Home

Amarendar Rao - Senior AWS Data Engineer

[email protected]

Location: Charlotte, North Carolina, USA

Relocation:

Visa:

AMARENDAR RAO T AWS Solutions Architect | Cloud Data Modernization & IaC Lead | Serverless & Database Reliability Engineer (15+ Years IT Experience) Email: [email protected] Phone: 571-771-3063 H1b C2C SUMMARY With 15+ years of experience in IT, I specialize in software development, configuration, build/release management, database development and administration. My expertise extends to the analysis, design, development, and operations of various database systems and cloud solution architecture. Proficient in managing windows/cloud-based applications. Experienced technologist proficient in crafting and implementing robust hosting solutions to cater to customers' escalating need for agile and cost-efficient computing solutions. Proficient in leveraging both private and public cloud technologies Exceptional understanding of SDLC, Traditional and Agile Methodologies. Experienced in Cloud automation using AWS Cloud Formation templates. Integrates with AWS CloudFormation Designer for visualizing and editing templates, as well as AWS CloudFormation CLI for advanced management tasks. Amazon Aurora PostgreSQL Deployment with CloudFormation. I have successfully collaborated with development teams and architects to drive best practices in cloud computing. Specializing in API Gateway, I have implemented and optimized solutions, ensuring seamless integration and robust performance. Proficient in leveraging AWS's cloud infrastructure and services to build, deploy, and manage Java-based applications. Experience in automation provisioning using Terraform. Extensive background in Active Directory, IAM, and Roles. Expert in AWS Architecture, VPC Networking, Terraform, AWS Primitives, and managing resource limits at a massive scale. Proficient in multiple coding languages including Python. Collaborated in the co-design of AWS Account Creation Automation. Helped in cost-saving network architecture changes. Sound understanding of networking concepts, security standard methodologies, and cloud infrastructure architecture. In-depth knowledge of cloud component architecture, encompassing Microservices, Containers, IaaS, Storage, and Security; well-versed in routing/switching technologies. Good understanding of cloud computing delivery models (IaaS, PaaS, and SaaS) and deployment models across Public, Private, and Hybrid Cloud services. Expertise in Big data analysis and analytics. Managing large datasets and deriving valuable insights. Application of Python for data manipulation, analysis, and scripting. Proficient in SQL (Structured Query Language) and expertise in Spark streaming. Experience in writing complex queries for database management and analysis. Expertise in deploying the content cloud platform on Amazon AWS using EC2, S3 and EBS. I have expertise with LINUX and Windows environments including Shell, Python scripts, version control tools like Subversion and GIT. Implemented AWS solutions using EC2, S3, RDS, EBS, Elastic Load Balancer and Auto scaling groups. Built and worked with Amazon EC2 instances, created the security groups on the instance level and used CloudWatch on EC2 instances. Experience involving configuring S3 versioning, lifecycle policies, backup files and archive files in glacier. I excel in Java application development, with proven expertise in designing scalable software solutions and implementing Java frameworks, libraries, and database integration for diverse applications. I have a deep understanding of UML design patterns that optimize application efficiency and maintainability. Proficient in leveraging AWS's cloud infrastructure and services to build, deploy, and manage Java-based applications. Snowflake Architecture and setup, Data Ingestion and Integration, Data Modeling and Optimization, Security, and performance tuning. Experience in performing scalability, high availability, migration, and consolidation of the Microsoft SQL Server Instances on Physical, VM s, AWS Cloud in coordination with (DevOps)/Platform, Network and Storage Teams. Worked on production engineering processes such as Incident management, root cause analysis, problem management, change management, release management and knowledge management. TECHNICAL SKILLS Build Tools/Scripting Lang(s)/ Programming Lang(s) MS Build, Bash, Shell, HTML, XML, Python, Java, C++, C, UML, Terraform, AngularJS, Nodejs Version Control Tools GIT, SVN Continuous Build Integration Jenkins, Jenkins Enterprise DevOps/Cloud Services Chef, Ansible, Docker, Kubernetes, Open Stack, AWS, AWS Glue, EMR, Redshift, SQS, SNS, Step Functions, Snowflake Operating Systems Windows, Linux, Unix Databases/Tools MS SQL, MySQL, Oracle, NoSQL, Tableau Bug Reporting Tools Bug Zilla, JIRA Monitoring Tools Splunk, Nagios, ELK, App Dynamics EDUCATION Master of Science in Computer Science from Sri Venkateswara University, Tirupati, India (1999). Bachelor of Science in Mathematics, Statistics and Computer Science from Kakatiya University, Warangal, India (1996). EXPERIENCE Cloud Engineer (IT Consultant) Tapas Cloud Com, US (Remote) (Feb 2023 Present) Responsibilities: Implement and Administrate AWS infrastructure to include support of AWS products such as: EC2, ELB, Auto Scaling, S3, RDS, Glue, Redshift, Lambda, Step Function, API Gateway, Networking & Content delivery (VPC, Route 53), IAM, CloudWatch, CloudTrail, Cloud Formation, Key Management. Manage network connectivity, database management, AMI lifecycle, server hardening, patching, package vulnerability, backups. Built and maintained scalable ETL pipelines using Python and PySpark on AWS Glue, integrating diverse data sources into Amazon Redshift, with optimized SQL transformations for analytics and reporting. Designed and optimized data storage and querying in Amazon Redshift, ensuring performance and scalability. Wrote SQL transformations, validations, and reporting queries to support analytics and business requirements. collaborated with development teams and architects to drive best practices in cloud computing. Specializing in API Gateway, I've implemented and optimized solutions, ensuring seamless integration and robust performance. Deploy API Gateway to consolidate endpoints for user registration, profile management, and data retrieval. Ensured data quality, error handling, and operational support for pipelines with monitoring via CloudWatch and CloudTrail. Integrated PyTest unit tests for Python-based ETL components to validate functionality and maintain reliability. Orchestrated workflows using AWS Step Functions to manage complex business processes. Implemented event-driven architectures leveraging AWS SNS and SQS for messaging and asynchronous processing. Developed Lambda functions (Python/Node.js) for serverless data transformations and integrations Architecture design of cloud solutions to address scalability, performance, security, availability, cost transformation requirements and representing design services and the status of Cloud Strategy. Leverage CloudFormation to create a reference deployment based on AWS best practices for security and high availability. Expertise in providing solutions / implementing solutions for Application Modernization on AWS Cloud. Proficient in developing Lambda functions using Node.js, leveraging its asynchronous and event-driven capabilities for efficient serverless computing. Demonstrated ability to design and implement serverless solutions on AWS, utilizing Lambda as the core computing service. Hands-on experience in writing clean, modular, and well-documented Node.js code for Lambda functions, ensuring maintainability and scalability. Expertise in integrating Lambda functions with other AWS services and external APIs to build robust serverless applications. Contributed to CI/CD pipelines for reproducible deployments using Terraform, CloudFormation, and AWS CDK. Skilled in optimizing Lambda functions for performance and cost-efficiency, considering factors such as memory allocation, execution time, and resource utilization. Experience in troubleshooting and debugging Lambda functions, utilizing logging and monitoring tools to identify and resolve issues promptly. Collaborative approach in working with development teams to design and implement Lambda-based solutions that meet business requirements and performance objectives. Experienced in using AWS Core Services CDK , Lambda , S3 , SQS , SNS , App Gateway etc. Design and build AWS cloud infrastructure using automation infrastructure-as-code (Iac). Proficient in building, changing, and versioning infrastructure efficiently. Using terraform defining infrastructure components and their configurations in code. Skilled in designing, developing, and deploying applications that leverage cloud-native principles and services and ability to monitor and optimize the performance of cloud-native applications. Understanding of microservices architecture and containerization technologies like Docker and Kubernetes. Experience with using cloud services for databases, storage, networking, and computing. Orchestrated workflows using AWS Step Functions to manage complex business processes. Integrates seamlessly with Continuous Integration/Continuous Deployment (CI/CD) pipelines. Installation, upgrading, and deletion of Kubernetes applications. Managed Kubernetes service provided by AWS EKS, simplifying the deployment, management, and scaling of Kubernetes clusters. Provisioning Kubernetes clusters through infrastructure as code tools like Terraform. Seamless integration with other AWS services like Elastic Load Balancing, Auto Scaling, and Amazon VPC for networking and scaling requirements. Built-in monitoring and logging integration with Amazon CloudWatch and AWS CloudTrail for visibility and auditability. Integrated various AWS services with Python applications for enhanced functionality. Wrote and executed unit tests using PyTest to ensure code quality and functionality. Environment: AWS, ETL, Python, PySpark, SQL, Node.js, Glue, Redshift, Lambda, Step Functions, SNS, SQS, S3, RDS, API Gateway, CloudWatch, CloudTrail, Git, Terraform, CloudFormation, CDK, Kubernetes (EKS) Cloud Engineer (IT Consultant) Botho Financial Services, Gaborone, Botswana (Jan 2017 - Dec 2022) Responsibilities I have successfully implemented and administered AWS infrastructure, providing support for a range of AWS products, including Amazon EC2 instances, Elastic Load Balancing, Amazon S3 storage, AWS Lambda functions, Amazon API Gateway, Amazon VPC, Route 53 for DNS, Amazon CloudFront CDN, and AWS Identity and Access Management (IAM), AWS ECS, AWS ECR, AWS RDS and AWS Glue. Used API Gateway to manage and expose RESTful APIs for product catalog, user authentication, and payment processing. Developed and optimized ETL pipelines using Python and PySpark on AWS Glue, enabling large-scale data processing and seamless integration of structured and semi-structured data into Amazon Redshift for analytics and reporting. Implemented data quality checks, monitoring, and error handling within ETL workflows, ensuring reliable data delivery and optimized query performance. Built and deployed Python-based applications on AWS, leveraging Pandas and NumPy for data manipulation and analysis. Designed and managed ETL pipelines using AWS Glue for data integration and transformation. Migrated critical production databases to AWS cloud with minimal downtime, ensuring data integrity and SLA compliance. Wrote and optimized SQL scripts for schema migration, data replication, and reporting. Utilized API Gateway to create RESTful APIs, triggering corresponding Lambda functions. Implement API Gateway features like request validation, response transformation, and usage plans for managing access. Implemented API Gateway to serve as an entry point for external clients. Use it for routing requests to appropriate microservices, aggregating responses, and managing common concerns like authentication and logging. Automated deployment and release distribution process. Developed and deployed Python applications on AWS, leveraging cloud infrastructure for scalability and reliability. Utilized Pandas and NumPy for data manipulation and analysis. Experience with developing, deploying, and managing AWS Lambda functions. Knowledge of integrating AWS Lambda with other AWS services such as API Gateway, S3, DynamoDB, and CloudWatch. Implemented event-driven architectures using AWS Lambda integrated with API Gateway, S3, DynamoDB, and CloudWatch. Designed RESTful APIs via API Gateway to trigger Lambda functions for product catalog, authentication, and payment processing. Ensured data quality and operational support by implementing monitoring, logging, and debugging for Lambda functions. Automated DB maintenance activities with PowerShell and integrated monitoring via CloudWatch. Provided 24/7 operational support for databases (SQL Server, MongoDB, MySQL, Postgres, Document DB). Utilized AWS SNS and SQS for messaging and asynchronous workflows Ability to write efficient, event-driven code for serverless applications. Familiarity with monitoring, logging, and debugging Lambda functions for operational efficiency. Built and deployed applications code to QA and PROD environments Designed the project workflows/pipelines using Jenkins as CI tool. Carried deployments and builds on various environments using continuous integration tool Jenkins. Expert in User Management and Plugin Management for Jenkins. Defined and provisioned AWS infrastructure using Terraform and CloudFormation, ensuring reproducible deployments. Managed containerized applications with Docker and AWS ECS/ECR, supporting microservices and middleware installations. Worked on Docker and created virtual instances with Docker. Experience working on Docker hub, creating Docker images, and managing multiple images primarily for middleware installations and domain configurations. Analyze, configure, and manage DevOps tool chain. Onboarding/Migrating on-premises applications to cloud environments. Migrated on premises applications to Cloud AWS environment using containerized methodologies. Automate DB maintenance activities using PowerShell. Migrate critical production databases to AWS cloud using transactional replication. Develop scripts for schema and data migration with minimal downtime, adhering to SLAs. Assist in migrating applications to the cloud and apply necessary procedures and fixes. Work on the AWS cloud, creating new EC2 instances, configuring Always On, and setting up PowerShell jobs to copy backups to S3 buckets while adhering to company standards. Strong ability to define, provision, and manage infrastructure using Terraform. Experience with writing modular, reusable, and maintainable Terraform configurations. Familiarity with Terraform's state management, including state files, remote state, and state locking. Competence in using Terraform for cloud providers such as AWS, and Google Cloud Platform. Proficiency in integrating Terraform with version control systems and CI/CD pipelines. Provide operational support and engineering solutions for various databases, including SQL Server, MongoDB, MySQL, Postgres, and Amazon DocumentDB. Set up source control, link databases to GITLAB, and facilitate the automation of production deployments. Review developer code, perform check-ins to version control, and handle deployments. Work in a DevOps database model, responsible for 24/7 operational support, collaborating with development and QA teams on application changes, code reviews, and deployments. Environment: AWS, ETL, Python, SQL, Glue, Redshift, Lambda, Step Functions, SNS, SQS, S3, RDS, API Gateway, CloudWatch, DynamoDB, Jenkins, GitLab, Terraform, CloudFormation, Docker, ECS/ECR Database Engineer Pathfinder Enterprises (Pty) Ltd, Francistown, Botswana (Nov 2007 - Dec 2016) Responsibilities Focus on enhancing SQL Server performance by closely monitoring usage, setting up baselines, and taking necessary actions. Consolidate databases from different servers and migrate them to a single server for cost-effective solutions, considering performance, IO throughput, and specs usage. Install and support SQL Server on Linux for non-critical production databases. Offer cost-effective disaster recovery solutions for SQL Server databases. Implement complex high availability and disaster recovery solutions, combining Always On and Log Shipping, as well as Replication and Mirroring, tailored to business requirements and customer SLAs. Migrate SSIS packages from SQL Server 2012 to SQL Server 2016. Focus on performance tuning for large databases ranging from 1 TB to 3.5 TB. Address new requirements and make necessary data/schema changes while adhering to company standards of development and change processes. Actively monitor and optimize maintenance plans, set up email alerts, and automate tasks. Set up and monitor SQL Server Transactional replication, troubleshoot data sync issues, and automate audit reports on logins, database users, and SQL Server Agent jobs. Implement Always On High Availability and Disaster Recovery (HADR) on SQL Server 2012. Support application vendors with database-related issues. Create maintenance plans for automation of backups, working with Net App Snap manager for snap backups. Provide appropriate Disaster Recovery (DR) solutions based on client requirements and limitations. Database migration and SQL Server upgrade projects from SQL Server 2005 to SQL Server 2008/2008R2 and SQL Server 2008 to SQL Server 2012. Write and execute Change steps for Change management on production systems. Apply patches for existing SQL Server 2008 instances. Set up and support MSSQL clusters, including active/passive and active/active clusters on Windows 2012. Participate in 24/7 On-Call production support, being accountable for the operational aspects of infrastructure management according to solution requirements and defined SLAs. Collaborate with others in Infrastructure Engineering to assess the production environment for continuous improvement and enhancement of existing environments. Plan and coordinate data migrations between SQL server instances as needed by development, QA, and Production. Support customers during the implementation process and troubleshoot issues preventing go live. Provide technical support for in-house and 3rd party developed SQL Server applications, including software installations, upgrades, problem diagnosis, performance monitoring, evaluation, and tuning recommendations. SQL Developer / SQL Server DBA ACEAR, Ambala, India (Jul 2006 - Oct 2007), HMRIT, Delhi, India (Apr 2005 - Jul 2006), Rai Foundation, New Delhi, India (Jan 2004 - Apr 2005), St. John PG Centre, Warangal, India (Nov 2001 - Jan 2004) and worked as a System Programmer in SRTIST, Nalgonda, India (Feb 1999 - Oct 2001). Responsibilities: Database design, development, maintenance, and troubleshooting. Database troubleshooting, upgrades, and maintenance. Write technical policies, procedures, and provide guidelines/roadmap for implementing policies and standards for Database Service Operations. Document security standards for Database Security, implement the standards, and validate. Develop custom scripts to automate the implementation of backup retention policy across the environment. Identify inefficiencies in current databases and investigate solutions. Work on a user/login remediation project to ensure compliance with security standards. Proactively monitor system performance, availability, and disk space using custom scripts, monitoring tools, and generate alerts at the threshold limit. Provide database maintenance support and monitor the database for 100% uptime. Create and manage automated scheduled jobs for DB Maintenance, Backups (involving native and NetBackup). Scheduled automated jobs for the improvement of performance like UPDATING STATISTICS, UPDATING INDEXES and DBCC checks Perform SQL Server database patching as needed during scheduled maintenance window. Using SQL Server profiler, performance tuned slow running stored procedures to decrease run times. Write scripts for internal DB monitoring, sending email alerts to on-call DBA. Experience in creating minor packages for loading data from different platforms and developing major packages with complex control flow and data flow transformations, including business intelligence, Split, and Join transformations. Work on fine-tuning databases, resolving performance issues, and proactively monitoring databases for optimum performance. Create and manage automated scheduled jobs for DB Maintenance, Backups. Proactively monitor system performance, availability, and resolve issues. Perform performance tuning at both server and database levels, taking corrective actions. CERTIFICATIONS AWS Fundamentals Specialization, DevOps on AWS Specialization, AWS Cloud Solutions Architect Professional Certificate, Building Serverless Applications on AWS Specialization, AWS Cloud Practitioner Essentials, Google Cloud Fundamentals: Core Infrastructure. CCNA, CCNA Security, CCNA Cyber Security Operations . Windows Server Management and Security, NDG Linux Essentials and Linux Unhatched. Certified ISO 9001:2015 Quality Management System Lead Auditor. Enterprise System Management and Security. Fundamentals of Scalable Data Science. Applied AI with Deep learning and Advanced Machine Learning and Signal Processing. Keywords: cprogramm cplusplus continuous integration continuous deployment quality analyst artificial intelligence javascript sthree database information technology golang microsoft mississippi Colorado