| Data Engineer (AWS RDS, Redshift, Databricks) at Remote, Remote, USA |
| Email: [email protected] |
|
From: DEBASISH PATTNAIK, MRTECHNOSOFT [email protected] Reply to: [email protected] Role: Data Engineer (AWS RDS, Redshift, Databricks) Location: Open to Remote Duration: Long-termJob Description We are seeking a skilled Data Engineer with expertise in AWS RDS, Redshift, and Databricks to design, build, and maintain scalable data pipelines and data management solutions. The ideal candidate will have strong experience in cloud-native data engineering practices and a proven ability to handle large-scale data processing tasks.Key Responsibilities Design and implement data pipelines using Databricks and AWS services like Redshift and RDS to support large-scale analytics and reporting. Develop ETL workflows to ingest, transform, and load data into Redshift from various data sources. Optimize database performance on AWS RDS and Redshift, including query tuning and indexing strategies. Build scalable and efficient data solutions to manage structured and semi-structured data. Collaborate with data scientists, analysts, and other stakeholders to understand requirements and deliver high-quality data solutions. Monitor and maintain data pipeline performance, identifying and resolving bottlenecks. Implement best practices for data security, governance, and compliance on AWS. Utilize Spark on Databricks for distributed data processing and analytics. Create and maintain technical documentation for data workflows and infrastructure.Required Skills AWS Expertise: In-depth knowledge of AWS RDS (PostgreSQL, MySQL, or Oracle). Hands-on experience with AWS Redshift, including its architecture, performance tuning, and Spectrum. Experience with AWS data ecosystem tools like S3, Lambda, and Glue. Databricks: Strong experience in using Databricks for data engineering workflows and integrating it with AWS. ETL & Data Integration: Expertise in creating and managing ETL pipelines for large-scale data ingestion and transformation. Proficiency in tools like Apache Airflow or AWS Step Functions. Programming: Proficient in Python or Scala for data engineering tasks, with knowledge of SQL for querying and optimization. Big Data Frameworks: Experience with Spark for processing large datasets. Database Skills: Proficient in schema design, partitioning strategies, and query optimization.Preferred Qualifications Experience with real-time data streaming tools like Kafka or Kinesis. Familiarity with BI tools like Tableau, QuickSight, or Power BI. Knowledge of data governance frameworks and tools. Experience working in Agile environments.Soft Skills Strong problem-solving and analytical capabilities. Excellent communication and collaboration skills. Ability to prioritize and manage multiple tasks efficiently. Thanks Debasish Pattnaik [email protected] www.mrtechnosoft.com Keywords: business intelligence sthree information technology Data Engineer (AWS RDS, Redshift, Databricks) [email protected] |
| [email protected] View All |
| 11:34 PM 07-Jan-25 |