| AWS Redshift RDS Databricks Data Engineer at Boston, Massachusetts, USA |
| Email: [email protected] |
|
From: DEBASISH PATTNAIK, MRTECHNOSOFT [email protected] Reply to: [email protected] AWS Redshift/RDS/Databricks Data Engineer Job Description Job Title: AWS Redshift/RDS/Databricks Data Engineer Location: [Boston,MA] Job Type: [Contract] Experience Level: [16+]Job Overview: We are seeking a highly skilled Data Engineer with expertise in AWS Redshift, RDS, and Databricks to design, develop, and optimize data pipelines and storage solutions. The ideal candidate should have strong experience working with large-scale data architectures, ETL processes, and cloud-based data platforms.Key Responsibilities: Design, develop, and maintain scalable data pipelines using AWS services, including Redshift, RDS, and Databricks. Optimize data storage and retrieval to ensure high performance and cost efficiency. Implement ETL/ELT processes for structured and unstructured data from multiple sources. Collaborate with data scientists, analysts, and software engineers to ensure seamless data integration. Ensure data security, governance, and compliance with best practices. Monitor and troubleshoot data pipeline issues, ensuring high availability and reliability. Automate data workflows using Python, Spark, or SQL-based solutions. Optimize query performance and database indexing strategies for AWS Redshift and RDS. Implement real-time and batch data processing solutions in Databricks. Stay updated with the latest advancements in cloud-based data engineering and recommend improvements.Required Skills & Qualifications: Strong experience with AWS Redshift, RDS, and Databricks. Proficiency in SQL for data modeling, transformation, and performance tuning. Hands-on experience with Python, PySpark, or Scala for data processing. Expertise in ETL/ELT pipelines and data ingestion strategies. Experience with AWS services such as S3, Lambda, Glue, Step Functions, and IAM. Familiarity with Apache Spark and distributed computing frameworks. Knowledge of data warehousing concepts, indexing, partitioning, and optimization. Strong understanding of database administration, including backup, replication, and disaster recovery. Experience with CI/CD pipelines and DevOps practices for data engineering. Ability to work in an Agile environment and collaborate with cross-functional teams.Nice to Have: Experience with Snowflake or BigQuery. Knowledge of Airflow or other workflow orchestration tools. Familiarity with ML/AI pipelines in Databricks. Hands-on experience with Kafka or Kinesis for real-time data processing.Education & Experience: Bachelors/Masters degree in Computer Science, Data Engineering, or a related field. [Specify required years] years of experience in data engineering with AWS technologies Thanks Debasish Pattnaik [email protected] www.mrtechnosoft.com Keywords: continuous integration continuous deployment artificial intelligence machine learning sthree Massachusetts AWS Redshift RDS Databricks Data Engineer [email protected] |
| [email protected] View All |
| 12:17 AM 05-Feb-25 |