| ETL ||Python Developer|| at Remote, Remote, USA |
| Email: [email protected] |
|
From: shivani sharma, adventatech [email protected] Reply to: [email protected] Job Description - Title: ETL/Python Developer Duration: 6+month Location: Remote (must be within the US) MUST HAVE. Strong Agricultural Exp is a MUST. Manager Notes: "The stack is very ETL heavy, maintaining orchestrator's jobs, and supported Redshift data warehouse infrastructure Technologies: Talend, Terraform, Python, Scripting, Docker, AWS: EC2, S3, Redshiftt Job Details: 1. ETL tools and technologies used in this project - Involves working with but not limited to: Talend (Talend Studio, Talend Remote Engine, Talend Management Console), Terraform, Python, scripting, Docker, AWS services like EC2, S3. 2. What tools are used for orchestrating the ETL processes - Custom built solutions involving Talend. 3. Redshift Spectrum or focusing on traditional Redshift clusters - Mainly traditional Redshift clusters. 4. What other AWS services or tools are integrated into the project - The infrastructure consists of multiple aspects of Talend (Talend Remote Engine, Talend Managament Console, Talend Studio), EC2 instances used to run Talend, Redshift and various repositories (Python and Terraform) for adding new sources and to manage and support the infrastructure. 5. Size of data that we are working with - 8400 GB 6. Are integrating data from multiple sources into the Redshift warehouse, or building a data pipeline for real-time analytics - Majority of it is building the Redshift data warehouse. 7. What does the day to day look like - Building and maintaining ETL pipelines. Also monitoring daily jobs and address the failures if any. Also work on optimizing the performance of Redshift infrastructure using Terraform. 8. Does the job include designing and building the ETL pipeline vs. maintaining existing ones - Both 9. I generally support the Data Analytics' team's. What does supporting Redshift infrastructure mean in this context - It includes managing the security aspects and fulfilling requests that come in from Cloud Engineering team. - Maintaining Talend infrastructure. - Also includes working with Terraform to maintain all the aspects of the Redshift cluster infrastructure using terraform templates. Keywords: sthree information technology ETL ||Python Developer|| [email protected] |
| [email protected] View All |
| 09:22 PM 22-Nov-24 |