| Lead Data Engineer with ETL and StreamSets Exp: Scottsdale AZ Onsite at Pima, Arizona, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1462983&uid= From: pritam Singh, kpg99 [email protected] Reply to: [email protected] Hello, Job Title: Lead Data Engineer with ETL and StreamSets Exp Location: 5801 N. Pima Rd Scottsdale, AZ 85250- Day one Onsite Duration: 6+ Month Contract Visa: USC or GC ONLY Note : - Having StreamSets exp is key for this role, pls get answers on below questions and share. If its found candidate is strong and comfortable with more than 4 yrs of exp in Streamsets, pls share profile along with answers of below questions on StreamSets. Make sure every submission has answers of below questions . pls share more profiles at earliest. 1. Total StreamSets experience in years & months (do not round to next year). Last used in which year Need Latest/current exp . Past or 2 3 years ago will not fly. 2. Which version of SCH, DC and TR have you used 3. For which project did you use StreamSets explain the use case in brief. Stages used, main StreamSets features used, challenges faced, solutions 4. Does StreamSets allow reusability how 5. What is the difference between processors and executors Which is the main processor is used in StreamSets orchestration The ideal candidate will have strong skills in Python programming, experience with cloud platforms (such as AWS, Azure, or GCP), and proficiency in containerization technologies like Kubernetes and Docker Although is mentioned Data/ETL Engineer but StreamSet is 100% must have and need hands on exp if Yellow and white belt certification is there its good to have if not hands recent exp is required. StramSet is ETL tool only and not niche skill, pls get strong hands exp in streamset, I need in submission email how many years of exp in stream set. For this assignment StreamSets hands on experience is mandatory and the candidate has to hit the ground running (no time to get certified and practice, etc.); its a T&M engagement Job Description: Overview: We are seeking a talented and motivated Software Developer with expertise in ETL (Extract, Transform, Load) processes, particularly within cloud environments. The ideal candidate will have strong skills in Python programming, experience with cloud platforms (such as AWS, Azure, or GCP), and proficiency in containerization technologies like Kubernetes and Docker. This role requires a proactive problem solver who can design, develop, and optimize ETL pipelines while ensuring scalability, reliability, and efficiency. Responsibilities: Design, develop, and maintain ETL pipelines to extract data from various sources, transform it according to business requirements, and load it into target data warehouses or databases. Collaborate with cross-functional teams to gather requirements, understand data sources, and define data transformation logic. Implement scalable and efficient ETL solutions within cloud environments, leveraging platform services and infrastructure as code. Optimize ETL processes for performance, reliability, and cost-effectiveness, considering factors like data volume, frequency, and latency requirements. Monitor and troubleshoot ETL pipelines to ensure data integrity, timely execution, and error handling. Stay up-to-date with emerging technologies and best practices in ETL, cloud computing, and data engineering, and propose innovative solutions to enhance the overall data ecosystem. Document ETL workflows, data mappings, and system configurations to facilitate knowledge sharing and maintainability. Provide technical guidance and support to junior team members, fostering a collaborative and learning-oriented environment. Requirements: Proven experience in software development with expertise in Python programming language. Hands-on experience with cloud platforms such as AWS, Azure, or GCP, including services like S3, EC2, Azure Data Factory, Google BigQuery, etc. Strong understanding of ETL concepts, methodologies, and best practices. Proficiency in containerization technologies such as Kubernetes and Docker. Experience with scripting languages for automation and orchestration tasks (e.g., Bash, Power). Familiarity with relational databases (e.g PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra). Excellent problem-solving skills with a focus on delivering high-quality solutions in a fast-paced environment. Effective communication skills with the ability to collaborate with cross-functional teams and articulate technical concepts to non-technical stakeholders. Preferred Qualifications: Master's degree in Computer Science, Engineering, or related field. Experience with data warehousing technologies and concepts (e.g., Snowflake, Redshift, star schema). Knowledge of data governance, security, and compliance standards in cloud environments. Certification in relevant cloud platforms (e.g., AWS Certified Developer, Azure Developer Associate, Google Cloud Professional Developer). Familiarity with agile development methodologies and DevOps practices. Contributions to open-source projects or active participation in developer communities. Thanks and Regards, Pritam Singh Sr. Technical Recruiter | KPG99 .INC Keywords: sthree information technology green card Arizona Lead Data Engineer with ETL and StreamSets Exp: Scottsdale AZ Onsite [email protected] http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1462983&uid= |
| [email protected] View All |
| 09:25 PM 07-Jun-24 |