Data Engineer :: Minneapolis, MN-- hybrid - LOCAL :: USC or H4EAD only at Minneapolis, Minnesota, USA |
Email: manisha.yadav@infotechspectrum.com |
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1630521&uid= From: manisha, infotechspectrum manisha.yadav@infotechspectrum.com Reply to: manisha.yadav@infotechspectrum.com POSITION Data Engineer - US BANK LOCATION Local to Minneapolis, MN-- hybrid LOCAL CONSULTANT WITH minnesota DL DURATION 3+ month contracts INTERVIEW TYPE Video Visa: USC or H4EAD only REQUIRED SKILLS Senior Software Engineer (Big Data - Spark) Primary Responsibilities: Azure, Spark, Kafka experience required Design and build highly available, scalable, resilient and fault tolerant distributed real-time data streaming system. Automate test coverage (90+%) for data pipelines, best practices and frameworks for unit, functional and integration tests. Automate CI and deployment processes and best practices for the production data pipelines. Build AI/ML model based alert mechanism and anomaly detection system for the product. The goal is have a self-annealing product. Basic Qualifications Bachelors degree or equivalent work experience. 5+ years of relevant experience. Preferred Skills/Experience 10+ years of overall experience in software development with 5-6 years of relevant experience in designing, developing, deploying and operating data streaming pipelines at scale. 3-4 years experience with Apache Kafka and Apache Spark Streaming Hands on experience with Spark Structured Streaming Experience in tuning Spark Data pipeline to achieve high throughput. Drive efforts to improve the data quality across data pipelines and implement system controls for managing data quality Programming proficiency in Scala or Java. Open Source Committer (Apache Spark or related Big Data open source technologies) Experience with Containers, Kubernetes and scaling elastically Strong background in algorithms and data structures and continuously develop and acquire new technical skills Lead and mentor junior engineers to ensure systems are built with highest quality and leveraging best practices. Experience in automating Spark pipeline deployment/testing (DevOps, CI/CD) Passion for data engineering and for enabling others by making the product easier to use. Excellent communication in sharing context to effectively collaborate with analytical partners, domain experts and other consumers of your work, preferably in supporting an engineering or product function Required Skills Summary: Apache Spark, Apache Kafka, Scala/Java, NoSQL Databases, Elasticsearch & Kibana, Kubernetes, Docker Containers Thank You Best Regards Manisha Yadav InfoTech Spectrum Inc 2060, Walsh Ave, #120, Santa Clara, CA 95050 Email : manisha.yadav@infotechspectrum.com Web: w ww.infotechspectrum.com Keywords: continuous integration continuous deployment artificial intelligence machine learning California Minnesota Data Engineer :: Minneapolis, MN-- hybrid - LOCAL :: USC or H4EAD only manisha.yadav@infotechspectrum.com http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1630521&uid= |
manisha.yadav@infotechspectrum.com View All |
03:45 AM 06-Aug-24 |