| Hiring || Data Engineer || Santa Clara, CA || Hybrid || Only locals || Banking - Financial || at Santa Clara, California, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=930070&uid= From: Justin Smith, Softcomsystems [email protected] Reply to: [email protected] Hello, Hope you are doing well, Kindly review the job details below. Should you think it's a good match with your background, please revert to this email with your updated resume and a good time to discuss the same. Role: Data Engineer Trusted Computing project Location: Santa Clara, CA (Hybrid) Note: Need only locals Duration: Long term contract Must Have: Scala,Data Bricks,DATA ARCHITECTURE,Databricks,Spark,Java,System Integration,Data Migration,Systems,Windows,ETL,Other,SQL Details: 10+ years of IT experience focusing on enterprise data architecture and management. Experience in Conceptual/Logical/Physical Data Modeling & expertise in Relational and Dimensional Data Modeling Experience with Databricks & on Prem , Structured Streaming, Delta Lake concepts, and Delta Live Tables required Experience with Spark scala and java programming Data Lake concepts such as time travel and schema evolution and optimization Structured Streaming and Delta Live Tables with Databricks a bonus Experience leading and architecting enterprise-wide initiatives specifically system integration, data migration, transformation, data warehouse build, data mart build, and data lakes implementation / support Advanced level understanding of streaming data pipelines and how they differ from batch systems Formalize concepts of how to handle late data, defining windows, and data freshness Advanced understanding of ETL and ELT and ETL/ELT tools such as Data Migration Service etc Understanding of concepts and implementation strategies for different incremental data loads such as tumbling window, sliding window, high watermark, etc. Familiarity and/or expertise with Great Expectations or other data quality/data validation frameworks a bonus Familiarity with concepts such as late data, defining windows, and how window definitions impact data freshness Advanced level SQL experience (Joins, Aggregation, Windowing functions, Common Table Expressions, RDBMS schema design performance optimization) Indexing and partitioning strategy experience Debug, troubleshoot, design and implement solutions to complex technical issues Experience with large-scale, high-performance enterprise big data application deployment and solution Architecture experience in AWS environment a bonus Familiarity working with Lambda specifically with how to push and pull data, how to use AWS tools to view data for processing massive data at scale a bonus Experience with Gitlabs and CloudWatch and ability to write and maintain gitlabs for supporting CI/CD pipelines Experience working with AWS Lambdas for configuration and optimization and experience with S3 Familiarity with Schema Registry, message formats such as Avro, ORC, etc. Ability to thrive in a team-based environment Experience briefing the benefits and constraints of technology solutions to technology partners, stakeholders, team members, and senior level of management Skillset: Java, Scala, S3, Glue, Redshift Location - Scottsdale" Best Regards, Justin Smith | Associate Recruiter | Softcom Systems, Inc Desk : (609) 333 - 9000 | Ext - 54 | Direct : 779-333-2421 : [email protected] LinkedIn: linkedin.com/in/justin-smith-9b1701248 www.Softcomsystems.com | 475, Wall Street , Princeton, NJ-08540 Keywords: continuous integration continuous deployment sthree information technology California New Jersey http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=930070&uid= |
| [email protected] View All |
| 08:15 PM 12-Dec-23 |