| Data Engineer - SFO, CA (Remote) at Remote, Remote, USA |
| Email: [email protected] |
|
Hi All, Data Engineer - 02 Positions SFO, CA (Remote) Interview Process: 3 Rounds of Interview Expertise: 5-9+ years of relevant industry experience with a BS/Masters, or 2+ years with a PhD Experience with distributed processing technologies and frameworks, such as Hadoop, Spark, Kafka, and distributed storage systems (e.g., HDFS, S3) Demonstrated ability to analyze large data sets to identify gaps and inconsistencies, provide data insights, and advance effective product solutions Expertise with ETL schedulers such as Apache Airflow, Luigi, Oozie, AWS Glue or similar frameworks Solid understanding of data warehousing concepts and hands-on experience with relational databases (e.g., PostgreSQL, MySQL) and columnar databases (e.g., Redshift, BigQuery, HBase, ClickHouse) Excellent written and verbal communication skills A Typical Day: Design, build, and maintain robust and efficient data pipelines that collect, process, and store data from various sources, including user interactions, financial details, and external data feeds. Develop data models that enable the efficient analysis and manipulation of data for merchandising optimization. Ensure data quality, consistency, and accuracy. Build scalable data pipelines (SparkSQL & Scala) leveraging Airflow scheduler/executor framework Collaborate with cross-functional teams, including Data Scientists, Product Managers, and Software Engineers, to define data requirements, and deliver data solutions that drive merchandising and sales improvements. Contribute to the broader Data Engineering community at Airbnb to influence tooling and standards to improve culture and productivity Improve code and data quality by leveraging and contributing to internal tools to automatically detect and mitigate issues. Skill Sets - Python, SQL (expert level), Spark and Scala (intermediate). Thanks & Regards, Pooja Marvel Infotech, Inc. Fax : 732-875-0333 Email:- [email protected] www.marvelinfotech.com MBE, SBE certified - State of NJ MBE NMSDC NYNJ Information transmitted by this e-mail is proprietary to Marvel Infotech, its associated companies and/ or its customers and is intended for use only by the individual or entity to which it is addressed, and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please delete this mail from your records. -- Keywords: sthree information technology California New Jersey Data Engineer - SFO, CA (Remote) [email protected] |
| [email protected] View All |
| 07:30 PM 14-Aug-24 |