| Bigdata Python-PySpark Developer - WHIPPANY, NJ - Onsite at Remote, Remote, USA |
| Email: justin@gacsol.com |
|
https://jobs.nvoids.com/job_details.jsp?id=1828200&uid= From: Justin Davis, GAC Solutions justin@gacsol.com Reply to: justin@gacsol.com Hi, I hope you're doing well! Only Local to NJ (out of NJ/NY client is not considering profiles) and lookig for seniors around 13-14+ Years of expeirence. Role Name: Bigdata Python/PySpark Developer Location: WHIPPANY, NJ - Onsite 6-12 months JOB DESCRIPTION: Job Summary: We are looking for a highly skilled Python/PySpark Developer with hands-on experience in Big Data technologies to join our dynamic data engineering team. The ideal candidate will have a strong background in building scalable, distributed data processing systems, using PySpark, and working on large datasets. You will collaborate with data scientists, engineers, and other stakeholders to design and implement efficient data pipelines. Key Responsibilities: Develop, optimize, and maintain ETL pipelines using PySpark to process large-scale datasets across distributed environments. Design and implement complex data transformation logic using PySpark and other Big Data tools. Work with various Big Data technologies such as Hadoop, Hive, HBase, Kafka, and Spark to build robust, scalable data systems. Collaborate with data engineers and data scientists to integrate data from multiple sources and create unified datasets. Write efficient, reusable, and scalable Python code to handle both batch and real-time data processing tasks. Ensure data quality, consistency, and reliability by implementing data validation, monitoring, and error handling. Fine-tune and optimize PySpark jobs to improve performance in distributed environments. Manage and maintain data flows in HDFS, ensuring scalability and fault tolerance. Perform data extraction, aggregation, and reporting using SQL and NoSQL databases. Participate in system design discussions and provide recommendations for architecture and performance improvements Thanks, Justin Davis Sr Technical Recruiter E: justin@gacsol.com Keywords: New Jersey New York Bigdata Python-PySpark Developer - WHIPPANY, NJ - Onsite justin@gacsol.com https://jobs.nvoids.com/job_details.jsp?id=1828200&uid= |
| justin@gacsol.com View All |
| 10:28 PM 09-Oct-24 |