| Azure Databricks Developer at Remote, Remote, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1196824&uid= From: sagar, 3ktechnologies [email protected] Reply to: [email protected] Position Title: Azure Databricks Developer Location: Remote Job Summary: As an Azure Databricks Developer, you will be responsible for designing, developing, and maintaining data engineering pipelines and analytics solutions on the Azure Databricks platform. This role involves collaborating with data scientists, analysts, and other stakeholders to implement scalable and efficient data processing workflows, machine learning models, and data visualizations to support business objectives. Key Responsibilities: Design, develop, and deploy data engineering pipelines on the Azure Databricks platform using Apache Spark, Databricks Delta, and other related technologies. Implement data ingestion, transformation, and storage processes to support the collection, cleansing, and enrichment of structured and unstructured data from various sources. Collaborate with data scientists and analysts to deploy machine learning models and analytics solutions on Azure Databricks, and integrate them with existing data pipelines and applications. Optimize data processing workflows and queries for performance, scalability, and cost-efficiency, leveraging techniques such as partitioning, caching, and data compression. Develop and maintain data visualizations and dashboards using tools such as Databricks SQL Analytics, Power BI, or Tableau to provide insights and actionable intelligence to stakeholders. Implement data security and compliance controls to protect sensitive data and ensure regulatory compliance (e.g., GDPR, HIPAA) on the Azure Databricks platform. Monitor and troubleshoot data engineering pipelines and analytics solutions for performance, reliability, and stability, and implement enhancements and optimizations as needed. Document data engineering processes, procedures, and best practices, and provide training and support to other team members as needed. Stay current with industry trends, technologies, and best practices in big data, cloud computing, and data engineering, and evaluate and recommend new tools and techniques to improve efficiency and effectiveness. Required Skills and Qualifications: Bachelor's degree in computer science, engineering, mathematics, or a related field. Proven experience (typically 2+ years) in data engineering, big data processing, or related roles, with a focus on developing and deploying data pipelines and analytics solutions on the Azure Databricks platform. Proficiency in Apache Spark programming using Python or Scala, and experience with Spark SQL for data manipulation and querying. Hands-on experience with Azure Databricks, including cluster configuration, job scheduling, notebook development, and integration with Azure services (e.g., Azure Data Lake Storage, Azure Synapse Analytics). Strong understanding of data storage and processing concepts, including distributed file systems, columnar storage formats, and parallel processing frameworks. Familiarity with data integration techniques and tools, such as ETL (Extract, Transform, Load) processes, data streaming, and real-time data processing. Experience with version control systems (e.g., Git) and CI/CD pipelines for automated testing, deployment, and monitoring of data engineering workflows. Excellent problem-solving and troubleshooting skills, with the ability to analyze complex data issues and implement effective solutions. Strong communication and collaboration skills, with the ability to work effectively in a team environment and interact with stakeholders from diverse backgrounds. Preferred Additional Skills: Certification in Azure Data Engineering or Azure Databricks is a plus. Knowledge of machine learning and statistical modeling techniques, and their applications in data engineering and analytics. Experience with other Azure services, such as Azure Machine Learning, Azure Data Factory, and Azure Synapse Analytics (formerly SQL Data Warehouse). Understanding of data governance frameworks and practices for managing data quality, privacy, and security. Familiarity with containerization and orchestration tools, such as Docker and Kubernetes, for deploying and managing data engineering workflows. Work Environment: Azure Databricks Developers typically work in office environments, often as part of cross-functional teams that include data scientists, analysts, software engineers, and business stakeholders. This role may involve occasional remote work and off-hours support to ensure the availability and reliability of data pipelines and analytics solutions. Conclusion: The role of an Azure Databricks Developer is essential for organizations looking to leverage the power of cloud-based big data processing and analytics to drive insights and innovation. By designing, developing, and maintaining scalable and efficient data pipelines and analytics solutions on the Azure Databricks platform, developers enable data-driven decision-making and business growth. With strong technical skills, analytical capabilities, and a collaborative mindset, Azure Databricks Developers play a key role in unlocking the value of data for organizations across various industries Thanks & Regards Vidhya Sagar Technical Recruiter 1114 Cadillac Ct, Milpitas, CA 95035 3K Technologies, LLC www.3ktechnologies.com [email protected] Analytics | BI | Big Data | Cloud |Software Engg. Keywords: continuous integration continuous deployment business intelligence California Connecticut http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1196824&uid= |
| [email protected] View All |
| 03:22 AM 08-Mar-24 |