| Position : Data Engineer with StrongDatabricks at Remote, Remote, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=2747596&uid=4e6f06da79584cd3a5a67e3c66695685 Position : Data Engineer with StrongDatabricks Role Details: Skillset Required: Databricks Location: Indianapolis, IN Engagement Type: Contract onsite 1. Role Objective Build, operate, and govern productiongrade data and analytics solutions that span Databricks (Pipelines, Delta Live Tables, Genie, AgentBricks) and Microsoft Fabric (Data Engineering, Lakehouse, Data Warehouse, PowerBI). Deliver fast, reliable, and costoptimized data flows while maintaining enterprisegrade security and observability. 2. Core Responsibilities Architecture & Design o Design endtoend ingestion, transformation, and serving layers across Databricks and Fabric. o Define datamodel standards (starschema, CDC, semistructured handling). Pipeline Development o Implement CICDready pipelines using Databricks Pipelines/Jobs API and Fabric pipelines (SparkSQL, notebooks). o Enable realtime streaming (Event Hub/Kafka Structured Streaming Fabric Lakehouse). Data Quality & Governance o Register assets in Unity Catalog & Fabric Lakehouse catalog; enforce rowlevel security, data masking, and Purview lineage. Performance & Cost Optimization o Tune Spark clusters, leverage Photon & Genie autotuning. o Use Fabrics hot/cold tiers, materialized views, and autoscale compute to keep spend under budget. Collaboration & Enablement o Partner with data scientists, analysts, and product owners to translate business needs into reliable data solutions. o Create reusable templates, documentation, and run knowledgesharing sessions on Databricks & Fabric best practices. 3. Minimum Required Skills Databricks 4+years with Pipelines, Delta Live Tables, Genie, AgentBricks; strong PySpark/Scala; Unity Catalog administration. Microsoft Fabric 3+years building Data Engineering, Lakehouse, and Data Warehouse pipelines; proficiency in Fabric notebooks (SparkSQL, Python). Azure Cloud ADLS Gen2, Event Hub, Service Bus, Azure Functions, Key Vault, Azure DevOps/GitHub Actions, Terraform/ARM. Data Modelling Starschema, CDC, handling JSON/Parquet/Avro. Governance & Security Unity Catalog, Microsoft Purview, rowlevel security, GDPR/CCPA compliance. CI/CD & Testing Automated unit/integration/endtoend tests; GitOps workflow. Observability Azure Monitor, Log Analytics, dashboards for pipeline health. Soft Skills Clear communication, stakeholder management, selfstarter in a fastmoving team. 4. Preferred / NicetoHave Databricks Certified Data Engineer (Associate/Professional). Microsoft Certified: Azure Data Engineer Associate. Experience with Genie AIassisted pipeline generation and Fabric Copilot. Knowledge of Delta Lake TimeTravel, ZOrdering, and Fabric Direct Lake query optimizations. Exposure to MLflow or Azure ML for modelserved pipelines. Regards Mohit : https://www.linkedin.com/in/mohit-saini-b21b73230/ -- Keywords: continuous integration continuous deployment artificial intelligence machine learning business intelligence information technology Position : Data Engineer with StrongDatabricks [email protected] http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=2747596&uid=4e6f06da79584cd3a5a67e3c66695685 |
| [email protected] View All |
| 01:40 AM 10-Sep-25 |