| Direct Client Req: Sr. Data Architect - Microsoft Fabric & Azure Databricks at Atlanta, Georgia, USA |
| Email: divya@diasoftwaresolutions.com |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1976534&uid= From: Divya, Dia Software solutions divya@diasoftwaresolutions.com Reply to: divya@diasoftwaresolutions.com Hi , I hope this message finds you well! I am reaching out to you about an exciting Direct client opportunity with one of our clients. Please review the requirements and let me know if you are interested in this position Job Title- Data Architect (Microsoft Fabric & Azure Databricks) Work Arrangement: Hybrid Work Location: Atlanta, Georgia Agency Interview Type: Either Web Cam or In Person Duration of the Project: 6 Months Exp: 15+ Job Description : Data Architecture: Design end-to-end data architecture leveraging Microsoft Fabric's capabilities. Design data flows within the Microsoft Fabric environment. Implement One Lake storage strategies. Configure Synapse Analytics workspaces. Establish Power BI integration patterns. Integration Design: Architect data integration patterns between IES Gateway and the analytics platform using Azure Databricks and Microsoft Fabric. Design Delta Lake architecture for IES Gateway data. Implement medallion architecture (Bronze/Silver/Gold layers). Create real-time data ingestion patterns. Establish data quality frameworks. Lakehouse Architecture: Implement modern data lake house architecture using Delta Lake, ensuring data reliability and performance. Data Governance: Establish data governance frameworks incorporating Microsoft Purview for data quality, lineage, and compliance. Implement row-level security. Configure Microsoft Purview policies. Establish data masking for sensitive information. Design audit logging mechanisms. Pipeline Development: Design scalable data pipelines using Azure Databricks for ETL/ELT processes and real-time data integration. Performance Optimization: Implement performance tuning strategies for large-scale data processing and analytics workloads. Optimize Spark configurations. Implement partitioning strategies. Design caching mechanisms. Establish monitoring frameworks. Security Framework: Design and implement security patterns aligned with federal and state requirements for sensitive data handling. Required Qualifications: Education: Bachelors degree in computer science or related field. Experience: 6+ years of experience in data architecture and engineering. 2+ years hands-on experience with Azure Databricks and Spark. Recent experience with Microsoft Fabric platform. Technical Skills: Microsoft Fabric Expertise: Data Integration: Combining and cleansing data from various sources. Data Pipeline Management: Creating, orchestrating, and troubleshooting data pipelines. Analytics Reporting: Building and delivering detailed reports and dashboards to derive meaningful insights from large datasets. Data Visualization Techniques: Representing data graphically in impactful and informative ways. Optimization and Security: Optimizing queries, improving performance, and securing data Azure Databricks Experience: Apache Spark Proficiency: Utilizing Spark for large-scale data processing and analytics. Data Engineering: Building and managing data pipelines, including ETL (Extract, Transform, Load) processes. Delta Lake: Implementing Delta Lake for data versioning, ACID transactions, and schema enforcement. Data Analysis and Visualization: Using Databricks notebooks for exploratory data analysis (EDA) and creating visualizations. Cluster Management: Configuring and managing Databricks clusters for optimized performance. (Ex: autoscaling and automatic termination) Integration with Azure Services: Integrating Databricks with other Azure services like Azure Data Lake, Azure SQL Database, and Azure Synapse Analytics. Machine Learning: Developing and deploying machine learning models using Databricks ML flow and other tools. Data Governance: Implementing data governance practices using Unity CatLog and Microsoft Purview Programming & Query Languages: SQL: Proficiency in SQL for querying and managing databases, including skills in SELECT statements, JOINs, subqueries, and window functions12. Python: Using Python for data manipulation, analysis, and scripting, including libraries like Pandas, NumPy, and Py Spark Data Modelling: Dimensional modelling Real-time data modelling patterns Preferred Experience: Azure DevOps Infrastructure as Code (Terraform) CI/CD for data pipelines Data mesh architecture THANK YOU & REGARDS DIVYA U | DIA SOFTWARE SOLUTIONS LLC. Austin, TX 78727 divya@diasoftwaresolutions.com Keywords: continuous integration continuous deployment machine learning access management business intelligence Texas Direct Client Req: Sr. Data Architect - Microsoft Fabric & Azure Databricks divya@diasoftwaresolutions.com http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1976534&uid= |
| divya@diasoftwaresolutions.com View All |
| 02:17 AM 03-Dec-24 |