| Senior Azure Data Engineer - Dallas, Texas at Dallas, Texas, USA |
| Email: [email protected] |
|
From: Priti Borkar, Quillion.ai [email protected] Reply to: [email protected] Hello Team, Please find the below job description, Job Title: Senior Azure Data Engineer - Dallas, Texas 3 days hybrid Local to TX only Job Overview We are seeking a highly experienced Senior Azure Data Engineer with a minimum of 8+ years of overall IT experience and a minimum of 5+ years working in Azure Cloud environments. The ideal candidate will be an expert in designing, developing, and optimizing enterprise-scale data solutions using Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage Gen2 (ADLS Gen2). This role requires strong expertise in big data processing using PySpark, Scala, or Python, and deep knowledge of SQL Server for ETL/ELT processing, query optimization, and data warehousing. The primary focus will be building scalable, secure, and high-performance data architectures to drive business intelligence, analytics, and machine learning initiatives. Key Responsibilities Design and implement data ingestion, transformation, and movement using Azure Data Factory (ADF), Azure Synapse, and Data Lake Collaborate with business stakeholders, data scientists, and engineers to build robust data solutions. Develop and manage high-volume, real-time, and batch data ingestion pipelines using Azure Data Factory (ADF). Implement event-driven architectures for real-time data movement and processing. Develop large-scale data processing solutions using Azure Databricks and Apache Spark with PySpark, Scala, or Python. Optimize data partitioning, caching, and indexing for efficient performance. Manage complex transformations and aggregations for structured and unstructured datasets. Design and implement high-performance data models in Azure Synapse Analytics using dedicated SQL Pools and Spark Pools. Optimize query performance, workload management, and cost efficiency in Synapse Analytics. Implement column store indexes, partitioning strategies, and data caching to enhance performance. Design and manage secure, scalable data lakes using Azure Data Lake Storage Gen2 (ADLS Gen2). Implement RBAC (Role-Based Access Control), encryption, and data masking to ensure security. Implement Azure Monitor, Log Analytics, and Application Insights for data pipeline monitoring and troubleshooting. Optimize cost management, auto-scaling, and performance tuning across Azure services. Key Skills Required 8+ years of experience in data engineering, ETL development, and cloud-based data integration Strong experience with Azure Data Factory (ADF): ETL/ELT pipeline orchestration and data movement. Azure Databricks: Large-scale data transformation and big data processing with Apache Spark, PySpark, Scala, or Python. Azure Synapse Analytics: Data warehousing, SQL Pools, Spark Pools, and performance tuning. Azure Data Lake Storage Gen2 (ADLS Gen2): Secure, scalable data lake architecture. SQL Server: Advanced T-SQL, stored procedures, indexing, and query optimization. Nice-to-Have Skills Power BI, Tableau, or other BI tools for data visualization. Financial/Banking experience. Best Regards, Priti Borkar [email protected] Keywords: artificial intelligence business intelligence information technology Texas Senior Azure Data Engineer - Dallas, Texas [email protected] |
| [email protected] View All |
| 04:48 AM 12-Mar-25 |