| Databricks Administrator (4 days week onsite) || Mountain View, CA. at Mountain View, California, USA |
| Email: himanshu@dmsvisions.com |
|
From: Himanshu, DMS Visions INC himanshu@dmsvisions.com Reply to: himanshu@dmsvisions.com Hi, Hope you are doing well. This is contract position, please revert me with updated resume if this JD matches your profile and you are interested, Title: Databricks Administrator (4 days/week onsite) Location: Mountain View, CA. Duration: 12+ Months Job Description: We are seeking a Databricks Administrator with subject matter expertise in AWS services and products for a long term contract position. The end client is just beginning their Databricks journey and is seeking a seasoned administrator to leadin setting up, managing, and securing the Databricks platform within an AWS environment, ensuring it runs smoothly for the entire team. Key Responsibilities: Databricks & Cloud Platform Administration: Deploy, configure, and maintain Databricks workspaces on AWS, ensuring high availability and security. Manage cluster configurations, autoscaling, and compute resources for optimal performance. Administer Unity Catalog, enforcing governance policies for structured and unstructured data assets. Security, Compliance & Data Governance: Implement Unity Catalog for centralized access control, enforcing RBAC (Role-Based Access Control), attribute-based policies, and lineage tracking. Define fine-grained permissions for tables, views, and files stored in AWS S3. Ensure compliance with banking industry regulations (e.g., SOX, PCI-DSS, GDPR) through data masking, encryption, and audit logging. Manage IAM roles and authentication mechanisms between AWS and Databricks. Performance Optimization & Monitoring: Optimize Spark job performance and cluster tuning using best practices. Monitor Databricks workspace health with CloudWatch, Datadog, or Splunk. Automate cost control measures through idle cluster termination, autoscaling policies, and query optimizations. Integration & Automation: Integrate Databricks with AWS services (S3, Glue, Redshift, Kinesis, IAM) and external databases. Automate workspace provisioning and Unity Catalog policies using Terraform or Infrastructure-as-Code (IaC). Implement CI/CD pipelines for Databricks notebooks, jobs, and configurations. Collaboration & Support: Work closely with data engineers, cloud architects, security teams, and compliance officers to align Databricks with banking security policies. Troubleshoot platform issues, user access problems, and data pipeline failures. Develop best practices and documentation for Databricks administration and Unity Catalog implementation. Qualifications & Experience: 5+ years of experience administering Databricks on AWS in an enterprise environment. Hands-on experience with Unity Catalog, including fine-grained permissions, lineage tracking, and governance policies. Strong knowledge of AWS services (S3, Glue, Redshift, IAM, Lambda, Kinesis). Expertise in Spark cluster management, Databricks Jobs, and SQL analytics. Experience in cloud security frameworks and data compliance for regulated industries. Familiarity with Terraform, CloudFormation, or Infrastructure-as-Code for automation. Understanding of networking & VPC configurations for Databricks on AWS. Preferred Skills: Experience with Databricks Delta Lake, MLflow, and AI/ML workloads. Proficiency in Python, Bash, or Power for scripting and automation. Knowledge of Airflow, Kafka, or other data orchestration tools. Databricks Certified Associate/Professional Administrator certification is a plus. Skills : Access Control Identity and Access Management (IAM) Airflow Cloud Computing Databricks Products Role-Based Access Control (RBAC) Python (Programming Language) Apache Spark Amazon S3 Terraform Amazon Web Services (AWS) If you are interested, please share your updated resume and suggest the best number & time to connect with you. Thanks & Regards Himanshu Gupta US IT RECRUITER, DMS VISIONS INC | (972) 433-0229 Ext-104 | ( 470) 467-9946 | dmsvisions.com | himanshu@dmsvisions.com LinkedIn: linkedin.com/in/himanshu-gupta-1888a1207 | 4645 Avon Lane, Suite 210, Frisco, TX 75033 | | Keywords: continuous integration continuous deployment artificial intelligence machine learning sthree information technology California Texas Databricks Administrator (4 days week onsite) || Mountain View, CA. himanshu@dmsvisions.com https://jobs.nvoids.com/job_details.jsp?id=2147224 |
| himanshu@dmsvisions.com View All |
| 10:57 PM 05-Feb-25 |