| Cloud Data Architecture Lead at Remote, Remote, USA |
| Email: [email protected] |
|
From: shivani sharma, adventatech [email protected] Reply to: [email protected] Job Description - Cloud Data Architecture Lead USC, GC Skype Need LinkedIn Requirements AWS Certification: Maintain and leverage AWS certification to design and implement cloud solutions. Cloud Architecture: Design and optimize cloud architecture for scalability and efficiency. Containers & Orchestration: Implement containers and orchestration tools for streamlined application deployment. Microservices Architecture: Design and manage microservices architecture for flexible and scalable systems. Cloud Environment Setup and Configurations: Set up and configure cloud environments to meet project requirements. Security & Access Management: Ensure secure access management and compliance within cloud environments. SQL, Python, Visualization & Analytical Tools: Use SQL, Python, and analytical tools for data processing and visualization. API Development & Management: Develop and manage APIs for seamless data integration and functionality. Education & Experience: Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field. 8+ years of experience in cloud architecture, data engineering, or a similar technical role, with at least 5 years of hands-on experience working with AWS and Databricks. Proven track record in architecting and deploying large-scale data engineering solutions on AWS and Databricks. Experience working with various data processing frameworks (e.g., Apache Spark, Apache Kafka, Airflow) and cloud-based data storage solutions. Technical Skills & Competencies: Deep expertise in AWS services, including but not limited to S3, EC2, Lambda, Glue, Redshift, Athena, and RDS. Strong experience with Databricks, including notebook creation, Spark-based processing, and managing Databricks clusters. Expertise in data engineering concepts, including ETL/ELT, data lakes, data pipelines, and real-time streaming architectures. Proficiency in programming languages such as Python, Scala, SQL, or Java for data processing and solution development. Experience with DevOps practices, CI/CD pipelines, containerization (e.g., Docker, Kubernetes), and infrastructure as code (e.g., Terraform, CloudFormation). Familiarity with machine learning workflows and tools, particularly those that integrate with Databricks (e.g., MLflow, Spark MLlib). Strong understanding of cloud security best practices, including IAM, encryption, and network security. Preferred Qualifications: Experience with big data frameworks (e.g., Hadoop, Spark) and container orchestration platforms (e.g., Kubernetes). Familiarity with data governance, privacy, and compliance frameworks. Key Responsibilities: Lead the design and implementation of cloud-based data architectures on AWS, utilizing a wide range of AWS services (e.g., S3, EC2, Lambda, RDS, Redshift, Athena, Glue). Architect and deploy scalable, secure, and cost-effective Databricks environments to process large volumes of data for analytics, data engineering, and machine learning. Provide leadership in designing modern data architectures, including real-time data pipelines, ETL/ELT workflows, and big data processing systems using Databricks and AWS technologies. Define and implement best practices for managing and optimizing data lakes, data warehouses, and data pipelines. Ensure architecture decisions align with business requirements, security policies, and compliance standards. Keywords: continuous integration continuous deployment sthree green card Cloud Data Architecture Lead [email protected] |
| [email protected] View All |
| 03:56 AM 27-Dec-24 |