Home

Yashwanth Majji - Senior AI/ML Engineer

[email protected]

Location: Charlotte, North Carolina, USA

Relocation:

Visa: Green Card

Resume file: Yashwanth_M_Sr AI Engineer_1793033544572.docx Please check the file(s) for viruses. Files are checked manually and then made available for download.

PROFESSIONAL SUMMARY Senior Data Scientist and data engineering professional with 12 years of experience building enterprise data pipelines, analytics solutions, and machine-learning systems across banking, healthcare, retail, and insurance. Hands-on experience with Azure Databricks, PySpark, Delta Lake, Azure Data Factory, ADLS, SQL, and modern lakehouse patterns for batch and streaming ingestion, transformation, cleansing, validation, and analytics. Developed reusable data ingestion and transformation workflows using Databricks Auto Loader, Spark Structured Streaming, Kafka, Airflow, and Dataflow, with schema validation, retry handling, and data-quality checkpoints. Designed curated enterprise data layers and integrated data from multiple applications and sources to support analytics, reporting, forecasting, and downstream machine-learning use cases. Applied practical data governance controls including PII redaction, de-identification, metadata-based access filters, role-based access, data contracts, and documented compliance practices for sensitive financial and healthcare data. Partnered with business and technical stakeholders to translate domain requirements into data processing, analytics, and production delivery requirements; documented architectures, data workflows, APIs, and operational runbooks. Built automated data validation and monitoring for nulls, malformed records, schema issues, data drift, and pipeline failures, improving reliability and trust in analytical datasets. Experienced in cloud data platforms across Microsoft Azure, Google Cloud Platform, and AWS, with a focus on scalable architecture, reusable engineering patterns, secure data handling, and production operations. TECHNICAL SKILLS Programming & Query: Python, SQL, PySpark, Scala, Java, Bash Data Platforms & Engineering: Azure Databricks, Apache Spark, Delta Lake, Azure Data Factory (ADF), ADLS, ETL/ELT, batch and streaming pipelines, data warehousing, lakehouse architecture Data Quality & Governance: data profiling, schema validation, data cleansing, data contracts, metadata, lineage documentation, data dictionaries, business definitions, data ownership/stewardship concepts, PII redaction, access controls Cloud: Microsoft Azure, Google Cloud Platform (GCP), AWS Orchestration & Streaming: Databricks Workflows, Apache Airflow, Kafka, Spark Structured Streaming, Google Dataflow, Pub/Sub Databases & Analytics: Azure SQL, SQL Server, PostgreSQL, Snowflake, BigQuery, Pandas, NumPy, statistics, feature engineering Monitoring & Delivery: Azure Monitor, CloudWatch, Prometheus, MLflow, Git, GitHub Actions, Jenkins, Terraform, Docker, Kubernetes, Agile, Jira, Confluence Reporting & Visualization: analytical dashboards and operational monitoring; Power BI exposure not listed in source resume PROFESSIONAL EXPERIENCE Bank of America - Charlotte, NC || March 2024 - Present Senior Machine Learning Engineer Responsibilities: Standardized streaming and batch ingestion patterns across Kafka, Databricks Auto Loader, ADLS, and Delta Lake, applying schema checks and reusable transformation logic before data was consumed by downstream analytics and AI services. Documented source-to-target data flows, data schemas, metadata filters, and governance controls for policy and customer-context datasets to improve traceability and consistency across retrieval and analytics components. Collaborated with banking domain stakeholders and engineering teams to clarify data interpretation, sensitive-data handling, access requirements, and validation rules for enterprise datasets. Led the end-to-end development of an enterprise-grade real-time GenAI agent-assist copilot using Azure OpenAI and LangChain to automatically summarize live customer calls and recommend context-aware, policy-aligned next steps within a sub-three-second latency target. Designed a highly scalable, event-driven microservices architecture using Kafka and Spark Structured Streaming to isolate transcript ingestion, retrieval, and generation processes across the copilot platform. Acted as the sole technical owner of the engineering lifecycle, coordinating directly with domain mentors during weekly product demos in Agile delivery cycles to translate complex banking compliance requirements into production-ready software scope. Structured a robust streaming data ingestion engine utilizing Databricks Auto Loader to consume live call transcriptions from Kafka pipelines and corporate policy updates from secure ADLS cloud storage containers. Built automated PII redaction and intent-tagging routines in PySpark to strip sensitive customer banking credentials and dynamically classify conversational focus areas before any downstream language model processing. Orchestrated optimized Delta Lake medallion layers on ADLS, integrating low-latency Azure SQL database lookups to provide unified historical customer profiles across all streaming analytics and evaluation workloads. Indexed extensive corporate financial policies into Azure AI Search, configuring a hybrid vector and keyword search engine equipped with granular metadata filters based on user roles and document versions. Selected high-performance Azure OpenAI GPT-4o for complex recommendation reasoning, while routing secondary classification queries to a smaller model to balance structural accuracy, API costs, and infrastructure footprint. Engineered a robust RAG pipeline featuring advanced cross-encoder reranking algorithms and cited evidence, constructing a strict fallback response protocol whenever available documentation was found to be statistically insufficient. Decreased critical p95 system response times and lowered ongoing token expenditure by integrating semantic prompt caching, optimized token limits, and chunked response streaming across concurrent agent desktop terminals. Developed production-ready ReAct-style autonomous agents using LangChain and LangGraph, embedding automated confidence thresholds, mandatory approval gates, and fluid human-in-the-loop escalation paths for ambiguous financial transactions. Applied strict object-oriented design patterns, strongly typed Python modules, and Pydantic data schemas to maintain highly reusable components across separate retrieval, prompt engineering, and localized tool execution modules. Established a meticulous automated validation framework by testing deployment candidates against a curated golden evaluation set, leveraging RAGAS faithfulness and relevance metrics alongside continuous manual technical reviews. Containerized the core FastAPI inference microservices and related streaming workers using Docker, publishing immutable, versioned container images to Azure Container Registry to facilitate instant, zero-downtime platform rollbacks. Deployed the containerized serving layer onto Azure Kubernetes Service (AKS) clusters with robust health probes, configuring horizontal pod autoscaling to seamlessly absorb extreme spikes in real-time banking call traffic. Automated deep continuous integration and deployment pipelines using GitHub Actions, integrating mandatory automated security image scanning, quality gates, and multi-stage testing environments for code promotions. Provisioned isolated, immutable infrastructure dependencies including AKS clusters, search registries, and key vaults using Terraform scripts to guarantee complete environment parity between development and production stages. Created live operational observability dashboards via Azure Monitor and Prometheus, tracking key metrics like retrieval quality, prompt latencies, token consumption rates, and structural agent execution errors. Wrote comprehensive PyTest unit and integration tests utilizing mocked LLM APIs, executing extreme stress and load testing using Locust to validate system resilience under heavy simulated agent traffic. Documented deep technical architecture diagrams, OpenAPI specifications, compliance data governance logs, and operational runbooks within Confluence to support seamless platform handoff to internal enterprise engineering teams. Environment: Azure OpenAI, GPT-4o, Python, FastAPI, LangChain, LangGraph, RAG, Kafka, Spark Structured Streaming, Databricks, PySpark, Delta Lake, ADLS, Azure SQL, Azure AI Search, Docker, Azure Container Registry, Azure Kubernetes Service (AKS), GitHub Actions, Terraform, Azure Key Vault, Azure Monitor, Prometheus, PyTest, Locust, Confluence, Agile. CVS Health - Hartford, CT || May 2022 - Feb 2024 Machine Learning Engineer Responsibilities: Integrated pharmacy, provider, patient, and wearable data into governed analytical stores, applying schema checks, de-identification, and consistent transformation logic across ingestion workflows. Documented dataset structures, source relationships, privacy controls, and operational data flows to support traceability and reliable downstream analytics across healthcare data sources. Designed and developed an enterprise-level agentic AI health companion using Gemini and LangGraph to efficiently guide active patients through intricate medication schedules, insurance coverage limits, and prescription cost questions. Architected a robust multi-agent backend ecosystem powered by GCP Pub/Sub and Dataflow pipelines to securely ingest, process, and unify siloed pharmacy metrics, provider data, and real-time wearable device streams. Maintained complete technical execution ownership spanning data pipeline construction, model evaluation, and multi-cloud deployment, leveraging rapid Agile sprint demonstrations to consistently refine evolving clinical requirements with stakeholders. Processed synthetic FHIR patient records and pharmacy fulfillment events via Dataflow, engineering resilient dead-letter queues and automated retry mechanisms to handle intermittent network exceptions gracefully without losing critical data. Stripped highly sensitive healthcare indicators from real-time data streams utilizing Cloud DLP, applying robust data de-identification practices before storing records inside a secure, interoperable Cloud Healthcare API FHIR store. Stored unified pharmacy, provider, and patient datasets in the Cloud Healthcare API FHIR store and BigQuery, providing the secure analytical and retrieval backbone for the health companion workloads. Indexed comprehensive medication user guides and provider benefit catalogs inside Vertex AI Vector Search, leveraging precise multi-field metadata filters to isolate relevant clinical documentation by specific insurance plan types. Selected Gemini multimodal architectures for complex reasoning across diverse document types, utilizing highly efficient, lighter models for quick intent routing to minimize operational latencies during high-concurrency periods. Implemented a dynamic RAG pipeline featuring hybrid vector-keyword search and cross-encoder reranking algorithms, ensuring all generated medical answers are strictly anchored to verified documentation with visible inline citations. Optimized prompt structures, context assembly mechanisms, and distributed response caching tiers to systematically lower token consumption and end-to-end processing latencies across highly conversational, multi-turn patient healthcare dialogs. Created highly resilient ReAct-style autonomous agents with LangGraph, embedding rigorous programmatic validation, manager approval checkpoints, and secure human-escalation pathways for high-risk, low-confidence, or clinical medical queries. Applied strict object-oriented development principles, typed Python modules, and Pydantic schemas to ensure all conversational tools, connectors, and agent state machines remain highly modular, readable, and easily testable. Validated system safety thresholds by performing comprehensive offline golden-set evaluations, checking core metrics like groundedness and task success while subjecting model layers to aggressive adversarial prompt injection attacks. Containerized core FastAPI microservices, asynchronous agent workers, and custom data processing jobs via Docker, securely pushing production-ready artifacts into a centralized Google Cloud Artifact Registry repository. Deployed core application components onto Google Kubernetes Engine (GKE) featuring dynamic cluster autoscaling, while deploying lightweight webhooks on Cloud Run to efficiently absorb sudden conversational traffic spikes. Automated continuous build, integration, and container image scanning workflows via GitHub Actions, managing staged deployment sequences across isolated development, testing, and production cloud environments seamlessly. Managed cloud infrastructure via Terraform scripts, provisioning isolated networking boundaries with VPC Service Controls under an enterprise-grade, least-privilege IAM access control framework to safeguard critical medical workloads. Set up comprehensive Cloud Monitoring and Cloud Logging infrastructure, building custom engineering dashboards to track real-time conversation turn lengths, prompt execution latencies, and vector index retrieval accuracy anomalies. Wrote rigorous PyTest unit and integration suites featuring completely mocked LLM behaviors and FHIR endpoints, conducting rigorous automated user acceptance testing based on complex, pre-scripted patient interaction scenarios. Documented system infrastructure topologies, OpenAPI specifications, strict data privacy controls, and operational troubleshooting runbooks in Confluence, collaborating with healthcare compliance officers to align all system outputs with enterprise Responsible-AI principles and data governance protocols. Environment: Google Cloud Platform (GCP), Gemini, Vertex AI, Vertex AI Vector Search, BigQuery, Cloud Healthcare API, FHIR, Pub/Sub, Dataflow, PySpark, Python, FastAPI, LangChain, LangGraph, RAG, Cloud DLP, Docker, Artifact Registry, Google Kubernetes Engine (GKE), Cloud Run, GitHub Actions, Terraform, Cloud Monitoring, Cloud Logging, PyTest, Confluence, Agile. Costco - Issaquah, WA || Oct 2020 - April 2022 Machine Learning Engineer Responsibilities: Established reusable PySpark and Databricks data preparation patterns for ingesting, cleansing, validating, and curating sales, pricing, promotion, and calendar data for analytical and forecasting workloads. Implemented data-quality checks for null values, malformed records, stale inputs, and schema issues, and organized curated Delta Lake datasets to support consistent downstream analysis. Documented pipeline architecture, transformation logic, data dependencies, and operational procedures to improve maintainability and handoff across analytics and platform teams. Built an end-to-end perishables demand-forecasting pipeline using PySpark, LightGBM, and MLflow on Databricks to significantly optimize production schedules, refine regional ordering quantities, and minimize costly grocery inventory spoilage. Architected a scalable batch-scoring framework built upon a modern Delta Lake medallion lakehouse, orchestrating complex data transformation steps via Apache Airflow alongside a FastAPI consumption layer. Translated intricate supply chain forecasting goals into concrete machine learning features during intensive one-week Agile sprints, establishing clear technical tradeoffs and accuracy thresholds during collaborative reviews with planners. Ingested massive historical grocery sales records, localized promotional calendars, and national holiday datasets from multi-cloud storage buckets and external REST APIs using scheduled Airflow DAGs. Engineered predictive historical features in PySpark, computing multi-day demand lags, rolling averages, price elasticity metrics, and promotional lift indicators to capture highly volatile, short-term grocery purchasing patterns effectively. Accelerated model training performance and downstream analysis queries by organizing massive database structures with optimized partitioning and Z-ordering indexes tuned directly on granular store-product compound keys. Established a centralized Databricks Feature Store with built-in automated data-quality validation checks to proactively block malformed records, null values, or stale inputs prior to initiating scheduled training pipelines. Selected LightGBM gradient boosting models after conducting exhaustive benchmark evaluations against Prophet and XGBoost baselines, capitalizing on its exceptional training velocity over massive, highly granular store-item time series. Generated highly actionable quantile demand forecasts providing lower, expected, and upper-bound target scenarios, empowering supply chain inventory planners to make data-driven, risk-managed stock purchasing decisions. Automated high-dimensional hyperparameter optimization routines using Optuna, tracking distinct experiments, training parameters, evaluation metrics, and finalized model weights cleanly within a centralized MLflow Model Registry dashboard. Utilized scikit-learn, LightGBM, and Pandas to build a modular, configuration-driven training framework that dynamically scales across diverse store groups without requiring code modifications or manual intervention. Applied disciplined object-oriented principles and strict Python type hinting throughout the codebase to drastically reduce software complexity, ensuring the entire analytics pipeline remains highly readable, maintainable, and extensible. Validated predictive forecasting accuracy via rigorous rolling-origin backtests measured against naive and seasonal baselines, tracking key business metrics including WAPE, forecast bias, and stockout risk parameters. Containerized the core FastAPI forecasting services and related batch inference workloads with Docker, securely storing versioned deployment images inside an enterprise-grade Azure Container Registry for automated deployment pipelines. Deployed the final predictive API tier onto Azure Kubernetes Service (AKS) clusters with health probes, while seamlessly scheduling high-volume nightly batch scoring jobs via native Databricks automated workflows. Automated continuous testing infrastructure, docker container compilation, and staged release gates using GitHub Actions pipelines to systematically promote verified machine learning candidates into the live production environment. Provisioned scalable cloud storage accounts, container registries, Kubernetes clusters, and isolated virtual networks using Terraform scripts to guarantee absolute structural identity between testing and production infrastructure tiers. Monitored forecast quality and data drift using Azure Monitor and Evidently, tracking pipeline health and model performance signals to flag degrading forecasts before they affected regional ordering decisions. Wrote PyTest unit and integration tests covering feature engineering logic, training pipelines, and the FastAPI forecasting endpoints to validate system behavior ahead of each staged production release. Documented pipeline architecture, forecasting methodology, and operational runbooks in Confluence, tracking delivery and planner feedback in Jira to support seamless handoff to supply chain and platform engineering teams. Environment: Python, PySpark, Databricks, LightGBM, XGBoost, Prophet, scikit-learn, Pandas, MLflow, Optuna, Airflow, Delta Lake, ADLS, FastAPI, Docker, Azure Container Registry, Azure Kubernetes Service (AKS), GitHub Actions, Terraform, Azure Monitor, Evidently, PyTest, Confluence, Jira, Agile. Homesite Insurance - Boston, MA || May 2018 - Sept 2020 Machine Learning Engineer Responsibilities: Developed an automated claims-severity and fraud-triage prediction pipeline using XGBoost and PySpark on AWS to help investigators intercept high-risk filings and prioritize manual review queues. Designed a scalable batch architecture on an AWS S3 data lake, using Amazon EMR for distributed processing and Apache Airflow for workflow scheduling. Managed the complete development loop as the primary technical owner, converting manual triage processes into structured, data-driven pipelines in partnership with fraud investigators and claims stakeholders. Consolidated claims, policy rules, and customer background profiles from relational databases into partitioned Parquet files on S3 with strict schema verification. Built sophisticated data-cleaning steps to handle highly skewed claim amounts, applying logarithmic transformations and custom outlier handling across the feature preparation stages. Stored curated claims datasets as partitioned Parquet on the S3 data lake with Hive-compatible organization, enabling fast distributed querying and repeatable batch scoring runs. Engineered large-scale claims and policy features in PySpark on EMR, building the modeling datasets that fed both severity prediction and fraud-triage scoring. Selected XGBoost after comparing multiple algorithms (Random Forest, LightGBM, Logistic Regression) for its superior performance on insurance tables. Built claims-severity prediction and fraud-triage classification models that scored incoming filings so investigators could intercept high-risk claims early. Handled extreme class imbalance (rare fraud incidents) by applying stratified cross-validation, class weighting, and synthetic resampling to stabilize model performance. Interpreted complex model decisions by integrating SHAP values, giving fraud investigators completely transparent, explainable feature drivers behind every alert score. Structured feature engineering and scoring code into reusable, modular Python components, keeping the training and inference logic consistent across batch and API scoring paths. Tracked experiments, parameters, and evaluation results in MLflow, validating candidate models against held-out claims data before promoting them into the production scoring pipeline. Packaged the scoring backend inside Docker containers, serving real-time predictions via a Flask API behind an AWS Load Balancer alongside EMR batch runs. Scheduled and orchestrated recurring batch scoring on EMR through Apache Airflow, coordinating feature preparation, model inference, and investigator alert delivery. Configured automated build, test, and container scanning workflows within a Jenkins CI/CD pipeline, tracking versions inside MLflow. Provisioned isolated cloud environments (S3, EMR, IAM roles) using Terraform templates to guarantee parity between testing and production. Set up AWS CloudWatch alarms and MLflow tracking to immediately flag data drift, sudden score shifts, or execution pipeline failures. Wrote PyTest unit and integration tests for feature logic and the scoring API, adding data-contract checks to block malformed inputs. Documented architecture, API specifications, model cards, and runbooks in Confluence for complete workflow traceability. Environment: Python, SQL, Pandas, NumPy, scikit-learn, XGBoost, LightGBM, SHAP, PySpark, Apache Spark, AWS S3, EMR, SageMaker, CloudWatch, IAM, Parquet, Hive, MLflow, Airflow, Flask, REST APIs, Docker, Jenkins, Git, Terraform, PyTest, Jupyter, Matplotlib, Seaborn, Jira, Confluence, Linux. Zensar Technologies - Hyderabad, India || Aug 2015 - Feb 2018 Python Developer Responsibilities: Created reusable data parsing, cleansing, validation, and transformation components for structured CSV, XML, and JSON data, with checks for missing fields, duplicate rows, and inconsistent formats. Documented data processing rules and SQL-based source relationships to make recurring data workflows more consistent, traceable, and maintainable. Developed core Python applications and background processing scripts to automate high-volume data transformation, reporting, and backend business logic, replacing labor-intensive operational workflows and reducing manual overhead significantly. Designed reusable, object-oriented Python modules and scheduled batch-processing services with clean, maintainable structures that other developers could extend across projects. Collaborated closely with QA and product teams within Agile development lifecycles to review code, patch bugs, and document requirements across the full SDLC. Created automated data parsing scripts to ingest and clean complex CSV, XML, and JSON files into application-ready formats. Utilized Pandas and NumPy to conduct initial data profiling, aggregations, and core analytical processing for business stakeholders. Wrote and fine-tuned complex SQL queries, multi-table joins, and stored procedures to optimize data retrieval speeds across relational databases. Programmed automated data-quality checkpoints to immediately capture and flag missing fields, duplicate rows, and inconsistent data formatting. Integrated custom Python backend modules seamlessly with external enterprise services using REST-based APIs. Automated recurring reporting and transformation jobs as scheduled batch-processing scripts, eliminating repetitive manual steps from daily operations. Built robust exception handling and logging into background scripts so processing failures surfaced early with clear, traceable error context. Applied object-oriented design principles to keep parsing, transformation, and reporting logic modular, readable, and reusable across applications. Ran background processing jobs on scheduled cycles in shared enterprise environments, coordinating releases and fixes with QA and product teams. Monitored scheduled job outputs and application logs to detect failed runs and data anomalies, resolving production issues in coordination with QA teams. Conducted structured unit testing using the unittest framework to identify and fix codebase issues early in the SDLC. Documented requirements, application behavior, and bug fixes for the QA and product teams, supporting smooth handoff and maintenance of the delivered Python applications. Environment: Python, SQL, Pandas, NumPy, REST APIs, CSV, XML, JSON, relational databases, unittest, PyUnit, logging, exception handling, Agile, SDLC. EDUCATION Bachelor of Technology (B.Tech.) in Computer Science, Osmania University Keywords: continuous integration continuous deployment quality analyst artificial intelligence business intelligence sthree Connecticut Delaware Massachusetts North Carolina Washington