Home

Hemanth T - Sr Software Engineer

[email protected]

Location: Remote, Remote, USA

Relocation:

Visa: F1

Hemanth T Email: [email protected] PROFESSIONAL SUMMARY 8 years of IT experience with specializing in Data Engineering and Cloud Solutions, designing and optimizing end to end ETL (Extract, Transform, Load) data pipelines, data warehousing, and analytics platforms across Amazon Web Services (AWS) and Google Cloud Platform (GCP). Architected multi cloud lakehouse platforms (AWS & GCP) with curated marts on Snowflake/Delta Lake, lifecycle policies, and workload isolation. Defined enterprise data standards, SLAs, guardrails, governance, and lineage; produced audit-ready policy evidence (HIPAA/PCI/SOX). Designed robust ETL/ELT ecosystems spanning batch and streaming, with CDC ingestion, medallion (bronze/silver/gold) layering, schema evolution controls, and reusable transformation patterns for reliability and scale. Built both batch and streaming ELT that actually scales. Using Databricks/Spark, Kafka/Kinesis, Glue/ADF, and EMR, I implemented medallion-layer pipelines with schema evolution, SCD 2 history, DLQs, and idempotent consumers. Architect cloud-native data lakes and warehouses on AWS and GCP, balancing performance and cost via storage formats, partitioning, compression, lifecycle policies, workload isolation, and curated marts optimized for BI and self service. Delivered CI/CD for data & Machine Learning with Jenkins, Git, Docker, Kubernetes, Terraform, environment promotions, canaries, and automated rollbacks. Established observability in CloudWatch, Datadog, Grafana, Splunk, SLIs/SLOs, golden signals, freshness, latency, error alerts, reduced MTTR with runbooks. Built governed feature stores on Delta Lake (PySpark) with full SCD-2 history for cross-domain analytics and ML reuse. Productionized ML on SageMaker/Databricks (XGBoost, LightGBM, scikit learn), imbalance handling, model calibration. Model data using 3NF, star, and snowflake schemas with conformed dimensions, surrogate keys, governed metrics, and versioned, well-documented datasets powering BI and ML feature stores with predictable semantics. Implement cloud-first ingestion frameworks for RDBMS, APIs, files, and streams using S3, Glue, Lambda, EMR, and Step Functions on GCP, deliver analytics flows with BigQuery, Dataflow, and Pub/Sub tuned with quotas and reservations. Establish platform CI/CD for data services (Jenkins, Git, Docker, Terraform) with environment promotion, change approvals, artifact versioning, contract checks, and automated validations treating pipelines and infrastructure as code. Implemented practical MLOps not just a registry. MLflow tracks lineage, metrics, and versions; Jenkins runs unit tests, data/feature drift checks, and simple bias screens; approved models promote automatically with canary traffic and rollback if health gates fail. Build observability and reliability with CloudWatch, Grafana, Datadog, and Splunk defining SLIs/SLOs, golden signals, proactive alerting, and runbooks to reduce MTTR and prevent repeat incidents. Delivered real-time Machine Learning with guard railed MLOps, streamed features via Kafka into stateless services on Kubernetes, exposing gRPC/REST scoring APIs consumed by claims adjudication, CRM, and ops apps backed by an MLflow model registry, CI gates for unit/bias/drift checks, and canary deployments with automatic rollback for safe, repeatable releases. Institute data quality at scale through schema/bounds/freshness checks, reconciliation and anomaly detection, automated remediation (including DLQs and replay), and lineage/metadata so consumers can trust data end-to-end. Partner with product, BI, and data science to translate ambiguous questions into technical plans, prioritize roadmaps, and ship curated, reusable data assets and feature sets with measurable business impact. TECHNICAL SKILLS Cloud Platforms AWS (S3, EC2, Glue, Redshift, Lambda), GCP (BigQuery, Dataflow, Pub/Sub) Data Engineering Snowflake, Databricks, Apache Spark (PySpark/Spark SQL), Delta Lake, Kafka Programming Java, JavaScript, Python, JSON, REST APIs DevOps & CI/CD Jenkins, Git, Docker, Kubernetes, Terraform Enterprise Platforms ServiceNow, Microsoft Dynamics CRM Databases & BI MySQL, PostgreSQL, SQL Server, Looker, Tableau, Power BI Other Tools CloudWatch, Grafana, Jira, Confluence PROFESSIONAL EXPERIENCE Sr Software Engineer | Prudential Financial May 2024 Present Location: Atlanta, Georgia. Define enterprise-wide data architecture standards, SLAs, and guardrails aligned to regulatory goals (HIPAA/PCI) and cost controls. Drive multi-cloud lakehouse patterns on AWS & GCP with robust lifecycle policies, workload isolation, and curated marts in Snowflake/Delta Lake. Build CI/CD for data services using Jenkins, Git, Docker, Kubernetes, and Terraform, with contract checks and automated validations. Enforce encryption, tokenization, IAM least-privilege, and VPC endpoints; document lineage and policy evidence for audits. Implement observability with CloudWatch, Grafana, and Datadog; define SLIs/SLOs and golden signals to reduce MTTR. Optimize Databricks/Spark jobs (AQE, partitioning, broadcast joins) and Snowflake (micro-partition pruning, result cache) to improve cost/perf. Implement advanced Spark job optimizations including AQE and caching to ensure stability under fluctuating data loads. Built an end-to-end claims fraud scoring system. I pulled together claims, policy, and provider data and organized it into a governed Delta Lake feature store (SCD-2) on Databricks with PySpark. Trained and tracked XGBoost/LightGBM models on SageMaker (stratified CV, class-imbalance handling, calibrated thresholds). Put it in production with real-time scoring, Kafka feeds stateless Docker services on Kubernetes that expose gRPC/REST into the claims adjudication flow. Wrapped it with Jenkins pipelines (unit, drift, bias checks), canary releases with automatic rollback, Terraform-managed infra, Datadog/Grafana monitoring for drift/latency/health, and clear playbooks plus Snowflake views so investigators can understand scores and auditors can verify them. Sr Software Engineer | WellCare May 2023 April 2024 Location: Tampa, FL Designed medallion-layer ELT with schema evolution and reusable transformations on Databricks, enforced quality via expectations and DLQs. Implemented streaming ingestion with Kafka for near-real-time subject areas, tuned backpressure, checkpoints, and idempotency. Right-sized Snowflake warehouses, materialized views, and clustering to meet tight SLAs under budget. Established CI/CD and environment promotion with Jenkins, Git, Docker, and Terraform. Built platform telemetry using CloudWatch, Grafana, Splunk, standardized runbooks for quicker incident recovery. Drive end-to-end architecture of healthcare data platforms ensuring compliance with HIPAA and SOX regulations through encryption, tokenization, and least privilege IAM. Develop real-time event streaming pipelines with Kafka, leveraging schema registries and backpressure controls to support sub-second ingestion SLAs. Optimize Snowflake warehouse sizing, materialized views, and caching to meet stringent latency and budget requirements. Built a Hierarchical Condition Category risk platform end to end, I pulled member, claims, pharmacy, and care-management data from AWS S3 and GCP BigQuery and shaped a governed feature store in Python. Trained gradient-boosting and logistic models with feature selection and SHAP, and kept clear lineage and governance records. Put batch and micro-batch scoring into Databricks/Delta Lake with conformed outputs in Snowflake, then published clinician-friendly Tableau dashboards (member risk, top drivers, confidence bands) with row-level security. Sr Software Engineer | Prudential Financial Feb 2022 Aug 2022 Location: Newark, NJ Built event-driven pipelines on Kafka and AWS Kinesis with idempotent consumers, schema registry, and DLQs. Engineered Snowflake warehouses with Streams/Tasks, clustering, and micro-partition pruning. Tuned Databricks/Spark jobs (AQE, caching, Z-ORDER) and checkpointing for stability and speed. Orchestrated ingestion on AWS with S3, Glue, Lambda, Step Functions; enforced governance and masking aligned to PCI. Established SLIs/SLOs and alerts in CloudWatch/Datadog for pipeline reliability. Migrated on-prem ETL to AWS with dual-run parity checks, backfills, and rollback plans; automated IAC with Terraform and CI with Jenkins. Implemented cloud-native data lakes on AWS using S3, Glue, Lambda, and Step Functions for ingestion orchestration. Built observability frameworks leveraging CloudWatch and Datadog, creating SLIs/SLOs and alerting for data pipeline reliability. Collaborated with product and analytics teams to translate business requirements into scalable data solutions. Automated CI/CD pipelines with Terraform and Jenkins to manage infrastructure as code for data platforms. Built a policy-lapse risk solution end to end: trained SageMaker models with time-based cross-validation using survival features engineered in PySpark on Delta Lake. Streamed behavioral web/app events via Kafka and Kinesis Firehose into curated feature tables, with reconciliation for late-arriving data. Served batch and near-real-time scores to Snowflake consumer marts and a CRM-facing API, layered with business rules for eligibility and exclusions. Operationalized with MLflow for versioning/approvals/rollbacks, Dockerized inference on Kubernetes, and performance tracking Software Engineer | Avya IT Pvt. Ltd. Jul 2018 Dec 2021 Location: Hyderabad, India Implemented scalable batch ETL on Apache Spark and Delta Lake; migrated legacy SQL jobs to Spark SQL for higher throughput. Built CDC with SCD-2, schema-evolution controls, and reusable transformations; improved observability with lineage and metadata. Integrated AWS S3 and on-demand EMR clusters; automated deployments with Jenkins and Docker; collaborated on governed marts for BI. Migrated Microsoft Dynamics CRM entities to an AWS lake using Java microservices and Glue jobs; modeled conformed dimensions and facts for analytics. Engineered customer 360 feature tables in Delta Lake; implemented quality checks, deduplication, and survivorship rules. Trained customer churn and upsell models in Amazon SageMaker (scikit-learn/XGBoost) using interaction, usage, and case history features. Exposed curated outputs to Power BI for sales and service; created score-aware dashboards and drill-downs for account managers. Containerized batch scoring with Docker; orchestrated runs via Jenkins; stored predictions and explanations in Snowflake for auditability. Instrumented model/data SLAs with Datadog (freshness, coverage, drift) and set alert thresholds for timely remediation. Associate Software Engineer | Dhatsol IT Solutions Pvt. Ltd. Jan 2017 - June 2018 Location: Hyderabad, India Built ETL with SQL/Python from RDBMS to the warehouse; designed 3NF/star schemas for BI and marts; automated orchestration with Airflow. Implemented validation and reconciliation checks; documented workflows and supported incident triage with clear runbooks. Ingested ServiceNow incidents/changes via Azure Data Factory; standardized categories and SLA mappings; deduplicated and stored curated tables in Azure SQL with audit columns for traceability. Built rule-based resolution-time calculators (median/percentile TTR by priority, assignment group, CI) and backlog risk indicators using static thresholds and SLA policies; implemented aging buckets for WIP tracking. Implemented control charts and time-window baselines to highlight unusual ticket volumes; surfaced exceptions using threshold checks rather than predictive models. Published interactive Power BI dashboards (SLA burn-down, backlog heatmaps, hot CIs, agent workload) with row-level security for support teams and managers. Automated daily data refreshes and KPI validations with ADF pipelines; triggered email/Teams notifications via Logic Apps when thresholds were breached. EDUCATION Northern Illinois University DeKalb, IL Master of Science in Computer Science Vignan s Foundation for Science Technology and Research Guntur, India Bachelor of Technology in Electronics and Communication Engineering Keywords: continuous integration continuous deployment machine learning business intelligence sthree information technology Florida Illinois New Jersey