| Senior Data Engineer with strong Kafka streaming Exp - Atlanta, GA and Chicago, IL(Onsite) at Atlanta, Georgia, USA |
| Email: [email protected] |
|
Processing description: http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=3000608&uid=f65fea946a5c40f6a4ff05a9ba2911c7 Your Email Title Role Name: Senior Data Engineer Location: Atlanta, GA and Chicago, IL(Onsite) Duration: 12 Months Role Summary: As a Senior Software Engineer, youll design and deliver real time data pipelines and event driven microservices. Youll implement resilient streaming patterns, contribute to data governance and CI/CD automation, and collaborate closely with platform and product teams to scale our data services. Responsibilities: Build reliable streaming applications using Confluent Kafka, Apache Flink, Hazelcast, Kafka Streams, Kafka Connect, and Schema Registry. Develop ETL/ELT pipelines for real time ingestion, transformation, and distribution; implement windowing, joins, and stateful processing. Implement distributed caching and in memory data grid integrations to reduce latency and improve throughput. Contribute to event gateway / event grid routing, schemas, topic design, ACLs, and dead letter strategies. Write clean, testable code for microservices (Java/Python), focusing on reliability, idempotency, and observability. Automate CI/CD pipelines, containerization (Docker), and deployments to Kubernetes. Participate in data governance: tagging, metadata updates, lineage capture, schema evolution, and data quality checks. Monitor production systems, perform performance tuning, troubleshoot backpressure/lag, and improve SLO attainment. Collaborate on design docs, code reviews, and cross team integrations. Required Qualifications: 7+ years in software engineering, with 3+ years focused on real time streaming or event driven systems. Strong hands-on experience with Kafka (topics, partitions, consumer groups), Schema Registry, Kafka Connect, and either Flink or Kafka Streams or Hazelcast. Solid understanding of ETL/ELT concepts, event time vs. processing time, checkpointing, state management, and exactly once/at least once semantics. Proficiency with microservices (Java /Python), APIs (REST/gRPC), Avro/JSON/protobuf, and contract testing. Experience with Docker, Kubernetes, and CI/CD tools (GitHub Actions/Azure DevOps/Jenkins or similar). Familiarity with distributed caching (Redis, Hazelcast) and in memory data grids. Cloud experience in at least one cloud platform (Azure/AWS/GCP). Knowledge of observability (metrics, logs, traces) and resilience (retries, timeouts, DLQs, circuit breakers). Exposure to data governance, metadata catalogs, and lineage tooling; schema evolution and compatibility (backward/forward/full). Core competencies to include Problem Solving, Ownership, Code Quality, Operational Mindset, Collaboration, Continuous Improvement. Nice to Have: CDC pipelines, Flink SQL, custom Kafka Connect connectors. Experience with feature flags, canary releases, and progressive delivery. Performance tuning: batch sizes, compression, serialization choices (Avro vs. Protobuf), and partitioning strategies. Best Regards, Vishal [email protected] www.smartcaliber.com Truth Lies in Heart Keywords: continuous integration continuous deployment Georgia Illinois Senior Data Engineer with strong Kafka streaming Exp - Atlanta, GA and Chicago, IL(Onsite) [email protected] http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=3000608&uid=f65fea946a5c40f6a4ff05a9ba2911c7 |
| [email protected] View All |
| 03:54 AM 17-Dec-25 |