Senior Kafka Platform Engineer (Automation & Kubernetes) jobs in United States
cer-icon
Apply on Employer Site
company-logo

Balyasny Asset Management L.P. ยท 3 days ago

Senior Kafka Platform Engineer (Automation & Kubernetes)

positionNew York, NY
timeFull-time
remoteOnsite
senioritySenior Level

Balyasny Asset Management L.P. is seeking a seasoned Kafka engineer to design, operate, and scale their event streaming platform. The role involves owning the Kafka core and automation, ensuring reliability and performance, and partnering with application teams on best practices.

Responsibilities

Architect, deploy, and operate production-grade Kafka clusters (self-managed and/or Confluent/MSK), including upgrades, capacity planning, multi-AZ/region DR, and performance tuning
Operate Kafka on Kubernetes using Operators, Helm, and GitOps, and build IaC-driven automation with guardrails for repeatable, compliant, zero-downtime provisioning and deployments
Implement and manage Kafka Connect, Schema Registry, and MirrorMaker 2/Cluster Linking; standardize connectors (e.g., Debezium) and build self-service patterns
Drive reliability: define SLOs/error budgets, on-call rotations, incident response, postmortems, runbooks, and automated remediation
Implement observability: metrics, logs, traces, lag monitoring, and capacity dashboards (e.g., Prometheus/Grafana, Burrow, Cruise Control, OpenTelemetry)
Secure the platform: TLS/mTLS, SASL (OAuth/SCRAM), RBAC/ACLs, secrets management, network policies, audit, and compliance automation
Guide event-streaming best practices: topic design, partitioning, compaction/retention, idempotency, ordering, schema evolution/compatibility, DLQs, EOS semantics
Partner with app, data, and SRE teams; provide enablement, documentation, and internal tooling for a great developer experience
Lead/mentor engineers and contribute to roadmap, standards, and platform strategy

Qualification

KafkaKubernetesInfrastructure as CodeSecurity practicesCloud experienceObservabilityPythonGoJavaBashLinux fundamentalsCommunication skillsMentoring

Required

Excellent communication and partnership skills with platform and application teams
Deep hands-on experience operating Kafka in production at scale (brokers, controllers, partitions, ISR, tiered storage/retention, rebalancing, replication, recovery)
Strong Kubernetes expertise running stateful systems
Automation first: Infrastructure as Code (Terraform), Helm, Operators, GitOps (Argo CD/Flux), and CI/CD (e.g., GitHub Actions/Jenkins) for platform lifecycle
Proficiency with one or more languages for tooling/automation: Python, Go, or Java; plus Bash and solid Linux fundamentals (networking, filesystems, JVM tuning basics)
Observability and reliability engineering for Kafka: Prometheus/Grafana, logging, alerting, lag monitoring, capacity/throughput modeling, performance tuning
Security for data in motion: TLS/mTLS, SASL/OAuth, ACL/RBAC, secrets management (e.g., Vault), and audit/compliance practices
Experience with Kafka ecosystem components: Kafka Connect, Schema Registry, MirrorMaker 2/Cluster Linking; familiarity with Cruise Control
Cloud experience (AWS/Azure/GCP) with networking, IAM, and one or more managed offerings (e.g., Confluent Cloud or AWS MSK)
Proven track record designing runbooks, leading incidents/postmortems, and driving platform roadmaps

Preferred

Data processing frameworks (Kafka Streams, Flink, Spark Structured Streaming) and EOS semantics
Experience with Strimzi or Confluent for Kubernetes in production
Knowledge of CDC patterns and tools (e.g., Debezium) and database connectors at scale
Multi-region architectures, cluster linking strategies, and disaster recovery drills

Company

Balyasny Asset Management L.P.

twittertwitter
Glassdoor4.3
company-logo
Balyasny Asset Management (BAM) is a diversified global investment firm founded in 2001 by Dmitry Balyasny, Scott Schroeder, and Taylor O'Malley.
dateFounded in 2001
location
Chicago, Illinois, USA
people-size1001-5000 employees
web-link
https://www.bamfunds.com/

H1B Sponsorship

Balyasny Asset Management L.P. has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (107)
2024 (85)
2023 (39)
2022 (44)
2021 (28)
2020 (18)

Funding

Current Stage
Late Stage

Leadership Team

leader-logo
David Black
Managing Director Business Development
linkedin
J
Jeremy Brunelli
Data Science & AI | Managing Director, Head of Geospatial & Alt Data Research
linkedin

Recent News

Pulse 2.0
Diagonal Therapeutics: $125 Million Series B Raised To Advance DIAG723 Clustering Antibody Into the Clinic
2026-01-12
pharmaphorum
Parabilis scores $305m in bumper biofinancing week
2026-01-11
GlobeNewswire
Diagonal Therapeutics Announces Oversubscribed $125 Million Series B Financing to Advance Clinical Development of Disease-Modifying, Clustering Antibody Pipeline
2026-01-09
Company data provided by crunchbase