CoreWeave · 1 day ago
Senior Software Engineer I, Inference
Sunnyvale, CA
Full-time
Hybrid
Mid, Senior Level
$139K/yr - $204K/yr
3+ years expCoreWeave is The Essential Cloud for AI™, providing technology and tools for innovators in the AI space. The Senior Software Engineer I, Inference will lead designs, improve engineering standards, and enhance the reliability of the Kubernetes-native inference platform.
AI InfrastructureArtificial Intelligence (AI)Cloud ComputingCloud InfrastructureInformation TechnologyMachine Learning
No H1B
U.S. Citizen Only
Responsibilities
Lead design reviews and drive architecture within the team; decompose multi-service work into clear milestones
Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release
Implement advanced optimizations (e.g., micro-batch schedulers, speculative decoding, KV-cache reuse) and quantify impact
Strengthen incident posture: capacity planning, autoscaling policy, graceful degradation, rollback/traffic-shift strategies
Mentor IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards
For IC4: own an area spanning multiple services and teams (e.g., request routing & adaptive scheduling, cost-per-token analytics, GPU resource isolation)
Qualification
Required
IC3: ~3–5 years; IC4: ~5–8 years industry experience building distributed systems or cloud services
Strong coding in Python or Go (C++ a plus) and deep familiarity with networked systems and performance
Hands-on experience with Kubernetes at production scale, CI/CD, and observability stacks (Prometheus, Grafana, OpenTelemetry)
Practical knowledge of inference internals: batching, caching, mixed precision (BF16/FP8), streaming token delivery
Proven track record improving tail latency (P95/P99) and service reliability through metrics-driven work
Preferred
Contributions to inference frameworks (vLLM, Triton, TensorRT-LLM, Ray Serve, TorchServe)
Experience with CUDA kernels, NCCL/SHARP, RDMA/NUMA, or GPU interconnect topologies
Leading multi-team initiatives or partnering with customers on mission-critical launches
Benefits
Medical, dental, and vision insurance - 100% paid for by CoreWeave
Company-paid Life Insurance
Voluntary supplemental life insurance
Short and long-term disability insurance
Flexible Spending Account
Health Savings Account
Tuition Reimbursement
Ability to Participate in Employee Stock Purchase Program (ESPP)
Mental Wellness Benefits through Spring Health
Family-Forming support provided by Carrot
Paid Parental Leave
Flexible, full-service childcare support with Kinside
401(k) with a generous employer match
Flexible PTO
Catered lunch each day in our office and data center locations
A casual work environment
A work culture focused on innovative disruption
Company
CoreWeave
CoreWeave is a cloud-based AI infrastructure company offering GPU cloud services to simplify AI and machine learning workloads.
1001-5000 employees
Funding
Current Stage
Public CompanyTotal Funding
$24.87BKey Investors
Jane Street CapitalStack CapitalCoatue
2025-12-08Post Ipo Debt· $2.54B
2025-11-12Post Ipo Debt· $2.5B
2025-08-20Post Ipo Secondary
Leadership Team
Michael Intrator
Chief Executive Officer
Brannin McBee
Founder & CDO
Recent News
2026-01-24
The Motley Fool
2026-01-24
Company data provided by crunchbase