CoreWeave · 1 day ago
Senior Software Engineer II, Inference
Bellevue, WA
Full-time
Hybrid
Senior Level
$165K/yr - $242K/yr
5+ years expCoreWeave is The Essential Cloud for AI™, providing technology and expertise to enable innovators to scale AI confidently. The Senior Software Engineer II, Inference will lead design reviews, drive architecture, and implement advanced optimizations for a Kubernetes-native inference platform while mentoring junior engineers and improving service reliability.
Artificial Intelligence (AI)Cloud ComputingCloud InfrastructureInformation TechnologyMachine Learning
No H1B
U.S. Citizen Only
Responsibilities
Lead design reviews and drive architecture within the team; decompose multi-service work into clear milestones
Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release
Implement advanced optimizations (e.g., micro-batch schedulers, speculative decoding, KV-cache reuse) and quantify impact
Strengthen incident posture: capacity planning, autoscaling policy, graceful degradation, rollback/traffic-shift strategies
Mentor IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards
Own an area spanning multiple services and teams (e.g., request routing & adaptive scheduling, cost-per-token analytics, GPU resource isolation)
Qualification
Required
~5–8 years industry experience building distributed systems or cloud services
Strong coding in Python or Go (C++ a plus) and deep familiarity with networked systems and performance
Hands-on experience with Kubernetes at production scale, CI/CD, and observability stacks (Prometheus, Grafana, OpenTelemetry)
Practical knowledge of inference internals: batching, caching, mixed precision (BF16/FP8), streaming token delivery
Proven track record improving tail latency (P95/P99) and service reliability through metrics-driven work
Lead design reviews and drive architecture within the team; decompose multi-service work into clear milestones
Define and own SLIs/SLOs; ensure post-incident actions land and reliability improves release-over-release
Implement advanced optimizations (e.g., micro-batch schedulers, speculative decoding, KV-cache reuse) and quantify impact
Strengthen incident posture: capacity planning, autoscaling policy, graceful degradation, rollback/traffic-shift strategies
Mentor IC1/IC2 engineers; review cross-team designs and elevate coding/testing standards
Own an area spanning multiple services and teams (e.g., request routing & adaptive scheduling, cost-per-token analytics, GPU resource isolation)
Preferred
Contributions to inference frameworks (vLLM, Triton, TensorRT-LLM, Ray Serve, TorchServe)
Experience with CUDA kernels, NCCL/SHARP, RDMA/NUMA, or GPU interconnect topologies
Leading multi-team initiatives or partnering with customers on mission-critical launches
Benefits
Medical, dental, and vision insurance - 100% paid for by CoreWeave
Company-paid Life Insurance
Voluntary supplemental life insurance
Short and long-term disability insurance
Flexible Spending Account
Health Savings Account
Tuition Reimbursement
Ability to Participate in Employee Stock Purchase Program (ESPP)
Mental Wellness Benefits through Spring Health
Family-Forming support provided by Carrot
Paid Parental Leave
Flexible, full-service childcare support with Kinside
401(k) with a generous employer match
Flexible PTO
Catered lunch each day in our office and data center locations
A casual work environment
A work culture focused on innovative disruption
Company
CoreWeave
CoreWeave is a cloud-based AI infrastructure company offering GPU cloud services to simplify AI and machine learning workloads.
1001-5000 employees
Funding
Current Stage
Public CompanyTotal Funding
$23.37BKey Investors
Jane Street CapitalStack CapitalCoatue
2025-12-08Post Ipo Debt· $2.54B
2025-11-12Post Ipo Debt· $1B
2025-08-20Post Ipo Secondary
Leadership Team
Michael Intrator
Chief Executive Officer
Nitin Agrawal
Chief Financial Officer
Recent News
TheStreet.com
2026-01-03
Company data provided by crunchbase