Anyscale · 1 day ago
Software Engineer - Model Serving Infrastructure
San Francisco, CA
Full-time
Onsite
Entry, Mid Level
$170K/yr - $237K/yr
2+ years expAnyscale is a company focused on democratizing distributed computing through its open-source project Ray, which enhances machine learning capabilities for developers. The role involves developing and maintaining systems for serving machine learning models in production, enhancing performance and autoscaling capabilities.
Artificial Intelligence (AI)Developer PlatformInformation TechnologyMachine LearningOpen Source
Responsibilities
Develop a highly available service for ML model serving
Enhance Ray Serve and our other libraries to simplify the development of next-generation ML applications in production
Improve our autoscaling capabilities to drive performance enhancements and cost savings
Optimize latency and throughput for both single- and multi-model serving scenarios
Qualification
Required
A solid background in algorithms, data structures, and system design
Experience working with modern machine learning tooling, including PyTorch, TensorFlow, and JAX
At least 2+ year of relevant work experience
Preferred
Experience in building and maintaining open-source projects
Experience in building and operating machine learning infrastructure in production
Experience in building highly available serving systems
Benefits
Stock Options
Healthcare plans, with premiums covered by Anyscale at 99%
401k Retirement Plan
Education & Wellbeing Stipend
Paid Parental Leave
Fertility Benefits
Flexible Time Off
Commute reimbursement
100% of in office meals covered
Company
Anyscale
Anyscale accelerates the development and productionization of any AI app, on any cloud, at any scale.
201-500 employees
H1B Sponsorship
Anyscale has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (33)
2024 (14)
2023 (10)
2022 (10)
2021 (4)
2020 (1)
Funding
Current Stage
Growth StageTotal Funding
$259MKey Investors
New Enterprise AssociatesAndreessen Horowitz
2022-08-23Series C· $99M
2021-12-07Series C· $100M
2020-10-21Series B· $40M
Leadership Team
Keerti Melkote
Chief Executive Officer
Ion Stoica
Co-Founder & Executive Chairman
Recent News
Foundation Capital
2026-01-02
2025-11-04
The New Stack
2025-10-23
Company data provided by crunchbase