AI Infrastructure Engineer, Model Serving Platform jobs in United States
cer-icon
Apply on Employer Site
company-logo

Scale AI · 4 hours ago

AI Infrastructure Engineer, Model Serving Platform

positionSan Francisco Bay Area
timeFull-time
remoteOnsite
seniorityMid, Senior Level
money$179K/yr - $224K/yr
date4+ years exp

Scale AI is focused on developing reliable AI systems for critical decisions. The AI Infrastructure Engineer will design and build platforms for scalable serving of LLMs, collaborating with teams to optimize models and ensure system performance.

Artificial Intelligence (AI)Data Collection and LabelingGenerative AIImage RecognitionMachine Learning
check
H1B Sponsor Likelynote

Responsibilities

Build and maintain fault-tolerant, high-performance systems for serving LLMs workloads at scale
Build an internal platform to empower LLM capability discovery
Collaborate with researchers and engineers to integrate and optimize models for production and research use cases
Conduct architecture and design reviews to uphold best practices in system design and scalability
Develop monitoring and observability solutions to ensure system health and performance
Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment

Qualification

LLM serving experienceBackend system designProgramming skillsContainersOrchestrationCloud infrastructureMonitoring solutionsProblem-solvingCollaboration

Required

4+ years of experience building large-scale, high-performance backend systems
Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++)
Experience with LLM serving and routing fundamentals (e.g. rate limiting, token streaming, load balancing, budgets, etc.)
Experience with LLM capabilities and concepts such as reasoning, tool calling, prompt templates, etc
Experience with containers and orchestration tools (e.g., Docker, Kubernetes)
Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code (e.g., Terraform)
Proven ability to solve complex problems and work independently in fast-moving environments

Preferred

Experience with modern LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM, or text-generation-inference

Benefits

Comprehensive health, dental and vision coverage
Retirement benefits
A learning and development stipend
Generous PTO
A commuter stipend

Company

Scale AI

twittertwittertwitter
company-logo
Scale’s mission is to develop reliable AI systems for the world’s most important decisions.
dateFounded in 2016
location
San Francisco, California, USA
people-size501-1000 employees
web-link
https://scale.com

H1B Sponsorship

Scale AI has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (82)
2024 (54)
2023 (29)
2022 (17)
2021 (10)
2020 (10)

Funding

Current Stage
Late Stage
Total Funding
$15.9B
Key Investors
MetaAccelTiger Global Management
2025-06-10Corporate Round· $14.3B
2025-06-04Series Unknown
2024-05-21Series F· $1B

Leadership Team

leader-logo
Jason Droege
Interim Chief Executive Officer
linkedin
leader-logo
Dennis Cinelli
Chief Financial Officer
linkedin

Recent News

Globe and Mail
Meta to buy Chinese-founded startup Manus as tech giant aims to boost advanced AI
2025-12-31
Investing.com
Meta acquires Manus to expand AI agent capabilities
2025-12-30
36kr.com
Surpassing Zuckerberg, a 22-year-old becomes a billionaire. In 2025, the wealth creation speed of AI refreshes human cognition.
2025-12-29
Company data provided by crunchbase