Software Engineering Intern, Cloud Inference jobs in United States
cer-icon
Apply on Employer Site
company-logo

Modular · 11 hours ago

Software Engineering Intern, Cloud Inference

positionLos Altos, CA
timeInternship
remoteHybrid
seniorityIntern
money$47/hr - $57/hr

Modular is on a mission to revolutionize AI infrastructure by rebuilding the AI software stack. The Software Engineering Intern on the Cloud Inference team will contribute to building a platform for serving foundation models efficiently and effectively.

AI InfrastructureArtificial Intelligence (AI)Generative AIMachine LearningSoftware
check
H1B Sponsor Likelynote

Responsibilities

Contribute directly to the core components of Mammoth
Designing high-throughput, low-latency inference services, with features such as KV-aware routing and disaggregated inference
Developing distributed KV-cache manager, KV-cache offloading, and other optimizations needed to improve cache utilization
Solving challenges in running large frontier models (e.g., DeepSeek R1) across multiple nodes
Extending Kubernetes APIs and building controllers to support multi-model, multi-node, and multi-cluster deployments

Qualification

Distributed systemsCloud infrastructureMachine learning systemsProgramming skillsKubernetesCuriosityProblem-solving

Required

Currently pursuing a Bachelor's or Master's degree in Computer Science, Software Engineering, Mathematics, or related field
Strong programming skills in any programming language
Interest in distributed systems, cloud infrastructure, or machine learning systems
Curiosity, problem-solving mindset, and ability to learn quickly in a fast-moving environment

Preferred

Familiarity with Kubernetes and cloud-native technologies
Strong programming skills in Go
Experience building efficient, scalable distributed systems
Understanding of LLMs and common serving optimizations

Benefits

Competitive Compensation.
Team Building Events.

Company

Modular

twittertwittertwitter
company-logo
Modular provides AI infrastructure for deployment, serving, and programming GPUs.
dateFounded in 2022
location
Palo Alto, California, USA
people-size51-200 employees
web-link
https://www.modular.com

H1B Sponsorship

Modular has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (10)
2024 (6)
2023 (8)
2022 (4)

Funding

Current Stage
Growth Stage
Total Funding
$380M
Key Investors
US Innovative Technology FundGeneral CatalystGoogle Ventures
2025-09-24Series C· $250M
2023-08-24Series B· $100M
2022-06-30Seed· $30M

Leadership Team

leader-logo
Chris Lattner
CEO + Co-Founder
linkedin
leader-logo
Tim Davis
Co-Founder & President
linkedin

Recent News

General Catalyst
Our Investment in Modular
2026-01-14
General Catalyst
General Catalyst Portfolio
2026-01-14
Greylock
Greylock Portfolio - Modular
2025-12-29
Company data provided by crunchbase