NVIDIA · 4 weeks ago
Senior System Software Engineer - Dynamo
United States
Full-time
Remote
Mid, Senior Level
$152K/yr - $242K/yr
3+ years expNVIDIA is a leading technology company known for its innovations in AI and deep learning. They are seeking a Senior System Software Engineer to develop open source software for AI model inference on GPUs, contributing to the Dynamo project and optimizing distributed inference components.
Artificial Intelligence (AI)SemiconductorConsumer GoodsHardwareSoftwareAppsAI InfrastructureConsumer ElectronicsFoundational AIGPUVirtual Reality
Responsibilities
Contribute to the development of disaggregated serving for Dynamo-supported inference engines (vLLM, SGLang, TRT-LLM) and expand to support multi-modal models for embedding disaggregation
Innovate in the management and transfer of large KV caches across heterogeneous memory and storage hierarchies, using the NVIDIA Optimized Transfer Library (NIXL) for low-latency, cost-effective data movement
Build new features to the Dynamo Rust Runtime Core Library and design, implement, and optimize distributed inference components in Rust and Python
Balance a variety of objectives: build robust, scalable, high performance software components to support our distributed inference workloads; work with team leads to prioritize features and capabilities; load-balance asynchronous requests across available resources; optimize prediction throughput under latency constraints; and integrate the latest open source technology
Qualification
Required
Masters or PhD or equivalent experience
3+ years in Computer Science, Computer Engineering, or related field
Ability to work in a fast-paced, agile team environment
Excellent Rust/Python/C++ programming and software design skills, including debugging, performance analysis, and test design
Experience with high scale distributed systems and ML systems
Preferred
Prior contributions to open-source AI inference frameworks (e.g., vLLM, TensorRT-LLM, SGLang)
Experience with GPU memory management, cache management, or high-performance networking
Understanding of LLM-specific inference challenges, such as context window scaling and multi-model agentic and reasoning workflows
Prior experience with disaggregated serving and multi modal models (Vision-Language models, Audio Language Models, Video Language Models)
Benefits
Equity
Benefits
Company
NVIDIA
NVIDIA is a computing platform company operating at the intersection of graphics, HPC, and AI.
10001+ employees
H1B Sponsorship
NVIDIA has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1877)
2024 (1355)
2023 (976)
2022 (835)
2021 (601)
2020 (529)
Funding
Current Stage
Public CompanyTotal Funding
$4.09BKey Investors
ARPA-EARK Investment ManagementSoftBank Vision Fund
2023-05-09Grant· $5M
2022-08-09Post Ipo Equity· $65M
2021-02-18Post Ipo Equity
Leadership Team
Jensen Huang
Founder and CEO
Michael Kagan
Chief Technology Officer
Recent News
2026-02-09
The Motley Fool
2026-02-09
Company data provided by crunchbase