NVIDIA · 1 week ago
Engineering Manager, Deep Learning Inference
Washington, United States
Full-time
Onsite
Senior Level
$224K/yr - $356K/yr
6+ years expNVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. The role involves shaping the software powering sophisticated AI systems and overseeing performance tuning and optimization of large-scale models for various AI applications.
AI InfrastructureArtificial Intelligence (AI)Consumer ElectronicsFoundational AIGPUHardwareSoftwareVirtual Reality
Hiring Manager
Bushra S.
Responsibilities
Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software
Drive the strategy, roadmap, and execution of NVIDIA’s inference frameworks engineering, focusing on SGLang
Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators
Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications
Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM)
Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA’s broader AI and software strategies
Foster a culture of technical excellence, open collaboration, and continuous innovation
Qualification
Required
MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field
6+ years of software development experience, including 3+ years in technical leadership or engineering management
Strong background in C/C++ software design and development; proficiency in Python is a plus
Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization
Proven record of deploying or optimizing deep learning models in production environments
Experience leading teams using Agile or collaborative software development practices
Preferred
Significant open-source contributions to deep learning or inference frameworks such as PyTorch, vLLM, SGLang, Triton, or TensorRT-LLM
Deep understanding of multi-GPU communications (NIXL, NCCL, NVSHMEM) and distributed inference architectures
Expertise in performance modeling, profiling, and system-level optimization across CPU and GPU platforms
Proven ability to mentor engineers, guide architectural decisions, and deliver complex projects with measurable impact
Publications, patents, or talks on LLM serving, model optimization, or GPU performance engineering
Benefits
Equity and benefits
Company
NVIDIA
NVIDIA is a computing platform company operating at the intersection of graphics, HPC, and AI.
10001+ employees
H1B Sponsorship
NVIDIA has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1877)
2024 (1355)
2023 (976)
2022 (835)
2021 (601)
2020 (529)
Funding
Current Stage
Public CompanyTotal Funding
$4.09BKey Investors
ARPA-EARK Investment ManagementSoftBank Vision Fund
2023-05-09Grant· $5M
2022-08-09Post Ipo Equity· $65M
2021-02-18Post Ipo Equity
Leadership Team
Jensen Huang
Founder and CEO
Michael Kagan
Chief Technology Officer
Recent News
2026-01-17
The Orlando Sentinel
2026-01-17
2026-01-17
Company data provided by crunchbase