Amazon · 3 months ago
Software Development Engineer AI/ML, Inference Serving, AWS Neuron
Cupertino, California, USA
Full-time
Onsite
Senior Level
$151K/yr - $262K/yr
5+ years expAmazon, through its subsidiary Annapurna Labs, is seeking a Software Development Engineer for its AWS Neuron team, which focuses on developing infrastructure for machine learning models. The role involves leading the design of distributed ML serving systems, optimizing performance, and collaborating with cross-functional teams to enhance inference capabilities for generative AI applications.
Artificial Intelligence (AI)DeliveryE-CommerceFoundational AIRetail
Responsibilities
Architect and lead the design of distributed ML serving systems optimized for generative AI workloads
Drive technical excellence in performance optimization and system reliability across the Neuron ecosystem
Design and implement scalable solutions for both offline and online inference workloads
Lead integration efforts with frameworks such as vLLM, SGLang, Torch XLA, TensorRT, and Triton
Develop and optimize system components for tensor/data parallelism and disaggregated serving
Implement and optimize custom PyTorch operators and NKI kernels
Mentor team members and provide technical leadership across multiple work streams
Drive architectural decisions that impact the entire Neuron serving stack
Collaborate with customers, product owners, and engineering teams to define technical strategy
Author technical documentation, design proposals, and architectural guidelines
Qualification
Required
5+ years of programming using a modern programming language such as Java, C++, or C#, including object-oriented design experience
5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
5+ years of non-internship professional software development experience
Experience as a mentor, tech lead or leading an engineering team
Preferred
Master's degree in computer science or equivalent
Deep expertise in ML Frameworks/Libraries such as JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, TensorRT
Benefits
Equity
Sign-on payments
A full range of medical, financial, and/or other benefits
Company
Amazon
Amazon is a tech firm with a focus on e-commerce, cloud computing, digital streaming, and artificial intelligence.
10001+ employees
H1B Sponsorship
Amazon has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (22803)
2024 (21175)
2023 (19057)
2022 (24088)
2021 (12233)
2020 (14881)
Funding
Current Stage
Public CompanyTotal Funding
$8.11BKey Investors
AmazonKleiner Perkins
2023-01-03Post Ipo Debt· $8B
2001-07-24Post Ipo Equity· $100M
1997-05-15IPO
Leadership Team
Douglas J. Herrington
CEO, Worldwide Amazon Stores
Werner Vogels
VP & CTO
Recent News
The Motley Fool
2026-01-09
2026-01-09
2026-01-09
Company data provided by crunchbase