Amazon Web Services (AWS) ยท 1 day ago
Software Development Engineer AI/ML, Inference Serving, AWS Neuron
Cupertino, CA
Full-time
Onsite
Senior Level
$151K/yr - $262K/yr
5+ years expAmazon Web Services (AWS) is seeking a Software Development Engineer to lead and architect next-generation model serving infrastructure for machine learning applications. This role involves designing distributed ML serving systems and collaborating with cross-functional teams to enhance performance and reliability.
ConsultingDevOpsInformation TechnologySoftwareWeb Development
Responsibilities
Architect and lead the design of distributed ML serving systems optimized for generative AI workloads
Drive technical excellence in performance optimization and system reliability across the Neuron ecosystem
Design and implement scalable solutions for both offline and online inference workloads
Lead integration efforts with frameworks such as vLLM, SGLang, Torch XLA, TensorRT, and Triton
Develop and optimize system components for tensor/data parallelism and disaggregated serving
Implement and optimize custom PyTorch operators and NKI kernels
Mentor team members and provide technical leadership across multiple work streams
Drive architectural decisions that impact the entire Neuron serving stack
Collaborate with customers, product owners, and engineering teams to define technical strategy
Author technical documentation, design proposals, and architectural guidelines
Leading design reviews and architectural discussions
Rapidly prototyping software to show customer value
Debugging complex performance issues across the stack
Mentoring junior engineers on system design and optimization
Collaborating with research teams on new ML serving capabilities
Driving technical decisions that shape the future of Neuron's inference stack
Qualification
Required
5+ years of programming using a modern programming language such as Java, C++, or C#, including object-oriented design experience
5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
5+ years of non-internship professional software development experience
Experience as a mentor, tech lead or leading an engineering team
Preferred
Master's degree in computer science or equivalent
Deep expertise in ML Frameworks/Libraries such as JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, TensorRT
Benefits
Equity
Sign-on payments
Full range of medical, financial, and/or other benefits
Company
Amazon Web Services (AWS)
Launched in 2006, Amazon Web Services (AWS) began exposing key infrastructure services to businesses in the form of web services -- now widely known as cloud computing.
10001+ employees
H1B Sponsorship
Amazon Web Services (AWS) has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (22803)
2024 (21175)
2023 (19057)
2022 (24088)
2021 (12233)
2020 (14881)
Funding
Current Stage
Late StageTotal Funding
unknownKey Investors
BIRD Foundation
2025-01-22Grant
Leadership Team
Matt Garman
Chief Executive Officer
Anand Desikan
CTO, CXO Advisor, and Enterprise Technologist
Recent News
2026-01-07
2026-01-07
2026-01-07
Company data provided by crunchbase