Microsoft · 7 hours ago
Senior AI Software Architect
Redmond, WA
Full-time
Onsite
Mid, Senior Level
$120K/yr - $235K/yr
4+ years expMicrosoft is a leading technology company committed to empowering every person and organization on the planet. They are seeking a Senior AI Software Architect to focus on model enablement and performance optimization for Maia accelerators, collaborating closely with hardware and software teams to ensure efficient model execution.
Application Performance ManagementArtificial Intelligence (AI)Business DevelopmentData ManagementDevOpsInformation ServicesInformation TechnologyManagement Information SystemsNetwork SecuritySoftware
Responsibilities
Port and optimize large-scale AI models (e.g., foundation models, diffusion models, YOLO) to run efficiently on Maia hardware
Integrate models using frameworks such as PyTorch, ONNX, vLLM, and SGLang
Apply techniques like KV cache quantization (e.g., BF16 → FP8), checkpointing, and re-sharding for efficient inference and training
Experiment with parallelism strategies (TP, PP) and analyze performance impacts across interconnects (NVLink vs PCIe)
Collaborate on improving inference pipelines, including KV caching in sglang/vllm and performance tuning at the PyTorch level
Work with Triton kernels for basic operations (e.g., FP8 dequantization) and assist in kernel performance analysis
Partner with hardware architects and kernel developers for co-design discussions
Communicate effectively with multiple stakeholders to align on performance goals and deliverables
Qualification
Required
Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter
Preferred
Bachelor's Degree in Computer Science or Engineering
3+ years of strong hands-on experience with PyTorch and model optimization techniques
Practical knowledge of quantization techniques like PTQ/QAT especially for KV cache quantization
Familiarity with parallelization strategies and distributed training concepts (e.g., sharding, allreduce)
2+ years of experience with AI inference stacks like SGLang/vLLM and performance profiling
Excellent problem-solving and communication skills; ability to work in a collaborative team environment
3+ years of experience in Triton kernels and CUDA programming (basic understanding is acceptable but willingness to learn is essential)
Experience with AI accelerator hardware and embedded systems
3+ years of prior work on efficient model checkpointing, resharding scripts, and large-scale model deployments for serving at scale
Company
Microsoft
Microsoft is a software corporation that develops, manufactures, licenses, supports, and sells a range of software products and services.
10001+ employees
H1B Sponsorship
Microsoft has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (7425)
2024 (9343)
2023 (7677)
2022 (11403)
2021 (7210)
2020 (7852)
Funding
Current Stage
Public CompanyTotal Funding
$1MKey Investors
Technology Venture Investors
2022-12-09Post Ipo Equity
1986-03-13IPO
1981-09-01Series Unknown· $1M
Leadership Team
Satya Nadella
Chairman and CEO
Vukani Mngxati
Chief Executive Officer - Microsft South Africa
Recent News
2025-12-31
Digital Trends
2025-12-31
Company data provided by crunchbase