Microsoft · 9 hours ago
Senior AI Hardware Architect
United States, California, Mountain View
Full-time
Hybrid
Mid, Senior Level
$120K/yr - $235K/yr
5+ years expMicrosoft is seeking a Senior AI Hardware Architect to join their AI Systems Architecture group. This role involves leading performance analysis and optimization of AI accelerator platforms, collaborating across teams to enhance data correlation and performance insight.
Agentic AIApplication Performance ManagementArtificial Intelligence (AI)Business DevelopmentDevOpsInformation ServicesInformation TechnologyManagement Information SystemsNetwork SecuritySoftware
Responsibilities
Lead performance analysis, profiling, and benchmarking across GPU and in-house AI accelerator architectures, applying rigorous data and statistical analysis to identify complex performance bottlenecks, root causes, and optimization opportunities across hardware, software, and system layers
Run and analyze end-to-end AI models on production-like serving infrastructure, performing deep dives into modern AI serving stacks (e.g., optimized LLM serving frameworks, schedulers, runtimes, and memory management systems) to understand performance behavior, scalability limits, and system-level trade-offs
Provide data-driven recommendations and architectural trade-offs to senior technical leadership, balancing performance, complexity, cost, quality, reliability, and development timelines to inform accelerator and system architecture decisions
Develop and implement technical solutions to complex performance, quality, and design challenges, including kernel-level optimization, architectural tuning, and system-level performance improvements across multiple products or feature areas
Correlate on-silicon measurements, software traces, and kernel execution behavior with architectural models and simulators, ensuring alignment between measured performance and architectural intent, and identifying gaps that drive future design enhancements
Design, build, and evolve data correlation, analysis, and visualization tools and workflows that scale performance insight, accelerate debugging, and improve clarity and communication of optimization opportunities across teams
Lead and contribute to design and performance documentation, including architecture reviews, performance reports, functional specifications, and customized analyses; communicate progress, risks, and recommendations within and across teams, and help identify and mitigate significant project risks
Qualification
Required
Master's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 3+ years technical engineering experience OR Bachelor's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 5+ years technical engineering experience OR equivalent experience
Ability to meet Microsoft, customer, and/or government security screening requirements for this role. These requirements include, but are not limited to, the following specialized security screenings: Microsoft Cloud Background Check: This position requires passing the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Preferred
Doctorate in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 3+ years technical engineering experience OR Master's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 6+ years technical engineering experience OR Bachelor's Degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or related field AND 8+ years technical engineering experience OR equivalent experience
MS or PhD in Machine Learning, Computer Architecture/Systems, Electrical Engineering, High-Performance Computing, or related areas
4+ years of experience in Computer Architecture, AI Systems, or closely related technical domains
Experience with GPU and AI accelerator architectures, including compute pipelines, memory hierarchies, interconnects, and parallel execution models
Demonstrated expertise in performance profiling, benchmarking, and root-cause analysis, using hardware performance counters, software traces, and workload-level measurements
Hands-on experience with kernel-level performance analysis and optimization, and correlating kernel behavior with architectural and system-level performance
Strong programming and scripting skills in Python and C/C++ for performance analysis, tooling, benchmarking, and automation
Experience with architectural modeling or simulators and correlating modeled behavior with measured hardware performance
Experience running and analyzing end-to-end AI models on serving or training infrastructure, with the ability to diagnose performance issues across hardware, runtime, and system layers
Hands-on experience with AI frameworks and runtimes, including PyTorch, and familiarity with modern AI serving stacks such as vLLM and SGLang frameworks
Ability to communicate complex technical concepts clearly through design documentation, performance reports, functional specifications, and technical presentations
Benefits
Certain roles may be eligible for benefits and other compensation.
Company
Microsoft
Microsoft is a software corporation that develops, manufactures, licenses, supports, and sells a range of software products and services.
10001+ employees
H1B Sponsorship
Microsoft has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (9192)
2024 (9343)
2023 (7677)
2022 (11403)
2021 (7210)
2020 (7852)
Funding
Current Stage
Public CompanyTotal Funding
$1MKey Investors
Technology Venture Investors
2022-12-09Post Ipo Equity
1986-03-13IPO
1981-09-01Series Unknown· $1M
Leadership Team
Satya Nadella
Chairman and CEO
Vukani Mngxati
Chief Executive Officer - Microsft South Africa
Recent News
2026-01-16
Morningstar.com
2026-01-16
Company data provided by crunchbase