Google · 2 hours ago
Staff Software Engineer, On-Device Machine Learning Infrastructure
Sunnyvale, CA, USA
Full-time
Onsite
Lead/Staff
$197K/yr - $291K/yr
8+ years expGoogle is seeking a Staff Software Engineer to develop on-device machine learning infrastructure that enhances performance and scalability. The role involves designing, developing, and deploying software solutions while collaborating with various teams to optimize generative AI performance across different hardware platforms.
AppsArtificial Intelligence (AI)Cloud StorageSearch EngineSEO
Responsibilities
Create roadmaps for developer-facing Application Programming Interfaces (APIs), Software Development Kits (SDKs), and tools, ensuring they meet the evolving needs of Large Language Models (LLMs) workflows
Solve technically tests problems that exceed the scope of a generalist Software Engineers, specifically around optimizing Generative AI performance across heterogeneous hardware (CPUs, GPUs, and EdgeTPUs)
Guide the team in designing resilient and robust systems, proactively anticipating scaling bottlenecks or shifts in usage as LLMs become increasingly complex
Coordinate efforts across multiple groups, including Android ML, ML Compiler, and DeepMind, to co-design performance and evaluation workflows
Provide technical mentorship, and implement new practices that address team needs and increase the velocity of your teammates
Qualification
Required
Bachelor's degree or equivalent practical experience
8 years of experience in software development
5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture
5 years of experience with one or more of the following: Speech/audio (e.g., technology duplicating and responding to the human voice), reinforcement learning (e.g., sequential decision making), ML infrastructure, or specialization in another ML field
5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning)
Preferred
Master's degree or PhD in Engineering, Computer Science, or a related technical field
8 years of experience with data structures/algorithms
3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects
Knowledge of ML converters/compilers and runtimes, and hardware-accelerated ML inference techniques
Understanding of Generative AI model architectures and their optimization for on-device execution
Proven track record of leading and delivering ML projects focused on on-device deployment (Android, iOS, web browsers, or embedded devices)
Benefits
Bonus
Equity
Benefits
Company
Google specializes in internet-related services and products, including search, advertising, and software. It is a sub-organization of Alphabet.
10001+ employees
H1B Sponsorship
Google has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (8763)
2024 (8872)
2023 (9682)
2022 (11626)
2021 (9109)
2020 (9785)
Funding
Current Stage
Public CompanyTotal Funding
$26.1MKey Investors
Andy Bechtolsheim
2004-08-19IPO
1999-06-07Series Unknown· $25M
1998-11-01Angel· $1M
Leadership Team
Sundar Pichai
CEO
Thomas Kurian
CEO - Google Cloud
Recent News
Small Business Trends
2026-01-24
2026-01-24
Search Engine Journal
2026-01-24
Company data provided by crunchbase