Modular · 1 month ago
Senior AI Kernel Engineer
United States
Full-time
Remote
Senior Level
$198K/yr - $286K/yr
5+ years expModular is on a mission to revolutionize AI infrastructure by rebuilding the AI software stack. They are seeking a Senior AI Kernel Engineer to lead the design and optimization of high-performance kernels for AI inference on GPUs and custom accelerators, collaborating closely with various teams to enhance performance.
Artificial Intelligence (AI)SoftwareAI InfrastructureGenerative AIMachine Learning
Responsibilities
Design, implement, and optimize performance-critical kernels for AI inference workloads (e.g., GEMM, attention, communication, fusion)
Lead kernel-level optimization efforts across single-GPU, multi-GPU, and heterogeneous hardware environments
Make informed trade-offs between latency, throughput, memory footprint, and numerical precision
Drive adoption of new hardware features (e.g., Tensor Cores, asynchronous execution, advanced memory spaces)
Analyze performance using profilers, hardware counters, and microbenchmarks; translate insights into concrete improvements
Work closely with compiler and runtime teams to influence code generation, scheduling, and kernel fusion strategies
Review and mentor other engineers on kernel design, performance tuning, and best practices
Contribute to technical roadmaps and long-term performance strategy for AI inference
Qualification
Required
5+ years of experience in performance-critical systems or kernel development (or equivalent depth of expertise)
Strong proficiency in C/C++ and low-level programming
Extensive hands-on experience with GPU kernel programming (CUDA, HIP, or equivalent)
Deep understanding of GPU architecture, including memory hierarchies, synchronization, and execution models
Proven track record of delivering measurable performance improvements in production systems
Strong problem-solving skills and ability to work independently on complex, ambiguous performance challenges
Preferred
Experience with PTX, assembly-level tuning, or code generation frameworks (e.g., Triton)
Experience optimizing distributed or multi-GPU inference pipelines
Familiarity with custom AI accelerators or domain-specific hardware
Understanding of modern AI models (e.g., transformers, LLMs, diffusion) from a systems and performance perspective
Contributions to open-source kernel libraries, compilers, or performance tools
Experience collaborating directly with hardware or compiler teams
Benefits
Premier insurance plans
Up to 5% 401k matching
Flexible paid time off
Stock options
Annual target bonus
Equity
Team Building Events
Regular team onsites and local meetups in Los Altos, CA as well as different cities
Traveling 2-4 times a year is expected for all roles
Company
Modular
Modular is an AI infrastructure company that develops a unified software stack for deployment, serving, and programming GPUs.
51-200 employees
H1B Sponsorship
Modular has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
94%
Represents job field similar to this job
Engineering and Development
Management and Executive
Trends of Total Sponsorships
*2026 (6)
2025 (10)
2024 (6)
2023 (8)
2022 (4)
Funding
Current Stage
Growth StageTotal Funding
$380MKey Investors
US Innovative Technology FundGeneral CatalystGoogle Ventures
2026-06-24Acquired
2025-09-24Series C· $250M
2023-08-24Series B· $100M
Leadership Team
Chris Lattner
CEO + Co-Founder
Tim Davis
Co-Founder & President
Recent News
2026-07-07
San Diego Business Journal
2026-07-04
2026-07-01
Company data provided by crunchbase