Gruve · 15 hours ago
Senior Engineering Leader - AI Infrastructure and Inferencing
Redwood City, CA
Full-time
Onsite
Senior Level, Lead/Staff
$240K/yr - $250K/yr
10+ years expGruve is an innovative software services startup dedicated to transforming enterprises to AI powerhouses. They are seeking a Senior Engineering Leader to build and lead a high-performing engineering team focused on AI inference software development and product development.
Artificial Intelligence (AI)Machine LearningSoftware
Responsibilities
Build, mentor, and scale a world-class engineering team of 10-15+ engineers
Foster a culture of technical excellence, collaboration, and continuous learning
Conduct performance reviews, career development planning, and succession planning
Define and execute the technical roadmap for AI inference infrastructure, AI toolchains, and AI software development
Make critical architectural decisions that balance performance, scalability, maintainability, and cost
Lead the development of AI inference systems and optimizations for AI workloads, including graph optimization, kernel fusion, and hardware-specific code generation to maximize inference performance
Oversee the end-to-end lifecycle of AI models from development through production deployment, including model fine-tuning, quantization, distillation, and serving infrastructure
Drive the design and implementation of scalable, low-latency inference APIs and platforms that serve models reliably at production scale with strict SLA requirements
Champion rigorous engineering practices including comprehensive technical specifications, design reviews, and documentation to ensure alignment and quality across complex projects
Partner effectively with research, product, and business stakeholders to translate requirements into technical solutions and communicate progress, trade-offs, and risks clearly
Own quarterly planning, roadmap prioritization, and on-time delivery of major initiatives
Establish metrics and KPIs to measure team performance and system health
Qualification
Required
10-15+ years of software engineering experience with at least 5+ years in engineering leadership roles managing teams of 5+ engineers
Proven track record of building and scaling high-performing engineering teams in high-growth technology companies
Deep expertise in systems programming languages (C++, Go, Rust, or similar) and architecture design
Strong background in AI model design, optimization, or adjacent systems-level programming (LLVM, MLIR, XLA, or similar frameworks)
Hands-on experience with AI/ML model development, training, and inference systems
Experience with model fine-tuning techniques and deployment optimization (quantization, pruning, etc.)
Demonstrated ability to design and build production-grade APIs and distributed systems
Strong understanding of spec-driven development processes and engineering best practices
Excellent communication skills with ability to influence across all levels of the organization
Bachelor's degree in Computer Science, Engineering, or related technical field (or equivalent practical experience)
Preferred
Master's or PhD in Computer Science, Machine Learning, or related field
Experience at leading AI/ML companies or research labs (OpenAI, Google DeepMind, Meta AI, Anthropic, etc.)
Direct experience with modern ML frameworks (PyTorch, JAX, TensorFlow) and their compilation stacks
Background in GPU programming (CUDA, Triton) and hardware acceleration for ML workloads
Experience with transformer architectures and large language model (LLM) inference optimization
Track record of shipping production ML systems serving millions of requests per day
Contributions to open-source compiler or ML infrastructure projects
Experience with cloud infrastructure (AWS, GCP, Azure) and containerization/orchestration (Kubernetes, Docker)
Benefits
Benefits
Company
Gruve
Gruve is a startup focused on transforming AI strategies into tangible outcomes for enterprises.
501-1000 employees
H1B Sponsorship
Gruve has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (5)
Funding
Current Stage
Late StageTotal Funding
$37.5MKey Investors
Mayfield Fund
2025-04-30Series A· $20M
2025-04-30Seed· $17.5M
Recent News
2025-05-23
Company data provided by crunchbase