MongoDB · 1 day ago
Senior Software Engineer, Inference Platform
Palo Alto
Full-time
Hybrid
Senior Level
$118K/yr - $231K/yr
5+ years expMongoDB is a leading database platform empowering innovation in the AI era. The Senior Engineer will build the next-generation inference platform for embedding models used in semantic search and AI-native experiences, collaborating with ML researchers to ensure high-scale, low-latency performance.
Cloud ComputingDatabaseOpen SourceSaaSSoftware
Responsibilities
Design and build components of a multi-tenant inference platform integrated directly with MongoDB Atlas, supporting semantic search and hybrid retrieval
Collaborate with AI engineers and researchers to productionize inference for embedding models and rerankers — enabling both batch and real-time use cases
Contribute to platform capabilities such as latency-aware routing, model versioning, health monitoring, and observability
Improve performance, autoscaling, GPU utilization, and resource efficiency in a cloud-native environment
Work across product, infrastructure, and ML teams to ensure the inference platform meets the scale, reliability, and latency demands of Atlas users
Gain hands-on experience with tools like vLLM and container orchestration with Kubernetes
Qualification
Required
5+ years of experience building backend or infrastructure systems at scale
Strong software engineering skills in languages such as Go, Rust, Python, or C++, with an emphasis on performance and reliability
Experienced in cloud-native architectures, distributed systems, and multi-tenant service design
Familiar with concepts in ML model serving and inference runtimes, even if not directly deploying models
Comfortable working across functional teams, including ML researchers, backend engineers, and platform teams
Motivated to work on systems integrated into MongoDB Atlas and used by thousands of developers
Preferred
Knowledge of vector search systems (e.g., Faiss, HNSW, ScaNN) is a plus
Experience integrating infrastructure with production ML workloads
Understanding of hybrid retrieval, prompt-driven systems, or retrieval-augmented generation (RAG)
Contributions to open-source infrastructure for ML serving or search
Benefits
Equity
Participation in the employee stock purchase program
Flexible paid time off
20 weeks fully-paid gender-neutral parental leave
Fertility and adoption assistance
401(k) plan
Mental health counseling
Access to transgender-inclusive health insurance coverage
Health benefits offerings
Company
MongoDB
MongoDB is a next-generation database that helps businesses transform their industries by harnessing the power of data.
5001-10000 employees
H1B Sponsorship
MongoDB has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (159)
2024 (150)
2023 (133)
2022 (79)
2021 (51)
2020 (30)
Funding
Current Stage
Public CompanyTotal Funding
$311MKey Investors
SalesforceEquityZenT. Rowe Price
2024-11-14Post Ipo Equity
2024-10-16Post Ipo Debt
2018-03-06Post Ipo Equity
Leadership Team
Jim Scharf
Chief Technology Officer
Ronnen Miller
SVP, Global Technical Services
Recent News
Investing.com
2026-01-06
2025-12-15
Company data provided by crunchbase