Senior AI/ML Engineer — LLM & Agent Stack jobs in United States
cer-icon
Apply on Employer Site
company-logo

TrueFoundry · 4 months ago

Senior AI/ML Engineer — LLM & Agent Stack

positionSan Mateo, CA
timeFull-time
remoteOnsite
senioritySenior Level
date5+ years exp

TrueFoundry is an enterprise-grade AI/ML platform that accelerates the development and deployment of GenAI and ML applications. They are seeking a Senior AI/ML Engineer to design and own core components enabling enterprise customers to run production agentic AI safely and efficiently, focusing on orchestration, integrations, and observability.

Artificial Intelligence (AI)DevOpsMachine Learning
check
H1B Sponsor Likelynote

Responsibilities

Architect and implement scalable agent orchestration patterns (graph-based executors, state management, multi-agent coordination) for production workloads
Own critical integrations: model adapters, LLM gateway hooks, vector DBs, tools & external APIs, and the platform's LLMops flows
Build and improve tracing, benchmarking and observability for LLMs and agents — token/cost accounting, latency p95, throughput, and correctness checks
Drive design for safety/guardrails: moderation hooks, human-in-the-loop checkpoints, replayable audit trails and policy enforcement
Mentor junior engineers, run design reviews, and improve engineering practices (testing, CI/CD, chaos testing for agents)
Work directly with strategic customers to prototype complex agentic solutions and translate them into product features

Qualification

LLM integrationAgent orchestrationDistributed systemsKubernetesPolicy enforcementPerformance tuningObservabilityCustomer engagementOpen-source contributionsMentoring

Required

5–8 years of software engineering with substantial experience building distributed systems, infra, or ML platforms
Deep practical experience integrating and deploying LLMs in production (RAG, retrieval, embeddings pipelines)
Hands-on experience with agent orchestration frameworks (LangGraph / LangChain or custom agent runtimes) and stateful workflow design
Strong systems knowledge: Kubernetes, container orchestration, service meshes, and performance tuning
Proven track record building observability, cost controls, and policy enforcement for production services

Preferred

Experience building or contributing to open-source LLM orchestration tools (LangGraph, LangChain, or similar)
Familiarity with enterprise constraints: on-prem/cloud hybrid deployments, data residency, compliance requirements
Background in security, privacy, or model governance for LLMs
Demonstrated leadership in cross-functional projects and direct customer engagement

Company

TrueFoundry

twittertwittertwitter
company-logo
TrueFoundry is a unified platform with an enterprise-grade AI Gateway - combining LLM, MCP, and Agent Gateway.
dateFounded in 2021
location
San Francisco, California, USA
people-size11-50 employees
web-link
http://truefoundry.com

H1B Sponsorship

TrueFoundry has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (3)
2024 (1)
2021 (1)

Funding

Current Stage
Early Stage
Total Funding
$21.3M
Key Investors
Intel Capital
2025-02-06Series A· $19M
2022-09-19Seed· $2.3M

Leadership Team

leader-logo
Nikunj Bajaj
Co-Founder & CEO
linkedin
leader-logo
Abhishek Choudhary
Co-Founder
linkedin

Recent News

Business Wire
TrueFoundry Launches AI Gateway on Product Hunt: Unified Control Plane for Agentic AI
2025-12-10
Indianweb2.com
NetApp Excellerator Cohort-14 Demo Day Showcases India’s Deep-Tech Startups Driving AI and Cloud Innovation
2025-11-07
Metrovacesa lanza el agente virtual conversacional MiA basado en IA generativa | CIO
How to build scalable agentic AI applications for enterprises
2025-09-25
Company data provided by crunchbase