Backend LLM Engineer jobs in United States
cer-icon
Apply on Employer Site
company-logo

LiteLLM (YC W23) · 2 days ago

Backend LLM Engineer

positionSan Francisco, CA, US
timeFull-time
remoteOnsite
seniorityEntry Level
money$120K/yr - $180K/yr
date1+ years exp

LiteLLM is an open-source LLM Gateway trusted by companies like NASA and Adobe, and they are seeking a Backend LLM Engineer to ensure the unification of API formats for calling LLM APIs. The role involves writing transformations for API requests and maintaining unified APIs across various LLM models.

Artificial Intelligence (AI)Developer ToolsSaaS

Responsibilities

Ensuring LiteLLM unifies the format for calling LLM APIs in the broader OpenAI + Anthropic spec
Writing transformations to convert API requests from OpenAI/Anthropic spec to various LLM provider formats
Building provider-agnostic unification functionality (e.g. session management across non-openai models for /v1/responses API, etc.)
Adding support for Anthropic and Bedrock Anthropic 'thinking' parameter
Handling provider-specific quirks like OpenAI o-1 streaming limitations
Maintaining ‘excellent’ unified API’s across /v1/messages, /v1/responses, /chat/completions for OpenAI/Gemini/Anthropic models across Azure, OpenAI API, Bedrock Invoke, Bedrock Converse, Vertex AI, Google AI Studio
Implementing cost tracking and logging for Anthropic API

Qualification

PythonFastAPILLM APIsRedisPostgresUser engagementWork ethicTeam collaboration

Required

1-2 years of backend/full-stack experience with production systems
Passion for open source and user engagement
Experience working with the OpenAI api (understand the difference between /chat/completions and /responses, and can speak to API-specific nuances)
Strong work ethic and ability to thrive in small teams
Eagerness to talk to users and help solve real problems
Strong Python and async framework experience (e.g., aiohttp, FastAPI)
Familiarity with LLM APIs (OpenAI, Azure, Hugging Face)
Proven ability to optimize scalable, high-performance systems

Company

LiteLLM (YC W23)

twittertwittertwitter
company-logo
LiteLLM is an open-source LLM Gateway with 18K+ stars on GitHub and trusted by companies like Rocket Money, Samsara, Lemonade, and Adobe.
dateFounded in 2023
location
San Francisco, California, USA
people-size2-10 employees
web-link
https://berri.ai

Funding

Current Stage
Early Stage
Total Funding
$0.5M
Key Investors
VentureBridge ProgramY Combinator
2023-06-14Seed
2023-04-05Pre Seed· $0.5M

Leadership Team

leader-logo
Ishaan Jaffer
Co-Founder & CTO
linkedin

Recent News

archive.org
Swartz Center for Entrepreneurship
2025-04-15
Company data provided by crunchbase