Machine Learning Engineer - Inference jobs in United States
cer-icon
Apply on Employer Site
company-logo

Together AI · 1 day ago

Machine Learning Engineer - Inference

positionUnited States
timeFull-time
remoteRemote
seniorityMid Level
money$160K/yr - $230K/yr
date3+ years exp

Together AI is a research-driven artificial intelligence company focused on creating innovative AI solutions. They are seeking a Machine Learning Engineer to join their Inference Engine team, where the role involves optimizing AI inference systems and collaborating with researchers and engineers to develop high-performance applications.

Artificial Intelligence (AI)Generative AIInternetIT InfrastructureOpen Source
check
H1B Sponsor Likelynote

Responsibilities

Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale
Develop and optimize runtime inference services for large-scale AI applications
Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world
Conduct design and code reviews to ensure high standards of quality
Create services, tools, and developer documentation to support the inference engine
Implement robust and fault-tolerant systems for data ingestion and processing

Qualification

PythonPyTorchHigh-performance librariesOperating systems conceptsAI inference systemsSpeculative decodingCUDA/Triton programmingRustCythonCompilers

Required

3+ years of experience writing high-performance, well-tested, production-quality code
Proficiency with Python and PyTorch
Demonstrated experience in building high performance libraries and tooling
Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale

Preferred

Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum
Knowledge of AI inference techniques such as speculative decoding
Knowledge of CUDA/Triton programming
Knowledge of Rust, Cython and compilers

Benefits

Competitive compensation
Startup equity
Health insurance
Other competitive benefits

Company

Together AI

twittertwittertwitter
company-logo
Together AI is a cloud-based platform designed for constructing open-source generative AI and infrastructure for developing AI models.
dateFounded in 2022
location
San Francisco, California, USA
people-size201-500 employees
web-link
https://www.together.ai

H1B Sponsorship

Together AI has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (19)
2024 (6)
2023 (3)

Funding

Current Stage
Growth Stage
Total Funding
$533.5M
Key Investors
Salesforce VenturesLux Capital
2025-02-20Series B· $305M
2024-03-13Series A· $106M
2023-11-29Series A· $102.5M

Leadership Team

leader-logo
Vipul Ved Prakash
Co-Founder & CEO
linkedin
leader-logo
Kae Ike Lim
Executive Assistant to Co-Founder and CEO
linkedin

Recent News

Morningstar.com
AI21 Labs and Together AI Partner to Expand Access to Open-Source Models
2025-11-27
prnasia.com
PEGATRON Strengthens AI Infrastructure Collaboration with Together AI and 5C for NVIDIA GB300 NVL72 and NVIDIA HGX B200 Liquid-Cooled Rack Deployment in U.S. Data Centers
2025-11-19
KrASIA
Coding tools Cursor and Windsurf found using Chinese AI in latest releases
2025-11-07
Company data provided by crunchbase