Dynamo AI · 6 months ago
ML Engineer — LLM Evaluation
San Francisco, CA
Full-time
Onsite
Mid LevelDynamo AI is a forward-thinking company focused on developing LLMs with an emphasis on safety, privacy, and responsibility. The ML Engineer role involves owning LLM evaluation processes, generating benchmarks, and delivering scalable production code to empower customers in deploying safe and responsible LLMs.
Artificial Intelligence (AI)Generative AIMachine Learning
Responsibilities
Own LLM evaluation processes and methods with a focus on generating benchmarks representative of real-world usage and safety vulnerabilities
Generate high quality synthetic data, curate labels, and conduct rigorous benchmarking
Deliver robust, scalable, and reproducible production code
Push the envelope by developing methods for benchmarking that revamps how we assess the best LLMs for harmlessness and helpfulness
Your research will directly empower our customers to more feasibly deploy safe and responsible LLMs
Co-author papers, patents, and presentations with our research team by integrating other members’ work with your vertical
Qualification
Required
Domain knowledge in LLM evaluation and data curation techniques
Extensive experience in designing and implementing LLM benchmarking, extending previous methods
Comfortability with leading end-to-end projects
Adaptability and flexibility. In both the academic and startup world, a new finding in the community may necessitate an abrupt shift in focus. You must be able to learn, implement, and extend state-of-the-art research
Preferred
Past research or projects in benchmarking LLMs
Company
Dynamo AI
The enterprise platform for enabling private, secure, and regulation-compliant Gen AI models.
Founded in 2021
51-200 employees
H1B Sponsorship
Dynamo AI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (2)
Funding
Current Stage
Growth StageTotal Funding
$19.25MKey Investors
Nexus Venture Partners
2023-08-16Series A· $15.1M
2022-09-20Seed· $4.15M
2022-05-01Pre Seed
Leadership Team
Christian Lau
Co-founder & Chief Product Officer; Ph.D. EECS @ MIT
Recent News
Silicon Prairie News
2025-09-27
Company data provided by crunchbase