Apply on Employer Site

Acceler8 Talent · 3 days ago

Lead LLM Evals Engineer

San Francisco, CA

Full-time

Onsite

Senior Level, Lead/Staff

$250K/yr - $350K/yr

Acceler8 Talent is an early-stage physical AI startup focused on building systems with general physical ability. They are seeking a Lead LLM Evals Engineer to own the evaluation and verification layer for agentic LLM systems, building eval harnesses and automated verifiers to ensure agents can effectively plan and execute workflows.

Staffing & Recruiting

H1B Sponsor Likely

Hiring Manager

Nick Bell

Responsibilities

Build eval harnesses for agentic LLM systems in complex workflows

Design verifiers for planning, execution, recovery, and constraint adherence

Turn eval failures into training signals with research and systems teams

Qualification

LLM evaluationAutomated verificationComplex workflowsAgentic systemsTeam collaboration

Required

Experience in building eval harnesses for agentic LLM systems in complex workflows

Ability to design verifiers for planning, execution, recovery, and constraint adherence

Experience in turning eval failures into training signals with research and systems teams

Company

Acceler8 Talent

Acceler8 Talent partners with technology companies to propel their funding and business forward.

Founded in 2019

Boston, Massachusetts, USA

11-50 employees

http://www.acceler8talent.com/

H1B Sponsorship

Acceler8 Talent has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2025 (1)

2023 (1)

Funding

Current Stage

Early Stage

Leadership Team

Sam McBroom

Director

Company data provided by crunchbase