Upwork · 1 week ago
Sr Lead Machine Learning Engineer
United States
Full-time
Remote
Senior Level, Lead/Staff
$195K/yr - $308K/yrUpwork is the world’s human and AI-powered work marketplace connecting businesses with skilled independent talent. The Sr Lead Machine Learning Engineer will define success metrics for AI agents and develop evaluation frameworks to enhance human+AI collaboration.
FreelanceMarketplacePeer to PeerRecruiting
Responsibilities
Design and implement comprehensive evaluation frameworks that reflect real-world task success for agentic systems, with a focus on human+AI collaboration outcomes
Build benchmarking pipelines that capture nuanced success indicators including trust calibration, intervention frequency, and agent handoff quality
Lead development of observability tools and instrumentation for analyzing agent behavior in production
Translate complex qualitative and quantitative signals into actionable insights that inform model iteration and product prioritization
Collaborate with researchers, engineers, and product teams to align evaluation methodologies with business and user goals
Own benchmarking infrastructure that enables reproducible, scalable evaluation across AI initiatives
Champion rigorous experimental design and statistical analysis across teams to ensure consistent and meaningful measurement standards
Qualification
Required
Proven experience designing evaluation systems for agentic or LLM-based AI, ideally in complex, interactive or open-ended environments
Deep expertise in statistical experimentation, benchmark creation, and human-AI interaction assessment
Fluency in building data pipelines and tooling using Python, SQL, and distributed data processing frameworks
Demonstrated ability to influence product and model roadmaps through evaluation insights and performance measurement
Adaptive-level proficiency in integrating AI tools into technical workflows for analysis, experimentation, and observability refinement
Benefits
Comprehensive medical coverage for you and your family
Unlimited PTO
A 401(k) plan with matching
12 weeks of paid parental leave
An Employee Stock Purchase Plan
Company
Upwork
Upwork is an online marketplace that connects corporate businesses and organizations with field professionals and independent talents.
Founded in 2014
501-1000 employees
H1B Sponsorship
Upwork has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (16)
2024 (28)
2023 (23)
2022 (35)
2021 (29)
2020 (22)
Funding
Current Stage
Public CompanyTotal Funding
$168.8MKey Investors
BenchmarkDAG VenturesSigma Partners
2018-10-02IPO
2017-01-01Series Unknown
2016-10-27Secondary Market
Leadership Team
Hayden Brown
President and Chief Executive Officer
Erica Gessert
Chief Financial Officer
Recent News
2026-01-09
2025-12-11
Company data provided by crunchbase