Scale AI · 2 weeks ago
Head of Evaluation and Oversight Research
Seattle, WA
Full-time
Onsite
Director/Executive
$252K/yr - $315K/yrScale AI is a leading data and evaluation partner for frontier AI companies, focusing on advancing the science of evaluating large language models. The Head of Evaluation and Oversight Research will lead a team in developing frameworks and benchmarks for AI models, collaborating with industry and academia, and publishing impactful research.
AI InfrastructureArtificial Intelligence (AI)Data Collection and LabelingGenerative AIImage RecognitionMachine Learning
Responsibilities
Lead a team of research scientists and engineers on foundational work in evaluation and oversight
Drive research initiatives on frameworks and benchmarks for frontier AI models, spanning reasoning, coding, multi-modal, and agentic behaviors
Design and advance scalable oversight methods, leveraging model-assisted evaluation, rubric-guided judgments, and recursive oversight
Collaborate with leading research labs across industry and academia
Publish research at top-tier venues and contribute to open-source benchmarking initiatives
Remain deeply engaged with the research community, both understanding trends and setting them
Qualification
Required
Track record of impactful research in machine learning, especially in generative AI, evaluation, or oversight
Significant experience leading ML research in academia or industry
Strong written and verbal communication skills for cross-functional collaboration
Experience building and mentoring teams of research scientists and engineers
Publications at major ML/AI conferences (e.g. NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR) and/or journals
Benefits
Comprehensive health, dental and vision coverage
Retirement benefits
A learning and development stipend
Generous PTO
Commuter stipend
Company
Scale AI
Scale’s mission is to develop reliable AI systems for the world’s most important decisions.
Founded in 2016
501-1000 employees
H1B Sponsorship
Scale AI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (82)
2024 (54)
2023 (29)
2022 (17)
2021 (10)
2020 (10)
Funding
Current Stage
Late StageTotal Funding
$15.9BKey Investors
MetaAccelTiger Global Management
2025-06-10Corporate Round· $14.3B
2025-06-04Series Unknown
2024-05-21Series F· $1B
Leadership Team
Jason Droege
Interim Chief Executive Officer
Dennis Cinelli
Chief Financial Officer
Recent News
CB Insights
2026-01-09
Crunchbase News
2026-01-07
Company data provided by crunchbase