Netflix · 8 hours ago
HDO Evaluation Program Manager
United States
Full-time
Remote
Mid, Senior Level
$230K/yr - $340K/yr
4+ years expNetflix is a global entertainment company seeking an experienced Human Evaluations Program Manager to support the evaluation and training of ML and Generative AI models. The role involves leading the execution of human evaluation initiatives, developing frameworks for AI evaluation, and collaborating with cross-functional teams to ensure high-quality evaluations aligned with Netflix's values.
Digital EntertainmentMedia and EntertainmentTVVideo Streaming
Responsibilities
Lead end-to-end execution of human evaluation and data operations initiatives—from intake and scoping to delivery
Develop and operationalize frameworks for evaluating GenAI and ML outputs
Collaborate across research, product, UX, and engineering to embed evaluation into model development cycles
Build and maintain project timelines, proactively manage blockers, and ensure timely execution
Develop clear, scalable guidelines and scoring rubrics to ensure consistent rater judgment
Oversee rater onboarding, calibration, and QA workflows
Define and monitor success metrics such as speed to IRR, throughput, and task effectiveness
Pilot and refine evaluation tasks to improve clarity, inter-rater reliability, and feedback quality
Build foundational documentation and drive adoption of best practices across teams
Track evaluation health and proactively communicate progress to stakeholders clearly and proactively
Anticipate and proactively resolve bottlenecks and blockers
Act as the connective tissue across multiple partners to ensure alignment and effective execution of evaluations at scale
Qualification
Required
4+ years of experience working in human evaluations, data collection, labeling, or annotation operations in GenAI/ML environments
Track record of implementing process improvements or quality control systems for data collection needs
Prior experience managing human annotation vendors, raters, or data labeling teams
Strong understanding of evaluation design, including guidelines, rubrics, and scoring protocols
Proven ability in end-to-end management of complex, cross-functional programs, demonstrating strong Program Management skills and clear accountability for successful delivery
Experience with human labeling platforms
Excellent written and verbal communication skills
Ability to synthesize feedback into clear recommendations and process improvements
Familiarity with responsible AI principles and how to embed them into evaluation design
Strong organizational skills and executional focus; ability to track details while seeing the bigger picture
Benefits
Health Plans
Mental Health support
A 401(k) Retirement Plan with employer match
Stock Option Program
Disability Programs
Health Savings and Flexible Spending Accounts
Family-forming benefits
Life and Serious Injury Benefits
Paid leave of absence programs
Full-time hourly employees accrue 35 days annually for paid time off to be used for vacation, holidays, and sick paid time off.
Full-time salaried employees are immediately entitled to flexible time off.
Company
Netflix
Netflix is an online streaming platform that enables users to watch TV shows and movies.
10001+ employees
H1B Sponsorship
Netflix has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (310)
2024 (309)
2023 (191)
2022 (261)
2021 (268)
2020 (225)
Funding
Current Stage
Public CompanyTotal Funding
$72.11BKey Investors
Wells FargoTCVGroupe Arnault
2025-12-05Post Ipo Debt· $67.2B
2024-08-01Post Ipo Debt· $1.8B
2018-05-05Post Ipo Debt· $2.67M
Leadership Team
Gregory Peters
Co-CEO
Ted Sarandos
Co-CEO
Recent News
2026-01-23
Company data provided by crunchbase