Anthropic · 1 week ago
Technical Program Manager, Model Evaluations
Seattle, WA
Full-time
Hybrid
Senior Level
$290K/yr - $365K/yr
5+ years expAnthropic is a public benefit corporation focused on creating reliable and beneficial AI systems. They are seeking a Technical Program Manager for model evaluations to coordinate the evaluation ecosystem, ensuring models meet performance and safety standards while collaborating across various teams.
Artificial Intelligence (AI)Foundational AIGenerative AIInformation TechnologyMachine Learning
Responsibilities
Standardize how evaluation results are generated, documented, compared across model versions, and communicated to stakeholders
Own end-to-end eval readiness for model launches—tracking which evals are ready, which need scores on past models, and which meet the bar for marketing materials
Establish and enforce clear criteria for eval inclusion: scores on historical models, state-of-the-art performance, and competitor comparisons
Coordinate between research teams, marketing, and product to consolidate eval status into a single source of truth
Maintain a high bar: ensure reported statistics reflect model capabilities in an honest, accurate, and transparent way
Get involved early in model development cycles, helping shape eval plans for RL environments
Partner with research and infrastructure teams to improve underlying evals infrastructure—eval-syncer reliability, results storage and querying, automation capabilities
Drive prioritization of eval tooling enhancements based on researcher needs
Identify patterns across launches and drive systemic fixes rather than point solutions
Work with PMs and researchers to improve and implement high priority evals launches
Maintain and prioritize the eval roadmap—working with cross-functional teams to identify which new evals are needed for upcoming launches and product requirements
Implement an operating model that reflects an evals environment with increasing complexity
Build lightweight but rigorous tracking systems—moving key information into structured formats that enable better decision-making
Create eval dashboards that provide real-time visibility into training progress on hero evals, enabling earlier intervention when scores look concerning
Document eval processes, requirements, and lessons learned to build institutional knowledge
Coordinate compute allocation for large-scale evals with infrastructure teams
Qualification
Required
5+ years of technical program management experience with a track record of bringing order to chaotic, high-stakes coordination problems
Possess scientific depth and a very high quality bar for data
Experience with ML/AI evaluation methodologies, benchmarking, or research quality assurance
Background in research operations, scientific publishing, or data quality management
Previous experience as data analyst, data scientist, or software engineer
Ability to build trust with research teams by understanding their work deeply enough to add value beyond coordination
Skilled at cross-functional coordination involving research, product, marketing, and engineering—navigating competing priorities and driving alignment
Working familiarity with data analysis tools (SQL, Python, or similar) for querying eval results and building dashboards
Familiarity with LLM capabilities and limitations and experience working with AI research teams
Excel at written and verbal communication, translating technical nuance for marketing stakeholders while maintaining precision
Thrive in unstructured environments with a bias toward action and a knack for creating clarity in ambiguous situations
Extremely high ownership and attention to detail
At least a Bachelor's degree in a related field or equivalent experience
Benefits
Equity
Benefits
Incentive compensation
Optional equity donation matching
Generous vacation and parental leave
Flexible working hours
A lovely office space in which to collaborate with colleagues
Company
Anthropic
Anthropic is an AI research company that focuses on the safety and alignment of AI systems with human values.
501-1000 employees
H1B Sponsorship
Anthropic has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (105)
2024 (13)
2023 (3)
2022 (4)
2021 (1)
Funding
Current Stage
Late StageTotal Funding
$33.74BKey Investors
Lightspeed Venture PartnersGoogleAmazon
2025-09-02Series F· $13B
2025-05-16Debt Financing· $2.5B
2025-03-03Series E· $3.5B
Leadership Team
Dario Amodei
CEO & Co-Founder
Daniela Amodei
President and co-founder
Recent News
2026-01-11
Insurance giant Allianz signs Claude Code deal with Anthropic | CIO
2026-01-11
Company data provided by crunchbase