Microsoft · 4 days ago
Technical Program Manager, Human Evaluation Operations
Mountain View, CA
Full-time
Onsite
Entry, Mid Level
$101K/yr - $199K/yr
2+ years expMicrosoft AI is building advanced AI systems and the Human Evaluation Operations team is essential for ensuring their models are safe and high-quality. They are seeking two Technical Program Managers to lead evaluation operations for model quality and safety, partnering with various teams to deliver high-quality human evaluations at scale.
Agentic AIApplication Performance ManagementArtificial Intelligence (AI)Business DevelopmentDevOpsInformation ServicesInformation TechnologyManagement Information SystemsNetwork SecuritySoftware
Responsibilities
Lead Human Evaluation Programs: Drive end‑to‑end human evaluation workflows supporting model quality, safety, and capability initiatives across MAI. Coordinate evaluation planning, task design alignment, and delivery with product squads, engineering, and research partners
Manage Evaluation Workforce & Readiness: Oversee the health, performance, and scalability of MAI’s human evaluation workforce—including onboarding, qualification, training, and continuous performance management—to ensure reliable, high‑quality evaluation signals
Operational Excellence & Quality Governance: Maintain high operational standards across human‑in‑the‑loop pipelines by monitoring quality signals, resolving issues, and guiding teams toward consistent, trustworthy evaluation outcomes
Cross‑Functional Program Leadership: Partner with product squads to scope evaluation needs, define instructions and scorecards, support experimentation, and ensure teams are equipped to use human evaluations effectively
Platform & Vendor Partnership Management: Represent MAI needs to platform and vendor partners, shaping their roadmaps and ensuring capacity, reliability, and compliance with MAI standards
Insights, Tooling, & Documentation: Provide evaluation insights to product teams, maintain essential documentation, and influence the evolution of internal tools, dashboards, and processes that enable scalable human evaluations
Specialized Evaluation Programs: Support domain‑specific or advanced evaluation initiatives (e.g., expert reviews, structured scoring programs) in collaboration with MAI stakeholders
Qualification
Required
Bachelor's Degree AND 2+ years experience in engineering, product/technical program management, data analysis, or product development OR equivalent experience
1+ year(s) of experience managing cross-functional and/or cross-team projects
Preferred
3+ years of technical program management, operations management, data operations, or equivalent experience
1+ year(s) of experience reading and/or writing code (e.g., sample documentation, product demos)
Experience working cross‑functionally with engineering, research, PM, vendors, and operations partners
Experience managing vendor relations or external workforce programs
Strong analytical skills and comfort working with dashboards, metrics, and evaluation data
Experience running human‑in‑the‑loop data pipelines (e.g., annotations, RLHF, safety evals, quality assurance, crowdsourcing)
Familiarity with LLM and AI model evaluation practices, data annotation platforms and systems
Ability to quickly understand product quality signals, debug task design issues, and iterate with engineering teams
Experience in operational excellence, process automation, or scaling manual workflows
Company
Microsoft
Microsoft is a software corporation that develops, manufactures, licenses, supports, and sells a range of software products and services.
10001+ employees
H1B Sponsorship
Microsoft has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (9192)
2024 (9343)
2023 (7677)
2022 (11403)
2021 (7210)
2020 (7852)
Funding
Current Stage
Public CompanyTotal Funding
$1MKey Investors
Technology Venture Investors
2022-12-09Post Ipo Equity
1986-03-13IPO
1981-09-01Series Unknown· $1M
Leadership Team
Satya Nadella
Chairman and CEO
Vukani Mngxati
Chief Executive Officer - Microsft South Africa
Recent News
2026-01-16
Morningstar.com
2026-01-16
Company data provided by crunchbase