Prolific · 16 hours ago
Data Quality Engineer, AI Business
United States
Full-time
Remote
Director/Executive
8+ years expProlific is redefining AI innovation by providing high-quality, diverse data for training AI models. As the Head of Quality for the AI Business, you will set global quality standards, lead a distributed quality organization, and ensure that quality is embedded throughout the project lifecycle.
Artificial Intelligence (AI)CrowdsourcingData Collection and LabelingMachine LearningMarket ResearchProduct ResearchPsychology
Responsibilities
Set the global quality strategy across annotation, human feedback, model evaluation, red-teaming, and multilingual data
Define and enforce quality standards, acceptance criteria, and playbooks across project and pipeline types
Own the quality measurement system, including sampling strategies, agreement metrics, defect taxonomies, and drift detection
Lead a globally distributed quality organisation and design operating rhythms that work asynchronously across regions
Ensure quality is embedded from project kickoff through delivery, including audits, remediation loops, and root-cause analysis
Act as the senior quality partner for customers, clearly explaining methodology, results, risks, and improvement plans
Partner with Product, Engineering, and Operations to embed quality into tooling, workflows, and automation
Hire, develop, and lead quality managers and senior individual contributors, building a culture of rigor and accountability
Qualification
Required
8–12+ years of experience in quality, data operations, trust and safety, or ML evaluation, with 3+ years leading managers or senior leads
Proven experience running quality programs across globally distributed teams and multiple regions
Strong statistical intuition (sampling, agreement, bias/variance) and comfort working with quality data
Fluency with SQL and dashboards; working knowledge of Python or R for analysis or automation
Deep understanding of human-in-the-loop systems and how data quality impacts model behaviour
Strong judgment balancing speed, cost, and quality, and the ability to make trade-offs explicit
Clear, confident communication with both engineers and executives
Preferred
Experience with multilingual or localisation-style QA
Background designing taxonomies or evaluation frameworks tied to model performance
Exposure to regulated or high-scrutiny environments where auditability matters
Benefits
Competitive salary
Benefits
Remote working
Base salary
Equity
Cash variable element, such as a bonus or commission
Company
Prolific
Building the most advanced global infrastructure for People Science.
51-200 employees
H1B Sponsorship
Prolific has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (12)
2024 (7)
2023 (16)
2022 (39)
2021 (16)
2020 (8)
Funding
Current Stage
Growth StageTotal Funding
$33.54M2023-07-11Series A· $32.34M
2019-12-04Seed· $1.2M
Leadership Team
Phelim Bradley
CEO
Recent News
BusinessCloud
2025-10-19
2025-09-12
Company data provided by crunchbase