Data Quality Engineer, AI Business jobs in United States
cer-icon
Apply on Employer Site
company-logo

Prolific · 1 day ago

Data Quality Engineer, AI Business

positionUnited States
timeFull-time
remoteRemote
senioritySenior Level
date5+ years exp

Prolific is redefining AI innovation by providing high-quality, diverse data for training AI models. As a Data Quality Engineer, you will ensure data quality and integrity for managed service studies, collaborating with cross-functional teams to implement measurement systems and automate quality checks.

Artificial Intelligence (AI)CrowdsourcingData Collection and LabelingMachine LearningMarket ResearchProduct ResearchPsychology
check
H1B Sponsor Likelynote

Responsibilities

Own end-to-end quality design for Prolific managed service studies, including rubrics, acceptance criteria, defect taxonomies, severity models, and clear definitions of done
Define, implement, and maintain quality measurement systems, including sampling plans, golden sets, calibration protocols, agreement targets, adjudication workflows, and drift detection
Build and deploy automated quality checks and launch gates using Python and SQL, such as schema and format validation, completeness checks, anomaly detection, consistency testing, and label distribution monitoring
Design and run launch readiness processes, including pre-launch checks, pilot calibration, ramp criteria, full-launch thresholds, and pause/rollback mechanisms
Partner with Product and Engineering to embed in-study quality controls and authenticity checks into workflows, tooling, and escalation paths
Write and continuously improve guidelines and training materials to keep participants, reviewers, and internal teams aligned on evolving quality standards
Investigate quality and integrity issues end to end, running root-cause analysis across guidelines, UX, screening, training, and operations, and driving corrective and preventive actions (CAPAs)
Build dashboards and operating cadences to track defect rates, rework, throughput versus quality trade-offs, integrity events, and SLA adherence
Lead calibration sessions and coach QA leads and reviewers to improve decision consistency, rubric application, and overall quality judgement
Translate one-off quality fixes into repeatable, scalable playbooks across customers, programs, and study types

Qualification

Quality engineeringPythonSQLStatistical conceptsAutomationCustomer empathyRoot-cause analysisData collection mechanicsIntegrity detection systemsCross-functional collaboration

Required

5+ years of experience in quality engineering, data or annotation quality, analytics engineering, trust and integrity, or ML/LLM evaluation operations
Strong proficiency in Python and SQL, with comfort applying statistical concepts such as sampling strategies, confidence levels, and agreement metrics
A proven track record of turning ambiguous or messy quality problems into clear metrics, automated checks, and durable process improvements
Strong quality systems thinking, with the ability to translate complex edge cases into clear rules, tests, rubrics, and governance mechanisms
Hands-on experience instrumenting workflows and implementing pragmatic automation that catches quality and integrity issues early
Demonstrated ability to influence cross-functional teams (Product, Engineering, Operations, Client teams) and drive change without direct authority
Strong customer empathy, with a clear understanding of what 'useful, trustworthy data' means for research, AI training, and evaluation use cases

Preferred

Familiarity with data collection mechanics (screeners, quota/routing constraints, study design patterns)
LLM evals, red teaming, or policy-based annotation experience
Data/versioning discipline (dataset lineage, change control, reproducibility)
Experience with integrity/fraud detection systems and anti-abuse tooling

Benefits

Competitive salary
Equity
Benefits
Opportunity to earn a cash variable element, such as a bonus or commission

Company

Prolific

twittertwittertwitter
company-logo
Building the most advanced global infrastructure for People Science.
dateFounded in 2014
location
Oxford, Oxfordshire, GBR
people-size51-200 employees
web-link
https://www.prolific.com/

H1B Sponsorship

Prolific has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (12)
2024 (7)
2023 (16)
2022 (39)
2021 (16)
2020 (8)

Funding

Current Stage
Growth Stage
Total Funding
$33.54M
2023-07-11Series A· $32.34M
2019-12-04Seed· $1.2M

Leadership Team

leader-logo
Phelim Bradley
CEO
linkedin

Recent News

TechRound
Expert Predictions For DeepTech In 2026, Part 2
2026-01-22
prnasia.com
New Research from UpGuard Reveals 68% of Security Leaders Admit to Unauthorized AI Usage
2025-11-10
BusinessCloud
King and Gustafsson big winners at Allstars Award
2025-10-19
Company data provided by crunchbase