Handshake · 2 weeks ago
Staff AI Research Scientist - Data Quality, Handshake AI
New York, NY
Full-time
Hybrid
Senior Level, Lead/Staff
5+ years expHandshake is the career network for the AI economy, connecting knowledge workers with educational institutions and employers. The Staff AI Research Scientist will lead research and development in data quality and post-training techniques to enhance large language model alignment and ensure data integrity for AI performance.
College RecruitingData Collection and LabelingEmploymentHuman ResourcesRecruiting
Responsibilities
Lead high-impact research on data quality frameworks for post-training LLMs — including techniques for preference consistency, label reliability, annotator calibration, and dataset auditing
Design and implement systems for identifying noisy, low-value, or adversarial data points in human feedback and synthetic comparison datasets
Drive strategy for aligning data collection, curation, and filtering with post-training objectives such as helpfulness, harmlessness, and faithfulness
Collaborate cross-functionally with engineers, alignment researchers, and product leaders to translate research into production-ready pipelines for RLHF and DPO
Mentor and influence junior researchers and engineers working on data-centric evaluation, reward modeling, and benchmark creation
Author foundational tools and metrics that connect supervision data characteristics to downstream LLM behavior and evaluation performance
Publish and present research that advances the field of data quality in LLM post-training, contributing to academic and industry best practices
Qualification
Required
PhD or equivalent experience in machine learning, NLP, or data-centric AI, with a track record of leadership in LLM post-training or data quality research
5 years of academic or industry experience post-doc
Deep expertise in RLHF, preference data pipelines, reward modeling, or evaluation systems
Demonstrated experience designing and scaling data quality infrastructure — from labeling frameworks and validation metrics to automated filtering and dataset optimization
Strong engineering proficiency in Python, PyTorch, and ecosystem tools for large-scale training and evaluation
A proven ability to define, lead, and execute complex research initiatives with clear business and technical impact
Strong communication and collaboration skills, with experience driving strategy across research, engineering, and product teams
Preferred
Experience with data valuation (e.g. influence functions, Shapley values), active learning, or human-in-the-loop systems
Contributions to open-source tools for dataset analysis, benchmarking, or reward model training
Familiarity with evaluation challenges such as annotation disagreement, subjective labeling, or multilingual feedback alignment
Interest in the long-term implications of data quality for AI safety, governance, and deployment ethics
Benefits
Equity in a fast-growing company
401(k) match, competitive compensation, financial coaching
Paid parental leave, fertility benefits, parental coaching
Medical, dental, and vision, mental health support, $500 wellness stipend
$2,000 learning stipend, ongoing development
Internet, commuting, and free lunch/gym in our SF office
Flexible PTO, 15 holidays + 2 flex days
Team outings & referral bonuses
Company
Handshake
Handshake is a college career network that helps students and recent graduates find their next opportunity.
501-1000 employees
H1B Sponsorship
Handshake has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (24)
2024 (14)
2023 (7)
2022 (15)
2021 (6)
2020 (4)
Funding
Current Stage
Late StageTotal Funding
$434MKey Investors
Notable CapitalEQT VenturesSpark Capital
2022-01-19Series F· $200M
2021-05-12Series E· $80M
2020-10-20Series D· $80M
Leadership Team
Garrett Lord
CEO - Co-Founder
Ben Christensen
Co-Founder
Recent News
Company data provided by crunchbase