Child Mind Institute · 23 hours ago
Rubric Engineering and Measurement Specialist
New York, NY
Full-time
Hybrid
Senior Level, Lead/Staff
$155K/yr - $190K/yr
8+ years expChild Mind Institute is dedicated to transforming the lives of children and families struggling with mental health and learning disorders. As the Rubric Engineering and Measurement Specialist, you will lead the implementation and validation of evaluation frameworks for assessing AI systems in clinical contexts, ensuring they are clinically grounded and operationally scalable.
EducationHealth CareMedicalNon ProfitService Industry
Responsibilities
Translate clinical symptom evaluations and manualized therapeutic protocols into reliable, consistent, precise, and scorable evaluation criteria
Design and implement metadata standards and protocols for multi-rater clinical evaluation systems, enabling crowd-sourced ratings from diverse clinical experts
Set reliability standards and adjudication logic for resolving disagreements among clinical annotators, ensuring consistency across large-scale annotation efforts
Establish and enforce quality assurance processes, versioning systems, and comprehensive documentation for all evaluation rubrics and measurement frameworks
Contribute to scientific publications, white papers, and technical documentation of measurement methodologies and validation studies
Collaborate with clinical leadership to ensure evaluation criteria accurately reflects clinical best practices, safety standards, and clinician feedback
Supervise and mentor a small team of data analysts and engineers, providing guidance on evaluation methodologies, statistical analysis, and rubric implementation
Collaborate with other technical project stakeholders, to ensure that technical definitions and requirements of all curated rubrics and resources are being met
Qualification
Required
PhD, PsyD, MD, or equivalent experience in measurement science, psychometrics, biostatistics, clinical psychology, epidemiology, or related field
8+ years of applied evaluation experience in clinical research, health outcomes research, or psychometric assessment
5+ years of experience with patient evaluation, clinical rating scales, or performance assessment frameworks
5+ years of experience in at least one scientific programming language (e.g., Python/R, Matlab) and related toolboxes or frameworks (e.g., Tidyverse, Scipy, Sklearn, Polars, Pytorch) is required. Experience can be concurrent with education
5+ years of experience with applied statistics, including reliability analysis, classical test theory, item response theory, or factor analysis
Demonstrated leadership of multi-rater or benchmark systems
Benefits
Medical insurance
401(k)
Paid parental leave
Dependent care
Flexible work schedules
Discounted tickets and entertainment perks programs
Company
Child Mind Institute
The Child Mind Institute is an independent national nonprofit dedicated to transforming the lives of children and families struggling with mental health and learning disorders.
201-500 employees
H1B Sponsorship
Child Mind Institute has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (11)
2024 (9)
2023 (11)
2022 (4)
2021 (8)
2020 (3)
Funding
Current Stage
Growth StageLeadership Team
Dr. Harold S. Koplewicz
President
Recent News
Social Media Today
2025-10-16
EIN Presswire
2025-08-21
Company data provided by crunchbase