Data Scientist (AI Quality & Evaluation) jobs in United States
cer-icon
Apply on Employer Site
company-logo

Bioscope AI ยท 11 hours ago

Data Scientist (AI Quality & Evaluation)

positionBoston, MA
timeFull-time
remoteOnsite
seniorityMid, Senior Level

Bioscope AI is seeking a Data Scientist to ensure the quality, reliability, and trustworthiness of their clinical AI outputs. The role involves developing evaluation frameworks, confidence scoring systems, and automated quality assurance processes that clinicians can trust.

Artificial Intelligence (AI)BiotechnologyCloud ComputingEnterprise SoftwareGeneticsHealth CareHealth DiagnosticsMedicalPrecision MedicineSaaS
check
H1B Sponsor Likelynote

Responsibilities

Design and implement automated evaluation pipelines that assess AI output quality, accuracy, and safety at scale
Develop uncertainty quantification systems where confidence scores meaningfully correlate with accuracy
Build comprehensive evaluation frameworks combining automated assessment with clinician-validated test cases
Implement feedback loops that continuously improve model outputs based on validation signals
Establish scalable quality gates that catch errors before they reach end users
Contribute to model alignment and fine-tuning efforts

Qualification

Deep learning frameworksModel evaluationPythonStatistical foundationsHealthcare AI experienceResearch paper implementationPublications in ML/AIEvaluation systems for LLMRLHF techniquesCommunication

Required

Strong foundation in deep learning frameworks (PyTorch) and LLM architectures
Experience with model evaluation, benchmarking, and quality metrics
Proficiency in Python and modern ML development tools
Strong statistical foundations
Ability to read, implement, and extend research papers
Excellent communication skills

Preferred

Master's degree in Computer Science, Machine Learning, Statistics, or related quantitative field (PhD preferred)
Publications in top ML/AI venues (NeurIPS, ICML, ICLR, ACL)
Experience with RLHF, DPO, or preference optimization techniques
Background in healthcare AI or regulated industries
Experience building evaluation systems for production LLM applications

Company

Bioscope AI

twittertwitter
company-logo
Bioscope AI offers a cloud CDS platform using genomics and AI to generate patient-specific insights with supporting evidence.
dateFounded in 2017
location
Carmel, Indiana, USA
people-size11-50 employees
web-link
https://www.bioscope.ai

H1B Sponsorship

Bioscope AI has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1)

Funding

Current Stage
Early Stage

Recent News

Newswire
Bioscope.AI Announces the "Try It on Yourself" Program Allowing Physicians to Use Artificial Intelligence and Genomics to Analyze Their Own Health
2025-10-22
Company data provided by crunchbase