Moody's Corporation · 12 hours ago
Asst Dir-Data Scientist
King of Prussia, PA
Full-time
Onsite
Lead/Staff, Director/Executive
$116K/yr - $169K/yr
2+ years expMoody's Corporation is a global leader in ratings and integrated risk assessment, advancing AI to move from insight to action. They are seeking an Assistant Director of Data Scientist to evaluate and validate large language models for analytical and decision-support systems, while collaborating with various teams to enhance credit analytics capabilities.
Financial Services
Responsibilities
Evaluate and validate large language models for production-grade analytical and decision-support systems
Design and implement evaluation frameworks for assessing LLM performance in credit analytics and decision-support contexts
Develop metrics and benchmarks to measure model robustness, reliability, consistency, and output quality
Analyze model behavior across diverse inputs, identifying failure modes, edge cases, and areas for improvement
Collaborate with model development and deployment teams to integrate validation processes into the model lifecycle
Conduct systematic assessments of model stability over time and across updates
Evaluate model outputs for bias, fairness, and economic relevance to credit risk applications
Develop and maintain documentation for evaluation methodologies, findings, and recommendations
Contribute to the advancement of best practices for LLM evaluation within the Credit COE
Qualification
Required
Ph.D. in Computer Science, Machine Learning, Natural Language Processing, Statistics, or a related quantitative field; or Master's degree with 2-3 years of experience in machine learning evaluation or a related area
Strong foundations in statistical methods, experimental design, and hypothesis testing
Experience evaluating machine learning or NLP models, including designing experiments and interpreting results
Familiarity with LLM evaluation benchmarks and methodologies
Strong programming skills in Python or R
Excellent communication skills in English (both written and verbal)
Preferred
Experience evaluating LLMs or generative AI systems
Experience with production machine learning systems
Exposure to cloud platforms such as AWS, GCP, or Azure
Publications or demonstrated work in model evaluation, benchmarking, or related areas
Benefits
Medical
Dental
Vision
Parental leave
Paid time off
A 401(k) plan with employee and company contribution opportunities
Life
Disability
Accident insurance
A discounted employee stock purchase plan
Tuition reimbursement
Company
Moody's Corporation
In a world shaped by increasingly interconnected risks, Moody’s helps customers develop a holistic view of these risks to advance their business and act decisively.
10001+ employees
H1B Sponsorship
Moody's Corporation has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (121)
2024 (120)
2023 (88)
2022 (112)
2021 (92)
2020 (64)
Funding
Current Stage
Late StageLeadership Team
Joshua Carroll
Chief Technology Officer
Dana Weinshank (MSW)
SVP-People Business Partner
Recent News
Hartford Business Journal
2024-06-05
GlobeNewswire News Room
2024-06-04
2024-06-01
Company data provided by crunchbase