Apply on Employer Site

LexisNexis Risk Solutions · 1 month ago

Senior Data Scientist III - AI Evaluation and Prompt Engineering

United States

Full-time

Remote

Mid, Senior Level

$124K/yr - $217K/yr

3+ years exp

LexisNexis Risk Solutions is part of RELX, a global provider of information-based analytics and decision tools. The role focuses on AI Evaluation and Prompt Engineering, where the Data Scientist will evaluate and improve large language model systems, build data-driven insights, and collaborate with product managers and data engineers to enhance legal research tools.

AnalyticsHealth CareInformation TechnologyInsurTechRisk ManagementSoftware

Culture & Values

H1B Sponsor Likely

Responsibilities

Evaluate and tune LLM-powered features, such as prompt optimization, retrieval-augmented generation (RAG) systems, and semantic search performance

Design and execute experiments to measure model quality, reliability, and user impact — translating technical findings into product recommendations

Develop and maintain data pipelines for evaluating, tracking, and improving system performance (e.g., accuracy, latency, cost, and relevance metrics)

Analyze structured and unstructured datasets (e.g., product usage logs, document metadata, LLM outputs) to identify patterns, insights, and areas for optimization

Collaborate with product managers to translate product goals into measurable data science questions, propose next steps, and inform roadmap priorities

Provide technical guidance to data engineers who build and maintain analytics and model evaluation infrastructure

Communicate results clearly — through written reports, dashboards, and presentations — to technical and non-technical stakeholders

Stay current on emerging practices in applied NLP, LLM evaluation, and data-driven product development, and thoughtfully adapt them to our environment

Qualification

Data scienceApplied NLPLLM-based systemsPythonSQLData explorationSelf-directionCuriosityCommunication skillsCollaboration

Required

3–6 years of experience in data science, applied NLP, or AI product analytics, preferably within a SaaS or research-heavy product environment

Strong proficiency in Python and data analysis libraries such as Pandas; solid working knowledge of SQL

Ability to design and evaluate LLM-based systems (e.g., RAG pipelines, prompt evaluations, output scoring), even if not specialized in deep learning

Experience with data exploration, experimentation, and reporting — from defining metrics to visualizing and interpreting results

Comfort working with document-based datasets (e.g., text corpora, metadata, embeddings) and understanding information retrieval / semantic search concepts

Excellent written and verbal communication skills — able to present complex ideas simply and persuasively across distributed teams

Proven ability to self-direct, learn new tools and concepts quickly, and apply them pragmatically

Strong sense of curiosity, patience, and collaboration — especially in working across different disciplines and cultures

Preferred

Familiarity with tools like LangChain, OpenAI API, Databricks, or similar LLM/AI development environments

Experience designing evaluation frameworks or product analytics systems for AI-driven products

Prior exposure to legal, financial, or corporate document datasets

Experience mentoring or informally guiding junior technical teammates

Benefits

Flexible remote-first work environment, with the option to work from our New York office

Comprehensive health coverage, including medical, dental, and vision plans

Retirement plan with inclusive risk benefits (disability, critical illness, life cover, and funeral cover)

Modern family benefits, including adoption, surrogacy, and parental leave

Paid study leave and professional development support

Well-being initiatives and opportunities for sabbaticals and personal growth

A culture that values work/life balance, clear communication, and continuous learning

Health Benefits: Comprehensive, multi-carrier program for medical, dental and vision benefits

Retirement Benefits: 401(k) with match and an Employee Share Purchase Plan

Wellbeing: Wellness platform with incentives, Headspace app subscription, Employee Assistance and Time-off Programs

Short-and-Long Term Disability, Life and Accidental Death Insurance, Critical Illness, and Hospital Indemnity

Family Benefits, including bonding and family care leaves, adoption and surrogacy benefits

Health Savings, Health Care, Dependent Care and Commuter Spending Accounts

In addition to annual Paid Time Off, we offer up to two days of paid leave each to participate in Employee Resource Groups and to volunteer with your charity of choice

Company

LexisNexis Risk Solutions

Glassdoor4.0

LexisNexis Risk Solutions provides information to assist customers in industry and government in assessing, predicting, and managing risk. It is a sub-organization of ChoicePoint.

Founded in 2000

Alpharetta, Georgia, USA

10001+ employees

https://risk.lexisnexis.com

H1B Sponsorship

LexisNexis Risk Solutions has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2025 (129)

2024 (132)

2023 (86)

2022 (98)

2021 (125)

2020 (53)