Alight Solutions · 3 days ago
AI Quality Lead - Virtual
United States
Full-time
Remote
Senior Level
$100K/yr - $160K/yrAlight Solutions is dedicated to championing people and improving lives. As the AI Quality Lead for the Artificial Intelligence Center of Excellence, you will architect and drive the end-to-end AI Quality program, leading the development of quality strategies and ensuring compliance and operational excellence.
ConsultingEmployee BenefitsFinanceHuman ResourcesSoftware
No H1B
Responsibilities
Design and implement the unified AI Quality strategy and SOP, scaling controls by risk tier (Tier 1–3)
Define and own success metrics, risk tiering, acceptance criteria, and operating thresholds for AI solutions
Lead the development of golden/adversarial test suites (prompt injection, misuse, safety, bias, functional automation)
Oversee the creation and maintenance of schema-level UI/API tests, agent contracts, and observability instrumentation
Own the org-wide evaluation harnesses (e.g., promptfoo , TruLens , Ragas), CI gating, and red-team playbooks
Manage independent spot checks, factsheets, registry entries, and evidence for governance and audit
Collaborate with Product Owners, Developers, SRE/Observability, and Governance to ensure compliance and operational excellence
Drive continuous improvement in fairness, explainability, safety, security, and operational SLOs
Mentor and grow the AI Quality team, including onboarding and coaching junior engineers
Experience with LLM, RAG, agentic, or classic ML systems
Familiarity with test automation frameworks, CI/CD pipelines, and evaluation tools ( promptfoo , TruLens , Ragas)
Knowledge of fairness, explainability, safety, and security standards (e.g., OWASP Top 10, MITRE ATLAS, Fairlearn , AIF360)
Experience with data integrity, lineage, and observability ( OpenTelemetry , New Relic, OpenLineage )
Familiarity with AI infrastructure, orchestration, data integrity, deployments, and model registry practices
Knowledge of international standards and regulatory frameworks for Responsible AI
Qualification
Required
Experience leading or building quality engineering or testing programs (AI/ML experience is a plus, but not required)
Ability to design and implement test strategies, automation frameworks, and quality standards
Excellent communication and leadership skills; ability to work cross-functionally and influence stakeholders
Experience with LLM, RAG, agentic, or classic ML systems
Familiarity with test automation frameworks, CI/CD pipelines, and evaluation tools (promptfoo, TruLens, Ragas)
Knowledge of fairness, explainability, safety, and security standards (e.g., OWASP Top 10, MITRE ATLAS, Fairlearn, AIF360)
Experience with data integrity, lineage, and observability (OpenTelemetry, New Relic, OpenLineage)
Familiarity with AI infrastructure, orchestration, data integrity, deployments, and model registry practices
Knowledge of international standards and regulatory frameworks for Responsible AI
Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field, or equivalent experience
Benefits
Variety of health coverage options
Wellbeing and support programs
Retirement
Vacation and sick leave
Maternity, paternity & adoption leave
Continuing education and training
Several voluntary benefit options
Company
Alight Solutions
Alight Solutions is a cloud-based human capital technology company that offers HR and financial solutions.
Founded in 2017
10001+ employees
Funding
Current Stage
Public CompanyTotal Funding
$1.64BKey Investors
Caterpillar Ventures
2023-08-22Post Ipo Secondary· $22.5M
2021-07-06Post Ipo Equity· $1.55B
2021-07-06IPO
Leadership Team
Donna Dorsey
Chief Human Resources Officer
Stephen Rush
Chief Commercial Officer
Recent News
Business Wire
2025-12-18
2025-12-16
Company data provided by crunchbase