Apply on Employer Site

Crowe · 2 months ago

AI Governance Testing & Validation Manager

Chicago IL USA

Full-time

Hybrid

Senior Level

$102K/yr - $204K/yr

5+ years exp

Crowe LLP is a leading public accounting, consulting, and technology firm in the United States. The AI Governance Testing & Validation Manager will lead independent testing and operational monitoring of AI systems, ensuring alignment with business goals and regulatory standards while mentoring consultants and evolving governance methodologies.

AccountingAdviceConsultingFinanceFinancial ServicesInformation TechnologyProfessional ServicesTax Consulting

No H1B

Responsibilities

Design and execute independent test plans for classical ML and LLMs/GenAI (functional accuracy, robustness, safety, toxicity, jailbreak/prompt-injection, hallucination/error rates); define acceptance criteria and go/no-go recommendations

Partner with teams to qualify opportunities, shape solutions/SOW/ELs, develop proposals and pricing, and contribute to pipeline reviews. Build client-ready collateral

Evolve Crowe’s AI Governance methodologies, accelerators, control libraries, templates, and training. Incorporate updates from standards/regulators into our playbooks (e.g., NIST’s GAI profile)

Publish insights, speak on webinars/events, and support marketing campaigns to grow brand presence

Supervise, coach, and develop consultants; manage engagement economics (scope, timeline, budget, quality) and support recruiting

Plan and run bias/fairness assessments using appropriate population slices and fairness metrics; document mitigations per NIST guidance on identifying/managing bias

Produce model explainability/transparency artifacts (e.g., model cards, method docs) and apply techniques (SHAP, LIME, feature attributions) aligned to NIST’s Four Principles of Explainable AI

Qualification

AI governance experiencePythonSQLNIST AI RMF knowledgeBias/fairness testingExplainability methodsProject managementNetworkingRelationship managementWritten communicationVerbal communication

Required

3+ years hands-on AI governance/Responsible AI experience (policy, controls, risk, compliance, or assurance of AI/ML systems)

5+ years in compliance, risk management, and/or professional services/consulting with client-facing delivery and team leadership

Strong Python and SQL (evaluation pipelines, data prep, metric computation, scripting CI jobs)

Demonstrated experience designing fairness/bias tests and applying explainability methods; ability to translate results for non-technical stakeholders

Practical knowledge of NIST AI RMF 1.0 (and GenAI profile), ISO/IEC 42001, and awareness of EU AI Act obligations for high-risk systems

Prior experience should include progressive responsibilities, including supervising and reviewing the work of others, and project management, including self-management of simultaneous work-streams and responsibilities

Strong written and verbal communication and comprehension both formally and informally to our clients and our teams, in a variety of formats and settings, including in interviews, meetings, calls, e-mails, reports, process narratives, presentations, etc

Networking and relationship management

Willingness to travel

Preferred

Experience operationalizing LLM/GenAI evaluations (adversarial/red-team testing, toxicity/harm scoring, retrieval/grounding, hallucination measurement, safety policies) consistent with NIST guidance

Hands-on with ML Ops/observability (e.g., model registries, data validation, drift detection), cloud (AWS/Azure/GCP), and containerization

Familiarity with governance and compliance platforms (e.g., GRC systems) and collaboration with privacy/security/legal

Bachelor's degree required; advanced degree a plus (CS, statistics, data science, information systems, or related)

Certification: AIGP – Artificial Intelligence Governance Professional (IAPP) or equivalent credential in AI governance/privacy/risk (e.g., CIPP/CIPM/CIPT with AI coursework, ISO/IEC 42001 implementer/auditor)