Supio · 4 hours ago
Manager, Software Engineering - AI Evals Framework
San Francisco, CA
Full-time
Onsite
Mid, Senior Level
2+ years expSupio is seeking Engineering Managers with AI Evals Framework experience to drive and scale the validation of their AI systems. The role involves leading a team of engineers and building tooling and platforms for reliable AI delivery across product teams.
Artificial Intelligence (AI)MarketingSoftware Engineering
Responsibilities
Lead and grow a team of 6–10 engineers while architecting a self-service evaluation platform used across product teams
Create the tooling that allows product teams to define, run, and visualize their own tests without being blocked by your team
Integrate our HITL workflows and subject-matter experts into the model evaluation pipeline through intuitive UI/UX and clear developer surfaces
Define and implement company-wide metrics for latency, cost, and accuracy across diverse tasks and products
Own the full lifecycle of our evaluation platform—from roadmap and architecture to execution, rollout, and continuous improvement
Qualification
Required
Track record of shipping complex technical products, internal platforms, or developer tools at startup speed
Deep hands-on experience building infrastructure for LLM evaluation, including datasets, runners, and metrics
Experience designing APIs and SDKs that other engineers genuinely enjoy using
Strong data engineering fundamentals; you care about dataset versioning, schema validation, and reproducibility
Bachelor's in CS or related field; 2+ years managing engineering teams or leading technical initiatives
Company
Supio
Supio is an accurate and dependable AI that assists attorneys in finding crucial information.
51-200 employees
H1B Sponsorship
Supio has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1)
Funding
Current Stage
Growth StageTotal Funding
$85MKey Investors
Sapphire Ventures
2025-04-30Series B· $60M
2024-08-27Series A· $25M
Leadership Team
Jerry Zhou
CEO
Kyle Lam
Co-Founder & CTO
Recent News
2025-11-14
Company data provided by crunchbase