Archetype AI · 1 week ago
World Model Evaluation Lead
United States
Full-time
Remote
Lead/StaffArchetype AI is developing an innovative AI platform to integrate AI into the real world. They are seeking a hands-on Evaluation Lead to design and implement evaluation techniques for assessing model performance in physical AI, and to build frameworks for testing and reporting on model effectiveness.
Artificial Intelligence (AI)Information TechnologySoftware
Responsibilities
Design and implement rigorous evaluation methodologies and benchmarks for measuring model effectiveness, reliability, alignment, and safety
Lead evaluation of model performance, ranging from offline experiments to full production model testing
Design and oversee the pipelines, dashboards, and tools that automate model evaluation
Design and oversee tools for A/B model testing, regression testing, and production model performance
Develop and implement strategies for evaluating physical AI models that can scale across a broad range of real-world use cases, sensor types, and edge cases
Plan, run, and oversee evaluations, across internal teams and external customers
Drive edge case discovery, red-teaming, safety, privacy, and risk evaluation - feeding back knowledge to key stakeholders in research and engineering teams
Qualification
Required
Extensive expertise in evaluating AI and machine learning models, ideally in physical AI or a related AI field
Experience in designing, implementing, and refining evaluation metrics
Deep understanding of machine learning, AI, and generative models
Excellent python and software engineering skills
Experience designing and building scaleable data pipelines and evaluation tools
Experience collaborating closely with key stakeholders from research, engineering, and product teams
Strong communication and documentation skills, with a bias for creating detailed evaluation reports that help drive model performance
Startup-ready mindset with the ability to thrive in high-velocity, high-ambiguity environments
Preferred
Experience evaluating real-world, real-time algorithms
Experience evaluating a broad range of sensor types, such as cameras, LIDAR, physical sensors, RF sensors, and beyond
A strong scientific approach to evaluation and understanding model performance
Experience in evaluating production algorithms
Experience building and curating data campaigns to create extensive test datasets
Experience managing internal teams and/or external vendors
Company
Archetype AI
Archetype AI develops Physical AI agents that harness real-world sensor data to enhance decision-making and automate processes.
Founded in 2023
11-50 employees
H1B Sponsorship
Archetype AI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
100%
Represents job field similar to this job
Engineering and Development
Trends of Total Sponsorships
*2026 (1)
Funding
Current Stage
Early StageTotal Funding
$48MKey Investors
Hitachi Ventures,IAG Capital PartnersComcast NBCUniversal LIFT LabsVenrock
2025-11-20Series A· $35M
2024-10-20Non Equity Assistance
2024-04-05Seed· $13M
Leadership Team
Jaime Lien
Co-Founder & Chief Scientist
Nick Gillian
Founder, CTO
Recent News
thecanadianpressnews.ca
2026-07-07
International Business Times
2026-05-26
Company data provided by crunchbase