Mindrift · 3 months ago
AI Agent Evaluation Analyst - AI Trainer
United States
Full-time
Remote
New Grad, Entry LevelMindrift is a company focused on using collective human intelligence to shape the future of AI. They are seeking an AI Agent Evaluation Analyst to validate and improve AI agents through quality assurance, research, and logical problem-solving in a flexible, project-based environment.
Computer Software
Responsibilities
Reviewing evaluation tasks and scenarios for logic, completeness, and realism
Identifying inconsistencies, missing assumptions, or unclear decision points
Helping define clear expected behaviors (gold standards) for AI agents
Annotating cause-effect relationships, reasoning paths, and plausible alternatives
Thinking through complex systems and policies as a human would to ensure agents are tested properly
Working closely with QA, writers, or developers to suggest refinements or edge case coverage
Qualification
Required
Curious and intellectually proactive contributors
Strong critical thinking skills
Comfortable with ambiguity and complexity
Ability to evaluate the soundness and consistency of complex setups
Experience in consulting, CHGK, Olympiads, case solving, or systems thinking
Ability to review evaluation tasks and scenarios for logic, completeness, and realism
Ability to identify inconsistencies, missing assumptions, or unclear decision points
Ability to help define clear expected behaviors (gold standards) for AI agents
Ability to annotate cause-effect relationships, reasoning paths, and plausible alternatives
Ability to think through complex systems and policies as a human would
Ability to work closely with QA, writers, or developers to suggest refinements or edge case coverage
Preferred
Analysts, researchers, or consultants
Students (senior undergrads / grad students) looking for an intellectually interesting gig
People open to a part-time and non-permanent opportunity
Company
Mindrift
Welcome to Mindrift — a space where innovation meets opportunity.
501-1000 employees
Funding
Current Stage
Late StageCompany data provided by crunchbase