Reflection AI · 21 hours ago
Member of Technical Staff - Evaluations
San Francisco, CA
Full-time
Onsite
Mid LevelReflection AI is on a mission to build open superintelligence and make it accessible to all. The role involves conducting comparative analysis, building evaluation systems, and collaborating with teams to improve model capabilities.
Computer Software
Responsibilities
Conduct critical comparative analysis to advance our understanding of model capabilities
Build and refine evaluation systems and processes that create tight feedback loops between data, evals, and model behavior
Develop generalizable evaluation frameworks that capture what matters for reasoning, alignment, and usefulness
Collaborate closely with pre-training, post-training, and applied teams to translate insights into model improvements
Push the boundaries of what’s measurable, from synthetic evals to human feedback and real-world interaction data
Qualification
Required
Strong statistical analysis and experimental design skills to rigorously measure model improvements
Familiarity with LLM evaluation methodologies: static benchmarks, human preference evals, and/or agentic tasks
High agency and thrive in a fast-paced startup environment; bias for impact over process
Excited to work in a new frontier lab, defining how we measure and accelerate progress toward more capable models
Collaborative, detail-oriented, and motivated by building the feedback loops that make models truly improve
Benefits
Comprehensive medical, dental, vision, life, and disability insurance.
Fully paid parental leave for all new parents, including adoptive and surrogate journeys.
Financial support for family planning.
Paid time off when you need it, relocation support, and more perks that optimize your time.
Lunch and dinner are provided daily.
Regular off-sites and team celebrations.
Company
Reflection AI
Frontier open intelligence accessible to all. Our team previously built frontier LLMs at labs like DeepMind, OpenAI, and Anthropic.
11-50 employees
H1B Sponsorship
Reflection AI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (5)
Funding
Current Stage
Early StageCompany data provided by crunchbase