Luma AI · 1 hour ago
Qualitative Evaluation Engineer
United States
Full-time
Remote
Senior Level
5+ years expLuma AI is focused on building multimodal AI to enhance human creativity and capabilities. They are seeking a Qualitative Evaluation Engineer to evaluate generative model performance and develop frameworks that capture qualitative nuances, collaborating with researchers and engineers to guide development based on real-world use cases.
Artificial Intelligence (AI)Generative AIVideoVideo Editing
Responsibilities
Evaluate generative model performance across diverse tasks, prompts, and modalities
Identify key failure modes, regression patterns, and edge cases that impact product quality
Develop and maintain qualitative evaluation frameworks that are scalable and reusable
Collaborate closely with technical artists and engineers to align evaluations with model capabilities and target use cases
Translate high-level product goals into concrete evaluative criteria
Lead qualitative studies, side-by-side comparisons, and human-in-the-loop evaluation efforts
Provide detailed feedback that informs model fine-tuning, dataset curation, and product UX
Stay informed about emerging evaluation standards in generative AI and creative tools
Qualification
Required
Master's degree or higher in Cognitive Science, Human-Computer Interaction (HCI), Design Research, Psychology, Media Studies, or a related field
5+ years of experience in product evaluation, UX research, model testing, or similar roles that involve structured qualitative assessment
Deep familiarity with creative workflows and real-world use cases for generative models (e.g., animation, filmmaking, digital art, VFX)
Strong systems thinking and the ability to define abstract qualities (like believability, identity retention, or scene coherence) in clear evaluative terms
Experience working cross-functionally with engineers, researchers, and creatives
Excellent written communication skills and the ability to synthesize nuanced judgments into clear, actionable insights
Preferred
Background in motion, visual effects, or storytelling pipelines
Experience evaluating AI-generated media (video, images, 3D)
Prior work on building internal tools for qualitative data collection or scoring
Familiarity with prompt engineering and reference-based input methods
Company
Luma AI
Luma AI develops tools that let users generate photorealistic images and videos from text, image, or video prompts.
11-50 employees
H1B Sponsorship
Luma AI has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (10)
2024 (3)
Funding
Current Stage
Growth StageTotal Funding
$1.06BKey Investors
HUMAINAndreessen HorowitzAmplify Partners
2025-11-19Series C· $900M
2024-12-06Series B· $90M
2024-01-09Series B· $43M
Leadership Team
Amit Jain
Co-Founder
Recent News
Crunchbase News
2025-12-10
2025-12-07
Company data provided by crunchbase