AI Evaluation manager at Luma AI

Palo Alto, California, United States

Apply Now

$300,000 – $350,000Compensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

AI, TechnologyIndustries

Requirements

Master’s degree or higher in Cognitive Science, Human-Computer Interaction (HCI), Design Research, Psychology, Media Studies, or a related field
5+ years of experience in product evaluation, UX research, model testing, or similar roles that involve structured qualitative assessment
Deep familiarity with creative workflows and real-world use cases for generative models (e.g., animation, filmmaking, digital art, VFX)
Strong systems thinking and the ability to define abstract qualities (like believability, identity retention, or scene coherence) in clear evaluative terms
Experience working cross-functionally with engineers, researchers, and creatives
Excellent written communication skills and the ability to synthesize nuanced judgments into clear, actionable insights

Responsibilities

Evaluate generative model performance across diverse tasks, prompts, and modalities
Identify key failure modes, regression patterns, and edge cases that impact product quality
Develop and maintain qualitative evaluation frameworks that are scalable and reusable
Collaborate closely with technical artists and engineers to align evaluations with model capabilities and target use cases
Translate high-level product goals into concrete evaluative criteria
Lead qualitative studies, side-by-side comparisons, and human-in-the-loop evaluation efforts
Provide detailed feedback that informs model fine-tuning, dataset curation, and product UX
Stay informed about emerging evaluation standards in generative AI and creative tools

Skills

Key technologies and capabilities for this role

Generative AIModel EvaluationUX ResearchQualitative AssessmentCreative WorkflowsHCICognitive ScienceSystems ThinkingHuman-in-the-Loop EvaluationDataset CurationModel Fine-Tuning

Questions & Answers

Common questions about this position

What is the salary range for the AI Evaluation Manager role?

The salary range is $300K - $350K.

Is this position remote or hybrid?

The position is hybrid.

What qualifications and skills are required for this role?

A Master’s degree or higher in fields like Cognitive Science, HCI, Design Research, Psychology, or related is required, along with 5+ years in product evaluation, UX research, or model testing. Key skills include deep familiarity with creative workflows for generative models, strong systems thinking, cross-functional collaboration experience, and excellent written communication.

What is the company culture like at Luma AI?

This information is not specified in the job description.

What makes a strong candidate for this AI Evaluation Manager position?

Strong candidates have a relevant Master’s or higher, 5+ years in qualitative assessment roles, experience with creative workflows in generative AI, systems thinking for defining abstract qualities, and cross-functional collaboration skills. Nice-to-haves include backgrounds in motion graphics, VFX, storytelling, AI media evaluation, building qualitative tools, or prompt engineering.

Luma AI

Develops multimodal AI technologies for creativity

About Luma AI

Luma AI develops multimodal artificial intelligence technologies that enhance human creativity and capabilities. Their main product, the Dream Machine, allows users to interact with various types of data, enabling creative professionals, businesses, and developers to explore innovative applications of AI. Unlike many competitors, Luma AI focuses on integrating multiple modes of interaction, which broadens the possibilities for users. The company operates on a subscription model, providing access to its AI tools and services, and aims to lead the way in AI-driven creativity and productivity.

San Francisco, CaliforniaHeadquarters

2021Year Founded

$84.9MTotal Funding

LATE_VCCompany Stage

Consumer Software, AI & Machine LearningIndustries

51-200Employees

Benefits

Company Equity

Stock Options

Risks

Competition from Google's Veo 2 and OpenAI's Sora Turbo challenges Luma AI's market position.

AWS's cost-reducing features may make Luma AI's services less competitive.

AI-generated content controversies could harm Luma AI's reputation if outputs are misleading.

Differentiation

Luma AI transforms text into 3D models, enhancing user interaction with digital content.

The Dream Machine integrates multimodal AI, offering a unique creative platform for users.

Luma AI's Photon models provide high-quality, personalized image generation capabilities.

Upsides

Luma AI's $90 million funding boosts its capacity for innovation and market expansion.

Dream Machine 1.5's advanced text-to-video features attract creative professionals and businesses.

Expanding Dream Machine into a mobile app increases accessibility and user engagement.

Land your dream remote job 3x faster with AI

Try Jobo Free