Research Engineer - Evaluations at Luma AI

Palo Alto, California, United States

Apply Now

$220,000 – $280,000Compensation

Mid-level (3 to 4 years), Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Artificial Intelligence, TechnologyIndustries

Requirements

Master's or PhD in Computer Science, Machine Learning, or a related technical field (or equivalent industry experience)
3+ years of experience building ML evaluation systems, model pipelines, or large-scale infrastructure
Hands-on experience working with visual data (images and/or video), including evaluation, modeling, or data preparation
Proficiency in Python and ML frameworks (PyTorch, JAX, or TensorFlow)
Familiarity with human-in-the-loop evaluation workflows and how to scale them with automation
Strong background in machine learning, with experience in generative models (diffusion, LLMs, multimodal architectures)
Strong software engineering skills (CI/CD, testing, data pipelines, distributed systems)

Responsibilities

Design and implement scalable pipelines for automated evaluation of generative models, with a focus on visual and multimodal outputs (image, video, text, audio)
Develop novel metrics and evaluation models that capture qualities like fidelity, coherence, temporal consistency, and alignment with human intent
Integrate evaluation signals into training loops (including reinforcement learning and reward modeling) to continuously improve model performance
Build infrastructure for large-scale regression testing, benchmarking, and monitoring of multimodal generative models
Collaborate with researchers running human studies to translate human evaluation frameworks into automated or semi-automated systems
Partner with model researchers to identify failure cases and build targeted evaluation harnesses
Maintain dashboards, reporting tools, and alerting systems to surface evaluation results to stakeholders
Stay current with emerging evaluation techniques in generative AI, multimodal LLMs, and perceptual quality assessment

Skills

Key technologies and capabilities for this role

Generative AIMultimodal ModelsComputer VisionVideo ProcessingReinforcement LearningReward ModelingML PipelinesEvaluation MetricsBenchmarkingMonitoringDashboardsLarge-scale Infrastructure

Questions & Answers

Common questions about this position

What is the salary range for the Research Engineer position?

The salary range is $220K - $280K.

Is this role remote or hybrid?

The position is hybrid.

What skills are required for this Research Engineer role?

Required qualifications include a Master's or PhD in Computer Science, Machine Learning, or related field, 3+ years building ML evaluation systems or pipelines, hands-on experience with visual data, proficiency in Python and ML frameworks like PyTorch, JAX, or TensorFlow, and strong software engineering skills.

What kind of collaboration is involved in this role?

You'll collaborate with research, engineering, and product teams, partner with researchers on human studies, and work with model researchers to identify failure cases.

What makes a strong candidate for this Research Engineer role?

Strong candidates have a Master's or PhD in a relevant field with 3+ years of ML evaluation or infrastructure experience, hands-on work with visual data, proficiency in Python and ML frameworks, and strong software engineering skills; nice-to-haves include RL/reward modeling and perceptual metrics experience.

Luma AI

Develops multimodal AI technologies for creativity

About Luma AI

Luma AI develops multimodal artificial intelligence technologies that enhance human creativity and capabilities. Their main product, the Dream Machine, allows users to interact with various types of data, enabling creative professionals, businesses, and developers to explore innovative applications of AI. Unlike many competitors, Luma AI focuses on integrating multiple modes of interaction, which broadens the possibilities for users. The company operates on a subscription model, providing access to its AI tools and services, and aims to lead the way in AI-driven creativity and productivity.

San Francisco, CaliforniaHeadquarters

2021Year Founded

$84.9MTotal Funding

LATE_VCCompany Stage

Consumer Software, AI & Machine LearningIndustries

51-200Employees

Benefits

Company Equity

Stock Options

Risks

Competition from Google's Veo 2 and OpenAI's Sora Turbo challenges Luma AI's market position.

AWS's cost-reducing features may make Luma AI's services less competitive.

AI-generated content controversies could harm Luma AI's reputation if outputs are misleading.

Differentiation

Luma AI transforms text into 3D models, enhancing user interaction with digital content.

The Dream Machine integrates multimodal AI, offering a unique creative platform for users.

Luma AI's Photon models provide high-quality, personalized image generation capabilities.

Upsides

Luma AI's $90 million funding boosts its capacity for innovation and market expansion.

Dream Machine 1.5's advanced text-to-video features attract creative professionals and businesses.

Expanding Dream Machine into a mobile app increases accessibility and user engagement.

Land your dream remote job 3x faster with AI

Try Jobo Free