Research Engineer - Evaluations at Luma AI

Palo Alto, California, United States

Apply Now

$220,000 – $280,000Compensation

Mid-level (3 to 4 years), Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Artificial Intelligence, TechnologyIndustries

Requirements

Master's or PhD in Computer Science, Machine Learning, or a related technical field (or equivalent industry experience)
3+ years of experience building ML evaluation systems, model pipelines, or large-scale infrastructure
Hands-on experience working with visual data (images and/or video), including evaluation, modeling, or data preparation
Proficiency in Python and ML frameworks (PyTorch, JAX, or TensorFlow)
Familiarity with human-in-the-loop evaluation workflows and how to scale them with automation
Strong background in machine learning, with experience in generative models (diffusion, LLMs, multimodal architectures)
Strong software engineering skills (CI/CD, testing, data pipelines, distributed systems)

Responsibilities

Design and implement scalable pipelines for automated evaluation of generative models, with a focus on visual and multimodal outputs (image, video, text, audio)
Develop novel metrics and evaluation models that capture qualities like fidelity, coherence, temporal consistency, and alignment with human intent
Integrate evaluation signals into training loops (including reinforcement learning and reward modeling) to continuously improve model performance
Build infrastructure for large-scale regression testing, benchmarking, and monitoring of multimodal generative models
Collaborate with researchers running human studies to translate human evaluation frameworks into automated or semi-automated systems
Partner with model researchers to identify failure cases and build targeted evaluation harnesses
Maintain dashboards, reporting tools, and alerting systems to surface evaluation results to stakeholders
Stay current with emerging evaluation techniques in generative AI, multimodal LLMs, and perceptual quality assessment

Skills

Generative AI

Multimodal Models

Computer Vision

Video Processing

Reinforcement Learning

Reward Modeling

ML Pipelines

Evaluation Metrics

Benchmarking

Monitoring

Dashboards

Large-scale Infrastructure

Luma AI

Develops multimodal AI technologies for creativity

About Luma AI

Luma AI develops multimodal artificial intelligence technologies that enhance human creativity and capabilities. Their main product, the Dream Machine, allows users to interact with various types of data, enabling creative professionals, businesses, and developers to explore innovative applications of AI. Unlike many competitors, Luma AI focuses on integrating multiple modes of interaction, which broadens the possibilities for users. The company operates on a subscription model, providing access to its AI tools and services, and aims to lead the way in AI-driven creativity and productivity.

San Francisco, CaliforniaHeadquarters

2021Year Founded

$84.9MTotal Funding

LATE_VCCompany Stage

Consumer Software, AI & Machine LearningIndustries

51-200Employees

Benefits

Company Equity

Stock Options

Risks

Competition from Google's Veo 2 and OpenAI's Sora Turbo challenges Luma AI's market position.

AWS's cost-reducing features may make Luma AI's services less competitive.

AI-generated content controversies could harm Luma AI's reputation if outputs are misleading.

Differentiation

Luma AI transforms text into 3D models, enhancing user interaction with digital content.

The Dream Machine integrates multimodal AI, offering a unique creative platform for users.

Luma AI's Photon models provide high-quality, personalized image generation capabilities.

Upsides

Luma AI's $90 million funding boosts its capacity for innovation and market expansion.

Dream Machine 1.5's advanced text-to-video features attract creative professionals and businesses.

Expanding Dream Machine into a mobile app increases accessibility and user engagement.

Land your dream remote job 3x faster with AI

Try Jobo Free