Senior Machine Learning Engineer - Hardware Abstractions & Performance Optimization

Position Overview

• Location Type: Remote
• Job Type: FullTime
• Salary: $220K - $300K

Luma's mission is to build multimodal AI to expand human imagination and capabilities. We believe that multimodality is critical for intelligence. To go beyond language models and build more aware, capable, and useful systems, the next step-change will come from vision. So, we are working on training and scaling up multimodal foundation models for systems that can see and understand, show and explain, and eventually interact with our world to effect change. We are looking for engineers with significant experience maintaining & designing highly efficient systems and code that can be optimized to run on multiple hardware platforms, bringing our state-of-the-art models to as many people at the best performance per dollar.

Requirements

• Significant experience maintaining & designing highly efficient systems and code.
• Experience optimizing for memory, latency, and throughput in PyTorch.
• Experience benchmarking and profiling GPU & CPU code in PyTorch for optimal device utilization (e.g., torch profiler, memory profilers, trace viewers, custom tooling).
• Experience with parallel inference, particularly with tensor parallelism and pipeline parallelism.
• Experience writing high-performance parallel C++.

Responsibilities

• Ensure efficient implementation of models & systems with a focus on designing, maintaining, and writing abstractions that scale beyond NVIDIA/CUDA hardware.
• Identify and remedy efficiency bottlenecks (memory, speed, utilization, communication) by profiling and implementing high-performance PyTorch code, deferring to Triton or similar kernel-level languages as necessary.
• Benchmark our products across a variety of hardware & software to help the product team understand the optimal tradeoffs between latency, throughput, and cost at various degrees of parallelism.
• Work together with our partners to help them identify bottlenecks and push forward new iterations of hardware and software.
• Work closely together with the rest of the research team to ensure systems are planned to be as efficient as possible from start to finish and raise potential issues for hardware integration.
• Write fused kernels for common hot paths, understand when to make use of lower-level features like tensor cores or warp intrinsics, and understand where these tools can be most impactful.

Bonus Experience

• Experience with non-NVIDIA systems.
• Experience using torch.compile / torch.XLA.
• Experience building tools & abstractions to ensure models run optimally on different hardware and software stacks.
• Experience working with transformer models and attention implementations.
• Experience building inference / demo prototype code (incl. Gradio, Docker etc.).