Senior Software Engineer, Vision Language Models

Position Overview

• Location Type: Remote
• Employment Type: Full-time
• Salary: Not specified

Motional is seeking a talented engineer to join our Data Mining team. This role focuses on leveraging foundation models like multi-modal encoding models, vision language models, and large language models to improve our autonomous driving vehicle's performance. The team develops billion-scale data workflows and algorithms to mine data from our fleet, identifying model errors, anomalies, and rare driving scenarios. This data is then used to train and validate our core ML products.

Requirements

• Education: BS in computer science, similar discipline, or equivalent experience.
• Experience: 3+ years of experience architecting and shipping high-performance, large-scale distributed systems.
• VLMs/Vision Encoders: Experience with deploying vision language models (VLMs) or large-scale vision encoders (e.g., CLIP) in production settings for image/video understanding, object detection, or searching.
• Cloud Services: Experience with core cloud services (e.g., AWS’s S3, Athena, RDS or similar).
• Software Engineering: Solid software engineering principles – including software design patterns, configuration management, source control, build processes, code reviews, testing methodologies, app containerization, continuous integration, and DevOps practices.
• Programming: Fluency in Python and experience on production-quality software development.

Responsibilities

• Develop Data Products: Utilize foundation models (multi-modal encoding models, VLMs, LLMs) adapted to the autonomous driving domain through pre-training, fine-tuning, and prompt optimization.
• Own Mining Workflows: Design and implement large-scale mining workflows to surface rare objects, model errors, and long-tail events.
• Build Datasets: Construct high-quality datasets to improve ML products through training and edge case validation.
• Data Processing Pipelines: Contribute to data processing pipelines that fuel the company's in-house billion-scale image search engine.
• Statistical Analysis: Provide statistical depth on model performance and generalization through rigorous error analysis across complex driving scenarios.

Bonus Points (Not Required)

• Advanced Degree: MS/PhD in computer science, machine learning, statistics, or computer vision.
• ML Techniques: Experience with at least one of the following ML techniques/models: Few-shot Learning, Metric Learning, Information Retrieval, Recommender Systems, Contrastive Learning, Semi-supervised Learning, Object Detection / Segmentation / Prediction.
• Deep Learning Frameworks: Experience with PyTorch or other deep learning frameworks (Jax, TensorFlow, etc.).
• A/B Testing: Experience with A/B testing methodologies and metrics tracking systems.
• Machine Learning Experience: Experience with machine learning lifecycle.

Company Information

Motional is a company focused on developing autonomous driving technology. They collect vast amounts of data from their robo-taxi fleet and use data mining and machine learning to improve the performance and safety of their vehicles.