Research Scientist, Latent State Inference for World Models at Toyota Research Institute

Los Altos, California, United States

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Automotive, Artificial Intelligence, RoboticsIndustries

Requirements

PhD in Computer Science, Machine Learning, Robotics, or a related field
Strong foundation in representation learning or state estimation for sequential decision-making
Robust experience in deep generative models (e.g., VAEs, diffusion models, autoregressive models)
Solid base in perception models from large-scale real-world sensor datasets from autonomous driving, robotics, or similar domains
Experience with latent world models, generative AI for perception, or contrastive learning
Familiarity with structure-from-motion, Gaussian splatting, or neural radiance fields (NeRFs)
Experience with multi-modal sensor fusion, state estimation, and SLAM techniques
Familiarity with uncertainty-aware perception, active perception, and predictive modeling
Accomplished publication record at top-tier conferences such as NeurIPS, CVPR, ICCV, ICLR, ICRA, CoRL, or RSS
Deep programming skills in Python and deep learning frameworks such as PyTorch or JAX
Excellent problem-solving skills and the ability to work in a fast-paced team

Responsibilities

Design and train learning-based systems that transform raw multimodal sensor data (e.g., images, lidar, radar) into compact, dynamic latent states suitable for use in learned world models
Investigate unsupervised, self-supervised, and contrastive methods to learn latent spaces that encode dynamics, semantics, and uncertainty
Incorporate temporal information and motion consistency into latent state estimation using recurrent, filtering, or transformer-based architectures
Combine data from heterogeneous modalities into a unified latent state representations that generalize across conditions and scenarios
Ensure the learned representations are resilient to occlusion, sensor degradation, and distributional shift
Collaborate on joint research agendas with world modeling and policy evaluation researchers to explore uncertainty modeling, interpretability, and representation bottlenecks
Publish novel research, contribute to open-source tools, and engage with the academic community at major ML and robotics conferences

Skills

Key technologies and capabilities for this role

Latent State InferenceWorld ModelsReinforcement LearningPerceptionPolicy EvaluationSensor Data ProcessingMulti-Agent ReasoningModel-Based RLAutonomous DrivingDeep Learning

Questions & Answers

Common questions about this position

What is the employment type for this position?

This is a full-time position.

Is this role remote or does it require working on-site?

This information is not specified in the job description.

What qualifications are required for this Research Scientist role?

A PhD in Computer Science, Machine Learning, Robotics, or a related field is required.

What kind of team will I be working with in this role?

You will work closely with researchers developing world models and those focused on policy evaluation within the Human Interactive Driving division's Extreme Performance Intelligent Control department.

What skills or experience make a strong candidate for this position?

Strong candidates should have expertise in designing learning-based systems for latent state inference from multimodal sensor data, experience with unsupervised/self-supervised methods, recurrent/transformer architectures, and a track record of publishing novel research in ML and robotics.

Toyota Research Institute

Research in mobility, safety, and automation

About Toyota Research Institute

Toyota Research Institute focuses on improving mobility through research and development in the automotive and technology sectors. The company works on enhancing safety, automated driving, robotics, materials science, and machine learning. Their products include advanced safety features and automated driving systems that aim to make driving safer and more efficient. Unlike many competitors, TRI emphasizes a research-driven approach, collaborating with various partners and licensing their innovations to enhance Toyota's offerings and maintain a competitive edge. The goal of TRI is to advance mobility solutions that improve quality of life and support the transition to zero-emissions transportation.

Los Altos, CaliforniaHeadquarters

2016Year Founded

$100MTotal Funding

SEEDCompany Stage

Robotics & Automation, Automotive & Transportation, AI & Machine LearningIndustries

201-500Employees

Benefits

Highly competitive benefits package

Robust programs to support the wellbeing, happiness, and health of our people and their families.

401(k) plan including matching and annual profit sharing along with total vacation and holidays totaling 38 days per year.

Subsidized commuter benefits and generous employee and vehicle allowances

2 paid days per year to participate in volunteer activities.

Maternity Leave program with 10 paid weeks plus baby bonding leave and Milk Stork for traveling moms

Baby Bonding Leave–an additional 16 paid weeks–to all new parents, including those who choose to adopt.

Back-up child and adult / elder care programs to help everyone thrive

WellBeats Virtual Fitness Trainer.

Risks

Public skepticism may undermine AI-assisted driving safety initiatives.

Boston Dynamics' high costs could impact TRI's humanoid robot project returns.

Complexity of real-world environments may delay TRI's robot skill deployment.

Differentiation

TRI focuses on AI to enhance human life and mobility.

Partnerships with Boston Dynamics and Stanford showcase TRI's collaborative innovation.

TRI's Driving Sensei concept integrates AI to improve driver skills and engagement.

Upsides

Generative AI allows TRI's robots to learn complex tasks without new coding.

TRI's AI-driven material synthesis could revolutionize EV battery development.

Autonomous tandem drifting collaboration with Stanford pushes vehicle dynamics boundaries.

Land your dream remote job 3x faster with AI

Try Jobo Free