Senior Deep Learning Software Engineer

Employment Type: Full-time Location Type: Hybrid

Position Overview

We are seeking a Senior Deep Learning Software Engineer to design and build our automated inference and deployment solution. This role is instrumental in defining a scalable architecture for Deep Learning (DL) inference, focusing on ease-of-use and compute efficiency. Your work will span multiple layers of the DL deployment stack, including developing features in high-level frameworks like PyTorch and JAX, designing and implementing a high-performance execution environment, low-level GPU optimizations, and developing custom GPU kernels in CUDA and/or Triton. This is an exceptional opportunity for passionate software engineers who bridge the boundaries of research and engineering, possessing a strong background in both machine learning fundamentals and software architecture & engineering.

What You'll Be Doing

• Play a pivotal role in defining a modular, scalable platform to seamlessly bridge training and deployment workflows, enabling tight integration of deployment tooling with training frameworks such as Megatron and Nemo.
• Leverage and build upon the PyTorch 2.0 ecosystem (TorchDynamo, torch.export, torch.compile, etc.) to analyze and extract standardized model graph representation from arbitrary PyTorch models for our automated deployment solution.
• Develop support for inference optimization techniques such as speculative decoding and LoRA.
• Collaborate with teams across NVIDIA to utilize performant kernel implementations within the automated deployment solution.
• Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities.
• Continuously innovate on inference performance to ensure NVIDIA's inference software solutions (TRT, TRT-LLM, TRT Model Optimizer) maintain and increase their market leadership.

What We Need to See

• Master's, PhD, or equivalent experience in Computer Science, AI, Applied Math, or a related field.
• 8+ years of relevant work or research experience in Deep Learning.
• Excellent software design skills, including debugging, performance analysis, and test design.
• Strong proficiency in Python, PyTorch, and related ML tools.
• Strong algorithms and programming fundamentals.
• Good written and verbal communication skills, with the ability to work independently and collaboratively in a fast-paced environment.

Ways to Stand Out

• Contributions to PyTorch, JAX, or other Machine Learning Frameworks.
• Knowledge of GPU architecture and compilation stack, and the capability to understand and debug end-to-end performance.
• Familiarity with NVIDIA's deep learning SDKs such as TensorRT.
• Prior experience in writing high-performance GPU kernels for machine learning workloads in frameworks such as CUDA, CUTLASS, or Triton.

Company Information

NVIDIA is increasingly known as "the AI computing company" and is widely considered one of the technology world’s most desirable employers. If you are creative, motivated, and love a challenge, we want to hear from you! Join our model optimization group, where you can help build real-time, cost-effective computing platforms driving our success in this exciting and rapidly-growing field.

Salary Range: $184,000 - $356,500 USD (base salary will be determined based on location, experience, and the pay of employees in similar positions). Additional Compensation: Eligible for equity and benefits.

NVIDIA accepts applications on an ongoing basis. NVIDIA is committed to fostering a diverse work environment and is proud to be an equal opportunity employer. We do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law.