Requirements

Candidates must have significant experience optimizing for memory, latency, and throughput in PyTorch, along with experience benchmarking and profiling GPU and CPU code for optimal device utilization. Experience with non-NVIDIA systems, using torch.compile or torch.XLA, and working with transformer models and attention implementations is required. Familiarity with high-performance Triton/CUDA and writing custom PyTorch kernels is preferred, as well as experience with parallel inference, particularly with tensor parallelism and pipeline parallelism. Additionally, experience in writing high-performance parallel C++ within an ML context and building inference or demo prototype code is a bonus.