AI Infrastructure Engineer at Abridge

Candidates should have strong experience in building and deploying machine learning models in production environments, with a deep understanding of container orchestration and distributed systems architecture. Expertise in Kubernetes administration, including custom resource definitions, operators, and cluster management, is required. Experience developing APIs and managing distributed systems for both batch and real-time workloads, along with excellent communication skills to interface between research and product engineering, is also necessary. Bonus points are given for expertise with model serving frameworks like NVIDIA Triton Server, VLLM, TRT-LLM, ML toolchains such as PyTorch or Tensorflow, GPU cluster management, CUDA optimization, infrastructure as code (Terraform, Ansible), GitOps practices, container registries, image optimization, multi-stage builds for ML workloads, and orchestrating ASR or LLM models for GenAI applications.