Senior HPC AI Cluster Engineer

Position Overview

• Location Type:
• Job Type: Full time
• Salary:

NVIDIA is seeking an experienced HPC Engineer to join the E2E software verification HPC/AI Infrastructure team. This role involves building supercomputers and HPC clusters utilizing groundbreaking technologies. The ideal candidate will be an outstanding architect for senior HPC roles, playing a key part in the most exciting computing hardware and software advancements in artificial intelligence and GPU computing. Responsibilities include providing insights on at-scale system design and tuning mechanisms for large-scale compute runs. You will work with the latest Accelerated computing and Deep Learning software and hardware platforms, collaborating with scientific researchers, developers, and customers to refine workflows and develop new, differentiated solutions. You will also interact with HPC, OS, GPU compute, and systems specialists to architect, develop, and bring up large-scale performance platforms.

Responsibilities

• Designing, implementing, and maintaining large-scale HPC/AI clusters with integrated monitoring, logging, and alerting.
• Managing Linux job/workload schedulers and orchestration tools.
• Developing and maintaining continuous integration and delivery pipelines.
• Creating tooling to automate the deployment and management of large-scale infrastructure environments.
• Automating operational monitoring and alerting.
• Enabling self-service consumption of resources.
• Deploying monitoring solutions for servers, networks, and storage.
• Troubleshooting and resolving issues from bare metal, operating system, software stack, to application level.
• Serving as a technical resource, developing, redefining, and documenting standard methodologies for internal teams.
• Supporting Research & Development activities and engaging in Proofs of Concept (POCs)/Proofs of Value (POVs) for future improvements.

Requirements

• Bachelor's Degree in Computer Science, Engineering, or a related field; or equivalent experience.
• 5+ years of experience.
• Knowledge of HPC and AI solution technologies, from CPUs and GPUs to high-speed interconnects and supporting software.
• Experience with job scheduling workloads and orchestration tools such as Slurm and Kubernetes.
• Excellent knowledge of Windows and Linux (Red Hat/CentOS and Ubuntu) networking (sockets, firewalls, iptables, Wireshark, etc.) and internals, Access Control Lists (ACLs), OS-level security protection, and common protocols (e.g., TCP, DHCP, DNS).
• Experience with multiple storage solutions such as Lustre, GPFS, ZFS, and XFS. Familiarity with newer and emerging storage technologies.
• Python programming and bash scripting experience.
• Comfort with automation and configuration management tools such as Jenkins, Ansible, Puppet/Chef.
• Deep knowledge of Networking Protocols like InfiniBand and Ethernet.
• Deep understanding and experience with virtual systems (e.g., VMware, Hyper-V, KVM, or Citrix).
• Familiarity with cloud computing platforms (e.g., AWS, Azure, Google Cloud).

Ways to Stand Out

• Knowledge of CPU and/or GPU architecture.
• Knowledge of Kubernetes and container-related microservice technologies.
• Experience with GPU-focused hardware/software (DGX, CUDA).
• Background with RDMA (InfiniBand or RoCE) fabrics.

Company Information

NVIDIA has been redefining computer graphics, PC gaming, and accelerated computing for over 25 years. We have a unique legacy of innovation fueled by great technology and amazing people. Today, we are harnessing the unlimited potential of AI to define the next era of computing, where our GPUs serve as the intelligence for computers, robots, and self-driving cars capable of understanding the world. Achieving the unprecedented requires vision, innovation, and the world's best talent. Our teams comprise driven, innovative professionals dedicated to pushing technological boundaries. We offer highly competitive salaries, a comprehensive benefits package, and a work environment that fosters diversity, inclusion, and flexibility. As an equal opportunity employer.