Manager, LLM Accuracy Evaluation at NVIDIA

Zurich, Zurich, Switzerland

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

AI, TechnologyIndustries

Requirements

BS, MS, or PhD in Computer Science, AI, Applied Math, or related field, or equivalent experience, with 7+ years of industry experience, including 3+ years in leadership
Proven success leading engineering teams and delivering complex AI/deep learning projects
Deep understanding of modern AI technologies—LLMs, multimodal models, retrieval-augmented generation, and agent frameworks—with the ability to guide technical strategy
Outstanding communication skills and the ability to partner effectively across organizations and with external collaborators
Demonstrated ability to mentor and grow engineering talent, fostering collaboration and technical excellence

Responsibilities

Lead and mentor a team of highly skilled engineers, fostering their growth while solving the most ambitious challenges in AI evaluation
Drive the accuracy evaluation of flagship AI models, coordinating efforts across internal teams and external partners to ensure timely, high-quality results
Collaborate with stakeholders across NVIDIA to balance speed of delivery with rigorous engineering practices
Develop and implement new methodologies for evaluating LLMs, multimodal systems, and agent frameworks at scale
Build a culture of innovation and excellence, encouraging continuous improvement and adoption of best practices in AI evaluation and deployment

Skills

Key technologies and capabilities for this role

LLMsRAGAI AgentsVision ModelsDeep LearningMultimodal ModelsGPU ClustersInference OptimizationAI Evaluation

Questions & Answers

Common questions about this position

What education and experience are required for the Manager, LLM Accuracy Evaluation role?

A BS, MS, or PhD in Computer Science, AI, Applied Math, or related field (or equivalent) is required, along with 7+ years of industry experience including 3+ years in leadership, proven success leading engineering teams on complex AI/deep learning projects, deep understanding of modern AI technologies like LLMs and multimodal models, outstanding communication skills, and ability to mentor engineering talent.

What does the role involve in terms of team leadership and AI evaluation?

The role requires leading and mentoring a team of engineers, driving accuracy evaluation of flagship AI models like Nemotron and GPT-4o, collaborating with stakeholders, developing new evaluation methodologies for LLMs and agents, and building a culture of innovation.

What is the compensation or salary for this position?

This information is not specified in the job description.

Is this a remote position, or what is the location policy?

This information is not specified in the job description.

What experience makes a candidate stand out for this role?

Candidates stand out with experience managing teams that shipped AI products using LLMs or multimodal models, hands-on expertise in deploying AI models with TensorRT or Triton, strong MLOps/DevOps background, managing large-scale AI evaluations on HPC clusters, and deep knowledge of cloud infrastructure, Docker, and Kubernetes.

NVIDIA

Designs GPUs and AI computing solutions

About NVIDIA

NVIDIA designs and manufactures graphics processing units (GPUs) and system on a chip units (SoCs) for various markets, including gaming, professional visualization, data centers, and automotive. Their products include GPUs tailored for gaming and professional use, as well as platforms for artificial intelligence (AI) and high-performance computing (HPC) that cater to developers, data scientists, and IT administrators. NVIDIA generates revenue through the sale of hardware, software solutions, and cloud-based services, such as NVIDIA CloudXR and NGC, which enhance experiences in AI, machine learning, and computer vision. What sets NVIDIA apart from competitors is its strong focus on research and development, allowing it to maintain a leadership position in a competitive market. The company's goal is to drive innovation and provide advanced solutions that meet the needs of a diverse clientele, including gamers, researchers, and enterprises.

Santa Clara, CaliforniaHeadquarters

1993Year Founded

$19.5MTotal Funding

IPOCompany Stage

Automotive & Transportation, Enterprise Software, AI & Machine Learning, GamingIndustries

10,001+Employees

Benefits

Company Equity

401(k) Company Match

Risks

Increased competition from AI startups like xAI could challenge NVIDIA's market position.

Serve Robotics' expansion may divert resources from NVIDIA's core GPU and AI businesses.

Integration of VinBrain may pose challenges and distract from NVIDIA's primary operations.

Differentiation

NVIDIA leads in AI and HPC solutions with cutting-edge GPU technology.

The company excels in diverse markets, including gaming, data centers, and autonomous vehicles.

NVIDIA's cloud services, like CloudXR, offer scalable solutions for AI and machine learning.

Upsides

Acquisition of VinBrain enhances NVIDIA's AI capabilities in the healthcare sector.

Investment in Nebius Group boosts NVIDIA's AI infrastructure and cloud platform offerings.

Serve Robotics' expansion, backed by NVIDIA, highlights growth in autonomous delivery services.

Land your dream remote job 3x faster with AI

Try Jobo Free