Software Engineer, Data Infrastructure - Research at OpenAI

San Francisco, California, United States

Apply Now

$250,000 – $380,000Compensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

AI, Machine LearningIndustries

Requirements

Strong engineering fundamentals with experience in distributed systems, data pipelines, or infrastructure
Experience building APIs, modular code, and scalable abstractions, with a focus on users and UX in abstraction design
Comfortable debugging bottlenecks across large fleets of machines
Pride in building infrastructure that “just works,” with a focus on reliability and scale
Collaborative, humble, and excited to own a foundational part of the ML stack
(Bonus) Background knowledge in data math, probability, or distributed data theory
(Bonus) Experience with GPU-scale distributed systems or dataset scaling for real-time data

Responsibilities

Design and maintain standardized dataset APIs, including for multimodal (MM) data that cannot fit in memory
Build proactive testing and scale validation pipelines for dataset loading at GPU scale
Collaborate with teammates to integrate datasets seamlessly into training and inference pipelines, ensuring smooth adoption and a great user experience
Document and maintain dataset interfaces so they are discoverable, consistent, and easy for other teams to adopt
Establish safeguards and validation systems to ensure datasets remain reproducible and unchanged once standardized
Debug and resolve performance bottlenecks in distributed dataset loading (e.g., straggler systems slowing global training)
Provide visualization and inspection tools to surface errors, bugs, or bottlenecks in datasets

Skills

distributed systems

dataset APIs

GPU scaling

multimodal data

performance optimization

training pipelines

inference pipelines

proactive testing

debugging bottlenecks

visualization tools

pipeline integration

OpenAI

Develops safe and beneficial AI technologies

About OpenAI

OpenAI develops and deploys artificial intelligence technologies aimed at benefiting humanity. The company creates advanced AI models capable of performing various tasks, such as automating processes and enhancing creativity. OpenAI's products, like Sora, allow users to generate videos from text descriptions, showcasing the versatility of its AI applications. Unlike many competitors, OpenAI operates under a capped profit model, which limits the profits it can make and ensures that excess earnings are redistributed to maximize the social benefits of AI. This commitment to safety and ethical considerations is central to its mission of ensuring that artificial general intelligence (AGI) serves all of humanity.

San Francisco, CaliforniaHeadquarters

2015Year Founded

$18,433.2MTotal Funding

LATE_VCCompany Stage

AI & Machine LearningIndustries

1,001-5,000Employees