Software Engineer, Networking - Inference at OpenAI

San Francisco, California, United States

Apply Now

$325,000 – $490,000Compensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

AI, TechnologyIndustries

Requirements

Deep experience designing and operating large-scale distributed systems, particularly load balancers, service gateways, or traffic routing layers
5+ years of experience designing in theory for and debugging in practice for the algorithmic and systems challenges of consistent hashing, sticky routing, and low-latency connection management
5+ years of experience as a software engineer and systems architect working on high-scale, high-reliability infrastructure
Strong debugging mindset and enjoy spending time in tracing, logs, and metrics to untangle distributed failures
Comfortable writing and reviewing production code in Rust or similar systems languages (C/C++, Java, Go, Zig, etc)
Operated in big tech or high-growth environments and excited to apply that experience in a faster-moving setting
Take ownership of problems end-to-end and excited to build something foundational

Responsibilities

Architect and build the gateway / network load balancer that fronts all research jobs, ensuring long-lived connections remain consistent and performant
Design traffic stickiness and routing strategies that optimize for both reliability and throughput
Instrument and debug complex distributed systems — with a focus on building world-class observability and debuggability tools (distributed tracing, logging, metrics)
Collaborate closely with researchers and ML engineers to understand how infrastructure decisions impact model performance and training dynamics
Own the end-to-end system lifecycle: from design and code to deploy, operate, and scale
Work in an outcome-oriented environment where everyone contributes across layers of the stack, from infra plumbing to performance tuning

Skills

Key technologies and capabilities for this role

Load BalancingDistributed SystemsNetworkingTraffic RoutingObservabilityDistributed TracingLoggingMetricsSystems ProgrammingPerformance Tuning

Questions & Answers

Common questions about this position

What is the salary range for this Software Engineer position?

The salary range is $325K - $490K.

Is this a remote position or does it require office work?

This information is not specified in the job description.

What skills and experience are required for this role?

Candidates need 5+ years designing and operating large-scale distributed systems like load balancers, 5+ years as a software engineer on high-scale infrastructure, deep experience with consistent hashing and sticky routing, a strong debugging mindset, and comfort with Rust or similar systems languages.

What is the team culture like at OpenAI's Inference team?

The team works in an outcome-oriented environment where everyone contributes across layers of the stack, from infra plumbing to performance tuning, and collaborates closely with researchers and ML engineers.

What makes a strong candidate for this Software Engineer role?

Strong candidates take ownership end-to-end, have experience in big tech or high-growth environments, and thrive building foundational systems with a focus on reliability and performance.

OpenAI

Develops safe and beneficial AI technologies

About OpenAI

OpenAI develops and deploys artificial intelligence technologies aimed at benefiting humanity. The company creates advanced AI models capable of performing various tasks, such as automating processes and enhancing creativity. OpenAI's products, like Sora, allow users to generate videos from text descriptions, showcasing the versatility of its AI applications. Unlike many competitors, OpenAI operates under a capped profit model, which limits the profits it can make and ensures that excess earnings are redistributed to maximize the social benefits of AI. This commitment to safety and ethical considerations is central to its mission of ensuring that artificial general intelligence (AGI) serves all of humanity.

San Francisco, CaliforniaHeadquarters

2015Year Founded

$18,433.2MTotal Funding

LATE_VCCompany Stage

AI & Machine LearningIndustries

1,001-5,000Employees

Benefits

Health insurance

Dental and vision insurance

Flexible spending account for healthcare and dependent care

Mental healthcare service

Fertility treatment coverage

401(k) with generous matching

20-week paid parental leave

Life insurance (complimentary)

AD&D insurance (complimentary)

Short-term/long-term disability insurance (complimentary)

Optional buy-up life insurance

Flexible work hours and unlimited paid time off (we encourage 4+ weeks per year)

Annual learning & development stipend

Regular team happy hours and outings

Daily catered lunch and dinner

Travel to domestic conferences

Risks

Elon Musk's legal battle may pose financial and reputational challenges for OpenAI.

Customizable ChatGPT personas could lead to privacy and ethical concerns.

Competitors like Anthropic raising capital may intensify market competition.

Differentiation

OpenAI's capped profit model prioritizes ethical AI development over unlimited profit.

OpenAI's AI models, like Sora, offer unique video creation from text descriptions.

OpenAI's focus on AGI aims to create AI systems smarter than humans.

Upsides

OpenAI's $6.6 billion funding boosts its AI research and computational capacity.

Customizable ChatGPT personas enhance user engagement and satisfaction.

OpenAI's 'Operator' AI agent could revolutionize workforce automation by 2025.

Land your dream remote job 3x faster with AI

Try Jobo Free