Backend Engineer, AI Observability

Salary: $125,000 - $225,000 annually (plus competitive equity package) Location Type: Not specified Employment Type: Not specified

The Opportunity

AI is rapidly transforming the world, from developing human-level intelligence to enhancing voice assistants and enabling large-scale genetic marker analysis. Arize AI is at the forefront of this revolution, providing the leading AI observability and evaluation platform that empowers AI engineers to build and deploy high-performing, reliable models. As the AI landscape evolves towards generative AI and agentic systems, Arize ensures teams have the essential tools to monitor, troubleshoot, and improve AI in production.

The Team

You will join our Backend Engineering team, responsible for building the highly scalable distributed services that power Arize’s ML observability platform. While Go is our primary language for these systems, the team also maintains services and tools in Python, Java, and TypeScript. We expect a high level of contribution from every individual, tackling challenges such as optimizing metric computation across billions of data points, designing next-generation OLAP database architectures, and researching/implementing advanced dimensionality reduction techniques. You will be a key player in driving product innovation, understanding how impactful engineering teams develop AI and LLM-powered applications, and building the tools to support them. Our product solutions span clean APIs for application instrumentation, interactive playgrounds for prompt engineering and agent development, and scaling real-time evaluation infrastructure.

What You’ll Do

• Write maintainable, scalable, and performant backend code primarily in Go, Java, and Python, with opportunities to work in TypeScript.
• Build high-volume and highly available analytics systems.
• Design and build APIs tailored to our customers’ Machine Learning and LLM workflows.
• Prototype, optimize, and maintain scalable backend services powering the Arize core platform.
• Extend and contribute back to open-source OLAP databases and distributed message queue frameworks.
• Develop and integrate collection tools for robust monitoring of ML and LLM pipelines.
• Research and implement cutting-edge visualization & dimensionality reduction algorithms in a distributed environment.
• Collaborate with product, design, and customer engineering teams to enhance and expand our product offerings.
• Contribute to the development of our in-house AI Agents.

What We’re Looking For

• 5+ years of experience working with high-performance backend systems.
• Strong experience writing Go, Python, TypeScript/Node, Java, or similar server programming languages.
• Enthusiasm and interest in the AI and LLM ecosystem, with a desire to learn and stay updated on emerging technologies.
• Previous work building and operating highly complex SaaS platforms/systems.
• Knowledge of working with public clouds & container orchestration (AWS, GCP, Azure, Kubernetes, etc.).

Bonus Points, But Not Required

• Experience with distributed stream processing (Kafka, Gazette, or similar).
• Experience with OLAP systems.
• Familiarity with system observability tooling like Prometheus.
• Working knowledge of Machine Learning and/or Data Science.
• First-hand experience working with large language models (LLMs) or developing AI products.

Company Information

Arize AI is the leading AI observability and evaluation platform.

Note: Actual compensation is determined based on a variety of job-related factors, including transferable work experience, skill sets, and qualifications. Total compensation includes a comprehensive benefits package, including medical, dental, vision, a 401(k) plan, and unlimited paid time off, a generous parental leave policy.