Member of Technical Staff, Model Serving

Position Overview

• Location Type: Remote
• Employment Type: Full-Time
• Salary: (Salary not provided in the description)

Cohere’s mission is to scale intelligence to serve humanity. They are training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. They believe their work is instrumental to the widespread adoption of AI. They obsess over what they build, with each person responsible for contributing to increasing the capabilities of their models and the value they drive for their customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft.

Why this Role?

• The Model Serving team is responsible for developing, deploying, and operating the AI platform delivering Cohere’s large language models through easy-to-use API endpoints.
• This role involves working closely with many teams to serve optimized LLM models to production in low latency, high throughput, and high availability environments.
• Opportunity to interface with customers and create customized deployments to meet their specific needs.

Requirements

• Experience with serving ML models in production
• Experience designing, implementing, and maintaining a production service at scale
• Strong intuition for system behavior and resource estimation under different workloads
• Familiarity with inference characteristics of deep learning models, specifically, Transformer based architectures
• Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or Inferentia), especially how they improve inference latency and throughput
• Strong understanding or working experience with distributed systems
• Experience in performance benchmarking, profiling, and optimization
• Experience with cloud infrastructure (e.g., AWS, GCP)
• Experience in Golang (or, other languages designed for high-performance scalable servers)

Responsibilities

• Develop and deploy machine learning serving infrastructure.
• Optimize LLM models for performance and scalability.
• Maintain and operate production AI platforms.
• Collaborate with various teams to integrate LLM models into customer applications.
• Contribute to the design and implementation of new features and improvements.

Company Information

• Mission: To scale intelligence to serve humanity.
• Focus: Training and deploying frontier models for developers and enterprises.
• Values: Obsession over building, working hard, moving fast, and prioritizing customer value.
• Diversity & Inclusion: Cohere values and celebrates diversity and strives to create an inclusive work environment for all. They welcome applicants from all backgrounds and are committed to providing equal opportunities.