Software Engineer - Model Performance
Baseten150,000 - 250,000 USD/year
- Full Time
- Mid-level (3 to 4 years), Senior (5 to 8 years)
Not SpecifiedCompensation
Junior (1 to 2 years)Experience Level
Full TimeJob Type
UnknownVisa
Artificial Intelligence, AI & Machine Learning, Software DevelopmentIndustries
Requirements
Candidates should possess experience with serving ML models in production, designing, implementing, and maintaining a production service at scale, and a strong intuition for system behavior and resource estimation under different workloads. Familiarity with inference characteristics of deep learning models, specifically Transformer based architectures, and computational characteristics of accelerators (GPUs, TPUs, and/or Inferentia) is required, along with experience in performance benchmarking, profiling, and optimization. Strong understanding or working experience with distributed systems and familiarity with cloud infrastructure (e.g., AWS, GCP) are also necessary, alongside proficiency in Golang (or other languages designed for high-performance scalable servers).
Responsibilities
As a Member of Technical Staff, Model Serving, you will be responsible for developing, deploying, and operating the AI platform delivering Cohere’s large language models through easy to use API endpoints, working closely with various teams to serve optimized LLM models to production in low latency, high throughput, and high availability environments. You will also have the opportunity to interface with customers and create customized deployments to meet their specific needs.
Skills
ML model serving
production service design
system behavior analysis
resource estimation
Transformer architectures
accelerator hardware (GPUs, TPUs, Inferentia)
distributed systems
performance benchmarking
profiling
optimization
cloud infrastructure (AWS, GCP)
Golang
Cohere
Provides NLP tools and LLMs via API
About Cohere
Cohere provides advanced Natural Language Processing (NLP) tools and Large Language Models (LLMs) through a user-friendly API. Their services cater to a wide range of clients, including businesses that want to improve their content generation, summarization, and search functions. Cohere's business model focuses on offering scalable and affordable generative AI tools, generating revenue by granting API access to pre-trained models that can handle tasks like text classification, sentiment analysis, and semantic search in multiple languages. The platform is customizable, enabling businesses to create smarter and faster solutions. With multilingual support, Cohere effectively addresses language barriers, making it suitable for international use.
Key Metrics
Toronto, CanadaHeadquarters
2019Year Founded
$914.4MTotal Funding
SERIES_DCompany Stage
AI & Machine LearningIndustries
501-1,000Employees