Voice AI Evaluation Lead

Remote

Apply with AI Apply

$135,000 – $165,000Compensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Voice AI, Speech Recognition, Artificial Intelligence, AI & Machine LearningIndustries

Requirements

Candidates should have experience designing, executing, and iterating on evaluation pipelines for ML models, proficiency in Python and data analysis libraries, and the ability to develop automated evaluation systems. It’s important to have comfort working with large-scale datasets and crafting meaningful performance metrics and visualizations, as well as experience using LLMs or internal tooling to accelerate analysis, QA, or pipeline prototyping. Strong communication skills, particularly when translating raw data into structured insights, documentation, or dashboards, and proven success working cross-functionally with research, engineering, QA, and product teams are required.

Responsibilities

The Voice AI Evaluation Lead will build and maintain scalable benchmarking pipelines for model evaluations across STT, TTS, and voice agent use cases, run regular evaluations of production and pre-release models on curated datasets, partner with Research, Data, and Engineering teams to develop new evaluation methodologies, design, define and refine evaluation metrics, author comprehensive model cards and internal reports, collaborate with Data Labeling Ops to source and annotate datasets, integrate model tests into CI/CD workflows, support Marketing and Product with data-backed comparisons, track market developments, and support GTM teams with benchmarking best practices.

Skills

Benchmarking pipelines

Model evaluation

Speech-to-text (STT)

Text-to-speech (TTS)

Voice AI

Data analysis

Evaluation metrics

Model cards

Cross-functional collaboration

Research methodologies

Deepgram

Speech recognition APIs for audio transcription

About Deepgram

Deepgram specializes in artificial intelligence for speech recognition, offering a set of APIs that developers can use to transcribe and understand audio content. Their technology allows clients, ranging from startups to large organizations like NASA, to process millions of audio minutes daily. Deepgram's APIs are designed to be fast, accurate, scalable, and cost-effective, making them suitable for businesses needing to handle large volumes of audio data. The company operates on a pay-per-use model, where clients are charged based on the amount of audio they transcribe, allowing Deepgram to grow its revenue alongside client usage. With a focus on the high-growth market of speech recognition, Deepgram is positioned for future success.

San Francisco, CaliforniaHeadquarters

2015Year Founded

$100.5MTotal Funding

SERIES_BCompany Stage

Data & Analytics, AI & Machine LearningIndustries

51-200Employees

Benefits

Comprehensive Health Plans

FSA Health Matching up to $1,000

Work from Home Ergonomic Stipend

Healthy Food & Snacks in offices

Community Groups

Unlimited Vacation

Risks

Increased competition from open-source solutions like OpenAI's Whisper threatens market share.

Recent layoffs suggest potential financial instability or strategic restructuring challenges.

Integration of Poised may cause disruptions in service or product development.

Differentiation

Deepgram's APIs offer fast, accurate, and scalable speech recognition solutions.

The acquisition of Poised enhances Deepgram's real-time feedback capabilities in virtual meetings.

Aura API provides low-latency, human-like voice models for conversational AI agents.

Upsides

Strategic partnership with Clarifai accelerates AI application development and market expansion.

Aura API positions Deepgram to capitalize on real-time conversational voice AI trends.

Deepgram's technology is used by large enterprises like NASA, indicating strong market trust.

Land your dream remote job 3x faster with AI

Try Jobo Free