Member of Technical Staff: AI Performance

Philippines

Apply with AI Apply

Not SpecifiedCompensation

Senior (5 to 8 years), Expert & Leadership (9+ years)Experience Level

Full TimeJob Type

UnknownVisa

SaaS, Artificial Intelligence, Customer Relationship Management (CRM)Industries

Requirements

Candidates should have 3-7 years of experience in systems, infra, or performance engineering roles with strong ownership of metrics and benchmarking. Fluency in Python and comfort working across full-stack and backend services is required. Experience building or using LLMs, vector-based search, or agentic frameworks in production environments is necessary, along with familiarity with LLM model serving infrastructure, model tuning workflows, and evaluation techniques in NLP, information retrieval, or human-centered AI. A deep appreciation for measuring what matters and strong product and user intuition are essential. Bonus points for experience contributing to academic or open-source benchmarking projects.

Responsibilities

The role involves designing and implementing agent evaluation pipelines to benchmark AI capabilities across real-world enterprise use cases, including building domain-specific benchmarks for various SaaS verticals. Responsibilities include developing performance benchmarks for latency, safety, cost-efficiency, and user-perceived quality, as well as creating search- and retrieval-oriented benchmarks. The position requires partnering with AI and infra teams to instrument models with telemetry for outcome-based evaluation and driving human-in-the-loop and programmatic testing methodologies. Additionally, the role contributes to DevRev’s open evaluation tooling and benchmarking frameworks.

Skills

Performance Benchmarking

LLM

SaaS

Agent Evaluation

Latency Optimization

Cost Efficiency

Retrieval

Multilingual Query Handling

Human-in-the-loop testing

Programmatic Testing

DevRev

Custom AI solutions for business efficiency

About DevRev

DevRev.ai focuses on enhancing business operations through its main product, Turing GenAI, which combines data engineering, knowledge graph, and customizable language model technologies. This tool helps businesses connect various departments like support, product management, and software development, allowing for smoother communication and collaboration. Clients can customize the AI solutions to fit their specific needs, whether they are small businesses or large corporations. Unlike many competitors, DevRev.ai emphasizes a tailored approach, ensuring that each solution is designed with a deep understanding of the client's unique challenges. The goal of DevRev.ai is to help businesses improve their efficiency and streamline their operations using advanced AI technologies.

Palo Alto, CaliforniaHeadquarters

2020Year Founded

$146.7MTotal Funding

SERIES_ACompany Stage

Enterprise Software, AI & Machine LearningIndustries

501-1,000Employees

Benefits

Hybrid Work Options

Flexible Work Hours

Risks

Competition from established CRM and AI companies like Salesforce and Microsoft.

Rapid AI technology development may render DevRev's offerings obsolete without innovation.

Potential data privacy concerns and regulatory scrutiny could impact DevRev's operations.

Differentiation

DevRev integrates development and CRM, bridging gaps between developers and customers.

The company offers customizable LLMs, enhancing AI-driven CRM solutions for enterprises.

DevRev's Turing GenAI combines data engineering and knowledge graphs for streamlined operations.

Upsides

Increased demand for AI-driven CRM solutions boosts DevRev's market potential.

Rising adoption of customizable LLMs expands opportunities for DevRev's Turing GenAI.

Recent $100M Series A funding indicates strong market confidence in DevRev.

Land your dream remote job 3x faster with AI

Try Jobo Free