Senior Data Scientist- AI Evaluation at Remitly

Netherlands

Apply Now

Not SpecifiedCompensation

Senior (5 to 8 years)Experience Level

Full TimeJob Type

UnknownVisa

Technology, AIIndustries

Requirements

Education/Experience: Master’s + 3 years, or Bachelor’s + 5 years, in CS, Data Science, Statistics, Computational Linguistics, or related field; strong track record shipping evaluation or ML analytics work
Technical: Strong Python and SQL; experience with LLM/NLP evaluation, data/versioning, testing/CI, and cloud-based workflows; familiarity with prompt/rubric design and LLM-as-judge patterns
Statistics: Comfortable with power analysis, CIs, hypothesis testing, inter-rater reliability, and error/slice analysis
Practices: Git, code reviews, reproducibility, documentation; ability to turn ambiguous product needs into executable study plans
Communication: Clear written/oral communication; ability to produce crisp dashboards and decision-ready summaries for non-technical stakeholders
Mindset: Ownership, curiosity, bias-for-action, and collaborative ways of working

Responsibilities

Study design & metrics — Translate product questions into hypotheses, tasks/rubrics, datasets, and success criteria; define metrics (accuracy/correctness, groundedness, reliability, safety/bias/toxicity) with acceptance thresholds
Pipelines & tooling — Build and maintain Python/SQL evaluation pipelines (data prep, prompt/rubric generation, LLM-as-judge with guardrails, scoring, QC, reporting); contribute to shared packages and CI
Statistical rigor — Plan for power, confidence intervals, inter-rater reliability (e.g., Cohen’s κ/ICC), calibration, and significance testing; document assumptions and limitations
SME integration — Partner with SME Ops and domain leads to create clear rater guidance, run calibration, monitor IRR, and incorporate feedback loops
Analytics & reporting — Create analyses that highlight regressions, safety risks, and improvement opportunities; deliver crisp write-ups and executive-level summaries
Governance & compliance — Produce audit-ready artifacts (evaluation plans, datasheets/model cards, risk logs); follow privacy/security guardrails and Responsible AI practices
Quality & reliability — Implement test hygiene (dataset/versioning, golden sets, seed control), observability, and failure analysis; help run post-release regression monitoring
Collaboration — Work closely with Product and Engineering to scope, estimate, and land evaluation work; participate in code reviews and design sessions alongside fellow Data Scientists

Skills

Key technologies and capabilities for this role

PythonSQLLLM EvaluationNLPStudy DesignMetrics DefinitionStatistical AnalysisCohen’s κICCInter-rater ReliabilityData PipelinesLLM-as-judgePrompt EngineeringResponsible AI

Questions & Answers

Common questions about this position

What education and experience are required for this Senior Data Scientist role?

A Master’s degree plus 3 years or Bachelor’s plus 5 years in CS, Data Science, Statistics, Computational Linguistics, or related field is required, along with a strong track record shipping evaluation or ML analytics work.

What technical skills are needed for this position?

Strong Python and SQL skills are required, plus experience with LLM/NLP evaluation, data/versioning, testing/CI, and cloud-based workflows, along with familiarity with prompting.

What does the team structure look like for this role?

The role is part of Elsevier’s AI Evaluation team, which designs, builds, and operates NLP/LLM evaluation solutions, partnering with Product, Technology, Domain SMEs, and Governance.

Is this a remote position or does it require office work?

This information is not specified in the job description.

What is the salary or compensation for this role?

This information is not specified in the job description.

Remitly

International money transfer for immigrants

About Remitly

Remitly focuses on international money transfers, helping immigrants send money to their families quickly and securely at lower costs than traditional banks. The company charges transaction fees and earns from the exchange rate margin, offering various transfer options like bank deposits and cash pickups. Remitly enhances user experience through its website and mobile app, allowing real-time tracking of transfers, and engages with immigrant communities by providing helpful resources and educational support. Its goal is to meet the unique needs of immigrants while ensuring fast, affordable, and reliable money transfer services.

Seattle, WashingtonHeadquarters

2011Year Founded

$423.9MTotal Funding

IPOCompany Stage

Fintech, Financial ServicesIndustries

1,001-5,000Employees

Benefits

Continuing Education or Travel Stipend

Office Culture

Flexible PTO, Schedules and Leaves

DEI Learning Opportunities

Community Engagement

Inclusive Benefits

Risks

Swift's integration into banks' channels poses a competitive threat to Remitly.

AI agents' rise may challenge Remitly's operational model in efficiency and cost-effectiveness.

Schall Law Firm's investigation could impact investor confidence and Remitly's reputation.

Differentiation

Remitly focuses on immigrants, offering competitive rates and fast, secure transactions.

The company leverages technology for a seamless user experience via its app and website.

Remitly engages with immigrant communities through content and educational initiatives.

Upsides

Increased digital wallet adoption in key markets expands Remitly's reach to unbanked individuals.

AI agents in operations could reduce costs and improve Remitly's efficiency.

Visa collaboration enhances Remitly's cross-border fund flow services, boosting customer satisfaction.

Land your dream remote job 3x faster with AI

Try Jobo Free