Lead Data Scientist

Responsibilities

• Own the design and evolution of evaluation frameworks for LLM-based systems, both offline and online
• Build and scale A/B testing to measure real customer impact and guide product decisions
• Define meaningful metrics that connect system performance to customer outcomes and business impact
• Identify failure modes across the assistant (reasoning, retrieval, tool use, tone) and drive improvements
• Work closely with engineers to iterate on agentic workflows and improve system behavior through prompt design
• Analyze conversation data to uncover new opportunities for improving the customer experience
• Collaborate daily with Product, Engineering, and Design teams to shape the assistant's behavior

Requirements

• Strong grounding in statistics with the ability to design and own A/B testing end-to-end
• Proven ability to evaluate models (classification, regression, or LLMs) and connect performance to business outcomes
• Experience writing production code in at least one of Python, TypeScript, or Java
• Familiarity with modern LLM/agentic architectures (e.g., Mastra, LangGraph or similar)
• Strong data skills including SQL, Snowflake, and data visualization
• Strong sense of ownership and a product-oriented mindset

Preferred Qualifications

• Experience with customer-facing agentic systems
• Experience with Bayesian inference, especially regarding A/B testing

About the Company

Wise is a global technology company building the best way to move and manage the world’s money. Our mission is to make money borderless—moving it with minimum fees, maximum ease, and full speed for everyone, everywhere.