Member of Technical Staff (Data Scientist, Evals)

Responsibilities

• Architect and maintain automated evaluation pipelines to assess answer quality across Perplexity's products
• Design evaluation sets and methods to measure the impact of tool calls and web search retrieval
• Develop VLM-based solutions to programmatically evaluate visual rendering of answers
• Review public benchmarks and academic evaluations for applicability to Perplexity
• Collaborate with technical leadership to measure and improve Answer Quality

Requirements

• PhD or MS in a technical field or equivalent experience
• 4+ years of experience in data science or machine learning
• Strong proficiency in Python and SQL
• Experience with AWS and Databricks
• Comfortable with agentic coding workflows and AI-assisted development tools

Preferred Qualifications

• 1+ years of experience working with LLMs at scale (LLM-as-a-judge)
• Experience with customer-facing web products or consumer apps at scale
• Strong research background applying methods to real-world ML problems
• Experience defining metrics like factual consistency, hallucination rate, and retrieval precision