Engineering Manager, Evals

About the Company

Our mission is to automate coding. The first step in our journey is to build the best tool for professional programmers, using a combination of inventive research, design, and engineering. Our organization is very flat, and our team is small and talent dense.

Responsibilities

• Set the eval roadmap end-to-end, determining what to measure and how signals influence shipping and training decisions
• Lead and grow a high-impact team of engineers and researchers building eval datasets and developer-friendly tools
• Guide the evolution of CursorBench to reflect real developer workflows and expand it with new evaluation properties
• Define crisp online quality signals and turn regressions into robust guardrails
• Integrate evaluations into decision-making cadences for launches, deploys, and model training loops

Requirements

• Experience leading engineering teams shipping production systems
• Strong people leadership and coaching skills
• Ability to align research, product, data, and infrastructure on quality metrics and processes
• Strong data acumen and ability to collaborate with data scientists and researchers
• Experience building and operating evaluation or measurement systems (e.g., AI evals, experimentation platforms, ranking/relevance, or reliability instrumentation)
• Strong opinions on model and agent behaviors and awareness of emerging research trends