Engineering Manager, Agent Prompts & Evals

Responsibilities

• Lead and grow a team of prompt engineers and platform software engineers
• Own the product-side eval platform including frameworks, dashboards, and CI integrations
• Manage system prompt infrastructure including versioning, deployment, and rollback tooling
• Act as a steady hand and operational backstop during high-stakes model launches
• Build durable collaboration with other evals groups and align shared roadmaps
• Recruit, close, and retain senior engineering talent
• Shape team investment priorities across frontier eval development and automation
• Drive the team to measure complex behaviors like behavioral drift and prompt quality

Requirements

• 8+ years in software engineering
• 3+ years managing engineering teams
• Experience leading platform, infrastructure, or developer-tooling teams
• Proven track record of building scalable tooling and processes
• Technical depth to engage in system design and pipeline architecture reviews
• Ability to build and maintain relationships with partner organizations
• Experience recruiting and closing senior ICs

Preferred Qualifications

• Prior exposure to LLM evals or ML experimentation platforms
• Experience with A/B testing infrastructure or feature flagging
• Background in devtools, CI/CD platforms, or testing infrastructure at scale
• Interest in AI safety and alignment

About the Company

Anthropic is a public benefit corporation dedicated to creating reliable, interpretable, and steerable AI systems. We work as a cohesive team on large-scale research efforts to ensure AI is safe and beneficial for society.