Staff Applied Scientist - Agentic Interfaces

Responsibilities

• Own the evaluation strategy for Datadog's AI agent integrations, defining metrics for quality, cost, and trajectory-level success.
• Build evaluation datasets, golden traces, and regression harnesses to ensure quality before customer deployment.
• Drive measurable improvements in retrieval relevance, tool-selection accuracy, and context efficiency.
• Conduct applied research on agent-data interaction, including tool selection in large catalogs and hallucination control.
• Partner with internal agent teams (Bits SRE, Bits Assistant, Bits Dev Agent) to unify measurement substrates.
• Provide technical leadership through design reviews, mentorship, and external representation in the open agent ecosystem.

Requirements

• BS/MS/PhD in a scientific field or equivalent experience.
• 10+ years of relevant engineering or applied science experience, including technical leadership roles.
• Proven track record of leading ML or GenAI initiatives from research through to production.
• Significant experience with the evaluation, experimentation, or measurement of ML systems at scale.
• Strong product mindset with the ability to drive initiatives across cross-functional teams.
• Ability to thrive in ambiguity and make sound technical decisions in undefined spaces.

Benefits

• New hire stock equity (RSUs) and employee stock purchase plan (ESPP).
• Continuous professional development, product training, and career pathing.
• Inclusive company culture with giving programs and Community Guilds (employee resource groups).
• Competitive global benefits and Spring Health benefits for employees and dependents.

About the Company

Datadog is the leading observability and security platform for the AI era, providing businesses with unified visibility across the technology stack to manage complexity at scale. Trusted globally by Fortune 500 companies and high-growth AI leaders, Datadog enables businesses to move faster with clarity and confidence.