Sr. SDET

Responsibilities

• Own end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification
• Design and build automation tooling and frameworks for AI/LLM-driven systems
• Develop and maintain evaluation frameworks to measure response quality, accuracy, and hallucination rates
• Drive automation coverage for critical AI workflows using deterministic and probabilistic validation
• Integrate AI quality checks into CI/CD pipelines
• Build tooling for LLM observability and debugging, including prompt tracing and response analysis
• Partner with Applied AI teams on prompt engineering and model selection
• Design and execute performance and load tests for AI services
• Identify and mitigate risks related to hallucinations, bias, and safety
• Define and track AI quality KPIs
• Mentor engineers and contribute to AI quality engineering practices

Requirements

• 5+ years of experience in software engineering or SDET roles
• Strong programming skills in Python, Java, or JavaScript
• Experience testing distributed, cloud-native SaaS systems and APIs
• Proficiency in coding with AI agents to accelerate development
• Hands-on exposure to LLMs or AI/ML systems (OpenAI, Claude, Gemini, etc.)
• Understanding of non-deterministic systems and probabilistic testing
• Experience building test frameworks and scalable automation systems
• Familiarity with AI evaluation techniques like benchmarking and golden datasets
• Experience with CI/CD pipelines such as Jenkins or GitHub Actions
• Bachelor’s degree in Computer Science or equivalent practical experience

About the Company

Dialpad is the AI-native business communications platform. We unify calling, messaging, meetings, and contact center on a single platform, powered by AI that understands every conversation in real time.