Machine Learning Eval Engineer

About the Company

Reducto is the agentic document platform for leading AI teams who demand enterprise performance at scale. We provide a comprehensive toolkit for working with documents the way a human would, combining custom in-house and leading frontier models to power efficient and accurate document workflows.

Responsibilities

• Design, build, and maintain evaluation benchmarks to reveal model performance and failure modes
• Develop metrics, heuristics, and workflows to automatically identify failure modes across real-world datasets
• Partner with ML engineers to turn evaluation insights into model improvements and training priorities
• Work with unstructured enterprise data including PDFs and spreadsheets to uncover edge cases
• Build lightweight internal and user-facing tools using Python frameworks like Flask to analyze model behavior
• Collaborate with customers to create bespoke benchmarks highlighting platform strengths

Requirements

• Strong Python skills with ability to build clean, reliable technical solutions
• Experience working with data infrastructure such as AWS S3
• Experience with OLAP or analytics systems like Tinybird
• Ability to work in a fast-changing, high-growth environment
• High bar for quality and precision
• Comfortable working with unstructured data

Preferred Qualifications

• Experience at an early-stage or high-growth startup
• Background in product thinking and building user-facing interfaces
• Experience working directly with customers to understand workflows
• Experience in AI/ML, data infrastructure, or document understanding systems

Benefits

• Unlimited PTO
• Daily free lunch at the office
• Reimbursed transportation costs
• Generous health insurance (medical, dental, and vision)
• $150/mo health and wellness budget
• Parental leave