Senior AI Engineer

About the Company

Metaforms is redefining how market research gets done. Our AI platform empowers the world's leading research agencies to scale 10x faster while maintaining uncompromising quality using AI Agents that augment survey programming, data processing, and project management.

Responsibilities

• Design and own synthetic data generation pipelines to improve model performance
• Develop evaluation datasets for domain-specific languages (DSLs)
• Build and maintain automated data quality checks and large-scale research data pipelines
• Define structured rubrics for evaluating AI outputs on nuanced research tasks
• Build continuous monitoring, logging, and performance optimization systems for production models
• Lead research and implementation of solutions for long-context handling and cascading error challenges
• Design and evaluate production-grade AI agent systems end-to-end
• Drive prompt engineering strategy and experimentation frameworks

Requirements

• 4+ years of engineering experience
• 2+ years focused on LLM systems or applied ML in production
• Deep experience with LLMs, synthetic data pipelines, and evaluation frameworks
• Strong Python skills
• Solid understanding of prompt engineering, model fine-tuning, and evaluation methodology
• Experience building and operating production AI systems
• Strong debugging skills for complex, non-deterministic system failures

Preferred Qualifications

• Experience with Go or TypeScript
• Experience with model monitoring, bias evaluation, or dataset management tooling
• Background in semantic parsing, DSLs, or structured output models
• Prior work on agentic systems, multi-step reasoning pipelines, or tool-use frameworks

Benefits

• Full family health insurance
• $1,000 USD annual learning and development budget
• Dedicated mentor and coaching support
• Free snacks and dinner at the office