ML Research Engineer

About the Company

Arcada Labs builds products used by millions of people to provide direct access to real human preference and judgment. We create the evolutionary pressure that pushes models toward what people actually want, collaborating with frontier labs like OpenAI, xAI, Meta, and Google DeepMind.

Responsibilities

• Design and run large-scale evaluations that measure how frontier models perform in real-world workflows
• Turn human preference votes and interaction traces into reliable signals about model capability, taste, reasoning, robustness, and agent behavior
• Develop ranking systems, analysis pipelines, and experimental methods for comparing models
• Identify where models fail and what those failures reveal about the next frontier of capability
• Work with engineers to turn research findings into user-facing products, leaderboards, and tools for frontier labs
• Contribute to internal research reports, external publications, and customer-facing analyses

Requirements

• Experience training, fine-tuning, or evaluating models (LLMs, reward models, preference models, or RLHF/DPO-style systems)
• Prior research experience, publications, open-source work, or hands-on work with frontier models
• Strong familiarity with modern AI systems, model evaluation, agentic workflows, and frontier model behavior
• Ability to turn vague real-world problems into concrete evaluation tasks and measurable systems
• Strong experimental judgment and statistical rigor
• Good taste for what matters in model behavior