Applied AI/ML Engineer

Responsibilities

• Design and maintain LLM prompt architecture, structured outputs, and orchestration systems
• Build evaluation infrastructure using tools like Langfuse, TensorZero, and Promptfoo
• Run structured experiments across models and configurations to optimize quality, cost, and latency
• Develop internal AI observability tools, dashboards, and debugging workflows
• Apply data science and lightweight modeling to improve matching and ranking systems
• Own the model serving layer, including deployment, inference infrastructure, and versioning
• Identify and implement fine-tuning opportunities using SFT, DPO, and RLHF
• Design vector databases and memory architectures for personalization
• Ensure AI systems are fair, unbiased, and auditable through human-in-the-loop processes

Requirements

• 5+ years of engineering experience
• Proven track record of building LLM-powered products in production
• Strong expertise in prompt engineering and evaluation frameworks
• Experience with vector databases and RAG architectures
• Proficiency with fine-tuning techniques (SFT, DPO, RLHF)
• Experience with Python, SQLAlchemy, FastAPI, and PostgreSQL
• Ability to thrive in high-ownership, early-stage environments

Preferred Qualifications

• Experience with recommender or ranking systems
• Knowledge of MLOps and model lifecycle infrastructure
• Experience building AI observability or evaluation frameworks
• Ability to build internal developer tools using React, TypeScript, and Python

About the Company

Overtone, led by the founder and former CEO of Hinge, is building a new kind of dating service designed to help users find meaningful connections by cutting through the noise.