Member of Technical Staff (Model Behavior Architect)

Responsibilities

• Design, test, and optimize context strategies and system prompts that shape answer engine behavior
• Build automated and semi-automated evaluation pipelines to measure model quality and catch regressions
• Partner with research and engineering to validate model behavior before and during rollouts
• Identify inconsistencies and failure modes in model outputs through research projects
• Work closely with design, product, and research teams to translate product goals into model behavior requirements
• Help engineers build intuition for prompt design, context engineering, and evaluation best practices
• Track the latest alignment, evaluation, and prompting techniques from industry and academia

Requirements

• 3+ years of experience working with LLMs in a product or research setting
• Experience designing evaluations, benchmarks, or metrics for AI systems
• Demonstrated experience with Python for prototyping, debugging, and automation
• Strong experience with Perplexity or other frontier AI models in production settings
• Strong written and verbal communication skills
• Ability to manage multiple concurrent projects in a fast-moving environment

Preferred Qualifications

• Experience with A/B testing or experimentation frameworks
• Track record of improving AI system performance through systematic evaluation and iteration