AI Engineer

Responsibilities

• Build new AI features end to end, from prototype to production
• Improve AI output quality through prompt engineering, model selection, retrieval, and evaluation
• Design and run evals that measure real output quality
• Iterate fast on prompts, agent designs, and orchestration patterns
• Partner with Product and AI Platform teams to translate requirements into working features
• Evaluate new models, tools, and techniques for quality, latency, cost, or reliability

Requirements

• Hands-on experience building LLM-powered features shipped to real users
• Production engineering experience in TypeScript/Node or Python
• Experience with LLM providers like Anthropic, OpenAI, Google Vertex, or AWS Bedrock
• Practical judgment in prompt engineering, retrieval, and agent design
• Track record of building evaluation systems to catch regressions
• Solid software engineering fundamentals for writing production code

Preferred Qualifications

• Experience with provider-abstraction libraries for multi-LLM workflows
• Familiarity with pgvector or other vector retrieval systems
• Experience with post-training or fine-tuning
• Experience deploying AI features on AWS Lambda or ECS Fargate
• Background in ML, NLP, or applied research
• Experience with structured output, function calling, and tool use at scale
• Experience with Anyscale Ray or similar distributed compute frameworks
• Open source contributions in the LLM or agent tooling space

About the Company

Fluency builds a platform that captures how work actually happens inside large organizations, measures productivity and process conformance, and analyzes where AI can do the work. We capture observable work data across tools and systems to help Fortune 500 customers like CVS Health, Aon, and PVH.