Member of Technical Staff, Evals & Post-Training Product

Responsibilities

• Build internal eval workflows to design and scale evaluation tooling used by internal teams
• Own fine-tuning product experiences including user-facing workflows for SFT, RFT, and model-improvement capabilities
• Work closely with users to partner with customers and internal stakeholders to understand evaluation and fine-tuning needs
• Convert bespoke workflows into productized solutions and triage issues

Requirements

• 1 - 7 years of software engineering experience
• Hands-on experience with LLM evaluations and/or post-training methods
• Product engineering skills with ability to work across backend systems and developer-facing surfaces
• Understanding of the GenAI lifecycle including prompting, dataset curation, fine-tuning, and productionizing agents
• User-centric mindset with willingness to triage GitHub issues and build products from scratch

Preferred Qualifications

• 3+ years of software engineering experience
• Domain-specific evaluation experience (e.g., medical, legal, coding)
• Prior contributions to open source developer tools or AI/ML repositories
• Knowledge of inference and hardware, including GPU constraints and optimization techniques
• Experience in fast-paced startup environments

About the Company

Fireworks is building the future of generative AI infrastructure, delivering high-quality models with industry-leading inference speed. A Series C company valued at $4 billion, Fireworks is backed by top investors including Benchmark and Sequoia.