Member of Technical Staff — Model Optimization and Inference

About the Company

Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence: a full-duplex audiovisual system that can listen, speak, react, interrupt, and respond like a real person.

Responsibilities

• Contribute to end-to-end inference optimization across LLMs, audio models, and diffusion-based components
• Implement and tune KV cache strategies for long-context conversations
• Work with and extend inference serving frameworks like vLLM, SGLang, or TensorRT-LLM
• Profile and benchmark end-to-end latency and throughput to eliminate bottlenecks
• Build internal profiling and testing infrastructure
• Accelerate diffusion model inference through distillation and custom kernel optimizations
• Apply quantization techniques (INT8, INT4, GPTQ, AWQ) to reduce memory footprint
• Partner with research and infrastructure teams to ensure optimized model shipping

Requirements

• BS, MS, or PhD in CS, ML, or a related field
• Strong fundamentals in LLM inference or ML systems (KV caching, memory layout, attention kernels)
• Exposure to inference serving frameworks (vLLM, SGLang, TensorRT-LLM)
• Strong Python and PyTorch skills
• Systematic approach to profiling and optimization
• Curiosity about diffusion inference, speculative decoding, or quantization

Preferred Qualifications

• Internship or research experience with LLM inference or model serving
• Contributions to open-source inference frameworks
• CUDA or Triton kernel development experience
• Publications in MLSys, model compression, or inference optimization
• Familiarity with multimodal or streaming inference architectures

Benefits

• HSA plan with ~$2,000 annual company contribution
• 15 days PTO plus public holidays and a year-end office closure
• Daily lunch, drinks, and snacks
• Commuter benefits
• Equity package