Senior/Lead Machine Learning Engineer, Serving

About the Company

Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models. Their technology powers experiences for companies like NVIDIA, Microsoft Xbox, and Niantic, serving hundreds of millions of end users across various AI applications.

Responsibilities

• Take models from research teams, containerize them, and optimize serving for production
• Optimize realtime inference and ensure high reliability and low latency
• Design benchmarks and prototypes to solve unclear technical problems
• Own the full cycle of model deployment and performance optimization

Requirements

• Deep understanding of serving frameworks like vLLM or TRT-LLM
• Experience with model acceleration techniques including quantization, distillation, and speculative decoding
• Proficiency in C++, CUDA, Rust, or highly optimized Python
• Experience with distributed systems, Kubernetes, Ray, and multi-GPU/multi-node inference
• PhD in CS, Physics, Math, or equivalent practical experience in ML systems
• Professional fluency in English

Preferred Qualifications

• Non-trivial systems programming projects or open-source contributions to inference engines
• Deep technical write-ups on systems or ML performance
• Experience handling thousands of concurrent connections in production environments