Posted 18 hours ago
Staff/Principal Machine Learning Engineer, Serving
InworldStaff/Principal Machine Learning Engineer, Serving
Requirements
Inference optimization (vLLM, TRT-LLM), Model acceleration (quantization, distillation), Proficiency in C++, CUDA, Rust, or optimized Python, Distributed systems (Kubernetes, Ray), PhD in CS, Physics, Math or equivalent experience
Skills
PythonCUDAKubernetesC#Machine Learning
About the role
About the Company
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models. Our technology powers experiences for companies like NVIDIA, Microsoft Xbox, and Niantic, representing hundreds of millions of end users.
Responsibilities
- Take models from research teams, containerize them, and optimize serving for production
- Optimize realtime inference and ensure high-performance model acceleration
- Design and implement scalable distributed systems for multi-GPU/multi-node inference
- Solve core latency and throughput problems through architectural improvements
- Ensure models run reliably in production while handling thousands of concurrent connections
Requirements
- Deep understanding of modern serving frameworks like vLLM or TRT-LLM
- Hands-on experience with quantization, distillation, and speculative decoding
- Proficiency in C++, CUDA, Rust, or highly optimized Python
- Experience with Kubernetes, Ray, and custom load balancing
- Ability to profile code and optimize performance on NVIDIA GPUs
- PhD in CS, Physics, Math, or equivalent practical experience building ML systems
- Legal right to work in the United Kingdom
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeStaff/Principal Machine Learning Engineer, Serving
Inworld · United Kingdom
