Posted 16 hours ago
Senior/Lead Machine Learning Engineer, Serving
InworldSenior/Lead Machine Learning Engineer, Serving
Requirements
Inference optimization (vLLM, TRT-LLM), Model acceleration (quantization, distillation), C++, CUDA, Rust, or optimized Python, Distributed systems (Kubernetes, Ray), PhD in CS, Physics, Math or equivalent
Skills
PythonCUDAKubernetes
About the role
About the Company
Inworld is a research lab of top researchers and engineers, building the world’s top-ranked realtime voice models. Their technology powers experiences for companies like NVIDIA, Microsoft Xbox, and Niantic, representing hundreds of millions of end users.
Responsibilities
- Take models from research teams, containerize them, and optimize serving
- Ensure models run reliably in production environments
- Optimize latency and throughput for realtime multimodal inference
- Design benchmarks and prototypes to solve unclear technical problems
Requirements
- Deep understanding of serving frameworks like vLLM or TRT-LLM
- Experience with quantization, distillation, and caching strategies
- Proficiency in C++, CUDA, Rust, or highly optimized Python
- Experience with Kubernetes, Ray, or multi-GPU/multi-node inference
- PhD in CS, Physics, Math, or equivalent practical experience
- Professional fluency in English
Preferred Qualifications
- Experience with continuous batching, paged attention, and speculative decoding
- Open-source contributions to major inference engines
- Non-trivial systems programming projects or technical write-ups
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeSenior/Lead Machine Learning Engineer, Serving
Inworld · Serbia
