Member of Technical Staff, Inference

About the Company

Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware.

Responsibilities

• Push the boundaries of LLM and diffusion model serving
• Optimize how models execute across diverse hardware and architectures
• Work at the core of vLLM to innovate on the inference engine
• Implement model architectures and inference techniques from research papers
• Contribute performant and maintainable code to complex ML codebases

Requirements

• Bachelor's degree in computer science, engineering, or equivalent experience
• Deep understanding of transformer architectures and their variants
• Strong programming skills in Python with experience in PyTorch internals
• Experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, or TGI
• Ability to debug in complex ML codebases

Preferred Qualifications

• Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving
• Familiarity with RL frameworks and algorithms for LLMs
• Experience with multimodal inference (audio/image/video/text)
• Contributions to open-source ML or system infrastructure projects
• Experience implementing core features in vLLM or contributing to vLLM integrations