Principal LLM Inference Engineer

Responsibilities

• Identify and prototype emerging LLM inference use cases suited to heterogeneous hardware deployments
• Build compelling proof-of-concept systems that demonstrate D-Matrix capabilities
• Develop and tune custom kernels and operator-level optimizations to maximize throughput and minimize latency
• Drive quantization, sparsity, and batching strategies tailored to D-Matrix computational model
• Build and maintain inference runtimes, serving frameworks, and evaluation tooling
• Contribute to distributed inference systems including tensor/pipeline parallelism and KV-cache management
• Work closely with hardware architects to provide actionable inference workload insights
• Partner with product and business development to translate POCs into customer-facing demonstrations

Requirements

• Bachelor’s degree in Computer Science, Electrical Engineering, or related field with 10+ years of engineering experience
• Master’s or PhD preferred with 6+ years of industry experience
• Strong proficiency in Python and C/C++
• Hands-on experience optimizing LLM inference (attention kernels, KV cache, quantization)
• Experience with major inference frameworks like vLLM, SGLang, or TensorRT-LLM
• Familiarity with GPU kernel programming (CUDA/Triton) and performance profiling tools

Preferred Qualifications

• Experience with heterogeneous compute deployments across dissimilar hardware
• Familiarity with custom silicon or ASIC-based inference
• Experience with distributed inference and production inference serving at scale
• Contributions to open-source inference or ML systems projects
• Familiarity with speculative decoding, mixture-of-experts, or long-context serving

About the Company

D-Matrix is focused on unleashing the potential of generative AI through software and hardware innovation. The D-Matrix Frontier Group works at the leading edge of LLM inference on heterogeneous hardware, building the tools, runtimes, and frameworks that enable frontier AI models to run efficiently across silicon, CPUs, and GPUs.