Inference Optimization ML Engineer

About the Company

Rhoda AI is building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots.

Responsibilities

• Own inference performance end-to-end, diagnosing and improving latency, throughput, and efficiency of large foundation models
• Build systematic performance attribution, including latency decomposition and bottleneck identification
• Apply optimization techniques such as quantization, pruning, distillation, operator fusion, and model compilation
• Optimize attention mechanisms, KV caching, and memory layouts for large multimodal models
• Work with kernel-level tooling like CUDA and Triton to implement or tune custom kernels
• Build benchmarking and regression detection infrastructure for latency baselines and throughput curves
• Collaborate with research engineers to translate model innovations into optimized implementations

Requirements

• 3+ years of experience in inference optimization, ML systems, or a related field
• Deep hands-on experience with PyTorch
• Strong understanding of compute, memory bandwidth, and I/O bottlenecks in large model inference
• Experience with quantization (INT8/FP8/AWQ), distillation, pruning, and compilation
• Familiarity with inference serving frameworks such as Triton, TensorRT, vLLM, or TorchServe
• Exceptional debugging and measurement ability

Preferred Qualifications

• GPU kernel or compiler-level experience (CUDA, Triton, graph capture, operator fusion)
• Experience with multimodal or video model inference
• Familiarity with edge/cloud hybrid deployment patterns and on-robot inference constraints
• Experience with speculative decoding, continuous batching, or other LLM serving optimizations
• Background in streaming or low-latency systems relevant to real-time robot control