Member of Technical Staff - ML Infrastructure & Performance

Responsibilities

• Optimize GPU performance using CUDA/Triton kernels, FlashAttention, paged attention, and CUDA Graphs
• Manage the serving stack including TensorRT-LLM, Triton Inference Server, vLLM, and TGI
• Implement continuous batching, on-GPU KV reuse, speculative decoding, and mixture-of-agents routing
• Optimize parallelism using FSDP/ZeRO, TP/PP/expert parallel, and NCCL tuning
• Handle quantization and PEFT including AWQ, GPTQ, FP8, LoRA, and DoRA serving
• Maintain systems using Ray, Kubernetes, Argo, and observability tools like Prometheus and Grafana
• Manage autoscaling, A/B infrastructure, and canary/rollback deployments

Requirements

• Experience with GPU performance optimization and CUDA/Triton kernels
• Proficiency with LLM serving stacks like vLLM or TensorRT-LLM
• Knowledge of distributed training and parallelism techniques (FSDP, ZeRO, TP/PP)
• Experience with quantization methods (AWQ, GPTQ, FP8)
• Experience with orchestration and observability tools (Kubernetes, Ray, Prometheus)

About the Company

Moonlake is building AI for creating real-time interactive content, focused on improving throughput, latency, and cost by deploying models significantly faster and cheaper.