Member of Technical Staff (AI Inference Engineer)

Responsibilities

• Support transformer-based retrieval, text-generation, and multimodal models in inference infrastructure
• Port in-house CUDA kernels to NVIDIA's CuTe DSL for GB200 and future hardware
• Develop internal Rust-based inference servers to handle growing traffic
• Profile and fix performance bottlenecks from network ingress to GPU kernel interleaving
• Build reliability and observability tools including dashboards, alerts, and automated remediation

Requirements

• 3+ years of professional software engineering experience in ML inference or high-performance systems
• Deep experience with GPU programming (CUDA, Triton, CUTLASS, or similar)
• Experience building and operating production distributed systems under real load
• Proficiency across Rust, Python, and CUDA/CuTe DSL
• Strong understanding of modern LLM architectures and inference optimization techniques
• Understanding of GPU architectures including memory hierarchy and tensor cores
• Familiarity with deep learning frameworks like PyTorch, JAX, or TensorFlow

Preferred Qualifications

• Experience with ML compilers and framework internals (PyTorch internals, torch.compile)
• Knowledge of distributed GPU communication (NCCL, NVLink, InfiniBand, RDMA)
• Experience with low-precision inference (INT8/FP8/FP4 quantization)
• Proficiency with profiling tools like Nsight Compute/Systems or CUDA-GDB
• Experience with Kubernetes and GPU scheduling