Member of Technical Staff (AI Inference Engineer)

Responsibilities

• Support transformer-based retrieval, text-generation, and multimodal models in inference infrastructure
• Port in-house CUDA kernels to NVIDIA's CuTe DSL for GB200 and future hardware
• Develop internal Rust-based inference server to handle growing traffic
• Profile and fix performance bottlenecks from network ingress to GPU kernels
• Build reliability dashboards, alerts, and automated remediation systems

Requirements

• 3+ years of professional software engineering experience in ML inference or high-performance systems
• Deep experience with GPU programming (CUDA, Triton, CUTLASS, or similar)
• Experience building and operating production distributed systems under real load
• Proficiency in Rust, Python, and CUDA/CuTe DSL
• Understanding of modern LLM architectures and inference optimization techniques
• Understanding of GPU architectures including memory hierarchy and tensor cores
• Familiarity with deep learning frameworks like PyTorch, JAX, or TensorFlow

Preferred Qualifications

• Knowledge of ML compilers and PyTorch internals (torch.compile, custom operators)
• Experience with distributed GPU communication (NCCL, NVLink, InfiniBand, RDMA)
• Expertise in low-precision inference (INT8/FP8/FP4 quantization)
• Proficiency with profiling tools like Nsight Compute/Systems or CUDA-GDB
• Experience with Kubernetes and GPU scheduling

About the Company

Perplexity builds and runs the inference engine behind every Perplexity query, deploying dozens of model architectures at scale with tight latency and cost budgets.