Machine Learning Performance Engineer

Responsibilities

• Optimize performance of ML models for both training and inference
• Improve efficient large-scale training, low-latency real-time inference, and high-throughput research inference
• Take a whole-systems approach spanning storage, networking, host, and GPU-level considerations
• Debug and optimize low-level CUDA and GPU utilization to ensure effective goodput

Requirements

• Understanding of modern ML techniques and toolsets
• Experience debugging a training run’s performance end to end
• Low-level GPU knowledge of PTX, SASS, warps, cooperative groups, Tensor Cores, and the memory hierarchy
• Debugging and optimisation experience using CUDA GDB, NSight Systems, and NSight Compute
• Library knowledge of Triton, CUTLASS, CUB, Thrust, cuDNN, and cuBLAS
• Intuition about latency and throughput of CUDA graph launch, tensor core arithmetic, warp-level synchronization, and asynchronous memory loads
• Background in Infiniband, RoCE, GPUDirect, PXN, rail optimisation, and NVLink for GPU clusters
• Understanding of collective algorithms for distributed GPU training in NCCL or MPI
• Fluency in English

About the Company

Jane Street is a quantitative trading firm where machine learning is a critical pillar of the global business. The ever-evolving trading environment serves as a unique, rapid-feedback platform for ML experimentation, allowing new ideas to be incorporated with relatively little friction.