Software Engineer, Inference

Responsibilities

• Own bring-up, correctness and performance of the OpenAI inference stack on AMD hardware
• Integrate internal model-serving infrastructure (e.g., vLLM, Triton) into a variety of GPU-backed systems
• Debug and optimize distributed inference workloads across memory, network, and compute layers
• Validate correctness, performance, and scalability of model execution on large GPU clusters
• Collaborate with partner teams to design and optimize high-performance GPU kernels using HIP, Triton, or other frameworks
• Build, integrate and tune collective communication libraries like RCCL to parallelize model execution

Requirements

• Experience writing or porting GPU kernels using HIP, CUDA, or Triton
• Familiarity with communication libraries like NCCL/RCCL
• Experience with distributed inference systems and scaling models across accelerator fleets
• Ability to solve end-to-end performance challenges across hardware and orchestration layers

Preferred Qualifications

• Contributions to open-source libraries like RCCL, Triton, or vLLM
• Experience with GPU performance tools such as Nsight, rocprof, or perf
• Prior experience deploying inference on non-NVIDIA GPU environments
• Knowledge of model/tensor parallelism, mixed precision, and serving 10B+ parameter models

About the Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.