Software Engineer, Model Inference

Responsibilities

• Optimize large-scale AI models for high-volume, low-latency, and high-availability production and research environments
• Work alongside ML researchers and product managers to bring latest technologies into production
• Introduce new techniques, tools, and architectures to improve performance, latency, throughput, and efficiency of the model inference stack
• Build tools to provide visibility into bottlenecks and instability, then design and implement solutions
• Optimize code and Azure VM fleets to maximize utilization of GPU RAM and FLOPs

Requirements

• 5+ years of professional software engineering experience
• Deep understanding of modern ML architectures and performance optimization for inference
• Experience architecting, building, observing, and debugging production distributed systems
• Familiarity with PyTorch and NVIDIA GPUs
• Experience with CUDA, NCCL, or InfiniBand

Preferred Qualifications

• Experience working on performance-critical distributed systems
• Experience refactoring production systems to handle rapid increases in scale
• Familiarity with HPC technologies such as MPI and NVLink

Benefits

• Competitive salary ($295K – $555K) and generous equity
• Medical, dental, and vision insurance for you and your family
• 401(k) retirement plan with employer match
• Paid parental leave and flexible PTO
• Daily meals in the office and meal delivery credits
• Annual learning and development stipend
• Mental health and wellness support

About the Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of AI capabilities and seek to safely deploy them to the world through our products.