Software Engineer, Inference - Performance Optimization

Responsibilities

• Build and refine performance models that translate microbenchmark results into cost-to-serve estimates
• Analyze inference workloads end to end across applications, models, and fleet infrastructure
• Enhance tooling to identify bottlenecks across layers for latency and throughput
• Partner with other teams to turn performance insights into concrete improvements and project how future changes affect inference

Requirements

• Deep expertise with performance profiling, benchmarking, analysis, and optimization
• Ability to reason from first principles about distributed systems, model inference, and hardware efficiency
• Comfort working across abstraction layers from application behavior to kernels, accelerators, networking, and fleet scheduling

About the Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.