Software Engineer, Data Infrastructure

Responsibilities

• Design and maintain standardized dataset APIs, including for multimodal data that cannot fit in memory
• Build proactive testing and scale validation pipelines for dataset loading at GPU scale
• Collaborate with teammates to integrate datasets into training and inference pipelines
• Document and maintain dataset interfaces for discoverability and consistency
• Establish safeguards and validation systems to ensure dataset reproducibility
• Debug and resolve performance bottlenecks in distributed dataset loading
• Provide visualization and inspection tools to surface errors or bottlenecks

Requirements

• Strong engineering fundamentals in distributed systems, data pipelines, or infrastructure
• Experience building APIs, modular code, and scalable abstractions
• Ability to debug bottlenecks across large fleets of machines
• Collaborative and humble mindset with a focus on reliability and scale

Preferred Qualifications

• Background knowledge in data math, probability, or distributed data theory
• Experience with GPU-scale distributed systems or dataset scaling for real-time data

About the Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.