Software Engineer, Data Acquisition

Responsibilities

• Own and lead engineering projects in data acquisition, including web crawling, data ingestion, and search.
• Develop and deploy highly scalable distributed systems capable of handling petabytes of data.
• Architect and implement algorithms for data indexing and search capabilities.
• Build and maintain backend services for data storage, including work with key-value databases and synchronization.
• Deploy solutions in a Kubernetes Infrastructure-as-Code environment and perform routine system checks.
• Collaborate with Data Processing, Architecture, Scaling, and Legal teams to ensure smooth data flow and compliance.

Requirements

• BS/MS/PhD in Computer Science or a related field.
• 4+ years of industry experience in software development.
• Strong expertise in large stateful distributed systems and data processing.
• Proficiency in Kubernetes and Infrastructure-as-Code concepts.
• Strong written and verbal communication skills.

Preferred Qualifications

• Experience with large-scale web crawlers.
• Willingness and enthusiasm for exploring new approaches and technologies.
• Ability to handle multiple tasks and adapt to changing priorities in a fast-paced environment.

Benefits

• Competitive salary ($293K – $385K) and generous equity.
• Comprehensive medical, dental, and vision insurance.
• 401(k) retirement plan with employer match.
• Paid parental leave and flexible PTO.
• Daily meals in the office and meal delivery credits.
• Annual learning and development stipend.
• Relocation support for eligible employees.

About the Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of AI capabilities and seek to safely deploy them to the world through our products.