Data Engineer

Responsibilities

• Design, build and manage data pipelines to integrate user event data into the data warehouse
• Develop canonical datasets to track key product metrics including growth, engagement, and revenue
• Collaborate with Infrastructure, Data Science, Product, Marketing, Finance, and Research teams
• Implement robust and fault-tolerant systems for data ingestion and processing
• Participate in data architecture and engineering decisions
• Ensure data security, integrity, and compliance with industry standards

Requirements

• 3+ years of experience as a data engineer
• 8+ years of software engineering experience
• Proficiency in Python, Scala, or Java
• Experience with distributed processing technologies like Hadoop or Flink
• Experience with distributed storage systems such as HDFS or S3
• Expertise with ETL schedulers like Airflow, Dagster, or Prefect
• Solid understanding of Spark including writing, debugging, and optimizing code

About the Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.