Data Engineer

Responsibilities

• Design, build and manage people data pipelines, ensuring all data is seamlessly integrated into the Databricks warehouse
• Develop canonical datasets to track key people metrics and People Innovation Labs product metrics
• Work collaboratively with Data Platform, Data Science, People Analytics, and Compensation and Equity teams
• Implement robust and fault-tolerant systems for data ingestion and processing
• Participate in data architecture and engineering decisions as the primary data engineering expert on the team
• Ensure the security, integrity, and compliance of data according to industry and company standards

Requirements

• 3+ years of experience as a data engineer
• 8+ years of software engineering experience
• Proficiency in Python, Scala, or Java
• Experience with Databricks and Snowflake
• Expertise with ETL schedulers such as Fivetran, Airflow, Dagster, or Prefect
• Experience with distributed processing frameworks like Spark, Hadoop, or Flink
• Experience with distributed storage systems such as HDFS or S3

About the Company

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity.