Senior Software Engineer, Data Infrastructure

About the Company

Arena is the platform for evaluating how AI models perform in the real world. Founded by researchers from UC Berkeley's SkyLab, we're on a mission to measure and advance the frontier of AI for real-world use.

Responsibilities

• Design and build robust data pipelines to ingest, process, and transform user vote data
• Collaborate with researchers and product leadership to understand product goals and necessary data
• Design and implement solutions to generate result dashboards and reports
• Ensure the integrity, data quality, and reliability of the pipelines
• Scale data infrastructure to accommodate increasing data volumes and evolving analytical needs

Requirements

• 5+ years of experience in software engineering with a focus on data engineering
• Proficiency in SQL
• Proficiency in Python, Scala, or R
• Hands-on experience with data processing frameworks like Apache Spark or Ray Data
• Experience with big data analytics platforms such as Databricks or Snowflake
• Demonstrated experience designing, implementing, optimizing, and debugging production data pipelines

Preferred Qualifications

• Prior work in data analytics or datalake platforms
• Experience with advanced data analysis tools like Delta lake or streaming tables
• Exposure to machine learning