Data Scientist

About the Company

Arena is the platform for evaluating how AI models perform in the real world. Founded by researchers from UC Berkeley's SkyLab, we're on a mission to measure and advance the frontier of AI for real-world use.

Responsibilities

• Design, implement, and analyze A/B tests, multi-armed bandits, and quasi-experimental methods
• Apply causal inference techniques to estimate treatment effects in non-randomized settings
• Develop and refine retention measurement frameworks including cohort and survival analysis
• Define and track core engagement metrics such as DAU, WAU, and MAU
• Build and maintain scalable data pipelines using PySpark, SQL, or big data tools
• Develop automated dashboards to monitor experiment performance and retention trends
• Partner with product, engineering, and marketing teams to translate business questions into data-driven analyses

Requirements

• 3+ years of experience in data science, analytics, or experimentation
• Strong background in statistics and causal inference
• Hands-on experience with SQL and Python (Pandas, NumPy, SciPy, StatsModels, Scikit-learn)
• Proficiency in experimentation tools
• Experience defining and analyzing retention metrics
• Familiarity with big data tools like PySpark or Hadoop

Preferred Qualifications

• Expertise in PySpark for large-scale data processing
• Experience with time-series forecasting, survival analysis, or uplift modeling
• Knowledge of ML for retention such as propensity models or clustering
• Experience with data visualization tools like Tableau, Looker, or Plotly
• Advanced degree (MS/PhD) in Statistics, Economics, Computer Science, or a quantitative field
• Experience with reinforcement learning or bandit algorithms
• Knowledge of MLOps or productionizing models using MLflow, Airflow, or Docker