Software Engineer, ML Data Systems

About the Company

Our mission is to automate coding. The first step in our journey is to build the best tool for professional programmers, using a combination of inventive research, design, and engineering. Our organization is very flat, and our team is small and talent dense.

Responsibilities

• Own the full data infrastructure ladder: patch, redesign, ship, and operate systems
• Ensure privacy guarantees and data correctness for model improvement and experimentation
• Design and ship replacements for legacy pipelines to ensure properties like point-in-time consistency
• Define instrumentation requirements for new product surfaces
• Manage schema evolution and validation to prevent silent degradation across consumers
• Optimize storage costs through retention and compression strategies

Requirements

• Deep experience with Spark (Databricks or open-source)
• Production experience with Ray Data
• Hands-on ownership of large data pipelines and storage systems
• Ability to debug performance issues across client instrumentation, streaming, storage, and networking layers
• Strong data modeling and maintainability skills
• Proven judgment regarding when to patch versus rebuild systems

Preferred Qualifications

• Experience running or scaling ClickHouse
• Familiarity with dbt, Dagster, or similar orchestration and modeling tools