Senior Backend Engineer

Responsibilities

• Build high-throughput bulk ingestion workflows to integrate datasets from multiple external providers
• Design and implement scalable entity-resolution solutions, including record linking, deduplication, clustering, and conflict arbitration
• Create and refine matching logic, decision rules, and similarity functions to align datasets
• Define and track data quality indicators such as overlap metrics, match precision/recall, and completeness
• Prepare training-ready datasets in formats such as TFRecords
• Develop processing components using Dataflow (Beam) and manage large analytical workloads in BigQuery
• Leverage frameworks like Ray to accelerate large-scale experiments and research-oriented data preparation
• Collaborate with ML researchers to evolve linkage strategies

Requirements

• Experience working with large, heterogeneous datasets from multiple providers
• Strong background in entity resolution, deduplication, or data unification
• Proficiency in Python for scalable data processing
• Experience with BigQuery, Google Dataflow, or Apache Beam
• Familiarity with data validation, normalization, and reconciliation
• Ability to craft matching and decision strategies balancing accuracy and efficiency
• Strong communication skills for collaboration with ML teams

Preferred Qualifications

• Knowledge of architecting Google Cloud Platform systems at scale
• Experience with distributed compute frameworks like Ray, Spark, or Flink
• Understanding of JAX-based ML pipelines or multihost training setups
• Familiarity with TFRecords or high-volume training data formats
• Exposure to ranking, clustering, or statistical similarity modeling
• Experience with Go, NextJS, or React Native

Benefits

• Highly competitive salary and equity
• Quarterly productivity budget
• Flexible time off
• Fantastic office location in Manhattan
• Productivity package including ChatGPT Plus, Claude Code, and Copilot
• Top notch private health, dental, and vision insurance
• 401(k) plan options with employer matching
• Concierge medical/primary care through One Medical and Rightway
• Mental health support from Spring Health
• Personalized life insurance and travel assistance