P
Posted 10 hours ago
Member of Technical Staff (Software Engineer, Data Platform)
PerplexityMember of Technical Staff (Software Engineer, Data Platform)
Requirements
5-8+ years software engineering experience, Production data infrastructure experience, Batch and streaming data processing, Python proficiency, Data orchestration expertise
Skills
PythonKafkaAirflowDatabricksSnowflake
About the role
Responsibilities
- Design and operate large-scale batch and streaming data pipelines for product features, AI training, and analytics
- Build event-driven and streaming systems for real-time ingestion and transformation
- Lead the architecture of data orchestration using tools like Airflow or Dagster
- Set guarantees for data correctness, freshness, lineage, and recoverability
- Build self-serve data platforms for engineers, data scientists, and analysts
- Improve developer experience through better abstractions and standards for data modeling and deployment
- Drive architectural decisions across storage, compute, orchestration, and data APIs
- Mentor engineers and review designs to raise the technical bar for data infrastructure
Requirements
- 5+ years (Senior) or 8+ years (Staff) of software engineering experience
- Strong experience building production data infrastructure systems
- Hands-on experience with batch and/or streaming data processing at scale
- Deep familiarity with data orchestration systems like Airflow or Dagster
- Proficiency in Python and at least one additional backend language (Go, TypeScript, etc.)
- Strong systems thinking regarding reliability, latency, cost, and complexity
- Experience supporting ML/AI workflows, training pipelines, or evaluation systems
- Familiarity with data quality, lineage, observability, and governance tooling
- Prior ownership of internal platforms used by multiple teams
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeMember of Technical Staff (Software Engineer, Data Platform)
Perplexity · San Francisco
