Staff Software Engineer, Data Infrastructure

Responsibilities

• Architect and build systems that ingest, store, and serve massive volumes of real-time operational data
• Design and operate a high-throughput, real-time data integration platform across diverse customer environments
• Architect a scalable open table format layer for reliable data storage at petabyte scale
• Build and optimize distributed data processing pipelines with Apache Spark and adjacent streaming technologies
• Drive performance, reliability, and cost efficiency across the full data infrastructure stack
• Collaborate with platform and product engineering teams to define data contracts, schemas, and integration patterns
• Establish best practices, tooling, and patterns for data infrastructure

Requirements

• 8+ years of experience architecting and operating large-scale data infrastructure systems in production
• Deep expertise with open table formats, particularly Apache Iceberg
• Extensive hands-on experience with Apache Spark for batch and streaming data processing
• Strong background in real-time data integration and stream processing (Kafka, Flink, etc.)
• Solid experience with data pipeline orchestration using Airflow
• Strong software engineering fundamentals in Python and/or Scala
• Extensive experience with AWS or comparable cloud platforms and S3-based data lake architectures
• Experience with Kubernetes and containerized deployment
• Degree in Computer Science, Engineering, or equivalent practical experience
• Located in San Francisco, New York, or Washington DC and open to working in office

About the Company

Peregrine helps public safety organizations, governments, and private-sector institutions address society’s challenges with an AI-enabled platform that turns siloed data into operational intelligence.