Senior Data Engineering Lead

Responsibilities

• Design and deliver core data platform capabilities including lakehouse, batch, streaming, and observability
• Build and operate scalable batch and streaming pipelines on Snowflake and/or Databricks
• Implement and optimize Apache Iceberg tables for consistent performance across compute engines
• Apply engineering practices including CI/CD, Infrastructure as Code (Terraform/CloudFormation), and automated testing
• Partner with Product, Analytics, and ML teams to implement data semantics and product contracts
• Optimize pipeline and query performance and cost through workload tuning and lifecycle management
• Own operational stability by troubleshooting issues, supporting backfills, and participating in incident response
• Provide technical leadership and mentoring for a small team or pod

Requirements

• 14+ years in software or data engineering
• BS/MS in Computer Science, Engineering, or equivalent experience
• Proficiency in Python and SQL
• Proven experience delivering Lakehouse solutions using Databricks and/or Snowflake on AWS
• Strong Spark development skills for both batch and streaming workloads
• Experience with Apache Iceberg and table/catalog governance
• Knowledge of production engineering practices including CI/CD, IaC, and observability

Preferred Qualifications

• Experience in financial services or asset management
• Experience with structured and alternative/unstructured data
• Familiarity with governance tooling such as Snowflake Horizon, Unity Catalog, Collibra, or Alation
• Experience using AI-assisted coding tools to maintain high code quality

About the Company

State Street is a global leader that helps institutional investors manage risk, respond to challenges, and drive performance and profitability. We are committed to fostering an inclusive environment where employees are empowered to reach their full potential through development opportunities and vibrant employee networks.