Data Engineer (Databricks), Assistant Vice President

Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark, Python, and Spark SQL
• Develop and optimize ETL/ELT workflows on Databricks using Delta Lake architecture
• Implement Lakehouse architecture (Bronze/Silver/Gold layers) for enterprise data platforms
• Manage Databricks Jobs, Workflows, and Notebooks for both batch and streaming workloads
• Design and optimize data lakehouse architectures and integrate data from SQL Server, Oracle, and other enterprise systems
• Ensure data quality through unit testing, validation frameworks, and automated checks
• Support data governance using Databricks Unity Catalog and AWS controls (IAM, KMS)
• Maintain CI/CD pipelines and automate deployment of Databricks assets

Requirements

• 8+ years of experience in Data Engineering or data platform development
• Strong hands-on experience with Databricks and Apache Spark
• Proficiency in PySpark, Python, and SQL
• Experience with AWS data platform services including S3, Glue, Lambda, and IAM
• Experience working with Delta Lake and lakehouse architecture
• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical discipline

Preferred Qualifications

• Hands-on experience with Databricks Unity Catalog and Workflows
• Experience building end-to-end data pipelines for analytics, reporting, or AI/ML use cases
• Familiarity with performance optimization techniques in Spark and Delta Lake
• Experience in Legal, Compliance, or Financial Services industries
• Exposure to unstructured data processing or NLP pipelines
• Experience with Power BI, Power Apps, or the Microsoft Power Platform

Benefits

• Retirement savings plan (401K) with company match
• Comprehensive insurance coverage including medical, dental, vision, and life insurance
• Paid time off including vacation, sick leave, and family care responsibilities
• Access to Employee Assistance Program
• Eligibility for annual performance-based awards

About the Company

State Street is a leading global provider of financial services to institutional investors. We help our clients manage risk, respond to challenges, and drive performance and profitability. We are committed to fostering an inclusive environment where every employee is empowered to reach their full potential.