Senior Data Engineer

Responsibilities

• Collaborate with cross-functional teams to develop end-to-end data pipelines and foundational data sets
• Design and own data architecture for large-scale projects while managing operational trade-offs
• Build and optimize sophisticated data pipelines, models, and visualizations across multiple domains
• Define and manage Service Level Agreements (SLAs) for all owned data sets
• Implement data security models, ensure privacy compliance, and evolve data governance processes
• Solve complex integration challenges using optimal ETL/ELT patterns for structured and unstructured data
• Maintain production processes by optimizing complex code using advanced algorithmic concepts
• Streamline data artifact development by optimizing pipelines, dashboards, and frameworks
• Mentor team members through actionable feedback to foster collective skill growth
• Maintain flexibility with availability during evening hours

Requirements

• Bachelor's degree in Computer Science, Computer Engineering, or a relevant technical field
• 5+ years of experience in data engineering, database engineering, business intelligence, data warehousing, and ETL tools
• 5+ years of experience with Python and building scalable Big Data solutions
• Proficiency in SQL, ETL/ELT, and data modeling
• Extensive experience in Snowflake and dbt
• Hands-on experience with RDBMS such as MySQL and Postgres
• Familiarity with integration tools like Airflow and Automic
• Working knowledge of CI/CD pipelines
• Deep understanding of the modern IaC ecosystem
• Experience with AWS services including Glue, Airflow (MWAA), DMS, EKS, and SNS
• Experience leveraging AI tools to transform workflows

Preferred Qualifications

• Experience with Spark and PySpark
• Master's degree in Computer Science or Computer Engineering

About the Company

Metropolis is creating the artificial intelligence to make the real world responsive, pioneering the Recognition Economy to transform parking and mobility through seamless, intelligent technology.