Senior Data Engineer

About the Company

Metropolis is an artificial intelligence company that uses computer vision technology to enable frictionless, checkout-free experiences in the real world. They are reimagining parking and building an ecosystem of technologies at the intersection of parking, mobility, and real estate.

Responsibilities

• Collaborate with engineers, product managers, and data scientists to develop end-to-end data pipelines and foundational data sets
• Design and own data architecture for large-scale projects, managing design and operational trade-offs
• Build, launch, and optimize sophisticated data pipelines, models, and visualizations
• Define and manage Service Level Agreements (SLAs) for owned data sets
• Implement data security models, ensure compliance, and evolve data governance processes
• Solve complex data integration challenges using optimal ETL/ELT patterns for structured and unstructured data
• Maintain and enhance production processes by optimizing complex code
• Mentor team members to foster skill growth and collaboration

Requirements

• Bachelor's degree in Computer Science, Computer Engineering, or a relevant technical field
• 5+ years of experience in data engineering, database engineering, BI, or data warehousing with large cloud datasets
• Proficiency in SQL, ETL/ELT, and data modeling with extensive experience in Snowflake and dbt
• 5+ years of experience with Python and building scalable Big Data solutions
• Hands-on experience with RDBMS such as MySQL, MS SQL Server, and Postgres
• Familiarity with integration tools like Airflow and Automic, and CI/CD pipelines
• Experience with Spark and PySpark for large-scale data processing
• Strong proficiency in Terraform is a must-have
• Ability to ensure flexibility with availability during evening hours