Data Engineer

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines to support business and analytical needs
• Work extensively with Databricks, Python, and PySpark to process large datasets
• Build and manage DAGs using Apache Airflow for workflow orchestration
• Collaborate with cross-functional teams to translate data requirements into engineering solutions
• Develop and optimize complex SQL queries and participate in data modeling activities
• Work with Amazon S3 for data storage, ingestion, and partitioning
• Ensure high standards of data quality, reliability, and performance

Requirements

• 2–5 years of experience in Data Engineering or related roles
• Strong Python programming skills for data manipulation and ETL workflows
• Hands-on experience with Databricks, PySpark, and Delta Lake
• Proficiency in Apache Airflow for designing and maintaining complex DAGs
• Advanced SQL expertise including query optimization and data modeling
• Practical knowledge of Amazon S3 for data lake architectures

Preferred Qualifications

• Experience with Argo Workflows or Kubernetes-native orchestration platforms
• Familiarity with data libraries such as pandas and numpy

About the Company

Analytical Wizards is part of the Definitive Healthcare family, focusing on unlocking the value of data and applications to solve complex business and societal challenges through innovation and ethical data practices.