Lead Data Engineer

Responsibilities

• Develop and deploy scalable ETL/ELT pipelines using Azure Databricks, PySpark, and SQL.
• Create Delta Lake tables with ACID transactions and schema evolution to support real-time and batch processing.
• Implement centralized data governance, access control, and data lineage tracking using Unity Catalog.
• Work within scrum teams to deliver high-quality database programming requirements for each sprint.
• Independently analyze, solve, and correct technical issues in real time to provide end-to-end resolution.
• Develop automated unit tests and apply SOLID development principles to maintain data integrity.
• Collaborate with product owners and business representatives to identify and satisfy data needs.

Requirements

• 7-10 years of experience as a Lead Data Engineer.
• Proven expertise in T-SQL programming and Azure Databricks.
• Hands-on experience with PySpark, Delta Lake, Unity Catalog, and ADLS Gen2.
• Proficiency with Microsoft TFS, Visual Studio, and Azure DevOps.
• Strong experience with Azure cloud platforms, including Azure SQL, Data Factory (ADF), and Data Lake.
• Analytical mindset with strong problem-solving and critical thinking skills.
• Ability to work independently with minimal supervision in a fast-paced environment.

Preferred Qualifications

• Previous experience working within the Healthcare domain.

About the Company

NationsBenefits is leading the transformation of the insurance industry by developing innovative benefits management solutions. We focus on modernizing complex back-office systems to create scalable, secure, and high-performing platforms that streamline operations for our clients through cloud-native architectures.