Senior Data Engineer

Responsibilities

• Design, develop, and maintain scalable data platforms and pipelines on Microsoft Azure
• Build and optimize batch and real-time data ingestion frameworks from multiple sources
• Implement data lake, data warehouse, and lakehouse architectures
• Develop and manage ETL/ELT processes using modern cloud-native practices
• Implement data transformation, cleansing, and quality frameworks to ensure reliability
• Collaborate with stakeholders, data scientists, and analysts to deliver scalable solutions
• Optimize data storage, processing, and query performance
• Implement security, governance, monitoring, and compliance best practices
• Participate in architecture reviews, code reviews, and technical mentoring

Requirements

• 8–10+ years of experience in Data Engineering and Enterprise Data Platform development
• Strong hands-on expertise with Microsoft Azure Data Services
• Proficiency in Azure Data Factory (ADF), Azure Synapse Analytics, and Azure Data Lake Storage (ADLS Gen2)
• Strong proficiency in SQL, query optimization, and database performance tuning
• Hands-on experience with PySpark, Apache Spark, and distributed data processing
• Strong programming skills in Python, Scala, or Java
• Experience with Git, CI/CD pipelines, and DevOps practices
• Deep understanding of dimensional modeling and modern lakehouse architectures

Preferred Qualifications

• Experience with Microsoft Fabric, OneLake, and Dataflows
• Experience with Databricks and Delta Lake implementations
• Knowledge of real-time streaming technologies like Azure Event Hubs or Apache Kafka
• Microsoft Azure Data Engineering certifications
• Experience supporting AI/ML and advanced analytics workloads
• Familiarity with data governance tools such as Microsoft Purview

About the Company

NorthBay Solutions is a fast-growing consulting partner focused on guiding organizations through their digital transformation journeys. Our globally distributed team of over 350 experts specializes in cloud migration, application modernization, and data analytics initiatives for high-profile organizations across financial services, healthcare, and retail.