Azure Data Engineer

Responsibilities

• Develop and implement data engineering solutions using Azure services such as Databricks, Data Factory, and Data Lake
• Work extensively with PySpark to process and transform large datasets efficiently
• Design, implement, and optimize Delta Lake architecture for scalable data storage and management
• Manage Databricks Jobs, Scheduling, Orchestration, Workflow, and REST API integration
• Collaborate with cross-functional teams to translate business requirements into data solutions
• Ensure data integrity, quality, and security in cloud-based data platforms
• Optimize data pipelines for performance and cost-efficiency
• Troubleshoot and resolve data processing issues in the Azure ecosystem

Requirements

• 6-8 years of experience in data engineering
• Expertise in Azure Databricks, Azure Data Factory, Azure Data Lake, and PySpark
• Strong knowledge of Databricks Jobs, Scheduling, Orchestration, and REST API integration
• Hands-on experience with Delta Lake Architecture and Implementation
• Experience building and maintaining scalable ETL/ELT pipelines
• Proficiency in SQL, Python, and Spark
• Experience with large-scale data processing and distributed computing

Preferred Qualifications

• Experience in Azure Synapse Analytics or Power BI
• Knowledge of Machine Learning and AI-based data processing techniques
• Azure certifications such as Microsoft Certified: Azure Data Engineer Associate
• Familiarity with CI/CD pipelines and DevOps practices

About the Company

Unionsys Technologies is a technology firm offering opportunities to work on cutting-edge cloud data technologies within a collaborative and innovative work culture.