Data Engineer

Responsibilities

• Design, develop, and manage scalable data pipelines and ETL workflows using Databricks, PySpark, and SQL
• Build and maintain data ingestion frameworks to extract data from SAP APIs, REST services, and relational databases
• Develop and optimize Delta Lake based data architecture for high-performance storage
• Implement data quality validation, monitoring, and error handling frameworks
• Build automated workflow orchestration and scheduling mechanisms
• Optimize Spark jobs and SQL queries to improve performance and reduce costs
• Collaborate with data analysts and business stakeholders to design efficient data solutions
• Support data platform engineering including cluster configuration and performance tuning
• Troubleshoot production pipeline failures and data inconsistencies

Requirements

• 2+ years of experience in Data Engineering and Data Pipeline Development
• Strong experience with Python, PySpark, and SQL
• Hands-on experience with Databricks and Delta Lake
• Experience integrating data from REST APIs and SAP systems
• Knowledge of data modeling, schema design, and ETL best practices
• Experience with cloud data platforms such as GCP, AWS, or Azure
• Proficiency in workflow orchestration and job scheduling
• Ability to optimize Spark workloads and troubleshoot large datasets

About the Company

NucleusTeq provides advanced technology solutions and data engineering expertise to help enterprises manage and leverage their data effectively.