Data Engineer

Responsibilities

• Design, build, and maintain scalable data pipelines using PySpark and Python
• Develop and optimize ETL workflows to process structured and semi-structured data
• Work with large datasets stored on AWS S3, and process them using Glue, EMR, or Lambda
• Collaborate with Data Scientists, Analysts, and Product teams to define data needs and deliver quality solutions
• Ensure data quality, integrity, and consistency across systems
• Manage orchestration using AWS Step Functions, Glue Workflows, or Airflow
• Monitor and optimize performance of pipelines, storage, and compute resources
• Participate in data modeling, schema design, and documentation

Requirements

• 4+ years of experience as a Data Engineer or similar role
• Strong programming skills in Python with experience in writing clean, scalable code
• Hands-on experience with PySpark and distributed data processing
• Proficient in AWS services: S3, Glue, EMR, Lambda, Athena, Redshift
• Good understanding of SQL and working with relational databases
• Familiarity with DevOps tools (Git, CI/CD) and job scheduling/orchestration tools
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field