AWS Data Engineering Tech Lead

Responsibilities

• Lead the design and development of scalable data pipelines and ETL processes on AWS
• Architect and implement big data solutions using PySpark, Spark, and Hadoop
• Develop and optimize data processing workflows using Python and SQL
• Work closely with stakeholders to understand data requirements and translate them into technical solutions
• Ensure data quality, reliability, performance, and security across data platforms
• Mentor and guide junior data engineers, conduct code reviews, and enforce best practices
• Optimize existing ETL jobs for performance and cost efficiency
• Collaborate with DevOps, analytics, and business teams for end-to-end data solutions

Requirements

• 5–7 years of experience in data engineering
• 1–2 years in a technical lead role
• Strong experience with AWS services (S3, EMR, Glue, Redshift, Athena, Lambda)
• Hands-on expertise in PySpark and Apache Spark
• Strong proficiency in SQL and performance tuning
• Solid experience in ETL design and development
• Advanced knowledge of Python for data engineering
• Experience working with Hadoop ecosystem
• Understanding of data warehousing and big data architecture
• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

Preferred Qualifications

• Experience with orchestration tools like Airflow or Step Functions
• Knowledge of CI/CD pipelines for data engineering
• Exposure to real-time data processing using Kafka or Kinesis
• Experience working in Agile/Scrum environments