Senior AWS Data Engineer

Responsibilities

• Design, develop, and maintain scalable data pipelines for processing large datasets
• Develop data processing scripts using Python
• Write optimized SQL queries for data extraction and transformation
• Process large volumes of data using Apache Spark (Scala or PySpark)
• Manage and store data using AWS S3
• Implement serverless data workflows using AWS Lambda
• Use AWS Glue Crawlers for data cataloging and metadata management
• Query data efficiently using Amazon Athena
• Develop dashboards or reports using Amazon QuickSight or other visualization tools
• Monitor and optimize data pipelines using AWS CloudWatch
• Work with Amazon EMR for big data processing
• Integrate alerting or messaging workflows using AWS SNS

Requirements

• 4–6 years of experience in Data Engineering or Big Data technologies
• Strong proficiency in Python programming
• Hands-on experience with SQL
• Experience with Apache Spark (Scala or PySpark)
• Experience working with AWS S3, Lambda, Glue Crawlers, and Athena
• Experience with any data visualization tool (QuickSight, Power BI, Tableau, etc.)
• Strong problem-solving and analytical skills

Preferred Qualifications

• Experience with Amazon EMR
• Knowledge of AWS SNS
• Experience with AWS CloudWatch monitoring