L
Posted 18 hours ago
Data Engineer
LumiqData Engineer
Requirements
3-6 years experience, PySpark, SQL, AWS Cloud Services, Data Pipeline architecture, Hadoop ecosystem, ETL processes
Skills
PySparkSQLAWS
About the role
Responsibilities
- Design, develop, and maintain scalable data pipelines
- Work extensively with PySpark for data processing and transformation
- Write optimized and complex queries using SQL
- Develop and manage data solutions on AWS (S3, EMR, Glue, Redshift, etc.)
- Handle large datasets using Hadoop ecosystem tools
- Ensure data quality, integrity, and security across systems
- Collaborate with data analysts, data scientists, and cross-functional teams
- Troubleshoot and optimize existing data workflows
Requirements
- 3 to 6 years of experience
- Strong hands-on experience in PySpark
- Proficiency in SQL
- Experience with AWS Cloud Services
- Strong understanding of Data Pipeline architecture
- Working knowledge of Hadoop ecosystem
- Good understanding of ETL processes
- Strong problem-solving and analytical skills
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeData Engineer
Lumiq · Noida