S
Posted 19 hours ago
Data Engineer
StatusNeo
Requirements
7+ years Data Engineering experience, Python expertise, PySpark proficiency, Google BigQuery experience, SQL skills, Apache Airflow, Git
Skills
PythonPySparkBigQuery
About the role
Responsibilities
- Design, develop, and maintain efficient and scalable ETL/ELT data pipelines using Python and PySpark
- Manage and optimize large datasets in Google BigQuery, ensuring high performance and cost efficiency
- Work closely with Data Scientists, Analysts, and Product Teams to understand data needs and deliver timely solutions
- Perform data modeling, transformation, and integration from multiple data sources
- Ensure data quality, validation, and integrity across all systems and workflows
- Build and maintain job orchestration and scheduling using tools like Apache Airflow
- Write efficient SQL queries and scripts for data extraction and reporting
- Maintain clear documentation of data workflows, architecture, and business logic
Requirements
- 7+ years of hands-on experience in Data Engineering
- Strong programming skills in Python with expertise in modular, production-ready code
- Proficiency in PySpark for distributed data processing
- Solid experience working with Google BigQuery, including query optimization and data management
- Strong SQL skills and experience with relational and columnar databases
- Experience with job orchestration tools like Apache Airflow
- Familiarity with version control tools like Git
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeData Engineer
StatusNeo · Mumbai