
Posted 9 hours ago
Data Engineer
Photon
Requirements
Strong SQL skills, Batch processing experience, Spark knowledge, Data lake experience, Data reconciliation expertise
Skills
SQLSparkData Engineering
About the role
Responsibilities
- Support ledger migration activities including data extraction, transformation, reconciliation, and validation
- Perform data lake clean-up, quality checks, and manage reporting pipelines
- Develop and validate SQL queries, batch jobs, and data processing logic
- Conduct regression testing on batch processes to ensure data accuracy
- Collaborate with engineering, product, and business teams to resolve data issues
- Document data mapping, validation rules, and migration exceptions
- Support production readiness and post-migration validation
Requirements
- Strong hands-on experience with SQL and large-scale data analysis
- Experience with batch processing and data reconciliation
- Working knowledge of Spark or similar distributed data processing technologies
- Experience with data lakes, reporting platforms, and data migration programs
- Ability to investigate root causes of data inconsistencies
- Understanding of data quality, controls, and auditability in enterprise environments
Preferred Qualifications
- Prior experience in financial services, banking, or ledger systems
- Experience with enterprise-scale platform transformation programs
- Exposure to operational data feeds and dashboards
- Ability to work in distributed teams across London, Bangalore, and Singapore
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeData Engineer
Photon · London
