
Posted 14 hours ago
Data Engineering Tech Lead
Deriv.comData Engineering Tech Lead
Requirements
8+ years in data engineering, Data modelling expertise, Cloud data stack experience, CI/CD proficiency, Hands-on coding ability
Skills
PythonSQLGCPBigQueryAirflowDBTKafka
About the role
Responsibilities
- Define, document, and drive adoption of technical standards for pipeline patterns, modeling conventions, and integration approaches.
- Make critical architectural decisions for data pipelines, modeling layers, and integration patterns.
- Own production reliability by implementing incident prevention, post-incident reviews, and systematic reduction of recurring failures.
- Build observability into all pipelines, including freshness checks, completeness validation, schema drift detection, and lineage tracking.
- Design scalable dimensional models, medallion layers, and integration patterns to handle increasing volume and complexity.
- Enforce data contracts, SLAs, and SLOs between producers and consumers.
- Automate scheduling, testing, deployment, and quality gates to reduce manual technical debt.
- Embed governance, PII handling, and access controls directly into the pipeline code.
Requirements
- 8+ years of experience in data engineering with a proven track record in technical leadership or a principal engineer capacity.
- Deep expertise in data modeling, including Kimball star schemas, Data Vault, or medallion architecture.
- Extensive experience with cloud data stacks, specifically GCP, BigQuery, Airflow, and Python.
- Strong hands-on experience with transformation tools such as dbt or Dataform.
- Proficiency in CI/CD for data pipelines, including version control and automated testing.
- Ability to work as a hands-on individual contributor, writing and reviewing production-grade code.
Preferred Qualifications
- Experience designing and operating real-time or event-driven pipelines using Kafka or Pub/Sub.
- Exposure to low-code integration tools like Fivetran or RudderStack.
- Experience with pipeline observability tooling for lineage and anomaly detection.
- Familiarity with open table formats such as Apache Iceberg or Delta Lake.
- Background in financial services, fintech, or highly regulated environments.
About the Company
Deriv.com is a global trading platform with a mission of "Trading for Anyone, Anywhere, Anytime." We power millions of traders around the clock through advanced AI, fraud detection models, and automated financial workflows. We value technical excellence, open sharing, and building reliable, governed infrastructure that can scale with our global user base.
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeData Engineering Tech Lead
Deriv.com · Cyberjaya
