
Posted 10 hours ago
Principle DataOps Architect
MarexPrinciple DataOps Architect
Requirements
Implementing DataOps or DevOps in complex environments, Designing CI/CD pipelines for data workloads, Hands-on experience with Databricks (AWS/Azure), Proficiency in Python and SQL, Infrastructure as Code using Terraform, Knowledge of software engineering SDLC, Experience in regulated environments
Skills
DataOpsDatabricksPythonSQLTerraformCI/CDAWSAzure
About the role
Responsibilities
- Design, implement, and operationalize the DataOps target state through consistent frameworks, tooling, and standards.
- Build and maintain CI/CD pipelines for data and ML workloads to enable automated promotion across environments.
- Define and implement test-driven development practices for data pipelines, including unit, integration, and end-to-end testing.
- Embed data reliability patterns aligned to the Medallion Architecture (Bronze, Silver, Gold).
- Implement data quality enforcement, monitoring, and observability frameworks to improve platform trust.
- Provide hands-on technical leadership and design advisory services to Data Engineering, AI/ML, and Analytics teams.
- Partner with Governance, Risk, and Control stakeholders to translate regulatory requirements into repeatable engineering controls.
Requirements
- Strong experience implementing DataOps or DevOps practices in complex data environments.
- Proven expertise designing and operating CI/CD pipelines for data and analytics workloads.
- Hands-on experience with Databricks (AWS and/or Azure) in production environments.
- Strong proficiency in Python and SQL.
- Experience applying Infrastructure as Code (IaC) using tools such as Terraform.
- Deep understanding of data platform reliability, observability, and quality controls.
- Strong knowledge of software engineering best practices and SDLC.
- Experience working in a regulated environment with knowledge of associated risk and compliance requirements.
Preferred Qualifications
- Experience in financial services, specifically with ETD and OTC derivative markets.
- Experience with orchestration platforms such as Azure Data Factory or Apache Airflow.
- Proficiency with data transformation platforms like dbt or Databricks Lakeflow.
- Experience supporting ML pipelines and model lifecycle operations.
- Databricks Certified Data Engineer Professional or DAMA Certified Data Management Professional (CDMP).
About the Company
Marex Group plc is a diversified global financial services platform providing essential liquidity, market access, and infrastructure services to clients across energy, commodities, and financial markets. With over 3,000 employees and 40 offices worldwide, Marex leverages technology-led services to provide award-winning data, insights, and analytics to the world's major commodity markets.
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freePrinciple DataOps Architect
Marex · London
