
Posted 2 days ago
Senior Data Engineering Lead
OptumSenior Data Engineering Lead
Requirements
Bachelor's Degree in IT, Engineering, Math, or CS, 7+ years in data engineering and storage systems, 7+ years in ETL/ELT design (Informatica, DataStage, SSIS), 7+ years in SQL, Python, or Scala, 5+ years in Microsoft Azure, Data Factory, Databricks, and Spark, 5+ years in healthcare data environments, Experience with RESTful APIs, Experience with Gen AI/LLM solutions and RAG pipelines
Skills
PythonAzureSQLSparkGenAI
About the role
Responsibilities
- Support the full data engineering lifecycle, including research, design, development, testing, and maintenance of data management solutions.
- Lead data acquisition efforts to gather data from structured and semi-structured source systems to power analytics across healthcare domains.
- Implement and maintain ETL/ELT methodologies to pull complex relational and dimensional data for DataMarts and reporting.
- Operationalize ML/GenAI pipelines by building scalable data pipelines, feature engineering frameworks, and model workflows.
- Build and optimize data pipelines for LLM/RAG use cases, including vector ingestion, embedding generation, and metadata enrichment.
- Drive data governance, security, and responsible AI practices to ensure compliance with privacy and enterprise standards.
- Partner with the broader analytics organization to recommend improvements to data systems and platform architecture.
- Utilize DevOps tools for code versioning and deployment, and leverage monitoring tools to ensure data integrity.
Requirements
- Bachelor's Degree in IT, Engineering, Math, Computer Science, or a related field.
- 7+ years of experience in data engineering, including ingestion, transformation, and storage.
- 7+ years of experience in designing ETL/ELT solutions using tools such as Informatica, DataStage, or SSIS.
- 7+ years of experience managing data assets using SQL, Python, or Scala.
- 5+ years of experience with Microsoft Azure, Azure Data Factory, Databricks, and Spark.
- 5+ years of experience working within healthcare data environments.
- Hands-on expertise in developing and integrating secure, scalable RESTful APIs.
- Experience building and operationalizing Gen AI/LLM solutions, including RAG pipelines and vector databases.
Benefits
- Comprehensive health plans and wellness coverage for you and your loved ones.
- Financial benefits, including retirement planning support.
- Emotional support through various tools and resources to build resilience and manage stress.
- Opportunities for professional growth, mentorship, and continuous learning.
About the Company
Optum is a global organization that delivers care, aided by technology, to help millions of people live healthier lives. We connect people with the care, pharmacy benefits, data, and resources they need to feel their best through innovation and a culture of inclusion.
ScoutJobs Agent
Get matches like this delivered daily
Sign up free — we'll pull jobs that fit your CV from across the web and rank them for you.
Get started — it's freeSenior Data Engineering Lead
Optum · Gurgaon
