Senior Data Engineer

Responsibilities

• Build and maintain high-performance streaming and batch data pipelines to power AI applications.
• Implement embedding generation workflows, vector store integrations, and retrieval pipelines for RAG systems and AI assistants.
• Develop and optimize scalable storage and retrieval patterns focusing on cost-efficient architecture.
• Implement AI-optimized data models and integrate pipelines with shared AI platform services like agent frameworks and feature stores.
• Embed end-to-end observability into data systems, including metrics, automated alerts, and drift detection.
• Drive the full development lifecycle from prototyping and implementation to testing and deployment.
• Collaborate with infrastructure, ML engineering, and product teams to deliver production-ready AI capabilities.

Requirements

• 5+ years of data engineering experience, with at least 1 year in a lead or senior technical role.
• Proven experience building and scaling streaming data pipelines (e.g., Kafka, Flink, Spark, Kinesis).
• Expert-level proficiency in Python, Java, or SQL.
• Hands-on experience with vector stores (e.g., Pinecone, Weaviate, FAISS, pgvector).
• Experience with workflow orchestration tools such as Airflow or Dagster.
• Bachelor's degree in Computer Science or a related quantitative field, or equivalent practical experience.

Preferred Qualifications

• Experience integrating AI-ready data stores with LLM orchestration frameworks like LangChain or LangGraph.
• Knowledge of observability and monitoring stacks such as Datadog or Prometheus.
• Background in governance and compliance practices for enterprise data platforms.
• Experience building data frameworks, registries, or accelerators adopted by multiple teams.

About the Company

The Walt Disney Company is a leading diversified international family entertainment and media enterprise. Disney Entertainment and ESPN Product & Technology is a global organization of engineers and technologists working to build the technological backbone for Disney’s media business globally, serving millions of people around the world through brands like Disney+, Hulu, and ESPN.