The Walt Disney Company · 3 hours ago
Lead Data Engineer, AI Data Foundations
San Francisco, CA, USA
Full-time
Hybrid
Senior Level, Lead/Staff
$152K/yr - $204K/yr
7+ years expThe Walt Disney Company is a global leader in entertainment and technology. They are seeking a Lead Data Engineer to design, build, and scale data foundations that support AI adoption across Ad Technology, ensuring reliable, real-time data for high-profile applications.
Amusement Park and ArcadeAnimationConsumer GoodsDigital MediaE-CommerceMedia and EntertainmentMulti-level MarketingPerforming ArtsResorts
Responsibilities
Design and build streaming and batch pipelines for AI applications, ensuring low-latency and high-throughput data flows
Implement embedding pipelines and vector store integrations to support retrieval, semantic search, and AI assistants
Optimize storage and retrieval patterns for scalability, compliance, and cost efficiency
Own the design of AI-optimized data stores that align with enterprise data architecture
Partner with the AI Core Engineering team to provide reliable pipelines into shared agents, registries, and services
Develop reusable data ingestion and transformation frameworks for cross-team adoption
Build observability into data pipelines with monitoring, alerting, and dashboards for latency, drift, and failures
Implement data quality checks, schema validation, and governance controls
Ensure pipelines meet enterprise standards for compliance, audit, and resilience
Mentor engineers, conduct design/code reviews, and enforce data engineering best practices
Collaborate with infra, product, and governance teams to align on priorities and safe adoption of AI
Drive delivery of high-profile AI applications by balancing execution speed with system reliability
Qualification
Required
7+ years of data engineering experience, with at least 2 years in a lead or senior technical role
Experience building and scaling streaming data pipelines in large-scale, distributed environments
Strong skills in Python, Java and SQL with expert level skill in either Python or Java
Proven experience building streaming data pipelines (e.g., Kafka, Flink, Spark, Kinesis)
Experience with embedding pipelines and vector stores (e.g., Pinecone, Weaviate, FAISS, pgvector)
Strong knowledge of data modeling, storage optimization, and retrieval patterns for large-scale systems
Hands-on experience with workflow orchestration tools (Airflow, Dagster, etc.)
Strong collaboration and communication skills, able to partner across AI engineering, infra, and product teams
Familiarity with testing, monitoring, and automation for data pipelines
Bachelor's degree in Computer Science, Engineering, or related technical field (Master's preferred), or equivalent experience
Preferred
Experience integrating AI-ready data stores with LLM orchestration frameworks (LangChain, LangGraph, etc.)
Knowledge of observability and monitoring stacks (Datadog, Prometheus, or equivalent)
Background in governance and compliance practices for enterprise data platforms
Experience building data frameworks, registries, or accelerators adopted by multiple teams
Experience ensuring data security, lineage, and auditability in enterprise data environments
Skilled at writing design documentation, driving system architecture reviews, and influencing data engineering culture
Benefits
A bonus and/or long-term incentive units may be provided as part of the compensation package
The full range of medical, financial, and/or other benefits
Company
The Walt Disney Company
The Walt Disney Company started as a cartoon studio and evolves into sports coverage and television shows.
10001+ employees
H1B Sponsorship
The Walt Disney Company has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (83)
2024 (63)
2023 (96)
2022 (130)
2021 (30)
2020 (40)
Funding
Current Stage
Public CompanyTotal Funding
$11BKey Investors
Citibank
2020-04-13Post Ipo Debt· $5B
2020-03-20Post Ipo Debt· $6B
1978-01-06IPO
Leadership Team
Christine M. McCarthy
Senior Executive Vice President & Chief Financial Officer
Jamie Power
SVP, Addressable Sales
Recent News
2026-01-08
2026-01-07
Company data provided by crunchbase