Principal Data Engineer - Databricks jobs in United States
cer-icon
Apply on Employer Site
company-logo

Alaska Airlines · 16 hours ago

Principal Data Engineer - Databricks

Alaska Airlines is creating an airline people love, and they are seeking a Principal Data Engineer to advance their Enterprise Data platform using Databricks. This role involves designing, optimizing, and implementing high-performance data pipelines, collaborating with various teams to deliver production-grade solutions, and driving innovation in data engineering practices.

Responsibilities

Act as lead subject matter expert in designing and implementing high-performance batch or real-time data pipelines using Apache Spark, Delta Live Tables, and Structured Streaming on Databricks
Collaborate with data scientists, analysts, and engineering teams to deliver robust, production grade solutions
Define and enforce best practices for workspace governance, job orchestration, and CI/CD integration
Optimize cluster configurations, spark parameters, and job execution plans for cost efficiency and speed
Evaluate and implement new Databricks features to enhance platform capabilities. Champion the usage and raise awareness of these new features across the teams
Exercise considerable latitude and initiative to troubleshoot and resolve performance bottlenecks, memory issues, and streaming latency challenges
Lead development and mentor engineers on real-time streaming solutions, spark internals, delta table optimization strategies, and advanced debugging techniques
Implement and manage Unity Catalog for centralized governance, data lineage, and fine-grained access control
Monitor Databricks costs, identify, evaluate and drive implementation for cost reduction opportunities
Lead efforts to standardize tagging and metadata practices across the AAG Databricks environment to improve cost attribution, accountability, and reporting
Focus on continuous improvement in system reliability through automation and monitoring, including implementation of automated unit testing. Document and create best practices for the development of future teams
Collaborate directly with Enterprise Architecture and Security to develop and plan the platforms architecture and security strategies
Act as a forward thinker to help ensure that new capabilities are evaluated, implemented, and operationalized

Qualification

DatabricksApache SparkReal-time streamingPythonSQLPySparkInfrastructure as CodeCI/CD pipelineUnity CatalogKafkaAgile methodologiesContinuous improvementCollaborationCommunicationMentoring

Required

7 years of experience in data engineering, big data platforms, and analytics, including independent work on large-scale data architectures or a related area
Bachelor's degree, or an additional two years of relevant training/experience in lieu of this degree
Expert-level proficiency in Databricks, Apache Spark
Deep understanding of Databricks configurations and hands-on experience with real-time streaming using Structured Streaming and Delta Live tables
Strong coding skills with Python, SQL and PySpark
Expert knowledge and experience in Infrastructure as Code (ARM, Terraform) and CI/CD pipeline (Azure DevOps and/or GitHub preferred)
Proven ability to optimize data pipelines for performance, reliability, and cost
Familiarity with Unity Catalog, Lakehouse Federation
Ability to work collaboratively with cross functional teams to understand objectives, gather automation requirements, write technical specifications, and perform in a lead role
Experience with Kafka
Excellent communication skills and a proven ability to collaborate or contribute to technical debate and design choices with a variety of team members
High school diploma or equivalent
Minimum age of 18
Must be authorized to work in the U.S

Preferred

Familiarity with Azure, MLflow
Certifications such as Databricks Certified Data Engineer Professional or Azure Solutions Architect Expert
Experience in Agile (Scrum) or Kanban development methodologies
Experience estimating effort and establishing timelines for projects
Experience conveying knowledge to diverse groups of people, such as classes, brown bag meetings, or seminars to ensure a consistent application of standards is established

Benefits

Free stand-by travel privileges on Alaska Airlines, Hawaiian Airlines & Horizon Air
Comprehensive well-being programs including medical, dental and vision benefits
Generous 401k match program
Quarterly and annual bonus plans
Generous holiday and paid time off

Company

Alaska Airlines

company-logo
Alaska Airlines provides rewarding careers in aviation by fueling a passion for service and helping others

H1B Sponsorship

Alaska Airlines has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (15)
2024 (10)
2023 (14)
2022 (13)
2021 (10)
2020 (13)

Funding

Current Stage
Late Stage

Leadership Team

leader-logo
Ben Minicucci
Chief Executive Officer
linkedin
leader-logo
Jason Berry
EVP & Chief Operating Officer
linkedin
Company data provided by crunchbase