ALLDATA · 6 hours ago
Sr. Data Platform Engineer
Elk Grove, CA
Full-time
Hybrid
Senior Level
$119K/yr - $208K/yr
7+ years expALLDATA is the industry’s #1 choice for unedited original equipment manufacturer (OEM) automotive repair and collision information. The Senior Data Platform Engineer is responsible for designing, implementing, and supporting scalable data solutions using modern big data technologies, while collaborating with various stakeholders to enhance analytics and reporting capabilities.
AutomotiveInformation TechnologyLogisticsSoftware
Responsibilities
Design, develop, and maintain robust data pipelines for ingesting, transforming, and storing large volumes of structured, semi-structured, and unstructured data
Implement data workflows and ETL processes using technologies such as Spark, Delta Lake, and cloud-native tools
Support the production use of big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform), ensuring high availability, scalability, and performance
Monitor, troubleshoot, and resolve issues in production data systems, including job failures, performance bottlenecks, and data quality concerns
Collaborate with data architects to translate business requirements into technical solutions, ensuring alignment with architectural standards and best practices
Optimize SQL queries, Spark jobs, and data processing workloads for efficiency and cost-effectiveness
Implement and maintain data governance, security, and compliance measures, including access controls, data masking, and audit logging
Integrate data workflows into CI/CD pipelines, automate deployment processes, and manage source control using Git and related DevOps tools
Document data pipelines, workflows, and operational procedures to support knowledge sharing and maintainability
Stay current with emerging big data technologies and recommend improvements to enhance reliability, scalability, and efficiency
Qualification
Required
7+ years of experience in data engineering or related roles, with hands-on experience building and supporting production data pipelines
Strong proficiency with big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform) and the Spark ecosystem
Experience with data lakehouse and warehouse architectures, including Delta Lake or similar technologies
Senior Data Engineer with deep expertise in Retrieval-Augmented Generation (RAG) systems and advanced chunking strategies for optimizing semantic search and LLM integration, delivering scalable solutions for enterprise knowledge retrieval and contextual AI applications
Advanced skills in PySpark, Python, SQL, and related data processing frameworks
Experience with cloud storage, messaging/streaming technologies (e.g., Apache Kafka, cloud Pub/Sub), and vector databases
Proven ability to optimize ETL jobs, tune cluster configurations, and troubleshoot production issues
Familiarity with data governance tools, catalog solutions, and security best practices
Experience integrating data workflows into CI/CD pipelines and using DevOps tools (e.g., Git, Jenkins, Terraform)
Strong problem-solving skills, attention to detail, and ability to work independently or as part of a team
Excellent communication skills to collaborate with cross-functional teams and document technical solutions
Experience working in an Agile environment
Preferred
Certifications in major big data platforms (e.g., Databricks, Snowflake, Google Cloud Platform) are advantageous
Exposure to related big data and streaming tools such as Apache Kafka, cloud messaging services, Apache Airflow, and BI/analytics tools is a plus
Google Professional Cloud Architect or better – Preferred, but not required
Azure Solutions Architect Expert – Preferred, but not required
Benefits
Competitive pay
Unrivaled company culture
Medical, dental and vision plans
Exclusive discounts and perks, including an AutoZone in-store discount
401(k) with company match and Stock Purchase Plan
AutoZoners Living Well Program for free mental health support
Opportunities for career growth
Paid time off
Life, and short- and long-term disability insurance options
Health Savings and Flexible Spending Accounts with wellness rewards
Tuition reimbursement
Company
ALLDATA
Alldata is the leading provider of OEM service and repair information to the professional automotive service and collision industries. It is a sub-organization of AutoZone.
201-500 employees
H1B Sponsorship
ALLDATA has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (4)
2024 (2)
2023 (5)
2022 (6)
2021 (6)
2020 (5)
Funding
Current Stage
Growth StageTotal Funding
unknown1996-02-07Acquired
Leadership Team
Satwinder Mangat
President, ALLDATA
Recent News
Globe Newswire
2024-03-01
Company data provided by crunchbase