STAFFXPERT LLC · 3 weeks ago
Lead Data Engineer
United States
Full-time
Remote
Senior Level
8+ years expSTAFFXPERT LLC is seeking a Lead Data Engineer to support and enhance their data engineering processes. The primary responsibilities include building and maintaining data pipelines, ensuring data quality, and collaborating with various teams to meet data requirements.
Management Consulting
Responsibilities
Serve as L3 support: triage high-severity incidents, perform advanced debugging/root-cause analysis, deploy hotfixes, and create runbooks for L2 teams
Build and maintain batch/streaming data pipelines using ETL/ELT tools (dbt,) to integrate and transform multi-source data
Implement data quality validation, monitoring, alerting, and documentation; optimize pipelines for performance, cost, and reliability (partitioning, indexing, error handling)
Partner with analytics, data science, and business teams to deliver data requirements, troubleshoot issues, and ensure SLAs for freshness/completeness
Qualification
Required
8–10+ years data engineering experience building and supporting production pipelines at scale
Design, build, and maintain data ingestion, transformation, and delivery pipelines across structured and semi-structured data sources
Develop modular, reusable data transformation logic using Python, SQL, and frameworks such as dbt
Implement data models and schemas optimized for analytics and reporting (star, snowflake, or dimensional)
Apply Medallion Architecture principles to organize data layers for quality, traceability, and performance
Use cloud-native data services such as AWS Glue, S3, Redshift, EMR or Azure Data Factory, ADLS, Synapse to manage data workflows
Set up and manage data pipeline orchestration, scheduling, and monitoring using Airflow, ADF, or equivalent tools
Apply data quality checks, validation logic, and logging mechanisms to ensure consistency and trust in data assets
Collaborate with analysts, scientists, and architects to design data models that align with business and analytical needs
Maintain code versioning, testing, and CI/CD standards for data pipeline development
Proven cloud data platform + orchestration experience (Snowflake/Big Query + Airflow/dbt)
L3 support experience: incident management, on-call rotations, debugging distributed data systems
Strong understanding of data engineering fundamentals — ETL/ELT design, data modelling, schema evolution, and data integrity
Proficient in Python and SQL for data transformation, automation, and workflow scripting
Hands-on experience with cloud-based data services in AWS (S3, Glue, Redshift, EMR) or Azure (ADLS, ADF, Synapse)
Working knowledge of distributed data processing concepts (Spark, Hive, or equivalent)
Familiarity with dbt for transformation design, testing, and data documentation
Awareness of Medallion Architecture and data layering concepts for scalable data management
Understanding of orchestration frameworks like Airflow or Data Factory for scheduling and monitoring pipelines
Knowledge of Git-based version control, CI/CD, and basic DevOps practices in data workflows
Preferred
Have an AI skill set, a little bit on Claude, ChatGPT, and other tool supports, or at least who can pick up those skills
Company
STAFFXPERT LLC
STAFFXPERT LLC is a dynamic and results-driven employment agency specializing in IT staffing and placement services, while also providing recruitment solutions across various industries.
11-50 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase