Be an early applicantLess than 25 applicants

Company

Tredence Inc. · 8 hours ago

QA Lead (Databricks)

United States

Full-time

Remote

Senior Level, Lead/Staff

Maximize your interview chances

AnalyticsArtificial Intelligence (AI)

Growth Opportunities

H1B Sponsor Likely

Insider Connection @Tredence Inc.

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Define and implement robust QA strategies, methodologies, and processes for data engineering projects.

Lead and mentor a team of QA engineers, ensuring alignment with best practices and project goals.

Collaborate with stakeholders to understand data quality requirements and translate them into test plans and cases.

Develop and maintain automated testing frameworks for ETL pipelines, data lakes, and data warehouses on Azure Data Factory, Databricks, and related services.

Design data validation and verification processes to ensure the accuracy and consistency of data across pipelines.

Automate regression, performance, and integration testing to minimize manual efforts.

Conduct root cause analysis for data quality issues and drive corrective actions.

Ensure compliance with data governance policies and data security best practices.

Develop data profiling and monitoring dashboards using tools like Azure Monitor, Power BI, or custom solutions.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

Azure Data FactoryAzure DatabricksSQLPythonCI/CDData GovernanceDatabricksPyTestDbtAzure MonitorPower BIMLflowBig DataAzure Data Engineer AssociateAzure Solutions Architect Expert

Required

Proficient in Azure Data Factory, Azure Databricks, Azure Synapse, and Azure Storage services.

Experience with Spark-based data engineering workflows, including Delta Lake.

Strong experience with tools like PyTest, dbt (data build tool), or similar testing frameworks for data pipelines.

Advanced SQL skills for data validation, with Python/Scala proficiency for automation.

Knowledge of CI/CD pipelines on Azure DevOps or GitHub Actions, focusing on data engineering workflows.

Familiarity with data governance tools and concepts such as metadata management, lineage, and data cataloging (e.g., Azure Purview).

Preferred

Experience with MLflow for data pipeline monitoring.

Knowledge of Big Data processing and distributed computing.

Certifications in Azure Data Engineer Associate or Azure Solutions Architect Expert.

Company

Tredence Inc.

Glassdoor

4.1

Tredence is a global data science solutions provider focused on solving the last mile problem in AI.

Founded in 2013

San Jose, California, USA

1,001-5,000 employees

http://tredence.com

H1B Sponsorship

Tredence Inc. has a track record of offering H1B sponsorships. Please note that this does not guarantee sponsorship for this specific role. Below presents additional info for your reference. (Data Powered by US Department of Labor)

Distribution of Different Job Fields Receiving Sponsorship

Represents job field similar to this job

Trends of Total Sponsorships

2023 (63)

2022 (91)

2021 (73)

2020 (60)