Acunor · 3 weeks ago
Databricks Engineer - Azure
United States
Full-time
Remote
Senior Level
$150K/yr - $160K/yr
7+ years expAcunor is seeking a highly skilled Data Engineer with deep expertise in Databricks and Azure Cloud to join a decision analytics and data engineering team. The role involves building, optimizing, and maintaining large-scale data pipelines that support enterprise analytics, reporting, and AI-driven insights, primarily for clients in the insurance and financial services domains.
AnalyticsCloud ManagementDatabaseInformation TechnologyStaffing Agency
Hiring Manager
Dharmendra Mutta
Responsibilities
Design, build, and enhance ETL/ELT data pipelines using Azure Data Factory, Databricks (PySpark, SQL, Python), and related services
Develop and manage Delta Live Tables, Autoloader, and Unity Catalog within the Databricks ecosystem for structured, incremental data processing
Implement data ingestion, transformation, and validation frameworks that ensure high performance, scalability, and reliability
Monitor data pipelines, troubleshoot issues, and ensure optimal system performance and SLA adherence
Collaborate with business analysts and reporting teams to define logical and physical data models supporting analytical and operational needs
Implement data warehousing and lakehouse solutions using Azure Data Lake and Delta Lake
Optimize data structures for query performance, cost efficiency, and reusability
Design and implement robust data quality checks and validation mechanisms to maintain integrity across sources and transformations
Automate repetitive processes using scripts, parameterized pipelines, and reusable frameworks
Conduct periodic audits and compliance checks aligned with governance policies
Contribute to metadata management, documentation, and lineage tracking
Qualification
Required
7–12 years of experience in Data Engineering with proven expertise in Databricks and Azure Cloud ecosystems
Strong hands-on experience in PySpark, Python, and SQL for data transformation, validation, and performance tuning
Solid understanding of Delta Lake architecture, ETL/ELT frameworks, and data warehousing principles
Proficiency with Azure services including Data Factory (ADF), Data Lake (ADLS), and Databricks Notebooks
Experience with Delta Live Tables, Unity Catalog, and Autoloader for batch and streaming data processing
Strong background in data modeling, performance optimization, and automation scripting
Familiarity with Agile methodologies and DevOps-based deployment practices (Git, CI/CD preferred)
Preferred
Experience in data migration projects (on-prem to cloud or multi-cloud)
Familiarity with Delta Sharing, Databricks SQL Warehouses, or MLflow for advanced use cases
Experience with data cataloging, lineage, or quality frameworks such as Purview, Collibra, or Great Expectations
Exposure to BI/reporting tools like Power BI or Tableau for end-to-end integration understanding
Company
Acunor
Acunor is an IT management company that offers staffing services including analytics, cloud, digital transformation, and data engineering.
51-200 employees
H1B Sponsorship
Acunor has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (5)
2024 (9)
2023 (1)
2022 (1)
Funding
Current Stage
Growth StageLeadership Team
Mona M.
Talent Research Partner
Company data provided by crunchbase