ScaleneWorks INC ยท 1 day ago
Data Engineer
ScaleneWorks INC is seeking a Data Engineer to join their team. The role involves working with Azure Databricks and various data services to optimize and manage data workflows and infrastructure.
Staffing & Recruiting
Responsibilities
Hands-on experience with Azure Databricks, Spark, and Python
Experience with Delta Live Tables (DLT) & Databricks SQL
Strong SQL and database background
Experience with Azure Functions, messaging services, or orchestration tools
Familiarity with data governance, lineage, or cataloging tools (e.g., Purview, Unity Catalog)
Experience monitoring and optimizing Databricks clusters or workflows
Experience working with Azure cloud data services and understanding how they integrate with Databricks and enterprise data platforms
Experience with Terraform for cloud infrastructure provisioning
Experience with GitHub and GitHub Actions for version control and CI/CD automation
Strong understanding of distributed computing concepts (partitions, joins, shuffles, cluster behavior)
Familiarity with SDLC and modern engineering practices
Ability to balance multiple priorities, work independently, and stay organized
Qualification
Required
Hands-on experience with Azure Databricks, Spark, and Python
Experience with Delta Live Tables (DLT) & Databricks SQL
Strong SQL and database background
Experience with Azure Functions, messaging services, or orchestration tools
Familiarity with data governance, lineage, or cataloging tools (e.g., Purview, Unity Catalog)
Experience monitoring and optimizing Databricks clusters or workflows
Experience working with Azure cloud data services and understanding how they integrate with Databricks and enterprise data platforms
Experience with Terraform for cloud infrastructure provisioning
Experience with GitHub and GitHub Actions for version control and CI/CD automation
Strong understanding of distributed computing concepts (partitions, joins, shuffles, cluster behavior)
Familiarity with SDLC and modern engineering practices
Ability to balance multiple priorities, work independently, and stay organized