Azure Databricks Engineer jobs in United States
info-icon
This job has closed.
company-logo

Jobs via Dice ยท 2 days ago

Azure Databricks Engineer

positionJersey City, NJ
timeFull-time
remoteHybrid
seniorityMid Level

Nityo Infotech Corporation is seeking an Azure Databricks Engineer to join their team. The role involves building and maintaining scalable ETL/ELT data pipelines, collaborating with cross-functional teams, and ensuring data governance and security standards are met.

Computer Software

Responsibilities

Build and maintain scalable ETL/ELT pipelines using Databricks
Leverage PySpark/Spark and SQL to transform and process large datasets
Integrate data from multiple sources including Azure Blob Storage, ADLS and other relational/non-relational systems
Work Closely with multiple teams to prepare data for dashboard and BI Tools
Collaborate with cross-functional teams to understand business requirements and deliver tailored data solutions
Optimize Databricks workloads for cost efficiency and performance
Monitor and troubleshoot data pipelines to ensure reliability and accuracy
Implement and manage data security, access controls and governance standards using Unity Catalog
Ensure compliance with organizational and regulatory data policies
Leverage Databricks Asset Bundles for seamless deployment of Databricks jobs, notebooks and configurations across environments
Manage version control for Databricks artifacts and collaborate with team to maintain development best practices

Qualification

DatabricksAzure Cloud ServicesPySparkDelta LakeUnity CatalogLakehouse ArchitectureData Pipeline DevelopmentGitLabRelational DatabasesData SecurityVersion ControlStreaming FrameworksReal-time Data SolutionsAzure Data Engineer AssociateDatabricks Certified

Required

Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Delta Live Pipelines, Databricks Runtime etc.)
Proficiency in Azure Cloud Services
Solid Understanding of Spark and PySpark for big data processing
Experience in relational databases
Knowledge on Databricks Asset Bundles and GitLab

Preferred

Familiarity with Databricks Runtimes and advanced configurations
Knowledge of streaming frameworks like Spark Streaming
Experience in developing real-time data solutions
Azure Data Engineer Associate or Databricks certified Data Engineer Associate certification

Company

Jobs via Dice

twitter
company-logo
Welcome to Jobs via Dice, the go-to destination for discovering the tech jobs you want.
people-size0-1 employees
web-link
https://www.dice.com

Funding

Current Stage
Early Stage
Company data provided by crunchbase