E-Solutions · 5 hours ago
Azure Databricks Architect
United States
Contract
Remote
Senior Level, Lead/Staff
12+ years expE-Solutions is seeking a Sr Azure Databricks Architect / SME to lead the architecture design and implementation of advanced analytics solutions using Azure Databricks Fabric. The ideal candidate will work closely with business stakeholders to deliver effective data solutions and oversee the development of data pipelines and ETL processes.
ConsultingHuman ResourcesRecruitingStaffing Agency
Hiring Manager
Pramod Singh
Responsibilities
Lead the architecture design and implementation of advanced analytics solutions using Azure Databricks Fabric
Work closely with business stakeholders and other IT teams to understand requirements and deliver effective solutions
Oversee the end to end implementation of data solutions ensuring alignment with business requirements and best practices
Lead the development of data pipelines and ETL processes using Azure Databricks PySpark and other relevant tools
Integrate Azure Databricks with other Azure services eg Azure Data Lake Azure Synapse Azure Data Factory and onpremise systems Spark expert Scala added advantage Fabric expert covering all the scenarios such as pipeline notebook development semantic modelling RTI etc Resource should also have CICD experience
Provide technical leadership and mentorship to the data engineering team fostering a culture of continuous learning and improvement Ensure proper documentation of architecture processes and data flows while ensuring compliance with security and governance standards Ensure best practices are followed in terms of code quality data security and scalability
Stay updated with the latest developments in Databricks and associated technologies to drive innovation
Qualification
Required
Lead the architecture design and implementation of advanced analytics solutions using Azure Databricks Fabric
Deep understanding of big data technologies, data engineering, and cloud computing with a strong focus on Azure Databricks along with Strong SQL
Work closely with business stakeholders and other IT teams to understand requirements and deliver effective solutions
Oversee the end to end implementation of data solutions ensuring alignment with business requirements and best practices
Lead the development of data pipelines and ETL processes using Azure Databricks, PySpark, and other relevant tools
Integrate Azure Databricks with other Azure services eg Azure Data Lake, Azure Synapse, Azure Data Factory and on-premise systems
Spark expert, Scala added advantage
Fabric expert covering all the scenarios such as pipeline, notebook development, semantic modelling, RTI etc
Resource should also have CICD experience
Provide technical leadership and mentorship to the data engineering team fostering a culture of continuous learning and improvement
Ensure proper documentation of architecture processes and data flows while ensuring compliance with security and governance standards
Ensure best practices are followed in terms of code quality, data security, and scalability
Stay updated with the latest developments in Databricks and associated technologies to drive innovation
Strong experience with Azure Databricks including cluster management, notebook development, and Delta Lake
Proficiency in big data technologies eg Hadoop, Spark and data processing frameworks eg PySpark
Deep understanding of Azure services like Azure Data Lake, Azure Synapse, and Azure Data Factory, Power BI, SSRS, SSAS, PSKU to F SKU
Experience with ETL/ELT processes, data warehousing, and building data lakes
Strong SQL skills and familiarity with NoSQL databases
Experience with CICD pipelines and version control systems like Git
Knowledge of cloud security best practices
Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders
Strong problem solving skills and a proactive approach to identifying and resolving issues
Leadership skills with the ability to manage and mentor a team of data engineers
Demonstrated expertise of 12 years in developing data ingestion and transformation pipelines using Databricks, Synapse notebooks, and Azure Data Factory
Solid understanding and hands-on experience with Delta tables, Delta Lake, and Azure Data Lake Storage Gen2
Experience in efficiently using Auto Loader and Delta Live tables for seamless data ingestion and transformation
Proficiency in building and optimizing query layers using Databricks SQL
Demonstrated experience integrating Databricks with Azure Synapse, ADLS Gen2, and Power BI for end-to-end analytics solutions
Prior experience in developing, optimizing, and deploying Power BI reports
Familiarity with modern CICD practices especially in the context of Databricks and cloud-native solutions
Preferred
Power BI for dashboarding and reporting
Microsoft Fabric for analytics and integration tasks
Spark Streaming for processing real-time data streams
Familiarity with Azure Resource Manager (ARM) templates for infrastructure as code best practices
Company
E-Solutions
E-Solutions is a talent acquisition company that offers staffing, application development, and strategic outsourcing services.
1001-5000 employees
H1B Sponsorship
E-Solutions has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (6)
2024 (11)
2023 (12)
2022 (11)
2021 (12)
2020 (16)
Funding
Current Stage
Late StageLeadership Team
ANKIT MISHRA
Head – Google Cloud Platform (GCP) - Global Delivery Practice | Client Partnering | Cloud Computing
Recent News
2024-05-06
2023-12-14
Company data provided by crunchbase