Apply on Employer Site

SMBC Group · 1 week ago

Cloud Data Specialist (Azure) - Vice President

Charlotte, NC

Full-time

Hybrid

Mid, Senior Level

7+ years exp

SMBC Group is a top-tier global financial group offering diverse financial services. They are seeking a Cloud Data Specialist with strong Azure DataFactory and Databricks knowledge to support Azure-based data integration and analytics pipelines, ensuring uptime and performance of critical workflows.

AdviceBankingFinancial Services

H1B Sponsor Likely

Responsibilities

Monitor, troubleshoot and support ADF pipelines and Databricks notebooks/jobs in Production

Extensive experience on Cloud solutions, specifically in Azure

Experience with Azure cloud services, Azure Data Factory, Gen 2, Azure Databases, Functions, Databricks, or similar technology

Good understanding of ETL/ELT

Analyze pipeline failures, spark job issues, data mismatches, cluster timeouts, resource unavailability and latency bottlenecks

Root cause analysis for incidents and outages

Understand the ADF Components and architecture

Knowledge of data integration techniques and best practices

Experience with connection to various data sources and destinations

Ability to orchestrate complex data workflows and transformation

Monitoring and troubleshooting data pipeline executions

Familiarity with ADF data flow activities for data transformations

Version control and deployment management using Azure DevOps or similar tools

Awareness of ADF Integration with Azure services like Azure Data Lake Storage, Azure Databricks, etc

Skills in implementing streaming and batch data ingestion using Delta Lake

Skills in implementing data pipelines and workflows in Databricks

Familiarity with Databricks notebooks for interactive data exploration and development

Integrating Databricks with Azure Services like ADLS Gen2, Azure SQLdb

Monitoring and optimizing Databricks jobs for cost efficiency

Proficiency in Git for managing code repositories, including branching, merging and pull requests

Support CI/CD pipelines for deployment using Azure DevOps

Participate in on-call rotation and ensure business continuity via proper DR strategies

Experience with RDBS systems like Azure SQL DB, Oracle, and NoSQL Databases like MongoDB

Understanding of indexing, partitioning, and other optimization techniques

Experience with stored procedure, functions, and triggers

Ensure High Availability (HA) of ADF pipelines and auto-scaling/failover readiness of Databricks clusters

Manage alerts, incidents and escalations using ServiceNow, Azure Monitor, Log Analytics etc

Proficiency in Git for managing code repositories, including branching, merging and pull requests

Experience with Confluence, ServiceNow & JIRA

Review and provide feedback on core code changes and support production deployment

Knowledge on ETL DataStage Application would be a plus

Azure Monitor, Application Insights, Log Analytics

Cluster and pipeline-level metrics and logs

Qualification

Azure Data FactoryAzure DatabricksSQLAzure cloud servicesETL/ELTData integration techniquesAzure DevOpsPythonPowerShellBashGitServiceNowJIRAConfluence

Required

7 years of experience

Strong hands-on experience with Azure Data Factory (ADF) - Pipeline orchestration, linked services, integration runtimes

Experience in Azure Data Bricks – running and debugging notebooks, clusters, spark job logs

Proficient in SQL – Writing/debugging queries, validating data

Good understanding of Azure Services: ADLS, Key Vault, Azure Monitor, Log Analytics

Familiarity with Azure DevOps pipelines and Git Integrations

Scripting knowledge: Python, PowerShell, or Bash

Ability to work on weekends for maintenance, production implementations, recovery tests and system verifications / validations

Ability to address production issues from home outside of normal business hours