EXL · 10 hours ago

Senior Cloud Engineer

United States

Full-time

Remote

Mid, Senior Level

4+ years exp

Maximize your interview chances

AnalyticsBanking

Insider Connection @EXL

Discover valuable connections within the company who might provide insights and potential referrals.
Get 3x more responses when you reach out via email instead of LinkedIn.

Responsibilities

Design, develop, and maintain large-scale data pipelines on Databricks, leveraging Apache Spark, Scala, Python, and other relevant technologies.

Collaborate with the Data Architecture Team to design and implement scalable, secure, and performant data architectures on Azure Databricks, ensuring alignment with business objectives.

Develop and maintain data processing workflows using Azure services, including:

Azure Data Factory

Azure Databricks

Azure Synapse Analytics

Azure HDInsight

Implement and utilize Medallion Architecture for end-to-end development.

Administer and manage Databricks clusters, including configuration, monitoring, and troubleshooting.

Ensure the highest level of data quality, security, and compliance by adhering to industry standards and best practices.

Optimize data pipeline performance for efficient resource utilization and minimal latency.

Troubleshoot and resolve cloud and data pipeline issues, providing technical guidance and mentorship to junior team members.

Qualification

Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.

DatabricksApache SparkAzure cloud technologiesScalaPythonSQLData architectureData modelingGovernance principlesDatabricks Delta LakeDatabricks QueryDatabricks JobsEPIC CaboodleSlicer DicerData pipeline optimizationData warehousingETL/ELT processesUS healthcare systems