Apply on Employer Site

Infinitive · 1 month ago

Databricks Platform Architect - Unity Catalog Enablement

McLean, VA

Full-time

Onsite

Senior Level, Lead/Staff

8+ years exp

Infinitive is a data & AI consultancy that enables global brands to deliver results through insights, innovation, and efficiency. The Databricks Platform Architect will define and shape a unified platform service that ensures scalable, governed, and cost-efficient data access across the bank.

AdvertisingInformation TechnologyInternetMarketing

Responsibilities

Define and champion the end-to-end architecture for the bank’s Databricks-based data platform, ensuring scalability, security, cost efficiency, and ease of adoption

Design a self-service platform layer that leverages Databricks Unity Catalog to deliver seamless data discovery, access, and observability across all environments

Establish architectural patterns and reference implementations that encourage enterprise-wide reuse and standardization

Lead the design and implementation of Databricks Unity Catalog as the central governance plane—defining catalog hierarchies, fine-grained access controls, and cross-environment lineage

Evaluate and implement metadata, RBAC/ABAC, and data masking capabilities to meet regulatory and compliance requirements (e.g., GLBA, GDPR, HIPAA)

Define the template architecture that allows Unity Catalog to operate as a scalable and cost-effective shared service across lines of business

Engineer platform capabilities that provide deep visibility into compute, storage, and catalog operations through integrated observability, monitoring, and FinOps practices

Develop resource optimization strategies to balance performance and cost while maintaining compliance and SLAs

Establish metrics, dashboards, and alerts to ensure the platform scales predictably under enterprise workloads

Architect streamlined RESTful/GraphQL APIs for secure, governed data access and metadata integration

Ensure interoperability with enterprise systems, APIs, and external data consumers using modern, consistent, and documented integration patterns

Guide teams in building Lakehouse-aligned data models that maximize reuse and governance

Oversee design of ETL/ELT architectures (Spark, PySpark, SQL) that integrate seamlessly with Unity Catalog for lineage and access tracking

Partner with engineering, data science, and risk teams to align platform design with business outcomes and regulatory expectations

Influence architecture steering committees and platform engineering groups to adopt the Databricks foundation as a managed, enterprise-wide service

Promote a culture of easy adoption through clear design patterns, documentation, and working sessions

Mentor engineers and architects on Databricks, Unity Catalog, and best practices for cost, scale, and observability

Contribute to internal architecture communities and upskill teams across multiple domains

Qualification

DatabricksUnity CatalogData ArchitectureAPI ArchitecturePythonSQLData GovernanceInfluence SkillsCommunication SkillsCollaboration Skills

Required

Education: Bachelor's or Master's degree in Computer Science, Engineering, or related field

Experience: 8+ years in data architecture or platform engineering, including experience designing enterprise-scale, distributed data environments

Databricks Expertise: Deep hands-on knowledge of Databricks, Delta Lake, Apache Spark, and Lakehouse principles

Unity Catalog Mastery: Demonstrated success architecting and operationalizing Databricks Unity Catalog for enterprise governance, metadata management, and access control

Programming & Data: Advanced proficiency in Python (PySpark) and SQL; experience with cloud data platforms (AWS, Azure, or GCP)

API Engineering: Strong background in API architecture (REST, GraphQL, OpenAPI) and applying best-in-class security and observability

Governance Knowledge: Expert-level understanding of data governance frameworks, data quality management, and regulatory compliance

Soft Skills: Outstanding communication and influence skills, with ability to advocate for design principles across executive, technical, and risk audiences