United IT · 2 days ago
Databricks Architect
Los Angeles, CA
Full-time
Hybrid
Senior Level, Lead/Staff
12+ years expUnited IT is seeking a Databricks Architect responsible for designing, implementing, and optimizing scalable data analytics and data engineering solutions on the Databricks Lakehouse Platform. The role involves collaborating with cross-functional teams to define data strategies and ensure platform reliability while enabling advanced analytics, machine learning, and business intelligence use cases.
ConsultingInformation TechnologyWeb Development
Responsibilities
Design end-to-end Databricks Lakehouse architectures for data ingestion, processing, storage, and consumption
Define and implement Delta Lake patterns, including medallion architecture (Bronze/Silver/Gold)
Develop scalable data pipelines using PySpark, Spark SQL, and Databricks workflows
Architect solutions for structured, semi-structured, and unstructured data
Build robust ETL/ELT pipelines with Databricks notebooks, jobs, and workflows
Design and implement high-performance streaming solutions using Structured Streaming
Optimize Spark jobs for cost, performance, and scalability
Implement CI/CD and automation using Databricks Repos, Git, and DevOps pipelines
Architect solutions across Azure/AWS/GCP leveraging native cloud services (e.g., Azure Data Factory, AWS Glue, GCP Dataflow)
Ensure security, governance, and compliance through Unity Catalog, RBAC, and encryption
Monitor workloads and optimize cluster configurations for performance and cost
Work closely with data engineers, data scientists, BI teams, and business stakeholders
Act as a subject matter expert (SME) for Databricks best practices, standards, and patterns
Conduct architectural reviews and guide teams on design decisions
Lead PoCs, evaluate new features, and drive platform adoption
Define standards for data quality, lineage, observability, and governance
Implement automated testing frameworks for pipelines and notebooks
Establish performance baselines and monitoring dashboards
Qualification
Required
12+ Years of experience
Design end-to-end Databricks Lakehouse architectures for data ingestion, processing, storage, and consumption
Define and implement Delta Lake patterns, including medallion architecture (Bronze/Silver/Gold)
Develop scalable data pipelines using PySpark, Spark SQL, and Databricks workflows
Architect solutions for structured, semi-structured, and unstructured data
Build robust ETL/ELT pipelines with Databricks notebooks, jobs, and workflows
Design and implement high-performance streaming solutions using Structured Streaming
Optimize Spark jobs for cost, performance, and scalability
Implement CI/CD and automation using Databricks Repos, Git, and DevOps pipelines
Architect solutions across Azure/AWS/GCP leveraging native cloud services (e.g., Azure Data Factory, AWS Glue, GCP Dataflow)
Ensure security, governance, and compliance through Unity Catalog, RBAC, and encryption
Monitor workloads and optimize cluster configurations for performance and cost
Work closely with data engineers, data scientists, BI teams, and business stakeholders
Act as a subject matter expert (SME) for Databricks best practices, standards, and patterns
Conduct architectural reviews and guide teams on design decisions
Lead PoCs, evaluate new features, and drive platform adoption
Define standards for data quality, lineage, observability, and governance
Implement automated testing frameworks for pipelines and notebooks
Establish performance baselines and monitoring dashboards
7+ years of experience in data engineering/architecture
3+ years of hands-on experience with Databricks
Strong expertise in Spark, PySpark, SQL, and distributed data processing
Deep understanding of Delta Lake features: ACID transactions, OPTIMIZE, ZORDER, Auto Loader
Experience with workflow orchestration, jobs, and Databricks REST APIs
Hands-on expertise with at least one cloud platform: Azure (preferred): ADF, ADLS, Key Vault, Event Hub, Azure DevOps; AWS: S3, Glue, Lambda, Kinesis; GCP: GCS, Dataflow, Pub/Sub
Familiarity with CI/CD, Git, DevOps, and Infrastructure-as-Code (Terraform preferred)
Strong analytical and problem-solving skills
Excellent communication and stakeholder management
Ability to lead design discussions and guide technical teams
Strong documentation and architectural blueprinting skills
Preferred
Databricks certifications, such as: Databricks Certified Data Engineer Professional, Databricks Certified Machine Learning Professional, Databricks Lakehouse Fundamentals
Experience with MLflow, Feature Store, or MLOps workflows
Experience working in regulated industries (BFSI, healthcare, etc.)
Company
United IT
United IT Solutions is a multi-disciplinary IT solutions company based in Irving, Texas, offering robust application development services, talent transformation and management, and continuous improvement and support.
51-200 employees
H1B Sponsorship
United IT has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (45)
2024 (39)
2023 (26)
2022 (17)
2021 (37)
2020 (31)
Funding
Current Stage
Growth StageLeadership Team
Ajay Govada
Chief Information Officer
Company data provided by crunchbase