Bright Vision Technologies · 6 hours ago
Databricks Platform Engineer
United States
Full-time
Remote
Mid LevelBright Vision Technologies is a forward-thinking software development company dedicated to building innovative solutions that help businesses automate and optimize their operations. They are seeking a skilled Databricks Platform Engineer to architect and implement scalable data solutions, design ETL/ELT pipelines, and drive performance optimization on the Databricks Lakehouse Platform.
Artificial Intelligence (AI)Cyber SecurityInformation TechnologySoftware
Responsibilities
Architect and implement scalable data solutions on the Databricks Lakehouse Platform, utilizing Apache Spark and PySpark for distributed processing at scale
Design and execute ETL/ELT pipelines with SQL and Delta Lake, ensuring ACID transactions, schema evolution, and reliable data ingestion from sources like AWS S3, Azure ADLS, and Google Cloud Storage (GCS)
Build and orchestrate Databricks Workflows for automated job scheduling, dependency management, and end-to-end data pipeline execution
Develop advanced Data Modeling strategies, including dimensional modeling and unified analytics layers, optimized for Databricks' lakehouse architecture
Drive Performance Optimization through Spark tuning, caching strategies, and Delta Lake optimizations to handle petabyte-scale workloads efficiently
Integrate with Cloud Platforms (AWS, Azure, GCP) for seamless hybrid deployments, leveraging native storage and compute resources
Implement CI/CD pipelines with Git for version-controlled Databricks notebooks, clusters, and workflows, enabling rapid iteration and deployment
Enforce Security & Access Controls using Unity Catalog, row/column-level security, and fine-grained permissions to protect sensitive data assets
Collaborate in Agile methodologies, contributing to sprint planning, code reviews, and iterative delivery of data engineering features
Monitor, troubleshoot, and scale Databricks environments, delivering cost-effective, high-performance data platforms that power business intelligence and ML use cases
Qualification
Required
Architect and implement scalable data solutions on the Databricks Lakehouse Platform, utilizing Apache Spark and PySpark for distributed processing at scale
Design and execute ETL/ELT pipelines with SQL and Delta Lake, ensuring ACID transactions, schema evolution, and reliable data ingestion from sources like AWS S3, Azure ADLS, and Google Cloud Storage (GCS)
Build and orchestrate Databricks Workflows for automated job scheduling, dependency management, and end-to-end data pipeline execution
Develop advanced Data Modeling strategies, including dimensional modeling and unified analytics layers, optimized for Databricks' lakehouse architecture
Drive Performance Optimization through Spark tuning, caching strategies, and Delta Lake optimizations to handle petabyte-scale workloads efficiently
Integrate with Cloud Platforms (AWS, Azure, GCP) for seamless hybrid deployments, leveraging native storage and compute resources
Implement CI/CD pipelines with Git for version-controlled Databricks notebooks, clusters, and workflows, enabling rapid iteration and deployment
Enforce Security & Access Controls using Unity Catalog, row/column-level security, and fine-grained permissions to protect sensitive data assets
Collaborate in Agile methodologies, contributing to sprint planning, code reviews, and iterative delivery of data engineering features
Monitor, troubleshoot, and scale Databricks environments, delivering cost-effective, high-performance data platforms that power business intelligence and ML use cases
Company
Bright Vision Technologies
Bright Vision Technologies is an information technology company that offers software development, AI, and cybersecurity services.
Founded in 2020
51-200 employees
H1B Sponsorship
Bright Vision Technologies has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (41)
2024 (14)
2023 (7)
2022 (12)
2021 (1)
Funding
Current Stage
Growth StageCompany data provided by crunchbase