CitiusTech · 10 hours ago
Data Architect - Databricks
United States
Full-time
Remote
Senior Level, Lead/Staff
15+ years exp
Maximize your interview chancesAnalyticsArtificial Intelligence (AI)
Hiring Manager
Disha Jain, AIRS PRC, CDR
Insider Connection @CitiusTech
Get 3x more responses when you reach out via email instead of LinkedIn.
Responsibilities
Design and architect scalable, high-performance, and cost-effective cloud-based solutions using AWS services and Databricks.
Develop comprehensive architecture blueprints that include data pipelines, data warehousing, and advanced analytics solutions.
Ensure that architecture aligns with the business requirements and complies with industry best practices and security standards. Oversee the setup and configuration of AWS environments, including S3 buckets, and other AWS services.
Validate IAM roles and policies, ensuring secure access to AWS resources. Optimize cloud infrastructure for cost, performance, and scalability.
Architect and implement big data processing solutions using Databricks, leveraging Apache Spark for data transformation, ETL processes, and machine learning workflows.
Design data pipelines that handle large-scale data ingestion, processing, and storage.
Implement data security, data governance, and compliance measures within the Databricks environment using Unity Catalog.
Work closely with Data engineers, DevOps teams, and other stakeholders to ensure seamless integration of solutions. Collaborate with project managers and business analysts to gather and interpret requirements, ensuring technical feasibility and alignment with business objectives.
Analyze and optimize system performance, identifying bottlenecks and implementing solutions to enhance processing speed and efficiency. Implement monitoring tools and strategies to proactively manage the health and performance of the AWS and Databricks environments.
Ensure that all solutions adhere to security best practices and compliance requirements, including data encryption, access control, and logging. Implement strategies to protect sensitive data and ensure compliance with relevant regulations, such as GDPR, HIPAA, etc.
Deep understanding of AWS services, including but not limited to EC2, S3, RDS, Lambda, IAM, VPC, and CloudFormation. Proficiency in Databricks, Apache Spark, and related big data technologies.
Strong scripting skills in Python, SQL, and experience with Terraform or CloudFormation.
Excellent problem-solving skills and the ability to work in a fast-paced environment.
Experience with Kubernetes, Docker, and containerized applications. Familiarity with data lake architectures, Delta Lake, and advanced analytics solutions. Strong understanding of CI/CD pipelines, DevOps practices, and Infrastructure as Code (IaC).
Qualification
Find out how your skills align with this job's requirements. If anything seems off, you can easily click on the tags to select or unselect skills to reflect your actual expertise.
Required
15+ Years of experience
Engineering Degree – BE/ME/BTech/MTech/BSc/MSc.
Design and architect scalable, high-performance, and cost-effective cloud-based solutions using AWS services and Databricks.
Develop comprehensive architecture blueprints that include data pipelines, data warehousing, and advanced analytics solutions.
Ensure that architecture aligns with the business requirements and complies with industry best practices and security standards.
Oversee the setup and configuration of AWS environments, including S3 buckets, and other AWS services.
Validate IAM roles and policies, ensuring secure access to AWS resources.
Optimize cloud infrastructure for cost, performance, and scalability.
Architect and implement big data processing solutions using Databricks, leveraging Apache Spark for data transformation, ETL processes, and machine learning workflows.
Design data pipelines that handle large-scale data ingestion, processing, and storage.
Implement data security, data governance, and compliance measures within the Databricks environment using Unity Catalog.
Work closely with Data engineers, DevOps teams, and other stakeholders to ensure seamless integration of solutions.
Collaborate with project managers and business analysts to gather and interpret requirements, ensuring technical feasibility and alignment with business objectives.
Analyze and optimize system performance, identifying bottlenecks and implementing solutions to enhance processing speed and efficiency.
Implement monitoring tools and strategies to proactively manage the health and performance of the AWS and Databricks environments.
Ensure that all solutions adhere to security best practices and compliance requirements, including data encryption, access control, and logging.
Implement strategies to protect sensitive data and ensure compliance with relevant regulations, such as GDPR, HIPAA, etc.
Deep understanding of AWS services, including but not limited to EC2, S3, RDS, Lambda, IAM, VPC, and CloudFormation.
Proficiency in Databricks, Apache Spark, and related big data technologies.
Strong scripting skills in Python, SQL, and experience with Terraform or CloudFormation.
Excellent problem-solving skills and the ability to work in a fast-paced environment.
Experience with Kubernetes, Docker, and containerized applications.
Familiarity with data lake architectures, Delta Lake, and advanced analytics solutions.
Strong understanding of CI/CD pipelines, DevOps practices, and Infrastructure as Code (IaC).
Mandatory skills: AWS, Databricks, Unity Catalog
Preferred
Technical certification in multiple technologies is desirable.
Company
CitiusTech
Major provider of technology services and solutions to healthcare technology companies, providers, payers and life sciences organizations
5,001-10,000 employees
H1B Sponsorship
CitiusTech has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2023 (359)
2022 (385)
2021 (437)
2020 (425)
Funding
Current Stage
Late StageTotal Funding
unknownKey Investors
Bain Capital Private EquityGeneral Atlantic
2022-10-20Private Equity· undefined
2019-07-12Acquired· undefined
2014-03-20Private Equity· undefined
Leadership Team
Rajan Kohli
CEO
Sudhir Kesavan
Chief Operating Officer
Recent News
2024-05-04
Company data provided by crunchbase
