Apply on Employer Site

Sev1Tech LLC · 2 weeks ago

Senior Databricks Data Engineer - Data Lake

Arlington, VA

Full-time

Onsite

Senior Level

Sev1Tech LLC is a company that provides IT, engineering, and program management solutions. They are seeking a Senior Databricks Data Engineer to take ownership of their AWS-hosted Databricks platform, focusing on operational excellence, automation, and governance.

ConsultingCyber SecurityInformation TechnologyInfrastructureSecurity

Growth Opportunities

No H1B

Security Clearance Required

U.S. Citizen Only

Hiring Manager

Amy Urquhart

Responsibilities

Serve as the hands-on administrator for an established, production AWS Databricks environment

Strengthen and mature the platform as it enters a major growth stage with increasing users, workloads, and data domains

Monitor, troubleshoot, and optimize workspace performance, cluster reliability, compute utilization, job operations, and platform health

Implement and maintain compute policies, cluster configurations, pools, serverless SQL warehouses, and workload management strategies

Lead ongoing efforts to automate provisioning, governance, monitoring, and operational processes across the platform

Manage and enhance Unity Catalog, metastore configuration, catalog hierarchies, permissions, and lineage

Define and enforce RBAC, workspace controls, access policies, and compliance frameworks

Implement security best practices across S3, IAM, KMS, secret scopes, encryption, and network configuration

Ensure adherence to FedRAMP/FISMA requirements and agency-specific data governance standards

Guide the creation of an automation roadmap for provisioning, user lifecycle, policy enforcement, workspace configuration, lineage, and monitoring

Build the frameworks and workflows required for scalable, consistent platform operations

Establish strong operational documentation, standards, and procedures

Work closely with cloud engineering teams to optimize S3, IAM, networking, logging, monitoring, and related AWS components

Enhance the platform’s integration with enterprise identity solutions (SAML, SCIM, Entra)

Partner with data engineers, analysts, and data owners to ensure best practices across jobs, clusters, Delta Lake designs, and data access patterns

Act as the platform SME, guiding teams toward secure, optimized usage of Databricks

Qualification

Databricks administrationAWS platform expertiseData governance principlesData ingestion pipelinesBig data technologiesSQL proficiencyProgramming languagesRelevant certificationsProblem-solving skillsInterpersonal skillsTechnical mentorship

Required

Bachelor's degree in computer science, information technology, or a related field. Equivalent experience will also be considered

Proven experience in building and configuring enterprise-level data lake solutions using Databricks in an AWS or Azure environment

In-depth knowledge of Databricks architecture, including workspaces, clusters, storage, notebook development, and automation capabilities

Strong expertise in designing and implementing data ingestion pipelines, data transformations, and data quality processes using Databricks

Experience with big data technologies such as Apache Spark, Apache Hive, Delta Lake, and Hadoop

Solid understanding of data governance principles, data modeling, data cataloging, and metadata management

Hands-on experience with cloud platforms like AWS or Azure, including relevant services like S3, EMR, Glue, Data Factory, etc

Proficiency in SQL and one or more programming languages (Python, Scala, or Java) for data manipulation and transformation

Knowledge of data security and privacy best practices, including data access controls, encryption, and data masking techniques

Strong problem-solving and analytical skills, with the ability to identify and resolve complex data-related issues

Excellent interpersonal and communication skills, with the ability to collaborate effectively with technical and non-technical stakeholders

Experience in a senior or lead role, providing technical guidance and mentorship to junior team members

Eligibility/Clearance Requirements : Must be eligible to obtain a Department of Homeland Security EOD clearance (Requirements 1. US Citizenship, 2. Favorable Background Investigation)

Preferred

Relevant certifications such as Databricks Certified Developer or Databricks Certified Professional are highly desirable

Active DHS/CISA suitability - 1st priority

Any DHS badge + DoD Top Secret - 2nd choice

DoD Top Secret + willingness to obtain DHS/CISA suitability - 3rd choice (it can take 10-60 days to obtain suitability – work can only begin once suitability is fully adjudicated)