Index Analytics LLC · 11 hours ago
Databricks Lead
United States
Full-time
Remote
Senior Level, Lead/Staff
$166K/yr - $220K/yr
8+ years expIndex Analytics, LLC is a rapidly growing, Baltimore-based small business providing health-related consulting services to the federal government. The Databricks Lead will lead the development and implementation of Databricks pipelines across multiple environments to support advanced data analytics, working as part of a cross-functional project team.
Enterprise SoftwareBig DataInformation TechnologyAnalyticsCRMData IntegrationData VisualizationInformation Services
No H1B
U.S. Citizen Only
Responsibilities
Architect, develop and implement data models and data engineering pipelines to support ingestion of diverse CMS data sources, applying complex business rules derived from Python/R code reviews and enable analytics through optimized Apache Spark datasets while maintaining strong security and governance practices
Engineer and manage end‑to‑end data processing workflows including receiving data files in multiple formats, applying validated business logic, loading transformed data into production pipelines, delivering recurring and ad‑hoc data extracts to meet stakeholder and program needs
Support and administer the Databricks environment including user and access control management for clusters, jobs, notebooks, DDL deployments, optimizing performance, monitoring usage and delivery of cost and utilization reporting
Ensure data quality and maintain documentation conducting comprehensive QA validation on all ingested and delivered datasets, managing codebooks and user training guides as well as maintaining complete metadata, source‑to‑target mappings and technical artifacts according to established schedules
Qualification
Required
U.S. citizen or otherwise authorized to work in the United States and able to demonstrate physical residency in the U.S. for at least three (3) of the past five (5) years
Bachelor's degree or higher in computer science or relevant discipline with eight (8) or more years of professional work experience, including six (6) or more years of data engineering experience designing and building pipelines across a variety of database technologies
Three (3) or more years of hands‑on experience with the Databricks Platform and developer tools including performance tuning, use of Databricks CLI and REST APIs, and cost optimization for Databricks compute and storage
Two (2) or more years of hands‑on programming experience with SAS, Python, JavaScript or similar tools
Working experience with Databricks integrations, including Delta Lake, LakeFlow and Lakehouse Federation
Proven experience with Databricks Unity Catalog and related governance/security frameworks as well as working knowledge of database security, including Access Control Lists (ACLs), Unity Catalog permissions, and RBAC controls
Proficiency with cloud platforms such as AWS or Azure, including database deployment automation and CI/CD pipelines in an AWS environment
Strong communication skills and commitment to continuous learning
Preferred
Databricks certified Data Engineer certifications are a plus
Experience working with unstructured and semi‑structured data (XML, JSON, Parquet, video, audio, images, PDF) is a plus
Benefits
Health and retirement benefits
Discretionary bonuses
Reimbursement for professional development opportunities
Company
Index Analytics LLC
Index Analytics LLC (Index) was established in May 2012 in Baltimore, Maryland, and currently supports mission-critical high visibility healthcare programs nationwide in collaboration with six federal health agencies.
201-500 employees
Funding
Current Stage
Growth StageLeadership Team
Raghu Akkapeddi
Founder & CEO
Recent News
Company data provided by crunchbase