Apply on Employer Site

Apex Systems · 1 day ago

Lead Data Engineer, Databricks Migration

San Francisco, CA

Full-time

Onsite

Senior Level, Lead/Staff

$70/hr - $80/hr

8+ years exp

Apex Systems is a world-class IT services company that serves thousands of clients across the globe. We are seeking a highly skilled Lead Data Engineer with deep hands-on experience in modern data engineering, BI platforms, and large-scale data migrations. This role requires strong technical leadership and end-to-end solution design capabilities, working closely with both onshore and offshore engineering teams.

Human ResourcesInformation TechnologyRecruiting

H1B Sponsor Likely

Responsibilities

Design scalable, future‑state data architectures and data models

Translate business requirements into logical and physical data models

Ensure data quality, governance, and reduced redundancy

Lead data integration from diverse sources (databases, warehouses, external systems)

Design and implement ETL/ELT pipelines to ensure data accuracy, consistency, and availability

Evaluate and implement BI tools, reporting platforms, and analytics technologies

Conduct POCs and guide tool configuration and rollout

Lead development of analytical solutions using SQL, Databricks, Tableau, and related technologies

Monitor and tune performance across data pipelines and BI environments

Optimize queries, data structures, and caching strategies

Serve as the technical lead to offshore engineering teams

Work cross‑functionally with product, engineering, data, and business stakeholders

Translate complex concepts for both technical and non‑technical audiences

Participate in design reviews, code reviews, and solution walkthroughs

Stay current on modern data engineering trends and best practices

Identify opportunities for automation, optimization, and innovation

Establish and maintain data governance policies

Define access controls, data privacy practices, and retention standards

Experience writing technical documentation, including design docs and source‑to‑target mappings

Ability to convert business requirements into clear, actionable Jira stories

Strong analytical skills: capable of working with large datasets to identify trends and insights

Experience building highly scalable BI solutions and distributed data applications

Comfortable working in a fast‑paced, agile, startup‑like environment

Strong interpersonal skills; collaborative, clear communicator, and able to mentor others

Self‑motivated, organized, and capable of managing multiple priorities

Enjoys contributing to a positive, fun, high‑performing team culture

Qualification

DatabricksSQLData ArchitecturePythonETL ToolsCloud PlatformsBI ToolsKafkaAPIsCI/CDScalaJavaR

Required

8–10 years of software development and data engineering experience

Minimum 7 years hands-on experience with SQL, Databricks, ADF, Datastage (or other ETL tools)

Minimum 7 years hands-on experience with SSAS cubes, Cognos, Tableau, ThoughtSpot, and other BI tools

Advanced SQL and Python proficiency

Strong database foundation across SQL and NoSQL systems

Experience writing SQL for raw data processing, Kafka ingestions, ADF pipelines, data validation, and QA

Familiarity with APIs for ingestion and data integration

Experience with modern cloud platforms (Azure, AWS, or equivalent)

Experience with Spark, Hive, and other distributed compute engines

Proficiency with CI/CD frameworks such as GitHub, Jenkins, etc

Programming/scripting familiarity: Scala, Java, Python, R (any combination)