Apply on Employer Site

DataHub · 4 days ago

Senior Backend Engineer - Catalog

Palo Alto, California, United States

Full-time

Hybrid

Mid, Senior Level

$225K/yr - $300K/yr

4+ years exp

Acryldata is an AI & Data Context Platform adopted by over 3,000 enterprises, including Apple and Netflix. They are seeking a Senior Backend Engineer to lead the development of DataHub's Platform framework, focusing on building scalable metadata ingestion systems and clean APIs to enhance data governance and processing capabilities.

Computer Software

Responsibilities

Lead development of DataHub's Platform framework

Build scalable, fault-tolerant ingestion systems for enterprise-scale metadata

Create clean, intuitive APIs for our connector ecosystem

Develop event-driven architectures for real-time metadata processing

Implement schema mapping between diverse systems and DataHub's unified model

Design versioning systems for AI assets

Qualification

Distributed systemsAdvanced PythonAPI designHigh-scale data processingAWSKubernetes/DockerCI/CD pipelinesMicroservice ArchitectureJava/Scala/Kotlin/C#/GoPython/TypeScript/Node.jsOpen-source contributionsLLM developmentEarly-stage startup experience

Required

4+ years building production-grade distributed systems

Advanced Python and API design expertise

Experience with high-scale data processing or integration frameworks

Strong systems knowledge and distributed architecture experience

Proven track record solving complex technical challenges

Built and maintained online applications serving live traffic at scale (100+ QPS)

Set up monitoring and alerting for services

Designed indexing, storage, and data architectures to make large-scale data accessible to online services

Designed and scaled distributed systems

Hands-on experience developing in a tight loop with LLMs and applying best practices for scalable LLM development

One of Java/Scala/Kotlin/C#/Go - very strong nice-to-have / borderline must-have

Python/TypeScript/Node.js - nice-to-have

AWS

Kubernetes/Docker

CI/CD deployment pipelines

Microservice Architecture

Preferred

Experience with DataHub or similar metadata/ETL frameworks (Airflow, Airbyte, dbt)

Open-source contributions

Experience building and maintaining services that make calls to LLMs in order to serve live traffic

Experience fine-tuning LLM-powered applications exposed to end users

Early-stage startup experience

Benefits

Competitive compensation

Equity for everyone

Remote Work

Location flexibility

Comprehensive health coverage

Flexible savings accounts

Support for every path to parenthood

Time off that works for you

Company

DataHub

Founded by data industry veterans and backed by LinkedIn, DataHub enables organizations to deploy AI in production through an enterprise-grade metadata platform handling 3M+ PyPI downloads monthly.

Founded in 2021

Palo Alto, California, US

51-200 employees