Apply on Employer Site

Ottometric · 22 hours ago

Principal Data Architect (ADAS/AV Data Platforms)

Greater Boston

Full-time

Onsite

Senior Level, Lead/Staff

5+ years exp

Ottometric is focused on autonomous vehicle data solutions, seeking a Principal Data Architect to lead their global data strategy. The role involves innovating data infrastructure and addressing complex challenges in storing and querying vast amounts of sensor data.

Information TechnologySoftware

Responsibilities

Architectural Innovation: Lead the R&D and design of a next-generation data lakehouse that supports the unique requirements of ADAS/AV, including 4D spatial-temporal querying and multi-modal data fusion

Deep Optimization: Go beyond standard implementations of Apache Iceberg to develop custom partitioning schemes, Z-ordering, and hidden indexing strategies tailored for LiDAR, radar, and video metadata

Theoretical Leadership: Apply advanced research in distributed systems to solve challenges regarding data consistency, deterministic "replay" of vehicle logs, and massive-scale data lineage

Strategic Storage R&D: Develop novel algorithms for data deduplication and "intelligent tiering," ensuring that rare "edge-case" driving data is preserved while optimizing the cost-to-performance ratio of the petabyte-scale lake

Cross-Functional Research: Partner with ML Research and Simulation teams to ensure the data architecture supports emerging paradigms like Foundation Models and End-to-End Autonomous Driving architectures

Technical Mentorship: Act as a high-level consultant and mentor to the broader Data Engineering organization, fostering an environment of analytical rigor and engineering excellence

Qualification

Apache IcebergHadoop ecosystemDistributed systemsData architectureQuery optimizationData systemsGeospatial indexingCloud deploymentsISO 26262SOTIFResearch & publicationTechnical mentorship

Required

PhD in Computer Science, Distributed Systems, Database Systems, or a related quantitative field

5+ years of experience in data systems, with a significant track record of designing large-scale distributed architectures

Deep, 'under-the-hood' knowledge of Apache Iceberg (specification and implementation) and the Hadoop ecosystem (HDFS, Spark, Trino/Presto)

Evidence of contributions to the field, such as publications in top-tier conferences (e.g., SIGMOD, VLDB, ICDE, OSDI) or a history of significant contributions to major open-source data projects

Expert-level understanding of query optimization, file format internals (Parquet/Avro), and the trade-offs of distributed consensus protocols