ECLARO · 21 hours ago
Data Platform & Visualization Engineer
Los Altos, CA
Contract
Hybrid
Entry, Mid Level
$85/hr - $90/hrECLARO is a leading technology solutions provider seeking a Data Platform and Visualization Engineer for their client in Los Altos, CA. The role involves building and evolving an internal data platform to support vehicle testing, experimentation, and machine learning workflows, including implementing data ingestion pipelines and web-based visualization tools.
Staffing & Recruiting
Responsibilities
Implement and extend data ingestion and processing workflows for large, heterogeneous datasets collected from vehicle tests and ML pipelines
Contribute to improving orchestration, scheduling, and reliability of long-running data workflows operating under real-world constraints
Integrate downstream automation such as metric computation, plotting, and LLM-based postprocess tooling
Implement backend services and APIs that support data indexing, metadata management, and experiment tracking
Build user-facing web-based tools and dashboard that allow users to browse datasets, inspect results, and understand experimental progress over time
Work with a SQL-backed database to store metrics, experiment metadata, and summaries, ensuring the data can be queried and accessed consistently across systems
Contribute to data traceability and provenance mechanisms that capture how datasets are generated, transformed, and consumed in ML workflows
Own and extend an existing data ingestion system responsible for uploading vehicle test data to Amazon S3
Improve ingestion orchestration to support:
Upload prioritization for small datasets
Deferred upload scheduling for large datasets during off-hours
Automatic discarding of data explicitly marked as trash
Persistent queueing and resumability across server restarts or failures
Maintain ingestion reliability under constrained network bandwidth
Extend the current web interface for clarity, reliability and extendability
Integrate ingestion workflows with post-processor, such as:
Existing LLM-based automatic annotation module
Automating plot generation (You come back to automatically generated plots as soon as data hits S3 - imagine that!)
Metric computation pipelines
Package and deploy the annotation system as a service (e.g., EC2-based)
Implement orchestration logic to trigger annotation jobs opportunistically when ingestion resources are idle
Store metrics, experiment metadata, plots and summaries in SQL-backed database layer
Implement and extend a SQL-backed metrics database using schemas defined by the team
Define schemas to support:
Multiple projects
Baselines vs experimental runs
Historical comparisons
Build automated pipelines to compute and register metrics after ingestion
Implement project-level leaderboard functionality to track:
Best performance per metric
Accepted baselines vs rejected experiments
Develop a web-based visualization interface to:
Display time-series progress
Visualize metric tradeoffs
Summarize experimental outcomes
Design and implement a data provenance system for ML datasets
Track:
Source S3 URLs
Post-processing operations applied to datasets
Implement a registry of post-processing functions with support for:
Easy addition and removal
Versioning and configuration tracking
Generate human-readable dataset identifiers
Enable lookup and inspection of dataset lineage via API and/or web interface
Qualification
Required
Experience with Python for backend services, data pipelines, and automation
Working knowledge of SQL, including writing queries and understanding database schemas
Experience building web-based tools, including: Backend APIs (e.g., FastAPI, Flask, or similar), Frontend applications using React or other modern frameworks
Familiarity with AWS and cloud-based storage or services
Comfortable working in Linux environments
Preferred
Interest in autonomous racing and vehicle dynamics research
Prior internship or project experience involving data pipelines, dashboards, or analytics tools
Exposure to data visualization libraries, ML workflows, or experiment tracking systems
Benefits
401k Retirement Savings Plan administered by Merrill Lynch
Commuter Check Pretax Commuter Benefits
Eligibility to purchase Medical, Dental & Vision Insurance through ECLARO
Company
ECLARO
ECLARO is an award-winning professional services firm headquartered in New York City and operating in the U.S., Canada and the Philippines.
Founded in 1999
1001-5000 employees
H1B Sponsorship
ECLARO has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (1)
2024 (1)
2023 (1)
2022 (2)
2020 (1)
Funding
Current Stage
Late StageLeadership Team
Nicholas Butcher
QA CTO Label Specialist
Dan Broderick
Chief Delivery Officer
Company data provided by crunchbase