Eon Systems PBC · 2 months ago
Data/Full Stack Engineer, Data Storage & Ingestion Consultant
San Francisco, CA
Full-time
Onsite
Senior Level
$100/hr - $300/hr
5+ years expEon Systems PBC is at the forefront of large-scale neuroscientific data collection, focusing on the development of brain emulation technology. They are seeking a Data/Full Stack Engineer to design and implement an end-to-end data pipeline for high-rate data ingestion and multi-petabyte storage, ensuring optimal performance and reliability.
Biotechnology
Responsibilities
Architect ingest & storage: Choose and implement an on-prem hardware and data pipeline design or a cloud/S3 alternative with explicit cost and performance tradeoffs at multi-petabyte scale
Set up a sustained-write ingest path ≥1 GB/s with adequate burst headroom (camera/frame-to-disk), including networking considerations, cooling, and throttling safeguards
Optimize footprint & cost: Incorporate on-the-fly compression/downsampling options and quantify CPU budget vs. write-speed tradeoffs; document when/where to compress to control $/PB
Integrate with acquisition workflows ensuring image data and metadata are compatible with downstream stitching/flat-field correction pipelines
Enable downstream compute: Expose the data to segmentation/analysis stacks (local GPU nodes or cloud)
Qualification
Required
5+ years designing and deploying high-throughput storage or HPC pipelines (≥1 GB/s sustained ingest) in production
Deep hands-on with: NVMe RAID/striping, ZFS/MDRAID/erasure coding, PCIe topology, NUMA pinning, Linux performance tuning, and NIC offload features
Proven delivery of multi-GB/s ingest systems and petabyte-scale storage in production (life-sciences, vision, HPC, or media)
Experience building tiered storage systems (NVMe → HDD/object) and validating real-world throughput under sustained load
Practical S3/object-storage know-how (AWS S3 and/or on-prem S3-compatible systems) with lifecycle, versioning, and cost controls
Data integrity & reliability: snapshots, scrubs, replication, erasure coding, and backup/DR for PB-scale systems
Networking: ••••25/40/100 GbE (SFP+/SFP28), RDMA/ RoCE/iWARP familiarity; switch config and path tuning
Ability to spec and rack hardware: selecting chassis/backplanes, RAID/HBA cards, NICs, and cooling strategies to prevent NVMe throttling under sustained writes
Preferred
Experience with microscopy or scientific imaging ingest at frame-to-disk speeds, including Micro-Manager-based pipelines and raw-to-containerized format conversions
Experience with life science imaging data a plus
Company
Eon Systems PBC
2-10 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase