Elloe AI | Immune System for AI · 5 months ago
Principal Engineer – Distributed Systems (GPU Edge + Inference)
United States
Full-time
Remote
Lead/StaffElloe AI is a company that serves as the trust layer for AI, ensuring safety and compliance for institutions like hospitals and banks. They are seeking a Principal Engineer to lead their GPU-edge inference systems, focusing on global infrastructure that enhances AI performance and compliance.
Customer ServiceSaaSSoftware
Responsibilities
Design zone-routing that ensures <50ms SLA in 10+ regions
Build fallback orchestration to handle compliance-aware rollbacks
Maximize utilization across 100K+ GPUs via mesh & load prediction
Integrate compliance overlays with VaultChain and SHAP triggers
Ship `/vault/audit`, `/inference/predict`, `/compliance/log` endpoints
Trace every edge request across governance and model layers
Qualification
Required
Senior systems engineer with GPU fleet experience (KubeRay, Istio, Envoy)
Operated real-time AI infra with 10M+ QPS loads
Comfortable with compliance observability and infra governance
Preferred
Timezone overlap with NY or EU preferred
Benefits
Top of market salary
Equity
Company
Elloe AI | Immune System for AI
Elloe AI is the immune system for AI — the real-time compliance layer making GenAI safe to deploy across regulated industries.
Founded in 2022
11-50 employees
Funding
Current Stage
Early StageTotal Funding
$1MKey Investors
Mad Ventures
2022-04-06Pre Seed· $1M
Company data provided by crunchbase