ThirdLaw | Runtime AI Safety · 2 months ago
Backend / Platform Engineer, AI Analytic Engines
United States
Full-time
Remote
Senior Level
5+ years expThirdLaw is focused on building the control layer for AI in the enterprise, addressing safety and compliance risks associated with AI agents. The role involves architecting and scaling systems for real-time evaluations, designing core evaluation engines, and creating frameworks for intervention policies.
Computer Software
Responsibilities
Architect scalable, low-latency services for running evaluations in real-time and batch, integrating with streaming data pipelines and trace-based event models
Design and build the core evaluation engine within ThirdLaw that applies heuristics, semantic models, and foundation model calls to detect violations across LLM inputs and outputs
Build a runtime intervention layer to determine and execute appropriate enforcement actions—such as block, redact, notify, escalate—based on evaluation results and risk context
Create reusable frameworks for pluggable evaluators and intervention policies, supporting no-code authoring and automated deployment pipelines
Configure and operationalize a vector database pipeline for RAG-like use cases
Build for scale. Mitigate blocking gRPC threads, implement micro-batching & streaming for LLM/embedding calls and add reliability controls such as queue-based back-pressure and graceful degradation paths
Qualification
Required
5+ years of Backend software engineering experience, including designing and shipping production software services
Strong coding proficiency in Python and/or Go
Deep experience with streaming data pipelines (e.g. Kafka, Pulsar, Redis Streams) and batch processing systems
Proven track record scaling high-QPS, low-latency services (p95/p99 ownership a plus)
Familiarity with vector databases (e.g. FAISS, Weaviate, Qdrant, pgvector) and embedding-based matching
Strong grasp of cloud-native infrastructure: containers, Kubernetes, serverless functions, CI/CD pipelines
Exposure to structured observability patterns, including OpenTelemetry (or similar tracing standards)
Comfortable designing—then defending—trade-offs around build vs buy vs OSS
Preferred
Experience with modern Python APIs & concurrency, e.g. gRPC, FastAPI, asyncio, multithreading/processes
Familiarity with ClickHouse, Apache Arrow, or fast analytical storage engines
Prior work on agent frameworks (e.g. LangChain, CrewAI, AutoGen) or LLM orchestration
Experience in trust & safety, compliance, or AI safety domains
Hands-on experience with secure enterprise integrations (authorization/authentication, webhooks, SIEM, IAM)
Benefits
Market cash compensation
Above-market equity
Well-designed and generous benefits
Company
ThirdLaw | Runtime AI Safety
ThirdLaw is the runtime AI safety platform for IT and Security teams.
2-10 employees
Funding
Current Stage
Early StageCompany data provided by crunchbase