ngrok · 1 day ago
Software Engineer III/Senior, AI Gateway
San Francisco, CA
Full-time
Hybrid
Mid, Senior Level
$180K/yr - $225K/yrngrok Inc. is an all-in-one cloud networking platform that secures, transforms, and routes traffic to services running anywhere. They are seeking a Software Engineer to build and evolve the AI Gateway, which involves designing AI traffic policies, tracking usage, and ensuring the reliability of AI systems in production.
AppsDeveloper APIsInternetSoftware
No H1B
Responsibilities
Build and evolve the AI Gateway: You’ll work on the AI-aware gateway components that classify and handle AI traffic in real time. This code runs directly in the request path and must be fast, safe, and predictable
Own AI traffic policy enforcement: You’ll design and implement AI Gateway Traffic Policy Objects—rate limits, usage caps, and access rules specific to AI workloads. These policies exist to prevent runaway costs, misuse, and accidental exposure without breaking legitimate traffic
Track AI usage and token consumption: You’ll build and maintain systems that accurately measure AI usage—requests, tokens, and related metadata—so customers can understand how their AI systems behave and what they’re consuming
Make AI behavior observable and explainable: You’ll expose clear, trustworthy signals around AI traffic: what was allowed or blocked, which policies applied, and how usage accumulated. When customers ask “what happened?”, the gateway should already know
Design abstractions that hide complexity: You’ll work with product and design to build AI-specific gateway primitives that feel intentional and safe, without leaking provider quirks or infrastructure details into customer workflows
Ship systems customers trust in production: You’ll collaborate closely with Gateway, Customer Data, and Platform teams to ensure AI usage data, policy enforcement, and billing signals line up—so customers can turn these features on with confidence
Qualification
Required
You're comfortable in a statically typed, compiled language such as Go, Rust, C++, or Java (with bonus points for Go)
Preferred
You've worked with AI/LLMs and can appreciate their unique brand of edge-cases
You care about developer experience and thoughtful abstractions
You enjoy defining system behavior, not just plumbing
You've thought about retries, limits, and costs before being asked
You like systems that move complexity from the user to the system
Extra credit if you've worked on AI platforms or inference infrastructure
Extra credit if you've worked on API gateways with product-level opinions
Extra credit if you've worked on usage limits, quotas, or billing-adjacent systems
Extra credit if you've worked on customer-facing observability tools
Benefits
We provide a 401(k) with a 100% match up to 3% of your salary and a 50% match up to another 2%.
We provide healthcare, dental, and vision with premiums fully covered on the base plan for employees. Half of premiums are covered for dependents.
We offer unlimited PTO and a culture in which the overwhelming majority of employees take more than four weeks.
Company
ngrok
Ngrok is an API-first ingress-as-a-service platform that helps developers build their apps faster and more securely with no code changes.
Founded in 2015
51-200 employees
Funding
Current Stage
Growth StageTotal Funding
$50MKey Investors
Lightspeed Venture Partners
2022-12-13Series A· $50M
Leadership Team
Alan Shreve
Chief Executive Officer
Peter Shafton
Chief Technology Officer
Recent News
2025-12-05
2025-10-14
Company data provided by crunchbase