Net2Source (N2S) · 3 hours ago
Sr. SRE Engineer -(Open AI |LLM)
Bellevue, WA
Contract
Onsite
Senior Level
$70/hr - $80/hrNet2Source (N2S) is seeking a Senior Site Reliability Engineer to design and operate enterprise AI Gateway infrastructure for OpenAI and internal LLM-based services. The role involves managing regional routing, developing deployment configurations, and ensuring operational readiness through monitoring frameworks and cross-functional collaboration.
B2BDeliveryHuman ResourcesInformation Technology
Hiring Manager
Bismillah Arzoo
Responsibilities
Design, deploy, and operate enterprise AI Gateway infrastructure supporting OpenAI and internal LLM-based services
Implement and manage regional routing (east/west), failover strategies, and upstream host configurations for AI traffic
Develop and maintain Helm charts, Kubernetes manifests, and Jinja templates for multi-environment deployments (dev, plab, qlab)
Enable per-API configuration for rate limiting, AI feature toggles, security credentials, and regional host overrides
Stay current with industry best practices for: AI Gateways and MCP servers, Secure LLM consumption patterns, Token handling, secrets management, and request isolation, Observability standards for AI platforms
Lead bi-weekly technical and operational syncs with AI Gateway vendors
Translate vendor capabilities, limitations, and roadmaps into actionable platform strategies
Communicate clearly in both technical and business terms with: Engineering teams, SRE, Security & compliance, Product and leadership stakeholders
Build and maintain monitoring and troubleshooting frameworks for AI workloads using Splunk and Grafana
Author and evolve SRE support cookbooks for proactive monitoring, incident response, and escalation
Analyze failure rates, latency spikes, and request flows across distributed AI systems
Support on-call readiness through actionable dashboards, alerts, and operational runbooks
Build CI pipelines to generate and deploy environment-specific configurations at scale
Automate service registration, deployment validation, and environment promotion
Enforce consistent naming, versioning, and deployment standards across clusters and environments
Act as a technical bridge between application teams, SRE, security, and platform engineering
Provide architectural guidance for teams onboarding to AI Gateway and Enterprise GPT platforms
Contribute to platform roadmaps, technical design reviews, and operational readiness planning
Qualification
Required
Strong experience with Kubernetes, Helm, and cloud-native networking
Hands-on experience with Istio / service mesh, routing rules, and traffic management
Proficiency in Python, Bash, and Jinja templating for infrastructure automation
Experience operating production-grade APIs with high reliability and observability standards
Deep understanding of SRE principles, monitoring, alerting, and incident management
Experience building observability frameworks using Splunk, Grafana, or similar tools
Strong ability to communicate complex technical issues in clear business terms
Experience working with AI/LLM APIs (OpenAI or similar) in an enterprise context
Preferred
Knowledge of MCP servers, AI gateway patterns, and LLM security models
Familiarity with security controls for AI platforms (secrets management, token handling, access controls)
Experience supporting multi-region, multi-environment deployments at scale
Strong documentation skills with a focus on operational clarity and enablement
CICD, Helm, IT infrastructure, Jinja, Kubernetes, computer science, containerization, continuous integration, information technology, software components, software development, software engineering, templating engines, virtualization
Company
Net2Source (N2S)
Net2Source (N2S) is a Minority owned global workforce solutions company recognized by SIA as the largest and fastest-growing Total Talent Solutions provider with a presence in 34 countries and in-house Glo-Cal (global and local) teams to support our clients.
Founded in 2007
1001-5000 employees
H1B Sponsorship
Net2Source (N2S) has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (3)
2024 (7)
2022 (8)
2021 (10)
2020 (25)
Funding
Current Stage
Late StageLeadership Team
Ashish Garg
Founder & CEO
Company data provided by crunchbase