Cohere · 9 hours ago
Site Reliability Engineer, Inference Infrastructure
San Francisco, CA
Full-time
Hybrid
Senior Level
5+ years expCohere is on a mission to scale intelligence to serve humanity by training and deploying frontier models for AI systems. The Site Reliability Engineer will be responsible for developing, deploying, and operating the AI platform that delivers large language models through API endpoints, ensuring high performance and reliability.
Artificial Intelligence (AI)Foundational AIGenerative AIMachine LearningNatural Language Processing
Responsibilities
Build self-service systems that automate managing, deploying and operating services
This includes our custom Kubernetes operators that support language model deployments
Automate environment observability and resilience. Enable all developers to troubleshoot and resolve problems
Take steps required to ensure we hit defined SLOs, including participation in an on-call rotation
Build strong relationships with internal developers and influence the Infrastructure team’s roadmap based on their feedback
Develop our team through knowledge sharing and an active review process
Qualification
Required
5+ years of engineering experience running production infrastructure at a large scale
Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
Experience with Kubernetes dev and production coding and support
Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
Experience in compute/storage/network resource and cost management
Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
The grit and adaptability to solve complex technical challenges that evolve day to day
Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference
Strong understanding or working experience with distributed systems
Experience in Golang, C++ or other languages designed for high-performance scalable servers
Benefits
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
6 weeks of vacation (30 working days!)
Company
Cohere
Cohere is an enterprise AI firm developing secure and private AI technology to address real-world business challenges.
201-500 employees
H1B Sponsorship
Cohere has a track record of offering H1B sponsorships. Please note that this does not
guarantee sponsorship for this specific role. Below presents additional info for your
reference. (Data Powered by US Department of Labor)
Distribution of Different Job Fields Receiving Sponsorship
Represents job field similar to this job
Trends of Total Sponsorships
2025 (11)
2024 (14)
2023 (13)
2022 (5)
2021 (2)
Funding
Current Stage
Late StageTotal Funding
$1.71BKey Investors
Government of CanadaTiger Global ManagementIndex Ventures
2025-09-24Series D· $100M
2025-08-14Series D· $500M
2025-06-17Secondary Market
Leadership Team
Aidan Gomez
cofounder + ceo
Ivan Zhang
Co-Founder
Recent News
Beyond Bylines
2026-01-11
2026-01-06
Crunchbase News
2026-01-06
Company data provided by crunchbase