Site Reliability Engineer, Inference Infrastructure
Cohere · Remote · Toronto · San Francisco · New York · Montreal ·
COMPENSATION
$190k – $260k
Listed on the employer's job board (Ashby)
- Fully remote friendly
- Kubernetes + GPU workloads
- Senior level role
- Global company
- On-call rotation
Join the Model Serving team at Cohere to build and operate the infrastructure behind enterprise AI models. You will focus on deploying optimized NLP models to production, ensuring low latency and high availability for customer-facing API endpoints.
The work requires close collaboration with internal developers to automate service management and create customized deployments. You will help define the next generation of AI platforms while maintaining rigorous reliability standards.
What you'll do
- Build self-service systems to automate service deployment and operations
- Develop custom Kubernetes operators for language model deployments
- Automate observability and resilience to aid developer troubleshooting
- Ensure defined SLOs are met through monitoring and on-call participation
- Influence infrastructure roadmap based on feedback from internal teams
- Create customized deployments to meet specific customer requirements
- Share knowledge and participate in active code review processes
What you'll need
- 5+ years of experience running production infrastructure at scale
- Experience designing distributed systems with Kubernetes and GPU workloads
- Proficiency in Kubernetes development, production coding, and support
- Experience with GCP, Azure, AWS, OCI, or hybrid cloud serving
- Strong background in complex Linux-based computing environments
- Knowledge of compute, storage, and network resource cost management
- Familiarity with accelerators like GPUs or TPUs and their latency impact
- Experience with Golang, C++, or similar high-performance languages
Stack
- Kubernetes
- GCP
- Azure
- AWS
- OCI
- Linux
- Golang
- C++
How you'll work
- Remote-friendly with offices in major global cities
- On-call rotation required for service reliability
- $500 home office stipend provided
- Co-working benefit available for remote employees
What you get
- Weekly lunch stipend equivalent to $75 or £75
- Full health and dental benefits including mental health budget
- Retirement matching via RRSP, 401K, or Pension Scheme
- 100% parental leave top-up for up to six months
- Six weeks of paid vacation annually
- Annual enrichment credits for arts, fitness, and learning
- Budget for travel to other offices and annual company offsite