Staff Software Engineer, AI Gateway
Docker · Remote · United States ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Staff level role
- Fully remote
- Go or Rust
- Distributed systems
- Visa sponsorship possible
Join Docker's globally distributed team to build the AI Gateway, a control point for managing AI agents from prototype to production. You will define the architecture for model routing, security, and observability, ensuring the system scales reliably for millions of users.
What you'll do
- Set technical direction for the AI Gateway core systems and control plane
- Identify and resolve infrastructure gaps before they impact production scale
- Solve complex problems in dynamic model selection and cost-based routing
- Serve as the expert for credential storage, caching, and rate limiting
- Build infrastructure for evaluating and backtesting AI agent behavior
- Mentor senior and mid-level engineers through code and design reviews
- Partner with product leadership to shape the future of the platform
What you'll need
- 8+ years of backend engineering experience with production-grade systems
- Deep expertise in distributed systems, cloud-native infrastructure, and APIs
- Proven track record building secure, high-throughput gateways or proxies
- Fluency in Go and/or Rust programming languages
- Ability to drive architectural fixes across team boundaries
- Strong product mindset with experience shaping technical strategy
- Bachelor's degree in Computer Science or equivalent practical experience
Nice to have
- Experience with OpenAI, Anthropic, or similar LLM APIs
- Familiarity with RAG architectures and agent frameworks like LangChain
- Knowledge of policy engines such as OPA or OpenTelemetry
- Understanding of OAuth 2.1, secret management, and cloud IAM
- Experience with Kubernetes, Docker, and microservices architecture
- History of setting technical standards adopted by other teams
Stack
- Go
- Rust
- Kubernetes
- Docker
- OpenTelemetry
- OPA
- LangChain
How you'll work
- Fully remote-first culture
- Work from anywhere globally
- Flexible schedule management
- Offices in Seattle and Paris available
What you get
- Generous PTO and quarterly Whaleness Days
- End-of-year Whaleness break
- Home office support setup
- Monthly technology stipend
- Annual learning and development budget
- 16 weeks paid parental leave
- Equity for all full-time employees
- Comprehensive medical and retirement benefits
Hiring process
- AI-assisted tools may be used in recruiting
- Optional interview recordings for calibration