Senior Software Engineer, AI Gateway
Docker · Remote · United States ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Senior Backend Engineer
- Fully remote (global)
- Go and/or Rust
- AI/ML infrastructure
- Visa sponsorship possible
Join Docker's globally distributed team to engineer the AI Gateway, which simplifies moving AI agents from prototype to production. You will build the infrastructure for model routing, security, and observability, enabling developers to create trustworthy autonomous workflows without managing complex underlying systems.
What you'll do
- Design core systems including the model router, MCP gateway, and control plane
- Build infrastructure for dynamic model selection, cost-based routing, and failover
- Implement secure credential storage, session summarization, caching, and rate limiting
- Develop APIs compatible with OpenAI interfaces and the Model Context Protocol
- Create tooling for evaluation, telemetry, replay, and backtesting of LLM workflows
- Lead architectural decisions and mentor engineers as the team scales
- Tune performance to maintain low latency under high production load
What you'll need
- 6+ years of backend engineering experience with production-grade systems
- Deep knowledge of distributed systems, cloud-native infrastructure, and API design
- Experience building secure, high-throughput services like gateways or proxies
- Fluency in Go and/or Rust
- Familiarity with AI/ML platforms or model serving infrastructure
- Bachelor's degree in Computer Science or equivalent practical experience
- Strong product mindset focused on building developer-facing tools
Nice to have
- Experience with OpenAI, Anthropic, or similar LLM APIs
- Familiarity with RAG architectures, vector databases, or agent frameworks
- Experience with policy engines like OPA or observability frameworks
- Understanding of OAuth2.1, secret management, and cloud IAM systems
- Experience with Kubernetes, Docker, and microservices architecture
Stack
- Go
- Rust
- OpenAI API
- Model Context Protocol
- Kubernetes
- Docker
- OPA
- OpenTelemetry
How you'll work
- Fully remote-first culture with global distribution
- Flexible schedule managed by the employee
- Offices in Seattle and Paris available for collaboration
- Visa sponsorship considered on a case-by-case basis
What you get
- Generous PTO and quarterly Whaleness Days
- End-of-year Whaleness break
- Home office support setup
- Monthly technology stipend of US$100 net
- Annual learning and development stipend
- 16 weeks paid parental leave after six months
- Equity for all full-time employees
- Comprehensive medical and retirement benefits vary by country
Hiring process
- Docker may use AI-assisted tools during recruiting
- Candidates may opt in to interview recordings for calibration