ML Ops Infrastructure Engineer
Deepgram · Remote · USA - Remote ·
COMPENSATION
$160k – $220k
Listed on the employer's job board (Ashby)
- Fully remote
- MLOps & Infrastructure
- Python + Kubernetes
- Voice AI platform
- Senior level
You will own the critical bridge between research and production at Deepgram, building the pipelines and deployment systems that take models from experimental to battle-tested at scale. Your work ensures model improvements are safely and reliably delivered to customers depending on real-time voice AI APIs.
The team operates with an AI-first mindset, expecting rapid adaptation and creative use of advanced AI tools. You will collaborate with research engineers to define quality gates and optimize serving infrastructure for latency, throughput, and cost efficiency.
What you'll do
- Design CI/CD pipelines tailored for ML model development and deployment
- Architect pipelines moving models from research to production environments
- Build A/B testing infrastructure for controlled model rollouts
- Implement monitoring for accuracy, latency, drift, and regression alerts
- Develop automated retraining pipelines triggered by data or performance changes
- Maintain build and test environments mirroring production for researchers
- Establish model versioning, artifact management, and rollback capabilities
- Build observability dashboards for real-time insight into model health
- Optimize model serving infrastructure for latency, throughput, and cost
What you'll need
- 4+ years in MLOps, DevOps, or infrastructure engineering focused on ML
- Strong proficiency in Python for building automation and tooling
- Deep experience with CI/CD systems for software and model delivery
- Hands-on experience with Docker and Kubernetes for containerized workloads
- Practical experience deploying and serving ML models in production
- Familiarity with model evaluation, validation, and QA processes
- Understanding of monitoring and observability principles for ML systems
- Strong problem-solving skills with a bias toward automation
Nice to have
- Experience with NVIDIA Triton, TensorRT, or ONNX Runtime
- Background in speech, audio, or real-time media ML systems
- Experience with Infrastructure as Code tools like Terraform or Pulumi
- Hands-on experience with Prometheus, Grafana, Datadog, or similar
- Familiarity with GPU-accelerated inference optimization
- Experience with feature stores, data versioning, or ML metadata
- Knowledge of canary deployment strategies for ML models
Stack
- Python
- Docker
- Kubernetes
- NVIDIA Triton Inference Server
- TensorRT
- ONNX Runtime
- Terraform
- Pulumi
- Prometheus
- Grafana
- Datadog
How you'll work
- Fully remote position
- AI-first mindset required
- Fast-paced environment with rapid change
- Expectation to actively use and experiment with AI tools