GPU Cloud Platform Engineer
Yotta Labs · Remote · United States · Hong Kong · Canada · Singapore ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote (Global)
- Kubernetes + GPU
- AI Infrastructure
- Senior Level
- Full-time
You will join the core infrastructure team to design and deploy next-generation AI compute clouds. Your work ensures high availability and efficiency for containerized AI models, including LLMs and generative systems, across heterogeneous hardware like NVIDIA and AMD GPUs.
The position requires orchestrating multi-cluster environments, optimizing scheduling strategies, and building unified monitoring systems. You will collaborate with IDC providers to plan large-scale GPU, network, and storage deployments that support both internal platforms and external customers.
What you'll do
- Build and operate large-scale GPU clusters ensuring stable compute, network, and storage.
- Conduct performance testing on multi-node clusters to identify and resolve bottlenecks.
- Deploy and orchestrate large models across multi-cluster environments using Kubernetes.
- Define Kubernetes multi-cluster configuration standards and optimize scheduling strategies.
- Build unified management and monitoring systems for cross-region resource tracking.
- Coordinate with IDC providers for planning and deploying large-scale infrastructure.
What you'll need
- Bachelor's degree in Computer Science or related field with 3+ years in DevOps.
- 5+ years in cloud-native or AI engineering with 2+ years in Kubernetes multi-cluster.
- Proficient in Docker, Helm, and Kubernetes ecosystem tools for deployment and scaling.
- Experience with monitoring tools like Prometheus and Grafana for GPU fault alerting.
- Hands-on experience with AWS, GCP, or Azure and cloud-native multi-cluster architecture.
- Familiarity with distributed file systems such as NFS, JuiceFS, CephFS, or Lustre.
- Understanding of high-performance communication protocols like IB, RoCE, and NVLink.
Nice to have
- Experience operating MaaS platforms or large-scale model inference clusters.
- Proficiency in CUDA programming and NCCL communication library.
- Ability to develop inference APIs and automation tools using Golang or Python.
- Experience with model quantization, static compilation, and multi-GPU parallelism.
- Knowledge of inference frameworks like Triton, vLLM, and SGLang.
- Experience with cluster management tools like Ray, Slurm, KubeSphere, or Karmada.
Stack
- Kubernetes
- Docker
- Helm
- Prometheus
- Grafana
- AWS
- GCP
- Azure
- CUDA
- Golang
- Python
- Triton
- vLLM
- SGLang
How you'll work
- Fully remote position open to candidates globally.
- Full-time employment type.