Forward Deployed Engineer - SRE
Andromeda Cluster · Remote · North America Remote / San Francisco, CA ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote (NA) or SF hybrid
- GPU cluster operations
- Kubernetes + Slurm
- Python, Go, Bash
- First FDE hire
You will join Andromeda as the first Forward Deployed Engineer, embedding directly with teams that run massive training and inference workloads. Your focus is on onboarding customers, diagnosing complex distributed system failures, and turning recurring issues into platform automation.
This position requires deep expertise in GPU infrastructure, high-speed networking, and production Kubernetes or Slurm environments. You act as the primary technical contact for key accounts, ensuring their clusters operate reliably while feeding insights back to the product roadmap.
What you'll do
- Onboard customers end-to-end, handling environment setup and orchestration choices
- Diagnose real-world failures like NCCL timeouts, stragglers, and I/O stalls
- Profile distributed training to improve MFU and reduce idle GPU time
- Ensure health of high-speed interconnects such as InfiniBand and NVLink
- Build automation for cluster provisioning, health checks, and driver management
- Lead incident response for multi-layer hardware and software failures
- Translate field observations into product improvements and roadmap changes
What you'll need
- Production experience operating NVIDIA A100/H100/H200/B200 GPU clusters
- Deep knowledge of InfiniBand, RoCE, or NVLink fabrics in distributed training
- Expert-level Linux skills including kernel tuning and driver management
- Strong experience running Kubernetes with GPU workloads or Slurm schedulers
- Proficiency in Python, Go, or Bash for building production-grade tools
- Experience with Infrastructure-as-Code tools like Terraform or Helm
- Ability to build monitoring for GPU telemetry using DCGM or nvidia-smi
- Proven track record leading incident response for complex distributed systems
Nice to have
- Experience with parallel file systems like VAST, WEKA, Lustre, or GPFS
- Background in solutions architecture or deployed SRE roles
- Hands-on experience operating production inference with autoscaling
- Contributions to open source projects or published technical deep-dives
- Experience working across heterogeneous cloud providers and regions
Stack
- NVIDIA GPUs
- InfiniBand
- RoCE
- NVLink
- Kubernetes
- Slurm
- Python
- Go
- Bash
- Terraform
- Helm
- DCGM
How you'll work
- Fully remote within North America
- Hybrid option available in San Francisco
- Full-time employment
What you get
- Competitive compensation with meaningful equity
- Healthcare, dental, and vision coverage
- 401(k) plan
- Unlimited PTO