Member of Technical Staff - Storage Infrastructure
Prime Intellect · Remote · San Francisco · Remote ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Storage Infrastructure
- Python or Go
- Distributed Systems
- AI Workloads
You will design and operate the storage systems that support frontier AI workloads at Prime Intellect. Your work ensures reliable, high-throughput access to datasets, checkpoints, and model artifacts as GPU clusters scale.
The team balances performance, durability, availability, and cost while building the open superintelligence stack. You will collaborate with compute and networking teams to deliver robust infrastructure for research and production models.
What you'll do
- Design storage architectures for training data, checkpoints, and inference artifacts
- Deploy and tune parallel filesystems, object storage, and local NVMe caching
- Benchmark throughput, latency, and metadata performance for AI workloads
- Build automation for provisioning, capacity planning, and lifecycle management
- Design replication, recovery, and backup procedures with durability targets
- Diagnose performance issues across applications, networks, and filesystems
- Implement access controls, monitoring, and tenant separation mechanisms
What you'll need
- 3+ years building or operating production distributed storage systems
- Experience with parallel filesystems like Lustre, BeeGFS, Ceph, or GPFS
- Strong Linux administration and performance troubleshooting skills
- Ability to automate infrastructure operations using Python, Go, or Bash
- Understanding of storage failure modes, data integrity, and recovery
- Knowledge of block, file, and object storage semantics and tradeoffs
- Experience with NVMe/SSD tuning, I/O profiling, and benchmarking
Nice to have
- Supporting large GPU training clusters and checkpoint workloads
- Experience with S3-compatible object storage or data tiering
- Knowledge of RDMA-enabled storage or GPUDirect Storage
- Kubernetes storage integrations or SLURM environment experience
- Contributions to open-source storage systems
Stack
- Lustre
- BeeGFS
- Ceph
- GPFS
- Linux
- Python
- Go
- Bash
- NVMe
- Kubernetes
- SLURM
How you'll work
- Fully remote position