Member of Technical Staff - Inference Runtime
Modal · Remote · San Francisco ·
COMPENSATION
$220k – $300k
Listed on the employer's job board (Ashby)
- Fully remote
- Rust + Go
- Linux systems focus
- GPU & RDMA optimization
- Senior level
Join Modal’s Inference Runtime team to own the container stack powering inference and training across their fleet. You will work at the intersection of Linux, containers, filesystems, and GPU drivers to ensure low-latency, secure, and efficient execution of demanding ML workloads.
This position focuses on deep systems engineering, requiring you to diagnose complex performance issues and contribute to open-source runtime technologies like gVisor while shaping the architecture of Modal’s platform.
What you'll do
- Accelerate container startup, checkpoint, and restore for large ML workloads
- Build multi-GPU snapshotting with efficient memory and RDMA handling
- Design zero-copy data paths between container memory, filesystems, and storage
- Optimize snapshot pipelines via parallel uploads and direct I/O techniques
- Improve filesystem performance across EROFS, FUSE, and overlay systems
- Extend sandboxed runtime support for new NVIDIA and AMD GPU capabilities
- Debug complex failures involving system calls, kernel behavior, and isolation
- Safely roll out runtime changes across heterogeneous fleets using feature flags
What you'll need
- Strong Linux systems knowledge including memory, filesystems, and scheduling
- Experience building or debugging container runtimes, sandboxes, or kernels
- Proficiency in Rust, Go, or similar systems languages
- Skill in profiling systems where I/O, synchronization, or kernel interactions dominate
- Ability to debug across abstraction boundaries from apps down to drivers
- Capacity to transform ambiguous production problems into reliable system designs
- Desire to own critical infrastructure and collaborate with internal teams
Nice to have
- Familiarity with gVisor, runsc, runc, or OCI runtimes
- Experience with Linux kernel development or virtualization modules
- Knowledge of CUDA, ROCm, or GPU driver virtualization
- Background in RDMA and high-performance networking
- Understanding of FUSE, EROFS, direct I/O, or page-cache behavior
- Experience with large-memory or multi-GPU inference systems
- Contributions to open-source systems software projects
Stack
- Rust
- Go
- Linux
- gVisor
- CUDA
- ROCm
- FUSE
- EROFS
How you'll work
- Fully remote position
- Global team distribution