Member of Technical Staff, ML Platform
Runway · Remote · Remote ·
COMPENSATION
$240k – $290k
Listed on the employer's job board (Ashby)
- Fully remote
- ML Infrastructure
- Python + PyTorch
- Kubernetes
- Senior level
Join the ML Platform team at Runway to design and build the systems that measure model quality. You will turn scattered evaluation work into a unified platform that generates samples, scores them with automated metrics and human annotations, and delivers results to researchers quickly. Your work will directly influence decisions on which models ship to millions of users.
What you'll do
- Own the evaluation platform end-to-end, including tooling for generation and annotation
- Design CLIs, APIs, GUIs, and storage layers for seamless evaluation workflows
- Collaborate with research teams to define metrics and generalize requirements
- Set standards for reproducibility, metric definitions, and reporting formats
- Support platform adoption across training, serving, and production efforts
- Contribute to broader ML Platform tools for training and serving frontier models
What you'll need
- 5+ years building ML infrastructure or data platforms in production
- Experience with evaluation, experimentation, or benchmarking systems
- Strong proficiency in Python and PyTorch
- Hands-on experience running large batch GPU workloads on Kubernetes
- Expertise in designing data pipelines and storage for large media volumes
- Familiarity with experimental statistics and inter-rater agreement
- Ability to build internal tools from command line to browser interfaces
- Capacity to lead technical direction and drive roadmaps independently
Nice to have
- Track record building evaluation suites for generative models
- Experience with LLM or VLM-as-judge pipelines
- Work with online experimentation platforms and product outcomes
- Prior experience evaluating agents or robotics policies
Stack
- Python
- PyTorch
- Kubernetes
- Cloud services
How you'll work
- Fully remote position
- Salary range applies to candidates in the U.S.