Senior ML Systems Engineer, Frameworks & Tooling
Cohere · Remote · London · San Francisco · New York · Paris · Toronto · Montreal ·
COMPENSATION
$205k – $380k
Listed on the employer's job board (Ashby)
- Senior ML Systems Engineer
- Fully remote or hybrid
- Distributed training focus
- JAX and CUDA expertise
- Global team collaboration
Join Cohere as a Senior ML Systems Engineer to design the core components enabling fast, reliable, and scalable model training. You will work at the intersection of large-scale training, distributed systems, and high-performance computing infrastructure.
Your work will involve building high-performance data pipelines, implementing performance profiling, and developing fault-tolerant checkpointing systems. You will have the autonomy to impact projects that bridge research ideas with thousands of GPUs.
What you'll do
- Build and own the training framework for large-scale LLM training
- Design distributed training abstractions including parallelism strategies
- Improve training throughput and stability on multi-node clusters
- Develop tooling for monitoring, logging, debugging, and developer ergonomics
- Collaborate with infra teams to support high-performance training environments
- Investigate and resolve performance bottlenecks across the ML systems stack
- Build robust systems for reproducible and debuggable large-scale runs
What you'll need
- Strong engineering experience in large-scale distributed training or HPC systems
- Deep familiarity with JAX internals, distributed training libraries, or custom kernels
- Experience with multi-node cluster orchestration like Slurm, Ray, or Kubernetes
- Comfort debugging performance issues across CUDA, NCCL, networking, and IO
- Experience working with containerized environments such as Docker or Singularity
- Track record of building tools that increase developer velocity for ML teams
- Excellent judgment around trade-offs between performance, complexity, and maintainability
Nice to have
- Experience with training LLMs or other large transformer architectures
- Contributions to ML frameworks like PyTorch, JAX, DeepSpeed, or Megatron
- Familiarity with evaluation and serving frameworks such as vLLM or TensorRT-LLM
- Experience with data pipeline optimization, sharded datasets, or caching strategies
- Background in performance engineering, profiling, or low-level systems
- Published papers at top-tier venues like NeurIPS, ICML, or ICLR
Stack
- JAX
- PyTorch
- CUDA
- NCCL
- Slurm
- Ray
- Kubernetes
- Docker
- Singularity
- DeepSpeed
- Megatron
- vLLM
- TensorRT-LLM
How you'll work
- Fully remote or based in key global offices
- No minimum in-office qualification requirement
- Core time zone alignment expected depending on location
- $500 home office stipend for remote workers
- Co-working benefit available for those not near an office
What you get
- Weekly lunch stipend of $75 or equivalent in local currency
- Full health and dental benefits including mental health budget
- Retirement savings matching via RRSP, 401K, or Pension Scheme
- 100% parental leave top-up for up to six months
- Annual enrichment benefits for arts, fitness, and workspace improvement
- Education and learning stipend for conferences and courses
- Six weeks of paid vacation totaling 30 working days
- Budget for traveling to other offices and annual company offsite