Audio Inference Engineer, Model Efficiency
Cohere · Remote · New York · San Francisco · Toronto · Montreal ·
COMPENSATION
$205k – $380k
Listed on the employer's job board (Ashby)
- Fully remote friendly
- C++ and Python
- Audio ML inference
- EST/PST time zones
- Senior level role
Join the Model Efficiency team at Cohere to build reliable machine learning systems and enhance audio inference serving. You will collaborate with training and infrastructure teams to identify bottlenecks and deliver solutions for audio processing and streaming.
The team is distributed globally but concentrated in EST and PST time zones. This position supports a remote-first culture with optional access to offices in major tech hubs.
What you'll do
- Advance core audio model serving metrics like latency and throughput
- Identify system bottlenecks in audio processing and streaming workloads
- Deliver creative solutions for real-time and streaming audio inference
- Collaborate with training and serving infrastructure teams
- Ensure seamless integration between model development and deployment
What you'll need
- Significant experience in high-performance audio or ML inference systems
- Proficiency with C++ and Python programming languages
- Hands-on experience with deep learning models for audio or speech
- Bias for action and strong results-oriented mindset
Nice to have
- Experience with GPU programming and low-level system optimization
- Knowledge of model parallelization techniques over multiple GPUs
- Background in duplex real-time streaming architectures
- Familiarity with PyTorch, TensorFlow, or specialized audio libraries
- Experience with inference frameworks like vLLM or TensorRT-LLM
- Expertise in sequence modeling and end-to-end audio pipeline optimization
Stack
- C++
- Python
- PyTorch
- TensorFlow
- vLLM
- SGLang
- TensorRT-LLM
How you'll work
- Remote-friendly with no minimum in-office requirement
- Preferred time zones are EST and PST
- Offices available in Toronto, London, NYC, SF, Montreal, Paris, Berlin, Seoul
- $500 home office stipend provided
- Co-working benefit for those not near an office
What you get
- Weekly lunch stipend of $75 or local equivalent
- Full health and dental benefits including mental health budget
- Retirement savings matching (RRSP, 401K, Pension)
- 100% parental leave top-up for up to 6 months
- 6 weeks of paid vacation annually
- Annual enrichment benefits for arts, fitness, and workspace
- Education stipend for conferences and courses
- Budget for traveling to other offices and annual offsite