Staff / Principal Machine Learning Engineer, Inference - UK
Inworld · Remote · UK ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Staff/Principal level
- Fully remote (UK)
- LLM inference focus
- C++ / CUDA / Rust
- No visa sponsorship
Join Inworld’s inference team to build the infrastructure behind modular APIs for high-volume, real-time workloads. You will optimize serving frameworks and accelerate models to support millions of daily users across healthcare, gaming, and education sectors.
The company values engineers who thrive in ambiguity and take full ownership of systems from research to production. You will work in a flat structure with minimal process, focusing on performance and latency as core product features.
What you'll do
- Optimize modern serving frameworks like vLLM or TRT-LLM for high throughput
- Implement model acceleration techniques such as quantization and speculative decoding
- Profile and optimize code for NVIDIA GPUs using C++, CUDA, or Rust
- Manage distributed systems scaling with Kubernetes and Ray for multi-GPU inference
- Take models from research to production, ensuring reliable serving and containerization
- Design benchmarks and prototypes to solve unclear latency or throughput problems
What you'll need
- Deep understanding of inference optimization and serving frameworks
- Hands-on experience with quantization, distillation, and caching strategies
- Proficiency in C++, CUDA, Rust, or highly optimized Python
- Experience with Kubernetes, Ray, and multi-node inference scaling
- PhD in CS, Physics, Math, or equivalent practical backend/ML systems experience
- Legal right to work in the United Kingdom is mandatory
Nice to have
- Open-source contributions to major inference engines
- Public technical write-ups or non-trivial systems programming projects
- Experience with continuous batching and paged attention mechanisms
Stack
- vLLM
- TRT-LLM
- C++
- CUDA
- Rust
- Python
- Kubernetes
- Ray
- NVIDIA GPUs
How you'll work
- Fully remote within the United Kingdom
- Full-time employment
- Flat organizational structure with fast iterations
- Minimal process overhead
What you get
- Base salary between £140,000 and £200,000
- Equity compensation included
- Standard benefits package
- Potential US visa and relocation support for San Francisco later