Senior / Lead Machine Learning Engineer, Inference - Germany
Inworld · Remote · Germany ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Senior / Lead level
- Fully remote (Germany)
- C++ / CUDA / Rust
- LLM Inference Optimization
- Visa sponsorship possible
Join Inworld's inference team to build the infrastructure behind modular APIs handling over 10 trillion LLM tokens monthly. You will optimize serving frameworks and accelerate models for high-volume, real-time workloads across healthcare, media, and companion apps. The team values engineers who thrive in ambiguity and prioritize impact over theoretical optimization.
What you'll do
- Optimize modern serving frameworks like vLLM or TRT-LLM for performance
- Implement model acceleration techniques such as quantization and distillation
- Profile code to maximize NVIDIA GPU efficiency and throughput
- Manage distributed systems using Kubernetes and Ray for scaling
- Take models from research to production with reliable containerization
- Handle thousands of concurrent connections with custom load balancing
What you'll need
- Proficiency in C++, CUDA, Rust, or highly optimized Python
- Deep understanding of inference optimization and serving frameworks
- Experience with distributed systems and multi-GPU inference
- PhD in CS, Physics, Math, or equivalent practical experience
- Professional fluency in English for daily collaboration with US teams
- Ability to thrive in ambiguity and define clear paths forward
Nice to have
- Open-source contributions to major inference engines
- Public technical write-ups or non-trivial systems projects
- Experience with speculative decoding and paged attention
Stack
- vLLM
- TRT-LLM
- C++
- CUDA
- Rust
- Python
- Kubernetes
- Ray
- NVIDIA GPUs
How you'll work
- Fully remote position based in Germany
- Collaboration with US-based leadership and engineering teams
- Flat structure with fast iterations and minimal process
What you get
- Potential U.S. visa and relocation support for San Francisco Bay Area
- Support for sharing work and open-source contributions