Senior / Lead Machine Learning Engineer, Inference - Serbia
Inworld · Remote · Serbia ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Senior / Lead level
- Fully remote (Serbia)
- C++ / CUDA / Rust
- LLM inference optimization
- Visa sponsorship possible
Join Inworld's inference team to build the infrastructure behind modular APIs handling over 10 trillion LLM tokens monthly. You will work on real-time serving, model research, and large-scale infrastructure for apps in healthcare, gaming, and education. The team values engineers who thrive in ambiguity and prioritize impact over theoretical optimization.
What you'll do
- Optimize modern serving frameworks like vLLM or TRT-LLM for high-volume workloads
- Implement model acceleration techniques such as quantization and speculative decoding
- Profile and optimize code for NVIDIA GPUs using C++, CUDA, Rust, or Python
- Manage distributed systems with Kubernetes, Ray, and multi-GPU inference
- Take models from research to production, ensuring reliable serving and containerization
- Handle thousands of concurrent connections with custom load balancing strategies
What you'll need
- Deep understanding of inference optimization and serving frameworks
- Hands-on experience with quantization, distillation, and caching strategies
- Proficiency in C++, CUDA, Rust, or highly optimized Python
- Experience with Kubernetes, Ray, and multi-node inference scaling
- Professional fluency in English for daily collaboration with US teams
- PhD in CS, Physics, Math, or equivalent practical backend/ML experience
- Ability to thrive in ambiguity and define solutions without clear roadmaps
Nice to have
- Public open-source contributions to major inference engines
- Non-trivial systems programming projects or technical write-ups
- Experience with continuous batching and paged attention mechanisms
Stack
- vLLM
- TRT-LLM
- C++
- CUDA
- Rust
- Python
- Kubernetes
- Ray
- NVIDIA GPUs
How you'll work
- Fully remote position based in Serbia
- Collaboration with US-based leadership and engineering teams
- Flat structure with fast iterations and minimal process
- Potential US visa and relocation support for San Francisco Bay Area