Member of Technical Staff, Inference
Inferact · Remote · Remote ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote (worldwide)
- vLLM core development
- Python + PyTorch
- Visa sponsorship available
- Senior level role
Join the team behind vLLM to accelerate AI progress by making inference cheaper and faster. You will work at the intersection of models and hardware, pushing the boundaries of serving large language and diffusion models. Your contributions will directly impact how the world runs AI inference by optimizing execution across various architectures.
What you'll do
- Optimize how models execute across diverse hardware and architectures
- Push boundaries in LLM and diffusion model serving capabilities
- Implement innovations required by new model architectures like MoE
- Work at the core of the vLLM inference engine
- Contribute performant and maintainable code to complex ML codebases
What you'll need
- Bachelor's degree or equivalent experience in CS or engineering
- Deep understanding of transformer architectures and variants
- Strong Python programming skills with PyTorch internals experience
- Experience with LLM inference systems like vLLM or TensorRT-LLM
- Ability to implement techniques from research papers
- Skill in debugging complex machine learning codebases
Nice to have
- Understanding of KV-cache memory management and prefix caching
- Familiarity with RL frameworks and algorithms for LLMs
- Experience with multimodal inference including audio and video
- Contributions to open-source ML or system infrastructure projects
- Implemented core features in vLLM or other inference engines
- Written technical blogs or side projects on LLM inference
Stack
- vLLM
- Python
- PyTorch
- TensorRT-LLM
- SGLang
- TGI
How you'll work
- Fully remote position available worldwide
- Timezone-flexible with Pacific Time overlap required
- Regular syncs scheduled during critical hours
What you get
- Competitive salary and equity package
- Visa sponsorship available on a case-by-case basis
- Health coverage appropriate to your location