Software Engineer, Inference
Luma · Remote · Remote, EU ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Systems engineering focus
- Python + Kubernetes
- GPU fleet management
- Scale: 1000+ machines
You will own the infrastructure that serves Luma's models, integrating new architectures into the inference engine and scaling deployments globally. The work involves scheduling, fleet management, and maintaining reliability across clusters and hardware providers to meet strict service level objectives.
This role is designed for engineers comfortable with model serving and Kubernetes at scale. You will collaborate with research and infrastructure teams to optimize efficiency, build internal tooling, and automate services for maximum uptime.
What you'll do
- Integrate new model architectures into the inference engine
- Optimize model efficiency and deployments with cross-functional teams
- Build tooling to profile and track inference job lifecycles
- Automate and maintain inference services for high reliability
- Manage workloads across clusters and diverse hardware providers
- Develop scheduling systems to optimize GPU usage against SLOs
- Maintain CI/CD pipelines for model checkpoints and SDKs
What you'll need
- Strong proficiency in Python and system architecture
- Experience deploying models with PyTorch or Hugging Face
- Familiarity with vLLM, SGLang, or TensorRT-LLM
- Background in queues, scheduling, and traffic control at scale
- Expertise in Linux, Docker, and Kubernetes orchestration
- Knowledge of Redis and S3-compatible storage solutions
Nice to have
- Experience with RDMA, RoCE, InfiniBand, or NVLink
- Background in large-scale ML systems with 100+ GPUs
- Proficiency in CUDA programming
- Experience with FFmpeg or multimedia processing
Stack
- Python
- PyTorch
- Hugging Face
- vLLM
- SGLang
- TensorRT-LLM
- Linux
- Docker
- Kubernetes
- Redis
- S3
- CUDA
- FFmpeg