Research Engineer - Pre-training
Pluralis Research · Remote · San Francisco · Australia ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote (global)
- Distributed ML systems
- PyTorch + FSDP/DeepSpeed
- Visa sponsorship available
- Senior engineering role
Join Pluralis Research to advance Protocol Learning, a method for training large AI models across decentralized consumer devices. You will engineer the infrastructure that scales this approach from 8B parameters to frontier models, handling heterogeneous hardware and unstable networks without central control.
The team solves unique challenges in communication-efficient training, elasticity, and robustness against malicious participants. This position requires building production-quality systems that enable trustless, sovereign AI development.
What you'll do
- Implement model-parallel training strategies including data, pipeline, and tensor parallelism.
- Optimize communication overhead to maintain model convergence over low-bandwidth links.
- Build elastic systems that handle node churn, checkpointing, and state recovery.
- Develop monitoring tools to track throughput, bottlenecks, and model quality across devices.
What you'll need
- Hands-on experience with distributed training using PyTorch, FSDP, DeepSpeed, or Megatron.
- Strong understanding of data, tensor, and pipeline parallelism concepts.
- Proficiency in production-quality Python with focus on concurrency and failure handling.
- Proven track record of shipping systems, research code, or serious personal projects.
- Professional-level English proficiency for written and spoken communication.
Nice to have
- Experience training or serving LLMs like Nemotron, Qwen, or OLMo.
- Knowledge of P2P networking and NAT traversal techniques.
- Background in post-training, reinforcement learning, or inference systems.
- Previous work at proprietary, open-weight, or open-source AI laboratories.
Stack
- PyTorch
- FSDP
- DeepSpeed
- Megatron
- Python
How you'll work
- Fully remote with team members distributed globally.
- Main teams located in Australia and North America.
- Requires comfort working across multiple time zones.
What you get
- Equity-heavy compensation package with significant ownership.
- High base salary for key technical contributors.
- Optional full visa sponsorship and relocation to Australia or US.
- Flexible work environment within a remote-first culture.