Research Scientist - Pre-training
Pluralis Research · Remote · San Francisco · Australia ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote (global)
- Distributed ML research
- PyTorch + FSDP/DeepSpeed
- Visa sponsorship available
- Senior research role
Join Pluralis Research to advance Protocol Learning, a method for training large AI models in a fully decentralized manner on small consumer-grade devices. You will help scale the system from an 8B model run to frontier-scale models, addressing challenges like communication efficiency and node churn.
The team operates globally with hubs in Australia and North America, working on open problems in trustless and sovereign AI infrastructure.
What you'll do
- Implement model-parallel training strategies including data, pipeline, and tensor parallelism
- Optimize communication overhead to maintain model convergence in high-latency networks
- Develop robust checkpointing and state synchronization for node churn resilience
- Build monitoring tools to track throughput and model quality across hundreds of devices
What you'll need
- Hands-on experience with distributed training using PyTorch, FSDP, DeepSpeed, or Megatron
- Strong understanding of data, tensor, and pipeline parallelism concepts
- Production-quality Python skills with focus on concurrency and failure handling
- Proven track record of shipping systems, research code, or serious personal projects
- Professional-level written and spoken English proficiency
Nice to have
- Experience training or serving LLMs such as Nemotron, Qwen, or OLMo
- Background in P2P networking and NAT traversal techniques
- Knowledge of post-training methods and Reinforcement Learning
- Experience with inference and serving systems
- Previous work at proprietary or open-source AI labs
Stack
- PyTorch
- FSDP
- DeepSpeed
- Megatron
- Python
How you'll work
- Fully remote with team members distributed globally
- Main teams located in Australia and North America
- Must be comfortable working across multiple time zones
What you get
- Equity-heavy compensation package with significant ownership
- High base salary component
- Optional full visa sponsorship and relocation to Australia or US
- Flexible work environment