Research
The science under the swarm
Protocol learning
Treat model training as an open network protocol: anyone can contribute compute, nobody needs a datacenter, and contributors, not a single lab, end up owning the model. Our version keeps the training stack and attaches an on-chain ownership ledger and a fee distribution.
Model parallelism over the internet
Instead of every node holding a full copy (data parallelism), the model is split into pipeline stages and each node holds one. Stage boundaries send activations constrained to a shared low-dimensional subspace, about 100× less traffic, so consumer links keep up.
Asynchronous, fault-tolerant pipelines
Nodes join, leave and run at different speeds. Microbatches are routed by measured throughput, replicas of a stage average among themselves, and a Nesterov-style correction compensates for stale gradients in asynchronous pipelines.
Verification and incentives
Work counts only when a trainer attests it and duplicate-forward spot checks agree. Stake gates entry and is slashable; each epoch's trading fees are split by verified points and committed on-chain as a merkle root.
Project docs
Design notes in this repository
Deep dive on Pluralis, Node0, Agora and adjacent systems, with sources.
What Eschaton keeps from Pluralis, what it changes, and why: points, verification, epochs.
The cross-component contract: accounts, merkle format, APIs, ports.
pump.fun creator-fee mechanics, measured on e/acc, and the routing ESC will use.
Papers
Reading list
- Ramasinghe et al., NeurIPS 2025 ↗
Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism
Subspace-constrained activations make pipeline stage boundaries ~100× cheaper with no convergence loss.
- Ajanthan et al., ICML 2025 ↗
Nesterov Method for Asynchronous Pipeline Parallel Optimization
Corrects gradient staleness in asynchronous pipelines with a delay-aware look-ahead.
- Pluralis, 2026 ↗
Agora: Collective and Permissionless Internet-Scale Pretraining of LLMs
The production system after Node0: admission, stage placement, health monitoring, trainer routing.
- Dolatabadi et al., 2026 ↗
SENTINEL: Stagewise Integrity Verification for Pipeline Parallel Decentralized Training
Statistical monitoring of inter-stage activations to flag corrupted stages; basis of our tie-break.
- Long, Hewa Koneputugodage et al., NeurIPS 2025 ↗
Unextractable Protocol Models
Time-varying invertible transforms so stolen shards can't be stitched into a usable model (roadmap here).
- Ryabinin et al., 2023 ↗
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
Randomized, throughput-aware pipelines over unreliable peers; the hivemind lineage Node0 builds on.
Background
From Pluralis Research
Code