NeutronTask: Scalable and Efficient Multi-GPU GNN Training with Task Parallelism

Summary: Introduce GNN task parallelism that partitions per-layer training tasks across GPUs (instead of graph partitioning), reducing neighbor replication, enabling intra-GPU shared neighbor-embedding reuse and overlapping subgraph computation. Combine with a task-decoupled training framework that releases intermediate data early to cut memory, enabling billion-scale full-graph multi-GPU training and 1.27×–5.47× speedups over NeutronStar/Sancus on 4×A5000. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: h71a59d88b6811c91
Venue: VLDB
Year: 2025
Pagerank: 5.5025409e-05
Overall Rank: 7,522 | 49.43%
DOI: 10.14778/3725688.3725700

Incoming Non-self Citations Over Time

Authors

1. Zhenbo Fu (Northeastern University)
2. Xin Ai (Northeastern University)
3. Qiange Wang (National University of Singapore)
4. Yanfeng Zhang (Northeastern University)
5. Shizhan Lu (Northeastern University)
6. Chaoyi Chen (Northeastern University)
7. Chunyu Cao (Northeastern University)
8. Hao Yuan (Northeastern University)
9. Zhewei Wei (Renmin University of China)
10. Yu Gu (Northeastern University)
11. Yingyou Wen (Neusoft AI Research)
12. Ge Yu (Northeastern University)

BibTeX Citation

@article{fu_vldb25,
        title = {{NeutronTask: Scalable and Efficient Multi-GPU GNN Training with Task Parallelism}},
        author = {Fu, Zhenbo and Ai, Xin and Wang, Qiange and Zhang, Yanfeng and Lu, Shizhan and Chen, Chaoyi and Cao, Chunyu and Yuan, Hao and Wei, Zhewei and Gu, Yu and Wen, Yingyou and Yu, Ge},
        journal = {PVLDB},
        series = {{VLDB} '25},
        volume = {18},
        number = {6},
        pages = {1705--1719},
        doi = {10.14778/3725688.3725700},
        url = {https://doi.org/10.14778/3725688.3725700},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,785	MGI: A Communication Framework for Data Processing in Massive GPU Infrastructures	2026	VLDB	4.9793485e-05
10,861	UniTG: A Unified System for Efficient and Seamless Textual Graph Learning	2026	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 16 of 16 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
211	AliGraph: A Comprehensive Graph Neural Network Platform	2019	VLDB	0.00024816965
1,134	SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks	2022	VLDB	0.00011893521
1,772	ByteGNN: Efficient Graph Neural Network Training at Large Scale	2022	VLDB	9.6792287e-05
2,279	NeutronStar: Distributed GNN Training with Hybrid Dependency Management	2022	SIGMOD	8.7062637e-05
2,506	DUCATI: A Dual-Cache Training System for Graph Neural Networks on Giant Graphs with the GPU	2023	SIGMOD	8.3774747e-05
2,596	Scalable and Efficient Full-Graph GNN Training for Large Graphs	2023	SIGMOD	8.2419456e-05
4,454	LargeEA: Aligning Entities for Large-scale Knowledge Graphs	2022	VLDB	6.5927535e-05
4,825	DAHA: Accelerating GNN Training with Data and Hardware Aware Execution Planning	2024	VLDB	6.3923554e-05
4,853	NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments	2024	VLDB	6.3805655e-05
5,051	Decoupled Graph Neural Networks for Large Dynamic Graphs	2023	VLDB	6.2961385e-05
5,899	Lotan: Bridging the Gap between GNNs and Scalable Graph Analytics Engines	2023	VLDB	5.9545431e-05
6,669	Efficient Training of Graph Neural Networks on Large Graphs	2024	VLDB	5.7147311e-05
6,793	XGNN: Boosting Multi-GPU GNN Training via Global GNN Memory Store	2024	VLDB	5.6810336e-05
6,852	OUTRE: An OUT-of-core De-REdundancy GNN Training Framework for Massive Graphs within A Single Machine	2024	VLDB	5.6647613e-05
6,924	HongTu: Scalable Full-Graph GNN Training on Multiple GPUs	2023	SIGMOD	5.6426299e-05
8,152	NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism	2025	VLDB	5.3896512e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,924	HongTu: Scalable Full-Graph GNN Training on Multiple GPUs	2023	SIGMOD
2	4,898	Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective	2024	VLDB
3	1,772	ByteGNN: Efficient Graph Neural Network Training at Large Scale	2022	VLDB
4	4,502	NeutronStream: A Dynamic GNN Training Framework with Sliding Window for Graph Streams	2024	VLDB
5	2,596	Scalable and Efficient Full-Graph GNN Training for Large Graphs	2023	SIGMOD
6	9,795	NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters	2026	SIGMOD
7	2,279	NeutronStar: Distributed GNN Training with Hybrid Dependency Management	2022	SIGMOD
8	11,043	NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments	2026	VLDB
9	4,853	NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments	2024	VLDB
10	8,152	NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism	2025	VLDB