NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism

Summary: NeutronTP replaces graph/data parallelism with feature-sliced tensor parallelism, eliminating cross-worker vertex dependencies and balancing GNN training loads. Decoupled aggregation/NN computation and memory-aware, overlap-oriented scheduling enable large-graph training, yielding 1.29–8.72× speedups. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 14020
Venue: VLDB
Year: 2025
Pagerank: 5.2528121e-05
Overall Rank: 9,546 | 34.51%
DOI: 10.14778/3705829.3705837

Incoming Non-self Citations Over Time

Authors

1. Xin Ai (Northeastern University)
2. Hao Yuan (Northeastern University)
3. Zeyu Ling (Northeastern University)
4. Qiange Wang (National University of Singapore)
5. Yanfeng Zhang (Northeastern University)
6. Zhenbo Fu (Northeastern University)
7. Chaoyi Chen (Northeastern University)
8. Yu Gu (Northeastern University)
9. Ge Yu (Northeastern University)

BibTeX Citation

@article{ai_vldb25,
        title = {{NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism}},
        author = {Ai, Xin and Yuan, Hao and Ling, Zeyu and Wang, Qiange and Zhang, Yanfeng and Fu, Zhenbo and Chen, Chaoyi and Gu, Yu and Yu, Ge},
        journal = {PVLDB},
        series = {{VLDB} '25},
        volume = {18},
        number = {2},
        pages = {173--186},
        doi = {10.14778/3705829.3705837},
        url = {https://doi.org/10.14778/3705829.3705837},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,357	DepCache: A KV Cache Management Framework for GraphRAG with Dependency Attention	2026	SIGMOD	5.093636e-05
10,521	Efficient GNN Training on Giant Graphs with Collective Batching and Scheduling	2026	VLDB	5.093636e-05
10,835	NeutronTask: Scalable and Efficient Multi-GPU GNN Training with Task Parallelism	2025	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
223	AliGraph: A Comprehensive Graph Neural Network Platform	2019	VLDB	0.00024182473
1,048	AGL: A Scalable System for Industrial-purpose Graph Machine Learning	2020	VLDB	0.00012433693
1,132	SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks	2022	VLDB	0.00012041292
1,863	ByteGNN: Efficient Graph Neural Network Training at Large Scale	2022	VLDB	9.5950349e-05
2,640	Scalable and Efficient Full-Graph GNN Training for Large Graphs	2023	SIGMOD	8.3074486e-05
2,695	NeutronStar: Distributed GNN Training with Hybrid Dependency Management	2022	SIGMOD	8.2468134e-05
4,980	Decoupled Graph Neural Networks for Large Dynamic Graphs	2023	VLDB	6.4133375e-05
5,316	Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective	2024	VLDB	6.2687017e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	5,316	Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective	2024	VLDB
2	9,729	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB
3	1,863	ByteGNN: Efficient Graph Neural Network Training at Large Scale	2022	VLDB
4	4,878	NeutronStream: A Dynamic GNN Training Framework with Sliding Window for Graph Streams	2024	VLDB
5	5,079	NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments	2024	VLDB
6	2,640	Scalable and Efficient Full-Graph GNN Training for Large Graphs	2023	SIGMOD
7	10,323	NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters	2026	SIGMOD
8	2,695	NeutronStar: Distributed GNN Training with Hybrid Dependency Management	2022	SIGMOD
9	10,596	NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments	2026	VLDB
10	10,835	NeutronTask: Scalable and Efficient Multi-GPU GNN Training with Task Parallelism	2025	VLDB