Distributed Learning of Fully Connected Neural Networks using Independent Subnet Training

Summary: Independent Subnet Training decomposes fully connected networks into narrow, same-depth subnets trained independently on partitioned data, periodically exchanging parameters. It enables model-parallel learning under privacy, memory, or slow-interconnect constraints while reducing communication and training time. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 12851
Venue: VLDB
Year: 2022
Pagerank: 5.1319012e-05
Overall Rank: 10,113 | 30.62%
DOI: 10.14778/3529337.3529343

Incoming Non-self Citations Over Time

Authors

1. Binhang Yuan (Rice University)
2. Cameron R. Wolfe (Rice University)
3. Chen Dun (Rice University)
4. Yuxin Tang (Rice University)
5. Anastasios Kyrillidis (Rice University)
6. Chris Jermaine (Rice University)

BibTeX Citation

@article{yuan_vldb22,
        title = {{Distributed Learning of Fully Connected Neural Networks using Independent Subnet Training}},
        author = {Yuan, Binhang and Wolfe, Cameron R. and Dun, Chen and Tang, Yuxin and Kyrillidis, Anastasios and Jermaine, Chris},
        journal = {PVLDB},
        series = {{VLDB} '22},
        volume = {15},
        number = {8},
        pages = {1581--1590},
        doi = {10.14778/3529337.3529343},
        url = {https://doi.org/10.14778/3529337.3529343},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,881	The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format	2024	SIGMOD	5.2040783e-05

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,150	DimmWitted: A Study of Main-Memory Statistical Analytics	2014	VLDB	0.00011943462

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	10,323	NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters	2026	SIGMOD
2	1,863	ByteGNN: Efficient Graph Neural Network Training at Large Scale	2022	VLDB
3	9,219	MemFlow: Memory-Aware Distributed Deep Learning	2020	SIGMOD
4	10,907	Effective and Efficient Distributed Temporal Graph Learning through Hotspot Memory Sharing	2025	VLDB
5	9,546	NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism	2025	VLDB
6	10,596	NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments	2026	VLDB
7	2,640	Scalable and Efficient Full-Graph GNN Training for Large Graphs	2023	SIGMOD
8	521	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB
9	4,956	Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce	2021	SIGMOD
10	9,729	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB