Database Paper Browser

Back to papers

Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce

Summary: Partial-reduce: heterogeneity-aware all-reduce variant for distributed ML; decomposes synchronous all-reduce into parallel-asynchronous blocks to tolerate stragglers. Converges to a stationary point at sublinear SGD rate; adds dynamic, staleness-aware averaging and group-generation to avoid update isolation; prototype yields 1.21x–2x speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
6055
Venue
SIGMOD
Year
2021
Pagerank
5.5640779e-05
Overall Rank
5,332 | 62.95%
DOI
10.1145/3448016.3452773

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 14 of 14 citing papers.

Rank Citing Paper Year Venue Pagerank
1,162 Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.00013573136
2,678 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.3224016e-05
3,475 BlindFL: Vertical Federated Machine Learning without Peeking into Your Data 2022 SIGMOD 7.0590045e-05
5,169 HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training 2022 SIGMOD 5.642415e-05
6,361 Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism 2023 VLDB 5.0903244e-05
7,143 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity 2024 VLDB 4.8143774e-05
7,535 Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent 2023 VLDB 4.7131061e-05
8,117 SDPipe: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 4.5788485e-05
8,343 FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation 2024 VLDB 4.5366487e-05
9,331 BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach 2023 SIGMOD 4.351469e-05
9,677 Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving 2025 SIGMOD 4.3006524e-05
9,787 The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format 2024 SIGMOD 4.2799988e-05
10,310 NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments 2026 VLDB 4.1905499e-05
10,596 SimRN: Trajectory Similarity Learning in Road Networks based on Distributed Deep Reinforcement Learning 2025 VLDB 4.1905499e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 6 of 6 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
411 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB 0.00023881138
1,044 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.00014465007
1,946 Heterogeneity-aware Distributed Parameter Servers 2017 SIGMOD 9.9983926e-05
2,660 Vertica-ML: Distributed Machine Learning in Vertica Database 2020 SIGMOD 8.3614044e-05
3,810 SketchML: Accelerating Distributed Machine Learning with Data Sketches 2018 SIGMOD 6.7375383e-05
4,972 PS2: Parameter Server on Spark 2019 SIGMOD 5.7894524e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
10,646 Heta: Distributed Training of Heterogeneous Graph Neural Networks 2025 VLDB 4.1905499e-05
1,407 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012163413
9,225 Towards an Optimized GROUP BY Abstraction for Large-Scale Machine Learning 2021 VLDB 4.3656789e-05
9,596 Scalable Graph Convolutional Network Training on Distributed-Memory Systems 2023 VLDB 4.3150788e-05
6,482 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB 5.0396586e-05
9,964 Distributed Learning of Fully Connected Neural Networks using Independent Subnet Training 2022 VLDB 4.2229209e-05
411 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB 0.00023881138
4,394 Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models 2017 VLDB 6.2173726e-05
8,117 SDPipe: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 4.5788485e-05
1,946 Heterogeneity-aware Distributed Parameter Servers 2017 SIGMOD 9.9983926e-05