DBScholar

Back to papers

Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce

Summary: Partial-reduce: heterogeneity-aware all-reduce variant for distributed ML; decomposes synchronous all-reduce into parallel-asynchronous blocks to tolerate stragglers. Converges to a stationary point at sublinear SGD rate; adds dynamic, staleness-aware averaging and group-generation to avoid update isolation; prototype yields 1.21x–2x speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
hc273ee7425e9821d
Venue
SIGMOD
Year
2021
Pagerank
6.3354086e-05
Overall Rank
4,965 | 66.64%
DOI
10.1145/3448016.3452773

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{miao_sigmod21,
        title = {{Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce}},
        author = {Miao, Xupeng and Nie, Xiaonan and Shao, Yingxia and Yang, Zhi and Jiang, Jiawei and Ma, Lingxiao and Cui, Bin},
        series = {{SIGMOD} '21},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3448016.3452773},
        url = {https://dl.acm.org/doi/10.1145/3448016.3452773},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 14 of 14 citing papers.

Rank Citing Paper Year Venue Pagerank
1,134 SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.0001188789
2,521 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.3481558e-05
3,312 BlindFL: Vertical Federated Machine Learning without Peeking into Your Data 2022 SIGMOD 7.4357331e-05
4,352 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity 2024 VLDB 6.6396584e-05
4,949 HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training 2022 SIGMOD 6.3405861e-05
5,011 Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism 2023 VLDB 6.3131083e-05
6,168 Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving 2025 SIGMOD 5.8603375e-05
7,831 FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation 2024 VLDB 5.4416736e-05
8,205 Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent 2023 VLDB 5.3769281e-05
8,257 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.3661896e-05
9,663 BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach 2023 SIGMOD 5.142891e-05
10,048 The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format 2024 SIGMOD 5.0896901e-05
11,052 NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments 2026 VLDB 4.9769913e-05
11,263 SimRN: Trajectory Similarity Learning in Road Networks based on Distributed Deep Reinforcement Learning 2025 VLDB 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 6 of 6 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Previous Page 1 / 1 Next

Semantically Similar Papers