DBScholar

Back to papers

Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce

Summary: Partial-reduce: heterogeneity-aware all-reduce variant for distributed ML; decomposes synchronous all-reduce into parallel-asynchronous blocks to tolerate stragglers. Converges to a stationary point at sublinear SGD rate; adds dynamic, staleness-aware averaging and group-generation to avoid update isolation; prototype yields 1.21x–2x speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
hc273ee7425e9821d
Venue
SIGMOD
Year
2021
Pagerank
6.3384091e-05
Overall Rank
4,963 | 66.64%
DOI
10.1145/3448016.3452773

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{miao_sigmod21,
        title = {{Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce}},
        author = {Miao, Xupeng and Nie, Xiaonan and Shao, Yingxia and Yang, Zhi and Jiang, Jiawei and Ma, Lingxiao and Cui, Bin},
        series = {{SIGMOD} '21},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3448016.3452773},
        url = {https://dl.acm.org/doi/10.1145/3448016.3452773},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 14 of 14 citing papers.

Rank Citing Paper Year Venue Pagerank
1,134 SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.00011893521
2,519 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.3521095e-05
3,311 BlindFL: Vertical Federated Machine Learning without Peeking into Your Data 2022 SIGMOD 7.4392547e-05
4,351 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity 2024 VLDB 6.642803e-05
4,945 HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training 2022 SIGMOD 6.3435891e-05
5,009 Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism 2023 VLDB 6.3158866e-05
6,166 Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving 2025 SIGMOD 5.8631131e-05
7,827 FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation 2024 VLDB 5.4442509e-05
8,197 Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent 2023 VLDB 5.3794747e-05
8,251 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.3687311e-05
9,656 BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach 2023 SIGMOD 5.1453267e-05
10,043 The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format 2024 SIGMOD 5.0921006e-05
11,043 NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments 2026 VLDB 4.9793485e-05
11,255 SimRN: Trajectory Similarity Learning in Road Networks based on Distributed Deep Reinforcement Learning 2025 VLDB 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 6 of 6 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Previous Page 1 / 1 Next

Semantically Similar Papers