DBScholar

Back to papers

BAGUA: Scaling up Distributed Learning with System Relaxations

Summary: MPI-style, modular BAGUA provides system-relaxation primitives (quantization, decentralization, delayed communication) for distributed data-parallel training. Enables rapid prototyping of advanced distributed-learning algorithms; delivers up to 2x end-to-end speedups over PyTorch-DDP/Horovod/BytePS on 128 GPUs and analyzes performance tradeoffs across network conditions. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
13139
Venue
VLDB
Year
2022
Pagerank
6.0557672e-05
Overall Rank
5,876 | 59.69%
DOI
10.14778/3503585.3503590

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{gan_vldb22,
        title = {{BAGUA: Scaling up Distributed Learning with System Relaxations}},
        author = {Gan, Shaoduo and Jiang, Jiawei and Yuan, Binhang and Zhang, Ce and Lian, Xiangru and Wang, Rui and Chang, Jianbin and Liu, Chengjun and Shi, Hongmei and Zhang, Shengzhuo and Li, Xianghong and Sun, Tengxu and Yang, Sen and Liu, Ji},
        journal = {PVLDB},
        series = {{VLDB} '22},
        volume = {15},
        number = {4},
        pages = {804--813},
        doi = {10.14778/3503585.3503590},
        url = {https://doi.org/10.14778/3503585.3503590},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 4 of 4 citing papers.

Rank Citing Paper Year Venue Pagerank
7,160 DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines 2022 CIDR 5.6855887e-05
10,684 Sequoia: An Accessible and Extensible Framework for Privacy-Preserving Machine Learning over Distributed Data 2025 SIGMOD 5.093636e-05
10,769 Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization 2025 SIGMOD 5.093636e-05
10,880 LobRA: Multi-tenant Fine-tuning over Heterogeneous Data 2025 VLDB 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
415 SystemML: Declarative Machine Learning on Spark 2016 VLDB 0.0001888524
521 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB 0.0001713368
1,079 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012258469
2,162 Heterogeneity-aware Distributed Parameter Servers 2017 SIGMOD 9.0581831e-05
2,347 Vertica-ML: Distributed Machine Learning in Vertica Database 2020 SIGMOD 8.7157552e-05
2,370 Declarative Recursive Computation on an RDBMS or, Why You Should Use a Database For Distributed Machine Learning 2019 VLDB 8.6795925e-05
2,693 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB 8.247004e-05
2,786 DB4ML – An In-Memory Database Kernel with Machine Learning Support 2020 SIGMOD 8.1207221e-05
5,195 Tensor Relational Algebra for Distributed Machine Learning System Design 2021 VLDB 6.3232105e-05
5,345 PS2: Parameter Server on Spark 2019 SIGMOD 6.2586047e-05
5,362 Mariana: Tencent Deep Learning Platform and its Applications 2014 VLDB 6.2481421e-05
6,743 In-database Distributed Machine Learning: Demonstration using Teradata SQL Engine 2019 VLDB 5.7859722e-05
9,672 DimBoost: Boosting Gradient Boosting Decision Tree to Higher Dimensions 2018 SIGMOD 5.2386185e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 3,782 CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers 2019 VLDB
2 6,519 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB
3 8,101 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB
4 532 MLbase: A Distributed Machine-learning System 2013 CIDR
5 5,195 Tensor Relational Algebra for Distributed Machine Learning System Design 2021 VLDB
6 9,371 Towards an Optimized GROUP BY Abstraction for Large-Scale Machine Learning 2021 VLDB
7 1,863 ByteGNN: Efficient Graph Neural Network Training at Large Scale 2022 VLDB
8 2,162 Heterogeneity-aware Distributed Parameter Servers 2017 SIGMOD
9 521 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB
10 4,956 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD