DBScholar

Back to papers

Heterogeneity-aware Distributed Parameter Servers

Summary: Heterogeneity-aware distributed parameter servers for SGD in heterogeneous clusters, addressing stragglers and sync bottlenecks. Proposes constant learning-rate pre-aggregation and delayed-update schedules with convergence guarantees; Tencent prototype yields 2–12x speedups and up to 6x fewer iterations vs Spark, Petuum, TF. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
ha5f19089820bfbc0
Venue
SIGMOD
Year
2017
Pagerank
8.8958335e-05
Overall Rank
2,184 | 85.32%
DOI
10.1145/3035918.3035933

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{jiang_sigmod17,
        title = {{Heterogeneity-aware Distributed Parameter Servers}},
        author = {Jiang, Jiawei and Cui, Bin and Zhang, Ce and Yu, Lele},
        series = {{SIGMOD} '17},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3035918.3035933},
        url = {https://dl.acm.org/doi/10.1145/3035918.3035933},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
1,134 SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.00011893521
1,152 Cerebro: A Data System for Optimized Deep Learning Model Selection 2020 VLDB 0.00011801961
2,519 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.3521095e-05
2,725 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB 8.090861e-05
3,240 Towards Demystifying Serverless Machine Learning Training 2021 SIGMOD 7.5002772e-05
3,524 MLog: Towards Declarative In-Database Machine Learning 2017 VLDB 7.2337006e-05
3,764 CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers 2019 VLDB 7.0394265e-05
3,872 SketchML: Accelerating Distributed Machine Learning with Data Sketches 2018 SIGMOD 6.9540368e-05
4,963 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD 6.3384091e-05
5,433 PS2: Parameter Server on Spark 2019 SIGMOD 6.1308667e-05
5,633 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD 6.0578661e-05
5,723 An Experimental Evaluation of Large Scale GBDT Systems 2019 VLDB 6.0178515e-05
5,877 BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees 2019 SIGMOD 5.9627218e-05
5,997 BAGUA: Scaling up Distributed Learning with System Relaxations 2022 VLDB 5.9198921e-05
6,458 In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle 2022 SIGMOD 5.7773842e-05
6,640 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB 5.7251952e-05
7,839 ExDRa: Exploratory Data Science on Federated Raw Data 2021 SIGMOD 5.4432099e-05
8,251 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.3687311e-05
8,488 Just Move It! Dynamic Parameter Allocation in Action 2021 VLDB 5.3317701e-05
9,795 NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters 2026 SIGMOD 5.1257999e-05
9,846 DimBoost: Boosting Gradient Boosting Decision Tree to Higher Dimensions 2018 SIGMOD 5.121318e-05
11,189 Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization 2025 SIGMOD 4.9793485e-05
12,296 LDA*: A Robust and Large-scale Topic Modeling System 2017 VLDB 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012092602
22 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.00055962491
105 The MADlib Analytics Library or MAD Skills, the SQL 2012 VLDB 0.00033638251
123 Schism: a Workload-Driven Approach to Database Replication and Partitioning 2010 VLDB 0.00030762995
195 Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design 2004 SIGMOD 0.00025628849
243 Automating Physical Database Design in a Parallel Database 2002 SIGMOD 0.00023358891
464 An Architecture for Parallel Topic Models 2010 VLDB 0.00017791851
730 Learning Generalized Linear Models Over Normalized Data 2015 SIGMOD 0.00014406936
1,009 Simulation of Database-Valued Markov Chains Using SimSQL 2013 SIGMOD 0.00012539827
1,167 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.00011729888
1,288 Hybrid-Range Partitioning Strategy: A New Declustering Strategy for Multiprocessor Database Machines 1990 VLDB 0.00011183651
4,777 Tornado: A System For Real-Time Iterative Analysis Over Evolving Data 2016 SIGMOD 6.4184698e-05
5,702 Large-scale Predictive Analytics in Vertica: Fast Data Transfer, Distributed Model Creation, and In-database Prediction 2015 SIGMOD 6.0312071e-05
12,342 Real-time Video Recommendation Exploration 2016 SIGMOD 4.9793485e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 9,795 NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters 2026 SIGMOD
2 522 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB
3 5,633 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD
4 2,725 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB
5 11,289 Heta: Distributed Training of Heterogeneous Graph Neural Networks 2025 VLDB
6 8,277 Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale 2022 VLDB
7 4,901 Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models 2017 VLDB
8 8,251 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB
9 6,640 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB
10 4,963 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD