DBScholar

Back to papers

Heterogeneity-aware Distributed Parameter Servers

Summary: Heterogeneity-aware distributed parameter servers for SGD in heterogeneous clusters, addressing stragglers and sync bottlenecks. Proposes constant learning-rate pre-aggregation and delayed-update schedules with convergence guarantees; Tencent prototype yields 2–12x speedups and up to 6x fewer iterations vs Spark, Petuum, TF. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
5361
Venue
SIGMOD
Year
2017
Pagerank
9.0581831e-05
Overall Rank
2,162 | 85.17%
DOI
10.1145/3035918.3035933

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{jiang_sigmod17,
        title = {{Heterogeneity-aware Distributed Parameter Servers}},
        author = {Jiang, Jiawei and Cui, Bin and Zhang, Ce and Yu, Lele},
        series = {{SIGMOD} '17},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3035918.3035933},
        url = {https://dl.acm.org/doi/10.1145/3035918.3035933},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
1,132 SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.00012041292
1,157 Cerebro: A Data System for Optimized Deep Learning Model Selection 2020 VLDB 0.00011924049
2,485 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.5145736e-05
2,693 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB 8.247004e-05
3,169 Towards Demystifying Serverless Machine Learning Training 2021 SIGMOD 7.6715222e-05
3,490 MLog: Towards Declarative In-Database Machine Learning 2017 VLDB 7.3667971e-05
3,782 CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers 2019 VLDB 7.1298979e-05
4,083 SketchML: Accelerating Distributed Machine Learning with Data Sketches 2018 SIGMOD 6.9160949e-05
4,956 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD 6.4290135e-05
5,345 PS2: Parameter Server on Spark 2019 SIGMOD 6.2586047e-05
5,589 An Experimental Evaluation of Large Scale GBDT Systems 2019 VLDB 6.1559057e-05
5,785 BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees 2019 SIGMOD 6.0892672e-05
5,876 BAGUA: Scaling up Distributed Learning with System Relaxations 2022 VLDB 6.0557672e-05
6,022 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD 6.0046123e-05
6,339 In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle 2022 SIGMOD 5.907165e-05
6,519 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB 5.8536062e-05
7,687 ExDRa: Exploratory Data Science on Federated Raw Data 2021 SIGMOD 5.5671645e-05
8,101 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.4870581e-05
8,334 Just Move It! Dynamic Parameter Allocation in Action 2021 VLDB 5.4518674e-05
9,672 DimBoost: Boosting Gradient Boosting Decision Tree to Higher Dimensions 2018 SIGMOD 5.2386185e-05
10,323 NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters 2026 SIGMOD 5.093636e-05
10,769 Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization 2025 SIGMOD 5.093636e-05
11,999 LDA*: A Robust and Large-scale Topic Modeling System 2017 VLDB 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012250108
20 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.00056944564
106 The MADlib Analytics Library or MAD Skills, the SQL 2012 VLDB 0.00033539462
126 Schism: a Workload-Driven Approach to Database Replication and Partitioning 2010 VLDB 0.00030779127
199 Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design 2004 SIGMOD 0.00025612088
246 Automating Physical Database Design in a Parallel Database 2002 SIGMOD 0.00023457421
452 An Architecture for Parallel Topic Models 2010 VLDB 0.00018146809
715 Learning Generalized Linear Models Over Normalized Data 2015 SIGMOD 0.00014655327
993 Simulation of Database-Valued Markov Chains Using SimSQL 2013 SIGMOD 0.00012789598
1,150 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.00011943462
1,271 Hybrid-Range Partitioning Strategy: A New Declustering Strategy for Multiprocessor Database Machines 1990 VLDB 0.0001138443
4,686 Tornado: A System For Real-Time Iterative Analysis Over Evolving Data 2016 SIGMOD 6.5620646e-05
5,584 Large-scale Predictive Analytics in Vertica: Fast Data Transfer, Distributed Model Creation, and In-database Prediction 2015 SIGMOD 6.1604087e-05
12,047 Real-time Video Recommendation Exploration 2016 SIGMOD 5.093636e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 10,323 NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters 2026 SIGMOD
2 521 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB
3 6,022 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD
4 2,693 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB
5 10,890 Heta: Distributed Training of Heterogeneous Graph Neural Networks 2025 VLDB
6 9,290 Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale 2022 VLDB
7 4,799 Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models 2017 VLDB
8 8,101 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB
9 6,519 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB
10 4,956 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD