Back to papers
Heterogeneity-aware Distributed Parameter Servers
Summary: Heterogeneity-aware distributed parameter servers for SGD in heterogeneous clusters, addressing stragglers and sync bottlenecks. Proposes constant learning-rate pre-aggregation and delayed-update schedules with convergence guarantees; Tencent prototype yields 2–12x speedups and up to 6x fewer iterations vs Spark, Petuum, TF.
(summarized by gpt-5-nano on Feb 09 2026)
- Paper ID
- 5300
- Venue
- SIGMOD
- Year
- 2017
- Pagerank
- 9.9983926e-05
- Overall Rank
- 1,946 | 86.48%
- DOI
-
10.1145/3035918.3035933
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 23 of 23 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 684 |
Cerebro: A Data System for Optimized Deep Learning Model Selection |
2020 |
VLDB |
0.00018152321 |
| 1,162 |
Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks |
2022 |
VLDB |
0.00013573136 |
| 2,446 |
FlexPS: Flexible Parallelism Control in Parameter Server Architecture |
2018 |
VLDB |
8.7988018e-05 |
| 2,678 |
HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework |
2022 |
VLDB |
8.3224016e-05 |
| 2,795 |
Towards Demystifying Serverless Machine Learning Training |
2021 |
SIGMOD |
8.1135606e-05 |
| 3,372 |
CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers |
2019 |
VLDB |
7.1633912e-05 |
| 3,810 |
SketchML: Accelerating Distributed Machine Learning with Data Sketches |
2018 |
SIGMOD |
6.7375383e-05 |
| 3,961 |
MLog: Towards Declarative In-Database Machine Learning |
2017 |
VLDB |
6.5824022e-05 |
| 4,924 |
An Experimental Evaluation of Large Scale GBDT Systems |
2019 |
VLDB |
5.8211961e-05 |
| 4,972 |
PS2: Parameter Server on Spark |
2019 |
SIGMOD |
5.7894524e-05 |
| 5,332 |
Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce |
2021 |
SIGMOD |
5.5640779e-05 |
| 5,730 |
BAGUA: Scaling up Distributed Learning with System Relaxations |
2022 |
VLDB |
5.3476346e-05 |
| 5,817 |
BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees |
2019 |
SIGMOD |
5.3154329e-05 |
| 5,992 |
NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access |
2022 |
SIGMOD |
5.2380905e-05 |
| 6,482 |
Dynamic Parameter Allocation in Parameter Servers |
2020 |
VLDB |
5.0396586e-05 |
| 6,573 |
In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle |
2022 |
SIGMOD |
5.0009513e-05 |
| 7,702 |
ExDRa: Exploratory Data Science on Federated Raw Data |
2021 |
SIGMOD |
4.6689015e-05 |
| 8,052 |
Just Move It! Dynamic Parameter Allocation in Action |
2021 |
VLDB |
4.5925868e-05 |
| 8,117 |
SDPipe: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training |
2023 |
VLDB |
4.5788485e-05 |
| 9,471 |
DimBoost: Boosting Gradient Boosting Decision Tree to Higher Dimensions |
2018 |
SIGMOD |
4.3300949e-05 |
| 10,027 |
NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters |
2026 |
SIGMOD |
4.1905499e-05 |
| 10,502 |
Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization |
2025 |
SIGMOD |
4.1905499e-05 |
| 11,803 |
LDA*: A Robust and Large-scale Topic Modeling System |
2017 |
VLDB |
4.1905499e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 14 of 14 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank |
Cited Paper |
Year |
Venue |
Pagerank |
| 4 |
Pregel: A System for Large-Scale Graph Processing |
2010 |
SIGMOD |
0.0019040811 |
| 39 |
Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud |
2012 |
VLDB |
0.00075263552 |
| 139 |
The MADlib Analytics Library or MAD Skills, the SQL |
2012 |
VLDB |
0.00042320525 |
| 208 |
Schism: a Workload-Driven Approach to Database Replication and Partitioning |
2010 |
VLDB |
0.00034478612 |
| 283 |
Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design |
2004 |
SIGMOD |
0.00029024583 |
| 285 |
Automating Physical Database Design in a Parallel Database |
2002 |
SIGMOD |
0.00028978423 |
| 330 |
An Architecture for Parallel Topic Models |
2010 |
VLDB |
0.00027271063 |
| 1,044 |
DimmWitted: A Study of Main-Memory Statistical Analytics |
2014 |
VLDB |
0.00014465007 |
| 1,165 |
Simulation of Database-Valued Markov Chains Using SimSQL |
2013 |
SIGMOD |
0.00013567206 |
| 1,172 |
Learning Generalized Linear Models Over Normalized Data |
2015 |
SIGMOD |
0.00013504249 |
| 1,264 |
Hybrid-Range Partitioning Strategy: A New Declustering Strategy for Multiprocessor Database Machines |
1990 |
VLDB |
0.00012939192 |
| 5,215 |
Tornado: A System For Real-Time Iterative Analysis Over Evolving Data |
2016 |
SIGMOD |
5.6231911e-05 |
| 5,400 |
Large-scale Predictive Analytics in Vertica: Fast Data Transfer, Distributed Model Creation, and In-database Prediction |
2015 |
SIGMOD |
5.5262667e-05 |
| 11,854 |
Real-time Video Recommendation Exploration |
2016 |
SIGMOD |
4.1905499e-05 |
Semantically Similar Papers