Back to papers
Heterogeneity-aware Distributed Parameter Servers
Summary: Heterogeneity-aware distributed parameter servers for SGD in heterogeneous clusters, addressing stragglers and sync bottlenecks. Proposes constant learning-rate pre-aggregation and delayed-update schedules with convergence guarantees; Tencent prototype yields 2–12x speedups and up to 6x fewer iterations vs Spark, Petuum, TF.
(summarized by gpt-5-nano on Feb 09 2026)
- Paper ID
- 5299
- Venue
- SIGMOD
- Year
- 2017
- Pagerank
- 0.00010012691
- Overall Rank
- 1,942 | 86.50%
- DOI
-
10.1145/3035918.3035933
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 23 of 23 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 683 |
Cerebro: A Data System for Optimized Deep Learning Model Selection |
2020 |
VLDB |
0.00018195476 |
| 1,160 |
Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks |
2022 |
VLDB |
0.00013586221 |
| 2,440 |
FlexPS: Flexible Parallelism Control in Parameter Server Architecture |
2018 |
VLDB |
8.8119143e-05 |
| 2,677 |
HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework |
2022 |
VLDB |
8.3268401e-05 |
| 2,791 |
Towards Demystifying Serverless Machine Learning Training |
2021 |
SIGMOD |
8.1206618e-05 |
| 3,363 |
CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers |
2019 |
VLDB |
7.1731921e-05 |
| 3,808 |
SketchML: Accelerating Distributed Machine Learning with Data Sketches |
2018 |
SIGMOD |
6.7455428e-05 |
| 3,958 |
MLog: Towards Declarative In-Database Machine Learning |
2017 |
VLDB |
6.5897636e-05 |
| 4,964 |
PS2: Parameter Server on Spark |
2019 |
SIGMOD |
5.7965988e-05 |
| 4,975 |
An Experimental Evaluation of Large Scale GBDT Systems |
2019 |
VLDB |
5.79026e-05 |
| 5,084 |
In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle |
2022 |
SIGMOD |
5.7091191e-05 |
| 5,333 |
Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce |
2021 |
SIGMOD |
5.5656575e-05 |
| 5,720 |
BAGUA: Scaling up Distributed Learning with System Relaxations |
2022 |
VLDB |
5.3527734e-05 |
| 5,806 |
BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees |
2019 |
SIGMOD |
5.3200643e-05 |
| 5,988 |
NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access |
2022 |
SIGMOD |
5.2430981e-05 |
| 6,471 |
Dynamic Parameter Allocation in Parameter Servers |
2020 |
VLDB |
5.0511668e-05 |
| 7,704 |
ExDRa: Exploratory Data Science on Federated Raw Data |
2021 |
SIGMOD |
4.6733838e-05 |
| 8,025 |
Just Move It! Dynamic Parameter Allocation in Action |
2021 |
VLDB |
4.6031105e-05 |
| 8,126 |
SDPipe: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training |
2023 |
VLDB |
4.5796615e-05 |
| 9,469 |
DimBoost: Boosting Gradient Boosting Decision Tree to Higher Dimensions |
2018 |
SIGMOD |
4.3342363e-05 |
| 10,027 |
NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters |
2026 |
SIGMOD |
4.1945683e-05 |
| 10,492 |
Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization |
2025 |
SIGMOD |
4.1945683e-05 |
| 11,795 |
LDA*: A Robust and Large-scale Topic Modeling System |
2017 |
VLDB |
4.1945683e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 14 of 14 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank |
Cited Paper |
Year |
Venue |
Pagerank |
| 4 |
Pregel: A System for Large-Scale Graph Processing |
2010 |
SIGMOD |
0.0019005923 |
| 37 |
Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud |
2012 |
VLDB |
0.0007522744 |
| 140 |
The MADlib Analytics Library or MAD Skills, the SQL |
2012 |
VLDB |
0.00042270404 |
| 209 |
Schism: a Workload-Driven Approach to Database Replication and Partitioning |
2010 |
VLDB |
0.00034468292 |
| 285 |
Automating Physical Database Design in a Parallel Database |
2002 |
SIGMOD |
0.0002899128 |
| 286 |
Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design |
2004 |
SIGMOD |
0.00028990057 |
| 328 |
An Architecture for Parallel Topic Models |
2010 |
VLDB |
0.0002728514 |
| 1,044 |
DimmWitted: A Study of Main-Memory Statistical Analytics |
2014 |
VLDB |
0.00014475229 |
| 1,158 |
Simulation of Database-Valued Markov Chains Using SimSQL |
2013 |
SIGMOD |
0.0001361064 |
| 1,167 |
Learning Generalized Linear Models Over Normalized Data |
2015 |
SIGMOD |
0.00013547713 |
| 1,266 |
Hybrid-Range Partitioning Strategy: A New Declustering Strategy for Multiprocessor Database Machines |
1990 |
VLDB |
0.00012946573 |
| 5,211 |
Tornado: A System For Real-Time Iterative Analysis Over Evolving Data |
2016 |
SIGMOD |
5.6284829e-05 |
| 5,395 |
Large-scale Predictive Analytics in Vertica: Fast Data Transfer, Distributed Model Creation, and In-database Prediction |
2015 |
SIGMOD |
5.5318806e-05 |
| 11,846 |
Real-time Video Recommendation Exploration |
2016 |
SIGMOD |
4.1945683e-05 |
Semantically Similar Papers