DBScholar

Back to papers

Heterogeneity-aware Distributed Parameter Servers

Summary: Heterogeneity-aware distributed parameter servers for SGD in heterogeneous clusters, addressing stragglers and sync bottlenecks. Proposes constant learning-rate pre-aggregation and delayed-update schedules with convergence guarantees; Tencent prototype yields 2–12x speedups and up to 6x fewer iterations vs Spark, Petuum, TF. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
ha5f19089820bfbc0
Venue
SIGMOD
Year
2017
Pagerank
8.8916253e-05
Overall Rank
2,186 | 85.31%
DOI
10.1145/3035918.3035933

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{jiang_sigmod17,
        title = {{Heterogeneity-aware Distributed Parameter Servers}},
        author = {Jiang, Jiawei and Cui, Bin and Zhang, Ce and Yu, Lele},
        series = {{SIGMOD} '17},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3035918.3035933},
        url = {https://dl.acm.org/doi/10.1145/3035918.3035933},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
1,134 SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.0001188789
1,152 Cerebro: A Data System for Optimized Deep Learning Model Selection 2020 VLDB 0.00011796404
2,521 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.3481558e-05
2,726 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB 8.0870349e-05
3,242 Towards Demystifying Serverless Machine Learning Training 2021 SIGMOD 7.4967268e-05
3,524 MLog: Towards Declarative In-Database Machine Learning 2017 VLDB 7.2303094e-05
3,766 CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers 2019 VLDB 7.0360981e-05
3,873 SketchML: Accelerating Distributed Machine Learning with Data Sketches 2018 SIGMOD 6.9507449e-05
4,965 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD 6.3354086e-05
5,437 PS2: Parameter Server on Spark 2019 SIGMOD 6.1279659e-05
5,635 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD 6.0549984e-05
5,724 An Experimental Evaluation of Large Scale GBDT Systems 2019 VLDB 6.0150027e-05
5,877 BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees 2019 SIGMOD 5.9599042e-05
5,999 BAGUA: Scaling up Distributed Learning with System Relaxations 2022 VLDB 5.9170896e-05
6,460 In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle 2022 SIGMOD 5.7746493e-05
6,644 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB 5.7224864e-05
7,843 ExDRa: Exploratory Data Science on Federated Raw Data 2021 SIGMOD 5.4406331e-05
8,257 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.3661896e-05
8,495 Just Move It! Dynamic Parameter Allocation in Action 2021 VLDB 5.3292461e-05
9,802 NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters 2026 SIGMOD 5.1233734e-05
9,853 DimBoost: Boosting Gradient Boosting Decision Tree to Higher Dimensions 2018 SIGMOD 5.1188938e-05
11,198 Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization 2025 SIGMOD 4.9769913e-05
12,302 LDA*: A Robust and Large-scale Topic Modeling System 2017 VLDB 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012087459
22 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.00055938421
105 The MADlib Analytics Library or MAD Skills, the SQL 2012 VLDB 0.00033633007
123 Schism: a Workload-Driven Approach to Database Replication and Partitioning 2010 VLDB 0.00030749898
195 Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design 2004 SIGMOD 0.00025619089
243 Automating Physical Database Design in a Parallel Database 2002 SIGMOD 0.00023349603
464 An Architecture for Parallel Topic Models 2010 VLDB 0.00017783871
731 Learning Generalized Linear Models Over Normalized Data 2015 SIGMOD 0.00014400356
1,009 Simulation of Database-Valued Markov Chains Using SimSQL 2013 SIGMOD 0.0001253551
1,167 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.0001172597
1,288 Hybrid-Range Partitioning Strategy: A New Declustering Strategy for Multiprocessor Database Machines 1990 VLDB 0.00011178484
4,781 Tornado: A System For Real-Time Iterative Analysis Over Evolving Data 2016 SIGMOD 6.4154314e-05
5,698 Large-scale Predictive Analytics in Vertica: Fast Data Transfer, Distributed Model Creation, and In-database Prediction 2015 SIGMOD 6.0300581e-05
12,348 Real-time Video Recommendation Exploration 2016 SIGMOD 4.9769913e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 8,985 FederatedScope: A Flexible Federated Learning Platform for Heterogeneity 2023 VLDB
2 523 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB
3 5,635 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD
4 2,726 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB
5 11,297 Heta: Distributed Training of Heterogeneous Graph Neural Networks 2025 VLDB
6 8,283 Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale 2022 VLDB
7 4,902 Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models 2017 VLDB
8 8,257 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB
9 6,644 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB
10 4,965 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD