DBScholar

Back to papers

Heterogeneity-aware Distributed Parameter Servers

Summary: Heterogeneity-aware distributed parameter servers for SGD in heterogeneous clusters, addressing stragglers and sync bottlenecks. Proposes constant learning-rate pre-aggregation and delayed-update schedules with convergence guarantees; Tencent prototype yields 2–12x speedups and up to 6x fewer iterations vs Spark, Petuum, TF. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
ha5f19089820bfbc0
Venue
SIGMOD
Year
2017
Pagerank
8.8807201e-05
Overall Rank
2,196 | 85.29%
DOI
10.1145/3035918.3035933

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{jiang_sigmod17,
        title = {{Heterogeneity-aware Distributed Parameter Servers}},
        author = {Jiang, Jiawei and Cui, Bin and Zhang, Ce and Yu, Lele},
        series = {{SIGMOD} '17},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3035918.3035933},
        url = {https://dl.acm.org/doi/10.1145/3035918.3035933},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
1,047 SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.00012313386
1,154 Cerebro: A Data System for Optimized Deep Learning Model Selection 2020 VLDB 0.00011762688
2,525 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.3302991e-05
2,737 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB 8.060868e-05
3,256 Towards Demystifying Serverless Machine Learning Training 2021 SIGMOD 7.4732677e-05
3,520 MLog: Towards Declarative In-Database Machine Learning 2017 VLDB 7.2180592e-05
3,770 CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers 2019 VLDB 7.0145345e-05
3,887 SketchML: Accelerating Distributed Machine Learning with Data Sketches 2018 SIGMOD 6.9277708e-05
4,961 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD 6.3246342e-05
5,454 PS2: Parameter Server on Spark 2019 SIGMOD 6.1077156e-05
5,648 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD 6.0350483e-05
5,738 An Experimental Evaluation of Large Scale GBDT Systems 2019 VLDB 5.9951855e-05
5,885 BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees 2019 SIGMOD 5.9421702e-05
6,011 BAGUA: Scaling up Distributed Learning with System Relaxations 2022 VLDB 5.8975241e-05
6,474 In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle 2022 SIGMOD 5.7555547e-05
6,655 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB 5.7036819e-05
7,860 ExDRa: Exploratory Data Science on Federated Raw Data 2021 SIGMOD 5.422643e-05
8,228 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.3581744e-05
8,511 Just Move It! Dynamic Parameter Allocation in Action 2021 VLDB 5.3116274e-05
9,835 NeutronHeter: Optimizing Distributed Graph Neural Network Training for Heterogeneous Clusters 2026 SIGMOD 5.1064323e-05
9,889 DimBoost: Boosting Gradient Boosting Decision Tree to Higher Dimensions 2018 SIGMOD 5.1020145e-05
11,257 Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization 2025 SIGMOD 4.9605343e-05
12,345 LDA*: A Robust and Large-scale Topic Modeling System 2017 VLDB 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012089097
22 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.00056028465
105 The MADlib Analytics Library or MAD Skills, the SQL 2012 VLDB 0.00033605348
122 Schism: a Workload-Driven Approach to Database Replication and Partitioning 2010 VLDB 0.00030701814
196 Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design 2004 SIGMOD 0.00025581936
243 Automating Physical Database Design in a Parallel Database 2002 SIGMOD 0.00023296011
464 An Architecture for Parallel Topic Models 2010 VLDB 0.0001776101
731 Learning Generalized Linear Models Over Normalized Data 2015 SIGMOD 0.00014363943
1,009 Simulation of Database-Valued Markov Chains Using SimSQL 2013 SIGMOD 0.00012518224
1,169 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.00011690039
1,291 Hybrid-Range Partitioning Strategy: A New Declustering Strategy for Multiprocessor Database Machines 1990 VLDB 0.00011147286
4,800 Tornado: A System For Real-Time Iterative Analysis Over Evolving Data 2016 SIGMOD 6.3942642e-05
5,707 Large-scale Predictive Analytics in Vertica: Fast Data Transfer, Distributed Model Creation, and In-database Prediction 2015 SIGMOD 6.011012e-05
12,391 Real-time Video Recommendation Exploration 2016 SIGMOD 4.9605343e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 9,005 FederatedScope: A Flexible Federated Learning Platform for Heterogeneity 2023 VLDB
2 526 PyTorch Distributed: Experiences on Accelerating Data Parallel Training 2020 VLDB
3 5,648 NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access 2022 SIGMOD
4 2,737 FlexPS: Flexible Parallelism Control in Parameter Server Architecture 2018 VLDB
5 11,352 Heta: Distributed Training of Heterogeneous Graph Neural Networks 2025 VLDB
6 8,298 Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale 2022 VLDB
7 4,927 Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models 2017 VLDB
8 8,228 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB
9 6,655 Dynamic Parameter Allocation in Parameter Servers 2020 VLDB
10 4,961 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD