Dynamic Parameter Allocation in Parameter Servers

Summary: Introduces dynamic parameter allocation to parameter servers, enabling parameter-access locality (PAL) for distributed ML. Lapse implements this design efficiently, achieving near-linear scaling and orders-of-magnitude speedups over existing systems. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 12274
Venue: VLDB
Year: 2020
Pagerank: 5.8536062e-05
Overall Rank: 6,519 | 55.28%
DOI: 10.14778/3407790.3407796

Incoming Non-self Citations Over Time

Authors

1. Alexander Renz-Wieland (Technical University of Berlin)
2. Rainer Gemulla (University of Mannheim)
3. Steffen Zeuch (German National Research Center for Information Technology; Technical University of Berlin)
4. Volker Markl (German National Research Center for Information Technology; Technical University of Berlin)

BibTeX Citation

@article{renzwieland_vldb20,
        title = {{Dynamic Parameter Allocation in Parameter Servers}},
        author = {Renz-Wieland, Alexander and Gemulla, Rainer and Zeuch, Steffen and Markl, Volker},
        journal = {PVLDB},
        series = {{VLDB} '20},
        volume = {13},
        number = {11},
        pages = {1877--1890},
        doi = {10.14778/3407790.3407796},
        url = {https://doi.org/10.14778/3407790.3407796},
        year = {2020}
}

Incoming Citations (Sorted by Pagerank)

Showing 7 of 7 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,169	Towards Demystifying Serverless Machine Learning Training	2021	SIGMOD	7.6715222e-05
4,067	Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches	2021	VLDB	6.9293511e-05
4,912	HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training	2022	SIGMOD	6.4481656e-05
5,305	Parallel Training of Knowledge Graph Embedding Models: A Comparison of Techniques	2022	VLDB	6.2732352e-05
6,022	NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access	2022	SIGMOD	6.0046123e-05
8,334	Just Move It! Dynamic Parameter Allocation in Action	2021	VLDB	5.4518674e-05
9,881	The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format	2024	SIGMOD	5.2040783e-05

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
20	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB	0.00056944564
452	An Architecture for Parallel Topic Models	2010	VLDB	0.00018146809
2,162	Heterogeneity-aware Distributed Parameter Servers	2017	SIGMOD	9.0581831e-05
2,615	NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion	2014	VLDB	8.340186e-05
2,693	FlexPS: Flexible Parallelism Control in Parameter Server Architecture	2018	VLDB	8.247004e-05
3,789	Husky: Towards a More Efficient and Expressive Distributed Computing Framework	2016	VLDB	7.1240627e-05
5,345	PS2: Parameter Server on Spark	2019	SIGMOD	6.2586047e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	9,729	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB
2	10,113	Distributed Learning of Fully Connected Neural Networks using Independent Subnet Training	2022	VLDB
3	2,473	PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel	2023	VLDB
4	1,079	Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML	2014	VLDB
5	521	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB
6	2,693	FlexPS: Flexible Parallelism Control in Parameter Server Architecture	2018	VLDB
7	4,956	Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce	2021	SIGMOD
8	6,022	NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access	2022	SIGMOD
9	2,162	Heterogeneity-aware Distributed Parameter Servers	2017	SIGMOD
10	8,334	Just Move It! Dynamic Parameter Allocation in Action	2021	VLDB