Harmony: Overcoming the Hurdles of GPU Memory Capacity to Train Massive DNN Models on Commodity Servers

Summary: Harmony rethinks GPU memory management and data movement to train massive DNNs on a single commodity server. Redesigned scheduling and CPU–GPU data paths cut swap by up to 100x and yield up to 7.6x throughput over optimized virtual memory baselines. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 12761
Venue: VLDB
Year: 2022
Pagerank: 4.4855009e-05
Overall Rank: 8,607 | 40.13%
DOI: 10.14778/3551793.3551828

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
7,152	Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity	2024	VLDB	4.8154191e-05
9,326	BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach	2023	SIGMOD	4.3556432e-05

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
411	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB	0.00023906921

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
5,561	Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses	2024	VLDB	5.4332062e-05
5,710	DynaHB: A Communication-Avoiding Asynchronous Distributed Framework with Hybrid Batches for Dynamic GNN Training	2024	VLDB	5.3590055e-05
10,656	Effective and Efficient Distributed Temporal Graph Learning through Hotspot Memory Sharing	2025	VLDB	4.1945683e-05
9,264	Model-Parallel Model Selection for Deep Learning Systems	2021	SIGMOD	4.3675421e-05
8,735	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	4.456315e-05
1,103	Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture	2021	VLDB	0.00014025101
9,265	COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression	2022	VLDB	4.3667558e-05
9,596	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB	4.319218e-05
9,170	MemFlow: Memory-Aware Distributed Deep Learning	2020	SIGMOD	4.3849075e-05
9,879	HARMONY: A Scalable Distributed Vector Database for High-Throughput Approximate Nearest Neighbor Search	2026	SIGMOD	4.2643674e-05