Analyzing and Mitigating Data Stalls in DNN Training

Summary: Data-pipeline stalls often dominate DNN training time for CV and audio models; large-scale study covers nine models, four datasets, three tasks. DS-Analyzer quantifies stalls; CoorDL's three loading techniques yield up to 5x speedups vs DALI. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 12594
Venue: VLDB
Year: 2021
Pagerank: 0.00011636174
Overall Rank: 1,500 | 89.58%
DOI: 10.14778/3446095.3446100

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 16 of 16 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
2,175	tf.data: A Machine Learning Data Processing Framework	2021	VLDB	9.3745231e-05
2,690	Accelerating Recommendation System Training by Leveraging Popular Choices	2022	VLDB	8.2911466e-05
3,694	Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines	2022	SIGMOD	6.8316905e-05
4,175	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB	6.3772575e-05
5,560	GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning	2023	SIGMOD	5.4350242e-05
6,059	Progressive Compressed Records: Taking a Byte out of Deep Learning Data	2021	VLDB	5.2272544e-05
7,656	Nautilus: An Optimized System for Deep Transfer Learning over Evolving Training Datasets	2022	SIGMOD	4.6826896e-05
8,343	FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation	2024	VLDB	4.5366487e-05
8,731	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	4.4520434e-05
8,733	Scheduling Data Processing Pipelines for Incremental Training on MLP-based Recommendation Models	2025	SIGMOD	4.4520434e-05
9,677	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD	4.3006524e-05
9,786	MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training	2025	SIGMOD	4.2799988e-05
10,183	Mixtera: A Data Plane for Foundation Model Training	2026	SIGMOD	4.1905499e-05
10,589	GPEmu: A GPU Emulator for Faster and Cheaper Prototyping and Evaluation of Deep Learning System Research	2025	VLDB	4.1905499e-05
10,776	cedar: Optimized and Unified Machine Learning Input Data Pipelines	2025	VLDB	4.1905499e-05
10,860	Analyzing Near-Network Hardware Acceleration with Co-Processing on DPUs	2025	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
684	Cerebro: A Data System for Optimized Deep Learning Model Selection	2020	VLDB	0.00018152321

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
13,185	Reimagining Deep Learning Systems Through the Lens of Data Systems	2024	VLDB	-
7,286	DAHA: Accelerating GNN Training with Data and Hardware Aware Execution Planning	2024	VLDB	4.7701372e-05
5,560	GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning	2023	SIGMOD	5.4350242e-05
7,055	Serving Deep Learning Models with Deduplication from Relational Databases	2022	VLDB	4.8428976e-05
5,570	Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses	2024	VLDB	5.4280174e-05
5,746	Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective	2024	VLDB	5.3429324e-05
4,175	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB	6.3772575e-05
8,731	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	4.4520434e-05
3,291	Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics	2021	VLDB	7.2607192e-05
3,694	Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines	2022	SIGMOD	6.8316905e-05