DBScholar

Back to papers

Analyzing and Mitigating Data Stalls in DNN Training

Summary: Finds input-pipeline stalls (storage fetch/preprocessing) dominate DNN training across models, datasets, and production hardware. DS-Analyzer provides differential what-if diagnosis, while CoorDL mitigates stalls and delivers up to 5× speedups over DALI. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
h3f0e5d33386356d4
Venue
VLDB
Year
2021
Pagerank
0.00010529036
Overall Rank
1,472 | 90.14%
DOI
10.14778/3446095.3446100
PDF
Download (CC BY-NC-ND 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{mohan_vldb21,
        title = {{Analyzing and Mitigating Data Stalls in DNN Training}},
        author = {Mohan, Jayashree and Phanishayee, Amar and Raniwala, Ashish and Chidambaram, Vijay},
        journal = {PVLDB},
        series = {{VLDB} '21},
        volume = {14},
        number = {5},
        pages = {771--784},
        doi = {10.14778/3446095.3446100},
        url = {https://doi.org/10.14778/3446095.3446100},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 16 of 16 citing papers.

Rank Citing Paper Year Venue Pagerank
2,062 tf.data: A Machine Learning Data Processing Framework 2021 VLDB 9.0820315e-05
2,692 Accelerating Recommendation System Training by Leveraging Popular Choices 2022 VLDB 8.1116599e-05
3,598 Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines 2022 SIGMOD 7.1596688e-05
3,800 FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline 2023 VLDB 6.9931619e-05
4,910 Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving 2025 SIGMOD 6.3468554e-05
5,410 GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning 2023 SIGMOD 6.1317831e-05
6,318 Progressive Compressed Records: Taking a Byte out of Deep Learning Data 2021 VLDB 5.7964881e-05
7,833 Nautilus: An Optimized System for Deep Transfer Learning over Evolving Training Datasets 2022 SIGMOD 5.4285082e-05
7,848 FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation 2024 VLDB 5.4236801e-05
9,097 TensorSocket: Shared Data Loading for Deep Learning Training 2026 SIGMOD 5.208561e-05
9,099 Scheduling Data Processing Pipelines for Incremental Training on MLP-based Recommendation Models 2025 SIGMOD 5.208561e-05
10,016 MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training 2025 SIGMOD 5.0825892e-05
10,040 cedar: Optimized and Unified Machine Learning Input Data Pipelines 2025 VLDB 5.0786422e-05
10,731 Mixtera: A Data Plane for Foundation Model Training 2026 SIGMOD 4.9605343e-05
11,314 GPEmu: A GPU Emulator for Faster and Cheaper Prototyping and Evaluation of Deep Learning System Research 2025 VLDB 4.9605343e-05
11,494 Analyzing Near-Network Hardware Acceleration with Co-Processing on DPUs 2025 VLDB 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1,154 Cerebro: A Data System for Optimized Deep Learning Model Selection 2020 VLDB 0.00011762688
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 13,734 Reimagining Deep Learning Systems Through the Lens of Data Systems 2024 VLDB
2 10,513 DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization 2026 SIGMOD
3 5,410 GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning 2023 SIGMOD
4 6,861 Serving Deep Learning Models with Deduplication from Relational Databases 2022 VLDB
5 4,848 Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses 2024 VLDB
6 4,875 Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective 2024 VLDB
7 3,800 FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline 2023 VLDB
8 9,097 TensorSocket: Shared Data Loading for Deep Learning Training 2026 SIGMOD
9 3,224 Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics 2021 VLDB
10 3,598 Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines 2022 SIGMOD