Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines

Summary: Analyzes data preprocessing pipelines across four domains, exposing bottlenecks and throughput–storage trade-offs. Presents an open-source profiler that auto-tunes preprocessing, delivering 3x–13x throughput gains with equivalent pipelines. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6300
Venue: SIGMOD
Year: 2022
Pagerank: 6.8340435e-05
Overall Rank: 3,698 | 74.28%
DOI: 10.1145/3514221.3517848

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 9 of 9 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,180	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB	6.3793352e-05
5,552	GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning	2023	SIGMOD	5.4402488e-05
8,735	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	4.456315e-05
8,737	Scheduling Data Processing Pipelines for Incremental Training on MLP-based Recommendation Models	2025	SIGMOD	4.456315e-05
9,677	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD	4.3047774e-05
9,805	MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training	2025	SIGMOD	4.2805224e-05
9,806	The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format	2024	SIGMOD	4.2805224e-05
10,492	Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization	2025	SIGMOD	4.1945683e-05
10,770	cedar: Optimized and Unified Machine Learning Input Data Pipelines	2025	VLDB	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,504	Analyzing and Mitigating Data Stalls in DNN Training	2021	VLDB	0.00011642333
2,170	tf.data: A Machine Learning Data Processing Framework	2021	VLDB	9.3821603e-05
3,293	Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics	2021	VLDB	7.2629834e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
6,469	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD	5.0519488e-05
8,735	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	4.456315e-05
5,552	GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning	2023	SIGMOD	5.4402488e-05
3,293	Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics	2021	VLDB	7.2629834e-05
2,456	Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities	2021	SIGMOD	8.7733773e-05
5,429	DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data	2023	SIGMOD	5.5087325e-05
2,170	tf.data: A Machine Learning Data Processing Framework	2021	VLDB	9.3821603e-05
1,504	Analyzing and Mitigating Data Stalls in DNN Training	2021	VLDB	0.00011642333
8,348	FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation	2024	VLDB	4.5410024e-05
4,180	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB	6.3793352e-05