Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines

Summary: Analyzes data preprocessing pipelines across four domains, exposing bottlenecks and throughput–storage trade-offs. Presents an open-source profiler that auto-tunes preprocessing, delivering 3x–13x throughput gains with equivalent pipelines. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6362
Venue: SIGMOD
Year: 2022
Pagerank: 7.3280673e-05
Overall Rank: 3,541 | 75.71%
DOI: 10.1145/3514221.3517848

Incoming Non-self Citations Over Time

Authors

1. Alexander Isenko (Technical University of Munich)
2. Ruben Mayer (Technical University of Munich)
3. Jeffrey Jedele (Technical University of Munich)
4. Hans-Arno Jacobsen (University of Toronto)

BibTeX Citation

@inproceedings{isenko_sigmod22,
        title = {{Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines}},
        author = {Isenko, Alexander and Mayer, Ruben and Jedele, Jeffrey and Jacobsen, Hans-Arno},
        series = {{SIGMOD} '22},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3514221.3517848},
        url = {https://dl.acm.org/doi/10.1145/3514221.3517848},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 9 of 9 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,764	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB	7.1446931e-05
5,282	GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning	2023	SIGMOD	6.2840712e-05
6,900	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD	5.7430032e-05
8,905	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	5.3483178e-05
8,907	Scheduling Data Processing Pipelines for Incremental Training on MLP-based Recommendation Models	2025	SIGMOD	5.3483178e-05
9,880	MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training	2025	SIGMOD	5.2040783e-05
9,881	The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format	2024	SIGMOD	5.2040783e-05
10,769	Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization	2025	SIGMOD	5.093636e-05
10,999	cedar: Optimized and Unified Machine Learning Input Data Pipelines	2025	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,446	Analyzing and Mitigating Data Stalls in DNN Training	2021	VLDB	0.0001076818
2,018	tf.data: A Machine Learning Data Processing Framework	2021	VLDB	9.3001937e-05
3,253	Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics	2021	VLDB	7.5936939e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,309	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD
2	8,905	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD
3	5,282	GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning	2023	SIGMOD
4	3,253	Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics	2021	VLDB
5	2,018	tf.data: A Machine Learning Data Processing Framework	2021	VLDB
6	2,657	Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities	2021	SIGMOD
7	5,291	DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data	2023	SIGMOD
8	1,446	Analyzing and Mitigating Data Stalls in DNN Training	2021	VLDB
9	8,321	FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation	2024	VLDB
10	3,764	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB