GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning

Summary: GoldMiner decouples data pre-processing from model training with stateless data workers that elastically pool cluster resources. By automatically extracting stateless pre-processing from pipelines, it scales across nodes, delivering up to 12.1x faster jobs and up to 2.5x better GPU utilization in large clusters. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6697
Venue: SIGMOD
Year: 2023
Pagerank: 5.4350242e-05
Overall Rank: 5,560 | 61.36%
DOI: 10.1145/3589773

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
8,343	FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation	2024	VLDB	4.5366487e-05
8,733	Scheduling Data Processing Pipelines for Incremental Training on MLP-based Recommendation Models	2025	SIGMOD	4.4520434e-05
10,502	Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization	2025	SIGMOD	4.1905499e-05
10,589	GPEmu: A GPU Emulator for Faster and Cheaper Prototyping and Evaluation of Deep Learning System Research	2025	VLDB	4.1905499e-05
10,776	cedar: Optimized and Unified Machine Learning Input Data Pipelines	2025	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,500	Analyzing and Mitigating Data Stalls in DNN Training	2021	VLDB	0.00011636174
2,175	tf.data: A Machine Learning Data Processing Framework	2021	VLDB	9.3745231e-05
3,694	Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines	2022	SIGMOD	6.8316905e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
8,807	FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement	2023	SIGMOD	4.4413307e-05
1,500	Analyzing and Mitigating Data Stalls in DNN Training	2021	VLDB	0.00011636174
4,807	Resource Elasticity for Large-Scale Machine Learning	2015	SIGMOD	5.9045148e-05
8,731	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	4.4520434e-05
8,343	FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation	2024	VLDB	4.5366487e-05
5,997	DLRover-RM: Resource Optimization for Deep Recommendation Models Training in the Cloud	2024	VLDB	5.2365238e-05
4,175	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB	6.3772575e-05
3,694	Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines	2022	SIGMOD	6.8316905e-05
2,175	tf.data: A Machine Learning Data Processing Framework	2021	VLDB	9.3745231e-05
10,297	Resilience-Aware Elastic Scaling for Cloud-Native Online DL Training on Multi-Tenant GPU Clusters	2026	VLDB	4.1905499e-05