DBScholar

Back to papers

GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning

Summary: GoldMiner decouples data pre-processing from model training with stateless data workers that elastically pool cluster resources. By automatically extracting stateless pre-processing from pipelines, it scales across nodes, delivering up to 12.1x faster jobs and up to 2.5x better GPU utilization in large clusters. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
6758
Venue
SIGMOD
Year
2023
Pagerank
6.2840712e-05
Overall Rank
5,282 | 63.77%
DOI
10.1145/3589773

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{zhao_sigmod23,
        title = {{GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning}},
        author = {Zhao, Hanyu and Yang, Zhi and Cheng, Yu and Tian, Chao and Ren, Shiru and Xiao, Wencong and Yuan, Man and Chen, Langshi and Liu, Kaibo and Zhang, Yang and Li, Yong and Lin, Wei},
        series = {{SIGMOD} '23},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3589773},
        url = {https://dl.acm.org/doi/10.1145/3589773},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank Citing Paper Year Venue Pagerank
8,321 FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation 2024 VLDB 5.4544352e-05
8,907 Scheduling Data Processing Pipelines for Incremental Training on MLP-based Recommendation Models 2025 SIGMOD 5.3483178e-05
10,769 Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization 2025 SIGMOD 5.093636e-05
10,844 GPEmu: A GPU Emulator for Faster and Cheaper Prototyping and Evaluation of Deep Learning System Research 2025 VLDB 5.093636e-05
10,999 cedar: Optimized and Unified Machine Learning Input Data Pipelines 2025 VLDB 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1,446 Analyzing and Mitigating Data Stalls in DNN Training 2021 VLDB 0.0001076818
2,018 tf.data: A Machine Learning Data Processing Framework 2021 VLDB 9.3001937e-05
3,541 Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines 2022 SIGMOD 7.3280673e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 8,883 FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement 2023 SIGMOD
2 1,446 Analyzing and Mitigating Data Stalls in DNN Training 2021 VLDB
3 4,049 Resource Elasticity for Large-Scale Machine Learning 2015 SIGMOD
4 8,321 FusionFlow: Accelerating Data Preprocessing for Machine Learning with CPU-GPU Cooperation 2024 VLDB
5 8,905 TensorSocket: Shared Data Loading for Deep Learning Training 2026 SIGMOD
6 6,954 DLRover-RM: Resource Optimization for Deep Recommendation Models Training in the Cloud 2024 VLDB
7 3,764 FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline 2023 VLDB
8 3,541 Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines 2022 SIGMOD
9 2,018 tf.data: A Machine Learning Data Processing Framework 2021 VLDB
10 10,583 Resilience-Aware Elastic Scaling for Cloud-Native Online DL Training on Multi-Tenant GPU Clusters 2026 VLDB