Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workload

Summary: Prestroid uses tree-convolution to predict SQL query resource usage from traces, reducing encoding/padding waste in large-scale DL training. On 19k Presto queries over 20PB, it outperforms baselines and cuts memory 13.5x and epoch time 3.45x, with up to 13.2x Azure savings. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h8bac035995e2e289
Venue: SIGMOD
Year: 2021
Pagerank: 6.8807882e-05
Overall Rank: 3,978 | 73.26%
DOI: 10.1145/3448016.3457546

Incoming Non-self Citations Over Time

Authors

1. Johan Kok Zhi Kang (National University of Singapore)
2. Gaurav (GrabTaxi Holdings)
3. Sien Yi Tan (GrabTaxi Holdings)
4. Feng Cheng (GrabTaxi Holdings)
5. Shixuan Sun (National University of Singapore)
6. Bingsheng He (National University of Singapore)

BibTeX Citation

@inproceedings{kang_sigmod21,
        title = {{Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workload}},
        author = {Kang, Johan Kok Zhi and Gaurav and Tan, Sien Yi and Cheng, Feng and Sun, Shixuan and He, Bingsheng},
        series = {{SIGMOD} '21},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3448016.3457546},
        url = {https://dl.acm.org/doi/10.1145/3448016.3457546},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 16 of 16 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
2,210	Lero: A Learning-to-Rank Query Optimizer	2023	VLDB	8.8257742e-05
2,250	QueryFormer: A Tree Transformer Model for Query Plan Representation	2022	VLDB	8.7533306e-05
2,885	Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction	2022	VLDB	7.9094988e-05
4,741	Machine Learning for Databases	2021	VLDB	6.4410027e-05
5,481	A Comparative Study and Component Analysis of Query Plan Representation Techniques in ML4DB Studies	2024	VLDB	6.1125124e-05
5,716	Detect, Distill and Update: Learned DB Systems Facing Out of Distribution Data	2023	SIGMOD	6.0194657e-05
5,865	Modeling Shifting Workloads for Learned Database Systems	2024	SIGMOD	5.9659203e-05
5,871	PilotScope: Steering Databases with Machine Learning Drivers	2024	VLDB	5.9639223e-05
6,791	dbET: Execution Time Distribution-based Plan Selection	2023	SIGMOD	5.6811782e-05
7,033	Rethinking Learned Cost Models: Why Start from Scratch?	2023	SIGMOD	5.6168499e-05
8,879	Spatial Query Optimization With Learning	2024	VLDB	5.2568354e-05
8,947	Machine Unlearning in Learned Databases: An Experimental Analysis	2024	SIGMOD	5.2532248e-05
9,720	APQO: An Adaptive Framework for Parametric Query Optimization	2026	SIGMOD	5.1349531e-05
10,698	LIO: A lightweight and interpretable query optimizer based on an evolutionary forest	2026	VLDB	4.9793485e-05
10,924	TATA: An Efficient Framework for Task Transfer in Query Plan Representation	2026	VLDB	4.9793485e-05
11,430	AXE: A Task Decomposition Approach to Learned LSM Tuning	2025	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 10 of 10 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
23	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00055406774
85	Learned Cardinalities: Estimating Correlated Joins with Deep Learning	2019	CIDR	0.00035864347
145	Neo: A Learned Query Optimizer	2019	VLDB	0.0002908188
461	An End-to-End Learning-based Cost Estimator	2020	VLDB	0.00017829982
1,156	QuickSel: Quick Selectivity Learning with Mixture Models	2020	SIGMOD	0.00011777105
2,209	The Case for Predictive Database Systems: Opportunities and Challenges	2011	CIDR	8.8326479e-05
3,060	Towards a Hands-Free Query Optimizer through Deep Learning	2019	CIDR	7.6928239e-05
4,912	A Top-Down Approach to Achieving Performance Predictability in Database Systems	2017	SIGMOD	6.3578361e-05
5,158	Facilitating SQL Query Composition and Analysis	2020	SIGMOD	6.2492918e-05
7,759	AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft	2020	VLDB	5.4575614e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,829	DBEst: Revisiting Approximate Query Processing Engines with Machine Learning Models	2019	SIGMOD
2	5,194	Database Workload Characterization with Query Plan Encoders	2022	VLDB
3	5,911	On Efficient Approximate Queries over Machine Learning Models	2023	VLDB
4	281	Accelerating Machine Learning Inference with Probabilistic Predicates	2018	SIGMOD
5	461	An End-to-End Learning-based Cost Estimator	2020	VLDB
6	12,145	Query-Driven Learning for Next Generation Predictive Modeling & Analytics	2019	SIGMOD
7	560	Plan-Structured Deep Neural Network Models for Query Performance Prediction	2019	VLDB
8	9,333	Deep Query Optimization	2019	SIGMOD
9	2,834	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD
10	2,885	Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction	2022	VLDB