Towards a Hands-Free Query Optimizer through Deep Learning

Summary: Vision to replace heuristic, hand-tuned optimizers with end-to-end deep RL agents that learn plan search, cost estimation, and operator choices, reducing manual tuning. Paper analyzes deployment challenges (exploration, reward sparsity, generalization) and sketches three DL approaches. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: haa21dd53ff53ce42
Venue: CIDR
Year: 2019
Pagerank: 7.6960881e-05
Overall Rank: 3,057 | 79.46%
DOI: -
PDF: Download (CC BY 3.0)

Incoming Non-self Citations Over Time

Authors

1. Ryan Marcus (Brandeis University)
2. Olga Papaemmanouil (Brandeis University)

BibTeX Citation

@inproceedings{marcus_cidr19,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '19},
        title = {{Towards a Hands-Free Query Optimizer through Deep Learning}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Marcus, Ryan and Papaemmanouil, Olga},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 19 of 19 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
144	Neo: A Learned Query Optimizer	2019	VLDB	0.00029090793
560	Plan-Structured Deep Neural Network Models for Query Performance Prediction	2019	VLDB	0.00016408613
1,525	Updatable Learned Index with Precise Positions	2021	VLDB	0.00010355133
1,735	Flow-Loss: Learning Cardinality Estimates That Matter	2021	VLDB	9.7566604e-05
1,849	Diagnosing Root Causes of Intermittent Slow Queries in Cloud Databases	2020	VLDB	9.5032324e-05
1,890	WeTune: Automatic Discovery and Verification of Query Rewrite Rules	2022	SIGMOD	9.4234723e-05
2,765	Instance-Optimized Data Layouts for Cloud Analytics Workloads	2021	SIGMOD	8.0401855e-05
2,963	Proving Query Equivalence Using Linear Integer Arithmetic	2023	SIGMOD	7.8035846e-05
3,272	Estimating Cardinalities with Deep Sketches	2019	SIGMOD	7.4711788e-05
3,710	CARMI: A Cache-Aware Learned Index with a Cost-based Construction Algorithm	2022	VLDB	7.0775695e-05
3,979	Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workload	2021	SIGMOD	6.8791817e-05
4,191	Kepler: Robust Learning for Faster Parametric Query Optimization	2023	SIGMOD	6.7425275e-05
5,675	HMAB: Self-Driving Hierarchy of Bandits for Integrated Physical Database Design Tuning	2023	VLDB	6.0401656e-05
6,081	From Auto-tuning One Size Fits All to Self-designed and Learned Data-intensive Systems	2019	SIGMOD	5.8897947e-05
6,298	Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective	2024	VLDB	5.8177684e-05
7,958	Robust Query Processing: Mission Possible	2020	VLDB	5.4164639e-05
8,373	GenJoin: Conditional Generative Plan-to-Plan Query Optimizer that Learns from Subplan Hints	2026	SIGMOD	5.3427119e-05
8,881	Tiresias: Enabling Predictive Autonomous Storage and Indexing	2022	VLDB	5.255975e-05
10,160	Generating Application-Specific Data Layouts for In-memory Databases	2019	VLDB	5.0683727e-05

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
15	How Good Are Query Optimizers, Really?	2016	VLDB	0.00061067652
40	The Case for Learned Index Structures	2018	SIGMOD	0.00046363107
98	LEO - DB2's LEarning Optimizer	2001	VLDB	0.00034099838
158	Deep Learning for Entity Matching: A Design Space Exploration	2018	SIGMOD	0.00028038831
224	Self-Driving Database Management Systems	2017	CIDR	0.00024011047
232	Adaptive Selectivity Estimation Using Query Feedback	1994	SIGMOD	0.0002378554
454	Self-tuning Histograms: Building Histograms Without Looking at Data	1999	SIGMOD	0.00017955913
569	Towards a Robust Query Optimizer: A Principled and Practical Approach	2005	SIGMOD	0.00016244162

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	7,031	The Case for Deep Query Optimisation	2020	CIDR
2	1,580	Deep Learning Models for Selectivity Estimation of Multi-Attribute Queries	2020	SIGMOD
3	7,546	Learned Offline Query Planning via Bayesian Optimization	2025	SIGMOD
4	318	DeepDB: Learn from Data, not from Queries!	2020	VLDB
5	560	Plan-Structured Deep Neural Network Models for Query Performance Prediction	2019	VLDB
6	11,832	Can Transfer Learning be used to build a Query Optimizer?	2022	CIDR
7	11,863	DeepO: A Learned Query Optimizer	2022	SIGMOD
8	6,298	Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective	2024	VLDB
9	4,743	Machine Learning for Databases	2021	VLDB
10	9,342	Deep Query Optimization	2019	SIGMOD