Towards a Hands-Free Query Optimizer through Deep Learning

Summary: Vision to replace heuristic, hand-tuned optimizers with end-to-end deep RL agents that learn plan search, cost estimation, and operator choices, reducing manual tuning. Paper analyzes deployment challenges (exploration, reward sparsity, generalization) and sketches three DL approaches. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: haa21dd53ff53ce42
Venue: CIDR
Year: 2019
Pagerank: 7.6809918e-05
Overall Rank: 3,062 | 79.48%
DOI: -
PDF: Download (CC BY 3.0)

Incoming Non-self Citations Over Time

Authors

1. Ryan Marcus (Brandeis University)
2. Olga Papaemmanouil (Brandeis University)

BibTeX Citation

@inproceedings{marcus_cidr19,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '19},
        title = {{Towards a Hands-Free Query Optimizer through Deep Learning}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Marcus, Ryan and Papaemmanouil, Olga},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 19 of 19 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
142	Neo: A Learned Query Optimizer	2019	VLDB	0.00029114937
545	Plan-Structured Deep Neural Network Models for Query Performance Prediction	2019	VLDB	0.0001657599
1,529	Updatable Learned Index with Precise Positions	2021	VLDB	0.00010320909
1,740	Flow-Loss: Learning Cardinality Estimates That Matter	2021	VLDB	9.7289523e-05
1,771	Diagnosing Root Causes of Intermittent Slow Queries in Cloud Databases	2020	VLDB	9.6775181e-05
1,888	WeTune: Automatic Discovery and Verification of Query Rewrite Rules	2022	SIGMOD	9.4148912e-05
2,772	Instance-Optimized Data Layouts for Cloud Analytics Workloads	2021	SIGMOD	8.0148928e-05
2,971	Proving Query Equivalence Using Linear Integer Arithmetic	2023	SIGMOD	7.7851336e-05
3,282	Estimating Cardinalities with Deep Sketches	2019	SIGMOD	7.4491295e-05
3,716	CARMI: A Cache-Aware Learned Index with a Cost-based Construction Algorithm	2022	VLDB	7.0541948e-05
3,967	Efficient Deep Learning Pipelines for Accurate Cost Estimations Over Large Scale Query Workload	2021	SIGMOD	6.8772718e-05
4,202	Kepler: Robust Learning for Faster Parametric Query Optimization	2023	SIGMOD	6.7258607e-05
5,687	HMAB: Self-Driving Hierarchy of Bandits for Integrated Physical Database Design Tuning	2023	VLDB	6.0201937e-05
6,095	From Auto-tuning One Size Fits All to Self-designed and Learned Data-intensive Systems	2019	SIGMOD	5.8706416e-05
6,109	Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective	2024	VLDB	5.8676956e-05
7,975	Robust Query Processing: Mission Possible	2020	VLDB	5.3985544e-05
8,389	GenJoin: Conditional Generative Plan-to-Plan Query Optimizer that Learns from Subplan Hints	2026	SIGMOD	5.3250456e-05
8,902	Tiresias: Enabling Predictive Autonomous Storage and Indexing	2022	VLDB	5.2387388e-05
10,195	Generating Application-Specific Data Layouts for In-memory Databases	2019	VLDB	5.0516135e-05

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
15	How Good Are Query Optimizers, Really?	2016	VLDB	0.00061463175
40	The Case for Learned Index Structures	2018	SIGMOD	0.00046249672
98	LEO - DB2's LEarning Optimizer	2001	VLDB	0.00034050614
158	Deep Learning for Entity Matching: A Design Space Exploration	2018	SIGMOD	0.00028049555
224	Self-Driving Database Management Systems	2017	CIDR	0.00023973185
234	Adaptive Selectivity Estimation Using Query Feedback	1994	SIGMOD	0.00023726296
457	Self-tuning Histograms: Building Histograms Without Looking at Data	1999	SIGMOD	0.00017915659
568	Towards a Robust Query Optimizer: A Principled and Practical Approach	2005	SIGMOD	0.00016221103

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	7,051	The Case for Deep Query Optimisation	2020	CIDR
2	1,583	Deep Learning Models for Selectivity Estimation of Multi-Attribute Queries	2020	SIGMOD
3	320	DeepDB: Learn from Data, not from Queries!	2020	VLDB
4	7,561	Learned Offline Query Planning via Bayesian Optimization	2025	SIGMOD
5	545	Plan-Structured Deep Neural Network Models for Query Performance Prediction	2019	VLDB
6	11,876	Can Transfer Learning be used to build a Query Optimizer?	2022	CIDR
7	11,907	DeepO: A Learned Query Optimizer	2022	SIGMOD
8	6,109	Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective	2024	VLDB
9	4,765	Machine Learning for Databases	2021	VLDB
10	9,365	Deep Query Optimization	2019	SIGMOD