Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective

Summary: Analyzes ML-centric pitfalls in learned query optimizers—training data generation, arbitrary train/validation splits, and inconsistent benchmarking—and introduces a standardized end-to-end ML evaluation framework for LQOs. Rigorous cross-split experiments show PostgreSQL often outperforms current LQOs, challenging claimed RL-driven gains. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: hbda81d656ffc3464
Venue: VLDB
Year: 2024
Pagerank: 5.8177833e-05
Overall Rank: 6,308 | 57.59%
DOI: 10.14778/3654621.3654625

Incoming Non-self Citations Over Time

Authors

1. Claude Lehmann (Zurich University of Applied Sciences)
2. Pavel Sulimov (Zurich University of Applied Sciences)
3. Kurt Stockinger (Zurich University of Applied Sciences)

BibTeX Citation

@article{lehmann_vldb24,
        title = {{Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective}},
        author = {Lehmann, Claude and Sulimov, Pavel and Stockinger, Kurt},
        journal = {PVLDB},
        series = {{VLDB} '24},
        volume = {17},
        number = {7},
        pages = {1565--1577},
        doi = {10.14778/3654621.3654625},
        url = {https://doi.org/10.14778/3654621.3654625},
        year = {2024}
}

Incoming Citations (Sorted by Pagerank)

Showing 12 of 12 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,950	Debunking the Myth of Join Ordering: Toward Robust SQL Analytics	2025	SIGMOD	6.3421691e-05
5,683	How Good are Learned Cost Models, Really? Insights from Query Optimization Tasks	2025	SIGMOD	6.0392183e-05
6,586	Can Large Language Models Be Query Optimizer for Relational Databases?	2026	SIGMOD	5.7430662e-05
7,460	T3: Accurate and Fast Performance Prediction for Relational Database Systems With Compiled Decision Trees	2025	SIGMOD	5.5215755e-05
7,977	The Holon Approach for Simultaneously Tuning Multiple Components in a Self-Driving Database Management System with Machine Learning via Synthesized Proto-Actions	2024	VLDB	5.4142519e-05
9,300	GenJoin: Conditional Generative Plan-to-Plan Query Optimizer that Learns from Subplan Hints	2026	SIGMOD	5.1987909e-05
9,670	Low Rank Learning for Offline Query Optimization	2025	SIGMOD	5.1452097e-05
9,777	Understanding Robustness Issues of Updatable Learned Indexes: [Experiments & Analysis]	2026	SIGMOD	5.1314952e-05
10,290	This is Going to Sound Crazy, But What If We Used Large Language Models to Boost Automatic Database Tuning Algorithms By Leveraging Prior History? We Will Find Better Configurations More Quickly Than Retraining From Scratch!	2026	SIGMOD	5.0431863e-05
10,484	NeurBench: A Benchmark Suite for Learned Database Components with Drift Modeling: [Experiments & Analysis]	2026	SIGMOD	4.9793485e-05
10,713	Robust Predicate Transfer with Dynamic Execution	2026	VLDB	4.9793485e-05
10,741	OBELISK: Efficient Offline Query Planning with Bayesian Optimization-Informed Language Model Reasoning	2026	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
15	How Good Are Query Optimizers, Really?	2016	VLDB	0.00061066921
53	Eddies: Continuously Adaptive Query Processing	2000	SIGMOD	0.00040860054
145	Neo: A Learned Query Optimizer	2019	VLDB	0.0002908188
318	DeepDB: Learn from Data, not from Queries!	2020	VLDB	0.00021167555
362	Bao: Making Learned Query Optimization Practical	2021	SIGMOD	0.00019989474
512	NeuroCard: One Cardinality Estimator for All Tables	2021	VLDB	0.00017050173
982	Cardinality Estimation in DBMS: A Comprehensive Benchmark Evaluation	2022	VLDB	0.00012714044
1,199	Balsa: Learning a Query Optimizer Without Expert Demonstrations	2022	SIGMOD	0.00011563985
2,210	Lero: A Learning-to-Rank Query Optimizer	2023	VLDB	8.8257742e-05
2,690	Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection	2022	VLDB	8.1258173e-05
3,060	Towards a Hands-Free Query Optimizer through Deep Learning	2019	CIDR	7.6928239e-05
3,487	LOGER: A Learned Optimizer towards Generating Efficient and Robust Query Execution Plans	2023	VLDB	7.263041e-05
4,258	LEON: A New Framework for ML-Aided Query Optimization	2023	VLDB	6.6994722e-05
6,710	Simple Adaptive Query Processing vs. Learned Query Optimizers: Observations and Analysis	2023	VLDB	5.7019157e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	11,826	Can Transfer Learning be used to build a Query Optimizer?	2022	CIDR
2	5,039	Leveraging Query Logs and Machine Learning for Parametric Query Optimization	2022	VLDB
3	8,332	Learned Offline Query Planning via Bayesian Optimization	2025	SIGMOD
4	5,683	How Good are Learned Cost Models, Really? Insights from Query Optimization Tasks	2025	SIGMOD
5	2,210	Lero: A Learning-to-Rank Query Optimizer	2023	VLDB
6	3,060	Towards a Hands-Free Query Optimizer through Deep Learning	2019	CIDR
7	11,857	DeepO: A Learned Query Optimizer	2022	SIGMOD
8	11,453	RankPQO: Learning-to-Rank for Parametric Query Optimization	2025	VLDB
9	6,710	Simple Adaptive Query Processing vs. Learned Query Optimizers: Observations and Analysis	2023	VLDB
10	6,586	Can Large Language Models Be Query Optimizer for Relational Databases?	2026	SIGMOD