NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions
Summary: Modular benchmark for LLM-NL2SQL, decomposing systems into schema selection, candidate generation, and query revision with fine-grained accuracy/efficiency metrics. Reveals major gaps in open-source methods, plus dataset/eval flaws (e.g., noisy gold SQL), providing a reproducible baseline for fair comparison. (summarized by gpt-5.4-mini on Apr 12 2026)
Incoming Non-self Citations Over Time
No non-self incoming citations found for this paper in this database.
Authors
- 1. Shizheng Hou (National University of Singapore)
- 2. Wenqi Pei (National University of Singapore)
- 3. Nuo Chen (National University of Singapore)
- 4. Quang-Trung Ta (National University of Singapore)
- 5. Peng Lu (Zhejiang University)
- 6. Beng Chin Ooi (Zhejiang University)
BibTeX Citation
@article{hou_vldb26,
title = {{NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions}},
author = {Hou, Shizheng and Pei, Wenqi and Chen, Nuo and Ta, Quang-Trung and Lu, Peng and Ooi, Beng Chin},
journal = {PVLDB},
series = {{VLDB} '26},
volume = {19},
number = {5},
pages = {1001--1015},
doi = {10.14778/3796195.3796211},
url = {https://doi.org/10.14778/3796195.3796211},
year = {2026}
}Incoming Citations (Sorted by Pagerank)
Showing 0 of 0 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 13 of 13 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 6,936 | Reliable Text-to-SQL with Adaptive Abstention | 2025 | SIGMOD |
| 2 | 10,556 | OpenSQL: Data-Efficient Text-to-SQL for Open-Source LLMs via Synthesized Intermediate Supervision | 2026 | VLDB |
| 3 | 10,122 | BenchPress: A Human-in-the-Loop Annotation System for Rapid Text-to-SQL Benchmark Curation | 2026 | CIDR |
| 4 | 2,039 | ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems | 2024 | VLDB |
| 5 | 9,870 | Natural Language to SQL: State of the Art and Open Problems | 2025 | VLDB |
| 6 | 865 | Natural language to SQL: Where are we today? | 2020 | VLDB |
| 7 | 279 | Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation | 2024 | VLDB |
| 8 | 2,602 | NL2SQL is a solved problem... Not! | 2024 | CIDR |
| 9 | 3,787 | Combining Small Language Models and Large Language Models for Zero-Shot NL2SQL | 2024 | VLDB |
| 10 | 2,852 | The Dawn of Natural Language to SQL: Are We Fully Ready? | 2024 | VLDB |