ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems

Summary: ScienceBenchmark: NL-to-SQL benchmark on three complex, domain-specific scientific databases with expert-curated NL/SQL pairs. Augments limited human data with GPT-3 syntheses and shows top Spider models fail, stressing need for domain-aware, data-efficient NL-to-SQL systems. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13747
Venue: VLDB
Year: 2024
Pagerank: 8.8218963e-05
Overall Rank: 2,435 | 83.08%
DOI: 10.14778/3636218.3636225

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 11 of 11 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
998	CodeS: Towards Building Open-source Language Models for Text-to-SQL	2024	SIGMOD	0.00014726344
3,978	OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale	2025	VLDB	6.5662694e-05
8,896	SQL-Factory: A Multi-Agent Framework for High-Quality and Large-Scale SQL Generation	2026	VLDB	4.4229886e-05
9,035	Sphinteract: Resolving Ambiguities in NL2SQL Through User Interaction	2025	VLDB	4.3997447e-05
9,151	The Power of Constraints in Natural Language to SQL Translation	2025	VLDB	4.380727e-05
10,051	Are Your LLM-based Text-to-SQL Models Secure? Exploring SQL Injection via Backdoor Attacks	2026	SIGMOD	4.1905499e-05
10,221	NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions	2026	VLDB	4.1905499e-05
10,268	OpenSQL: Data-Efficient Text-to-SQL for Open-Source LLMs via Synthesized Intermediate Supervision	2026	VLDB	4.1905499e-05
10,720	CoLA: Model Collaboration for Log-based Anomaly Detection	2025	VLDB	4.1905499e-05
10,774	SiriusBI: A Comprehensive LLM-Powered Solution for Data Analytics in Business Intelligence	2025	VLDB	4.1905499e-05
10,841	Natural Language to SQL: State of the Art and Open Problems	2025	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
205	Constructing an Interactive Natural Language Interface for Relational Databases	2015	VLDB	0.00034651121
535	ATHENA: An Ontology-Driven System for Natural Language Querying over Relational Data Stores	2016	VLDB	0.00020718836
565	NaLIR: An Interactive Natural Language Interface for Querying Relational Databases	2014	SIGMOD	0.00019945986
1,168	SODA: Generating SQL for Business Users	2012	VLDB	0.00013531301
1,592	The SDSS SkyServer - Public Access to the Sloan Digital Sky Survey Data	2002	SIGMOD	0.00011215586
2,325	DBPal: A Fully Pluggable NL2SQL Training Pipeline	2020	SIGMOD	9.0277894e-05
3,506	MT-TeQL: Evaluating and Augmenting Neural NLIDB on Real-world Linguistic and Schema Variations	2022	VLDB	7.0301746e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
10,841	Natural Language to SQL: State of the Art and Open Problems	2025	VLDB	4.1905499e-05
9,973	BenchPress: A Human-in-the-Loop Annotation System for Rapid Text-to-SQL Benchmark Curation	2026	CIDR	4.1905499e-05
5,448	SNAILS: Schema Naming Assessments for Improved LLM-Based SQL Inference	2025	SIGMOD	5.4980173e-05
4,829	Synthesizing Natural Language to Visualization (NL2VIS) Benchmarks from NL2SQL Benchmarks	2021	SIGMOD	5.8890126e-05
1,731	CatSQL: Towards Real World Natural Language to SQL Applications	2023	VLDB	0.00010723825
2,987	NL2SQL is a solved problem... Not!	2024	CIDR	7.77529e-05
366	Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation	2024	VLDB	0.00025580097
3,666	The Dawn of Natural Language to SQL: Are We Fully Ready?	2024	VLDB	6.8606092e-05
973	Natural language to SQL: Where are we today?	2020	VLDB	0.0001488435
10,221	NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions	2026	VLDB	4.1905499e-05