ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems

Summary: ScienceBenchmark: NL-to-SQL benchmark on three complex, domain-specific scientific databases with expert-curated NL/SQL pairs. Augments limited human data with GPT-3 syntheses and shows top Spider models fail, stressing need for domain-aware, data-efficient NL-to-SQL systems. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13746
Venue: VLDB
Year: 2024
Pagerank: 8.8285962e-05
Overall Rank: 2,433 | 83.08%
DOI: 10.14778/3636218.3636225

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 11 of 11 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
998	CodeS: Towards Building Open-source Language Models for Text-to-SQL	2024	SIGMOD	0.00014729379
3,978	OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale	2025	VLDB	6.5725884e-05
8,896	SQL-Factory: A Multi-Agent Framework for High-Quality and Large-Scale SQL Generation	2026	VLDB	4.427232e-05
9,032	Sphinteract: Resolving Ambiguities in NL2SQL Through User Interaction	2025	VLDB	4.4039656e-05
9,151	The Power of Constraints in Natural Language to SQL Translation	2025	VLDB	4.3849295e-05
10,051	Are Your LLM-based Text-to-SQL Models Secure? Exploring SQL Injection via Backdoor Attacks	2026	SIGMOD	4.1945683e-05
10,221	NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions	2026	VLDB	4.1945683e-05
10,268	OpenSQL: Data-Efficient Text-to-SQL for Open-Source LLMs via Synthesized Intermediate Supervision	2026	VLDB	4.1945683e-05
10,713	CoLA: Model Collaboration for Log-based Anomaly Detection	2025	VLDB	4.1945683e-05
10,768	SiriusBI: A Comprehensive LLM-Powered Solution for Data Analytics in Business Intelligence	2025	VLDB	4.1945683e-05
10,837	Natural Language to SQL: State of the Art and Open Problems	2025	VLDB	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
206	Constructing an Interactive Natural Language Interface for Relational Databases	2015	VLDB	0.00034667032
535	ATHENA: An Ontology-Driven System for Natural Language Querying over Relational Data Stores	2016	VLDB	0.00020727678
567	NaLIR: An Interactive Natural Language Interface for Querying Relational Databases	2014	SIGMOD	0.00019966681
1,168	SODA: Generating SQL for Business Users	2012	VLDB	0.00013541143
1,591	The SDSS SkyServer - Public Access to the Sloan Digital Sky Survey Data	2002	SIGMOD	0.00011226338
2,321	DBPal: A Fully Pluggable NL2SQL Training Pipeline	2020	SIGMOD	9.03609e-05
3,501	MT-TeQL: Evaluating and Augmenting Neural NLIDB on Real-world Linguistic and Schema Variations	2022	VLDB	7.0366785e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
10,837	Natural Language to SQL: State of the Art and Open Problems	2025	VLDB	4.1945683e-05
9,974	BenchPress: A Human-in-the-Loop Annotation System for Rapid Text-to-SQL Benchmark Curation	2026	CIDR	4.1945683e-05
5,437	SNAILS: Schema Naming Assessments for Improved LLM-Based SQL Inference	2025	SIGMOD	5.5033018e-05
4,825	Synthesizing Natural Language to Visualization (NL2VIS) Benchmarks from NL2SQL Benchmarks	2021	SIGMOD	5.8946721e-05
1,732	CatSQL: Towards Real World Natural Language to SQL Applications	2023	VLDB	0.00010732004
2,988	NL2SQL is a solved problem... Not!	2024	CIDR	7.7761714e-05
369	Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation	2024	VLDB	0.0002547515
3,662	The Dawn of Natural Language to SQL: Are We Fully Ready?	2024	VLDB	6.8672143e-05
984	Natural language to SQL: Where are we today?	2020	VLDB	0.00014857465
10,221	NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions	2026	VLDB	4.1945683e-05