SmartBench: Demonstrating Automatic Generation of Comprehensive Benchmarks for Question Answering Over Knowledge Graphs

Summary: SmartBench: automatic benchmark generation for QA over any KG. It addresses benchmark variability, stagnation, and KG diversity by guaranteeing coverage of NL question/query properties seen in the literature, as long as the KG contains those properties. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 13045
Venue: VLDB
Year: 2022
Pagerank: 5.2917932e-05
Overall Rank: 9,286 | 36.30%
DOI: 10.14778/3554821.3554869

Incoming Non-self Citations Over Time

Authors

1. Abdelghny Orogat (Carleton University)
2. Ahmed El-Roby (Carleton University)

BibTeX Citation

@article{orogat_vldb22,
        title = {{SmartBench: Demonstrating Automatic Generation of Comprehensive Benchmarks for Question Answering Over Knowledge Graphs}},
        author = {Orogat, Abdelghny and El-Roby, Ahmed},
        journal = {PVLDB},
        series = {{VLDB} '22},
        volume = {15},
        number = {12},
        pages = {3662--3665},
        doi = {10.14778/3554821.3554869},
        url = {https://doi.org/10.14778/3554821.3554869},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
5,969	Maestro: Automatic Generation of Comprehensive Benchmarks for Question Answering Over Knowledge Graphs	2023	SIGMOD	6.0247307e-05
6,149	Efficient Exact Subgraph Matching via GNN-based Path Dominance Embedding	2024	VLDB	5.9581422e-05
10,354	Continuous Subgraph Matching via Cost-Model-based Dynamic Vertex Dominance Embeddings	2026	SIGMOD	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 2 of 2 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
11,694	CBench: Towards Better Evaluation of Question Answering Over Knowledge Graphs	2021	VLDB	5.093636e-05
13,457	CBench: Demonstrating Comprehensive Evaluation of Question Answering Systems over Knowledge Graphs Through Deep Analysis of Benchmarks	2021	VLDB	-

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	7,369	SmartBench: A Benchmark For Data Management In Smart Spaces	2020	VLDB
2	10,663	Dialogue Benchmark Generation from Knowledge Graphs with Cost-Effective Retrieval-Augmented LLMs	2025	SIGMOD
3	3,981	Natural Language Question Answering over RDF — A Graph Data Driven Approach	2014	SIGMOD
4	4,056	A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs	2020	VLDB
5	3,699	KBQA: Learning Question Answering over QA Corpora and Knowledge Bases	2017	VLDB
6	5,671	Question Answering Over Knowledge Graphs: Question Understanding Via Template Decomposition	2018	VLDB
7	10,023	A Universal Question-Answering Platform for Knowledge Graphs	2023	SIGMOD
8	5,969	Maestro: Automatic Generation of Comprehensive Benchmarks for Question Answering Over Knowledge Graphs	2023	SIGMOD
9	11,694	CBench: Towards Better Evaluation of Question Answering Over Knowledge Graphs	2021	VLDB
10	13,457	CBench: Demonstrating Comprehensive Evaluation of Question Answering Systems over Knowledge Graphs Through Deep Analysis of Benchmarks	2021	VLDB