DBScholar

Back to papers

Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment

Summary: SmartCrawl enables progressive deep-web crawling via keyword queries to enrich a local database under top-k and query-budget limits. It constructs a local data-driven query pool and iteratively issues high-benefit queries to maximize coverage, addressing ΔD and top-k effects with novel benefit estimation and optimizations. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
haa19bb0f83843538
Venue
SIGMOD
Year
2019
Pagerank
5.2904354e-05
Overall Rank
8,736 | 41.27%
DOI
10.1145/3299869.3319899

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{wang_sigmod19,
        title = {{Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment}},
        author = {Wang, Pei and Shea, Ryan and Wang, Jiannan and Wu, Eugene},
        series = {{SIGMOD} '19},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3299869.3319899},
        url = {https://dl.acm.org/doi/10.1145/3299869.3319899},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank Citing Paper Year Venue Pagerank
4,409 LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes 2024 VLDB 6.6120807e-05
8,964 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.2482562e-05
9,511 ActiveDeeper: A Model-based Active Data Enrichment System 2020 VLDB 5.1701756e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
164 Mining Frequent Patterns without Candidate Generation 2000 SIGMOD 0.00027412227
348 Crawling the Hidden Web 2001 VLDB 0.00020262145
367 InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables 2012 SIGMOD 0.00019884846
484 Data Integration for the Relational Web 2009 VLDB 0.00017548541
717 Finding Related Tables 2012 SIGMOD 0.00014532116
814 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013719268
1,053 Answering Table Queries on the Web using Column Keywords 2012 VLDB 0.00012274222
1,344 Detecting Data Errors: Where are we and what needs to be done? 2016 VLDB 0.00010956518
1,517 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB 0.00010400233
1,582 Google’s Deep-Web Crawl 2008 VLDB 0.00010173382
3,245 InfoGather+: Semantic Matching and Annotation of Numeric and Time-Varying Attributes in Web Tables 2013 SIGMOD 7.4957822e-05
3,759 Toward Large Scale Integration: Building a MetaQuerier over Databases on the Web 2005 CIDR 7.0430209e-05
5,985 A Random Walk Approach to Sampling Hidden Databases 2007 SIGMOD 5.926154e-05
8,065 Mining a Search Engine’s Corpus: Efficient Yet Unbiased Sampling and Aggregate Estimation 2011 SIGMOD 5.3952009e-05
8,833 Unbiased Estimation of Size and Other Aggregates Over Hidden Web Databases 2010 SIGMOD 5.267646e-05
9,856 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB 5.1185704e-05
9,857 Attribute Domain Discovery for Hidden Web Databases 2011 SIGMOD 5.1185704e-05
12,226 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD 4.9793485e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 1,265 Focused Crawling Using Context Graphs 2000 VLDB
2 12,717 Optimizing Content Freshness of Relations Extracted From the Web Using Keyword Search 2010 SIGMOD
3 1,517 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB
4 9,826 Exploration of Deep Web Repositories 2011 VLDB
5 348 Crawling the Hidden Web 2001 VLDB
6 8,338 Accurate and Efficient Crawling for Relevant Websites 2004 VLDB
7 4,610 Probe, Count, and Classify: Categorizing Hidden-Web Databases 2001 SIGMOD
8 1,582 Google’s Deep-Web Crawl 2008 VLDB
9 12,226 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD
10 9,856 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB