DBScholar

Back to papers

Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment

Summary: SmartCrawl enables progressive deep-web crawling via keyword queries to enrich a local database under top-k and query-budget limits. It constructs a local data-driven query pool and iteratively issues high-benefit queries to maximize coverage, addressing ΔD and top-k effects with novel benefit estimation and optimizations. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
haa19bb0f83843538
Venue
SIGMOD
Year
2019
Pagerank
5.2705326e-05
Overall Rank
8,763 | 41.27%
DOI
10.1145/3299869.3319899

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{wang_sigmod19,
        title = {{Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment}},
        author = {Wang, Pei and Shea, Ryan and Wang, Jiannan and Wu, Eugene},
        series = {{SIGMOD} '19},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3299869.3319899},
        url = {https://dl.acm.org/doi/10.1145/3299869.3319899},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank Citing Paper Year Venue Pagerank
4,419 LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes 2024 VLDB 6.5905711e-05
8,994 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.2284279e-05
9,549 ActiveDeeper: A Model-based Active Data Enrichment System 2020 VLDB 5.1506407e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
165 Mining Frequent Patterns without Candidate Generation 2000 SIGMOD 0.00027323925
349 Crawling the Hidden Web 2001 VLDB 0.0002018926
367 InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables 2012 SIGMOD 0.00019882429
484 Data Integration for the Relational Web 2009 VLDB 0.00017535511
705 Finding Related Tables 2012 SIGMOD 0.00014626858
816 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013671567
1,037 Answering Table Queries on the Web using Column Keywords 2012 VLDB 0.00012357454
1,306 Detecting Data Errors: Where are we and what needs to be done? 2016 VLDB 0.00011076619
1,519 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB 0.0001036122
1,584 Google’s Deep-Web Crawl 2008 VLDB 0.00010136933
3,257 InfoGather+: Semantic Matching and Annotation of Numeric and Time-Varying Attributes in Web Tables 2013 SIGMOD 7.4732058e-05
3,767 Toward Large Scale Integration: Building a MetaQuerier over Databases on the Web 2005 CIDR 7.0171417e-05
5,996 A Random Walk Approach to Sampling Hidden Databases 2007 SIGMOD 5.9037651e-05
8,086 Mining a Search Engine’s Corpus: Efficient Yet Unbiased Sampling and Aggregate Estimation 2011 SIGMOD 5.3748177e-05
8,861 Unbiased Estimation of Size and Other Aggregates Over Hidden Web Databases 2010 SIGMOD 5.2477452e-05
9,898 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB 5.0992325e-05
9,899 Attribute Domain Discovery for Hidden Web Databases 2011 SIGMOD 5.0992325e-05
12,275 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD 4.9605343e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 1,271 Focused Crawling Using Context Graphs 2000 VLDB
2 12,766 Optimizing Content Freshness of Relations Extracted From the Web Using Keyword Search 2010 SIGMOD
3 1,519 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB
4 9,866 Exploration of Deep Web Repositories 2011 VLDB
5 349 Crawling the Hidden Web 2001 VLDB
6 8,355 Accurate and Efficient Crawling for Relevant Websites 2004 VLDB
7 4,623 Probe, Count, and Classify: Categorizing Hidden-Web Databases 2001 SIGMOD
8 1,584 Google’s Deep-Web Crawl 2008 VLDB
9 12,275 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD
10 9,898 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB