DBScholar

Back to papers

Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment

Summary: SmartCrawl enables progressive deep-web crawling via keyword queries to enrich a local database under top-k and query-budget limits. It constructs a local data-driven query pool and iteratively issues high-benefit queries to maximize coverage, addressing ΔD and top-k effects with novel benefit estimation and optimizations. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
5773
Venue
SIGMOD
Year
2019
Pagerank
5.3807913e-05
Overall Rank
8,705 | 40.28%
DOI
10.1145/3299869.3319899

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{wang_sigmod19,
        title = {{Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment}},
        author = {Wang, Pei and Shea, Ryan and Wang, Jiannan and Wu, Eugene},
        series = {{SIGMOD} '19},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3299869.3319899},
        url = {https://dl.acm.org/doi/10.1145/3299869.3319899},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank Citing Paper Year Venue Pagerank
7,893 LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes 2024 VLDB 5.5209866e-05
8,801 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.3687048e-05
9,348 ActiveDeeper: A Model-based Active Data Enrichment System 2020 VLDB 5.2843006e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
161 Mining Frequent Patterns without Candidate Generation 2000 SIGMOD 0.00027981772
340 Crawling the Hidden Web 2001 VLDB 0.00020711119
367 InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables 2012 SIGMOD 0.00019979463
493 Data Integration for the Relational Web 2009 VLDB 0.00017558709
739 Finding Related Tables 2012 SIGMOD 0.0001448231
791 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013995705
1,055 Answering Table Queries on the Web using Column Keywords 2012 VLDB 0.00012389274
1,351 Detecting Data Errors: Where are we and what needs to be done? 2016 VLDB 0.00011064851
1,483 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB 0.00010633832
1,554 Google’s Deep-Web Crawl 2008 VLDB 0.00010372108
3,181 InfoGather+: Semantic Matching and Annotation of Numeric and Time-Varying Attributes in Web Tables 2013 SIGMOD 7.6595256e-05
3,690 Toward Large Scale Integration: Building a MetaQuerier over Databases on the Web 2005 CIDR 7.2007927e-05
5,868 A Random Walk Approach to Sampling Hidden Databases 2007 SIGMOD 6.0613238e-05
7,903 Mining a Search Engine’s Corpus: Efficient Yet Unbiased Sampling and Aggregate Estimation 2011 SIGMOD 5.5182155e-05
8,671 Unbiased Estimation of Size and Other Aggregates Over Hidden Web Databases 2010 SIGMOD 5.3876663e-05
9,683 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB 5.2352357e-05
9,684 Attribute Domain Discovery for Hidden Web Databases 2011 SIGMOD 5.2352357e-05
11,927 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD 5.093636e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 1,403 Focused Crawling Using Context Graphs 2000 VLDB
2 12,426 Optimizing Content Freshness of Relations Extracted From the Web Using Keyword Search 2010 SIGMOD
3 1,483 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB
4 9,651 Exploration of Deep Web Repositories 2011 VLDB
5 340 Crawling the Hidden Web 2001 VLDB
6 8,166 Accurate and Efficient Crawling for Relevant Websites 2004 VLDB
7 4,524 Probe, Count, and Classify: Categorizing Hidden-Web Databases 2001 SIGMOD
8 1,554 Google’s Deep-Web Crawl 2008 VLDB
9 11,927 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD
10 9,683 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB