Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection

Summary: Proposes metasearch over hidden-web databases via adaptive probes to produce content summaries with absolute word-frequency estimates. Introduces a hierarchical selection algorithm using these summaries and an induced taxonomy to surpass flat methods, validated on 50 real databases. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h3c601fd59d6d4416
Venue: VLDB
Year: 2002
Pagerank: 0.00010400233
Overall Rank: 1,517 | 89.81%
DOI: 10.1016/B978-155860869-6/50042-1

Incoming Non-self Citations Over Time

Authors

1. Panagiotis G. Ipeirotis (Columbia University)
2. Luis Gravano (Columbia University)

BibTeX Citation

@article{ipeirotis_vldb02,
        title = {{Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection}},
        author = {Ipeirotis, Panagiotis G. and Gravano, Luis},
        journal = {PVLDB},
        series = {{VLDB} '02},
        doi = {10.1016/B978-155860869-6/50042-1},
        url = {https://doi.org/10.1016/B978-155860869-6/50042-1},
        year = {2002}
}

Incoming Citations (Sorted by Pagerank)

Showing 15 of 15 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
997	To Search or to Crawl? Towards a Query Optimizer for Text-Centric Tasks	2006	SIGMOD	0.00012633809
1,582	Google’s Deep-Web Crawl	2008	VLDB	0.00010173382
1,872	Information Sharing Across Private Databases	2003	SIGMOD	9.4632771e-05
4,549	Harnessing the Deep Web: Present and Future	2009	CIDR	6.5410269e-05
5,845	Effective Keyword-based Selection of Relational Databases	2007	SIGMOD	5.9729417e-05
6,997	Facet Discovery for Structured Web Search: A Query-log Mining Approach	2011	SIGMOD	5.6254522e-05
8,065	Mining a Search Engine’s Corpus: Efficient Yet Unbiased Sampling and Aggregate Estimation	2011	SIGMOD	5.3952009e-05
8,736	Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment	2019	SIGMOD	5.2904354e-05
8,833	Unbiased Estimation of Size and Other Aggregates Over Hidden Web Databases	2010	SIGMOD	5.267646e-05
9,856	Optimal Algorithms for Crawling a Hidden Database in the Web	2012	VLDB	5.1185704e-05
9,857	Attribute Domain Discovery for Hidden Web Databases	2011	SIGMOD	5.1185704e-05
12,576	Rank Discovery From Web Databases	2013	VLDB	4.9793485e-05
12,785	Privacy Preservation of Aggregates in Hidden Databases: Why and How?	2009	SIGMOD	4.9793485e-05
12,810	Kosmix: High-Performance Topic Exploration using the Deep Web	2009	VLDB	4.9793485e-05
13,058	When one Sample is not Enough: Improving Text Database Selection Using Shrinkage	2004	SIGMOD	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,631	Automatic Discovery of Language Models for Text Databases	1999	SIGMOD	0.00010022634
1,655	Determining Text Databases to Search in the Internet	1998	VLDB	9.9733472e-05
4,610	Probe, Count, and Classify: Categorizing Hidden-Web Databases	2001	SIGMOD	6.5030302e-05
5,137	STARTS: Stanford Proposal for Internet Meta-Searching	1997	SIGMOD	6.2582423e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	8,736	Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment	2019	SIGMOD
2	14,320	A Method of Re-ranking Web Search Results Using their Hidden Hyperlink Structure	2002	VLDB
3	8,833	Unbiased Estimation of Size and Other Aggregates Over Hidden Web Databases	2010	SIGMOD
4	1,655	Determining Text Databases to Search in the Internet	1998	VLDB
5	5,845	Effective Keyword-based Selection of Relational Databases	2007	SIGMOD
6	13,058	When one Sample is not Enough: Improving Text Database Selection Using Shrinkage	2004	SIGMOD
7	3,346	Distributed Hypertext Resource Discovery Through Examples	1999	VLDB
8	9,856	Optimal Algorithms for Crawling a Hidden Database in the Web	2012	VLDB
9	9,108	Efficient and Effective Metasearch for Text Databases Incorporating Linkages among Documents	2001	SIGMOD
10	4,610	Probe, Count, and Classify: Categorizing Hidden-Web Databases	2001	SIGMOD