Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection

Summary: Proposes metasearch over hidden-web databases via adaptive probes to produce content summaries with absolute word-frequency estimates. Introduces a hierarchical selection algorithm using these summaries and an induced taxonomy to surpass flat methods, validated on 50 real databases. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h3c601fd59d6d4416
Venue: VLDB
Year: 2002
Pagerank: 0.0001036122
Overall Rank: 1,519 | 89.82%
DOI: 10.1016/B978-155860869-6/50042-1

Incoming Non-self Citations Over Time

Authors

1. Panagiotis G. Ipeirotis (Columbia University)
2. Luis Gravano (Columbia University)

BibTeX Citation

@article{ipeirotis_vldb02,
        title = {{Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection}},
        author = {Ipeirotis, Panagiotis G. and Gravano, Luis},
        journal = {PVLDB},
        series = {{VLDB} '02},
        doi = {10.1016/B978-155860869-6/50042-1},
        url = {https://doi.org/10.1016/B978-155860869-6/50042-1},
        year = {2002}
}

Incoming Citations (Sorted by Pagerank)

Showing 15 of 15 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
998	To Search or to Crawl? Towards a Query Optimizer for Text-Centric Tasks	2006	SIGMOD	0.00012590871
1,584	Google’s Deep-Web Crawl	2008	VLDB	0.00010136933
1,880	Information Sharing Across Private Databases	2003	SIGMOD	9.427621e-05
4,569	Harnessing the Deep Web: Present and Future	2009	CIDR	6.5170432e-05
5,855	Effective Keyword-based Selection of Relational Databases	2007	SIGMOD	5.9516101e-05
7,014	Facet Discovery for Structured Web Search: A Query-log Mining Approach	2011	SIGMOD	5.60446e-05
8,086	Mining a Search Engine’s Corpus: Efficient Yet Unbiased Sampling and Aggregate Estimation	2011	SIGMOD	5.3748177e-05
8,763	Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment	2019	SIGMOD	5.2705326e-05
8,861	Unbiased Estimation of Size and Other Aggregates Over Hidden Web Databases	2010	SIGMOD	5.2477452e-05
9,898	Optimal Algorithms for Crawling a Hidden Database in the Web	2012	VLDB	5.0992325e-05
9,899	Attribute Domain Discovery for Hidden Web Databases	2011	SIGMOD	5.0992325e-05
12,625	Rank Discovery From Web Databases	2013	VLDB	4.9605343e-05
12,834	Privacy Preservation of Aggregates in Hidden Databases: Why and How?	2009	SIGMOD	4.9605343e-05
12,859	Kosmix: High-Performance Topic Exploration using the Deep Web	2009	VLDB	4.9605343e-05
13,105	When one Sample is not Enough: Improving Text Database Selection Using Shrinkage	2004	SIGMOD	4.9605343e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,638	Automatic Discovery of Language Models for Text Databases	1999	SIGMOD	9.9886199e-05
1,663	Determining Text Databases to Search in the Internet	1998	VLDB	9.9393733e-05
4,623	Probe, Count, and Classify: Categorizing Hidden-Web Databases	2001	SIGMOD	6.4814577e-05
5,163	STARTS: Stanford Proposal for Internet Meta-Searching	1997	SIGMOD	6.2349815e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	8,763	Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment	2019	SIGMOD
2	14,365	A Method of Re-ranking Web Search Results Using their Hidden Hyperlink Structure	2002	VLDB
3	8,861	Unbiased Estimation of Size and Other Aggregates Over Hidden Web Databases	2010	SIGMOD
4	1,663	Determining Text Databases to Search in the Internet	1998	VLDB
5	5,855	Effective Keyword-based Selection of Relational Databases	2007	SIGMOD
6	13,105	When one Sample is not Enough: Improving Text Database Selection Using Shrinkage	2004	SIGMOD
7	3,355	Distributed Hypertext Resource Discovery Through Examples	1999	VLDB
8	9,898	Optimal Algorithms for Crawling a Hidden Database in the Web	2012	VLDB
9	9,136	Efficient and Effective Metasearch for Text Databases Incorporating Linkages among Documents	2001	SIGMOD
10	4,623	Probe, Count, and Classify: Categorizing Hidden-Web Databases	2001	SIGMOD