Accurate and Efficient Crawling for Relevant Websites

Summary: Two-level focused website crawler with graph-based external site selection and per-site focused page crawling. Models websites as first-class units and beats prior site-adapted focused crawlers by efficient, targeted intra-site discovery. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 9342
Venue: VLDB
Year: 2004
Pagerank: 5.4749372e-05
Overall Rank: 8,166 | 43.98%
DOI: 10.1016/B978-012088469-8.50037-1

Incoming Non-self Citations Over Time

Authors

1. Martin Ester (Simon Fraser University)
2. Hans-Peter Kriegel (University of Munich)
3. Matthias Schubert (University of Munich)

BibTeX Citation

@article{ester_vldb04,
        title = {{Accurate and Efficient Crawling for Relevant Websites}},
        author = {Ester, Martin and Kriegel, Hans-Peter and Schubert, Matthias},
        journal = {PVLDB},
        series = {{VLDB} '04},
        pages = {396},
        doi = {10.1016/B978-012088469-8.50037-1},
        url = {https://doi.org/10.1016/B978-012088469-8.50037-1},
        year = {2004}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
5,749	RankMass Crawler: A Crawler with High Personalized PageRank Coverage Guarantee	2007	VLDB	6.1008193e-05

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,403	Focused Crawling Using Context Graphs	2000	VLDB	0.000108876
2,624	Enhanced hypertext categorization using hyperlinks	1998	SIGMOD	8.3307304e-05
3,383	Distributed Hypertext Resource Discovery Through Examples	1999	VLDB	7.4557297e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,483	Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection	2002	VLDB
2	1,554	Google’s Deep-Web Crawl	2008	VLDB
3	12,354	Efficient Verification of Web-Content Searching Through Authenticated Web Crawlers	2012	VLDB
4	12,808	The BINGO! System for Information Portal Generation and Expert Web Search	2003	CIDR
5	9,683	Optimal Algorithms for Crawling a Hidden Database in the Web	2012	VLDB
6	5,749	RankMass Crawler: A Crawler with High Personalized PageRank Coverage Guarantee	2007	VLDB
7	7,532	The Evolution of the Web and Implications for an Incremental Crawler	2000	VLDB
8	8,705	Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment	2019	SIGMOD
9	340	Crawling the Hidden Web	2001	VLDB
10	1,403	Focused Crawling Using Context Graphs	2000	VLDB