DBScholar

Back to papers

Google’s Deep-Web Crawl

Summary: An automated, web-scale system surfaces Deep-Web content by synthesizing valid HTML-form submissions across millions of multilingual forms. It infers text/type inputs and prunes Cartesian-product combinations to generate only indexable URLs, powering Google at >1,000 queries/s. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
9941
Venue
VLDB
Year
2008
Pagerank
0.00010372108
Overall Rank
1,554 | 89.34%
DOI
10.14778/1454159.1454163

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{madhavan_vldb08,
        title = {{Google’s Deep-Web Crawl}},
        author = {Madhavan, Jayant and Ko, David and Kot, Łucja and Ganapathy, Vignesh and Rasmussen, Alex and Halevy, Alon},
        journal = {PVLDB},
        series = {{VLDB} '08},
        volume = {1},
        number = {2},
        pages = {1241--1252},
        doi = {10.14778/1454159.1454163},
        url = {https://doi.org/10.14778/1454159.1454163},
        year = {2008}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 22 citing papers.

Rank Citing Paper Year Venue Pagerank
1,316 Harvesting Relational Tables from Lists on the Web 2009 VLDB 0.00011181216
1,497 Generic Schema Matching, Ten Years Later 2011 VLDB 0.00010568859
2,169 Data Integration: After the Teenage Years 2017 PODS 9.0436759e-05
2,351 From Data Fusion to Knowledge Fusion 2014 VLDB 8.7099618e-05
2,978 Transform-Data-by-Example (TDE): An Extensible Search Engine for Data Transformations 2018 VLDB 7.9030989e-05
4,471 Harnessing the Deep Web: Present and Future 2009 CIDR 6.6813981e-05
4,546 DataXFormer: An Interactive Data Transformation Tool 2015 SIGMOD 6.6374215e-05
5,389 A First Tutorial on Dataspaces 2008 VLDB 6.2346998e-05
5,745 DIADEM: Thousands of Websites to a Single Database 2014 VLDB 6.1018855e-05
6,092 DataXFormer: Leveraging the Web for Semantic Transformations 2015 CIDR 5.9794237e-05
6,931 Automatically Incorporating New Sources in Keyword Search-Based Data Integration 2010 SIGMOD 5.736358e-05
8,378 Discovering the Skyline of Web Databases 2016 VLDB 5.4387677e-05
8,705 Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment 2019 SIGMOD 5.3807913e-05
8,801 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.3687048e-05
9,683 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB 5.2352357e-05
12,083 Query Reranking As A Service 2016 VLDB 5.093636e-05
12,418 Schema Clustering and Retrieval for Multi-domain Pay-As-You-Go Data Integration Systems 2010 SIGMOD 5.093636e-05
12,426 Optimizing Content Freshness of Relations Extracted From the Web Using Keyword Search 2010 SIGMOD 5.093636e-05
12,435 Creating and Exploring Web Form Repositories 2010 SIGMOD 5.093636e-05
12,495 Privacy Preservation of Aggregates in Hidden Databases: Why and How? 2009 SIGMOD 5.093636e-05
12,520 Kosmix: High-Performance Topic Exploration using the Deep Web 2009 VLDB 5.093636e-05
13,750 Answering Web Questions Using Structured Data - Dream or Reality? 2009 VLDB -
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
263 Reconciling Schemas of Disparate Data Sources: A Machine-Learning Approach 2001 SIGMOD 0.0002305693
298 Answering Queries Using Templates With Binding Patterns (Extended Abstract) 1995 PODS 0.00022138018
340 Crawling the Hidden Web 2001 VLDB 0.00020711119
791 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013995705
969 Web-scale Data Integration: You can only afford to Pay As You Go 2007 CIDR 0.00012892148
1,483 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB 0.00010633832
2,372 Instance-based Schema Matching for Web Databases by Domain-specific Query Probing 2004 VLDB 8.6786716e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 8,166 Accurate and Efficient Crawling for Relevant Websites 2004 VLDB
2 12,435 Creating and Exploring Web Form Repositories 2010 SIGMOD
3 91 WebTables: Exploring the Power of Tables on the Web 2008 VLDB
4 11,927 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD
5 9,651 Exploration of Deep Web Repositories 2011 VLDB
6 4,524 Probe, Count, and Classify: Categorizing Hidden-Web Databases 2001 SIGMOD
7 9,683 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB
8 340 Crawling the Hidden Web 2001 VLDB
9 8,705 Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment 2019 SIGMOD
10 4,471 Harnessing the Deep Web: Present and Future 2009 CIDR