DBScholar

Back to papers

Google’s Deep-Web Crawl

Summary: An automated, web-scale system surfaces Deep-Web content by synthesizing valid HTML-form submissions across millions of multilingual forms. It infers text/type inputs and prunes Cartesian-product combinations to generate only indexable URLs, powering Google at >1,000 queries/s. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
he4b234e59c2182cb
Venue
VLDB
Year
2008
Pagerank
0.00010168682
Overall Rank
1,582 | 89.37%
DOI
10.14778/1454159.1454163

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{madhavan_vldb08,
        title = {{Google’s Deep-Web Crawl}},
        author = {Madhavan, Jayant and Ko, David and Kot, Łucja and Ganapathy, Vignesh and Rasmussen, Alex and Halevy, Alon},
        journal = {PVLDB},
        series = {{VLDB} '08},
        volume = {1},
        number = {2},
        pages = {1241--1252},
        doi = {10.14778/1454159.1454163},
        url = {https://doi.org/10.14778/1454159.1454163},
        year = {2008}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 22 citing papers.

Rank Citing Paper Year Venue Pagerank
1,337 Harvesting Relational Tables from Lists on the Web 2009 VLDB 0.00010982201
1,505 Generic Schema Matching, Ten Years Later 2011 VLDB 0.00010449986
2,192 Data Integration: After the Teenage Years 2017 PODS 8.8826744e-05
2,394 From Data Fusion to Knowledge Fusion 2014 VLDB 8.529467e-05
2,744 Transform-Data-by-Example (TDE): An Extensible Search Engine for Data Transformations 2018 VLDB 8.064168e-05
4,551 Harnessing the Deep Web: Present and Future 2009 CIDR 6.5380754e-05
4,587 DataXFormer: An Interactive Data Transformation Tool 2015 SIGMOD 6.5148596e-05
5,526 A First Tutorial on Dataspaces 2008 VLDB 6.0922984e-05
5,867 DIADEM: Thousands of Websites to a Single Database 2014 VLDB 5.9629039e-05
6,139 DataXFormer: Leveraging the Web for Semantic Transformations 2015 CIDR 5.870906e-05
7,067 Automatically Incorporating New Sources in Keyword Search-Based Data Integration 2010 SIGMOD 5.6066383e-05
8,554 Discovering the Skyline of Web Databases 2016 VLDB 5.3146691e-05
8,744 Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment 2019 SIGMOD 5.287931e-05
8,974 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.2457737e-05
9,863 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB 5.1161474e-05
12,382 Query Reranking As A Service 2016 VLDB 4.9769913e-05
12,715 Schema Clustering and Retrieval for Multi-domain Pay-As-You-Go Data Integration Systems 2010 SIGMOD 4.9769913e-05
12,723 Optimizing Content Freshness of Relations Extracted From the Web Using Keyword Search 2010 SIGMOD 4.9769913e-05
12,732 Creating and Exploring Web Form Repositories 2010 SIGMOD 4.9769913e-05
12,791 Privacy Preservation of Aggregates in Hidden Databases: Why and How? 2009 SIGMOD 4.9769913e-05
12,816 Kosmix: High-Performance Topic Exploration using the Deep Web 2009 VLDB 4.9769913e-05
14,066 Answering Web Questions Using Structured Data - Dream or Reality? 2009 VLDB -
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
269 Reconciling Schemas of Disparate Data Sources: A Machine-Learning Approach 2001 SIGMOD 0.00022610366
301 Answering Queries Using Templates With Binding Patterns (Extended Abstract) 1995 PODS 0.00021673683
348 Crawling the Hidden Web 2001 VLDB 0.00020252678
815 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013713173
992 Web-scale Data Integration: You can only afford to Pay As You Go 2007 CIDR 0.00012641942
1,517 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB 0.00010395336
2,419 Instance-based Schema Matching for Web Databases by Domain-specific Query Probing 2004 VLDB 8.4890861e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 8,342 Accurate and Efficient Crawling for Relevant Websites 2004 VLDB
2 12,732 Creating and Exploring Web Form Repositories 2010 SIGMOD
3 89 WebTables: Exploring the Power of Tables on the Web 2008 VLDB
4 12,232 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD
5 9,833 Exploration of Deep Web Repositories 2011 VLDB
6 4,612 Probe, Count, and Classify: Categorizing Hidden-Web Databases 2001 SIGMOD
7 9,863 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB
8 348 Crawling the Hidden Web 2001 VLDB
9 8,744 Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment 2019 SIGMOD
10 4,551 Harnessing the Deep Web: Present and Future 2009 CIDR