DBScholar

Back to papers

Google’s Deep-Web Crawl

Summary: An automated, web-scale system surfaces Deep-Web content by synthesizing valid HTML-form submissions across millions of multilingual forms. It infers text/type inputs and prunes Cartesian-product combinations to generate only indexable URLs, powering Google at >1,000 queries/s. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
he4b234e59c2182cb
Venue
VLDB
Year
2008
Pagerank
0.00010136933
Overall Rank
1,584 | 89.39%
DOI
10.14778/1454159.1454163

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{madhavan_vldb08,
        title = {{Google’s Deep-Web Crawl}},
        author = {Madhavan, Jayant and Ko, David and Kot, Łucja and Ganapathy, Vignesh and Rasmussen, Alex and Halevy, Alon},
        journal = {PVLDB},
        series = {{VLDB} '08},
        volume = {1},
        number = {2},
        pages = {1241--1252},
        doi = {10.14778/1454159.1454163},
        url = {https://doi.org/10.14778/1454159.1454163},
        year = {2008}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 22 citing papers.

Rank Citing Paper Year Venue Pagerank
1,336 Harvesting Relational Tables from Lists on the Web 2009 VLDB 0.00010969394
1,503 Generic Schema Matching, Ten Years Later 2011 VLDB 0.00010428871
2,200 Data Integration: After the Teenage Years 2017 PODS 8.8631564e-05
2,396 From Data Fusion to Knowledge Fusion 2014 VLDB 8.5109743e-05
2,746 Transform-Data-by-Example (TDE): An Extensible Search Engine for Data Transformations 2018 VLDB 8.0458815e-05
4,569 Harnessing the Deep Web: Present and Future 2009 CIDR 6.5170432e-05
4,600 DataXFormer: An Interactive Data Transformation Tool 2015 SIGMOD 6.4965061e-05
5,545 A First Tutorial on Dataspaces 2008 VLDB 6.0723258e-05
5,880 DIADEM: Thousands of Websites to a Single Database 2014 VLDB 5.9431868e-05
6,157 DataXFormer: Leveraging the Web for Semantic Transformations 2015 CIDR 5.8532316e-05
7,085 Automatically Incorporating New Sources in Keyword Search-Based Data Integration 2010 SIGMOD 5.5883722e-05
8,557 Discovering the Skyline of Web Databases 2016 VLDB 5.3017168e-05
8,763 Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment 2019 SIGMOD 5.2705326e-05
8,994 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.2284279e-05
9,898 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB 5.0992325e-05
12,425 Query Reranking As A Service 2016 VLDB 4.9605343e-05
12,758 Schema Clustering and Retrieval for Multi-domain Pay-As-You-Go Data Integration Systems 2010 SIGMOD 4.9605343e-05
12,766 Optimizing Content Freshness of Relations Extracted From the Web Using Keyword Search 2010 SIGMOD 4.9605343e-05
12,775 Creating and Exploring Web Form Repositories 2010 SIGMOD 4.9605343e-05
12,834 Privacy Preservation of Aggregates in Hidden Databases: Why and How? 2009 SIGMOD 4.9605343e-05
12,859 Kosmix: High-Performance Topic Exploration using the Deep Web 2009 VLDB 4.9605343e-05
14,106 Answering Web Questions Using Structured Data - Dream or Reality? 2009 VLDB -
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
271 Reconciling Schemas of Disparate Data Sources: A Machine-Learning Approach 2001 SIGMOD 0.00022566676
301 Answering Queries Using Templates With Binding Patterns (Extended Abstract) 1995 PODS 0.00021612162
349 Crawling the Hidden Web 2001 VLDB 0.0002018926
816 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013671567
994 Web-scale Data Integration: You can only afford to Pay As You Go 2007 CIDR 0.00012610224
1,519 Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection 2002 VLDB 0.0001036122
2,432 Instance-based Schema Matching for Web Databases by Domain-specific Query Probing 2004 VLDB 8.4621184e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 8,355 Accurate and Efficient Crawling for Relevant Websites 2004 VLDB
2 12,775 Creating and Exploring Web Form Repositories 2010 SIGMOD
3 88 WebTables: Exploring the Power of Tables on the Web 2008 VLDB
4 12,275 Deeper: A Data Enrichment System Powered by Deep Web 2018 SIGMOD
5 9,866 Exploration of Deep Web Repositories 2011 VLDB
6 4,623 Probe, Count, and Classify: Categorizing Hidden-Web Databases 2001 SIGMOD
7 9,898 Optimal Algorithms for Crawling a Hidden Database in the Web 2012 VLDB
8 349 Crawling the Hidden Web 2001 VLDB
9 8,763 Progressive Deep Web Crawling Through Keyword Queries For Data Enrichment 2019 SIGMOD
10 4,569 Harnessing the Deep Web: Present and Future 2009 CIDR