DBScholar

Back to papers

Web-scale Data Integration: You can only afford to Pay As You Go

Summary: Argues that traditional tightly-coupled integration fails at web scale (Deep Web, Google Base) due to extreme heterogeneity and scale. Proposes PAYGO, a dataspaces-inspired pay-as-you-go architecture that delivers incremental, best-effort, cost-aware integration to maximize utility under limited resources. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
h22f1f81c337a5188
Venue
CIDR
Year
2007
Pagerank
0.00012647155
Overall Rank
991 | 93.34%
DOI
-

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{madhavan_cidr07,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '07},
        title = {{Web-scale Data Integration: You can only afford to Pay As You Go}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Madhavan, Jayant and Jeffery, Shawn R. and Cohen, Shirley and Dong, Xin (Luna) and Ko, David and Yu, Cong and Halevy, Alon},
        year = {2007}
}

Incoming Citations (Sorted by Pagerank)

Showing 28 of 28 citing papers.

Rank Citing Paper Year Venue Pagerank
319 Making Database Systems Usable 2007 SIGMOD 0.00021164857
579 Incremental Knowledge Base Construction Using DeepDive 2015 VLDB 0.00016086569
587 Management of Probabilistic Data: Foundations and Challenges 2007 PODS 0.00015933201
888 Pay-as-you-go User Feedback for Dataspace Systems 2008 SIGMOD 0.00013255989
1,582 Google’s Deep-Web Crawl 2008 VLDB 0.00010173382
2,477 A Relational Approach to Incrementally Extracting and Querying Structure in Unstructured Data 2007 VLDB 8.4079432e-05
2,888 DB&IR: Both Sides Now (Extended Abstract) 2007 SIGMOD 7.9072366e-05
3,250 How Large Language Models Will Disrupt Data Management 2023 VLDB 7.4938546e-05
3,738 BLAST: a Loosely Schema-aware Meta-blocking Approach for Entity Resolution 2016 VLDB 7.0627813e-05
4,518 Arnold: Declarative Crowd-Machine Data Integration 2013 CIDR 6.5632796e-05
4,549 Harnessing the Deep Web: Present and Future 2009 CIDR 6.5410269e-05
5,128 iTrails: Pay-as-you-go Information Integration in Dataspaces 2007 VLDB 6.260847e-05
5,143 Schema-agnostic vs Schema-based Configurations for Blocking Methods on Homogeneous Data 2016 VLDB 6.2569019e-05
5,498 HAMSTER: Using Search Clicklogs for Schema and Taxonomy Matching 2009 VLDB 6.1035636e-05
5,678 A Hierarchical Approach to Model Web Query Interfaces for Web Source Integration 2009 VLDB 6.0409774e-05
7,260 Entity Resolution On-Demand 2022 VLDB 5.571581e-05
8,059 A Grammar-based Entity Representation Framework for Data Cleaning 2009 SIGMOD 5.3965759e-05
8,182 SXPath - Extending XPath towards Spatial Querying on Web Documents 2011 VLDB 5.3819907e-05
8,920 Search Driven Analysis of Heterogeneous XML Data 2009 CIDR 5.2559789e-05
8,964 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.2482562e-05
9,170 The Role of Schema Matching in Large Enterprises 2009 CIDR 5.2141009e-05
9,199 Learning to Extract Form Labels 2008 VLDB 5.2087995e-05
10,317 Stop Word and Related Problems in Web Interface Integration 2009 VLDB 5.0372479e-05
12,601 Mob Data Sourcing 2012 SIGMOD 4.9793485e-05
12,670 Pay-As-You-Go Mapping Selection in Dataspaces 2011 SIGMOD 4.9793485e-05
12,709 Schema Clustering and Retrieval for Multi-domain Pay-As-You-Go Data Integration Systems 2010 SIGMOD 4.9793485e-05
12,726 Creating and Exploring Web Form Repositories 2010 SIGMOD 4.9793485e-05
12,804 Vispedia: On-demand Data Integration for Interactive Visualization and Exploration 2009 SIGMOD 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
24 Data Integration: A Theoretical Perspective 2002 PODS 0.00055043581
190 Applying Model Management to Classical Meta Data Problems 2003 CIDR 0.00025768763
230 Reference Reconciliation in Complex Information Spaces 2005 SIGMOD 0.00023871933
269 Reconciling Schemas of Disparate Data Sources: A Machine-Learning Approach 2001 SIGMOD 0.00022620777
348 Crawling the Hidden Web 2001 VLDB 0.00020262145
474 Automated Ranking of Database Query Results 2003 CIDR 0.00017672562
814 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013719268
1,114 Principles of Dataspace Systems 2006 PODS 0.00011964697
1,567 Crossing the Structure Chasm 2003 CIDR 0.00010213092
2,273 A Platform for Personal Information Management and Integration 2005 CIDR 8.7118476e-05
2,462 DBLife: A Community Information Management Platform for the Database Research Community 2007 CIDR 8.4317833e-05
2,661 WISE-Integrator: An Automatic Integrator of Web Search Interfaces for E-Commerce 2003 VLDB 8.1602863e-05
3,759 Toward Large Scale Integration: Building a MetaQuerier over Databases on the Web 2005 CIDR 7.0430209e-05
6,659 A Dataspace Odyssey: The iMeMex Personal Dataspace Management System 2007 CIDR 5.7179562e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 13,335 Data Integration in the Large: The Challenge of Reuse 1994 VLDB
2 13,094 Architectural Issues and Solutions in the Development of Data-Intensive Web Applications 2003 CIDR
3 2,158 Open Data Integration 2018 VLDB
4 814 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD
5 6,697 Web Data Management 2011 SIGMOD
6 2,120 An Analysis of Structured Data on the Web 2012 VLDB
7 12,657 Schema-As-You-Go: On Probabilistic Tagging and Querying of Wide Tables 2011 SIGMOD
8 3,759 Toward Large Scale Integration: Building a MetaQuerier over Databases on the Web 2005 CIDR
9 12,709 Schema Clustering and Retrieval for Multi-domain Pay-As-You-Go Data Integration Systems 2010 SIGMOD
10 1,667 Bootstrapping Pay-As-You-Go Data Integration Systems 2008 SIGMOD