DBScholar

Back to papers

Web-scale Data Integration: You can only afford to Pay As You Go

Summary: Argues that traditional tightly-coupled integration fails at web scale (Deep Web, Google Base) due to extreme heterogeneity and scale. Proposes PAYGO, a dataspaces-inspired pay-as-you-go architecture that delivers incremental, best-effort, cost-aware integration to maximize utility under limited resources. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
94
Venue
CIDR
Year
2007
Pagerank
0.00012892148
Overall Rank
969 | 93.36%
DOI
-

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{madhavan_cidr07,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '07},
        title = {{Web-scale Data Integration: You can only afford to Pay As You Go}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Madhavan, Jayant and Jeffery, Shawn R. and Cohen, Shirley and Dong, Xin (Luna) and Ko, David and Yu, Cong and Halevy, Alon},
        year = {2007}
}

Incoming Citations (Sorted by Pagerank)

Showing 28 of 28 citing papers.

Rank Citing Paper Year Venue Pagerank
314 Making Database Systems Usable 2007 SIGMOD 0.00021574636
574 Management of Probabilistic Data: Foundations and Challenges 2007 PODS 0.00016278855
579 Incremental Knowledge Base Construction Using DeepDive 2015 VLDB 0.00016217563
866 Pay-as-you-go User Feedback for Dataspace Systems 2008 SIGMOD 0.00013519288
1,554 Google’s Deep-Web Crawl 2008 VLDB 0.00010372108
2,724 A Relational Approach to Incrementally Extracting and Querying Structure in Unstructured Data 2007 VLDB 8.2048095e-05
2,877 DB&IR: Both Sides Now (Extended Abstract) 2007 SIGMOD 8.0070039e-05
3,536 How Large Language Models Will Disrupt Data Management 2023 VLDB 7.3297343e-05
3,665 BLAST: a Loosely Schema-aware Meta-blocking Approach for Entity Resolution 2016 VLDB 7.2142217e-05
4,428 Arnold: Declarative Crowd-Machine Data Integration 2013 CIDR 6.7094035e-05
4,471 Harnessing the Deep Web: Present and Future 2009 CIDR 6.6813981e-05
5,006 iTrails: Pay-as-you-go Information Integration in Dataspaces 2007 VLDB 6.4034229e-05
5,018 Schema-agnostic vs Schema-based Configurations for Blocking Methods on Homogeneous Data 2016 VLDB 6.3993079e-05
5,492 HAMSTER: Using Search Clicklogs for Schema and Taxonomy Matching 2009 VLDB 6.2001088e-05
5,545 A Hierarchical Approach to Model Web Query Interfaces for Web Source Integration 2009 VLDB 6.1795204e-05
7,743 Entity Resolution On-Demand 2022 VLDB 5.5545622e-05
7,894 A Grammar-based Entity Representation Framework for Data Cleaning 2009 SIGMOD 5.5204248e-05
8,022 SXPath - Extending XPath towards Spatial Querying on Web Documents 2011 VLDB 5.5055198e-05
8,757 Search Driven Analysis of Heterogeneous XML Data 2009 CIDR 5.3766157e-05
8,801 Effective Entity Augmentation By Querying External Data Sources 2023 VLDB 5.3687048e-05
9,014 The Role of Schema Matching in Large Enterprises 2009 CIDR 5.3321017e-05
9,031 Learning to Extract Form Labels 2008 VLDB 5.3283239e-05
10,095 Stop Word and Related Problems in Web Interface Integration 2009 VLDB 5.1528643e-05
12,310 Mob Data Sourcing 2012 SIGMOD 5.093636e-05
12,379 Pay-As-You-Go Mapping Selection in Dataspaces 2011 SIGMOD 5.093636e-05
12,418 Schema Clustering and Retrieval for Multi-domain Pay-As-You-Go Data Integration Systems 2010 SIGMOD 5.093636e-05
12,435 Creating and Exploring Web Form Repositories 2010 SIGMOD 5.093636e-05
12,514 Vispedia: On-demand Data Integration for Interactive Visualization and Exploration 2009 SIGMOD 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
22 Data Integration: A Theoretical Perspective 2002 PODS 0.00056204792
183 Applying Model Management to Classical Meta Data Problems 2003 CIDR 0.00026283359
228 Reference Reconciliation in Complex Information Spaces 2005 SIGMOD 0.00023941271
263 Reconciling Schemas of Disparate Data Sources: A Machine-Learning Approach 2001 SIGMOD 0.0002305693
340 Crawling the Hidden Web 2001 VLDB 0.00020711119
466 Automated Ranking of Database Query Results 2003 CIDR 0.00018014467
791 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD 0.00013995705
1,162 Principles of Dataspace Systems 2006 PODS 0.00011866011
1,540 Crossing the Structure Chasm 2003 CIDR 0.00010430413
2,236 A Platform for Personal Information Management and Integration 2005 CIDR 8.8902064e-05
2,405 DBLife: A Community Information Management Platform for the Database Research Community 2007 CIDR 8.6217909e-05
2,621 WISE-Integrator: An Automatic Integrator of Web Search Interfaces for E-Commerce 2003 VLDB 8.3342664e-05
3,690 Toward Large Scale Integration: Building a MetaQuerier over Databases on the Web 2005 CIDR 7.2007927e-05
6,541 A Dataspace Odyssey: The iMeMex Personal Dataspace Management System 2007 CIDR 5.8471331e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 13,045 Data Integration in the Large: The Challenge of Reuse 1994 VLDB
2 12,804 Architectural Issues and Solutions in the Development of Data-Intensive Web Applications 2003 CIDR
3 2,216 Open Data Integration 2018 VLDB
4 791 An Interactive Clustering-based Approach to Integrating Source Query Interfaces on the Deep Web 2004 SIGMOD
5 6,573 Web Data Management 2011 SIGMOD
6 2,096 An Analysis of Structured Data on the Web 2012 VLDB
7 12,366 Schema-As-You-Go: On Probabilistic Tagging and Querying of Wide Tables 2011 SIGMOD
8 3,690 Toward Large Scale Integration: Building a MetaQuerier over Databases on the Web 2005 CIDR
9 12,418 Schema Clustering and Retrieval for Multi-domain Pay-As-You-Go Data Integration Systems 2010 SIGMOD
10 1,649 Bootstrapping Pay-As-You-Go Data Integration Systems 2008 SIGMOD