Industry-Scale Duplicate Detection

Summary: Extends the DogmatiX duplicate-detection prototype from hierarchical XML to a 60M-person industrial credit database. Addresses both matching accuracy and scalability, with evaluation on real-world data where errors affect credit access and identity fraud. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h5732a22a11e9a756
Venue: VLDB
Year: 2008
Pagerank: 5.9508297e-05
Overall Rank: 5,859 | 60.73%
DOI: 10.14778/1454159.1454165

Incoming Non-self Citations Over Time

Authors

1. Melanie Weis (Hasso Plattner Institute)
2. Felix Naumann (Hasso Plattner Institute)
3. Ulrich Jehle (SCHUFA Holding AG)
4. Jens Lufter (SCHUFA Holding AG)
5. Holger Schuster (SCHUFA Holding AG)

BibTeX Citation

@article{weis_vldb08,
        title = {{Industry-Scale Duplicate Detection}},
        author = {Weis, Melanie and Naumann, Felix and Jehle, Ulrich and Lufter, Jens and Schuster, Holger},
        journal = {PVLDB},
        series = {{VLDB} '08},
        pages = {1253},
        doi = {10.14778/1454159.1454165},
        url = {https://doi.org/10.14778/1454159.1454165},
        year = {2008}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
619	Reasoning about Record Matching Rules	2009	VLDB	0.00015503747
7,884	Learning Over Dirty Data Without Cleaning	2020	SIGMOD	5.4169982e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
60	The Merge/Purge Problem for Large Databases	1995	SIGMOD	0.00039343755
308	Eliminating Fuzzy Duplicates in Data Warehouses	2002	VLDB	0.0002143466
1,582	Example-driven Design of Efficient Record Matching Queries	2007	VLDB	0.0001015019
2,558	DogmatiX Tracks down Duplicates in XML	2005	SIGMOD	8.2766441e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,074	Crowd-Based Deduplication: An Adaptive Approach	2015	SIGMOD
2	6,539	Parallel Discrepancy Detection and Incremental Detection	2021	VLDB
3	2,469	Leveraging Aggregate Constraints For Deduplication	2007	SIGMOD
4	2,954	Distributed Data Deduplication	2016	VLDB
5	6,929	Efficient Discovery of XML Data Redundancies	2006	VLDB
6	5,544	MDedup: Duplicate Detection with Matching Dependencies	2020	VLDB
7	886	Framework for Evaluating Clustering Algorithms in Duplicate Detection	2009	VLDB
8	308	Eliminating Fuzzy Duplicates in Data Warehouses	2002	VLDB
9	3,437	Modeling and Querying Possible Repairs in Duplicate Detection	2009	VLDB
10	2,558	DogmatiX Tracks down Duplicates in XML	2005	SIGMOD