Industry-Scale Duplicate Detection

Summary: Extends the DogmatiX duplicate-detection prototype from hierarchical XML to a 60M-person industrial credit database. Addresses both matching accuracy and scalability, with evaluation on real-world data where errors affect credit access and identity fraud. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 9942
Venue: VLDB
Year: 2008
Pagerank: 6.0977946e-05
Overall Rank: 5,754 | 60.53%
DOI: 10.14778/1454159.1454165

Incoming Non-self Citations Over Time

Authors

1. Melanie Weis (Hasso Plattner Institute)
2. Felix Naumann (Hasso Plattner Institute)
3. Ulrich Jehle (SCHUFA Holding AG)
4. Jens Lufter (SCHUFA Holding AG)
5. Holger Schuster (SCHUFA Holding AG)

BibTeX Citation

@article{weis_vldb08,
        title = {{Industry-Scale Duplicate Detection}},
        author = {Weis, Melanie and Naumann, Felix and Jehle, Ulrich and Lufter, Jens and Schuster, Holger},
        journal = {PVLDB},
        series = {{VLDB} '08},
        pages = {1253},
        doi = {10.14778/1454159.1454165},
        url = {https://doi.org/10.14778/1454159.1454165},
        year = {2008}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
619	Reasoning about Record Matching Rules	2009	VLDB	0.00015707247
7,880	Learning Over Dirty Data Without Cleaning	2020	SIGMOD	5.5244204e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
58	The Merge/Purge Problem for Large Databases	1995	SIGMOD	0.00040116748
306	Eliminating Fuzzy Duplicates in Data Warehouses	2002	VLDB	0.00021839661
1,560	Example-driven Design of Efficient Record Matching Queries	2007	VLDB	0.00010361222
2,531	DogmatiX Tracks down Duplicates in XML	2005	SIGMOD	8.4574631e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	3,983	Crowd-Based Deduplication: An Adaptive Approach	2015	SIGMOD
2	6,403	Parallel Discrepancy Detection and Incremental Detection	2021	VLDB
3	2,410	Leveraging Aggregate Constraints For Deduplication	2007	SIGMOD
4	2,893	Distributed Data Deduplication	2016	VLDB
5	6,771	Efficient Discovery of XML Data Redundancies	2006	VLDB
6	5,391	MDedup: Duplicate Detection with Matching Dependencies	2020	VLDB
7	871	Framework for Evaluating Clustering Algorithms in Duplicate Detection	2009	VLDB
8	306	Eliminating Fuzzy Duplicates in Data Warehouses	2002	VLDB
9	3,371	Modeling and Querying Possible Repairs in Duplicate Detection	2009	VLDB
10	2,531	DogmatiX Tracks down Duplicates in XML	2005	SIGMOD