Dedoop: Efficient Deduplication with Hadoop

Summary: Dedoop offers browser-based composition of MapReduce entity-resolution workflows, including blocking, matching, and learned classifiers. It compiles them to Hadoop jobs with load balancing and redundant-comparison avoidance for scalable, observable deduplication. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 10613
Venue: VLDB
Year: 2012
Pagerank: 9.2512023e-05
Overall Rank: 2,055 | 85.91%
DOI: 10.14778/2367502.2367523

Incoming Non-self Citations Over Time

Authors

1. Lars Kolb (University of Leipzig)
2. Andreas Thor (University of Leipzig)
3. Erhard Rahm (University of Leipzig)

BibTeX Citation

@article{kolb_vldb12,
        title = {{Dedoop: Efficient Deduplication with Hadoop}},
        author = {Kolb, Lars and Thor, Andreas and Rahm, Erhard},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {12},
        pages = {1878--1881},
        doi = {10.14778/2367502.2367523},
        url = {https://doi.org/10.14778/2367502.2367523},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 14 of 14 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
529	Magellan: Toward Building Entity Matching Management Systems	2016	VLDB	0.00017096361
1,323	Data Cleaning: Overview and Emerging Challenges	2016	SIGMOD	0.00011152602
2,398	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	8.631172e-05
2,893	Distributed Data Deduplication	2016	VLDB	7.983961e-05
3,251	Keys for Graphs	2015	VLDB	7.5963955e-05
3,611	NADEEF/ER: Generic and Interactive Entity Resolution	2014	SIGMOD	7.2580469e-05
6,392	Qualitative Data Cleaning	2016	VLDB	5.8883971e-05
6,403	Parallel Discrepancy Detection and Incremental Detection	2021	VLDB	5.8859374e-05
6,708	Serving Deep Learning Models with Deduplication from Relational Databases	2022	VLDB	5.7964983e-05
7,179	CleanM: An Optimizable Query Language for Unified Scale-Out Data Cleaning	2017	VLDB	5.6802479e-05
7,204	WOO: A Scalable and Multi-tenant Platform for Continuous Knowledge Base Synthesis	2013	VLDB	5.6739087e-05
8,908	Privacy and Accuracy-Aware AI/ML Model Deduplication	2025	SIGMOD	5.3483178e-05
9,627	Making It Tractable to Catch Duplicates and Conflicts in Graphs	2023	SIGMOD	5.2434488e-05
9,997	HyperBlocker: Accelerating Rule-based Blocking in Entity Resolution using GPUs	2025	VLDB	5.1814573e-05

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
248	Evaluation of entity resolution approaches on real-world match problems	2010	VLDB	0.00023278354

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	9,510	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB
2	9,263	MapReduce Algorithms for Big Data Analysis	2012	VLDB
3	6,147	HadoopDB in Action: Building Real World Applications	2010	SIGMOD
4	3,671	Exploiting MapReduce-based Similarity Joins	2012	SIGMOD
5	5,021	Online Deduplication for Databases	2017	SIGMOD
6	5,391	MDedup: Duplicate Detection with Matching Dependencies	2020	VLDB
7	660	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB
8	2,893	Distributed Data Deduplication	2016	VLDB
9	13,707	MapDupReducer: Detecting Near Duplicates over Massive Datasets	2010	SIGMOD
10	9,369	Redoop Infrastructure for Recurring Big Data Queries	2014	VLDB