Dedoop: Efficient Deduplication with Hadoop

Summary: Dedoop offers browser-based specification of complex ER workflows (blocking, similarity functions, ML-generated classifiers) for MapReduce-based deduplication on Hadoop. It auto-translates workflows into MapReduce jobs, visualizes results and workload, and uses blocking-aware load balancing to minimize comparisons and balance cluster usage. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 10426
Venue: VLDB
Year: 2012
Pagerank: 9.2962961e-05
Overall Rank: 2,205 | 84.68%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 14 of 14 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
705	Magellan: Toward Building Entity Matching Management Systems	2016	VLDB	0.00017779048
1,629	Data Cleaning: Overview and Emerging Challenges	2016	SIGMOD	0.00011073148
2,728	Keys for Graphs	2015	VLDB	8.2228869e-05
3,000	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	7.7447724e-05
3,526	Distributed Data Deduplication	2016	VLDB	7.0056559e-05
3,568	NADEEF/ER: Generic and Interactive Entity Resolution	2014	SIGMOD	6.9588883e-05
6,103	WOO: A Scalable and Multi-tenant Platform for Continuous Knowledge Base Synthesis	2013	VLDB	5.2054474e-05
6,691	Parallel Discrepancy Detection and Incremental Detection	2021	VLDB	4.9573939e-05
7,013	Qualitative Data Cleaning	2016	VLDB	4.8576683e-05
7,055	Serving Deep Learning Models with Deduplication from Relational Databases	2022	VLDB	4.8428976e-05
7,233	CleanM: An Optimizable Query Language for Unified Scale-Out Data Cleaning	2017	VLDB	4.788267e-05
9,489	Making It Tractable to Catch Duplicates and Conflicts in Graphs	2023	SIGMOD	4.3300131e-05
9,846	HyperBlocker: Accelerating Rule-based Blocking in Entity Resolution using GPUs	2025	VLDB	4.2680295e-05
10,508	Privacy and Accuracy-Aware AI/ML Model Deduplication	2025	SIGMOD	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
318	Evaluation of entity resolution approaches on real-world match problems	2010	VLDB	0.00027850417

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
9,360	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB	4.3476444e-05
9,113	MapReduce Algorithms for Big Data Analysis	2012	VLDB	4.3889284e-05
5,850	HadoopDB in Action: Building Real World Applications	2010	SIGMOD	5.3008257e-05
4,130	Exploiting MapReduce-based Similarity Joins	2012	SIGMOD	6.4205218e-05
5,240	Online Deduplication for Databases	2017	SIGMOD	5.6060424e-05
6,047	MDedup: Duplicate Detection with Matching Dependencies	2020	VLDB	5.2355891e-05
789	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00016602215
3,526	Distributed Data Deduplication	2016	VLDB	7.0056559e-05
13,521	MapDupReducer: Detecting Near Duplicates over Massive Datasets	2010	SIGMOD	-
9,270	Redoop Infrastructure for Recurring Big Data Queries	2014	VLDB	4.362604e-05