Database Paper Browser

Back to papers

Distributed Data Deduplication

Summary: Distributed data deduplication in a shared-nothing setting; leverages parallelism to prune pairwise comparisons after blocking. Dis-Dedup—a distribution strategy that minimizes the maximum per-node workload with theoretical guarantees; experiments on synthetic and real data show scalable speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
11374
Venue
VLDB
Year
2016
Pagerank
7.0066139e-05
Overall Rank
3,528 | 75.46%
DOI
-

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 15 of 15 citing papers.

Rank Citing Paper Year Venue Pagerank
712 Magellan: Toward Building Entity Matching Management Systems 2016 VLDB 0.00017732426
2,175 Falcon: Scaling Up Hands-Off Crowdsourced Entity Matching to Build Cloud Services 2017 SIGMOD 9.3644117e-05
5,434 Auto-FuzzyJoin: Auto-Program Fuzzy Similarity Joins Without Labeled Examples 2021 SIGMOD 5.5045402e-05
5,613 Distributed implementations of dependency discovery algorithms 2019 VLDB 5.4102298e-05
6,690 Parallel Discrepancy Detection and Incremental Detection 2021 VLDB 4.9621556e-05
7,013 Qualitative Data Cleaning 2016 VLDB 4.8619024e-05
7,061 Serving Deep Learning Models with Deduplication from Relational Databases 2022 VLDB 4.8463881e-05
7,153 Submodularity of Distributed Join Computation 2018 SIGMOD 4.8153963e-05
8,092 Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications 2023 SIGMOD 4.587921e-05
8,396 Optimizing Declarative Graph Queries at Large Scale 2019 SIGMOD 4.5276541e-05
8,422 Deducing Certain Fixes to Graphs 2019 VLDB 4.5167705e-05
9,487 Making It Tractable to Catch Duplicates and Conflicts in Graphs 2023 SIGMOD 4.3341665e-05
9,846 HyperBlocker: Accelerating Rule-based Blocking in Entity Resolution using GPUs 2025 VLDB 4.2721228e-05
10,499 Privacy and Accuracy-Aware AI/ML Model Deduplication 2025 SIGMOD 4.1945683e-05
11,438 New Algorithms for Monotone Classification 2021 PODS 4.1945683e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
67 The Merge/Purge Problem for Large Databases 1995 SIGMOD 0.00061348205
280 Eliminating Fuzzy Duplicates in Data Warehouses 2002 VLDB 0.00029113044
319 Evaluation of entity resolution approaches on real-world match problems 2010 VLDB 0.00027781866
447 Efficient Parallel Set-Similarity Joins Using MapReduce 2010 SIGMOD 0.00022900171
588 Practical Skew Handling in Parallel Joins 1992 VLDB 0.00019604754
814 Entity Resolution: Theory, Practice & Open Challenges 2012 VLDB 0.00016370594
1,074 Processing Theta-Joins using MapReduce* 2011 SIGMOD 0.00014260096
1,939 From Theory to Practice: Efficient Join Query Evaluation in a Parallel Database System 2015 SIGMOD 0.00010025655
2,212 Skew in Parallel Query Processing 2014 PODS 9.2771827e-05
2,231 Dedoop: Efficient Deduplication with Hadoop 2012 VLDB 9.2304499e-05
3,141 ClusterJoin: A Similarity Joins Framework using Map-Reduce 2014 VLDB 7.4829448e-05
3,377 Demonstration of the Myria Big Data Management Service 2014 SIGMOD 7.1624478e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
9,581 Sharing Aggregate Computation for Distributed Queries 2007 SIGMOD 4.3227214e-05
3,360 Modeling and Querying Possible Repairs in Duplicate Detection 2009 VLDB 7.1742067e-05
7,715 Query Centric Partitioning and Allocation for Partially Replicated Database Systems 2017 SIGMOD 4.6699261e-05
5,613 Distributed implementations of dependency discovery algorithms 2019 VLDB 5.4102298e-05
6,690 Parallel Discrepancy Detection and Incremental Detection 2021 VLDB 4.9621556e-05
4,619 Crowd-Based Deduplication: An Adaptive Approach 2015 SIGMOD 6.0444854e-05
1,957 On the Design and Scalability of Distributed Shared-Data Databases 2015 SIGMOD 9.9598319e-05
3,821 Locality-aware Partitioning in Parallel Database Systems 2015 SIGMOD 6.7281515e-05
2,618 Distributing A Database For Parallelism 1983 SIGMOD 8.4447319e-05
5,236 Online Deduplication for Databases 2017 SIGMOD 5.611324e-05