SpongeFiles: Mitigating Data Skew in MapReduce Using Distributed Memory
Summary: Introduces SpongeFiles, a distributed-memory abstraction for MapReduce that stores spilled data as a logical byte array across memory and disk, reducing data skew. Nearest-capacity routing lets idle nodes tap neighbor memory/disk, yielding up to 55% overall speedup (85% under contention) on Hadoop/Pig. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
Incoming Citations (Sorted by Pagerank)
Showing 1 of 1 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 6,856 | Liquid: Unifying Nearline and Offline Big Data Integration | 2015 | CIDR | 4.901516e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 9 of 9 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 3 | Pig Latin: A Not-So-Foreign Language for Data Processing | 2008 | SIGMOD | 0.0024217964 |
| 54 | PNUTS: Yahoo!'s Hosted Data Serving Platform | 2008 | VLDB | 0.0006582344 |
| 548 | Practical Skew Handling in Parallel Joins | 1992 | VLDB | 0.00020369531 |
| 679 | Skew-Aware Automatic Database Partitioning in Shared-Nothing, Parallel OLTP Systems | 2012 | SIGMOD | 0.00018211621 |
| 776 | Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience | 2009 | VLDB | 0.00016765694 |
| 852 | A Taxonomy and Performance Model of Data Skew Effects in Parallel Joins | 1991 | VLDB | 0.00015896035 |
| 1,339 | SkewTune: Mitigating Skew in MapReduce Applications | 2012 | SIGMOD | 0.00012492978 |
| 1,355 | Handling Data Skew in Multiprocessor Database Computers Using Partition Tuning | 1991 | VLDB | 0.00012417228 |
| 8,460 | Piranha: Optimizing Short Jobs in Hadoop | 2013 | VLDB | 4.5008938e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| Overall Rank | Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 5,587 | Magnet: Push-based Shuffle Service for Large-scale Data Processing | 2020 | VLDB | 5.4193445e-05 |
| 3,193 | Column-Oriented Storage Techniques for MapReduce | 2011 | VLDB | 7.4073719e-05 |
| 4,068 | Advanced Partitioning Techniques for Massively Distributed Computation | 2012 | SIGMOD | 6.4748133e-05 |
| 7,510 | Hone: "Scaling Down" Hadoop on Shared-Memory Systems | 2013 | VLDB | 4.7135369e-05 |
| 2,211 | ReStore: Reusing Results of MapReduce Jobs | 2012 | VLDB | 9.2823037e-05 |
| 12,148 | SkewTune in Action: Mitigating Skew in MapReduce Applications | 2012 | VLDB | 4.1905499e-05 |
| 2,476 | A Platform for Scalable One-Pass Analytics using MapReduce | 2011 | SIGMOD | 8.6907971e-05 |
| 11,941 | FP-Hadoop: Efficient Execution of Parallel Jobs Over Skewed Data | 2015 | VLDB | 4.1905499e-05 |
| 1,339 | SkewTune: Mitigating Skew in MapReduce Applications | 2012 | SIGMOD | 0.00012492978 |
| 11,843 | An Efficient MapReduce Cube Algorithm for Varied Data Distributions | 2016 | SIGMOD | 4.1905499e-05 |