SkewTune: Mitigating Skew in MapReduce Applications

Summary: SkewTune automatically mitigates MapReduce skew without extra user input, as a drop-in Hadoop extension. It uses idle-node detection to repartition a straggler's unprocessed data, preserves input order for concatenation-based output reconstruction, and incurs minimal overhead when skew is absent. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h6bd1b7236e7dc74f
Venue: SIGMOD
Year: 2012
Pagerank: 0.00010893993
Overall Rank: 1,360 | 90.89%
DOI: 10.1145/2213836.2213840

Incoming Non-self Citations Over Time

Authors

1. YongChul Kwon (University of Washington)
2. Magdalena Balazinska (University of Washington)
3. Bill Howe (University of Washington)
4. Jerome Rolia (Hewlett Packard Enterprise)

BibTeX Citation

@inproceedings{kwon_sigmod12,
        title = {{SkewTune: Mitigating Skew in MapReduce Applications}},
        author = {Kwon, YongChul and Balazinska, Magdalena and Howe, Bill and Rolia, Jerome},
        series = {{SIGMOD} '12},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2213836.2213840},
        url = {https://dl.acm.org/doi/10.1145/2213836.2213840},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 21 of 21 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
436	Shark: SQL and Rich Analytics at Scale	2013	SIGMOD	0.00018274105
1,547	Upper and Lower Bounds on the Cost of a Map-Reduce Computation	2013	VLDB	0.00010268377
2,561	Minimal MapReduce Algorithms	2013	SIGMOD	8.2743492e-05
3,701	A General and Parallel Platform for Mining Co-Movement Patterns over Large-scale Trajectories	2017	VLDB	7.0650979e-05
4,532	LocationSpark: A Distributed In-Memory Data Management System for Big Spatial Data	2016	VLDB	6.5412636e-05
5,236	A Padded Encoding Scheme to Accelerate Scans by Leveraging Skew	2015	SIGMOD	6.2037372e-05
5,756	Scalable Progressive Analytics on Big Data in the Cloud	2013	VLDB	5.987569e-05
5,842	Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture	2013	SIGMOD	5.9568426e-05
6,808	Hadoop's Adolescence: An analysis of Hadoop usage in scientific workloads	2013	VLDB	5.6608074e-05
6,880	SquirrelJoin: Network-Aware Distributed Join Processing with Lazy Partitioning	2017	VLDB	5.6423904e-05
7,284	Submodularity of Distributed Join Computation	2018	SIGMOD	5.5505534e-05
7,653	MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs	2014	VLDB	5.4592529e-05
8,852	Toward Progress Indicators on Steroids for Big Data Systems	2013	CIDR	5.2488117e-05
9,048	The Power of Nested Parallelism in Big Data Processing – Hitting Three Flies with One Slap –	2021	SIGMOD	5.2155403e-05
9,232	Dalton: Learned Partitioning for Distributed Data Streams	2023	VLDB	5.1888006e-05
9,341	SpongeFiles: Mitigating Data Skew in MapReduce Using Distributed Memory	2014	SIGMOD	5.1779529e-05
12,081	Fangorn: Adaptive Execution Framework for Heterogeneous Workloads on Shared Clusters	2021	VLDB	4.9605343e-05
12,238	An Experimental Evaluation of Garbage Collectors on Big Data Applications	2019	VLDB	4.9605343e-05
12,473	FP-Hadoop: Efficient Execution of Parallel Jobs Over Skewed Data	2015	VLDB	4.9605343e-05
12,488	Big Data Research: Will Industry Solve all the Problems?	2015	VLDB	4.9605343e-05
12,676	SkewTune in Action: Mitigating Skew in MapReduce Applications	2012	VLDB	4.9605343e-05

Outgoing Citations (Sorted by Pagerank)

Showing 16 of 16 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
9	Online Aggregation	1997	SIGMOD	0.00076079812
385	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00019419661
491	Practical Skew Handling in Parallel Joins	1992	VLDB	0.00017383221
655	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00015076669
677	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00014835154
836	A Taxonomy and Performance Model of Data Skew Effects in Parallel Joins	1991	VLDB	0.00013534835
944	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR	0.00012927479
982	Parallel Evaluation of Conjunctive Queries	2011	PODS	0.00012687016
1,236	Highly Available, Fault-Tolerant, Parallel Dataflows	2004	SIGMOD	0.00011382214
1,633	Adaptive Parallel Aggregation Algorithms	1995	SIGMOD	0.00010002415
2,299	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD	8.6639611e-05
2,684	A Latency and Fault-Tolerance Optimizer for Online Parallel Query Plans	2011	SIGMOD	8.1185673e-05
2,990	Clustera: An Integrated Computation And Data Management System	2008	VLDB	7.7609997e-05
3,977	Estimation of Query-Result Distribution and its Application in Parallel-Join Load Balancing	1996	VLDB	6.8707656e-05
5,562	Efficient outer join data skew handling in parallel DBMS	2009	VLDB	6.0663747e-05
12,676	SkewTune in Action: Mitigating Skew in MapReduce Applications	2012	VLDB	4.9605343e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,036	Skew Handling Techniques in Sort-Merge Join	2002	SIGMOD
2	7,653	MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs	2014	VLDB
3	1,483	Skew in Parallel Query Processing	2014	PODS
4	491	Practical Skew Handling in Parallel Joins	1992	VLDB
5	12,347	Runtime Optimization of Join Location in Parallel Data Management Systems	2017	VLDB
6	12,380	An Efficient MapReduce Cube Algorithm for Varied Data Distributions	2016	SIGMOD
7	9,341	SpongeFiles: Mitigating Data Skew in MapReduce Using Distributed Memory	2014	SIGMOD
8	1,259	Handling Data Skew in Multiprocessor Database Computers Using Partition Tuning	1991	VLDB
9	12,473	FP-Hadoop: Efficient Execution of Parallel Jobs Over Skewed Data	2015	VLDB
10	12,676	SkewTune in Action: Mitigating Skew in MapReduce Applications	2012	VLDB