Multi-Query Optimization in MapReduce Framework

Summary: MapReduce multi-job optimization: grouping merges jobs for shared scans and map outputs; partial-materialization shares map outputs. An algorithm partitions the batch and assigns per-group techniques; Hadoop experiments show large gains over MRShare. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 10805
Venue: VLDB
Year: 2014
Pagerank: 6.8289978e-05
Overall Rank: 3,703 | 74.25%
DOI: -

Incoming Non-self Citations Over Time

Authors

1. Guoping Wang
2. Chee-Yong Chan

Incoming Citations (Sorted by Pagerank)

Showing 13 of 13 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,922	Selecting Subexpressions to Materialize at Datacenter Scale	2018	VLDB	0.00010082599
2,576	S4: Top-k Spreadsheet-Style Search for Query Discovery	2015	SIGMOD	8.5112408e-05
3,129	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.5008242e-05
3,550	Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems	2018	VLDB	6.9843512e-05
4,174	Computation Reuse in Analytics Job Service at Microsoft	2018	SIGMOD	6.3856219e-05
6,469	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD	5.0519488e-05
6,777	Revisiting Reuse in Main Memory Database Systems	2017	SIGMOD	4.9288776e-05
7,351	Distributed Outlier Detection using Compressive Sensing	2015	SIGMOD	4.7545562e-05
8,859	Pipemizer: An Optimizer for Analytics Data Pipelines	2022	VLDB	4.4344107e-05
10,850	Mayura: Exploiting Similarities in Motifs for Temporal Co-Mining	2025	VLDB	4.1945683e-05
11,506	Robust Voice Querying with MUVE: Optimally Visualizing Results of Phonetically Similar Queries	2021	VLDB	4.1945683e-05
11,882	Parallel Evaluation of Multi-Semi-Joins	2016	VLDB	4.1945683e-05
11,958	Shared Execution of Recurring Workloads in MapReduce	2015	VLDB	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 10 of 10 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
3	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0024183614
70	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00059533166
780	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00016775082
868	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00015789681
947	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00015114576
1,280	Automatic Optimization for MapReduce Programs	2011	VLDB	0.0001285503
2,205	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.2920002e-05
2,747	Stubby: A Transformation-based Optimizer for MapReduce Workflows	2012	VLDB	8.1828918e-05
5,903	Building Wavelet Histograms on Large Data in MapReduce	2012	VLDB	5.2791351e-05
8,358	MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish	2011	VLDB	4.5372998e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
960	A Comparison of Join Algorithms for Log Processing in MapReduce	2010	SIGMOD	0.00015012242
1,280	Automatic Optimization for MapReduce Programs	2011	VLDB	0.0001285503
42	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00073498298
7,294	Optimization for iterative queries on MapReduce	2014	VLDB	4.773119e-05
868	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00015789681
1,615	The Performance of MapReduce: An In-depth Study	2010	VLDB	0.00011132319
15	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD	0.0010654262
3,062	Efficient Multi-way Theta-Join Processing Using MapReduce	2012	VLDB	7.6343994e-05
2,674	Minimal MapReduce Algorithms	2013	SIGMOD	8.3328645e-05
947	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00015114576