Multi-Query Optimization in MapReduce Framework

Summary: MapReduce multi-job optimization: grouping merges jobs for shared scans and map outputs; partial-materialization shares map outputs. An algorithm partitions the batch and assigns per-group techniques; Hadoop experiments show large gains over MRShare. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 10806
Venue: VLDB
Year: 2014
Pagerank: 6.8211506e-05
Overall Rank: 3,709 | 74.23%
DOI: -

Incoming Non-self Citations Over Time

Authors

1. Guoping Wang
2. Chee-Yong Chan

Incoming Citations (Sorted by Pagerank)

Showing 13 of 13 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,921	Selecting Subexpressions to Materialize at Datacenter Scale	2018	VLDB	0.00010085899
2,577	S4: Top-k Spreadsheet-Style Search for Query Discovery	2015	SIGMOD	8.5066436e-05
3,188	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.4142307e-05
3,551	Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems	2018	VLDB	6.9776252e-05
4,171	Computation Reuse in Analytics Job Service at Microsoft	2018	SIGMOD	6.3800823e-05
6,464	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD	5.0471003e-05
6,777	Revisiting Reuse in Main Memory Database Systems	2017	SIGMOD	4.9242249e-05
7,347	Distributed Outlier Detection using Compressive Sensing	2015	SIGMOD	4.7502395e-05
8,859	Pipemizer: An Optimizer for Analytics Data Pipelines	2022	VLDB	4.4301601e-05
10,854	Mayura: Exploiting Similarities in Motifs for Temporal Co-Mining	2025	VLDB	4.1905499e-05
11,510	Robust Voice Querying with MUVE: Optimally Visualizing Results of Phonetically Similar Queries	2021	VLDB	4.1905499e-05
11,890	Parallel Evaluation of Multi-Semi-Joins	2016	VLDB	4.1905499e-05
11,966	Shared Execution of Recurring Workloads in MapReduce	2015	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 10 of 10 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
3	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0024217964
70	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00059744625
776	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00016765694
866	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00015771189
947	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00015112344
1,281	Automatic Optimization for MapReduce Programs	2011	VLDB	0.00012837411
2,211	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.2823037e-05
2,754	Stubby: A Transformation-based Optimizer for MapReduce Workflows	2012	VLDB	8.1720428e-05
5,908	Building Wavelet Histograms on Large Data in MapReduce	2012	VLDB	5.2731311e-05
8,355	MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish	2011	VLDB	4.5328622e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
962	A Comparison of Join Algorithms for Log Processing in MapReduce	2010	SIGMOD	0.00015003834
1,281	Automatic Optimization for MapReduce Programs	2011	VLDB	0.00012837411
42	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00073570328
7,293	Optimization for iterative queries on MapReduce	2014	VLDB	4.7668182e-05
866	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00015771189
1,615	The Performance of MapReduce: An In-depth Study	2010	VLDB	0.00011137225
15	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD	0.0010668335
3,068	Efficient Multi-way Theta-Join Processing Using MapReduce	2012	VLDB	7.6241861e-05
2,714	Minimal MapReduce Algorithms	2013	SIGMOD	8.2426646e-05
947	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00015112344