Automatic Optimization for MapReduce Programs

Summary: Manimal statically analyzes free-form MapReduce code to infer relational data semantics (e.g., selections/projections) without programmer annotations. It automatically applies database-style optimizations, achieving up to 1,121% speedups on existing jobs. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h17bb5a85de59b43b
Venue: VLDB
Year: 2011
Pagerank: 0.00011213384
Overall Rank: 1,281 | 91.39%
DOI: 10.14778/2020.2021

Incoming Non-self Citations Over Time

Authors

1. Eaman Jahani (University of Michigan)
2. Michael J. Cafarella (University of Michigan)
3. Christopher Ré (University of Wisconsin)

BibTeX Citation

@article{jahani_vldb11,
        title = {{Automatic Optimization for MapReduce Programs}},
        author = {Jahani, Eaman and Cafarella, Michael J. and Ré, Christopher},
        journal = {PVLDB},
        series = {{VLDB} '11},
        volume = {4},
        number = {6},
        pages = {385--396},
        doi = {10.14778/2020.2021},
        url = {https://doi.org/10.14778/2020.2021},
        year = {2011}
}

Incoming Citations (Sorted by Pagerank)

Showing 19 of 19 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
753	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014237583
782	DBSherlock: A Performance Diagnostic Tool for Transactional Databases	2016	SIGMOD	0.00014022168
1,072	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00012168947
1,708	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.8224587e-05
2,195	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	8.8781177e-05
2,423	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	8.4877894e-05
2,527	Stubby: A Transformation-based Optimizer for MapReduce Workflows	2012	VLDB	8.3425785e-05
2,573	Minimal MapReduce Algorithms	2013	SIGMOD	8.2821647e-05
2,903	Column-Oriented Storage Techniques for MapReduce	2011	VLDB	7.8810111e-05
3,066	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.6877117e-05
3,216	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.5234702e-05
3,755	Parallel Algorithms for Constructing Range and Nearest-Neighbor Searching Data Structures	2016	PODS	7.0452836e-05
4,293	A Method for Optimizing Opaque Filter Queries	2020	SIGMOD	6.6819917e-05
5,274	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.2005706e-05
6,840	Towards Unified Ad-hoc Data Processing	2014	SIGMOD	5.6669233e-05
9,693	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB	5.1399537e-05
10,009	Streaming Space Complexity of Nearly All Functions of One Variable on Frequency Vectors	2016	PODS	5.0979044e-05
12,465	Anti-Combining for MapReduce	2014	SIGMOD	4.9793485e-05
12,554	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
2	R-Trees: A Dynamic Index Structure For Spatial Searching	1984	SIGMOD	0.001992968
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.001052036
12	C-Store: A Column-oriented DBMS	2005	VLDB	0.00068998927
27	Database Architecture Optimized for the New Bottleneck: Memory Access	1999	VLDB	0.0005158963
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00045546775
61	Integrating Compression and Execution in Column-Oriented Database Systems	2006	SIGMOD	0.000392237
75	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD	0.0003704106
120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.000311132
292	LINQ: Reconciling Objects, Relations and XML in the .NET Framework	2006	SIGMOD	0.00021947834
384	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.0001948031
673	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.0001488755

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,654	Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications	2018	SIGMOD
2	1,072	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
3	43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD
4	12,465	Anti-Combining for MapReduce	2014	SIGMOD
5	2,282	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
6	1,466	The Performance of MapReduce: An In-depth Study	2010	VLDB
7	75	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD
8	2,573	Minimal MapReduce Algorithms	2013	SIGMOD
9	3,216	Multi-Query Optimization in MapReduce Framework	2014	VLDB
10	753	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB