Automatic Optimization for MapReduce Programs

Summary: Manimal statically analyzes free-form MapReduce code to infer relational data semantics (e.g., selections/projections) without programmer annotations. It automatically applies database-style optimizations, achieving up to 1,121% speedups on existing jobs. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h17bb5a85de59b43b
Venue: VLDB
Year: 2011
Pagerank: 0.00011208192
Overall Rank: 1,282 | 91.39%
DOI: 10.14778/2020.2021

Incoming Non-self Citations Over Time

Authors

1. Eaman Jahani (University of Michigan)
2. Michael J. Cafarella (University of Michigan)
3. Christopher Ré (University of Wisconsin)

BibTeX Citation

@article{jahani_vldb11,
        title = {{Automatic Optimization for MapReduce Programs}},
        author = {Jahani, Eaman and Cafarella, Michael J. and Ré, Christopher},
        journal = {PVLDB},
        series = {{VLDB} '11},
        volume = {4},
        number = {6},
        pages = {385--396},
        doi = {10.14778/2020.2021},
        url = {https://doi.org/10.14778/2020.2021},
        year = {2011}
}

Incoming Citations (Sorted by Pagerank)

Showing 19 of 19 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
754	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014231311
783	DBSherlock: A Performance Diagnostic Tool for Transactional Databases	2016	SIGMOD	0.0001401708
1,073	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00012163258
1,708	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.8184303e-05
2,197	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	8.8740089e-05
2,424	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	8.483813e-05
2,527	Stubby: A Transformation-based Optimizer for MapReduce Workflows	2012	VLDB	8.3386546e-05
2,573	Minimal MapReduce Algorithms	2013	SIGMOD	8.2782871e-05
2,903	Column-Oriented Storage Techniques for MapReduce	2011	VLDB	7.877293e-05
3,068	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.6841028e-05
3,217	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.5199872e-05
3,757	Parallel Algorithms for Constructing Range and Nearest-Neighbor Searching Data Structures	2016	PODS	7.0419501e-05
4,292	A Method for Optimizing Opaque Filter Queries	2020	SIGMOD	6.6792739e-05
5,278	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.1976381e-05
6,845	Towards Unified Ad-hoc Data Processing	2014	SIGMOD	5.6642425e-05
9,699	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB	5.1375205e-05
10,014	Streaming Space Complexity of Nearly All Functions of One Variable on Frequency Vectors	2016	PODS	5.0954911e-05
12,471	Anti-Combining for MapReduce	2014	SIGMOD	4.9769913e-05
12,560	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	4.9769913e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
2	R-Trees: A Dynamic Index Structure For Spatial Searching	1984	SIGMOD	0.0019923528
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0010515896
12	C-Store: A Column-oriented DBMS	2005	VLDB	0.0006897844
27	Database Architecture Optimized for the New Bottleneck: Memory Access	1999	VLDB	0.00051576613
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.0004552807
61	Integrating Compression and Execution in Column-Oriented Database Systems	2006	SIGMOD	0.00039236924
75	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD	0.0003702496
120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.00031099083
292	LINQ: Reconciling Objects, Relations and XML in the .NET Framework	2006	SIGMOD	0.00021937893
384	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00019471648
675	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00014880686

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,073	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
2	4,656	Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications	2018	SIGMOD
3	12,471	Anti-Combining for MapReduce	2014	SIGMOD
4	43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD
5	2,285	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
6	1,466	The Performance of MapReduce: An In-depth Study	2010	VLDB
7	75	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD
8	2,573	Minimal MapReduce Algorithms	2013	SIGMOD
9	3,217	Multi-Query Optimization in MapReduce Framework	2014	VLDB
10	754	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB