Automatic Optimization for MapReduce Programs

Summary: Manimal statically analyzes free-form MapReduce code to infer relational data semantics (e.g., selections/projections) without programmer annotations. It automatically applies database-style optimizations, achieving up to 1,121% speedups on existing jobs. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h17bb5a85de59b43b
Venue: VLDB
Year: 2011
Pagerank: 0.00011187304
Overall Rank: 1,286 | 91.39%
DOI: 10.14778/2020.2021

Incoming Non-self Citations Over Time

Authors

1. Eaman Jahani (University of Michigan)
2. Michael J. Cafarella (University of Michigan)
3. Christopher Ré (University of Wisconsin)

BibTeX Citation

@article{jahani_vldb11,
        title = {{Automatic Optimization for MapReduce Programs}},
        author = {Jahani, Eaman and Cafarella, Michael J. and Ré, Christopher},
        journal = {PVLDB},
        series = {{VLDB} '11},
        volume = {4},
        number = {6},
        pages = {385--396},
        doi = {10.14778/2020.2021},
        url = {https://doi.org/10.14778/2020.2021},
        year = {2011}
}

Incoming Citations (Sorted by Pagerank)

Showing 19 of 19 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
757	DBSherlock: A Performance Diagnostic Tool for Transactional Databases	2016	SIGMOD	0.00014202187
759	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014194746
1,074	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00012131641
1,706	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.804657e-05
2,124	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	8.9915327e-05
2,429	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	8.4657925e-05
2,532	Stubby: A Transformation-based Optimizer for MapReduce Workflows	2012	VLDB	8.3132509e-05
2,561	Minimal MapReduce Algorithms	2013	SIGMOD	8.2743492e-05
2,918	Column-Oriented Storage Techniques for MapReduce	2011	VLDB	7.8517976e-05
3,071	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.6670408e-05
3,228	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.4957341e-05
3,761	Parallel Algorithms for Constructing Range and Nearest-Neighbor Searching Data Structures	2016	PODS	7.0186907e-05
4,302	A Method for Optimizing Opaque Filter Queries	2020	SIGMOD	6.6614501e-05
5,297	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.1773472e-05
6,862	Towards Unified Ad-hoc Data Processing	2014	SIGMOD	5.6458401e-05
9,728	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB	5.1205326e-05
10,050	Streaming Space Complexity of Nearly All Functions of One Variable on Frequency Vectors	2016	PODS	5.0786422e-05
12,514	Anti-Combining for MapReduce	2014	SIGMOD	4.9605343e-05
12,603	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	4.9605343e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
2	R-Trees: A Dynamic Index Structure For Spatial Searching	1984	SIGMOD	0.0019892478
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0010494385
12	C-Store: A Column-oriented DBMS	2005	VLDB	0.00068985264
27	Database Architecture Optimized for the New Bottleneck: Memory Access	1999	VLDB	0.0005146739
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00045453255
61	Integrating Compression and Execution in Column-Oriented Database Systems	2006	SIGMOD	0.00039323387
76	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD	0.00036932687
120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.00031017868
295	LINQ: Reconciling Objects, Relations and XML in the .NET Framework	2006	SIGMOD	0.00021876398
385	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00019419661
677	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00014835154

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,670	Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications	2018	SIGMOD
2	1,074	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
3	12,514	Anti-Combining for MapReduce	2014	SIGMOD
4	43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD
5	1,470	The Performance of MapReduce: An In-depth Study	2010	VLDB
6	2,299	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
7	76	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD
8	2,561	Minimal MapReduce Algorithms	2013	SIGMOD
9	3,228	Multi-Query Optimization in MapReduce Framework	2014	VLDB
10	759	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB