Automatic Optimization for MapReduce Programs

Summary: Manimal statically analyzes free-form MapReduce code to infer relational data semantics (e.g., selections/projections) without programmer annotations. It automatically applies database-style optimizations, achieving up to 1,121% speedups on existing jobs. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 10458
Venue: VLDB
Year: 2011
Pagerank: 0.000114432
Overall Rank: 1,257 | 91.38%
DOI: 10.14778/2020.2021

Incoming Non-self Citations Over Time

Authors

1. Eaman Jahani (University of Michigan)
2. Michael J. Cafarella (University of Michigan)
3. Christopher Ré (University of Wisconsin)

BibTeX Citation

@article{jahani_vldb11,
        title = {{Automatic Optimization for MapReduce Programs}},
        author = {Jahani, Eaman and Cafarella, Michael J. and Ré, Christopher},
        journal = {PVLDB},
        series = {{VLDB} '11},
        volume = {4},
        number = {6},
        pages = {385--396},
        doi = {10.14778/2020.2021},
        url = {https://doi.org/10.14778/2020.2021},
        year = {2011}
}

Incoming Citations (Sorted by Pagerank)

Showing 19 of 19 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014522606
798	DBSherlock: A Performance Diagnostic Tool for Transactional Databases	2016	SIGMOD	0.00013919795
1,054	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00012390673
1,694	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.9953156e-05
2,164	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	9.0521951e-05
2,398	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	8.631172e-05
2,486	Stubby: A Transformation-based Optimizer for MapReduce Workflows	2012	VLDB	8.5143189e-05
2,539	Minimal MapReduce Algorithms	2013	SIGMOD	8.4526595e-05
2,849	Column-Oriented Storage Techniques for MapReduce	2011	VLDB	8.053191e-05
3,008	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.8578871e-05
3,163	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.6784171e-05
3,685	Parallel Algorithms for Constructing Range and Nearest-Neighbor Searching Data Structures	2016	PODS	7.2030591e-05
4,264	A Method for Optimizing Opaque Filter Queries	2020	SIGMOD	6.7937529e-05
5,156	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.3410921e-05
6,714	Towards Unified Ad-hoc Data Processing	2014	SIGMOD	5.7959651e-05
9,510	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB	5.2576928e-05
9,821	Streaming Space Complexity of Nearly All Functions of One Variable on Frequency Vectors	2016	PODS	5.214913e-05
12,174	Anti-Combining for MapReduce	2014	SIGMOD	5.093636e-05
12,263	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
2	R-Trees: A Dynamic Index Structure For Spatial Searching	1984	SIGMOD	0.0020210012
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0010686205
12	C-Store: A Column-oriented DBMS	2005	VLDB	0.00069513174
29	Database Architecture Optimized for the New Bottleneck: Memory Access	1999	VLDB	0.00052093615
44	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00046055057
60	Integrating Compression and Execution in Column-Oriented Database Systems	2006	SIGMOD	0.0003955489
72	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD	0.00037695852
120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.00031680027
292	LINQ: Reconciling Objects, Relations and XML in the .NET Framework	2006	SIGMOD	0.00022259549
372	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.0001981521
660	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00015198804

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,054	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
2	4,599	Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications	2018	SIGMOD
3	12,174	Anti-Combining for MapReduce	2014	SIGMOD
4	44	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD
5	1,436	The Performance of MapReduce: An In-depth Study	2010	VLDB
6	2,265	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
7	72	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD
8	2,539	Minimal MapReduce Algorithms	2013	SIGMOD
9	3,163	Multi-Query Optimization in MapReduce Framework	2014	VLDB
10	735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB