Stubby: A Transformation-based Optimizer for MapReduce Workflows

Summary: Stubby cost-based optimizes MapReduce workflows via plan-to-plan transformations, covering a comprehensive plan space across workflow interfaces. It selectively searches only plans that available metadata can enumerate and cost, enabling extensible optimization under incomplete information. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: hbde8f92957347773
Venue: VLDB
Year: 2012
Pagerank: 8.3425785e-05
Overall Rank: 2,527 | 83.02%
DOI: 10.14778/2350229.2350258

Incoming Non-self Citations Over Time

Authors

1. Harold Lim (Duke University)
2. Herodotos Herodotou (Duke University)
3. Shivnath Babu (Duke University)

BibTeX Citation

@article{lim_vldb12,
        title = {{Stubby: A Transformation-based Optimizer for MapReduce Workflows}},
        author = {Lim, Harold and Herodotou, Herodotos and Babu, Shivnath},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {11},
        pages = {1196--1207},
        doi = {10.14778/2350229.2350258},
        url = {https://doi.org/10.14778/2350229.2350258},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 13 of 13 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,081	Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML	2014	VLDB	0.00012123917
2,264	An Intermediate Representation for Optimizing Machine Learning Pipelines	2019	VLDB	8.7289107e-05
2,573	Minimal MapReduce Algorithms	2013	SIGMOD	8.2821647e-05
2,689	Major Technical Advancements in Apache Hive	2014	SIGMOD	8.1264718e-05
2,723	Exploiting Matrix Dependency for Efficient Distributed Matrix Computation	2015	SIGMOD	8.0944934e-05
3,066	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.6877117e-05
3,101	On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML	2018	VLDB	7.649219e-05
3,216	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.5234702e-05
3,556	WANalytics: Analytics for a Geo-Distributed Data-Intensive World	2015	CIDR	7.2084485e-05
4,069	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD	6.821366e-05
4,818	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB	6.3964573e-05
9,254	QMapper for Smart Grid: Migrating SQL-based Application to Hive	2015	SIGMOD	5.2056825e-05
9,815	IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows	2015	SIGMOD	5.1257999e-05

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00045546775
118	The EXODUS Optimizer Generator	1987	SIGMOD	0.00031392616
651	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00015130782
753	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014237583
823	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00013648332
1,281	Automatic Optimization for MapReduce Programs	2011	VLDB	0.00011213384
1,898	Querying Multiple Features of Groups in Relational Databases	1996	VLDB	9.4085557e-05
2,672	A Latency and Fault-Tolerance Optimizer for Online Parallel Query Plans	2011	SIGMOD	8.1491683e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,072	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
2	3,214	Efficient and Provable Multi-Query Optimization	2017	PODS
3	8,215	Execution Primitives for Scalable Joins and Aggregations in Map Reduce	2014	VLDB
4	1,924	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB
5	12,465	Anti-Combining for MapReduce	2014	SIGMOD
6	7,632	MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs	2014	VLDB
7	8,663	MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish	2011	VLDB
8	1,281	Automatic Optimization for MapReduce Programs	2011	VLDB
9	3,216	Multi-Query Optimization in MapReduce Framework	2014	VLDB
10	753	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB