Stubby: A Transformation-based Optimizer for MapReduce Workflows

Summary: Stubby cost-based optimizes MapReduce workflows via plan-to-plan transformations, covering a comprehensive plan space across workflow interfaces. It selectively searches only plans that available metadata can enumerate and cost, enabling extensible optimization under incomplete information. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 10545
Venue: VLDB
Year: 2012
Pagerank: 8.5143189e-05
Overall Rank: 2,486 | 82.95%
DOI: 10.14778/2350229.2350258

Incoming Non-self Citations Over Time

Authors

1. Harold Lim (Duke University)
2. Herodotos Herodotou (Duke University)
3. Shivnath Babu (Duke University)

BibTeX Citation

@article{lim_vldb12,
        title = {{Stubby: A Transformation-based Optimizer for MapReduce Workflows}},
        author = {Lim, Harold and Herodotou, Herodotos and Babu, Shivnath},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {11},
        pages = {1196--1207},
        doi = {10.14778/2350229.2350258},
        url = {https://doi.org/10.14778/2350229.2350258},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 13 of 13 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,079	Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML	2014	VLDB	0.00012258469
2,239	An Intermediate Representation for Optimizing Machine Learning Pipelines	2019	VLDB	8.8875753e-05
2,539	Minimal MapReduce Algorithms	2013	SIGMOD	8.4526595e-05
2,681	Exploiting Matrix Dependency for Efficient Distributed Matrix Computation	2015	SIGMOD	8.2632778e-05
2,706	Major Technical Advancements in Apache Hive	2014	SIGMOD	8.2287564e-05
3,008	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.8578871e-05
3,163	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.6784171e-05
3,205	On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML	2018	VLDB	7.6386536e-05
3,494	WANalytics: Analytics for a Geo-Distributed Data-Intensive World	2015	CIDR	7.3642231e-05
4,481	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD	6.6754521e-05
4,742	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB	6.5269203e-05
9,079	QMapper for Smart Grid: Migrating SQL-based Application to Hive	2015	SIGMOD	5.3251649e-05
9,639	IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows	2015	SIGMOD	5.2434488e-05

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
44	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00046055057
119	The EXODUS Optimizer Generator	1987	SIGMOD	0.0003183602
642	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00015395331
735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014522606
803	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00013899943
1,257	Automatic Optimization for MapReduce Programs	2011	VLDB	0.000114432
1,860	Querying Multiple Features of Groups in Relational Databases	1996	VLDB	9.5965783e-05
2,622	A Latency and Fault-Tolerance Optimizer for Online Parallel Query Plans	2011	SIGMOD	8.3330136e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,054	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
2	3,268	Efficient and Provable Multi-Query Optimization	2017	PODS
3	8,069	Execution Primitives for Scalable Joins and Aggregations in Map Reduce	2014	VLDB
4	1,883	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB
5	12,174	Anti-Combining for MapReduce	2014	SIGMOD
6	7,546	MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs	2014	VLDB
7	8,497	MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish	2011	VLDB
8	1,257	Automatic Optimization for MapReduce Programs	2011	VLDB
9	3,163	Multi-Query Optimization in MapReduce Framework	2014	VLDB
10	735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB