MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish

Summary: Starfish delivers the first cost-based optimizer for MapReduce programs with black-box map/reduce functions, enabling self-tuning for big-data analytics. A profiler, what-if engine, and Visualizer support execution insight, cost estimation under resource/data changes, and automatic optimization of MapReduce workflows. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 10414
Venue: VLDB
Year: 2011
Pagerank: 5.41357e-05
Overall Rank: 8,497 | 41.71%
DOI: 10.14778/3402755.3402791

Incoming Non-self Citations Over Time

Authors

1. Herodotos Herodotou (Duke University)
2. Fei Dong (Duke University)
3. Shivnath Babu (Duke University)

BibTeX Citation

@article{herodotou_vldb11,
        title = {{MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish}},
        author = {Herodotou, Herodotos and Dong, Fei and Babu, Shivnath},
        journal = {PVLDB},
        series = {{VLDB} '11},
        volume = {4},
        number = {12},
        pages = {1446--1449},
        doi = {10.14778/3402755.3402791},
        url = {https://doi.org/10.14778/3402755.3402791},
        year = {2011}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,163	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.6784171e-05

Outgoing Citations (Sorted by Pagerank)

Showing 2 of 2 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014522606
923	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR	0.00013189886

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,054	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
2	1,791	Cheetah: A High Performance, Custom Data Warehouse on Top of MapReduce	2010	VLDB
3	2,265	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
4	1,257	Automatic Optimization for MapReduce Programs	2011	VLDB
5	3,163	Multi-Query Optimization in MapReduce Framework	2014	VLDB
6	2,486	Stubby: A Transformation-based Optimizer for MapReduce Workflows	2012	VLDB
7	9,263	MapReduce Algorithms for Big Data Analysis	2012	VLDB
8	9,510	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB
9	923	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR
10	735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB