DBScholar

Back to papers

Starfish: A Self-tuning System for Big Data Analytics

Summary: Starfish automatically tunes Hadoop MapReduce workflows to improve runtime, resource utilization, and cloud cost without manual knob fiddling. It adapts self-tuning DB techniques—cost models, profiling and reconfiguration—to MapReduce’s workload variability and pay‑as‑you‑go environments. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
hb6187a396092158c
Venue
CIDR
Year
2011
Pagerank
0.00012959992
Overall Rank
940 | 93.69%
DOI
-
PDF
Download (CC BY 3.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{herodotou_cidr11,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '11},
        title = {{Starfish: A Self-tuning System for Big Data Analytics}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Herodotou, Herodotos and Lim, Harold and Luo, Gang and Borisov, Nedyalko and Dong, Liang and Cetin, Fatma Bilgen and Babu, Shivnath},
        year = {2011}
}

Incoming Citations (Sorted by Pagerank)

Showing 32 of 32 citing papers.

Rank Citing Paper Year Venue Pagerank
754 Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs 2011 VLDB 0.00014231311
1,082 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012118261
1,223 Data Management in Machine Learning: Challenges, Techniques, and Systems 2017 SIGMOD 0.00011468426
1,236 Dhalion: Self-Regulating Stream Processing in Heron 2017 VLDB 0.00011400483
1,351 SkewTune: Mitigating Skew in MapReduce Applications 2012 SIGMOD 0.00010929229
1,699 Black or White? How to Develop an AutoTuner for Memory-based Analytics 2020 SIGMOD 9.8403606e-05
1,708 PerfXplain: Debugging MapReduce Job Performance 2012 VLDB 9.8184303e-05
3,029 Autoscaling Tiered Cloud Storage in Anna 2019 VLDB 7.7341355e-05
3,144 Early Accurate Results for Advanced Analytics on MapReduce 2012 VLDB 7.5947114e-05
3,518 A Comparative Evaluation of Systems for Scalable Linear Algebra-based Analytics 2018 VLDB 7.236638e-05
3,632 VISTA: Optimized System for Declarative Feature Transfer from Deep CNNs at Scale 2020 SIGMOD 7.146238e-05
4,309 Comparative Evaluation of Big-Data Systems on Scientific Image Analytics Workloads 2017 VLDB 6.6703601e-05
5,392 PREDIcT: Towards Predicting the Runtime of Large Scale Iterative Analytics 2013 VLDB 6.1494698e-05
5,507 Magnet: Push-based Shuffle Service for Large-scale Data Processing 2020 VLDB 6.0989657e-05
5,969 Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems 2019 VLDB 5.9296488e-05
6,113 iQCAR: inter-Query Contention Analyzer for Data Analytics Frameworks 2019 SIGMOD 5.880013e-05
6,248 The Cosmos Big Data Platform at Microsoft: Over a Decade of Progress and a Decade to Look Forward 2021 VLDB 5.8345657e-05
6,453 Towards General and Efficient Online Tuning for Spark 2023 VLDB 5.7790502e-05
6,758 Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities 2022 SIGMOD 5.6877154e-05
7,071 Saturn: An Optimized Data System for Multi-Large-Model Deep Learning Workloads 2024 VLDB 5.6056639e-05
7,086 KEA: Tuning an Exabyte-Scale Data Infrastructure 2021 SIGMOD 5.6002958e-05
7,638 MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs 2014 VLDB 5.4767699e-05
7,767 AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft 2020 VLDB 5.4550466e-05
8,671 MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish 2011 VLDB 5.2903241e-05
9,199 Tempo: Robust and Self-Tuning Resource Management in Multi-tenant Parallel Databases 2016 VLDB 5.2088173e-05
9,264 QMapper for Smart Grid: Migrating SQL-based Application to Hive 2015 SIGMOD 5.2032182e-05
9,822 IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows 2015 SIGMOD 5.1233734e-05
10,757 LakeHelm: Zero-Shot Lakehouse Advisor for Joint Engine-Format Selection and Configuration 2026 VLDB 4.9769913e-05
11,439 CloudGlide: Deconstructing the Landscape of Cloud-Based Analytics 2025 VLDB 4.9769913e-05
11,598 Agile-Ant: Self-managing Distributed Cache Management for Cost Optimization of Big Data Applications 2024 VLDB 4.9769913e-05
11,854 Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications 2022 SIGMOD 4.9769913e-05
12,595 Optimization Strategies for A/B Testing on HADOOP 2013 VLDB 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 5 of 5 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
120 HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads 2009 VLDB 0.00031099083
154 MAD Skills: New Analysis Practices for Big Data 2009 VLDB 0.00028568843
675 Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing) 2010 VLDB 0.00014880686
823 MRShare: Sharing Across Multiple Queries in MapReduce 2010 VLDB 0.00013642066
1,466 The Performance of MapReduce: An In-depth Study 2010 VLDB 0.00010565007
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 8,451 CARTILAGE: Adding Flexibility to the Hadoop Skeleton 2013 SIGMOD
2 8,631 Piranha: Optimizing Short Jobs in Hadoop 2013 VLDB
3 3,446 Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing 2019 SIGMOD
4 2,285 A Platform for Scalable One-Pass Analytics using MapReduce 2011 SIGMOD
5 9,823 Supporting Scalable Analytics with Latency Constraints 2015 VLDB
6 13,875 Big Data Science Needs Big Data Middleware 2015 CIDR
7 2,233 Shark: Fast Data Analysis Using Coarse-grained Distributed Memory 2012 SIGMOD
8 432 Shark: SQL and Rich Analytics at Scale 2013 SIGMOD
9 5,969 Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems 2019 VLDB
10 8,671 MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish 2011 VLDB