Database Paper Browser

Back to papers

Starfish: A Self-tuning System for Big Data Analytics

Summary: Starfish automatically tunes Hadoop MapReduce workflows to improve runtime, resource utilization, and cloud cost without manual knob fiddling. It adapts self-tuning DB techniques—cost models, profiling and reconfiguration—to MapReduce’s workload variability and pay‑as‑you‑go environments. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
169
Venue
CIDR
Year
2011
Pagerank
0.00014312777
Overall Rank
1,071 | 92.56%
DOI
-

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 31 of 31 citing papers.

Rank Citing Paper Year Venue Pagerank
868 Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs 2011 VLDB 0.00015789681
1,084 Dhalion: Self-Regulating Stream Processing in Heron 2017 VLDB 0.00014209714
1,334 SkewTune: Mitigating Skew in MapReduce Applications 2012 SIGMOD 0.0001250413
1,402 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012180605
1,532 Data Management in Machine Learning: Challenges, Techniques, and Systems 2017 SIGMOD 0.00011472681
1,534 PerfXplain: Debugging MapReduce Job Performance 2012 VLDB 0.00011468393
1,902 Black or White? How to Develop an AutoTuner for Memory-based Analytics 2020 SIGMOD 0.00010157713
2,886 VISTA: Optimized System for Declarative Feature Transfer from Deep CNNs at Scale 2020 SIGMOD 7.9612767e-05
3,279 Early Accurate Results for Advanced Analytics on MapReduce 2012 VLDB 7.2855494e-05
3,343 Comparative Evaluation of Big-Data Systems on Scientific Image Analytics Workloads 2017 VLDB 7.1967343e-05
3,659 Autoscaling Tiered Cloud Storage in Anna 2019 VLDB 6.8696023e-05
3,948 A Comparative Evaluation of Systems for Scalable Linear Algebra-based Analytics 2018 VLDB 6.5959084e-05
5,688 PREDIcT: Towards Predicting the Runtime of Large Scale Iterative Analytics 2013 VLDB 5.3702808e-05
5,888 Magnet: Push-based Shuffle Service for Large-scale Data Processing 2020 VLDB 5.2873617e-05
6,124 iQCAR: inter-Query Contention Analyzer for Data Analytics Frameworks 2019 SIGMOD 5.1988046e-05
6,261 The Cosmos Big Data Platform at Microsoft: Over a Decade of Progress and a Decade to Look Forward 2021 VLDB 5.1350714e-05
6,268 Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems 2019 VLDB 5.133857e-05
6,757 KEA: Tuning an Exabyte-Scale Data Infrastructure 2021 SIGMOD 4.9372134e-05
6,871 Towards General and Efficient Online Tuning for Spark 2023 VLDB 4.8997004e-05
7,296 Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities 2022 SIGMOD 4.7723197e-05
7,304 MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs 2014 VLDB 4.7684491e-05
7,684 AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft 2020 VLDB 4.6796855e-05
8,358 MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish 2011 VLDB 4.5372998e-05
8,924 QMapper for Smart Grid: Migrating SQL-based Application to Hive 2015 SIGMOD 4.427232e-05
9,066 Tempo: Robust and Self-Tuning Resource Management in Multi-tenant Parallel Databases 2016 VLDB 4.4035481e-05
9,503 IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows 2015 SIGMOD 4.3341665e-05
9,603 Saturn: An Optimized Data System for Multi-Large-Model Deep Learning Workloads 2024 VLDB 4.3177432e-05
10,852 CloudGlide: Deconstructing the Landscape of Cloud-Based Analytics 2025 VLDB 4.1945683e-05
11,056 Agile-Ant: Self-managing Distributed Cache Management for Cost Optimization of Big Data Applications 2024 VLDB 4.1945683e-05
11,341 Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications 2022 SIGMOD 4.1945683e-05
12,101 Optimization Strategies for A/B Testing on HADOOP 2013 VLDB 4.1945683e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 5 of 5 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
157 HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads 2009 VLDB 0.00040397359
168 MAD Skills: New Analysis Practices for Big Data 2009 VLDB 0.00038946305
794 Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing) 2010 VLDB 0.00016605103
947 MRShare: Sharing Across Multiple Queries in MapReduce 2010 VLDB 0.00015114576
1,615 The Performance of MapReduce: An In-depth Study 2010 VLDB 0.00011132319
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
7,958 CARTILAGE: Adding Flexibility to the Hadoop Skeleton 2013 SIGMOD 4.613363e-05
8,464 Piranha: Optimizing Short Jobs in Hadoop 2013 VLDB 4.5052127e-05
3,973 Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing 2019 SIGMOD 6.5758017e-05
2,476 A Platform for Scalable One-Pass Analytics using MapReduce 2011 SIGMOD 8.6960139e-05
9,504 Supporting Scalable Analytics with Latency Constraints 2015 VLDB 4.3341665e-05
13,356 Big Data Science Needs Big Data Middleware 2015 CIDR -
2,488 Shark: Fast Data Analysis Using Coarse-grained Distributed Memory 2012 SIGMOD 8.6683713e-05
542 Shark: SQL and Rich Analytics at Scale 2013 SIGMOD 0.00020595648
6,268 Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems 2019 VLDB 5.133857e-05
8,358 MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish 2011 VLDB 4.5372998e-05