Database Paper Browser

Back to papers

Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs

Summary: First cost-based optimizer for MapReduce programs, tackling the large configuration parameter space with black-box map/reduce functions. Profiler for unmodified programs and a what-if cost estimator enable data-driven optimization; prototype on Hadoop, with cross-domain evaluation. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
10175
Venue
VLDB
Year
2011
Pagerank
0.00015771189
Overall Rank
866 | 93.99%
DOI
-

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 37 of 37 citing papers.

Rank Citing Paper Year Venue Pagerank
981 Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads 2012 VLDB 0.00014866268
1,017 Robust Estimation of Resource Consumption for SQL Queries using Statistical Techniques 2012 VLDB 0.00014627121
1,407 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012163413
1,980 Split Query Processing in Polybase 2013 SIGMOD 9.8740305e-05
2,528 Pregelix: Big(ger) Graph Analytics on A Dataflow Engine 2015 VLDB 8.5953347e-05
2,616 Opening the Black Boxes in Data Flow Optimization 2012 VLDB 8.4457819e-05
2,672 Cumulon: Optimizing Statistical Data Analysis in the Cloud 2013 SIGMOD 8.3334428e-05
2,714 Minimal MapReduce Algorithms 2013 SIGMOD 8.2426646e-05
2,754 Stubby: A Transformation-based Optimizer for MapReduce Workflows 2012 VLDB 8.1720428e-05
2,929 WANalytics: Analytics for a Geo-Distributed Data-Intensive World 2015 CIDR 7.8792902e-05
3,435 Real-time Workload Pattern Analysis for Large-scale Cloud Databases 2023 VLDB 7.0946114e-05
3,709 Multi-Query Optimization in MapReduce Framework 2014 VLDB 6.8211506e-05
4,435 Clash of the Titans: MapReduce vs. Spark for Large Scale Data Analytics 2015 VLDB 6.18493e-05
4,807 Resource Elasticity for Large-Scale Machine Learning 2015 SIGMOD 5.9045148e-05
5,110 Only Aggressive Elephants are Fast Elephants 2012 VLDB 5.6868273e-05
5,617 HYPER: Hypothetical Reasoning With What-If and How-To Queries Using a Probabilistic Causal Approach 2022 SIGMOD 5.4085897e-05
5,699 PREDIcT: Towards Predicting the Runtime of Large Scale Iterative Analytics 2013 VLDB 5.3655096e-05
6,265 Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems 2019 VLDB 5.1294788e-05
6,493 Memory-Aware Framework for Efficient Second-Order Random Walk on Large Graphs 2020 SIGMOD 5.0344095e-05
6,565 Toward Interpretable and Actionable Data Analysis with Explanations and Causality 2022 VLDB 5.0033542e-05
6,817 Hadoop's Adolescence: An analysis of Hadoop usage in scientific workloads 2013 VLDB 4.9110239e-05
7,095 Fast Multi-Column Sorting in Main-Memory Column-Stores 2016 SIGMOD 4.8289712e-05
7,099 KEA: Tuning an Exabyte-Scale Data Infrastructure 2021 SIGMOD 4.8263529e-05
7,154 Submodularity of Distributed Join Computation 2018 SIGMOD 4.8107765e-05
7,298 MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs 2014 VLDB 4.7651301e-05
8,355 MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish 2011 VLDB 4.5328622e-05
8,754 Hyperspace: The Indexing Subsystem of Azure Synapse 2021 VLDB 4.4520434e-05
8,927 QMapper for Smart Grid: Migrating SQL-based Application to Hive 2015 SIGMOD 4.4229886e-05
9,360 Efficient Big Data Processing in Hadoop MapReduce 2012 VLDB 4.3476444e-05
9,505 Supporting Scalable Analytics with Latency Constraints 2015 VLDB 4.3300131e-05
9,735 UDAO: A Next-Generation Unified Data Analytics Optimizer 2019 VLDB 4.2901665e-05
11,370 PACk: An Efficient Partition-based Distributed Agglomerative Hierarchical Clustering Algorithm for Deduplication 2022 VLDB 4.1905499e-05
11,639 Automated Performance Management for the Big Data Stack 2019 CIDR 4.1905499e-05
11,673 Cost-Effective, Workload-Adaptive Migration of Big Data Applications to the Cloud 2019 SIGMOD 4.1905499e-05
11,699 An Experimental Evaluation of Garbage Collectors on Big Data Applications 2019 VLDB 4.1905499e-05
12,067 Workload Management for Big Data Analytics 2013 SIGMOD 4.1905499e-05
12,109 Optimization Strategies for A/B Testing on HADOOP 2013 VLDB 4.1905499e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
158 HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads 2009 VLDB 0.00040401371
408 HaLoop: Efficient Iterative Data Processing on Large Clusters 2010 VLDB 0.00023939456
789 Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing) 2010 VLDB 0.00016602215
947 MRShare: Sharing Across Multiple Queries in MapReduce 2010 VLDB 0.00015112344
962 A Comparison of Join Algorithms for Log Processing in MapReduce 2010 SIGMOD 0.00015003834
1,048 Starfish: A Self-tuning System for Big Data Analytics 2011 CIDR 0.00014442178
1,281 Automatic Optimization for MapReduce Programs 2011 VLDB 0.00012837411
1,615 The Performance of MapReduce: An In-depth Study 2010 VLDB 0.00011137225
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
158 HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads 2009 VLDB 0.00040401371
15 Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters 2007 SIGMOD 0.0010668335
9,360 Efficient Big Data Processing in Hadoop MapReduce 2012 VLDB 4.3476444e-05
981 Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads 2012 VLDB 0.00014866268
2,714 Minimal MapReduce Algorithms 2013 SIGMOD 8.2426646e-05
2,476 A Platform for Scalable One-Pass Analytics using MapReduce 2011 SIGMOD 8.6907971e-05
1,615 The Performance of MapReduce: An In-depth Study 2010 VLDB 0.00011137225
3,709 Multi-Query Optimization in MapReduce Framework 2014 VLDB 6.8211506e-05
1,281 Automatic Optimization for MapReduce Programs 2011 VLDB 0.00012837411
8,355 MapReduce Programming and Cost-based Optimization? Crossing this Chasm with Starfish 2011 VLDB 4.5328622e-05