Database Paper Browser

Back to papers

Dynamically Optimizing Queries over Large Scale Data Platforms

Summary: Dynamic optimization of queries on large-scale data platforms, handling opaque UDFs and cross-relational correlations. Pilot runs estimate selectivities to seed a cost-based plan, after which plans evolve during execution to yield up to 2x (Jaql) and 4x (Hive) gains over hand-written left-deep baselines. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
4902
Venue
SIGMOD
Year
2014
Pagerank
5.7586174e-05
Overall Rank
5,014 | 65.12%
DOI
10.1145/2588555.2610531

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 11 of 11 citing papers.

Rank Citing Paper Year Venue Pagerank
1,758 Sampling-Based Query Re-Optimization 2016 SIGMOD 0.00010655546
2,083 Towards a Learning Optimizer for Shared Clouds 2019 VLDB 9.5834572e-05
2,156 SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning 2018 VLDB 9.4170209e-05
2,219 SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning 2019 SIGMOD 9.2623533e-05
2,928 WANalytics: Analytics for a Geo-Distributed Data-Intensive World 2015 CIDR 7.8812874e-05
4,174 Computation Reuse in Analytics Job Service at Microsoft 2018 SIGMOD 6.3856219e-05
4,802 Resource Elasticity for Large-Scale Machine Learning 2015 SIGMOD 5.9114415e-05
8,158 MONSOON: Multi-Step Optimization and Execution of Queries with Partially Obscured Predicates 2020 SIGMOD 4.5730772e-05
9,124 Dynamic Speculative Optimizations for SQL Compilation in Apache Spark 2020 VLDB 4.391961e-05
9,125 On-Demand State Separation for Cloud Data Warehousing 2022 VLDB 4.3917246e-05
9,706 Distributed Numerical and Machine Learning Computations via Two-Phase Execution of Aggregated Join Trees 2021 VLDB 4.2992942e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 22 of 22 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1 Access Path Selection in a Relational Database Management System 1979 SIGMOD 0.0040449103
42 A Comparison of Approaches to Large-Scale Data Analysis 2009 SIGMOD 0.00073498298
70 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00059533166
99 On the Propagation of Errors in the Size of Join Results 1991 SIGMOD 0.00050022914
106 Extensible/Rule Based Query Rewrite Optimization in Starburst 1992 SIGMOD 0.00048400734
220 Efficient Mid-Query Re-Optimization of Sub-Optimal Query Execution Plans 1998 SIGMOD 0.00033194808
224 CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies 2004 SIGMOD 0.00032746205
378 Towards Estimation Error Guarantees for Distinct Values 2000 PODS 0.0002497492
542 Shark: SQL and Rich Analytics at Scale 2013 SIGMOD 0.00020595648
650 Robust Query Processing through Progressive Optimization 2004 SIGMOD 0.00018659177
727 On Synopses for Distinct-Value Estimation Under Multiset Operations 2007 SIGMOD 0.00017508726
780 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00016775082
960 A Comparison of Join Algorithms for Log Processing in MapReduce 2010 SIGMOD 0.00015012242
1,265 Jaql: A Scripting Language for Large Scale Semistructured Data Analysis 2011 VLDB 0.00012947629
1,272 Proactive Re-Optimization 2005 SIGMOD 0.00012920076
1,464 Online Aggregation for Large MapReduce Jobs 2011 VLDB 0.00011865546
1,727 BigBench: Towards an Industry Standard Benchmark for Big Data Analytics 2013 SIGMOD 0.00010740936
1,797 Effective Use of Block-Level Sampling in Statistics Estimation 2004 SIGMOD 0.00010523169
2,611 Opening the Black Boxes in Data Flow Optimization 2012 VLDB 8.4536967e-05
2,747 Stubby: A Transformation-based Optimizer for MapReduce Workflows 2012 VLDB 8.1828918e-05
5,297 Continuous Cloud-Scale Query Optimization and Processing 2013 VLDB 5.5801669e-05
8,165 Progressive Optimization in a Shared-Nothing Parallel Database 2007 SIGMOD 4.5717277e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
6,374 Optimization of Conjunctive Predicates for Main Memory Column Stores 2016 VLDB 5.0927058e-05
3,727 Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection 2022 VLDB 6.8141709e-05
6,173 Exploiting Soft and Hard Correlations in Big Data Query Optimization 2016 VLDB 5.1699414e-05
1,619 Adaptive Optimization of Very Large Join Queries 2018 SIGMOD 0.00011111678
5,727 Enabling Incremental Query Re-Optimization 2016 SIGMOD 5.3510544e-05
1,070 Analyzing Plan Diagrams of Database Query Optimizers 2005 VLDB 0.00014316791
329 Accelerating Machine Learning Inference with Probabilistic Predicates 2018 SIGMOD 0.00027249545
9,305 Parallelizing Query Optimization on Shared-Nothing Architectures 2016 VLDB 4.3577129e-05
339 Optimization of Dynamic Query Evaluation Plans 1994 SIGMOD 0.00026851113
2,860 Optimization of Queries with User-defined Predicates 1996 VLDB 7.9934503e-05