DBScholar

Back to papers

Dynamically Optimizing Queries over Large Scale Data Platforms

Summary: Dynamic optimization of queries on large-scale data platforms, handling opaque UDFs and cross-relational correlations. Pilot runs estimate selectivities to seed a cost-based plan, after which plans evolve during execution to yield up to 2x (Jaql) and 4x (Hive) gains over hand-written left-deep baselines. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h9c6add33e9df1fde
Venue
SIGMOD
Year
2014
Pagerank
6.821366e-05
Overall Rank
4,069 | 72.65%
DOI
10.1145/2588555.2610531

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{karanasos_sigmod14,
        title = {{Dynamically Optimizing Queries over Large Scale Data Platforms}},
        author = {Karanasos, Konstantinos and Balmin, Andrey and Kutsch, Marcel and Özcan, Fatma and Ercegovac, Vuk and Xia, Chunyang and Jackson, Jesse},
        series = {{SIGMOD} '14},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2588555.2610531},
        url = {https://dl.acm.org/doi/10.1145/2588555.2610531},
        year = {2014}
}

Incoming Citations (Sorted by Pagerank)

Showing 12 of 12 citing papers.

Rank Citing Paper Year Venue Pagerank
1,257 Sampling-Based Query Re-Optimization 2016 SIGMOD 0.00011310561
1,433 Towards a Learning Optimizer for Shared Clouds 2019 VLDB 0.00010677711
1,603 SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning 2019 SIGMOD 0.00010097649
1,800 SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning 2018 VLDB 9.6093317e-05
3,545 Computation Reuse in Analytics Job Service at Microsoft 2018 SIGMOD 7.2134803e-05
3,556 WANalytics: Analytics for a Geo-Distributed Data-Intensive World 2015 CIDR 7.2084485e-05
4,116 Resource Elasticity for Large-Scale Machine Learning 2015 SIGMOD 6.7961306e-05
8,345 MONSOON: Multi-Step Optimization and Execution of Queries with Partially Obscured Predicates 2020 SIGMOD 5.3511996e-05
9,316 On-Demand State Separation for Cloud Data Warehousing 2022 VLDB 5.1958405e-05
9,321 Dynamic Speculative Optimizations for SQL Compilation in Apache Spark 2020 VLDB 5.1943472e-05
10,021 Distributed Numerical and Machine Learning Computations via Two-Phase Execution of Aggregated Join Trees 2021 VLDB 5.0943545e-05
10,982 A Portable Middleware for Plan-Based Adaptive Query Processing 2026 VLDB 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 22 of 22 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1 Access Path Selection in a Relational Database Management System 1979 SIGMOD 0.0023947656
31 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00049839909
43 A Comparison of Approaches to Large-Scale Data Analysis 2009 SIGMOD 0.00045546775
91 On the Propagation of Errors in the Size of Join Results 1991 SIGMOD 0.0003475226
109 Extensible/Rule Based Query Rewrite Optimization in Starburst 1992 SIGMOD 0.0003276959
149 Efficient Mid-Query Re-Optimization of Sub-Optimal Query Execution Plans 1998 SIGMOD 0.00028981723
160 CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies 2004 SIGMOD 0.00027837289
295 Towards Estimation Error Guarantees for Distinct Values 2000 PODS 0.00021914399
432 Shark: SQL and Rich Analytics at Scale 2013 SIGMOD 0.00018339357
481 Robust Query Processing through Progressive Optimization 2004 SIGMOD 0.00017603972
651 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00015130782
707 On Synopses for Distinct-Value Estimation Under Multiset Operations 2007 SIGMOD 0.00014640173
799 A Comparison of Join Algorithms for Log Processing in MapReduce 2010 SIGMOD 0.00013889081
836 Proactive Re-Optimization 2005 SIGMOD 0.00013557047
1,022 Online Aggregation for Large MapReduce Jobs 2011 VLDB 0.00012438826
1,037 Jaql: A Scripting Language for Large Scale Semistructured Data Analysis 2011 VLDB 0.00012376819
1,702 BigBench: Towards an Industry Standard Benchmark for Big Data Analytics 2013 SIGMOD 9.8361594e-05
1,826 Effective Use of Block-Level Sampling in Statistics Estimation 2004 SIGMOD 9.5575424e-05
2,195 Opening the Black Boxes in Data Flow Optimization 2012 VLDB 8.8781177e-05
2,527 Stubby: A Transformation-based Optimizer for MapReduce Workflows 2012 VLDB 8.3425785e-05
4,818 Continuous Cloud-Scale Query Optimization and Processing 2013 VLDB 6.3964573e-05
8,185 Progressive Optimization in a Shared-Nothing Parallel Database 2007 SIGMOD 5.381532e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 6,106 Optimization of Conjunctive Predicates for Main Memory Column Stores 2016 VLDB
2 2,690 Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection 2022 VLDB
3 6,074 Exploiting Soft and Hard Correlations in Big Data Query Optimization 2016 VLDB
4 1,186 Adaptive Optimization of Very Large Join Queries 2018 SIGMOD
5 5,667 Enabling Incremental Query Re-Optimization 2016 SIGMOD
6 996 Analyzing Plan Diagrams of Database Query Optimizers 2005 VLDB
7 281 Accelerating Machine Learning Inference with Probabilistic Predicates 2018 SIGMOD
8 9,621 Parallelizing Query Optimization on Shared-Nothing Architectures 2016 VLDB
9 289 Optimization of Dynamic Query Evaluation Plans 1994 SIGMOD
10 2,516 Optimization of Queries with User-defined Predicates 1996 VLDB