DBScholar

Back to papers

Dynamically Optimizing Queries over Large Scale Data Platforms

Summary: Dynamic optimization of queries on large-scale data platforms, handling opaque UDFs and cross-relational correlations. Pilot runs estimate selectivities to seed a cost-based plan, after which plans evolve during execution to yield up to 2x (Jaql) and 4x (Hive) gains over hand-written left-deep baselines. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
4964
Venue
SIGMOD
Year
2014
Pagerank
6.6754521e-05
Overall Rank
4,481 | 69.26%
DOI
10.1145/2588555.2610531

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{karanasos_sigmod14,
        title = {{Dynamically Optimizing Queries over Large Scale Data Platforms}},
        author = {Karanasos, Konstantinos and Balmin, Andrey and Kutsch, Marcel and Özcan, Fatma and Ercegovac, Vuk and Xia, Chunyang and Jackson, Jesse},
        series = {{SIGMOD} '14},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2588555.2610531},
        url = {https://dl.acm.org/doi/10.1145/2588555.2610531},
        year = {2014}
}

Incoming Citations (Sorted by Pagerank)

Showing 11 of 11 citing papers.

Rank Citing Paper Year Venue Pagerank
1,256 Sampling-Based Query Re-Optimization 2016 SIGMOD 0.00011457194
1,468 Towards a Learning Optimizer for Shared Clouds 2019 VLDB 0.00010686496
1,712 SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning 2019 SIGMOD 9.9492299e-05
1,815 SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning 2018 VLDB 9.6894541e-05
3,494 WANalytics: Analytics for a Geo-Distributed Data-Intensive World 2015 CIDR 7.3642231e-05
3,605 Computation Reuse in Analytics Job Service at Microsoft 2018 SIGMOD 7.2640711e-05
4,049 Resource Elasticity for Large-Scale Machine Learning 2015 SIGMOD 6.9369379e-05
8,211 MONSOON: Multi-Step Optimization and Execution of Queries with Partially Obscured Predicates 2020 SIGMOD 5.4658735e-05
9,154 On-Demand State Separation for Cloud Data Warehousing 2022 VLDB 5.3123022e-05
9,167 Dynamic Speculative Optimizations for SQL Compilation in Apache Spark 2020 VLDB 5.3099969e-05
9,840 Distributed Numerical and Machine Learning Computations via Two-Phase Execution of Aggregated Join Trees 2021 VLDB 5.2103367e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 22 of 22 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1 Access Path Selection in a Relational Database Management System 1979 SIGMOD 0.0024089429
32 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00050111008
44 A Comparison of Approaches to Large-Scale Data Analysis 2009 SIGMOD 0.00046055057
89 On the Propagation of Errors in the Size of Join Results 1991 SIGMOD 0.00035031529
110 Extensible/Rule Based Query Rewrite Optimization in Starburst 1992 SIGMOD 0.0003309592
151 Efficient Mid-Query Re-Optimization of Sub-Optimal Query Execution Plans 1998 SIGMOD 0.00029161879
159 CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies 2004 SIGMOD 0.00028129426
288 Towards Estimation Error Guarantees for Distinct Values 2000 PODS 0.00022296371
425 Shark: SQL and Rich Analytics at Scale 2013 SIGMOD 0.00018704491
492 Robust Query Processing through Progressive Optimization 2004 SIGMOD 0.0001756877
642 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00015395331
689 On Synopses for Distinct-Value Estimation Under Multiset Operations 2007 SIGMOD 0.00014940023
769 A Comparison of Join Algorithms for Log Processing in MapReduce 2010 SIGMOD 0.00014166872
829 Proactive Re-Optimization 2005 SIGMOD 0.00013769838
1,009 Online Aggregation for Large MapReduce Jobs 2011 VLDB 0.00012684342
1,021 Jaql: A Scripting Language for Large Scale Semistructured Data Analysis 2011 VLDB 0.00012606673
1,693 BigBench: Towards an Industry Standard Benchmark for Big Data Analytics 2013 SIGMOD 9.9965799e-05
1,806 Effective Use of Block-Level Sampling in Statistics Estimation 2004 SIGMOD 9.7112151e-05
2,164 Opening the Black Boxes in Data Flow Optimization 2012 VLDB 9.0521951e-05
2,486 Stubby: A Transformation-based Optimizer for MapReduce Workflows 2012 VLDB 8.5143189e-05
4,742 Continuous Cloud-Scale Query Optimization and Processing 2013 VLDB 6.5269203e-05
8,064 Progressive Optimization in a Shared-Nothing Parallel Database 2007 SIGMOD 5.4949524e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 6,009 Optimization of Conjunctive Predicates for Main Memory Column Stores 2016 VLDB
2 2,762 Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection 2022 VLDB
3 6,044 Exploiting Soft and Hard Correlations in Big Data Query Optimization 2016 VLDB
4 1,286 Adaptive Optimization of Very Large Join Queries 2018 SIGMOD
5 5,552 Enabling Incremental Query Re-Optimization 2016 SIGMOD
6 984 Analyzing Plan Diagrams of Database Query Optimizers 2005 VLDB
7 295 Accelerating Machine Learning Inference with Probabilistic Predicates 2018 SIGMOD
8 9,446 Parallelizing Query Optimization on Shared-Nothing Architectures 2016 VLDB
9 289 Optimization of Dynamic Query Evaluation Plans 1994 SIGMOD
10 2,465 Optimization of Queries with User-defined Predicates 1996 VLDB