DBScholar

Back to papers

Steering Query Optimizers: A Practical Take on Big Data Workloads

Summary: Steering query optimizers for big data; Bao adapted to SCOPE. Introduces rule signatures, a pipeline for recurring configs, and a learning method for unseen workloads; evaluated on 150K daily jobs with 7–30% latency savings, up to 90% on subset. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h89a5213c5639869f
Venue
SIGMOD
Year
2021
Pagerank
6.2678118e-05
Overall Rank
5,110 | 65.66%
DOI
10.1145/3448016.3457568
PDF
Download (CC BY 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{negi_sigmod21,
        title = {{Steering Query Optimizers: A Practical Take on Big Data Workloads}},
        author = {Negi, Parimarjan and Interlandi, Matteo and Marcus, Ryan and Alizadeh, Mohammad and Kraska, Tim and Friedman, Marc and Jindal, Alekh},
        series = {{SIGMOD} '21},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3448016.3457568},
        url = {https://dl.acm.org/doi/10.1145/3448016.3457568},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 22 citing papers.

Rank Citing Paper Year Venue Pagerank
2,209 Lero: A Learning-to-Rank Query Optimizer 2023 VLDB 8.8360101e-05
2,393 A Learned Query Rewrite System using Monte Carlo Tree Search 2022 VLDB 8.5298464e-05
2,844 FactorJoin: A New Cardinality Estimation Framework for Join Queries 2023 SIGMOD 7.9446987e-05
3,565 Auto-WLM: Machine Learning Enhanced Workload Management in Amazon Redshift 2023 SIGMOD 7.200937e-05
3,948 Deploying a Steered Query Optimizer in Production at Microsoft 2022 SIGMOD 6.9051584e-05
4,240 LEON: A New Framework for ML-Aided Query Optimization 2023 VLDB 6.7064546e-05
4,677 AutoSteer: Learned Query Optimization for Any SQL Database 2023 VLDB 6.4721041e-05
5,700 PilotScope: Steering Databases with Machine Learning Drivers 2024 VLDB 6.028998e-05
5,973 Towards instance-optimized data systems 2021 VLDB 5.9281867e-05
6,579 Towards Query Optimizer as a Service (QOaaS) in a Unified LakeHouse Ecosystem: Can One QO Rule Them All? 2025 CIDR 5.7421583e-05
7,356 PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost! 2021 VLDB 5.5421826e-05
7,546 Learned Offline Query Planning via Bayesian Optimization 2025 SIGMOD 5.4966669e-05
7,813 Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward 2021 VLDB 5.4474823e-05
8,046 Efficiently Computing Join Orders with Heuristic Search 2023 SIGMOD 5.3992239e-05
8,349 SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft 2021 VLDB 5.3485189e-05
8,357 Towards Building Autonomous Data Services on Azure 2023 SIGMOD 5.3463255e-05
8,972 GEqO: ML-Accelerated Semantic Equivalence Detection 2023 SIGMOD 5.2469075e-05
9,134 Hit the Gym: Accelerating Query Execution to Efficiently Bootstrap Behavior Models for Self-Driving Database Management Systems 2024 VLDB 5.2223611e-05
9,635 Low Rank Learning for Offline Query Optimization 2025 SIGMOD 5.1453041e-05
10,035 QO-Insight: Inspecting Steered Query Optimizers 2023 VLDB 5.0901047e-05
10,319 Intra-Query Runtime Elasticity for Cloud-Native Data Analysis 2025 SIGMOD 5.0362412e-05
10,418 AgenticScholar: Agentic Data Management with Pipeline Orchestration for Scholarly Corpora 2026 SIGMOD 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 17 of 17 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
15 How Good Are Query Optimizers, Really? 2016 VLDB 0.00061067652
30 SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets 2008 VLDB 0.00050475202
52 The Snowflake Elastic Data Warehouse 2016 SIGMOD 0.00041210636
85 Learned Cardinalities: Estimating Correlated Joins with Deep Learning 2019 CIDR 0.00035876108
144 Neo: A Learned Query Optimizer 2019 VLDB 0.00029090793
379 Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources 2018 SIGMOD 0.00019507406
406 Deep Unsupervised Cardinality Estimation 2020 VLDB 0.00019050182
510 NeuroCard: One Cardinality Estimator for All Tables 2021 VLDB 0.00017059914
617 F1: A Distributed SQL Database That Scales 2013 VLDB 0.00015548752
691 Selectivity Estimation for Range Predicates using Lightweight Models 2019 VLDB 0.00014737455
1,432 Towards a Learning Optimizer for Shared Clouds 2019 VLDB 0.00010676754
1,607 The Picasso Database Query Optimizer Visualizer 2010 VLDB 0.00010085995
2,833 Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings 2020 SIGMOD 7.9539771e-05
3,208 Efficiently Approximating Selectivity Functions using Low Overhead Regression Models 2020 VLDB 7.5355264e-05
3,544 Computation Reuse in Analytics Job Service at Microsoft 2018 SIGMOD 7.2108612e-05
6,114 Robustness Metrics for Relational Query Execution Plans 2018 VLDB 5.8796178e-05
7,767 AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft 2020 VLDB 5.4550466e-05
Previous Page 1 / 1 Next

Semantically Similar Papers