DBScholar

Back to papers

Steering Query Optimizers: A Practical Take on Big Data Workloads

Summary: Steering query optimizers for big data; Bao adapted to SCOPE. Introduces rule signatures, a pipeline for recurring configs, and a learning method for unseen workloads; evaluated on 150K daily jobs with 7–30% latency savings, up to 90% on subset. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h89a5213c5639869f
Venue
SIGMOD
Year
2021
Pagerank
6.2493606e-05
Overall Rank
5,126 | 65.65%
DOI
10.1145/3448016.3457568
PDF
Download (CC BY 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{negi_sigmod21,
        title = {{Steering Query Optimizers: A Practical Take on Big Data Workloads}},
        author = {Negi, Parimarjan and Interlandi, Matteo and Marcus, Ryan and Alizadeh, Mohammad and Kraska, Tim and Friedman, Marc and Jindal, Alekh},
        series = {{SIGMOD} '21},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3448016.3457568},
        url = {https://dl.acm.org/doi/10.1145/3448016.3457568},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 22 citing papers.

Rank Citing Paper Year Venue Pagerank
2,190 Lero: A Learning-to-Rank Query Optimizer 2023 VLDB 8.8848923e-05
2,389 A Learned Query Rewrite System using Monte Carlo Tree Search 2022 VLDB 8.5228689e-05
2,852 FactorJoin: A New Cardinality Estimation Framework for Join Queries 2023 SIGMOD 7.9201312e-05
3,328 Auto-WLM: Machine Learning Enhanced Workload Management in Amazon Redshift 2023 SIGMOD 7.4094448e-05
3,951 Deploying a Steered Query Optimizer in Production at Microsoft 2022 SIGMOD 6.8884038e-05
4,247 LEON: A New Framework for ML-Aided Query Optimization 2023 VLDB 6.6920059e-05
4,683 AutoSteer: Learned Query Optimization for Any SQL Database 2023 VLDB 6.4545989e-05
5,713 PilotScope: Steering Databases with Machine Learning Drivers 2024 VLDB 6.0090633e-05
5,982 Towards instance-optimized data systems 2021 VLDB 5.9096991e-05
6,590 Towards Query Optimizer as a Service (QOaaS) in a Unified LakeHouse Ecosystem: Can One QO Rule Them All? 2025 CIDR 5.7233093e-05
7,377 PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost! 2021 VLDB 5.5238616e-05
7,561 Learned Offline Query Planning via Bayesian Optimization 2025 SIGMOD 5.4784915e-05
7,830 Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward 2021 VLDB 5.4294933e-05
8,064 Efficiently Computing Join Orders with Heuristic Search 2023 SIGMOD 5.3813708e-05
8,362 SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft 2021 VLDB 5.3308358e-05
8,372 Towards Building Autonomous Data Services on Azure 2023 SIGMOD 5.3287108e-05
8,992 GEqO: ML-Accelerated Semantic Equivalence Detection 2023 SIGMOD 5.2295579e-05
9,154 Hit the Gym: Accelerating Query Execution to Efficiently Bootstrap Behavior Models for Self-Driving Database Management Systems 2024 VLDB 5.2050937e-05
9,662 Low Rank Learning for Offline Query Optimization 2025 SIGMOD 5.1283524e-05
10,074 QO-Insight: Inspecting Steered Query Optimizers 2023 VLDB 5.0732737e-05
10,347 Intra-Query Runtime Elasticity for Cloud-Native Data Analysis 2025 SIGMOD 5.0195882e-05
10,484 AgenticScholar: Agentic Data Management with Pipeline Orchestration for Scholarly Corpora 2026 SIGMOD 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 17 of 17 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
15 How Good Are Query Optimizers, Really? 2016 VLDB 0.00061463175
30 SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets 2008 VLDB 0.00050364918
52 The Snowflake Elastic Data Warehouse 2016 SIGMOD 0.00041439488
84 Learned Cardinalities: Estimating Correlated Joins with Deep Learning 2019 CIDR 0.00035920657
142 Neo: A Learned Query Optimizer 2019 VLDB 0.00029114937
382 Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources 2018 SIGMOD 0.0001946654
405 Deep Unsupervised Cardinality Estimation 2020 VLDB 0.00019020138
508 NeuroCard: One Cardinality Estimator for All Tables 2021 VLDB 0.00017108973
617 F1: A Distributed SQL Database That Scales 2013 VLDB 0.00015511217
690 Selectivity Estimation for Range Predicates using Lightweight Models 2019 VLDB 0.00014703258
1,436 Towards a Learning Optimizer for Shared Clouds 2019 VLDB 0.00010658836
1,604 The Picasso Database Query Optimizer Visualizer 2010 VLDB 0.00010086711
2,828 Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings 2020 SIGMOD 7.9476509e-05
3,212 Efficiently Approximating Selectivity Functions using Low Overhead Regression Models 2020 VLDB 7.5145409e-05
3,550 Computation Reuse in Analytics Job Service at Microsoft 2018 SIGMOD 7.1895215e-05
6,128 Robustness Metrics for Relational Query Execution Plans 2018 VLDB 5.860258e-05
7,764 AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft 2020 VLDB 5.4403231e-05
Previous Page 1 / 1 Next

Semantically Similar Papers