Back to papers
Steering Query Optimizers: A Practical Take on Big Data Workloads
Summary: Steering query optimizers for big data; Bao adapted to SCOPE. Introduces rule signatures, a pipeline for recurring configs, and a learning method for unseen workloads; evaluated on 150K daily jobs with 7–30% latency savings, up to 90% on subset.
(summarized by gpt-5-nano on Feb 09 2026)
- Paper ID
- 6258
- Venue
- SIGMOD
- Year
- 2021
- Pagerank
- 5.2367998e-05
- Overall Rank
- 5,994 | 58.35%
- DOI
-
10.1145/3448016.3457568
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 21 of 21 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 3,241 |
A Learned Query Rewrite System using Monte Carlo Tree Search |
2022 |
VLDB |
7.32744e-05 |
| 3,345 |
Lero: A Learning-to-Rank Query Optimizer |
2023 |
VLDB |
7.1908499e-05 |
| 3,992 |
FactorJoin: A New Cardinality Estimation Framework for Join Queries |
2023 |
SIGMOD |
6.5519369e-05 |
| 4,592 |
Auto-WLM: Machine Learning Enhanced Workload Management in Amazon Redshift |
2023 |
SIGMOD |
6.056004e-05 |
| 4,687 |
Deploying a Steered Query Optimizer in Production at Microsoft |
2022 |
SIGMOD |
5.9915268e-05 |
| 5,339 |
LEON: A New Framework for ML-Aided Query Optimization |
2023 |
VLDB |
5.5596755e-05 |
| 5,654 |
AutoSteer: Learned Query Optimization for Any SQL Database |
2023 |
VLDB |
5.3882121e-05 |
| 6,298 |
Towards instance-optimized data systems |
2021 |
VLDB |
5.1182917e-05 |
| 6,883 |
PilotScope: Steering Databases with Machine Learning Drivers |
2024 |
VLDB |
4.8918682e-05 |
| 7,652 |
Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward |
2021 |
VLDB |
4.6831938e-05 |
| 8,149 |
Efficiently Computing Join Orders with Heuristic Search |
2023 |
SIGMOD |
4.5715614e-05 |
| 8,196 |
SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft |
2021 |
VLDB |
4.5568952e-05 |
| 8,219 |
PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost! |
2021 |
VLDB |
4.551524e-05 |
| 8,378 |
Towards Building Autonomous Data Services on Azure |
2023 |
SIGMOD |
4.5275731e-05 |
| 8,579 |
Towards Query Optimizer as a Service (QOaaS) in a Unified LakeHouse Ecosystem: Can One QO Rule Them All? |
2025 |
CIDR |
4.4877266e-05 |
| 8,660 |
Learned Offline Query Planning via Bayesian Optimization |
2025 |
SIGMOD |
4.4680058e-05 |
| 8,780 |
GEqO: ML-Accelerated Semantic Equivalence Detection |
2023 |
SIGMOD |
4.4485568e-05 |
| 9,012 |
Hit the Gym: Accelerating Query Execution to Efficiently Bootstrap Behavior Models for Self-Driving Database Management Systems |
2024 |
VLDB |
4.4059413e-05 |
| 9,581 |
Low Rank Learning for Offline Query Optimization |
2025 |
SIGMOD |
4.3186744e-05 |
| 9,709 |
QO-Insight: Inspecting Steered Query Optimizers |
2023 |
VLDB |
4.2951473e-05 |
| 10,501 |
Intra-Query Runtime Elasticity for Cloud-Native Data Analysis |
2025 |
SIGMOD |
4.1905499e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 17 of 17 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank |
Cited Paper |
Year |
Venue |
Pagerank |
| 22 |
SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets |
2008 |
VLDB |
0.00084679526 |
| 71 |
How Good Are Query Optimizers, Really? |
2016 |
VLDB |
0.00059446482 |
| 167 |
The Snowflake Elastic Data Warehouse |
2016 |
SIGMOD |
0.00039408116 |
| 203 |
Learned Cardinalities: Estimating Correlated Joins with Deep Learning |
2019 |
CIDR |
0.00034868567 |
| 329 |
Neo: A Learned Query Optimizer |
2019 |
VLDB |
0.00027301488 |
| 542 |
Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources |
2018 |
SIGMOD |
0.00020522627 |
| 752 |
Deep Unsupervised Cardinality Estimation |
2020 |
VLDB |
0.00017138049 |
| 900 |
F1: A Distributed SQL Database That Scales |
2013 |
VLDB |
0.00015466804 |
| 905 |
NeuroCard: One Cardinality Estimator for All Tables |
2021 |
VLDB |
0.00015423174 |
| 1,239 |
Selectivity Estimation for Range Predicates using Lightweight Models |
2019 |
VLDB |
0.00013091459 |
| 1,297 |
The Picasso Database Query Optimizer Visualizer |
2010 |
VLDB |
0.00012732768 |
| 2,080 |
Towards a Learning Optimizer for Shared Clouds |
2019 |
VLDB |
9.5954034e-05 |
| 3,623 |
Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings |
2020 |
SIGMOD |
6.9017341e-05 |
| 3,955 |
Efficiently Approximating Selectivity Functions using Low Overhead Regression Models |
2020 |
VLDB |
6.5895015e-05 |
| 4,171 |
Computation Reuse in Analytics Job Service at Microsoft |
2018 |
SIGMOD |
6.3800823e-05 |
| 6,763 |
Robustness Metrics for Relational Query Execution Plans |
2018 |
VLDB |
4.9291549e-05 |
| 7,685 |
AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft |
2020 |
VLDB |
4.6753414e-05 |
Semantically Similar Papers
| Overall Rank |
Paper |
Year |
Venue |
Pagerank |
| 9,117 |
Deep Query Optimization |
2019 |
SIGMOD |
4.3885415e-05 |
| 5,012 |
Dynamically Optimizing Queries over Large Scale Data Platforms |
2014 |
SIGMOD |
5.7543101e-05 |
| 7,564 |
Modeling Shifting Workloads for Learned Database Systems |
2024 |
SIGMOD |
4.7049893e-05 |
| 3,658 |
Towards a Hands-Free Query Optimizer through Deep Learning |
2019 |
CIDR |
6.8700949e-05 |
| 6,639 |
Leveraging Query Logs and Machine Learning for Parametric Query Optimization |
2022 |
VLDB |
4.976781e-05 |
| 5,654 |
AutoSteer: Learned Query Optimization for Any SQL Database |
2023 |
VLDB |
5.3882121e-05 |
| 5,309 |
Continuous Cloud-Scale Query Optimization and Processing |
2013 |
VLDB |
5.5714729e-05 |
| 3,623 |
Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings |
2020 |
SIGMOD |
6.9017341e-05 |
| 6,671 |
Incorporating Super-Operators in Big-Data Query Optimizers |
2020 |
VLDB |
4.9625353e-05 |
| 4,687 |
Deploying a Steered Query Optimizer in Production at Microsoft |
2022 |
SIGMOD |
5.9915268e-05 |