Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings
Summary: Workload-driven cost models for big data queries, integrated into a Cascade-style optimizer to optimize plans and containers. In production, Cleo/SCOPE sees 2–3 orders higher accuracy and 20x correlation; ~70% plan changes cut latency and save resources. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Tarique Siddiqui (Microsoft; University of Illinois Urbana-Champaign)
- 2. Alekh Jindal (Microsoft)
- 3. Shi Qiao (Microsoft)
- 4. Hiren Patel (Microsoft)
- 5. Wangchao Le (Microsoft)
BibTeX Citation
@inproceedings{siddiqui_sigmod20,
title = {{Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings}},
author = {Siddiqui, Tarique and Jindal, Alekh and Qiao, Shi and Patel, Hiren and Le, Wangchao},
series = {{SIGMOD} '20},
booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
publisher = {Association for Computing Machinery},
doi = {10.1145/3318464.3380584},
url = {https://dl.acm.org/doi/10.1145/3318464.3380584},
year = {2020}
}Incoming Citations (Sorted by Pagerank)
Showing 40 of 40 citing papers.
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 24 of 24 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 2,619 | Zero-Shot Cost Models for Out-of-the-box Learned Cost Prediction | 2022 | VLDB |
| 2 | 5,549 | A Cost Model for Clustered Object-Oriented Databases | 1995 | VLDB |
| 3 | 4,209 | Schedule Optimization for Data Processing Flows on the Cloud | 2011 | SIGMOD |
| 4 | 11,484 | Learned Cost Models for Query Optimization: From Batch to Streaming Systems | 2025 | VLDB |
| 5 | 1,436 | Towards a Learning Optimizer for Shared Clouds | 2019 | VLDB |
| 6 | 4,641 | How Good are Learned Cost Models, Really? Insights from Query Optimization Tasks | 2025 | SIGMOD |
| 7 | 9,476 | Saving Money for Analytical Workloads in the Cloud | 2024 | VLDB |
| 8 | 2,162 | Towards Cost-Optimal Query Processing in the Cloud | 2021 | VLDB |
| 9 | 4,842 | Continuous Cloud-Scale Query Optimization and Processing | 2013 | VLDB |
| 10 | 5,126 | Steering Query Optimizers: A Practical Take on Big Data Workloads | 2021 | SIGMOD |