Adaptive and Robust Query Execution for Lakehouses at Scale
Summary: Presents Adaptive Query Execution (AQE), using pipeline breakers to gather runtime statistics and re-optimize plans despite stale metadata and misestimates. Deployed at Databricks at exabyte scale, delivering up to 25× TPC-DS speedups and improved robustness. (summarized by gpt-5.6-luna on Jul 21 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Maryann Xue (Databricks)
- 2. Yingyi Bu (Databricks)
- 3. Abhishek Somani (Databricks)
- 4. Wenchen Fan (Databricks)
- 5. Ziqi Liu (Databricks)
- 6. Steven Chen (Databricks)
- 7. Herman van Hovell (Databricks)
- 8. Bart Samwel (Databricks)
- 9. Mostafa Mokhtar (Databricks)
- 10. RK Korlapati (Databricks)
- 11. Andy Lam (Databricks)
- 12. Yunxiao Ma (Databricks)
- 13. Vuk Ercegovac (Databricks)
- 14. Jiexing Li (Databricks)
- 15. Alexander Behm (Databricks)
- 16. Yuanjian Li (Databricks)
- 17. Xiao Li (Databricks)
- 18. Sriram Krishnamurthy (Databricks)
- 19. Amit Shukla (Databricks)
- 20. Michalis Petropoulos (Databricks)
- 21. Sameer Paranjpye (Databricks)
- 22. Reynold Xin (Databricks)
- 23. Matei Zaharia (Databricks)
BibTeX Citation
@article{xue_vldb24,
title = {{Adaptive and Robust Query Execution for Lakehouses at Scale}},
author = {Xue, Maryann and Bu, Yingyi and Somani, Abhishek and Fan, Wenchen and Liu, Ziqi and Chen, Steven and van Hovell, Herman and Samwel, Bart and Mokhtar, Mostafa and Korlapati, RK and Lam, Andy and Ma, Yunxiao and Ercegovac, Vuk and Li, Jiexing and Behm, Alexander and Li, Yuanjian and Li, Xiao and Krishnamurthy, Sriram and Shukla, Amit and Petropoulos, Michalis and Paranjpye, Sameer and Xin, Reynold and Zaharia, Matei},
journal = {PVLDB},
series = {{VLDB} '24},
volume = {17},
number = {12},
pages = {3947--3959},
doi = {10.14778/3685800.3685818},
url = {https://doi.org/10.14778/3685800.3685818},
year = {2024}
}Incoming Citations (Sorted by Pagerank)
Showing 11 of 11 citing papers.
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 24 of 24 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 13,560 | Robust Data Transformations | 2015 | CIDR |
| 2 | 1,138 | Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics | 2021 | CIDR |
| 3 | 4,742 | Continuous Cloud-Scale Query Optimization and Processing | 2013 | VLDB |
| 4 | 8,448 | Towards Query Optimizer as a Service (QOaaS) in a Unified LakeHouse Ecosystem: Can One QO Rule Them All? | 2025 | CIDR |
| 5 | 1,824 | Photon: A Fast Query Engine for Lakehouse Systems | 2022 | SIGMOD |
| 6 | 4,649 | BigLake: BigQuery’s Evolution toward a Multi-Cloud Lakehouse | 2024 | SIGMOD |
| 7 | 8,615 | A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning | 2024 | VLDB |
| 8 | 1,022 | A Scalable, Predictable Join Operator for Highly Concurrent Data Warehouses | 2009 | VLDB |
| 9 | 10,536 | Active Data Lakes: Regaining Physical Data Independence Without Losing Interoperability | 2026 | VLDB |
| 10 | 7,780 | Petabyte-Scale Row-Level Operations in Data Lakehouses | 2024 | VLDB |