LOCAT: Low-Overhead Online Configuration Auto-Tuning of Spark SQL Applications

Summary: LOCAT uses online Bayesian optimization to tune Spark SQL configs, with QCSA to drop insensitive queries, DAGP for data-size modeling, and IICP to tune only influential parameters. Low-overhead, data-size-adaptive tuning yields 4.1–9.7× faster optimization and 1.9–2.4× speedups on TPC-DS, TPC-H, and HiBench on ARM/x86. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6530
Venue: SIGMOD
Year: 2022
Pagerank: 6.4779623e-05
Overall Rank: 4,854 | 66.70%
DOI: 10.1145/3514221.3526157

Incoming Non-self Citations Over Time

Authors

1. Jinhan Xin (Shenzhen University; University of Chinese Academy of Sciences)
2. Kai Hwang (Chinese University of Hong Kong)
3. Zhibin Yu (Huawei; Shenzhen University)

BibTeX Citation

@inproceedings{xin_sigmod22,
        title = {{LOCAT: Low-Overhead Online Configuration Auto-Tuning of Spark SQL Applications}},
        author = {Xin, Jinhan and Hwang, Kai and Yu, Zhibin},
        series = {{SIGMOD} '22},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3514221.3526157},
        url = {https://dl.acm.org/doi/10.1145/3514221.3526157},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 9 of 9 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
6,344	Towards General and Efficient Online Tuning for Spark	2023	VLDB	5.9060457e-05
8,457	Towards Resource Efficiency: Practical Insights into Large-Scale Spark Workloads at ByteDance	2024	VLDB	5.4217837e-05
8,615	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB	5.4005602e-05
9,771	Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment	2025	SIGMOD	5.2209769e-05
10,105	SCompression: Enhancing Database Knob Tuning Efficiency Through Slice-Based OLTP Workload Compression	2025	VLDB	5.1435736e-05
10,565	LakeHelm: Zero-Shot Lakehouse Advisor for Joint Engine-Format Selection and Configuration	2026	VLDB	5.093636e-05
10,660	Centrum: Model-based Database Auto-tuning with Minimal Distributional Assumptions	2025	SIGMOD	5.093636e-05
11,083	Graph Transformers for Query Plan Representation: Potentials and Challenges	2025	VLDB	5.093636e-05
11,091	LEAP: A Low-cost Spark SQL Query Optimizer using Pairwise Comparison	2025	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 5 of 5 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
24	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00054865648
32	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00050111008
334	An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning	2019	SIGMOD	0.00020875082
498	QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning	2019	VLDB	0.00017440583
1,686	Black or White? How to Develop an AutoTuner for Memory-based Analytics	2020	SIGMOD	0.00010008686

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	2,594	Big Data Analytics with Datalog Queries on Spark	2016	SIGMOD
2	10,886	AQETuner: Reliable Query-level Configuration Tuning for Analytical Query Engines	2025	VLDB
3	5,059	Steering Query Optimizers: A Practical Take on Big Data Workloads	2021	SIGMOD
4	9,771	Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment	2025	SIGMOD
5	5,865	Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems	2019	VLDB
6	10,384	MCTuner: Spatial Decomposition-Enhanced Database Tuning via LLM-Guided Exploration	2026	SIGMOD
7	11,091	LEAP: A Low-cost Spark SQL Query Optimizer using Pairwise Comparison	2025	VLDB
8	4,011	Towards Dynamic and Safe Configuration Tuning for Cloud Databases	2022	SIGMOD
9	8,615	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB
10	6,344	Towards General and Efficient Online Tuning for Spark	2023	VLDB