LOCAT: Low-Overhead Online Configuration Auto-Tuning of Spark SQL Applications

Summary: LOCAT uses online Bayesian optimization to tune Spark SQL configs, with QCSA to drop insensitive queries, DAGP for data-size modeling, and IICP to tune only influential parameters. Low-overhead, data-size-adaptive tuning yields 4.1–9.7× faster optimization and 1.9–2.4× speedups on TPC-DS, TPC-H, and HiBench on ARM/x86. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h767e35a0d8024885
Venue: SIGMOD
Year: 2022
Pagerank: 6.4721364e-05
Overall Rank: 4,681 | 68.53%
DOI: 10.1145/3514221.3526157

Incoming Non-self Citations Over Time

Authors

1. Jinhan Xin (Shenzhen University; University of Chinese Academy of Sciences)
2. Kai Hwang (Chinese University of Hong Kong)
3. Zhibin Yu (Huawei; Shenzhen University)

BibTeX Citation

@inproceedings{xin_sigmod22,
        title = {{LOCAT: Low-Overhead Online Configuration Auto-Tuning of Spark SQL Applications}},
        author = {Xin, Jinhan and Hwang, Kai and Yu, Zhibin},
        series = {{SIGMOD} '22},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3514221.3526157},
        url = {https://dl.acm.org/doi/10.1145/3514221.3526157},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 10 of 10 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
6,450	Towards General and Efficient Online Tuning for Spark	2023	VLDB	5.7817872e-05
8,004	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB	5.4089097e-05
8,601	Towards Resource Efficiency: Practical Insights into Large-Scale Spark Workloads at ByteDance	2024	VLDB	5.3051732e-05
9,023	Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment	2025	SIGMOD	5.2348677e-05
9,956	Graph Transformers for Query Plan Representation: Potentials and Challenges	2025	VLDB	5.1038322e-05
10,333	SCompression: Enhancing Database Knob Tuning Efficiency Through Slice-Based OLTP Workload Compression	2025	VLDB	5.0281656e-05
10,747	LakeHelm: Zero-Shot Lakehouse Advisor for Joint Engine-Format Selection and Configuration	2026	VLDB	4.9793485e-05
10,887	MFTune: An Efficient Multi-fidelity Framework for Spark SQL Configuration Tuning	2026	VLDB	4.9793485e-05
11,103	Centrum: Model-based Database Auto-tuning with Minimal Distributional Assumptions	2025	SIGMOD	4.9793485e-05
11,443	LEAP: A Low-cost Spark SQL Query Optimizer using Pairwise Comparison	2025	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 5 of 5 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
23	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00055406774
31	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00049839909
314	An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning	2019	SIGMOD	0.00021282642
437	QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning	2019	VLDB	0.00018315867
1,699	Black or White? How to Develop an AutoTuner for Memory-based Analytics	2020	SIGMOD	9.8445322e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	11,287	AQETuner: Reliable Query-level Configuration Tuning for Analytical Query Engines	2025	VLDB
2	5,110	Steering Query Optimizers: A Practical Take on Big Data Workloads	2021	SIGMOD
3	9,023	Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment	2025	SIGMOD
4	5,967	Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems	2019	VLDB
5	10,580	MCTuner: Spatial Decomposition-Enhanced Database Tuning via LLM-Guided Exploration	2026	SIGMOD
6	11,443	LEAP: A Low-cost Spark SQL Query Optimizer using Pairwise Comparison	2025	VLDB
7	4,079	Towards Dynamic and Safe Configuration Tuning for Cloud Databases	2022	SIGMOD
8	10,887	MFTune: An Efficient Multi-fidelity Framework for Spark SQL Configuration Tuning	2026	VLDB
9	8,004	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB
10	6,450	Towards General and Efficient Online Tuning for Spark	2023	VLDB