Scarf: Self-Adaptive Tuning via Multi-Objective Reinforcement Learning for Apache Flink

Summary: Scarf applies multi-objective RL to jointly tune Flink throughput and resource usage, learning an offline Pareto frontier. Workload-sensitivity clustering and transferable GNN actor-critics enable fast knob selection and adaptation across dynamic, heterogeneous jobs. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 14484
Venue: VLDB
Year: 2026
Pagerank: 5.093636e-05
Overall Rank: 10,547 | 27.64%
DOI: 10.14778/3801059.3801066

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Liu Liu (Zhejiang University)
2. Shenghao Gong (Zhejiang University)
3. Ziquan Fang (Zhejiang University)
4. Yunjun Gao (Zhejiang University)

BibTeX Citation

@article{liu_vldb26,
        title = {{Scarf: Self-Adaptive Tuning via Multi-Objective Reinforcement Learning for Apache Flink}},
        author = {Liu, Liu and Gong, Shenghao and Fang, Ziquan and Gao, Yunjun},
        journal = {PVLDB},
        series = {{VLDB} '26},
        volume = {19},
        number = {7},
        pages = {1516--1529},
        doi = {10.14778/3801059.3801066},
        url = {https://doi.org/10.14778/3801059.3801066},
        year = {2026}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 22 of 22 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
86	Automatic Database Management System Tuning Through Large-scale Machine Learning	2017	SIGMOD	0.00035316107
334	An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning	2019	SIGMOD	0.00020875082
347	Tuning Database Configuration Parameters with iTuned	2009	VLDB	0.00020651582
361	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB	0.00020138717
498	QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning	2019	VLDB	0.00017440583
983	Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management	2013	SIGMOD	0.0001283214
1,190	Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark	2018	SIGMOD	0.00011743246
1,224	Dhalion: Self-Regulating Stream Processing in Heron	2017	VLDB	0.00011596911
1,337	DB-BERT: A Database Tuning Tool that "Reads the Manual"	2022	SIGMOD	0.00011117488
1,422	State Management in Apache Flink: Consistent Stateful Distributed Stream Processing	2017	VLDB	0.00010823039
2,540	Multi-Objective Parametric Query Optimization	2015	VLDB	8.45187e-05
2,740	Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation	2022	VLDB	8.1855759e-05
3,343	LlamaTune: Sample-Efficient DBMS Configuration Tuning	2022	VLDB	7.4983591e-05
3,587	HUNTER: An Online Cloud Database Hybrid Tuning System for Personalized Requirements	2022	SIGMOD	7.2829345e-05
3,926	UDO: Universal Database Optimization using Reinforcement Learning	2021	VLDB	7.0128068e-05
4,117	Schedule Optimization for Data Processing Flows on the Cloud	2011	SIGMOD	6.8919486e-05
4,880	An Incremental Anytime Algorithm for Multi-Objective Query Optimization	2015	SIGMOD	6.4656221e-05
5,124	Approximation Schemes for Many-Objective Query Optimization	2014	SIGMOD	6.3567981e-05
6,344	Towards General and Efficient Online Tuning for Spark	2023	VLDB	5.9060457e-05
8,615	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB	5.4005602e-05
9,898	ContTune: Continuous Tuning by Conservative Bayesian Optimization for Distributed Stream Data Processing Systems	2023	VLDB	5.1997534e-05
9,902	UDAO: A Next-Generation Unified Data Analytics Optimizer	2019	VLDB	5.1997534e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	8,237	Meta-Dataflows: Efficient Exploratory Dataflow Jobs	2018	SIGMOD
2	8,615	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB
3	11,665	Klink: Progress-Aware Scheduling for Streaming Data Systems	2021	SIGMOD
4	4,049	Resource Elasticity for Large-Scale Machine Learning	2015	SIGMOD
5	9,898	ContTune: Continuous Tuning by Conservative Bayesian Optimization for Distributed Stream Data Processing Systems	2023	VLDB
6	5,171	An Efficient Transfer Learning Based Configuration Adviser for Database Tuning	2024	VLDB
7	1,422	State Management in Apache Flink: Consistent Stateful Distributed Stream Processing	2017	VLDB
8	8,248	Demonstrating PDSP-Bench: A Benchmarking System for Parallel and Distributed Stream Processing	2025	SIGMOD
9	6,344	Towards General and Efficient Online Tuning for Spark	2023	VLDB
10	7,194	Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning	2018	VLDB