A Systematic Study on Early Stopping Metrics in HPO and the Implications of Uncertainty

Summary: Systematic study of early-stopping metric choice in HPO/NAS: using training loss (vs. validation loss) in early stages boosts HPO outcomes up to 24.76%. Introduce uncertainty-aware metrics that add up to ~4% extra gain under budget constraints, improving reliability and resource efficiency for scalable early-stopping. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 14006
Venue: VLDB
Year: 2025
Pagerank: 5.093636e-05
Overall Rank: 10,827 | 25.72%
DOI: 10.14778/3725688.3725689

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Jiawei Guan (Renmin University of China)
2. Feng Zhang (Renmin University of China)
3. Xiaoyong Du (Renmin University of China)
4. Jiesong Liu (North Carolina State University)
5. Xipeng Shen (North Carolina State University)

BibTeX Citation

@article{guan_vldb25,
        title = {{A Systematic Study on Early Stopping Metrics in HPO and the Implications of Uncertainty}},
        author = {Guan, Jiawei and Zhang, Feng and Du, Xiaoyong and Liu, Jiesong and Shen, Xipeng},
        journal = {PVLDB},
        series = {{VLDB} '25},
        volume = {18},
        number = {6},
        pages = {1551--1564},
        doi = {10.14778/3725688.3725689},
        url = {https://doi.org/10.14778/3725688.3725689},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 15 of 15 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
26	Models and Issues in Data Stream Systems	2002	PODS	0.00052982574
112	HoloClean: Holistic Data Repairs with Probabilistic Inference	2017	VLDB	0.00032801121
205	Snorkel: Rapid Training Data Creation with Weak Supervision	2018	VLDB	0.00025235185
498	QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning	2019	VLDB	0.00017440583
1,004	Democratizing Data Science through Interactive Curation of ML Pipelines	2019	SIGMOD	0.00012701932
1,756	SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle	2020	CIDR	9.8172465e-05
2,740	Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation	2022	VLDB	8.1855759e-05
3,116	ResTune: Resource Oriented Tuning Boosted by Meta-Learning for Cloud Databases	2021	SIGMOD	7.7390737e-05
3,272	VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition	2021	VLDB	7.5775321e-05
3,587	HUNTER: An Online Cloud Database Hybrid Tuning System for Personalized Requirements	2022	SIGMOD	7.2829345e-05
3,950	MagicScaler: Uncertainty-aware, Predictive Autoscaling	2023	VLDB	6.9980354e-05
4,011	Towards Dynamic and Safe Configuration Tuning for Cloud Databases	2022	SIGMOD	6.959982e-05
4,118	Rafiki: Machine Learning as an Analytics Service System	2019	VLDB	6.8908973e-05
9,290	Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale	2022	VLDB	5.2910774e-05
11,425	Homomorphic Compression: Making Text Processing on Compression Unlimited	2023	SIGMOD	5.093636e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	8,741	Improvements on the k-center Problem for Uncertain Data	2018	PODS
2	2,762	Cost-based or Learning-based? A Hybrid Query Optimizer for Query Plan Selection	2022	VLDB
3	2,740	Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation	2022	VLDB
4	13,406	High Dimensional Differentially Private Stochastic Optimization with Heavy-tailed Data	2022	PODS
5	9,529	Hippo: Sharing Computations in Hyper-Parameter Optimization	2022	VLDB
6	4,799	Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models	2017	VLDB
7	7,000	A Cost-based Optimizer for Gradient Descent Optimization	2017	SIGMOD
8	10,453	ESTune: Bayesian Uncertainty-Guided Early Stopping for Database Configuration Tuning	2026	SIGMOD
9	8,646	Supporting User-Defined Functions on Uncertain Data	2013	VLDB
10	9,290	Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale	2022	VLDB