LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning

Summary: LEAD performs in-loop iterative data selection for instruction tuning, avoiding costly full-dataset inference by estimating sample utility via Instance-Level Dynamic Uncertainty (IDU): instantaneous loss, gradient-based loss-change approximation, and exponential smoothing. A two-stage coarse-to-fine pipeline (MAB cluster prioritization + IDU fine selection) yields ~6–11% avg gains using 2.5% of data and 5–10× faster training. (summarized by gpt-5-mini on Mar 13 2026)

Paper ID: h0d7c3705b86b574e
Venue: VLDB
Year: 2026
Pagerank: 5.601767e-05
Overall Rank: 7,090 | 52.34%
DOI: 10.14778/3778092.3778103

Incoming Non-self Citations Over Time

Authors

1. Xiaotian Lin (Hong Kong University of Science and Technology)
2. Yanlin Qi (Université Paris Cité)
3. Yizhang Zhu (Hong Kong University of Science and Technology)
4. Themis Palpanas (Université Paris Cité)
5. Chengliang Chai (Beijing Institute of Technology)
6. Nan Tang (Hong Kong University of Science and Technology)
7. Yuyu Luo (Hong Kong University of Science and Technology)

BibTeX Citation

@article{lin_vldb26,
        title = {{LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning}},
        author = {Lin, Xiaotian and Qi, Yanlin and Zhu, Yizhang and Palpanas, Themis and Chai, Chengliang and Tang, Nan and Luo, Yuyu},
        journal = {PVLDB},
        series = {{VLDB} '26},
        volume = {19},
        number = {3},
        pages = {426--439},
        doi = {10.14778/3778092.3778103},
        url = {https://doi.org/10.14778/3778092.3778103},
        year = {2026}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,821	Data-efficient Online Training for Direct Alignment in LLMs	2026	VLDB	4.9793485e-05
11,002	Demonstrating TOFFEE: A Learned System for Synthesizing Data Agent Trajectories at Scale	2026	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
205	Snorkel: Rapid Training Data Creation with Weak Supervision	2018	VLDB	0.00025181304
3,941	GoodCore: Data-effective and Data-efficient Machine Learning through Coreset Selection over Incomplete Data	2023	SIGMOD	6.9138042e-05
4,864	Synthesizing Natural Language to Visualization (NL2VIS) Benchmarks from NL2SQL Benchmarks	2021	SIGMOD	6.3770469e-05
4,867	Selective Data Acquisition in the Wild for Model Charging	2022	VLDB	6.3749346e-05
7,042	Human-in-the-loop Outlier Detection	2020	SIGMOD	5.6142998e-05
7,258	Learned Data-aware Image Representations of Line Charts for Similarity Search	2023	SIGMOD	5.572114e-05
7,583	Data Imputation with Limited Data Redundancy Using Data Lakes	2025	VLDB	5.4906208e-05
9,210	Natural Language to SQL: State of the Art and Open Problems	2025	VLDB	5.2061511e-05
9,470	VisClean: Interactive Cleaning for Progressive Visualization	2020	VLDB	5.1714419e-05
10,433	DeepEye-SQL: A Software-Engineering-Inspired Text-to-SQL Framework	2026	SIGMOD	4.9793485e-05
11,270	Weak-to-Strong Prompts with Lightweight-to-Powerful LLMs for High-Accuracy, Low-Cost, and Explainable Data Transformation	2025	VLDB	4.9793485e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	10,488	OctoSelector: Efficient and Effective Batch-Aware Model Selection for Large Language Models	2026	SIGMOD
2	11,161	ScaleLLM: A Technique for Scalable LLM-augmented Data Systems	2025	SIGMOD
3	10,435	DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization	2026	SIGMOD
4	8,997	Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees	2026	SIGMOD
5	10,838	Unified Static–Dynamic Pruning for Efficient LLM Inference	2026	VLDB
6	8,888	Optimized Batch Prompting for Cost-effective LLMs	2025	VLDB
7	11,767	Self-Training for Label-Efficient Information Extraction from Semi-Structured Web-Pages	2023	VLDB
8	11,061	LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning	2026	VLDB
9	10,821	Data-efficient Online Training for Direct Alignment in LLMs	2026	VLDB
10	10,711	BRIEF: Bi-level Coreset Selection for Efficient Instruction Tuning in LLMs	2026	VLDB