LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning
Summary: LEAD performs in-loop iterative data selection for instruction tuning, avoiding costly full-dataset inference by estimating sample utility via Instance-Level Dynamic Uncertainty (IDU): instantaneous loss, gradient-based loss-change approximation, and exponential smoothing. A two-stage coarse-to-fine pipeline (MAB cluster prioritization + IDU fine selection) yields ~6–11% avg gains using 2.5% of data and 5–10× faster training. (summarized by gpt-5-mini on Mar 13 2026)
Incoming Non-self Citations Over Time
No non-self incoming citations found for this paper in this database.
Authors
- 1. Xiaotian Lin (Hong Kong University of Science and Technology)
- 2. Yanlin Qi (Université Paris Cité)
- 3. Yizhang Zhu (Hong Kong University of Science and Technology)
- 4. Themis Palpanas (Université Paris Cité)
- 5. Chengliang Chai (Beijing Institute of Technology)
- 6. Nan Tang (Hong Kong University of Science and Technology)
- 7. Yuyu Luo (Hong Kong University of Science and Technology)
BibTeX Citation
@article{lin_vldb26,
title = {{LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning}},
author = {Lin, Xiaotian and Qi, Yanlin and Zhu, Yizhang and Palpanas, Themis and Chai, Chengliang and Tang, Nan and Luo, Yuyu},
journal = {PVLDB},
series = {{VLDB} '26},
volume = {19},
number = {3},
pages = {426--439},
doi = {10.14778/3778092.3778103},
url = {https://doi.org/10.14778/3778092.3778103},
year = {2026}
}Incoming Citations (Sorted by Pagerank)
Showing 0 of 0 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|
Outgoing Citations (Sorted by Pagerank)
Showing 11 of 11 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 10,132 | Making Prompts First-Class Citizens for Adaptive LLM Pipelines | 2026 | CIDR |
| 2 | 6,101 | LLM for Data Management | 2024 | VLDB |
| 3 | 10,276 | OctoSelector: Efficient and Effective Batch-Aware Model Selection for Large Language Models | 2026 | SIGMOD |
| 4 | 10,733 | ScaleLLM: A Technique for Scalable LLM-augmented Data Systems | 2025 | SIGMOD |
| 5 | 10,219 | DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization | 2026 | SIGMOD |
| 6 | 8,829 | Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees | 2026 | SIGMOD |
| 7 | 10,856 | Optimized Batch Prompting for Cost-effective LLMs | 2025 | VLDB |
| 8 | 11,455 | Self-Training for Label-Efficient Information Extraction from Semi-Structured Web-Pages | 2023 | VLDB |
| 9 | 10,614 | LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning | 2026 | VLDB |
| 10 | 10,527 | BRIEF: Bi-level Coreset Selection for Efficient Instruction Tuning in LLMs | 2026 | VLDB |