mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs
Summary: mLoRA: LoRA-aware pipeline parallelism and a LoRA-efficient operator to parallelize multiple LoRA adapter fine-tuning across GPUs/machines, reducing communication and improving GPU utilization. Cuts average fine-tuning time ~30% vs FSDP and enables larger models on fewer GPUs. (summarized by gpt-5-mini on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Zhengmao Ye (Sichuan University)
- 2. Dengchun Li (Sichuan University)
- 3. Zetao Hu (Sichuan University)
- 4. Tingfeng Lan (Sichuan University)
- 5. Jian Sha (Ant Financial)
- 6. Shicong Zhang (Zhejiang New Internet Exchange Center)
- 7. Lei Duan (Sichuan University)
- 8. Jie Zuo (Sichuan University)
- 9. Hui Lu (University of Texas)
- 10. Yuanchun Zhou (Chinese Academy of Sciences)
- 11. Mingjie Tang (Sichuan University)
BibTeX Citation
@article{ye_vldb25,
title = {{mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs}},
author = {Ye, Zhengmao and Li, Dengchun and Hu, Zetao and Lan, Tingfeng and Sha, Jian and Zhang, Shicong and Duan, Lei and Zuo, Jie and Lu, Hui and Zhou, Yuanchun and Tang, Mingjie},
journal = {PVLDB},
series = {{VLDB} '25},
volume = {18},
number = {6},
pages = {1948--1961},
doi = {10.14778/3725688.3725718},
url = {https://doi.org/10.14778/3725688.3725718},
year = {2025}
}Incoming Citations (Sorted by Pagerank)
Showing 1 of 1 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 11,282 | LobRA: Multi-tenant Fine-tuning over Heterogeneous Data | 2025 | VLDB | 4.9793485e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 7 of 7 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 174 | Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation | 2024 | VLDB | 0.00026790979 |
| 501 | Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes | 2024 | VLDB | 0.00017267905 |
| 522 | PyTorch Distributed: Experiences on Accelerating Data Parallel Training | 2020 | VLDB | 0.00016912378 |
| 2,231 | GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization | 2024 | VLDB | 8.7982985e-05 |
| 2,246 | PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel | 2023 | VLDB | 8.7637336e-05 |
| 4,351 | Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity | 2024 | VLDB | 6.642803e-05 |
| 4,589 | ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models | 2024 | VLDB | 6.5144711e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 13,578 | Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking | 2026 | SIGMOD |
| 2 | 2,246 | PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel | 2023 | VLDB |
| 3 | 11,189 | Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization | 2025 | SIGMOD |
| 4 | 9,158 | Database Perspective on LLM Inference Systems | 2025 | VLDB |
| 5 | 11,061 | LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning | 2026 | VLDB |
| 6 | 10,838 | Unified Static–Dynamic Pruning for Efficient LLM Inference | 2026 | VLDB |
| 7 | 10,042 | MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training | 2025 | SIGMOD |
| 8 | 10,757 | Mil: Cost-guided Minimum Makespan Scheduling for Applications of Multiple LLMs | 2026 | VLDB |
| 9 | 10,435 | DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization | 2026 | SIGMOD |
| 10 | 11,282 | LobRA: Multi-tenant Fine-tuning over Heterogeneous Data | 2025 | VLDB |