mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs

Summary: mLoRA: LoRA-aware pipeline parallelism and a LoRA-efficient operator to parallelize multiple LoRA adapter fine-tuning across GPUs/machines, reducing communication and improving GPU utilization. Cuts average fine-tuning time ~30% vs FSDP and enables larger models on fewer GPUs. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13851
Venue: VLDB
Year: 2025
Pagerank: 4.4893996e-05
Overall Rank: 8,518 | 40.80%
DOI: 10.14778/3725688.3725718

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,634	LobRA: Multi-tenant Fine-tuning over Heterogeneous Data	2025	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
366	Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation	2024	VLDB	0.00025580097
411	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB	0.00023881138
1,088	Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes	2024	VLDB	0.00014158762
2,907	PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel	2023	VLDB	7.9322286e-05
3,105	GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization	2024	VLDB	7.5567226e-05
5,098	ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models	2024	VLDB	5.6943033e-05
7,143	Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity	2024	VLDB	4.8143774e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
7,143	Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity	2024	VLDB	4.8143774e-05
6,361	Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism	2023	VLDB	5.0903244e-05
9,269	Model-Parallel Model Selection for Deep Learning Systems	2021	SIGMOD	4.3633561e-05
10,122	TranSQL+: Serving Large Language Models with SQL on Low-Resource Hardware	2026	SIGMOD	4.1905499e-05
2,907	PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel	2023	VLDB	7.9322286e-05
10,502	Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization	2025	SIGMOD	4.1905499e-05
13,152	Database Perspective on LLM Inference Systems	2025	VLDB	-
10,328	LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning	2026	VLDB	4.1905499e-05
9,786	MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training	2025	SIGMOD	4.2799988e-05
10,634	LobRA: Multi-tenant Fine-tuning over Heterogeneous Data	2025	VLDB	4.1905499e-05