mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs

Summary: mLoRA: LoRA-aware pipeline parallelism and a LoRA-efficient operator to parallelize multiple LoRA adapter fine-tuning across GPUs/machines, reducing communication and improving GPU utilization. Cuts average fine-tuning time ~30% vs FSDP and enables larger models on fewer GPUs. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13850
Venue: VLDB
Year: 2025
Pagerank: 4.4937074e-05
Overall Rank: 8,520 | 40.73%
DOI: 10.14778/3725688.3725718

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,626	LobRA: Multi-tenant Fine-tuning over Heterogeneous Data	2025	VLDB	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
369	Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation	2024	VLDB	0.0002547515
411	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB	0.00023906921
1,116	Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes	2024	VLDB	0.00013890154
2,902	PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel	2023	VLDB	7.93939e-05
3,114	GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization	2024	VLDB	7.5451724e-05
5,099	ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models	2024	VLDB	5.6997784e-05
7,152	Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity	2024	VLDB	4.8154191e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
7,152	Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity	2024	VLDB	4.8154191e-05
6,377	Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism	2023	VLDB	5.0911095e-05
9,264	Model-Parallel Model Selection for Deep Learning Systems	2021	SIGMOD	4.3675421e-05
10,122	TranSQL+: Serving Large Language Models with SQL on Low-Resource Hardware	2026	SIGMOD	4.1945683e-05
2,902	PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel	2023	VLDB	7.93939e-05
10,492	Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization	2025	SIGMOD	4.1945683e-05
13,138	Database Perspective on LLM Inference Systems	2025	VLDB	-
10,316	LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning	2026	VLDB	4.1945683e-05
9,805	MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training	2025	SIGMOD	4.2805224e-05
10,626	LobRA: Multi-tenant Fine-tuning over Heterogeneous Data	2025	VLDB	4.1945683e-05