Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity

Summary: Flash-LLM makes unstructured sparsity efficient on restrictive GPU tensor cores via “Load-as-Sparse, Compute-as-Dense,” targeting skinny, bandwidth-bound generative-model multiplications. It delivers up to 3.8× higher inference throughput and substantially lower cost on OPT-30B–175B. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h3508effaa5e9dc1e
Venue: VLDB
Year: 2024
Pagerank: 6.642803e-05
Overall Rank: 4,351 | 70.75%
DOI: 10.14778/3626292.3626303

Incoming Non-self Citations Over Time

Authors

1. Haojun Xia (University of Sydney)
2. Zhen Zheng (Alibaba)
3. Yuchao Li (Alibaba)
4. Donglin Zhuang (University of Sydney)
5. Zhongzhu Zhou (University of Sydney)
6. Xiafei Qiu (Alibaba)
7. Yong Li (Alibaba)
8. Wei Lin (Alibaba)
9. Shuaiwen Leon Song (University of Sydney)

BibTeX Citation

@article{xia_vldb24,
        title = {{Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity}},
        author = {Xia, Haojun and Zheng, Zhen and Li, Yuchao and Zhuang, Donglin and Zhou, Zhongzhu and Qiu, Xiafei and Li, Yong and Lin, Wei and Song, Shuaiwen Leon},
        journal = {PVLDB},
        series = {{VLDB} '24},
        volume = {17},
        number = {2},
        pages = {211--224},
        doi = {10.14778/3626292.3626303},
        url = {https://doi.org/10.14778/3626292.3626303},
        year = {2024}
}

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
6,166	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD	5.8631131e-05
8,887	mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs	2025	VLDB	5.2559789e-05
10,818	ThunderGNN: Unlocking Tensor Cores for Graph Neural Networks	2026	VLDB	4.9793485e-05
10,838	Unified Static–Dynamic Pruning for Efficient LLM Inference	2026	VLDB	4.9793485e-05
11,189	Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization	2025	SIGMOD	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
329	Can Foundation Models Wrangle Your Data?	2023	VLDB	0.00020858443
522	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB	0.00016912378
1,740	TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data	2022	VLDB	9.7439205e-05
3,450	MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud	2023	VLDB	7.2921762e-05
3,764	CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers	2019	VLDB	7.0394265e-05
4,442	From BERT to GPT-3 Codex: Harnessing the Potential of Very Large Language Models for Data Management	2022	VLDB	6.5988336e-05
4,963	Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce	2021	SIGMOD	6.3384091e-05
5,009	Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism	2023	VLDB	6.3158866e-05
5,439	Optimizing Tensor Programs on Flexible Storage	2023	SIGMOD	6.1279464e-05
8,555	Harmony: Overcoming the Hurdles of GPU Memory Capacity to Train Massive DNN Models on Commodity Servers	2022	VLDB	5.3152366e-05
8,812	FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement	2023	SIGMOD	5.2722828e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	8,812	FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement	2023	SIGMOD
2	10,472	KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference	2026	SIGMOD
3	8,888	Optimized Batch Prompting for Cost-effective LLMs	2025	VLDB
4	6,860	SpareLLM: Automatically Selecting Task-Specific Minimum-Cost Large Language Models under Equivalence Constraint	2025	SIGMOD
5	9,158	Database Perspective on LLM Inference Systems	2025	VLDB
6	10,435	DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization	2026	SIGMOD
7	6,560	RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference	2026	VLDB
8	10,757	Mil: Cost-guided Minimum Makespan Scheduling for Applications of Multiple LLMs	2026	VLDB
9	10,042	MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training	2025	SIGMOD
10	10,838	Unified Static–Dynamic Pruning for Efficient LLM Inference	2026	VLDB