HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems

Summary: HotPrefix: hotness-aware KV-cache scheduler that tracks long-tail prefix reuse and pins hot prefixes in GPU HBM while offloading cold prefixes to CPU RAM. Overlaps KV transfers with computation to avoid redundant KV recomputation, cut HBM pressure, and boost latency/throughput for shared-prefix LLM inference. (summarized by gpt-5-mini on Feb 11 2026)

Paper ID: 7515
Venue: SIGMOD
Year: 2026
Pagerank: 5.7727874e-05
Overall Rank: 6,788 | 53.43%
DOI: 10.1145/3749168

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{li_sigmod26,
        title = {{HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems}},
        author = {Li, Yuhang and Gu, Rong and Huan, Chengying and Wang, Zhibin and Yao, Renjie and Tian, Chen and Chen, Guihai},
        series = {{SIGMOD} '26},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3749168},
        url = {https://dl.acm.org/doi/10.1145/3749168},
        year = {2026}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,191	AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System	2026	SIGMOD	5.093636e-05
10,210	CoDec: Prefix-Shared Decoding Kernel for LLMs	2026	SIGMOD	5.093636e-05
10,225	Dynamic Flat Filter: A Unified Framework for Scalable and Stable Fingerprint-Based Filters	2026	SIGMOD	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
637	2Q: A Low Overhead High Performance Buffer Management Replacement Algorithm	1994	VLDB	0.0001547341

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	13,291	OrbitFlow: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfiguration	2026	VLDB
2	6,432	RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference	2026	VLDB
3	10,459	From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generation	2026	SIGMOD
4	10,357	DepCache: A KV Cache Management Framework for GraphRAG with Dependency Attention	2026	SIGMOD
5	10,210	CoDec: Prefix-Shared Decoding Kernel for LLMs	2026	SIGMOD
6	6,900	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD
7	2,816	Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation	2025	SIGMOD
8	4,430	PQCache: Product Quantization-based KVCache for Long Context LLM Inference	2025	SIGMOD
9	10,191	AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System	2026	SIGMOD
10	10,259	KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference	2026	SIGMOD