Buoy: Efficient and Effective Cache Replacement for Prefix Caching

Summary: Buoy replaces LRU with a prefix- and tier-aware policy for LLM KV-cache management, combining quick demotion with prefix-matched access characteristics to improve hit rates and cache-hit latency. Real-world traces show 6–93% higher hit rates and 8–16% greater prefill throughput than LRU. (summarized by gpt-6-luna on Oct 22 2026)

Paper ID: h5172e27d5ac9574b
Venue: SIGMOD
Year: 2027
Pagerank: 4.9605343e-05
Overall Rank: 10,386 | 30.39%
DOI: 10.1145/3837101
PDF: Download (CC BY 4.0)

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Liang Wang (Huazhong University of Science and Technology)
2. Ranjun Jia (Huazhong University of Science and Technology)
3. Kai Wang (Huazhong University of Science and Technology)
4. Kai Lu (Huazhong University of Science and Technology)
5. Jiguang Wan (Huazhong University of Science and Technology)

BibTeX Citation

@inproceedings{wang_sigmod27,
        title = {{Buoy: Efficient and Effective Cache Replacement for Prefix Caching}},
        author = {Wang, Liang and Jia, Ranjun and Wang, Kai and Lu, Kai and Wan, Jiguang},
        series = {{SIGMOD} '27},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3837101},
        url = {https://dl.acm.org/doi/10.1145/3837101},
        year = {2027}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
207	The LRU-K Page Replacement Algorithm For Database Disk Buffering	1993	SIGMOD	0.00024963419
496	Adaptive Self-Tuning Memory in DB2	2006	VLDB	0.00017347275
572	2Q: A Low Overhead High Performance Buffer Management Replacement Algorithm	1994	VLDB	0.00016154673
2,197	Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation	2025	SIGMOD	8.8751404e-05
3,844	HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems	2026	SIGMOD	6.9683696e-05
3,972	PQCache: Product Quantization-based KVCache for Long Context LLM Inference	2025	SIGMOD	6.8744954e-05
6,944	DepCache: A KV Cache Management Framework for GraphRAG with Dependency Attention	2026	SIGMOD	5.6219388e-05
9,100	NeutronRAG: Towards Understanding the Effectiveness of RAG from a Data Retrieval Perspective	2025	SIGMOD	5.208561e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,910	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD
2	9,844	ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models	2025	VLDB
3	6,944	DepCache: A KV Cache Management Framework for GraphRAG with Dependency Attention	2026	SIGMOD
4	10,718	From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generation	2026	SIGMOD
5	10,847	Efficient Cooperation-Aware Key and Value Management for LLM Inference	2026	VLDB
6	10,549	KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference	2026	SIGMOD
7	3,972	PQCache: Product Quantization-based KVCache for Long Context LLM Inference	2025	SIGMOD
8	10,485	AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System	2026	SIGMOD
9	2,197	Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation	2025	SIGMOD
10	3,844	HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems	2026	SIGMOD