AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System

Summary: AlignedServe introduces prefix-aware batching that groups requests by KV-cache length, eliminating intra-iteration decode bubbles. Its CPU-buffered orchestration, batch scheduling, and novel GPU-to-GPU KV-cache prefetching improve LLM throughput up to 1.98× and reduce latency 7.4×. (summarized by gpt-5.6-luna on Jul 26 2026)

Paper ID: 7382
Venue: SIGMOD
Year: 2026
Pagerank: 5.093636e-05
Overall Rank: 10,191 | 30.09%
DOI: 10.1145/3802009

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Fengyao Bai (Sun Yat-Sen University)
2. Hongbin Zhang (Sun Yat-Sen University)
3. Zhitao Chen (Sun Yat-Sen University)
4. Jiangsu Du (Sun Yat-Sen University)
5. Zhiguang Chen (Sun Yat-Sen University)
6. Yutong Lu (Sun Yat-Sen University)

BibTeX Citation

@inproceedings{bai_sigmod26,
        title = {{AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System}},
        author = {Bai, Fengyao and Zhang, Hongbin and Chen, Zhitao and Du, Jiangsu and Chen, Zhiguang and Lu, Yutong},
        series = {{SIGMOD} '26},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3802009},
        url = {https://dl.acm.org/doi/10.1145/3802009},
        year = {2026}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
2,816	Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation	2025	SIGMOD	8.0959781e-05
6,788	HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems	2026	SIGMOD	5.7727874e-05
6,900	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD	5.7430032e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	13,343	Database Perspective on LLM Inference Systems	2025	VLDB
2	2,816	Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation	2025	SIGMOD
3	10,228	Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective	2026	SIGMOD
4	13,291	OrbitFlow: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfiguration	2026	VLDB
5	10,259	KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference	2026	SIGMOD
6	4,430	PQCache: Product Quantization-based KVCache for Long Context LLM Inference	2025	SIGMOD
7	9,000	Optimizing Inference Serving on Serverless Platforms	2022	VLDB
8	6,788	HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems	2026	SIGMOD
9	13,288	Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking	2026	SIGMOD
10	6,900	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD