Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking

Summary: OmniServe co-serves latency-sensitive and best-effort LLMs by piggybacking BE Attention on CPUs, asynchronously overlapping CPU/GPU streams to reduce interference. Dynamic batching sustains GPU utilization, improving LS SLO attainment 1.48× and BE throughput 9.85×. (summarized by gpt-5.6-luna on Jul 26 2026)

Paper ID: 7480
Venue: SIGMOD
Year: 2026
Pagerank: -
Overall Rank: 13,288 | 8.84%
DOI: 10.1145/3802107

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Zizhao Mo (University of Macau)
2. Junlin Chen (University of Macau)
3. Huanle Xu (University of Macau)
4. Chengzhong Xu (University of Macau)

BibTeX Citation

@inproceedings{mo_sigmod26,
        title = {{Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking}},
        author = {Mo, Zizhao and Chen, Junlin and Xu, Huanle and Xu, Chengzhong},
        series = {{SIGMOD} '26},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3802107},
        url = {https://dl.acm.org/doi/10.1145/3802107},
        year = {2026}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 0 of 0 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	10,219	DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization	2026	SIGMOD
2	6,788	HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems	2026	SIGMOD
3	10,259	KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference	2026	SIGMOD
4	13,343	Database Perspective on LLM Inference Systems	2025	VLDB
5	10,420	SG-Serve: Efficient Model Serving for Subgraph-based Graph Representation Learning	2026	SIGMOD
6	10,228	Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective	2026	SIGMOD
7	9,000	Optimizing Inference Serving on Serverless Platforms	2022	VLDB
8	13,291	OrbitFlow: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfiguration	2026	VLDB
9	10,191	AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System	2026	SIGMOD
10	6,900	Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving	2025	SIGMOD