DBScholar

Back to papers

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving

Summary: Hybrid KV+hidden-state cache expands batch size under GPU memory limits for LLM inference. Adaptive scheduling with formal optimization and guarantees tunes batch composition, yielding up to 8.8× throughput vs SOTA on 13B–66B models. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h01d34dd363184bc4
Venue
SIGMOD
Year
2025
Pagerank
6.3468554e-05
Overall Rank
4,910 | 67.09%
DOI
10.1145/3725394

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{gao_sigmod25,
        title = {{Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving}},
        author = {Gao, Shihong and Zhang, Xin and Shen, Yanyan and Chen, Lei},
        series = {{SIGMOD} '25},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3725394},
        url = {https://dl.acm.org/doi/10.1145/3725394},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 4 of 4 citing papers.

Rank Citing Paper Year Venue Pagerank
5,376 RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference 2026 VLDB 6.139877e-05
10,410 StaleFlow: Staleness-Aware Data Management for Mitigating Data Skewness in Fully Disaggregated RL Post-Training 2027 SIGMOD 4.9605343e-05
10,485 AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System 2026 SIGMOD 4.9605343e-05
10,847 Efficient Cooperation-Aware Key and Value Management for LLM Inference 2026 VLDB 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 35 of 35 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
278 Accelerating Machine Learning Inference with Probabilistic Predicates 2018 SIGMOD 0.00022277737
1,047 SANCUS: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks 2022 VLDB 0.00012313386
1,472 Analyzing and Mitigating Data Stalls in DNN Training 2021 VLDB 0.00010529036
2,434 Accelerating Large Scale Real-Time GNN Inference using Channel Pruning 2021 VLDB 8.4608438e-05
2,513 DUCATI: A Dual-Cache Training System for Graph Neural Networks on Giant Graphs with the GPU 2023 SIGMOD 8.3467757e-05
2,525 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.3302991e-05
2,913 AI Meets Database: AI4DB and DB4AI 2021 SIGMOD 7.855786e-05
3,027 Zebra: When Temporal Graph Neural Networks Meet Temporal Personalized PageRank 2023 VLDB 7.7178623e-05
3,224 Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics 2021 VLDB 7.503511e-05
3,411 Orca: Scalable Temporal Graph Neural Network Training with Theoretical Guarantees 2023 SIGMOD 7.3074792e-05
3,598 Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines 2022 SIGMOD 7.1596688e-05
3,800 FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline 2023 VLDB 6.9931619e-05
3,966 Optimizing Machine Learning Inference Queries with Correlative Proxy Models 2022 VLDB 6.8776157e-05
3,972 PQCache: Product Quantization-based KVCache for Long Context LLM Inference 2025 SIGMOD 6.8744954e-05
4,096 Self-Tuning Query Scheduling for Analytical Workloads 2021 SIGMOD 6.7949494e-05
4,360 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity 2024 VLDB 6.6274323e-05
4,785 ETC: Efficient Training of Temporal Graph Neural Networks over Large-scale Dynamic Graphs 2024 VLDB 6.4033344e-05
4,850 DAHA: Accelerating GNN Training with Data and Hardware Aware Execution Planning 2024 VLDB 6.3682022e-05
4,961 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD 6.3246342e-05
4,974 HET-GMP: A Graph-based System Approach to Scaling Large Embedding Model Training 2022 SIGMOD 6.3196773e-05
5,011 Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism 2023 VLDB 6.3030066e-05
6,339 EARLY: Efficient and Reliable Graph Neural Network for Dynamic Graphs 2023 SIGMOD 5.7918126e-05
6,841 Distribution-Based Query Scheduling 2013 VLDB 5.6512839e-05
6,849 SIMPLE: Efficient Temporal Graph Neural Network Training at Scale with Dynamic Data Placement 2024 SIGMOD 5.6488938e-05
6,978 Towards Optimal Transaction Scheduling 2024 VLDB 5.6130331e-05
7,262 Transaction Scheduling: From Conflicts to Runtime Conflicts 2023 SIGMOD 5.5560872e-05
7,950 Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines 2024 VLDB 5.402957e-05
8,228 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.3581744e-05
9,486 MorphStream: Adaptive Scheduling for Scalable Transactional Stream Processing on Multicores 2023 SIGMOD 5.1562396e-05
10,015 Capsule*: An Out-of-Core Training Mechanism for Colossal GNNs 2025 SIGMOD 5.0825892e-05
10,016 MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training 2025 SIGMOD 5.0825892e-05
10,017 The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format 2024 SIGMOD 5.0825892e-05
10,018 Demonstration of Accelerating Machine Learning Inference Queries with Correlative Proxy Models 2022 VLDB 5.0825892e-05
10,054 ETO: Accelerating Optimization of DNN Operators by High-Performance Tensor Program Reuse 2022 VLDB 5.0779982e-05
13,716 STile: Searching Hybrid Sparse Formats for Sparse Deep Learning Operators Automatically 2024 SIGMOD -
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 10,909 Unified Static–Dynamic Pruning for Efficient LLM Inference 2026 VLDB
2 3,972 PQCache: Product Quantization-based KVCache for Long Context LLM Inference 2025 SIGMOD
3 10,549 KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference 2026 SIGMOD
4 10,847 Efficient Cooperation-Aware Key and Value Management for LLM Inference 2026 VLDB
5 3,844 HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems 2026 SIGMOD
6 9,196 Optimizing Inference Serving on Serverless Platforms 2022 VLDB
7 9,834 Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective 2026 SIGMOD
8 13,628 OrbitFlow: SLO-Aware Long-Context LLM Serving with Fine-Grained KV Cache Reconfiguration 2026 VLDB
9 13,625 Serving Hybrid LLM Loads with SLO Guarantees Using CPU-GPU Attention Piggybacking 2026 SIGMOD
10 10,485 AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System 2026 SIGMOD