DBScholar

Back to papers

MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training

Summary: Memo enables ultra-long context LLM training via fine-grained activation memory management: offloads activations to CPU after each layer and fetches them in backprop with token-wise recomputation. Bi-level MIP optimizes cross-layer memory reuse to curb fragmentation and communication, delivering MFU gains over Megatron-LM and DeepSpeed. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
7110
Venue
SIGMOD
Year
2025
Pagerank
5.2040783e-05
Overall Rank
9,880 | 32.22%
DOI
10.1145/3709703

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{zhao_sigmod25,
        title = {{MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training}},
        author = {Zhao, Pinxue and Zhang, Hailin and Fu, Fangcheng and Nie, Xiaonan and Liu, Qibin and Yang, Fang and Peng, Yuanbo and Jiao, Dian and Li, Shuaipeng and Xue, Jinbao and Tao, Yangyu and Cui, Bin},
        series = {{SIGMOD} '25},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3709703},
        url = {https://dl.acm.org/doi/10.1145/3709703},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank Citing Paper Year Venue Pagerank
6,900 Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving 2025 SIGMOD 5.7430032e-05
10,380 Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment 2026 SIGMOD 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 20 of 20 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1,446 Analyzing and Mitigating Data Stalls in DNN Training 2021 VLDB 0.0001076818
2,018 tf.data: A Machine Learning Data Processing Framework 2021 VLDB 9.3001937e-05
2,473 PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel 2023 VLDB 8.5326287e-05
2,485 HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework 2022 VLDB 8.5145736e-05
2,504 Concurrent Analytical Query Processing with GPUs 2014 VLDB 8.4963369e-05
3,134 Efficient Join Algorithms For Large Database Tables in a Multi-GPU Environment 2021 VLDB 7.7231028e-05
3,530 MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud 2023 VLDB 7.3379782e-05
3,541 Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines 2022 SIGMOD 7.3280673e-05
3,634 Orca: Scalable Temporal Graph Neural Network Training with Theoretical Guarantees 2023 SIGMOD 7.2358691e-05
3,764 FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline 2023 VLDB 7.1446931e-05
3,874 Tensors: An abstraction for general data processing 2021 VLDB 7.0561161e-05
4,647 Memory Management Techniques for Large-Scale Persistent-Main-Memory Systems 2017 VLDB 6.5879506e-05
5,003 Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism 2023 VLDB 6.4065691e-05
5,195 Tensor Relational Algebra for Distributed Machine Learning System Design 2021 VLDB 6.3232105e-05
5,414 Optimizing Tensor Programs on Flexible Storage 2023 SIGMOD 6.2258658e-05
6,730 SIMPLE: Efficient Temporal Graph Neural Network Training at Scale with Dynamic Data Placement 2024 SIGMOD 5.7895038e-05
8,101 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB 5.4870581e-05
8,230 TOD: GPU-accelerated Outlier Detection via Tensor Operations 2023 VLDB 5.4619615e-05
9,520 Experimental Analysis of Large-scale Learnable Vector Storage Compression 2024 VLDB 5.2561283e-05
9,556 CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models 2024 SIGMOD 5.2528121e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 2,816 Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation 2025 SIGMOD
2 4,430 PQCache: Product Quantization-based KVCache for Long Context LLM Inference 2025 SIGMOD
3 10,432 Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management 2026 SIGMOD
4 8,725 mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs 2025 VLDB
5 10,259 KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference 2026 SIGMOD
6 10,024 Near-Duplicate Sequence Search at Scale for Large Language Model Memorization Evaluation 2023 SIGMOD
7 6,432 RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference 2026 VLDB
8 9,219 MemFlow: Memory-Aware Distributed Deep Learning 2020 SIGMOD
9 10,219 DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization 2026 SIGMOD
10 7,075 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity 2024 VLDB