Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent
Summary: Angel-PTM is a scalable Transformer pre-training system using page-granular hierarchical-memory management and unified compute/data/communication scheduling. SSD-backed model scaling with lock-free updates enables efficient GPT-3 175B and T5-MoE 1.2T training. (summarized by gpt-5.6-luna on Jul 24 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Xiaonan Nie (Peking University)
- 2. Yi Liu (Tencent)
- 3. Fangcheng Fu (Peking University)
- 4. Jinbao Xue (Tencent)
- 5. Dian Jiao (Tencent)
- 6. Xupeng Miao (Carnegie Mellon University)
- 7. Yangyu Tao (Tencent)
- 8. Bin Cui (Peking University; Peking University (Qingdao))
BibTeX Citation
@article{nie_vldb23,
title = {{Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent}},
author = {Nie, Xiaonan and Liu, Yi and Fu, Fangcheng and Xue, Jinbao and Jiao, Dian and Miao, Xupeng and Tao, Yangyu and Cui, Bin},
journal = {PVLDB},
series = {{VLDB} '23},
volume = {16},
number = {12},
pages = {3781--3794},
doi = {10.14778/3611540.3611564},
url = {https://doi.org/10.14778/3611540.3611564},
year = {2023}
}Incoming Citations (Sorted by Pagerank)
Showing 5 of 5 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 6,954 | DLRover-RM: Resource Optimization for Deep Recommendation Models Training in the Cloud | 2024 | VLDB | 5.7303405e-05 |
| 8,101 | SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training | 2023 | VLDB | 5.4870581e-05 |
| 8,883 | FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement | 2023 | SIGMOD | 5.351513e-05 |
| 10,769 | Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization | 2025 | SIGMOD | 5.093636e-05 |
| 10,880 | LobRA: Multi-tenant Fine-tuning over Heterogeneous Data | 2025 | VLDB | 5.093636e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 8 of 8 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 1,959 | VF2Boost: Very Fast Vertical Federated Gradient Boosting for Cross-Enterprise Learning | 2021 | SIGMOD | 9.4090198e-05 |
| 2,485 | HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework | 2022 | VLDB | 8.5145736e-05 |
| 2,504 | Concurrent Analytical Query Processing with GPUs | 2014 | VLDB | 8.4963369e-05 |
| 4,083 | SketchML: Accelerating Distributed Machine Learning with Data Sketches | 2018 | SIGMOD | 6.9160949e-05 |
| 4,956 | Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce | 2021 | SIGMOD | 6.4290135e-05 |
| 5,003 | Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism | 2023 | VLDB | 6.4065691e-05 |
| 8,883 | FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement | 2023 | SIGMOD | 5.351513e-05 |
| 10,114 | Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates | 2022 | VLDB | 5.1319012e-05 |
Previous
Page 1 / 1
Next