DBScholar

Back to papers

MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud

Summary: MiCS shrinks communication collectives to exploit heterogeneous cloud links, cut slow-link traffic/latency, and amortize global synchronization. On AWS, it reaches 2.89× state-of-the-art throughput and trains a 100B-parameter model on 512 GPUs with 99.4% weak scaling. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
13387
Venue
VLDB
Year
2023
Pagerank
7.3379782e-05
Overall Rank
3,530 | 75.79%
DOI
10.14778/3561261.3561265

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{zhang_vldb23,
        title = {{MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud}},
        author = {Zhang, Zhen and Zheng, Shuai and Wang, Yida and Chiu, Justin and Karypis, George and Chilimbi, Trishul and Li, Mu and Jin, Xin},
        journal = {PVLDB},
        series = {{VLDB} '23},
        volume = {16},
        number = {1},
        pages = {37--50},
        doi = {10.14778/3561261.3561265},
        url = {https://doi.org/10.14778/3561261.3561265},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank Citing Paper Year Venue Pagerank
2,473 PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel 2023 VLDB 8.5326287e-05
3,536 How Large Language Models Will Disrupt Data Management 2023 VLDB 7.3297343e-05
7,075 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity 2024 VLDB 5.7099047e-05
9,880 MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training 2025 SIGMOD 5.2040783e-05
10,880 LobRA: Multi-tenant Fine-tuning over Heterogeneous Data 2025 VLDB 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 0 of 0 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 10,596 NeutronCloud: Resource-Aware Distributed GNN Training in Fluctuating Cloud Environments 2026 VLDB
2 4,956 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD
3 8,101 SDP_PIPE: A Semi-Decentralized Framework for Heterogeneity-aware Pipeline-parallel Training 2023 VLDB
4 3,764 FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline 2023 VLDB
5 10,583 Resilience-Aware Elastic Scaling for Cloud-Native Online DL Training on Multi-Tenant GPU Clusters 2026 VLDB
6 2,640 Scalable and Efficient Full-Graph GNN Training for Large Graphs 2023 SIGMOD
7 1,234 Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture 2021 VLDB
8 3,782 CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers 2019 VLDB
9 9,729 Scalable Graph Convolutional Network Training on Distributed-Memory Systems 2023 VLDB
10 9,469 How Can We Train Deep Learning Models Across Clouds and Continents? An Experimental Study 2024 VLDB