DBScholar

Back to papers

Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization

Summary: Malleus enables straggler-resilient hybrid training via per-GPU profiling and a planning algorithm that optimizes GPU groups, pipelines, layers, and data. It re-plans and migrates state on the fly to sustain stability, operating under dynamic straggler distributions and delivering 2.63–5.28x efficiency on LLMs up to 110B. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h7a4cd5cbab7f25f9
Venue
SIGMOD
Year
2025
Pagerank
4.9605343e-05
Overall Rank
11,257 | 24.55%
DOI
10.1145/3725322

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

BibTeX Citation

@inproceedings{li_sigmod25,
        title = {{Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization}},
        author = {Li, Haoyang and Fu, Fangcheng and Ge, Hao and Lin, Sheng and Wang, Xuanyu and Niu, Jiawen and Wang, Yujie and Zhang, Hailin and Nie, Xiaonan and Cui, Bin},
        series = {{SIGMOD} '25},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3725322},
        url = {https://dl.acm.org/doi/10.1145/3725322},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank Citing Paper Year Venue Pagerank
10,410 StaleFlow: Staleness-Aware Data Management for Mitigating Data Skewness in Fully Disaggregated RL Post-Training 2027 SIGMOD 4.9605343e-05
10,650 Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment 2026 SIGMOD 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1,612 PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel 2023 VLDB 0.00010061487
2,117 The Dawn of Natural Language to SQL: Are We Fully Ready? 2024 VLDB 9.0012524e-05
2,196 Heterogeneity-aware Distributed Parameter Servers 2017 SIGMOD 8.8807201e-05
2,227 GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization 2024 VLDB 8.7924681e-05
3,598 Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines 2022 SIGMOD 7.1596688e-05
3,887 SketchML: Accelerating Distributed Machine Learning with Data Sketches 2018 SIGMOD 6.9277708e-05
4,335 ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models 2024 VLDB 6.6432811e-05
4,360 Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity 2024 VLDB 6.6274323e-05
5,011 Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism 2023 VLDB 6.3030066e-05
5,230 Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications 2023 SIGMOD 6.2059854e-05
5,410 GoldMiner: Elastic Scaling of Training Data Pre-Processing Pipelines for Deep Learning 2023 SIGMOD 6.1317831e-05
6,011 BAGUA: Scaling up Distributed Learning with System Relaxations 2022 VLDB 5.8975241e-05
7,261 Data and AI Model Markets: Opportunities for Data and Model Sharing, Discovery, and Integration 2023 VLDB 5.5563116e-05
7,950 Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines 2024 VLDB 5.402957e-05
8,220 Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent 2023 VLDB 5.3591486e-05
8,833 FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement 2023 SIGMOD 5.2528075e-05
9,686 How Can We Train Deep Learning Models Across Clouds and Continents? An Experimental Study 2024 VLDB 5.1258854e-05
9,691 BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach 2023 SIGMOD 5.1258854e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 1,612 PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel 2023 VLDB
2 8,916 mLoRA: Fine-Tuning LoRA Adapters via Highly-Efficient Pipeline Parallelism in Multiple GPUs 2025 VLDB
3 4,911 Parallel Training of Knowledge Graph Embedding Models: A Comparison of Techniques 2022 VLDB
4 1,083 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB
5 5,011 Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism 2023 VLDB
6 9,629 Model-Parallel Model Selection for Deep Learning Systems 2021 SIGMOD
7 10,415 ZenFlow: Enabling Stall-Free Offloading for LLM Training 2027 SIGMOD
8 4,961 Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce 2021 SIGMOD
9 10,513 DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization 2026 SIGMOD
10 10,650 Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment 2026 SIGMOD