MemFlow: Memory-Aware Distributed Deep Learning

Summary: MemFlow is a memory-aware distributed DNN optimizer that jointly optimizes memory usage vs training time to yield Pareto configs. It builds a memory-estimated task graph, simulates parallelism, and uses MCMC to explore recomputation vs compute tradeoffs. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5910
Venue: SIGMOD
Year: 2020
Pagerank: 5.3047301e-05
Overall Rank: 9,219 | 36.75%
DOI: 10.1145/3318464.3384416

Incoming Non-self Citations Over Time

Authors

1. Neil Band (Harvard University)

BibTeX Citation

@inproceedings{band_sigmod20,
        title = {{MemFlow: Memory-Aware Distributed Deep Learning}},
        author = {Band, Neil},
        series = {{SIGMOD} '20},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3318464.3384416},
        url = {https://dl.acm.org/doi/10.1145/3318464.3384416},
        year = {2020}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
8,328	Deep Learning: Systems and Responsibility	2021	SIGMOD	5.4535681e-05
9,729	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB	5.2289669e-05

Outgoing Citations (Sorted by Pagerank)

Showing 0 of 0 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	7,194	Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning	2018	VLDB
2	8,883	FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement	2023	SIGMOD
3	8,454	D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks	2024	VLDB
4	10,219	DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization	2026	SIGMOD
5	10,113	Distributed Learning of Fully Connected Neural Networks using Independent Subnet Training	2022	VLDB
6	3,764	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB
7	1,246	Distributed Evaluation of Subgraph Queries Using Worst-case Optimal Low-Memory Dataflows	2018	VLDB
8	9,880	MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training	2025	SIGMOD
9	9,729	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB
10	10,907	Effective and Efficient Distributed Temporal Graph Learning through Hotspot Memory Sharing	2025	VLDB