CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers

Summary: CROSSBOW scales synchronous multi-GPU training without forcing large batches, using SMA: replicas independently descend while synchronously steering toward a global average trajectory. It packs and auto-tunes replicas per GPU, yielding 1.3–4× faster training than TensorFlow on eight GPUs. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h117fe0e8ecaab046
Venue: VLDB
Year: 2019
Pagerank: 7.0394265e-05
Overall Rank: 3,764 | 74.70%
DOI: 10.14778/3342263.3342276

Incoming Non-self Citations Over Time

Authors

1. Alexandros Koliousis (Imperial College London)
2. Pijika Watcharapichat (Microsoft)
3. Matthias Weidlich (Humboldt University of Berlin)
4. Luo Mai (Imperial College London)
5. Paolo Costa (Microsoft)
6. Peter Pietzuch (Imperial College London)

BibTeX Citation

@article{koliousis_vldb19,
        title = {{CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers}},
        author = {Koliousis, Alexandros and Watcharapichat, Pijika and Weidlich, Matthias and Mai, Luo and Costa, Paolo and Pietzuch, Peter},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {11},
        pages = {1399--1413},
        doi = {10.14778/3342263.3342276},
        url = {https://doi.org/10.14778/3342263.3342276},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 8 of 8 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,152	Cerebro: A Data System for Optimized Deep Learning Model Selection	2020	VLDB	0.00011801961
2,284	Pump Up the Volume: Processing Large Data on GPUs with Fast Interconnects	2020	SIGMOD	8.6954168e-05
3,053	MG-Join: A Scalable Join for Massively Parallel Multi-GPU Architectures	2021	SIGMOD	7.7019663e-05
4,095	Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches	2021	VLDB	6.8095767e-05
4,351	Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity	2024	VLDB	6.642803e-05
8,998	ANN Softmax: Acceleration of Extreme Classification Training	2022	VLDB	5.2400492e-05
9,067	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	5.2283159e-05
10,043	The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format	2024	SIGMOD	5.0921006e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,167	DimmWitted: A Study of Main-Memory Statistical Analytics	2014	VLDB	0.00011729888
2,184	Heterogeneity-aware Distributed Parameter Servers	2017	SIGMOD	8.8958335e-05
2,725	FlexPS: Flexible Parallelism Control in Parameter Server Architecture	2018	VLDB	8.090861e-05
4,901	Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models	2017	VLDB	6.3647386e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,095	Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches	2021	VLDB
2	9,595	Model-Parallel Model Selection for Deep Learning Systems	2021	SIGMOD
3	4,963	Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce	2021	SIGMOD
4	3,450	MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud	2023	VLDB
5	522	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB
6	9,900	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB
7	9,067	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD
8	5,009	Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism	2023	VLDB
9	1,152	Cerebro: A Data System for Optimized Deep Learning Model Selection	2020	VLDB
10	3,811	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB