CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers

Summary: CROSSBOW scales synchronous multi-GPU training without forcing large batches, using SMA: replicas independently descend while synchronously steering toward a global average trajectory. It packs and auto-tunes replicas per GPU, yielding 1.3–4× faster training than TensorFlow on eight GPUs. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 12021
Venue: VLDB
Year: 2019
Pagerank: 7.1298979e-05
Overall Rank: 3,782 | 74.06%
DOI: 10.14778/3342263.3342276

Incoming Non-self Citations Over Time

Authors

1. Alexandros Koliousis (Imperial College London)
2. Pijika Watcharapichat (Microsoft)
3. Matthias Weidlich (Humboldt University of Berlin)
4. Luo Mai (Imperial College London)
5. Paolo Costa (Microsoft)
6. Peter Pietzuch (Imperial College London)

BibTeX Citation

@article{koliousis_vldb19,
        title = {{CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers}},
        author = {Koliousis, Alexandros and Watcharapichat, Pijika and Weidlich, Matthias and Mai, Luo and Costa, Paolo and Pietzuch, Peter},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {11},
        pages = {1399--1413},
        doi = {10.14778/3342263.3342276},
        url = {https://doi.org/10.14778/3342263.3342276},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 8 of 8 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,157	Cerebro: A Data System for Optimized Deep Learning Model Selection	2020	VLDB	0.00011924049
2,566	Pump Up the Volume: Processing Large Data on GPUs with Fast Interconnects	2020	SIGMOD	8.4116562e-05
3,227	MG-Join: A Scalable Join for Massively Parallel Multi-GPU Architectures	2021	SIGMOD	7.6217889e-05
4,067	Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches	2021	VLDB	6.9293511e-05
7,075	Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity	2024	VLDB	5.7099047e-05
8,855	ANN Softmax: Acceleration of Extreme Classification Training	2022	VLDB	5.3573227e-05
8,905	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD	5.3483178e-05
9,881	The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format	2024	SIGMOD	5.2040783e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1,150	DimmWitted: A Study of Main-Memory Statistical Analytics	2014	VLDB	0.00011943462
2,162	Heterogeneity-aware Distributed Parameter Servers	2017	SIGMOD	9.0581831e-05
2,693	FlexPS: Flexible Parallelism Control in Parameter Server Architecture	2018	VLDB	8.247004e-05
4,799	Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models	2017	VLDB	6.5024714e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,067	Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches	2021	VLDB
2	9,414	Model-Parallel Model Selection for Deep Learning Systems	2021	SIGMOD
3	4,956	Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce	2021	SIGMOD
4	3,530	MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud	2023	VLDB
5	521	PyTorch Distributed: Experiences on Accelerating Data Parallel Training	2020	VLDB
6	8,905	TensorSocket: Shared Data Loading for Deep Learning Training	2026	SIGMOD
7	9,729	Scalable Graph Convolutional Network Training on Distributed-Memory Systems	2023	VLDB
8	5,003	Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism	2023	VLDB
9	1,157	Cerebro: A Data System for Optimized Deep Learning Model Selection	2020	VLDB
10	3,764	FastFlow: Accelerating Deep Learning Model Training with Smart Offloading of Input Data Pipeline	2023	VLDB