DBScholar

Back to papers

Distributed Join Algorithms on Thousands of Cores

Summary: MPI-based radix-hash and sort-merge joins scale to 4,096 cores and 4.8 TB, reaching 48.7B tuples/s with SIMD, one-sided operations, and RDMA. Communication scheduling and compute/network balance dominate scalability; sort-merge nears its model peak, unlike hashing. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
hada67e191ff880b0
Venue
VLDB
Year
2017
Pagerank
7.9903139e-05
Overall Rank
2,802 | 81.17%
DOI
10.14778/3055540.3055546

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{barthels_vldb17,
        title = {{Distributed Join Algorithms on Thousands of Cores}},
        author = {Barthels, Claude and Müller, Ingo and Schneider, Timo and Alonso, Gustavo and Hoefler, Torsten},
        journal = {PVLDB},
        series = {{VLDB} '17},
        volume = {10},
        number = {5},
        pages = {517--528},
        doi = {10.14778/3055540.3055546},
        url = {https://doi.org/10.14778/3055540.3055546},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
1,241 BatchDB: Efficient Isolated Execution of Hybrid OLTP+OLAP Workloads for Interactive Applications 2017 SIGMOD 0.00011378254
1,366 Designing Distributed Tree-based Index Structures for Fast RDMA-capable Networks 2019 SIGMOD 0.00010912114
1,423 Procella: Unifying serving and analytical data at YouTube 2019 VLDB 0.00010719956
1,966 Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory 2022 SIGMOD 9.3039057e-05
3,227 Farview: Disaggregated Memory with Operator Off-loading for Database Engines 2022 CIDR 7.5083649e-05
3,608 DFI: The Data Flow Interface for High-Speed Networks 2021 SIGMOD 7.1650501e-05
3,626 Triton Join: Efficiently Scaling to a Large Join State on GPUs with Fast Interconnects 2022 SIGMOD 7.1524537e-05
3,953 Tensors: An abstraction for general data processing 2021 VLDB 6.9027109e-05
4,414 Strong consistency is not hard to get: Two-Phase Locking and Two-Phase Commit on Thousands of Cores 2019 VLDB 6.609914e-05
4,931 FPGA-based Data Partitioning 2017 SIGMOD 6.348544e-05
5,371 Distributed GPU Joins on Fast RDMA-capable Networks 2023 SIGMOD 6.1584802e-05
6,213 Data Partitioning for In-Memory Systems: Myths, Challenges, and Opportunities 2019 CIDR 5.8479612e-05
6,534 DPI: The Data Processing Interface for Modern Networks 2019 CIDR 5.7541988e-05
7,451 Modularis: Modular Relational Analytics over Heterogeneous Distributed Platforms 2021 VLDB 5.5236802e-05
7,729 Hardware-Oblivious SIMD Parallelism for In-Memory Column-Stores 2020 CIDR 5.4657611e-05
8,269 Accelerate Distributed Joins with Predicate Transfer 2025 SIGMOD 5.3648571e-05
8,403 The Case for Learned In-Memory Joins 2023 VLDB 5.3389852e-05
8,859 OLAP on Modern Chiplet-Based Processors 2024 VLDB 5.2632587e-05
9,429 Efficiently Joining Large Relations on Multi-GPU Systems 2025 VLDB 5.1786456e-05
9,659 Parallel Query Processing: To Separate Communication from Computation 2022 SIGMOD 5.1453267e-05
10,304 Data Chunk Compaction in Vectorized Execution 2025 SIGMOD 5.0400722e-05
10,801 PystachIO: Efficient Distributed GPU Query Processing with PyTorch over Fast Networks & Fast Storage 2026 VLDB 4.9793485e-05
11,865 Scaling Equi-Joins 2022 SIGMOD 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
210 Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs 2009 VLDB 0.00024851502
251 Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited 2014 VLDB 0.00023143736
361 Design and Evaluation of Main Memory Hash Join Algorithms for Multi-core CPUs 2011 SIGMOD 0.00020006406
423 Massively Parallel Sort-Merge Joins in Main Memory Multi-Core Database Systems 2012 VLDB 0.00018491327
777 To Join or Not to Join? Thinking Twice about Joins before Feature Selection 2016 SIGMOD 0.00014054709
889 Rack-Scale In-Memory Join Processing using RDMA 2015 SIGMOD 0.00013247362
945 The End of Slow Networks: It's Time for a Redesign 2016 VLDB 0.00012939225
1,028 High-Speed Query Processing over High-Speed Networks 2016 VLDB 0.00012422084
1,059 Accelerating Relational Databases by Leveraging Remote Memory and RDMA 2016 SIGMOD 0.00012224884
1,995 Track Join: Distributed Joins with Minimal Network Traffic 2014 SIGMOD 9.2169073e-05
5,817 VectorH: Taking SQL-on-Hadoop to the Next Level 2016 SIGMOD 5.9840356e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 9,429 Efficiently Joining Large Relations on Multi-GPU Systems 2025 VLDB
2 249 A Performance Evaluation of Four Parallel Join Algorithms in a Shared-Nothing Multiprocessor Environment 1989 SIGMOD
3 889 Rack-Scale In-Memory Join Processing using RDMA 2015 SIGMOD
4 156 Multiprocessor Hash-Based Join Algorithms 1985 VLDB
5 423 Massively Parallel Sort-Merge Joins in Main Memory Multi-Core Database Systems 2012 VLDB
6 361 Design and Evaluation of Main Memory Hash Join Algorithms for Multi-core CPUs 2011 SIGMOD
7 12,383 Let's Rethink Join Optimization in Distributed Systems 2015 CIDR
8 3,059 Design and Evaluation of Parallel Pipelined Join Algorithms 1987 SIGMOD
9 210 Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs 2009 VLDB
10 251 Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited 2014 VLDB