SquirrelJoin: Network-Aware Distributed Join Processing with Lazy Partitioning

Summary: SquirrelJoin mitigates transient network skew in distributed joins via in-memory lazy partitions, dynamically assigned using throughput-based completion-time estimates. A Flink implementation cuts join latency up to 2.9× under contention with small fixed overhead. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 11598
Venue: VLDB
Year: 2017
Pagerank: 5.7814194e-05
Overall Rank: 6,762 | 53.61%
DOI: 10.14778/3137628.3137650

Incoming Non-self Citations Over Time

Authors

1. Lukas Rupprecht (Imperial College London)
2. William Culhane (Imperial College London)
3. Peter Pietzuch (Imperial College London)

BibTeX Citation

@article{rupprecht_vldb17,
        title = {{SquirrelJoin: Network-Aware Distributed Join Processing with Lazy Partitioning}},
        author = {Rupprecht, Lukas and Culhane, William and Pietzuch, Peter},
        journal = {PVLDB},
        series = {{VLDB} '17},
        volume = {10},
        number = {11},
        pages = {1250--1261},
        doi = {10.14778/3137628.3137650},
        url = {https://doi.org/10.14778/3137628.3137650},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,227	MG-Join: A Scalable Join for Massively Parallel Multi-GPU Architectures	2021	SIGMOD	7.6217889e-05
3,306	Frontier: Resilient Edge Processing for the Internet of Things	2018	VLDB	7.5387095e-05
6,957	Design and Analysis of a Processing-in-DIMM Join Algorithm: A Case Study with UPMEM DIMMs	2023	SIGMOD	5.7303405e-05
8,721	Accelerate Distributed Joins with Predicate Transfer	2025	SIGMOD	5.3772617e-05
9,628	INEv: In-Network Evaluation for Event Stream Processing	2023	SIGMOD	5.2434488e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
7	Implementation Techniques For Main Memory Database Systems	1984	SIGMOD	0.00083340894
24	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00054865648
53	Eddies: Continuously Adaptive Query Processing	2000	SIGMOD	0.00041071971
769	A Comparison of Join Algorithms for Log Processing in MapReduce	2010	SIGMOD	0.00014166872
1,319	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00011175005
1,657	Handling Data Skew in Parallel Joins in Shared-Nothing Systems	2008	SIGMOD	0.00010096933
1,974	Track Join: Distributed Joins with Minimal Network Traffic	2014	SIGMOD	9.3658402e-05
2,642	CoHadoop: Flexible Data Placement and Its Exploitation in Hadoop	2011	VLDB	8.3059948e-05
2,892	Scalable and Adaptive Online Joins	2014	VLDB	7.9852178e-05
4,283	Lifting the Burden of History from Adaptive Query Processing	2004	VLDB	6.7836167e-05
5,187	A Software-Defined Networking based Approach for Performance Management of Analytical Queries on Distributed Data Stores	2014	SIGMOD	6.3279213e-05
5,911	RPJ: Producing Fast Join Results on Streams through Rate-based Optimization	2005	SIGMOD	6.0443263e-05
7,822	Resource Bricolage for Parallel Database Systems	2015	VLDB	5.5371287e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	2,926	Distributed Join Algorithms on Thousands of Cores	2017	VLDB
2	8,721	Accelerate Distributed Joins with Predicate Transfer	2025	SIGMOD
3	7,111	Submodularity of Distributed Join Computation	2018	SIGMOD
4	12,001	Runtime Optimization of Join Location in Parallel Data Management Systems	2017	VLDB
5	1,236	Handling Data Skew in Multiprocessor Database Computers Using Partition Tuning	1991	VLDB
6	6,596	Near-Optimal Distributed Band-Joins through Recursive Partitioning	2020	SIGMOD
7	1,974	Track Join: Distributed Joins with Minimal Network Traffic	2014	SIGMOD
8	7,856	AJoin: Ad-hoc Stream Joins at Scale	2020	VLDB
9	12,090	Let's Rethink Join Optimization in Distributed Systems	2015	CIDR
10	2,892	Scalable and Adaptive Online Joins	2014	VLDB