SquirrelJoin: Network-Aware Distributed Join Processing with Lazy Partitioning

Summary: SquirrelJoin is a network-aware distributed join using lazy partitioning to mitigate transient skew in shared clusters. In-memory lazy partitions are dynamically allocated via throughput estimates to minimize join time; implemented in Apache Flink, it achieves up to 2.9x speedups with modest overhead. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 11411
Venue: VLDB
Year: 2017
Pagerank: 4.8420027e-05
Overall Rank: 7,056 | 50.97%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,216	Frontier: Resilient Edge Processing for the Internet of Things	2018	VLDB	7.3675613e-05
4,000	MG-Join: A Scalable Join for Massively Parallel Multi-GPU Architectures	2021	SIGMOD	6.5419402e-05
8,777	Accelerate Distributed Joins with Predicate Transfer	2025	SIGMOD	4.4492064e-05
9,143	Design and Analysis of a Processing-in-DIMM Join Algorithm: A Case Study with UPMEM DIMMs	2023	SIGMOD	4.381112e-05
9,490	INEv: In-Network Evaluation for Event Stream Processing	2023	SIGMOD	4.3300131e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
9	Implementation Techniques For Main Memory Database Systems	1984	SIGMOD	0.001426258
66	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00061707583
116	Eddies: Continuously Adaptive Query Processing	2000	SIGMOD	0.00046191288
962	A Comparison of Join Algorithms for Log Processing in MapReduce	2010	SIGMOD	0.00015003834
1,339	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00012492978
1,919	Handling Data Skew in Parallel Joins in Shared-Nothing Systems	2008	SIGMOD	0.00010097452
2,441	CoHadoop: Flexible Data Placement and Its Exploitation in Hadoop	2011	VLDB	8.8106295e-05
2,518	Track Join: Distributed Joins with Minimal Network Traffic	2014	SIGMOD	8.6052941e-05
3,384	Scalable and Adaptive Online Joins	2014	VLDB	7.153329e-05
4,945	Lifting the Burden of History from Adaptive Query Processing	2004	VLDB	5.8115261e-05
5,142	A Software-Defined Networking based Approach for Performance Management of Analytical Queries on Distributed Data Stores	2014	SIGMOD	5.6618933e-05
6,349	RPJ: Producing Fast Join Results on Streams through Rate-based Optimization	2005	SIGMOD	5.0957869e-05
7,914	Resource Bricolage for Parallel Database Systems	2015	VLDB	4.6136905e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
3,428	Distributed Join Algorithms on Thousands of Cores	2017	VLDB	7.1002401e-05
8,777	Accelerate Distributed Joins with Predicate Transfer	2025	SIGMOD	4.4492064e-05
7,154	Submodularity of Distributed Join Computation	2018	SIGMOD	4.8107765e-05
11,805	Runtime Optimization of Join Location in Parallel Data Management Systems	2017	VLDB	4.1905499e-05
1,355	Handling Data Skew in Multiprocessor Database Computers Using Partition Tuning	1991	VLDB	0.00012417228
6,618	Near-Optimal Distributed Band-Joins through Recursive Partitioning	2020	SIGMOD	4.9864636e-05
2,518	Track Join: Distributed Joins with Minimal Network Traffic	2014	SIGMOD	8.6052941e-05
8,078	AJoin: Ad-hoc Stream Joins at Scale	2020	VLDB	4.5873626e-05
11,898	Let's Rethink Join Optimization in Distributed Systems	2015	CIDR	4.1905499e-05
3,384	Scalable and Adaptive Online Joins	2014	VLDB	7.153329e-05