SquirrelJoin: Network-Aware Distributed Join Processing with Lazy Partitioning

Summary: SquirrelJoin is a network-aware distributed join using lazy partitioning to mitigate transient skew in shared clusters. In-memory lazy partitions are dynamically allocated via throughput estimates to minimize join time; implemented in Apache Flink, it achieves up to 2.9x speedups with modest overhead. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 11410
Venue: VLDB
Year: 2017
Pagerank: 4.8465382e-05
Overall Rank: 7,060 | 50.89%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,210	Frontier: Resilient Edge Processing for the Internet of Things	2018	VLDB	7.3746627e-05
4,002	MG-Join: A Scalable Join for Massively Parallel Multi-GPU Architectures	2021	SIGMOD	6.545665e-05
8,781	Accelerate Distributed Joins with Predicate Transfer	2025	SIGMOD	4.4534753e-05
9,142	Design and Analysis of a Processing-in-DIMM Join Algorithm: A Case Study with UPMEM DIMMs	2023	SIGMOD	4.3853149e-05
9,488	INEv: In-Network Evaluation for Event Stream Processing	2023	SIGMOD	4.3341665e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
9	Implementation Techniques For Main Memory Database Systems	1984	SIGMOD	0.0014279444
66	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00061639801
115	Eddies: Continuously Adaptive Query Processing	2000	SIGMOD	0.00046221215
960	A Comparison of Join Algorithms for Log Processing in MapReduce	2010	SIGMOD	0.00015012242
1,334	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.0001250413
1,915	Handling Data Skew in Parallel Joins in Shared-Nothing Systems	2008	SIGMOD	0.00010104123
2,439	CoHadoop: Flexible Data Placement and Its Exploitation in Hadoop	2011	VLDB	8.8190594e-05
2,526	Track Join: Distributed Joins with Minimal Network Traffic	2014	SIGMOD	8.5968612e-05
3,382	Scalable and Adaptive Online Joins	2014	VLDB	7.1597145e-05
4,943	Lifting the Burden of History from Adaptive Query Processing	2004	VLDB	5.8170713e-05
5,142	A Software-Defined Networking based Approach for Performance Management of Analytical Queries on Distributed Data Stores	2014	SIGMOD	5.6673393e-05
6,338	RPJ: Producing Fast Join Results on Streams through Rate-based Optimization	2005	SIGMOD	5.105321e-05
7,913	Resource Bricolage for Parallel Database Systems	2015	VLDB	4.6180739e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
3,443	Distributed Join Algorithms on Thousands of Cores	2017	VLDB	7.0887214e-05
8,781	Accelerate Distributed Joins with Predicate Transfer	2025	SIGMOD	4.4534753e-05
7,153	Submodularity of Distributed Join Computation	2018	SIGMOD	4.8153963e-05
11,797	Runtime Optimization of Join Location in Parallel Data Management Systems	2017	VLDB	4.1945683e-05
1,365	Handling Data Skew in Multiprocessor Database Computers Using Partition Tuning	1991	VLDB	0.00012368421
6,619	Near-Optimal Distributed Band-Joins through Recursive Partitioning	2020	SIGMOD	4.9910152e-05
2,526	Track Join: Distributed Joins with Minimal Network Traffic	2014	SIGMOD	8.5968612e-05
8,075	AJoin: Ad-hoc Stream Joins at Scale	2020	VLDB	4.5917655e-05
11,890	Let's Rethink Join Optimization in Distributed Systems	2015	CIDR	4.1945683e-05
3,382	Scalable and Adaptive Online Joins	2014	VLDB	7.1597145e-05