Data Chunk Compaction in Vectorized Execution

Summary: Addresses data-chunk fragmentation in vectorized analytics caused by hash-join processing. Proposes a runtime-learning compaction threshold and hash-join-specific logical compaction to minimize movement, delivering up to 63% speedups in DuckDB on TPC-H/DS. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 7025
Venue: SIGMOD
Year: 2025
Pagerank: 4.1905499e-05
Overall Rank: 10,384 | 27.84%
DOI: 10.1145/3709676

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Yiming Qiao
2. Huanchen Zhang

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
7,122	Debunking the Myth of Join Ordering: Toward Robust SQL Analytics	2025	SIGMOD	4.8199209e-05
10,241	Robust Predicate Transfer with Dynamic Execution	2026	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 31 of 31 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
35	MonetDB/X100: Hyper-Pipelining Query Execution	2005	CIDR	0.00076209479
71	How Good Are Query Optimizers, Really?	2016	VLDB	0.00059446482
81	Cache Conscious Algorithms for Relational Query Processing	1994	VLDB	0.00055253195
132	Integrating Compression and Execution in Column-Oriented Database Systems	2006	SIGMOD	0.00043697853
167	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00039408116
185	DuckDB: an Embeddable Analytical Database	2019	SIGMOD	0.00036529607
241	DB2 with BLU Acceleration: So Much More than Just a Column Store	2013	VLDB	0.00031314629
350	Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs	2009	VLDB	0.00026368305
363	CockroachDB: The Resilient Geo-Distributed SQL Database	2020	SIGMOD	0.00025750338
417	Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age	2014	SIGMOD	0.00023734582
848	Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask	2018	VLDB	0.00015933538
959	Rethinking SIMD Vectorization for In-Memory Databases	2015	SIGMOD	0.00015034808
1,260	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD	0.00012990853
1,800	An Experimental Comparison of Thirteen Relational Equi-Joins in Main Memory	2016	SIGMOD	0.00010494121
1,865	Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last	2018	VLDB	0.00010271333
1,939	Procella: Unifying serving and analytical data at YouTube	2019	VLDB	0.00010019433
2,060	Dremel: A Decade of Interactive SQL Analysis at Web Scale	2020	VLDB	9.6585115e-05
2,472	Photon: A Fast Query Engine for Lakehouse Systems	2022	SIGMOD	8.7156826e-05
2,533	Velox: Meta’s Unified Execution Engine	2022	VLDB	8.5870599e-05
3,428	Distributed Join Algorithms on Thousands of Cores	2017	VLDB	7.1002401e-05
3,719	To Partition, or Not to Partition, That is the Join Question in a Real System	2021	SIGMOD	6.8141176e-05
3,881	Micro Adaptivity in Vectorwise	2013	SIGMOD	6.6638918e-05
4,498	ClickHouse - Lightning Fast Analytics for Everyone	2024	VLDB	6.1351257e-05
4,516	An Empirical Evaluation of Columnar Storage Formats	2024	VLDB	6.1146215e-05
6,218	Charting the Design Space of Query Execution using VOILA	2021	VLDB	5.1462707e-05
6,332	Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine	2024	SIGMOD	5.1021765e-05
7,443	From X100 to Vectorwise: Opportunities, Challenges and Things Most Researchers do not Think About	2012	SIGMOD	4.7240917e-05
8,476	Analyzing Vectorized Hash Tables Across CPU Architectures	2023	VLDB	4.4972784e-05
9,662	Efficient Query Re-optimization with Judicious Subquery Selections	2023	SIGMOD	4.3056334e-05
9,856	Towards Unifying Query Interpretation and Compilation	2023	CIDR	4.2652623e-05
11,039	Blitzcrank: Fast Semantic Compression for In-memory Online Transaction Processing	2024	VLDB	4.1905499e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
1,890	Caching Multidimensional Queries Using Chunks	1998	SIGMOD	0.0001018961
5,323	Can Learned Models Replace Hash Functions?	2023	VLDB	5.5671086e-05
1,098	Query Optimization In Compressed Database Systems	2001	SIGMOD	0.00014070252
538	Design and Evaluation of Main Memory Hash Join Algorithms for Multi-core CPUs	2011	SIGMOD	0.00020632609
1,260	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD	0.00012990853
8,575	Robust and Budget-Constrained Encoding Configurations for In-Memory Database Systems	2022	VLDB	4.4880409e-05
1,193	Fast Computation of Sparse Datacubes	1997	VLDB	0.00013412915
5,677	Joins on Encoded and Partitioned Data	2014	VLDB	5.376511e-05
8,271	Adaptive Factorization Using Linear-Chained Hash Tables	2025	CIDR	4.539672e-05
10,643	Saving Private Hash Join	2025	VLDB	4.1905499e-05