Data Chunk Compaction in Vectorized Execution

Summary: Addresses data-chunk fragmentation in vectorized analytics caused by hash-join processing. Proposes a runtime-learning compaction threshold and hash-join-specific logical compaction to minimize movement, delivering up to 63% speedups in DuckDB on TPC-H/DS. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: hedb995c67bf591e7
Venue: SIGMOD
Year: 2025
Pagerank: 5.0400722e-05
Overall Rank: 10,304 | 30.73%
DOI: 10.1145/3709676

Incoming Non-self Citations Over Time

Authors

1. Yiming Qiao (Tsinghua University)
2. Huanchen Zhang (Shanghai Qi Zhi Institute; Tsinghua University)

BibTeX Citation

@inproceedings{qiao_sigmod25,
        title = {{Data Chunk Compaction in Vectorized Execution}},
        author = {Qiao, Yiming and Zhang, Huanchen},
        series = {{SIGMOD} '25},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3709676},
        url = {https://dl.acm.org/doi/10.1145/3709676},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,950	Debunking the Myth of Join Ordering: Toward Robust SQL Analytics	2025	SIGMOD	6.3421691e-05
10,713	Robust Predicate Transfer with Dynamic Execution	2026	VLDB	4.9793485e-05
10,932	IMLane: Composable Framework for Efficient AI Function Execution in Database Engine	2026	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 31 of 31 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
14	MonetDB/X100: Hyper-Pipelining Query Execution	2005	CIDR	0.00064031282
15	How Good Are Query Optimizers, Really?	2016	VLDB	0.00061066921
52	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00041219077
61	Integrating Compression and Execution in Column-Oriented Database Systems	2006	SIGMOD	0.000392237
71	DuckDB: an Embeddable Analytical Database	2019	SIGMOD	0.00037720227
76	Cache Conscious Algorithms for Relational Query Processing	1994	VLDB	0.00036898845
163	DB2 with BLU Acceleration: So Much More than Just a Column Store	2013	VLDB	0.0002749118
185	CockroachDB: The Resilient Geo-Distributed SQL Database	2020	SIGMOD	0.00026072278
210	Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs	2009	VLDB	0.00024851502
215	Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age	2014	SIGMOD	0.00024598661
605	Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask	2018	VLDB	0.00015647561
627	Rethinking SIMD Vectorization for In-Memory Databases	2015	SIGMOD	0.00015460957
906	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD	0.00013160654
1,135	Dremel: A Decade of Interactive SQL Analysis at Web Scale	2020	VLDB	0.00011891907
1,266	An Experimental Comparison of Thirteen Relational Equi-Joins in Main Memory	2016	SIGMOD	0.00011269175
1,341	Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last	2018	VLDB	0.00010968512
1,423	Procella: Unifying serving and analytical data at YouTube	2019	VLDB	0.00010719956
1,424	Velox: Meta's Unified Execution Engine	2022	VLDB	0.00010719232
1,487	Photon: A Fast Query Engine for Lakehouse Systems	2022	SIGMOD	0.00010521722
2,802	Distributed Join Algorithms on Thousands of Cores	2017	VLDB	7.9903139e-05
2,818	To Partition, or Not to Partition, That is the Join Question in a Real System	2021	SIGMOD	7.9739791e-05
3,001	ClickHouse - Lightning Fast Analytics for Everyone	2024	VLDB	7.7667824e-05
3,100	Micro Adaptivity in Vectorwise	2013	SIGMOD	7.6505054e-05
3,808	An Empirical Evaluation of Columnar Storage Formats	2024	VLDB	7.0097354e-05
4,074	Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine	2024	SIGMOD	6.8198673e-05
5,543	Charting the Design Space of Query Execution using VOILA	2021	VLDB	6.0889645e-05
7,384	From X100 to Vectorwise: Opportunities, Challenges and Things Most Researchers do not Think About	2012	SIGMOD	5.5385293e-05
7,814	Analyzing Vectorized Hash Tables Across CPU Architectures	2023	VLDB	5.4482093e-05
8,164	Efficient Query Re-optimization with Judicious Subquery Selections	2023	SIGMOD	5.3852872e-05
10,193	Towards Unifying Query Interpretation and Compilation	2023	CIDR	5.0633497e-05
11,579	Blitzcrank: Fast Semantic Compression for In-memory Online Transaction Processing	2024	VLDB	4.9793485e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	900	Query Optimization In Compressed Database Systems	2001	SIGMOD
2	10,768	One Join Order Does Not Fit All: Reducing Intermediate Results with Per-Split Query Plans	2026	VLDB
3	361	Design and Evaluation of Main Memory Hash Join Algorithms for Multi-core CPUs	2011	SIGMOD
4	906	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD
5	8,563	Robust and Budget-Constrained Encoding Configurations for In-Memory Database Systems	2022	VLDB
6	1,068	Fast Computation of Sparse Datacubes	1997	VLDB
7	5,423	Joins on Encoded and Partitioned Data	2014	VLDB
8	5,489	Compressed Representations of Conjunctive Query Results	2018	PODS
9	7,774	Adaptive Factorization Using Linear-Chained Hash Tables	2025	CIDR
10	9,566	Saving Private Hash Join	2025	VLDB