Data Chunk Compaction in Vectorized Execution

Summary: Addresses data-chunk fragmentation in vectorized analytics caused by hash-join processing. Proposes a runtime-learning compaction threshold and hash-join-specific logical compaction to minimize movement, delivering up to 63% speedups in DuckDB on TPC-H/DS. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 7086
Venue: SIGMOD
Year: 2025
Pagerank: 5.093636e-05
Overall Rank: 10,662 | 26.85%
DOI: 10.1145/3709676

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Yiming Qiao (Tsinghua University)
2. Huanchen Zhang (Shanghai Qi Zhi Institute; Tsinghua University)

BibTeX Citation

@inproceedings{qiao_sigmod25,
        title = {{Data Chunk Compaction in Vectorized Execution}},
        author = {Qiao, Yiming and Zhang, Huanchen},
        series = {{SIGMOD} '25},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3709676},
        url = {https://dl.acm.org/doi/10.1145/3709676},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
5,529	Debunking the Myth of Join Ordering: Toward Robust SQL Analytics	2025	SIGMOD	6.18591e-05
10,529	Robust Predicate Transfer with Dynamic Execution	2026	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 31 of 31 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
14	MonetDB/X100: Hyper-Pipelining Query Execution	2005	CIDR	0.0006312782
18	How Good Are Query Optimizers, Really?	2016	VLDB	0.00059284255
60	Integrating Compression and Execution in Column-Oriented Database Systems	2006	SIGMOD	0.0003955489
66	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00038561587
74	Cache Conscious Algorithms for Relational Query Processing	1994	VLDB	0.00037330605
103	DuckDB: an Embeddable Analytical Database	2019	SIGMOD	0.00034161428
165	DB2 with BLU Acceleration: So Much More than Just a Column Store	2013	VLDB	0.00027693424
208	CockroachDB: The Resilient Geo-Distributed SQL Database	2020	SIGMOD	0.00024968157
209	Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs	2009	VLDB	0.00024932174
241	Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age	2014	SIGMOD	0.00023654664
634	Rethinking SIMD Vectorization for In-Memory Databases	2015	SIGMOD	0.00015533814
649	Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask	2018	VLDB	0.00015320656
941	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD	0.00013078348
1,265	An Experimental Comparison of Thirteen Relational Equi-Joins in Main Memory	2016	SIGMOD	0.00011415709
1,379	Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last	2018	VLDB	0.00010970894
1,453	Dremel: A Decade of Interactive SQL Analysis at Web Scale	2020	VLDB	0.00010742227
1,494	Procella: Unifying serving and analytical data at YouTube	2019	VLDB	0.00010577585
1,757	Velox: Meta's Unified Execution Engine	2022	VLDB	9.8166984e-05
1,824	Photon: A Fast Query Engine for Lakehouse Systems	2022	SIGMOD	9.6734544e-05
2,926	Distributed Join Algorithms on Thousands of Cores	2017	VLDB	7.9549783e-05
2,962	To Partition, or Not to Partition, That is the Join Question in a Real System	2021	SIGMOD	7.9170451e-05
3,195	Micro Adaptivity in Vectorwise	2013	SIGMOD	7.6481177e-05
3,653	ClickHouse - Lightning Fast Analytics for Everyone	2024	VLDB	7.2241143e-05
3,877	An Empirical Evaluation of Columnar Storage Formats	2024	VLDB	7.0532293e-05
5,469	Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine	2024	SIGMOD	6.2073056e-05
5,765	Charting the Design Space of Query Execution using VOILA	2021	VLDB	6.0953705e-05
7,276	From X100 to Vectorwise: Opportunities, Challenges and Things Most Researchers do not Think About	2012	SIGMOD	5.6582613e-05
8,443	Analyzing Vectorized Hash Tables Across CPU Architectures	2023	VLDB	5.4243766e-05
9,756	Efficient Query Re-optimization with Judicious Subquery Selections	2023	SIGMOD	5.2258278e-05
10,008	Towards Unifying Query Interpretation and Compilation	2023	CIDR	5.1785299e-05
11,246	Blitzcrank: Fast Semantic Compression for In-memory Online Transaction Processing	2024	VLDB	5.093636e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	921	Query Optimization In Compressed Database Systems	2001	SIGMOD
2	4,780	Can Learned Models Replace Hash Functions?	2023	VLDB
3	360	Design and Evaluation of Main Memory Hash Join Algorithms for Multi-core CPUs	2011	SIGMOD
4	941	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD
5	8,410	Robust and Budget-Constrained Encoding Configurations for In-Memory Database Systems	2022	VLDB
6	1,083	Fast Computation of Sparse Datacubes	1997	VLDB
7	5,357	Joins on Encoded and Partitioned Data	2014	VLDB
8	5,383	Compressed Representations of Conjunctive Query Results	2018	PODS
9	8,232	Adaptive Factorization Using Linear-Chained Hash Tables	2025	CIDR
10	10,888	Saving Private Hash Join	2025	VLDB