On Reducing Space Amplification with Multi-Column Compaction in Apache IoTDB

Summary: Introduces Multi-Column Compaction for Apache IoTDB’s LSM engine, targeting space amplification from out-of-order inserts, updates, duplicate keys, obsolete values, and overlapping bitmaps. A heuristic file selector with prefetching and compaction caching improves reduction efficiency. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 13704
Venue: VLDB
Year: 2024
Pagerank: 5.093636e-05
Overall Rank: 11,257 | 22.77%
DOI: 10.14778/3681954.3681977

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Chenguang Fang (Tsinghua University)
2. Zijie Chen (Tsinghua University)
3. Shaoxu Song (Tsinghua University)
4. Xiangdong Huang (Tsinghua University)
5. Chen Wang (Tsinghua University)
6. Jianmin Wang (Tsinghua University)

BibTeX Citation

@article{fang_vldb24,
        title = {{On Reducing Space Amplification with Multi-Column Compaction in Apache IoTDB}},
        author = {Fang, Chenguang and Chen, Zijie and Song, Shaoxu and Huang, Xiangdong and Wang, Chen and Wang, Jianmin},
        journal = {PVLDB},
        series = {{VLDB} '24},
        volume = {17},
        number = {11},
        pages = {2974--2986},
        doi = {10.14778/3681954.3681977},
        url = {https://doi.org/10.14778/3681954.3681977},
        year = {2024}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 16 of 16 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
271	TiDB: A Raft-based HTAP Database	2020	VLDB	0.00022703024
282	Efficient Management of Transitive Relationships in Large Data and Knowledge Bases	1989	SIGMOD	0.0002242162
472	Optimizing Space Amplification in RocksDB	2017	CIDR	0.000179044
831	Dostoevsky: Better Space-Time Trade-Offs for LSM-Tree Based Key-Value Stores via Adaptive Removal of Superfluous Merging	2018	SIGMOD	0.00013748512
857	SlimDB: A Space-Efficient Key-Value Storage Engine For Semi-Sorted Data	2017	VLDB	0.00013568886
1,213	MyRocks: LSM-Tree Database Storage Engine Serving Facebook's Social Graph	2020	VLDB	0.00011646797
1,347	X-Engine: An Optimized Storage Engine for Large-scale E-commerce Transaction Processing	2019	SIGMOD	0.00011073571
1,608	Apache IoTDB: Time-series Database for Internet of Things	2020	VLDB	0.00010227331
1,874	Storage Management in AsterixDB	2014	VLDB	9.5750002e-05
2,755	Lethe: A Tunable Delete-Aware LSM Engine	2020	SIGMOD	8.163097e-05
3,005	Constructing and Analyzing the LSM Compaction Design Space	2021	VLDB	7.8608206e-05
3,054	Spooky: Granulating LSM-Tree Compactions Correctly	2022	VLDB	7.8090808e-05
3,793	Apache IoTDB: A Time Series Database for IoT Applications	2023	SIGMOD	7.1217835e-05
4,826	Design Tradeoffs of Data Access Methods	2016	SIGMOD	6.4904836e-05
5,229	Dissecting, Designing, and Optimizing LSM-based Data Stores	2022	SIGMOD	6.307514e-05
11,381	Grouping Time Series for Efficient Columnar Storage	2023	SIGMOD	5.093636e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	3,793	Apache IoTDB: A Time Series Database for IoT Applications	2023	SIGMOD
2	10,667	In-Database Time Series Clustering	2025	SIGMOD
3	9,457	Rethinking The Compaction Policies in LSM-trees	2025	SIGMOD
4	8,409	Time Series Representation for Visualization in Apache IoTDB	2024	SIGMOD
5	10,840	Migration-Free Elastic Storage of Time Series in Apache IoTDB	2025	VLDB
6	4,762	Time Series Data Encoding for Efficient Storage: A Comparative Analysis in Apache IoTDB	2022	VLDB
7	3,005	Constructing and Analyzing the LSM Compaction Design Space	2021	VLDB
8	8,973	Distance-based Outlier Query Optimization in Apache IoTDB	2024	VLDB
9	11,381	Grouping Time Series for Efficient Columnar Storage	2023	SIGMOD
10	10,923	Improving Time Series Data Compression in Apache IoTDB	2025	VLDB