Towards Functional Decomposition of Storage Formats

Summary: Shows that tying compression blocks to row‑skipping partitions in columnar formats forces a compressibility vs scan-performance tradeoff; proposes splitting into a storage layer + Search Acceleration Layer (SAL). Finds SALs benefit from fine-grained partitions (~10–100 rows), with optimal size varying by metadata, data, and query, enabling independent tuning and improved tradeoffs. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 544
Venue: CIDR
Year: 2025
Pagerank: 5.2110542e-05
Overall Rank: 9,836 | 32.52%
DOI: -

Incoming Non-self Citations Over Time

Authors

1. Martin Prammer (Carnegie Mellon University)
2. Xinyu Zeng (Tsinghua University)
3. Ruijun Meng (Tsinghua University)
4. Wes McKinney (Posit Public Benefit Corporation)
5. Huanchen Zhang (Tsinghua University)
6. Andrew Pavlo (Carnegie Mellon University)
7. Jignesh M. Patel (Carnegie Mellon University)

BibTeX Citation

@inproceedings{prammer_cidr25,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '25},
        title = {{Towards Functional Decomposition of Storage Formats}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Prammer, Martin and Zeng, Xinyu and Meng, Ruijun and McKinney, Wes and Zhang, Huanchen and Pavlo, Andrew and Patel, Jignesh M.},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,277	F3: The Open-Source Data File Format for the Future	2026	SIGMOD	5.2937431e-05
10,057	AnyBlox: A Framework for Self-Decoding Datasets	2025	VLDB	5.1664022e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
49	Weaving Relations for Cache Performance	2001	VLDB	0.00043781096
66	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00038561587
227	Small Materialized Aggregates: A Light Weight Index Structure for Data Warehousing	1998	VLDB	0.00023958508
237	Amazon Redshift and the Case for Simpler Data Warehouses	2015	SIGMOD	0.0002369895
520	Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores	2020	VLDB	0.00017136828
749	Data Lake Management: Challenges and Opportunities	2019	VLDB	0.00014379989
1,138	Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics	2021	CIDR	0.00012023643
2,477	Azure Data Lake Store: A Hyperscale Distributed File Service for Big Data Analytics	2017	SIGMOD	8.5239378e-05
2,788	BtrBlocks: Efficient Columnar Compression for Data Lakes	2023	SIGMOD	8.1205155e-05
3,877	An Empirical Evaluation of Columnar Storage Formats	2024	VLDB	7.0532293e-05
5,037	A Deep Dive into Common Open Formats for Analytical DBMSs	2023	VLDB	6.3914026e-05
7,285	Bullion: A Column Store for Machine Learning	2025	CIDR	5.6559604e-05
7,469	Optimizing Collections of Bloom Filters within a Space Budget	2024	VLDB	5.609743e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	3,876	Cracking the Database Store	2005	CIDR
2	10,509	FlatStor: An Efficient Embedded-Index Based Columnar Data Layout for Multimodal Data Workloads	2026	VLDB
3	7,780	Petabyte-Scale Row-Level Operations in Data Lakehouses	2024	VLDB
4	5,715	Design and Evaluation of Storage Organizations for Read-Optimized Main Memory Databases	2013	VLDB
5	11,274	Partition, Don’t Sort! Compression Boosters for Cloud Data Ingestion Pipelines	2024	VLDB
6	6,653	Adaptive Data Skipping in Main-Memory Systems	2016	SIGMOD
7	6,076	Self-Organizing Data Containers	2022	CIDR
8	12,191	A Partitioning Framework for Aggressive Data Skipping	2014	VLDB
9	3,877	An Empirical Evaluation of Columnar Storage Formats	2024	VLDB
10	3,106	Skipping-oriented Partitioning for Columnar Layouts	2017	VLDB