Database Paper Browser

Back to papers

An Empirical Evaluation of Columnar Storage Formats

Summary: Revisits Parquet and ORC with a stress-test benchmark on modern hardware and workloads, pinpointing internal choices that favor today's analytics: default dictionary encoding, integer encodings optimized for decode speed, optional block compression, and finer-grained auxiliary structures. Shows format inefficiencies for common ML workflows and GPU decoding, and derives concrete guidelines for next-generation columnar formats. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
13391
Venue
VLDB
Year
2024
Pagerank
6.1146215e-05
Overall Rank
4,516 | 68.62%
DOI
10.14778/3626292.3626298

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 20 of 20 citing papers.

Rank Citing Paper Year Venue Pagerank
3,419 LeCo: Lightweight Compression via Learning Serial Correlations 2024 SIGMOD 7.1127752e-05
5,571 A Deep Dive into Common Open Formats for Analytical DBMSs 2023 VLDB 5.4279553e-05
7,470 Bullion: A Column Store for Machine Learning 2025 CIDR 4.7159125e-05
7,866 Two Birds With One Stone: Designing a Hybrid Cloud Storage Engine for HTAP 2024 VLDB 4.6282779e-05
9,127 Apache TsFile: An IoT-native Time Series File Format 2024 VLDB 4.3867836e-05
9,203 F3: The Open-Source Data File Format for the Future 2026 SIGMOD 4.3701616e-05
9,646 The FastLanes File Format 2025 VLDB 4.3067693e-05
9,700 Towards Functional Decomposition of Storage Formats 2025 CIDR 4.2967256e-05
10,105 RABIT: Efficient Range Queries with Bitmap Indexing 2026 SIGMOD 4.1905499e-05
10,175 Improving LZ4 for Effective Compression and Efficient Query 2026 SIGMOD 4.1905499e-05
10,193 Predictive Translation: High-Performance Buffer Management Without the Trade-Offs 2026 SIGMOD 4.1905499e-05
10,196 PTO: A Workload-driven Predictive Table Optimizer for Lakehouse Systems 2026 SIGMOD 4.1905499e-05
10,220 FlatStor: An Efficient Embedded-Index Based Columnar Data Layout for Multimodal Data Workloads 2026 VLDB 4.1905499e-05
10,241 Robust Predicate Transfer with Dynamic Execution 2026 VLDB 4.1905499e-05
10,384 Data Chunk Compaction in Vectorized Execution 2025 SIGMOD 4.1905499e-05
10,494 Femur: A Flexible Framework for Fast and Secure Querying from Public Key-Value Store 2025 SIGMOD 4.1905499e-05
10,504 Nested Parquet Is Flat, Why Not Use It? How To Scan Nested Data With On-the-Fly Key Generation and Joins 2025 SIGMOD 4.1905499e-05
10,773 The HANA Native Query Engine for Lakehouse Systems 2025 VLDB 4.1905499e-05
10,858 LiquidCache: Efficient Pushdown Caching for Cloud-Native Data Analytics 2025 VLDB 4.1905499e-05
10,860 Analyzing Near-Network Hardware Acceleration with Co-Processing on DPUs 2025 VLDB 4.1905499e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 37 of 37 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
20 C-Store: A Column-oriented DBMS 2005 VLDB 0.00086163998
70 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00059744625
80 Weaving Relations for Cache Performance 2001 VLDB 0.00055735291
109 Dremel: Interactive Analysis of Web-Scale Datasets 2010 VLDB 0.00048217028
123 A Decomposition Storage Model 1985 SIGMOD 0.00045235743
132 Integrating Compression and Execution in Column-Oriented Database Systems 2006 SIGMOD 0.00043697853
167 The Snowflake Elastic Data Warehouse 2016 SIGMOD 0.00039408116
211 Gorilla: A Fast, Scalable, In-Memory Time Series Database 2015 VLDB 0.0003401421
424 Amazon Redshift and the Case for Simpler Data Warehouses 2015 SIGMOD 0.00023604384
494 Milvus: A Purpose-Built Vector Data Management System 2021 SIGMOD 0.00021769407
739 Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores 2020 VLDB 0.00017365933
1,169 SuRF: Practical Range Query Filtering with Fast Succinct Tries 2018 SIGMOD 0.00013530267
1,267 BitWeaving: Fast Scans for Main Memory Data Processing 2013 SIGMOD 0.00012917585
1,356 Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics 2021 CIDR 0.00012409986
1,939 Procella: Unifying serving and analytical data at YouTube 2019 VLDB 0.00010019433
1,993 Column Imprints: A Secondary Index Structure 2013 SIGMOD 9.8403548e-05
2,044 A Study of the Fundamental Performance Characteristics of GPUs and CPUs for Database Analytics 2020 SIGMOD 9.6963999e-05
2,060 Dremel: A Decade of Interactive SQL Analysis at Web Scale 2020 VLDB 9.6585115e-05
2,070 Chimp: Efficient Lossless Floating Point Compression for Time Series Databases 2022 VLDB 9.6331032e-05
2,123 SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures 2014 VLDB 9.4836925e-05
2,937 DSB: A Decision Support Benchmark for Workload-Driven and Traditional Database Systems 2021 VLDB 7.8552033e-05
2,998 Major Technical Advancements in Apache Hive 2014 SIGMOD 7.7477264e-05
3,419 LeCo: Lightweight Compression via Learning Serial Correlations 2024 SIGMOD 7.1127752e-05
3,611 Column Sketches: A Scan Accelerator for Rapid and Robust Predicate Evaluation 2018 SIGMOD 6.9178844e-05
3,614 SNARF: A Learning-Enhanced Range Filter 2022 VLDB 6.9124805e-05
3,642 BtrBlocks: Efficient Columnar Compression for Data Lakes 2023 SIGMOD 6.8876984e-05
4,520 The FastLanes Compression Layout: Decoding >100 Billion Integers per Second with Scalar Code 2023 VLDB 6.1119645e-05
4,665 Napa: Powering Scalable Data Warehousing with Robust Query Performance at Google 2021 VLDB 6.0057914e-05
5,018 Orchestrating Data Placement and Query Execution in Heterogeneous CPU-GPU DBMS 2022 VLDB 5.7503878e-05
5,039 Tile-based Lightweight Integer Compression in GPU 2022 SIGMOD 5.7369993e-05
5,847 Order-Preserving Key Compression for In-Memory Search Trees 2020 SIGMOD 5.3040014e-05
6,237 Self-Organizing Data Containers 2022 CIDR 5.1371094e-05
6,366 Good to the Last Bit: Data-Driven Encoding with CodecDB 2021 SIGMOD 5.0892171e-05
6,716 Shared Foundations: Modernizing Meta's Data Lakehouse 2023 CIDR 4.9462431e-05
7,110 Wide Table Layout Optimization based on Column Ordering and Duplication 2017 SIGMOD 4.8228761e-05
7,427 Selection Pushdown in Column Stores using Bit Manipulation Instructions 2023 SIGMOD 4.7282014e-05
8,727 Columnar Formats for Schemaless LSM-based Document Stores 2022 VLDB 4.4534547e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
8,596 LST-Bench: Benchmarking Log-Structured Tables in the Cloud 2024 SIGMOD 4.4839589e-05
9,646 The FastLanes File Format 2025 VLDB 4.3067693e-05
3,642 BtrBlocks: Efficient Columnar Compression for Data Lakes 2023 SIGMOD 6.8876984e-05
9,203 F3: The Open-Source Data File Format for the Future 2026 SIGMOD 4.3701616e-05
6,799 Understanding Insights into the Basic Structure and Essential Issues of Table Placement Methods in Clusters 2013 VLDB 4.9181295e-05
6,668 Mainlining Databases: Supporting Fast Transactional Workloads on Universal Columnar Data File Formats 2021 VLDB 4.9647176e-05
7,427 Selection Pushdown in Column Stores using Bit Manipulation Instructions 2023 SIGMOD 4.7282014e-05
9,700 Towards Functional Decomposition of Storage Formats 2025 CIDR 4.2967256e-05
3,193 Column-Oriented Storage Techniques for MapReduce 2011 VLDB 7.4073719e-05
5,571 A Deep Dive into Common Open Formats for Analytical DBMSs 2023 VLDB 5.4279553e-05