Back to papers
A Deep Dive into Common Open Formats for Analytical DBMSs
Summary: Systematic evaluation of Arrow, Parquet, and ORC against OLAP DBMS requirements, showing how layout, vectorization, compression, metadata, and mmap trade-offs affect query efficiency and integration. Identifies co‑design opportunities for unified in‑memory/on‑disk representation and practical guidance for implementers.
(summarized by gpt-5-mini on Feb 09 2026)
- Paper ID
- 13145
- Venue
- VLDB
- Year
- 2023
- Pagerank
- 5.4279553e-05
- Overall Rank
- 5,571 | 61.29%
- DOI
-
10.14778/3611479.3611507
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 11 of 11 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 4,498 |
ClickHouse - Lightning Fast Analytics for Everyone |
2024 |
VLDB |
6.1351257e-05 |
| 7,866 |
Two Birds With One Stone: Designing a Hybrid Cloud Storage Engine for HTAP |
2024 |
VLDB |
4.6282779e-05 |
| 9,127 |
Apache TsFile: An IoT-native Time Series File Format |
2024 |
VLDB |
4.3867836e-05 |
| 9,203 |
F3: The Open-Source Data File Format for the Future |
2026 |
SIGMOD |
4.3701616e-05 |
| 9,646 |
The FastLanes File Format |
2025 |
VLDB |
4.3067693e-05 |
| 9,700 |
Towards Functional Decomposition of Storage Formats |
2025 |
CIDR |
4.2967256e-05 |
| 10,220 |
FlatStor: An Efficient Embedded-Index Based Columnar Data Layout for Multimodal Data Workloads |
2026 |
VLDB |
4.1905499e-05 |
| 10,504 |
Nested Parquet Is Flat, Why Not Use It? How To Scan Nested Data With On-the-Fly Key Generation and Joins |
2025 |
SIGMOD |
4.1905499e-05 |
| 10,748 |
Beyond Compression: A Comprehensive Evaluation of Lossless Floating-Point Compression |
2025 |
VLDB |
4.1905499e-05 |
| 10,858 |
LiquidCache: Efficient Pushdown Caching for Cloud-Native Data Analytics |
2025 |
VLDB |
4.1905499e-05 |
| 10,860 |
Analyzing Near-Network Hardware Acceleration with Co-Processing on DPUs |
2025 |
VLDB |
4.1905499e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 26 of 26 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank |
Cited Paper |
Year |
Venue |
Pagerank |
| 109 |
Dremel: Interactive Analysis of Web-Scale Datasets |
2010 |
VLDB |
0.00048217028 |
| 132 |
Integrating Compression and Execution in Column-Oriented Database Systems |
2006 |
SIGMOD |
0.00043697853 |
| 424 |
Amazon Redshift and the Case for Simpler Data Warehouses |
2015 |
SIGMOD |
0.00023604384 |
| 496 |
Column-Stores vs. Row-Stores: How Different Are They Really? |
2008 |
SIGMOD |
0.00021705611 |
| 659 |
The Making of TPC-DS |
2006 |
VLDB |
0.00018514913 |
| 1,267 |
BitWeaving: Fast Scans for Main Memory Data Processing |
2013 |
SIGMOD |
0.00012917585 |
| 1,356 |
Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics |
2021 |
CIDR |
0.00012409986 |
| 1,608 |
Qd-tree: Learning Data Layouts for Big Data Analytics |
2020 |
SIGMOD |
0.00011169837 |
| 2,123 |
SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures |
2014 |
VLDB |
9.4836925e-05 |
| 2,262 |
SQL Server Column Store Indexes |
2011 |
SIGMOD |
9.1629311e-05 |
| 2,533 |
Velox: Meta’s Unified Execution Engine |
2022 |
VLDB |
8.5870599e-05 |
| 2,619 |
Decomposed Bounded Floats for Fast Compression and Queries |
2021 |
VLDB |
8.4427442e-05 |
| 3,044 |
Azure Data Lake Store: A Hyperscale Distributed File Service for Big Data Analytics |
2017 |
SIGMOD |
7.6624689e-05 |
| 3,611 |
Column Sketches: A Scan Accelerator for Rapid and Robust Predicate Evaluation |
2018 |
SIGMOD |
6.9178844e-05 |
| 4,516 |
An Empirical Evaluation of Columnar Storage Formats |
2024 |
VLDB |
6.1146215e-05 |
| 4,672 |
FlexPushdownDB: Hybrid Pushdown and Caching in a Cloud DBMS |
2021 |
VLDB |
6.001444e-05 |
| 5,105 |
Analyzing and Comparing Lakehouse Storage Systems |
2023 |
CIDR |
5.6916018e-05 |
| 5,124 |
Accelerating Generalized Linear Models with MLWeaving: A One-Size-Fits-All System for Any-Precision Learning |
2019 |
VLDB |
5.6747536e-05 |
| 5,879 |
Column Partition and Permutation for Run Length Encoding in Columnar Databases |
2020 |
SIGMOD |
5.2875479e-05 |
| 6,237 |
Self-Organizing Data Containers |
2022 |
CIDR |
5.1371094e-05 |
| 6,311 |
VergeDB: A Database for IoT Analytics on Edge Devices |
2021 |
CIDR |
5.1112212e-05 |
| 6,366 |
Good to the Last Bit: Data-Driven Encoding with CodecDB |
2021 |
SIGMOD |
5.0892171e-05 |
| 6,668 |
Mainlining Databases: Supporting Fast Transactional Workloads on Universal Columnar Data File Formats |
2021 |
VLDB |
4.9647176e-05 |
| 7,127 |
Jigsaw: A Data Storage and Query Processing Engine for Irregular Table Partitioning |
2021 |
SIGMOD |
4.8184276e-05 |
| 7,431 |
CompressDB: Enabling Efficient Compressed Data Direct Processing for Various Databases |
2022 |
SIGMOD |
4.7274757e-05 |
| 8,090 |
PIDS: Attribute Decomposition for Improved Compression and Query Performance in Columnar Storage |
2020 |
VLDB |
4.5853298e-05 |
Semantically Similar Papers
| Overall Rank |
Paper |
Year |
Venue |
Pagerank |
| 638 |
Towards a Unified Architecture for in-RDBMS Analytics |
2012 |
SIGMOD |
0.00018810785 |
| 9,700 |
Towards Functional Decomposition of Storage Formats |
2025 |
CIDR |
4.2967256e-05 |
| 2,998 |
Major Technical Advancements in Apache Hive |
2014 |
SIGMOD |
7.7477264e-05 |
| 6,332 |
Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine |
2024 |
SIGMOD |
5.1021765e-05 |
| 1,050 |
A Common Database Approach for OLTP and OLAP Using an In-Memory Column Database |
2009 |
SIGMOD |
0.00014425459 |
| 10,248 |
Active Data Lakes: Regaining Physical Data Independence Without Losing Interoperability |
2026 |
VLDB |
4.1905499e-05 |
| 7,864 |
Operational Analytics Data Management Systems |
2016 |
VLDB |
4.6284093e-05 |
| 6,668 |
Mainlining Databases: Supporting Fast Transactional Workloads on Universal Columnar Data File Formats |
2021 |
VLDB |
4.9647176e-05 |
| 3,753 |
Choosing A Cloud DBMS: Architectures and Tradeoffs |
2019 |
VLDB |
6.7850001e-05 |
| 4,516 |
An Empirical Evaluation of Columnar Storage Formats |
2024 |
VLDB |
6.1146215e-05 |