Mainlining Databases: Supporting Fast Transactional Workloads on Universal Columnar Data File Formats

Summary: In-memory DBMS storage emits universal columnar blocks (Apache Arrow) tailored to data usage, reducing OLTP-analytic data movement. Relaxed update formats with lightweight transforms to read-optimized layouts on cold data and fast Arrow exports in NoisePage yield OLTP parity and large export speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h5f8aeee22b38aeff
Venue: VLDB
Year: 2021
Pagerank: 5.8042139e-05
Overall Rank: 6,295 | 57.81%
DOI: 10.14778/3436905.3436913
PDF: Download (CC BY-NC-ND 4.0)

Incoming Non-self Citations Over Time

Authors

1. Tianyu Li (Massachusetts Institute of Technology)
2. Matthew Butrovich (Carnegie Mellon University)
3. Amadou Ngom (Massachusetts Institute of Technology)
4. Wan Shen Lim (Carnegie Mellon University)
5. Wes McKinney (Ursa Labs)
6. Andrew Pavlo (Carnegie Mellon University)

BibTeX Citation

@article{li_vldb21,
        title = {{Mainlining Databases: Supporting Fast Transactional Workloads on Universal Columnar Data File Formats}},
        author = {Li, Tianyu and Butrovich, Matthew and Ngom, Amadou and Lim, Wan Shen and McKinney, Wes and Pavlo, Andrew},
        journal = {PVLDB},
        series = {{VLDB} '21},
        volume = {14},
        number = {4},
        pages = {534--546},
        doi = {10.14778/3436905.3436913},
        url = {https://doi.org/10.14778/3436905.3436913},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 11 of 11 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,373	A Deep Dive into Common Open Formats for Analytical DBMSs	2023	VLDB	6.6191786e-05
4,817	Permutable Compiled Queries: Dynamically Adapting Compiled Queries without Recompiling	2021	VLDB	6.388743e-05
5,788	Diva: Making MVCC Systems HTAP-Friendly	2022	SIGMOD	5.9763511e-05
6,132	ConnectorX: Accelerating Data Loading From Databases to Dataframes	2022	VLDB	5.8593078e-05
6,626	Everything is a Transaction: Unifying Logical Concurrency Control and Physical Data Structure Maintenance in Database Management Systems	2021	CIDR	5.7130318e-05
7,273	Scalable and Robust Snapshot Isolation for High-Performance Storage Engines	2023	VLDB	5.5522008e-05
7,715	Tastes Great! Less Filling! High Performance and Accurate Training Data Collection for Self-Driving Database Management Systems	2022	SIGMOD	5.4556951e-05
8,559	Deploying Computational Storage for HTAP DBMSs Takes More Than Just Computation Offloading	2023	VLDB	5.3015086e-05
9,830	TiQuE: Improving the Transactional Performance of Analytical Systems for True Hybrid Workloads	2023	VLDB	5.1065586e-05
10,631	Counting Is All You Need for Instant Tuple Discovery: Enabling Real-Time HTAP in Standalone DBMSs	2026	SIGMOD	4.9605343e-05
11,258	Moving on From Group Commit: Autonomous Commit Enables High Throughput and Low Latency on NVMe SSDs	2025	SIGMOD	4.9605343e-05

Outgoing Citations (Sorted by Pagerank)

Showing 24 of 24 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
7	Implementation Techniques For Main Memory Database Systems	1984	SIGMOD	0.00081781676
14	MonetDB/X100: Hyper-Pipelining Query Execution	2005	CIDR	0.00063972112
70	The End of an Architectural Era (It’s Time for a Complete Rewrite)	2007	VLDB	0.00037761637
172	High-Performance Concurrency Control Mechanisms for Main-Memory Databases	2012	VLDB	0.00027021768
228	Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems	2015	SIGMOD	0.00023872855
236	OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases	2014	VLDB	0.00023639649
332	Column-Stores vs. Row-Stores: How Different Are They Really?	2008	SIGMOD	0.00020739447
431	HYRISE—A Main Memory Hybrid Storage Engine	2011	VLDB	0.00018411252
447	Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores	2015	VLDB	0.00018126139
549	Faster: A Concurrent Key-Value Store with In-Place Updates	2018	SIGMOD	0.00016530268
574	Efficient Transaction Processing in SAP HANA Database – The End of a Column Store Myth	2012	SIGMOD	0.00016142754
606	Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask	2018	VLDB	0.00015609823
729	Anti-Caching: A New Approach to Database Management System Architecture	2013	VLDB	0.00014393711
863	An Empirical Evaluation of In-Memory Multi-Version Concurrency Control	2017	VLDB	0.00013368866
894	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD	0.00013179915
1,188	Bridging the Archipelago between Row-Stores and Column-Stores for Hybrid Workloads	2016	SIGMOD	0.00011580449
1,328	Building a Bw-Tree Takes More Than Just Buzz Words	2018	SIGMOD	0.00011013165
1,350	Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last	2018	VLDB	0.00010934877
1,383	H2O: A Hands-free Adaptive Store	2014	SIGMOD	0.00010826996
2,390	Low-Overhead Asynchronous Checkpointing in Main-Memory Database Systems	2016	SIGMOD	8.5227929e-05
2,636	Hybrid Garbage Collection for Multi-Version Concurrency Control in SAP HANA	2016	SIGMOD	8.176465e-05
3,308	Compacting Transactional Data in Hybrid OLTP&OLAP Databases	2012	VLDB	7.4195851e-05
3,547	Read-Optimized Databases, In Depth	2008	VLDB	7.1905227e-05
4,132	Don’t Hold My Data Hostage – A Case For Client Protocol Redesign	2017	VLDB	6.7741062e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,942	Instant Loading for Main Memory Databases	2013	VLDB
2	27	Database Architecture Optimized for the New Bottleneck: Memory Access	1999	VLDB
3	3,215	Columnar Storage and List-based Processing for Graph Database Management Systems	2021	VLDB
4	10,299	Shared Load(ing): Efficient Bulk Loading into Optimized Storage	2020	CIDR
5	5,839	Design and Evaluation of Storage Organizations for Read-Optimized Main Memory Databases	2013	VLDB
6	635	Performance Tradeoffs in Read-Optimized Databases	2006	VLDB
7	3,986	The Impact of Columnar In-Memory Databases on Enterprise Systems: Implications of Eliminating Transaction-Maintained Aggregates	2014	VLDB
8	976	A Common Database Approach for OLTP and OLAP Using an In-Memory Column Database	2009	SIGMOD
9	3,809	An Empirical Evaluation of Columnar Storage Formats	2024	VLDB
10	1,188	Bridging the Archipelago between Row-Stores and Column-Stores for Hybrid Workloads	2016	SIGMOD