SplitDF: Splitting Dataframes for Memory-Efficient Data Analysis

Summary: Introduce “splitting”: lossless join decomposition adding join keys to reduce dataframe redundancy while preserving a unified tabular view, no FD discovery required. SplitDF (Ibis/DuckDB) with automated SplitGen (Velox) achieves 19–61% memory savings with minimal API change. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13449
Venue: VLDB
Year: 2024
Pagerank: 4.1945683e-05
Overall Rank: 11,024 | 23.31%
DOI: 10.14778/3665844.3665849

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Aarati Kakaraparthy
2. Jignesh M. Patel

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
167	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00039180521
185	DuckDB: an Embeddable Analytical Database	2019	SIGMOD	0.00036538405
224	CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies	2004	SIGMOD	0.00032746205
894	A Hybrid Approach to Functional Dependency Discovery	2016	SIGMOD	0.00015556428
1,047	Functional Dependency Discovery: An Experimental Evaluation of Seven Algorithms	2015	VLDB	0.00014459715
1,427	Towards Scalable Dataframe Systems	2020	VLDB	0.0001204248
1,625	Data Profiling with Metanome	2015	VLDB	0.00011094926
2,077	Efficient Discovery of Approximate Dependencies	2018	VLDB	9.6001836e-05
2,528	Velox: Meta’s Unified Execution Engine	2022	VLDB	8.59454e-05
2,549	GORDIAN: Efficient and Scalable Discovery of Composite Keys	2006	VLDB	8.5641554e-05
2,954	Magpie: Python at Speed and Scale using Cloud Backends	2021	CIDR	7.8262582e-05
4,127	A Statistical Perspective on Discovering Functional Dependencies in Noisy Data	2020	SIGMOD	6.4310458e-05
4,773	PolyFrame: A Retargetable Query-based Approach to Scaling Dataframes	2021	VLDB	5.9320139e-05
9,416	When sweet and cute isn't enough anymore: Solving scalability issues in Python Pandas with Grizzly	2020	CIDR	4.3441378e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
5,118	AdaptDB: Adaptive Partitioning for Distributed Joins	2017	VLDB	5.6820984e-05
8,275	Adaptive Factorization Using Linear-Chained Hash Tables	2025	CIDR	4.5439841e-05
2,443	Data Management for Data Science: Towards Embedded Analytics	2020	CIDR	8.8078476e-05
4,773	PolyFrame: A Retargetable Query-based Approach to Scaling Dataframes	2021	VLDB	5.9320139e-05
10,372	Data Chunk Compaction in Vectorized Execution	2025	SIGMOD	4.1945683e-05
3,763	Flexible Rule-Based Decomposition and Metadata Independence in Modin: A Parallel Dataframe System	2022	VLDB	6.7801795e-05
1,427	Towards Scalable Dataframe Systems	2020	VLDB	0.0001204248
10,635	Saving Private Hash Join	2025	VLDB	4.1945683e-05
6,541	ConnectorX: Accelerating Data Loading From Databases to Dataframes	2022	VLDB	5.0216945e-05
8,915	DQDF: Data-Quality-Aware Dataframes	2022	VLDB	4.427232e-05