Data Partitioning for In-Memory Systems: Myths, Challenges, and Opportunities

Summary: Re-evaluates partitioning for multi-core in-memory DBs, exposing overlooked factors (tuple size, row vs column) that invalidate narrow prior studies. Proposes a broad benchmark and shows simple partitioning methods are often robust across realistic settings. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: h0b0d58a50138a47f
Venue: CIDR
Year: 2019
Pagerank: 5.8261473e-05
Overall Rank: 6,228 | 58.26%
DOI: -
PDF: Download (CC BY 3.0)

Incoming Non-self Citations Over Time

Authors

1. Zuyu Zhang (University of Wisconsin)
2. Harshad Deshmukh (University of Wisconsin)
3. Jignesh M. Patel (University of Wisconsin)

BibTeX Citation

@inproceedings{zhang_cidr19,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '19},
        title = {{Data Partitioning for In-Memory Systems: Myths, Challenges, and Opportunities}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Zhang, Zuyu and Deshmukh, Harshad and Patel, Jignesh M.},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 6 of 6 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,603	Adopting Worst-Case Optimal Joins in Relational Database Systems	2020	VLDB	0.00010090639
2,827	To Partition, or Not to Partition, That is the Join Question in a Real System	2021	SIGMOD	7.9486247e-05
3,622	Triton Join: Efficiently Scaling to a Large Join State on GPUs with Fast Interconnects	2022	SIGMOD	7.1350619e-05
7,122	Design and Analysis of a Processing-in-DIMM Join Algorithm: A Case Study with UPMEM DIMMs	2023	SIGMOD	5.580601e-05
7,927	A Practical Approach to Groupjoin and Nested Aggregates	2021	VLDB	5.4082473e-05
9,093	A Design Space Exploration and Evaluation for Main-Memory Hash Joins in Storage Class Memory	2023	VLDB	5.208561e-05

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
27	Database Architecture Optimized for the New Bottleneck: Memory Access	1999	VLDB	0.0005146739
77	Cache Conscious Algorithms for Relational Query Processing	1994	VLDB	0.00036821827
212	Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs	2009	VLDB	0.00024786576
250	Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited	2014	VLDB	0.00023080446
360	Design and Evaluation of Main Memory Hash Join Algorithms for Multi-core CPUs	2011	SIGMOD	0.00019959166
423	Massively Parallel Sort-Merge Joins in Main Memory Multi-Core Database Systems	2012	VLDB	0.00018450254
627	Rethinking SIMD Vectorization for In-Memory Databases	2015	SIGMOD	0.00015430074
662	Fast Sort on CPUs and GPUs: A Case for Bandwidth Oblivious SIMD Sort	2010	SIGMOD	0.00014972958
722	Efficient Implementation of Sorting on Multi-Core SIMD CPU Architecture	2008	VLDB	0.00014470465
997	What happens during a Join? Dissecting CPU and Memory Optimization Effects	2000	VLDB	0.00012595178
1,119	A Comprehensive Study of Main-Memory Partitioning and its Application to Large-Scale Comparison- and Radix-Sort	2014	SIGMOD	0.00011927356
1,213	A Seven-Dimensional Analysis of Hashing Methods and its Implications on Query Processing	2016	VLDB	0.00011492443
1,269	An Experimental Comparison of Thirteen Relational Equi-Joins in Main Memory	2016	SIGMOD	0.00011236561
2,485	A Memory Bandwidth-Efficient Hybrid Radix Sort on GPUs	2017	SIGMOD	8.3800709e-05
2,806	Distributed Join Algorithms on Thousands of Cores	2017	VLDB	7.9657254e-05
4,084	PARADIS: An Efficient Parallel Algorithm for In-place Radix Sort	2015	VLDB	6.7991616e-05
4,958	FPGA-based Data Partitioning	2017	SIGMOD	6.3254925e-05
5,202	On the Surprising Difficulty of Simple Things: the Case of Radix Partitioning	2015	VLDB	6.2175649e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,585	On the Design and Scalability of Distributed Shared-Data Databases	2015	SIGMOD
2	2,823	Horizontal Data Partitioning In Database Design	1982	SIGMOD
3	7,725	Query Centric Partitioning and Allocation for Partially Replicated Database Systems	2017	SIGMOD
4	4,707	Adaptive NUMA-aware data placement and task scheduling for analytical workloads in main-memory column-stores	2017	VLDB
5	3,094	Skipping-oriented Partitioning for Columnar Layouts	2017	VLDB
6	5,202	On the Surprising Difficulty of Simple Things: the Case of Radix Partitioning	2015	VLDB
7	3,258	Locality-aware Partitioning in Parallel Database Systems	2015	SIGMOD
8	1,892	Automated Partitioning Design in Parallel Database Systems	2011	SIGMOD
9	7,958	A Non-Uniform Data Fragmentation Strategy for Parallel Main-Memory Database Systems	1995	VLDB
10	1,119	A Comprehensive Study of Main-Memory Partitioning and its Application to Large-Scale Comparison- and Radix-Sort	2014	SIGMOD