Data Partitioning for In-Memory Systems: Myths, Challenges, and Opportunities

Summary: Re-evaluates partitioning for multi-core in-memory DBs, exposing overlooked factors (tuple size, row vs column) that invalidate narrow prior studies. Proposes a broad benchmark and shows simple partitioning methods are often robust across realistic settings. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 323
Venue: CIDR
Year: 2019
Pagerank: 5.9537202e-05
Overall Rank: 6,161 | 57.74%
DOI: -

Incoming Non-self Citations Over Time

Authors

1. Zuyu Zhang (University of Wisconsin)
2. Harshad Deshmukh (University of Wisconsin)
3. Jignesh M. Patel (University of Wisconsin)

BibTeX Citation

@inproceedings{zhang_cidr19,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '19},
        title = {{Data Partitioning for In-Memory Systems: Myths, Challenges, and Opportunities}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Zhang, Zuyu and Deshmukh, Harshad and Patel, Jignesh M.},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 6 of 6 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,740	Adopting Worst-Case Optimal Joins in Relational Database Systems	2020	VLDB	9.875587e-05
2,962	To Partition, or Not to Partition, That is the Join Question in a Real System	2021	SIGMOD	7.9170451e-05
4,535	Triton Join: Efficiently Scaling to a Large Join State on GPUs with Fast Interconnects	2022	SIGMOD	6.6419266e-05
6,957	Design and Analysis of a Processing-in-DIMM Join Algorithm: A Case Study with UPMEM DIMMs	2023	SIGMOD	5.7303405e-05
7,787	A Practical Approach to Groupjoin and Nested Aggregates	2021	VLDB	5.5449593e-05
8,904	A Design Space Exploration and Evaluation for Main-Memory Hash Joins in Storage Class Memory	2023	VLDB	5.3483178e-05

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
29	Database Architecture Optimized for the New Bottleneck: Memory Access	1999	VLDB	0.00052093615
74	Cache Conscious Algorithms for Relational Query Processing	1994	VLDB	0.00037330605
209	Sort vs. Hash Revisited: Fast Join Implementation on Modern Multi-Core CPUs	2009	VLDB	0.00024932174
252	Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited	2014	VLDB	0.00023242719
360	Design and Evaluation of Main Memory Hash Join Algorithms for Multi-core CPUs	2011	SIGMOD	0.00020182846
423	Massively Parallel Sort-Merge Joins in Main Memory Multi-Core Database Systems	2012	VLDB	0.00018725853
634	Rethinking SIMD Vectorization for In-Memory Databases	2015	SIGMOD	0.00015533814
678	Fast Sort on CPUs and GPUs: A Case for Bandwidth Oblivious SIMD Sort	2010	SIGMOD	0.00015061068
712	Efficient Implementation of Sorting on Multi-Core SIMD CPU Architecture	2008	VLDB	0.0001468812
987	What happens during a Join? Dissecting CPU and Memory Optimization Effects	2000	VLDB	0.00012814017
1,177	A Comprehensive Study of Main-Memory Partitioning and its Application to Large-Scale Comparison- and Radix-Sort	2014	SIGMOD	0.00011808761
1,265	An Experimental Comparison of Thirteen Relational Equi-Joins in Main Memory	2016	SIGMOD	0.00011415709
1,278	A Seven-Dimensional Analysis of Hashing Methods and its Implications on Query Processing	2016	VLDB	0.00011362007
2,667	A Memory Bandwidth-Efficient Hybrid Radix Sort on GPUs	2017	SIGMOD	8.2756346e-05
2,926	Distributed Join Algorithms on Thousands of Cores	2017	VLDB	7.9549783e-05
4,021	PARADIS: An Efficient Parallel Algorithm for In-place Radix Sort	2015	VLDB	6.9501893e-05
4,840	FPGA-based Data Partitioning	2017	SIGMOD	6.483442e-05
5,094	On the Surprising Difficulty of Simple Things: the Case of Radix Partitioning	2015	VLDB	6.3657592e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,637	On the Design and Scalability of Distributed Shared-Data Databases	2015	SIGMOD
2	2,855	Horizontal Data Partitioning In Database Design	1982	SIGMOD
3	7,585	Query Centric Partitioning and Allocation for Partially Replicated Database Systems	2017	SIGMOD
4	4,607	Adaptive NUMA-aware data placement and task scheduling for analytical workloads in main-memory column-stores	2017	VLDB
5	3,106	Skipping-oriented Partitioning for Columnar Layouts	2017	VLDB
6	5,094	On the Surprising Difficulty of Simple Things: the Case of Radix Partitioning	2015	VLDB
7	3,200	Locality-aware Partitioning in Parallel Database Systems	2015	SIGMOD
8	1,852	Automated Partitioning Design in Parallel Database Systems	2011	SIGMOD
9	7,801	A Non-Uniform Data Fragmentation Strategy for Parallel Main-Memory Database Systems	1995	VLDB
10	1,177	A Comprehensive Study of Main-Memory Partitioning and its Application to Large-Scale Comparison- and Radix-Sort	2014	SIGMOD