Dalton: Learned Partitioning for Distributed Data Streams

Summary: Dalton: an RL-based, lightweight partitioner for distributed streams that memoizes recent state to minimize per-tuple overhead and rapidly adapt to unknown, changing hot-key skews. Scales via cooperative learning across instances (no centralized bottleneck), achieving 1.3–6.7× higher throughput. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13306
Venue: VLDB
Year: 2023
Pagerank: 4.2818172e-05
Overall Rank: 9,797 | 31.85%
DOI: 10.14778/3570690.3570699

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,840	Learned Cost Models for Query Optimization: From Batch to Streaming Systems	2025	VLDB	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
288	Storm @Twitter	2014	SIGMOD	0.00028939871
538	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB	0.00020678804
1,098	Trill: A High-Performance Incremental Query Processor for Diverse Analytics	2015	VLDB	0.00014114442
1,334	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.0001250413
3,550	Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems	2018	VLDB	6.9843512e-05
3,762	SABER: Window-Based Hybrid Stream Processing for Heterogeneous Architectures	2016	SIGMOD	6.7804471e-05
4,383	Incremental Record Linkage	2014	VLDB	6.2383094e-05
4,488	Analyzing Efficient Stream Processing on Modern Hardware	2019	VLDB	6.145117e-05
6,629	A Holistic View of Stream Partitioning Costs	2017	VLDB	4.9880986e-05
6,759	AStream: Ad-hoc Shared Stream Processing	2019	SIGMOD	4.9352213e-05
8,596	Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems	2020	SIGMOD	4.4887993e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
6,619	Near-Optimal Distributed Band-Joins through Recursive Partitioning	2020	SIGMOD	4.9910152e-05
7,715	Query Centric Partitioning and Allocation for Partially Replicated Database Systems	2017	SIGMOD	4.6699261e-05
9,313	Providing Resiliency to Load Variations in Distributed Stream Processing	2006	VLDB	4.3565355e-05
3,821	Locality-aware Partitioning in Parallel Database Systems	2015	SIGMOD	6.7281515e-05
8,596	Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems	2020	SIGMOD	4.4887993e-05
2,494	Streaming Graph Partitioning: An Experimental Study	2018	VLDB	8.6508229e-05
3,076	Learning a Partitioning Advisor for Cloud Databases	2020	SIGMOD	7.6107677e-05
6,629	A Holistic View of Stream Partitioning Costs	2017	VLDB	4.9880986e-05
7,372	Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning	2018	VLDB	4.7496881e-05
3,382	Scalable and Adaptive Online Joins	2014	VLDB	7.1597145e-05