Dalton: Learned Partitioning for Distributed Data Streams

Summary: Dalton: an RL-based, lightweight partitioner for distributed streams that memoizes recent state to minimize per-tuple overhead and rapidly adapt to unknown, changing hot-key skews. Scales via cooperative learning across instances (no centralized bottleneck), achieving 1.3–6.7× higher throughput. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13307
Venue: VLDB
Year: 2023
Pagerank: 4.2777144e-05
Overall Rank: 9,800 | 31.89%
DOI: 10.14778/3570690.3570699

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,844	Learned Cost Models for Query Optimization: From Batch to Streaming Systems	2025	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
287	Storm @Twitter	2014	SIGMOD	0.00028917909
536	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB	0.00020651621
1,097	Trill: A High-Performance Incremental Query Processor for Diverse Analytics	2015	VLDB	0.00014083973
1,339	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00012492978
3,551	Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems	2018	VLDB	6.9776252e-05
3,771	SABER: Window-Based Hybrid Stream Processing for Heterogeneous Architectures	2016	SIGMOD	6.7738535e-05
4,380	Incremental Record Linkage	2014	VLDB	6.2325868e-05
4,490	Analyzing Efficient Stream Processing on Modern Hardware	2019	VLDB	6.137834e-05
6,628	A Holistic View of Stream Partitioning Costs	2017	VLDB	4.9833118e-05
6,760	AStream: Ad-hoc Shared Stream Processing	2019	SIGMOD	4.9304882e-05
8,594	Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems	2020	SIGMOD	4.4844961e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
6,618	Near-Optimal Distributed Band-Joins through Recursive Partitioning	2020	SIGMOD	4.9864636e-05
7,713	Query Centric Partitioning and Allocation for Partially Replicated Database Systems	2017	SIGMOD	4.6662571e-05
9,318	Providing Resiliency to Load Variations in Distributed Stream Processing	2006	VLDB	4.3523601e-05
3,825	Locality-aware Partitioning in Parallel Database Systems	2015	SIGMOD	6.7225803e-05
8,594	Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems	2020	SIGMOD	4.4844961e-05
2,497	Streaming Graph Partitioning: An Experimental Study	2018	VLDB	8.6427191e-05
3,066	Learning a Partitioning Advisor for Cloud Databases	2020	SIGMOD	7.6255556e-05
6,628	A Holistic View of Stream Partitioning Costs	2017	VLDB	4.9833118e-05
7,366	Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning	2018	VLDB	4.7459576e-05
3,384	Scalable and Adaptive Online Joins	2014	VLDB	7.153329e-05