Dalton: Learned Partitioning for Distributed Data Streams

Summary: Dalton: an RL-based, lightweight partitioner for distributed streams that memoizes recent state to minimize per-tuple overhead and rapidly adapt to unknown, changing hot-key skews. Scales via cooperative learning across instances (no centralized bottleneck), achieving 1.3–6.7× higher throughput. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13494
Venue: VLDB
Year: 2023
Pagerank: 5.3189314e-05
Overall Rank: 9,128 | 37.38%
DOI: 10.14778/3570690.3570699

Incoming Non-self Citations Over Time

Authors

1. Eleni Zapridou (EPFL)
2. Ioannis Mytilinis (Oracle)
3. Anastasia Ailamaki (EPFL)

BibTeX Citation

@article{zapridou_vldb23,
        title = {{Dalton: Learned Partitioning for Distributed Data Streams}},
        author = {Zapridou, Eleni and Mytilinis, Ioannis and Ailamaki, Anastasia},
        journal = {PVLDB},
        series = {{VLDB} '23},
        volume = {16},
        number = {3},
        pages = {491--504},
        doi = {10.14778/3570690.3570699},
        url = {https://doi.org/10.14778/3570690.3570699},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,306	Workload-Aware Incremental Reclustering in Cloud Data Warehouses	2026	SIGMOD	5.093636e-05
11,065	Learned Cost Models for Query Optimization: From Batch to Streaming Systems	2025	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
220	Storm @Twitter	2014	SIGMOD	0.00024244587
361	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB	0.00020138717
710	Trill: A High-Performance Incremental Query Processor for Diverse Analytics	2015	VLDB	0.00014715033
1,319	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00011175005
3,060	SABER: Window-Based Hybrid Stream Processing for Heterogeneous Architectures	2016	SIGMOD	7.8062155e-05
3,117	Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems	2018	VLDB	7.7382559e-05
3,851	Analyzing Efficient Stream Processing on Modern Hardware	2019	VLDB	7.0735328e-05
3,916	Incremental Record Linkage	2014	VLDB	7.0185679e-05
6,410	A Holistic View of Stream Partitioning Costs	2017	VLDB	5.884019e-05
6,482	AStream: Ad-hoc Shared Stream Processing	2019	SIGMOD	5.8668582e-05
8,568	Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems	2020	SIGMOD	5.4118703e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,596	Near-Optimal Distributed Band-Joins through Recursive Partitioning	2020	SIGMOD
2	7,585	Query Centric Partitioning and Allocation for Partially Replicated Database Systems	2017	SIGMOD
3	9,460	Providing Resiliency to Load Variations in Distributed Stream Processing	2006	VLDB
4	3,200	Locality-aware Partitioning in Parallel Database Systems	2015	SIGMOD
5	8,568	Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems	2020	SIGMOD
6	2,221	Streaming Graph Partitioning: An Experimental Study	2018	VLDB
7	2,499	Learning a Partitioning Advisor for Cloud Databases	2020	SIGMOD
8	6,410	A Holistic View of Stream Partitioning Costs	2017	VLDB
9	7,194	Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning	2018	VLDB
10	2,892	Scalable and Adaptive Online Joins	2014	VLDB