DBScholar

Back to papers

Megaphone: Latency-conscious state migration for distributed streaming dataflows

Summary: Megaphone enables latency-conscious state migration for streaming dataflows by subdividing migrations into configurable granularity to limit spikes. Ahead-of-time prep; unmodified Timely dataflow library with a compatible interface, delivering large latency reductions and modest overhead. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h1f965962ebdda413
Venue
VLDB
Year
2019
Pagerank
7.3318754e-05
Overall Rank
3,398 | 77.17%
DOI
10.14778/3329772.3329777
PDF
Download (CC BY-NC-ND 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{hoffmann_vldb19,
        title = {{Megaphone: Latency-conscious state migration for distributed streaming dataflows}},
        author = {Hoffmann, Moritz and Lattuada, Andrea and McSherry, Frank and Kalavri, Vasiliki and Liagouris, John and Roscoe, Timothy},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {9},
        pages = {1002--1015},
        doi = {10.14778/3329772.3329777},
        url = {https://doi.org/10.14778/3329772.3329777},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 16 of 16 citing papers.

Rank Citing Paper Year Venue Pagerank
4,110 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD 6.797656e-05
4,453 Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka 2021 SIGMOD 6.59179e-05
5,939 StreamOps: Cloud-Native Runtime Management for Streaming Services in ByteDance 2023 VLDB 5.9378976e-05
6,270 Beyond Analytics: The Evolution of Stream Processing Systems 2020 SIGMOD 5.8275729e-05
8,515 Tackling Hardware/Software co-design from a database perspective 2020 CIDR 5.3242523e-05
9,817 Scabbard: Single-Node Fault-Tolerant Stream Processing 2022 VLDB 5.1233734e-05
10,089 ContTune: Continuous Tuning by Conservative Bayesian Optimization for Distributed Stream Data Processing Systems 2023 VLDB 5.0806786e-05
10,228 Towards Resource-adaptive Query Execution in Cloud Native Databases 2024 CIDR 5.054828e-05
10,552 Accelerating Stream Processing Engines via Hardware Offloading 2026 SIGMOD 4.9769913e-05
10,743 APEROL: Adaptive Parallel Edge-to-cloud Runtime Optimization for Layered Workflow Execution 2026 VLDB 4.9769913e-05
10,783 Incremental Stream Query Deployment under Continuous Infrastructure Changes in the Cloud-Edge Continuum 2026 VLDB 4.9769913e-05
10,855 Meerkat: Scalable, Network-Aware Failure Recovery for the Internet of Things 2026 VLDB 4.9769913e-05
10,871 Fugue: Online Elasticity for Distributed Stateful Stream Processing 2026 VLDB 4.9769913e-05
11,087 Transactional Cloud Applications Go with the (Data)Flow 2025 CIDR 4.9769913e-05
11,208 Styx: Transactional Stateful Functions on Streaming Dataflows 2025 SIGMOD 4.9769913e-05
11,761 Fries: Fast and Consistent Runtime Reconfiguration in Dataflow Systems with Transactional Guarantees 2023 VLDB 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
218 MillWheel: Fault-Tolerant Stream Processing at Internet Scale 2013 VLDB 0.00024379041
325 The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing 2015 VLDB 0.0002095522
739 Zephyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms 2011 SIGMOD 0.00014327549
893 Albatross: Lightweight Elasticity in Shared Storage Databases for the Cloud using Live Data Migration 2011 VLDB 0.00013213905
970 Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management 2013 SIGMOD 0.00012776109
1,123 Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark 2018 SIGMOD 0.00011926683
1,236 Dhalion: Self-Regulating Stream Processing in Heron 2017 VLDB 0.00011400483
1,402 State Management in Apache Flink: Consistent Stateful Distributed Stream Processing 2017 VLDB 0.00010766851
2,425 Squall: Fine-Grained Live Reconfiguration for Partitioned Main Memory Databases 2015 SIGMOD 8.4820179e-05
3,034 Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems 2018 VLDB 7.7301387e-05
3,371 Dynamic Plan Migration for Continuous Queries Over Data Streams 2004 SIGMOD 7.3659127e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 970 Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management 2013 SIGMOD
2 5,296 Optimizing Data-intensive Systems in Disaggregated Data Centers with TELEPORT 2022 SIGMOD
3 3,371 Dynamic Plan Migration for Continuous Queries Over Data Streams 2004 SIGMOD
4 10,162 Railgun: managing large streaming windows under MAD requirements 2021 VLDB
5 8,410 Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines 2020 SIGMOD
6 9,041 Stateful Entities: Object-oriented Cloud Applications as Distributed Dataflows 2023 CIDR
7 9,823 Supporting Scalable Analytics with Latency Constraints 2015 VLDB
8 11,208 Styx: Transactional Stateful Functions on Streaming Dataflows 2025 SIGMOD
9 10,871 Fugue: Online Elasticity for Distributed Stateful Stream Processing 2026 VLDB
10 4,110 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD