DBScholar

Back to papers

Megaphone: Latency-conscious state migration for distributed streaming dataflows

Summary: Megaphone enables latency-conscious state migration for streaming dataflows by subdividing migrations into configurable granularity to limit spikes. Ahead-of-time prep; unmodified Timely dataflow library with a compatible interface, delivering large latency reductions and modest overhead. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h1f965962ebdda413
Venue
VLDB
Year
2019
Pagerank
7.3076317e-05
Overall Rank
3,409 | 77.16%
DOI
10.14778/3329772.3329777
PDF
Download (CC BY-NC-ND 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{hoffmann_vldb19,
        title = {{Megaphone: Latency-conscious state migration for distributed streaming dataflows}},
        author = {Hoffmann, Moritz and Lattuada, Andrea and McSherry, Frank and Kalavri, Vasiliki and Liagouris, John and Roscoe, Timothy},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {9},
        pages = {1002--1015},
        doi = {10.14778/3329772.3329777},
        url = {https://doi.org/10.14778/3329772.3329777},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 16 of 16 citing papers.

Rank Citing Paper Year Venue Pagerank
4,129 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD 6.7751787e-05
4,475 Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka 2021 SIGMOD 6.5699934e-05
5,952 StreamOps: Cloud-Native Runtime Management for Streaming Services in ByteDance 2023 VLDB 5.9182633e-05
6,281 Beyond Analytics: The Evolution of Stream Processing Systems 2020 SIGMOD 5.8083033e-05
8,530 Tackling Hardware/Software co-design from a database perspective 2020 CIDR 5.3067157e-05
9,851 Scabbard: Single-Node Fault-Tolerant Stream Processing 2022 VLDB 5.1064323e-05
10,123 ContTune: Continuous Tuning by Conservative Bayesian Optimization for Distributed Stream Data Processing Systems 2023 VLDB 5.0638787e-05
10,259 Towards Resource-adaptive Query Execution in Cloud Native Databases 2024 CIDR 5.0381136e-05
10,617 Accelerating Stream Processing Engines via Hardware Offloading 2026 SIGMOD 4.9605343e-05
10,804 APEROL: Adaptive Parallel Edge-to-cloud Runtime Optimization for Layered Workflow Execution 2026 VLDB 4.9605343e-05
10,844 Incremental Stream Query Deployment under Continuous Infrastructure Changes in the Cloud-Edge Continuum 2026 VLDB 4.9605343e-05
10,916 Meerkat: Scalable, Network-Aware Failure Recovery for the Internet of Things 2026 VLDB 4.9605343e-05
10,932 Fugue: Online Elasticity for Distributed Stateful Stream Processing 2026 VLDB 4.9605343e-05
11,148 Transactional Cloud Applications Go with the (Data)Flow 2025 CIDR 4.9605343e-05
11,267 Styx: Transactional Stateful Functions on Streaming Dataflows 2025 SIGMOD 4.9605343e-05
11,805 Fries: Fast and Consistent Runtime Reconfiguration in Dataflow Systems with Transactional Guarantees 2023 VLDB 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
219 MillWheel: Fault-Tolerant Stream Processing at Internet Scale 2013 VLDB 0.0002429959
328 The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing 2015 VLDB 0.00020889528
742 Zephyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms 2011 SIGMOD 0.00014284689
895 Albatross: Lightweight Elasticity in Shared Storage Databases for the Cloud using Live Data Migration 2011 VLDB 0.0001317406
973 Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management 2013 SIGMOD 0.00012734154
1,128 Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark 2018 SIGMOD 0.00011887295
1,244 Dhalion: Self-Regulating Stream Processing in Heron 2017 VLDB 0.00011363001
1,409 State Management in Apache Flink: Consistent Stateful Distributed Stream Processing 2017 VLDB 0.00010732706
2,435 Squall: Fine-Grained Live Reconfiguration for Partitioned Main Memory Databases 2015 SIGMOD 8.4576994e-05
3,043 Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems 2018 VLDB 7.7045781e-05
3,380 Dynamic Plan Migration for Continuous Queries Over Data Streams 2004 SIGMOD 7.3415892e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 973 Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management 2013 SIGMOD
2 5,312 Optimizing Data-intensive Systems in Disaggregated Data Centers with TELEPORT 2022 SIGMOD
3 10,197 Railgun: managing large streaming windows under MAD requirements 2021 VLDB
4 3,380 Dynamic Plan Migration for Continuous Queries Over Data Streams 2004 SIGMOD
5 8,427 Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines 2020 SIGMOD
6 9,063 Stateful Entities: Object-oriented Cloud Applications as Distributed Dataflows 2023 CIDR
7 9,857 Supporting Scalable Analytics with Latency Constraints 2015 VLDB
8 11,267 Styx: Transactional Stateful Functions on Streaming Dataflows 2025 SIGMOD
9 10,932 Fugue: Online Elasticity for Distributed Stateful Stream Processing 2026 VLDB
10 4,129 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD