DBScholar

Back to papers

Megaphone: Latency-conscious state migration for distributed streaming dataflows

Summary: Megaphone enables latency-conscious state migration for streaming dataflows by subdividing migrations into configurable granularity to limit spikes. Ahead-of-time prep; unmodified Timely dataflow library with a compatible interface, delivering large latency reductions and modest overhead. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
11990
Venue
VLDB
Year
2019
Pagerank
7.337549e-05
Overall Rank
3,531 | 75.78%
DOI
10.14778/3329772.3329777

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{hoffmann_vldb19,
        title = {{Megaphone: Latency-conscious state migration for distributed streaming dataflows}},
        author = {Hoffmann, Moritz and Lattuada, Andrea and McSherry, Frank and Kalavri, Vasiliki and Liagouris, John and Roscoe, Timothy},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {9},
        pages = {1002--1015},
        doi = {10.14778/3329772.3329777},
        url = {https://doi.org/10.14778/3329772.3329777},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 15 of 15 citing papers.

Rank Citing Paper Year Venue Pagerank
4,279 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD 6.7865878e-05
4,389 Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka 2021 SIGMOD 6.7304854e-05
5,821 StreamOps: Cloud-Native Runtime Management for Streaming Services in ByteDance 2023 VLDB 6.07598e-05
6,680 Beyond Analytics: The Evolution of Stream Processing Systems 2020 SIGMOD 5.8049764e-05
8,341 Tackling Hardware/Software co-design from a database perspective 2020 CIDR 5.4486102e-05
9,633 Scabbard: Single-Node Fault-Tolerant Stream Processing 2022 VLDB 5.2434488e-05
9,898 ContTune: Continuous Tuning by Conservative Bayesian Optimization for Distributed Stream Data Processing Systems 2023 VLDB 5.1997534e-05
10,039 Towards Resource-adaptive Query Execution in Cloud Native Databases 2024 CIDR 5.1723558e-05
10,337 Accelerating Stream Processing Engines via Hardware Offloading 2026 SIGMOD 5.093636e-05
10,551 APEROL: Adaptive Parallel Edge-to-cloud Runtime Optimization for Layered Workflow Execution 2026 VLDB 5.093636e-05
10,573 Incremental Stream Query Deployment under Continuous Infrastructure Changes in the Cloud-Edge Continuum 2026 VLDB 5.093636e-05
10,579 Meerkat: Scalable, Network-Aware Failure Recovery for the Internet of Things 2026 VLDB 5.093636e-05
10,633 Transactional Cloud Applications Go with the (Data)Flow 2025 CIDR 5.093636e-05
10,782 Styx: Transactional Stateful Functions on Streaming Dataflows 2025 SIGMOD 5.093636e-05
11,443 Fries: Fast and Consistent Runtime Reconfiguration in Dataflow Systems with Transactional Guarantees 2023 VLDB 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
224 MillWheel: Fault-Tolerant Stream Processing at Internet Scale 2013 VLDB 0.00024130894
361 The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing 2015 VLDB 0.00020138717
741 Zephyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms 2011 SIGMOD 0.00014442062
902 Albatross: Lightweight Elasticity in Shared Storage Databases for the Cloud using Live Data Migration 2011 VLDB 0.00013328979
983 Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management 2013 SIGMOD 0.0001283214
1,190 Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark 2018 SIGMOD 0.00011743246
1,224 Dhalion: Self-Regulating Stream Processing in Heron 2017 VLDB 0.00011596911
1,422 State Management in Apache Flink: Consistent Stateful Distributed Stream Processing 2017 VLDB 0.00010823039
2,407 Squall: Fine-Grained Live Reconfiguration for Partitioned Main Memory Databases 2015 SIGMOD 8.6184333e-05
3,117 Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems 2018 VLDB 7.7382559e-05
3,321 Dynamic Plan Migration for Continuous Queries Over Data Streams 2004 SIGMOD 7.5216049e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 10,337 Accelerating Stream Processing Engines via Hardware Offloading 2026 SIGMOD
2 983 Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management 2013 SIGMOD
3 5,196 Optimizing Data-intensive Systems in Disaggregated Data Centers with TELEPORT 2022 SIGMOD
4 3,321 Dynamic Plan Migration for Continuous Queries Over Data Streams 2004 SIGMOD
5 9,968 Railgun: managing large streaming windows under MAD requirements 2021 VLDB
6 8,309 Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines 2020 SIGMOD
7 8,874 Stateful Entities: Object-oriented Cloud Applications as Distributed Dataflows 2023 CIDR
8 9,640 Supporting Scalable Analytics with Latency Constraints 2015 VLDB
9 10,782 Styx: Transactional Stateful Functions on Streaming Dataflows 2025 SIGMOD
10 4,279 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD