Database Paper Browser

Back to papers

Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management

Summary: Externalised operator state with explicit primitives enables dynamic scale-out and fault-tolerant recovery for stateful stream processing. Checkpoints are partitioned across new VMs for scale-out; recovery restores state and replays tuples, demonstrated on EC2 with up to 50 VMs and L=350. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
4692
Venue
SIGMOD
Year
2013
Pagerank
0.00013168869
Overall Rank
1,226 | 91.49%
DOI
-

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 40 of 40 citing papers.

Rank Citing Paper Year Venue Pagerank
1,616 Realtime Data Processing at Facebook 2016 SIGMOD 0.00011133818
2,270 S-Store: Streaming Meets Transaction Processing 2015 VLDB 9.149662e-05
2,339 Samza: Stateful Scalable Stream Processing at LinkedIn 2017 VLDB 9.0029559e-05
3,216 Frontier: Resilient Edge Processing for the Internet of Things 2018 VLDB 7.3675613e-05
3,384 Scalable and Adaptive Online Joins 2014 VLDB 7.153329e-05
3,551 Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems 2018 VLDB 6.9776252e-05
3,771 SABER: Window-Based Hybrid Stream Processing for Heterogeneous Architectures 2016 SIGMOD 6.7738535e-05
4,051 Megaphone: Latency-conscious state migration for distributed streaming dataflows 2019 VLDB 6.4932782e-05
4,490 Analyzing Efficient Stream Processing on Modern Hardware 2019 VLDB 6.137834e-05
4,797 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD 5.9101178e-05
4,824 Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka 2021 SIGMOD 5.8902451e-05
5,292 StreamOps: Cloud-Native Runtime Management for Streaming Services in ByteDance 2023 VLDB 5.5784769e-05
5,357 LightSaber: Efficient Window Aggregation on Multi-core Processors 2020 SIGMOD 5.5512343e-05
5,657 FluxQuery: An Execution Framework for Highly Interactive Query Workloads 2016 SIGMOD 5.3873634e-05
5,670 BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures 2019 SIGMOD 5.3799054e-05
5,942 Clonos: Consistent Causal Recovery for Highly-Available Streaming Dataflows 2021 SIGMOD 5.2591128e-05
6,628 A Holistic View of Stream Partitioning Costs 2017 VLDB 4.9833118e-05
6,649 Grizzly: Efficient Stream Processing Through Adaptive Query Compilation 2020 SIGMOD 4.9724735e-05
6,726 Beyond Analytics: The Evolution of Stream Processing Systems 2020 SIGMOD 4.9444528e-05
6,856 Liquid: Unifying Nearline and Offline Big Data Integration 2015 CIDR 4.901516e-05
7,231 MgCrab: Transaction Crabbing for Live Migration in Deterministic Database Systems 2019 VLDB 4.7895465e-05
7,371 Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile 2021 VLDB 4.744863e-05
8,004 Rethinking Stateful Stream Processing with RDMA 2022 SIGMOD 4.6048388e-05
8,081 Meta-Dataflows: Efficient Exploratory Dataflow Jobs 2018 SIGMOD 4.5871672e-05
8,478 Optimization of Threshold Functions over Streams 2021 VLDB 4.4968401e-05
9,138 Phoebe: A Learning-based Checkpoint Optimizer 2021 VLDB 4.3842765e-05
9,220 Elasticutor: Rapid Elasticity for Realtime Stateful Stream Processing 2019 SIGMOD 4.3670162e-05
9,497 Scabbard: Single-Node Fault-Tolerant Stream Processing 2022 VLDB 4.3300131e-05
9,505 Supporting Scalable Analytics with Latency Constraints 2015 VLDB 4.3300131e-05
9,732 ContTune: Continuous Tuning by Conservative Bayesian Optimization for Distributed Stream Data Processing Systems 2023 VLDB 4.2901665e-05
9,882 Towards Resource-adaptive Query Execution in Cloud Native Databases 2024 CIDR 4.2594931e-05
10,043 Accelerating Stream Processing Engines via Hardware Offloading 2026 SIGMOD 4.1905499e-05
10,259 Scarf: Self-Adaptive Tuning via Multi-Objective Reinforcement Learning for Apache Flink 2026 VLDB 4.1905499e-05
10,413 CloudJump II: Optimizing Cloud Databases for Shared Storage 2025 SIGMOD 4.1905499e-05
11,245 Fries: Fast and Consistent Runtime Reconfiguration in Dataflow Systems with Transactional Guarantees 2023 VLDB 4.1905499e-05
11,700 Minimizing Cost by Reducing Scaling Operations in Distributed Stream Processing 2019 VLDB 4.1905499e-05
11,812 State Management in Apache Flink 2017 VLDB 4.1905499e-05
11,827 Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance 2017 VLDB 4.1905499e-05
11,917 CE-Storm: Confidential Elastic Processing of Data Streams 2015 SIGMOD 4.1905499e-05
11,933 Smooth Task Migration in Apache Storm 2015 SIGMOD 4.1905499e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
191 The Design of the Borealis Stream Processing Engine 2005 CIDR 0.00035714897
194 Query Processing, Resource Management, and Approximation in a Data Stream Management System 2003 CIDR 0.00035402379
585 DBToaster: Higher-order Delta Processing for Dynamic, Frequently Fresh Views 2012 VLDB 0.00019682634
601 Linear Road: A Stream Data Management Benchmark 2004 VLDB 0.00019372971
962 A Comparison of Join Algorithms for Log Processing in MapReduce 2010 SIGMOD 0.00015003834
2,710 Design, Implementation, and Evaluation of the Linear Road Benchmark on the Stream Processing Core 2006 SIGMOD 8.2580301e-05
5,044 Massive Scale-out of Expensive Continuous Queries 2011 VLDB 5.7338982e-05
5,047 Run-Time Operator State Spilling for Memory Intensive Long-Running Queries 2006 SIGMOD 5.7321615e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
4,490 Analyzing Efficient Stream Processing on Modern Hardware 2019 VLDB 6.137834e-05
9,123 On-Demand State Separation for Cloud Data Warehousing 2022 VLDB 4.3875153e-05
1,362 Highly Available, Fault-Tolerant, Parallel Dataflows 2004 SIGMOD 0.00012380613
6,628 A Holistic View of Stream Partitioning Costs 2017 VLDB 4.9833118e-05
11,827 Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance 2017 VLDB 4.1905499e-05
9,318 Providing Resiliency to Load Variations in Distributed Stream Processing 2006 VLDB 4.3523601e-05
3,887 Fault-tolerant Stream Processing using a Distributed, Replicated File System 2008 VLDB 6.6601639e-05
10,866 How Reliable Are Streams? End-to-End Processing-Guarantee Validation and Performance Benchmarking of Stream Processing Systems 2025 VLDB 4.1905499e-05
5,044 Massive Scale-out of Expensive Continuous Queries 2011 VLDB 5.7338982e-05
11,812 State Management in Apache Flink 2017 VLDB 4.1905499e-05