Highly Available, Fault-Tolerant, Parallel Dataflows

Summary: Flux is a fault-tolerant dataflow operator that provides high availability for long-running, parallel continuous queries. It fuses partitioned parallelism with process-pairs HA to enable piecemeal, on-the-fly recovery while minimizing changes to operator code. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 3633
Venue: SIGMOD
Year: 2004
Pagerank: 0.00011657178
Overall Rank: 1,209 | 91.71%
DOI: 10.1145/1007568.1007662

Incoming Non-self Citations Over Time

Authors

1. Mehul A. Shah (University of California Berkeley)
2. Joseph M. Hellerstein (Intel; University of California Berkeley)
3. Eric Brewer (University of California Berkeley)

BibTeX Citation

@inproceedings{shah_sigmod04,
        title = {{Highly Available, Fault-Tolerant, Parallel Dataflows}},
        author = {Shah, Mehul A. and Hellerstein, Joseph M. and Brewer, Eric},
        series = {{SIGMOD} '04},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/1007568.1007662},
        url = {https://dl.acm.org/doi/10.1145/1007568.1007662},
        year = {2004}
}

Incoming Citations (Sorted by Pagerank)

Showing 18 of 18 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
127	The Design of the Borealis Stream Processing Engine	2005	CIDR	0.00030738755
1,319	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00011175005
1,789	Fault-Tolerance in the Borealis Distributed Stream Processing System	2005	SIGMOD	9.7502324e-05
1,915	S-Store: Streaming Meets Transaction Processing	2015	VLDB	9.4884706e-05
2,622	A Latency and Fault-Tolerance Optimizer for Online Parallel Query Plans	2011	SIGMOD	8.3330136e-05
3,306	Frontier: Resilient Edge Processing for the Internet of Things	2018	VLDB	7.5387095e-05
3,776	Fault-tolerant Stream Processing using a Distributed, Replicated File System	2008	VLDB	7.1347542e-05
5,241	Load Management and High Availability in the Medusa Distributed Stream Processing System	2004	SIGMOD	6.3019989e-05
5,521	Staying FIT: Efficient Load Shedding Techniques for Distributed Stream Processing	2007	VLDB	6.1881166e-05
5,574	Lineage-driven Fault Injection	2015	SIGMOD	6.1681803e-05
5,756	Clonos: Consistent Causal Recovery for Highly-Available Streaming Dataflows	2021	SIGMOD	6.097233e-05
8,258	Correctness in Stream Processing: Challenges and Opportunities	2022	CIDR	5.4574671e-05
8,711	Customizable Parallel Execution of Scientific Stream Queries	2005	VLDB	5.3781587e-05
8,841	Parallel Querying with Non-Dedicated Computers	2005	VLDB	5.358974e-05
9,531	Shrink - Prescribing Resiliency Solutions for Streaming	2017	VLDB	5.2540895e-05
9,643	Borealis-R: A Replication-Transparent Stream Processing System for Wide-Area Monitoring Applications	2008	SIGMOD	5.2434488e-05
12,022	Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance	2017	VLDB	5.093636e-05
12,449	iFlow: An Approach for Fast and Reliable Internet-Scale Stream Processing Utilizing Detouring and Replication	2010	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
25	NiagaraCQ: A Scalable Continuous Query System for Internet Databases	2000	SIGMOD	0.00054667018
98	Encapsulation of Parallelism in the Volcano Query Processing System	1990	SIGMOD	0.00034510605
111	TelegraphCQ: Continuous Dataflow Processing for an Uncertain World	2003	CIDR	0.00032990214
150	Query Processing, Resource Management, and Approximation in a Data Stream Management System	2003	CIDR	0.00029208207
167	Monitoring Streams – A New Class of Data Management Applications	2002	VLDB	0.0002757283
1,127	Implementing Recoverable Requests Using Queues	1990	SIGMOD	0.00012070177
1,291	Managing Intra-operator Parallelism in Parallel Database Systems	1995	VLDB	0.00011307625
6,910	A Database System for Real-Time Event Aggregation in Telecommunication	1998	VLDB	5.741257e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	9,670	Optimistic Recovery for Iterative Dataflows in Action	2015	SIGMOD
2	1,945	Continuous Analytics Over Discontinuous Streams	2010	SIGMOD
3	2,892	Scalable and Adaptive Online Joins	2014	VLDB
4	2,622	A Latency and Fault-Tolerance Optimizer for Online Parallel Query Plans	2011	SIGMOD
5	983	Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management	2013	SIGMOD
6	4,918	Massive Scale-out of Expensive Continuous Queries	2011	VLDB
7	12,861	Fault-tolerant, Load-balancing Queries in Telegraph	2001	SIGMOD
8	1,789	Fault-Tolerance in the Borealis Distributed Stream Processing System	2005	SIGMOD
9	12,022	Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance	2017	VLDB
10	9,596	Cost-based Fault-tolerance for Parallel Data Processing	2015	SIGMOD