Highly Available, Fault-Tolerant, Parallel Dataflows

Summary: Flux is a fault-tolerant dataflow operator that provides high availability for long-running, parallel continuous queries. It fuses partitioned parallelism with process-pairs HA to enable piecemeal, on-the-fly recovery while minimizing changes to operator code. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h4a16b822b4541f08
Venue: SIGMOD
Year: 2004
Pagerank: 0.00011425028
Overall Rank: 1,228 | 91.75%
DOI: 10.1145/1007568.1007662

Incoming Non-self Citations Over Time

Authors

1. Mehul A. Shah (University of California Berkeley)
2. Joseph M. Hellerstein (Intel; University of California Berkeley)
3. Eric Brewer (University of California Berkeley)

BibTeX Citation

@inproceedings{shah_sigmod04,
        title = {{Highly Available, Fault-Tolerant, Parallel Dataflows}},
        author = {Shah, Mehul A. and Hellerstein, Joseph M. and Brewer, Eric},
        series = {{SIGMOD} '04},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/1007568.1007662},
        url = {https://dl.acm.org/doi/10.1145/1007568.1007662},
        year = {2004}
}

Incoming Citations (Sorted by Pagerank)

Showing 18 of 18 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
127	The Design of the Borealis Stream Processing Engine	2005	CIDR	0.00030427614
1,351	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00010934347
1,828	Fault-Tolerance in the Borealis Distributed Stream Processing System	2005	SIGMOD	9.5516021e-05
1,938	S-Store: Streaming Meets Transaction Processing	2015	VLDB	9.3342542e-05
2,672	A Latency and Fault-Tolerance Optimizer for Online Parallel Query Plans	2011	SIGMOD	8.1491683e-05
3,370	Frontier: Resilient Edge Processing for the Internet of Things	2018	VLDB	7.3695613e-05
3,790	Fault-tolerant Stream Processing using a Distributed, Replicated File System	2008	VLDB	7.0190297e-05
4,866	Clonos: Consistent Causal Recovery for Highly-Available Streaming Dataflows	2021	SIGMOD	6.3754084e-05
5,356	Load Management and High Availability in the Medusa Distributed Stream Processing System	2004	SIGMOD	6.1620521e-05
5,648	Lineage-driven Fault Injection	2015	SIGMOD	6.0523358e-05
5,653	Staying FIT: Efficient Load Shedding Techniques for Distributed Stream Processing	2007	VLDB	6.0509231e-05
8,424	Correctness in Stream Processing: Challenges and Opportunities	2022	CIDR	5.3350162e-05
8,876	Customizable Parallel Execution of Scientific Stream Queries	2005	VLDB	5.2576285e-05
9,005	Parallel Querying with Non-Dedicated Computers	2005	VLDB	5.2388349e-05
9,633	Shrink - Prescribing Resiliency Solutions for Streaming	2017	VLDB	5.1471224e-05
9,819	Borealis-R: A Replication-Transparent Stream Processing System for Wide-Area Monitoring Applications	2008	SIGMOD	5.1257999e-05
12,317	Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance	2017	VLDB	4.9793485e-05
12,740	iFlow: An Approach for Fast and Reliable Internet-Scale Stream Processing Utilizing Detouring and Replication	2010	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 8 of 8 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
25	NiagaraCQ: A Scalable Continuous Query System for Internet Databases	2000	SIGMOD	0.00053930011
93	Encapsulation of Parallelism in the Volcano Query Processing System	1990	SIGMOD	0.00034622929
112	TelegraphCQ: Continuous Dataflow Processing for an Uncertain World	2003	CIDR	0.00032459277
152	Query Processing, Resource Management, and Approximation in a Data Stream Management System	2003	CIDR	0.0002867034
171	Monitoring Streams – A New Class of Data Management Applications	2002	VLDB	0.00027011593
1,149	Implementing Recoverable Requests Using Queues	1990	SIGMOD	0.00011809579
1,271	Managing Intra-operator Parallelism in Parallel Database Systems	1995	VLDB	0.00011252507
7,044	A Database System for Real-Time Event Aggregation in Telecommunication	1998	VLDB	5.6140722e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	9,824	Optimistic Recovery for Iterative Dataflows in Action	2015	SIGMOD
2	1,985	Continuous Analytics Over Discontinuous Streams	2010	SIGMOD
3	2,953	Scalable and Adaptive Online Joins	2014	VLDB
4	2,672	A Latency and Fault-Tolerance Optimizer for Online Parallel Query Plans	2011	SIGMOD
5	968	Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management	2013	SIGMOD
6	5,013	Massive Scale-out of Expensive Continuous Queries	2011	VLDB
7	13,151	Fault-tolerant, Load-balancing Queries in Telegraph	2001	SIGMOD
8	1,828	Fault-Tolerance in the Borealis Distributed Stream Processing System	2005	SIGMOD
9	12,317	Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance	2017	VLDB
10	9,773	Cost-based Fault-tolerance for Parallel Data Processing	2015	SIGMOD