Correctness in Stream Processing: Challenges and Opportunities

Summary: Documents correctness challenges in distributed real-time stream processing—engineering and performance advances outpaced formal correctness, leading to hard-to-detect, catastrophic errors in platforms like Storm, Flink, Spark. Advocates rigorous formalization and formal-methods-based verification of stream processing programs and systems. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 438
Venue: CIDR
Year: 2022
Pagerank: 5.4574671e-05
Overall Rank: 8,258 | 43.35%
DOI: -

Incoming Non-self Citations Over Time

Authors

1. Caleb Stanford (University of Pennsylvania)
2. Konstantinos Kallas (University of Pennsylvania)
3. Rajeev Alur (University of Pennsylvania)

BibTeX Citation

@inproceedings{stanford_cidr22,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '22},
        title = {{Correctness in Stream Processing: Challenges and Opportunities}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Stanford, Caleb and Kallas, Konstantinos and Alur, Rajeev},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
11,086	How Reliable Are Streams? End-to-End Processing-Guarantee Validation and Performance Benchmarking of Stream Processing Systems	2025	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 10 of 10 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
455	Differential dataflow	2013	CIDR	0.00018133241
710	Trill: A High-Performance Incremental Query Processor for Diverse Analytics	2015	VLDB	0.00014715033
1,190	Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark	2018	SIGMOD	0.00011743246
1,209	Highly Available, Fault-Tolerant, Parallel Dataflows	2004	SIGMOD	0.00011657178
1,422	State Management in Apache Flink: Consistent Stateful Distributed Stream Processing	2017	VLDB	0.00010823039
1,789	Fault-Tolerance in the Borealis Distributed Stream Processing System	2005	SIGMOD	9.7502324e-05
1,895	Samza: Stateful Scalable Stream Processing at LinkedIn	2017	VLDB	9.5260291e-05
4,279	Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines	2020	SIGMOD	6.7865878e-05
5,756	Clonos: Consistent Causal Recovery for Highly-Available Streaming Dataflows	2021	SIGMOD	6.097233e-05
11,633	Synchronization Schemas	2021	PODS	5.093636e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	12,306	Declarative Error Management for Robust Data-Intensive Applications	2012	SIGMOD
2	13,666	Verifying Computations with Streaming Interactive Proofs	2012	VLDB
3	11,633	Synchronization Schemas	2021	PODS
4	1,350	Automating Large-Scale Data Quality Verification	2018	VLDB
5	4,389	Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka	2021	SIGMOD
6	361	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB
7	4,907	One SQL to Rule Them All – an Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables	2019	SIGMOD
8	1,190	Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark	2018	SIGMOD
9	11,086	How Reliable Are Streams? End-to-End Processing-Guarantee Validation and Performance Benchmarking of Stream Processing Systems	2025	VLDB
10	6,502	Watermarks in Stream Processing Systems: Semantics and Comparative Analysis of Apache Flink and Google Cloud Dataflow	2021	VLDB