Database Paper Browser

Back to papers

Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka

Summary: Kafka's persistent log provides strong correctness under failures and out-of-order data. Kafka Streams uses read–process–write over log-append state with idempotent/transactional exactly-once and revision-based speculative processing for early results, enabling scalable correctness–performance–cost trade-offs. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
6246
Venue
SIGMOD
Year
2021
Pagerank
5.8902451e-05
Overall Rank
4,824 | 66.48%
DOI
10.1145/3448016.3457556

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 7 of 7 citing papers.

Rank Citing Paper Year Venue Pagerank
5,292 StreamOps: Cloud-Native Runtime Management for Streaming Services in ByteDance 2023 VLDB 5.5784769e-05
6,769 Watermarks in Stream Processing Systems: Semantics and Comparative Analysis of Apache Flink and Google Cloud Dataflow 2021 VLDB 4.9274854e-05
8,402 Serverless State Management Systems 2024 CIDR 4.5167942e-05
8,911 What's the Difference? Incremental Processing with Change Queries in Snowflake 2023 SIGMOD 4.4229886e-05
9,323 Disaggregated State Management in Apache Flink® 2.0 2025 VLDB 4.351469e-05
9,731 DARQ Matter Binds Everything: Performant and Composable Cloud Programming via Resilient Steps 2023 SIGMOD 4.2901665e-05
11,084 Towards Millions of Database Transmission Services in the Cloud 2024 VLDB 4.1905499e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 20 of 20 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
22 SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets 2008 VLDB 0.00084679526
143 TelegraphCQ: Continuous Dataflow Processing for an Uncertain World 2003 CIDR 0.00041688468
191 The Design of the Borealis Stream Processing Engine 2005 CIDR 0.00035714897
314 MillWheel: Fault-Tolerant Stream Processing at Internet Scale 2013 VLDB 0.00028059664
324 Gigascope: A Stream Database for Network Applications 2003 SIGMOD 0.00027465124
433 Flexible Time Management in Data Stream Systems 2004 PODS 0.00023336894
536 The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing 2015 VLDB 0.00020651621
1,081 Dhalion: Self-Regulating Stream Processing in Heron 2017 VLDB 0.00014201838
1,097 Trill: A High-Performance Incremental Query Processor for Diverse Analytics 2015 VLDB 0.00014083973
1,226 Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management 2013 SIGMOD 0.00013168869
1,549 Out-of-Order Processing: A New Architecture for High-Performance Stream Systems 2008 VLDB 0.00011407374
2,188 Aurora: A Data Stream Management System 2003 SIGMOD 9.3378116e-05
2,270 S-Store: Streaming Meets Transaction Processing 2015 VLDB 9.149662e-05
2,339 Samza: Stateful Scalable Stream Processing at LinkedIn 2017 VLDB 9.0029559e-05
3,551 Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems 2018 VLDB 6.9776252e-05
4,051 Megaphone: Latency-conscious state migration for distributed streaming dataflows 2019 VLDB 6.4932782e-05
5,271 Consistent Regions: Guaranteed Tuple Processing in IBM Streams 2016 VLDB 5.5924143e-05
5,766 Building a Replicated Logging System with Apache Kafka 2015 VLDB 5.3353079e-05
6,133 Optimal and General Out-of-Order Sliding-Window Aggregation 2019 VLDB 5.1917864e-05
6,856 Liquid: Unifying Nearline and Offline Big Data Integration 2015 CIDR 4.901516e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
6,769 Watermarks in Stream Processing Systems: Semantics and Comparative Analysis of Apache Flink and Google Cloud Dataflow 2021 VLDB 4.9274854e-05
9,417 Kora: A Cloud-Native Event Streaming Platform For Kafka 2023 VLDB 4.3399748e-05
7,941 Correctness in Stream Processing: Challenges and Opportunities 2022 CIDR 4.6089395e-05
2,339 Samza: Stateful Scalable Stream Processing at LinkedIn 2017 VLDB 9.0029559e-05
11,827 Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance 2017 VLDB 4.1905499e-05
1,995 Fault-Tolerance in the Borealis Distributed Stream Processing System 2005 SIGMOD 9.83817e-05
11,362 KafkaDirect: Zero-copy Data Access for Apache Kafka over RDMA Networks 2022 SIGMOD 4.1905499e-05
11,812 State Management in Apache Flink 2017 VLDB 4.1905499e-05
10,866 How Reliable Are Streams? End-to-End Processing-Guarantee Validation and Performance Benchmarking of Stream Processing Systems 2025 VLDB 4.1905499e-05
5,766 Building a Replicated Logging System with Apache Kafka 2015 VLDB 5.3353079e-05