DBScholar

Back to papers

The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing

Summary: Introduces the Dataflow Model for massive-scale, unbounded, out-of-order streams, treating results as perpetually evolving rather than eventually complete. Exposes principled tradeoffs among event-time correctness, latency, and cost via watermarks, triggers, and retractions. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
h3eb25e9eb82dd88c
Venue
VLDB
Year
2015
Pagerank
0.00020889528
Overall Rank
328 | 97.81%
DOI
10.14778/2824032.2824076
PDF
Download (CC BY-NC-ND 3.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{akidau_vldb15,
        title = {{The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing}},
        author = {Akidau, Tyler and Bradshaw, Robert and Chambers, Craig and Chernyak, Slava and Fernández-Moctezuma, Rafael J. and Lax, Reuven and McVeety, Sam and Mills, Daniel and Perry, Frances and Schmidt, Eric and Whittle, Sam},
        journal = {PVLDB},
        series = {{VLDB} '15},
        volume = {8},
        number = {12},
        pages = {1792--1803},
        doi = {10.14778/2824032.2824076},
        url = {https://doi.org/10.14778/2824032.2824076},
        year = {2015}
}

Incoming Citations (Sorted by Pagerank)

Showing 50 of 72 citing papers.

Rank Citing Paper Year Venue Pagerank
1,128 Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark 2018 SIGMOD 0.00011887295
1,310 Realtime Data Processing at Facebook 2016 SIGMOD 0.00011065568
1,409 State Management in Apache Flink: Consistent Stateful Distributed Stream Processing 2017 VLDB 0.00010732706
1,785 Samza: Stateful Scalable Stream Processing at LinkedIn 2017 VLDB 9.6438157e-05
2,796 Lethe: A Tunable Delete-Aware LSM Engine 2020 SIGMOD 7.9849253e-05
2,831 F1 Query: Declarative Querying at Scale 2018 VLDB 7.945337e-05
2,871 SnappyData: A Unified Cluster for Streaming, Transactions, and Interactive Analytics 2017 CIDR 7.9024682e-05
3,043 Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems 2018 VLDB 7.7045781e-05
3,395 Timon: A Timestamped Event Database for Efficient Telemetry Data Processing and Analytics 2020 SIGMOD 7.3281547e-05
3,409 Megaphone: Latency-conscious state migration for distributed streaming dataflows 2019 VLDB 7.3076317e-05
3,761 Parallel Algorithms for Constructing Range and Nearest-Neighbor Searching Data Structures 2016 PODS 7.0186907e-05
3,939 Analyzing Efficient Stream Processing on Modern Hardware 2019 VLDB 6.8967981e-05
4,129 Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines 2020 SIGMOD 6.7751787e-05
4,475 Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka 2021 SIGMOD 6.5699934e-05
4,908 One SQL to Rule Them All – an Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables 2019 SIGMOD 6.3470129e-05
4,942 LightSaber: Efficient Window Aggregation on Multi-core Processors 2020 SIGMOD 6.3309707e-05
5,644 TcpRT: Instrument and Diagnostic Analysis System for Service Quality of Cloud Databases at Massive Scale in Real-time 2018 SIGMOD 6.0362111e-05
5,861 Optimal and General Out-of-Order Sliding-Window Aggregation 2019 VLDB 5.9505926e-05
5,952 StreamOps: Cloud-Native Runtime Management for Streaming Services in ByteDance 2023 VLDB 5.9182633e-05
6,189 Helios: Hyperscale Indexing for the Cloud & Edge 2020 VLDB 5.8391363e-05
6,240 Providing Streaming Joins as a Service at Facebook 2018 VLDB 5.8234435e-05
6,281 Beyond Analytics: The Evolution of Stream Processing Systems 2020 SIGMOD 5.8083033e-05
6,553 A Holistic View of Stream Partitioning Costs 2017 VLDB 5.7306258e-05
6,607 AStream: Ad-hoc Shared Stream Processing 2019 SIGMOD 5.7187564e-05
6,635 Watermarks in Stream Processing Systems: Semantics and Comparative Analysis of Apache Flink and Google Cloud Dataflow 2021 VLDB 5.7098902e-05
6,807 Intermittent Query Processing 2019 VLDB 5.661199e-05
6,809 Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile 2021 VLDB 5.660585e-05
7,340 Ananke: A Streaming Framework for Live Forward Provenance 2021 VLDB 5.5333914e-05
7,432 POLAR: Adaptive and Non-invasive Join Order Selection via Plans of Least Resistance 2024 VLDB 5.5133928e-05
7,680 CYADB: A Database that Covers Your Ask 2018 VLDB 5.4565877e-05
7,823 Erebus: Explaining the Outputs of Data Streaming Queries 2023 VLDB 5.4300469e-05
7,857 Quill: Efficient, Transferable, and Rich Analytics at Scale 2016 VLDB 5.4230146e-05
7,996 Rethinking Stateful Stream Processing with RDMA 2022 SIGMOD 5.3941543e-05
8,427 Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines 2020 SIGMOD 5.3184107e-05
8,705 Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems 2020 SIGMOD 5.2707935e-05
8,880 VStream: A Distributed Streaming Vector Search System 2025 VLDB 5.2447315e-05
8,930 InvaliDB: Scalable Push-Based Real-Time Queries on Top of Pull-Based Databases (Extended) 2020 VLDB 5.2361195e-05
8,977 Differentially Private Stream Processing at Scale 2024 VLDB 5.2336589e-05
9,232 Dalton: Learned Partitioning for Distributed Data Streams 2023 VLDB 5.1888006e-05
9,260 Streaming Democratized: Ease Across the Latency Spectrum with Delayed View Semantics and Snowflake Dynamic Tables 2025 SIGMOD 5.1860131e-05
9,354 CrocodileDB in Action: Resource-Efficient Query Execution by Exploiting Time Slackness 2020 VLDB 5.1751422e-05
9,511 Elasticutor: Rapid Elasticity for Realtime Stateful Stream Processing 2019 SIGMOD 5.1516241e-05
9,683 Disaggregated State Management in Apache Flink 2.0 2025 VLDB 5.1258854e-05
9,770 Vortex: A Stream-oriented Storage Engine For Big Data Analytics 2024 SIGMOD 5.115551e-05
9,958 GeaFlow: A Graph Extended and Accelerated Dataflow System 2023 SIGMOD 5.0910746e-05
10,040 cedar: Optimized and Unified Machine Learning Input Data Pipelines 2025 VLDB 5.0786422e-05
10,197 Railgun: managing large streaming windows under MAD requirements 2021 VLDB 5.0514463e-05
10,640 Enjima: A Resource-Adaptive Stream Processing System 2026 SIGMOD 4.9605343e-05
10,731 Mixtera: A Data Plane for Foundation Model Training 2026 SIGMOD 4.9605343e-05
10,800 Scarf: Self-Adaptive Tuning via Multi-Objective Reinforcement Learning for Apache Flink 2026 VLDB 4.9605343e-05
Previous Page 1 / 2 Next

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Previous Page 1 / 1 Next

Semantically Similar Papers