DBScholar

Back to papers

The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing

Summary: Introduces the Dataflow Model for massive-scale, unbounded, out-of-order streams, treating results as perpetually evolving rather than eventually complete. Exposes principled tradeoffs among event-time correctness, latency, and cost via watermarks, triggers, and retractions. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
h3eb25e9eb82dd88c
Venue
VLDB
Year
2015
Pagerank
0.00020964941
Overall Rank
325 | 97.82%
DOI
10.14778/2824032.2824076

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{akidau_vldb15,
        title = {{The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing}},
        author = {Akidau, Tyler and Bradshaw, Robert and Chambers, Craig and Chernyak, Slava and Fernández-Moctezuma, Rafael J. and Lax, Reuven and McVeety, Sam and Mills, Daniel and Perry, Frances and Schmidt, Eric and Whittle, Sam},
        journal = {PVLDB},
        series = {{VLDB} '15},
        volume = {8},
        number = {12},
        pages = {1792--1803},
        doi = {10.14778/2824032.2824076},
        url = {https://doi.org/10.14778/2824032.2824076},
        year = {2015}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 72 citing papers.

Rank Citing Paper Year Venue Pagerank
10,767 IncreQueryFusion: On-demand Data Fusion Framework in Dynamic Data Lakes 2026 VLDB 4.9793485e-05
10,862 Fugue: Online Elasticity for Distributed Stateful Stream Processing 2026 VLDB 4.9793485e-05
10,922 Virtualizing Recursion: Just-In-Time Graph Analytics in a Hyperscale Relational Warehouse 2026 VLDB 4.9793485e-05
10,927 A Decade of Apache Spark Structured Streaming: How We Evolved The Architecture To Meet Real-World Needs 2026 VLDB 4.9793485e-05
10,997 Demonstration of BobFlow: Human-Agent Collaboration Using Dataflows on Data Science Tasks 2026 VLDB 4.9793485e-05
11,036 The Dataflow Model Revisited Or: That Feeling When You Realize Every Problem You've Been Solving Is a Database Problem 2026 VLDB 4.9793485e-05
11,129 Oceanus: Enable SLO-Aware Vertical Autoscaling for Cloud-Native Streaming Services in Tencent 2025 SIGMOD 4.9793485e-05
11,276 Unraveling the Impact of Window Semantics: Optimizing Join Order for Efficient Stream Processing 2025 VLDB 4.9793485e-05
11,372 Scribe: How Meta transports terabytes per second in real time 2025 VLDB 4.9793485e-05
11,436 Analyzing Near-Network Hardware Acceleration with Co-Processing on DPUs 2025 VLDB 4.9793485e-05
11,439 How Reliable Are Streams? End-to-End Processing-Guarantee Validation and Performance Benchmarking of Stream Processing Systems 2025 VLDB 4.9793485e-05
11,506 PECJ: Stream Window Join on Disorder Data Streams with Proactive Error Compensation 2024 SIGMOD 4.9793485e-05
11,670 Raising the Level of Abstraction for Time-State Analytics With the Timeline Framework 2023 CIDR 4.9793485e-05
11,755 Fries: Fast and Consistent Runtime Reconfiguration in Dataflow Systems with Transactional Guarantees 2023 VLDB 4.9793485e-05
11,800 Portals: A Showcase of Multi-Dataflow Stateful Serverless 2023 VLDB 4.9793485e-05
11,940 Synchronization Schemas 2021 PODS 4.9793485e-05
11,972 Klink: Progress-Aware Scheduling for Streaming Data Systems 2021 SIGMOD 4.9793485e-05
11,990 Real-time Data Infrastructure at Uber 2021 SIGMOD 4.9793485e-05
12,005 In the Land of Data Streams where Synopses are Missing, One Framework to Bring Them All 2021 VLDB 4.9793485e-05
12,213 Robust, Scalable, Real-Time Event Time Series Aggregation at Twitter 2018 SIGMOD 4.9793485e-05
12,232 Challenges and Experiences in Building an Efficient Apache Beam Runner For IBM Streams 2018 VLDB 4.9793485e-05
12,304 CarStream: An Industrial System of Big Data Processing for Internet-of-Vehicles 2017 VLDB 4.9793485e-05
Previous Page 2 / 2 Next

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
25 NiagaraCQ: A Scalable Continuous Query System for Internet Databases 2000 SIGMOD 0.00053930011
31 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00049839909
218 MillWheel: Fault-Tolerant Stream Processing at Internet Scale 2013 VLDB 0.00024390324
429 Flexible Time Management in Data Stream Systems 2004 PODS 0.00018439857
510 TelegraphCQ: Continuous Dataflow Processing 2003 SIGMOD 0.00017065714
651 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00015130782
685 Trill: A High-Performance Incremental Query Processor for Diverse Analytics 2015 VLDB 0.00014782777
895 Consistent Streaming Through Time: A Vision for Event Stream Processing 2007 CIDR 0.00013211204
1,197 Out-of-Order Processing: A New Architecture for High-Performance Stream Systems 2008 VLDB 0.00011566535
1,633 Summingbird: A Framework for Integrating Batch and Online MapReduce Computations 2014 VLDB 0.00010019617
1,674 Semantics and Evaluation Techniques for Window Aggregates in Data Streams 2005 SIGMOD 9.922398e-05
3,251 SECRET: A Model for Analysis of the Execution Semantics of Stream Processing Systems 2010 VLDB 7.4932341e-05
3,324 A Heartbeat Mechanism and its Application in Gigascope 2005 VLDB 7.4285048e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 3,339 Optimizing Analytic Data Flows for Multiple Execution Engines 2012 SIGMOD
2 2,116 Generating Example Data for Dataflow Programs 2009 SIGMOD
3 9,816 Supporting Scalable Analytics with Latency Constraints 2015 VLDB
4 8,550 Topology-aware Parallel Data Processing: Models, Algorithms and Systems at Scale 2020 CIDR
5 4,188 Schedule Optimization for Data Processing Flows on the Cloud 2011 SIGMOD
6 2,227 Spinning Fast Iterative Data Flows 2012 VLDB
7 3,995 An Approach to Optimize Data Processing in Business Processes 2007 VLDB
8 26 Models and Issues in Data Stream Systems 2002 PODS
9 2,621 Process And Dataflow Control In Distributed Data-Intensive Systems 1988 SIGMOD
10 11,036 The Dataflow Model Revisited Or: That Feeling When You Realize Every Problem You've Been Solving Is a Database Problem 2026 VLDB