MillWheel: Fault-Tolerant Stream Processing at Internet Scale

Summary: MillWheel is a Google-scale, low-latency stream-processing framework where users define computation graphs while the system manages persistent state, record flow, and fault tolerance. Its distinctive logical-time programming model simplifies time-based aggregation alongside scalable, resilient execution. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 10729
Venue: VLDB
Year: 2013
Pagerank: 0.00024130894
Overall Rank: 224 | 98.47%
DOI: 10.14778/2536222.2536229

Incoming Non-self Citations Over Time

Authors

1. Tyler Akidau (Google)
2. Alex Balikov (Google)
3. Kaya Bekiroğlu (Google)
4. Slava Chernyak (Google)
5. Josh Haberman (Google)
6. Reuven Lax (Google)
7. Sam McVeety (Google)
8. Daniel Mills (Google)
9. Paul Nordstrom (Google)
10. Sam Whittle (Google)

BibTeX Citation

@article{akidau_vldb13,
        title = {{MillWheel: Fault-Tolerant Stream Processing at Internet Scale}},
        author = {Akidau, Tyler and Balikov, Alex and Bekiroğlu, Kaya and Chernyak, Slava and Haberman, Josh and Lax, Reuven and McVeety, Sam and Mills, Daniel and Nordstrom, Paul and Whittle, Sam},
        journal = {PVLDB},
        series = {{VLDB} '13},
        volume = {6},
        number = {11},
        pages = {1033--1044},
        doi = {10.14778/2536222.2536229},
        url = {https://doi.org/10.14778/2536222.2536229},
        year = {2013}
}

Incoming Citations (Sorted by Pagerank)

Showing 16 of 66 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,633	Scabbard: Single-Node Fault-Tolerant Stream Processing	2022	VLDB	5.2434488e-05
9,640	Supporting Scalable Analytics with Latency Constraints	2015	VLDB	5.2434488e-05
9,738	GeaFlow: A Graph Extended and Accelerated Dataflow System	2023	SIGMOD	5.227679e-05
10,695	Oceanus: Enable SLO-Aware Vertical Autoscaling for Cloud-Native Streaming Services in Tencent	2025	SIGMOD	5.093636e-05
10,712	CORE+: A Complex Event Recognition Engine in C++	2025	SIGMOD	5.093636e-05
10,996	Scribe: How Meta transports terabytes per second in real time	2025	VLDB	5.093636e-05
11,460	Out-of-Order Sliding-Window Aggregation with Efficient Bulk Evictions and Insertions	2023	VLDB	5.093636e-05
11,506	Making Cache Monotonic and Consistent	2023	VLDB	5.093636e-05
11,633	Synchronization Schemas	2021	PODS	5.093636e-05
11,665	Klink: Progress-Aware Scheduling for Streaming Data Systems	2021	SIGMOD	5.093636e-05
11,820	InvaliDB: Scalable Push-Based Real-Time Queries on Top of Pull-Based Databases (Extended)	2020	VLDB	5.093636e-05
11,868	Online Template Induction for Machine-Generated Emails	2019	VLDB	5.093636e-05
11,914	Robust, Scalable, Real-Time Event Time Series Aggregation at Twitter	2018	SIGMOD	5.093636e-05
11,933	Challenges and Experiences in Building an Efficient Apache Beam Runner For IBM Streams	2018	VLDB	5.093636e-05
12,022	Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance	2017	VLDB	5.093636e-05
12,050	The Challenges of Global-scale Data Management	2016	SIGMOD	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 10 of 10 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
26	Models and Issues in Data Stream Systems	2002	PODS	0.00052982574
127	The Design of the Borealis Stream Processing Engine	2005	CIDR	0.00030738755
150	Query Processing, Resource Management, and Approximation in a Data Stream Management System	2003	CIDR	0.00029208207
220	Storm @Twitter	2014	SIGMOD	0.00024244587
424	Flexible Time Management in Data Stream Systems	2004	PODS	0.0001870735
505	TelegraphCQ: Continuous Dataflow Processing	2003	SIGMOD	0.00017285498
1,223	Out-of-Order Processing: A New Architecture for High-Performance Stream Systems	2008	VLDB	0.00011602265
1,696	Gigascope: High Performance Network Monitoring with an SQL Interface	2002	SIGMOD	9.9810227e-05
3,332	A Heartbeat Mechanism and its Application in Gigascope	2005	VLDB	7.5129799e-05
3,776	Fault-tolerant Stream Processing using a Distributed, Replicated File System	2008	VLDB	7.1347542e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	361	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB
2	3,776	Fault-tolerant Stream Processing using a Distributed, Replicated File System	2008	VLDB
3	13,523	Fault-Tolerance for Distributed Iterative Dataflows in Action	2018	VLDB
4	983	Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management	2013	SIGMOD
5	1,789	Fault-Tolerance in the Borealis Distributed Stream Processing System	2005	SIGMOD
6	1,209	Highly Available, Fault-Tolerant, Parallel Dataflows	2004	SIGMOD
7	9,596	Cost-based Fault-tolerance for Parallel Data Processing	2015	SIGMOD
8	9,968	Railgun: managing large streaming windows under MAD requirements	2021	VLDB
9	1,171	Photon: Fault-tolerant and Scalable Joining of Continuous Data Streams	2013	SIGMOD
10	12,022	Toward High-Performance Distributed Stream Processing via Approximate Fault Tolerance	2017	VLDB