DBScholar

Back to papers

Challenges and Experiences in Building an Efficient Apache Beam Runner For IBM Streams

Summary: IBM Streams' Beam runner optimizes event-time windows by indexing inter-dependent states, garbage-collecting stale keys, and tuning bundle sizes. On NEXMark, it outruns Flink and Spark, demonstrating efficient, enterprise Beam integration on Streams. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h5703e5ad5d9ad631
Venue
VLDB
Year
2018
Pagerank
4.9793485e-05
Overall Rank
12,232 | 17.76%
DOI
10.14778/3229863.3229864

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

BibTeX Citation

@article{li_vldb18,
        title = {{Challenges and Experiences in Building an Efficient Apache Beam Runner For IBM Streams}},
        author = {Li, Shen and Gerver, Paul and MacMillan, John and Debrunner, Daniel and Marshall, William and Wu, Kun-Lung},
        journal = {PVLDB},
        series = {{VLDB} '18},
        volume = {11},
        number = {12},
        pages = {1742--1754},
        doi = {10.14778/3229863.3229864},
        url = {https://doi.org/10.14778/3229863.3229864},
        year = {2018}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank Citing Paper Year Venue Pagerank
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 9 of 9 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
218 MillWheel: Fault-Tolerant Stream Processing at Internet Scale 2013 VLDB 0.00024390324
231 Storm @Twitter 2014 SIGMOD 0.00023841089
325 The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing 2015 VLDB 0.00020964941
606 Twitter Heron: Stream Processing at Scale 2015 SIGMOD 0.00015635133
1,233 Dhalion: Self-Regulating Stream Processing in Heron 2017 VLDB 0.00011405873
1,402 State Management in Apache Flink: Consistent Stateful Distributed Stream Processing 2017 VLDB 0.00010771949
1,771 Samza: Stateful Scalable Stream Processing at LinkedIn 2017 VLDB 9.6803752e-05
2,739 General Incremental Sliding-Window Aggregation 2015 VLDB 8.0721161e-05
4,309 Consistent Regions: Guaranteed Tuple Processing in IBM Streams 2016 VLDB 6.6732011e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 4,400 Shared Arrangements: practical inter-query sharing for streaming dataflows 2020 VLDB
2 8,417 Demonstrating PDSP-Bench: A Benchmarking System for Parallel and Distributed Stream Processing 2025 SIGMOD
3 6,123 Parallel Index-based Stream Join on a Multicore CPU 2020 SIGMOD
4 6,599 AStream: Ad-hoc Shared Stream Processing 2019 SIGMOD
5 12,624 Building User-defined Runtime Adaptation Routines for Stream Processing Applications 2012 VLDB
6 3,922 Analyzing Efficient Stream Processing on Modern Hardware 2019 VLDB
7 8,581 Changing Engines in Midstream: A Java Stream Computational Model for Big Data Processing 2014 VLDB
8 10,927 A Decade of Apache Spark Structured Streaming: How We Evolved The Architecture To Meet Real-World Needs 2026 VLDB
9 4,884 One SQL to Rule Them All – an Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables 2019 SIGMOD
10 1,123 Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark 2018 SIGMOD