Database Paper Browser

Back to papers

Summingbird: A Framework for Integrating Batch and Online MapReduce Computations

Summary: Summingbird is a Scala DSL that unifies batch and online MapReduce in a single framework, using dataflow abstractions (sources, sinks, stores) and executing on Hadoop (Scalding/Cascading) or Storm without code changes. Hybrid processing mode transparently fuses batch and online results via algebraic structures, imposing aggregations constraints that guide applicability while reducing duplication for Twitter-scale analytics. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
10804
Venue
VLDB
Year
2014
Pagerank
0.00010532024
Overall Rank
1,794 | 87.53%
DOI
-

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 14 of 14 citing papers.

Rank Citing Paper Year Venue Pagerank
538 The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing 2015 VLDB 0.00020678804
824 Twitter Heron: Stream Processing at Scale 2015 SIGMOD 0.0001623129
1,613 Realtime Data Processing at Facebook 2016 SIGMOD 0.00011140777
2,163 Elastic Machine Learning Algorithms in Amazon SageMaker 2020 SIGMOD 9.3949234e-05
2,853 Timon: A Timestamped Event Database for Efficient Telemetry Data Processing and Analytics 2020 SIGMOD 8.0108722e-05
2,919 RaSQL: Greater Power and Performance for Big Data Analytics with Recursive-aggregate-SQL on Spark 2019 SIGMOD 7.9047279e-05
3,319 Sketching Linear Classifiers over Data Streams 2018 SIGMOD 7.226439e-05
5,971 Optimal and General Out-of-Order Sliding-Window Aggregation 2019 VLDB 5.2480159e-05
6,593 Out of Many We are One: Measuring Item Batch with Clock-Sketch 2021 SIGMOD 4.9999287e-05
7,237 CleanM: An Optimizable Query Language for Unified Scale-Out Data Cleaning 2017 VLDB 4.7928651e-05
7,998 Data Management for Social Networking 2016 PODS 4.6101889e-05
9,318 Disaggregated State Management in Apache Flink® 2.0 2025 VLDB 4.3556432e-05
11,709 Robust, Scalable, Real-Time Event Time Series Aggregation at Twitter 2018 SIGMOD 4.1945683e-05
11,805 CarStream: An Industrial System of Big Data Processing for Internet-of-Vehicles 2017 VLDB 4.1945683e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pig Latin: A Not-So-Foreign Language for Data Processing 2008 SIGMOD 0.0024183614
205 Monitoring Streams – A New Class of Data Management Applications 2002 VLDB 0.00034731577
288 Storm @Twitter 2014 SIGMOD 0.00028939871
314 MillWheel: Fault-Tolerant Stream Processing at Internet Scale 2013 VLDB 0.00028084774
1,286 Photon: Fault-tolerant and Scalable Joining of Continuous Data Streams 2013 SIGMOD 0.0001282373
1,467 SPADE: The System S Declarative Stream Processing Engine 2008 SIGMOD 0.00011849864
2,198 Continuous Analytics Over Discontinuous Streams 2010 SIGMOD 9.308495e-05
2,605 Muppet: MapReduce-Style Processing of Fast Data 2012 VLDB 8.4646171e-05
3,601 Large-Scale Machine Learning at Twitter 2012 SIGMOD 6.9315087e-05
4,572 The Unified Logging Infrastructure for Data Analytics at Twitter 2012 VLDB 6.0760183e-05
6,131 Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture 2013 SIGMOD 5.1956688e-05
6,734 Analytics for the Real-Time Web 2011 VLDB 4.9461128e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
22 SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets 2008 VLDB 0.0008456613
8,108 Execution Primitives for Scalable Joins and Aggregations in Map Reduce 2014 VLDB 4.5846987e-05
1,326 Starling: A Scalable Query Engine on Cloud Functions 2020 SIGMOD 0.00012576952
2,736 Online Aggregation and Continuous Query support in MapReduce 2010 SIGMOD 8.2043187e-05
2,476 A Platform for Scalable One-Pass Analytics using MapReduce 2011 SIGMOD 8.6960139e-05
4,677 Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications 2018 SIGMOD 6.0047822e-05
824 Twitter Heron: Stream Processing at Scale 2015 SIGMOD 0.0001623129
2,818 Implicit Parallelism through Deep Language Embedding 2015 SIGMOD 8.0665558e-05
11,502 In the Land of Data Streams where Synopses are Missing, One Framework to Bring Them All 2021 VLDB 4.1945683e-05
11,709 Robust, Scalable, Real-Time Event Time Series Aggregation at Twitter 2018 SIGMOD 4.1945683e-05