Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines
Summary: A 3500-container streaming pipeline with SLO on tail latency contends with co-tenant load, stalls, and resource delays. Spur detects slow instances with pruning to drive speculative replication, yielding 10–38% p99 gains using 0.5% extra resources. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Ke Wang (Carnegie Mellon University)
- 2. Avrilia Floratou (Microsoft)
- 3. Ashvin Agrawal (Microsoft)
- 4. Daniel Musgrave (Netflix)
BibTeX Citation
@inproceedings{wang_sigmod20,
title = {{Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines}},
author = {Wang, Ke and Floratou, Avrilia and Agrawal, Ashvin and Musgrave, Daniel},
series = {{SIGMOD} '20},
booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
publisher = {Association for Computing Machinery},
doi = {10.1145/3318464.3386142},
url = {https://dl.acm.org/doi/10.1145/3318464.3386142},
year = {2020}
}Incoming Citations (Sorted by Pagerank)
Showing 2 of 2 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 6,787 | Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile | 2021 | VLDB | 5.6820544e-05 |
| 7,806 | Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward | 2021 | VLDB | 5.4500623e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 9 of 9 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 30 | SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets | 2008 | VLDB | 0.00050495102 |
| 231 | Storm @Twitter | 2014 | SIGMOD | 0.00023841089 |
| 325 | The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing | 2015 | VLDB | 0.00020964941 |
| 606 | Twitter Heron: Stream Processing at Scale | 2015 | SIGMOD | 0.00015635133 |
| 685 | Trill: A High-Performance Incremental Query Processor for Diverse Analytics | 2015 | VLDB | 0.00014782777 |
| 1,123 | Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark | 2018 | SIGMOD | 0.0001193233 |
| 1,233 | Dhalion: Self-Regulating Stream Processing in Heron | 2017 | VLDB | 0.00011405873 |
| 1,771 | Samza: Stateful Scalable Stream Processing at LinkedIn | 2017 | VLDB | 9.6803752e-05 |
| 9,813 | Dhalion in Action: Automatic Management of Streaming Applications | 2018 | VLDB | 5.1257999e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 9,633 | Shrink - Prescribing Resiliency Solutions for Streaming | 2017 | VLDB |
| 2 | 968 | Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management | 2013 | SIGMOD |
| 3 | 9,640 | Providing Resiliency to Load Variations in Distributed Stream Processing | 2006 | VLDB |
| 4 | 5,653 | Staying FIT: Efficient Load Shedding Techniques for Distributed Stream Processing | 2007 | VLDB |
| 5 | 6,541 | A Holistic View of Stream Partitioning Costs | 2017 | VLDB |
| 6 | 11,559 | Intelligent Pooling: Proactive Resource Provisioning in Large-scale Cloud Service | 2024 | VLDB |
| 7 | 4,818 | Continuous Cloud-Scale Query Optimization and Processing | 2013 | VLDB |
| 8 | 8,678 | Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems | 2020 | SIGMOD |
| 9 | 7,356 | Parallelism-Optimizing Data Placement for Faster Data-Parallel Computations | 2023 | VLDB |
| 10 | 9,816 | Supporting Scalable Analytics with Latency Constraints | 2015 | VLDB |