Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines
Summary: A 3500-container streaming pipeline with SLO on tail latency contends with co-tenant load, stalls, and resource delays. Spur detects slow instances with pruning to drive speculative replication, yielding 10–38% p99 gains using 0.5% extra resources. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Ke Wang (Carnegie Mellon University)
- 2. Avrilia Floratou (Microsoft)
- 3. Ashvin Agrawal (Microsoft)
- 4. Daniel Musgrave (Netflix)
BibTeX Citation
@inproceedings{wang_sigmod20,
title = {{Spur: Mitigating Slow Instances in Large-Scale Streaming Pipelines}},
author = {Wang, Ke and Floratou, Avrilia and Agrawal, Ashvin and Musgrave, Daniel},
series = {{SIGMOD} '20},
booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
publisher = {Association for Computing Machinery},
doi = {10.1145/3318464.3386142},
url = {https://dl.acm.org/doi/10.1145/3318464.3386142},
year = {2020}
}Incoming Citations (Sorted by Pagerank)
Showing 2 of 2 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 7,294 | Hazelcast Jet: Low-latency Stream Processing at the 99.99th Percentile | 2021 | VLDB | 5.6532162e-05 |
| 7,661 | Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward | 2021 | VLDB | 5.5736026e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 9 of 9 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 30 | SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets | 2008 | VLDB | 0.00051174276 |
| 220 | Storm @Twitter | 2014 | SIGMOD | 0.00024244587 |
| 361 | The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing | 2015 | VLDB | 0.00020138717 |
| 612 | Twitter Heron: Stream Processing at Scale | 2015 | SIGMOD | 0.0001573018 |
| 710 | Trill: A High-Performance Incremental Query Processor for Diverse Analytics | 2015 | VLDB | 0.00014715033 |
| 1,190 | Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark | 2018 | SIGMOD | 0.00011743246 |
| 1,224 | Dhalion: Self-Regulating Stream Processing in Heron | 2017 | VLDB | 0.00011596911 |
| 1,895 | Samza: Stateful Scalable Stream Processing at LinkedIn | 2017 | VLDB | 9.5260291e-05 |
| 9,637 | Dhalion in Action: Automatic Management of Streaming Applications | 2018 | VLDB | 5.2434488e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 9,531 | Shrink - Prescribing Resiliency Solutions for Streaming | 2017 | VLDB |
| 2 | 983 | Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management | 2013 | SIGMOD |
| 3 | 9,460 | Providing Resiliency to Load Variations in Distributed Stream Processing | 2006 | VLDB |
| 4 | 5,521 | Staying FIT: Efficient Load Shedding Techniques for Distributed Stream Processing | 2007 | VLDB |
| 5 | 6,410 | A Holistic View of Stream Partitioning Costs | 2017 | VLDB |
| 6 | 11,222 | Intelligent Pooling: Proactive Resource Provisioning in Large-scale Cloud Service | 2024 | VLDB |
| 7 | 4,742 | Continuous Cloud-Scale Query Optimization and Processing | 2013 | VLDB |
| 8 | 8,568 | Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems | 2020 | SIGMOD |
| 9 | 8,091 | Parallelism-Optimizing Data Placement for Faster Data-Parallel Computations | 2023 | VLDB |
| 10 | 9,640 | Supporting Scalable Analytics with Latency Constraints | 2015 | VLDB |