Capturing and Querying Structural Provenance in Spark with Pebble
Summary: Pebble captures and queries structural provenance for nested data in Spark, tracking access and modification of both top-level and nested items. Tree-pattern provenance queries and a Jupyter GUI enable debugging pipelines in a standalone Spark library. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
No non-self incoming citations found for this paper in this database.
Authors
- 1. Ralf Diestelkämper (University of Stuttgart)
- 2. Melanie Herschel (University of Stuttgart)
BibTeX Citation
@inproceedings{diestelkamper_sigmod19,
title = {{Capturing and Querying Structural Provenance in Spark with Pebble}},
author = {Diestelkämper, Ralf and Herschel, Melanie},
series = {{SIGMOD} '19},
booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
publisher = {Association for Computing Machinery},
doi = {10.1145/3299869.3320225},
url = {https://dl.acm.org/doi/10.1145/3299869.3320225},
year = {2019}
}Incoming Citations (Sorted by Pagerank)
Showing 0 of 0 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 2 of 2 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 1,388 | Provenance for Generalized Map and Reduce Workflows | 2011 | CIDR | 0.00010821908 |
| 1,765 | Putting Lipstick on Pig: Enabling Database-style Workflow Provenance | 2012 | VLDB | 9.6955585e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 8,583 | OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs | 2023 | VLDB |
| 2 | 1,388 | Provenance for Generalized Map and Reduce Workflows | 2011 | CIDR |
| 3 | 4,801 | Improving Reproducibility of Data Science Pipelines through Transparent Provenance Capture | 2020 | VLDB |
| 4 | 10,927 | A Decade of Apache Spark Structured Streaming: How We Evolved The Architecture To Meet Real-World Needs | 2026 | VLDB |
| 5 | 11,902 | DPDS: Assisting Data Science with Data Provenance | 2022 | VLDB |
| 6 | 10,194 | LPStream: Fine-grained Lazy Provenance for Stream Processing | 2026 | SIGMOD |
| 7 | 1,564 | Titian: Data Provenance Support in Spark | 2016 | VLDB |
| 8 | 1,123 | Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark | 2018 | SIGMOD |
| 9 | 7,212 | Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science | 2021 | VLDB |
| 10 | 10,024 | Debugging Missing Answers for Spark Queries over Nested Data with Breadcrumb | 2021 | VLDB |