Debugging Large-Scale Data Science Pipelines using Dagger
Summary: End-to-end debugger for data pipelines; Dagger isolates data-centric errors from transformations to model inputs. It enables inter-module (black-box blocks) and intra-module (inspect DataFrames/Python objects) debugging, demonstrated on Intel BI workloads. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. El Kindi Rezig (Massachusetts Institute of Technology)
- 2. Ashrita Brahmaroutu (Intel)
- 3. Nesime Tatbul (Intel; Massachusetts Institute of Technology)
- 4. Mourad Ouzzani (Qatar Computing Research Institute)
- 5. Nan Tang (Qatar Computing Research Institute)
- 6. Timothy Mattson (Intel)
- 7. Samuel Madden (Massachusetts Institute of Technology)
- 8. Michael Stonebraker (Massachusetts Institute of Technology)
BibTeX Citation
@article{rezig_vldb20,
title = {{Debugging Large-Scale Data Science Pipelines using Dagger}},
author = {Rezig, El Kindi and Brahmaroutu, Ashrita and Tatbul, Nesime and Ouzzani, Mourad and Tang, Nan and Mattson, Timothy and Madden, Samuel and Stonebraker, Michael},
journal = {PVLDB},
series = {{VLDB} '20},
volume = {13},
number = {12},
pages = {2993--2996},
doi = {10.14778/3415478.3415527},
url = {https://doi.org/10.14778/3415478.3415527},
year = {2020}
}Incoming Citations (Sorted by Pagerank)
Showing 3 of 3 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 7,068 | DataPrism: Exposing Disconnect between Data and Systems | 2022 | SIGMOD | 5.7119157e-05 |
| 11,607 | Machine Programming: Turning Data into Programmer Productivity | 2022 | VLDB | 5.093636e-05 |
| 13,435 | Data Cleaning in the Era of Data Science: Challenges and Opportunities | 2021 | CIDR | - |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 8 of 8 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 963 | The Data Civilizer System | 2017 | CIDR | 0.00012935145 |
| 1,351 | Detecting Data Errors: Where are we and what needs to be done? | 2016 | VLDB | 0.00011064851 |
| 1,392 | Northstar: An Interactive Data Science System | 2018 | VLDB | 0.00010936065 |
| 1,670 | MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis | 2018 | SIGMOD | 0.00010045615 |
| 3,146 | HELIX: Accelerating Human-in-the-loop Machine Learning | 2018 | VLDB | 7.7091591e-05 |
| 4,373 | Data Debugging and Exploration with Vizier | 2019 | SIGMOD | 6.738145e-05 |
| 5,404 | Dagger: A Data (not code) Debugger | 2020 | CIDR | 6.2294192e-05 |
| 8,242 | Data Civilizer 2.0: A Holistic Framework for Data Preparation and Analytics | 2019 | VLDB | 5.459288e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 9,245 | Towards Observability for Production Machine Learning Pipelines | 2022 | VLDB |
| 2 | 11,043 | mlidea: Interactively Improving ML Data Preparation Code via “Shadow Pipelines” | 2025 | VLDB |
| 3 | 6,794 | Error Diagnosis and Data Profiling with Data X-Ray | 2015 | VLDB |
| 4 | 13,348 | Bridging Disciplines in Data Management Research to Solve Complex Data Problems | 2025 | VLDB |
| 5 | 4,518 | MLINSPECT: A Data Distribution Debugger for Machine Learning Pipelines | 2021 | SIGMOD |
| 6 | 8,185 | BugDoc: Algorithms to Debug Computational Processes | 2020 | SIGMOD |
| 7 | 11,353 | Reconstructing and Querying ML Pipeline Intermediates | 2023 | CIDR |
| 8 | 4,373 | Data Debugging and Exploration with Vizier | 2019 | SIGMOD |
| 9 | 9,418 | BugDoc: A System for Debugging Computational Pipelines | 2020 | SIGMOD |
| 10 | 5,404 | Dagger: A Data (not code) Debugger | 2020 | CIDR |