Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science

Summary: Fine-grained, element-level provenance for ML preprocessing pipelines. Formalizes a core set of preprocessing operators and provenance patterns; introduces an application-level Python library and evaluates scalability/overhead on real pipelines to enable debugging queries. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 12571
Venue: VLDB
Year: 2021
Pagerank: 4.5723431e-05
Overall Rank: 8,163 | 43.22%
DOI: 10.14778/3436905.3436911

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 4 of 4 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,118	Towards Observability for Production Machine Learning Pipelines	2022	VLDB	4.3928288e-05
9,231	Modyn: Data-Centric Machine Learning Pipeline Orchestration	2025	SIGMOD	4.3690661e-05
10,419	Unified Lineage System: Tracking Data Provenance at Scale	2025	SIGMOD	4.1945683e-05
11,396	DPDS: Assisting Data Science with Data Provenance	2022	VLDB	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
31	Provenance Semirings	2007	PODS	0.0007857786
921	Democratizing Data Science through Interactive Curation of ML Pipelines	2019	SIGMOD	0.00015337438
1,427	Towards Scalable Dataframe Systems	2020	VLDB	0.0001204248
1,824	DBNotes: A Post-It System for Relational Databases based on Provenance	2005	SIGMOD	0.00010405194
2,027	Titian: Data Provenance Support in Spark	2016	VLDB	9.7437067e-05
2,152	MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis	2018	SIGMOD	9.4239787e-05
2,280	SMOKE: Fine-grained Lineage at Interactive Speed	2018	VLDB	9.1111033e-05
2,463	noWorkflow: a Tool for Collecting, Analyzing, and Managing Provenance from Python Scripts	2017	VLDB	8.7561396e-05
4,003	Data Platform for Machine Learning	2019	SIGMOD	6.54347e-05
4,424	PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models	2020	SIGMOD	6.198474e-05
5,114	TPC-DI: The First Industry Benchmark for Data Integration	2014	VLDB	5.6863051e-05
7,384	The VADA Architecture for Cost-Effective Data Wrangling	2017	SIGMOD	4.7445719e-05
8,341	BugDoc: Algorithms to Debug Computational Processes	2020	SIGMOD	4.5433282e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
8,394	Hypothetical Reasoning via Provenance Abstraction	2019	SIGMOD	4.527807e-05
11,665	Ursprung: Provenance for Large-Scale Analytics Environments	2019	SIGMOD	4.1945683e-05
1,765	Efficient Lineage Tracking For Scientific Workflows	2008	SIGMOD	0.00010630348
2,173	Querying Data Provenance	2010	SIGMOD	9.3676609e-05
11,471	On Optimizing the Trade-off between Privacy and Utility in Data Provenance	2021	SIGMOD	4.1945683e-05
5,843	Tracing Lineage Beyond Relational Operators	2007	VLDB	5.3032967e-05
8,729	OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs	2023	VLDB	4.4582221e-05
2,456	Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities	2021	SIGMOD	8.7733773e-05
11,396	DPDS: Assisting Data Science with Data Provenance	2022	VLDB	4.1945683e-05
5,086	Improving Reproducibility of Data Science Pipelines through Transparent Provenance Capture	2020	VLDB	5.7078462e-05