Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science

Summary: Fine-grained, element-level provenance for ML preprocessing pipelines. Formalizes a core set of preprocessing operators and provenance patterns; introduces an application-level Python library and evaluates scalability/overhead on real pipelines to enable debugging queries. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 12572
Venue: VLDB
Year: 2021
Pagerank: 4.567959e-05
Overall Rank: 8,166 | 43.25%
DOI: 10.14778/3436905.3436911

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 4 of 4 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,116	Towards Observability for Production Machine Learning Pipelines	2022	VLDB	4.3886184e-05
9,234	Modyn: Data-Centric Machine Learning Pipeline Orchestration	2025	SIGMOD	4.3648789e-05
10,429	Unified Lineage System: Tracking Data Provenance at Scale	2025	SIGMOD	4.1905499e-05
11,399	DPDS: Assisting Data Science with Data Provenance	2022	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
31	Provenance Semirings	2007	PODS	0.00078516827
917	Democratizing Data Science through Interactive Curation of ML Pipelines	2019	SIGMOD	0.00015324193
1,427	Towards Scalable Dataframe Systems	2020	VLDB	0.00012033503
1,824	DBNotes: A Post-It System for Relational Databases based on Provenance	2005	SIGMOD	0.00010395269
2,030	Titian: Data Provenance Support in Spark	2016	VLDB	9.734332e-05
2,157	MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis	2018	SIGMOD	9.4153917e-05
2,286	SMOKE: Fine-grained Lineage at Interactive Speed	2018	VLDB	9.102574e-05
2,462	noWorkflow: a Tool for Collecting, Analyzing, and Managing Provenance from Python Scripts	2017	VLDB	8.7477473e-05
4,006	Data Platform for Machine Learning	2019	SIGMOD	6.5371762e-05
4,423	PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models	2020	SIGMOD	6.1925724e-05
5,115	TPC-DI: The First Industry Benchmark for Data Integration	2014	VLDB	5.6808821e-05
7,384	The VADA Architecture for Cost-Effective Data Wrangling	2017	SIGMOD	4.7400209e-05
8,335	BugDoc: Algorithms to Debug Computational Processes	2020	SIGMOD	4.538972e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
8,392	Hypothetical Reasoning via Provenance Abstraction	2019	SIGMOD	4.5234647e-05
11,670	Ursprung: Provenance for Large-Scale Analytics Environments	2019	SIGMOD	4.1905499e-05
1,763	Efficient Lineage Tracking For Scientific Workflows	2008	SIGMOD	0.00010626896
11,474	On Optimizing the Trade-off between Privacy and Utility in Data Provenance	2021	SIGMOD	4.1905499e-05
2,182	Querying Data Provenance	2010	SIGMOD	9.3596252e-05
5,854	Tracing Lineage Beyond Relational Operators	2007	VLDB	5.2982313e-05
8,725	OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs	2023	VLDB	4.453957e-05
2,456	Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities	2021	SIGMOD	8.7649259e-05
11,399	DPDS: Assisting Data Science with Data Provenance	2022	VLDB	4.1905499e-05
5,088	Improving Reproducibility of Data Science Pipelines through Transparent Provenance Capture	2020	VLDB	5.7023581e-05