OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs

Summary: OneProvenance: a log-based system that efficiently extracts coarse-grained provenance from DB query event logs by recovering query execution dependencies via lightweight log analysis and novel event transformations. Adds filtering optimizations to cut noise and capture execution dependencies, yielding up to ~18× faster extraction than prior work and deployed at scale in Microsoft Purview. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13196
Venue: VLDB
Year: 2023
Pagerank: 4.453957e-05
Overall Rank: 8,725 | 39.37%
DOI: 10.14778/3611540.3611555

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,435	Real-time Workload Pattern Analysis for Large-scale Cloud Databases	2023	VLDB	7.0946114e-05
10,429	Unified Lineage System: Tracking Data Provenance at Scale	2025	SIGMOD	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 23 of 23 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
149	Trio: A System for Integrated Management of Data, Accuracy, and Lineage	2005	CIDR	0.00041085859
519	Data Integration for the Relational Web	2009	VLDB	0.00021148006
561	An Annotation Management System for Relational Databases	2004	VLDB	0.00020099434
609	Goods: Organizing Google's Datasets	2016	SIGMOD	0.00019223217
830	Guided Data Repair	2011	VLDB	0.00016125759
936	Data Lake Management: Challenges and Opportunities	2019	VLDB	0.00015197838
1,179	Table Union Search on Open Data	2018	VLDB	0.00013458551
1,274	The Data Civilizer System	2017	CIDR	0.00012869297
1,278	DataHub: Collaborative Data Science & Dataset Version Management at Scale	2015	CIDR	0.00012851949
1,412	VisTrails: Visualization meets Data Management	2006	SIGMOD	0.00012112718
1,565	Principles of Dataset Versioning: Exploring the Recreation/Storage Tradeoff	2015	VLDB	0.00011336478
1,677	Graceful Database Schema Evolution: the PRISM Workbench	2008	VLDB	0.00010929631
1,862	Bootstrapping Pay-As-You-Go Data Integration Systems	2008	SIGMOD	0.00010292134
1,868	Update Exchange with Mappings and Provenance	2007	VLDB	0.0001026365
2,031	Putting Lipstick on Pig: Enabling Database-style Workflow Provenance	2012	VLDB	9.7341007e-05
2,142	LSH Ensemble: Internet-Scale Domain Search	2016	VLDB	9.4461701e-05
2,242	Procedural Extensions of SQL: Understanding their usage in the wild	2021	VLDB	9.2124242e-05
2,267	Ground: A Data Context Service	2017	CIDR	9.1554363e-05
2,286	SMOKE: Fine-grained Lineage at Interactive Speed	2018	VLDB	9.102574e-05
4,785	Ibis: A Provenance Manager for Multi-Layer Systems	2011	CIDR	5.9196501e-05
4,802	CLAMS: Bringing Quality to Data Lakes	2016	SIGMOD	5.9064959e-05
5,088	Improving Reproducibility of Data Science Pipelines through Transparent Provenance Capture	2020	VLDB	5.7023581e-05
7,838	Dependency-Driven Analytics: a Compass for Uncharted Data Oceans	2017	CIDR	4.6338262e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
4,853	Provenance for Natural Language Queries	2017	VLDB	5.8711821e-05
8,166	Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science	2021	VLDB	4.567959e-05
1,106	Provenance for Aggregate Queries	2011	PODS	0.00013976386
8,965	Computing How-Provenance for SPARQL Queries via Query Rewriting	2021	VLDB	4.4163852e-05
2,903	Data Provenance at Internet Scale: Architecture, Experiences, and the Road Ahead	2017	CIDR	7.9403877e-05
8,228	You Say 'What', I Hear 'Where' and 'Why' - (Mis-)Interpreting SQL to Derive Fine-Grained Provenance	2018	VLDB	4.549777e-05
7,556	Interactive Query Explanations Using Fine Grained Provenance	2022	SIGMOD	4.7072603e-05
2,182	Querying Data Provenance	2010	SIGMOD	9.3596252e-05
6,187	On Provenance Minimization	2011	PODS	5.1611195e-05
11,670	Ursprung: Provenance for Large-Scale Analytics Environments	2019	SIGMOD	4.1905499e-05