SMOKE: Fine-grained Lineage at Interactive Speed

Summary: SMOKE is an in-memory DB engine that tightly integrates lineage capture into physical operators to minimize overhead and accelerate lineage queries. It uses compact lineage representations and upfront-query-aware optimizations to deliver interactive latency (sub-150 ms) and multi-order-of-magnitude improvements over prior systems on real workloads. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 11779
Venue: VLDB
Year: 2018
Pagerank: 9.102574e-05
Overall Rank: 2,286 | 84.12%
DOI: 10.14778/3184470.3184475

Incoming Non-self Citations Over Time

Authors

1. Fotis Psallidas
2. Eugene Wu

Incoming Citations (Sorted by Pagerank)

Showing 25 of 25 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,354	Northstar: An Interactive Data Science System	2018	VLDB	0.00012424105
2,536	DeepLens: Towards a Visual Data Management System	2019	CIDR	8.5817195e-05
3,158	Fine-Grained, Secure and Efficient Data Provenance on Blockchain Systems	2019	VLDB	7.466958e-05
4,779	LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems	2021	SIGMOD	5.9259373e-05
5,706	Putting Things into Context: Rich Explanations for Query Answers using Join Graphs	2021	SIGMOD	5.3633001e-05
5,743	Explaining Wrong Queries Using Small Examples	2019	SIGMOD	5.3432106e-05
5,815	Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data	2020	SIGMOD	5.3158788e-05
6,405	Fine-Grained Lineage for Safer Notebook Interactions	2021	VLDB	5.0707934e-05
6,840	Towards Democratizing Relational Data Visualization	2019	SIGMOD	4.9058387e-05
7,556	Interactive Query Explanations Using Fine Grained Provenance	2022	SIGMOD	4.7072603e-05
8,166	Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science	2021	VLDB	4.567959e-05
8,267	Rumble: Data Independence for Large Messy Data Sets	2021	VLDB	4.5410785e-05
8,725	OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs	2023	VLDB	4.453957e-05
8,886	Provenance-based Data Skipping	2022	VLDB	4.4237386e-05
9,204	Compact, Tamper-Resistant Archival of Fine-Grained Provenance	2021	VLDB	4.3701044e-05
9,600	Optimizing Dataflow Systems for Scalable Interactive Visualization	2024	SIGMOD	4.3136057e-05
9,705	Distributed Numerical and Machine Learning Computations via Two-Phase Execution of Aggregated Join Trees	2021	VLDB	4.2951758e-05
9,920	ProvCite: Provenance-based Data Citation	2019	VLDB	4.250874e-05
10,024	LPStream: Fine-grained Lazy Provenance for Stream Processing	2026	SIGMOD	4.1905499e-05
10,389	FastPDB: Towards Bag-Probabilistic Queries at Interactive Speeds	2025	SIGMOD	4.1905499e-05
10,429	Unified Lineage System: Tracking Data Provenance at Scale	2025	SIGMOD	4.1905499e-05
10,890	FaDE: More Than a Million What-ifs Per Second	2025	VLDB	4.1905499e-05
11,455	Flow Provenance in Temporal Interaction Networks	2021	SIGMOD	4.1905499e-05
11,522	A Demonstration of RELIC: A System for REtrospective Lineage InferenCe of Data Workflows	2021	VLDB	4.1905499e-05
11,716	Demonstration of Smoke: A Deep Breath of Data-Intensive Lineage Applications	2018	SIGMOD	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 28 of 28 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
59	Efficiently Compiling Efficient Query Plans for Modern Hardware	2011	VLDB	0.0006445664
192	HoloClean: Holistic Data Repairs with Probabilistic Inference	2017	VLDB	0.00035692958
213	Scorpion: Explaining Away Outliers in Aggregate Queries	2013	VLDB	0.0003371037
300	Trio: A System for Data, Uncertainty, and Lineage	2006	VLDB	0.00028503742
561	An Annotation Management System for Relational Databases	2004	VLDB	0.00020099434
1,441	Provenance for Generalized Map and Reduce Workflows	2011	CIDR	0.00011950114
1,534	Data Management in Machine Learning: Challenges, Techniques, and Systems	2017	SIGMOD	0.00011462072
1,622	Row-wise Parallel Predicate Evaluation	2008	VLDB	0.00011104582
1,627	Data Profiling with Metanome	2015	VLDB	0.0001108421
1,646	Caravan: Provisioning for What-If Analysis	2013	CIDR	0.00011026644
1,804	M4: A Visualization-Oriented Time Series Data Aggregation	2014	VLDB	0.00010481427
1,824	DBNotes: A Post-It System for Relational Databases based on Provenance	2005	SIGMOD	0.00010395269
2,030	Titian: Data Provenance Support in Spark	2016	VLDB	9.734332e-05
2,182	Querying Data Provenance	2010	SIGMOD	9.3596252e-05
2,318	On Improving User Response Times in Tableau	2015	SIGMOD	9.04679e-05
2,655	Explaining Query Answers with Explanation-Ready Databases	2016	VLDB	8.3638668e-05
2,770	The Semiring Framework for Database Provenance	2017	PODS	8.1495762e-05
2,903	Data Provenance at Internet Scale: Architecture, Experiences, and the Road Ahead	2017	CIDR	7.9403877e-05
3,977	UGuide – User-Guided Discovery of FD-Detectable Errors	2017	SIGMOD	6.5670094e-05
4,160	Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe?	2017	SIGMOD	6.3886736e-05
4,853	Provenance for Natural Language Queries	2017	VLDB	5.8711821e-05
5,212	Explaining Outputs in Modern Data Analytics	2016	VLDB	5.6239563e-05
5,671	Descriptive and Prescriptive Data Cleaning	2014	SIGMOD	5.3797808e-05
5,870	Combining Design and Performance in a Data Visualization Management System	2017	CIDR	5.2914369e-05
6,076	Factorizing Complex Predicates in Queries to Exploit Indexes	2003	SIGMOD	5.2207439e-05
6,383	A Demonstration of DBWipes: Clean as You Query	2012	VLDB	5.0831604e-05
6,777	Revisiting Reuse in Main Memory Database Systems	2017	SIGMOD	4.9242249e-05
8,590	Wisteria: Nurturing Scalable Data Cleaning Infrastructure	2015	VLDB	4.4851741e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
8,049	Thrifty Query Execution via Incrementability	2020	SIGMOD	4.5939412e-05
9,466	Memory Efficient Scheduling of Query Pipeline Execution	2022	CIDR	4.3314303e-05
8,725	OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs	2023	VLDB	4.453957e-05
7,556	Interactive Query Explanations Using Fine Grained Provenance	2022	SIGMOD	4.7072603e-05
10,429	Unified Lineage System: Tracking Data Provenance at Scale	2025	SIGMOD	4.1905499e-05
7,754	Lineage Processing over Correlated Probabilistic Databases	2010	SIGMOD	4.6556861e-05
5,854	Tracing Lineage Beyond Relational Operators	2007	VLDB	5.2982313e-05
11,355	Lineage Resource Manager	2022	SIGMOD	4.1905499e-05
1,763	Efficient Lineage Tracking For Scientific Workflows	2008	SIGMOD	0.00010626896
11,716	Demonstration of Smoke: A Deep Breath of Data-Intensive Lineage Applications	2018	SIGMOD	4.1905499e-05