LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems

Summary: Fine-grained lineage tracing and reuse in ML systems (LIMA) to break coarse, black-box limits. Multi-level traces, loop/function dedup, and cross-hierarchy reuse enable low-overhead provenance with versioning, compatible with task parallelism and operator fusion, delivering up to 12.4x speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6070
Venue: SIGMOD
Year: 2021
Pagerank: 6.9157444e-05
Overall Rank: 4,177 | 70.98%
DOI: 10.1145/3448016.3452788

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 15 of 15 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
7,033	DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines	2022	CIDR	5.7748146e-05
7,476	Nautilus: An Optimized System for Deep Transfer Learning over Evolving Training Datasets	2022	SIGMOD	5.6710868e-05
7,559	ExDRa: Exploratory Data Science on Federated Raw Data	2021	SIGMOD	5.6533871e-05
7,788	Provenance-Enabled Explainable AI	2024	SIGMOD	5.6035684e-05
7,847	Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications	2023	SIGMOD	5.6015205e-05
8,441	UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads	2022	VLDB	5.4955352e-05
9,778	The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format	2024	SIGMOD	5.2743647e-05
9,922	ElasticNotebook: Enabling Live Migration for Computational Notebooks	2024	VLDB	5.2412669e-05
10,252	CAPS: Cost-Aware ML Pipeline Selection	2026	VLDB	5.1725247e-05
10,303	Morphing-based Compression for Data-centric ML Pipelines	2026	VLDB	5.1725247e-05
10,429	Unified Lineage System: Tracking Data Provenance at Scale	2025	SIGMOD	5.1725247e-05
10,479	Alsatian: Optimizing Model Search for Deep Transfer Learning	2025	SIGMOD	5.1725247e-05
10,636	CatDB: Data-catalog-guided, LLM-based Generation of Data-centric ML Pipelines	2025	VLDB	5.1725247e-05
10,846	ML-Asset Management: Curation, Discovery, and Utilization	2025	VLDB	5.1725247e-05
11,341	Redundancy Elimination in Distributed Matrix Computation	2022	SIGMOD	5.1725247e-05

Outgoing Citations (Sorted by Pagerank)

Showing 50 of 54 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0010826673
17	Provenance Semirings	2007	PODS	0.00059742957
85	Automated Selection of Materialized Views and Indexes for SQL Databases	2000	VLDB	0.00035865404
127	Efficient and Extensible Algorithms for Multi Query Optimization	2000	SIGMOD	0.00031128834
257	Database Cracking	2007	CIDR	0.00023290946
381	Why Not?	2009	SIGMOD	0.0001962695
398	MauveDB: Supporting Model-based User Views in Database Systems	2006	SIGMOD	0.00019336176
403	SystemML: Declarative Machine Learning on Spark	2016	VLDB	0.00019232796
504	Goods: Organizing Google's Datasets	2016	SIGMOD	0.00017409807
589	Incremental Knowledge Base Construction Using DeepDive	2015	VLDB	0.00016077388
635	Materialization Optimizations for Feature Selection Workloads	2014	SIGMOD	0.00015616114
989	Democratizing Data Science through Interactive Curation of ML Pipelines	2019	SIGMOD	0.00012871011
1,009	The DataPath System: A Data-Centric Analytic Processing Engine for Large Data Warehouses	2010	SIGMOD	0.00012763646
1,064	DataHub: Collaborative Data Science & Dataset Version Management at Scale	2015	CIDR	0.00012444051
1,140	Efficient Exploitation of Similar Subexpressions for Query Processing	2007	SIGMOD	0.00012092506
1,214	Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML	2014	VLDB	0.00011714799
1,347	Provenance for Generalized Map and Reduce Workflows	2011	CIDR	0.00011181443
1,432	An Architecture for Compiling UDF-centric Workflows	2015	VLDB	0.00010905682
1,451	VisTrails: Visualization meets Data Management	2006	SIGMOD	0.0001082042
1,536	HELIX: Holistic Optimization for Accelerating Iterative Machine Learning	2019	VLDB	0.00010490973
1,570	Update Exchange with Mappings and Provenance	2007	VLDB	0.00010390225
1,578	Titian: Data Provenance Support in Spark	2016	VLDB	0.00010366201
1,649	MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis	2018	SIGMOD	0.00010189983
1,737	Selecting Subexpressions to Materialize at Datacenter Scale	2018	VLDB	9.9473651e-05
1,753	Putting Lipstick on Pig: Enabling Database-style Workflow Provenance	2012	VLDB	9.9195501e-05
1,764	SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle	2020	CIDR	9.890878e-05
1,771	SMOKE: Fine-grained Lineage at Interactive Speed	2018	VLDB	9.8683462e-05
1,863	Predictable Performance for Unpredictable Workloads	2009	VLDB	9.6827833e-05
1,915	Elastic Machine Learning Algorithms in Amazon SageMaker	2020	SIGMOD	9.5806135e-05
2,010	Data Market Platforms: Trading Data Assets to Solve Data Problems	2020	VLDB	9.3907822e-05
2,046	LINVIEW: Incremental View Maintenance for Complex Analytical Queries	2014	SIGMOD	9.3348757e-05
2,130	An Architecture for Recycling Intermediates in a Column-store	2009	SIGMOD	9.1825258e-05
2,189	An Intermediate Representation for Optimizing Machine Learning Pipelines	2019	VLDB	9.0524569e-05
2,192	Cumulon: Optimizing Statistical Data Analysis in the Cloud	2013	SIGMOD	9.0425859e-05
2,274	Evaluating End-to-End Optimization for Data Analytics Applications in Weld	2018	VLDB	8.8988246e-05
2,805	Fine-Grained, Secure and Efficient Data Provenance on Blockchain Systems	2019	VLDB	8.1677906e-05
2,887	Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations	2019	SIGMOD	8.0623053e-05
3,211	On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML	2018	VLDB	7.7053042e-05
3,390	Incremental View Maintenance with Triple Lock Factorization Benefits	2018	SIGMOD	7.5251617e-05
3,509	Cloudy with High Chance of DBMS: A 10-year Prediction for Enterprise-Grade ML	2020	CIDR	7.4227748e-05
4,059	Data Integration and Machine Learning: A Natural Synergy	2018	SIGMOD	6.9975008e-05
4,151	The Missing Piece in Complex Analytics: Low Latency, Scalable Model Management and Serving with Velox	2015	CIDR	6.9299082e-05
4,155	SPOOF: Sum-Product Optimization and Operator Fusion for Large-Scale Machine Learning	2017	CIDR	6.9267903e-05
4,450	Juneau: Data Lake Management for Jupyter	2019	VLDB	6.753337e-05
4,924	SPORES: Sum-Product Optimization via Relational Equality Saturation for Large Scale Linear Algebra	2020	VLDB	6.5073091e-05
5,606	Optimizing Machine Learning Workloads in Collaborative Environments	2020	SIGMOD	6.2160995e-05
5,754	Incrementally Maintaining Classification using an RDBMS	2011	VLDB	6.1624967e-05
5,840	Your notebook is not crumby enough, REPLace it	2020	CIDR	6.1304757e-05
5,985	Fine-Grained Lineage for Safer Notebook Interactions	2021	VLDB	6.0801131e-05
6,170	Lightweight Inspection of Data Preprocessing in Native Machine Learning Pipelines	2021	CIDR	6.0279644e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
9,237	Towards an Optimized GROUP BY Abstraction for Large-Scale Machine Learning	2021	VLDB	5.3573019e-05
7,847	Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications	2023	SIGMOD	5.6015205e-05
1,842	Efficient Lineage Tracking For Scientific Workflows	2008	SIGMOD	9.7225975e-05
2,189	An Intermediate Representation for Optimizing Machine Learning Pipelines	2019	VLDB	9.0524569e-05
6,170	Lightweight Inspection of Data Preprocessing in Native Machine Learning Pipelines	2021	CIDR	6.0279644e-05
7,925	Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science	2021	VLDB	5.58238e-05
3,211	On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML	2018	VLDB	7.7053042e-05
5,606	Optimizing Machine Learning Workloads in Collaborative Environments	2020	SIGMOD	6.2160995e-05
2,623	Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities	2021	SIGMOD	8.3971735e-05
6,235	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD	6.0072575e-05