Optimizing Machine Learning Workloads in Collaborative Environments

Summary: Introduces Experiment Graph (EG) to persist artifacts (data/models) as vertices and ML operations as edges for collaborative ML workloads. Proposes two materialization strategies and a linear-time reuse algorithm to cache artifacts and plan execution, yielding up to 10x speedups on repeats and ~50% on edits. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5992
Venue: SIGMOD
Year: 2020
Pagerank: 6.1170243e-05
Overall Rank: 5,699 | 60.91%
DOI: 10.1145/3318464.3389715

Incoming Non-self Citations Over Time

Authors

1. Behrouz Derakhshan (German National Research Center for Information Technology)
2. Alireza Rezaei Mahdiraji (German National Research Center for Information Technology)
3. Ziawasch Abedjan (Technical University of Berlin)
4. Tilmann Rabl (Hasso Plattner Institute; University of Potsdam)
5. Volker Markl (German National Research Center for Information Technology; Technical University of Berlin)

BibTeX Citation

@inproceedings{derakhshan_sigmod20,
        title = {{Optimizing Machine Learning Workloads in Collaborative Environments}},
        author = {Derakhshan, Behrouz and Mahdiraji, Alireza Rezaei and Abedjan, Ziawasch and Rabl, Tilmann and Markl, Volker},
        series = {{SIGMOD} '20},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3318464.3389715},
        url = {https://dl.acm.org/doi/10.1145/3318464.3389715},
        year = {2020}
}

Incoming Citations (Sorted by Pagerank)

Showing 10 of 10 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,634	Orca: Scalable Temporal Graph Neural Network Training with Theoretical Guarantees	2023	SIGMOD	7.2358691e-05
4,067	Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches	2021	VLDB	6.9293511e-05
4,240	LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems	2021	SIGMOD	6.809685e-05
6,309	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD	5.9189554e-05
6,538	UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads	2022	VLDB	5.8477764e-05
7,687	ExDRa: Exploratory Data Science on Federated Raw Data	2021	SIGMOD	5.5671645e-05
10,540	CAPS: Cost-Aware ML Pipeline Selection	2026	VLDB	5.093636e-05
10,750	Alsatian: Optimizing Model Search for Deep Transfer Learning	2025	SIGMOD	5.093636e-05
11,066	ML-Asset Management: Curation, Discovery, and Utilization	2025	VLDB	5.093636e-05
11,674	Enforcing Constraints for Machine Learning Systems via Declarative Feature Selection: An Experimental Study	2021	SIGMOD	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 10 of 10 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
640	Materialization Optimizations for Feature Selection Workloads	2014	SIGMOD	0.00015409494
1,004	Democratizing Data Science through Interactive Curation of ML Pipelines	2019	SIGMOD	0.00012701932
1,077	DataHub: Collaborative Data Science & Dataset Version Management at Scale	2015	CIDR	0.00012269438
1,349	Principles of Dataset Versioning: Exploring the Recreation/Storage Tradeoff	2015	VLDB	0.00011069122
1,569	HELIX: Holistic Optimization for Accelerating Iterative Machine Learning	2019	VLDB	0.00010335423
1,670	MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis	2018	SIGMOD	0.00010045615
1,883	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.5421713e-05
1,902	Ground: A Data Context Service	2017	CIDR	9.506714e-05
3,146	HELIX: Accelerating Human-in-the-loop Machine Learning	2018	VLDB	7.7091591e-05
7,065	Dataset Relationship Management	2019	CIDR	5.7123431e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,246	Distributed Evaluation of Subgraph Queries Using Worst-case Optimal Low-Memory Dataflows	2018	VLDB
2	9,371	Towards an Optimized GROUP BY Abstraction for Large-Scale Machine Learning	2021	VLDB
3	4,240	LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems	2021	SIGMOD
4	1,349	Principles of Dataset Versioning: Exploring the Recreation/Storage Tradeoff	2015	VLDB
5	13,469	Towards Scalable Online Machine Learning Collaborations with OpenML	2021	VLDB
6	2,029	Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads	2018	VLDB
7	4,049	Resource Elasticity for Large-Scale Machine Learning	2015	SIGMOD
8	2,657	Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities	2021	SIGMOD
9	6,038	Efficient Construction of Approximate Ad-Hoc ML models Through Materialization and Reuse	2018	VLDB
10	6,309	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD