Meta-Dataflows: Efficient Exploratory Dataflow Jobs
Summary: Meta-Dataflows (MDFs) model exploratory dataflow workflows with two primitives: explore and choose. Reuse intermediates, prune branches, and align memory planning with future access to accelerate exploratory runs (up to 90%). (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Raul Castro Fernandez (Massachusetts Institute of Technology)
- 2. William Culhane (Imperial College London)
- 3. Pijika Watcharapichat (Imperial College London)
- 4. Matthias Weidlich (Humboldt University of Berlin)
- 5. Victoria Lopez Morales (Imperial College London)
- 6. Peter Pietzuch (Imperial College London)
BibTeX Citation
@inproceedings{fernandez_sigmod18,
title = {{Meta-Dataflows: Efficient Exploratory Dataflow Jobs}},
author = {Fernandez, Raul Castro and Culhane, William and Watcharapichat, Pijika and Weidlich, Matthias and Morales, Victoria Lopez and Pietzuch, Peter},
series = {{SIGMOD} '18},
booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
publisher = {Association for Computing Machinery},
doi = {10.1145/3183713.3183760},
url = {https://dl.acm.org/doi/10.1145/3183713.3183760},
year = {2018}
}Incoming Citations (Sorted by Pagerank)
Showing 2 of 2 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 1,756 | SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle | 2020 | CIDR | 9.8172465e-05 |
| 2,239 | An Intermediate Representation for Optimizing Machine Learning Pipelines | 2019 | VLDB | 8.8875753e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 8 of 8 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 289 | Optimization of Dynamic Query Evaluation Plans | 1994 | SIGMOD | 0.00022278876 |
| 474 | Dynamic Query Evaluation Plans | 1989 | SIGMOD | 0.00017894953 |
| 919 | Provenance and Scientific Workflows: Challenges and Opportunities | 2008 | SIGMOD | 0.00013214041 |
| 983 | Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management | 2013 | SIGMOD | 0.0001283214 |
| 1,442 | An Architecture for Compiling UDF-centric Workflows | 2015 | VLDB | 0.00010778486 |
| 3,913 | An Approach to Optimize Data Processing in Business Processes | 2007 | VLDB | 7.0230514e-05 |
| 4,595 | The "Big Data" Ecosystem at LinkedIn | 2013 | SIGMOD | 6.6137084e-05 |
| 6,889 | An Algebraic Approach for Data-Centric Scientific Workflows | 2011 | VLDB | 5.7455532e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 361 | The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing | 2015 | VLDB |
| 2 | 4,117 | Schedule Optimization for Data Processing Flows on the Cloud | 2011 | SIGMOD |
| 3 | 11,207 | Pasta: A Cost-Based Optimizer for Generating Pipelining Schedules for Dataflow DAGs | 2024 | SIGMOD |
| 4 | 2,681 | Exploiting Matrix Dependency for Efficient Distributed Matrix Computation | 2015 | SIGMOD |
| 5 | 1,246 | Distributed Evaluation of Subgraph Queries Using Worst-case Optimal Low-Memory Dataflows | 2018 | VLDB |
| 6 | 3,821 | Distributed Subgraph Matching on Timely Dataflow | 2019 | VLDB |
| 7 | 8,958 | The Power of Nested Parallelism in Big Data Processing – Hitting Three Flies with One Slap – | 2021 | SIGMOD |
| 8 | 3,275 | Optimizing Analytic Data Flows for Multiple Execution Engines | 2012 | SIGMOD |
| 9 | 4,845 | Explaining Outputs in Modern Data Analytics | 2016 | VLDB |
| 10 | 2,196 | Spinning Fast Iterative Data Flows | 2012 | VLDB |