Back to papers
Opening the Black Boxes in Data Flow Optimization
Summary: Optimizes data flows with black-box UDFs by extracting a few operator properties to determine safe reordering, via static analysis of UDF code. The optimizer enables selection reordering, bushy join ordering, and limited aggregation push-down for non-relational flows, achieving relational-like rewrite power without relying on operator semantics.
(summarized by gpt-5-nano on Feb 09 2026)
- Paper ID
- 10364
- Venue
- VLDB
- Year
- 2012
- Pagerank
- 8.4457819e-05
- Overall Rank
- 2,616 | 81.83%
- DOI
-
-
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 21 of 21 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 1,407 |
Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML |
2014 |
VLDB |
0.00012163413 |
| 1,884 |
Tuplex: Data Science in Python at Native Code Speed |
2021 |
SIGMOD |
0.00010206514 |
| 2,179 |
Spinning Fast Iterative Data Flows |
2012 |
VLDB |
9.3632007e-05 |
| 2,355 |
An Intermediate Representation for Optimizing Machine Learning Pipelines |
2019 |
VLDB |
8.9727612e-05 |
| 3,270 |
RHEEM: Enabling Cross-Platform Data Processing - May The Big Data Be With You! - |
2018 |
VLDB |
7.3013196e-05 |
| 4,909 |
A Method for Optimizing Opaque Filter Queries |
2020 |
SIGMOD |
5.8354189e-05 |
| 5,012 |
Dynamically Optimizing Queries over Large Scale Data Platforms |
2014 |
SIGMOD |
5.7543101e-05 |
| 5,438 |
The NebulaStream Platform: Data and Application Management for the Internet of Things |
2020 |
CIDR |
5.5041764e-05 |
| 5,486 |
Containerized Execution of UDFs: An Experimental Evaluation |
2022 |
VLDB |
5.481452e-05 |
| 6,079 |
Opportunistic Physical Design for Big Data Analytics |
2014 |
SIGMOD |
5.2194019e-05 |
| 6,321 |
The BUDS Language for Distributed Bayesian Machine Learning |
2017 |
SIGMOD |
5.1075833e-05 |
| 7,303 |
DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines |
2022 |
CIDR |
4.7632836e-05 |
| 7,807 |
Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques |
2023 |
VLDB |
4.6417687e-05 |
| 8,435 |
Not Black-Box Anymore! Enabling Analytics-Aware Optimizations in Teradata Vantage |
2021 |
VLDB |
4.5075971e-05 |
| 9,191 |
POLAR: Adaptive and Non-invasive Join Order Selection via Plans of Least Resistance |
2024 |
VLDB |
4.3738237e-05 |
| 9,384 |
Versatile Optimization of UDF-heavy Data Flows with Sofa |
2014 |
SIGMOD |
4.3432098e-05 |
| 9,520 |
PAXQuery: Parallel Analytical XML Processing |
2015 |
SIGMOD |
4.3282242e-05 |
| 9,846 |
HyperBlocker: Accelerating Rule-based Blocking in Entity Resolution using GPUs |
2025 |
VLDB |
4.2680295e-05 |
| 10,776 |
cedar: Optimized and Unified Machine Learning Input Data Pipelines |
2025 |
VLDB |
4.1905499e-05 |
| 12,038 |
How Achaeans Would Construct Columns in Troy |
2013 |
CIDR |
4.1905499e-05 |
| 12,047 |
Iterative Parallel Data Processing with Stratosphere: An Inside Look |
2013 |
SIGMOD |
4.1905499e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 13 of 13 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank |
Cited Paper |
Year |
Venue |
Pagerank |
| 1 |
Access Path Selection in a Relational Database Management System |
1979 |
SIGMOD |
0.0040465394 |
| 3 |
Pig Latin: A Not-So-Foreign Language for Data Processing |
2008 |
SIGMOD |
0.0024217964 |
| 22 |
SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets |
2008 |
VLDB |
0.00084679526 |
| 51 |
Including Group-By in Query Optimization |
1994 |
VLDB |
0.00067060283 |
| 70 |
Hive - A Warehousing Solution Over a Map-Reduce Framework |
2009 |
VLDB |
0.00059744625 |
| 866 |
Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs |
2011 |
VLDB |
0.00015771189 |
| 909 |
Tenzing A SQL Implementation On The MapReduce Framework |
2011 |
VLDB |
0.0001540191 |
| 1,265 |
Jaql: A Scripting Language for Large Scale Semistructured Data Analysis |
2011 |
VLDB |
0.00012933537 |
| 1,281 |
Automatic Optimization for MapReduce Programs |
2011 |
VLDB |
0.00012837411 |
| 1,344 |
Dynamic Programming Strikes Back |
2008 |
SIGMOD |
0.00012477274 |
| 1,361 |
SQL/MapReduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions |
2009 |
VLDB |
0.00012393719 |
| 2,510 |
One Size Fits All? – Part 2: Benchmarking Results |
2007 |
CIDR |
8.6241461e-05 |
| 4,208 |
FERRY - Database-Supported Program Execution |
2009 |
SIGMOD |
6.3511489e-05 |
Semantically Similar Papers