DBScholar

Back to papers

Opening the Black Boxes in Data Flow Optimization

Summary: Optimizes data flows with black-box UDFs by extracting a few operator properties to determine safe reordering, via static analysis of UDF code. The optimizer enables selection reordering, bushy join ordering, and limited aggregation push-down for non-relational flows, achieving relational-like rewrite power without relying on operator semantics. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h1efa77d09424640d
Venue
VLDB
Year
2012
Pagerank
8.9915327e-05
Overall Rank
2,124 | 85.77%
DOI
10.14778/2350229.2350244

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{hueske_vldb12,
        title = {{Opening the Black Boxes in Data Flow Optimization}},
        author = {Hueske, Fabian and Peters, Mathias and Sax, Matthias J. and Rheinländer, Astrid and Bergmann, Rico and Krettek, Aljoscha and Tzoumas, Kostas},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {11},
        pages = {1256--1267},
        doi = {10.14778/2350229.2350244},
        url = {https://doi.org/10.14778/2350229.2350244},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 22 citing papers.

Rank Citing Paper Year Venue Pagerank
1,083 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012089325
1,812 Tuplex: Data Science in Python at Native Code Speed 2021 SIGMOD 9.5747668e-05
2,240 Spinning Fast Iterative Data Flows 2012 VLDB 8.7716808e-05
2,278 An Intermediate Representation for Optimizing Machine Learning Pipelines 2019 VLDB 8.6993865e-05
3,416 RHEEM: Enabling Cross-Platform Data Processing - May The Big Data Be With You! - 2018 VLDB 7.3046808e-05
4,088 Dynamically Optimizing Queries over Large Scale Data Platforms 2014 SIGMOD 6.7973616e-05
4,302 A Method for Optimizing Opaque Filter Queries 2020 SIGMOD 6.6614501e-05
5,363 The NebulaStream Platform: Data and Application Management for the Internet of Things 2020 CIDR 6.1475521e-05
5,847 Opportunistic Physical Design for Big Data Analytics 2014 SIGMOD 5.9537345e-05
6,159 The BUDS Language for Distributed Bayesian Machine Learning 2017 SIGMOD 5.851731e-05
6,428 Containerized Execution of UDFs: An Experimental Evaluation 2022 VLDB 5.7709339e-05
6,902 DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines 2022 CIDR 5.6365602e-05
7,432 POLAR: Adaptive and Non-invasive Join Order Selection via Plans of Least Resistance 2024 VLDB 5.5133928e-05
8,231 Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques 2023 VLDB 5.3573407e-05
8,620 Not Black-Box Anymore! Enabling Analytics-Aware Optimizations in Teradata Vantage 2021 VLDB 5.2868261e-05
9,475 HyperBlocker: Accelerating Rule-based Blocking in Entity Resolution using GPUs 2025 VLDB 5.157794e-05
9,727 Versatile Optimization of UDF-heavy Data Flows with Sofa 2014 SIGMOD 5.1207461e-05
9,877 PAXQuery: Parallel Analytical XML Processing 2015 SIGMOD 5.1046184e-05
10,040 cedar: Optimized and Unified Machine Learning Input Data Pipelines 2025 VLDB 5.0786422e-05
10,388 CADENZA: Compiling Natural-Language Intent into Task-Specific Operator DAGs for Semantic Query Processing 2027 SIGMOD 4.9605343e-05
12,567 How Achaeans Would Construct Columns in Troy 2013 CIDR 4.9605343e-05
12,577 Iterative Parallel Data Processing with Stratosphere: An Inside Look 2013 SIGMOD 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1 Access Path Selection in a Relational Database Management System 1979 SIGMOD 0.0023948467
6 Pig Latin: A Not-So-Foreign Language for Data Processing 2008 SIGMOD 0.0010494385
30 SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets 2008 VLDB 0.00050364918
31 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00049707539
73 Including Group-By in Query Optimization 1994 VLDB 0.00037420958
759 Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs 2011 VLDB 0.00014194746
876 Tenzing: A SQL Implementation On The MapReduce Framework 2011 VLDB 0.00013272245
881 Dynamic Programming Strikes Back 2008 SIGMOD 0.0001324243
1,040 Jaql: A Scripting Language for Large Scale Semistructured Data Analysis 2011 VLDB 0.00012336829
1,286 Automatic Optimization for MapReduce Programs 2011 VLDB 0.00011187304
1,289 SQL/MapReduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions 2009 VLDB 0.0001116552
2,477 One Size Fits All? – Part 2: Benchmarking Results 2007 CIDR 8.392837e-05
3,936 FERRY - Database-Supported Program Execution 2009 SIGMOD 6.8977705e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 6,063 Dependency-Aware Reordering for Parallelizing Query Optimization in Multi-Core CPUs 2009 SIGMOD
2 5,673 Enabling Incremental Query Re-Optimization 2016 SIGMOD
3 1,102 A General Framework for the Optimization of Object-Oriented Queries 1992 SIGMOD
4 144 Optimization of Nonrecursive Queries 1986 VLDB
5 691 Query Optimization in the Presence of Foreign Functions 1993 VLDB
6 4,708 Static Optimization of Conjunctive Queries with Sliding Windows Over Infinite Streams 2004 SIGMOD
7 6,305 Declarative Sub-Operators for Universal Data Processing 2023 VLDB
8 7,371 SQL Query Optimization: Reordering for a General Class of Queries 1996 SIGMOD
9 4,088 Dynamically Optimizing Queries over Large Scale Data Platforms 2014 SIGMOD
10 2,521 Optimization of Queries with User-defined Predicates 1996 VLDB