DBScholar

Back to papers

Opening the Black Boxes in Data Flow Optimization

Summary: Optimizes data flows with black-box UDFs by extracting a few operator properties to determine safe reordering, via static analysis of UDF code. The optimizer enables selection reordering, bushy join ordering, and limited aggregation push-down for non-relational flows, achieving relational-like rewrite power without relying on operator semantics. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h1efa77d09424640d
Venue
VLDB
Year
2012
Pagerank
8.8740089e-05
Overall Rank
2,197 | 85.24%
DOI
10.14778/2350229.2350244

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{hueske_vldb12,
        title = {{Opening the Black Boxes in Data Flow Optimization}},
        author = {Hueske, Fabian and Peters, Mathias and Sax, Matthias J. and Rheinländer, Astrid and Bergmann, Rico and Krettek, Aljoscha and Tzoumas, Kostas},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {11},
        pages = {1256--1267},
        doi = {10.14778/2350229.2350244},
        url = {https://doi.org/10.14778/2350229.2350244},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 21 of 21 citing papers.

Rank Citing Paper Year Venue Pagerank
1,082 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012118261
1,803 Tuplex: Data Science in Python at Native Code Speed 2021 SIGMOD 9.602292e-05
2,228 Spinning Fast Iterative Data Flows 2012 VLDB 8.7996087e-05
2,266 An Intermediate Representation for Optimizing Machine Learning Pipelines 2019 VLDB 8.7248802e-05
3,402 RHEEM: Enabling Cross-Platform Data Processing - May The Big Data Be With You! - 2018 VLDB 7.3269775e-05
4,069 Dynamically Optimizing Queries over Large Scale Data Platforms 2014 SIGMOD 6.8184364e-05
4,292 A Method for Optimizing Opaque Filter Queries 2020 SIGMOD 6.6792739e-05
5,349 The NebulaStream Platform: Data and Application Management for the Internet of Things 2020 CIDR 6.1679453e-05
5,836 Opportunistic Physical Design for Big Data Analytics 2014 SIGMOD 5.9732317e-05
6,144 The BUDS Language for Distributed Bayesian Machine Learning 2017 SIGMOD 5.8705515e-05
6,418 Containerized Execution of UDFs: An Experimental Evaluation 2022 VLDB 5.7899052e-05
6,883 DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines 2022 CIDR 5.6552024e-05
7,413 POLAR: Adaptive and Non-invasive Join Order Selection via Plans of Least Resistance 2024 VLDB 5.5316834e-05
8,220 Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques 2023 VLDB 5.3745507e-05
8,600 Not Black-Box Anymore! Enabling Analytics-Aware Optimizations in Teradata Vantage 2021 VLDB 5.3043615e-05
9,467 HyperBlocker: Accelerating Rule-based Blocking in Entity Resolution using GPUs 2025 VLDB 5.1709969e-05
9,698 Versatile Optimization of UDF-heavy Data Flows with Sofa 2014 SIGMOD 5.1377347e-05
9,841 PAXQuery: Parallel Analytical XML Processing 2015 SIGMOD 5.1215512e-05
10,005 cedar: Optimized and Unified Machine Learning Input Data Pipelines 2025 VLDB 5.0954911e-05
12,524 How Achaeans Would Construct Columns in Troy 2013 CIDR 4.9769913e-05
12,534 Iterative Parallel Data Processing with Stratosphere: An Inside Look 2013 SIGMOD 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
1 Access Path Selection in a Relational Database Management System 1979 SIGMOD 0.0023943337
6 Pig Latin: A Not-So-Foreign Language for Data Processing 2008 SIGMOD 0.0010515896
30 SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets 2008 VLDB 0.00050475202
31 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00049821554
73 Including Group-By in Query Optimization 1994 VLDB 0.0003750677
754 Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs 2011 VLDB 0.00014231311
876 Tenzing: A SQL Implementation On The MapReduce Framework 2011 VLDB 0.00013304112
883 Dynamic Programming Strikes Back 2008 SIGMOD 0.00013263866
1,037 Jaql: A Scripting Language for Large Scale Semistructured Data Analysis 2011 VLDB 0.00012371105
1,282 Automatic Optimization for MapReduce Programs 2011 VLDB 0.00011208192
1,285 SQL/MapReduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions 2009 VLDB 0.0001119616
2,478 One Size Fits All? – Part 2: Benchmarking Results 2007 CIDR 8.4063622e-05
3,932 FERRY - Database-Supported Program Execution 2009 SIGMOD 6.9151928e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 6,051 Dependency-Aware Reordering for Parallelizing Query Optimization in Multi-Core CPUs 2009 SIGMOD
2 5,663 Enabling Incremental Query Re-Optimization 2016 SIGMOD
3 1,099 A General Framework for the Optimization of Object-Oriented Queries 1992 SIGMOD
4 143 Optimization of Nonrecursive Queries 1986 VLDB
5 689 Query Optimization in the Presence of Foreign Functions 1993 VLDB
6 4,695 Static Optimization of Conjunctive Queries with Sliding Windows Over Infinite Streams 2004 SIGMOD
7 6,301 Declarative Sub-Operators for Universal Data Processing 2023 VLDB
8 7,347 SQL Query Optimization: Reordering for a General Class of Queries 1996 SIGMOD
9 4,069 Dynamically Optimizing Queries over Large Scale Data Platforms 2014 SIGMOD
10 2,516 Optimization of Queries with User-defined Predicates 1996 VLDB