Opening the Black Boxes in Data Flow Optimization

Summary: Optimizes data flows with black-box UDFs by extracting a few operator properties to determine safe reordering, via static analysis of UDF code. The optimizer enables selection reordering, bushy join ordering, and limited aggregation push-down for non-relational flows, achieving relational-like rewrite power without relying on operator semantics. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 10551
Venue: VLDB
Year: 2012
Pagerank: 9.0521951e-05
Overall Rank: 2,164 | 85.16%
DOI: 10.14778/2350229.2350244

Incoming Non-self Citations Over Time

Authors

1. Fabian Hueske (Technical University of Berlin)
2. Mathias Peters (Humboldt University of Berlin)
3. Matthias J. Sax (Humboldt University of Berlin)
4. Astrid Rheinländer (Humboldt University of Berlin)
5. Rico Bergmann (Humboldt University of Berlin)
6. Aljoscha Krettek (Technical University of Berlin)
7. Kostas Tzoumas (Technical University of Berlin)

BibTeX Citation

@article{hueske_vldb12,
        title = {{Opening the Black Boxes in Data Flow Optimization}},
        author = {Hueske, Fabian and Peters, Mathias and Sax, Matthias J. and Rheinländer, Astrid and Bergmann, Rico and Krettek, Aljoscha and Tzoumas, Kostas},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {11},
        pages = {1256--1267},
        doi = {10.14778/2350229.2350244},
        url = {https://doi.org/10.14778/2350229.2350244},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 21 of 21 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,079	Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML	2014	VLDB	0.00012258469
1,768	Tuplex: Data Science in Python at Native Code Speed	2021	SIGMOD	9.8041636e-05
2,196	Spinning Fast Iterative Data Flows	2012	VLDB	8.9704984e-05
2,239	An Intermediate Representation for Optimizing Machine Learning Pipelines	2019	VLDB	8.8875753e-05
3,351	RHEEM: Enabling Cross-Platform Data Processing - May The Big Data Be With You! -	2018	VLDB	7.4937347e-05
4,264	A Method for Optimizing Opaque Filter Queries	2020	SIGMOD	6.7937529e-05
4,481	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD	6.6754521e-05
5,215	The NebulaStream Platform: Data and Application Management for the Internet of Things	2020	CIDR	6.3124972e-05
5,720	Opportunistic Physical Design for Big Data Analytics	2014	SIGMOD	6.1092082e-05
6,030	The BUDS Language for Distributed Bayesian Machine Learning	2017	SIGMOD	6.0009079e-05
6,553	Containerized Execution of UDFs: An Experimental Evaluation	2022	VLDB	5.8417517e-05
7,160	DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines	2022	CIDR	5.6855887e-05
8,115	Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques	2023	VLDB	5.4842672e-05
8,450	Not Black-Box Anymore! Enabling Analytics-Aware Optimizations in Teradata Vantage	2021	VLDB	5.4231788e-05
8,494	POLAR: Adaptive and Non-invasive Join Order Selection via Plans of Least Resistance	2024	VLDB	5.4142129e-05
9,509	Versatile Optimization of UDF-heavy Data Flows with Sofa	2014	SIGMOD	5.2581466e-05
9,661	PAXQuery: Parallel Analytical XML Processing	2015	SIGMOD	5.2409648e-05
9,997	HyperBlocker: Accelerating Rule-based Blocking in Entity Resolution using GPUs	2025	VLDB	5.1814573e-05
10,999	cedar: Optimized and Unified Machine Learning Input Data Pipelines	2025	VLDB	5.093636e-05
12,227	How Achaeans Would Construct Columns in Troy	2013	CIDR	5.093636e-05
12,237	Iterative Parallel Data Processing with Stratosphere: An Inside Look	2013	SIGMOD	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
1	Access Path Selection in a Relational Database Management System	1979	SIGMOD	0.0024089429
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0010686205
30	SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets	2008	VLDB	0.00051174276
32	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00050111008
71	Including Group-By in Query Optimization	1994	VLDB	0.00038021159
735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00014522606
872	Tenzing: A SQL Implementation On The MapReduce Framework	2011	VLDB	0.00013486409
1,013	Dynamic Programming Strikes Back	2008	SIGMOD	0.00012652549
1,021	Jaql: A Scripting Language for Large Scale Semistructured Data Analysis	2011	VLDB	0.00012606673
1,257	Automatic Optimization for MapReduce Programs	2011	VLDB	0.000114432
1,264	SQL/MapReduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions	2009	VLDB	0.00011416393
2,451	One Size Fits All? – Part 2: Benchmarking Results	2007	CIDR	8.560157e-05
3,864	FERRY - Database-Supported Program Execution	2009	SIGMOD	7.0628657e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,018	Dependency-Aware Reordering for Parallelizing Query Optimization in Multi-Core CPUs	2009	SIGMOD
2	5,552	Enabling Incremental Query Re-Optimization	2016	SIGMOD
3	1,073	A General Framework for the Optimization of Object-Oriented Queries	1992	SIGMOD
4	143	Optimization of Nonrecursive Queries	1986	VLDB
5	677	Query Optimization in the Presence of Foreign Functions	1993	VLDB
6	4,596	Static Optimization of Conjunctive Queries with Sliding Windows Over Infinite Streams	2004	SIGMOD
7	6,253	Declarative Sub-Operators for Universal Data Processing	2023	VLDB
8	7,272	SQL Query Optimization: Reordering for a General Class of Queries	1996	SIGMOD
9	4,481	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD
10	2,465	Optimization of Queries with User-defined Predicates	1996	VLDB