An Architecture for Compiling UDF-centric Workflows

Summary: A compilation architecture for UDF-centric analytics workflows, jointly exploiting data, UDF, and hardware properties to generate specialized code. Implemented in Tupleware, it delivers up to 1000× speedups over existing systems. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h7e3165f3de77278f
Venue: VLDB
Year: 2015
Pagerank: 0.00010597074
Overall Rank: 1,449 | 90.29%
DOI: 10.14778/2824032.2824045
PDF: Download (CC BY-NC-ND 3.0)

Incoming Non-self Citations Over Time

Authors

1. Andrew Crotty (Brown University)
2. Alex Galakatos (Brown University)
3. Kayhan Dursun (Brown University)
4. Tim Kraska (Brown University)
5. Carsten Binnig (Brown University)
6. Ugur Cetintemel (Brown University)
7. Stan Zdonik (Brown University)

BibTeX Citation

@article{crotty_vldb15,
        title = {{An Architecture for Compiling UDF-centric Workflows}},
        author = {Crotty, Andrew and Galakatos, Alex and Dursun, Kayhan and Kraska, Tim and Binnig, Carsten and Cetintemel, Ugur and Zdonik, Stan},
        journal = {PVLDB},
        series = {{VLDB} '15},
        volume = {8},
        number = {12},
        pages = {1466--1477},
        doi = {10.14778/2824032.2824045},
        url = {https://doi.org/10.14778/2824032.2824045},
        year = {2015}
}

Incoming Citations (Sorted by Pagerank)

Showing 33 of 33 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
40	The Case for Learned Index Structures	2018	SIGMOD	0.00046249672
415	SystemML: Declarative Machine Learning on Spark	2016	VLDB	0.00018619173
606	Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask	2018	VLDB	0.00015609823
928	The End of Slow Networks: It's Time for a Redesign	2016	VLDB	0.00013012601
1,197	Weld: A Common Runtime for High Performance Data Analytics	2017	CIDR	0.00011548039
1,226	Data Management in Machine Learning: Challenges, Techniques, and Systems	2017	SIGMOD	0.00011432354
1,350	Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last	2018	VLDB	0.00010934877
1,567	HELIX: Holistic Optimization for Accelerating Iterative Machine Learning	2019	VLDB	0.00010179013
1,804	IDEBench: A Benchmark for Interactive Data Exploration	2020	SIGMOD	9.5971444e-05
1,812	Tuplex: Data Science in Python at Native Code Speed	2021	SIGMOD	9.5747668e-05
2,258	Evaluating End-to-End Optimization for Data Analytics Applications in Weld	2018	VLDB	8.7387151e-05
2,433	How to Architect a Query Compiler, Revisited	2018	SIGMOD	8.4614165e-05
3,104	On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML	2018	VLDB	7.6281929e-05
3,331	SPOOF: Sum-Product Optimization and Operator Fusion for Large-Scale Machine Learning	2017	CIDR	7.4065701e-05
3,654	User-Defined Operators: Efficiently Integrating Custom Algorithms into Modern Databases	2022	VLDB	7.1147757e-05
3,658	Davos: A System for Interactive Data-Driven Decision Making	2021	VLDB	7.1100991e-05
3,746	In-RDBMS Hardware Acceleration of Advanced Analytics	2018	VLDB	7.0369561e-05
4,354	LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems	2021	SIGMOD	6.6318545e-05
5,246	Babelfish: Efficient Execution of Polyglot Queries	2022	VLDB	6.1985833e-05
5,787	Database Technology for the Masses: Sub-Operators as First-Class Entities	2021	VLDB	5.9768075e-05
5,944	Hillview: A trillion-cell spreadsheet for big data	2019	VLDB	5.9213827e-05
6,428	Containerized Execution of UDFs: An Experimental Evaluation	2022	VLDB	5.7709339e-05
7,470	Modularis: Modular Relational Analytics over Heterogeneous Distributed Platforms	2021	VLDB	5.5032592e-05
7,887	Building Advanced SQL Analytics From Low-Level Plan Operators	2021	SIGMOD	5.4162508e-05
8,002	Efficient Execution of User-Defined Functions in SQL Queries	2023	VLDB	5.3927693e-05
8,191	Pangea: Monolithic Distributed Storage for Data Analytics	2019	VLDB	5.3648409e-05
8,387	Meta-Dataflows: Efficient Exploratory Dataflow Jobs	2018	SIGMOD	5.325795e-05
8,701	Adaptive Code Generation for Data-Intensive Analytics	2021	VLDB	5.2714222e-05
9,512	Query Compilation Without Regrets	2024	SIGMOD	5.1515819e-05
9,672	QURE: AI-Assisted and Automatically Verified UDF Inlining	2025	SIGMOD	5.1267792e-05
9,691	BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach	2023	SIGMOD	5.1258854e-05
10,061	Tuplex: Robust, Efficient Analytics When Python Rules	2019	VLDB	5.0753869e-05
11,847	To UDFs and Beyond: Demonstration of a Fully Decomposed Data Processor for General Data Wrangling Tasks	2023	VLDB	4.9605343e-05

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
21	Efficiently Compiling Efficient Query Plans for Modern Hardware	2011	VLDB	0.00056737487
30	SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets	2008	VLDB	0.00050364918
385	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00019419661
498	Building Efficient Query Engines in a High-Level Language	2014	VLDB	0.00017323427
766	Conjunctive Selection Conditions in Main Memory	2002	PODS	0.00014105436
876	Tenzing: A SQL Implementation On The MapReduce Framework	2011	VLDB	0.00013272245
1,074	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00012131641
1,169	DimmWitted: A Study of Main-Memory Statistical Analytics	2014	VLDB	0.00011690039
3,111	Micro Adaptivity in Vectorwise	2013	SIGMOD	7.6224912e-05
4,903	ASTERIX: An Open Source System for "Big Data" Management and Analysis (Demo)	2012	VLDB	6.3480568e-05
5,316	Performance Analysis of a Relational Data Base Management System	1979	SIGMOD	6.168031e-05
6,808	Hadoop's Adolescence: An analysis of Hadoop usage in scientific workloads	2013	VLDB	5.6608074e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,756	How to Architect a Query Compiler	2016	SIGMOD
2	11,847	To UDFs and Beyond: Demonstration of a Fully Decomposed Data Processor for General Data Wrangling Tasks	2023	VLDB
3	1,812	Tuplex: Data Science in Python at Native Code Speed	2021	SIGMOD
4	6,305	Declarative Sub-Operators for Universal Data Processing	2023	VLDB
5	11,236	UDFBench: A Tool for Benchmarking UDF Queries on SQL Engines	2025	SIGMOD
6	6,413	Functional-Style SQL UDFs With a Capital 'F'	2020	SIGMOD
7	9,729	The UDFBench Benchmark for General-purpose UDF Queries	2025	VLDB
8	5,723	Accelerating Python UDFs in Vectorized Query Execution	2022	CIDR
9	2,139	Tupleware: "Big" Data, Big Analytics, Small Clusters	2015	CIDR
10	8,002	Efficient Execution of User-Defined Functions in SQL Queries	2023	VLDB