Back to papers
Evaluating End-to-End Optimization for Data Analytics Applications in Weld
Summary: Proposes Weld, a common runtime for data analytics libraries that enables cross-library optimizations and pipelining under imperative APIs. An automatic adaptive optimizer uses lightweight measurements to make data-dependent runtime decisions with low overhead, delivering up to 23x single-thread and 80x on eight-thread speedups, plus 3.75x gains over rule-based optimization with incremental porting of 4–5 operators.
(summarized by gpt-5-nano on Feb 09 2026)
- Paper ID
- 11596
- Venue
- VLDB
- Year
- 2018
- Pagerank
- 7.9403097e-05
- Overall Rank
- 2,904 | 79.82%
- DOI
-
10.14778/3213880.3213890
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 25 of 25 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 1,273 |
Amazon Redshift Re-invented |
2022 |
SIGMOD |
0.00012870386 |
| 1,354 |
Northstar: An Interactive Data Science System |
2018 |
VLDB |
0.00012424105 |
| 1,884 |
Tuplex: Data Science in Python at Native Code Speed |
2021 |
SIGMOD |
0.00010206514 |
| 2,355 |
An Intermediate Representation for Optimizing Machine Learning Pipelines |
2019 |
VLDB |
8.9727612e-05 |
| 2,472 |
Photon: A Fast Query Engine for Lakehouse Systems |
2022 |
SIGMOD |
8.7156826e-05 |
| 3,333 |
A Demonstration of Willump: A Statistically-Aware End-to-end Optimizer for Machine Learning Inference |
2020 |
VLDB |
7.2064333e-05 |
| 3,409 |
End-to-end Optimization of Machine Learning Prediction Queries |
2022 |
SIGMOD |
7.1240791e-05 |
| 3,610 |
EVA: A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized Views |
2022 |
SIGMOD |
6.919859e-05 |
| 4,779 |
LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems |
2021 |
SIGMOD |
5.9259373e-05 |
| 4,930 |
User-Defined Operators: Efficiently Integrating Custom Algorithms into Modern Databases |
2022 |
VLDB |
5.8171482e-05 |
| 5,734 |
Evolution of a Compiling Query Engine |
2021 |
VLDB |
5.3471832e-05 |
| 5,741 |
Babelfish: Efficient Execution of Polyglot Queries |
2022 |
VLDB |
5.3450701e-05 |
| 6,863 |
Declarative Sub-Operators for Universal Data Processing |
2023 |
VLDB |
4.9003859e-05 |
| 7,309 |
The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development |
2020 |
SIGMOD |
4.7611148e-05 |
| 8,057 |
Biathlon: Harnessing Model Resilience for Accelerating ML Inference Pipelines |
2024 |
VLDB |
4.5903427e-05 |
| 8,098 |
Modularis: Modular Relational Analytics over Heterogeneous Distributed Platforms |
2021 |
VLDB |
4.5824106e-05 |
| 8,253 |
Automating and Optimizing Data-Centric What-If Analyses on Native Machine Learning Pipelines |
2023 |
SIGMOD |
4.5444167e-05 |
| 8,580 |
Efficient Execution of User-Defined Functions in SQL Queries |
2023 |
VLDB |
4.4876382e-05 |
| 8,593 |
Towards A Polyglot Framework for Factorized ML |
2021 |
VLDB |
4.4846362e-05 |
| 9,331 |
BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach |
2023 |
SIGMOD |
4.351469e-05 |
| 9,765 |
The UDFBench Benchmark for General-purpose UDF Queries |
2025 |
VLDB |
4.2815042e-05 |
| 10,177 |
InferF: Declarative Factorization of AI/ML Inferences over Joins |
2026 |
SIGMOD |
4.1905499e-05 |
| 10,481 |
Approximating Opaque Top-k Queries |
2025 |
SIGMOD |
4.1905499e-05 |
| 10,721 |
Towards Designing Future-Proof Data Processing Systems |
2025 |
VLDB |
4.1905499e-05 |
| 10,972 |
Query Compilation Without Regrets |
2024 |
SIGMOD |
4.1905499e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 13 of 13 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Semantically Similar Papers
| Overall Rank |
Paper |
Year |
Venue |
Pagerank |
| 4,254 |
Optimizing I/O for Big Array Analytics |
2012 |
VLDB |
6.3084831e-05 |
| 8,585 |
A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning |
2024 |
VLDB |
4.4856045e-05 |
| 638 |
Towards a Unified Architecture for in-RDBMS Analytics |
2012 |
SIGMOD |
0.00018810785 |
| 3,920 |
On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML |
2018 |
VLDB |
6.6246708e-05 |
| 2,179 |
Spinning Fast Iterative Data Flows |
2012 |
VLDB |
9.3632007e-05 |
| 1,884 |
Tuplex: Data Science in Python at Native Code Speed |
2021 |
SIGMOD |
0.00010206514 |
| 6,191 |
Accelerating Python UDFs in Vectorized Query Execution |
2022 |
CIDR |
5.1598046e-05 |
| 10,901 |
Welding Natural Language Queries to Analytics IRs with LLMs |
2024 |
CIDR |
4.1905499e-05 |
| 1,875 |
An Architecture for Compiling UDF-centric Workflows |
2015 |
VLDB |
0.00010243959 |
| 1,751 |
Weld: A Common Runtime for High Performance Data Analytics |
2017 |
CIDR |
0.00010673874 |