Evaluating End-to-End Optimization for Data Analytics Applications in Weld
Summary: Proposes Weld, a common runtime for data analytics libraries that enables cross-library optimizations and pipelining under imperative APIs. An automatic adaptive optimizer uses lightweight measurements to make data-dependent runtime decisions with low overhead, delivering up to 23x single-thread and 80x on eight-thread speedups, plus 3.75x gains over rule-based optimization with incremental porting of 4–5 operators. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Shoumik Palkar (Stanford University)
- 2. James Thomas (Stanford University)
- 3. Deepak Narayanan (Stanford University)
- 4. Pratiksha Thaker (Stanford University)
- 5. Rahul Palamuttam (Stanford University)
- 6. Parimajan Negi (Stanford University)
- 7. Anil Shanbhag (Massachusetts Institute of Technology)
- 8. Malte Schwarzkopf (Massachusetts Institute of Technology)
- 9. Holger Pirk (Imperial College London)
- 10. Saman Amarasinghe (Massachusetts Institute of Technology)
- 11. Samuel Madden (Massachusetts Institute of Technology)
- 12. Matei Zaharia (Stanford University)
BibTeX Citation
@article{palkar_vldb18,
title = {{Evaluating End-to-End Optimization for Data Analytics Applications in Weld}},
author = {Palkar, Shoumik and Thomas, James and Narayanan, Deepak and Thaker, Pratiksha and Palamuttam, Rahul and Negi, Parimajan and Shanbhag, Anil and Schwarzkopf, Malte and Pirk, Holger and Amarasinghe, Saman and Madden, Samuel and Zaharia, Matei},
journal = {PVLDB},
series = {{VLDB} '18},
volume = {11},
number = {9},
pages = {1002--1015},
doi = {10.14778/3213880.3213890},
url = {https://doi.org/10.14778/3213880.3213890},
year = {2018}
}Incoming Citations (Sorted by Pagerank)
Showing 26 of 26 citing papers.
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 13 of 13 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 4,212 | Optimizing I/O for Big Array Analytics | 2012 | VLDB |
| 2 | 503 | Towards a Unified Architecture for in-RDBMS Analytics | 2012 | SIGMOD |
| 3 | 8,004 | A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning | 2024 | VLDB |
| 4 | 3,101 | On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML | 2018 | VLDB |
| 5 | 2,227 | Spinning Fast Iterative Data Flows | 2012 | VLDB |
| 6 | 1,803 | Tuplex: Data Science in Python at Native Code Speed | 2021 | SIGMOD |
| 7 | 5,719 | Accelerating Python UDFs in Vectorized Query Execution | 2022 | CIDR |
| 8 | 11,468 | Welding Natural Language Queries to Analytics IRs with LLMs | 2024 | CIDR |
| 9 | 1,444 | An Architecture for Compiling UDF-centric Workflows | 2015 | VLDB |
| 10 | 1,193 | Weld: A Common Runtime for High Performance Data Analytics | 2017 | CIDR |