Weld: A Common Runtime for High Performance Data Analytics
Summary: Weld presents a common data-parallel IR and runtime to optimize across disjoint libraries (SQL, ML, graph, array), eliminating expensive cross-library data movement. By operator fusion and whole-workflow code generation, Weld plugs into Spark/TensorFlow/NumPy/Pandas and yields up to 30× end-to-end speedups. (summarized by gpt-5-mini on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Shoumik Palkar (Stanford University)
- 2. James J. Thomas (Stanford University)
- 3. Anil Shanbhag (Massachusetts Institute of Technology)
- 4. Deepak Narayanan (Stanford University)
- 5. Holger Pirk (Massachusetts Institute of Technology)
- 6. Malte Schwarzkopf (Massachusetts Institute of Technology)
- 7. Saman Amarasinghe (Massachusetts Institute of Technology)
- 8. Matei Zaharia (Stanford University)
BibTeX Citation
@inproceedings{palkar_cidr17,
address = {Amsterdam, Netherlands},
series = {{CIDR} '17},
title = {{Weld: A Common Runtime for High Performance Data Analytics}},
booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
author = {Palkar, Shoumik and Thomas, James J. and Shanbhag, Anil and Narayanan, Deepak and Pirk, Holger and Schwarzkopf, Malte and Amarasinghe, Saman and Zaharia, Matei},
year = {2017}
}Incoming Citations (Sorted by Pagerank)
Showing 39 of 39 citing papers.
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 12 of 12 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 12,461 | Palette: Enabling Scalable Analytics for Big-Memory, Multicore Machines | 2014 | SIGMOD |
| 2 | 9,922 | Unified Data Analytics: State-of-the-art and Open Problems | 2022 | VLDB |
| 3 | 2,053 | tf.data: A Machine Learning Data Processing Framework | 2021 | VLDB |
| 4 | 2,754 | Implicit Parallelism through Deep Language Embedding | 2015 | SIGMOD |
| 5 | 6,996 | Bridging the Gap Between HPC and Big Data Frameworks | 2017 | VLDB |
| 6 | 8,600 | Flare & Lantern: Efficiently Swapping Horses Midstream | 2019 | VLDB |
| 7 | 2,227 | Spinning Fast Iterative Data Flows | 2012 | VLDB |
| 8 | 11,468 | Welding Natural Language Queries to Analytics IRs with LLMs | 2024 | CIDR |
| 9 | 1,444 | An Architecture for Compiling UDF-centric Workflows | 2015 | VLDB |
| 10 | 2,249 | Evaluating End-to-End Optimization for Data Analytics Applications in Weld | 2018 | VLDB |