Database Paper Browser

Back to papers

Weld: A Common Runtime for High Performance Data Analytics

Summary: Weld presents a common data-parallel IR and runtime to optimize across disjoint libraries (SQL, ML, graph, array), eliminating expensive cross-library data movement. By operator fusion and whole-workflow code generation, Weld plugs into Spark/TensorFlow/NumPy/Pandas and yields up to 30× end-to-end speedups. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
291
Venue
CIDR
Year
2017
Pagerank
0.00010683647
Overall Rank
1,750 | 87.83%
DOI
-

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 35 of 35 citing papers.

Rank Citing Paper Year Venue Pagerank
853 Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask 2018 VLDB 0.00015940507
1,666 HELIX: Holistic Optimization for Accelerating Iterative Machine Learning 2019 VLDB 0.0001096361
2,122 SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle 2020 CIDR 9.4989076e-05
2,350 An Intermediate Representation for Optimizing Machine Learning Pipelines 2019 VLDB 8.9788641e-05
2,473 Photon: A Fast Query Engine for Lakehouse Systems 2022 SIGMOD 8.7237281e-05
2,896 Evaluating End-to-End Optimization for Data Analytics Applications in Weld 2018 VLDB 7.9452051e-05
3,265 RHEEM: Enabling Cross-Platform Data Processing - May The Big Data Be With You! - 2018 VLDB 7.3083672e-05
3,293 Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics 2021 VLDB 7.2629834e-05
3,918 On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML 2018 VLDB 6.6315176e-05
4,033 In-RDBMS Hardware Acceleration of Advanced Analytics 2018 VLDB 6.5113267e-05
4,677 Automatically Leveraging MapReduce Frameworks for Data-Intensive Applications 2018 SIGMOD 6.0047822e-05
4,948 Designing an Open Framework for Query Optimization and Compilation 2022 VLDB 5.8116879e-05
4,957 Doing More with Less: Characterizing Dataset Downsampling for AutoML 2021 VLDB 5.8035715e-05
5,427 The NebulaStream Platform: Data and Application Management for the Internet of Things 2020 CIDR 5.509468e-05
5,731 Babelfish: Efficient Execution of Polyglot Queries 2022 VLDB 5.3502065e-05
6,221 Charting the Design Space of Query Execution using VOILA 2021 VLDB 5.1512158e-05
6,245 Iterative Query Processing based on Unified Optimization Techniques 2019 SIGMOD 5.1391202e-05
6,291 Lightweight Inspection of Data Preprocessing in Native Machine Learning Pipelines 2021 CIDR 5.1269764e-05
6,378 Mitigating the Impedance Mismatch between Prediction Query Execution and Database Engine 2025 SIGMOD 5.0909804e-05
6,525 Database Technology for the Masses: Sub-Operators as First-Class Entities 2021 VLDB 5.027205e-05
6,701 YeSQL: “You extend SQL” with Rich and Highly Performant User-Defined Functions in Relational Databases 2022 VLDB 4.9561066e-05
6,863 Declarative Sub-Operators for Universal Data Processing 2023 VLDB 4.905092e-05
7,476 Lachesis: Automatic Partitioning for UDF-Centric Analytics 2021 VLDB 4.7188928e-05
7,925 Architecting a Query Compiler for Spatial Workloads 2020 SIGMOD 4.6153403e-05
8,051 Building Advanced SQL Analytics From Low-Level Plan Operators 2021 SIGMOD 4.5969549e-05
8,248 Flare & Lantern: Efficiently Swapping Horses Midstream 2019 VLDB 4.5509332e-05
8,583 Efficient Execution of User-Defined Functions in SQL Queries 2023 VLDB 4.4919445e-05
8,595 Towards A Polyglot Framework for Factorized ML 2021 VLDB 4.4889397e-05
9,332 PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development 2018 SIGMOD 4.3556432e-05
9,608 Unified Data Analytics: State-of-the-art and Open Problems 2022 VLDB 4.3177432e-05
9,823 Thriving in the No Man’s Land between Compilers and Databases 2019 CIDR 4.2754485e-05
9,884 SQL Engines Excel at the Execution of Imperative Programs 2024 VLDB 4.2635782e-05
10,332 Cephalopod - Virtual Data Model Composition through Partial Query Translation 2025 CIDR 4.1945683e-05
10,897 Welding Natural Language Queries to Analytics IRs with LLMs 2024 CIDR 4.1945683e-05
10,969 Query Compilation Without Regrets 2024 SIGMOD 4.1945683e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
37 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.0007522744
60 Efficiently Compiling Efficient Query Plans for Modern Hardware 2011 VLDB 0.00064439773
66 Spark SQL: Relational Data Processing in Spark 2015 SIGMOD 0.00061639801
140 The MADlib Analytics Library or MAD Skills, the SQL 2012 VLDB 0.00042270404
704 Building Efficient Query Engines in a High-Level Language 2014 VLDB 0.00017900583
851 The case against specialized graph analytics engines 2015 CIDR 0.0001594441
1,239 A Demonstration of SciDB: A Science-Oriented DBMS 2009 VLDB 0.00013102195
1,343 NoDB: Efficient Query Execution on Raw Data Files 2012 SIGMOD 0.00012482538
1,873 An Architecture for Compiling UDF-centric Workflows 2015 VLDB 0.00010253002
2,014 Voodoo - A Vector Algebra for Portable Database Performance on Modern Hardware 2016 VLDB 9.7904029e-05
2,449 GraphMat: High performance graph analytics made productive 2015 VLDB 8.7915899e-05
2,709 Vertexica: Your Relational Friend for Graph Analytics! 2014 VLDB 8.2530203e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
11,972 Palette: Enabling Scalable Analytics for Big-Memory, Multicore Machines 2014 SIGMOD 4.1945683e-05
9,608 Unified Data Analytics: State-of-the-art and Open Problems 2022 VLDB 4.3177432e-05
2,170 tf.data: A Machine Learning Data Processing Framework 2021 VLDB 9.3821603e-05
2,818 Implicit Parallelism through Deep Language Embedding 2015 SIGMOD 8.0665558e-05
7,019 Bridging the Gap Between HPC and Big Data Frameworks 2017 VLDB 4.860057e-05
8,248 Flare & Lantern: Efficiently Swapping Horses Midstream 2019 VLDB 4.5509332e-05
2,172 Spinning Fast Iterative Data Flows 2012 VLDB 9.3706587e-05
10,897 Welding Natural Language Queries to Analytics IRs with LLMs 2024 CIDR 4.1945683e-05
1,873 An Architecture for Compiling UDF-centric Workflows 2015 VLDB 0.00010253002
2,896 Evaluating End-to-End Optimization for Data Analytics Applications in Weld 2018 VLDB 7.9452051e-05