Database Paper Browser

Back to papers

Resource Elasticity for Large-Scale Machine Learning

Summary: Resource elasticity for declarative large-scale ML using memory-aware optimization and runtime plan migration. Introduces a memory-optimizer for near-optimal memory configurations and dynamic plan migration under YARN-style resource negotiation; up to 21x gains with low overhead and no static provisioning. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
5049
Venue
SIGMOD
Year
2015
Pagerank
5.9114415e-05
Overall Rank
4,802 | 66.60%
DOI
10.1145/2723372.2749432

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 13 of 13 citing papers.

Rank Citing Paper Year Venue Pagerank
557 SystemML: Declarative Machine Learning on Spark 2016 VLDB 0.00020197988
834 Learning Linear Regression Models over Factorized Joins 2016 SIGMOD 0.00016135159
1,532 Data Management in Machine Learning: Challenges, Techniques, and Systems 2017 SIGMOD 0.00011472681
1,967 Compressed Linear Algebra for Large-Scale Machine Learning 2016 VLDB 9.9131712e-05
2,122 SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle 2020 CIDR 9.4989076e-05
3,918 On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML 2018 VLDB 6.6315176e-05
4,505 SPOOF: Sum-Product Optimization and Operator Fusion for Large-Scale Machine Learning 2017 CIDR 6.1327108e-05
4,833 MNC: Structure-Exploiting Sparsity Estimation for Matrix Expressions 2019 SIGMOD 5.8916346e-05
5,993 DLRover-RM: Resource Optimization for Deep Recommendation Models Training in the Cloud 2024 VLDB 5.2415551e-05
7,296 Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities 2022 SIGMOD 4.7723197e-05
8,262 FuseME: Distributed Matrix Computation Engine based on Cuboid-based Fused Operator and Plan Generation 2022 SIGMOD 4.5467867e-05
9,326 BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach 2023 SIGMOD 4.3556432e-05
11,389 CDI-E: An Elastic Cloud Service for Data Engineering 2022 VLDB 4.1945683e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 29 of 29 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
168 MAD Skills: New Analysis Practices for Big Data 2009 VLDB 0.00038946305
258 DB2 Design Advisor: Integrated Automatic Physical Database Design 2004 VLDB 0.0003022091
761 Materialization Optimizations for Feature Selection Workloads 2014 SIGMOD 0.00017053783
868 Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs 2011 VLDB 0.00015789681
892 Zephyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms 2011 SIGMOD 0.00015559876
1,044 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.00014475229
1,076 RIOT: I/O-Efficient Numerical Computing without SQL 2009 CIDR 0.00014248449
1,121 Albatross: Lightweight Elasticity in Shared Storage Databases for the Cloud using Live Data Migration 2011 VLDB 0.00013856011
1,402 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012180605
1,534 PerfXplain: Debugging MapReduce Job Performance 2012 VLDB 0.00011468393
1,684 Fuxi: a Fault-Tolerant Resource Management and Job Scheduling System at Internet Scale 2014 VLDB 0.0001091857
1,770 ParaTimer: A Progress Indicator for MapReduce DAGs 2010 SIGMOD 0.00010618229
2,249 Orca: A Modular Query Optimizer Architecture for Big Data 2014 SIGMOD 9.2034693e-05
2,418 Tupleware: "Big" Data, Big Analytics, Small Clusters 2015 CIDR 8.8556595e-05
2,484 Efficient Use of the Query Optimizer for Automated Physical Design 2007 VLDB 8.6864279e-05
2,667 Cumulon: Optimizing Statistical Data Analysis in the Cloud 2013 SIGMOD 8.3413995e-05
2,828 Automatic Physical Design Tuning: Workload as a Sequence 2006 SIGMOD 8.0548516e-05
2,848 Exploiting Matrix Dependency for Efficient Distributed Matrix Computation 2015 SIGMOD 8.0208832e-05
3,284 Configuration-Parametric Query Optimization for Physical Design Tuning 2008 SIGMOD 7.2790444e-05
3,675 Accordion: Elastic Scalability for Database Systems Supporting Distributed Transactions 2014 VLDB 6.8555664e-05
4,516 A Characterization of the Sensitivity of Query Optimization to Storage Access Cost Parameters 2003 SIGMOD 6.1178799e-05
4,906 Machine Learning for Big Data 2013 SIGMOD 5.8389053e-05
5,014 Dynamically Optimizing Queries over Large Scale Data Platforms 2014 SIGMOD 5.7586174e-05
6,157 Compression Aware Physical Database Design 2011 VLDB 5.1801143e-05
6,542 Profiling R on a Contemporary Processor 2015 VLDB 5.0216639e-05
7,050 REEF: Retainable Evaluator Execution Framework 2013 VLDB 4.85001e-05
7,134 Incremental Elasticity For Array Databases 2014 SIGMOD 4.822331e-05
7,304 MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs 2014 VLDB 4.7684491e-05
9,666 Breaking the Chains: On Declarative Data Analysis and Data Independence in the Big Data Era 2014 VLDB 4.3082114e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
2,163 Elastic Machine Learning Algorithms in Amazon SageMaker 2020 SIGMOD 9.3949234e-05
6,304 Elastic Pipelining in an In-Memory Database Cluster 2016 SIGMOD 5.1210182e-05
3,918 On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML 2018 VLDB 6.6315176e-05
6,191 Automatic Optimization of Matrix Implementations for Distributed Machine Learning and Linear Algebra 2021 SIGMOD 5.1642282e-05
6,330 Efficient Construction of Approximate Ad-Hoc ML models Through Materialization and Reuse 2018 VLDB 5.1077416e-05
6,156 Optimizing Tensor Programs on Flexible Storage 2023 SIGMOD 5.1802603e-05
10,408 Managed Resource Scaling in Amazon EMR 2025 SIGMOD 4.1945683e-05
13,343 M3: Scaling Up Machine Learning via Memory Mapping 2016 SIGMOD -
1,967 Compressed Linear Algebra for Large-Scale Machine Learning 2016 VLDB 9.9131712e-05
1,402 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012180605