DBScholar

Back to papers

Resource Elasticity for Large-Scale Machine Learning

Summary: Resource elasticity for declarative large-scale ML using memory-aware optimization and runtime plan migration. Introduces a memory-optimizer for near-optimal memory configurations and dynamic plan migration under YARN-style resource negotiation; up to 21x gains with low overhead and no static provisioning. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h168f857a99170187
Venue
SIGMOD
Year
2015
Pagerank
6.7929814e-05
Overall Rank
4,117 | 72.33%
DOI
10.1145/2723372.2749432

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{huang_sigmod15,
        title = {{Resource Elasticity for Large-Scale Machine Learning}},
        author = {Huang, Botong and Boehm, Matthias and Tian, Yuanyuan and Reinwald, Berthold and Tatikonda, Shirish and Reiss, Frederick R.},
        series = {{SIGMOD} '15},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2723372.2749432},
        url = {https://dl.acm.org/doi/10.1145/2723372.2749432},
        year = {2015}
}

Incoming Citations (Sorted by Pagerank)

Showing 14 of 14 citing papers.

Rank Citing Paper Year Venue Pagerank
416 SystemML: Declarative Machine Learning on Spark 2016 VLDB 0.00018650998
521 Learning Linear Regression Models over Factorized Joins 2016 SIGMOD 0.00016923519
1,223 Data Management in Machine Learning: Challenges, Techniques, and Systems 2017 SIGMOD 0.00011468426
1,614 Compressed Linear Algebra for Large-Scale Machine Learning 2016 VLDB 0.00010067153
1,669 SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle 2020 CIDR 9.9324573e-05
2,978 In-Database Learning with Sparse Tensors 2018 PODS 7.7872011e-05
3,103 On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML 2018 VLDB 7.6456038e-05
3,331 SPOOF: Sum-Product Optimization and Operator Fusion for Large-Scale Machine Learning 2017 CIDR 7.4138851e-05
4,330 MNC: Structure-Exploiting Sparsity Estimation for Matrix Expressions 2019 SIGMOD 6.6564176e-05
6,758 Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities 2022 SIGMOD 5.6877154e-05
7,097 DLRover-RM: Resource Optimization for Deep Recommendation Models Training in the Cloud 2024 VLDB 5.5991152e-05
8,528 FuseME: Distributed Matrix Computation Engine based on Cuboid-based Fused Operator and Plan Generation 2022 SIGMOD 5.3212942e-05
9,663 BladeDISC: Optimizing Dynamic Shape Machine Learning Workloads via Compiler Approach 2023 SIGMOD 5.142891e-05
11,901 CDI-E: An Elastic Cloud Service for Data Engineering 2022 VLDB 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 29 of 29 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
154 MAD Skills: New Analysis Practices for Big Data 2009 VLDB 0.00028568843
187 DB2 Design Advisor: Integrated Automatic Physical Database Design 2004 VLDB 0.00025914764
654 Materialization Optimizations for Feature Selection Workloads 2014 SIGMOD 0.00015096817
739 Zephyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms 2011 SIGMOD 0.00014327549
754 Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs 2011 VLDB 0.00014231311
893 Albatross: Lightweight Elasticity in Shared Storage Databases for the Cloud using Live Data Migration 2011 VLDB 0.00013213905
1,045 RIOT: I/O-Efficient Numerical Computing without SQL 2009 CIDR 0.00012327118
1,082 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB 0.00012118261
1,167 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.0001172597
1,277 Orca: A Modular Query Optimizer Architecture for Big Data 2014 SIGMOD 0.00011234276
1,688 ParaTimer: A Progress Indicator for MapReduce DAGs 2010 SIGMOD 9.860716e-05
1,708 PerfXplain: Debugging MapReduce Job Performance 2012 VLDB 9.8184303e-05
2,025 Efficient Use of the Query Optimizer for Automated Physical Design 2007 VLDB 9.1650298e-05
2,125 Tupleware: "Big" Data, Big Analytics, Small Clusters 2015 CIDR 9.0017828e-05
2,230 Fuxi: a Fault-Tolerant Resource Management and Job Scheduling System at Internet Scale 2014 VLDB 8.7990209e-05
2,249 Cumulon: Optimizing Statistical Data Analysis in the Cloud 2013 SIGMOD 8.7544468e-05
2,724 Exploiting Matrix Dependency for Efficient Distributed Matrix Computation 2015 SIGMOD 8.090674e-05
2,776 Automatic Physical Design Tuning: Workload as a Sequence 2006 SIGMOD 8.0278951e-05
3,041 Configuration-Parametric Query Optimization for Physical Design Tuning 2008 SIGMOD 7.718081e-05
3,061 Accordion: Elastic Scalability for Database Systems Supporting Distributed Transactions 2014 VLDB 7.6928177e-05
4,069 Dynamically Optimizing Queries over Large Scale Data Platforms 2014 SIGMOD 6.8184364e-05
4,894 A Characterization of the Sensitivity of Query Optimization to Storage Access Cost Parameters 2003 SIGMOD 6.3648584e-05
4,935 Machine Learning for Big Data 2013 SIGMOD 6.3447168e-05
5,856 Compression Aware Physical Database Design 2011 VLDB 5.9682143e-05
6,403 Profiling R on a Contemporary Processor 2015 VLDB 5.7950827e-05
6,967 Incremental Elasticity For Array Databases 2014 SIGMOD 5.6301184e-05
7,178 REEF: Retainable Evaluator Execution Framework 2013 VLDB 5.5909045e-05
7,638 MRTuner: A Toolkit to Enable Holistic Optimization for MapReduce Jobs 2014 VLDB 5.4767699e-05
9,994 Breaking the Chains: On Declarative Data Analysis and Data Independence in the Big Data Era 2014 VLDB 5.0982945e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 1,928 Elastic Machine Learning Algorithms in Amazon SageMaker 2020 SIGMOD
2 6,291 Elastic Pipelining in an In-Memory Database Cluster 2016 SIGMOD
3 3,103 On Optimizing Operator Fusion Plans for Large-Scale Machine Learning in SystemML 2018 VLDB
4 6,171 Automatic Optimization of Matrix Implementations for Distributed Machine Learning and Linear Algebra 2021 SIGMOD
5 6,146 Efficient Construction of Approximate Ad-Hoc ML models Through Materialization and Reuse 2018 VLDB
6 5,444 Optimizing Tensor Programs on Flexible Storage 2023 SIGMOD
7 11,137 Managed Resource Scaling in Amazon EMR 2025 SIGMOD
8 13,862 M3: Scaling Up Machine Learning via Memory Mapping 2016 SIGMOD
9 1,614 Compressed Linear Algebra for Large-Scale Machine Learning 2016 VLDB
10 1,082 Hybrid Parallelization Strategies for Large-Scale Machine Learning in SystemML 2014 VLDB