Database Paper Browser

Back to papers

Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing

Summary: Fine-grained instance-level modeling with a MaxCompute-based architecture decomposes resource optimization into simpler, multi-objective decisions (partition count, placement, per-instance resources). Novel predictive models and optimization methods enable sub-second RO and yield 37–72% latency and 43–78% cost reductions on production workloads. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
12790
Venue
VLDB
Year
2022
Pagerank
5.5457532e-05
Overall Rank
5,368 | 62.66%
DOI
10.14778/3551793.3551855

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 8 of 8 citing papers.

Rank Citing Paper Year Venue Pagerank
5,701 Resource Management in Aurora Serverless 2024 VLDB 5.3647775e-05
5,832 Stage: Query Execution Time Prediction in Amazon Redshift 2024 SIGMOD 5.3111109e-05
7,889 Cost-Intelligent Data Analytics in the Cloud 2024 CIDR 4.6253386e-05
8,020 The Holon Approach for Simultaneously Tuning Multiple Components in a Self-Driving Database Management System with Machine Learning via Synthesized Proto-Actions 2024 VLDB 4.6040862e-05
8,617 A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning 2024 VLDB 4.4846425e-05
8,956 T3: Accurate and Fast Performance Prediction for Relational Database Systems With Compiled Decision Trees 2025 SIGMOD 4.4214154e-05
10,726 Improving DBMS Scheduling Decisions with Accurate Performance Prediction on Concurrent Queries 2025 VLDB 4.1945683e-05
10,859 Graph Transformers for Query Plan Representation: Potentials and Challenges 2025 VLDB 4.1945683e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 41 of 41 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
70 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00059533166
183 Automatic Database Management System Tuning Through Large-scale Machine Learning 2017 SIGMOD 0.00036721403
333 Neo: A Learned Query Optimizer 2019 VLDB 0.00027206884
514 An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning 2019 SIGMOD 0.0002124895
542 Shark: SQL and Rich Analytics at Scale 2013 SIGMOD 0.00020595648
758 Deep Unsupervised Cardinality Estimation 2020 VLDB 0.0001706608
780 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00016775082
806 An End-to-End Learning-based Cost Estimator 2020 VLDB 0.00016434274
884 Plan-Structured Deep Neural Network Models for Query Performance Prediction 2019 VLDB 0.00015654004
910 NeuroCard: One Cardinality Estimator for All Tables 2021 VLDB 0.00015423056
1,647 Parametric Query Optimization for Linear and Piecewise Linear Cost Functions 2002 VLDB 0.00011033757
1,684 Fuxi: a Fault-Tolerant Resource Management and Job Scheduling System at Internet Scale 2014 VLDB 0.0001091857
2,083 Towards a Learning Optimizer for Shared Clouds 2019 VLDB 9.5834572e-05
2,364 Deep Learning Models for Selectivity Estimation of Multi-Attribute Queries 2020 SIGMOD 8.9554751e-05
2,568 Towards Cost-Optimal Query Processing in the Cloud 2021 VLDB 8.5239227e-05
2,659 Multi-Objective Parametric Query Optimization 2015 VLDB 8.3604734e-05
2,762 FLAT: Fast, Lightweight and Accurate Method for Cardinality Estimation 2021 VLDB 8.1585394e-05
2,783 Flow-Loss: Learning Cardinality Estimates That Matter 2021 VLDB 8.1293383e-05
3,216 WiSeDB: A Learning-based Workload Management Advisor for Cloud Databases 2016 VLDB 7.3601267e-05
3,499 Fauce: Fast and Accurate Deep Ensembles with Uncertainty for Cardinality Estimation 2021 VLDB 7.0376445e-05
3,522 ResTune: Resource Oriented Tuning Boosted by Meta-Learning for Cloud Databases 2021 SIGMOD 7.0096727e-05
3,580 Query Performance Prediction for Concurrent Queries using Graph Embedding 2020 VLDB 6.9500996e-05
3,625 Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings 2020 SIGMOD 6.9055212e-05
3,924 A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation 2021 SIGMOD 6.6271553e-05
4,543 FACE: A Normalizing Flow based Cardinality Estimator 2022 VLDB 6.1011198e-05
4,590 MB2: Decomposed Behavior Modeling for Self-Driving Database Management Systems 2021 SIGMOD 6.0620053e-05
4,700 Schedule Optimization for Data Processing Flows on the Cloud 2011 SIGMOD 5.9882572e-05
4,874 Approximation Schemes for Many-Objective Query Optimization 2014 SIGMOD 5.8594632e-05
5,075 An Incremental Anytime Algorithm for Multi-Objective Query Optimization 2015 SIGMOD 5.7172118e-05
5,469 Learned Cardinality Estimation for Similarity Queries 2021 SIGMOD 5.4898192e-05
6,368 Pre-training Summarization Models of Structured Datasets for Cardinality Estimation 2022 VLDB 5.0937722e-05
6,667 Leveraging Query Logs and Machine Learning for Parametric Query Optimization 2022 VLDB 4.9688874e-05
7,358 Weighted Distinct Sampling: Cardinality Estimation for SPJ Queries 2021 SIGMOD 4.7529363e-05
7,372 Model-Free Control for Distributed Stream Data Processing using Deep Reinforcement Learning 2018 VLDB 4.7496881e-05
7,913 Resource Bricolage for Parallel Database Systems 2015 VLDB 4.6180739e-05
8,576 PostCENN: PostgreSQL with Machine Learning Models for Cardinality Estimation 2021 VLDB 4.4927989e-05
9,066 Tempo: Robust and Self-Tuning Resource Management in Multi-tenant Parallel Databases 2016 VLDB 4.4035481e-05
9,194 Phoebe: A Learning-based Checkpoint Optimizer 2021 VLDB 4.3761777e-05
9,546 Trident: Task Scheduling over Tiered Storage Systems in Big Data Platforms 2021 VLDB 4.3259935e-05
9,547 Optimistic Recovery for Iterative Dataflows in Action 2015 SIGMOD 4.3259935e-05
9,736 UDAO: A Next-Generation Unified Data Analytics Optimizer 2019 VLDB 4.2942813e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

Overall Rank Paper Year Venue Pagerank
2,459 Multi-dimensional Resource Scheduling for Parallel Queries 1996 SIGMOD 8.7676516e-05
4,061 Advanced Partitioning Techniques for Massively Distributed Computation 2012 SIGMOD 6.483587e-05
4,802 Resource Elasticity for Large-Scale Machine Learning 2015 SIGMOD 5.9114415e-05
3,703 Multi-Query Optimization in MapReduce Framework 2014 VLDB 6.8289978e-05
4,700 Schedule Optimization for Data Processing Flows on the Cloud 2011 SIGMOD 5.9882572e-05
5,297 Continuous Cloud-Scale Query Optimization and Processing 2013 VLDB 5.5801669e-05
3,625 Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings 2020 SIGMOD 6.9055212e-05
8,617 A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning 2024 VLDB 4.4846425e-05
11,415 Budget-Conscious Fine-Grained Configuration Optimization for Spatio-Temporal Applications 2022 VLDB 4.1945683e-05
9,504 Supporting Scalable Analytics with Latency Constraints 2015 VLDB 4.3341665e-05