PREDIcT: Towards Predicting the Runtime of Large Scale Iterative Analytics

Summary: PREDIcT predicts runtime for large-scale iterative analytics by using small sample runs plus per-iteration features aligned with dataset characteristics. It tolerates diverse convergence dynamics and high iteration-to-iteration variability (up to 100x), achieving 10-30% relative error on scale-free graphs. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 10656
Venue: VLDB
Year: 2013
Pagerank: 5.3655096e-05
Overall Rank: 5,699 | 60.40%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 7 of 7 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
2,154	SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning	2018	VLDB	9.4176683e-05
2,222	SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning	2019	SIGMOD	9.2598438e-05
6,507	Expand your Training Limits! Generating Training Data for ML-based Data Management	2021	SIGMOD	5.0273414e-05
7,291	Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities	2022	SIGMOD	4.7677422e-05
9,614	Auto-Approximation of Graph Computing	2014	VLDB	4.3136057e-05
11,059	Agile-Ant: Self-managing Distributed Cache Management for Cost Optimization of Big Data Applications	2024	VLDB	4.1905499e-05
11,343	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 15 of 15 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
4	Pregel: A System for Large-Scale Graph Processing	2010	SIGMOD	0.0019040811
39	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB	0.00075263552
79	An Amateur's Introduction to Recursive Query Processing Strategies	1986	SIGMOD	0.00056010993
169	MAD Skills: New Analysis Practices for Big Data	2009	VLDB	0.00038967371
181	LEO - DB2's LEarning Optimizer	2001	VLDB	0.00036970794
408	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00023939456
531	Random Sampling for Histogram Construction: How much is enough?	1998	SIGMOD	0.0002079072
716	Performance Prediction for Concurrent Database Workloads	2011	SIGMOD	0.00017623897
866	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00015771189
1,017	Robust Estimation of Resource Consumption for SQL Queries using Statistical Techniques	2012	VLDB	0.00014627121
1,048	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR	0.00014442178
1,449	Asynchronous Large-Scale Graph Processing Made Easy	2013	CIDR	0.00011928716
1,768	ParaTimer: A Progress Indicator for MapReduce DAGs	2010	SIGMOD	0.00010608933
2,113	When Can We Trust Progress Estimators for SQL Queries?	2005	SIGMOD	9.5207583e-05
2,179	Spinning Fast Iterative Data Flows	2012	VLDB	9.3632007e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
5,266	Incrementalizing Graph Algorithms	2021	SIGMOD	5.5949839e-05
1,876	Large-Scale Distributed Graph Computing Systems: An Experimental Evaluation	2015	VLDB	0.00010242818
2,916	Pregel Algorithms for Graph Connectivity Problems with Performance Guarantees	2014	VLDB	7.9062736e-05
11,655	Query-Driven Learning for Next Generation Predictive Modeling & Analytics	2019	SIGMOD	4.1905499e-05
2,332	Optimizing Graph Algorithms on Pregel-like Systems	2014	VLDB	9.0173968e-05
4,075	Towards Predicting Query Execution Time for Concurrent and Dynamic Database Workloads	2013	VLDB	6.4699689e-05
2,179	Spinning Fast Iterative Data Flows	2012	VLDB	9.3632007e-05
7,778	Runtime Variation in Big Data Analytics	2023	SIGMOD	4.6491879e-05
1,684	Fast Iterative Graph Computation with Block Updates	2013	VLDB	0.00010912102
11,158	Predicting Query Execution time for JIT Compiled Database Engines	2023	CIDR	4.1905499e-05