PREDIcT: Towards Predicting the Runtime of Large Scale Iterative Analytics

Summary: PREDIcT predicts runtime for large-scale iterative analytics by using small sample runs plus per-iteration features aligned with dataset characteristics. It tolerates diverse convergence dynamics and high iteration-to-iteration variability (up to 100x), achieving 10-30% relative error on scale-free graphs. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 10655
Venue: VLDB
Year: 2013
Pagerank: 5.3702808e-05
Overall Rank: 5,688 | 60.44%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 7 of 7 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
2,156	SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning	2018	VLDB	9.4170209e-05
2,219	SkinnerDB: Regret-Bounded Query Evaluation via Reinforcement Learning	2019	SIGMOD	9.2623533e-05
6,519	Expand your Training Limits! Generating Training Data for ML-based Data Management	2021	SIGMOD	5.0316686e-05
7,296	Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities	2022	SIGMOD	4.7723197e-05
9,614	Auto-Approximation of Graph Computing	2014	VLDB	4.3177432e-05
11,056	Agile-Ant: Self-managing Distributed Cache Management for Cost Optimization of Big Data Applications	2024	VLDB	4.1945683e-05
11,341	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 15 of 15 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
4	Pregel: A System for Large-Scale Graph Processing	2010	SIGMOD	0.0019005923
37	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB	0.0007522744
77	An Amateur's Introduction to Recursive Query Processing Strategies	1986	SIGMOD	0.00057043861
168	MAD Skills: New Analysis Practices for Big Data	2009	VLDB	0.00038946305
182	LEO - DB2's LEarning Optimizer	2001	VLDB	0.00036962631
413	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00023904409
530	Random Sampling for Histogram Construction: How much is enough?	1998	SIGMOD	0.00020803682
718	Performance Prediction for Concurrent Database Workloads	2011	SIGMOD	0.0001763106
868	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB	0.00015789681
1,019	Robust Estimation of Resource Consumption for SQL Queries using Statistical Techniques	2012	VLDB	0.00014625603
1,071	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR	0.00014312777
1,452	Asynchronous Large-Scale Graph Processing Made Easy	2013	CIDR	0.00011919499
1,770	ParaTimer: A Progress Indicator for MapReduce DAGs	2010	SIGMOD	0.00010618229
2,111	When Can We Trust Progress Estimators for SQL Queries?	2005	SIGMOD	9.5286436e-05
2,172	Spinning Fast Iterative Data Flows	2012	VLDB	9.3706587e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
5,292	Incrementalizing Graph Algorithms	2021	SIGMOD	5.5816687e-05
1,877	Large-Scale Distributed Graph Computing Systems: An Experimental Evaluation	2015	VLDB	0.00010236803
2,927	Pregel Algorithms for Graph Connectivity Problems with Performance Guarantees	2014	VLDB	7.8823626e-05
11,650	Query-Driven Learning for Next Generation Predictive Modeling & Analytics	2019	SIGMOD	4.1945683e-05
2,336	Optimizing Graph Algorithms on Pregel-like Systems	2014	VLDB	9.0109891e-05
4,088	Towards Predicting Query Execution Time for Concurrent and Dynamic Database Workloads	2013	VLDB	6.4603918e-05
2,172	Spinning Fast Iterative Data Flows	2012	VLDB	9.3706587e-05
7,778	Runtime Variation in Big Data Analytics	2023	SIGMOD	4.653651e-05
1,685	Fast Iterative Graph Computation with Block Updates	2013	VLDB	0.0001091808
11,155	Predicting Query Execution time for JIT Compiled Database Engines	2023	CIDR	4.1945683e-05