Optimization for iterative queries on MapReduce

Summary: OptIQ optimizes iterative MapReduce queries by partitioning into invariant and variant views and materializing the invariant part. It reuses the invariant and incrementally evaluates the variant view, skipping converged tuples for up to 5x speedups on PageRank and k-means. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: hf67b8007c03f22ae
Venue: VLDB
Year: 2014
Pagerank: 5.5572894e-05
Overall Rank: 7,308 | 50.87%
DOI: 10.14778/2732279.2732286

Incoming Non-self Citations Over Time

Authors

1. Makoto Onizuka (Nippon Telegraph and Telephone Software Innovation Center)
2. Hiroyuki Kato (National Institute of Informatics)
3. Soichiro Hidaka (National Institute of Informatics)
4. Keisuke Nakano (University of Electro-Communications)
5. Zhenjiang Hu (National Institute of Informatics)

BibTeX Citation

@article{onizuka_vldb14,
        title = {{Optimization for iterative queries on MapReduce}},
        author = {Onizuka, Makoto and Kato, Hiroyuki and Hidaka, Soichiro and Nakano, Keisuke and Hu, Zhenjiang},
        journal = {PVLDB},
        series = {{VLDB} '14},
        volume = {7},
        number = {4},
        pages = {241--252},
        doi = {10.14778/2732279.2732286},
        url = {https://doi.org/10.14778/2732279.2732286},
        year = {2014}
}

Incoming Citations (Sorted by Pagerank)

Showing 4 of 4 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,066	Scalable Big Graph Processing in MapReduce	2014	SIGMOD	7.6877117e-05
4,154	Asynchronous and Fault-Tolerant Recursive Datalog Evaluation in Shared-Nothing Engines	2015	VLDB	6.776227e-05
6,088	Iterative Query Processing based on Unified Optimization Techniques	2019	SIGMOD	5.8908778e-05
11,126	Dynamic Pruning for Recursive Joins	2025	SIGMOD	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
3	Pregel: A System for Large-Scale Graph Processing	2010	SIGMOD	0.0012092602
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.001052036
22	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB	0.00055962491
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00045546775
62	Maintaining Views Incrementally	1993	SIGMOD	0.00039045511
120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.000311132
384	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.0001948031
651	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00015130782
978	Incremental Query Evaluation in a Ring of Databases	2010	PODS	0.00012731074
2,480	REX: Recursive, Delta-Based Data-Centric Computation	2012	VLDB	8.4035081e-05
2,603	PLANET: Massively Parallel Learning of Tree Ensembles with MapReduce	2009	VLDB	8.2312028e-05
9,437	MapReduce Algorithms for Big Data Analysis	2012	VLDB	5.1778231e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	2,006	IO-Top-k: Index-access Optimized Top-k Query Processing	2006	VLDB
2	2,282	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
3	2,931	Efficient Multi-way Theta-Join Processing Using MapReduce	2012	VLDB
4	75	Map-Reduce-Merge: Simplified Relational Data Processing on Large Clusters	2007	SIGMOD
5	1,281	Automatic Optimization for MapReduce Programs	2011	VLDB
6	3,214	Efficient and Provable Multi-Query Optimization	2017	PODS
7	9,767	IQ: The Case for Iterative Querying for Knowledge	2011	CIDR
8	8,635	An Iterative Method for Distributed Database Design	1991	VLDB
9	3,731	An Efficient Query Indexing Mechanism for Filtering Geo-Textual Data	2013	SIGMOD
10	3,216	Multi-Query Optimization in MapReduce Framework	2014	VLDB