DBScholar

Back to papers

HaLoop: Efficient Iterative Data Processing on Large Clusters

Summary: HaLoop, a modified Hadoop MapReduce, adds loop-aware scheduling and caching to support iterative data mining workloads. By reusing state across iterations, it achieves 1.85x faster runtimes and shuffles only 4% of data between map and reduce. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h1a14d9d34a478e91
Venue
VLDB
Year
2010
Pagerank
0.00019419661
Overall Rank
385 | 97.43%
DOI
10.14778/1920841.1920881

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{bu_vldb10,
        title = {{HaLoop: Efficient Iterative Data Processing on Large Clusters}},
        author = {Bu, Yingyi and Howe, Bill and Balazinska, Magdalena and Ernst, Michael D.},
        journal = {PVLDB},
        series = {{VLDB} '10},
        volume = {3},
        number = {1},
        pages = {285--296},
        doi = {10.14778/1920841.1920881},
        url = {https://doi.org/10.14778/1920841.1920881},
        year = {2010}
}

Incoming Citations (Sorted by Pagerank)

Showing 39 of 39 citing papers.

Rank Citing Paper Year Venue Pagerank
392 One Trillion Edges: Graph Processing at Facebook-Scale 2015 VLDB 0.0001917435
436 Shark: SQL and Rich Analytics at Scale 2013 SIGMOD 0.00018274105
444 Differential dataflow 2013 CIDR 0.0001815754
759 Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs 2011 VLDB 0.00014194746
1,009 Simulation of Database-Valued Markov Chains Using SimSQL 2013 SIGMOD 0.00012518224
1,074 Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads 2012 VLDB 0.00012131641
1,286 Automatic Optimization for MapReduce Programs 2011 VLDB 0.00011187304
1,360 SkewTune: Mitigating Skew in MapReduce Applications 2012 SIGMOD 0.00010893993
1,449 An Architecture for Compiling UDF-centric Workflows 2015 VLDB 0.00010597074
1,488 Photon: A Fast Query Engine for Lakehouse Systems 2022 SIGMOD 0.00010487094
1,498 Asynchronous Large-Scale Graph Processing Made Easy 2013 CIDR 0.00010445866
1,552 Distributed SociaLite: A Datalog-Based Language for Large-Scale Graph Analysis 2013 VLDB 0.00010252208
2,011 Pregelix: Big(ger) Graph Analytics on A Dataflow Engine 2015 VLDB 9.1800407e-05
2,083 LINVIEW: Incremental View Maintenance for Complex Analytical Queries 2014 SIGMOD 9.0563677e-05
2,139 Tupleware: "Big" Data, Big Analytics, Small Clusters 2015 CIDR 8.9729383e-05
2,240 Spinning Fast Iterative Data Flows 2012 VLDB 8.7716808e-05
2,250 Cumulon: Optimizing Statistical Data Analysis in the Cloud 2013 SIGMOD 8.7477605e-05
2,421 epiC: an Extensible and Scalable System for Processing Big Data 2014 VLDB 8.4793059e-05
2,491 REX: Recursive, Delta-Based Data-Centric Computation 2012 VLDB 8.3733589e-05
2,720 Exploiting Matrix Dependency for Efficient Distributed Matrix Computation 2015 SIGMOD 8.0778267e-05
2,758 Implicit Parallelism through Deep Language Embedding 2015 SIGMOD 8.0283838e-05
3,042 Knowledge Expansion over Probabilistic Knowledge Bases 2014 SIGMOD 7.7063913e-05
3,154 Early Accurate Results for Advanced Analytics on MapReduce 2012 VLDB 7.5708527e-05
3,329 Keys for Graphs 2015 VLDB 7.4086832e-05
3,787 M3R: Increased Performance for In-Memory Hadoop Jobs 2012 VLDB 6.9994582e-05
4,175 Asynchronous and Fault-Tolerant Recursive Datalog Evaluation in Shared-Nothing Engines 2015 VLDB 6.7506691e-05
5,394 PREDIcT: Towards Predicting the Runtime of Large Scale Iterative Analytics 2013 VLDB 6.1351004e-05
5,707 Large-scale Predictive Analytics in Vertica: Fast Data Transfer, Distributed Model Creation, and In-database Prediction 2015 SIGMOD 6.011012e-05
6,092 Exploiting Soft and Hard Correlations in Big Data Query Optimization 2016 VLDB 5.8723624e-05
7,329 Optimization for iterative queries on MapReduce 2014 VLDB 5.5364687e-05
8,125 Hone: “Scaling Down” Hadoop on Shared-Memory Systems 2013 VLDB 5.3739121e-05
8,268 Large-scale Complex Analytics on Semi-structured Datasets using AsterixDB and Spark 2016 VLDB 5.3491386e-05
8,732 sPCA: Scalable Principal Component Analysis for Big Data on Distributed Platforms 2015 SIGMOD 5.2705676e-05
9,582 Redoop Infrastructure for Recurring Big Data Queries 2014 VLDB 5.1411627e-05
9,632 Hybrid Pulling/Pushing for I/O-Efficient Distributed and Iterative Graph Computing 2016 SIGMOD 5.1358859e-05
9,864 Optimistic Recovery for Iterative Dataflows in Action 2015 SIGMOD 5.1063146e-05
12,488 Big Data Research: Will Industry Solve all the Problems? 2015 VLDB 4.9605343e-05
12,514 Anti-Combining for MapReduce 2014 SIGMOD 4.9605343e-05
12,676 SkewTune in Action: Mitigating Skew in MapReduce Applications 2012 VLDB 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 5 of 5 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012089097
6 Pig Latin: A Not-So-Foreign Language for Data Processing 2008 SIGMOD 0.0010494385
43 A Comparison of Approaches to Large-Scale Data Analysis 2009 SIGMOD 0.00045453255
67 An Amateur's Introduction to Recursive Query Processing Strategies 1986 SIGMOD 0.0003786983
120 HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads 2009 VLDB 0.00031017868
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 7,285 Experimental Analysis of Distributed Graph Systems 2018 VLDB
2 1,960 Fast Iterative Graph Computation with Block Updates 2013 VLDB
3 1,074 Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads 2012 VLDB
4 2,372 Online Aggregation and Continuous Query support in MapReduce 2010 SIGMOD
5 2,240 Spinning Fast Iterative Data Flows 2012 VLDB
6 677 Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing) 2010 VLDB
7 9,582 Redoop Infrastructure for Recurring Big Data Queries 2014 VLDB
8 1,470 The Performance of MapReduce: An In-depth Study 2010 VLDB
9 12,496 Shared Execution of Recurring Workloads in MapReduce 2015 VLDB
10 2,299 A Platform for Scalable One-Pass Analytics using MapReduce 2011 SIGMOD