Agile-Ant: Self-managing Distributed Cache Management for Cost Optimization of Big Data Applications

Summary: Agile-Ant self-manages distributed in-memory clusters by identifying hot datasets and auto-scaling memory to cache them on-the-fly without app/data/resource priors or human tuning. Aims for cost-optimal caching under multi-tenancy; reduces execution cost 78.3% vs baseline and beats prior approaches. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13718
Venue: VLDB
Year: 2024
Pagerank: 5.093636e-05
Overall Rank: 11,264 | 22.72%
DOI: 10.14778/3681954.3681990

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Hani Al-Sayeh (Technical University of Ilmenau)
2. Muhammad Attahir Jibril (Technical University of Ilmenau)
3. Kai-Uwe Sattler (Technical University of Ilmenau)

BibTeX Citation

@article{alsayeh_vldb24,
        title = {{Agile-Ant: Self-managing Distributed Cache Management for Cost Optimization of Big Data Applications}},
        author = {Al-Sayeh, Hani and Jibril, Muhammad Attahir and Sattler, Kai-Uwe},
        journal = {PVLDB},
        series = {{VLDB} '24},
        volume = {17},
        number = {11},
        pages = {3151--3164},
        doi = {10.14778/3681954.3681990},
        url = {https://doi.org/10.14778/3681954.3681990},
        year = {2024}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
24	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00054865648
66	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00038561587
84	Learned Cardinalities: Estimating Correlated Joins with Deep Learning	2019	CIDR	0.00035838391
694	Cardinality Estimation Done Right: Index-Based Join Sampling	2017	CIDR	0.00014911698
720	Anti-Caching: A New Approach to Database Management System Architecture	2013	VLDB	0.00014632914
923	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR	0.00013189886
1,516	Cardinality Estimation: An Experimental Survey	2018	VLDB	0.00010520885
1,686	Black or White? How to Develop an AutoTuner for Memory-based Analytics	2020	SIGMOD	0.00010008686
4,908	Using Cloud Functions as Accelerator for Elastic Data Analytics	2023	SIGMOD	6.4488784e-05
5,287	PREDIcT: Towards Predicting the Runtime of Large Scale Iterative Analytics	2013	VLDB	6.2815429e-05
11,539	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	5.093636e-05
11,603	SparkCAD: Caching Anomalies Detector for Spark Applications	2022	VLDB	5.093636e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	8,933	Caching at the Web Scale	2017	VLDB
2	2,034	Efficient Distributed Memory Management with RDMA and Caching	2018	VLDB
3	5,348	Crystal: A Unified Cache Storage System for Analytical Databases	2021	VLDB
4	9,640	Supporting Scalable Analytics with Latency Constraints	2015	VLDB
5	3,109	Managing Large Dynamic Graphs Efficiently	2012	SIGMOD
6	381	Query Caching and Optimization in Distributed Mediator Systems	1996	SIGMOD
7	720	Anti-Caching: A New Approach to Database Management System Architecture	2013	VLDB
8	8,512	Linear Elastic Caching via Ski Rental	2025	CIDR
9	4,764	ReCache: Reactive Caching for Fast Analytics over Heterogeneous Data	2018	VLDB
10	11,539	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD