Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance

Summary: Month-long EC2 study spanning CPU, I/O, network, and MapReduce exposes large bimodal performance variance, tied to VM types and availability zones. Shows cloud noise threatens database wall-clock evaluations and SLAs, and offers measurement guidance. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: hf23153350c3306d8
Venue: VLDB
Year: 2010
Pagerank: 0.00013199466
Overall Rank: 893 | 94.02%
DOI: 10.14778/1920841.1920901

Incoming Non-self Citations Over Time

Authors

1. Jörg Schad (Saarland University)
2. Jens Dittrich (Saarland University)
3. Jorge-Arnulfo Quiané-Ruiz (Saarland University)

BibTeX Citation

@article{schad_vldb10,
        title = {{Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance}},
        author = {Schad, Jörg and Dittrich, Jens and Quiané-Ruiz, Jorge-Arnulfo},
        journal = {PVLDB},
        series = {{VLDB} '10},
        volume = {3},
        number = {1},
        pages = {460--471},
        doi = {10.14778/1920841.1920901},
        url = {https://doi.org/10.14778/1920841.1920901},
        year = {2010}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
52	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00041439488
236	OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases	2014	VLDB	0.00023639649
677	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00014835154
1,244	Dhalion: Self-Regulating Stream Processing in Heron	2017	VLDB	0.00011363001
1,294	An Inquiry into Machine Learning-based Automatic Configuration Tuning Services on Real-World Database Management Systems	2021	VLDB	0.00011129451
1,706	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.804657e-05
2,277	Data Consistency Properties and the Trade-offs in Commercial Cloud Storages: the Consumers’ Perspective	2011	CIDR	8.7010392e-05
2,828	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD	7.9476509e-05
2,878	Choosing A Cloud DBMS: Architectures and Tradeoffs	2019	VLDB	7.8955094e-05
4,058	Exploiting Cloud Object Storage for High-Performance Analytics	2023	VLDB	6.8134824e-05
5,297	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.1773472e-05
6,526	PLANET: Making Progress with Commit Processing in Unpredictable Environments	2014	SIGMOD	5.7380126e-05
7,377	PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost!	2021	VLDB	5.5238616e-05
7,764	AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft	2020	VLDB	5.4403231e-05
7,944	Runtime Variation in Big Data Analytics	2023	SIGMOD	5.4051579e-05
7,987	Resource Bricolage for Parallel Database Systems	2015	VLDB	5.3957073e-05
8,003	InTime: Towards Performance Predictability In Byzantine Fault Tolerant Proof-of-Stake Consensus	2025	SIGMOD	5.392702e-05
9,031	ByteCard: Enhancing ByteDance’s Data Warehouse with Learned Cardinality Estimation	2024	SIGMOD	5.219814e-05
9,774	Kora: A Cloud-Native Event Streaming Platform For Kafka	2023	VLDB	5.115551e-05
11,898	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	4.9605343e-05
12,364	Lifting the Haze off the Cloud: A Consumer-Centric Market for Database Computation in the Cloud	2017	VLDB	4.9605343e-05
12,603	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	4.9605343e-05
14,043	RAFT at Work: Speeding-Up MapReduce Applications under Task and Node Failures	2011	SIGMOD	-

Outgoing Citations (Sorted by Pagerank)

Showing 2 of 2 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00045453255
1,331	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD	0.00010997846

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	5,097	Releasing Cloud Databases from the Chains of Performance Prediction Models	2017	CIDR
2	10,079	From Worst-Case to Average-Case Analysis: Accurate Latency Predictions for Key-Value Storage Engines	2020	SIGMOD
3	4,842	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB
4	1,452	Consistency Rationing in the Cloud: Pay only when it matters	2009	VLDB
5	3,303	Cloud Analytics Benchmark	2023	VLDB
6	1,331	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD
7	9,177	Benchmarking Scalability and Elasticity of Distributed Database Systems	2014	VLDB
8	3,401	RTP: Robust Tenant Placement for Elastic In-Memory Database Clusters	2013	SIGMOD
9	1,470	The Performance of MapReduce: An In-depth Study	2010	VLDB
10	7,944	Runtime Variation in Big Data Analytics	2023	SIGMOD