Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance

Summary: Month-long EC2 study spanning CPU, I/O, network, and MapReduce exposes large bimodal performance variance, tied to VM types and availability zones. Shows cloud noise threatens database wall-clock evaluations and SLAs, and offers measurement guidance. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: hf23153350c3306d8
Venue: VLDB
Year: 2010
Pagerank: 0.00013227162
Overall Rank: 892 | 94.01%
DOI: 10.14778/1920841.1920901

Incoming Non-self Citations Over Time

Authors

1. Jörg Schad (Saarland University)
2. Jens Dittrich (Saarland University)
3. Jorge-Arnulfo Quiané-Ruiz (Saarland University)

BibTeX Citation

@article{schad_vldb10,
        title = {{Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance}},
        author = {Schad, Jörg and Dittrich, Jens and Quiané-Ruiz, Jorge-Arnulfo},
        journal = {PVLDB},
        series = {{VLDB} '10},
        volume = {3},
        number = {1},
        pages = {460--471},
        doi = {10.14778/1920841.1920901},
        url = {https://doi.org/10.14778/1920841.1920901},
        year = {2010}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
52	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00041219077
235	OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases	2014	VLDB	0.00023697028
673	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.0001488755
1,233	Dhalion: Self-Regulating Stream Processing in Heron	2017	VLDB	0.00011405873
1,289	An Inquiry into Machine Learning-based Automatic Configuration Tuning Services on Real-World Database Management Systems	2021	VLDB	0.00011162479
1,708	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.8224587e-05
2,261	Data Consistency Properties and the Trade-offs in Commercial Cloud Storages: the Consumers’ Perspective	2011	CIDR	8.7336351e-05
2,834	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD	7.9560627e-05
2,966	Choosing A Cloud DBMS: Architectures and Tradeoffs	2019	VLDB	7.8039817e-05
4,034	Exploiting Cloud Object Storage for High-Performance Analytics	2023	VLDB	6.8393244e-05
5,274	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.2005706e-05
6,512	PLANET: Making Progress with Commit Processing in Unpredictable Environments	2014	SIGMOD	5.7597381e-05
7,363	PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost!	2021	VLDB	5.5418564e-05
7,759	AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft	2020	VLDB	5.4575614e-05
7,925	Runtime Variation in Big Data Analytics	2023	SIGMOD	5.4256585e-05
7,967	Resource Bricolage for Parallel Database Systems	2015	VLDB	5.415894e-05
7,982	InTime: Towards Performance Predictability In Byzantine Fault Tolerant Proof-of-Stake Consensus	2025	SIGMOD	5.4131553e-05
9,012	ByteCard: Enhancing ByteDance’s Data Warehouse with Learned Cardinality Estimation	2024	SIGMOD	5.2375369e-05
9,737	Kora: A Cloud-Native Event Streaming Platform For Kafka	2023	VLDB	5.1349531e-05
11,848	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	4.9793485e-05
12,315	Lifting the Haze off the Cloud: A Consumer-Centric Market for Database Computation in the Cloud	2017	VLDB	4.9793485e-05
12,554	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	4.9793485e-05
13,998	RAFT at Work: Speeding-Up MapReduce Applications under Task and Node Failures	2011	SIGMOD	-

Outgoing Citations (Sorted by Pagerank)

Showing 2 of 2 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00045546775
1,322	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD	0.00011034098

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	5,087	Releasing Cloud Databases from the Chains of Performance Prediction Models	2017	CIDR
2	10,035	From Worst-Case to Average-Case Analysis: Accurate Latency Predictions for Key-Value Storage Engines	2020	SIGMOD
3	4,818	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB
4	1,447	Consistency Rationing in the Cloud: Pay only when it matters	2009	VLDB
5	3,316	Cloud Analytics Benchmark	2023	VLDB
6	1,322	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD
7	9,146	Benchmarking Scalability and Elasticity of Distributed Database Systems	2014	VLDB
8	3,391	RTP: Robust Tenant Placement for Elastic In-Memory Database Clusters	2013	SIGMOD
9	1,466	The Performance of MapReduce: An In-depth Study	2010	VLDB
10	7,925	Runtime Variation in Big Data Analytics	2023	SIGMOD