Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance

Summary: Month-long EC2 study spanning CPU, I/O, network, and MapReduce exposes large bimodal performance variance, tied to VM types and availability zones. Shows cloud noise threatens database wall-clock evaluations and SLAs, and offers measurement guidance. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: hf23153350c3306d8
Venue: VLDB
Year: 2010
Pagerank: 0.00013221727
Overall Rank: 892 | 94.01%
DOI: 10.14778/1920841.1920901

Incoming Non-self Citations Over Time

Authors

1. Jörg Schad (Saarland University)
2. Jens Dittrich (Saarland University)
3. Jorge-Arnulfo Quiané-Ruiz (Saarland University)

BibTeX Citation

@article{schad_vldb10,
        title = {{Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance}},
        author = {Schad, Jörg and Dittrich, Jens and Quiané-Ruiz, Jorge-Arnulfo},
        journal = {PVLDB},
        series = {{VLDB} '10},
        volume = {3},
        number = {1},
        pages = {460--471},
        doi = {10.14778/1920841.1920901},
        url = {https://doi.org/10.14778/1920841.1920901},
        year = {2010}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
52	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00041210636
235	OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases	2014	VLDB	0.00023689995
675	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00014880686
1,236	Dhalion: Self-Regulating Stream Processing in Heron	2017	VLDB	0.00011400483
1,289	An Inquiry into Machine Learning-based Automatic Configuration Tuning Services on Real-World Database Management Systems	2021	VLDB	0.00011159167
1,708	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.8184303e-05
2,262	Data Consistency Properties and the Trade-offs in Commercial Cloud Storages: the Consumers’ Perspective	2011	CIDR	8.7295599e-05
2,833	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD	7.9539771e-05
2,968	Choosing A Cloud DBMS: Architectures and Tradeoffs	2019	VLDB	7.8004755e-05
4,036	Exploiting Cloud Object Storage for High-Performance Analytics	2023	VLDB	6.8360867e-05
5,278	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.1976381e-05
6,514	PLANET: Making Progress with Commit Processing in Unpredictable Environments	2014	SIGMOD	5.7570121e-05
7,356	PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost!	2021	VLDB	5.5421826e-05
7,767	AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft	2020	VLDB	5.4550466e-05
7,930	Runtime Variation in Big Data Analytics	2023	SIGMOD	5.42309e-05
7,971	Resource Bricolage for Parallel Database Systems	2015	VLDB	5.4133337e-05
7,987	InTime: Towards Performance Predictability In Byzantine Fault Tolerant Proof-of-Stake Consensus	2025	SIGMOD	5.4105928e-05
9,010	ByteCard: Enhancing ByteDance’s Data Warehouse with Learned Cardinality Estimation	2024	SIGMOD	5.2371313e-05
9,742	Kora: A Cloud-Native Event Streaming Platform For Kafka	2023	VLDB	5.1325223e-05
11,854	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	4.9769913e-05
12,321	Lifting the Haze off the Cloud: A Consumer-Centric Market for Database Computation in the Cloud	2017	VLDB	4.9769913e-05
12,560	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	4.9769913e-05
14,003	RAFT at Work: Speeding-Up MapReduce Applications under Task and Node Failures	2011	SIGMOD	-

Outgoing Citations (Sorted by Pagerank)

Showing 2 of 2 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
43	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.0004552807
1,322	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD	0.00011029244

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	5,090	Releasing Cloud Databases from the Chains of Performance Prediction Models	2017	CIDR
2	10,040	From Worst-Case to Average-Case Analysis: Accurate Latency Predictions for Key-Value Storage Engines	2020	SIGMOD
3	4,820	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB
4	1,447	Consistency Rationing in the Cloud: Pay only when it matters	2009	VLDB
5	3,311	Cloud Analytics Benchmark	2023	VLDB
6	1,322	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD
7	9,155	Benchmarking Scalability and Elasticity of Distributed Database Systems	2014	VLDB
8	3,391	RTP: Robust Tenant Placement for Elastic In-Memory Database Clusters	2013	SIGMOD
9	1,466	The Performance of MapReduce: An In-depth Study	2010	VLDB
10	7,930	Runtime Variation in Big Data Analytics	2023	SIGMOD