Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance

Summary: Month-long EC2 study spanning CPU, I/O, network, and MapReduce exposes large bimodal performance variance, tied to VM types and availability zones. Shows cloud noise threatens database wall-clock evaluations and SLAs, and offers measurement guidance. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 10210
Venue: VLDB
Year: 2010
Pagerank: 0.00013357681
Overall Rank: 895 | 93.87%
DOI: 10.14778/1920841.1920901

Incoming Non-self Citations Over Time

Authors

1. Jörg Schad (Saarland University)
2. Jens Dittrich (Saarland University)
3. Jorge-Arnulfo Quiané-Ruiz (Saarland University)

BibTeX Citation

@article{schad_vldb10,
        title = {{Runtime Measurements in the Cloud: Observing, Analyzing, and Reducing Variance}},
        author = {Schad, Jörg and Dittrich, Jens and Quiané-Ruiz, Jorge-Arnulfo},
        journal = {PVLDB},
        series = {{VLDB} '10},
        volume = {3},
        number = {1},
        pages = {460--471},
        doi = {10.14778/1920841.1920901},
        url = {https://doi.org/10.14778/1920841.1920901},
        year = {2010}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
66	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00038561587
226	OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases	2014	VLDB	0.00024027277
660	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00015198804
1,224	Dhalion: Self-Regulating Stream Processing in Heron	2017	VLDB	0.00011596911
1,344	An Inquiry into Machine Learning-based Automatic Configuration Tuning Services on Real-World Database Management Systems	2021	VLDB	0.00011094717
1,694	PerfXplain: Debugging MapReduce Job Performance	2012	VLDB	9.9953156e-05
2,228	Data Consistency Properties and the Trade-offs in Commercial Cloud Storages: the Consumers’ Perspective	2011	CIDR	8.9082751e-05
2,822	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD	8.0898536e-05
3,071	Choosing A Cloud DBMS: Architectures and Tradeoffs	2019	VLDB	7.7885156e-05
4,103	Exploiting Cloud Object Storage for High-Performance Analytics	2023	VLDB	6.8993506e-05
5,156	Only Aggressive Elephants are Fast Elephants	2012	VLDB	6.3410921e-05
6,477	PLANET: Making Progress with Commit Processing in Unpredictable Environments	2014	SIGMOD	5.8682525e-05
7,619	AutoToken: Predicting Peak Parallelism for Big Data Analytics at Microsoft	2020	VLDB	5.5810604e-05
7,762	Runtime Variation in Big Data Analytics	2023	SIGMOD	5.5501898e-05
7,819	InTime: Towards Performance Predictability In Byzantine Fault Tolerant Proof-of-Stake Consensus	2025	SIGMOD	5.5373997e-05
7,822	Resource Bricolage for Parallel Database Systems	2015	VLDB	5.5371287e-05
8,040	PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost!	2021	VLDB	5.5018396e-05
8,849	ByteCard: Enhancing ByteDance’s Data Warehouse with Learned Cardinality Estimation	2024	SIGMOD	5.3577504e-05
9,562	Kora: A Cloud-Native Event Streaming Platform For Kafka	2023	VLDB	5.2528121e-05
11,539	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	5.093636e-05
12,020	Lifting the Haze off the Cloud: A Consumer-Centric Market for Database Computation in the Cloud	2017	VLDB	5.093636e-05
12,263	ClouDiA: A Deployment Advisor for Public Clouds	2013	VLDB	5.093636e-05
13,686	RAFT at Work: Speeding-Up MapReduce Applications under Task and Node Failures	2011	SIGMOD	-

Outgoing Citations (Sorted by Pagerank)

Showing 2 of 2 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
44	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00046055057
1,348	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD	0.00011071078

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,987	Releasing Cloud Databases from the Chains of Performance Prediction Models	2017	CIDR
2	11,766	From Worst-Case to Average-Case Analysis: Accurate Latency Predictions for Key-Value Storage Engines	2020	SIGMOD
3	4,742	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB
4	1,423	Consistency Rationing in the Cloud: Pay only when it matters	2009	VLDB
5	3,838	Cloud Analytics Benchmark	2023	VLDB
6	1,348	An Evaluation of Alternative Architectures for Transaction Processing in the Cloud	2010	SIGMOD
7	8,985	Benchmarking Scalability and Elasticity of Distributed Database Systems	2014	VLDB
8	3,373	RTP: Robust Tenant Placement for Elastic In-Memory Database Clusters	2013	SIGMOD
9	1,436	The Performance of MapReduce: An In-depth Study	2010	VLDB
10	7,762	Runtime Variation in Big Data Analytics	2023	SIGMOD