Data Warehousing and Analytics Infrastructure at Facebook

Summary: Facebook's scalable data warehouse uses Scribe, Hadoop, Hive to unify log collection, storage, and analytics for BI dashboards and feature services. Stores >15PB (2.5PB compressed), ingests ~60TB/day (10TB compressed); discusses design choices, day-to-day operations, and planned improvements. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 4315
Venue: SIGMOD
Year: 2010
Pagerank: 8.3634079e-05
Overall Rank: 2,658 | 81.53%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 17 of 17 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,616	Realtime Data Processing at Facebook	2016	SIGMOD	0.00011133818
1,838	Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing	2014	VLDB	0.00010353323
2,211	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.2823037e-05
2,929	WANalytics: Analytics for a Geo-Distributed Data-Intensive World	2015	CIDR	7.8792902e-05
4,563	The Unified Logging Infrastructure for Data Analytics at Twitter	2012	VLDB	6.0752535e-05
4,859	The "Big Data" Ecosystem at LinkedIn	2013	SIGMOD	5.8680444e-05
5,110	Only Aggressive Elephants are Fast Elephants	2012	VLDB	5.6868273e-05
6,135	Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture	2013	SIGMOD	5.1908958e-05
6,278	The Cosmos Big Data Platform at Microsoft: Over a Decade of Progress and a Decade to Look Forward	2021	VLDB	5.1241654e-05
6,280	Cheetah: Accelerating Database Queries with Switch Pruning	2020	SIGMOD	5.1239052e-05
6,856	Liquid: Unifying Nearline and Offline Big Data Integration	2015	CIDR	4.901516e-05
7,306	Compliant Geo-distributed Query Processing	2021	SIGMOD	4.7626702e-05
7,926	Data Management for Social Networking	2016	PODS	4.6107647e-05
9,360	Efficient Big Data Processing in Hadoop MapReduce	2012	VLDB	4.3476444e-05
10,429	Unified Lineage System: Tracking Data Provenance at Scale	2025	SIGMOD	4.1905499e-05
11,857	The Challenges of Global-scale Data Management	2016	SIGMOD	4.1905499e-05
11,966	Shared Execution of Recurring Workloads in MapReduce	2015	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
80	Weaving Relations for Cache Performance	2001	VLDB	0.00055735291

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
3,606	Large-Scale Machine Learning at Twitter	2012	SIGMOD	6.9246371e-05
2,340	Efficient Processing of Data Warehousing Queries in a Split Execution Environment	2011	SIGMOD	9.001663e-05
158	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.00040401371
2,476	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD	8.6907971e-05
7,879	Emerging Trends in the Enterprise Data Analytics: Connecting Hadoop and DB2 Warehouse	2011	SIGMOD	4.6253726e-05
1,497	Apache Hadoop Goes Realtime at Facebook	2011	SIGMOD	0.00011668393
4,859	The "Big Data" Ecosystem at LinkedIn	2013	SIGMOD	5.8680444e-05
3,966	Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing	2019	SIGMOD	6.5782437e-05
6,843	Petabyte Scale Databases and Storage Systems at Facebook	2013	SIGMOD	4.9046389e-05
70	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00059744625