The Unified Logging Infrastructure for Data Analytics at Twitter

Summary: Twitter’s production logging system unifies heterogeneous application logs into flexible, well-structured Thrift client events. It also pre-materializes compact session sequences, accelerating common analytics and enabling faster service experimentation. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: hb52e94c395aef685
Venue: VLDB
Year: 2012
Pagerank: 6.7037533e-05
Overall Rank: 4,249 | 71.44%
DOI: 10.14778/2367502.2367518

Incoming Non-self Citations Over Time

Authors

1. George Lee (Twitter)
2. Jimmy Lin (Twitter)
3. Chuang Liu (Twitter)
4. Andrew Lorek (Twitter)
5. Dmitriy Ryaboy (Twitter)

BibTeX Citation

@article{lee_vldb12,
        title = {{The Unified Logging Infrastructure for Data Analytics at Twitter}},
        author = {Lee, George and Lin, Jimmy and Liu, Chuang and Lorek, Andrew and Ryaboy, Dmitriy},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {12},
        pages = {1771--1780},
        doi = {10.14778/2367502.2367518},
        url = {https://doi.org/10.14778/2367502.2367518},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 12 of 12 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,622	Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing	2014	VLDB	0.00010051462
1,633	Summingbird: A Framework for Integrating Batch and Online MapReduce Computations	2014	VLDB	0.00010019617
3,556	WANalytics: Analytics for a Geo-Distributed Data-Intensive World	2015	CIDR	7.2084485e-05
4,177	GraphJet: Real-Time Content Recommendations at Twitter	2016	VLDB	6.757191e-05
4,691	The "Big Data" Ecosystem at LinkedIn	2013	SIGMOD	6.4685355e-05
5,828	Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture	2013	SIGMOD	5.9786171e-05
5,838	PreLog: A Pre-trained Model for Log Analytics	2024	SIGMOD	5.9752181e-05
6,245	The Cosmos Big Data Platform at Microsoft: Over a Decade of Progress and a Decade to Look Forward	2021	VLDB	5.837329e-05
7,404	Compliant Geo-distributed Query Processing	2021	SIGMOD	5.5357067e-05
8,177	Data Management for Social Networking	2016	PODS	5.3829638e-05
11,343	CoLA: Model Collaboration for Log-based Anomaly Detection	2025	VLDB	4.9793485e-05
12,213	Robust, Scalable, Real-Time Event Time Series Aggregation at Twitter	2018	SIGMOD	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 9 of 9 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.001052036
120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.000311132
651	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00015130782
673	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.0001488755
1,302	Apache Hadoop Goes Realtime at Facebook	2011	SIGMOD	0.00011111126
2,173	Efficient Processing of Data Warehousing Queries in a Split Execution Environment	2011	SIGMOD	8.91924e-05
2,230	Data Warehousing and Analytics Infrastructure at Facebook	2010	SIGMOD	8.799877e-05
3,007	Llama: Leveraging Columnar Storage for Scalable Join Processing in the MapReduce Framework	2011	SIGMOD	7.7607173e-05
3,788	Large-Scale Machine Learning at Twitter	2012	SIGMOD	7.0195464e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	4,691	The "Big Data" Ecosystem at LinkedIn	2013	SIGMOD
2	1,072	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
3	2,687	CoHadoop: Flexible Data Placement and Its Exploitation in Hadoop	2011	VLDB
4	606	Twitter Heron: Stream Processing at Scale	2015	SIGMOD
5	799	A Comparison of Join Algorithms for Log Processing in MapReduce	2010	SIGMOD
6	2,230	Data Warehousing and Analytics Infrastructure at Facebook	2010	SIGMOD
7	9,816	Supporting Scalable Analytics with Latency Constraints	2015	VLDB
8	2,282	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
9	3,788	Large-Scale Machine Learning at Twitter	2012	SIGMOD
10	5,828	Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture	2013	SIGMOD