Real-time Data Infrastructure at Uber

Summary: Uber's real-time data stack: PB-scale ingestion/processing built on OSS with Uber-specific refinements to meet latency and scale. Highlights three scaling challenges per component, and real-time use cases (incentives, fraud, ML) with key lessons. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h990918e39a4256a4
Venue: SIGMOD
Year: 2021
Pagerank: 4.9793485e-05
Overall Rank: 11,990 | 19.39%
DOI: 10.1145/3448016.3457552

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Yupeng Fu (Uber)
2. Chinmay Soman (Uber)

BibTeX Citation

@inproceedings{fu_sigmod21,
        title = {{Real-time Data Infrastructure at Uber}},
        author = {Fu, Yupeng and Soman, Chinmay},
        series = {{SIGMOD} '21},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3448016.3457552},
        url = {https://dl.acm.org/doi/10.1145/3448016.3457552},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 16 of 16 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
23	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00055406774
234	TiDB: A Raft-based HTAP Database	2020	VLDB	0.00023765806
325	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB	0.00020964941
606	Twitter Heron: Stream Processing at Scale	2015	SIGMOD	0.00015635133
685	Trill: A High-Performance Incremental Query Processor for Diverse Analytics	2015	VLDB	0.00014782777
1,135	Dremel: A Decade of Interactive SQL Analysis at Web Scale	2020	VLDB	0.00011891907
1,175	Photon: Fault-tolerant and Scalable Joining of Continuous Data Streams	2013	SIGMOD	0.00011649474
1,236	Druid: A Real-time Analytical Data Store	2014	SIGMOD	0.00011402848
1,241	BatchDB: Efficient Isolated Execution of Hybrid OLTP+OLAP Workloads for Interactive Applications	2017	SIGMOD	0.00011378254
1,303	Realtime Data Processing at Facebook	2016	SIGMOD	0.0001110427
1,423	Procella: Unifying serving and analytical data at YouTube	2019	VLDB	0.00010719956
1,576	F1 – The Fault-Tolerant Distributed RDBMS Supporting Google's Ad Business	2012	SIGMOD	0.00010191761
2,997	F1 Lightning: HTAP as a Service	2020	VLDB	7.7689996e-05
3,846	Alibaba Hologres: A Cloud-Native Service for Hybrid Serving/Analytical Processing	2020	VLDB	6.9851711e-05
4,172	Pinot: Realtime OLAP for 530 Million Users	2018	SIGMOD	6.7598973e-05
6,174	Helios: Hyperscale Indexing for the Cloud & Edge	2020	VLDB	5.8612829e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	12,494	Design and Implementation of a Real-Time Interactive Analytics System for Large Spatio-Temporal Data	2014	VLDB
2	11,383	Ursa: A Lakehouse-Native Data Streaming Engine for Kafka	2025	VLDB
3	12,130	Automated Performance Management for the Big Data Stack	2019	CIDR
4	2,499	Towards Scalable Real-time Analytics: An Architecture for Scale-out of OLxP Workloads	2015	VLDB
5	5,828	Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture	2013	SIGMOD
6	9,816	Supporting Scalable Analytics with Latency Constraints	2015	VLDB
7	3,787	Solving Big Data Challenges for Enterprise Application Performance Management	2012	VLDB
8	3,455	Scaling Spark in the Real World: Performance and Usability	2015	VLDB
9	2,230	Data Warehousing and Analytics Infrastructure at Facebook	2010	SIGMOD
10	1,303	Realtime Data Processing at Facebook	2016	SIGMOD