Real-time Data Infrastructure at Uber

Summary: Uber's real-time data stack: PB-scale ingestion/processing built on OSS with Uber-specific refinements to meet latency and scale. Highlights three scaling challenges per component, and real-time use cases (incentives, fraud, ML) with key lessons. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6303
Venue: SIGMOD
Year: 2021
Pagerank: 5.093636e-05
Overall Rank: 11,683 | 19.85%
DOI: 10.1145/3448016.3457552

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Yupeng Fu (Uber)
2. Chinmay Soman (Uber)

BibTeX Citation

@inproceedings{fu_sigmod21,
        title = {{Real-time Data Infrastructure at Uber}},
        author = {Fu, Yupeng and Soman, Chinmay},
        series = {{SIGMOD} '21},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3448016.3457552},
        url = {https://dl.acm.org/doi/10.1145/3448016.3457552},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 16 of 16 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
24	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00054865648
271	TiDB: A Raft-based HTAP Database	2020	VLDB	0.00022703024
361	The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing	2015	VLDB	0.00020138717
612	Twitter Heron: Stream Processing at Scale	2015	SIGMOD	0.0001573018
710	Trill: A High-Performance Incremental Query Processor for Diverse Analytics	2015	VLDB	0.00014715033
1,171	Photon: Fault-tolerant and Scalable Joining of Continuous Data Streams	2013	SIGMOD	0.00011826434
1,238	BatchDB: Efficient Isolated Execution of Hybrid OLTP+OLAP Workloads for Interactive Applications	2017	SIGMOD	0.00011543368
1,242	Druid: A Real-time Analytical Data Store	2014	SIGMOD	0.00011516162
1,389	Realtime Data Processing at Facebook	2016	SIGMOD	0.00010943767
1,453	Dremel: A Decade of Interactive SQL Analysis at Web Scale	2020	VLDB	0.00010742227
1,494	Procella: Unifying serving and analytical data at YouTube	2019	VLDB	0.00010577585
1,593	F1 – The Fault-Tolerant Distributed RDBMS Supporting Google's Ad Business	2012	SIGMOD	0.00010252961
2,972	F1 Lightning: HTAP as a Service	2020	VLDB	7.9114157e-05
4,102	Alibaba Hologres: A Cloud-Native Service for Hybrid Serving/Analytical Processing	2020	VLDB	6.8993947e-05
4,231	Pinot: Realtime OLAP for 530 Million Users	2018	SIGMOD	6.8177976e-05
6,045	Helios: Hyperscale Indexing for the Cloud & Edge	2020	VLDB	5.9957544e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	12,203	Design and Implementation of a Real-Time Interactive Analytics System for Large Spatio-Temporal Data	2014	VLDB
2	11,018	Ursa: A Lakehouse-Native Data Streaming Engine for Kafka	2025	VLDB
3	11,829	Automated Performance Management for the Big Data Stack	2019	CIDR
4	2,583	Towards Scalable Real-time Analytics: An Architecture for Scale-out of OLxP Workloads	2015	VLDB
5	5,748	Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture	2013	SIGMOD
6	9,640	Supporting Scalable Analytics with Latency Constraints	2015	VLDB
7	3,963	Solving Big Data Challenges for Enterprise Application Performance Management	2012	VLDB
8	3,411	Scaling Spark in the Real World: Performance and Usability	2015	VLDB
9	2,191	Data Warehousing and Analytics Infrastructure at Facebook	2010	SIGMOD
10	1,389	Realtime Data Processing at Facebook	2016	SIGMOD