Photon: A Fast Query Engine for Lakehouse Systems

Summary: Photon is a vectorized Lakehouse query engine, delivering fast queries on raw data lakes and Parquet via Spark API. Design choices (vectorization vs. code generation), memory manager, and SQL/Spark integration enable 10x gains and a 100TB TPC-DS record. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6431
Venue: SIGMOD
Year: 2022
Pagerank: 8.7156826e-05
Overall Rank: 2,472 | 82.83%
DOI: 10.1145/3514221.3526054

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 28 of 28 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,642	BtrBlocks: Efficient Columnar Compression for Data Lakes	2023	SIGMOD	6.8876984e-05
4,241	The Composable Data Management System Manifesto	2023	VLDB	6.3258298e-05
4,498	ClickHouse - Lightning Fast Analytics for Everyone	2024	VLDB	6.1351257e-05
4,873	Exploiting Cloud Object Storage for High-Performance Analytics	2023	VLDB	5.8557568e-05
5,105	Analyzing and Comparing Lakehouse Storage Systems	2023	CIDR	5.6916018e-05
5,537	Presto: A Decade of SQL Analytics at Meta	2023	SIGMOD	5.453017e-05
6,332	Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine	2024	SIGMOD	5.1021765e-05
6,683	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	4.9593505e-05
7,427	Selection Pushdown in Column Stores using Bit Manipulation Instructions	2023	SIGMOD	4.7282014e-05
7,544	Is Perfect Hashing Practical for OLAP Systems?	2024	CIDR	4.710321e-05
7,820	Deep Lake: a Lakehouse for Deep Learning	2023	CIDR	4.6394471e-05
7,917	Terabyte-Scale Analytics in the Blink of an Eye	2026	VLDB	4.6129625e-05
8,174	Sigma Workbook: A Spreadsheet for Cloud Data Warehouses	2022	VLDB	4.5638044e-05
8,477	Excalibur: A Virtual Machine for Adaptive Fine-grained JIT-Compiled Query Execution based on VOILA	2023	VLDB	4.4971772e-05
8,856	Composable Data Management: An Execution Overview	2024	VLDB	4.4303659e-05
9,856	Towards Unifying Query Interpretation and Compilation	2023	CIDR	4.2652623e-05
9,972	End-to-End Declarative Data Analytics: Co-designing Engines, Interfaces, and Cloud Infrastructure	2026	CIDR	4.1905499e-05
10,384	Data Chunk Compaction in Vectorized Execution	2025	SIGMOD	4.1905499e-05
10,504	Nested Parquet Is Flat, Why Not Use It? How To Scan Nested Data With On-the-Fly Key Generation and Joins	2025	SIGMOD	4.1905499e-05
10,721	Towards Designing Future-Proof Data Processing Systems	2025	VLDB	4.1905499e-05
10,773	The HANA Native Query Engine for Lakehouse Systems	2025	VLDB	4.1905499e-05
10,783	Magnus: A Holistic Approach to Data Management for Large-Scale Machine Learning Workloads	2025	VLDB	4.1905499e-05
10,793	AnalyticDB-PG: A Cloud-native High-performance Data Warehouse in Alibaba Cloud	2025	VLDB	4.1905499e-05
10,808	GraphAr: An Efficient Storage Scheme for Graph Data in Data Lakes	2025	VLDB	4.1905499e-05
10,858	LiquidCache: Efficient Pushdown Caching for Cloud-Native Data Analytics	2025	VLDB	4.1905499e-05
10,972	Query Compilation Without Regrets	2024	SIGMOD	4.1905499e-05
11,093	Simple (yet Efficient) Function Authoring for Vectorized Engines	2024	VLDB	4.1905499e-05
11,271	Big Data Analytic Toolkit: A general-purpose, modular, and heterogeneous acceleration toolkit for data analytical engines	2023	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 19 of 19 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
35	MonetDB/X100: Hyper-Pipelining Query Execution	2005	CIDR	0.00076209479
52	Database Architecture Optimized for the new Bottleneck: Memory Access	1999	VLDB	0.00066322421
59	Efficiently Compiling Efficient Query Plans for Modern Hardware	2011	VLDB	0.0006445664
66	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00061707583
113	Encapsulation of Parallelism in the Volcano Query Processing System	1990	SIGMOD	0.0004673401
167	The Snowflake Elastic Data Warehouse	2016	SIGMOD	0.00039408116
310	The Vertica Analytic Database: C-Store 7 Years Later	2012	VLDB	0.0002815547
343	Implementing Database Operations Using SIMD Instructions	2002	SIGMOD	0.00026756534
408	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00023939456
739	Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores	2020	VLDB	0.00017365933
848	Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask	2018	VLDB	0.00015933538
959	Rethinking SIMD Vectorization for In-Memory Databases	2015	SIGMOD	0.00015034808
1,260	Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation	2016	SIGMOD	0.00012990853
1,356	Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics	2021	CIDR	0.00012409986
1,751	Weld: A Common Runtime for High Performance Data Analytics	2017	CIDR	0.00010673874
1,865	Relaxed Operator Fusion for In-Memory Databases: Making Compilation, Vectorization, and Prefetching Work Together At Last	2018	VLDB	0.00010271333
2,904	Evaluating End-to-End Optimization for Data Analytics Applications in Weld	2018	VLDB	7.9403097e-05
3,881	Micro Adaptivity in Vectorwise	2013	SIGMOD	6.6638918e-05
4,269	Maximizing Persistent Memory Bandwidth Utilization for OLAP Workloads	2021	SIGMOD	6.2950779e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
8,502	New Query Optimization Techniques in the Spark Engine of Azure Synapse	2022	VLDB	4.491819e-05
10,196	PTO: A Workload-driven Predictive Table Optimizer for Lakehouse Systems	2026	SIGMOD	4.1905499e-05
6,397	BigLake: BigQuery’s Evolution toward a Multi-Cloud Lakehouse	2024	SIGMOD	5.0749432e-05
4,498	ClickHouse - Lightning Fast Analytics for Everyone	2024	VLDB	6.1351257e-05
1,356	Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics	2021	CIDR	0.00012409986
10,773	The HANA Native Query Engine for Lakehouse Systems	2025	VLDB	4.1905499e-05
9,856	Towards Unifying Query Interpretation and Compilation	2023	CIDR	4.2652623e-05
7,907	Petabyte-Scale Row-Level Operations in Data Lakehouses	2024	VLDB	4.6161532e-05
6,683	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	4.9593505e-05
9,807	Photon: A High-Performance Query Engine for the Lakehouse	2022	CIDR	4.275302e-05