Shark: SQL and Rich Analytics at Scale

Summary: Shark unifies SQL and analytics on clusters via a distributed memory abstraction into a single scalable engine. In-memory columnar storage, replanning, and fault tolerance enable SQL and ML, 100x faster than Hive/Hadoop, competitive with MPP. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 4698
Venue: SIGMOD
Year: 2013
Pagerank: 0.00018911753
Overall Rank: 416 | 97.12%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 50 of 54 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
25	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00055280049
1,041	Fine-grained Partitioning for Aggressive Data Skipping	2014	SIGMOD	0.00012565893
1,162	Simba: Efficient In-Memory Spatial Analytics	2016	SIGMOD	0.00011985579
1,345	Scuba: Diving into Data at Facebook	2013	VLDB	0.00011188833
1,389	Knowing When You’re Wrong: Building Fast and Reliable Approximate Query Processing Systems	2014	SIGMOD	0.00011029933
1,470	From Theory to Practice: Efficient Join Query Evaluation in a Parallel Database System	2015	SIGMOD	0.00010746951
1,577	Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing	2014	VLDB	0.00010368033
1,669	Skew in Parallel Query Processing	2014	PODS	0.00010137031
1,885	SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures	2014	VLDB	9.6357703e-05
2,131	Quickstep: A Data Platform Based on the Scaling-Up Approach	2018	VLDB	9.1824992e-05
2,360	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	8.7662564e-05
2,559	Towards Scalable Real-time Analytics: An Architecture for Scale-out of OLxP Workloads	2015	VLDB	8.4826976e-05
2,585	WideTable: An Accelerator for Analytical Data Processing	2014	VLDB	8.4540632e-05
2,804	HAWQ: A Massively Parallel Processing SQL Engine in Hadoop	2014	SIGMOD	8.1680595e-05
2,916	Column Sketches: A Scan Accelerator for Rapid and Robust Predicate Evaluation	2018	SIGMOD	8.0325296e-05
3,152	Locality-aware Partitioning in Parallel Database Systems	2015	SIGMOD	7.7601909e-05
3,453	WANalytics: Analytics for a Geo-Distributed Data-Intensive World	2015	CIDR	7.4746036e-05
3,553	Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe?	2017	SIGMOD	7.3775219e-05
3,650	In-RDBMS Hardware Acceleration of Advanced Analytics	2018	VLDB	7.291319e-05
4,239	WANalytics: Geo-Distributed Analytics for a Data Intensive World	2015	SIGMOD	6.8751474e-05
4,419	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD	6.7759069e-05
4,787	Design Tradeoffs of Data Access Methods	2016	SIGMOD	6.574662e-05
5,366	Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing	2022	VLDB	6.3169865e-05
5,639	Opportunistic Physical Design for Big Data Analytics	2014	SIGMOD	6.2030946e-05
6,039	A Performance Study of Big Data on Small Nodes	2015	VLDB	6.0592432e-05
6,164	Elastic Pipelining in an In-Memory Database Cluster	2016	SIGMOD	6.0304568e-05
6,247	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	6.0034176e-05
6,248	Towards General and Efficient Online Tuning for Spark	2023	VLDB	6.0022621e-05
6,536	Understanding Insights into the Basic Structure and Essential Issues of Table Placement Methods in Clusters	2013	VLDB	5.9059287e-05
6,544	Adaptive Data Skipping in Main-Memory Systems	2016	SIGMOD	5.9038156e-05
6,629	Liquid: Unifying Nearline and Offline Big Data Integration	2015	CIDR	5.8773042e-05
6,715	JetScope: Reliable and Interactive Analytics at Cloud Scale	2015	VLDB	5.8520075e-05
6,785	SparkR: Scaling R Programs with Spark	2016	SIGMOD	5.8328772e-05
7,037	Bubble Execution: Resource-aware Reliable Analytics at Cloud Scale	2018	VLDB	5.7720294e-05
7,120	Kodiak: Leveraging Materialized Views For Very Low-Latency Analytics Over High-Dimensional Web-Scale Data	2016	VLDB	5.7519336e-05
7,368	Decentralized Actor Scheduling and Reference-based Storage in Xorbits: a Native Scalable Data Science Engine	2025	VLDB	5.6897772e-05
7,583	Quill: Efficient, Transferable, and Rich Analytics at Scale	2016	VLDB	5.6487011e-05
7,587	JoinBoost: Grow Trees Over Normalized Data Using Only SQL	2023	VLDB	5.6470468e-05
7,608	Using VDMS to Index and Search 100M Images	2021	VLDB	5.6423505e-05
8,046	SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft	2021	VLDB	5.5585663e-05
8,321	Parallel-Correctness and Transferability for Conjunctive Queries	2015	PODS	5.5080112e-05
8,329	Piranha: Optimizing Short Jobs in Hadoop	2013	VLDB	5.5065836e-05
8,472	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB	5.4888877e-05
8,951	QMapper for Smart Grid: Migrating SQL-based Application to Hive	2015	SIGMOD	5.4076395e-05
9,448	Cost-based Fault-tolerance for Parallel Data Processing	2015	SIGMOD	5.3309226e-05
9,678	Introduction to Spark 2.0 for Database Researchers	2016	SIGMOD	5.293328e-05
10,415	Dynamic Pruning for Recursive Joins	2025	SIGMOD	5.1725247e-05
11,695	Integration of Large-Scale Data Processing Systems and Traditional Parallel Database Technology	2019	VLDB	5.1725247e-05
11,839	Logical Aspects of Massively Parallel and Distributed Systems	2016	PODS	5.1725247e-05
11,890	Parallel Evaluation of Multi-Semi-Joins	2016	VLDB	5.1725247e-05

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
3	Pregel: A System for Large-Scale Graph Processing	2010	SIGMOD	0.0012294368
12	C-Store: A Column-oriented DBMS	2005	VLDB	0.00070779065
21	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB	0.00057138114
31	SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets	2008	VLDB	0.00051798124
42	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00046908254
51	Dremel: Interactive Analysis of Web-Scale Datasets	2010	VLDB	0.00043359268
53	Eddies: Continuously Adaptive Query Processing	2000	SIGMOD	0.00041541642
121	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.00032158142
150	Efficient Mid-Query Re-Optimization of Sub-Optimal Query Execution Plans	1998	SIGMOD	0.00029370827
153	MAD Skills: New Analysis Practices for Big Data	2009	VLDB	0.00029010702
371	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00020014052
409	Cost-based Query Scrambling for Initial Delays	1998	SIGMOD	0.0001903673
561	Towards a Unified Architecture for in-RDBMS Analytics	2012	SIGMOD	0.00016568485
810	Tenzing A SQL Implementation On The MapReduce Framework	2011	VLDB	0.00013936806
1,219	Processing a Trillion Cells per Mouse Click	2012	VLDB	0.00011705588
1,323	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00011264233
1,756	Cheetah: A High Performance, Custom Data Warehouse on Top of MapReduce	2010	VLDB	9.9092914e-05
1,775	Distributed Data-Parallel Computing Using a High-Level Programming Language	2009	SIGMOD	9.8630311e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
6,247	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	6.0034176e-05
1,885	SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures	2014	VLDB	9.6357703e-05
7,583	Quill: Efficient, Transferable, and Rich Analytics at Scale	2016	VLDB	5.6487011e-05
25	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00055280049
2,804	HAWQ: A Massively Parallel Processing SQL Engine in Hadoop	2014	SIGMOD	8.1680595e-05
3,510	Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing	2019	SIGMOD	7.4221305e-05
1,212	Blink and It's Done: Interactive Queries on Very Large Data	2012	VLDB	0.00011741576
4,727	SharkDB: An In-Memory Storage System for Massive Trajectory Data	2015	SIGMOD	6.6048147e-05
900	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR	0.00013374475
2,168	Shark: Fast Data Analysis Using Coarse-grained Distributed Memory	2012	SIGMOD	9.0925999e-05