Shark: SQL and Rich Analytics at Scale

Summary: Shark unifies SQL and analytics on clusters via a distributed memory abstraction into a single scalable engine. In-memory columnar storage, replanning, and fault tolerance enable SQL and ML, 100x faster than Hive/Hadoop, competitive with MPP. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 4698
Venue: SIGMOD
Year: 2013
Pagerank: 0.00020615453
Overall Rank: 539 | 96.26%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 50 of 54 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
66	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00061707583
1,301	Simba: Efficient In-Memory Spatial Analytics	2016	SIGMOD	0.00012702767
1,473	Fine-grained Partitioning for Aggressive Data Skipping	2014	SIGMOD	0.00011786148
1,488	Scuba: Diving into Data at Facebook	2013	VLDB	0.00011690191
1,838	Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing	2014	VLDB	0.00010353323
1,867	Knowing When You’re Wrong: Building Fast and Reliable Approximate Query Processing Systems	2014	SIGMOD	0.00010264932
1,938	From Theory to Practice: Efficient Join Query Evaluation in a Parallel Database System	2015	SIGMOD	0.00010025547
2,123	SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures	2014	VLDB	9.4836925e-05
2,216	Skew in Parallel Query Processing	2014	PODS	9.2693784e-05
2,409	WideTable: An Accelerator for Analytical Data Processing	2014	VLDB	8.867144e-05
2,777	Quickstep: A Data Platform Based on the Scaling-Up Approach	2018	VLDB	8.1346418e-05
2,850	Towards Scalable Real-time Analytics: An Architecture for Scale-out of OLxP Workloads	2015	VLDB	8.0194228e-05
2,929	WANalytics: Analytics for a Geo-Distributed Data-Intensive World	2015	CIDR	7.8792902e-05
3,000	BigDansing: A System for Big Data Cleansing	2015	SIGMOD	7.7447724e-05
3,070	HAWQ: A Massively Parallel Processing SQL Engine in Hadoop	2014	SIGMOD	7.6157391e-05
3,611	Column Sketches: A Scan Accelerator for Rapid and Robust Predicate Evaluation	2018	SIGMOD	6.9178844e-05
3,825	Locality-aware Partitioning in Parallel Database Systems	2015	SIGMOD	6.7225803e-05
4,040	In-RDBMS Hardware Acceleration of Advanced Analytics	2018	VLDB	6.5052227e-05
4,053	WANalytics: Geo-Distributed Analytics for a Data Intensive World	2015	SIGMOD	6.4916925e-05
4,160	Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe?	2017	SIGMOD	6.3886736e-05
5,012	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD	5.7543101e-05
5,118	Design Tradeoffs of Data Access Methods	2016	SIGMOD	5.6781464e-05
5,373	Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing	2022	VLDB	5.5410059e-05
5,841	A Performance Study of Big Data on Small Nodes	2015	VLDB	5.3062554e-05
6,079	Opportunistic Physical Design for Big Data Analytics	2014	SIGMOD	5.2194019e-05
6,301	Elastic Pipelining in an In-Memory Database Cluster	2016	SIGMOD	5.1172165e-05
6,489	Towards General and Efficient Online Tuning for Spark	2023	VLDB	5.0373773e-05
6,683	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	4.9593505e-05
6,783	SparkR: Scaling R Programs with Spark	2016	SIGMOD	4.9221272e-05
6,799	Understanding Insights into the Basic Structure and Essential Issues of Table Placement Methods in Clusters	2013	VLDB	4.9181295e-05
6,807	Adaptive Data Skipping in Main-Memory Systems	2016	SIGMOD	4.915939e-05
6,856	Liquid: Unifying Nearline and Offline Big Data Integration	2015	CIDR	4.901516e-05
6,898	Decentralized Actor Scheduling and Reference-based Storage in Xorbits: a Native Scalable Data Science Engine	2025	VLDB	4.8878659e-05
7,079	JetScope: Reliable and Interactive Analytics at Cloud Scale	2015	VLDB	4.8353804e-05
7,205	Kodiak: Leveraging Materialized Views For Very Low-Latency Analytics Over High-Dimensional Web-Scale Data	2016	VLDB	4.7965293e-05
7,367	Using VDMS to Index and Search 100M Images	2021	VLDB	4.7459078e-05
7,386	Bubble Execution: Resource-aware Reliable Analytics at Cloud Scale	2018	VLDB	4.7398757e-05
7,600	Quill: Efficient, Transferable, and Rich Analytics at Scale	2016	VLDB	4.697054e-05
7,920	JoinBoost: Grow Trees Over Normalized Data Using Only SQL	2023	VLDB	4.6120304e-05
8,196	SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft	2021	VLDB	4.5568952e-05
8,213	Parallel-Correctness and Transferability for Conjunctive Queries	2015	PODS	4.5533861e-05
8,460	Piranha: Optimizing Short Jobs in Hadoop	2013	VLDB	4.5008938e-05
8,585	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB	4.4856045e-05
8,927	QMapper for Smart Grid: Migrating SQL-based Application to Hive	2015	SIGMOD	4.4229886e-05
9,451	Cost-based Fault-tolerance for Parallel Data Processing	2015	SIGMOD	4.3364392e-05
9,585	Introduction to Spark 2.0 for Database Researchers	2016	SIGMOD	4.31773e-05
10,415	Dynamic Pruning for Recursive Joins	2025	SIGMOD	4.1905499e-05
11,695	Integration of Large-Scale Data Processing Systems and Traditional Parallel Database Technology	2019	VLDB	4.1905499e-05
11,839	Logical Aspects of Massively Parallel and Distributed Systems	2016	PODS	4.1905499e-05
11,890	Parallel Evaluation of Multi-Semi-Joins	2016	VLDB	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
4	Pregel: A System for Large-Scale Graph Processing	2010	SIGMOD	0.0019040811
20	C-Store: A Column-oriented DBMS	2005	VLDB	0.00086163998
22	SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets	2008	VLDB	0.00084679526
39	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB	0.00075263552
42	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00073570328
109	Dremel: Interactive Analysis of Web-Scale Datasets	2010	VLDB	0.00048217028
116	Eddies: Continuously Adaptive Query Processing	2000	SIGMOD	0.00046191288
158	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB	0.00040401371
169	MAD Skills: New Analysis Practices for Big Data	2009	VLDB	0.00038967371
221	Efficient Mid-Query Re-Optimization of Sub-Optimal Query Execution Plans	1998	SIGMOD	0.00033182072
408	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00023939456
457	Cost-based Query Scrambling for Initial Delays	1998	SIGMOD	0.00022699974
638	Towards a Unified Architecture for in-RDBMS Analytics	2012	SIGMOD	0.00018810785
909	Tenzing A SQL Implementation On The MapReduce Framework	2011	VLDB	0.0001540191
1,339	SkewTune: Mitigating Skew in MapReduce Applications	2012	SIGMOD	0.00012492978
1,470	Processing a Trillion Cells per Mouse Click	2012	VLDB	0.00011827274
1,719	Distributed Data-Parallel Computing Using a High-Level Programming Language	2009	SIGMOD	0.00010759065
1,945	Cheetah: A High Performance, Custom Data Warehouse on Top of MapReduce	2010	VLDB	0.0001000706

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
6,683	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	4.9593505e-05
2,123	SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures	2014	VLDB	9.4836925e-05
7,600	Quill: Efficient, Transferable, and Rich Analytics at Scale	2016	VLDB	4.697054e-05
66	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00061707583
3,070	HAWQ: A Massively Parallel Processing SQL Engine in Hadoop	2014	SIGMOD	7.6157391e-05
3,966	Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing	2019	SIGMOD	6.5782437e-05
1,151	Blink and It's Done: Interactive Queries on Very Large Data	2012	VLDB	0.00013634671
4,706	SharkDB: An In-Memory Storage System for Massive Trajectory Data	2015	SIGMOD	5.9773995e-05
1,048	Starfish: A Self-tuning System for Big Data Analytics	2011	CIDR	0.00014442178
2,487	Shark: Fast Data Analysis Using Coarse-grained Distributed Memory	2012	SIGMOD	8.6609871e-05