Spark SQL: Relational Data Processing in Spark

Summary: Relational processing integrated into Spark via DataFrame API, unifying SQL queries with Spark's functional workflow. Catalyst, a Scala-based extensible optimizer, enables composable rules, code generation, JSON schema inference, and federation to databases. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5023
Venue: SIGMOD
Year: 2015
Pagerank: 0.00055280697
Overall Rank: 25 | 99.83%
DOI: 10.1145/2723372.2742797

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 6 of 206 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
11,699	An Experimental Evaluation of Garbage Collectors on Big Data Applications	2019	VLDB	5.1725167e-05
11,757	An Authorization Model for Multi-Provider Queries	2018	VLDB	5.1725167e-05
11,761	Effective Temporal Dependence Discovery in Time Series Data	2018	VLDB	5.1725167e-05
11,782	Query Processing Techniques for Big Spatial-Keyword Data	2017	SIGMOD	5.1725167e-05
11,956	Tutorial: SQL-on-Hadoop Systems	2015	VLDB	5.1725167e-05
13,110	Blink Twice - Automatic Workload Pinning and Regression Detection for Versionless Apache Spark using Retries	2025	SIGMOD	-

Outgoing Citations (Sorted by Pagerank)

Showing 15 of 15 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
6	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0010826893
21	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB	0.00057138029
42	A Comparison of Approaches to Large-Scale Data Analysis	2009	SIGMOD	0.00046905759
51	Dremel: Interactive Analysis of Web-Scale Datasets	2010	VLDB	0.00043401322
122	The EXODUS Optimizer Generator	1987	SIGMOD	0.00031727197
153	MAD Skills: New Analysis Practices for Big Data	2009	VLDB	0.00029010571
322	Impala: A Modern, Open-Source SQL Engine for Hadoop	2015	CIDR	0.00021314511
416	Shark: SQL and Rich Analytics at Scale	2013	SIGMOD	0.00018911836
520	Building Efficient Query Engines in a High-Level Language	2014	VLDB	0.00017224358
1,235	Extracting Schema from Semistructured Data	1998	SIGMOD	0.00011628442
1,717	Sinew: A SQL System for Multi-Structured Data	2014	SIGMOD	0.00010019304
1,776	Distributed Data-Parallel Computing Using a High-Level Programming Language	2009	SIGMOD	9.8629747e-05
1,825	G-OLA: Generalized On-Line Aggregation for Interactive Analysis on Big Data	2015	SIGMOD	9.7685715e-05
2,755	Inferring XML Schema Definitions from XML Data	2007	VLDB	8.227934e-05
3,508	Rethinking Data-Intensive Science Using Scalable Analytics Systems	2015	SIGMOD	7.4290429e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
3,358	Scaling Spark in the Real World: Performance and Usability	2015	VLDB	7.5513638e-05
416	Shark: SQL and Rich Analytics at Scale	2013	SIGMOD	0.00018911836
403	SystemML: Declarative Machine Learning on Spark	2016	VLDB	0.00019232751
2,553	Big Data Analytics with Datalog Queries on Spark	2016	SIGMOD	8.4879475e-05
11,580	RASQL: A Powerful Language and its System for Big Data Applications	2020	SIGMOD	5.1725167e-05
1,139	Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark	2018	SIGMOD	0.00012096709
6,785	SparkR: Scaling R Programs with Spark	2016	SIGMOD	5.8327875e-05
9,678	Introduction to Spark 2.0 for Database Researchers	2016	SIGMOD	5.2933195e-05
9,031	Dynamic Speculative Optimizations for SQL Compilation in Apache Spark	2020	VLDB	5.3924601e-05
2,882	RaSQL: Greater Power and Performance for Big Data Analytics with Recursive-aggregate-SQL on Spark	2019	SIGMOD	8.0670857e-05