Introduction to Spark 2.0 for Database Researchers
Summary: Introduces Spark 2.0 to database researchers, focusing on Catalyst, whole-stage code generation, and hands-on use of DataFrames, SQL, streaming, and ML pipelines. Emphasizes engine internals and a practical path to hack Spark by extending the query optimizer to accelerate distributed joins. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Michael Armbrust
- 2. Doug Bateman
- 3. Reynold Xin
- 4. Matei Zaharia
Incoming Citations (Sorted by Pagerank)
Showing 1 of 1 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 1,534 | Data Management in Machine Learning: Challenges, Techniques, and Systems | 2017 | SIGMOD | 0.00011462072 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 4 of 4 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 66 | Spark SQL: Relational Data Processing in Spark | 2015 | SIGMOD | 0.00061707583 |
| 539 | Shark: SQL and Rich Analytics at Scale | 2013 | SIGMOD | 0.00020615453 |
| 3,536 | Scaling Spark in the Real World: Performance and Usability | 2015 | VLDB | 6.9938207e-05 |
| 6,783 | SparkR: Scaling R Programs with Spark | 2016 | SIGMOD | 4.9221272e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| Overall Rank | Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 1,546 | Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark | 2018 | SIGMOD | 0.00011418993 |
| 9,517 | [Demo] Low-latency Spark Queries on Updatable Data | 2019 | SIGMOD | 4.3294347e-05 |
| 8,585 | A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning | 2024 | VLDB | 4.4856045e-05 |
| 6,783 | SparkR: Scaling R Programs with Spark | 2016 | SIGMOD | 4.9221272e-05 |
| 6,265 | Speedup Your Analytics: Automatic Parameter Tuning for Databases and Big Data Systems | 2019 | VLDB | 5.1294788e-05 |
| 7,019 | Bridging the Gap Between HPC and Big Data Frameworks | 2017 | VLDB | 4.8553946e-05 |
| 3,207 | Big Data Analytics with Datalog Queries on Spark | 2016 | SIGMOD | 7.3847098e-05 |
| 66 | Spark SQL: Relational Data Processing in Spark | 2015 | SIGMOD | 0.00061707583 |
| 3,536 | Scaling Spark in the Real World: Performance and Usability | 2015 | VLDB | 6.9938207e-05 |
| 13,362 | Trends and Challenges in Big Data Processing | 2016 | VLDB | - |