DBScholar

Back to papers

Big Data Analytics with Datalog Queries on Spark

Summary: BigDatalog enables concise declarative Datalog queries for large-scale analytics on Spark. It uses compilation and optimization to efficiently support recursion on Spark, with empirical comparisons against top Datalog systems showing Spark-based analytics viable. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
5318
Venue
SIGMOD
Year
2016
Pagerank
8.3646367e-05
Overall Rank
2,594 | 82.21%
DOI
10.1145/2882903.2915229

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{shkapsky_sigmod16,
        title = {{Big Data Analytics with Datalog Queries on Spark}},
        author = {Shkapsky, Alexander and Yang, Mohan and Interlandi, Matteo and Chiu, Hsuan and Condie, Tyson and Zaniolo, Carlo},
        series = {{SIGMOD} '16},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2882903.2915229},
        url = {https://dl.acm.org/doi/10.1145/2882903.2915229},
        year = {2016}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
2,935 RaSQL: Greater Power and Performance for Big Data Analytics with Recursive-aggregate-SQL on Spark 2019 SIGMOD 7.9444531e-05
3,872 All-in-One: Graph Processing in RDBMSs Revisited 2017 SIGMOD 7.0580243e-05
3,874 Tensors: An abstraction for general data processing 2021 VLDB 7.0561161e-05
4,534 Shared Arrangements: practical inter-query sharing for streaming dataflows 2020 VLDB 6.6420049e-05
4,943 Datalog Unchained 2021 PODS 6.4332239e-05
4,972 On the Optimization of Recursive Relational Queries: Application to Graph Queries 2020 SIGMOD 6.4193234e-05
5,783 Scaling-Up In-Memory Datalog Processing: Observations and Techniques 2019 VLDB 6.0899186e-05
6,203 Automating Incremental and Asynchronous Evaluation for Recursive Aggregate Data Processing 2020 SIGMOD 5.9447217e-05
6,204 Complete Event Trend Detection in High-Rate Event Streams 2017 SIGMOD 5.9444229e-05
7,035 Optimizing Recursive Queries with Program Synthesis 2022 SIGMOD 5.7216112e-05
8,318 Optimizing Declarative Graph Queries at Large Scale 2019 SIGMOD 5.4548766e-05
8,858 Optimizing Parallel Recursive Datalog Evaluation on Multicore Machines 2022 SIGMOD 5.3567857e-05
8,982 Automatic Index Selection for Large-Scale Datalog Computation 2019 VLDB 5.3397715e-05
9,478 Parallel Query Processing: To Separate Communication from Computation 2022 SIGMOD 5.2634238e-05
9,959 Datalog with First-Class Facts 2025 VLDB 5.1879626e-05
9,960 Optimizing Nested Recursive Queries 2024 SIGMOD 5.1879626e-05
10,582 FlowLog: Efficient and Extensible Datalog via Incrementality 2026 VLDB 5.093636e-05
10,690 Dynamic Pruning for Recursive Joins 2025 SIGMOD 5.093636e-05
11,261 Efficient Enumeration of Recursive Plans in Transformation-based Query Optimizers 2024 VLDB 5.093636e-05
11,336 The Vadalog Parallel System: Distributed Reasoning with Datalog+/- 2024 VLDB 5.093636e-05
11,360 Templating Shuffles 2023 CIDR 5.093636e-05
11,539 Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications 2022 SIGMOD 5.093636e-05
11,842 Ariadne: Online Provenance for Big Graph Analytics 2019 SIGMOD 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012250108
20 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.00056944564
24 Spark SQL: Relational Data Processing in Spark 2015 SIGMOD 0.00054865648
32 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00050111008
343 A Message Passing Framework for Logical Query Evaluation 1986 SIGMOD 0.00020669253
642 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00015395331
972 Declarative Networking: Language, Execution and Optimization 2006 SIGMOD 0.00012874501
1,539 Distributed SociaLite: A Datalog-Based Language for Large-Scale Graph Analysis 2013 VLDB 0.000104329
1,623 Relational Transducers for Declarative Networking 2011 PODS 0.0001020106
2,196 Spinning Fast Iterative Data Flows 2012 VLDB 8.9704984e-05
2,247 A Framework for the Parallel Processing of Datalog Queries 1990 SIGMOD 8.8713691e-05
2,454 REX: Recursive, Delta-Based Data-Centric Computation 2012 VLDB 8.5560058e-05
3,076 A New Paradigm For Parallel And Distributed Rule-Processing 1990 SIGMOD 7.7817058e-05
3,097 Evita Raced: Metacompilation for Declarative Networks 2008 VLDB 7.7619658e-05
4,002 Monotonic Aggregation in Deductive Databases 1992 PODS 6.9661579e-05
4,078 Asynchronous and Fault-Tolerant Recursive Datalog Evaluation in Shared-Nothing Engines 2015 VLDB 6.9209348e-05
4,121 Distributed Processing Of Logic Programs 1988 SIGMOD 6.8889854e-05
4,464 Why A Single Parallelization Strategy Is Not Enough In Knowledge Bases 1989 PODS 6.6860087e-05
4,745 Graph Queries in a Next-Generation Datalog System 2013 VLDB 6.526185e-05
5,850 Parallelizing Datalog Programs by Generalized Pivoting 1991 PODS 6.0664613e-05
9,226 Collaborative Access Control in WebdamLog 2015 SIGMOD 5.3027519e-05
Previous Page 1 / 1 Next

Semantically Similar Papers