DBScholar

Back to papers

Big Data Analytics with Datalog Queries on Spark

Summary: BigDatalog enables concise declarative Datalog queries for large-scale analytics on Spark. It uses compilation and optimization to efficiently support recursion on Spark, with empirical comparisons against top Datalog systems showing Spark-based analytics viable. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
hef9214a62ff7e7d3
Venue
SIGMOD
Year
2016
Pagerank
8.1926426e-05
Overall Rank
2,636 | 82.29%
DOI
10.1145/2882903.2915229

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{shkapsky_sigmod16,
        title = {{Big Data Analytics with Datalog Queries on Spark}},
        author = {Shkapsky, Alexander and Yang, Mohan and Interlandi, Matteo and Chiu, Hsuan and Condie, Tyson and Zaniolo, Carlo},
        series = {{SIGMOD} '16},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2882903.2915229},
        url = {https://dl.acm.org/doi/10.1145/2882903.2915229},
        year = {2016}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
2,939 RaSQL: Greater Power and Performance for Big Data Analytics with Recursive-aggregate-SQL on Spark 2019 SIGMOD 7.8318902e-05
3,801 All-in-One: Graph Processing in RDBMSs Revisited 2017 SIGMOD 7.0128676e-05
3,955 Tensors: An abstraction for general data processing 2021 VLDB 6.8994712e-05
4,402 Shared Arrangements: practical inter-query sharing for streaming dataflows 2020 VLDB 6.6120647e-05
5,054 On the Optimization of Recursive Relational Queries: Application to Graph Queries 2020 SIGMOD 6.2946294e-05
5,070 Datalog Unchained 2021 PODS 6.2869206e-05
5,868 Scaling-Up In-Memory Datalog Processing: Observations and Techniques 2019 VLDB 5.962124e-05
6,312 Automating Incremental and Asynchronous Evaluation for Recursive Aggregate Data Processing 2020 SIGMOD 5.8149275e-05
6,335 Complete Event Trend Detection in High-Rate Event Streams 2017 SIGMOD 5.8089072e-05
7,141 Optimizing Recursive Queries with Program Synthesis 2022 SIGMOD 5.5979985e-05
8,362 Optimizing Nested Recursive Queries 2024 SIGMOD 5.3456573e-05
8,493 Optimizing Declarative Graph Queries at Large Scale 2019 SIGMOD 5.3301105e-05
9,026 Optimizing Parallel Recursive Datalog Evaluation on Multicore Machines 2022 SIGMOD 5.2341149e-05
9,125 Automatic Index Selection for Large-Scale Datalog Computation 2019 VLDB 5.2245517e-05
9,666 Parallel Query Processing: To Separate Communication from Computation 2022 SIGMOD 5.142891e-05
10,153 Datalog with First-Class Facts 2025 VLDB 5.0691578e-05
10,202 Ariadne: Online Provenance for Big Graph Analytics 2019 SIGMOD 5.0599411e-05
10,341 Efficient Enumeration of Recursive Plans in Transformation-based Query Optimizers 2024 VLDB 5.0230745e-05
10,893 FlowLog: Efficient and Extensible Datalog via Incrementality 2026 VLDB 4.9769913e-05
11,135 Dynamic Pruning for Recursive Joins 2025 SIGMOD 4.9769913e-05
11,660 The Vadalog Parallel System: Distributed Reasoning with Datalog+/- 2024 VLDB 4.9769913e-05
11,682 Templating Shuffles 2023 CIDR 4.9769913e-05
11,854 Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications 2022 SIGMOD 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012087459
22 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.00055938421
23 Spark SQL: Relational Data Processing in Spark 2015 SIGMOD 0.00055384955
31 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00049821554
349 A Message Passing Framework for Logical Query Evaluation 1986 SIGMOD 0.00020245855
651 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00015123701
1,000 Declarative Networking: Language, Execution and Optimization 2006 SIGMOD 0.00012601156
1,550 Distributed SociaLite: A Datalog-Based Language for Large-Scale Graph Analysis 2013 VLDB 0.00010285521
1,630 Relational Transducers for Declarative Networking 2011 PODS 0.00010028813
2,228 Spinning Fast Iterative Data Flows 2012 VLDB 8.7996087e-05
2,297 A Framework for the Parallel Processing of Datalog Queries 1990 SIGMOD 8.6799663e-05
2,481 REX: Recursive, Delta-Based Data-Centric Computation 2012 VLDB 8.3995962e-05
3,137 A New Paradigm For Parallel And Distributed Rule-Processing 1990 SIGMOD 7.6044211e-05
3,142 Evita Raced: Metacompilation for Declarative Networks 2008 VLDB 7.595954e-05
4,079 Monotonic Aggregation in Deductive Databases 1992 PODS 6.8154895e-05
4,153 Asynchronous and Fault-Tolerant Recursive Datalog Evaluation in Shared-Nothing Engines 2015 VLDB 6.7730626e-05
4,206 Distributed Processing Of Logic Programs 1988 SIGMOD 6.7322742e-05
4,557 Why A Single Parallelization Strategy Is Not Enough In Knowledge Bases 1989 PODS 6.5347712e-05
4,836 Graph Queries in a Next-Generation Datalog System 2013 VLDB 6.3856256e-05
5,964 Parallelizing Datalog Programs by Generalized Pivoting 1991 PODS 5.9303286e-05
9,411 Collaborative Access Control in WebdamLog 2015 SIGMOD 5.1818074e-05
Previous Page 1 / 1 Next

Semantically Similar Papers