DBScholar

Back to papers

Big Data Analytics with Datalog Queries on Spark

Summary: BigDatalog enables concise declarative Datalog queries for large-scale analytics on Spark. It uses compilation and optimization to efficiently support recursion on Spark, with empirical comparisons against top Datalog systems showing Spark-based analytics viable. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
hef9214a62ff7e7d3
Venue
SIGMOD
Year
2016
Pagerank
8.1655777e-05
Overall Rank
2,644 | 82.28%
DOI
10.1145/2882903.2915229

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{shkapsky_sigmod16,
        title = {{Big Data Analytics with Datalog Queries on Spark}},
        author = {Shkapsky, Alexander and Yang, Mohan and Interlandi, Matteo and Chiu, Hsuan and Condie, Tyson and Zaniolo, Carlo},
        series = {{SIGMOD} '16},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2882903.2915229},
        url = {https://dl.acm.org/doi/10.1145/2882903.2915229},
        year = {2016}
}

Incoming Citations (Sorted by Pagerank)

Showing 23 of 23 citing papers.

Rank Citing Paper Year Venue Pagerank
2,953 RaSQL: Greater Power and Performance for Big Data Analytics with Recursive-aggregate-SQL on Spark 2019 SIGMOD 7.8060043e-05
3,805 All-in-One: Graph Processing in RDBMSs Revisited 2017 SIGMOD 6.99039e-05
3,969 Tensors: An abstraction for general data processing 2021 VLDB 6.8769925e-05
4,422 Shared Arrangements: practical inter-query sharing for streaming dataflows 2020 VLDB 6.5902096e-05
5,073 On the Optimization of Recursive Relational Queries: Application to Graph Queries 2020 SIGMOD 6.2738154e-05
5,089 Datalog Unchained 2021 PODS 6.2661323e-05
5,881 Scaling-Up In-Memory Datalog Processing: Observations and Techniques 2019 VLDB 5.94241e-05
6,325 Automating Incremental and Asynchronous Evaluation for Recursive Aggregate Data Processing 2020 SIGMOD 5.7957351e-05
6,346 Complete Event Trend Detection in High-Rate Event Streams 2017 SIGMOD 5.7896994e-05
7,167 Optimizing Recursive Queries with Program Synthesis 2022 SIGMOD 5.5794956e-05
8,376 Optimizing Nested Recursive Queries 2024 SIGMOD 5.3279812e-05
8,508 Optimizing Declarative Graph Queries at Large Scale 2019 SIGMOD 5.3124867e-05
9,047 Optimizing Parallel Recursive Datalog Evaluation on Multicore Machines 2022 SIGMOD 5.2168077e-05
9,144 Automatic Index Selection for Large-Scale Datalog Computation 2019 VLDB 5.2072761e-05
9,695 Parallel Query Processing: To Separate Communication from Computation 2022 SIGMOD 5.1258854e-05
10,188 Datalog with First-Class Facts 2025 VLDB 5.052396e-05
10,235 Ariadne: Online Provenance for Big Graph Analytics 2019 SIGMOD 5.0432098e-05
10,368 Efficient Enumeration of Recursive Plans in Transformation-based Query Optimizers 2024 VLDB 5.0064651e-05
10,954 FlowLog: Efficient and Extensible Datalog via Incrementality 2026 VLDB 4.9605343e-05
11,195 Dynamic Pruning for Recursive Joins 2025 SIGMOD 4.9605343e-05
11,707 The Vadalog Parallel System: Distributed Reasoning with Datalog+/- 2024 VLDB 4.9605343e-05
11,728 Templating Shuffles 2023 CIDR 4.9605343e-05
11,898 Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications 2022 SIGMOD 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
3 Pregel: A System for Large-Scale Graph Processing 2010 SIGMOD 0.0012089097
22 Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud 2012 VLDB 0.00056028465
23 Spark SQL: Relational Data Processing in Spark 2015 SIGMOD 0.00055523999
31 Hive - A Warehousing Solution Over a Map-Reduce Framework 2009 VLDB 0.00049707539
351 A Message Passing Framework for Logical Query Evaluation 1986 SIGMOD 0.00020181666
655 Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience 2009 VLDB 0.00015076669
1,002 Declarative Networking: Language, Execution and Optimization 2006 SIGMOD 0.00012560011
1,552 Distributed SociaLite: A Datalog-Based Language for Large-Scale Graph Analysis 2013 VLDB 0.00010252208
1,635 Relational Transducers for Declarative Networking 2011 PODS 9.9989545e-05
2,240 Spinning Fast Iterative Data Flows 2012 VLDB 8.7716808e-05
2,308 A Framework for the Parallel Processing of Datalog Queries 1990 SIGMOD 8.6514834e-05
2,491 REX: Recursive, Delta-Based Data-Centric Computation 2012 VLDB 8.3733589e-05
3,147 A New Paradigm For Parallel And Distributed Rule-Processing 1990 SIGMOD 7.5793335e-05
3,153 Evita Raced: Metacompilation for Declarative Networks 2008 VLDB 7.5712705e-05
4,102 Monotonic Aggregation in Deductive Databases 1992 PODS 6.7929603e-05
4,175 Asynchronous and Fault-Tolerant Recursive Datalog Evaluation in Shared-Nothing Engines 2015 VLDB 6.7506691e-05
4,224 Distributed Processing Of Logic Programs 1988 SIGMOD 6.710075e-05
4,575 Why A Single Parallelization Strategy Is Not Enough In Knowledge Bases 1989 PODS 6.5132348e-05
4,858 Graph Queries in a Next-Generation Datalog System 2013 VLDB 6.3650212e-05
5,971 Parallelizing Datalog Programs by Generalized Pivoting 1991 PODS 5.9115526e-05
9,436 Collaborative Access Control in WebdamLog 2015 SIGMOD 5.1646737e-05
Previous Page 1 / 1 Next

Semantically Similar Papers