Incorporating Super-Operators in Big-Data Query Optimizers

Summary: Fuses shuffle-inducing Join, UnionAll, Spool, and GroupBy subqueries into parametric streaming super-operators, targeting dominant big-data costs. Abstract operator trees and matching avoid rule explosion; SCOPE reduces resource cost 1.7× and latency 1.5× without slower optimization. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 12418
Venue: VLDB
Year: 2020
Pagerank: 5.9470844e-05
Overall Rank: 6,194 | 57.51%
DOI: 10.14778/3368289.3368299

Incoming Non-self Citations Over Time

Authors

1. Jyoti Leeka (Microsoft)
2. Kaushik Rajan (Microsoft)

BibTeX Citation

@article{leeka_vldb20,
        title = {{Incorporating Super-Operators in Big-Data Query Optimizers}},
        author = {Leeka, Jyoti and Rajan, Kaushik},
        journal = {PVLDB},
        series = {{VLDB} '20},
        volume = {13},
        number = {3},
        pages = {348--360},
        doi = {10.14778/3368289.3368299},
        url = {https://doi.org/10.14778/3368289.3368299},
        year = {2020}
}

Incoming Citations (Sorted by Pagerank)

Showing 10 of 10 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,603	The Art of Balance: A RateupDB Experience of Building a CPU/GPU Hybrid Database Product	2021	VLDB	6.6105578e-05
6,121	The Cosmos Big Data Platform at Microsoft: Over a Decade of Progress and a Decade to Look Forward	2021	VLDB	5.9688569e-05
7,899	Modularis: Modular Relational Analytics over Heterogeneous Distributed Platforms	2021	VLDB	5.5195553e-05
8,164	SlabCity: Whole-Query Optimization using Program Synthesis	2023	VLDB	5.4750309e-05
8,465	Predicate Pushdown for Data Science Pipelines	2023	SIGMOD	5.4194578e-05
9,224	Phoebe: A Learning-based Checkpoint Optimizer	2021	VLDB	5.3035811e-05
9,478	Parallel Query Processing: To Separate Communication from Computation	2022	SIGMOD	5.2634238e-05
9,607	Sharing Opportunities for OLTP Workloads in Different Isolation Levels	2020	VLDB	5.2473912e-05
10,139	Leveraging Query Optimizers to Verify the Soundness of LLM-based Query Rewrites for Real-World Workloads, and More!	2026	CIDR	5.093636e-05
11,120	Welding Natural Language Queries to Analytics IRs with LLMs	2024	CIDR	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
23	Efficiently Compiling Efficient Query Plans for Modern Hardware	2011	VLDB	0.00054886415
24	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00054865648
32	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00050111008
237	Amazon Redshift and the Case for Simpler Data Warehouses	2015	SIGMOD	0.0002369895
534	Building Efficient Query Engines in a High-Level Language	2014	VLDB	0.00017046514
543	Orthogonal Optimization of Subqueries and Aggregation	2001	SIGMOD	0.0001682612
2,021	WinMagic : Subquery Elimination Using Window Aggregation	2003	SIGMOD	9.297379e-05
2,385	Groupwise Processing of Relational Queries	1997	VLDB	8.6492848e-05
2,477	Azure Data Lake Store: A Hyperscale Distributed File Service for Big Data Analytics	2017	SIGMOD	8.5239378e-05
2,769	A Layered Aggregate Engine for Analytics Workloads	2019	SIGMOD	8.1465406e-05
3,964	Hyper Dimension Shuffle: Efficient Data Repartition at Petabyte Scale in SCOPE	2019	VLDB	6.9855158e-05
3,970	Counting, Enumerating, and Sampling of Execution Plans in a Cost-Based Query Optimizer	2000	SIGMOD	6.9835426e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,022	A Scalable, Predictable Join Operator for Highly Concurrent Data Warehouses	2009	VLDB
2	10,296	Succinct Structure Representations for Efficient Query Optimization	2026	SIGMOD
3	1,154	Efficient Exploitation of Similar Subexpressions for Query Processing	2007	SIGMOD
4	8,439	New Query Optimization Techniques in the Spark Engine of Azure Synapse	2022	VLDB
5	2,822	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD
6	4,742	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB
7	4,481	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD
8	2,164	Opening the Black Boxes in Data Flow Optimization	2012	VLDB
9	3,964	Hyper Dimension Shuffle: Efficient Data Repartition at Petabyte Scale in SCOPE	2019	VLDB
10	5,059	Steering Query Optimizers: A Practical Take on Big Data Workloads	2021	SIGMOD