Incorporating Super-Operators in Big-Data Query Optimizers

Summary: Fuses shuffle-inducing Join, UnionAll, Spool, and GroupBy subqueries into parametric streaming super-operators, targeting dominant big-data costs. Abstract operator trees and matching avoid rule explosion; SCOPE reduces resource cost 1.7× and latency 1.5× without slower optimization. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h60d1097f25cf9060
Venue: VLDB
Year: 2020
Pagerank: 5.8223076e-05
Overall Rank: 6,290 | 57.72%
DOI: 10.14778/3368289.3368299

Incoming Non-self Citations Over Time

Authors

1. Jyoti Leeka (Microsoft)
2. Kaushik Rajan (Microsoft)

BibTeX Citation

@article{leeka_vldb20,
        title = {{Incorporating Super-Operators in Big-Data Query Optimizers}},
        author = {Leeka, Jyoti and Rajan, Kaushik},
        journal = {PVLDB},
        series = {{VLDB} '20},
        volume = {13},
        number = {3},
        pages = {348--360},
        doi = {10.14778/3368289.3368299},
        url = {https://doi.org/10.14778/3368289.3368299},
        year = {2020}
}

Incoming Citations (Sorted by Pagerank)

Showing 10 of 10 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,137	The Art of Balance: A RateupDB Experience of Building a CPU/GPU Hybrid Database Product	2021	VLDB	6.7861661e-05
6,245	The Cosmos Big Data Platform at Microsoft: Over a Decade of Progress and a Decade to Look Forward	2021	VLDB	5.837329e-05
7,451	Modularis: Modular Relational Analytics over Heterogeneous Distributed Platforms	2021	VLDB	5.5236802e-05
7,931	SlabCity: Whole-Query Optimization using Program Synthesis	2023	VLDB	5.4238328e-05
8,153	Predicate Pushdown for Data Science Pipelines	2023	SIGMOD	5.3891786e-05
9,358	Phoebe: A Learning-based Checkpoint Optimizer	2021	VLDB	5.1869771e-05
9,659	Parallel Query Processing: To Separate Communication from Computation	2022	SIGMOD	5.1453267e-05
9,782	Sharing Opportunities for OLTP Workloads in Different Isolation Levels	2020	VLDB	5.1296539e-05
10,359	Leveraging Query Optimizers to Verify the Soundness of LLM-based Query Rewrites for Real-World Workloads, and More!	2026	CIDR	4.9793485e-05
11,468	Welding Natural Language Queries to Analytics IRs with LLMs	2024	CIDR	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
21	Efficiently Compiling Efficient Query Plans for Modern Hardware	2011	VLDB	0.00056855599
23	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00055406774
31	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00049839909
233	Amazon Redshift and the Case for Simpler Data Warehouses	2015	SIGMOD	0.00023783585
495	Building Efficient Query Engines in a High-Level Language	2014	VLDB	0.00017370758
552	Orthogonal Optimization of Subqueries and Aggregation	2001	SIGMOD	0.0001653835
2,045	WinMagic : Subquery Elimination Using Window Aggregation	2003	SIGMOD	9.1329128e-05
2,438	Groupwise Processing of Relational Queries	1997	VLDB	8.4652613e-05
2,446	Azure Data Lake Store: A Hyperscale Distributed File Service for Big Data Analytics	2017	SIGMOD	8.4547121e-05
2,663	A Layered Aggregate Engine for Analytics Workloads	2019	SIGMOD	8.1581558e-05
3,895	Hyper Dimension Shuffle: Efficient Data Repartition at Petabyte Scale in SCOPE	2019	VLDB	6.9393157e-05
4,038	Counting, Enumerating, and Sampling of Execution Plans in a Cost-Based Query Optimizer	2000	SIGMOD	6.8389401e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,034	A Scalable, Predictable Join Operator for Highly Concurrent Data Warehouses	2009	VLDB
2	10,508	Succinct Structure Representations for Efficient Query Optimization	2026	SIGMOD
3	1,131	Efficient Exploitation of Similar Subexpressions for Query Processing	2007	SIGMOD
4	8,535	New Query Optimization Techniques in the Spark Engine of Azure Synapse	2022	VLDB
5	2,834	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD
6	4,818	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB
7	4,069	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD
8	2,195	Opening the Black Boxes in Data Flow Optimization	2012	VLDB
9	3,895	Hyper Dimension Shuffle: Efficient Data Repartition at Petabyte Scale in SCOPE	2019	VLDB
10	5,110	Steering Query Optimizers: A Practical Take on Big Data Workloads	2021	SIGMOD