QaaD (Query-as-a-Data): Scalable Execution of Massive Number of Small Queries in Spark

Summary: Query-merging via microRDD converts many small Spark queries into a few larger ones; queries embedded as data enable shared inputs. Dynamic partition sizing minimizes runtime overhead, yielding 10.6x-36.6x speedups over Spark for small queries. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6637
Venue: SIGMOD
Year: 2023
Pagerank: 4.1945683e-05
Overall Rank: 11,197 | 22.11%
DOI: 10.1145/3589279

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
3	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0024183614
979	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.0001488055
1,922	Selecting Subexpressions to Materialize at Datacenter Scale	2018	VLDB	0.00010082599
1,943	Procella: Unifying serving and analytical data at YouTube	2019	VLDB	0.00010012569
3,265	RHEEM: Enabling Cross-Platform Data Processing - May The Big Data Be With You! -	2018	VLDB	7.3083672e-05
3,973	Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing	2019	SIGMOD	6.5758017e-05
5,118	AdaptDB: Adaptive Partitioning for Distributed Joins	2017	VLDB	5.6820984e-05
5,671	LSched: A Workload-Aware Learned Query Scheduler for Analytical Database Systems	2022	SIGMOD	5.3803919e-05
7,461	Scalable Multi-Query Execution using Reinforcement Learning	2021	SIGMOD	4.723898e-05
7,476	Lachesis: Automatic Partitioning for UDF-Centric Analytics	2021	VLDB	4.7188928e-05
8,197	SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft	2021	VLDB	4.5607121e-05
8,464	Piranha: Optimizing Short Jobs in Hadoop	2013	VLDB	4.5052127e-05
9,330	Parallel Query Processing: To Separate Communication from Computation	2022	SIGMOD	4.3556432e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
9,504	Supporting Scalable Analytics with Latency Constraints	2015	VLDB	4.3341665e-05
11,943	A Demonstration of AQWA: Adaptive Query-Workload-Aware Partitioning of Big Spatial Data	2015	VLDB	4.1945683e-05
8,197	SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft	2021	VLDB	4.5607121e-05
3,535	Scaling Spark in the Real World: Performance and Usability	2015	VLDB	6.9992495e-05
9,516	[Demo] Low-latency Spark Queries on Updatable Data	2019	SIGMOD	4.3335877e-05
7,059	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	4.8477825e-05
7,905	S2RDF: RDF Querying with SPARQL on Spark	2016	VLDB	4.6211706e-05
8,617	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB	4.4846425e-05
8,506	New Query Optimization Techniques in the Spark Engine of Azure Synapse	2022	VLDB	4.4957661e-05
3,200	Big Data Analytics with Datalog Queries on Spark	2016	SIGMOD	7.3912411e-05