QaaD (Query-as-a-Data): Scalable Execution of Massive Number of Small Queries in Spark

Summary: Query-merging via microRDD converts many small Spark queries into a few larger ones; queries embedded as data enable shared inputs. Dynamic partition sizing minimizes runtime overhead, yielding 10.6x-36.6x speedups over Spark for small queries. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6638
Venue: SIGMOD
Year: 2023
Pagerank: 4.1905499e-05
Overall Rank: 11,199 | 22.17%
DOI: 10.1145/3589279

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
3	Pig Latin: A Not-So-Foreign Language for Data Processing	2008	SIGMOD	0.0024217964
981	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00014866268
1,921	Selecting Subexpressions to Materialize at Datacenter Scale	2018	VLDB	0.00010085899
1,939	Procella: Unifying serving and analytical data at YouTube	2019	VLDB	0.00010019433
3,270	RHEEM: Enabling Cross-Platform Data Processing - May The Big Data Be With You! -	2018	VLDB	7.3013196e-05
3,966	Apache Hive: From MapReduce to Enterprise-grade Big Data Warehousing	2019	SIGMOD	6.5782437e-05
5,116	AdaptDB: Adaptive Partitioning for Distributed Joins	2017	VLDB	5.6805476e-05
5,682	LSched: A Workload-Aware Learned Query Scheduler for Analytical Database Systems	2022	SIGMOD	5.3752251e-05
7,459	Lachesis: Automatic Partitioning for UDF-Centric Analytics	2021	VLDB	4.7199075e-05
7,461	Scalable Multi-Query Execution using Reinforcement Learning	2021	SIGMOD	4.7193668e-05
8,196	SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft	2021	VLDB	4.5568952e-05
8,460	Piranha: Optimizing Short Jobs in Hadoop	2013	VLDB	4.5008938e-05
9,335	Parallel Query Processing: To Separate Communication from Computation	2022	SIGMOD	4.351469e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
9,505	Supporting Scalable Analytics with Latency Constraints	2015	VLDB	4.3300131e-05
11,951	A Demonstration of AQWA: Adaptive Query-Workload-Aware Partitioning of Big Spatial Data	2015	VLDB	4.1905499e-05
8,196	SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft	2021	VLDB	4.5568952e-05
3,536	Scaling Spark in the Real World: Performance and Usability	2015	VLDB	6.9938207e-05
9,517	[Demo] Low-latency Spark Queries on Updatable Data	2019	SIGMOD	4.3294347e-05
6,683	Adaptive and Robust Query Execution for Lakehouses at Scale	2024	VLDB	4.9593505e-05
7,904	S2RDF: RDF Querying with SPARQL on Spark	2016	VLDB	4.616742e-05
8,502	New Query Optimization Techniques in the Spark Engine of Azure Synapse	2022	VLDB	4.491819e-05
8,585	A Spark Optimizer for Adaptive, Fine-Grained Parameter Tuning	2024	VLDB	4.4856045e-05
3,207	Big Data Analytics with Datalog Queries on Spark	2016	SIGMOD	7.3847098e-05