Pipelining in Multi-Query Optimization

Summary: Introduce a general model for pipelined multi-query optimization and a necessary and sufficient condition to decide schedule validity under finite buffer constraints. Prove NP-hardness of finding minimum-cost valid schedules, propose a greedy heuristic, and show TPC-D performance gains. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: h2def42a55ead9ef3
Venue: PODS
Year: 2001
Pagerank: 0.00013541546
Overall Rank: 833 | 94.42%
DOI: 10.1145/375551.375561

Incoming Non-self Citations Over Time

Authors

1. Nilesh N. Dalvi (Indian Institute of Technology Mumbai)
2. Sumit K Sanghai (Indian Institute of Technology Mumbai)
3. Prasan Roy (AT&T)
4. S. Sudarshan (Indian Institute of Technology Mumbai)

BibTeX Citation

@inproceedings{dalvi_pods01,
        address = {New York, NY, USA},
        series = {{PODS} '01},
        title = {{Pipelining in Multi-Query Optimization}},
        url = {https://dl.acm.org/doi/10.1145/375551.375561},
        doi = {10.1145/375551.375561},
        booktitle = {Proceedings of the {ACM} {SIGMOD} Symposium on {Principles} of {Database} {Systems}},
        publisher = {Association for Computing Machinery},
        author = {Dalvi, Nilesh N. and Sanghai, Sumit K and Roy, Prasan and Sudarshan, S.},
        year = {2001}
}

Incoming Citations (Sorted by Pagerank)

Showing 26 of 26 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
112	TelegraphCQ: Continuous Dataflow Processing for an Uncertain World	2003	CIDR	0.00032341214
390	QPipe: A Simultaneously Pipelined Relational Query Engine	2005	SIGMOD	0.00019232929
670	SharedDB: Killing One Thousand Queries With One Stone	2012	VLDB	0.00014929358
768	Cooperative Scans: Dynamic Bandwidth Sharing in a DBMS	2007	VLDB	0.00014096379
2,237	Shared Workload Optimization	2014	VLDB	8.7777395e-05
2,701	On Improving User Response Times in Tableau	2015	SIGMOD	8.0959949e-05
2,781	Multiple Query Optimization on the D-Wave 2X Adiabatic Quantum Computer	2016	VLDB	8.0065695e-05
3,226	Efficient and Provable Multi-Query Optimization	2017	PODS	7.4987658e-05
3,361	Exact Cardinality Query Optimization for Optimizer Testing	2009	VLDB	7.3621595e-05
4,013	An Approach to Optimize Data Processing in Business Processes	2007	VLDB	6.8408682e-05
4,082	The Case for Precision Sharing	2004	VLDB	6.8010384e-05
4,411	Sharing Data and Work Across Concurrent Analytical Queries	2013	VLDB	6.5996653e-05
4,506	Optimization of Continuous Queries with Shared Expensive Filters	2007	PODS	6.5558725e-05
4,594	To Share or Not to Share?	2007	VLDB	6.4994689e-05
5,598	Efficient Computation of Multiple Group By Queries	2005	SIGMOD	6.053914e-05
5,794	Lemo: A Cache-Enhanced Learned Optimizer for Concurrent Queries	2023	SIGMOD	5.9759435e-05
6,088	QoX-Driven ETL Design: Reducing the Cost of ETL Consulting Engagements	2009	SIGMOD	5.8739792e-05
6,234	Materialization and Reuse Optimizations for Production Data Science Pipelines	2022	SIGMOD	5.825343e-05
7,042	Near-Optimal Algorithms for Shared Filter Evaluation in Data Stream Systems	2008	SIGMOD	5.597835e-05
7,347	Scalable Multi-Query Execution using Reinforcement Learning	2021	SIGMOD	5.5308106e-05
7,833	Nautilus: An Optimized System for Deep Transfer Learning over Evolving Training Datasets	2022	SIGMOD	5.4285082e-05
8,067	Adaptive Optimizations of Recursive Queries in Teradata	2012	SIGMOD	5.3810658e-05
8,998	Pipemizer: An Optimizer for Analytics Data Pipelines	2022	VLDB	5.2273528e-05
9,502	Optimizing Complex Queries with Multiple Relation Instances	2008	SIGMOD	5.1533173e-05
10,322	Request Window: an Approach to Improve Throughput of RDBMS-based Data Integration System by Utilizing Data Sharing Across Concurrent Distributed Queries	2007	VLDB	5.0255524e-05
11,603	Pasta: A Cost-Based Optimizer for Generating Pipelining Schedules for Dataflow DAGs	2024	SIGMOD	4.9605343e-05

Outgoing Citations (Sorted by Pagerank)

Showing 5 of 5 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
129	Efficient and Extensible Algorithms for Multi Query Optimization	2000	SIGMOD	0.00030314403
842	Exploiting Inter-Operation Parallelism in XPRS	1992	SIGMOD	0.00013501181
961	Materialized View Selection for Multidimensional Datasets	1998	VLDB	0.0001281898
1,410	Anatomy of a Modular Multiple Query Optimizer	1988	VLDB	0.00010730897
2,115	Scheduling Problems in Parallel Query Optimization	1995	PODS	9.008867e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	2,151	Optimization of Multi-Way Join Queries for Parallel Execution	1991	VLDB
2	6,063	Dependency-Aware Reordering for Parallelizing Query Optimization in Multi-Core CPUs	2009	SIGMOD
3	3,310	On Optimal Processor Allocation To Support Pipelined Hash Joins	1993	SIGMOD
4	3,226	Efficient and Provable Multi-Query Optimization	2017	PODS
5	2,309	Multi-dimensional Resource Scheduling for Parallel Queries	1996	SIGMOD
6	1,809	Dynamic Pipeline Scheduling for Improving Interactive Query Performance	2001	VLDB
7	6,302	Elastic Pipelining in an In-Memory Database Cluster	2016	SIGMOD
8	2,115	Scheduling Problems in Parallel Query Optimization	1995	PODS
9	2,182	Optimization Algorithms for Exploiting the Parallelism-Communication Tradeoff in Pipelined Parallelism	1994	VLDB
10	9,822	Memory Efficient Scheduling of Query Pipeline Execution	2022	CIDR