Run-Time Operator State Spilling for Memory Intensive Long-Running Queries

Summary: Proposes run-time state spilling for memory-heavy, long-running queries with multiple stateful operators; shows inter-operator dependencies necessitating plan-level spill strategies. Introduces bottom-up operator-level spilling and partition-level strategies (local/global output, with penalty), implemented in D-CAPE; experiments favor global-output approaches. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 3765
Venue: SIGMOD
Year: 2006
Pagerank: 5.7321615e-05
Overall Rank: 5,047 | 64.93%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,226	Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management	2013	SIGMOD	0.00013168869
2,339	Samza: Stateful Scalable Stream Processing at LinkedIn	2017	VLDB	9.0029559e-05
4,152	Query Suspend and Resume	2007	SIGMOD	6.4009383e-05
6,871	State-Slice: New Paradigm of Multi-query Optimization of Window-based Stream Queries	2006	VLDB	4.8966771e-05
6,880	E-Cube: Multi-Dimensional Event Sequence Analysis Using Hierarchical Pattern Query Sharing	2011	SIGMOD	4.8934688e-05

Outgoing Citations (Sorted by Pagerank)

Showing 16 of 16 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
41	NiagaraCQ: A Scalable Continuous Query System for Internet Databases	2000	SIGMOD	0.00073944756
43	Models and Issues in Data Stream Systems	2002	PODS	0.00072660894
113	Encapsulation of Parallelism in the Volcano Query Processing System	1990	SIGMOD	0.0004673401
191	The Design of the Borealis Stream Processing Engine	2005	CIDR	0.00035714897
244	Continuously Adaptive Continuous Queries over Streams	2002	SIGMOD	0.00031037999
548	Practical Skew Handling in Parallel Joins	1992	VLDB	0.00020369531
723	Load Shedding in a Data Stream Manager	2003	VLDB	0.00017493042
984	Streaming Queries over Streaming Data	2002	VLDB	0.00014840586
1,063	Tradeoffs in Processing Complex Join Queries via Hashing in Multiprocessor Database Machines	1990	VLDB	0.00014352292
1,136	Chain: Operator Scheduling for Memory Minimization in Data Stream Systems	2003	SIGMOD	0.00013745517
1,233	Maximizing the Output Rate of Multi-Way Join Queries over Streaming Information Sources	2003	VLDB	0.0001312635
1,343	Scalable Distributed Stream Processing	2003	CIDR	0.00012478318
1,851	Using Segmented Right-Deep Trees for the Execution of Pipelined Hash Joins	1992	VLDB	0.00010326646
3,038	How To Roll a Join: Asynchronous Incremental View Maintenance	2000	SIGMOD	7.6712282e-05
5,440	Shrinking the Warehouse Update Window	1999	SIGMOD	5.5025519e-05
9,083	A Dynamically Adaptive Distributed System for Processing Complex Continuous Queries	2005	VLDB	4.3951708e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
6,336	Dependency-Aware Reordering for Parallelizing Query Optimization in Multi-Core CPUs	2009	SIGMOD	5.1009488e-05
1,904	Characterizing Memory Requirements for Queries over Continuous Data Streams	2002	PODS	0.00010144623
5,044	Massive Scale-out of Expensive Continuous Queries	2011	VLDB	5.7338982e-05
9,466	Memory Efficient Scheduling of Query Pipeline Execution	2022	CIDR	4.3314303e-05
10,992	High-Performance Query Processing with NVMe Arrays: Spilling without Killing Performance	2024	SIGMOD	4.1905499e-05
4,934	Static Optimization of Conjunctive Queries with Sliding Windows Over Infinite Streams	2004	SIGMOD	5.815192e-05
6,871	State-Slice: New Paradigm of Multi-query Optimization of Window-based Stream Queries	2006	VLDB	4.8966771e-05
3,770	Dynamic Plan Migration for Continuous Queries Over Data Streams	2004	SIGMOD	6.7743365e-05
1,136	Chain: Operator Scheduling for Memory Minimization in Data Stream Systems	2003	SIGMOD	0.00013745517
9,083	A Dynamically Adaptive Distributed System for Processing Complex Continuous Queries	2005	VLDB	4.3951708e-05