Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques

Summary: C-SQL: a formal SQL-like language to express cluster placement as constraint optimization. Using query-optimizer techniques (incremental view maintenance, predicate pushdown) scales DCM to 10^4 nodes, yielding ~3000× DB latency speedup and up to 300× reduction in problem size (Kubernetes scheduler case study). (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 13295
Venue: VLDB
Year: 2023
Pagerank: 5.4842672e-05
Overall Rank: 8,115 | 44.33%
DOI: 10.14778/3603581.3603599

Incoming Non-self Citations Over Time

Authors

1. Kexin Rong (Georgia Institute of Technology; VMware)
2. Mihai Budiu (Feldera)
3. Athinagoras Skiadopoulos (Stanford University)
4. Lalith Suresh (Feldera)
5. Amy Tai (Google)

BibTeX Citation

@article{rong_vldb23,
        title = {{Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques}},
        author = {Rong, Kexin and Budiu, Mihai and Skiadopoulos, Athinagoras and Suresh, Lalith and Tai, Amy},
        journal = {PVLDB},
        series = {{VLDB} '23},
        volume = {16},
        number = {10},
        pages = {2618--2631},
        doi = {10.14778/3603581.3603599},
        url = {https://doi.org/10.14778/3603581.3603599},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
5,355	Solver-In-The-Loop Cluster Resource Management for Database-as-a-Service	2023	VLDB	6.2500992e-05

Outgoing Citations (Sorted by Pagerank)

Showing 19 of 19 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
61	Maintaining Views Incrementally	1993	SIGMOD	0.00039026867
132	Predicate Migration: Optimizing Queries with Expensive Predicates	1993	SIGMOD	0.00030378624
284	NoScope: Optimizing Neural Network Queries over Video at Scale	2017	VLDB	0.00022370521
295	Accelerating Machine Learning Inference with Probabilistic Predicates	2018	SIGMOD	0.00022238183
375	Incremental Maintenance of Views with Duplicates	1995	SIGMOD	0.00019681204
391	View Maintenance in a Warehousing Environment	1995	SIGMOD	0.00019370653
438	DBToaster: Higher-order Delta Processing for Dynamic, Frequently Fresh Views	2012	VLDB	0.00018471721
455	Differential dataflow	2013	CIDR	0.00018133241
929	Materialized View Selection and Maintenance Using Multi-Query Optimization	2001	SIGMOD	0.00013152103
1,037	Cost-Based Optimization for Magic: Algebra and Implementation	1996	SIGMOD	0.00012494928
1,124	Query Optimization by Predicate Move-Around	1994	VLDB	0.00012087356
1,133	Algorithms for Deferred View Maintenance	1996	SIGMOD	0.00012039326
1,750	Lazy Maintenance of Materialized Views	2007	VLDB	9.8392477e-05
2,164	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	9.0521951e-05
3,549	DBOS: A DBMS-oriented Operating System	2022	VLDB	7.3212765e-05
3,650	Implementation of Magic-sets in a Relational Database System	1994	SIGMOD	7.2249961e-05
3,847	Operator Placement for In-Network Stream Query Processing	2005	PODS	7.0758352e-05
7,384	CONSTRAINT QUERY LANGUAGES (preliminary report)	1990	PODS	5.6282573e-05
7,870	Optimizing Iceberg Queries with Complex Joins	2017	SIGMOD	5.5272726e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	7,484	On Scale Independence for Querying Big Data	2014	PODS
2	10,969	Improving DBMS Scheduling Decisions with Accurate Performance Prediction on Concurrent Queries	2025	VLDB
3	10,694	Managed Resource Scaling in Amazon EMR	2025	SIGMOD
4	7,585	Query Centric Partitioning and Allocation for Partially Replicated Database Systems	2017	SIGMOD
5	1,092	Automated Demand-driven Resource Scaling in Relational Database-as-a-Service	2016	SIGMOD
6	5,935	Towards Energy-Efficient Database Cluster Design	2012	VLDB
7	4,481	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD
8	8,887	A Model for Query Execution Over Heterogeneous Instances	2024	CIDR
9	9,446	Parallelizing Query Optimization on Shared-Nothing Architectures	2016	VLDB
10	4,742	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB