Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques

Summary: C-SQL: a formal SQL-like language to express cluster placement as constraint optimization. Using query-optimizer techniques (incremental view maintenance, predicate pushdown) scales DCM to 10^4 nodes, yielding ~3000× DB latency speedup and up to 300× reduction in problem size (Kubernetes scheduler case study). (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: h5432d9ddb4fe670a
Venue: VLDB
Year: 2023
Pagerank: 5.3770961e-05
Overall Rank: 8,213 | 44.79%
DOI: 10.14778/3603581.3603599

Incoming Non-self Citations Over Time

Authors

1. Kexin Rong (Georgia Institute of Technology; VMware)
2. Mihai Budiu (Feldera)
3. Athinagoras Skiadopoulos (Stanford University)
4. Lalith Suresh (Feldera)
5. Amy Tai (Google)

BibTeX Citation

@article{rong_vldb23,
        title = {{Scaling a Declarative Cluster Manager Architecture with Query Optimization Techniques}},
        author = {Rong, Kexin and Budiu, Mihai and Skiadopoulos, Athinagoras and Suresh, Lalith and Tai, Amy},
        journal = {PVLDB},
        series = {{VLDB} '23},
        volume = {16},
        number = {10},
        pages = {2618--2631},
        doi = {10.14778/3603581.3603599},
        url = {https://doi.org/10.14778/3603581.3603599},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,902	Solver-In-The-Loop Cluster Resource Management for Database-as-a-Service	2023	VLDB	6.3639631e-05

Outgoing Citations (Sorted by Pagerank)

Showing 19 of 19 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
62	Maintaining Views Incrementally	1993	SIGMOD	0.00039045511
125	Predicate Migration: Optimizing Queries with Expensive Predicates	1993	SIGMOD	0.00030470848
271	NoScope: Optimizing Neural Network Queries over Video at Scale	2017	VLDB	0.00022560564
281	Accelerating Machine Learning Inference with Probabilistic Predicates	2018	SIGMOD	0.00022295232
388	Incremental Maintenance of Views with Duplicates	1995	SIGMOD	0.00019350381
398	View Maintenance in a Warehousing Environment	1995	SIGMOD	0.00019152162
408	DBToaster: Higher-order Delta Processing for Dynamic, Frequently Fresh Views	2012	VLDB	0.00018900199
442	Differential dataflow	2013	CIDR	0.00018210463
923	Materialized View Selection and Maintenance Using Multi-Query Optimization	2001	SIGMOD	0.00013064753
1,032	Cost-Based Optimization for Magic: Algebra and Implementation	1996	SIGMOD	0.00012401489
1,119	Query Optimization by Predicate Move-Around	1994	VLDB	0.00011950395
1,150	Algorithms for Deferred View Maintenance	1996	SIGMOD	0.00011809311
1,779	Lazy Maintenance of Materialized Views	2007	VLDB	9.6583009e-05
2,195	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	8.8781177e-05
3,463	DBOS: A DBMS-oriented Operating System	2022	VLDB	7.2821994e-05
3,707	Implementation of Magic-sets in a Relational Database System	1994	SIGMOD	7.0820673e-05
3,933	Operator Placement for In-Network Stream Query Processing	2005	PODS	6.9176947e-05
7,525	CONSTRAINT QUERY LANGUAGES (preliminary report)	1990	PODS	5.5024265e-05
8,023	Optimizing Iceberg Queries with Complex Joins	2017	SIGMOD	5.4054487e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	7,610	On Scale Independence for Querying Big Data	2014	PODS
2	10,148	Improving DBMS Scheduling Decisions with Accurate Performance Prediction on Concurrent Queries	2025	VLDB
3	11,128	Managed Resource Scaling in Amazon EMR	2025	SIGMOD
4	7,706	Query Centric Partitioning and Allocation for Partially Replicated Database Systems	2017	SIGMOD
5	1,067	Automated Demand-driven Resource Scaling in Relational Database-as-a-Service	2016	SIGMOD
6	6,057	Towards Energy-Efficient Database Cluster Design	2012	VLDB
7	4,069	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD
8	9,045	A Model for Query Execution Over Heterogeneous Instances	2024	CIDR
9	9,621	Parallelizing Query Optimization on Shared-Nothing Architectures	2016	VLDB
10	4,818	Continuous Cloud-Scale Query Optimization and Processing	2013	VLDB