Selecting Subexpressions to Materialize at Datacenter Scale

Summary: Selects cross-job subexpressions to materialize at datacenter scale, formulating the problem as ILP/bipartite labeling. BIG SUBS, a distributed vertex-centric algorithm, handles tens of thousands of jobs and cuts machine-hours by up to 40% on SCOPE workloads. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: h6542a7c977054b41
Venue: VLDB
Year: 2018
Pagerank: 9.7050769e-05
Overall Rank: 1,757 | 88.23%
DOI: 10.14778/3192965.3192971

Incoming Non-self Citations Over Time

Authors

1. Alekh Jindal (Microsoft)
2. Konstantinos Karanasos (Microsoft)
3. Sriram Rao (Microsoft)
4. Hiren Patel (Microsoft)

BibTeX Citation

@article{jindal_vldb18,
        title = {{Selecting Subexpressions to Materialize at Datacenter Scale}},
        author = {Jindal, Alekh and Karanasos, Konstantinos and Rao, Sriram and Patel, Hiren},
        journal = {PVLDB},
        series = {{VLDB} '18},
        volume = {11},
        number = {7},
        pages = {800--812},
        doi = {10.14778/3192965.3192971},
        url = {https://doi.org/10.14778/3192965.3192971},
        year = {2018}
}

Incoming Citations (Sorted by Pagerank)

Showing 37 of 37 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,567	HELIX: Holistic Optimization for Accelerating Iterative Machine Learning	2019	VLDB	0.00010179013
2,792	EVA: A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized Views	2022	SIGMOD	7.9895218e-05
2,828	Cost Models for Big Data Query Processing: Learning, Retrofitting, and Our Findings	2020	SIGMOD	7.9476509e-05
3,158	GenRewrite: Query Rewriting via Large Language Models	2026	SIGMOD	7.5676704e-05
3,294	Automated Verification of Query Equivalence Using Satisfiability Modulo Theories	2019	VLDB	7.4297513e-05
3,687	openGauss: An Autonomous Database System	2021	VLDB	7.0797286e-05
4,022	Automated Generation of Materialized Views in Oracle	2020	VLDB	6.8345671e-05
4,354	LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems	2021	SIGMOD	6.6318545e-05
5,210	Crystal: A Unified Cache Storage System for Analytical Databases	2021	VLDB	6.2142008e-05
5,453	Eraser: Eliminating Performance Regression on Learned Query Optimizer	2024	VLDB	6.1077838e-05
5,793	Optimizing Data Pipelines for Machine Learning in Feature Stores	2023	VLDB	5.9761004e-05
6,260	The Cosmos Big Data Platform at Microsoft: Over a Decade of Progress and a Decade to Look Forward	2021	VLDB	5.815273e-05
6,768	Multi-Tenant Cloud Data Services: State-of-the-Art, Challenges and Opportunities	2022	SIGMOD	5.6693759e-05
6,807	Intermittent Query Processing	2019	VLDB	5.661199e-05
6,835	CrocodileDB: Efficient Database Execution through Intelligent Deferment	2020	CIDR	5.6527238e-05
6,977	Jigsaw: A Data Storage and Query Processing Engine for Irregular Table Partitioning	2021	SIGMOD	5.6135617e-05
7,162	Sibyl: Forecasting Time-Evolving Query Workloads	2024	SIGMOD	5.5798063e-05
7,347	Scalable Multi-Query Execution using Reinforcement Learning	2021	SIGMOD	5.5308106e-05
7,585	SIEVE: Effective Filtered Vector Search with Collection of Indexes	2025	VLDB	5.4747603e-05
7,621	SageDB: An Instance-Optimized Data Analytics System	2022	VLDB	5.4664683e-05
8,362	SparkCruise: Workload Optimization in Managed Spark Clusters at Microsoft	2021	VLDB	5.3308358e-05
8,539	New Query Optimization Techniques in the Spark Engine of Azure Synapse	2022	VLDB	5.3048225e-05
8,724	View Selection over Knowledge Graphs in Triple Stores	2021	VLDB	5.2705676e-05
8,992	GEqO: ML-Accelerated Semantic Equivalence Detection	2023	SIGMOD	5.2295579e-05
9,041	Optimizing the cloud? Don't train models. Build oracles!	2024	CIDR	5.2183053e-05
9,117	Hyperspace: The Indexing Subsystem of Azure Synapse	2021	VLDB	5.208561e-05
9,394	Phoebe: A Learning-based Checkpoint Optimizer	2021	VLDB	5.1676892e-05
9,559	UniView: A Unified Autonomous Materialized View Management System for Various Databases	2024	VLDB	5.1477909e-05
9,780	When sweet and cute isn't enough anymore: Solving scalability issues in Python Pandas with Grizzly	2020	CIDR	5.115551e-05
10,034	SparkCruise: Handsfree Computation Reuse in Spark	2019	VLDB	5.0795508e-05
10,195	Generating Application-Specific Data Layouts for In-memory Databases	2019	VLDB	5.0516135e-05
10,966	Benchmarking the Full Pipeline of Materialized-View-Based Query Rewriting	2026	VLDB	4.9605343e-05
11,490	Mayura: Exploiting Similarities in Motifs for Temporal Co-Mining	2025	VLDB	4.9605343e-05
11,519	Oligolithic Cross-task Optimizations across Isolated Workloads*	2024	CIDR	4.9605343e-05
11,766	QaaD (Query-as-a-Data): Scalable Execution of Massive Number of Small Queries in Spark	2023	SIGMOD	4.9605343e-05
11,898	Juggler: Autonomous Cost Optimization and Performance Prediction of Big Data Applications	2022	SIGMOD	4.9605343e-05
13,758	PikePlace: Generating Intelligence for Marketplace Datasets	2023	VLDB	-

Outgoing Citations (Sorted by Pagerank)

Showing 27 of 27 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
3	Pregel: A System for Large-Scale Graph Processing	2010	SIGMOD	0.0012089097
11	Implementing Data Cubes Efficiently	1996	SIGMOD	0.00070860346
25	NiagaraCQ: A Scalable Continuous Query System for Internet Databases	2000	SIGMOD	0.00053734242
31	Hive - A Warehousing Solution Over a Map-Reduce Framework	2009	VLDB	0.00049707539
89	Automated Selection of Materialized Views and Indexes for SQL Databases	2000	VLDB	0.00035265687
129	Efficient and Extensible Algorithms for Multi Query Optimization	2000	SIGMOD	0.00030314403
514	TelegraphCQ: Continuous Dataflow Processing	2003	SIGMOD	0.00017005839
707	Main-Memory Scan Sharing For Multi-Core CPUs	2008	VLDB	0.00014613324
768	Cooperative Scans: Dynamic Bandwidth Sharing in a DBMS	2007	VLDB	0.00014096379
784	The case against specialized graph analytics engines	2015	CIDR	0.00014001272
825	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00013600553
1,137	Efficient Exploitation of Similar Subexpressions for Query Processing	2007	SIGMOD	0.00011856159
1,550	Data Warehouse Configuration	1997	VLDB	0.00010257652
1,936	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.3375824e-05
2,092	An Architecture for Recycling Intermediates in a Column-store	2009	SIGMOD	9.0432353e-05
2,237	Shared Workload Optimization	2014	VLDB	8.7777395e-05
2,360	Vertexica: Your Relational Friend for Graph Analytics!	2014	VLDB	8.5628814e-05
2,452	Azure Data Lake Store: A Hyperscale Distributed File Service for Big Data Analytics	2017	SIGMOD	8.4280184e-05
3,226	Efficient and Provable Multi-Query Optimization	2017	PODS	7.4987658e-05
3,228	Multi-Query Optimization in MapReduce Framework	2014	VLDB	7.4957341e-05
3,550	Computation Reuse in Analytics Job Service at Microsoft	2018	SIGMOD	7.1895215e-05
4,492	Recurring Job Optimization in Scope	2012	SIGMOD	6.5612113e-05
5,113	ROBUS: Fair Cache Allocation for Data-parallel Workloads	2017	SIGMOD	6.2569879e-05
5,847	Opportunistic Physical Design for Big Data Analytics	2014	SIGMOD	5.9537345e-05
7,366	Kodiak: Leveraging Materialized Views For Very Low-Latency Analytics Over High-Dimensional Web-Scale Data	2016	VLDB	5.5268158e-05
8,335	View Selection in Semantic Web Databases	2012	VLDB	5.3369178e-05
9,186	Delta: Scalable Data Dissemination under Capacity Constraints	2014	VLDB	5.1979837e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	10,811	Characterizing Parallel Subgraph Matching Performance: A Systematic Study of Interactions, Scalability, and Enumeration	2026	VLDB
2	1,046	Parallel Subgraph Listing in a Large-Scale Graph	2014	SIGMOD
3	10,281	Towards the Scheduling of Vertex-constrained Multi Subgraph Matching Query	2020	SIGMOD
4	3,885	Distributed Subgraph Matching on Timely Dataflow	2019	VLDB
5	6,294	Incorporating Super-Operators in Big-Data Query Optimizers	2020	VLDB
6	440	Efficient Subgraph Matching on Billion Node Graphs	2012	VLDB
7	10,498	Beyond Maximum Common Subgraph: A Framework Maximizing Shared Computation for Multi-Query Subgraph Matching	2026	SIGMOD
8	4,133	HUGE: An Efficient and Scalable Subgraph Enumeration System	2021	SIGMOD
9	1,253	Distributed Evaluation of Subgraph Queries Using Worst-case Optimal Low-Memory Dataflows	2018	VLDB
10	1,137	Efficient Exploitation of Similar Subexpressions for Query Processing	2007	SIGMOD