Exploiting Soft and Hard Correlations in Big Data Query Optimization

Summary: EXORD exploits both hard and soft correlations to accelerate big-data query optimization, addressing uncertainty that RDBMS-based methods ignore. A three-phase pipeline validates soft correlations, selects/prepares with violation handling, and deploys them in optimization; a cost-benefit model yields a polynomial-time heuristic; Hive/Hadoop prototype achieves >10x speedup with low overhead. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 11212
Venue: VLDB
Year: 2016
Pagerank: 5.1699414e-05
Overall Rank: 6,173 | 57.06%
DOI: -

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
2,483	Discovery of Approximate (and Exact) Denial Constraints	2020	VLDB	8.6864916e-05
6,498	Memory-Aware Framework for Efficient Second-Order Random Walk on Large Graphs	2020	SIGMOD	5.0392468e-05
9,410	Leveraging Application Data Constraints to Optimize Database-Backed Web Applications	2023	VLDB	4.3441378e-05

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
224	CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies	2004	SIGMOD	0.00032746205
413	HaLoop: Efficient Iterative Data Processing on Large Clusters	2010	VLDB	0.00023904409
555	Discovering Denial Constraints	2013	VLDB	0.00020254908
780	Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience	2009	VLDB	0.00016775082
794	Hadoop++: Making a Yellow Elephant Run Like a Cheetah (Without It Even Noticing)	2010	VLDB	0.00016605103
947	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00015114576
1,615	The Performance of MapReduce: An In-depth Study	2010	VLDB	0.00011132319
1,863	Cheetah: A High Performance, Custom Data Warehouse on Top of MapReduce	2010	VLDB	0.00010286531
1,974	BHUNT: Automatic Discovery of Fuzzy Algebraic Constraints in Relational Data	2003	VLDB	9.8866171e-05
2,205	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.2920002e-05
2,439	CoHadoop: Flexible Data Placement and Its Exploitation in Hadoop	2011	VLDB	8.8190594e-05
2,837	Correlation Maps: A Compressed Access Method for Exploiting Soft Functional Dependencies	2009	VLDB	8.0414149e-05
3,867	CORADD: Correlation Aware Database Designer for Materialized Views and Indexes	2010	VLDB	6.683173e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
10,149	CorrBound: Cardinality Estimation Accounting for Inter- and Intra-relation Correlations	2026	SIGMOD	4.1945683e-05
3,867	CORADD: Correlation Aware Database Designer for Materialized Views and Indexes	2010	VLDB	6.683173e-05
684	Towards a Robust Query Optimizer: A Principled and Practical Approach	2005	SIGMOD	0.00018179769
8,979	High Performance Stream Query Processing With Correlation-Aware Partitioning	2014	VLDB	4.4170433e-05
3,824	Correlation Sketches for Approximate Join-Correlation Queries	2021	SIGMOD	6.7260705e-05
2,837	Correlation Maps: A Compressed Access Method for Exploiting Soft Functional Dependencies	2009	VLDB	8.0414149e-05
7,653	CORDS: Automatic Generation of Correlation Statistics in DB2	2004	VLDB	4.6875371e-05
8,869	Sharing-Aware Horizontal Partitioning for Exploiting Correlations During Query Processing	2010	VLDB	4.4320338e-05
224	CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies	2004	SIGMOD	0.00032746205
5,014	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD	5.7586174e-05