Opportunistic Physical Design for Big Data Analytics

Summary: Treats materialized intermediates from exploratory MapReduce jobs as an opportunistic physical design. Proposes a semantic UDF model to reuse UDF-containing views and a minimum-cost rewrite algorithm (with provable guarantees); Hive prototype shows dramatic speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h502f41c29b1fd959
Venue: SIGMOD
Year: 2014
Pagerank: 5.9732317e-05
Overall Rank: 5,836 | 60.78%
DOI: 10.1145/2685555.2680152

Incoming Non-self Citations Over Time

Authors

1. Jeff LeFevre (University of California Santa Cruz)
2. Jagan Sankaranarayanan (NEC Corporation)
3. Hakan Hacıgümüş (NEC Corporation)
4. Junichi Tatemura (NEC Corporation)
5. Neoklis Polyzotis (University of California Santa Cruz)
6. Michael J. Carey (University of California Irvine)

BibTeX Citation

@inproceedings{lefevre_sigmod14,
        title = {{Opportunistic Physical Design for Big Data Analytics}},
        author = {LeFevre, Jeff and Sankaranarayanan, Jagan and Hacıgümüş, Hakan and Tatemura, Junichi and Polyzotis, Neoklis and Carey, Michael J.},
        series = {{SIGMOD} '14},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2685555.2680152},
        url = {https://dl.acm.org/doi/10.1145/2685555.2680152},
        year = {2014}
}

Incoming Citations (Sorted by Pagerank)

Showing 6 of 6 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,747	Selecting Subexpressions to Materialize at Datacenter Scale	2018	VLDB	9.7303647e-05
3,841	MISO: Souping Up Big Data Query Processing with a Multistore System	2014	SIGMOD	6.98877e-05
5,310	CliffGuard: A Principled Framework for Finding Robust Database Designs	2015	SIGMOD	6.184026e-05
7,534	Invisible Glue: Scalable Self-Tuning Multi-Stores	2015	CIDR	5.4985818e-05
8,600	Not Black-Box Anymore! Enabling Analytics-Aware Optimizations in Teradata Vantage	2021	VLDB	5.3043615e-05
9,169	Making Data Clouds Smarter at Keebo: Automated Warehouse Optimization using Data Learning	2023	SIGMOD	5.2139049e-05

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
72	Answering Queries Using Views (Extended Abstract)	1995	PODS	0.0003753293
88	Automated Selection of Materialized Views and Indexes for SQL Databases	2000	VLDB	0.00035340164
105	The MADlib Analytics Library or MAD Skills, the SQL	2012	VLDB	0.00033633007
432	Shark: SQL and Rich Analytics at Scale	2013	SIGMOD	0.00018331051
553	Optimizing Queries Using Materialized Views: A Practical, Scalable Solution	2001	SIGMOD	0.00016518678
823	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00013642066
936	A Scalable Algorithm for Answering Queries Using Views	2000	VLDB	0.00012983552
1,054	Answering Complex SQL Queries Using Automatic Summary Tables	2000	SIGMOD	0.00012263425
1,073	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00012163258
1,925	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.3643089e-05
2,197	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	8.8740089e-05
2,285	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD	8.6924793e-05
3,339	Optimizing Analytic Data Flows for Multiple Execution Engines	2012	SIGMOD	7.4045097e-05
3,841	MISO: Souping Up Big Data Query Processing with a Multistore System	2014	SIGMOD	6.98877e-05
4,232	On the Content of Materialized Aggregate Views	2000	PODS	6.7114333e-05
6,227	Scalable Query Rewriting: A Graph-Based Approach	2011	SIGMOD	5.8428038e-05
6,792	Hadoop's Adolescence: An analysis of Hadoop usage in scientific workloads	2013	VLDB	5.6794384e-05
9,341	Odyssey: A Multi-Store System for Evolutionary Analytics	2013	VLDB	5.1898232e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,706	Algorithms for Materialized View Design in Data Warehousing Environment	1997	VLDB
2	553	Optimizing Queries Using Materialized Views: A Practical, Scalable Solution	2001	SIGMOD
3	120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB
4	9,823	Supporting Scalable Analytics with Latency Constraints	2015	VLDB
5	3,217	Multi-Query Optimization in MapReduce Framework	2014	VLDB
6	754	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB
7	2,285	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
8	5,638	Revisiting Reuse in Main Memory Database Systems	2017	SIGMOD
9	1,073	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
10	4,069	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD