Opportunistic Physical Design for Big Data Analytics

Summary: Treats materialized intermediates from exploratory MapReduce jobs as an opportunistic physical design. Proposes a semantic UDF model to reuse UDF-containing views and a minimum-cost rewrite algorithm (with provable guarantees); Hive prototype shows dramatic speedups. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 4945
Venue: SIGMOD
Year: 2014
Pagerank: 6.1092082e-05
Overall Rank: 5,720 | 60.76%
DOI: 10.1145/2685555.2680152

Incoming Non-self Citations Over Time

Authors

1. Jeff LeFevre (University of California Santa Cruz)
2. Jagan Sankaranarayanan (NEC Corporation)
3. Hakan Hacıgümüş (NEC Corporation)
4. Junichi Tatemura (NEC Corporation)
5. Neoklis Polyzotis (University of California Santa Cruz)
6. Michael J. Carey (University of California Irvine)

BibTeX Citation

@inproceedings{lefevre_sigmod14,
        title = {{Opportunistic Physical Design for Big Data Analytics}},
        author = {LeFevre, Jeff and Sankaranarayanan, Jagan and Hacıgümüş, Hakan and Tatemura, Junichi and Polyzotis, Neoklis and Carey, Michael J.},
        series = {{SIGMOD} '14},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2685555.2680152},
        url = {https://dl.acm.org/doi/10.1145/2685555.2680152},
        year = {2014}
}

Incoming Citations (Sorted by Pagerank)

Showing 6 of 6 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,765	Selecting Subexpressions to Materialize at Datacenter Scale	2018	VLDB	9.8079546e-05
3,783	MISO: Souping Up Big Data Query Processing with a Multistore System	2014	SIGMOD	7.1298683e-05
5,222	CliffGuard: A Principled Framework for Finding Robust Database Designs	2015	SIGMOD	6.3103741e-05
7,428	Invisible Glue: Scalable Self-Tuning Multi-Stores	2015	CIDR	5.6205394e-05
8,450	Not Black-Box Anymore! Enabling Analytics-Aware Optimizations in Teradata Vantage	2021	VLDB	5.4231788e-05
9,008	Making Data Clouds Smarter at Keebo: Automated Warehouse Optimization using Data Learning	2023	SIGMOD	5.3335632e-05

Outgoing Citations (Sorted by Pagerank)

Showing 18 of 18 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
69	Answering Queries Using Views (Extended Abstract)	1995	PODS	0.00038090878
87	Automated Selection of Materialized Views and Indexes for SQL Databases	2000	VLDB	0.00035281619
106	The MADlib Analytics Library or MAD Skills, the SQL	2012	VLDB	0.00033539462
425	Shark: SQL and Rich Analytics at Scale	2013	SIGMOD	0.00018704491
559	Optimizing Queries Using Materialized Views: A Practical, Scalable Solution	2001	SIGMOD	0.00016528822
803	MRShare: Sharing Across Multiple Queries in MapReduce	2010	VLDB	0.00013899943
913	A Scalable Algorithm for Answering Queries Using Views	2000	VLDB	0.00013253821
1,034	Answering Complex SQL Queries Using Automatic Summary Tables	2000	SIGMOD	0.00012516197
1,054	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB	0.00012390673
1,883	ReStore: Reusing Results of MapReduce Jobs	2012	VLDB	9.5421713e-05
2,164	Opening the Black Boxes in Data Flow Optimization	2012	VLDB	9.0521951e-05
2,265	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD	8.8398946e-05
3,275	Optimizing Analytic Data Flows for Multiple Execution Engines	2012	SIGMOD	7.5747814e-05
3,783	MISO: Souping Up Big Data Query Processing with a Multistore System	2014	SIGMOD	7.1298683e-05
4,155	On the Content of Materialized Aggregate Views	2000	PODS	6.866935e-05
6,090	Scalable Query Rewriting: A Graph-Based Approach	2011	SIGMOD	5.9796288e-05
6,736	Hadoop's Adolescence: An analysis of Hadoop usage in scientific workloads	2013	VLDB	5.787547e-05
9,160	Odyssey: A Multi-Store System for Evolutionary Analytics	2013	VLDB	5.3114559e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,743	Algorithms for Materialized View Design in Data Warehousing Environment	1997	VLDB
2	559	Optimizing Queries Using Materialized Views: A Practical, Scalable Solution	2001	SIGMOD
3	120	HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads	2009	VLDB
4	9,640	Supporting Scalable Analytics with Latency Constraints	2015	VLDB
5	3,163	Multi-Query Optimization in MapReduce Framework	2014	VLDB
6	735	Profiling, What-if Analysis, and Cost-based Optimization of MapReduce Programs	2011	VLDB
7	2,265	A Platform for Scalable One-Pass Analytics using MapReduce	2011	SIGMOD
8	5,947	Revisiting Reuse in Main Memory Database Systems	2017	SIGMOD
9	1,054	Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads	2012	VLDB
10	4,481	Dynamically Optimizing Queries over Large Scale Data Platforms	2014	SIGMOD