Searching Data Lakes for Nested and Joined Data

Summary: Extends Juneau to search data lakes for hierarchical tables/JSON by matching ranked multi-table join-and-transform views. Novel index/sketch-based result merging scales nested-data discovery, outperforming heuristics in relevance, coverage, speed, and downstream ML utility. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 13735
Venue: VLDB
Year: 2024
Pagerank: 5.093636e-05
Overall Rank: 11,270 | 22.68%
DOI: 10.14778/3681954.3682005

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Yi Zhang (Amazon)
2. Peter Baile Chen (Massachusetts Institute of Technology)
3. Zachary G. Ives (University of Pennsylvania)

BibTeX Citation

@article{zhang_vldb24,
        title = {{Searching Data Lakes for Nested and Joined Data}},
        author = {Zhang, Yi and Chen, Peter Baile and Ives, Zachary G.},
        journal = {PVLDB},
        series = {{VLDB} '24},
        volume = {17},
        number = {11},
        pages = {3346--3359},
        doi = {10.14778/3681954.3682005},
        url = {https://doi.org/10.14778/3681954.3682005},
        year = {2024}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 23 of 23 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
5	Optimal Aggregation Algorithms for Middleware [Extended Abstract]	2001	PODS	0.0010828372
27	Fast Algorithms for Mining Association Rules	1994	VLDB	0.00052255472
91	WebTables: Exploring the Power of Tables on the Web	2008	VLDB	0.00034838835
137	Relational Databases for Querying XML Documents: Limitations and Opportunities	1999	VLDB	0.00029877792
318	Storing Semistructured Data with STORED	1999	SIGMOD	0.00021380325
397	TURL: Table Understanding through Representation Learning	2021	VLDB	0.00019278189
499	Supporting Incremental Join Queries on Ranked Inputs	2001	VLDB	0.00017431827
514	Goods: Organizing Google's Datasets	2016	SIGMOD	0.00017178673
749	Data Lake Management: Challenges and Opportunities	2019	VLDB	0.00014379989
753	Recovering Semantics of Tables on the Web	2011	VLDB	0.00014336094
779	JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes	2019	SIGMOD	0.00014092047
963	The Data Civilizer System	2017	CIDR	0.00012935145
1,055	Answering Table Queries on the Web using Column Keywords	2012	VLDB	0.00012389274
1,303	Finding Related Tables in Data Lakes for Interactive Data Science	2020	SIGMOD	0.0001123653
1,495	LSH Ensemble: Internet-Scale Domain Search	2016	VLDB	0.00010571481
2,192	Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning	2023	VLDB	8.9767241e-05
2,315	SANTOS: Relationship-based Semantic Table Union Search	2023	SIGMOD	8.7605101e-05
2,744	Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks	2020	SIGMOD	8.1781662e-05
3,073	DeepJoin: Joinable Table Discovery with Pre-trained Language Models	2023	VLDB	7.785842e-05
3,271	Ten Years of WebTables	2018	VLDB	7.5779937e-05
4,123	Integrating Data Lake Tables	2023	VLDB	6.8878137e-05
6,192	Pre-trained Embeddings for Entity Resolution: An Experimental Analysis	2023	VLDB	5.947284e-05
7,690	Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes	2021	SIGMOD	5.5669307e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,931	Automatically Incorporating New Sources in Keyword Search-Based Data Integration	2010	SIGMOD
2	779	JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes	2019	SIGMOD
3	878	Table Union Search on Open Data	2018	VLDB
4	3,015	Organizing Data Lakes for Navigation	2020	SIGMOD
5	2,192	Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning	2023	VLDB
6	5,344	Discovering Related Data At Scale	2021	VLDB
7	3,228	Correlation Sketches for Approximate Join-Correlation Queries	2021	SIGMOD
8	7,893	LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes	2024	VLDB
9	3,073	DeepJoin: Joinable Table Discovery with Pre-trained Language Models	2023	VLDB
10	1,303	Finding Related Tables in Data Lakes for Interactive Data Science	2020	SIGMOD