DBScholar

Back to papers

Data Lake Management: Challenges and Opportunities

Summary: Data lakes create new management problems—dataset discovery and evolving needs for extraction, cleaning, integration, versioning, and metadata. This tutorial surveys state-of-the-art approaches, challenges, and opportunities unique to data-lake ecosystems for guiding future research. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
hc68a3db3407b5542
Venue
VLDB
Year
2019
Pagerank
0.00015476813
Overall Rank
624 | 95.81%
DOI
10.14778/3352063.3352116
PDF
Download (CC BY-NC-ND 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{nargesian_vldb19,
        title = {{Data Lake Management: Challenges and Opportunities}},
        author = {Nargesian, Fatemeh and Zhu, Erkang and Miller, Renee J. and Pu, Ken Q. and Arocena, Patricia C.},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {12},
        pages = {1986--1989},
        doi = {10.14778/3352063.3352116},
        url = {https://doi.org/10.14778/3352063.3352116},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 45 of 45 citing papers.

Rank Citing Paper Year Venue Pagerank
496 Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes 2024 VLDB 0.00017318538
1,293 Finding Related Tables in Data Lakes for Interactive Data Science 2020 SIGMOD 0.00011144991
1,854 Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning 2023 VLDB 9.4961138e-05
1,929 CHORUS: Foundation Models for Unified Data Discovery and Exploration 2024 VLDB 9.3551286e-05
1,990 SANTOS: Relationship-based Semantic Table Union Search 2023 SIGMOD 9.2443028e-05
2,227 GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization 2024 VLDB 8.8007923e-05
4,031 Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models 2021 SIGMOD 6.8374909e-05
4,159 Integrating Data Lake Tables 2023 VLDB 6.7687144e-05
4,411 LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes 2024 VLDB 6.6089506e-05
4,906 Data-Sharing Markets: Model, Protocol, and Algorithms to Incentivize the Formation of Data-Sharing Consortia 2023 SIGMOD 6.3594272e-05
5,018 Discovering Related Data At Scale 2021 VLDB 6.3107365e-05
5,074 Data Collection and Quality Challenges for Deep Learning 2020 VLDB 6.2850141e-05
5,100 Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V 2023 VLDB 6.271266e-05
5,109 Data-Driven Domain Discovery for Structured Datasets 2020 VLDB 6.2679672e-05
5,393 SchemaPile: A Large Collection of Relational Database Schemas 2024 SIGMOD 6.1494131e-05
5,407 RONIN: Data Lake Exploration 2021 VLDB 6.1433552e-05
5,673 Subgraph Matching over Graph Federation 2022 VLDB 6.0404425e-05
5,933 Doing More with Less: Characterizing Dataset Downsampling for AutoML 2021 VLDB 5.9393751e-05
7,199 Cross Modal Data Discovery over Structured and Unstructured Data Lakes 2023 VLDB 5.5864204e-05
7,263 Entity Resolution On-Demand 2022 VLDB 5.5693872e-05
7,588 Unity Catalog: Open and Universal Governance for the Lakehouse and Beyond 2025 SIGMOD 5.4884845e-05
8,590 OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs 2023 VLDB 5.3067446e-05
8,984 DataLoom: Simplifying Data Loading with LLMs 2024 VLDB 5.2428922e-05
9,919 FusionQuery: On-demand Fusion Queries over Multi-source Heterogeneous Data 2024 VLDB 5.1079647e-05
10,027 Towards Functional Decomposition of Storage Formats 2025 CIDR 5.0917543e-05
10,085 LiquidCache: Efficient Pushdown Caching for Cloud-Native Data Analytics 2025 VLDB 5.0806786e-05
10,115 EquiTensors: Learning Fair Integrations of Heterogeneous Urban Data 2021 SIGMOD 5.0765311e-05
10,344 QueryArtisan: Generating Data Manipulation Codes for Ad-hoc Analysis in Data Lakes 2025 VLDB 5.0176429e-05
10,510 Shape-Agnostic Table Overlap Discovery: A Maximum Common Subhypergraph Approach 2026 SIGMOD 4.9769913e-05
10,631 AutoDDG: Automated Dataset Description Generation using Large Language Models 2026 SIGMOD 4.9769913e-05
10,684 Qualitative Join Discovery in Data Lakes using Examples 2026 SIGMOD 4.9769913e-05
10,777 IncreQueryFusion: On-demand Data Fusion Framework in Dynamic Data Lakes 2026 VLDB 4.9769913e-05
10,903 MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery 2026 VLDB 4.9769913e-05
10,976 QEC lake: Data Lake for AI-enhanced Quantum Error Correction 2026 VLDB 4.9769913e-05
11,047 Semantic Data Systems: From Data Management to Data Understanding 2026 VLDB 4.9769913e-05
11,209 Table Overlap Estimation through Graph Embeddings 2025 SIGMOD 4.9769913e-05
11,303 OpenForge: Probabilistic Metadata Integration 2025 VLDB 4.9769913e-05
11,396 A Demonstration of QueryArtisan: Real-Time Data Lake Analysis via Dynamically Generated Data Manipulation Code 2025 VLDB 4.9769913e-05
11,400 GraphAr: An Efficient Storage Scheme for Graph Data in Data Lakes 2025 VLDB 4.9769913e-05
11,424 Sort it Like You Mean It: Discovering Semantically Interesting Attribute Augmentations to Sort Tables 2025 VLDB 4.9769913e-05
11,472 Towards an Objective Metric for Data Value Through Relevance 2024 CIDR 4.9769913e-05
11,518 Determining the Largest Overlap between Tables 2024 SIGMOD 4.9769913e-05
11,604 Searching Data Lakes for Nested and Joined Data 2024 VLDB 4.9769913e-05
11,612 KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection 2024 VLDB 4.9769913e-05
11,931 Detecting Layout Templates in Complex Multiregion Files 2022 VLDB 4.9769913e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 26 of 26 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
89 WebTables: Exploring the Power of Tables on the Web 2008 VLDB 0.00035115118
367 InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables 2012 SIGMOD 0.00019876273
428 Big Data Integration 2013 VLDB 0.00018457189
484 Data Integration for the Relational Web 2009 VLDB 0.00017541069
509 Goods: Organizing Google's Datasets 2016 SIGMOD 0.00017063491
694 JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes 2019 SIGMOD 0.00014721161
716 Guided Data Repair 2011 VLDB 0.00014546968
718 Finding Related Tables 2012 SIGMOD 0.00014526813
765 Table Union Search on Open Data 2018 VLDB 0.00014119196
972 The Data Civilizer System 2017 CIDR 0.00012757732
1,053 Answering Table Queries on the Web using Column Keywords 2012 VLDB 0.00012268624
1,086 DataHub: Collaborative Data Science & Dataset Version Management at Scale 2015 CIDR 0.0001209772
1,273 Discovering Queries based on Example Tuples 2014 SIGMOD 0.00011243661
1,319 LSH Ensemble: Internet-Scale Domain Search 2016 VLDB 0.00011042864
1,355 Principles of Dataset Versioning: Exploring the Recreation/Storage Tradeoff 2015 VLDB 0.00010921447
1,668 Bootstrapping Pay-As-You-Go Data Integration Systems 2008 SIGMOD 9.9333074e-05
1,776 Sample-Driven Schema Mapping 2012 SIGMOD 9.6631141e-05
1,881 Ground: A Data Context Service 2017 CIDR 9.4372419e-05
2,160 Open Data Integration 2018 VLDB 8.9373643e-05
2,352 Schema Extraction for Tabular Data on the Web 2013 VLDB 8.5888676e-05
2,925 Constance: An Intelligent Data Lake System 2016 SIGMOD 7.8462376e-05
3,913 Navigating the Data Lake with DATAMARAN: Automatically Extracting Structure from Log Datasets 2018 SIGMOD 6.9274273e-05
4,020 CLAMS: Bringing Quality to Data Lakes 2016 SIGMOD 6.8481527e-05
4,395 Extracting Databases from Dark Data with DeepDive 2016 SIGMOD 6.6175689e-05
6,247 Interactive Navigation of Open Data Linkages 2017 VLDB 5.8347624e-05
6,823 Error Diagnosis and Data Profiling with Data X-Ray 2015 VLDB 5.6701292e-05
Previous Page 1 / 1 Next

Semantically Similar Papers