DBScholar

Back to papers

Data Lake Management: Challenges and Opportunities

Summary: Data lakes create new management problems—dataset discovery and evolving needs for extraction, cleaning, integration, versioning, and metadata. This tutorial surveys state-of-the-art approaches, challenges, and opportunities unique to data-lake ecosystems for guiding future research. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
hc68a3db3407b5542
Venue
VLDB
Year
2019
Pagerank
0.00015446839
Overall Rank
623 | 95.83%
DOI
10.14778/3352063.3352116
PDF
Download (CC BY-NC-ND 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{nargesian_vldb19,
        title = {{Data Lake Management: Challenges and Opportunities}},
        author = {Nargesian, Fatemeh and Zhu, Erkang and Miller, Renee J. and Pu, Ken Q. and Arocena, Patricia C.},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {12},
        pages = {1986--1989},
        doi = {10.14778/3352063.3352116},
        url = {https://doi.org/10.14778/3352063.3352116},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 45 of 45 citing papers.

Rank Citing Paper Year Venue Pagerank
494 Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes 2024 VLDB 0.00017350554
1,268 Finding Related Tables in Data Lakes for Interactive Data Science 2020 SIGMOD 0.00011243615
1,742 Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning 2023 VLDB 9.7283207e-05
1,838 CHORUS: Foundation Models for Unified Data Discovery and Exploration 2024 VLDB 9.5195856e-05
1,929 SANTOS: Relationship-based Semantic Table Union Search 2023 SIGMOD 9.3458857e-05
2,227 GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization 2024 VLDB 8.7924681e-05
4,049 Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models 2021 SIGMOD 6.8169373e-05
4,158 Integrating Data Lake Tables 2023 VLDB 6.7595989e-05
4,419 LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes 2024 VLDB 6.5905711e-05
4,930 Data-Sharing Markets: Model, Protocol, and Algorithms to Incentivize the Formation of Data-Sharing Consortia 2023 SIGMOD 6.3383989e-05
5,036 Discovering Related Data At Scale 2021 VLDB 6.2907736e-05
5,090 Data-Driven Domain Discovery for Structured Datasets 2020 VLDB 6.265808e-05
5,094 Data Collection and Quality Challenges for Deep Learning 2020 VLDB 6.264232e-05
5,120 Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V 2023 VLDB 6.254003e-05
5,383 SchemaPile: A Large Collection of Relational Database Schemas 2024 SIGMOD 6.1379374e-05
5,423 RONIN: Data Lake Exploration 2021 VLDB 6.1230414e-05
5,676 Subgraph Matching over Graph Federation 2022 VLDB 6.0246247e-05
5,945 Doing More with Less: Characterizing Dataset Downsampling for AutoML 2021 VLDB 5.9197358e-05
7,217 Cross Modal Data Discovery over Structured and Unstructured Data Lakes 2023 VLDB 5.5679482e-05
7,278 Entity Resolution On-Demand 2022 VLDB 5.5511396e-05
7,605 Unity Catalog: Open and Universal Governance for the Lakehouse and Beyond 2025 SIGMOD 5.4703362e-05
8,609 OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs 2023 VLDB 5.2892055e-05
9,003 DataLoom: Simplifying Data Loading with LLMs 2024 VLDB 5.2256178e-05
9,954 FusionQuery: On-demand Fusion Queries over Multi-source Heterogeneous Data 2024 VLDB 5.0910746e-05
10,065 Towards Functional Decomposition of Storage Formats 2025 CIDR 5.0749178e-05
10,095 Table Overlap Estimation through Graph Embeddings 2025 SIGMOD 5.068372e-05
10,099 Determining the Largest Overlap between Tables 2024 SIGMOD 5.068372e-05
10,119 LiquidCache: Efficient Pushdown Caching for Cloud-Native Data Analytics 2025 VLDB 5.0638787e-05
10,152 EquiTensors: Learning Fair Integrations of Heterogeneous Urban Data 2021 SIGMOD 5.0597449e-05
10,371 QueryArtisan: Generating Data Manipulation Codes for Ad-hoc Analysis in Data Lakes 2025 VLDB 5.0010514e-05
10,576 Shape-Agnostic Table Overlap Discovery: A Maximum Common Subhypergraph Approach 2026 SIGMOD 4.9605343e-05
10,692 AutoDDG: Automated Dataset Description Generation using Large Language Models 2026 SIGMOD 4.9605343e-05
10,745 Qualitative Join Discovery in Data Lakes using Examples 2026 SIGMOD 4.9605343e-05
10,838 IncreQueryFusion: On-demand Data Fusion Framework in Dynamic Data Lakes 2026 VLDB 4.9605343e-05
10,964 MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery 2026 VLDB 4.9605343e-05
11,037 QEC lake: Data Lake for AI-enhanced Quantum Error Correction 2026 VLDB 4.9605343e-05
11,108 Semantic Data Systems: From Data Management to Data Understanding 2026 VLDB 4.9605343e-05
11,358 OpenForge: Probabilistic Metadata Integration 2025 VLDB 4.9605343e-05
11,449 A Demonstration of QueryArtisan: Real-Time Data Lake Analysis via Dynamically Generated Data Manipulation Code 2025 VLDB 4.9605343e-05
11,453 GraphAr: An Efficient Storage Scheme for Graph Data in Data Lakes 2025 VLDB 4.9605343e-05
11,477 Sort it Like You Mean It: Discovering Semantically Interesting Attribute Augmentations to Sort Tables 2025 VLDB 4.9605343e-05
11,524 Towards an Objective Metric for Data Value Through Relevance 2024 CIDR 4.9605343e-05
11,651 Searching Data Lakes for Nested and Joined Data 2024 VLDB 4.9605343e-05
11,659 KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection 2024 VLDB 4.9605343e-05
11,975 Detecting Layout Templates in Complex Multiregion Files 2022 VLDB 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 26 of 26 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
88 WebTables: Exploring the Power of Tables on the Web 2008 VLDB 0.00035500667
367 InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables 2012 SIGMOD 0.00019882429
422 Big Data Integration 2013 VLDB 0.00018451149
484 Data Integration for the Relational Web 2009 VLDB 0.00017535511
512 Goods: Organizing Google's Datasets 2016 SIGMOD 0.00017036842
679 JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes 2019 SIGMOD 0.00014802
705 Finding Related Tables 2012 SIGMOD 0.00014626858
717 Guided Data Repair 2011 VLDB 0.00014531658
758 Table Union Search on Open Data 2018 VLDB 0.00014200657
972 The Data Civilizer System 2017 CIDR 0.00012742827
1,037 Answering Table Queries on the Web using Column Keywords 2012 VLDB 0.00012357454
1,089 DataHub: Collaborative Data Science & Dataset Version Management at Scale 2015 CIDR 0.00012062983
1,275 Discovering Queries based on Example Tuples 2014 SIGMOD 0.00011213614
1,325 LSH Ensemble: Internet-Scale Domain Search 2016 VLDB 0.000110184
1,362 Principles of Dataset Versioning: Exploring the Recreation/Storage Tradeoff 2015 VLDB 0.0001088824
1,673 Bootstrapping Pay-As-You-Go Data Integration Systems 2008 SIGMOD 9.9018228e-05
1,787 Sample-Driven Schema Mapping 2012 SIGMOD 9.6333179e-05
1,878 Ground: A Data Context Service 2017 CIDR 9.431234e-05
2,097 Open Data Integration 2018 VLDB 9.0397587e-05
2,289 Schema Extraction for Tabular Data on the Web 2013 VLDB 8.6813411e-05
2,936 Constance: An Intelligent Data Lake System 2016 SIGMOD 7.825556e-05
3,897 Navigating the Data Lake with DATAMARAN: Automatically Extracting Structure from Log Datasets 2018 SIGMOD 6.9202872e-05
4,016 CLAMS: Bringing Quality to Data Lakes 2016 SIGMOD 6.8370819e-05
4,403 Extracting Databases from Dark Data with DeepDive 2016 SIGMOD 6.6033854e-05
6,252 Interactive Navigation of Open Data Linkages 2017 VLDB 5.8179392e-05
6,837 Error Diagnosis and Data Profiling with Data X-Ray 2015 VLDB 5.6518411e-05
Previous Page 1 / 1 Next

Semantically Similar Papers