DBScholar

Back to papers

Data Lake Management: Challenges and Opportunities

Summary: Data lakes create new management problems—dataset discovery and evolving needs for extraction, cleaning, integration, versioning, and metadata. This tutorial surveys state-of-the-art approaches, challenges, and opportunities unique to data-lake ecosystems for guiding future research. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
hc68a3db3407b5542
Venue
VLDB
Year
2019
Pagerank
0.00015479826
Overall Rank
624 | 95.81%
DOI
10.14778/3352063.3352116

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{nargesian_vldb19,
        title = {{Data Lake Management: Challenges and Opportunities}},
        author = {Nargesian, Fatemeh and Zhu, Erkang and Miller, Renee J. and Pu, Ken Q. and Arocena, Patricia C.},
        journal = {PVLDB},
        series = {{VLDB} '19},
        volume = {12},
        number = {12},
        pages = {1986--1989},
        doi = {10.14778/3352063.3352116},
        url = {https://doi.org/10.14778/3352063.3352116},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 45 of 45 citing papers.

Rank Citing Paper Year Venue Pagerank
501 Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes 2024 VLDB 0.00017267905
1,293 Finding Related Tables in Data Lakes for Interactive Data Science 2020 SIGMOD 0.00011149857
1,852 Semantics-aware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning 2023 VLDB 9.5004635e-05
1,932 CHORUS: Foundation Models for Unified Data Discovery and Exploration 2024 VLDB 9.34643e-05
1,988 SANTOS: Relationship-based Semantic Table Union Search 2023 SIGMOD 9.2482448e-05
2,231 GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization 2024 VLDB 8.7982985e-05
4,030 Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models 2021 SIGMOD 6.8407272e-05
4,159 Integrating Data Lake Tables 2023 VLDB 6.7717519e-05
4,409 LakeBench: A Benchmark for Discovering Joinable and Unionable Tables in Data Lakes 2024 VLDB 6.6120807e-05
4,905 Data-Sharing Markets: Model, Protocol, and Algorithms to Incentivize the Formation of Data-Sharing Consortia 2023 SIGMOD 6.3624391e-05
5,015 Discovering Related Data At Scale 2021 VLDB 6.3135569e-05
5,071 Data Collection and Quality Challenges for Deep Learning 2020 VLDB 6.2879908e-05
5,097 Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V 2023 VLDB 6.2742361e-05
5,106 Data-Driven Domain Discovery for Structured Datasets 2020 VLDB 6.2706946e-05
5,385 SchemaPile: A Large Collection of Relational Database Schemas 2024 SIGMOD 6.1523255e-05
5,401 RONIN: Data Lake Exploration 2021 VLDB 6.1462648e-05
5,672 Subgraph Matching over Graph Federation 2022 VLDB 6.0433033e-05
5,933 Doing More with Less: Characterizing Dataset Downsampling for AutoML 2021 VLDB 5.942188e-05
7,197 Cross Modal Data Discovery over Structured and Unstructured Data Lakes 2023 VLDB 5.5890662e-05
7,260 Entity Resolution On-Demand 2022 VLDB 5.571581e-05
7,582 Unity Catalog: Open and Universal Governance for the Lakehouse and Beyond 2025 SIGMOD 5.4910839e-05
8,583 OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance From Database Query Event Logs 2023 VLDB 5.3092552e-05
8,986 DataLoom: Simplifying Data Loading with LLMs 2024 VLDB 5.2428439e-05
9,912 FusionQuery: On-demand Fusion Queries over Multi-source Heterogeneous Data 2024 VLDB 5.1103839e-05
10,022 Towards Functional Decomposition of Storage Formats 2025 CIDR 5.0941658e-05
10,080 LiquidCache: Efficient Pushdown Caching for Cloud-Native Data Analytics 2025 VLDB 5.0830849e-05
10,111 EquiTensors: Learning Fair Integrations of Heterogeneous Urban Data 2021 SIGMOD 5.0789354e-05
10,337 QueryArtisan: Generating Data Manipulation Codes for Ad-hoc Analysis in Data Lakes 2025 VLDB 5.0200193e-05
10,499 Shape-Agnostic Table Overlap Discovery: A Maximum Common Subhypergraph Approach 2026 SIGMOD 4.9793485e-05
10,620 AutoDDG: Automated Dataset Description Generation using Large Language Models 2026 SIGMOD 4.9793485e-05
10,673 Qualitative Join Discovery in Data Lakes using Examples 2026 SIGMOD 4.9793485e-05
10,767 IncreQueryFusion: On-demand Data Fusion Framework in Dynamic Data Lakes 2026 VLDB 4.9793485e-05
10,894 MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery 2026 VLDB 4.9793485e-05
10,967 QEC lake: Data Lake for AI-enhanced Quantum Error Correction 2026 VLDB 4.9793485e-05
11,038 Semantic Data Systems: From Data Management to Data Understanding 2026 VLDB 4.9793485e-05
11,200 Table Overlap Estimation through Graph Embeddings 2025 SIGMOD 4.9793485e-05
11,295 OpenForge: Probabilistic Metadata Integration 2025 VLDB 4.9793485e-05
11,390 A Demonstration of QueryArtisan: Real-Time Data Lake Analysis via Dynamically Generated Data Manipulation Code 2025 VLDB 4.9793485e-05
11,394 GraphAr: An Efficient Storage Scheme for Graph Data in Data Lakes 2025 VLDB 4.9793485e-05
11,418 Sort it Like You Mean It: Discovering Semantically Interesting Attribute Augmentations to Sort Tables 2025 VLDB 4.9793485e-05
11,466 Towards an Objective Metric for Data Value Through Relevance 2024 CIDR 4.9793485e-05
11,512 Determining the Largest Overlap between Tables 2024 SIGMOD 4.9793485e-05
11,598 Searching Data Lakes for Nested and Joined Data 2024 VLDB 4.9793485e-05
11,606 KGFabric: A Scalable Knowledge Graph Warehouse for Enterprise Data Interconnection 2024 VLDB 4.9793485e-05
11,925 Detecting Layout Templates in Complex Multiregion Files 2022 VLDB 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 26 of 26 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
89 WebTables: Exploring the Power of Tables on the Web 2008 VLDB 0.00035129658
367 InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables 2012 SIGMOD 0.00019884846
427 Big Data Integration 2013 VLDB 0.00018465558
484 Data Integration for the Relational Web 2009 VLDB 0.00017548541
509 Goods: Organizing Google's Datasets 2016 SIGMOD 0.00017071087
694 JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes 2019 SIGMOD 0.00014727089
716 Guided Data Repair 2011 VLDB 0.00014553463
717 Finding Related Tables 2012 SIGMOD 0.00014532116
765 Table Union Search on Open Data 2018 VLDB 0.00014125424
972 The Data Civilizer System 2017 CIDR 0.00012763234
1,053 Answering Table Queries on the Web using Column Keywords 2012 VLDB 0.00012274222
1,086 DataHub: Collaborative Data Science & Dataset Version Management at Scale 2015 CIDR 0.0001210322
1,272 Discovering Queries based on Example Tuples 2014 SIGMOD 0.00011248674
1,318 LSH Ensemble: Internet-Scale Domain Search 2016 VLDB 0.00011047393
1,355 Principles of Dataset Versioning: Exploring the Recreation/Storage Tradeoff 2015 VLDB 0.0001092646
1,667 Bootstrapping Pay-As-You-Go Data Integration Systems 2008 SIGMOD 9.9378851e-05
1,776 Sample-Driven Schema Mapping 2012 SIGMOD 9.6674087e-05
1,880 Ground: A Data Context Service 2017 CIDR 9.4415148e-05
2,158 Open Data Integration 2018 VLDB 8.941016e-05
2,351 Schema Extraction for Tabular Data on the Web 2013 VLDB 8.5928057e-05
2,924 Constance: An Intelligent Data Lake System 2016 SIGMOD 7.8498312e-05
3,912 Navigating the Data Lake with DATAMARAN: Automatically Extracting Structure from Log Datasets 2018 SIGMOD 6.9306061e-05
4,019 CLAMS: Bringing Quality to Data Lakes 2016 SIGMOD 6.8512817e-05
4,393 Extracting Databases from Dark Data with DeepDive 2016 SIGMOD 6.620281e-05
6,244 Interactive Navigation of Open Data Linkages 2017 VLDB 5.8374791e-05
6,817 Error Diagnosis and Data Profiling with Data X-Ray 2015 VLDB 5.6727207e-05
Previous Page 1 / 1 Next

Semantically Similar Papers