Navigating the Data Lake with DATAMARAN: Automatically Extracting Structure from Log Datasets

Summary: Datamaran automatically extracts structure from semi-structured log data, identifying endpoints and filtering noise. It discovers structures without boundaries, achieving 95% extraction accuracy on GitHub logs, ~66% higher than unsupervised schemes. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h48f2350b78516a56
Venue: SIGMOD
Year: 2018
Pagerank: 6.9306061e-05
Overall Rank: 3,912 | 73.70%
DOI: 10.1145/3183713.3183746

Incoming Non-self Citations Over Time

Authors

1. Yihan Gao (University of Illinois Urbana-Champaign)
2. Silu Huang (University of Illinois Urbana-Champaign)
3. Aditya Parameswaran (University of Illinois Urbana-Champaign)

BibTeX Citation

@inproceedings{gao_sigmod18,
        title = {{Navigating the Data Lake with DATAMARAN: Automatically Extracting Structure from Log Datasets}},
        author = {Gao, Yihan and Huang, Silu and Parameswaran, Aditya},
        series = {{SIGMOD} '18},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3183713.3183746},
        url = {https://dl.acm.org/doi/10.1145/3183713.3183746},
        year = {2018}
}

Incoming Citations (Sorted by Pagerank)

Showing 8 of 8 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
624	Data Lake Management: Challenges and Opportunities	2019	VLDB	0.00015479826
2,641	Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks	2020	SIGMOD	8.1787073e-05
4,594	AS-Parser: Log Parsing Based on Adaptive Segmentation	2023	SIGMOD	6.5105437e-05
5,097	Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V	2023	VLDB	6.2742361e-05
5,167	Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples	2023	VLDB	6.2462501e-05
8,160	PIDS: Attribute Decomposition for Improved Compression and Query Performance in Columnar Storage	2020	VLDB	5.3871591e-05
10,605	Visual Template Inference for Data Extraction from Documents	2026	SIGMOD	4.9793485e-05
12,186	Enabling Data Science for the Majority	2019	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 17 of 17 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
89	WebTables: Exploring the Power of Tables on the Web	2008	VLDB	0.00035129658
95	Potter's Wheel: An Interactive Data Cleaning System	2001	VLDB	0.00034382643
316	Annotating and Searching Web Tables Using Entities, Types and Relationships	2010	VLDB	0.00021221602
367	InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables	2012	SIGMOD	0.00019884846
509	Goods: Organizing Google's Datasets	2016	SIGMOD	0.00017071087
515	RoadRunner: Towards Automatic Data Extraction from Large Web Sites	2001	VLDB	0.00016970049
599	Extracting Structured Data from Web Pages	2003	SIGMOD	0.0001575536
717	Finding Related Tables	2012	SIGMOD	0.00014532116
757	Recovering Semantics of Tables on the Web	2011	VLDB	0.00014205602
1,115	Foofah: Transforming Data By Example	2017	SIGMOD	0.00011964111
1,337	Harvesting Relational Tables from Lists on the Web	2009	VLDB	0.00010987014
1,697	Answering Table Augmentation Queries from Unstructured Lists on the Web [Extended Abstract]	2009	VLDB	9.8545388e-05
1,977	Data Wrangling: The Challenging Journey from the Wild to the Lake	2015	CIDR	9.2730808e-05
2,924	Constance: An Intelligent Data Lake System	2016	SIGMOD	7.8498312e-05
3,245	InfoGather+: Semantic Matching and Annotation of Numeric and Time-Varying Attributes in Web Tables	2013	SIGMOD	7.4957822e-05
4,021	Robust Web Extraction: An Approach Based on a Probabilistic Tree-Edit Model	2009	SIGMOD	6.8498852e-05
5,670	Joint Unsupervised Structure Discovery and Information Extraction	2011	SIGMOD	6.0445956e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	735	Automatic segmentation of text into structured records	2001	SIGMOD
2	7,197	Cross Modal Data Discovery over Structured and Unstructured Data Lakes	2023	VLDB
3	12,368	Graph-based Exploration of Non-graph Datasets	2016	VLDB
4	12,236	CoreKG: a Knowledge Lake Service	2018	VLDB
5	10,804	Document-to-Database: Extraction Meets Relational Semantics	2026	VLDB
6	9,247	Data Lakes Empowered by Knowledge Graph Technologies	2021	SIGMOD
7	2,926	Organizing Data Lakes for Navigation	2020	SIGMOD
8	11,390	A Demonstration of QueryArtisan: Real-Time Data Lake Analysis via Dynamically Generated Data Manipulation Code	2025	VLDB
9	10,337	QueryArtisan: Generating Data Manipulation Codes for Ad-hoc Analysis in Data Lakes	2025	VLDB
10	501	Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes	2024	VLDB