Navigating the Data Lake with DATAMARAN: Automatically Extracting Structure from Log Datasets

Summary: Datamaran automatically extracts structure from semi-structured log data, identifying endpoints and filtering noise. It discovers structures without boundaries, achieving 95% extraction accuracy on GitHub logs, ~66% higher than unsupervised schemes. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5528
Venue: SIGMOD
Year: 2018
Pagerank: 7.0417782e-05
Overall Rank: 3,894 | 73.29%
DOI: 10.1145/3183713.3183746

Incoming Non-self Citations Over Time

Authors

1. Yihan Gao (University of Illinois Urbana-Champaign)
2. Silu Huang (University of Illinois Urbana-Champaign)
3. Aditya Parameswaran (University of Illinois Urbana-Champaign)

BibTeX Citation

@inproceedings{gao_sigmod18,
        title = {{Navigating the Data Lake with DATAMARAN: Automatically Extracting Structure from Log Datasets}},
        author = {Gao, Yihan and Huang, Silu and Parameswaran, Aditya},
        series = {{SIGMOD} '18},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3183713.3183746},
        url = {https://dl.acm.org/doi/10.1145/3183713.3183746},
        year = {2018}
}

Incoming Citations (Sorted by Pagerank)

Showing 8 of 8 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
749	Data Lake Management: Challenges and Opportunities	2019	VLDB	0.00014379989
2,744	Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks	2020	SIGMOD	8.1781662e-05
4,502	AS-Parser: Log Parsing Based on Adaptive Segmentation	2023	SIGMOD	6.6599757e-05
4,990	Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V	2023	VLDB	6.4105738e-05
5,403	Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples	2023	VLDB	6.2306083e-05
8,000	PIDS: Attribute Decomposition for Improved Compression and Query Performance in Columnar Storage	2020	VLDB	5.5092344e-05
10,414	Visual Template Inference for Data Extraction from Documents	2026	SIGMOD	5.093636e-05
11,886	Enabling Data Science for the Majority	2019	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 17 of 17 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
91	WebTables: Exploring the Power of Tables on the Web	2008	VLDB	0.00034838835
94	Potter's Wheel: An Interactive Data Cleaning System	2001	VLDB	0.00034616103
317	Annotating and Searching Web Tables Using Entities, Types and Relationships	2010	VLDB	0.00021402049
367	InfoGather: Entity Augmentation and Attribute Discovery By Holistic Matching with Web Tables	2012	SIGMOD	0.00019979463
503	RoadRunner: Towards Automatic Data Extraction from Large Web Sites	2001	VLDB	0.00017314037
514	Goods: Organizing Google's Datasets	2016	SIGMOD	0.00017178673
588	Extracting Structured Data from Web Pages	2003	SIGMOD	0.00016092668
739	Finding Related Tables	2012	SIGMOD	0.0001448231
753	Recovering Semantics of Tables on the Web	2011	VLDB	0.00014336094
1,165	Foofah: Transforming Data By Example	2017	SIGMOD	0.00011860616
1,316	Harvesting Relational Tables from Lists on the Web	2009	VLDB	0.00011181216
1,684	Answering Table Augmentation Queries from Unstructured Lists on the Web [Extended Abstract]	2009	VLDB	0.0001001843
1,947	Data Wrangling: The Challenging Journey from the Wild to the Lake	2015	CIDR	9.4326202e-05
2,913	Constance: An Intelligent Data Lake System	2016	SIGMOD	7.9684737e-05
3,181	InfoGather+: Semantic Matching and Annotation of Numeric and Time-Varying Attributes in Web Tables	2013	SIGMOD	7.6595256e-05
3,944	Robust Web Extraction: An Approach Based on a Probabilistic Tree-Edit Model	2009	SIGMOD	7.0058157e-05
5,554	Joint Unsupervised Structure Discovery and Information Extraction	2011	SIGMOD	6.1775485e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	5,344	Discovering Related Data At Scale	2021	VLDB
2	717	Automatic segmentation of text into structured records	2001	SIGMOD
3	7,572	Cross Modal Data Discovery over Structured and Unstructured Data Lakes	2023	VLDB
4	12,075	Graph-based Exploration of Non-graph Datasets	2016	VLDB
5	11,937	CoreKG: a Knowledge Lake Service	2018	VLDB
6	9,071	Data Lakes Empowered by Knowledge Graph Technologies	2021	SIGMOD
7	3,015	Organizing Data Lakes for Navigation	2020	SIGMOD
8	11,025	A Demonstration of QueryArtisan: Real-Time Data Lake Analysis via Dynamically Generated Data Manipulation Code	2025	VLDB
9	10,109	QueryArtisan: Generating Data Manipulation Codes for Ad-hoc Analysis in Data Lakes	2025	VLDB
10	713	Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes	2024	VLDB