Exploratory Training: When Annotators Learn About Data

Summary: Game-theoretic collaborative active learning where annotator beliefs evolve with data as users revise strategies. Theory and algorithms for joint user-system convergence to a shared target model, reducing interactions, validated on real-world studies. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h9c08ba71afc5c4e9
Venue: SIGMOD
Year: 2023
Pagerank: 5.2840289e-05
Overall Rank: 8,756 | 41.13%
DOI: 10.1145/3589280

Incoming Non-self Citations Over Time

Authors

1. Rajesh Shrestha (Oregon State University)
2. Omeed Habibelahian (Oregon State University)
3. Arash Termehchy (Oregon State University)
4. Paolo Papotti (EURECOM)

BibTeX Citation

@inproceedings{shrestha_sigmod23,
        title = {{Exploratory Training: When Annotators Learn About Data}},
        author = {Shrestha, Rajesh and Habibelahian, Omeed and Termehchy, Arash and Papotti, Paolo},
        series = {{SIGMOD} '23},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3589280},
        url = {https://dl.acm.org/doi/10.1145/3589280},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,109	Generation of Training Examples for Tabular Natural Language Inference	2023	SIGMOD	5.2280464e-05
11,121	User-Centric Property Graph Repairs	2025	SIGMOD	4.9793485e-05
11,428	Versatile Property Graph Transformations	2025	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
104	HoloClean: Holistic Data Repairs with Probabilistic Inference	2017	VLDB	0.00033690989
160	CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies	2004	SIGMOD	0.00027837289
483	ActiveClean: Interactive Data Cleaning For Statistical Modeling	2016	VLDB	0.00017590977
500	Dependencies Revisited for Improving Data Quality	2008	PODS	0.00017280722
716	Guided Data Repair	2011	VLDB	0.00014553463
883	HoloDetect: Few-Shot Learning for Error Detection	2019	SIGMOD	0.00013268059
1,120	Snuba: Automating Weak Supervision to Label Training Data	2019	VLDB	0.00011946047
1,344	Detecting Data Errors: Where are we and what needs to be done?	2016	VLDB	0.00010956518
1,361	Computing Optimal Repairs for Functional Dependencies	2018	PODS	0.00010917917
1,537	Query Steering for Interactive Data Exploration	2013	CIDR	0.00010319063
1,805	Raha: A Configuration-Free Error Detection System	2019	SIGMOD	9.59842e-05
1,971	Discovery of Approximate (and Exact) Denial Constraints	2020	VLDB	9.2873596e-05
2,051	Messing Up with BART: Error Generation for Evaluating Data-Cleaning Algorithms	2016	VLDB	9.1199511e-05
2,125	Discovery of Genuine Functional Dependencies from Relational Data with Missing Values	2018	VLDB	9.0033119e-05
2,443	Approximate Denial Constraints	2020	VLDB	8.4586656e-05
2,648	Explore-by-Example: An Automatic Query Steering Framework for Interactive Data Exploration	2014	SIGMOD	8.1761271e-05
3,422	UGuide – User-Guided Discovery of FD-Detectable Errors	2017	SIGMOD	7.3138834e-05
4,226	Scalable Discovery of Unique Column Combinations	2014	VLDB	6.7183259e-05
4,650	Temporal Rules Discovery for Web Data Cleaning	2016	VLDB	6.4858157e-05
9,858	HILDA'22: The SIGMOD 2022 Workshop on Human-in-the-Loop Data Analytics	2022	SIGMOD	5.1184018e-05
10,016	The Data Interaction Game	2018	SIGMOD	5.0959403e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,699	Toward Interpretable and Actionable Data Analysis with Explanations and Causality	2022	VLDB
2	4,867	Selective Data Acquisition in the Wild for Model Charging	2022	VLDB
3	12,198	Entity Matching with Active Monotone Classification	2018	PODS
4	4,497	Automating Exploratory Data Analysis via Machine Learning: An Overview	2020	SIGMOD
5	7,052	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB
6	9,444	Intelligent Agents for Data Exploration	2024	VLDB
7	3,015	Data Acquisition for Improving Machine Learning Models	2021	VLDB
8	6,162	Guided Exploration of User Groups	2020	VLDB
9	4,507	Optimization for Active Learning-based Interactive Database Exploration	2019	VLDB
10	10,016	The Data Interaction Game	2018	SIGMOD