Exploratory Training: When Annotators Learn About Data

Summary: Game-theoretic collaborative active learning where annotator beliefs evolve with data as users revise strategies. Theory and algorithms for joint user-system convergence to a shared target model, reducing interactions, validated on real-world studies. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h9c08ba71afc5c4e9
Venue: SIGMOD
Year: 2023
Pagerank: 5.2815275e-05
Overall Rank: 8,764 | 41.10%
DOI: 10.1145/3589280

Incoming Non-self Citations Over Time

Authors

1. Rajesh Shrestha (Oregon State University)
2. Omeed Habibelahian (Oregon State University)
3. Arash Termehchy (Oregon State University)
4. Paolo Papotti (EURECOM)

BibTeX Citation

@inproceedings{shrestha_sigmod23,
        title = {{Exploratory Training: When Annotators Learn About Data}},
        author = {Shrestha, Rajesh and Habibelahian, Omeed and Termehchy, Arash and Papotti, Paolo},
        series = {{SIGMOD} '23},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3589280},
        url = {https://dl.acm.org/doi/10.1145/3589280},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,118	Generation of Training Examples for Tabular Natural Language Inference	2023	SIGMOD	5.2255715e-05
11,130	User-Centric Property Graph Repairs	2025	SIGMOD	4.9769913e-05
11,434	Versatile Property Graph Transformations	2025	VLDB	4.9769913e-05

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
104	HoloClean: Holistic Data Repairs with Probabilistic Inference	2017	VLDB	0.00033676943
160	CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies	2004	SIGMOD	0.00027827605
483	ActiveClean: Interactive Data Cleaning For Statistical Modeling	2016	VLDB	0.00017584249
501	Dependencies Revisited for Improving Data Quality	2008	PODS	0.00017273167
716	Guided Data Repair	2011	VLDB	0.00014546968
884	HoloDetect: Few-Shot Learning for Error Detection	2019	SIGMOD	0.00013263269
1,120	Snuba: Automating Weak Supervision to Label Training Data	2019	VLDB	0.000119406
1,344	Detecting Data Errors: Where are we and what needs to be done?	2016	VLDB	0.00010951939
1,362	Computing Optimal Repairs for Functional Dependencies	2018	PODS	0.00010912749
1,538	Query Steering for Interactive Data Exploration	2013	CIDR	0.00010314313
1,805	Raha: A Configuration-Free Error Detection System	2019	SIGMOD	9.5938877e-05
1,972	Discovery of Approximate (and Exact) Denial Constraints	2020	VLDB	9.2829634e-05
2,054	Messing Up with BART: Error Generation for Evaluating Data-Cleaning Algorithms	2016	VLDB	9.1158071e-05
2,127	Discovery of Genuine Functional Dependencies from Relational Data with Missing Values	2018	VLDB	8.9990594e-05
2,445	Approximate Denial Constraints	2020	VLDB	8.4546614e-05
2,648	Explore-by-Example: An Automatic Query Steering Framework for Interactive Data Exploration	2014	SIGMOD	8.172373e-05
3,421	UGuide – User-Guided Discovery of FD-Detectable Errors	2017	SIGMOD	7.3120636e-05
4,226	Scalable Discovery of Unique Column Combinations	2014	VLDB	6.7151469e-05
4,652	Temporal Rules Discovery for Web Data Cleaning	2016	VLDB	6.4828106e-05
9,865	HILDA'22: The SIGMOD 2022 Workshop on Human-in-the-Loop Data Analytics	2022	SIGMOD	5.1159789e-05
10,021	The Data Interaction Game	2018	SIGMOD	5.0935279e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,703	Toward Interpretable and Actionable Data Analysis with Explanations and Causality	2022	VLDB
2	4,869	Selective Data Acquisition in the Wild for Model Charging	2022	VLDB
3	12,204	Entity Matching with Active Monotone Classification	2018	PODS
4	4,499	Automating Exploratory Data Analysis via Machine Learning: An Overview	2020	SIGMOD
5	7,054	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB
6	9,453	Intelligent Agents for Data Exploration	2024	VLDB
7	3,016	Data Acquisition for Improving Machine Learning Models	2021	VLDB
8	6,165	Guided Exploration of User Groups	2020	VLDB
9	4,509	Optimization for Active Learning-based Interactive Database Exploration	2019	VLDB
10	10,021	The Data Interaction Game	2018	SIGMOD