Exploratory Training: When Annotators Learn About Data

Summary: Game-theoretic collaborative active learning where annotator beliefs evolve with data as users revise strategies. Theory and algorithms for joint user-system convergence to a shared target model, reducing interactions, validated on real-world studies. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h9c08ba71afc5c4e9
Venue: SIGMOD
Year: 2023
Pagerank: 5.2640635e-05
Overall Rank: 8,784 | 41.12%
DOI: 10.1145/3589280

Incoming Non-self Citations Over Time

Authors

1. Rajesh Shrestha (Oregon State University)
2. Omeed Habibelahian (Oregon State University)
3. Arash Termehchy (Oregon State University)
4. Paolo Papotti (EURECOM)

BibTeX Citation

@inproceedings{shrestha_sigmod23,
        title = {{Exploratory Training: When Annotators Learn About Data}},
        author = {Shrestha, Rajesh and Habibelahian, Omeed and Termehchy, Arash and Papotti, Paolo},
        series = {{SIGMOD} '23},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3589280},
        url = {https://dl.acm.org/doi/10.1145/3589280},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,073	Generation of Training Examples for Tabular Natural Language Inference	2023	SIGMOD	5.2117663e-05
11,190	User-Centric Property Graph Repairs	2025	SIGMOD	4.9605343e-05
11,487	Versatile Property Graph Transformations	2025	VLDB	4.9605343e-05

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
100	HoloClean: Holistic Data Repairs with Probabilistic Inference	2017	VLDB	0.00033951027
160	CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies	2004	SIGMOD	0.00027775994
460	ActiveClean: Interactive Data Cleaning For Statistical Modeling	2016	VLDB	0.00017836918
502	Dependencies Revisited for Improving Data Quality	2008	PODS	0.00017239652
717	Guided Data Repair	2011	VLDB	0.00014531658
853	HoloDetect: Few-Shot Learning for Error Detection	2019	SIGMOD	0.00013437314
1,123	Snuba: Automating Weak Supervision to Label Training Data	2019	VLDB	0.00011901566
1,306	Detecting Data Errors: Where are we and what needs to be done?	2016	VLDB	0.00011076619
1,314	Computing Optimal Repairs for Functional Dependencies	2018	PODS	0.000110547
1,542	Query Steering for Interactive Data Exploration	2013	CIDR	0.00010280869
1,731	Raha: A Configuration-Free Error Detection System	2019	SIGMOD	9.7585913e-05
1,980	Discovery of Approximate (and Exact) Denial Constraints	2020	VLDB	9.2527187e-05
2,053	Messing Up with BART: Error Generation for Evaluating Data-Cleaning Algorithms	2016	VLDB	9.1054597e-05
2,131	Discovery of Genuine Functional Dependencies from Relational Data with Missing Values	2018	VLDB	8.9867472e-05
2,451	Approximate Denial Constraints	2020	VLDB	8.4303553e-05
2,656	Explore-by-Example: An Automatic Query Steering Framework for Interactive Data Exploration	2014	SIGMOD	8.1460698e-05
3,432	UGuide – User-Guided Discovery of FD-Detectable Errors	2017	SIGMOD	7.288198e-05
4,245	Scalable Discovery of Unique Column Combinations	2014	VLDB	6.6958006e-05
4,653	Temporal Rules Discovery for Web Data Cleaning	2016	VLDB	6.4706099e-05
9,900	HILDA'22: The SIGMOD 2022 Workshop on Human-in-the-Loop Data Analytics	2022	SIGMOD	5.0990622e-05
10,057	The Data Interaction Game	2018	SIGMOD	5.0766855e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,718	Toward Interpretable and Actionable Data Analysis with Explanations and Causality	2022	VLDB
2	4,890	Selective Data Acquisition in the Wild for Model Charging	2022	VLDB
3	12,247	Entity Matching with Active Monotone Classification	2018	PODS
4	4,519	Automating Exploratory Data Analysis via Machine Learning: An Overview	2020	SIGMOD
5	7,074	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB
6	9,483	Intelligent Agents for Data Exploration	2024	VLDB
7	3,024	Data Acquisition for Improving Machine Learning Models	2021	VLDB
8	6,182	Guided Exploration of User Groups	2020	VLDB
9	4,526	Optimization for Active Learning-based Interactive Database Exploration	2019	VLDB
10	10,057	The Data Interaction Game	2018	SIGMOD