Exploratory Training: When Annotators Learn About Data

Summary: Game-theoretic collaborative active learning where annotator beliefs evolve with data as users revise strategies. Theory and algorithms for joint user-system convergence to a shared target model, reducing interactions, validated on real-world studies. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 6700
Venue: SIGMOD
Year: 2023
Pagerank: 5.4053095e-05
Overall Rank: 8,594 | 41.04%
DOI: 10.1145/3589280

Incoming Non-self Citations Over Time

Authors

1. Rajesh Shrestha (Oregon State University)
2. Omeed Habibelahian (Oregon State University)
3. Arash Termehchy (Oregon State University)
4. Paolo Papotti (EURECOM)

BibTeX Citation

@inproceedings{shrestha_sigmod23,
        title = {{Exploratory Training: When Annotators Learn About Data}},
        author = {Shrestha, Rajesh and Habibelahian, Omeed and Termehchy, Arash and Papotti, Paolo},
        series = {{SIGMOD} '23},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3589280},
        url = {https://dl.acm.org/doi/10.1145/3589280},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
8,945	Generation of Training Examples for Tabular Natural Language Inference	2023	SIGMOD	5.3480421e-05
10,681	User-Centric Property Graph Repairs	2025	SIGMOD	5.093636e-05
11,069	Versatile Property Graph Transformations	2025	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
112	HoloClean: Holistic Data Repairs with Probabilistic Inference	2017	VLDB	0.00032801121
159	CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies	2004	SIGMOD	0.00028129426
494	Dependencies Revisited for Improving Data Quality	2008	PODS	0.00017526549
582	ActiveClean: Interactive Data Cleaning For Statistical Modeling	2016	VLDB	0.00016148948
714	Guided Data Repair	2011	VLDB	0.00014662041
946	HoloDetect: Few-Shot Learning for Error Detection	2019	SIGMOD	0.00013054126
1,094	Snuba: Automating Weak Supervision to Label Training Data	2019	VLDB	0.00012214617
1,351	Detecting Data Errors: Where are we and what needs to be done?	2016	VLDB	0.00011064851
1,399	Computing Optimal Repairs for Functional Dependencies	2018	PODS	0.00010893312
1,506	Query Steering for Interactive Data Exploration	2013	CIDR	0.00010545273
1,955	Discovery of Approximate (and Exact) Denial Constraints	2020	VLDB	9.4166032e-05
2,123	Discovery of Genuine Functional Dependencies from Relational Data with Missing Values	2018	VLDB	9.1372798e-05
2,142	Messing Up with BART: Error Generation for Evaluating Data-Cleaning Algorithms	2016	VLDB	9.0975882e-05
2,297	Raha: A Configuration-Free Error Detection System	2019	SIGMOD	8.7897221e-05
2,677	Explore-by-Example: An Automatic Query Steering Framework for Interactive Data Exploration	2014	SIGMOD	8.2682753e-05
2,685	Approximate Denial Constraints	2020	VLDB	8.2594662e-05
3,380	UGuide – User-Guided Discovery of FD-Detectable Errors	2017	SIGMOD	7.457539e-05
4,183	Scalable Discovery of Unique Column Combinations	2014	VLDB	6.8467773e-05
4,598	Temporal Rules Discovery for Web Data Cleaning	2016	VLDB	6.6127851e-05
9,680	HILDA'22: The SIGMOD 2022 Workshop on Human-in-the-Loop Data Analytics	2022	SIGMOD	5.235881e-05
9,832	The Data Interaction Game	2018	SIGMOD	5.2124469e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,576	Toward Interpretable and Actionable Data Analysis with Explanations and Causality	2022	VLDB
2	4,808	Selective Data Acquisition in the Wild for Model Charging	2022	VLDB
3	11,898	Entity Matching with Active Monotone Classification	2018	PODS
4	4,395	Automating Exploratory Data Analysis via Machine Learning: An Overview	2020	SIGMOD
5	6,908	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB
6	9,293	Intelligent Agents for Data Exploration	2024	VLDB
7	2,994	Data Acquisition for Improving Machine Learning Models	2021	VLDB
8	6,039	Guided Exploration of User Groups	2020	VLDB
9	4,420	Optimization for Active Learning-based Interactive Database Exploration	2019	VLDB
10	9,832	The Data Interaction Game	2018	SIGMOD