Cost-Effective Data Annotation using Game-Based Crowdsourcing
Summary: CrowdGame enables cost-efficient data annotation via game-based crowdsourcing. Two worker cohorts (rule generators and rule refuters) play a minimax game to optimize coverage vs. precision, using Bayesian estimation for rule quality and task selection; demonstrated on entity matching and relation extraction, outperforming SOTA. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Jingru Yang (Renmin University of China)
- 2. Ju Fan (Renmin University of China)
- 3. Zhewei Wei (Beihang University; Renmin University of China)
- 4. Guoliang Li (Tsinghua University)
- 5. Tongyu Liu (Renmin University of China)
- 6. Xiaoyong Du (Renmin University of China)
BibTeX Citation
@article{yang_vldb19,
title = {{Cost-Effective Data Annotation using Game-Based Crowdsourcing}},
author = {Yang, Jingru and Fan, Ju and Wei, Zhewei and Li, Guoliang and Liu, Tongyu and Du, Xiaoyong},
journal = {PVLDB},
series = {{VLDB} '19},
volume = {12},
number = {1},
pages = {57--70},
doi = {10.14778/3275536.3275541},
url = {https://doi.org/10.14778/3275536.3275541},
year = {2019}
}Incoming Citations (Sorted by Pagerank)
Showing 8 of 8 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 4,329 | Relational Data Synthesis using Generative Adversarial Networks: A Design Space Exploration | 2020 | VLDB | 6.6597797e-05 |
| 5,814 | Domain Adaptation for Deep Entity Resolution | 2022 | SIGMOD | 5.9852808e-05 |
| 5,850 | Adaptive Rule Discovery for Labeling Text Data | 2021 | SIGMOD | 5.9719174e-05 |
| 8,476 | DADER: Hands-Off Entity Resolution with Domain Adaptation | 2022 | VLDB | 5.333639e-05 |
| 8,664 | CrowdGame: A Game-Based Crowdsourcing System for Cost-Effective Data Labeling | 2019 | SIGMOD | 5.2926663e-05 |
| 10,006 | Hierarchical Entity Resolution using an Oracle | 2022 | SIGMOD | 5.0979044e-05 |
| 10,244 | Towards Interpretable and Learnable Risk Analysis for Entity Resolution | 2020 | SIGMOD | 5.0525742e-05 |
| 10,578 | LLM-Powered Interactive Graph Search: A Scalable and Practical Approach | 2026 | SIGMOD | 4.9793485e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 20 of 20 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 5,379 | Efficient Knowledge Graph Accuracy Evaluation | 2019 | VLDB |
| 2 | 2,642 | Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning | 2015 | VLDB |
| 3 | 5,250 | An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems | 2013 | SIGMOD |
| 4 | 198 | CrowdER: Crowdsourcing Entity Resolution | 2012 | VLDB |
| 5 | 4,661 | CrowdMatcher: Crowd-Assisted Schema Matching | 2014 | SIGMOD |
| 6 | 7,500 | Crowdsourced Data Management: Overview and Challenges | 2017 | SIGMOD |
| 7 | 871 | Leveraging Transitive Relations for Crowdsourced Joins | 2013 | SIGMOD |
| 8 | 3,171 | Truth Inference in Crowdsourcing: Is the Problem Solved? | 2017 | VLDB |
| 9 | 4,623 | Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach | 2016 | SIGMOD |
| 10 | 8,664 | CrowdGame: A Game-Based Crowdsourcing System for Cost-Effective Data Labeling | 2019 | SIGMOD |