CrowdGame: A Game-Based Crowdsourcing System for Cost-Effective Data Labeling

Summary: CrowdGame generates candidate labeling rules and employs a game-based crowdsourcing loop to select high-coverage, high-accuracy rules, cutting labeling cost while preserving quality. A UI-enabled deployment applies these rules to entity matching and relation extraction, demonstrating scalable, cost-effective data labeling. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5791
Venue: SIGMOD
Year: 2019
Pagerank: 5.4139128e-05
Overall Rank: 8,495 | 41.72%
DOI: 10.1145/3299869.3320221

Incoming Non-self Citations Over Time

Authors

1. Tongyu Liu (Renmin University of China)
2. Jingru Yang (Renmin University of China)
3. Ju Fan (Renmin University of China)
4. Zhewei Wei (Renmin University of China)
5. Guoliang Li (Tsinghua University)
6. Xiaoyong Du (Renmin University of China)

BibTeX Citation

@inproceedings{liu_sigmod19,
        title = {{CrowdGame: A Game-Based Crowdsourcing System for Cost-Effective Data Labeling}},
        author = {Liu, Tongyu and Yang, Jingru and Fan, Ju and Wei, Zhewei and Li, Guoliang and Du, Xiaoyong},
        series = {{SIGMOD} '19},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3299869.3320221},
        url = {https://dl.acm.org/doi/10.1145/3299869.3320221},
        year = {2019}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,261	Relational Data Synthesis using Generative Adversarial Networks: A Design Space Exploration	2020	VLDB	6.7982037e-05
6,816	Inspector Gadget: A Data Programming-based Labeling System for Industrial Images	2021	VLDB	5.7649828e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
205	Snorkel: Rapid Training Data Creation with Weak Supervision	2018	VLDB	0.00025235185
1,094	Snuba: Automating Weak Supervision to Label Training Data	2019	VLDB	0.00012214617
3,262	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD	7.5846052e-05
6,908	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB	5.7415834e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,270	CDAS: A Crowdsourcing Data Analytics System	2012	VLDB
2	5,343	Crowdsourcing Applications and Platforms: A Data Management Perspective	2011	VLDB
3	5,474	CrowdQ: Crowdsourced Query Understanding	2013	CIDR
4	196	CrowdER: Crowdsourcing Entity Resolution	2012	VLDB
5	4,538	Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach	2016	SIGMOD
6	7,357	Crowdsourced Data Management: Overview and Challenges	2017	SIGMOD
7	852	Leveraging Transitive Relations for Crowdsourced Joins	2013	SIGMOD
8	4,568	CrowdMatcher: Crowd-Assisted Schema Matching	2014	SIGMOD
9	2,626	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB
10	6,908	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB