Minimizing Efforts in Validating Crowd Answers

Summary: Proposes a probabilistic model to identify the most beneficial validation questions in crowdsourced answers, enhancing correctness and flagging faulty workers. Demonstrates up to 50% expert savings and near-perfect correctness after only ~20% of questions are validated. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 4951
Venue: SIGMOD
Year: 2015
Pagerank: 4.5366717e-05
Overall Rank: 8,362 | 41.83%
DOI: 10.1145/2723372.2723731

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,773	Cleaning Crowdsourced Labels Using Oracles for Statistical Classification	2019	VLDB	6.7758649e-05
7,648	User Guidance for Efficient Fact Checking	2019	VLDB	4.6889787e-05
11,770	Staging User Feedback toward Rapid Conflict Resolution in Data Fusion	2017	SIGMOD	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
55	Efficiently Updating Materialized Views	1986	SIGMOD	0.00065762967
371	A Bayesian Approach to Discovering Truth from Conflicting Sources for Data Integration	2012	VLDB	0.00025389696
509	On Active Learning of Record Matching Packages	2010	SIGMOD	0.00021409518
643	Corleone: Hands-Off Crowdsourcing for Entity Matching	2014	SIGMOD	0.00018754451
692	Pay-as-you-go User Feedback for Dataspace Systems	2008	SIGMOD	0.00018083948
833	Guided Data Repair	2011	VLDB	0.00016138432
1,246	Truth Discovery and Copying Detection in a Dynamic World	2009	VLDB	0.0001307161
1,491	CDAS: A Crowdsourcing Data Analytics System	2012	VLDB	0.00011694982
1,716	Chimera: Large-Scale Classification using Machine Learning, Rules, and Crowdsourcing	2014	VLDB	0.00010795718
2,452	Data Fusion – Resolving Data Conflicts for Integration	2009	VLDB	8.7839322e-05
3,100	Crowd Mining	2013	SIGMOD	7.5634778e-05
4,651	Whom to Ask? Jury Selection for Decision Making Tasks on Micro-blog Services	2012	VLDB	6.022931e-05
5,081	Reducing Uncertainty of Schema Matching via Crowdsourcing	2013	VLDB	5.7132042e-05
7,549	SOLOMON: Seeking the Truth Via Copying Detection	2010	VLDB	4.7137426e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
3,118	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB	7.5379338e-05
3,322	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD	7.2230626e-05
5,734	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB	5.3482904e-05
1,164	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD	0.00013564823
2,937	Truth Inference in Crowdsourcing: Is the Problem Solved?	2017	VLDB	7.853108e-05
4,479	Optimal Crowd-Powered Rating and Filtering Algorithms	2014	VLDB	6.149053e-05
1,242	Question Selection for Crowd Entity Resolution	2013	VLDB	0.00013096655
7,178	Towards Globally Optimal Crowdsourcing Quality Management: The Uniform Worker Setting	2016	SIGMOD	4.8085946e-05
4,827	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD	5.8938399e-05
11,902	The Importance of Being Expert: Efficient Max-Finding in Crowdsourcing	2015	SIGMOD	4.1945683e-05