Minimizing Efforts in Validating Crowd Answers

Summary: Proposes a probabilistic model to identify the most beneficial validation questions in crowdsourced answers, enhancing correctness and flagging faulty workers. Demonstrates up to 50% expert savings and near-perfect correctness after only ~20% of questions are validated. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 4952
Venue: SIGMOD
Year: 2015
Pagerank: 4.5324924e-05
Overall Rank: 8,358 | 41.92%
DOI: 10.1145/2723372.2723731

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 3 of 3 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,767	Cleaning Crowdsourced Labels Using Oracles for Statistical Classification	2019	VLDB	6.7748725e-05
7,647	User Guidance for Efficient Fact Checking	2019	VLDB	4.6844821e-05
11,778	Staging User Feedback toward Rapid Conflict Resolution in Data Fusion	2017	SIGMOD	4.1905499e-05

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
55	Efficiently Updating Materialized Views	1986	SIGMOD	0.00065687017
372	A Bayesian Approach to Discovering Truth from Conflicting Sources for Data Integration	2012	VLDB	0.00025371138
507	On Active Learning of Record Matching Packages	2010	SIGMOD	0.00021474096
641	Corleone: Hands-Off Crowdsourcing for Entity Matching	2014	SIGMOD	0.00018759417
686	Pay-as-you-go User Feedback for Dataspace Systems	2008	SIGMOD	0.00018107664
830	Guided Data Repair	2011	VLDB	0.00016125759
1,245	Truth Discovery and Copying Detection in a Dynamic World	2009	VLDB	0.00013080304
1,491	CDAS: A Crowdsourcing Data Analytics System	2012	VLDB	0.00011685731
1,713	Chimera: Large-Scale Classification using Machine Learning, Rules, and Crowdsourcing	2014	VLDB	0.00010785323
2,453	Data Fusion – Resolving Data Conflicts for Integration	2009	VLDB	8.7815427e-05
3,104	Crowd Mining	2013	SIGMOD	7.5577057e-05
4,648	Whom to Ask? Jury Selection for Decision Making Tasks on Micro-blog Services	2012	VLDB	6.01726e-05
5,080	Reducing Uncertainty of Schema Matching via Crowdsourcing	2013	VLDB	5.7081186e-05
7,547	SOLOMON: Seeking the Truth Via Copying Detection	2010	VLDB	4.7093124e-05

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
3,020	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB	7.6987905e-05
3,324	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD	7.2163115e-05
5,744	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB	5.343155e-05
1,154	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD	0.00013616867
2,944	Truth Inference in Crowdsourcing: Is the Problem Solved?	2017	VLDB	7.8457167e-05
4,483	Optimal Crowd-Powered Rating and Filtering Algorithms	2014	VLDB	6.1431599e-05
1,242	Question Selection for Crowd Entity Resolution	2013	VLDB	0.00013088272
7,178	Towards Globally Optimal Crowdsourcing Quality Management: The Uniform Worker Setting	2016	SIGMOD	4.80398e-05
4,832	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD	5.8883457e-05
11,910	The Importance of Being Expert: Efficient Max-Finding in Crowdsourcing	2015	SIGMOD	4.1905499e-05