Back to papers
Truth Inference in Crowdsourcing: Is the Problem Solved?
Summary: Comprehensive survey of truth inference in crowdsourcing, detailing task types, task/worker modeling, and inference techniques. Systematic empirical comparison of 17 methods on 5 real datasets using a unified experimental framework to help practitioners select suitable algorithms.
(summarized by gpt-5-nano on Feb 09 2026)
- Paper ID
- 11551
- Venue
- VLDB
- Year
- 2017
- Pagerank
- 7.8457167e-05
- Overall Rank
- 2,944 | 79.55%
- DOI
-
-
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 16 of 16 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 3,466 |
AI Meets Database: AI4DB and DB4AI |
2021 |
SIGMOD |
7.0645718e-05 |
| 3,767 |
Cleaning Crowdsourced Labels Using Oracles for Statistical Classification |
2019 |
VLDB |
6.7748725e-05 |
| 5,254 |
CDB: A Crowd-Powered Database System |
2018 |
VLDB |
5.5991922e-05 |
| 5,420 |
Mining an "Anti-Knowledge Base" from Wikipedia Updates with Applications to Fact Checking and Beyond |
2020 |
VLDB |
5.5154502e-05 |
| 5,744 |
Efficient Algorithms for Crowd-Aided Categorization |
2020 |
VLDB |
5.343155e-05 |
| 6,780 |
Domain-Aware Multi-Truth Discovery from Conflicting Sources |
2018 |
VLDB |
4.9230376e-05 |
| 6,873 |
Cost-Effective Data Annotation using Game-Based Crowdsourcing |
2019 |
VLDB |
4.8963037e-05 |
| 7,116 |
Crowdsourced Data Management: Overview and Challenges |
2017 |
SIGMOD |
4.8219732e-05 |
| 7,580 |
Human-in-the-loop Outlier Detection |
2020 |
SIGMOD |
4.7023767e-05 |
| 7,667 |
Human-in-the-loop Data Integration |
2017 |
VLDB |
4.6791871e-05 |
| 9,415 |
Ground Truth Inference for Weakly Supervised Entity Matching |
2023 |
SIGMOD |
4.3399748e-05 |
| 11,404 |
SA-Q: Observing, Evaluating, and Enhancing the Quality of the Results of Sentiment Analysis Tools |
2022 |
VLDB |
4.1905499e-05 |
| 11,542 |
Quality of Sentiment Analysis Tools: The Reasons of Inconsistency |
2021 |
VLDB |
4.1905499e-05 |
| 11,713 |
A Rating-Ranking Method for Crowdsourced Top-k Computation |
2018 |
SIGMOD |
4.1905499e-05 |
| 11,796 |
CDB: Optimizing Queries with Crowd-Based Selections and Joins |
2017 |
SIGMOD |
4.1905499e-05 |
| 11,807 |
Truth Discovery for Spatio-Temporal Events from Crowdsourced Data |
2017 |
VLDB |
4.1905499e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 16 of 16 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank |
Cited Paper |
Year |
Venue |
Pagerank |
| 94 |
CrowdDB: Answering Queries with Crowdsourcing |
2011 |
SIGMOD |
0.00051273089 |
| 246 |
Crowdsourced Databases: Query Processing with People |
2011 |
CIDR |
0.00030952631 |
| 265 |
CrowdER: Crowdsourcing Entity Resolution |
2012 |
VLDB |
0.00029904018 |
| 266 |
Human-powered Sorts and Joins |
2012 |
VLDB |
0.00029884758 |
| 854 |
So Who Won? Dynamic Max Discovery with the Crowd |
2012 |
SIGMOD |
0.00015879917 |
| 1,154 |
CrowdScreen: Algorithms for Filtering Data with Humans |
2012 |
SIGMOD |
0.00013616867 |
| 1,242 |
Question Selection for Crowd Entity Resolution |
2013 |
VLDB |
0.00013088272 |
| 1,491 |
CDAS: A Crowdsourcing Data Analytics System |
2012 |
VLDB |
0.00011685731 |
| 2,571 |
Resolving Conflicts in Heterogeneous Data by Truth Discovery and Source Reliability Estimation |
2014 |
SIGMOD |
8.5157375e-05 |
| 2,796 |
Finish Them!: Pricing Algorithms for Human Computation |
2014 |
VLDB |
8.1119959e-05 |
| 3,268 |
QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications |
2015 |
SIGMOD |
7.3027561e-05 |
| 3,324 |
iCrowd: An Adaptive Crowdsourcing Framework |
2015 |
SIGMOD |
7.2163115e-05 |
| 4,016 |
A Confidence-Aware Approach for Truth Discovery on Long-Tail Data |
2015 |
VLDB |
6.5280895e-05 |
| 4,453 |
CLAMShell: Speeding up Crowds for Low-latency Data Labeling |
2016 |
VLDB |
6.1690121e-05 |
| 4,580 |
Crowdsourced Top-k Algorithms: An Experimental Evaluation |
2016 |
VLDB |
6.0646299e-05 |
| 5,369 |
Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach |
2016 |
SIGMOD |
5.5436995e-05 |
Semantically Similar Papers
| Overall Rank |
Paper |
Year |
Venue |
Pagerank |
| 4,580 |
Crowdsourced Top-k Algorithms: An Experimental Evaluation |
2016 |
VLDB |
6.0646299e-05 |
| 5,369 |
Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach |
2016 |
SIGMOD |
5.5436995e-05 |
| 5,458 |
Crowdsourcing Applications and Platforms: A Data Management Perspective |
2011 |
VLDB |
5.4952003e-05 |
| 4,832 |
An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems |
2013 |
SIGMOD |
5.8883457e-05 |
| 7,116 |
Crowdsourced Data Management: Overview and Challenges |
2017 |
SIGMOD |
4.8219732e-05 |
| 8,358 |
Minimizing Efforts in Validating Crowd Answers |
2015 |
SIGMOD |
4.5324924e-05 |
| 6,873 |
Cost-Effective Data Annotation using Game-Based Crowdsourcing |
2019 |
VLDB |
4.8963037e-05 |
| 1,842 |
Crowdsourcing Algorithms for Entity Resolution |
2014 |
VLDB |
0.00010342022 |
| 1,242 |
Question Selection for Crowd Entity Resolution |
2013 |
VLDB |
0.00013088272 |
| 5,411 |
Truth Discovery and Crowdsourcing Aggregation: A Unified Perspective |
2015 |
VLDB |
5.5206368e-05 |