Back to papers
Truth Inference in Crowdsourcing: Is the Problem Solved?
Summary: Comprehensive survey of truth inference in crowdsourcing, detailing task types, task/worker modeling, and inference techniques. Systematic empirical comparison of 17 methods on 5 real datasets using a unified experimental framework to help practitioners select suitable algorithms.
(summarized by gpt-5-nano on Feb 09 2026)
- Paper ID
- 11550
- Venue
- VLDB
- Year
- 2017
- Pagerank
- 7.853108e-05
- Overall Rank
- 2,937 | 79.57%
- DOI
-
-
Incoming Non-self Citations Over Time
Incoming Citations (Sorted by Pagerank)
Showing 16 of 16 citing papers.
| Rank |
Citing Paper |
Year |
Venue |
Pagerank |
| 3,473 |
AI Meets Database: AI4DB and DB4AI |
2021 |
SIGMOD |
7.062864e-05 |
| 3,773 |
Cleaning Crowdsourced Labels Using Oracles for Statistical Classification |
2019 |
VLDB |
6.7758649e-05 |
| 5,279 |
CDB: A Crowd-Powered Database System |
2018 |
VLDB |
5.5902418e-05 |
| 5,412 |
Mining an "Anti-Knowledge Base" from Wikipedia Updates with Applications to Fact Checking and Beyond |
2020 |
VLDB |
5.5207515e-05 |
| 5,734 |
Efficient Algorithms for Crowd-Aided Categorization |
2020 |
VLDB |
5.3482904e-05 |
| 6,780 |
Domain-Aware Multi-Truth Discovery from Conflicting Sources |
2018 |
VLDB |
4.9277708e-05 |
| 6,868 |
Cost-Effective Data Annotation using Game-Based Crowdsourcing |
2019 |
VLDB |
4.9010083e-05 |
| 7,117 |
Crowdsourced Data Management: Overview and Challenges |
2017 |
SIGMOD |
4.826509e-05 |
| 7,575 |
Human-in-the-loop Outlier Detection |
2020 |
SIGMOD |
4.7068909e-05 |
| 7,668 |
Human-in-the-loop Data Integration |
2017 |
VLDB |
4.6834075e-05 |
| 9,409 |
Ground Truth Inference for Weakly Supervised Entity Matching |
2023 |
SIGMOD |
4.3441378e-05 |
| 11,401 |
SA-Q: Observing, Evaluating, and Enhancing the Quality of the Results of Sentiment Analysis Tools |
2022 |
VLDB |
4.1945683e-05 |
| 11,538 |
Quality of Sentiment Analysis Tools: The Reasons of Inconsistency |
2021 |
VLDB |
4.1945683e-05 |
| 11,707 |
A Rating-Ranking Method for Crowdsourced Top-k Computation |
2018 |
SIGMOD |
4.1945683e-05 |
| 11,788 |
CDB: Optimizing Queries with Crowd-Based Selections and Joins |
2017 |
SIGMOD |
4.1945683e-05 |
| 11,799 |
Truth Discovery for Spatio-Temporal Events from Crowdsourced Data |
2017 |
VLDB |
4.1945683e-05 |
Outgoing Citations (Sorted by Pagerank)
Showing 16 of 16 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank |
Cited Paper |
Year |
Venue |
Pagerank |
| 94 |
CrowdDB: Answering Queries with Crowdsourcing |
2011 |
SIGMOD |
0.00051013264 |
| 249 |
Crowdsourced Databases: Query Processing with People |
2011 |
CIDR |
0.00030740523 |
| 263 |
CrowdER: Crowdsourcing Entity Resolution |
2012 |
VLDB |
0.00029862413 |
| 267 |
Human-powered Sorts and Joins |
2012 |
VLDB |
0.00029690405 |
| 859 |
So Who Won? Dynamic Max Discovery with the Crowd |
2012 |
SIGMOD |
0.00015870894 |
| 1,164 |
CrowdScreen: Algorithms for Filtering Data with Humans |
2012 |
SIGMOD |
0.00013564823 |
| 1,242 |
Question Selection for Crowd Entity Resolution |
2013 |
VLDB |
0.00013096655 |
| 1,491 |
CDAS: A Crowdsourcing Data Analytics System |
2012 |
VLDB |
0.00011694982 |
| 2,567 |
Resolving Conflicts in Heterogeneous Data by Truth Discovery and Source Reliability Estimation |
2014 |
SIGMOD |
8.5239306e-05 |
| 2,792 |
Finish Them!: Pricing Algorithms for Human Computation |
2014 |
VLDB |
8.1197186e-05 |
| 3,263 |
QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications |
2015 |
SIGMOD |
7.3097573e-05 |
| 3,322 |
iCrowd: An Adaptive Crowdsourcing Framework |
2015 |
SIGMOD |
7.2230626e-05 |
| 4,011 |
A Confidence-Aware Approach for Truth Discovery on Long-Tail Data |
2015 |
VLDB |
6.5343479e-05 |
| 4,451 |
CLAMShell: Speeding up Crowds for Low-latency Data Labeling |
2016 |
VLDB |
6.1738675e-05 |
| 4,579 |
Crowdsourced Top-k Algorithms: An Experimental Evaluation |
2016 |
VLDB |
6.070469e-05 |
| 5,362 |
Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach |
2016 |
SIGMOD |
5.5473503e-05 |
Semantically Similar Papers
| Overall Rank |
Paper |
Year |
Venue |
Pagerank |
| 4,579 |
Crowdsourced Top-k Algorithms: An Experimental Evaluation |
2016 |
VLDB |
6.070469e-05 |
| 5,362 |
Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach |
2016 |
SIGMOD |
5.5473503e-05 |
| 5,450 |
Crowdsourcing Applications and Platforms: A Data Management Perspective |
2011 |
VLDB |
5.5003491e-05 |
| 4,827 |
An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems |
2013 |
SIGMOD |
5.8938399e-05 |
| 7,117 |
Crowdsourced Data Management: Overview and Challenges |
2017 |
SIGMOD |
4.826509e-05 |
| 8,362 |
Minimizing Efforts in Validating Crowd Answers |
2015 |
SIGMOD |
4.5366717e-05 |
| 6,868 |
Cost-Effective Data Annotation using Game-Based Crowdsourcing |
2019 |
VLDB |
4.9010083e-05 |
| 1,242 |
Question Selection for Crowd Entity Resolution |
2013 |
VLDB |
0.00013096655 |
| 1,841 |
Crowdsourcing Algorithms for Entity Resolution |
2014 |
VLDB |
0.00010348858 |
| 5,405 |
Truth Discovery and Crowdsourcing Aggregation: A Unified Perspective |
2015 |
VLDB |
5.5257718e-05 |