Towards Globally Optimal Crowdsourcing Quality Management: The Uniform Worker Setting

Summary: Under a uniform worker model, the paper devises algorithms that provably achieve the global optimum of the maximum-likelihood estimates for task answers and worker quality (yes/no and rating tasks). It prunes the mapping space to preserve optimality, characterizes the problem's complexity, and often outperforms EM-based estimates. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5206
Venue: SIGMOD
Year: 2016
Pagerank: 5.5718685e-05
Overall Rank: 7,665 | 47.42%
DOI: 10.1145/2882903.2882953

Incoming Non-self Citations Over Time

Authors

1. Akash Das Sarma (Stanford University)
2. Aditya Parameswaran (University of Illinois Urbana-Champaign)
3. Jennifer Widom (Stanford University)

BibTeX Citation

@inproceedings{sarma_sigmod16,
        title = {{Towards Globally Optimal Crowdsourcing Quality Management: The Uniform Worker Setting}},
        author = {Sarma, Akash Das and Parameswaran, Aditya and Widom, Jennifer},
        series = {{SIGMOD} '16},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2882903.2882953},
        url = {https://dl.acm.org/doi/10.1145/2882903.2882953},
        year = {2016}
}

Incoming Citations (Sorted by Pagerank)

Showing 2 of 2 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
4,451	GOGGLES: Automatic Image Labeling with Affinity Coding	2020	SIGMOD	6.6952549e-05
8,807	Understanding Workers, Developing Effective Tasks, and Enhancing Marketplace Dynamics: A Study of a Large Crowdsourcing Marketplace	2017	VLDB	5.3671183e-05

Outgoing Citations (Sorted by Pagerank)

Showing 4 of 4 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
196	CrowdER: Crowdsourcing Entity Resolution	2012	VLDB	0.00025780596
997	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD	0.00012755983
1,293	Entity Resolution with Iterative Blocking	2009	SIGMOD	0.00011292804
1,443	Crowdsourcing Algorithms for Entity Resolution	2014	VLDB	0.00010773106

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	11,995	CrowdDQS: Dynamic Question Selection in Crowdsourcing Systems	2017	SIGMOD
2	3,262	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD
3	6,311	Worker Skill Estimation in Team-Based Tasks	2015	VLDB
4	12,101	The Importance of Being Expert: Efficient Max-Finding in Crowdsourcing	2015	SIGMOD
5	5,813	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB
6	3,904	QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications	2015	SIGMOD
7	5,143	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD
8	997	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD
9	8,351	Minimizing Efforts in Validating Crowd Answers	2015	SIGMOD
10	3,914	Optimal Crowd-Powered Rating and Filtering Algorithms	2014	VLDB