QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications

Summary: QASCA is a quality-aware online task assignment system for crowdsourcing on AMT. It models probabilistic ground truths from worker answers to optimize Accuracy and F-score, delivering linear-time assignment and achieving about 8% quality gains across five real applications. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5109
Venue: SIGMOD
Year: 2015
Pagerank: 7.0304212e-05
Overall Rank: 3,904 | 73.22%
DOI: 10.1145/2723372.2749430

Incoming Non-self Citations Over Time

Authors

1. Yudian Zheng (University of Hong Kong)
2. Jiannan Wang (University of California Berkeley)
3. Guoliang Li (Tsinghua University)
4. Reynold Cheng (University of Hong Kong)
5. Jianhua Feng (Tsinghua University)

BibTeX Citation

@inproceedings{zheng_sigmod15,
        title = {{QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications}},
        author = {Zheng, Yudian and Wang, Jiannan and Li, Guoliang and Cheng, Reynold and Feng, Jianhua},
        series = {{SIGMOD} '15},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2723372.2749430},
        url = {https://dl.acm.org/doi/10.1145/2723372.2749430},
        year = {2015}
}

Incoming Citations (Sorted by Pagerank)

Showing 17 of 17 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,231	Truth Inference in Crowdsourcing: Is the Problem Solved?	2017	VLDB	7.6191767e-05
3,281	Cleaning Crowdsourced Labels Using Oracles for Statistical Classification	2019	VLDB	7.5706653e-05
4,538	Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach	2016	SIGMOD	6.6404776e-05
5,211	CDB: A Crowd-Powered Database System	2018	VLDB	6.3161892e-05
5,813	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB	6.0780519e-05
6,899	Human-in-the-loop Outlier Detection	2020	SIGMOD	5.7431609e-05
6,908	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB	5.7415834e-05
7,357	Crowdsourced Data Management: Overview and Challenges	2017	SIGMOD	5.6346837e-05
7,493	Human-in-the-loop Data Integration	2017	VLDB	5.6046905e-05
7,729	Subjective Knowledge Base Construction Powered By Crowdsourcing and Knowledge Base	2018	SIGMOD	5.5574654e-05
9,401	CDB: Optimizing Queries with Crowd-Based Selections and Joins	2017	SIGMOD	5.2755515e-05
10,305	Weighted Set Multi-Cover on Bounded Universe and Applications in Package Recommendation	2026	SIGMOD	5.093636e-05
11,806	CONCIERGE: Improving Constrained Search Results by Data Melioration	2020	VLDB	5.093636e-05
11,912	A Rating-Ranking Method for Crowdsourced Top-k Computation	2018	SIGMOD	5.093636e-05
11,956	Worker Recommendation for Crowdsourced Q&A Services: A Triple-Factor Aware Approach	2018	VLDB	5.093636e-05
11,995	CrowdDQS: Dynamic Question Selection in Crowdsourcing Systems	2017	SIGMOD	5.093636e-05
12,019	DOCS: Domain-Aware Crowdsourcing System	2017	VLDB	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 21 of 21 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
90	CrowdDB: Answering Queries with Crowdsourcing	2011	SIGMOD	0.00034951786
196	CrowdER: Crowdsourcing Entity Resolution	2012	VLDB	0.00025780596
251	Crowdsourced Databases: Query Processing with People	2011	CIDR	0.00023261113
265	Human-powered Sorts and Joins	2012	VLDB	0.00022935368
356	Efficient Parallel Set-Similarity Joins Using MapReduce	2010	SIGMOD	0.00020303289
504	A Bayesian Approach to Discovering Truth from Conflicting Sources for Data Integration	2012	VLDB	0.00017300628
743	So Who Won? Dynamic Max Discovery with the Crowd	2012	SIGMOD	0.00014421358
852	Leveraging Transitive Relations for Crowdsourced Joins	2013	SIGMOD	0.00013604253
866	Pay-as-you-go User Feedback for Dataspace Systems	2008	SIGMOD	0.00013519288
933	Question Selection for Crowd Entity Resolution	2013	VLDB	0.00013111293
955	Truth Finding on the Deep Web: Is the Problem Solved?	2013	VLDB	0.00012996675
975	Can We Beat the Prefix Filtering? An Adaptive Framework for Similarity Join and Search	2012	SIGMOD	0.00012870645
997	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD	0.00012755983
1,270	CDAS: A Crowdsourcing Data Analytics System	2012	VLDB	0.00011388453
1,273	Fusing Data with Correlations	2014	SIGMOD	0.00011384191
1,736	A Sample-and-Clean Framework for Fast and Accurate Query Processing on Dirty Data	2014	SIGMOD	9.8984415e-05
1,910	Counting with the Crowd	2013	VLDB	9.4972788e-05
2,409	Deco: A System for Declarative Crowdsourcing	2012	VLDB	8.6145078e-05
3,184	Evaluating Entity Resolution Results	2010	VLDB	7.6575598e-05
4,539	Reducing Uncertainty of Schema Matching via Crowdsourcing	2013	VLDB	6.6403822e-05
5,143	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD	6.3476753e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	2,626	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB
2	7,417	An Experimental Evaluation of Task Assignment in Spatial Crowdsourcing	2018	VLDB
3	8,351	Minimizing Efforts in Validating Crowd Answers	2015	SIGMOD
4	933	Question Selection for Crowd Entity Resolution	2013	VLDB
5	11,956	Worker Recommendation for Crowdsourced Q&A Services: A Triple-Factor Aware Approach	2018	VLDB
6	7,665	Towards Globally Optimal Crowdsourcing Quality Management: The Uniform Worker Setting	2016	SIGMOD
7	5,143	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD
8	1,270	CDAS: A Crowdsourcing Data Analytics System	2012	VLDB
9	3,262	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD
10	11,995	CrowdDQS: Dynamic Question Selection in Crowdsourcing Systems	2017	SIGMOD