A Rating-Ranking Method for Crowdsourced Top-k Computation

Summary: Introduces rating-ranking crowdsourcing for top-k, avoiding quadratic pairwise comparisons. A unified model blends coarse rating and fine ranking questions with query selection/assignment, yielding cost-efficient, accurate top-k results. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: h498da42a9cfa4a00
Venue: SIGMOD
Year: 2018
Pagerank: 4.9793485e-05
Overall Rank: 12,212 | 17.90%
DOI: 10.1145/3183713.3183762

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Kaiyu Li (Tsinghua University)
2. Xiaohang Zhang (Tsinghua University)
3. Guoliang Li (Tsinghua University)

BibTeX Citation

@inproceedings{li_sigmod18,
        title = {{A Rating-Ranking Method for Crowdsourced Top-k Computation}},
        author = {Li, Kaiyu and Zhang, Xiaohang and Li, Guoliang},
        series = {{SIGMOD} '18},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3183713.3183762},
        url = {https://dl.acm.org/doi/10.1145/3183713.3183762},
        year = {2018}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
7,042	Human-in-the-loop Outlier Detection	2020	SIGMOD	5.6142998e-05

Outgoing Citations (Sorted by Pagerank)

Showing 20 of 20 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
92	CrowdDB: Answering Queries with Crowdsourcing	2011	SIGMOD	0.00034672523
198	CrowdER: Crowdsourcing Entity Resolution	2012	VLDB	0.00025555196
257	Crowdsourced Databases: Query Processing with People	2011	CIDR	0.00022962347
266	Human-powered Sorts and Joins	2012	VLDB	0.00022739124
749	So Who Won? Dynamic Max Discovery with the Crowd	2012	SIGMOD	0.00014265279
871	Leveraging Transitive Relations for Crowdsourced Joins	2013	SIGMOD	0.00013343705
987	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD	0.00012660627
1,926	Counting with the Crowd	2013	VLDB	9.3637786e-05
2,429	Deco: A System for Declarative Crowdsourcing	2012	VLDB	8.4801295e-05
2,653	CrowdFill: Collecting Structured Data from the Crowd	2014	SIGMOD	8.1732164e-05
3,171	Truth Inference in Crowdsourcing: Is the Problem Solved?	2017	VLDB	7.5691557e-05
3,197	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD	7.5455503e-05
3,839	QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications	2015	SIGMOD	6.9921604e-05
4,623	Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach	2016	SIGMOD	6.4978811e-05
4,917	Crowdsourced Top-k Queries by Confidence-Aware Pairwise Judgments	2017	SIGMOD	6.3553844e-05
5,024	Crowdsourced Top-k Algorithms: An Experimental Evaluation	2016	VLDB	6.3096186e-05
6,397	Human-in-the-loop Data Integration	2017	VLDB	5.7989499e-05
7,500	Crowdsourced Data Management: Overview and Challenges	2017	SIGMOD	5.5083793e-05
9,583	CDB: Optimizing Queries with Crowd-Based Selections and Joins	2017	SIGMOD	5.1571823e-05
12,314	DOCS: Domain-Aware Crowdsourcing System	2017	VLDB	4.9793485e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	6,502	Ranking Objects by Exploiting Relationships: Computing Top-K over Aggregation	2006	SIGMOD
2	5,250	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD
3	3,838	Optimal Crowd-Powered Rating and Filtering Algorithms	2014	VLDB
4	8,449	Efficient Top-K Processing Over Query-Dependent Functions	2008	VLDB
5	7,394	Efficient and Generic Evaluation of Ranked Queries	2011	SIGMOD
6	5,570	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB
7	4,478	Top-k Sorting Under Partial Order Information	2018	SIGMOD
8	4,917	Crowdsourced Top-k Queries by Confidence-Aware Pairwise Judgments	2017	SIGMOD
9	12,308	A Confidence-Aware Top-k Query Processing Toolkit on Crowdsourcing	2017	VLDB
10	5,024	Crowdsourced Top-k Algorithms: An Experimental Evaluation	2016	VLDB