Crowdsourcing Practice for Efficient Data Labeling: Aggregation, Incremental Relabeling, and Pricing

Summary: Tutorial linking theory and industry practice in crowdsourced data labeling from Yandex. Focuses on aggregation, incremental relabeling, and dynamic pricing, with real marketplace demos and practical guidance for efficient, high-quality labeling. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 5824
Venue: SIGMOD
Year: 2020
Pagerank: 4.427232e-05
Overall Rank: 8,920 | 37.95%
DOI: 10.1145/3318464.3383127

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
11,538	Quality of Sentiment Analysis Tools: The Reasons of Inconsistency	2021	VLDB	4.1945683e-05

Outgoing Citations (Sorted by Pagerank)

Showing 0 of 0 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
5,734	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB	5.3482904e-05
4,827	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD	5.8938399e-05
5,450	Crowdsourcing Applications and Platforms: A Data Management Perspective	2011	VLDB	5.5003491e-05
5,405	Truth Discovery and Crowdsourcing Aggregation: A Unified Perspective	2015	VLDB	5.5257718e-05
13,464	Designing a Scalable Crowdsourcing Platform	2012	SIGMOD	-
6,868	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB	4.9010083e-05
3,118	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB	7.5379338e-05
12,113	Mob Data Sourcing	2012	SIGMOD	4.1945683e-05
7,117	Crowdsourced Data Management: Overview and Challenges	2017	SIGMOD	4.826509e-05
8,343	CrowdGame: A Game-Based Crowdsourcing System for Cost-Effective Data Labeling	2019	SIGMOD	4.5429217e-05