Crowdsourcing Practice for Efficient Data Labeling: Aggregation, Incremental Relabeling, and Pricing
Summary: Tutorial linking theory and industry practice in crowdsourced data labeling from Yandex. Focuses on aggregation, incremental relabeling, and dynamic pricing, with real marketplace demos and practical guidance for efficient, high-quality labeling. (summarized by gpt-5-nano on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Alexey Drutsa (Yandex)
- 2. Valentina Fedorova (Yandex)
- 3. Dmitry Ustalov (Yandex)
- 4. Olga Megorskaya (Yandex)
- 5. Evfrosiniya Zerminova (Yandex)
- 6. Daria Baidakova (Yandex)
BibTeX Citation
@inproceedings{drutsa_sigmod20,
title = {{Crowdsourcing Practice for Efficient Data Labeling: Aggregation, Incremental Relabeling, and Pricing}},
author = {Drutsa, Alexey and Fedorova, Valentina and Ustalov, Dmitry and Megorskaya, Olga and Zerminova, Evfrosiniya and Baidakova, Daria},
series = {{SIGMOD} '20},
booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
publisher = {Association for Computing Machinery},
doi = {10.1145/3318464.3383127},
url = {https://dl.acm.org/doi/10.1145/3318464.3383127},
year = {2020}
}
Incoming Citations (Sorted by Pagerank)
Showing 1 of 1 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 11,735 | Quality of Sentiment Analysis Tools: The Reasons of Inconsistency | 2021 | VLDB | 5.093636e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 0 of 0 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 5,813 | Efficient Algorithms for Crowd-Aided Categorization | 2020 | VLDB |
| 2 | 5,143 | An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems | 2013 | SIGMOD |
| 3 | 5,343 | Crowdsourcing Applications and Platforms: A Data Management Perspective | 2011 | VLDB |
| 4 | 5,981 | Truth Discovery and Crowdsourcing Aggregation: A Unified Perspective | 2015 | VLDB |
| 5 | 13,665 | Designing a Scalable Crowdsourcing Platform | 2012 | SIGMOD |
| 6 | 6,908 | Cost-Effective Data Annotation using Game-Based Crowdsourcing | 2019 | VLDB |
| 7 | 2,626 | Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning | 2015 | VLDB |
| 8 | 12,310 | Mob Data Sourcing | 2012 | SIGMOD |
| 9 | 7,357 | Crowdsourced Data Management: Overview and Challenges | 2017 | SIGMOD |
| 10 | 8,495 | CrowdGame: A Game-Based Crowdsourcing System for Cost-Effective Data Labeling | 2019 | SIGMOD |