Designing a Scalable Crowdsourcing Platform

Summary: Describes CrowdFlower, scalable crowdsourcing turning millions of micro-tasks into a 24/7 data-labeling engine. Quality rests on gold standards, consensus and peer review; a channel marketplace enables cost–speed–quality tradeoffs with CML tooling. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 4660
Venue: SIGMOD
Year: 2012
Pagerank: -
Overall Rank: 13,665 | 6.25%
DOI: 10.1145/2213836.2213951

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Chris Van Pelt (CrowdFlower)
2. Alex Sorokin (CrowdFlower)

BibTeX Citation

@inproceedings{pelt_sigmod12,
        title = {{Designing a Scalable Crowdsourcing Platform}},
        author = {Van Pelt, Chris and Sorokin, Alex},
        series = {{SIGMOD} '12},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/2213836.2213951},
        url = {https://dl.acm.org/doi/10.1145/2213836.2213951},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 0 of 0 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	5,752	CyLog/Crowd4U: A Declarative Platform for Complex Data-centric Crowdsourcing	2012	VLDB
2	10,029	Human Factors in Crowdsourcing	2016	VLDB
3	1,270	CDAS: A Crowdsourcing Data Analytics System	2012	VLDB
4	7,357	Crowdsourced Data Management: Overview and Challenges	2017	SIGMOD
5	6,097	Crowds, Clouds, and Algorithms: Exploring the Human Side of "Big Data" Applications	2010	SIGMOD
6	90	CrowdDB: Answering Queries with Crowdsourcing	2011	SIGMOD
7	3,262	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD
8	2,626	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB
9	11,925	Crowdsourcing Analytics with CrowdCur	2018	SIGMOD
10	12,068	Collaborative Crowdsourcing with Crowd4U	2016	VLDB