CLAMShell: Speeding up Crowds for Low-latency Data Labeling

Summary: Presents CLAMShell to accelerate crowds for fast labeling; builds latency taxonomy and deployment profiles. Mitigates stragglers, pool maintenance, retainer pools, and active learning yield large speedups and lower variance; validated on MTurk. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 11517
Venue: VLDB
Year: 2016
Pagerank: 6.9835263e-05
Overall Rank: 3,971 | 72.76%
DOI: 10.14778/2856318.2856324

Incoming Non-self Citations Over Time

Authors

1. Daniel Haas (University of California Berkeley)
2. Jiannan Wang (Simon Fraser University)
3. Eugene Wu (Columbia University)
4. Michael J. Franklin (University of California Berkeley)

BibTeX Citation

@article{haas_vldb16,
        title = {{CLAMShell: Speeding up Crowds for Low-latency Data Labeling}},
        author = {Haas, Daniel and Wang, Jiannan and Wu, Eugene and Franklin, Michael J.},
        journal = {PVLDB},
        series = {{VLDB} '16},
        volume = {9},
        number = {4},
        pages = {372--383},
        doi = {10.14778/2856318.2856324},
        url = {https://doi.org/10.14778/2856318.2856324},
        year = {2016}
}

Incoming Citations (Sorted by Pagerank)

Showing 12 of 12 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
1,323	Data Cleaning: Overview and Emerging Challenges	2016	SIGMOD	0.00011152602
1,643	Falcon: Scaling Up Hands-Off Crowdsourced Entity Matching to Build Cloud Services	2017	SIGMOD	0.00010134956
2,313	Active Learning for ML Enhanced Database Systems	2020	SIGMOD	8.762627e-05
3,231	Truth Inference in Crowdsourcing: Is the Problem Solved?	2017	VLDB	7.6191767e-05
3,281	Cleaning Crowdsourced Labels Using Oracles for Statistical Classification	2019	VLDB	7.5706653e-05
3,496	Waldo: An Adaptive Human Interface for Crowd Entity Resolution	2017	SIGMOD	7.3634471e-05
4,610	Top-k Sorting Under Partial Order Information	2018	SIGMOD	6.6079578e-05
4,959	QFix: Diagnosing Errors through Query Histories	2017	SIGMOD	6.4283324e-05
4,966	Rotom: A Meta-Learned Data Augmentation Framework for Entity Matching, Data Cleaning, Text Classification, and Beyond	2021	SIGMOD	6.4225454e-05
6,816	Inspector Gadget: A Data Programming-based Labeling System for Industrial Images	2021	VLDB	5.7649828e-05
6,908	Cost-Effective Data Annotation using Game-Based Crowdsourcing	2019	VLDB	5.7415834e-05
7,357	Crowdsourced Data Management: Overview and Challenges	2017	SIGMOD	5.6346837e-05

Outgoing Citations (Sorted by Pagerank)

Showing 16 of 16 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
12	C-Store: A Column-oriented DBMS	2005	VLDB	0.00069513174
38	Hekaton: SQL Server’s Memory-Optimized OLTP Engine	2013	SIGMOD	0.00047648573
51	Dremel: Interactive Analysis of Web-Scale Datasets	2010	VLDB	0.0004291425
90	CrowdDB: Answering Queries with Crowdsourcing	2011	SIGMOD	0.00034951786
196	CrowdER: Crowdsourcing Entity Resolution	2012	VLDB	0.00025780596
265	Human-powered Sorts and Joins	2012	VLDB	0.00022935368
439	Corleone: Hands-Off Crowdsourcing for Entity Matching	2014	SIGMOD	0.00018464913
516	Data Curation at Scale: The Data Tamer System	2013	CIDR	0.00017171198
1,472	M4: A Visualization-Oriented Time Series Data Aggregation	2014	VLDB	0.00010668946
1,736	A Sample-and-Clean Framework for Fast and Accurate Query Processing on Dirty Data	2014	SIGMOD	9.8984415e-05
1,903	Instant Loading for Main Memory Databases	2013	VLDB	9.5049156e-05
1,910	Counting with the Crowd	2013	VLDB	9.4972788e-05
2,626	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB	8.3291889e-05
3,314	Finish Them!: Pricing Algorithms for Human Computation	2014	VLDB	7.5279588e-05
5,056	The Case for Data Visualization Management Systems	2014	VLDB	6.3823595e-05
8,620	Wisteria: Nurturing Scalable Data Cleaning Infrastructure	2015	VLDB	5.3991092e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	1,270	CDAS: A Crowdsourcing Data Analytics System	2012	VLDB
2	1,910	Counting with the Crowd	2013	VLDB
3	20	Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud	2012	VLDB
4	10,031	tDP: An Optimal-Latency Budget Allocation Strategy for Crowdsourced MAXIMUM Operations	2015	SIGMOD
5	852	Leveraging Transitive Relations for Crowdsourced Joins	2013	SIGMOD
6	2,636	CrowdFill: Collecting Structured Data from the Crowd	2014	SIGMOD
7	9,003	Planting Trees for scalable and efficient Canonical Hub Labeling	2020	VLDB
8	9,640	Supporting Scalable Analytics with Latency Constraints	2015	VLDB
9	2,626	Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning	2015	VLDB
10	8,495	CrowdGame: A Game-Based Crowdsourcing System for Cost-Effective Data Labeling	2019	SIGMOD