Optimal Crowd-Powered Rating and Filtering Algorithms

Summary: Optimal crowd-powered filtering for data management; relaxes prior assumptions. Two approaches: a generalization yielding optimal but intractable solutions, and an efficient near-optimal strategy; achieves up to 30% error reduction in peer evaluation. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 11133
Venue: VLDB
Year: 2014
Pagerank: 7.0204052e-05
Overall Rank: 3,914 | 73.15%
DOI: 10.14778/2732232.2732234

Incoming Non-self Citations Over Time

Authors

1. Aditya Parameswaran (Stanford University; University of Illinois Urbana-Champaign)
2. Stephen Boyd (Stanford University)
3. Hector Garcia-Molina (Stanford University)
4. Ashish Gupta (Stanford University)
5. Neoklis Polyzotis (Google; University of California Santa Cruz)
6. Jennifer Widom (Stanford University)

BibTeX Citation

@article{parameswaran_vldb14,
        title = {{Optimal Crowd-Powered Rating and Filtering Algorithms}},
        author = {Parameswaran, Aditya and Boyd, Stephen and Garcia-Molina, Hector and Gupta, Ashish and Polyzotis, Neoklis and Widom, Jennifer},
        journal = {PVLDB},
        series = {{VLDB} '14},
        volume = {7},
        number = {9},
        pages = {685--696},
        doi = {10.14778/2732232.2732234},
        url = {https://doi.org/10.14778/2732232.2732234},
        year = {2014}
}

Incoming Citations (Sorted by Pagerank)

Showing 12 of 12 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
2,570	Query-Oriented Data Cleaning with Oracles	2015	SIGMOD	8.4061164e-05
3,262	iCrowd: An Adaptive Crowdsourcing Framework	2015	SIGMOD	7.5846052e-05
3,314	Finish Them!: Pricing Algorithms for Human Computation	2014	VLDB	7.5279588e-05
5,813	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB	6.0780519e-05
7,357	Crowdsourced Data Management: Overview and Challenges	2017	SIGMOD	5.6346837e-05
7,788	A Natural Language Interface for Querying General and Individual Knowledge	2015	VLDB	5.5446256e-05
8,788	Managing General and Individual Knowledge in Crowd Mining Applications	2015	CIDR	5.3726124e-05
8,807	Understanding Workers, Developing Effective Tasks, and Enhancing Marketplace Dynamics: A Study of a Large Crowdsourcing Marketplace	2017	VLDB	5.3671183e-05
9,401	CDB: Optimizing Queries with Crowd-Based Selections and Joins	2017	SIGMOD	5.2755515e-05
9,812	Interactive Graph Search for Multiple Targets on DAGs	2025	VLDB	5.214913e-05
10,382	LLM-Powered Interactive Graph Search: A Scalable and Practical Approach	2026	SIGMOD	5.093636e-05
12,101	The Importance of Being Expert: Efficient Max-Finding in Crowdsourcing	2015	SIGMOD	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
90	CrowdDB: Answering Queries with Crowdsourcing	2011	SIGMOD	0.00034951786
196	CrowdER: Crowdsourcing Entity Resolution	2012	VLDB	0.00025780596
251	Crowdsourced Databases: Query Processing with People	2011	CIDR	0.00023261113
265	Human-powered Sorts and Joins	2012	VLDB	0.00022935368
743	So Who Won? Dynamic Max Discovery with the Crowd	2012	SIGMOD	0.00014421358
997	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD	0.00012755983
2,789	Crowd Mining	2013	SIGMOD	8.1203892e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	7,222	Hear the Whole Story: Towards the Diversity of Opinion in Crowdsourcing Markets	2015	VLDB
2	12,013	A Confidence-Aware Top-k Query Processing Toolkit on Crowdsourcing	2017	VLDB
3	4,915	Crowdsourced Top-k Algorithms: An Experimental Evaluation	2016	VLDB
4	11,912	A Rating-Ranking Method for Crowdsourced Top-k Computation	2018	SIGMOD
5	5,143	An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems	2013	SIGMOD
6	12,101	The Importance of Being Expert: Efficient Max-Finding in Crowdsourcing	2015	SIGMOD
7	8,351	Minimizing Efforts in Validating Crowd Answers	2015	SIGMOD
8	5,813	Efficient Algorithms for Crowd-Aided Categorization	2020	VLDB
9	7,665	Towards Globally Optimal Crowdsourcing Quality Management: The Uniform Worker Setting	2016	SIGMOD
10	997	CrowdScreen: Algorithms for Filtering Data with Humans	2012	SIGMOD