DBScholar

Back to papers

Crowdsourced Databases: Query Processing with People

Summary: Presents Qurk, a crowdsourced DBMS that exposes humans on MTurk as query operators to perform filtering, sorting, joining, and aggregation via HIT workflows. Analyzes execution and optimization challenges unique to human-in-the-loop queries—cost/latency/accuracy tradeoffs, batching, operator design, and techniques to minimize HITs and improve result quality. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
h4f9346900ff72c4d
Venue
CIDR
Year
2011
Pagerank
0.00022914527
Overall Rank
257 | 98.28%
DOI
-
PDF
Download (CC BY 3.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{marcus_cidr11,
        address = {Amsterdam, Netherlands},
        series = {{CIDR} '11},
        title = {{Crowdsourced Databases: Query Processing with People}},
        booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
        author = {Marcus, Adam and Wu, Eugene and Karger, David R. and Madden, Samuel and Miller, Robert C.},
        year = {2011}
}

Incoming Citations (Sorted by Pagerank)

Showing 48 of 48 citing papers.

Rank Citing Paper Year Venue Pagerank
92 CrowdDB: Answering Queries with Crowdsourcing 2011 SIGMOD 0.00034613457
199 CrowdER: Crowdsourcing Entity Resolution 2012 VLDB 0.0002551523
266 Human-powered Sorts and Joins 2012 VLDB 0.00022701006
434 Corleone: Hands-Off Crowdsourcing for Entity Matching 2014 SIGMOD 0.00018323777
630 DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing 2025 VLDB 0.00015402076
991 CrowdScreen: Algorithms for Filtering Data with Humans 2012 SIGMOD 0.00012632003
1,267 CDAS: A Crowdsourcing Data Analytics System 2012 VLDB 0.00011253716
1,650 Falcon: Scaling Up Hands-Off Crowdsourced Entity Matching to Build Cloud Services 2017 SIGMOD 9.9690948e-05
1,932 Counting with the Crowd 2013 VLDB 9.3441665e-05
2,205 Demonstration of Qurk: A Query Processor for Human Operators 2011 SIGMOD 8.8525313e-05
2,618 dbTouch: Analytics at your Fingertips 2013 CIDR 8.1988906e-05
2,842 Crowd Mining 2013 SIGMOD 7.9345347e-05
3,028 Truth Inference in Crowdsourcing: Is the Problem Solved? 2017 VLDB 7.7177453e-05
3,210 iCrowd: An Adaptive Crowdsourcing Framework 2015 SIGMOD 7.5170875e-05
3,659 Revisiting Prompt Engineering via Declarative Crowdsourcing 2024 CIDR 7.1094444e-05
3,841 Optimal Crowd-Powered Rating and Filtering Algorithms 2014 VLDB 6.9697568e-05
3,849 QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications 2015 SIGMOD 6.9657571e-05
4,237 Whom to Ask? Jury Selection for Decision Making Tasks on Micro-blog Services 2012 VLDB 6.7026224e-05
4,537 Arnold: Declarative Crowd-Machine Data Integration 2013 CIDR 6.5399966e-05
4,559 Databases Unbound: Querying All of the World’s Bytes with AI 2024 VLDB 6.5238675e-05
4,626 Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach 2016 SIGMOD 6.480594e-05
4,812 Hybrid Querying Over Relational Databases and Large Language Models 2025 CIDR 6.3907046e-05
4,996 SemBench: A Benchmark for Semantic Query Processing Engines 2026 VLDB 6.3095672e-05
5,051 CDB: A Crowd-Powered Database System 2018 VLDB 6.2863221e-05
5,276 An Online Cost Sensitive Decision-Making Method in Crowdsourcing Systems 2013 SIGMOD 6.1866921e-05
5,478 Crowdsourcing Applications and Platforms: A Data Management Perspective 2011 VLDB 6.0999674e-05
5,586 Efficient Algorithms for Crowd-Aided Categorization 2020 VLDB 6.0582193e-05
5,627 CrowdQ: Crowdsourced Query Understanding 2013 CIDR 6.0440058e-05
6,404 Human-in-the-loop Data Integration 2017 VLDB 5.7797097e-05
6,791 Crowdsourced Data Management: Overview and Challenges 2017 SIGMOD 5.6647309e-05
6,881 Estimating the Impact of Unknown Unknowns on Aggregate Query Results 2016 SIGMOD 5.6415853e-05
6,948 Interactive Graph Search 2019 SIGMOD 5.6210529e-05
6,959 Inspector Gadget: A Data Programming-based Labeling System for Industrial Images 2021 VLDB 5.6184124e-05
7,064 Human-in-the-loop Outlier Detection 2020 SIGMOD 5.5930864e-05
7,113 Answering Planning Queries with the Crowd 2013 VLDB 5.5807503e-05
7,805 Pushing the Boundaries of Crowd-enabled Databases with Query-driven Schema Expansion 2012 VLDB 5.4329573e-05
7,882 Semantic Integrity Constraints: Declarative Guardrails for AI-Augmented Data Processing Systems 2025 VLDB 5.4181983e-05
8,972 Managing General and Individual Knowledge in Crowd Mining Applications 2015 CIDR 5.2347984e-05
8,982 Understanding Workers, Developing Effective Tasks, and Enhancing Marketplace Dynamics: A Study of a Large Crowdsourcing Marketplace 2017 VLDB 5.2317838e-05
9,408 Query-Guided Resolution in Uncertain Databases 2023 SIGMOD 5.1672232e-05
9,617 CDB: Optimizing Queries with Crowd-Based Selections and Joins 2017 SIGMOD 5.1376962e-05
11,190 User-Centric Property Graph Repairs 2025 SIGMOD 4.9605343e-05
12,202 Anti-Freeze for Large and Complex Spreadsheets: Asynchronous Formula Computation 2019 SIGMOD 4.9605343e-05
12,261 A Rating-Ranking Method for Crowdsourced Top-k Computation 2018 SIGMOD 4.9605343e-05
12,357 A Confidence-Aware Top-k Query Processing Toolkit on Crowdsourcing 2017 VLDB 4.9605343e-05
12,363 DOCS: Domain-Aware Crowdsourcing System 2017 VLDB 4.9605343e-05
12,443 The Importance of Being Expert: Efficient Max-Finding in Crowdsourcing 2015 SIGMOD 4.9605343e-05
12,650 Mob Data Sourcing 2012 SIGMOD 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 1 of 1 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
260 Answering Queries using Humans, Algorithms and Databases 2011 CIDR 0.00022866018
Previous Page 1 / 1 Next

Semantically Similar Papers