DBScholar

Back to papers

Human-powered Sorts and Joins

Summary: Integrates crowds into a declarative workflow (Qurk) to support sorting and joining via MTurk in a DBMS-like interface. Uses task batching, numeric ratings, and pre-filtering to dramatically reduce cost while preserving accuracy and latency. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
10520
Venue
VLDB
Year
2012
Pagerank
0.00022935368
Overall Rank
265 | 98.19%
DOI
10.14778/2047485.2047487

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{marcus_vldb12,
        title = {{Human-powered Sorts and Joins}},
        author = {Marcus, Adam and Wu, Eugene and Karger, David and Madden, Samuel and Miller, Robert},
        journal = {PVLDB},
        series = {{VLDB} '12},
        volume = {5},
        number = {1},
        pages = {13--24},
        doi = {10.14778/2047485.2047487},
        url = {https://doi.org/10.14778/2047485.2047487},
        year = {2012}
}

Incoming Citations (Sorted by Pagerank)

Showing 41 of 41 citing papers.

Rank Citing Paper Year Venue Pagerank
141 Deep Entity Matching with Pre-Trained Language Models 2021 VLDB 0.0002964847
196 CrowdER: Crowdsourcing Entity Resolution 2012 VLDB 0.00025780596
439 Corleone: Hands-Off Crowdsourcing for Entity Matching 2014 SIGMOD 0.00018464913
743 So Who Won? Dynamic Max Discovery with the Crowd 2012 SIGMOD 0.00014421358
852 Leveraging Transitive Relations for Crowdsourced Joins 2013 SIGMOD 0.00013604253
933 Question Selection for Crowd Entity Resolution 2013 VLDB 0.00013111293
997 CrowdScreen: Algorithms for Filtering Data with Humans 2012 SIGMOD 0.00012755983
1,443 Crowdsourcing Algorithms for Entity Resolution 2014 VLDB 0.00010773106
1,643 Falcon: Scaling Up Hands-Off Crowdsourced Entity Matching to Build Cloud Services 2017 SIGMOD 0.00010134956
1,910 Counting with the Crowd 2013 VLDB 9.4972788e-05
2,626 Scaling Up Crowd-Sourcing to Very Large Datasets: A Case for Active Learning 2015 VLDB 8.3291889e-05
3,231 Truth Inference in Crowdsourcing: Is the Problem Solved? 2017 VLDB 7.6191767e-05
3,262 iCrowd: An Adaptive Crowdsourcing Framework 2015 SIGMOD 7.5846052e-05
3,314 Finish Them!: Pricing Algorithms for Human Computation 2014 VLDB 7.5279588e-05
3,496 Waldo: An Adaptive Human Interface for Crowd Entity Resolution 2017 SIGMOD 7.3634471e-05
3,904 QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications 2015 SIGMOD 7.0304212e-05
3,914 Optimal Crowd-Powered Rating and Filtering Algorithms 2014 VLDB 7.0204052e-05
3,971 CLAMShell: Speeding up Crowds for Low-latency Data Labeling 2016 VLDB 6.9835263e-05
3,983 Crowd-Based Deduplication: An Adaptive Approach 2015 SIGMOD 6.9739851e-05
4,050 Revisiting Prompt Engineering via Declarative Crowdsourcing 2024 CIDR 6.9368666e-05
4,245 Interactive Join Query Inference with JIM 2014 VLDB 6.8057947e-05
4,428 Arnold: Declarative Crowd-Machine Data Integration 2013 CIDR 6.7094035e-05
4,538 Cost-Effective Crowdsourced Entity Resolution: A Partial-Order Approach 2016 SIGMOD 6.6404776e-05
4,821 Crowdsourced Top-k Queries by Confidence-Aware Pairwise Judgments 2017 SIGMOD 6.4935866e-05
4,915 Crowdsourced Top-k Algorithms: An Experimental Evaluation 2016 VLDB 6.4477856e-05
5,201 Argonaut: Macrotask Crowdsourcing for Complex Data Processing 2015 VLDB 6.3201995e-05
5,813 Efficient Algorithms for Crowd-Aided Categorization 2020 VLDB 6.0780519e-05
6,548 Query Optimization over Crowdsourced Data 2013 VLDB 5.8443704e-05
7,021 Budget Constrained Interactive Search for Multiple Targets 2021 VLDB 5.7242766e-05
7,134 PromptEM: Prompt-tuning for Low-resource Generalized Entity Matching 2023 VLDB 5.6942765e-05
7,222 Hear the Whole Story: Towards the Diversity of Opinion in Crowdsourcing Markets 2015 VLDB 5.6679185e-05
7,493 Human-in-the-loop Data Integration 2017 VLDB 5.6046905e-05
8,039 Algorithmic Complexity Attacks on Dynamic Learned Indexes 2024 VLDB 5.5021992e-05
8,868 SemBench: A Benchmark for Semantic Query Processing Engines 2026 VLDB 5.3546848e-05
9,401 CDB: Optimizing Queries with Crowd-Based Selections and Joins 2017 SIGMOD 5.2755515e-05
10,030 Skyline Queries with Noisy Comparisons 2015 PODS 5.1741314e-05
10,031 tDP: An Optimal-Latency Budget Allocation Strategy for Crowdsourced MAXIMUM Operations 2015 SIGMOD 5.1741314e-05
11,912 A Rating-Ranking Method for Crowdsourced Top-k Computation 2018 SIGMOD 5.093636e-05
11,995 CrowdDQS: Dynamic Question Selection in Crowdsourcing Systems 2017 SIGMOD 5.093636e-05
12,013 A Confidence-Aware Top-k Query Processing Toolkit on Crowdsourcing 2017 VLDB 5.093636e-05
12,101 The Importance of Being Expert: Efficient Max-Finding in Crowdsourcing 2015 SIGMOD 5.093636e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
90 CrowdDB: Answering Queries with Crowdsourcing 2011 SIGMOD 0.00034951786
250 Answering Queries using Humans, Algorithms and Databases 2011 CIDR 0.00023261164
251 Crowdsourced Databases: Query Processing with People 2011 CIDR 0.00023261113
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 7,493 Human-in-the-loop Data Integration 2017 VLDB
2 9,401 CDB: Optimizing Queries with Crowd-Based Selections and Joins 2017 SIGMOD
3 997 CrowdScreen: Algorithms for Filtering Data with Humans 2012 SIGMOD
4 4,915 Crowdsourced Top-k Algorithms: An Experimental Evaluation 2016 VLDB
5 5,474 CrowdQ: Crowdsourced Query Understanding 2013 CIDR
6 1,910 Counting with the Crowd 2013 VLDB
7 852 Leveraging Transitive Relations for Crowdsourced Joins 2013 SIGMOD
8 90 CrowdDB: Answering Queries with Crowdsourcing 2011 SIGMOD
9 2,165 Demonstration of Qurk: A Query Processor for Human Operators 2011 SIGMOD
10 251 Crowdsourced Databases: Query Processing with People 2011 CIDR