ScaleDoc: Scaling LLM-based Predicates over Large Document Collections

Summary: ScaleDoc decouples LLM predicate execution: offline document representations enable query-time proxy models to filter most documents. Contrastive scoring and adaptive cascades preserve accuracy, yielding 2× speedups and up to 85% fewer LLM calls. (summarized by gpt-5.6-luna on Jul 26 2026)

Paper ID: hd71b6ff64a2f217b
Venue: SIGMOD
Year: 2026
Pagerank: 4.9793485e-05
Overall Rank: 10,498 | 29.42%
DOI: 10.1145/3802106

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Hengrui Zhang (Tsinghua University)
2. Yulong Hui (Tsinghua University)
3. Yihao Liu (Tsinghua University)
4. Huanchen Zhang (Shanghai Qi Zhi Institute; Tsinghua University)

BibTeX Citation

@inproceedings{zhang_sigmod26,
        title = {{ScaleDoc: Scaling LLM-based Predicates over Large Document Collections}},
        author = {Zhang, Hengrui and Hui, Yulong and Liu, Yihao and Zhang, Huanchen},
        series = {{SIGMOD} '26},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3802106},
        url = {https://dl.acm.org/doi/10.1145/3802106},
        year = {2026}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 13 of 13 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
271	NoScope: Optimizing Neural Network Queries over Video at Scale	2017	VLDB	0.00022560564
281	Accelerating Machine Learning Inference with Probabilistic Predicates	2018	SIGMOD	0.00022295232
501	Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes	2024	VLDB	0.00017267905
683	DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing	2025	VLDB	0.00014817539
1,932	CHORUS: Foundation Models for Unified Data Discovery and Exploration	2024	VLDB	9.34643e-05
2,776	Approximate Selection with Guarantees using Proxies	2020	VLDB	8.0309448e-05
2,982	Semantic Operators and Their Optimization: Enabling LLM-Based Data Processing with Accuracy Guarantees in LOTUS	2025	VLDB	7.7845174e-05
3,969	Optimizing Machine Learning Inference Queries with Correlative Proxy Models	2022	VLDB	6.8899861e-05
4,924	SNAILS: Schema Naming Assessments for Improved LLM-Based SQL Inference	2025	SIGMOD	6.3512934e-05
5,272	InferDB: In-Database Machine Learning Inference Using Indexes	2024	VLDB	6.2010954e-05
8,361	LEAP: LLM-powered End-to-end Automatic Library for Processing Social Science Queries on Unstructured Data	2025	VLDB	5.3474284e-05
8,997	Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guarantees	2026	SIGMOD	5.2410834e-05
9,645	Self-Enhancing Video Data Management System for Compositional Events with Large Language Models	2025	SIGMOD	5.1453267e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	10,527	In-context Clustering-based Entity Resolution with Large Language Models: A Design Space Exploration	2026	SIGMOD
2	683	DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing	2025	VLDB
3	10,766	A Comparative Evaluation of Schema Subsetting for LLM-based NL-to-SQL over Large-Schema Databases	2026	VLDB
4	12,259	Scalable Semantic Querying of Text	2018	VLDB
5	11,008	iPDB: SQL with ML and LLM Predicates (Towards a Database Engine for AI)	2026	VLDB
6	10,422	Bridging the Gap: Cardinality Estimation for Semantic Queries on Unstructured Data	2026	SIGMOD
7	4,736	Scalable Ad-hoc Entity Extraction from Text Collections	2008	VLDB
8	8,632	DocDB: A Database for Unstructured Document Analysis	2025	VLDB
9	10,690	Task Cascades for Efficient Unstructured Data Processing	2026	SIGMOD
10	11,161	ScaleLLM: A Technique for Scalable LLM-augmented Data Systems	2025	SIGMOD