Not Small Enough? SegPQ: A Learned Approach to Compress Product Quantization Codebooks

Summary: SegPQ losslessly compresses PQ codebooks via learned ε-bounded piecewise-linear models plus low-bit residuals, with a provable near-optimal bit bound. SIMD-aware decompression cuts memory up to 4.7× for billion-scale ANN search at only 3.3% overhead. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID: 14184
Venue: VLDB
Year: 2025
Pagerank: 5.093636e-05
Overall Rank: 10,944 | 24.92%
DOI: 10.14778/3749646.3749650

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

1. Qiyu Liu (Southwest University)
2. Yanlin Qi (Harbin Engineering University)
3. Siyuan Han (Hong Kong University of Science and Technology)
4. Jingshu Peng (ByteDance)
5. Jin Li (Harvard University)
6. Lei Chen (Hong Kong University of Science and Technology)

BibTeX Citation

@article{liu_vldb25,
        title = {{Not Small Enough? SegPQ: A Learned Approach to Compress Product Quantization Codebooks}},
        author = {Liu, Qiyu and Qi, Yanlin and Han, Siyuan and Peng, Jingshu and Li, Jin and Chen, Lei},
        journal = {PVLDB},
        series = {{VLDB} '25},
        volume = {18},
        number = {11},
        pages = {3730--3743},
        doi = {10.14778/3749646.3749650},
        url = {https://doi.org/10.14778/3749646.3749650},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank

Outgoing Citations (Sorted by Pagerank)

Showing 11 of 11 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
43	The Case for Learned Index Structures	2018	SIGMOD	0.00046060254
447	ALEX: An Updatable Adaptive Learned Index	2020	SIGMOD	0.00018322593
477	The PGM-index: a fully-dynamic compressed learned index with provable worst-case bounds	2020	VLDB	0.00017851226
592	Distance-Based Indexing For High-Dimensional Metric Spaces	1997	SIGMOD	0.00016029789
805	Cache locality is not enough: High-Performance Nearest Neighbor Search with Product Quantization Fast Scan	2016	VLDB	0.00013891999
2,023	Decomposed Bounded Floats for Fast Compression and Queries	2021	VLDB	9.2950046e-05
2,572	DeltaPQ: Lossless Product Quantization Code Compression for High Dimensional Similarity Search	2020	VLDB	8.4027322e-05
4,073	Stable Learned Bloom Filters for Data Streams	2020	VLDB	6.9242783e-05
4,780	Can Learned Models Replace Hash Functions?	2023	VLDB	6.5118885e-05
8,709	Why Are Learned Indexes So Effective but Sometimes Ineffective?	2025	VLDB	5.3789739e-05
8,791	HAP: An Efficient Hamming Space Index Based on Augmented Pigeonhole Principle	2022	SIGMOD	5.3717005e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	10,548	JHQ: Johnson-Lindenstrauss Enhanced Hierarchical Quantization for High-Dimensional Approximate Nearest Neighbor Search	2026	VLDB
2	6,851	Maximum Inner Product is Query-Scaled Nearest Neighbor	2025	VLDB
3	8,622	Accelerating Product Quantization Query Execution Runtime	2021	SIGMOD
4	9,520	Experimental Analysis of Large-scale Learnable Vector Storage Compression	2024	VLDB
5	1,357	RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search	2024	SIGMOD
6	10,525	Quantization Meets Projection: A Happy Marriage for Approximate k-Nearest Neighbor Search	2026	VLDB
7	2,519	Similarity search in the blink of an eye with compressed indices	2023	VLDB
8	9,889	SAQ: Pushing the Limits of Vector Quantization through Code Adjustment and Dimension Segmentation	2026	SIGMOD
9	3,108	Practical and Asymptotically Optimal Quantization of High-Dimensional Vectors in Euclidean Space for Approximate Nearest Neighbor Search	2025	SIGMOD
10	2,572	DeltaPQ: Lossless Product Quantization Code Compression for High Dimensional Similarity Search	2020	VLDB