Divide and conquer the space
The inverted file index, or IVF, speeds search by first partitioning the vector space into clusters. A clustering step learns a set of centroids, and every vector is assigned to the nearest centroid, forming a bucket.
How a query is answered
At query time you compare the query to the centroids, pick the few closest buckets, and search only the vectors inside them. You skip the rest of the database entirely.
The key knob
The parameter often called nprobe sets how many buckets you search.
- A small nprobe is fast but may miss neighbors that fell into an unsearched bucket.
- A larger nprobe raises recall at the cost of more comparisons.
Where IVF fits
IVF is memory efficient and trains quickly, and it pairs naturally with quantization to shrink storage further. It works best when clusters are well balanced and the data has clear structure.
Key idea
IVF partitions vectors into centroid buckets and searches only the buckets nearest the query, with nprobe trading recall against speed.