Files
faiss/c_api/IndexIVF_c.cpp
T
Alexandr Guzhva 799bf3aaec Introduce early stop facilities for IVF, attempt 2 (#5160)
Summary:
# Introduce early-stop controls for IVF search

This PR adds optional early-stop controls to IVF search through `SearchParametersIVF`.

IVF search looks in a sequence of buckets. Normally it keeps looking until it has checked all requested buckets. This PR adds a few ways to stop earlier:

- stop after enough candidate vectors have been checked,
- stop FastScan after enough buckets have been checked,
- stop range search after several nearby buckets produce no results.

All new fields default to disabled, so existing searches keep their current behavior unless callers opt in.

## New SearchParametersIVF fields

Default values:

| Field | Default | Effect |
|---|---:|---|
| `ensure_topk_full` | `false` | disabled; keep the existing hard `max_codes` behavior |
| `max_lists_num` | `0` | disabled; FastScan is bounded only by `nprobe` |
| `max_empty_result_buckets` | `0` | disabled; range search scans all requested probes |

`ensure_topk_full`

For k-NN search, `max_codes` limits how many candidate vectors are scanned. With `ensure_topk_full=false`, the search stops when the budget is reached.

With `ensure_topk_full=true`, very small budgets become less aggressive: `max_codes` is treated as at least `k` post-filter scans. This helps when `max_codes < k`, or when an `IDSelector` filters out many candidates before a distance is computed.

In the generic IVF path this is supported for `parallel_mode` 0 and 3. In FastScan k-NN it is supported by the per-query implementations 10 and 11.

`max_lists_num`

FastScan k-NN can also stop after visiting a limited number of inverted lists. `0` means disabled. With `ensure_topk_full`, this list budget is treated as at least `k` lists. This is only used by the FastScan per-query path (`search_implem_10`, also used for implementation 11).

When FastScan is left on `implem = 0`, setting an early-stop k-NN option makes the dispatcher choose the per-query implementation automatically. If an incompatible FastScan implementation is selected explicitly, Faiss throws instead of ignoring the option.

`max_empty_result_buckets`

This is for range search. It stops probing a query after that many consecutive probed lists add no in-radius results. The counter resets as soon as a list adds at least one result.

`0` means disabled. Setting this trades recall for less work: once nearby buckets repeatedly produce no hits, farther buckets are less likely to help.

The generic IVF range path supports this in `parallel_mode == 0`. FastScan range search also supports it through implementation 10. With `implem = 0`, FastScan selects implementation 10 automatically when this option is set; explicit incompatible implementations throw.

## Selector-aware scan accounting

This PR adds `InvertedListScannerStats`, returned by inverted-list scanner methods. It currently reports:

- `scan_cnt`: number of distances actually computed,
- `nheap_updates`: number of result updates.

The important behavior change is that `IndexIVFStats::ndis` now uses `scan_cnt`. With an `IDSelector`, candidates rejected by the selector are not counted as computed distances. Without a selector, this matches the previous count because every scanned code reaches distance computation.

This same post-filter count is used by `ensure_topk_full`, so a restrictive selector no longer consumes the search budget with candidates that were never eligible for the result heap.

## FastScan behavior

FastScan k-NN now applies `max_codes`, `max_lists_num`, and `ensure_topk_full` in the per-query implementation. The code keeps separate counters per query, so multi-query batches have independent budgets.

For `max_codes`, FastScan uses the result handler's scanned-row count. This means filtered-out rows do not consume the code budget.

FastScan range search rejects k-NN-only options (`max_codes`, `max_lists_num`, and `ensure_topk_full`). It accepts `max_empty_result_buckets` when the selected implementation can preserve per-query probe order.

## API exposure

The new fields are exposed through:

- C++ `SearchParametersIVF`,
- Python bindings,
- the C API.

The statistics is tracked via `InvertedListScanner::scan_codes(..., ResultHandler& handler)` call. In order to make it work inside `IndexIVF::search()`, it has been decided to transition it into this particular override of `InvertedListScanner::scan_codes()`. As a result, non-`ResultHandler` version of `scan_codes()` has been marked as non-virtual, so the custom `InvertedListScanner` implementations should override `ResultHandler`-based versions only. Another consequence is that `IndexIVFPQFastScanScanner` and `IndexIVFRaBitQFastScanScanner` accept only `HeapResultHandler` types of `ResultHandler`. These are the compromises that had to be made.

## Tests

The tests cover:

- default parameters preserving baseline k-NN and range-search results,
- `ensure_topk_full` filling more result slots under tight budgets,
- selector-aware `IndexIVFStats::ndis`,
- range-search early exit and counter reset behavior,
- unsupported parallel modes and FastScan implementations,
- FastScan k-NN and range-search early-stop behavior,
- Python and C API field exposure.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5160

Reviewed By: limqiying

Differential Revision: D103044128

Pulled By: mnorris11

fbshipit-source-id: 80f1241c13ffc8173a3da0f2974fa9b6dd318890
2026-05-01 12:23:05 -07:00

194 lines
5.5 KiB
C++

/*
* Copyright (c) Meta Platforms, Inc. and affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
// -*- c++ -*-
#include "IndexIVF_c.h"
#include <faiss/IndexIVF.h>
#include "Clustering_c.h"
#include "Index_c.h"
#include "impl/AuxIndexStructures_c.h"
#include "macros_impl.h"
using faiss::IndexIVF;
using faiss::IndexIVFStats;
using faiss::SearchParametersIVF;
/// SearchParametersIVF definitions
DEFINE_DESTRUCTOR(SearchParametersIVF)
DEFINE_SEARCH_PARAMETERS_DOWNCAST(SearchParametersIVF)
int faiss_SearchParametersIVF_new(FaissSearchParametersIVF** p_sp) {
try {
SearchParametersIVF* sp = new SearchParametersIVF;
*p_sp = reinterpret_cast<FaissSearchParametersIVF*>(sp);
}
CATCH_AND_HANDLE
}
int faiss_SearchParametersIVF_new_with(
FaissSearchParametersIVF** p_sp,
FaissIDSelector* sel,
size_t nprobe,
size_t max_codes) {
try {
SearchParametersIVF* sp = new SearchParametersIVF;
sp->sel = reinterpret_cast<faiss::IDSelector*>(sel);
sp->nprobe = nprobe;
sp->max_codes = max_codes;
*p_sp = reinterpret_cast<FaissSearchParametersIVF*>(sp);
}
CATCH_AND_HANDLE
}
DEFINE_GETTER_PERMISSIVE(SearchParametersIVF, const FaissIDSelector*, sel)
DEFINE_GETTER(SearchParametersIVF, size_t, nprobe)
DEFINE_SETTER(SearchParametersIVF, size_t, nprobe)
DEFINE_GETTER(SearchParametersIVF, size_t, max_codes)
DEFINE_SETTER(SearchParametersIVF, size_t, max_codes)
DEFINE_GETTER(SearchParametersIVF, size_t, max_lists_num)
DEFINE_SETTER(SearchParametersIVF, size_t, max_lists_num)
DEFINE_GETTER(SearchParametersIVF, int, ensure_topk_full)
DEFINE_SETTER_STATIC(SearchParametersIVF, bool, int, ensure_topk_full)
DEFINE_GETTER(SearchParametersIVF, size_t, max_empty_result_buckets)
DEFINE_SETTER(SearchParametersIVF, size_t, max_empty_result_buckets)
/// IndexIVF definitions
DEFINE_DESTRUCTOR(IndexIVF)
DEFINE_INDEX_DOWNCAST(IndexIVF)
/// number of possible key values
DEFINE_GETTER(IndexIVF, size_t, nlist)
/// number of probes at query time
DEFINE_GETTER(IndexIVF, size_t, nprobe)
DEFINE_SETTER(IndexIVF, size_t, nprobe)
/// quantizer that maps vectors to inverted lists
DEFINE_GETTER_PERMISSIVE(IndexIVF, FaissIndex*, quantizer)
/**
* = 0: use the quantizer as index in a kmeans training
* = 1: just pass on the training set to the train() of the quantizer
* = 2: kmeans training on a flat index + add the centroids to the quantizer
*/
DEFINE_GETTER(IndexIVF, char, quantizer_trains_alone)
/// whether object owns the quantizer
DEFINE_GETTER(IndexIVF, int, own_fields)
DEFINE_SETTER(IndexIVF, int, own_fields)
using faiss::IndexIVF;
int faiss_IndexIVF_merge_from(
FaissIndexIVF* index,
FaissIndexIVF* other,
idx_t add_id) {
try {
reinterpret_cast<IndexIVF*>(index)->merge_from(
*reinterpret_cast<IndexIVF*>(other), add_id);
}
CATCH_AND_HANDLE
}
int faiss_IndexIVF_copy_subset_to(
const FaissIndexIVF* index,
FaissIndexIVF* other,
int subset_type,
idx_t a1,
idx_t a2) {
try {
reinterpret_cast<const IndexIVF*>(index)->copy_subset_to(
*reinterpret_cast<IndexIVF*>(other),
static_cast<faiss::InvertedLists::subset_type_t>(subset_type),
a1,
a2);
}
CATCH_AND_HANDLE
}
int faiss_IndexIVF_search_preassigned(
const FaissIndexIVF* index,
idx_t n,
const float* x,
idx_t k,
const idx_t* assign,
const float* centroid_dis,
float* distances,
idx_t* labels,
int store_pairs) {
try {
reinterpret_cast<const IndexIVF*>(index)->search_preassigned(
n, x, k, assign, centroid_dis, distances, labels, store_pairs);
}
CATCH_AND_HANDLE
}
size_t faiss_IndexIVF_get_list_size(
const FaissIndexIVF* index,
size_t list_no) {
return reinterpret_cast<const IndexIVF*>(index)->get_list_size(list_no);
}
int faiss_IndexIVF_make_direct_map(
FaissIndexIVF* index,
int new_maintain_direct_map) {
try {
reinterpret_cast<IndexIVF*>(index)->make_direct_map(
static_cast<bool>(new_maintain_direct_map));
}
CATCH_AND_HANDLE
}
double faiss_IndexIVF_imbalance_factor(const FaissIndexIVF* index) {
return reinterpret_cast<const IndexIVF*>(index)
->invlists->imbalance_factor();
}
/// display some stats about the inverted lists
void faiss_IndexIVF_print_stats(const FaissIndexIVF* index) {
reinterpret_cast<const IndexIVF*>(index)->invlists->print_stats();
}
/// get inverted lists ids
void faiss_IndexIVF_invlists_get_ids(
const FaissIndexIVF* index,
size_t list_no,
idx_t* invlist) {
const idx_t* list =
reinterpret_cast<const IndexIVF*>(index)->invlists->get_ids(
list_no);
size_t list_size =
reinterpret_cast<const IndexIVF*>(index)->get_list_size(list_no);
memcpy(invlist, list, list_size * sizeof(idx_t));
}
int faiss_IndexIVF_train_encoder(
FaissIndexIVF* index,
idx_t n,
const float* x,
const idx_t* assign) {
try {
reinterpret_cast<IndexIVF*>(index)->train_encoder(n, x, assign);
}
CATCH_AND_HANDLE
}
void faiss_IndexIVFStats_reset(FaissIndexIVFStats* stats) {
reinterpret_cast<IndexIVFStats*>(stats)->reset();
}
FaissIndexIVFStats* faiss_get_indexIVF_stats() {
return reinterpret_cast<FaissIndexIVFStats*>(&faiss::indexIVF_stats);
}