Summary:
Iterator-backed IVF scans ignored the configured `IDSelector`, and array-backed
scans applied it. An iterator-backed scan therefore returned IDs that the
selector excludes. `InvertedListScanner::iterate_codes` and
`InvertedListScanner::iterate_codes_range` now test `sel` for every entry.
`search_preassigned` also keeps the selector for an iterable list. The function
clears `sel` when the selector is an `IDSelectorRange` that sets
`assume_sorted`, because an array-backed list bounds a sorted section instead of
testing every entry. An iterable list holds no contiguous section to bound, so
it keeps the selector and takes the generic path. `range_search` holds no
sorted-range shortcut and needs no equivalent change.
The test loops over both values of `assume_sorted`, and it checks the KNN search
and the range search.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5592
Test Plan:
`buck2 test fbcode//faiss/tests:test_ivf_index fbcode//faiss/tests:test_custom_result_handler fbcode//faiss/tests:test_hnsw fbcode//faiss/tests:test_rabitq_simd_neon fbcode//faiss/tests:test_params_override fbcode//faiss/tests:test_hnsw_rabitq fbcode//faiss/tests:test_rabitq fbcode//faiss/tests:test_search_params fbcode//faiss/tests:test_io_corrupted fbcode//faiss/tests:test_rabitq_utils`, run at the top of the stack on aarch64: 173 pass, 0 fail, 3 skip.
`buck2 build fbcode//faiss:faiss` succeeds in the static aarch64 build and with `-c faiss.dynamic_dispatch=true`.
Reviewed By: alibeklfc
Differential Revision: D121846696
Pulled By: mnorris11
fbshipit-source-id: 09c8a3673fea6cc53f3370f05f3b6d399dfb4548
lyang24
76978eee4a