Summary:
- Fix compiler warnings in all 20 FAISS C++ test files
- Fixes include `-Wshadow`, `-Wunused-parameter`, and signed/unsigned comparisons
All changes are mechanical. No functional changes.
Part 13/13 of the compiler warnings cleanup (split from https://github.com/facebookresearch/faiss/issues/4810 per maintainer request).
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4944
Test Plan:
- [x] No functional changes — all fixes are mechanical renames and annotations
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Reviewed By: mnorris11
Differential Revision: D96996609
Pulled By: alibeklfc
fbshipit-source-id: 72555246a4ccf540eb5422e64018c8d5e2660dce
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4760
This enables IndexIVFRaBitQFastScan to work with the standard IVF search interface by implementing the InvertedListScanner pattern. The scanner follows the same design as IVFPQFastScanScanner - processing one query at a time with SIMD-optimized distance computation via pq4_accumulate_loop, while applying RaBitQ-specific distance corrections through the IVFRaBitQHeapHandler.
This is needed for compatibility with low-level IVF APIs that use scanners directly, and for use cases requiring fine-grained control over inverted list scanning.
Reviewed By: junjieqi, mdouze
Differential Revision: D90634862
fbshipit-source-id: 88f0fae73f03663eaca821734d9aea317e8ee49b
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4537
Description
-
Adds InvertedListScanner for IVFPQFastScan. Right now IVFPQFastScan does not support a Scanner because the codes are PQFS packed. This is required from Unicorn to simplify the integration layer.
We can implement a scanner based on the search_implem_10. We can't do distance_to_code but we can do the whole scan. (that is what is done in this diff in the new `IVFPQFastScanScanner.scan_codes`).
Summary of changes (v10)
-
- High level: there's 2 heaps at play during a `scan_codes` call. One is the uint16_t heap inside the HeapHandler, which tracks the best results for a given iteration of nprobe. The other tracks the results across iterations and is in float: heap_distances. At the end of each call, we find the best results from the current iteration (`curr_dists`) then `heap_addn` those to the output distances `distances`).
- Moved `make_knn_handler` to the IndexIVFFastScan.h declaration so it can be used outside just IndexIVFFastScan.cpp. It is moved out of the anonymous namespace in the .cpp file too.
- Add the IVFPQFastScanScanner to IndexIVFPQFastScan. This impl for scan_codes is directly translated from search_implem_10 in in IndexIVFPQFastScan, because both search_implem_10 and scan_codes process 1 query and 1 invlist at a time (technically, search_implem_10 processes `n` queries at a time, just not batched. search_implem_10 has a simple loop over `n`, so search_implem_10 can be factored out to 1 query for this Scanner easily.). The only changes are removal of extraneous stuff from search_implem_10.
- Changes to test_lowlevel_ivf.cpp to test this. Refactored the test because IVFPQFastScanScanner does not behave like other scanners.
v15 vs v16
-
v15 matches search_implem_10 exactly. The unit tests pass with bit-exact match. They were not passing before due to the biases and LUT not matching exactly to search_implem_10.
Problems:
1. It requires quantizer search. Otherwise the biases and LUT will not match search_implem_10. The LUT is built from several inverted lists.
2. It requires mutable state across scan_codes calls. I called them lut_counter and bias_counter.
(I think even though this matches results, these problems make this option a no-go. It is mostly here to show the investigation.)
v16 does not use quantizer search and has no mutable state.
- cq.dis == distance from query (xi) to centroid represented by list_no.
- cq.ids == list_no
Both are of length 1 since we just have 1 list which is set explicitly.
Then we can call compute_LUT as normal.
The LUTs and biases are completely different from an existing approach, so results will not match an existing approach.
Match percentage between search_implem_10 and various by_residual approaches (I called it `recall` in the test):
by_residual_IP: 0.9745
by_residual_L2: 0.936
Thus I set a baseline of 0.9 in the test.
Reference impl:
https://github.com/facebookresearch/faiss/blob/514b44fca8542bafe8640adcbf1cccce1900f74c/faiss/IndexIVFFastScan.cpp#L918-L992
ntotal vs number of blocks?
-
- index.bbs is often 32 https://github.com/facebookresearch/faiss/wiki/Fast-accumulation-of-PQ-and-AQ-codes-(FastScan)#implementation. This is the number of codes per block.
~~Why do we need this io_simd_dis and io_simd_ids?~~
-
Old, but keeping it struck out. This describes the uint16_t heap kept over multiple scans.
~~Other scanners can scan over multiple lists with `set_query` then `set_list`, then calling `scan_codes`. Why? Because they take the `float* distances`. The heap can operate on these floats to re-use values from last iteration.~~
~~IVFPQFastScan is different. Codes are pq4 packed. The HeapHandler has to get initialized with values from scanning the prior list.~~
~~If the user is only setting 1 list and 1 query for the lifetime of this scanner (which I believe is Unicorn's use case), it does not matter because there's no need to use prior results and the user can follow the test named IVFPQFS_null_params. But if the user wants to use like the other Scanners, they can follow the other 2 tests in test_lowlevel_ivf.cpp that pass the heap size.~~
Reviewed By: ddrcoder, mdouze
Differential Revision: D80114737
fbshipit-source-id: a28392d444bc4dbe5d12e0fd84884e691cf00cf0
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4400
Unicorn now uses faiss opaquely, and we need support for decode vectors without decoding listnos, since we used encode_vectors without encoding listno in the codes.
Add decode_vectors, the inverse of encode_vectors, that for now doesn't decode listnos at all (can be changed)
Reviewed By: luciang, mnorris11
Differential Revision: D76870386
fbshipit-source-id: 5a4315b104253b10871e9d7095bb2fbcfa5ca453
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4029
Remove headers flagged by facebook-unused-include-check over fbcode.faiss.
+ format and autodeps
This is a codemod. It was automatically generated and will be landed once it is approved and tests are passing in sandcastle.
You have been added as a reviewer by Sentinel or Butterfly.
Autodiff project: uif
Autodiff partition: fbcode.faiss
Autodiff bookmark: ad.uif.fbcode.faiss
Reviewed By: dtolnay
Differential Revision: D65957849
fbshipit-source-id: f6199250db595defd56f5e7b2828f838702e9a16
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3455
Code quality control by reducing the number of prints
Reviewed By: junjieqi
Differential Revision: D57502194
fbshipit-source-id: a6cd65ed4cc49590ce73d2978d41b640b5259c17
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2582
A few more or less cosmetic improvements
* Index::idx_t was in the Index object, which does not make much sense, this diff moves it to faiss::idx_t
* replace multiprocessing.dummy with multiprocessing.pool
* add Alexandr as a core contributor of Faiss in the README ;-)
```
for i in $( find . -name \*.cu -o -name \*.cuh -o -name \*.h -o -name \*.cpp ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
For the fbcode deps:
```
for i in $( fbgs Index::idx_t --exclude fbcode/faiss -l ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
Reviewed By: algoriddle
Differential Revision: D41437507
fbshipit-source-id: 8300f2a3ae97cace6172f3f14a9be3a83999fb89
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2277
* extend a specialized AVX2 version for IVFPQScannerT::scan_list_with_table to cover IVFPQScannerT::scan_list_polysemous_hc as well
* lower the comparison precision in test_lowlevel_ivf tests from EXPECT_EQ to EXPECT-FLOAT_EQ because of the AVX2 change in IVFPQScannerT::scan_list_polysemous_hc, otherwise tests fail
Reviewed By: mdouze
Differential Revision: D34964138
fbshipit-source-id: 1d304a8f6eda040fa4c626676b4d492f2c12f04f
Changelog:
- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
