facebookresearch/faiss
1
0
Fork 0
mirror of https://github.com/facebookresearch/faiss.git synced 2026-10-12 02:13:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,335 Commits 268 Branches 33 Tags
ae4466d57e9a72977c58e79f6ed49d9a905f07ff
Commit Graph
2335 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jaeyoon Jeong ae4466d57e Avoid widening fp16 data for k-means++ initialization (#5693) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5693

Run `KMEANS_PLUS_PLUS` and `AFK_MC2` initialization directly from packed fp16 rows. Decode bounded per-thread row blocks, dispatch conversion once, and preserve the existing fp32 distance, RNG, and accumulation behavior without an `nx * d` fp32 copy.

Reviewed By: mnorris11

Differential Revision: D122781508

fbshipit-source-id: d3ed17ff9903fd08eb0d1221a6c562c46af82ff8
 2026-10-02 18:24:52 -07:00
Michael Norris ea9151d71b RaBitQ: broadcast one sign word per 32 dims in the AVX2 RBQ9 kernel (#5696) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5696

`compute_inner_product<SIMDLevel::AVX2>` scores 9-bit RaBitQ codes (`ex_bits == 8`) in groups of 8 dimensions. Before this change, each group loaded one sign byte and broadcast it. The broadcast uses the shuffle port, which limits the loop.

This change makes these changes to the loop:

- The loop loads one 32-bit sign word and broadcasts it once for each 32 dimensions.
- Each group of 8 dimensions ANDs the word with `positions << 8k` and compares the result with `cmpeq`. A `cmpgt` against zero is not correct here, because bit 31 of the shifted positions is a sign bit.
- The loop uses two accumulators.
- An 8-dimension loop handles the remaining full groups. `ip_scalar` handles the tail, as before.

The two accumulators change the float summation order. The benchmark checksums agree to 6 significant digits.

AI explanation of why:
```
  What limits a loop like this

  A loop runs no faster than its tightest limit. This loop has three possible limits:
  1. Micro-op count. The core issues about 4 micro-ops per cycle.
  2. Port 5, the shuffle port. Broadcasts and byte-to-int expansions can run only there.
  3. The FMA chain. With one accumulator, each FMA must wait for the previous one. Each FMA takes 4 cycles, so one group of 8 dims takes at least 4 cycles.

  Measured values, per group of 8 dims (d=768, codes in L2)

  ┌────────────────────────────────────────┬────────┬──────────────┬─────────────┬─────────────────┐
  │                  loop                  │ cycles │ instructions │ port 5 µops │ FMA chain floor │
  ├────────────────────────────────────────┼────────┼──────────────┼─────────────┼─────────────────┤
  │ V1 (contributor), 1 accumulator        │ 5.53   │ 18.4         │ 4.12        │ 4.0             │
  ├────────────────────────────────────────┼────────┼──────────────┼─────────────┼─────────────────┤
  │ V1, 2 accumulators                     │ 5.63   │ 15.4         │ 4.78        │ 2.0             │
  ├────────────────────────────────────────┼────────┼──────────────┼─────────────┼─────────────────┤
  │ sign word, 1 accumulator               │ 4.43   │ 10.4         │ 2.95        │ 4.0             │
  ├────────────────────────────────────────┼────────┼──────────────┼─────────────┼─────────────────┤
  │ sign word, 2 accumulators (D122893433) │ 3.77   │ 10.5         │ 2.99        │ 2.0             │
  └────────────────────────────────────────┴────────┴──────────────┴─────────────┴─────────────────┘

  With 4,096 codes, which do not fit in L2, the values are 6.20, 5.96, 5.15 and 4.66 cycles. The new loop is 1.33x faster than V1 there, which agrees with the
  faiss benchmark (1.33x–1.36x).

  Step 1: the sign-word broadcast removes micro-ops

  In V1, each group of 8 dims gets its sign byte in four instructions:
  movzx  eax, BYTE PTR [sign + i/8]   ; load one byte
  vmovd  xmm1, eax                    ; copy it into a vector register (port 5)
  vpbroadcastd ymm1, xmm1             ; copy it into all 8 lanes (port 5)
  The loop also does its index and loop-control work (shr, mov, add, cmp, jae) once per 8 dims. In total, each group needs 18 instructions.

  The new loop loads a 32-bit sign word once per 32 dims. The compiler turns the memcpy plus _mm256_set1_epi32 into one instruction:
  vpbroadcastd ymm0, DWORD PTR [sign + i/8]
  A broadcast that reads directly from memory runs on the load port only. It does not use port 5. Each of the four groups then needs only vpand and vpcmpeqd
  against its shifted position constant. The compiler loads these constants before the loop, so they cost nothing per pass. The loop-control work also runs once
  per 32 dims, not once per 8.

  Result: instructions per group go from 18.4 to 10.4, and port 5 micro-ops from 4.12 to 2.95. Cycles go from 5.53 to 4.43, which is 1.25x.

  Step 2: the second accumulator shortens the FMA chain

  After step 1, the micro-op limit is about 2.6 cycles and the port 5 limit is about 3.0 cycles. But one accumulator still forces 4 cycles per group. The measured
  4.43 cycles is close to that floor, so the FMA chain is now the limit.

  With two accumulators, two chains run at the same time, and the floor falls to 2.0 cycles. The next limit is port 5 at about 3 cycles. Cycles go from 4.43 to
  3.77, which is 1.18x.

  Why the second accumulator alone did not help V1

  V1 needs 5.53 cycles because of its micro-op count and port 5, not because of the FMA chain: 5.53 is well above the 4-cycle floor. A second accumulator removes a
  limit that V1 does not reach. My earlier standalone test found the same result. Thus each change helps only together with the other:
  - The broadcast change brings the loop down to the FMA floor.
  - The second accumulator then lowers the floor.

  The cmpeq change

  cmpeq is not faster than cmpgt; both cost the same. It is necessary for correctness: the group that tests bits 24–31 uses 128 << 24 = 0x80000000. As a signed
  integer this value is negative, so cmpgt(x, 0) would give the wrong result for that lane.
```

Reviewed By: junjieqi

Differential Revision: D122893433

fbshipit-source-id: e6c42d33e138614589e8f80c059df3652a7ddfcd
 2026-10-02 07:58:02 -07:00
ChenLiqing ea29bac75d RaBitQ: add batch-four estimates and 9-bit SIMD scoring (#5647) 
Summary:
Extend RaBitQ's distance-computation API with four independent one-bit estimates and add optimized kernels for standard RaBitQ codes.

- Add `RaBitQDistanceComputer::distance_to_code_1bit_batch_4`, with a default implementation that delegates to four single-code evaluations.
- For non-centered, four-bit query quantization, reuse query bit planes across four database codes. The optimized kernel uses the existing `AVX512_VPOPCNT` dispatch level; other SIMD levels and query modes retain fallbacks.
- Add AVX2 and AVX-512 full-distance kernels for 9-bit database codes, exploiting byte-aligned extra codes rather than scalar extraction. The scalar tail reads only the required extra bytes.
- Add `-mbmi2` to the x86 SIMD compile flags in `xplat.bzl` and `CMakeLists.txt`. The `AVX2` and higher SIMD levels now require CPUID BMI2. The RaBitQ kernels call `_pext_u64` without `__BMI2__` guards.
- Use local fixed-size `memcpy` for byte-aligned scalar bit-plane loads.

## API contract and scope

Call `set_query()` before evaluating a batch. The four row pointers may be non-contiguous or repeated. Results are returned in input order. The method computes estimates only: it does not refine candidates, update thresholds, or increment search counters.

Existing query modes and database precisions remain supported. No changes to HNSW traversal, filtering, graph construction, defaults, or serialized code layout are included. This is an opt-in batch API; existing search call sites are unchanged.

## Tests and benchmark

New standalone tests cover batch/single agreement for database bits 1–9, query bits 0–8, both centering modes, L2/IP, available SIMD overrides, repeated/non-contiguous row pointers, query reuse, and unchanged statistics. Independent references check packed multi-bit values and unaligned/tail buffers, including exactly sized RBQ9 extra codes. A round-trip test checks stored codes and distances.

`bench_rabitq_batch_estimates` compares four single-code estimates with the batch API at dimensions 128, 768, 960, and 1536. It prepares queries outside timing, shuffles code pointers, warms both paths, alternates measurement order, and prints per-code times and checksums. It is a scorer microbenchmark, not an end-to-end search benchmark.

Local validation (GCC 12, Release dynamic-dispatch build, Intel Xeon Platinum 8375C):

- Built `faiss_test` and `bench_rabitq_batch_estimates`.
- 61 tests passed with `--gtest_filter='TestLowLevelIVF.*:*RaBitQ*:*HNSW*'`, including all five new tests.
- 47 individual-process tests passed with `ctest -R 'RaBitQ|HNSW'`.
- All five new tests passed with the changed RaBitQ translation units and test file instrumented by ASan/UBSan, including leak detection. Unchanged dependencies were linked from the Release build; this was not a fully instrumented library build.
- Checked NONE, AVX2, AVX512 and AVX512_VPOPCNT on the local host. Clang 15 syntax checks passed for the three affected SIMD units, with baseline BMI2 disabled for the AVX2/AVX512 checks. ARM, non-BMI2 hardware, and native SPR execution were not performed locally. A CPU without BMI2 now uses the `NONE` level.
- No on-disk format changes; code and distance round-trip checks passed.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5647

Reviewed By: junjieqi

Differential Revision: D122390129

Pulled By: mnorris11

fbshipit-source-id: f3f1b1817e3fbcbf7896f3b4cf86eebcbba6f6b2
 2026-10-02 07:58:02 -07:00
Evan Haque 76c0e01e0f Add GPU-accelerated IVFPQ search to the Metal backend (#5449) 
Summary:
This is a follow-up to https://github.com/facebookresearch/faiss/issues/5288, which added `MetalIndexIVFPQ` with GPU-resident storage while delegating search to the CPU.

Adds GPU-accelerated IVFPQ search to the Metal backend, including optimized scanning through `M <= 64`.

- Add Metal kernels for 8-bit PQ distance computation, inverted-list scanning, and top-k merging
- Support optimized GPU scanning through `M <= 64`
- Compute distance contributions only for PQ codes encountered in each list when `d <= 256`
- Add a 32-thread cooperative kernel for lists containing at most 128 vectors
- Retain the cached precomputed-LUT kernel for suitable `M <= 16` workloads
- Stream precomputed terms for larger dimensions without allocating an `M * 256` threadgroup LUT
- Support both L2 and inner product metrics
- Add exact segmented top-k selection for inverted lists of any length
- Add grouped multi-round merging without a fixed `nprobe * k` candidate limit
- Compact invalid per-list padding before merging when lists contain fewer than `k` results
- Reuse dynamically sized Metal scratch buffers across searches
- Retain the legacy per-probe LUT path with optional FP16 lookup tables
- Fall back to CPU search for unsupported configurations, search parameters, or Metal failures

## Changes

- **Modified:** `MetalDistance.metal` - on demand PQ scans, large-M precomputed scan, short-list specialization, segmented top-k, and compacting grouped merge
- **Modified:** `MetalDistance.h/.mm` - IVFPQ search orchestration and support for `M <= 64`
- **Modified:** `MetalKernels.h/.mm` - dispatch for cached, on-demand, short-list, and large-M scan kernels
- **Modified:** `MetalIndexIVFPQ.h/.mm` - GPU scan selection, reusable buffers, centroid uploads, precomputed terms, and CPU fallback
- **Modified:** `MetalIndex.h` - add the `useFloat16` LUT configuration
- **Modified:** `TestMetalIndexIVFPQ.mm` - direct-GPU scan validation and search-parameter fallback coverage

## Differences from CUDA IVFPQ

**Training:** `MetalIndexIVFPQ::train` delegates training to its CPU index, as introduced in https://github.com/facebookresearch/faiss/issues/5288. CUDA can train the coarse and product quantizers on GPU. Training is a one-time cost.

**Add path:** Coarse assignment, residual computation, and PQ encoding remain on the CPU. The encoded PQ codes are stored in GPU-resident Metal buffers. CUDA performs these operations on GPU.

**Coarse quantization:** The coarse quantizer search runs on the CPU. The selected lists and coarse distances are passed to the Metal scan.

**Distance computation:** For `d <= 256`, the preferred Metal path evaluates only the PQ codes encountered while scanning a list. This avoids constructing or loading all 256 lookup entries for every subquantizer, which is especially beneficial for short IVF lists.

For other supported dimensions, Metal uses the CPU IVFPQ precomputed-table decomposition. The query-independent centroid/PQ term is computed once per trained index and the query term once per batch.

**Threadgroup memory:** The existing cached lookup-table kernel remains available for suitable `M <= 16` workloads. Larger `M` values do not expand its `M * 256` threadgroup LUT, which would exceed Metal threadgroup-memory limits. Instead, they use the on-demand or streamed-precomputed scan paths.

**List scanning:** Short lists use a 32-thread cooperative kernel with approximately 2 KiB of threadgroup scratch. Longer lists use segmented selection to maintain an exact running top-k.

**Top-k merge:** Per-list results are merged in groups over multiple rounds. Invalid padding is compacted when lists contain fewer than `k` results.

**Fallback:** The optimized scan supports `M <= 64`, `d / M <= 256`, and `k <= 512`. Unsupported configurations fall back to the legacy Metal path or CPU search. Search options that the GPU path cannot preserve, including selectors, scan budgets, and polysemous filtering, are forwarded to the CPU with their original parameters.

## Build and test

```bash
cmake -B build \
  -DFAISS_ENABLE_GPU=OFF \
  -DFAISS_ENABLE_METAL=ON \
  -DBUILD_TESTING=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_PREFIX_PATH="$(brew --prefix libomp)" \
  .

cmake --build build \
  --target faiss faiss_metal TestMetalIndexIVFPQ \
  -j$(sysctl -n hw.logicalcpu)

cd build && ctest -R TestMetalIndexIVFPQ --output-on-failure
```
All six IVFPQ tests pass. Direct-GPU regression coverage exercises 800 combinations across `M=8,16,17,32,48,64`, L2 and inner product, `k` through 512, short and long lists, both merge modes, and 64-bit IDs.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5449

Reviewed By: junjieqi

Differential Revision: D113864779

Pulled By: mnorris11

fbshipit-source-id: 741c12b2b014ef28a82569c6b0344352351a7325
 2026-10-01 21:07:27 -07:00
lyang24 0db2f30886 rabitq neon kernels (#5532) 
Summary:
This diff adds native ARM NEON kernels for the RaBitQ bitwise popcount
operations and for the multi-bit inner product. The kernels replace the scalar
fallbacks on ARM. They handle every supported bit width and every unaligned
dimension.

- `tests/test_rabitq_simd_neon.cpp` holds the correctness test, and
  `tests/BUCK` holds its `cpp_unittest` target. The target carries
  `target_compatible_with` on `ovr_config//cpu:arm64`, which mirrors the
  x86_64 gate on `test_rabitq_simd`.
- `tests/test_rabitq_simd_util.h` holds the shared `kDims` list and the shared
  `random_bytes` helper. `test_rabitq_simd.cpp` and `test_rabitq_simd_dd.cpp`
  now use them too.
- The three kernels that index a fixed array of eight accumulators assert
  `qb <= kMaxQueryBits`. `RaBitQuantizer::set_query` caps `qb` at 8, but the
  kernels are public templates and carry no other bound. The scalar kernel and
  the AVX-512 kernel need no such assertion, because they shift inside the loop
  and hold one accumulator.

The multi-bit test compares with a relative tolerance of `2e-5`. The NEON kernel
accumulates in eight lanes, so it does not reproduce the scalar sum bit for bit.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5532

Test Plan:
`buck2 test fbcode//faiss/tests:test_ivf_index fbcode//faiss/tests:test_custom_result_handler fbcode//faiss/tests:test_hnsw fbcode//faiss/tests:test_rabitq_simd_neon fbcode//faiss/tests:test_params_override fbcode//faiss/tests:test_hnsw_rabitq fbcode//faiss/tests:test_rabitq fbcode//faiss/tests:test_search_params fbcode//faiss/tests:test_io_corrupted fbcode//faiss/tests:test_rabitq_utils`, run at the top of the stack on aarch64: 173 pass, 0 fail, 3 skip.

`buck2 build fbcode//faiss:faiss` succeeds in the static aarch64 build and with `-c faiss.dynamic_dispatch=true`.

The host holds Neoverse V2 cores with ASIMDDP, so `test_rabitq_simd_neon` runs
its kernels natively rather than skipping.

Reviewed By: alibeklfc

Differential Revision: D121846717

Pulled By: mnorris11

fbshipit-source-id: de079613a52ced326045830fb799dfc9b169fe2e
 2026-10-01 10:36:44 -07:00
lyang24 6c51305e64 Avoid double-advancing HNSW visited tables (#5596) 
Summary:
The outer HNSW search loop called `vt->advance()` a second time. `HNSW::search`
already ends with `vt.advance()` at `impl/HNSW.cpp:1861`, so the caller in
`IndexHNSW.cpp` advanced the visited table once more per query. This diff
removes that call and adds a comment that names the owner of the advance.

The sibling call in `search_level_0` stays, because `search_level_0_impl` does
not advance.

**Microbenchmark.** The host holds 72 Cooper Lake cores. The build is
`mode/opt`. One core is pinned, and the two binaries interleave over 5 rounds.
The table reports the median of the medians. `IndexHNSWFlat`, `d=64`, `M=16`,
`use_visited_hashset=false` to select the versioned array, one query for each
iteration.

| ntotal | before | after | ratio |
|---|---|---|---|
| 100k | 44.2 us | 44.2 us | 1.00x |
| 400k | 63.8 us | 69.1 us | 0.92x |

**The change gives no measurable speed gain.** Both rows sit inside a 10% to 27%
spread. The arithmetic agrees: `visno` is a `uint8_t` that cycles from 1 to 254,
so the code clears the table once for every 250 advances. Two advances for each
query halve that interval, which costs about 800 extra bytes of memset for each
query against a search of 44 us to 69 us.

The value of the diff is the removal of a redundant call, not a speed gain.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5596

Test Plan:
`buck2 test fbcode//faiss/tests:test_ivf_index fbcode//faiss/tests:test_custom_result_handler fbcode//faiss/tests:test_hnsw fbcode//faiss/tests:test_rabitq_simd_neon fbcode//faiss/tests:test_params_override fbcode//faiss/tests:test_hnsw_rabitq fbcode//faiss/tests:test_rabitq fbcode//faiss/tests:test_search_params fbcode//faiss/tests:test_io_corrupted fbcode//faiss/tests:test_rabitq_utils`, run at the top of the stack on aarch64: 173 pass, 0 fail, 3 skip.

`buck2 build fbcode//faiss:faiss` succeeds in the static aarch64 build and with `-c faiss.dynamic_dispatch=true`.

Reviewed By: alibeklfc

Differential Revision: D121846632

Pulled By: mnorris11

fbshipit-source-id: 8928c4d4648ce061e63c084ded9c74fa17ef66bd
 2026-10-01 10:36:44 -07:00
lyang24 76978eee4a Fix selector handling for IVF iterator scans (#5592) 
Summary:
Iterator-backed IVF scans ignored the configured `IDSelector`, and array-backed
scans applied it. An iterator-backed scan therefore returned IDs that the
selector excludes. `InvertedListScanner::iterate_codes` and
`InvertedListScanner::iterate_codes_range` now test `sel` for every entry.

`search_preassigned` also keeps the selector for an iterable list. The function
clears `sel` when the selector is an `IDSelectorRange` that sets
`assume_sorted`, because an array-backed list bounds a sorted section instead of
testing every entry. An iterable list holds no contiguous section to bound, so
it keeps the selector and takes the generic path. `range_search` holds no
sorted-range shortcut and needs no equivalent change.

The test loops over both values of `assume_sorted`, and it checks the KNN search
and the range search.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5592

Test Plan:
`buck2 test fbcode//faiss/tests:test_ivf_index fbcode//faiss/tests:test_custom_result_handler fbcode//faiss/tests:test_hnsw fbcode//faiss/tests:test_rabitq_simd_neon fbcode//faiss/tests:test_params_override fbcode//faiss/tests:test_hnsw_rabitq fbcode//faiss/tests:test_rabitq fbcode//faiss/tests:test_search_params fbcode//faiss/tests:test_io_corrupted fbcode//faiss/tests:test_rabitq_utils`, run at the top of the stack on aarch64: 173 pass, 0 fail, 3 skip.

`buck2 build fbcode//faiss:faiss` succeeds in the static aarch64 build and with `-c faiss.dynamic_dispatch=true`.

Reviewed By: alibeklfc

Differential Revision: D121846696

Pulled By: mnorris11

fbshipit-source-id: 09c8a3673fea6cc53f3370f05f3b6d399dfb4548
 2026-10-01 10:36:44 -07:00
lyang24 41023120df Defer IVF list scan setup until needed (#5595) 
Summary:
The IVF list scan prepared state that an early return then discarded. This diff
moves `scanner->set_list`, the `ScopedCodes` fetch and the `ScopedIds`
allocation after every check that can return first. A list that an
`IDSelectorRange` excludes no longer builds a lookup table and no longer fetches
its codes. The diff also folds the `jmin` offset into the initial codes pointer,
and it replaces `std::unique_ptr<ScopedIds>` with `std::optional<ScopedIds>`.

The diff also fixes a null-pointer dereference. `search_preassigned` cleared
`sel` for a sorted `IDSelectorRange` before it tested
`!(sel && store_pairs)`. That combination then reached
`IDSelectorRange::find_sorted_ids_bounds`, which reads `ids[0]`. The `ids`
pointer is null when the caller sets `store_pairs`. The test now runs before the
code clears `sel`, so the combination throws.
`IVF.sorted_range_selector_rejects_store_pairs` covers it.

**Microbenchmark.** The host holds 72 Cooper Lake cores. The build is
`mode/opt`. One core is pinned, and the two binaries interleave over 5 rounds.
The table reports the median of the medians. `IndexIVFPQ`, `d=64`,
`nlist=1024`, 100k vectors, `nprobe=1024`, sorted `IDSelectorRange`.

| ids that pass the range | before | after | ratio |
|---|---|---|---|
| 1% | 836 us | 587 us | **1.43x** |
| 10% | 894 us | 926 us | 0.97x |

The gain grows with the number of lists that the range excludes. At 10% the
saving sits inside the noise.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5595

Test Plan:
`buck2 test fbcode//faiss/tests:test_ivf_index fbcode//faiss/tests:test_custom_result_handler fbcode//faiss/tests:test_hnsw fbcode//faiss/tests:test_rabitq_simd_neon fbcode//faiss/tests:test_params_override fbcode//faiss/tests:test_hnsw_rabitq fbcode//faiss/tests:test_rabitq fbcode//faiss/tests:test_search_params fbcode//faiss/tests:test_io_corrupted fbcode//faiss/tests:test_rabitq_utils`, run at the top of the stack on aarch64: 173 pass, 0 fail, 3 skip.

`buck2 build fbcode//faiss:faiss` succeeds in the static aarch64 build and with `-c faiss.dynamic_dispatch=true`.

Reviewed By: alibeklfc

Differential Revision: D121846650

Pulled By: mnorris11

fbshipit-source-id: aa3d1d3e6f903ef9e57be5c3dd824a5b1a090228
 2026-10-01 10:36:44 -07:00
Jaeyoon Jeong 38460ee969 Fix Faiss fp16 parity tests across architectures (#5695) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5695

Correct GPU tests to decode the exact packed fp16 values passed to `search_ex`; the generic software converter and legacy `roundToHalf` disagree on half-way ties. Keep labels exact while comparing CPU distances with the normal scaled `1e-5` BLAS/SIMD tolerance.

___

Differential Revision: D122785161

fbshipit-source-id: 057ded12bdc018add4bef39e514217b83b59c2cb
 2026-10-01 07:33:29 -07:00
Jaeyoon Jeong 430a41ee4c Native fp16 k-means training for Faiss IVF (#5671) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5671

Train IVF coarse quantizers from packed fp16 (IEEE binary16) data without an fp32 copy of the training set, and let the k-means assignment step consume fp16 natively.

**k-means: `Clustering::train_ex(n, x, NumericType, index, weights)`**
Float16 runs k-means directly on the fp16 rows, without a codec:
- NaN/Inf check on the fp16 bit patterns
- subsampling and random initialization copy fp16 rows
- the centroid update accumulates fp16 rows into the fp32 centroids with F16C/NEON (`compute_centroids_fp16`)
- the assignment step passes blocks of at most `decode_block_size` fp16 rows to `Index::search_ex(Float16)`
Only k-means++ / AFK-MC2 initialization still widens the (subsampled) training set.

**Assignment: `Index::search_ex(Float16)`**
- Default for every index: widen bounded blocks and call `search()`.
- `IndexFlat` (L2 / IP): the BLAS kernels take fp16 queries (`knn_L2sqr_fp16`, `knn_inner_product_fp16`). Queries are widened inside the kernel in chunks of at most 16 MiB, each reused for all database tiles; results are bit-identical to the fp32 path on the rounded queries. The AVX2 / AVX512 / SVE top-1 kernels share one templated body. Selectors, small inputs, d <= 32 and database-parallel searches widen chunks and use the fp32 path. `IndexFlat1D` and `IndexFlatPanorama` keep their own search.
- GPU: every `GpuIndex` accepts Float16 (queries are copied to the device as fp16 and widened there); `GpuIndexFlat` with `useFloat16` storage searches fp16 queries directly with the fp16 GEMM.

**IVF**
- `IndexIVF::train_ex(Float16)` and `Level1Quantizer::train_q1_ex` use the native path; only the `train_encoder_num_vectors()` rows sampled for the encoder are widened. `IndexIVFFlat` trains only the coarse quantizer; `IndexIVFFlatDedup` rejects Float16.
- `train_encoded` (IVF and Clustering) remains the generic codec path, e.g. for decode-time normalization.
- `faiss::fp16_to_fp32` (faiss/utils/utils.h) is the shared bulk widening helper.

**Performance** (opt build, AMD Genoa: F16C conversion only, no fp16 arithmetic)
- top-1 knn, d=256, 65536 queries x 1024 centroids: fp32 22.2 ms, fp16 25.3 ms
- k-means, 1024 centroids, 10 iterations: fp32 270 ms, fp16 319 ms, identical centroids
On CPUs without fp16 arithmetic the GEMM still runs in fp32 on widened tiles, so fp16 is not faster than fp32 there. The gains are memory (fp16 data plus a bounded fp32 tile instead of a full fp32 copy) and a single hook (`search_ex`) for fp16-capable backends; GPU `useFloat16` already runs the fp16 GEMM directly.

Reviewed By: mnorris11

Differential Revision: D121848262

fbshipit-source-id: 1400d3eeb7c713359895923dcb86c544570cba69
 2026-10-01 04:06:09 -07:00
Junjie Qi 88a28bcd80 Faiss OSS nightly autofix (#5686) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5686

Automated fix for the Faiss OSS nightly build.

___

Differential Revision: D121930308

fbshipit-source-id: 357958ce1d0b70e175ac3f7d381ec588575809da
 2026-09-29 13:39:58 -07:00
Zhang Jinhanihb2032lyd1992Yuansheng
 		6c8378e4f1 Add RVV implementation for decode_bf16_simd (#5641) 
Summary:
Add a RISC-V Vector (RVV) implementation to `decode_bf16_simd()`.

The RVV path preserves the existing bit-level conversion semantics:

1. load BF16 values as unsigned 16-bit integers;
2. zero-extend them to unsigned 32-bit integers;
3. shift left by 16 bits;
4. reinterpret the resulting bits as FP32.

Inputs shorter than 16 elements retain the scalar path to avoid short-vector overhead. Larger inputs use fixed-VL RVV chunks followed by a runtime-VL tail.

## Validation

Tested on native RISC-V hardware with GCC 15.3.0 and `-march=rv64gcv -mabi=lp64d`, based on Faiss commit `b5a14632a6f2bfdc824c86d683b9d6e9205c2604`.

Correctness:

- exhaustively tested all 65,536 BF16 bit patterns;
- tested 31 input sizes, including zero, short inputs, vector boundaries, tails, and large arrays;
- checked guard elements for output overwrites;
- zero correctness failures.

Representative microbenchmark speedups over the scalar implementation:

| Elements | Speedup |
|---:|---:|
| 7 | 1.01x |
| 16 | 1.58x |
| 31 | 2.43x |
| 64 | 3.46x |
| 128 | 3.88x |
| 1024 | 4.38x |
| 4096 | 4.37x |
| 65536 | 4.30x |

## Scope

This change only modifies `faiss/utils/bf16.h`. It is independent of the active ScalarQuantizer RVV changes in PRs https://github.com/facebookresearch/faiss/issues/5535 and https://github.com/facebookresearch/faiss/issues/5539.

## Notes

- The `n == 1` fast path and the `n < 16` scalar loop exist because at VLEN=128 an `e16m1` chunk covers only 8 elements, and for very short inputs the `vsetvl`/load/extend/store sequence costs more than the equivalent scalar work. The measured crossover is at 16 elements.
- The bulk loop uses a hoisted `vsetvlmax_e16m1()` so `vsetvl` is not re-executed per iteration; only the final partial chunk calls `vsetvl` again.
- `vuint16m1` → `vuint32m2` uses `vzext.vf2` (LMUL widening), so the 32-bit shift and the FP32 store run at `m2`. This keeps register pressure low while doubling the elements per chunk.
- No rounding or saturation is involved: BF16 → FP32 is an exact bit widening, so the vector path is bit-identical to the scalar path for every input, including NaN payloads and subnormals.
- The guard is `#elif defined(__riscv_vector)` inside the existing arch dispatch, so the AVX2/AVX512 paths are untouched and a build without RVV falls through to the scalar loop unchanged.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5641

Reviewed By: alibeklfc

Differential Revision: D121445956

Pulled By: mnorris11

fbshipit-source-id: 4b4ec0bad6473030849ae9275cbdad0bc3e087ee

Co-authored-by: ihb2032 <[email protected]>
Co-authored-by: lyd1992 <[email protected]>
Co-authored-by: Yuansheng <[email protected]>
 2026-09-28 17:52:28 -07:00
Michael Norris fdb9535c15 Make search stats collection configurable (#5670) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5670

Follow-up to D121368933 and public PR https://github.com/facebookresearch/faiss/pull/5657.

HNSW and RaBitQ search statistics are available to C++ and Python. Faiss collects them by default. Every search updates shared counters. Under high concurrency, these updates are expensive. In the 64-thread relaxed-atomic profile, `HNSWStats::combine_atomic` uses 74.14% of the sampled user cycles.

This diff adds one process-wide flag in `faiss/utils/utils.h`:

- `set_search_stats_enabled(bool)`
- `get_search_stats_enabled()`

The flag is a relaxed `std::atomic<bool>`. It is private to `utils.cpp`. The default is enabled, for backward compatibility. Callers that do not read the statistics can disable collection. Then the searches skip the global atomic updates.

At the moment, the flag controls `hnsw_stats` and `rabitq_stats`. The name is generic on purpose. The other global stats objects (for example `indexIVF_stats`, `indexPQ_stats` and `indexPanorama_stats`) use plain `+=` on shared counters, so they have the same scaling problem. Later diffs can put these objects under the same flag without a new API. The documentation says that the values of global stats objects are unspecified when the flag is disabled. Thus, more coverage does not change the contract.

API design:

- Faiss often exposes a plain global for a setting, for example `distance_compute_blas_threshold`. This diff does not do that. The flag is for concurrent searches, and a thread can change it while other threads search. A plain `bool` is then a data race, and TSAN reports it. This stack fixed TSAN races in D121097648.
- SWIG cannot wrap `std::atomic<bool>`. Thus, the setter and the getter are the only API. Python calls `faiss.set_search_stats_enabled(False)`.

Other changes:

- `bench_hnsw_stats` has a `stats` argument. `stats:0` disables collection, and `stats:1` enables it. One binary measures the two modes.
- The tests save the flag value and restore it after the test. The C++ tests use `SearchStatsEnabledGuard`. The Python tests use `addCleanup`. Thus, a failed assertion does not change the flag for later tests in the same process.

Mode/opt benchmark from V12, five sequential repetitions (mean real-time throughput):

| External pthreads | Relaxed atomic stats enabled | Stats disabled | Change |
| ---: | ---: | ---: | ---: |
| 1 | 2.516 M/s | 2.411 M/s | 0.96x |
| 2 | 3.919 M/s | 4.164 M/s | 1.06x |
| 4 | 7.494 M/s | 8.260 M/s | 1.10x |
| 8 | 12.717 M/s | 14.170 M/s | 1.11x |
| 16 | 16.074 M/s | 22.979 M/s | 1.43x |
| 32 | 19.929 M/s | 36.841 M/s | 1.85x |
| 64 | 22.101 M/s | 66.642 M/s | 3.02x |

The 0.96x value at 1 pthread is noise. A longer pinned measurement in the test plan shows no slowdown. At 1 thread, the atomic adds have no contention, so the expected gain is small.

The disabled path scales 27.64x from 1 to 64 pthreads. The enabled path scales 8.78x. The disabled path does not scale linearly, because this one-vector search is very small. At 64 threads, the OpenMP runtime uses about 28% of the sampled cycles. Allocator, RTTI and shared-cache costs also contribute.

Reviewed By: luciang

Differential Revision: D121374405

fbshipit-source-id: a54ee81a68a63df01c93bd5098171658883cd5ab
 2026-09-26 21:59:14 -07:00
Zhang Jinhan e24a371a70 feat(rvv): add vector arithmetic kernels (#5655) 
Summary:
On RISC-V, `fvec_add` and `fvec_sub` dispatch through `with_simd_level_256bit`. The level mask of that dispatcher (`AVAILABLE_SIMD_LEVELS_AVX2_NEON`) holds no `RISCV_RVV` bit. RVV hosts therefore use the scalar NONE implementation.

This PR adds native RVV kernels for the two functions and fixes the dispatch. The kernels follow the vector-length-agnostic approach of the RVV kernels in the tree. The kernels use `__riscv_vsetvl` (m8) and adapt to any VLEN at runtime. The kernels assume no fixed vector width.

## Changes

- Add RVV kernels in `faiss/utils/simd_impl/distances_rvv.cpp` for:
  - `fvec_add` (vector plus vector, and vector plus scalar)
  - `fvec_sub`
- Add the `AVAILABLE_SIMD_LEVELS_BASE_NO_AVX512` mask in `faiss/impl/simd_dispatch.h`.
  - The mask holds NONE, AVX2, ARM_NEON, and RISCV_RVV.
  - The mask holds no AVX512 bit, so AVX512 machines keep the AVX2 path.
- Route the three dispatchers (`fvec_sub`, `fvec_add`, scalar `fvec_add`) in `faiss/utils/distances_dispatch.h` through the new mask.
- Stable already provides the `compute_PQ_dis_tables_dsub2` RVV kernel and the `ProductQuantizer` RISC-V path, so this PR does not change them.
- This PR changes no behavior on other platforms.

## Performance

The benchmark ran on the original patch. The original patch also held the `compute_PQ_dis_tables_dsub2` kernel. Measured on SG2044 (64-core RISC-V, RVV 1.0), Faiss C++ benchmark suite, rcq-search case:

| Metric | Baseline | Patched | Change |
|---|---|---|---|
| instructions | 7.25 T | 6.87 T | -5.2% |
| cycles | 4.60 T | 4.30 T | -6.5% |
| wall time | 238.1 s | 223.6 s | -6.1% |
| branches | 334.6 G | 305.0 G | -8.8% |

The `fvec_add` hotspot in the perf record drops from 1.06% (scalar NONE) to 0.18% (RVV) of samples.

## Correctness

- `sq-accuracy` reconstruction errors match the baseline on all 9 quantizer types.
- `ndiff_for_idempotence` is 0.
- The correctness run used the original patch.

### Co-authors

- ihb2032 <[email protected]>
- lyd1992 <[email protected]>
- Yuansheng <[email protected]>

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5655

Test Plan:
`buck2 build fbcode//faiss:faiss` passes.

`buck2 test` passes on the distance tests `test_distances_dispatch` and `test_distances_simd`: 9 pass, 0 fail, 8 skip. The skips occur because the host provides one SIMD level.

`buck2 test` passes on the PQ tests `test_pq_encoding`, `test_pq_code_distance`, `test_ivfpq_indexing`, and `test_disable_pq_sdc_tables`: 17 pass, 0 fail.

Buck does not compile `faiss/utils/simd_impl/distances_rvv.cpp`. The host has no RISC-V toolchain. The new kernels still await review on RVV hardware.

Reviewed By: alibeklfc

Differential Revision: D121445525

Pulled By: mnorris11

fbshipit-source-id: ae3c1685e319bbb9523d87a3cbce8b1fd5310ce3
 2026-09-25 14:10:46 -07:00
Michael Norris 7f6bf10b1c Halve IndexSQFastScan rerank memory with a split-nibble layout (#5662) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5662

`IndexIVFSQFastScan` kept whole original codes in a parallel inverted list, so
an 8-bit quantizer cost 12 bits per dimension against 8 for plain SQ8. It now
keeps only the bits the scan does not read, which is the same split the flat
index uses in the diff below.

## The split is bit-width generic

An 8-bit code keeps 4 leftover bits. A 6-bit code keeps 2. `sq_fastscan_utils.h` packs the remainder with `BitstringWriter`, which lays bits out exactly as `Codec6bit` and `Codec8bit` do.

The LUT reconstruction collapses to one formula: `step = 2^(bits-4)`, `offset = step / 2`, `scale = 2^bits - 1`. At 4 bits this reduces to the native `(v + 0.5) / 15`, so the native and split paths no longer need separate constants. This also corrects a small bias, because the 8-bit case previously used an offset of 7.5 where the true band midpoint is 8.0.

## `IndexIVFSQFastScan` splits as well

The IVF form kept whole codes in a parallel inverted list. It now keeps only the leftover bits.

## Measured results, flat index, 200k vectors

| index | before | after |
| --- | --- | --- |
| `SQ8fs` | 12 bits/dim | **8.00 bits/dim** |
| `SQ6fs` | 10 bits/dim | **6.00 bits/dim** |

The IVF form reaches 1.37x `SQ8` at d=64 and 1.10x at d=512, against 1.79x before. The remainder is the ids duplicated into the parallel list and the block padding. Both are per-vector, so the ratio falls as `d` grows.

Recall does not drop. The split reaches 0.914 to 0.932 for `SQ6fs` and 0.969 to 0.980 for `SQ8fs` at `rerank_factor=4`, measured against exact neighbours restricted to the admitted set.

Also fixes a double-append in `add()`. `IndexFastScan::add` blocks adds over 65536 and calls `add()` virtually. The override therefore re-enters once per chunk, and every side-table entry was written twice for a large index.

## Serialization

The `ISfs` payload now carries the leftover bits instead of whole original
codes. The fourcc does not change, because `ISfs` has not been released with
rerank state and no stored index is affected.

Reviewed By: junjieqi

Differential Revision: D121092769

fbshipit-source-id: d6e8a4f541ce73057eba86215a83a0a80f7e106d
 2026-09-25 13:47:19 -07:00
Michael Norris 94a4331fee Add reranked types to IndexSQFastScan and fix flat RaBitQ filtering (#5661) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5661

Two gaps in the flat fast-scan indexes, both found while benchmarking
IDSelector filtering.

`IndexSQFastScan` accepted only the native 4-bit quantizers. It now accepts
the reranked types as well, the same set `IndexIVFSQFastScan` handles:
QT_6bit, QT_8bit, QT_8bit_uniform, QT_8bit_direct and QT_8bit_direct_signed.
Each code is SPLIT: the SIMD scan reads the top 4 bits, and `lo_codes` keeps
the rest for rerank. The total is the quantizer's own width, so `SQ8fs` holds
8 bits per dimension, the same as plain `SQ8`. A flat index needs no direct
map for this, unlike the IVF form, because the position is the id. `SQ8fs` and
`SQ6fs` now resolve in `index_factory`.

`IndexRaBitQFastScan` forwarded an `IDSelector` through every layer but its
kernel never applied it, so a filtered search returned ids outside the
admitted set. `RaBitQHeapHandler::handle` now tests the selector per code,
after the SIMD block, which is where the PQ and SQ handlers test theirs. The
entry point no longer refuses a parameter object.

The type predicates and the nibble packing move to `impl/sq_fastscan_utils.h`.
The flat and IVF classes then cannot disagree about which quantizers rerank,
or about how a nibble is laid out.

The `ISfs` payload gains the rerank factor and the original codes. Both are
written for every quantizer, and the codes are empty for a native 4-bit index.
`ISfs` has not been released, so no stored index is affected.

Reviewed By: junjieqi

Differential Revision: D121036639

fbshipit-source-id: 40e51fc348a483cf5da947fa7db56ff10f1af773
 2026-09-25 13:47:19 -07:00
Michael Norris dcf8ebb346 Support IDSelector in flat fast-scan search (#5660) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5660

`IndexFastScan::search` rejected every `SearchParameters` object, so a flat
fast-scan index could not take an `IDSelector` at all. `IndexSQFastScan`
avoided this by unpacking all `ntotal` codes into a fresh buffer on every
call, then running the generic scanner. That allocates and rewrites 12.8 MB
per query at 200k vectors and d=128.

The kernel already supports a selector. `make_fast_scan_knn_scanner` takes an
`IDSelector*`, and the result handlers test it after computing a 32-code SIMD
block. The flat path passed `nullptr` in that slot while the IVF path passed
the real selector. This forwards it instead, and deletes the unpack fallback.

`adjust_id` returns `j0 + 32 * b + j` and applies `id_map` when one is set, so
the id the selector tests is correct for the flat layout with no change.

Two paths still refuse a selector, both deliberately:

- `implem` 2, 3 and 4 have no scanner and cannot apply one. The default
`implem` of 0 never selects them.
- `IndexRaBitQFastScan` keeps its refusal here. The next diff adds the
  selector test to its kernel and removes the refusal.

Reviewed By: junjieqi

Differential Revision: D121036637

fbshipit-source-id: d199debb81244275b96a540bdfb8d8dd5fc31c18
 2026-09-25 13:47:19 -07:00
Michael Norris ec94295adc Bound the fast scan LUT scale by the accumulated sum (#5658) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5658

A fast scan kernel sums `M` uint8 table entries into a uint16 that can wrap around to 0. The scale that quantizes the table must therefore bound the whole sum that gets accumulated, not only one entry. Seven call sites each held their own copy of that rule. Only four applied it.

This adds `quantize_lut::fastscan_lut_scale` and routes all seven through it. The bound also reserves a margin for `round_tab`, which rounds half up, so `n` terms can gain `n/2` over the unrounded sum. `IndexIVFRaBitQFastScan` also folds its per-probe bias into the bound, because that bias is rounded into the same accumulator.

`IndexPQFastScan` only reaches the bound when a sub-vector holds one dimension. A nearest centroid keeps the selected entries near the bottom of each column. `IndexSQFastScan` sets `M = d`, so it fails above about `d = 512`. At `d = 1024` it goes from R@1 0.00 to 1.00.

Search speed does not change. The scale is computed once per query, and the inner loop is untouched.

An index at `M <= 256` quantizes to the same bytes as before, because the sum term does not bind there.

Reviewed By: junjieqi

Differential Revision: D120638521

fbshipit-source-id: 27d5dd15f5a60a9c80cec1f63f53ac9c6c51b8ff
 2026-09-25 13:47:19 -07:00
marcelo-cjl 3f08095cee fix: release SuperKMeans sample buffer after rotation (#5648) 
Summary:
- release the temporary subsampled training buffer as soon as rotation has populated `X_tilde`
- avoid retaining that buffer during Forgy initialization and the SuperKMeans iteration loop
- preserve the existing algorithm and rotation-time peak-memory behavior

## Memory result

One build run on Cohere 1M x 768 FP32 with SCANN (`nlist=1024`, `sub_dim=2`, `with_raw_data=false`, 16 threads):

- SuperKMeans before this change: 6749.1 MiB average RSS
- SuperKMeans with this change: 6425.4 MiB average RSS
- reduction: 323.7 MiB (4.8% of total average RSS; 8.5% of build RSS above baseline)

The released sample buffer is 768 MiB. The measured reduction is smaller because average RSS is time-weighted over the full build; the buffer is still required during sampling and rotation.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5648

Test Plan:
- built the AVX2 CPU `faiss_test` target in Release mode
- `faiss_test --gtest_filter='AdSampling.*:PdxLayout.*:BlockL2.*:SuperKMeansAssignIteration.*'` (13 tests passed)
- Knowhere vendored integration test for SuperKMeans (3 cases, 44 assertions passed)

Closes https://github.com/facebookresearch/faiss/issues/5650

Reviewed By: mnorris11

Differential Revision: D121294074

Pulled By: alibeklfc

fbshipit-source-id: c8e6f35f1421907887f9756661912df6ee4c2b87
 2026-09-25 12:56:06 -07:00
anthony-zy 1768a5a8e9 Add RVV SIMD specialization for product quantizer distance-table computation (#5543) 
Summary:
Add a RISC-V Vector Extension implementation for the product-quantizer
distance-table computation, and wire it into the SIMD dispatch path. The
kernels are runtime vector-length aware (vsetvl-driven, e32m1) rather than
assuming a fixed vector width.

Changes
- compute_PQ_dis_tables_dsub2<RISCV_RVV> (utils/distances_rvv.cpp): RVV
  kernel for both inner-product and L2 dsub=2 distance tables. Centroids are
  first split into even/odd dimension planes (c0_all / c1_all) so each
  subquantizer's ksub entries load as contiguous vectors; the hot loop
  processes two subquantizers per iteration:
      IP:  r = c0*x0 + c1*x1         (vfmul + vfmacc)
      L2:  r = (c0-x0)^2 + (c1-x1)^2 (vfsub + vfmul + vfadd)
- Add RVV specializations for the accompanying distance primitives:
  fvec_norm_L2sqr, fvec_L2sqr_ny, fvec_L2sqr_ny_transposed and
  fvec_inner_products_ny are vectorized (strided vlse32 over ny); the
  remaining fvec_* and VectorDistance specializations fall back to
  SIMDLevel::NONE.
- compute_distance_tables / compute_inner_prod_tables: route dsub == 2 &&
  nbits >= 3 && nbits < 8 through compute_PQ_dis_tables_dsub2 when
  COMPILE_SIMD_RISCV_RVV is defined, matching the AVX2 / ARM_NEON fast path.
- compute_PQ_dis_tables_dsub2_dispatch: extend the dispatch mask to include
  RISCV_RVV.

### Review fixes

The AI reviewers reported 6 valid defects. This version corrects all of them.

- The RVV `compute_PQ_dis_tables_dsub2` unrolled the subquantizer loop by 2
  and had no tail. An odd `M` left the last `ksub` table entries unwritten.
  The kernel now writes the last subquantizer in a tail loop.
- `fvec_inner_products_ny<RISCV_RVV>` loaded every chunk from `y + j`. The
  base pointer missed the `i * d` offset. Each chunk after the first repeated
  the first vectors. The kernel now loads from `y + i * d + j`.
- `fvec_norm_L2sqr<RISCV_RVV>` ran one `vfredusum` per vector chunk. A
  `d=768` input at VLEN=128 cost 24 serialized reductions. The function again
  keeps a tail-undisturbed accumulator, and it reduces once after the loop.
- The `dsub2` kernels require `ksub % 8 == 0`. The dispatch condition accepted
  every `nbits < 8`, so an `nbits` of 1 or 2 made `compute_distance_tables`
  throw. The condition now also requires `nbits >= 3`. This defect also
  affected the AVX2 and the ARM NEON paths.
- `search_sdc` dropped the per-thread `q_row` pointer precomputation. That
  change added `M` multiplications per database vector on every architecture.
  This version restores the precomputation.
- The `ny` kernels fixed the chunk width at the VLEN=128 value. They now read
  `VLMAX` with `vsetvlmax`. Hardware with a larger VLEN uses the whole vector
  register.

The diff also removes 2 unreadable comments from the RVV kernel.

Caveat: the benchmark below ran on the previous version. That version included
the `search_sdc` change that this version reverts.

### Performance (sg2044)
Cmd: 1M 768 <PQ> 32 100 16 1 1000 1 <metric>  (search ns/query, lower is better)

| #  | CONFIG   | M   | NPROBE | METRIC | F1(ns)  | PQ1(ns) | F2(ns)  | PQ2(ns) | Δ_PQ%   | Δ_F%   |
|----|----------|-----|--------|--------|---------|---------|---------|---------|---------|--------|
| 01 | PQ8x4    | 4   | 8      | L2     | 6656870 | 536988  | 6582134 | 495970  | -7.64%  | -1.12% |
| 02 | PQ8x4    | 4   | 8      | IP     | 6228497 | 475320  | 6164081 | 481118  | +1.22%  | -1.03% |
| 03 | PQ8x8    | 8   | 8      | L2     | 6626171 | 545230  | 6580586 | 500343  | -8.23%  | -0.69% |
| 04 | PQ8x8    | 8   | 8      | IP     | 6228452 | 519710  | 6165707 | 479864  | -7.67%  | -1.01% |
| 05 | PQ8x16   | 16  | 8      | L2     | 6615305 | 697070  | 6582239 | 624059  | -10.47% | -0.50% |
| 06 | PQ8x16   | 16  | 8      | IP     | 6227067 | 642949  | 6164572 | 587271  | -8.66%  | -1.00% |
| 07 | PQ8x32   | 32  | 8      | L2     | 6624323 | 1185655 | 6581329 | 902298  | -23.90% | -0.65% |
| 08 | PQ8x32   | 32  | 8      | IP     | 6239930 | 1155545 | 6164198 | 785403  | -32.03% | -1.21% |
| 09 | PQ8x64   | 64  | 8      | L2     | 6624196 | 2006275 | 6578815 | 1687191 | -15.90% | -0.69% |
| 10 | PQ8x64   | 64  | 8      | IP     | 6225719 | 1800678 | 6164767 | 1512760 | -15.99% | -0.98% |
| 11 | PQ8x96   | 96  | 8      | L2     | 6617525 | 3108377 | 6582946 | 2701758 | -13.08% | -0.52% |
| 12 | PQ8x96   | 96  | 8      | IP     | 6239711 | 2657958 | 6170006 | 2320803 | -12.68% | -1.12% |
| 13 | PQ8x384  | 384 | 8      | L2     | 6620602 | 14029275| 6583820 | 12318712| -12.19% | -0.56% |
| 14 | PQ8x384  | 384 | 8      | IP     | 6228776 | 12089134| 6166143 | 10509785| -13.06% | -1.01% |
| 15 | PQ8x32   | 32  | 4      | L2     | 3321625 | 773663  | 3298874 | 491225  | -36.51% | -0.68% |
| 16 | PQ8x32   | 32  | 16     | L2     |13223858 | 2184906 |13152640 | 1645444 | -24.69% | -0.54% |
| 17 | PQ8x32   | 32  | 32     | L2     |26430233 | 3405062 |26331426 | 3138563 | -7.83%  | -0.37% |
| 18 | PQ8x64   | 64  | 4      | L2     | 3320850 | 1191054 | 3298426 | 969309  | -18.62% | -0.68% |
| 19 | PQ8x64   | 64  | 16     | L2     |13207340 | 3944868 |13149902 | 3252654 | -17.55% | -0.43% |
| 20 | PQ8x64   | 64  | 32     | L2     |26467696 | 6725098 |26335327 | 6667493 | -0.86%  | -0.50% |
| 21 | PQ8x16   | 16  | 4      | L2     | 3316496 | 427653  | 3304051 | 367407  | -14.09% | -0.38% |
| 22 | PQ8x16   | 16  | 16     | L2     |13203491 | 1198002 |13150061 | 1121421 | -6.39%  | -0.40% |
| 23 | PQ8x16   | 16  | 32     | L2     |26438401 | 2199527 |26280149 | 2143973 | -2.53%  | -0.60% |

Δ_PQ%: PQ baseline -> RVV; Δ_F%: Flat baseline -> RVV.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5543

Test Plan:
`buck build fbcode//faiss:faiss` passes.

`buck2 test` passes on the PQ and the distance C++ tests:
`test_disable_pq_sdc_tables`, `test_distances_dispatch`, `test_distances_simd`,
`test_ivfpq_codec`, `test_ivfpq_indexing`, `test_pq_encoding` and
`test_pq_code_distance`. The result is 27 pass and 0 fail. The host is aarch64,
so 8 tests skip. Those tests need a second SIMD level to compare against.

This diff adds 2 tests to `faiss/tests/test_product_quantizer.py`:
- `test_dsub2_4bit` covers `dsub=2` with `nbits=4`. `ksub` is 16, so the SIMD
  `dsub2` kernel runs.
- `test_dsub2_2bit` covers `dsub=2` with `nbits=2`. `ksub` is 4, so the generic
  path must run instead.

I checked the `nbits >= 3` fix with a temporary C++ test on ARM NEON. The test
compares `compute_distance_tables` and `compute_inner_prod_tables` against
`compute_distance_table` and `compute_inner_prod_table`, for `nbits` of 1, 2, 3
and 4. The test fails with a `ksub % 8 == 0` assertion before the fix. The test
passes after the fix. I removed the temporary test before submit.

Limits of the local verification:
- I did not run `faiss/tests/test_product_quantizer.py` on this devserver. The
  `fbcode//faiss/python:pyfaiss-py-gen` SWIG step fails on aarch64 with
  `Exec format error`. The failure also reproduces without my changes. CI runs
  the Python tests on x86.
- I did not compile `utils/simd_impl/distances_rvv.cpp`. This devserver has no
  RISC-V toolchain. `xplat.bzl` registers no RVV sources, so Buck and internal
  CI do not build that file either. Only the OSS CMake build on rv64 compiles
  it. A reviewer with RVV hardware should confirm the 4 RVV kernel fixes.

Reviewed By: juancarpio27

Differential Revision: D117914440

Pulled By: mnorris11

fbshipit-source-id: 3539a116d47f4a96c1051bc10e6440deae8c5302
 2026-09-24 18:31:50 -07:00
Yedidya Feldblum b1c1496545 migrate from bit-manipulation builtins in fbcode/faiss (#5667) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5667

Replaces `__builtin_ctz` with `std::countr_zero` in `impl/fast_scan/simd_result_handlers.h`'s per-lane result-scanning loops. `utils/popcount.h` and `impl/platform_macros.h` are portability wrappers (their sole purpose is bridging MSVC/GCC-Clang builtins), so they're left out of scope here.

___

Differential Revision: D121529297

fbshipit-source-id: e42b3fb70eacc5fc4d2e2ffe7dfca64662ef2016
 2026-09-24 14:08:11 -07:00
Zhang Jinhanihb2032lyd1992Yuansheng
 		f323c18d5a Add RVV encoding for QT_8bit_direct (#5640) 
Summary:
RVV `QT_8bit_direct` currently inherits scalar encoding. This adds an RVV encoder using `e32m4`, explicit round-toward-zero float-to-unsigned conversion, and two narrowing operations before storing bytes.

The generic encoder becomes overridable so the RVV specialization can provide the implementation. The portable scalar conversion domain is finite inputs in `(-1, 256)`, whose truncated values are representable in `uint8_t`; negative fractions greater than -1 produce zero. The new regression test covers that domain, including signed zero, subnormals, values around integer boundaries, the largest float below 256, short tails, and output guards. This PR is independent of the FP16 and direct-signed-distance changes.

### Performance

Measured on a native SG2044 RISC-V host (VLEN=128), GCC 15.1, Release `-O3`, dynamic dispatch (`FAISS_OPT_LEVEL=dd`), `rv64gcv_zvfhmin/lp64d`, one thread pinned to CPU 2. The baseline is official commit `80a16564f86530dbf0bfaf96c2b71feffeb5093f`; the candidate is that same baseline plus only this kernel change. These measurements were not collected on the newer PR base `2ed4c106e9fb9686e7727e5daf8ad6ad1e164109`. The affected scalar-quantizer source files are unchanged between those bases, and the submitted kernel differs from the measured one only in comments/formatting.

| Public path | Dimension | Baseline ns/element | Candidate ns/element | Paired speedup | 95% interval |
|---|---:|---:|---:|---:|---:|
| Direct-u8 encode | 16 | 1.839 | 1.345 | 1.366x | [1.343, 1.379] |
| Direct-u8 encode | 32 | 1.839 | 1.061 | 1.750x | [1.690, 1.775] |
| Direct-u8 encode | 128 | 1.640 | 0.823 | 1.937x | [1.901, 1.992] |
| Direct-u8 encode | 768 | 1.530 | 0.740 | 2.059x | [1.883, 2.105] |

Geometric mean of the dimension-specific speedups at d=32/128/768: Direct-u8 encode **1.911x**.

There are three consecutive sessions, each with four alternating ABBA/BAAB blocks per dimension/path: **48 paired blocks** for this candidate. A is the baseline and B is the candidate. Each call is calibrated to at least 0.1 s (the shortest formal call in the full campaign was 0.192 s), with three warm-up batches and `n = max(32, floor(32768/d))`. Each block uses the ratio of the two-call geometric mean times. Reported speedup is the median of the three session medians; intervals use 5,000 hierarchical bootstrap resamples, resampling sessions and then blocks within each ABBA/BAAB order stratum. The timing columns are separate medians, so their quotient need not equal the paired speedup. All 48 paired blocks favored this candidate.

The timed operation is public `ScalarQuantizer::compute_codes`, using fixed-seed (718) finite input batches. Direct-u8 input values are `float(rng()%16777216)/65536.f`.

These results describe public encoding/distance throughput on one non-exclusive host, not end-to-end ANN search speedup. The intervals describe these sessions only, with no multiple-comparison correction; they do not establish portability across machines or vector lengths.

### Validation

- Native public encoding oracle: **7,724,670** legal-domain input elements across the five RISC-V rounding modes; zero differing output bytes or write-guard failures. Inputs include negative fractions, signed zeros, subnormals, all byte-value boundaries, and values just below 256.
- Original benchmark baseline full C++ suite: **273 passed, 7 skipped, 0 failed**.
- Submission version on `2ed4c106e9fb9686e7727e5daf8ad6ad1e164109`: all three independent candidate builds succeeded; this PR's focused C++ suite (`NONE`: 11 passed, 5 skipped, 0 failed; `RISCV_RVV`: 12 passed, 4 skipped, 0 failed) and the public-path oracle above pass on native RISC-V. The newly added RVV regression test is executed and passed in the RVV run, and skipped when RVV is disabled in the NONE run.
- All touched C++ files pass clang-format 21.1.8; `git diff --check` clean.

### Notes

- Touches three files: `faiss/impl/scalar_quantizer/quantizers.h` (one-line `final` → `override` so the generic encoder can be specialized), `faiss/impl/scalar_quantizer/sq-rvv.cpp` (the RVV encoder), and `tests/test_scalar_quantizer.cpp` (one regression test).
- The conversion uses `vfcvt.rtz.xu.f.v` (round toward zero, float to unsigned) rather than a signed conversion. The generic encoder is `code[i] = (uint8_t)x[i]`, which is a C++ float-to-unsigned-integral truncation, so round-toward-zero is the matching mode. Negative fractions greater than -1 truncate to 0 in both paths.
- Two narrowing shifts (`vnsrl` by 0) are used to go `u32m4 → u16m2 → u8m1` before the byte store, avoiding a separate saturating narrow instruction.
- The kernel is VLEN-agnostic: `e32m4` lanes with `vsetvl`-driven loop bounds. At VLEN=128 this processes 16 elements per iteration.
- Quantization is not clamped. Inputs outside `(-1, 256)` are outside the portable domain — the scalar path casts out-of-range floats to `uint8_t`, which is undefined behavior for values outside that range, so no portable contract exists to preserve there.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5640

Reviewed By: trang-nm-nguyen

Differential Revision: D121445607

Pulled By: mnorris11

fbshipit-source-id: 16f34cf77aa0cac2fd99ccb245f2ffbc2047d6c4

Co-authored-by: ihb2032 <[email protected]>
Co-authored-by: lyd1992 <[email protected]>
Co-authored-by: Yuansheng <[email protected]>
 2026-09-24 12:35:28 -07:00
Lucian Grijincu 2496f63f76 Avoid search stats contention during concurrent search (#5663) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5663

Follow-up to D121097648 (public PR: https://github.com/facebookresearch/faiss/pull/5657).

Related public PR: https://github.com/facebookresearch/faiss/pull/4910

Use relaxed atomic additions for HNSW, RaBitQ, and IVFPQ stats aggregation. Builds with `std::atomic_ref` support use `fetch_add(..., std::memory_order_relaxed)`; older standard libraries fall back to `__atomic_fetch_add(..., __ATOMIC_RELAXED)`. This removes the global OpenMP critical sections that serialize concurrent searches while preserving exact counter updates.

Benchmark design: mode/opt `bench_hnsw_stats` binary, one shared HNSW index, `N` external pthreads, one OpenMP thread per search (each worker calls `omp_set_num_threads(1)` once before its timed loop), five repetitions per point.

Performance:

| External pthreads | Critical M/s | Relaxed atomic M/s | Speedup |
|---:|---:|---:|---:|
| 1 | 2.228 | 2.503 | 1.12x |
| 2 | 3.716 | 3.853 | 1.04x |
| 4 | 5.662 | 7.357 | 1.30x |
| 8 | 6.384 | 12.843 | 2.01x |
| 16 | 2.697 | 15.804 | 5.86x |
| 32 | 0.951 | 18.910 | 19.88x |
| 64 | 0.117 | 21.866 | 186.89x |

At 64 pthreads, the feature-detected `atomic_ref` path measured 21.918 M/s mean over 10 repetitions (21.907 M/s median, 2.59% CV). The `__atomic_fetch_add` fallback measured 22.563 M/s mean (22.511 M/s median, 3.24% CV).

The 64-thread profile attributes 97.38% of cycles in the critical-section version to lock acquisition and waiting. With relaxed atomic aggregation, 74.14% is in the stats combine path instead. The remaining non-linear scaling, 21.866 M/s at 64 threads versus the 2.503 M/s single-thread baseline, comes from cache-line contention on the global counters and shared-index memory-bandwidth and cache pressure.

Reviewed By: mnorris11

Differential Revision: D121368933

fbshipit-source-id: 8dd26313e55aa73c0cdd379c7648efacd4f5f1d8
 2026-09-23 19:27:47 -07:00
ihb2032 5e74b1bdde feat: implement RVV optimized scalar quantizer codecs and distance computation (#5535) 
Summary:
This PR adds RISC-V Vector Extension SIMD support for Faiss scalar quantizer.

The implementation introduces runtime vector-length aware RVV kernels using the m8 vector configuration, instead of assuming a fixed vector width. This matches the variable-length nature of RVV hardware and allows the implementation to scale across different RISC-V vector implementations.

## Changes

- Add RVV implementations for scalar quantizer codecs:
  - 8-bit codec decoding
  - 4-bit codec decoding
  - (6-bit stays scalar: the RVV gather decoder measured 12-22% slower
    than the scalar path, so QT_6bit falls back until a faster decoder
    exists)

- Add RVV quantizer reconstruction support for:
  - Uniform quantizers
  - Non-uniform quantizers
  - FP16 (Zvfhmin is the enforced minimum ISA of the RISCV_RVV level; a
    build without it fails compilation rather than silently falling back)
  - BF16
  - Direct 8-bit quantization
  - Signed 8-bit quantization
  - LloydMax quantizers (1/2/3/4/8-bit)

- Add RVV optimized similarity implementations:
  - L2 distance
  - Inner product

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5535

Reviewed By: mnorris11

Differential Revision: D116996553

Pulled By: juancarpio27

fbshipit-source-id: dc726948fa602a04a757d8f9cab8eea41ac1979a
 2026-09-23 12:00:59 -07:00
Jure Bajic 02cacc5293 Fix TSAN races in side hsnw_stats and rabitq_stats (#5657) 
Summary:
These TSAN races can be detected when running a search in parallel over the IVF_HNSW index. This PR just wraps the updates of these inside `omp_critical` points.

```
WARNING: ThreadSanitizer: data race (pid=485)
  Write of size 8 at 0xaaaadb3db920 by thread T1443:
    #0 faiss::IndexHNSW::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/exception_ptr.h (arangod+0x7b95d44) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)

  Previous write of size 8 at 0xaaaadb3db920 by thread T59 (mutexes: write M0):
    #0 faiss::IndexHNSW::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/exception_ptr.h (arangod+0x7b95d44) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
    https://github.com/facebookresearch/faiss/issues/1 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const::$_0::operator()(long, float const*, float*, long*, faiss::IndexIVFStats*) const /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:351 (arangod+0x7ae5c04) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
    https://github.com/facebookresearch/faiss/issues/2 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const (.omp_outlined) /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:391 (arangod+0x7ae5ae8) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
    https://github.com/facebookresearch/faiss/issues/3 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:384 (arangod+0x7ade660) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
    https://github.com/facebookresearch/faiss/issues/4 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:384 (arangod+0x7ade624) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)

 Location is global 'faiss::hnsw_stats' of size 32 at 0xaaaadb3db920 (arangod+0xa5eb920)

  Mutex M0 (0xffff39ed9968) created at:
    #0 pthread_mutex_lock <null> (arangod+0x2414098) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
    https://github.com/facebookresearch/faiss/issues/1 __gthread_mutex_lock /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/aarch64-linux-gnu/c++/13/bits/gthr-default.h:749 (arangod+0x3820b80) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
    https://github.com/facebookresearch/faiss/issues/2 lock /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/std_mutex.h:113 (arangod+0x3820b80)
    https://github.com/facebookresearch/faiss/issues/3 lock_guard /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/std_mutex.h:249 (arangod+0x3820b80)

Thread T1443 (tid=6126, running) created by thread T138 at:
    #0 pthread_create <null> (arangod+0x2411d9c) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
    https://github.com/facebookresearch/faiss/issues/1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) <null> (libstdc++.so.6+0xe1c64) (BuildId: 7c239d1743727054caa3df1ab866bfa72cd4dd93)

 Thread T59 'SchedWorker' (tid=812, running) created by thread T56 at:
    #0 pthread_create <null> (arangod+0x2411d9c) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
```

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5657

Reviewed By: alibeklfc

Differential Revision: D121097648

Pulled By: mnorris11

fbshipit-source-id: 55455e7dab133360d4aa193f3853e3ae12f2f881
 2026-09-22 12:24:46 -07:00
anthony-zy bf8be4a79b fix(ci): use GCC 14 riscv64 cross-compiler for RVV tuple/zvfhmin intrinsics (#5653) 
Summary:
fix(ci): use GCC 14 riscv64 cross-compiler for RVV tuple/zvfhmin intrinsics

The linux-riscv64-DD CI job installs the default cross-compiler from
Ubuntu noble, which resolves to GCC 13.3. GCC 13's <riscv_vector.h>
does not export the RVV tuple types (vuint8m1x3_t) nor the zvfhmin
f16 intrinsics (vfloat16m1_t, __riscv_vle16_v_f16m1,
__riscv_vfwcvt_f_f_v_f32m2) used by the SQ RVV kernels, so building
sq-rvv.cpp failed with "'vuint8m1x3_t' was not declared in this
scope" (and similar) in the Codec6bit software-pipelined loops and
the QuantizerFP16 fast paths.

The intrinsics used are all legal under -march=rv64gcv_zvfhmin (only
vle16/vse16/vfwcvt/vfncvt on f16) and are fully supported starting
with GCC 14, so no source changes are needed.

Explicitly install gcc-14-riscv64-linux-gnu / g++-14-riscv64-linux-gnu
and point the toolchain file at the versioned binaries, avoiding any
reliance on update-alternatives defaults. A version self-check in the
workflow makes the resolved compiler visible in the CI log.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5653

Reviewed By: alibeklfc

Differential Revision: D120734761

Pulled By: mnorris11

fbshipit-source-id: d82cf2c99dfd6e8720f31bdadb718806f9e71044
 2026-09-21 16:23:17 -07:00
Alibek Zhakubayev 98af2cf627 Fix osx-arm64 conda build: drop redundant openblas pin, allow single-platform release dispatch (#5652) 
Summary:
The `OSX arm64 packages` job failed in the v1.15.1 release run ([35131196327](https://github.com/facebookresearch/faiss/actions/runs/35131196327)), so `faiss-cpu` 1.15.1 is on the pytorch channel for linux-64, linux-aarch64 and win-64 but **not** osx-arm64. PyPI is unaffected.

### Root cause

Upstream conda-forge inconsistency, not a faiss regression. On 2026-09-10 conda-forge published two new `libopenblas 0.3.34` builds for osx-arm64 without matching `openblas 0.3.34` metapackages:

| `openblas 0.3.34` accepts | `libopenblas 0.3.34` available |
|---|---|
| `openmp_he657e61_0` | `openmp_he657e61_0` (07-17) |
| `pthreads_hddb8425_0` | `pthreads_hddb8425_0` (07-17) |
| `pthreads_h60d1960_1` | `pthreads_h60d1960_1` (08-30) |
| `openmp_h4f80526_1` | `openmp_h4f80526_1` (08-30) |
| — | `pthreads_hb42d564_1` (09-10) ← no `openblas` |
| — | `openmp_h5e6e99c_1` (09-10) ← no `openblas` |

`libfaiss` built fine against `openmp_h4f80526_1`. The failure came when solving the `faiss-cpu` env: the unpinned `libopenblas =0.3.34` resolves to the newest build `openmp_h5e6e99c_1`, which no `openblas 0.3.34` accepts:

```
package libfaiss-1.15.1-py3.12_hcb8d3e5_0_cpu requires openblas 0.3.34.*,
but none of the providers can be installed
```

The recipe is vulnerable because the `libfaiss` output pins **both** `openblas` and `libopenblas` on osx-arm64 — `# [not x86_64]` was written for linux-aarch64 but also matches macOS ARM. The `faiss-cpu` output in the same file already pins only `libopenblas` on osx.

### Changes

1. **`conda/faiss/meta.yaml`** (lines 72, 86) — narrow the selector to `# [linux and not x86_64]`, so osx-arm64 constrains only `libopenblas` (what `libfaiss` actually links: `libopenblas.0.dylib`). linux-aarch64 and x86_64 behavior unchanged.

2. **`.github/workflows/build-release.yml`** — add `workflow_dispatch` with a `platforms` choice input. The workflow was `workflow_call`-only, so a single failed conda leg could not be rebuilt without pushing a tag. Each job gains an `if:` that is a no-op on tag pushes (`github.event_name` is `push`), preserving current release behavior.

### Recovery plan for 1.15.1

After this lands, dispatch `build-release.yml` from `main` with `platforms: osx-arm64`. The conda version comes from `git describe`, so this produces **1.15.1 build 1** — same version, next build number, no retagging and no PyPI involvement.

### Note

If conda-forge publishes the missing `openblas` builds, the old recipe would start working again on its own. This change removes the coupling so the build no longer depends on that.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5652

Test Plan:
- [ ] CI passes
- [ ] Dispatch `build-release.yml` with `platforms: osx-arm64` and confirm the conda solve succeeds
- [ ] Confirm `faiss-cpu` 1.15.1 appears for osx-arm64 on the pytorch channel

Reviewed By: mnorris11

Differential Revision: D120400090

Pulled By: alibeklfc

fbshipit-source-id: 0c27e8244bfe25bea4a45466e4e2376c04e0c1f8
 2026-09-21 13:25:58 -07:00
Mulugeta Mammo b0d9e80288 Add fast scan variants of SQ indexes - IndexSQFastScan and IndexIVFSQFastScan (#5341) 
Summary:
This PR introduces `IndexSQFastScan` and `IndexIVFSQFastScan`, fast scan variants of scalar quantized indexes.

**IndexSQFastScan** (33f2cb639bd7a888b2ab7578083b73753c2fd3d3)

Introduces `IndexSQFastScan`, a fast scan variant of `IndexScalarQuantizer` that leverages the PQ4 FastScan SIMD infrastructure (`vpshufb`) for accelerated search over scalar-quantized codes.

Three code paths depending on quantizer type:

- Native 4-bit (`QT_4bit`, `QT_4bit_uniform`): codes map directly onto the 16-level PQ4 fast scan. ~11x speedup, no precision loss.

- Rerank (`QT_6bit`, `QT_8bit`, `QT_8bit_uniform`, `QT_8bit_direct`, `QT_8bit_direct_signed`): re-quantize to 4-bit for a coarse fast scan first pass, then rerank top-k*rerank_factor candidates with exact original-precision distances. ~7-9x speedup, near-zero precision loss at rerank_factor=4.

- Fallback (`QT_fp16`, `QT_bf16`, `TurboQuant`, etc.): delegates to the ScalarQuantizer's own SIMD-optimized scanner. Same speed as IndexScalarQuantizer — no fast scan benefit, but provides a unified interface for all quantizer types.

**benchmark**: `benchs/bench_index_sq_fastscan.py`

```text
+---------------------+----------------------+---------+---------+--------+--------+
| Category            | QType                | Speedup | ms/q SQ | R@1    | R@10   |
+---------------------+----------------------+---------+---------+--------+--------+
| Native 4-bit (SIMD) | QT_4bit              | 11.9x   | 0.240   | 0.7714 | 0.9985 |
|                     | QT_4bit_uniform      | 11.7x   | 0.235   | 0.7180 | 0.9952 |
+---------------------+----------------------+----------+---------+--------+--------+
| Rerank (rf=4)       | QT_6bit              |  9.3x   | 0.202   | 0.9371 | 1.0000 |
|                     | QT_8bit              |  8.4x   | 0.173   | 0.9783 | 1.0000 |
|                     | QT_8bit_uniform      |  6.7x   | 0.138   | 0.9728 | 1.0000 |
|                     | QT_8bit_direct       |  4.7x   | 0.101   | 0.9914 | 1.0000 |
|                     | QT_8bit_direct_signed|  4.9x   | 0.101   | 0.5999 | 0.6591 |
+---------------------+----------------------+----------+---------+--------+--------+
| Fallback (~1.0x)    | QT_fp16              |  0.9x   | 0.373   | 0.9914 | 1.0000 |
|                     | QT_bf16              |  1.0x   | 0.365   | 0.9914 | 1.0000 |
|                     | QT_8bit_tqmse        |  1.0x   | 0.261   | 0.6033 | 0.9561 |
|                     | QT_5bit_tq           |  1.0x   | 3.013   | 0.2680 | 0.6248 |
|                     | QT_4bit_tq           |  1.0x   | 2.416   | 0.1511 | 0.3994 |
|                     | QT_3bit_tq           |  1.0x   | 0.411   | 0.0487 | 0.1691 |
|                     | QT_2bit_tq           |  1.1x   | 0.231   | 0.0376 | 0.1532 |
|                     | QT_4bit_tqmse        |  1.0x   | 0.290   | 0.0012 | 0.0064 |
|                     | QT_3bit_tqmse        |  1.0x   | 0.345   | 0.0000 | 0.0001 |
|                     | QT_2bit_tqmse        |  1.0x   | 0.266   | 0.0000 | 0.0001 |
|                     | QT_1bit_tqmse        |  1.0x   | 0.280   | 0.0000 | 0.0001 |
+---------------------+----------------------+----------+---------+--------+--------+

Key: Speedup = IndexScalarQuantizer ms / IndexSQFastScan ms
     R@1, R@10 = IndexSQFastScan recall (matches SQ for all fallback types)
     Rerank shows rf=4 (rf=2 and rf=8 yield same recall, ~0.1x speedup variation)
     Dataset: SIFT1M, k=32
```

**IndexIVFSQFastScan** (ebfa072816df2fbd80a6b463a913866f862587f6)

Similar to the above, this commit introduces `IndexIVFSQFastScan`, an IVF variant of scalar quantized fast scan that extends the PQ4 FastScan SIMD infrastructure (vpshufb) to inverted-list indexes with scalar quantization. Uses BlockInvertedLists for packed 4-bit codes scanned via SIMD. Will continue to update the PR until it's feature complete.

Three code paths depending on quantizer type:

- Native 4-bit (QT_4bit, QT_4bit_uniform): codes pack directly into the SIMD block layout with no precision loss. ~12x speedup.

- Rerank (QT_6bit, QT_8bit, QT_8bit_uniform, QT_8bit_direct, QT_8bit_direct_signed): re-quantized to 4-bit for the SIMD coarse pass, then reranked using exact original-precision codes stored in a parallel ArrayInvertedLists. ~8-10x speedup, near-zero precision loss at rerank_factor=2.

- Fallback (QT_fp16, QT_bf16, TurboQuant, etc.): delegates to IndexIVF's InvertedListScanner path. Same behavior as IndexIVFScalarQuantizer — no fast scan benefit, but provides a unified interface for all quantizer types.

**benchmark (SIFT1M, d=128, k=32, SapphireRapids)**: `benchs/bench_index_ivf_sq_fastscan.py`

```text
+---------------------+----------------------+--------+-------------------------------------------+
| Category            | QType                | nprobe | Speedup | ms/q SQ | R@1 FS | R@10 FS     |
+---------------------+----------------------+--------+---------+---------+--------+-------------+
| Native 4-bit (SIMD) | QT_4bit              |      1 |   7.1x  |  0.007  | 0.4582 | 0.5456      |
|                     |                      |      4 |  10.4x  |  0.007  | 0.6903 | 0.8667      |
|                     |                      |     16 |  12.9x  |  0.022  | 0.7668 | 0.9885      |
|                     |                      |     64 |  12.1x  |  0.070  | 0.7725 | 0.9983      |
|                     |                      |    256 |  11.3x  |  0.230  | 0.7725 | 0.9983      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_4bit_uniform      |      1 |  15.8x  |  0.006  | 0.4397 | 0.5449      |
|                     |                      |      4 |  10.5x  |  0.007  | 0.6419 | 0.8650      |
|                     |                      |     16 |  10.4x  |  0.018  | 0.7113 | 0.9855      |
|                     |                      |     64 |  10.9x  |  0.063  | 0.7162 | 0.9951      |
|                     |                      |    256 |  11.0x  |  0.222  | 0.7162 | 0.9951      |
+---------------------+----------------------+--------+---------+---------+--------+-------------+
| Rerank (rf=4)       | QT_6bit              |      1 |   4.9x  |  0.007  | 0.4582 | 0.5456      |
|                     |                      |      4 |   4.8x  |  0.007  | 0.6903 | 0.8667      |
|                     |                      |     16 |   7.9x  |  0.018  | 0.7668 | 0.9885      |
|                     |                      |     64 |  10.4x  |  0.067  | 0.7725 | 0.9983      |
|                     |                      |    256 |  10.2x  |  0.221  | 0.7725 | 0.9983      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_8bit              |      1 |  17.1x  |  0.007  | 0.4582 | 0.5456      |
|                     |                      |      4 |  12.5x  |  0.008  | 0.6903 | 0.8667      |
|                     |                      |     16 |   8.9x  |  0.015  | 0.7668 | 0.9885      |
|                     |                      |     64 |   8.4x  |  0.049  | 0.7725 | 0.9983      |
|                     |                      |    256 |   9.3x  |  0.188  | 0.7725 | 0.9983      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_8bit_uniform      |      1 |  18.2x  |  0.007  | 0.4397 | 0.5449      |
|                     |                      |      4 |  11.1x  |  0.008  | 0.6419 | 0.8650      |
|                     |                      |     16 |   8.3x  |  0.014  | 0.7113 | 0.9855      |
|                     |                      |     64 |   7.8x  |  0.045  | 0.7162 | 0.9951      |
|                     |                      |    256 |   7.8x  |  0.164  | 0.7162 | 0.9951      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_8bit_direct       |      1 |   8.6x  |  0.007  | 0.4083 | 0.5426      |
|                     |                      |      4 |   7.0x  |  0.008  | 0.5987 | 0.8587      |
|                     |                      |     16 |   6.8x  |  0.013  | 0.6550 | 0.9756      |
|                     |                      |     64 |   6.1x  |  0.035  | 0.6581 | 0.9850      |
|                     |                      |    256 |   7.4x  |  0.150  | 0.6581 | 0.9850      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_8bit_direct_signed|      1 |  16.5x  |  0.006  | 0.4257 | 0.5447      |
|                     |                      |      4 |  10.1x  |  0.007  | 0.6286 | 0.8633      |
|                     |                      |     16 |   7.1x  |  0.012  | 0.6919 | 0.9831      |
|                     |                      |     64 |   5.7x  |  0.033  | 0.6963 | 0.9923      |
|                     |                      |    256 |   7.1x  |  0.145  | 0.6963 | 0.9923      |
+---------------------+----------------------+--------+---------+---------+--------+-------------+
| Fallback (~1.0x)    | QT_fp16              |      1 |   1.0x  |  0.007  | 0.5434 | 0.5462      |
|                     |                      |      4 |   1.1x  |  0.013  | 0.8607 | 0.8678      |
|                     |                      |     16 |   0.9x  |  0.035  | 0.9814 | 0.9902      |
|                     |                      |     64 |   0.9x  |  0.115  | 0.9912 | 1.0000      |
|                     |                      |    256 |   0.9x  |  0.481  | 0.9912 | 1.0000      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_bf16              |      1 |   1.0x  |  0.008  | 0.5427 | 0.5462      |
|                     |                      |      4 |   1.1x  |  0.013  | 0.8581 | 0.8678      |
|                     |                      |     16 |   1.0x  |  0.035  | 0.9778 | 0.9902      |
|                     |                      |     64 |   1.0x  |  0.115  | 0.9876 | 1.0000      |
|                     |                      |    256 |   1.0x  |  0.487  | 0.9876 | 1.0000      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_5bit_tq           |      1 |   1.0x  |  0.016  | 0.1866 | 0.3827      |
|                     |                      |     64 |   1.0x  |  0.794  | 0.2680 | 0.6248      |
|                     |                      |    256 |   1.0x  |  2.885  | 0.2680 | 0.6248      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_4bit_tq           |      1 |   1.0x  |  0.013  | 0.1182 | 0.2841      |
|                     |                      |     64 |   0.9x  |  0.618  | 0.1511 | 0.3994      |
|                     |                      |    256 |   0.9x  |  2.298  | 0.1511 | 0.3994      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_3bit_tq           |      1 |   1.1x  |  0.008  | 0.0526 | 0.1599      |
|                     |                      |     64 |   0.8x  |  0.092  | 0.0488 | 0.1696      |
|                     |                      |    256 |   0.8x  |  0.357  | 0.0487 | 0.1691      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_2bit_tq           |      1 |   1.2x  |  0.008  | 0.0450 | 0.1526      |
|                     |                      |     64 |   1.1x  |  0.072  | 0.0377 | 0.1534      |
|                     |                      |    256 |   1.1x  |  0.224  | 0.0376 | 0.1532      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_4bit_tqmse        |      1 |   1.0x  |  0.007  | 0.0619 | 0.1851      |
|                     |                      |     64 |   1.0x  |  0.072  | 0.0002 | 0.0010      |
|                     |                      |    256 |   0.9x  |  0.275  | 0.0000 | 0.0000      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_3bit_tqmse        |      1 |   1.0x  |  0.007  | 0.0621 | 0.1852      |
|                     |                      |     64 |   1.0x  |  0.090  | 0.0006 | 0.0032      |
|                     |                      |    256 |   1.0x  |  0.350  | 0.0000 | 0.0000      |
|                     +----------------------+--------+---------+---------+--------+-------------+
|                     | QT_2bit_tqmse        |      1 |   1.1x  |  0.007  | 0.0621 | 0.1852      |
|                     |                      |     64 |   1.0x  |  0.068  | 0.0015 | 0.0080      |
|                     |                      |    256 |   1.0x  |  0.244  | 0.0000 | 0.0000      |
+---------------------+----------------------+--------+---------+---------+--------+-------------+

Key: Speedup = IndexIVFScalarQuantizer ms / IndexIVFSQFastScan ms
     R@1, R@10 = IVFSQFastScan recall (matches IVF-SQ for all fallback types)
     Rerank shows rf=4 only (rf=2,8 yield similar results)
     Fallback shows nprobe=1,64,256 only (abbreviated for low-recall types)
     Dataset: SIFT1M, nlist=256, k=32
```

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5341

Reviewed By: junjieqi

Differential Revision: D117259284

Pulled By: mnorris11

fbshipit-source-id: 41455097ff3ae1285aeb4cf88e07abc5b5b13171
 2026-09-18 20:24:54 -07:00
Mihai Capotă 4864f4d420 SVS: Add ParameterSpace.set_index_parameter support for Vamana (#5644) 
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5644

Reviewed By: alibeklfc

Differential Revision: D120290334

Pulled By: mnorris11

fbshipit-source-id: c947484d0704789553a7d4d80a0ef7185e59962a
 2026-09-18 10:15:10 -07:00
Aaryan Mehta 3f4838ead3 Batch the IVF ScalarQuantizer scan four codes at a time (#5456) 
Summary:
The IVF ScalarQuantizer inverted-list scan distances one code at a time through `run_scan_codes1`, even though the SQ distance computers already expose a four-at-a-time kernel, `query_to_codes_batch_4` (the one HNSW uses through `distances_batch_4`). This adds `run_scan_codes4`, a batched inner loop that distances four consecutive codes per step and then applies the same threshold compare, `add_result`, and threshold refresh as `run_scan_codes1`, in id order, so the sequence of heap updates is identical. The two SQ scanners route their no-selector path through it; the selector path stays on `run_scan_codes1`.

Four independent accumulator chains let the out-of-order core overlap the latency-bound reduction that a single code cannot hide. Results are unchanged: the batched kernel reconstructs each code, which is bit-identical to the single-code path for every quantizer except the uniform ones, whose single-code path predecodes the query, so a compile-time check keeps those on the scalar scan. Verified bit-identical distances and ids against the scalar scan for QT_8bit, QT_8bit_uniform, QT_6bit, QT_4bit, QT_fp16, and QT_8bit_direct, in both metrics, at d in {64, 128, 256}.

Measured on AVX2, end-to-end `IndexIVFScalarQuantizer` search of QT_8bit, single thread, batched against the scalar scan:

| metric | d | scalar | batched | speedup |
| --- | --- | --- | --- | --- |
| L2 | 128 | 60.4 ms | 40.3 ms | 1.50x |
| L2 | 256 | 170.9 ms | 105.7 ms | 1.62x |
| L2 | 256, out of cache | 403.5 ms | 262.3 ms | 1.54x |
| IP | 128 | 46.5 ms | 38.4 ms | 1.21x |
| IP | 256 | 108.7 ms | 83.8 ms | 1.30x |

The L2 gain is the larger one, around 1.5-1.6x, and it holds when the index spills out of cache. Inner product gains less since its shorter chain is already partly overlapped by the core. When batching does not help it matches the scalar scan, so there is no regression.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5456

Reviewed By: junjieqi

Differential Revision: D118857126

Pulled By: mnorris11

fbshipit-source-id: df7b02c2ede8699c599956370b87a949f1b7db9d
 2026-09-17 23:20:26 -07:00
郑紫阳10295048 0f23e6a2ef Add RVV SIMD specialization for scalar quantizer distance compute (#5539) 
Summary:
This PR adds RISC-V Vector Extension SIMD specialization for the scalar
quantizer distance computation, following the codec/quantizer/similarity
specialization already present for other SIMD levels. The kernels use the
runtime vector-length aware m8 vector configuration and are shared across
all RVV hardware widths, instead of assuming a fixed vector length.

Changes
Add RVV specialized L2 / inner-product distance kernels for the scalar
quantizer codecs:

- QT_4bit / QT_4bit_uniform
- QT_6bit
- QT_8bit / QT_8bit_uniform
- QT_8bit_direct / QT_8bit_direct_signed
- QT_bf16
- QT_fp16

Each kernel reuses the reconstructed (quantizer) L2 / IP formulation that
avoids per-dimension reconstruction, keeping the hot loop in the integer
domain and applying the final scale at the tail.

Performance
Cmd: 1M 768 <SQ> 32 100 16 1 1000 1 <metric>  (search ms/query, lower is better)
Δ_SQ%: SQ scalar baseline -> RVV;
Δ_F%: Faiss flat baseline -> RVV.

| 序号 | QT_TYPE               | Metric | F1(ms) | SQ1(ms) | F2(ms) | SQ2(ms) | Δ_SQ%   | Δ_F%   |
|------|-----------------------|--------|--------|---------|--------|---------|---------|--------|
| 01   | QT_4bit               | L2     | 1.841  | 6.231   | 1.824  | 1.450   | -76.73% | -0.92% |
| 02   | QT_4bit               | IP     | 1.652  | 6.086   | 1.674  | 1.273   | -79.08% | +1.33% |
| 03   | QT_4bit_uniform       | L2     | 1.806  | 2.560   | 1.806  | 1.135   | -55.66% | +0.00% |
| 04   | QT_4bit_uniform       | IP     | 1.612  | 6.007   | 1.597  | 1.244   | -79.29% | -0.93% |
| 05   | QT_6bit               | L2     | 1.801  | 5.701   | 1.805  | 1.500   | -73.69% | +0.22% |
| 06   | QT_6bit               | IP     | 1.608  | 5.706   | 1.619  | 1.249   | -78.11% | +0.68% |
| 07   | QT_8bit               | L2     | 1.821  | 3.736   | 1.907  | 1.394   | -62.69% | +4.72% |
| 08   | QT_8bit               | IP     | 1.704  | 3.378   | 1.730  | 1.277   | -62.20% | +1.53% |
| 09   | QT_8bit_direct        | L2     | 1.977  | 0.280   | 1.947  | 0.054   | -80.71% | -1.52% |
| 10   | QT_8bit_direct        | IP     | 1.709  | 0.458   | 1.740  | 0.099   | -78.38% | +1.81% |
| 11   | QT_8bit_direct_signed | L2     | 1.581  | 3.234   | 1.572  | 0.761   | -76.47% | -0.57% |
| 12   | QT_8bit_direct_signed | IP     | 1.740  | 0.414   | 1.748  | 0.148   | -64.25% | +0.46% |
| 13   | QT_8bit_uniform       | L2     | 1.943  | 3.204   | 1.928  | 0.927   | -71.07% | -0.77% |
| 14   | QT_8bit_uniform       | IP     | 1.727  | 3.121   | 1.744  | 0.947   | -69.66% | +0.98% |
| 15   | QT_bf16               | L2     | 1.862  | 2.889   | 1.894  | 1.466   | -49.26% | +1.72% |
| 16   | QT_bf16               | IP     | 1.619  | 2.846   | 1.793  | 1.204   | -57.70% | +10.75% |
| 17   | QT_fp16               | L2     | 2.011  | 5.691   | 1.950  | 1.570   | -72.41% | -3.03% |
| 18   | QT_fp16               | IP     | 1.797  | 5.584   | 1.790  | 1.189   | -78.71% | -0.39% |

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5539

Reviewed By: mnorris11

Differential Revision: D117914328

Pulled By: juancarpio27

fbshipit-source-id: 879863b9ab4da6d1c8685892ccc3d64190975181
 2026-09-17 13:34:43 -07:00
YingzuoLiu 3778f5892d Fix IDSelector handling in unbounded HNSW search (#5508) 
Summary:
- apply `SearchParametersHNSW.sel` to the unbounded HNSW search path
- keep filtered nodes available for graph traversal while excluding them from the result set
- add regression coverage for L2 and inner-product search, restrictive `efSearch`, and an empty selector

## Root cause

The bounded HNSW path applies the selector through its result handler. The unbounded path did not receive the selector and used the same heap for traversal pruning and final results, so IDs rejected by `SearchParameters.sel` could be returned.

## Implementation

When a selector is present, the unbounded path maintains a separate result heap containing only accepted IDs. The existing candidate heap remains unchanged and can still contain rejected nodes, preserving traversal through them. When no selector is present, the existing unfiltered result behavior is preserved.

## Validation

- issue reproducer: unbounded search now returns ID 1 at distance 100.0 instead of rejected ID 0
- `tests/test_search_params.py`: 52 passed
- HNSW C++ tests: 14 passed
- full CPU C++ test suite: 278 passed, 5 skipped
- clang-format and `git diff --check`: passed

Fixes https://github.com/facebookresearch/faiss/issues/5504

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5508

Reviewed By: alibeklfc

Differential Revision: D120364656

Pulled By: mnorris11

fbshipit-source-id: 074234b80cf7b2e06a85d57489a1c214dfd3195b
 2026-09-16 17:55:51 -07:00
Alibek Zhakubayev 1dbb3d25b0 Update backward compatibility test to v1.15.1 (#5646) 
Summary:
- Update backward compatibility test to use faiss-cpu=1.15.1

Note: the test was previously pinned to faiss-cpu=1.14.3, so this skips over 1.15.0.

**Do not land until faiss-cpu=1.15.1 is available on the pytorch conda channel** — the conda package is published by the tag-triggered release build. Landing earlier will break this workflow on main.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5646

Test Plan:
- [ ] CI passes
- [ ] Backward compatibility test uses correct version

Reviewed By: mnorris11

Differential Revision: D120265026

Pulled By: alibeklfc

fbshipit-source-id: 9459958af352716d5aef25e45043444105f22ff1
 2026-09-16 13:58:02 -07:00
Alibek Zhakubayev 75c755c5a1 Add missing CHANGELOG entry for #5632 in v1.15.1 (#5651) 
Summary:
- https://github.com/facebookresearch/faiss/issues/5632 (Add explicit IVFPQ scanner distance modes) landed between the changelog generation and the version bump in https://github.com/facebookresearch/faiss/issues/5645, so it ships in 1.15.1 but was missing from `CHANGELOG.md`.
- Adds it under `Added` in the 1.15.1 section.

Changelog-only; no code changes.

**Time-sensitive:** the v1.15.1 tag will be cut from the latest `main`. Landing this before tagging gets the entry into the released changelog.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5651

Test Plan: - [ ] CI passes (docs-only change)

Reviewed By: mnorris11

Differential Revision: D120360726

Pulled By: alibeklfc

fbshipit-source-id: 2854a7c44958ec4528ec87804382c312ea195ad2
v1.15.1 		2026-09-16 09:06:59 -07:00
Alibek Zhakubayev d63faeaf5a Increment to next release, v1.15.1 (#5645) 
Summary:
- Increment version to 1.15.1
- Update CHANGELOG.md with commits since v1.15.0 (57 commits)
- Bump version in CMakeLists.txt, setup.py, Index.h, INSTALL.md

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5645

Test Plan:
- [ ] CI passes
- [ ] CHANGELOG entries are correctly categorized

Reviewed By: mnorris11

Differential Revision: D120264788

Pulled By: alibeklfc

fbshipit-source-id: d95d68c9eb0c7c5b06aaeab7d8d0015b99d23145
 2026-09-15 20:42:13 -07:00
Jaeyoon Jeong 3b3711c112 Add explicit IVFPQ scanner distance modes (#5632) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5632

Add an explicit IVFPQ scanner mode that preserves precomputed-table behavior by default and can decode PQ codes to compute residual L2 distances on demand. Keep the existing scanner factory overload compatible.

Reviewed By: mdouze

Differential Revision: D119608525

fbshipit-source-id: b60b2c9771bdfd95d9bee2e81b72c9c3e7abb82c
 2026-09-15 17:32:23 -07:00
Alibek Zhakubayev 3bb48643ab Pin faiss-gpu-cuvs RAPIDS deps to the 26.06 build series (#5643) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5643

`faiss-gpu-cuvs` 1.15.0 fails at import on a clean install:

```
ImportError: libfaiss.so: undefined symbol: _ZTIN3rmm9bad_allocE
```

## Root cause, measured

RAPIDS 26.08 wrapped RMM in a per-release ABI inline namespace. `nm -D` on the published `librmm-cu13` wheels:

| Version | `typeinfo for rmm::bad_alloc` | Exported instead |
| --- | --- | --- |
| 26.2.0 | present | |
| 26.4.0 | present | |
| 26.6.0 | present | |
| 26.8.0 | **absent** | `typeinfo for rmm::_RMM_26_8::bad_alloc` |

`pyproject-gpu-cuvs.toml` builds against `==26.06.*` in `before-all` but declares `>=26.06,<27` at runtime and in `test-requires`. A fresh install therefore pairs a 26.06-built `libfaiss.so` with a 26.8 `librmm.so` that renamed every `rmm::` symbol. Nothing vendored covers the gap: `repair-wheel-command` hands every RAPIDS `lib*.so*` under `/opt/cuvs` to `auditwheel --exclude`.

The 1.15.0 wheel was uploaded 2026-08-03, two days before RAPIDS 26.08 reached PyPI. It published green and began failing for every new installer afterwards, which matches `build-pip-gpu / Build & test faiss-gpu-cuvs` going red on 2026-08-05 after 68 consecutive passes.

An exact-series pin rather than a narrower range is required: the inline namespace encodes the release, so **any** range spanning more than one RAPIDS minor series is broken by construction.

## This diff

- Pin `libcuvs-cu13`, `libraft-cu13`, and `librmm-cu13` to `==26.06.*` in both `dependencies` and `test-requires`.
- Declare `libraft-cu13`. `faiss/python/__init__.py` imports `libraft` and calls `libraft.load_library()`, but only `before-all` installed it, so at runtime its version floated on `libcuvs-cu13` transitive metadata.
- Correct the cuSOLVER soname in the `auditwheel` exclude list from `libcusolver.so.11` to `libcusolver.so.12`, which is what CUDA 13.2 ships.
- Update the `_preload_gpu_libs` fix-it message, which told users to install the range this diff removes.
- Add `TestCuvsRapidsPinning` to `tests/test_wheel_smoke_gpu.py`.

## Regression guard

`tests/test_wheel_smoke_gpu.py` is already the `test-command` for the cuVS wheel, so the new class runs inside `build-pip-gpu.yml` with no workflow change and no new test file. It asserts three properties of the built wheel's own `Requires-Dist`:

1. every RAPIDS distribution `_preload_gpu_libs` dlopens is declared
2. each is pinned to `==26.06.*` rather than an open range
3. the installed versions sit in that series

`_RAPIDS_SERIES` duplicates the series from `pyproject-gpu-cuvs.toml` because the wheel does not ship that file. The duplication is the tripwire: a RAPIDS bump has to touch both places deliberately.

## Prior attempt

D117099095 made the version-pin change on 2026-08-23 and was abandoned on 2026-09-04 without landing. It never failed review. `faiss/public_tld/` is ShipIt-mirrored with `requireExportConfig`, so `github-export-checks` held it Unpublished pending a `facebookresearch/faiss` PR that a bot cannot open, and three pings over 12 days went unanswered. This diff needs that same export.

## Notes

The published 1.15.0 metadata declares only `libcuvs-cu13`, not `librmm-cu13`. That is not a fbsource/GitHub divergence: `librmm-cu13` was added to `dependencies` on 2026-08-14, after the 08-03 release, and has never shipped.

`rapids-logger` is installed by `before-all` but stays undeclared here. `librmm-cu13` owns that link-time dependency and carries it in its own metadata; an unversioned duplicate would not make the resolve safer.

Reviewed By: mnorris11

Differential Revision: D120162004

fbshipit-source-id: eb6dc316a4fba2a4228eab62d7b86ac8ea94a1b7
 2026-09-15 14:42:20 -07:00
Trang Nguyen 7c3b55c01b Make Panorama bounds checks overflow-safe (#5616) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5616

Follow up on D118495504 by closing two arithmetic gaps in the Panorama
bounds validation. Avoid signed overflow and reject negative counts in
`IndexFlatPanorama::reconstruct_n`, and compute the serialized
`IndexFlatPanorama` batch count without overflowing
`ntotal + batch_size - 1`.

Add regression coverage for negative reconstruction counts and a maximal
serialized batch size that previously admitted empty storage for a
non-empty index.

Reviewed By: mnorris11

Differential Revision: D119272683

fbshipit-source-id: 7ea841bd4bd90d0358b78797441fa3cc815db46a
 2026-09-14 21:26:48 -07:00
Trang Nguyen fb2d57a312 Fix leak and unbounded lists[] in OnDisk invlist deserialization (#5560) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5560

`OnDiskInvertedListsIOHook::read` allocates its `OnDiskInvertedLists` with a
bare `new` and then keeps reading. Any throw after that point leaks the whole
object -- including `od->lists`, which `READVECTOR` will happily size up to
`get_deserialization_vector_byte_limit()`. Throwing after that point is the
common case for malformed input: the `slots` and `filename` reads can fail, and
`do_mmap()` on an attacker-controlled filename fails essentially always.

This is what the `faiss_read_index_binary_fuzzer` health check has been tripping
over. The one CI stack that survived tpx truncation names
`OnDiskInvertedLists.cpp:722` (`vector<OnDiskOneList>::resize`) as the largest
live allocation, reached via `IBFf` -> `read_index` -> `IxRP` -> IVF ->
`read_InvertedLists`. Because the harness runs with `detect_leaks: 0`, the
leaked blocks accumulate across a 120 s session; peak RSS across the failing
runs climbed 978 -> 1142 -> 1249 -> 1815 -> 4356 MB against `-rss_limit_mb=4096`.
That is why the check reads as flaky (23.6%) rather than as a hard red, and why
the saved crash artifacts do not reproduce when replayed one at a time.

Switch both `read` and `read_ArrayInvertedLists` to hold the object in a
`std::unique_ptr` and `release()` on success. The public signature still returns
`InvertedLists*`, so this is a local scope guard only -- the same shape
`read_InvertedLists_up` in `index_read.cpp` already uses.

Also reject `lists.size() != nlist` at read time. Every `OnDiskInvertedLists`
accessor (`list_size`, `get_codes`, `get_ids`, ...) indexes `lists[list_no]`
with `list_no` bounded only against `nlist`, so a file whose `lists` vector is
shorter than its `nlist` is a straight out-of-bounds read. Every construction
path in the class keeps `lists.size() == nlist`, so this rejects only malformed
input.

Reviewed By: mnorris11

Differential Revision: D118487565

fbshipit-source-id: 6f01c937b24a2a98620e4ca2bde141674af1b19d
 2026-09-14 21:23:31 -07:00
Michael Norris a74867cf56 faiss HNSW: delete the flag and the lock-based graph build (#5611) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5611

D112025877 added the `faiss::hnsw_deterministic_build` flag. The flag chose
between the lock-based graph build and the deterministic graph build. This
diff deletes the flag. The deterministic build is now the only build.

Reviewed By: trang-nm-nguyen

Differential Revision: D118342471

fbshipit-source-id: d3c5e5238aca2bcda463eb27959ec22827222c15
 2026-09-14 13:13:11 -07:00
Ge Gao 4fdbc9dc18 Support ZeroCopyIOReader for IndexRaBitQ deserialization (#5629) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5629

`IndexRaBitQ` keeps its database in `codes`, a `MaybeOwnedVector<uint8_t>` inherited from `IndexFlatCodes`, but both read branches (`Ixrq` for single-bit and `Ixrr` for multi-bit) used the raw `READVECTOR` macro. So `codes` was always an owning heap vector even when the caller passed a view-capable reader, while every other flat-codes index already goes through `read_vector`.

This switches both branches to `read_vector`, which dispatches on the reader type in `read_vector_base` and makes `codes` a view of the caller's buffer. Any other reader falls back to `READVECTOR`, so existing callers are unaffected.

A zero-copy RaBitQ index now works like this:
```
#include <faiss/impl/zerocopy_io.h>

faiss::ZeroCopyIOReader reader(buffer.data(), buffer.size());
std::unique_ptr<faiss::Index> index_zc(faiss::read_index(&reader));
```
The buffer must outlive the index, as with any `faiss::ZeroCopyIOReader`.

Reviewed By: mnorris11

Differential Revision: D119297402

fbshipit-source-id: 95c5ba050cb3a0636a0899efbe886616a7a8a2fd
 2026-09-14 13:03:13 -07:00
Michael Norris b5a14632a6 Run the eight SIMD kernel tests that no build executed (#5620) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5620

**TL;DR:** Eight SIMD kernel tests never ran, and two byte-domain kernels trust their caller. This change makes three fixes:

1. **Six tests never compiled.** `test_rabitq_simd.cpp` hides them behind `COMPILE_SIMD_*` macros that the faiss target does not export.
2. **Two tests skipped themselves in a static build.** `TestSQByteDirectAcrossSIMDLevels` needs `SIMDLevel::NONE` as a reference, and a static build does not report that level.
3. **The byte kernels never test their dimension requirement.** `DistanceComputerByte` and `DistanceComputerByteSigned` read 16 bytes per loop step.

## 1. Six tests never compiled

`test_rabitq_simd.cpp` wraps two blocks in `#ifdef COMPILE_SIMD_AVX2` and `#ifdef COMPILE_SIMD_AVX512`. The faiss target sets those macros as `preprocessor_flags`, and it does not export them. A test translation unit never sees them. So the preprocessor deleted both blocks. The file holds nine tests and ran three.

The AVX2 block loses its guard, because every x86 faiss build holds the AVX2 kernels. Its three tests also lose an `#ifdef FAISS_ENABLE_DD` around a runtime skip, which the same problem disabled. `is_simd_level_available()` reports the right answer in a static build too.

The AVX512 block moves to a dynamic dispatch target. A static x86 build compiles the AVX2 kernels only, so the test fails to link there rather than skips. `test_rabitq_simd_vpopcnt.cpp` becomes `test_rabitq_simd_dd.cpp`, because it now holds every RaBitQ test that a static build cannot link. A new `test_rabitq_simd_util.h` holds the scalar reference that both files call.

## 2. Two tests skipped themselves in a static build

`TestSQByteDirectAcrossSIMDLevels` measured the index a second time at `SIMDLevel::NONE`, then compared the two runs. A static build reports only the level it compiles, and omits `NONE`, so both tests skipped.

The tests now compute the distances from the definition. The distances are integers, so the reference is exact. The tests check absolute values now, instead of the agreement of two faiss paths.

## 3. The byte kernels trust the caller

`DistanceComputerByte` and `DistanceComputerByteSigned` read 16 bytes per loop step, so they require `d % 16 == 0`. `sq-dispatch.h` enforces that, and no other caller builds them.

A caller that ignores the requirement reads up to 15 bytes past each code and returns a wrong distance. Each constructor now tests the requirement and throws. The AVX2 and the ARM_NEON specializations both get the test.

Reviewed By: alibeklfc

Differential Revision: D119545379

fbshipit-source-id: ddfdcc30d3662f469b2ec3dbb9d5b9f4dafa21aa
 2026-09-10 20:05:56 -07:00
Michael Norris 6bbb068ad8 Measure 20-byte Hamming codes eight at a time (#5587) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5587

**TL;DR:** Adds a batched 20-byte Hamming kernel that measures eight codes per call, the only thing that helps the 160-bit width, worth 1.21x end to end.

A 20-byte code has no vector kernel at any x86 SIMD level. `HammingComputer20` measures one distance with three scalar popcounts at every level, including `AVX512_VPOPCNT`, so a scan of 160-bit codes runs the same code everywhere.

Vectorizing a single 20-byte distance does not help. A masked 256-bit load plus `vpopcntq` plus a horizontal reduction measures the same as the scalar body, because the reduction costs more than the three popcounts it replaces.

Measuring several codes per call does help, because it amortizes that reduction. This adds `hamming_batch()` to `HammingComputer20` at `AVX512_VPOPCNT`, which measures eight codes at once:

- Eight codes span 160 bytes. `_mm512_popcnt_epi8` gives a count per byte.
- `_mm512_sad_epu8` sums each aligned 8-byte group.
- A 20-byte code covers two whole groups plus half of a group it shares with its neighbour, so each code needs one scalar popcount for its half.

`IVFBinaryScannerL2::scan_codes` and `IVFBinaryScannerL2::scan_codes_range` call `hamming_batch()` when the computer declares a `batch_size`, then measure the remaining codes one at a time. A computer without a `batch_size` keeps the existing loop, so no other code size changes.

The scanner holds the tiled query, and the constructor rejects a code size that does not match the computer. A caller therefore cannot reach the batch kernel with a stride the kernel does not read.

The byte-wise popcount needs `AVX512_BITALG`, which is a separate CPUID bit from `AVX512_VPOPCNTDQ`. This adds the flag to the `AVX512_VPOPCNT` level and requires both bits to select it. Every CPU that has VPOPCNTDQ also has BITALG, so no CPU loses the level.

An alternative byte-wise popcount built from `vpshufb`, which needs neither bit and would run at the plain `AVX512` level, measures only 1.08x against 1.54x, so it is not worth the wider reach.

Reviewed By: alibeklfc

Differential Revision: D118971285

fbshipit-source-id: 5fd4637c972e5c55e2695b32b9b15fba8181a242
 2026-09-10 09:32:36 -07:00
Michael Norris 3c7042f17a Let the binary Hamming paths reach the VPOPCNT kernels 
Summary:
**TL;DR:** Points `IndexBinaryIVF` at the VPOPCNT level and instantiates its scanner in the translation unit compiled with `-mavx512vpopcntdq`, worth 1.58x at 256-bit and 1.41x at 512-bit on Genoa.

Two things kept the binary Hamming paths on scalar popcount, whatever the host.

`IndexBinaryIVF` builds its scanner and runs its searches through `with_simd_level`, which uses the BASE level mask. That mask holds no `AVX512_VPOPCNT` bit, so on a host that reports that level the dispatch falls through to `AVX512`, where the wide computers use scalar popcount.

Two build variants, `faiss_omp_mock` and `faiss_no_multithreading`, hardcoded dynamic dispatch off. They compiled the AVX2 kernels only, so a consumer of either variant could not use runtime dispatch at all, whatever the build setting said.

This change:

- Moves the three dispatch sites in `IndexBinaryIVF.cpp` to `with_simd_level_with_vpopcnt`. Below that level the dispatch falls through to `AVX512`, as before.
- Instantiates `IndexBinaryIVF_impl.h` in `hamming_avx512_vpopcnt.cpp`. That is the one translation unit compiled with `-mavx512vpopcntdq`, which the VPOPCNT computers need. Without it the new dispatch has no symbol to call. The include must follow the computer specializations, so it is placed after them.
- Lets both build variants honour the dynamic dispatch setting.

`HammingComputer16` and `HammingComputer20` still inherit the scalar body at this level, so a 16-byte or 20-byte code gains nothing here.

`IVFBinaryScannerL2::scan_codes` also reads the heap top into a local `bound` instead of loading `simi[0]` on every code. Only `scan_codes` writes `simi`, so it refreshes `bound` after each `heap_replace_top` and the two forms return the same results. The test plan measures it.

Reviewed By: alibeklfc

Differential Revision: D118874433

fbshipit-source-id: dc74294ecaf178919fdd713abc60c44bbaa80347
 2026-09-10 09:32:36 -07:00
lyang24 e6bd1d252c Add AVX512 VPOPCNT dynamic dispatch level (#5531) 
Summary:
## What

  Split the existing VPOPCNTDQ RaBitQ and Hamming kernels from the full Sapphire Rapids SIMD level.

  DD builds now expose an `AVX512_VPOPCNT` capability for CPUs such as Ice Lake and Zen 4. The fallback chain is:

  `AVX512_SPR -> AVX512_VPOPCNT -> AVX512 -> AVX2 -> NONE`

  The ordinary static AVX512 target remains unchanged and contains no VPOPCNT instructions.

  ## Why

  The VPOPCNT kernels only require baseline AVX-512 plus `AVX512_VPOPCNTDQ`. Tying them to the SPR level unnecessarily excluded CPUs that support VPOPCNTDQ but not AVX512-FP16, notably Zen 4.

  ## Changes made while importing

  Two changes were needed on top of the pull request.

  **Feature detection read the wrong CPUID leaf.** `AVX512_VPOPCNTDQ` and `AVX512_VNNI` were read from leaf 1 ECX. Both live in leaf 7 subleaf 0 ECX, at bits 14 and 11. Leaf 1 ECX holds unrelated bits at those positions, so the new level was never selected on any CPU. Measured on an AMD Genoa host before the fix: the level reported as unavailable and detection returned `AVX512`. After the fix the same host reports `AVX512_VPOPCNT`.

  **The build definitions outside CMake were not updated.** The pull request renames three files. The other build description still named the old paths and had no entry for the new level, so the library did not build. This adds the level constant, its compiler flags, its `COMPILE_SIMD_AVX512_VPOPCNT` define, and the renamed paths.

**The new level was not registered in the compiled-level mask.** `compiled_simd_levels()` reports which levels a binary holds, and `FAISS_SIMD_LEVEL` walks down from the requested level until it finds one that is present. The new level had no entry, so the override always stepped past it to `AVX512` and printed that the level was not compiled in. This was reported by `test_dispatch_with_env_var` on the open-source dynamic-dispatch build.

  **The test helper inferred available levels from `/proc/cpuinfo`.** A CPU feature does not imply that a level is compiled into the build, so the helper listed a level the override then refused. It now asks `SIMDConfig::is_simd_level_available()`, which reports compiled and supported. That is a subset of compiled, so every level it returns is one the override accepts.

  The mask and helper names also follow the naming used in the parent diff, so `AVAILABLE_SIMD_LEVELS_BASE_WITH_VPOPCNT` and `with_simd_level_with_vpopcnt`.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5531

Test Plan:
```
buck2 test fbcode//mode/opt -c faiss.dynamic_dispatch=true \
  fbcode//faiss/tests:test_simd_levels fbcode//faiss/tests:test_rabitq_simd
```
Result: all pass.

The build failures the imported version reported are cleared. All three configurations build:
```
buck2 build fbcode//mode/dev fbcode//faiss:faiss fbcode//faiss:faiss_no_multithreading fbcode//faiss:faiss_omp_mock
buck2 build fbcode//mode/opt -c faiss.dynamic_dispatch=true <same three targets>
buck2 build fbcode//mode/dev -c fbcode.arch=aarch64 <same three targets>
```

Detection on an AMD Genoa host, which has AVX512_VPOPCNTDQ and AVX512_BF16 but not AVX512_FP16:

| | before the CPUID fix | after |
| --- | --- | --- |
| detected level | `AVX512` | `AVX512_VPOPCNT` |
| `AVX512_VPOPCNT` available | 0 | 1 |
| `AVX512_SPR` available | 0 | 0 |

Reviewed By: alibeklfc

Differential Revision: D118969754

Pulled By: mnorris11

fbshipit-source-id: d1ca52b614829a3d4b322c9fecf0c004f2174dab
 2026-09-10 09:32:36 -07:00
Michael Norris 9f993087b0 Name the SIMD level masks after what they hold (#5600) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5600

**TL;DR:** Renames the dispatch masks from `A0`/`A1`/`A2` to names that say which levels they hold; no behaviour change.

The dispatch masks were named `A0`, `A1` and `A2`. The names say nothing about which levels a mask holds, so a caller had to read `simd_dispatch.h` to pick one. That is how several call sites ended up on a mask with no `ARM_SVE` bit while a dedicated SVE kernel existed.

This renames every mask and helper after its contents. It changes no behaviour.

| Old name | New name | Holds |
| --- | --- | --- |
| `AVAILABLE_SIMD_LEVELS_A0` | `..._BASE` | NONE, AVX2, AVX512, ARM_NEON, RISCV_RVV |
| `AVAILABLE_SIMD_LEVELS_A0_SPR` | `..._BASE_WITH_SPR` | BASE + AVX512_SPR |
| `AVAILABLE_SIMD_LEVELS_A1` | `..._BASE_WITH_SVE` | BASE + ARM_SVE |

The helpers follow: `with_simd_level_a0_spr` becomes `with_simd_level_with_spr`, and `with_simd_level_a1` becomes `with_simd_level_with_sve`.

`BASE` is the right name for the default mask because `ARM_NEON` belongs to it. NEON is mandatory on aarch64, the way AVX2 is the x86 baseline, while `ARM_SVE` is optional. So the ARM fallback chain is SVE to NEON to NONE.

Reviewed By: alibeklfc

Differential Revision: D119025991

fbshipit-source-id: 2b6681de2d83df6468569a365d05341088742cf8
 2026-09-10 09:32:36 -07:00
Michael Norris fc13c81fb8 Stop aarch64 falling back to SIMDLevel::NONE in faiss dispatch (#5572) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5572

**TL;DR:** Stops aarch64 running scalar code where a NEON or SVE kernel exists, adds the two missing NEON byte-domain kernels, and fixes `FAISS_SIMD_LEVEL` falling all the way to `NONE` on an uncompiled level.

On aarch64 several faiss dispatch paths ran scalar code, or ran an `ARM_NEON` kernel where an `ARM_SVE` kernel already existed. This change enforces the order SVE -> NEON -> NONE on an ARM host. It also adds the one kernel pair that was missing.

T287037898 reported the scalar-quantizer half. On aarch64 `IndexScalarQuantizer` with `QT_8bit_direct` or `QT_8bit_direct_signed` saved memory but lost throughput. `Refine(SQ8)` has the same problem, because `IndexRefine::search` calls `refine_index->get_distance_computer()`.

## What each call site gets

| Call site | Before | After |
| --- | --- | --- |
| `QT_8bit_direct` on aarch64 | float-domain `DCTemplate` | NEON `DistanceComputerByte` |
| `QT_8bit_direct_signed` on aarch64 | float-domain `DCTemplate` | NEON `DistanceComputerByteSigned` (new) |
| `IndexFlat` distance computers, SVE host | `ARM_NEON` | `ARM_SVE` |
| `AdditiveQuantizer::compute_centroid_norms`, SVE host | `ARM_NEON` | `ARM_SVE` |
| `SuperKMeans` `block_l2`, SVE host | `ARM_NEON` | `ARM_SVE` |
| `pq_code_distance` wrappers, SVE host | `ARM_NEON` | `ARM_SVE` |
| `with_VectorDistance`, SVE host | `ARM_NEON` | `ARM_SVE` |
| `FAISS_SIMD_LEVEL=ARM_SVE`, build without SVE | `NONE` | nearest compiled level |

## Three causes, fixed separately

**1. Two missing kernels.** `sq-neon.cpp` held a scalar `DistanceComputerByte<Sim, ARM_NEON>` that nothing could reach, and no `DistanceComputerByteSigned<Sim, ARM_NEON>` at all. Both are now real NEON kernels. L2 and the unsigned inner product use `vabdq_u8`, `vmull_u8` and `vpadalq_u16`. The bias-encoded inner product uses `veorq_u8`, `vmull_s8` and `vpadalq_s16`. Both agree bit for bit with the AVX2 specializations, which the tests require.

Two invariants deserve a note:

- The L2 path keeps an unsigned accumulator. A `vmull_u8` square reaches 255^2 = 65025, which an int16 lane would read as negative.
- For x in 0 to 255, `x ^ 0x80` read as `int8` is exactly `x - 128`. That is how the kernel removes the +128 bias before `vmull_s8`.

**2. Dispatch chains that list only x86 levels.** The `if constexpr` chains in `sq-dispatch.h` enumerated x86 levels only, so an ARM host fell through to the float path. This adds `ARM_NEON` to four chains: two in `select_distance_computer_body`, and two in `sq_select_InvertedListScanner`. The chain in `is_dimension_compatible` already included ARM.

This does not add `ARM_SVE`. The scalar-quantizer entry points dispatch with a mask that holds no `ARM_SVE` bit, so an SVE host already falls through to the `ARM_NEON` case and now gets these kernels. Adding `ARM_SVE` would instantiate the empty primary template in `distance_computers.h`. This corrects that template's stale comment.

**3. Level masks that hide existing SVE kernels.** The default mask holds no `ARM_SVE` bit, so the dispatch fell through `case ARM_SVE` to `ARM_NEON`. This uses `with_simd_level_a1` at every site where a real SVE kernel exists and links: `IndexFlat.cpp` at four sites, `AdditiveQuantizer::compute_centroid_norms`, `block_l2` in `SuperKMeans.cpp`, all three wrappers in `pq_code_distance-generic.cpp`, and `with_VectorDistance` in `distances_dispatch.h`.

The `IndexFlat` sites matter most. `faiss::fvec_L2sqr()` already used the SVE mask, so on an SVE host the free function ran SVE while `IndexFlatL2`'s distance computer ran NEON.

Also: `FAISS_SIMD_LEVEL=ARM_SVE` on a build without SVE compiled in used to skip every level and run at `NONE`, because `with_selected_simd_levels` has no case label for an uncompiled level. It now walks down to the nearest compiled level. The override is still honoured when the CPU lacks the level, since forcing a level is the point of it. Only uncompiled levels are corrected.

The unused `AVAILABLE_SIMD_LEVELS_A2` is deleted. It is the NEON-to-NONE trap in constant form, with no users.

This change keeps the existing mask names. A follow-up renames them to say what they hold.

## Out of scope

In rough order of remaining value:

- `IndexPQ.cpp` and `IndexIVFPQ.cpp` still pin PQ to `ARM_NEON` on an SVE host. A mask change is not enough. `pq_code_distance-sve.cpp` includes only `pq_scan_impl.h`, and `with_HammingComputer<ARM_SVE>` has no complete type.
- `distances_aarch64.cpp` forwards five `ARM_NEON` specializations to `<SIMDLevel::NONE>`, so a non-SVE aarch64 host runs scalar code in the IVFFlat scan.
- `rabitq_neon.cpp` forwards all six `ARM_NEON` specializations to `<SIMDLevel::NONE>`. There is no SVE variant.
- There is no `block_l2<ARM_NEON>` and no `exhaustive_L2sqr_blas_cmax<ARM_NEON>`.
- Part two of T287037898, which is SDOT and SMMLA. `FEAT_DotProd` and `FEAT_I8MM` are not expressible as a `SIMDLevel`. The right shape is a runtime `getauxval(AT_HWCAP)` check inside the ARM translation unit, which follows the `SIMDConfig::avx512_split` precedent. Note the reporter's caveat: SMMLA shows no improvement until the scan loop is tiled, so the tiling must land in the same change.

Reviewed By: alibeklfc

Differential Revision: D118682598

fbshipit-source-id: 1ab588d9fa130e8246bf996025fc11d8954323be
 2026-09-10 09:32:36 -07:00
Michael Norris 00928e3f76 Fix ROCm CI: build in a dedicated conda env instead of the runner's broken base (#5545) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5545

The `linux-x86_64-GPU-w-ROCm-cmake` job fails installing AMD's ROCm torch wheels: they exist for one CPython ABI only, and the MI325 runner's interpreter is 3.14, not the 3.12 the action asks for.

Nothing in `build_cmake` actually constrained the interpreter. `setup-miniconda` requests `python-version: '3.12'` and `Configure build environment` asks for `python=3.12` again, but the runner's conda base has `conda-meta` listing both `python 3.12.9` and `cpython 3.14.6`, so conda reports "already installed" while `bin/python` is 3.14. Forcing it does downgrade the interpreter, but strands conda's own compiled extensions (`pydantic_core`), and conda builds its argument parser through its plugins -- so every subsequent conda call in the job dies. Working around that with `CONDA_NO_PLUGINS` in turn drops the libmamba solver, which conda then rejects because `solver: libmamba` stays configured. Meanwhile the base env is inconsistent across three Python ABIs (conda lists ~200 conflicting packages), so there is no in-place repair to converge on.

So stop repairing base. For ROCm only, `conda create` a fresh `$CONDA/envs/faiss` prefix on 3.12 and point `$CONDA` at it, leaving base untouched so conda keeps running on its own interpreter. `$CONDA` is the single lever the later steps already read (`-DPYTHON_EXECUTABLE`, `setup.py install`, the import smoke tests), so redirecting it is enough.

Making that lever actually hold:
- Pass `-p "$CONDA"` on every `conda install`, and `conda activate "$CONDA"`. A bare `conda install` targets the active env, which is base regardless of `$CONDA`. No-op for the non-ROCm jobs, where `$CONDA` is base.
- Write base's `bin` to `$GITHUB_PATH` *before* the env's. The runner prepends entries as it reads them, so the last line written ends up first on PATH: this leaves the env's interpreter ahead of base's while keeping base reachable for `conda` itself.
- Drive the ROCm torch step through `"$CONDA/bin/python" -m pip` rather than bare `python`/`pip3`, so it does not depend on PATH ordering at all.
- `rm -rf` the prefix before `conda create`: it survives between runs on this self-hosted runner and `conda create` refuses an existing one. A fresh prefix each run is the point.
- Derive the `libpthread_nonshared.a` symlink from `$CONDA` instead of a hardcoded `$HOME/miniconda3`, since `sysroot_linux-64` now installs into the build's prefix, and assert the target exists -- `ln -s` does not check, so a wrong path would surface much later as a link failure.
- `conda list -p "$CONDA"` for the diagnostics, so the logs describe the env under test rather than base.

Torch wheel installation:
- Resolve through `--find-links` as a flat index instead of hardcoding a `cp312-cp312-linux_x86_64` filename; the hardcoded tag is what made an interpreter mismatch surface as "not a supported wheel on this platform".
- Keep a retag fallback for pip builds that reject the unqualified `linux_x86_64` tag, but gate it on the resolver's own error text so a transient failure (network, disk) fails loudly instead of being retagged over.
- `--ignore-installed`, because this runner is self-hosted and a wrong-ABI torch from an earlier run otherwise satisfies the requirement and skips tag matching.
- Use AMD's wheels built against the system ROCm rather than the PyTorch Foundation `rocm7.2` wheels, which bundle an older ROCr lacking `hsa_amd_memory_get_preferred_copy_engine` and collide with the system `libamdhip64`. `ROCM_VERSION` is exported to `$GITHUB_ENV` so the apt release and the wheel release cannot drift apart.
- Assert on interpreter version, torch's install location, and `torch.version.hip` rather than printing them. Each of these failures previously surfaced several steps later as something unrelated-looking.

Note for infra: the base env on this runner is corrupt independently of this change (mixed 3.12/3.13/3.14 packages, and truncated `lib/python3.1/site-packages` paths from a `sys.version[:3]` bug upstream). Wiping it would let several of the workarounds above be dropped.

Reviewed By: alibeklfc

Differential Revision: D117896658

fbshipit-source-id: 26f98c53e5daf747dd81e0fd35232b4160f62377
 2026-09-10 08:01:09 -07:00
Trang Nguyen 5702cafcb7 Remove dead TurboQuant projection state (#5618) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5618

Remove the unused projection buffers and eager `init_projection` work from `ScalarQuantizer::TurboQuantRefine`. Quantizers construct the required projection from `trained` when used, so rebuilding it during training and deserialization only added an unbounded quadratic allocation and cubic QR operation.

Keep `seed` and `qjl_type` as the serialized configuration, enforce that the helper remains a small config-only struct, and cover both projection modes in deserialization tests.

___

Differential Revision: D119243693

fbshipit-source-id: 53f982b00dd849cbe88983ffd27809c2fa738a4c
 2026-09-09 11:05:57 -07:00
Trang Nguyen 545550f6a2 Use unittest assertions and f-strings in test_index_composite (#5571) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5571

`test_index_composite.py` used bare `assert` statements throughout. Inside a
`unittest.TestCase` these are worse than the `self.assert*` helpers in two
ways: a failure prints only `AssertionError` with no operand values, so you
have to re-run under a debugger to see what the actual number was, and the
whole assertion is compiled out under `python -O`, silently turning the test
into a no-op.

This converts every bare `assert` in the file to the matching `self.assert*`
call (`assertEqual`, `assertTrue`, `assertFalse`, `assertNotEqual`,
`assertGreater`) and replaces the two remaining printf-style format strings
with f-strings.

Mechanical change only — no assertion was strengthened, weakened, or
reordered, and no test logic was touched. The `raise AssertionError()` idioms
guarding `try`/`except RuntimeError` blocks are left alone; converting those
to `assertRaises` would change control flow and belongs in its own diff.

Split out of D118606761 so that the `IndexIVFFlatDedup::remove_ids` bug fix
stays reviewable on its own.

Note this file is exported to the open source facebookresearch/faiss repo, so
this diff needs an OSS export alongside it.

Reviewed By: mnorris11

Differential Revision: D118721255

fbshipit-source-id: b6933a6119955f2dbd7590f4e39632d700a537c4
 2026-09-08 14:08:01 -07:00