mirror of
https://github.com/facebookresearch/faiss.git
synced 2026-10-12 02:13:36 +00:00
561e6ecdf3337e32542b82c735bfaf83d9e3afaa
2269
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 William
|
561e6ecdf3 |
Merge shard results by metric type rather than assuming similarity (#5509)
Summary:
`IndexShardsTemplate::search` picked the comparator for merging per-shard results by testing for `METRIC_L2`:
```cpp
if (this->metric_type == METRIC_L2) {
merge_knn_results<idx_t, CMin<distance_t, int>>(...); // smaller is better
} else {
merge_knn_results<idx_t, CMax<distance_t, int>>(...); // larger is better
}
```
Everything that is not `METRIC_L2` was therefore merged as a similarity. `METRIC_L1`, `METRIC_Linf` and `METRIC_Lp` return distances, so once results crossed a shard boundary they came back farthest-first, while the same vectors in a single unsharded index came back nearest-first.
This PR selects the comparator with the existing `is_similarity_metric()` helper from `MetricType.h`, which is exactly the distinction the branch needs.
Fixes https://github.com/facebookresearch/faiss/issues/5503
## Behaviour change
`is_similarity_metric()` is true only for `METRIC_INNER_PRODUCT` and `METRIC_Jaccard`, so:
| metric | before | after |
| --- | --- | --- |
| `METRIC_L2` | `CMin` | `CMin` (unchanged) |
| `METRIC_INNER_PRODUCT` | `CMax` | `CMax` (unchanged) |
| `METRIC_L1`, `METRIC_Linf`, `METRIC_Lp`, `METRIC_Canberra`, `METRIC_BrayCurtis`, `METRIC_JensenShannon` | `CMax` | **`CMin`** |
| `METRIC_Jaccard` | `CMax` | `CMax` (unchanged) |
`IndexShardsTemplate<IndexBinary>` is also unaffected: `IndexBinary::metric_type` defaults to `METRIC_L2`, which keeps `CMin` as before.
## Reproduction
Against the released `faiss-cpu` 1.15.0 wheel, using the reporter's example:
```python
import faiss, numpy as np
xb0 = np.array([[0., 0.]], dtype='float32')
xb1 = np.array([[10., 0.]], dtype='float32')
xq = np.array([[1., 0.]], dtype='float32')
flat = faiss.IndexFlat(2, faiss.METRIC_L1)
flat.add(np.vstack([xb0, xb1]))
print(flat.search(xq, 2))
s0 = faiss.IndexFlat(2, faiss.METRIC_L1); s0.add(xb0)
s1 = faiss.IndexFlat(2, faiss.METRIC_L1); s1.add(xb1)
shards = faiss.IndexShards(2, False, True)
shards.add_shard(s0); shards.add_shard(s1)
print(shards.search(xq, 2))
```
```
single IndexFlat D,I = [[1. 9.]] [[0 1]] # nearest first, correct
IndexShards D,I = [[9. 1.]] [[1 0]] # farthest first, reversed
```
## Test
Adds `Shards::test_shards_distance_metric_ordering` to `tests/test_meta_index.py`. It splits the dataset across three `METRIC_L1` shards and checks that each result row is ordered nearest-first and matches the distances an unsharded `IndexFlat(METRIC_L1)` returns. Distances rather than labels are compared so that equidistant neighbours may be returned in either order.
The test fails on `main` (the rows come back reversed) and passes with this change.
## Testing notes
I reproduced the bug against the released 1.15.0 wheel as shown above, but I was not able to build faiss from source on this machine (Windows, no local C++ toolchain), so I have not executed the C++ build or run the test suite locally. The added test's pass/fail claim above follows from the comparator change rather than from a local run. Please treat CI as the gate, and I am happy to adjust if anything in the suite disagrees.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5509
Reviewed By: juancarpio27
Differential Revision: D117536893
Pulled By: mnorris11
fbshipit-source-id: 8568437eb596a202262fc455748063fafca202c1
|
||
|
Fu Xiaonan
|
613e0acac5 |
Fix IndexIDMap range search parameters (#5536)
Summary: - preserve the concrete search-parameter type when `IndexIDMap::range_search` translates an external-ID selector - retain IVF-specific per-query fields such as `nprobe` by temporarily replacing only `sel`, using the same scoped restoration pattern as `search_ex` - add a deterministic regression test with two IVF lists and external IDs ## Testing - confirmed the new test fails against FAISS 1.14.2 with `IndexIVF params have incorrect type` - built the current source on macOS arm64 with Python bindings (`FAISS_OPT_LEVEL=generic`, GPU and Metal disabled) - `python -m pytest -q tests/test_search_params.py` (51 passed) - `clang-format 21 --dry-run --Werror faiss/IndexIDMap.cpp` - `git diff --check` Closes https://github.com/facebookresearch/faiss/issues/5523 Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5536 Reviewed By: mnorris11 Differential Revision: D117233222 Pulled By: juancarpio27 fbshipit-source-id: 28e750f63fb681cc915ad41443c582739d8c3597 |
||
|
Juan Carpio Castellanos
|
8b01f308f0 |
Validate IndexScalarQuantizer codes buffer size on deserialization (#5540)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5540 When reading a serialized ScalarQuantizer-based flat index, the codes buffer length was not checked against ntotal * code_size, unlike the equivalent read paths for other flat-codes index types. A crafted index file with an inflated ntotal but a short codes buffer causes out-of-bounds reads during decode/reconstruct. Add the same size check used by the other index types right after the codes buffer is read. ___ Differential Revision: D117319393 fbshipit-source-id: 70dd1f09c8daf7393b992908fa808b111d0a02a1 |
||
|
ThatikondV
|
317a207bca |
ARM SVE optimization for L2 distance kernels adapted to new simd_impl structure (#5098)
Summary: This PR introduces ARM SVE (Scalable Vector Extension) optimization for L2 metric functions. The new implementation extends the existing NEON/SVE SIMD logic with SVE vectorized kernels for distance computations, improving both indexing and search performance. **Benchmarking Setup:** To evaluate the performance impact of the new SVE-enabled L2 distance computations, a dedicated benchmarking script was developed. This script systematically compares the earlier FAISS version and the latest SVE-optimized version using the SIFT1M dataset with 128-dimensional vectors. The benchmarks cover all major index types like FLAT, IVF_FLAT, IVF_PQ, IVF_SQ8, and HNSW. **Results:** Index | OSS QPS | SVE QPS | Improvement -- | -- | -- | -- Flat (L2) | 888.53 | 1188.12 | 33.70% IVF1024-Flat | 33,437.76 | 45,871.00 | 37.20% IVF1024-PQ32x8 | 33,597.01 | 40,650.00 | 21.00% HNSW (M=16, ef=64) | 72,837.67 | 77,482.00 | 6.40% IVF1024-SQ8 | 17,002.79 | 20,657.00 | 21.50% The above results clearly demonstrate that enabling SVE vectorization yields consistent performance gains across all tested index types, where both indexing and search times improved significantly. These improvements validate the effectiveness of the SVE implementation in accelerating L2 distance computations on modern ARM platforms. **Co-author:** This work was jointly developed with abhijain1204fujitsu. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5098 Test Plan: Run on an aarch64 host with SVE2, `svcntw() == 4` (128-bit vectors). **The SVE code only exists in dynamic-dispatch builds.** In the default static aarch64 Buck build, `xplat.bzl` filters `utils/simd_impl/distances_arm_sve.cpp` out of `srcs` and `SINGLE_SIMD_LEVEL` resolves to `ARM_NEON`, so a static-mode run exercises nothing here. CI runs both modes. ## Correctness ``` buck2 test -c faiss.dynamic_dispatch=true \ fbcode//faiss/tests:test_distances_simd \ fbcode//faiss/tests:test_distances_dispatch \ fbcode//faiss/tests:test_pq_encoding \ fbcode//faiss/tests:test_ivfpq_indexing \ fbcode//faiss/tests:test_ivfpq_codec \ fbcode//faiss/tests:test_pq_code_distance \ fbcode//faiss/tests:test_disable_pq_sdc_tables # Pass 32. Fail 0. buck2 test -c faiss.dynamic_dispatch=true \ fbcode//faiss/tests:test_build_blocks fbcode//faiss/tests:test_product_quantizer # Pass 105. Fail 0. buck2 test -c faiss.dynamic_dispatch=true fbcode//faiss/tests:test_index # Pass buck2 test fbcode//faiss/tests:test_distances_simd fbcode//faiss/tests:test_distances_dispatch # static mode: Pass 9. Skip 6 (only one SIMD level available). ``` `buck2 build fbcode//mode/opt fbcode//faiss:faiss -c faiss.dynamic_dispatch=true` and the static build both clean. `arc f` / `arc lint` clean. `test_distances_dispatch` forces every available level and compares against `SIMDLevel::NONE`; in DD mode on this host it reports `NONE`/`ARM_NEON`/`ARM_SVE` and no longer skips, so the `ARM_SVE` specializations are genuinely under test. Added coverage in `tests/test_distances_simd.cpp`: - `TestFvecL2sqrNy.D2`: `dim` swept over `{1,2,3,4,5,8,12,16,24,32,48,64}` (was `{2,4,8,12}`) so the `lanes1`..`lanes4` kernels are reached on 128/256/512-bit SVE, not just the `d1/d2/d4/d8` cases. - Both `_y_transposed` tests: `ny` swept over `{5,8,11,16,32}` (was hardcoded `11`, which never produced a full-vector block on any SVE width, so the no-tail path was untested). ## Kernel microbenchmark `mode/opt`, DD, pinned core, alternating runs, min-of-N x median-of-7, run-to-run spread <1%. `ny = 256` (ksub), `d = dsub`. vs this diff's parent: | kernel | speedup range over parent | |---|---| | `fvec_L2sqr_ny` | +17% to +725% (dsub 3 and 6 hit the scalar fallback, unchanged) | | `fvec_L2sqr_ny_nearest` | +48% to +130% | | `fvec_L2sqr_ny_transposed` | +234% to +298% | | `fvec_L2sqr_ny_nearest_y_transposed` | +318% to +658% | ## Index-level `faiss.contrib.datasets.SyntheticDataset`, d=128, nb=400k, nq=2k, k=1, single-threaded, min/best of 3 alternating rounds. "kernels" = SVE kernels with the pre-existing A0 dispatch; "wired" = this diff. PQ encode, `add()` seconds: | Index | parent | kernels only | wired | speedup | |---|---|---|---|---| | IVF1024-PQ32x8 | 16.17 | 16.16 | 6.32 | **2.56x** | | PQ32x8 (flat) | 14.80 | 14.82 | 4.97 | **2.98x** | | Flat / IVF-Flat / HNSW / IVF-SQ8 | - | - | - | 1.00x | Search QPS is unchanged everywhere (-0.9% to +0.5%, inside 0.2-3.0% run-to-run variance). recall@1 is identical to the parent for every index. The "kernels only" column is 1.00x because `ProductQuantizer` dispatched through `with_simd_level()` = `AVAILABLE_SIMD_LEVELS_A0`, which lacks the `ARM_SVE` bit, so the `case SIMDLevel::ARM_SVE:` arm failed its `if constexpr` and fell through to `ARM_NEON`. Verified by putting `abort()` in `fvec_L2sqr_ny_nearest<ARM_SVE>`: PQ encoding completed without hitting it, while `test_distances_dispatch` (A1) aborted immediately. The three `with_simd_level_a1` call sites in `impl/ProductQuantizer.cpp` are what close that gap. `compute_inner_prod_table` still uses A0 and so cannot reach the pre-existing `fvec_inner_products_ny<ARM_SVE>`; left alone as it is outside this diff's L2 scope and unmeasured here. Reviewed By: junjieqi Differential Revision: D106149015 Pulled By: mnorris11 fbshipit-source-id: c1144fc8fd0eefb30c9dcc8084b472152623a88e |
||
|
tandede
|
2832ac3f67 |
Fix empty-index BLAS search results (#5528)
Summary: - initialize result handlers when an exhaustive search has no database vectors - process empty results in the configured query block size, preserving bounded temporary memory for large query batches - cover L2 and inner-product search across top-1, heap, and reservoir result handlers ## Problem Searching an empty `IndexFlat` produced correct sentinel values for small query batches, but could leave the caller's distance and label buffers untouched once `nq * d` selected the BLAS path. The BLAS helpers return early for `ny == 0` before the result handler's `begin_multiple` / `end_multiple` lifecycle initializes and finalizes the output. This change handles an empty database before choosing the sequential or BLAS implementation. It uses the existing result-handler lifecycle so each handler supplies its own correct neutral distance and `-1` label semantics. Fixes https://github.com/facebookresearch/faiss/issues/3830. ## Validation - `python -m unittest test_index.TestIndexFlat_ARM_NEON.test_empty_index_with_blas` - `python -m unittest test_index` (49 tests passed) - `faiss_test --gtest_brief=1` (290 passed, 1 platform-specific skip) - `clang-format 21 --dry-run --Werror faiss/utils/distances.cpp` ## Verification of the regression test We confirmed `test_empty_index_with_blas` is a true regression test by running it with and without the `utils/distances.cpp` change: | State | Result | |-------|--------| | Fix applied | 1 passed, 0 failed | | `utils/distances.cpp` reverted to parent, test retained | 6 failed | The 6 failures are every subtest (2 metrics x 3 values of `k`), covering the `Top1`, `Heap`, and `Reservoir` result handlers. The failure output reports `ACTUAL: array([[42., 42., ...` against `DESIRED: array(-3.402823e+38)`. The test pre-fills `D` with `42` and `I` with `99`, so this confirms the output buffers are never written without the fix, rather than written with incorrect values. The passing run is clean under ASan and UBSan. ## Note for issue #3830 The repro in the issue (`d=1`, `nq=20`) no longer reproduces on `main`. When the issue was filed, `distance_compute_blas_threshold` defaulted to `20` and the branch condition was `nx < threshold`, which put `nq=19` on the sequential path (correct sentinels) and `nq=20` on the BLAS path (uninitialized output). The threshold is now dimension-aware (`nx * d < distance_compute_blas_threshold`, default `128000`), so `d=1` requires `nq >= 128000` to select BLAS. The defect was relocated, not fixed. A repro that still holds today is: ``` faiss.IndexFlatL2(128).search(np.random.random((1000, 128)).astype('float32'), 1) # nx * d == 128000 ``` This change removes the defect at its source, for all `nx`, all `d`, both metrics, and all three result handlers. The `k > ntotal` case with `ntotal > 0` was never affected: `begin_multiple` already initializes all `k` slots, so the unfilled tail is correctly padded. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5528 Reviewed By: mnorris11, trang-nm-nguyen Differential Revision: D116650117 Pulled By: alibeklfc fbshipit-source-id: b40bdf7ade7640b2e433765b3877e2cea3546cf2 |
||
|
Michael Norris
|
e135a1a667 |
Fix OSX arm64 nightly: bump libopenblas from 0.3.33 to 0.3.34 (#5538)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5538 The `OSX arm64 nightlies` job has been failing to solve the `libfaiss` environment since D115218647: ``` ExplainedDependencyNeedsBuildingError: Unsatisfiable dependencies for platform osx-arm64: {MatchSpec("openblas==0.3.34=openmp_hea878ba_0")} - package libopenblas-0.3.33-pthreads_hddb8425_0 has constraint openblas >=0.3.33,<0.3.34.0a0 conflicting with openblas-0.3.34-openmp_hea878ba_0 ``` `conda/faiss/meta.yaml` pins both outputs of the OpenBLAS feedstock, and on osx-arm64 both selectors apply at once: `openblas =0.3.34 # [not x86_64]` and `libopenblas =0.3.33 # [osx]`. Since `libopenblas 0.3.33` constrains `openblas >=0.3.33,<0.3.34.0a0`, the two specs are mutually exclusive and the recipe can never solve. D115218647 bumped `openblas` to 0.3.34 to get `Linux arm64 (conda)` green and left the osx `libopenblas` pins at 0.3.33 on the grounds that that package was resolving fine. It was — in isolation. But the two pins are version-coupled, so the bump broke Mac. The pins also land on different platforms, which is why only OSX fails: `# [not x86_64]` gives linux-arm64 the `openblas` pin on its own. Bump the six osx `libopenblas` pins to 0.3.34 so both feedstock outputs agree. This matches how the same breakage was fixed the last time `openblas` moved: D98293081 bumped `openblas` 0.3.30 to 0.3.32 and broke this job, D98718117 fixed it by bumping `libopenblas` to match, and D104902441 moved both together when going 0.3.32 to 0.3.33 and stayed green. Reverting `openblas` to 0.3.33 is not an option here — it would re-break `linux-arm64-nightly`. The GPU recipes need no change: they carry no osx `libopenblas` pins, since GPU builds are Linux-only. Reviewed By: junjieqi Differential Revision: D117300512 fbshipit-source-id: fe55ef37f4a3054689eacb2f7f64a21ff1d1e8b2 |
||
|
lyang24
|
7059eaf7da |
Add AVX512 bitplane kernel for multi-bit RaBitQ inner product (#5522)
Summary:
`compute_inner_product<AVX512>` only had a dedicated kernel for `ex_bits == 1`; every wider configuration fell through to `ip_bitplane_avx2`, so AVX512 builds ran 256-bit code for the RaBitQ widths used in practice (`nb_bits` 4-5). This adds `ip_bitplane_avx512`: a bitplane is already a bitmask, so it goes straight into a mask register and is consumed by a single masked add instead of the AVX2 broadcast/and/cmpgt/and/fmadd sequence, which also takes the GPR->vector broadcast off the dependency chain; and it processes 16 dimensions per iteration instead of 8. `_pext_u64` still runs once per 8 dims per plane, so only the vector side gets cheaper, and the memory access pattern is unchanged. The now-unreachable copy of `ip_bitplane_avx2` in this translation unit is removed; the AVX2 translation unit keeps its own. Top-10 labels are unchanged across `nb_bits` in {2,3,4,5,7,8} and both metrics, with distance deviation at the level expected from reassociating the accumulation from 8 to 16 lanes; `nb_bits == 2` stays bit-identical since that path is untouched.
== Benchmark ==
We measured `compute_inner_product<AVX512>` directly, A/B against this diff's base revision. Same host, same binary target, `mode/opt`, dynamic dispatch enabled, 5 repetitions, medians reported. Speedup is base time divided by diff time.
| d | `ex_bits` 2 | `ex_bits` 3 | `ex_bits` 4 | `ex_bits` 7 |
| --- | --- | --- | --- | --- |
| 128 | 1.19x | 1.18x | 1.10x | 1.04x |
| 136 | 1.15x | 1.14x | 1.07x | 1.02x |
| 768 | 1.33x | 1.23x | 1.12x | 1.00x |
| 1024 | 1.33x | 1.26x | 1.13x | 1.00x |
1. The production widths (`nb_bits` 4-5, i.e. `ex_bits` 3-4) gain **1.07x to 1.26x**. Best case overall is **1.33x** at `ex_bits` 2 and large `d`.
2. The gain decays to nothing by `ex_bits` 7 (1.00x-1.04x). `_pext_u64` work per plane is unchanged by this diff, so at wide `ex_bits` the GPR side dominates and the vector saving is no longer measurable.
3. Speedup grows with `d`, because the per-call `v_weights` and `pext_masks` setup is a fixed cost amortized over more dimensions.
4. `d = 136` runs 3-4% below `d = 128` at every width. That is the half-width cleanup step paying full 512-bit execution cost for 8 lanes of useful work.
Validity: `compute_inner_product<NONE>` was benchmarked in the same binaries as a control. Its code is unchanged by this diff and reads 1.00x in 15 of 16 cells, putting the noise floor at 0.1%-1.9%. The single exception (`d = 768`, `ex_bits` 2, 0.84x) made the diff side look slower, so it does not inflate any reported gain. AVX512 runs 3x-4x faster than scalar in both builds, confirming the dispatched path is the one being measured. These are kernel-level numbers; end-to-end index search gains will be smaller.
The harness was a temporary microbenchmark calling the kernel directly and is not included in this diff.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5522
Reviewed By: mnorris11
Differential Revision: D116356583
Pulled By: alibeklfc
fbshipit-source-id: 388bae1e564e70f7136bfd595ff276e05ca9e25f
|
||
|
Michael Norris
|
e2cf9e1ebf |
Accept ragged code sizes in faiss::hammings(), and fix the vectorization axis for wide codes (#5520)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5520 Two changes to `faiss::hammings()`: it accepts ragged code sizes, and the kernel for wide codes is told which axis to vectorize. Throughout, **ncodes** is number of bytes per vector (so, 32 for a 256-bit vector), a **word** is one 64-bit (8-byte) unit. A code size is **word-multiple** when `ncodes % 8 == 0` and **ragged** otherwise, so a code is `nwords = ncodes / 8` whole words followed by `rem = ncodes % 8` trailing bytes. ## 1. "Ragged" code sizes Ragged here doesn't mean variable vector lengths, it means non-divisible by 8 ncodes. `hammings()` throws unless the code size is word-multiple, even though the other Hamming entry points (`hammings_knn_hc`, `hamming_range_search`) have always accepted ragged sizes. Callers who need them have to reach past `hammings()` into `with_simd_level_a0_spr` and `with_HammingComputer` and hand-roll the sweep; Laser does exactly that in `KnnFaissBinaryIndex::measureAllCentroids` (D115840549) and can drop it after this. Ragged sizes go to `hammings_ragged_dispatch()`, behind its own `hammings_ragged_fixSL<SL>` entry point so the word-multiple kernels keep the code they generate today. ## 2. Vectorization axis for wide codes All-pairs Hamming is a doubly nested loop: over candidates, and over the words of one code. Only one of them gets vectorized, and clang picks the candidate loop. That means each vector lane wants word `k` of a different candidate, and since candidates are `ncodes` bytes apart the load becomes a strided gather: ``` across candidates (today) within one code (this diff) vpgatherqq (%r8,%zmm19,1),%zmm20 vpxorq (%rbx,%r12,1),%zmm4,%zmm5 ``` Vectorizing the other loop instead reads one whole code contiguously, which is a plain vector load. `#pragma clang loop vectorize(disable)` on the candidate loop is what expresses that: it does not disable vectorization, it takes the candidate loop out of the running so the word loop is what gets vectorized. The arithmetic is identical either way. It is applied only when `nwords >= 4`, i.e. once a code is at least a full 256-bit register wide. That is the widest threshold that costs nothing anywhere: | threshold | ncodes 8 | ncodes 16 | ncodes 32 | ncodes 64 | | --- | --- | --- | --- | --- | | not applied | +0.0 / -0.1 | +0.0 / -0.6 | +0.1 / -0.2 | -0.1 / +0.2 | | `nwords >= 1` | **+45.3 / +63.5** | -10.4 / **+12.2** | -12.6 / -22.7 | -40.7 / -63.1 | | `nwords >= 2` | +0.1 / +0.0 | -10.5 / **+11.7** | -12.6 / -23.7 | -40.7 / -63.1 | | `nwords >= 4` | +0.2 / +0.1 | -0.1 / +0.1 | -12.7 / -23.7 | -40.7 / -63.1 | (AVX2 / AVX512, percent against the parent commit.) At one word the candidate loop is already contiguous, so taking it out of the running costs 45-63%. At two words it helps AVX2 at ncodes 16 and hurts AVX512 by the same amount, which would make the behaviour SIMD-level dependent. Four words is where both levels agree. The pragma is clang-only and guarded with `#if defined(__clang__)`, since MSVC warns C4068 on an unknown pragma. Other compilers therefore never see it and keep today's behaviour: correct, just without the gain. ## 3. Why the ragged kernel is templated A ragged code's loads must be `memcpy` because it is not 8-byte aligned. Both `nwords` and `rem` are template parameters, which is worth an order of magnitude: | ncodes | `<NW, REM>` | `<NW>`, rem runtime | neither templated | | --- | --- | --- | --- | | 2 | **0.487** | 8.720 | 8.724 | | 4 | **0.433** | 9.419 | 9.164 | | 10 | **0.868** | 9.344 | 10.160 | | 12 | **0.692** | 10.387 | 11.472 | | 20 | **1.066** | 9.879 | 10.730 | | 33 | **2.646** | 10.757 | 10.769 | Fig 1: ragged table comparing templates NW and REM (AVX2, ns per pair.) `rem` is the critical part: at a constant size `memcpy` folds into a single instruction, whereas at a runtime size it becomes a real `memcpy` call per candidate, costing 10-20x. That is why the dispatch switches on `ncodes % 8` rather than passing the remainder as an argument. `NW` is worth a further ~2x, but only for 1 to 3 whole words, where a compile-time trip count lets the word loop unroll to straight-line popcounts: | ncodes | NW templated | NW runtime | | --- | --- | --- | | 2, 4 (0 words) | 0.489 / 0.433 | 0.489 / 0.413 | | 10 (1 word) | **0.873** | 1.730 | | 12 (1 word) | **0.696** | 1.725 | | 20 (2 words) | **1.065** | 2.086 | | 33 (4 words) | 2.644 | 2.589 | Fig 2: Ragged table comparing just NW templating vs not having it templated So `NW` is templated for 0-3 words and left runtime from 4 up, which is where it stops paying. ## Results AMD Genoa (EPYC, Zen 4), `mode/opt`, interleaved A/B against the parent commit, one code size per process, 250 samples per cell, min ns per (query, candidate) pair. Word-multiple sizes: | ncodes | AVX2 | AVX512 | | --- | --- | --- | | 32 | **-12.6%** (1.440 -> 1.258) | **-23.3%** (1.352 -> 1.037) | | 64 | **-40.6%** (3.421 -> 2.031) | **-63.1%** (5.680 -> 2.094) | | 40 | +0.0% | **-4.0%** (2.954 -> 2.837) | | 48 | +0.1% | **-4.7%** (3.530 -> 3.365) | | 56 | +0.0% | **+4.0%** (3.944 -> 4.101) | | 8, 16, 24, 128, 160 | within +/-0.3% | within +/-0.8% | Fig 3: result table comparing against baseline (aka before this diff) ncodes 32 and 64 are the sizes whose codes are exactly one vector register wide, which is where the gather cost the most. Absolute cost of every code size with this diff, so the ragged ones can be read against the word-multiple ones: | ncodes | kind | AVX2 ns/pair | AVX2 ns/byte | AVX512 ns/pair | AVX512 ns/byte | | --- | --- | --- | --- | --- | --- | | 2 | ragged | 0.487 | 0.243 | 0.476 | 0.238 | | 4 | ragged | 0.434 | 0.108 | 0.441 | 0.110 | | 8 | word-multiple | 0.240 | 0.030 | 0.215 | 0.027 | | 10 | ragged | 0.879 | 0.088 | 0.789 | 0.079 | | 12 | ragged | 0.693 | 0.058 | 0.693 | 0.058 | | 16 | word-multiple | 0.713 | 0.045 | 0.573 | 0.036 | | 20 | ragged | 1.065 | 0.053 | 1.127 | 0.056 | | 24 | word-multiple | 2.082 | 0.087 | 2.081 | 0.087 | | 32 | word-multiple | 1.257 | 0.039 | 1.036 | 0.032 | | 33 | ragged | 2.644 | 0.080 | 3.160 | 0.096 | | 40 | word-multiple | 2.650 | 0.066 | 2.837 | 0.071 | | 48 | word-multiple | 3.110 | 0.065 | 3.361 | 0.070 | | 56 | word-multiple | 3.452 | 0.062 | 4.096 | 0.073 | | 64 | word-multiple | 2.039 | 0.032 | 2.102 | 0.033 | | 128 | word-multiple | 6.033 | 0.047 | 5.348 | 0.042 | | 160 | word-multiple | 8.164 | 0.051 | 5.731 | 0.036 | Fig 4: result table showing the ns/pair and ns/byte performance. ragged has no "before" because it is newly supported with this diff. Read down the ns/byte columns and the sizes with their own kernel (8, 16, 32, 64) sit at 0.027-0.045, the new ragged kernel at 0.053-0.096 once past a single word, and the shared runtime-word-count kernel (24, 40, 48, 56, 128, 160) at 0.036-0.087. That last group is the weakest: ncodes 24 costs 2.082 ns/pair against 1.257 for ncodes 32, and ncodes 56 costs 3.452 against 2.039 for ncodes 64, both more expensive than a larger code. It vectorizes across candidates for the same reason described above, but its word count is not known at compile time so the same fix does not apply. Left alone here. The two ragged sizes below one word (2 and 4) are dominated by fixed per-call work rather than the compare itself, which is why their ns/byte is high. Reviewed By: alibeklfc Differential Revision: D115982541 fbshipit-source-id: 081c600973fbdf30852527fb6beaae4e29cadfa3 |
||
|
Michael Norris
|
2027d3b331 |
Make cuVS dispatch opt-in rather than build-implied (#5529)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5529 `GpuIndexConfig::use_cuvs`, `GpuClonerOptions::use_cuvs` and `GpuDistanceParams::use_cuvs` each defaulted to `true` whenever the library was compiled with `USE_NVIDIA_CUVS`, and to `false` otherwise. That makes the choice of backend a property of how the binary was built rather than a choice the caller made. Code that constructs a default `GpuIndexFlatConfig()` and never mentions cuVS starts dispatching to a different implementation, with its own numerical behaviour, the moment it is linked against a cuVS-enabled build -- and there is no process-wide way to opt back out, since the flag lives per-object with no environment variable or global setter behind it. Default all three to `false`, so an unset flag means "no". The flag is otherwise unchanged: callers that want cuVS set it explicitly, which is what every cuVS test already does. Builds without cuVS are unaffected, since the flag was already `false` there. `GpuIndexCagra` never consults `use_cuvs` -- it is unconditionally cuVS-backed -- so CAGRA is unaffected. One knock-on worth flagging: `test_gpu_index_ivfsq.py` and `test_index_cpu_to_gpu.py` were picking up cuVS from the default rather than setting it, so when built against a cuVS-enabled library they now exercise the classical path. Neither asserts which backend ran, so both still pass. Threading an explicit `use_cuvs=True` through them, so those variants keep earning their name, is a follow-up. Reviewed By: juancarpio27 Differential Revision: D116567346 fbshipit-source-id: 6cc3419a5563785f81626582885bec2a1ab0770d |
||
|
Bhavik Sheth
|
920a631851 |
Don't raise HNSW entry_point to a point that isn't linked yet (#5524)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5524 `hnsw_add_vertices_deterministic` promoted `entry_point` to `order[i0]` before that point had any links. Within a single add() the promoted point gets linked moments later by the same bucket's batches, so nothing is lost. Across add() calls it orphans the whole graph built by earlier calls: the new entry point has no path back to any of it, and every subsequent search starts there. Defer the raise by one prefix-doubling batch. The first batch is exactly {order[i0]} (grow = done == 0 ? 1 : done), so once it returns the point is linked into the existing graph and is a valid entry point. Later batches in the same bucket then see the raised max_level, which preserves upper-level connectivity -- deferring to the end of the bucket instead would lose that. Bootstrap (entry_point == -1) is unchanged: with an empty graph there is nothing to orphan and nothing to link to. entry_point and max_level end up at the same values as before; only the moment of promotion moves. Indexes built with a single add() are bit-identical. Determinism is preserved. Found via OVIS Search, which builds 4.15B-vector HNSW32 shards 1M vectors per add(). Recall against exhaustive ground truth sat at 68% and would not respond to efConstruction 40/256/512 or efSearch 120..1600. Modelling it as "the first P vectors of each shard are unreachable" gives recall = (N-P)/N and fits five independent production indexes at P = 62.2M +/- 1.3M, across 2x in N and both fp32 and SQ8. Why this survived so long: it is only reachable through repeated add(). Both existing deterministic-build tests, test_graph_based.py and bench_hnsw_deterministic.py, call add() exactly once. Reviewed By: mnorris11 Differential Revision: D116192917 fbshipit-source-id: 99503f11d322a26fb17b87ebff74808f4cbd9a5c |
||
|
Michael Norris
|
a39e260688 |
Fold multi-GPU CAGRA build into train(), delete trainMultiGpu (#5521)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5521 Previous diff: D114685755. Now aarch64 passes. `GpuIndexCagra` had two multi-GPU build entry points totalling 16 positional arguments, neither of which fits the `Index` API. This removes one, folds the other into `train()`, and speeds up the build. **Usage.** Listing more than one device in the config selects the multi-GPU build; `train()` routes to it: ```python devices = faiss.Int32Vector() for i in range(8): devices.push_back(i) an = faiss.AllNeighborsCagraConfig() an.n_clusters = 16 # 0 = auto: max(2 * n_devices, 4) an.overlap_factor = 2 # must be >= 2 an.ivf_pq_search_batch_size = 8192 # 0 = cuVS default; caps IVF-PQ memory config = faiss.GpuIndexCagraConfig() config.graph_degree = 32 config.intermediate_graph_degree = 32 config.build_algo = faiss.graph_build_algo_IVF_PQ config.devices = devices # >1 device selects the multi-GPU build config.all_neighbors_params = an index = faiss.GpuIndexCagra(res, d, faiss.METRIC_L2, config) index.train(xb) # xb must stay alive until copyTo() completes cpu_index = faiss.IndexHNSWCagra() cpu_index.base_level_only = True index.copyTo(cpu_index) # required: the GPU index is not searchable on this path ``` Float32-only, does not copy `x`, and leaves `index_` empty so `copyTo()` is the only valid follow-up. Single-GPU behaviour is unchanged when `devices` is empty. **Deleted `trainMultiGpu`.** Sharded SNMG CAGRA produces per-shard graphs with zero cross-shard edges by construction, so it needs post-hoc stitching just to be usable, and it lost to the `all_neighbors` path on both build time and recall. It had no callers outside the benchmark and one test. **Folded `trainAllNeighbors` into `train()`.** Its 6 trailing bare scalars now live in the constructor-time config struct, which is the established Faiss GPU convention: | Old argument | Now | | --- | --- | | `devices` | `GpuIndexCagraConfig::devices`, also the dispatch predicate | | `build_algo` (0/1/2) | `GpuIndexCagraConfig::build_algo` | | `refinement_rate` | `AllNeighborsCagraConfig::refinement_rate` | | `n_clusters`, `overlap_factor`, `ivfpq_search_batch` | new `AllNeighborsCagraConfig` | This also kills a live footgun: the old `int build_algo` used an encoding (0=NN-descent, 1=brute-force, 2=IVF-PQ) that disagreed with the `graph_build_algo` enum in the same header (0=IVF_PQ, 1=NN_DESCENT). Both callers set `config.build_algo` and then passed an unrelated int, and the config field was silently dead. `BRUTE_FORCE` is appended to `graph_build_algo` (at the end, so existing values do not renumber) and the config field is now the single source of truth. **Graph pruning no longer copies the graph off the device.** Previously only detour counting ran on the GPU and pruning plus reverse-graph construction ran on the host, which was 35% of total build time. This part is **temporary and self-removing**. cuVS has no public API to prune a graph that is already on the device: its exported `helpers::optimize` takes host matrices, and the dispatch behind it erases the mdspan accessor to host memory, so cuVS's own device code path is unreachable from outside. Until that is fixed upstream, this reaches into cuVS's internal headers when the build has them available, and otherwise falls back to the public host API -- correct either way, just slower on the fallback. **Open-source builds get the fallback**, since cuVS does not install those headers; the `train()` docs say so. An upstream patch adding a `device_matrix_view` overload to `cuvs::neighbors::cagra::helpers::optimize` is prepared and will be submitted to rapidsai/cuvs. When it ships, the internal include, the build flag guarding it, and the fallback branch all get deleted and every build gets the fast path. At 50M vectors on 8 GPUs the optimize step goes from 119.1s to 1.65s (72x) and end-to-end build->serialize from 8.0 to 5.4 minutes, recall unchanged. **Collapsed the benchmark to one path.** With the stitching approaches gone, `bench_approaches.py` is a single-path tool for validating and tuning the production build, and reports a per-phase build breakdown plus an efSearch sweep of recall and QPS. Deliberately *not* merged into the config: - `ivf_pq_params` / `ivf_pq_search_params` are still not consulted on this path. cuVS derives them from the dataset shape and those derived values beat the static defaults in the faiss structs, so only the knobs cuVS cannot infer are overridden. - `faiss::cagra_build_algo` only has `{IVF_PQ, NN_DESCENT}`, so a `BRUTE_FORCE` config would silently degrade to NN-descent on the single-GPU path; `train_ex()` now rejects it there. `GpuIndexBinaryCagra` shares this config struct and has no multi-GPU build, so it rejects `devices.size() > 1` rather than silently building on one device. Behaviour changes: the default `build_algo` on the multi-GPU path is now `IVF_PQ` (the config default) rather than NN-descent (the old argument default), and `IndexHNSW::num_base_level_search_entrypoints` goes 32 -> 256, which measured better on both recall and QPS at every efSearch. Reviewed By: junjieqi Differential Revision: D116187873 fbshipit-source-id: 265be7dc0670de262cb1a937ca4b2f530578e237 |
||
|
Trevor Mathisen
|
1f93154314 |
Revert D114685755: Fold multi-GPU CAGRA build into train(), delete trainMultiGpu
Differential Revision: D114685755 Original commit changeset: 4eb72793284d Original Phabricator Diff: D114685755 fbshipit-source-id: e0d325ef77a323885736a7eb04deb21301d36fc4 |
||
|
Michael Norris
|
6644dcfa55 |
Fold multi-GPU CAGRA build into train(), delete trainMultiGpu (#5500)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5500 `GpuIndexCagra` had two multi-GPU build entry points totalling 16 positional arguments, neither of which fits the `Index` API. This removes one, folds the other into `train()`, and speeds up the build. **Usage.** Listing more than one device in the config selects the multi-GPU build; `train()` routes to it: ```python devices = faiss.Int32Vector() for i in range(8): devices.push_back(i) an = faiss.AllNeighborsCagraConfig() an.n_clusters = 16 # 0 = auto: max(2 * n_devices, 4) an.overlap_factor = 2 # must be >= 2 an.ivf_pq_search_batch_size = 8192 # 0 = cuVS default; caps IVF-PQ memory config = faiss.GpuIndexCagraConfig() config.graph_degree = 32 config.intermediate_graph_degree = 32 config.build_algo = faiss.graph_build_algo_IVF_PQ config.devices = devices # >1 device selects the multi-GPU build config.all_neighbors_params = an index = faiss.GpuIndexCagra(res, d, faiss.METRIC_L2, config) index.train(xb) # xb must stay alive until copyTo() completes cpu_index = faiss.IndexHNSWCagra() cpu_index.base_level_only = True index.copyTo(cpu_index) # required: the GPU index is not searchable on this path ``` Float32-only, does not copy `x`, and leaves `index_` empty so `copyTo()` is the only valid follow-up. Single-GPU behaviour is unchanged when `devices` is empty. **Deleted `trainMultiGpu`.** Sharded SNMG CAGRA produces per-shard graphs with zero cross-shard edges by construction, so it needs post-hoc stitching just to be usable, and it lost to the `all_neighbors` path on both build time and recall. It had no callers outside the benchmark and one test. **Folded `trainAllNeighbors` into `train()`.** Its 6 trailing bare scalars now live in the constructor-time config struct, which is the established Faiss GPU convention: | Old argument | Now | | --- | --- | | `devices` | `GpuIndexCagraConfig::devices`, also the dispatch predicate | | `build_algo` (0/1/2) | `GpuIndexCagraConfig::build_algo` | | `refinement_rate` | `AllNeighborsCagraConfig::refinement_rate` | | `n_clusters`, `overlap_factor`, `ivfpq_search_batch` | new `AllNeighborsCagraConfig` | This also kills a live footgun: the old `int build_algo` used an encoding (0=NN-descent, 1=brute-force, 2=IVF-PQ) that disagreed with the `graph_build_algo` enum in the same header (0=IVF_PQ, 1=NN_DESCENT). Both callers set `config.build_algo` and then passed an unrelated int, and the config field was silently dead. `BRUTE_FORCE` is appended to `graph_build_algo` (at the end, so existing values do not renumber) and the config field is now the single source of truth. **Graph pruning no longer copies the graph off the device.** Previously only detour counting ran on the GPU and pruning plus reverse-graph construction ran on the host, which was 35% of total build time. This part is **temporary and self-removing**. cuVS has no public API to prune a graph that is already on the device: its exported `helpers::optimize` takes host matrices, and the dispatch behind it erases the mdspan accessor to host memory, so cuVS's own device code path is unreachable from outside. Until that is fixed upstream, this reaches into cuVS's internal headers when the build has them available, and otherwise falls back to the public host API -- correct either way, just slower on the fallback. **Open-source builds get the fallback**, since cuVS does not install those headers; the `train()` docs say so. An upstream patch adding a `device_matrix_view` overload to `cuvs::neighbors::cagra::helpers::optimize` is prepared and will be submitted to rapidsai/cuvs. When it ships, the internal include, the build flag guarding it, and the fallback branch all get deleted and every build gets the fast path. At 50M vectors on 8 GPUs the optimize step goes from 119.1s to 1.65s (72x) and end-to-end build->serialize from 8.0 to 5.4 minutes, recall unchanged. **Collapsed the benchmark to one path.** With the stitching approaches gone, `bench_approaches.py` is a single-path tool for validating and tuning the production build, and reports a per-phase build breakdown plus an efSearch sweep of recall and QPS. Deliberately *not* merged into the config: - `ivf_pq_params` / `ivf_pq_search_params` are still not consulted on this path. cuVS derives them from the dataset shape and those derived values beat the static defaults in the faiss structs, so only the knobs cuVS cannot infer are overridden. - `faiss::cagra_build_algo` only has `{IVF_PQ, NN_DESCENT}`, so a `BRUTE_FORCE` config would silently degrade to NN-descent on the single-GPU path; `train_ex()` now rejects it there. `GpuIndexBinaryCagra` shares this config struct and has no multi-GPU build, so it rejects `devices.size() > 1` rather than silently building on one device. Behaviour changes: the default `build_algo` on the multi-GPU path is now `IVF_PQ` (the config default) rather than NN-descent (the old argument default), and `IndexHNSW::num_base_level_search_entrypoints` goes 32 -> 256, which measured better on both recall and QPS at every efSearch. Reviewed By: alibeklfc Differential Revision: D114685755 fbshipit-source-id: 4eb72793284d2e9f19c77e20fdcc6c28be4bf3bb |
||
|
Michael Norris
|
80a16564f8 |
Enable Faiss dynamic dispatch for OVIS and VeST fbpkgs (#5515)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5515 Enable `faiss.dynamic_dispatch=true` on the three `vector_search` fbpkgs that link Faiss, and add a probe that logs which SIMD kernels each process actually dispatched to. Also adds some logs in VeST. **fbpkgs covered** - `ovis_cli` (`cli/BUCK`) - `vector_search_offline_knn` (`offline_knn/BUCK`) - `vector_search_index_trainer` (`training/BUCK`) — the VeST trainer Without the flag, Faiss is compiled with static AVX2 SIMD selection. With it, all compiled-in SIMD variants (AVX2, AVX512, AVX512-SPR) are built and the best supported one is selected at runtime via CPUID. **This is not a speedup, and is not intended as one.** These workloads *build* indexes rather than search them, and the AVX-512 win for SQ4 lives on the scan path, not the encode path. Measured end-to-end as perf-neutral — see the test plan. The reasons to land it are consistency across the `vector_search` fbpkgs, unlocking AVX-512 for any future search-side work in these binaries, and the observability below. **SIMD probe.** `log_faiss_simd_level()` in `training/utils.py` logs `level`, `dispatched_level`, `has_dynamic_dispatch` and `compile_options`. It is called from `assignment_worker.py` (datasink worker) and from `datasinks/common.py::_build_merged_file_blocking` (the spawned leaf-build children are separate processes and do not inherit parent Faiss state, so each reports its own; guarded to log once per process rather than once per cluster). This probe is load-bearing: Faiss logs nothing about its own dispatch state, so the earlier VeST DD A/B could not distinguish "DD did not help" from "DD never engaged". With the probe, that ambiguity is resolved — DD demonstrably engages, and the null result is real. Reviewed By: bshethmeta Differential Revision: D107286701 fbshipit-source-id: 48cada6220c2b379f9774514edd48f2f5858f39a |
||
|
Michael Norris
|
e2f9ccaccc |
Fix SQtqmse codebook scale: fold 1/sqrt(d) into Lloyd-Max table (#5517)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5517 `SQtqmse` reconstructions have inflated norms and badly degraded recall since D102408184. Reported as a 1.14.2 -> 1.14.3 regression: at `d=64`, median decoded norm of a unit-norm input went `0.9965` -> `1.1503` (+15%), while `SQtqmse8` looked "essentially unchanged". Fixes https://github.com/facebookresearch/faiss/issues/5317 Root cause - D102408184 replaced the `tqmse` codebook construction with hardcoded Lloyd-Max tables: ``` - scalar_quantizer::train_TurboQuantMSE(d, 4, trained); + populate_lloyd_max_trained(4, trained); ``` The old codebook was built for one coordinate of a unit-norm vector in R^d, so its centroids scaled as `1/sqrt(d)`. The new tables are optimal for `N(0, 1)` and do not depend on `d` at all. That is correct for `_eden` and `_tq`, which rescale each vector to unit variance before lookup. Plain `tqmse` does not: it encodes raw unit-norm vectors, whose components are ~`1/sqrt(d)` (~`0.125` at `d=64`), against a codebook ~50x too wide. Nearly everything falls in the innermost cells, so the codebook collapses to a few levels and every component decodes too large: | d | bits | distinct codes used (before D102408184 -> after) | | --- | --- | --- | | 64 | 4 | 16/16 -> **4/16** | | 768 | 4 | 16/16 -> **2/16** | | 768 | 8 | 256/256 -> **32/256** | Gets worse as `d` grows. Fix - Give `populate_lloyd_max_trained` a `scale` argument that multiplies the table. The five `tqmse` cases pass `1/sqrt(d)` (the standard deviation of a unit-norm vector's components) so the codebook once again (like 1.14.2) matches the data it encodes. **No change for `_eden` / `_tq`.** They pass no `scale`, so it defaults to `1` and every table entry is bit-identical to today. Scaling `trained` rather than the encode path is what keeps this a one-line fix per callsite: every SIMD specialization and distance computer reads `this->centroids`, a pointer into `trained`. Notes - - NOT A REVERT TO 1.14.2, but that is fine. 1.14.2 trained on the exact unit-sphere marginal; this scales a Gaussian approximation of it, so centroids differ by up to ~20% at 8 bits. The approximation is not worse: max component error is lower than 1.14.2 and recall is restored (see test plan). The only consequence is that `tqmse` codes are not comparable across 1.14.2 / 1.14.3 / this version -- harmless, because the codebook ships with the index (below). - SERIALIZATION STILL COMPATIBLE: Existing 1.14.3-written `tqmse` indexes stay readable and self-consistent: `read_ScalarQuantizer` loads `trained` verbatim, so decode matches how they were encoded, and `trained` length is unchanged so size validation still passes. They are degraded, not corrupt, and need re-indexing to benefit. No version guard added. Reviewed By: alibeklfc Differential Revision: D115440646 fbshipit-source-id: a559f6a6a50a65dc3ab32ff3f17cda8796b2d4ae |
||
|
Michael Norris
|
a424dcb809 |
faiss HNSW: add opt-in deterministic lock-free graph build (faiss::hnsw_deterministic_build) (#5486)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5486 TLDR: adds deterministic HNSW build inspired by ParlayANN. The deterministic build is reproducible AND faster than the lock-based build at every scale and thread count we measured. This is gated behind `faiss::hnsw_deterministic_build`. We plan to test internally, then make this the default flow and remove the existing flow. -- https://arxiv.org/abs/2305.04359 -- HOW TO ENABLE -- ``` C++: faiss::hnsw_deterministic_build = true; Python: faiss.cvar.hnsw_deterministic_build = True ``` Nothing else is needed, and nothing changes until you do it: the flag defaults to false, so this diff is a no-op on its own. An embedder driving this from its own configuration system should resolve the value once at startup and assign the global, rather than per index: changing it between two `add()` calls would leave one index built by each algorithm. `python/__init__.pyi` now declares `faiss.cvar`. It was missing, so any Python consumer assigning a faiss global failed Pyre with "No attribute `cvar` in module `faiss`" -- fixed here rather than with a `pyre-ignore` in each consumer. WHEN IT IS WORTH ENABLING -- Only for builds that run multi-threaded. At one thread the existing lock-based build is *already* deterministic, so the flag buys nothing beyond ~6% build time. Verified: lock-based at OMP=1 is byte-identical across runs, while at OMP=8 46% of neighbor slots differ. A consumer that pins `omp_set_num_threads(1)` should not bother. similarities to parlayANN: - add vertices in doubling batches against frozen snapshot - defer adding reciprocal edges immediately, add them later after parallel phase differences from ParlayANN: - re-uses Faiss HNSW pruning in `shrink_neighbor_list` --- AI (with a bunch of edits) explanation in more detail: -- What changed - `IndexHNSW::add` and `IndexBinaryHNSW::add` pick the build at runtime. Both paths are compiled in and **the default is unchanged** (lock-based); nothing switches until the flag or the hook says so. - Turning it on: ``` C++: faiss::hnsw_deterministic_build = true; Python: faiss.cvar.hnsw_deterministic_build = True ``` - A single plain `bool` is the whole API: libfaiss takes no dependency on any configuration system, and an embedder that wants one resolves it itself and assigns the global: ``` faiss.cvar.hnsw_deterministic_build = <your config lookup> ``` - Resolve it once at startup, not per index. `add()` reads the flag each call, so changing it between two `add()` calls on one index would build part of the graph with each algorithm -- reproducible in neither. - **This flag is transitional.** Once the deterministic build is validated in production, the follow-up makes it the only path and deletes the lock-based build and the flag. - The deterministic path now supports the CAGRA level-0 import configuration: `init_level0=false` skips the level-0-only bucket (level 0 is supplied by the imported CAGRA graph), and `keep_max_size_level0` fills the base layer to 2*M. So `IndexHNSWCagra` (CPU) and `GpuIndexCagra::copyTo(IndexHNSWCagra*)` build through the deterministic path. - `IndexBinaryHNSW::add` is gated by the same flag and shares the same deterministic implementation: `hnsw_add_vertices_deterministic` takes `make_distance_computer` / `set_query` callbacks, so the binary index just supplies its own Hamming distance computer. It keeps its own lock-based `hnsw_add_vertices` for the default path. Background -- HNSW construction in Faiss was non-deterministic under parallel builds: multiple runs of `IndexHNSW::add` with the same data and seeds could produce different graphs, a problem for persistence, crash recovery, and replication (the ParlayANN motivation, https://arxiv.org/abs/2305.04359). Sources of non-determinism were: (1) the reciprocal-link write race in `add_links_starting_from_impl`; (2) floating-point distance ties resolved in heap/visitation order; (3) the entry-point bootstrap `#pragma omp critical` race. Algorithm (adapted from ParlayANN to Faiss's level-batched structure): - Per level bucket (highest first, deterministic shuffle), points are inserted in prefix-doubling sub-batches (batch sizes 1, 2, 4, ... capped at 2% of the index). - Phase A (`HNSW::compute_forward_links_deterministic`, parallel): each point greedily descends and computes its forward links against the immutable snapshot from the end of the previous sub-batch, writing only its own neighbor slots. Reciprocal-edge requests are collected, not applied, so this phase is race-free. - Phase B (`HNSW::merge_reverse_links_deterministic`, parallel): reverse edges are grouped by destination with a fixed-size 256-bucket radix partition on the low bits of `dest` (a small constant bucket count, independent of `ntotal` and thread count, so grouping stays O(edges) in memory), each bucket sorted by `(level, dest)` and merged in parallel. Every affected node is merged exactly once in a total order (distance, ties by id) and re-pruned with the same RNG heuristic. Because every `dest` maps to exactly one bucket, distinct nodes touch disjoint slots (no locks) and the merge is order- and thread-count-independent. The Phase-B parallel-for uses `schedule(static)` — the libomp dynamic dispatcher segfaults in some build configs (the pre-existing lock-based build carried the same warning). Guarantee: the resulting graph is reproducible across runs at a fixed thread count and, in practice, across thread counts (the merge is fully order-independent). Recall matches the previous default at every efSearch. ## Performance: build time (40M, d=128, M=32, efC=64, 166 threads) 10-round interleaved timing study (one deterministic + one lock-based build per round, so both see identical host conditions): deterministic per-round s: 285.58 275.03 280.84 272.62 273.73 272.61 272.05 272.90 269.87 272.29 lock-based per-round s: 306.23 352.97 322.76 294.23 339.32 303.04 291.46 341.96 359.31 282.92 deterministic: min=269.87 mean=274.75 median=272.76 max=285.58 std=4.53 lock-based: min=282.92 mean=319.42 median=314.50 max=359.31 std=26.12 det/lock: mean=0.860 (deterministic ~14% faster), median=0.867 The deterministic build is ~14% faster than the lock-based build at 40M and ~6x more stable run-to-run (std 4.53s vs 26.12s), since it does not depend on lock-contention timing. Peak RSS ~66GB vs ~56GB. Recall matches at every efSearch (byte-identical graph across builds). ## Performance: search time Back on the deterministic HEAD, tree clean. Here's the matched A/B — same 40M synthetic data, same machine (AMD Genoa, 166 cores), search_repeat=100, deterministic (my HEAD) vs lock-based (parent commit). Since my diff doesn't touch search() at all, any difference is purely graph structure + measurement noise. Search QPS: deterministic vs lock-based (40M synthetic, repeat=100) HNSW16 ``` ┌──────────┬─────────────────┬─────────┬──────────┬───────┐ │ efSearch │ recall det/lock │ QPS det │ QPS lock │ Δ │ ├──────────┼─────────────────┼─────────┼──────────┼───────┤ │ 64 │ 0.828/0.820 │ 170,329 │ 177,995 │ −4.3% │ ├──────────┼─────────────────┼─────────┼──────────┼───────┤ │ 128 │ 0.866/0.862 │ 112,727 │ 110,727 │ +1.8% │ ├──────────┼─────────────────┼─────────┼──────────┼───────┤ │ 256 │ 0.886/0.888 │ 59,815 │ 56,784 │ +5.3% │ └──────────┴─────────────────┴─────────┴──────────┴───────┘ ``` HNSW32 ``` ┌──────────┬─────────────────┬─────────┬──────────┬───────┐ │ efSearch │ recall det/lock │ QPS det │ QPS lock │ Δ │ ├──────────┼─────────────────┼─────────┼──────────┼───────┤ │ 64 │ 0.935/0.930 │ 110,186 │ 108,411 │ +1.6% │ ├──────────┼─────────────────┼─────────┼──────────┼───────┤ │ 128 │ 0.958/0.953 │ 68,019 │ 66,308 │ +2.6% │ ├──────────┼─────────────────┼─────────┼──────────┼───────┤ │ 256 │ 0.965/0.960 │ 37,624 │ 35,828 │ +5.0% │ └──────────┴─────────────────┴─────────┴──────────┴───────┘ ``` HNSW32,SQ8 ``` ┌──────────┬─────────────────┬─────────┬──────────┬────────┐ │ efSearch │ recall det/lock │ QPS det │ QPS lock │ Δ │ ├──────────┼─────────────────┼─────────┼──────────┼────────┤ │ 64 │ 0.926/0.934 │ 220,713 │ 198,325 │ +11.3% │ ├──────────┼─────────────────┼─────────┼──────────┼────────┤ │ 128 │ 0.948/0.953 │ 117,504 │ 129,173 │ −9.0% │ ├──────────┼─────────────────┼─────────┼──────────┼────────┤ │ 256 │ 0.961/0.963 │ 58,582 │ 65,551 │ −10.6% │ └──────────┴─────────────────┴─────────┴──────────┴────────┘ ``` (Low-ef points ef16/32 omitted from the verdict — even at 100 repeats their std is ~8–20%, too noisy; ef128/256 std is ~3–5%.) Verdict: no search-QPS regression - Pure HNSW (16, 32): QPS at parity — within ±5%, and actually slightly faster deterministic at the high-recall points (ef128/256), with equal-or-better recall. - HNSW32,SQ8: more scatter (±10%, mixed direction) — but it tracks small correlated recall differences (det ef256 is 0.961 vs 0.963), i.e. the two different graphs sit at slightly different recall/QPS operating points, not a systematic slowdown. Search code is identical, so this is graph-structure + noise, not a code regression. If you want it pinned down, a recall-matched (interpolated) comparison would remove the operating-point confound. - Bonus: the deterministic build was 2–3× faster in every case (e.g. HNSW32: 277 s vs 527 s; HNSW16: 164 s vs 429 s) — consistent with all prior results. ## Single-threaded (OMP_NUM_THREADS=1) Customers frequently build with OMP=1 or OpenMP disabled, so this case matters. Measured at 1M / d=128 / M=32 / efC=64, single-threaded: build time: lock-based 188.36s vs deterministic 176.40s (0.94x -> deterministic ~6% FASTER) peak RSS: 1.6 GB (both, identical) recall@10 ef 16/32/64/128: lock-based .8830/.9387/.9676/.9853 vs deterministic .8832/.9381/.9625/.9798 No single-threaded regression: the deterministic build is slightly faster (it avoids the per-node OpenMP lock ops), uses the same memory, and matches recall within noise. Note the lock-based build was already deterministic at a single thread, so single-threaded users lose nothing and gain a small speedup. ## Serialization compatibility No on-disk format change, verified in `index_read.cpp` / `index_write.cpp`: - `hnsw_deterministic_build` is a namespace-scope global, not a field on `HNSW` or `IndexHNSW`, so it is not part of any serialized struct. Like `retain_locks`, it is a pure runtime build flag. - `write_HNSW` / `read_HNSW` and the `IndexHNSW` field layout are unchanged. The subtype fourcc tags, header, CAGRA block, graph CSR (entry_point / max_level / levels / offsets / neighbors / efC / efS), and storage are all as before. - `keep_max_size_level0` is still serialized only for the CAGRA subtype (`IHc2`/`IHNc`); `init_level0` is build-only (not serialized). - The deterministic build emits the same HNSW CSR structure (only neighbor content differs), so old indexes read unchanged and new indexes remain readable by older Faiss. - Verified by the `io_and_retest` serialize -> deserialize -> re-search round-trips in `test_graph_based.py` / `test_hnsw.cpp` (all pass). - Measured on a 1M index: written with the flag off, flag flipped on, read back -> `neighbors`, `offsets`, `entry_point`, `max_level` byte-identical and search ids+distances identical. The flag is read only inside `add()`. - Mixed-mode append (lock-built graph extended by a deterministic `add()`) was measured against brute-force ground truth at 220k and is indistinguishable from either pure build -- recall@10 within +-0.02 of both at ef 32/64/128. No existing test covers this, since every test builds one way in one process. ## CAGRA API for HNSW build on multi-GPU (aka D106837134) — MAST verification Verified end-to-end on MAST (8x H100 Grand Teton, Approach D, 100M vectors) with this change in the build — the multi-GPU CAGRA -> HNSW graph-build time is comparable to the D106837134 baseline (no regression): all_neighbors build: 367.4s optimize: 231.7s copyTo: 18.4s serialize: 28.9s (66 GB) INDEX build -> serialize total: 661.7s (11.0 min) [D106837134 baseline: 721s] recall@10 (tiled 100M): ef64 0.7746, ef128 0.8830, ef256 0.9429 - This confirms this CPU-side change builds, links, and runs in the GPU CAGRA binary at scale and does not regress the pipeline. Note the Approach-D run uses copyTo(base_level_only=True), which imports the CAGRA graph directly as HNSW level 0 and skips add(), so it does not itself route through the deterministic add(). - The deterministic CAGRA level-0 import this change adds (the copyTo path with base_level_only=False: init_level0=false skips the level-0 bucket; keep_max_size_level0 fills the base layer) is covered by passing unit tests: `Test_IndexHNSWCagra_BaseLevelOnly_RangeSearch` (C++), `test_hnsw_no_init_level0`, and `test_hnsw_cagra_IP` / `_base_level_only` (Python). ## Behavioral note: level-0 base layer under keep_max_size_level0 (reviewers, please note) One deliberate difference from the lock-based build, in the CAGRA base-layer case only: the old build gated the "fill the level-0 list up to 2*M" behavior on the inserted point's OWN top level (`keep_max_size_level0 && pt_level == 0`), so a level>=1 node's level-0 list could be pruned below 2*M. The deterministic build gates on the LINK level (`keep_max_size_level0 && level == 0`), so EVERY node's level-0 list is filled to 2*M when `keep_max_size_level0` is set (not only the level-0-only points). This is a strict superset of the old coverage -- it fills exactly to the 2*M slot capacity (no overflow) and yields a fuller/denser base layer for CPU `IndexHNSWCagra`, which is what `GpuIndexCagra::copyFrom(IndexHNSWCagra*)` reads back. It is INERT for the default build (`keep_max_size_level0` defaults to false, so the gate is never true) and never affects a non-CAGRA graph. Called out explicitly so reviewers know the CPU `IndexHNSWCagra` base-layer graph is intentionally denser than the pre-diff build; worth a sanity check against GPU `copyFrom` expectations. Reviewed By: pankajsingh88 Differential Revision: D112025877 fbshipit-source-id: dd111a5a3674c08200905bfea1d056f0fa5b3d42 |
||
|
Aaryan Mehta
|
2688c34f94 |
Parallelize the k-means++ D^2 seeding update (#5457)
Summary: The k-means++ centroid initialization (`init_kmeans_plus_plus` in `ClusteringInitialization`) recomputes each point's squared distance to the newly chosen centroid in a serial loop, while the Lloyd iterations that follow (assignment through `index.search`, and the centroid update) are already OpenMP-parallel. This adds a `#pragma omp parallel for` to that per-point min-distance update. Each `min_distances[i]` is an independent write, and the serial cumulative sum and D^2 sampling that pick the next centroid are untouched, so the chosen-centroid sequence and the output stay bit-identical to the serial version. The determinism test in `test_clustering_initialization.py` continues to hold, and a standalone check confirms the seeding output is bit-identical across 1, 2, 4, and 10 threads. Only the k-means++ init path is affected; RANDOM (the default) and AFK-MC2 are unchanged. The loop is memory-bandwidth-bound, so it is not a linear speedup: on a 10-core dual-channel machine the seeding update measures about 2.7x at 4 threads and saturates there. End-to-end the seeding is a small share of a full clustering, so the effect on `Clustering.train` is modest and shrinks as niter grows. The value is removing the last serial loop in an otherwise parallel init, most visible for large n with many centroids. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5457 Reviewed By: mnorris11 Differential Revision: D115373675 Pulled By: alibeklfc fbshipit-source-id: 485236ed4eb4310480ef4f48a20a731597d07e33 |
||
|
Michael Norris
|
151452ec94 |
Fix SIMD level detection in Faiss (#5507)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5507 `SIMDConfig::has_dynamic_dispatch()` was defined inline in `simd_levels.h`, where it resolved `#ifdef FAISS_ENABLE_DD` at include time. That macro is only set while faiss itself is compiled, so any code outside the library that included the header always evaluated it to `false` — reporting that dynamic dispatch was unavailable even when linked against a faiss built with it enabled. Callers using it to decide whether `SIMDConfig::set_level()` will do anything would therefore skip setting the SIMD level, and silently fall back to the auto-detected one. Move the definition into `simd_levels.cpp`, which is compiled as part of the library and does see the macro. Callers now get the answer for the faiss they actually link against. The function is no longer `constexpr` and so cannot be used in constant expressions. It had no such uses. Also bumps the conda `openblas` pin for non-x86_64 platforms from 0.3.33 to 0.3.34, which is required to get the `Linux arm64 (conda)` build green. conda-forge now serves `libopenblas 0.3.34`, which depends on `openblas >=0.3.34,<0.3.35`, so the old pin left the environment unsolvable: ``` ExplainedDependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-aarch64 - package libfaiss-1.15.0-py3.12_h3f4af3b_10_cpu requires openblas 0.3.33.*, but none of the providers can be installed ``` The separate `libopenblas` pins for osx are left at 0.3.33, since that package is resolving fine. Reviewed By: junjieqi Differential Revision: D115218647 fbshipit-source-id: 7163e18011dc9b82e93cd68fb600d24ef7012732 |
||
|
Rohan Rajesh Mali
|
02dbb4e42f |
Fix data race in NNDescent::update() (#5413)
Summary: `NNDescent::update()` Step 3 (the reverse-link phase) has a data race on each node's candidate `pool`. While some threads read `other.pool.back().distance`, the thread that owns that node reorders the same pool via `std::make_heap` at the end of its own iteration, with no shared synchronization. Reported as https://github.com/facebookresearch/faiss/issues/5276. The originally proposed change (moving `LockGuard guard(other.lock)` before the `other.pool.back()` read) does not remove the race: the concurrent writer is the unlocked `std::make_heap`, so locking only the reader establishes no mutual exclusion. The race is also not with `Nhood::insert()`'s heap operations: `join()` and `update()` run in strictly separate phases (`join()` completes before `update()` begins), so `insert()` never runs concurrently with Step 3. We verified this: `Nhood::insert()` has exactly two call sites, both inside the `join()` lambda, and `update()` has exactly one call site, sequentially after `join()` returns. This diff removes the concurrency at its source. `std::make_heap` is moved out of the reverse-link loop into a separate `#pragma omp parallel for` that runs after the loop's implicit barrier. During the reverse-link loop no thread reorders or resizes a pool, so the `other.pool.back()` reads are safe without a lock (the lock continues to guard only the `rnn_*` updates, as in the original code). The only element write in that loop is to `flag`, a distinct memory location from the `distance` being read, so it is not a race either. There is no added lock traffic on the hot read path. **Behavior change: fewer reverse links, faster build, one extra iteration to converge.** This is not a behavior-preserving change, and the effect is larger than the race fix itself. Step 2 sorts each `pool` ascending, so `pool.back()` is the maximum (worst) distance. In the previous code `std::make_heap` had usually already run on the node being read, and in a max-heap built from sorted input the last element is a near-minimum leaf: | `L` (pool size) | rank of `back()` | |---|---| | 30 | 6 of 30 (21st percentile) | | 82 (default, `L = K + 50`) | 1 of 82 (second smallest) | The gate `nn.distance > other.pool.back().distance` therefore compared against roughly the pool minimum before, and against the true maximum now, so materially fewer reverse links are admitted. Reverse links feed `nn_new`/`nn_old` in Step 4 and drive the candidate pairs enumerated by the next `join()`, so each iteration refines the graph less and does less work. Measured on `TestNNDescentKNNG` (`K=10`, `S=10`, `R=32`, `L=30`, `iter=5`), graph recall per iteration: | iteration | before | after | |---|---|---| | 1 | 0.692 | 0.627 | | 2 | 0.973 | 0.958 | | 3 | 0.999 | 0.998 | | 4 | 1.000 | 1.000 | Convergence is approximately one iteration slower and the converged graph is unchanged: both reach 1.000 and both clear the test's `> 0.99` assertion. The build is correspondingly cheaper, at 1608 ms versus 2186 ms on the same workload, because the previous behavior computed many more candidate distances per iteration. The new gate matches the documented intent of the pre-existing comment ("push itself into `other.rnn_new` if it is not in the candidate pool of the other side"), which holds only when `back()` is the worst candidate. The previous threshold was an artifact of the race: which pools were already heapified depended on thread scheduling, so the extra exploration was nondeterministic and undefined behavior besides. There is nothing principled to preserve. Users at the default `iter = 10` are unaffected because both configurations saturate. Exposure is limited to reduced `iter` settings and to datasets needing more than 10 iterations. Credit: wonder-fly for reporting and diagnosing the race in #5276, and RohanMali2003 for PR #5413. The final fix takes a different approach (deferring `make_heap`) than the originally proposed lock reordering. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5413 Test Plan: - `buck test fbcode//mode/dev fbcode//faiss/tests:test_nndescent`: Pass 4. - `buck test fbcode//mode/dev fbcode//faiss/tests:test_graph_based -- --regex NNDescent`: Pass 6. - `arc f` / `arc lint` clean on `faiss/impl/NNDescent.cpp`. A/B verification of the behavior change: we reverted only the `make_heap` placement, rebuilt, and compared per-iteration graph recall from `TestNNDescentKNNG` verbose output. Numbers are in the Summary. No test in the repository guards this race, on either revision. We verified this by building and running both revisions: the graph-quality tests pass identically before and after the fix, because they measure converged recall and both configurations converge. An earlier draft of this diff added a C++ end-to-end test; we removed it because it measured `recall@10 = 1.0000` on both revisions, so it could not fail for the reason it claimed to guard, and it duplicated `TestNNDescentKNNG` at 20x to 100x the runtime. ThreadSanitizer note: fbcode `dev-tsan` serializes OpenMP to a single thread (verified: OMP threads in use = 1 even with `OMP_NUM_THREADS`/`OMP_THREAD_LIMIT` overrides), so this race cannot be reproduced through `update()` under local TSan. It is reproducible under an OSS/cmake TSan build with real OpenMP threads, which is how #5276 was found. The fix rests on the concurrency analysis in the Summary, on quality parity at convergence, and on the original OSS TSan report. Reviewed By: mnorris11 Differential Revision: D112612388 Pulled By: alibeklfc fbshipit-source-id: 32f3532603d5ff01ed0c89d444cb9a636c25f5ae |
||
|
Juan Carpio Castellanos
|
a9ecee221d |
Reject AdditiveQuantizer with zero-bit code size on deserialization (#5506)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5506 Deserializing an AdditiveQuantizer-based index (residual quantizer, local-search quantizer, and their product variants) validated that the number of sub-quantizers M is positive, but not that the resulting per-vector code size is nonzero. A crafted index can set every sub-quantizer's bit width to 0, which makes the derived code size 0 while the index still declares a positive vector count. Such an index ends up with an empty codes buffer whose data pointer is null/unspecified. Decoding any vector then dereferences that pointer inside the bitstring reader used to unpack per-vector codes, causing a segfault on fully attacker-controlled input. Reject this configuration right where the code size is derived, with a descriptive exception. A quantizer that stores zero bits per vector cannot decode anything and is never produced by real training, so this only rejects malformed/crafted inputs. Reviewed By: trang-nm-nguyen Differential Revision: D115173394 fbshipit-source-id: 289c16f1da7280af04a2be6a5629d5af6154a3e7 |
||
|
Juan Carpio Castellanos
|
4de3bbb601 |
Faiss OSS nightly autofix (#5505)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5505 Automated fix for the Faiss OSS nightly build. ___ Differential Revision: D115137724 fbshipit-source-id: 91fc0317baf2b1273b9750a7d2aa23601fe29123 |
||
|
Lengning Liu
|
da3191e754 |
IDSelectorWithContext: extend context hook to IVFPQ/PQR, multibit RaBitQ, and Panorama scanners (#5491)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5491 D113139231 added `faiss::IDSelectorWithContext` and wired exactly one scan loop (`run_scan_codes1`, used by IVFFlat / IVFSQ). This diff extends the same "hand the selector its scan context" hook to the remaining linear IVF per-candidate selector sites, so `IDSelectorWithContext` is honored uniformly instead of being silently dropped depending on index type or a runtime parameter (`nb_bits`). A new shared helper `IDSelectorContextDispatch` (in `IDSelector.h`) centralizes the once-per-list `dynamic_cast` + per-candidate dispatch. `run_scan_codes1` is refactored onto it, and the new sites reuse it: - `IVFPQ` / `IVFPQR`: `WrappedSearchResult::skip_entry` — one edit covers the table / pointer / on-the-fly / polysemous variants, and IVFPQR reuses the IVFPQ scanner. - multibit `RaBitQ` (`nb_bits >= 2`): the two hand-written `scan_codes_multibit` loops (scalar + SIMD). This closes a real footgun where the hook was honored at 1 bit (via `run_scan_codes1`) but silently dropped at `nb_bits >= 2` within the same index type. - `IVFFlatPanorama`: `progressive_filter_batch` — the scan-order position is the full-list `global_idx`, so a lookahead crosses batch boundaries correctly. No behavior change: `is_member_with_context` returns a verdict identical to `is_member`; the context is a prefetch-only side channel. Selectors that do not implement the sub-interface, and `store_pairs` scans (synthetic ids), fall back to `is_member`. The base `IDSelector` is untouched, so every existing selector is unaffected. Binary size: one small inline struct plus a cached pointer; no new template dimensions. FastScan (`IVFPQFastScan` / `IVFRaBitQFastScan`) is intentionally deferred to a benchmark-gated follow-up: its result-lane scan is sparse and it is LUT/SIMD-bound, so the prefetch benefit there is speculative (correctness holds either way). Out of scope: `IVFSpectralHash` (asserts no selector), binary IVF, and HNSW (separate selector paths). Stacks on D113139231. Reviewed By: mnorris11 Differential Revision: D113913501 fbshipit-source-id: 011f52015f3e87b5b7cf91291edbe020bedadbeb |
||
|
Lengning Liu
|
edddbc998e |
IDSelectorWithContext: scan-context hook for locality-aware selectors (#5490)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5490 Adds `faiss::IDSelectorWithContext`, an `IDSelector` sub-interface whose `is_member_with_context(id, ids, list_size, j)` also receives the surrounding scan context, so an implementation can exploit locality across a scan (e.g. prefetch data for an entry it will be asked about soon). The IVF inner scan loop `run_scan_codes1` `dynamic_cast`s the selector once per inverted list (RTTI, as already used for `IDSelectorRange` in IndexIVF.cpp) and, when it matches, calls `is_member_with_context` instead of `is_member` — a single predicted branch per candidate, no template dimension, and the base `IDSelector` is unchanged so every existing selector is unaffected. No behavior change on its own: a selector that does not implement the sub-interface takes the plain `is_member` path. First consumer is admarket's `FilterableIDSelector` (stacked on top). Reviewed By: mnorris11 Differential Revision: D113139231 fbshipit-source-id: 48a5ede7b502771b0b9a66593719502c5202c6da |
||
|
jeevan6996
|
b5ca016ded |
Fix IndexBinaryIVF reconstruct_n and search_and_reconstruct byte stride (#5463)
Summary: - use `code_size` instead of bit dimension `d` when placing `IndexBinaryIVF::reconstruct_n` outputs - add a regression test covering compact Python `reconstruct_n` output for binary IVF indexes Fixes https://github.com/facebookresearch/faiss/issues/5386. ## Duplicate / ownership checks - issue https://github.com/facebookresearch/faiss/issues/5386 is open and unassigned - maintainer comment says a fix would be reviewed - searched open PRs for https://github.com/facebookresearch/faiss/issues/5386, `IndexBinaryIVF reconstruct_n`, and `code_size`; no matching PR found before opening this PR ## Testing - `git diff --check` - `cmake -B build-codex-py312-cpu -DFAISS_ENABLE_GPU=OFF -DFAISS_ENABLE_METAL=OFF -DFAISS_ENABLE_CUVS=OFF -DFAISS_ENABLE_PYTHON=ON -DBUILD_TESTING=OFF -DBUILD_SHARED_LIBS=ON -DFAISS_OPT_LEVEL=generic -DOpenMP_ROOT=/opt/homebrew/opt/libomp -DPython_EXECUTABLE=/opt/homebrew/bin/python3.12 -DCMAKE_CXX_FLAGS=-isystem/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 .` - `cmake --build build-codex-py312-cpu --target faiss swigfaiss -j4` - `python3.12 setup.py build` from `build-codex-py312-cpu/faiss/python` - `python3.12 -m unittest test_index_binary.TestBinaryIVF_ARM_NEON.test_ivf_reconstruct_n_uses_code_size_stride` - `python3.12 -m unittest test_index_binary.TestBinaryIVF_ARM_NEON` Note: `BUILD_TESTING=ON` attempted to fetch googletest, but local DNS/network resolution blocked that fetch. I used the focused Python binary-IVF tests against a local CPU/Python build instead. ## Added during import `IndexBinaryIVF::search_and_reconstruct` has the identical defect, so it is fixed here too. It strided `recons` by the bit dimension `d` and, for the not-found (`key < 0`) padding, memset `d` bytes -- but the header documents `recons` as size `(n, k, d / 8)`. Both now use `code_size`, matching the base `IndexBinary::search_and_reconstruct`. The `memset` is the worse of the two: it unconditionally wrote `8 * code_size` bytes whenever a query returned fewer than `k` neighbors. The regression test in this diff covers `reconstruct_n` only; `search_and_reconstruct` coverage is being added separately. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5463 Reviewed By: junjieqi Differential Revision: D114687715 Pulled By: mnorris11 fbshipit-source-id: 86c38f53edc90f1e5a776462b7a5e55b77b5fdcf |
||
|
Akash Nayar
|
76c67b7b73 |
Update default Panorama batch size to 1024 (#5441)
Summary: ## Changes This PR unifies the default Panorama batch size to 1024 going forward. This was shown to offer better performance than the current values ([paper](https://arxiv.org/abs/2510.00566), figure 11). ## Performance ### IVFFlatPanorama SIFT1M | nprobe | qps_before | qps_after | speedup | |---|---|---|---| | 1 | 5980.4 | 6343.7 | 1.06x | | 2 | 3537.7 | 3933.2 | 1.11x | | 4 | 2105.4 | 2419.9 | 1.15x | | 8 | 1245.2 | 1466.4 | 1.18x | | 16 | 729.2 | 887.4 | 1.22x | | 32 | 429.6 | 538.0 | 1.25x | | 64 | 249.0 | 319.3 | 1.28x | GIST1M | nprobe | qps_before | qps_after | speedup | |---|---|---|---| | 1 | 1826.0 | 1494.7 | 0.82x | | 2 | 1091.0 | 1001.7 | 0.92x | | 4 | 642.2 | 639.0 | 0.99x | | 8 | 368.7 | 392.7 | 1.07x | | 16 | 212.2 | 233.9 | 1.10x | | 32 | 122.2 | 138.6 | 1.13x | | 64 | 71.3 | 82.6 | 1.16x | ## FlatL2Panorama SIFT1M | qps_before | qps_after | speedup | |---|---|---| | 124.8 | 133.4 | 1.07x | GIST1M | qps_before | qps_after | speedup | |---|---|---| | 44.1 | 46.2 | 1.05x | Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5441 Reviewed By: alibeklfc Differential Revision: D114387063 Pulled By: mnorris11 fbshipit-source-id: b71ba6ced074c4998aee3183df236b692626cd34 |
||
|
ihb2032
|
3b50babc59 |
Optimize RVV batch-4 distance kernels (#5469)
Summary: Summary This PR refactors the RVV distance implementation and adds native vectorized batch-4 kernels for inner product and squared L2 distance. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5469 Reviewed By: alibeklfc Differential Revision: D114382677 Pulled By: mnorris11 fbshipit-source-id: c55354c296c1c9c96a8e6cec9388fb3514773218 |
||
|
Kn0688
|
b3942aba5a |
Avoid signed int overflow in VStackInvertedLists binary search (#5482)
Summary: translate_list_no() for VStackInvertedLists used int indices for the binary search over vil->ils, with i1 initialized from the size_t ils.size() (a narrowing conversion). The midpoint computation (i0 + i1) / 2 could overflow int, which is undefined behavior. Use size_t indices and compute the midpoint as i0 + (i1 - i0) / 2, which cannot overflow. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5482 Reviewed By: trang-nm-nguyen Differential Revision: D114388075 Pulled By: mnorris11 fbshipit-source-id: b7e7234b16b64c01d2e29d973155ccb97673bb8d |
||
|
Michael Norris
|
20f14b31a6 |
Increment to next release, v1.15.0 (#5494)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5494 Reviewed By: trang-nm-nguyen Differential Revision: D114413409 Pulled By: mnorris11 fbshipit-source-id: b30b14ed3a5a55486ae379700c7fda5773e2620cv1.15.0 |
||
|
Alibek Zhakubayev
|
4332ceb994 |
Enable byte-LUT gather path for QT_8bit_eden distance computation (#5498)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5498 `EDENQuantizer::supports_byte_lut` excludes `nb_bits == 8`, forcing the highest-precision EDEN scalar-quantizer variant onto a fully scalar, per-dimension bit-unpacking distance path via `eden_utils::get_distance_computer` (`EDENOptimizedDistanceComputer`), the computer used by `IndexEDEN` and `IndexIVFEDEN`. ## Root cause `build_code_dot_lut` gates on `supports_byte_lut(nb_bits)` before constructing a per-query LUT. For `nb_bits == 8`, the check returned false, so `lut_kind` stayed `CodeDotLUTKind::None` and every call to `distance_to_code` fell back to `compute_code_dot_reference`, a per-dimension loop through `BitstringReader::read` (bit-shift, mask, buffer-position bookkeeping per dimension). `QT_1bit_eden`, `QT_2bit_eden`, and `QT_4bit_eden` already have this LUT gated through the same predicate. The generic "Byte" LUT branch in `build_code_dot_lut` (lines ~191-210) was already correct for `nb_bits == 8` (`values_per_byte == 1`, one 256-entry row per dimension) but unreachable because of the early return. ## Fix Add `nb_bits == 8` to `supports_byte_lut`'s predicate. This is the only change required to `EDENQuantizer.cpp`: no new code paths, no signature changes. `distance_to_code` now uses `compute_code_dot_lut` (direct `table[code[i]]` array reads, partially unrolled) in place of the bit-unpacking reference loop. Note this fix activates the scalar table-lookup path reachable via `distance_to_code` (used by both `IndexEDEN`'s brute-force scan and `IndexIVFEDEN`'s per-list scan). It does not activate the AVX2/AVX512 *batch*-gather kernels (`compute_code_dot_lut_batch_8_avx2`, `compute_code_dot_lut_batch_8_avx512`/`_batch_16_avx512`) for `IndexEDEN::search` specifically: `IndexEDEN.cpp`'s `use_eden_batch_scan` gate was not updated by this diff and still returns `false` for `nb_bits == 8`, so those kernels remain unreachable for flat EDEN search. That gap is logged as a separate follow-up (see Notes) rather than bundled into this diff, since it touches a different file/function and needs its own benchmark. ## Benchmark `IndexEDEN` built via `index_factory(d, "EDEN8")`, `SyntheticDataset`, k=10, best-of-5 search latency (`mode/opt`): | d | threads | before | after | speedup | |---|---|---|---|---| | 128 | 1 | 27,194.11 ms (36.8 QPS) | 8,629.36 ms (115.9 QPS) | 3.15x | | 128 | 56 | 1,197.53 ms (835.1 QPS) | 392.58 ms (2,547.3 QPS) | 3.05x | | 768 | 1 | 161,997.57 ms (6.2 QPS) | 64,010.54 ms (15.6 QPS) | 2.53x | | 768 | 56 | 5,250.05 ms (190.5 QPS) | 2,886.25 ms (346.5 QPS) | 1.82x | nb=100,000, nq=1,000 in all four configurations. The measured speedup comes from replacing the scalar bit-unpacking loop with a scalar table lookup (both single-threaded per-code cost, no new SIMD instructions on this reachable path) -- it exceeds the 10% impact-gate threshold by a wide margin regardless of mechanism. ## Tests Added `bits=8` to three existing parity/regression loops in `tests/test_eden.py` (`test_distance_computer_matches_reference_l2`, `test_high_dimensional_search_matches_distance_computer`, `test_search_matches_none_simd_level`) -- these previously covered only `[1, 2, 4]` and never exercised the code path this diff activates (`test_scalar_quantizer.cpp`'s existing `QT_8bit_eden` coverage exercises a separate implementation, `ScalarQuantizer`'s generic `QuantizerLloydMax<8, SL>` dispatch in `quantizers.h`, not `EDENQuantizer.cpp`'s `eden_utils::get_distance_computer`). The new assertions directly validate the new LUT math against a NumPy reference and against `SIMDLevel::NONE` output; all pass, confirming bit-exact parity with the pre-fix scalar reference computation. `test_scalar_quantizer` (16 tests), `test_eden` (8 tests, includes the 3 newly-added `bits=8` assertions) pass. Full Monday quantization rotation suite (`test_pq_encoding`, `test_scalar_quantizer_correctness`, `test_residual_quantizer`, `test_local_search_quantizer`, `test_standalone_codec`, `test_product_quantizer`, `test_rabitq`, `test_rabitq_fastscan`) passes: 245 pass, 0 fail (2 skips are a pre-existing devserver limitation, `SIMDLevel.NONE` unavailable, unrelated to this change). `buck2 build fbcode//faiss:faiss` clean, 0 warnings. ## Notes for reviewer Two related follow-ups identified during self-review, logged to the FAISS improver findings backlog rather than bundled into this diff: 1. `IndexEDEN.cpp`'s `use_eden_batch_scan` doesn't include `nb_bits == 8`, so the AVX2/AVX512 batch-gather kernels remain unreachable for flat EDEN search even after this fix. Mechanical follow-up, needs its own benchmark to quantify incremental gain. 2. `EDENOptimizedDistanceComputer::set_query` (and hence `IVFEDENDistanceComputer::operator()`, reachable via the generic `Index::get_distance_computer()` API used by `IndexRefine`, HNSW-on-flat-storage, NNDescent, NSG) rebuilds a full LUT on every single-code query. This is a pre-existing pattern already present for `nb_bits` 4 at `d < 256` (unbounded byte-LUT path, same as this diff's default for `nb_bits == 8`), not introduced by this diff, but worth a dedicated look given `nb_bits == 8` has no half-byte-style size cap the way 1/2/4-bit do. ___ Differential Revision: D114590565 fbshipit-source-id: eacae7855324c5f593b3de7ea8df6c84c171080e |
||
|
generatedunixname923199350665076
|
6187d940e3 |
clang-diagnostic-unused-parameter in GpuCloner.cpp (#5480)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5480 Reviewed By: limqiying Differential Revision: D113537493 fbshipit-source-id: d10114baf0b335810faf7b286a36de3346ec826a |
||
|
Ivan Sopin
|
37313e7b64 |
Fix two memory leaks in shard_ivf_index_centroids (#5495)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5495 `handle_ivf` / `handle_binary_ivf` (behind `shard_ivf_index_centroids` / `shard_binary_ivf_index_centroids`) leaked memory in two places; this fixes both: 1. **Leaked `clone`.** Each function clones the input index into a `clone` whose quantizer is reset, then clones `clone` once per shard. The per-shard clones are deleted, but `clone` itself was never freed — one leaked `IndexIVF` (with its sub-objects) per call. Added the missing `delete clone;` in both helpers. 2. **Leaked quantizer in the `generate_ids=true` path.** That path replaces the per-shard clone's quantizer with `new IndexIDMap2(quantizer)`, but the IDMap did not own the wrapped quantizer, so the cloned quantizer (and its codes) leaked per shard. Set `own_fields = true` on the IDMap so it is freed together with the shard index. Surfaced by a LeakSanitizer failure in a new Galileo unit test that shards a trained IVF model; the fix also benefits the existing LaserKNN sharded-training caller. Reviewed By: mnorris11 Differential Revision: D112336282 fbshipit-source-id: 080657355299a45c2f1dbb36cab9304d825daa5d |
||
|
lyang24
|
7a4e7975d1 |
Fix RaBitQ fast scan aux offsets for large bbs (#5421)
Summary: Fix RaBitQ FastScan aux-factor lookup for bbs > 32 so SIMD sub-blocks read factors from the correct vector offsets. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5421 Test Plan: Added regression coverage for the bbs>32 aux-factor offset fix. Non-IVF path (imported C++ test, now wired into the internal Buck build via a new `test_rabitq_fastscan_cpp` target): buck test fbcode//faiss/tests:test_rabitq_fastscan_cpp => Pass 1 IVF path (new end-to-end regression test `test_ivf_large_bbs_aux_offsets`; `nlist=1` + `bbs=64` so a single list spans multiple bbs blocks and the `b>=1` SIMD sub-blocks exercise `idx_base % bbs != 0`): buck test fbcode//faiss/tests:test_rabitq_fastscan -- test_ivf_large_bbs_aux_offsets => Pass 1 Red/green verified: the IVF test FAILS on the pre-fix offset (`aux_base + j * storage_size`) and PASSES with the fix (`aux_base + ((idx_base % bbs) + j) * storage_size`). Reviewed By: mnorris11 Differential Revision: D112829433 Pulled By: alibeklfc fbshipit-source-id: 0271fddc459a6af6675ebfb9078c44e9476733b3 |
||
|
lyang24
|
b1d61c3b88 |
Fuse RaBitQ AND-dot and popcount scan (#5412)
Summary: Fuse RaBitQ’s AND-dot and doc-side popcount into a single SIMD pass, avoiding a second scan of the same binary code. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5412 Reviewed By: mnorris11 Differential Revision: D112829641 Pulled By: alibeklfc fbshipit-source-id: d8015661fe27e656d34a210ddeec7789aed1802e |
||
|
Alibek Zhakubayev
|
9579d8f0a4 |
Add type stubs for SVS index classes + IndexIVFFlatPanorama.batch_size (#5489)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5489 Fbcode/faiss/python/__init__.pyi is missing type stubs for 7 SWIG-exposed SVS (Intel Scalable Vector Search) index classes and one field on IndexIVFFlatPanorama, causing Pyre `[missing-attribute]` errors for any Python caller that constructs or type-annotates these classes. `swigfaiss.swig` `%include`s and `DOWNCAST`s `IndexSVSFlat`, `IndexSVSVamana`, `IndexSVSVamanaLVQ`, `IndexSVSVamanaLeanVec`, `IndexSVSIVF`, `IndexSVSIVFLVQ`, and `IndexSVSIVFLeanVec` (lines 763-769, 928-934) -- all 7 classes are live, constructible Python types today. None of the 7 appear anywhere in `__init__.pyi`: 0 matches for each. Verified against current trunk before implementing, not from a stale backlog snapshot. This diff adds: - `SVSStorageKind = int` module-level type alias plus the 10 constant values (`SVS_FP32`, `SVS_FP16`, `SVS_SQ8`, `SVS_LVQ4x0/4x4/4x8`, `SVS_LVQ8x0`, `SVS_LeanVec4x4/4x8/8x8`, `SVS_count`), matching the existing `MetricType = int` / `METRIC_L2: int` convention used for every other plain (non-`enum class`) C++ enum in this file -- `SVSStorageKind` is declared as an unscoped `enum` in `svs/IndexSVSVamana.h`, so SWIG exposes its values as bare top-level names, not `SVSStorageKind_`-prefixed names (that prefix convention only applies to C++11 `enum class` types, e.g. `ClusteringInitMethod`). - `SearchParametersSVSVamana` and `SearchParametersSVSIVF` stubs (`search_window_size`/`search_buffer_capacity`, `n_probes`/`k_reorder` respectively), the two `SearchParameters` subclasses defined alongside the SVS index headers that were equally unstubbed. - `IndexSVSFlat`, `IndexSVSVamana` (+ `is_lvq_leanvec_enabled()` static method), `IndexSVSVamanaLVQ`, `IndexSVSVamanaLeanVec` (+ `train_with_queries` support surfaced via the base `Index.train()` Python wrapper, not a separate stub method -- see below), `IndexSVSIVF` (+ `is_lvq_leanvec_enabled()`), `IndexSVSIVFLVQ`, and `IndexSVSIVFLeanVec`, each with the constructor signature and public config fields read directly from their C++ headers (`svs/IndexSVS*.h`). Internal implementation-detail members (`impl`, `mmap_owner`, `stored_vectors`, `stored_vectors_valid`, `training_data` -- raw pointers into the unwrapped `svs_runtime::*` library, or reconstruction-cache storage) are deliberately excluded, matching this file's existing convention of omitting internal storage fields even when technically public (e.g. `IndexIVFFlatPanorama.cum_sums` is similarly excluded today). - `IndexIVFFlatPanorama.batch_size: int` field and constructor parameter (default `Panorama::kDefaultBatchSize` = 128, expressed as `= ...` per this file's existing convention for non-literal defaults). The class already had `n_levels` stubbed but was missing `batch_size`, added in the same commit (D111132644-era) as the class itself. Confirmed `train_with_queries` (a raw SWIG-exposed C++ method used internally by `Index`'s Python `train()` wrapper for out-of-distribution training data, per `class_wrappers.py`) does not need its own stub entry: the same pattern already excludes `train_c`/`train_ex`, the other two raw per-numeric-type dispatch targets of the same wrapper -- callers use `index.train(x, xq_train=...)`, not the raw method name directly. This is a stub-only change: `python/__init__.pyi` is consumed exclusively by static type checkers (Pyre, mypy, pyright) and IDE autocompletion, not by the Python runtime, so no behavior change is possible. Validated with `python3 -c "import ast; ast.parse(open('python/__init__.pyi').read())"` (0 syntax errors) and a full-file identifier-occurrence grep confirming each new symbol appears exactly where intended with no accidental duplicate class/constant definitions. `buck2 build fbcode//faiss:faiss` and `fbcode//faiss/python:pyfaiss` both build clean (0 warnings, 0 errors); `buck2 test fbcode//faiss/tests:test_io` (Friday rotation target) passes 30/30. ___ Differential Revision: D114349814 fbshipit-source-id: 4bfe5ad481438d1727247235240634a5a02a73b1 |
||
|
Michael Norris
|
a3d59f0999 |
faiss DD: use 256-bit fast-scan QBS kernel on AMD Zen 4 (split AVX-512) (#5488)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5488 ## TLDR make PQFS faster on AMD Bergamo specifically, without regressing for RaBitQFS, by changing AVX512 --> AVX2 for the FastScan part. The RaBitQ avx512 popcount is still avx512, it is just the FastScan part that stays on AVX2. Under dynamic dispatch (`faiss.dynamic_dispatch=true`) the fast-scan QBS search path picks the 512-bit accumulate kernel whenever the runtime SIMD level is AVX-512. On AMD Zen 4 / Zen 4c ("Bergamo", family 0x19) that is the wrong choice: Zen 4 splits 512-bit ops over two separate 256-bit parts. - This can execute operations in 2 separate operations, so 2 cycles compared to a single cycle in avx2. It can sometimes be slower or faster depending on the operation, because more new instructions are supported in avx512 like vpopcnt which are completely missing in AVX2. - Why split at all? because older CPUs had to downclock for avx512 instructions, while this approach does not, so it can be faster for certain workloads, and it allows support for avx512 instructions for free. Captured from benchmark infra, matched AVX2-static vs AVX-512-DD operating points): - Bergamo `PQ8x4fs`: median AVX-512/AVX2 = 0.861 (nq=1) / 0.860 (batched) across 15 datasets, spread [0.857, 0.867], 14/15 datasets below 0.95 (worst `sift-1M` = 0.60). - Bergamo `PQ16x4fs`: 0.870 (nq=1) / 0.878 (batched) across 14 datasets. The magnitude is tight and reproduces on ~every dataset in both the single-query and batched regimes, well below the Bergamo benchmark noise floor (SVS-Vamana negative control median ~1.05), so it is real signal. It is isolated to the bbs=32 QBS path: `PQ*x4fs_64` (bbs=64) and `IVF*,PQ*x4fs` are unaffected, and on Intel Skylake / Cooper Lake the same factories are FASTER under AVX-512 (1.01-1.71x), so the fix must be keyed to Zen 4 only. Fix (3 files): - `simd_levels.{h,cpp}`: add `SIMDConfig::avx512_split`, set by raw-CPUID detection (vendor == AuthenticAMD && display_family == 0x19), run once at load time from `auto_detect_simd_level()` in both DD and static builds. - `dispatching.h` `ScannerMixIn::accumulate_loop_qbs`: this is the live QBS search dispatch for BOTH PQ fast-scan and RaBitQ fast-scan. When the CPU splits avx512, route to the 256-bit (AVX2) QBS kernel (`pq4_accumulate_loop_qbs_fixed_scaler_256<AVX2>`, the same one the existing unknown-qbs fallback already uses) instead of the 512-bit kernel. Process- constant runtime branch, hoisted out of the inner accumulate loop. Intel AVX-512 keeps the 512-bit kernel unchanged. Note: the fix is placed in `dispatching.h` (the path IndexFastScan / IndexIVFFastScan search actually take, via `ScannerMixIn`), NOT in `decompose_qbs.h` -- the QBS block kernel there is only reached by the `accumulate_to_mem` test utility, not by any search path, so routing it there would be inert for search. Scope: RaBitQ fast-scan shares this same PQ4 QBS accumulate, so it is also routed to 256-bit on Zen 4. That is not expected to regress it -- the 256-bit kernel is >= the 512-bit one on Zen 4 (the whole premise), and RaBitQ's AVX-512 popcount (folded into the query LUT) and multibit FP-refine kernels are untouched. An earlier version of this diff also added a Cooper Lake RaBitQ-popcount cap and a native `vpopcntq` path; the Cooper Lake regression that motivated them did not survive additional benchmark runs, so both were dropped to keep this a single-purpose, evidence-backed fix. Validated on Bergamo (see Test Plan) Reviewed By: junjieqi Differential Revision: D113617130 fbshipit-source-id: c59b522134f40cc7a7daf6abaa3c3f6d5b139fe1 |
||
|
Juan Carpio Castellanos
|
4d7491540d |
Bound lattice_nbit loop in IndexLattice ctor to fix deserialization timeout (T277092535) (#5484)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5484 Agentic fuzzing (faiss_read_index_binary_fuzzer) found a multi-second timeout while deserializing an IndexLattice. After D113405533 bounded the ZnSphereCodecRec decode-cache build, the same crafted input now hangs one frame up, in the IndexLattice constructor: lattice_nbit = 0; while (!(((uint64_t)1 << lattice_nbit) >= zn_sphere_codec.nv)) { lattice_nbit++; } zn_sphere_codec.nv (number of Zn lattice points) grows as O(r2^(dsq/2)) and is a uint64_t; a crafted r2/dsq makes it exceed 2^63 (and overflow). Once lattice_nbit reaches 64 the shift '(uint64_t)1 << lattice_nbit' is undefined -- on x86 the shift count is masked mod 64 so it wraps to a small value that stays < nv, and the loop never terminates. That is an effectively infinite loop reached purely from attacker-controlled bytes, exceeding the fuzzer's 10s per-input timeout. Fix: reject nv > 2^63 inside the loop with a catchable FAISS_THROW_IF_NOT. A lattice code wider than 63 bits is never valid, and real lattice configs (the ZnSphereCodecAlt fallback is dsq=8/r2=14; factory configs use small power-of-two dsq with small r2) need only a handful of bits, so they are orders of magnitude under the bound. run_fuzz_iteration already swallows FaissException. Reviewed By: mnorris11 Differential Revision: D113834749 fbshipit-source-id: cb2ab1278a1913c84393b6bb6d4543255000a656 |
||
|
Michael Norris
|
a032797bd8 |
faiss DD: SQ AVX-512 falls back to AVX2 when d%16!=0 (#5478)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5478 TLDR: missed a DD fallback of AVX512 to sequential when d was not divisible by 16. No unit test because too hard to measure timing in fallback without flakiness. - Under dynamic dispatch, scalar-quantizer (SQ) search is catastrophically slow for vector dimensions that are not a multiple of 16. Measured on `ai_infra.vench_benchmark_results` (nq=1), AVX-512-vs-AVX2 median speedup by dimension: dim 128 -> 1.13x dim 512 -> 1.31x dim 1024 -> 1.42x dim 200 -> 0.199x dim 768 -> 1.42x dim 2048 -> 1.39x (worst `IVF*,SQfp16` at d=200: ~0.05x, i.e. ~19x slower than AVX2) dim=200 is the only benchmarked dimension that is not a multiple of 16, and it is the only one that regresses; every multiple-of-16 dim gets a 1.1-1.4x gain. This affects all x86 microarchitectures (not just one CPU). Root cause: `is_dimension_compatible<AVX512>()` (`sq-dispatch.h`) requires `d % 16 == 0`, so for d=200 the AVX-512 SQ factory returns `nullptr`. The dispatch in `ScalarQuantizer::get_distance_computer` / `select_InvertedListScanner` / `select_quantizer` then falls back directly from AVX-512 to the scalar `SIMDLevel::NONE` path, skipping AVX2. `with_simd_level_spr` dispatches once at the host's top level (AVX-512), so the `nullptr` from the AVX-512 factory drops straight to NONE rather than trying AVX2. On an AVX-512 host, d=200 therefore runs a scalar per-component loop, while an AVX2 host runs the fast 8-wide kernel (d=200 is a multiple of 8). Fix: in all three SQ dispatch sites, when the top level is an AVX-512 variant and its factory declines, try AVX2 before the scalar path. AVX2 needs only `d % 8 == 0` (satisfied by d=200), is always available on an AVX-512 host, and is dramatically faster than scalar. This recovers d=200 SQ from ~0.2x to ~1.0x (parity with AVX2, which is what non-DD prod already uses). Chose the AVX2 fallback over adding a masked 512-bit tail to the float SQ distance/decode loops (`DCTemplate<...,AVX512>`): the masked-tail approach would touch the decode path of every codec (8/4/6-bit, fp16, bf16) with fiddly masked loads over packed sub-byte codes, a much larger and riskier change, for only a marginal gain over AVX2 at these dimensions. The AVX2 fallback is minimal and fully removes the regression. My questions: -- Why can't we replace with_simd_level and always fall back? -- ``` They are not interchangeable with with_simd_level_fallback. There are two independent differences, and the first is the disqualifying one: a) Null-decline semantics + multiple invocation. with_simd_level_fallback is built for selection/factory actions that return a nullable pointer, where null means "this level declines, drop down a level." That's the whole point of dispatch_simd_level_or_lower (simd_dispatch.h:89-103): if (auto result = action.template operator()<current_level>()) { return result; // non-null: accept } // null: recurse to get_simd_fallback(level) This (i) requires the return value to be truthiness-testable, and (ii) may call the action multiple times (once per level until one accepts). Your three SQ sites fit perfectly — they return SQuantizer* / SQDistanceComputer* / InvertedListScanner* and the old code hand-rolled the exact if (q) return q; else NONE fallback that this now encapsulates. with_simd_level / with_simd_level_256bit back work kernels — e.g. the fvec_L2sqr<level>(...) compute loop in the with_simd_level doc example (simd_dispatch.h:209-216). Those: - often return void or a non-pointer — if (auto result = ...) is ill-formed on void and would misread a legitimate 0.0/false as "declined," and - must run exactly once at the selected level; re-invoking a side-effecting loop across levels would be wrong. So you can't fold them in without breaking every void/compute caller. ``` Did we miss any call sites? -- ``` No remaining migration candidates — your diff already covers the whole pattern The audit traced every pointer-returning dispatch site in fbcode/faiss/ down to its underlying <SL> factory to check for the nullable-decline semantics (return nullptr to reject a SIMD level → fall back lower). Findings: 1. The manual decline pattern existed only in ScalarQuantizer.cpp — the three sites your diff already migrated (select_quantizer, get_distance_computer, select_InvertedListScanner). The declining factories they call live in impl/scalar_quantizer/sq-dispatch.h and are the only return nullptr-to-decline factories in the entire tree (each does if (!is_dimension_compatible<SL>(d)) return nullptr; plus per-qtype declines for eden 5/6/7-bit). 2. with_simd_level_spr is fully removed — zero references remain anywhere in fbcode/faiss/. 3. Every other pointer-returning dispatch site is not a candidate — the factory always returns a valid object, so wrapping it in with_simd_level_fallback would be a no-op. Notable ones: ┌───────────────────────────────────┬──────────────────────────────────────────────┬────────────────────────┐ │ Site │ Factory │ Why not │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ IndexPQ.cpp:85 │ get_PQFlatCodesDistanceComputer<SL> │ never null │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ IndexIVFPQ.cpp:529 │ make_IVFPQInvertedListScanner<SL> │ never null │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ IndexBinaryIVF.cpp:470 │ make_binary_ivf_scanner_fixSL<SL> │ never null │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ IndexBinaryHNSW.cpp:298 │ make_binary_hnsw_distance_computer_fixSL<SL> │ never null │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ IndexIVFSpectralHash.cpp:225 │ make_spectral_hash_scanner_fixSL<SL> │ never null │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ EDENQuantizer.cpp:873 │ make_distance_computer_for_level<SL> │ every specialization │ │ │ │ returns valid │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ RaBitQuantizer.cpp:662 │ inline new RaBitQDistanceComputer…<SL> │ always dc.release() │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ fast_scan.cpp:449/461/474/490/514 │ make_*_scanner_impl<SL> │ always valid │ │ │ │ unique_ptr │ ├───────────────────────────────────┼──────────────────────────────────────────────┼────────────────────────┤ │ │ │ returns bool compute │ │ distances_fused.cpp:54 │ exhaustive_L2sqr_fused_cmax<SL> │ result, NONE is a real │ │ │ │ specialization │ └───────────────────────────────────┴──────────────────────────────────────────────┴────────────────────────┘ 4. All the with_simd_level / with_simd_level_256bit / with_simd_level_a0_spr bulk callers (dozens across utils/distances.cpp, hamming.cpp, ProductQuantizer.cpp, ResidualQuantizer.cpp, etc.) are run-once void/scalar compute kernels — correctly left alone, as we discussed. ``` Reviewed By: alibeklfc Differential Revision: D113787304 fbshipit-source-id: 31d08fac07f25511ebfed9c0e1937bf76ee44ba9 |
||
|
Juan Carpio Castellanos
|
5987f523f4 |
Reject nbits==0 ProductQuantizer on deserialization (T279464247) (#5483)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5483 Agentic fuzzing (faiss_rmdt_post_read_reconstruct_fuzzer) found a null-pointer read in PQDecoderGeneric::decode(): a serialized index whose ProductQuantizer declares nbits == 0 derives code_size == 0 (fbcode/faiss/impl/ProductQuantizer.cpp:69), which makes the codes-size consistency check in the deserializer trivially satisfied by an empty codes vector for any ntotal ("0 == ntotal * 0"). The first reconstruct() call then hands codes.data() == nullptr down to PQDecoderGeneric::decode(), which dereferences it unconditionally (reg = *code). D112368333 patched this for the IxPq (IndexPQ) branch specifically by adding a `code_size > 0 || ntotal == 0` guard local to that branch. That missed every other deserialization path that shares the same read_ProductQuantizer() entry point and the same nbits==0 degenerate case -- in particular Index2Layer ("Ix2L"), whose pq.code_size feeds code_size_2 via validate_code_size_match(), which only checks equality and not positivity. The finding kept regressing (crashbot re-flagged it on 2026-07-24, 07-27, 07-28) because the crash still reproduces, now via Index2Layer::sa_decode -> ProductQuantizer::decode -> PQDecoderGeneric::decode. Fix: move the validation to the shared deserializer entry point, read_ProductQuantizer() (fbcode/faiss/impl/index_read.cpp), by tightening the existing nbits upper-bound check into a range check requiring nbits >= 1. This is "Option 1" from the original AI Security Assessment, which the assessment itself noted is the correct place for the check: nbits == 0 is a legitimate in-memory state for a default-constructed, untrained PQ (ProductQuantizer() : ProductQuantizer(0, 1, 0)), so the check can't live in set_derived_values(); it belongs at deserialization, where nbits == 0 always means a corrupt/malicious index. This closes the gap for every current and future caller of read_ProductQuantizer (IxPq, IVFPQ, IVFPQR, Index2Layer, IMI, PQFastScan, ...), not just IndexPQ. Reviewed By: mnorris11 Differential Revision: D114027516 fbshipit-source-id: 571ededc9bd055b2fd220f7d2431e01c9ba7fcbd |
||
|
lyang24
|
88b6c47cf5 |
Implement IVFRaBitQ FastScan conversion constructor (#5422)
Summary: Fix broken IndexIVFRaBitQFastScan(IndexIVFRaBitQ) conversion constructor by packing existing IVF RaBitQ codes into FastScan blocks while preserving aux data. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5422 Reviewed By: alibeklfc Differential Revision: D112829289 Pulled By: bshethmeta fbshipit-source-id: 4d616a7f31b28678902f972b855599fae2decae3 |
||
|
Juan Carpio Castellanos
|
a3e8c67c60 |
Fix heap-buffer-overflow in ResidualCoarseQuantizer deserialization (T280514309) (#5485)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5485 Fixes T280514309: an attacker-tunable heap-buffer-overflow (write) that fires during `faiss::read_index()` when deserializing a malformed `ResidualCoarseQuantizer` index (`ImRQ` fourcc). Root cause: the FAISS index format encodes the centroid count twice - once as the index header field `ntotal`, and once implicitly as `2^sum(rq.nbits)` (`tot_bits`). On the `ImRQ` read path these two fields are restored independently and never reconciled. `set_beam_factor()` then sizes the `centroid_norms` buffer from `ntotal` (IndexAdditiveQuantizer.cpp:516) while `compute_centroid_norms()` writes `2^tot_bits` floats (AdditiveQuantizer.cpp:386). A crafted index with `ntotal < 2^tot_bits`, `beam_factor <= 0`, and `metric_type == METRIC_L2` overflows the heap allocation. The trained-constructor invariant `FAISS_THROW_IF_NOT(rq.tot_bits <= 63)` and `ntotal == 1 << tot_bits` are bypassed by deserialization, which assigns fields directly onto a default-constructed object. Fix: reconcile the two fields in the `ImRQ` branch of `read_index_up()` before `set_beam_factor()` runs, matching the cross-field consistency checks that sibling index types already perform. We restore the `tot_bits <= 63` bound (also avoiding UB in `(size_t)1 << tot_bits`) and require `ntotal == 2^tot_bits`, which holds by construction for every legitimately-trained ResidualCoarseQuantizer. Reviewed By: mnorris11 Differential Revision: D113832696 fbshipit-source-id: 4bc2f9f5451ba6e6b95cdd33abbe8bafba295b6f |
||
|
Binozo
|
aff8589c9e |
c_api: expose ScalarQuantizer codec and complete FaissQuantizerType enum (#5477)
Summary: Hi, the `c_api` is missing exports for the FaissScalarQuantizer scalar quantizer. I've added them. Additionally I synced the `c_api/IndexScalarQuantizer_c.h` `FaissQuantizerType` enum with the cpp side which is required because `faiss_IndexScalarQuantizer_sq` may return an enum value which is not contained in the `c_api` enum definition. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5477 Reviewed By: trang-nm-nguyen Differential Revision: D114077316 Pulled By: mnorris11 fbshipit-source-id: 6c714a86daa20a490b3892c3a78356b7beebc92e |
||
|
Tom Jackson
|
13aa39b791 |
Clone index support for HadamardRotation (#5475)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5475 Reviewed By: mnorris11 Differential Revision: D113851808 fbshipit-source-id: fde2442bdd00fd1a6020b182019342a782215c98 |
||
|
generatedunixname923199350665076
|
a9683455b7 |
clang-diagnostic-unused-parameter in bench_result_handler_overhead.cpp (#5481)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5481 Reviewed By: limqiying Differential Revision: D113537265 fbshipit-source-id: a92d436db6385d8476609b3b1c648a870a7a0305 |
||
|
generatedunixname923199350665076
|
d6d3dce8d7 |
clang-diagnostic-unused-parameter in bench_cppcontrib_sa_decode.cpp (#5479)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5479 Reviewed By: limqiying Differential Revision: D113537359 fbshipit-source-id: 458da98d641ecb38d4b85bfa8dd3bb7ecdef35fb |
||
|
Tia Lim
|
bc102d61b8 |
Bound IVFPQ precomputed-table allocation at deserialization time (#5464)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5464 The `faiss_read_index_binary_fuzzer_ci_diff_time` health-check has been failing (T276499517). Reproduced at HEAD: libFuzzer reports `out-of-memory (malloc(2147483648))` with the stack `AlignedTableTightAlloc<float,32>::resize` <- `initialize_IVFPQ_precomputed_table` <- `read_ivfpq` <- `read_index_binary` (the `IBFf` path). Root cause: `read_ivfpq()` calls `IndexIVFPQ::precompute_table()`, which builds a derived table of `quantizer->ntotal * pq.M * pq.ksub` floats. That table is NOT stored in the serialized index — it is recomputed on load from attacker-controlled header fields — and faiss only capped it at `precomputed_table_max_bytes` (2 GB). Every other derived allocation in `index_read.cpp` is already bounded by the runtime-configurable `get_deserialization_vector_byte_limit()` (which the fuzzer sets to 128 MB), but this precompute path was not, so a tiny crafted header can drive a multi-GB allocation and OOM. Fix: bound `ntotal * pq.M * pq.ksub` by `get_deserialization_vector_byte_limit()` (overflow-safe via `mul_no_overflow`) before calling `precompute_table()`, mirroring the existing `read_ProductQuantizer` guard style. For normal library use the limit defaults to 1 TB, so realistic tables (<= 2 GB) still load exactly as before — no behavioral change. Also adds a deterministic regression test `ReadIndexDeserialize.IVFPQPrecomputedTableExceedsByteLimit` that serializes a real trained IVFPQ and confirms reading it back under a low byte limit throws instead of allocating. Reviewed By: mnorris11, alibeklfc Differential Revision: D111974816 fbshipit-source-id: 89b23905cd1b9406d957471c574afc0c2b1c91a9 |
||
|
Himanshu Gupta
|
247527b349 |
Fix reconstruct_n stubs resolving to torch.Tensor for numpy calls (#5411)
Summary: Index.reconstruct_n(n0, ni) returns a numpy array by default, but the first matching overload was the torch one, so mypy resolved buffer-less calls to torch.Tensor and rejected code assigning the result to an ndarray. The buffer-less return type cannot be resolved statically: importing faiss.contrib.torch_utils replaces the method with one that allocates a torch.Tensor, while the default implementation allocates an ndarray. Reordering the overloads (or adding a numpy-returning overload for that shape) would just move the false positive onto torch_utils users, so type the no-buffer shape as Any and keep exact types on the overloads that take an explicit x. Applied to Index, IndexPreTransform and IndexBinary, which all carried the same overload pair. Fixes https://github.com/facebookresearch/faiss/issues/5232 Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5411 Reviewed By: mnorris11 Differential Revision: D112830101 Pulled By: bshethmeta fbshipit-source-id: 0ec9877047dec2023949dafeeffd79af3e14a641 |
||
|
lyang24
|
81031a9789 |
RaBitQ fastscan avx512 lut quant (#5397)
Summary: Vectorize RaBitQ FastScan LUT quantization with AVX-512 to reduce query setup overhead <img width="1672" height="941" alt="exec-4dd00ef0-e5cf-4d1e-bc4f-2757cba24840" src="https://github.com/user-attachments/assets/1ab6b779-c9c2-4a03-82fe-d5ffa4ae9d14" /> Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5397 Reviewed By: mnorris11 Differential Revision: D113798910 Pulled By: alibeklfc fbshipit-source-id: 68e803939ed9c4d8d1e67bebaf5cc9804138f940 |
||
|
ivan-digital
|
16c5d782e0 |
Don't redefine __PRETTY_FUNCTION__ under clang-cl (#5474)
Summary: `platform_macros.h` redefines `__PRETTY_FUNCTION__` to `__FUNCSIG__` whenever `_MSC_VER` is defined. That is right for MSVC, which has no `__PRETTY_FUNCTION__`, but clang-cl also defines `_MSC_VER` while already providing `__PRETTY_FUNCTION__` in clang's own format. The macro therefore replaces a working definition with one in a different format. Anything that parses it then misbehaves, and does so far from the cause. magic_enum derives enumerator names from `__PRETTY_FUNCTION__`, so with clang-cl every translation unit that includes any FAISS header silently loses enum reflection. The failure surfaces as a static assertion about an unrelated enum, with nothing pointing back at FAISS. Found while building a project that embeds FAISS with clang-cl on Windows. Bisecting the include list showed reflection failing with any FAISS header present and succeeding with none; restricting the shim to Microsoft's compiler fixes it, and FAISS itself builds unchanged. MSVC is unaffected: it does not define `__clang__`, so it still gets the shim. Non-Windows targets never reach this branch. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5474 Reviewed By: mnorris11 Differential Revision: D113781946 Pulled By: trang-nm-nguyen fbshipit-source-id: 44616e4cf967b7f2178e1e2ead3715b6308cd9ea |
||
|
Michael Norris
|
35c5495b8f |
Remove confusing double-negative throw-macro conditions (#5471)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5471 Audited `fbcode/faiss` for uses of the `FAISS_THROW_IF_NOT*` macros whose condition itself contains a negation (`!` or `!=`). Because `FAISS_THROW_IF_NOT(X)` throws when `X` is *false*, a negated condition reads as a double negative — e.g. `FAISS_THROW_IF_NOT_MSG(!params, "...")` actually throws when `params` *is* set, and `FAISS_THROW_IF_NOT(ptr != nullptr)` throws when `ptr` *is* null. Rewrites (all behavior-preserving): - `FAISS_THROW_IF_NOT_MSG(!X, MSG)` -> `FAISS_THROW_IF_MSG(X, MSG)` - `FAISS_THROW_IF_NOT_MSG(A != B, MSG)` -> `FAISS_THROW_IF_MSG(A == B, MSG)` - `FAISS_THROW_IF_NOT(ptr != nullptr)` -> `FAISS_THROW_IF_NOT(ptr)` (pointer truthiness) - `FAISS_THROW_IF_NOT(!X)` / `FAISS_THROW_IF_NOT(A != B)` -> `FAISS_THROW_IF_MSG(...)` with a concise message (there is no bare `FAISS_THROW_IF` / `FAISS_THROW_IF_FMT`, so positive-polarity throws must carry a message). - Compound conditions were rewritten via De Morgan's law so the surviving condition has no `!`/`!=` where practical (e.g. `M != 0 && K != 0` -> throw on `M == 0 || K == 0`). Also documented this convention in `faiss/.claude/CLAUDE.md` so future changes avoid reintroducing double negatives. Not changed: three `FAISS_THROW_IF_NOT_FMT(...)` sites with negated conditions (`impl/index_read.cpp`, `impl/index_write.cpp`, `invlists/OnDiskInvertedLists.cpp`). There is no `FAISS_THROW_IF_FMT` macro, so their printf-style format arguments cannot move to `FAISS_THROW_IF_MSG` without adding a new macro. Reviewed By: alibeklfc Differential Revision: D113577769 fbshipit-source-id: 6537b4dbd656adc783bc2972161650c8342643a8 |
||
|
Michael Norris
|
a20b22d0d4 |
Bound decode-cache compute in ZnSphereCodecRec deserialization (#5467)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5467 Agentic fuzzing (T279545561, T279518049 — the same signature via two harnesses) found a multi-second timeout in `ZnSphereCodecRec::decode` reached while deserializing an `IndexLattice`. The `ZnSphereCodecRec` constructor builds a decode cache by calling `decode()` once per cache entry, and each `decode()` costs O(`dim`). The pre-existing cap bounded only stored cache memory (`total_cache_entries * dimsub <= 2^27`, with `dimsub = 2^min(3, log2_dim-1)` clamped at 8 ), so a crafted index with large `dim`/`r2` stayed under it while driving billions of `decode` operations, exceeding the fuzzer's 10s per-input timeout. Replace that memory-only cap with a single bound `total_cache_entries * dim <= 2^27`. One check covers both cost dimensions at once: - Construction cost is one O(`dim`) `decode()` per entry, i.e. `entries * dim` operations (~1.3e8 at the cap, sub-second). - Cache memory is `entries * dimsub` floats. Because `dimsub < dim` always (`dimsub = 2^min(3, log2_dim-1) < 2^log2_dim = dim`), `entries * dim <= 2^27` implies `entries * dimsub <= 2^27`. So the single compute check subsumes the old memory bound — a separate memory check would be strictly redundant. `dim >= 1` is guaranteed by the constructor's power-of-two check, so there is no divide-by-zero, and the `2^27` constant is reused for alignment with the `nv_cum` cap above. Placing the check in the constructor protects every construction path (deserialization, `index_factory`, direct construction) and runs once at construction, not on the search hot path. Real lattice configs (the `ZnSphereCodecAlt` fallback is dim=8/r2=14; factory configs use small power-of-two dim with small r2) are orders of magnitude under the bound; only crafted blow-ups are rejected, now with a catchable `FaissException` that `run_fuzz_iteration` already swallows. Reviewed By: alibeklfc Differential Revision: D113405533 fbshipit-source-id: 9ed07fd6cbd3f99f230cc4ceb1bad80772dd1d4c |