mirror of
https://github.com/facebookresearch/faiss.git
synced 2026-10-12 02:13:36 +00:00
3f08095cee9d36698ad5d72bf19fb7b2dfcb1f8c
2317
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 marcelo-cjl
|
3f08095cee |
fix: release SuperKMeans sample buffer after rotation (#5648)
Summary: - release the temporary subsampled training buffer as soon as rotation has populated `X_tilde` - avoid retaining that buffer during Forgy initialization and the SuperKMeans iteration loop - preserve the existing algorithm and rotation-time peak-memory behavior ## Memory result One build run on Cohere 1M x 768 FP32 with SCANN (`nlist=1024`, `sub_dim=2`, `with_raw_data=false`, 16 threads): - SuperKMeans before this change: 6749.1 MiB average RSS - SuperKMeans with this change: 6425.4 MiB average RSS - reduction: 323.7 MiB (4.8% of total average RSS; 8.5% of build RSS above baseline) The released sample buffer is 768 MiB. The measured reduction is smaller because average RSS is time-weighted over the full build; the buffer is still required during sampling and rotation. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5648 Test Plan: - built the AVX2 CPU `faiss_test` target in Release mode - `faiss_test --gtest_filter='AdSampling.*:PdxLayout.*:BlockL2.*:SuperKMeansAssignIteration.*'` (13 tests passed) - Knowhere vendored integration test for SuperKMeans (3 cases, 44 assertions passed) Closes https://github.com/facebookresearch/faiss/issues/5650 Reviewed By: mnorris11 Differential Revision: D121294074 Pulled By: alibeklfc fbshipit-source-id: c8e6f35f1421907887f9756661912df6ee4c2b87 |
||
|
anthony-zy
|
1768a5a8e9 |
Add RVV SIMD specialization for product quantizer distance-table computation (#5543)
Summary:
Add a RISC-V Vector Extension implementation for the product-quantizer
distance-table computation, and wire it into the SIMD dispatch path. The
kernels are runtime vector-length aware (vsetvl-driven, e32m1) rather than
assuming a fixed vector width.
Changes
- compute_PQ_dis_tables_dsub2<RISCV_RVV> (utils/distances_rvv.cpp): RVV
kernel for both inner-product and L2 dsub=2 distance tables. Centroids are
first split into even/odd dimension planes (c0_all / c1_all) so each
subquantizer's ksub entries load as contiguous vectors; the hot loop
processes two subquantizers per iteration:
IP: r = c0*x0 + c1*x1 (vfmul + vfmacc)
L2: r = (c0-x0)^2 + (c1-x1)^2 (vfsub + vfmul + vfadd)
- Add RVV specializations for the accompanying distance primitives:
fvec_norm_L2sqr, fvec_L2sqr_ny, fvec_L2sqr_ny_transposed and
fvec_inner_products_ny are vectorized (strided vlse32 over ny); the
remaining fvec_* and VectorDistance specializations fall back to
SIMDLevel::NONE.
- compute_distance_tables / compute_inner_prod_tables: route dsub == 2 &&
nbits >= 3 && nbits < 8 through compute_PQ_dis_tables_dsub2 when
COMPILE_SIMD_RISCV_RVV is defined, matching the AVX2 / ARM_NEON fast path.
- compute_PQ_dis_tables_dsub2_dispatch: extend the dispatch mask to include
RISCV_RVV.
### Review fixes
The AI reviewers reported 6 valid defects. This version corrects all of them.
- The RVV `compute_PQ_dis_tables_dsub2` unrolled the subquantizer loop by 2
and had no tail. An odd `M` left the last `ksub` table entries unwritten.
The kernel now writes the last subquantizer in a tail loop.
- `fvec_inner_products_ny<RISCV_RVV>` loaded every chunk from `y + j`. The
base pointer missed the `i * d` offset. Each chunk after the first repeated
the first vectors. The kernel now loads from `y + i * d + j`.
- `fvec_norm_L2sqr<RISCV_RVV>` ran one `vfredusum` per vector chunk. A
`d=768` input at VLEN=128 cost 24 serialized reductions. The function again
keeps a tail-undisturbed accumulator, and it reduces once after the loop.
- The `dsub2` kernels require `ksub % 8 == 0`. The dispatch condition accepted
every `nbits < 8`, so an `nbits` of 1 or 2 made `compute_distance_tables`
throw. The condition now also requires `nbits >= 3`. This defect also
affected the AVX2 and the ARM NEON paths.
- `search_sdc` dropped the per-thread `q_row` pointer precomputation. That
change added `M` multiplications per database vector on every architecture.
This version restores the precomputation.
- The `ny` kernels fixed the chunk width at the VLEN=128 value. They now read
`VLMAX` with `vsetvlmax`. Hardware with a larger VLEN uses the whole vector
register.
The diff also removes 2 unreadable comments from the RVV kernel.
Caveat: the benchmark below ran on the previous version. That version included
the `search_sdc` change that this version reverts.
### Performance (sg2044)
Cmd: 1M 768 <PQ> 32 100 16 1 1000 1 <metric> (search ns/query, lower is better)
| # | CONFIG | M | NPROBE | METRIC | F1(ns) | PQ1(ns) | F2(ns) | PQ2(ns) | Δ_PQ% | Δ_F% |
|----|----------|-----|--------|--------|---------|---------|---------|---------|---------|--------|
| 01 | PQ8x4 | 4 | 8 | L2 | 6656870 | 536988 | 6582134 | 495970 | -7.64% | -1.12% |
| 02 | PQ8x4 | 4 | 8 | IP | 6228497 | 475320 | 6164081 | 481118 | +1.22% | -1.03% |
| 03 | PQ8x8 | 8 | 8 | L2 | 6626171 | 545230 | 6580586 | 500343 | -8.23% | -0.69% |
| 04 | PQ8x8 | 8 | 8 | IP | 6228452 | 519710 | 6165707 | 479864 | -7.67% | -1.01% |
| 05 | PQ8x16 | 16 | 8 | L2 | 6615305 | 697070 | 6582239 | 624059 | -10.47% | -0.50% |
| 06 | PQ8x16 | 16 | 8 | IP | 6227067 | 642949 | 6164572 | 587271 | -8.66% | -1.00% |
| 07 | PQ8x32 | 32 | 8 | L2 | 6624323 | 1185655 | 6581329 | 902298 | -23.90% | -0.65% |
| 08 | PQ8x32 | 32 | 8 | IP | 6239930 | 1155545 | 6164198 | 785403 | -32.03% | -1.21% |
| 09 | PQ8x64 | 64 | 8 | L2 | 6624196 | 2006275 | 6578815 | 1687191 | -15.90% | -0.69% |
| 10 | PQ8x64 | 64 | 8 | IP | 6225719 | 1800678 | 6164767 | 1512760 | -15.99% | -0.98% |
| 11 | PQ8x96 | 96 | 8 | L2 | 6617525 | 3108377 | 6582946 | 2701758 | -13.08% | -0.52% |
| 12 | PQ8x96 | 96 | 8 | IP | 6239711 | 2657958 | 6170006 | 2320803 | -12.68% | -1.12% |
| 13 | PQ8x384 | 384 | 8 | L2 | 6620602 | 14029275| 6583820 | 12318712| -12.19% | -0.56% |
| 14 | PQ8x384 | 384 | 8 | IP | 6228776 | 12089134| 6166143 | 10509785| -13.06% | -1.01% |
| 15 | PQ8x32 | 32 | 4 | L2 | 3321625 | 773663 | 3298874 | 491225 | -36.51% | -0.68% |
| 16 | PQ8x32 | 32 | 16 | L2 |13223858 | 2184906 |13152640 | 1645444 | -24.69% | -0.54% |
| 17 | PQ8x32 | 32 | 32 | L2 |26430233 | 3405062 |26331426 | 3138563 | -7.83% | -0.37% |
| 18 | PQ8x64 | 64 | 4 | L2 | 3320850 | 1191054 | 3298426 | 969309 | -18.62% | -0.68% |
| 19 | PQ8x64 | 64 | 16 | L2 |13207340 | 3944868 |13149902 | 3252654 | -17.55% | -0.43% |
| 20 | PQ8x64 | 64 | 32 | L2 |26467696 | 6725098 |26335327 | 6667493 | -0.86% | -0.50% |
| 21 | PQ8x16 | 16 | 4 | L2 | 3316496 | 427653 | 3304051 | 367407 | -14.09% | -0.38% |
| 22 | PQ8x16 | 16 | 16 | L2 |13203491 | 1198002 |13150061 | 1121421 | -6.39% | -0.40% |
| 23 | PQ8x16 | 16 | 32 | L2 |26438401 | 2199527 |26280149 | 2143973 | -2.53% | -0.60% |
Δ_PQ%: PQ baseline -> RVV; Δ_F%: Flat baseline -> RVV.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5543
Test Plan:
`buck build fbcode//faiss:faiss` passes.
`buck2 test` passes on the PQ and the distance C++ tests:
`test_disable_pq_sdc_tables`, `test_distances_dispatch`, `test_distances_simd`,
`test_ivfpq_codec`, `test_ivfpq_indexing`, `test_pq_encoding` and
`test_pq_code_distance`. The result is 27 pass and 0 fail. The host is aarch64,
so 8 tests skip. Those tests need a second SIMD level to compare against.
This diff adds 2 tests to `faiss/tests/test_product_quantizer.py`:
- `test_dsub2_4bit` covers `dsub=2` with `nbits=4`. `ksub` is 16, so the SIMD
`dsub2` kernel runs.
- `test_dsub2_2bit` covers `dsub=2` with `nbits=2`. `ksub` is 4, so the generic
path must run instead.
I checked the `nbits >= 3` fix with a temporary C++ test on ARM NEON. The test
compares `compute_distance_tables` and `compute_inner_prod_tables` against
`compute_distance_table` and `compute_inner_prod_table`, for `nbits` of 1, 2, 3
and 4. The test fails with a `ksub % 8 == 0` assertion before the fix. The test
passes after the fix. I removed the temporary test before submit.
Limits of the local verification:
- I did not run `faiss/tests/test_product_quantizer.py` on this devserver. The
`fbcode//faiss/python:pyfaiss-py-gen` SWIG step fails on aarch64 with
`Exec format error`. The failure also reproduces without my changes. CI runs
the Python tests on x86.
- I did not compile `utils/simd_impl/distances_rvv.cpp`. This devserver has no
RISC-V toolchain. `xplat.bzl` registers no RVV sources, so Buck and internal
CI do not build that file either. Only the OSS CMake build on rv64 compiles
it. A reviewer with RVV hardware should confirm the 4 RVV kernel fixes.
Reviewed By: juancarpio27
Differential Revision: D117914440
Pulled By: mnorris11
fbshipit-source-id: 3539a116d47f4a96c1051bc10e6440deae8c5302
|
||
|
Yedidya Feldblum
|
b1c1496545 |
migrate from bit-manipulation builtins in fbcode/faiss (#5667)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5667 Replaces `__builtin_ctz` with `std::countr_zero` in `impl/fast_scan/simd_result_handlers.h`'s per-lane result-scanning loops. `utils/popcount.h` and `impl/platform_macros.h` are portability wrappers (their sole purpose is bridging MSVC/GCC-Clang builtins), so they're left out of scope here. ___ Differential Revision: D121529297 fbshipit-source-id: e42b3fb70eacc5fc4d2e2ffe7dfca64662ef2016 |
||
|
|
f323c18d5a |
Add RVV encoding for QT_8bit_direct (#5640)
Summary: RVV `QT_8bit_direct` currently inherits scalar encoding. This adds an RVV encoder using `e32m4`, explicit round-toward-zero float-to-unsigned conversion, and two narrowing operations before storing bytes. The generic encoder becomes overridable so the RVV specialization can provide the implementation. The portable scalar conversion domain is finite inputs in `(-1, 256)`, whose truncated values are representable in `uint8_t`; negative fractions greater than -1 produce zero. The new regression test covers that domain, including signed zero, subnormals, values around integer boundaries, the largest float below 256, short tails, and output guards. This PR is independent of the FP16 and direct-signed-distance changes. ### Performance Measured on a native SG2044 RISC-V host (VLEN=128), GCC 15.1, Release `-O3`, dynamic dispatch (`FAISS_OPT_LEVEL=dd`), `rv64gcv_zvfhmin/lp64d`, one thread pinned to CPU 2. The baseline is official commit `80a16564f86530dbf0bfaf96c2b71feffeb5093f`; the candidate is that same baseline plus only this kernel change. These measurements were not collected on the newer PR base `2ed4c106e9fb9686e7727e5daf8ad6ad1e164109`. The affected scalar-quantizer source files are unchanged between those bases, and the submitted kernel differs from the measured one only in comments/formatting. | Public path | Dimension | Baseline ns/element | Candidate ns/element | Paired speedup | 95% interval | |---|---:|---:|---:|---:|---:| | Direct-u8 encode | 16 | 1.839 | 1.345 | 1.366x | [1.343, 1.379] | | Direct-u8 encode | 32 | 1.839 | 1.061 | 1.750x | [1.690, 1.775] | | Direct-u8 encode | 128 | 1.640 | 0.823 | 1.937x | [1.901, 1.992] | | Direct-u8 encode | 768 | 1.530 | 0.740 | 2.059x | [1.883, 2.105] | Geometric mean of the dimension-specific speedups at d=32/128/768: Direct-u8 encode **1.911x**. There are three consecutive sessions, each with four alternating ABBA/BAAB blocks per dimension/path: **48 paired blocks** for this candidate. A is the baseline and B is the candidate. Each call is calibrated to at least 0.1 s (the shortest formal call in the full campaign was 0.192 s), with three warm-up batches and `n = max(32, floor(32768/d))`. Each block uses the ratio of the two-call geometric mean times. Reported speedup is the median of the three session medians; intervals use 5,000 hierarchical bootstrap resamples, resampling sessions and then blocks within each ABBA/BAAB order stratum. The timing columns are separate medians, so their quotient need not equal the paired speedup. All 48 paired blocks favored this candidate. The timed operation is public `ScalarQuantizer::compute_codes`, using fixed-seed (718) finite input batches. Direct-u8 input values are `float(rng()%16777216)/65536.f`. These results describe public encoding/distance throughput on one non-exclusive host, not end-to-end ANN search speedup. The intervals describe these sessions only, with no multiple-comparison correction; they do not establish portability across machines or vector lengths. ### Validation - Native public encoding oracle: **7,724,670** legal-domain input elements across the five RISC-V rounding modes; zero differing output bytes or write-guard failures. Inputs include negative fractions, signed zeros, subnormals, all byte-value boundaries, and values just below 256. - Original benchmark baseline full C++ suite: **273 passed, 7 skipped, 0 failed**. - Submission version on `2ed4c106e9fb9686e7727e5daf8ad6ad1e164109`: all three independent candidate builds succeeded; this PR's focused C++ suite (`NONE`: 11 passed, 5 skipped, 0 failed; `RISCV_RVV`: 12 passed, 4 skipped, 0 failed) and the public-path oracle above pass on native RISC-V. The newly added RVV regression test is executed and passed in the RVV run, and skipped when RVV is disabled in the NONE run. - All touched C++ files pass clang-format 21.1.8; `git diff --check` clean. ### Notes - Touches three files: `faiss/impl/scalar_quantizer/quantizers.h` (one-line `final` → `override` so the generic encoder can be specialized), `faiss/impl/scalar_quantizer/sq-rvv.cpp` (the RVV encoder), and `tests/test_scalar_quantizer.cpp` (one regression test). - The conversion uses `vfcvt.rtz.xu.f.v` (round toward zero, float to unsigned) rather than a signed conversion. The generic encoder is `code[i] = (uint8_t)x[i]`, which is a C++ float-to-unsigned-integral truncation, so round-toward-zero is the matching mode. Negative fractions greater than -1 truncate to 0 in both paths. - Two narrowing shifts (`vnsrl` by 0) are used to go `u32m4 → u16m2 → u8m1` before the byte store, avoiding a separate saturating narrow instruction. - The kernel is VLEN-agnostic: `e32m4` lanes with `vsetvl`-driven loop bounds. At VLEN=128 this processes 16 elements per iteration. - Quantization is not clamped. Inputs outside `(-1, 256)` are outside the portable domain — the scalar path casts out-of-range floats to `uint8_t`, which is undefined behavior for values outside that range, so no portable contract exists to preserve there. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5640 Reviewed By: trang-nm-nguyen Differential Revision: D121445607 Pulled By: mnorris11 fbshipit-source-id: 16f34cf77aa0cac2fd99ccb245f2ffbc2047d6c4 Co-authored-by: ihb2032 <[email protected]> Co-authored-by: lyd1992 <[email protected]> Co-authored-by: Yuansheng <[email protected]> |
||
|
Lucian Grijincu
|
2496f63f76 |
Avoid search stats contention during concurrent search (#5663)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5663 Follow-up to D121097648 (public PR: https://github.com/facebookresearch/faiss/pull/5657). Related public PR: https://github.com/facebookresearch/faiss/pull/4910 Use relaxed atomic additions for HNSW, RaBitQ, and IVFPQ stats aggregation. Builds with `std::atomic_ref` support use `fetch_add(..., std::memory_order_relaxed)`; older standard libraries fall back to `__atomic_fetch_add(..., __ATOMIC_RELAXED)`. This removes the global OpenMP critical sections that serialize concurrent searches while preserving exact counter updates. Benchmark design: mode/opt `bench_hnsw_stats` binary, one shared HNSW index, `N` external pthreads, one OpenMP thread per search (each worker calls `omp_set_num_threads(1)` once before its timed loop), five repetitions per point. Performance: | External pthreads | Critical M/s | Relaxed atomic M/s | Speedup | |---:|---:|---:|---:| | 1 | 2.228 | 2.503 | 1.12x | | 2 | 3.716 | 3.853 | 1.04x | | 4 | 5.662 | 7.357 | 1.30x | | 8 | 6.384 | 12.843 | 2.01x | | 16 | 2.697 | 15.804 | 5.86x | | 32 | 0.951 | 18.910 | 19.88x | | 64 | 0.117 | 21.866 | 186.89x | At 64 pthreads, the feature-detected `atomic_ref` path measured 21.918 M/s mean over 10 repetitions (21.907 M/s median, 2.59% CV). The `__atomic_fetch_add` fallback measured 22.563 M/s mean (22.511 M/s median, 3.24% CV). The 64-thread profile attributes 97.38% of cycles in the critical-section version to lock acquisition and waiting. With relaxed atomic aggregation, 74.14% is in the stats combine path instead. The remaining non-linear scaling, 21.866 M/s at 64 threads versus the 2.503 M/s single-thread baseline, comes from cache-line contention on the global counters and shared-index memory-bandwidth and cache pressure. Reviewed By: mnorris11 Differential Revision: D121368933 fbshipit-source-id: 8dd26313e55aa73c0cdd379c7648efacd4f5f1d8 |
||
|
ihb2032
|
5e74b1bdde |
feat: implement RVV optimized scalar quantizer codecs and distance computation (#5535)
Summary:
This PR adds RISC-V Vector Extension SIMD support for Faiss scalar quantizer.
The implementation introduces runtime vector-length aware RVV kernels using the m8 vector configuration, instead of assuming a fixed vector width. This matches the variable-length nature of RVV hardware and allows the implementation to scale across different RISC-V vector implementations.
## Changes
- Add RVV implementations for scalar quantizer codecs:
- 8-bit codec decoding
- 4-bit codec decoding
- (6-bit stays scalar: the RVV gather decoder measured 12-22% slower
than the scalar path, so QT_6bit falls back until a faster decoder
exists)
- Add RVV quantizer reconstruction support for:
- Uniform quantizers
- Non-uniform quantizers
- FP16 (Zvfhmin is the enforced minimum ISA of the RISCV_RVV level; a
build without it fails compilation rather than silently falling back)
- BF16
- Direct 8-bit quantization
- Signed 8-bit quantization
- LloydMax quantizers (1/2/3/4/8-bit)
- Add RVV optimized similarity implementations:
- L2 distance
- Inner product
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5535
Reviewed By: mnorris11
Differential Revision: D116996553
Pulled By: juancarpio27
fbshipit-source-id: dc726948fa602a04a757d8f9cab8eea41ac1979a
|
||
|
Jure Bajic
|
02cacc5293 |
Fix TSAN races in side hsnw_stats and rabitq_stats (#5657)
Summary:
These TSAN races can be detected when running a search in parallel over the IVF_HNSW index. This PR just wraps the updates of these inside `omp_critical` points.
```
WARNING: ThreadSanitizer: data race (pid=485)
Write of size 8 at 0xaaaadb3db920 by thread T1443:
#0 faiss::IndexHNSW::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/exception_ptr.h (arangod+0x7b95d44) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
Previous write of size 8 at 0xaaaadb3db920 by thread T59 (mutexes: write M0):
#0 faiss::IndexHNSW::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/exception_ptr.h (arangod+0x7b95d44) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
https://github.com/facebookresearch/faiss/issues/1 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const::$_0::operator()(long, float const*, float*, long*, faiss::IndexIVFStats*) const /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:351 (arangod+0x7ae5c04) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
https://github.com/facebookresearch/faiss/issues/2 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const (.omp_outlined) /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:391 (arangod+0x7ae5ae8) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
https://github.com/facebookresearch/faiss/issues/3 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:384 (arangod+0x7ade660) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
https://github.com/facebookresearch/faiss/issues/4 faiss::IndexIVF::search(long, float const*, long, float*, long*, faiss::SearchParameters const*) const /root/project/3rdParty/faiss/faiss/IndexIVF.cpp:384 (arangod+0x7ade624) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
Location is global 'faiss::hnsw_stats' of size 32 at 0xaaaadb3db920 (arangod+0xa5eb920)
Mutex M0 (0xffff39ed9968) created at:
#0 pthread_mutex_lock <null> (arangod+0x2414098) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
https://github.com/facebookresearch/faiss/issues/1 __gthread_mutex_lock /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/aarch64-linux-gnu/c++/13/bits/gthr-default.h:749 (arangod+0x3820b80) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
https://github.com/facebookresearch/faiss/issues/2 lock /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/std_mutex.h:113 (arangod+0x3820b80)
https://github.com/facebookresearch/faiss/issues/3 lock_guard /usr/lib/gcc/aarch64-linux-gnu/13/../../../../include/c++/13/bits/std_mutex.h:249 (arangod+0x3820b80)
Thread T1443 (tid=6126, running) created by thread T138 at:
#0 pthread_create <null> (arangod+0x2411d9c) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
https://github.com/facebookresearch/faiss/issues/1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State>>, void (*)()) <null> (libstdc++.so.6+0xe1c64) (BuildId: 7c239d1743727054caa3df1ab866bfa72cd4dd93)
Thread T59 'SchedWorker' (tid=812, running) created by thread T56 at:
#0 pthread_create <null> (arangod+0x2411d9c) (BuildId: 6c4d32ad66c51d1f6946b33e752288fa07e2d2c8)
```
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5657
Reviewed By: alibeklfc
Differential Revision: D121097648
Pulled By: mnorris11
fbshipit-source-id: 55455e7dab133360d4aa193f3853e3ae12f2f881
|
||
|
anthony-zy
|
bf8be4a79b |
fix(ci): use GCC 14 riscv64 cross-compiler for RVV tuple/zvfhmin intrinsics (#5653)
Summary: fix(ci): use GCC 14 riscv64 cross-compiler for RVV tuple/zvfhmin intrinsics The linux-riscv64-DD CI job installs the default cross-compiler from Ubuntu noble, which resolves to GCC 13.3. GCC 13's <riscv_vector.h> does not export the RVV tuple types (vuint8m1x3_t) nor the zvfhmin f16 intrinsics (vfloat16m1_t, __riscv_vle16_v_f16m1, __riscv_vfwcvt_f_f_v_f32m2) used by the SQ RVV kernels, so building sq-rvv.cpp failed with "'vuint8m1x3_t' was not declared in this scope" (and similar) in the Codec6bit software-pipelined loops and the QuantizerFP16 fast paths. The intrinsics used are all legal under -march=rv64gcv_zvfhmin (only vle16/vse16/vfwcvt/vfncvt on f16) and are fully supported starting with GCC 14, so no source changes are needed. Explicitly install gcc-14-riscv64-linux-gnu / g++-14-riscv64-linux-gnu and point the toolchain file at the versioned binaries, avoiding any reliance on update-alternatives defaults. A version self-check in the workflow makes the resolved compiler visible in the CI log. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5653 Reviewed By: alibeklfc Differential Revision: D120734761 Pulled By: mnorris11 fbshipit-source-id: d82cf2c99dfd6e8720f31bdadb718806f9e71044 |
||
|
Alibek Zhakubayev
|
98af2cf627 |
Fix osx-arm64 conda build: drop redundant openblas pin, allow single-platform release dispatch (#5652)
Summary: The `OSX arm64 packages` job failed in the v1.15.1 release run ([35131196327](https://github.com/facebookresearch/faiss/actions/runs/35131196327)), so `faiss-cpu` 1.15.1 is on the pytorch channel for linux-64, linux-aarch64 and win-64 but **not** osx-arm64. PyPI is unaffected. ### Root cause Upstream conda-forge inconsistency, not a faiss regression. On 2026-09-10 conda-forge published two new `libopenblas 0.3.34` builds for osx-arm64 without matching `openblas 0.3.34` metapackages: | `openblas 0.3.34` accepts | `libopenblas 0.3.34` available | |---|---| | `openmp_he657e61_0` | `openmp_he657e61_0` (07-17) | | `pthreads_hddb8425_0` | `pthreads_hddb8425_0` (07-17) | | `pthreads_h60d1960_1` | `pthreads_h60d1960_1` (08-30) | | `openmp_h4f80526_1` | `openmp_h4f80526_1` (08-30) | | — | `pthreads_hb42d564_1` (09-10) ← no `openblas` | | — | `openmp_h5e6e99c_1` (09-10) ← no `openblas` | `libfaiss` built fine against `openmp_h4f80526_1`. The failure came when solving the `faiss-cpu` env: the unpinned `libopenblas =0.3.34` resolves to the newest build `openmp_h5e6e99c_1`, which no `openblas 0.3.34` accepts: ``` package libfaiss-1.15.1-py3.12_hcb8d3e5_0_cpu requires openblas 0.3.34.*, but none of the providers can be installed ``` The recipe is vulnerable because the `libfaiss` output pins **both** `openblas` and `libopenblas` on osx-arm64 — `# [not x86_64]` was written for linux-aarch64 but also matches macOS ARM. The `faiss-cpu` output in the same file already pins only `libopenblas` on osx. ### Changes 1. **`conda/faiss/meta.yaml`** (lines 72, 86) — narrow the selector to `# [linux and not x86_64]`, so osx-arm64 constrains only `libopenblas` (what `libfaiss` actually links: `libopenblas.0.dylib`). linux-aarch64 and x86_64 behavior unchanged. 2. **`.github/workflows/build-release.yml`** — add `workflow_dispatch` with a `platforms` choice input. The workflow was `workflow_call`-only, so a single failed conda leg could not be rebuilt without pushing a tag. Each job gains an `if:` that is a no-op on tag pushes (`github.event_name` is `push`), preserving current release behavior. ### Recovery plan for 1.15.1 After this lands, dispatch `build-release.yml` from `main` with `platforms: osx-arm64`. The conda version comes from `git describe`, so this produces **1.15.1 build 1** — same version, next build number, no retagging and no PyPI involvement. ### Note If conda-forge publishes the missing `openblas` builds, the old recipe would start working again on its own. This change removes the coupling so the build no longer depends on that. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5652 Test Plan: - [ ] CI passes - [ ] Dispatch `build-release.yml` with `platforms: osx-arm64` and confirm the conda solve succeeds - [ ] Confirm `faiss-cpu` 1.15.1 appears for osx-arm64 on the pytorch channel Reviewed By: mnorris11 Differential Revision: D120400090 Pulled By: alibeklfc fbshipit-source-id: 0c27e8244bfe25bea4a45466e4e2376c04e0c1f8 |
||
|
Mulugeta Mammo
|
b0d9e80288 |
Add fast scan variants of SQ indexes - IndexSQFastScan and IndexIVFSQFastScan (#5341)
Summary:
This PR introduces `IndexSQFastScan` and `IndexIVFSQFastScan`, fast scan variants of scalar quantized indexes.
**IndexSQFastScan** (33f2cb639bd7a888b2ab7578083b73753c2fd3d3)
Introduces `IndexSQFastScan`, a fast scan variant of `IndexScalarQuantizer` that leverages the PQ4 FastScan SIMD infrastructure (`vpshufb`) for accelerated search over scalar-quantized codes.
Three code paths depending on quantizer type:
- Native 4-bit (`QT_4bit`, `QT_4bit_uniform`): codes map directly onto the 16-level PQ4 fast scan. ~11x speedup, no precision loss.
- Rerank (`QT_6bit`, `QT_8bit`, `QT_8bit_uniform`, `QT_8bit_direct`, `QT_8bit_direct_signed`): re-quantize to 4-bit for a coarse fast scan first pass, then rerank top-k*rerank_factor candidates with exact original-precision distances. ~7-9x speedup, near-zero precision loss at rerank_factor=4.
- Fallback (`QT_fp16`, `QT_bf16`, `TurboQuant`, etc.): delegates to the ScalarQuantizer's own SIMD-optimized scanner. Same speed as IndexScalarQuantizer — no fast scan benefit, but provides a unified interface for all quantizer types.
**benchmark**: `benchs/bench_index_sq_fastscan.py`
```text
+---------------------+----------------------+---------+---------+--------+--------+
| Category | QType | Speedup | ms/q SQ | R@1 | R@10 |
+---------------------+----------------------+---------+---------+--------+--------+
| Native 4-bit (SIMD) | QT_4bit | 11.9x | 0.240 | 0.7714 | 0.9985 |
| | QT_4bit_uniform | 11.7x | 0.235 | 0.7180 | 0.9952 |
+---------------------+----------------------+----------+---------+--------+--------+
| Rerank (rf=4) | QT_6bit | 9.3x | 0.202 | 0.9371 | 1.0000 |
| | QT_8bit | 8.4x | 0.173 | 0.9783 | 1.0000 |
| | QT_8bit_uniform | 6.7x | 0.138 | 0.9728 | 1.0000 |
| | QT_8bit_direct | 4.7x | 0.101 | 0.9914 | 1.0000 |
| | QT_8bit_direct_signed| 4.9x | 0.101 | 0.5999 | 0.6591 |
+---------------------+----------------------+----------+---------+--------+--------+
| Fallback (~1.0x) | QT_fp16 | 0.9x | 0.373 | 0.9914 | 1.0000 |
| | QT_bf16 | 1.0x | 0.365 | 0.9914 | 1.0000 |
| | QT_8bit_tqmse | 1.0x | 0.261 | 0.6033 | 0.9561 |
| | QT_5bit_tq | 1.0x | 3.013 | 0.2680 | 0.6248 |
| | QT_4bit_tq | 1.0x | 2.416 | 0.1511 | 0.3994 |
| | QT_3bit_tq | 1.0x | 0.411 | 0.0487 | 0.1691 |
| | QT_2bit_tq | 1.1x | 0.231 | 0.0376 | 0.1532 |
| | QT_4bit_tqmse | 1.0x | 0.290 | 0.0012 | 0.0064 |
| | QT_3bit_tqmse | 1.0x | 0.345 | 0.0000 | 0.0001 |
| | QT_2bit_tqmse | 1.0x | 0.266 | 0.0000 | 0.0001 |
| | QT_1bit_tqmse | 1.0x | 0.280 | 0.0000 | 0.0001 |
+---------------------+----------------------+----------+---------+--------+--------+
Key: Speedup = IndexScalarQuantizer ms / IndexSQFastScan ms
R@1, R@10 = IndexSQFastScan recall (matches SQ for all fallback types)
Rerank shows rf=4 (rf=2 and rf=8 yield same recall, ~0.1x speedup variation)
Dataset: SIFT1M, k=32
```
**IndexIVFSQFastScan** (ebfa072816df2fbd80a6b463a913866f862587f6)
Similar to the above, this commit introduces `IndexIVFSQFastScan`, an IVF variant of scalar quantized fast scan that extends the PQ4 FastScan SIMD infrastructure (vpshufb) to inverted-list indexes with scalar quantization. Uses BlockInvertedLists for packed 4-bit codes scanned via SIMD. Will continue to update the PR until it's feature complete.
Three code paths depending on quantizer type:
- Native 4-bit (QT_4bit, QT_4bit_uniform): codes pack directly into the SIMD block layout with no precision loss. ~12x speedup.
- Rerank (QT_6bit, QT_8bit, QT_8bit_uniform, QT_8bit_direct, QT_8bit_direct_signed): re-quantized to 4-bit for the SIMD coarse pass, then reranked using exact original-precision codes stored in a parallel ArrayInvertedLists. ~8-10x speedup, near-zero precision loss at rerank_factor=2.
- Fallback (QT_fp16, QT_bf16, TurboQuant, etc.): delegates to IndexIVF's InvertedListScanner path. Same behavior as IndexIVFScalarQuantizer — no fast scan benefit, but provides a unified interface for all quantizer types.
**benchmark (SIFT1M, d=128, k=32, SapphireRapids)**: `benchs/bench_index_ivf_sq_fastscan.py`
```text
+---------------------+----------------------+--------+-------------------------------------------+
| Category | QType | nprobe | Speedup | ms/q SQ | R@1 FS | R@10 FS |
+---------------------+----------------------+--------+---------+---------+--------+-------------+
| Native 4-bit (SIMD) | QT_4bit | 1 | 7.1x | 0.007 | 0.4582 | 0.5456 |
| | | 4 | 10.4x | 0.007 | 0.6903 | 0.8667 |
| | | 16 | 12.9x | 0.022 | 0.7668 | 0.9885 |
| | | 64 | 12.1x | 0.070 | 0.7725 | 0.9983 |
| | | 256 | 11.3x | 0.230 | 0.7725 | 0.9983 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_4bit_uniform | 1 | 15.8x | 0.006 | 0.4397 | 0.5449 |
| | | 4 | 10.5x | 0.007 | 0.6419 | 0.8650 |
| | | 16 | 10.4x | 0.018 | 0.7113 | 0.9855 |
| | | 64 | 10.9x | 0.063 | 0.7162 | 0.9951 |
| | | 256 | 11.0x | 0.222 | 0.7162 | 0.9951 |
+---------------------+----------------------+--------+---------+---------+--------+-------------+
| Rerank (rf=4) | QT_6bit | 1 | 4.9x | 0.007 | 0.4582 | 0.5456 |
| | | 4 | 4.8x | 0.007 | 0.6903 | 0.8667 |
| | | 16 | 7.9x | 0.018 | 0.7668 | 0.9885 |
| | | 64 | 10.4x | 0.067 | 0.7725 | 0.9983 |
| | | 256 | 10.2x | 0.221 | 0.7725 | 0.9983 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_8bit | 1 | 17.1x | 0.007 | 0.4582 | 0.5456 |
| | | 4 | 12.5x | 0.008 | 0.6903 | 0.8667 |
| | | 16 | 8.9x | 0.015 | 0.7668 | 0.9885 |
| | | 64 | 8.4x | 0.049 | 0.7725 | 0.9983 |
| | | 256 | 9.3x | 0.188 | 0.7725 | 0.9983 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_8bit_uniform | 1 | 18.2x | 0.007 | 0.4397 | 0.5449 |
| | | 4 | 11.1x | 0.008 | 0.6419 | 0.8650 |
| | | 16 | 8.3x | 0.014 | 0.7113 | 0.9855 |
| | | 64 | 7.8x | 0.045 | 0.7162 | 0.9951 |
| | | 256 | 7.8x | 0.164 | 0.7162 | 0.9951 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_8bit_direct | 1 | 8.6x | 0.007 | 0.4083 | 0.5426 |
| | | 4 | 7.0x | 0.008 | 0.5987 | 0.8587 |
| | | 16 | 6.8x | 0.013 | 0.6550 | 0.9756 |
| | | 64 | 6.1x | 0.035 | 0.6581 | 0.9850 |
| | | 256 | 7.4x | 0.150 | 0.6581 | 0.9850 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_8bit_direct_signed| 1 | 16.5x | 0.006 | 0.4257 | 0.5447 |
| | | 4 | 10.1x | 0.007 | 0.6286 | 0.8633 |
| | | 16 | 7.1x | 0.012 | 0.6919 | 0.9831 |
| | | 64 | 5.7x | 0.033 | 0.6963 | 0.9923 |
| | | 256 | 7.1x | 0.145 | 0.6963 | 0.9923 |
+---------------------+----------------------+--------+---------+---------+--------+-------------+
| Fallback (~1.0x) | QT_fp16 | 1 | 1.0x | 0.007 | 0.5434 | 0.5462 |
| | | 4 | 1.1x | 0.013 | 0.8607 | 0.8678 |
| | | 16 | 0.9x | 0.035 | 0.9814 | 0.9902 |
| | | 64 | 0.9x | 0.115 | 0.9912 | 1.0000 |
| | | 256 | 0.9x | 0.481 | 0.9912 | 1.0000 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_bf16 | 1 | 1.0x | 0.008 | 0.5427 | 0.5462 |
| | | 4 | 1.1x | 0.013 | 0.8581 | 0.8678 |
| | | 16 | 1.0x | 0.035 | 0.9778 | 0.9902 |
| | | 64 | 1.0x | 0.115 | 0.9876 | 1.0000 |
| | | 256 | 1.0x | 0.487 | 0.9876 | 1.0000 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_5bit_tq | 1 | 1.0x | 0.016 | 0.1866 | 0.3827 |
| | | 64 | 1.0x | 0.794 | 0.2680 | 0.6248 |
| | | 256 | 1.0x | 2.885 | 0.2680 | 0.6248 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_4bit_tq | 1 | 1.0x | 0.013 | 0.1182 | 0.2841 |
| | | 64 | 0.9x | 0.618 | 0.1511 | 0.3994 |
| | | 256 | 0.9x | 2.298 | 0.1511 | 0.3994 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_3bit_tq | 1 | 1.1x | 0.008 | 0.0526 | 0.1599 |
| | | 64 | 0.8x | 0.092 | 0.0488 | 0.1696 |
| | | 256 | 0.8x | 0.357 | 0.0487 | 0.1691 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_2bit_tq | 1 | 1.2x | 0.008 | 0.0450 | 0.1526 |
| | | 64 | 1.1x | 0.072 | 0.0377 | 0.1534 |
| | | 256 | 1.1x | 0.224 | 0.0376 | 0.1532 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_4bit_tqmse | 1 | 1.0x | 0.007 | 0.0619 | 0.1851 |
| | | 64 | 1.0x | 0.072 | 0.0002 | 0.0010 |
| | | 256 | 0.9x | 0.275 | 0.0000 | 0.0000 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_3bit_tqmse | 1 | 1.0x | 0.007 | 0.0621 | 0.1852 |
| | | 64 | 1.0x | 0.090 | 0.0006 | 0.0032 |
| | | 256 | 1.0x | 0.350 | 0.0000 | 0.0000 |
| +----------------------+--------+---------+---------+--------+-------------+
| | QT_2bit_tqmse | 1 | 1.1x | 0.007 | 0.0621 | 0.1852 |
| | | 64 | 1.0x | 0.068 | 0.0015 | 0.0080 |
| | | 256 | 1.0x | 0.244 | 0.0000 | 0.0000 |
+---------------------+----------------------+--------+---------+---------+--------+-------------+
Key: Speedup = IndexIVFScalarQuantizer ms / IndexIVFSQFastScan ms
R@1, R@10 = IVFSQFastScan recall (matches IVF-SQ for all fallback types)
Rerank shows rf=4 only (rf=2,8 yield similar results)
Fallback shows nprobe=1,64,256 only (abbreviated for low-recall types)
Dataset: SIFT1M, nlist=256, k=32
```
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5341
Reviewed By: junjieqi
Differential Revision: D117259284
Pulled By: mnorris11
fbshipit-source-id: 41455097ff3ae1285aeb4cf88e07abc5b5b13171
|
||
|
Mihai Capotă
|
4864f4d420 |
SVS: Add ParameterSpace.set_index_parameter support for Vamana (#5644)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5644 Reviewed By: alibeklfc Differential Revision: D120290334 Pulled By: mnorris11 fbshipit-source-id: c947484d0704789553a7d4d80a0ef7185e59962a |
||
|
Aaryan Mehta
|
3f4838ead3 |
Batch the IVF ScalarQuantizer scan four codes at a time (#5456)
Summary:
The IVF ScalarQuantizer inverted-list scan distances one code at a time through `run_scan_codes1`, even though the SQ distance computers already expose a four-at-a-time kernel, `query_to_codes_batch_4` (the one HNSW uses through `distances_batch_4`). This adds `run_scan_codes4`, a batched inner loop that distances four consecutive codes per step and then applies the same threshold compare, `add_result`, and threshold refresh as `run_scan_codes1`, in id order, so the sequence of heap updates is identical. The two SQ scanners route their no-selector path through it; the selector path stays on `run_scan_codes1`.
Four independent accumulator chains let the out-of-order core overlap the latency-bound reduction that a single code cannot hide. Results are unchanged: the batched kernel reconstructs each code, which is bit-identical to the single-code path for every quantizer except the uniform ones, whose single-code path predecodes the query, so a compile-time check keeps those on the scalar scan. Verified bit-identical distances and ids against the scalar scan for QT_8bit, QT_8bit_uniform, QT_6bit, QT_4bit, QT_fp16, and QT_8bit_direct, in both metrics, at d in {64, 128, 256}.
Measured on AVX2, end-to-end `IndexIVFScalarQuantizer` search of QT_8bit, single thread, batched against the scalar scan:
| metric | d | scalar | batched | speedup |
| --- | --- | --- | --- | --- |
| L2 | 128 | 60.4 ms | 40.3 ms | 1.50x |
| L2 | 256 | 170.9 ms | 105.7 ms | 1.62x |
| L2 | 256, out of cache | 403.5 ms | 262.3 ms | 1.54x |
| IP | 128 | 46.5 ms | 38.4 ms | 1.21x |
| IP | 256 | 108.7 ms | 83.8 ms | 1.30x |
The L2 gain is the larger one, around 1.5-1.6x, and it holds when the index spills out of cache. Inner product gains less since its shorter chain is already partly overlapped by the core. When batching does not help it matches the scalar scan, so there is no regression.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5456
Reviewed By: junjieqi
Differential Revision: D118857126
Pulled By: mnorris11
fbshipit-source-id: df7b02c2ede8699c599956370b87a949f1b7db9d
|
||
|
郑紫阳10295048
|
0f23e6a2ef |
Add RVV SIMD specialization for scalar quantizer distance compute (#5539)
Summary: This PR adds RISC-V Vector Extension SIMD specialization for the scalar quantizer distance computation, following the codec/quantizer/similarity specialization already present for other SIMD levels. The kernels use the runtime vector-length aware m8 vector configuration and are shared across all RVV hardware widths, instead of assuming a fixed vector length. Changes Add RVV specialized L2 / inner-product distance kernels for the scalar quantizer codecs: - QT_4bit / QT_4bit_uniform - QT_6bit - QT_8bit / QT_8bit_uniform - QT_8bit_direct / QT_8bit_direct_signed - QT_bf16 - QT_fp16 Each kernel reuses the reconstructed (quantizer) L2 / IP formulation that avoids per-dimension reconstruction, keeping the hot loop in the integer domain and applying the final scale at the tail. Performance Cmd: 1M 768 <SQ> 32 100 16 1 1000 1 <metric> (search ms/query, lower is better) Δ_SQ%: SQ scalar baseline -> RVV; Δ_F%: Faiss flat baseline -> RVV. | 序号 | QT_TYPE | Metric | F1(ms) | SQ1(ms) | F2(ms) | SQ2(ms) | Δ_SQ% | Δ_F% | |------|-----------------------|--------|--------|---------|--------|---------|---------|--------| | 01 | QT_4bit | L2 | 1.841 | 6.231 | 1.824 | 1.450 | -76.73% | -0.92% | | 02 | QT_4bit | IP | 1.652 | 6.086 | 1.674 | 1.273 | -79.08% | +1.33% | | 03 | QT_4bit_uniform | L2 | 1.806 | 2.560 | 1.806 | 1.135 | -55.66% | +0.00% | | 04 | QT_4bit_uniform | IP | 1.612 | 6.007 | 1.597 | 1.244 | -79.29% | -0.93% | | 05 | QT_6bit | L2 | 1.801 | 5.701 | 1.805 | 1.500 | -73.69% | +0.22% | | 06 | QT_6bit | IP | 1.608 | 5.706 | 1.619 | 1.249 | -78.11% | +0.68% | | 07 | QT_8bit | L2 | 1.821 | 3.736 | 1.907 | 1.394 | -62.69% | +4.72% | | 08 | QT_8bit | IP | 1.704 | 3.378 | 1.730 | 1.277 | -62.20% | +1.53% | | 09 | QT_8bit_direct | L2 | 1.977 | 0.280 | 1.947 | 0.054 | -80.71% | -1.52% | | 10 | QT_8bit_direct | IP | 1.709 | 0.458 | 1.740 | 0.099 | -78.38% | +1.81% | | 11 | QT_8bit_direct_signed | L2 | 1.581 | 3.234 | 1.572 | 0.761 | -76.47% | -0.57% | | 12 | QT_8bit_direct_signed | IP | 1.740 | 0.414 | 1.748 | 0.148 | -64.25% | +0.46% | | 13 | QT_8bit_uniform | L2 | 1.943 | 3.204 | 1.928 | 0.927 | -71.07% | -0.77% | | 14 | QT_8bit_uniform | IP | 1.727 | 3.121 | 1.744 | 0.947 | -69.66% | +0.98% | | 15 | QT_bf16 | L2 | 1.862 | 2.889 | 1.894 | 1.466 | -49.26% | +1.72% | | 16 | QT_bf16 | IP | 1.619 | 2.846 | 1.793 | 1.204 | -57.70% | +10.75% | | 17 | QT_fp16 | L2 | 2.011 | 5.691 | 1.950 | 1.570 | -72.41% | -3.03% | | 18 | QT_fp16 | IP | 1.797 | 5.584 | 1.790 | 1.189 | -78.71% | -0.39% | Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5539 Reviewed By: mnorris11 Differential Revision: D117914328 Pulled By: juancarpio27 fbshipit-source-id: 879863b9ab4da6d1c8685892ccc3d64190975181 |
||
|
YingzuoLiu
|
3778f5892d |
Fix IDSelector handling in unbounded HNSW search (#5508)
Summary: - apply `SearchParametersHNSW.sel` to the unbounded HNSW search path - keep filtered nodes available for graph traversal while excluding them from the result set - add regression coverage for L2 and inner-product search, restrictive `efSearch`, and an empty selector ## Root cause The bounded HNSW path applies the selector through its result handler. The unbounded path did not receive the selector and used the same heap for traversal pruning and final results, so IDs rejected by `SearchParameters.sel` could be returned. ## Implementation When a selector is present, the unbounded path maintains a separate result heap containing only accepted IDs. The existing candidate heap remains unchanged and can still contain rejected nodes, preserving traversal through them. When no selector is present, the existing unfiltered result behavior is preserved. ## Validation - issue reproducer: unbounded search now returns ID 1 at distance 100.0 instead of rejected ID 0 - `tests/test_search_params.py`: 52 passed - HNSW C++ tests: 14 passed - full CPU C++ test suite: 278 passed, 5 skipped - clang-format and `git diff --check`: passed Fixes https://github.com/facebookresearch/faiss/issues/5504 Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5508 Reviewed By: alibeklfc Differential Revision: D120364656 Pulled By: mnorris11 fbshipit-source-id: 074234b80cf7b2e06a85d57489a1c214dfd3195b |
||
|
Alibek Zhakubayev
|
1dbb3d25b0 |
Update backward compatibility test to v1.15.1 (#5646)
Summary: - Update backward compatibility test to use faiss-cpu=1.15.1 Note: the test was previously pinned to faiss-cpu=1.14.3, so this skips over 1.15.0. **Do not land until faiss-cpu=1.15.1 is available on the pytorch conda channel** — the conda package is published by the tag-triggered release build. Landing earlier will break this workflow on main. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5646 Test Plan: - [ ] CI passes - [ ] Backward compatibility test uses correct version Reviewed By: mnorris11 Differential Revision: D120265026 Pulled By: alibeklfc fbshipit-source-id: 9459958af352716d5aef25e45043444105f22ff1 |
||
|
Alibek Zhakubayev
|
75c755c5a1 |
Add missing CHANGELOG entry for #5632 in v1.15.1 (#5651)
Summary: - https://github.com/facebookresearch/faiss/issues/5632 (Add explicit IVFPQ scanner distance modes) landed between the changelog generation and the version bump in https://github.com/facebookresearch/faiss/issues/5645, so it ships in 1.15.1 but was missing from `CHANGELOG.md`. - Adds it under `Added` in the 1.15.1 section. Changelog-only; no code changes. **Time-sensitive:** the v1.15.1 tag will be cut from the latest `main`. Landing this before tagging gets the entry into the released changelog. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5651 Test Plan: - [ ] CI passes (docs-only change) Reviewed By: mnorris11 Differential Revision: D120360726 Pulled By: alibeklfc fbshipit-source-id: 2854a7c44958ec4528ec87804382c312ea195ad2v1.15.1 |
||
|
Alibek Zhakubayev
|
d63faeaf5a |
Increment to next release, v1.15.1 (#5645)
Summary: - Increment version to 1.15.1 - Update CHANGELOG.md with commits since v1.15.0 (57 commits) - Bump version in CMakeLists.txt, setup.py, Index.h, INSTALL.md Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5645 Test Plan: - [ ] CI passes - [ ] CHANGELOG entries are correctly categorized Reviewed By: mnorris11 Differential Revision: D120264788 Pulled By: alibeklfc fbshipit-source-id: d95d68c9eb0c7c5b06aaeab7d8d0015b99d23145 |
||
|
Jaeyoon Jeong
|
3b3711c112 |
Add explicit IVFPQ scanner distance modes (#5632)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5632 Add an explicit IVFPQ scanner mode that preserves precomputed-table behavior by default and can decode PQ codes to compute residual L2 distances on demand. Keep the existing scanner factory overload compatible. Reviewed By: mdouze Differential Revision: D119608525 fbshipit-source-id: b60b2c9771bdfd95d9bee2e81b72c9c3e7abb82c |
||
|
Alibek Zhakubayev
|
3bb48643ab |
Pin faiss-gpu-cuvs RAPIDS deps to the 26.06 build series (#5643)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5643 `faiss-gpu-cuvs` 1.15.0 fails at import on a clean install: ``` ImportError: libfaiss.so: undefined symbol: _ZTIN3rmm9bad_allocE ``` ## Root cause, measured RAPIDS 26.08 wrapped RMM in a per-release ABI inline namespace. `nm -D` on the published `librmm-cu13` wheels: | Version | `typeinfo for rmm::bad_alloc` | Exported instead | | --- | --- | --- | | 26.2.0 | present | | | 26.4.0 | present | | | 26.6.0 | present | | | 26.8.0 | **absent** | `typeinfo for rmm::_RMM_26_8::bad_alloc` | `pyproject-gpu-cuvs.toml` builds against `==26.06.*` in `before-all` but declares `>=26.06,<27` at runtime and in `test-requires`. A fresh install therefore pairs a 26.06-built `libfaiss.so` with a 26.8 `librmm.so` that renamed every `rmm::` symbol. Nothing vendored covers the gap: `repair-wheel-command` hands every RAPIDS `lib*.so*` under `/opt/cuvs` to `auditwheel --exclude`. The 1.15.0 wheel was uploaded 2026-08-03, two days before RAPIDS 26.08 reached PyPI. It published green and began failing for every new installer afterwards, which matches `build-pip-gpu / Build & test faiss-gpu-cuvs` going red on 2026-08-05 after 68 consecutive passes. An exact-series pin rather than a narrower range is required: the inline namespace encodes the release, so **any** range spanning more than one RAPIDS minor series is broken by construction. ## This diff - Pin `libcuvs-cu13`, `libraft-cu13`, and `librmm-cu13` to `==26.06.*` in both `dependencies` and `test-requires`. - Declare `libraft-cu13`. `faiss/python/__init__.py` imports `libraft` and calls `libraft.load_library()`, but only `before-all` installed it, so at runtime its version floated on `libcuvs-cu13` transitive metadata. - Correct the cuSOLVER soname in the `auditwheel` exclude list from `libcusolver.so.11` to `libcusolver.so.12`, which is what CUDA 13.2 ships. - Update the `_preload_gpu_libs` fix-it message, which told users to install the range this diff removes. - Add `TestCuvsRapidsPinning` to `tests/test_wheel_smoke_gpu.py`. ## Regression guard `tests/test_wheel_smoke_gpu.py` is already the `test-command` for the cuVS wheel, so the new class runs inside `build-pip-gpu.yml` with no workflow change and no new test file. It asserts three properties of the built wheel's own `Requires-Dist`: 1. every RAPIDS distribution `_preload_gpu_libs` dlopens is declared 2. each is pinned to `==26.06.*` rather than an open range 3. the installed versions sit in that series `_RAPIDS_SERIES` duplicates the series from `pyproject-gpu-cuvs.toml` because the wheel does not ship that file. The duplication is the tripwire: a RAPIDS bump has to touch both places deliberately. ## Prior attempt D117099095 made the version-pin change on 2026-08-23 and was abandoned on 2026-09-04 without landing. It never failed review. `faiss/public_tld/` is ShipIt-mirrored with `requireExportConfig`, so `github-export-checks` held it Unpublished pending a `facebookresearch/faiss` PR that a bot cannot open, and three pings over 12 days went unanswered. This diff needs that same export. ## Notes The published 1.15.0 metadata declares only `libcuvs-cu13`, not `librmm-cu13`. That is not a fbsource/GitHub divergence: `librmm-cu13` was added to `dependencies` on 2026-08-14, after the 08-03 release, and has never shipped. `rapids-logger` is installed by `before-all` but stays undeclared here. `librmm-cu13` owns that link-time dependency and carries it in its own metadata; an unversioned duplicate would not make the resolve safer. Reviewed By: mnorris11 Differential Revision: D120162004 fbshipit-source-id: eb6dc316a4fba2a4228eab62d7b86ac8ea94a1b7 |
||
|
Trang Nguyen
|
7c3b55c01b |
Make Panorama bounds checks overflow-safe (#5616)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5616 Follow up on D118495504 by closing two arithmetic gaps in the Panorama bounds validation. Avoid signed overflow and reject negative counts in `IndexFlatPanorama::reconstruct_n`, and compute the serialized `IndexFlatPanorama` batch count without overflowing `ntotal + batch_size - 1`. Add regression coverage for negative reconstruction counts and a maximal serialized batch size that previously admitted empty storage for a non-empty index. Reviewed By: mnorris11 Differential Revision: D119272683 fbshipit-source-id: 7ea841bd4bd90d0358b78797441fa3cc815db46a |
||
|
Trang Nguyen
|
fb2d57a312 |
Fix leak and unbounded lists[] in OnDisk invlist deserialization (#5560)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5560 `OnDiskInvertedListsIOHook::read` allocates its `OnDiskInvertedLists` with a bare `new` and then keeps reading. Any throw after that point leaks the whole object -- including `od->lists`, which `READVECTOR` will happily size up to `get_deserialization_vector_byte_limit()`. Throwing after that point is the common case for malformed input: the `slots` and `filename` reads can fail, and `do_mmap()` on an attacker-controlled filename fails essentially always. This is what the `faiss_read_index_binary_fuzzer` health check has been tripping over. The one CI stack that survived tpx truncation names `OnDiskInvertedLists.cpp:722` (`vector<OnDiskOneList>::resize`) as the largest live allocation, reached via `IBFf` -> `read_index` -> `IxRP` -> IVF -> `read_InvertedLists`. Because the harness runs with `detect_leaks: 0`, the leaked blocks accumulate across a 120 s session; peak RSS across the failing runs climbed 978 -> 1142 -> 1249 -> 1815 -> 4356 MB against `-rss_limit_mb=4096`. That is why the check reads as flaky (23.6%) rather than as a hard red, and why the saved crash artifacts do not reproduce when replayed one at a time. Switch both `read` and `read_ArrayInvertedLists` to hold the object in a `std::unique_ptr` and `release()` on success. The public signature still returns `InvertedLists*`, so this is a local scope guard only -- the same shape `read_InvertedLists_up` in `index_read.cpp` already uses. Also reject `lists.size() != nlist` at read time. Every `OnDiskInvertedLists` accessor (`list_size`, `get_codes`, `get_ids`, ...) indexes `lists[list_no]` with `list_no` bounded only against `nlist`, so a file whose `lists` vector is shorter than its `nlist` is a straight out-of-bounds read. Every construction path in the class keeps `lists.size() == nlist`, so this rejects only malformed input. Reviewed By: mnorris11 Differential Revision: D118487565 fbshipit-source-id: 6f01c937b24a2a98620e4ca2bde141674af1b19d |
||
|
Michael Norris
|
a74867cf56 |
faiss HNSW: delete the flag and the lock-based graph build (#5611)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5611 D112025877 added the `faiss::hnsw_deterministic_build` flag. The flag chose between the lock-based graph build and the deterministic graph build. This diff deletes the flag. The deterministic build is now the only build. Reviewed By: trang-nm-nguyen Differential Revision: D118342471 fbshipit-source-id: d3c5e5238aca2bcda463eb27959ec22827222c15 |
||
|
Ge Gao
|
4fdbc9dc18 |
Support ZeroCopyIOReader for IndexRaBitQ deserialization (#5629)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5629 `IndexRaBitQ` keeps its database in `codes`, a `MaybeOwnedVector<uint8_t>` inherited from `IndexFlatCodes`, but both read branches (`Ixrq` for single-bit and `Ixrr` for multi-bit) used the raw `READVECTOR` macro. So `codes` was always an owning heap vector even when the caller passed a view-capable reader, while every other flat-codes index already goes through `read_vector`. This switches both branches to `read_vector`, which dispatches on the reader type in `read_vector_base` and makes `codes` a view of the caller's buffer. Any other reader falls back to `READVECTOR`, so existing callers are unaffected. A zero-copy RaBitQ index now works like this: ``` #include <faiss/impl/zerocopy_io.h> faiss::ZeroCopyIOReader reader(buffer.data(), buffer.size()); std::unique_ptr<faiss::Index> index_zc(faiss::read_index(&reader)); ``` The buffer must outlive the index, as with any `faiss::ZeroCopyIOReader`. Reviewed By: mnorris11 Differential Revision: D119297402 fbshipit-source-id: 95c5ba050cb3a0636a0899efbe886616a7a8a2fd |
||
|
Michael Norris
|
b5a14632a6 |
Run the eight SIMD kernel tests that no build executed (#5620)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5620 **TL;DR:** Eight SIMD kernel tests never ran, and two byte-domain kernels trust their caller. This change makes three fixes: 1. **Six tests never compiled.** `test_rabitq_simd.cpp` hides them behind `COMPILE_SIMD_*` macros that the faiss target does not export. 2. **Two tests skipped themselves in a static build.** `TestSQByteDirectAcrossSIMDLevels` needs `SIMDLevel::NONE` as a reference, and a static build does not report that level. 3. **The byte kernels never test their dimension requirement.** `DistanceComputerByte` and `DistanceComputerByteSigned` read 16 bytes per loop step. ## 1. Six tests never compiled `test_rabitq_simd.cpp` wraps two blocks in `#ifdef COMPILE_SIMD_AVX2` and `#ifdef COMPILE_SIMD_AVX512`. The faiss target sets those macros as `preprocessor_flags`, and it does not export them. A test translation unit never sees them. So the preprocessor deleted both blocks. The file holds nine tests and ran three. The AVX2 block loses its guard, because every x86 faiss build holds the AVX2 kernels. Its three tests also lose an `#ifdef FAISS_ENABLE_DD` around a runtime skip, which the same problem disabled. `is_simd_level_available()` reports the right answer in a static build too. The AVX512 block moves to a dynamic dispatch target. A static x86 build compiles the AVX2 kernels only, so the test fails to link there rather than skips. `test_rabitq_simd_vpopcnt.cpp` becomes `test_rabitq_simd_dd.cpp`, because it now holds every RaBitQ test that a static build cannot link. A new `test_rabitq_simd_util.h` holds the scalar reference that both files call. ## 2. Two tests skipped themselves in a static build `TestSQByteDirectAcrossSIMDLevels` measured the index a second time at `SIMDLevel::NONE`, then compared the two runs. A static build reports only the level it compiles, and omits `NONE`, so both tests skipped. The tests now compute the distances from the definition. The distances are integers, so the reference is exact. The tests check absolute values now, instead of the agreement of two faiss paths. ## 3. The byte kernels trust the caller `DistanceComputerByte` and `DistanceComputerByteSigned` read 16 bytes per loop step, so they require `d % 16 == 0`. `sq-dispatch.h` enforces that, and no other caller builds them. A caller that ignores the requirement reads up to 15 bytes past each code and returns a wrong distance. Each constructor now tests the requirement and throws. The AVX2 and the ARM_NEON specializations both get the test. Reviewed By: alibeklfc Differential Revision: D119545379 fbshipit-source-id: ddfdcc30d3662f469b2ec3dbb9d5b9f4dafa21aa |
||
|
Michael Norris
|
6bbb068ad8 |
Measure 20-byte Hamming codes eight at a time (#5587)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5587 **TL;DR:** Adds a batched 20-byte Hamming kernel that measures eight codes per call, the only thing that helps the 160-bit width, worth 1.21x end to end. A 20-byte code has no vector kernel at any x86 SIMD level. `HammingComputer20` measures one distance with three scalar popcounts at every level, including `AVX512_VPOPCNT`, so a scan of 160-bit codes runs the same code everywhere. Vectorizing a single 20-byte distance does not help. A masked 256-bit load plus `vpopcntq` plus a horizontal reduction measures the same as the scalar body, because the reduction costs more than the three popcounts it replaces. Measuring several codes per call does help, because it amortizes that reduction. This adds `hamming_batch()` to `HammingComputer20` at `AVX512_VPOPCNT`, which measures eight codes at once: - Eight codes span 160 bytes. `_mm512_popcnt_epi8` gives a count per byte. - `_mm512_sad_epu8` sums each aligned 8-byte group. - A 20-byte code covers two whole groups plus half of a group it shares with its neighbour, so each code needs one scalar popcount for its half. `IVFBinaryScannerL2::scan_codes` and `IVFBinaryScannerL2::scan_codes_range` call `hamming_batch()` when the computer declares a `batch_size`, then measure the remaining codes one at a time. A computer without a `batch_size` keeps the existing loop, so no other code size changes. The scanner holds the tiled query, and the constructor rejects a code size that does not match the computer. A caller therefore cannot reach the batch kernel with a stride the kernel does not read. The byte-wise popcount needs `AVX512_BITALG`, which is a separate CPUID bit from `AVX512_VPOPCNTDQ`. This adds the flag to the `AVX512_VPOPCNT` level and requires both bits to select it. Every CPU that has VPOPCNTDQ also has BITALG, so no CPU loses the level. An alternative byte-wise popcount built from `vpshufb`, which needs neither bit and would run at the plain `AVX512` level, measures only 1.08x against 1.54x, so it is not worth the wider reach. Reviewed By: alibeklfc Differential Revision: D118971285 fbshipit-source-id: 5fd4637c972e5c55e2695b32b9b15fba8181a242 |
||
|
Michael Norris
|
3c7042f17a |
Let the binary Hamming paths reach the VPOPCNT kernels
Summary: **TL;DR:** Points `IndexBinaryIVF` at the VPOPCNT level and instantiates its scanner in the translation unit compiled with `-mavx512vpopcntdq`, worth 1.58x at 256-bit and 1.41x at 512-bit on Genoa. Two things kept the binary Hamming paths on scalar popcount, whatever the host. `IndexBinaryIVF` builds its scanner and runs its searches through `with_simd_level`, which uses the BASE level mask. That mask holds no `AVX512_VPOPCNT` bit, so on a host that reports that level the dispatch falls through to `AVX512`, where the wide computers use scalar popcount. Two build variants, `faiss_omp_mock` and `faiss_no_multithreading`, hardcoded dynamic dispatch off. They compiled the AVX2 kernels only, so a consumer of either variant could not use runtime dispatch at all, whatever the build setting said. This change: - Moves the three dispatch sites in `IndexBinaryIVF.cpp` to `with_simd_level_with_vpopcnt`. Below that level the dispatch falls through to `AVX512`, as before. - Instantiates `IndexBinaryIVF_impl.h` in `hamming_avx512_vpopcnt.cpp`. That is the one translation unit compiled with `-mavx512vpopcntdq`, which the VPOPCNT computers need. Without it the new dispatch has no symbol to call. The include must follow the computer specializations, so it is placed after them. - Lets both build variants honour the dynamic dispatch setting. `HammingComputer16` and `HammingComputer20` still inherit the scalar body at this level, so a 16-byte or 20-byte code gains nothing here. `IVFBinaryScannerL2::scan_codes` also reads the heap top into a local `bound` instead of loading `simi[0]` on every code. Only `scan_codes` writes `simi`, so it refreshes `bound` after each `heap_replace_top` and the two forms return the same results. The test plan measures it. Reviewed By: alibeklfc Differential Revision: D118874433 fbshipit-source-id: dc74294ecaf178919fdd713abc60c44bbaa80347 |
||
|
lyang24
|
e6bd1d252c |
Add AVX512 VPOPCNT dynamic dispatch level (#5531)
Summary: ## What Split the existing VPOPCNTDQ RaBitQ and Hamming kernels from the full Sapphire Rapids SIMD level. DD builds now expose an `AVX512_VPOPCNT` capability for CPUs such as Ice Lake and Zen 4. The fallback chain is: `AVX512_SPR -> AVX512_VPOPCNT -> AVX512 -> AVX2 -> NONE` The ordinary static AVX512 target remains unchanged and contains no VPOPCNT instructions. ## Why The VPOPCNT kernels only require baseline AVX-512 plus `AVX512_VPOPCNTDQ`. Tying them to the SPR level unnecessarily excluded CPUs that support VPOPCNTDQ but not AVX512-FP16, notably Zen 4. ## Changes made while importing Two changes were needed on top of the pull request. **Feature detection read the wrong CPUID leaf.** `AVX512_VPOPCNTDQ` and `AVX512_VNNI` were read from leaf 1 ECX. Both live in leaf 7 subleaf 0 ECX, at bits 14 and 11. Leaf 1 ECX holds unrelated bits at those positions, so the new level was never selected on any CPU. Measured on an AMD Genoa host before the fix: the level reported as unavailable and detection returned `AVX512`. After the fix the same host reports `AVX512_VPOPCNT`. **The build definitions outside CMake were not updated.** The pull request renames three files. The other build description still named the old paths and had no entry for the new level, so the library did not build. This adds the level constant, its compiler flags, its `COMPILE_SIMD_AVX512_VPOPCNT` define, and the renamed paths. **The new level was not registered in the compiled-level mask.** `compiled_simd_levels()` reports which levels a binary holds, and `FAISS_SIMD_LEVEL` walks down from the requested level until it finds one that is present. The new level had no entry, so the override always stepped past it to `AVX512` and printed that the level was not compiled in. This was reported by `test_dispatch_with_env_var` on the open-source dynamic-dispatch build. **The test helper inferred available levels from `/proc/cpuinfo`.** A CPU feature does not imply that a level is compiled into the build, so the helper listed a level the override then refused. It now asks `SIMDConfig::is_simd_level_available()`, which reports compiled and supported. That is a subset of compiled, so every level it returns is one the override accepts. The mask and helper names also follow the naming used in the parent diff, so `AVAILABLE_SIMD_LEVELS_BASE_WITH_VPOPCNT` and `with_simd_level_with_vpopcnt`. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5531 Test Plan: ``` buck2 test fbcode//mode/opt -c faiss.dynamic_dispatch=true \ fbcode//faiss/tests:test_simd_levels fbcode//faiss/tests:test_rabitq_simd ``` Result: all pass. The build failures the imported version reported are cleared. All three configurations build: ``` buck2 build fbcode//mode/dev fbcode//faiss:faiss fbcode//faiss:faiss_no_multithreading fbcode//faiss:faiss_omp_mock buck2 build fbcode//mode/opt -c faiss.dynamic_dispatch=true <same three targets> buck2 build fbcode//mode/dev -c fbcode.arch=aarch64 <same three targets> ``` Detection on an AMD Genoa host, which has AVX512_VPOPCNTDQ and AVX512_BF16 but not AVX512_FP16: | | before the CPUID fix | after | | --- | --- | --- | | detected level | `AVX512` | `AVX512_VPOPCNT` | | `AVX512_VPOPCNT` available | 0 | 1 | | `AVX512_SPR` available | 0 | 0 | Reviewed By: alibeklfc Differential Revision: D118969754 Pulled By: mnorris11 fbshipit-source-id: d1ca52b614829a3d4b322c9fecf0c004f2174dab |
||
|
Michael Norris
|
9f993087b0 |
Name the SIMD level masks after what they hold (#5600)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5600 **TL;DR:** Renames the dispatch masks from `A0`/`A1`/`A2` to names that say which levels they hold; no behaviour change. The dispatch masks were named `A0`, `A1` and `A2`. The names say nothing about which levels a mask holds, so a caller had to read `simd_dispatch.h` to pick one. That is how several call sites ended up on a mask with no `ARM_SVE` bit while a dedicated SVE kernel existed. This renames every mask and helper after its contents. It changes no behaviour. | Old name | New name | Holds | | --- | --- | --- | | `AVAILABLE_SIMD_LEVELS_A0` | `..._BASE` | NONE, AVX2, AVX512, ARM_NEON, RISCV_RVV | | `AVAILABLE_SIMD_LEVELS_A0_SPR` | `..._BASE_WITH_SPR` | BASE + AVX512_SPR | | `AVAILABLE_SIMD_LEVELS_A1` | `..._BASE_WITH_SVE` | BASE + ARM_SVE | The helpers follow: `with_simd_level_a0_spr` becomes `with_simd_level_with_spr`, and `with_simd_level_a1` becomes `with_simd_level_with_sve`. `BASE` is the right name for the default mask because `ARM_NEON` belongs to it. NEON is mandatory on aarch64, the way AVX2 is the x86 baseline, while `ARM_SVE` is optional. So the ARM fallback chain is SVE to NEON to NONE. Reviewed By: alibeklfc Differential Revision: D119025991 fbshipit-source-id: 2b6681de2d83df6468569a365d05341088742cf8 |
||
|
Michael Norris
|
fc13c81fb8 |
Stop aarch64 falling back to SIMDLevel::NONE in faiss dispatch (#5572)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5572 **TL;DR:** Stops aarch64 running scalar code where a NEON or SVE kernel exists, adds the two missing NEON byte-domain kernels, and fixes `FAISS_SIMD_LEVEL` falling all the way to `NONE` on an uncompiled level. On aarch64 several faiss dispatch paths ran scalar code, or ran an `ARM_NEON` kernel where an `ARM_SVE` kernel already existed. This change enforces the order SVE -> NEON -> NONE on an ARM host. It also adds the one kernel pair that was missing. T287037898 reported the scalar-quantizer half. On aarch64 `IndexScalarQuantizer` with `QT_8bit_direct` or `QT_8bit_direct_signed` saved memory but lost throughput. `Refine(SQ8)` has the same problem, because `IndexRefine::search` calls `refine_index->get_distance_computer()`. ## What each call site gets | Call site | Before | After | | --- | --- | --- | | `QT_8bit_direct` on aarch64 | float-domain `DCTemplate` | NEON `DistanceComputerByte` | | `QT_8bit_direct_signed` on aarch64 | float-domain `DCTemplate` | NEON `DistanceComputerByteSigned` (new) | | `IndexFlat` distance computers, SVE host | `ARM_NEON` | `ARM_SVE` | | `AdditiveQuantizer::compute_centroid_norms`, SVE host | `ARM_NEON` | `ARM_SVE` | | `SuperKMeans` `block_l2`, SVE host | `ARM_NEON` | `ARM_SVE` | | `pq_code_distance` wrappers, SVE host | `ARM_NEON` | `ARM_SVE` | | `with_VectorDistance`, SVE host | `ARM_NEON` | `ARM_SVE` | | `FAISS_SIMD_LEVEL=ARM_SVE`, build without SVE | `NONE` | nearest compiled level | ## Three causes, fixed separately **1. Two missing kernels.** `sq-neon.cpp` held a scalar `DistanceComputerByte<Sim, ARM_NEON>` that nothing could reach, and no `DistanceComputerByteSigned<Sim, ARM_NEON>` at all. Both are now real NEON kernels. L2 and the unsigned inner product use `vabdq_u8`, `vmull_u8` and `vpadalq_u16`. The bias-encoded inner product uses `veorq_u8`, `vmull_s8` and `vpadalq_s16`. Both agree bit for bit with the AVX2 specializations, which the tests require. Two invariants deserve a note: - The L2 path keeps an unsigned accumulator. A `vmull_u8` square reaches 255^2 = 65025, which an int16 lane would read as negative. - For x in 0 to 255, `x ^ 0x80` read as `int8` is exactly `x - 128`. That is how the kernel removes the +128 bias before `vmull_s8`. **2. Dispatch chains that list only x86 levels.** The `if constexpr` chains in `sq-dispatch.h` enumerated x86 levels only, so an ARM host fell through to the float path. This adds `ARM_NEON` to four chains: two in `select_distance_computer_body`, and two in `sq_select_InvertedListScanner`. The chain in `is_dimension_compatible` already included ARM. This does not add `ARM_SVE`. The scalar-quantizer entry points dispatch with a mask that holds no `ARM_SVE` bit, so an SVE host already falls through to the `ARM_NEON` case and now gets these kernels. Adding `ARM_SVE` would instantiate the empty primary template in `distance_computers.h`. This corrects that template's stale comment. **3. Level masks that hide existing SVE kernels.** The default mask holds no `ARM_SVE` bit, so the dispatch fell through `case ARM_SVE` to `ARM_NEON`. This uses `with_simd_level_a1` at every site where a real SVE kernel exists and links: `IndexFlat.cpp` at four sites, `AdditiveQuantizer::compute_centroid_norms`, `block_l2` in `SuperKMeans.cpp`, all three wrappers in `pq_code_distance-generic.cpp`, and `with_VectorDistance` in `distances_dispatch.h`. The `IndexFlat` sites matter most. `faiss::fvec_L2sqr()` already used the SVE mask, so on an SVE host the free function ran SVE while `IndexFlatL2`'s distance computer ran NEON. Also: `FAISS_SIMD_LEVEL=ARM_SVE` on a build without SVE compiled in used to skip every level and run at `NONE`, because `with_selected_simd_levels` has no case label for an uncompiled level. It now walks down to the nearest compiled level. The override is still honoured when the CPU lacks the level, since forcing a level is the point of it. Only uncompiled levels are corrected. The unused `AVAILABLE_SIMD_LEVELS_A2` is deleted. It is the NEON-to-NONE trap in constant form, with no users. This change keeps the existing mask names. A follow-up renames them to say what they hold. ## Out of scope In rough order of remaining value: - `IndexPQ.cpp` and `IndexIVFPQ.cpp` still pin PQ to `ARM_NEON` on an SVE host. A mask change is not enough. `pq_code_distance-sve.cpp` includes only `pq_scan_impl.h`, and `with_HammingComputer<ARM_SVE>` has no complete type. - `distances_aarch64.cpp` forwards five `ARM_NEON` specializations to `<SIMDLevel::NONE>`, so a non-SVE aarch64 host runs scalar code in the IVFFlat scan. - `rabitq_neon.cpp` forwards all six `ARM_NEON` specializations to `<SIMDLevel::NONE>`. There is no SVE variant. - There is no `block_l2<ARM_NEON>` and no `exhaustive_L2sqr_blas_cmax<ARM_NEON>`. - Part two of T287037898, which is SDOT and SMMLA. `FEAT_DotProd` and `FEAT_I8MM` are not expressible as a `SIMDLevel`. The right shape is a runtime `getauxval(AT_HWCAP)` check inside the ARM translation unit, which follows the `SIMDConfig::avx512_split` precedent. Note the reporter's caveat: SMMLA shows no improvement until the scan loop is tiled, so the tiling must land in the same change. Reviewed By: alibeklfc Differential Revision: D118682598 fbshipit-source-id: 1ab588d9fa130e8246bf996025fc11d8954323be |
||
|
Michael Norris
|
00928e3f76 |
Fix ROCm CI: build in a dedicated conda env instead of the runner's broken base (#5545)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5545 The `linux-x86_64-GPU-w-ROCm-cmake` job fails installing AMD's ROCm torch wheels: they exist for one CPython ABI only, and the MI325 runner's interpreter is 3.14, not the 3.12 the action asks for. Nothing in `build_cmake` actually constrained the interpreter. `setup-miniconda` requests `python-version: '3.12'` and `Configure build environment` asks for `python=3.12` again, but the runner's conda base has `conda-meta` listing both `python 3.12.9` and `cpython 3.14.6`, so conda reports "already installed" while `bin/python` is 3.14. Forcing it does downgrade the interpreter, but strands conda's own compiled extensions (`pydantic_core`), and conda builds its argument parser through its plugins -- so every subsequent conda call in the job dies. Working around that with `CONDA_NO_PLUGINS` in turn drops the libmamba solver, which conda then rejects because `solver: libmamba` stays configured. Meanwhile the base env is inconsistent across three Python ABIs (conda lists ~200 conflicting packages), so there is no in-place repair to converge on. So stop repairing base. For ROCm only, `conda create` a fresh `$CONDA/envs/faiss` prefix on 3.12 and point `$CONDA` at it, leaving base untouched so conda keeps running on its own interpreter. `$CONDA` is the single lever the later steps already read (`-DPYTHON_EXECUTABLE`, `setup.py install`, the import smoke tests), so redirecting it is enough. Making that lever actually hold: - Pass `-p "$CONDA"` on every `conda install`, and `conda activate "$CONDA"`. A bare `conda install` targets the active env, which is base regardless of `$CONDA`. No-op for the non-ROCm jobs, where `$CONDA` is base. - Write base's `bin` to `$GITHUB_PATH` *before* the env's. The runner prepends entries as it reads them, so the last line written ends up first on PATH: this leaves the env's interpreter ahead of base's while keeping base reachable for `conda` itself. - Drive the ROCm torch step through `"$CONDA/bin/python" -m pip` rather than bare `python`/`pip3`, so it does not depend on PATH ordering at all. - `rm -rf` the prefix before `conda create`: it survives between runs on this self-hosted runner and `conda create` refuses an existing one. A fresh prefix each run is the point. - Derive the `libpthread_nonshared.a` symlink from `$CONDA` instead of a hardcoded `$HOME/miniconda3`, since `sysroot_linux-64` now installs into the build's prefix, and assert the target exists -- `ln -s` does not check, so a wrong path would surface much later as a link failure. - `conda list -p "$CONDA"` for the diagnostics, so the logs describe the env under test rather than base. Torch wheel installation: - Resolve through `--find-links` as a flat index instead of hardcoding a `cp312-cp312-linux_x86_64` filename; the hardcoded tag is what made an interpreter mismatch surface as "not a supported wheel on this platform". - Keep a retag fallback for pip builds that reject the unqualified `linux_x86_64` tag, but gate it on the resolver's own error text so a transient failure (network, disk) fails loudly instead of being retagged over. - `--ignore-installed`, because this runner is self-hosted and a wrong-ABI torch from an earlier run otherwise satisfies the requirement and skips tag matching. - Use AMD's wheels built against the system ROCm rather than the PyTorch Foundation `rocm7.2` wheels, which bundle an older ROCr lacking `hsa_amd_memory_get_preferred_copy_engine` and collide with the system `libamdhip64`. `ROCM_VERSION` is exported to `$GITHUB_ENV` so the apt release and the wheel release cannot drift apart. - Assert on interpreter version, torch's install location, and `torch.version.hip` rather than printing them. Each of these failures previously surfaced several steps later as something unrelated-looking. Note for infra: the base env on this runner is corrupt independently of this change (mixed 3.12/3.13/3.14 packages, and truncated `lib/python3.1/site-packages` paths from a `sys.version[:3]` bug upstream). Wiping it would let several of the workarounds above be dropped. Reviewed By: alibeklfc Differential Revision: D117896658 fbshipit-source-id: 26f98c53e5daf747dd81e0fd35232b4160f62377 |
||
|
Trang Nguyen
|
5702cafcb7 |
Remove dead TurboQuant projection state (#5618)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5618 Remove the unused projection buffers and eager `init_projection` work from `ScalarQuantizer::TurboQuantRefine`. Quantizers construct the required projection from `trained` when used, so rebuilding it during training and deserialization only added an unbounded quadratic allocation and cubic QR operation. Keep `seed` and `qjl_type` as the serialized configuration, enforce that the helper remains a small config-only struct, and cover both projection modes in deserialization tests. ___ Differential Revision: D119243693 fbshipit-source-id: 53f982b00dd849cbe88983ffd27809c2fa738a4c |
||
|
Trang Nguyen
|
545550f6a2 |
Use unittest assertions and f-strings in test_index_composite (#5571)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5571 `test_index_composite.py` used bare `assert` statements throughout. Inside a `unittest.TestCase` these are worse than the `self.assert*` helpers in two ways: a failure prints only `AssertionError` with no operand values, so you have to re-run under a debugger to see what the actual number was, and the whole assertion is compiled out under `python -O`, silently turning the test into a no-op. This converts every bare `assert` in the file to the matching `self.assert*` call (`assertEqual`, `assertTrue`, `assertFalse`, `assertNotEqual`, `assertGreater`) and replaces the two remaining printf-style format strings with f-strings. Mechanical change only — no assertion was strengthened, weakened, or reordered, and no test logic was touched. The `raise AssertionError()` idioms guarding `try`/`except RuntimeError` blocks are left alone; converting those to `assertRaises` would change control flow and belongs in its own diff. Split out of D118606761 so that the `IndexIVFFlatDedup::remove_ids` bug fix stays reviewable on its own. Note this file is exported to the open source facebookresearch/faiss repo, so this diff needs an OSS export alongside it. Reviewed By: mnorris11 Differential Revision: D118721255 fbshipit-source-id: b6933a6119955f2dbd7590f4e39632d700a537c4 |
||
|
Trang Nguyen
|
1e5780fbbf |
Fix OOB read in Panorama::reconstruct via IndexPreTransform (T287094917) (#5562)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5562 Lionhead harness `faiss_rmdt_post_read_reconstruct_fuzzer` found a heap out-of-bounds read at `Panorama.cpp:171`, reported as `READ of size 5122903332` in `__asan_memcpy`. Decoding the 232-byte crash input, it is `IxPT -> IxPT -> IxPT -> IxFp` where every `IndexPreTransform` carries an **empty** transform chain and the innermost `IndexFlatIPPanorama` has `d = 0x4C565349 = 1280725833` (so `code_size = d * 4 = 5122903332`, exactly the ASAN read size), `ntotal = 0`, and an empty `codes` vector. Three independent defects combine: 1. `IndexFlatPanorama::reconstruct` never bounds-checks `key`. It overrides `IndexFlat::reconstruct`, which does have `FAISS_THROW_IF_NOT(key < ntotal)`, and drops the guard. Overriding `reconstruct_n` likewise removes `IndexFlatCodes::reconstruct_n`'s `i0 + ni <= ntotal` check. With `ntotal == 0` and empty `codes`, `reconstruct(0, ...)` memcpy's `code_size` bytes out of a zero-length buffer. `Panorama::reconstruct` takes an unsized `const uint8_t*`, so it cannot self-validate -- the caller has to. 2. The `IxFP`/`IxFp` read branch does not range-check `d`. It is the one index type that does not go through `read_index_header`, so it misses that function's `FAISS_CHECK_RANGE(d, 0, (1 << 20) + 1)` and `ntotal >= 0` checks. The `codes.size()` equality check below it does not catch this, because `ntotal == 0` makes `num_slots == 0` and an empty `codes` is then "valid". 3. `IndexPreTransform` deserialization skips all consistency checks when the chain is empty -- the `chain[0].d_in == ixpt->d` / `chain[nt-1].d_out == index->d` validation sits entirely inside `if (nt > 0)`. With `nt == 0`, `IndexPreTransform::reconstruct` passes the caller's buffer (sized from the outer `d`) straight to the sub-index, so a mismatched sub-index `d` is also a write-side overflow, for any sub-index type. Each is fixed: - `IndexFlatPanorama::reconstruct` / `reconstruct_n` now range-check against `ntotal`. - The `IxFP`/`IxFp` branch applies the same `d` and `ntotal` checks as `read_index_header`. - `read_index` requires `index->d == ixpt->d` when the `IxPT` chain is empty. Two adjacent gaps in the same audit are closed as well: - `IHfP` (`IndexHNSWFlatPanorama`) never validated `cum_sums.size()`, unlike the equivalent `IxFP` check. Both `get_cum_sum()` and `HNSW.cpp`'s Panorama search path index it at stride `pano.n_levels + 1`, so a short vector is an OOB read at search time. It also had no explicit `n_levels > 0` guard. - `ArrayInvertedListsPanorama::get_single_code` bounds-checked with `assert`, which is compiled out in opt builds. It reaches the same `Panorama::reconstruct` sink via `IndexIVFFlatPanorama::reconstruct_from_offset`. Reviewed By: mnorris11 Differential Revision: D118495504 fbshipit-source-id: 46bc7a1a6c9335a5589e20221b9478ae5505b9da |
||
|
lyang24
|
7f4cedda9c |
Add staged RaBitQ storage to HNSW (#5526)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5526 Reviewed By: junjieqi, mnorris11 Differential Revision: D116650696 Pulled By: alibeklfc fbshipit-source-id: 68ec701da84bd72a71e83ae5fadc041b2890eefe |
||
|
Binozo
|
2ed4c106e9 |
Fix: Windows SIMD detection (#5497)
Summary: Hi, while testing on windows x64 systems I've noticed that the `dd` optimization compilation target doesn't work properly. Causing crashes on non-avx2 systems for instance. This pr fixes the SIMD level detection. MSVC doesn't define `__x86_64__` but does define `_M_X64`. This is what this patch is based on. I got some nice [GitHub actions](https://github.com/Binozo/Faiss.NET/actions/runs/30251011773/job/89928775910) and tested the patch on it. ##### Based on https://github.com/facebookresearch/faiss/issues/5476 but reopened because I made a mistake while rebasing Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5497 Reviewed By: alibeklfc Differential Revision: D117897685 Pulled By: mnorris11 fbshipit-source-id: 80987895d17945bac9a4c5f0eb14beb67c84cbd5 |
||
|
kzahiri1
|
b4c66ba603 |
Fix reverse_index_factory/get_code_size round-trip for HNSW, IMI and IVFPQR (#5564)
Summary:
Three defects in `contrib/factory_tools.py` where a reversed factory string rebuilds a different index or fails to parse. `test_get_code_size_hnsw_roundtrip` already pins this invariant for flat HNSW; these cases escape it.
1. **HNSW storage dropped.** The `IndexHNSW` branch never read `index.storage`, so `HNSW32,SQ8` and `HNSW32,PQ16x8` both reversed to `"HNSW32"` and rebuilt as `IndexHNSWFlat`. `get_code_size` then applied the flat `d*4` formula: at d=128 it reported 768 bytes/vector for an index whose storage is 128 plus the 256-byte graph estimate, so 384.
2. **IMI regex fields transposed.** `index_factory.cpp:295` parses `IMI2x([0-9]+)`, but the regex read `IMI\d+x2`. Every real IMI key (`IMI2x10,PQ16`, and the ones in `demos/demo_auto_tune.py`) raised `RuntimeError("cannot parse")`.
3. **IndexIVFPQR matched by the IndexIVFPQ check**, dropping the refinement stage: `IVF64,PQ8+16` reversed to `IVF64,PQ8x8`, 8 bytes instead of 24.
Consumers are `benchs/bench_all_ivf/parse_bench_all_ivf.py` (imports `get_code_size` as `unitsize`) and `benchs/bench_fw/index.py`.
Flat storage stays implicit as the bare `HNSW<M>` string, and a null `storage` pointer still returns `HNSW<M>` instead of raising.
**Behavior change worth flagging:** `IndexHNSW2Level` storage is `Index2Layer`, which `reverse_index_factory` does not handle, so it now raises `NotImplementedError` instead of silently returning a wrong `"HNSW32"`. It is not reachable from `index_factory`.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5564
Test Plan:
Four tests added to `TestFactoryTools`. All four fail on unpatched `factory_tools.py` and pass with it:
```
# pristine contrib/factory_tools.py, new tests present
4 failed, 7 passed
FAILED test_get_code_size_hnsw_non_flat_storage
FAILED test_get_code_size_imi
FAILED test_hnsw_storage_reverse_index_factory
FAILED test_ivfpqr_reverse_index_factory
# with the fix
11 passed
```
Full `tests/test_contrib.py tests/test_factory.py`, pristine vs branch:
```
pristine: 2 failed, 88 passed
branch: 2 failed, 92 passed
```
Failure sets are identical. The two are `TestComputeGT::test_compute_GT_gpu` and `test_compute_GT_ip_gpu`, pre-existing and unrelated (`AttributeError: module 'faiss' has no attribute 'GpuMultipleClonerOptions'` on a CPU-only build).
An A/B sweep over 40 factory strings changes exactly the 7 expected lines and nothing else:
```
- HNSW32,SQ8 IndexHNSWSQ HNSW32 768
+ HNSW32,SQ8 IndexHNSWSQ HNSW32,SQ8 384
- HNSW32,SQfp16 IndexHNSWSQ HNSW32 768
+ HNSW32,SQfp16 IndexHNSWSQ HNSW32,SQfp16 512
- HNSW32,PQ16x8 IndexHNSWPQ HNSW32 768
+ HNSW32,PQ16x8 IndexHNSWPQ HNSW32,PQ16x8 272
- IVF64,PQ8+16 IndexIVFPQR IVF64,PQ8x8 8
+ IVF64,PQ8+16 IndexIVFPQR IVF64,PQ8+16 24
- IMI2x5,Flat IndexIVFFlat IMI2x5,Flat <RuntimeError>
+ IMI2x5,Flat IndexIVFFlat IMI2x5,Flat 512
- IMI2x5,PQ8 IndexIVFPQ IMI2x5,PQ8x8 <RuntimeError>
+ IMI2x5,PQ8 IndexIVFPQ IMI2x5,PQ8x8 8
- IMI2x8,PQ8x8 IndexIVFPQ IMI2x8,PQ8x8 <RuntimeError>
+ IMI2x8,PQ8x8 IndexIVFPQ IMI2x8,PQ8x8 8
```
flake8 at the 80-char limit from CONTRIBUTING reports the same 12 pre-existing findings before and after, none in the changed lines.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed By: mnorris11
Differential Revision: D118661065
Pulled By: trang-nm-nguyen
fbshipit-source-id: b50a9522361e3fd14f276b36196d3c728567b980
|
||
|
ethanglaser
|
cfbf4ec5ff |
SVS: allow option to disable vector storing (#5525)
Summary: Allow avoiding storing original vectors, which may not necessarily be used, in which case they would be wasting memory Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5525 Reviewed By: alibeklfc Differential Revision: D118722620 Pulled By: mnorris11 fbshipit-source-id: 9757fa3d272187308c750fb6c6d0e3a6e8ab2bb3 |
||
|
Aalekh Patel
|
cda4f6e713 |
Expose IndexBinaryIVF.set_direct_map_type via the C API (#5451)
Summary: Expose the `set_direct_map_type` method of binary IVF indexes via the C API. This enables support for the `index.set_direct_map_type` API in languages other than Python. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5451 Reviewed By: alibeklfc Differential Revision: D114681970 Pulled By: mnorris11 fbshipit-source-id: 1342eeb225d6fd9fdea76c6b9417cedde55c1d3a |
||
|
generatedunixname2238252550246476
|
c0084f5cd2 |
Clamp EDEN unbiased L2 distances to be non-negative (#5569)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5569 https://www.internalfb.com/monitoring/oncall_operator?board=1003685189388103&workItem=4614322948811275 --- `IndexEDEN` / `IndexIVFEDEN` with the unbiased 1-bit EDEN scale can return negative `METRIC_L2` distances, violating the contract that L2 distances are never negative. Root cause: for the unbiased scale, `distance_from_code_dot` in `faiss/impl/scalar_quantizer/EDENQuantizer.cpp` reports `query_base + l2_norm_term - 2*scale*<q_code, x_res>`, where the encoder sets `l2_norm_term = ||r||^2` (the *original* residual norm) rather than the reconstructed-code norm `scale^2 * ||q_code||^2`. Unlike the biased scale (where the three terms complete the square `||x_res - r_y||^2 >= 0`), the unbiased branch is an unbiased *estimator* of the squared distance, so for a query residual that aligns strongly with the code direction the subtracted term can exceed `query_base + ||r||^2` and the result goes negative. Every distance path (reference, LUT, and the AVX2/AVX-512 batch variants) routes through `distance_from_code_dot`, and none of them clamped the result. Fix: clamp the L2 branch of `distance_from_code_dot` to zero. This is the single choke point, so it covers every distance and search path. The clamp only floors estimates that were already below the true minimum of zero. Design note for reviewers: clamping biases the unbiased estimator upward for the very closest matches (whose estimates are the ones that dip below zero), and collapses their fine ordering at exactly zero. The alternative — leaking negative squared distances to callers that take a sqrt or compare against a radius — is worse, and this is the most reversible fix. If the team prefers to preserve the strict unbiased estimator and instead relax the non-negativity expectation, that is a larger design decision to make separately. Reviewed By: trang-nm-nguyen Differential Revision: D118611745 fbshipit-source-id: 8c3d6a279fb15a92697589909f89eca218915e52 |
||
|
Trang Nguyen
|
6457e941f5 |
Contain DirectoryArchiver entries within the extraction root (#5561)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5561 `svs::lib::DirectoryArchiver::read_file` builds its output path as `root / filename`, where `filename` is read straight off the deserialization stream. Neither guard on that path contains it: - `std::filesystem::operator/` **discards `root` entirely** when the right operand is absolute, so `filename = "/etc/cron.d/x"` yields `path = "/etc/cron.d/x"`. - A `..` component climbs out of `root` normally. - `check_file(path, out|binary)` only verifies that `path.parent_path()` is an existing directory. It performs no containment check — and that check is satisfied by construction anyway, because `read_file` calls `fs::create_directories(parent_dir)` two lines earlier. The result is that `faiss::read_index_up()` on untrusted bytes can create or overwrite an arbitrary file with attacker-controlled content, bounded only by the process's filesystem permissions. The harness manifest already rates this surface `REMOTE` / `PRE_AUTH` / `ZERO_CLICK`. This is not theoretical. The new tests fail on unpatched code with `std::filesystem::exists(probe)` returning **true** — the file really is written outside the temp root. Note the deserialization then fails with an unrelated `"File could not be opened for reading"`, so the write is a silent side effect that never surfaces as an error to the caller. The fix normalizes both sides and rejects any entry whose resolved path is not under `root`, before `create_directories` runs so that a rejected entry leaves nothing behind. Legitimate entries are unaffected: plain names, nested `sub/dir/name`, `./name`, and interior `..` that stays inside `root` all still resolve normally — only escapes are rejected. `third-party/svs/v0.4.0` is pristine vendored code (`c5e5dbb35bbf`, never patched internally), so this also needs reporting upstream to Intel via their vulnerability handling process (`src/SECURITY.md`) rather than as a public PR, so it survives the next version bump. The related unbounded-allocation defect in `Archiver::read_name`, on the same call path, is fixed separately in the next diff in this stack (T287110736). Reviewed By: mnorris11 Differential Revision: D118494326 fbshipit-source-id: 7f7851111375eda831e4a7dac0acb7cdd1f80dc8 |
||
|
Tamim Ehsan
|
ac60182fc0 |
RaBitQ: Fix zero residual sign convention (#5553)
Summary: The RaBitQ 1 bit quantization, code is quantized comparing positive vs negative or zero https://github.com/facebookresearch/faiss/blob/718002f7cf269f3f3752dde04a4cc3f8eabb0050/faiss/impl/RaBitQuantizer.cpp#L147-L154 However, during ex bit quantization, the signs are flipped under condition negative vs positive or zero. https://github.com/facebookresearch/faiss/blob/718002f7cf269f3f3752dde04a4cc3f8eabb0050/faiss/impl/RaBitQuantizerMultiBit.cpp#L305-L310 Alternative implementation can be taking the equality in the one bit path. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5553 Reviewed By: mnorris11 Differential Revision: D118529759 Pulled By: alibeklfc fbshipit-source-id: 319f0ec06952c73616acac25c37c63dfa7f9203c |
||
|
generatedunixname26860585020226206
|
1a0cfa64e0 |
Apply the deserialization byte limit to SVS IVF (#5568)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5568 `read_index` can cap the bytes one read pulls from an untrusted index file (`set_deserialization_vector_byte_limit`). Three of the five SVS deserialization sites pass it down; the two SVS IVF sites do not, so it is ignored on every SVS IVF index. Impact: a corrupt or hostile IVF file can drive an unbounded read into the third-party SVS reader. The faiss fuzzers set this limit to stop that. Fix: pass `get_deserialization_vector_byte_limit()` at both sites, matching Flat and Vamana. Sentinel-Council-Run: council_1788372198_15590275 Sentinel-Harness: claude *Modify your team agent prompt, check stats, and leave feedback: https://www.internalfb.com/sentinel_agent/rotations/faiss* Model used: Claude Opus 5 (1M context), claude-opus-5[1m] Reviewed By: trang-nm-nguyen Differential Revision: D118524339 fbshipit-source-id: 4ed77fc6cd6a1894a1846cd427d299bd1c677f15 |
||
|
marcelo-cjl
|
256980cb6e |
Add SVE low-dimensional nearest fast path and spherical SuperKMeans support (#5530)
Summary:
This PR adds two related changes for the SCANN coarse quantizer / PQ encoding path on ARM SVE: a low-dimensional nearest-neighbor fast path and spherical SuperKMeans support.
### 1. Low-dimensional L2sqr nearest fast path (d in {2,4,8})
`fvec_L2sqr_ny_nearest<ARM_SVE>` previously wrote all `ny` distances to the scratch buffer and then performed a separate linear scan. For PQ encoding (`compute_1_code`) each call has `d = dsub` and `ny = ksub`, making this the hot path. The new path maps one centroid to each SVE lane, gathers its `D` components, and keeps lane-local minimum distances and ids across batches. A predicated tail handles non-multiple `ny` values, equal distances preserve the first index, and only one final horizontal reduction is needed. The scratch buffer is not written, matching the x86 AVX2/AVX512 `D2/D4/D8` implementations.
`ProductQuantizer::compute_code` already uses the A1 dispatch on current `main`, so the ARM_SVE specialization remains reachable from PQ encoding after this rebase.
### 2. SuperKMeans spherical (inner-product) support
SuperKMeans previously only supported L2 clustering; the header explicitly excluded IP/cosine. With `cp.spherical=true`:
- `TrainState::R` becomes `std::unique_ptr<VectorTransform>`; power-of-two `d` uses the fast `HadamardRotation`, L2 keeps `RandomRotationMatrix`.
- Forgy initialization produces unit centroids before the first assignment. Each iteration computes and splits centroids first, then applies spherical centroid post-processing once, matching the `Clustering` flow.
- `HadamardRotation::reverse_transform` fills the missing inverse.
- `ClusteringParameters::use_super_kmeans` (default false) lets `Level1Quantizer::train_q1` route coarse quantizer training through SuperKMeans when explicitly enabled; the Python type stubs expose the new option.
- `block_l2<ARM_SVE>` completes the SuperKMeans SIMD kernels for the ADSampling pruning loop.
## Tests
- Low-dim nearest across SIMD levels: `d` in `{2,4,8}` and `ny` around common SIMD widths through 257, including non-multiple tails and first-index tie behavior. The dispatch test source is included in DD builds.
- Spherical objective close to vanilla spherical Clustering; unit-norm centroids.
- `use_super_kmeans` field inherited from `ClusteringParameters`.
## Benchmark
qwen 4096-dim, IP, nlist=1024, sub_dim=4, 1 thread, 50k rows:
| stage | baseline | optimized | change |
| --- | ---: | ---: | --- |
| Train | 67.73s | 24.36s | -64% |
| Add | 10.86s | 7.17s | -34% |
| Build | 78.59s | 31.52s | -60% |
Train speedup comes from SuperKMeans spherical coarse quantizer training; Add speedup comes from the low-dim SVE nearest fast path in PQ encoding. recall@10 unchanged vs Clustering baseline (diff <= 0.0002).
## Notes
This change targets ARM SVE. The x86 (AVX2/AVX512) implementations already have equivalent low-dim nearest specializations. The generic ARM SVE distance path comes from current `main`; this PR adds the low-dimensional nearest specializations on top of it.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5530
Reviewed By: mnorris11
Differential Revision: D117563661
Pulled By: alibeklfc
fbshipit-source-id: 47ab1064d781a45d009923b4d5e62525718e83bf
|
||
|
Trang Nguyen
|
718002f7cf |
Fix distance conversion in IndexBinaryFromFloat for non-L2 metrics (#5549)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5549 `IndexBinaryFromFloat::search()` converted the wrapped float index's distances back to Hamming distances with a hardcoded `round(df[i] / 4.0)`, which is only correct for `METRIC_L2`. For any other metric the returned distances were silently wrong. `binary_to_real()` maps bits to `{-1, +1}`, so the relation between the float distance and the Hamming distance depends on the metric: - `METRIC_L2` (squared): `float_dist = 4 * hamming` - `METRIC_INNER_PRODUCT`: `float_dist = d - 2 * hamming` - `METRIC_L1`: `float_dist = 2 * hamming` Branch on `index->metric_type` and apply the matching inverse. `METRIC_L2` remains the default/fallback, so existing behavior is unchanged. Each mapping is monotonic in the Hamming distance, so the result order the wrapped index produces is already correct and needs no reordering. Reported in https://github.com/facebookresearch/faiss/issues/5546. ___ Differential Revision: D118160087 fbshipit-source-id: f2ce2b97ebfd47461346508f9878b1af7e321b12 |
||
|
generatedunixname1395027625275998
|
af9eeffcf7 |
Reserve cs vector capacity in hammings_knn_mc_impl to avoid reallocations (#5548)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5548 Reviewed By: rjaber Differential Revision: D118055043 fbshipit-source-id: 47f6ecdf6197610ccbcd60a466a3bfc77cfcc4e1 |
||
|
Chao Zhang
|
3a716ead1a |
Add Windows ARM64 support to CI (#5544)
Summary: Hi team, I'm from Microsoft and recently I'm working on improving Python ecosystem support for Windows on Arm. So I updates the CI workflows to add support for Windows ARM64 build and release. Could you please help to review? Thanks. **Windows ARM64 support and OpenBLAS installation improvements:** * Added a new PowerShell script `install-openblas.ps1` that automatically detects the runner architecture (x64 or ARM64), downloads the appropriate OpenBLAS binary, verifies its hash, and installs it to `C:\openblas`. This ensures consistent and secure OpenBLAS setup on all Windows runners. * Updated the build matrix in `.github/workflows/build-pip.yml` to include `windows-11-arm`, enabling CI builds and tests on Windows ARM64 runners. * Refactored the Windows `before-all` build step in `pyproject.toml` to use the new PowerShell script, replacing the previous hardcoded x64-only download command. This allows seamless support for both x64 and ARM64 architectures. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5544 Reviewed By: mnorris11 Differential Revision: D117818430 Pulled By: trang-nm-nguyen fbshipit-source-id: dac342878d063828437952e1b19f76e6699e1b6b |
||
|
mayuriphad
|
02ea14372c |
Fix range_search_max_results to respect all similarity metrics (#5533)
Summary: ## Bug `range_search_max_results()` (and its helper `apply_maxres()`) in `contrib/exhaustive_search.py` decide whether to keep the highest- or lowest-scoring results when pruning the accumulated results down to `max_results`, based on: ```python keep_max=index.metric_type == faiss.METRIC_INNER_PRODUCT ``` This hardcodes the assumption that `METRIC_INNER_PRODUCT` is the only "similarity" metric (higher = better). But `faiss.is_similarity_metric()` (see `faiss/MetricType.h`) also treats `METRIC_Jaccard` as a similarity metric, and this same, correct check is already used a few lines above in `range_search_gpu()` in the very same file (`keep_max = faiss.is_similarity_metric(index_gpu.metric_type)`). As a result, when `range_search_max_results` is used with a `METRIC_Jaccard` index and the result set grows past `max_results`, the pruning logic incorrectly discards the *highest*-similarity matches and keeps the *lowest*-similarity ones, and moves the returned radius in the wrong direction. ## Root cause Two call sites in `contrib/exhaustive_search.py` compare `index.metric_type` directly against `faiss.METRIC_INNER_PRODUCT` instead of using the library's own `faiss.is_similarity_metric()` helper, which is the single source of truth for which metrics are "larger is better". ## Fix Replace both occurrences of `index.metric_type == faiss.METRIC_INNER_PRODUCT` with `faiss.is_similarity_metric(index.metric_type)`, matching the existing pattern used in `range_search_gpu` in the same module. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5533 Test Plan: Added `test_query_iterator_similarity_metric` to `tests/test_contrib.py`. It exercises `range_search_max_results` against a lightweight fake index using `METRIC_Jaccard` (real `IndexFlat.range_search` doesn't support `METRIC_Jaccard` directly, so a fake index is used to isolate the pruning logic under test) and asserts that pruning keeps the highest-scoring results and raises the radius accordingly. - Confirmed the new test fails on `main` (old code keeps the *lowest*-similarity results and lowers the radius) and passes after the fix, by swapping the module in-place against an installed `faiss-cpu` and running the assertions directly (full repo `pytest` collection isn't possible in this environment without rebuilding the C++ extension for `main`, since the installed wheel is older and `tests/common_faiss_tests.py` on `main` references APIs from a newer build). - The fix is a pure two-line change that only affects the boolean passed to `apply_maxres`, verified by direct comparison of before/after behavior with the reproduction script. Reviewed By: mnorris11 Differential Revision: D117001619 Pulled By: alibeklfc fbshipit-source-id: 27e6757951d63a4d4f4f998640a1726ef8f6c220 |
||
|
Jeremy Schoemaker
|
bd7087453e |
fix(faiss): use is_similarity_metric for IndexShardsIVF merge (#5541)
Summary: IndexShardsIVF merges non-L2 distance metrics as similarities, reversing L1 and other metrics at IndexShardsIVF.cpp:230. The search method checked only METRIC_L2 for CMin and used CMax for everything else. Metrics such as L1, Linf, Lp, Canberra and BrayCurtis are distances and require CMin. Only inner product and Jaccard are similarities. Fix: replace the METRIC_L2 check with !is_similarity_metric(metric_type) so all distance metrics use CMin and only similarities use CMax. Add the MetricType header for the helper. Verified by syntax check and inspecting merge logic. The diff is minimal and does not change formatting elsewhere. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5541 Reviewed By: mnorris11 Differential Revision: D117542369 Pulled By: juancarpio27 fbshipit-source-id: 8cab4275cc9868c46100bc60ed14349544fd2aab |
||
|
William
|
561e6ecdf3 |
Merge shard results by metric type rather than assuming similarity (#5509)
Summary:
`IndexShardsTemplate::search` picked the comparator for merging per-shard results by testing for `METRIC_L2`:
```cpp
if (this->metric_type == METRIC_L2) {
merge_knn_results<idx_t, CMin<distance_t, int>>(...); // smaller is better
} else {
merge_knn_results<idx_t, CMax<distance_t, int>>(...); // larger is better
}
```
Everything that is not `METRIC_L2` was therefore merged as a similarity. `METRIC_L1`, `METRIC_Linf` and `METRIC_Lp` return distances, so once results crossed a shard boundary they came back farthest-first, while the same vectors in a single unsharded index came back nearest-first.
This PR selects the comparator with the existing `is_similarity_metric()` helper from `MetricType.h`, which is exactly the distinction the branch needs.
Fixes https://github.com/facebookresearch/faiss/issues/5503
## Behaviour change
`is_similarity_metric()` is true only for `METRIC_INNER_PRODUCT` and `METRIC_Jaccard`, so:
| metric | before | after |
| --- | --- | --- |
| `METRIC_L2` | `CMin` | `CMin` (unchanged) |
| `METRIC_INNER_PRODUCT` | `CMax` | `CMax` (unchanged) |
| `METRIC_L1`, `METRIC_Linf`, `METRIC_Lp`, `METRIC_Canberra`, `METRIC_BrayCurtis`, `METRIC_JensenShannon` | `CMax` | **`CMin`** |
| `METRIC_Jaccard` | `CMax` | `CMax` (unchanged) |
`IndexShardsTemplate<IndexBinary>` is also unaffected: `IndexBinary::metric_type` defaults to `METRIC_L2`, which keeps `CMin` as before.
## Reproduction
Against the released `faiss-cpu` 1.15.0 wheel, using the reporter's example:
```python
import faiss, numpy as np
xb0 = np.array([[0., 0.]], dtype='float32')
xb1 = np.array([[10., 0.]], dtype='float32')
xq = np.array([[1., 0.]], dtype='float32')
flat = faiss.IndexFlat(2, faiss.METRIC_L1)
flat.add(np.vstack([xb0, xb1]))
print(flat.search(xq, 2))
s0 = faiss.IndexFlat(2, faiss.METRIC_L1); s0.add(xb0)
s1 = faiss.IndexFlat(2, faiss.METRIC_L1); s1.add(xb1)
shards = faiss.IndexShards(2, False, True)
shards.add_shard(s0); shards.add_shard(s1)
print(shards.search(xq, 2))
```
```
single IndexFlat D,I = [[1. 9.]] [[0 1]] # nearest first, correct
IndexShards D,I = [[9. 1.]] [[1 0]] # farthest first, reversed
```
## Test
Adds `Shards::test_shards_distance_metric_ordering` to `tests/test_meta_index.py`. It splits the dataset across three `METRIC_L1` shards and checks that each result row is ordered nearest-first and matches the distances an unsharded `IndexFlat(METRIC_L1)` returns. Distances rather than labels are compared so that equidistant neighbours may be returned in either order.
The test fails on `main` (the rows come back reversed) and passes with this change.
## Testing notes
I reproduced the bug against the released 1.15.0 wheel as shown above, but I was not able to build faiss from source on this machine (Windows, no local C++ toolchain), so I have not executed the C++ build or run the test suite locally. The added test's pass/fail claim above follows from the comparator change rather than from a local run. Please treat CI as the gate, and I am happy to adjust if anything in the suite disagrees.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5509
Reviewed By: juancarpio27
Differential Revision: D117536893
Pulled By: mnorris11
fbshipit-source-id: 8568437eb596a202262fc455748063fafca202c1
|
||
|
Fu Xiaonan
|
613e0acac5 |
Fix IndexIDMap range search parameters (#5536)
Summary: - preserve the concrete search-parameter type when `IndexIDMap::range_search` translates an external-ID selector - retain IVF-specific per-query fields such as `nprobe` by temporarily replacing only `sel`, using the same scoped restoration pattern as `search_ex` - add a deterministic regression test with two IVF lists and external IDs ## Testing - confirmed the new test fails against FAISS 1.14.2 with `IndexIVF params have incorrect type` - built the current source on macOS arm64 with Python bindings (`FAISS_OPT_LEVEL=generic`, GPU and Metal disabled) - `python -m pytest -q tests/test_search_params.py` (51 passed) - `clang-format 21 --dry-run --Werror faiss/IndexIDMap.cpp` - `git diff --check` Closes https://github.com/facebookresearch/faiss/issues/5523 Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5536 Reviewed By: mnorris11 Differential Revision: D117233222 Pulled By: juancarpio27 fbshipit-source-id: 28e750f63fb681cc915ad41443c582739d8c3597 |