mirror of
https://github.com/facebookresearch/faiss.git
synced 2026-10-12 02:13:36 +00:00
28b2b66cf279ff2bdb7b8a7fd387c7ea328cb642
2000
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Matthijs Douze
|
28b2b66cf2 |
Inline PQ code distance kernels into scanner TUs (#5159)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5159 After the SIMD dispatch refactoring, the PQ code distance implementations (pq_code_distance_single_impl<SL>, pq_code_distance_four_impl<SL>) lived in separate translation units from the scanner loops (IVFPQScannerT::scan_list_with_table, PQDistanceComputer::distance_to_code). The compiler could not inline the SIMD gather/accumulate code into the hot inner loops. This diff converts the per-SIMD pq_code_distance .cpp files to .h headers and includes them in the corresponding scanner TUs before the scanner _impl.h includes. This puts the kernel definitions in the same TU, enabling the compiler to inline the AVX2/AVX512 vgatherdps code directly into scan_list_with_table and scan_list_polysemous_hc. Changes: - pq_code_distance-avx2.cpp → pq_code_distance-avx2.h (header, #pragma once) - pq_code_distance-avx512.cpp → pq_code_distance-avx512.h (same) - New pq_code_distance-generic.h with inline NONE/ARM_NEON specializations - Scanner TUs (avx2.cpp, avx512.cpp, neon.cpp) include the PQ distance headers - PQCodeDistance gains static constexpr simd_level member - scan_list_polysemous uses PQCodeDist::simd_level for Hamming computer dispatch (was hardcoded to SIMDLevel::NONE) - Scanner TUs include per-ISA hamming_computer headers for SIMD Hamming dispatch - Build files (xplat.bzl, CMakeLists.txt) updated Verified via objdump that scan_list_with_table and scan_list_polysemous_hc contain zero calls to pq_code_distance_*_impl — all AVX2/AVX512 gather code is fully inlined. Reviewed By: algoriddle Differential Revision: D102942787 fbshipit-source-id: d2979f9cd629652ac1de4886afd3c335fbb4fac8 |
||
|
Gergely Szilvasy
|
417c53e0ce |
Add NoneSIMDLevel context manager for cross-level reference checks (#5158)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5158 Several DD-dispatched code paths produce integer outputs (codes, top-k indices) where bit-exact equivalence between SIMD levels is the right correctness bar, but the existing tests assert only level- independent properties (decode error, recall) that wouldn't catch silent dispatch drift between AVX2/AVX512/NEON/NONE implementations. Adds NoneSIMDLevel context manager in common_faiss_tests.py: codes = quantizer.compute_codes(xb) with NoneSIMDLevel(): codes_none = quantizer.compute_codes(xb) np.testing.assert_array_equal(codes, codes_none) The context manager pins SIMDConfig to NONE for the duration of the block, restores the previous level on exit (incl. exception), and raises SkipTest when NONE isn't compiled in (static-AVX2 lane). Adds test_codes_match_none / test_PQ4_search_matches_none methods to five existing for_all_simd_levels test classes: * TestScalarQuantizerEncodeDecode (test_scalar_quantizer_correctness) * TestProductQuantizer (test_product_quantizer) * TestRaBitQuantizerEncodeDecode (test_rabitq) * TestSearch (test_fast_scan; checks (D, I) output of PQ16x4fs search) * TestResidualQuantizer (test_residual_quantizer; replaces inlined set_level/restore from D102821184 with the new helper) Caveat documented in the docstring: SkipTest raised inside subTest is swallowed; tests using both must do an explicit availability check before the loop. Applied where needed. Reviewed By: mdouze Differential Revision: D102989453 fbshipit-source-id: 475ee504271e0505615841ef82263776245419e0 |
||
|
Ishwar Bhati
|
46def46902 |
Enable SVS IVF Index in FAISS (#4801)
Summary:
This PR introduces SVS IVF (Inverted File) index support in FAISS, complementing the existing SVS Vamana graph‑based index. The implementation integrates Intel's Scalable Vector Search (SVS) runtime to provide a high‑performance, partition‑based ANN index with strong scalability on modern hardware.
## Key Advantages of SVS IVF
- **AMX‑accelerated builds** for significantly faster index training on supported Intel platforms
- **Hierarchical clustering** (default ON) enabling much faster k‑means on large datasets
- **LVQ and LeanVec compression** applied at search time to boost throughput while preserving recall
- **Intra-query parallelism** to reduce latency for small‑batch or single‑query workloads
- **Static index mode** for read‑only workloads with optimized memory layout and search performance
## New Factory Strings
| Factory String | IndexClass | Storage |
|---|---|---|
| `SVSIVF{nlist}` | IndexSVSIVF | FP32 (default) |
| `SVSIVF{nlist},FP16` | IndexSVSIVF | Float16 |
| `SVSIVF{nlist},SQI8` | IndexSVSIVF | Scalar quantized int8 |
| `SVSIVF{nlist},LVQ4x0` | IndexSVSIVFLVQ | LVQ primary=4, no residual |
| `SVSIVF{nlist},LVQ4x4` | IndexSVSIVFLVQ | LVQ primary=4, residual=4 |
| `SVSIVF{nlist},LVQ4x8` | IndexSVSIVFLVQ | LVQ primary=4, residual=8 |
| `SVSIVF{nlist},LVQ8x0` | IndexSVSIVFLVQ | LVQ primary=8, no residual |
| `SVSIVF{nlist},LeanVec4x4_{dims}` | IndexSVSIVFLeanVec | LeanVec 4x4 |
| `SVSIVF{nlist},LeanVec4x8_{dims}` | IndexSVSIVFLeanVec | LeanVec 4x8 |
| `SVSIVF{nlist},LeanVec8x8_{dims}` | IndexSVSIVFLeanVec | LeanVec 8x8 |
## Static vs Dynamic IVF Index
SVS IVF supports two modes controlled by the `is_static` constructor parameter:
### Dynamic IVF (default, `is_static = false`)
- Supports `train()`, `add()`, `remove_ids()`, and `search()`
- Vectors can be added or removed after index creation
- Suitable for workloads where the dataset changes over time
### Static IVF (`is_static = true`)
- **All data must be provided during `train()`** — the index is built and populated in a single step
- `add()` and `remove_ids()` will throw an exception after creation
- Optimized for **read‑only workloads** where the dataset is fixed at build time
- Uses `svs_runtime::IVFIndex` (static) instead of `svs_runtime::DynamicIVFIndex`
- Serialization and deserialization are fully supported
**Usage Examples(C++):**
```cpp
// Dynamic IVF with intra-query parallelism
faiss::IndexSVSIVF index(d, nlist);
// intra_query_threads must be set BEFORE train() — changes after
// train() are not propagated to the SVS backend (current SVS runtime
// API limitation, will be fixed in a future SVS version).
index.intra_query_threads = 4;
index.n_probes = 20;
index.train(n, train_data);
index.add(n, base_data);
index.search(nq, queries, k, distances, labels);
// num_threads CAN be changed between searches (synced at search time).
index.num_threads = 8;
index.search(nq, queries, k, distances, labels);
// Static IVF (read-only, all data at build time)
faiss::IndexSVSIVF index(d, nlist, faiss::METRIC_L2,
faiss::SVSStorageKind::SVS_FP32, /*is_static=*/true);
index.intra_query_threads = 4; // set before train()
index.train(n, data); // builds and populates the index
index.search(nq, queries, k, distances, labels);
// index.add(...) // throws — not supported on static index
// index.remove_ids(...) // throws — not supported on static index
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4801
Reviewed By: limqiying
Differential Revision: D102536698
Pulled By: mnorris11
fbshipit-source-id: cbc2b882c1f7728ed54a0bd25ceacb225d5a6c74
|
||
|
Alibek Zhakubayev
|
585ba7954a |
Add pip install support via scikit-build-core + cibuildwheel (#4862)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4862 Add official pip wheel packaging for `faiss-cpu`, enabling `pip install faiss-cpu` from PyPI. Builds binary wheels across Linux x86_64/aarch64, macOS arm64/x86_64, and Windows x86_64 for Python 3.10-3.14. Linux and macOS use Python Stable ABI (abi3) — a single cp310-abi3 wheel per platform works on all Python 3.10+ versions without per-version rebuilds. **BLAS strategy:** - Linux (x86_64 and aarch64): Threaded OpenBLAS (`libopenblaso`). The CMake build prefers `libopenblaso` (OpenMP-threaded) or `libopenblasp` (pthreads-threaded) over the serial default on RHEL/Fedora-based manylinux containers. - macOS: Apple Accelerate (found automatically). - Windows: Prebuilt OpenBLAS from GitHub releases (v0.3.30). **SIMD optimization strategy:** - Linux and macOS: Dynamic Dispatch (`FAISS_OPT_LEVEL=dd`) compiles SIMD-hot kernel files at multiple ISA levels (generic + AVX2 + AVX-512 on x86_64, NEON + SVE on aarch64) into a single library. Runtime CPUID detection selects the optimal code path automatically. - Windows: MSVC does not support DD, so uses `FAISS_OPT_LEVEL=generic`. Shared libraries (`BUILD_SHARED_LIBS=ON`) are used to avoid MSVC's single-pass static archive scanning issue. **New files:** - `pyproject.toml`: scikit-build-core build backend config with cibuildwheel settings. Version is dynamically extracted from CMakeLists.txt. Enables abi3 on Linux/macOS via SWIG 4.2+ and NumPy 2.0+ Limited API support. Enables LTO and dead-code stripping for smaller wheel sizes. - `.github/workflows/build-pip.yml`: CI workflow using cibuildwheel to build binary wheels on 5 platform runners. All GitHub Actions are pinned to commit SHAs for supply chain security. Publishes wheels to PyPI via OIDC trusted publishers on tag push. - `THIRD_PARTY_NOTICES`: License notices for bundled dependencies (OpenBLAS, LLVM OpenMP). - `tests/test_wheel_smoke.py`: 12-test smoke suite validating import, OpenMP, BLAS (FlatL2/FlatIP), index factory (IVF+PQ), HNSW, serialization roundtrip, GC safety, contrib imports, and SIMD level detection. **Modified files:** - `faiss/CMakeLists.txt`: Added threaded OpenBLAS fallback — prefers `libopenblaso` (OpenMP-threaded) or `libopenblasp` (pthreads-threaded) over the serial default on RHEL/Fedora-based manylinux containers. - `python/CMakeLists.txt`: Added `install()` targets for scikit-build-core wheel packaging, including all Dynamic Dispatch SWIG variants (avx2/avx512/avx512_spr/sve). Added abi3 support: conditionally uses `Python::SABIModule`, defines `Py_LIMITED_API`, and sets `.abi3.so` suffix when `SKBUILD_SABI_VERSION` is set. - `.github/workflows/build.yml`: Wired `build-pip.yml` into the root CI trigger so pip builds run alongside conda builds. - `tests/BUCK`: Added `python_pytest` target for the smoke test. Reviewed By: mnorris11 Differential Revision: D95258115 fbshipit-source-id: 3593ed4fa5c1d8b7bebfc61c933bb3204c9666eb |
||
|
Alibek Zhakubayev
|
9942229393 |
Fix add_sa_codes silently accepting non-int64 ids, corrupting stored labels (#5171)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5171 **Problem 1 — add_sa_codes silently stores wrong ids when passed non-int64 input** replacement_add_sa_codes in class_wrappers.py passed the ids array directly to swig_ptr without dtype coercion. The C++ signature is add_sa_codes(idx_t n, const uint8_t* codes, const idx_t* xids) where idx_t = int64_t (8 bytes per element). When the caller passed int32 ids — the default output of np.arange(n) on most platforms — swig_ptr returned a pointer to 4-byte data. The C++ layer then read every 8 consecutive bytes as one int64, producing garbage labels for every stored vector. No exception was raised, no warning was emitted. This is inconsistent with every other ids-accepting wrapper in the same file. replacement_add_with_ids, replacement_remove_ids, and replacement_permute_entries all call np.ascontiguousarray(ids, dtype='int64') before swig_ptr. add_sa_codes was the only exception. Callers using np.arange(n) without an explicit dtype — common in tutorials and quick-start code — silently ingested a corrupt index. Fix: add ids = np.ascontiguousarray(ids, dtype='int64') before swig_ptr(ids), matching the pattern used by all other ids-accepting wrappers. The call is a no-op for callers already passing int64, so there is no behavior change for correct inputs. **Problem 2 — sa_encode calls sa_code_size() twice (one redundant SWIG dispatch)** replacement_sa_encode called self.sa_code_size() once in the if codes is None branch and once in the else branch. Since sa_code_size() is a virtual C++ method accessed via SWIG, this is one avoidable cross-boundary call per sa_encode invocation. The value is a pure function of index state and cannot change within a call. replacement_sa_decode in the same block already uses the correct pattern, reusing the code size from codes.shape. The inconsistency between sa_encode and sa_decode confirms this was an oversight. Fix: hoist self.sa_code_size() into code_size = self.sa_code_size() once before the branch, matching the pattern in sa_decode. Reviewed By: limqiying Differential Revision: D103384269 fbshipit-source-id: 3f28a18b7e951c24d23db676be475bb6b298df2f |
||
|
Alexandr Guzhva
|
799bf3aaec |
Introduce early stop facilities for IVF, attempt 2 (#5160)
Summary: # Introduce early-stop controls for IVF search This PR adds optional early-stop controls to IVF search through `SearchParametersIVF`. IVF search looks in a sequence of buckets. Normally it keeps looking until it has checked all requested buckets. This PR adds a few ways to stop earlier: - stop after enough candidate vectors have been checked, - stop FastScan after enough buckets have been checked, - stop range search after several nearby buckets produce no results. All new fields default to disabled, so existing searches keep their current behavior unless callers opt in. ## New SearchParametersIVF fields Default values: | Field | Default | Effect | |---|---:|---| | `ensure_topk_full` | `false` | disabled; keep the existing hard `max_codes` behavior | | `max_lists_num` | `0` | disabled; FastScan is bounded only by `nprobe` | | `max_empty_result_buckets` | `0` | disabled; range search scans all requested probes | `ensure_topk_full` For k-NN search, `max_codes` limits how many candidate vectors are scanned. With `ensure_topk_full=false`, the search stops when the budget is reached. With `ensure_topk_full=true`, very small budgets become less aggressive: `max_codes` is treated as at least `k` post-filter scans. This helps when `max_codes < k`, or when an `IDSelector` filters out many candidates before a distance is computed. In the generic IVF path this is supported for `parallel_mode` 0 and 3. In FastScan k-NN it is supported by the per-query implementations 10 and 11. `max_lists_num` FastScan k-NN can also stop after visiting a limited number of inverted lists. `0` means disabled. With `ensure_topk_full`, this list budget is treated as at least `k` lists. This is only used by the FastScan per-query path (`search_implem_10`, also used for implementation 11). When FastScan is left on `implem = 0`, setting an early-stop k-NN option makes the dispatcher choose the per-query implementation automatically. If an incompatible FastScan implementation is selected explicitly, Faiss throws instead of ignoring the option. `max_empty_result_buckets` This is for range search. It stops probing a query after that many consecutive probed lists add no in-radius results. The counter resets as soon as a list adds at least one result. `0` means disabled. Setting this trades recall for less work: once nearby buckets repeatedly produce no hits, farther buckets are less likely to help. The generic IVF range path supports this in `parallel_mode == 0`. FastScan range search also supports it through implementation 10. With `implem = 0`, FastScan selects implementation 10 automatically when this option is set; explicit incompatible implementations throw. ## Selector-aware scan accounting This PR adds `InvertedListScannerStats`, returned by inverted-list scanner methods. It currently reports: - `scan_cnt`: number of distances actually computed, - `nheap_updates`: number of result updates. The important behavior change is that `IndexIVFStats::ndis` now uses `scan_cnt`. With an `IDSelector`, candidates rejected by the selector are not counted as computed distances. Without a selector, this matches the previous count because every scanned code reaches distance computation. This same post-filter count is used by `ensure_topk_full`, so a restrictive selector no longer consumes the search budget with candidates that were never eligible for the result heap. ## FastScan behavior FastScan k-NN now applies `max_codes`, `max_lists_num`, and `ensure_topk_full` in the per-query implementation. The code keeps separate counters per query, so multi-query batches have independent budgets. For `max_codes`, FastScan uses the result handler's scanned-row count. This means filtered-out rows do not consume the code budget. FastScan range search rejects k-NN-only options (`max_codes`, `max_lists_num`, and `ensure_topk_full`). It accepts `max_empty_result_buckets` when the selected implementation can preserve per-query probe order. ## API exposure The new fields are exposed through: - C++ `SearchParametersIVF`, - Python bindings, - the C API. The statistics is tracked via `InvertedListScanner::scan_codes(..., ResultHandler& handler)` call. In order to make it work inside `IndexIVF::search()`, it has been decided to transition it into this particular override of `InvertedListScanner::scan_codes()`. As a result, non-`ResultHandler` version of `scan_codes()` has been marked as non-virtual, so the custom `InvertedListScanner` implementations should override `ResultHandler`-based versions only. Another consequence is that `IndexIVFPQFastScanScanner` and `IndexIVFRaBitQFastScanScanner` accept only `HeapResultHandler` types of `ResultHandler`. These are the compromises that had to be made. ## Tests The tests cover: - default parameters preserving baseline k-NN and range-search results, - `ensure_topk_full` filling more result slots under tight budgets, - selector-aware `IndexIVFStats::ndis`, - range-search early exit and counter reset behavior, - unsupported parallel modes and FastScan implementations, - FastScan k-NN and range-search early-stop behavior, - Python and C API field exposure. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5160 Reviewed By: limqiying Differential Revision: D103044128 Pulled By: mnorris11 fbshipit-source-id: 80f1241c13ffc8173a3da0f2974fa9b6dd318890 |
||
|
Michael Norris
|
812010fe19 |
Remove flaky test (#5169)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5169 Created from CodeHub with https://fburl.com/edit-in-codehub ___ overriding_review_checks_triggers_an_audit_and_retroactive_review Oncall Short Name: faiss Differential Revision: D103406045 fbshipit-source-id: 20ab4bc62548b64c47014ccedd4496af9d0f4eb7 |
||
|
Binozo
|
c5ddbc7104 |
c_api: fix IndexShards own_indices getter/setter name mismatch (#5165)
Summary: `IndexShards_c.cpp` defines: ```cpp DEFINE_GETTER(IndexShards, int, own_indices) DEFINE_SETTER(IndexShards, int, own_indices) ``` while `IndexShards_c.h` defines ```cpp FAISS_DECLARE_GETTER_SETTER(IndexShards, int, own_fields) ``` this mismatch causes neither `faiss_IndexShards_set_own_indices` and `faiss_IndexShards_set_own_fields` to be exported in the c_api. This patch completes the migration to `indices` instead of `fields` in the c_api. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5165 Reviewed By: mnorris11 Differential Revision: D103238014 Pulled By: trang-nm-nguyen fbshipit-source-id: 5a972cc6b6611088f06e3388677e0cd10d9037c0 |
||
|
Alibek Zhakubayev
|
66c9d08265 |
Add SuperKMeans foundations: math primitives, PDX layout, SIMD kernels
Summary:
Adds the foundational building blocks for SuperKMeans — a faster k-means alternative to `faiss::Clustering` for IVF training and other large-k workloads (diff [2/3] adds the algorithm itself; diff [3/3] wires the IVF integration and Python binding). This diff is pure infrastructure with zero impact on existing FAISS behavior.
## Why these components exist
SuperKMeans replaces the brute-force nearest-centroid assignment step of Lloyd's k-means with a two-phase scheme: a partial GEMM over the leading d/8 dimensions followed by progressive pruning over the remaining dimensions. This requires four new primitives that do not exist in the FAISS codebase today:
1. **A chi-squared statistical threshold for pruning** — to decide, after computing partial L2 distances over only the first d_prime dimensions, whether a centroid is statistically unlikely to be the nearest. The threshold comes from the chi-squared distribution's quantile function, which in turn requires the inverse standard normal CDF.
2. **A cache-friendly data layout for the trailing dimensions** — the pruning phase sweeps across surviving centroids in 64-dim blocks. Standard row-major layout causes cache misses because each centroid's block is far from the next. A block-column-major (PDX) layout interleaves all k centroids' values for the same block contiguously, making the per-block L2 sweep cache-friendly.
3. **SIMD-optimized block L2 distance kernels** — the inner loop of the pruning phase computes L2 distance over 64-dim PDX blocks. Vectorizing this with AVX2/AVX-512 intrinsics is where the wall-time reduction comes from (the pruning logic itself is scalar; only the distance kernel needs SIMD).
4. **Shared clustering helpers** — SuperKMeans reuses four functions from vanilla `Clustering` (subsample, split_clusters, compute_centroids, get_actual_rng_seed). These are currently in Clustering.cpp's anonymous namespace. Lifting them into `faiss::detail::` avoids code duplication.
## What is added
### faiss::detail::normal_quantile and chi2_quantile_wh (impl/AdSampling.{h,cpp})
Three pure-math functions under `faiss::detail::`:
- `normal_quantile(p)`: inverse standard normal CDF. Three-branch rational polynomial approximation (Acklam). Absolute error < 1.15e-9. No external dependencies (no scipy, no Boost).
- `chi2_quantile_wh(p, alpha)`: chi-squared quantile via the Wilson-Hilferty cube-root approximation. For degrees of freedom p >= 16 (the minimum d_prime SuperKMeans uses), the approximation is within 2% of scipy.stats.chi2.ppf. Domain constraint documented: for very small alpha and small p (< 4), the cube can go negative; in practice p >= 16 and alpha >= 0.9375 so this never happens.
- `precompute_ad_thresholds(d, epsilon)`: builds a per-prefix-length coefficient table of size d+1. Entry coeff[p] is the normalized chi-squared quantile at significance epsilon for p degrees of freedom. The SuperKMeans pruning loop reads coeff[d_prime] at the GEMM boundary and coeff[true_block_end] at each PDX block boundary. Input validated via FAISS_THROW_IF_NOT (not debug-only assert).
### faiss::detail::pdxify / de_pdxify (impl/PdxLayout.{h,cpp})
Row-major to block-column-major (PDX) transform and its inverse. Given k centroids in d_trail trailing dimensions with block_size = 64:
- Row-major: centroid j's dims are at `Y[j * d_trail ... j * d_trail + d_trail - 1]`
- PDX: block b contains all k centroids' dims `[b*64 ... (b+1)*64-1]` interleaved as `Y_pdx[offset + j * block_dim + dim_in_block]`
The trailing block handles `d_trail % 64` dims (non-power-of-2 safe). Both transforms are pure memcpy — no arithmetic, no rounding, bit-exact inverse.
Also provides `compute_partial_norms(X, n, d, p, norms)`: computes `||X[i, 0:p]||^2` for the L2-from-IP identity used in the GEMM phase.
### faiss::detail:: clustering helpers (impl/ClusteringHelpers.{h,cpp})
Four functions lifted from `Clustering.cpp`'s anonymous namespace into `faiss::detail::`:
- `subsample_training_set`: cap training data to k * max_points_per_centroid via random permutation
- `split_clusters`: replace empty centroids with perturbations of the largest cluster
- `compute_centroids`: partition-by-centroid OpenMP reduction (no cross-thread float accumulation, so bit-deterministic at fixed thread count)
- `get_actual_rng_seed`: resolve negative seed to high-resolution clock
Pure refactor. Existing call sites in Clustering.cpp updated to qualified names. Zero behavior change — verified by existing test_clustering (13 Python tests) passing unchanged.
### Per-arch block_l2 SIMD kernels (utils/simd_impl/super_kmeans_*.{h,cpp})
The inner loop of SuperKMeans's pruning phase: compute L2 distance over a PDX block of n floats (typically 64, but can be smaller for the trailing block).
- **Primary template** (super_kmeans_kernels.h): scalar fallback. Any architecture (x86, aarch64, anything) compiles and runs correctly via this path.
- **AVX2 specialization** (super_kmeans_kernels_avx2.cpp): 8-wide `_mm256` loop. Horizontal reduction uses a shuffle+add tree (`_mm_movehdup_ps` / `_mm_movehl_ps` / `_mm_add_ss`), not `_mm_hadd_ps` (which has 3-cycle latency on Skylake-class cores vs single-cycle for each shuffle/add).
- **AVX-512 specialization** (super_kmeans_kernels_avx512.cpp): 16-wide `_mm512` loop with `_mm512_fmadd_ps`. Horizontal reduction via `_mm512_reduce_add_ps` (a compiler pseudo-intrinsic that GCC/Clang expand to a shuffle+add tree — no separate manual reduction needed).
- **Runtime dispatch** (super_kmeans_dispatch.h): `block_l2_dispatch()` uses the existing FAISS `with_selected_simd_levels` template machinery to select AVX-512 > AVX2 > scalar at process startup.
Known perf gap: aarch64 (NEON/SVE) specializations are not implemented. aarch64 falls through to the scalar primary template. Validating SVE requires a Graviton-class host; deferred to a focused follow-up.
## Existing code changes
`Clustering.cpp`: the four helper functions are now called via `faiss::detail::subsample_training_set(...)` etc. instead of unqualified calls within the anonymous namespace. The function bodies are moved to ClusteringHelpers.cpp. No logic changes, no behavioral changes.
Next in stack: [2/3] adds the SuperKMeans class that consumes all of these primitives.
Reviewed By: mnorris11
Differential Revision: D102713322
fbshipit-source-id: ba5943843b4a3270dd78b6e5421e1ba1f92e33ac
|
||
|
Evandabest
|
66cea52433 |
Add Metal GPU backend for Apple Silicon (IndexFlat) (#5144)
Summary: ## Summary Adds a new Metal compute backend so FAISS can run GPU-accelerated similarity search on Apple Silicon (M1/M2/M3/M4/M5). This is just an initial pr, it introduces the build system integration, core Metal infrastructure, and a working `IndexFlat` with L2 and inner product search. I have some follow up prs with more functionality that I plan to submit. Addresses https://github.com/facebookresearch/faiss/issues/2386 — FAISS currently has no GPU acceleration path on Apple Silicon, forcing CPU-only execution. This Metal backend enables GPU-accelerated search on M-series chips, with follow-up PRs adding IVF indexes, Python bindings, and kernel optimizations. All Metal code lives in a new `faiss/gpu_metal/` directory, keeping `faiss/gpu/` (CUDA/ROCm) completely untouched. The backend is opt-in via `cmake -DFAISS_ENABLE_METAL=ON -DFAISS_ENABLE_GPU=OFF`. ### What's included - **Build system**: New `FAISS_ENABLE_METAL` CMake option. When enabled, requires Apple platform, links Metal/MetalKit/MPS/Foundation frameworks, and builds `libfaiss_metal.a`. Three small additions to root `CMakeLists.txt`. - **MetalResources**: Owns `MTLDevice`, `MTLCommandQueue`, and buffer allocation/deallocation. Mirrors `GpuResources` roles. - **MetalIndex**: Base class for Metal-backed indexes (mirrors `GpuIndex`). Single-device only (device 0). - **MetalIndexFlat**: Flat index storing vectors in an `MTLBuffer`. Supports `add`, `add_with_ids`, `reset`, `search` for both `METRIC_L2` and `METRIC_INNER_PRODUCT`. - **MSL compute kernels**: `l2_squared_matrix`, `ip_matrix` (distance computation), and `topk` (sorted insertion, k ≤ 256). Embedded as a compile-time string in `MetalFlatKernels.mm`. - **Backend abstraction**: `get_num_gpus()`, `index_cpu_to_metal_gpu()`, `index_metal_gpu_to_cpu()`, `StandardMetalResources`, and a `GpuIndexFlat` typedef for API parity. - **Tests**: 8 gtest cases in `TestMetalIndexFlat.mm` — L2/IP search correctness vs CPU `IndexFlat`, `add_with_ids`, reset, empty search, `get_num_gpus`, and CPU↔Metal cloning in both directions. All tests skip gracefully if no Metal device is available. - Python/SWIG bindings ### Planned follow up prs - Larger k support (tiling, heap-based top-k) - IVFFlat, IVFPQ, IVFSQ, BinaryFlat - Kernel optimizations (fused distance+top-k, GEMM-tiled distance, simdgroup/threadgroup parallel selection) ### Build and test ```bash mkdir -p build_metal && cd build_metal cmake .. -DFAISS_ENABLE_METAL=ON -DFAISS_ENABLE_GPU=OFF \ -DBUILD_TESTING=ON -DFAISS_ENABLE_PYTHON=OFF \ -DCMAKE_PREFIX_PATH="/opt/homebrew/opt/libomp;/opt/homebrew" cmake --build . --target faiss_metal TestMetalIndexFlat ctest -R TestMetalIndexFlat ``` > The `mkdir -p` avoids the error if the directory exists, and `-DFAISS_ENABLE_PYTHON=OFF` skips the Python dependency that isn't needed for this PR. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5144 Reviewed By: alibeklfc Differential Revision: D102284936 Pulled By: mnorris11 fbshipit-source-id: 812fc6ceae251b7b33d25f6861fc37b8d7b475c1 |
||
|
ivan-digital
|
17fd3332c7 |
Add database-parallel flat search for few-query workloads (#5000)
Summary: When the number of queries (`nx`) is smaller than the available thread count, the current flat search underutilizes CPU cores — each thread scans the entire database for a single query. This PR adds a database-parallel code path that divides the database across threads instead, with per-thread BLAS calls and heap merging. - Activates automatically when `nx < omp_get_max_threads()` and the database is large enough - Falls back to the existing path when an `IDSelector` is active or the database is too small to benefit - Works for both `METRIC_INNER_PRODUCT` (CMin) and `METRIC_L2` (CMax) ## Benchmarks 12 threads, `d=128`, `k=10`: | Index | ny | nx | Before | After | Speedup | |-------|------|------|--------|-------|---------| | FlatIP | 1,000,000 | 4 | 56.7 ms | 8.4 ms | **6.7x** | | FlatIP | 1,000,000 | 1 | 14.1 ms | 5.0 ms | **2.8x** | | FlatL2 | 1,000,000 | 4 | 56.7 ms | 14.0 ms | **4.0x** | | FlatL2 | 1,000,000 | 1 | 14.1 ms | 9.5 ms | **1.5x** | | FlatIP | 200,000 | 4 | 11.5 ms | 1.6 ms | **7.0x** | | FlatL2 | 200,000 | 4 | 11.4 ms | 2.8 ms | **4.2x** | No regression for `nx >= nthreads` (existing path is unchanged). ## Motivation Addresses https://github.com/facebookresearch/faiss/issues/4121. The original issue demonstrated that for few-query workloads over large databases, parallelizing over database segments instead of queries can yield multi-fold speedups. This is especially relevant for serving workloads where queries arrive one at a time. Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5000 Test Plan: - [x] All existing `TestIndexFlat` tests pass (15/15) - [x] Full `test_index.py` passes (47/47 including 9 new tests) - [x] `test_search_params.py` + `test_index_composite.py` pass (81/81) - [x] New `TestDbParallelSearch` covers: IP, L2, k=1, k=200, single query, few queries, thread scaling consistency - [x] Cross-validated against NumPy brute-force for IP and L2 Reviewed By: mdouze Differential Revision: D99676059 Pulled By: mnorris11 fbshipit-source-id: 3fbd17a71b446a4b4932d53d9ebc254c4510a0ca |
||
|
Tom Jackson
|
c41405d525 |
Enforce minimum training set size for ScalarQuantizer, check nullptr (#5141)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5141 Reviewed By: mnorris11 Differential Revision: D102078734 fbshipit-source-id: f2c9899b5a34b620a359e01dc1e530bfdd5aa960 |
||
|
Trang Nguyen
|
8380e2543d |
Fix avx512 unit test (#5161)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5161 T225351715 fix task as part of oncall duty Reviewed By: mnorris11 Differential Revision: D103035738 fbshipit-source-id: 70b0400be265fe1595ec880ee8cc92fbc177e924 |
||
|
Trang Nguyen
|
71448c07d3 |
Fix issue with svs tests (#5162)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5162 T225351715 Fixing failed unit test as part of oncall duty. test_svs_py are failing due to timeout. In this diff, we are fixing 2 issues with the tests: 1. setUp runs before every test method — with 11 tests per class × 8 classes = ~57 times, each regenerating the same numpy arrays. setUpClass runs once per class, so data is generated ~8 times instead of ~57. The data is deterministic (fixed np.random.seed(1234)) and no test mutates xb/xq — tests only read them. So regenerating identical arrays per-method is pure waste. 2. Reduce test size Reviewed By: mnorris11 Differential Revision: D103043664 fbshipit-source-id: ac79b23910c2d936168dc7021b06f57c47a32782 |
||
|
Michael Norris
|
44c1972aec |
Enable ccache in GitHub Actions cmake builds (#5157)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5157 TLDR - ccache is like how BUCK works with partial cached artifacts for builds, but in the OSS world. https://github.com/facebookresearch/faiss/actions/runs/25118621287/job/73635614448?pr=5157 it sped up the jobs a lot, cuVS down to 40 minutes! AI stuff - Add ccache to the `build_cmake` composite action to speed up CI builds. Uses `hendrikmuhs/ccache-action@v1` to install ccache, restore/save caches keyed by build configuration (OS, arch, opt level, GPU, cuVS, ROCm, SVS), and set a 2GB max cache size. Passes `CMAKE_C_COMPILER_LAUNCHER`, `CMAKE_CXX_COMPILER_LAUNCHER`, and `CMAKE_CUDA_COMPILER_LAUNCHER` to cmake so all C, C++, and CUDA compilations go through ccache. On first run the cache will be cold and there is no speedup. Subsequent runs on the same branch (or PRs against main) will see cache hits for unchanged translation units, which should significantly reduce build times for incremental CI runs. Reviewed By: junjieqi Differential Revision: D102926935 fbshipit-source-id: b8808a3a48edee271547a0421960261aba6ff425 |
||
|
Gergely Szilvasy
|
c7f07bbc0a |
Add C++ per-SIMD-level tests for distance utility functions (#5154)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5154 The templated distance functions in faiss/utils/distances.h -- fvec_L1, fvec_Linf, fvec_L2sqr_batch_4, fvec_inner_product_batch_4, fvec_L2sqr_ny_transposed, fvec_L2sqr_ny_nearest -- are exercised transitively by IndexFlat search at the binary's compiled/dispatched SIMD level. Direct per-level tests catch tail-handling bugs that the typical aligned-d transitive coverage masks. Adds tests/test_distances_dispatch.cpp with one TEST per function, sweeping every SIMDConfig-available level via SIMDConfig::set_level() and asserting equivalence against the SIMDLevel::NONE reference. Dimensions sweep {1, 3, 7, 8, 9, 15, 16, 17, 31, 32, 33, 64} so non- multiple-of-8 dims exercise the tail paths. Tolerances are calibrated to function semantics: * fvec_L2sqr_ny_nearest returns size_t; compared bit-exactly via EXPECT_EQ on the index. The internal distances_tmp_buffer is not part of the public contract (some SIMD impls don't write into it), so it isn't compared. * fvec_Linf is a max, not a sum, so reordering doesn't change the result; compared bit-exactly with EXPECT_NEAR(0.0). * The L1 / batch_4 / L2sqr_ny_transposed variants accumulate over d terms; SIMD reductions can differ by a few ULPs from scalar. Tolerance scales linearly with d (1e-6 per term, floor 1e-6). For fvec_L2sqr_ny_transposed the test uses the contract layout y[i + k * d_offset] (per distances_simd.cpp:62) and picks d_offset = ny + 5 so the stride is non-trivial AND every (i, k) access in [0, ny) x [0, d) is in-bounds. NaN/Inf inputs are out of scope: rand_vec produces uniform [-1, 1]. ___ overriding_review_checks_triggers_an_audit_and_retroactive_review Oncall Short Name: fair_umami_cluster Differential Revision: D102821270 fbshipit-source-id: d25a9da80f5c623487bdf4868a107d2b59dfe833 |
||
|
Gergely Szilvasy
|
0242758a3d |
Add bit-exact RQ encode cross-level test (#5155)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5155 The for_all_simd_levels decorator (D101822335) makes existing tests run once per SIMD level, but only catches dispatch drift when the test's correctness assertions compare against a level-independent reference. RQ encode (residual_quantizer_encode_steps.cpp:96,369, dispatched via with_simd_level_256bit) is integer-valued, but test_residual_quantizer's existing assertions are tolerance-based -- silent dispatch drift between AVX2 and NONE could go undetected. Adds test_compute_codes_matches_none in test_residual_quantizer.py: at the decorator's level, run rq.compute_codes(xb), then switch to SIMDLevel.NONE, run the reference, restore the level, assert np.testing.assert_array_equal. Static-build lanes have only one level available, so the test self.skipTest()'s there. LSQ icm_encode_step already has a bit-exact Python reference comparison (via icm_encode_step_ref) at every level, so cross-level NONE comparison is transitively guaranteed -- no addition needed there. test_extra_distances.py already has L1/Linf cases. Reviewed By: mdouze Differential Revision: D102821184 fbshipit-source-id: 9061794474deb36a7e72194b68d42bb598967737 |
||
|
Gergely Szilvasy
|
2f68a37307 |
Add per-SIMD-level tests for hamming.h utils + fix crosshamming bug (#5153)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5153 The standalone Hamming-distance entry points in faiss/utils/hamming.h -- hammings, hammings_knn_mc, hamming_count_thres, match_hamming_thres, crosshamming_count_thres, generalized_hammings_knn_hc -- are dynamic- dispatch entry points (with_simd_level dispatch in hamming.cpp), but had no Python test that exercised them across SIMD levels. The IndexBinary tests cover hammings_knn_hc and hamming_range_search via the flat / range index; everything else was untested. Adds tests/test_hamming_utils.py with for_all_simd_levels. For each function the test compares the result against a NumPy popcount-xor reference, and additionally: * For hammings_knn_mc and generalized_hammings_knn_hc: asserts each returned distance is no greater than the true k-th smallest (catches an impl that returns a non-top-k set), asserts unique IDs per row, asserts ordered-output for ordered=True. * For match_hamming_thres: decodes the (i, j) int64-pair packing of the output (per hamming_distance/common.h match_hamming_thres_impl) and verifies the returned set of pairs equals the reference set of below- threshold pairs exactly. * For *_thres functions: uses ht values that produce non-trivial match counts (assertGreater(expected, 0)), so a no-op implementation cannot pass vacuously; matches the implementation's `<= ht` predicate. Source fix in faiss/utils/hamming_distance/common.h: crosshamming_count_thres_impl had a long-standing bug: `bs2 = bs1 + 2;` hardcoded an offset of 2 uint64s instead of `nwords`. This is correct only for nwords=2 (ncodes=16); for ncodes=8 it skips a pair and reads out of bounds, and for ncodes>=32 it reads from non-element-aligned memory. The crosshamming test exposed this for ncodes=32 (off by 21 matches in 3160 pairs); the one-line fix `bs2 = bs1 + nwords` corrects it. Reviewed By: mdouze Differential Revision: D102821132 fbshipit-source-id: d60f87d9726005904c90c01c68109ac851cf807c |
||
|
Alibek Zhakubayev
|
e09de2d070 |
Harden compute_query_factors: add preconditions, handle aliasing, remove dead checks
Summary: Harden `compute_query_factors` in RaBitQUtils.cpp: - Add `FAISS_THROW_IF_NOT(d > 0)` precondition, which lets us simplify `inv_d_sqrt` and remove unreachable `d == 0` branches. - Detect query/rotated_q pointer aliasing (used by the IVF residual path in `IndexIVFRaBitQFastScan::compute_residual_LUT`) and assert the invariant that aliasing is only valid when `centroid == nullptr`. Skip the memcpy when aliased to avoid overlapping-buffer UB. - Remove dead defensive checks (`if (d > 0 && !rotated_q.empty())`) that were always true after the `resize(d)` call. - Reuse the cached `qr_to_c_L2sqr` value for `qr_norm_L2sqr` in the `METRIC_INNER_PRODUCT` + `centroid == nullptr` path, avoiding a redundant `fvec_norm_L2sqr` call. Reviewed By: mnorris11 Differential Revision: D102682365 fbshipit-source-id: d4811a9ab98b09cb65d7035a72833522ca6829bf |
||
|
Alibek Zhakubayev
|
6a9936c60d |
Hoist per-iteration vector allocations in IndexIVFRaBitQFastScan::compute_LUT
Summary: Move `rotated_q` and `centroid_buf` vector allocations from inside the inner loop to the `#pragma omp parallel` block scope. Previously each of the `n * nprobe` iterations allocated and freed these vectors on the heap. Now each thread allocates once and reuses capacity across all its iterations. Split `#pragma omp parallel for` into `#pragma omp parallel` + `#pragma omp for` to create a per-thread scope for the buffers. Reviewed By: mnorris11 Differential Revision: D102678232 fbshipit-source-id: 601022dd65943d11d53c7e5a91015553ad75870b |
||
|
Justin Gibbs
|
6c70444d08 |
Validate code_size during deserialization to prevent oversized allocations (#5151)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5151 Several index types read code_size directly from the serialized stream independently of the quantizer parameters that determine its correct value. When the stored code_size is corrupt but ntotal is 0, the existing consistency check (codes.size() == ntotal * code_size) passes trivially. A subsequent search then allocates (code_size * sizeof(float)) bytes in GenericFlatCodesDistanceComputer, which can trigger an OOM exception. Two layers of protection: 1. Cross-validate the deserialized code_size against the quantizer-derived value for all index types that read code_size from the stream: IndexResidualQuantizer, IndexLocalSearchQuantizer, IndexProductResidualQuantizer, IndexProductLocalSearchQuantizer, IndexIVFAdditiveQuantizer, IndexIVFScalarQuantizer, IndexLSH, and Index2Layer. The quantizer code_size is computed from validated parameters via set_derived_values() and is always authoritative. 2. For IndexLattice, where code_size is derived from constructor parameters (scale_nbit, lattice_nbit, nsq) rather than read from the stream, validate that code_size does not exceed the uncompressed vector size (d * sizeof(float)). IndexLattice is a lossy compressor, so its code_size must always be smaller than the uncompressed representation. A corrupt scale_nbit can overflow the total_nbit computation, producing a code_size that wraps to a huge value; this bound catches that before any allocation is attempted. Reviewed By: mnorris11 Differential Revision: D102360605 fbshipit-source-id: 1f0e7262a0e0e4566d7813b7da1bf0102e6fd9bf |
||
|
Justin Gibbs
|
9dbb81ca9d |
Set own_fields after reading Index2Layer sub-quantizer during deserialization (#5147)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5147 Index2Layer deserialization reads its sub-quantizer via read_index() which allocates a new Index on the heap, but never sets q1.own_fields = true. Since Level1Quantizer::own_fields defaults to false, the deserialized Index2Layer does not own its quantizer and never frees it — leaking the allocation both on the normal path (when the Index2Layer is eventually destroyed) and on the error path (when a subsequent deserialization step throws and the partially-constructed Index2Layer is cleaned up). Every other index type in read_index_up that reads a sub-index via read_index sets own_fields = true afterwards (IndexIVF, IndexPreTransform, IndexIDMap, IndexRefine, IndexHNSW, IndexNSG, IndexNNDescent, etc.). Index2Layer was the only one missing it. Reviewed By: mnorris11 Differential Revision: D102357926 fbshipit-source-id: 66a64ebe406c27aeebe45924028670227feeb9d0 |
||
|
anish k
|
01b22b4cc9 |
Fix memory leak warning for unordered_multimap in IndexIVFFlatDedup Python binding (#1667) (#5145)
Summary: When running the Python test suite, SWIG emits a warning: ``` swig/python detected a memory leak of type 'std::unordered_multimap< faiss::Index::idx_t,faiss::Index::idx_t > *', no destructor found. ``` The root cause is that `faiss::IndexIVFFlatDedup::instances` is a `std::unordered_multimap<idx_t, idx_t>` member, and SWIG has no built-in destructor registration for that STL type. When SWIG wraps the field, it generates a raw pointer wrapper with no cleanup, causing the leak warning whenever an `IndexIVFFlatDedup` object is garbage-collected on the Python side. The fix adds `%ignore faiss::IndexIVFFlatDedup::instances` in `faiss/python/swigfaiss.swig` before the `%include <faiss/IndexIVFFlat.h>` directive. This prevents SWIG from generating the untyped wrapper for the internal dedup map (which was never exposed in the Python type stubs anyway) and eliminates the memory leak warning. Fixes https://github.com/facebookresearch/faiss/issues/1667 Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5145 Reviewed By: trang-nm-nguyen Differential Revision: D102668055 Pulled By: mnorris11 fbshipit-source-id: e865d9463aecc4d9c37447602b33e1cf71011007 |
||
|
Michael Norris
|
a31dad31df |
Generated from a GitHub Pull Request. Run 'jf sync' on this diff to load the correct commit data.
Reviewed By: alibeklfc Differential Revision: D102487191 fbshipit-source-id: ba14efa264653acf42a2eece8e28a8b0d0d50f99 |
||
|
Gergely Szilvasy
|
c3460c980b |
Convert binary index Hamming callers to dynamic dispatch (#5071)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5071 Phase 2 of the Hamming DD conversion. Converts 5 external callers of dispatch_HammingComputer to use per-ISA TUs so the correct HammingComputer structs (AVX2/NEON/generic) are used at runtime. Previously, these files compiled as common TUs with no ISA flags, causing hamdis-inl.h to fall through to generic-inl.h — scalar HammingComputer structs even on AVX2-capable machines in DD mode. Converted files: - IndexBinaryHNSW.cpp: get_distance_computer() factory - IndexBinaryIVF.cpp: get_InvertedListScanner() + search_preassigned() - IndexBinaryHash.cpp: 4 search entry points (knn+range × 2 index types) - IndexIVFSpectralHash.cpp: get_InvertedListScanner() factory - IndexPQ.cpp: polysemous_inner_loop dispatch NOT converted: IndexIVFPQ.cpp polysemous path — deeply interleaved with PQ distance computation inside IVFPQScannerT, niche (off by default, polysemous_ht=0). See ~/ivfpq_polysemous_dd.md for analysis. Reviewed By: mdouze Differential Revision: D100020358 fbshipit-source-id: df787e5c8c095d86041f70e21cd5ae2e4e418c3b |
||
|
Trang Nguyen
|
130fc2481a |
Fix flaky test (#5150)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5150 https://www.internalfb.com/intern/test/844425196588481 We are receving alert due to unhealthy test. Claude is suggesting me to reduce the test size, I think it is worth a try. Reviewed By: mnorris11 Differential Revision: D102641694 fbshipit-source-id: 05c38a01e646e0f35d205e63c4c489eee8b6d986 |
||
|
Gergely Szilvasy
|
ecac6511a2 |
Convert hamming.cpp public API to dynamic dispatch (#5070)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5070 Convert all 8 public Hamming distance functions in `hamming.cpp` to dynamic dispatch so they use the best available SIMD implementation at runtime instead of always running at scalar speed in DD mode. This integrates Matthijs's refactor (D100004339): deduplicates the HammingComputer code across the 4 per-ISA backend files into a shared `common.h`. It is Phase 1 of the Hamming DD conversion; Phase 2 (future diff) will migrate external callers that use `dispatch_HammingComputer` directly. Key changes: - **DD dispatch**: the 8 public hamming.cpp functions now dispatch via `with_simd_level` (mask A0: NONE/AVX2/AVX512/ARM_NEON) to SL-templated `*_fixSL<SL>` entry points. - **`_fixSL` suffix** on SL-templated entry points (per Matthijs's D100020358 convention), distinguishing them from the non-templated DD entry points that share the same base name. Avoids silent-dispatch-vs- pinned-specialization confusion at call sites. - **New `hamming_avx512.cpp` TU** — makes the pre-existing AVX-512 HammingComputer specializations in `avx512-inl.h` reachable from Buck DD builds (previously reachable only from OSS CMake `faiss_avx512` / `faiss_avx512_spr` targets). The AVX-512 TU gives SKX/CLX/Ice Lake clients ZMM auto-vectorization of the hamming kernels; the `_mm512_popcnt_epi64` fast path remains gated on VPOPCNTDQ (active in OSS `faiss_avx512_spr` CMake target only, per pre-existing design). - **ODR fix via SIMDLevel templatization**: the 9 ISA-varying `HammingComputer*` / `GenHammingComputer*` structs are now primary templates `*_tpl<SIMDLevel>` with scalar bodies in `common.h`, and per-ISA specializations in the respective `-inl.h` files. Each TU picks its specialization via a `using`-alias in hamdis-inl.h keyed on `FAISS_HAMMING_SL`. Distinct mangled names per SL eliminate the latent ODR hazard where multiple DD TUs with different flags saw differently-laid-out structs sharing the `faiss::HammingComputer*` name. Verified by disassembly: distinct `<SIMDLevel)0|1|2>` suffixes per TU. - **Silent OOB fix in `HammingComputer20::set`** (inherited from D100004339): pre-diff `uint64_t a[2]` read 8 bytes on a 20-byte code (past-end read, tripped by ASan); post-diff `uint32_t a32[4]` reads exactly 4 bytes. - **`faiss_hamming_sl_collapse()` constexpr** in hamdis-inl.h makes the AVX512_SPR→AVX512 / ARM_SVE→ARM_NEON collapse explicit at the alias level, rather than relying on the raw arch-macro ladder happening to match. Applies only to HammingComputer aliases; does not change the dispatcher. - **popcount.h extraction**: moved `popcount32` / `popcount64` out of `common.h` into a standalone `faiss/utils/popcount.h` so callers outside the hamming subtree (rabitq_simd.h, rabitq_avx2/512.cpp, partitioning_simdlib256.h, PolysemousTraining.cpp) have a clean include. Migrated all `__builtin_popcount*` call sites. Silent Windows correctness improvement: pre-diff `__builtin_popcountl` on `uint64_t` truncated to 32 bits on MSVC (LLP64, `long` is 32-bit); post-diff `popcount64` always dispatches to `__popcnt64`. The pre-diff `__builtin_popcount → __popcnt` macro shim in `platform_macros.h` is removed — no longer needed now that all call sites use the helpers. - **`FAISS_MAYBE_UNUSED` on Windows**: previously defined only in the Linux/OSX branch of platform_macros.h; added to the MSVC branch so the templated HammingComputer `set(..., int code_size)` ctors compile cleanly on Windows under `-Wunused-parameter` + NDEBUG. Reviewed By: mdouze Differential Revision: D99994593 fbshipit-source-id: 1f54e09fbade3d12b23fb787640d0b206e499859 |
||
|
Akshay Mahesh
|
748e4d47ad |
Revert D97138228: Generated from a GitHub Pull Request. Run 'jf sync' on this diff to load the correct commit data.
Differential Revision: D97138228 Original commit changeset: 311eafd5186a Original Phabricator Diff: D97138228 fbshipit-source-id: c0aead65b62bdf6a3ccf0009ee5c3d57e5707304 |
||
|
Michael Norris
|
bdc882c4d3 |
Generated from a GitHub Pull Request. Run 'jf sync' on this diff to load the correct commit data.
Reviewed By: limqiying Differential Revision: D97138228 fbshipit-source-id: 311eafd5186a77e1ba32805a4723662e75815593 |
||
|
Fabrizio (Misto) Milo
|
a7116ed4ec |
Add TurboQuant (CPU) (#5049)
Summary: This PR adds initial TurboQuant (see https://github.com/facebookresearch/faiss/issues/4990) support to Faiss and integrates it into the main codepaths needed for local evaluation. Changes in this PR: - add `IndexTurboQuantMSE` and the underlying `TurboQuantizer` implementation - add cloning and serialization support for TurboQuant indexes - expose TurboQuant through the Python SWIG bindings - extend quantizer benchmarking to cover TurboQuant and additional datasets - add unit tests for TurboQuant reconstruction behavior ## Preliminary Benchmarks These are preliminary local results from a macOS CPU-only run. Command: ```bash python bench_quantizer.py glove 100x4 turboquant pq rq eval on glove 100x4 maxtrain=100000 No training set: training on database ===== PQ training time: 0.594 s encode time: 0.317 reconstruction error: 0.010 recall@1: 0.7036 recons_err_compat 0.100 code_size: 50 B/vector ===== RQ training time: 208.554 s max_beam_size=1 encode time: 2.977 reconstruction error: 0.027 recall@1: 0.6034 recons_err_compat 0.162 code_size: 50 B/vector max_beam_size=2 encode time: 5.774 reconstruction error: 0.023 recall@1: 0.6280 recons_err_compat 0.151 code_size: 50 B/vector max_beam_size=4 encode time: 12.271 reconstruction error: 0.020 recall@1: 0.6512 recons_err_compat 0.140 code_size: 50 B/vector max_beam_size=8 encode time: 25.617 reconstruction error: 0.018 recall@1: 0.6751 recons_err_compat 0.131 code_size: 50 B/vector max_beam_size=16 encode time: 50.257 reconstruction error: 0.016 recall@1: 0.6890 recons_err_compat 0.123 code_size: 50 B/vector max_beam_size=32 encode time: 105.272 reconstruction error: 0.014 recall@1: 0.7067 recons_err_compat 0.116 code_size: 50 B/vector ===== TurboQuant training time: 0.002 s encode time: 0.080 reconstruction error: 0.009 recall@1: 0.7189 recons_err_compat 0.095 code_size: 50 B/vector ``` ### Initial takeaway On this macOS CPU-only benchmark, TurboQuant shows: - better `recall@1` than PQ at the same code size - lower reconstruction error than PQ - higher `recall@1` than the best RQ setting tested here - much lower training and encoding cost than RQ ### TODO - [x] add JQL support (only MSE at the moment) - reproduce the paper benchmarks with same datasets (need a beefy machine but script is ready) - ~add CUDA support~ (will be in another PR if this is accepted) Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5049 Test Plan: buck build fbcode//faiss:faiss buck test fbcode//faiss/tests:test_factory buck test fbcode//faiss/tests:test_scalar_quantizer buck test fbcode//faiss/tests:test_scalar_quantizer_correctness buck build fbcode//faiss/tests/fb:faiss_fuzzer_helpers Reviewed By: mdouze Differential Revision: D102195204 Pulled By: mnorris11 fbshipit-source-id: a302541a1fc5c229e697f28e639cba633babd072 |
||
|
Justin Gibbs
|
dfce6e948e |
Fix OMP exception safety in IndexHNSW search (#5133)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5133 Exceptions thrown inside OpenMP parallel regions in IndexHNSW search methods call std::terminate because the OpenMP specification does not allow exceptions to escape parallel regions. This is triggered by corrupt serialized index data that causes allocation failures (e.g. VisitedTable construction with absurdly large ntotal) or other errors in DistanceComputer, but can occur with any exception thrown during the HNSW search loop. Wrap the OpenMP parallel bodies in hnsw_search(), IndexHNSW::search_level_0(), and IndexHNSW2Level::search() with per-thread try/catch blocks that capture exceptions via std::exception_ptr and re-throw them on the main thread after the parallel region completes. This follows the same pattern established in IndexFlatCodes and IndexNNDescent/IndexNSG. VisitedTable, SingleResultHandler, and DistanceComputer are moved from stack to heap allocation (std::unique_ptr) because the constructors are now inside try/catch blocks. If a stack-allocated object's constructor throws, the object never comes into existence, but OpenMP requires all threads to participate in the worksharing loop that follows — any reference to the uninitialized object would be undefined behavior. With std::unique_ptr, the variable starts as nullptr before the try block, remains nullptr if construction fails, and the interrupt flag ensures all loop iterations skip via continue without dereferencing it. Reviewed By: mnorris11 Differential Revision: D101638521 fbshipit-source-id: 3920be5e41b1bff0490a14e2571b3dbeace64397 |
||
|
Alibek Zhakubayev
|
c6273349f9 |
Fix sa_decode offset bug and integer overflow in IndexRefine (#5143)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5143 This diff fixes four bugs in `IndexRefine.cpp` and adds a regression test. **Bug 1 (critical): `sa_decode` reads wrong bytes** `IndexRefine::sa_encode` writes each vector's codes as `[base_codes (cs1 bytes) | refine_codes (cs2 bytes)]` (lines 197-199). The encode writes base codes at offset 0 and refine codes at offset `cs1`: ``` memcpy(b, tmp1.get() + cs1 * i, cs1); // base at b+0 memcpy(b + cs1, tmp2.get() + cs2 * i, cs2); // refine at b+cs1 ``` `sa_decode` must extract the refine portion to pass to `refine_index->sa_decode`. The old code read from `bytes + i * (cs1 + cs2)` (offset 0), which extracted the base codes instead of the refine codes. The fix adds `+ cs1` to skip past the base codes: ``` // Old (wrong): memcpy(..., bytes + i * (cs1 + cs2), cs2); // New (fixed): memcpy(..., bytes + i * (cs1 + cs2) + cs1, cs2); ``` This mirrors the write offset in `sa_encode` line 199. Without this fix, `sa_decode` silently produces wrong reconstructions by feeding base-index codes to the refine decoder. **Bug 2: `int` loop variable causes overflow with large inputs** Three validation loops used `int i` as the loop counter: ``` for (int i = 0; i < n * k_base; i++) ``` Both `n` and `k_base` are `idx_t` (int64_t), so `n * k_base` can exceed `INT_MAX` (2^31 - 1). When the `int` counter reaches `INT_MAX`, incrementing it is signed integer overflow (undefined behavior). In practice this causes an infinite loop or out-of-bounds access. Changed to `idx_t i` in all three search methods: `IndexRefine::search`, `IndexRefineFlat::search`, and `IndexRefinePanorama::search`. **Bug 3: Wrong class name in error message** `IndexRefinePanorama::search` had the error message `"IndexRefineFlat params have incorrect type"` -- a copy-paste error from `IndexRefineFlat::search`. Fixed to `"IndexRefinePanorama params have incorrect type"`. **Bug 4: Missing overflow guard on `n * k_base` allocation** The product `n * k_base` is used in `new idx_t[n * k_base]` before the loop. If the product overflows int64_t, it could wrap to a small positive value, causing a too-small allocation followed by out-of-bounds writes from `base_index->search`. Added `FAISS_THROW_IF_NOT_MSG(n <= INT64_MAX / k_base, ...)` before the allocation in all three search methods. Division by zero is impossible because `k_base >= k > 0` is checked earlier. **Cleanup: Redundant `sa_code_size()` call** The allocation in `sa_decode` called `refine_index->sa_code_size()` a second time instead of using the already-computed `cs2`. Replaced with `cs2`. Reviewed By: junjieqi Differential Revision: D101903134 fbshipit-source-id: 37848d280d50447216ec3c76c598c3e212ea0971 |
||
|
Matthijs Douze
|
8d1b964c11 |
Enable dynamic dispatch build on Windows (#5127)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5127 Attempt to enable FAISS dynamic dispatch (DD) mode on Windows/MSVC. Changes: - CMakeLists.txt: Remove if(NOT WIN32) guard from DD section, add MSVC per-file SIMD flags (/arch:AVX2, /arch:AVX512, /bigobj) alongside existing GCC/Clang flags - build-pull-request.yml: Add windows-x86_64-DD-cmake job that runs immediately (no dependency on linux-x86_64-cmake), builds with MSVC and FAISS_OPT_LEVEL=dd, runs C++ tests and Python tests This diff is expected to fail on Windows due to MSVC requiring explicit template specialization declarations (C++ §17.8.3) which GCC/Clang don't enforce. The CI failure will surface the exact errors to guide the fix. UPDATE: in fact, MSVC does not seem to require this. After fixing build scripts and a few classical Windows error, the C++ and python tests pass. Reviewed By: algoriddle Differential Revision: D101649751 fbshipit-source-id: 765ef8483d02652ce58625cd22baab8870acf718 |
||
|
Michael Norris
|
a25bcb1e22 |
Rely on cuVS for default values (#5140)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5140 Full logs: P2284358998 ## TLDR: Fix vs No-fix: Is the improvement consistent? yes, seems like doing this fix is better than doing nothing. Net result: faster overall. 26/60 configs improve by >5%, only 4 regress by >5%. ┌───────────────────────────────┬─────────────┬────────────┐ │ Worst regressions │ ms increase │ % increase │ ├───────────────────────────────┼─────────────┼────────────┤ │ M=100, bs=10000, np=16, k=100 │ +10.0 ms │ +8.4% │ ├───────────────────────────────┼─────────────┼────────────┤ │ M=50, bs=10000, np=16, k=100 │ +6.3 ms │ +7.7% │ ├───────────────────────────────┼─────────────┼────────────┤ │ M=50, bs=10000, np=64, k=100 │ +10.2 ms │ +6.2% │ ├───────────────────────────────┼─────────────┼────────────┤ │ M=100, bs=1000, np=32, k=100 │ +11.8 ms │ +5.2% │ └───────────────────────────────┴─────────────┴────────────┘ ┌────────────────────────────────┬───────────┬──────────┐ │ Best improvements │ ms saved │ % faster │ ├────────────────────────────────┼───────────┼──────────┤ │ M=50, bs=1000, np=16, k=100 │ -35.8 ms │ -22.7% │ ├────────────────────────────────┼───────────┼──────────┤ │ M=100, bs=10000, np=256, k=500 │ -359.8 ms │ -10.7% │ ├────────────────────────────────┼───────────┼──────────┤ │ M=100, bs=1000, np=128, k=1000 │ -240.9 ms │ -10.4% │ └────────────────────────────────┴───────────┴──────────┘ No fix in Faiss D101399711 (migration only, no fix) vs baseline More regressions: 14/60 configs regress >5%, 2 exceed 10%. ┌───────────────────────────────┬─────────────┬────────────┐ │ Worst regressions │ ms increase │ % increase │ ├───────────────────────────────┼─────────────┼────────────┤ │ M=50, bs=10000, np=32, k=1000 │ +110.5 ms │ +10.6% │ ├───────────────────────────────┼─────────────┼────────────┤ │ M=50, bs=10000, np=16, k=1000 │ +68.3 ms │ +10.1% │ ├───────────────────────────────┼─────────────┼────────────┤ │ M=50, bs=10000, np=64, k=1000 │ +147.7 ms │ +9.2% │ └───────────────────────────────┴─────────────┴────────────┘ ## what about variance? Variance summary (see mnorris11 notes, I don't agree with everything) The search is mostly stable, with occasional single-run outliers. ┌─────────────────────┬───────────────────────────────────┐ │ Metric │ Value │ ├─────────────────────┼───────────────────────────────────┤ │ Median CV │ 3.3% │ ├─────────────────────┼───────────────────────────────────┤ │ 90th percentile CV │ 9.8% │ ├─────────────────────┼───────────────────────────────────┤ │ Most stable configs │ bs=1000, nprobe≥64: CV 0.3-1.7% │ ├─────────────────────┼───────────────────────────────────┤ │ Noisiest configs │ bs=10000, nprobe=16: CV up to 39% │ └─────────────────────┴───────────────────────────────────┘ What drives variance: absolute runtime. Sub-200ms configs (low nprobe, k=100) are noisy because GPU scheduling jitter is proportionally large. Configs >1 second are rock-solid (CV < 2%). Outlier pattern: Every high-CV config is caused by a single extreme outlier (e.g., one run at 1124ms when 49 others are at 120-155ms), not general measurement noise. Using median instead of mean would remove these. The median values in the results file should be more reliable for these fast configs. Are the cross-trial differences real? - For stable configs (bs=1000, nprobe≥64, CV < 2%): the 8-11% improvements at high nprobe are definitely real — intra-trial noise is well below 2%, so a 10% shift is ~5 standard errors. - For noisy configs (bs=10000, nprobe=16, CV 5-15%): the 5-8% regressions are borderline — they're statistically detectable with 50 runs, but could partly reflect system-state differences between trials (GPU thermals, background load) rather than code changes. ******[mnorris11 note: this explanation sounds like BS, there was nothing else going on in the GPU...]****** - To be fully confident about the small regressions at low nprobe, you'd want same-machine A/B testing (build both versions, alternate runs within one script). ******[mnorris11 note: this is nonsense, this is what we did?]****** Reviewed By: weidbd2025 Differential Revision: D101500046 fbshipit-source-id: 9a2759a64a3ba8e398b6b36ca40e31cf6aaa5ba0 |
||
|
Junjie Qi
|
1b4b995b97 |
Fix implicit integer precision loss from 64-bit to 32-bit (#5091)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5091 Fix 172 `clang-diagnostic-shorten-64-to-32` lint warnings across 29 files by adding explicit `static_cast<int>()` or widening variable types where `size_t`/`idx_t` (64-bit) values were implicitly narrowed to `int`/`int32_t` (32-bit). The fixes fall into two categories: - **Explicit casts**: Where the receiving API requires `int` and the value is known to fit (e.g., vector dimensions, sub-quantizer counts, cluster counts, BLAS parameters) - **Type widening**: Where the variable was unnecessarily narrow (e.g., `int nprobe` → `size_t nprobe`, `int list_no` → `size_t list_no`) Reviewed By: limqiying Differential Revision: D100588996 fbshipit-source-id: 153fe3d557a102c5adb7831915c1b3c8cecae22b |
||
|
Junjie Qi
|
b0568942ef |
Fix miscellaneous lint warnings (#5093)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5093 Fix remaining miscellaneous lint warnings across 10 files: - `facebook-hte-MultTypeDeclaration`: Split mixed-type declaration in AutoTune.cpp - `facebook-hte-IdenticalOperands`: Rename variable in build.cpp to avoid false positive - `facebook-hte-BadImplicitCast`: Add explicit cast in Index.cpp - `performance-inefficient-vector-operation`: Add reserve() in IndexBinaryIVF.cpp - `performance-for-range-copy`: Use const reference in IndexBinaryHash.cpp range-for - `facebook-hte-UnassignedReleasedUniquePointer`: Capture release() results in IVFlib.cpp, IndexPreTransform.cpp - `facebook-hte-UnqualifiedCall-sqrt`: Use std::sqrt() in MatrixStats.cpp - `facebook-unused-include-check`: Remove unused includes in IndexIVF.cpp, IndexNNDescent.cpp, IndexNSG.cpp - `clang-diagnostic-switch-enum`: Add missing enum cases in IndexAdditiveQuantizer.cpp, IndexIVFAdditiveQuantizer.cpp Reviewed By: pankajsingh88 Differential Revision: D100592786 fbshipit-source-id: 1d324e30d79e967c345737bae6991e9a443622ee |
||
|
generatedunixname2012458493028417
|
ed11f283d6 |
Fix broken fbcode//faiss/tests:test_contrib - test_checkpoint (test_contrib.TestBigBatchSearch) (#5139)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5139 Reviewed By: junjieqi Differential Revision: D101979704 fbshipit-source-id: b3b1575fd3431dca7d3b5e7ec86e4009810095fb |
||
|
Alibek Zhakubayev
|
277c53d299 |
Fix PCA training bug and memory safety issues in VectorTransform.cpp (#5138)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5138 ## Summary This diff fixes several bugs and memory safety issues in VectorTransform.cpp: ### 1. Bug fix: Wrong beta parameter in PCAMatrix::train (sgemm_ call) In the code path where n < d_in (Gram matrix approach), the sgemm_ call that computes `PCAMat = xc * gram` incorrectly uses beta=1.0 instead of beta=0.0. This means the computation is actually `PCAMat = xc * gram + PCAMat` instead of `PCAMat = xc * gram`. On the first training call this works by accident because std::vector::resize zero-initializes new elements. However, if PCAMatrix::train() is called a second time (e.g., retraining with different data), PCAMat retains stale values from the previous training, corrupting the PCA matrix and producing incorrect dimensionality reduction results. ### 2. Memory leak fix: eig() function Replaced raw `new double[]` with `std::vector<double>` for the LAPACK workspace buffer. The old code would leak memory if dsyev_ threw an exception. ### 3. Memory leak fix: LinearTransform::transform_transpose() Replaced raw `new float[]` with `std::vector<float>` for the bias-corrected buffer. The old code would leak memory if sgemm_ threw an exception between allocation and the manual delete[]. ### 4. Missing error check: OPQ SVD workspace query Added FAISS_THROW_IF_NOT_FMT check after the sgesvd_ workspace query in OPQMatrix::train(). Previously, if the workspace query failed, the returned workspace size would be garbage, leading to either a crash or silent data corruption in the subsequent SVD computation. Reviewed By: junjieqi, mnorris11 Differential Revision: D101975473 fbshipit-source-id: 57e74d8cc55d119bfee99f164caaf4d64b08a7ce |
||
|
Akash Nayar
|
06228410c6 |
Panorama Optimizations (#5041)
Summary: _Note: Should be merged before https://github.com/facebookresearch/faiss/issues/4970 (IVFPQPanorama)._ ## Changes ### Performance This PR implements various optimizations to Panorama (L2Flat and IVFFlat). 1. Disaggregate distance computation from pruning decisions to avoid branches in distance computation hotpath. 2. Early batch processing termination when no points are remaining. 3. Manually unrolled distance kernel. 4. Template distance computation on level width for autovectorization. 5. `if constexpr (C::is_max)` instead of `C::cmp` for autovectorized pruning. 6. Byteset for vectorized compacting of active indices using `_pext_u64`. 7. Template distance computation and pruning on first level (no `active_indices` indirection) to let it autovectorize. 8. Hoist buffer allocations into `IndexFlat`/`IVFFlatScannerPanorama`. 9. Expose `batch_size` as a parameter for IVFFlatPanorama (for consistency with `IndexFlatPanorama` but also because 1024 `batch_size` can improve performance). ### Other - Define `kDefaultBatchSize` once in `Panorama.h` (previously defined in 5 separate locations). - Allow `bench_flat_l2_panorama.py` and `bench_ivf_flat_panorama.py` to accept `gist1M` or `sift1M` as dataset to bench on. ## Results Together, these optimizations enable powerful additional speedups, especially on lower-dimensional datasets like SIFT (128d), by dramatically minimizing Panorama's overhead: **GIST1M (IVF128, nlist=128, nlevels=16)** | nprobe | Recall@10 | Old Speedup | New Speedup | _Additional_ Speedup | |--------|-----------|-------------------------|-------------------------|--------------------| | 1 | 0.1439 | 3.92x | 3.93x | 1.00x | | 2 | 0.2605 | 4.71x | 5.19x | 1.10x | | 4 | 0.4369 | 5.53x | 6.75x | 1.22x | | 8 | 0.6470 | 6.37x | 8.21x | 1.29x | | 16 | 0.8780 | 7.30x | 9.74x | 1.33x | | 32 | 0.9764 | 8.33x | 11.29x | 1.36x | | 64 | 0.9868 | 9.38x | 12.74x | 1.36x | **SIFT1M (IVF128, nlist=128, nlevels=8)** | nprobe | Recall@10 | Old Speedup | New Speedup | _Additional_ Speedup | |--------|-----------|-------------------------|-------------------------|--------------------| | 1 | 0.2678 | 1.20x | 1.81x | 1.52x | | 2 | 0.4584 | 1.38x | 2.23x | 1.62x | | 4 | 0.6855 | 1.59x | 2.70x | 1.70x | | 8 | 0.8760 | 1.83x | 3.44x | 1.88x | | 16 | 0.9679 | 2.11x | 4.72x | 2.24x | | 32 | 0.9855 | 2.44x | 5.61x | 2.30x | | 64 | 0.9861 | 2.74x | 6.39x | 2.33x | ### Raw Data Collected by running the new benches on `main` and this branch. On main, you cannot specify `batch_size` so remove the `{1024}` from the factory string in the new benches to run them there. The results above are calculated from the following raw data as follows: 1. For each experiment (i.e., GIST (old) or SIFT (new), calculate the Panorama speedups for each `nprobe` ((original ms per query) / (pano ms per query)) 2. For each pairing of (old) and (new) results, calculate the additional speedup by calculating (new speedup) / (old speedup). #### Before (`main`) GIST1M: ``` ======IVF128,Flat nprobe 1, Recall@10: 0.145200, speed: 2.705442 ms/query, dims scanned: 100.00% nprobe 2, Recall@10: 0.260800, speed: 5.456891 ms/query, dims scanned: 100.00% nprobe 4, Recall@10: 0.441900, speed: 10.895120 ms/query, dims scanned: 100.00% nprobe 8, Recall@10: 0.648200, speed: 21.676788 ms/query, dims scanned: 100.00% nprobe 16, Recall@10: 0.878000, speed: 43.142261 ms/query, dims scanned: 100.00% nprobe 32, Recall@10: 0.975400, speed: 84.498397 ms/query, dims scanned: 100.00% nprobe 64, Recall@10: 0.986800, speed: 160.092644 ms/query, dims scanned: 100.00% ======PCA960,IVF128,FlatPanorama16 nprobe 1, Recall@10: 0.143900, speed: 0.689507 ms/query, dims scanned: 12.96% nprobe 2, Recall@10: 0.260500, speed: 1.158416 ms/query, dims scanned: 11.18% nprobe 4, Recall@10: 0.436900, speed: 1.968814 ms/query, dims scanned: 9.90% nprobe 8, Recall@10: 0.647000, speed: 3.401469 ms/query, dims scanned: 8.91% nprobe 16, Recall@10: 0.878000, speed: 5.912757 ms/query, dims scanned: 8.10% nprobe 32, Recall@10: 0.976400, speed: 10.147847 ms/query, dims scanned: 7.44% nprobe 64, Recall@10: 0.986800, speed: 17.074573 ms/query, dims scanned: 6.93% ``` SIFT1M: ``` ======IVF128,Flat nprobe 1, Recall@10: 0.267480, speed: 0.285990 ms/query, dims scanned: 100.00% nprobe 2, Recall@10: 0.457520, speed: 0.564067 ms/query, dims scanned: 100.00% nprobe 4, Recall@10: 0.685320, speed: 1.111833 ms/query, dims scanned: 100.00% nprobe 8, Recall@10: 0.877210, speed: 2.195088 ms/query, dims scanned: 100.00% nprobe 16, Recall@10: 0.967730, speed: 4.338444 ms/query, dims scanned: 100.00% nprobe 32, Recall@10: 0.985400, speed: 8.500538 ms/query, dims scanned: 100.00% nprobe 64, Recall@10: 0.986100, speed: 16.349893 ms/query, dims scanned: 100.00% ======PCA128,IVF128,FlatPanorama8 nprobe 1, Recall@10: 0.267670, speed: 0.239243 ms/query, dims scanned: 27.97% nprobe 2, Recall@10: 0.458320, speed: 0.408590 ms/query, dims scanned: 24.42% nprobe 4, Recall@10: 0.685480, speed: 0.699694 ms/query, dims scanned: 21.50% nprobe 8, Recall@10: 0.875930, speed: 1.197310 ms/query, dims scanned: 19.06% nprobe 16, Recall@10: 0.967760, speed: 2.055968 ms/query, dims scanned: 16.98% nprobe 32, Recall@10: 0.985370, speed: 3.481555 ms/query, dims scanned: 15.26% nprobe 64, Recall@10: 0.985980, speed: 5.977346 ms/query, dims scanned: 14.02% ``` #### After (`optimize-pano`) GIST1M: ``` ======IVF128,Flat nprobe 1, Recall@10: 0.145200, speed: 2.625779 ms/query, dims scanned: 100.00% nprobe 2, Recall@10: 0.260800, speed: 5.285007 ms/query, dims scanned: 100.00% nprobe 4, Recall@10: 0.441900, speed: 10.555867 ms/query, dims scanned: 100.00% nprobe 8, Recall@10: 0.648200, speed: 21.012494 ms/query, dims scanned: 100.00% nprobe 16, Recall@10: 0.878000, speed: 41.794143 ms/query, dims scanned: 100.00% nprobe 32, Recall@10: 0.975400, speed: 81.865038 ms/query, dims scanned: 100.00% nprobe 64, Recall@10: 0.986800, speed: 155.067333 ms/query, dims scanned: 100.00% ======PCA960,IVF128,FlatPanorama16_1024 nprobe 1, Recall@10: 0.143900, speed: 0.668800 ms/query, dims scanned: 20.33% nprobe 2, Recall@10: 0.260500, speed: 1.018440 ms/query, dims scanned: 14.81% nprobe 4, Recall@10: 0.436900, speed: 1.563622 ms/query, dims scanned: 11.72% nprobe 8, Recall@10: 0.647000, speed: 2.557981 ms/query, dims scanned: 9.82% nprobe 16, Recall@10: 0.878000, speed: 4.292616 ms/query, dims scanned: 8.56% nprobe 32, Recall@10: 0.976400, speed: 7.248832 ms/query, dims scanned: 7.68% nprobe 64, Recall@10: 0.986800, speed: 12.171319 ms/query, dims scanned: 7.06% ``` SIFT1M: ``` ======IVF128,Flat nprobe 1, Recall@10: 0.267480, speed: 0.295904 ms/query, dims scanned: 100.00% nprobe 2, Recall@10: 0.457520, speed: 0.583204 ms/query, dims scanned: 100.00% nprobe 4, Recall@10: 0.685320, speed: 1.150055 ms/query, dims scanned: 100.00% nprobe 8, Recall@10: 0.877210, speed: 2.425575 ms/query, dims scanned: 100.00% nprobe 16, Recall@10: 0.967730, speed: 5.509365 ms/query, dims scanned: 100.00% nprobe 32, Recall@10: 0.985400, speed: 10.794491 ms/query, dims scanned: 100.00% nprobe 64, Recall@10: 0.986100, speed: 20.727924 ms/query, dims scanned: 100.00% ======PCA128,IVF128,FlatPanorama8_1024 nprobe 1, Recall@10: 0.267750, speed: 0.163266 ms/query, dims scanned: 34.97% nprobe 2, Recall@10: 0.458370, speed: 0.261109 ms/query, dims scanned: 27.97% nprobe 4, Recall@10: 0.685540, speed: 0.425977 ms/query, dims scanned: 23.30% nprobe 8, Recall@10: 0.875990, speed: 0.704580 ms/query, dims scanned: 19.98% nprobe 16, Recall@10: 0.967860, speed: 1.167465 ms/query, dims scanned: 17.45% nprobe 32, Recall@10: 0.985470, speed: 1.925296 ms/query, dims scanned: 15.50% nprobe 64, Recall@10: 0.986080, speed: 3.245793 ms/query, dims scanned: 14.14% ``` Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5041 Reviewed By: alibeklfc Differential Revision: D101753364 Pulled By: mnorris11 fbshipit-source-id: e6da1aa05e465e83632239bc69548bf8f5353d49 |
||
|
Gergely Szilvasy
|
f4338ade1b |
Parameterize existing tests with @for_all_simd_levels (#5134)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5134 Expand DD test coverage by applying for_all_simd_levels to existing test classes that exercise DD-dispatched code paths but were previously pinned to a single SIMD level. This is the "mega decorator" diff from the DD test coverage gaps plan -- pure decorator additions, no new test logic. Follow-up diffs add new test files and numerical cross-level assertions for gaps the decorator alone cannot close. Classes decorated (grouped by area): * Binary Hamming non-IVF: TestRange and TestKnn in test_binary_hashindex.py; TestBinarySearchParams in test_binary_search_params.py; TestIndexBinaryFromFloat in test_index_binary_from_float.py; TestSpectralHash in test_index_accuracy.py. * IVFPQ / search: EvalIVFPQAccuracy in test_index.py; TestSelector and TestSearchParams in test_search_params.py. * Flat / refine / Panorama: TestIndexFlat, TestIndexFlatL2, TestIndexFlatL2Panorama, TestScalarQuantizer in test_index.py; TestDistanceComputer, TestIndexRefineSearchParams, TestIndexRefineRangeSearch in test_refine.py; TestIndexRefinePanorama, TestIndexFlatPanorama, TestIndexHNSWFlatPanorama, TestIndexIVFFlatPanorama in their respective files. * Quantizer encode: TestResidualQuantizer, TestIndexResidualQuantizerSearch in test_residual_quantizer.py; TestComponents, TestLocalSearchQuantizer in test_local_search_quantizer.py. * Fast scan: TestFastScanFiltering, TestBlockSkipConsistency, TestFastScanRangeSearchFilter in test_fastscan_filter.py. * Broader index tests (search-exercising classes only): TestParameterSpace in test_autotune.py; TestSpectralHash in test_factory.py; TestMerge1, TestMerge2, TestRemoveFastScan in test_merge_index.py. BUCK changes move decorated tests from the legacy-listing lists to the simd_levels lists (which use supports_static_listing = False, required for dynamic class name generation), and add the new entries test_binary_search_params, test_fastscan_filter, test_refine_panorama, test_hnsw_panorama. ___ overriding_review_checks_triggers_an_audit_and_retroactive_review Oncall Short Name: fair_umami_cluster Differential Revision: D101822335 fbshipit-source-id: 9a432c6d6bee201c3731713226405a8c8ecebbe6 |
||
|
Alibek Zhakubayev
|
8312540763 |
Fix integer type truncation in IDSelectorBatch bloom filter mask (#5136)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5136 Fixes integer truncation in `IDSelectorBatch::is_member` on platforms where `long` is 32-bit (Windows LLP64). **Root cause.** `IDSelectorBatch::mask` is declared as `idx_t` (i.e. `int64_t` — see `MetricType.h:51`) and is computed in the constructor as `mask = ((idx_t)1 << nbits) - 1`, where `nbits = ceil(log2(n)) + 5`. For bloom filters sized for `n >= ~134M` ids, `nbits >= 32` and `mask` requires more than 32 bits to represent. The expression `i & mask` therefore yields a 64-bit `idx_t`. The previous code stored the result in a local `long im`: - LP64 ABI (Linux, macOS x86_64/arm64): `long` is 64-bit — no truncation, behaves correctly. - LLP64 ABI (Windows x86_64, MinGW): `long` is 32-bit — silently truncates the upper bits. After truncation, `im >> 3` indexes the wrong bloom slot and `1 << (im & 7)` tests the wrong bit. This produces false negatives in the bloom filter, causing `is_member` to incorrectly return `false` for ids that are in the set, which silently drops legitimate matches during selection. **Fix.** Change the local from `long im` to `idx_t im` so its type matches both operands of `i & mask`. This eliminates the platform-dependent truncation. As a small follow-on cleanup, change the early `return 0;` to `return false;` to match the function's `bool` return type (no behavior change — both compile to the same value). **Scope.** Intentionally narrow. Earlier iterations of this diff also widened `DirectMap::update_codes` from `int n` to `idx_t n` and added an `if (ii < 0) return false;` guard in `IDSelectorBitmap::is_member`. Both were reverted after review: - The `DirectMap::update_codes` widening was cosmetic: its sole caller `IndexIVF::update_vectors` still takes `int n` (see `IndexIVF.h:357`), so widening the inner type cannot unlock any larger batch size. Lifting the 2^31 cap would require widening the public virtual `update_vectors`, all overrides, and the C API in `IndexIVFFlat_c.{h,cpp}` — out of scope here, and a separate diff if desired. - The `IDSelectorBitmap` negative-id guard was redundant: per `[conv.integral]` the existing `uint64_t i = ii;` for negative `ii` produces a value in `[2^63, 2^64)`, so `i >> 3 >= 2^60`, which is unconditionally `>= n` for any physically realizable bitmap (`n` is bounded far below 2^60 by addressable memory). The pre-existing `(i >> 3) >= n` check already handles the case correctly. Reviewed By: mnorris11 Differential Revision: D101801522 fbshipit-source-id: 719d6dcc26ece5faf0dfb927e4639e322cf1a6fd |
||
|
Gergely Szilvasy
|
9d567497ec |
Wire 512-bit QBS kernels into fast scan DD dispatch (#5075)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5075 In DD mode, the QBS (bbs=32) accumulate path always used 256-bit kernels, even in the AVX512 per-ISA TU. The 512-bit kernels in kernels_simd512.h were dead because bare simdlib aliases resolve to _tpl<NONE> in DD mode, and 512-bit NONE types don't exist (empty primary templates). Fix: add function-local using declarations in both 512-bit kernel functions to bind types to explicit AVX512/AVX2 levels. Create accumulate_loops_512.h with FixedStorage512 (a non-virtual intermediate handler that bridges the AVX2→NONE type gap via storeu/loadu at the handler boundary) and the 512-bit QBS accumulate loop. Wire it into dispatching.h's ScannerMixIn behind an Reviewed By: mdouze Differential Revision: D100151879 fbshipit-source-id: b801f897f2d061a8448842f42edcdeb3a447eafd |
||
|
Alibek Zhakubayev
|
9cbc8da28e |
Fix integer overflow and unbounded loop in Clustering.cpp (#5130)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5130 This diff fixes four bugs in `Clustering.cpp`, four of which trigger for datasets with more than 2,147,483,647 vectors (`INT_MAX`), and one that can trigger regardless of dataset size. ## Bug 1: Integer truncation in fast subsampling — out-of-bounds memory access **Location**: `subsample_training_set()`, line 96 **Before**: ```cpp std::vector<int> perm; // ... perm[i] = rng.rand_int(nx); ``` **Bug**: `rand_int(int max)` takes an `int` parameter. When `nx` is `idx_t` (`int64_t`) and exceeds `INT_MAX`, the implicit narrowing conversion truncates `nx` to `int`. On two's complement (all target platforms), a value like `3,000,000,000` becomes `-1,294,967,296`. The function then generates a "random" index in a garbage range. These values are stored in `perm` and used as array indices: ```cpp memcpy(x_new + i * line_size, x + perm[i] * line_size, line_size); ``` A negative `perm[i]` produces an out-of-bounds read from before the start of `x`. This is undefined behavior that can crash or silently corrupt data. **Fix**: ```cpp std::vector<idx_t> perm; // ... perm[i] = rng.rand_int64() % nx; ``` Two changes: (1) `perm` is now `std::vector<idx_t>` so it can hold indices > `INT_MAX`. (2) `rand_int64()` returns `int64_t`, and `% nx` produces a value in `[0, nx)` without any narrowing. The result is stored losslessly in `idx_t`. ## Bug 2: Missing guard in standard subsampling path **Location**: `subsample_training_set()`, lines 99-108 **Before**: ```cpp } else { perm.resize(nx); rand_perm(perm.data(), nx, actual_seed); } ``` **Bug**: `rand_perm(int* perm, size_t n, int64_t seed)` takes `int*` and internally does `perm[i] = i`. When `nx > INT_MAX`, the value `i` (a `size_t`) is narrowed to `int` on assignment, wrapping to negative values. These negative values are then used as dataset indices — same out-of-bounds access as Bug 1. **Fix**: ```cpp } else { FAISS_THROW_IF_NOT_FMT( nx <= static_cast<idx_t>(std::numeric_limits<int>::max()), "Dataset too large (%" PRId64 ") for standard subsampling; " "set use_faster_subsampling=true", nx); std::vector<int> int_perm(nx); rand_perm(int_perm.data(), nx, actual_seed); perm.assign(int_perm.begin(), int_perm.end()); } ``` Three parts: (1) A guard that fails early with a clear error message directing the user to the fast path (which handles large datasets correctly via Bug 1 fix). (2) A temporary `std::vector<int>` to satisfy `rand_perm`'s `int*` API — safe because the guard guarantees `nx <= INT_MAX`. (3) Copy into the `idx_t` perm vector so both paths produce the same type for downstream code. We chose not to change `rand_perm`'s signature from `int*` to `idx_t*` because it is a public API in `faiss/utils/random.h` and changing it would break all callers. ## Bug 3: Infinite loop in split_clusters **Location**: `split_clusters()`, lines 239-265 **Before**: ```cpp for (cj = 0; true; cj = (cj + 1) % k) { float p = (hassign[cj] - 1.0) / (float)(n - k); float r = rng.rand_float(); if (r < p) { break; } } ``` **Bug**: This loop probabilistically selects a cluster to split (to replace an empty cluster). The probability of picking cluster `cj` is `p = (hassign[cj] - 1) / (n - k)`. When `hassign[cj] = 1` (cluster has exactly one vector), `p = 0 / (n - k) = 0`. No random float `r` satisfies `r < 0`, so that cluster is never picked. **Proof of infinite loop**: If all non-empty clusters have exactly 1 vector assigned (which happens with bad initialization, adversarial data, or too many clusters), then every `p = 0` and the loop condition `true` is never broken. The loop spins forever, hanging the process. Even in non-degenerate cases, the loop can be extremely slow. Example: `n = 10,000,000`, `k = 1000`, largest cluster has 50,000 vectors. Per-cluster probability: `p = 49999 / 9999000 ≈ 0.005`. Expected iterations to find a match: ~200. But with smaller clusters or larger `n`, this grows without bound. **Fix**: ```cpp size_t max_tries = 10 * k; size_t n_tries = 0; bool found = false; for (cj = 0; n_tries < max_tries; cj = (cj + 1) % k) { float p = (hassign[cj] - 1.0) / (float)(n - k); float r = rng.rand_float(); if (r < p) { found = true; break; } n_tries++; } if (!found) { cj = 0; for (size_t j = 1; j < k; j++) { if (hassign[j] > hassign[cj]) { cj = j; } } } ``` After `10 * k` attempts (10 full passes through all clusters), the loop falls back to deterministically picking the largest cluster. This is semantically correct because the probabilistic selection is already weighted by cluster size — larger clusters have higher `p`. The deterministic fallback produces the most likely outcome of the probabilistic selection. Termination is guaranteed in O(k) time. ## Bug 4: Integer overflow in objective accumulation loop **Location**: `Clustering::train_encoded()`, line 535 **Before**: ```cpp for (int j = 0; j < nx; j++) { obj += dis[j]; } ``` **Bug**: `nx` is `idx_t` (`int64_t`). When `nx > INT_MAX`, `int j` overflows at 2,147,483,647. Signed integer overflow is undefined behavior per the C++ standard. In practice on two's complement, `j` wraps to `-2,147,483,648`, which satisfies `j < nx`, so the loop continues with a negative index. `dis[j]` with negative `j` is an out-of-bounds read — crash or garbage accumulation. **Proof**: For `nx = 3,000,000,000`: - `j` increments from 0 to 2,147,483,647 (correct) - Next increment: UB, typically wraps to -2,147,483,648 - `-2,147,483,648 < 3,000,000,000` is true (signed/unsigned comparison promotes to unsigned, but even with signed comparison it's true) - `dis[-2147483648]` — out-of-bounds access **Fix**: ```cpp for (idx_t j = 0; j < nx; j++) { obj += dis[j]; } ``` `idx_t` matches `nx`'s type. The loop variable can represent all valid indices up to `nx`. Reviewed By: mnorris11 Differential Revision: D101624009 fbshipit-source-id: b961f2677f7e7b93642fe795cfe6ca77812573d3 |
||
|
Alibek Zhakubayev
|
3c4056d822 |
Fix race condition in HNSW::add_with_locks (#5129)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5129 ## Bug `HNSW::add_with_locks()` updates two shared member variables — `max_level` and `entry_point` — after releasing the per-node lock, without any synchronization: ```cpp omp_unset_lock(&locks[pt_id]); if (pt_level > max_level) { // read shared state max_level = pt_level; // write shared state entry_point = pt_id; // write shared state } ``` This function is called from inside `#pragma omp for` in `hnsw_add_vertices()` (IndexHNSW.cpp), meaning multiple threads execute it concurrently. The unprotected check-then-act pattern is a classic TOCTOU race condition. ## Proof by interleaving Suppose `max_level = 2` and two threads finish their link-building simultaneously: - Thread A: `pt_level = 4`, `pt_id = 100` - Thread B: `pt_level = 3`, `pt_id = 200` | Step | Thread A | Thread B | max_level | entry_point | |------|---------------------------------------|---------------------------------------|-----------|-------------| | 0 | — | — | 2 | (level-2 node) | | 1 | reads `4 > 2` -> true | | 2 | | | 2 | | reads `3 > 2` -> true | 2 | | | 3 | writes `max_level = 4` | | 4 | | | 4 | writes `entry_point = 100` | | 4 | 100 | | 5 | | writes `max_level = 3` | 3 | 100 | | 6 | | writes `entry_point = 200` | 3 | 200 | **Result**: `max_level = 3`, `entry_point = 200` (a node at level 3). But node 100 exists at level 4 — the true maximum. The HNSW invariant that `entry_point` is a node at `max_level` is violated. ## Consequence Search starts from `entry_point` and walks down from `max_level`. With a wrong entry point at a lower level, the upper levels of the graph are never traversed during search, leading to silently degraded recall. The index does not crash and still returns results — they are just worse. ## Fix Wrap the check-and-update in `#pragma omp critical` to make it atomic: ```cpp #pragma omp critical { if (pt_level > max_level) { max_level = pt_level; entry_point = pt_id; } } ``` This guarantees that only one thread executes the block at a time. In the interleaving above, Thread B would enter the critical section after Thread A completes, see `max_level = 4`, evaluate `3 > 4` as false, and correctly skip the write. ## Note on the read at line 561 `int level = max_level` reads `max_level` without synchronization. This is technically a data race under the C++ memory model, but it is benign: reading a stale value just means the greedy search starts one level too low, which the algorithm handles correctly (it still finds correct neighbors, just slightly less efficiently). Adding synchronization here would introduce overhead on every iteration of a hot loop for negligible benefit. ## Why existing tests did not catch this 1. **Tiny race window**: both threads must pass the `if` check in the few nanoseconds before either writes — extremely unlikely per run. 2. **Subtle consequence**: a wrong entry point degrades recall slightly but does not crash or return wrong types. Tests assert recall thresholds (e.g., recall > 0.9), not exact values. 3. **Rare trigger condition**: the race only fires when two nodes added concurrently both exceed the current `max_level`. Higher HNSW levels are exponentially less probable by design — most nodes are level 0, and the highest levels typically have only 1-2 nodes, making concurrent contention on `max_level` near-impossible in practice. Reviewed By: mnorris11 Differential Revision: D101444067 fbshipit-source-id: 82b9fdafed0b7c3cc26eb4d6c7e3536e6e12bee3 |
||
|
Tom Jackson
|
11e5d39160 |
Add optional persistent locks to IndexHNSW for incremental adds (#5031)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5031 `IndexHNSW` allocates an initializes locks for `ntotal+n` nodes on every call to `add()`. This makes batched insertion very costly, and incremental insertion prohibitively so. This diff introduces optional persistent locks for `IndexHNSW` to improve incremental `add()` performance. Previously, `omp_lock_t` arrays of size `ntotal+n` were created/destroyed on each `add()` call. Now locks can be retained via a new `retain_locks` flag (default: false), using a new `HNSW::Lock` RAII wrapper with geometric growth. RFC: Instead of `retain_locks` being the only way to opt into this new behavior, this could be inferred on the first incremental add. That is, clear the locks after insertion iff `n0 == 0`. Workloads which call `add()` once would be unaffected, but workloads which call `add()` repeatedly would 1) forego the clearing of the lock vector after `add()` call #2, and reuse locks for all subsequent calls. The downside would be the lack of the ability to reclaim the locks after insertion without HNSW-specific behavior at the call site. Reviewed By: mdouze Differential Revision: D98232750 fbshipit-source-id: ef55cd9e4eb79793267a29a06502a582873e6a74 |
||
|
generatedunixname923199350665076
|
bdf877f90c |
facebook-unused-include-check in IndexNNDescent.cpp (#5132)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5132 Reviewed By: limqiying, junjieqi Differential Revision: D101359141 fbshipit-source-id: 7d78875eed114367d4a45215e058f5fa9ebf06a1 |
||
|
generatedunixname1242365081173837
|
632427fd0d |
fbcode/faiss/invlists (#5048)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5048 Reviewed By: mnorris11, hanle11 Differential Revision: D99419595 fbshipit-source-id: 9c1214c6f4b88bf41e9d1851dd0acb5c7c5001ef |
||
|
Matthijs Douze
|
e23c66175d |
Replace dispatch_HammingComputer with with_HammingComputer (#5126)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5126 Replace the dispatch_HammingComputer + Run_XXX consumer struct pattern with with_HammingComputer that takes a C++20 template lambda directly. This eliminates boilerplate wrapper structs across 8 files. Before: struct Run_foo { using T = void; template<class HC, class... T> void f(T... a) { foo<HC>(a...); } }; Run_foo r; dispatch_HammingComputer(code_size, r, args...); After: with_HammingComputer(code_size, [&]<class HC>() { foo<HC>(args...); }); Reviewed By: algoriddle Differential Revision: D101350351 fbshipit-source-id: 02a346e8c33ffdb49153cbe13415b748f0a1e847 |
||
|
Gergely Szilvasy
|
ed7e1f2435 |
Hoist SIMD dispatch outside loops in 4 call sites (#5074)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5074 Move `with_simd_level` / `with_simd_level_256bit` calls outside the enclosing loops so the SIMD level is resolved once rather than on every iteration. Sites fixed: - distances.cpp: knn_inner_products_by_idx, knn_L2sqr_by_idx - NeuralNet.cpp: ZnLUTCodec::encode - ClusteringInitialization.cpp: init_kmpp_plus_plus Reviewed By: mdouze Differential Revision: D100144174 fbshipit-source-id: bd2369ed4fd9c3b5b54e435c7ee66a03f0e152df |
||
|
Gergely Szilvasy
|
6e64c5d687 |
Bump GCC pin in faiss-gpu conda recipe to fix AVX2 SIMD miscompilation (#5125)
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5125 The faiss-gpu conda recipe pins `{{ compiler('cxx') }} =12.4` (GCC 12.4). GCC 12.4 miscompiles the 16-bin SIMD histogram reduction in `partitioning_simdlib256.h`, producing correct results for bins 0-7 but near-zero for bins 8-15. This causes `test_16bin_bounded_bigrange` in `TestHistograms_AVX2` to fail in the CUDA 12.6 GPU nightly. The bug is in GCC 12's code generation for the AVX2 cross-lane reduction chain (`_mm256_hadd_epi16` → `_mm256_permute2f128_si256` → `_mm256_permutevar8x32_epi32`). GCC 13 and 14 both compile this correctly. The CPU-only `faiss/meta.yaml` leaves the compiler unpinned (gets GCC 14), which is why only the GPU nightly fails. The GCC 12.4 pin was introduced in D84193438 as part of a batch nightly fix — not a deliberate CUDA compatibility constraint. CUDA 12.6 supports up to GCC 13.x as host compiler (GCC 14 requires CUDA 12.9+), so we widen the pin to `>=12.4,<14`. Reproduced locally: GCC 12.4 fails, GCC 13.4 passes, GCC 14.2 passes — all on the same faiss source, same test, same machine. Reviewed By: mdouze Differential Revision: D101601476 fbshipit-source-id: 8e36c83a9df67ba66408faa4ca392e1bd46d7c87 |