Summary:
## What
Split the existing VPOPCNTDQ RaBitQ and Hamming kernels from the full Sapphire Rapids SIMD level.
DD builds now expose an `AVX512_VPOPCNT` capability for CPUs such as Ice Lake and Zen 4. The fallback chain is:
`AVX512_SPR -> AVX512_VPOPCNT -> AVX512 -> AVX2 -> NONE`
The ordinary static AVX512 target remains unchanged and contains no VPOPCNT instructions.
## Why
The VPOPCNT kernels only require baseline AVX-512 plus `AVX512_VPOPCNTDQ`. Tying them to the SPR level unnecessarily excluded CPUs that support VPOPCNTDQ but not AVX512-FP16, notably Zen 4.
## Changes made while importing
Two changes were needed on top of the pull request.
**Feature detection read the wrong CPUID leaf.** `AVX512_VPOPCNTDQ` and `AVX512_VNNI` were read from leaf 1 ECX. Both live in leaf 7 subleaf 0 ECX, at bits 14 and 11. Leaf 1 ECX holds unrelated bits at those positions, so the new level was never selected on any CPU. Measured on an AMD Genoa host before the fix: the level reported as unavailable and detection returned `AVX512`. After the fix the same host reports `AVX512_VPOPCNT`.
**The build definitions outside CMake were not updated.** The pull request renames three files. The other build description still named the old paths and had no entry for the new level, so the library did not build. This adds the level constant, its compiler flags, its `COMPILE_SIMD_AVX512_VPOPCNT` define, and the renamed paths.
**The new level was not registered in the compiled-level mask.** `compiled_simd_levels()` reports which levels a binary holds, and `FAISS_SIMD_LEVEL` walks down from the requested level until it finds one that is present. The new level had no entry, so the override always stepped past it to `AVX512` and printed that the level was not compiled in. This was reported by `test_dispatch_with_env_var` on the open-source dynamic-dispatch build.
**The test helper inferred available levels from `/proc/cpuinfo`.** A CPU feature does not imply that a level is compiled into the build, so the helper listed a level the override then refused. It now asks `SIMDConfig::is_simd_level_available()`, which reports compiled and supported. That is a subset of compiled, so every level it returns is one the override accepts.
The mask and helper names also follow the naming used in the parent diff, so `AVAILABLE_SIMD_LEVELS_BASE_WITH_VPOPCNT` and `with_simd_level_with_vpopcnt`.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5531
Test Plan:
```
buck2 test fbcode//mode/opt -c faiss.dynamic_dispatch=true \
fbcode//faiss/tests:test_simd_levels fbcode//faiss/tests:test_rabitq_simd
```
Result: all pass.
The build failures the imported version reported are cleared. All three configurations build:
```
buck2 build fbcode//mode/dev fbcode//faiss:faiss fbcode//faiss:faiss_no_multithreading fbcode//faiss:faiss_omp_mock
buck2 build fbcode//mode/opt -c faiss.dynamic_dispatch=true <same three targets>
buck2 build fbcode//mode/dev -c fbcode.arch=aarch64 <same three targets>
```
Detection on an AMD Genoa host, which has AVX512_VPOPCNTDQ and AVX512_BF16 but not AVX512_FP16:
| | before the CPUID fix | after |
| --- | --- | --- |
| detected level | `AVX512` | `AVX512_VPOPCNT` |
| `AVX512_VPOPCNT` available | 0 | 1 |
| `AVX512_SPR` available | 0 | 0 |
Reviewed By: alibeklfc
Differential Revision: D118969754
Pulled By: mnorris11
fbshipit-source-id: d1ca52b614829a3d4b322c9fecf0c004f2174dab
Summary:
I've verified the successful compilation via docker + qemu combination.
Also, this PR introduces all the needed overrides, which rely on default implementation, but later can be populated with the vectorized code.
Also, fixes a typo in SQ4U.
algoriddle I would appreciate if you could take a look whenever you have time. Thanks!
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5156
Reviewed By: mdouze
Differential Revision: D102911491
Pulled By: mnorris11
fbshipit-source-id: f3555987cdbf8d32a4fbc9d07ec2307f0f8a112a
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4798
Add ARM SVE (Scalable Vector Extension) support for FAISS distance
functions.
- Add ARM_SVE to SIMDLevel enum and dispatch macros
- Add distances_arm_sve.cpp with SVE implementations
- Wire up SVE in both build systems:
- Buck: SIMD_FILES/SIMD_FLAGS in xplat.bzl
- CMake: FAISS_SIMD_SVE_SRC, COMPILE_SIMD_ARM_SVE definition, and per-file -march=armv8.2-a+sve flags in DD mode
- SVE is disabled by default (ARM_SVE: False in dispatch_config)
Reviewed By: mdouze
Differential Revision: D92819013
fbshipit-source-id: cb37fc09efb34364fc88dde3389b76232fb29d59
lyang24