Files
faiss/tests/test_simd_levels.cpp
lyang24 e6bd1d252c Add AVX512 VPOPCNT dynamic dispatch level (#5531)
Summary:
## What

  Split the existing VPOPCNTDQ RaBitQ and Hamming kernels from the full Sapphire Rapids SIMD level.

  DD builds now expose an `AVX512_VPOPCNT` capability for CPUs such as Ice Lake and Zen 4. The fallback chain is:

  `AVX512_SPR -> AVX512_VPOPCNT -> AVX512 -> AVX2 -> NONE`

  The ordinary static AVX512 target remains unchanged and contains no VPOPCNT instructions.

  ## Why

  The VPOPCNT kernels only require baseline AVX-512 plus `AVX512_VPOPCNTDQ`. Tying them to the SPR level unnecessarily excluded CPUs that support VPOPCNTDQ but not AVX512-FP16, notably Zen 4.

  ## Changes made while importing

  Two changes were needed on top of the pull request.

  **Feature detection read the wrong CPUID leaf.** `AVX512_VPOPCNTDQ` and `AVX512_VNNI` were read from leaf 1 ECX. Both live in leaf 7 subleaf 0 ECX, at bits 14 and 11. Leaf 1 ECX holds unrelated bits at those positions, so the new level was never selected on any CPU. Measured on an AMD Genoa host before the fix: the level reported as unavailable and detection returned `AVX512`. After the fix the same host reports `AVX512_VPOPCNT`.

  **The build definitions outside CMake were not updated.** The pull request renames three files. The other build description still named the old paths and had no entry for the new level, so the library did not build. This adds the level constant, its compiler flags, its `COMPILE_SIMD_AVX512_VPOPCNT` define, and the renamed paths.

**The new level was not registered in the compiled-level mask.** `compiled_simd_levels()` reports which levels a binary holds, and `FAISS_SIMD_LEVEL` walks down from the requested level until it finds one that is present. The new level had no entry, so the override always stepped past it to `AVX512` and printed that the level was not compiled in. This was reported by `test_dispatch_with_env_var` on the open-source dynamic-dispatch build.

  **The test helper inferred available levels from `/proc/cpuinfo`.** A CPU feature does not imply that a level is compiled into the build, so the helper listed a level the override then refused. It now asks `SIMDConfig::is_simd_level_available()`, which reports compiled and supported. That is a subset of compiled, so every level it returns is one the override accepts.

  The mask and helper names also follow the naming used in the parent diff, so `AVAILABLE_SIMD_LEVELS_BASE_WITH_VPOPCNT` and `with_simd_level_with_vpopcnt`.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5531

Test Plan:
```
buck2 test fbcode//mode/opt -c faiss.dynamic_dispatch=true \
  fbcode//faiss/tests:test_simd_levels fbcode//faiss/tests:test_rabitq_simd
```
Result: all pass.

The build failures the imported version reported are cleared. All three configurations build:
```
buck2 build fbcode//mode/dev fbcode//faiss:faiss fbcode//faiss:faiss_no_multithreading fbcode//faiss:faiss_omp_mock
buck2 build fbcode//mode/opt -c faiss.dynamic_dispatch=true <same three targets>
buck2 build fbcode//mode/dev -c fbcode.arch=aarch64 <same three targets>
```

Detection on an AMD Genoa host, which has AVX512_VPOPCNTDQ and AVX512_BF16 but not AVX512_FP16:

| | before the CPUID fix | after |
| --- | --- | --- |
| detected level | `AVX512` | `AVX512_VPOPCNT` |
| `AVX512_VPOPCNT` available | 0 | 1 |
| `AVX512_SPR` available | 0 | 0 |

Reviewed By: alibeklfc

Differential Revision: D118969754

Pulled By: mnorris11

fbshipit-source-id: d1ca52b614829a3d4b322c9fecf0c004f2174dab
2026-09-10 09:32:36 -07:00

194 lines
7.3 KiB
C++

/*
* Copyright (c) Meta Platforms, Inc. and affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
// Core SIMDConfig API tests - works in both static and DD modes.
// Hardware execution tests (DD-only) are in separate files:
// - test_simd_levels_x86_avx2.cpp (compiled with AVX2 flags)
// - test_simd_levels_x86_avx512.cpp (compiled with AVX512 flags)
#include <gtest/gtest.h>
#include <faiss/impl/FaissException.h>
#include <faiss/utils/simd_levels.h>
#include <faiss/utils/utils.h>
// Helper to check if we're in DD mode
static bool is_dd_mode() {
return faiss::get_compile_options().find("DD") != std::string::npos;
}
TEST(SIMDConfig, get_level_returns_valid_level) {
// Works in both static and DD modes
faiss::SIMDLevel level = faiss::SIMDConfig::get_level();
EXPECT_NE(level, faiss::SIMDLevel::COUNT);
EXPECT_GE(static_cast<int>(level), 0);
EXPECT_LT(
static_cast<int>(level), static_cast<int>(faiss::SIMDLevel::COUNT));
}
TEST(SIMDConfig, supported_simd_levels_not_empty) {
// Works in both static and DD modes
// Current level should always be in supported levels
EXPECT_TRUE(
faiss::SIMDConfig::is_simd_level_available(
faiss::SIMDConfig::get_level()));
}
TEST(SIMDConfig, set_level_to_supported_level_succeeds) {
// Works in both static and DD modes
faiss::SIMDLevel original_level = faiss::SIMDConfig::get_level();
// Setting to any supported level should succeed
for (int i = 0; i < static_cast<int>(faiss::SIMDLevel::COUNT); ++i) {
auto level = static_cast<faiss::SIMDLevel>(i);
if (faiss::SIMDConfig::is_simd_level_available(level)) {
EXPECT_NO_THROW(faiss::SIMDConfig::set_level(level))
<< "set_level(" << faiss::to_string(level)
<< ") should succeed";
EXPECT_EQ(faiss::SIMDConfig::get_level(), level);
}
}
// Restore original level
faiss::SIMDConfig::set_level(original_level);
}
TEST(SIMDConfig, set_level_to_unsupported_level_throws) {
// Works in both static and DD modes
// Find a level that's NOT supported
faiss::SIMDLevel unsupported = faiss::SIMDLevel::COUNT;
for (int i = 0; i < static_cast<int>(faiss::SIMDLevel::COUNT); ++i) {
auto level = static_cast<faiss::SIMDLevel>(i);
if (!faiss::SIMDConfig::is_simd_level_available(level)) {
unsupported = level;
break;
}
}
if (unsupported != faiss::SIMDLevel::COUNT) {
EXPECT_THROW(
faiss::SIMDConfig::set_level(unsupported),
faiss::FaissException)
<< "set_level(" << faiss::to_string(unsupported)
<< ") should throw";
}
}
TEST(SIMDConfig, static_mode_has_single_level) {
// Static mode should have exactly 1 level: the compiled-in level
if (is_dd_mode()) {
GTEST_SKIP() << "DD build - has multiple levels";
}
int count = 0;
for (int i = 0; i < static_cast<int>(faiss::SIMDLevel::COUNT); ++i) {
if (faiss::SIMDConfig::is_simd_level_available(
static_cast<faiss::SIMDLevel>(i))) {
++count;
}
}
EXPECT_EQ(count, 1)
<< "Static mode should have exactly 1 level (compiled-in)";
}
TEST(SIMDConfig, get_level_name_matches_level) {
// Works in both static and DD modes
faiss::SIMDLevel level = faiss::SIMDConfig::get_level();
std::string name = faiss::SIMDConfig::get_level_name();
std::string expected = faiss::to_string(level);
EXPECT_EQ(name, expected);
}
TEST(SIMDConfig, get_dispatched_level_matches_get_level) {
// Works in both static and DD modes
// Verifies that dispatch mechanism returns the same level as get_level()
faiss::SIMDLevel level = faiss::SIMDConfig::get_level();
faiss::SIMDLevel dispatched = faiss::SIMDConfig::get_dispatched_level();
EXPECT_EQ(level, dispatched)
<< "get_level() returned " << faiss::to_string(level)
<< " but get_dispatched_level() returned "
<< faiss::to_string(dispatched);
}
TEST(SIMDLevel, to_string_all_levels) {
EXPECT_EQ("NONE", faiss::to_string(faiss::SIMDLevel::NONE));
EXPECT_EQ("AVX2", faiss::to_string(faiss::SIMDLevel::AVX2));
EXPECT_EQ("AVX512", faiss::to_string(faiss::SIMDLevel::AVX512));
EXPECT_EQ(
"AVX512_VPOPCNT",
faiss::to_string(faiss::SIMDLevel::AVX512_VPOPCNT));
EXPECT_EQ("AVX512_SPR", faiss::to_string(faiss::SIMDLevel::AVX512_SPR));
EXPECT_EQ("ARM_NEON", faiss::to_string(faiss::SIMDLevel::ARM_NEON));
EXPECT_EQ("ARM_SVE", faiss::to_string(faiss::SIMDLevel::ARM_SVE));
EXPECT_EQ("RISCV_RVV", faiss::to_string(faiss::SIMDLevel::RISCV_RVV));
// COUNT should throw
EXPECT_THROW(
faiss::to_string(faiss::SIMDLevel::COUNT), faiss::FaissException);
}
TEST(SIMDLevel, to_simd_level_all_strings) {
EXPECT_EQ(faiss::SIMDLevel::NONE, faiss::to_simd_level("NONE"));
EXPECT_EQ(faiss::SIMDLevel::AVX2, faiss::to_simd_level("AVX2"));
EXPECT_EQ(faiss::SIMDLevel::AVX512, faiss::to_simd_level("AVX512"));
EXPECT_EQ(
faiss::SIMDLevel::AVX512_VPOPCNT,
faiss::to_simd_level("AVX512_VPOPCNT"));
EXPECT_EQ(faiss::SIMDLevel::AVX512_SPR, faiss::to_simd_level("AVX512_SPR"));
EXPECT_EQ(faiss::SIMDLevel::ARM_NEON, faiss::to_simd_level("ARM_NEON"));
EXPECT_EQ(faiss::SIMDLevel::ARM_SVE, faiss::to_simd_level("ARM_SVE"));
EXPECT_EQ(faiss::SIMDLevel::RISCV_RVV, faiss::to_simd_level("RISCV_RVV"));
// Invalid strings should throw
EXPECT_THROW(faiss::to_simd_level("INVALID"), faiss::FaissException);
EXPECT_THROW(faiss::to_simd_level(""), faiss::FaissException);
}
TEST(SIMDConfig, modern_hardware_has_simd_support) {
// In DD mode, verify modern hardware detects SIMD support
if (!is_dd_mode()) {
GTEST_SKIP() << "Static build - level is fixed at compile time";
}
faiss::SIMDLevel detected = faiss::SIMDConfig::auto_detect_simd_level();
#if defined(__x86_64__) || defined(_M_X64)
// All modern x86_64 machines (Haswell 2013+) support at least AVX2
EXPECT_NE(detected, faiss::SIMDLevel::NONE)
<< "x86_64 machines should support at least AVX2";
#elif defined(__aarch64__) || defined(_M_ARM64)
// NEON is mandatory on aarch64
EXPECT_NE(detected, faiss::SIMDLevel::NONE)
<< "ARM64 machines should support at least NEON";
#endif
}
TEST(CompileOptions, lists_expected_levels) {
std::string options = faiss::get_compile_options();
// All supported levels (except NONE) should be in compile options
for (int i = 0; i < static_cast<int>(faiss::SIMDLevel::COUNT); ++i) {
auto level = static_cast<faiss::SIMDLevel>(i);
if (!faiss::SIMDConfig::is_simd_level_available(level)) {
continue;
}
if (level == faiss::SIMDLevel::NONE) {
continue; // NONE is not reported in options
}
std::string name = faiss::to_string(level);
EXPECT_NE(options.find(name), std::string::npos)
<< "Supported level " << name
<< " should be in compile options: " << options;
}
// DD mode should have "DD" marker
if (is_dd_mode()) {
EXPECT_NE(options.find("DD"), std::string::npos)
<< "DD mode should have 'DD' in compile options: " << options;
}
}