Summary:
- The current Faiss conda recipe does not specify the OpenBLAS threading variant.
- On Linux AArch64, the dependency solver selects the pthreads build.
- This causes problems when Faiss calls BLAS from within an existing OpenMP parallel region due to nested threading
- As a result, the benchmark below (with an OOB conda install of faiss) only yields 7% of the throughput reachable when an OMP build of OpenBLAS is selected (what this PR does)
- This PR fixes the issue, and as a result accelerates affected workloads by ~ 14x
```
# SPDX-FileCopyrightText: Copyright 2026 Arm Limited and/or its affiliates <[email protected]>
# SPDX-License-Identifier: MIT
import ctypes
import statistics
import time
import faiss
import numpy as np
rng = np.random.default_rng(1234)
d = 128
nlist = 4096
nq = 40_000
quantizer = faiss.IndexFlatL2(d)
quantizer.add(rng.random((nlist, d), dtype=np.float32))
index = faiss.IndexIVFPQFastScan(quantizer, d, nlist, 128, 4)
# index is empty - this only triggeres the course quantizer
index.is_trained = True
index.nprobe = 64
xq = rng.random((nq, d), dtype=np.float32)
openblas = ctypes.CDLL("libopenblas.so.0")
openblas.openblas_get_parallel.restype = ctypes.c_int
openblas.openblas_get_num_threads.restype = ctypes.c_int
print(f"nq={nq:,}, d={d}, nlist={nlist}, nprobe={index.nprobe}")
print(f"OpenMP threads={faiss.omp_get_max_threads()}")
print(
f"OpenBLAS parallel={openblas.openblas_get_parallel()}, "
f"threads={openblas.openblas_get_num_threads()}"
)
print(f"BLAS threshold={faiss.cvar.distance_compute_blas_threshold}")
index.search(xq, 10)
times = []
for _ in range(20):
start = time.perf_counter()
index.search(xq, 10)
times.append(time.perf_counter() - start)
elapsed = statistics.median(times)
print(f"median={elapsed:.6f}s, throughput={nq / elapsed:,.0f} QPS")
```
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5669
Reviewed By: weixianghong
Differential Revision: D124380509
Pulled By: mnorris11
fbshipit-source-id: 16e1521e7aaf7d65c6a4ce8e404a47bdb26c1198
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5507
`SIMDConfig::has_dynamic_dispatch()` was defined inline in `simd_levels.h`, where it resolved `#ifdef FAISS_ENABLE_DD` at include time. That macro is only set while faiss itself is compiled, so any code outside the library that included the header always evaluated it to `false` — reporting that dynamic dispatch was unavailable even when linked against a faiss built with it enabled.
Callers using it to decide whether `SIMDConfig::set_level()` will do anything would therefore skip setting the SIMD level, and silently fall back to the auto-detected one.
Move the definition into `simd_levels.cpp`, which is compiled as part of the library and does see the macro. Callers now get the answer for the faiss they actually link against.
The function is no longer `constexpr` and so cannot be used in constant expressions. It had no such uses.
Also bumps the conda `openblas` pin for non-x86_64 platforms from 0.3.33 to 0.3.34, which is required to get the `Linux arm64 (conda)` build green. conda-forge now serves `libopenblas 0.3.34`, which depends on `openblas >=0.3.34,<0.3.35`, so the old pin left the environment unsolvable:
```
ExplainedDependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-aarch64
- package libfaiss-1.15.0-py3.12_h3f4af3b_10_cpu requires openblas 0.3.33.*,
but none of the providers can be installed
```
The separate `libopenblas` pins for osx are left at 0.3.33, since that package is resolving fine.
Reviewed By: junjieqi
Differential Revision: D115218647
fbshipit-source-id: 7163e18011dc9b82e93cd68fb600d24ef7012732
Summary:
This pull request adds support for memory-mapped I/O (mmap) loading of static SVS (Scalable Vector Search) indices in Faiss, enabling faster and more memory-efficient index loading. The main changes include implementing mmap-based deserialization for static Vamana and Flat indices, updating the index loading logic to use mmap when requested, and adding comprehensive tests for the new functionality.
**Memory-mapped index loading support:**
* Added `map_to` methods to `IndexSVSVamana` and `IndexSVSFlat` for loading static indices directly from memory-mapped files, including logic to manage memory mapping ownership and adjust file positions after mapping.
**Index and file structure updates:**
* Extended `IndexSVSVamana` and `IndexSVSFlat` classes to hold a reference to the mmap owner, ensuring the mapped memory remains valid for the lifetime of the index. Also added forward declarations and includes for mmap-related types.
* Modified `reset` methods to release memory mappings when the index is reset, preventing resource leaks.
**Testing and interface improvements:**
* Enhanced test utilities and added new tests to verify correct mmap loading for various static Vamana index types, including FP32, FP16, SQI8, LVQ4x4, and LeanVec4x4 in `tests/test_svs.cpp`
These changes collectively provide robust and efficient support for memory-mapped loading of static SVS indices, improving both performance and usability for large-scale vector search applications.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5271
Reviewed By: alibeklfc
Differential Revision: D108313487
Pulled By: mnorris11
fbshipit-source-id: 4ae6dc3e69363b0c792cd56ad26b01bf6a853d28
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5209
Three fixes for GitHub Actions CI build/test failures:
1. sqlite3 "database is locked" error (cuVS cmake builds)
conda-libmamba-solver 26.4.0 uses a SQLite shards cache for package index
data. When consecutive conda install commands run in the same job step,
the SQLite lock from the first install is not fully released before the
second install begins, causing:
sqlite3.OperationalError: database is locked
Fix: Add conda clean --index-cache between the cuVS and PyTorch conda
installs to flush the shards cache and release the lock.
2. OpenBLAS version pin conflict (ARM64 conda builds)
conda-forge removed openblas 0.3.32, making it unresolvable. The conda
recipes pinned 0.3.32 for non-x86_64 (ARM64) and macOS builds, causing:
package libfaiss requires openblas 0.3.32.*, but none of the providers can be installed
Fix: Bump all openblas/libopenblas pins from 0.3.32 to 0.3.33 in the
conda recipes and from 0.3.29 to 0.3.33 in the cmake CI action.
3. Flaky SVS LeanVec test aborts (SVSLL tests)
IndexSVSVamanaLeanVec::~IndexSVSVamanaLeanVec() explicitly called
IndexSVSVamana::~IndexSVSVamana(), but C++ destructor chaining already
calls the base destructor automatically. This caused double-destruction of
base class members (notably std::vector<float> stored_vectors), resulting
in a double-free and non-deterministic SIGABRT. The IVF variant
(IndexSVSIVFLeanVec) correctly omits this call.
Fix: Remove the explicit base destructor call; let C++ handle it.
Reviewed By: pankajsingh88
Differential Revision: D104902441
fbshipit-source-id: c4ab80a4c1fdeb9976c4e089dfb95814284824a7
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5192
conda-forge added MKL 2026.0.0 which uses a new soname (libmkl_intel_lp64.so.3
instead of .so.2). The build env now resolves mkl-devel >=2024.2.2 to 2026.0.0,
but the test env gets mkl 2025.3.1 because pytorch 2.10.0 is pinned to that
version. This soname mismatch causes ImportError at test time.
Adding <2026 upper bound forces both build and test envs to use MKL 2025.x,
keeping sonames consistent until pytorch supports MKL 2026.
🤖 This diff was automatically generated by myclaw_faiss_monitor
Reviewed By: mnorris11
Differential Revision: D104219406
fbshipit-source-id: e4d256eade13fe4e193a8ed8a48bbdb53616898e
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5180
finally passes: https://github.com/facebookresearch/faiss/actions/runs/25449492630?pr=5180
Two GitHub Actions CI jobs are failing due to conda environment resolution issues.
**cuVS**: The `cuda-toolkit=12.9` metapackage includes visual tools (nsight-compute) that require `krb5 >=1.21.3,<1.22.0a0`, which conflicts with `mamba`/`libcurl` needing `krb5 >=1.22.2,<1.23.0a0`. It also has Windows-only variants requiring `__win`. Removing `cuda-toolkit=12.9` and keeping only `cuda-version=12.9` lets `libcuvs` pull in just the CUDA packages it actually needs.
**ROCm**: The latest miniforge ships `conda 25.7.0` with `conda-libmamba-solver 25.11.0`, which references `RepodataCache.cache_path_shards` — an attribute that does not exist in `conda 25.7.0`. The previous constraint `"conda<=25.07"` (numerically equal to `<=25.7.0`) did not force a downgrade. Pinning to `"conda<25.5"` along with `"conda-libmamba-solver<25.9"` ensures a compatible version pair.
Reviewed By: alibeklfc
Differential Revision: D103892827
fbshipit-source-id: 88c91a4741950cac4fe1f6ab56ae00fcf19c96cd
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5125
The faiss-gpu conda recipe pins `{{ compiler('cxx') }} =12.4` (GCC 12.4). GCC 12.4 miscompiles the 16-bin SIMD histogram reduction in `partitioning_simdlib256.h`, producing correct results for bins 0-7 but near-zero for bins 8-15. This causes `test_16bin_bounded_bigrange` in `TestHistograms_AVX2` to fail in the CUDA 12.6 GPU nightly.
The bug is in GCC 12's code generation for the AVX2 cross-lane reduction chain (`_mm256_hadd_epi16` → `_mm256_permute2f128_si256` → `_mm256_permutevar8x32_epi32`). GCC 13 and 14 both compile this correctly. The CPU-only `faiss/meta.yaml` leaves the compiler unpinned (gets GCC 14), which is why only the GPU nightly fails.
The GCC 12.4 pin was introduced in D84193438 as part of a batch nightly fix — not a deliberate CUDA compatibility constraint. CUDA 12.6 supports up to GCC 13.x as host compiler (GCC 14 requires CUDA 12.9+), so we widen the pin to `>=12.4,<14`.
Reproduced locally: GCC 12.4 fails, GCC 13.4 passes, GCC 14.2 passes — all on the same faiss source, same test, same machine.
Reviewed By: mdouze
Differential Revision: D101601476
fbshipit-source-id: 8e36c83a9df67ba66408faa4ca392e1bd46d7c87
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4994🤖 This diff was automatically generated by myclaw_faiss_monitor
The arm64 nightly build (Linux arm64) started failing on Mar 26 because
conda-forge removed openblas 0.3.30 for linux-aarch64. The solver cannot
satisfy the exact pin `openblas =0.3.30` since only 0.3.32 is available.
Fix: bump `openblas =0.3.30` to `=0.3.32` in all three conda recipes
(faiss, faiss-gpu, faiss-gpu-cuvs) for the `# [not x86_64]` targets
(arm64 Linux). OSX libopenblas pins are unaffected (different package).
Reviewed By: mnorris11
Differential Revision: D98293081
fbshipit-source-id: d2371a6e1bf1c920fb130cc35fb385ec43aa8e56
Summary:
Installs [SVS v0.2.0](https://github.com/intel/ScalableVectorSearch/releases/tag/v0.2.0) which includes updates for LeanVec OOD and IVF to be integrated in subsequent PRs. Also removes intel conda channel logic as libsvs-runtime is now available on conda-forge.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4860
Reviewed By: alibeklfc
Differential Revision: D95164622
Pulled By: mnorris11
fbshipit-source-id: 3b2bbaa8c5283e6aa5ac5ee250e8eac4ccba0e5a
Summary:
This PR migrates from using SVS via tarball to using a conda package
Currently the package is available at the intel channel. In future releases, it will be available from conda-forge and the channel configuration can be removed
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4747
Reviewed By: subhadeepkaran
Differential Revision: D91484831
Pulled By: alibeklfc
fbshipit-source-id: b94dba1f03b4083ae8aa35cdd258ad71c421d30b
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/issues/4450
This pull request introduces support for [Intel ScalableVectorSearch](https://github.com/intel/ScalableVectorSearch), integrating Intel's proprietary LVQ and LeanVec technologies in binary form. The following index types are now supported: `IndexSVSVamana`, `IndexSVSVamanaLVQ`, `IndexSVSVamanaLeanVec`, and `IndexSVSFlat`. `IndexSVSVamana` and `IndexSVSFlat` utilizes SVS open-source float32/float16/int8 implementation.
Key features and enhancements include:
- Implemented `search`, `add`, and `remove_ids`
- Implemented filter search and `range_search`
- Enabled SVS open-source fp16 and scalar quantization implementation
- Enabled factory methods
- Enabled save/load
- Enabled Python bindings
- Enable fallback mechanism that fallbacks to 8-bit scalar quantization if LVQ/LeanVec is used on non-intel hardware CPUs
- Added examples in both Python and C++ under the `tutorial/` directory
- Added `FAISS_ENABLE_SVS` flag to allows users to optionally enable SVS
TODOs:
- Support more indices such as `IndexSVSIVF` and `IndexSVSFlat` with LVQ and LeanVec
- Documentation
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4548
Reviewed By: subhadeepkaran
Differential Revision: D84713772
Pulled By: mnorris11
fbshipit-source-id: 290365eb3337f685dd46696935b621c57dad3174
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4604
Nightly for GPU builds (cuvs and non-cuvs) is failing again after numpy2 upgrade.
- Fixed it by upgrading CUDA, using the right deps (pytorch-gpu instead of the deprecated pytorch-cuda), pinning pytorch >=2.7 for GPU builds, upgrading MKL
- SVE requires pytorch<2.5 for the testing flow, otherwise it times out for 1 test.
Reviewed By: subhadeepkaran
Differential Revision: D84193438
fbshipit-source-id: 71868d5f9deb973ee431e10ee765a450d4f1f876
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4523
This upgrades OSS Faiss to numpy 2.3.3. I tried to fix issues based on https://numpy.org/devdocs/numpy_2_0_migration_guide.html.
Overall, numpy2 causes some test issues on various platforms/optimization levels.
Test changes that are OK:
- test_contrib.py: test_float: changing to use item(): this is just the new way of getting the value, like recommended in the migration guide.
Test changes that indicate some problem:
- test_contrib.py: test_ivf_train_2level: This fails on many platforms (avx2, conda installs). I could not find what the issue was. I had to increase the bounds slightly.
- test_index_accuracy.py: test_OIVFPQ: This fails just on AVX512. I had to increase the iterations from 10 to 12.
- test_local_search_quantizer.py and test_standalone_codec.py: Some of these break on Windows. Others needed lighter tolerances.
Debugging attempts: Especially for test_ivf_train_2level, I tried to unsuccessfully track down what might have changed. I suspected the type promotion change from numpy1 to 2.
```
- np.float32(3) + 3. now returns a float32 when it previously returned a float64.
- np.array([3], dtype=np.float32) + np.float64(3) will now return a float64 array. (The higher precision of the scalar is not ignored.)
```
I changed all datasets to be explicitly float64s, and checked for any constants in the code path. No luck.
Devmate was less than useless, it was actively a waste of time (as usual...)
Reviewed By: subhadeepkaran
Differential Revision: D79900113
fbshipit-source-id: 541cfa6f1fa1d60c84d855ce71f0e1de7ea7c7f1
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4442
Trying to set up conda nightly for 12.4.
It already works in cmake, builds are using 12.4 by default
Reviewed By: ramilbakhshyiev
Differential Revision: D78716765
fbshipit-source-id: 1a7c0d71da40e58df2036647f39aab967cf676ff
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4198
1. pins lief due to `AttributeError: type object 'CLASS' has no attribute 'CLASS64'` (just set it to last passing nightly version)
2. pins mkl in gpu builds due to it trying to pull in 2024.2.2 which conflicts with 2023 in the libfaiss.
Added nightlies to make sure they pass https://github.com/facebookresearch/faiss/actions/runs/13422430425/job/37498020894. Not all passed: I'm not sure the `build-pull-request / Linux x86_64 GPU w/ cuVS nightlies (CUDA 12.4.0)` nightly is actually broken, but this unblocks the PR builds for now.
Reviewed By: junjieqi
Differential Revision: D69860604
fbshipit-source-id: 2da623c71b03c22d581b78655253a863fbafd3ed
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4126
Good resource on overriding channels to make sure we aren't using `defaults`:https://stackoverflow.com/questions/67695893/how-do-i-completely-purge-and-disable-the-default-channel-in-anaconda-and-switch
Explanation of changes:
-
- changed to miniforge from miniconda: this ensures we only pull in from conda-defaults when creating the environment
- architecture: ARM64 and aarch64 are the same thing. But there is no miniforge package for ARM64, so we need to make it check for aarch64 instead. However, mac breaks this rule, and does have macOS-arm64! So there is a conditional for mac to use arm64. https://github.com/conda-forge/miniforge/releases/
- action.yml mkl 2022.2.1 change: conda-forge and defaults have completely different dependencies. Defaults required intel-openmp, but now on conda-forge, mkl 2023.1 or higher requires llvm-openmp >=14.0.6, but this is incompatible with the pytorch build <2.5 which requires llvm-openmp<14.0. We would need to upgrade Python to 3.12 first, upgrade Pytorch build, then upgrade this mkl. (The meta.yaml changes are the ones that narrow it to 2022.2.1 during `conda build faiss`.) So, this has just been changed to 2022.2.1.
- mkl now requires _openmp_mutex of type "llvm" instead of "gnu": prior non-cuVS builds all used gnu, because intel-openmp from anaconda defaults channel does not require llvm-openmp. Now we need to remove the gnu one which is automatically pulled in during miniconda setup, and only keep the llvm version of _openmp_mutex.
- liblief: The above changes tried to pull in liblief 0.15. This results in an error like `AttributeError: module 'lief._lief.ELF' has no attribute 'ELF_CLASS'`. When I checked passing PR builds on defaults, they use lief 0.12, so I pinned that one for Python 3.9 3.10 3.11. For Python 3.12, we need lief 0.14 or higher.
- gcc_linux-64 =11.2 for faiss-gpu on cudatoolkit-11.2: GPU builds kept trying to reference 11.2 when 14.2 was installed. I couldn't figure out why, or how to point it to the 14.2 installed on the host. Current nightly builds still reference 11.2, so I gave up and pinned 11.2 to keep it the same. Moving to 14.2 will take some more investigation.
- meta.yaml mkl 2023.0 vs 2023.1 with python versions: 3.9, 3.10, and 3.11 pass with 2023.0, but python 3.12 needs mkl 2023.1 or higher. Otherwise we get:
```
INTEL MKL ERROR: $PREFIX/lib/python3.12/site-packages/faiss/../../.././libmkl_def.so.2: undefined symbol: mkl_sparse_optimize_bsr_trsm_i8.
Intel MKL FATAL ERROR: Cannot load libmkl_def.so.2.
```
so the solution was to put a bunch of conditions in in faiss/meta.yaml.
We should be able to use Jinja macros to reduce duplication but it requires some investigation. It was failing: https://github.com/facebookresearch/faiss/actions/runs/12915187334/job/36016477707?pr=4126 (paste of logs here: P1716887936). This can be a future BE task.
Macro example (the `-` signs remove whitespace lines before and after)
```
{% macro inclmkldevel() %}
{%- if PY_VER == '3.9' or PY_VER == '3.10' or PY_VER == '3.11' -%}
- mkl-devel =2023.0 # [x86_64]
- liblief =0.12.3 # [not win]
- python_abi <3.12
{%- elif PY_VER == '3.12' %}
- mkl-devel >=2023.2.0 # [x86_64]
- liblief =0.15.1 # [not win]
- python_abi =3.12
{% endif -%}
{% endmacro %}
```
The python_abi was required to be pinned inside these conditions because otherwise several builds got this error:
```
File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/utils.py", line 1919, in insert_variant_versions
matches = [regex.match(pkg) for pkg in reqs]
^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'list'
```
Unit test notes:
-
- test_gpu_basics.py: GPU residual quantizer: Debugged extensively with Matthijs. The problem is in the C++ -> Python conversion. The C++ side prints the right values, but when getting it back to Python, it is filled with junk data. It is only reproducible on CUDA 11.4.4 after switching channels. It is likely a compiler problem. We discussed, and resolved to create a C++ side unit test (so this diff creates TestGpuResidualQuantizer) to verify the functionality and disable the Python unit test, but leave it in the codebase with a comment. Matthijs made extensive notes in https://docs.google.com/document/d/1MjMdOpPgx-MArdrYJZCaQlRqlrhSj5Y1Z9lTyiab8jc/edit?usp=sharing .
- test_contrib.py: this now hangs forever and times out the runner for Windows on Python 3.12. I have it skipping now.
- test_mem_leak.cpp seems flaky. It sometimes fails, then passes with rerun.
Unfixed issues:
-
- I noticed sometimes downloads will fail with the text like below. It passes on re-run.
```
libgomp-14.2.0-h77fa898_1.conda extraction failed
Warning: error libmamba Error when extracting package: Could not chdir info/recipe/parent/patches/0005-Hardcode-HAVE_ALIGNED_ALLOC-1-in-libstdc-v3-configur.patch
error libmamba Error when extracting package: Could not chdir info/recipe/parent/patches/0005-Hardcode-HAVE_ALIGNED_ALLOC-1-in-libstdc-v3-configur.patch
Warning: Found incorrect download: libgomp. Aborting
Found incorrect download: libgomp. Aborting
Warning:
```
Green build and tests for both build pull request and nightlies: https://github.com/facebookresearch/faiss/actions/runs/12956402963/job/36148818361
Reviewed By: asadoughi
Differential Revision: D68043874
fbshipit-source-id: b105a1e3e6272763ad9daab7fc6f05a79f01c9e2
Summary:
Remove the dependency on `raft::compiled` and modify GPU implementations to use cuVS backend in place of RAFT.
A deeper insight into the dependency:
FAISS gets the ANN algorithm implementations such as IVF-Flat and IVF-PQ from cuVS. RAFT is meant to be a lightweight C++ header-only template library that cuVS relies on for the more fundamental / low-level utilities. Some examples of these are RAFT's device mdarray and mdspan objects; the RAFT resource object (`raft::resource`) that takes care of the stream ordering of device functions; linear algebra functions such as mapping, reduction, BLAS routines etc. A lot of the cuVS functions take the RAFT mdspan objects as arguments (for example `raft::device_matrix_view`). Therefore FAISS relies on both cuVS and RAFT. FAISS gets RAFT headers through cuVS and uses them to create the function arguments that can be consumed by cuVS. Note that we are not explicitly linking FAISS against `raft::raft` or `raft::compiled`. Only the required headers are included and compiled rather than compiling the whole RAFT shared library. This is the reason we still see mentions of `raft` in FAISS.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3549
Reviewed By: ramilbakhshyiev
Differential Revision: D62041013
Pulled By: asadoughi
fbshipit-source-id: 7230dcc06cf47baf95873adc1dec2adca4a8f82a
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3922
We need to be able to build external modules into FAISS, but don't have an example yet. This diff shows what CMakeLists.txt changes need to happen to incorporate an external module.
Reference: https://github.com/facebookresearch/faiss/pull/3699/
Reviewed By: mdouze
Differential Revision: D63991471
fbshipit-source-id: 0c1cd25eabbffb01d2a7170d6725a0c4a13c5bf0
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3466
Flattening Conda include directories breaks CUDA 11.4.4 build on Ubuntu 20 / v5 kernel. This change updates the logic to only flatten includes on Ubuntu 22 / v6 kernel (aka as running on GitHub Actions runners).
Reviewed By: algoriddle
Differential Revision: D57602154
fbshipit-source-id: 00c14ca7c64644b8b86483ac6b4d40c6d8f12372
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3459
When building with CUDA 11.4.4, CMake does not properly include files under Conda environment. This workaround flattens the include sub-directories in to the include root. It will unblock us for now while we are looking for a fix through CMakeLists files or figure out if it's a CMake bug and it gets fixed.
Reviewed By: junjieqi
Differential Revision: D57545169
fbshipit-source-id: 9cbdd0866e00e899cc889930a59448da55d873c2
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3440
This change is required to unblock the migration to GitHub Actions. `cuda-toolkit` was only specified in the `libfaiss` package and it was not available in `faiss-gpu` or `faiss-gpu-raft`. This currently works on CircleCI because the runner image has CUDA toolkit of the needed version installed on the system and the build logic falls back to that but breaks on GitHub Actions because their runner images do not come with CUDA toolkit pre-installed.
Reviewed By: junjieqi
Differential Revision: D57371597
fbshipit-source-id: 8bededd53e2528f033fac797b296d74b47f9403e
Summary:
The CMakeLists.txt in faiss/gpu uses the $<LINK_LIBRARY:WHOLE_ARCHIVE expression which requires at least cmake 3.24.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3305
Reviewed By: mlomeli1
Differential Revision: D56234500
Pulled By: algoriddle
fbshipit-source-id: dfe7df3379c5250dedec7d1988cffa889fc1c393
Summary:
Considering that only GPU classic works on Pascal and only if compiled with cuda 11.4, this sets up the conda builds as follows.
tl;dr: Pascal will only be supported by faiss-gpu on cuda 11.
## faiss-gpu
1. build on cuda 11.4.4, supports Pascal, works with pytorch-cuda=11 (including hosts with cuda 12 drivers)
2. build on cuda 12.1.1, does NOT support Pascal, works with pytorch-cuda=12
## faiss-gpu-raft
1. build on cuda 11.8.0, does NOT support Pascal, works with pytorch-cuda=11.8
2. build on cuda 12.1.1, does NOT support Pascal, works with pytorch-cuda=12
The reason faiss-gpu-raft is built with and supports only 11.8 is due to the risk of clobbering otherwise between libraft and pytorch-cuda dependencies.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3237
Reviewed By: mdouze
Differential Revision: D53354121
Pulled By: algoriddle
fbshipit-source-id: 86108e975168759572f84922bf0ccef55ae75ef8
Summary:
1. Split the GPU code into a separate static library to reduce build times.
2. faiss-gpu now depends on pytorch-cuda to stop clobbering of CUDA libraries when Faiss and Pytorch installed in the same environment
3. Add CUDA 12 (we support both 11.8 and 12.1)
4. Add Python 3.12
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3222
Reviewed By: mlomeli1
Differential Revision: D53215033
Pulled By: algoriddle
fbshipit-source-id: d48c5707af20b7e5ff72c3aa27b0e677259f22f8
Summary:
This fixes the build, except for MacOS, where there's a problem with cmake + OpenMP. We can fix it separately.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2856
Reviewed By: mlomeli1
Differential Revision: D45704458
Pulled By: algoriddle
fbshipit-source-id: 0c09036ae5fa34ab114b857f407a35603986613a
Summary:
RAFT requires cmake 3.23.1, pulling it from conda-forge. We continue to keep the dependency on conda-forge minimal, hence the ordering of the channels and the pinning of sysroot to a specific version.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2808
Test Plan:
contbuild
Imported from GitHub, without a `Test Plan:` line.
Reviewed By: mlomeli1
Differential Revision: D44746827
Pulled By: algoriddle
fbshipit-source-id: ad576b11b257203bd0cafd57c2c2e7fd8d10ca98
Summary:
1. GPU builds use CircleCI base image, no docker
2. Switched to CUDA 11.4 (used to be 11.3)
3. Merged all build jobs into two parameterized targets: `build_cmake` and `build_conda`.
4. Cleaned up test execution, fixed bug of Python GPU tests not running on PRs
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2803
Reviewed By: mlomeli1
Differential Revision: D44541714
Pulled By: algoriddle
fbshipit-source-id: aa09ae638ecb6ef556d42f27a4bfaddad7355e50
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2791
Removed building for 3.7 (D44373285 - not supported for M1), adding 3.10 with this diff.
The only change that matters is in `conda_build_config.yaml`, the others are about making the configs consistent between CPU and GPU.
Reviewed By: mlomeli1
Differential Revision: D44405573
fbshipit-source-id: ad933e08834593e55a35075c602e5f509a813e73
Fadi Arafeh