Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5127
Attempt to enable FAISS dynamic dispatch (DD) mode on Windows/MSVC.
Changes:
- CMakeLists.txt: Remove if(NOT WIN32) guard from DD section, add MSVC per-file
SIMD flags (/arch:AVX2, /arch:AVX512, /bigobj) alongside existing GCC/Clang flags
- build-pull-request.yml: Add windows-x86_64-DD-cmake job that runs immediately
(no dependency on linux-x86_64-cmake), builds with MSVC and FAISS_OPT_LEVEL=dd,
runs C++ tests and Python tests
This diff is expected to fail on Windows due to MSVC requiring explicit template
specialization declarations (C++ §17.8.3) which GCC/Clang don't enforce. The CI
failure will surface the exact errors to guide the fix.
UPDATE: in fact, MSVC does not seem to require this. After fixing build scripts and a few classical Windows error, the C++ and python tests pass.
Reviewed By: algoriddle
Differential Revision: D101649751
fbshipit-source-id: 765ef8483d02652ce58625cd22baab8870acf718
Summary:
Extend the POSIX mmap reader to Apple platforms and use MAP_FAILED for mmap error checks while keeping madvise best-effort.
Update the C++ and Python mmap tests to exercise Darwin, and stop linking faiss_test against the Python example extension so Python-enabled test builds can run on macOS.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5058
Reviewed By: junjieqi
Differential Revision: D100351635
Pulled By: alibeklfc
fbshipit-source-id: f384251a634c7d3154103dbc763293b52b093ee8
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4818
Update call sites that were wrapping read_index/read_index_binary in
unique_ptr to use the new read_index_up/read_index_binary_up functions
directly.
Reviewed By: junjieqi
Differential Revision: D93151783
fbshipit-source-id: a71f0060a5ecea28f7aaf53b9867efc53cb06f54
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4250
This is an attempt to re-land the diff stack D69972250 D70982449
It was reverted because the bottom of the stack did not pass the tests.
The original code comes from Alexandr Guzhva's https://github.com/facebookresearch/faiss/pull/4199
To the adsmarket steward: the diff was already accepted by your team (see D70982449), but reverted for an independent reason. So should be easy to accept now.
Reviewed By: mengdilin
Differential Revision: D71614511
fbshipit-source-id: 94139b4a4d457afe0d37ac95342537414aa81e7a
Summary:
This PR introduces a backport of a combination of https://github.com/zilliztech/knowhere/pull/996 and https://github.com/zilliztech/knowhere/pull/1032 that allow to have memory-mapped and zerocopy indces.
The root underlying idea is that we replace certain `std::vector<>` containers with a custom `faiss::MaybeOwnedVector<>` container, which may behave either as `std::vector<>`, or as a view of a certain pointer / descriptor. We don't replace all the instances of `std::vector<>`, but the largest ones.
This change affects `IndexFlatCodes`-based and `IndexHNSW` CPU indices.
(done) alter IVF lists as well.
(done) alter binary indices as well.
Memory-mapped index works like this:
```C++
std::unique_ptr<faiss::Index> index_mm(
faiss::read_index(filenamename.c_str(), faiss::IO_FLAG_MMAP_IFC));
```
In theory, it should be ready to be used from Python. All the descriptor management should be working.
Zero-copy index works like this:
```C++
#include <faiss/impl/zerocopy_io.h>
faiss::ZeroCopyIOReader reader(buffer.data(), buffer.size());
std::unique_ptr<faiss::Index> index_zc(faiss::read_index(&reader));
```
All the pointer management for `faiss::ZeroCopyIOReader` should be handled manually.
I'm not sure how to plug this into Python yet, maybe, some ref-counting is required.
(done) some refactoring
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4199
Reviewed By: mengdilin
Differential Revision: D69972250
Pulled By: mdouze
fbshipit-source-id: 98a3f94d6884814873d3534ee25f960892ef1076
Matthijs Douze