Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5349
# this is a no-op change
Ran `black -l 80` over all tracked Python files under `fbcode/faiss/`, using line length 80 to match faiss's documented coding style (`CONTRIBUTING.md`: "80 character line length (both for C++ and Python)"). 172 files were reformatted; 28 were already compliant.
This is a pure-formatting change. `black` runs its AST-equivalence safety check on every file, so no identifiers, string/numeric values, operators, or control flow were altered — only whitespace, indentation, line wrapping, blank lines, trailing commas, and string-quote normalization (single to double quotes).
`fbcode/faiss/` is intentionally excluded from Meta's standard Python autoformatter (`arc f` / `arc lint -a` report no formatter for these paths) because it mirrors to public GitHub, so the Python had drifted out of any consistent style. The C++ is already clang-format-clean (0 of 807 files change), so this effort is Python-only. This diff brings the Python to canonical `black` formatting.
Excluded `demos/index_pq_flat_separate_codes_from_codebook.py`: it contains a Jupyter `!rm` shell-magic line and is not valid Python, so `black` cannot parse it.
This is the base of a 3-diff stack: (1) this black reformat; (2) `.flake8` alignment (max-line-length 80 + black-compatible ignores E203/W503); (3) genuine lint fixes (F401/F403/F841/B011).
Reviewed By: mnorris11
Differential Revision: D109736099
fbshipit-source-id: 9a054635766ed11aa3760a788395babd3d198abf
Summary:
This PR adds a Python HNSW tutorial corresponding to the existing C++ HNSW tutorial.
The example creates synthetic vectors, builds an IndexHNSWFlat index, runs a search, and prints the nearest-neighbor IDs and distances for the final queries.
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5260
Reviewed By: mnorris11
Differential Revision: D107272616
Pulled By: alibeklfc
fbshipit-source-id: 721d7a1a95aa79150d6cb9653bd803065c82dcd7
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/issues/4450
This pull request introduces support for [Intel ScalableVectorSearch](https://github.com/intel/ScalableVectorSearch), integrating Intel's proprietary LVQ and LeanVec technologies in binary form. The following index types are now supported: `IndexSVSVamana`, `IndexSVSVamanaLVQ`, `IndexSVSVamanaLeanVec`, and `IndexSVSFlat`. `IndexSVSVamana` and `IndexSVSFlat` utilizes SVS open-source float32/float16/int8 implementation.
Key features and enhancements include:
- Implemented `search`, `add`, and `remove_ids`
- Implemented filter search and `range_search`
- Enabled SVS open-source fp16 and scalar quantization implementation
- Enabled factory methods
- Enabled save/load
- Enabled Python bindings
- Enable fallback mechanism that fallbacks to 8-bit scalar quantization if LVQ/LeanVec is used on non-intel hardware CPUs
- Added examples in both Python and C++ under the `tutorial/` directory
- Added `FAISS_ENABLE_SVS` flag to allows users to optionally enable SVS
TODOs:
- Support more indices such as `IndexSVSIVF` and `IndexSVSFlat` with LVQ and LeanVec
- Documentation
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/4548
Reviewed By: subhadeepkaran
Differential Revision: D84713772
Pulled By: mnorris11
fbshipit-source-id: 290365eb3337f685dd46696935b621c57dad3174
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3494
This tasks focus on the refine index construction tutorial with different index refinement on fp16/sq8 quantization. The python version was added a while ago.
Reviewed By: junjieqi
Differential Revision: D58161983
fbshipit-source-id: 1c598fe612b5dee3952c5f7398e6802e117f141d
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3480
This tutorial summarize the methods to construct different indexs for PQFastScan refinement. It shows how the choice can impact on accuracy.
Reviewed By: junjieqi
Differential Revision: D57799598
fbshipit-source-id: a75c52c60a5217366f3361676da8f03f0c4a9feb
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3474
This commit focus on the cpp version of PQfastscan tutorial with index refinement by defining the k factor.
Reviewed By: junjieqi
Differential Revision: D57680905
fbshipit-source-id: 980c2990172f24ec9a4f870685e706195883408f
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3468
This commit includes the tutorial for PQFastScan in the cpp environment.
Reviewed By: junjieqi
Differential Revision: D57631441
fbshipit-source-id: f5e17eee2a584ebfc9ff63868d741d0da6b3b413
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3465
This commit include python version of tutorial for FastScan. It includes all the parameters enabled within PQFastScan.
Reviewed By: junjieqi
Differential Revision: D57594044
fbshipit-source-id: cb12679b6fc241a654b9545c5bc7bd0517aa1813
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3439
LLVM-15 has a warning `-Wunused-but-set-variable` which we treat as an error because it's so often diagnostic of a code issue. Unused variables can compromise readability or, worse, performance.
This diff either (a) removes an unused variable and, possibly, it's associated code, or (b) qualifies the variable with `[[maybe_unused]]`, mostly in cases where the variable _is_ used, but, eg, in an `assert` statement that isn't present in production code.
- If you approve of this diff, please use the "Accept & Ship" button :-)
Reviewed By: palmje, junjieqi
Differential Revision: D57344013
fbshipit-source-id: adf410139d2e6ca69a26ccdbff8511c9b7620489
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2582
A few more or less cosmetic improvements
* Index::idx_t was in the Index object, which does not make much sense, this diff moves it to faiss::idx_t
* replace multiprocessing.dummy with multiprocessing.pool
* add Alexandr as a core contributor of Faiss in the README ;-)
```
for i in $( find . -name \*.cu -o -name \*.cuh -o -name \*.h -o -name \*.cpp ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
For the fbcode deps:
```
for i in $( fbgs Index::idx_t --exclude fbcode/faiss -l ) ; do
sed -i s/Index::idx_t/idx_t/ $i
done
```
Reviewed By: algoriddle
Differential Revision: D41437507
fbshipit-source-id: 8300f2a3ae97cace6172f3f14a9be3a83999fb89
Changelog:
- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
* Refactors Makefiles and add configure script.
* Give MKL higher priority in configure script.
* Clean up Linux example makefile.inc.
* Cleanup makefile.inc examples.
* Fix python clean Makefile target.
* Regen swig wrappers.
* Remove useless CUDAFLAGS variable.
* Fix python linking flags.
* Separate compile and link phase in python makefile.
* Add macro to look for swig.
* Add CUDA check in configure script.
* Cleanup make depend targets.
* Cleanup CUDA flags.
* Fix linking flags.
* Fix python GPU linking.
* Remove useless flags from python gpu module linking.
* Add check for cuda libs.
* Cleanup GPU targets.
* Clean up test target.
* Add cpu/gpu targets to python makefile.
* Clean up tutorial Makefile.
* Remove stale OS var from example makefiles.
* Clean up cuda example flags.
* Remove transitional print import.
* Add example for multiple GPUs in python.
* Add example on GPU in C++.
* Add example on multiple GPUs in C++.
* Add IVFFlat example on GPU in python.
* Add Makefile for C++ tutorial examples.
* Add IVF index on GPU example in C++.
* moved most FAISS_ASSERT calls to C++ exceptions, and adjusted
memory allocation to avoid mem leaks
* added an IndexIVFScalarQuantizer type that offers an
intermediate compression between IVFFlat and IVFPQ
* support removal of indices in IndexIDMap / IndexFlat combination
* various fixes in GPU code
* add initial cmake support
* update cmake, add cmake instructions to INSTALL
* update findopenmp and INSTALL
* change FindOpenBLAS.cmake to cater for macports
- change cblas.h to openblas_config.h since macports does not ship
cblas.h with openblas.
* revise INSTALL for cmake
Tia Lim