mirror of
https://github.com/facebookresearch/faiss.git
synced 2026-10-11 22:50:00 +00:00
Alibek Zhakubayev
a238933d17
Summary: Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5444 We fix every remaining flake8 E501 (line > 80 chars) violation in fbcode/faiss/ (excluding gpu/), closing out a lint-cleanup initiative that a June 29, 2026 diff stack (9a05463576 -> d304e29b9a -> 7d140d86ca) explicitly promised but left 41% incomplete. The middle diff in that stack aligned .flake8's max-line-length to black's 80-char width and stated the remaining 277 violations after the config change were "genuine lint (E501 long strings/URLs, F401/F841/B011, etc.), addressed in later commits in this stack." The follow-up commit fixed only the F401/F403/F841/B011 categories (45 issues) and never touched E501, leaving 192 violations across 53 files live in trunk. A direct re-scan today found 201 violations across 54 files (two files gained lines since the original count). We fixed 183 violations across 52 files. Two of the originally-flagged files, benchs/bench_rabitq.py and demos/index_pq_flat_separate_codes_from_codebook.py, already carry a file-level `# flake8: noqa` directive (the second is a Bento/Grimaldi notebook export that also fails standalone py_compile due to embedded shell magic) -- flake8 ignores both files entirely, so no changes were needed there and they are excluded from this diff. Fix approach, by category: - The large majority of violations are prose: docstrings and comments rewrapped at word boundaries, preserving each file's existing indentation and docstring conventions. No wording was changed. - Code lines (function calls, f-strings, assert messages, long argument lists) were reflowed in black-compatible style: parenthesization, comma breaks, or implicit adjacent string-literal concatenation. Every touched file is confirmed unchanged by `black --line-length 80 --diff` after the edit. - A small number of genuinely unwrappable long URLs (no reasonable break point) were suppressed with `# noqa: E501` rather than force-wrapped, matching the precedent set by the original black-format diff stack. - demos/offline_ivf/offline_ivf.py: a repeated, non-trivial ratio expression used in both an assert condition and its (lazily-evaluated) failure message was hoisted into a single local variable, eliminating the duplicate computation. The expression is pure and deterministic (built from already-computed arrays), so this is behaviorally identical to the original -- not a functional change. - `arc lint -a` additionally auto-fixed a real bug it detected while processing these files: `raise NotImplemented` -> `raise NotImplementedError` in contrib/evaluation.py (the sentinel object NotImplemented is not an exception type; raising it produces a TypeError masking the intended NotImplementedError). It also added a missing trailing newline in benchs/fb/bench_index_binary_from_float.py and, in the same file, rewrote the `#!/usr/bin/env python2` shebang to `#!/usr/bin/env fbpython` -- that rewrite was reverted by hand, since the file still does `import cPickle` (a Python 2-only module), so the auto-fix would have made the shebang describe an interpreter the script cannot actually run under. Fixing the underlying py2-to-py3 migration is out of scope for this line-length-only sweep. - Rebasing onto master (this diff's base commit had drifted) surfaced two concurrent upstream fixes to `check_ref_knn_with_draws(...)` calls in tests/test_ivf_flat_panorama.py and tests/test_refine_panorama.py (an unrelated in-flight correctness fix passing `D_panorama`/`atol` instead of a stale `D_regular` arg with no `atol`). The rebase took upstream's corrected arguments and re-applied only this diff's line-wrap on top. We verified zero functional change with an AST-level check: every string and f-string literal in every touched file was extracted before and after the edit and compared with whitespace normalized, confirming no wording, value, or expression was altered beyond the intended `# noqa: E501` suffixes and the one documented ratio-hoisting simplification above. Reviewed By: mnorris11 Differential Revision: D112525458 fbshipit-source-id: e4b85d73dafcaac779f3522a383a66e490fa942a
Demos for a few Faiss functionalities
demo_auto_tune.py
Demonstrates the auto-tuning functionality of Faiss
demo_ondisk_ivf.py
Shows how to construct a Faiss index that stores the inverted file data on disk, eg. when it does not fit in RAM. The script works on a small dataset (sift1M) for demonstration and proceeds in stages:
0: train on the dataset
1-4: build 4 indexes, each containing 1/4 of the dataset. This can be done in parallel on several machines
5: merge the 4 indexes into one that is written directly to disk (needs not to fit in RAM)
6: load and test the index