mirror of
https://github.com/facebookresearch/faiss.git
synced 2026-10-12 02:13:36 +00:00
Harden OnDiskInvertedLists mmap reader against malformed list sizes (#5312)
Summary: ## What `OnDiskInvertedListsIOHook::read_ArrayInvertedLists()` builds each inverted list's `offset` by accumulating `size * (sizeof(idx_t) + code_size)` over the deserialized `sizes` vector. The per-list `size` values come straight from the serialized index and were not bounds-checked: the multiply could overflow `size_t`, and the running offset was never re-validated against the mapped file size inside the loop. The only existing guard (`FAISS_THROW_IF_NOT(o <= totsize)` just above the loop) runs once and validates only the starting offset. As a result, a malformed index loaded via `read_index(path, IO_FLAG_MMAP)` could leave one or more `lists[i].offset` pointing past the end of the mmap'd region. `get_codes()` / `get_ids()` only guard against `INVALID_OFFSET`, so at search time the IVF scan reads from an out-of-range pointer — an out-of-bounds read. ## Fix Inside the offset-accumulation loop, this change: - uses `mul_no_overflow` for `size * (sizeof(idx_t) + code_size)`, - uses `add_no_overflow` when advancing the running offset, and - re-validates `o <= totsize` on every iteration with a descriptive message. This mirrors the deserialization hardening already present for the Array and Block inverted-list readers in `faiss/impl/index_read.cpp` (which use `mul_no_overflow` + per-list byte-limit checks), bringing the OnDisk mmap reader to parity. Well-formed indexes are unaffected — the guards only reject inputs whose declared list sizes do not fit the mapped file. ## Verification - The changed translation unit passes `clang++ -std=c++17 -fsyntax-only` cleanly. The three helpers (`mul_no_overflow`, `add_no_overflow`, `FAISS_THROW_IF_NOT_FMT`) are already in scope via `faiss/impl/FaissAssert.h` and used throughout this file. - No behavioral change for valid indexes; the new checks only fire on malformed inputs. Reported via coordinated disclosure (huntr). Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5312 Reviewed By: mnorris11 Differential Revision: D108627881 Pulled By: alibeklfc fbshipit-source-id: c0925315f2e80f6a6a25800fb4bdee253376f981
This commit is contained in:
committed by
meta-codesync[bot]
parent
5437256fc5
commit
c428ca1e38
@@ -799,7 +799,19 @@ InvertedLists* OnDiskInvertedListsIOHook::read_ArrayInvertedLists(
|
||||
OnDiskInvertedLists::List& l = ails->lists[i];
|
||||
l.size = l.capacity = sizes[i];
|
||||
l.offset = o;
|
||||
o += l.size * (sizeof(idx_t) + ails->code_size);
|
||||
size_t elem_size = add_no_overflow(
|
||||
sizeof(idx_t), ails->code_size, "OnDisk inverted list element");
|
||||
size_t list_bytes =
|
||||
mul_no_overflow(l.size, elem_size, "OnDisk inverted list");
|
||||
o = add_no_overflow(o, list_bytes, "OnDisk inverted list offset");
|
||||
FAISS_THROW_IF_NOT_FMT(
|
||||
o <= ails->totsize,
|
||||
"inverted list %zu at offset %zu with %zu bytes exceeds "
|
||||
"mapped file size %zu",
|
||||
i,
|
||||
l.offset,
|
||||
list_bytes,
|
||||
ails->totsize);
|
||||
}
|
||||
// resume normal reading of file
|
||||
fseek(fdesc, o, SEEK_SET);
|
||||
|
||||
Reference in New Issue
Block a user