Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5562
Lionhead harness `faiss_rmdt_post_read_reconstruct_fuzzer` found a
heap out-of-bounds read at `Panorama.cpp:171`, reported as
`READ of size 5122903332` in `__asan_memcpy`.
Decoding the 232-byte crash input, it is `IxPT -> IxPT -> IxPT -> IxFp`
where every `IndexPreTransform` carries an **empty** transform chain and
the innermost `IndexFlatIPPanorama` has `d = 0x4C565349 = 1280725833`
(so `code_size = d * 4 = 5122903332`, exactly the ASAN read size),
`ntotal = 0`, and an empty `codes` vector. Three independent defects
combine:
1. `IndexFlatPanorama::reconstruct` never bounds-checks `key`. It
overrides `IndexFlat::reconstruct`, which does have
`FAISS_THROW_IF_NOT(key < ntotal)`, and drops the guard. Overriding
`reconstruct_n` likewise removes `IndexFlatCodes::reconstruct_n`'s
`i0 + ni <= ntotal` check. With `ntotal == 0` and empty `codes`,
`reconstruct(0, ...)` memcpy's `code_size` bytes out of a zero-length
buffer. `Panorama::reconstruct` takes an unsized `const uint8_t*`, so it
cannot self-validate -- the caller has to.
2. The `IxFP`/`IxFp` read branch does not range-check `d`. It is the one
index type that does not go through `read_index_header`, so it misses
that function's `FAISS_CHECK_RANGE(d, 0, (1 << 20) + 1)` and
`ntotal >= 0` checks. The `codes.size()` equality check below it does
not catch this, because `ntotal == 0` makes `num_slots == 0` and an
empty `codes` is then "valid".
3. `IndexPreTransform` deserialization skips all consistency checks when
the chain is empty -- the `chain[0].d_in == ixpt->d` /
`chain[nt-1].d_out == index->d` validation sits entirely inside
`if (nt > 0)`. With `nt == 0`, `IndexPreTransform::reconstruct` passes
the caller's buffer (sized from the outer `d`) straight to the
sub-index, so a mismatched sub-index `d` is also a write-side overflow,
for any sub-index type.
Each is fixed:
- `IndexFlatPanorama::reconstruct` / `reconstruct_n` now range-check
against `ntotal`.
- The `IxFP`/`IxFp` branch applies the same `d` and `ntotal` checks as
`read_index_header`.
- `read_index` requires `index->d == ixpt->d` when the `IxPT` chain is
empty.
Two adjacent gaps in the same audit are closed as well:
- `IHfP` (`IndexHNSWFlatPanorama`) never validated `cum_sums.size()`,
unlike the equivalent `IxFP` check. Both `get_cum_sum()` and
`HNSW.cpp`'s Panorama search path index it at stride
`pano.n_levels + 1`, so a short vector is an OOB read at search time.
It also had no explicit `n_levels > 0` guard.
- `ArrayInvertedListsPanorama::get_single_code` bounds-checked with
`assert`, which is compiled out in opt builds. It reaches the same
`Panorama::reconstruct` sink via
`IndexIVFFlatPanorama::reconstruct_from_offset`.
Reviewed By: mnorris11
Differential Revision: D118495504
fbshipit-source-id: 46bc7a1a6c9335a5589e20221b9478ae5505b9da
Trang Nguyen
1e5780fbbf