Stop copying QueryFactorsData per block in RaBitQHeapHandler

Summary:
`RaBitQHeapHandler::handle()` in `IndexRaBitQFastScan.h` copied `rabitq_utils::QueryFactorsData` by value once per (query, 32-vector block). The struct owns `std::vector<float> rotated_q`, so each copy is a heap malloc + free inside the scan's innermost loop: about 312M allocations per 10,000-query pass on sift-1M.

It now binds a `const&` to `context->query_factors[q]`, or to a function-local `static const` empty instance when there is no context. `compute_1bit_adjusted_distance` already takes `const&`, so nothing downstream changes.

**Why this showed up as an ARM "page-fault storm"**
The copy is wasteful on every platform. It becomes catastrophic with jemalloc heap profiling on aarch64, when the allocation happens on an OpenMP worker thread:
- Heap profiling is on by default in task 0 of every Tupperware job: the TW scheduler appends `prof:true,prof_final:false,prof_prefix:/tmp/jeprof` when `taskID % contHeapEnabledRatio == 0` (`tupperware/scheduler/JobSpecUtil.cpp`). jemalloc then backtraces one allocation per 512 KiB on average (`lg_prof_sample` 19), through libunwind on aarch64 (`config.prof_libunwind=1`).
- On OpenMP workers the stack runs through `__kmp_invoke_microtask`, libomp's assembly entry, which has no usable CFI. DWARF unwinding fails there, and libunwind's aarch64 fallback (`get_frame_state`) calls `get_proc_name`, which maps the frame's whole ELF image and unmaps it on every call. Every map and unmap serialises on `mmap_lock`.
- gdb on a reproducer without FAISS: all 95 `get_proc_name` fallbacks hit `__kmp_invoke_microtask`; the same allocations on plain pthreads hit none.
- The fault count tracks the sampling rate exactly: 3,053,990 / 1,525,976 / 764,008 faults per pass at `lg_prof_sample` 18 / 19 / 20. That is about 305k samples per pass (160 GB of 512 B copies ÷ 512 KiB) at roughly 5 faults each.

**Numbers**, hr_rabitqfs, sift-1M, recall 0.3567 throughout.

ServiceLab Grace (72 threads, heap profiling on as in production), nq=10000:

| | QPS | minor faults / pass |
|---|---|---|
| before | 91 | 1.53M |
| after (E2362759441136789) | 17,186 | 858 |

devbig334 Neoverse-V2, environment matched to Tupperware (no `KMP_*` variables), nq=10000, single runs on a busy shared host, so treat as indicative:

| | QPS | kcycles/query | minor faults / pass |
|---|---|---|---|
| before, profiling off | 9,342 | 13,223 | 322 |
| after, profiling off | 11,083 | 11,831 | 341 |
| before, profiling on | 93 | 33,563 | 1,525,976 |
| after, profiling on | 10,970 | 11,756 | 840 |

Without profiling the copy alone costs about 10% more cycles per query.

**Not fixed here: `IndexIVFRaBitQFastScan`.** Its scan already reads query factors by reference, but a ServiceLab sweep with this fix and profiling on still shows up to 30k (1-bit) and 57k (multi-bit) faults per pass with up to 58 threads blocked. The allocation site is being located and will be fixed in a separate diff.

Reviewed By: mnorris11

Differential Revision: D123990523

fbshipit-source-id: 0112bdbe78ac0fd647ea6199b00000193f96788e
This commit is contained in:
Weixiang Hong
2026-10-08 13:19:13 -07:00
committed by meta-codesync[bot]
parent 3a98062426
commit 6692d7ab37
+6 -4
View File
@@ -195,10 +195,12 @@ struct RaBitQHeapHandler
float* const heap_dis = heap_distances + q * k;
int64_t* const heap_ids = heap_labels + q * k;
rabitq_utils::QueryFactorsData query_factors_data = {};
if (context && context->query_factors != nullptr) {
query_factors_data = context->query_factors[q];
}
// Not a copy: QueryFactorsData owns a std::vector (malloc per block).
static const rabitq_utils::QueryFactorsData kNoQueryFactors{};
const rabitq_utils::QueryFactorsData& query_factors_data =
(context && context->query_factors != nullptr)
? context->query_factors[q]
: kNoQueryFactors;
const float one_a = normalizers ? (1.0f / normalizers[2 * q]) : 1.0f;
const float bias = normalizers ? normalizers[2 * q + 1] : 0.0f;