mirror of
https://github.com/facebookresearch/faiss.git
synced 2026-10-12 02:13:36 +00:00
Stop copying QueryFactorsData per block in RaBitQHeapHandler
Summary: `RaBitQHeapHandler::handle()` in `IndexRaBitQFastScan.h` copied `rabitq_utils::QueryFactorsData` by value once per (query, 32-vector block). The struct owns `std::vector<float> rotated_q`, so each copy is a heap malloc + free inside the scan's innermost loop: about 312M allocations per 10,000-query pass on sift-1M. It now binds a `const&` to `context->query_factors[q]`, or to a function-local `static const` empty instance when there is no context. `compute_1bit_adjusted_distance` already takes `const&`, so nothing downstream changes. **Why this showed up as an ARM "page-fault storm"** The copy is wasteful on every platform. It becomes catastrophic with jemalloc heap profiling on aarch64, when the allocation happens on an OpenMP worker thread: - Heap profiling is on by default in task 0 of every Tupperware job: the TW scheduler appends `prof:true,prof_final:false,prof_prefix:/tmp/jeprof` when `taskID % contHeapEnabledRatio == 0` (`tupperware/scheduler/JobSpecUtil.cpp`). jemalloc then backtraces one allocation per 512 KiB on average (`lg_prof_sample` 19), through libunwind on aarch64 (`config.prof_libunwind=1`). - On OpenMP workers the stack runs through `__kmp_invoke_microtask`, libomp's assembly entry, which has no usable CFI. DWARF unwinding fails there, and libunwind's aarch64 fallback (`get_frame_state`) calls `get_proc_name`, which maps the frame's whole ELF image and unmaps it on every call. Every map and unmap serialises on `mmap_lock`. - gdb on a reproducer without FAISS: all 95 `get_proc_name` fallbacks hit `__kmp_invoke_microtask`; the same allocations on plain pthreads hit none. - The fault count tracks the sampling rate exactly: 3,053,990 / 1,525,976 / 764,008 faults per pass at `lg_prof_sample` 18 / 19 / 20. That is about 305k samples per pass (160 GB of 512 B copies ÷ 512 KiB) at roughly 5 faults each. **Numbers**, hr_rabitqfs, sift-1M, recall 0.3567 throughout. ServiceLab Grace (72 threads, heap profiling on as in production), nq=10000: | | QPS | minor faults / pass | |---|---|---| | before | 91 | 1.53M | | after (E2362759441136789) | 17,186 | 858 | devbig334 Neoverse-V2, environment matched to Tupperware (no `KMP_*` variables), nq=10000, single runs on a busy shared host, so treat as indicative: | | QPS | kcycles/query | minor faults / pass | |---|---|---|---| | before, profiling off | 9,342 | 13,223 | 322 | | after, profiling off | 11,083 | 11,831 | 341 | | before, profiling on | 93 | 33,563 | 1,525,976 | | after, profiling on | 10,970 | 11,756 | 840 | Without profiling the copy alone costs about 10% more cycles per query. **Not fixed here: `IndexIVFRaBitQFastScan`.** Its scan already reads query factors by reference, but a ServiceLab sweep with this fix and profiling on still shows up to 30k (1-bit) and 57k (multi-bit) faults per pass with up to 58 threads blocked. The allocation site is being located and will be fixed in a separate diff. Reviewed By: mnorris11 Differential Revision: D123990523 fbshipit-source-id: 0112bdbe78ac0fd647ea6199b00000193f96788e
This commit is contained in:
committed by
meta-codesync[bot]
parent
3a98062426
commit
6692d7ab37
@@ -195,10 +195,12 @@ struct RaBitQHeapHandler
|
||||
float* const heap_dis = heap_distances + q * k;
|
||||
int64_t* const heap_ids = heap_labels + q * k;
|
||||
|
||||
rabitq_utils::QueryFactorsData query_factors_data = {};
|
||||
if (context && context->query_factors != nullptr) {
|
||||
query_factors_data = context->query_factors[q];
|
||||
}
|
||||
// Not a copy: QueryFactorsData owns a std::vector (malloc per block).
|
||||
static const rabitq_utils::QueryFactorsData kNoQueryFactors{};
|
||||
const rabitq_utils::QueryFactorsData& query_factors_data =
|
||||
(context && context->query_factors != nullptr)
|
||||
? context->query_factors[q]
|
||||
: kNoQueryFactors;
|
||||
|
||||
const float one_a = normalizers ? (1.0f / normalizers[2 * q]) : 1.0f;
|
||||
const float bias = normalizers ? normalizers[2 * q + 1] : 0.0f;
|
||||
|
||||
Reference in New Issue
Block a user