Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/5663
Follow-up to D121097648 (public PR: https://github.com/facebookresearch/faiss/pull/5657).
Related public PR: https://github.com/facebookresearch/faiss/pull/4910
Use relaxed atomic additions for HNSW, RaBitQ, and IVFPQ stats aggregation. Builds with `std::atomic_ref` support use `fetch_add(..., std::memory_order_relaxed)`; older standard libraries fall back to `__atomic_fetch_add(..., __ATOMIC_RELAXED)`. This removes the global OpenMP critical sections that serialize concurrent searches while preserving exact counter updates.
Benchmark design: mode/opt `bench_hnsw_stats` binary, one shared HNSW index, `N` external pthreads, one OpenMP thread per search (each worker calls `omp_set_num_threads(1)` once before its timed loop), five repetitions per point.
Performance:
| External pthreads | Critical M/s | Relaxed atomic M/s | Speedup |
|---:|---:|---:|---:|
| 1 | 2.228 | 2.503 | 1.12x |
| 2 | 3.716 | 3.853 | 1.04x |
| 4 | 5.662 | 7.357 | 1.30x |
| 8 | 6.384 | 12.843 | 2.01x |
| 16 | 2.697 | 15.804 | 5.86x |
| 32 | 0.951 | 18.910 | 19.88x |
| 64 | 0.117 | 21.866 | 186.89x |
At 64 pthreads, the feature-detected `atomic_ref` path measured 21.918 M/s mean over 10 repetitions (21.907 M/s median, 2.59% CV). The `__atomic_fetch_add` fallback measured 22.563 M/s mean (22.511 M/s median, 3.24% CV).
The 64-thread profile attributes 97.38% of cycles in the critical-section version to lock acquisition and waiting. With relaxed atomic aggregation, 74.14% is in the stats combine path instead. The remaining non-linear scaling, 21.866 M/s at 64 threads versus the 2.503 M/s single-thread baseline, comes from cache-line contention on the global counters and shared-index memory-bandwidth and cache pressure.
Reviewed By: mnorris11
Differential Revision: D121368933
fbshipit-source-id: 8dd26313e55aa73c0cdd379c7648efacd4f5f1d8
Lucian Grijincu
2496f63f76