gemm_f32_parity: compute capability 6.1
gdn qkv    10240 x 2560 : 1278 sizes from T 256 to 8192, 0 with different bits (bitwise equal)
BENCH gdn qkv    T   560 | cuBLAS FP16    6.48 ms | FP32 copies    4.44 ms (the conversions included) | 1.46x
BENCH gdn qkv    T  2048 | cuBLAS FP16   15.43 ms | FP32 copies   14.21 ms (the conversions included) | 1.09x
BENCH gdn qkv    T  4096 | cuBLAS FP16   49.01 ms | FP32 copies   25.54 ms (the conversions included) | 1.92x
BENCH gdn qkv    T  7040 | cuBLAS FP16   55.75 ms | FP32 copies   46.52 ms (the conversions included) | 1.20x
BENCH gdn qkv    T  7041 | cuBLAS FP16   85.38 ms | FP32 copies   44.74 ms (the conversions included) | 1.91x
qsa query  12288 x 2560 : 1278 sizes from T 256 to 8192, 0 with different bits (bitwise equal)
BENCH qsa query  T   560 | cuBLAS FP16    8.26 ms | FP32 copies    6.27 ms (the conversions included) | 1.32x
BENCH qsa query  T  2048 | cuBLAS FP16   29.38 ms | FP32 copies   19.08 ms (the conversions included) | 1.54x
BENCH qsa query  T  4096 | cuBLAS FP16   48.58 ms | FP32 copies   38.18 ms (the conversions included) | 1.27x
BENCH qsa query  T  7040 | cuBLAS FP16   79.49 ms | FP32 copies   61.34 ms (the conversions included) | 1.30x
BENCH qsa query  T  7041 | cuBLAS FP16  102.37 ms | FP32 copies   64.72 ms (the conversions included) | 1.58x
gdn gate    6144 x 2560 : 1235 sizes from T 512 to 8192, 0 with different bits (bitwise equal)
BENCH gdn gate   T   560 | cuBLAS FP16    2.90 ms | FP32 copies    3.05 ms (the conversions included) | 0.95x
BENCH gdn gate   T  2048 | cuBLAS FP16   12.10 ms | FP32 copies    8.96 ms (the conversions included) | 1.35x
BENCH gdn gate   T  4096 | cuBLAS FP16   26.77 ms | FP32 copies   17.70 ms (the conversions included) | 1.51x
BENCH gdn gate   T  7040 | cuBLAS FP16   39.19 ms | FP32 copies   33.55 ms (the conversions included) | 1.17x
BENCH gdn gate   T  7041 | cuBLAS FP16   47.23 ms | FP32 copies   33.52 ms (the conversions included) | 1.41x
gdn out     2560 x 6144 : 705 sizes from T 4096 to 8192, 0 with different bits (bitwise equal)
BENCH gdn out    T  4096 | cuBLAS FP16   24.59 ms | FP32 copies   18.34 ms (the conversions included) | 1.34x
BENCH gdn out    T  7040 | cuBLAS FP16   35.33 ms | FP32 copies   30.60 ms (the conversions included) | 1.15x
BENCH gdn out    T  7041 | cuBLAS FP16   48.76 ms | FP32 copies   33.42 ms (the conversions included) | 1.46x
qsa out     2560 x 4096 : 705 sizes from T 4096 to 8192, 0 with different bits (bitwise equal)
BENCH qsa out    T  4096 | cuBLAS FP16   12.29 ms | FP32 copies   13.29 ms (the conversions included) | 0.93x
BENCH qsa out    T  7040 | cuBLAS FP16   26.60 ms | FP32 copies   22.31 ms (the conversions included) | 1.19x
BENCH qsa out    T  7041 | cuBLAS FP16   22.26 ms | FP32 copies   22.36 ms (the conversions included) | 1.00x
gemm_f32_parity: 5201 sizes, OK (bitwise)
