* errors are with C++11 narrowing warnings (treated as errors in strict builds) when initializing at::cuda::CUDAGuard with a non-constant char cast to c10::DeviceIndex (signed char).
* errors are with C++11 narrowing warnings (treated as errors in strict builds) when initializing at::cuda::CUDAGuard with a non-constant char cast to c10::DeviceIndex (signed char).
* errors are with C++11 narrowing warnings (treated as errors in strict builds) when initializing at::cuda::CUDAGuard with a non-constant char cast to c10::DeviceIndex (signed char).
* errors are with C++11 narrowing warnings (treated as errors in strict builds) when initializing at::cuda::CUDAGuard with a non-constant char cast to c10::DeviceIndex (signed char).
* Update flash_api_stable.cpp
* upstream cutlass v4.2.1 csrc
When testing the deterministic option for the GQA case, we found it fell into deadlock issues. Initialization dk and dv_semaphore to zeros to fix this issue.
* use LPT order in varlen kernel
* add prefill decode benchmark script
* add sort in prepare
* add full implementation:
* add varlen kvhead swizzle
* add settings for swizzle ablation
* add correction term for sort when causal
* remove ablation options from frontend and clean up comments
* add comments in prepare kernel
* remove debug code and scripts
* put back defaults in tests
* remove excess Nones returned in python interface for varlen
* revert opinionated change to setup.py on cuda version 12.9
* force inline sort op and make east const
* more templating in varlen scheduler to cure some register spilling
* fix exploding build by splitting compilation and add qol macros for hdimdiff
* fix metadata mismatch with seqlenk in test script
* extend prepare kernel to >992 batches and always call it for varlen
* do inter-batch sort per every 992 batches
* better names in combine and fix prepare condition in api
* separate d & dv (interface)
* separate d & dv (api)
* separate d & dv (template)
* separate d & dv (mainloop)
* separate d & dv (epilogue)
* update test
* disable backward test when attention_chunk != 0
* extend backward d > dv to d != dv
---------
Co-authored-by: monk.ey <monk.ey@kakaocorp.com>
skarupke