mirror of
https://github.com/BillyOutlast/flash-attention-prebuild-wheels-rocm.git
synced 2026-07-01 01:27:54 -04:00
Junya Morioka
800 KiB
800 KiB
flash-attention pre-build wheels
This repository provides wheels for the pre-built flash-attention.
Since building flash-attention takes a very long time and is resource-intensive, I also build and provide combinations of CUDA and PyTorch that are not officially distributed.
The building Github Actions Workflow can be found here.
The built packages are available on the release page.
This repository uses a self-hosted runner and AWS CodeBuild for building the wheels. If you find this project helpful, please consider sponsoring to help maintain the infrastructure!
Table of Contents
Install
- Select the versions for Python, CUDA, PyTorch, and flash_attn.
flash_attn-[flash_attn Version]+cu[CUDA Version]torch[PyTorch Version]-cp[Python Version]-cp[Python Version]-linux_x86_64.whl
# Example: Python 3.11, CUDA 12.4, PyTorch 2.5, and flash_attn 2.6.3
flash_attn-2.6.3+cu124torch2.5-cp312-cp312-linux_x86_64.whl
-
Find the corresponding version of a wheel from the below Package section and releases
-
Direct Install or Download and Local Install
# Direct Install
pip install https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.0.0/flash_attn-2.6.3+cu124torch2.5-cp312-cp312-linux_x86_64.whl
# Download and Local Install
wget https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.0.0/flash_attn-2.6.3+cu124torch2.5-cp312-cp312-linux_x86_64.whl
pip install ./flash_attn-2.6.3+cu124torch2.5-cp312-cp312-linux_x86_64.whl
Packages
🐧 Linux x86_64
Flash-Attention 2.8.3
Packages for Flash-Attention 2.8.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.9 | 2.5 | 12.4 | Download1 |
| 3.9 | 2.5 | 12.6 | Download1 |
| 3.9 | 2.6 | 12.4 | Download1 |
| 3.9 | 2.6 | 12.6 | Download1 |
| 3.9 | 2.7 | 12.4 | Download1 |
| 3.9 | 2.7 | 12.6 | Download1 |
| 3.9 | 2.8 | 12.4 | Download1 |
| 3.9 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.6 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.6 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.6 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.10 | 2.9 | 12.6 | Download1 |
| 3.10 | 2.9 | 12.8 | Download1 |
| 3.10 | 2.9 | 13.0 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.6 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.6 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.6 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.9 | 12.6 | Download1, Download2 |
| 3.11 | 2.9 | 12.8 | Download1, Download2 |
| 3.11 | 2.9 | 13.0 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.6 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.6 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.6 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.6 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1, Download2 |
| 3.12 | 2.9 | 12.8 | Download1, Download2 |
| 3.12 | 2.9 | 13.0 | Download1 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.6 | 12.6 | Download1 |
| 3.13 | 2.6 | 12.8 | Download1 |
| 3.13 | 2.6 | 12.9 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.7 | 12.8 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.8 | Download1 |
| 3.13 | 2.8 | 12.9 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1, Download2 |
| 3.13 | 2.9 | 12.8 | Download1, Download2 |
| 3.13 | 2.9 | 13.0 | Download1 |
Flash-Attention 2.8.2
Packages for Flash-Attention 2.8.2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.9 | Download1 |
Flash-Attention 2.8.1
Packages for Flash-Attention 2.8.1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.8.0
Packages for Flash-Attention 2.8.0
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1, Download2 |
| 3.10 | 2.4 | 12.8 | Download1, Download2 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1, Download2 |
| 3.10 | 2.5 | 12.8 | Download1, Download2 |
| 3.10 | 2.6 | 12.4 | Download1, Download2 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1, Download2 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1, Download2 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1, Download2 |
| 3.11 | 2.6 | 12.8 | Download1, Download2 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1, Download2 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1, Download2 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1, Download2 |
| 3.12 | 2.5 | 12.8 | Download1, Download2 |
| 3.12 | 2.6 | 12.4 | Download1, Download2 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.7.4.post1
Packages for Flash-Attention 2.7.4.post1
Flash-Attention 2.7.4
Packages for Flash-Attention 2.7.4
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1, Download2 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1, Download2 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1, Download2 |
| 3.10 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.10 | 2.8 | 12.4 | Download1, Download2 |
| 3.10 | 2.8 | 12.8 | Download1, Download2 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.10 | 2.9 | 12.8 | Download1 |
| 3.10 | 2.9 | 13.0 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1, Download2 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1, Download2 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1, Download2 |
| 3.11 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.11 | 2.8 | 12.4 | Download1, Download2 |
| 3.11 | 2.8 | 12.8 | Download1, Download2 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.9 | 12.8 | Download1 |
| 3.11 | 2.9 | 13.0 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1, Download2 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1, Download2 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1, Download2 |
| 3.12 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.12 | 2.8 | 12.4 | Download1, Download2 |
| 3.12 | 2.8 | 12.8 | Download1, Download2 |
| 3.12 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.9 | 12.8 | Download1 |
| 3.12 | 2.9 | 13.0 | Download1 |
| 3.13 | 2.9 | 12.8 | Download1 |
| 3.13 | 2.9 | 13.0 | Download1 |
Flash-Attention 2.7.3
Packages for Flash-Attention 2.7.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.7.2.post1
Packages for Flash-Attention 2.7.2.post1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.7.0.post2
Packages for Flash-Attention 2.7.0.post2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.6.3
Packages for Flash-Attention 2.6.3
Flash-Attention 2.5.9
Packages for Flash-Attention 2.5.9
Flash-Attention 2.5.6
Packages for Flash-Attention 2.5.6
Flash-Attention 2.4.3
Packages for Flash-Attention 2.4.3
Flash-Attention 1.0.9
Packages for Flash-Attention 1.0.9
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
🪟 Windows x86_64
Flash-Attention 2.8.3
Packages for Flash-Attention 2.8.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.6 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.6 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.6 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.9 | 12.4 | Download1 |
| 3.10 | 2.9 | 12.6 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.6 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.6 | Download1 |
| 3.11 | 2.9 | 12.4 | Download1 |
| 3.11 | 2.9 | 12.6 | Download1, Download2 |
| 3.11 | 2.9 | 13.0 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.6 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.6 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.6 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.6 | Download1 |
| 3.12 | 2.9 | 12.4 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1, Download2 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.6 | 12.6 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
| 3.13 | 2.9 | 12.4 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1, Download2 |
| 3.13 | 2.9 | 13.0 | Download1 |
Flash-Attention 2.8.2
Packages for Flash-Attention 2.8.2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
Flash-Attention 2.7.4.post1
Packages for Flash-Attention 2.7.4.post1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.6 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.6 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.9 | 12.4 | Download1 |
| 3.10 | 2.9 | 12.6 | Download1 |
| 3.11 | 2.5 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.6 | Download1 |
| 3.11 | 2.8 | 12.6 | Download1 |
| 3.11 | 2.9 | 12.6 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.6 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.6 | Download1 |
| 3.12 | 2.8 | 12.6 | Download1 |
| 3.12 | 2.9 | 12.4 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1 |
| 3.13 | 2.6 | 12.6 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
| 3.13 | 2.9 | 12.4 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1 |
Flash-Attention 2.7.4
Packages for Flash-Attention 2.7.4
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1, Download2 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
Flash-Attention 2.6.3
Packages for Flash-Attention 2.6.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.5.9
Packages for Flash-Attention 2.5.9
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
🐧 🐧 Linux x86_64
Flash-Attention 2.8.3
Packages for Flash-Attention 2.8.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.9 | 2.5 | 12.4 | Download1 |
| 3.9 | 2.5 | 12.6 | Download1 |
| 3.9 | 2.6 | 12.4 | Download1 |
| 3.9 | 2.6 | 12.6 | Download1 |
| 3.9 | 2.7 | 12.4 | Download1 |
| 3.9 | 2.7 | 12.6 | Download1 |
| 3.9 | 2.8 | 12.4 | Download1 |
| 3.9 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.6 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.6 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.6 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.10 | 2.9 | 12.6 | Download1 |
| 3.10 | 2.9 | 12.8 | Download1 |
| 3.10 | 2.9 | 13.0 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.6 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.6 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.6 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.9 | 12.6 | Download1, Download2 |
| 3.11 | 2.9 | 12.8 | Download1, Download2 |
| 3.11 | 2.9 | 13.0 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.6 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.6 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.6 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.6 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1, Download2 |
| 3.12 | 2.9 | 12.8 | Download1, Download2 |
| 3.12 | 2.9 | 13.0 | Download1 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.6 | 12.6 | Download1 |
| 3.13 | 2.6 | 12.8 | Download1 |
| 3.13 | 2.6 | 12.9 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.7 | 12.8 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.8 | Download1 |
| 3.13 | 2.8 | 12.9 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1, Download2 |
| 3.13 | 2.9 | 12.8 | Download1, Download2 |
| 3.13 | 2.9 | 13.0 | Download1 |
Flash-Attention 2.8.2
Packages for Flash-Attention 2.8.2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.9 | Download1 |
Flash-Attention 2.8.1
Packages for Flash-Attention 2.8.1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.8.0
Packages for Flash-Attention 2.8.0
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1, Download2 |
| 3.10 | 2.4 | 12.8 | Download1, Download2 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1, Download2 |
| 3.10 | 2.5 | 12.8 | Download1, Download2 |
| 3.10 | 2.6 | 12.4 | Download1, Download2 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1, Download2 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1, Download2 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1, Download2 |
| 3.11 | 2.6 | 12.8 | Download1, Download2 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1, Download2 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1, Download2 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1, Download2 |
| 3.12 | 2.5 | 12.8 | Download1, Download2 |
| 3.12 | 2.6 | 12.4 | Download1, Download2 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.7.4.post1
Packages for Flash-Attention 2.7.4.post1
Flash-Attention 2.7.4
Packages for Flash-Attention 2.7.4
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1, Download2 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1, Download2 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1, Download2 |
| 3.10 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.10 | 2.8 | 12.4 | Download1, Download2 |
| 3.10 | 2.8 | 12.8 | Download1, Download2 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1, Download2 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1, Download2 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1, Download2 |
| 3.11 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.11 | 2.8 | 12.4 | Download1, Download2 |
| 3.11 | 2.8 | 12.8 | Download1, Download2 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1, Download2 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1, Download2 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1, Download2 |
| 3.12 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.12 | 2.8 | 12.4 | Download1, Download2 |
| 3.12 | 2.8 | 12.8 | Download1, Download2 |
| 3.12 | 2.8 | 12.9 | Download1 |
Flash-Attention 2.7.3
Packages for Flash-Attention 2.7.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.7.2.post1
Packages for Flash-Attention 2.7.2.post1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.7.0.post2
Packages for Flash-Attention 2.7.0.post2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.6.3
Packages for Flash-Attention 2.6.3
Flash-Attention 2.5.9
Packages for Flash-Attention 2.5.9
Flash-Attention 2.5.6
Packages for Flash-Attention 2.5.6
Flash-Attention 2.4.3
Packages for Flash-Attention 2.4.3
Flash-Attention 1.0.9
Packages for Flash-Attention 1.0.9
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
🐧 🐧 🐧 Linux x86_64
Flash-Attention 2.8.3
Packages for Flash-Attention 2.8.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.9 | 2.5 | 12.4 | Download1 |
| 3.9 | 2.5 | 12.6 | Download1 |
| 3.9 | 2.6 | 12.4 | Download1 |
| 3.9 | 2.6 | 12.6 | Download1 |
| 3.9 | 2.7 | 12.4 | Download1 |
| 3.9 | 2.7 | 12.6 | Download1 |
| 3.9 | 2.8 | 12.4 | Download1 |
| 3.9 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.6 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.6 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.6 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.10 | 2.9 | 12.6 | Download1 |
| 3.10 | 2.9 | 12.8 | Download1 |
| 3.10 | 2.9 | 13.0 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.6 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.6 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.6 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.9 | 12.6 | Download1, Download2 |
| 3.11 | 2.9 | 12.8 | Download1, Download2 |
| 3.11 | 2.9 | 13.0 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.6 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.6 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.6 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.6 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1, Download2 |
| 3.12 | 2.9 | 12.8 | Download1, Download2 |
| 3.12 | 2.9 | 13.0 | Download1 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.6 | 12.6 | Download1 |
| 3.13 | 2.6 | 12.8 | Download1 |
| 3.13 | 2.6 | 12.9 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.7 | 12.8 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.8 | Download1 |
| 3.13 | 2.8 | 12.9 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1, Download2 |
| 3.13 | 2.9 | 12.8 | Download1, Download2 |
| 3.13 | 2.9 | 13.0 | Download1 |
Flash-Attention 2.8.2
Packages for Flash-Attention 2.8.2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.9 | Download1 |
Flash-Attention 2.8.1
Packages for Flash-Attention 2.8.1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.8.0
Packages for Flash-Attention 2.8.0
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1, Download2 |
| 3.10 | 2.4 | 12.8 | Download1, Download2 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1, Download2 |
| 3.10 | 2.5 | 12.8 | Download1, Download2 |
| 3.10 | 2.6 | 12.4 | Download1, Download2 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1, Download2 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1, Download2 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1, Download2 |
| 3.11 | 2.6 | 12.8 | Download1, Download2 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1, Download2 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1, Download2 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1, Download2 |
| 3.12 | 2.5 | 12.8 | Download1, Download2 |
| 3.12 | 2.6 | 12.4 | Download1, Download2 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.7.4.post1
Packages for Flash-Attention 2.7.4.post1
Flash-Attention 2.7.4
Packages for Flash-Attention 2.7.4
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1, Download2 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1, Download2 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.9 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1, Download2 |
| 3.10 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.10 | 2.8 | 12.4 | Download1, Download2 |
| 3.10 | 2.8 | 12.8 | Download1, Download2 |
| 3.10 | 2.8 | 12.9 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1, Download2 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1, Download2 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.9 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1, Download2 |
| 3.11 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.11 | 2.8 | 12.4 | Download1, Download2 |
| 3.11 | 2.8 | 12.8 | Download1, Download2 |
| 3.11 | 2.8 | 12.9 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1, Download2 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1, Download2 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.9 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1, Download2 |
| 3.12 | 2.7 | 12.8 | Download1, Download2, Download3 |
| 3.12 | 2.8 | 12.4 | Download1, Download2 |
| 3.12 | 2.8 | 12.8 | Download1, Download2 |
| 3.12 | 2.8 | 12.9 | Download1 |
Flash-Attention 2.7.3
Packages for Flash-Attention 2.7.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.7.2.post1
Packages for Flash-Attention 2.7.2.post1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.7.0.post2
Packages for Flash-Attention 2.7.0.post2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
Flash-Attention 2.6.3
Packages for Flash-Attention 2.6.3
Flash-Attention 2.5.9
Packages for Flash-Attention 2.5.9
Flash-Attention 2.5.6
Packages for Flash-Attention 2.5.6
Flash-Attention 2.4.3
Packages for Flash-Attention 2.4.3
Flash-Attention 1.0.9
Packages for Flash-Attention 1.0.9
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.0 | 11.8 | Download1 |
| 3.10 | 2.1 | 11.8 | Download1 |
| 3.10 | 2.1 | 12.1 | Download1 |
| 3.10 | 2.1 | 12.4 | Download1 |
| 3.10 | 2.2 | 11.8 | Download1 |
| 3.10 | 2.2 | 12.1 | Download1 |
| 3.10 | 2.2 | 12.4 | Download1 |
| 3.10 | 2.3 | 11.8 | Download1 |
| 3.10 | 2.3 | 12.1 | Download1 |
| 3.10 | 2.3 | 12.4 | Download1 |
| 3.10 | 2.4 | 11.8 | Download1 |
| 3.10 | 2.4 | 12.1 | Download1 |
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 11.8 | Download1 |
| 3.10 | 2.5 | 12.1 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.0 | 11.8 | Download1 |
| 3.11 | 2.1 | 11.8 | Download1 |
| 3.11 | 2.1 | 12.1 | Download1 |
| 3.11 | 2.1 | 12.4 | Download1 |
| 3.11 | 2.2 | 11.8 | Download1 |
| 3.11 | 2.2 | 12.1 | Download1 |
| 3.11 | 2.2 | 12.4 | Download1 |
| 3.11 | 2.3 | 11.8 | Download1 |
| 3.11 | 2.3 | 12.1 | Download1 |
| 3.11 | 2.3 | 12.4 | Download1 |
| 3.11 | 2.4 | 11.8 | Download1 |
| 3.11 | 2.4 | 12.1 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.5 | 11.8 | Download1 |
| 3.11 | 2.5 | 12.1 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.2 | 11.8 | Download1 |
| 3.12 | 2.2 | 12.1 | Download1 |
| 3.12 | 2.2 | 12.4 | Download1 |
| 3.12 | 2.3 | 11.8 | Download1 |
| 3.12 | 2.3 | 12.1 | Download1 |
| 3.12 | 2.3 | 12.4 | Download1 |
| 3.12 | 2.4 | 11.8 | Download1 |
| 3.12 | 2.4 | 12.1 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 11.8 | Download1 |
| 3.12 | 2.5 | 12.1 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
🪟 🪟 Windows x86_64
Flash-Attention 2.8.3
Packages for Flash-Attention 2.8.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.6 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.6 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.6 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.6 | Download1 |
| 3.10 | 2.9 | 12.4 | Download1 |
| 3.10 | 2.9 | 12.6 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.6 | Download1 |
| 3.11 | 2.8 | 12.4 | Download1 |
| 3.11 | 2.8 | 12.6 | Download1 |
| 3.11 | 2.9 | 12.4 | Download1 |
| 3.11 | 2.9 | 12.6 | Download1, Download2 |
| 3.11 | 2.9 | 13.0 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.6 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.6 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.6 | Download1 |
| 3.12 | 2.8 | 12.4 | Download1 |
| 3.12 | 2.8 | 12.6 | Download1 |
| 3.12 | 2.9 | 12.4 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1, Download2 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.6 | 12.6 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
| 3.13 | 2.9 | 12.4 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1, Download2 |
| 3.13 | 2.9 | 13.0 | Download1 |
Flash-Attention 2.8.2
Packages for Flash-Attention 2.8.2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
Flash-Attention 2.7.4.post1
Packages for Flash-Attention 2.7.4.post1
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.5 | 12.6 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.6 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.8 | 12.4 | Download1 |
| 3.10 | 2.9 | 12.4 | Download1 |
| 3.10 | 2.9 | 12.6 | Download1 |
| 3.11 | 2.5 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.6 | Download1 |
| 3.11 | 2.8 | 12.6 | Download1 |
| 3.11 | 2.9 | 12.6 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.6 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.6 | Download1 |
| 3.12 | 2.8 | 12.6 | Download1 |
| 3.12 | 2.9 | 12.4 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1 |
| 3.13 | 2.6 | 12.6 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
| 3.13 | 2.9 | 12.4 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1 |
Flash-Attention 2.7.4
Packages for Flash-Attention 2.7.4
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1, Download2 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
Flash-Attention 2.6.3
Packages for Flash-Attention 2.6.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.5.9
Packages for Flash-Attention 2.5.9
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
🪟 🪟 🪟 Windows x86_64
Flash-Attention 2.8.3
Packages for Flash-Attention 2.8.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.11 | 2.9 | 12.6 | Download1 |
| 3.12 | 2.9 | 12.6 | Download1 |
| 3.13 | 2.9 | 12.6 | Download1 |
Flash-Attention 2.8.2
Packages for Flash-Attention 2.8.2
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
| 3.13 | 2.6 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.4 | Download1 |
| 3.13 | 2.7 | 12.6 | Download1 |
| 3.13 | 2.8 | 12.4 | Download1 |
| 3.13 | 2.8 | 12.6 | Download1 |
Flash-Attention 2.7.4
Packages for Flash-Attention 2.7.4
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.10 | 2.8 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1, Download2 |
| 3.11 | 2.8 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.8 | 12.8 | Download1 |
Flash-Attention 2.6.3
Packages for Flash-Attention 2.6.3
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.6 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
Flash-Attention 2.5.9
Packages for Flash-Attention 2.5.9
| Python | PyTorch | CUDA | package |
|---|---|---|---|
| 3.10 | 2.4 | 12.4 | Download1 |
| 3.10 | 2.4 | 12.8 | Download1 |
| 3.10 | 2.5 | 12.4 | Download1 |
| 3.10 | 2.5 | 12.8 | Download1 |
| 3.10 | 2.6 | 12.4 | Download1 |
| 3.10 | 2.6 | 12.8 | Download1 |
| 3.10 | 2.7 | 12.4 | Download1 |
| 3.10 | 2.7 | 12.8 | Download1 |
| 3.11 | 2.4 | 12.4 | Download1 |
| 3.11 | 2.4 | 12.8 | Download1 |
| 3.11 | 2.5 | 12.4 | Download1 |
| 3.11 | 2.5 | 12.8 | Download1 |
| 3.11 | 2.6 | 12.4 | Download1 |
| 3.11 | 2.6 | 12.8 | Download1 |
| 3.11 | 2.7 | 12.4 | Download1 |
| 3.11 | 2.7 | 12.8 | Download1 |
| 3.12 | 2.4 | 12.4 | Download1 |
| 3.12 | 2.4 | 12.8 | Download1 |
| 3.12 | 2.5 | 12.4 | Download1 |
| 3.12 | 2.5 | 12.8 | Download1 |
| 3.12 | 2.6 | 12.4 | Download1 |
| 3.12 | 2.6 | 12.8 | Download1 |
| 3.12 | 2.7 | 12.4 | Download1 |
| 3.12 | 2.7 | 12.8 | Download1 |
History
History of this repository is available here.
Self build
If you cannot find the version you are looking for, you can fork this repository and create a wheel on GitHub Actions.
- Fork this repository
- Edit workflow file
.github/workflows/build.ymlto set the version you want to build. - Add tag
v*.*.*to trigger the build workflow.
Please note that depending on the combination of versions, it may not be possible to build.
Self-Hosted Runner Build
In some version combinations, you cannot build wheels on GitHub-hosted runners due to job time limitations. To build the wheels for these versions, you can use self-hosted runners.
git clone https://github.com/mjun0812/flash-attention-prebuild-wheels.git
cd self-hosted-runner
cp env.template env
Edit env file to set the environment variables.
# Edit env
PERSONAL_ACCESS_TOKEN=[Github Personal Access Token]
Edit compose.yml file if you use repository folked from this repository.
services:
runner:
privileged: true
build:
context: .
dockerfile: Dockerfile
args:
REPOSITORY_URL: [Target Repository URL]
PERSONAL_ACCESS_TOKEN: $PERSONAL_ACCESS_TOKEN
GH_RUNNER_VERSION: 2.324.0
RUNNER_NAME: self-hosted-runner
RUNNER_GROUP: default
RUNNER_LABELS: self-hosted
TARGET_ARCH: x64
Then, build and run the docker container.
# Build and run
docker compose build
docker compose up -d
Original Repository
@inproceedings{dao2022flashattention,
title={Flash{A}ttention: Fast and Memory-Efficient Exact Attention with {IO}-Awareness},
author={Dao, Tri and Fu, Daniel Y. and Ermon, Stefano and Rudra, Atri and R{\'e}, Christopher},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
year={2022}
}
@inproceedings{dao2023flashattention2,
title={Flash{A}ttention-2: Faster Attention with Better Parallelism and Work Partitioning},
author={Dao, Tri},
booktitle={International Conference on Learning Representations (ICLR)},
year={2024}
}