mirror of
https://github.com/modelscope/FunASR.git
synced 2026-10-12 02:19:18 +00:00
feat(runtime): add Windows Blackwell CUDA asset
Signed-off-by: LauraGPT <[email protected]>
This commit is contained in:
LauraGPT
@@ -145,6 +145,30 @@ jobs:
|
||||
-DGGML_FMA=OFF
|
||||
-DGGML_F16C=OFF
|
||||
-DGGML_BMI2=OFF
|
||||
- os: windows-latest
|
||||
name: windows-x64-cuda-blackwell
|
||||
cuda: true
|
||||
cuda_version: '13.2.0'
|
||||
cuda_architectures: '120'
|
||||
build_target: llama-funasr-sensevoice
|
||||
timeout_minutes: 90
|
||||
cmake_flags: >-
|
||||
-DGGML_NATIVE=OFF
|
||||
-DGGML_CUDA=ON
|
||||
-DGGML_CUDA_FORCE_CUBLAS=ON
|
||||
-DGGML_CUDA_FA=OFF
|
||||
-DGGML_CUDA_NCCL=OFF
|
||||
-DGGML_CUDA_GRAPHS=OFF
|
||||
-DGGML_AVX=OFF
|
||||
-DGGML_AVX2=OFF
|
||||
-DGGML_AVX_VNNI=OFF
|
||||
-DGGML_AVX512=OFF
|
||||
-DGGML_AVX512_VBMI=OFF
|
||||
-DGGML_AVX512_VNNI=OFF
|
||||
-DGGML_AVX512_BF16=OFF
|
||||
-DGGML_FMA=OFF
|
||||
-DGGML_F16C=OFF
|
||||
-DGGML_BMI2=OFF
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: ${{ matrix.timeout_minutes || 30 }}
|
||||
defaults:
|
||||
@@ -257,4 +281,4 @@ jobs:
|
||||
gh release create "$tag" dist/* \
|
||||
--repo "${{ github.repository }}" \
|
||||
--title "FunASR llama.cpp runtime $version" \
|
||||
--notes "Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer and Fun-ASR-Nano with built-in FSMN-VAD. Download the default quantized model with \`bash download-funasr-model.sh <sensevoice|paraformer|nano>\` (the helper requires the Hugging Face CLI: \`pip install -U huggingface_hub\`), then run \`llama-funasr-cli\` / \`llama-funasr-sensevoice\` / \`llama-funasr-paraformer\`. Use the default x64 asset for maximum CPU compatibility; use the x64-avx2 asset on CPUs with AVX2/FMA/F16C/BMI2 for higher throughput. The Vulkan assets are \`linux-x64-vulkan\` and \`windows-x64-vulkan\`; they require a working Vulkan driver/ICD and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend vulkan\`. Build from source with \`-DGGML_VULKAN=ON\` to validate platform-specific GPU stacks. The Windows CUDA asset is \`windows-x64-cuda\`; it requires an NVIDIA driver compatible with the CUDA Toolkit version configured by the release workflow, targets CUDA architecture 86, and enables SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend cuda\`. Build from source for other GPU architectures. No Python ASR runtime or local build is required. Docs: $docs"
|
||||
--notes "Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer and Fun-ASR-Nano with built-in FSMN-VAD. Download the default quantized model with \`bash download-funasr-model.sh <sensevoice|paraformer|nano>\` (the helper requires the Hugging Face CLI: \`pip install -U huggingface_hub\`), then run \`llama-funasr-cli\` / \`llama-funasr-sensevoice\` / \`llama-funasr-paraformer\`. Use the default x64 asset for maximum CPU compatibility; use the x64-avx2 asset on CPUs with AVX2/FMA/F16C/BMI2 for higher throughput. The Vulkan assets are \`linux-x64-vulkan\` and \`windows-x64-vulkan\`; they require a working Vulkan driver/ICD and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend vulkan\`. Build from source with \`-DGGML_VULKAN=ON\` to validate platform-specific GPU stacks. The Windows CUDA assets are \`windows-x64-cuda\` for CUDA architecture 86 and \`windows-x64-cuda-blackwell\` for CUDA architecture 120 (RTX 50 / Blackwell); both require a compatible NVIDIA driver and enable SenseVoiceSmall graph execution with \`llama-funasr-sensevoice ... --backend cuda\`. Build from source for other GPU architectures. A successful CI build verifies code generation and archive integrity, not execution on physical Blackwell hardware. No Python ASR runtime or local build is required. Docs: $docs"
|
||||
|
||||
+13
-10
@@ -93,11 +93,11 @@ FSMN-VAD without Python.
|
||||
|
||||
### Optional Windows CUDA backend for SenseVoiceSmall
|
||||
|
||||
The CPU release ZIPs are portable packages. Tagged releases also publish
|
||||
`funasr-llamacpp-windows-x64-cuda.zip` for SenseVoiceSmall graph execution on
|
||||
NVIDIA GPUs that match CUDA architecture 86. Download the CUDA ZIP from
|
||||
[runtime-llamacpp-v0.1.9](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.9),
|
||||
then select the backend at runtime:
|
||||
The CPU release ZIPs are portable packages. Tagged releases publish separate
|
||||
SenseVoiceSmall CUDA packages: `funasr-llamacpp-windows-x64-cuda.zip` targets
|
||||
CUDA architecture 86, while `funasr-llamacpp-windows-x64-cuda-blackwell.zip`
|
||||
targets CUDA architecture 120 (`sm_120`) for RTX 50 / Blackwell GPUs. Select the
|
||||
matching archive, then enable the backend at runtime:
|
||||
|
||||
```bash
|
||||
# From the extracted windows-x64-cuda package:
|
||||
@@ -105,7 +105,9 @@ then select the backend at runtime:
|
||||
-m sensevoice-small-q8.gguf --vad fsmn-vad.gguf -a sample.wav --backend cuda
|
||||
```
|
||||
|
||||
Build from source to target other GPU architectures:
|
||||
The Blackwell package uses the same command from its extracted directory. Build
|
||||
from source to target other GPU architectures or to reproduce the architecture
|
||||
120 build locally:
|
||||
|
||||
```bash
|
||||
cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
|
||||
@@ -115,10 +117,11 @@ cmake --build build-cuda -j --target llama-funasr-sensevoice
|
||||
-m sensevoice-small-f16.gguf -a sample.wav --backend cuda
|
||||
```
|
||||
|
||||
Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. RTX 50 /
|
||||
Blackwell cards report compute capability 12.0 (`sm_120`), so the current
|
||||
`windows-x64-cuda` prebuilt package for architecture 86 will not cover those
|
||||
cards.
|
||||
Use the matching `CMAKE_CUDA_ARCHITECTURES` value for your GPU. A successful
|
||||
release workflow build verifies architecture 120 code generation and ZIP
|
||||
integrity; it does not prove execution on physical Blackwell hardware. Keep
|
||||
hardware-specific reports open until the matching archive is retested on the
|
||||
reported GPU.
|
||||
|
||||
`--backend cpu` remains the default and is what the portable cross-platform
|
||||
prebuilt binaries use. The CUDA package requires an NVIDIA driver compatible
|
||||
|
||||
@@ -16,6 +16,19 @@ def test_windows_cuda_release_asset_is_in_matrix():
|
||||
assert "timeout_minutes: 90" in workflow
|
||||
|
||||
|
||||
def test_windows_blackwell_cuda_release_asset_is_in_matrix():
|
||||
workflow = WORKFLOW.read_text(encoding="utf-8")
|
||||
entry = workflow.split("name: windows-x64-cuda-blackwell", maxsplit=1)[1].split(
|
||||
" - os:", maxsplit=1
|
||||
)[0]
|
||||
|
||||
assert "cuda: true" in entry
|
||||
assert "cuda_version: '13.2.0'" in entry
|
||||
assert "cuda_architectures: '120'" in entry
|
||||
assert "build_target: llama-funasr-sensevoice" in entry
|
||||
assert "timeout_minutes: 90" in entry
|
||||
|
||||
|
||||
def test_linux_vulkan_release_asset_is_in_matrix():
|
||||
workflow = WORKFLOW.read_text(encoding="utf-8")
|
||||
|
||||
@@ -88,10 +101,13 @@ def test_release_notes_explain_cpu_and_cuda_windows_assets():
|
||||
assert "--backend cuda" in readme
|
||||
assert "Windows CUDA" in readme
|
||||
assert "CUDA architecture 86" in readme
|
||||
assert "windows-x64-cuda-blackwell" in readme
|
||||
assert "CUDA architecture 120" in readme
|
||||
assert "Build from source" in readme
|
||||
assert "other GPU architectures" in readme
|
||||
assert "CMAKE_CUDA_ARCHITECTURES=120" in readme
|
||||
assert "sm_120" in readme
|
||||
assert "does not prove execution on physical Blackwell hardware" in readme
|
||||
|
||||
|
||||
def test_release_notes_explain_vulkan_linux_asset():
|
||||
@@ -134,6 +150,16 @@ def test_github_release_notes_mention_vulkan_asset():
|
||||
assert "--backend vulkan" in release_notes
|
||||
assert "GGML_VULKAN=ON" in release_notes
|
||||
|
||||
|
||||
def test_github_release_notes_distinguish_cuda_architecture_assets():
|
||||
workflow = WORKFLOW.read_text(encoding="utf-8")
|
||||
release_notes = workflow.split('--notes "', maxsplit=1)[1]
|
||||
|
||||
assert "windows-x64-cuda-blackwell" in release_notes
|
||||
assert "CUDA architecture 86" in release_notes
|
||||
assert "CUDA architecture 120" in release_notes
|
||||
|
||||
|
||||
def test_readme_documents_lightweight_http_server():
|
||||
readme = (ROOT / "runtime" / "llama.cpp" / "README.md").read_text(encoding="utf-8")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user