modelscope/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR.git synced 2026-10-12 02:19:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(llama.cpp): fix default GGUF quickstart (#3234)

Browse Source 
* docs(llama.cpp): fix default GGUF quickstart

Signed-off-by: 游雁 <[email protected]>

* fix(llama.cpp): make release quickstart executable

Signed-off-by: 游雁 <[email protected]>

* fix(llama.cpp): validate Windows release layout

Signed-off-by: 游雁 <[email protected]>

* test(llama.cpp): require runnable quickstart lines

Signed-off-by: 游雁 <[email protected]>

---------

Signed-off-by: 游雁 <[email protected]>
This commit is contained in:
zhifu gao
2026-07-15 20:15:07 +08:00
committed by GitHub
parent 2497cb5e8e
commit 68a21aafbb
4 changed files with 129 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/test-llamacpp-downloader.yml
+31
View File
@@ -0,0 +1,31 @@
name: Test llama.cpp downloader
on:
workflow_dispatch:
pull_request:
paths:
- 'README.md'
- 'README_zh.md'
- 'runtime/llama.cpp/download-funasr-model.sh'
- 'runtime/llama.cpp/tests/test_download_funasr_model.sh'
- '.github/workflows/test-llamacpp-downloader.yml'
push:
branches:
- main
paths:
- 'README.md'
- 'README_zh.md'
- 'runtime/llama.cpp/download-funasr-model.sh'
- 'runtime/llama.cpp/tests/test_download_funasr_model.sh'
- '.github/workflows/test-llamacpp-downloader.yml'
permissions:
contents: read
jobs:
contract:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run downloader and README contract tests
run: bash runtime/llama.cpp/tests/test_download_funasr_model.sh
README.md
+10 -3
View File
@@ -246,13 +246,20 @@ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-
Run **SenseVoice / Paraformer / Fun-ASR-Nano** as a **single self-contained binary** on CPU and edge devices — this is to FunASR what [whisper.cpp](https://github.com/ggml-org/whisper.cpp) is to Whisper, but with **~3× lower CER than whisper.cpp on Chinese**. Built-in FSMN-VAD, no Python at runtime.
```bash
# 1) Grab a prebuilt binary from Releases (Linux / macOS / Windows), then:
# Linux / macOS: run from the extracted release directory
bash download-funasr-model.sh sensevoice ./gguf # or: paraformer | nano
llama-funasr-sensevoice -m ./gguf/SenseVoiceSmall-f16.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
# → 欢迎大家来体验达摩院推出的语音识别模型
```
**Prebuilt binaries:** [Releases](../../releases) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)
```powershell
# Windows PowerShell: run from the extracted archive root (with the `hf` CLI installed)
hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf
hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav
```
**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)
[OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)
README_zh.md
+19 -1
View File
@@ -274,7 +274,25 @@ curl http://localhost:8000/v1/audio/transcriptions \
docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12
```
> **CPU / 边缘部署(无需 GPU、无需 Python):** 用 **llama.cpp / GGUF** 跑 Fun-ASR-Nano / SenseVoice / Paraformer —— 单个自包含二进制,对标 whisper.cpp。详见 [runtime/llama.cpp/](./runtime/llama.cpp/)。
### CPU / 边缘部署 - llama.cpp / GGUF（无需 GPU、无需 Python）
在 CPU 和边缘设备上用单个自包含二进制运行 **SenseVoice / Paraformer / Fun-ASR-Nano**，无需 Python 运行环境，并内置 FSMN-VAD。
```bash
# Linux / macOS：在解压后的发布目录中执行
bash download-funasr-model.sh sensevoice ./gguf # 也可使用 paraformer 或 nano
./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav
# -> 欢迎大家来体验达摩院推出的语音识别模型
```
```powershell
# Windows PowerShell：在解压根目录执行（需已安装 `hf` CLI）
hf download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir .\gguf
hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf
.\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav
```
**预编译二进制：** [Releases](https://github.com/modelscope/FunASR/releases) · **下载与快速开始：** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型：** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测：** [runtime/llama.cpp/](./runtime/llama.cpp/)
[OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)
runtime/llama.cpp/tests/test_download_funasr_model.sh
+69
View File
@@ -2,6 +2,7 @@
set -euo pipefail
ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
REPO_ROOT=$(cd "$ROOT/../.." && pwd)
SCRIPT="$ROOT/download-funasr-model.sh"
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
@@ -62,6 +63,8 @@ bash "$SCRIPT" sensevoice "$TMP/out" >/dev/null
assert_log "download FunAudioLLM/SenseVoiceSmall-GGUF sensevoice-small-q8.gguf --local-dir $TMP/out"
assert_log "download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir $TMP/out"
assert_no_log "--include *.gguf"
DEFAULT_SENSEVOICE_MODEL=$(awk '$1 == "download" && $2 == "FunAudioLLM/SenseVoiceSmall-GGUF" { print $3; exit }' "$HF_LOG")
[[ -n "$DEFAULT_SENSEVOICE_MODEL" ]]
reset_case
bash "$SCRIPT" paraformer "$TMP/out" f16 >/dev/null
@@ -106,4 +109,70 @@ if HF_SKIP_CREATE=1 bash "$SCRIPT" sensevoice "$TMP/out" >"$TMP/stdout" 2>"$TMP/
fi
grep -F "no GGUF files found in $TMP/out" "$TMP/stderr" >/dev/null
assert_readme_quickstart() {
local readme=$1
local section_prefix=$2
local section
local quickstart
local powershell
if ! section=$(awk -v prefix="$section_prefix" '
!found && index($0, prefix) == 1 { found = 1 }
found && /^---$/ { exit }
found { print }
END { if (!found) exit 1 }
' "$readme"); then
printf 'missing CPU/edge section in %s\n' "$readme" >&2
exit 1
fi
if ! quickstart=$(printf '%s\n' "$section" | awk '
!seen && /^```bash$/ { seen = 1; in_block = 1; next }
in_block && /^```$/ { complete = 1; exit }
in_block { print }
END { if (!seen || !complete) exit 1 }
'); then
printf 'missing Bash quickstart block in %s\n' "$readme" >&2
exit 1
fi
if ! powershell=$(printf '%s\n' "$section" | awk '
!seen && /^```powershell$/ { seen = 1; in_block = 1; next }
in_block && /^```$/ { complete = 1; exit }
in_block { print }
END { if (!seen || !complete) exit 1 }
'); then
printf 'missing PowerShell quickstart block in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Eq '^bash download-funasr-model\.sh sensevoice \./gguf([[:space:]]+#.*)?$' <<<"$quickstart"; then
printf 'missing default SenseVoice download command in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx "./llama-funasr-sensevoice -m ./gguf/$DEFAULT_SENSEVOICE_MODEL --vad ./gguf/fsmn-vad.gguf -a audio.wav" <<<"$quickstart" >/dev/null; then
printf 'README command does not run the default sensevoice-small-q8.gguf in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx "hf download FunAudioLLM/SenseVoiceSmall-GGUF $DEFAULT_SENSEVOICE_MODEL --local-dir .\\gguf" <<<"$powershell" >/dev/null; then
printf 'missing Windows SenseVoice download command in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx 'hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf' <<<"$powershell" >/dev/null; then
printf 'missing Windows FSMN-VAD download command in %s\n' "$readme" >&2
exit 1
fi
if ! grep -Fx ".\\llama-funasr-sensevoice.exe -m .\\gguf\\$DEFAULT_SENSEVOICE_MODEL --vad .\\gguf\\fsmn-vad.gguf -a audio.wav" <<<"$powershell" >/dev/null; then
printf 'missing Windows PowerShell quickstart in %s\n' "$readme" >&2
exit 1
fi
if ! grep -F '[Releases](https://github.com/modelscope/FunASR/releases)' <<<"$section" >/dev/null; then
printf 'non-portable Releases link in %s\n' "$readme" >&2
exit 1
fi
}
assert_readme_quickstart "$REPO_ROOT/README.md" '### CPU / Edge'
assert_readme_quickstart "$REPO_ROOT/README_zh.md" '### CPU / 边缘部署'
echo "download-funasr-model contract tests passed"