diff --git a/README.md b/README.md index a858877e6..045353f1c 100644 --- a/README.md +++ b/README.md @@ -306,7 +306,7 @@ GPUs report compute capability 12.0 (`sm_120`) and should use the CPU package or build from source with `-DCMAKE_CUDA_ARCHITECTURES=120` until a dedicated CUDA asset is published. -**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.3](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.3/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.3/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.3/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/en/deploy/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/) +**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.4](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/en/deploy/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html) @@ -330,6 +330,7 @@ asset is published. ## What's new +- 2026/08/29: **llama.cpp runtime v0.2.4** — fixes intermittent empty SenseVoiceSmall transcripts with F16 GGUF models. Query embeddings are now decoded according to their GGML F16/F32 type instead of reading F16 storage as F32. The exact v0.2.3 AVX2 asset reproduced 22 blank outputs across 298 completed runs; the fixed head produced one identical non-empty result in 100/100 runs and matched the Q8 model byte-for-byte. The release workflow will build nine Linux, macOS, and Windows archives from one exact commit. [Fix ->](https://github.com/modelscope/FunASR/pull/3550) · [Release ->](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) - 2026/08/29: **v1.4.6 on PyPI** — the realtime WebSocket server keeps 20-second pings but no longer closes healthy sessions by default when queued decoding delays pong handling; operators can still configure a positive timeout after measuring production queue and decode latency. This release also avoids `IndexError` on empty English timestamp segments and groups adjacent subtitle words into readable cues. Upgrade with `python -m pip install -U "funasr==1.4.6"`. The GitHub Release carries the verified nine-platform llama.cpp v0.2.3 runtime archives. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.6) - 2026/08/29: **llama.cpp runtime v0.2.3** — adds flushed stage boundaries after backend initialization for model loading, audio/VAD, graph construction and allocation, and compute. These diagnostics narrow Windows AMD Vulkan `0xC0000005` failures that occur after `vulkan backend ready`; they do not claim the hardware-specific crash is fixed. Nine Linux, macOS, and Windows archives are published from one exact commit. [Troubleshooting boundaries ->](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [Release ->](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3) - 2026/08/28: **v1.4.5 on PyPI** — `torchaudio` is no longer a hard inference dependency. Feature extraction uses `torchaudio.compliance.kaldi` when available or the optional `kaldi-native-fbank` backend, while operations that still require `torchaudio` now fail with an actionable message. The fallback was verified end to end on Ascend 910B: 70.47 s of audio processed in 1.15 s (RTF 0.016). Upgrade with `python -m pip install -U "funasr==1.4.5"`; on systems without `torchaudio`, install `python -m pip install -U "funasr[knf]==1.4.5"`. The GitHub Release also carries the verified nine-platform llama.cpp v0.2.1 runtime archives. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.5) diff --git a/README_ja.md b/README_ja.md index 7a3072af7..b9885ab1d 100644 --- a/README_ja.md +++ b/README_ja.md @@ -99,6 +99,7 @@ Whisper は単一モデルですが、**FunASR はツールキット**です— ## 最新情報 +- 2026/08/29：**llama.cpp runtime v0.2.4** — F16 GGUF model で transcript が断続的に空になる問題を修正しました。Query embedding は F16 storage を F32 として読むのではなく、GGML の F16/F32 type に従って decode されます。正確な v0.2.3 AVX2 asset では 298 回の完了 run 中 22 回の空出力を再現しましたが、修正後は 100/100 回で同一の非空出力となり、Q8 model と byte 単位で一致しました。Release workflow は同一の exact commit から 9 platform archive を構築します。[修正 →](https://github.com/modelscope/FunASR/pull/3550) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) - 2026/08/29：**v1.4.6 を PyPI に公開** — realtime WebSocket server は 20 秒間隔の ping を維持しつつ、decode queue によって pong 処理が遅れた正常な session を default では切断しなくなりました。運用環境の queue と decode latency を測定したうえで、正の timeout を明示設定することもできます。空の英語 timestamp segment による `IndexError` を防ぎ、隣接する subtitle word を読みやすい cue にまとめます。更新：`python -m pip install -U "funasr==1.4.6"`。GitHub Release には検証済みの llama.cpp v0.2.3 9 platform runtime archive も含まれます。[Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.6) - 2026/08/29：**llama.cpp runtime v0.2.3** — backend 初期化後に、model 読み込み、audio/VAD、graph の構築と割り当て、compute の各段階を即時 flush する境界ログを追加しました。`vulkan backend ready` 後に起きる Windows AMD Vulkan `0xC0000005` の発生箇所を絞り込むための診断であり、hardware 固有の crash が修正済みとはしていません。同一の exact commit から Linux、macOS、Windows 向け 9 archive を公開します。[トラブルシューティング →](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3) - 2026/08/28：**v1.4.5 を PyPI に公開** — `torchaudio` は推論の必須依存関係ではなくなりました。特徴抽出は利用可能な場合に `torchaudio.compliance.kaldi` を使い、そうでなければ任意の `kaldi-native-fbank` backend を利用できます。引き続き `torchaudio` が必要な処理は、実行可能な依存関係メッセージを返します。この fallback は Ascend 910B で end-to-end 検証済みで、70.47 秒の音声を 1.15 秒（RTF 0.016）で処理しました。更新：`python -m pip install -U "funasr==1.4.5"`。`torchaudio` がない環境：`python -m pip install -U "funasr[knf]==1.4.5"`。GitHub Release には検証済みの llama.cpp v0.2.1 9 platform runtime archive も含まれます。[Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.5) diff --git a/README_ko.md b/README_ko.md index 224d1e688..bdd447d1f 100644 --- a/README_ko.md +++ b/README_ko.md @@ -99,6 +99,7 @@ Whisper는 단일 모델이지만, **FunASR는 툴킷**입니다. 용도에 맞 ## 최신 소식 +- 2026/08/29: **llama.cpp runtime v0.2.4** — F16 GGUF model에서 transcript가 간헐적으로 비는 문제를 수정했습니다. Query embedding은 F16 storage를 F32로 읽지 않고 GGML F16/F32 type에 맞게 decode됩니다. 정확한 v0.2.3 AVX2 asset은 완료된 298회 실행 중 22회의 빈 출력을 재현했지만, 수정 후에는 100/100회 동일한 비어 있지 않은 결과를 냈고 Q8 model과 byte 단위로 일치했습니다. Release workflow가 동일한 exact commit에서 9개 platform archive를 빌드합니다. [수정 →](https://github.com/modelscope/FunASR/pull/3550) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) - 2026/08/29: **v1.4.6 PyPI 공개** — realtime WebSocket server는 20초 간격 ping을 유지하면서 decode queue 때문에 pong 처리가 늦어진 정상 session을 기본적으로 종료하지 않습니다. 운영 환경의 queue 및 decode latency를 측정한 뒤 양수 timeout을 명시적으로 설정할 수도 있습니다. 빈 영어 timestamp segment에서 발생하는 `IndexError`를 방지하고 인접 subtitle word를 읽기 쉬운 cue로 묶습니다. 업데이트: `python -m pip install -U "funasr==1.4.6"`. GitHub Release에는 검증된 llama.cpp v0.2.3 9개 platform runtime archive도 포함됩니다. [Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.6) - 2026/08/29: **llama.cpp runtime v0.2.3** — backend 초기화 이후 model load, audio/VAD, graph build와 allocation, compute 단계마다 즉시 flush되는 경계 로그를 추가했습니다. `vulkan backend ready` 이후 발생하는 Windows AMD Vulkan `0xC0000005`의 위치를 좁히기 위한 진단이며, hardware별 crash가 수정됐다고 주장하지 않습니다. 동일한 exact commit에서 Linux, macOS, Windows용 archive 9개를 공개합니다. [문제 해결 경계 →](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3) - 2026/08/28: **v1.4.5 PyPI 공개** — `torchaudio`는 더 이상 inference의 필수 dependency가 아닙니다. Feature extraction은 사용할 수 있으면 `torchaudio.compliance.kaldi`를 사용하고, 그렇지 않으면 선택형 `kaldi-native-fbank` backend를 사용할 수 있습니다. 여전히 `torchaudio`가 필요한 작업은 해결 방법이 포함된 dependency 메시지를 반환합니다. 이 fallback은 Ascend 910B에서 end-to-end로 검증됐으며 70.47초 오디오를 1.15초(RTF 0.016)에 처리했습니다. 업데이트: `python -m pip install -U "funasr==1.4.5"`. `torchaudio`가 없는 환경: `python -m pip install -U "funasr[knf]==1.4.5"`. GitHub Release에는 검증된 llama.cpp v0.2.1 9개 platform runtime archive도 함께 제공됩니다. [Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.5) diff --git a/README_zh.md b/README_zh.md index c6fc7ad2b..a919880a1 100644 --- a/README_zh.md +++ b/README_zh.md @@ -149,6 +149,7 @@ Whisper 是单个模型，**FunASR 是一个工具箱**——按场景挑模型 ## 最新动态 +- 2026/08/29：**llama.cpp runtime v0.2.4** — 修复 F16 GGUF 模型偶发空转写。运行时现在按 GGML 的 F16/F32 类型解码查询 embedding，不再把 F16 存储误读为 F32。精确的 v0.2.3 AVX2 发布包在 298 次完整运行中复现 22 次空结果；修复后 100/100 次输出稳定唯一，并与 Q8 模型逐字节一致。发布工作流将在同一准确提交上构建九个 Linux、macOS 与 Windows 压缩包。[修复 →](https://github.com/modelscope/FunASR/pull/3550) · [发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) - 2026/08/29：**v1.4.6 已发布到 PyPI** — 实时 WebSocket 服务继续每 20 秒发送 ping，但当排队解码延迟 pong 处理时，默认不再误关仍健康的连接；运维人员仍可在测量生产队列与解码延迟后显式配置正数超时。本版本同时避免空英文时间戳片段触发 `IndexError`，并将相邻字幕词组合为更易读的字幕段。升级命令：`python -m pip install -U "funasr==1.4.6"`。GitHub 发布页同时提供已验证的 llama.cpp v0.2.3 九平台运行包。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.4.6) - 2026/08/29：**llama.cpp runtime v0.2.3** — 后端初始化完成后新增可立即刷新的阶段边界，覆盖模型加载、音频/VAD、计算图构建与分配、推理计算，可继续缩小发生在 `vulkan backend ready` 之后的 Windows AMD Vulkan `0xC0000005` 崩溃范围；本版本不宣称已经修复该硬件相关崩溃。九个 Linux、macOS 与 Windows 压缩包由同一准确提交构建并发布。[排障边界 →](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3) - 2026/08/28：**v1.4.5 已发布到 PyPI** — `torchaudio` 不再是推理的硬依赖。特征提取会优先使用 `torchaudio.compliance.kaldi`，不可用时可切换到可选的 `kaldi-native-fbank` 后端；仍然必须使用 `torchaudio` 的操作会给出可执行的依赖提示。该 fallback 已在 Ascend 910B 上端到端验证：70.47 秒音频用时 1.15 秒（RTF 0.016）。常规升级：`python -m pip install -U "funasr==1.4.5"`；没有 `torchaudio` 的环境请安装：`python -m pip install -U "funasr[knf]==1.4.5"`。GitHub 发布页同时提供已验证的 llama.cpp v0.2.1 九平台运行包。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.4.5) @@ -363,7 +364,7 @@ Windows Vulkan ZIP 使用显卡驱动提供的系统 Vulkan loader，不需要 compute capability 12.0（`sm_120`），在专用 CUDA 产物发布前，请使用 CPU 包，或从 源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=120`。 -**预编译二进制：** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.3](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.3/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.3/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.3/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始：** [funasr.com/deploy/llama-cpp](https://www.funasr.com/deploy/llama-cpp.html) · **GGUF 模型：** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测：** [runtime/llama.cpp/](./runtime/llama.cpp/) +**预编译二进制：** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.4](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始：** [funasr.com/deploy/llama-cpp](https://www.funasr.com/deploy/llama-cpp.html) · **GGUF 模型：** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测：** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html) diff --git a/runtime/llama.cpp/current-release.txt b/runtime/llama.cpp/current-release.txt index 19448e06d..055729335 100644 --- a/runtime/llama.cpp/current-release.txt +++ b/runtime/llama.cpp/current-release.txt @@ -1 +1 @@ -runtime-llamacpp-v0.2.3 +runtime-llamacpp-v0.2.4 diff --git a/runtime/llama.cpp/tests/test_download_funasr_model.sh b/runtime/llama.cpp/tests/test_download_funasr_model.sh index 4b7c8ed40..da42b0963 100755 --- a/runtime/llama.cpp/tests/test_download_funasr_model.sh +++ b/runtime/llama.cpp/tests/test_download_funasr_model.sh @@ -170,8 +170,8 @@ assert_readme_quickstart() { printf 'non-portable Releases link in %s\n' "$readme" >&2 exit 1 fi - if ! grep -F 'runtime-llamacpp-v0.2.3' <<<"$section" >/dev/null; then - printf 'missing current runtime v0.2.3 link in %s\n' "$readme" >&2 + if ! grep -F 'runtime-llamacpp-v0.2.4' <<<"$section" >/dev/null; then + printf 'missing current runtime v0.2.4 link in %s\n' "$readme" >&2 exit 1 fi if ! grep -F 'funasr-llamacpp-linux-x64-vulkan' <<<"$section" >/dev/null; then