From 331369fd64ed5d350937670c90f350e5fe263e67 Mon Sep 17 00:00:00 2001 From: LauraGPT <18321252+LauraGPT@users.noreply.github.com> Date: Wed, 2 Sep 2026 07:21:20 +0000 Subject: [PATCH] docs: clarify Fun-ASR-Nano vLLM precision boundary --- docs/vllm_guide.md | 7 +++++++ docs/vllm_guide_zh.md | 6 ++++++ tests/test_docs_funasr_install_commands.py | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/docs/vllm_guide.md b/docs/vllm_guide.md index 5aedf56ae..1894bf29b 100644 --- a/docs/vllm_guide.md +++ b/docs/vllm_guide.md @@ -872,6 +872,13 @@ Streaming: `serve_realtime_ws.RealtimeASRSession` **Q: Slow first startup?** vLLM initialization takes 60–90 s (KV Cache + CUDA Graph warmup). Subsequent inferences are instant. +**Q: What happens when Fun-ASR-Nano vLLM uses `dtype="fp16"`?** +The audio frontend and adaptor remain float16, but FunASR runs the Qwen3 decoder +in bfloat16 because vLLM float16 decoding can produce degraded repeated output. +This is automatic and keeps the two-byte decoder weight footprint. On hardware +without BF16 support, use `dtype="fp32"`; the vLLM path does not claim an +end-to-end FP16 decoder mode. + **Q: vLLM returns repeated punctuation such as `!!!!!!!!` but PyTorch/HF generate is normal. What should I check?** This usually means the audio frontend and checkpoint can work, but the vLLM prompt-embedding path or decoding parameters differ from the upstream runner. diff --git a/docs/vllm_guide_zh.md b/docs/vllm_guide_zh.md index 7b65fc621..0568fcd42 100644 --- a/docs/vllm_guide_zh.md +++ b/docs/vllm_guide_zh.md @@ -866,6 +866,12 @@ RTFx 102 → 46。CER 不变。默认关闭。 **Q: 首次慢？** vLLM 初始化 60-90s，之后即时。 +**Q: Fun-ASR-Nano vLLM 使用 `dtype="fp16"` 时实际会怎样？** +音频 frontend 与 adaptor 仍使用 FP16，但 FunASR 会让 Qwen3 decoder 使用 BF16， +因为 vLLM 的 FP16 decoder 可能产生退化的重复输出。该行为自动生效，decoder 权重 +仍是两字节精度；不支持 BF16 的硬件请使用 `dtype="fp32"`。vLLM 路径不宣称支持 +端到端 FP16 decoder。 + **Q: vLLM 输出连续标点（例如 `!!!!!!!!`），但 PyTorch/HF generate 正常，应该先查什么？** 这通常说明音频 frontend 和 checkpoint 本身能工作，但 vLLM prompt-embedding 路径或解码参数和 upstream runner 不一致。改模型前先检查这些项： diff --git a/tests/test_docs_funasr_install_commands.py b/tests/test_docs_funasr_install_commands.py index c5eec5990..2f22d83f4 100644 --- a/tests/test_docs_funasr_install_commands.py +++ b/tests/test_docs_funasr_install_commands.py @@ -341,8 +341,8 @@ def test_top_level_readmes_surface_current_release_and_edge_runtime(): } for name, text in readmes.items(): - assert 'python -m pip install -U "funasr==1.4.12"' in text, name - assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.12" in text, name + assert 'python -m pip install -U "funasr==1.4.13"' in text, name + assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.13" in text, name assert "runtime-llamacpp-v0.2.6" in text, name assert "https://www.funasr.com/en/deploy/llama-cpp.html" in readmes["README.md"]