docs: clarify Fun-ASR-Nano vLLM precision boundary

This commit is contained in:
LauraGPT
2026-09-02 07:21:20 +00:00
parent f2332313ac
commit 331369fd64
3 changed files with 15 additions and 2 deletions
+7
View File
@@ -872,6 +872,13 @@ Streaming: `serve_realtime_ws.RealtimeASRSession`
**Q: Slow first startup?**
vLLM initialization takes 60–90 s (KV Cache + CUDA Graph warmup). Subsequent inferences are instant.
**Q: What happens when Fun-ASR-Nano vLLM uses `dtype="fp16"`?**
The audio frontend and adaptor remain float16, but FunASR runs the Qwen3 decoder
in bfloat16 because vLLM float16 decoding can produce degraded repeated output.
This is automatic and keeps the two-byte decoder weight footprint. On hardware
without BF16 support, use `dtype="fp32"`; the vLLM path does not claim an
end-to-end FP16 decoder mode.
**Q: vLLM returns repeated punctuation such as `!!!!!!!!` but PyTorch/HF generate is normal. What should I check?**
This usually means the audio frontend and checkpoint can work, but the vLLM
prompt-embedding path or decoding parameters differ from the upstream runner.
+6
View File
@@ -866,6 +866,12 @@ RTFx 102 → 46。CER 不变。默认关闭。
**Q: 首次慢?**
vLLM 初始化 60-90s,之后即时。
**Q: Fun-ASR-Nano vLLM 使用 `dtype="fp16"` 时实际会怎样?**
音频 frontend 与 adaptor 仍使用 FP16,但 FunASR 会让 Qwen3 decoder 使用 BF16,
因为 vLLM 的 FP16 decoder 可能产生退化的重复输出。该行为自动生效,decoder 权重
仍是两字节精度;不支持 BF16 的硬件请使用 `dtype="fp32"`。vLLM 路径不宣称支持
端到端 FP16 decoder。
**Q: vLLM 输出连续标点(例如 `!!!!!!!!`),但 PyTorch/HF generate 正常,应该先查什么?**
这通常说明音频 frontend 和 checkpoint 本身能工作,但 vLLM prompt-embedding
路径或解码参数和 upstream runner 不一致。改模型前先检查这些项:
+2 -2
View File
@@ -341,8 +341,8 @@ def test_top_level_readmes_surface_current_release_and_edge_runtime():
}
for name, text in readmes.items():
assert 'python -m pip install -U "funasr==1.4.12"' in text, name
assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.12" in text, name
assert 'python -m pip install -U "funasr==1.4.13"' in text, name
assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.13" in text, name
assert "runtime-llamacpp-v0.2.6" in text, name
assert "https://www.funasr.com/en/deploy/llama-cpp.html" in readmes["README.md"]