mirror of
https://github.com/modelscope/FunASR.git
synced 2026-10-12 02:19:18 +00:00
docs: clarify Fun-ASR-Nano vLLM precision boundary
This commit is contained in:
@@ -872,6 +872,13 @@ Streaming: `serve_realtime_ws.RealtimeASRSession`
|
||||
**Q: Slow first startup?**
|
||||
vLLM initialization takes 60–90 s (KV Cache + CUDA Graph warmup). Subsequent inferences are instant.
|
||||
|
||||
**Q: What happens when Fun-ASR-Nano vLLM uses `dtype="fp16"`?**
|
||||
The audio frontend and adaptor remain float16, but FunASR runs the Qwen3 decoder
|
||||
in bfloat16 because vLLM float16 decoding can produce degraded repeated output.
|
||||
This is automatic and keeps the two-byte decoder weight footprint. On hardware
|
||||
without BF16 support, use `dtype="fp32"`; the vLLM path does not claim an
|
||||
end-to-end FP16 decoder mode.
|
||||
|
||||
**Q: vLLM returns repeated punctuation such as `!!!!!!!!` but PyTorch/HF generate is normal. What should I check?**
|
||||
This usually means the audio frontend and checkpoint can work, but the vLLM
|
||||
prompt-embedding path or decoding parameters differ from the upstream runner.
|
||||
|
||||
@@ -866,6 +866,12 @@ RTFx 102 → 46。CER 不变。默认关闭。
|
||||
**Q: 首次慢?**
|
||||
vLLM 初始化 60-90s,之后即时。
|
||||
|
||||
**Q: Fun-ASR-Nano vLLM 使用 `dtype="fp16"` 时实际会怎样?**
|
||||
音频 frontend 与 adaptor 仍使用 FP16,但 FunASR 会让 Qwen3 decoder 使用 BF16,
|
||||
因为 vLLM 的 FP16 decoder 可能产生退化的重复输出。该行为自动生效,decoder 权重
|
||||
仍是两字节精度;不支持 BF16 的硬件请使用 `dtype="fp32"`。vLLM 路径不宣称支持
|
||||
端到端 FP16 decoder。
|
||||
|
||||
**Q: vLLM 输出连续标点(例如 `!!!!!!!!`),但 PyTorch/HF generate 正常,应该先查什么?**
|
||||
这通常说明音频 frontend 和 checkpoint 本身能工作,但 vLLM prompt-embedding
|
||||
路径或解码参数和 upstream runner 不一致。改模型前先检查这些项:
|
||||
|
||||
@@ -341,8 +341,8 @@ def test_top_level_readmes_surface_current_release_and_edge_runtime():
|
||||
}
|
||||
|
||||
for name, text in readmes.items():
|
||||
assert 'python -m pip install -U "funasr==1.4.12"' in text, name
|
||||
assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.12" in text, name
|
||||
assert 'python -m pip install -U "funasr==1.4.13"' in text, name
|
||||
assert "https://github.com/modelscope/FunASR/releases/tag/v1.4.13" in text, name
|
||||
assert "runtime-llamacpp-v0.2.6" in text, name
|
||||
|
||||
assert "https://www.funasr.com/en/deploy/llama-cpp.html" in readmes["README.md"]
|
||||
|
||||
Reference in New Issue
Block a user