Fail closed for native, remote-code, and vLLM Fun-ASR-Nano timestamp inference when checkpoint CTC weights are missing, partial, shape-mismatched, DDP-prefixed, or wrapped.
Fixes#3208.
Preserve supported accelerator device types, including npu, when choosing Fun-ASR-Nano autocast device type. This prevents npu devices from falling back to CPU autocast before real torch_npu execution can be validated.
Add a Qwen3-ASR streaming WebSocket server example, bilingual notes about VAD/STOP behavior, vLLM version constraints, model download guidance, and smoke tests for the example docs and exception logging.
The realtime WS server's interim (partial) path re-encodes the entire growing
speech segment from its start on every chunk, so a length-L segment costs O(L^2)
total re-encoding. Under high concurrency this saturates the GPU and long-segment
requests time out (e.g. a 17-29s utterance failing at 48 concurrent streams).
Cap the interim re-decode to the most recent `--partial-window-sec` seconds
(default 15; lower to 8-10 for high concurrency). Final transcripts are unchanged
- completed VAD segments are still decoded in full by _decode_segment / the
is_final path; only the interim hypothesis is bounded.
Root cause and validation by @qiulang on issue #3006: he reproduced the failure
on an L20 (long audio, 48 concurrent streams), confirmed bounding the window
restores most of the concurrency, and pinpointed that the growing window (not
partial frequency) is the dominant load for this LLM-ASR. Thanks!
Co-authored-by: LauraGPT <[email protected]>
FsmnVADStreaming.forward() is inference-only (no loss computation),
so standard train_ds.py training does not work. Removing the
finetune.sh and docs that were incorrectly added.
The parameter tuning guide (vad_kwargs) remains valid and is
documented in the issue thread (#2961).
Allow custom local paths or alternative model names for VAD and
speaker models. Set --spk-model to empty string to disable speaker
model loading. Fixes#2964.
- Fix TypeError when scheduler_conf is not provided (NoneType unpacking)
- Add explicit scheduler and scheduler_conf to VAD finetune.sh
- Fixes user report in #2961
- Add finetune.sh for FSMN-VAD model fine-tuning
- Add English and Chinese documentation for VAD fine-tuning
- Include parameter tuning guide for quick adjustments without fine-tuning
- Addresses user requests in #2961
- repetition_penalty is incompatible with enable_prompt_embeds=True in vLLM
- Removed from both serve_realtime_ws.py and serve_vllm.py
- Added truncate_repetition() post-processing to serve_vllm.py as alternative
- serve_realtime_ws.py already has detect_and_fix_hallucination() for this
Root cause: vLLM's repetition penalty implementation does scatter on prompt
token IDs, but prompt_embeds mode has no token IDs, causing index OOB.
- Implemented cross-entropy loss computation in WhisperWarp.forward()
- Added fine-tuning example script and documentation
- Users can now fine-tune any Whisper model on custom data via FunASR training framework
- Merged serve_vllm.py logic into funasr-server (_server_app.py)
- Fun-ASR-Nano now uses FunASRNanoVLLM with repetition_penalty=1.3
- Falls back to AutoModel if vLLM unavailable (e.g. CPU or driver mismatch)
- Added /asr endpoint with VAD + timestamps + SPK support
- Non-LLM models (sensevoice, paraformer) still use AutoModel
- Verified: 120.wav repetition bug completely fixed
- Bump version to 1.3.6
- Single script supports all models (Fun-ASR-Nano, GLM-ASR, AutoModelVLLM)
- Remove old model-specific benchmark from examples/
- Update docs with unified benchmark commands
Root cause: Fun-ASR-Nano uses use_low_frame_rate=True which computes
effective audio token count from fbank length (3x stride-2 conv formula).
The vLLM implementation was missing this step, sending 86 tokens where
PyTorch only uses 11 - causing CER to degrade from 8.94% to 19.68%.
Fix: Apply the same formula in both _encode_audio() and batch encode path.
Also removed <think> tokens from prompt suffix (not needed, model generates
directly without thinking for this task).
Results (184 files, 11541s, 1947 VAD segments):
PyTorch: RTFx=19.0, CER=8.94%
vLLM: RTFx=393.9, CER=8.91% (20.7x faster, CER consistent)
Also added benchmark_vllm.py for reproducible evaluation.
zhifu gao