* feat(nano): support LoRA finetuning of the Qwen3 LLM
The FunASR-Nano model config already declares llm_conf.use_lora and
llm_conf.lora_conf (r/lora_alpha/lora_dropout/target_modules), but nothing
consumed them: use_lora was silently ignored and training fell back to the
full-parameter LLM path. Wire it up:
- When llm_conf.use_lora is true, replace the LLM target Linear layers
(default q_proj/v_proj) with lora.Linear adapters, sharing the frozen base
weight and adding trainable lora_A/lora_B. The adapter params are created in
the base weight's dtype (bf16), so the LoRA path does not depend on autocast.
- Pairs with the existing lora_only / mark_only_lora_as_trainable flow in
train.py / train_ds.py: lora_only=true freezes every non-LoRA parameter for
pure-LoRA training; lora_only=false + unfrozen encoder/adaptor conf keeps
them trainable while LoRA-tweaking only the LLM.
- Checkpoints save the full state dict (unchanged base + adapter params), so a
LoRA run can be resumed or decoded with the same use_lora=true config; for a
standalone deployment checkpoint, fold with W' = W + alpha/r * B @ A.
- Add lora_finetune.sh and document the recipe in finetune.md / finetune_zh.md.
Validated end-to-end with the real model: 56 adapters injected (28 Qwen3
layers x q_proj+v_proj), optimizer updates only the 112 LoRA params, and a
2-step smoke training run through funasr-train-ds completes and saves a
checkpoint whose lora keys are bf16 and match the base dtype.
Co-Authored-By: Claude <[email protected]>
* test(nano): add offline unit tests for LoRA injection
Drive FunASRNano._apply_lora_to_llm against a tiny fake LLM mirroring the
Qwen3 parameter layout (layers.<i>.self_attn.{q,k,v,o}_proj / mlp.{gate,up,down}_proj).
No model download needed.
Covers the points requested in review:
1. only configured target modules are replaced;
2. the initial output matches the original linear layers (lora_B init zero);
3. base weights stay frozen while lora_A/lora_B are trainable (llm_conf.freeze
applied before injection, as in the real init path);
4. a non-zero adapter survives a state-dict save/load round trip and train/eval
transitions (including the eval-merge / train-unmerge convention);
5. the no-matching-target configuration is an explicit no-op with a warning.
Verified honest: on upstream main (no _apply_lora_to_llm) all 7 fail with
AttributeError; with the fix 7/7 pass. Full nano set: 34 passed.
Co-Authored-By: Claude <[email protected]>
---------
Co-authored-by: Claude <[email protected]>
Fail closed for native, remote-code, and vLLM Fun-ASR-Nano timestamp inference when checkpoint CTC weights are missing, partial, shape-mismatched, DDP-prefixed, or wrapped.
Fixes#3208.
Preserve supported accelerator device types, including npu, when choosing Fun-ASR-Nano autocast device type. This prevents npu devices from falling back to CPU autocast before real torch_npu execution can be validated.
Add a Qwen3-ASR streaming WebSocket server example, bilingual notes about VAD/STOP behavior, vLLM version constraints, model download guidance, and smoke tests for the example docs and exception logging.
The realtime WS server's interim (partial) path re-encodes the entire growing
speech segment from its start on every chunk, so a length-L segment costs O(L^2)
total re-encoding. Under high concurrency this saturates the GPU and long-segment
requests time out (e.g. a 17-29s utterance failing at 48 concurrent streams).
Cap the interim re-decode to the most recent `--partial-window-sec` seconds
(default 15; lower to 8-10 for high concurrency). Final transcripts are unchanged
- completed VAD segments are still decoded in full by _decode_segment / the
is_final path; only the interim hypothesis is bounded.
Root cause and validation by @qiulang on issue #3006: he reproduced the failure
on an L20 (long audio, 48 concurrent streams), confirmed bounding the window
restores most of the concurrency, and pinpointed that the growing window (not
partial frequency) is the dominant load for this LLM-ASR. Thanks!
Co-authored-by: LauraGPT <[email protected]>
LauraGPT