Correct the single-character typo without changing training commands or links.
Co-authored-by: baofeidyz <[email protected]>
Signed-off-by: zhifu gao <[email protected]>
* feat(nano): support LoRA finetuning of the Qwen3 LLM
The FunASR-Nano model config already declares llm_conf.use_lora and
llm_conf.lora_conf (r/lora_alpha/lora_dropout/target_modules), but nothing
consumed them: use_lora was silently ignored and training fell back to the
full-parameter LLM path. Wire it up:
- When llm_conf.use_lora is true, replace the LLM target Linear layers
(default q_proj/v_proj) with lora.Linear adapters, sharing the frozen base
weight and adding trainable lora_A/lora_B. The adapter params are created in
the base weight's dtype (bf16), so the LoRA path does not depend on autocast.
- Pairs with the existing lora_only / mark_only_lora_as_trainable flow in
train.py / train_ds.py: lora_only=true freezes every non-LoRA parameter for
pure-LoRA training; lora_only=false + unfrozen encoder/adaptor conf keeps
them trainable while LoRA-tweaking only the LLM.
- Checkpoints save the full state dict (unchanged base + adapter params), so a
LoRA run can be resumed or decoded with the same use_lora=true config; for a
standalone deployment checkpoint, fold with W' = W + alpha/r * B @ A.
- Add lora_finetune.sh and document the recipe in finetune.md / finetune_zh.md.
Validated end-to-end with the real model: 56 adapters injected (28 Qwen3
layers x q_proj+v_proj), optimizer updates only the 112 LoRA params, and a
2-step smoke training run through funasr-train-ds completes and saves a
checkpoint whose lora keys are bf16 and match the base dtype.
Co-Authored-By: Claude <[email protected]>
* test(nano): add offline unit tests for LoRA injection
Drive FunASRNano._apply_lora_to_llm against a tiny fake LLM mirroring the
Qwen3 parameter layout (layers.<i>.self_attn.{q,k,v,o}_proj / mlp.{gate,up,down}_proj).
No model download needed.
Covers the points requested in review:
1. only configured target modules are replaced;
2. the initial output matches the original linear layers (lora_B init zero);
3. base weights stay frozen while lora_A/lora_B are trainable (llm_conf.freeze
applied before injection, as in the real init path);
4. a non-zero adapter survives a state-dict save/load round trip and train/eval
transitions (including the eval-merge / train-unmerge convention);
5. the no-matching-target configuration is an explicit no-op with a warning.
Verified honest: on upstream main (no _apply_lora_to_llm) all 7 fail with
AttributeError; with the fix 7/7 pass. Full nano set: 34 passed.
Co-Authored-By: Claude <[email protected]>
---------
Co-authored-by: Claude <[email protected]>
Fail closed for native, remote-code, and vLLM Fun-ASR-Nano timestamp inference when checkpoint CTC weights are missing, partial, shape-mismatched, DDP-prefixed, or wrapped.
Fixes#3208.
LauraGPT