modelscope/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR.git synced 2026-10-12 02:19:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: honor submodel device placement across inference calls

Browse Source
This commit is contained in:
damnwenxi
2026-09-14 09:20:40 +08:00
parent 486b4b7ceb
commit aa353d6d22
6 changed files with 234 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/python_api.md
+25 -1
View File
@@ -23,7 +23,7 @@ The wrapper accepts `**kwargs`, not a universally validated parameter schema. A
| `ngpu` | `1` | Zero selects CPU. This is not a multi-GPU serving or sharding configuration. |
| `ncpu` | `4` | Positive CPU thread count; invalid values use the fallback and values below 1 clamp to 1. Sets process-wide PyTorch threads. |
| `vad_model`, `punc_model`, `spk_model` | `None` | Optional components, built once. Use `vad_model`, not the rejected typo `vda_model`. |
| `vad_kwargs`, `punc_kwargs`, `spk_kwargs` | `{}` | Component configuration dictionaries. Device is inherited; hub and CPU threads are inherited unless provided in the component dictionary. |
| `vad_kwargs`, `punc_kwargs`, `spk_kwargs` | `{}` | Component configuration dictionaries. Device, hub and CPU threads are inherited unless explicitly provided in the component dictionary. Device inherits the resolved ASR device (after fallback). Caller dictionaries are copied before defaults are applied. |
| `vad_model_revision`, `punc_model_revision`, `spk_model_revision` | `"master"` each | Separate component revisions, not inherited from the ASR revision. These top-level values replace `model_revision` in the component dictionaries. |
| `spk_mode` | `"punc_segment"` | Use `"punc_segment"` or `"vad_segment"`. The validation also accepts legacy `"default"`, but the sentence-building branches do not implement it; do not select it. |
| `disable_update` | `False` | Disables the FunASR package version check only, not model downloading. |
@@ -51,6 +51,30 @@ The wrapper accepts `**kwargs`, not a universally validated parameter schema. A
`generate()` restores saved construction configuration before merging call options. Repeat request-specific options on each call; do not rely on the previous call's language, batch settings, or hotwords. This configuration reset does not establish thread safety or reset every model attribute: for example, a speaker-mode fallback mutates `self.spk_mode`. Serialize access to a shared instance unless your own concurrency tests establish otherwise.
### Place submodels on separate devices
For example, keep FSMN VAD and punctuation on CPU while ASR uses Apple Silicon MPS:
```python
model = AutoModel(
model="paraformer-zh",
device="mps",
vad_model="fsmn-vad",
vad_kwargs={"device": "cpu"},
punc_model="ct-punc",
punc_kwargs={"device": "cpu"},
)
results = model.generate(input="audio.wav")
```
`spk_kwargs={"device": "cpu"}` works the same way when a speaker model is configured.
Each model resolves unavailable-device fallback independently at construction.
The selected devices remain fixed across calls, including when shared runtime
options are merged. A `device` passed to `generate()` is ignored for placement;
it does not move loaded weights. Construct another `AutoModel` to change devices.
Other runtime options, such as `hotword` and `batch_size_s`, continue to be merged.
No internal model replacement or `ComputeScores` patch is needed.
## Local Files and Batching
Install the SDK using the [installation guide](installation/installation.md). The three examples below are standalone scripts with command-line arguments and require prepared local model snapshots and real audio. Their syntax and wrapper contracts are checked without downloading weights; they are not reported as full inference tests.