modelscope/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR.git synced 2026-10-12 02:19:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(audio): support explicit PCM16 input without container probing

Browse Source 
Signed-off-by: LauraGPT <[email protected]>
This commit is contained in:
LauraGPT
2026-10-01 15:17:23 +00:00
parent 4a84879ae8
commit 9d611293f4
7 changed files with 309 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/python_api.md
+9 -1
View File
@@ -104,7 +104,15 @@ for result in results:
`batch_size=1` processes a list serially through direct inference; raise it only for a model that supports batching and after measuring memory use. `batch_size_s` has a different meaning and only controls the wrapper's VAD path. VAD segmentation is not incremental microphone streaming, and it does not guarantee unlimited recording length or bounded total memory.
Use a one-dimensional mono float32 waveform with a known sample rate. For models using the common audio loader, `fs` in `generate()` describes the array's source sample rate (fallback 16000); the model frontend supplies the target rate. Do not pass a stereo matrix or a list of numbers and assume it will be treated as one waveform: a Python list is normally a list of inputs. File decoding depends on the installed audio backends. Top-level bytes are handled by `load_bytes`: recognized containers are decoded/resampled to 16 kHz, otherwise bytes are interpreted as int16 PCM without sample-rate metadata. Prefer a decoded array with an explicit rate for ambiguous inputs. See [audio loading](../funasr/utils/load_utils.py) and [byte-input tests](../tests/test_load_audio_bytes.py).
Use a one-dimensional mono float32 waveform with a known sample rate. For models using the common audio loader, `fs` in `generate()` describes the array's source sample rate (fallback 16000); the model frontend supplies the target rate. Do not pass a stereo matrix or a list of numbers and assume it will be treated as one waveform: a Python list is normally a list of inputs. File decoding depends on the installed audio backends. With the default `input_format="auto"`, top-level bytes are handled by `load_bytes`: recognized containers are decoded/resampled to 16 kHz, otherwise bytes are interpreted as int16 PCM without sample-rate metadata. Prefer a decoded array with an explicit rate for ambiguous inputs. See [audio loading](../funasr/utils/load_utils.py) and [byte-input tests](../tests/test_load_audio_bytes.py).
For known **mono signed 16-bit little-endian raw PCM**, explicitly bypass format detection:
```python
result = model.generate(input=pcm_bytes, input_format="pcm_s16le", fs=16000)
```
This option also applies to byte strings in a batch and to the VAD preprocessing path. It normalizes samples to float32 without invoking a container decoder; raw bytes can otherwise accidentally resemble a file header. It does not resample at byte-conversion time: use the actual source sample rate in `fs`. Do not select this format for WAV/MP3 file bytes, float PCM, other bit depths, big-endian samples, or interleaved stereo. A partial two-byte sample raises `ValueError`; assemble complete samples in the receiving layer. Streaming callers must still preserve their per-session `cache`, chunk settings, and final-chunk handling. This opt-in path avoids ambiguous probing, but does not diagnose every source of inference delays.
`prepare_data_iterator` also accepts manifests. A `.scp` line can be `utterance_id /path/to/audio.wav`; `.jsonl` records use `{"source": "/path/to/audio.wav", "key": "utterance_id"}`. `.json` is not parsed as an arbitrary JSON array by this iterator. Do not expose unrestricted paths or URLs directly to untrusted callers.