modelscope/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR.git synced 2026-10-12 02:19:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,986 Commits 468 Branches 66 Tags
main
Commit Graph
7 Commits
Author SHA1 Message Date
Dingyuan Wang b0d44bd792 llama.cpp: add srt output (#3480) 
Add optional SRT output to Fun-ASR-Nano, SenseVoiceSmall, and Paraformer llama.cpp runtimes. Preserve existing non-SRT output behavior and document VAD-aligned subtitle generation.
 2026-08-11 11:27:36 +08:00
zhifu gao 9f1d220432 feat: q8_0 export option (--wtype q8_0) for SenseVoice/Paraformer/Nano encoder GGUF (#3013) 2026-06-22 02:39:32 +08:00
zhifu gao 9cc0ebcc66 fix: detok review findings (null gguf_get_arr_str guard + utf-8 tokens.json read) (#3005) 2026-06-20 17:39:32 +08:00
zhifu gao 807bcd5647 feat: print transcription text in the binaries (embed vocab, in-binary detok) (#3004) 
SenseVoice and Paraformer binaries now output text directly (whisper-cli style),
no Python detok step.

- export_{sensevoice,paraformer}_gguf.py: embed the vocab as a GGUF string array
  (sv.vocab = SentencePiece pieces; pf.vocab = tokens.json) so the binary is self-contained.
- funasr-sensevoice.cpp: CTC ids -> text (pieces + ▁->space), drop <|...|> meta by
  default; --ids (raw), --keep-tags (keep meta tags).
- funasr-paraformer.cpp: ids -> text (tokens.json, drop @@ continuations); --ids for raw.
- both fall back to ids if the GGUF has no vocab. READMEs updated to text output.

Verified: C++ text == reference detok (sentencepiece / tokens.json) on sample and the
full 002 clip; ids unchanged so CER is unchanged.
 2026-06-20 16:39:25 +08:00
zhifu gao e615ed0dc0 feat: built-in FSMN-VAD (--vad) for all three runtimes — single-binary segmentation (#2998) 
Add a native ggml FSMN-VAD front end so the binaries segment long audio themselves,
with no Python at runtime. New `--vad fsmn-vad.gguf` flag on llama-funasr-cli,
llama-funasr-sensevoice and llama-funasr-paraformer runs FSMN-VAD (fbank + LFR + CMVN +
FSMN encoder + E2EVadModel state machine, in funasr-common/funasr_vad.h) and decodes
each speech segment, then concatenates.

- funasr-common/funasr_vad.h: single-header VAD; segment boundaries within ~10 ms of
  PyTorch fsmn-vad.generate on the 184-clip set (DEFAULT_SILENCE_SCHEDULE replicated,
  chunk-stepped silence schedule + speech_noise_thres 0.5).
- funasr-vad/: standalone VAD tool (wav -> [start_ms,end_ms] segments).
- export_vad_gguf.py: export FSMN-VAD encoder + CMVN to GGUF (1.7 MB, 24 tensors).
- BENCHMARKS.md: bare binary with built-in VAD reaches the reference end-to-end —
  full-184 micro-CER SenseVoice 8.01 / Paraformer 9.85 / Fun-ASR-Nano 8.30.

Additive only: no existing code is modified outside runtime/llama.cpp/.
 2026-06-20 13:26:27 +08:00
zhifu gaoandLauraGPT a83e1d9c38 feat: accept any audio input (any rate/channels, wav/mp3/flac) via miniaudio (#2994) 
* feat(runtime): accept any audio (any sample-rate/channels, wav/mp3/flac) via miniaudio

Replaces the 16k-mono-PCM16-only WAV reader with a miniaudio-based loader
(funasr-common/funasr_audio.h) that decodes wav/mp3/flac at any sample rate and
channel count, resampling to 16 kHz mono. Verified: a 44.1k stereo file and an 8k
mono file both transcribe correctly across cli/sensevoice/paraformer.

* fix(audio): inline loader (avoid duplicate symbols), null-path guard, check decoder read result (gemini review)

---------

Co-authored-by: LauraGPT <[email protected]>
 2026-06-19 22:40:02 +08:00
zhifu gaoandLauraGPT b7da5cc706 Add llama.cpp / GGUF runtime for FunASR models (#2988) 
* Add llama.cpp/GGUF runtime for Fun-ASR-Nano, SenseVoice and Paraformer

Run FunASR models entirely on the llama.cpp / ggml stack: CPU / edge, single
binary, no Python at runtime, quantized GGUF weights (like whisper.cpp for
Whisper). Adds runtime/llama.cpp/ with three models sharing a common ggml SAN-M
encoder / FSMN / attention and kaldi fbank front end:

- fun-asr-nano : SAN-M encoder + adaptor + Qwen3-0.6B LLM (audio embeds injected
  via llama_decode embedding input, llava/mtmd style); integrated WAV->text CLI.
- sensevoice   : SAN-M encoder + CTC; WAV->CTC ids.
- paraformer   : SAN-M encoder + CIF predictor + SAN-M decoder (non-autoregressive).

Each ships a GGUF export script and a README (architecture, quickstart, accuracy,
gotchas). Validated against the PyTorch reference (encoder cosine ~1.0; SenseVoice
CTC ids identical; Paraformer text identical; Fun-ASR-Nano CER within 0.02% under
identical conditions). Adds a new directory only; no existing code is modified.

* docs: add comprehensive design document (architecture, GGUF format, validation, gotchas)

* docs: link README to DESIGN.md

* fix: address code review (WAV ch/bits validation, chunk guard, null/leak checks)

Fixes gemini-code-assist findings: validate channels/bits in the WAV reader
(reject non-16-bit/zero-channel instead of dividing by zero), guard chunk size
>=1, null-check fopen/llama_context, and fclose/ggml_free on all paths.

---------

Co-authored-by: LauraGPT <[email protected]>
 2026-06-19 12:06:42 +08:00