Prefer matching discrete GPU devices while retaining a matching integrated GPU as fallback. Tested with the pinned llama.cpp revision in CPU and Vulkan builds; this addresses the iGPU selector bug without claiming to fix the separate RX 9070 XT backend crash.
Refresh the pinned llama.cpp revision with AMD Vulkan submission batching and DeviceLost diagnostics, adapt the sampler API, and document Windows troubleshooting. Refs #3479.
* feat(runtime): add Windows Vulkan release asset
* fix(ci): install SPIR-V headers for Windows Vulkan
* fix(runtime): make Windows Vulkan package standalone
Document the current sm_86 Windows CUDA prebuilt limit and direct RTX 50 / Blackwell users to CPU binaries or source builds with CMAKE_CUDA_ARCHITECTURES=120 until a dedicated asset is available.
Adds a Windows x64 CUDA release asset for the llama.cpp SenseVoice runtime, with CUDA Toolkit setup, architecture-86 build flags, focused SenseVoice target packaging, docs, and workflow regression tests. Manual workflow dispatch 29656771776 passed all build matrix jobs including build-windows-x64-cuda.
SenseVoice and Paraformer binaries now output text directly (whisper-cli style),
no Python detok step.
- export_{sensevoice,paraformer}_gguf.py: embed the vocab as a GGUF string array
(sv.vocab = SentencePiece pieces; pf.vocab = tokens.json) so the binary is self-contained.
- funasr-sensevoice.cpp: CTC ids -> text (pieces + ▁->space), drop <|...|> meta by
default; --ids (raw), --keep-tags (keep meta tags).
- funasr-paraformer.cpp: ids -> text (tokens.json, drop @@ continuations); --ids for raw.
- both fall back to ids if the GGUF has no vocab. READMEs updated to text output.
Verified: C++ text == reference detok (sentencepiece / tokens.json) on sample and the
full 002 clip; ids unchanged so CER is unchanged.
* test: numerical regression harness (frozen golden vs ggml/VAD/CIF/CTC output)
Adds tests/ — runs each runtime tool on a fixed 6 s clip and diffs against frozen
golden output, catching regressions in the ggml graphs, the FSMN-VAD state machine,
the CIF predictor and CTC decode.
- tests/run_regression.sh: auto-detects which tools are built; VAD model is auto-fetched
(1.7 MB), ASR GGUFs tested when present or with RUN_FULL=1 (downloads from HF).
Non-zero exit on any mismatch. BIN_DIR/MODELS_DIR overridable.
- tests/sample.wav (~6 s) + tests/golden/*.txt: golden captured on Linux x86-64 with the
f16 GGUFs from FunAudioLLM/*-GGUF.
Verified locally: all present tools PASS; default mode fetches VAD + skips absent models.
* test: golden as text (binaries now detok in-process; merge after in-binary-detok)
* test: fix sensevoice golden (correct transcription text)
- download-funasr-model.sh: detect `hf` (new CLI) or fall back to `huggingface-cli`
(older huggingface_hub); fail with an install hint if neither exists.
- convert-funasr-to-gguf.py: catch ModuleNotFoundError and print how to install the
missing package instead of a raw traceback.
Make the runtime a downloadable product (whisper.cpp-style), no manual export needed:
- download-funasr-model.sh: one command to pull pre-converted GGUF from Hugging Face
(FunAudioLLM/*-GGUF), incl. fsmn-vad for built-in --vad. No Python ML env required.
- convert-funasr-to-gguf.py: one-step HF/ModelScope checkpoint -> GGUF (wraps the
validated export_*.py; discovered relative to itself so it works in any layout).
- CMakeLists.txt: standalone, CI-friendly top-level build. FetchContent pins llama.cpp
(provides ggml + llama), static libs -> self-contained binaries in build/bin/, no
hardcoded paths; builds on Linux/macOS/Windows x64/arm64. `cmake -B build && cmake --build build -j`.
- README: download-first + standalone-build quickstart; links to the HF GGUF repos.
Verified on Linux: all targets build via FetchContent (static, no shared-lib deps);
hf download -> run reproduces canonical output. Additive; runtime/llama.cpp/ only.
- funasr_vad.h: define M_PI fallback (not guaranteed on MSVC) for cross-platform CI;
guard against audio shorter than one frame (empty fbank -> OOB in lfr);
validate required GGUF tensors up front (fail with a message, not a segfault);
shrink the no_alloc metadata context (512MB -> 16MB); drop a redundant ggml_cont
on an already-contiguous row slice in the FSMN memory loop.
- export_vad_gguf.py: use a with-statement when reading am.mvn (no fd leak).
No behavior change: VAD segment boundaries are byte-identical on the 184-clip set.
Add a native ggml FSMN-VAD front end so the binaries segment long audio themselves,
with no Python at runtime. New `--vad fsmn-vad.gguf` flag on llama-funasr-cli,
llama-funasr-sensevoice and llama-funasr-paraformer runs FSMN-VAD (fbank + LFR + CMVN +
FSMN encoder + E2EVadModel state machine, in funasr-common/funasr_vad.h) and decodes
each speech segment, then concatenates.
- funasr-common/funasr_vad.h: single-header VAD; segment boundaries within ~10 ms of
PyTorch fsmn-vad.generate on the 184-clip set (DEFAULT_SILENCE_SCHEDULE replicated,
chunk-stepped silence schedule + speech_noise_thres 0.5).
- funasr-vad/: standalone VAD tool (wav -> [start_ms,end_ms] segments).
- export_vad_gguf.py: export FSMN-VAD encoder + CMVN to GGUF (1.7 MB, 24 tensors).
- BENCHMARKS.md: bare binary with built-in VAD reaches the reference end-to-end —
full-184 micro-CER SenseVoice 8.01 / Paraformer 9.85 / Fun-ASR-Nano 8.30.
Additive only: no existing code is modified outside runtime/llama.cpp/.
* feat(runtime): accept any audio (any sample-rate/channels, wav/mp3/flac) via miniaudio
Replaces the 16k-mono-PCM16-only WAV reader with a miniaudio-based loader
(funasr-common/funasr_audio.h) that decodes wav/mp3/flac at any sample rate and
channel count, resampling to 16 kHz mono. Verified: a 44.1k stereo file and an 8k
mono file both transcribe correctly across cli/sensevoice/paraformer.
* fix(audio): inline loader (avoid duplicate symbols), null-path guard, check decoder read result (gemini review)
---------
Co-authored-by: LauraGPT <[email protected]>
LauraGPT