- README.md: add vLLM/dynamic VAD to What's new, add vLLM row to features
- README_zh.md: same updates in Chinese
- Fun-ASR-Nano README/README_zh: add built-in vLLM and WebSocket service links
Features:
- AutoModelVLLM: generic vLLM inference for all LLM-based ASR models
- serve_realtime_ws.py: streaming WebSocket ASR with VAD+SPK+hotwords
- DynamicStreamingVAD: dynamic silence threshold (default on for all models)
- Fun-ASR-Nano batch_size fix for VAD pipeline
- Comprehensive vLLM documentation (docs/vllm_guide.md)
- Modify FsmnVADStreaming.inference() to support dynamic_silence=True (default)
- Non-streaming schedule: gradually tighten threshold as speech accumulates
- Streaming uses STREAMING_SILENCE_SCHEDULE via DynamicStreamingVAD wrapper
- Can disable with dynamic_silence=False or customize via silence_schedule param
- Tracks VAD state to only accumulate during active speech
- New funasr/utils/dynamic_vad.py: generic dynamic-threshold streaming VAD
- Supports both streaming (feed/finalize) and non-streaming (process) APIs
- Configurable silence_schedule for custom threshold curves
- Refactor serve_realtime_ws.py to use DynamicStreamingVAD
- Add documentation in vllm_guide.md appendix
- One-liner tagline: 170x faster than Whisper, 50+ languages
- Quick Start with human-readable output (speaker labels + timestamps)
- Why FunASR comparison table (vs Whisper vs Cloud)
- Merged GPU/CPU benchmark into one compact table
- Model Zoo with self-hosted + community models
- Detailed examples moved to docs site (modelscope.github.io/FunASR)
- Star History chart
- 538 lines → 210 lines
trainer.py now accepts use_bf16=True in train_conf. BF16 is preferred
over FP16 on modern GPUs (A100/H100) as it has wider dynamic range
and doesn't need loss scaling (GradScaler).
Usage: ++train_conf.use_bf16=true
Previously, all autoregressive models (Conformer, Transformer, Branchformer,
E-Branchformer) raised NotImplementedError for batch_size>1 inference,
forcing single-utterance decoding which is ~50x slower.
Now when batch_size>1, the model automatically uses CTC greedy decoding
(argmax + collapse repeats + remove blanks) which is fully parallelizable.
Performance on AISHELL-1 test (Conformer, 150 epochs):
- Attention decoder (batch=1): 4.92% CER, very slow
- CTC greedy batch (batch=32): 5.34% CER, ~50x faster
Users can still use batch_size=1 for attention decoder beam search when
they need the best accuracy.
Press hotkey → record → auto-transcribe → paste to cursor.
Works with funasr-server as backend. Supports macOS/Linux/Windows.
No secondary development needed - just pip install and run.
Enables AI tools to call FunASR speech recognition via Model Context Protocol.
Supports transcribe_audio tool with auto language detection and speaker diarization.
Drop-in replacement for /v1/audio/transcriptions endpoint.
Works with any agent framework (LangChain, AutoGen, etc).
Supports sensevoice, paraformer, and fun-asr-nano models.
Conformer, Transformer, and Branchformer models raise
NotImplementedError for batch decoding. Only Paraformer and
e_paraformer support batch_size>1 for inference. Revert the
non-Paraformer recipes back to batch_size=1.
* fix(onnxruntime): Fix Windows MSVC compilation errors
- Fix CMake syntax warning in src/CMakeLists.txt (message function quoting)
- Replace Python-style 'and' with C++ '&&' in ct-transformer-online.cpp
- Fix ssize_t redefinition conflict between kaldi and openfst on Windows
- Update yaml-cpp CMake policies from OLD to NEW for modern CMake compatibility
These changes enable successful compilation with MSVC on Windows (VS 2022)
while maintaining compatibility with other platforms.
Tested with: Visual Studio 2022, Windows 11, CMake 3.x
* Add realtime mic app and optional FFmpeg build
zhifu gao