mirror of
https://github.com/modelscope/FunASR.git
synced 2026-10-12 02:19:18 +00:00
vishnupratap3790
@@ -37,7 +37,7 @@ pip install torch torchaudio
|
||||
pip install funasr
|
||||
```
|
||||
|
||||
**Flagship model — Fun-ASR-Nano** (LLM-ASR for Chinese, English, and Japanese, including Chinese dialects; needs a GPU):
|
||||
**Flagship model — Fun-ASR-Nano** (LLM-ASR for Chinese, English, and Japanese, plus Chinese dialect groups and regional accents; needs a GPU):
|
||||
|
||||
```python
|
||||
from funasr import AutoModel
|
||||
@@ -50,7 +50,7 @@ print(result[0]["text"])
|
||||
|
||||
For the separate 31-language checkpoint, use
|
||||
[Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512).
|
||||
Language coverage is checkpoint-specific.
|
||||
Language coverage is checkpoint-specific, so Nano and MLT-Nano should be treated as distinct model choices.
|
||||
|
||||
On CPU (or for five-language ASR plus emotion and audio-event tags), use
|
||||
**SenseVoiceSmall**. The pipeline below composes SenseVoiceSmall with FSMN-VAD
|
||||
@@ -141,7 +141,7 @@ Requirements: Python ≥ 3.8. Install PyTorch + torchaudio first ([pytorch.org](
|
||||
|
||||
| Model | Task | Languages | Params | Links |
|
||||
|-------|------|-----------|--------|-------|
|
||||
| **Fun-ASR-Nano** | ASR | zh/en/ja + Chinese dialects | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) |
|
||||
| **Fun-ASR-Nano** | ASR | zh/en/ja + Chinese dialects and accents | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) |
|
||||
| **Fun-ASR-MLT-Nano** | ASR | 31 languages | 800M | [⭐](https://www.modelscope.cn/models/FunAudioLLM/Fun-ASR-MLT-Nano-2512) [🤗](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) |
|
||||
| **SenseVoiceSmall** | ASR + emotion + events | zh/en/ja/ko/yue | 234M | [⭐](https://www.modelscope.cn/models/iic/SenseVoiceSmall) [🤗](https://huggingface.co/FunAudioLLM/SenseVoiceSmall) |
|
||||
| **Paraformer-zh** | ASR + timestamps | zh/en | 220M | [⭐](https://www.modelscope.cn/models/iic/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch/summary) [🤗](https://huggingface.co/funasr/paraformer-zh) |
|
||||
|
||||
+1
-1
@@ -61,7 +61,7 @@ SenseVoiceSmall自体ではなく、CAM++によって提供されます。
|
||||
### なぜFunASRを選ぶのか？
|
||||
|
||||
Whisper は単一モデルですが、**FunASR はツールキット**です——用途に応じて
|
||||
**Fun-ASR-Nano**（中・英・日と中国語方言、GPU）、
|
||||
**Fun-ASR-Nano**（中・英・日と中国語方言・地域アクセント、GPU）、
|
||||
**Fun-ASR-MLT-Nano**（31言語）、**SenseVoiceSmall**（5言語ASRと感情・
|
||||
音声イベント）、**Paraformer**（低遅延ストリーミング）を選べます。
|
||||
下の表はツールキット全体の機能と、それを提供するモデルまたはパイプラインを示します：
|
||||
|
||||
+1
-1
@@ -61,7 +61,7 @@ CAM++에서 제공합니다.
|
||||
### 왜 FunASR인가?
|
||||
|
||||
Whisper는 단일 모델이지만, **FunASR는 툴킷**입니다. 용도에 맞게
|
||||
**Fun-ASR-Nano**(중국어/영어/일본어 및 중국어 방언, GPU),
|
||||
**Fun-ASR-Nano**(중국어/영어/일본어 및 중국어 방언/지역 억양, GPU),
|
||||
**Fun-ASR-MLT-Nano**(31개 언어), **SenseVoiceSmall**(5개 언어 ASR와
|
||||
감정·오디오 이벤트), **Paraformer**(저지연 스트리밍)를 선택하세요.
|
||||
아래 표는 툴킷 전체의 기능과 이를 제공하는 모델 또는 파이프라인을 보여 줍니다:
|
||||
|
||||
+1
-1
@@ -71,7 +71,7 @@ result = model.generate(input="https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics
|
||||
|
||||
需要 31 语种时，请改用独立的
|
||||
[Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512)
|
||||
checkpoint。不同 checkpoint 的语言范围不能混用。
|
||||
checkpoint。Nano 与 MLT-Nano 的语言范围不同，使用时请按 checkpoint 选择。
|
||||
|
||||
使用 vLLM 做高吞吐批处理：
|
||||
|
||||
|
||||
+1
-1
@@ -107,7 +107,7 @@ echo "$result" | jq -r '.text'
|
||||
| sensevoice | 50+ | ~70ms/10s | General use, multilingual |
|
||||
| paraformer | zh + mixed | ~60ms/10s | Chinese production (with punctuation) |
|
||||
| paraformer-en | en | ~60ms/10s | English |
|
||||
| fun-asr-nano | 31 | varies | Encoder+LLM, complex audio |
|
||||
| fun-asr-nano | zh/en/ja + Chinese dialects/accents | varies | Encoder+LLM, complex audio |
|
||||
|
||||
## Legacy CLI
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ Use this guide when you are choosing a first model, comparing FunASR with Whispe
|
||||
|
||||
## Fast default path
|
||||
|
||||
If you have a GPU, start with the flagship **Fun-ASR-Nano** — an LLM-based ASR model (SenseVoice encoder + a Qwen3 decoder) covering 31 languages, with the strongest accuracy on hard cases, context, and proper nouns:
|
||||
If you have a GPU, start with the flagship **Fun-ASR-Nano** — an LLM-based ASR model (SenseVoice encoder + a Qwen3 decoder) for Chinese, English, and Japanese, plus Chinese dialect groups and regional accents, with strong accuracy on hard cases, context, and proper nouns:
|
||||
|
||||
```python
|
||||
from funasr import AutoModel
|
||||
@@ -37,7 +37,7 @@ Switch to Paraformer when your workload is Mandarin-only and you want character-
|
||||
| Fast multilingual private transcription | SenseVoice-Small | Strong default with ASR, emotion tags, audio event tags, and CPU viability. | [README quick start](../README.md#quick-start) |
|
||||
| Mandarin production ASR | Paraformer-Large | Mature Chinese ASR path with VAD and punctuation. | [Tutorial](./tutorial/README.md) |
|
||||
| English-only route in the OpenAI API example | `paraformer-en` alias | Smaller English route for API compatibility checks. | [OpenAI API example](../examples/openai_api/) |
|
||||
| LLM-based ASR or 31-language experiments | Fun-ASR-Nano | LLM-based model path; use vLLM when decoder throughput matters. | [vLLM guide](./vllm_guide.md) |
|
||||
| LLM-based ASR or Chinese/English/Japanese + dialect experiments | Fun-ASR-Nano | LLM-based model path; use vLLM when decoder throughput matters. | [vLLM guide](./vllm_guide.md) |
|
||||
| Live captions or call-center streams | Runtime WebSocket service | Designed for long-lived streaming sessions and partial results. | [Runtime service docs](../runtime/readme.md) |
|
||||
| Batch archive processing | SenseVoice-Small or Paraformer-Large | Stable offline transcription path; caller owns manifests, retries, and logs. | [Batch ASR example](../examples/batch_asr_improved.py) |
|
||||
| Migration from Whisper/cloud ASR | SenseVoice-Small first, then benchmark alternatives | Gives a strong baseline before deeper model-specific tuning. | [Migration guide](./migration_from_whisper.md) |
|
||||
@@ -51,7 +51,7 @@ The `examples/openai_api` server exposes short aliases so application teams do n
|
||||
| `sensevoice` | `iic/SenseVoiceSmall` | You want the default private speech API with multilingual ASR, event tags, and good CPU/GPU behavior. |
|
||||
| `paraformer` | `paraformer-zh` with VAD and punctuation | You want a Mandarin-oriented production route. |
|
||||
| `paraformer-en` | `paraformer-en` with VAD | You want a compact English route in OpenAI-style clients. |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | You are evaluating LLM-based ASR, 31-language coverage, or vLLM acceleration. |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | You are evaluating LLM-based ASR for Chinese, English, Japanese, and Chinese dialect/accent coverage, or want to test vLLM acceleration. |
|
||||
|
||||
For Ascend NPU deployments, treat `fun-asr-nano` separately from SenseVoice / Paraformer. The Fun-ASR-Nano PyTorch `AutoModel` path has community compatibility evidence on 310P3 after the NPU autocast fix, but it was much slower than CPU in that smoke test; `AutoModelVLLM` still depends on vLLM-Ascend operator support and has hit Qwen3 rotary / `TransData` failures. Use CUDA/vLLM, standard PyTorch CPU/GPU, or GGUF runtime for production unless you are actively validating an Ascend backend.
|
||||
|
||||
@@ -91,7 +91,7 @@ For migration work, use the [migration benchmark example](../examples/migration/
|
||||
|
||||
## Practical recommendations
|
||||
|
||||
- With a GPU, default to Fun-ASR-Nano — the flagship LLM-based model (31 languages), strongest on hard, contextual, and proper-noun-heavy audio.
|
||||
- With a GPU, default to Fun-ASR-Nano — the flagship LLM-based model for Chinese, English, Japanese, and Chinese dialects/accents, strongest on hard, contextual, and proper-noun-heavy audio. For the separate 31-language checkpoint, use Fun-ASR-MLT-Nano.
|
||||
- On CPU, or for multilingual + emotion workloads, use SenseVoice-Small (fast non-autoregressive, CPU-viable).
|
||||
- Use Paraformer when your production traffic is primarily Mandarin and you want timestamps or hotwords.
|
||||
- Use the streaming runtime when partial results and long-lived connections matter more than a single final transcript.
|
||||
|
||||
@@ -27,7 +27,7 @@ result = model.generate(input="meeting.wav")
|
||||
| 高速な多言語プライベート文字起こし | SenseVoice-Small | ASR、感情タグ、音声イベントタグ、CPU/GPU の扱いやすさがそろった標準ルート。 | [README quick start](../README_ja.md#クイックスタート) |
|
||||
| 中国語中心の本番 ASR | Paraformer-Large | VAD と句読点復元を組み合わせた成熟した中国語 ASR ルート。 | [Tutorial](./tutorial/README.md) |
|
||||
| OpenAI API 例の英語ルート | `paraformer-en` alias | OpenAI-style client で互換性を確認しやすい軽量な英語ルート。 | [OpenAI API example](../examples/openai_api/README_ja.md) |
|
||||
| LLM-based ASR や 31 言語の評価 | Fun-ASR-Nano | LLM-based モデル。decoder throughput が重要なら vLLM を使います。 | [vLLM guide](./vllm_guide.md) |
|
||||
| LLM-based ASR や中英日 + 中国語方言・地域アクセントの評価 | Fun-ASR-Nano | LLM-based モデル。decoder throughput が重要なら vLLM を使います。 | [vLLM guide](./vllm_guide.md) |
|
||||
| ライブ字幕やコールセンターストリーム | Runtime WebSocket service | 長時間接続、部分結果、エンドポイント検出に向いたランタイム。 | [Runtime service docs](../runtime/readme.md) |
|
||||
| Whisper / cloud ASR からの移行 | SenseVoice-Small で baseline を作り、必要に応じて比較 | まず強い標準ルートで評価してから、用途別に詰めるのが安全です。 | [Migration guide](./migration_from_whisper.md) |
|
||||
|
||||
@@ -40,7 +40,7 @@ result = model.generate(input="meeting.wav")
|
||||
| `sensevoice` | `iic/SenseVoiceSmall` | 多言語 ASR、イベントタグ、CPU/GPU 両対応の標準プライベート音声 API。 |
|
||||
| `paraformer` | `paraformer-zh` + VAD + punctuation | 中国語中心の本番ルート。 |
|
||||
| `paraformer-en` | `paraformer-en` + VAD | OpenAI-style client の英語互換性チェック。 |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | LLM-based ASR、31 言語、vLLM acceleration の評価。 |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | LLM-based ASR の中英日・中国語方言/地域アクセント評価、または vLLM acceleration の確認。 |
|
||||
|
||||
接続前にサービスを確認します。
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ result = model.generate(input="meeting.wav")
|
||||
| 빠른 다국어 프라이빗 전사 | SenseVoice-Small | ASR, 감정 태그, 음성 이벤트 태그, CPU/GPU 사용성이 균형 잡힌 기본 경로입니다. | [README quick start](../README_ko.md#빠른-시작) |
|
||||
| 중국어 중심 프로덕션 ASR | Paraformer-Large | VAD와 문장부호 복원을 함께 쓰는 성숙한 중국어 ASR 경로입니다. | [Tutorial](./tutorial/README.md) |
|
||||
| OpenAI API 예제의 영어 경로 | `paraformer-en` alias | OpenAI-style client에서 호환성을 확인하기 쉬운 가벼운 영어 경로입니다. | [OpenAI API example](../examples/openai_api/README_ko.md) |
|
||||
| LLM-based ASR 또는 31개 언어 평가 | Fun-ASR-Nano | LLM-based 모델입니다. decoder throughput이 중요하면 vLLM을 사용합니다. | [vLLM guide](./vllm_guide.md) |
|
||||
| LLM-based ASR 또는 중영일 + 중국어 방언/지역 억양 평가 | Fun-ASR-Nano | LLM-based 모델입니다. decoder throughput이 중요하면 vLLM을 사용합니다. | [vLLM guide](./vllm_guide.md) |
|
||||
| 라이브 자막 또는 콜센터 스트림 | Runtime WebSocket service | 장시간 연결, 부분 결과, endpointing에 맞춘 런타임입니다. | [Runtime service docs](../runtime/readme.md) |
|
||||
| Whisper / cloud ASR에서 전환 | SenseVoice-Small로 baseline을 만들고 필요하면 비교 | 강한 기본 경로로 먼저 평가한 뒤 용도별로 조정하는 편이 안전합니다. | [Migration guide](./migration_from_whisper.md) |
|
||||
|
||||
@@ -40,7 +40,7 @@ result = model.generate(input="meeting.wav")
|
||||
| `sensevoice` | `iic/SenseVoiceSmall` | 다국어 ASR, 이벤트 태그, CPU/GPU 동작이 균형 잡힌 기본 프라이빗 음성 API. |
|
||||
| `paraformer` | `paraformer-zh` + VAD + punctuation | 중국어 중심 프로덕션 경로. |
|
||||
| `paraformer-en` | `paraformer-en` + VAD | OpenAI-style client의 영어 호환성 확인. |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | LLM-based ASR, 31개 언어, vLLM acceleration 평가. |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | LLM-based ASR의 중영일·중국어 방언/지역 억양 평가, 또는 vLLM acceleration 확인. |
|
||||
|
||||
클라이언트를 연결하기 전에 서비스를 확인하세요.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
## 默认快速路径
|
||||
|
||||
如果有 GPU，先从旗舰 **Fun-ASR-Nano** 开始 —— 基于 LLM 的识别模型（SenseVoice 编码器 + Qwen3 解码），覆盖 31 语种，在难例、上下文和专名上精度最强：
|
||||
如果有 GPU，先从旗舰 **Fun-ASR-Nano** 开始 —— 基于 LLM 的识别模型（SenseVoice 编码器 + Qwen3 解码），覆盖中文、英文、日语，以及 7 种中文方言和 26 种地域口音，在难例、上下文和专名上精度最强：
|
||||
|
||||
```python
|
||||
from funasr import AutoModel
|
||||
@@ -37,7 +37,7 @@ result = model.generate(input="meeting.wav")
|
||||
| 快速多语种私有转写 | SenseVoice-Small | 兼顾 ASR、情感标签、音频事件标签和 CPU 可用性。 | [README 快速开始](../README_zh.md#快速开始) |
|
||||
| 中文生产 ASR | Paraformer-Large | 成熟中文 ASR 路径，可组合 VAD 和标点。 | [教程](./tutorial/README_zh.md) |
|
||||
| OpenAI API 示例中的英文路由 | `paraformer-en` alias | 适合在 OpenAI 风格客户端里验证较轻量英文路径。 | [OpenAI API 示例](../examples/openai_api/README_zh.md) |
|
||||
| LLM-based ASR 或 31 语种实验 | Fun-ASR-Nano | LLM-based 模型路径；解码吞吐敏感时配合 vLLM。 | [vLLM 指南](./vllm_guide.md) |
|
||||
| LLM-based ASR 或中文/英文/日语 + 方言实验 | Fun-ASR-Nano | LLM-based 模型路径；解码吞吐敏感时配合 vLLM。 | [vLLM 指南](./vllm_guide.md) |
|
||||
| 实时字幕或客服流式音频 | Runtime WebSocket 服务 | 面向长连接流式会话和中间结果。 | [Runtime 服务文档](../runtime/readme_cn.md) |
|
||||
| 录音归档批处理 | SenseVoice-Small 或 Paraformer-Large | 稳定离线转写路径；调用方负责 manifest、重试和日志。 | [批处理示例](../examples/batch_asr_improved.py) |
|
||||
| 从 Whisper/云端 ASR 迁移 | 先用 SenseVoice-Small，再 benchmark 其他模型 | 先建立强基线，再做模型专项调优。 | [迁移指南](./migration_from_whisper_zh.md) |
|
||||
@@ -51,7 +51,7 @@ result = model.generate(input="meeting.wav")
|
||||
| `sensevoice` | `iic/SenseVoiceSmall` | 默认私有语音 API，多语种 ASR、事件标签和 CPU/GPU 行为较均衡。 |
|
||||
| `paraformer` | `paraformer-zh` + VAD + 标点 | 中文生产流量优先尝试。 |
|
||||
| `paraformer-en` | `paraformer-en` + VAD | OpenAI 风格客户端里的英文轻量路由。 |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | 评估 LLM-based ASR、31 语种覆盖或 vLLM 加速。 |
|
||||
| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | 评估 LLM-based ASR 的中文、英文、日语与中文方言/口音覆盖，或测试 vLLM 加速。 |
|
||||
|
||||
如果部署目标是昇腾 NPU，请把 `fun-asr-nano` 和 SenseVoice / Paraformer 分开看。Fun-ASR-Nano 的 PyTorch `AutoModel` 路径在修复 NPU autocast 后已有 310P3 社区兼容性 smoke 结果，但该测试明显慢于 CPU；`AutoModelVLLM` 仍依赖 vLLM-Ascend 算子支持，并已遇到 Qwen3 rotary / `TransData` 失败。生产部署优先使用 CUDA/vLLM、标准 PyTorch CPU/GPU 或 GGUF runtime，除非你正在主动验证 Ascend 后端。
|
||||
|
||||
@@ -93,7 +93,7 @@ SDK、JavaScript、工作流、Postman、OpenAPI、Docker 和 Kubernetes 路径
|
||||
|
||||
- demo、私有 API、Agent 语音输入和多语种场景优先试 SenseVoice-Small。
|
||||
- 中文生产流量优先试 Paraformer，尤其是希望走成熟非自回归 ASR 路径时。
|
||||
- 明确需要 LLM-based 模型路径或 vLLM 加速实验时，再试 Fun-ASR-Nano。
|
||||
- 明确需要 LLM-based 模型路径或 vLLM 加速实验时，再试 Fun-ASR-Nano；如需单独的 31 语种覆盖，请改用 Fun-ASR-MLT-Nano。
|
||||
- 需要中间结果和长连接时，优先使用 streaming runtime，而不是普通 HTTP 转写接口。
|
||||
- 生产 runbook 中固定模型 alias，保证 benchmark 和问题复现可追踪。
|
||||
- 遇到阻塞时，用 [Deployment Help issue](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) 提供模型、设备、命令、日志、音频时长和运行路径。
|
||||
|
||||
@@ -146,7 +146,8 @@ print(res[0]["text"])
|
||||
print(res[0]["timestamps"]) # character-level timestamps
|
||||
```
|
||||
Notes:
|
||||
- Fun-ASR-Nano is trained on tens of millions of hours of data, supporting 31 languages including Chinese dialects.
|
||||
- Fun-ASR-Nano is trained on tens of millions of hours of data and supports Chinese, English, and Japanese, plus Chinese dialect groups and regional accents.
|
||||
- For the separate 31-language checkpoint, use Fun-ASR-MLT-Nano.
|
||||
- Supports hotwords: `hotwords=["keyword1", "keyword2"]`
|
||||
- Supports speaker diarization: add `spk_model="cam++"` and `punc_model="ct-punc"` to get `sentence_info` with speaker labels.
|
||||
- Requires: `pip install tiktoken huggingface_hub`
|
||||
|
||||
@@ -40,7 +40,7 @@ In your Dify app settings:
|
||||
|
||||
| Model Name | Best For | Speed |
|
||||
|-----------|----------|-------|
|
||||
| `fun-asr-nano` | 31 languages + Chinese dialects | 340x realtime (vLLM) |
|
||||
| `fun-asr-nano` | zh/en/ja + Chinese dialects/accents | 340x realtime (vLLM) |
|
||||
| `sensevoice` | Ultra-fast, 5 languages | 170x realtime |
|
||||
| `paraformer` | Chinese (classic, stable) | 120x realtime |
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
「[简体中文](README_zh.md)」|「English」
|
||||
|
||||
Fun-ASR is an end-to-end speech recognition large model launched by Tongyi Lab. It is trained on tens of millions of hours of real speech data, possessing powerful contextual understanding capabilities and industry adaptability. It supports low-latency real-time transcription and covers 31 languages. It excels in vertical domains such as education and finance, accurately recognizing professional terminology and industry expressions, effectively addressing challenges like "hallucination" generation and language confusion, achieving "clear hearing, understanding meaning, and accurate writing."
|
||||
Fun-ASR is an end-to-end speech recognition large model launched by Tongyi Lab. It is trained on tens of millions of hours of real speech data, possessing powerful contextual understanding capabilities and industry adaptability. The Nano checkpoint supports low-latency real-time transcription for Chinese, English, and Japanese, plus Chinese dialect groups and regional accents; the separate MLT-Nano checkpoint extends coverage to 31 languages. It excels in vertical domains such as education and finance, accurately recognizing professional terminology and industry expressions, effectively addressing challenges like "hallucination" generation and language confusion, achieving "clear hearing, understanding meaning, and accurate writing."
|
||||
|
||||
<div align="center">
|
||||
<img src="images/funasr-v2.png">
|
||||
@@ -34,7 +34,7 @@ Online Experience:
|
||||
|
||||
# What's New 🔥
|
||||
|
||||
- 2025/12: [Fun-ASR-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) is an end-to-end speech recognition large model trained on tens of millions of hours real speech data. It supports low-latency real-time transcription and covers 31 languages.
|
||||
- 2025/12: [Fun-ASR-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) is an end-to-end speech recognition large model trained on tens of millions of hours real speech data. It supports low-latency real-time transcription for Chinese, English, and Japanese, plus Chinese dialect groups and regional accents; the separate MLT-Nano checkpoint covers 31 languages.
|
||||
- 2024/7: [FunASR](https://github.com/modelscope/FunASR) is a fundamental speech recognition toolkit that offers a variety of features, including speech recognition (ASR), Voice Activity Detection (VAD), Punctuation Restoration, Language Models, Speaker Verification, Speaker Diarization and multi-talker ASR.
|
||||
|
||||
# Core Features 🎯
|
||||
@@ -45,7 +45,7 @@ Online Experience:
|
||||
- **Chinese Dialects and Regional Accents:**
|
||||
- Supports **7 major dialects**: Wu, Cantonese, Min, Hakka, Gan, Xiang, Jin
|
||||
- Covers **26 regional accents**: including Henan, Shaanxi, Hubei, Sichuan, Chongqing, Yunnan, Guizhou, Guangdong, Guangxi and more than 20 other regions
|
||||
- **Multi-language Free Speech:** Supports recognition of **31 languages**, with focused optimization on East and Southeast Asian languages, supporting free language switching and mixed recognition.
|
||||
- **Multi-language Free Speech:** The separate **Fun-ASR-MLT-Nano** checkpoint supports recognition of **31 languages**, with focused optimization on East and Southeast Asian languages, supporting free language switching and mixed recognition.
|
||||
- **Music Background Lyric Recognition:** Enhanced speech recognition performance under music background interference, supporting accurate recognition of lyric content in songs.
|
||||
|
||||
# Environment Setup 🐍
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
「简体中文」|「[English](README.md)」
|
||||
|
||||
Fun-ASR 是通义实验室推出的端到端语音识别大模型，是基于数千万小时真实语音数据训练而成，具备强大的上下文理解能力与行业适应性，支持低延迟实时听写，并且覆盖 31 个语种。在教育、金融等垂直领域表现出色，能准确识别专业术语与行业表达，有效应对"幻觉"生成和语种混淆等挑战，实现"听得清、懂其意、写得准"。
|
||||
Fun-ASR 是通义实验室推出的端到端语音识别大模型，是基于数千万小时真实语音数据训练而成，具备强大的上下文理解能力与行业适应性，支持低延迟实时听写。Fun-ASR-Nano 支持中文、英文、日语，以及 7 种中文方言和 26 种地域口音；独立的 Fun-ASR-MLT-Nano checkpoint 则覆盖 31 个语种。在教育、金融等垂直领域表现出色，能准确识别专业术语与行业表达，有效应对"幻觉"生成和语种混淆等挑战，实现"听得清、懂其意、写得准"。
|
||||
|
||||
<div align="center">
|
||||
<img src="images/funasr-v2.png">
|
||||
@@ -34,7 +34,7 @@ Fun-ASR 是通义实验室推出的端到端语音识别大模型，是基于数
|
||||
|
||||
# 最新动态 🔥
|
||||
|
||||
- 2025/12: [Fun-ASR-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) 是一款基于数千万小时真实语音数据训练的端到端语音识别大模型。它支持低延迟实时转写，并涵盖 31 种语言识别功能。
|
||||
- 2025/12: [Fun-ASR-Nano-2512](https://modelscope.cn/models/FunAudioLLM/Fun-ASR-Nano-2512) 是一款基于数千万小时真实语音数据训练的端到端语音识别大模型。它支持低延迟实时转写，覆盖中文、英文、日语，以及 7 种中文方言和 26 种地域口音；独立的 Fun-ASR-MLT-Nano checkpoint 则覆盖 31 种语言。
|
||||
- 2024/7: [FunASR](https://github.com/modelscope/FunASR) 是一款功能全面的语音识别基础工具包，集成了多项核心功能，包括自动语音识别（ASR）、语音活动检测（VAD）、标点恢复、语言模型、说话人验证、说话人日志记录以及多说话人语音识别。
|
||||
|
||||
# 核心特性 🎯
|
||||
@@ -45,7 +45,7 @@ Fun-ASR 是通义实验室推出的端到端语音识别大模型，是基于数
|
||||
- **中文方言与地方口音：**
|
||||
- 支持 **7 大方言**：吴语、粤语、闽语、客家话、赣语、湘语、晋语
|
||||
- 覆盖 **26 个地区口音**：包括河南、陕西、湖北、四川、重庆、云南、贵州、广东、广西等 20 多个地区
|
||||
- **多语言自由说：** 支持 **31 种语言**识别，重点优化东亚与东南亚语种，支持语种自由切换和混合识别。
|
||||
- **多语言自由说：** 独立的 **Fun-ASR-MLT-Nano** checkpoint 支持 **31 种语言**识别，重点优化东亚与东南亚语种，支持语种自由切换和混合识别。
|
||||
- **音乐背景歌词识别：** 强化在音乐背景干扰下的语音识别性能，支持对歌曲中歌词内容的精准识别。
|
||||
|
||||
# 环境安装 🐍
|
||||
|
||||
@@ -82,11 +82,11 @@ body { font-family:'Inter',system-ui,sans-serif; background:#080810; color:#e0e0
|
||||
<span class="feat feat-green">Streaming ASR</span>
|
||||
<span class="feat feat-yellow">Speaker Diarization <span style="font-size:9px;opacity:0.7">(Beta)</span></span>
|
||||
<span class="feat feat-purple">Hotword Customization</span>
|
||||
<span class="feat feat-blue">31 Languages · 7 Dialects</span>
|
||||
<span class="feat feat-blue">Nano: zh/en/ja + dialects · MLT-Nano: 31 languages</span>
|
||||
</div>
|
||||
<p class="hero-desc">
|
||||
基于 <b>FunASR</b> 的 <b>vLLM 推理引擎</b>，实现流式语音识别服务。支持实时 VAD 分句、说话人分离 <span style="color:#f59e0b;font-size:11px">(Beta)</span>、<b>热词定制化</b>（加载人名、地名等实体词列表，提升专有名词识别准确率）、31种语言及中文方言。所有代码与模型已全部开源。<br>
|
||||
<span style="font-size:12px">Streaming ASR with vLLM engine, real-time VAD, speaker diarization <span style="color:#f59e0b">(Beta)</span>, <b>hotword customization</b> (names, places, entities), 31 languages & Chinese dialects. Fully open-sourced.</span>
|
||||
基于 <b>FunASR</b> 的 <b>vLLM 推理引擎</b>，实现流式语音识别服务。支持实时 VAD 分句、说话人分离 <span style="color:#f59e0b;font-size:11px">(Beta)</span>、<b>热词定制化</b>（加载人名、地名等实体词列表，提升专有名词识别准确率）。Nano 覆盖中文、英文、日语及中文方言/口音；MLT-Nano 覆盖 31 种语言。所有代码与模型已全部开源。<br>
|
||||
<span style="font-size:12px">Streaming ASR with vLLM engine, real-time VAD, speaker diarization <span style="color:#f59e0b">(Beta)</span>, <b>hotword customization</b> (names, places, entities), and checkpoint-specific language coverage: Nano for zh/en/ja plus Chinese dialects/accents, MLT-Nano for 31 languages. Fully open-sourced.</span>
|
||||
· <a href="https://www.funasr.com" target="_blank">www.funasr.com</a>
|
||||
</p>
|
||||
</div>
|
||||
|
||||
@@ -44,7 +44,7 @@ ssh -L 10095:localhost:10095 <server>
|
||||
- **流式 VAD 分句**：动态静音阈值，自然断句
|
||||
- **说话人分离 (Beta)**：流式 ID 分配 + 最终重聚类
|
||||
- **热词定制化**：加载人名、地名等实体词文件
|
||||
- **语种指定**：31 种语言 + 中文方言
|
||||
- **语种指定**：Nano 支持中/英/日及中文方言/口音；MLT-Nano 支持 31 种语言
|
||||
- **幻觉检测**：自动检测重复模式并截断
|
||||
|
||||
## 文件列表
|
||||
|
||||
@@ -107,7 +107,7 @@ Now your AI assistant can transcribe local audio files while keeping the audio i
|
||||
| `sensevoice` | General purpose, emotion | 170x GPU / 17x CPU |
|
||||
| `paraformer` | Chinese production | 120x GPU / 15x CPU |
|
||||
| `paraformer-en` | English production | 120x GPU / 15x CPU |
|
||||
| `fun-asr-nano` | 31 languages, LLM-based | 17x GPU |
|
||||
| `fun-asr-nano` | zh/en/ja + Chinese dialects/accents, LLM-based | 17x GPU |
|
||||
|
||||
## Get Started
|
||||
|
||||
|
||||
@@ -91,7 +91,7 @@ curl http://localhost:8000/v1/audio/transcriptions \
|
||||
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | Emotion detection |
|
||||
| `paraformer` | 120x realtime | 15x realtime | zh/en | Punctuation |
|
||||
| `paraformer-en` | 120x realtime | 15x realtime | en | English only |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based, timestamps |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | zh/en/ja + Chinese dialects/accents | LLM-based, timestamps |
|
||||
|
||||
## API Endpoints
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ curl http://localhost:8000/v1/audio/transcriptions \
|
||||
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | 感情・イベントタグ |
|
||||
| `paraformer` | 120x realtime | 15x realtime | zh/en | 句読点復元 |
|
||||
| `paraformer-en` | 120x realtime | 15x realtime | en | 英語認識 |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based、タイムスタンプ |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 中英日 + 中国語方言・地域アクセント | LLM-based、タイムスタンプ |
|
||||
|
||||
## API エンドポイント
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ curl http://localhost:8000/v1/audio/transcriptions \
|
||||
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | 감정 및 이벤트 태그 |
|
||||
| `paraformer` | 120x realtime | 15x realtime | zh/en | 문장부호 복원 |
|
||||
| `paraformer-en` | 120x realtime | 15x realtime | en | 영어 인식 |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based, 타임스탬프 |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 중영일 + 중국어 방언/지역 억양 | LLM-based, 타임스탬프 |
|
||||
|
||||
## API 엔드포인트
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ curl http://localhost:8000/v1/audio/transcriptions \
|
||||
| `sensevoice` | 170x realtime | 17x realtime | zh/en/ja/ko/yue | 情感与事件标签 |
|
||||
| `paraformer` | 120x realtime | 15x realtime | zh/en | 标点恢复 |
|
||||
| `paraformer-en` | 120x realtime | 15x realtime | en | 英文识别 |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 31 languages | LLM-based，时间戳 |
|
||||
| `fun-asr-nano` | 17x realtime | 3.6x realtime | 中/英/日 + 中文方言/口音 | LLM-based，时间戳 |
|
||||
|
||||
## API 端点
|
||||
|
||||
|
||||
@@ -55,7 +55,7 @@ python funasr_input.py --lang zh # 语言
|
||||
|------|------|------|
|
||||
| sensevoice (默认) | 170x GPU / 17x CPU | 中/英/日/韩/粤 |
|
||||
| paraformer | 120x GPU / 15x CPU | 中/英 |
|
||||
| fun-asr-nano | 17x GPU | 31种语言 |
|
||||
| fun-asr-nano | 17x GPU | 中/英/日 + 中文方言/口音 |
|
||||
|
||||
## 架构
|
||||
|
||||
|
||||
@@ -34,7 +34,8 @@ class FunASRNano(nn.Module):
|
||||
"""Fun-ASR-Nano: End-to-End ASR Large Model.
|
||||
|
||||
Trained on tens of millions of hours of real speech data.
|
||||
Supports 31 languages including Chinese dialects and regional accents.
|
||||
Language coverage is checkpoint-specific: Nano supports Chinese, English,
|
||||
and Japanese plus Chinese dialects/accents; MLT-Nano supports 31 languages.
|
||||
|
||||
Features:
|
||||
- Character-level timestamps (via CTC forced alignment)
|
||||
|
||||
Reference in New Issue
Block a user