- Quick Start · Benchmark · Migration guide · Use cases · Deployment matrix · Models · Agent Integration · Docs · Contribute + Quick Start · Benchmark · Model selection · Migration guide · Use cases · Deployment matrix · Models · Agent Integration · Docs · Contribute
--- @@ -66,7 +66,7 @@ That's it. **One model, one call** — VAD segmentation, speech recognition, pun | Cost | Free | Free | $0.006/min+ | | CPU viable | ✅ 17x realtime | ❌ Too slow | N/A | -Planning a switch from Whisper or a cloud ASR provider? Use the [migration guide](./docs/migration_from_whisper.md) and [benchmark example](./examples/migration/) to test representative audio, map features, and roll out safely. +Choosing a first model? Start with the [model selection guide](./docs/model_selection.md). Planning a switch from Whisper or a cloud ASR provider? Use the [migration guide](./docs/migration_from_whisper.md) and [benchmark example](./examples/migration/) to test representative audio, map features, and roll out safely. --- diff --git a/README_zh.md b/README_zh.md index 26b0289e1..543f845c6 100644 --- a/README_zh.md +++ b/README_zh.md @@ -21,7 +21,7 @@- 快速开始 · 性能评测 · 迁移指南 · 场景速览 · 部署选型 · 模型列表 · Agent 集成 · 文档 · 贡献 + 快速开始 · 性能评测 · 模型选择 · 迁移指南 · 场景速览 · 部署选型 · 模型列表 · Agent 集成 · 文档 · 贡献
--- @@ -65,7 +65,7 @@ result = model.generate(input="meeting.wav") | 费用 | 免费 | 免费 | ¥0.04/分钟起 | | CPU 可用 | ✅ 17 倍实时 | ❌ 太慢 | 不适用 | -计划从 Whisper 或云端 ASR 切换？请按 [迁移指南](./docs/migration_from_whisper_zh.md) 和 [评测示例](./examples/migration/) 用代表性音频评测、映射功能并安全上线。 +还不确定先用哪个模型？先看 [模型选择指南](./docs/model_selection_zh.md)。计划从 Whisper 或云端 ASR 切换？请按 [迁移指南](./docs/migration_from_whisper_zh.md) 和 [评测示例](./examples/migration/) 用代表性音频评测、映射功能并安全上线。 --- diff --git a/docs/deployment_matrix.md b/docs/deployment_matrix.md index 2572f65e9..4ba579eee 100644 --- a/docs/deployment_matrix.md +++ b/docs/deployment_matrix.md @@ -21,7 +21,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm ### I want to try FunASR in five minutes -Use the Python API from the README. It is the shortest route for validating installation, model download, device selection, and basic output shape. +Use the Python API from the README. It is the shortest route for validating installation, model download, device selection, and basic output shape. If you are unsure which model to start with, use the [model selection guide](./model_selection.md). ### I want a local replacement for cloud transcription diff --git a/docs/deployment_matrix_zh.md b/docs/deployment_matrix_zh.md index ab2fb8f02..a09ece909 100644 --- a/docs/deployment_matrix_zh.md +++ b/docs/deployment_matrix_zh.md @@ -21,7 +21,7 @@ ### 我想五分钟内试跑 FunASR -使用 README 里的 Python API。它是验证安装、模型下载、设备选择和基础输出格式的最短路径。 +使用 README 里的 Python API。它是验证安装、模型下载、设备选择和基础输出格式的最短路径。如果还不确定先用哪个模型，请看 [模型选择指南](./model_selection_zh.md)。 ### 我想替代云端转写服务 diff --git a/docs/index.rst b/docs/index.rst index 551b30437..6d2e04c07 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -106,6 +106,8 @@ Overview :maxdepth: 1 :caption: Application + ./model_selection.md + ./model_selection_zh.md ./migration_from_whisper.md ./migration_from_whisper_zh.md ./use_case_showcase.md diff --git a/docs/migration_from_whisper.md b/docs/migration_from_whisper.md index ac1aba2c7..1853ecd3b 100644 --- a/docs/migration_from_whisper.md +++ b/docs/migration_from_whisper.md @@ -28,7 +28,7 @@ Stay on your current pipeline if you need a managed service with no operations w | Existing workflow | FunASR path | What to validate | |---|---|---| -| Whisper file transcription | [README quick start](../README.md#quick-start) with SenseVoice, Paraformer, or Fun-ASR-Nano | Transcript quality, timestamps, speed, model download, CPU/GPU behavior. | +| Whisper file transcription | [README quick start](../README.md#quick-start) and [model selection guide](./model_selection.md) with SenseVoice, Paraformer, or Fun-ASR-Nano | Transcript quality, timestamps, speed, model download, CPU/GPU behavior. | | Whisper plus pyannote | `spk_model="cam++"` with VAD and punctuation | Speaker labels, speaker changes, overlapping speech, long silences. | | OpenAI audio API or cloud batch ASR | [OpenAI-compatible API example](../examples/openai_api/) | `/v1/audio/transcriptions`, response format, client compatibility, upload limits. | | Dify/LangChain/AutoGen agent audio | [Client recipes](../examples/openai_api/CLIENTS.md) or [MCP server](../examples/mcp_server/) | Tool latency, file handling, auth boundary, error reporting. | diff --git a/docs/migration_from_whisper_zh.md b/docs/migration_from_whisper_zh.md index bcf4e3243..589169bd5 100644 --- a/docs/migration_from_whisper_zh.md +++ b/docs/migration_from_whisper_zh.md @@ -28,7 +28,7 @@ | 现有流程 | FunASR 路径 | 需要验证什么 | |---|---|---| -| Whisper 文件转写 | 使用 SenseVoice、Paraformer 或 Fun-ASR-Nano 的 [README 快速开始](../README_zh.md#快速开始) | 转写质量、时间戳、速度、模型下载、CPU/GPU 行为。 | +| Whisper 文件转写 | 使用 [README 快速开始](../README_zh.md#快速开始) 和 [模型选择指南](./model_selection_zh.md)，在 SenseVoice、Paraformer 或 Fun-ASR-Nano 中选型 | 转写质量、时间戳、速度、模型下载、CPU/GPU 行为。 | | Whisper + pyannote | VAD、标点和 `spk_model="cam++"` | 说话人标签、换人位置、重叠说话、长静音。 | | OpenAI 音频 API 或云端批量 ASR | [OpenAI 兼容 API 示例](../examples/openai_api/README_zh.md) | `/v1/audio/transcriptions`、响应格式、客户端兼容性、上传限制。 | | Dify/LangChain/AutoGen Agent 音频 | [客户端配方](../examples/openai_api/CLIENTS.md) 或 [MCP 服务](../examples/mcp_server/) | 工具延迟、文件处理、鉴权边界、错误返回。 | diff --git a/docs/model_selection.md b/docs/model_selection.md new file mode 100644 index 000000000..7dfc6daba --- /dev/null +++ b/docs/model_selection.md @@ -0,0 +1,87 @@ +# FunASR Model Selection Guide + +Use this guide when you are choosing a first model, comparing FunASR with Whisper or a cloud ASR provider, or deciding which model alias to expose through the OpenAI-compatible API. + +## Fast default path + +If you are unsure, start with **SenseVoice-Small**: + +```python +from funasr import AutoModel + +model = AutoModel( + model="iic/SenseVoiceSmall", + vad_model="fsmn-vad", + spk_model="cam++", + device="cuda", # use "cpu" for a portable smoke test +) +result = model.generate(input="meeting.wav") +``` + +It is the best first choice for demos, private APIs, multilingual transcription, speaker-aware meeting transcripts, and agent voice input. Switch only when your workload has a clear requirement such as Mandarin production accuracy, streaming latency, or LLM-based ASR experiments. + +## Decision table + +| Need | Start with | Why | Next doc | +|---|---|---|---| +| Fast multilingual private transcription | SenseVoice-Small | Strong default with ASR, emotion tags, audio event tags, and CPU viability. | [README quick start](../README.md#quick-start) | +| Mandarin production ASR | Paraformer-Large | Mature Chinese ASR path with VAD and punctuation. | [Tutorial](./tutorial/README.md) | +| English-only route in the OpenAI API example | `paraformer-en` alias | Smaller English route for API compatibility checks. | [OpenAI API example](../examples/openai_api/) | +| LLM-based ASR or 31-language experiments | Fun-ASR-Nano | LLM-based model path; use vLLM when decoder throughput matters. | [vLLM guide](./vllm_guide.md) | +| Live captions or call-center streams | Runtime WebSocket service | Designed for long-lived streaming sessions and partial results. | [Runtime service docs](../runtime/readme.md) | +| Batch archive processing | SenseVoice-Small or Paraformer-Large | Stable offline transcription path; caller owns manifests, retries, and logs. | [Batch ASR example](../examples/batch_asr_improved.py) | +| Migration from Whisper/cloud ASR | SenseVoice-Small first, then benchmark alternatives | Gives a strong baseline before deeper model-specific tuning. | [Migration guide](./migration_from_whisper.md) | + +## OpenAI-compatible API aliases + +The `examples/openai_api` server exposes short aliases so application teams do not need to know model repository IDs: + +| Alias | Underlying path | Use when | +|---|---|---| +| `sensevoice` | `iic/SenseVoiceSmall` | You want the default private speech API with multilingual ASR, event tags, and good CPU/GPU behavior. | +| `paraformer` | `paraformer-zh` with VAD and punctuation | You want a Mandarin-oriented production route. | +| `paraformer-en` | `paraformer-en` with VAD | You want a compact English route in OpenAI-style clients. | +| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | You are evaluating LLM-based ASR, 31-language coverage, or vLLM acceleration. | + +Check the live service before wiring clients: + +```bash +curl http://localhost:8000/v1/models +python examples/openai_api/smoke_test.py --base-url http://localhost:8000 --model sensevoice +``` + +For SDK, JavaScript, workflow, Postman, OpenAPI, Docker, and Kubernetes paths, start from the [OpenAI API example](../examples/openai_api/). + +## Runtime choice by workload + +| Workload | Runtime path | Notes | +|---|---|---| +| Notebook or one-off evaluation | Python `AutoModel` | Shortest path for install, model download, and output-shape checks. | +| Internal HTTP service | OpenAI-compatible API | Reuse OpenAI-style clients, Dify, n8n, LangChain, AutoGen, and HTTP nodes. | +| Repeatable local container demo | Docker Compose API | CPU-first smoke test; adapt the image before using CUDA. | +| Internal cluster service | Kubernetes API template | Private `ClusterIP`, persistent model cache, `/health` probes, and port-forward smoke test. | +| Live audio | Runtime WebSocket service | Validate chunk size, VAD, endpointing, reconnects, and client backpressure with real audio. | +| LLM-based ASR throughput | vLLM path for Fun-ASR-Nano | vLLM accelerates autoregressive decoding; it does not apply to non-autoregressive Paraformer. | + +See the [deployment matrix](./deployment_matrix.md) when you are choosing between these paths. + +## Benchmark before committing + +Do not choose a model from a single clean demo file. Use a small representative set first: + +- 20-50 audio files that cover short clips, long meetings, silence, noise, overlapping speakers, domain vocabulary, and target languages. +- Record model name, model revision, FunASR version, device, CPU/GPU type, CUDA/PyTorch version, runtime path, batch size, and whether warmup/model download time is excluded. +- Track quality with your normal WER/CER or human review process, not only transcript readability. +- Track latency, throughput, memory, failures, and upload size limits together. +- Keep at least one public sample for smoke tests and at least one private realistic sample for deployment validation. + +For migration work, use the [migration benchmark example](../examples/migration/) and the [migration guide](./migration_from_whisper.md). + +## Practical recommendations + +- Start with SenseVoice-Small for demos, private APIs, agent voice input, and multilingual workloads. +- Use Paraformer when your production traffic is primarily Mandarin and you want the mature non-autoregressive ASR path. +- Use Fun-ASR-Nano when you specifically want the LLM-based model path or vLLM acceleration experiments. +- Use the streaming runtime when partial results and long-lived connections matter more than a single final transcript. +- Keep model aliases stable in production runbooks so benchmark results and bug reports are reproducible. +- Open a [Deployment Help issue](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) with model, device, command, logs, audio duration, and runtime path when you get stuck. diff --git a/docs/model_selection_zh.md b/docs/model_selection_zh.md new file mode 100644 index 000000000..dc883e560 --- /dev/null +++ b/docs/model_selection_zh.md @@ -0,0 +1,87 @@ +# FunASR 模型选择指南 + +当你第一次选择模型、评估是否从 Whisper 或云端 ASR 迁移，或者准备通过 OpenAI 兼容 API 暴露模型别名时，可以先看这份指南。 + +## 默认快速路径 + +如果还不确定，先从 **SenseVoice-Small** 开始： + +```python +from funasr import AutoModel + +model = AutoModel( + model="iic/SenseVoiceSmall", + vad_model="fsmn-vad", + spk_model="cam++", + device="cuda", # 便携 smoke test 可改为 "cpu" +) +result = model.generate(input="meeting.wav") +``` + +它适合 demo、私有 API、多语种转写、带说话人信息的会议转写和 Agent 语音输入。只有当你的场景明确需要中文生产识别、低延迟流式结果或 LLM-based ASR 实验时，再切换到其他路径。 + +## 决策表 + +| 需求 | 优先尝试 | 原因 | 下一步文档 | +|---|---|---|---| +| 快速多语种私有转写 | SenseVoice-Small | 兼顾 ASR、情感标签、音频事件标签和 CPU 可用性。 | [README 快速开始](../README_zh.md#快速开始) | +| 中文生产 ASR | Paraformer-Large | 成熟中文 ASR 路径，可组合 VAD 和标点。 | [教程](./tutorial/README_zh.md) | +| OpenAI API 示例中的英文路由 | `paraformer-en` alias | 适合在 OpenAI 风格客户端里验证较轻量英文路径。 | [OpenAI API 示例](../examples/openai_api/README_zh.md) | +| LLM-based ASR 或 31 语种实验 | Fun-ASR-Nano | LLM-based 模型路径；解码吞吐敏感时配合 vLLM。 | [vLLM 指南](./vllm_guide.md) | +| 实时字幕或客服流式音频 | Runtime WebSocket 服务 | 面向长连接流式会话和中间结果。 | [Runtime 服务文档](../runtime/readme_cn.md) | +| 录音归档批处理 | SenseVoice-Small 或 Paraformer-Large | 稳定离线转写路径；调用方负责 manifest、重试和日志。 | [批处理示例](../examples/batch_asr_improved.py) | +| 从 Whisper/云端 ASR 迁移 | 先用 SenseVoice-Small，再 benchmark 其他模型 | 先建立强基线，再做模型专项调优。 | [迁移指南](./migration_from_whisper_zh.md) | + +## OpenAI 兼容 API 别名 + +`examples/openai_api` 服务提供短别名，应用团队不需要了解具体模型仓库 ID： + +| Alias | 底层路径 | 适合场景 | +|---|---|---| +| `sensevoice` | `iic/SenseVoiceSmall` | 默认私有语音 API，多语种 ASR、事件标签和 CPU/GPU 行为较均衡。 | +| `paraformer` | `paraformer-zh` + VAD + 标点 | 中文生产流量优先尝试。 | +| `paraformer-en` | `paraformer-en` + VAD | OpenAI 风格客户端里的英文轻量路由。 | +| `fun-asr-nano` | `FunAudioLLM/Fun-ASR-Nano-2512` | 评估 LLM-based ASR、31 语种覆盖或 vLLM 加速。 | + +接入客户端前先检查在线服务： + +```bash +curl http://localhost:8000/v1/models +python examples/openai_api/smoke_test.py --base-url http://localhost:8000 --model sensevoice +``` + +SDK、JavaScript、工作流、Postman、OpenAPI、Docker 和 Kubernetes 路径可从 [OpenAI API 示例](../examples/openai_api/README_zh.md) 开始。 + +## 按工作负载选择运行路径 + +| 工作负载 | 运行路径 | 说明 | +|---|---|---| +| Notebook 或一次性评估 | Python `AutoModel` | 验证安装、模型下载和输出结构的最短路径。 | +| 内部 HTTP 服务 | OpenAI 兼容 API | 复用 OpenAI 风格客户端、Dify、n8n、LangChain、AutoGen 和 HTTP 节点。 | +| 可复现本地容器 demo | Docker Compose API | CPU-first smoke test；使用 CUDA 前先适配镜像。 | +| 集群内私有服务 | Kubernetes API 模板 | 私有 `ClusterIP`、持久化模型缓存、`/health` probes 和 port-forward smoke test。 | +| 实时音频 | Runtime WebSocket 服务 | 用真实音频验证 chunk size、VAD、断句、重连和客户端背压。 | +| LLM-based ASR 吞吐 | Fun-ASR-Nano 的 vLLM 路径 | vLLM 加速自回归解码；不适用于非自回归 Paraformer。 | + +选择部署方式时可以参考 [部署选型表](./deployment_matrix_zh.md)。 + +## 上线前先 benchmark + +不要只用一个干净 demo 文件选型。先准备一个小而有代表性的集合： + +- 20-50 条音频，覆盖短音频、长会议、静音、噪声、多人重叠、领域词汇和目标语言。 +- 记录模型名、模型版本、FunASR 版本、设备、CPU/GPU 型号、CUDA/PyTorch 版本、运行路径、batch size，以及是否排除 warmup/模型下载时间。 +- 使用你已有的 WER/CER 流程或人工审阅，不要只看转写文本是否“读起来还行”。 +- 同时记录延迟、吞吐、内存、失败样例和上传大小限制。 +- 保留至少一个公开样例用于 smoke test，也保留至少一个真实私有样本用于部署验证。 + +迁移场景可以使用 [迁移评测示例](../examples/migration/) 和 [迁移指南](./migration_from_whisper_zh.md)。 + +## 实用建议 + +- demo、私有 API、Agent 语音输入和多语种场景优先试 SenseVoice-Small。 +- 中文生产流量优先试 Paraformer，尤其是希望走成熟非自回归 ASR 路径时。 +- 明确需要 LLM-based 模型路径或 vLLM 加速实验时，再试 Fun-ASR-Nano。 +- 需要中间结果和长连接时，优先使用 streaming runtime，而不是普通 HTTP 转写接口。 +- 生产 runbook 中固定模型 alias，保证 benchmark 和问题复现可追踪。 +- 遇到阻塞时，用 [Deployment Help issue](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) 提供模型、设备、命令、日志、音频时长和运行路径。 diff --git a/docs/use_case_showcase.md b/docs/use_case_showcase.md index b93996cf8..802a99a5d 100644 --- a/docs/use_case_showcase.md +++ b/docs/use_case_showcase.md @@ -6,7 +6,7 @@ FunASR is useful far beyond a single offline transcription command. This page co | Goal | Start here | Why it matters | |---|---|---| -| Transcribe one file locally | [README quick start](../README.md#quick-start) | Verify install and model download in minutes. | +| Transcribe one file locally | [README quick start](../README.md#quick-start) and [model selection guide](./model_selection.md) | Verify install, model choice, and model download in minutes. | | Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. | | Migrate from Whisper/cloud ASR | [Migration guide](./migration_from_whisper.md) | Map existing pipelines to FunASR, benchmark representative audio, and plan a safe rollout. | | Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/), [client recipes](../examples/openai_api/CLIENTS.md), [JavaScript/TypeScript recipes](../examples/openai_api/JAVASCRIPT.md), and [workflow recipes](../examples/openai_api/WORKFLOWS.md) | Reuse LangChain, Dify, n8n, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. | @@ -70,6 +70,8 @@ Use this path when deciding whether FunASR is a good replacement for Whisper or ## Model selection hints +For a deeper comparison of SenseVoice, Paraformer, Fun-ASR-Nano, streaming runtime, and OpenAI API aliases, use the [model selection guide](./model_selection.md). + | Need | Good first choice | Notes | |---|---|---| | Fast multilingual transcription | SenseVoice-Small | Strong default for local demos and private APIs. | diff --git a/docs/use_case_showcase_zh.md b/docs/use_case_showcase_zh.md index 190de8bbe..d741ec6ea 100644 --- a/docs/use_case_showcase_zh.md +++ b/docs/use_case_showcase_zh.md @@ -6,7 +6,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部 | 目标 | 从这里开始 | 为什么重要 | |---|---|---| -| 本地转写一个文件 | [README 快速开始](../README_zh.md#快速开始) | 几分钟内验证安装、模型下载和首次推理。 | +| 本地转写一个文件 | [README 快速开始](../README_zh.md#快速开始) 和 [模型选择指南](./model_selection_zh.md) | 几分钟内验证安装、模型选择、模型下载和首次推理。 | | 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 | | 从 Whisper/云端 ASR 迁移 | [迁移指南](./migration_from_whisper_zh.md) | 将现有流水线映射到 FunASR，用代表性音频评测并规划安全上线。 | | 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/README_zh.md)、[客户端配方](../examples/openai_api/CLIENTS.md)、[JavaScript/TypeScript 配方](../examples/openai_api/JAVASCRIPT_zh.md) 和 [工作流配方](../examples/openai_api/WORKFLOWS_zh.md) | 复用 LangChain、Dify、n8n、AutoGen 等 OpenAI 风格客户端，音频不出内网。 | @@ -70,6 +70,8 @@ curl http://localhost:8000/v1/audio/transcriptions \ ## 模型选择建议 +如需更完整地比较 SenseVoice、Paraformer、Fun-ASR-Nano、streaming runtime 和 OpenAI API alias，请看 [模型选择指南](./model_selection_zh.md)。 + | 需求 | 推荐先试 | 说明 | |---|---|---| | 快速多语种转写 | SenseVoice-Small | 本地 demo 和私有 API 的稳妥默认选择。 |