mirror of
https://github.com/modelscope/FunASR.git
synced 2026-10-12 02:19:18 +00:00
docs: add deployment decision matrix
Add bilingual deployment matrix docs and link them from README, docs, and use-case pages.
This commit is contained in:
zhifu gao
@@ -196,7 +196,7 @@ curl http://localhost:8000/v1/audio/transcriptions \
|
||||
docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12
|
||||
```
|
||||
|
||||
[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)
|
||||
[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)
|
||||
|
||||
---
|
||||
|
||||
|
||||
+1
-1
@@ -195,7 +195,7 @@ curl http://localhost:8000/v1/audio/transcriptions \
|
||||
docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12
|
||||
```
|
||||
|
||||
[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)
|
||||
[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# FunASR Deployment Matrix
|
||||
|
||||
Use this page to choose the shortest deployment path for a product, demo, benchmark, or internal workflow. Start with the smallest surface that satisfies the job, then move to heavier runtimes only when throughput, latency, or integration requirements demand it.
|
||||
|
||||
## Quick decision table
|
||||
|
||||
| Path | Best for | Start here | Operational notes |
|
||||
|---|---|---|---|
|
||||
| Python API | Notebooks, offline jobs, first model evaluation | [README quick start](../README.md#quick-start) | Lowest ceremony; caller owns batching, retries, and files. |
|
||||
| OpenAI-compatible API | Private speech API, agents, Dify/LangChain/AutoGen-style clients | [OpenAI API example](../examples/openai_api/) | Easiest integration for apps that already support OpenAI audio APIs. |
|
||||
| Docker Compose API | Reproducible local smoke test or small internal service | [OpenAI API Docker docs](../examples/openai_api/#docker-deployment) | CPU by default; adapt the image before using CUDA in containers. |
|
||||
| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |
|
||||
| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. |
|
||||
| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. |
|
||||
| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. |
|
||||
| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. |
|
||||
| Triton runtime | Specialized high-performance serving | [Triton runtime docs](../runtime/triton_gpu/README.md) | Heavier setup; choose when your team already operates Triton/GPU serving. |
|
||||
|
||||
## Common choices
|
||||
|
||||
### I want to try FunASR in five minutes
|
||||
|
||||
Use the Python API from the README. It is the shortest route for validating installation, model download, device selection, and basic output shape.
|
||||
|
||||
### I want a local replacement for cloud transcription
|
||||
|
||||
Use the OpenAI-compatible API. It exposes `/v1/audio/transcriptions`, `/v1/models`, `/health`, and Swagger docs. Start with `sensevoice`, run `examples/openai_api/smoke_test.sh`, then connect existing SDK or HTTP clients using [client recipes](../examples/openai_api/CLIENTS.md).
|
||||
|
||||
### I want a repeatable container demo
|
||||
|
||||
Use `examples/openai_api/docker-compose.yml` for a CPU-mode smoke test:
|
||||
|
||||
```bash
|
||||
cd examples/openai_api
|
||||
cp .env.example .env
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Keep CPU mode until you have a CUDA-capable PyTorch/FunASR image. After that, set `FUNASR_DEVICE=cuda` and verify with the same smoke test.
|
||||
|
||||
### I need streaming or live captioning
|
||||
|
||||
Use the runtime WebSocket service. Validate chunk size, VAD, endpointing, punctuation, speaker diarization, reconnect behavior, and client backpressure with real audio before production rollout.
|
||||
|
||||
### I need more LLM-based ASR throughput
|
||||
|
||||
Use the vLLM path for Fun-ASR-Nano. Benchmark with your own audio distribution and watch GPU memory, tensor parallel size, first-token latency, and warmup time.
|
||||
|
||||
## Readiness checklist
|
||||
|
||||
- Pick a model alias and pin it in deployment notes.
|
||||
- Record FunASR version, model version, device, CUDA/PyTorch version, Docker image tag, and command line.
|
||||
- Run a short public smoke sample and at least one realistic private sample.
|
||||
- Log audio duration, model, device, latency, response format, and error type for every request.
|
||||
- Add upload-size limits, authentication, TLS, and rate limits before exposing an API outside a trusted network.
|
||||
- For streaming, test silence, noise, overlapping speakers, long sessions, reconnects, and slow clients.
|
||||
- For benchmark claims, include input duration, hardware, batch size, model, runtime path, and whether model download/warmup time is excluded.
|
||||
|
||||
## When to open an issue
|
||||
|
||||
Use [Deployment Help](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) for runtime, Docker, vLLM, Triton, Android, browser, or agent integration problems. Include your deployment path, exact command/config, logs, model, device, and audio characteristics.
|
||||
@@ -0,0 +1,61 @@
|
||||
# FunASR 部署选型表
|
||||
|
||||
这个页面帮助你为产品、demo、benchmark 或内部工作流选择最短部署路径。先选择能满足目标的最小方案，只有在吞吐、延迟或集成方式有明确要求时，再切换到更重的运行时。
|
||||
|
||||
## 快速决策表
|
||||
|
||||
| 路径 | 适合场景 | 从这里开始 | 运维提示 |
|
||||
|---|---|---|---|
|
||||
| Python API | Notebook、离线任务、首次模型评测 | [README 快速开始](../README_zh.md#快速开始) | 最简单；调用方自己负责批处理、重试和文件管理。 |
|
||||
| OpenAI 兼容 API | 私有语音 API、Agent、Dify/LangChain/AutoGen 风格客户端 | [OpenAI API 示例](../examples/openai_api/) | 已支持 OpenAI audio API 的应用最容易接入。 |
|
||||
| Docker Compose API | 可复现本地 smoke test 或小型内部服务 | [OpenAI API Docker 文档](../examples/openai_api/#docker-deployment) | 默认 CPU；容器里使用 CUDA 前需要先适配 CUDA-capable 镜像。 |
|
||||
| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |
|
||||
| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐；不适用于非自回归 Paraformer。 |
|
||||
| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 |
|
||||
| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 |
|
||||
| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 |
|
||||
| Triton Runtime | 专门的高性能推理服务 | [Triton 文档](../runtime/triton_gpu/README.md) | 配置更重；适合已经在运维 Triton/GPU serving 的团队。 |
|
||||
|
||||
## 常见选择
|
||||
|
||||
### 我想五分钟内试跑 FunASR
|
||||
|
||||
使用 README 里的 Python API。它是验证安装、模型下载、设备选择和基础输出格式的最短路径。
|
||||
|
||||
### 我想替代云端转写服务
|
||||
|
||||
使用 OpenAI 兼容 API。它提供 `/v1/audio/transcriptions`、`/v1/models`、`/health` 和 Swagger docs。先用 `sensevoice` 跑通 `examples/openai_api/smoke_test.sh`，再根据 [客户端配方](../examples/openai_api/CLIENTS.md) 接入 SDK 或 HTTP 客户端。
|
||||
|
||||
### 我想要可复现的容器 demo
|
||||
|
||||
使用 `examples/openai_api/docker-compose.yml` 跑 CPU smoke test：
|
||||
|
||||
```bash
|
||||
cd examples/openai_api
|
||||
cp .env.example .env
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
在没有 CUDA-capable PyTorch/FunASR 镜像前保持 CPU 模式。准备好 CUDA 镜像后，再设置 `FUNASR_DEVICE=cuda` 并用同一个 smoke test 验证。
|
||||
|
||||
### 我需要流式识别或实时字幕
|
||||
|
||||
使用 Runtime WebSocket 服务。上线前请用真实音频验证 chunk size、VAD、断句、标点、说话人分离、重连行为和客户端背压。
|
||||
|
||||
### 我需要更高的 LLM-based ASR 吞吐
|
||||
|
||||
Fun-ASR-Nano 走 vLLM 路径。请用自己的音频分布做 benchmark，并关注 GPU 显存、tensor parallel size、首 token 延迟和 warmup 时间。
|
||||
|
||||
## 上线检查清单
|
||||
|
||||
- 选择模型 alias，并写入部署说明。
|
||||
- 记录 FunASR 版本、模型版本、设备、CUDA/PyTorch 版本、Docker 镜像 tag 和启动命令。
|
||||
- 跑一个公开短音频 smoke sample，再跑至少一个真实私有样本。
|
||||
- 每次请求记录音频时长、模型、设备、延迟、响应格式和错误类型。
|
||||
- API 暴露到可信网络外之前，增加上传大小限制、鉴权、TLS 和限流。
|
||||
- 流式场景需要测试静音、噪声、多人重叠、长连接、重连和慢客户端。
|
||||
- 发布 benchmark 结论时，说明输入时长、硬件、batch size、模型、运行路径，以及是否排除模型下载和 warmup 时间。
|
||||
|
||||
## 什么时候开 issue
|
||||
|
||||
Runtime、Docker、vLLM、Triton、Android、浏览器或 Agent 集成问题，请使用 [Deployment Help](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md)。请附上部署路径、完整命令/config、日志、模型、设备和音频特征。
|
||||
@@ -71,6 +71,8 @@ Overview
|
||||
:maxdepth: 1
|
||||
:caption: Runtime and Service
|
||||
|
||||
./deployment_matrix.md
|
||||
./deployment_matrix_zh.md
|
||||
./runtime/readme.md
|
||||
./runtime/docs/SDK_tutorial_online.md
|
||||
./runtime/docs/SDK_tutorial.md
|
||||
|
||||
@@ -10,6 +10,7 @@ FunASR is useful far beyond a single offline transcription command. This page co
|
||||
| Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. |
|
||||
| Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/) and [client recipes](../examples/openai_api/CLIENTS.md) | Reuse LangChain, Dify, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. |
|
||||
| Add speech input to agents | [MCP server](../examples/mcp_server/) and [voice input](../examples/voice_input/) | Connect local ASR to Claude, Cursor, and desktop agent workflows. |
|
||||
| Choose a deployment path | [Deployment matrix](./deployment_matrix.md) | Compare Python API, OpenAI API, Docker Compose, WebSocket, vLLM, MCP, batch, subtitles, and Triton. |
|
||||
| Serve streaming ASR | [Runtime service docs](../runtime/readme.md) | Run WebSocket or service-mode ASR for live captioning and call-center style workloads. |
|
||||
| Accelerate LLM-based ASR | [vLLM guide](./vllm_guide.md) | Use tensor parallel decoding and streaming service support for Fun-ASR-Nano. |
|
||||
| Generate subtitles | [Subtitle example](../examples/subtitle/) | Turn long audio or video into subtitle files for media workflows. |
|
||||
|
||||
@@ -10,6 +10,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部
|
||||
| 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 |
|
||||
| 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/) 和 [客户端配方](../examples/openai_api/CLIENTS.md) | 复用 LangChain、Dify、AutoGen 等 OpenAI 风格客户端，音频不出内网。 |
|
||||
| 给 Agent 增加语音输入 | [MCP 服务](../examples/mcp_server/) 和 [语音输入示例](../examples/voice_input/) | 将本地 ASR 接入 Claude、Cursor 和桌面 Agent 工作流。 |
|
||||
| 选择部署路径 | [部署选型表](./deployment_matrix_zh.md) | 对比 Python API、OpenAI API、Docker Compose、WebSocket、vLLM、MCP、批处理、字幕和 Triton。 |
|
||||
| 部署流式 ASR | [Runtime 服务文档](../runtime/readme_cn.md) | 面向实时字幕、客服、会议等低延迟场景。 |
|
||||
| 加速 LLM-based ASR | [vLLM 指南](./vllm_guide.md) | 为 Fun-ASR-Nano 使用 tensor parallel 解码和流式服务能力。 |
|
||||
| 生成字幕 | [字幕示例](../examples/subtitle/) | 将长音频或视频转成字幕文件。 |
|
||||
|
||||
Reference in New Issue
Block a user