diff --git a/README.md b/README.md index 6d3c646f2..15ae1e03a 100644 --- a/README.md +++ b/README.md @@ -196,7 +196,7 @@ curl http://localhost:8000/v1/audio/transcriptions \ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12 ``` -[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html) +[OpenAI API example →](./examples/openai_api/) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html) --- diff --git a/README_zh.md b/README_zh.md index cb11511c8..782a46f27 100644 --- a/README_zh.md +++ b/README_zh.md @@ -195,7 +195,7 @@ curl http://localhost:8000/v1/audio/transcriptions \ docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12 ``` -[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html) +[OpenAI API 示例 →](./examples/openai_api/) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html) --- diff --git a/docs/deployment_matrix.md b/docs/deployment_matrix.md new file mode 100644 index 000000000..83f19667c --- /dev/null +++ b/docs/deployment_matrix.md @@ -0,0 +1,61 @@ +# FunASR Deployment Matrix + +Use this page to choose the shortest deployment path for a product, demo, benchmark, or internal workflow. Start with the smallest surface that satisfies the job, then move to heavier runtimes only when throughput, latency, or integration requirements demand it. + +## Quick decision table + +| Path | Best for | Start here | Operational notes | +|---|---|---|---| +| Python API | Notebooks, offline jobs, first model evaluation | [README quick start](../README.md#quick-start) | Lowest ceremony; caller owns batching, retries, and files. | +| OpenAI-compatible API | Private speech API, agents, Dify/LangChain/AutoGen-style clients | [OpenAI API example](../examples/openai_api/) | Easiest integration for apps that already support OpenAI audio APIs. | +| Docker Compose API | Reproducible local smoke test or small internal service | [OpenAI API Docker docs](../examples/openai_api/#docker-deployment) | CPU by default; adapt the image before using CUDA in containers. | +| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. | +| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. | +| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. | +| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. | +| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. | +| Triton runtime | Specialized high-performance serving | [Triton runtime docs](../runtime/triton_gpu/README.md) | Heavier setup; choose when your team already operates Triton/GPU serving. | + +## Common choices + +### I want to try FunASR in five minutes + +Use the Python API from the README. It is the shortest route for validating installation, model download, device selection, and basic output shape. + +### I want a local replacement for cloud transcription + +Use the OpenAI-compatible API. It exposes `/v1/audio/transcriptions`, `/v1/models`, `/health`, and Swagger docs. Start with `sensevoice`, run `examples/openai_api/smoke_test.sh`, then connect existing SDK or HTTP clients using [client recipes](../examples/openai_api/CLIENTS.md). + +### I want a repeatable container demo + +Use `examples/openai_api/docker-compose.yml` for a CPU-mode smoke test: + +```bash +cd examples/openai_api +cp .env.example .env +docker compose up --build +``` + +Keep CPU mode until you have a CUDA-capable PyTorch/FunASR image. After that, set `FUNASR_DEVICE=cuda` and verify with the same smoke test. + +### I need streaming or live captioning + +Use the runtime WebSocket service. Validate chunk size, VAD, endpointing, punctuation, speaker diarization, reconnect behavior, and client backpressure with real audio before production rollout. + +### I need more LLM-based ASR throughput + +Use the vLLM path for Fun-ASR-Nano. Benchmark with your own audio distribution and watch GPU memory, tensor parallel size, first-token latency, and warmup time. + +## Readiness checklist + +- Pick a model alias and pin it in deployment notes. +- Record FunASR version, model version, device, CUDA/PyTorch version, Docker image tag, and command line. +- Run a short public smoke sample and at least one realistic private sample. +- Log audio duration, model, device, latency, response format, and error type for every request. +- Add upload-size limits, authentication, TLS, and rate limits before exposing an API outside a trusted network. +- For streaming, test silence, noise, overlapping speakers, long sessions, reconnects, and slow clients. +- For benchmark claims, include input duration, hardware, batch size, model, runtime path, and whether model download/warmup time is excluded. + +## When to open an issue + +Use [Deployment Help](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md) for runtime, Docker, vLLM, Triton, Android, browser, or agent integration problems. Include your deployment path, exact command/config, logs, model, device, and audio characteristics. diff --git a/docs/deployment_matrix_zh.md b/docs/deployment_matrix_zh.md new file mode 100644 index 000000000..613f56e69 --- /dev/null +++ b/docs/deployment_matrix_zh.md @@ -0,0 +1,61 @@ +# FunASR 部署选型表 + +这个页面帮助你为产品、demo、benchmark 或内部工作流选择最短部署路径。先选择能满足目标的最小方案,只有在吞吐、延迟或集成方式有明确要求时,再切换到更重的运行时。 + +## 快速决策表 + +| 路径 | 适合场景 | 从这里开始 | 运维提示 | +|---|---|---|---| +| Python API | Notebook、离线任务、首次模型评测 | [README 快速开始](../README_zh.md#快速开始) | 最简单;调用方自己负责批处理、重试和文件管理。 | +| OpenAI 兼容 API | 私有语音 API、Agent、Dify/LangChain/AutoGen 风格客户端 | [OpenAI API 示例](../examples/openai_api/) | 已支持 OpenAI audio API 的应用最容易接入。 | +| Docker Compose API | 可复现本地 smoke test 或小型内部服务 | [OpenAI API Docker 文档](../examples/openai_api/#docker-deployment) | 默认 CPU;容器里使用 CUDA 前需要先适配 CUDA-capable 镜像。 | +| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 | +| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐;不适用于非自回归 Paraformer。 | +| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 | +| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 | +| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 | +| Triton Runtime | 专门的高性能推理服务 | [Triton 文档](../runtime/triton_gpu/README.md) | 配置更重;适合已经在运维 Triton/GPU serving 的团队。 | + +## 常见选择 + +### 我想五分钟内试跑 FunASR + +使用 README 里的 Python API。它是验证安装、模型下载、设备选择和基础输出格式的最短路径。 + +### 我想替代云端转写服务 + +使用 OpenAI 兼容 API。它提供 `/v1/audio/transcriptions`、`/v1/models`、`/health` 和 Swagger docs。先用 `sensevoice` 跑通 `examples/openai_api/smoke_test.sh`,再根据 [客户端配方](../examples/openai_api/CLIENTS.md) 接入 SDK 或 HTTP 客户端。 + +### 我想要可复现的容器 demo + +使用 `examples/openai_api/docker-compose.yml` 跑 CPU smoke test: + +```bash +cd examples/openai_api +cp .env.example .env +docker compose up --build +``` + +在没有 CUDA-capable PyTorch/FunASR 镜像前保持 CPU 模式。准备好 CUDA 镜像后,再设置 `FUNASR_DEVICE=cuda` 并用同一个 smoke test 验证。 + +### 我需要流式识别或实时字幕 + +使用 Runtime WebSocket 服务。上线前请用真实音频验证 chunk size、VAD、断句、标点、说话人分离、重连行为和客户端背压。 + +### 我需要更高的 LLM-based ASR 吞吐 + +Fun-ASR-Nano 走 vLLM 路径。请用自己的音频分布做 benchmark,并关注 GPU 显存、tensor parallel size、首 token 延迟和 warmup 时间。 + +## 上线检查清单 + +- 选择模型 alias,并写入部署说明。 +- 记录 FunASR 版本、模型版本、设备、CUDA/PyTorch 版本、Docker 镜像 tag 和启动命令。 +- 跑一个公开短音频 smoke sample,再跑至少一个真实私有样本。 +- 每次请求记录音频时长、模型、设备、延迟、响应格式和错误类型。 +- API 暴露到可信网络外之前,增加上传大小限制、鉴权、TLS 和限流。 +- 流式场景需要测试静音、噪声、多人重叠、长连接、重连和慢客户端。 +- 发布 benchmark 结论时,说明输入时长、硬件、batch size、模型、运行路径,以及是否排除模型下载和 warmup 时间。 + +## 什么时候开 issue + +Runtime、Docker、vLLM、Triton、Android、浏览器或 Agent 集成问题,请使用 [Deployment Help](https://github.com/modelscope/FunASR/issues/new?template=deployment_help.md)。请附上部署路径、完整命令/config、日志、模型、设备和音频特征。 diff --git a/docs/index.rst b/docs/index.rst index a80518c1a..fdbe8dc7d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -71,6 +71,8 @@ Overview :maxdepth: 1 :caption: Runtime and Service + ./deployment_matrix.md + ./deployment_matrix_zh.md ./runtime/readme.md ./runtime/docs/SDK_tutorial_online.md ./runtime/docs/SDK_tutorial.md diff --git a/docs/use_case_showcase.md b/docs/use_case_showcase.md index 8ff30c27a..0cb1dae92 100644 --- a/docs/use_case_showcase.md +++ b/docs/use_case_showcase.md @@ -10,6 +10,7 @@ FunASR is useful far beyond a single offline transcription command. This page co | Compare accuracy and speed | [Benchmark report](https://modelscope.github.io/FunASR/benchmark.html) | Reproduce the 184-file long-audio benchmark before choosing a model. | | Build a private speech API | [OpenAI-compatible API example](../examples/openai_api/) and [client recipes](../examples/openai_api/CLIENTS.md) | Reuse LangChain, Dify, AutoGen, and other OpenAI-style clients without sending audio to a cloud ASR provider. | | Add speech input to agents | [MCP server](../examples/mcp_server/) and [voice input](../examples/voice_input/) | Connect local ASR to Claude, Cursor, and desktop agent workflows. | +| Choose a deployment path | [Deployment matrix](./deployment_matrix.md) | Compare Python API, OpenAI API, Docker Compose, WebSocket, vLLM, MCP, batch, subtitles, and Triton. | | Serve streaming ASR | [Runtime service docs](../runtime/readme.md) | Run WebSocket or service-mode ASR for live captioning and call-center style workloads. | | Accelerate LLM-based ASR | [vLLM guide](./vllm_guide.md) | Use tensor parallel decoding and streaming service support for Fun-ASR-Nano. | | Generate subtitles | [Subtitle example](../examples/subtitle/) | Turn long audio or video into subtitle files for media workflows. | diff --git a/docs/use_case_showcase_zh.md b/docs/use_case_showcase_zh.md index d111a0cc7..84264b7d7 100644 --- a/docs/use_case_showcase_zh.md +++ b/docs/use_case_showcase_zh.md @@ -10,6 +10,7 @@ FunASR 不只是一个离线转写命令。这个页面把常见的评测、部 | 对比准确率和速度 | [性能评测报告](https://modelscope.github.io/FunASR/zh/benchmark.html) | 选型前先查看 184 条长音频评测结果。 | | 搭建私有语音 API | [OpenAI 兼容 API 示例](../examples/openai_api/) 和 [客户端配方](../examples/openai_api/CLIENTS.md) | 复用 LangChain、Dify、AutoGen 等 OpenAI 风格客户端,音频不出内网。 | | 给 Agent 增加语音输入 | [MCP 服务](../examples/mcp_server/) 和 [语音输入示例](../examples/voice_input/) | 将本地 ASR 接入 Claude、Cursor 和桌面 Agent 工作流。 | +| 选择部署路径 | [部署选型表](./deployment_matrix_zh.md) | 对比 Python API、OpenAI API、Docker Compose、WebSocket、vLLM、MCP、批处理、字幕和 Triton。 | | 部署流式 ASR | [Runtime 服务文档](../runtime/readme_cn.md) | 面向实时字幕、客服、会议等低延迟场景。 | | 加速 LLM-based ASR | [vLLM 指南](./vllm_guide.md) | 为 Fun-ASR-Nano 使用 tensor parallel 解码和流式服务能力。 | | 生成字幕 | [字幕示例](../examples/subtitle/) | 将长音频或视频转成字幕文件。 |