modelscope/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR.git synced 2026-10-11 22:39:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: clarify low-code workflow deployment and response contracts

Browse Source 
Signed-off-by: zhifu gao <[email protected]>
This commit is contained in:
zhifu gao
2026-09-07 16:08:47 +00:00
parent 42ee5421fc
commit 160494c4c4
3 changed files with 375 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/openai_api/WORKFLOWS.md
+87 -61
View File
@@ -2,65 +2,89 @@
[中文](WORKFLOWS_zh.md)
Use this guide when you want Dify, n8n, webhook workers, or another workflow engine to call a private FunASR speech API. Start with the local smoke test in this directory, then replace `localhost` with the reachable service name inside your network.
Use this guide when you want Dify, n8n, webhook workers, or another workflow engine to call a private FunASR speech API. These are multipart HTTP recipes, not a compatibility guarantee for every workflow product or version.
The example server has **no built-in authentication or upload limit**. A client URL containing `localhost` does not restrict the server's listening address. Keep local checks on loopback; before sharing, configure TLS, gateway authentication, upload/time/rate limits, private health/model/schema access, and audio/transcript retention using the [security and gateway guide](SECURITY.md). A placeholder API key does not authenticate the server.
## Server preflight
First prepare the checkout and environment in the [example README](README.md#quick-start). Its fixed source revision is not a dependency/model lock or proof of a clean installation. The commands below start in that checkout's root, with its `.venv` already prepared. If the server is already running, skip startup and proceed to the checks; do not start a second process on the same port.
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
source ../../.venv/bin/activate
python server.py --host 127.0.0.1 --model sensevoice --device cpu --port 8000
```
From the workflow host or container:
After preparing CUDA dependencies, replace the CPU command with `python server.py --host 127.0.0.1 --model sensevoice --device cuda --port 8000`. For the distinct packaged `funasr-server` route, use [Agent integration](../../docs/agent_integration.md); startup defaults, aliases and response fields differ from this example.
In a second terminal on the same host, enter the same checkout's `examples/openai_api` directory and activate the same environment. Wait for model loading, then check the local service:
```bash
export FUNASR_BASE_URL=http://<funasr-host>:8000
source ../../.venv/bin/activate
export FUNASR_BASE_URL="http://127.0.0.1:8000"
curl -fsS "$FUNASR_BASE_URL/health"
curl -fsS "$FUNASR_BASE_URL/v1/models"
curl -fsS "$FUNASR_BASE_URL/openapi.json"
```
If the workflow engine runs in Docker, `localhost` usually means the workflow container itself. Use a Docker Compose service name, Kubernetes service name, or LAN host name instead.
Health and schema checks do not establish acoustic correctness. For transcription, use an existing local audio file as `meeting.wav` below; the README's public Chinese smoke sample is not a multilingual accuracy benchmark.
If the workflow engine runs in Docker, `localhost` usually means the workflow container itself. A host-loopback server is not automatically reachable from that container. Deliberately configure a private gateway/container network, then replace `FUNASR_BASE_URL` and the worker's `FUNASR_URL` with the address reachable from the workflow runtime. Use real gateway credentials when required; the local curl/Python examples below do not add authentication headers. Do not solve connectivity by exposing an unauthenticated port publicly.
## Postman smoke test
Before configuring a low-code tool, you can import the [Postman collection](POSTMAN.md) and run health, model-list, and transcription requests from a GUI. For schema-driven imports, use the [OpenAPI spec](OPENAPI.md). Set `FUNASR_BASE_URL`, choose a local audio file for the multipart `file` field, and keep `MODEL_ALIAS=sensevoice` for the first test.
For offline multi-speaker meetings, set `MODEL_ALIAS=moss-transcribe-diarize`
and keep `response_format=verbose_json` so downstream nodes receive native
anonymous speaker segments. Use the [MOSS deployment guide](../../docs/moss_transcribe_diarize.md)
for the isolated GPU service and file-duration limits.
For offline multi-speaker meetings, first prepare the isolated third-party MOSS service in the [MOSS deployment guide](../../docs/moss_transcribe_diarize.md), including its GPU and file-duration limits. Then set `MODEL_ALIAS=moss-transcribe-diarize` and keep `response_format=verbose_json` to preserve available native anonymous speaker segments. Do not add external VAD or `spk=true`; recording-local labels are not verified identities or stable IDs across recordings. Changing a client alias alone does not prepare that service.
## Multipart HTTP request
Every workflow engine eventually needs to send this request shape:
| Field | Value |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Body type | `multipart/form-data` |
| File field | `file` |
| Text field | `model=sensevoice` |
| Text field | `response_format=verbose_json` |
| Timeout | Set according to maximum audio duration, for example 300 seconds for long files. |
- **Method:** `POST`
- **URL:** `http://<funasr-host>:8000/v1/audio/transcriptions`
- **Body type:** `multipart/form-data`
- **File field:** `file`
- **Text field:** `model=sensevoice`
- **Text field:** `response_format=verbose_json`
- **Timeout:** Set according to maximum audio duration, for example 300 seconds for long files.
Equivalent curl command:
```bash
curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \
curl -fsS "$FUNASR_BASE_URL/v1/audio/transcriptions" \
-F file=@meeting.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
Typical JSON fields to map downstream:
Map `text` to the transcript. `response_format=verbose_json` selects a format: it does not enable diarization or force timestamps. Check the [client response contract](CLIENTS.md#response-formats) before consuming the other fields:
| Path | Use |
|---|---|
| `text` | Plain transcript for a chatbot, ticket, or knowledge-base step. |
| `segments` | Timestamps and speaker labels when `verbose_json` is requested. |
| `duration` | Audio processing time reported by the API, useful for logs. |
| `model` | Model alias used for the request. |
- **Example `server.py`:** `segments` comes only from model-provided `sentence_info`, otherwise `segments=[]`. `duration` is elapsed seconds around `generate()`, excluding initial model loading, not recording length. `model` is the resolved request alias; `language` echoes the submitted hint or `auto`, not detected language.
- **Packaged `funasr-server`:** `duration` is audio length in seconds, with 0 possible if fallback audio metadata cannot be read. It can return coarse text-based segments, not forced alignment. Its verbose response includes `task` and segment `id`/`words`, but no top-level `model`; language can use backend detection. For non-native diarization models, `spk=true` opts into a separate speaker pipeline, subject to its model/dependencies. The example has no `spk` form field.
Segment `start`/`end` use seconds, not the SDK's millisecond coordinates. A `speaker` field can be absent, null, numeric or a string; labels do not identify a person. Neither nonempty segments nor `verbose_json` guarantees accurate subtitle alignment. HTTP display text strips SenseVoice rich tags; it is not the SDK raw-tag result or a dedicated emotion/event response. SDK fields/options such as `timestamp`, `timestamps`, `ctc_timestamps`, `use_itn`, hotwords and raw arrays are not additional form fields; raw SDK timestamps are not automatically converted into HTTP segments.
Illustrative example-server response with no `sentence_info`; `0.42` is processing time, not audio duration:
```json
{"text": "recognized speech", "segments": [], "language": "auto", "duration": 0.42, "model": "sensevoice"}
```
Illustrative packaged-server response for a 3.2-second file and a coarse fallback segment, without speaker opt-in. These examples describe schemas, not new model measurements:
```json
{
"task": "transcribe",
"language": "en",
"duration": 3.2,
"text": "recognized speech",
"segments": [
{"id": 0, "start": 0.0, "end": 3.2, "text": "recognized speech", "words": []}
]
}
```
## Dify custom tool or HTTP node
@@ -75,21 +99,33 @@ Configure an HTTP request node or custom tool with:
- Body: `multipart/form-data`
- File part: `file`, bound to the uploaded audio variable
- Text parts: `model=sensevoice`, `response_format=verbose_json`
- Output variable: map `text` as the transcript, and keep `segments` when timestamps or speaker labels matter
- Output variable: map `text` as the transcript; inspect `segments` availability and provenance before using timestamps or speaker labels
### Audio URL path
Some workflow tools pass a file URL rather than raw multipart bytes. In that case, add a small internal worker:
Some workflow tools pass a file URL rather than raw multipart bytes. A URL string in the multipart `file` field is not an audio upload. Prefer direct binary uploads or controlled storage object IDs resolved by a reviewed storage client.
1. Dify sends the audio URL and metadata to the worker.
The following sketch is only for an operator-approved URL in trusted storage. Destination allowlists, private-network access policy, redirect validation, download byte limits and authentication are **not implemented**. `requests.get` follows redirects and buffers the entire response; its timeout is not a byte cap or a complete end-to-end deadline. Do not pass user-supplied URLs to this helper. Before accepting them, require a reviewed download boundary enforcing all those controls, including blocking unintended private/metadata destinations and checking each redirect. Being inside a trusted network is not an SSRF defense.
For this trusted-input illustration:
1. An operator supplies an approved audio URL and metadata to the worker.
2. The worker downloads the file from trusted storage.
3. The worker posts multipart data to FunASR.
4. The worker returns `text`, `segments`, and operational logs to Dify.
4. The worker returns the service JSON; downstream nodes check optional fields. Logs must not expose signed URLs, credentials or private transcripts.
In the same activated client environment, install the separate HTTP dependency:
```bash
python -m pip install requests
```
Set `FUNASR_URL` for the prepared service; this default is for a worker on the same host. The function definitions can be imported by your worker, but do not create an HTTP listener or implement inbound authentication/upload limits.
```python
import requests
FUNASR_URL = "http://funasr-api:8000/v1/audio/transcriptions"
FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions"
def transcribe_from_url(audio_url: str) -> dict:
audio_response = requests.get(audio_url, timeout=120)
@@ -101,7 +137,7 @@ def transcribe_from_url(audio_url: str) -> dict:
return response.json()
```
Keep this worker inside your trusted network and validate allowed URL domains before downloading user-provided links.
Keep this illustration restricted to approved inputs; hostname validation alone is not a complete safe-download policy.
## n8n HTTP Request node
@@ -109,39 +145,31 @@ A common n8n flow is: trigger -> binary audio data -> HTTP Request -> transcript
Recommended HTTP Request settings:
| n8n setting | Value |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Send Body | enabled |
| Body Content Type | `Form-Data` / multipart |
| Binary file field | `file` |
| Additional form fields | `model=sensevoice`, `response_format=verbose_json` |
| Response Format | JSON |
| Timeout | Increase for long recordings. |
- **Method:** `POST`
- **URL:** `http://<funasr-host>:8000/v1/audio/transcriptions`
- **Send Body:** enabled
- **Body Content Type:** `Form-Data` / multipart
- **Binary file field:** `file`
- **Additional form fields:** `model=sensevoice`, `response_format=verbose_json`
- **Response Format:** JSON
- **Timeout:** Increase for long recordings.
After the request, use `{{$json.text}}` as the transcript. If `verbose_json` is enabled, route `{{$json.segments}}` to subtitle, speaker, or QA steps.
After the request, use `{{$json.text}}` as the transcript. Route `{{$json.segments}}` onward only after checking it exists and is useful for the task; empty or coarse segments must not be treated as verified subtitle timing or diarization. Node labels and binary-property configuration vary with the installed n8n version; `file` is the outgoing multipart field, not necessarily the name of the incoming binary property.
### n8n OpenAI Audio node
The built-in OpenAI node can also run its Audio > Transcribe operation against
FunASR. In OpenAI credentials, set Base URL to
`http://<funasr-host>:8000/v1` and provide any non-empty API key. The node
always sends `model=whisper-1`; FunASR maps that compatibility alias to the
model selected when the server starts. This path is transcription-only. Use the
HTTP Request node above when you need `verbose_json`, segments, or speaker
labels.
For OpenAI Audio > Transcribe node versions that send `model=whisper-1`, FunASR maps that compatibility alias to the model selected at server startup; it does not select a Whisper checkpoint. Set Base URL to your reachable service URL with `/v1`. A non-empty placeholder key is only for an unprotected local endpoint; supply real credentials for an authenticated gateway. Verify the installed node version's request behavior rather than assuming all versions match. Use this recipe for plain transcription; use the HTTP Request node for explicit `response_format` and any supported speaker opt-in, still subject to the response limits above.
## Webhook worker pattern
Use this when the workflow engine cannot send multipart files reliably or when audio needs pre-processing.
Use this when the workflow engine cannot send multipart files reliably or when audio needs pre-processing. This POSIX temporary-file example uses the same `requests` dependency and closes the upload handle. It accepts bytes already held in memory: apply upload limits before buffering them. It is a function, not a protected webhook server.
```python
from pathlib import Path
import tempfile
import requests
FUNASR_URL = "http://localhost:8000/v1/audio/transcriptions"
FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions"
def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/wav") -> dict:
with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix or ".wav") as tmp:
@@ -158,7 +186,7 @@ def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/w
return response.json()
```
This worker is also the right place to add audio conversion, file-size checks, request IDs, authentication, and retries.
Audio conversion, file-size checks, request IDs, inbound/upstream authentication and retry policy are not implemented here. Enforce them at the appropriate boundary before shared use; retries can repeat expensive transcription work.
## Production guardrails
@@ -171,11 +199,9 @@ This worker is also the right place to add audio conversion, file-size checks, r
## Troubleshooting
| Symptom | Fix |
|---|---|
| Workflow can call `/health` but transcription fails | Confirm the request is `multipart/form-data` and the binary field is named `file`. |
| `localhost` connection fails from Dify or n8n | Use the host, Compose service, or Kubernetes service reachable from the workflow runtime. |
| Response has no `segments` | Set `response_format=verbose_json`. |
| Requests time out | Increase HTTP timeout or split long recordings. |
| First request is slow | Preload the model with `--model sensevoice` and use `/health` as a readiness check. |
| Unknown model alias | Call `/v1/models` and use one of the returned aliases. |
- **Workflow can call `/health` but transcription fails:** Confirm the request is `multipart/form-data` and the binary field is named `file`.
- **`localhost` connection fails from Dify or n8n:** Use the host, Compose service, or Kubernetes service reachable from the workflow runtime.
- **Response has no usable `segments`:** Check the format and deployed schema, then the model's `sentence_info` and speaker configuration; `verbose_json` alone cannot create timestamps or labels.
- **Requests time out:** Increase HTTP timeout or split long recordings.
- **First request is slow:** Preload the model with `--model sensevoice` and use `/health` as a readiness check.
- **Unknown model alias:** Call `/v1/models` and use one of the returned aliases.
examples/openai_api/WORKFLOWS_zh.md
+87 -58
View File
@@ -2,64 +2,89 @@
[English](WORKFLOWS.md)
当你希望让 Dify、n8n、HTTP 节点、webhook worker 或其他低代码工作流引擎调用私有 FunASR 语音 API 时，可以从这份配方开始。建议先在本目录跑通本地 smoke test，再把 `localhost` 换成工作流运行环境能够访问到的服务名或内网地址。
当你希望让 Dify、n8n、HTTP 节点、webhook worker 或其他低代码工作流引擎调用私有 FunASR 语音 API 时，可以从这份配方开始。这些是 multipart HTTP 接入示例，不是对所有工作流产品或版本的兼容保证。
示例服务**没有内置鉴权或上传大小限制**。客户端 URL 中的 `localhost` 不会限制服务端监听地址。本地检查只绑定 loopback；共享前按[安全与网关指南](SECURITY_zh.md)配置 TLS、网关鉴权、上传/时限/速率限制、health/model/schema 私有访问和音频/转写保留策略。占位 API key 不会让服务具备鉴权。
## 服务预检
先按[示例 README](README_zh.md#快速开始)准备 checkout 和环境。其中固定的源码 revision 不是依赖/模型锁定，也不代表已验证全新安装。以下命令从该 checkout 根目录开始，假定 `.venv` 已准备好。如果服务已经运行，跳过启动直接检查，不要在同一端口启动第二个进程。
```bash
cd examples/openai_api
python server.py --model sensevoice --device cuda --port 8000
source ../../.venv/bin/activate
python server.py --host 127.0.0.1 --model sensevoice --device cpu --port 8000
```
在工作流所在主机或容器中执行：
准备好 CUDA 依赖后，可将 CPU 命令替换为 `python server.py --host 127.0.0.1 --model sensevoice --device cuda --port 8000`。另一个实现即打包的 `funasr-server` 路线见 [Agent 集成指南](../../docs/agent_integration_zh.md)，其启动默认值、别名和响应字段与本示例不同。
在同一主机的第二个终端进入同一 checkout 的 `examples/openai_api` 目录，激活同一环境。等待模型加载后检查本地服务：
```bash
export FUNASR_BASE_URL=http://<funasr-host>:8000
source ../../.venv/bin/activate
export FUNASR_BASE_URL="http://127.0.0.1:8000"
curl -fsS "$FUNASR_BASE_URL/health"
curl -fsS "$FUNASR_BASE_URL/v1/models"
curl -fsS "$FUNASR_BASE_URL/openapi.json"
```
如果工作流引擎运行在 Docker 中，`localhost` 通常指的是工作流容器自身。请改用 Docker Compose service name、Kubernetes service name 或内网主机名。
健康与 schema 检查不验证声学正确性。下方转写命令的 `meeting.wav` 应替换为已有本地音频；README 的公开中文 smoke 样例不是多语言准确率基准。
如果工作流引擎运行在 Docker 中，`localhost` 通常指的是工作流容器自身；主机 loopback 服务不会自动对容器可达。请明确配置私有网关/容器网络，再将 `FUNASR_BASE_URL` 和 worker 的 `FUNASR_URL` 替换为工作流运行时能访问的地址。需要网关鉴权时使用真实凭据；下方本地 curl/Python 示例没有添加鉴权 header。不要为解决连通性而把未鉴权端口直接暴露到公网。
## Postman smoke test
在配置低代码工具前，可以先导入 [Postman collection](POSTMAN_zh.md)，从图形界面跑通 health、模型列表和转写请求；需要按 schema 导入时可使用 [OpenAPI spec](OPENAPI_zh.md)。设置 `FUNASR_BASE_URL`，在 multipart `file` 字段选择本地音频文件，第一次测试建议保持 `MODEL_ALIAS=sensevoice`。
处理离线多人会议时，将 `MODEL_ALIAS` 改为 `moss-transcribe-diarize`，并保留
`response_format=verbose_json`，下游节点即可收到模型原生匿名说话人 segments。
独立 GPU 服务与文件时长边界见 [MOSS 部署指南](../../docs/moss_transcribe_diarize_zh.md)。
处理离线多人会议时，先按 [MOSS 部署指南](../../docs/moss_transcribe_diarize_zh.md)准备独立的第三方 MOSS 服务，包括 GPU 和文件时长边界。再将 `MODEL_ALIAS` 改为 `moss-transcribe-diarize`，保留 `response_format=verbose_json` 以保留可用的原生匿名说话人 segments。不要添加外部 VAD 或 `spk=true`；录音内标签不是已验证身份，也不是跨录音稳定编号。只修改客户端别名不会完成服务环境准备。
## Multipart HTTP 请求
所有工作流引擎最终都需要发出下面这种请求：
| 字段 | 值 |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Body type | `multipart/form-data` |
| File field | `file` |
| Text field | `model=sensevoice` |
| Text field | `response_format=verbose_json` |
| Timeout | 根据最长音频时长设置，例如长录音可先设为 300 秒。 |
- **Method:** `POST`
- **URL:** `http://<funasr-host>:8000/v1/audio/transcriptions`
- **Body type:** `multipart/form-data`
- **File field:** `file`
- **Text field:** `model=sensevoice`
- **Text field:** `response_format=verbose_json`
- **Timeout:** 根据最长音频时长设置，例如长录音可先设为 300 秒。
等价 curl 命令：
```bash
curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \
curl -fsS "$FUNASR_BASE_URL/v1/audio/transcriptions" \
-F file=@meeting.wav \
-F model=sensevoice \
-F response_format=verbose_json
```
常用响应字段映射：
将 `text` 映射为转写文本。`response_format=verbose_json` 只选择格式，不开启说话人分离，也不强制生成时间戳。消费其他字段前，先看[客户端响应契约](CLIENTS.md#response-formats)：
| 路径 | 用途 |
|---|---|
| `text` | 纯文本转写结果，可传给聊天机器人、工单、知识库或后续摘要节点。 |
| `segments` | 请求 `verbose_json` 时返回的时间戳、说话人等分段信息。 |
| `duration` | API 返回的音频处理时长，适合写入日志和监控。 |
| `model` | 本次请求使用的模型别名。 |
- **示例 `server.py`：** `segments` 只来自模型返回的 `sentence_info`，否则为 `segments=[]`。`duration` 是 `generate()` 调用周围的耗时秒数，不含初次模型加载，不是录音时长。`model` 是解析后的请求别名；`language` 回显提交的提示或 `auto`，不是检测出的语言。
- **打包 `funasr-server`：** `duration` 是音频时长秒数，fallback 无法读取音频元数据时可能为 0。它可能返回按文本生成的粗粒度片段，不是强制对齐。verbose 响应有 `task` 和片段 `id`/`words`，但没有顶层 `model`；语言可使用后端检测结果。对非原生说话人模型，`spk=true` 请求独立说话人流程，仍依赖其模型和环境。示例服务没有 `spk` 表单字段。
片段 `start`/`end` 单位为秒，不是 SDK 的毫秒坐标。`speaker` 字段可能缺失、为 null、数字或字符串；标签不标识真实身份。非空片段或 `verbose_json` 都不保证准确字幕对齐。HTTP 展示文本会剥离 SenseVoice 富标签，不等于 SDK 原始标签结果，也不是独立的情绪/事件响应。SDK 字段/选项如 `timestamp`、`timestamps`、`ctc_timestamps`、`use_itn`、hotwords 和原始数组不是额外表单字段；原始 SDK 时间戳不会自动转成 HTTP segments。
以下是没有 `sentence_info` 的示例服务响应；`0.42` 是处理耗时，不是音频时长：
```json
{"text": "recognized speech", "segments": [], "language": "auto", "duration": 0.42, "model": "sensevoice"}
```
以下是 3.2 秒文件对应的打包服务响应示意，含粗粒度 fallback 片段，未开启说话人选项。这些示例说明 schema，不是新模型测量：
```json
{
"task": "transcribe",
"language": "en",
"duration": 3.2,
"text": "recognized speech",
"segments": [
{"id": 0, "start": 0.0, "end": 3.2, "text": "recognized speech", "words": []}
]
}
```
## Dify 自定义工具或 HTTP 节点
@@ -74,21 +99,33 @@ curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \
- Body: `multipart/form-data`
- File part: `file`，绑定到上传音频变量
- Text parts: `model=sensevoice`、`response_format=verbose_json`
- Output variable: 把 `text` 映射为转写文本；需要时间戳或说话人信息时保留 `segments`
- Output variable: 把 `text` 映射为转写文本；使用时间戳或说话人标签前检查 `segments` 是否可用及其来源
### 音频 URL 转写
有些工作流工具只能传文件 URL，而不能直接传 multipart 二进制。此时建议加一个内网 worker：
有些工作流工具只能传文件 URL，而不能直接传 multipart 二进制。multipart `file` 字段中的 URL 字符串不是音频上传。优先直接上传二进制，或传受控存储对象 ID 并由经过审查的存储客户端解析。
1. Dify 将音频 URL 和元数据发送给 worker。
下面的 sketch 仅用于运维人员批准的可信存储 URL。目标 allowlist、私网访问策略、重定向校验、下载字节上限与鉴权均**未实现**。`requests.get` 会跟随重定向并将整个响应缓存在内存中；其 timeout 不是字节限制，也不是完整的端到端截止时限。不要把用户提供的 URL 传给这个 helper。接受这些 URL 前，必须使用经过审查的下载边界执行全部控制，包括阻止非预期私网/元数据目标并检查每次重定向。位于可信内网并不能防止 SSRF。
对于这个可信输入示意：
1. 运维人员向 worker 提供已批准的音频 URL 和元数据。
2. worker 从可信存储下载音频。
3. worker 使用 multipart 请求调用 FunASR。
4. worker 将 `text`、`segments` 和运行日志返回给 Dify。
4. worker 返回服务 JSON，由下游检查可选字段。日志不得暴露签名 URL、凭据或私有转写。
在同一已激活的客户端环境安装独立 HTTP 依赖：
```bash
python -m pip install requests
```
按已准备的服务设置 `FUNASR_URL`；默认值用于同主机 worker。工作流可导入以下函数定义，但它们不创建 HTTP 监听器，也不实现入站鉴权/上传限制。
```python
import requests
FUNASR_URL = "http://funasr-api:8000/v1/audio/transcriptions"
FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions"
def transcribe_from_url(audio_url: str) -> dict:
audio_response = requests.get(audio_url, timeout=120)
@@ -100,7 +137,7 @@ def transcribe_from_url(audio_url: str) -> dict:
return response.json()
```
请把这个 worker 放在可信内网中，并在下载用户提供的链接前校验允许访问的域名或存储桶。
请把此示意限制在已批准输入内；仅校验主机名不是完整的安全下载策略。
## n8n HTTP Request 节点
@@ -108,37 +145,31 @@ def transcribe_from_url(audio_url: str) -> dict:
推荐 HTTP Request 配置：
| n8n 设置 | 值 |
|---|---|
| Method | `POST` |
| URL | `http://<funasr-host>:8000/v1/audio/transcriptions` |
| Send Body | enabled |
| Body Content Type | `Form-Data` / multipart |
| Binary file field | `file` |
| Additional form fields | `model=sensevoice`、`response_format=verbose_json` |
| Response Format | JSON |
| Timeout | 长录音场景需要调大。 |
- **Method:** `POST`
- **URL:** `http://<funasr-host>:8000/v1/audio/transcriptions`
- **Send Body:** enabled
- **Body Content Type:** `Form-Data` / multipart
- **Binary file field:** `file`
- **Additional form fields:** `model=sensevoice`、`response_format=verbose_json`
- **Response Format:** JSON
- **Timeout:** 长录音场景需要调大。
请求之后，使用 `{{$json.text}}` 作为转写文本。如果启用了 `verbose_json`，可以把 `{{$json.segments}}` 传给字幕、说话人分析或质检节点。
请求之后，使用 `{{$json.text}}` 作为转写文本。先检查 `{{$json.segments}}` 存在且适合任务，再传给后续节点；空片段或粗粒度片段不能作为已验证字幕时刻或说话人分离结果。节点标签和二进制属性配置随所安装的 n8n 版本变化；`file` 是发出的 multipart 字段名，不一定是传入二进制属性的名字。
### n8n OpenAI Audio 节点
也可以让内置 OpenAI 节点的 Audio > Transcribe 操作调用 FunASR。在 OpenAI
凭据中将 Base URL 设为 `http://<funasr-host>:8000/v1`，并填写任意非空 API
key。该节点固定发送 `model=whisper-1`，FunASR 会将这个兼容别名映射到服务
启动时选择的模型。此路径只返回转写文本；需要 `verbose_json`、分段或说话人
标签时，仍使用上面的 HTTP Request 节点。
对于发送 `model=whisper-1` 的 OpenAI Audio > Transcribe 节点版本，FunASR 会把该兼容别名映射到服务启动时选择的模型，而不是选择 Whisper checkpoint。Base URL 使用可达服务地址并带 `/v1`。非空占位 key 仅适用于未保护的本地端点，访问鉴权网关时应提供真实凭据。请核验所安装节点版本的请求行为，不要假定所有版本相同。此配方用于纯文本转写；需要显式 `response_format` 或服务支持的说话人选项时使用 HTTP Request 节点，仍受上述响应边界限制。
## Webhook worker 模式
当工作流引擎不能稳定发送 multipart 文件，或音频需要预处理时，可以把转写封装成一个内部 webhook worker。
当工作流引擎不能稳定发送 multipart 文件，或音频需要预处理时，可以使用这个函数。此 POSIX 临时文件示例使用同一个 `requests` 依赖，并关闭上传句柄。它接受已在内存中的 bytes，应在缓冲之前限制上传大小；它是函数，不是受保护的 webhook 服务。
```python
from pathlib import Path
import tempfile
import requests
FUNASR_URL = "http://localhost:8000/v1/audio/transcriptions"
FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions"
def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/wav") -> dict:
with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix or ".wav") as tmp:
@@ -155,7 +186,7 @@ def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/w
return response.json()
```
这个 worker 也适合集中做音频转码、文件大小限制、请求 ID、鉴权、重试和审计日志。
这里未实现音频转码、文件大小检查、请求 ID、入站/上游鉴权和重试策略。共享前在相应边界执行这些控制；重试可能重复执行昂贵的转写工作。
## 生产环境护栏
@@ -168,11 +199,9 @@ def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/w
## 故障排查
| 现象 | 处理方式 |
|---|---|
| 工作流能访问 `/health`，但转写失败 | 确认请求是 `multipart/form-data`，且二进制字段名是 `file`。 |
| Dify 或 n8n 访问 `localhost` 失败 | 换成工作流运行时可访问的主机名、Compose service name 或 Kubernetes service name。 |
| 响应中没有 `segments` | 设置 `response_format=verbose_json`。 |
| 请求超时 | 调大 HTTP timeout，或先切分长录音。 |
| 第一次请求很慢 | 使用 `--model sensevoice` 预加载模型，并用 `/health` 做 readiness check。 |
| 模型别名未知 | 调用 `/v1/models`，使用返回列表中的别名。 |
- **工作流能访问 `/health`，但转写失败:** 确认请求是 `multipart/form-data`，且二进制字段名是 `file`。
- **Dify 或 n8n 访问 `localhost` 失败:** 换成工作流运行时可访问的主机名、Compose service name 或 Kubernetes service name。
- **响应中没有可用 `segments`:** 检查格式和已部署 schema，再检查模型的 `sentence_info` 与说话人配置；仅设置 `verbose_json` 不能创建时间戳或标签。
- **请求超时:** 调大 HTTP timeout，或先切分长录音。
- **第一次请求很慢:** 使用 `--model sensevoice` 预加载模型，并用 `/health` 做 readiness check。
- **模型别名未知:** 调用 `/v1/models`，使用返回列表中的别名。
tests/test_service_docs_contract.py
+201
View File
@@ -6,6 +6,8 @@ from pathlib import Path
import re
import shlex
import subprocess
import tempfile
from types import SimpleNamespace
import unittest
from urllib.parse import unquote, urlsplit
@@ -15,6 +17,13 @@ EXAMPLE = ROOT / "examples/openai_api"
READMES = [EXAMPLE / name for name in ("README.md", "README_zh.md", "README_ja.md", "README_ko.md")]
DOCS = [EXAMPLE / "CLIENTS.md", *READMES]
PACKAGED = ROOT / "funasr/bin/_server_app.py"
WORKFLOW_DOCS = [EXAMPLE / name for name in ("WORKFLOWS.md", "WORKFLOWS_zh.md")]
# Frozen from the two pre-edit workflow guides, not translated from one language.
WORKFLOW_HEADINGS = {
"WORKFLOWS.md": "Low-code Workflow Recipes for the FunASR OpenAI-Compatible API|Server preflight|Postman smoke test|Multipart HTTP request|Dify custom tool or HTTP node|Direct file upload path|Audio URL path|n8n HTTP Request node|n8n OpenAI Audio node|Webhook worker pattern|Production guardrails|Troubleshooting",
"WORKFLOWS_zh.md": "FunASR OpenAI 兼容 API 低代码工作流配方|服务预检|Postman smoke test|Multipart HTTP 请求|Dify 自定义工具或 HTTP 节点|直接上传文件|音频 URL 转写|n8n HTTP Request 节点|n8n OpenAI Audio 节点|Webhook worker 模式|生产环境护栏|故障排查",
}
# Frozen from the pre-edit README inventory, excluding fenced code comments.
LEGACY_HEADINGS = {
@@ -317,5 +326,197 @@ class ServiceDocsContract(unittest.TestCase):
self.assertIn("does not enable diarization", clients)
class WorkflowDocsContract(unittest.TestCase):
def test_workflow_explanatory_sections_use_complete_readable_lists(self):
sections = [
(("Multipart HTTP request", "Multipart HTTP 请求"),
["Method", "URL", "Body type", "File field", "Text field", "Text field", "Timeout"],
[("POST",), ("/v1/audio/transcriptions",), ("multipart/form-data",), ("`file`",),
("model=sensevoice",), ("response_format=verbose_json",), ("300",)]),
(("n8n HTTP Request node", "n8n HTTP Request 节点"),
["Method", "URL", "Send Body", "Body Content Type", "Binary file field", "Additional form fields", "Response Format", "Timeout"],
[("POST",), ("/v1/audio/transcriptions",), ("enabled",), ("Form-Data", "multipart"),
("`file`",), ("model=sensevoice", "response_format=verbose_json"), ("JSON",), ()]),
]
symptoms = [
["Workflow can call `/health` but transcription fails", "`localhost` connection fails from Dify or n8n",
"Response has no usable `segments`", "Requests time out", "First request is slow", "Unknown model alias"],
["工作流能访问 `/health`，但转写失败", "Dify 或 n8n 访问 `localhost` 失败", "响应中没有可用 `segments`",
"请求超时", "第一次请求很慢", "模型别名未知"],
]
fixes = [("multipart/form-data", "`file`"), ("Compose", "Kubernetes"),
("sentence_info", "verbose_json"), ("HTTP timeout",), ("--model sensevoice", "/health"), ("/v1/models",)]
for language, path in enumerate(WORKFLOW_DOCS):
text = path.read_text(encoding="utf-8")
cases = [*sections, (("Troubleshooting", "故障排查"), symptoms[language], fixes)]
for titles, labels, tokens in cases:
with self.subTest(path=path.name, section=titles[language]):
section = text.split("## " + titles[language] + "\n", 1)[1].split("\n## ", 1)[0]
self.assertNotRegex(section, r"(?m)^\s*\|")
self.assertNotRegex(section, r"(?i)<table\b")
lines = section.splitlines()
start = next((i for i, line in enumerate(lines) if line.startswith("- **")), None)
self.assertIsNotNone(start, "Missing explanation list")
items = []
for line in lines[start:]:
if not line.startswith("- "):
break
match = re.fullmatch(r"- \*\*(.+?):\*\* (.+)", line)
self.assertIsNotNone(match, line)
items.append(match.groups())
self.assertEqual([label for label, _ in items], labels)
for (_, value), required in zip(items, tokens):
for token in required:
self.assertIn(token, value)
def test_preserves_workflow_headings_and_links(self):
for path in WORKFLOW_DOCS:
text = path.read_text(encoding="utf-8")
source = re.sub(r"^```[^\n]*\n.*?^```[^\n]*$", "", text, flags=re.M | re.S)
actual = re.findall(r"^(#{1,6}) (.+)$", source, re.M)
titles = WORKFLOW_HEADINGS[path.name].split("|")
levels = [1, 2, 2, 2, 2, 3, 3, 2, 3, 2, 2, 2]
self.assertEqual([(len(level), title) for level, title in actual], list(zip(levels, titles)))
suffix = "_zh" if path.stem.endswith("_zh") else ""
preserved = {"WORKFLOWS.md" if suffix else "WORKFLOWS_zh.md",
f"POSTMAN{suffix}.md", f"OPENAPI{suffix}.md", f"SECURITY{suffix}.md",
f"../../docs/moss_transcribe_diarize{suffix}.md"}
links = set(re.findall(r"\[[^\]]+\]\(([^)]+)\)", text))
self.assertLessEqual(preserved, links)
for link in links:
parts = urlsplit(link)
if parts.scheme or parts.netloc:
continue
target = (path.parent / unquote(parts.path)).resolve()
self.assertIn(ROOT.resolve(), target.parents)
self.assertNotIn(".mcp-tasks", target.parts)
self.assertTrue(target.is_file(), link)
if parts.fragment:
self.assertIn(unquote(parts.fragment), headings(target.read_text(encoding="utf-8")), link)
def test_workflow_preflight_uses_prepared_loopback_server(self):
options = {ast.literal_eval(arg) for node in ast.walk(tree(EXAMPLE / "server.py"))
if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute)
and node.func.attr == "add_argument" for arg in node.args
if isinstance(arg, ast.Constant) and isinstance(arg.value, str)}
for path in WORKFLOW_DOCS:
text = path.read_text(encoding="utf-8")
prefix = text.split("```bash", 1)[0]
suffix = "_zh" if path.stem.endswith("_zh") else ""
self.assertIn(f"](README{suffix}.md", prefix)
self.assertIn(f"](SECURITY{suffix}.md)", prefix)
self.assertIn(f"](../../docs/agent_integration{suffix}.md)", text)
startup = blocks(text, "bash")[0]
commands = [shlex.split(line, comments=True) for line in startup.splitlines() if line.strip()]
self.assertIn(["source", "../../.venv/bin/activate"], commands)
launches = [args for args in commands if args[:2] == ["python", "server.py"]]
self.assertEqual(len(launches), 1)
args = launches[0]
for option, expected in (("--host", "127.0.0.1"), ("--model", "sensevoice"), ("--device", "cpu")):
self.assertIn(option, args)
self.assertEqual(args[args.index(option) + 1], expected)
self.assertLessEqual({arg for arg in args if arg.startswith("--")}, options)
def test_workflow_url_assignments_are_copyable_and_checks_are_explicit(self):
for path in WORKFLOW_DOCS:
codes = blocks(path.read_text(encoding="utf-8"), "bash")
assignments = []
for code in codes:
self.assertNotRegex(code, r"<[^\n>]+>")
result = subprocess.run(["bash", "-n"], input=code, text=True, capture_output=True)
self.assertEqual(result.returncode, 0, result.stderr)
for line in code.replace("\\\n", " ").splitlines():
args = shlex.split(line, comments=True)
if args[:1] == ["export"]:
assignments.extend(arg for arg in args[1:] if arg.startswith("FUNASR_BASE_URL="))
self.assertEqual(assignments, ["FUNASR_BASE_URL=http://127.0.0.1:8000"])
for endpoint in ("health", "v1/models", "openapi.json"):
self.assertIn(f'"$FUNASR_BASE_URL/{endpoint}"', "\n".join(codes))
def test_workflow_json_examples_match_actual_service_response_builders(self):
handler = function(EXAMPLE / "server.py", "transcribe")
response = next(node for node in ast.walk(handler) if isinstance(node, ast.Dict)
and any(isinstance(key, ast.Constant) and key.value == "duration" for key in node.keys))
example = eval(compile(ast.Expression(response), "example-response", "eval"),
{"text": "recognized speech", "segments": [], "language": None,
"elapsed": 0.42, "model": "sensevoice"})
namespace = {}
functions = [function(PACKAGED, name) for name in
("_split_text_for_openai_segments", "build_openai_fallback_segments",
"resolve_transcription_language", "build_openai_verbose_json")]
exec(compile(ast.Module(body=functions, type_ignores=[]), str(PACKAGED), "exec"), namespace)
result = {"text": "recognized speech", "duration": 3.2, "language": "en",
"segments": namespace["build_openai_fallback_segments"]("recognized speech", 3.2)}
packaged = namespace["build_openai_verbose_json"](result)
for path in WORKFLOW_DOCS:
text = path.read_text(encoding="utf-8")
samples = [json.loads(code) for code in blocks(text, "json")]
self.assertEqual(samples, [example, packaged], path.name)
self.assertIn("](CLIENTS.md#response-formats)", text)
for marker in ("sentence_info", "segments=[]", "spk=true", "ctc_timestamps", "use_itn", "hotwords"):
self.assertIn(marker, text)
for speaker in (None, 0, "SPK0", "S01"):
result["segments"][0]["speaker"] = speaker
segment = namespace["build_openai_verbose_json"](result)["segments"][0]
if speaker is None:
self.assertNotIn("speaker", segment)
else:
self.assertEqual(segment["speaker"], speaker)
def test_workflow_url_worker_warns_before_code_and_declares_dependency(self):
for path in WORKFLOW_DOCS:
text = path.read_text(encoding="utf-8")
section = text.split("### Audio URL path" if path.name == "WORKFLOWS.md" else "### 音频 URL 转写", 1)[1]
warning = section.split("```python", 1)[0]
markers = ("not implemented", "redirect", "private-network", "byte", "trusted") if path.name == "WORKFLOWS.md" else (
"未实现", "重定向", "私网", "字节", "可信")
for marker in markers:
self.assertIn(marker, warning)
self.assertIn("python -m pip install requests", text[:text.index("```python")])
def test_workflow_python_examples_are_equivalent_and_send_real_multipart_fields(self):
snippets = [blocks(path.read_text(encoding="utf-8"), "python") for path in WORKFLOW_DOCS]
self.assertEqual([len(items) for items in snippets], [2, 2])
self.assertEqual([ast.dump(ast.parse(code)) for code in snippets[0]],
[ast.dump(ast.parse(code)) for code in snippets[1]])
allowed = set(form_defaults(function(EXAMPLE / "server.py", "transcribe"))) | {"file"}
for code, name in zip(snippets[0], ("transcribe_from_url", "transcribe_bytes")):
calls = []
handles = []
payload = b"test audio bytes, no acoustic inference"
def post(url, *, files, data, timeout):
self.assertEqual(urlsplit(url).path, "/v1/audio/transcriptions")
self.assertEqual(set(files), {"file"})
self.assertLessEqual(set(data) | set(files), allowed)
self.assertEqual(data, {"model": "sensevoice", "response_format": "verbose_json"})
self.assertGreater(timeout, 0)
body = files["file"][1]
if hasattr(body, "read"):
handles.append(body)
body = body.read()
self.assertEqual(body, payload)
calls.append(url)
return SimpleNamespace(raise_for_status=lambda: None, json=lambda: {"text": "hello", "segments": []})
downloads = []
def get(url, *, timeout):
downloads.append(url)
self.assertGreater(timeout, 0)
return SimpleNamespace(content=payload, raise_for_status=lambda: None)
parsed = ast.parse(code)
# Substitute only the HTTP boundary; execute the documented worker body unchanged.
parsed.body = [node for node in parsed.body if not isinstance(node, (ast.Import, ast.ImportFrom))]
namespace = {"Path": Path, "tempfile": tempfile, "requests": SimpleNamespace(get=get, post=post)}
exec(compile(parsed, name, "exec"), namespace)
args = ("https://storage.example.invalid/approved.wav",) if name == "transcribe_from_url" else ("audio.wav", payload)
self.assertEqual(namespace[name](*args), {"text": "hello", "segments": []})
self.assertEqual(len(calls), 1)
self.assertEqual(downloads, list(args) if name == "transcribe_from_url" else [])
self.assertTrue(all(handle.closed for handle in handles))
if __name__ == "__main__":
unittest.main()