diff --git a/examples/openai_api/WORKFLOWS.md b/examples/openai_api/WORKFLOWS.md index 3cdab6ba6..4c5f0d5d1 100644 --- a/examples/openai_api/WORKFLOWS.md +++ b/examples/openai_api/WORKFLOWS.md @@ -2,65 +2,89 @@ [中文](WORKFLOWS_zh.md) -Use this guide when you want Dify, n8n, webhook workers, or another workflow engine to call a private FunASR speech API. Start with the local smoke test in this directory, then replace `localhost` with the reachable service name inside your network. +Use this guide when you want Dify, n8n, webhook workers, or another workflow engine to call a private FunASR speech API. These are multipart HTTP recipes, not a compatibility guarantee for every workflow product or version. + +The example server has **no built-in authentication or upload limit**. A client URL containing `localhost` does not restrict the server's listening address. Keep local checks on loopback; before sharing, configure TLS, gateway authentication, upload/time/rate limits, private health/model/schema access, and audio/transcript retention using the [security and gateway guide](SECURITY.md). A placeholder API key does not authenticate the server. ## Server preflight +First prepare the checkout and environment in the [example README](README.md#quick-start). Its fixed source revision is not a dependency/model lock or proof of a clean installation. The commands below start in that checkout's root, with its `.venv` already prepared. If the server is already running, skip startup and proceed to the checks; do not start a second process on the same port. + ```bash cd examples/openai_api -python server.py --model sensevoice --device cuda --port 8000 +source ../../.venv/bin/activate +python server.py --host 127.0.0.1 --model sensevoice --device cpu --port 8000 ``` -From the workflow host or container: +After preparing CUDA dependencies, replace the CPU command with `python server.py --host 127.0.0.1 --model sensevoice --device cuda --port 8000`. For the distinct packaged `funasr-server` route, use [Agent integration](../../docs/agent_integration.md); startup defaults, aliases and response fields differ from this example. + +In a second terminal on the same host, enter the same checkout's `examples/openai_api` directory and activate the same environment. Wait for model loading, then check the local service: ```bash -export FUNASR_BASE_URL=http://:8000 +source ../../.venv/bin/activate +export FUNASR_BASE_URL="http://127.0.0.1:8000" curl -fsS "$FUNASR_BASE_URL/health" curl -fsS "$FUNASR_BASE_URL/v1/models" +curl -fsS "$FUNASR_BASE_URL/openapi.json" ``` -If the workflow engine runs in Docker, `localhost` usually means the workflow container itself. Use a Docker Compose service name, Kubernetes service name, or LAN host name instead. +Health and schema checks do not establish acoustic correctness. For transcription, use an existing local audio file as `meeting.wav` below; the README's public Chinese smoke sample is not a multilingual accuracy benchmark. + +If the workflow engine runs in Docker, `localhost` usually means the workflow container itself. A host-loopback server is not automatically reachable from that container. Deliberately configure a private gateway/container network, then replace `FUNASR_BASE_URL` and the worker's `FUNASR_URL` with the address reachable from the workflow runtime. Use real gateway credentials when required; the local curl/Python examples below do not add authentication headers. Do not solve connectivity by exposing an unauthenticated port publicly. ## Postman smoke test Before configuring a low-code tool, you can import the [Postman collection](POSTMAN.md) and run health, model-list, and transcription requests from a GUI. For schema-driven imports, use the [OpenAPI spec](OPENAPI.md). Set `FUNASR_BASE_URL`, choose a local audio file for the multipart `file` field, and keep `MODEL_ALIAS=sensevoice` for the first test. -For offline multi-speaker meetings, set `MODEL_ALIAS=moss-transcribe-diarize` -and keep `response_format=verbose_json` so downstream nodes receive native -anonymous speaker segments. Use the [MOSS deployment guide](../../docs/moss_transcribe_diarize.md) -for the isolated GPU service and file-duration limits. +For offline multi-speaker meetings, first prepare the isolated third-party MOSS service in the [MOSS deployment guide](../../docs/moss_transcribe_diarize.md), including its GPU and file-duration limits. Then set `MODEL_ALIAS=moss-transcribe-diarize` and keep `response_format=verbose_json` to preserve available native anonymous speaker segments. Do not add external VAD or `spk=true`; recording-local labels are not verified identities or stable IDs across recordings. Changing a client alias alone does not prepare that service. ## Multipart HTTP request Every workflow engine eventually needs to send this request shape: -| Field | Value | -|---|---| -| Method | `POST` | -| URL | `http://:8000/v1/audio/transcriptions` | -| Body type | `multipart/form-data` | -| File field | `file` | -| Text field | `model=sensevoice` | -| Text field | `response_format=verbose_json` | -| Timeout | Set according to maximum audio duration, for example 300 seconds for long files. | +- **Method:** `POST` +- **URL:** `http://:8000/v1/audio/transcriptions` +- **Body type:** `multipart/form-data` +- **File field:** `file` +- **Text field:** `model=sensevoice` +- **Text field:** `response_format=verbose_json` +- **Timeout:** Set according to maximum audio duration, for example 300 seconds for long files. Equivalent curl command: ```bash -curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \ +curl -fsS "$FUNASR_BASE_URL/v1/audio/transcriptions" \ -F file=@meeting.wav \ -F model=sensevoice \ -F response_format=verbose_json ``` -Typical JSON fields to map downstream: +Map `text` to the transcript. `response_format=verbose_json` selects a format: it does not enable diarization or force timestamps. Check the [client response contract](CLIENTS.md#response-formats) before consuming the other fields: -| Path | Use | -|---|---| -| `text` | Plain transcript for a chatbot, ticket, or knowledge-base step. | -| `segments` | Timestamps and speaker labels when `verbose_json` is requested. | -| `duration` | Audio processing time reported by the API, useful for logs. | -| `model` | Model alias used for the request. | +- **Example `server.py`:** `segments` comes only from model-provided `sentence_info`, otherwise `segments=[]`. `duration` is elapsed seconds around `generate()`, excluding initial model loading, not recording length. `model` is the resolved request alias; `language` echoes the submitted hint or `auto`, not detected language. +- **Packaged `funasr-server`:** `duration` is audio length in seconds, with 0 possible if fallback audio metadata cannot be read. It can return coarse text-based segments, not forced alignment. Its verbose response includes `task` and segment `id`/`words`, but no top-level `model`; language can use backend detection. For non-native diarization models, `spk=true` opts into a separate speaker pipeline, subject to its model/dependencies. The example has no `spk` form field. + +Segment `start`/`end` use seconds, not the SDK's millisecond coordinates. A `speaker` field can be absent, null, numeric or a string; labels do not identify a person. Neither nonempty segments nor `verbose_json` guarantees accurate subtitle alignment. HTTP display text strips SenseVoice rich tags; it is not the SDK raw-tag result or a dedicated emotion/event response. SDK fields/options such as `timestamp`, `timestamps`, `ctc_timestamps`, `use_itn`, hotwords and raw arrays are not additional form fields; raw SDK timestamps are not automatically converted into HTTP segments. + +Illustrative example-server response with no `sentence_info`; `0.42` is processing time, not audio duration: + +```json +{"text": "recognized speech", "segments": [], "language": "auto", "duration": 0.42, "model": "sensevoice"} +``` + +Illustrative packaged-server response for a 3.2-second file and a coarse fallback segment, without speaker opt-in. These examples describe schemas, not new model measurements: + +```json +{ + "task": "transcribe", + "language": "en", + "duration": 3.2, + "text": "recognized speech", + "segments": [ + {"id": 0, "start": 0.0, "end": 3.2, "text": "recognized speech", "words": []} + ] +} +``` ## Dify custom tool or HTTP node @@ -75,21 +99,33 @@ Configure an HTTP request node or custom tool with: - Body: `multipart/form-data` - File part: `file`, bound to the uploaded audio variable - Text parts: `model=sensevoice`, `response_format=verbose_json` -- Output variable: map `text` as the transcript, and keep `segments` when timestamps or speaker labels matter +- Output variable: map `text` as the transcript; inspect `segments` availability and provenance before using timestamps or speaker labels ### Audio URL path -Some workflow tools pass a file URL rather than raw multipart bytes. In that case, add a small internal worker: +Some workflow tools pass a file URL rather than raw multipart bytes. A URL string in the multipart `file` field is not an audio upload. Prefer direct binary uploads or controlled storage object IDs resolved by a reviewed storage client. -1. Dify sends the audio URL and metadata to the worker. +The following sketch is only for an operator-approved URL in trusted storage. Destination allowlists, private-network access policy, redirect validation, download byte limits and authentication are **not implemented**. `requests.get` follows redirects and buffers the entire response; its timeout is not a byte cap or a complete end-to-end deadline. Do not pass user-supplied URLs to this helper. Before accepting them, require a reviewed download boundary enforcing all those controls, including blocking unintended private/metadata destinations and checking each redirect. Being inside a trusted network is not an SSRF defense. + +For this trusted-input illustration: + +1. An operator supplies an approved audio URL and metadata to the worker. 2. The worker downloads the file from trusted storage. 3. The worker posts multipart data to FunASR. -4. The worker returns `text`, `segments`, and operational logs to Dify. +4. The worker returns the service JSON; downstream nodes check optional fields. Logs must not expose signed URLs, credentials or private transcripts. + +In the same activated client environment, install the separate HTTP dependency: + +```bash +python -m pip install requests +``` + +Set `FUNASR_URL` for the prepared service; this default is for a worker on the same host. The function definitions can be imported by your worker, but do not create an HTTP listener or implement inbound authentication/upload limits. ```python import requests -FUNASR_URL = "http://funasr-api:8000/v1/audio/transcriptions" +FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions" def transcribe_from_url(audio_url: str) -> dict: audio_response = requests.get(audio_url, timeout=120) @@ -101,7 +137,7 @@ def transcribe_from_url(audio_url: str) -> dict: return response.json() ``` -Keep this worker inside your trusted network and validate allowed URL domains before downloading user-provided links. +Keep this illustration restricted to approved inputs; hostname validation alone is not a complete safe-download policy. ## n8n HTTP Request node @@ -109,39 +145,31 @@ A common n8n flow is: trigger -> binary audio data -> HTTP Request -> transcript Recommended HTTP Request settings: -| n8n setting | Value | -|---|---| -| Method | `POST` | -| URL | `http://:8000/v1/audio/transcriptions` | -| Send Body | enabled | -| Body Content Type | `Form-Data` / multipart | -| Binary file field | `file` | -| Additional form fields | `model=sensevoice`, `response_format=verbose_json` | -| Response Format | JSON | -| Timeout | Increase for long recordings. | +- **Method:** `POST` +- **URL:** `http://:8000/v1/audio/transcriptions` +- **Send Body:** enabled +- **Body Content Type:** `Form-Data` / multipart +- **Binary file field:** `file` +- **Additional form fields:** `model=sensevoice`, `response_format=verbose_json` +- **Response Format:** JSON +- **Timeout:** Increase for long recordings. -After the request, use `{{$json.text}}` as the transcript. If `verbose_json` is enabled, route `{{$json.segments}}` to subtitle, speaker, or QA steps. +After the request, use `{{$json.text}}` as the transcript. Route `{{$json.segments}}` onward only after checking it exists and is useful for the task; empty or coarse segments must not be treated as verified subtitle timing or diarization. Node labels and binary-property configuration vary with the installed n8n version; `file` is the outgoing multipart field, not necessarily the name of the incoming binary property. ### n8n OpenAI Audio node -The built-in OpenAI node can also run its Audio > Transcribe operation against -FunASR. In OpenAI credentials, set Base URL to -`http://:8000/v1` and provide any non-empty API key. The node -always sends `model=whisper-1`; FunASR maps that compatibility alias to the -model selected when the server starts. This path is transcription-only. Use the -HTTP Request node above when you need `verbose_json`, segments, or speaker -labels. +For OpenAI Audio > Transcribe node versions that send `model=whisper-1`, FunASR maps that compatibility alias to the model selected at server startup; it does not select a Whisper checkpoint. Set Base URL to your reachable service URL with `/v1`. A non-empty placeholder key is only for an unprotected local endpoint; supply real credentials for an authenticated gateway. Verify the installed node version's request behavior rather than assuming all versions match. Use this recipe for plain transcription; use the HTTP Request node for explicit `response_format` and any supported speaker opt-in, still subject to the response limits above. ## Webhook worker pattern -Use this when the workflow engine cannot send multipart files reliably or when audio needs pre-processing. +Use this when the workflow engine cannot send multipart files reliably or when audio needs pre-processing. This POSIX temporary-file example uses the same `requests` dependency and closes the upload handle. It accepts bytes already held in memory: apply upload limits before buffering them. It is a function, not a protected webhook server. ```python from pathlib import Path import tempfile import requests -FUNASR_URL = "http://localhost:8000/v1/audio/transcriptions" +FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions" def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/wav") -> dict: with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix or ".wav") as tmp: @@ -158,7 +186,7 @@ def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/w return response.json() ``` -This worker is also the right place to add audio conversion, file-size checks, request IDs, authentication, and retries. +Audio conversion, file-size checks, request IDs, inbound/upstream authentication and retry policy are not implemented here. Enforce them at the appropriate boundary before shared use; retries can repeat expensive transcription work. ## Production guardrails @@ -171,11 +199,9 @@ This worker is also the right place to add audio conversion, file-size checks, r ## Troubleshooting -| Symptom | Fix | -|---|---| -| Workflow can call `/health` but transcription fails | Confirm the request is `multipart/form-data` and the binary field is named `file`. | -| `localhost` connection fails from Dify or n8n | Use the host, Compose service, or Kubernetes service reachable from the workflow runtime. | -| Response has no `segments` | Set `response_format=verbose_json`. | -| Requests time out | Increase HTTP timeout or split long recordings. | -| First request is slow | Preload the model with `--model sensevoice` and use `/health` as a readiness check. | -| Unknown model alias | Call `/v1/models` and use one of the returned aliases. | +- **Workflow can call `/health` but transcription fails:** Confirm the request is `multipart/form-data` and the binary field is named `file`. +- **`localhost` connection fails from Dify or n8n:** Use the host, Compose service, or Kubernetes service reachable from the workflow runtime. +- **Response has no usable `segments`:** Check the format and deployed schema, then the model's `sentence_info` and speaker configuration; `verbose_json` alone cannot create timestamps or labels. +- **Requests time out:** Increase HTTP timeout or split long recordings. +- **First request is slow:** Preload the model with `--model sensevoice` and use `/health` as a readiness check. +- **Unknown model alias:** Call `/v1/models` and use one of the returned aliases. diff --git a/examples/openai_api/WORKFLOWS_zh.md b/examples/openai_api/WORKFLOWS_zh.md index 7e05b7c07..23a9f71fe 100644 --- a/examples/openai_api/WORKFLOWS_zh.md +++ b/examples/openai_api/WORKFLOWS_zh.md @@ -2,64 +2,89 @@ [English](WORKFLOWS.md) -当你希望让 Dify、n8n、HTTP 节点、webhook worker 或其他低代码工作流引擎调用私有 FunASR 语音 API 时，可以从这份配方开始。建议先在本目录跑通本地 smoke test，再把 `localhost` 换成工作流运行环境能够访问到的服务名或内网地址。 +当你希望让 Dify、n8n、HTTP 节点、webhook worker 或其他低代码工作流引擎调用私有 FunASR 语音 API 时，可以从这份配方开始。这些是 multipart HTTP 接入示例，不是对所有工作流产品或版本的兼容保证。 + +示例服务**没有内置鉴权或上传大小限制**。客户端 URL 中的 `localhost` 不会限制服务端监听地址。本地检查只绑定 loopback；共享前按[安全与网关指南](SECURITY_zh.md)配置 TLS、网关鉴权、上传/时限/速率限制、health/model/schema 私有访问和音频/转写保留策略。占位 API key 不会让服务具备鉴权。 ## 服务预检 +先按[示例 README](README_zh.md#快速开始)准备 checkout 和环境。其中固定的源码 revision 不是依赖/模型锁定，也不代表已验证全新安装。以下命令从该 checkout 根目录开始，假定 `.venv` 已准备好。如果服务已经运行，跳过启动直接检查，不要在同一端口启动第二个进程。 + ```bash cd examples/openai_api -python server.py --model sensevoice --device cuda --port 8000 +source ../../.venv/bin/activate +python server.py --host 127.0.0.1 --model sensevoice --device cpu --port 8000 ``` -在工作流所在主机或容器中执行： +准备好 CUDA 依赖后，可将 CPU 命令替换为 `python server.py --host 127.0.0.1 --model sensevoice --device cuda --port 8000`。另一个实现即打包的 `funasr-server` 路线见 [Agent 集成指南](../../docs/agent_integration_zh.md)，其启动默认值、别名和响应字段与本示例不同。 + +在同一主机的第二个终端进入同一 checkout 的 `examples/openai_api` 目录，激活同一环境。等待模型加载后检查本地服务： ```bash -export FUNASR_BASE_URL=http://:8000 +source ../../.venv/bin/activate +export FUNASR_BASE_URL="http://127.0.0.1:8000" curl -fsS "$FUNASR_BASE_URL/health" curl -fsS "$FUNASR_BASE_URL/v1/models" +curl -fsS "$FUNASR_BASE_URL/openapi.json" ``` -如果工作流引擎运行在 Docker 中，`localhost` 通常指的是工作流容器自身。请改用 Docker Compose service name、Kubernetes service name 或内网主机名。 +健康与 schema 检查不验证声学正确性。下方转写命令的 `meeting.wav` 应替换为已有本地音频；README 的公开中文 smoke 样例不是多语言准确率基准。 + +如果工作流引擎运行在 Docker 中，`localhost` 通常指的是工作流容器自身；主机 loopback 服务不会自动对容器可达。请明确配置私有网关/容器网络，再将 `FUNASR_BASE_URL` 和 worker 的 `FUNASR_URL` 替换为工作流运行时能访问的地址。需要网关鉴权时使用真实凭据；下方本地 curl/Python 示例没有添加鉴权 header。不要为解决连通性而把未鉴权端口直接暴露到公网。 ## Postman smoke test 在配置低代码工具前，可以先导入 [Postman collection](POSTMAN_zh.md)，从图形界面跑通 health、模型列表和转写请求；需要按 schema 导入时可使用 [OpenAPI spec](OPENAPI_zh.md)。设置 `FUNASR_BASE_URL`，在 multipart `file` 字段选择本地音频文件，第一次测试建议保持 `MODEL_ALIAS=sensevoice`。 -处理离线多人会议时，将 `MODEL_ALIAS` 改为 `moss-transcribe-diarize`，并保留 -`response_format=verbose_json`，下游节点即可收到模型原生匿名说话人 segments。 -独立 GPU 服务与文件时长边界见 [MOSS 部署指南](../../docs/moss_transcribe_diarize_zh.md)。 +处理离线多人会议时，先按 [MOSS 部署指南](../../docs/moss_transcribe_diarize_zh.md)准备独立的第三方 MOSS 服务，包括 GPU 和文件时长边界。再将 `MODEL_ALIAS` 改为 `moss-transcribe-diarize`，保留 `response_format=verbose_json` 以保留可用的原生匿名说话人 segments。不要添加外部 VAD 或 `spk=true`；录音内标签不是已验证身份，也不是跨录音稳定编号。只修改客户端别名不会完成服务环境准备。 ## Multipart HTTP 请求 所有工作流引擎最终都需要发出下面这种请求： -| 字段 | 值 | -|---|---| -| Method | `POST` | -| URL | `http://:8000/v1/audio/transcriptions` | -| Body type | `multipart/form-data` | -| File field | `file` | -| Text field | `model=sensevoice` | -| Text field | `response_format=verbose_json` | -| Timeout | 根据最长音频时长设置，例如长录音可先设为 300 秒。 | +- **Method:** `POST` +- **URL:** `http://:8000/v1/audio/transcriptions` +- **Body type:** `multipart/form-data` +- **File field:** `file` +- **Text field:** `model=sensevoice` +- **Text field:** `response_format=verbose_json` +- **Timeout:** 根据最长音频时长设置，例如长录音可先设为 300 秒。 等价 curl 命令： ```bash -curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \ +curl -fsS "$FUNASR_BASE_URL/v1/audio/transcriptions" \ -F file=@meeting.wav \ -F model=sensevoice \ -F response_format=verbose_json ``` -常用响应字段映射： +将 `text` 映射为转写文本。`response_format=verbose_json` 只选择格式，不开启说话人分离，也不强制生成时间戳。消费其他字段前，先看[客户端响应契约](CLIENTS.md#response-formats)： -| 路径 | 用途 | -|---|---| -| `text` | 纯文本转写结果，可传给聊天机器人、工单、知识库或后续摘要节点。 | -| `segments` | 请求 `verbose_json` 时返回的时间戳、说话人等分段信息。 | -| `duration` | API 返回的音频处理时长，适合写入日志和监控。 | -| `model` | 本次请求使用的模型别名。 | +- **示例 `server.py`：** `segments` 只来自模型返回的 `sentence_info`，否则为 `segments=[]`。`duration` 是 `generate()` 调用周围的耗时秒数，不含初次模型加载，不是录音时长。`model` 是解析后的请求别名；`language` 回显提交的提示或 `auto`，不是检测出的语言。 +- **打包 `funasr-server`：** `duration` 是音频时长秒数，fallback 无法读取音频元数据时可能为 0。它可能返回按文本生成的粗粒度片段，不是强制对齐。verbose 响应有 `task` 和片段 `id`/`words`，但没有顶层 `model`；语言可使用后端检测结果。对非原生说话人模型，`spk=true` 请求独立说话人流程，仍依赖其模型和环境。示例服务没有 `spk` 表单字段。 + +片段 `start`/`end` 单位为秒，不是 SDK 的毫秒坐标。`speaker` 字段可能缺失、为 null、数字或字符串；标签不标识真实身份。非空片段或 `verbose_json` 都不保证准确字幕对齐。HTTP 展示文本会剥离 SenseVoice 富标签，不等于 SDK 原始标签结果，也不是独立的情绪/事件响应。SDK 字段/选项如 `timestamp`、`timestamps`、`ctc_timestamps`、`use_itn`、hotwords 和原始数组不是额外表单字段；原始 SDK 时间戳不会自动转成 HTTP segments。 + +以下是没有 `sentence_info` 的示例服务响应；`0.42` 是处理耗时，不是音频时长： + +```json +{"text": "recognized speech", "segments": [], "language": "auto", "duration": 0.42, "model": "sensevoice"} +``` + +以下是 3.2 秒文件对应的打包服务响应示意，含粗粒度 fallback 片段，未开启说话人选项。这些示例说明 schema，不是新模型测量： + +```json +{ + "task": "transcribe", + "language": "en", + "duration": 3.2, + "text": "recognized speech", + "segments": [ + {"id": 0, "start": 0.0, "end": 3.2, "text": "recognized speech", "words": []} + ] +} +``` ## Dify 自定义工具或 HTTP 节点 @@ -74,21 +99,33 @@ curl "$FUNASR_BASE_URL/v1/audio/transcriptions" \ - Body: `multipart/form-data` - File part: `file`，绑定到上传音频变量 - Text parts: `model=sensevoice`、`response_format=verbose_json` -- Output variable: 把 `text` 映射为转写文本；需要时间戳或说话人信息时保留 `segments` +- Output variable: 把 `text` 映射为转写文本；使用时间戳或说话人标签前检查 `segments` 是否可用及其来源 ### 音频 URL 转写 -有些工作流工具只能传文件 URL，而不能直接传 multipart 二进制。此时建议加一个内网 worker： +有些工作流工具只能传文件 URL，而不能直接传 multipart 二进制。multipart `file` 字段中的 URL 字符串不是音频上传。优先直接上传二进制，或传受控存储对象 ID 并由经过审查的存储客户端解析。 -1. Dify 将音频 URL 和元数据发送给 worker。 +下面的 sketch 仅用于运维人员批准的可信存储 URL。目标 allowlist、私网访问策略、重定向校验、下载字节上限与鉴权均**未实现**。`requests.get` 会跟随重定向并将整个响应缓存在内存中；其 timeout 不是字节限制，也不是完整的端到端截止时限。不要把用户提供的 URL 传给这个 helper。接受这些 URL 前，必须使用经过审查的下载边界执行全部控制，包括阻止非预期私网/元数据目标并检查每次重定向。位于可信内网并不能防止 SSRF。 + +对于这个可信输入示意： + +1. 运维人员向 worker 提供已批准的音频 URL 和元数据。 2. worker 从可信存储下载音频。 3. worker 使用 multipart 请求调用 FunASR。 -4. worker 将 `text`、`segments` 和运行日志返回给 Dify。 +4. worker 返回服务 JSON，由下游检查可选字段。日志不得暴露签名 URL、凭据或私有转写。 + +在同一已激活的客户端环境安装独立 HTTP 依赖： + +```bash +python -m pip install requests +``` + +按已准备的服务设置 `FUNASR_URL`；默认值用于同主机 worker。工作流可导入以下函数定义，但它们不创建 HTTP 监听器，也不实现入站鉴权/上传限制。 ```python import requests -FUNASR_URL = "http://funasr-api:8000/v1/audio/transcriptions" +FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions" def transcribe_from_url(audio_url: str) -> dict: audio_response = requests.get(audio_url, timeout=120) @@ -100,7 +137,7 @@ def transcribe_from_url(audio_url: str) -> dict: return response.json() ``` -请把这个 worker 放在可信内网中，并在下载用户提供的链接前校验允许访问的域名或存储桶。 +请把此示意限制在已批准输入内；仅校验主机名不是完整的安全下载策略。 ## n8n HTTP Request 节点 @@ -108,37 +145,31 @@ def transcribe_from_url(audio_url: str) -> dict: 推荐 HTTP Request 配置： -| n8n 设置 | 值 | -|---|---| -| Method | `POST` | -| URL | `http://:8000/v1/audio/transcriptions` | -| Send Body | enabled | -| Body Content Type | `Form-Data` / multipart | -| Binary file field | `file` | -| Additional form fields | `model=sensevoice`、`response_format=verbose_json` | -| Response Format | JSON | -| Timeout | 长录音场景需要调大。 | +- **Method:** `POST` +- **URL:** `http://:8000/v1/audio/transcriptions` +- **Send Body:** enabled +- **Body Content Type:** `Form-Data` / multipart +- **Binary file field:** `file` +- **Additional form fields:** `model=sensevoice`、`response_format=verbose_json` +- **Response Format:** JSON +- **Timeout:** 长录音场景需要调大。 -请求之后，使用 `{{$json.text}}` 作为转写文本。如果启用了 `verbose_json`，可以把 `{{$json.segments}}` 传给字幕、说话人分析或质检节点。 +请求之后，使用 `{{$json.text}}` 作为转写文本。先检查 `{{$json.segments}}` 存在且适合任务，再传给后续节点；空片段或粗粒度片段不能作为已验证字幕时刻或说话人分离结果。节点标签和二进制属性配置随所安装的 n8n 版本变化；`file` 是发出的 multipart 字段名，不一定是传入二进制属性的名字。 ### n8n OpenAI Audio 节点 -也可以让内置 OpenAI 节点的 Audio > Transcribe 操作调用 FunASR。在 OpenAI -凭据中将 Base URL 设为 `http://:8000/v1`，并填写任意非空 API -key。该节点固定发送 `model=whisper-1`，FunASR 会将这个兼容别名映射到服务 -启动时选择的模型。此路径只返回转写文本；需要 `verbose_json`、分段或说话人 -标签时，仍使用上面的 HTTP Request 节点。 +对于发送 `model=whisper-1` 的 OpenAI Audio > Transcribe 节点版本，FunASR 会把该兼容别名映射到服务启动时选择的模型，而不是选择 Whisper checkpoint。Base URL 使用可达服务地址并带 `/v1`。非空占位 key 仅适用于未保护的本地端点，访问鉴权网关时应提供真实凭据。请核验所安装节点版本的请求行为，不要假定所有版本相同。此配方用于纯文本转写；需要显式 `response_format` 或服务支持的说话人选项时使用 HTTP Request 节点，仍受上述响应边界限制。 ## Webhook worker 模式 -当工作流引擎不能稳定发送 multipart 文件，或音频需要预处理时，可以把转写封装成一个内部 webhook worker。 +当工作流引擎不能稳定发送 multipart 文件，或音频需要预处理时，可以使用这个函数。此 POSIX 临时文件示例使用同一个 `requests` 依赖，并关闭上传句柄。它接受已在内存中的 bytes，应在缓冲之前限制上传大小；它是函数，不是受保护的 webhook 服务。 ```python from pathlib import Path import tempfile import requests -FUNASR_URL = "http://localhost:8000/v1/audio/transcriptions" +FUNASR_URL = "http://127.0.0.1:8000/v1/audio/transcriptions" def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/wav") -> dict: with tempfile.NamedTemporaryFile(suffix=Path(filename).suffix or ".wav") as tmp: @@ -155,7 +186,7 @@ def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/w return response.json() ``` -这个 worker 也适合集中做音频转码、文件大小限制、请求 ID、鉴权、重试和审计日志。 +这里未实现音频转码、文件大小检查、请求 ID、入站/上游鉴权和重试策略。共享前在相应边界执行这些控制；重试可能重复执行昂贵的转写工作。 ## 生产环境护栏 @@ -168,11 +199,9 @@ def transcribe_bytes(filename: str, payload: bytes, content_type: str = "audio/w ## 故障排查 -| 现象 | 处理方式 | -|---|---| -| 工作流能访问 `/health`，但转写失败 | 确认请求是 `multipart/form-data`，且二进制字段名是 `file`。 | -| Dify 或 n8n 访问 `localhost` 失败 | 换成工作流运行时可访问的主机名、Compose service name 或 Kubernetes service name。 | -| 响应中没有 `segments` | 设置 `response_format=verbose_json`。 | -| 请求超时 | 调大 HTTP timeout，或先切分长录音。 | -| 第一次请求很慢 | 使用 `--model sensevoice` 预加载模型，并用 `/health` 做 readiness check。 | -| 模型别名未知 | 调用 `/v1/models`，使用返回列表中的别名。 | +- **工作流能访问 `/health`，但转写失败:** 确认请求是 `multipart/form-data`，且二进制字段名是 `file`。 +- **Dify 或 n8n 访问 `localhost` 失败:** 换成工作流运行时可访问的主机名、Compose service name 或 Kubernetes service name。 +- **响应中没有可用 `segments`:** 检查格式和已部署 schema，再检查模型的 `sentence_info` 与说话人配置；仅设置 `verbose_json` 不能创建时间戳或标签。 +- **请求超时:** 调大 HTTP timeout，或先切分长录音。 +- **第一次请求很慢:** 使用 `--model sensevoice` 预加载模型，并用 `/health` 做 readiness check。 +- **模型别名未知:** 调用 `/v1/models`，使用返回列表中的别名。 diff --git a/tests/test_service_docs_contract.py b/tests/test_service_docs_contract.py index d85f6d9c1..6f0dc5bcb 100644 --- a/tests/test_service_docs_contract.py +++ b/tests/test_service_docs_contract.py @@ -6,6 +6,8 @@ from pathlib import Path import re import shlex import subprocess +import tempfile +from types import SimpleNamespace import unittest from urllib.parse import unquote, urlsplit @@ -15,6 +17,13 @@ EXAMPLE = ROOT / "examples/openai_api" READMES = [EXAMPLE / name for name in ("README.md", "README_zh.md", "README_ja.md", "README_ko.md")] DOCS = [EXAMPLE / "CLIENTS.md", *READMES] PACKAGED = ROOT / "funasr/bin/_server_app.py" +WORKFLOW_DOCS = [EXAMPLE / name for name in ("WORKFLOWS.md", "WORKFLOWS_zh.md")] + +# Frozen from the two pre-edit workflow guides, not translated from one language. +WORKFLOW_HEADINGS = { + "WORKFLOWS.md": "Low-code Workflow Recipes for the FunASR OpenAI-Compatible API|Server preflight|Postman smoke test|Multipart HTTP request|Dify custom tool or HTTP node|Direct file upload path|Audio URL path|n8n HTTP Request node|n8n OpenAI Audio node|Webhook worker pattern|Production guardrails|Troubleshooting", + "WORKFLOWS_zh.md": "FunASR OpenAI 兼容 API 低代码工作流配方|服务预检|Postman smoke test|Multipart HTTP 请求|Dify 自定义工具或 HTTP 节点|直接上传文件|音频 URL 转写|n8n HTTP Request 节点|n8n OpenAI Audio 节点|Webhook worker 模式|生产环境护栏|故障排查", +} # Frozen from the pre-edit README inventory, excluding fenced code comments. LEGACY_HEADINGS = { @@ -317,5 +326,197 @@ class ServiceDocsContract(unittest.TestCase): self.assertIn("does not enable diarization", clients) +class WorkflowDocsContract(unittest.TestCase): + def test_workflow_explanatory_sections_use_complete_readable_lists(self): + sections = [ + (("Multipart HTTP request", "Multipart HTTP 请求"), + ["Method", "URL", "Body type", "File field", "Text field", "Text field", "Timeout"], + [("POST",), ("/v1/audio/transcriptions",), ("multipart/form-data",), ("`file`",), + ("model=sensevoice",), ("response_format=verbose_json",), ("300",)]), + (("n8n HTTP Request node", "n8n HTTP Request 节点"), + ["Method", "URL", "Send Body", "Body Content Type", "Binary file field", "Additional form fields", "Response Format", "Timeout"], + [("POST",), ("/v1/audio/transcriptions",), ("enabled",), ("Form-Data", "multipart"), + ("`file`",), ("model=sensevoice", "response_format=verbose_json"), ("JSON",), ()]), + ] + symptoms = [ + ["Workflow can call `/health` but transcription fails", "`localhost` connection fails from Dify or n8n", + "Response has no usable `segments`", "Requests time out", "First request is slow", "Unknown model alias"], + ["工作流能访问 `/health`，但转写失败", "Dify 或 n8n 访问 `localhost` 失败", "响应中没有可用 `segments`", + "请求超时", "第一次请求很慢", "模型别名未知"], + ] + fixes = [("multipart/form-data", "`file`"), ("Compose", "Kubernetes"), + ("sentence_info", "verbose_json"), ("HTTP timeout",), ("--model sensevoice", "/health"), ("/v1/models",)] + for language, path in enumerate(WORKFLOW_DOCS): + text = path.read_text(encoding="utf-8") + cases = [*sections, (("Troubleshooting", "故障排查"), symptoms[language], fixes)] + for titles, labels, tokens in cases: + with self.subTest(path=path.name, section=titles[language]): + section = text.split("## " + titles[language] + "\n", 1)[1].split("\n## ", 1)[0] + self.assertNotRegex(section, r"(?m)^\s*\|") + self.assertNotRegex(section, r"(?i)]+>") + result = subprocess.run(["bash", "-n"], input=code, text=True, capture_output=True) + self.assertEqual(result.returncode, 0, result.stderr) + for line in code.replace("\\\n", " ").splitlines(): + args = shlex.split(line, comments=True) + if args[:1] == ["export"]: + assignments.extend(arg for arg in args[1:] if arg.startswith("FUNASR_BASE_URL=")) + self.assertEqual(assignments, ["FUNASR_BASE_URL=http://127.0.0.1:8000"]) + for endpoint in ("health", "v1/models", "openapi.json"): + self.assertIn(f'"$FUNASR_BASE_URL/{endpoint}"', "\n".join(codes)) + + def test_workflow_json_examples_match_actual_service_response_builders(self): + handler = function(EXAMPLE / "server.py", "transcribe") + response = next(node for node in ast.walk(handler) if isinstance(node, ast.Dict) + and any(isinstance(key, ast.Constant) and key.value == "duration" for key in node.keys)) + example = eval(compile(ast.Expression(response), "example-response", "eval"), + {"text": "recognized speech", "segments": [], "language": None, + "elapsed": 0.42, "model": "sensevoice"}) + namespace = {} + functions = [function(PACKAGED, name) for name in + ("_split_text_for_openai_segments", "build_openai_fallback_segments", + "resolve_transcription_language", "build_openai_verbose_json")] + exec(compile(ast.Module(body=functions, type_ignores=[]), str(PACKAGED), "exec"), namespace) + result = {"text": "recognized speech", "duration": 3.2, "language": "en", + "segments": namespace["build_openai_fallback_segments"]("recognized speech", 3.2)} + packaged = namespace["build_openai_verbose_json"](result) + for path in WORKFLOW_DOCS: + text = path.read_text(encoding="utf-8") + samples = [json.loads(code) for code in blocks(text, "json")] + self.assertEqual(samples, [example, packaged], path.name) + self.assertIn("](CLIENTS.md#response-formats)", text) + for marker in ("sentence_info", "segments=[]", "spk=true", "ctc_timestamps", "use_itn", "hotwords"): + self.assertIn(marker, text) + for speaker in (None, 0, "SPK0", "S01"): + result["segments"][0]["speaker"] = speaker + segment = namespace["build_openai_verbose_json"](result)["segments"][0] + if speaker is None: + self.assertNotIn("speaker", segment) + else: + self.assertEqual(segment["speaker"], speaker) + + def test_workflow_url_worker_warns_before_code_and_declares_dependency(self): + for path in WORKFLOW_DOCS: + text = path.read_text(encoding="utf-8") + section = text.split("### Audio URL path" if path.name == "WORKFLOWS.md" else "### 音频 URL 转写", 1)[1] + warning = section.split("```python", 1)[0] + markers = ("not implemented", "redirect", "private-network", "byte", "trusted") if path.name == "WORKFLOWS.md" else ( + "未实现", "重定向", "私网", "字节", "可信") + for marker in markers: + self.assertIn(marker, warning) + self.assertIn("python -m pip install requests", text[:text.index("```python")]) + + def test_workflow_python_examples_are_equivalent_and_send_real_multipart_fields(self): + snippets = [blocks(path.read_text(encoding="utf-8"), "python") for path in WORKFLOW_DOCS] + self.assertEqual([len(items) for items in snippets], [2, 2]) + self.assertEqual([ast.dump(ast.parse(code)) for code in snippets[0]], + [ast.dump(ast.parse(code)) for code in snippets[1]]) + allowed = set(form_defaults(function(EXAMPLE / "server.py", "transcribe"))) | {"file"} + for code, name in zip(snippets[0], ("transcribe_from_url", "transcribe_bytes")): + calls = [] + handles = [] + payload = b"test audio bytes, no acoustic inference" + + def post(url, *, files, data, timeout): + self.assertEqual(urlsplit(url).path, "/v1/audio/transcriptions") + self.assertEqual(set(files), {"file"}) + self.assertLessEqual(set(data) | set(files), allowed) + self.assertEqual(data, {"model": "sensevoice", "response_format": "verbose_json"}) + self.assertGreater(timeout, 0) + body = files["file"][1] + if hasattr(body, "read"): + handles.append(body) + body = body.read() + self.assertEqual(body, payload) + calls.append(url) + return SimpleNamespace(raise_for_status=lambda: None, json=lambda: {"text": "hello", "segments": []}) + + downloads = [] + + def get(url, *, timeout): + downloads.append(url) + self.assertGreater(timeout, 0) + return SimpleNamespace(content=payload, raise_for_status=lambda: None) + + parsed = ast.parse(code) + # Substitute only the HTTP boundary; execute the documented worker body unchanged. + parsed.body = [node for node in parsed.body if not isinstance(node, (ast.Import, ast.ImportFrom))] + namespace = {"Path": Path, "tempfile": tempfile, "requests": SimpleNamespace(get=get, post=post)} + exec(compile(parsed, name, "exec"), namespace) + args = ("https://storage.example.invalid/approved.wav",) if name == "transcribe_from_url" else ("audio.wav", payload) + self.assertEqual(namespace[name](*args), {"text": "hello", "segments": []}) + self.assertEqual(len(calls), 1) + self.assertEqual(downloads, list(args) if name == "transcribe_from_url" else []) + self.assertTrue(all(handle.closed for handle in handles)) + + if __name__ == "__main__": unittest.main()