feat(moss): support vLLM diarized responses

Signed-off-by: LauraGPT <[email protected]>

docs/moss_transcribe_diarize.md

@@ -21,7 +21,9 @@ This guide was verified against:
 - source: `OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`
 - model: `OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`
 - license: Apache-2.0 in the pinned upstream source and model metadata
-- vLLM nightly index: `68b4a1d582818e67adc903bf1b8fc5a5447da2fa`
+- vLLM release: `v0.27.1@6e448d0ea9bf3d88d898b65449ca6dc2aec170ac`
+- vLLM CUDA 12.9 x86_64 wheel SHA-256:
+  `bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b`
 
 Pin all three revisions. The model uses `trust_remote_code`; do not execute a
 floating model revision in a production service.
@@ -76,36 +78,48 @@ model = AutoModel(
     backend="vllm",
     vllm_base_url="http://127.0.0.1:8898/v1",
     vllm_model="moss-transcribe-diarize",
+    vllm_response_format="diarized_json",
     disable_update=True,
 )
 result = model.generate("audio.wav")[0]
 ```
 
 The vLLM adapter sends the documented OpenAI-compatible multipart request and
-normalizes the returned raw text. Authentication can be supplied with
-`vllm_api_key`; keep it in a secret store rather than source code.
+normalizes the official speaker-attributed `segments` directly into
+`sentence_info`. Authentication can be supplied with `vllm_api_key`; keep it
+in a secret store rather than source code.
+
+`vllm_response_format="diarized_json"` is the structured production path on
+the pinned vLLM revision. The compatibility default remains `json` so existing
+clients keep the exact tagged generation in `raw_text`; in structured mode,
+`raw_text` is the cleaned `text` returned by vLLM and the authoritative speaker
+metadata is in `sentence_info`.
 
 ## Choose a serving path
 
 | Path | Environment | Response contract | Use when |
 |---|---|---|---|
-| vLLM | CUDA 12 (`cu129`) or CUDA 13 (`cu130`) | `response_format=json` returns raw `[start][Sxx]text[end]` text | You already operate vLLM or need its scheduler |
+| vLLM | CUDA 12 (`cu129`) or CUDA 13 (`cu130`) | `response_format=diarized_json` returns OpenAI-compatible speaker segments; `response_format=json` keeps raw `[start][Sxx]text[end]` text | You already operate vLLM or need its scheduler |
 | SGLang Omni | CUDA 13 in the current upstream guide | `response_format=verbose_json` returns parsed segments | You need structured speaker segments directly from the API |
 | Transformers | PyTorch process | Python objects and raw tagged text | Evaluation, debugging, or custom preprocessing |
 
 The two HTTP backends share `/v1/audio/transcriptions`, but their documented
-response formats are not interchangeable. Do not promise vLLM `segments`
-without testing the exact vLLM revision.
+response formats are not interchangeable. The pinned vLLM revision supports
+`diarized_json`; test the exact server revision before promising `segments`.
 
 ## vLLM
 
-Create an isolated environment and install the upstream-pinned nightly build:
+Create an isolated environment and install the verified vLLM release:
 
 ```bash
 uv venv --python 3.12 .venv-moss
-uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \
-  --torch-backend=auto \
-  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
+curl -fL \
+  https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl \
+  -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl
+echo "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b  vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl" \
+  | sha256sum -c -
+uv pip install --python .venv-moss/bin/python --torch-backend=auto \
+  "vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl"
 ```
 
 The `audio` extra is required. A plain `vllm` install can start the server but
@@ -124,13 +138,13 @@ CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \
   --port 8898
 ```
 
-Submit audio and validate the speaker-tagged text:
+Submit audio and validate the official speaker-attributed response:
 
 ```bash
 curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
   -F file=@audio.wav \
   -F model=moss-transcribe-diarize \
-  -F response_format=json \
+  -F response_format=diarized_json \
   -F temperature=0 \
   | tee moss-transcription.json
 
@@ -140,19 +154,36 @@ import json
 with open("moss-transcription.json", encoding="utf-8") as stream:
     payload = json.load(stream)
 text = payload.get("text", "")
+segments = payload.get("segments", [])
 assert text.strip(), payload
-assert "[S01]" in text, text
-print(text)
+assert segments, payload
+assert all(
+    isinstance(item.get("speaker"), str)
+    and item.get("text")
+    and item.get("start") <= item.get("end")
+    for item in segments
+), payload
+print(text, sorted({item["speaker"] for item in segments}))
 PY
 ```
 
+For an audit of the model's compact tagged generation, repeat the request with
+`response_format=json` and verify `[S01]` appears in the returned `text`.
+
 For long recordings, test `max_completion_tokens` against the longest expected
 meeting. A larger limit can increase memory use and tail latency.
 
 ### Reproduced vLLM contract
 
-The FunASR ecosystem validation used vLLM
-`0.23.1rc1.dev949+g68b4a1d58`, Torch `2.11.0+cu129`, and one H100 80GB:
+The structured-response validation used vLLM 0.27.1, Torch
+`2.13.0+cu129`, and one H100 80GB. The 15.1685-second two-speaker probe
+(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)
+returned two `diarized_json` segments labelled `S01` and `S02`; the same
+request through the FunASR adapter returned two monotonic `sentence_info`
+entries with the same speakers.
+
+Earlier raw-response validation on vLLM
+`0.23.1rc1.dev949+g68b4a1d58`, Torch `2.11.0+cu129`, and one H100 80GB found:
 
 - the bundled 6.000-second sample
   (`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`)
@@ -165,6 +196,12 @@ This proves the pinned API and speaker-return contract on those inputs. It is
 not a diarization accuracy result, overlap test, throughput benchmark, or
 production capacity claim.
 
+The older nightly commit
+`68b4a1d582818e67adc903bf1b8fc5a5447da2fa` predates vLLM
+[`#48543`](https://github.com/vllm-project/vllm/pull/48543). It accepts
+`response_format=json` but returns HTTP 400 for `diarized_json`; upgrade to the
+verified release instead of silently retrying an expensive transcription.
+
 The FunASR adapter was also tested on both `backend="hf"` and
 `backend="vllm"` with model revision
 `e8681d68e7042738ffca8ac8212bc8fcb1131ab8`. The 15.1685-second two-speaker

docs/moss_transcribe_diarize_zh.md

@@ -19,7 +19,9 @@ MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话
 - 源码：`OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`
 - 模型：`OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`
 - 许可证：固定上游源码和模型元数据中的 Apache-2.0
-- vLLM nightly 索引：`68b4a1d582818e67adc903bf1b8fc5a5447da2fa`
+- vLLM 发布版：`v0.27.1@6e448d0ea9bf3d88d898b65449ca6dc2aec170ac`
+- vLLM CUDA 12.9 x86_64 wheel SHA-256：
+  `bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b`
 
 三者都要固定。模型依赖 `trust_remote_code`，生产服务不要执行浮动的模型 revision。
 
@@ -68,34 +70,46 @@ model = AutoModel(
     backend="vllm",
     vllm_base_url="http://127.0.0.1:8898/v1",
     vllm_model="moss-transcribe-diarize",
+    vllm_response_format="diarized_json",
     disable_update=True,
 )
 result = model.generate("audio.wav")[0]
 ```
 
-vLLM 适配器发送官方文档中的 OpenAI-compatible multipart 请求，并统一解析返回文本。
-认证可通过 `vllm_api_key` 提供；请放在 secret store，不要写入源码。
+vLLM 适配器发送官方文档中的 OpenAI-compatible multipart 请求，并把官方返回的
+speaker `segments` 直接映射到 `sentence_info`。认证可通过 `vllm_api_key` 提供；
+请放在 secret store，不要写入源码。
+
+固定 vLLM revision 的生产结构化路径是
+`vllm_response_format="diarized_json"`。为保持已有客户端兼容，默认值仍是 `json`，
+此时 `raw_text` 保留原始标签生成；结构化模式下，`raw_text` 是 vLLM 返回的清理后
+`text`，权威说话人信息位于 `sentence_info`。
 
 ## 选择服务后端
 
 | 路径 | 环境 | 响应契约 | 适用场景 |
 |---|---|---|---|
-| vLLM | CUDA 12（`cu129`）或 CUDA 13（`cu130`） | `response_format=json` 返回原始 `[start][Sxx]text[end]` 文本 | 已运行 vLLM，或需要其调度能力 |
+| vLLM | CUDA 12（`cu129`）或 CUDA 13（`cu130`） | `response_format=diarized_json` 返回 OpenAI-compatible 说话人 segments；`response_format=json` 保留原始 `[start][Sxx]text[end]` 文本 | 已运行 vLLM，或需要其调度能力 |
 | SGLang Omni | 当前上游指南为 CUDA 13 | `response_format=verbose_json` 返回解析后的 segments | API 需要直接返回结构化说话人分段 |
 | Transformers | PyTorch 进程 | Python 对象和原始标签文本 | 评测、排障或自定义预处理 |
 
 两个 HTTP 后端都使用 `/v1/audio/transcriptions`，但官方文档中的响应格式不能混为
-一谈。没有对 exact vLLM revision 做验证时，不要承诺 vLLM 返回 `segments`。
+一谈。固定的 vLLM revision 支持 `diarized_json`；承诺 `segments` 前仍要验证实际
+部署的 exact server revision。
 
 ## vLLM
 
-创建隔离环境并安装上游固定的 nightly build：
+创建隔离环境并安装已验证的 vLLM 发布版：
 
 ```bash
 uv venv --python 3.12 .venv-moss
-uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \
-  --torch-backend=auto \
-  --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
+curl -fL \
+  https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl \
+  -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl
+echo "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b  vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl" \
+  | sha256sum -c -
+uv pip install --python .venv-moss/bin/python --torch-backend=auto \
+  "vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl"
 ```
 
 必须安装 `audio` extra。只安装 plain `vllm` 时服务可以启动，但因为没有音频解码器，
@@ -113,13 +127,13 @@ CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \
   --port 8898
 ```
 
-提交音频并验证说话人标签文本：
+提交音频并验证官方结构化说话人响应：
 
 ```bash
 curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
   -F file=@audio.wav \
   -F model=moss-transcribe-diarize \
-  -F response_format=json \
+  -F response_format=diarized_json \
   -F temperature=0 \
   | tee moss-transcription.json
 
@@ -129,18 +143,34 @@ import json
 with open("moss-transcription.json", encoding="utf-8") as stream:
     payload = json.load(stream)
 text = payload.get("text", "")
+segments = payload.get("segments", [])
 assert text.strip(), payload
-assert "[S01]" in text, text
-print(text)
+assert segments, payload
+assert all(
+    isinstance(item.get("speaker"), str)
+    and item.get("text")
+    and item.get("start") <= item.get("end")
+    for item in segments
+), payload
+print(text, sorted({item["speaker"] for item in segments}))
 PY
 ```
 
+如需审计模型原始紧凑标签生成，请把请求改为 `response_format=json`，并验证返回
+`text` 中包含 `[S01]`。
+
 长音频需要按业务最长会议验证 `max_completion_tokens`。增大上限也会影响显存和尾延迟。
 
 ### 已复现的 vLLM 契约
 
-FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
-`2.11.0+cu129` 和一张 H100 80GB：
+结构化响应验证使用 vLLM 0.27.1、Torch `2.13.0+cu129` 和一张 H100 80GB。
+15.1685 秒双说话人样例
+（`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`）
+返回两个 `diarized_json` segments，标签为 `S01`、`S02`；同一请求通过 FunASR
+适配器后，返回两个时间单调且说话人一致的 `sentence_info`。
+
+较早的 raw-response 验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
+`2.11.0+cu129` 和一张 H100 80GB，结果如下：
 
 - 仓库内 6.000 秒样例
   （`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`）
@@ -152,6 +182,11 @@ FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
 这证明固定版本对这些输入满足 API 和说话人返回契约，不是 diarization 精度、重叠语音、
 吞吐或生产容量结论。
 
+旧 nightly commit `68b4a1d582818e67adc903bf1b8fc5a5447da2fa` 早于 vLLM
+[`#48543`](https://github.com/vllm-project/vllm/pull/48543)：它支持
+`response_format=json`，但 `diarized_json` 会返回 HTTP 400。应升级到已验证发布版，
+不要静默重试一次成本较高的转写请求。
+
 FunASR 适配器还分别使用 `backend="hf"` 和 `backend="vllm"`，按模型 revision
 `e8681d68e7042738ffca8ac8212bc8fcb1131ab8` 做了真实测试。15.1685 秒的双说话人
 样例（`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`）通过

funasr/models/moss_transcribe_diarize/model.py

@@ -7,6 +7,7 @@ https://github.com/OpenMOSS/MOSS-Transcribe-Diarize
 
 import copy
 import io
+import math
 import mimetypes
 import os
 import re
@@ -127,6 +128,66 @@ def _result_from_transcript(key, transcript):
     }
 
 
+def _result_from_diarized_response(key, response):
+    """Normalize vLLM's OpenAI-compatible diarized JSON response."""
+    text = response.get("text") if isinstance(response, dict) else None
+    segments = response.get("segments") if isinstance(response, dict) else None
+    if not isinstance(text, str) or not isinstance(segments, list):
+        raise RuntimeError(
+            "vLLM diarized_json response requires text and segments fields"
+        )
+
+    sentence_info = []
+    timestamps = []
+    for index, segment in enumerate(segments):
+        if not isinstance(segment, dict):
+            raise RuntimeError(
+                "vLLM diarized_json segment %d must be an object" % index
+            )
+        start = segment.get("start")
+        end = segment.get("end")
+        segment_text = segment.get("text")
+        speaker = segment.get("speaker")
+        numeric = (
+            isinstance(start, (int, float))
+            and not isinstance(start, bool)
+            and isinstance(end, (int, float))
+            and not isinstance(end, bool)
+        )
+        if (
+            not numeric
+            or not math.isfinite(start)
+            or not math.isfinite(end)
+            or start < 0
+            or end < start
+            or not isinstance(segment_text, str)
+            or not isinstance(speaker, str)
+            or not speaker
+        ):
+            raise RuntimeError(
+                "vLLM diarized_json segment %d has an invalid contract" % index
+            )
+        timestamp = [int(round(start * 1000)), int(round(end * 1000))]
+        timestamps.append(timestamp)
+        sentence_info.append(
+            {
+                "start": timestamp[0],
+                "end": timestamp[1],
+                "text": segment_text,
+                "sentence": segment_text,
+                "spk": speaker,
+                "timestamp": [timestamp],
+            }
+        )
+    return {
+        "key": key,
+        "text": text,
+        "raw_text": text,
+        "timestamp": timestamps,
+        "sentence_info": sentence_info,
+    }
+
+
 @tables.register("model_classes", "MOSS-Transcribe-Diarize")
 @tables.register("model_classes", DEFAULT_MODEL)
 class MossTranscribeDiarize(nn.Module):
@@ -157,6 +218,9 @@ class MossTranscribeDiarize(nn.Module):
         self.vllm_model = kwargs.get("vllm_model", DEFAULT_MODEL)
         self.vllm_api_key = kwargs.get("vllm_api_key", "EMPTY")
         self.vllm_timeout = float(kwargs.get("vllm_timeout", 600.0))
+        self.vllm_response_format = kwargs.get("vllm_response_format", "json").lower()
+        if self.vllm_response_format not in {"json", "diarized_json"}:
+            raise ValueError("vllm_response_format must be 'json' or 'diarized_json'")
         self.http_session = kwargs.get("http_session")
 
         self.hf_model = None
@@ -240,10 +304,18 @@ class MossTranscribeDiarize(nn.Module):
         results = []
         for index, audio in enumerate(inputs):
             if self.backend == "vllm":
-                transcript = self._transcribe_vllm(audio, **kwargs)
+                response = self._transcribe_vllm(audio, **kwargs)
+                if self.vllm_response_format == "diarized_json":
+                    results.append(
+                        _result_from_diarized_response(keys[index], response)
+                    )
+                else:
+                    results.append(
+                        _result_from_transcript(keys[index], response["text"])
+                    )
             else:
                 transcript = self._transcribe_hf(audio, **kwargs)
-            results.append(_result_from_transcript(keys[index], transcript))
+                results.append(_result_from_transcript(keys[index], transcript))
         return results, {
             "batch_data_time": max(audio_duration, 1e-9),
             "forward": time.perf_counter() - started,
@@ -324,7 +396,7 @@ class MossTranscribeDiarize(nn.Module):
         filename, payload, content_type = self._multipart_audio(audio)
         data = {
             "model": self.vllm_model,
-            "response_format": "json",
+            "response_format": self.vllm_response_format,
             "temperature": str(kwargs.get("temperature", 0)),
         }
         prompt = kwargs.get("prompt")
@@ -345,11 +417,15 @@ class MossTranscribeDiarize(nn.Module):
         )
         response.raise_for_status()
         result = response.json()
-        if not isinstance(result, dict) or not isinstance(result.get("text"), str):
+        if not isinstance(result, dict):
+            raise RuntimeError("vLLM transcription response must be a JSON object")
+        if self.vllm_response_format == "json" and not isinstance(
+            result.get("text"), str
+        ):
             raise RuntimeError(
                 "vLLM transcription response did not contain a text field"
             )
-        return result["text"]
+        return result
 
     @staticmethod
     def _multipart_audio(audio):

tests/test_moss_transcribe_diarize_docs.py

@@ -25,15 +25,19 @@ def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) ->
         "cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3",
         "OpenMOSS-Team/MOSS-Transcribe-Diarize",
         "e8681d68e7042738ffca8ac8212bc8fcb1131ab8",
-        "68b4a1d582818e67adc903bf1b8fc5a5447da2fa",
+        "6e448d0ea9bf3d88d898b65449ca6dc2aec170ac",
+        "bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b",
         "vllm[audio]",
-        "0.23.1rc1.dev949+g68b4a1d58",
+        "vLLM 0.27.1",
+        "68b4a1d582818e67adc903bf1b8fc5a5447da2fa",
         "dbb32bcfed2e8226bedf64248a9f4a44685b293a4696d18fb4cfa701b04db912",
         "43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8",
         "S01 -> S02 -> S01",
         "/v1/audio/transcriptions",
         "response_format=json",
+        "response_format=diarized_json",
         "response_format=verbose_json",
+        'vllm_response_format="diarized_json"',
         "[S01]",
         "Apache-2.0",
         "sentence_info",

tests/test_moss_transcribe_diarize_model.py

@@ -138,6 +138,89 @@ class MossVllmBackendTest(unittest.TestCase):
         self.assertEqual(content_type, "audio/wav")
         response.raise_for_status.assert_called_once_with()
 
+    def test_normalizes_official_vllm_diarized_json_response(self):
+        session = MagicMock()
+        response = MagicMock()
+        response.json.return_value = {
+            "task": "transcribe",
+            "duration": 3.5,
+            "text": "hello again",
+            "segments": [
+                {
+                    "type": "transcript.text.segment",
+                    "id": "seg_0",
+                    "start": 0.25,
+                    "end": 1.5,
+                    "text": "hello",
+                    "speaker": "S01",
+                },
+                {
+                    "type": "transcript.text.segment",
+                    "id": "seg_1",
+                    "start": 2.0,
+                    "end": 3.5,
+                    "text": "again",
+                    "speaker": "S02",
+                },
+            ],
+        }
+        session.post.return_value = response
+
+        model = MossTranscribeDiarize(
+            backend="vllm",
+            vllm_base_url="http://vllm.test:8000/v1",
+            vllm_response_format="diarized_json",
+            http_session=session,
+        )
+        results, _ = model.inference(
+            [np.zeros(1600, dtype=np.float32)], key=["meeting"]
+        )
+
+        self.assertEqual(
+            session.post.call_args.kwargs["data"]["response_format"],
+            "diarized_json",
+        )
+        self.assertEqual(results[0]["text"], "hello again")
+        self.assertEqual(results[0]["raw_text"], "hello again")
+        self.assertEqual(results[0]["timestamp"], [[250, 1500], [2000, 3500]])
+        self.assertEqual(
+            [(item["spk"], item["text"]) for item in results[0]["sentence_info"]],
+            [("S01", "hello"), ("S02", "again")],
+        )
+
+    def test_rejects_unsupported_vllm_response_format(self):
+        with self.assertRaisesRegex(ValueError, "json.*diarized_json"):
+            MossTranscribeDiarize(
+                backend="vllm",
+                vllm_base_url="http://vllm.test:8000/v1",
+                vllm_response_format="verbose_json",
+            )
+
+    def test_rejects_malformed_vllm_diarized_segment(self):
+        session = MagicMock()
+        response = MagicMock()
+        response.json.return_value = {
+            "text": "broken",
+            "segments": [
+                {
+                    "start": 2.0,
+                    "end": 1.0,
+                    "text": "broken",
+                    "speaker": "S01",
+                }
+            ],
+        }
+        session.post.return_value = response
+        model = MossTranscribeDiarize(
+            backend="vllm",
+            vllm_base_url="http://vllm.test:8000/v1",
+            vllm_response_format="diarized_json",
+            http_session=session,
+        )
+
+        with self.assertRaisesRegex(RuntimeError, "segment 0.*invalid contract"):
+            model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
+
 
 if __name__ == "__main__":
     unittest.main()

web-pages/product-site/data/deployments.json

@@ -1,6 +1,6 @@
 {
   "schema_version": 1,
-  "verified": "2026-08-29",
+  "verified": "2026-08-30",
   "deployments": [
     {
       "id": "moss-transcribe-diarize",
@@ -13,10 +13,10 @@
       "models": ["OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)"],
       "operating_systems": ["Linux"],
       "interfaces": ["OpenAI-compatible HTTP", "vLLM", "SGLang Omni", "Transformers"],
-      "tested": {"funasr": "AutoModel adapter@a3a2de0a; third-party model@e8681d68", "runtime": "Transformers 5.16.1 + vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129 / H100 80GB", "verified": "2026-08-29"},
+      "tested": {"funasr": "AutoModel diarized_json adapter; third-party model@e8681d68", "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB", "verified": "2026-08-30"},
       "commands": {
         "install": [
-          "uv venv --python 3.12 .venv-moss && uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' --torch-backend=auto --extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129",
+          "uv venv --python 3.12 .venv-moss && curl -fL https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl && echo 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b  vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl' | sha256sum -c - && uv pip install --python .venv-moss/bin/python --torch-backend=auto \"vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl\"",
           "HF_HUB_ENABLE_HF_TRANSFER=1 hf download OpenMOSS-Team/MOSS-Transcribe-Diarize --revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 --local-dir .models/moss-transcribe-diarize"
         ],
         "launch": [
@@ -27,9 +27,9 @@
           "curl -fsS http://127.0.0.1:8898/v1/models"
         ],
         "smoke": [
-          "curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F file=@runtime/llama.cpp/tests/sample.wav -F model=moss-transcribe-diarize -F response_format=json -F temperature=0 | tee /tmp/moss-transcription.json",
-          "python - <<'PY'\nimport json\n\nwith open('/tmp/moss-transcription.json', encoding='utf-8') as stream:\n    payload = json.load(stream)\ntext = payload.get('text', '')\nassert text.strip(), payload\nassert '[S01]' in text, text\nprint(text)\nPY",
-          "python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='vllm', vllm_base_url='http://127.0.0.1:8898/v1', vllm_model='moss-transcribe-diarize', disable_update=True)\nresult = model.generate('runtime/llama.cpp/tests/sample.wav')[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"
+          "curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F file=@runtime/llama.cpp/tests/sample.wav -F model=moss-transcribe-diarize -F response_format=diarized_json -F temperature=0 | tee /tmp/moss-transcription.json",
+          "python - <<'PY'\nimport json\n\nwith open('/tmp/moss-transcription.json', encoding='utf-8') as stream:\n    payload = json.load(stream)\ntext = payload.get('text', '')\nsegments = payload.get('segments', [])\nassert text.strip() and segments, payload\nassert all(isinstance(item.get('speaker'), str) and item.get('text') and item.get('start') <= item.get('end') for item in segments), payload\nprint(text, sorted({item['speaker'] for item in segments}))\nPY",
+          "python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='vllm', vllm_base_url='http://127.0.0.1:8898/v1', vllm_model='moss-transcribe-diarize', vllm_response_format='diarized_json', disable_update=True)\nresult = model.generate('runtime/llama.cpp/tests/sample.wav')[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"
         ]
       },
       "evidence": [
@@ -37,22 +37,24 @@
         {"label": "upstream source pinned at cb765f2", "url": "https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/commit/cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3"},
         {"label": "official model snapshot pinned at e8681d68", "url": "https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize/tree/e8681d68e7042738ffca8ac8212bc8fcb1131ab8"},
         {"label": "official vLLM and SGLang Omni serving guide", "url": "https://github.com/OpenMOSS/MOSS-Transcribe-Diarize/blob/cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3/README.md"},
+        {"label": "vLLM official diarized_json response contract", "url": "https://github.com/vllm-project/vllm/pull/48543"},
+        {"label": "verified vLLM 0.27.1 release", "url": "https://github.com/vllm-project/vllm/releases/tag/v0.27.1"},
         {"label": "FunASR ecosystem integration guide", "url": "https://github.com/modelscope/FunASR/blob/main/docs/moss_transcribe_diarize.md"},
         {"label": "FunASR AutoModel adapter merge and exact-head CI", "url": "https://github.com/modelscope/FunASR/pull/3558"}
       ],
       "benchmarks": [
         {
           "model": "OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68",
-          "runtime": "vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129",
+          "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129",
           "hardware": "NVIDIA H100 80GB HBM3",
-          "workload": "OpenAI-compatible transcription API smoke test plus synthetic A-to-B-to-A speaker-return probe",
-          "audio": "Bundled 6.000 s Mandarin sample; synthetic 19.147 s file built as sample A + 0.8 s silence + sample B + 0.8 s silence + sample A",
-          "settings": "Pinned model and vLLM revisions; vllm[audio]; response_format=json; temperature=0; one request at a time",
-          "timing_scope": "Client wall time after /health became ready; local HTTP and decoding included; model download and engine startup excluded",
-          "result": "Both requests returned HTTP 200. The 6.000 s sample produced S01 with timestamps 0.96-5.94. The 19.147 s probe produced S01 -> S02 -> S01 with timestamps covering 0.96-19.08. Observed client times were 1.034 s and 0.171 s respectively.",
-          "qualification": "A functional API contract and speaker-return probe on synthetic concatenation, not a diarization accuracy study, overlap test, throughput benchmark, or production capacity promise.",
+          "workload": "OpenAI-compatible diarized_json API smoke plus FunASR AutoModel structured-response mapping",
+          "audio": "15.1685 s two-speaker probe; SHA256 43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8",
+          "settings": "Pinned model revision; vLLM 0.27.1; vllm[audio]; response_format=diarized_json; temperature=0; max_completion_tokens=768",
+          "timing_scope": "Functional response validation after /health became ready; no latency or throughput claim",
+          "result": "Direct vLLM HTTP and the FunASR adapter each returned two non-empty monotonic segments with speakers S01 and S02.",
+          "qualification": "A structured response contract smoke, not a diarization accuracy study, overlap test, throughput benchmark, or production capacity promise.",
           "source": "https://github.com/modelscope/FunASR/blob/main/docs/moss_transcribe_diarize.md",
-          "verified": "2026-08-29"
+          "verified": "2026-08-30"
         }
       ],
       "translations": {
@@ -62,11 +64,11 @@
           "fit": ["会议、访谈、播客和通话的多人长音频", "希望通过同一个模型输出文本、时间戳与说话人编号", "已有 NVIDIA GPU，并需要 OpenAI 兼容的音频转写接口"],
           "not_fit": ["实时流式字幕或低延迟端点检测", "CPU、桌面边缘设备或 Windows 原生部署", "要求 FunASR 自有模型权重的项目"],
           "selection_reason": "模型端到端联合生成转写与说话人标签，用户不必再拼接外部 VAD、ASR 和 diarization 管线；FunASR AutoModel 为 Transformers 与 vLLM 提供统一的结构化结果。",
-          "primary_limitation": "这是 OpenMOSS 的第三方模型，不是 FunASR 模型；“无需外部 VAD”不表示模型内部没有分块或分段。当前 vLLM 路径固定 nightly commit，升级前必须重跑真实多人长音频。",
+          "primary_limitation": "这是 OpenMOSS 的第三方模型，不是 FunASR 模型；“无需外部 VAD”不表示模型内部没有分块或分段。当前 vLLM 路径固定 0.27.1 发布版，升级前必须重跑真实多人长音频。",
           "status_label": "社区验证",
-          "operations": ["固定模型 revision、FunASR adapter merge、vLLM nightly commit、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "FunASR AutoModel 保留 raw_text，并统一返回 text、timestamp 和带 spk 的 sentence_info；直接调用 vLLM 时 json 仍是原始 [Sxx] 标记文本"],
+          "operations": ["固定模型 revision、FunASR adapter merge、vLLM 0.27.1、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "FunASR AutoModel 可用 vLLM diarized_json 直接统一 text、timestamp 和带 spk 的 sentence_info；json 兼容路径仍在 raw_text 保留原始 [Sxx] 标记文本"],
           "security": ["把服务绑定内网，由网关完成认证、TLS、限流及音频大小/时长限制", "固定并审计 trust_remote_code 对应的模型 revision，不执行浮动 main 代码", "隔离模型缓存、上传临时目录和生成日志，按数据保留策略清理原始音频"],
-          "troubleshooting": ["启动前确认 CUDA 12 使用 cu129 nightly 源，CUDA 13 使用上游文档列出的 cu130 源", "长音频被截断时提高 max_completion_tokens，并同时监控显存、延迟和输出完整性", "直接调用 vLLM 时 json 没有 segments 字段是当前接口边界；使用 FunASR AutoModel 归一化结果，或改用并单独验证 SGLang Omni verbose_json"]
+          "troubleshooting": ["CUDA 12 使用已校验 SHA256 的 vLLM 0.27.1 cu129 wheel；其他 CUDA 栈需按上游发布资产重新验证", "长音频被截断时提高 max_completion_tokens，并同时监控显存、延迟和输出完整性", "需要结构化说话人 segments 时使用 vLLM diarized_json；旧 68b4a1d5 nightly 会返回 HTTP 400，json 只返回原始 [Sxx] 标签文本"]
         },
         "en": {
           "name": "MOSS unified transcription and diarization",
@@ -74,11 +76,11 @@
           "fit": ["Multi-speaker meetings, interviews, podcasts, and calls", "One model output for text, timestamps, and speaker identifiers", "NVIDIA GPU deployments that need an OpenAI-compatible audio transcription endpoint"],
           "not_fit": ["Realtime streaming captions or low-latency endpoint detection", "CPU, desktop-edge, or native Windows deployments", "Projects that require FunASR-owned model weights"],
           "selection_reason": "The model jointly emits transcription and speaker labels so users do not need to assemble an external VAD, ASR, and diarization pipeline; FunASR AutoModel provides one structured result across Transformers and vLLM.",
-          "primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins a nightly commit and must be revalidated on real long multi-speaker audio before upgrades.",
+          "primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins release 0.27.1 and must be revalidated on real long multi-speaker audio before upgrades.",
           "status_label": "Community verified",
-          "operations": ["Pin the model revision, FunASR adapter merge, vLLM nightly commit, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "FunASR AutoModel preserves raw_text and normalizes text, timestamp, and sentence_info with spk; direct vLLM json still returns raw [Sxx]-tagged text"],
+          "operations": ["Pin the model revision, FunASR adapter merge, vLLM 0.27.1, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "FunASR AutoModel can map vLLM diarized_json directly into text, timestamp, and sentence_info with spk; the json compatibility path still preserves raw [Sxx]-tagged text in raw_text"],
           "security": ["Bind workers to a private network and put authentication, TLS, rate limits, and audio size/duration limits at the gateway", "Pin and audit the trust_remote_code model revision instead of executing a floating main revision", "Isolate model caches, upload directories, and generation logs; remove source audio according to retention policy"],
-          "troubleshooting": ["Use the cu129 nightly source for CUDA 12 and the upstream cu130 source for CUDA 13", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "A direct vLLM json response without segments is the current API boundary; normalize it with FunASR AutoModel or use and separately validate SGLang Omni verbose_json"]
+          "troubleshooting": ["For CUDA 12, use the SHA256-verified vLLM 0.27.1 cu129 wheel; revalidate another CUDA stack against its upstream release asset", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "Use vLLM diarized_json for structured speaker segments; the old 68b4a1d5 nightly returns HTTP 400, while json only returns raw [Sxx]-tagged text"]
         }
       }
     },

web-pages/product-site/tests/browser/product-site.spec.ts

@@ -186,7 +186,9 @@ for (const viewport of [
     await expect(page.getByText('FunASR AutoModel', { exact: false }).first()).toBeVisible();
     await expect(page.getByText('OpenMOSS', { exact: false }).first()).toBeVisible();
     await expect(page.getByText('sentence_info', { exact: false }).first()).toBeVisible();
+    await expect(page.getByText('diarized_json', { exact: false }).first()).toBeVisible();
     await expect(page.locator('a[href="https://github.com/modelscope/FunASR/pull/3558"]')).toBeVisible();
+    await expect(page.locator('a[href="https://github.com/vllm-project/vllm/pull/48543"]')).toBeVisible();
     await expect(page.locator('a[href="/en/deploy/moss-transcribe-diarize.html"]')).toBeVisible();
 
     const layout = await page.evaluate(() => ({

web-pages/product-site/tests/test_registry.py

@@ -107,14 +107,15 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
         'OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)'
     ]
     assert entry['tested'] == {
-        'funasr': 'AutoModel adapter@a3a2de0a; third-party model@e8681d68',
-        'runtime': 'Transformers 5.16.1 + vLLM 0.23.1rc1.dev949+g68b4a1d58 / Torch 2.11.0+cu129 / H100 80GB',
-        'verified': '2026-08-29',
+        'funasr': 'AutoModel diarized_json adapter; third-party model@e8681d68',
+        'runtime': 'vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB',
+        'verified': '2026-08-30',
     }
 
     install = '\n'.join(entry['commands']['install'])
     assert 'vllm[audio]' in install
-    assert 'wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129' in install
+    assert 'vllm-0.27.1%2Bcu129' in install
+    assert 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b' in install
     assert 'OpenMOSS-Team/MOSS-Transcribe-Diarize' in install
     assert 'e8681d68e7042738ffca8ac8212bc8fcb1131ab8' in install
 
@@ -123,10 +124,11 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
     assert '--trust-remote-code' in launch
     smoke = '\n'.join(entry['commands']['smoke'])
     assert '/v1/audio/transcriptions' in smoke
-    assert 'response_format=json' in smoke
-    assert '[S01]' in smoke
+    assert 'response_format=diarized_json' in smoke
+    assert "payload.get('segments'" in smoke
     assert "from funasr import AutoModel" in smoke
     assert "backend='vllm'" in smoke
+    assert "vllm_response_format='diarized_json'" in smoke
     assert "result['raw_text']" in smoke
     assert "result['sentence_info']" in smoke
 
@@ -141,6 +143,7 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
         'e8681d68e7042738ffca8ac8212bc8fcb1131ab8'
     ) in evidence_urls
     assert 'https://github.com/modelscope/FunASR/pull/3558' in evidence_urls
+    assert 'https://github.com/vllm-project/vllm/pull/48543' in evidence_urls
 
     english = entry['translations']['en']
     assert 'OpenMOSS' in english['summary']
@@ -151,14 +154,16 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
     assert 'FunASR AutoModel API' not in ' '.join(english['not_fit'])
     assert 'sentence_info' in english['operations'][-1]
     assert 'raw_text' in english['operations'][-1]
+    assert 'diarized_json' in english['operations'][-1]
+    assert 'diarized_json' in english['troubleshooting'][-1]
     assert 'internal segmentation' in english['primary_limitation']
     assert 'FunASR model' in english['primary_limitation']
     assert any(
         benchmark['hardware'] == 'NVIDIA H100 80GB HBM3'
-        and '6.000 s' in benchmark['audio']
-        and '19.147 s' in benchmark['audio']
-        and 'S01 -> S02 -> S01' in benchmark['result']
-        and 'functional API contract' in benchmark['qualification']
+        and '15.1685 s' in benchmark['audio']
+        and '43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8' in benchmark['audio']
+        and 'S01 and S02' in benchmark['result']
+        and 'contract smoke' in benchmark['qualification']
         for benchmark in entry['benchmarks']
     )
 
@@ -167,7 +172,7 @@ def test_audio_cpp_contract_tracks_mainline_nano_and_sensevoice(valid_registry):
     entry = next(item for item in valid_registry['deployments'] if item['id'] == 'audio-cpp')
     llama_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'llama-cpp')
 
-    assert valid_registry['verified'] == '2026-08-29'
+    assert valid_registry['verified'] == '2026-08-30'
     assert entry['maturity'] == 'community-verified'
     assert entry['selector_rank'] > llama_cpp['selector_rank']
     assert entry['tested'] == {
@@ -333,7 +338,7 @@ def test_sensevoice_native_server_contract_tracks_merged_runtime(valid_registry)
     llama_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'llama-cpp')
     audio_cpp = next(item for item in valid_registry['deployments'] if item['id'] == 'audio-cpp')
 
-    assert valid_registry['verified'] == '2026-08-29'
+    assert valid_registry['verified'] == '2026-08-30'
     assert entry['maturity'] == 'production-verified'
     assert llama_cpp['selector_rank'] < entry['selector_rank'] < audio_cpp['selector_rank']
     assert entry['tested'] == {