feat(moss): add SGLang AutoModel backend

Signed-off-by: LauraGPT <[email protected]>
This commit is contained in:
LauraGPT
2026-08-30 15:21:25 +00:00
parent c7ec8bd007
commit 783c24adfb
11 changed files with 399 additions and 31 deletions
+1 -1
View File
@@ -15,7 +15,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm
| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |
| ONNX/C++ runtime | High-concurrency CPU services or embedded realtime ASR | [ONNX runtime docs](../runtime/onnxruntime/readme.md) | Keep this path when latency/concurrency is already proven; add text post-processing for fixed business terms before moving to GPU LLMs. |
| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. |
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`) or vLLM (`backend="vllm"`), or serve it independently through native SGLang Omni. SGLang Omni is not an `AutoModel` backend. The model remains published and maintained by OpenMOSS. |
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`), vLLM (`backend="vllm"`), or SGLang Omni (`backend="sglang"`). The model remains published and maintained by OpenMOSS. |
| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. |
| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. |
| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. |
+1 -1
View File
@@ -13,7 +13,7 @@
| Kubernetes API | Cluster service 向け internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP` から開始。公開範囲を広げる前に auth、TLS、network policy、GPU scheduling を追加します。 |
| Runtime WebSocket service | Live captions、meeting、call-center stream | [Runtime service docs](../runtime/readme.md) | partial result、endpointing、long-lived audio stream が重要な場合に使います。 |
| vLLM acceleration | Fun-ASR-Nano の LLM-based ASR throughput 向上 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput 向け。non-autoregressive Paraformer には適用しません。 |
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF(`backend="hf"`)または vLLM(`backend="vllm"`)を選択でき、native SGLang Omni から独立して serve することもできます。SGLang Omni は `AutoModel` backend ではなく、model の公開・保守主体は OpenMOSS のままです。 |
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF(`backend="hf"`)、vLLM(`backend="vllm"`)、または SGLang Omni(`backend="sglang"`)を選択できます。model の公開・保守主体は OpenMOSS のままです。 |
| MCP server | Claude/Cursor/desktop agent の speech tool | [MCP example](../examples/mcp_server/) | ASR 結果を local tool として Agent に渡したい場合に便利です。 |
| Subtitle generator | 長時間 audio/video から SRT/VTT 作成 | [Subtitle example](../examples/subtitle/) | readability が重要な場合は verbose segment と speaker label を使います。 |
| Batch ASR script | Archive、meeting、dataset、繰り返し offline run | [Batch example](../examples/batch_asr_improved.py) | production では queue、manifest、retry log を追加してください。 |
+1 -1
View File
@@ -13,7 +13,7 @@
| Kubernetes API | Cluster service용 internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP`부터 시작합니다. 범위를 넓히기 전에 auth, TLS, network policy, GPU scheduling을 추가하세요. |
| Runtime WebSocket service | Live captions, meeting, call-center stream | [Runtime service docs](../runtime/readme.md) | partial result, endpointing, long-lived audio stream이 중요할 때 사용합니다. |
| vLLM acceleration | Fun-ASR-Nano의 LLM-based ASR throughput 향상 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput용입니다. non-autoregressive Paraformer에는 적용되지 않습니다. |
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`) 또는 vLLM(`backend="vllm"`)을 선택하거나 native SGLang Omni에서 독립적으로 serve할 수 있습니다. SGLang Omni는 `AutoModel` backend가 아니며, model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`), vLLM(`backend="vllm"`) 또는 SGLang Omni(`backend="sglang"`)를 선택할 수 있습니다. model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
| MCP server | Claude/Cursor/desktop agent speech tool | [MCP example](../examples/mcp_server/) | ASR 결과를 local tool로 Agent에 전달할 때 유용합니다. |
| Subtitle generator | 긴 audio/video에서 SRT/VTT 생성 | [Subtitle example](../examples/subtitle/) | readability가 중요하면 verbose segment와 speaker label을 사용합니다. |
| Batch ASR script | Archive, meeting, dataset, 반복 offline run | [Batch example](../examples/batch_asr_improved.py) | production에서는 queue, manifest, retry log를 추가하세요. |
+1 -1
View File
@@ -15,7 +15,7 @@
| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |
| ONNX/C++ Runtime | 高并发 CPU 服务或嵌入式实时 ASR | [ONNX Runtime 文档](../runtime/onnxruntime/readme.md) | 如果延迟和并发已经验证,不要轻易替换;固定业务词优先做文本后处理。 |
| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐;不适用于非自回归 Paraformer。 |
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型,已接入 FunASR `AutoModel`;可选本地 HF(`backend="hf"`)或 vLLM(`backend="vllm"`),也可通过原生 SGLang Omni 独立服务。SGLang Omni 不是 `AutoModel` backend。模型仍由 OpenMOSS 发布和维护。 |
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型,已接入 FunASR `AutoModel`;可选本地 HF(`backend="hf"`)、vLLM(`backend="vllm"`)或 SGLang Omni(`backend="sglang"`)。模型仍由 OpenMOSS 发布和维护。 |
| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 |
| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 |
| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 |
+37 -2
View File
@@ -6,8 +6,8 @@ This guide connects the third-party
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
model to the FunASR deployment ecosystem. The model is published by OpenMOSS
under Apache-2.0; it is not a FunASR model. FunASR provides an adapter for its
public Transformers and vLLM interfaces while retaining the OpenMOSS model
name, license, and upstream revision.
public Transformers, vLLM, and SGLang Omni interfaces while retaining the
OpenMOSS model name, license, and upstream revision.
MOSS-Transcribe-Diarize jointly generates transcription, timestamps, and
speaker labels such as `[S01]`. An application therefore does not need to
@@ -259,6 +259,16 @@ claims as measurements of your host.
Follow the pinned upstream SGLang Omni installation guide for CUDA 13, then
download the immutable model snapshot and serve the local directory:
```bash
git clone https://github.com/sgl-project/sglang-omni.git
git -C sglang-omni checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e
```
This source pin includes the transcription API's `max_new_tokens` forwarding.
The original #914 merge predates that request field, so it is not sufficient
for the long-audio command below even though its published H100 benchmark
remains useful upstream evidence.
```bash
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
@@ -287,6 +297,31 @@ current `verbose_json` contract, the speaker identifier is retained as the
Parse and validate that prefix before wiring the response into subtitles,
meeting notes, or analytics.
The FunASR adapter performs that validation and maps the official SGLang
segments into the same `sentence_info` contract as the HF and vLLM paths:
```python
from funasr import AutoModel
model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
backend="sglang",
sglang_base_url="http://127.0.0.1:8898/v1",
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
max_new_tokens=65536,
disable_update=True,
)
result = model.generate(input="audio.wav", max_new_tokens=65536)[0]
for segment in result["sentence_info"]:
print(segment["start"], segment["end"], segment["spk"], segment["text"])
```
Do not pass `vad_model` or `spk_model`: MOSS performs segmentation and
anonymous speaker attribution jointly, and external splitting can destroy
speaker consistency across long turns. The adapter preserves the upstream
tagged transcript in `raw_text`, strips only the validated `[Sxx]` prefix from
each normalized segment, and fails closed if SGLang omits that prefix.
The native runtime was merged in SGLang Omni
[#914](https://github.com/sgl-project/sglang-omni/pull/914). Its single-H100
Seed-TTS EN benchmark completed 1088/1088 clips with no request failures. WER
+35 -2
View File
@@ -5,8 +5,8 @@
本文把第三方
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
模型接入 FunASR 部署生态。模型由 OpenMOSS 以 Apache-2.0 发布,不是 FunASR
自有模型。FunASR 只为其公开的 Transformers 与 vLLM 接口提供适配器,并保留
OpenMOSS 模型名称、许可证和上游 revision。
自有模型。FunASR 为其公开的 Transformers、vLLM 与 SGLang Omni 接口提供适配器,
并保留 OpenMOSS 模型名称、许可证和上游 revision。
MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话人标签,应用侧
不必再拼接外部 VAD、ASR 和 diarization 管线。这里描述的是部署形态,不表示模型
@@ -231,6 +231,15 @@ LocalAI `master@a7cc5873ef5b7c909fc9ff7d349d51738ba9bb05` 已包含
按固定上游 SGLang Omni 安装指南准备 CUDA 13 环境,然后下载不可变模型快照并从本地目录启动:
```bash
git clone https://github.com/sgl-project/sglang-omni.git
git -C sglang-omni checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e
```
该源码 pin 已把 `max_new_tokens` 传入 transcription 生成请求。最初的 #914
merge 早于这个请求字段,因此虽然它的 H100 benchmark 仍可作为上游证据,却不能
支撑下方长音频命令。
```bash
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
@@ -257,6 +266,30 @@ curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
`verbose_json` 合同把说话人编号保留为 `segments[].text` 的 `[Sxx]` 前缀,
并没有单独的 `speaker` 字段。接入字幕、会议纪要或分析系统前,应解析并校验此前缀。
FunASR 适配器会完成该校验,并把 SGLang 官方 segments 映射为与 HF、vLLM
一致的 `sentence_info` 合同:
```python
from funasr import AutoModel
model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
backend="sglang",
sglang_base_url="http://127.0.0.1:8898/v1",
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
max_new_tokens=65536,
disable_update=True,
)
result = model.generate(input="audio.wav", max_new_tokens=65536)[0]
for segment in result["sentence_info"]:
print(segment["start"], segment["end"], segment["spk"], segment["text"])
```
不要传入 `vad_model` 或 `spk_model`:MOSS 在一次生成中联合完成分段和匿名说话人
归属,外部分段会破坏长轮次中的说话人一致性。适配器把上游带标签原文保存在
`raw_text`,只从标准化 segment 中移除已经校验的 `[Sxx]` 前缀;如果 SGLang
没有返回此前缀,则明确失败,不会伪造说话人身份。
原生 runtime 已通过 SGLang Omni
[#914](https://github.com/sgl-project/sglang-omni/pull/914) 合并。其单张 H100
Seed-TTS EN benchmark 完成 1088/1088 条请求且无请求失败。WER 是在移除
+128 -5
View File
@@ -29,6 +29,7 @@ DEFAULT_PROMPT = (
_NUMBER = r"(?:\d+(?:\.\d+)?|\.\d+)"
_SEGMENT_START = re.compile(r"\[(%s)\]\[(S\d+)\]" % _NUMBER)
_TIMESTAMP = re.compile(r"\[(%s)\]" % _NUMBER)
_SGLANG_SPEAKER_PREFIX = re.compile(r"^\s*\[(S\d{2,})\]\s*")
def _parse_transcript(transcript):
@@ -188,6 +189,87 @@ def _result_from_diarized_response(key, response):
}
def _result_from_sglang_response(key, response):
"""Normalize SGLang Omni's MOSS ``verbose_json`` response."""
raw_text = response.get("text") if isinstance(response, dict) else None
segments = response.get("segments") if isinstance(response, dict) else None
if not isinstance(raw_text, str) or not isinstance(segments, list):
raise RuntimeError(
"SGLang verbose_json response requires text and segments fields"
)
source_segments = _parse_transcript(raw_text)
sentence_info = []
timestamps = []
texts = []
previous_start = None
for index, segment in enumerate(segments):
if not isinstance(segment, dict):
raise RuntimeError("SGLang segment %d must be an object" % index)
start = segment.get("start")
end = segment.get("end")
tagged_text = segment.get("text")
numeric = (
isinstance(start, (int, float))
and not isinstance(start, bool)
and isinstance(end, (int, float))
and not isinstance(end, bool)
)
if (
not numeric
or not math.isfinite(start)
or not math.isfinite(end)
or start < 0
or end < start
or not isinstance(tagged_text, str)
):
raise RuntimeError("SGLang segment %d has an invalid contract" % index)
if previous_start is not None and start < previous_start:
raise RuntimeError("SGLang segment %d has non-monotonic start time" % index)
previous_start = start
speaker_match = _SGLANG_SPEAKER_PREFIX.match(tagged_text)
if speaker_match is None:
raise RuntimeError(
"SGLang segment %d is missing the required speaker prefix" % index
)
text = tagged_text[speaker_match.end() :].strip()
if not text:
raise RuntimeError("SGLang segment %d has an invalid contract" % index)
if index >= len(source_segments):
raise RuntimeError(
"SGLang segment %d is not backed by raw transcript markers" % index
)
_, _, source_speaker, source_text = source_segments[index]
if source_speaker != speaker_match.group(1) or source_text != text:
raise RuntimeError(
"SGLang segment %d is not backed by raw transcript markers" % index
)
timestamp = [int(round(start * 1000)), int(round(end * 1000))]
timestamps.append(timestamp)
texts.append(text)
sentence_info.append(
{
"start": timestamp[0],
"end": timestamp[1],
"text": text,
"sentence": text,
"spk": speaker_match.group(1),
"timestamp": [timestamp],
}
)
if len(source_segments) != len(segments):
raise RuntimeError(
"SGLang segments are not backed by raw transcript speaker markers"
)
return {
"key": key,
"text": _join_texts(texts),
"raw_text": raw_text,
"timestamp": timestamps,
"sentence_info": sentence_info,
}
@tables.register("model_classes", "MOSS-Transcribe-Diarize")
@tables.register("model_classes", DEFAULT_MODEL)
class MossTranscribeDiarize(nn.Module):
@@ -203,8 +285,8 @@ class MossTranscribeDiarize(nn.Module):
)
self.backend = kwargs.get("backend", "hf").lower()
if self.backend not in {"hf", "vllm"}:
raise ValueError("backend must be 'hf' or 'vllm'")
if self.backend not in {"hf", "vllm", "sglang"}:
raise ValueError("backend must be 'hf', 'vllm', or 'sglang'")
self.model_path = (
kwargs.get("model_path") or kwargs.get("model") or DEFAULT_MODEL
)
@@ -225,12 +307,19 @@ class MossTranscribeDiarize(nn.Module):
if self.vllm_response_format not in {"json", "diarized_json"}:
raise ValueError("vllm_response_format must be 'json' or 'diarized_json'")
self.http_session = kwargs.get("http_session")
self.sglang_base_url = kwargs.get("sglang_base_url")
self.sglang_model = kwargs.get("sglang_model", DEFAULT_MODEL)
self.sglang_api_key = kwargs.get("sglang_api_key", "EMPTY")
self.sglang_timeout = float(kwargs.get("sglang_timeout", 600.0))
self.hf_model = None
self.processor = None
if self.backend == "vllm":
if not self.vllm_base_url:
raise ValueError("vllm_base_url is required when backend='vllm'")
elif self.backend == "sglang":
if not self.sglang_base_url:
raise ValueError("sglang_base_url is required when backend='sglang'")
else:
self._load_hf_backend(kwargs)
@@ -316,6 +405,9 @@ class MossTranscribeDiarize(nn.Module):
results.append(
_result_from_transcript(keys[index], response["text"])
)
elif self.backend == "sglang":
response = self._transcribe_sglang(audio, **kwargs)
results.append(_result_from_sglang_response(keys[index], response))
else:
transcript = self._transcribe_hf(audio, **kwargs)
results.append(_result_from_transcript(keys[index], transcript))
@@ -384,8 +476,9 @@ class MossTranscribeDiarize(nn.Module):
generated, skip_special_tokens=True
).strip()
def _transcriptions_url(self):
base = self.vllm_base_url.rstrip("/")
@staticmethod
def _transcriptions_url(base_url):
base = base_url.rstrip("/")
if base.endswith("/v1/audio/transcriptions"):
return base
if not base.endswith("/v1"):
@@ -415,7 +508,7 @@ class MossTranscribeDiarize(nn.Module):
if self.vllm_api_key and self.vllm_api_key != "EMPTY":
headers["Authorization"] = "Bearer " + self.vllm_api_key
response = session.post(
self._transcriptions_url(),
self._transcriptions_url(self.vllm_base_url),
data=data,
files={"file": (filename, payload, content_type)},
headers=headers,
@@ -433,6 +526,36 @@ class MossTranscribeDiarize(nn.Module):
)
return result
def _transcribe_sglang(self, audio, **kwargs):
import requests
session = self.http_session or requests.Session()
filename, payload, content_type = self._multipart_audio(audio)
data = {
"model": self.sglang_model,
"response_format": "verbose_json",
"temperature": str(kwargs.get("temperature", 0)),
"max_new_tokens": str(kwargs.get("max_new_tokens", self.max_new_tokens)),
}
prompt = kwargs.get("prompt")
if prompt:
data["prompt"] = prompt
headers = {}
if self.sglang_api_key and self.sglang_api_key != "EMPTY":
headers["Authorization"] = "Bearer " + self.sglang_api_key
response = session.post(
self._transcriptions_url(self.sglang_base_url),
data=data,
files={"file": (filename, payload, content_type)},
headers=headers,
timeout=self.sglang_timeout,
)
response.raise_for_status()
result = response.json()
if not isinstance(result, dict):
raise RuntimeError("SGLang transcription response must be a JSON object")
return result
@staticmethod
def _multipart_audio(audio):
if isinstance(audio, np.ndarray):
@@ -26,6 +26,7 @@ def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) ->
"OpenMOSS-Team/MOSS-Transcribe-Diarize",
"e8681d68e7042738ffca8ac8212bc8fcb1131ab8",
"6e448d0ea9bf3d88d898b65449ca6dc2aec170ac",
"3f819f9cdae3d4eeec22f73306c9067a1ec2542e",
"bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b",
"vllm[audio]",
"vLLM 0.27.1",
@@ -38,6 +39,9 @@ def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) ->
"response_format=diarized_json",
"response_format=verbose_json",
'vllm_response_format="diarized_json"',
'backend="sglang"',
'sglang_base_url="http://127.0.0.1:8898/v1"',
"max_new_tokens=65536",
"[S01]",
"Apache-2.0",
"sentence_info",
@@ -66,6 +70,7 @@ def test_all_deployment_matrices_link_moss_guide(matrix: Path) -> None:
assert "moss_transcribe_diarize.md" in moss_row
assert '`backend="hf"`' in moss_row
assert '`backend="vllm"`' in moss_row
assert '`backend="sglang"`' in moss_row
for stale_claim in (
"not a FunASR-owned model or `AutoModel` backend",
"不是 FunASR 自有模型或 `AutoModel` 后端",
+166
View File
@@ -264,5 +264,171 @@ class MossVllmBackendTest(unittest.TestCase):
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
class MossSglangBackendTest(unittest.TestCase):
def test_auto_model_sglang_path_bypasses_weight_download(self):
from funasr.auto.auto_model import AutoModel
with patch(
"funasr.auto.auto_model.download_model",
side_effect=AssertionError(
"SGLang client mode must not download model weights"
),
):
auto_model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
backend="sglang",
sglang_base_url="http://sglang.test:8000/v1",
device="cpu",
disable_update=True,
)
self.assertIsInstance(auto_model.model, MossTranscribeDiarize)
def test_normalizes_official_sglang_verbose_json_response(self):
session = MagicMock()
response = MagicMock()
response.json.return_value = {
"text": "[0.25][S01]hello[1.5][2.0][S02]again[3.5]",
"segments": [
{"start": 0.25, "end": 1.5, "text": "[S01]hello"},
{"start": 2.0, "end": 3.5, "text": "[S02] again"},
],
}
session.post.return_value = response
model = MossTranscribeDiarize(
backend="sglang",
sglang_base_url="http://sglang.test:8000/v1",
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
sglang_api_key="secret",
http_session=session,
)
results, _ = model.inference(
[np.zeros(1600, dtype=np.float32)],
key=["meeting"],
prompt="transcribe and diarize",
max_new_tokens=8192,
)
request = session.post.call_args
self.assertEqual(
request.args[0],
"http://sglang.test:8000/v1/audio/transcriptions",
)
self.assertEqual(
request.kwargs["data"],
{
"model": "OpenMOSS-Team/MOSS-Transcribe-Diarize",
"response_format": "verbose_json",
"temperature": "0",
"prompt": "transcribe and diarize",
"max_new_tokens": "8192",
},
)
self.assertEqual(request.kwargs["headers"], {"Authorization": "Bearer secret"})
self.assertEqual(results[0]["text"], "hello again")
self.assertEqual(
results[0]["raw_text"],
"[0.25][S01]hello[1.5][2.0][S02]again[3.5]",
)
self.assertEqual(results[0]["timestamp"], [[250, 1500], [2000, 3500]])
self.assertEqual(
[(item["spk"], item["text"]) for item in results[0]["sentence_info"]],
[("S01", "hello"), ("S02", "again")],
)
def test_rejects_sglang_segment_without_speaker_prefix(self):
session = MagicMock()
response = MagicMock()
response.json.return_value = {
"text": "hello",
"segments": [{"start": 0.0, "end": 1.0, "text": "hello"}],
}
session.post.return_value = response
model = MossTranscribeDiarize(
backend="sglang",
sglang_base_url="http://sglang.test:8000/v1",
http_session=session,
)
with self.assertRaisesRegex(RuntimeError, "segment 0.*speaker prefix"):
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
def test_rejects_sglang_fallback_speaker_not_backed_by_raw_transcript(self):
session = MagicMock()
response = MagicMock()
response.json.return_value = {
"text": "hello",
"segments": [{"start": 0.0, "end": 1.0, "text": "[S01]hello"}],
}
session.post.return_value = response
model = MossTranscribeDiarize(
backend="sglang",
sglang_base_url="http://sglang.test:8000/v1",
http_session=session,
)
with self.assertRaisesRegex(RuntimeError, "not backed by raw transcript"):
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
def test_rejects_one_digit_sglang_speaker_label(self):
session = MagicMock()
response = MagicMock()
response.json.return_value = {
"text": "[0][S1]hello[1]",
"segments": [{"start": 0.0, "end": 1.0, "text": "[S1]hello"}],
}
session.post.return_value = response
model = MossTranscribeDiarize(
backend="sglang",
sglang_base_url="http://sglang.test:8000/v1",
http_session=session,
)
with self.assertRaisesRegex(RuntimeError, "segment 0.*speaker prefix"):
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
def test_rejects_sglang_segments_with_decreasing_start_times(self):
session = MagicMock()
response = MagicMock()
response.json.return_value = {
"text": "[2][S01]late[3][1][S02]early[1.5]",
"segments": [
{"start": 2.0, "end": 3.0, "text": "[S01]late"},
{"start": 1.0, "end": 1.5, "text": "[S02]early"},
],
}
session.post.return_value = response
model = MossTranscribeDiarize(
backend="sglang",
sglang_base_url="http://sglang.test:8000/v1",
http_session=session,
)
with self.assertRaisesRegex(RuntimeError, "segment 1.*non-monotonic"):
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
def test_accepts_sglang_timestamp_clamp_when_speaker_and_text_match_raw(self):
session = MagicMock()
response = MagicMock()
response.json.return_value = {
"text": "[0][S01]hello[99]",
"segments": [{"start": 0.0, "end": 10.0, "text": "[S01]hello"}],
}
session.post.return_value = response
model = MossTranscribeDiarize(
backend="sglang",
sglang_base_url="http://sglang.test:8000/v1",
http_session=session,
)
results, _ = model.inference(
[np.zeros(1600, dtype=np.float32)], key=["meeting"]
)
self.assertEqual(results[0]["timestamp"], [[0, 10000]])
self.assertEqual(results[0]["sentence_info"][0]["spk"], "S01")
if __name__ == "__main__":
unittest.main()
+15 -14
View File
@@ -13,7 +13,7 @@
"models": ["OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)"],
"operating_systems": ["Linux"],
"interfaces": ["OpenAI-compatible HTTP", "vLLM", "SGLang Omni", "Transformers", "LocalAI / moss-transcribe.cpp"],
"tested": {"funasr": "AutoModel diarized_json adapter; third-party model@e8681d68", "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB", "verified": "2026-08-30"},
"tested": {"funasr": "AutoModel vLLM + SGLang adapters; third-party model@e8681d68", "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB", "verified": "2026-08-30"},
"commands": {
"install": [
"uv venv --python 3.12 .venv-moss && curl -fL https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl && echo 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl' | sha256sum -c - && uv pip install --python .venv-moss/bin/python --torch-backend=auto \"vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl\"",
@@ -61,14 +61,14 @@
},
{
"id": "sglang-omni",
"tested": "SGLang Omni merge 8458f76a / single H100 upstream benchmark",
"tested": "SGLang Omni 3f819f9c / FunASR adapter contract-tested / #914 H100 upstream benchmark",
"translations": {
"zh": {"name": "SGLang Omni 原生服务", "summary": "使用已合并的原生 MOSS pipeline 和 OpenAI-compatible verbose_json 接口;它是独立上游 runtime,不是 FunASR AutoModel backend。"},
"en": {"name": "Native SGLang Omni serving", "summary": "Use the merged native MOSS pipeline and OpenAI-compatible verbose_json endpoint. This is an independent upstream runtime, not a FunASR AutoModel backend."}
"zh": {"name": "SGLang Omni + FunASR AutoModel", "summary": "使用已合并的原生 MOSS pipeline 和 OpenAI-compatible verbose_json 接口,并通过 FunASR AutoModel 校验 [Sxx] 前缀、统一为 sentence_info。"},
"en": {"name": "SGLang Omni + FunASR AutoModel", "summary": "Use the merged native MOSS pipeline and OpenAI-compatible verbose_json endpoint, then validate [Sxx] prefixes and normalize them into sentence_info through FunASR AutoModel."}
},
"commands": {
"install": [
"git clone https://github.com/sgl-project/sglang-omni.git && cd sglang-omni && git checkout 8458f76ab25f5ba9152b05929b40e07618aff2ce && uv venv .venv -p 3.12 && uv pip install --python .venv/bin/python -v -e .",
"git clone https://github.com/sgl-project/sglang-omni.git && cd sglang-omni && git checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e && uv venv .venv -p 3.12 && uv pip install --python .venv/bin/python -v -e .",
"HF_HUB_ENABLE_HF_TRANSFER=1 hf download OpenMOSS-Team/MOSS-Transcribe-Diarize --revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 --local-dir .models/moss-transcribe-diarize",
"curl -fsSL https://raw.githubusercontent.com/modelscope/FunASR/8d65a38a8f4f3b5301be72905096219dde443f73/runtime/llama.cpp/tests/sample.wav -o moss-sample.wav && echo 'ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e moss-sample.wav' | sha256sum -c -"
],
@@ -81,7 +81,8 @@
],
"smoke": [
"curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F [email protected] -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize -F response_format=verbose_json | tee /tmp/moss-sglang-transcription.json",
"python - <<'PY'\nimport json\nimport re\n\nwith open('/tmp/moss-sglang-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\nsegments = payload.get('segments', [])\nassert payload.get('text', '').strip() and segments, payload\nassert all(item.get('start') <= item.get('end') and re.match(r'^\\[S\\d{2,}\\]', item.get('text', '')) for item in segments), payload\nprint(payload['text'])\nprint([(item['start'], item['end'], item['text'][:5]) for item in segments])\nPY"
"python - <<'PY'\nimport json\nimport re\n\nwith open('/tmp/moss-sglang-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\nsegments = payload.get('segments', [])\nassert payload.get('text', '').strip() and segments, payload\nassert all(item.get('start') <= item.get('end') and re.match(r'^\\[S\\d{2,}\\]', item.get('text', '')) for item in segments), payload\nprint(payload['text'])\nprint([(item['start'], item['end'], item['text'][:5]) for item in segments])\nPY",
"python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='sglang', sglang_base_url='http://127.0.0.1:8898/v1', sglang_model='OpenMOSS-Team/MOSS-Transcribe-Diarize', max_new_tokens=65536, disable_update=True)\nresult = model.generate('moss-sample.wav', max_new_tokens=65536)[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"
]
}
}
@@ -134,24 +135,24 @@
"summary": "通过 FunASR AutoModel 或 OpenAI 兼容服务接入 OpenMOSS 发布的第三方 Apache-2.0 模型,一次生成长音频转写、时间戳和说话人标签。",
"fit": ["会议、访谈、播客和通话的多人长音频", "希望通过同一个模型输出文本、时间戳与说话人编号", "已有 NVIDIA GPU,并需要 vLLM 或 SGLang Omni 的 OpenAI 兼容音频转写接口", "通过 LocalAI / moss-transcribe.cpp 在 CPU 或桌面边缘 GPU 上运行第三方 GGUF 实现"],
"not_fit": ["实时流式字幕或低延迟端点检测", "要求 FunASR AutoModel 与 LocalAI C++ 路径输出完全相同结果的部署", "未经目标硬件复测的原生 Windows 部署", "要求 FunASR 自有模型权重的项目"],
"selection_reason": "模型端到端联合生成转写与说话人标签,用户不必再拼接外部 VAD、ASR 和 diarization 管线;FunASR AutoModel 为 Transformers 与 vLLM 提供统一的结构化结果,SGLang Omni 提供独立的原生上游服务,LocalAI 则提供独立的第三方 C++/GGUF 边缘路径。",
"primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。vLLM 路径固定 0.27.1,SGLang Omni 路径固定 merge 8458f76a;升级前必须重跑真实多人长音频。SGLang Omni 不是 FunASR AutoModel backend,其 verbose_json 把说话人编号保留在 segments[].text 的 [Sxx] 前缀。LocalAI / moss-transcribe.cpp 也是独立第三方重写。",
"selection_reason": "模型端到端联合生成转写与说话人标签,用户不必再拼接外部 VAD、ASR 和 diarization 管线;FunASR AutoModel 为 Transformers、vLLM 与 SGLang Omni 提供统一的结构化结果,LocalAI 则提供独立的第三方 C++/GGUF 边缘路径。",
"primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。vLLM 路径固定 0.27.1,SGLang Omni 路径固定 3f819f9c;升级前必须重跑真实多人长音频。SGLang Omni 的 verbose_json 把说话人编号保留在 segments[].text 的 [Sxx] 前缀,FunASR adapter 会校验该分段能够回溯到 raw_text 后再解析。LocalAI / moss-transcribe.cpp 也是独立第三方重写。",
"status_label": "社区验证",
"operations": ["固定模型 revision、FunASR adapter merge、vLLM 0.27.1、SGLang Omni merge 8458f76a、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "SGLang Omni 的 1088 条单说话人英文 benchmark 不覆盖 diarization 或 timestamp 准确率,生产前必须单独验证", "LocalAI 路径固定第三方 C++ backend 与 GGUF revision,并分别验证 CPU/GPU backend、量化精度和 OpenAI 接口响应", "FunASR AutoModel 可用 vLLM diarized_json 直接统一 text、timestamp 和带 spk 的 sentence_info;json 兼容路径仍在 raw_text 保留原始 [Sxx] 标记文本"],
"operations": ["固定模型 revision、FunASR adapter merge、vLLM 0.27.1、SGLang Omni 3f819f9c、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "SGLang Omni #914 的 1088 条单说话人英文 benchmark 不覆盖 diarization 或 timestamp 准确率,生产前必须单独验证", "LocalAI 路径固定第三方 C++ backend 与 GGUF revision,并分别验证 CPU/GPU backend、量化精度和 OpenAI 接口响应", "FunASR AutoModel 可把 vLLM diarized_json 与 SGLang Omni verbose_json 统一为 text、timestamp 和带 spk 的 sentence_info;SGLang 分段必须能回溯到 raw_text,避免接受上游合成的 S01 fallback"],
"security": ["把服务绑定内网,由网关完成认证、TLS、限流及音频大小/时长限制", "固定并审计 trust_remote_code 对应的模型 revision,不执行浮动 main 代码", "隔离模型缓存、上传临时目录和生成日志,按数据保留策略清理原始音频"],
"troubleshooting": ["CUDA 12 使用已校验 SHA256 的 vLLM 0.27.1 cu129 wheel;SGLang Omni 当前固定 CUDA 13 安装契约和 merge 8458f76a", "长音频被截断时提高 max_completion_tokens,并同时监控显存、延迟和输出完整性", "vLLM diarized_json 提供独立 speaker 字段;SGLang Omni verbose_json 当前把说话人编号放在 segments[].text 的 [Sxx] 前缀"]
"troubleshooting": ["CUDA 12 使用已校验 SHA256 的 vLLM 0.27.1 cu129 wheel;SGLang Omni 当前固定 CUDA 13 安装契约和 3f819f9c", "长音频被截断时,vLLM 提高 max_completion_tokens,SGLang Omni 提高 max_new_tokens,并同时监控显存、延迟和输出完整性", "vLLM diarized_json 提供独立 speaker 字段;SGLang Omni verbose_json 当前把说话人编号放在 segments[].text 的 [Sxx] 前缀"]
},
"en": {
"name": "MOSS unified transcription and diarization",
"summary": "Use FunASR AutoModel or an OpenAI-compatible service with the third-party Apache-2.0 model published by OpenMOSS to produce long-form transcripts, timestamps, and speaker labels in one pass.",
"fit": ["Multi-speaker meetings, interviews, podcasts, and calls", "One model output for text, timestamps, and speaker identifiers", "NVIDIA GPU deployments that need a vLLM or SGLang Omni OpenAI-compatible audio transcription endpoint", "The third-party LocalAI / moss-transcribe.cpp GGUF path on CPU or desktop-edge GPUs"],
"not_fit": ["Realtime streaming captions or low-latency endpoint detection", "Deployments that require identical output from FunASR AutoModel and the LocalAI C++ path", "Native Windows deployments without validation on the target hardware", "Projects that require FunASR-owned model weights"],
"selection_reason": "The model jointly emits transcription and speaker labels so users do not need to assemble an external VAD, ASR, and diarization pipeline; FunASR AutoModel provides one structured result across Transformers and vLLM, SGLang Omni provides an independent native upstream service, and LocalAI exposes a separate third-party C++/GGUF edge path.",
"primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins 0.27.1 and the SGLang Omni path pins merge 8458f76a; revalidate real long multi-speaker audio before upgrades. SGLang Omni is not a FunASR AutoModel backend, and its verbose_json keeps the speaker identifier in the [Sxx] prefix of segments[].text. LocalAI / moss-transcribe.cpp is another independent third-party reimplementation.",
"selection_reason": "The model jointly emits transcription and speaker labels so users do not need to assemble an external VAD, ASR, and diarization pipeline; FunASR AutoModel provides one structured result across Transformers, vLLM, and SGLang Omni, while LocalAI exposes a separate third-party C++/GGUF edge path.",
"primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins 0.27.1 and the SGLang Omni path pins 3f819f9c; revalidate real long multi-speaker audio before upgrades. SGLang Omni verbose_json keeps the speaker identifier in the [Sxx] prefix of segments[].text, which the FunASR adapter parses only after validating that the segment is backed by raw_text. LocalAI / moss-transcribe.cpp is another independent third-party reimplementation.",
"status_label": "Community verified",
"operations": ["Pin the model revision, FunASR adapter merge, vLLM 0.27.1, SGLang Omni merge 8458f76a, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "The 1088-clip SGLang Omni single-speaker English benchmark does not cover diarization or timestamp accuracy; validate both independently before production", "For LocalAI, pin the third-party C++ backend and GGUF revision, then validate CPU/GPU backends, quantization accuracy, and the OpenAI response independently", "FunASR AutoModel can map vLLM diarized_json directly into text, timestamp, and sentence_info with spk; the json compatibility path still preserves raw [Sxx]-tagged text in raw_text"],
"operations": ["Pin the model revision, FunASR adapter merge, vLLM 0.27.1, SGLang Omni 3f819f9c, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "The 1088-clip SGLang Omni #914 single-speaker English benchmark does not cover diarization or timestamp accuracy; validate both independently before production", "For LocalAI, pin the third-party C++ backend and GGUF revision, then validate CPU/GPU backends, quantization accuracy, and the OpenAI response independently", "FunASR AutoModel maps both vLLM diarized_json and SGLang Omni verbose_json into text, timestamp, and sentence_info with spk; SGLang segments must be backed by raw_text so an upstream synthesized S01 fallback is not accepted"],
"security": ["Bind workers to a private network and put authentication, TLS, rate limits, and audio size/duration limits at the gateway", "Pin and audit the trust_remote_code model revision instead of executing a floating main revision", "Isolate model caches, upload directories, and generation logs; remove source audio according to retention policy"],
"troubleshooting": ["For CUDA 12, use the SHA256-verified vLLM 0.27.1 cu129 wheel; the SGLang Omni path currently pins its CUDA 13 installation contract and merge 8458f76a", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "vLLM diarized_json returns a separate speaker field; SGLang Omni verbose_json currently keeps the speaker identifier in the [Sxx] prefix of segments[].text"]
"troubleshooting": ["For CUDA 12, use the SHA256-verified vLLM 0.27.1 cu129 wheel; the SGLang Omni path currently pins its CUDA 13 installation contract and 3f819f9c", "If long audio is truncated, raise max_completion_tokens for vLLM or max_new_tokens for SGLang Omni while monitoring memory, latency, and output completeness", "vLLM diarized_json returns a separate speaker field; SGLang Omni verbose_json currently keeps the speaker identifier in the [Sxx] prefix of segments[].text"]
}
}
},
@@ -107,7 +107,7 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
'OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)'
]
assert entry['tested'] == {
'funasr': 'AutoModel diarized_json adapter; third-party model@e8681d68',
'funasr': 'AutoModel vLLM + SGLang adapters; third-party model@e8681d68',
'runtime': 'vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB',
'verified': '2026-08-30',
}
@@ -120,7 +120,7 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
'vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB; FunASR adapter verified'
)
assert runtime_paths['sglang-omni']['tested'] == (
'SGLang Omni merge 8458f76a / single H100 upstream benchmark'
'SGLang Omni 3f819f9c / FunASR adapter contract-tested / #914 H100 upstream benchmark'
)
sglang_commands = '\n'.join(
@@ -128,12 +128,15 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
for group in ('install', 'launch', 'health', 'smoke')
for command in runtime_paths['sglang-omni']['commands'][group]
)
assert 'git checkout 8458f76ab25f5ba9152b05929b40e07618aff2ce' in sglang_commands
assert 'git checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e' in sglang_commands
assert 'sgl-omni serve' in sglang_commands
assert '--model-path .models/moss-transcribe-diarize' in sglang_commands
assert 'response_format=verbose_json' in sglang_commands
assert "payload.get('segments'" in sglang_commands
assert 'backend=\'sglang\'' not in sglang_commands
assert "from funasr import AutoModel" in sglang_commands
assert "backend='sglang'" in sglang_commands
assert "sglang_base_url='http://127.0.0.1:8898/v1'" in sglang_commands
assert 'max_new_tokens=65536' in sglang_commands
install = '\n'.join(entry['commands']['install'])
assert 'vllm[audio]' in install
@@ -188,6 +191,8 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
assert 'diarized_json' in english['operations'][-1]
assert 'diarized_json' in english['troubleshooting'][-1]
assert 'internal segmentation' in english['primary_limitation']
assert 'not a FunASR AutoModel backend' not in english['primary_limitation']
assert 'SGLang Omni' in english['operations'][-1]
assert 'FunASR model' in english['primary_limitation']
assert any('LocalAI' in item and 'GGUF' in item for item in english['fit'])
assert not any(