mirror of
https://github.com/modelscope/FunASR.git
synced 2026-10-12 02:19:18 +00:00
feat(moss): add SGLang AutoModel backend
Signed-off-by: LauraGPT <[email protected]>
This commit is contained in:
@@ -15,7 +15,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm
|
||||
| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |
|
||||
| ONNX/C++ runtime | High-concurrency CPU services or embedded realtime ASR | [ONNX runtime docs](../runtime/onnxruntime/readme.md) | Keep this path when latency/concurrency is already proven; add text post-processing for fixed business terms before moving to GPU LLMs. |
|
||||
| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. |
|
||||
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`) or vLLM (`backend="vllm"`), or serve it independently through native SGLang Omni. SGLang Omni is not an `AutoModel` backend. The model remains published and maintained by OpenMOSS. |
|
||||
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`), vLLM (`backend="vllm"`), or SGLang Omni (`backend="sglang"`). The model remains published and maintained by OpenMOSS. |
|
||||
| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. |
|
||||
| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. |
|
||||
| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. |
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
| Kubernetes API | Cluster service 向け internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP` から開始。公開範囲を広げる前に auth、TLS、network policy、GPU scheduling を追加します。 |
|
||||
| Runtime WebSocket service | Live captions、meeting、call-center stream | [Runtime service docs](../runtime/readme.md) | partial result、endpointing、long-lived audio stream が重要な場合に使います。 |
|
||||
| vLLM acceleration | Fun-ASR-Nano の LLM-based ASR throughput 向上 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput 向け。non-autoregressive Paraformer には適用しません。 |
|
||||
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF(`backend="hf"`)または vLLM(`backend="vllm"`)を選択でき、native SGLang Omni から独立して serve することもできます。SGLang Omni は `AutoModel` backend ではなく、model の公開・保守主体は OpenMOSS のままです。 |
|
||||
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF(`backend="hf"`)、vLLM(`backend="vllm"`)、または SGLang Omni(`backend="sglang"`)を選択できます。model の公開・保守主体は OpenMOSS のままです。 |
|
||||
| MCP server | Claude/Cursor/desktop agent の speech tool | [MCP example](../examples/mcp_server/) | ASR 結果を local tool として Agent に渡したい場合に便利です。 |
|
||||
| Subtitle generator | 長時間 audio/video から SRT/VTT 作成 | [Subtitle example](../examples/subtitle/) | readability が重要な場合は verbose segment と speaker label を使います。 |
|
||||
| Batch ASR script | Archive、meeting、dataset、繰り返し offline run | [Batch example](../examples/batch_asr_improved.py) | production では queue、manifest、retry log を追加してください。 |
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
| Kubernetes API | Cluster service용 internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP`부터 시작합니다. 범위를 넓히기 전에 auth, TLS, network policy, GPU scheduling을 추가하세요. |
|
||||
| Runtime WebSocket service | Live captions, meeting, call-center stream | [Runtime service docs](../runtime/readme.md) | partial result, endpointing, long-lived audio stream이 중요할 때 사용합니다. |
|
||||
| vLLM acceleration | Fun-ASR-Nano의 LLM-based ASR throughput 향상 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput용입니다. non-autoregressive Paraformer에는 적용되지 않습니다. |
|
||||
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`) 또는 vLLM(`backend="vllm"`)을 선택하거나 native SGLang Omni에서 독립적으로 serve할 수 있습니다. SGLang Omni는 `AutoModel` backend가 아니며, model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
|
||||
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`), vLLM(`backend="vllm"`) 또는 SGLang Omni(`backend="sglang"`)를 선택할 수 있습니다. model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
|
||||
| MCP server | Claude/Cursor/desktop agent speech tool | [MCP example](../examples/mcp_server/) | ASR 결과를 local tool로 Agent에 전달할 때 유용합니다. |
|
||||
| Subtitle generator | 긴 audio/video에서 SRT/VTT 생성 | [Subtitle example](../examples/subtitle/) | readability가 중요하면 verbose segment와 speaker label을 사용합니다. |
|
||||
| Batch ASR script | Archive, meeting, dataset, 반복 offline run | [Batch example](../examples/batch_asr_improved.py) | production에서는 queue, manifest, retry log를 추가하세요. |
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |
|
||||
| ONNX/C++ Runtime | 高并发 CPU 服务或嵌入式实时 ASR | [ONNX Runtime 文档](../runtime/onnxruntime/readme.md) | 如果延迟和并发已经验证,不要轻易替换;固定业务词优先做文本后处理。 |
|
||||
| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐;不适用于非自回归 Paraformer。 |
|
||||
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型,已接入 FunASR `AutoModel`;可选本地 HF(`backend="hf"`)或 vLLM(`backend="vllm"`),也可通过原生 SGLang Omni 独立服务。SGLang Omni 不是 `AutoModel` backend。模型仍由 OpenMOSS 发布和维护。 |
|
||||
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型,已接入 FunASR `AutoModel`;可选本地 HF(`backend="hf"`)、vLLM(`backend="vllm"`)或 SGLang Omni(`backend="sglang"`)。模型仍由 OpenMOSS 发布和维护。 |
|
||||
| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 |
|
||||
| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 |
|
||||
| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 |
|
||||
|
||||
@@ -6,8 +6,8 @@ This guide connects the third-party
|
||||
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
|
||||
model to the FunASR deployment ecosystem. The model is published by OpenMOSS
|
||||
under Apache-2.0; it is not a FunASR model. FunASR provides an adapter for its
|
||||
public Transformers and vLLM interfaces while retaining the OpenMOSS model
|
||||
name, license, and upstream revision.
|
||||
public Transformers, vLLM, and SGLang Omni interfaces while retaining the
|
||||
OpenMOSS model name, license, and upstream revision.
|
||||
|
||||
MOSS-Transcribe-Diarize jointly generates transcription, timestamps, and
|
||||
speaker labels such as `[S01]`. An application therefore does not need to
|
||||
@@ -259,6 +259,16 @@ claims as measurements of your host.
|
||||
Follow the pinned upstream SGLang Omni installation guide for CUDA 13, then
|
||||
download the immutable model snapshot and serve the local directory:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/sgl-project/sglang-omni.git
|
||||
git -C sglang-omni checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e
|
||||
```
|
||||
|
||||
This source pin includes the transcription API's `max_new_tokens` forwarding.
|
||||
The original #914 merge predates that request field, so it is not sufficient
|
||||
for the long-audio command below even though its published H100 benchmark
|
||||
remains useful upstream evidence.
|
||||
|
||||
```bash
|
||||
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
|
||||
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
|
||||
@@ -287,6 +297,31 @@ current `verbose_json` contract, the speaker identifier is retained as the
|
||||
Parse and validate that prefix before wiring the response into subtitles,
|
||||
meeting notes, or analytics.
|
||||
|
||||
The FunASR adapter performs that validation and maps the official SGLang
|
||||
segments into the same `sentence_info` contract as the HF and vLLM paths:
|
||||
|
||||
```python
|
||||
from funasr import AutoModel
|
||||
|
||||
model = AutoModel(
|
||||
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
backend="sglang",
|
||||
sglang_base_url="http://127.0.0.1:8898/v1",
|
||||
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
max_new_tokens=65536,
|
||||
disable_update=True,
|
||||
)
|
||||
result = model.generate(input="audio.wav", max_new_tokens=65536)[0]
|
||||
for segment in result["sentence_info"]:
|
||||
print(segment["start"], segment["end"], segment["spk"], segment["text"])
|
||||
```
|
||||
|
||||
Do not pass `vad_model` or `spk_model`: MOSS performs segmentation and
|
||||
anonymous speaker attribution jointly, and external splitting can destroy
|
||||
speaker consistency across long turns. The adapter preserves the upstream
|
||||
tagged transcript in `raw_text`, strips only the validated `[Sxx]` prefix from
|
||||
each normalized segment, and fails closed if SGLang omits that prefix.
|
||||
|
||||
The native runtime was merged in SGLang Omni
|
||||
[#914](https://github.com/sgl-project/sglang-omni/pull/914). Its single-H100
|
||||
Seed-TTS EN benchmark completed 1088/1088 clips with no request failures. WER
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
本文把第三方
|
||||
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
|
||||
模型接入 FunASR 部署生态。模型由 OpenMOSS 以 Apache-2.0 发布,不是 FunASR
|
||||
自有模型。FunASR 只为其公开的 Transformers 与 vLLM 接口提供适配器,并保留
|
||||
OpenMOSS 模型名称、许可证和上游 revision。
|
||||
自有模型。FunASR 为其公开的 Transformers、vLLM 与 SGLang Omni 接口提供适配器,
|
||||
并保留 OpenMOSS 模型名称、许可证和上游 revision。
|
||||
|
||||
MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话人标签,应用侧
|
||||
不必再拼接外部 VAD、ASR 和 diarization 管线。这里描述的是部署形态,不表示模型
|
||||
@@ -231,6 +231,15 @@ LocalAI `master@a7cc5873ef5b7c909fc9ff7d349d51738ba9bb05` 已包含
|
||||
|
||||
按固定上游 SGLang Omni 安装指南准备 CUDA 13 环境,然后下载不可变模型快照并从本地目录启动:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/sgl-project/sglang-omni.git
|
||||
git -C sglang-omni checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e
|
||||
```
|
||||
|
||||
该源码 pin 已把 `max_new_tokens` 传入 transcription 生成请求。最初的 #914
|
||||
merge 早于这个请求字段,因此虽然它的 H100 benchmark 仍可作为上游证据,却不能
|
||||
支撑下方长音频命令。
|
||||
|
||||
```bash
|
||||
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
|
||||
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
|
||||
@@ -257,6 +266,30 @@ curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
|
||||
`verbose_json` 合同把说话人编号保留为 `segments[].text` 的 `[Sxx]` 前缀,
|
||||
并没有单独的 `speaker` 字段。接入字幕、会议纪要或分析系统前,应解析并校验此前缀。
|
||||
|
||||
FunASR 适配器会完成该校验,并把 SGLang 官方 segments 映射为与 HF、vLLM
|
||||
一致的 `sentence_info` 合同:
|
||||
|
||||
```python
|
||||
from funasr import AutoModel
|
||||
|
||||
model = AutoModel(
|
||||
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
backend="sglang",
|
||||
sglang_base_url="http://127.0.0.1:8898/v1",
|
||||
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
max_new_tokens=65536,
|
||||
disable_update=True,
|
||||
)
|
||||
result = model.generate(input="audio.wav", max_new_tokens=65536)[0]
|
||||
for segment in result["sentence_info"]:
|
||||
print(segment["start"], segment["end"], segment["spk"], segment["text"])
|
||||
```
|
||||
|
||||
不要传入 `vad_model` 或 `spk_model`:MOSS 在一次生成中联合完成分段和匿名说话人
|
||||
归属,外部分段会破坏长轮次中的说话人一致性。适配器把上游带标签原文保存在
|
||||
`raw_text`,只从标准化 segment 中移除已经校验的 `[Sxx]` 前缀;如果 SGLang
|
||||
没有返回此前缀,则明确失败,不会伪造说话人身份。
|
||||
|
||||
原生 runtime 已通过 SGLang Omni
|
||||
[#914](https://github.com/sgl-project/sglang-omni/pull/914) 合并。其单张 H100
|
||||
Seed-TTS EN benchmark 完成 1088/1088 条请求且无请求失败。WER 是在移除
|
||||
|
||||
@@ -29,6 +29,7 @@ DEFAULT_PROMPT = (
|
||||
_NUMBER = r"(?:\d+(?:\.\d+)?|\.\d+)"
|
||||
_SEGMENT_START = re.compile(r"\[(%s)\]\[(S\d+)\]" % _NUMBER)
|
||||
_TIMESTAMP = re.compile(r"\[(%s)\]" % _NUMBER)
|
||||
_SGLANG_SPEAKER_PREFIX = re.compile(r"^\s*\[(S\d{2,})\]\s*")
|
||||
|
||||
|
||||
def _parse_transcript(transcript):
|
||||
@@ -188,6 +189,87 @@ def _result_from_diarized_response(key, response):
|
||||
}
|
||||
|
||||
|
||||
def _result_from_sglang_response(key, response):
|
||||
"""Normalize SGLang Omni's MOSS ``verbose_json`` response."""
|
||||
raw_text = response.get("text") if isinstance(response, dict) else None
|
||||
segments = response.get("segments") if isinstance(response, dict) else None
|
||||
if not isinstance(raw_text, str) or not isinstance(segments, list):
|
||||
raise RuntimeError(
|
||||
"SGLang verbose_json response requires text and segments fields"
|
||||
)
|
||||
source_segments = _parse_transcript(raw_text)
|
||||
|
||||
sentence_info = []
|
||||
timestamps = []
|
||||
texts = []
|
||||
previous_start = None
|
||||
for index, segment in enumerate(segments):
|
||||
if not isinstance(segment, dict):
|
||||
raise RuntimeError("SGLang segment %d must be an object" % index)
|
||||
start = segment.get("start")
|
||||
end = segment.get("end")
|
||||
tagged_text = segment.get("text")
|
||||
numeric = (
|
||||
isinstance(start, (int, float))
|
||||
and not isinstance(start, bool)
|
||||
and isinstance(end, (int, float))
|
||||
and not isinstance(end, bool)
|
||||
)
|
||||
if (
|
||||
not numeric
|
||||
or not math.isfinite(start)
|
||||
or not math.isfinite(end)
|
||||
or start < 0
|
||||
or end < start
|
||||
or not isinstance(tagged_text, str)
|
||||
):
|
||||
raise RuntimeError("SGLang segment %d has an invalid contract" % index)
|
||||
if previous_start is not None and start < previous_start:
|
||||
raise RuntimeError("SGLang segment %d has non-monotonic start time" % index)
|
||||
previous_start = start
|
||||
speaker_match = _SGLANG_SPEAKER_PREFIX.match(tagged_text)
|
||||
if speaker_match is None:
|
||||
raise RuntimeError(
|
||||
"SGLang segment %d is missing the required speaker prefix" % index
|
||||
)
|
||||
text = tagged_text[speaker_match.end() :].strip()
|
||||
if not text:
|
||||
raise RuntimeError("SGLang segment %d has an invalid contract" % index)
|
||||
if index >= len(source_segments):
|
||||
raise RuntimeError(
|
||||
"SGLang segment %d is not backed by raw transcript markers" % index
|
||||
)
|
||||
_, _, source_speaker, source_text = source_segments[index]
|
||||
if source_speaker != speaker_match.group(1) or source_text != text:
|
||||
raise RuntimeError(
|
||||
"SGLang segment %d is not backed by raw transcript markers" % index
|
||||
)
|
||||
timestamp = [int(round(start * 1000)), int(round(end * 1000))]
|
||||
timestamps.append(timestamp)
|
||||
texts.append(text)
|
||||
sentence_info.append(
|
||||
{
|
||||
"start": timestamp[0],
|
||||
"end": timestamp[1],
|
||||
"text": text,
|
||||
"sentence": text,
|
||||
"spk": speaker_match.group(1),
|
||||
"timestamp": [timestamp],
|
||||
}
|
||||
)
|
||||
if len(source_segments) != len(segments):
|
||||
raise RuntimeError(
|
||||
"SGLang segments are not backed by raw transcript speaker markers"
|
||||
)
|
||||
return {
|
||||
"key": key,
|
||||
"text": _join_texts(texts),
|
||||
"raw_text": raw_text,
|
||||
"timestamp": timestamps,
|
||||
"sentence_info": sentence_info,
|
||||
}
|
||||
|
||||
|
||||
@tables.register("model_classes", "MOSS-Transcribe-Diarize")
|
||||
@tables.register("model_classes", DEFAULT_MODEL)
|
||||
class MossTranscribeDiarize(nn.Module):
|
||||
@@ -203,8 +285,8 @@ class MossTranscribeDiarize(nn.Module):
|
||||
)
|
||||
|
||||
self.backend = kwargs.get("backend", "hf").lower()
|
||||
if self.backend not in {"hf", "vllm"}:
|
||||
raise ValueError("backend must be 'hf' or 'vllm'")
|
||||
if self.backend not in {"hf", "vllm", "sglang"}:
|
||||
raise ValueError("backend must be 'hf', 'vllm', or 'sglang'")
|
||||
self.model_path = (
|
||||
kwargs.get("model_path") or kwargs.get("model") or DEFAULT_MODEL
|
||||
)
|
||||
@@ -225,12 +307,19 @@ class MossTranscribeDiarize(nn.Module):
|
||||
if self.vllm_response_format not in {"json", "diarized_json"}:
|
||||
raise ValueError("vllm_response_format must be 'json' or 'diarized_json'")
|
||||
self.http_session = kwargs.get("http_session")
|
||||
self.sglang_base_url = kwargs.get("sglang_base_url")
|
||||
self.sglang_model = kwargs.get("sglang_model", DEFAULT_MODEL)
|
||||
self.sglang_api_key = kwargs.get("sglang_api_key", "EMPTY")
|
||||
self.sglang_timeout = float(kwargs.get("sglang_timeout", 600.0))
|
||||
|
||||
self.hf_model = None
|
||||
self.processor = None
|
||||
if self.backend == "vllm":
|
||||
if not self.vllm_base_url:
|
||||
raise ValueError("vllm_base_url is required when backend='vllm'")
|
||||
elif self.backend == "sglang":
|
||||
if not self.sglang_base_url:
|
||||
raise ValueError("sglang_base_url is required when backend='sglang'")
|
||||
else:
|
||||
self._load_hf_backend(kwargs)
|
||||
|
||||
@@ -316,6 +405,9 @@ class MossTranscribeDiarize(nn.Module):
|
||||
results.append(
|
||||
_result_from_transcript(keys[index], response["text"])
|
||||
)
|
||||
elif self.backend == "sglang":
|
||||
response = self._transcribe_sglang(audio, **kwargs)
|
||||
results.append(_result_from_sglang_response(keys[index], response))
|
||||
else:
|
||||
transcript = self._transcribe_hf(audio, **kwargs)
|
||||
results.append(_result_from_transcript(keys[index], transcript))
|
||||
@@ -384,8 +476,9 @@ class MossTranscribeDiarize(nn.Module):
|
||||
generated, skip_special_tokens=True
|
||||
).strip()
|
||||
|
||||
def _transcriptions_url(self):
|
||||
base = self.vllm_base_url.rstrip("/")
|
||||
@staticmethod
|
||||
def _transcriptions_url(base_url):
|
||||
base = base_url.rstrip("/")
|
||||
if base.endswith("/v1/audio/transcriptions"):
|
||||
return base
|
||||
if not base.endswith("/v1"):
|
||||
@@ -415,7 +508,7 @@ class MossTranscribeDiarize(nn.Module):
|
||||
if self.vllm_api_key and self.vllm_api_key != "EMPTY":
|
||||
headers["Authorization"] = "Bearer " + self.vllm_api_key
|
||||
response = session.post(
|
||||
self._transcriptions_url(),
|
||||
self._transcriptions_url(self.vllm_base_url),
|
||||
data=data,
|
||||
files={"file": (filename, payload, content_type)},
|
||||
headers=headers,
|
||||
@@ -433,6 +526,36 @@ class MossTranscribeDiarize(nn.Module):
|
||||
)
|
||||
return result
|
||||
|
||||
def _transcribe_sglang(self, audio, **kwargs):
|
||||
import requests
|
||||
|
||||
session = self.http_session or requests.Session()
|
||||
filename, payload, content_type = self._multipart_audio(audio)
|
||||
data = {
|
||||
"model": self.sglang_model,
|
||||
"response_format": "verbose_json",
|
||||
"temperature": str(kwargs.get("temperature", 0)),
|
||||
"max_new_tokens": str(kwargs.get("max_new_tokens", self.max_new_tokens)),
|
||||
}
|
||||
prompt = kwargs.get("prompt")
|
||||
if prompt:
|
||||
data["prompt"] = prompt
|
||||
headers = {}
|
||||
if self.sglang_api_key and self.sglang_api_key != "EMPTY":
|
||||
headers["Authorization"] = "Bearer " + self.sglang_api_key
|
||||
response = session.post(
|
||||
self._transcriptions_url(self.sglang_base_url),
|
||||
data=data,
|
||||
files={"file": (filename, payload, content_type)},
|
||||
headers=headers,
|
||||
timeout=self.sglang_timeout,
|
||||
)
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
if not isinstance(result, dict):
|
||||
raise RuntimeError("SGLang transcription response must be a JSON object")
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _multipart_audio(audio):
|
||||
if isinstance(audio, np.ndarray):
|
||||
|
||||
@@ -26,6 +26,7 @@ def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) ->
|
||||
"OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
"e8681d68e7042738ffca8ac8212bc8fcb1131ab8",
|
||||
"6e448d0ea9bf3d88d898b65449ca6dc2aec170ac",
|
||||
"3f819f9cdae3d4eeec22f73306c9067a1ec2542e",
|
||||
"bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b",
|
||||
"vllm[audio]",
|
||||
"vLLM 0.27.1",
|
||||
@@ -38,6 +39,9 @@ def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) ->
|
||||
"response_format=diarized_json",
|
||||
"response_format=verbose_json",
|
||||
'vllm_response_format="diarized_json"',
|
||||
'backend="sglang"',
|
||||
'sglang_base_url="http://127.0.0.1:8898/v1"',
|
||||
"max_new_tokens=65536",
|
||||
"[S01]",
|
||||
"Apache-2.0",
|
||||
"sentence_info",
|
||||
@@ -66,6 +70,7 @@ def test_all_deployment_matrices_link_moss_guide(matrix: Path) -> None:
|
||||
assert "moss_transcribe_diarize.md" in moss_row
|
||||
assert '`backend="hf"`' in moss_row
|
||||
assert '`backend="vllm"`' in moss_row
|
||||
assert '`backend="sglang"`' in moss_row
|
||||
for stale_claim in (
|
||||
"not a FunASR-owned model or `AutoModel` backend",
|
||||
"不是 FunASR 自有模型或 `AutoModel` 后端",
|
||||
|
||||
@@ -264,5 +264,171 @@ class MossVllmBackendTest(unittest.TestCase):
|
||||
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
|
||||
|
||||
|
||||
class MossSglangBackendTest(unittest.TestCase):
|
||||
def test_auto_model_sglang_path_bypasses_weight_download(self):
|
||||
from funasr.auto.auto_model import AutoModel
|
||||
|
||||
with patch(
|
||||
"funasr.auto.auto_model.download_model",
|
||||
side_effect=AssertionError(
|
||||
"SGLang client mode must not download model weights"
|
||||
),
|
||||
):
|
||||
auto_model = AutoModel(
|
||||
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
backend="sglang",
|
||||
sglang_base_url="http://sglang.test:8000/v1",
|
||||
device="cpu",
|
||||
disable_update=True,
|
||||
)
|
||||
|
||||
self.assertIsInstance(auto_model.model, MossTranscribeDiarize)
|
||||
|
||||
def test_normalizes_official_sglang_verbose_json_response(self):
|
||||
session = MagicMock()
|
||||
response = MagicMock()
|
||||
response.json.return_value = {
|
||||
"text": "[0.25][S01]hello[1.5][2.0][S02]again[3.5]",
|
||||
"segments": [
|
||||
{"start": 0.25, "end": 1.5, "text": "[S01]hello"},
|
||||
{"start": 2.0, "end": 3.5, "text": "[S02] again"},
|
||||
],
|
||||
}
|
||||
session.post.return_value = response
|
||||
|
||||
model = MossTranscribeDiarize(
|
||||
backend="sglang",
|
||||
sglang_base_url="http://sglang.test:8000/v1",
|
||||
sglang_model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
sglang_api_key="secret",
|
||||
http_session=session,
|
||||
)
|
||||
results, _ = model.inference(
|
||||
[np.zeros(1600, dtype=np.float32)],
|
||||
key=["meeting"],
|
||||
prompt="transcribe and diarize",
|
||||
max_new_tokens=8192,
|
||||
)
|
||||
|
||||
request = session.post.call_args
|
||||
self.assertEqual(
|
||||
request.args[0],
|
||||
"http://sglang.test:8000/v1/audio/transcriptions",
|
||||
)
|
||||
self.assertEqual(
|
||||
request.kwargs["data"],
|
||||
{
|
||||
"model": "OpenMOSS-Team/MOSS-Transcribe-Diarize",
|
||||
"response_format": "verbose_json",
|
||||
"temperature": "0",
|
||||
"prompt": "transcribe and diarize",
|
||||
"max_new_tokens": "8192",
|
||||
},
|
||||
)
|
||||
self.assertEqual(request.kwargs["headers"], {"Authorization": "Bearer secret"})
|
||||
self.assertEqual(results[0]["text"], "hello again")
|
||||
self.assertEqual(
|
||||
results[0]["raw_text"],
|
||||
"[0.25][S01]hello[1.5][2.0][S02]again[3.5]",
|
||||
)
|
||||
self.assertEqual(results[0]["timestamp"], [[250, 1500], [2000, 3500]])
|
||||
self.assertEqual(
|
||||
[(item["spk"], item["text"]) for item in results[0]["sentence_info"]],
|
||||
[("S01", "hello"), ("S02", "again")],
|
||||
)
|
||||
|
||||
def test_rejects_sglang_segment_without_speaker_prefix(self):
|
||||
session = MagicMock()
|
||||
response = MagicMock()
|
||||
response.json.return_value = {
|
||||
"text": "hello",
|
||||
"segments": [{"start": 0.0, "end": 1.0, "text": "hello"}],
|
||||
}
|
||||
session.post.return_value = response
|
||||
model = MossTranscribeDiarize(
|
||||
backend="sglang",
|
||||
sglang_base_url="http://sglang.test:8000/v1",
|
||||
http_session=session,
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(RuntimeError, "segment 0.*speaker prefix"):
|
||||
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
|
||||
|
||||
def test_rejects_sglang_fallback_speaker_not_backed_by_raw_transcript(self):
|
||||
session = MagicMock()
|
||||
response = MagicMock()
|
||||
response.json.return_value = {
|
||||
"text": "hello",
|
||||
"segments": [{"start": 0.0, "end": 1.0, "text": "[S01]hello"}],
|
||||
}
|
||||
session.post.return_value = response
|
||||
model = MossTranscribeDiarize(
|
||||
backend="sglang",
|
||||
sglang_base_url="http://sglang.test:8000/v1",
|
||||
http_session=session,
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(RuntimeError, "not backed by raw transcript"):
|
||||
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
|
||||
|
||||
def test_rejects_one_digit_sglang_speaker_label(self):
|
||||
session = MagicMock()
|
||||
response = MagicMock()
|
||||
response.json.return_value = {
|
||||
"text": "[0][S1]hello[1]",
|
||||
"segments": [{"start": 0.0, "end": 1.0, "text": "[S1]hello"}],
|
||||
}
|
||||
session.post.return_value = response
|
||||
model = MossTranscribeDiarize(
|
||||
backend="sglang",
|
||||
sglang_base_url="http://sglang.test:8000/v1",
|
||||
http_session=session,
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(RuntimeError, "segment 0.*speaker prefix"):
|
||||
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
|
||||
|
||||
def test_rejects_sglang_segments_with_decreasing_start_times(self):
|
||||
session = MagicMock()
|
||||
response = MagicMock()
|
||||
response.json.return_value = {
|
||||
"text": "[2][S01]late[3][1][S02]early[1.5]",
|
||||
"segments": [
|
||||
{"start": 2.0, "end": 3.0, "text": "[S01]late"},
|
||||
{"start": 1.0, "end": 1.5, "text": "[S02]early"},
|
||||
],
|
||||
}
|
||||
session.post.return_value = response
|
||||
model = MossTranscribeDiarize(
|
||||
backend="sglang",
|
||||
sglang_base_url="http://sglang.test:8000/v1",
|
||||
http_session=session,
|
||||
)
|
||||
|
||||
with self.assertRaisesRegex(RuntimeError, "segment 1.*non-monotonic"):
|
||||
model.inference([np.zeros(1600, dtype=np.float32)], key=["meeting"])
|
||||
|
||||
def test_accepts_sglang_timestamp_clamp_when_speaker_and_text_match_raw(self):
|
||||
session = MagicMock()
|
||||
response = MagicMock()
|
||||
response.json.return_value = {
|
||||
"text": "[0][S01]hello[99]",
|
||||
"segments": [{"start": 0.0, "end": 10.0, "text": "[S01]hello"}],
|
||||
}
|
||||
session.post.return_value = response
|
||||
model = MossTranscribeDiarize(
|
||||
backend="sglang",
|
||||
sglang_base_url="http://sglang.test:8000/v1",
|
||||
http_session=session,
|
||||
)
|
||||
|
||||
results, _ = model.inference(
|
||||
[np.zeros(1600, dtype=np.float32)], key=["meeting"]
|
||||
)
|
||||
|
||||
self.assertEqual(results[0]["timestamp"], [[0, 10000]])
|
||||
self.assertEqual(results[0]["sentence_info"][0]["spk"], "S01")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
"models": ["OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)"],
|
||||
"operating_systems": ["Linux"],
|
||||
"interfaces": ["OpenAI-compatible HTTP", "vLLM", "SGLang Omni", "Transformers", "LocalAI / moss-transcribe.cpp"],
|
||||
"tested": {"funasr": "AutoModel diarized_json adapter; third-party model@e8681d68", "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB", "verified": "2026-08-30"},
|
||||
"tested": {"funasr": "AutoModel vLLM + SGLang adapters; third-party model@e8681d68", "runtime": "vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB", "verified": "2026-08-30"},
|
||||
"commands": {
|
||||
"install": [
|
||||
"uv venv --python 3.12 .venv-moss && curl -fL https://github.com/vllm-project/vllm/releases/download/v0.27.1/vllm-0.27.1%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl -o vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl && echo 'bf0d52faa2a51e7a01c6856a7a8a2d1307fd0ff711415d34168a67ffac0fa47b vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl' | sha256sum -c - && uv pip install --python .venv-moss/bin/python --torch-backend=auto \"vllm[audio] @ file://$PWD/vllm-0.27.1+cu129-cp38-abi3-manylinux_2_28_x86_64.whl\"",
|
||||
@@ -61,14 +61,14 @@
|
||||
},
|
||||
{
|
||||
"id": "sglang-omni",
|
||||
"tested": "SGLang Omni merge 8458f76a / single H100 upstream benchmark",
|
||||
"tested": "SGLang Omni 3f819f9c / FunASR adapter contract-tested / #914 H100 upstream benchmark",
|
||||
"translations": {
|
||||
"zh": {"name": "SGLang Omni 原生服务", "summary": "使用已合并的原生 MOSS pipeline 和 OpenAI-compatible verbose_json 接口;它是独立上游 runtime,不是 FunASR AutoModel backend。"},
|
||||
"en": {"name": "Native SGLang Omni serving", "summary": "Use the merged native MOSS pipeline and OpenAI-compatible verbose_json endpoint. This is an independent upstream runtime, not a FunASR AutoModel backend."}
|
||||
"zh": {"name": "SGLang Omni + FunASR AutoModel", "summary": "使用已合并的原生 MOSS pipeline 和 OpenAI-compatible verbose_json 接口,并通过 FunASR AutoModel 校验 [Sxx] 前缀、统一为 sentence_info。"},
|
||||
"en": {"name": "SGLang Omni + FunASR AutoModel", "summary": "Use the merged native MOSS pipeline and OpenAI-compatible verbose_json endpoint, then validate [Sxx] prefixes and normalize them into sentence_info through FunASR AutoModel."}
|
||||
},
|
||||
"commands": {
|
||||
"install": [
|
||||
"git clone https://github.com/sgl-project/sglang-omni.git && cd sglang-omni && git checkout 8458f76ab25f5ba9152b05929b40e07618aff2ce && uv venv .venv -p 3.12 && uv pip install --python .venv/bin/python -v -e .",
|
||||
"git clone https://github.com/sgl-project/sglang-omni.git && cd sglang-omni && git checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e && uv venv .venv -p 3.12 && uv pip install --python .venv/bin/python -v -e .",
|
||||
"HF_HUB_ENABLE_HF_TRANSFER=1 hf download OpenMOSS-Team/MOSS-Transcribe-Diarize --revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 --local-dir .models/moss-transcribe-diarize",
|
||||
"curl -fsSL https://raw.githubusercontent.com/modelscope/FunASR/8d65a38a8f4f3b5301be72905096219dde443f73/runtime/llama.cpp/tests/sample.wav -o moss-sample.wav && echo 'ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e moss-sample.wav' | sha256sum -c -"
|
||||
],
|
||||
@@ -81,7 +81,8 @@
|
||||
],
|
||||
"smoke": [
|
||||
"curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions -F [email protected] -F model=OpenMOSS-Team/MOSS-Transcribe-Diarize -F response_format=verbose_json | tee /tmp/moss-sglang-transcription.json",
|
||||
"python - <<'PY'\nimport json\nimport re\n\nwith open('/tmp/moss-sglang-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\nsegments = payload.get('segments', [])\nassert payload.get('text', '').strip() and segments, payload\nassert all(item.get('start') <= item.get('end') and re.match(r'^\\[S\\d{2,}\\]', item.get('text', '')) for item in segments), payload\nprint(payload['text'])\nprint([(item['start'], item['end'], item['text'][:5]) for item in segments])\nPY"
|
||||
"python - <<'PY'\nimport json\nimport re\n\nwith open('/tmp/moss-sglang-transcription.json', encoding='utf-8') as stream:\n payload = json.load(stream)\nsegments = payload.get('segments', [])\nassert payload.get('text', '').strip() and segments, payload\nassert all(item.get('start') <= item.get('end') and re.match(r'^\\[S\\d{2,}\\]', item.get('text', '')) for item in segments), payload\nprint(payload['text'])\nprint([(item['start'], item['end'], item['text'][:5]) for item in segments])\nPY",
|
||||
"python - <<'PY'\nfrom funasr import AutoModel\n\nmodel = AutoModel(model='OpenMOSS-Team/MOSS-Transcribe-Diarize', backend='sglang', sglang_base_url='http://127.0.0.1:8898/v1', sglang_model='OpenMOSS-Team/MOSS-Transcribe-Diarize', max_new_tokens=65536, disable_update=True)\nresult = model.generate('moss-sample.wav', max_new_tokens=65536)[0]\nassert result['raw_text'] and result['sentence_info'], result\nprint(result['text'])\nprint(result['sentence_info'])\nPY"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -134,24 +135,24 @@
|
||||
"summary": "通过 FunASR AutoModel 或 OpenAI 兼容服务接入 OpenMOSS 发布的第三方 Apache-2.0 模型,一次生成长音频转写、时间戳和说话人标签。",
|
||||
"fit": ["会议、访谈、播客和通话的多人长音频", "希望通过同一个模型输出文本、时间戳与说话人编号", "已有 NVIDIA GPU,并需要 vLLM 或 SGLang Omni 的 OpenAI 兼容音频转写接口", "通过 LocalAI / moss-transcribe.cpp 在 CPU 或桌面边缘 GPU 上运行第三方 GGUF 实现"],
|
||||
"not_fit": ["实时流式字幕或低延迟端点检测", "要求 FunASR AutoModel 与 LocalAI C++ 路径输出完全相同结果的部署", "未经目标硬件复测的原生 Windows 部署", "要求 FunASR 自有模型权重的项目"],
|
||||
"selection_reason": "模型端到端联合生成转写与说话人标签,用户不必再拼接外部 VAD、ASR 和 diarization 管线;FunASR AutoModel 为 Transformers 与 vLLM 提供统一的结构化结果,SGLang Omni 提供独立的原生上游服务,LocalAI 则提供独立的第三方 C++/GGUF 边缘路径。",
|
||||
"primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。vLLM 路径固定 0.27.1,SGLang Omni 路径固定 merge 8458f76a;升级前必须重跑真实多人长音频。SGLang Omni 不是 FunASR AutoModel backend,其 verbose_json 把说话人编号保留在 segments[].text 的 [Sxx] 前缀。LocalAI / moss-transcribe.cpp 也是独立第三方重写。",
|
||||
"selection_reason": "模型端到端联合生成转写与说话人标签,用户不必再拼接外部 VAD、ASR 和 diarization 管线;FunASR AutoModel 为 Transformers、vLLM 与 SGLang Omni 提供统一的结构化结果,LocalAI 则提供独立的第三方 C++/GGUF 边缘路径。",
|
||||
"primary_limitation": "这是 OpenMOSS 的第三方模型,不是 FunASR 模型;“无需外部 VAD”不表示模型内部没有分块或分段。vLLM 路径固定 0.27.1,SGLang Omni 路径固定 3f819f9c;升级前必须重跑真实多人长音频。SGLang Omni 的 verbose_json 把说话人编号保留在 segments[].text 的 [Sxx] 前缀,FunASR adapter 会校验该分段能够回溯到 raw_text 后再解析。LocalAI / moss-transcribe.cpp 也是独立第三方重写。",
|
||||
"status_label": "社区验证",
|
||||
"operations": ["固定模型 revision、FunASR adapter merge、vLLM 0.27.1、SGLang Omni merge 8458f76a、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "SGLang Omni 的 1088 条单说话人英文 benchmark 不覆盖 diarization 或 timestamp 准确率,生产前必须单独验证", "LocalAI 路径固定第三方 C++ backend 与 GGUF revision,并分别验证 CPU/GPU backend、量化精度和 OpenAI 接口响应", "FunASR AutoModel 可用 vLLM diarized_json 直接统一 text、timestamp 和带 spk 的 sentence_info;json 兼容路径仍在 raw_text 保留原始 [Sxx] 标记文本"],
|
||||
"operations": ["固定模型 revision、FunASR adapter merge、vLLM 0.27.1、SGLang Omni 3f819f9c、CUDA/Torch 与 trust_remote_code 审计结果", "用真实会议验证说话人一致性、重叠语音、长静音、时间戳和最大生成长度", "SGLang Omni #914 的 1088 条单说话人英文 benchmark 不覆盖 diarization 或 timestamp 准确率,生产前必须单独验证", "LocalAI 路径固定第三方 C++ backend 与 GGUF revision,并分别验证 CPU/GPU backend、量化精度和 OpenAI 接口响应", "FunASR AutoModel 可把 vLLM diarized_json 与 SGLang Omni verbose_json 统一为 text、timestamp 和带 spk 的 sentence_info;SGLang 分段必须能回溯到 raw_text,避免接受上游合成的 S01 fallback"],
|
||||
"security": ["把服务绑定内网,由网关完成认证、TLS、限流及音频大小/时长限制", "固定并审计 trust_remote_code 对应的模型 revision,不执行浮动 main 代码", "隔离模型缓存、上传临时目录和生成日志,按数据保留策略清理原始音频"],
|
||||
"troubleshooting": ["CUDA 12 使用已校验 SHA256 的 vLLM 0.27.1 cu129 wheel;SGLang Omni 当前固定 CUDA 13 安装契约和 merge 8458f76a", "长音频被截断时提高 max_completion_tokens,并同时监控显存、延迟和输出完整性", "vLLM diarized_json 提供独立 speaker 字段;SGLang Omni verbose_json 当前把说话人编号放在 segments[].text 的 [Sxx] 前缀"]
|
||||
"troubleshooting": ["CUDA 12 使用已校验 SHA256 的 vLLM 0.27.1 cu129 wheel;SGLang Omni 当前固定 CUDA 13 安装契约和 3f819f9c", "长音频被截断时,vLLM 提高 max_completion_tokens,SGLang Omni 提高 max_new_tokens,并同时监控显存、延迟和输出完整性", "vLLM diarized_json 提供独立 speaker 字段;SGLang Omni verbose_json 当前把说话人编号放在 segments[].text 的 [Sxx] 前缀"]
|
||||
},
|
||||
"en": {
|
||||
"name": "MOSS unified transcription and diarization",
|
||||
"summary": "Use FunASR AutoModel or an OpenAI-compatible service with the third-party Apache-2.0 model published by OpenMOSS to produce long-form transcripts, timestamps, and speaker labels in one pass.",
|
||||
"fit": ["Multi-speaker meetings, interviews, podcasts, and calls", "One model output for text, timestamps, and speaker identifiers", "NVIDIA GPU deployments that need a vLLM or SGLang Omni OpenAI-compatible audio transcription endpoint", "The third-party LocalAI / moss-transcribe.cpp GGUF path on CPU or desktop-edge GPUs"],
|
||||
"not_fit": ["Realtime streaming captions or low-latency endpoint detection", "Deployments that require identical output from FunASR AutoModel and the LocalAI C++ path", "Native Windows deployments without validation on the target hardware", "Projects that require FunASR-owned model weights"],
|
||||
"selection_reason": "The model jointly emits transcription and speaker labels so users do not need to assemble an external VAD, ASR, and diarization pipeline; FunASR AutoModel provides one structured result across Transformers and vLLM, SGLang Omni provides an independent native upstream service, and LocalAI exposes a separate third-party C++/GGUF edge path.",
|
||||
"primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins 0.27.1 and the SGLang Omni path pins merge 8458f76a; revalidate real long multi-speaker audio before upgrades. SGLang Omni is not a FunASR AutoModel backend, and its verbose_json keeps the speaker identifier in the [Sxx] prefix of segments[].text. LocalAI / moss-transcribe.cpp is another independent third-party reimplementation.",
|
||||
"selection_reason": "The model jointly emits transcription and speaker labels so users do not need to assemble an external VAD, ASR, and diarization pipeline; FunASR AutoModel provides one structured result across Transformers, vLLM, and SGLang Omni, while LocalAI exposes a separate third-party C++/GGUF edge path.",
|
||||
"primary_limitation": "This is an OpenMOSS third-party model, not a FunASR model; no external VAD requirement does not imply an absence of internal segmentation. The vLLM path pins 0.27.1 and the SGLang Omni path pins 3f819f9c; revalidate real long multi-speaker audio before upgrades. SGLang Omni verbose_json keeps the speaker identifier in the [Sxx] prefix of segments[].text, which the FunASR adapter parses only after validating that the segment is backed by raw_text. LocalAI / moss-transcribe.cpp is another independent third-party reimplementation.",
|
||||
"status_label": "Community verified",
|
||||
"operations": ["Pin the model revision, FunASR adapter merge, vLLM 0.27.1, SGLang Omni merge 8458f76a, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "The 1088-clip SGLang Omni single-speaker English benchmark does not cover diarization or timestamp accuracy; validate both independently before production", "For LocalAI, pin the third-party C++ backend and GGUF revision, then validate CPU/GPU backends, quantization accuracy, and the OpenAI response independently", "FunASR AutoModel can map vLLM diarized_json directly into text, timestamp, and sentence_info with spk; the json compatibility path still preserves raw [Sxx]-tagged text in raw_text"],
|
||||
"operations": ["Pin the model revision, FunASR adapter merge, vLLM 0.27.1, SGLang Omni 3f819f9c, CUDA/Torch stack, and trust_remote_code audit", "Validate speaker consistency, overlap, long silence, timestamps, and generation limits on real meetings", "The 1088-clip SGLang Omni #914 single-speaker English benchmark does not cover diarization or timestamp accuracy; validate both independently before production", "For LocalAI, pin the third-party C++ backend and GGUF revision, then validate CPU/GPU backends, quantization accuracy, and the OpenAI response independently", "FunASR AutoModel maps both vLLM diarized_json and SGLang Omni verbose_json into text, timestamp, and sentence_info with spk; SGLang segments must be backed by raw_text so an upstream synthesized S01 fallback is not accepted"],
|
||||
"security": ["Bind workers to a private network and put authentication, TLS, rate limits, and audio size/duration limits at the gateway", "Pin and audit the trust_remote_code model revision instead of executing a floating main revision", "Isolate model caches, upload directories, and generation logs; remove source audio according to retention policy"],
|
||||
"troubleshooting": ["For CUDA 12, use the SHA256-verified vLLM 0.27.1 cu129 wheel; the SGLang Omni path currently pins its CUDA 13 installation contract and merge 8458f76a", "If long audio is truncated, raise max_completion_tokens while monitoring memory, latency, and output completeness", "vLLM diarized_json returns a separate speaker field; SGLang Omni verbose_json currently keeps the speaker identifier in the [Sxx] prefix of segments[].text"]
|
||||
"troubleshooting": ["For CUDA 12, use the SHA256-verified vLLM 0.27.1 cu129 wheel; the SGLang Omni path currently pins its CUDA 13 installation contract and 3f819f9c", "If long audio is truncated, raise max_completion_tokens for vLLM or max_new_tokens for SGLang Omni while monitoring memory, latency, and output completeness", "vLLM diarized_json returns a separate speaker field; SGLang Omni verbose_json currently keeps the speaker identifier in the [Sxx] prefix of segments[].text"]
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
@@ -107,7 +107,7 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
|
||||
'OpenMOSS-Team/MOSS-Transcribe-Diarize (third-party Apache-2.0 model)'
|
||||
]
|
||||
assert entry['tested'] == {
|
||||
'funasr': 'AutoModel diarized_json adapter; third-party model@e8681d68',
|
||||
'funasr': 'AutoModel vLLM + SGLang adapters; third-party model@e8681d68',
|
||||
'runtime': 'vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB',
|
||||
'verified': '2026-08-30',
|
||||
}
|
||||
@@ -120,7 +120,7 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
|
||||
'vLLM 0.27.1 / Torch 2.13.0+cu129 / H100 80GB; FunASR adapter verified'
|
||||
)
|
||||
assert runtime_paths['sglang-omni']['tested'] == (
|
||||
'SGLang Omni merge 8458f76a / single H100 upstream benchmark'
|
||||
'SGLang Omni 3f819f9c / FunASR adapter contract-tested / #914 H100 upstream benchmark'
|
||||
)
|
||||
|
||||
sglang_commands = '\n'.join(
|
||||
@@ -128,12 +128,15 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
|
||||
for group in ('install', 'launch', 'health', 'smoke')
|
||||
for command in runtime_paths['sglang-omni']['commands'][group]
|
||||
)
|
||||
assert 'git checkout 8458f76ab25f5ba9152b05929b40e07618aff2ce' in sglang_commands
|
||||
assert 'git checkout 3f819f9cdae3d4eeec22f73306c9067a1ec2542e' in sglang_commands
|
||||
assert 'sgl-omni serve' in sglang_commands
|
||||
assert '--model-path .models/moss-transcribe-diarize' in sglang_commands
|
||||
assert 'response_format=verbose_json' in sglang_commands
|
||||
assert "payload.get('segments'" in sglang_commands
|
||||
assert 'backend=\'sglang\'' not in sglang_commands
|
||||
assert "from funasr import AutoModel" in sglang_commands
|
||||
assert "backend='sglang'" in sglang_commands
|
||||
assert "sglang_base_url='http://127.0.0.1:8898/v1'" in sglang_commands
|
||||
assert 'max_new_tokens=65536' in sglang_commands
|
||||
|
||||
install = '\n'.join(entry['commands']['install'])
|
||||
assert 'vllm[audio]' in install
|
||||
@@ -188,6 +191,8 @@ def test_moss_transcribe_diarize_contract_tracks_third_party_upstream(valid_regi
|
||||
assert 'diarized_json' in english['operations'][-1]
|
||||
assert 'diarized_json' in english['troubleshooting'][-1]
|
||||
assert 'internal segmentation' in english['primary_limitation']
|
||||
assert 'not a FunASR AutoModel backend' not in english['primary_limitation']
|
||||
assert 'SGLang Omni' in english['operations'][-1]
|
||||
assert 'FunASR model' in english['primary_limitation']
|
||||
assert any('LocalAI' in item and 'GGUF' in item for item in english['fit'])
|
||||
assert not any(
|
||||
|
||||
Reference in New Issue
Block a user