diff --git a/docs/vllm_guide.md b/docs/vllm_guide.md index 9ca4c825b..056798695 100644 --- a/docs/vllm_guide.md +++ b/docs/vllm_guide.md @@ -549,7 +549,7 @@ Client (microphone / audio stream) serve_realtime_ws.py - Audio arrives frame by frame; processing starts immediately - Natural sentence segmentation based on VAD endpoints - Confirmed segment text is locked and never changes; partial text updates in real time -- Streaming speaker assignment + global re-clustering on STOP +- Optional streaming speaker assignment (`--enable-spk`) + global re-clustering on STOP - First-word latency ~480 ms ### 6.2 Starting the Service @@ -559,6 +559,16 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/ --port 10095 --language 中文 --hotword-file hotword_list ``` +For multi-client or long continuous-speech workloads, start by bounding partial previews and lowering the refresh rate: + +```bash +CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py \ + --port 10095 --language 中文 \ + --partial-window-sec 8 --decode-interval 0.8 +``` + +Speaker diarization is disabled by default; add `--enable-spk` only when the `spk` field is required. + ### 6.3 WebSocket Protocol **Connection**: `ws://host:10095` @@ -571,18 +581,18 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/ | Hotwords | `"HOTWORDS:word1,word2"` | Optional | | Language | `"LANGUAGE:中文"` | Optional | | Audio | `binary` | PCM16 16 kHz mono | -| End | `"STOP"` | Final decode + SPK re-clustering | +| End | `"STOP"` | Final decode; SPK re-clustering only when `--enable-spk` is enabled | **Server → Client**: ```json {"event": "started"} -{"sentences": [{"text":"你好","start":300,"end":1200,"spk":0}], "partial": "世界", "is_final": false} +{"sentences": [{"text":"你好","start":300,"end":1200}], "partial": "世界", "is_final": false} {"sentences": [...], "is_final": true} {"event": "stopped"} ``` -**Fields**: `sentences[]` = locked segments, `partial` = text being spoken (may change), `is_final` = true after STOP. +**Fields**: `sentences[]` = locked segments, `partial` = text being spoken (may change), `is_final` = true after STOP. When `--enable-spk` is enabled, `sentences[]` also includes `spk`. **Sequence diagram**: ``` @@ -651,16 +661,15 @@ Because each refresh re-encodes from the sentence start, the longer a sentence, **Usage guidance** - Normal conversational speech has natural pauses, so VAD splits it into relatively short utterances and each partial's cost is naturally bounded — **usually nothing to worry about**. -- Only **very long, pauseless continuous speech** (e.g. reading aloud) makes a single utterance keep growing and the partial preview progressively slower — this is an inherent property of the full-context encoder, **not a configuration issue**. If your scenario has such audio and is sensitive to preview latency, you can bound the partial encoding window in code (e.g. encode only the last few seconds); this is a **code change, not a config option**, and it does not affect locked sentences (which still run on the full audio, with identical final transcripts). +- Only **very long, pauseless continuous speech** (e.g. reading aloud) makes a single utterance keep growing and the partial preview progressively slower. `serve_realtime_ws.py` bounds provisional previews with `--partial-window-sec 15` by default; for multi-client or continuous-monologue load tests, try `8-10` and raise `--decode-interval` to `0.8-1.0`. This only affects provisional `partial`; VAD-locked sentences and STOP final output still run on the full audio. -### 6.6 Cost of speaker diarization (SPK) and how to disable it +### 6.6 Cost of speaker diarization (SPK) and how to enable it -At startup, `serve_realtime_ws.py` **loads the SPK model by default** (hardcoded to `iic/speech_eres2netv2_sv_zh-cn_16k-common`) and runs speaker assignment for each VAD-completed sentence during streaming. Note: +`serve_realtime_ws.py` **does not load** the SPK model by default. It loads `--spk-model` (default `iic/speech_eres2netv2_sv_zh-cn_16k-common`) only when started with `--enable-spk`, then runs speaker assignment for each VAD-completed sentence during streaming. Note: - **SPK is of limited effectiveness on Fun-ASR-Nano** (see #2944); most real-time ASR scenarios do not need speaker separation. - **Streaming SPK is expensive and grows with the session**: each sentence re-clusters **all historical embeddings** (**O(N²)**, more expensive per sentence as the session grows) and **synchronously blocks the event loop**; the session also **re-clusters everything again** at the end, so the per-sentence clustering during streaming is overwritten by the final result — redundant as far as the final output is concerned. This is especially pronounced under long sessions + high concurrency. -- **Disabling currently requires a code change**: `spk_model` is loaded by default and hardcoded in `serve_realtime_ws.py`, with no flag. -- **Recommendation**: don't load `spk_model` when speaker separation isn't needed; if diarization is required, take labels from the final result, or use a Paraformer + cam++ pipeline. +- **Recommendation**: keep the default off for multi-client live ASR; if diarization is required, add `--enable-spk` and treat the final STOP-time labels as authoritative. ### 6.7 Production concurrency and multi-process deployment diff --git a/docs/vllm_guide_zh.md b/docs/vllm_guide_zh.md index 32cc640f9..a502df415 100644 --- a/docs/vllm_guide_zh.md +++ b/docs/vllm_guide_zh.md @@ -553,7 +553,7 @@ asyncio.run(offline_ws("audio.wav")) - 音频逐帧到达，边收边处理 - 基于 VAD 端点自然分句 - 确认段文字锁定不变，partial 实时更新 -- 流式说话人分配 + STOP 时全局重聚类 +- 可选流式说话人分配（`--enable-spk`）+ STOP 时全局重聚类 - 首字延迟 ~480ms ### 6.2 启动服务 @@ -563,6 +563,16 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/ --port 10095 --language 中文 --hotword-file 热词列表 ``` +多客户端或长时间连续语音场景，建议先限制 partial 预览窗口并适当降低刷新频率： + +```bash +CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py \ + --port 10095 --language 中文 \ + --partial-window-sec 8 --decode-interval 0.8 +``` + +说话人分离默认关闭；只有确实需要 `spk` 字段时再加 `--enable-spk`。 + ### 6.3 WebSocket 协议 **连接**：`ws://host:10095` @@ -575,18 +585,18 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/ | 热词 | `"HOTWORDS:词1,词2"` | 可选 | | 语种 | `"LANGUAGE:中文"` | 可选 | | 音频 | `binary` | PCM16 16kHz mono | -| 结束 | `"STOP"` | 最终解码 + SPK 重聚类 | +| 结束 | `"STOP"` | 最终解码；启用 `--enable-spk` 时会做 SPK 重聚类 | **服务端 → 客户端**： ```json {"event": "started"} -{"sentences": [{"text":"你好","start":300,"end":1200,"spk":0}], "partial": "世界", "is_final": false} +{"sentences": [{"text":"你好","start":300,"end":1200}], "partial": "世界", "is_final": false} {"sentences": [...], "is_final": true} {"event": "stopped"} ``` -**字段**：`sentences[]` = 已锁定，`partial` = 正在说（会变），`is_final` = STOP 后为 true。 +**字段**：`sentences[]` = 已锁定，`partial` = 正在说（会变），`is_final` = STOP 后为 true。启用 `--enable-spk` 后，`sentences[]` 会包含 `spk`。 **时序**： ``` @@ -655,16 +665,15 @@ Fun-ASR-Nano 的声学编码器（SenseVoice）是**全上下文、非流式** **使用建议** - 正常对话语音有自然停顿，VAD 会把它切成一句句较短的语音，每句 partial 的开销自然受限，**通常无需关注**。 -- 只有**超长、不停顿的连续语音**（如长篇朗读）会让单句不断变长、partial 预览逐渐变慢——这是全上下文编码器的固有特性，**不是配置问题**。若你的场景确有此类音频且对预览延迟敏感，可在代码侧限制 partial 的编码窗口（如只编最近若干秒）；这属于**代码改动、非配置项**，且不影响锁定句（仍走完整音频、最终转写不变）。 +- 只有**超长、不停顿的连续语音**（如长篇朗读）会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 15` 限制临时预览窗口；多客户端或连续独白压测时可降到 `8-10`，并把 `--decode-interval` 提高到 `0.8-1.0`。这只影响临时 `partial`，VAD 锁定句和 STOP 最终结果仍走完整音频。 -### 6.6 说话人分离（SPK）的代价与关闭 +### 6.6 说话人分离（SPK）的代价与开关 -`serve_realtime_ws.py` 启动时**默认加载** SPK 模型（目前写死为 `iic/speech_eres2netv2_sv_zh-cn_16k-common`），并在流式中对每个 VAD 完成句调用一次说话人分配。需要注意： +`serve_realtime_ws.py` 默认**不加载** SPK 模型。只有启动时显式加 `--enable-spk`，才会加载 `--spk-model`（默认 `iic/speech_eres2netv2_sv_zh-cn_16k-common`）并在流式中对每个 VAD 完成句调用一次说话人分配。需要注意： - **Fun-ASR-Nano 上 SPK 效果有限**（见 #2944），多数实时 ASR 场景并不需要说话人分离。 - **流式 SPK 代价高且随会话变长**：每句对**全部历史 embedding** 做一次全量重聚类（**O(N²)**，会话越长每句越贵），且**同步阻塞事件循环**；而会话结束时还会**全量重聚一遍**，流式期间每句的聚类结果会被最终结果覆盖——对最终输出而言属于重复计算。长会话 + 高并发下尤其明显。 -- **目前关闭需改代码**：spk_model 在 `serve_realtime_ws.py` 中默认加载、且写死，没有开关. -- **建议**：不需要说话人分离时不加载 spk_model；确需 diarization 时，从最终结果取标签，或改用 Paraformer + cam++ 方案。 +- **建议**：多客户端实时转写优先保持默认关闭；确需 diarization 时再加 `--enable-spk`，并以 STOP 后的最终 `spk` 标签为准。 ### 6.7 生产并发与多进程部署 diff --git a/docs/vllm_guide_zh_v2.md b/docs/vllm_guide_zh_v2.md index 32cc640f9..a502df415 100644 --- a/docs/vllm_guide_zh_v2.md +++ b/docs/vllm_guide_zh_v2.md @@ -553,7 +553,7 @@ asyncio.run(offline_ws("audio.wav")) - 音频逐帧到达，边收边处理 - 基于 VAD 端点自然分句 - 确认段文字锁定不变，partial 实时更新 -- 流式说话人分配 + STOP 时全局重聚类 +- 可选流式说话人分配（`--enable-spk`）+ STOP 时全局重聚类 - 首字延迟 ~480ms ### 6.2 启动服务 @@ -563,6 +563,16 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/ --port 10095 --language 中文 --hotword-file 热词列表 ``` +多客户端或长时间连续语音场景，建议先限制 partial 预览窗口并适当降低刷新频率： + +```bash +CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py \ + --port 10095 --language 中文 \ + --partial-window-sec 8 --decode-interval 0.8 +``` + +说话人分离默认关闭；只有确实需要 `spk` 字段时再加 `--enable-spk`。 + ### 6.3 WebSocket 协议 **连接**：`ws://host:10095` @@ -575,18 +585,18 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/ | 热词 | `"HOTWORDS:词1,词2"` | 可选 | | 语种 | `"LANGUAGE:中文"` | 可选 | | 音频 | `binary` | PCM16 16kHz mono | -| 结束 | `"STOP"` | 最终解码 + SPK 重聚类 | +| 结束 | `"STOP"` | 最终解码；启用 `--enable-spk` 时会做 SPK 重聚类 | **服务端 → 客户端**： ```json {"event": "started"} -{"sentences": [{"text":"你好","start":300,"end":1200,"spk":0}], "partial": "世界", "is_final": false} +{"sentences": [{"text":"你好","start":300,"end":1200}], "partial": "世界", "is_final": false} {"sentences": [...], "is_final": true} {"event": "stopped"} ``` -**字段**：`sentences[]` = 已锁定，`partial` = 正在说（会变），`is_final` = STOP 后为 true。 +**字段**：`sentences[]` = 已锁定，`partial` = 正在说（会变），`is_final` = STOP 后为 true。启用 `--enable-spk` 后，`sentences[]` 会包含 `spk`。 **时序**： ``` @@ -655,16 +665,15 @@ Fun-ASR-Nano 的声学编码器（SenseVoice）是**全上下文、非流式** **使用建议** - 正常对话语音有自然停顿，VAD 会把它切成一句句较短的语音，每句 partial 的开销自然受限，**通常无需关注**。 -- 只有**超长、不停顿的连续语音**（如长篇朗读）会让单句不断变长、partial 预览逐渐变慢——这是全上下文编码器的固有特性，**不是配置问题**。若你的场景确有此类音频且对预览延迟敏感，可在代码侧限制 partial 的编码窗口（如只编最近若干秒）；这属于**代码改动、非配置项**，且不影响锁定句（仍走完整音频、最终转写不变）。 +- 只有**超长、不停顿的连续语音**（如长篇朗读）会让单句不断变长、partial 预览逐渐变慢。`serve_realtime_ws.py` 默认用 `--partial-window-sec 15` 限制临时预览窗口；多客户端或连续独白压测时可降到 `8-10`，并把 `--decode-interval` 提高到 `0.8-1.0`。这只影响临时 `partial`，VAD 锁定句和 STOP 最终结果仍走完整音频。 -### 6.6 说话人分离（SPK）的代价与关闭 +### 6.6 说话人分离（SPK）的代价与开关 -`serve_realtime_ws.py` 启动时**默认加载** SPK 模型（目前写死为 `iic/speech_eres2netv2_sv_zh-cn_16k-common`），并在流式中对每个 VAD 完成句调用一次说话人分配。需要注意： +`serve_realtime_ws.py` 默认**不加载** SPK 模型。只有启动时显式加 `--enable-spk`，才会加载 `--spk-model`（默认 `iic/speech_eres2netv2_sv_zh-cn_16k-common`）并在流式中对每个 VAD 完成句调用一次说话人分配。需要注意： - **Fun-ASR-Nano 上 SPK 效果有限**（见 #2944），多数实时 ASR 场景并不需要说话人分离。 - **流式 SPK 代价高且随会话变长**：每句对**全部历史 embedding** 做一次全量重聚类（**O(N²)**，会话越长每句越贵），且**同步阻塞事件循环**；而会话结束时还会**全量重聚一遍**，流式期间每句的聚类结果会被最终结果覆盖——对最终输出而言属于重复计算。长会话 + 高并发下尤其明显。 -- **目前关闭需改代码**：spk_model 在 `serve_realtime_ws.py` 中默认加载、且写死，没有开关. -- **建议**：不需要说话人分离时不加载 spk_model；确需 diarization 时，从最终结果取标签，或改用 Paraformer + cam++ 方案。 +- **建议**：多客户端实时转写优先保持默认关闭；确需 diarization 时再加 `--enable-spk`，并以 STOP 后的最终 `spk` 标签为准。 ### 6.7 生产并发与多进程部署 diff --git a/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py b/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py index 54cc6dfd3..a8d8e5049 100644 --- a/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py +++ b/examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py @@ -249,6 +249,7 @@ class RealtimeASRSession: self.vad_fed_samples = 0 self.prev_text = "" self.last_partial_text = "" + self.last_partial_start_ms = 0 self.last_decode_samples = 0 self.locked_sentences = [] self.prev_seg_text = "" @@ -311,14 +312,7 @@ class RealtimeASRSession: return self._build_response(is_final) if self.vad.current_speech_start is not None: - seg_start_sample = int(self.vad.current_speech_start * self.sample_rate / 1000) - seg_audio = self.audio_buffer[seg_start_sample:] - # Bound the interim re-decode cost for a long ongoing segment: only - # re-encode the most recent partial_window_samples. The full segment is - # still decoded once the VAD boundary fires (see _decode_segment and the - # is_final path), so only the interim hypothesis is affected. - if self.partial_window_samples and len(seg_audio) > self.partial_window_samples: - seg_audio = seg_audio[-self.partial_window_samples:] + seg_audio, partial_start_ms = self.get_partial_decode_audio() else: self.last_decode_samples = len(self.audio_buffer) self.last_partial_text = "" @@ -347,6 +341,7 @@ class RealtimeASRSession: self.last_decode_samples = len(self.audio_buffer) self.last_partial_text = text + self.last_partial_start_ms = partial_start_ms if text.strip() and not self.first_decode_done: self.first_decode_done = True @@ -359,6 +354,20 @@ class RealtimeASRSession: return self._build_response(is_final) + def get_partial_decode_audio(self): + """Return the bounded audio window used for unstable partial decoding.""" + seg_start_sample = int(self.vad.current_speech_start * self.sample_rate / 1000) + decode_start_sample = seg_start_sample + + if self.partial_window_samples: + min_start = len(self.audio_buffer) - self.partial_window_samples + if min_start > decode_start_sample: + decode_start_sample = min_start + + decode_start_sample = max(0, decode_start_sample) + start_ms = int(decode_start_sample * 1000 / self.sample_rate) + return self.audio_buffer[decode_start_sample:], start_ms + @torch.no_grad() def _decode_segment(self, seg): """Decode a completed VAD segment via vLLM.""" @@ -387,7 +396,12 @@ class RealtimeASRSession: duration_ms = int(len(self.audio_buffer) * 1000 / self.sample_rate) sentences = list(self.locked_sentences) partial = self.last_partial_text - partial_start = self.vad.current_speech_start or duration_ms + if partial: + partial_start = self.last_partial_start_ms + elif self.vad.current_speech_start is not None: + partial_start = self.vad.current_speech_start + else: + partial_start = duration_ms if is_final: return {"sentences": sentences, "partial": "", "partial_start_ms": 0, @@ -403,6 +417,7 @@ class RealtimeASRSession: self.vad.reset() self.prev_text = "" self.last_partial_text = "" + self.last_partial_start_ms = 0 self.last_decode_samples = 0 self.locked_sentences = [] if self.spk_tracker: @@ -445,19 +460,34 @@ def load_models(args): logger.info("Loading VAD: fsmn-vad (streaming)") _vad_model = AutoModel(model="fsmn-vad", device=args.device, disable_update=True) - logger.info("Loading SPK: eres2netv2") - _spk_model = AutoModel(model="iic/speech_eres2netv2_sv_zh-cn_16k-common", device=args.device, disable_update=True) + if getattr(args, "enable_spk", False): + logger.info(f"Loading SPK: {args.spk_model}") + _spk_model = AutoModel(model=args.spk_model, device=args.device, disable_update=True) + else: + _spk_model = None + logger.info("SPK disabled; use --enable-spk to include speaker diarization") logger.info("All models ready!") return _vllm_engine, _asr_kwargs, _vad_model, _spk_model +def create_speaker_tracker(spk_model, args): + if not getattr(args, "enable_spk", False) or spk_model is None: + return None + return HybridSpeakerTracker(spk_model, args.device) + + async def handle_client(websocket, args): vllm_engine, asr_kwargs, vad_model, spk_model = load_models(args) vad = DynamicStreamingVAD(vad_model) - spk_tracker = HybridSpeakerTracker(spk_model, args.device) - session = RealtimeASRSession(vllm_engine, asr_kwargs, vad, spk_tracker=spk_tracker, - partial_window_sec=getattr(args, 'partial_window_sec', 15.0)) + spk_tracker = create_speaker_tracker(spk_model, args) + session = RealtimeASRSession( + vllm_engine, + asr_kwargs, + vad, + spk_tracker=spk_tracker, + partial_window_sec=getattr(args, 'partial_window_sec', 15.0), + ) logger.info(f"Client connected: {websocket.remote_address}") decode_interval = args.decode_interval @@ -516,7 +546,7 @@ async def main(args): await asyncio.Future() -if __name__ == "__main__": +def build_arg_parser(): parser = argparse.ArgumentParser(description="Fun-ASR-Nano Streaming WebSocket Server") parser.add_argument("--port", type=int, default=10095) parser.add_argument("--model", type=str, default="FunAudioLLM/Fun-ASR-Nano-2512") @@ -531,11 +561,17 @@ if __name__ == "__main__": "every chunk (O(L^2)), which saturates the GPU under concurrency and times " "out long-segment requests. Lower it (e.g. 8-10) for high-concurrency " "self-hosting; <=0 disables (legacy behaviour). Final transcripts are unaffected.") + parser.add_argument("--enable-spk", action="store_true", help="Enable streaming speaker diarization.") + parser.add_argument("--spk-model", type=str, default="iic/speech_eres2netv2_sv_zh-cn_16k-common") parser.add_argument("--hotword-file", type=str, default="热词列表") parser.add_argument("--language", type=str, default=None, help="Language hint (e.g. 中文, English, 日本語)") parser.add_argument("--dtype", type=str, default="bf16", choices=["bf16", "fp16", "fp32"]) parser.add_argument("--tensor-parallel-size", type=int, default=1) parser.add_argument("--gpu-memory-utilization", type=float, default=0.8) parser.add_argument("--max-model-len", type=int, default=2048) - args = parser.parse_args() + return parser + + +if __name__ == "__main__": + args = build_arg_parser().parse_args() asyncio.run(main(args)) diff --git a/tests/test_realtime_ws_service.py b/tests/test_realtime_ws_service.py new file mode 100644 index 000000000..f33a34ccd --- /dev/null +++ b/tests/test_realtime_ws_service.py @@ -0,0 +1,122 @@ +import importlib.util +import sys +import types +from pathlib import Path + +import numpy as np + + +REPO_ROOT = Path(__file__).resolve().parents[1] +SERVICE_PATH = REPO_ROOT / "examples" / "industrial_data_pretraining" / "fun_asr_nano" / "serve_realtime_ws.py" + + +def load_service_module(): + websockets_stub = types.SimpleNamespace( + exceptions=types.SimpleNamespace(ConnectionClosed=Exception), + serve=lambda *args, **kwargs: None, + ) + sys.modules.setdefault("websockets", websockets_stub) + + module_name = "serve_realtime_ws_under_test" + sys.modules.pop(module_name, None) + spec = importlib.util.spec_from_file_location(module_name, SERVICE_PATH) + module = importlib.util.module_from_spec(spec) + assert spec.loader is not None + spec.loader.exec_module(module) + return module + + +def test_cli_defaults_disable_speaker_and_bound_partial_window(): + module = load_service_module() + + assert hasattr(module, "build_arg_parser") + args = module.build_arg_parser().parse_args([]) + + assert args.enable_spk is False + assert args.partial_window_sec == 15.0 + + +def test_create_speaker_tracker_skips_spk_when_disabled(): + module = load_service_module() + args = types.SimpleNamespace(enable_spk=False, device="cuda:0") + + assert module.create_speaker_tracker(object(), args) is None + + +def test_partial_decode_audio_uses_recent_window(): + module = load_service_module() + + class DummyVad: + current_speech_start = 0 + + def reset(self): + pass + + sample_rate = 16000 + session = module.RealtimeASRSession( + vllm_engine=object(), + asr_kwargs={}, + vad=DummyVad(), + spk_tracker=None, + sample_rate=sample_rate, + chunk_ms=960, + partial_window_sec=2.0, + ) + session.audio_buffer = np.arange(sample_rate * 5, dtype=np.float32) + + audio, start_ms = session.get_partial_decode_audio() + + assert start_ms == 3000 + np.testing.assert_array_equal(audio, session.audio_buffer[-sample_rate * 2:]) + + +def test_partial_decode_audio_keeps_short_current_segment(): + module = load_service_module() + + class DummyVad: + current_speech_start = 1500 + + def reset(self): + pass + + sample_rate = 16000 + session = module.RealtimeASRSession( + vllm_engine=object(), + asr_kwargs={}, + vad=DummyVad(), + spk_tracker=None, + sample_rate=sample_rate, + chunk_ms=960, + partial_window_sec=10.0, + ) + session.audio_buffer = np.arange(sample_rate * 5, dtype=np.float32) + + audio, start_ms = session.get_partial_decode_audio() + assert start_ms == 1500 + np.testing.assert_array_equal(audio, session.audio_buffer[int(sample_rate * 1.5):]) + + +def test_empty_partial_response_preserves_zero_speech_start(): + module = load_service_module() + + class DummyVad: + current_speech_start = 0 + + def reset(self): + pass + + sample_rate = 16000 + session = module.RealtimeASRSession( + vllm_engine=object(), + asr_kwargs={}, + vad=DummyVad(), + spk_tracker=None, + sample_rate=sample_rate, + chunk_ms=960, + ) + session.audio_buffer = np.zeros(sample_rate, dtype=np.float32) + session.last_partial_text = "" + + response = session._build_response(is_final=False) + + assert response["partial_start_ms"] == 0