Files
zero/internal/dictation/server_test.go
Anandan 87158a1c90 feat: voice dictation (speech-to-text) (#557)
* ## Speech-to-text dictation

Adds voice dictation to Zero. Run `/voice` to enter hold-to-record mode, then hold **Space** to dictate and release to transcribe (run `/voice` again to turn it off so Space types normally). Pick a backend with `/stt-model`.

### Backends
- **Local (sherpa-onnx)** — offline, no API key. Auto-downloads a model via `/stt-model`; the engine binary is fetched once and shared across models. Family auto-detection covers the mainstream offline families (Whisper, transducer/Zipformer, Moonshine, Paraformer, SenseVoice, and the CTC families FireRedASR, NeMo, TeleSpeech, WeNet, Zipformer-CTC, Dolphin, Omnilingual — plus the encoder/decoder families Canary and FireRedASR-AED).
- **Cloud batch** — Groq and OpenAI Whisper. Keys resolve from a configured provider, the encrypted credential store, or the env var; a missing/invalid key is entered inline.
- **Streaming** — a warm local sherpa-onnx websocket server (live transcript as you speak) on desktop, or Deepgram / OpenAI Realtime.

### Highlights
- Live audio-reactive waveform while recording (real mic levels when streaming; a synthesized wave for batch).
- `/stt-model` download chooser: recommended models first, Live/Batch tag per row, size + download state, and a search filter.
- Streaming transcript is inserted live into the composer and committed on stop; batch inserts on stop. Insert-for-review by default (`stt.autoSubmit` to fire straight at the agent).
- The warm streaming server is released when `/voice` is turned off and on exit.

### Implementation notes
- No CGO and no bundled runtimes — every OS integration shells out via `exec.Command` with discrete argv (never a shell string).
- The streaming server manager mirrors the LSP manager's warm-subprocess lifecycle.
- OS-boundary code (recorders, transcribers, server) is unit-tested via injected fakes; real microphone / websocket behavior is covered by env-gated smoke tests.

### Testing
- `go build ./...`, `go vet ./...`, and the config / dictation / cli / tui suites pass.
- Verified end-to-end against real downloaded binaries + models (offline batch, streaming websocket server, and the model auto-download), including the FireRedASR-CTC and NeMo streaming transducer families.

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **New Features**
  * Added speech-to-text dictation with both batch and streaming modes, supporting local (sherpa-onnx) and cloud providers, including WAV/M4A detection.
  * Introduced voice dictation mode with STT model selection, local engine/model auto-download, and an inline prompt to save cloud API keys.
  * Added new TUI commands for voice/STT model selection and enabled F1–F12 key bindings.
* **Bug Fixes**
  * Improved streaming dictation to preserve accumulated transcript partials and keep the editor region consistent.
  * Refined TUI status/picker rendering, including clearer loading/search behavior and more reliable selection alignment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

* fix(dictation): address CodeRabbit review feedback

- validateSTTConfig: add NumThreads bounds check
- downloadVerifyExtract: stage extraction into a sibling temp dir,
  atomic-rename into destDir on success so a partial extract cannot
  leave a truncated engine binary that passes fileExists() on next run
- startProcess: close pipe fds when cmd.Start() fails
- ServerManager.Shutdown: honor the caller's context deadline
- audioSpillDir: tighten any pre-existing directory to 0700
- startVariantDownload: refuse re-entry while a download is in flight
- maybeOpenSTTDownloadPicker: defer Provider=local persistence to
  applyEngineComponents (commit-on-success), matching the cloud flow
- fetchSTTModelsCmd: bound with a 30s timeout client+context
- newSTTDownloadPickerFrom.isCurrent: compare path segments via
  filepath.Base to avoid substring false-positives

* fix(dictation): harden recorder.Stop, Esc-cancel priority, streaming region

Three correctness issues from PR review:

1) recorder.Stop() panicked (nil-deref) when called on a StartStreaming
   session because StartStreaming intentionally leaves r.proc nil. Reject
   that misuse with a clear error. Termux's batch path also leaves
   r.proc nil by design — keep that path working.

2) Esc was consumed by an active dictation session before the
   confirming second Esc could cancel a pending run, so a user
   mid-recording who double-Esc's to kill the run found the first Esc
   swallowed by dictation. Gate dictation-cancel on the Esc not being
   a pending-run confirm.

3) applyStreamingText hard-reset the cursor to regionStart on every
   partial, yanking focus away from users editing elsewhere in the
   composer, and stored regionStart/End as absolute indices that became
   stale the moment the user typed or pasted before the live region.
   Preserve the cursor when it's outside the region, and snapshot the
   prefix BEFORE the region so the next partial can detect external
   edits and shift [start,end) to stay aligned.
2026-07-07 11:13:14 +08:00

190 lines
5.4 KiB
Go

package dictation
import (
"context"
"encoding/binary"
"errors"
"io"
"math"
"os"
"os/exec"
"path/filepath"
"sync"
"testing"
)
func streamingModelDir(t *testing.T) string {
t.Helper()
dir := t.TempDir()
for _, f := range []string{"encoder.onnx", "decoder.onnx", "joiner.onnx", "tokens.txt"} {
if err := os.WriteFile(filepath.Join(dir, f), []byte("x"), 0o600); err != nil {
t.Fatal(err)
}
}
return dir
}
func TestServerManagerLazyStartAndReuse(t *testing.T) {
var startCount int
var mu sync.Mutex
starter := func(spec commandSpec) (processHandle, io.ReadCloser, error) {
mu.Lock()
startCount++
mu.Unlock()
if spec.name != "sherpa-onnx-online-websocket-server" {
t.Errorf("binary = %q", spec.name)
}
return newFakeProcess(), nil, nil
}
alive := true
m := NewServerManager(ServerConfig{
ModelPath: streamingModelDir(t),
Port: 6006,
starter: starter,
healthCheck: func(context.Context, int) error { return nil },
aliveCheck: func(int) bool { return alive },
})
url, err := m.EnsureRunning(context.Background())
if err != nil {
t.Fatalf("EnsureRunning: %v", err)
}
if url != "ws://127.0.0.1:6006" {
t.Errorf("url = %q", url)
}
// Second call reuses the warm server (alive) — no new start.
if _, err := m.EnsureRunning(context.Background()); err != nil {
t.Fatalf("EnsureRunning reuse: %v", err)
}
if startCount != 1 {
t.Errorf("expected 1 start (warm reuse), got %d", startCount)
}
}
func TestServerManagerRestartsCrashedServer(t *testing.T) {
var startCount int
starter := func(commandSpec) (processHandle, io.ReadCloser, error) {
startCount++
return newFakeProcess(), nil, nil
}
alive := true
m := NewServerManager(ServerConfig{
ModelPath: streamingModelDir(t),
starter: starter,
healthCheck: func(context.Context, int) error { return nil },
aliveCheck: func(int) bool { return alive },
})
if _, err := m.EnsureRunning(context.Background()); err != nil {
t.Fatal(err)
}
alive = false // simulate a crashed server
if _, err := m.EnsureRunning(context.Background()); err != nil {
t.Fatal(err)
}
if startCount != 2 {
t.Errorf("expected a restart (2 starts), got %d", startCount)
}
}
func TestServerManagerMissingBinaryIsSetupError(t *testing.T) {
m := NewServerManager(ServerConfig{
ModelPath: streamingModelDir(t),
starter: func(commandSpec) (processHandle, io.ReadCloser, error) { return nil, nil, exec.ErrNotFound },
})
_, err := m.EnsureRunning(context.Background())
var setupErr *SetupError
if !errors.As(err, &setupErr) {
t.Fatalf("want *SetupError, got %v", err)
}
}
func TestServerManagerMissingModelIsSetupError(t *testing.T) {
m := NewServerManager(ServerConfig{
ModelPath: t.TempDir(), // empty dir, no onnx/tokens
starter: func(commandSpec) (processHandle, io.ReadCloser, error) { return newFakeProcess(), nil, nil },
})
_, err := m.EnsureRunning(context.Background())
var setupErr *SetupError
if !errors.As(err, &setupErr) {
t.Fatalf("want *SetupError for missing streaming model, got %v", err)
}
}
func TestServerManagerShutdown(t *testing.T) {
proc := newFakeProcess()
m := NewServerManager(ServerConfig{
ModelPath: streamingModelDir(t),
starter: func(commandSpec) (processHandle, io.ReadCloser, error) { return proc, nil, nil },
healthCheck: func(context.Context, int) error { return nil },
aliveCheck: func(int) bool { return true },
})
if _, err := m.EnsureRunning(context.Background()); err != nil {
t.Fatal(err)
}
if err := m.Shutdown(context.Background()); err != nil {
t.Fatalf("Shutdown: %v", err)
}
if !proc.stopped && !proc.killed {
t.Error("Shutdown should stop the server process")
}
if m.URL() != "" {
t.Error("URL should be empty after shutdown")
}
}
func TestOnlineServerCommandBuildsTransducerArgs(t *testing.T) {
spec, err := onlineServerCommand(ServerConfig{Binary: "sherpa-onnx-online-websocket-server", ModelPath: streamingModelDir(t), Port: 6007})
if err != nil {
t.Fatal(err)
}
joined := ""
for _, a := range spec.args {
joined += a + " "
}
for _, want := range []string{"--port=6007", "--tokens=", "--encoder=", "--decoder=", "--joiner=", "--decoding-method=greedy_search"} {
if !contains(joined, want) {
t.Errorf("missing %q in args: %s", want, joined)
}
}
}
func TestPCM16ToFloat32LE(t *testing.T) {
// int16 max (32767) and min (-32768), little-endian.
pcm := []byte{0xFF, 0x7F, 0x00, 0x80}
out := pcm16ToFloat32LE(pcm)
if len(out) != 8 {
t.Fatalf("expected 8 bytes (2 float32), got %d", len(out))
}
f0 := math.Float32frombits(binary.LittleEndian.Uint32(out[0:4]))
f1 := math.Float32frombits(binary.LittleEndian.Uint32(out[4:8]))
if f0 < 0.99 || f0 > 1.0 {
t.Errorf("32767 → %f, want ~1.0", f0)
}
if f1 != -1.0 {
t.Errorf("-32768 → %f, want -1.0", f1)
}
}
func TestParseSherpaStreamResult(t *testing.T) {
text, segment, final, ok := parseSherpaStreamResult([]byte(`{"text":" hello world ","segment":2,"is_final":true}`))
if !ok || text != "hello world" || segment != 2 || !final {
t.Errorf("parsed = (%q, %d, %v, %v)", text, segment, final, ok)
}
if _, _, _, ok := parseSherpaStreamResult([]byte("not json")); ok {
t.Error("non-JSON should not parse")
}
}
func contains(hay, needle string) bool {
return len(needle) == 0 || (len(hay) >= len(needle) && indexOf(hay, needle) >= 0)
}
func indexOf(hay, needle string) int {
for i := 0; i+len(needle) <= len(hay); i++ {
if hay[i:i+len(needle)] == needle {
return i
}
}
return -1
}