mirror of
https://github.com/Twigpine/zero.git
synced 2026-11-10 13:11:31 -10:00
* ## Speech-to-text dictation Adds voice dictation to Zero. Run `/voice` to enter hold-to-record mode, then hold **Space** to dictate and release to transcribe (run `/voice` again to turn it off so Space types normally). Pick a backend with `/stt-model`. ### Backends - **Local (sherpa-onnx)** — offline, no API key. Auto-downloads a model via `/stt-model`; the engine binary is fetched once and shared across models. Family auto-detection covers the mainstream offline families (Whisper, transducer/Zipformer, Moonshine, Paraformer, SenseVoice, and the CTC families FireRedASR, NeMo, TeleSpeech, WeNet, Zipformer-CTC, Dolphin, Omnilingual — plus the encoder/decoder families Canary and FireRedASR-AED). - **Cloud batch** — Groq and OpenAI Whisper. Keys resolve from a configured provider, the encrypted credential store, or the env var; a missing/invalid key is entered inline. - **Streaming** — a warm local sherpa-onnx websocket server (live transcript as you speak) on desktop, or Deepgram / OpenAI Realtime. ### Highlights - Live audio-reactive waveform while recording (real mic levels when streaming; a synthesized wave for batch). - `/stt-model` download chooser: recommended models first, Live/Batch tag per row, size + download state, and a search filter. - Streaming transcript is inserted live into the composer and committed on stop; batch inserts on stop. Insert-for-review by default (`stt.autoSubmit` to fire straight at the agent). - The warm streaming server is released when `/voice` is turned off and on exit. ### Implementation notes - No CGO and no bundled runtimes — every OS integration shells out via `exec.Command` with discrete argv (never a shell string). - The streaming server manager mirrors the LSP manager's warm-subprocess lifecycle. - OS-boundary code (recorders, transcribers, server) is unit-tested via injected fakes; real microphone / websocket behavior is covered by env-gated smoke tests. ### Testing - `go build ./...`, `go vet ./...`, and the config / dictation / cli / tui suites pass. - Verified end-to-end against real downloaded binaries + models (offline batch, streaming websocket server, and the model auto-download), including the FireRedASR-CTC and NeMo streaming transducer families. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added speech-to-text dictation with both batch and streaming modes, supporting local (sherpa-onnx) and cloud providers, including WAV/M4A detection. * Introduced voice dictation mode with STT model selection, local engine/model auto-download, and an inline prompt to save cloud API keys. * Added new TUI commands for voice/STT model selection and enabled F1–F12 key bindings. * **Bug Fixes** * Improved streaming dictation to preserve accumulated transcript partials and keep the editor region consistent. * Refined TUI status/picker rendering, including clearer loading/search behavior and more reliable selection alignment. <!-- end of auto-generated comment: release notes by coderabbit.ai --> * fix(dictation): address CodeRabbit review feedback - validateSTTConfig: add NumThreads bounds check - downloadVerifyExtract: stage extraction into a sibling temp dir, atomic-rename into destDir on success so a partial extract cannot leave a truncated engine binary that passes fileExists() on next run - startProcess: close pipe fds when cmd.Start() fails - ServerManager.Shutdown: honor the caller's context deadline - audioSpillDir: tighten any pre-existing directory to 0700 - startVariantDownload: refuse re-entry while a download is in flight - maybeOpenSTTDownloadPicker: defer Provider=local persistence to applyEngineComponents (commit-on-success), matching the cloud flow - fetchSTTModelsCmd: bound with a 30s timeout client+context - newSTTDownloadPickerFrom.isCurrent: compare path segments via filepath.Base to avoid substring false-positives * fix(dictation): harden recorder.Stop, Esc-cancel priority, streaming region Three correctness issues from PR review: 1) recorder.Stop() panicked (nil-deref) when called on a StartStreaming session because StartStreaming intentionally leaves r.proc nil. Reject that misuse with a clear error. Termux's batch path also leaves r.proc nil by design — keep that path working. 2) Esc was consumed by an active dictation session before the confirming second Esc could cancel a pending run, so a user mid-recording who double-Esc's to kill the run found the first Esc swallowed by dictation. Gate dictation-cancel on the Esc not being a pending-run confirm. 3) applyStreamingText hard-reset the cursor to regionStart on every partial, yanking focus away from users editing elsewhere in the composer, and stored regionStart/End as absolute indices that became stale the moment the user typed or pasted before the live region. Preserve the cursor when it's outside the region, and snapshot the prefix BEFORE the region so the next partial can detect external edits and shift [start,end) to stay aligned.
97 lines
3.4 KiB
Go
97 lines
3.4 KiB
Go
package cli
|
|
|
|
import (
|
|
"errors"
|
|
"testing"
|
|
|
|
"github.com/Gitlawb/zero/internal/config"
|
|
"github.com/Gitlawb/zero/internal/dictation"
|
|
)
|
|
|
|
func TestFactoryFallsBackToBatchWhenLocalStreamingNotConfigured(t *testing.T) {
|
|
t.Setenv("GROQ_API_KEY", "gsk-test")
|
|
resolved := config.ResolvedConfig{STT: config.STTConfig{
|
|
Provider: config.STTProviderGroq, // cloud batch
|
|
// StreamProvider unset → defaults to local, which has no model configured.
|
|
}}
|
|
factory := newDictationTranscriberFactory(resolved, "", newDictationServerManager(resolved.STT))
|
|
|
|
tr, streaming, err := factory(resolved.STT, true) // prefer streaming
|
|
if err != nil {
|
|
t.Fatalf("expected batch fallback, got error: %v", err)
|
|
}
|
|
if streaming {
|
|
t.Error("should have fallen back to batch (streaming=false)")
|
|
}
|
|
if tr == nil {
|
|
t.Error("expected a Groq batch transcriber")
|
|
}
|
|
}
|
|
|
|
func TestFactoryCloudProviderIgnoresLeftoverLocalModel(t *testing.T) {
|
|
// Regression: pick local once (LocalModelPath gets set), then switch to a
|
|
// cloud batch provider. streamProvider stays unset (defaults to "local"), so
|
|
// the streaming factory must NOT fire up the sherpa server on the leftover
|
|
// path — it should fall back to the cloud batch pipeline the user chose.
|
|
t.Setenv("GROQ_API_KEY", "gsk-test")
|
|
stt := config.STTConfig{
|
|
Provider: config.STTProviderGroq, // cloud batch, chosen now
|
|
LocalModelPath: "/leftover/model/dir", // set by an earlier local selection
|
|
}
|
|
resolved := config.ResolvedConfig{STT: stt}
|
|
factory := newDictationTranscriberFactory(resolved, "", newDictationServerManager(stt))
|
|
|
|
tr, streaming, err := factory(stt, true) // prefer streaming
|
|
if err != nil {
|
|
t.Fatalf("expected cloud batch fallback, got error: %v", err)
|
|
}
|
|
if streaming {
|
|
t.Error("a cloud provider must not stream through the local sherpa server")
|
|
}
|
|
if tr == nil {
|
|
t.Error("expected a Groq batch transcriber")
|
|
}
|
|
}
|
|
|
|
func TestFactorySurfacesExplicitCloudStreamingError(t *testing.T) {
|
|
// Deepgram chosen explicitly but no key resolvable → surface the setup error.
|
|
resolved := config.ResolvedConfig{STT: config.STTConfig{
|
|
StreamProvider: config.STTProviderDeepgram,
|
|
}}
|
|
factory := newDictationTranscriberFactory(resolved, "", nil)
|
|
|
|
_, streaming, err := factory(resolved.STT, true)
|
|
if err == nil {
|
|
t.Fatal("expected a setup error for missing Deepgram key")
|
|
}
|
|
if !streaming {
|
|
t.Error("an explicit cloud streaming choice should report streaming=true even on error")
|
|
}
|
|
var setupErr *dictation.SetupError
|
|
if !errors.As(err, &setupErr) {
|
|
t.Errorf("want *SetupError, got %v", err)
|
|
}
|
|
}
|
|
|
|
func TestFactoryLocalBatchSetupErrorWhenNoModel(t *testing.T) {
|
|
resolved := config.ResolvedConfig{STT: config.STTConfig{Provider: config.STTProviderLocal}}
|
|
factory := newDictationTranscriberFactory(resolved, "", newDictationServerManager(resolved.STT))
|
|
|
|
_, _, err := factory(resolved.STT, true)
|
|
var setupErr *dictation.SetupError
|
|
if !errors.As(err, &setupErr) {
|
|
t.Fatalf("want *SetupError guiding local model setup, got %v", err)
|
|
}
|
|
}
|
|
|
|
func TestResolveSTTKeyPrefersConfiguredProvider(t *testing.T) {
|
|
resolved := config.ResolvedConfig{
|
|
Providers: []config.ProviderProfile{
|
|
{Name: "groq", CatalogID: "groq", BaseURL: "https://api.groq.com/openai/v1", APIKey: "gsk-configured"},
|
|
},
|
|
}
|
|
if got := resolveSTTKey(resolved, "", dictation.ProviderGroq, "GROQ_API_KEY"); got != "gsk-configured" {
|
|
t.Errorf("key = %q, want the configured provider's key", got)
|
|
}
|
|
}
|