mirror of
https://github.com/Twigpine/zero.git
synced 2026-11-08 17:56:14 -10:00
68facda077
* zeroruntime: add ImageBlock type and NormalizeImageMediaType * zeroruntime: add optional Images field to Message * zeroruntime: add SeedMessagesWithImages, delegate SeedMessages with nil * Emit Anthropic image source blocks for user image turns * Add inlineData part type to Gemini request schema * Serialize user image attachments as Gemini inlineData parts * test(openai): pin text-only chat request serialization before content refactor * feat(openai): allow structured multimodal content parts in chat request * feat(openai): map image attachments to image_url content parts * test(openai): assert image content parts serialize over the request body * modelregistry: add SupportsVision capability gate * Add shared image-file loader package * cli: parse repeatable exec --image flag into imagePaths * cli: resolve exec image attachments via shared loader with sniff and size cap * agent: thread image attachments through Options into the seeded user turn * cli: resolve and vision-gate exec image attachments into the agent run * cli: document exec --image flag in help output * streamjson: add InputImage type and InputEvent.Images field * streamjson: whitelist images field on message events only * streamjson: allow empty content on message events that carry images * streamjson: add ResolveImages to decode, normalize and cap input images * streamjson: cover image parse-and-resolve round-trip and text-only invariance * Add /image command definition to TUI * Add synchronous vision check for TUI model * Implement /image attach, clear, and non-vision refusal in TUI * Render pending image chips above the input in both skins * Thread pending images into the agent turn and clear on submit * Wire stream-json images through to the agent run Stream-json message images were parsed and validated but never reached agent.Options.Images: streamjson.ResolveImages had no production caller, so images sent over stream-json input were silently dropped. resolveExecPrompt now also returns the resolved stream-json images (nil for text input and for image-free stream-json input). The exec runner merges them with --image attachments before the vision gate, so both input sources flow through the same drop+warn and the same agent.Options.Images wiring. Adds an exec-level end-to-end test that feeds a base64 image on a stream-json message event via stdin on a vision model and asserts the image reaches the agent run (the provider request's user turn). * Re-check TUI vision support at submit, not only at /image attach A user could attach an image on a vision model and then switch to a non-vision model via /model before submitting; the images were threaded unconditionally into the agent turn, reaching a model that rejects them. The commandPrompt submit path now re-runs the vision check against the current effective model. When the active model can't accept images, the pending images are dropped (not threaded) and an inline notice mirroring the headless drop+warn wording is appended. Pending image state is still cleared after submit either way. Adds a test that attaches on a vision model, switches the model id to a non-vision one, submits, and asserts the run received no images, an inline notice was appended, and pending state was cleared. * Bound memory with an os.Stat size pre-check in imageinput.LoadFile LoadFile read the whole file into memory before checking the size cap, so a multi-gigabyte file allocated a huge buffer only to be discarded. Add an os.Stat size pre-check before os.ReadFile: a file whose size exceeds MaxImageBytes is rejected with the same 10 MiB-limit error before any read; a missing file surfaces the same not-found error. The existing post-read len(data) check is kept as a TOCTOU backstop in case the file grows between Stat and ReadFile. * OpenAI: only the user role emits image content-parts The Anthropic and Gemini providers only attach images in their user branches, but OpenAI funnels every role through a single mapMessage, which would emit image content-parts for any role carrying Images. Guard the content-parts path to the user role: a non-user message that happens to carry Images keeps the plain string/nil content path. Pins the invariant with TestMapMessageNonUserRolesNeverCarryImages, asserting assistant/tool/system messages with Images set still serialize plain content (no content-parts array). * Deep-copy image buffers so attachments are never aliased Add a shared CloneImageBlocks helper in zeroruntime that deep-copies a slice of ImageBlock including each Data byte slice (returning nil for nil/empty input). Use it in SeedMessagesWithImages so a seeded user turn owns an independent copy of the caller's image bytes, and in the agent loop's copyMessages so image bytes are deep-copied alongside ToolCalls instead of staying aliased across history/request/result copies. Tests assert that mutating the caller's original Data bytes after seeding leaves the seeded message unchanged, and that copyMessages produces independent image bytes. * Accept image-only stream-json turns and bound base64 image size Resolve stream-json images before the prompt in the exec input path and tolerate an empty prompt when images are present, so an image-only message event (empty content + one image) proceeds with prompt="" and the image reaches the agent run instead of being rejected. A turn carrying neither text nor images still errors. In ResolveImages, reject an oversized payload from its ENCODED length (DecodedLen) BEFORE the base64 decode so a huge blob never allocates a decode buffer just to be capped afterward; the post-decode length check stays as a backstop. Tests cover the image-only exec turn reaching the agent and the pre-decode oversize rejection (asserting the size gate fires before any decode). * Bound the image read in imageinput.LoadFile with a LimitReader Replace os.ReadFile with os.Open + io.ReadAll over an io.LimitReader(f, MaxImageBytes+1). The os.Stat pre-check is only a fast-path hint (a non-regular file reports a misleading size and the file can grow between stat and read); the LimitReader is the real bound, so at most one byte past the cap is ever buffered and an oversized or unbounded source can never allocate a multi-gigabyte buffer. The file is closed via defer and the existing size message is reused. * Fix RenderChat frame-height off-by-one with image chips cmdRegion emits TWO rows when ImageChips is set (a chip row above the input line), but the body/overlay height accounting assumed a one-row command region, overflowing the fixed frame by one line. Compute the fixed-row count from the actual command-region height (1 or 2 rows) so the composed frame stays exactly Height rows in both cases. A RenderChat test asserts the rendered frame line-count equals the requested height both with and without ImageChips set. * Reject an empty inline --image= value Route the inline --image= form through requiredInlineFlagValue, the same empty-rejection the other inline flags use, so --image= errors with "--image requires a value" instead of appending an empty image path. A parse test covers the empty inline value. * imageinput: reject non-regular files (FIFO/dir) before opening --------- Co-authored-by: KRATOS <[email protected]> Co-authored-by: Gnanam <[email protected]>
50 lines
1.5 KiB
Go
50 lines
1.5 KiB
Go
package cli
|
|
|
|
import (
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
func TestParseExecImageFlagRepeatable(t *testing.T) {
|
|
options, help, err := parseExecArgs([]string{
|
|
"--image", "one.png",
|
|
"--image=two.jpg",
|
|
"describe these",
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("parseExecArgs returned error: %v", err)
|
|
}
|
|
if help {
|
|
t.Fatal("help = true, want false")
|
|
}
|
|
want := []string{"one.png", "two.jpg"}
|
|
if len(options.imagePaths) != len(want) {
|
|
t.Fatalf("imagePaths = %#v, want %#v", options.imagePaths, want)
|
|
}
|
|
for i := range want {
|
|
if options.imagePaths[i] != want[i] {
|
|
t.Fatalf("imagePaths[%d] = %q, want %q", i, options.imagePaths[i], want[i])
|
|
}
|
|
}
|
|
if strings.Join(options.promptParts, " ") != "describe these" {
|
|
t.Fatalf("promptParts = %#v", options.promptParts)
|
|
}
|
|
}
|
|
|
|
func TestParseExecImageFlagRequiresValue(t *testing.T) {
|
|
if _, _, err := parseExecArgs([]string{"--image"}); err == nil ||
|
|
!strings.Contains(err.Error(), "--image requires a value") {
|
|
t.Fatalf("expected --image requires a value error, got %v", err)
|
|
}
|
|
}
|
|
|
|
// TestParseExecImageInlineFlagRejectsEmpty locks that the inline `--image=` form
|
|
// rejects an empty value (the same empty-rejection the other inline flags use)
|
|
// instead of appending an empty image path.
|
|
func TestParseExecImageInlineFlagRejectsEmpty(t *testing.T) {
|
|
if _, _, err := parseExecArgs([]string{"--image=", "prompt"}); err == nil ||
|
|
!strings.Contains(err.Error(), "--image requires a value") {
|
|
t.Fatalf("expected --image requires a value error for empty inline value, got %v", err)
|
|
}
|
|
}
|