Compare commits

..
Author SHA1 Message Date
JATMNandAyush7614 88ec97d563 fix(tools): publish file writes atomically instead of truncating in place (#1214)
* fix(tools): publish file writes atomically instead of truncating in place

Stage replacements before publication so failed writes preserve existing
contents, successful renames detach hard-link siblings, and creates publish
without replacing raced files. Preserve modes and contained symlinks, with
the existing validated in-place fallback when replacement is refused.

Rebuild Ayush7614's complete change from Twigpine/zero#1159 on current main.
The net patch is unchanged; original authorship is retained.

Original-commit: d97d1cf7ae
Original-commit: 9b994c1903
Original-commit: 48f65ca634
Original-PR: https://github.com/Twigpine/zero/pull/1159
Fixes: #1156

* fix(tools): preserve atomic publication when replacement is refused

Keep regular-file preimages and hard-link siblings intact after rename errors. Record published creates even when staging cleanup fails while preserving the cleanup error. Make the permission regression fixture independent of umask.

---------

Co-authored-by: Ayush7614 <[email protected]>
2026-11-09 12:25:24 -08:00
Vasanth T b32f7ce4d6 fix(agent): stop a denied tool looping past the repeated-failure halt (#866)
* fix(agent): stop a denied tool looping past the repeated-failure halt

The repeated-failure guard keys its streak on the first 80 characters of the
error text. A permission denial reads "Error: Permission denied for <tool>:
<reason>", and reason names the path or command that was refused, so the text
differs on every call while describing the same unchanging refusal. Each call
therefore rebuilt the record at count 1 and toolFailureStopAt was never
reached.

Not hypothetical. A headless run made 384 denied calls over 26 minutes under a
halt set to 6, produced no files, and reported nothing. #702 already hit this
shape once and fixed it by making one error message id-invariant; that works
per message and needs every future message to remember. Denials now key on
their DenialCategory instead, which is a small closed enum the loop already
sets on the result, so the class is fixed rather than one instance of it.

Adds a second, content-blind counter beside the streak. The signature-keyed
one cannot by construction see a tool that fails with a genuinely different
error every time, and that is still a tool that is not working. It counts
consecutive failures regardless of the error and is cleared only by a success
of that same tool, so changing how a tool fails is not progress and neither is
some other tool succeeding. It stops at 12 rather than 6 on purpose: a model
iterating on a tricky edit legitimately fails a few times with different errors
while converging, which is the same reasoning that moved toolFailureStopAt from
4 to 6.

Two counters, tripping on either, is what both of the agent CLIs I compared
against arrived at independently after hitting this bug — a tight bound on
identical failures ORed with a looser bound that no amount of varying the error
can reset.

Every guard is mutation-checked. Reverting the denial re-key fails
TestPermissionDenialStreakSurvivesVaryingReasonText; deleting the content-blind
bound, or letting a signature change reset it, each fail
TestToolFailingWithDifferentErrorsEveryTimeStillStops and
TestSuccessResetsBothFailureCounters.

One existing test call site gains the new parameter.

* fix(agent): count denials as failures and report the bound that tripped

Addresses both blocking findings from @anandh8x's review. He was right on
both, and the first was fatal: the previous commit was a no-op in production.

loop.go passed isRetriableToolError as the guard's `failed` flag, and that
returns false for any categorized denial (a policy refusal is deliberately not
retriable). observeToolResult therefore took its success branch and DELETED the
record before it could key on DenialReason, so a denied tool still looped to the
turn limit. The re-key was correct and unreachable.

The flag is now split. `failed` counts a denial toward the streaks; `hintable`
stays retriable-only, because a schema hint is the wrong response to a refusal —
the call shape is fine, the answer was no. Collapsing the two is what made a
caller unable to express "count this but do not coach the model about it".

Second finding: outcome.Count returned the signature-keyed record.count even
when the content-blind counter was what tripped the stop. With twelve distinct
errors that count is 1, so the final answer told the user a tool "failed 1 time
in a row with the same error". The outcome now carries the counter that actually
fired plus a Varied flag, and the stop answer says "each with a different error"
in that case.

Every earlier test passed while the production path was broken, because they
called observeToolResult directly with failed=true. So the important addition
here is TestRunStopsARepeatedlyDeniedToolAtTheFailureBound, which drives Run
itself: a tool that always prompts, an approver that always denies, and a
different command per turn so the denial reason varies as it does in a real run.

Verified by reverting the fix: the run makes 10 denied calls instead of halting
at 6 and dies on the no-output guard 13 turns later, while the helper-level test
stays green — which is precisely why this shipped in the first place.

* fix(agent): count uncategorized policy refusals in the halt guard

Reported by jatmn, and he is right that the guard missed the paths it most
needed to cover.

countedFailure asked `DenialReason != DenialNone`, but a category is only
attached where a TYPED denial is built. A headless run leaves
OnPermissionRequest nil, so the loop never reaches that branch and the registry
returns a bare `Error: Permission required ...` with no category. A sandbox
preflight denial on a non-shell tool loses its SandboxDecision converting to
ToolResult and arrives as an uncategorized `Sandbox block`. isRetriableToolError
rejects both, so both operands were false, observeToolResult took its success
branch, and the record the guard accumulates was cleared. The same refused call
could then repeat to MaxTurns, which is the loop this PR exists to stop.

The text patterns for those outcomes already existed, enumerated inside
isRetriableToolError. They simply were not reachable from the counting question.
They are now a shared isPolicyRefusal predicate that both callers use, so the
two questions cannot drift apart again, which is how they diverged in the first
place.

Also, denials no longer feed the execution-profile failure-streak trigger. That
trigger restores the displaced turn budget and reasoning effort on the theory
that a tool is struggling and needs room. A policy refusal is not a struggling
tool, it is an answer, and spending the one-shot escalation on one contradicts
the trigger's documented retriable-failure contract. Denials still count for the
halt; they just no longer buy more budget.

On coverage, honestly: the new tests pin the PREDICATE, including both
uncategorized shapes, and I verified by mutation that removing the text branch
fails them. They do NOT pin the wiring. Mutating countedFailure leaves them
green, which is the same unit-versus-call-path gap that produced the original
defect here. A Run-level test through the headless path is what would close it
and this commit does not add one.

* fix(agent): stop the halt answer claiming a pattern the counters do not track

jatmn's P3. The final answer overclaimed in both directions.

The content-blind bound said "each with a different error". anyErrorCount only
establishes that the tool failed consecutively without a success. Five A, five B
and two C reaches 12 without any signature repeating six times, and three of
those errors were shared, so "each different" is false. It now says "varying
errors", which is what reaching 12 without tripping the signature bound actually
proves: no signature repeated six times in a row.

The signature bound said "with the same error", which is false the other way for
a denial streak. A denial keys on its CATEGORY precisely because the prose
embeds the path or command refused and therefore differs on every call. That
streak now reports as refused rather than as one repeated error, carried on a
Refused flag derived from the signature prefix.

The one claim that IS justified is kept: an error-signature streak really did
repeat the same signature, so that wording stands.

The existing denial test asserted the old "same error" phrasing, so it was
describing the very defect this fixes; it now expects the refusal wording.

Tests: the three wordings against their counters, plus the mixed-signature
regression jatmn asked for, driven through the real counter rather than asserted
about the strings in isolation, so it proves the 5/5/2 run trips the
content-blind bound and is not described as all-different.

* fix(agent): do not classify successful tool output as a policy refusal

isPolicyRefusal decides on denial category, then permission metadata,
then output text. None of those questions is meaningful about a call the
tool completed, and the last one is answered by content the model does
not control.

isRetriableToolError gated on StatusError before calling in, so the
boundary held while that was the only caller. Extracting the helper and
calling it from the counting path dropped the gate: an allowed bash
printing "Sandbox block", or a read_file returning a document that quotes
it, set policyRefusal, made countedFailure true, and recorded a failure
against the tool's signature. Six such successes tripped the
same-signature stop and ended a healthy run with "the `bash` tool failed
6 times in a row with the same error".

The gate belongs in the classifier rather than at each caller, because
the next caller will forget it too.

Covered by a direct StatusOK classifier case over every signal the helper
reads, and by an end-to-end run of ten successful greps whose output
quotes the phrase. Both fail against the ungated helper: the run halts at
6 with the refusal answer above.

* test(agent): drive the uncategorized refusal paths through Run

The categorized denial was never the gap. The gap is a refusal arriving
with DenialReason empty, because a category is attached only where a
typed denial is built: a headless run leaves OnPermissionRequest nil and
the registry gate returns a bare "Permission required for ...", and a
sandbox preflight denial on a non-shell tool loses its SandboxDecision
converting to ToolResult and arrives as a bare "Sandbox block".

Testing that through the helper proves nothing. The first version of this
fix passed every helper test while being a no-op in production, because
the loop asked a different question than the tests did. Both cases here
go through Run and pin what the loop does with the classification: halt
at the bound, never execute the tool, and withhold the profile's one-shot
failure escalation.

Each half of the wiring falsifies the tests on its own. Dropping
policyRefusal from countedFailure lets the headless refusal run 13 turns
instead of halting at 6. Dropping the empty-outcome branch for the
posture controller reports posture_escalations = 1 instead of 0.

* fix(agent): classify a policy refusal from provenance, never from tool output

isPolicyRefusal fell back to matching phrases in the model-visible output, and
output is tool-controlled. bash preserves arbitrary stdout and stderr on a
StatusError for any nonzero exit, so an allowed command running
`printf 'Sandbox block\n' >&2; exit 1` had actually executed, carried no denial
category, and was still classified as refused. read_file returning a document
that quotes one of the phrases did the same, which is the likelier way a real
session hits it. The loop then withheld the retry hint and the profile
failure-streak recovery and accumulated the executed failure toward the refusal
halt, so a later stop told the user a tool had been refused when it had run.

The registry already had the structured facts and dropped them on the floor.
Every path that returns BEFORE the tool runs now carries one marker naming which
gate refused: sandbox deny, sandbox approval required, permission required,
permission denied. That includes the two cases that were previously
uncategorized, the headless prompt refusal and the sandbox preflight denial on a
non-shell tool, neither of which builds a typed DenialReason. isPolicyRefusal
reads DenialReason, permission metadata and that marker, and nothing else.

markStructuredSandboxDenial already stated this rule for the sandbox adapter,
"Classification is never inferred from stdout or stderr"; this carries the same
guarantee across the remaining gates.

Coverage runs in both directions. The existing refusal fixtures now carry the
provenance their production paths attach rather than relying on their text, and
there is a Run-level regression where an allowed tool that ran, failed, and
printed each recognized phrase still receives the retry hint. Reverting the
classifier to substrings fails all three tests, including every phrase of the
Run-level one.

* fix(tools): give capture_artifact's configuration refusals the same provenance as the gates

capture_artifact rejects in RejectBeforePermission, which the registry returns
straight back before any of the gates that attach provenance. Its
valid-but-unavailable calls therefore reached the classifier with no denial
category, no permission metadata and no refusal marker, so they were read as
ordinary retriable failures: the model got the schema hint telling it to fix
arguments that were already valid, and the call could consume the profile
failure-streak escalation, for a tool that never executed and that no argument
change can enable.

PolicyRefusalToolNotEnabled existed for exactly this and I never wired it. The
missing-artifact-directory and disabled-driver branches carry it now.

The malformed-argument branch deliberately stays an ordinary error. That one IS
fixable by trying again differently, which is what the hint is for, so marking
every early rejection would trade one wrong answer for another. Both directions
are covered.

Checked the rest of the class rather than only the reported tool: web_fetch,
browser_launch, browser_connect, browser_open, desktop_windows,
desktop_snapshot and terminal_session all reject on arguments alone, which is
correctly retriable. capture_artifact was the only one refusing on
configuration.

Also rebased onto current main rather than carrying the two merge commits, per
the same requirement raised on #886.

* fix(agent): give a marker-only refusal the same identity the guard keys on

The registry marks its pre-execution refusals in metadata, and isPolicyRefusal
read that marker while observeToolResult keyed on DenialReason, which those
paths leave empty. The guard fell back to errorSignature(output), so two
refusals of the same category with different wording looked like two different
failures and the streak restarted at 1 on every call.

A model alternating capture_artifact's browser_screenshot and browser_pdf
against a disabled driver is refused identically each time, and never tripped
the six-call refusal halt. Only the generic twelve-error fallback stopped the
run, reporting varied errors rather than a repeated refusal.

The category is derived once now, at the boundary where a tools.Result becomes a
ToolResult, and both the classification and the streak read that one value. It
had to go in twice, because a RejectBeforePermission refusal takes its own
constructor, and that is the route capture_artifact actually takes. Deriving it
at the producers instead would have left the same gap for the next path that
returns before the gates.

One behaviour change worth stating plainly rather than burying. A headless
prompt refusal is marked too, so it now carries a category and the stop answer
says the tool was refused rather than that it failed with the same error. The
bound is unchanged, and the new wording is the accurate one: the tool never ran.
The test that pinned the old wording is updated, along with the comment that
explained why it was uncategorized.

* fix(agent): type the refusal identity and carry the halt to terminal status

Three defects with one shape: provenance was encoded in a string, recovered by
inspecting that string, and then not carried far enough.

The same-identity streak keyed on one string namespace holding both a
normalized error signature and a synthetic "denial:<category>" key, with
provenance recovered afterwards by testing for that prefix. Trusted provenance
and untrusted content in one namespace is a namespace the untrusted side can
write into: a command printing exactly "denial:permission_denied" and exiting
non-zero acquired the identity of a real permission refusal and the run
reported it as refused although it executed every time. The identity is now a
(kind, key) pair, so no output can spell a refusal.

The content-blind bound reported only that the failures varied, and it fires
precisely when no identity repeated, so the identity present at the end says
nothing about the eleven before it. Alternating two refusal categories reached
twelve without either reaching six and the answer described a tool that never
ran as having failed. The record now carries the aggregate, and the guard
returns a typed cause instead of two overlapping booleans, with the mixed case
named rather than left to whichever field a switch tested first.

The halt returned straight out of the tool loop, so it never crossed the
completion gate the max-turns paths go through. Under RequireCompletionSignal
zero exec treats only Incomplete as exit 4, so a task denied six times came
back as a successful automation result having done none of the work.

Also closes two coverage gaps that made the markers untestable: reverting
either the sandbox deny marker in registry.go or the disabled-driver refusal in
local_capture.go left both suites green. The sandbox case now runs a real
engine evaluation through Run and asserts the body was skipped; the capture
case configures an artifact root with one driver enabled so it reaches the
disabled-driver branch instead of returning at the missing-directory one, with
a sibling case pinning that a malformed argument stays retriable.

* fix(agent): close out advertised tool calls by what actually happened

Parallel read-ahead broke the assumption the aborted placeholders were written
under. executeParallelReadBatch runs an entire eligible run of read calls before
the loop consumes any of them, so "not consumed yet" stopped meaning "not
executed". Every terminal branch closed out the calls after the current index as
aborted, and a sibling that had already run was recorded that way: its real
result discarded, and its callbacks, trace counter, task observation, loaded
tools and images lost with it. Where the sibling is a successful read,
execution may already have committed file-observation credit for content the
model never receives, so the authorization state disagreed with the transcript.

Each remaining call is now put in the state that is true of it. A completed one
is finalized exactly once with the same bookkeeping the main path performs; an
unstarted one still gets a placeholder so every tool_use keeps its answering
tool_result. The guard is deliberately not consulted for a drained sibling: it
cannot reverse a decision already made, it is only owed an honest record.

All three early returns go through one helper rather than repeating the
assumption, so the next stop condition inherits the fix instead of the bug.

* fix(agent): authenticate refusal provenance and separate completion from abort

Two things this branch left keyed on something that could disagree with the
fact it stands for.

A tool that ran and failed could claim the registry refused it before it ran.
IsPolicyRefusalResult trusted a metadata key, and Registry.RunWithOptions
forwards an executed result and its Meta unchanged, so a tool could set it by
mistake, by copying metadata forward from something it called, or on purpose.
The loop then withheld the retry hint, suppressed the failure-streak recovery,
counted the call in refusal accounting, and could tell the user a tool was
refused when it had executed. That is the output-text trust problem one layer
down. The execution boundary now strips the marker, so no value survives
running, recognized or invented. Pre-execution refusals are untouched, including
RejectBeforePermission, which decides before any of this.

precomputedResultFor treated any batch entry carrying an abort error as
unstarted. Producing a result and asking the run to stop are different facts,
and executeToolCall's cancelled-permission path returns both: an earlier
sibling reaching a terminal branch would discard the real cancellation result
and write an aborted placeholder over it. The batch now records what it ran,
where that is known, and the placeholder is reserved for entries that produced
nothing. The terminal decision is unchanged; draining only makes the record
honest.

* test(agent): pin a drained read-ahead sibling's image through the guard halt

The halt drains read-ahead siblings that already ran and records them as completed, images included. Images now reach the transcript through the turn's deferred builder, and nothing checked that a drained sibling's image still went through it: dropping it left every test green. The probe tool returns an image from the sibling and the run must carry exactly one image message.

* chore: follow the Twigpine module path

main moved the module to github.com/Twigpine/zero in #1105. Rename this
branch's imports to match, including the two in internal/workspaceindex
that main itself still carries until #1153 lands.

* chore(agent): drop two unused zero-value constants

failureKindNone and toolFailureCauseNone were never referenced, so the pinned unused check flagged them. The zero values stay reserved with a blank identifier.

* fix(agent): report mixed guard halts as mixed and pair drained siblings

- toolFailureIncompleteReason gains the mixed branch, so a headless run
  halted by the content-blind bound over failures and refusals says the
  tool failed or was refused, as the stop answer does.
- closeOutRemaining sets ToolCallProviderID on a drained sibling's tool
  result, matching the consumption path, so a strict replay can pair it.
- executedToolResult strips permission_action as well as the refusal
  marker. The loop reads a denied permission action as a refusal, so an
  executed failure carrying one was counted toward the refusal halt. The
  key is a tools constant now, and the loop uses it too.
- The parallel read-ahead comment says what differs from sequential
  execution: a halt can leave up to seven sibling reads already run.

Reported by @jatmn.
2026-11-09 08:02:41 -08:00
Vasanth T 8e9c3ec4ce fix(mcp): scrub the injected AI/ML API key from server responses (#1197)
* fix(mcp): scrub the injected AI/ML API key from server responses

Zero injects the user's AI/ML API key into requests to mcp.aimlapi.com
(#1193) and keeps it out of config, but only an HTTP error status left
the server's text out. A JSON-RPC error message, the method of an
unexpected message, a tool result (MCP reports a failed tool call as a
result with isError, not as an error) and a legacy SSE endpoint that
fails to parse all reached the caller as the server wrote them, so a
server that echoed the key put it in errors, in tool output sent to the
model, and in anything that prints them.

A response to a request that carried the injected key now has the key
replaced with [REDACTED] in its error text, its method and every string
of its result before anything reads it, on streamable HTTP and on the
legacy SSE transport. The SSE client scrubs both the key its stream
request carried and the one on the POST, since the key is resolved per
request. A user Authorization header, OAuth and project-configured
servers are unchanged, since nothing was injected.

The scrub uses a new redaction.RedactSecretValues: the split-aware
known-value pass RedactString applies to ExtraSecretValues, without the
shape patterns, which could otherwise rewrite tool output or base64
payloads that never held the key. A result without the key keeps its
original bytes.

* fix(mcp): scrub every key a call sent, and transport errors too

Review found two gaps. The key is resolved again for each redirect hop,
but the scrub used only the final hop's key, so a same-origin server
answering the last hop could echo an earlier hop's key. And a legacy SSE
POST that failed at the transport quoted the endpoint URL the server
chose, which can hold the stream's key, with no scrub at all.

A collector in the request context now records every key
applyAIMLAPIRequestHeaders injects during a call. net/http gives each
redirect hop the first request's context, so a later hop's key is
recorded too. Both transports scrub the response and every error the
call returns with those keys, the stream request's own errors included,
and the SSE client adds the keys its stream request sent. A scrubbed
error unwraps only to the context sentinel the original matched, so a
cancellation still reads as one without the chain that held the key.

Reported by @jatmn.
2026-11-09 07:49:09 -08:00
github-actions[bot] ef1ca39b09 chore(main): release 0.13.0 (#1190)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-09 14:20:51 +08:00
Victor Lowther bbf779e691 fix(tui): make reasoning stream rendering linear (coalesced deltas, O(1) append, tail-bounded preview) (#1198)
* feat(tui): coalesce reasoning deltas with kind-switch flush

Extend the stream coalescer to batch agentReasoningMsg deltas alongside
assistant-text at ~60fps, flushing on kind switches so interleaved
prose and reasoning never reorder. Fixes the O(n^2) re-render
pathology where each thinking delta re-rendered the whole accumulated
block.

* perf(tui): make reasoning stream append and preview rendering O(1) per delta

streamingReasoning accumulated with string += delta, the same O(n^2)
append pathology that streamingText had already been migrated away
from for. Switch it to []byte so each reasoning delta is an amortized
O(1) append, read through streamingReasoningString().

The live preview also paid per-frame costs on the growing string:
reasoningPreviewLines split the entire accumulated reasoning to show
the last two lines, and previewTail rune-converted whole lines. Both
now walk backwards over only the tail they render, so preview cost is
bounded by the visible width instead of segment length.

* fix(tui): clear fired coalescer timer so later deltas re-arm and never stall

* fix(tui): normalize invalid UTF-8 in truncated reasoning preview tail
2026-11-09 13:29:28 +08:00
Kevin Codex 47f78c6624 feat(providers): point OpenGateway at api.twigpine.com and trust its live model list (#1208)
Rename the recommended gateway provider to "Twigpine OpenGateway" and move
it to https://api.twigpine.com/v1. The catalog id (gitlawb-opengateway) and
GITLAWB_OPENGATEWAY_API_KEY stay, and "twigpine" / "twigpine opengateway"
are added as aliases.

- Default model is now xiaomi/mimo-v2.6-pro, an exact id from the live
  /v1/models list, so onboarding keeps it selected after live discovery
  (an alias default made setup jump to the first sorted model).
- The live models URL is https://api.twigpine.com/v1/models. Entries
  marked status "retired" are skipped in both the catalog parse and the
  live /models probe.
- The curated list is reduced to an offline fallback (auto,
  xiaomi/mimo-v2.6-pro, xiaomi/mimo-v2.6-flash). The live list replaces it
  whenever the gateway is reachable.
- Profiles saved with the old https://opengateway.gitlawb.com/v1 base URL
  keep resolving: the descriptor lists it in LegacyBaseURLs, and the
  resolver and picker treat it as the catalog endpoint, so the catalog env
  var binding is kept.
2026-11-09 11:56:14 +08:00
56 changed files with 5106 additions and 219 deletions

No files matched your search

+1 -1
View File
@@ -1 +1 @@
{".":"0.12.0"}
{".":"0.13.0"}
+22
View File
@@ -6,6 +6,28 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once the first release is
tagged. Until then, source builds report the version `dev`.
## [0.13.0](https://github.com/Twigpine/zero/compare/v0.12.0...v0.13.0) (2026-11-09)
### Features
* **mcp:** add AI/ML API as an opt-in built-in server ([#1193](https://github.com/Twigpine/zero/issues/1193)) ([16ccd78](https://github.com/Twigpine/zero/commit/16ccd78765b6a2a56b7370e3031710d718aa4757))
* **plugins:** add CLI enable/disable for plugin.json ([#1202](https://github.com/Twigpine/zero/issues/1202)) ([05c1b5e](https://github.com/Twigpine/zero/commit/05c1b5e16098e1c60212316d20ae8e055a27b5ae))
* **providers:** point OpenGateway at api.twigpine.com and trust its live model list ([#1208](https://github.com/Twigpine/zero/issues/1208)) ([47f78c6](https://github.com/Twigpine/zero/commit/47f78c66247aa96c8753d6bb95e935126ff1761e))
* **web-fetch:** add an opt-in AI/ML API Toolbox scraping backend ([#1192](https://github.com/Twigpine/zero/issues/1192)) ([c9f1794](https://github.com/Twigpine/zero/commit/c9f179482a7f6d7111ee64fa9cb8ffa2517c2397))
* **web-search:** add AI/ML API Toolbox as an opt-in backend ([#1189](https://github.com/Twigpine/zero/issues/1189)) ([9873645](https://github.com/Twigpine/zero/commit/98736457bdac6b0272e032728a5485cf557780fe))
### Bug Fixes
* **hooks:** dispatch specialistStart/Stop lifecycle hooks ([#1204](https://github.com/Twigpine/zero/issues/1204)) ([2ca84cc](https://github.com/Twigpine/zero/commit/2ca84cc127d9369765004b72cf95049fecc6c28b))
* **tui:** make reasoning stream rendering linear (coalesced deltas, O(1) append, tail-bounded preview) ([#1198](https://github.com/Twigpine/zero/issues/1198)) ([bbf779e](https://github.com/Twigpine/zero/commit/bbf779e69168d62bb1103aefdd1b10e53433209e))
### Performance Improvements
* **tui:** bounded file read, LRU rendering cache, and size caps for full-file view ([#1203](https://github.com/Twigpine/zero/issues/1203)) ([e034a93](https://github.com/Twigpine/zero/commit/e034a93815629c68976dcdab7dcc08bbe9b03693))
## [0.12.0](https://github.com/Twigpine/zero/compare/v0.11.0...v0.12.0) (2026-11-02)
+4
View File
@@ -307,6 +307,10 @@ attribution headers `X-AIMLAPI-Source: agent/zero` and `X-AIMLAPI-Partner-ID`,
and gets the key under the rules above. Other MCP servers never receive these
headers or the key.
If the server echoes back the key Zero sent it, Zero replaces the key with
`[REDACTED]` in that server's error messages and tool results before anything
reads them, and reports an HTTP error status without the response body.
### As a server — expose Zero's tools to another agent
```bash
@@ -0,0 +1,143 @@
package agent
import (
"context"
"strconv"
"strings"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// A CONFIGURATION REFUSAL IS NOT A RETRIABLE FAILURE, EVEN WHEN IT ARRIVES
// EARLY.
//
// capture_artifact rejects in RejectBeforePermission, which the registry returns
// straight back before any of the gates that attach provenance. Its
// valid-but-unavailable calls therefore reached the classifier with no denial
// category, no permission metadata and no refusal marker, and were read as
// ordinary retriable failures: the model got the schema hint and the call could
// consume the profile failure-streak escalation, for a tool that never executed
// and that no argument change can enable.
func TestDisabledCaptureArtifactIsAPolicyRefusalNotARetriableFailure(t *testing.T) {
// No artifacts directory and no enabled driver: valid arguments, unavailable
// tool. This is the shape an operator produces by configuration alone.
registry := tools.NewRegistry()
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{}) {
registry.Register(tool)
}
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
"action": "browser_screenshot",
"name": "shot",
}, tools.RunOptions{PermissionGranted: true})
if result.Status != tools.StatusError {
t.Fatalf("SETUP INVALID: expected the disabled tool to refuse, got %s: %s", result.Status, result.Output)
}
if !tools.IsPolicyRefusalResult(result) {
t.Fatalf("the refusal carries no provenance, so nothing downstream can tell it from a failed command: %#v", result.Meta)
}
// Through the conversion the loop performs, which is where the marker has to
// survive to be worth anything.
converted := ToolResult{
Status: result.Status,
Output: result.Output,
Meta: result.Meta,
}
if !isPolicyRefusal(converted) {
t.Error("the marker did not survive into the agent-facing result")
}
if isRetriableToolError(converted) {
t.Error("a tool that never executed, and that no argument change can enable, was marked retriable: the model gets a schema hint telling it to fix arguments that were already valid")
}
}
// The malformed-argument branch must stay retriable. That one IS fixable by
// trying again differently, which is what the hint exists for, so marking every
// early rejection would trade one wrong answer for another.
func TestMalformedCaptureArtifactArgumentsStayRetriable(t *testing.T) {
registry := tools.NewRegistry()
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{}) {
registry.Register(tool)
}
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
"action": "not_a_real_action",
}, tools.RunOptions{PermissionGranted: true})
if result.Status != tools.StatusError {
t.Fatalf("SETUP INVALID: expected invalid arguments to fail, got %s", result.Status)
}
if tools.IsPolicyRefusalResult(result) {
t.Fatalf("a malformed-argument error was marked a policy refusal, so the model is denied the hint that would let it fix the call: %q", result.Output)
}
converted := ToolResult{Status: result.Status, Output: result.Output, Meta: result.Meta}
if !isRetriableToolError(converted) {
t.Error("invalid arguments should stay retriable")
}
}
// alwaysRefusingCaptureTool stands in for the disabled tool at Run level, so the
// loop consequence can be observed rather than inferred.
type alwaysRefusingCaptureTool struct{ ran int }
func (tool *alwaysRefusingCaptureTool) Name() string { return "capture_artifact" }
func (tool *alwaysRefusingCaptureTool) Description() string { return "test capture tool" }
func (tool *alwaysRefusingCaptureTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"action": {Type: "string"}},
Required: []string{"action"},
AdditionalProperties: false,
}
}
func (tool *alwaysRefusingCaptureTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "captures artifacts"}
}
func (tool *alwaysRefusingCaptureTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{}
}
func (tool *alwaysRefusingCaptureTool) RejectBeforePermission(map[string]any) (tools.Result, bool) {
return tools.Result{
Status: tools.StatusError,
Output: "Error: capture_artifact is disabled because no artifact directory is configured.",
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalToolNotEnabled},
}, true
}
// THE LOOP CONSEQUENCE. A refusal must not draw the retry hint, because the hint
// tells the model to fix arguments that were already valid and the tool will
// refuse identically next time.
func TestRunDoesNotHintARefusedCaptureArtifact(t *testing.T) {
tool := &alwaysRefusingCaptureTool{}
registry := tools.NewRegistry()
registry.Register(tool)
calls := toolFailureHintAt + 1
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
for i := range calls {
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "capture_artifact", `{"action":"browser_screenshot"}`))
}
turns = append(turns, textTurn("gave up on the screenshot"))
result, err := Run(context.Background(), "take a screenshot", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
})
if err != nil {
t.Fatal(err)
}
if tool.ran != 0 {
t.Fatalf("SETUP INVALID: the tool executed %d times; it must be refused before Run", tool.ran)
}
for _, message := range result.Messages {
if strings.Contains(message.Content, toolFailureHintMarker) {
t.Fatal("a refused, never-executed tool drew the retry hint, which tells the model to fix arguments that were already valid")
}
}
}
@@ -0,0 +1,112 @@
package agent
import (
"context"
"strconv"
"strings"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// alternatingRefusalCaptureTool refuses every call with the same CATEGORY and a
// different MESSAGE, which is what a real disabled driver does: the refusal
// names the action, and the model is free to alternate valid actions.
type alternatingRefusalCaptureTool struct{ ran int }
func (tool *alternatingRefusalCaptureTool) Name() string { return "capture_artifact" }
func (tool *alternatingRefusalCaptureTool) Description() string { return "test capture tool" }
func (tool *alternatingRefusalCaptureTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"action": {Type: "string"}},
Required: []string{"action"},
AdditionalProperties: false,
}
}
func (tool *alternatingRefusalCaptureTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "captures artifacts"}
}
func (tool *alternatingRefusalCaptureTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{}
}
func (tool *alternatingRefusalCaptureTool) RejectBeforePermission(args map[string]any) (tools.Result, bool) {
action, _ := args["action"].(string)
return tools.Result{
Status: tools.StatusError,
// Action-specific wording, same category. This is the whole point: the
// prose differs on every call while the refusal never changes.
Output: "Error: Local control driver for " + action + " is disabled.",
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalToolNotEnabled},
}, true
}
// A REFUSAL KEYS ON ITS CATEGORY, INCLUDING WHEN THE CATEGORY IS ONLY A MARKER.
//
// The registry marks pre-execution refusals in metadata, and isPolicyRefusal
// read that marker while observeToolResult keyed on DenialReason, which those
// paths leave empty. The guard fell back to errorSignature(output), so two
// refusals of the same category with different wording looked like two
// different failures and the streak restarted at 1 every call.
//
// A model alternating capture_artifact's browser_screenshot and browser_pdf
// against a disabled driver is refused identically each time and never tripped
// the six-call halt. Only the generic twelve-error fallback stopped the run,
// reporting varied errors rather than a repeated refusal.
func TestAlternatingRefusedActionsStillTripTheRefusalHalt(t *testing.T) {
tool := &alternatingRefusalCaptureTool{}
registry := tools.NewRegistry()
registry.Register(tool)
// More calls than the refusal halt but FEWER than the generic any-error
// fallback, so only the category-keyed streak can stop this.
calls := toolFailureAnyErrorStopAt - 1
if calls <= toolFailureStopAt {
t.Fatalf("SETUP INVALID: %d calls cannot distinguish the refusal halt from the generic fallback", calls)
}
actions := []string{"browser_screenshot", "browser_pdf"}
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
for i := range calls {
action := actions[i%len(actions)]
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "capture_artifact", `{"action":"`+action+`"}`))
}
turns = append(turns, textTurn("gave up"))
result, err := Run(context.Background(), "capture something", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
})
if err != nil {
t.Fatal(err)
}
if tool.ran != 0 {
t.Fatalf("SETUP INVALID: the tool executed %d times; it must be refused before Run", tool.ran)
}
refusals := 0
for _, message := range result.Messages {
if strings.Contains(message.Content, "is disabled") {
refusals++
}
}
if refusals > toolFailureStopAt {
t.Errorf("the run made %d refused calls; the six-call refusal halt never tripped because the streak re-keyed on each action's wording", refusals)
}
// And the halt has to read as a repeated refusal, not as varied errors.
stop := strings.ToLower(strings.Join(messageContents(result.Messages), "\n"))
if strings.Contains(stop, toolFailureHintMarker) {
t.Error("a refused, never-executed tool drew the retry hint")
}
}
func messageContents(messages []zeroruntime.Message) []string {
out := make([]string, 0, len(messages))
for _, message := range messages {
out = append(out, message.Content)
}
return out
}
@@ -0,0 +1,93 @@
package agent
import (
"context"
"testing"
"github.com/Twigpine/zero/internal/localcontrol"
"github.com/Twigpine/zero/internal/tools"
)
// THE DISABLED-DRIVER BRANCH, REACHED FOR REAL.
//
// RejectBeforePermission refuses on two separate conditions, and its neighbour
// test constructs empty options, so it returns at the FIRST one (no artifact
// directory configured) and the second is never evaluated. Reverting the
// disabled-driver branch in local_capture.go to a plain errorResult therefore
// left the whole agent and tools suites green.
//
// This is the shape an operator actually produces: an artifact root IS
// configured and one driver IS enabled, and the model asks for an action owned
// by a different, disabled driver. Without provenance that call is read as a
// retriable failure, so it collects a schema hint and can spend the profile's
// failure-streak escalation, for a tool no argument change can enable.
func TestCaptureArtifactDisabledDriverIsAPolicyRefusal(t *testing.T) {
registry := tools.NewRegistry()
options := tools.LocalControlArtifactOptions{
ArtifactsDir: t.TempDir(),
// Configured and enabled, so the tool as a whole is available and the
// missing-directory branch cannot fire.
Browser: localcontrol.BrowserOptions{Enabled: true, Driver: "test-browser", HelperPath: "browser-helper"},
// Terminal deliberately left disabled.
}
for _, tool := range tools.NewLocalControlArtifactTools(options) {
registry.Register(tool)
}
// An enabled driver's action must still be accepted, or the test below would
// pass for a tool that refuses everything.
if result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
"action": "browser_screenshot", "name": "shot",
}, tools.RunOptions{PermissionGranted: true}); tools.IsPolicyRefusalResult(result) {
t.Fatalf("SETUP INVALID: the enabled browser driver was refused as policy: %s", result.Output)
}
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
"action": "terminal_snapshot", "name": "snap", "session": "test-session",
}, tools.RunOptions{PermissionGranted: true})
if result.Status != tools.StatusError {
t.Fatalf("SETUP INVALID: the disabled terminal driver did not refuse: %s / %s", result.Status, result.Output)
}
if !tools.IsPolicyRefusalResult(result) {
t.Fatalf("a driver disabled by configuration refuses without provenance, so it reads as a retriable failure: %#v", result.Meta)
}
if got := result.Meta["policy_refusal"]; got != tools.PolicyRefusalToolNotEnabled {
t.Errorf("policy_refusal = %q, want %q", got, tools.PolicyRefusalToolNotEnabled)
}
if isRetriableToolError(ToolResult{Status: result.Status, Output: result.ModelOutput(), Meta: result.Meta}) {
t.Error("the refusal is still classified as retriable, so the model gets a schema hint for a configuration decision")
}
}
// AND THE EARLY REJECTIONS MUST NOT COLLAPSE INTO ONE ANSWER.
//
// RejectBeforePermission refuses on configuration; argument validation refuses
// on a fixable mistake, and it runs FIRST, which is how the disabled-driver
// branch stayed uncovered. A malformed call must stay an ordinary retriable
// error, because trying again differently is exactly the right response to it.
func TestCaptureArtifactMalformedArgumentsStayRetriable(t *testing.T) {
registry := tools.NewRegistry()
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{
ArtifactsDir: t.TempDir(),
Terminal: localcontrol.TerminalOptions{Enabled: true, Driver: "test-terminal", HelperPath: "terminal-helper"},
}) {
registry.Register(tool)
}
// The driver IS enabled, so nothing here is a configuration decision. The
// call is simply missing the session the action requires.
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
"action": "terminal_snapshot", "name": "snap",
}, tools.RunOptions{PermissionGranted: true})
if result.Status != tools.StatusError {
t.Fatalf("SETUP INVALID: the malformed call did not fail: %s", result.Output)
}
if tools.IsPolicyRefusalResult(result) {
t.Errorf("a fixable argument mistake was marked a policy refusal, so the model loses the schema hint that would fix it: %s", result.Output)
}
if !isRetriableToolError(ToolResult{Status: result.Status, Output: result.ModelOutput(), Meta: result.Meta}) {
t.Errorf("a fixable argument mistake is not retriable: %s", result.Output)
}
}
+192 -13
View File
@@ -55,6 +55,22 @@ const (
// error, so this only affects true same-error loops.
toolFailureStopAt = 6
// toolFailureAnyErrorStopAt halts a tool that keeps failing with DIFFERENT
// errors. The streak above cannot see that case by construction: a changed
// signature rebuilds the record at 1, so a tool whose error text varies every
// call never reaches toolFailureStopAt however long it loops.
//
// That is not hypothetical. A headless run made 384 denied calls over 26
// minutes without tripping a halt set to 6, because each denial named a
// different path. Keying denials on their category (see observeToolResult)
// fixes that case; this bound covers every error with no category to key on.
//
// Deliberately well above toolFailureStopAt. A model iterating on a genuinely
// tricky edit legitimately fails several times with different errors while it
// converges, and this must not cut those runs short — the same reasoning that
// moved toolFailureStopAt from 4 to 6. Both counters reset on success.
toolFailureAnyErrorStopAt = 12
// maxContinueNudges bounds how many times the headless completion gate
// (Options.RequireCompletionSignal) re-prompts a model that stopped without a
// tool call while work clearly remained. Once spent, the run finalizes as
@@ -324,15 +340,91 @@ func acceptanceVerificationNudge(objective string) string {
const toolFailureHintMarker = "kept failing with the same error"
type toolFailureRecord struct {
count int
errSig string
hintShown bool
count int
identity failureIdentity
// sawExecuted and sawRefused are the AGGREGATE over the whole streak, not
// the current identity. The content-blind bound needs them because it fires
// exactly when no identity repeated, so the last one is unrepresentative.
sawExecuted bool
sawRefused bool
hintShown bool
// anyErrorCount counts consecutive failures of this tool REGARDLESS of the
// error, and is cleared only by a success. count above restarts whenever the
// signature changes, which is exactly what a varying error message defeats;
// this one cannot be reset by changing the text.
anyErrorCount int
}
// failureKind separates a failure the TOOL produced from one POLICY produced.
//
// The two used to share one string namespace, with provenance recovered
// afterwards by testing that string for a "denial:" prefix. Trusted provenance
// and untrusted content in one namespace is a namespace the untrusted side can
// write into: a command that printed exactly "denial:permission_denied" and
// exited non-zero acquired the identity of a real permission refusal, merged
// its streak with one, and had the run report it as refused although it ran
// every time. The kind is carried as its own field so no output can spell it.
type failureKind uint8
const (
// The zero value: no failure recorded.
_ failureKind = iota
// failureKindExecuted: the tool ran and returned an error. The key is the
// normalized error signature, which is tool-controlled text.
failureKindExecuted
// failureKindRefused: policy refused the call before the tool ran. The key is
// the denial category, a small closed enum the loop sets.
failureKindRefused
)
// failureIdentity is what the same-identity streak counts. Comparing the pair
// means an executed error can never equal a refusal however it is spelled.
type failureIdentity struct {
kind failureKind
key string
}
// toolFailureCause is why the guard stopped, decided where the facts are rather
// than reconstructed by the caller from overlapping booleans.
type toolFailureCause uint8
const (
// The zero value: the guard did not stop.
_ toolFailureCause = iota
// toolFailureCauseSameError: the same executed error signature, six times.
toolFailureCauseSameError
// toolFailureCauseSameRefusal: the same denial category, six times. The prose
// differs on every call because it names what was refused, so this is NOT the
// same-error case.
toolFailureCauseSameRefusal
// toolFailureCauseVariedExecuted: the content-blind bound, and every failure
// was the tool executing and failing.
toolFailureCauseVariedExecuted
// toolFailureCauseVariedRefused: the content-blind bound, and every failure
// was a policy refusal. Alternating two categories reaches twelve without
// either reaching six, and the tool never ran once.
toolFailureCauseVariedRefused
// toolFailureCauseVariedMixed: the content-blind bound over both kinds.
// Named deliberately rather than left to whichever field a switch tested
// first.
toolFailureCauseVariedMixed
)
type toolFailureOutcome struct {
InjectHint bool
Stop bool
Count int
// Cause is set only when Stop is. It carries the aggregate fact about the
// sequence that tripped a bound, which the last record alone cannot supply:
// the content-blind bound is reached precisely when no single identity
// repeated enough, so the identity present at the end says nothing about the
// eleven before it.
Cause toolFailureCause
}
// Refused reports whether policy, rather than the tool, produced the failures.
func (cause toolFailureCause) Refused() bool {
return cause == toolFailureCauseSameRefusal || cause == toolFailureCauseVariedRefused
}
// errorSignature normalizes a tool error to a short, comparable signature so
@@ -358,12 +450,47 @@ func toolFailureHint(toolName, schemaJSON, errOutput string) string {
// toolFailureStopAnswer is the final answer when the repeated-failure guard halts
// a run.
func toolFailureStopAnswer(toolName string, count int) string {
return "Agent stopped: the `" + toolName + "` tool failed " + strconv.Itoa(count) +
" times in a row with the same error, so I halted instead of looping further. " +
func toolFailureStopAnswer(toolName string, count int, cause toolFailureCause) string {
// Each branch claims only what its counter established, and the mixed case is
// spelled out rather than decided by whichever field a switch happened to
// test first. The wording used to overclaim in both directions: the
// content-blind bound said every failure differed, which it does not track,
// and it described a run of policy refusals as the tool failing, when the
// tool had not run at all.
verb, tail := " tool failed ", " times in a row with the same error, "
switch cause {
case toolFailureCauseSameRefusal:
verb, tail = " tool was refused ", " times in a row, "
case toolFailureCauseVariedExecuted:
tail = " times in a row with varying errors, "
case toolFailureCauseVariedRefused:
verb, tail = " tool was refused ", " times in a row for different reasons, "
case toolFailureCauseVariedMixed:
verb, tail = " tool failed or was refused ", " times in a row, "
}
return "Agent stopped: the `" + toolName + "`" + verb + strconv.Itoa(count) +
tail + "so I halted instead of looping further. " +
"Please check the request or adjust the tool arguments."
}
// toolFailureIncompleteReason describes a guard halt for the headless
// completion gate. A guard halt is BY DEFINITION unfinished work: the run
// stopped because a tool would not stop failing, not because the task was done.
//
// It makes the same claim as toolFailureStopAnswer for the same halt, including
// the mixed case: a headless caller reads this reason and an interactive one
// reads the stop answer, and the two must not describe one halt differently.
// Reported by @jatmn.
func toolFailureIncompleteReason(toolName string, cause toolFailureCause) string {
switch {
case cause.Refused():
return "halted after `" + toolName + "` was refused repeatedly, without completing the task"
case cause == toolFailureCauseVariedMixed:
return "halted after `" + toolName + "` failed or was refused repeatedly, without completing the task"
}
return "halted after `" + toolName + "` failed repeatedly, without completing the task"
}
// The no-output stop answer is assembled from these fixed parts (only the turn
// count varies). IsNoProgressStop matches all three so a legitimate message that
// merely quotes the marker substring is not misclassified as a failed empty run.
@@ -472,28 +599,68 @@ func newGuardState() *guardState {
// observeToolResult tracks repeated identical failures of a tool. A successful
// result clears that tool's failure streak. Returns whether to inject a one-shot
// corrective hint and/or stop the run.
func (state *guardState) observeToolResult(name string, failed bool, output string) toolFailureOutcome {
// hintable is separate from failed on purpose. A categorized denial MUST count
// toward the streaks — that is the whole point of keying on the category — but a
// schema hint is the wrong response to a policy refusal: the call shape is fine,
// the answer was no. Collapsing the two is what made the earlier version of this
// fix a no-op, since the caller could only pass a flag that excluded denials.
func (state *guardState) observeToolResult(name string, failed bool, hintable bool, output string, denial DenialCategory) toolFailureOutcome {
if state.toolFailures == nil {
state.toolFailures = map[string]*toolFailureRecord{}
}
if !failed {
delete(state.toolFailures, name) // success resets the streak
delete(state.toolFailures, name) // success resets both counters
return toolFailureOutcome{}
}
sig := errorSignature(output)
// A denial keys on its CATEGORY, not its prose. The message embeds the path
// or command that was refused, so it differs on every call while describing
// the same unchanging refusal — which rebuilt the record at 1 each time and
// let a denied tool loop indefinitely under a halt set to 6. The category is
// a small closed enum the loop already sets on the result.
identity := failureIdentity{kind: failureKindExecuted, key: errorSignature(output)}
if denial != DenialNone {
identity = failureIdentity{kind: failureKindRefused, key: string(denial)}
}
record := state.toolFailures[name]
if record == nil || record.errSig != sig {
record = &toolFailureRecord{count: 1, errSig: sig}
if record == nil {
record = &toolFailureRecord{identity: identity}
state.toolFailures[name] = record
}
if record.identity != identity {
// A different failure restarts the same-identity streak but NOT the
// content-blind one: changing how a tool fails is not progress.
record.count = 1
record.identity = identity
record.hintShown = false
} else {
record.count++
}
record.anyErrorCount++
if identity.kind == failureKindRefused {
record.sawRefused = true
} else {
record.sawExecuted = true
}
outcome := toolFailureOutcome{Count: record.count}
if record.count >= toolFailureStopAt {
switch {
case record.count >= toolFailureStopAt:
outcome.Stop = true
outcome.Cause = toolFailureCauseSameError
if identity.kind == failureKindRefused {
outcome.Cause = toolFailureCauseSameRefusal
}
return outcome
case record.anyErrorCount >= toolFailureAnyErrorStopAt:
// Report the counter that actually tripped. record.count is the
// same-identity streak and is often 1 here, which would describe a tool
// that failed a dozen different ways as having failed once.
outcome.Stop = true
outcome.Count = record.anyErrorCount
outcome.Cause = record.variedCause()
return outcome
}
if record.count >= toolFailureHintAt && !record.hintShown {
if hintable && record.count >= toolFailureHintAt && !record.hintShown {
record.hintShown = true
outcome.InjectHint = true
}
@@ -603,3 +770,15 @@ func (state *guardState) planReminder(turn int) string {
return ""
}
// variedCause classifies a content-blind stop by what the whole streak held.
func (record *toolFailureRecord) variedCause() toolFailureCause {
switch {
case record.sawRefused && record.sawExecuted:
return toolFailureCauseVariedMixed
case record.sawRefused:
return toolFailureCauseVariedRefused
default:
return toolFailureCauseVariedExecuted
}
}
+220 -1
View File
@@ -2,6 +2,8 @@ package agent
import (
"context"
"path/filepath"
"strconv"
"strings"
"testing"
@@ -192,7 +194,7 @@ func TestUnknownExecSessionProbingTripsFailureHalt(t *testing.T) {
var state guardState
var stoppedAt int
for i := 1; i <= toolFailureStopAt; i++ {
out := state.observeToolResult(tools.WriteStdinToolName, true, tools.UnknownExecSessionError(i))
out := state.observeToolResult(tools.WriteStdinToolName, true, true, tools.UnknownExecSessionError(i), "")
if out.Stop {
stoppedAt = i
break
@@ -203,6 +205,223 @@ func TestUnknownExecSessionProbingTripsFailureHalt(t *testing.T) {
}
}
// A permission denial repeats forever when the streak is keyed on the error
// TEXT, because the denial message embeds the path or command that varies per
// call.
//
// Observed, not theorised: a headless run made 384 denied calls over 26 minutes
// without tripping a halt that stops at 6. Every denial carried the same typed
// category and a different reason string, so errorSignature differed each time
// and the record was rebuilt at count 1 on every call.
//
// TestUnknownExecSessionErrorSignatureIsIDInvariant fixed one message this way.
// Keying on the category fixes the class, without needing every future denial
// message to remember to be invariant.
func TestPermissionDenialStreakSurvivesVaryingReasonText(t *testing.T) {
var state guardState
stoppedAt := 0
for i := 1; i <= toolFailureStopAt; i++ {
// The shape the loop actually produces: same tool, same category, a
// different path every time.
output := "Error: Permission denied for write_file: cannot write " +
filepath.Join("C:", "ws", "pkg", "file"+strconv.Itoa(i)+".go")
out := state.observeToolResult("write_file", true, true, output, DenialPermissionDenied)
if out.Stop {
stoppedAt = i
break
}
}
if stoppedAt != toolFailureStopAt {
t.Fatalf("denials with varying reason text stopped at %d, want %d", stoppedAt, toolFailureStopAt)
}
}
// The content-blind bound. A tool failing over and over with genuinely
// DIFFERENT errors and no denial category is still a tool that is not working,
// and the same-signature streak can never see it.
//
// Bounded well above toolFailureStopAt on purpose: a model iterating on a
// tricky edit legitimately fails a few times with different errors while it
// converges, and cutting that short is the regression
// TestSuccessResetsBothFailureCounters guards.
func TestToolFailingWithDifferentErrorsEveryTimeStillStops(t *testing.T) {
var state guardState
stoppedAt := 0
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
out := state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), "")
if out.Stop {
stoppedAt = i
break
}
}
if stoppedAt != toolFailureAnyErrorStopAt {
t.Fatalf("a tool failing with a new error each call stopped at %d, want %d", stoppedAt, toolFailureAnyErrorStopAt)
}
}
// Neither counter may outlive a success, or a long run that fails occasionally
// and recovers would eventually halt for no reason. This is the property that
// keeps the content-blind bound safe to add.
func TestSuccessResetsBothFailureCounters(t *testing.T) {
var state guardState
for i := 1; i < toolFailureAnyErrorStopAt; i++ {
if out := state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), ""); out.Stop {
t.Fatalf("stopped at %d before the success that should reset it", i)
}
}
state.observeToolResult("bash", false, false, "ok", "")
// Same again from zero. Reaching the bound a second time proves the counter
// restarted rather than merely paused.
stoppedAt := 0
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
if out := state.observeToolResult("bash", true, true, "later failure "+strconv.Itoa(i), ""); out.Stop {
stoppedAt = i
break
}
}
if stoppedAt != toolFailureAnyErrorStopAt {
t.Fatalf("after a success the tool stopped at %d, want a full fresh %d", stoppedAt, toolFailureAnyErrorStopAt)
}
}
// Records are keyed per tool, so ANOTHER tool succeeding in between must not
// clear the failing tool's streak.
//
// This is the realistic shape of the run that motivated the fix: the model kept
// making progress elsewhere — reading files, updating its plan — while one tool
// was refused over and over. A reset keyed on "something succeeded" rather than
// "this tool succeeded" would make the halt unreachable in exactly the runs that
// need it.
func TestAnotherToolSucceedingDoesNotClearAFailingToolsStreak(t *testing.T) {
var state guardState
stoppedAt := 0
for i := 1; i <= toolFailureStopAt; i++ {
state.observeToolResult("read_file", false, false, "ok", "")
out := state.observeToolResult("write_file", true, false,
"Error: Permission denied for write_file: "+strconv.Itoa(i), DenialPermissionDenied)
if out.Stop {
stoppedAt = i
break
}
}
if stoppedAt != toolFailureStopAt {
t.Fatalf("denials interleaved with another tool's successes stopped at %d, want %d", stoppedAt, toolFailureStopAt)
}
}
// alwaysPromptingTool is never allowed to run: it exists so a Run-level test can
// drive real permission denials through the loop.
type alwaysPromptingTool struct{ ran int }
func (tool *alwaysPromptingTool) Name() string { return "bash" }
func (tool *alwaysPromptingTool) Description() string { return "test shell tool" }
func (tool *alwaysPromptingTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
Required: []string{"command"},
AdditionalProperties: false,
}
}
func (tool *alwaysPromptingTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs shell commands"}
}
func (tool *alwaysPromptingTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{Status: tools.StatusOK, Output: "should never run"}
}
// The regression for the gap this whole change existed to close, driven through
// Run rather than through the guard helper.
//
// The first version of this fix was a no-op in production and every one of its
// unit tests passed, because they called observeToolResult directly with
// failed=true. The loop passes isRetriableToolError, which returns false for a
// categorized denial, so the guard took its success branch and deleted the
// record before it could key on the category. A denied tool still looped to the
// turn limit.
//
// This drives the real path: a tool that always prompts, an approver that always
// denies, and a different command each turn so the denial REASON — and therefore
// the error text — varies exactly as it does in a real run.
func TestRunStopsARepeatedlyDeniedToolAtTheFailureBound(t *testing.T) {
tool := &alwaysPromptingTool{}
registry := tools.NewRegistry()
registry.Register(tool)
// Comfortably more turns than the bound, so reaching the bound is what stops
// the run rather than exhausting the provider or MaxTurns.
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
for i := range toolFailureStopAt + 4 {
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
`{"command":"touch /etc/file`+strconv.Itoa(i)+`"}`))
}
provider := &mockProvider{turns: turns}
denials := 0
result, err := Run(context.Background(), "do the thing", provider, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
OnPermissionRequest: func(_ context.Context, request PermissionRequest) (PermissionDecision, error) {
denials++
// The varying half. In production this is the refused path or command;
// here it is the command, which lands in the denial message the guard
// used to key on.
return PermissionDecision{
Action: PermissionDecisionDeny,
Reason: "refused " + request.ToolName + " call " + strconv.Itoa(denials),
}, nil
},
})
if err != nil {
t.Fatal(err)
}
if denials != toolFailureStopAt {
t.Errorf("the run made %d denied calls, want it halted at %d", denials, toolFailureStopAt)
}
if tool.ran != 0 {
t.Errorf("the denied tool executed %d times; the denial must precede execution", tool.ran)
}
want := toolFailureStopAnswer("bash", toolFailureStopAt, toolFailureCauseSameRefusal)
if result.FinalAnswer != want {
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
}
}
// The stop message must describe the bound that actually tripped. When the
// content-blind counter is what halts the run, the same-signature streak is
// usually 1, and reporting that would tell the user a tool failed once after it
// failed a dozen different ways.
func TestVariedFailureStopAnswerReportsTheRightCounter(t *testing.T) {
var state guardState
var outcome toolFailureOutcome
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
outcome = state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), "")
if outcome.Stop {
break
}
}
if !outcome.Stop {
t.Fatal("never stopped")
}
if outcome.Cause != toolFailureCauseVariedExecuted {
t.Errorf("Cause = %v for a stop driven by the content-blind counter over executed failures", outcome.Cause)
}
if outcome.Count != toolFailureAnyErrorStopAt {
t.Errorf("Count = %d, want the counter that tripped (%d)", outcome.Count, toolFailureAnyErrorStopAt)
}
answer := toolFailureStopAnswer("bash", outcome.Count, outcome.Cause)
if !strings.Contains(answer, "varying errors") {
t.Errorf("stop answer describes the wrong cause: %q", answer)
}
if strings.Contains(answer, "with the same error") {
t.Errorf("stop answer claims a same-error loop after distinct failures: %q", answer)
}
}
func TestGuardStateResetsToolOnlyStreakOnEmptyNonToolTurn(t *testing.T) {
var state guardState
toolOnly := zeroruntime.CollectedStream{
+211 -25
View File
@@ -719,6 +719,60 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// executed concurrently, consumed strictly in order below.
var precomputed []precomputedToolResult
precomputedStart, precomputedEnd := 0, 0
// closeOutRemaining terminates every advertised call from `from` onward
// TRUTHFULLY, and every early return in this loop goes through it.
//
// Read-ahead broke the assumption the abort placeholders were written
// under. executeParallelReadBatch runs an entire eligible run before the
// loop consumes any of it, so "not consumed yet" stopped meaning "not
// executed": a sibling can already have run, committed its side effects
// and any authorization credit they carry, and still sit past the index
// where a terminal branch fires. Recording it as aborted threw that result
// away, so the transcript disagreed with what had actually happened, and
// its callbacks, trace counter, task observation and images went with it.
//
// So each remaining call is put in the one state that is true of it:
// completed, and finalized exactly once with the same bookkeeping the main
// path performs, or unstarted, and aborted. The guard is deliberately NOT
// consulted for a drained sibling: these results cannot reverse a stop
// decision that has already been made, they are only owed an honest record.
closeOutRemaining := func(from int) {
for next := from; next < len(collected.ToolCalls); next++ {
sibling, ran := precomputedResultFor(precomputed, precomputedStart, precomputedEnd, next)
if !ran {
messages = appendAbortedToolResults(messages, collected.ToolCalls[next:next+1])
continue
}
nextCall := collected.ToolCalls[next]
if options.OnToolCall != nil {
options.OnToolCall(nextCall)
}
options.Trace.Counter(trace.CounterToolCalls, 1)
recordOutputBudgetTrace(options.Trace, sibling)
task.observe(taskStateEvent{kind: taskStateEventToolResult, arguments: nextCall.Arguments, toolResult: sibling})
if options.OnToolResult != nil {
options.OnToolResult(sibling)
}
for _, name := range sibling.LoadedTools {
loaded[name] = true
}
// The same fields the consumption path below records, provider call
// ID included: a strict replay pairs the result to its tool_use by
// that ID, and a finalized sibling without it was the one result in
// the turn that could not be paired. Reported by @jatmn.
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleTool,
Content: sibling.ModelOutput(),
ToolCallID: sibling.ToolCallID,
ToolCallProviderID: nextCall.ProviderCallID,
IsError: sibling.Status == tools.StatusError,
ChangedFiles: append([]string(nil), sibling.ChangedFiles...),
})
if len(sibling.Images) > 0 {
toolResultsWithImages = append(toolResultsWithImages, sibling)
}
}
}
for index, call := range collected.ToolCalls {
// When this call starts a consecutive run of >= 2 auto-allowed read-only
// calls, execute the whole run concurrently now (see parallel_tools.go).
@@ -783,13 +837,13 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
abortErr = ctx.Err()
}
if abortErr != nil {
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
closeOutRemaining(index + 1)
messages = append(messages, buildToolImageMessages()...)
result.Messages = copyMessages(messages)
return result, abortErr
}
if stopReason := stopReasonFromToolResult(toolResult); stopReason != "" {
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
closeOutRemaining(index + 1)
messages = append(messages, buildToolImageMessages()...)
result.FinalAnswer = toolResult.ModelOutput()
result.StopReason = stopReason
@@ -804,8 +858,29 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// aren't fixed by reformatting the call, so a "match this schema" hint
// would misdirect the model toward JSON shape or blocked behavior.
retriableFailure := isRetriableToolError(toolResult)
outcome := guards.observeToolResult(call.Name, retriableFailure, toolResult.ModelOutput())
posture.observeToolOutcome(outcome, toolResult)
// A categorized denial is NOT retriable — retrying it verbatim is
// pointless — but it is still a failure the streaks must count, or a
// refused tool loops until the turn limit. Passing retriableFailure for
// both is what let that happen: observeToolResult took its success
// branch and deleted the record before it could key on the category.
// isPolicyRefusal rather than DenialReason alone: the categories are
// only attached where a typed denial is built, so a headless prompt
// refusal and an uncategorized sandbox block were counted as SUCCESS
// and cleared the record they were supposed to accumulate.
policyRefusal := isPolicyRefusal(toolResult)
countedFailure := retriableFailure || policyRefusal
outcome := guards.observeToolResult(call.Name, countedFailure, retriableFailure, toolResult.ModelOutput(), toolResult.DenialReason)
// The profile's failure-streak trigger is for RETRIABLE failures: it
// restores the displaced turn budget and reasoning effort on the theory
// that the tool is struggling and needs room. A policy refusal is not a
// struggling tool, it is an answer, and spending the one-shot
// escalation on one contradicts that trigger's own contract. Denials
// still count for the halt above; they just do not buy more budget.
if policyRefusal {
posture.observeToolOutcome(toolFailureOutcome{}, toolResult)
} else {
posture.observeToolOutcome(outcome, toolResult)
}
if outcome.Stop {
// The assistant message advertised EVERY collected tool call, but
// the guard halts mid-turn so the calls after this one never run.
@@ -813,17 +888,22 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// every tool_use has a matching tool_result and the recorded
// messages stay valid for a strict provider replay (Anthropic
// rejects a tool_use with no answering tool_result).
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
closeOutRemaining(index + 1)
messages = append(messages, buildToolImageMessages()...)
result.FinalAnswer = toolFailureStopAnswer(call.Name, outcome.Count)
// Same headless contract as the empty-turn guard: halting on a
// repeated failure is an unfinished run, not success. Gate-off
// (interactive / --no-completion-gate) keeps the old behavior.
result.FinalAnswer = toolFailureStopAnswer(call.Name, outcome.Count, outcome.Cause)
result.Messages = copyMessages(messages)
// THE HALT HAS TO CROSS THE TERMINAL-STATUS BOUNDARY TOO.
//
// This branch returns straight out, so it never reached the
// completion gate that the max-turns paths below go through. Under
// RequireCompletionSignal `zero exec` treats only Incomplete as exit
// 4 and otherwise emits run_end("success", 0), so a task that was
// denied six times, or failed through the twelve-call bound, became
// a successful automation result having done none of the work.
if options.RequireCompletionSignal {
result.Incomplete = true
result.IncompleteReason = "the " + call.Name + " tool failed " + strconv.Itoa(outcome.Count) + " times in a row with the same error"
result.IncompleteReason = toolFailureIncompleteReason(call.Name, outcome.Cause)
}
result.Messages = copyMessages(messages)
return result, nil
}
if outcome.InjectHint && failureHint == "" {
@@ -1605,9 +1685,41 @@ func executeToolCall(ctx context.Context, registry *tools.Registry, call ToolCal
// the Run turn loop performs the actual provider switch. Empty for every
// ordinary tool result.
RequestedModel: result.Meta["escalate_to_model"],
// ONE REFUSAL IDENTITY, derived here so classification and streak
// accounting cannot disagree.
//
// The registry marks its pre-execution refusals in metadata, and
// isPolicyRefusal read that marker while observeToolResult keyed on
// DenialReason, which those paths leave empty. The guard therefore fell
// back to errorSignature(output), and two refusals of the same category
// with different wording looked like two different failures. A model
// alternating capture_artifact's browser_screenshot and browser_pdf against
// a disabled driver is refused identically each time, and never tripped the
// six-call refusal halt: only the generic twelve-error fallback stopped it,
// reporting varied errors rather than a repeated refusal.
DenialReason: denialCategoryForResult(result),
}, nil
}
// denialCategoryForResult gives every pre-execution refusal a stable category,
// whether it was built as a typed denial or only marked in metadata.
//
// The mapping lives here, at the one boundary tools.Result becomes ToolResult,
// rather than at each producer. A marker without a category is the shape that
// caused this: it classified as a refusal and keyed as an error signature.
func denialCategoryForResult(result tools.Result) DenialCategory {
switch result.Meta[tools.PolicyRefusalMeta] {
case tools.PolicyRefusalToolNotEnabled:
return DenialFiltered
case tools.PolicyRefusalPermissionDenied, tools.PolicyRefusalPermissionRequired:
return DenialPermissionDenied
case tools.PolicyRefusalSandboxDenied, tools.PolicyRefusalSandboxApproval:
return DenialSandboxBlock
default:
return DenialNone
}
}
const sandboxNamespaceLimitedReason = "sandbox output is limited to the sandbox PID namespace; host/global state requires approval"
func maybeRetryUnsandboxedAfterSandboxRestriction(ctx context.Context, registry *tools.Registry, call ToolCall, tool tools.Tool, args map[string]any, result tools.Result, permissionMode PermissionMode, options Options, progressCallback func(streamjson.Event)) (tools.Result, *ToolResult, bool, PermissionDecisionAction, string, []string, error) {
@@ -1938,6 +2050,10 @@ func toolResultFromPrePermissionReject(call ToolCall, result tools.Result) ToolR
Display: display,
LoadedTools: loadedToolsFromResult(meta),
RequestedModel: meta["escalate_to_model"],
// The SAME identity the executed path derives. This is the route a
// RejectBeforePermission refusal takes, so leaving it empty here is what
// made the marker classify as a refusal and key as an error signature.
DenialReason: denialCategoryForResult(tools.Result{Meta: meta}),
}
}
@@ -2109,21 +2225,64 @@ func isRetriableToolError(result ToolResult) bool {
// A categorized denial (filtered / permission / sandbox) is a policy decision,
// not a transient failure — never retry it. This is robust to message wording
// (the text checks below remain as a fallback for results lacking the field).
return !isPolicyRefusal(result)
}
// isPolicyRefusal reports a result the run refused on policy grounds: a
// permission gate, a filter, a sandbox preflight, or a hook.
//
// Split out of isRetriableToolError so the two questions cannot drift apart,
// which is exactly what they had done. The halt guard asked
// `DenialReason != DenialNone`, and a category is only attached on the paths
// that build a TYPED denial. A headless run leaves OnPermissionRequest nil, so
// the loop never reaches its typed-denial branch and the registry returns a bare
// `Error: Permission required ...` carrying no category. A sandbox preflight
// denial on a non-shell tool loses its SandboxDecision converting to ToolResult
// and arrives as an uncategorized `Sandbox block`.
//
// Both are refusals, and both were counted as SUCCESS, so observeToolResult
// cleared the record and the same refused call could repeat to MaxTurns. That is
// the loop this guard exists to stop. The text checks below already enumerated
// these outcomes for the retriable question; they simply were not reachable from
// the counting one.
func isPolicyRefusal(result ToolResult) bool {
// A REFUSAL IS A FAILURE. Everything below inspects metadata and then output
// text, and neither is meaningful on a result the tool completed.
//
// isRetriableToolError gates on this before it calls here, so the boundary
// held while that was the only caller. Extracting this helper and calling it
// straight from the counting path dropped the gate: an allowed `bash`
// printing "Sandbox block", or a read_file returning a document that quotes
// one of these phrases, was counted as a denial. Six such successes tripped
// the same-signature stop and ended a healthy run with a refusal answer.
//
// The gate belongs here rather than at each caller, because the next caller
// will forget it too.
if result.Status != tools.StatusError {
return false
}
if result.DenialReason != DenialNone {
return false
return true
}
if result.Meta["permission_action"] == string(PermissionActionDeny) {
return false
if result.Meta[tools.PermissionActionMeta] == string(PermissionActionDeny) {
return true
}
switch {
case strings.Contains(result.Output, "is not enabled for this run"),
strings.Contains(result.Output, "Permission denied for "),
strings.Contains(result.Output, "Permission required for "),
strings.Contains(result.Output, "Sandbox block"),
strings.Contains(result.Output, "Sandbox approval required for "):
return false
}
return true
// STRUCTURED ONLY. This used to fall back to matching phrases in Output
// ("Sandbox block", "Permission denied for ", and so on), which cannot work:
// Output is tool-controlled. `bash` preserves arbitrary stdout and stderr on
// a StatusError for any nonzero exit, so an ALLOWED command running
// `printf 'Sandbox block\n' >&2; exit 1` has actually executed, carries no
// DenialReason, and was still classified as refused. The loop then withheld
// the schema hint and the profile failure-streak recovery, counted the
// executed failure toward the refusal halt, and a later stop told the user a
// tool had been refused when it had run.
//
// read_file returning a document that quotes one of those phrases did the
// same thing, which is the more likely way a real session hits it.
//
// The registry now marks every path that returns BEFORE the tool runs, so
// the question is answerable from provenance. Nothing below reads Output.
return tools.IsPolicyRefusalResult(tools.Result{Meta: result.Meta})
}
// scrubInterceptedOutput mirrors the registry's scrubResultSecrets boundary for
@@ -2689,7 +2848,7 @@ func deniedPermissionResult(call ToolCall, reason string, requestEvent Permissio
Output: "Error: Permission denied for " + call.Name + ": " + reason,
DenialReason: denial,
Meta: map[string]string{
"permission_action": string(event.Action),
tools.PermissionActionMeta: string(event.Action),
},
}
}
@@ -2715,7 +2874,7 @@ func canceledPermissionResult(call ToolCall, reason string, requestEvent Permiss
Output: "Error: Permission approval cancelled for " + call.Name + ": " + reason,
DenialReason: DenialApprovalCanceled,
Meta: map[string]string{
"permission_action": string(event.Action),
tools.PermissionActionMeta: string(event.Action),
},
}
}
@@ -3569,6 +3728,33 @@ func toolResultImageMessage(result ToolResult, options Options) (zeroruntime.Mes
}, true
}
// precomputedResultFor reports the read-ahead result for call index, and whether
// that call actually produced one. A batch entry that never started is still
// unstarted as far as the transcript is concerned.
//
// PRODUCING A RESULT AND ASKING THE RUN TO STOP ARE DIFFERENT FACTS. Treating a
// non-nil abortErr as "unstarted" conflated them, and a cancelled permission
// request is exactly the case where they disagree: executeToolCall builds the
// cancellation result and returns it WITH ErrPermissionApprovalCanceled, so an
// earlier sibling reaching a terminal branch discarded a real result and wrote
// an aborted placeholder in its place. The transcript then denied that the call
// had completed permission handling, and the cancellation was missing from
// OnToolResult, the trace counters and the task observation even though the
// permission event had already happened.
//
// The terminal decision is not affected: draining such a sibling only records
// what occurred, it cannot reverse a stop already selected.
func precomputedResultFor(precomputed []precomputedToolResult, start, end, index int) (ToolResult, bool) {
if index < start || index >= end {
return ToolResult{}, false
}
entry := precomputed[index-start]
if !entry.completed {
return ToolResult{}, false
}
return entry.result, true
}
func modelAcceptsToolImages(options Options) bool {
if options.SupportsVision != nil {
return options.SupportsVision(options.Model)
+16 -4
View File
@@ -685,11 +685,23 @@ func TestIsRetriableToolError(t *testing.T) {
{"success", ToolResult{Status: tools.StatusOK}, false},
{"bad arguments", ToolResult{Status: tools.StatusError, Output: "Error: Failed to parse arguments for x: bad json"}, true},
{"execution failure", ToolResult{Status: tools.StatusError, Output: "Error: read foo.txt: no such file"}, true},
{"disabled tool", ToolResult{Status: tools.StatusError, Output: `Error: Tool "x" is not enabled for this run.`}, false},
// Each refusal carries the provenance its production path attaches: the
// disabled-tool path sets a denial category, the registry gates set the
// pre-execution marker. Text is never what decides.
{"disabled tool", ToolResult{Status: tools.StatusError, Output: `Error: Tool "x" is not enabled for this run.`, DenialReason: DenialFiltered}, false},
{"permission denied (meta)", ToolResult{Status: tools.StatusError, Output: "Error: Permission denied for x: needs approval", Meta: map[string]string{"permission_action": "deny"}}, false},
{"permission required", ToolResult{Status: tools.StatusError, Output: "Error: Permission required for x: approve first"}, false},
{"sandbox block", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox block: outside_workspace"}, false},
{"sandbox approval", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox approval required for x: network"}, false},
{"permission required", ToolResult{Status: tools.StatusError, Output: "Error: Permission required for x: approve first", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalPermissionRequired}}, false},
{"sandbox block", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox block: outside_workspace", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied}}, false},
{"sandbox approval", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox approval required for x: network", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxApproval}}, false},
// THE INVERSE, which is the failure this classifier had. An allowed tool
// that RAN, failed, and happened to print one of those phrases must stay
// retriable: `bash` preserves arbitrary stdout and stderr on any nonzero
// exit, and read_file returns whatever the file says.
{"executed failure printing the sandbox phrase", ToolResult{Status: tools.StatusError, Output: "Sandbox block\n"}, true},
{"executed failure printing a permission denial", ToolResult{Status: tools.StatusError, Output: "cp: Permission denied for /etc/hosts"}, true},
{"executed failure printing a permission requirement", ToolResult{Status: tools.StatusError, Output: "Permission required for sudo"}, true},
{"executed failure printing the approval phrase", ToolResult{Status: tools.StatusError, Output: "Sandbox approval required for curl"}, true},
{"executed failure quoting the disabled-tool phrase", ToolResult{Status: tools.StatusError, Output: "grep: web_fetch is not enabled for this run"}, true},
// Structured denial categories are authoritative regardless of message text.
{"denial: filtered", ToolResult{Status: tools.StatusError, Output: "anything", DenialReason: DenialFiltered}, false},
{"denial: permission", ToolResult{Status: tools.StatusError, Output: "anything", DenialReason: DenialPermissionDenied}, false},
+74
View File
@@ -0,0 +1,74 @@
package agent
import (
"context"
"strconv"
"strings"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// mixedOutcomeTool needs permission on every call. The test denies every other
// request, and a call that is allowed executes and fails with an error naming
// its own item, so no single identity repeats and only the content-blind bound
// can stop the run.
type mixedOutcomeTool struct{ ran int }
func (tool *mixedOutcomeTool) Name() string { return "bash" }
func (tool *mixedOutcomeTool) Description() string { return "probe" }
func (tool *mixedOutcomeTool) Parameters() tools.Schema { return tools.Schema{Type: "object"} }
func (tool *mixedOutcomeTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs a command"}
}
func (tool *mixedOutcomeTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{Status: tools.StatusError, Output: "Error: item " + strconv.Itoa(tool.ran) + " failed"}
}
// A MIXED HALT READS THE SAME HEADLESS AS INTERACTIVE.
//
// The stop answer for the content-blind bound over executed failures and
// refusals says the tool "failed or was refused". The headless reason for the
// same halt said only that it "failed repeatedly", so zero exec and the
// interactive answer disagreed about one stop. Reported by @jatmn.
func TestAMixedGuardHaltIsReportedAsMixedUnderTheCompletionGate(t *testing.T) {
tool := &mixedOutcomeTool{}
registry := tools.NewRegistry()
registry.Register(tool)
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
for i := range toolFailureAnyErrorStopAt + 4 {
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "bash", `{"item":`+strconv.Itoa(i)+`}`))
}
asked := 0
result, err := Run(context.Background(), "run them", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
RequireCompletionSignal: true,
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
asked++
if asked%2 == 0 {
return PermissionDecision{Action: PermissionDecisionAllow}, nil
}
return PermissionDecision{Action: PermissionDecisionDeny}, nil
},
})
if err != nil {
t.Fatal(err)
}
if tool.ran == 0 || asked <= tool.ran {
t.Fatalf("the run did not mix executed failures with refusals (ran %d of %d asked), so this proves nothing", tool.ran, asked)
}
if !strings.Contains(result.FinalAnswer, "failed or was refused") {
t.Fatalf("the halt was not the mixed content-blind bound: %q", result.FinalAnswer)
}
if !result.Incomplete {
t.Fatal("a guard halt under the completion gate must be Incomplete")
}
if !strings.Contains(result.IncompleteReason, "failed or was refused") || !strings.Contains(result.IncompleteReason, "bash") {
t.Errorf("IncompleteReason = %q, want it to say the tool failed or was refused, as the stop answer does", result.IncompleteReason)
}
}
@@ -0,0 +1,232 @@
package agent
import (
"context"
"strconv"
"strings"
"sync"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// readAheadTool is read-only and thread-safe, so a consecutive run of its calls
// is executed as one parallel batch before the loop consumes any of it.
type readAheadTool struct {
mu sync.Mutex
ran int
}
func (t *readAheadTool) Name() string { return "read_probe" }
func (t *readAheadTool) Description() string { return "read-only probe" }
func (t *readAheadTool) Parameters() tools.Schema {
return tools.Schema{Type: "object", Properties: map[string]tools.PropertySchema{"id": {Type: "string"}}}
}
func (t *readAheadTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "test"}
}
func (t *readAheadTool) Capabilities() tools.ToolCapabilities {
return tools.ToolCapabilities{Effect: tools.EffectReadOnly, ThreadSafe: true}
}
func (t *readAheadTool) Run(_ context.Context, args map[string]any) tools.Result {
t.mu.Lock()
t.ran++
t.mu.Unlock()
id, _ := args["id"].(string)
// Varying output, so no identity repeats and only the content-blind counter
// can halt the run.
return tools.Result{Status: tools.StatusError, Output: "failure for " + id}
}
func readCall(id string) []zeroruntime.StreamEvent {
return []zeroruntime.StreamEvent{
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: id, ToolCallProviderID: "provider-" + id, ToolName: "read_probe"},
{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: id, ArgumentsFragment: `{"id":"` + id + `"}`},
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: id},
}
}
// READ-AHEAD MEANS "NOT CONSUMED YET" NO LONGER MEANS "NOT EXECUTED".
//
// executeParallelReadBatch runs an entire eligible run of read calls before the
// loop consumes any of them. Every terminal branch in the consumption loop used
// to close out the calls after the current index with aborted placeholders, on
// the assumption that they had not run. A sibling that had already executed was
// therefore recorded as aborted: its real result was discarded, the transcript
// disagreed with what had happened, and its callbacks, trace counter, task
// observation and images were lost with it. Where such a sibling is a
// successful read, execution may already have committed file-observation credit
// for content the model never receives.
func TestParallelSiblingsAreFinalizedWhenTheGuardHalts(t *testing.T) {
tool := &readAheadTool{}
registry := tools.NewRegistry()
registry.Register(tool)
// Eleven single-call turns bring the content-blind counter to one below its
// bound, so the first call of the pair below trips it.
var turns [][]zeroruntime.StreamEvent
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
id := "s" + strconv.Itoa(i)
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
}
// Then one turn advertising two parallel-eligible calls. Both execute in the
// batch; the halt fires while consuming the first.
pair := append(readCall("pA"), readCall("pB")...)
turns = append(turns, append(pair, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
turns = append(turns, append(readCall("unreached"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
var reported []ToolResult
var announced int
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAuto,
MaxTurns: len(turns) + 5,
OnToolCall: func(ToolCall) { announced++ },
OnToolResult: func(r ToolResult) { reported = append(reported, r) },
})
if err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
}
// Every execution is reported exactly once, and the announcements match.
if tool.ran != len(reported) {
t.Errorf("the tool executed %d times but %d results were reported; a completed call was recorded as aborted",
tool.ran, len(reported))
}
if announced != len(reported) {
t.Errorf("OnToolCall fired %d times against %d results; the two callbacks disagree", announced, len(reported))
}
// Strict pairing: one tool result per advertised tool call, and the drained
// sibling carries its REAL output rather than the aborted placeholder.
byID := map[string]int{}
var siblingContent string
for _, message := range result.Messages {
if message.Role != zeroruntime.MessageRoleTool {
continue
}
byID[message.ToolCallID]++
if message.ToolCallID == "pB" {
siblingContent = message.Content
}
// A strict replay pairs each result to its tool_use by the provider
// call ID, so the drained sibling needs it as much as the consumed call.
if (message.ToolCallID == "pA" || message.ToolCallID == "pB") && message.ToolCallProviderID != "provider-"+message.ToolCallID {
t.Errorf("tool result %s carries provider call ID %q, want %q", message.ToolCallID, message.ToolCallProviderID, "provider-"+message.ToolCallID)
}
}
for _, id := range []string{"pA", "pB"} {
if byID[id] != 1 {
t.Errorf("tool call %s has %d results, want exactly 1", id, byID[id])
}
}
if siblingContent == abortedToolResultNotice {
t.Error("the sibling that already ran was recorded as aborted")
}
if !strings.Contains(siblingContent, "failure for pB") {
t.Errorf("the sibling's real output is missing from the transcript: %q", siblingContent)
}
// And the drained sibling must not reverse the decision that was already
// made: the halt is still attributed to the call that tripped it.
if !strings.Contains(result.FinalAnswer, strconv.Itoa(toolFailureAnyErrorStopAt)) {
t.Errorf("the stop answer changed after draining siblings: %q", result.FinalAnswer)
}
}
// imageReadTool is readAheadTool with an image on one named call, so a drained
// sibling's image can be followed from execution to the transcript.
type imageReadTool struct {
readAheadTool
imageFor string
}
func (t *imageReadTool) Run(ctx context.Context, args map[string]any) tools.Result {
result := t.readAheadTool.Run(ctx, args)
if id, _ := args["id"].(string); id == t.imageFor {
result.Images = []zeroruntime.ImageBlock{{MediaType: "image/png", Data: []byte("\x89PNG")}}
}
return result
}
// The drained sibling's image is part of what ran, so it follows the tool
// results like any other call's image. Images reach the transcript through the
// turn's deferred builder, which the halt path calls after closing out the
// siblings; a drain that skipped the sibling's image would lose it silently.
func TestParallelSiblingImageSurvivesTheGuardHalt(t *testing.T) {
tool := &imageReadTool{imageFor: "pB"}
registry := tools.NewRegistry()
registry.Register(tool)
var turns [][]zeroruntime.StreamEvent
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
id := "s" + strconv.Itoa(i)
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
}
pair := append(readCall("pA"), readCall("pB")...)
turns = append(turns, append(pair, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAuto,
MaxTurns: len(turns) + 5,
SupportsVision: func(string) bool { return true },
})
if err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
}
images := 0
for _, message := range result.Messages {
if message.Role == zeroruntime.MessageRoleUser {
images += len(message.Images)
}
}
if images != 1 {
t.Errorf("the drained sibling's image reached the transcript %d times, want once", images)
}
}
// A call that genuinely never ran still gets an aborted placeholder, so the fix
// did not simply stop aborting anything.
func TestUnstartedCallsStillGetAbortedPlaceholders(t *testing.T) {
tool := &readAheadTool{}
registry := tools.NewRegistry()
registry.Register(tool)
var turns [][]zeroruntime.StreamEvent
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
id := "s" + strconv.Itoa(i)
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
}
// One eligible pair, then a THIRD call outside the batch window is not what
// happens here: the whole run is eligible, so instead advertise a single call
// and let the halt fire on it, leaving nothing precomputed beyond it.
turns = append(turns, append(readCall("solo"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
turns = append(turns, append(readCall("unreached"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAuto,
MaxTurns: len(turns) + 5,
})
if err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
t.Fatalf("the guard did not halt: %q", result.FinalAnswer)
}
if tool.ran != toolFailureAnyErrorStopAt {
t.Errorf("the tool ran %d times, want %d; nothing should execute past the halt",
tool.ran, toolFailureAnyErrorStopAt)
}
}
+30 -3
View File
@@ -2,6 +2,7 @@ package agent
import (
"context"
"strings"
"sync"
"github.com/Twigpine/zero/internal/tools"
@@ -12,8 +13,17 @@ import (
// them one after another serializes pure I/O waits. A consecutive run of
// capability-safe read-only calls is executed concurrently instead; results are
// then consumed in the original call order, so guard counters, message
// ordering, abort semantics, and the surface's call/result event pairing are
// byte-identical to sequential execution. Runs never span a mutating call: a
// ordering, abort semantics, and the surface's call/result event pairing match
// sequential execution.
//
// What can differ is how many calls RAN. A run executes before the guard
// consumes its first result, so a halt on an earlier call can leave up to
// maxParallelReadTools-1 siblings already executed. The guard bounds count
// consumed results, not executions, and the extra executions are reads by the
// eligibility rule below; closeOutRemaining records them without feeding the
// guard. Reported by @jatmn.
//
// Runs never span a mutating call: a
// read that follows a write must observe the write, so eligibility is decided
// per consecutive run, not per batch.
//
@@ -36,6 +46,15 @@ const maxParallelReadTools = 8
type precomputedToolResult struct {
result ToolResult
abortErr error
// completed records whether this call PRODUCED a result, which is a
// different question from whether it asks the enclosing run to stop.
//
// A cancelled permission request is both: executeToolCall builds the
// cancellation result, with its call ID, its message and its denial category,
// and returns it together with ErrPermissionApprovalCanceled. Reading the
// error as "never started" throws that result away, and the transcript then
// denies that permission handling happened at all.
completed bool
}
// parallelSafeToolCall reports whether call may run concurrently with its
@@ -189,7 +208,15 @@ func executeParallelReadBatch(ctx context.Context, registry *tools.Registry, cal
semaphore <- struct{}{}
defer func() { <-semaphore }()
result, abortErr := executeToolCall(ctx, registry, calls[index], permissionMode, batchOptions)
results[index-start] = precomputedToolResult{result: result, abortErr: abortErr}
// Recorded HERE, where both halves are in hand, rather than inferred later
// from the error. A populated call ID is what every path that produced a
// result sets and what a context cancellation before execution leaves
// empty.
results[index-start] = precomputedToolResult{
result: result,
abortErr: abortErr,
completed: strings.TrimSpace(result.ToolCallID) != "",
}
}(index)
}
waitGroup.Wait()
@@ -0,0 +1,116 @@
package agent
import (
"context"
"strconv"
"strings"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// forgedMarkerTool executes, fails, and claims in its metadata that the registry
// refused it before it ran.
type forgedMarkerTool struct {
ran int
key string
value string
}
func (tool *forgedMarkerTool) Name() string { return "bash" }
func (tool *forgedMarkerTool) Description() string { return "test shell tool" }
func (tool *forgedMarkerTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
Required: []string{"command"},
AdditionalProperties: false,
}
}
func (tool *forgedMarkerTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
}
func (tool *forgedMarkerTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{
Status: tools.StatusError,
Output: "Error: the script exited 1",
Meta: map[string]string{tool.key: tool.value},
}
}
// THE FORGED MARKER MUST NOT CHANGE THE RUN.
//
// Classifying from metadata instead of from output text is only an improvement
// while the metadata is something an executed result cannot produce. The
// registry strips the key at the execution boundary; this is the same claim
// stated where it is spent, because the unit test proves the boundary and this
// proves the loop behaves.
//
// An executed failure that is misread as a refusal loses the schema hint, stops
// the profile failure streak from recovering, and is counted toward the refusal
// halt, so the run ends early and the final answer can say a tool was refused
// when it ran every time it was asked.
func TestRunTreatsAForgedRefusalMarkerAsAnExecutedFailure(t *testing.T) {
for _, forged := range []struct{ key, value string }{
{tools.PolicyRefusalMeta, tools.PolicyRefusalSandboxDenied},
{tools.PolicyRefusalMeta, tools.PolicyRefusalPermissionDenied},
{tools.PolicyRefusalMeta, tools.PolicyRefusalToolNotEnabled},
{tools.PolicyRefusalMeta, "something-the-registry-never-emits"},
// The loop reads a denied permission action as a refusal too, so an
// executed failure carrying one is the same forgery. Reported by @jatmn.
{tools.PermissionActionMeta, string(PermissionActionDeny)},
} {
value := forged.value
t.Run(forged.key+"="+value, func(t *testing.T) {
tool := &forgedMarkerTool{key: forged.key, value: value}
registry := tools.NewRegistry()
registry.Register(tool)
// One more call than the hint threshold, so the hint has to have been
// injected by the last one, then a final text turn to end the run.
calls := toolFailureHintAt + 1
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
for i := range calls {
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash", `{"command":"./flaky.sh"}`))
}
turns = append(turns, textTurn("gave up on the script"))
var denials []DenialCategory
result, err := Run(context.Background(), "run the script", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
OnToolResult: func(toolResult ToolResult) {
denials = append(denials, toolResult.DenialReason)
},
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
t.Error("permission was requested for an allow-safety tool; this test must exercise the executed-failure path")
return PermissionDecision{Action: PermissionDecisionDeny}, nil
},
})
if err != nil {
t.Fatal(err)
}
if tool.ran != calls {
t.Fatalf("the tool ran %d times, want %d; the run was halted by a marker it set on its own result", tool.ran, calls)
}
for index, denial := range denials {
if denial != DenialNone {
t.Errorf("result %d was recorded as denial %q; the tool executed", index, denial)
}
}
var hinted bool
for _, message := range result.Messages {
if strings.Contains(message.Content, toolFailureHintMarker) {
hinted = true
break
}
}
if !hinted {
t.Errorf("no retry hint was injected for an executed failure carrying %q, so the model got no correction", value)
}
})
}
}
@@ -0,0 +1,196 @@
package agent
import (
"context"
"strconv"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/trace"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// THE UNCATEGORIZED REFUSALS ARE THE ONES THIS CHANGE EXISTS FOR, SO THEY ARE THE
// ONES THAT MUST BE DRIVEN THROUGH Run.
//
// A categorized denial is easy to test and was never the gap. The gap is a
// refusal that arrives with DenialReason empty, because a category is attached
// only where a TYPED denial is built:
//
// - a headless run leaves OnPermissionRequest nil, so the loop never reaches
// its typed-denial branch and the registry gate returns a bare
// `Permission required for ...`
// - a sandbox preflight denial on a non-shell tool loses its SandboxDecision
// converting to ToolResult and arrives as a bare `Sandbox block`
//
// Both were counted as SUCCESS, which cleared the very record they were meant to
// accumulate, and the refused call repeated to MaxTurns.
//
// Asserting that through the helper alone would prove nothing: the first version
// of this fix passed every helper test while being a no-op in production,
// because the loop asked a different question than the tests did. So each case
// below goes through Run and pins the three things the loop actually does with
// the classification: halt at the bound, never execute the tool, and withhold
// the profile's one-shot failure escalation.
// headlessPromptTool needs approval and never gets it, because the run has no
// approver. The registry refuses it before Run() is reached, exactly as a
// headless run does, so nothing here fakes the refusal.
type headlessPromptTool struct{ ran int }
func (tool *headlessPromptTool) Name() string { return "bash" }
func (tool *headlessPromptTool) Description() string { return "test shell tool" }
func (tool *headlessPromptTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
Required: []string{"command"},
AdditionalProperties: false,
}
}
func (tool *headlessPromptTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs shell commands"}
}
func (tool *headlessPromptTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{Status: tools.StatusOK, Output: "should never run"}
}
// A headless prompt refusal must halt at the same-signature bound rather than
// repeat to MaxTurns.
//
// It used to key on the refusal TEXT, which the registry happens to hold
// constant here, and the stop then described a repeated failure. The registry
// marks this path, so the category is now derived at the ToolResult boundary and
// the streak keys on that instead. The bound is unchanged; the wording is more
// honest, because a headless prompt refusal is a refusal and not a tool that
// keeps failing.
func TestRunStopsAnUncategorizedHeadlessRefusalAtTheFailureBound(t *testing.T) {
tool := &headlessPromptTool{}
registry := tools.NewRegistry()
registry.Register(tool)
// Comfortably more turns than the bound, so reaching the bound is what stops
// the run rather than exhausting the provider or MaxTurns.
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
for i := range toolFailureStopAt + 4 {
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
`{"command":"touch /etc/file`+strconv.Itoa(i)+`"}`))
}
provider := &mockProvider{turns: turns}
recorder := trace.NewRecorder("policy-refusal-session", "run-1", "fast")
result, err := Run(context.Background(), "do the thing", provider, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
Trace: recorder,
// Armed well below the halt bound, so a refusal counted as a retriable
// failure would visibly spend it.
Profile: &ProfilePolicy{
Name: "fast",
Escalate: &PostureEscalation{MaxTurns: 999, OnToolFailureStreak: 2},
},
// OnPermissionRequest deliberately nil: this IS the headless path.
})
if err != nil {
t.Fatal(err)
}
if len(provider.requests) != toolFailureStopAt {
t.Errorf("the run made %d turns, want it halted at %d; an uncategorized refusal is not being counted", len(provider.requests), toolFailureStopAt)
}
if tool.ran != 0 {
t.Errorf("the refused tool executed %d times; the registry gate must precede execution", tool.ran)
}
// Categorized now, so the stop answer says refused. Previously the loop could
// not tell and reported a repeated failure.
want := toolFailureStopAnswer("bash", toolFailureStopAt, toolFailureCauseSameRefusal)
if result.FinalAnswer != want {
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
}
assertNoPostureEscalation(t, recorder)
}
// uncategorizedSandboxTool returns the shape a sandbox preflight denial has by
// the time the loop sees it on a non-shell tool: StatusError, `Sandbox block`
// prose, and no category, because the SandboxDecision does not survive the
// conversion to ToolResult. The path varies per call exactly as the real message
// does, since it names what was refused.
type uncategorizedSandboxTool struct{ ran int }
func (tool *uncategorizedSandboxTool) Name() string { return "write_file" }
func (tool *uncategorizedSandboxTool) Description() string { return "test write tool" }
func (tool *uncategorizedSandboxTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"path": {Type: "string"}},
Required: []string{"path"},
AdditionalProperties: false,
}
}
func (tool *uncategorizedSandboxTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectWrite, Permission: tools.PermissionAllow, Reason: "writes files"}
}
func (tool *uncategorizedSandboxTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{
Status: tools.StatusError,
Output: "Sandbox block: write to /out-" + strconv.Itoa(tool.ran) + ".txt is outside the workspace",
}
}
// Varying text plus no category means every call restarts the same-signature
// streak at 1, so the same-signature bound is never reached and only the
// content-blind counter can halt the run. Before the fix nothing halted it at
// all: each refusal was read as a success, deleted the record, and the call
// repeated until MaxTurns.
func TestRunStopsAnUncategorizedVaryingSandboxRefusalAtTheVariedBound(t *testing.T) {
tool := &uncategorizedSandboxTool{}
registry := tools.NewRegistry()
registry.Register(tool)
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
for i := range toolFailureAnyErrorStopAt + 4 {
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "write_file",
`{"path":"/out-`+strconv.Itoa(i)+`.txt"}`))
}
provider := &mockProvider{turns: turns}
recorder := trace.NewRecorder("policy-refusal-session", "run-2", "fast")
result, err := Run(context.Background(), "write the files", provider, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
Trace: recorder,
Profile: &ProfilePolicy{
Name: "fast",
Escalate: &PostureEscalation{MaxTurns: 999, OnToolFailureStreak: 2},
},
})
if err != nil {
t.Fatal(err)
}
if tool.ran != toolFailureAnyErrorStopAt {
t.Errorf("the tool was called %d times, want the run halted at %d", tool.ran, toolFailureAnyErrorStopAt)
}
want := toolFailureStopAnswer("write_file", toolFailureAnyErrorStopAt, toolFailureCauseVariedExecuted)
if result.FinalAnswer != want {
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
}
assertNoPostureEscalation(t, recorder)
}
// The profile's failure-streak trigger restores turn budget and reasoning effort
// on the theory that a struggling tool needs room. A refusal is not a struggling
// tool, it is an answer, and spending the one-shot escalation on one buys turns
// that will be refused identically. The loop keeps this contract by passing an
// empty outcome to the controller for a refusal, which is only observable from
// outside as the escalation never firing.
func assertNoPostureEscalation(t *testing.T, recorder *trace.Recorder) {
t.Helper()
if got := recorder.Finish().Counter(trace.CounterPostureEscalations); got != 0 {
t.Errorf("posture_escalations = %d, want 0; a policy refusal spent the one-shot failure escalation", got)
}
}
@@ -0,0 +1,265 @@
package agent
import (
"context"
"strconv"
"strings"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// A REFUSAL IS A FAILURE, AND THE CLASSIFIER MUST SAY SO BEFORE IT READS ANYTHING
// ELSE.
//
// isPolicyRefusal decides on denial category, then permission metadata, then
// output text. None of those questions is meaningful about a call the tool
// completed, and the last one is answered by content the model does not control:
// a `bash` that greps the sandbox docs, a read_file returning this very source
// file, any successful command whose output happens to carry one of the phrases.
//
// isRetriableToolError gated on StatusError before calling in, so the boundary
// held while that was the only caller. Extracting the helper and calling it from
// the counting path dropped the gate, and every successful result was put to a
// question that only failures can answer.
//
// Each case below is a SUCCESS. None may be read as a refusal.
func TestPolicyRefusalNeverClassifiesASuccessfulResult(t *testing.T) {
cases := []struct {
name string
result ToolResult
// structured marks a result carrying real provenance: a denial category,
// permission metadata, or the registry's pre-execution marker. Only those
// may be read as a refusal when they fail. The rest carry nothing but
// text, which the tool controls, so they must stay ordinary failures no
// matter which phrase they happen to contain.
structured bool
}{
{
name: "output quotes the sandbox block phrase",
result: ToolResult{Status: tools.StatusOK, Output: "grep: docs/sandbox.md: Sandbox block"},
},
{
name: "output quotes the disabled-tool phrase",
result: ToolResult{Status: tools.StatusOK, Output: "web_fetch is not enabled for this run"},
},
{
name: "output quotes a permission denial",
result: ToolResult{Status: tools.StatusOK, Output: "log line: Permission denied for bash"},
},
{
name: "output quotes a permission requirement",
result: ToolResult{Status: tools.StatusOK, Output: "log line: Permission required for bash"},
},
{
name: "output quotes a sandbox approval prompt",
result: ToolResult{Status: tools.StatusOK, Output: "Sandbox approval required for curl"},
},
{
// Metadata is no more trustworthy than text on a completed call. A
// result carrying a category with an OK status is not a denial; every
// path that attaches one fails the call.
name: "stale denial category on a completed call",
result: ToolResult{Status: tools.StatusOK, DenialReason: DenialSandboxBlock, Output: "ok"},
structured: true,
},
{
name: "stale permission metadata on a completed call",
result: ToolResult{
Status: tools.StatusOK,
Meta: map[string]string{"permission_action": string(PermissionActionDeny)},
Output: "ok",
},
structured: true,
},
{
name: "registry refusal marker on a completed call",
result: ToolResult{
Status: tools.StatusOK,
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied},
Output: "ok",
},
structured: true,
},
}
for _, testCase := range cases {
t.Run(testCase.name, func(t *testing.T) {
if isPolicyRefusal(testCase.result) {
t.Errorf("a successful result was classified as a policy refusal, so a healthy call counts toward the denial streak: %+v", testCase.result)
}
failed := testCase.result
failed.Status = tools.StatusError
switch {
case testCase.structured && !isPolicyRefusal(failed):
t.Errorf("a failure carrying real provenance was not classified as a refusal, so the gate broke the classifier: %+v", failed)
case !testCase.structured && isPolicyRefusal(failed):
t.Errorf("a failure carrying only text was classified as a refusal; output is tool-controlled and cannot decide this: %+v", failed)
}
})
}
}
// blockTextShellTool runs freely and prints text the classifier keys on. This
// is not contrived: `Sandbox block` appears in this repo's own docs and error
// strings, so any grep, cat, or file read that crosses them produces exactly
// this result.
type blockTextShellTool struct{ ran int }
func (tool *blockTextShellTool) Name() string { return "bash" }
func (tool *blockTextShellTool) Description() string { return "test shell tool" }
func (tool *blockTextShellTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
Required: []string{"command"},
AdditionalProperties: false,
}
}
func (tool *blockTextShellTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
}
func (tool *blockTextShellTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{
Status: tools.StatusOK,
Output: "docs/windows-sandbox.md:41: Sandbox block (write jail) is reported as ...",
}
}
// The same gap driven through Run, because the classifier being wrong only
// matters if the loop acts on it, and it does: the counting path calls
// isPolicyRefusal directly, and a true answer records a failure against the
// tool's signature. Six identical successes then look like six identical
// denials, and the run ends on a stop answer telling the user a tool it never
// refused was refused six times.
//
// The tool succeeds every turn and the run is given more turns than the bound,
// so anything that halts before the provider's final turn is the guard.
func TestRunDoesNotHaltOnSuccessfulOutputThatQuotesARefusal(t *testing.T) {
tool := &blockTextShellTool{}
registry := tools.NewRegistry()
registry.Register(tool)
calls := toolFailureStopAt + 4
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
for i := range calls {
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
`{"command":"grep -n 'Sandbox block' docs/windows-sandbox.md"}`))
}
turns = append(turns, textTurn("found the docs"))
provider := &mockProvider{turns: turns}
result, err := Run(context.Background(), "search the docs", provider, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
t.Error("the loop asked permission for an allow-safety tool; this test must exercise the success path")
return PermissionDecision{Action: PermissionDecisionDeny}, nil
},
})
if err != nil {
t.Fatal(err)
}
if tool.ran != calls {
t.Errorf("the tool ran %d times, want %d; the run was halted mid-way by its own success output", tool.ran, calls)
}
if result.FinalAnswer != "found the docs" {
t.Errorf("final answer =\n %q\nwant\n %q\n(a stop answer here means successful output was counted as a refusal)",
result.FinalAnswer, "found the docs")
}
}
// failingPhraseTool RAN and failed. Its stderr happens to contain a phrase the
// classifier used to key on, which is the everyday case: `bash` preserves
// arbitrary stdout and stderr on any nonzero exit, so
// `printf 'Sandbox block\n' >&2; exit 1` produces exactly this.
type failingPhraseTool struct {
ran int
output string
}
func (tool *failingPhraseTool) Name() string { return "bash" }
func (tool *failingPhraseTool) Description() string { return "test shell tool" }
func (tool *failingPhraseTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
Required: []string{"command"},
AdditionalProperties: false,
}
}
func (tool *failingPhraseTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
}
func (tool *failingPhraseTool) Run(context.Context, map[string]any) tools.Result {
tool.ran++
return tools.Result{Status: tools.StatusError, Output: tool.output}
}
// AN EXECUTED FAILURE IS RETRIABLE, WHATEVER IT PRINTED, and the observable
// difference is the schema hint.
//
// isRetriableToolError feeds `hintable` into the guard, and the guard injects
// the retry hint at toolFailureHintAt. Classifying an executed failure as a
// policy refusal makes hintable false, so the model is told nothing and simply
// repeats the call: the run loses the one correction it had. That is on top of
// the failure being accumulated as a denial.
//
// Each phrase below is one the classifier used to match on. The tool is
// allow-safety and runs every time, so nothing here is refused by anything.
func TestRunHintsAnExecutedFailureThatPrintsARefusalPhrase(t *testing.T) {
for _, output := range []string{
"Sandbox block\n",
"cp: Permission denied for /etc/hosts",
"Permission required for sudo",
"Sandbox approval required for curl",
"grep: web_fetch is not enabled for this run",
} {
t.Run(output, func(t *testing.T) {
tool := &failingPhraseTool{output: output}
registry := tools.NewRegistry()
registry.Register(tool)
// One more call than the hint threshold, so the hint has to have been
// injected by the last one, then a final text turn to end the run.
calls := toolFailureHintAt + 1
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
for i := range calls {
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash", `{"command":"./flaky.sh"}`))
}
turns = append(turns, textTurn("gave up on the script"))
provider := &mockProvider{turns: turns}
result, err := Run(context.Background(), "run the script", provider, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
t.Error("permission was requested for an allow-safety tool; this test must exercise the executed-failure path")
return PermissionDecision{Action: PermissionDecisionDeny}, nil
},
})
if err != nil {
t.Fatal(err)
}
if tool.ran != calls {
t.Fatalf("the tool ran %d times, want %d; the run was halted by output it merely printed", tool.ran, calls)
}
var hinted bool
for _, message := range result.Messages {
if strings.Contains(message.Content, toolFailureHintMarker) {
hinted = true
break
}
}
if !hinted {
t.Errorf("no retry hint was injected for an executed failure printing %q; it was classified as a policy refusal, so the model got no correction", output)
}
})
}
}
+84
View File
@@ -0,0 +1,84 @@
package agent
import (
"testing"
"github.com/Twigpine/zero/internal/tools"
)
// UNCATEGORIZED REFUSALS MUST STILL COUNT.
//
// The halt guard asked DenialReason != DenialNone, but a category is only
// attached where a TYPED denial is built. A headless run leaves
// OnPermissionRequest nil, so the loop never reaches that branch and the
// registry returns a bare "Error: Permission required ..." with no category. A
// sandbox preflight denial on a non-shell tool loses its SandboxDecision in the
// conversion and arrives as an uncategorized "Sandbox block".
//
// Both were counted as SUCCESS, which cleared the very record the guard
// accumulates, so the same refused call could repeat to MaxTurns. That is the
// loop this PR exists to stop.
func TestUncategorizedPolicyRefusalsAreCounted(t *testing.T) {
for _, testCase := range []struct {
name string
result ToolResult
}{
{
// Still uncategorized in the DenialReason sense: a headless run never
// reaches the branch that builds a typed denial. It carries the
// registry's pre-execution marker instead, which is what makes it
// classifiable without reading the message.
name: "headless prompt refusal carries no denial category",
result: ToolResult{
Status: tools.StatusError,
Output: `Error: Permission required for bash: The tool is marked "prompt" and was not executed.`,
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalPermissionRequired},
},
},
{
// The SandboxDecision is still dropped in the conversion for a
// non-shell tool; the marker is what survives it.
name: "sandbox preflight denial loses its decision in conversion",
result: ToolResult{
Status: tools.StatusError,
Output: "Sandbox block: write outside the workspace",
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied},
},
},
{
name: "an explicitly denied permission action",
result: ToolResult{Status: tools.StatusError, Output: "refused", Meta: map[string]string{"permission_action": string(PermissionActionDeny)}},
},
{
name: "a categorized denial still counts",
result: ToolResult{Status: tools.StatusError, Output: "denied", DenialReason: DenialFiltered},
},
} {
t.Run(testCase.name, func(t *testing.T) {
if !isPolicyRefusal(testCase.result) {
t.Fatal("refusal was treated as a success, so the guard's record is cleared and the call can repeat to MaxTurns")
}
if isRetriableToolError(testCase.result) {
t.Error("a policy refusal is not retriable: repeating it verbatim cannot help")
}
})
}
}
// An ordinary failure is still retriable and still not a refusal, or the guard
// would halt on things the model can legitimately fix by trying again.
func TestOrdinaryFailuresAreNotPolicyRefusals(t *testing.T) {
for _, output := range []string{
"Error: file not found",
"Error: invalid JSON in arguments",
"Error: connection reset",
} {
result := ToolResult{Status: tools.StatusError, Output: output}
if isPolicyRefusal(result) {
t.Errorf("%q was classified as a policy refusal", output)
}
if !isRetriableToolError(result) {
t.Errorf("%q should stay retriable", output)
}
}
}
@@ -0,0 +1,87 @@
package agent
import (
"context"
"errors"
"fmt"
"testing"
"github.com/Twigpine/zero/internal/tools"
)
// PRODUCING A RESULT AND ASKING THE RUN TO STOP ARE DIFFERENT FACTS.
//
// closeOutRemaining decides between finalizing a completed sibling and writing
// an aborted placeholder, and it used to make that decision from abortErr. A
// cancelled permission request is exactly where the two facts disagree:
// executeToolCall builds the cancellation result, with its call ID, its message
// and its denial category, and returns it TOGETHER with
// ErrPermissionApprovalCanceled. Reading the error as "never started" discards a
// real result and records the opposite of what happened, and the cancellation
// then goes missing from OnToolResult, the trace counters and the task
// observation even though the permission event already fired.
func TestABatchEntryWithAResultAndAnAbortErrorIsDrained(t *testing.T) {
canceled := canceledPermissionResult(
ToolCall{ID: "call-b", Name: "read_probe"},
"cancelled in TUI",
PermissionEvent{ToolName: "read_probe"},
)
precomputed := []precomputedToolResult{
{result: ToolResult{ToolCallID: "call-a", Name: "read_probe", Status: tools.StatusOK}, completed: true},
{
result: canceled,
abortErr: fmt.Errorf("%w for read_probe", ErrPermissionApprovalCanceled),
completed: true,
},
}
sibling, ran := precomputedResultFor(precomputed, 0, 2, 1)
if !ran {
t.Fatal("a cancelled permission result was treated as a call that never started")
}
if sibling.ToolCallID != "call-b" {
t.Errorf("ToolCallID = %q, want the cancellation result rather than an empty placeholder", sibling.ToolCallID)
}
if sibling.DenialReason != DenialApprovalCanceled {
t.Errorf("DenialReason = %q, want the cancellation preserved", sibling.DenialReason)
}
}
// An entry that never produced a result is still unstarted, or the fix would
// have stopped aborting anything.
func TestABatchEntryWithNoResultIsStillUnstarted(t *testing.T) {
precomputed := []precomputedToolResult{
{abortErr: errors.New("cancelled before the tool ran")},
}
if _, ran := precomputedResultFor(precomputed, 0, 1, 0); ran {
t.Error("an entry that produced nothing was finalized as if it had run")
}
}
// A call outside the batch window never started either.
func TestACallOutsideTheBatchWindowIsUnstarted(t *testing.T) {
precomputed := []precomputedToolResult{{result: ToolResult{ToolCallID: "call-a"}, completed: true}}
if _, ran := precomputedResultFor(precomputed, 0, 1, 1); ran {
t.Error("a call past the batch window was reported as executed")
}
}
// And the flag is recorded where the truth is known rather than inferred later,
// so a real batch marks what it ran.
func TestTheBatchRecordsWhatItRan(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(&readAheadTool{})
calls := []ToolCall{
{ID: "call-a", Name: "read_probe", Arguments: `{"id":"a"}`},
{ID: "call-b", Name: "read_probe", Arguments: `{"id":"b"}`},
}
results := executeParallelReadBatch(context.Background(), registry, calls, 0, 2, PermissionModeAuto, Options{Registry: registry})
for index, entry := range results {
if !entry.completed {
t.Errorf("entry %d executed but was not recorded as completed: %#v", index, entry)
}
if entry.result.ToolCallID != calls[index].ID {
t.Errorf("entry %d is keyed to %q, want %q", index, entry.result.ToolCallID, calls[index].ID)
}
}
}
+176
View File
@@ -0,0 +1,176 @@
package agent
import (
"context"
"strconv"
"strings"
"testing"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// legacyDenialSignaturePrefix is the string provenance used to be encoded in.
// The identity was one namespace holding both a normalized error signature and
// a synthetic "denial:<category>" key, and provenance was recovered afterwards
// with strings.HasPrefix. It survives here, and only here, so the regression
// names the exact spelling that used to collide.
const legacyDenialSignaturePrefix = "denial:"
func runUntilGuardHalt(t *testing.T, gate bool) Result {
t.Helper()
tool := &uncategorizedSandboxTool{}
registry := tools.NewRegistry()
registry.Register(tool)
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
for i := range toolFailureAnyErrorStopAt + 4 {
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "write_file", `{"path":"/o-`+strconv.Itoa(i)+`.txt"}`))
}
result, err := Run(context.Background(), "write", &mockProvider{turns: turns}, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
RequireCompletionSignal: gate,
})
if err != nil {
t.Fatal(err)
}
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
}
return result
}
// A GUARD HALT IS UNFINISHED WORK, AND HEADLESS HAS TO HEAR ABOUT IT.
//
// The halt returns straight out of the tool loop, so it never reached the
// completion gate the max-turns paths go through. zero exec treats only
// Result.Incomplete as exit 4 and otherwise reports run_end success with exit
// 0, so a task denied six times, or failed through the twelve-call bound, came
// back as a successful automation result having done none of the work asked.
func TestGuardHaltIsIncompleteUnderTheCompletionGate(t *testing.T) {
headless := runUntilGuardHalt(t, true)
if !headless.Incomplete {
t.Error("a guard halt reports the run as complete; zero exec would call a run that did nothing a success")
}
if !strings.Contains(headless.IncompleteReason, "write_file") {
t.Errorf("the incomplete reason does not name the tool that halted the run: %q", headless.IncompleteReason)
}
// And the interactive default is untouched: Incomplete exists for the
// headless gate, so setting it without one would be inventing a status.
if interactive := runUntilGuardHalt(t, false); interactive.Incomplete {
t.Error("Incomplete was set without RequireCompletionSignal, changing interactive behaviour")
}
}
// executedDenialLookalikeTool runs, fails, and prints exactly the string the
// guard used to store for a real permission refusal.
type executedDenialLookalikeTool struct{ ran int }
func (t *executedDenialLookalikeTool) Name() string { return "bash" }
func (t *executedDenialLookalikeTool) Description() string { return "probe" }
func (t *executedDenialLookalikeTool) Parameters() tools.Schema { return tools.Schema{Type: "object"} }
func (t *executedDenialLookalikeTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionAllow}
}
func (t *executedDenialLookalikeTool) Run(context.Context, map[string]any) tools.Result {
t.ran++
return tools.Result{Status: tools.StatusError, Output: legacyDenialSignaturePrefix + string(DenialPermissionDenied)}
}
// TRUSTED PROVENANCE AND UNTRUSTED CONTENT MUST NOT SHARE A NAMESPACE.
//
// While the identity was a single string, a command that printed exactly
// "denial:permission_denied" and exited non-zero took on the identity of a real
// permission refusal: it merged streaks with one, and the run told the user the
// tool had been refused when it had executed every time.
func TestAnExecutedErrorCannotImpersonateARefusal(t *testing.T) {
tool := &executedDenialLookalikeTool{}
registry := tools.NewRegistry()
registry.Register(tool)
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
for i := range toolFailureStopAt + 4 {
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "bash", `{}`))
}
result, err := Run(context.Background(), "run it", &mockProvider{turns: turns}, Options{
Registry: registry, PermissionMode: PermissionModeAsk, MaxTurns: len(turns) + 5,
})
if err != nil {
t.Fatal(err)
}
if tool.ran == 0 {
t.Fatal("the tool never ran, so this no longer covers an executed failure")
}
if strings.Contains(result.FinalAnswer, "was refused") {
t.Errorf("the tool executed %d times and nothing refused it, but the run says it was refused: %q", tool.ran, result.FinalAnswer)
}
if !strings.Contains(result.FinalAnswer, "with the same error") {
t.Errorf("an executed failure repeating identically should read as the same error: %q", result.FinalAnswer)
}
}
// AND THE AGGREGATE IS ITS OWN FACT.
//
// The content-blind bound fires precisely when no single identity repeated
// enough, so the identity present at the end says nothing about the eleven
// before it. Alternating two refusal categories reaches twelve without either
// reaching six; the tool never ran once, and the answer used to describe that
// as the tool failing with varying errors.
func TestTheContentBlindBoundKeepsRefusalProvenance(t *testing.T) {
for _, testCase := range []struct {
name string
kinds []DenialCategory
wantCause toolFailureCause
wantAnswer string
}{
{
name: "alternating refusal categories",
kinds: []DenialCategory{DenialPermissionDenied, DenialSandboxBlock},
wantCause: toolFailureCauseVariedRefused,
wantAnswer: "was refused",
},
{
// Mixed is DECIDED, not inherited from whichever field a switch
// tested first. Neither "failed" alone nor "refused" alone is true of
// this sequence, so the wording says both.
name: "executed failures mixed with refusals",
kinds: []DenialCategory{DenialNone, DenialPermissionDenied},
wantCause: toolFailureCauseVariedMixed,
wantAnswer: "failed or was refused",
},
{
name: "executed failures only",
kinds: []DenialCategory{DenialNone, DenialNone},
wantCause: toolFailureCauseVariedExecuted,
wantAnswer: "with varying errors",
},
} {
t.Run(testCase.name, func(t *testing.T) {
state := newGuardState()
var outcome toolFailureOutcome
for index := 0; index < toolFailureAnyErrorStopAt; index++ {
// Distinct prose every call, exactly as a real refusal or error
// naming what it touched would produce.
outcome = state.observeToolResult("bash", true, false,
"failure on item "+strconv.Itoa(index), testCase.kinds[index%len(testCase.kinds)])
if outcome.Stop {
break
}
}
if !outcome.Stop {
t.Fatal("twelve consecutive failures did not trip the content-blind bound")
}
if outcome.Count != toolFailureAnyErrorStopAt {
t.Errorf("Count = %d, want the counter that tripped (%d)", outcome.Count, toolFailureAnyErrorStopAt)
}
if outcome.Cause != testCase.wantCause {
t.Errorf("Cause = %v, want %v", outcome.Cause, testCase.wantCause)
}
answer := toolFailureStopAnswer("bash", outcome.Count, outcome.Cause)
if !strings.Contains(answer, testCase.wantAnswer) {
t.Errorf("stop answer = %q, want it to contain %q", answer, testCase.wantAnswer)
}
})
}
}
@@ -0,0 +1,103 @@
package agent
import (
"context"
"path/filepath"
"strconv"
"strings"
"testing"
"github.com/Twigpine/zero/internal/sandbox"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// deniedWriteTool records every execution. Its body must never run: the sandbox
// refuses the call at the registry boundary, before the tool is reached.
type deniedWriteTool struct{ ran int }
func (t *deniedWriteTool) Name() string { return "write_file" }
func (t *deniedWriteTool) Description() string { return "test write tool" }
func (t *deniedWriteTool) Parameters() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]tools.PropertySchema{"path": {Type: "string"}},
Required: []string{"path"},
}
}
func (t *deniedWriteTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectWrite, Permission: tools.PermissionAllow, Reason: "writes files"}
}
func (t *deniedWriteTool) Run(context.Context, map[string]any) tools.Result {
t.ran++
return tools.Result{Status: tools.StatusOK, Output: "wrote"}
}
// THROUGH THE REAL SANDBOX, NOT A FAKE STANDING WHERE IT WOULD BE.
//
// The neighbouring uncategorized test deliberately hand-builds the shape a
// refusal has when no category survives, which is a fact about the LOOP. This
// one covers the producer: a real sandbox.Engine evaluation denies the call,
// the registry turns that into a categorized refusal, and the guard keys on the
// category. Without it, reverting refusalResult(..., PolicyRefusalSandboxDenied)
// in registry.go to a plain errorResult left both the agent and tools suites
// green, so nothing depended on the marker at all.
//
// Every assertion here fails if a different link breaks: ran proves the body was
// skipped, the request count proves the CATEGORY bound halted it rather than the
// content-blind one, and the wording proves the provenance survived conversion.
func TestRunHaltsARealSandboxDenialOnTheCategoryBound(t *testing.T) {
workspace := t.TempDir()
forbidden := t.TempDir()
policy := sandbox.DefaultPolicy()
// Applies whenever the sandbox is enforcing, so this does not depend on the
// workspace boundary being reachable from a synthetic request.
policy.DenyWrite = []string{forbidden}
tool := &deniedWriteTool{}
registry := tools.NewRegistry()
registry.Register(tool)
// A different path every call. Note that errorSignature truncates at 80
// characters and the message opens with a fixed 40-character prefix followed
// by a long temp root, so the signatures collide here anyway: the request
// count below pins WHICH bound halted the run, but it does not by itself
// prove the streak was held by the category. The wording assertion is the one
// that discriminates, and it is the one that fails when the marker is gone.
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
for i := range toolFailureStopAt + 4 {
target := filepath.Join(forbidden, "escape-"+strconv.Itoa(i)+".txt")
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "write_file", `{"path":`+strconv.Quote(target)+`}`))
}
provider := &mockProvider{turns: turns}
result, err := Run(context.Background(), "write the files", provider, Options{
Registry: registry,
PermissionMode: PermissionModeAsk,
MaxTurns: len(turns) + 5,
Cwd: workspace,
Sandbox: sandbox.NewEngine(sandbox.EngineOptions{
WorkspaceRoot: workspace,
Policy: policy,
}),
})
if err != nil {
t.Fatal(err)
}
if tool.ran != 0 {
t.Errorf("the tool body ran %d times; the sandbox must refuse before execution", tool.ran)
}
if len(provider.requests) != toolFailureStopAt {
t.Errorf("the run made %d turns, want the category bound at %d; a categorized refusal whose prose varies must still halt on its category",
len(provider.requests), toolFailureStopAt)
}
if want := toolFailureStopAnswer("write_file", toolFailureStopAt, toolFailureCauseSameRefusal); result.FinalAnswer != want {
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
}
if !strings.Contains(result.FinalAnswer, "was refused") {
t.Errorf("the refusal provenance did not survive to the answer: %q", result.FinalAnswer)
}
}
@@ -0,0 +1,73 @@
package agent
import (
"strings"
"testing"
)
// THE ANSWER MUST CLAIM ONLY WHAT THE COUNTER ESTABLISHED.
//
// anyErrorCount establishes that the tool failed consecutively without a
// success. It does NOT establish that every failure differed: five A, five B and
// two C reach the content-blind bound of 12 without any signature repeating six
// times, and three of those errors were shared. The old wording called that
// "each with a different error".
//
// The signature bound has the inverse problem. A denial streak keys on the
// CATEGORY, and the prose behind it embeds the path or command refused, so it
// differs on every call. Calling that "the same error" is false in the other
// direction.
func TestStopAnswerDoesNotOverclaimTheFailurePattern(t *testing.T) {
varied := toolFailureStopAnswer("bash", 12, toolFailureCauseVariedExecuted)
if strings.Contains(varied, "each with a different error") {
t.Errorf("the content-blind bound does not track that every failure differed: %q", varied)
}
if !strings.Contains(varied, "varying errors") {
t.Errorf("the varied stop should still say the errors varied: %q", varied)
}
refused := toolFailureStopAnswer("bash", 6, toolFailureCauseSameRefusal)
if strings.Contains(refused, "same error") {
t.Errorf("a denial streak covers refusals of different paths, so it is not the same error: %q", refused)
}
if !strings.Contains(refused, "refused") {
t.Errorf("a denial streak should say it was refused: %q", refused)
}
// The one claim that IS justified: a signature streak really did repeat the
// same error signature, so that wording stays.
same := toolFailureStopAnswer("bash", 6, toolFailureCauseSameError)
if !strings.Contains(same, "same error") {
t.Errorf("a signature streak may still be described as the same error: %q", same)
}
}
// The mixed-signature case jatmn asked for, driven through the real counter
// rather than asserted about the wording in isolation: a run whose errors vary
// must trip the content-blind bound and must not be described as all-different.
func TestMixedSignatureStreakTripsTheContentBlindBound(t *testing.T) {
var state guardState
var outcome toolFailureOutcome
// Five of one error, five of another, two of a third: twelve failures, no
// signature repeating six times in a row.
for index, output := range []string{
"Error: A", "Error: A", "Error: A", "Error: A", "Error: A",
"Error: B", "Error: B", "Error: B", "Error: B", "Error: B",
"Error: C", "Error: C",
} {
outcome = state.observeToolResult("bash", true, true, output, DenialNone)
if outcome.Stop && index < 11 {
t.Fatalf("halted early at failure %d", index+1)
}
}
if !outcome.Stop {
t.Fatal("twelve consecutive failures did not trip the content-blind bound")
}
if outcome.Cause != toolFailureCauseVariedExecuted {
t.Errorf("a mixed-signature streak of executed failures should report varied-executed, got %v", outcome.Cause)
}
answer := toolFailureStopAnswer("bash", outcome.Count, outcome.Cause)
if strings.Contains(answer, "each with a different error") {
t.Errorf("three of these errors were shared, so they were not each different: %q", answer)
}
}
+173 -5
View File
@@ -1,13 +1,19 @@
package mcp
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"strings"
"sync"
"github.com/Twigpine/zero/internal/aimlapi"
"github.com/Twigpine/zero/internal/config"
"github.com/Twigpine/zero/internal/redaction"
)
// The AI/ML API MCP server authenticates with the user's AI/ML API key.
@@ -20,11 +26,17 @@ import (
// config.LoadAIMLAPIServiceKey, which in the same call decides whether it may
// be sent (no proxy profile, a single token), so a provider change applies to
// the next request and a withheld or invalid key means no Authorization
// header. It lives only in that request's headers: never in Server, its
// Identity (OAuth token binding), config.json, list output or errors (an
// error response to such a request is reported without its body). A
// user-configured Authorization header, OAuth, or a project-touched server
// always goes without it.
// header. It lives only in that request's headers, and for the length of the
// call in the collector its context carries (aimlapiSentKeys), so the reply can
// be scrubbed: never in Server, its Identity (OAuth token binding), config.json,
// list output, errors or tool output. Every key the call sent, redirect hops
// included, and on the SSE transport the keys the stream request sent, is
// scrubbed from the response's error text, method and result before anything
// reads them (scrubAIMLAPIKeys) and from any error the call returns, a
// transport error quoting a URL the server chose among them
// (scrubAIMLAPIKeyError). An error status to such a request is also reported
// without its body. A user-configured Authorization header, OAuth, or a
// project-touched server always goes without it.
var loadAIMLAPIServiceKey = config.LoadAIMLAPIServiceKey
var aimlapiMCPHost = func() string {
@@ -56,6 +68,9 @@ func applyAIMLAPIRequestHeaders(server Server, request *http.Request) {
return
}
request.Header.Set("Authorization", "Bearer "+key)
if sent, ok := request.Context().Value(aimlapiSentKeysContextKey{}).(*aimlapiSentKeys); ok {
sent.add(key)
}
}
// reapplyAIMLAPIRedirectHeaders gives a redirect hop the automatic headers its
@@ -101,6 +116,159 @@ func hasUserAuthorizationHeader(server Server) bool {
return false
}
// aimlapiSentKeys collects every key applyAIMLAPIRequestHeaders puts on the
// requests of one call. The first request's context carries it, and net/http
// gives each redirect hop that same context, so a key resolved again for a later
// hop is collected too. The final hop's key alone is not enough: a same-origin
// server answering the last hop has seen every earlier one.
type aimlapiSentKeys struct {
mu sync.Mutex
keys []string
}
type aimlapiSentKeysContextKey struct{}
// withAIMLAPISentKeys returns ctx carrying a fresh collector for one call.
func withAIMLAPISentKeys(ctx context.Context) (context.Context, *aimlapiSentKeys) {
sent := &aimlapiSentKeys{}
return context.WithValue(ctx, aimlapiSentKeysContextKey{}, sent), sent
}
func (sent *aimlapiSentKeys) add(key string) {
sent.mu.Lock()
defer sent.mu.Unlock()
for _, existing := range sent.keys {
if existing == key {
return
}
}
sent.keys = append(sent.keys, key)
}
// list returns a copy of the keys collected so far.
func (sent *aimlapiSentKeys) list() []string {
sent.mu.Lock()
defer sent.mu.Unlock()
return append([]string(nil), sent.keys...)
}
// scrubAIMLAPIKeys removes the injected keys from a response before any of it
// is read: the error text, the method name and every string in the result,
// object member names included. A server can echo the credential it was sent,
// and MCP reports a failed tool call as a result rather than an error, so the
// result needs it as much as the error text. Only the known values go, split
// forms included (redaction.RedactSecretValues); a result that holds none of
// them keeps its original bytes.
func scrubAIMLAPIKeys(message rpcMessage, keys ...string) rpcMessage {
var sent []string
for _, key := range keys {
if strings.TrimSpace(key) != "" {
sent = append(sent, key)
}
}
if len(sent) == 0 {
return message
}
if message.Error != nil {
scrubbed := *message.Error
scrubbed.Message = redaction.RedactSecretValues(scrubbed.Message, sent...)
message.Error = &scrubbed
}
message.Method = redaction.RedactSecretValues(message.Method, sent...)
message.Result = scrubAIMLAPIKeysFromJSON(message.Result, sent)
return message
}
func scrubAIMLAPIKeysFromJSON(raw json.RawMessage, keys []string) json.RawMessage {
if len(raw) == 0 {
return raw
}
decoder := json.NewDecoder(bytes.NewReader(raw))
decoder.UseNumber()
var value any
if err := decoder.Decode(&value); err != nil {
// Decoding it into the caller's type fails the same way, and that
// error does not quote the text.
return raw
}
scrubbed, changed := scrubAIMLAPIKeysFromValue(value, keys)
if !changed {
return raw
}
encoded, err := json.Marshal(scrubbed)
if err != nil {
// Never hand back the unscrubbed bytes: an empty result reads as
// nothing rather than as the key.
return nil
}
return encoded
}
func scrubAIMLAPIKeysFromValue(value any, keys []string) (any, bool) {
switch typed := value.(type) {
case string:
scrubbed := redaction.RedactSecretValues(typed, keys...)
return scrubbed, scrubbed != typed
case []any:
changed := false
for index, item := range typed {
if scrubbed, itemChanged := scrubAIMLAPIKeysFromValue(item, keys); itemChanged {
typed[index] = scrubbed
changed = true
}
}
return typed, changed
case map[string]any:
changed := false
scrubbed := make(map[string]any, len(typed))
for name, item := range typed {
scrubbedName := redaction.RedactSecretValues(name, keys...)
scrubbedItem, itemChanged := scrubAIMLAPIKeysFromValue(item, keys)
if itemChanged || scrubbedName != name {
changed = true
}
scrubbed[scrubbedName] = scrubbedItem
}
return scrubbed, changed
}
return value, false
}
// scrubAIMLAPIKeyError returns err with the injected keys removed from its text.
// An error the keys do not appear in keeps its chain. One they do appear in is
// replaced by its scrubbed text, because its chain still holds the key (a
// *url.Error keeps the URL the server chose); the replacement unwraps only to
// the context sentinel the original matched, so a cancellation still reads as
// one.
func scrubAIMLAPIKeyError(err error, keys ...string) error {
if err == nil || len(keys) == 0 {
return err
}
text := err.Error()
scrubbed := redaction.RedactSecretValues(text, keys...)
if scrubbed == text {
return err
}
replacement := &aimlapiScrubbedError{text: scrubbed}
for _, sentinel := range []error{context.Canceled, context.DeadlineExceeded} {
if errors.Is(err, sentinel) {
replacement.cause = sentinel
break
}
}
return replacement
}
// aimlapiScrubbedError is an error whose text had an injected key removed.
type aimlapiScrubbedError struct {
text string
cause error
}
func (scrubbed *aimlapiScrubbedError) Error() string { return scrubbed.text }
func (scrubbed *aimlapiScrubbedError) Unwrap() error { return scrubbed.cause }
// aimlapiUnauthorizedHint explains how to authenticate when the AI/ML API MCP
// server rejects a request without OAuth. It names no key material.
func aimlapiUnauthorizedHint(server Server, status int) string {
+658
View File
@@ -0,0 +1,658 @@
package mcp
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"sync"
"testing"
"time"
"github.com/Twigpine/zero/internal/aimlapi"
)
// scrubTestKey has no credential shape, so only the known-value scrub can find
// it; a shape pattern would leave it alone.
const scrubTestKey = "aiml0123456789abcdef0123456789abcd"
// aimlapiEcho stands in for mcp.aimlapi.com over streamable HTTP. It answers
// each JSON-RPC request with reply(method, id), as a JSON body or as one SSE
// event, and records the Authorization header every request carried.
type aimlapiEcho struct {
sse bool
reply func(method string, id json.RawMessage) string
mu sync.Mutex
auth []string
}
func (echo *aimlapiEcho) RoundTrip(request *http.Request) (*http.Response, error) {
var body []byte
if request.Body != nil {
body, _ = io.ReadAll(request.Body)
_ = request.Body.Close()
}
echo.mu.Lock()
echo.auth = append(echo.auth, request.Header.Get("Authorization"))
echo.mu.Unlock()
var message struct {
ID json.RawMessage `json:"id"`
Method string `json:"method"`
}
_ = json.Unmarshal(body, &message)
if len(message.ID) == 0 {
return fakeHTTPResponse(request, http.StatusAccepted, ""), nil
}
reply := echo.reply(message.Method, message.ID)
response := fakeHTTPResponse(request, http.StatusOK, reply)
if echo.sse {
response.Header.Set("Content-Type", "text/event-stream")
response.Body = io.NopCloser(strings.NewReader("event: message\ndata: " + reply + "\n\n"))
}
return response, nil
}
// assertSent fails unless every request carried want as its Authorization, so a
// test cannot pass by never reaching the injected-key path.
func (echo *aimlapiEcho) assertSent(t *testing.T, want string) {
t.Helper()
echo.mu.Lock()
defer echo.mu.Unlock()
if len(echo.auth) == 0 {
t.Fatal("the fake server was never called")
}
for _, got := range echo.auth {
if got != want {
t.Fatalf("Authorization = %q, want %q", got, want)
}
}
}
func rpcReply(id json.RawMessage, field string, value any) string {
encoded, err := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": id, field: value})
if err != nil {
panic(err)
}
return string(encoded)
}
// echoedKeys are the spellings of the key a server echoes back: whole, and
// split by a NUL and by a zero-width space, which a plain replace would miss.
func echoedKeys() []string {
return []string{
scrubTestKey,
scrubTestKey[:10] + string(rune(0)) + scrubTestKey[10:],
scrubTestKey[:20] + string(rune(0x200b)) + scrubTestKey[20:],
}
}
// assertNoPartOf fails if any eight-byte run of key survives in text.
func assertNoPartOf(t *testing.T, text string, key string) {
t.Helper()
for start := 0; start+8 <= len(key); start++ {
if strings.Contains(text, key[start:start+8]) {
t.Fatalf("text still holds part of the key (%q): %q", key[start:start+8], text)
}
}
}
// assertKeyScrubbed fails if any part of scrubTestKey survives in text, or if
// text does not show the replacement.
func assertKeyScrubbed(t *testing.T, text string) {
t.Helper()
assertNoPartOf(t, text, scrubTestKey)
if !strings.Contains(text, "[REDACTED]") {
t.Fatalf("text = %q, want the key replaced by [REDACTED]", text)
}
}
func TestScrubAIMLAPIKeysKeepsAResultWithoutTheKeyByteForByte(t *testing.T) {
raw := json.RawMessage(`{"tools": [ {"name": "b", "description": "password=hunter2"} ], "a": 1.50}`)
if got := scrubAIMLAPIKeysFromJSON(raw, []string{scrubTestKey}); string(got) != string(raw) {
t.Fatalf("result without the key = %s, want the original bytes %s", got, raw)
}
withKey := json.RawMessage(`{"content": [ {"type": "text", "text": "key ` + scrubTestKey + `"} ], "n": 1.50}`)
got := scrubAIMLAPIKeysFromJSON(withKey, []string{scrubTestKey})
assertKeyScrubbed(t, string(got))
var decoded struct {
Content []Content `json:"content"`
N json.Number `json:"n"`
}
if err := json.Unmarshal(got, &decoded); err != nil || len(decoded.Content) != 1 || decoded.Content[0].Text != "key [REDACTED]" || decoded.N != "1.50" {
t.Fatalf("scrubbed result = %s (%v), want valid JSON with only the key replaced and the number as written", got, err)
}
}
func newEchoNetworkClient(server Server, echo *aimlapiEcho) *networkClient {
return &networkClient{server: server, client: mcpHTTPClient(server, echo), nextID: 1}
}
func TestAIMLAPIMCPScrubsTheInjectedKeyFromResponses(t *testing.T) {
isolateAIMLAPIKeyLookup(t, scrubTestKey)
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
for _, sse := range []bool{false, true} {
transport := "json"
if sse {
transport = "sse"
}
t.Run(transport+"/error message", func(t *testing.T) {
for _, echoed := range echoedKeys() {
echo := &aimlapiEcho{sse: sse, reply: func(_ string, id json.RawMessage) string {
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "invalid credential " + echoed + " for this account"})
}}
err := newEchoNetworkClient(server, echo).initialize(context.Background())
if err == nil {
t.Fatal("initialize succeeded against a server that answered with an error")
}
echo.assertSent(t, "Bearer "+scrubTestKey)
assertKeyScrubbed(t, err.Error())
if !strings.Contains(err.Error(), "invalid credential [REDACTED] for this account") {
t.Fatalf("error = %q, want the rest of the server text kept", err)
}
}
})
t.Run(transport+"/tool result", func(t *testing.T) {
const image = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="
for _, echoed := range echoedKeys() {
echo := &aimlapiEcho{sse: sse, reply: func(_ string, id json.RawMessage) string {
return rpcReply(id, "result", map[string]any{
"isError": true,
"content": []any{
map[string]any{"type": "text", "text": "upstream rejected key " + echoed},
map[string]any{"type": "image", "mimeType": "image/png", "data": image},
},
})
}}
result, err := newEchoNetworkClient(server, echo).CallTool(context.Background(), "generate", map[string]any{"prompt": "x"})
if err != nil {
t.Fatalf("CallTool: %v", err)
}
echo.assertSent(t, "Bearer "+scrubTestKey)
if len(result.Content) != 2 || !result.IsError {
t.Fatalf("result = %+v, want the error result with both blocks", result)
}
assertKeyScrubbed(t, result.Content[0].Text)
if result.Content[0].Text != "upstream rejected key [REDACTED]" {
t.Fatalf("text block = %q, want only the key replaced", result.Content[0].Text)
}
if result.Content[1].Data != image {
t.Fatalf("image data changed: %q", result.Content[1].Data)
}
}
})
t.Run(transport+"/result without the key", func(t *testing.T) {
// Shapes stay as the server wrote them: only the known value goes.
const description = "Set password=hunter2 or pass Authorization: Bearer other-token"
echo := &aimlapiEcho{sse: sse, reply: func(_ string, id json.RawMessage) string {
return rpcReply(id, "result", map[string]any{"tools": []any{map[string]any{
"name": "generate",
"description": description,
"inputSchema": map[string]any{"type": "object", "properties": map[string]any{"api_key": map[string]any{"type": "string"}}},
}}})
}}
tools, err := newEchoNetworkClient(server, echo).ListTools(context.Background())
if err != nil {
t.Fatalf("ListTools: %v", err)
}
echo.assertSent(t, "Bearer "+scrubTestKey)
if len(tools) != 1 || tools[0].Description != description {
t.Fatalf("tools = %+v, want the description unchanged", tools)
}
properties, _ := tools[0].InputSchema["properties"].(map[string]any)
if _, ok := properties["api_key"]; !ok {
t.Fatalf("inputSchema = %v, want the api_key property kept", tools[0].InputSchema)
}
})
}
t.Run("json/unexpected method", func(t *testing.T) {
echo := &aimlapiEcho{reply: func(string, json.RawMessage) string {
return `{"jsonrpc":"2.0","method":"notifications/` + scrubTestKey + `"}`
}}
err := newEchoNetworkClient(server, echo).initialize(context.Background())
if err == nil || !strings.Contains(err.Error(), "got method") {
t.Fatalf("initialize error = %v, want the unexpected method reported", err)
}
echo.assertSent(t, "Bearer "+scrubTestKey)
assertKeyScrubbed(t, err.Error())
})
}
func TestAIMLAPIMCPLeavesResponsesAloneWithoutTheInjectedKey(t *testing.T) {
isolateAIMLAPIKeyLookup(t, scrubTestKey)
// A user-configured Authorization header means nothing was injected, so the
// server text is reported as written; this change scrubs only what Zero
// itself attached.
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
server.Headers = map[string]string{"Authorization": "Bearer " + scrubTestKey}
echo := &aimlapiEcho{reply: func(_ string, id json.RawMessage) string {
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "invalid credential " + scrubTestKey})
}}
err := newEchoNetworkClient(server, echo).initialize(context.Background())
echo.assertSent(t, "Bearer "+scrubTestKey)
if err == nil || !strings.Contains(err.Error(), "invalid credential "+scrubTestKey) {
t.Fatalf("error = %v, want the server text unchanged when no key was injected", err)
}
}
// aimlapiSSEEcho stands in for mcp.aimlapi.com over the legacy SSE transport:
// the GET opens a stream that starts with the endpoint event, and each POST is
// answered on that stream with reply(method, id).
type aimlapiSSEEcho struct {
endpoint string
reply func(method string, id json.RawMessage) string
// postErr, when set, fails every POST at the transport.
postErr error
// rawEvent, when set, writes its whole event to the stream instead of a
// message event built from reply.
rawEvent func(method string, id json.RawMessage) string
events chan string
mu sync.Mutex
auth []string
}
func newAIMLAPISSEEcho(endpoint string, reply func(string, json.RawMessage) string) *aimlapiSSEEcho {
return &aimlapiSSEEcho{endpoint: endpoint, reply: reply, events: make(chan string, 8)}
}
func (echo *aimlapiSSEEcho) RoundTrip(request *http.Request) (*http.Response, error) {
echo.mu.Lock()
echo.auth = append(echo.auth, request.Header.Get("Authorization"))
echo.mu.Unlock()
if request.Method == http.MethodGet {
reader, writer := io.Pipe()
go func() {
defer writer.Close()
if _, err := io.WriteString(writer, "event: endpoint\ndata: "+echo.endpoint+"\n\n"); err != nil {
return
}
for {
select {
case event := <-echo.events:
if _, err := io.WriteString(writer, event); err != nil {
return
}
case <-request.Context().Done():
return
}
}
}()
response := fakeHTTPResponse(request, http.StatusOK, "")
response.Header.Set("Content-Type", "text/event-stream")
response.Body = reader
return response, nil
}
if echo.postErr != nil {
return nil, echo.postErr
}
var body []byte
if request.Body != nil {
body, _ = io.ReadAll(request.Body)
_ = request.Body.Close()
}
var message struct {
ID json.RawMessage `json:"id"`
Method string `json:"method"`
}
_ = json.Unmarshal(body, &message)
if len(message.ID) > 0 {
if echo.rawEvent != nil {
echo.events <- echo.rawEvent(message.Method, message.ID)
} else {
echo.events <- "event: message\ndata: " + echo.reply(message.Method, message.ID) + "\n\n"
}
}
return fakeHTTPResponse(request, http.StatusAccepted, ""), nil
}
// rotateAIMLAPIKeys makes the key resolver hand out keys in order, repeating the
// last, the way a provider change between two requests of one call would.
func rotateAIMLAPIKeys(t *testing.T, keys ...string) {
t.Helper()
var mu sync.Mutex
previous := loadAIMLAPIServiceKey
loadAIMLAPIServiceKey = func() (string, error) {
mu.Lock()
defer mu.Unlock()
key := keys[0]
if len(keys) > 1 {
keys = keys[1:]
}
return key, nil
}
t.Cleanup(func() { loadAIMLAPIServiceKey = previous })
}
func (echo *aimlapiSSEEcho) assertSent(t *testing.T, want string, requests int) {
t.Helper()
echo.mu.Lock()
defer echo.mu.Unlock()
if len(echo.auth) < requests {
t.Fatalf("the fake server saw %d requests, want at least %d", len(echo.auth), requests)
}
for _, got := range echo.auth {
if got != want {
t.Fatalf("Authorization = %q, want %q", got, want)
}
}
}
func newEchoSSEClient(echo *aimlapiSSEEcho) *remoteSSEClient {
server := Server{Name: "aimlapi", Type: ServerTypeSSE, URL: aimlapi.MCPServerURL}
return &remoteSSEClient{
server: server,
client: mcpHTTPClient(server, echo),
nextID: 1,
pending: map[string]chan ssePendingResponse{},
}
}
func TestAIMLAPIMCPSSEScrubsTheInjectedKeyFromStreamAnswers(t *testing.T) {
isolateAIMLAPIKeyLookup(t, scrubTestKey)
for _, echoed := range echoedKeys() {
echo := newAIMLAPISSEEcho("/messages", func(method string, id json.RawMessage) string {
if method == "tools/call" {
return rpcReply(id, "result", map[string]any{"isError": true, "content": []any{map[string]any{"type": "text", "text": "rejected " + echoed}}})
}
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "invalid credential " + echoed})
})
client := newEchoSSEClient(echo)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
if err := client.openStream(ctx); err != nil {
cancel()
t.Fatalf("openStream: %v", err)
}
_, listErr := client.ListTools(ctx)
result, callErr := client.CallTool(ctx, "generate", map[string]any{})
_ = client.Close()
cancel()
echo.assertSent(t, "Bearer "+scrubTestKey, 3)
if listErr == nil {
t.Fatal("ListTools succeeded against a server that answered with an error")
}
assertKeyScrubbed(t, listErr.Error())
if callErr != nil || len(result.Content) != 1 {
t.Fatalf("CallTool = %+v, %v, want the error result", result, callErr)
}
assertKeyScrubbed(t, result.Content[0].Text)
}
}
func TestAIMLAPIMCPSSEScrubsTheStreamKeyAndThePostKey(t *testing.T) {
// The key is resolved for every request, so the stream and a later POST can
// carry different keys when the provider changes in between. An answer on
// the stream comes from a server that was sent both.
const streamKey = "strm9f8e7d6c5b4a39281706f5e4d3c2b1"
const postKey = "post1a2b3c4d5e6f708192a3b4c5d6e7f8"
rotateAIMLAPIKeys(t, streamKey, postKey)
echo := newAIMLAPISSEEcho("/messages", func(_ string, id json.RawMessage) string {
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "stream key " + streamKey + ", post key " + postKey})
})
client := newEchoSSEClient(echo)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := client.openStream(ctx); err != nil {
t.Fatalf("openStream: %v", err)
}
_, err := client.ListTools(ctx)
_ = client.Close()
echo.mu.Lock()
auth := append([]string(nil), echo.auth...)
echo.mu.Unlock()
if len(auth) != 2 || auth[0] != "Bearer "+streamKey || auth[1] != "Bearer "+postKey {
t.Fatalf("Authorization headers = %q, want the stream key and then the post key", auth)
}
if err == nil {
t.Fatal("ListTools succeeded against a server that answered with an error")
}
assertNoPartOf(t, err.Error(), streamKey)
assertNoPartOf(t, err.Error(), postKey)
if !strings.Contains(err.Error(), "stream key [REDACTED], post key [REDACTED]") {
t.Fatalf("error = %q, want both keys replaced", err)
}
}
func TestAIMLAPIMCPSSEScrubsTheInjectedKeyFromAnEndpointError(t *testing.T) {
isolateAIMLAPIKeyLookup(t, scrubTestKey)
// An invalid escape makes the endpoint fail to parse, and the parse error
// quotes the endpoint the server sent.
echo := newAIMLAPISSEEcho("/messages/"+scrubTestKey+"%zz", func(string, json.RawMessage) string { return "" })
client := newEchoSSEClient(echo)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
err := client.openStream(ctx)
_ = client.Close()
echo.assertSent(t, "Bearer "+scrubTestKey, 1)
if err == nil || !strings.Contains(err.Error(), "endpoint") {
t.Fatalf("openStream error = %v, want the endpoint parse failure", err)
}
assertKeyScrubbed(t, err.Error())
}
// aimlapiHopEcho answers the first request with a 307 to location and hands the
// second hop to second, recording the Authorization each hop carried.
type aimlapiHopEcho struct {
location string
second func(request *http.Request, id json.RawMessage) (*http.Response, error)
mu sync.Mutex
auth []string
}
func (echo *aimlapiHopEcho) RoundTrip(request *http.Request) (*http.Response, error) {
var body []byte
if request.Body != nil {
body, _ = io.ReadAll(request.Body)
_ = request.Body.Close()
}
echo.mu.Lock()
echo.auth = append(echo.auth, request.Header.Get("Authorization"))
hop := len(echo.auth)
echo.mu.Unlock()
if hop == 1 {
response := fakeHTTPResponse(request, http.StatusTemporaryRedirect, "")
response.Header.Set("Location", echo.location)
return response, nil
}
var message struct {
ID json.RawMessage `json:"id"`
}
_ = json.Unmarshal(body, &message)
return echo.second(request, message.ID)
}
func TestAIMLAPIMCPScrubsEveryKeySentAcrossRedirectHops(t *testing.T) {
// The key is resolved again for a redirect hop, so the hops of one call can
// carry different keys, and the same-origin server answering the last hop
// has seen them all.
const first = "aimlfirst0123456789abcdef01234567"
const second = "aimlsecond0123456789abcdef0123456"
for _, tc := range []struct {
name string
location string
// hopKey is the key the redirect hop must carry, "" for none.
hopKey string
second func(*http.Request, json.RawMessage) (*http.Response, error)
}{
{
name: "error echoes the first hop's key",
location: "https://mcp.aimlapi.com/mcp/",
hopKey: second,
second: func(request *http.Request, id json.RawMessage) (*http.Response, error) {
return fakeHTTPResponse(request, http.StatusOK, rpcReply(id, "error", map[string]any{"code": -32001, "message": "stale " + first})), nil
},
},
{
name: "status body after a hop that dropped the key",
location: "https://[email protected]/mcp",
second: func(request *http.Request, _ json.RawMessage) (*http.Response, error) {
return fakeHTTPResponse(request, http.StatusPaymentRequired, `{"error":"invalid credential `+first+`"}`), nil
},
},
{
name: "transport error quoting the redirect target",
location: "https://mcp.aimlapi.com/mcp?session=" + first,
hopKey: second,
second: func(*http.Request, json.RawMessage) (*http.Response, error) {
return nil, errors.New("connection reset by peer")
},
},
} {
t.Run(tc.name, func(t *testing.T) {
rotateAIMLAPIKeys(t, first, second)
echo := &aimlapiHopEcho{location: tc.location, second: tc.second}
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
client := &networkClient{server: server, client: mcpHTTPClient(server, echo), nextID: 1}
err := client.initialize(context.Background())
echo.mu.Lock()
auth := append([]string(nil), echo.auth...)
echo.mu.Unlock()
if len(auth) != 2 || auth[0] != "Bearer "+first {
t.Fatalf("Authorization per hop = %q, want the first key on the first request and a redirect hop", auth)
}
if tc.hopKey != "" && auth[1] != "Bearer "+tc.hopKey {
t.Fatalf("redirect hop Authorization = %q, want the key resolved again for it", auth[1])
}
// With no key, net/http fills Basic auth from the hop's own userinfo.
if tc.hopKey == "" && (strings.Contains(auth[1], first) || strings.Contains(auth[1], second)) {
t.Fatalf("redirect hop Authorization = %q, want no automatic key", auth[1])
}
if err == nil {
t.Fatal("initialize succeeded against a failing server")
}
assertNoPartOf(t, err.Error(), first)
assertNoPartOf(t, err.Error(), second)
if !strings.Contains(err.Error(), "[REDACTED]") {
t.Fatalf("error = %q, want the echoed key replaced", err)
}
})
}
}
func TestAIMLAPIMCPSSEScrubsTheStreamKeyFromAFailedPost(t *testing.T) {
// The server chose an endpoint holding the key its stream request carried.
// A POST to it that fails at the transport quotes that URL, and the POST
// itself carried a different key, so only the stream's keys can scrub it.
const streamKey = "strm0a1b2c3d4e5f60718293a4b5c6d7e8"
const postKey = "post9e8d7c6b5a4f30291807f6e5d4c3b2"
rotateAIMLAPIKeys(t, streamKey, postKey)
echo := newAIMLAPISSEEcho("/messages/"+streamKey, func(string, json.RawMessage) string { return "" })
echo.postErr = errors.New("connection reset by peer")
client := newEchoSSEClient(echo)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := client.openStream(ctx); err != nil {
t.Fatalf("openStream: %v", err)
}
_, err := client.ListTools(ctx)
_ = client.Close()
echo.mu.Lock()
auth := append([]string(nil), echo.auth...)
echo.mu.Unlock()
if len(auth) != 2 || auth[0] != "Bearer "+streamKey || auth[1] != "Bearer "+postKey {
t.Fatalf("Authorization headers = %q, want the stream key and then the post key", auth)
}
if err == nil || !strings.Contains(err.Error(), "post MCP SSE message") {
t.Fatalf("ListTools error = %v, want the failed POST", err)
}
assertNoPartOf(t, err.Error(), streamKey)
assertNoPartOf(t, err.Error(), postKey)
if !strings.Contains(err.Error(), "/messages/[REDACTED]") {
t.Fatalf("error = %q, want the endpoint quoted with the key replaced", err)
}
}
func TestAIMLAPIMCPSSEScrubsTheKeyFromALaterEndpointError(t *testing.T) {
// An endpoint event can arrive after the stream is open. Its parse error
// then fails the pending requests directly rather than returning from
// openStream, so it has to be scrubbed where it is made.
isolateAIMLAPIKeyLookup(t, scrubTestKey)
echo := newAIMLAPISSEEcho("/messages", nil)
echo.rawEvent = func(string, json.RawMessage) string {
return "event: endpoint\ndata: /messages/" + scrubTestKey + "%zz\n\n"
}
client := newEchoSSEClient(echo)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := client.openStream(ctx); err != nil {
t.Fatalf("openStream: %v", err)
}
_, err := client.ListTools(ctx)
_ = client.Close()
echo.assertSent(t, "Bearer "+scrubTestKey, 2)
if err == nil || !strings.Contains(err.Error(), "endpoint") {
t.Fatalf("ListTools error = %v, want the later endpoint's parse failure", err)
}
assertKeyScrubbed(t, err.Error())
}
func TestAIMLAPIMCPSSEScrubsKeysFromAFailedStreamRedirect(t *testing.T) {
// The stream request follows redirects like any other, so a hop that fails
// at the transport quotes the target the server chose, which can hold the
// key the first hop carried.
const first = "strmfirst0123456789abcdef01234567"
const second = "strmsecond0123456789abcdef0123456"
rotateAIMLAPIKeys(t, first, second)
echo := &aimlapiHopEcho{
location: "https://mcp.aimlapi.com/mcp?session=" + first,
second: func(*http.Request, json.RawMessage) (*http.Response, error) {
return nil, errors.New("connection reset by peer")
},
}
server := Server{Name: "aimlapi", Type: ServerTypeSSE, URL: aimlapi.MCPServerURL}
client := &remoteSSEClient{server: server, client: mcpHTTPClient(server, echo), nextID: 1, pending: map[string]chan ssePendingResponse{}}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
err := client.openStream(ctx)
_ = client.Close()
echo.mu.Lock()
auth := append([]string(nil), echo.auth...)
echo.mu.Unlock()
if len(auth) != 2 || auth[0] != "Bearer "+first || auth[1] != "Bearer "+second {
t.Fatalf("Authorization per hop = %q, want the first key and then the second", auth)
}
if err == nil || !strings.Contains(err.Error(), "open MCP SSE stream") {
t.Fatalf("openStream error = %v, want the failed stream request", err)
}
assertNoPartOf(t, err.Error(), first)
assertNoPartOf(t, err.Error(), second)
if !strings.Contains(err.Error(), "session=[REDACTED]") {
t.Fatalf("error = %q, want the redirect target quoted with the key replaced", err)
}
}
func TestScrubAIMLAPIKeyErrorKeepsOnlyTheContextSentinel(t *testing.T) {
for _, sentinel := range []error{context.Canceled, context.DeadlineExceeded} {
original := fmt.Errorf("post MCP SSE message to aimlapi: %w", &url.Error{Op: "Post", URL: "https://mcp.aimlapi.com/messages/" + scrubTestKey, Err: sentinel})
scrubbed := scrubAIMLAPIKeyError(original, scrubTestKey)
assertKeyScrubbed(t, scrubbed.Error())
if !errors.Is(scrubbed, sentinel) {
t.Fatalf("scrubbed error %q no longer reads as %v", scrubbed, sentinel)
}
if unwrapped := errors.Unwrap(scrubbed); unwrapped != sentinel {
t.Fatalf("scrubbed error unwraps to %v, want only %v", unwrapped, sentinel)
}
var urlErr *url.Error
if errors.As(scrubbed, &urlErr) {
t.Fatalf("scrubbed error still exposes the *url.Error holding the key: %v", urlErr.URL)
}
}
if scrubbed := scrubAIMLAPIKeyError(fmt.Errorf("decode: %s", scrubTestKey), scrubTestKey); errors.Unwrap(scrubbed) != nil {
t.Fatalf("scrubbed error without a context sentinel unwraps to %v, want nothing", errors.Unwrap(scrubbed))
}
untouched := fmt.Errorf("call MCP: %w", context.Canceled)
if got := scrubAIMLAPIKeyError(untouched, scrubTestKey); got != untouched {
t.Fatalf("an error without the key was replaced: %v", got)
}
}
+49 -18
View File
@@ -35,6 +35,11 @@ type remoteSSEClient struct {
pending map[string]chan ssePendingResponse
streamErr error
closed bool
// streamKeys are the injected AI/ML API keys the stream request carried,
// redirect hops included. openStream sets them once, before the reader
// starts; the endpoint and the messages on the stream come from a server
// that was sent them.
streamKeys []string
}
type ssePendingResponse struct {
@@ -259,12 +264,13 @@ func (client *remoteSSEClient) request(ctx context.Context, method string, param
client.pending[key] = responseChannel
client.mu.Unlock()
if err := client.post(ctx, rpcMessage{
sentKeys, err := client.post(ctx, rpcMessage{
JSONRPC: "2.0",
ID: id,
Method: method,
Params: rawParams,
}); err != nil {
})
if err != nil {
client.removePending(key)
return err
}
@@ -274,14 +280,15 @@ func (client *remoteSSEClient) request(ctx context.Context, method string, param
if response.err != nil {
return response.err
}
if !rpcIDMatches(response.message.ID, id) {
message := scrubAIMLAPIKeys(response.message, append(append([]string(nil), client.streamKeys...), sentKeys...)...)
if !rpcIDMatches(message.ID, id) {
return fmt.Errorf("MCP %s response id mismatch for server %s", method, client.server.Name)
}
if response.message.Error != nil {
return fmt.Errorf("MCP %s failed: %s", method, response.message.Error.Message)
if message.Error != nil {
return fmt.Errorf("MCP %s failed: %s", method, message.Error.Message)
}
if target != nil && len(response.message.Result) > 0 {
if err := json.Unmarshal(response.message.Result, target); err != nil {
if target != nil && len(message.Result) > 0 {
if err := json.Unmarshal(message.Result, target); err != nil {
return fmt.Errorf("decode MCP %s result: %w", method, err)
}
}
@@ -320,11 +327,12 @@ func (client *remoteSSEClient) notify(ctx context.Context, method string, params
if err != nil {
return err
}
return client.post(ctx, rpcMessage{
_, err = client.post(ctx, rpcMessage{
JSONRPC: "2.0",
Method: method,
Params: rawParams,
})
return err
}
func (client *networkClient) post(ctx context.Context, message rpcMessage, expectResponse bool) (result rpcMessage, err error) {
@@ -332,6 +340,11 @@ func (client *networkClient) post(ctx context.Context, message rpcMessage, expec
if err != nil {
return rpcMessage{}, err
}
ctx, sent := withAIMLAPISentKeys(ctx)
// From here an error can quote the server or a URL it chose, a redirect
// target among them, so every key the call sent, on any hop, is scrubbed
// from it. This runs last, after the body is closed.
defer func() { err = scrubAIMLAPIKeyError(err, sent.list()...) }()
request, err := http.NewRequestWithContext(ctx, http.MethodPost, client.server.URL, bytes.NewReader(body))
if err != nil {
return rpcMessage{}, fmt.Errorf("create MCP %s request for %s: %w", client.server.Type, client.server.Name, err)
@@ -357,12 +370,18 @@ func (client *networkClient) post(ctx context.Context, message rpcMessage, expec
if response.StatusCode == http.StatusNoContent {
return rpcMessage{}, fmt.Errorf("MCP %s server %s returned no response body", client.server.Type, client.server.Name)
}
return client.decodeResponse(response)
decoded, err := client.decodeResponse(response)
if err != nil {
return rpcMessage{}, err
}
return scrubAIMLAPIKeys(decoded, sent.list()...), nil
}
func (client *remoteSSEClient) openStream(ctx context.Context) error {
func (client *remoteSSEClient) openStream(ctx context.Context) (err error) {
streamCtx, cancel := context.WithCancel(ctx)
client.streamCancel = cancel
streamCtx, sent := withAIMLAPISentKeys(streamCtx)
defer func() { err = scrubAIMLAPIKeyError(err, sent.list()...) }()
request, err := http.NewRequestWithContext(streamCtx, http.MethodGet, client.server.URL, nil)
if err != nil {
@@ -380,6 +399,7 @@ func (client *remoteSSEClient) openStream(ctx context.Context) error {
return err
}
client.streamBody = response.Body
client.streamKeys = sent.list()
ready := make(chan error, 1)
go client.readStream(response.Body, ready)
@@ -391,31 +411,41 @@ func (client *remoteSSEClient) openStream(ctx context.Context) error {
}
}
func (client *remoteSSEClient) post(ctx context.Context, message rpcMessage) (err error) {
// post sends message to the endpoint and returns the injected AI/ML API keys the
// request carried, redirect hops included, since the answer arrives on the
// stream from a server that was sent them. The endpoint is the server's own
// choice and can hold one of the stream's keys, and a transport error quotes it,
// so the stream's keys are scrubbed from any error as well as the POST's.
func (client *remoteSSEClient) post(ctx context.Context, message rpcMessage) (sentKeys []string, err error) {
ctx, sent := withAIMLAPISentKeys(ctx)
defer func() {
sentKeys = sent.list()
err = scrubAIMLAPIKeyError(err, append(append([]string(nil), client.streamKeys...), sentKeys...)...)
}()
endpointURL, err := client.currentEndpoint()
if err != nil {
return err
return nil, err
}
body, err := json.Marshal(message)
if err != nil {
return err
return nil, err
}
request, err := http.NewRequestWithContext(ctx, http.MethodPost, endpointURL, bytes.NewReader(body))
if err != nil {
return fmt.Errorf("create MCP SSE post for %s: %w", client.server.Name, err)
return nil, fmt.Errorf("create MCP SSE post for %s: %w", client.server.Name, err)
}
client.applyPostHeaders(request)
response, err := client.client.Do(request)
if err != nil {
return fmt.Errorf("post MCP SSE message to %s: %w", client.server.Name, err)
return nil, fmt.Errorf("post MCP SSE message to %s: %w", client.server.Name, err)
}
defer closeResponseBody(&err, client.server, response.Body)
if response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
return httpStatusError(client.server, response)
return nil, httpStatusError(client.server, response)
}
_, _ = io.Copy(io.Discard, response.Body)
return nil
return nil, nil
}
func (client *networkClient) applyHeaders(request *http.Request) {
@@ -526,7 +556,8 @@ func (client *remoteSSEClient) readStream(reader io.Reader, ready chan<- error)
func (client *remoteSSEClient) setEndpoint(value string) error {
endpointURL, err := resolveSSEEndpointURL(client.server.URL, strings.TrimSpace(value))
if err != nil {
return err
// A URL parse error quotes the endpoint, which is the server's text.
return scrubAIMLAPIKeyError(err, client.streamKeys...)
}
client.mu.Lock()
client.endpointURL = endpointURL
@@ -0,0 +1,14 @@
package observability
import (
"os"
"github.com/Twigpine/zero/internal/pathjail"
)
// renameNoReplace publishes a crash report without replacing an existing one.
// The implementation lives in pathjail because the file tools need the same
// atomic no-replace guarantee.
func renameNoReplace(root *os.Root, oldname, newname string) error {
return pathjail.RenameNoReplace(root, oldname, newname)
}
@@ -1,19 +0,0 @@
//go:build darwin
package observability
import (
"fmt"
"os"
"golang.org/x/sys/unix"
)
func renameNoReplace(root *os.Root, oldname, newname string) error {
directory, err := root.Open(".")
if err != nil {
return fmt.Errorf("open crash directory for publication: %w", err)
}
defer directory.Close()
return unix.RenameatxNp(int(directory.Fd()), oldname, int(directory.Fd()), newname, unix.RENAME_EXCL)
}
@@ -1,19 +0,0 @@
//go:build linux
package observability
import (
"fmt"
"os"
"golang.org/x/sys/unix"
)
func renameNoReplace(root *os.Root, oldname, newname string) error {
directory, err := root.Open(".")
if err != nil {
return fmt.Errorf("open crash directory for publication: %w", err)
}
defer directory.Close()
return unix.Renameat2(int(directory.Fd()), oldname, int(directory.Fd()), newname, unix.RENAME_NOREPLACE)
}
+12 -2
View File
@@ -142,12 +142,22 @@ func trimTrailingSeparators(path string) string {
var randomBytes = rand.Read
// CreateTemp makes an O_EXCL temp file named prefix.<random>suffix inside dir,
// relative to handle, and returns it with its relative path.
// relative to handle, and returns it with its relative path. The file is
// created 0o600 so bytes staged in it are not readable before publication.
//
// O_EXCL rather than a fixed name: a link planted at a predictable temp path
// between the check and the create is the same race this package exists to
// close, and O_EXCL makes the kernel refuse it rather than us.
func CreateTemp(handle *os.Root, dir, prefix, suffix string) (*os.File, string, error) {
return CreateTempMode(handle, dir, prefix, suffix, 0o600)
}
// CreateTempMode is CreateTemp with an explicit create mode. The mode is
// passed to the kernel's exclusive create, so the process umask still applies;
// callers that must publish the permissions a normal create would have been
// granted should read the returned file's mode rather than assigning perm
// later, because Chmod does not consult the umask.
func CreateTempMode(handle *os.Root, dir, prefix, suffix string, perm os.FileMode) (*os.File, string, error) {
// prefix and suffix are NAME fragments, not path fragments. A separator or a
// "."/".." in either would relocate the file out of dir — a prefix of
// "../escaped", or a suffix of ".tmp/../evil" that also erases the random
@@ -164,7 +174,7 @@ func CreateTemp(handle *os.Root, dir, prefix, suffix string) (*os.File, string,
return nil, "", fmt.Errorf("generate a temporary name: %w", err)
}
relative := filepath.Join(dir, prefix+"."+hex.EncodeToString(noise)+suffix)
file, err := handle.OpenFile(relative, os.O_RDWR|os.O_CREATE|os.O_EXCL, 0o600)
file, err := handle.OpenFile(relative, os.O_RDWR|os.O_CREATE|os.O_EXCL, perm)
if err == nil {
return file, relative, nil
}
+28
View File
@@ -0,0 +1,28 @@
package pathjail
import (
"fmt"
"os"
"path/filepath"
)
// RenameNoReplace renames oldname to newname within the directory bound by
// root, and refuses to replace an existing newname with the same atomic
// guarantee the kernel gives a create's exclusive open. A caller that loses
// the race receives fs.ErrExist and nothing at newname changes.
//
// Both names must be single path components: they are resolved relative to
// root's directory handle, so a separator (or an ancestor link) must not be
// able to redirect the syscall outside the checked root. Callers with a path
// in a subdirectory open that directory with Root.OpenRoot first.
//
// Platforms whose kernel cannot publish without replacing fall back to
// ErrUnsupported, which lets the caller use a no-replace hard link instead.
func RenameNoReplace(root *os.Root, oldname, newname string) error {
for _, name := range []string{oldname, newname} {
if name == "" || name == "." || name == ".." || filepath.Base(name) != name {
return fmt.Errorf("%w: rename name %q must be a single path component", ErrEscapes, name)
}
}
return renameNoReplace(root, oldname, newname)
}
@@ -0,0 +1,22 @@
//go:build darwin
package pathjail
import (
"fmt"
"os"
"golang.org/x/sys/unix"
)
func renameNoReplace(root *os.Root, oldname, newname string) error {
directory, err := root.Open(".")
if err != nil {
return fmt.Errorf("open directory for rename: %w", err)
}
defer directory.Close()
if err := unix.RenameatxNp(int(directory.Fd()), oldname, int(directory.Fd()), newname, unix.RENAME_EXCL); err != nil {
return fmt.Errorf("rename %s to %s: %w", oldname, newname, err)
}
return nil
}
@@ -0,0 +1,22 @@
//go:build linux
package pathjail
import (
"fmt"
"os"
"golang.org/x/sys/unix"
)
func renameNoReplace(root *os.Root, oldname, newname string) error {
directory, err := root.Open(".")
if err != nil {
return fmt.Errorf("open directory for rename: %w", err)
}
defer directory.Close()
if err := unix.Renameat2(int(directory.Fd()), oldname, int(directory.Fd()), newname, unix.RENAME_NOREPLACE); err != nil {
return fmt.Errorf("rename %s to %s: %w", oldname, newname, err)
}
return nil
}
@@ -1,6 +1,6 @@
//go:build !darwin && !linux && !windows
package observability
package pathjail
import (
"errors"
@@ -0,0 +1,79 @@
package pathjail
import (
"errors"
"os"
"path/filepath"
"testing"
)
func openRenameRoot(t *testing.T) (*os.Root, string) {
t.Helper()
base := t.TempDir()
root, err := os.OpenRoot(base)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = root.Close() })
return root, base
}
// A CREATE'S NAME MUST NOT BE LOST TO A RACE. The no-replace rename reports the
// collision as fs.ErrExist and leaves both files exactly as they were, so the
// caller can refuse the create instead of replacing the other writer.
func TestRenameNoReplaceRefusesAnExistingDestination(t *testing.T) {
root, base := openRenameRoot(t)
if err := os.WriteFile(filepath.Join(base, "source"), []byte("source\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(base, "target"), []byte("target\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := RenameNoReplace(root, "source", "target"); !errors.Is(err, os.ErrExist) {
t.Fatalf("RenameNoReplace over an existing name = %v, want os.ErrExist", err)
}
for name, want := range map[string]string{"source": "source\n", "target": "target\n"} {
got, err := os.ReadFile(filepath.Join(base, name))
if err != nil || string(got) != want {
t.Fatalf("%s = %q, err=%v; a refused rename must not move either file", name, got, err)
}
}
}
func TestRenameNoReplacePublishesAnAbsentDestination(t *testing.T) {
root, base := openRenameRoot(t)
if err := os.WriteFile(filepath.Join(base, "source"), []byte("whole\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := RenameNoReplace(root, "source", "target"); err != nil {
t.Fatalf("RenameNoReplace = %v", err)
}
if got, err := os.ReadFile(filepath.Join(base, "target")); err != nil || string(got) != "whole\n" {
t.Fatalf("target = %q, err=%v", got, err)
}
if _, err := os.Stat(filepath.Join(base, "source")); !os.IsNotExist(err) {
t.Fatalf("source survived a successful rename: %v", err)
}
}
// The syscall is given a directory handle and bare names. Allowing a
// separator here would let an ancestor component resolve outside the directory
// the caller checked, so it is refused before the syscall.
func TestRenameNoReplaceRejectsPathComponents(t *testing.T) {
root, _ := openRenameRoot(t)
for _, tc := range []struct{ oldname, newname string }{
{"", "target"},
{"source", ""},
{".", "target"},
{"source", ".."},
{"dir/source", "target"},
{"source", "dir/target"},
{filepath.Join("dir", "source"), "target"},
} {
if err := RenameNoReplace(root, tc.oldname, tc.newname); !errors.Is(err, ErrEscapes) {
t.Errorf("RenameNoReplace(%q, %q) = %v, want ErrEscapes", tc.oldname, tc.newname, err)
}
}
}
@@ -1,6 +1,6 @@
//go:build windows
package observability
package pathjail
import (
"errors"
@@ -0,0 +1,40 @@
package redaction
import "testing"
func TestRedactSecretValuesRemovesOnlyTheKnownValues(t *testing.T) {
const key = "aiml-0123456789abcdef0123456789abcdef"
nul := string(rune(0))
zeroWidthSpace := string(rune(0x200b))
for _, tc := range []struct {
name string
value string
want string
// shaped marks text RedactString would rewrite by shape alone, so the
// case shows that RedactSecretValues leaves shapes to RedactString.
shaped bool
}{
{name: "whole", value: "invalid key " + key + " for this account", want: "invalid key " + RedactedSecret + " for this account"},
{name: "bearer form", value: "got Bearer " + key, want: "got Bearer " + RedactedSecret},
{name: "split by NUL", value: "invalid key " + key[:9] + nul + key[9:], want: "invalid key " + RedactedSecret},
{name: "split by a zero-width space", value: "value " + key[:20] + zeroWidthSpace + key[20:] + ".", want: "value " + RedactedSecret + "."},
{name: "password assignment", value: "password=hunter2", want: "password=hunter2", shaped: true},
{name: "another bearer token", value: "Authorization: Bearer some-other-token", want: "Authorization: Bearer some-other-token", shaped: true},
{name: "openai shaped key", value: "use sk-proj-abcdefghijklmnopqrstuvwxyz0123456789", want: "use sk-proj-abcdefghijklmnopqrstuvwxyz0123456789", shaped: true},
} {
t.Run(tc.name, func(t *testing.T) {
if got := RedactSecretValues(tc.value, key); got != tc.want {
t.Fatalf("RedactSecretValues(%q) = %q, want %q", tc.value, got, tc.want)
}
if tc.shaped && RedactString(tc.value, Options{}) == tc.value {
t.Fatalf("RedactString(%q) left it unchanged, so this case does not show the shape patterns are off", tc.value)
}
})
}
for _, secrets := range [][]string{nil, {""}, {" "}} {
if got := RedactSecretValues("token "+key, secrets...); got != "token "+key {
t.Fatalf("RedactSecretValues with secrets %q = %q, want the text unchanged", secrets, got)
}
}
}
+13
View File
@@ -628,6 +628,19 @@ func secretMatchHasDigit(s string) bool {
return false
}
// RedactSecretValues replaces every occurrence of each of secrets in value with
// RedactedSecret, whole or split by invisible separators, exactly as
// RedactString treats Options.ExtraSecretValues. Unlike RedactString it runs
// none of the shape patterns, so text that does not contain one of the values
// comes back unchanged. That suits content where only known values must go,
// such as tool output, which a shape pattern could otherwise rewrite.
func RedactSecretValues(value string, secrets ...string) string {
if len(secrets) == 0 {
return value
}
return redactExtraSecretValues(value, secrets, RedactedSecret)
}
func RedactValue(value any, options Options) any {
return redactReflect(reflect.ValueOf(value), redactionContext{
options: options,
+332 -28
View File
@@ -5,20 +5,63 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"strings"
"syscall"
"github.com/Twigpine/zero/internal/pathjail"
)
var errFileChangedDuringWrite = errors.New("file changed on disk before the write committed")
// errFilePublishedWithCleanupFailure marks an error after publication. Callers
// must record the committed bytes while still reporting the cleanup failure.
var errFilePublishedWithCleanupFailure = errors.New("file published, but staging cleanup failed")
// fileWriteBeforeCommit is a deterministic test hook. Production leaves it
// nil; tests use it to replace a path after observation but before opening the
// object that will actually be mutated.
var fileWriteBeforeCommit func(path string)
// fileWriteBeforePublish is a deterministic test hook that runs after the
// replacement bytes have been staged and before the destination's identity is
// re-checked for the publish. Tests use it to install a competing writer at the
// last point before publication.
var fileWriteBeforePublish func(path string)
// fileWriteStat is a deterministic test seam for proving that the opened-file
// identity is captured before the final preimage comparison. Production uses
// the file descriptor directly.
var fileWriteStat = func(file *os.File) (os.FileInfo, error) { return file.Stat() }
// fileWriteTempWrite is a deterministic test seam for a failure while the
// replacement bytes are written to the side file, after the observed file's
// identity and preimage have been accepted. Production writes through
// io.WriteString. The failure it stands in for (a full disk, a failing
// device) is exactly the moment a truncate-then-write has already destroyed
// the observed file.
var fileWriteTempWrite = func(file *os.File, content string) error {
_, err := io.WriteString(file, content)
return err
}
// fileWriteRename publishes the staged side file over the destination.
// Tests replace it to exercise refused atomic publication deterministically.
var fileWriteRename = func(root *os.Root, oldName, newName string) error {
return root.Rename(oldName, newName)
}
// fileWriteNoReplaceRename publishes a create without replacing an existing
// name. Tests replace it to exercise the hard-link fallback that filesystems
// without a no-replace rename use.
var fileWriteNoReplaceRename = publishRootedNoReplace
// fileWriteRemove removes a staging name. Tests inject cleanup failures after
// publication to distinguish a committed create from a failed write.
var fileWriteRemove = func(root *os.Root, name string) error {
return root.Remove(name)
}
// commitRootedFileContents binds an overwrite to the file identity and bytes
// that the caller observed, opening every component through an already-open
// *os.Root descriptor rather than re-resolving the absolute path. Because the
@@ -26,11 +69,26 @@ var fileWriteStat = func(file *os.File) (os.FileInfo, error) { return file.Stat(
// swapped for a symlink that escapes the workspace between validation and this
// call is refused by the kernel instead of redirecting the write.
//
// A create uses exclusive creation THROUGH THE ROOT. An overwrite opens the
// observed object without truncation, verifies identity/content through that
// handle, then truncates and writes the same handle. relativePath is the target
// expressed relative to root; absolutePath is retained only for the test hook
// and error reporting.
// Both publishes are atomic against a concurrent reader: the replacement bytes
// are written whole to a side file in the target's directory and the target is
// swapped wholesale. A create publishes through a kernel no-replace rename, so
// losing the create race refuses instead of replacing the raced file. An
// overwrite publishes through a rename after re-checking that the destination
// still names the observed object; that check is the last operation before the
// rename, but no supported platform offers a compare-and-swap rename, so a
// writer that replaces the name between the check and the syscall can still be
// replaced. A failure at any point before the publish removes the side file
// and leaves the observed bytes standing — the caller's file outlives a write
// that could not complete, and no partial state is ever visible at the path
// itself.
//
// A final-component symlink keeps main's behaviour: the link stays a link and
// the resolved file is what is replaced. A hard link is detached by the rename
// instead of rewritten through the shared inode. If the platform refuses the
// rename, the overwrite fails and the observed file remains intact.
//
// relativePath is the target expressed relative to root; absolutePath is
// retained only for the test hooks and error reporting.
//
// expectedInfo nil means the caller observed a missing path. expectedContent
// may be nil for an existing but unreadable file; that path may still be
@@ -41,16 +99,7 @@ func commitRootedFileContents(root *os.Root, absolutePath, relativePath string,
}
if expectedInfo == nil {
file, err := root.OpenFile(relativePath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
if err != nil {
return err
}
openedInfo, err := fileWriteStat(file)
if err != nil {
_ = file.Close()
return err
}
return writeAndVerifyRootedFileIdentity(root, relativePath, file, openedInfo, content, false)
return commitRootedFileCreate(root, relativePath, content)
}
flags := os.O_WRONLY
@@ -86,24 +135,239 @@ func commitRootedFileContents(root *os.Root, absolutePath, relativePath string,
return errFileChangedDuringWrite
}
}
return writeAndVerifyRootedFileIdentity(root, relativePath, file, openedInfo, content, true)
}
func writeAndVerifyRootedFileIdentity(root *os.Root, relativePath string, file *os.File, openedInfo os.FileInfo, content string, truncate bool) error {
if truncate {
if err := file.Truncate(0); err != nil {
_ = file.Close()
return err
}
if _, err := file.Seek(0, io.SeekStart); err != nil {
_ = file.Close()
return err
}
// A device, FIFO or socket is not a file this path should replace with a
// regular one, and the previous in-place write is the only publish that
// keeps its semantics. Regular files (including symlink targets) go through
// staging and rename.
if !openedInfo.Mode().IsRegular() {
return writeRootedFileInPlace(root, relativePath, file, openedInfo, expectedContent, content)
}
if _, err := io.WriteString(file, content); err != nil {
mode := openedInfo.Mode().Perm()
publishPath, err := resolveRootedFinalPath(root, relativePath)
if err != nil {
_ = file.Close()
return err
}
if publishPath != relativePath {
targetInfo, statErr := root.Stat(publishPath)
if statErr != nil || !os.SameFile(openedInfo, targetInfo) {
_ = file.Close()
return fmt.Errorf("%w: link target changed", errFileChangedDuringWrite)
}
}
if err := file.Close(); err != nil {
return err
}
return commitRootedFileOverwrite(root, absolutePath, relativePath, publishPath, openedInfo, mode, content)
}
// commitRootedFileCreate publishes a brand-new file whose absence the caller
// observed. The bytes are written whole to a private side file beside the
// target and handed to the name with a kernel no-replace rename: the create is
// either absent or the whole file, a concurrent create is refused rather than
// replaced, and a write that cannot complete leaves no file at all.
func commitRootedFileCreate(root *os.Root, relativePath, content string) error {
tempName, err := writeRootedSideFile(root, relativePath, 0o644, true, content)
if err != nil {
return err
}
if err := publishRootedCreate(root, tempName, relativePath); err != nil {
if errors.Is(err, errFilePublishedWithCleanupFailure) {
return err
}
return errors.Join(err, removeWriteSideFile(root, tempName))
}
return nil
}
// publishRootedCreate hands the staged side file to a name that must not
// already exist. RenameNoReplace is the primary primitive; a hard link is the
// no-replace fallback for filesystems whose rename cannot promise it. Neither
// fallback copies into a visible target, so a reader sees only the absent name
// or the complete file.
func publishRootedCreate(root *os.Root, tempName, relativePath string) error {
renameErr := fileWriteNoReplaceRename(root, tempName, relativePath)
if renameErr == nil {
return nil
}
if errors.Is(renameErr, os.ErrExist) {
return fmt.Errorf("creating %s: %w", relativePath, renameErr)
}
if !renameNoReplaceUnsupported(renameErr) {
return fmt.Errorf("creating %s: %w", relativePath, renameErr)
}
if linkErr := root.Link(tempName, relativePath); linkErr != nil {
return fmt.Errorf("creating %s: %w", relativePath, errors.Join(renameErr, linkErr))
}
// The staging name now shares the published inode. Remove only the name,
// without chmod-based retries that could change the published file's mode.
if cleanupErr := fileWriteRemove(root, tempName); cleanupErr != nil && !errors.Is(cleanupErr, os.ErrNotExist) {
return fmt.Errorf("creating %s: %w: %w", relativePath, errFilePublishedWithCleanupFailure, cleanupErr)
}
return nil
}
// publishRootedNoReplace performs the no-replace rename through a root handle
// bound to the target's own directory. Binding the directory first is what
// keeps the raw rename inside the workspace: a parent link swapped after the
// caller resolved the path is refused when the directory is opened, and both
// names are single components below that handle.
func publishRootedNoReplace(root *os.Root, tempName, relativePath string) error {
parent := filepath.Dir(relativePath)
if filepath.Dir(tempName) != parent {
return fmt.Errorf("staging file %s is not beside %s", tempName, relativePath)
}
directory, err := root.OpenRoot(parent)
if err != nil {
return err
}
defer directory.Close()
return pathjail.RenameNoReplace(directory, filepath.Base(tempName), filepath.Base(relativePath))
}
// renameNoReplaceUnsupported reports whether the platform or filesystem told
// us only that it cannot perform a no-replace rename, so the no-replace hard
// link is the right fallback. A collision is not an unsupported operation and
// is handled by the caller before this is consulted.
func renameNoReplaceUnsupported(err error) bool {
return errors.Is(err, errors.ErrUnsupported) || errors.Is(err, syscall.EINVAL)
}
// commitRootedFileOverwrite publishes a replacement for an object whose
// identity and preimage just passed validation: the bytes are written to a
// side file beside the target, the observed identity is re-checked against the
// path as late as the rename allows, and a rename hands the path over in one
// step. A failure while the side file is being written — ENOSPC, a failing
// device — therefore costs a removed side file, not the observed file.
//
// Refused renames fail without mutating the observed inode. Writing in place
// here would expose partial content and rewrite hard-link siblings, defeating
// the guarantees that staging provides.
func commitRootedFileOverwrite(root *os.Root, absolutePath, relativePath, publishPath string, openedInfo os.FileInfo, mode os.FileMode, content string) error {
tempName, err := writeRootedSideFile(root, publishPath, mode, false, content)
if err != nil {
return err
}
if fileWriteBeforePublish != nil {
fileWriteBeforePublish(absolutePath)
}
if err := verifyRootedPublishTarget(root, publishPath, openedInfo); err != nil {
return errors.Join(err, removeWriteSideFile(root, tempName))
}
if err := fileWriteRename(root, tempName, publishPath); err != nil {
renameErr := fmt.Errorf("writing %s: atomic replacement refused: %w", relativePath, err)
return errors.Join(renameErr, removeWriteSideFile(root, tempName))
}
return nil
}
// verifyRootedPublishTarget refuses to replace anything but the exact object
// whose identity and preimage were validated. It is deliberately the last
// operation before the rename; no portable rename offers a compare-and-swap,
// so this check is the strongest condition available before an atomic replace.
func verifyRootedPublishTarget(root *os.Root, publishPath string, openedInfo os.FileInfo) error {
info, err := root.Lstat(publishPath)
if err != nil || info.Mode()&(os.ModeSymlink|os.ModeIrregular) != 0 || !os.SameFile(openedInfo, info) {
return fmt.Errorf("%w: path identity changed", errFileChangedDuringWrite)
}
return nil
}
// resolveRootedFinalPath follows final-component symlinks inside the root and
// returns the path of the file the caller's link resolves to. Publishing over
// that path preserves the link, unlike a rename over the link itself.
func resolveRootedFinalPath(root *os.Root, relativePath string) (string, error) {
current := relativePath
for links := 0; links < 255; links++ {
info, err := root.Lstat(current)
if err != nil {
return "", err
}
if info.Mode()&os.ModeSymlink == 0 {
return current, nil
}
target, err := root.Readlink(current)
if err != nil {
return "", err
}
if filepath.IsAbs(target) || filepath.VolumeName(target) != "" {
return "", fmt.Errorf("%w: %s is an absolute link", errFileChangedDuringWrite, current)
}
next := filepath.Clean(filepath.Join(filepath.Dir(current), target))
if next == ".." || strings.HasPrefix(next, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("%w: %s points outside the workspace", errFileChangedDuringWrite, current)
}
current = next
}
return "", fmt.Errorf("%w: more than 255 links at %s", errFileChangedDuringWrite, relativePath)
}
// writeRootedSideFile writes content to a fresh side file beside the target,
// relative to the root handle, and leaves it closed and ready to publish. The
// caller owns removing the side file when the publish that follows does not
// commit.
//
// umaskDerived selects the permission contract. A create asks the kernel for
// 0o644 and remembers what the process umask actually granted; the bytes are
// made owner-only while they are staged, then the granted bits are restored.
// An overwrite stages 0o600 and finally carries the observed file's permission
// bits over, so the published file looks like the one it replaced.
func writeRootedSideFile(root *os.Root, relativePath string, mode os.FileMode, umaskDerived bool, content string) (string, error) {
createMode := mode.Perm()
if !umaskDerived {
createMode = 0o600
}
temp, tempName, err := pathjail.CreateTempMode(root, filepath.Dir(relativePath), "zero-write", ".tmp", createMode)
if err != nil {
return "", fmt.Errorf("writing %s: %w", relativePath, err)
}
granted := mode.Perm()
if umaskDerived {
info, statErr := temp.Stat()
if statErr != nil {
_ = temp.Close()
_ = removeWriteSideFile(root, tempName)
return "", fmt.Errorf("writing %s: %w", relativePath, statErr)
}
granted = info.Mode().Perm()
if chmodErr := temp.Chmod(0o600); chmodErr != nil {
_ = temp.Close()
_ = removeWriteSideFile(root, tempName)
return "", fmt.Errorf("writing %s: %w", relativePath, chmodErr)
}
}
writeErr := func() error {
if err := fileWriteTempWrite(temp, content); err != nil {
return err
}
return temp.Chmod(granted.Perm())
}()
closeErr := temp.Close()
if writeErr != nil || closeErr != nil {
_ = removeWriteSideFile(root, tempName)
return "", fmt.Errorf("writing %s: %w", relativePath, errors.Join(writeErr, closeErr))
}
return tempName, nil
}
// writeRootedFileInPlace publishes through an already-validated handle. It is
// the non-regular path, so it keeps the previous truncate-and-write behaviour
// and restores the observed bytes if a write fails and the preimage is available.
func writeRootedFileInPlace(root *os.Root, relativePath string, file *os.File, openedInfo os.FileInfo, expectedContent *string, content string) error {
if err := file.Truncate(0); err != nil {
_ = file.Close()
return err
}
if _, err := file.Seek(0, io.SeekStart); err != nil {
_ = file.Close()
return err
}
if _, err := io.WriteString(file, content); err != nil {
restoreErr := restoreRootedFile(file, expectedContent)
closeErr := file.Close()
return errors.Join(err, restoreErr, closeErr)
}
if err := file.Close(); err != nil {
return err
}
@@ -113,3 +377,43 @@ func writeAndVerifyRootedFileIdentity(root *os.Root, relativePath string, file *
}
return nil
}
// restoreRootedFile puts the observed bytes back after a failed in-place
// write. The write already truncated the file, so this is best effort: the
// caller receives both the original failure and any restore failure.
func restoreRootedFile(file *os.File, preimage *string) error {
if preimage == nil {
return nil
}
if err := file.Truncate(0); err != nil {
return fmt.Errorf("restoring the preimage: %w", err)
}
if _, err := file.Seek(0, io.SeekStart); err != nil {
return fmt.Errorf("restoring the preimage: %w", err)
}
if _, err := io.WriteString(file, *preimage); err != nil {
return fmt.Errorf("restoring the preimage: %w", err)
}
return nil
}
// removeWriteSideFile removes a staged replacement that never reached (or did
// not survive) its publish, through the root handle.
func removeWriteSideFile(root *os.Root, name string) error {
err := fileWriteRemove(root, name)
if err == nil || errors.Is(err, os.ErrNotExist) {
return nil
}
// Windows refuses to delete a read-only file. The staged file only carries
// that bit after its published mode was applied; clear it through the root
// and retry. The name belongs to a random side file this commit created.
if errors.Is(err, os.ErrPermission) {
if chmodErr := root.Chmod(name, 0o600); chmodErr == nil {
err = fileWriteRemove(root, name)
if err == nil || errors.Is(err, os.ErrNotExist) {
return nil
}
}
}
return err
}
+369
View File
@@ -0,0 +1,369 @@
package tools
import (
"context"
"errors"
"io"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
func installFileWriteRename(t *testing.T, rename func(*os.Root, string, string) error) {
t.Helper()
prior := fileWriteRename
fileWriteRename = rename
t.Cleanup(func() { fileWriteRename = prior })
}
func installFileWriteBeforePublish(t *testing.T, hook func(string)) {
t.Helper()
prior := fileWriteBeforePublish
fileWriteBeforePublish = hook
t.Cleanup(func() { fileWriteBeforePublish = prior })
}
func installFileWriteNoReplaceRename(t *testing.T, rename func(*os.Root, string, string) error) {
t.Helper()
prior := fileWriteNoReplaceRename
fileWriteNoReplaceRename = rename
t.Cleanup(func() { fileWriteNoReplaceRename = prior })
}
// A FINAL-COMPONENT SYMLINK STAYS A LINK. Main opened the target through the
// link and wrote through it, so replacing the link with a regular file leaves
// the link's target stuck on its old bytes. Publishing over the resolved path
// preserves both halves: the link is still a link, and the file it names has
// the new content.
//
// The public tools reject a symlink during path validation today, so this is
// asserted at the commit layer, which is where the publish semantics live and
// where a scoped write that resolves through an extra root reaches them.
func TestOverwriteKeepsAnInWorkspaceSymlink(t *testing.T) {
for _, tc := range []struct {
name string
chain bool
}{
{name: "direct link"},
{name: "link chain", chain: true},
} {
t.Run(tc.name, func(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "target.txt")
if err := os.WriteFile(target, []byte("observed\n"), 0o644); err != nil {
t.Fatal(err)
}
link := filepath.Join(root, "link.txt")
if err := os.Symlink("target.txt", link); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
publishPath, links := "link.txt", []string{link}
if tc.chain {
chained := filepath.Join(root, "chained.txt")
if err := os.Symlink("link.txt", chained); err != nil {
t.Fatalf("create second symlink: %v", err)
}
publishPath, links = "chained.txt", append(links, chained)
}
workspace, err := os.OpenRoot(root)
if err != nil {
t.Fatal(err)
}
defer workspace.Close()
expectedInfo, err := workspace.Stat(publishPath)
if err != nil {
t.Fatal(err)
}
preimage := "observed\n"
if err := commitRootedFileContents(workspace, filepath.Join(root, publishPath), publishPath, expectedInfo, &preimage, "zero\n"); err != nil {
t.Fatalf("symlink overwrite: %v", err)
}
for _, path := range links {
info, err := os.Lstat(path)
if err != nil {
t.Fatalf("inspect %s: %v", path, err)
}
if info.Mode()&os.ModeSymlink == 0 {
t.Fatalf("%s is no longer a symlink: mode=%v", path, info.Mode())
}
}
if got, err := os.ReadFile(target); err != nil || string(got) != "zero\n" {
t.Fatalf("link target = %q, err=%v; the link must keep pointing at the rewritten file", got, err)
}
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
}
for _, entry := range entries {
if strings.HasSuffix(entry.Name(), ".tmp") {
t.Fatalf("staging file %s survived a successful publish", entry.Name())
}
}
})
}
}
// An open reader keeps the observed inode. Platforms that cannot replace the
// name while it is open must fail without rewriting that reader's bytes.
func TestOverwriteWithAnOpenReaderPreservesAtomicPublication(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "existing.txt")
if err := os.WriteFile(target, []byte("observed\n"), 0o644); err != nil {
t.Fatal(err)
}
reader, err := os.Open(target)
if err != nil {
t.Fatal(err)
}
defer reader.Close()
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "existing.txt", "content": "zero\n", "overwrite": true,
})
want := "zero\n"
if runtime.GOOS == "windows" && result.Status == StatusError && strings.Contains(result.Output, "atomic replacement refused") {
want = "observed\n"
} else if result.Status != StatusOK {
t.Fatalf("overwrite with an open reader = %s: %s", result.Status, result.Output)
}
if got, err := os.ReadFile(target); err != nil || string(got) != want {
t.Fatalf("target = %q, err=%v", got, err)
}
if got, err := io.ReadAll(reader); err != nil || string(got) != "observed\n" {
t.Fatalf("open reader = %q, err=%v; publication must preserve the observed inode", got, err)
}
}
// A refused atomic replacement must leave both names of the observed inode
// untouched. Exercise both production callers and their session baselines.
func TestOverwriteRefusedReplacementPreservesThePreimageAndHardLink(t *testing.T) {
for _, name := range []string{"write", "edit"} {
t.Run(name, func(t *testing.T) {
root := t.TempDir()
root, err := filepath.EvalSymlinks(root)
if err != nil {
t.Fatal(err)
}
target := filepath.Join(root, "existing.txt")
sibling := filepath.Join(root, "sibling.txt")
preimage := []byte("observed\n")
if err := os.WriteFile(target, preimage, 0o644); err != nil {
t.Fatal(err)
}
if err := os.Link(target, sibling); err != nil {
t.Skipf("hard links unavailable: %v", err)
}
info, err := os.Stat(target)
if err != nil {
t.Fatal(err)
}
tracker := NewFileTracker()
tracker.Record(target, preimage, info)
tracker.RecordSeenRange(target, 1, 1, 1)
installFileWriteRename(t, func(*os.Root, string, string) error {
return errors.New("replacement refused")
})
tool := NewScopedWriteFileTool(root, nil)
args := map[string]any{"path": "existing.txt", "content": "zero\n", "overwrite": true}
if name == "edit" {
tool = NewScopedEditFileTool(root, nil)
args = map[string]any{"path": "existing.txt", "old_string": "observed", "new_string": "zero"}
}
registry := NewRegistry()
registry.Register(tool)
result := registry.RunWithOptions(context.Background(), tool.Name(), args, RunOptions{FileTracker: tracker, PermissionGranted: true})
if result.Status != StatusError || !strings.Contains(result.Output, "replacement refused") {
t.Fatalf("refused overwrite = %s: %s", result.Status, result.Output)
}
for _, path := range []string{target, sibling} {
got, err := os.ReadFile(path)
if err != nil || string(got) != string(preimage) {
t.Fatalf("%s = %q, err=%v; a refused publish must preserve the shared preimage", path, got, err)
}
}
if err := tracker.CheckConflict(target, preimage); err != nil {
t.Fatalf("tracker changed after refused publication: %v", err)
}
entries, err := os.ReadDir(root)
if err != nil || len(entries) != 2 {
t.Fatalf("refused publish left staging files: %v, err=%v", entries, err)
}
})
}
}
// A competing writer that replaces the path during a refused rename keeps its
// own file. No in-place write may follow the failed publication.
func TestOverwriteRefusedRenameLeavesACompetingReplacementUntouched(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "existing.txt")
if err := os.WriteFile(target, []byte("observed\n"), 0o644); err != nil {
t.Fatal(err)
}
installFileWriteRename(t, func(*os.Root, string, string) error {
preserved := target + ".preserved"
if err := os.Rename(target, preserved); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(target, []byte("observed\n"), 0o644); err != nil {
t.Fatal(err)
}
return errors.New("replacement refused")
})
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "existing.txt", "content": "zero\n", "overwrite": true,
})
if result.Status != StatusError || !strings.Contains(result.Output, "replacement refused") {
t.Fatalf("replaced target = %s: %s", result.Status, result.Output)
}
if got, err := os.ReadFile(target); err != nil || string(got) != "observed\n" {
t.Fatalf("replacement = %q, err=%v; a refused rename must leave the competing file untouched", got, err)
}
}
// A FILESYSTEM MAY NOT OFFER A NO-REPLACE RENAME. The hard-link fallback is
// still atomic against readers — a reader sees the absent name or the complete
// inode — and refuses a name another writer created.
func TestCreateFallsBackToAHardLinkWhenNoReplaceRenameIsUnsupported(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "created.txt")
installFileWriteNoReplaceRename(t, func(*os.Root, string, string) error {
return errors.ErrUnsupported
})
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "created.txt", "content": "zero\n",
})
if result.Status != StatusOK {
t.Fatalf("hard-link create = %s: %s", result.Status, result.Output)
}
if got, err := os.ReadFile(target); err != nil || string(got) != "zero\n" {
t.Fatalf("created file = %q, err=%v", got, err)
}
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 || entries[0].Name() != "created.txt" {
t.Fatalf("root holds %d entries; the staging name must not survive the publish", len(entries))
}
}
func TestCreateHardLinkFallbackStillRefusesAnExistingName(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "existing.txt")
if err := os.WriteFile(target, []byte("other writer\n"), 0o644); err != nil {
t.Fatal(err)
}
installFileWriteNoReplaceRename(t, func(*os.Root, string, string) error {
return errors.ErrUnsupported
})
workspace, err := os.OpenRoot(root)
if err != nil {
t.Fatal(err)
}
defer workspace.Close()
if err := commitRootedFileContents(workspace, target, "existing.txt", nil, nil, "zero\n"); !errors.Is(err, os.ErrExist) {
t.Fatalf("hard-link create over an existing name = %v, want os.ErrExist", err)
}
if got, err := os.ReadFile(target); err != nil || string(got) != "other writer\n" {
t.Fatalf("existing file = %q, err=%v; the fallback must not replace it", got, err)
}
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 || entries[0].Name() != "existing.txt" {
t.Fatalf("root holds %d entries; a refused create must remove its staging file", len(entries))
}
}
// A REPLACEMENT INSTALLED AFTER THE BYTES WERE STAGED must not be silently
// overwritten by the publish. The identity check immediately before the rename
// is what keeps the conflict guarantee for the window that remains.
func TestOverwriteRefusesAReplacementInstalledBeforePublish(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "existing.txt")
if err := os.WriteFile(target, []byte("observed\n"), 0o644); err != nil {
t.Fatal(err)
}
installFileWriteBeforePublish(t, func(path string) {
preserved := path + ".preserved"
if err := os.Rename(path, preserved); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("competing\n"), 0o644); err != nil {
t.Fatal(err)
}
})
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "existing.txt", "content": "zero\n", "overwrite": true,
})
if result.Status != StatusError || !strings.Contains(result.Output, errFileChangedDuringWrite.Error()) {
t.Fatalf("pre-publish replacement = %s: %s", result.Status, result.Output)
}
if got, err := os.ReadFile(target); err != nil || string(got) != "competing\n" {
t.Fatalf("competing file = %q, err=%v; the publish must refuse a replacement it did not observe", got, err)
}
}
// Publication and cleanup are separate outcomes. Even when cleanup fails, the
// session must know that the new file exists and contains the requested bytes.
func TestCreateCleanupFailureRecordsThePublishedFile(t *testing.T) {
for _, cleanupErr := range []error{errors.New("injected staging cleanup failure"), os.ErrPermission} {
t.Run(cleanupErr.Error(), func(t *testing.T) {
root := t.TempDir()
root, err := filepath.EvalSymlinks(root)
if err != nil {
t.Fatal(err)
}
target := filepath.Join(root, "created.txt")
var publishedMode os.FileMode
installFileWriteNoReplaceRename(t, func(root *os.Root, oldName, _ string) error {
info, err := root.Stat(oldName)
if err != nil {
t.Fatal(err)
}
publishedMode = info.Mode().Perm()
return errors.ErrUnsupported
})
prior := fileWriteRemove
fileWriteRemove = func(*os.Root, string) error { return cleanupErr }
t.Cleanup(func() { fileWriteRemove = prior })
tracker := NewFileTracker()
registry := NewRegistry()
registry.Register(NewScopedWriteFileTool(root, nil))
result := registry.RunWithOptions(context.Background(), "write_file", map[string]any{
"path": "created.txt", "content": "zero\n",
}, RunOptions{FileTracker: tracker, PermissionGranted: true})
if result.Status != StatusError || !strings.Contains(result.Output, "published") || !strings.Contains(result.Output, cleanupErr.Error()) {
t.Fatalf("published create with cleanup failure = %s: %s", result.Status, result.Output)
}
if got, err := os.ReadFile(target); err != nil || string(got) != "zero\n" {
t.Fatalf("published content = %q, err=%v", got, err)
}
if info, err := os.Stat(target); err != nil || info.Mode().Perm() != publishedMode {
t.Fatalf("published mode changed after cleanup failure: %v, err=%v, want %v", info, err, publishedMode)
}
if version, ok := tracker.Version(target); !ok || version.Hash != HashContent([]byte("zero\n")) {
t.Fatalf("published file missing from tracker: %+v, tracked=%v", version, ok)
}
if !tracker.SeenWhole(target) {
t.Fatal("published bytes must be recorded as seen")
}
if created := tracker.CreatedFiles(); len(created) != 1 || created[0] != target {
t.Fatalf("created files = %v, want %s", created, target)
}
if len(result.ChangedFiles) != 1 || result.ChangedFiles[0] != "created.txt" {
t.Fatalf("changed files = %v", result.ChangedFiles)
}
})
}
}
+68
View File
@@ -350,3 +350,71 @@ func TestWriteFileRefusesSameNameSubstitutionWithIdenticalBytes(t *testing.T) {
t.Fatalf("substituted object = %q, err=%v; the write must not land in an object the preimage did not observe", got, err)
}
}
// AN OVERWRITTEN PATH'S NEIGHBOURS THROUGH ANOTHER NAME KEEP THEIR CONTENT.
//
// The replace-overwrite truncates and rewrites the observed file in place, and
// truncation follows the inode: a sibling reached through a hard link loses
// its content when this path's content is replaced, because both names were
// pointing at the same bytes. Publishing through a rename detaches this name
// from the shared inode instead, so the link sibling keeps what it shares.
func TestOverwriteDoesNotTruncateAHardlinkSibling(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "target.txt")
sibling := filepath.Join(root, "sibling.txt")
if err := os.WriteFile(target, []byte("shared\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Link(target, sibling); err != nil {
t.Skipf("hard links unavailable: %v", err)
}
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "target.txt", "content": "replaced\n", "overwrite": true,
})
if result.Status != StatusOK {
t.Fatalf("overwrite status = %s: %s", result.Status, result.Output)
}
shared, err := os.ReadFile(sibling)
if err != nil || string(shared) != "shared\n" {
t.Fatalf("hard-link sibling = %q, err=%v; replacing this name must not rewrite the bytes the sibling shares", shared, err)
}
}
// THE PUBLISHED FILE LOOKS LIKE THE FILE IT REPLACED. The permission bits are
// carried over from the file that was observed, not from the side file the
// bytes were staged in.
func TestOverwriteKeepsTheObservedPermissionBits(t *testing.T) {
// 0o640 differs from the 0o600 the side file is staged with, so a publish
// that forgot to carry the observed bits cannot pass by coincidence.
want := os.FileMode(0o640)
if runtime.GOOS == "windows" {
// A Windows file carries only the read-only bit, so a 0o640 file is
// reported as 0o666; the write bit, not the exact value, is the fact
// being pinned.
want = os.FileMode(0o666)
}
root := t.TempDir()
target := filepath.Join(root, "existing.txt")
if err := os.WriteFile(target, []byte("observed\n"), want); err != nil {
t.Fatal(err)
}
if err := os.Chmod(target, want); err != nil {
t.Fatal(err)
}
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "existing.txt", "content": "zero\n", "overwrite": true,
})
if result.Status != StatusOK {
t.Fatalf("overwrite status = %s: %s", result.Status, result.Output)
}
info, err := os.Stat(target)
if err != nil {
t.Fatal(err)
}
if got := info.Mode().Perm(); got != want {
t.Fatalf("published permission bits = %o, want %o", got, want)
}
}
+56
View File
@@ -0,0 +1,56 @@
//go:build unix
package tools
import (
"context"
"os"
"path/filepath"
"syscall"
"testing"
)
// THE PUBLISHED FILE CARRIES THE PERMISSIONS THE PROCESS UMASK PRODUCES, NOT
// THE RAW MODE AN ASSIGNMENT WOULD SET.
//
// The create path opens its staging file with the same exclusive 0o644 create
// this branch of the write path always honored, and the kernel applies the
// process umask to that open. The publish must carry the granted bits — taken
// from the staging file itself — because a chmod never consults the umask: a
// process that requests tight defaults on new files would find every write it
// publishes shared with the world. Under the restrictive umask a plain mode
// assignment would publish 0o644 where the kernel granted 0o600, so the test
// fails before the granted bits are carried.
func TestCreatePublishesTheUmaskedPermissionsOfTheStagedCreate(t *testing.T) {
for name, testCase := range map[string]struct {
umask int
want os.FileMode
}{
"a restrictive umask keeps every stranger out": {umask: 0o077, want: 0o600},
"a permissive umask keeps the readable bits": {umask: 0o022, want: 0o644},
} {
t.Run(name, func(t *testing.T) {
previous := syscall.Umask(testCase.umask)
defer syscall.Umask(previous)
root := t.TempDir()
target := filepath.Join(root, "created.txt")
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "created.txt", "content": "zero\n",
})
if result.Status != StatusOK {
t.Fatalf("create status = %s: %s", result.Status, result.Output)
}
info, err := os.Stat(target)
if err != nil {
t.Fatal(err)
}
if got := info.Mode().Perm(); got != testCase.want {
t.Fatalf("published mode = %#o, want the %#o the kernel granted the staging create under this umask; a raw mode assignment would publish %#o and share the file with hands the umask excludes", got, testCase.want, os.FileMode(0o644))
}
if got, readErr := os.ReadFile(target); readErr != nil || string(got) != "zero\n" {
t.Fatalf("published content = %q, err=%v", got, readErr)
}
})
}
}
@@ -0,0 +1,128 @@
package tools
import (
"context"
"errors"
"os"
"path/filepath"
"testing"
)
func installFileWriteTempWrite(t *testing.T, write func(*os.File, string) error) {
t.Helper()
prior := fileWriteTempWrite
fileWriteTempWrite = write
t.Cleanup(func() { fileWriteTempWrite = prior })
}
// A WRITE THAT CANNOT COMPLETE MUST NOT TAKE THE FILE IT REPLACED WITH IT.
//
// Running out of disk space part-way through a write used to be fatal to the
// preimage: the overwrite truncated the observed file first, so the failed
// write left an empty husk where a working file stood. The replacement bytes
// are now written whole to a side file, so a failure there costs the side file
// and the observed bytes survive untouched. Each case reopens the target after
// the tool has run and checks the preimage is what still stands, and that no
// side file was left behind.
func TestOverwriteFailureKeepsThePreimageStanding(t *testing.T) {
for name, run := range map[string]func(string) Result{
"write overwrite": func(root string) Result {
return NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "existing.txt", "content": "zero\n", "overwrite": true,
})
},
"edit": func(root string) Result {
return NewScopedEditFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "existing.txt", "old_string": "observed", "new_string": "zero",
})
},
} {
t.Run(name, func(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "existing.txt")
if err := os.WriteFile(target, []byte("observed\n"), 0o644); err != nil {
t.Fatal(err)
}
// The write that stands in for ENOSPC fails after the file was
// observed, accepted and staged — the exact point the old
// truncate-then-write had nothing left to lose.
writeErr := errors.New("injected unrecoverable write failure")
installFileWriteTempWrite(t, func(*os.File, string) error { return writeErr })
result := run(root)
if result.Status != StatusError {
t.Fatalf("failed-write overwrite = %s, want error: %s", result.Status, result.Output)
}
got, err := os.ReadFile(target)
if err != nil || string(got) != "observed\n" {
t.Fatalf("preimage = %q, err=%v; the observed file must outlive a write that failed", got, err)
}
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 || entries[0].Name() != "existing.txt" {
t.Fatalf("root holds %d entries; a failed write must remove its side file", len(entries))
}
})
}
}
// A NEW FILE WHOSE WRITE FAILS MUST NOT APPEAR AT ALL. An exclusive create used
// to leave whatever the write had managed to write behind, half-written and
// complete-looking to anything that did not read past the visible length.
func TestCreateFailureCreatesNothing(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "created.txt")
writeErr := errors.New("injected unrecoverable write failure")
installFileWriteTempWrite(t, func(*os.File, string) error { return writeErr })
result := NewScopedWriteFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "created.txt", "content": "zero\n",
})
if result.Status != StatusError {
t.Fatalf("failed-write create = %s, want error: %s", result.Status, result.Output)
}
if _, statErr := os.Stat(target); !os.IsNotExist(statErr) {
t.Fatalf("target = %v; a create whose bytes never made it must leave no file", statErr)
}
entries, err := os.ReadDir(root)
if err != nil {
t.Fatal(err)
}
if len(entries) != 0 {
t.Fatalf("root holds %d entries; a failed create must remove its side file", len(entries))
}
}
// AT THE LAYER THE COMMIT RUNS, the injected failure is reported as the
// failure it injected and nothing else, so a caller can distinguish a failed
// write from a refused write.
func TestCommitRootedOverwriteFailedWriteIsTheWriteError(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "existing.txt")
if err := os.WriteFile(target, []byte("observed\n"), 0o644); err != nil {
t.Fatal(err)
}
workspace, err := os.OpenRoot(root)
if err != nil {
t.Fatal(err)
}
defer workspace.Close()
info, err := os.Stat(target)
if err != nil {
t.Fatal(err)
}
preimage := "observed\n"
writeErr := errors.New("injected unrecoverable write failure")
installFileWriteTempWrite(t, func(*os.File, string) error { return writeErr })
err = commitRootedFileContents(workspace, filepath.Join(root, "existing.txt"), "existing.txt", info, &preimage, "zero\n")
if !errors.Is(err, writeErr) {
t.Fatalf("commit error = %v, want the injected write failure", err)
}
got, readErr := os.ReadFile(target)
if readErr != nil || string(got) != "observed\n" {
t.Fatalf("preimage = %q, err=%v; the observed file must outlive a write that failed", got, readErr)
}
}
+14 -2
View File
@@ -76,11 +76,23 @@ func (tool captureArtifactTool) RejectBeforePermission(args map[string]any) (Res
if err != nil {
return errorResult("Error: Invalid arguments for capture_artifact: " + err.Error()), true
}
// CONFIGURATION, NOT ARGUMENTS, so these carry provenance. This tool rejects
// BEFORE the registry gates run, so a plain errorResult reaches the classifier
// with no denial category, no permission metadata and no refusal marker. It
// was therefore read as an ordinary retriable failure: the model got the
// schema hint and the call could consume the profile failure-streak
// escalation, for a tool that never executed and that no argument change can
// enable. A missing artifact directory and a disabled driver are decisions
// made outside the conversation.
//
// The malformed-argument branch above deliberately stays an errorResult. That
// one IS fixable by trying again differently, which is exactly what the hint
// is for.
if strings.TrimSpace(tool.artifactsDir) == "" {
return errorResult("Error: capture_artifact is disabled because no artifact directory is configured."), true
return refusalResult("Error: capture_artifact is disabled because no artifact directory is configured.", PolicyRefusalToolNotEnabled), true
}
if !tool.actionEnabled(request.action) {
return errorResult("Error: Local control driver for " + request.action + " is disabled."), true
return refusalResult("Error: Local control driver for "+request.action+" is disabled.", PolicyRefusalToolNotEnabled), true
}
return Result{}, false
}
@@ -0,0 +1,162 @@
package tools
import (
"context"
"testing"
"github.com/Twigpine/zero/internal/sandbox"
)
// forgingTool executes and fails while claiming the registry refused it before
// it ran. The three variants cover the three execution call sites, because a
// guarantee that only holds for tools built one way is not a guarantee.
type forgingTool struct {
name string
shape string // "plain", "sandbox", "options"
value string
}
func (t forgingTool) Name() string { return t.name }
func (t forgingTool) Description() string { return "executes and fails" }
func (t forgingTool) Parameters() Schema { return Schema{Type: "object", AdditionalProperties: false} }
func (t forgingTool) Safety() Safety {
return Safety{SideEffect: SideEffectRead, Permission: PermissionAllow}
}
func (t forgingTool) executed() Result {
result := errorResult("Error: the upstream call failed, try a different path")
result.Meta = map[string]string{PolicyRefusalMeta: t.value, PermissionActionMeta: "deny", "duration_ms": "12"}
return result
}
func (t forgingTool) Run(context.Context, map[string]any) Result { return t.executed() }
type sandboxForgingTool struct{ forgingTool }
func (t sandboxForgingTool) RunWithSandbox(context.Context, map[string]any, *sandbox.Engine) Result {
return t.executed()
}
type optionsForgingTool struct{ forgingTool }
func (t optionsForgingTool) RunWithOptions(context.Context, map[string]any, RunOptions) Result {
return t.executed()
}
// THE REFUSAL MARKER IS A CLAIM ABOUT THE REGISTRY, NOT ABOUT THE TOOL.
//
// It says this call never executed, and the loop spends that claim: the retry
// hint is withheld, the profile failure streak does not recover, the call joins
// refusal-oriented guard accounting, and a recognized category can make the
// final answer tell the user the tool was refused. Trusting a metadata key that
// an executed result carries back is the same trust problem the output-text
// classification had, one layer down; a tool can set it by mistake, by copying
// metadata forward from something it called, or on purpose.
//
// So the boundary strips it. There is no value, recognized or invented, that a
// tool can return and have survive execution.
func TestAnExecutedFailureCannotClaimItWasRefused(t *testing.T) {
for _, value := range []string{
PolicyRefusalSandboxDenied, // a recognized category
PolicyRefusalPermissionDenied, // another, which changes the final wording
"something-the-registry-never-emits", // unknown but nonempty, which was enough
} {
for _, shape := range []string{"plain", "sandbox", "options"} {
t.Run(shape+"/"+value, func(t *testing.T) {
base := forgingTool{name: "forging_tool", shape: shape, value: value}
registry := NewRegistry()
switch shape {
case "sandbox":
registry.Register(sandboxForgingTool{base})
case "options":
registry.Register(optionsForgingTool{base})
default:
registry.Register(base)
}
options := RunOptions{}
if shape == "sandbox" {
options.Sandbox = sandbox.NewEngine(sandbox.EngineOptions{})
}
result := registry.RunWithOptions(context.Background(), "forging_tool", map[string]any{}, options)
if IsPolicyRefusalResult(result) {
t.Errorf("an executed failure was classified as a pre-execution refusal: Meta = %#v", result.Meta)
}
if got := result.Meta[PolicyRefusalMeta]; got != "" {
t.Errorf("the forged marker survived execution as %q", got)
}
// The permission action the loop reads as a refusal goes the same way.
if got := result.Meta[PermissionActionMeta]; got != "" {
t.Errorf("a forged permission action survived execution as %q", got)
}
if result.Status != StatusError {
t.Errorf("Status = %q, want the executed failure preserved", result.Status)
}
// Ordinary metadata is not collateral.
if got := result.Meta["duration_ms"]; got != "12" {
t.Errorf("unrelated metadata was dropped: Meta = %#v", result.Meta)
}
})
}
}
}
// And a real refusal still says so, or the strip would have removed the fact
// rather than authenticated it.
func TestARealRegistryRefusalKeepsItsMarker(t *testing.T) {
registry := NewRegistry()
registry.Register(denyTool{reason: "writes outside the workspace."})
result := registry.RunWithOptions(context.Background(), "deny_tool", map[string]any{}, RunOptions{})
if !IsPolicyRefusalResult(result) {
t.Fatalf("a permission denial lost its provenance: Meta = %#v", result.Meta)
}
if got := result.Meta[PolicyRefusalMeta]; got != PolicyRefusalPermissionDenied {
t.Errorf("category = %q, want %q", got, PolicyRefusalPermissionDenied)
}
}
// rejectingTool refuses on configuration, before the registry's own gates and
// before any execution. That refusal is genuine and has to survive.
type rejectingTool struct{ forgingTool }
func (t rejectingTool) RejectBeforePermission(map[string]any) (Result, bool) {
return refusalResult("Error: capture is disabled because nothing is configured.", PolicyRefusalToolNotEnabled), true
}
func TestAPreExecutionRejectionKeepsItsMarker(t *testing.T) {
registry := NewRegistry()
registry.Register(rejectingTool{forgingTool{name: "rejecting_tool"}})
result := registry.RunWithOptions(context.Background(), "rejecting_tool", map[string]any{}, RunOptions{})
if !IsPolicyRefusalResult(result) {
t.Fatalf("a configuration refusal that never executed lost its provenance: Meta = %#v", result.Meta)
}
if got := result.Meta[PolicyRefusalMeta]; got != PolicyRefusalToolNotEnabled {
t.Errorf("category = %q, want %q", got, PolicyRefusalToolNotEnabled)
}
}
// A tool that reuses one metadata map across calls must not see the strip.
func TestStrippingDoesNotMutateTheToolsOwnMetadata(t *testing.T) {
shared := map[string]string{PolicyRefusalMeta: PolicyRefusalSandboxDenied, "duration_ms": "12"}
registry := NewRegistry()
registry.Register(sharedMetaTool{meta: shared})
_ = registry.RunWithOptions(context.Background(), "shared_meta_tool", map[string]any{}, RunOptions{})
if _, present := shared[PolicyRefusalMeta]; !present {
t.Error("the boundary deleted a key out of the tool's own map")
}
}
type sharedMetaTool struct{ meta map[string]string }
func (t sharedMetaTool) Name() string { return "shared_meta_tool" }
func (t sharedMetaTool) Description() string { return "reuses one map" }
func (t sharedMetaTool) Parameters() Schema {
return Schema{Type: "object", AdditionalProperties: false}
}
func (t sharedMetaTool) Safety() Safety {
return Safety{SideEffect: SideEffectRead, Permission: PermissionAllow}
}
func (t sharedMetaTool) Run(context.Context, map[string]any) Result {
return Result{Status: StatusError, Output: "Error: failed", Meta: t.meta}
}
+11 -7
View File
@@ -242,12 +242,12 @@ func (registry *Registry) RunWithOptions(ctx context.Context, name string, args
})
sandboxDecision = &d
if d.Action == sandbox.ActionDeny {
res := errorResult(d.ErrorString())
res := refusalResult(d.ErrorString(), PolicyRefusalSandboxDenied)
res.SandboxDecision = sandboxDecision
return res
}
if d.Action == sandbox.ActionPrompt && !options.PermissionGranted {
res := errorResult("Error: Sandbox approval required for " + name + ": " + d.Reason)
res := refusalResult("Error: Sandbox approval required for "+name+": "+d.Reason, PolicyRefusalSandboxApproval)
res.SandboxDecision = sandboxDecision
return res
}
@@ -261,18 +261,22 @@ func (registry *Registry) RunWithOptions(ctx context.Context, name string, args
case PermissionAllow:
case PermissionPrompt:
if !options.PermissionGranted && !sandboxGrantAuthorized {
res := errorResult("Error: Permission required for " + name + ": " + tool.Safety().Reason + ` The tool is marked "prompt" and was not executed.`)
res := refusalResult("Error: Permission required for "+name+": "+tool.Safety().Reason+` The tool is marked "prompt" and was not executed.`, PolicyRefusalPermissionRequired)
res.SandboxDecision = sandboxDecision
return res
}
default:
res := errorResult("Error: Permission denied for " + name + ": " + tool.Safety().Reason)
res := refusalResult("Error: Permission denied for "+name+": "+tool.Safety().Reason, PolicyRefusalPermissionDenied)
res.SandboxDecision = sandboxDecision
return res
}
// Past this point the tool RUNS, so every result below is an executed one and
// passes through executedToolResult. See its comment: the refusal marker is a
// claim about the registry, and a result that came back from the tool is proof
// the opposite happened.
if optioned, ok := tool.(optionsAwareTool); ok {
res := optioned.RunWithOptions(ctx, args, options)
res := executedToolResult(optioned.RunWithOptions(ctx, args, options))
if res.SandboxDecision == nil {
res.SandboxDecision = sandboxDecision
}
@@ -281,12 +285,12 @@ func (registry *Registry) RunWithOptions(ctx context.Context, name string, args
if options.Sandbox != nil {
if sandboxed, ok := tool.(sandboxAwareTool); ok {
res := sandboxed.RunWithSandbox(ctx, args, options.Sandbox)
res := executedToolResult(sandboxed.RunWithSandbox(ctx, args, options.Sandbox))
res.SandboxDecision = sandboxDecision
return res
}
}
res := tool.Run(ctx, args)
res := executedToolResult(tool.Run(ctx, args))
res.SandboxDecision = sandboxDecision
return res
}
+93
View File
@@ -2,6 +2,7 @@ package tools
import (
"context"
"strings"
"github.com/Twigpine/zero/internal/execution"
"github.com/Twigpine/zero/internal/sandbox"
@@ -59,6 +60,55 @@ const (
SandboxDenialKindNetwork = "network"
)
// PolicyRefusalMeta marks a Result the registry produced INSTEAD of running the
// tool, and its value names which gate refused.
//
// This exists because output is tool-controlled and therefore cannot classify
// anything. `bash` preserves arbitrary stdout and stderr on a StatusError for
// any nonzero exit, so an ALLOWED command that prints "Sandbox block" and exits
// 1 is indistinguishable, by text, from a sandbox refusal that never ran. The
// two have opposite meanings for retry and for the failure-streak accounting:
// one is a command that executed and failed, the other is a command the policy
// stopped. markStructuredSandboxDenial already states this rule for the sandbox
// adapter ("Classification is never inferred from stdout or stderr"); this
// carries the same guarantee across the rest of the pre-execution gates.
//
// Set it on every path that returns before the tool runs. A refusal without it
// reads as an ordinary execution failure, which is the safe direction (retried
// rather than counted as a denial) but still wrong.
const PolicyRefusalMeta = "policy_refusal"
// PermissionActionMeta carries the permission decision on a result the loop
// builds for a call it stopped. The loop reads "deny" there as a refusal, so it
// makes the same claim about the call as PolicyRefusalMeta and the execution
// boundary strips it the same way.
const PermissionActionMeta = "permission_action"
// The gates that can refuse before execution. Values are stable strings because
// they are written into result metadata that session readers persist.
const (
PolicyRefusalSandboxDenied = "sandbox_denied"
PolicyRefusalSandboxApproval = "sandbox_approval_required"
PolicyRefusalPermissionRequired = "permission_required"
PolicyRefusalPermissionDenied = "permission_denied"
PolicyRefusalToolNotEnabled = "tool_not_enabled"
)
// refusalResult builds the error Result for a gate that refused to run a tool,
// carrying the marker so classification never has to read Output.
func refusalResult(output string, category string) Result {
result := errorResult(output)
result.Meta = map[string]string{PolicyRefusalMeta: category}
return result
}
// IsPolicyRefusalResult reports whether the registry refused this call before
// the tool ran. Exported so the agent loop classifies from the marker rather
// than from model-visible text.
func IsPolicyRefusalResult(result Result) bool {
return strings.TrimSpace(result.Meta[PolicyRefusalMeta]) != ""
}
type Safety struct {
SideEffect SideEffect
Permission Permission
@@ -294,3 +344,46 @@ func promptSafety(sideEffect SideEffect, reason string) Safety {
Reason: reason,
}
}
// executedToolResult strips the pre-execution refusal marker from a result the
// tool produced by RUNNING.
//
// THE MARKER IS A CLAIM ABOUT THE REGISTRY, NOT ABOUT THE TOOL. It says this
// call never executed, and the loop spends that claim: the retry hint is
// withheld, the profile failure streak does not recover, the call is counted in
// refusal-oriented guard accounting, and a recognized category can make the
// final answer tell the user the tool was refused. A result that came back from
// Run is proof the opposite happened, so a tool setting the key, by mistake, by
// copying metadata forward from something it called, or deliberately, would
// forge that claim one layer below the output text this branch stopped trusting.
//
// PermissionActionMeta goes for the same reason. The loop reads a "deny" there
// as a refusal, so an executed failure carrying one was counted toward the
// refusal halt as if the user had refused the call. Reported by @jatmn.
//
// Stripping rather than validating the value keeps it unforgeable for future
// implementations as well: there is no spelling a tool can return that survives
// execution. Genuine pre-execution refusals are untouched, including
// RejectBeforePermission, which decides before any of this and never runs the
// tool. Ordinary metadata is preserved.
func executedToolResult(result Result) Result {
_, refusal := result.Meta[PolicyRefusalMeta]
_, action := result.Meta[PermissionActionMeta]
if !refusal && !action {
return result
}
// Copied rather than deleted in place: the map belongs to the tool, and a
// tool that reuses one across calls would see this mutation.
meta := make(map[string]string, len(result.Meta))
for key, value := range result.Meta {
if key == PolicyRefusalMeta || key == PermissionActionMeta {
continue
}
meta[key] = value
}
if len(meta) == 0 {
meta = nil
}
result.Meta = meta
return result
}
+10 -2
View File
@@ -2,6 +2,7 @@ package tools
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
@@ -146,8 +147,9 @@ func (tool writeFileTool) RunWithOptions(ctx context.Context, args map[string]an
if priorContentKnown {
expectedContent = &priorContent
}
if err := commitRootedFileContents(root, absolutePath, rootedRelative, priorInfo, expectedContent, content); err != nil {
return errorResult("Error writing file " + relativePath + ": " + err.Error())
commitErr := commitRootedFileContents(root, absolutePath, rootedRelative, priorInfo, expectedContent, content)
if commitErr != nil && !errors.Is(commitErr, errFilePublishedWithCleanupFailure) {
return errorResult("Error writing file " + relativePath + ": " + commitErr.Error())
}
modelKnownContent := content
// Optional format-on-write (ZERO_FORMAT_ON_WRITE). Must run BEFORE the
@@ -193,7 +195,13 @@ func (tool writeFileTool) RunWithOptions(ctx context.Context, args map[string]an
if finalContentKnown {
summary += inlineDiagnostics(ctx, options, absolutePath, relativePath)
}
if commitErr != nil {
summary += "\n" + commitErr.Error()
}
result := okResult(summary)
if commitErr != nil {
result.Status = StatusError
}
result.ChangedFiles = []string{relativePath}
// Do not pretend an unreadable overwrite was a creation. The write may be
// valid, but ACP only receives an exact before/after pair we actually saw.
+1 -1
View File
@@ -160,7 +160,7 @@ func (m model) handleBTWCommand(question string) (model, tea.Cmd) {
side.plan.clear()
side.planDetailGen++
side.streamingText = nil
side.streamingReasoning = ""
side.streamingReasoning = nil
side.streamingReasoningExpanded = false
side.clearStreamingToolCall()
side.resetStreamingFade()
+67 -33
View File
@@ -7,19 +7,27 @@ import (
tea "charm.land/bubbletea/v2"
)
// streamCoalesceInterval is roughly one 60fps frame. Assistant-text deltas that
// arrive within this window are merged into a single agentTextMsg, so the render
// rate decouples from the token rate: a fast local model (100+ tok/s) no longer
// forces 100+ full Update→View cycles (each re-parsing the growing markdown) per
// second. Rendering stays smooth regardless of provider speed.
// streamCoalesceInterval is roughly one 60fps frame. Assistant-text and
// reasoning deltas that arrive within this window are merged into a single
// message, so the render rate decouples from the token rate: a fast local
// model (100+ tok/s) no longer forces 100+ full Update→View cycles (each
// re-rendering the growing markdown or thinking block) per second. Reasoning
// matters as much as text here: interimBlock re-renders the ENTIRE accumulated
// thinking string every frame, so uncoalesced reasoning deltas are O(n) work
// per token (O(n²) per segment) and the message queue backs up faster than it
// drains — the UI keeps replaying "thoughts" long after the provider stopped
// thinking. Rendering stays smooth regardless of provider speed.
const streamCoalesceInterval = 16 * time.Millisecond
// textCoalescer batches agentTextMsg deltas before forwarding them to the Bubble
// Tea program. Any OTHER message flushes the pending text first, so ordering
// between streamed prose and tool-call / reasoning / row / usage messages is
// preserved. The turn's final agentResponseMsg does not pass through here (it is
// a tea.Cmd return, not a sink message), but the model drops deltas whose runID
// is no longer active, so a flush that races just past end-of-turn is harmless.
// textCoalescer batches agentTextMsg and agentReasoningMsg deltas before
// forwarding them to the Bubble Tea program. Any OTHER message flushes the
// pending stream first, so ordering between streamed prose/reasoning and
// tool-call / row / usage messages is preserved. A switch between text and
// reasoning (or between runs) also flushes the buffered kind first, so
// interleaved thinking and prose never reorder. The turn's final
// agentResponseMsg does not pass through here (it is a tea.Cmd return, not a
// sink message), but the model drops deltas whose runID is no longer active,
// so a flush that races just past end-of-turn is harmless.
//
// Sink messages originate from the single agent goroutine and so arrive
// serially; the only concurrent caller is the flush timer. The mutex guards the
@@ -37,7 +45,11 @@ type textCoalescer struct {
mu sync.Mutex
buf []byte
runID int
timer coalesceTimer
// reasoning records whether the buffered deltas are reasoning (true) or
// prose (false). Only one kind is ever buffered at a time: a kind switch
// flushes first, which preserves interleaved order.
reasoning bool
timer coalesceTimer
}
// coalesceTimer is the subset of *time.Timer the coalescer needs. Abstracted
@@ -60,50 +72,72 @@ func (c *textCoalescer) send(msg tea.Msg) {
c.mu.Lock()
defer c.mu.Unlock()
text, ok := msg.(agentTextMsg)
if !ok {
// Non-text message: flush buffered text first (preserving order), then
var (
delta string
runID int
reasoning bool
streaming bool
)
switch m := msg.(type) {
case agentTextMsg:
delta, runID, streaming = m.delta, m.runID, true
case agentReasoningMsg:
delta, runID, reasoning, streaming = m.delta, m.runID, true, true
}
if !streaming {
// Non-stream message: flush buffered stream first (preserving order), then
// forward it — both under the lock so nothing can interleave between them.
c.drainAndForwardLocked()
c.forward(msg)
return
}
// A delta for a different run than the one buffered: flush the old run's text
// before buffering the new run's. In practice runs are sequential (the prior
// run's end already flushed via a non-text message), so this is belt-and-braces.
if len(c.buf) > 0 && text.runID != c.runID {
// A kind or run switch: flush the buffered kind before buffering the new
// one, so interleaved reasoning/prose is delivered in arrival order. In
// practice runs are sequential (the prior run's end already flushed via a
// non-stream message), so the run switch is belt-and-braces.
if len(c.buf) > 0 && (runID != c.runID || reasoning != c.reasoning) {
c.drainAndForwardLocked()
}
c.runID = text.runID
c.buf = append(c.buf, text.delta...)
c.runID = runID
c.reasoning = reasoning
c.buf = append(c.buf, delta...)
if c.timer == nil {
c.timer = c.afterFunc(c.flush)
}
}
// flush forwards any buffered text as one agentTextMsg. Runs on the timer
// goroutine; the lock it takes serializes it against send so its output can't be
// reordered around a concurrent non-text message.
// flush forwards any buffered stream deltas as one coalesced message. Runs on
// the timer goroutine; the lock it takes serializes it against send so its
// output can't be reordered around a concurrent non-stream message.
func (c *textCoalescer) flush() {
c.mu.Lock()
defer c.mu.Unlock()
c.drainAndForwardLocked()
}
// drainAndForwardLocked forwards any buffered text as one agentTextMsg and stops
// the timer, all while the caller holds c.mu — so a text flush and any non-text
// forward are strictly ordered and never interleave. A no-op when nothing is
// buffered. string(c.buf) copies, so reusing the backing array via [:0] is safe.
// drainAndForwardLocked forwards any buffered deltas as one coalesced
// agentTextMsg or agentReasoningMsg (whichever kind was buffered) and stops the
// timer, all while the caller holds c.mu — so a flush and any concurrent
// forward are strictly ordered and never interleave. Nothing is forwarded when
// nothing is buffered, but the timer is still cleared: an empty delta can arm
// a timer that fires against an empty buffer, and leaving that timer non-nil
// would make send() skip arming a new one, stalling all later deltas until a
// kind switch or non-stream message. string(c.buf) copies, so reusing the
// backing array via [:0] is safe.
func (c *textCoalescer) drainAndForwardLocked() {
if len(c.buf) == 0 {
return
}
if c.timer != nil {
c.timer.Stop()
c.timer = nil
}
msg := agentTextMsg{runID: c.runID, delta: string(c.buf)}
if len(c.buf) == 0 {
return
}
text := string(c.buf)
c.buf = c.buf[:0]
c.forward(msg)
if c.reasoning {
c.forward(agentReasoningMsg{runID: c.runID, delta: text})
return
}
c.forward(agentTextMsg{runID: c.runID, delta: text})
}
+107
View File
@@ -180,3 +180,110 @@ func TestCoalescerTimerFlushes(t *testing.T) {
t.Fatalf("timer-flushed message = %#v", got[0])
}
}
// An empty delta can arm a timer whose flush fires against an empty buffer.
// That fired timer must be cleared, or send() sees a non-nil timer and skips
// arming a new one, stalling all later deltas until a kind switch or non-stream
// message happens to flush them.
func TestCoalescerEmptyDeltaDoesNotStrandTimer(t *testing.T) {
rec := &recorder{}
c := newTextCoalescer(rec.forward)
var (
mu sync.Mutex
armCount int
fireLatest func()
)
c.afterFunc = func(fn func()) coalesceTimer {
mu.Lock()
defer mu.Unlock()
armCount++
fireLatest = fn
return &manualTimer{}
}
// Empty delta arms a timer; it fires against an empty buffer.
c.send(agentTextMsg{runID: 1, delta: ""})
mu.Lock()
first := armCount
mu.Unlock()
if first != 1 {
t.Fatalf("empty delta should arm exactly one timer, armed %d", first)
}
fireLatest()
// A real delta after the fired timer must arm a NEW timer, so it can be
// delivered on schedule instead of stalling forever.
c.send(agentTextMsg{runID: 1, delta: "after stall"})
mu.Lock()
second := armCount
mu.Unlock()
if second != 2 {
t.Fatalf("delta after empty-buffer timer fire should re-arm the timer (arm count %d), so later deltas stall", second)
}
fireLatest()
got := rec.snapshot()
if len(got) != 1 {
t.Fatalf("expected exactly one forwarded message, got %d: %#v", len(got), got)
}
if text, ok := got[0].(agentTextMsg); !ok || text.delta != "after stall" {
t.Fatalf("forwarded message = %#v, want agentTextMsg{\"after stall\"}", got[0])
}
}
// Rapid reasoning deltas within one frame collapse into a single
// agentReasoningMsg, exactly like text deltas. Without this, every reasoning
// token forces a full Update→View cycle that re-renders the whole growing
// thinking block, so the UI falls behind the stream and keeps "streaming"
// thoughts long after the provider finished.
func TestCoalescerBatchesReasoningDeltas(t *testing.T) {
rec := &recorder{}
c := newTextCoalescer(rec.forward)
c.afterFunc = func(func()) coalesceTimer { return &manualTimer{} }
c.send(agentReasoningMsg{runID: 1, delta: "think"})
c.send(agentReasoningMsg{runID: 1, delta: "ing…"})
if got := rec.snapshot(); len(got) != 0 {
t.Fatalf("reasoning deltas should buffer, forwarded %d early: %#v", len(got), got)
}
c.flush()
got := rec.snapshot()
if len(got) != 1 {
t.Fatalf("expected 1 coalesced message, got %d: %#v", len(got), got)
}
reasoning, ok := got[0].(agentReasoningMsg)
if !ok || reasoning.delta != "thinking…" || reasoning.runID != 1 {
t.Fatalf("coalesced message = %#v, want agentReasoningMsg{1, \"thinking…\"}", got[0])
}
}
// A text delta following buffered reasoning flushes the reasoning first, and
// vice versa, so interleaved thinking/prose never reorders.
func TestCoalescerPreservesReasoningTextOrder(t *testing.T) {
rec := &recorder{}
c := newTextCoalescer(rec.forward)
c.afterFunc = func(func()) coalesceTimer { return &manualTimer{} }
c.send(agentReasoningMsg{runID: 1, delta: "thought"})
c.send(agentTextMsg{runID: 1, delta: "answer"})
c.send(agentReasoningMsg{runID: 1, delta: "more thought"})
c.flush()
got := rec.snapshot()
if len(got) != 3 {
t.Fatalf("expected reasoning, text, reasoning in order, got %d: %#v", len(got), got)
}
if r, ok := got[0].(agentReasoningMsg); !ok || r.delta != "thought" {
t.Fatalf("first message must be the flushed reasoning, got %#v", got[0])
}
if tx, ok := got[1].(agentTextMsg); !ok || tx.delta != "answer" {
t.Fatalf("second message must be the flushed text, got %#v", got[1])
}
if r, ok := got[2].(agentReasoningMsg); !ok || r.delta != "more thought" {
t.Fatalf("third message must be the final reasoning, got %#v", got[2])
}
}
+71 -25
View File
@@ -435,7 +435,7 @@ type model struct {
// A []byte (not strings.Builder) because the model is copied by value on every
// Update, which would trip strings.Builder's copy check.
streamingText []byte
streamingReasoning string // live provider reasoning for the current segment
streamingReasoning []byte // live provider reasoning for the current segment; []byte for the same O(1)-append reason as streamingText (see above) — read via streamingReasoningString()
streamingReasoningExpanded bool
// turnStreamedRunes accumulates every reasoning+answer rune streamed in the
// current turn so the working line can show a live, monotonic token estimate.
@@ -2475,7 +2475,7 @@ func (m model) updateModel(msg tea.Msg) (tea.Model, tea.Cmd) {
if msg.runID != m.activeRunID {
return m, nil
}
m.streamingReasoning += msg.delta
m.streamingReasoning = append(m.streamingReasoning, msg.delta...)
m.turnStreamedRunes += utf8.RuneCountInString(msg.delta)
// Reasoning IS live provider output, so refresh the activity clock — else the
// quiet-generation hint can wrongly read "still generating…" mid-think.
@@ -2801,7 +2801,7 @@ func (m model) updateModel(msg tea.Msg) (tea.Model, tea.Cmd) {
}
for _, row := range msg.rows {
if row.kind == rowReasoning {
m.streamingReasoning = ""
m.streamingReasoning = nil
m.streamingReasoningExpanded = false
}
m.transcript = appendTranscriptRow(m.transcript, row)
@@ -2810,7 +2810,7 @@ func (m model) updateModel(msg tea.Msg) (tea.Model, tea.Cmd) {
// A failed turn has no final answer row to supersede the streamed
// text the user already watched — keep the partial answer instead of
// letting it vanish from history.
if row, ok := reasoningTranscriptRow("", msg.runID, m.streamingReasoning); ok {
if row, ok := reasoningTranscriptRow("", msg.runID, m.streamingReasoningString()); ok {
m.transcript = appendTranscriptRow(m.transcript, row)
}
if text := strings.TrimRight(m.streamingTextString(), "\n"); strings.TrimSpace(text) != "" {
@@ -2835,7 +2835,7 @@ func (m model) updateModel(msg tea.Msg) (tea.Model, tea.Cmd) {
}
}
m.streamingText = nil
m.streamingReasoning = ""
m.streamingReasoning = nil
m.streamingReasoningExpanded = false
if msg.goalAware {
m = m.reconcileGoalAfterRun(msg.usageEvents, msg.err)
@@ -3045,7 +3045,7 @@ func (m model) updateModel(msg tea.Msg) (tea.Model, tea.Cmd) {
return m, nil
}
if msg.row.kind == rowReasoning {
m.streamingReasoning = ""
m.streamingReasoning = nil
m.streamingReasoningExpanded = false
}
// A tool call ends the current streamed text segment. The segment is the
@@ -3053,9 +3053,9 @@ func (m model) updateModel(msg tea.Msg) (tea.Model, tea.Cmd) {
// non-final assistant row so it stays in history instead of silently
// vanishing when the tool card replaces the interim block.
if msg.row.kind == rowToolCall {
if row, ok := reasoningTranscriptRow("", msg.runID, m.streamingReasoning); ok {
if row, ok := reasoningTranscriptRow("", msg.runID, m.streamingReasoningString()); ok {
m.transcript = appendTranscriptRow(m.transcript, row)
m.streamingReasoning = ""
m.streamingReasoning = nil
m.streamingReasoningExpanded = false
}
if text := strings.TrimRight(m.streamingTextString(), "\n"); strings.TrimSpace(text) != "" {
@@ -3973,7 +3973,7 @@ func (m model) liveReasoningBodyCap() int {
func (m model) interimBlock(width int) string {
text := strings.TrimRight(m.streamingTextString(), "\n")
reasoning := strings.TrimRight(m.streamingReasoning, "\n")
reasoning := strings.TrimRight(m.streamingReasoningString(), "\n")
blocks := []string{}
if strings.TrimSpace(reasoning) != "" {
blocks = append(blocks, renderReasoningBlock(reasoning, m.streamingReasoningExpanded, width, true, 0, m.liveReasoningBodyCap()))
@@ -4196,20 +4196,36 @@ func formatWorkingElapsed(d time.Duration) string {
// stream as a dimmed preview so a long "Thinking" phase shows live, changing
// content instead of a static header. Each line shows its TAIL (the most recent
// text) so a single continuously-growing reasoning line still visibly moves as
// tokens arrive. Returns nil when there is no reasoning text.
// tokens arrive. Only the tail is ever shown, so it scans backwards from the end
// instead of splitting the whole (continuously growing) string on every frame.
// Returns nil when there is no reasoning text.
func reasoningPreviewLines(reasoning string, width int) []string {
var lines []string
for _, raw := range strings.Split(strings.TrimSpace(reasoning), "\n") {
if t := strings.TrimSpace(raw); t != "" {
lines = append(lines, t)
// Collect up to the last 2 non-blank lines, walking backwards from the end.
var tail [2]string
count := 0
rest := reasoning
for count < 2 {
end := len(rest)
for end > 0 && rest[end-1] == '\n' {
end--
}
if end == 0 {
break
}
lineStart := strings.LastIndexByte(rest[:end], '\n') + 1
if t := strings.TrimSpace(rest[lineStart:end]); t != "" {
tail[1-count] = t
count++
}
if lineStart == 0 {
break
}
rest = rest[:lineStart-1]
}
if len(lines) == 0 {
if count == 0 {
return nil
}
if len(lines) > 2 {
lines = lines[len(lines)-2:]
}
lines := tail[2-count:]
avail := width - 2
if avail < 8 {
avail = 8
@@ -4223,16 +4239,37 @@ func reasoningPreviewLines(reasoning string, width int) []string {
// previewTail returns the last `width` runes of s, prefixed with "…" when text
// was dropped, so a streaming preview shows the newest content. s is plain text
// (reasoning deltas carry no ANSI), so rune counting is a safe width proxy.
// (reasoning deltas carry no ANSI), so rune counting is a safe width proxy. It
// decodes only the last `width` runes instead of converting the whole line,
// because a single growing reasoning line is re-previewed on every frame.
func previewTail(s string, width int) string {
runes := []rune(s)
if width <= 0 || len(runes) <= width {
if width <= 0 || s == "" {
return s
}
if width == 1 {
return string(runes[len(runes)-1:])
_, size := utf8.DecodeLastRuneInString(s)
return s[len(s)-size:]
}
return "…" + string(runes[len(runes)-(width-1):])
// Walk back at most `width` runes, remembering where the last width-1 runes
// start. If fewer than width runes exist, nothing was dropped; if exactly
// width exist, the original whole-line return still applies. Only when a
// width+1-th rune exists do we truncate and prefix "…".
i := len(s)
cut := len(s)
for consumed := 0; i > 0 && consumed < width; consumed++ {
_, size := utf8.DecodeLastRuneInString(s[:i])
i -= size
if consumed == width-2 {
cut = i
}
}
if i == 0 {
return s
}
// Re-encode through []rune so invalid UTF-8 bytes in the tail become
// replacement characters, matching the whole-string rune conversion this
// truncation replaced.
return "…" + string([]rune(s[cut:]))
}
func (m model) appendStreamingCursor(lines []string, width int) []string {
@@ -5698,7 +5735,7 @@ func (m *model) cancelRun() {
// A cancelled run must terminate visibly in the transcript: first the
// partial streamed answer (if any), then the cancellation marker — the
// session log gets the same marker below.
if row, ok := reasoningTranscriptRow("", m.activeRunID, m.streamingReasoning); ok {
if row, ok := reasoningTranscriptRow("", m.activeRunID, m.streamingReasoningString()); ok {
m.transcript = appendTranscriptRow(m.transcript, row)
}
if text := strings.TrimRight(m.streamingTextString(), "\n"); strings.TrimSpace(text) != "" {
@@ -5736,7 +5773,7 @@ func (m *model) cancelRun() {
// The interim block renders streamingText live; a cancelled run's partial
// answer must not leak into (and concatenate with) the next turn's stream.
m.streamingText = nil
m.streamingReasoning = ""
m.streamingReasoning = nil
m.streamingReasoningExpanded = false
// Hard-stop the fade and drop the per-line age map. The next turn's
// first agentTextMsg will seed a fresh lineAges slice and restart
@@ -6414,6 +6451,15 @@ func (m model) streamingTextString() string {
return string(m.streamingText)
}
// streamingReasoningString returns the accumulated live reasoning. Like
// streamingText it is stored as []byte so each reasoning delta is an O(1)
// amortized append instead of the O(n²) that string += delta incurs across a
// long thinking segment; the conversion here is bounded by the segment length,
// the same cost the renderer already pays in interimBlock.
func (m model) streamingReasoningString() string {
return string(m.streamingReasoning)
}
func (m model) sendToolCallStreamStart(runID int, id, name string) {
if m.runtimeMessageSink == nil {
return
+1 -1
View File
@@ -765,7 +765,7 @@ func TestMouseClickTogglesStreamingReasoning(t *testing.T) {
m.mouseCapture = true
m.pending = true
m.activeRunID = 1
m.streamingReasoning = "private **thought**"
m.streamingReasoning = []byte("private **thought**")
width := m.chatColumnWidth()
body, selectable := m.transcriptBody(width, "")
+1 -1
View File
@@ -48,7 +48,7 @@ func TestTranscriptBodyItemsShiftSelectableLinesByItemStart(t *testing.T) {
func TestTranscriptBodyItemsKeepPendingInterimSelectableLocal(t *testing.T) {
m := mouseTestModel()
m.pending = true
m.streamingReasoning = "private thought"
m.streamingReasoning = []byte("private thought")
width := m.chatColumnWidth()
layout := layoutTranscriptBodyItems(m.transcriptBodyItems(width, "", false))
+1 -1
View File
@@ -1020,7 +1020,7 @@ func (m model) renderSelectableReasoningRow(rowIndex int, row transcriptRow, wid
}
func (m model) renderSelectableStreamingReasoning(width int, startBodyY int) []transcriptSelectableLine {
_, selectable := m.renderSelectableReasoningBlock(-1, m.streamingReasoning, m.streamingReasoningExpanded, true, 0, width, startBodyY)
_, selectable := m.renderSelectableReasoningBlock(-1, m.streamingReasoningString(), m.streamingReasoningExpanded, true, 0, width, startBodyY)
for index := range selectable {
selectable[index].live = true
}
+49 -3
View File
@@ -352,7 +352,7 @@ func TestWorkingTokenIndicatorAccumulatesAcrossSegmentClears(t *testing.T) {
// Simulate the segment boundary that clears the live buffers, then stream
// answer text in the next segment.
m.streamingReasoning = ""
m.streamingReasoning = nil
m.streamingText = nil
updated, _ = m.Update(agentTextMsg{runID: rid, delta: strings.Repeat("b", 40)})
m = updated.(model)
@@ -373,6 +373,40 @@ func TestWorkingTokenIndicatorAccumulatesAcrossSegmentClears(t *testing.T) {
}
}
func TestReasoningPreviewLines(t *testing.T) {
plain := func(lines []string) []string {
out := make([]string, 0, len(lines))
for _, line := range lines {
out = append(out, ansiPattern.ReplaceAllString(line, ""))
}
return out
}
if got := reasoningPreviewLines("", 80); got != nil {
t.Errorf("empty reasoning must render no preview, got %q", got)
}
if got := reasoningPreviewLines(" \n\n \n", 80); got != nil {
t.Errorf("blank-only reasoning must render no preview, got %q", got)
}
got := plain(reasoningPreviewLines("first line\n\n second line \nthird\n\n", 200))
want := []string{" second line", " third"}
if len(got) != len(want) {
t.Fatalf("preview lines = %q, want %q", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Errorf("preview line %d = %q, want %q", i, got[i], want[i])
}
}
// A single continuously-growing line shows its TAIL, newest last.
single := plain(reasoningPreviewLines(strings.Repeat("a", 500)+" tail", 20))
if len(single) != 1 || !strings.Contains(single[0], "tail") {
t.Fatalf("single growing line must preview its tail, got %q", single)
}
if strings.HasPrefix(strings.TrimLeft(single[0], " "), "a") {
t.Errorf("long line must be truncated with an ellipsis, got %q", single[0])
}
}
func TestPreviewTail(t *testing.T) {
cases := []struct {
in string
@@ -383,12 +417,24 @@ func TestPreviewTail(t *testing.T) {
{"exactlyten", 10, "exactlyten"},
{"abcdefghijklmnop", 6, "…lmnop"}, // tail with leading ellipsis
{"", 8, ""},
{"four", 4, "four"}, // exactly width runes: no ellipsis
{"five!", 4, "…ve!"}, // width+1 runes: truncate to width incl. ellipsis
{"x", 1, "x"}, // width 1 keeps the last rune
{"日本語abc", 4, "…abc"}, // multibyte runes counted, not bytes
{"héllo wörld", 6, "…wörld"}, // accented tail
{"🙂🙂🙂", 2, "…🙂"}, // 4-byte emoji runes
}
for _, c := range cases {
if got := previewTail(c.in, c.width); got != c.want {
t.Errorf("previewTail(%q,%d) = %q, want %q", c.in, c.width, got, c.want)
}
}
// A truncated tail must normalize invalid UTF-8 to replacement characters,
// matching the whole-string rune conversion this bounded-tail path replaced
// (a raw 0xFF would otherwise reach the renderer as invalid bytes).
if got := previewTail("abcdefgh\xff", 4); got != "…gh\uFFFD" {
t.Errorf("previewTail with invalid byte = %q, want %q", got, "…gh\uFFFD")
}
}
// The fix: during a think (no answer text yet) the streaming reasoning TAIL is
@@ -399,7 +445,7 @@ func TestInterimBlockShowsReasoningPreviewWhileThinking(t *testing.T) {
base := time.Date(2026, 6, 18, 23, 0, 0, 0, time.UTC)
m.now = func() time.Time { return base.Add(90 * time.Second) }
m.turnStartedAt = base
m.streamingReasoning = "analyzing the layout\nthe patch was corrupt so re-planning the css edits"
m.streamingReasoning = []byte("analyzing the layout\nthe patch was corrupt so re-planning the css edits")
m.streamingText = nil // thinking phase: no answer yet
got := plainRender(t, m.interimBlock(96))
@@ -418,7 +464,7 @@ func TestInterimBlockNoPreviewWhenReasoningExpanded(t *testing.T) {
m.width = 100
m.now = func() time.Time { return time.Date(2026, 6, 18, 23, 0, 0, 0, time.UTC) }
m.streamingReasoningExpanded = true
m.streamingReasoning = "only line of reasoning here"
m.streamingReasoning = []byte("only line of reasoning here")
m.streamingText = nil
got := plainRender(t, m.interimBlock(96))
if strings.Count(got, "only line of reasoning here") != 1 {
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@twigpine/zero",
"version": "0.12.0",
"version": "0.13.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@twigpine/zero",
"version": "0.12.0",
"version": "0.13.0",
"cpu": [
"x64",
"arm64"
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@twigpine/zero",
"version": "0.12.0",
"version": "0.13.0",
"description": "Zero — a fast, multi-provider terminal coding agent.",
"type": "module",
"module": "bin/zero.js",