Compare commits

..
Author SHA1 Message Date
e034a93815 perf(tui): bounded file read, LRU rendering cache, and size caps for full-file view (#1203)
* perf(tui): bounded file read, LRU rendering cache, and size caps for full-file view

- Bound full-file view reads (4000 lines, 1 MiB total, 4 KiB per line) with async loading and an LRU render cache capped at 64 entries and 8 MiB.
- Refresh both BTW surfaces when a tool result (such as write_stdin) carries both exact changed paths and an unknown-scope or aggregated directory summary outside Git.
- Gate Run details FILES mouse hit-testing on Run details being the visible foreground overlay so higher-priority modals (speech API-key prompt, leader help) block clicks behind them.

(cherry picked from commit 100d847aaf)

* fix(tui): gate full-view read errors on snapshot validity

renderFileViewFull returned renderedContent whenever hasError was set,
before the snapshot-validity check that governs every other completed
snapshot. A /btw fork of a full view that had settled on a read error
invalidates the inherited snapshot and clears renderedContent, but the
error latch survived, so the side pane rendered an empty body instead of
the loading placeholder until its own load landed. Reloads of an errored
view (resize, refresh, theme) likewise kept showing the stale error.

Serve the error snapshot through the same validity predicate as content
snapshots, so a pending or invalidated load falls through to "Loading…".
Error displays still do not count as cache hits.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit a1887f4b03)

* fix(tui): keep file-view revisions monotonic and revalidate on git sweeps

Shared path revisions reset to zero on every cache purge, while each surface
kept its own required revision. Three sites could also raise that revision
past the shared one by incrementing it locally. After a purge, an exact-path
edit made on the other BTW surface could produce a shared revision no higher
than this surface's own. recoverInvalidatedFileView then saw nothing new, and
the view stayed stale. Revisions now come from one cache-wide clock that
survives purges. A purge raises the epoch floor to that clock instead of
resetting it, and each surface adopts the revision the cache returns.

Every git sweep invalidated each swept path and forced a full refresh. The
swept list is cumulative since the baseline, and each live sweep is launched
beside the tool-result, turn-end or BTW-return handling that already reloads
or rechecks the open view. The sweep that followed therefore reread and
re-highlighted an unchanged file and flashed "Loading…". A sweep now
revalidates the open view instead. It waits for an in-flight load to land,
then rereads the bounded source in the background and compares it with the
displayed snapshot's identity (hash and truncation flags). The view reloads
only if the bytes changed. A mutation the swept list omits is still caught.

Regression tests (all fail on 311290c0 and pass here):
- TestFileViewCrossSurfaceExactEditAfterPurgeReloadsOtherSurface
- TestFileViewGitSweepRevalidatesWithoutDuplicateReload
- TestFileViewGitSweepDuringLoadDefersToThatLoad
- TestFileViewStaleRevalidationDoesNotReloadNewerSnapshot
- TestFileViewCachePurgeResetsPathRevisions (now asserts monotonicity)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* refactor(tui): return source identity from loadAndRender directly

1ce4f96e added loadAndRenderSource and left loadAndRender as a two-value
wrapper. After that change only tests called the wrapper, so make deadcode
newly reported it. Fold the wrapper away: loadAndRender now returns the
rendered text, the source identity, and the error, and its callers take the
value they need. Behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(tui): close three file-view staleness gaps from review

- fileViewSourceHash joined lines with separators, so an empty file (no
  lines) and a file holding one newline (one empty line) hashed alike.
  Sweep revalidation compares identities without the file size, so adding
  or removing that blank line went undetected. Every line is now
  terminated, which keeps the encoding injective.
- A load cancelled inside the bounded read returned errFileViewSuperseded
  with no lines. loadAndRender then treated it as a read error and evicted
  the path, discarding an entry a newer load had already committed. The
  sentinel now returns before the eviction branch.
- A cancelled run keeps going until it finishes, and agentRowMsg ignores its
  late tool results. Its completion only flushed session events, so a write
  that finished during cancellation left an open full-file view stale. The
  flush completion now runs the unknown-scope refresh.

Regression tests (each fails on ca1cdaf3 and passes here):
- TestFileViewGitSweepDetectsBlankLineAddedToEmptyFile
- TestFileViewSupersededReadKeepsNewerCacheEntry
- TestFileViewCancelledRunCompletionRefreshesOpenView

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(tui): gate Run details Enter on the visible overlay

The Run details Enter drill-in keyed off the raw runDetailsOpen flag,
while mouse FILES picks use runDetailsVisible(). The flag outlives the
overlay: clicking a specialist card beside Run details enters the
subchat, which hides the overlay but leaves runDetailsOpen set, and
Enter then opened the parent's file view from inside the subchat.

Use runDetailsVisible() and the shared runDetailsInnerWidth() geometry
on the keyboard path, matching the mouse path. The branch's existing
Esc/Ctrl+B and key-swallow behavior is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: hazyhaar <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-07 10:28:29 -08:00
2ca84cc127 fix(hooks): dispatch specialistStart/Stop lifecycle hooks (#1204)
* fix(hooks): dispatch specialistStart/Stop and allow matchers

These events were documented and accepted by hooks add, but never fired.
Wire advisory dispatch from specialist accounting and allow specialist
name matchers so EXTENDING.md matches runtime behavior.

* fix(hooks): unlock before specialistStop hooks and fire once

Release accountingMu before running lifecycle hooks, and dispatch
specialistStop only when the stop event is newly recorded so concurrent
finishers cannot re-run arbitrary hook commands.

* fix(hooks): fire specialistStop without relying on session append

Decouple stop-hook dispatch from parent-session event append so swarm
members (empty parent) and TaskOutput still get exactly one hook via a
shared LifecycleHooks claim, and wire that bridge into TaskOutput.

* fix(hooks): claim specialistStop durably and gate hooks on real launches

Address the review findings on specialist lifecycle dispatch:

- specialistStop is now claimed in the parent session's event log via a
  record-once specialist_stop_hook event, sharing the accounting run-ID
  equivalence (empty runId is a catch-all). A later reaper in another
  process (TaskOutput poll, recovered TaskStop, restart) no longer re-runs
  the hook. The in-process claim remains only as fallback for swarm
  members with no parent session and for session-store errors.
- Recovered TaskStop records the terminal transition after Kill, so an
  explicitly stopped recovered task emits specialistStop exactly once.
- specialistStart fires only after the child process actually started
  (onStarted after Start()); a launch that fails before exec records
  attempt accounting but no public start/stop hooks. Reapers derive the
  same eligibility from the durable stop event's childStarted flag.

Adds a CLI startup wire-up test asserting both real entry points
(interactive TUI, headless exec) attach the lifecycle bridge to the same
LifecycleHooks the TaskOutput tool holds, fixes the stale import path on
the rebased branch, and documents the once-per-run semantics.

* fix(hooks): gate background exit accounting behind the start decision

A background child that exits immediately can run its launcher exit
callback before Executor.LaunchBackground returns, so the terminal
accounting could claim and dispatch specialistStop before runBackground
dispatched specialistStart, and the recorded stop missed the pid that
SetPID had not stored yet.

The exit callback now waits on a settled channel closed when runBackground
returns (after the start hook and SetPID), from its own goroutine so a
synchronous injected launcher cannot deadlock waiting for a signal only
the launch return can close.

Regression test: TestRunBackgroundDefersExitUntilStartDecisionCompletes
drives a synchronous fast-exit launcher and asserts start-before-stop hook
order, the recorded stop pid, and no deadlock. It fails on the previous
code with hooks "specialistStop,specialistStart".

* fix(hooks): record specialistStop only after the orphan kill succeeds

When SetPID fails after a background child has started, runBackground kills
the unrecorded child and records the stop. If TerminateProcess fails, the
child may still be running, but the branch still marked the task errored,
recorded the stop, took the durable specialist_stop_hook claim and fired
specialistStop. The hook could then run while the sub-agent was alive, and
the one-shot claim suppressed the hook on the child's real exit. Marking the
task non-running also let a TaskOutput or TaskStop reap claim the stop
before the launcher's Wait observed the exit.

On a failed kill, surface the joined error and return without the status
update or the stop. The task stays running, so reapers remain no-ops, and the
launcher's exit callback marks it exited and records the single stop when the
child ends. A successful kill keeps the immediate stop as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(hooks): keep the prompt on a failed orphan kill and share one stop claim

When SetPID fails and terminating the unrecorded child also fails, the child
may still be running and the task stays running. That branch still deleted
the --file prompt, so a child that had not yet read it failed at startup.
The launcher's exit callback already removes the prompt once the child
really exits, and Runtime.Close removes tracked prompts at shutdown, so the
failed-kill branch no longer removes it.

dispatchSpecialistStopHook made two independent dispatch decisions: the
durable specialist_stop_hook claim, and the in-process claim used when the
session store errors. A won durable claim never recorded the in-process
claim, so on one bridge a transient store error before or after a durable
claim dispatched the same stop twice, breaking the documented at-most-once
per process. A won durable claim now also takes the in-process claim
(including for an ineligible stop), and the hook runs only if both are free.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Ayush7614 <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-07 10:19:56 -08:00
05c1b5e160 feat(plugins): add CLI enable/disable for plugin.json (#1202)
* feat(plugins): add CLI enable/disable for plugin.json

Add zero plugins enable|disable so installed plugins can be toggled
without removing them, matching hooks/MCP parity and EXTENDING docs.

* fix(plugins): serialize toggles with installs and preserve manifest bytes

Address review findings on the enable/disable CLI:

- Take the same cross-process installtxn lock plugins add/remove take on
  the plugin's install root before the final lookup and read, and publish
  plugin.json with installtxn.WriteFileAtomically while holding it.
- Re-resolve the plugin under the lock so a concurrent install can never
  be overwritten with manifest bytes read before it committed, retrying
  when precedence moves to the other root; a plugin removed under the
  lock reports ErrNotInstalled instead of being recreated.
- Edit the enabled value in place (a targeted token walk) so field
  order, formatting, and numeric literals survive; the old
  decode/re-encode dropped precision (9007199254740993 became
  9007199254740992) and sorted the keys.
- Fail closed on manifest stat errors, refresh a clean plugins.lock hash
  after a toggle, and leave pre-existing drift in place instead of
  absorbing it into the lockfile.
- Redact the text-mode output, reject an empty plugin id as a usage
  error, and document project-over-user precedence in the help text.

* fix(plugins): recover interrupted install transactions before toggling

SetEnabledByID takes the plugin root's installtxn lock but went straight
to reading and rewriting the tree. installtxn.Recover requires every
holder of that lock to resolve interrupted transactions first and to
abort on its error, as Install and Remove already do.

Without it, a toggle after an install killed between its tree swap and
its lockfile publish edited the unpublished tree, and the next install
or removal restored the retained backup over it, silently undoing the
enable/disable. A transaction recovery could not resolve also failed to
stop the toggle.

Call installtxn.Recover with the plugins reconciler right after taking
the lock, and add regression tests for every interrupted state that
leaves the plugin discoverable, the unresolved-transaction abort, and a
plugin removed while the toggle waits on the lock.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(plugins): say when a plugin toggle takes effect

Zero reads plugin.json "enabled" only when it activates plugins at
startup (runExec and the interactive TUI bootstrap), and nothing reloads
them afterwards. A running interactive session therefore keeps the
tools, hooks, and skills of a plugin disabled from another terminal.

State that in the plugins docs and in the enable/disable help so the
documented activation rule matches what the command does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(plugins): pin toggled manifests, redact toggle errors, honor last enabled key

Address the review findings on the plugin enable/disable toggle.

Pin the plugin directory before editing its manifest. The install lock
only serializes cooperating installers, so swapping the plugin directory
for a link after discovery redirected the read and the atomic publish
through it, overwriting another manifest. The toggle now opens the
directory under its install root with pathjail, refuses a link or
reparse point, checks that the handle is the directory entry itself,
and reads, stats, creates the temporary file, and renames through that
handle.

Redact every stderr message the toggle writes. Parser and not-installed
usage errors echoed the raw argument, so a mistyped credential reached
stderr unredacted while the toggle's other output was redacted.

Validate only the last "enabled" value, as the loader does. The loader
keeps the last duplicate key, but the toggle rejected an earlier,
overwritten non-boolean value, so a plugin discovered as enabled could
not be toggled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(plugins): keep the toggle on the pinned plugin directory throughout

The previous fix pinned the manifest read and publish to a handle on the
plugin directory but still hashed the tree for plugins.lock by path, and
reported success without checking that the path still named the pinned
directory. If the directory was replaced after it was opened, the edit
landed in the moved-away original while the lockfile check and refresh
read the replacement, and the command reported a state the installed
plugin did not have.

Hash the tree before and after the edit through the pinned handle, and
confirm that the plugin's directory entry is still the pinned directory
before refreshing the lockfile or reporting any result, including a
no-op. hashTree now delegates to the same rooted walker, so path and
handle callers hash identical bytes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(plugins): hash plugin trees through native os.Root calls

20b484f moved hashTree onto an os.Root handle but walked it through
handle.FS(). That adapter accepts only fs.ValidPath names, which must be
UTF-8, so a plugin holding a Linux-valid name such as byte 0xff failed
with "lstat ...: invalid argument". copyTree and the previous walker
accept those names, so Install, the toggle's lock refresh, and Info's
drift check all regressed for such plugins.

Walk the handle with its own Open/Lstat methods instead. The walk stays
pinned to the directory, and the hash bytes match the merge-base
implementation (checked on a fixture with nested and non-UTF-8 names,
an exec bit, a symlink, .git and a FIFO).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Ayush7614 <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-07 09:56:16 -08:00
16ccd78765 feat(mcp): add AI/ML API as an opt-in built-in server (#1193)
* feat(aimlapi): shared service key and attribution for aimlapi.com services

Adds the pieces the Toolbox and MCP integrations share:

- internal/aimlapi: X-AIMLAPI-Source (agent/zero), the fixed Toolbox and
  MCP URLs, and an exact-hostname check (https, default port, no
  userinfo) for api/tools/mcp.aimlapi.com before the key or attribution
  headers are attached. The partner id still comes from ResolvePartnerID.
- internal/config: AIMLAPIServiceKey picks the key from the aimlapi
  profile (or AIMLAPI_API_KEY without one), reading the credential
  store, and withholds it when any aimlapi-keyed profile points at a
  proxy base URL or AIMLAPI_INFERENCE_URL is overridden. Keys that are
  not a single token are refused.

* feat(mcp): AI/ML API MCP server as an opt-in built-in default

Seeds `aimlapi` (https://mcp.aimlapi.com/mcp, streamable HTTP) next to
Exa as a built-in MCP server that ships disabled: its tool runs and
generations are billed to the user's AI/ML API balance. The disable is
marked as explicitly set, so a project config cannot lift it; the user
turns it on with `zero mcp enable aimlapi`, which now writes
`"disabled": false` for a default that ships off (and `zero mcp add
aimlapi` does the same). `zero doctor` reports the untouched default as
a pass rather than a warning.

The MCP client attaches the AI/ML API key and the X-AIMLAPI-Source /
X-AIMLAPI-Partner-ID headers in one place (internal/mcp/aimlapi.go),
called from the three header-application points. Both apply only to a
request whose own URL is exactly https://mcp.aimlapi.com on the default
port, whatever the server is named. The key is resolved for every
request through config.LoadAIMLAPIServiceKey, so switching the aimlapi
provider to a proxy, or a key that is not a single token, means no
Authorization header on the next request. It is never put in config,
the server identity used for OAuth token binding, list output or errors.
A user Authorization header, `"auth": "oauth"`, or a project-touched
server goes without it. A 401 from that host explains how to add a key
or sign in with `zero mcp oauth login`.

* fix(aimlapi): validate whitespace and preserve store errors

* fix(mcp): tailor unauthorized hints to credential policy

* docs(mcp): clarify automatic credential eligibility

* test(mcp): cover edited disabled built-in defaults

* fix(aimlapi): report store reads after proxy eligibility checks

* docs(aimlapi): document MCP key precedence and clarify 401 hints

Operator copy now matches normalizeProvider resolution (inline key,
apiKeyEnv, then credential store) and warns that AIMLAPI_API_KEY can
bill MCP before a stored profile key. SECURITY and service-key godoc
stay aligned. 401 hints no longer imply a proxy-only failure mode.

Adds a regression test that resolved profile keys win over the store.

Co-authored-by: Cursor <[email protected]>

* fix(mcp): keep the aimlapi key off redirects and error text, persist enable

Redirect hops: net/http copies the first request's headers onto every
redirect hop without calling the header hooks, so the automatic AI/ML API
key and attribution followed a same-origin hop that fails the exact-URL
rule (userinfo) and an eligible hop reused the first hop's key. The
redirect guard now restores the server's own values for those headers and
applies the per-request rule again.

Error text: httpStatusError copied up to 1 KiB of the response body into
the error, so a body echoing the injected key (or its prefix, when cut by
the read limit) reached CLI output. When the request carried the automatic
key, the body is now omitted instead of matched.

Enable persistence: MCPServerConfig had no serializer and Disabled is
omitempty, so any read-modify-write of config.json (SetTheme,
SetActiveProvider, ...) dropped `"disabled": false` and the shipped
disable turned aimlapi off again. MarshalJSON now writes the flag whenever
it was explicitly set; `zero mcp add` keeps writing only a disable.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Stan <[email protected]>
Co-authored-by: Hugo <[email protected]>
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 20:13:53 -08:00
c9f179482a feat(web-fetch): add an opt-in AI/ML API Toolbox scraping backend (#1192)
* feat(web-fetch): AI/ML API Toolbox as an opt-in web_fetch backend

With ZERO_WEBFETCH_PROVIDER=aimlapi, web_fetch fetches the page as Markdown
through the Toolbox tool search1api/crawl instead of a direct GET. A key
alone never selects it; unset or unknown values keep the direct fetch.

- The URL checks (pre-permission and resolved host) run before the URL is
  sent, so loopback, private and metadata URLs never reach the crawler.
- The key is resolved on every call through LoadAIMLAPIServiceKey; a
  missing, proxied or invalid key is an error with no request and no
  fallback. A failed run (is_error, failed status, no content) is also an
  error.
- Requests go only to aimlapi.ToolboxBaseURL, checked with
  IsAttributedURL before the key and attribution headers are attached;
  redirects are refused. The client uses the default transport, so the
  proxy environment applies and the target-site SSRF pin does not.
- Every run carries max_cost_usd 0.005, wait_ms 30000 and a fresh
  idempotency key. 202 runs are polled (hint clamped to 250 ms..5 s,
  120 s overall). 429/502/503/504 are retried with the identical body,
  honouring Retry-After (capped at 10 s); max_cost_exceeded is not.
- Output keeps the result shape with a Backend line, respects max_bytes,
  refuses format raw, and scrubs the key and redacts all Toolbox text.

* fix(web-fetch): honor poll retries and reject raw before permission

* docs(web-fetch): explain the bounded run response

* test(web-fetch): rename the Toolbox fake that collides with web_search tests

#1189 added a fakeToolbox type with a recorded method to the tools test
package. The web_fetch Toolbox tests declared their own fakeToolbox with a
different shape, so the package's tests no longer compiled once both were on
main. Rename the web_fetch one to webFetchFakeToolbox, matching the file's
webFetch prefix. Test-only; no behavior change.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(web-fetch): describe the aimlapi backend in the tool schema

With ZERO_WEBFETCH_PROVIDER=aimlapi the max_bytes and format descriptions
still promised direct-fetch behavior: raw body bytes before conversion and
HTML conversion with a raw option. The Toolbox backend returns crawled
Markdown, applies max_bytes to that Markdown and refuses format "raw", as
the README says. Say so in both descriptions so the model sees the same
contract.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(web-fetch): keep the AI/ML API key out of Toolbox errors and output

The Toolbox backend scrubbed the key with a literal string replace. A key
echoed back split by an invisible character (NUL, ZWSP) passed it on every
path the model sees: error bodies, run errors, page content and the run_id
meta field. scrubWebFetchKey now uses scrubAIMLAPIResult, the same
split-aware redaction the web_search backend uses.

A failed Toolbox request also returned the *url.Error from client.Do. That
error quotes the request URL, and a polling URL carries the run id
percent-escaped, a form no key scrubbing can match. send now reports the
cause without the URL, as web_search does.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(web-fetch): state only established billing for Toolbox runs

The README said every fetch is a billed Toolbox run, while the next
sentence says a run over max_cost_usd is refused and not charged, and a
code comment said a 404 comes back as an unbilled is_error run. The
AI/ML API Toolbox documentation says you pay for runs but not whether a
failed run is billed. Both now say only that Toolbox runs are billed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Stan <[email protected]>
Co-authored-by: Hugo <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 19:20:17 -08:00
98736457bd feat(web-search): add AI/ML API Toolbox as an opt-in backend (#1189)
* feat(aimlapi): shared service key and attribution for aimlapi.com services

Adds the pieces the Toolbox and MCP integrations share:

- internal/aimlapi: X-AIMLAPI-Source (agent/zero), the fixed Toolbox and
  MCP URLs, and an exact-hostname check (https, default port, no
  userinfo) for api/tools/mcp.aimlapi.com before the key or attribution
  headers are attached. The partner id still comes from ResolvePartnerID.
- internal/config: AIMLAPIServiceKey picks the key from the aimlapi
  profile (or AIMLAPI_API_KEY without one), reading the credential
  store, and withholds it when any aimlapi-keyed profile points at a
  proxy base URL or AIMLAPI_INFERENCE_URL is overridden. Keys that are
  not a single token are refused.

* feat(web-search): AI/ML API Toolbox as an opt-in web_search backend

ZERO_WEBSEARCH_PROVIDER=aimlapi runs the Toolbox tool web/search at the
fixed https://tools.aimlapi.com/v1/tools URL. It is selected only by that
setting; an AI/ML API key alone never enables it or registers web_search.

- The key comes from config.LoadAIMLAPIServiceKey on every search; a
  missing, proxy-profile or invalid key fails before any request.
  ZERO_WEBSEARCH_BASE_URL and ZERO_WEBSEARCH_API_KEY are ignored.
- The key and attribution headers are attached only after
  aimlapi.IsAttributedURL accepts the URL, and redirects are refused.
- Every run carries max_cost_usd 0.025 and a fresh idempotency key;
  limit maps to max_results and domains to include_domains, with the
  local domain filter kept.
- A queued run (202) is polled until it ends, honouring
  next_poll_hint_ms clamped to 250 ms..5 s. This backend gets a 30 s
  budget with wait_ms 20 s; the generic backends keep 10 s.
- Non-2xx responses, is_error runs, failed/stopped/timed_out runs and
  malformed bodies are errors carrying the error code and request id,
  with the key scrubbed.

* fix(aimlapi): preserve store errors and isolate Windows config tests

* fix(web-search): redact results and preserve domain errors

* docs(web-search): clarify minimum search charge

* fix(aimlapi): reject Unicode whitespace in service keys

* fix(aimlapi): report store reads after proxy eligibility checks

* fix(web-search): keep plain no-results for unscoped backends

Only a backend that applied the domain allowlist itself (the AI/ML API
Toolbox) should turn an empty result into the "no results matched
domains" error. A generic backend never saw the allowlist, so an empty
result from it is an ordinary "No results for query", as before this
branch.

Also clear ZERO_WEBSEARCH_PROVIDER in the unconfigured registry test:
ZERO_WEBSEARCH_PROVIDER=aimlapi alone now registers web_search, so an
exported value flipped the test's oracle.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(aimlapi): resolve the Toolbox key without an active provider

A user config that lists several providers but marks none active (or
names a removed one) makes Resolve return ErrNoActiveProvider together
with the normalized profiles. The app keeps running on that partial
config, but LoadAIMLAPIServiceKey stopped on the error, so web_search
failed even with an aimlapi profile or AIMLAPI_API_KEY set. Look the
key up in the partial config instead; the proxy check still scans every
resolved profile, and all other resolve errors are still returned.

Also state the key rules in the README as implemented: user config
only (project .zero providers are not used), which aimlapi profile is
used, and that any aimlapi profile or any profile reading
AIMLAPI_API_KEY on a non-canonical base URL withholds the key.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(aimlapi): withhold the Toolbox key for dropped proxy profiles

Resolve drops a non-active profile it cannot normalize (an unknown or
unsupported catalog entry, a missing base URL, ...), so the proxy check
never saw it. A dropped profile that reads AIMLAPI_API_KEY from a proxy
still shows the key belongs to that proxy, yet the key was sent to
tools.aimlapi.com. LoadAIMLAPIServiceKey now also checks the configured
user profiles, merged the same way Resolve merges them, and withholds
the key when a dropped profile carries it and is not explicitly on the
canonical base URL.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(web-search): pin the generic backend route in BASE_URL registration tests

defaultSearchBackend now checks ZERO_WEBSEARCH_PROVIDER=aimlapi before
ZERO_WEBSEARCH_BASE_URL. TestWebSearchRegisteredInCoreNetworkTools and
TestCoreNetworkToolsExposeSafetyMetadata set only the base URL and inherit
the provider from the process environment, so with an exported
ZERO_WEBSEARCH_PROVIDER=aimlapi they took the Toolbox route and still passed
even if generic base-URL registration broke. Clear the provider so they
exercise the generic route they describe, matching
TestCoreNetworkToolsOmitWebSearchWhenUnconfigured.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): redact caller-supplied secrets split by invisible characters

RedactString replaced Options.ExtraSecretValues only as contiguous text,
and the gap-tolerant pass from #1067 covers only the built-in credential
shapes. An opaque key such as the AI/ML API service key, echoed by a
service with a NUL or zero-width space inside it, matched neither pass.
Both fragments then reached Toolbox web_search errors (before clipping)
and successful results sent to the model and transcript.

Match each caller-supplied secret with runs of the same invisible
separators allowed between its characters, right after the contiguous
replacement. Separators inside the match are covered; those beside it
stay. Text without a separator takes the old path unchanged.

Regressions: split explicit values in the redaction package, and split
keys in the Toolbox HTTP error, run error and successful-result paths.
Raised by Copilot on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): claim each caller secret whole or split before shorter ones

RedactString replaced every ExtraSecretValues entry contiguously, longest
first, and only then ran the split match. A shorter secret inside a
longer one that was split by an invisible character was replaced first,
so "abc\x00def" with secrets abcdef and abc became "[REDACTED]\x00def"
and the rest of the longer secret stayed visible.

Run the contiguous and split match together per value, longest first,
so a longer value is claimed in either form before any shorter one.
Regressions cover a shorter secret at the start, end and middle of a
split longer one, and alongside it. Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): merge overlapping caller-secret occurrences before replacing

ExtraSecretValues were replaced one occurrence at a time, so each
replacement could consume bytes another occurrence needed. With secret
abcabc, "abc\x00abcabc" became "abc\x00[REDACTED]". A key whose first and
last characters match lost all but one character the same way
("xY7q​[REDACTED]"), and two partly overlapping values left the
rest of one visible. Reordering the passes cannot fix this: the
contiguous and regexp replacements are both non-overlapping.

Find every occurrence of every value, whole or split by invisible
separators, in one separator-free copy of the text (linear KMP, all
overlapping starts), map the spans back, merge overlapping ones, and
replace each merged span once. Occurrences that only touch stay
separate replacements; separators beside a match stay; tab, newline
and carriage return are still not gaps. This honours the existing
TestRedactString_OverlappingExtraSecrets contract for both forms.
Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(web-search): keep the request URL out of Toolbox transport errors

When http.Client.Do failed, the backend scrubbed err.Error() from the
*url.Error, which quotes the request URL. A polling URL carries the run
id the Toolbox returned, percent-escaped, so a key echoed in that id
appeared as "Lp9%00Xr4" or "Lp9%E2%80%8BXr4". The key scrubber matches
the key whole or split by raw invisible separators, not percent-escapes,
so the key stayed recoverable in the web_search error.

Report only the cause inside the *url.Error. The message prefix, the
redirect refusal text and key scrubbing are unchanged. The regression
fails a polling GET after a queued run whose id carries a NUL- or
ZWSP-split key and checks the output has neither the polling URL nor
the key. Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(redaction): always redact a caller secret as written

Once the text held any invisible separator, redactExtraSecretValues
matched each caller secret only with separators removed from both. That
replaced the contiguous match instead of adding to it: a secret made
only of separators compacted to nothing and was skipped, and separators
at either end of a secret fell outside its match. RedactString("a\x00b")
with secret "\x00" returned the text unchanged, where main replaced it.

In the separator path, also record every occurrence of the secret as
written, then merge spans as before. Text without a separator takes the
unchanged path. Raised by CodeRabbit on #1189.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Stan <[email protected]>
Co-authored-by: Hugo <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-04 18:24:09 -08:00
ShryukGrandhi e59632406b test(agent): isolate user home, config and cache dirs for the agent tests (#1184)
Sandboxed tool runs in the agent tests keep runtime state and lease files
under os.UserCacheDir, so go test ./internal/agent/ left
zero/runtime/v1/*.lease files in the developer's real cache. TestMain
now points every per-user root at its temp dir via
testutil.IsolateUserDirs, as internal/tui does, and a guard test pins it.

Refs #1103

Signed-off-by: ShryukGrandhi <[email protected]>
2026-11-04 12:27:48 +08:00
github-actions[bot] 7c55a09c07 chore(main): release 0.12.0 (#1150)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-03 08:10:03 +08:00
Octopusandocto-patch d99f92b81a feat(providers): add MiniMax-M3.1-Flash to catalog (#1180)
* feat(providers): add MiniMax-M3.1-Flash to catalog

* fix(providers): forward MiniMax adaptive effort

---------

Co-authored-by: octo-patch <[email protected]>
2026-11-02 18:28:17 +08:00
zzucs 5aa54fe1d2 fix(agent): mark headless guardrail stops incomplete (#1182)
* fix(agent): mark headless guardrail stops incomplete

The empty-turn and repeated-tool-failure guards returned the run as a
successful final answer even with RequireCompletionSignal enabled, so
zero exec reported exit 0 and a success run_end for work it had just
abandoned.

Set Result.Incomplete and a reason at both stops when the headless gate
is on. Thresholds, stop answers, retry behavior, interactive runs, and
--no-completion-gate are unchanged.

Fixes #1154

* fix(agent): count consecutive empty turns in stop reason

The headless no-output stop reason used result.Turns, which counts every
turn in the run, so a run with an earlier tool-call turn claimed "stopped
after 4 turns with no output" for a 3-turn empty streak. Use the guard's
emptyTurns counter and say "consecutive".

Pinned by a case with a successful tool turn before the empty streak.
2026-11-02 14:25:40 +08:00
oniisan f912f80d00 fix(chatgpt): refresh subscription defaults and compatibility version (#1179)
* fix(chatgpt): refresh subscription defaults and compatibility version

Replace retired preset suggestions with vendor-documented Sol, Terra, Astra, and Luna IDs. Update the Codex discovery compatibility floor for Terra and verify that saved model selections are retained.

Refs Twigpine/zero#1177 (defaults portion)

* test(chatgpt): assert discovered model in compatibility probe

Verify the live response contains exactly gpt-future-coder in addition to checking the Codex compatibility query value.

Refs Twigpine/zero#1177

* docs(chatgpt): align Terra description with vendor guidance
2026-11-02 14:25:00 +08:00
ShryukGrandhi 9c4e3693e7 test(providerio): widen the idle window in the content-reset test (#1172)
TestScanSSEDataWithContextContentResetsOnData wrote a data line every 30ms against a 100ms idle timeout, so a ~70ms scheduling stall on a shared runner tripped the idle watchdog. Use a 500ms idle timeout (600ms content window) and 25 writes (~750ms), which keeps every gap far under the idle timeout while the run still outlasts the content window.

Signed-off-by: ShryukGrandhi <[email protected]>
2026-11-02 14:24:24 +08:00
zzucs 63962081cf fix(sessions): capture freeform apply_patch checkpoints (#1183)
* fix(sessions): capture freeform apply_patch checkpoints

Freeform apply_patch arguments are raw patch text, but both checkpoint
paths passed them to json.Unmarshal and dropped the error, so they found
no mutation targets and recorded no checkpoint. The tool still executed,
leaving rewind unable to restore the edit.

Resolve targets through a shared agent.CheckpointTargets helper that
prepares freeform input on the registered built-in apply_patch, exactly
as executeToolCall does. This keeps absolute patch headers
workspace-relative (rewriting them and setting cwd) instead of handing
absolute paths to the checkpoint layer. Non-freeform calls now use the
same tolerant decoder as execution, so the two cannot disagree.

Covers the CLI, spec-draft, and TUI capture paths.

Fixes #1162

* test(sessions): canonicalize freeform absolute-header fixtures

t.TempDir returns the symlinked /var spelling on macOS and short names on
Windows, while the apply_patch tool resolves the physical root. The
absolute-header cases must build the header from the resolved root, as the
repo's cross-platform test guidance requires.
2026-11-02 14:23:58 +08:00
Vasanth T 2e10a4b60b test(swarm): wait for every slot to be released before checking the drain (#1181)
A finishing member's task reads Done before its slot is freed: the
supervisor completes the task, removes the member and only then frees
the slot in afterExit. The test waited for Done == 5 and checked the
running count straight after, so a busy runner could see one slot still
held. It now waits for the drain itself.
2026-10-29 06:30:41 -07:00
Vasanth T 4247a07aba test(tools): stop the tty session before removing its root (#1175)
TestExecCommandTTYSkipsInteractiveBlock runs ssh to a bogus host under
tty with a 250ms yield. When ssh is still resolving the host after the
yield, the session keeps running in the test's directory and nothing
stops it, so on Windows t.TempDir's cleanup fails with a sharing
violation. It failed Smoke (windows-latest) on a README-only PR.

Stop the session in a cleanup, and build the tool on execTestRoot,
which retries the removal while Windows still holds the directory right
after the stop, as the other exec tests that leave a session do.
2026-10-27 06:25:41 -07:00
Vasanth T b837448dd4 docs: list Vercel under Partners (#1176)
Adds Vercel beside GMI Cloud in the Partners table of both READMEs,
with Vercel's official logotype in light and dark variants, taken
unchanged from its brand page apart from a trailing newline.
2026-10-27 15:29:59 +08:00
Vasanth T a0b036a3c7 docs: add Vercel OSS Program badge (#1174)
* docs: add Vercel OSS Program badge

* docs: add the Vercel OSS Program badge to README_ZH too

The two READMEs carry the same badge row, so the Chinese one gets the
badge as well. Both now list alt before src, like the other badges in
the row.
2026-10-27 12:30:50 +05:30
Vasanth T 164150633a feat(skills): install a skill's bundled files and point the skill tool at them (#1167)
* feat(skills): install a skill's bundled files and point the skill tool at them

A skill was only its SKILL.md: install copied that one file, and the skill
tool returned only the markdown body, so a skill whose instructions run its
own scripts was broken by installing it, and a hand-placed skill directory
gave the model no way to find the files beside it.

Install now copies the whole skill directory, leaving out hidden entries
(.git, .env) and symlinks, which it neither follows nor copies but reports.
A script stays executable; everything else is installed 0644. A tree over
1,000 files or 32 MiB is refused. skills.lock records a treeHash beside the
SKILL.md hash; skill info reports drift in bundled files, and an entry from
before this is still checked on SKILL.md alone. A reinstall that changed
only a bundled file is reported as an update.

Both skill tools end their output with the skill's directory and a list of
up to 50 bundled files when there are any; a SKILL.md-only skill is
returned unchanged.

Closes #584

* fix(skills): compare a reinstall over an old lock entry with SKILL.md alone

A lock entry written before tree hashes has no treeHash, so the reinstall
compared the new tree hash with an empty string and always counted as an
update. Re-adding an unchanged skill that is only a SKILL.md printed
"Updated" and "the files bundled with it changed" when there were none.

That entry comes from an install that copied SKILL.md alone, so compare
with that: it is an update when SKILL.md changed or the reinstall now
brings bundled files. InstallResult gains previousTreeHash, and the CLI
uses it to say the bundled files are now installed rather than changed
on the first reinstall over such an entry.

* fix(skills): match the tree hash when recovering an interrupted install

lockReconciler matched both trees of an interrupted workspace on the
SKILL.md hash alone. A reinstall that changes only a bundled file keeps
SKILL.md, so a kill between the tree swap and the lockfile publish left
two trees carrying the recorded hash, and recovery kept the new one and
retired the backup the lockfile still described.

A tree now matches an entry with a tree hash only when both hashes
match. An entry from before tree hashes is still matched on SKILL.md
alone. A tree hash that cannot be computed is returned as an error, the
way an unreadable SKILL.md already is, so recovery decides nothing about
a tree it could not read.

Also bring the package comment up to date: install copies the skill
directory, not SKILL.md alone.
2026-10-25 07:24:59 -07:00
226 changed files with 16675 additions and 14369 deletions

No files matched your search

+1 -1
View File
@@ -1 +1 @@
{".":"0.11.0"}
{".":"0.12.0"}
+24
View File
@@ -6,6 +6,30 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once the first release is
tagged. Until then, source builds report the version `dev`.
## [0.12.0](https://github.com/Twigpine/zero/compare/v0.11.0...v0.12.0) (2026-11-02)
### Features
* **providers:** add MiniMax-M3.1-Flash to catalog ([#1180](https://github.com/Twigpine/zero/issues/1180)) ([d99f92b](https://github.com/Twigpine/zero/commit/d99f92b81a671f186cede81614c89b8e8a7f9451))
* **sandbox:** support self-exec pattern to eliminate separate helper binary ([#946](https://github.com/Twigpine/zero/issues/946)) ([8c4e759](https://github.com/Twigpine/zero/commit/8c4e7597989ef9b3ca67b7703dfd15d92372aed2))
* **skills:** install a skill's bundled files and point the skill tool at them ([#1167](https://github.com/Twigpine/zero/issues/1167)) ([1641506](https://github.com/Twigpine/zero/commit/164150633a8b2e6ae864ae02fd7105b201eeb68e))
### Bug Fixes
* **agent:** mark headless guardrail stops incomplete ([#1182](https://github.com/Twigpine/zero/issues/1182)) ([5aa54fe](https://github.com/Twigpine/zero/commit/5aa54fe1d2b419c9bbef73605a77919c9007cf7b))
* bump Go to 1.26.10 for GO-2026-6809 ([#1168](https://github.com/Twigpine/zero/issues/1168)) ([6393a90](https://github.com/Twigpine/zero/commit/6393a90ed618a3536e09c02852f4ec829b97545b))
* **chatgpt:** refresh subscription defaults and compatibility version ([#1179](https://github.com/Twigpine/zero/issues/1179)) ([f912f80](https://github.com/Twigpine/zero/commit/f912f80d002602e11ba0ef8548ecee58442874aa))
* **deps:** refresh the lockfile to hono 4.13.13 ([#1171](https://github.com/Twigpine/zero/issues/1171)) ([de1443b](https://github.com/Twigpine/zero/commit/de1443b2edb44fd3747cc77200d54dc8cffcd0a7))
* **oauth:** reuse the crash-safe secret store and sync encrypted publications ([#1165](https://github.com/Twigpine/zero/issues/1165)) ([84c1b15](https://github.com/Twigpine/zero/commit/84c1b1554076679a342e9cac61f8f8cdbaf3a429))
* **redaction:** redact credentials whose trigger a greedy span swallowed ([#1158](https://github.com/Twigpine/zero/issues/1158)) ([a97db88](https://github.com/Twigpine/zero/commit/a97db8832fdba077f2d31bdf03ecef8d7c774fe1))
* **sessions:** capture freeform apply_patch checkpoints ([#1183](https://github.com/Twigpine/zero/issues/1183)) ([6396208](https://github.com/Twigpine/zero/commit/63962081cf130c2171407936ce3beb7021d238b8))
* **sessions:** preserve image attachment summaries on resume ([#1152](https://github.com/Twigpine/zero/issues/1152)) ([1e84849](https://github.com/Twigpine/zero/commit/1e848492e5c1919d187fb5e1eb5e7ef9bc6140dc)), closes [#1135](https://github.com/Twigpine/zero/issues/1135)
* **tools:** grep stops where a file's line breaks stop ([#1160](https://github.com/Twigpine/zero/issues/1160)) ([aa9ba84](https://github.com/Twigpine/zero/commit/aa9ba8412eb8917daa2ca5cb3fdbd0a39276fa88))
* **workspaceindex:** import gitsafe under the Twigpine module path ([#1153](https://github.com/Twigpine/zero/issues/1153)) ([ef4da13](https://github.com/Twigpine/zero/commit/ef4da1323f1097863ae278224b5f304c2fff909a))
* **workspaceindex:** run Scan's git calls through gitsafe ([#1149](https://github.com/Twigpine/zero/issues/1149)) ([2ffe183](https://github.com/Twigpine/zero/commit/2ffe1831b0950b1463ec46d487d03a5fcb679cc6))
## [0.11.0](https://github.com/Twigpine/zero/compare/v0.10.0...v0.11.0) (2026-10-15)
+80
View File
@@ -15,6 +15,7 @@
<img alt="Go 1.26.10+" src="https://img.shields.io/badge/Go-1.26.10+-00ADD8?logo=go&logoColor=white">
<img alt="25+ providers" src="https://img.shields.io/badge/providers-25+-34E2EA">
<a href="https://discord.gg/CaQDS6wdFn"><img alt="Discord" src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white"></a>
<a href="https://vercel.com/open-source-program"><img alt="Vercel OSS Program" src="https://vercel.com/oss/program-badge-2026.svg"></a>
<br>
<strong>English</strong> | <a href="README_ZH.md">中文</a>
</p>
@@ -87,9 +88,18 @@ zero exec --output-format stream-json < turns.jsonl
</picture>
</a>
</td>
<td align="center" width="150" height="80">
<a href="https://vercel.com/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/vercel-logo-dark.svg">
<img src="docs/assets/vercel-logo.svg" alt="Vercel logo" width="120">
</picture>
</a>
</td>
</tr>
<tr>
<td align="center"><a href="https://www.gmicloud.ai/"><strong>GMI Cloud</strong></a></td>
<td align="center"><a href="https://vercel.com/"><strong>Vercel</strong></a></td>
</tr>
</table>
@@ -387,6 +397,76 @@ and can rewrite every request through it, so this is the trust you accept by
configuring one. Leave the variables unset for the checks to be enforced end to
end.
The built-in `web_search` tool is registered only when a search backend is
configured: `ZERO_WEBSEARCH_BASE_URL` (with an optional `ZERO_WEBSEARCH_API_KEY`)
for a generic JSON endpoint, `ZERO_WEBSEARCH_PROVIDER=searxng` with a SearXNG
base URL, or the AI/ML API Toolbox:
```bash
export ZERO_WEBSEARCH_PROVIDER=aimlapi
```
The AI/ML API backend runs the Toolbox tool `web/search`, and every search is
billed to your AI/ML API balance: about $0.0013 per result, with a documented
minimum of $0.0091 per search. Each search is sent with a $0.025 spending cap,
which the Toolbox enforces. It is used only when
`ZERO_WEBSEARCH_PROVIDER` is set to `aimlapi`; an AI/ML API key alone never
enables it. The key is read from your user configuration only; provider
profiles in a project `.zero` config are not used. It is the aimlapi profile's
key (the active profile when it is aimlapi, otherwise the first aimlapi
profile), including a key in the credential store, or `AIMLAPI_API_KEY` when no
aimlapi profile exists. It is resolved again on every search;
`ZERO_WEBSEARCH_API_KEY` is not used. When any aimlapi profile, or any profile
that reads `AIMLAPI_API_KEY`, points at a base URL other than
`https://api.aimlapi.com/v1`, or `AIMLAPI_INFERENCE_URL` is overridden, the key
may belong to a proxy, so it is not sent and the search fails without a request.
Requests go only to
`https://tools.aimlapi.com` (`ZERO_WEBSEARCH_BASE_URL` is ignored and redirects
are refused) and carry `X-AIMLAPI-Source: agent/zero` and an
`X-AIMLAPI-Partner-ID` header. The `domains` argument is sent to the Toolbox as
`include_domains`, and results are still filtered locally.
### AI/ML API Toolbox backend for `web_fetch`
`web_fetch` can fetch pages through the AI/ML API Toolbox tool
`search1api/crawl` instead of a direct HTTP GET. It is off unless you choose it:
```bash
export ZERO_WEBFETCH_PROVIDER=aimlapi
```
Having an AI/ML API key never switches it on by itself, and any other value (or
none) keeps the direct fetch. The variable is read on every fetch.
- **Billing:** Toolbox runs are billed to your AI/ML API balance, about
$0.0013 per run. Each run is sent with `max_cost_usd` 0.005, so a run
priced above that is refused before it starts and not charged. Rate-limited
or gateway-failed requests are retried with the same idempotency key, so a
retry is never billed twice.
- **Key:** the key comes from your `aimlapi` provider profile (including a key
in the credential store), or from `AIMLAPI_API_KEY` when there is no such
profile. It is withheld, and `web_fetch` returns an error, when any `aimlapi`
profile or any profile using `AIMLAPI_API_KEY` points at a base URL other than
`https://api.aimlapi.com/v1`, or when `AIMLAPI_INFERENCE_URL` is overridden:
the key may then belong to a proxy.
- **Destination:** requests go only to the fixed `https://tools.aimlapi.com`
endpoint, with `Authorization: Bearer <key>` and the attribution headers
`X-AIMLAPI-Source: agent/zero` and `X-AIMLAPI-Partner-ID`. Redirects are
refused. `HTTP_PROXY`/`HTTPS_PROXY` apply to these requests as to Zero's other
aimlapi.com calls.
- **No fallback:** without a usable key, or when the run fails, `web_fetch`
returns an error and does not fetch the page directly. Unset the variable to
go back to direct fetches.
- **Remote fetch:** the page is fetched by a remote service, which may serve a
cached copy, and the full URL (query string included) is sent to it. The URL
checks above still run first, so loopback, private and metadata URLs are
never handed to it. The result is Markdown with a
`Backend: aimlapi (search1api/crawl)` line; `max_bytes` cuts the Markdown,
and `format: "raw"` is refused because the original response is not
available. The complete Toolbox run response is limited to 8 MiB before JSON
decoding; larger responses return an error even if `max_bytes` is smaller.
The remote run may already have been billed when this limit is detected.
For local dev servers, use shell commands such as `curl` through `exec_command`
so the normal sandbox and permission policy applies. Long-running commands stay
attached to a background terminal session and can be listed or stopped from the
+10
View File
@@ -15,6 +15,7 @@
<img alt="Go 1.26.10+" src="https://img.shields.io/badge/Go-1.26.10+-00ADD8?logo=go&logoColor=white">
<img alt="25+ providers" src="https://img.shields.io/badge/providers-25+-34E2EA">
<a href="https://discord.gg/CaQDS6wdFn"><img alt="Discord" src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white"></a>
<a href="https://vercel.com/open-source-program"><img alt="Vercel OSS Program" src="https://vercel.com/oss/program-badge-2026.svg"></a>
<br>
<a href="README.md">English</a> | <strong>中文</strong>
</p>
@@ -85,9 +86,18 @@ zero exec --output-format stream-json < turns.jsonl
</picture>
</a>
</td>
<td align="center" width="150" height="80">
<a href="https://vercel.com/">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/vercel-logo-dark.svg">
<img src="docs/assets/vercel-logo.svg" alt="Vercel logo" width="120">
</picture>
</a>
</td>
</tr>
<tr>
<td align="center"><a href="https://www.gmicloud.ai/"><strong>GMI Cloud</strong></a></td>
<td align="center"><a href="https://vercel.com/"><strong>Vercel</strong></a></td>
</tr>
</table>
+13 -1
View File
@@ -33,4 +33,16 @@ Please include, where possible:
depend on a user explicitly disabling the sandbox (`--skip-permissions-unsafe`), are generally
out of scope — but please report anything you are unsure about.
- Zero sends no telemetry. Network calls go to the model/tool providers you configure (and, for
keyless web search, to Exa's hosted MCP endpoint at `https://mcp.exa.ai/mcp`).
keyless web search, to Exa's hosted MCP endpoint at `https://mcp.exa.ai/mcp`). The built-in
AI/ML API MCP server (`https://mcp.aimlapi.com/mcp`) is off until you run
`zero mcp enable aimlapi`. Once enabled, the Bearer key follows the aimlapi
profile resolution order (inline key, then the profile's `apiKeyEnv` — default
`AIMLAPI_API_KEY` — then the credential store), or `AIMLAPI_API_KEY` alone when
no aimlapi profile exists; a non-empty `AIMLAPI_API_KEY` can override a stored
profile key for MCP billing. The automatic key is withheld if an aimlapi profile
(or a profile reading `AIMLAPI_API_KEY`) uses a base URL other than
`https://api.aimlapi.com/v1`, `AIMLAPI_INFERENCE_URL` points elsewhere, a
non-empty user Authorization header is configured, OAuth is selected, project
config has touched the server, or the key is missing or not a single token
(including Unicode whitespace). See
[docs/EXTENDING.md](docs/EXTENDING.md#built-in-servers).
+65 -2
View File
@@ -110,6 +110,8 @@ Discovery roots (earlier wins on name collisions):
A missing directory is fine — Zero just omits it. Management commands (`zero skill add` / `zero skill remove`) always write to the primary Zero dir only; `list` / `info` search primary + `~/.agents/skills`. `skill info` reports lock source/hash when present and whether the on-disk `SKILL.md` has drifted from the recorded hash.
For a skill installed since Zero started installing bundled files, `skill info` also reports drift in any file installed with it.
```text
~/.local/share/zero/skills/ # primary (install/remove target)
run-benchmarks/
@@ -140,6 +142,15 @@ Only `name` and `description` are recognized in the frontmatter today. The `name
The `skill` tool lets the agent load any discovered skill by name (primary, agents, or plugin).
A skill can ship files beside its `SKILL.md`, in the [agentskills.io](https://agentskills.io/specification) layout: `scripts/`, `references/`, `assets/` or anything else the instructions refer to. `zero skill add` installs the whole skill directory, leaving out hidden entries such as `.git` and not following or copying symlinks (it lists any it skipped). A skill larger than 1,000 files or 32 MiB is refused. When the `skill` tool loads a skill that has such files, its output ends with the skill's directory and a list of the files, so instructions like "run `scripts/fill.py`" can be followed. Running a bundled script goes through the normal tool permissions; installing or loading a skill never runs anything.
```text
pdf/
SKILL.md
scripts/fill.py
references/form-fields.md
```
## 4. Hooks
Hooks fire shell commands on lifecycle events. Configure them in JSON:
@@ -192,7 +203,11 @@ Events the agent emits (in dispatch order):
| `specialistStart` | A sub-agent is spawned | yes (specialist name) |
| `specialistStop` | A sub-agent ends | yes (specialist name) |
A hook's exit code decides what happens next: `0` continues, non-zero blocks the tool call (`beforeTool`) or surfaces an error (`afterTool`). Hook execution is recorded in the audit log; the audit is reachable from the agent's view of past actions, not from a dedicated `zero doctor` check.
For swarm members the matcher subject is the **agent type** (`subagent`, `teammate`, …), not a registered specialist profile name — swarm members run from an inline manifest keyed by agent type.
`specialistStart` fires once the child process has actually started; a launch that fails before exec (missing binary, spawn error) records no public lifecycle hooks. `specialistStop` fires once per terminal stop: completion, error, explicit stop of a recovered task, and a background task reaped by a later `TaskOutput` all share one durable claim, so a hook that notifies or cleans up does not run twice for the same sub-agent run.
A hook's exit code decides what happens next: `0` continues, non-zero blocks the tool call (`beforeTool`) or surfaces an error (`afterTool`). `sessionStart` / `sessionEnd` / `specialistStart` / `specialistStop` hooks are advisory (failures are audited but do not interrupt the run). Hook execution is recorded in the audit log; the audit is reachable from the agent's view of past actions, not from a dedicated `zero doctor` check.
> **Roadmap.** An in-UI hooks manager is on the backlog. Today you edit the JSON directly.
@@ -246,6 +261,52 @@ or use a new server name. OAuth tokens are bound to the resolved server identity
so changing a server target can require `zero mcp oauth login <server>` again.
OAuth server names ending in `.<32 hex chars>` are reserved for token storage.
#### Built-in servers
Zero seeds two MCP servers before your config is merged. Both show up in
`zero mcp list`, and any field can be overridden in your user config.
- `exa` (`https://mcp.exa.ai/mcp`) is on by default for keyless web search.
`zero mcp disable exa` turns it off.
- `aimlapi` (`https://mcp.aimlapi.com/mcp`, the AI/ML API Toolbox, model
discovery, image/video/audio generation, chat, embeddings and balance tools)
is **off by default**, because its tool runs and generations are billed to
your AI/ML API balance. Turn it on with `zero mcp enable aimlapi`, which
writes only `"disabled": false` to your user config, and off again with
`zero mcp disable aimlapi`. A project config cannot turn it on.
When `aimlapi` connects, Zero sends your AI/ML API key as
`Authorization: Bearer <key>`. The key is resolved again on every request and
is never written to `config.json`. With an aimlapi.com provider profile
(`zero providers setup aimlapi`), the key is chosen in this order:
1. An inline key on the profile (if set in config).
2. Otherwise the environment variable named by the profile's `apiKeyEnv`
(the catalog default is `AIMLAPI_API_KEY`).
3. Otherwise the key held in the credential store when the profile uses
`apiKeyStored`.
When there is no aimlapi profile, Zero uses `AIMLAPI_API_KEY` alone. If both
`AIMLAPI_API_KEY` and a stored profile key exist, step 2 wins over the store
whenever the inline key is empty — MCP requests may bill the environment key's
account, not the stored key.
No automatic key is sent when an aimlapi profile, or any profile reading
`AIMLAPI_API_KEY`, uses a base URL other than `https://api.aimlapi.com/v1`, or
when `AIMLAPI_INFERENCE_URL` is overridden, because that credential may belong
to a proxy. No key is sent when the resolved value is not a single token (for
example a comma- or newline-separated pool), when you set your own
non-empty `Authorization` header, when the server uses `"auth": "oauth"`, or
when project config touched the server. Without a key the server answers 401.
To sign in with your aimlapi.com account instead, set `"auth": "oauth"` on the
server and run `zero mcp oauth login aimlapi`.
Every request to exactly `https://mcp.aimlapi.com` (https, default port,
whatever the server is named, including one you add by hand) also carries the
attribution headers `X-AIMLAPI-Source: agent/zero` and `X-AIMLAPI-Partner-ID`,
and gets the key under the rules above. Other MCP servers never receive these
headers or the key.
### As a server — expose Zero's tools to another agent
```bash
@@ -290,10 +351,12 @@ Install and manage:
zero plugins add ./github-pr-review # copy into ~/.config/zero/plugins/ or ./.zero/plugins/
zero plugins list
zero plugins info github-pr-review # show manifest path, enabled state, lock metadata
zero plugins disable github-pr-review # sets "enabled": false in plugin.json
zero plugins enable github-pr-review # sets "enabled": true in plugin.json
zero plugins remove github-pr-review # alias: rm
```
A plugin is enabled by being present in the plugins directory and disabled by removing it (or by the user setting `"enabled": false` in its `plugin.json`). Plugins are not enabled or disabled by a CLI subcommand today.
A plugin is active when it is present under a plugins directory and its `plugin.json` has `"enabled": true` (the default when the field is omitted). Use `zero plugins disable <id>` / `zero plugins enable <id>` to toggle that field without removing the plugin. Pass `--user` to target only user plugins and ignore `./.zero/plugins`; without it, a project plugin with the same id takes precedence. The toggle edits the `enabled` value in place — the rest of the manifest keeps its formatting, field order, and numeric literals — and serializes with `plugins add` / `plugins remove` on the same plugins root. Zero reads plugins when it starts, so the change applies to the next `zero exec` run or interactive session; an interactive session that is already running keeps the plugin tools, hooks, and skills it started with until you restart it.
Plugin commands run with the plugin directory as their working directory. Use relative paths; the loader resolves them at activation time.
+4
View File
@@ -634,6 +634,10 @@ Core tool groups include:
`HTTP_PROXY`/`HTTPS_PROXY` set, the proxy is dialed instead and resolves the
target itself, so destination enforcement ends at the proxy; see the note in
the README.
With `ZERO_WEBFETCH_PROVIDER=aimlapi` (opt-in; a key alone does not select
it), the same URL checks run and the page is then fetched as Markdown by the
billed AI/ML API Toolbox tool `search1api/crawl`, with no fallback to a
direct fetch; see the README.
- **Local-control tools**: config-gated wrappers for browser, terminal, desktop,
and artifact helpers backed by packages such as `internal/localcontrol` and
`internal/browser`.
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="261" height="52" fill="none" viewBox="0 0 261 52"><path fill="white" d="M59.8 52H0L29.9 0zm67.82-38.45q4.9 0 8.81 2.13a15.5 15.5 0 0 1 6.22 6.32q2.3 4.2 2.38 10.26v2.06h-26.35q.27 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.76-4.93l9.16.67q-1.68 4.98-6.29 7.99t-10.63 3q-5.53 0-9.64-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m62.55 0q4.47 0 8.18 1.66a15.3 15.3 0 0 1 6.15 4.6q2.38 3 2.87 7.05l-9.23.47a8 8 0 0 0-2.8-5 7.7 7.7 0 0 0-5.17-1.86q-4.33 0-6.7 3-2.4 3-2.39 8.52t2.38 8.52q2.37 3 6.71 3 3.15 0 5.39-1.87 2.24-1.92 2.72-5.46l9.3.4a14.7 14.7 0 0 1-2.87 7.33 16 16 0 0 1-6.15 4.86 21 21 0 0 1-8.39 1.66q-5.53 0-9.64-2.26a16 16 0 0 1-6.44-6.46 20 20 0 0 1-2.3-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m38.66 0q4.9 0 8.8 2.13a15.5 15.5 0 0 1 6.22 6.32q2.31 4.2 2.38 10.26v2.06h-26.35q.28 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.77-4.93l9.15.67q-1.68 5-6.29 7.99t-10.62 3q-5.53 0-9.65-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26M86.9 36.69l17.24-34.33h10.8L89.96 49.63h-6.12L58.85 2.36h10.81zm71.62-15.55a11 11 0 0 1 2.47-4.48q2.28-2.31 6.38-2.31h3.4v7.26h-3.47q-2.91 0-4.79.8a5.8 5.8 0 0 0-2.77 2.5q-.9 1.72-.9 4.37v20.35h-8.89V14.35h8.33zm101.73 28.5h-8.95V2.35h8.95zM127.62 20.26q-3.7 0-6 2.2-2.32 2.2-2.87 6.2h17.05q-.48-4.34-2.72-6.33a7.8 7.8 0 0 0-5.46-2.07m101.2 0q-3.7 0-6 2.2-2.31 2.2-2.87 6.2H237q-.5-4.34-2.73-6.33a7.8 7.8 0 0 0-5.46-2.07" style="fill:white;fill-opacity:1;"/></svg>

After

Width:  |  Height:  |  Size: 1.6 KiB

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="261" height="52" fill="none" viewBox="0 0 261 52"><path fill="black" d="M59.8 52H0L29.9 0zm67.82-38.45q4.9 0 8.81 2.13a15.5 15.5 0 0 1 6.22 6.32q2.3 4.2 2.38 10.26v2.06h-26.35q.27 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.76-4.93l9.16.67q-1.68 4.98-6.29 7.99t-10.63 3q-5.53 0-9.64-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m62.55 0q4.47 0 8.18 1.66a15.3 15.3 0 0 1 6.15 4.6q2.38 3 2.87 7.05l-9.23.47a8 8 0 0 0-2.8-5 7.7 7.7 0 0 0-5.17-1.86q-4.33 0-6.7 3-2.4 3-2.39 8.52t2.38 8.52q2.37 3 6.71 3 3.15 0 5.39-1.87 2.24-1.92 2.72-5.46l9.3.4a14.7 14.7 0 0 1-2.87 7.33 16 16 0 0 1-6.15 4.86 21 21 0 0 1-8.39 1.66q-5.53 0-9.64-2.26a16 16 0 0 1-6.44-6.46 20 20 0 0 1-2.3-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m38.66 0q4.9 0 8.8 2.13a15.5 15.5 0 0 1 6.22 6.32q2.31 4.2 2.38 10.26v2.06h-26.35q.28 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.77-4.93l9.15.67q-1.68 5-6.29 7.99t-10.62 3q-5.53 0-9.65-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26M86.9 36.69l17.24-34.33h10.8L89.96 49.63h-6.12L58.85 2.36h10.81zm71.62-15.55a11 11 0 0 1 2.47-4.48q2.28-2.31 6.38-2.31h3.4v7.26h-3.47q-2.91 0-4.79.8a5.8 5.8 0 0 0-2.77 2.5q-.9 1.72-.9 4.37v20.35h-8.89V14.35h8.33zm101.73 28.5h-8.95V2.35h8.95zM127.62 20.26q-3.7 0-6 2.2-2.32 2.2-2.87 6.2h17.05q-.48-4.34-2.72-6.33a7.8 7.8 0 0 0-5.46-2.07m101.2 0q-3.7 0-6 2.2-2.31 2.2-2.87 6.2H237q-.5-4.34-2.73-6.33a7.8 7.8 0 0 0-5.46-2.07" style="fill:black;fill-opacity:1;"/></svg>

After

Width:  |  Height:  |  Size: 1.6 KiB

+34
View File
@@ -0,0 +1,34 @@
package agent
import "github.com/Twigpine/zero/internal/tools"
// CheckpointTargets returns the workspace-relative paths a tool call will mutate,
// resolving its arguments exactly as executeToolCall does. Freeform apply_patch
// input is normalized through the registered built-in tool, so absolute headers
// are rewritten relative to the selected write root and cwd is set: checkpoint
// capture and execution therefore agree on the files at stake.
//
// Best-effort: nil when the call has no capture targets or its arguments cannot
// be resolved. Checkpoint capture never fails a run.
func CheckpointTargets(workspaceRoot string, registry *tools.Registry, call ToolCall) []string {
if registry == nil {
return nil
}
args := map[string]any{}
if call.Freeform {
tool, ok := registry.Get(call.Name)
if !ok || !tools.IsBuiltInApplyPatch(tool) {
return nil
}
prepared, err := tools.PrepareFreeformApplyPatchArguments(tool, call.Arguments)
if err != nil {
return nil
}
args = prepared
} else if call.Arguments != "" {
if err := decodeToolArguments(call.Arguments, &args); err != nil {
return nil
}
}
return tools.MutationTargets(workspaceRoot, call.Name, args)
}
+99
View File
@@ -0,0 +1,99 @@
package agent
import (
"encoding/json"
"os"
"path/filepath"
"slices"
"testing"
"github.com/Twigpine/zero/internal/tools"
)
func TestCheckpointTargets(t *testing.T) {
root := t.TempDir()
// Absolute headers must be compared against the physical root: t.TempDir
// differs from it on macOS (/var -> /private/var) and via Windows short names.
if resolved, err := filepath.EvalSymlinks(root); err == nil {
root = resolved
}
if err := os.WriteFile(filepath.Join(root, "main.txt"), []byte("before\n"), 0o644); err != nil {
t.Fatal(err)
}
relativePatch := "*** Begin Patch\n*** Update File: main.txt\n@@\n-before\n+after\n*** End Patch\n"
absolutePatch := "*** Begin Patch\n*** Update File: " + filepath.Join(root, "main.txt") + "\n@@\n-before\n+after\n*** End Patch\n"
multiFilePatch := "*** Begin Patch\n*** Update File: main.txt\n@@\n-before\n+after\n*** Add File: notes.txt\n+new\n*** End Patch\n"
jsonArgs, err := json.Marshal(map[string]any{"patch": relativePatch})
if err != nil {
t.Fatal(err)
}
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(root, nil))
unbuiltinRegistry := tools.NewRegistry()
unbuiltinRegistry.Register(&customApplyPatchTool{})
for _, tc := range []struct {
name string
registry *tools.Registry
call ToolCall
want []string
}{
{
name: "freeform relative header",
call: ToolCall{Name: "apply_patch", Arguments: relativePatch, Freeform: true},
want: []string{"main.txt"},
},
{
name: "freeform absolute header stays workspace-relative",
call: ToolCall{Name: "apply_patch", Arguments: absolutePatch, Freeform: true},
want: []string{"main.txt"},
},
{
name: "freeform multi-file patch",
call: ToolCall{Name: "apply_patch", Arguments: multiFilePatch, Freeform: true},
want: []string{"main.txt", "notes.txt"},
},
{
name: "json arguments",
call: ToolCall{Name: "apply_patch", Arguments: string(jsonArgs)},
want: []string{"main.txt"},
},
{
name: "freeform traversal is rejected",
call: ToolCall{Name: "apply_patch", Arguments: "*** Begin Patch\n*** Update File: ../escape.txt\n@@\n-x\n+y\n*** End Patch\n", Freeform: true},
want: nil,
},
{
name: "freeform custom apply_patch is not prepared",
registry: unbuiltinRegistry,
call: ToolCall{Name: "apply_patch", Arguments: relativePatch, Freeform: true},
want: nil,
},
{
name: "unknown freeform tool",
call: ToolCall{Name: "read_file", Arguments: "raw", Freeform: true},
want: nil,
},
{
name: "malformed json arguments",
call: ToolCall{Name: "apply_patch", Arguments: "{"},
want: nil,
},
} {
t.Run(tc.name, func(t *testing.T) {
reg := registry
if tc.registry != nil {
reg = tc.registry
}
got := CheckpointTargets(root, reg, tc.call)
if !slices.Equal(got, tc.want) {
t.Fatalf("CheckpointTargets() = %#v, want %#v", got, tc.want)
}
})
}
if got := CheckpointTargets(root, nil, ToolCall{Name: "apply_patch", Arguments: relativePatch}); got != nil {
t.Fatalf("CheckpointTargets(nil registry) = %#v, want nil", got)
}
}
+64
View File
@@ -80,6 +80,9 @@ func TestRunStopsAfterConsecutiveEmptyTurns(t *testing.T) {
if result.FinalAnswer == maxTurnsAnswer {
t.Fatalf("no-output guard must stop before reaching maxTurns, got max-turns answer")
}
if result.Incomplete {
t.Fatalf("interactive run must not be marked Incomplete; reason=%q", result.IncompleteReason)
}
}
func TestRunResetsEmptyTurnCounterOnVisibleOutput(t *testing.T) {
@@ -513,6 +516,22 @@ func repeatedFlakyTurns(n int) [][]zeroruntime.StreamEvent {
return turns
}
// alwaysSucceedingTool is the successful counterpart to alwaysFailingTool, used
// to put a productive tool turn before an empty-turn streak.
type alwaysSucceedingTool struct{}
func (alwaysSucceedingTool) Name() string { return "ok" }
func (alwaysSucceedingTool) Description() string { return "always succeeds for testing" }
func (alwaysSucceedingTool) Parameters() tools.Schema {
return tools.Schema{Type: "object", AdditionalProperties: false}
}
func (alwaysSucceedingTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow}
}
func (alwaysSucceedingTool) Run(context.Context, map[string]any) tools.Result {
return tools.Result{Status: tools.StatusOK, Output: "ok"}
}
func TestRunStopsAfterRepeatedToolFailures(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(alwaysFailingTool{})
@@ -529,6 +548,51 @@ func TestRunStopsAfterRepeatedToolFailures(t *testing.T) {
if len(provider.requests) != toolFailureStopAt {
t.Fatalf("expected stop at %d failures, made %d requests", toolFailureStopAt, len(provider.requests))
}
if result.Incomplete {
t.Fatalf("interactive run must not be marked Incomplete; reason=%q", result.IncompleteReason)
}
}
// Under the headless completion gate both guardrail stops are unfinished runs,
// so they must set Result.Incomplete and a reason; exec maps that to exit 4.
// Thresholds and stop answers stay pinned by the two tests above.
func TestHeadlessGuardStopsAreIncomplete(t *testing.T) {
for _, tc := range []struct {
name string
turns [][]zeroruntime.StreamEvent
wantTurns int
wantReason string
}{
{"empty_turns", [][]zeroruntime.StreamEvent{emptyTurn(), emptyTurn(), emptyTurn()}, maxEmptyTurns, "3 consecutive turns with no output"},
{"empty_turns_after_progress", [][]zeroruntime.StreamEvent{
toolTurn("ok-1", "ok", "{}"),
emptyTurn(), emptyTurn(), emptyTurn(),
}, maxEmptyTurns + 1, "3 consecutive turns with no output"},
{"repeated_tool_failures", repeatedFlakyTurns(10), toolFailureStopAt, "flaky"},
} {
t.Run(tc.name, func(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(alwaysFailingTool{})
registry.Register(alwaysSucceedingTool{})
result, err := Run(context.Background(), "go", &mockProvider{turns: tc.turns}, Options{
Registry: registry,
MaxTurns: 12,
RequireCompletionSignal: true,
})
if err != nil {
t.Fatal(err)
}
if result.Turns != tc.wantTurns {
t.Fatalf("guard did not trigger: turns=%d want=%d", result.Turns, tc.wantTurns)
}
if !result.Incomplete {
t.Fatalf("guardrail stop under the headless gate must be Incomplete; final=%q", result.FinalAnswer)
}
if !strings.Contains(result.IncompleteReason, tc.wantReason) {
t.Fatalf("IncompleteReason = %q, want it to mention %q", result.IncompleteReason, tc.wantReason)
}
})
}
}
func TestRunInjectsToolFailureHintWithSchema(t *testing.T) {
+15
View File
@@ -587,6 +587,14 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// counted toward the runaway cap so we stop before burning maxTurns.
if guards.observeTurn(collected) {
result.FinalAnswer = noOutputStopAnswer(result.Turns)
// Under the headless completion gate this stop is unfinished work,
// not a successful final answer: the guard exists precisely because
// the model made no progress. Interactive runs keep the gate off, so
// their behavior is unchanged.
if options.RequireCompletionSignal {
result.Incomplete = true
result.IncompleteReason = "stopped after " + strconv.Itoa(guards.emptyTurns) + " consecutive turns with no output"
}
result.Messages = copyMessages(messages)
return result, nil
}
@@ -808,6 +816,13 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
messages = append(messages, buildToolImageMessages()...)
result.FinalAnswer = toolFailureStopAnswer(call.Name, outcome.Count)
// Same headless contract as the empty-turn guard: halting on a
// repeated failure is an unfinished run, not success. Gate-off
// (interactive / --no-completion-gate) keeps the old behavior.
if options.RequireCompletionSignal {
result.Incomplete = true
result.IncompleteReason = "the " + call.Name + " tool failed " + strconv.Itoa(outcome.Count) + " times in a row with the same error"
}
result.Messages = copyMessages(messages)
return result, nil
}
+45
View File
@@ -1,14 +1,20 @@
package agent
import (
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/Twigpine/zero/internal/config"
"github.com/Twigpine/zero/internal/sandbox"
"github.com/Twigpine/zero/internal/testutil"
)
// isolatedUserDirsRoot is where TestMain points every per-user directory.
var isolatedUserDirsRoot string
// TestMain points userConfigDirForPrompt at an empty, package-wide temp
// directory for every test in this package. Without this, buildSystemPrompt
// falls back to the real config.UserConfigDir, so any developer with a
@@ -17,17 +23,56 @@ import (
// across contributor machines. Tests that specifically exercise user
// guidelines stub userConfigDirForPrompt themselves via
// withSystemPromptTestUserConfigDir(Func) and restore this default on cleanup.
//
// It also points every per-user directory root at the same temp directory:
// sandboxed tool runs keep their runtime state and lease files under the user
// cache dir, so without this they land in the developer's real cache.
func TestMain(m *testing.M) {
dir, err := os.MkdirTemp("", "zero-agent-tests-*")
if err != nil {
panic(err)
}
isolatedUserDirsRoot = dir
restore, err := testutil.IsolateUserDirs(dir)
if err != nil {
os.RemoveAll(dir)
panic(err)
}
userConfigDirForPrompt = func() (string, error) { return dir, nil }
code := m.Run()
if err := restore(); err != nil {
fmt.Fprintln(os.Stderr, "agent tests: restore user dirs:", err)
code = 1
}
os.RemoveAll(dir)
os.Exit(code)
}
// Guards TestMain: the per-user directories the package can read or write
// must resolve inside the isolated root on every OS.
func TestUserDirsAreIsolatedFromTheDeveloperHome(t *testing.T) {
if isolatedUserDirsRoot == "" {
t.Fatal("TestMain did not record an isolated root")
}
home, err := os.UserHomeDir()
if err != nil {
t.Fatal(err)
}
userConfig, err := config.UserConfigDir()
if err != nil {
t.Fatal(err)
}
cache, err := os.UserCacheDir()
if err != nil {
t.Fatal(err)
}
for name, dir := range map[string]string{"home": home, "config": userConfig, "cache": cache} {
if !testutil.PathWithin(isolatedUserDirsRoot, dir) {
t.Errorf("%s dir %q is outside the isolated root %q", name, dir, isolatedUserDirsRoot)
}
}
}
func TestCoreSystemPromptIncludesCodingQualityRules(t *testing.T) {
prompt := strings.Join(strings.Fields(strings.ToLower(buildSystemPrompt(Options{}))), " ")
if strings.Contains(prompt, "clickable file paths") {
+57
View File
@@ -0,0 +1,57 @@
package aimlapi
import (
"net/http"
"net/url"
"strings"
)
const (
// SourceHeaderName and SourceValue identify Zero as the calling agent on
// requests to aimlapi.com services (the Toolbox and the MCP server).
SourceHeaderName = "X-AIMLAPI-Source"
SourceValue = "agent/zero"
// CanonicalInferenceBaseURL is the catalog inference endpoint. A provider
// profile whose base URL differs is treated as a proxy: its key is never sent
// to other aimlapi.com services.
CanonicalInferenceBaseURL = "https://api.aimlapi.com/v1"
// ToolboxBaseURL is the fixed AI/ML API Toolbox REST base. It is not
// configurable, so the key and attribution can only reach tools.aimlapi.com.
ToolboxBaseURL = "https://tools.aimlapi.com/v1/tools"
// MCPServerURL is the AI/ML API MCP server (streamable HTTP).
MCPServerURL = "https://mcp.aimlapi.com/mcp"
)
// attributedHosts are the only hostnames that receive the AI/ML API key or the
// attribution headers. Matching is exact; a suffix or substring match would
// also accept hosts such as tools.aimlapi.com.example.net.
var attributedHosts = [...]string{"api.aimlapi.com", "tools.aimlapi.com", "mcp.aimlapi.com"}
// IsAttributedURL reports whether target is an https URL on one of the
// aimlapi.com service hosts, on the default port and without userinfo.
func IsAttributedURL(target *url.URL) bool {
if target == nil || !strings.EqualFold(target.Scheme, "https") || target.User != nil {
return false
}
if port := target.Port(); port != "" && port != "443" {
return false
}
host := strings.ToLower(target.Hostname())
for _, candidate := range attributedHosts {
if host == candidate {
return true
}
}
return false
}
// ApplyAttributionHeaders sets the source and partner headers on header only
// when target is an aimlapi.com service URL; any other target is left as is.
func ApplyAttributionHeaders(header http.Header, target *url.URL) {
if header == nil || !IsAttributedURL(target) {
return
}
header.Set(SourceHeaderName, SourceValue)
header.Set(PartnerHeaderName, ResolvePartnerID(""))
}
+54
View File
@@ -0,0 +1,54 @@
package aimlapi
import (
"net/http"
"net/url"
"testing"
)
func TestIsAttributedURLMatchesExactServiceHostsOnly(t *testing.T) {
cases := map[string]bool{
"https://tools.aimlapi.com/v1/tools/run": true,
"https://mcp.aimlapi.com/mcp": true,
"https://api.aimlapi.com/v1": true,
"https://TOOLS.aimlapi.com:443/v1/tools": true,
"http://tools.aimlapi.com/v1/tools/run": false,
"https://tools.aimlapi.com:8443/v1/tools": false,
"https://[email protected]/v1/tools": false,
"https://tools.aimlapi.com.example.net/run": false,
"https://eviltools.aimlapi.com/run": false,
"https://aimlapi.com/app": false,
"https://auth.aimlapi.com/": false,
"https://example.com/?h=tools.aimlapi.com": false,
}
for raw, want := range cases {
target, err := url.Parse(raw)
if err != nil {
t.Fatalf("parse %q: %v", raw, err)
}
if got := IsAttributedURL(target); got != want {
t.Errorf("IsAttributedURL(%q) = %v, want %v", raw, got, want)
}
}
if IsAttributedURL(nil) {
t.Fatal("IsAttributedURL(nil) = true")
}
}
func TestApplyAttributionHeadersSetsSourceAndPartnerOnServiceHosts(t *testing.T) {
t.Setenv("AIMLAPI_PARTNER_ID", "")
header := http.Header{}
ApplyAttributionHeaders(header, &url.URL{Scheme: "https", Host: "tools.aimlapi.com", Path: "/v1/tools/run"})
if got := header.Get(SourceHeaderName); got != "agent/zero" {
t.Fatalf("%s = %q, want agent/zero", SourceHeaderName, got)
}
if got := header.Get(PartnerHeaderName); got != DefaultPartnerID {
t.Fatalf("%s = %q, want %q", PartnerHeaderName, got, DefaultPartnerID)
}
other := http.Header{}
ApplyAttributionHeaders(other, &url.URL{Scheme: "https", Host: "tools.aimlapi.com.example.net"})
if len(other) != 0 {
t.Fatalf("attribution reached a non-aimlapi host: %#v", other)
}
}
+25 -2
View File
@@ -964,6 +964,7 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
// report can be combined with the plugin activation's, and emit at most one
// notice when project hooks/plugins were dropped for an untrusted workspace.
hookDispatcher, hookSkip := newHookDispatcherWithExtra(workspaceRoot, pluginActivation.hooks, trustRoot, executionRunner)
specialistRuntime.attachLifecycleHooks(hookDispatcher)
emitTrustNotice(stderr, hookSkip, pluginActivation.trustSkip, mcpSkip)
// From the engine the session's commands run through, so the notice
// describes the sandbox they actually get.
@@ -1221,6 +1222,9 @@ type agentToolRuntime struct {
// the orchestrator's system-prompt delegation section. Populated alongside the
// tools, so it is present exactly when the Task tool is.
specialists []agent.SpecialistInfo
// lifecycleHooks is shared with the specialist executor so the hooks
// dispatcher (built later in startup) can be attached before any specialist runs.
lifecycleHooks *specialist.LifecycleHooks
}
// specialistInfos returns the runtime's specialist summaries, nil-safe so the
@@ -1232,12 +1236,26 @@ func (r *agentToolRuntime) specialistInfos() []agent.SpecialistInfo {
return r.specialists
}
func (r *agentToolRuntime) attachLifecycleHooks(dispatcher *hooks.Dispatcher) {
if r == nil || r.lifecycleHooks == nil || dispatcher == nil {
return
}
r.lifecycleHooks.Dispatch = func(ctx context.Context, event string, specialistName string, payload map[string]any) {
dispatcher.Dispatch(ctx, hooks.DispatchInput{
Event: hooks.Event(event),
ToolName: specialistName,
Payload: payload,
})
}
}
func registerSpecialistTools(registry *tools.Registry, workspaceRoot string, maxTeamSize int) (*agentToolRuntime, error) {
paths, err := specialist.DefaultPaths(workspaceRoot)
if err != nil {
return nil, err
}
executor := specialist.Executor{Paths: paths}
lifecycleHooks := &specialist.LifecycleHooks{}
executor := specialist.Executor{Paths: paths, LifecycleHooks: lifecycleHooks}
runtime, err := specialist.RegisterTools(registry, executor)
if err != nil {
return nil, err
@@ -1256,7 +1274,12 @@ func registerSpecialistTools(registry *tools.Registry, workspaceRoot string, max
return nil, err
}
swarm.RegisterTools(registry, sw)
return &agentToolRuntime{specialist: runtime, swarm: sw, specialists: specialistSummaries(paths)}, nil
return &agentToolRuntime{
specialist: runtime,
swarm: sw,
specialists: specialistSummaries(paths),
lifecycleHooks: lifecycleHooks,
}, nil
}
// specialistSummaries loads the available specialists (built-ins + user/project
+13 -1
View File
@@ -92,13 +92,25 @@ func runSkillAdd(args []string, dir string, stdout io.Writer, stderr io.Writer)
lines := []string{}
if result.Updated {
lines = append(lines, fmt.Sprintf("Updated skill %q.", result.Name))
lines = append(lines, fmt.Sprintf(" hash: %s -> %s", shortHash(result.PreviousHash), shortHash(result.Hash)))
switch {
case result.PreviousHash != result.Hash:
lines = append(lines, fmt.Sprintf(" hash: %s -> %s", shortHash(result.PreviousHash), shortHash(result.Hash)))
case result.PreviousTreeHash == "":
// The install this replaced predates tree hashes and copied SKILL.md
// alone, so the bundled files are new rather than changed.
lines = append(lines, " SKILL.md is unchanged; the files bundled with it are now installed too")
default:
lines = append(lines, " SKILL.md is unchanged; the files bundled with it changed")
}
} else {
lines = append(lines, fmt.Sprintf("Installed skill %q.", result.Name))
lines = append(lines, " hash: "+result.Hash)
}
lines = append(lines, " source: "+result.Source)
lines = append(lines, " path: "+result.Path)
if len(result.SkippedLinks) > 0 {
lines = append(lines, " not copied (symlinks): "+strings.Join(result.SkippedLinks, ", "))
}
if _, err := fmt.Fprintln(stdout, redaction.RedactString(strings.Join(lines, "\n"), redaction.Options{})); err != nil {
return exitCrash
}
+132
View File
@@ -7,6 +7,8 @@ import (
"path/filepath"
"strings"
"testing"
"github.com/Twigpine/zero/internal/skills"
)
func writeSourceSkillDir(t *testing.T, dir string, content string) string {
@@ -391,3 +393,133 @@ func TestRunDistributionCommandsDoNotLaunchTUI(t *testing.T) {
}
}
}
// A reinstall that only changed a bundled script must still read as an update
// (#584): the SKILL.md hash is the same, so "hash: X -> X" would say nothing.
func TestRunSkillAddReportsAnUpdateToBundledFilesOnly(t *testing.T) {
skillsDir := t.TempDir()
src := writeSourceSkillDir(t, filepath.Join(t.TempDir(), "src"),
"---\nname: pdf\ndescription: Fill forms.\n---\nRun scripts/fill.py.\n")
script := filepath.Join(src, "scripts", "fill.py")
if err := os.MkdirAll(filepath.Dir(script), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(script, []byte("print('v1')\n"), 0o755); err != nil {
t.Fatal(err)
}
deps := appDeps{skillsDir: func() string { return skillsDir }}
var stdout, stderr bytes.Buffer
if exit := runWithDeps([]string{"skill", "add", src}, &stdout, &stderr, deps); exit != 0 {
t.Fatalf("first skill add exit = %d, stderr = %s", exit, stderr.String())
}
if err := os.WriteFile(script, []byte("print('v2')\n"), 0o755); err != nil {
t.Fatal(err)
}
stdout.Reset()
stderr.Reset()
if exit := runWithDeps([]string{"skill", "add", src}, &stdout, &stderr, deps); exit != 0 {
t.Fatalf("second skill add exit = %d, stderr = %s", exit, stderr.String())
}
for _, want := range []string{`Updated skill "pdf".`, "SKILL.md is unchanged; the files bundled with it changed"} {
if !strings.Contains(stdout.String(), want) {
t.Fatalf("second add output does not say %q:\n%s", want, stdout.String())
}
}
}
// An install from before tree hashes copied SKILL.md alone and recorded no tree
// hash. Adding the skill again over that entry must not claim bundled files
// changed: there are none, or they are installed for the first time.
func TestRunSkillAddOverAnEntryWithoutATreeHash(t *testing.T) {
for _, tc := range []struct {
name string
script bool
want []string
notWant []string
}{
{
name: "SKILL.md only",
want: []string{`Installed skill "pdf".`},
notWant: []string{"Updated skill", "files bundled with it"},
},
{
name: "with a bundled script",
script: true,
want: []string{`Updated skill "pdf".`, "SKILL.md is unchanged; the files bundled with it are now installed too"},
notWant: []string{"files bundled with it changed"},
},
} {
t.Run(tc.name, func(t *testing.T) {
skillsDir := t.TempDir()
src := writeSourceSkillDir(t, filepath.Join(t.TempDir(), "src"),
"---\nname: pdf\ndescription: Fill forms.\n---\nRun scripts/fill.py.\n")
if tc.script {
script := filepath.Join(src, "scripts", "fill.py")
if err := os.MkdirAll(filepath.Dir(script), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(script, []byte("print('fill')\n"), 0o755); err != nil {
t.Fatal(err)
}
}
deps := appDeps{skillsDir: func() string { return skillsDir }}
var stdout, stderr bytes.Buffer
if exit := runWithDeps([]string{"skill", "add", src}, &stdout, &stderr, deps); exit != 0 {
t.Fatalf("first skill add exit = %d, stderr = %s", exit, stderr.String())
}
lock, err := skills.ReadLock(skillsDir)
if err != nil {
t.Fatal(err)
}
entry := lock["pdf"]
entry.TreeHash = ""
lock["pdf"] = entry
data, err := json.Marshal(lock)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(skillsDir, skills.LockFileName), data, 0o644); err != nil {
t.Fatal(err)
}
stdout.Reset()
stderr.Reset()
if exit := runWithDeps([]string{"skill", "add", src}, &stdout, &stderr, deps); exit != 0 {
t.Fatalf("second skill add exit = %d, stderr = %s", exit, stderr.String())
}
for _, want := range tc.want {
if !strings.Contains(stdout.String(), want) {
t.Fatalf("second add output does not say %q:\n%s", want, stdout.String())
}
}
for _, notWant := range tc.notWant {
if strings.Contains(stdout.String(), notWant) {
t.Fatalf("second add output says %q:\n%s", notWant, stdout.String())
}
}
})
}
}
func TestRunSkillAddListsTheSymlinksItDidNotCopy(t *testing.T) {
skillsDir := t.TempDir()
src := writeSourceSkillDir(t, filepath.Join(t.TempDir(), "src"),
"---\nname: pdf\ndescription: Fill forms.\n---\nbody\n")
outside := filepath.Join(t.TempDir(), "notes.md")
if err := os.WriteFile(outside, []byte("outside\n"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(src, "notes.md")); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
var stdout, stderr bytes.Buffer
deps := appDeps{skillsDir: func() string { return skillsDir }}
if exit := runWithDeps([]string{"skill", "add", src}, &stdout, &stderr, deps); exit != 0 {
t.Fatalf("skill add exit = %d, stderr = %s", exit, stderr.String())
}
if !strings.Contains(stdout.String(), "not copied (symlinks): notes.md") {
t.Fatalf("add output does not list the skipped symlink:\n%s", stdout.String())
}
}
+2 -1
View File
@@ -655,6 +655,7 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
// can be combined with the plugin activation's, and emit at most one notice when
// project hooks/plugins were dropped for an untrusted workspace.
hookDispatcher, hookSkip := newHookDispatcherWithExtra(workspaceRoot, pluginActivation.hooks, trustRoot, executionRunner)
specialistRuntime.attachLifecycleHooks(hookDispatcher)
emitTrustNotice(stderr, hookSkip, pluginActivation.trustSkip, mcpSkip)
result, err := agent.Run(runCtx, agentPrompt, provider, agent.Options{
MaxTurns: resolved.MaxTurns,
@@ -712,7 +713,7 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
"arguments": call.Arguments,
})
// Snapshot before-state of files this call will mutate (safe rewind).
if checkpoint, ok := sessionRecorder.captureCheckpoint(workspaceRoot, call); ok {
if checkpoint, ok := sessionRecorder.captureCheckpoint(workspaceRoot, registry, call); ok {
writer.checkpoint(checkpoint)
}
},
@@ -0,0 +1,206 @@
package cli
import (
"bytes"
"context"
"encoding/json"
"os"
"path/filepath"
"testing"
"github.com/Twigpine/zero/internal/agent"
"github.com/Twigpine/zero/internal/config"
"github.com/Twigpine/zero/internal/sessions"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/zeroruntime"
)
func freeformApplyPatchArguments(root string, absolute bool) string {
target := "main.txt"
if absolute {
target = filepath.Join(root, "main.txt")
}
return "*** Begin Patch\n*** Update File: " + target + "\n@@\n-before\n+after\n*** End Patch\n"
}
// captureCheckpoint must resolve a freeform apply_patch call through the same
// preparation execution uses, so the before-state is captured and rewind can
// restore it. The JSON control pins the unchanged path.
func TestCaptureCheckpointFreeformApplyPatch(t *testing.T) {
for _, tc := range []struct {
name string
freeform bool
absolute bool
jsonArgs bool
}{
{name: "freeform", freeform: true},
{name: "freeform absolute header", freeform: true, absolute: true},
{name: "json arguments", jsonArgs: true},
} {
t.Run(tc.name, func(t *testing.T) {
t.Setenv("XDG_DATA_HOME", t.TempDir())
t.Setenv("ZERO_CHECKPOINTS", "on")
root := t.TempDir()
// The absolute-header case needs the physical root spelling the tool
// resolves (macOS /var vs /private/var, Windows short names).
if resolved, err := filepath.EvalSymlinks(root); err == nil {
root = resolved
}
if err := os.WriteFile(filepath.Join(root, "main.txt"), []byte("before\n"), 0o644); err != nil {
t.Fatal(err)
}
store := sessions.NewStore(sessions.StoreOptions{})
session, err := store.Create(sessions.CreateInput{Title: "freeform checkpoint"})
if err != nil {
t.Fatal(err)
}
if _, err := store.AppendEvent(session.SessionID, sessions.AppendEventInput{
Type: sessions.EventMessage,
Payload: map[string]string{"role": "user", "content": "apply the patch"},
}); err != nil {
t.Fatal(err)
}
recorder := &execSessionRecorder{prepared: sessions.PreparedExec{Store: store, Session: session}}
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(root, nil))
call := agent.ToolCall{ID: "call", Name: "apply_patch", Freeform: tc.freeform}
if tc.jsonArgs {
encoded, err := json.Marshal(map[string]string{"patch": freeformApplyPatchArguments(root, false)})
if err != nil {
t.Fatal(err)
}
call.Arguments = string(encoded)
} else {
call.Arguments = freeformApplyPatchArguments(root, tc.absolute)
}
event, ok := recorder.captureCheckpoint(root, registry, call)
if !ok || event.Type != sessions.EventSessionCheckpoint {
t.Fatalf("captureCheckpoint() = %#v, %t; want a checkpoint event", event, ok)
}
var payload sessions.CheckpointPayload
if err := json.Unmarshal(event.Payload, &payload); err != nil {
t.Fatalf("decode checkpoint payload: %v", err)
}
if len(payload.Files) != 1 || payload.Files[0].Path != "main.txt" || payload.Files[0].Blob == "" {
t.Fatalf("checkpoint files = %#v, want main.txt with a blob", payload.Files)
}
// Simulate the tool's mutation, then rewind past it.
if err := os.WriteFile(filepath.Join(root, "main.txt"), []byte("after\n"), 0o644); err != nil {
t.Fatal(err)
}
report, err := store.ApplyRewind(session.SessionID, root, 1)
if err != nil {
t.Fatalf("ApplyRewind: %v", err)
}
if report.FilesRestored != 1 {
t.Fatalf("restore report = %#v, want one restored file", report)
}
content, err := os.ReadFile(filepath.Join(root, "main.txt"))
if err != nil {
t.Fatal(err)
}
if string(content) != "before\n" {
t.Fatalf("restored content = %q, want before newline", content)
}
})
}
}
// freeformPatchExecProvider emits one native freeform apply_patch call, then a
// final text turn.
type freeformPatchExecProvider struct {
patch string
calls int
}
func (provider *freeformPatchExecProvider) StreamCompletion(ctx context.Context, request zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
ch := make(chan zeroruntime.StreamEvent, 5)
calls := provider.calls
provider.calls++
if calls == 0 {
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "call_patch", ToolName: "apply_patch", ToolCallFreeform: true}
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: "call_patch", ArgumentsFragment: provider.patch}
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "call_patch"}
} else {
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventText, Content: "patched"}
}
ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}
close(ch)
return ch, nil
}
// End-to-end: exec captures the freeform checkpoint and `sessions rewind`
// restores the pre-patch content.
func TestRunExecFreeformApplyPatchCheckpointsForRewind(t *testing.T) {
isolateCLIUserState(t)
t.Setenv("ZERO_CHECKPOINTS", "on")
root := t.TempDir()
if err := os.WriteFile(filepath.Join(root, "main.txt"), []byte("before\n"), 0o644); err != nil {
t.Fatal(err)
}
provider := &freeformPatchExecProvider{patch: freeformApplyPatchArguments(root, false)}
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{
"exec", "--init-session-id", "freeform_rewind", "--skip-permissions-unsafe",
"--output-format", "stream-json", "apply the patch",
}, &stdout, &stderr, appDeps{
getwd: func() (string, error) { return root, nil },
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return execResolvedConfig(), nil
},
newProvider: func(config.ProviderProfile) (zeroruntime.Provider, error) {
return provider, nil
},
})
if exitCode != exitSuccess {
t.Fatalf("exit = %d, want %d; stderr=%q stdout=%q", exitCode, exitSuccess, stderr.String(), stdout.String())
}
content, err := os.ReadFile(filepath.Join(root, "main.txt"))
if err != nil {
t.Fatal(err)
}
if string(content) != "after\n" {
t.Fatalf("patched content = %q, want after newline", content)
}
store := sessions.NewStore(sessions.StoreOptions{})
events, err := store.ReadEvents("freeform_rewind")
if err != nil {
t.Fatalf("ReadEvents: %v", err)
}
checkpoints := 0
for _, event := range events {
if event.Type == sessions.EventSessionCheckpoint {
checkpoints++
}
}
if checkpoints != 1 {
t.Fatalf("checkpoint events = %d, want 1; events=%v", checkpoints, sessionEventTypes(events))
}
report, err := store.ApplyRewind("freeform_rewind", root, 1)
if err != nil {
t.Fatalf("ApplyRewind: %v", err)
}
if report.FilesRestored != 1 {
t.Fatalf("restore report = %#v, want one restored file", report)
}
content, err = os.ReadFile(filepath.Join(root, "main.txt"))
if err != nil {
t.Fatal(err)
}
if string(content) != "before\n" {
t.Fatalf("rewound content = %q, want before newline", content)
}
}
func sessionEventTypes(events []sessions.Event) []string {
types := make([]string, 0, len(events))
for _, event := range events {
types = append(types, string(event.Type))
}
return types
}
@@ -0,0 +1,90 @@
package cli
import (
"bytes"
"context"
"slices"
"strings"
"testing"
"github.com/Twigpine/zero/internal/config"
"github.com/Twigpine/zero/internal/zeroruntime"
)
// guardStopExecProvider emits turns with no visible text and no tool calls, so
// the empty-turn guardrail stops the run after its consecutive-empty-turn cap.
type guardStopExecProvider struct{}
func (guardStopExecProvider) StreamCompletion(ctx context.Context, request zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
ch := make(chan zeroruntime.StreamEvent, 1)
select {
case <-ctx.Done():
close(ch)
return ch, ctx.Err()
case ch <- zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}:
}
close(ch)
return ch, nil
}
// A guardrail stop in headless exec must use the existing INCOMPLETE contract
// (exit 4, error event, run_end status), and --no-completion-gate must keep
// reporting success. isolateCLIUserState also turns off the models.dev refresh
// that runExec starts in the background (see the #1152/#1161 flake).
func TestRunExecGuardrailStopIncompleteContract(t *testing.T) {
for _, tc := range []struct {
name string
args []string
wantExit int
wantStatus string
}{
{"gate on", []string{"exec", "--output-format", "stream-json", "complete the task"}, exitIncomplete, "incomplete"},
{"gate off", []string{"exec", "--no-completion-gate", "--output-format", "stream-json", "complete the task"}, exitSuccess, "success"},
} {
t.Run(tc.name, func(t *testing.T) {
isolateCLIUserState(t)
cwd := t.TempDir()
var stdout, stderr bytes.Buffer
exitCode := runWithDeps(tc.args, &stdout, &stderr, appDeps{
getwd: func() (string, error) { return cwd, nil },
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return execResolvedConfig(), nil
},
newProvider: func(config.ProviderProfile) (zeroruntime.Provider, error) {
return guardStopExecProvider{}, nil
},
})
if exitCode != tc.wantExit {
t.Fatalf("exit code = %d, want %d; stderr=%q", exitCode, tc.wantExit, stderr.String())
}
events := decodeJSONLines(t, stdout.String())
if len(events) == 0 || events[len(events)-1]["type"] != "run_end" {
t.Fatalf("expected run_end as the last event, got %#v", events)
}
runEnd := events[len(events)-1]
if runEnd["status"] != tc.wantStatus {
t.Fatalf("run_end status = %v, want %q", runEnd["status"], tc.wantStatus)
}
if runEnd["exitCode"] != float64(tc.wantExit) {
t.Fatalf("run_end exitCode = %v, want %d", runEnd["exitCode"], tc.wantExit)
}
finalEvent := findJSONEvent(t, events, "final")
if text, _ := finalEvent["text"].(string); !strings.Contains(text, "no output") {
t.Fatalf("final text = %q, want the guard stop answer", text)
}
if tc.wantStatus == "incomplete" {
errorEvent := findJSONEvent(t, events, "error")
if errorEvent["code"] != "incomplete" {
t.Fatalf("error code = %v, want incomplete", errorEvent["code"])
}
if message, _ := errorEvent["message"].(string); !strings.Contains(message, "no output") {
t.Fatalf("error message = %q, want the guard reason", message)
}
} else if slices.Contains(jsonEventTypes(events), "error") {
t.Fatalf("gate-off run must not emit an error event: %#v", events)
}
})
}
}
+2 -7
View File
@@ -1,7 +1,6 @@
package cli
import (
"encoding/json"
"fmt"
"io"
"strings"
@@ -19,15 +18,11 @@ type execSessionRecorder struct {
// captureCheckpoint snapshots the before-state of files a tool call will mutate.
// Best-effort: failures never affect the run. Returns the checkpoint event and
// true when one was recorded.
func (recorder *execSessionRecorder) captureCheckpoint(workspaceRoot string, call agent.ToolCall) (sessions.Event, bool) {
func (recorder *execSessionRecorder) captureCheckpoint(workspaceRoot string, registry *tools.Registry, call agent.ToolCall) (sessions.Event, bool) {
if recorder.prepared.Store == nil || recorder.prepared.Session.SessionID == "" {
return sessions.Event{}, false
}
var args map[string]any
if call.Arguments != "" {
_ = json.Unmarshal([]byte(call.Arguments), &args)
}
targets := tools.MutationTargets(workspaceRoot, call.Name, args)
targets := agent.CheckpointTargets(workspaceRoot, registry, call)
if len(targets) == 0 {
return sessions.Event{}, false
}
+1 -1
View File
@@ -148,7 +148,7 @@ func runExecSpecDraft(run execSpecDraftRun) int {
"name": call.Name,
"arguments": call.Arguments,
})
if checkpoint, ok := sessionRecorder.captureCheckpoint(run.workspaceRoot, call); ok {
if checkpoint, ok := sessionRecorder.captureCheckpoint(run.workspaceRoot, run.registry, call); ok {
writer.checkpoint(checkpoint)
}
},
+116
View File
@@ -2,6 +2,7 @@ package cli
import (
"context"
"errors"
"fmt"
"io"
"strings"
@@ -82,11 +83,105 @@ func runPlugins(args []string, stdout io.Writer, stderr io.Writer, deps appDeps)
return runPluginInfo(args[1:], stdout, stderr, deps)
case "remove", "rm":
return runPluginRemove(args[1:], deps.pluginsDir(), stdout, stderr)
case "enable":
return runPluginsToggle(args[1:], stdout, stderr, deps, false)
case "disable":
return runPluginsToggle(args[1:], stdout, stderr, deps, true)
default:
return writeExecUsageError(stderr, fmt.Sprintf("unknown plugins subcommand %q", args[0]))
}
}
type pluginToggleOptions struct {
json bool
user bool
}
// runPluginsToggle implements `zero plugins enable|disable <id>`.
//
// Every message it writes, including usage errors that echo the caller's
// arguments, passes through the redaction pipeline: an argument is user input
// and can be a mistyped credential.
func runPluginsToggle(args []string, stdout io.Writer, stderr io.Writer, deps appDeps, disabled bool) int {
commandName := "enable"
if disabled {
commandName = "disable"
}
usageError := func(message string) int {
return writeExecUsageError(stderr, redaction.RedactString(message, redaction.Options{}))
}
options, positional, help, err := parsePluginToggleArgs(args, commandName)
if err != nil {
return usageError(err.Error())
}
if help {
if err := writePluginsToggleHelp(stdout, commandName); err != nil {
return exitCrash
}
return exitSuccess
}
if len(positional) != 1 {
return usageError(fmt.Sprintf("usage: zero plugins %s <id> [--user] [--json]", commandName))
}
pluginID := strings.TrimSpace(positional[0])
if pluginID == "" {
return usageError(fmt.Sprintf("usage: zero plugins %s <id> [--user] [--json]", commandName))
}
cwd, err := deps.getwd()
if err != nil {
return writeAppError(stderr, "failed to resolve workspace: "+redaction.ErrorMessage(err, redaction.Options{}), exitCrash)
}
loadOptions := plugins.LoadOptions{Cwd: cwd, ExcludeProject: options.user}
result, err := plugins.SetEnabledByID(loadOptions, pluginID, !disabled)
if err != nil {
if errors.Is(err, plugins.ErrNotInstalled) {
return usageError(err.Error())
}
return writeAppError(stderr, redaction.ErrorMessage(err, redaction.Options{}), exitCrash)
}
state := "enabled"
if disabled {
state = "disabled"
}
if options.json {
if err := writePrettyJSON(stdout, redaction.RedactValue(result, redaction.Options{})); err != nil {
return exitCrash
}
return exitSuccess
}
message := fmt.Sprintf("Plugin %s is now %s in %s.\n", result.ID, state, result.ManifestPath)
if !result.Changed {
message = fmt.Sprintf("Plugin %s was already %s in %s.\n", result.ID, state, result.ManifestPath)
}
if _, err := fmt.Fprint(stdout, redaction.RedactString(message, redaction.Options{})); err != nil {
return exitCrash
}
return exitSuccess
}
func parsePluginToggleArgs(args []string, command string) (pluginToggleOptions, []string, bool, error) {
options := pluginToggleOptions{}
positional := []string{}
for _, arg := range args {
switch arg {
case "-h", "--help", "help":
return options, positional, true, nil
case "--json":
options.json = true
case "--user":
options.user = true
default:
if strings.HasPrefix(arg, "-") {
return options, positional, false, execUsageError{fmt.Sprintf("unknown plugins %s flag %q", command, arg)}
}
positional = append(positional, arg)
}
}
return options, positional, false, nil
}
func runHooks(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) int {
if len(args) == 0 {
return writeExecUsageError(stderr, "hooks subcommand required. Use `zero hooks list`.")
@@ -566,10 +661,31 @@ Commands:
info <id> Show plugin details and lockfile metadata
add <git-url|path> Install a plugin (manifest-validated, pinned in plugins.lock)
remove <id> Remove an installed plugin and its lockfile entry
enable <id> Enable a plugin via its plugin.json
disable <id> Disable a plugin via its plugin.json
`)
return err
}
func writePluginsToggleHelp(w io.Writer, command string) error {
_, err := fmt.Fprintf(w, `Usage:
zero plugins %s <id> [flags]
Flags:
--user Target only user plugins (ignore ./.zero/plugins)
--json Print command result as JSON
-h, --help Show this help
Without --user, a project plugin with the same id takes precedence over a user
plugin, matching the loader's discovery order.
Zero reads plugins when it starts, so the change applies to the next zero exec
run or interactive session. A running interactive session keeps the plugins it
started with until you restart it.
`, command)
return err
}
func writePluginsListHelp(w io.Writer) error {
_, err := fmt.Fprint(w, `Usage:
zero plugins list [flags]
+232
View File
@@ -5,6 +5,7 @@ import (
"context"
"encoding/json"
"errors"
"os"
"path/filepath"
"strings"
"testing"
@@ -333,3 +334,234 @@ func TestRunMCPPermissionsHelpDoesNotOpenStore(t *testing.T) {
})
}
}
func TestRunPluginsEnableDisable(t *testing.T) {
cwd := t.TempDir()
home := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", home)
t.Setenv("HOME", home)
pluginDir := filepath.Join(cwd, ".zero", "plugins", "demo")
if err := os.MkdirAll(pluginDir, 0o700); err != nil {
t.Fatal(err)
}
manifestPath := filepath.Join(pluginDir, "plugin.json")
if err := os.WriteFile(manifestPath, []byte(`{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0"
}
`), 0o600); err != nil {
t.Fatal(err)
}
deps := appDeps{getwd: func() (string, error) { return cwd, nil }}
var stdout bytes.Buffer
var stderr bytes.Buffer
exitCode := runWithDeps([]string{"plugins", "disable", "zero.demo", "--json"}, &stdout, &stderr, deps)
if exitCode != exitSuccess {
t.Fatalf("disable exit=%d stderr=%s", exitCode, stderr.String())
}
var disablePayload plugins.SetEnabledResult
if err := json.Unmarshal(stdout.Bytes(), &disablePayload); err != nil {
t.Fatalf("decode disable JSON: %v\n%s", err, stdout.String())
}
if disablePayload.ID != "zero.demo" || disablePayload.Enabled || !disablePayload.Changed {
t.Fatalf("unexpected disable payload: %#v", disablePayload)
}
raw, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(raw), `"enabled": false`) {
t.Fatalf("manifest not disabled: %s", raw)
}
stdout.Reset()
stderr.Reset()
exitCode = runWithDeps([]string{"plugins", "disable", "zero.demo"}, &stdout, &stderr, deps)
if exitCode != exitSuccess {
t.Fatalf("noop disable exit=%d stderr=%s", exitCode, stderr.String())
}
if !strings.Contains(stdout.String(), "was already disabled") {
t.Fatalf("noop disable output: %s", stdout.String())
}
stdout.Reset()
stderr.Reset()
exitCode = runWithDeps([]string{"plugins", "enable", "zero.demo"}, &stdout, &stderr, deps)
if exitCode != exitSuccess {
t.Fatalf("enable exit=%d stderr=%s", exitCode, stderr.String())
}
if !strings.Contains(stdout.String(), "is now enabled") {
t.Fatalf("enable output: %s", stdout.String())
}
stdout.Reset()
stderr.Reset()
exitCode = runWithDeps([]string{"plugins", "enable", "missing-plugin"}, &stdout, &stderr, deps)
if exitCode != exitUsage {
t.Fatalf("missing plugin exit=%d want %d stderr=%s", exitCode, exitUsage, stderr.String())
}
}
// The text-mode toggle output runs through the redaction pipeline, so a
// manifest path that contains something secret-shaped is masked.
func TestRunPluginsToggleTextOutputIsRedacted(t *testing.T) {
cwd := t.TempDir()
home := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", home)
t.Setenv("HOME", home)
secretDir := "sk-proj-abcdefghijklmnopqrstuvwxyz"
pluginDir := filepath.Join(cwd, ".zero", "plugins", secretDir)
if err := os.MkdirAll(pluginDir, 0o700); err != nil {
t.Fatal(err)
}
manifestPath := filepath.Join(pluginDir, "plugin.json")
if err := os.WriteFile(manifestPath, []byte(`{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0"
}
`), 0o600); err != nil {
t.Fatal(err)
}
deps := appDeps{getwd: func() (string, error) { return cwd, nil }}
var stdout bytes.Buffer
var stderr bytes.Buffer
exitCode := runWithDeps([]string{"plugins", "disable", "zero.demo"}, &stdout, &stderr, deps)
if exitCode != exitSuccess {
t.Fatalf("disable exit=%d stderr=%s", exitCode, stderr.String())
}
if strings.Contains(stdout.String(), secretDir) {
t.Fatalf("toggle text leaked a secret-shaped manifest path: %q", stdout.String())
}
if !strings.Contains(stdout.String(), "[REDACTED]") {
t.Fatalf("toggle text was not redacted: %q", stdout.String())
}
stdout.Reset()
stderr.Reset()
exitCode = runWithDeps([]string{"plugins", "disable", "zero.demo"}, &stdout, &stderr, deps)
if exitCode != exitSuccess {
t.Fatalf("noop disable exit=%d stderr=%s", exitCode, stderr.String())
}
if !strings.Contains(stdout.String(), "was already disabled") {
t.Fatalf("noop toggle output: %q", stdout.String())
}
if strings.Contains(stdout.String(), secretDir) {
t.Fatalf("noop toggle text leaked a secret-shaped manifest path: %q", stdout.String())
}
}
func TestRunPluginsToggleRejectsEmptyID(t *testing.T) {
deps := appDeps{getwd: func() (string, error) { return t.TempDir(), nil }}
for _, subcommand := range []string{"enable", "disable"} {
t.Run(subcommand, func(t *testing.T) {
var stdout bytes.Buffer
var stderr bytes.Buffer
exitCode := runWithDeps([]string{"plugins", subcommand, ""}, &stdout, &stderr, deps)
if exitCode != exitUsage {
t.Fatalf("exit=%d want %d stderr=%s", exitCode, exitUsage, stderr.String())
}
if !strings.Contains(stderr.String(), "usage: zero plugins "+subcommand) {
t.Fatalf("stderr = %q", stderr.String())
}
})
}
}
func TestRunPluginsDisableUserIgnoresProject(t *testing.T) {
home := t.TempDir()
cwd := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", home)
t.Setenv("HOME", home)
userManifest := filepath.Join(home, "zero", "plugins", "demo", "plugin.json")
projectManifest := filepath.Join(cwd, ".zero", "plugins", "demo", "plugin.json")
for _, path := range []string{userManifest, projectManifest} {
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(`{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
"enabled": true
}
`), 0o600); err != nil {
t.Fatal(err)
}
}
deps := appDeps{getwd: func() (string, error) { return cwd, nil }}
var stdout bytes.Buffer
var stderr bytes.Buffer
exitCode := runWithDeps([]string{"plugins", "disable", "zero.demo", "--user"}, &stdout, &stderr, deps)
if exitCode != exitSuccess {
t.Fatalf("disable --user exit=%d stderr=%s", exitCode, stderr.String())
}
if !strings.Contains(stdout.String(), "is now disabled") {
t.Fatalf("disable --user output: %s", stdout.String())
}
userRaw, err := os.ReadFile(userManifest)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(userRaw), `"enabled": false`) {
t.Fatalf("user manifest not disabled: %s", userRaw)
}
projectRaw, err := os.ReadFile(projectManifest)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(projectRaw), `"enabled": false`) {
t.Fatalf("project manifest should stay enabled: %s", projectRaw)
}
}
// The toggle's usage errors echo the caller's arguments, which can be a
// mistyped credential, so they pass through redaction like its other output.
func TestRunPluginsToggleUsageErrorsAreRedacted(t *testing.T) {
home := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", home)
t.Setenv("HOME", home)
deps := appDeps{getwd: func() (string, error) { return t.TempDir(), nil }}
const secret = "sk-proj-abcdefghijklmnopqrstuvwxyz"
cases := []struct {
name string
args []string
want string
}{
{"unknown flag", []string{"zero.demo", "--token=" + secret}, "unknown plugins"},
{"missing plugin", []string{secret}, "plugin is not installed"},
}
for _, subcommand := range []string{"enable", "disable"} {
for _, tc := range cases {
t.Run(subcommand+" "+tc.name, func(t *testing.T) {
var stdout bytes.Buffer
var stderr bytes.Buffer
args := append([]string{"plugins", subcommand}, tc.args...)
exitCode := runWithDeps(args, &stdout, &stderr, deps)
if exitCode != exitUsage {
t.Fatalf("exit=%d want %d stderr=%s", exitCode, exitUsage, stderr.String())
}
if strings.Contains(stderr.String(), secret) {
t.Fatalf("usage error leaked the argument: %q", stderr.String())
}
if !strings.Contains(stderr.String(), tc.want) || !strings.Contains(stderr.String(), "[REDACTED]") {
t.Fatalf("stderr = %q, want a redacted %q error", stderr.String(), tc.want)
}
})
}
}
}
+1 -1
View File
@@ -319,7 +319,7 @@ Flags:
--command <cmd> Command to run (required)
--name <name> Human-readable hook name
--description <text> Hook description
--matcher <pattern> Tool matcher (beforeTool/afterTool only)
--matcher <pattern> Tool/specialist matcher (beforeTool/afterTool/specialistStart/specialistStop)
--arg <value> Command argument (repeatable)
--user Write to user config instead of the project
--json Print command result as JSON
+32 -3
View File
@@ -741,6 +741,13 @@ func (cfg *mcpWritableConfig) upsertServer(name string, server config.MCPServerC
if err != nil {
return false, err
}
if !mergedServer.Disabled && config.IsDefaultMCPServerDisabled(name) {
// Adding a server under the name of a default that ships disabled means
// the user wants it on; the omitted flag would otherwise keep it off.
if data, err = setMCPServerRawDisabledFalse(data); err != nil {
return false, err
}
}
cfg.serverRaw[name] = data
if _, err := cfg.removeLegacyServer(name); err != nil {
return false, err
@@ -800,11 +807,29 @@ func mergeMCPServerRaw(existing json.RawMessage, server config.MCPServerConfig)
if key == "oauth" && preserveRawOAuth {
continue
}
// `mcp add` replaces the entry: it writes only a disable, never a stale
// explicit false carried over from the old entry (upsertServer adds the
// false a default that ships disabled needs).
if key == "disabled" && string(value) == "false" {
continue
}
merged[key] = value
}
return json.Marshal(merged)
}
func setMCPServerRawDisabledFalse(data json.RawMessage) (json.RawMessage, error) {
var server map[string]json.RawMessage
if err := json.Unmarshal(data, &server); err != nil {
return nil, err
}
if server == nil {
server = map[string]json.RawMessage{}
}
server["disabled"] = json.RawMessage("false")
return json.Marshal(server)
}
func (cfg *mcpWritableConfig) removeServer(name string) (bool, error) {
cfg.ensureRaw()
removed := false
@@ -856,15 +881,19 @@ func (cfg *mcpWritableConfig) setServerDisabled(name string, disabled bool) (boo
server = map[string]json.RawMessage{}
}
current := false
// A default that ships disabled (aimlapi) stays off until an entry says
// otherwise, so an absent flag reads as disabled and enabling it must be
// written out explicitly: dropping the flag would leave it off.
shippedDisabled := config.IsDefaultMCPServerDisabled(name)
current := shippedDisabled
if rawDisabled, ok := server["disabled"]; ok && len(rawDisabled) > 0 && string(rawDisabled) != "null" {
if err := json.Unmarshal(rawDisabled, &current); err != nil {
return false, false, err
}
}
changed := current != disabled
if disabled {
data, err := json.Marshal(true)
if disabled || shippedDisabled {
data, err := json.Marshal(disabled)
if err != nil {
return false, false, err
}
+260
View File
@@ -2,10 +2,18 @@ package cli
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"github.com/Twigpine/zero/internal/aimlapi"
"github.com/Twigpine/zero/internal/config"
)
@@ -44,3 +52,255 @@ func TestRunMCPDisableSeededExaDefault(t *testing.T) {
t.Fatal("expected `mcp disable exa` to turn the seeded default off in the resolved config")
}
}
// aimlapiWire stands in for https://mcp.aimlapi.com/mcp at the transport layer:
// requests keep their real URL, nothing is dialed, and each one is recorded.
type aimlapiWire struct {
mu sync.Mutex
requests []*http.Request
}
func (wire *aimlapiWire) RoundTrip(request *http.Request) (*http.Response, error) {
var body []byte
if request.Body != nil {
body, _ = io.ReadAll(request.Body)
_ = request.Body.Close()
}
wire.mu.Lock()
wire.requests = append(wire.requests, request.Clone(context.Background()))
wire.mu.Unlock()
var message struct {
ID json.RawMessage `json:"id"`
Method string `json:"method"`
}
_ = json.Unmarshal(body, &message)
respond := func(status int, payload string) *http.Response {
return &http.Response{StatusCode: status, Header: http.Header{"Content-Type": {"application/json"}}, Body: io.NopCloser(strings.NewReader(payload)), Request: request}
}
if request.URL.String() != aimlapi.MCPServerURL {
return respond(http.StatusBadGateway, `{}`), nil
}
if len(message.ID) == 0 {
return respond(http.StatusAccepted, ""), nil
}
result := `{"protocolVersion":"2024-11-05","capabilities":{},"serverInfo":{"name":"fake","version":"1"}}`
if message.Method == "tools/list" {
result = `{"tools":[{"name":"tools_search","description":"Search the Toolbox","inputSchema":{"type":"object"}}]}`
}
return respond(http.StatusOK, fmt.Sprintf(`{"jsonrpc":"2.0","id":%s,"result":%s}`, message.ID, result)), nil
}
func (wire *aimlapiWire) recorded() []*http.Request {
wire.mu.Lock()
defer wire.mu.Unlock()
return append([]*http.Request(nil), wire.requests...)
}
// isolateAIMLAPIMCPCommand redirects every root the real resolve path reads to
// a temporary home, sets an AI/ML API key in the environment, and routes the
// default HTTP transport to a recorder. It returns the user config path.
func isolateAIMLAPIMCPCommand(t *testing.T, key string) (string, *aimlapiWire, appDeps) {
t.Helper()
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("XDG_STATE_HOME", filepath.Join(home, ".state"))
t.Setenv("XDG_CACHE_HOME", filepath.Join(home, ".cache"))
t.Setenv("APPDATA", filepath.Join(home, "AppData", "Roaming"))
t.Setenv("LOCALAPPDATA", filepath.Join(home, "AppData", "Local"))
t.Setenv("ZERO_CRED_STORAGE", "file")
t.Setenv("ZERO_PROVIDER", "")
t.Setenv("AIMLAPI_INFERENCE_URL", "")
t.Setenv("AIMLAPI_PARTNER_ID", "")
t.Setenv("AIMLAPI_API_KEY", key)
configPath, err := config.DefaultUserConfigPath()
if err != nil {
t.Fatal(err)
}
writeMCPCommandRawConfig(t, configPath, `{}`)
wire := &aimlapiWire{}
previous := http.DefaultTransport
http.DefaultTransport = wire
t.Cleanup(func() { http.DefaultTransport = previous })
workspace := t.TempDir()
return configPath, wire, appDeps{getwd: func() (string, error) { return workspace, nil }}
}
func TestRunMCPEnableAIMLAPIDefaultWritesOnlyTheToggle(t *testing.T) {
configPath, wire, deps := isolateAIMLAPIMCPCommand(t, "aiml-secret-key")
var out, errBuf bytes.Buffer
if code := runWithDeps([]string{"mcp", "disable", "aimlapi", "--json"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("disable exit=%d stderr=%s", code, errBuf.String())
}
if !strings.Contains(out.String(), `"changed": false`) {
t.Fatalf("disabling the off-by-default server should be a no-op: %s", out.String())
}
out.Reset()
errBuf.Reset()
if code := runWithDeps([]string{"mcp", "enable", "aimlapi", "--json"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("enable exit=%d stderr=%s", code, errBuf.String())
}
if !strings.Contains(out.String(), `"changed": true`) {
t.Fatalf("enable should report a change: %s", out.String())
}
data, err := os.ReadFile(configPath)
if err != nil {
t.Fatal(err)
}
var written struct {
MCP struct {
Servers map[string]map[string]any `json:"servers"`
} `json:"mcp"`
}
if err := json.Unmarshal(data, &written); err != nil {
t.Fatal(err)
}
if entry := written.MCP.Servers["aimlapi"]; len(entry) != 1 || entry["disabled"] != false {
t.Fatalf("enable must write only the toggle, got %#v in %s", entry, data)
}
if strings.Contains(string(data), "aiml-secret-key") || strings.Contains(string(data), "Authorization") {
t.Fatalf("config holds credential material: %s", data)
}
cfg, err := config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
if err != nil {
t.Fatal(err)
}
if server := cfg.Servers["aimlapi"]; server.Disabled || config.IsUnconfiguredDefault("aimlapi", server) {
t.Fatalf("enabled aimlapi = %#v, want enabled and user-configured", server)
}
// An unrelated save rewrites the whole file through FileConfig; the
// explicit enable has to survive it or the shipped disable comes back.
if _, err := config.SetTheme(configPath, "dark"); err != nil {
t.Fatal(err)
}
cfg, err = config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
if err != nil {
t.Fatal(err)
}
if cfg.Servers["aimlapi"].Disabled {
data, _ := os.ReadFile(configPath)
t.Fatalf("an unrelated config save turned aimlapi off again: %s", data)
}
if len(wire.recorded()) != 0 {
t.Fatal("enable must not connect")
}
out.Reset()
if code := runWithDeps([]string{"mcp", "disable", "aimlapi"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("disable exit=%d stderr=%s", code, errBuf.String())
}
cfg, err = config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
if err != nil {
t.Fatal(err)
}
if !cfg.Servers["aimlapi"].Disabled {
t.Fatal("disable after enable must turn aimlapi off again")
}
}
func TestRunMCPAddUnderAIMLAPIDefaultNameEnablesIt(t *testing.T) {
configPath, _, deps := isolateAIMLAPIMCPCommand(t, "")
var out, errBuf bytes.Buffer
if code := runWithDeps([]string{"mcp", "add", "aimlapi", "--type", "http", "--url", aimlapi.MCPServerURL, "--auth", "oauth"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("add exit=%d stderr=%s", code, errBuf.String())
}
cfg, err := config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
if err != nil {
t.Fatal(err)
}
if server := cfg.Servers["aimlapi"]; server.Disabled || server.Auth != "oauth" {
t.Fatalf("added aimlapi = %#v, want enabled with oauth", server)
}
if code := runWithDeps([]string{"mcp", "add", "aimlapi", "--type", "http", "--url", aimlapi.MCPServerURL, "--disabled"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("add --disabled exit=%d stderr=%s", code, errBuf.String())
}
if cfg, err = config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath}); err != nil || !cfg.Servers["aimlapi"].Disabled {
t.Fatalf("add --disabled left aimlapi enabled: %#v, %v", cfg.Servers["aimlapi"], err)
}
}
// Through the real resolve path (no injected resolveMCPConfig): the default is
// listed as disabled, never connects, and once enabled it reaches session
// startup and `zero mcp check`, which connects with the key on the wire.
func TestAIMLAPIMCPDefaultThroughRealResolvePath(t *testing.T) {
_, wire, deps := isolateAIMLAPIMCPCommand(t, "aiml-secret-key")
var out, errBuf bytes.Buffer
if code := runWithDeps([]string{"mcp", "list"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("list exit=%d stderr=%s", code, errBuf.String())
}
if !strings.Contains(out.String(), "aimlapi [http] disabled https://mcp.aimlapi.com/mcp") || strings.Contains(out.String(), "aiml-secret-key") {
t.Fatalf("list output should show the disabled aimlapi default and no key:\n%s", out.String())
}
out.Reset()
if code := runWithDeps([]string{"mcp", "list", "--json"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("list --json exit=%d stderr=%s", code, errBuf.String())
}
if !strings.Contains(out.String(), `"aimlapi"`) || strings.Contains(out.String(), "aiml-secret-key") {
t.Fatalf("list --json = %s", out.String())
}
out.Reset()
errBuf.Reset()
if code := runWithDeps([]string{"mcp", "check", "aimlapi"}, &out, &errBuf, deps); code == exitSuccess || !strings.Contains(errBuf.String(), "is disabled") {
t.Fatalf("check of the disabled default: exit=%d stderr=%s", code, errBuf.String())
}
resolved, err := fillAppDeps(deps).resolveMCPConfig(t.TempDir(), true)
if err != nil {
t.Fatal(err)
}
critical, optional := splitMCPStartupConfig(resolved)
if _, ok := critical.Servers["aimlapi"]; ok {
t.Fatal("the disabled default must not start with a session")
}
if _, ok := optional.Servers["aimlapi"]; ok {
t.Fatal("the disabled default must not start with a session")
}
if len(wire.recorded()) != 0 {
t.Fatalf("the disabled default connected: %d requests", len(wire.recorded()))
}
if code := runWithDeps([]string{"mcp", "enable", "aimlapi"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("enable exit=%d stderr=%s", code, errBuf.String())
}
resolved, err = fillAppDeps(deps).resolveMCPConfig(t.TempDir(), true)
if err != nil {
t.Fatal(err)
}
critical, _ = splitMCPStartupConfig(resolved)
if _, ok := critical.Servers["aimlapi"]; !ok {
t.Fatal("an enabled aimlapi must start with the session as a user-configured server")
}
out.Reset()
errBuf.Reset()
if code := runWithDeps([]string{"mcp", "check", "aimlapi", "--json"}, &out, &errBuf, deps); code != exitSuccess {
t.Fatalf("check exit=%d stdout=%s stderr=%s", code, out.String(), errBuf.String())
}
if !strings.Contains(out.String(), `"status": "ok"`) || !strings.Contains(out.String(), `"toolCount": 1`) {
t.Fatalf("check output = %s", out.String())
}
if strings.Contains(out.String()+errBuf.String(), "aiml-secret-key") {
t.Fatalf("check output leaks the key: %s %s", out.String(), errBuf.String())
}
requests := wire.recorded()
if len(requests) == 0 {
t.Fatal("check did not connect")
}
for _, request := range requests {
if request.URL.String() != aimlapi.MCPServerURL {
t.Fatalf("request went to %s", request.URL)
}
if got := request.Header.Get("Authorization"); got != "Bearer aiml-secret-key" {
t.Fatalf("Authorization = %q, want the AI/ML API key", got)
}
if request.Header.Get(aimlapi.SourceHeaderName) != aimlapi.SourceValue || request.Header.Get(aimlapi.PartnerHeaderName) == "" {
t.Fatalf("attribution headers missing: %v", request.Header)
}
}
}
+3 -12
View File
@@ -174,7 +174,7 @@ func runSandboxSetup(args []string, stdout io.Writer, stderr io.Writer, deps app
if !setupHelper.Available() {
return writeAppError(stderr, "Windows sandbox setup helper is not available", exitProvider)
}
setupPlan, err := zeroSandbox.BuildWindowsSandboxSetupArgs(zeroSandbox.WindowsSandboxSetupArgsOptions{
setupArgs, err := zeroSandbox.BuildWindowsSandboxSetupArgs(zeroSandbox.WindowsSandboxSetupArgsOptions{
CommandCWD: workspaceRoot,
WorkspaceRoots: []string{workspaceRoot},
PermissionProfile: profile,
@@ -182,18 +182,9 @@ func runSandboxSetup(args []string, stdout io.Writer, stderr io.Writer, deps app
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
setupArgs := append(append([]string{}, setupHelper.ArgsPrefix...), setupPlan.Args...)
// BUILDING THE ARGS ALREADY WROTE TO DISK. Selecting the runtime root takes a
// lease, and taking a lease creates the runtime tree and the lease file when
// they are not there. A helper that never publishes a marker leaves that tree
// behind, attested by nothing, so the undo belongs on this side of the process
// boundary where the record of it lives.
setupArgs = append(append([]string{}, setupHelper.ArgsPrefix...), setupArgs...)
if err := deps.runSandboxSetupHelper(setupHelper.Name, setupArgs, stdout, stderr); err != nil {
message := "Windows sandbox setup failed: " + err.Error()
if undo := setupPlan.Rollback(); undo != nil {
message += "\nleftover sandbox runtime state could not be removed: " + undo.Error()
}
return writeAppError(stderr, message, exitProvider)
return writeAppError(stderr, "Windows sandbox setup failed: "+err.Error(), exitProvider)
}
return writeSandboxSetupResult(stdout, options.json, sandboxSetupResult{
Platform: "windows",
@@ -0,0 +1,145 @@
package cli
import (
"bytes"
"context"
"io"
"testing"
"github.com/Twigpine/zero/internal/config"
"github.com/Twigpine/zero/internal/hooks"
"github.com/Twigpine/zero/internal/mcp"
"github.com/Twigpine/zero/internal/specialist"
"github.com/Twigpine/zero/internal/tools"
"github.com/Twigpine/zero/internal/tui"
"github.com/Twigpine/zero/internal/zeroruntime"
)
func TestRegisterSpecialistToolsWiresLifecycleHooksIntoTaskOutput(t *testing.T) {
registry := tools.NewRegistry()
runtime, err := registerSpecialistTools(registry, t.TempDir(), 0)
if err != nil {
t.Fatalf("registerSpecialistTools returned error: %v", err)
}
t.Cleanup(func() { closeSpecialistRuntime(io.Discard, runtime) })
if runtime.lifecycleHooks == nil {
t.Fatal("lifecycleHooks is nil after registerSpecialistTools")
}
raw, ok := registry.Get("TaskOutput")
if !ok {
t.Fatal("TaskOutput tool not registered")
}
outputTool, ok := raw.(*specialist.OutputTool)
if !ok {
t.Fatalf("TaskOutput type = %T, want *specialist.OutputTool", raw)
}
if outputTool.LifecycleHooks != runtime.lifecycleHooks {
t.Fatal("TaskOutput LifecycleHooks is not the shared registerSpecialistTools bridge")
}
dispatcher := hooks.NewDispatcher(hooks.DispatcherOptions{})
runtime.attachLifecycleHooks(dispatcher)
if runtime.lifecycleHooks.Dispatch == nil {
t.Fatal("attachLifecycleHooks left Dispatch nil")
}
if outputTool.LifecycleHooks.Dispatch == nil {
t.Fatal("TaskOutput did not observe Dispatch after attachLifecycleHooks")
}
}
// TestRealEntryPointsAttachLifecycleHooks closes the gap the registration-level
// test above cannot see: wiring the bridge when tools register is not enough
// unless the real startup paths actually call attachLifecycleHooks before any
// run. Both surfaces that can launch specialists must do so — the interactive
// TUI and headless exec. attachLifecycleHooks mutates the same
// *specialist.LifecycleHooks the TaskOutput tool holds, so a nil Dispatch on
// that tool after a full startup proves the call never ran and the
// specialistStart/specialistStop hooks would silently never fire.
func TestRealEntryPointsAttachLifecycleHooks(t *testing.T) {
requireTaskOutputDispatchAttached := func(t *testing.T, registry *tools.Registry) {
t.Helper()
if registry == nil {
t.Fatal("startup did not reach the run registry")
}
raw, ok := registry.Get("TaskOutput")
if !ok {
t.Fatal("TaskOutput tool not registered by startup")
}
outputTool, ok := raw.(*specialist.OutputTool)
if !ok {
t.Fatalf("TaskOutput type = %T, want *specialist.OutputTool", raw)
}
if outputTool.LifecycleHooks == nil || outputTool.LifecycleHooks.Dispatch == nil {
t.Fatal("startup left the specialist lifecycle bridge unattached: specialistStart/specialistStop hooks would never fire")
}
}
t.Run("interactive TUI", func(t *testing.T) {
isolateCLIUserState(t)
cwd := t.TempDir()
var registry *tools.Registry
// Empty args route runWithDeps to the interactive TUI path; the injected
// runTUI receives the same registry specialist tools registered into.
exitCode := runWithDeps([]string{}, io.Discard, io.Discard, appDeps{
getwd: func() (string, error) { return cwd, nil },
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return config.ResolvedConfig{
Provider: config.ProviderProfile{
Name: "p",
ProviderKind: config.ProviderKindOpenAICompatible,
BaseURL: "http://127.0.0.1/v1",
Model: "m",
},
}, nil
},
resolveMCPConfig: func(string, bool) (config.MCPConfig, error) { return config.MCPConfig{}, nil },
runTUI: func(_ context.Context, options tui.Options) int {
registry = options.Registry
return exitSuccess
},
})
if exitCode != exitSuccess {
t.Fatalf("runWithDeps exit = %d, want %d", exitCode, exitSuccess)
}
requireTaskOutputDispatchAttached(t, registry)
})
t.Run("headless exec", func(t *testing.T) {
isolateCLIUserState(t)
cwd := t.TempDir()
var registry *tools.Registry
var stdout, stderr bytes.Buffer
// --auto high is what registers specialist tools on the exec surface.
// MCP registration is the one startup chokepoint that sees the run
// registry before attachLifecycleHooks runs, so capture it there and
// assert after the run completes.
exitCode := runWithDeps([]string{
"exec", "--auto", "high", "--cwd", cwd, "--init-session-id", "hook_wireup", "hello",
}, &stdout, &stderr, appDeps{
getwd: func() (string, error) { return cwd, nil },
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return execResolvedConfig(), nil
},
newProvider: func(config.ProviderProfile) (zeroruntime.Provider, error) {
return echoExecProvider{}, nil
},
resolveMCPConfig: func(string, bool) (config.MCPConfig, error) {
return config.MCPConfig{Servers: map[string]config.MCPServerConfig{
"wireup": {Type: "stdio", Command: "wireup-mcp"},
}}, nil
},
newMCPStore: func() (*mcp.PermissionStore, error) { return nil, nil },
registerMCPTools: func(_ context.Context, captured *tools.Registry, _ config.MCPConfig, _ mcp.RegisterOptions) (mcpToolRuntime, error) {
registry = captured
return closeFunc(func() error { return nil }), nil
},
})
if exitCode != exitSuccess {
t.Fatalf("runWithDeps exit = %d, want %d (stderr=%q)", exitCode, exitSuccess, stderr.String())
}
requireTaskOutputDispatchAttached(t, registry)
})
}
+168
View File
@@ -0,0 +1,168 @@
package config
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"unicode"
"github.com/Twigpine/zero/internal/aimlapi"
"github.com/Twigpine/zero/internal/providercatalog"
)
const aimlapiKeyEnv = "AIMLAPI_API_KEY"
var (
// ErrAIMLAPIKeyMissing means no AI/ML API key is configured.
ErrAIMLAPIKeyMissing = errors.New("no AI/ML API key is configured; set up aimlapi.com with /provider or set AIMLAPI_API_KEY")
// ErrAIMLAPIProxyProfile means the aimlapi profile points at a proxy, so its
// key belongs to that proxy and is withheld from aimlapi.com services.
ErrAIMLAPIProxyProfile = errors.New("the aimlapi provider points at a proxy base URL; its key is not sent to aimlapi.com services")
// ErrAIMLAPIKeyInvalid means the key cannot be sent as a single Bearer token.
ErrAIMLAPIKeyInvalid = errors.New("the AI/ML API key contains whitespace, a comma or a control character")
)
// AIMLAPIServiceKey returns the AI/ML API key that may be sent to the
// aimlapi.com Toolbox and MCP server. It is resolved again on every call, so a
// provider change is seen by the next request.
//
// The key comes from the aimlapi provider profile (the active one when it is
// aimlapi, otherwise the first), including a key held in the credential store,
// or from AIMLAPI_API_KEY when no aimlapi profile exists. The key is withheld
// when that profile, any aimlapi profile, or any profile reading
// AIMLAPI_API_KEY points at a base URL other than the canonical
// https://api.aimlapi.com/v1, and when AIMLAPI_INFERENCE_URL is overridden:
// in those cases the credential may belong to a proxy.
func AIMLAPIServiceKey(resolved ResolvedConfig, store APIKeyGetter, getenv func(string) string) (string, error) {
if getenv == nil {
getenv = os.Getenv
}
if override := strings.TrimSpace(getenv("AIMLAPI_INFERENCE_URL")); override != "" && !sameBaseURL(override, aimlapi.CanonicalInferenceBaseURL) {
return "", ErrAIMLAPIProxyProfile
}
for _, profile := range resolved.Providers {
if aimlapiKeyedProfile(profile) && !sameBaseURL(profile.BaseURL, aimlapi.CanonicalInferenceBaseURL) {
return "", ErrAIMLAPIProxyProfile
}
}
if profile, ok := aimlapiServiceProfile(resolved); ok {
if !sameBaseURL(profile.BaseURL, aimlapi.CanonicalInferenceBaseURL) {
return "", ErrAIMLAPIProxyProfile
}
if store != nil && profile.APIKeyStored && strings.TrimSpace(profile.APIKey) == "" && strings.TrimSpace(profile.Name) != "" {
key, found, err := store.Get(strings.TrimSpace(profile.Name))
if err != nil {
return "", fmt.Errorf("read AI/ML API credential store: %w", err)
}
if found {
profile.APIKey = key
}
}
return validAIMLAPIServiceKey(profile.APIKey)
}
return validAIMLAPIServiceKey(getenv(aimlapiKeyEnv))
}
// LoadAIMLAPIServiceKey resolves the user configuration, the environment and
// the user credential store, then applies AIMLAPIServiceKey. Project config is
// not read: provider keys are user-scoped. A config with no active provider
// still resolves its profiles (ErrNoActiveProvider), so the key is looked up in
// that partial config; every other resolve error is returned. Resolve drops a
// non-active profile it cannot normalize, so the configured profiles are
// checked as well: such a profile still withholds the key when it carries it.
func LoadAIMLAPIServiceKey() (string, error) {
userConfigPath, err := DefaultUserConfigPath()
if err != nil {
return "", err
}
existing, err := existingConfigFile(userConfigPath)
if err != nil {
return "", err
}
resolved, err := Resolve(ResolveOptions{UserConfigPath: existing, ExcludeProject: true})
if err != nil && !errors.Is(err, ErrNoActiveProvider) {
return "", err
}
if existing != "" {
fileConfig, err := loadConfigFile(existing)
if err != nil {
return "", err
}
var configured FileConfig
mergeConfig(&configured, fileConfig)
if droppedAIMLAPIProxyProfile(configured.Providers, resolved.Providers) {
return "", ErrAIMLAPIProxyProfile
}
}
// Check proxy eligibility and already-resolved keys before opening a store.
key, keyErr := AIMLAPIServiceKey(resolved, nil, os.Getenv)
if !errors.Is(keyErr, ErrAIMLAPIKeyMissing) {
return key, keyErr
}
profile, ok := aimlapiServiceProfile(resolved)
if !ok || !profile.APIKeyStored {
return key, keyErr
}
store, err := ProviderKeyStoreAt(filepath.Dir(userConfigPath))
if err != nil {
return "", fmt.Errorf("open AI/ML API credential store: %w", err)
}
return AIMLAPIServiceKey(resolved, store, os.Getenv)
}
func isAIMLAPIProfile(profile ProviderProfile) bool {
return strings.EqualFold(providercatalog.NormalizeID(profile.CatalogID), providercatalog.AIMLAPIID)
}
// aimlapiKeyedProfile reports whether profile carries an AI/ML API credential:
// an aimlapi catalog profile, or any profile that reads AIMLAPI_API_KEY.
func aimlapiKeyedProfile(profile ProviderProfile) bool {
return isAIMLAPIProfile(profile) || strings.EqualFold(strings.TrimSpace(profile.APIKeyEnv), aimlapiKeyEnv)
}
// droppedAIMLAPIProxyProfile reports whether a configured profile missing from
// the resolved list (Resolve could not normalize it) carries an AI/ML API
// credential and is not explicitly on the canonical base URL. Its endpoint was
// never resolved, so an empty base URL counts as a proxy.
func droppedAIMLAPIProxyProfile(configured, resolved []ProviderProfile) bool {
kept := make(map[string]bool, len(resolved))
for _, profile := range resolved {
kept[profile.Name] = true
}
for _, profile := range configured {
if kept[strings.TrimSpace(profile.Name)] || !aimlapiKeyedProfile(profile) {
continue
}
if !sameBaseURL(profile.BaseURL, aimlapi.CanonicalInferenceBaseURL) {
return true
}
}
return false
}
func aimlapiServiceProfile(resolved ResolvedConfig) (ProviderProfile, bool) {
if isAIMLAPIProfile(resolved.Provider) {
return resolved.Provider, true
}
for _, profile := range resolved.Providers {
if isAIMLAPIProfile(profile) {
return profile, true
}
}
return ProviderProfile{}, false
}
func validAIMLAPIServiceKey(raw string) (string, error) {
key := strings.TrimSpace(raw)
if key == "" {
return "", ErrAIMLAPIKeyMissing
}
for _, r := range key {
if unicode.IsSpace(r) || r <= ' ' || r == 0x7f || r == ',' || (r >= 0x80 && r <= 0x9f) {
return "", ErrAIMLAPIKeyInvalid
}
}
return key, nil
}
+282
View File
@@ -0,0 +1,282 @@
package config
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
)
func aimlapiServiceKeyEnv(values map[string]string) func(string) string {
return func(name string) string { return values[name] }
}
func TestAIMLAPIServiceKeyUsesCanonicalProfileKey(t *testing.T) {
resolved := ResolvedConfig{Providers: []ProviderProfile{
{Name: "openai", BaseURL: "https://api.openai.com/v1", APIKey: "sk-openai"},
{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1/", APIKey: " aiml-key "},
}}
key, err := AIMLAPIServiceKey(resolved, nil, aimlapiServiceKeyEnv(nil))
if err != nil || key != "aiml-key" {
t.Fatalf("AIMLAPIServiceKey = %q, %v; want aiml-key", key, err)
}
}
func TestAIMLAPIServiceKeyReadsCredentialStore(t *testing.T) {
resolved := ResolvedConfig{Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKeyStored: true}}
store := fakeKeyGetter{keys: map[string]string{"aimlapi": "stored-key"}}
key, err := AIMLAPIServiceKey(resolved, store, aimlapiServiceKeyEnv(nil))
if err != nil || key != "stored-key" {
t.Fatalf("AIMLAPIServiceKey = %q, %v; want stored-key", key, err)
}
}
func TestAIMLAPIServiceKeyPrefersResolvedProfileKeyOverCredentialStore(t *testing.T) {
resolved := ResolvedConfig{Provider: ProviderProfile{
Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1",
APIKey: "env-key", APIKeyStored: true,
}}
store := fakeKeyGetter{keys: map[string]string{"aimlapi": "stored-key"}}
key, err := AIMLAPIServiceKey(resolved, store, aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "env-key"}))
if err != nil || key != "env-key" {
t.Fatalf("AIMLAPIServiceKey = %q, %v; want env-key from resolved profile, not stored-key", key, err)
}
}
func TestAIMLAPIServiceKeyFallsBackToEnvironmentWithoutProfile(t *testing.T) {
key, err := AIMLAPIServiceKey(ResolvedConfig{}, nil, aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "env-key"}))
if err != nil || key != "env-key" {
t.Fatalf("AIMLAPIServiceKey = %q, %v; want env-key", key, err)
}
if _, err := AIMLAPIServiceKey(ResolvedConfig{}, nil, aimlapiServiceKeyEnv(nil)); !errors.Is(err, ErrAIMLAPIKeyMissing) {
t.Fatalf("missing key error = %v, want ErrAIMLAPIKeyMissing", err)
}
}
func TestAIMLAPIServiceKeyWithholdsProxyCredentials(t *testing.T) {
env := aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "proxy-key"})
cases := map[string]ResolvedConfig{
"active aimlapi profile on a proxy": {Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://proxy.example/v1", APIKey: "proxy-key"}},
"a second aimlapi profile on a proxy": {Providers: []ProviderProfile{
{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKey: "proxy-key"},
{Name: "aimlapi-proxy", CatalogID: "aimlapi", BaseURL: "https://proxy.example/v1", APIKey: "proxy-key"},
}},
"a custom profile reading AIMLAPI_API_KEY": {Providers: []ProviderProfile{
{Name: "gateway", BaseURL: "https://proxy.example/v1", APIKeyEnv: "AIMLAPI_API_KEY", APIKey: "proxy-key"},
}},
}
for name, resolved := range cases {
key, err := AIMLAPIServiceKey(resolved, nil, env)
if !errors.Is(err, ErrAIMLAPIProxyProfile) || key != "" {
t.Errorf("%s: AIMLAPIServiceKey = %q, %v; want ErrAIMLAPIProxyProfile", name, key, err)
}
}
overridden := aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "proxy-key", "AIMLAPI_INFERENCE_URL": "https://staging.example/v1"})
if key, err := AIMLAPIServiceKey(ResolvedConfig{}, nil, overridden); !errors.Is(err, ErrAIMLAPIProxyProfile) || key != "" {
t.Fatalf("inference URL override: AIMLAPIServiceKey = %q, %v; want ErrAIMLAPIProxyProfile", key, err)
}
}
func TestAIMLAPIServiceKeyRejectsKeysThatAreNotOneToken(t *testing.T) {
for _, raw := range []string{"first\u00a0second", "first\u2003second", "first,second", "first\nsecond", "first\r\nsecond", "a b", "key\x00tail", "key\u0085tail"} {
resolved := ResolvedConfig{Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKey: raw}}
if key, err := AIMLAPIServiceKey(resolved, nil, aimlapiServiceKeyEnv(nil)); !errors.Is(err, ErrAIMLAPIKeyInvalid) || key != "" {
t.Errorf("key %q: AIMLAPIServiceKey = %q, %v; want ErrAIMLAPIKeyInvalid", raw, key, err)
}
}
}
func TestLoadAIMLAPIServiceKeyResolvesUserConfig(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("ZERO_CRED_STORAGE", "file")
t.Setenv("ZERO_PROVIDER", "")
t.Setenv("AIMLAPI_INFERENCE_URL", "")
t.Setenv("AIMLAPI_API_KEY", "env-key")
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "env-key" {
t.Fatalf("without config: LoadAIMLAPIServiceKey = %q, %v; want env-key", key, err)
}
path, err := DefaultUserConfigPath()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
t.Fatal(err)
}
proxy := `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi","baseURL":"https://proxy.example/v1","model":"m"}]}`
if err := os.WriteFile(path, []byte(proxy), 0o600); err != nil {
t.Fatal(err)
}
if key, err := LoadAIMLAPIServiceKey(); !errors.Is(err, ErrAIMLAPIProxyProfile) || key != "" {
t.Fatalf("proxy profile: LoadAIMLAPIServiceKey = %q, %v; want ErrAIMLAPIProxyProfile", key, err)
}
canonical := `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi"}]}`
if err := os.WriteFile(path, []byte(canonical), 0o600); err != nil {
t.Fatal(err)
}
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "env-key" {
t.Fatalf("canonical profile: LoadAIMLAPIServiceKey = %q, %v; want env-key", key, err)
}
}
// Several profiles and no activeProvider make Resolve return
// ErrNoActiveProvider with the normalized profiles; the interactive app keeps
// running on that partial config, so the service key must still resolve.
func TestLoadAIMLAPIServiceKeyWithoutActiveProvider(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("ZERO_CRED_STORAGE", "file")
t.Setenv("ZERO_PROVIDER", "")
t.Setenv("AIMLAPI_INFERENCE_URL", "")
t.Setenv("AIMLAPI_API_KEY", "env-key")
path, err := DefaultUserConfigPath()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
t.Fatal(err)
}
other := `{"name":"other","provider":"openai","model":"m","apiKey":"other-key"}`
for _, tc := range []struct {
name string
active string
profile string
want string
wantErr error
}{
{name: "environment key", profile: `{"name":"second","provider":"openai","model":"m","apiKey":"second-key"}`, want: "env-key"},
{name: "stale active provider", active: "gone", profile: `{"name":"second","provider":"openai","model":"m","apiKey":"second-key"}`, want: "env-key"},
{name: "aimlapi profile key", profile: `{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKey":"profile-key"}`, want: "profile-key"},
{name: "proxy profile", profile: `{"name":"aimlapi","catalogID":"aimlapi","baseURL":"https://proxy.example/v1","model":"m"}`, wantErr: ErrAIMLAPIProxyProfile},
} {
data := `{"activeProvider":"` + tc.active + `","providers":[` + other + `,` + tc.profile + `]}`
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
t.Fatal(err)
}
key, err := LoadAIMLAPIServiceKey()
if tc.wantErr != nil {
if !errors.Is(err, tc.wantErr) || key != "" {
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %v", tc.name, key, err, tc.wantErr)
}
continue
}
if err != nil || key != tc.want {
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %q", tc.name, key, err, tc.want)
}
}
}
// Resolve drops a non-active profile it cannot normalize (here an unknown
// catalog entry). Such a profile still shows that AIMLAPI_API_KEY belongs to a
// proxy, so the key is withheld whether or not a provider is active.
func TestLoadAIMLAPIServiceKeyWithholdsForDroppedProxyProfile(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("ZERO_CRED_STORAGE", "file")
t.Setenv("ZERO_PROVIDER", "")
t.Setenv("AIMLAPI_INFERENCE_URL", "")
t.Setenv("AIMLAPI_API_KEY", "env-key")
path, err := DefaultUserConfigPath()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
t.Fatal(err)
}
canonical := `{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKey":"profile-key"}`
other := `{"name":"other","provider":"openai","model":"m","apiKey":"other-key"}`
dropped := `{"name":"future","catalogID":"no-such-catalog","baseURL":"https://proxy.example/v1","apiKeyEnv":"AIMLAPI_API_KEY","model":"m"}`
unrelated := `{"name":"future","catalogID":"no-such-catalog","baseURL":"https://proxy.example/v1","apiKeyEnv":"OTHER_API_KEY","model":"m"}`
for _, tc := range []struct {
name string
data string
want string
wantErr error
}{
{name: "active aimlapi profile", data: `{"activeProvider":"aimlapi","providers":[` + canonical + `,` + dropped + `]}`, wantErr: ErrAIMLAPIProxyProfile},
{name: "no active provider", data: `{"providers":[` + other + `,` + dropped + `]}`, wantErr: ErrAIMLAPIProxyProfile},
{name: "dropped profile without the key", data: `{"activeProvider":"aimlapi","providers":[` + canonical + `,` + unrelated + `]}`, want: "profile-key"},
} {
if err := os.WriteFile(path, []byte(tc.data), 0o600); err != nil {
t.Fatal(err)
}
key, err := LoadAIMLAPIServiceKey()
if tc.wantErr != nil {
if !errors.Is(err, tc.wantErr) || key != "" {
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %v", tc.name, key, err, tc.wantErr)
}
continue
}
if err != nil || key != tc.want {
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %q", tc.name, key, err, tc.want)
}
}
}
func TestLoadAIMLAPIServiceKeyStoreOpenFailure(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("ZERO_PROVIDER", "")
t.Setenv("AIMLAPI_INFERENCE_URL", "")
t.Setenv("AIMLAPI_API_KEY", "env-key")
t.Setenv("ZERO_CRED_STORAGE", "invalid-test-storage")
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "env-key" {
t.Fatalf("environment key must not require a store: %q, %v", key, err)
}
path, err := DefaultUserConfigPath()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
t.Fatal(err)
}
t.Setenv("AIMLAPI_API_KEY", "")
data := `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKeyStored":true}]}`
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
t.Fatal(err)
}
if key, err := LoadAIMLAPIServiceKey(); key != "" || err == nil || !strings.Contains(err.Error(), "unknown storage") {
t.Fatalf("stored key must report store failure: %q, %v", key, err)
}
t.Setenv("AIMLAPI_INFERENCE_URL", "https://proxy.example/v1")
if key, err := LoadAIMLAPIServiceKey(); key != "" || !errors.Is(err, ErrAIMLAPIProxyProfile) {
t.Fatalf("proxy rejection must precede store open: %q, %v", key, err)
}
t.Setenv("AIMLAPI_INFERENCE_URL", "")
data = `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKeyStored":true,"apiKey":"inline-key"}]}`
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
t.Fatal(err)
}
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "inline-key" {
t.Fatalf("inline key must not require a store: %q, %v", key, err)
}
}
func TestAIMLAPIServiceKeyReportsStoreReadFailure(t *testing.T) {
want := errors.New("test credential read failure")
resolved := ResolvedConfig{Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKeyStored: true}}
if key, err := AIMLAPIServiceKey(resolved, fakeKeyGetter{err: want}, aimlapiServiceKeyEnv(nil)); key != "" || !errors.Is(err, want) {
t.Fatalf("read failure was hidden: %q, %v", key, err)
}
resolved.Provider.BaseURL = "https://proxy.example/v1"
if _, err := AIMLAPIServiceKey(resolved, fakeKeyGetter{err: want}, aimlapiServiceKeyEnv(nil)); !errors.Is(err, ErrAIMLAPIProxyProfile) {
t.Fatalf("proxy check must precede store read: %v", err)
}
}
+51
View File
@@ -0,0 +1,51 @@
package config
import (
"encoding/json"
"os"
"path/filepath"
"testing"
)
func TestEnsureChatGPTDefaultPreservesSavedSelection(t *testing.T) {
for _, saved := range []string{"", "gpt-explicit-saved"} {
name := "new profile"
if saved != "" {
name = "saved selection"
}
t.Run(name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.json")
if saved != "" {
body, err := json.Marshal(FileConfig{ActiveProvider: "codex", Providers: []ProviderProfile{{Name: "codex", CatalogID: "chatgpt", Model: saved}}})
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, body, 0600); err != nil {
t.Fatal(err)
}
}
ensured, err := EnsureCatalogProvider(path, "chatgpt")
if err != nil {
t.Fatal(err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var cfg FileConfig
if err := json.Unmarshal(body, &cfg); err != nil {
t.Fatal(err)
}
want := "gpt-6.1-sol"
if saved != "" {
want = saved
}
if len(cfg.Providers) != 1 || cfg.Providers[0].Model != want {
t.Fatalf("ChatGPT default/selection = %#v, want model %q", cfg.Providers, want)
}
if saved != "" && (ensured.Name != "codex" || ensured.Created) {
t.Fatalf("saved subscription profile was replaced: %#v", ensured)
}
})
}
}
+27 -5
View File
@@ -3,26 +3,48 @@ package config
import (
"reflect"
"strings"
"github.com/Twigpine/zero/internal/aimlapi"
)
// DefaultMCPServers returns the MCP servers Zero ships ENABLED by default so
// web search and page fetching work out of the box with no setup and no API
// key. They are seeded before user/project config is merged (see ResolveMCP),
// so a user can override any field — for example add an API-key header to lift
// Exa's anonymous rate limit — or disable it entirely with
// DefaultMCPServers returns the MCP servers Zero ships built in. Exa is ENABLED
// by default so web search and page fetching work out of the box with no setup
// and no API key. They are seeded before user/project config is merged (see
// ResolveMCP), so a user can override any field — for example add an API-key
// header to lift Exa's anonymous rate limit — or disable it entirely with
// `zero mcp disable <name>` (which writes `"disabled": true`).
//
// Exa's hosted MCP server works anonymously with rate limits. Users can add an
// Exa API key for higher limits.
//
// The AI/ML API server ships DISABLED: its tool runs and generations are billed
// to the user's AI/ML API balance, so it only connects after
// `zero mcp enable aimlapi` (which writes `"disabled": false`). The disable is
// marked as explicitly set so the lower-trust project layer cannot lift it
// (see mergeMCPServer); only the user and CLI-override scopes can. The key is
// attached per request by the MCP client and is never part of this config.
func DefaultMCPServers() map[string]MCPServerConfig {
return map[string]MCPServerConfig{
"exa": {
Type: "http",
URL: "https://mcp.exa.ai/mcp",
},
"aimlapi": {
Type: "http",
URL: aimlapi.MCPServerURL,
Disabled: true,
disabledSet: true,
},
}
}
// IsDefaultMCPServerDisabled reports whether name is a built-in default that
// ships disabled, so enabling it has to be written out as `"disabled": false`
// rather than by dropping the flag.
func IsDefaultMCPServerDisabled(name string) bool {
return DefaultMCPServers()[strings.TrimSpace(name)].Disabled
}
// IsDefaultMCPServer reports whether name is one of Zero's built-in default MCP
// servers. The config commands use it so a default can be disabled/enabled even
// though it is not written to the user's config file until overridden.
+89
View File
@@ -291,3 +291,92 @@ func TestResolveMCPDisabledRetiredEntryStaysReEnableable(t *testing.T) {
t.Fatal("the disable must still carry to the successor")
}
}
func writeMCPDefaultsTestConfig(t *testing.T, data string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.json")
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
t.Fatal(err)
}
return path
}
func TestResolveMCPSeedsDisabledAIMLAPIDefault(t *testing.T) {
if !IsDefaultMCPServer("aimlapi") || !IsDefaultMCPServerDisabled("aimlapi") || IsDefaultMCPServerDisabled("exa") {
t.Fatal("aimlapi must be a built-in default that ships disabled, exa one that ships enabled")
}
// No user config at all, and a user config that never mentions aimlapi.
for name, options := range map[string]ResolveOptions{
"no config": {},
"unrelated user file": {UserConfigPath: writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"exa":{"disabled":true}}}}`)},
"user entry with only headers": {UserConfigPath: writeMCPDefaultsTestConfig(t,
`{"mcp":{"servers":{"aimlapi":{"headers":{"X-Trace":"1"}}}}}`)},
} {
cfg, err := ResolveMCP(options)
if err != nil {
t.Fatalf("%s: ResolveMCP: %v", name, err)
}
server, ok := cfg.Servers["aimlapi"]
if !ok {
t.Fatalf("%s: the aimlapi default must be visible in the resolved config", name)
}
if server.Type != "http" || server.URL != "https://mcp.aimlapi.com/mcp" || !server.Disabled {
t.Fatalf("%s: aimlapi default = %#v, want a disabled http server at the MCP URL", name, server)
}
}
cfg, err := ResolveMCP(ResolveOptions{})
if err != nil {
t.Fatal(err)
}
if !IsUnconfiguredDefault("aimlapi", cfg.Servers["aimlapi"]) {
t.Fatal("the untouched aimlapi default should be reported as unconfigured")
}
if exa := cfg.Servers["exa"]; exa.Disabled || !IsUnconfiguredDefault("exa", exa) {
t.Fatalf("exa default changed: %#v", exa)
}
}
func TestResolveMCPUserEnablesAIMLAPIDefault(t *testing.T) {
path := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":false}}}}`)
cfg, err := ResolveMCP(ResolveOptions{UserConfigPath: path})
if err != nil {
t.Fatalf("ResolveMCP: %v", err)
}
server := cfg.Servers["aimlapi"]
if server.Disabled || server.URL != "https://mcp.aimlapi.com/mcp" || server.Type != "http" {
t.Fatalf("user enable did not turn the default on: %#v", server)
}
if IsUnconfiguredDefault("aimlapi", server) {
t.Fatal("an enabled aimlapi is user-configured, so its failures must be reported")
}
enable := MCPServerConfig{Disabled: false, disabledSet: true}
cfg, err = ResolveMCP(ResolveOptions{Overrides: Overrides{MCP: MCPConfig{Servers: map[string]MCPServerConfig{"aimlapi": enable}}}})
if err != nil {
t.Fatal(err)
}
if cfg.Servers["aimlapi"].Disabled {
t.Fatal("the CLI override scope must be able to enable the aimlapi default")
}
}
func TestResolveMCPProjectCannotEnableAIMLAPIDefault(t *testing.T) {
project := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":false}}}}`)
cfg, err := ResolveMCP(ResolveOptions{ProjectConfigPath: project})
if err != nil {
t.Fatalf("ResolveMCP: %v", err)
}
if !cfg.Servers["aimlapi"].Disabled {
t.Fatal("a project config must not turn on the billed aimlapi default")
}
user := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":false}}}}`)
projectDisable := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":true}}}}`)
cfg, err = ResolveMCP(ResolveOptions{UserConfigPath: user, ProjectConfigPath: projectDisable})
if err != nil {
t.Fatalf("ResolveMCP: %v", err)
}
if cfg.Servers["aimlapi"].Disabled {
t.Fatal("the user's explicit enable wins over the project layer, as for any server")
}
}
+33
View File
@@ -724,6 +724,39 @@ func (cfg LocalControlDriverConfig) MarshalJSON() ([]byte, error) {
return json.Marshal(raw)
}
// MarshalJSON writes "disabled" whenever it was explicitly set, not only when
// true. A read-modify-write of config.json (SetTheme, SetActiveProvider, ...)
// would otherwise drop `"disabled": false`, which turns a default that ships
// disabled (aimlapi) back off and lets project config override a user's
// explicit enable (see mergeMCPServer).
func (server MCPServerConfig) MarshalJSON() ([]byte, error) {
type rawServer struct {
Type string `json:"type,omitempty"`
Command string `json:"command,omitempty"`
Args []string `json:"args,omitempty"`
Env map[string]string `json:"env,omitempty"`
URL string `json:"url,omitempty"`
Headers map[string]string `json:"headers,omitempty"`
Auth string `json:"auth,omitempty"`
OAuth *MCPOAuthConfig `json:"oauth,omitempty"`
Disabled *bool `json:"disabled,omitempty"`
}
raw := rawServer{
Type: server.Type,
Command: server.Command,
Args: server.Args,
Env: server.Env,
URL: server.URL,
Headers: server.Headers,
Auth: server.Auth,
OAuth: server.OAuth,
}
if server.disabledSet || server.Disabled {
raw.Disabled = &server.Disabled
}
return json.Marshal(raw)
}
func (server *MCPServerConfig) UnmarshalJSON(data []byte) error {
type rawServer struct {
Type string `json:"type"`
+4 -70
View File
@@ -99,77 +99,11 @@ func windowsSandboxSetupCheck(goos string, backend sandbox.Backend, workspaceRoo
return &result
}
profile := sandbox.PermissionProfileFromPolicy(workspaceRoot, doctorSandboxPolicy(sandboxConfig), scope)
// A RECORDED ROOT IS HISTORY, NOT PROOF. Doctor pins the marker's runtime
// root so the stamp check can run without taking a lease, but a command does
// not pin blindly: it derives the current cache and fallback candidates and
// honours the marker only when its root is one of them. Run setup with the
// cache at A, relocate the cache so commands derive B, and the stamped A tree
// remains: pinning A here reported a healthy sandbox immediately before every
// real command rejected A and failed on the out-of-date marker.
//
// So ask the command's own question first, through the command's own
// function. Only a root a command would still select is used to validate
// the marker; anything else is reported as out of date, with the remedy.
//
// THREE STATES, NOT TWO. The helper answers current, stale, or "I could not
// tell", and the last one used to fall through to the pinned branch: a valid
// old marker then made doctor report healthy while selectSandboxRuntimeRoot,
// reached by BuildCommandPlan, propagated the same resolution error and stopped
// every command from launching. Historical state is not a substitute for a
// failed current selection, so an error is surfaced as its own warning with the
// cause, and the pinned validation profile is built only once currentness has
// actually been established.
recorded, current, rootErr := sandbox.WindowsSandboxRecordedRuntimeRootIsCurrent(sandboxHome, workspaceRoot)
if rootErr != nil {
result := check("sandbox.backend", "Sandbox backend", StatusWarn, fmt.Sprintf("Native sandbox backend %s is installed, but the runtime root a command would select cannot be resolved (%s), so the recorded setup cannot be trusted and commands are likely to fail to launch.", backend.Name, rootErr.Error()), map[string]any{
"backend": string(backend.Name),
"platform": goos,
"supportLevel": string(backend.SupportLevel()),
"setupStatus": "runtime-root-unresolved",
"error": rootErr.Error(),
"remedy": "run `zero sandbox setup` to prepare the Windows native sandbox for the current cache location",
})
return &result
}
if recorded != "" && !current {
result := check("sandbox.backend", "Sandbox backend", StatusWarn, fmt.Sprintf("Native sandbox backend %s is installed, but the runtime root setup recorded (%s) is not one a command would select now, so the setup is out of date.", backend.Name, recorded), map[string]any{
"backend": string(backend.Name),
"platform": goos,
"supportLevel": string(backend.SupportLevel()),
"setupStatus": "runtime-root-stale",
"runtimeRoot": recorded,
"remedy": "run `zero sandbox setup` to prepare the Windows native sandbox for the current cache location",
})
return &result
}
setupConfig := sandbox.WindowsSandboxSetupConfig{
SandboxHome: sandboxHome,
CommandCWD: workspaceRoot,
WorkspaceRoots: []string{workspaceRoot},
// The same augmentation setup and the command plan apply, so doctor
// fingerprints what a real command fingerprints. Checking the bare profile
// made doctor call a correctly prepared machine "out of date", which is the
// mismatch this pairing exists to close. Safe to resolve in this process:
// doctor runs in the operator's shell, not behind the sandbox TEMP
// redirection that stops the runner deriving these for itself.
//
// The RECORDED runtime root goes on the profile as well, because that is
// what makes the stamp check run at all. Without it profile.Runtime is nil,
// validateWindowsSandboxRuntimeStamp returns nil early, and doctor reported
// a healthy sandbox on a machine whose runtime tree had been evicted and
// silently recreated without the capability ACE -- precisely the state the
// stamp exists to catch, and the state where every sandboxed command then
// fails with nothing explaining why.
//
// Read from the marker rather than selected, so doctor takes no lease and
// creates nothing, and read ONCE: this is the root whose currentness was
// checked above, not a second read that a rewritten marker could change. A
// marker from an older schema records no root, which leaves this exactly as
// it was.
PermissionProfile: sandbox.WindowsSandboxProfileWithRuntimeRoots(
sandbox.PermissionProfileWithRuntimeRoot(profile, recorded),
[]string{workspaceRoot},
),
SandboxHome: sandboxHome,
CommandCWD: workspaceRoot,
WorkspaceRoots: []string{workspaceRoot},
PermissionProfile: profile,
}
if err := sandbox.ValidateWindowsSandboxSetupMarker(setupConfig); err != nil {
result := check("sandbox.backend", "Sandbox backend", StatusWarn, fmt.Sprintf("Native sandbox backend %s is installed, but Windows sandbox setup is missing or out of date: %v.", backend.Name, err), map[string]any{
@@ -1,247 +0,0 @@
package doctor
import (
"os"
"runtime"
"strings"
"testing"
"github.com/Twigpine/zero/internal/config"
"github.com/Twigpine/zero/internal/sandbox"
)
// DOCTOR HAS TO ASK THE ONE QUESTION THE MARKER CANNOT ANSWER.
//
// The marker hashes ACL-plan entries, which are pathnames. Whether the directory
// those pathnames resolve to is still the tree setup provisioned is a different
// question, and the runtime stamp is what answers it. validateWindowsSandboxRuntimeStamp
// returns nil early when profile.Runtime is nil, which is correct for the setup
// side and for unrestricted profiles, and was wrong here: doctor built its
// profile with PermissionProfileFromPolicy, which never sets Runtime, so the
// check was skipped and `zero doctor` reported a healthy sandbox on exactly the
// state the stamp was added to detect -- an evicted runtime tree, silently
// recreated with ordinary permissions and no capability ACE, where every
// sandboxed command then fails with nothing explaining why.
// doctorRuntimeCandidate returns a runtime root this workspace would really
// select, taken from the candidate set the Windows plan folds in.
//
// It has to be a REAL candidate. An arbitrary path is already in the ACL plan on
// one side and not the other, so the plan hashes diverge and validation fails
// before the stamp is ever consulted -- which would make this test fail without
// the fix for a reason that has nothing to do with the stamp.
// redirectUserCache points os.UserCacheDir at test-owned storage.
//
// WITHOUT THIS THE TEST WRITES INTO THE DEVELOPER'S REAL CACHE. The runtime
// candidate is derived from the process's actual user cache directory, so the
// test was creating, recursively removing, recreating and then cleanup-removing
// a directory under the real %LocalAppData%zerountime (or ~/.cache/zero on
// Unix). The workspace digest made a collision with a live runtime tree
// unlikely rather than impossible, and "unlikely" is not the standard for a
// test that calls RemoveAll.
//
// All three variables, because os.UserCacheDir reads a different one per
// platform: %LocalAppData% on Windows, $XDG_CACHE_HOME or $HOME on Unix, and
// $HOME on macOS.
func redirectUserCache(t *testing.T) {
t.Helper()
cache := t.TempDir()
t.Setenv("LOCALAPPDATA", cache)
t.Setenv("XDG_CACHE_HOME", cache)
t.Setenv("HOME", cache)
}
func doctorRuntimeCandidate(t *testing.T, workspace string) string {
t.Helper()
bare := doctorProfile(t, workspace)
augmented := sandbox.WindowsSandboxProfileWithRuntimeRoots(bare, []string{workspace})
existing := map[string]bool{}
for _, root := range bare.FileSystem.WriteRoots {
existing[root.Root] = true
}
for _, root := range augmented.FileSystem.WriteRoots {
if !existing[root.Root] {
return root.Root
}
}
t.Skip("no runtime candidate is derivable in this environment")
return ""
}
func doctorProfile(t *testing.T, workspace string) sandbox.PermissionProfile {
t.Helper()
scope, err := sandbox.NewScope(workspace, nil)
if err != nil {
t.Fatalf("NewScope: %v", err)
}
return sandbox.PermissionProfileFromPolicy(workspace, doctorSandboxPolicy(config.SandboxConfig{}), scope)
}
func writeDoctorSetupMarker(t *testing.T, home, workspace, runtimeRoot string) {
t.Helper()
profile := doctorProfile(t, workspace)
setup := sandbox.WindowsSandboxSetupConfig{
SandboxHome: home,
CommandCWD: workspace,
WorkspaceRoots: []string{workspace},
PermissionProfile: sandbox.WindowsSandboxProfileWithRuntimeRoots(
sandbox.PermissionProfileWithRuntimeRoot(profile, runtimeRoot),
[]string{workspace},
),
}
if _, err := sandbox.WriteWindowsSandboxSetupMarker(setup); err != nil {
t.Fatalf("WriteWindowsSandboxSetupMarker: %v", err)
}
}
func TestDoctorReportsAnEvictedRuntimeTree(t *testing.T) {
home := t.TempDir()
workspace := t.TempDir()
t.Setenv("ZERO_WINDOWS_SANDBOX_HOME", home)
redirectUserCache(t)
runtimeRoot := doctorRuntimeCandidate(t, workspace)
// Belt and braces: if the redirection above ever stops working, fail loudly
// rather than quietly operating on the developer's real cache.
if !strings.HasPrefix(runtimeRoot, os.TempDir()) && !strings.Contains(runtimeRoot, t.Name()) {
t.Fatalf("the runtime candidate %q is outside test-owned storage; this test creates and removes that path", runtimeRoot)
}
if err := os.MkdirAll(runtimeRoot, 0o700); err != nil {
t.Fatalf("create the runtime root: %v", err)
}
t.Cleanup(func() { _ = os.RemoveAll(runtimeRoot) })
writeDoctorSetupMarker(t, home, workspace, runtimeRoot)
backend := sandbox.Backend{Name: sandbox.BackendWindowsRestrictedToken}
// A healthy machine first, or the eviction assertion below would be satisfied
// by a check that warns unconditionally.
if result := windowsSandboxSetupCheck("windows", backend, workspace, config.SandboxConfig{}); result != nil {
t.Fatalf("a freshly set-up machine was reported unhealthy: %s", result.Message)
}
// cleanupSandboxRuntimeRoots evicts inactive roots on an age and count policy,
// and the next run recreates the pathname with inherited permissions. The plan
// hash never moves, so only the stamp can tell.
if err := os.RemoveAll(runtimeRoot); err != nil {
t.Fatalf("evict the runtime root: %v", err)
}
if err := os.MkdirAll(runtimeRoot, 0o700); err != nil {
t.Fatalf("recreate the pathname the way an ordinary run would: %v", err)
}
result := windowsSandboxSetupCheck("windows", backend, workspace, config.SandboxConfig{})
if result == nil {
t.Fatal("doctor reported a healthy sandbox while the provisioned runtime tree was gone; every sandboxed command on this machine would fail with nothing explaining why")
}
if !strings.Contains(strings.ToLower(result.Message), "setup") {
t.Errorf("the warning does not point at setup: %s", result.Message)
}
}
// UNKNOWN IS NOT CURRENT.
//
// WindowsSandboxRecordedRuntimeRootIsCurrent answers three ways: the recorded
// root is still one a command would select, it is not, or the inputs a command
// selects from cannot be resolved at all. The third used to fall through to the
// pinned branch, so a valid old marker and stamp made doctor report healthy
// while selectSandboxRuntimeRoot — reached by BuildCommandPlan — propagated the
// same resolution failure and stopped every command from launching. Doctor and
// the command disagreed about the same machine.
//
// Stale and unknown are different states, so this is deliberately separate from
// the cache-A-to-cache-B test rather than another assertion on the same boolean.
func TestDoctorWarnsWhenTheRuntimeRootCannotBeResolved(t *testing.T) {
home := t.TempDir()
workspace := t.TempDir()
t.Setenv("ZERO_WINDOWS_SANDBOX_HOME", home)
redirectUserCache(t)
runtimeRoot := doctorRuntimeCandidate(t, workspace)
if !strings.HasPrefix(runtimeRoot, os.TempDir()) && !strings.Contains(runtimeRoot, t.Name()) {
t.Fatalf("the runtime candidate %q is outside test-owned storage", runtimeRoot)
}
if err := os.MkdirAll(runtimeRoot, 0o700); err != nil {
t.Fatalf("create the runtime root: %v", err)
}
t.Cleanup(func() { _ = os.RemoveAll(runtimeRoot) })
writeDoctorSetupMarker(t, home, workspace, runtimeRoot)
backend := sandbox.Backend{Name: sandbox.BackendWindowsRestrictedToken}
// Healthy first, so the assertion below cannot be satisfied by a check that
// warns unconditionally.
if result := windowsSandboxSetupCheck("windows", backend, workspace, config.SandboxConfig{}); result != nil {
t.Fatalf("SETUP INVALID: a freshly set-up machine was reported unhealthy: %s", result.Message)
}
// Now take away the inputs the selection is derived from, leaving the marker
// and the provisioned tree exactly as they were. os.UserCacheDir fails with no
// location to read, which is the same error a command would hit.
t.Setenv("LOCALAPPDATA", "")
t.Setenv("XDG_CACHE_HOME", "")
t.Setenv("HOME", "")
result := windowsSandboxSetupCheck("windows", backend, workspace, config.SandboxConfig{})
if result == nil {
t.Fatal("doctor reported a healthy sandbox while the runtime root a command selects could not be resolved; every command would fail to launch with doctor saying nothing")
}
if status, _ := result.Details["setupStatus"].(string); status != "runtime-root-unresolved" {
t.Errorf("setupStatus = %q, want runtime-root-unresolved so the unknown state is not reported as staleness: %s", status, result.Message)
}
if _, ok := result.Details["error"]; !ok {
t.Errorf("the warning does not carry the resolver cause, so an operator cannot act on it: %+v", result.Details)
}
}
// A CACHE DIRECTORY THAT RESOLVES TO NOTHING IS UNRESOLVED TOO, not only one
// that fails to resolve. os.UserCacheDir succeeds on a value like "." and the
// selection a command makes refuses it after canonicalizing, with "user cache
// directory is unavailable". The diagnostic behind this check did not, so doctor
// went on to compare the recorded root against candidates derived from an empty
// cache root and reported staleness, which sends an operator to re-run setup on
// a machine where setup cannot select a root either.
//
// The sandbox package pins the two entry points to one error through the
// resolver seam on every platform. This is the consumer's half, and it is
// Windows-only because that is where the environment can produce the value:
// %LocalAppData% is returned as it is, while a relative XDG_CACHE_HOME is an
// error on Linux and a relative HOME still joins to a usable path on macOS.
// Reported by @jatmn.
func TestDoctorReportsAnEmptyCacheRootAsUnresolvedNotStale(t *testing.T) {
if runtime.GOOS != "windows" {
t.Skip("only %LocalAppData% hands os.UserCacheDir a value that canonicalizes to nothing")
}
home := t.TempDir()
workspace := t.TempDir()
t.Setenv("ZERO_WINDOWS_SANDBOX_HOME", home)
redirectUserCache(t)
runtimeRoot := doctorRuntimeCandidate(t, workspace)
if !strings.HasPrefix(runtimeRoot, os.TempDir()) && !strings.Contains(runtimeRoot, t.Name()) {
t.Fatalf("the runtime candidate %q is outside test-owned storage", runtimeRoot)
}
if err := os.MkdirAll(runtimeRoot, 0o700); err != nil {
t.Fatalf("create the runtime root: %v", err)
}
t.Cleanup(func() { _ = os.RemoveAll(runtimeRoot) })
writeDoctorSetupMarker(t, home, workspace, runtimeRoot)
backend := sandbox.Backend{Name: sandbox.BackendWindowsRestrictedToken}
if result := windowsSandboxSetupCheck("windows", backend, workspace, config.SandboxConfig{}); result != nil {
t.Fatalf("SETUP INVALID: a freshly set-up machine was reported unhealthy: %s", result.Message)
}
for _, value := range []string{".", " . "} {
t.Setenv("LOCALAPPDATA", value)
result := windowsSandboxSetupCheck("windows", backend, workspace, config.SandboxConfig{})
if result == nil {
t.Fatalf("LOCALAPPDATA=%q: doctor reported a healthy sandbox while no command can select a runtime root", value)
}
if status, _ := result.Details["setupStatus"].(string); status != "runtime-root-unresolved" {
t.Errorf("LOCALAPPDATA=%q: setupStatus = %q, want runtime-root-unresolved: %s", value, status, result.Message)
}
if cause, _ := result.Details["error"].(string); !strings.Contains(cause, "user cache directory is unavailable") {
t.Errorf("LOCALAPPDATA=%q: the warning does not carry the resolver cause a command would report: %+v", value, result.Details)
}
}
}
+2 -2
View File
@@ -779,7 +779,7 @@ func normalizeDefinition(raw any, field string) (Definition, error) {
return Definition{}, err
}
if matcher != "" && !eventSupportsMatcher(event) {
return Definition{}, manifestError{fieldPath: field + ".matcher", message: "matcher is only supported for beforeTool and afterTool hooks."}
return Definition{}, manifestError{fieldPath: field + ".matcher", message: "matcher is only supported for beforeTool, afterTool, specialistStart, and specialistStop hooks."}
}
command, err := requiredString(obj, field+".command")
if err != nil {
@@ -843,7 +843,7 @@ func matchesHookMatcher(matcher string, toolName string) bool {
}
func eventSupportsMatcher(event Event) bool {
return event == EventBeforeTool || event == EventAfterTool
return event == EventBeforeTool || event == EventAfterTool || event == EventSpecialistStart || event == EventSpecialistStop
}
func toDiagnostic(err error, source ConfigSource, path string) Diagnostic {
+41 -1
View File
@@ -180,7 +180,7 @@ func TestLoadConfigPreservesUserDisabledStateWhenProjectOmitsEnabled(t *testing.
}
func TestLoadConfigRejectsMatchersOnLifecycleHooks(t *testing.T) {
for _, event := range []string{"sessionStart", "specialistStart"} {
for _, event := range []string{"sessionStart", "sessionEnd"} {
t.Run(event, func(t *testing.T) {
dir := t.TempDir()
projectConfigPath := filepath.Join(dir, "hooks.json")
@@ -210,6 +210,46 @@ func TestLoadConfigRejectsMatchersOnLifecycleHooks(t *testing.T) {
}
}
func TestLoadConfigAcceptsMatchersOnSpecialistHooks(t *testing.T) {
for _, tc := range []struct {
event Event
id string
}{
{event: EventSpecialistStart, id: "zero.explorer-start"},
{event: EventSpecialistStop, id: "zero.explorer-stop"},
} {
t.Run(string(tc.event), func(t *testing.T) {
dir := t.TempDir()
projectConfigPath := filepath.Join(dir, "hooks.json")
writeHookJSON(t, projectConfigPath, map[string]any{
"hooks": []any{map[string]any{
"id": tc.id,
"event": string(tc.event),
"matcher": "explorer",
"command": "node",
}},
})
result, err := LoadConfig(LoadOptions{
UserConfigPath: filepath.Join(dir, "missing-user-hooks.json"),
ProjectConfigPath: projectConfigPath,
})
if err != nil {
t.Fatalf("LoadConfig returned error: %v", err)
}
if len(result.Config.Hooks) != 1 || result.Config.Hooks[0].Matcher != "explorer" {
t.Fatalf("specialist matcher hook = %#v", result.Config.Hooks)
}
if got := hookIDs(Select(result.Config, SelectInput{Event: tc.event, ToolName: "explorer"})); !reflect.DeepEqual(got, []string{tc.id}) {
t.Fatalf("matched selection = %#v", got)
}
if got := Select(result.Config, SelectInput{Event: tc.event, ToolName: "worker"}); len(got) != 0 {
t.Fatalf("unmatched specialist should be empty, got %#v", got)
}
})
}
}
func TestConfigStorePersistsUpdates(t *testing.T) {
configPath := filepath.Join(t.TempDir(), "hooks.json")
store, err := NewConfigStore(StoreOptions{ConfigPath: configPath})
+119
View File
@@ -0,0 +1,119 @@
package mcp
import (
"fmt"
"net/http"
"net/url"
"strings"
"github.com/Twigpine/zero/internal/aimlapi"
"github.com/Twigpine/zero/internal/config"
)
// The AI/ML API MCP server authenticates with the user's AI/ML API key.
//
// Invariant: the key and the attribution headers are attached only to an
// outgoing request whose own URL is https://mcp.aimlapi.com on the default port
// with no userinfo (exact hostname, see aimlapi.IsAttributedURL), whatever the
// server is named. The key is resolved again for every request, redirect hops
// included (see reapplyAIMLAPIRedirectHeaders), through
// config.LoadAIMLAPIServiceKey, which in the same call decides whether it may
// be sent (no proxy profile, a single token), so a provider change applies to
// the next request and a withheld or invalid key means no Authorization
// header. It lives only in that request's headers: never in Server, its
// Identity (OAuth token binding), config.json, list output or errors (an
// error response to such a request is reported without its body). A
// user-configured Authorization header, OAuth, or a project-touched server
// always goes without it.
var loadAIMLAPIServiceKey = config.LoadAIMLAPIServiceKey
var aimlapiMCPHost = func() string {
parsed, err := url.Parse(aimlapi.MCPServerURL)
if err != nil {
panic(err)
}
return parsed.Hostname()
}()
func isAIMLAPIMCPURL(target *url.URL) bool {
return aimlapi.IsAttributedURL(target) && strings.EqualFold(target.Hostname(), aimlapiMCPHost)
}
// aimlapiAutomaticHeaders are the headers applyAIMLAPIRequestHeaders may add.
var aimlapiAutomaticHeaders = [...]string{"Authorization", aimlapi.SourceHeaderName, aimlapi.PartnerHeaderName}
// applyAIMLAPIRequestHeaders runs after the server's own headers are set.
func applyAIMLAPIRequestHeaders(server Server, request *http.Request) {
if request == nil || !isAIMLAPIMCPURL(request.URL) {
return
}
aimlapi.ApplyAttributionHeaders(request.Header, request.URL)
if request.Header.Get("Authorization") != "" || strings.EqualFold(server.Auth, ServerAuthOAuth) || server.ProjectConfigured {
return
}
key, err := loadAIMLAPIServiceKey()
if err != nil || key == "" {
return
}
request.Header.Set("Authorization", "Bearer "+key)
}
// reapplyAIMLAPIRedirectHeaders gives a redirect hop the automatic headers its
// own URL earns. net/http copies the first request's headers onto every hop
// without calling the header hooks, so without this a key or attribution added
// for the first URL would follow a same-origin hop that fails isAIMLAPIMCPURL
// (userinfo, say), and an eligible hop would reuse the first hop's key instead
// of resolving it again. The hop gets back the server's own values for these
// headers, then the same rule as any other request.
func reapplyAIMLAPIRedirectHeaders(server Server, request *http.Request, first *http.Request) {
if request == nil || first == nil || !isAIMLAPIMCPURL(first.URL) {
return
}
for _, name := range aimlapiAutomaticHeaders {
request.Header.Del(name)
}
for name, value := range server.Headers {
for _, automatic := range aimlapiAutomaticHeaders {
if strings.EqualFold(name, automatic) {
request.Header.Set(name, value)
}
}
}
applyAIMLAPIRequestHeaders(server, request)
}
// sentAIMLAPIServiceKey reports whether request went out carrying the key
// applyAIMLAPIRequestHeaders injects, as opposed to the server's own
// Authorization header or OAuth.
func sentAIMLAPIServiceKey(server Server, request *http.Request) bool {
if request == nil || !isAIMLAPIMCPURL(request.URL) || strings.EqualFold(server.Auth, ServerAuthOAuth) || server.ProjectConfigured {
return false
}
return request.Header.Get("Authorization") != "" && !hasUserAuthorizationHeader(server)
}
func hasUserAuthorizationHeader(server Server) bool {
for name, value := range server.Headers {
if strings.EqualFold(name, "Authorization") && value != "" {
return true
}
}
return false
}
// aimlapiUnauthorizedHint explains how to authenticate when the AI/ML API MCP
// server rejects a request without OAuth. It names no key material.
func aimlapiUnauthorizedHint(server Server, status int) string {
if status != http.StatusUnauthorized || strings.EqualFold(server.Auth, ServerAuthOAuth) {
return ""
}
target, err := url.Parse(server.URL)
if err != nil || !isAIMLAPIMCPURL(target) {
return ""
}
oauthHint := fmt.Sprintf("To sign in instead, set \"auth\": \"oauth\" on this server and run `zero mcp oauth login %s`", server.Name)
if server.ProjectConfigured || hasUserAuthorizationHeader(server) {
return " (" + oauthHint + ")"
}
return fmt.Sprintf(" (set up the aimlapi.com provider with `zero providers setup aimlapi` or set AIMLAPI_API_KEY; automatic injection is also disabled for proxy profiles, OAuth, project config, and non-empty Authorization headers — see docs/EXTENDING.md#built-in-servers. To sign in instead, set \"auth\": \"oauth\" on this server and run `zero mcp oauth login %s`)", server.Name)
}
+537
View File
@@ -0,0 +1,537 @@
package mcp
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"github.com/Twigpine/zero/internal/aimlapi"
"github.com/Twigpine/zero/internal/config"
)
// aimlapiRecorder stands in for mcp.aimlapi.com: it records every request it
// is handed (the URL stays https://mcp.aimlapi.com/mcp, nothing is dialed) and
// answers JSON-RPC, or 401 with the server's OAuth challenge when unauthorized.
type aimlapiRecorder struct {
mu sync.Mutex
headers []http.Header
urls []string
unauthorized bool
// redirectFirstTo, when set, answers the first request with a 307 to it.
redirectFirstTo string
// failStatus, when set, answers every request with it and failBody.
failStatus int
failBody string
}
func (recorder *aimlapiRecorder) RoundTrip(request *http.Request) (*http.Response, error) {
var body []byte
if request.Body != nil {
body, _ = io.ReadAll(request.Body)
_ = request.Body.Close()
}
recorder.mu.Lock()
recorder.headers = append(recorder.headers, request.Header.Clone())
recorder.urls = append(recorder.urls, request.URL.String())
unauthorized := recorder.unauthorized
redirect := recorder.redirectFirstTo != "" && len(recorder.headers) == 1
recorder.mu.Unlock()
if redirect {
response := fakeHTTPResponse(request, http.StatusTemporaryRedirect, "")
response.Header.Set("Location", recorder.redirectFirstTo)
return response, nil
}
if recorder.failStatus != 0 {
return fakeHTTPResponse(request, recorder.failStatus, recorder.failBody), nil
}
if unauthorized {
response := fakeHTTPResponse(request, http.StatusUnauthorized, `{"error":"invalid_token"}`)
response.Header.Set("WWW-Authenticate", `Bearer resource_metadata="https://mcp.aimlapi.com/.well-known/oauth-protected-resource/mcp"`)
return response, nil
}
var message struct {
ID json.RawMessage `json:"id"`
Method string `json:"method"`
}
_ = json.Unmarshal(body, &message)
if len(message.ID) == 0 {
return fakeHTTPResponse(request, http.StatusAccepted, ""), nil
}
result := `{}`
switch message.Method {
case "initialize":
result = `{"protocolVersion":"2024-11-05","capabilities":{},"serverInfo":{"name":"fake","version":"1"}}`
case "tools/list":
result = `{"tools":[{"name":"tools_search","inputSchema":{"type":"object"}}]}`
}
return fakeHTTPResponse(request, http.StatusOK, fmt.Sprintf(`{"jsonrpc":"2.0","id":%s,"result":%s}`, message.ID, result)), nil
}
func (recorder *aimlapiRecorder) recorded() []http.Header {
recorder.mu.Lock()
defer recorder.mu.Unlock()
return append([]http.Header(nil), recorder.headers...)
}
func fakeHTTPResponse(request *http.Request, status int, body string) *http.Response {
header := http.Header{}
if body != "" {
header.Set("Content-Type", "application/json")
}
return &http.Response{
StatusCode: status,
Header: header,
Body: io.NopCloser(bytes.NewBufferString(body)),
Request: request,
}
}
// isolateAIMLAPIKeyLookup points the real key resolver at an empty temporary
// home and returns the user config path it reads.
func isolateAIMLAPIKeyLookup(t *testing.T, envKey string) string {
t.Helper()
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
t.Setenv("ZERO_CRED_STORAGE", "file")
t.Setenv("ZERO_PROVIDER", "")
t.Setenv("AIMLAPI_INFERENCE_URL", "")
t.Setenv("AIMLAPI_PARTNER_ID", "")
t.Setenv("AIMLAPI_API_KEY", envKey)
path, err := config.DefaultUserConfigPath()
if err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
t.Fatal(err)
}
return path
}
func writeAIMLAPIProviderConfig(t *testing.T, path string, baseURL string, apiKey string) {
t.Helper()
profile := map[string]any{"name": "aimlapi", "catalogID": "aimlapi", "model": "m", "apiKey": apiKey}
if baseURL != "" {
profile["baseURL"] = baseURL
}
data, err := json.Marshal(map[string]any{"activeProvider": "aimlapi", "providers": []any{profile}})
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, data, 0o600); err != nil {
t.Fatal(err)
}
}
func newRecordedNetworkClient(server Server, recorder *aimlapiRecorder) *networkClient {
return &networkClient{server: server, client: mcpHTTPClient(server, recorder), nextID: 1}
}
func aimlapiTestServer(name string, rawURL string) Server {
return Server{Name: name, Type: ServerTypeHTTP, URL: rawURL}
}
func assertAIMLAPIAttribution(t *testing.T, header http.Header) {
t.Helper()
if got := header.Get(aimlapi.SourceHeaderName); got != aimlapi.SourceValue {
t.Fatalf("%s = %q, want %q", aimlapi.SourceHeaderName, got, aimlapi.SourceValue)
}
if got, want := header.Get(aimlapi.PartnerHeaderName), aimlapi.ResolvePartnerID(""); got == "" || got != want {
t.Fatalf("%s = %q, want %q", aimlapi.PartnerHeaderName, got, want)
}
}
func TestAIMLAPIMCPRequestsCarryServiceKeyAndAttribution(t *testing.T) {
isolateAIMLAPIKeyLookup(t, "aiml-secret")
// A hand-added server on the exact host is treated like the built-in one.
for _, name := range []string{"aimlapi", "media"} {
recorder := &aimlapiRecorder{}
client := newRecordedNetworkClient(aimlapiTestServer(name, aimlapi.MCPServerURL), recorder)
if err := client.initialize(context.Background()); err != nil {
t.Fatalf("%s: initialize: %v", name, err)
}
if _, err := client.ListTools(context.Background()); err != nil {
t.Fatalf("%s: ListTools: %v", name, err)
}
requests := recorder.recorded()
if len(requests) != 3 {
t.Fatalf("%s: recorded %d requests, want initialize, initialized and tools/list", name, len(requests))
}
for _, header := range requests {
if got := header.Get("Authorization"); got != "Bearer aiml-secret" {
t.Fatalf("%s: Authorization = %q, want the AI/ML API key", name, got)
}
assertAIMLAPIAttribution(t, header)
}
}
}
func TestAIMLAPIMCPHeadersOnlyForExactHost(t *testing.T) {
isolateAIMLAPIKeyLookup(t, "aiml-secret")
for _, rawURL := range []string{
"https://mcp.aimlapi.com.example.net/mcp",
"http://mcp.aimlapi.com/mcp",
"https://mcp.aimlapi.com:8443/mcp",
"https://[email protected]/mcp",
"https://api.aimlapi.com/mcp",
"https://evilmcp.aimlapi.com/mcp",
"https://mcp.exa.ai/mcp",
} {
recorder := &aimlapiRecorder{}
server := aimlapiTestServer("other", rawURL)
server.Headers = map[string]string{"X-Custom": "kept"}
client := newRecordedNetworkClient(server, recorder)
if err := client.initialize(context.Background()); err != nil {
t.Fatalf("%s: initialize: %v", rawURL, err)
}
for _, header := range recorder.recorded() {
if strings.Contains(rawURL, "user@") && strings.HasPrefix(header.Get("Authorization"), "Basic ") {
// net/http turns URL userinfo into Basic auth on its own.
header.Del("Authorization")
}
// Byte-for-byte what any other MCP server gets.
want := http.Header{
"X-Custom": {"kept"},
"Content-Type": {"application/json"},
"Accept": {"application/json, text/event-stream"},
}
if fmt.Sprint(header) != fmt.Sprint(want) {
t.Fatalf("%s: headers = %v, want %v", rawURL, header, want)
}
}
}
// Port 443 spelled out and a different host case are still the exact host.
for _, rawURL := range []string{"https://mcp.aimlapi.com:443/mcp", "https://MCP.AIMLAPI.COM/mcp"} {
recorder := &aimlapiRecorder{}
client := newRecordedNetworkClient(aimlapiTestServer("aimlapi", rawURL), recorder)
if err := client.initialize(context.Background()); err != nil {
t.Fatalf("%s: initialize: %v", rawURL, err)
}
header := recorder.recorded()[0]
if header.Get("Authorization") != "Bearer aiml-secret" {
t.Fatalf("%s: Authorization = %q, want the key", rawURL, header.Get("Authorization"))
}
assertAIMLAPIAttribution(t, header)
}
}
// The key and its eligibility are resolved for every request, so switching the
// aimlapi profile to a proxy keeps the proxy's key off mcp.aimlapi.com on the
// very next request of an already-connected client and on new connections.
func TestAIMLAPIMCPKeyIsResolvedPerRequestAcrossProviderSwitch(t *testing.T) {
path := isolateAIMLAPIKeyLookup(t, "")
writeAIMLAPIProviderConfig(t, path, "", "canonical-key")
recorder := &aimlapiRecorder{}
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
client := newRecordedNetworkClient(server, recorder)
if err := client.initialize(context.Background()); err != nil {
t.Fatalf("initialize: %v", err)
}
if got := recorder.recorded()[0].Get("Authorization"); got != "Bearer canonical-key" {
t.Fatalf("canonical profile: Authorization = %q, want Bearer canonical-key", got)
}
writeAIMLAPIProviderConfig(t, path, "https://proxy.example/v1", "proxy-key")
if _, err := client.ListTools(context.Background()); err != nil {
t.Fatalf("ListTools after switch: %v", err)
}
fresh := newRecordedNetworkClient(server, recorder)
if err := fresh.initialize(context.Background()); err != nil {
t.Fatalf("initialize after switch: %v", err)
}
requests := recorder.recorded()
for _, header := range requests[2:] {
if got := header.Get("Authorization"); got != "" {
t.Fatalf("proxy profile: Authorization = %q, want none", got)
}
assertAIMLAPIAttribution(t, header)
}
for _, header := range requests {
if strings.Contains(fmt.Sprint(header), "proxy-key") {
t.Fatalf("proxy key reached mcp.aimlapi.com: %v", header)
}
}
}
func TestAIMLAPIMCPUnusableKeySendsNoAuthorization(t *testing.T) {
cases := map[string]string{
"multi-line key pool": "first-key\nsecond-key",
"comma key pool": "first-key,second-key",
"no key": "",
}
for name, apiKey := range cases {
t.Run(name, func(t *testing.T) {
path := isolateAIMLAPIKeyLookup(t, "")
writeAIMLAPIProviderConfig(t, path, "", apiKey)
recorder := &aimlapiRecorder{unauthorized: true}
client := newRecordedNetworkClient(aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), recorder)
err := client.initialize(context.Background())
if err == nil {
t.Fatal("initialize succeeded against a 401")
}
requests := recorder.recorded()
if len(requests) != 1 {
t.Fatalf("recorded %d requests, want one intact request", len(requests))
}
header := requests[0]
if got := header.Get("Authorization"); got != "" {
t.Fatalf("Authorization = %q, want none", got)
}
if header.Get("Content-Type") != "application/json" {
t.Fatalf("request headers broken: %v", header)
}
assertAIMLAPIAttribution(t, header)
if !strings.Contains(err.Error(), "returned HTTP 401") || !strings.Contains(err.Error(), "zero mcp oauth login aimlapi") || !strings.Contains(err.Error(), "AIMLAPI_API_KEY") {
t.Fatalf("401 error lacks the authentication hint: %v", err)
}
if strings.Contains(err.Error(), "first-key") {
t.Fatalf("error leaks key material: %v", err)
}
})
}
}
func TestAIMLAPIMCPUnauthorizedHintOnlyForAIMLAPIWithoutOAuth(t *testing.T) {
if hint := aimlapiUnauthorizedHint(aimlapiTestServer("docs", "https://mcp.example.com/mcp"), http.StatusUnauthorized); hint != "" {
t.Fatalf("hint for another server = %q", hint)
}
oauth := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
oauth.Auth = ServerAuthOAuth
if hint := aimlapiUnauthorizedHint(oauth, http.StatusUnauthorized); hint != "" {
t.Fatalf("hint for an OAuth server = %q", hint)
}
if hint := aimlapiUnauthorizedHint(aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), http.StatusForbidden); hint != "" {
t.Fatalf("hint for 403 = %q", hint)
}
}
func TestAIMLAPIMCPKeyDefersToUserAuthorizationOAuthAndProjectConfig(t *testing.T) {
isolateAIMLAPIKeyLookup(t, "aiml-secret")
userHeader := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
userHeader.Headers = map[string]string{"authorization": "Bearer user-token"}
oauth := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
oauth.Auth = ServerAuthOAuth
project := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
project.ProjectConfigured = true
cases := []struct {
name string
server Server
want string
}{
{"user Authorization header", userHeader, "Bearer user-token"},
{"OAuth server", oauth, ""},
{"project-configured server", project, ""},
}
for _, tc := range cases {
recorder := &aimlapiRecorder{}
client := newRecordedNetworkClient(tc.server, recorder)
if err := client.initialize(context.Background()); err != nil {
t.Fatalf("%s: initialize: %v", tc.name, err)
}
header := recorder.recorded()[0]
if got := header.Get("Authorization"); got != tc.want {
t.Fatalf("%s: Authorization = %q, want %q", tc.name, got, tc.want)
}
assertAIMLAPIAttribution(t, header)
}
}
func TestAIMLAPIMCPSSERequestsCarryKeyAndAttribution(t *testing.T) {
isolateAIMLAPIKeyLookup(t, "aiml-secret")
server := Server{Name: "aimlapi", Type: ServerTypeSSE, URL: aimlapi.MCPServerURL}
client := &remoteSSEClient{server: server}
apply := map[string]func(*http.Request){
"stream": client.applyStreamHeaders,
"post": client.applyPostHeaders,
}
for name, applyHeaders := range apply {
request, err := http.NewRequest(http.MethodPost, aimlapi.MCPServerURL, nil)
if err != nil {
t.Fatal(err)
}
applyHeaders(request)
if got := request.Header.Get("Authorization"); got != "Bearer aiml-secret" {
t.Fatalf("%s: Authorization = %q, want the key", name, got)
}
assertAIMLAPIAttribution(t, request.Header)
other, err := http.NewRequest(http.MethodPost, "https://mcp.aimlapi.com.example.net/mcp", nil)
if err != nil {
t.Fatal(err)
}
applyHeaders(other)
if other.Header.Get("Authorization") != "" || other.Header.Get(aimlapi.SourceHeaderName) != "" || other.Header.Get(aimlapi.PartnerHeaderName) != "" {
t.Fatalf("%s: lookalike host got AI/ML API headers: %v", name, other.Header)
}
}
}
func TestAIMLAPIMCPKeyStaysOutOfServerConfigAndIdentity(t *testing.T) {
enabled := config.MCPConfig{Servers: map[string]config.MCPServerConfig{
"aimlapi": {Type: "http", URL: aimlapi.MCPServerURL},
}}
isolateAIMLAPIKeyLookup(t, "")
withoutKey, err := NormalizeConfig(enabled)
if err != nil {
t.Fatal(err)
}
isolateAIMLAPIKeyLookup(t, "aiml-secret")
withKey, err := NormalizeConfig(enabled)
if err != nil {
t.Fatal(err)
}
if len(withKey) != 1 || withKey[0].Identity != withoutKey[0].Identity {
t.Fatalf("server identity depends on the key: %#v vs %#v", withKey, withoutKey)
}
if strings.Contains(fmt.Sprintf("%#v", withKey), "aiml-secret") {
t.Fatalf("normalized server carries the key: %#v", withKey)
}
// The shipped default is off, so it is not even normalized for a session.
defaults, err := NormalizeConfig(config.MCPConfig{Servers: config.DefaultMCPServers()})
if err != nil {
t.Fatal(err)
}
for _, server := range defaults {
if server.Name == "aimlapi" {
t.Fatalf("disabled aimlapi default was normalized for connection: %#v", server)
}
}
}
func TestAIMLAPIMCPUnauthorizedHintRespectsCredentialEligibility(t *testing.T) {
for _, tc := range []struct {
name string
headers map[string]string
project bool
wantSetup bool
}{
{"default", nil, false, true},
{"project", nil, true, false},
{"custom bearer", map[string]string{"Authorization": "Bearer manual"}, false, false},
{"mixed case header", map[string]string{"aUtHoRiZaTiOn": "Bearer manual"}, false, false},
{"empty header", map[string]string{"authorization": ""}, false, true},
} {
t.Run(tc.name, func(t *testing.T) {
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
server.Headers = tc.headers
server.ProjectConfigured = tc.project
hint := aimlapiUnauthorizedHint(server, http.StatusUnauthorized)
if strings.Contains(hint, "zero providers setup aimlapi") != tc.wantSetup || !strings.Contains(hint, "zero mcp oauth login aimlapi") {
t.Fatalf("incorrect hint: %s", hint)
}
})
}
}
// net/http copies the first request's headers onto every redirect hop without
// calling the header hooks, so the redirect guard has to apply the per-request
// rule again: a same-origin hop that fails the exact-URL rule (userinfo) gets
// neither the key nor attribution, an eligible hop gets a freshly resolved key,
// and a user's own Authorization header still follows as before.
func TestAIMLAPIMCPRedirectHopsReapplyAutomaticHeaders(t *testing.T) {
isolateAIMLAPIKeyLookup(t, "")
userHeader := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
userHeader.Headers = map[string]string{"authorization": "Bearer user-token"}
for _, tc := range []struct {
name string
server Server
location string
wantFirst string
wantHop string
wantAttribution bool
}{
{"userinfo hop", aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), "https://[email protected]/mcp", "Bearer aiml-first", "", false},
{"eligible hop", aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), "https://mcp.aimlapi.com/mcp/", "Bearer aiml-first", "Bearer aiml-second", true},
{"user header hop", userHeader, "https://[email protected]/mcp", "Bearer user-token", "Bearer user-token", false},
} {
t.Run(tc.name, func(t *testing.T) {
keys := []string{"aiml-first", "aiml-second"}
previous := loadAIMLAPIServiceKey
loadAIMLAPIServiceKey = func() (string, error) {
key := keys[0]
if len(keys) > 1 {
keys = keys[1:]
}
return key, nil
}
t.Cleanup(func() { loadAIMLAPIServiceKey = previous })
recorder := &aimlapiRecorder{redirectFirstTo: tc.location}
client := newRecordedNetworkClient(tc.server, recorder)
if err := client.initialize(context.Background()); err != nil {
t.Fatalf("initialize: %v", err)
}
requests := recorder.recorded()
if len(requests) < 2 || recorder.urls[1] != tc.location {
t.Fatalf("redirect was not followed: %v", recorder.urls)
}
if got := requests[0].Get("Authorization"); got != tc.wantFirst {
t.Fatalf("first hop Authorization = %q, want %q", got, tc.wantFirst)
}
hop := requests[1]
// With no key, net/http fills Basic auth from the hop's own userinfo;
// only the automatic key must be absent.
if got := hop.Get("Authorization"); (tc.wantHop == "" && strings.Contains(got, "aiml-")) || (tc.wantHop != "" && got != tc.wantHop) {
t.Fatalf("redirect hop Authorization = %q, want %q", got, tc.wantHop)
}
if tc.wantAttribution {
assertAIMLAPIAttribution(t, hop)
} else if hop.Get(aimlapi.SourceHeaderName) != "" || hop.Get(aimlapi.PartnerHeaderName) != "" {
t.Fatalf("redirect hop kept attribution headers: %v", hop)
}
})
}
}
// An error body can echo the key Zero injected; the body is left out of the
// error rather than matched, so a key cut by the 1024-byte read limit cannot
// leak its prefix either. Requests without the automatic key keep the detail.
func TestAIMLAPIMCPStatusErrorOmitsBodyWhenKeyWasInjected(t *testing.T) {
const key = "sk-echo-0123456789abcdef"
isolateAIMLAPIKeyLookup(t, key)
for _, tc := range []struct {
name string
body string
}{
{"echoed key", `{"error":"invalid credential ` + key + `"}`},
{"key across the read limit", strings.Repeat("x", 1020) + key},
} {
t.Run(tc.name, func(t *testing.T) {
recorder := &aimlapiRecorder{failStatus: http.StatusPaymentRequired, failBody: tc.body}
err := newRecordedNetworkClient(aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), recorder).initialize(context.Background())
if err == nil {
t.Fatal("initialize succeeded against a failing server")
}
if got := recorder.recorded()[0].Get("Authorization"); got != "Bearer "+key {
t.Fatalf("request did not carry the injected key: %q", got)
}
if strings.Contains(err.Error(), key[:4]) || !strings.Contains(err.Error(), "HTTP 402") {
t.Fatalf("error = %q, want HTTP 402 without the echoed key", err)
}
})
}
userHeader := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
userHeader.Headers = map[string]string{"Authorization": "Bearer user-token"}
recorder := &aimlapiRecorder{failStatus: http.StatusPaymentRequired, failBody: `{"error":"insufficient balance"}`}
err := newRecordedNetworkClient(userHeader, recorder).initialize(context.Background())
if err == nil || !strings.Contains(err.Error(), "insufficient balance") {
t.Fatalf("error without the automatic key = %v, want the response detail", err)
}
}
+11 -2
View File
@@ -427,6 +427,7 @@ func (client *networkClient) applyHeaders(request *http.Request) {
if client.sessionID != "" {
request.Header.Set("Mcp-Session-Id", client.sessionID)
}
applyAIMLAPIRequestHeaders(client.server, request)
}
func (client *remoteSSEClient) applyStreamHeaders(request *http.Request) {
@@ -434,6 +435,7 @@ func (client *remoteSSEClient) applyStreamHeaders(request *http.Request) {
request.Header.Set(key, value)
}
request.Header.Set("Accept", "text/event-stream")
applyAIMLAPIRequestHeaders(client.server, request)
}
func (client *remoteSSEClient) applyPostHeaders(request *http.Request) {
@@ -442,6 +444,7 @@ func (client *remoteSSEClient) applyPostHeaders(request *http.Request) {
}
request.Header.Set("Content-Type", "application/json")
request.Header.Set("Accept", "application/json")
applyAIMLAPIRequestHeaders(client.server, request)
}
func closeResponseBody(errp *error, server Server, body io.Closer) {
@@ -607,12 +610,18 @@ func (client *remoteSSEClient) failPending(err error) {
}
func httpStatusError(server Server, response *http.Response) error {
hint := aimlapiUnauthorizedHint(server, response.StatusCode)
if sentAIMLAPIServiceKey(server, response.Request) {
// The body is server text that may echo the key Zero injected, whole or
// cut by the read limit, so it is left out rather than matched.
return fmt.Errorf("MCP %s server %s returned HTTP %d (response body omitted: the request carried your AI/ML API key)%s", server.Type, server.Name, response.StatusCode, hint)
}
body, _ := io.ReadAll(io.LimitReader(response.Body, 1024))
detail := strings.TrimSpace(string(body))
if detail == "" {
return fmt.Errorf("MCP %s server %s returned HTTP %d", server.Type, server.Name, response.StatusCode)
return fmt.Errorf("MCP %s server %s returned HTTP %d%s", server.Type, server.Name, response.StatusCode, hint)
}
return fmt.Errorf("MCP %s server %s returned HTTP %d: %s", server.Type, server.Name, response.StatusCode, detail)
return fmt.Errorf("MCP %s server %s returned HTTP %d: %s%s", server.Type, server.Name, response.StatusCode, detail, hint)
}
func decodeSSERPCMessage(reader io.Reader) (rpcMessage, error) {
+1
View File
@@ -28,6 +28,7 @@ func checkMCPRedirect(server Server) func(*http.Request, []*http.Request) error
if !sameMCPOrigin(via[0].URL, request.URL) {
return fmt.Errorf("MCP %s server %s refused cross-origin redirect to %s", server.Type, server.Name, mcpOrigin(request.URL))
}
reapplyAIMLAPIRedirectHeaders(server, request, via[0])
return nil
}
}
+1 -1
View File
@@ -374,7 +374,7 @@ func (tool skillTool) Run(_ context.Context, args map[string]any) tools.Result {
available := make([]string, 0, len(merged))
for _, skill := range merged {
if skill.Name == name {
return tools.Result{Status: tools.StatusOK, Output: skill.Content}
return tools.Result{Status: tools.StatusOK, Output: skills.ToolOutput(skill)}
}
available = append(available, skill.Name)
}
+25
View File
@@ -673,3 +673,28 @@ func TestExpandPluginRootPathRelative(t *testing.T) {
}
}
}
// The multi-root skill tool is the one the CLI registers, so it has to point at
// a skill's bundled files the same way the core tool does (#584).
func TestNewSkillToolPointsAtTheSkillsBundledFiles(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
dir := t.TempDir()
writeTestSkill(t, dir, "pdf", "Run scripts/fill.py.")
script := filepath.Join(dir, "pdf", "scripts", "fill.py")
if err := os.MkdirAll(filepath.Dir(script), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(script, []byte("print('fill')\n"), 0o755); err != nil {
t.Fatal(err)
}
got := NewSkillTool(dir, nil).Run(context.Background(), map[string]any{"name": "pdf"})
if got.Status != tools.StatusOK {
t.Fatalf("Status = %s (output: %s)", got.Status, got.Output)
}
if !strings.Contains(got.Output, "- scripts/fill.py") || !strings.Contains(got.Output, filepath.Join(dir, "pdf")) {
t.Fatalf("output does not point at the bundled script:\n%s", got.Output)
}
}
+552
View File
@@ -0,0 +1,552 @@
package plugins
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
"github.com/Twigpine/zero/internal/installtxn"
"github.com/Twigpine/zero/internal/pathjail"
)
// SetEnabledResult describes a successful enable/disable of one plugin manifest.
type SetEnabledResult struct {
ID string `json:"id"`
Enabled bool `json:"enabled"`
Changed bool `json:"changed"`
Source Source `json:"source"`
ManifestPath string `json:"manifestPath"`
}
// maxEnabledRelockAttempts bounds how many times SetEnabledByID re-resolves the
// plugin after it moves to another install root while waiting for the lock.
const maxEnabledRelockAttempts = 4
// FindByID returns the loaded plugin with the given id after normal discovery
// and precedence (project overrides user). A missing id returns ok=false.
func FindByID(options LoadOptions, id string) (LoadedPlugin, bool, error) {
id = strings.TrimSpace(id)
if id == "" {
return LoadedPlugin{}, false, errors.New("plugin id is required")
}
result, err := Load(options)
if err != nil {
return LoadedPlugin{}, false, err
}
for _, plugin := range result.Plugins {
if plugin.ID == id {
return plugin, true, nil
}
}
return LoadedPlugin{}, false, nil
}
// SetEnabledByID finds a plugin by id (respecting LoadOptions) and toggles the
// top-level "enabled" field of its plugin.json.
//
// The read-modify-write is serialized with plugin installs and removals: it
// takes the same cross-process installtxn lock they take on the plugin's
// install root, runs the same interrupted-transaction recovery they run under
// it, re-resolves the plugin under that lock, and publishes the
// manifest with an atomic rename while still holding it, so a concurrent
// install can never be interleaved between the read and the write. Only the
// bytes of the "enabled" value change, so field order, formatting, and numeric
// literals in the rest of the manifest survive the toggle.
func SetEnabledByID(options LoadOptions, id string, enabled bool) (SetEnabledResult, error) {
for attempt := 0; ; attempt++ {
plugin, ok, err := FindByID(options, id)
if err != nil {
return SetEnabledResult{}, err
}
if !ok {
return SetEnabledResult{}, fmt.Errorf("%w: %q", ErrNotInstalled, id)
}
// Take the install root lock before the final lookup and read so an
// install or remove cannot swap the tree between them.
unlock, err := installtxn.Lock(plugin.Root)
if err != nil {
return SetEnabledResult{}, err
}
// Put back anything an earlier install or removal was killed mid-commit,
// and abort if a transaction could not be resolved: editing a tree that is
// still owed a restore loses the edit when the next install or removal
// puts the backup back over it. See installtxn.Recover.
if err := installtxn.Recover(plugin.Root, reconcileInterrupted(plugin.Root)); err != nil {
unlock()
return SetEnabledResult{}, err
}
final, ok, err := FindByID(options, id)
if err != nil {
unlock()
return SetEnabledResult{}, err
}
if !ok {
unlock()
return SetEnabledResult{}, fmt.Errorf("%w: %q", ErrNotInstalled, id)
}
if final.Root != plugin.Root {
// Precedence moved to another root while this lock was pending;
// retry against the root that wins now so the edit still serializes
// with that root's installs.
unlock()
if attempt >= maxEnabledRelockAttempts {
return SetEnabledResult{}, fmt.Errorf("plugin %q kept changing roots while waiting for the install lock", id)
}
continue
}
changed, err := setEnabledLocked(final, enabled)
unlock()
if err != nil {
return SetEnabledResult{}, err
}
return SetEnabledResult{
ID: final.ID,
Enabled: enabled,
Changed: changed,
Source: final.Source,
ManifestPath: final.ManifestPath,
}, nil
}
}
// setEnabledLocked performs the manifest read-modify-write while the caller
// holds the installtxn lock for the plugin's install root.
//
// The lock only serializes cooperating installers; it does not stop the plugin
// directory from being swapped after discovery found it. So the whole update
// works on one directory, pinned by openPluginDir: the manifest is read and
// published, and the tree hashed for the lockfile, through that handle, never
// by re-resolving the path.
func setEnabledLocked(plugin LoadedPlugin, enabled bool) (bool, error) {
pinned, err := openPluginDir(plugin)
if err != nil {
return false, err
}
defer pinned.Close()
return setEnabledPinned(plugin, pinned, enabled)
}
// setEnabledPinned is setEnabledLocked on an already pinned plugin directory.
//
// The edit lands in the pinned directory even if the path now names another
// one, so before refreshing the lockfile or reporting any result it confirms
// that the directory entry still is the pinned directory. Otherwise the
// command would report a state that the installed plugin does not have.
func setEnabledPinned(plugin LoadedPlugin, pinned *pinnedPluginDir, enabled bool) (bool, error) {
manifestName := filepath.Base(plugin.ManifestPath)
raw, perm, err := readManifest(pinned.dir, manifestName)
if err != nil {
return false, err
}
data, changed, err := toggleEnabledManifest(raw, enabled)
if err != nil {
return false, err
}
if !changed {
if err := pinned.verifyLive(); err != nil {
return false, err
}
return false, nil
}
// Read the lockfile before the edit so the refresh below only runs when the
// installed tree still matched its recorded hash; a toggle must never mask
// pre-existing drift.
lock, entry, hashClean := cleanLockEntry(plugin, pinned.dir)
if err := publishManifest(pinned.dir, manifestName, data, perm); err != nil {
return false, err
}
if !hashClean {
if err := pinned.verifyLive(); err != nil {
return false, err
}
return true, nil
}
after, err := hashRootedTree(pinned.dir)
if err != nil {
return true, fmt.Errorf("plugin manifest updated but %s hash refresh failed: %w", LockFileName, err)
}
if err := pinned.verifyLive(); err != nil {
return false, err
}
entry.Hash = after
lock[plugin.ID] = entry
if err := writeLock(plugin.Root, lock); err != nil {
return true, fmt.Errorf("plugin manifest updated but %s hash refresh failed: %w", LockFileName, err)
}
return true, nil
}
// errPluginDirReplaced reports that the plugin's directory entry no longer
// names the directory the toggle pinned and edited.
var errPluginDirReplaced = errors.New("plugin directory was replaced while it was being updated; the plugin installed there now was not changed")
// pinnedPluginDir is the plugin directory discovery found, held open for the
// length of one toggle.
type pinnedPluginDir struct {
root *os.Root // the plugin's install root
name string // the plugin directory's entry name in root
path string // the plugin directory's path, for messages
dir *os.Root // the plugin directory itself
}
// Close releases both handles.
func (p *pinnedPluginDir) Close() {
_ = p.dir.Close()
_ = p.root.Close()
}
// verifyLive confirms that the plugin's entry in its install root is still a
// real directory, not a symlink, junction, or other reparse point, and is the
// very directory p.dir holds open.
func (p *pinnedPluginDir) verifyLive() error {
opened, err := p.dir.Stat(".")
if err != nil {
return fmt.Errorf("inspect plugin directory: %w", err)
}
entry, err := p.root.Lstat(p.name)
if errors.Is(err, fs.ErrNotExist) {
return fmt.Errorf("%w: %s", errPluginDirReplaced, p.path)
}
if err != nil {
return fmt.Errorf("inspect plugin directory: %w", err)
}
if entry.Mode()&(os.ModeSymlink|os.ModeIrregular) != 0 {
return fmt.Errorf("%w: plugin directory %s", pathjail.ErrReparse, p.path)
}
if !os.SameFile(entry, opened) {
return fmt.Errorf("%w: %s", errPluginDirReplaced, p.path)
}
return nil
}
// openPluginDir pins the plugin directory discovery found, confined to the
// plugin's install root.
//
// A link swapped in before the open is refused here. One swapped in after it
// cannot redirect anything, because every later read, write, and hash is
// relative to the returned handle rather than to a path; verifyLive then
// detects that the path no longer names the edited directory.
func openPluginDir(plugin LoadedPlugin) (*pinnedPluginDir, error) {
root, name, err := pathjail.Open(plugin.Root, plugin.PluginDir)
if err != nil {
return nil, fmt.Errorf("open plugin directory: %w", err)
}
if name == "." || filepath.Base(name) != name {
_ = root.Close()
return nil, fmt.Errorf("%w: plugin directory %s is not an entry of %s", pathjail.ErrEscapes, plugin.PluginDir, plugin.Root)
}
if err := pathjail.RefuseReparse(root, name); err != nil {
_ = root.Close()
return nil, fmt.Errorf("open plugin directory: %w", err)
}
dir, err := root.OpenRoot(name)
if err != nil {
_ = root.Close()
return nil, fmt.Errorf("open plugin directory: %w", err)
}
pinned := &pinnedPluginDir{root: root, name: name, path: plugin.PluginDir, dir: dir}
if err := pinned.verifyLive(); err != nil {
pinned.Close()
return nil, err
}
return pinned, nil
}
// readManifest reads name from dir and returns its bytes and permission bits.
// Failing on a stat error keeps the publish from widening the permissions.
func readManifest(dir *os.Root, name string) ([]byte, os.FileMode, error) {
file, err := dir.Open(name)
if err != nil {
return nil, 0, fmt.Errorf("read plugin manifest: %w", err)
}
defer file.Close()
info, err := file.Stat()
if err != nil {
return nil, 0, fmt.Errorf("inspect plugin manifest: %w", err)
}
raw, err := io.ReadAll(file)
if err != nil {
return nil, 0, fmt.Errorf("read plugin manifest: %w", err)
}
return raw, info.Mode().Perm(), nil
}
// publishManifest replaces name in dir with data by renaming a complete
// temporary sibling over it, so a concurrent reader sees either the old
// manifest or the new one, never a partial write.
func publishManifest(dir *os.Root, name string, data []byte, perm os.FileMode) error {
temp, tempName, err := pathjail.CreateTemp(dir, ".", ".zero-manifest", "")
if err != nil {
return fmt.Errorf("write plugin manifest: %w", err)
}
defer func() { _ = dir.Remove(tempName) }()
if err := temp.Chmod(perm); err != nil {
_ = temp.Close()
return fmt.Errorf("write plugin manifest: %w", err)
}
if _, err := temp.Write(data); err != nil {
_ = temp.Close()
return fmt.Errorf("write plugin manifest: %w", err)
}
if err := temp.Sync(); err != nil {
_ = temp.Close()
return fmt.Errorf("write plugin manifest: %w", err)
}
if err := temp.Close(); err != nil {
return fmt.Errorf("write plugin manifest: %w", err)
}
if err := dir.Rename(tempName, name); err != nil {
return fmt.Errorf("write plugin manifest: %w", err)
}
return nil
}
// cleanLockEntry reports whether the pinned plugin directory dir still matches
// the lockfile entry recorded for the plugin and returns both for a refresh.
func cleanLockEntry(plugin LoadedPlugin, dir *os.Root) (map[string]LockEntry, LockEntry, bool) {
lock, err := ReadLock(plugin.Root)
if err != nil {
return nil, LockEntry{}, false
}
entry, ok := lock[plugin.ID]
if !ok {
return nil, LockEntry{}, false
}
current, err := hashRootedTree(dir)
if err != nil || current != entry.Hash {
return nil, LockEntry{}, false
}
return lock, entry, true
}
// toggleEnabledManifest returns raw with the top-level "enabled" member set to
// enabled. A missing or null value counts as enabled, matching the loader's
// optionalBool fallback. changed reports whether the result differs from raw.
func toggleEnabledManifest(raw []byte, enabled bool) (out []byte, changed bool, err error) {
dec := json.NewDecoder(bytes.NewReader(raw))
openTok, err := dec.Token()
if err != nil {
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
}
if delim, ok := openTok.(json.Delim); !ok || delim != '{' {
return nil, false, errors.New("plugin manifest must be a JSON object")
}
openEnd := dec.InputOffset()
current := true
lastInvalid := false
var valueStart, valueEnd int64 = -1, -1
for dec.More() {
keyTok, err := dec.Token()
if err != nil {
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
}
key, ok := keyTok.(string)
if !ok {
return nil, false, errors.New("plugin manifest must be a JSON object")
}
keyEnd := dec.InputOffset()
if key != "enabled" {
if err := skipManifestValue(dec); err != nil {
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
}
continue
}
valueTok, err := dec.Token()
if err != nil {
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
}
var literal string
switch value := valueTok.(type) {
case bool:
current = value
literal = "false"
if value {
literal = "true"
}
case nil:
current = true
literal = "null"
default:
// Only the last occurrence of a duplicate key counts, as in the
// loader's json.Unmarshal, so an earlier non-boolean value that a
// later one overwrites is not an error. Remember it and decide after
// the last member.
if err := skipManifestComposite(dec, valueTok); err != nil {
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
}
lastInvalid = true
continue
}
lastInvalid = false
start, err := enabledValueOffset(raw, keyEnd)
if err != nil {
return nil, false, err
}
if start+int64(len(literal)) > int64(len(raw)) ||
!bytes.Equal(raw[start:start+int64(len(literal))], []byte(literal)) {
return nil, false, errors.New("parse plugin manifest: cannot locate the enabled value")
}
// A duplicate key keeps the last occurrence, matching json.Unmarshal.
valueStart, valueEnd = start, start+int64(len(literal))
}
closingTok, err := dec.Token()
if err != nil {
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
}
if delim, ok := closingTok.(json.Delim); !ok || delim != '}' {
return nil, false, errors.New("plugin manifest must be a JSON object")
}
closingEnd := dec.InputOffset()
if _, err := dec.Token(); err != io.EOF {
return nil, false, errors.New("parse plugin manifest: unexpected data after the top-level object")
}
if lastInvalid {
return nil, false, ManifestError{FieldPath: "enabled", Message: "Expected a boolean."}
}
if current == enabled {
return nil, false, nil
}
newLiteral := "false"
if enabled {
newLiteral = "true"
}
if valueStart >= 0 {
out := make([]byte, 0, len(raw)-int(valueEnd-valueStart)+len(newLiteral))
out = append(out, raw[:valueStart]...)
out = append(out, newLiteral...)
out = append(out, raw[valueEnd:]...)
return out, true, nil
}
return insertEnabledMember(raw, openEnd, closingEnd-1, newLiteral), true, nil
}
// enabledValueOffset returns the byte offset of the value that follows the
// colon after an "enabled" key token that ended at keyEnd.
func enabledValueOffset(raw []byte, keyEnd int64) (int64, error) {
i := keyEnd
for i < int64(len(raw)) && isJSONSpace(raw[i]) {
i++
}
if i >= int64(len(raw)) || raw[i] != ':' {
return 0, errors.New("parse plugin manifest: missing colon after key")
}
i++
for i < int64(len(raw)) && isJSONSpace(raw[i]) {
i++
}
return i, nil
}
// insertEnabledMember adds a top-level "enabled" member to a manifest that has
// none, mirroring the document's own whitespace style where it can. openEnd is
// the byte offset just after the opening brace and bracePos is the byte index
// of the closing brace.
func insertEnabledMember(raw []byte, openEnd, bracePos int64, literal string) []byte {
insertPos := bracePos
for insertPos > openEnd && isJSONSpace(raw[insertPos-1]) {
insertPos--
}
newline := "\n"
if bytes.Contains(raw, []byte("\r\n")) {
newline = "\r\n"
}
first := openEnd
for first < bracePos && isJSONSpace(raw[first]) {
first++
}
empty := first >= bracePos
multiline := bytes.IndexByte(raw[openEnd:bracePos], '\n') >= 0
member := `"enabled": ` + literal
var insert string
switch {
case empty && !multiline:
insert = member
case empty && multiline:
insert = newline + closingLineIndent(raw, openEnd, bracePos) + " " + member
case !empty && !multiline:
insert = "," + member
default:
insert = "," + newline + memberLineIndent(raw, openEnd, first) + member
}
out := make([]byte, 0, len(raw)+len(insert))
out = append(out, raw[:insertPos]...)
out = append(out, insert...)
out = append(out, raw[insertPos:]...)
return out
}
// memberLineIndent returns the indentation of the line the first member of a
// JSON object starts on. first is the byte index of that member's first token.
func memberLineIndent(raw []byte, openEnd, first int64) string {
lineStart := first
for lineStart > openEnd && raw[lineStart-1] != '\n' {
lineStart--
}
end := lineStart
for end < first && (raw[end] == ' ' || raw[end] == '\t') {
end++
}
return string(raw[lineStart:end])
}
// closingLineIndent returns the indentation before an object's closing brace.
func closingLineIndent(raw []byte, openEnd, bracePos int64) string {
start := bracePos
for start > openEnd && (raw[start-1] == ' ' || raw[start-1] == '\t') {
start--
}
return string(raw[start:bracePos])
}
// skipManifestValue consumes the next JSON value from dec.
func skipManifestValue(dec *json.Decoder) error {
tok, err := dec.Token()
if err != nil {
return err
}
return skipManifestComposite(dec, tok)
}
// skipManifestComposite consumes the rest of the value whose first token tok
// was just read from dec. A scalar is already complete; an object or array is
// consumed through its matching closing delimiter.
func skipManifestComposite(dec *json.Decoder, tok json.Token) error {
delim, ok := tok.(json.Delim)
if !ok || (delim != '{' && delim != '[') {
return nil
}
depth := 1
for depth > 0 {
tok, err := dec.Token()
if err != nil {
return err
}
if inner, ok := tok.(json.Delim); ok {
switch inner {
case '{', '[':
depth++
case '}', ']':
depth--
}
}
}
return nil
}
func isJSONSpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
+999
View File
@@ -0,0 +1,999 @@
package plugins
import (
"errors"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
"github.com/Twigpine/zero/internal/installtxn"
"github.com/Twigpine/zero/internal/pathjail"
)
func enableTestOptions(root string) LoadOptions {
return LoadOptions{Roots: []Root{{Source: SourceUser, Path: root}}}
}
func writeRawManifest(t *testing.T, root, dir, raw string) string {
t.Helper()
pluginDir := filepath.Join(root, dir)
if err := os.MkdirAll(pluginDir, 0o700); err != nil {
t.Fatal(err)
}
manifestPath := filepath.Join(pluginDir, "plugin.json")
if err := os.WriteFile(manifestPath, []byte(raw), 0o600); err != nil {
t.Fatal(err)
}
return manifestPath
}
func demoManifest(enabled bool) string {
state := "false"
if enabled {
state = "true"
}
return `{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
"enabled": ` + state + `
}
`
}
func TestSetEnabledByIDPreservesManifestBytes(t *testing.T) {
root := t.TempDir()
raw := `{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
"limits": {
"maxBytes": 9007199254740993
},
"description": "keep me",
"enabled": true,
"tools": []
}
`
manifestPath := writeRawManifest(t, root, "demo", raw)
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
if err != nil {
t.Fatalf("SetEnabledByID(false): %v", err)
}
if !result.Changed || result.Enabled {
t.Fatalf("unexpected result: %#v", result)
}
disabled, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
want := strings.Replace(raw, `"enabled": true`, `"enabled": false`, 1)
if string(disabled) != want {
t.Fatalf("toggle changed more than the enabled value:\n got %q\nwant %q", disabled, want)
}
if !strings.Contains(string(disabled), "9007199254740993") {
t.Fatal("large integer literal was rounded by the toggle")
}
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
if err != nil || !ok {
t.Fatalf("FindByID after disable: ok=%v err=%v", ok, err)
}
if plugin.Enabled {
t.Fatal("loader still reports the plugin enabled after disable")
}
result, err = SetEnabledByID(enableTestOptions(root), "zero.demo", true)
if err != nil {
t.Fatalf("SetEnabledByID(true): %v", err)
}
if !result.Changed || !result.Enabled {
t.Fatalf("unexpected enable result: %#v", result)
}
enabled, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if string(enabled) != raw {
t.Fatalf("round trip did not restore the original bytes:\n got %q\nwant %q", enabled, raw)
}
}
func TestSetEnabledByIDTogglesManifestAndIsIdempotent(t *testing.T) {
root := t.TempDir()
writePluginManifest(t, filepath.Join(root, "demo"), map[string]any{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
"description": "keep me",
"tools": []any{},
})
manifestPath := filepath.Join(root, "demo", "plugin.json")
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
if err != nil {
t.Fatalf("SetEnabledByID(false): %v", err)
}
if !result.Changed {
t.Fatal("expected first disable to change the manifest")
}
raw, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(raw), `"enabled": false`) {
t.Fatalf("manifest not disabled: %s", raw)
}
if !strings.Contains(string(raw), `"keep me"`) {
t.Fatalf("description was not preserved: %s", raw)
}
result, err = SetEnabledByID(enableTestOptions(root), "zero.demo", false)
if err != nil {
t.Fatalf("SetEnabledByID(false) again: %v", err)
}
if result.Changed {
t.Fatal("expected second disable to be a no-op")
}
result, err = SetEnabledByID(enableTestOptions(root), "zero.demo", true)
if err != nil {
t.Fatalf("SetEnabledByID(true): %v", err)
}
if !result.Changed {
t.Fatal("expected enable to change the manifest")
}
}
func TestSetEnabledByIDInsertsMissingField(t *testing.T) {
t.Run("pretty manifest", func(t *testing.T) {
root := t.TempDir()
raw := `{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0"
}
`
manifestPath := writeRawManifest(t, root, "demo", raw)
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
t.Fatalf("SetEnabledByID(false): %v", err)
}
disabled, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
wantDisabled := `{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
"enabled": false
}
`
if string(disabled) != wantDisabled {
t.Fatalf("after insert:\n got %q\nwant %q", disabled, wantDisabled)
}
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", true); err != nil {
t.Fatalf("SetEnabledByID(true): %v", err)
}
enabled, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
wantEnabled := strings.Replace(wantDisabled, `"enabled": false`, `"enabled": true`, 1)
if string(enabled) != wantEnabled {
t.Fatalf("after replace:\n got %q\nwant %q", enabled, wantEnabled)
}
})
t.Run("compact manifest", func(t *testing.T) {
root := t.TempDir()
raw := `{"schemaVersion":1,"id":"zero.demo","name":"Demo","version":"1.0.0"}`
manifestPath := writeRawManifest(t, root, "demo", raw)
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
t.Fatalf("SetEnabledByID(false): %v", err)
}
disabled, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
want := `{"schemaVersion":1,"id":"zero.demo","name":"Demo","version":"1.0.0","enabled": false}`
if string(disabled) != want {
t.Fatalf("compact insert:\n got %q\nwant %q", disabled, want)
}
})
t.Run("null value", func(t *testing.T) {
root := t.TempDir()
raw := `{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
"enabled": null
}
`
manifestPath := writeRawManifest(t, root, "demo", raw)
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", true)
if err != nil {
t.Fatalf("SetEnabledByID(true): %v", err)
}
if result.Changed {
t.Fatal("null counts as enabled, so enabling must be a no-op")
}
unchanged, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if string(unchanged) != raw {
t.Fatalf("no-op toggle rewrote the manifest: %q", unchanged)
}
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
t.Fatalf("SetEnabledByID(false): %v", err)
}
disabled, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
want := strings.Replace(raw, `"enabled": null`, `"enabled": false`, 1)
if string(disabled) != want {
t.Fatalf("null replacement:\n got %q\nwant %q", disabled, want)
}
})
}
func TestToggleEnabledManifestFormattingEdges(t *testing.T) {
cases := []struct {
name string
in string
enabled bool
want string
changed bool
}{
{"empty object", `{}`, false, `{"enabled": false}`, true},
{"empty object with space", `{ }`, false, `{"enabled": false }`, true},
{"empty multiline", "{\n}", false, "{\n \"enabled\": false\n}", true},
{"compact members", `{"a":1}`, false, `{"a":1,"enabled": false}`, true},
{"crlf members", "{\r\n \"a\": 1\r\n}", false, "{\r\n \"a\": 1,\r\n \"enabled\": false\r\n}", true},
{"nested enabled untouched", `{"nested":{"enabled":true},"a":2}`, false, `{"nested":{"enabled":true},"a":2,"enabled": false}`, true},
{"replace with true", `{"enabled": false}`, true, `{"enabled": true}`, true},
{"replace with false", `{"enabled":true}`, false, `{"enabled":false}`, true},
{"null to false", `{"enabled": null}`, false, `{"enabled": false}`, true},
{"null to true is a no-op", `{"enabled": null}`, true, `{"enabled": null}`, false},
{"already enabled is a no-op", `{"enabled": true}`, true, `{"enabled": true}`, false},
// The loader keeps the last duplicate, so an overwritten non-boolean
// value is not an error and only the last value is rewritten.
{"overwritten string", `{"enabled":"obsolete","enabled":true}`, false, `{"enabled":"obsolete","enabled":false}`, true},
{"overwritten composite", `{"enabled":{"a":[1,{"enabled":2}]},"enabled":null}`, false, `{"enabled":{"a":[1,{"enabled":2}]},"enabled":false}`, true},
{"overwritten number is a no-op", `{"enabled":1,"enabled":false}`, false, `{"enabled":1,"enabled":false}`, false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
out, changed, err := toggleEnabledManifest([]byte(tc.in), tc.enabled)
if err != nil {
t.Fatalf("toggleEnabledManifest: %v", err)
}
if changed != tc.changed {
t.Fatalf("changed = %v, want %v", changed, tc.changed)
}
if !tc.changed {
if out != nil {
t.Fatalf("no-op returned bytes: %q", out)
}
return
}
if string(out) != tc.want {
t.Fatalf("got %q, want %q", out, tc.want)
}
})
}
}
func TestToggleEnabledManifestRejectsMalformedInput(t *testing.T) {
for _, raw := range []string{`{"enabled": "yes"}`, `{"enabled": 1}`, `{"enabled": []}`, `{"enabled": {}}`, `{"enabled": true, "enabled": "yes"}`} {
t.Run("value "+raw, func(t *testing.T) {
_, _, err := toggleEnabledManifest([]byte(raw), false)
if err == nil {
t.Fatal("expected error")
}
var manifestErr ManifestError
if !errors.As(err, &manifestErr) {
t.Fatalf("error = %v, want ManifestError", err)
}
if !strings.Contains(err.Error(), "enabled") {
t.Fatalf("error = %v, want it to name the enabled field", err)
}
})
}
for _, raw := range []string{``, `{`, `[]`, `null`, `"str"`, `{} {}`, `{}{}`} {
t.Run("document "+raw, func(t *testing.T) {
if _, _, err := toggleEnabledManifest([]byte(raw), false); err == nil {
t.Fatalf("toggleEnabledManifest(%q) succeeded, want error", raw)
}
})
}
}
// TestSetEnabledByIDHonorsTheLastDuplicateEnabledValue keeps the toggle in step
// with the loader on a manifest whose earlier "enabled" value is overwritten:
// the loader discovers the plugin from the last value, so the toggle must be
// able to change it too.
func TestSetEnabledByIDHonorsTheLastDuplicateEnabledValue(t *testing.T) {
root := t.TempDir()
writeRawManifest(t, root, "demo", `{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
"enabled": "obsolete",
"enabled": true
}
`)
loadedEnabled := func() bool {
t.Helper()
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
if err != nil || !ok {
t.Fatalf("FindByID: ok=%v err=%v", ok, err)
}
return plugin.Enabled
}
if !loadedEnabled() {
t.Fatal("the loader should discover the plugin as enabled from the last value")
}
for _, enabled := range []bool{false, true} {
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", enabled)
if err != nil {
t.Fatalf("SetEnabledByID(%v): %v", enabled, err)
}
if !result.Changed {
t.Fatalf("SetEnabledByID(%v) reported no change", enabled)
}
if got := loadedEnabled(); got != enabled {
t.Fatalf("after SetEnabledByID(%v) the loader sees enabled=%v", enabled, got)
}
}
}
// TestSetEnabledRefusesASwappedPluginDirectory swaps the plugin directory that
// discovery found for a link between the lookup and the edit. The install lock
// does not cover that window, because it only serializes cooperating
// installers. The edit must fail without following the link: neither the
// link's target nor the moved-away original may change.
func TestSetEnabledRefusesASwappedPluginDirectory(t *testing.T) {
cases := []struct {
name string
// link returns the symlink target to plant and the manifest it leads to.
link func(t *testing.T, root string) (target string, manifest string)
}{
{"to a directory outside the plugins root", func(t *testing.T, root string) (string, string) {
manifest := writeRawManifest(t, t.TempDir(), "elsewhere", demoManifest(true))
return filepath.Dir(manifest), manifest
}},
{"to another plugin in the same root", func(t *testing.T, root string) (string, string) {
// A relative link stays inside the root, so confinement to the
// root alone would follow it.
manifest := writeRawManifest(t, root, "other", strings.Replace(demoManifest(true), "zero.demo", "zero.other", 1))
return "other", manifest
}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
root := t.TempDir()
raw := demoManifest(true)
writeRawManifest(t, root, "demo", raw)
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
if err != nil || !ok {
t.Fatalf("FindByID: ok=%v err=%v", ok, err)
}
target, targetManifest := tc.link(t, root)
targetBefore, err := os.ReadFile(targetManifest)
if err != nil {
t.Fatal(err)
}
moved := filepath.Join(t.TempDir(), "moved-away")
if err := os.Rename(plugin.PluginDir, moved); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, plugin.PluginDir); err != nil {
t.Skipf("cannot create a directory symlink here: %v", err)
}
if _, err := setEnabledLocked(plugin, false); !errors.Is(err, pathjail.ErrReparse) {
t.Fatalf("setEnabledLocked through a swapped directory: err = %v, want %v", err, pathjail.ErrReparse)
}
if after, err := os.ReadFile(targetManifest); err != nil || string(after) != string(targetBefore) {
t.Fatalf("the edit followed the link and changed %s (err %v):\n got %q\nwant %q", targetManifest, err, after, targetBefore)
}
if after, err := os.ReadFile(filepath.Join(moved, "plugin.json")); err != nil || string(after) != raw {
t.Fatalf("the moved-away manifest changed (err %v): %q", err, after)
}
})
}
}
// TestSetEnabledDetectsAPluginDirectoryReplacedAfterOpening replaces the live
// plugin directory after the toggle pinned it. The edit then lands in the
// moved-away directory, so the toggle must not refresh the lockfile from either
// tree and must not report success: the plugin installed at the path now is
// unchanged.
func TestSetEnabledDetectsAPluginDirectoryReplacedAfterOpening(t *testing.T) {
for _, tc := range []struct {
name string
pinned bool // the pinned manifest's enabled value
changes bool // whether disabling it is a real edit
}{
{"edit", true, true},
{"no-op", false, false},
} {
t.Run(tc.name, func(t *testing.T) {
root := t.TempDir()
manifestPath := writeRawManifest(t, root, "demo", demoManifest(tc.pinned))
pluginDir := filepath.Dir(manifestPath)
hash, err := hashTree(pluginDir)
if err != nil {
t.Fatal(err)
}
if err := writeLock(root, map[string]LockEntry{"zero.demo": {Source: "./demo", Hash: hash}}); err != nil {
t.Fatal(err)
}
lockPath := filepath.Join(root, LockFileName)
lockBefore, err := os.ReadFile(lockPath)
if err != nil {
t.Fatal(err)
}
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
if err != nil || !ok {
t.Fatalf("FindByID: ok=%v err=%v", ok, err)
}
pinned, err := openPluginDir(plugin)
if err != nil {
t.Fatalf("openPluginDir: %v", err)
}
defer pinned.Close()
moved := filepath.Join(t.TempDir(), "moved-away")
if err := os.Rename(pluginDir, moved); err != nil {
t.Fatal(err)
}
replacement := strings.Replace(demoManifest(true), `"version": "1.0.0"`, `"version": "2.0.0"`, 1)
writeRawManifest(t, root, "demo", replacement)
// The lockfile check hashes the pinned tree, which still matches the
// recorded hash, not the replacement now at the path.
if _, _, clean := cleanLockEntry(plugin, pinned.dir); !clean {
t.Error("cleanLockEntry hashed the replacement at the path instead of the pinned directory")
}
if _, err := setEnabledPinned(plugin, pinned, false); !errors.Is(err, errPluginDirReplaced) {
t.Fatalf("setEnabledPinned after the directory was replaced: err = %v, want %v", err, errPluginDirReplaced)
}
if after, err := os.ReadFile(manifestPath); err != nil || string(after) != replacement {
t.Fatalf("the replacement manifest changed (err %v): %q", err, after)
}
if after, err := os.ReadFile(lockPath); err != nil || string(after) != string(lockBefore) {
t.Fatalf("the lockfile changed (err %v):\n got %s\nwant %s", err, after, lockBefore)
}
movedRaw, err := os.ReadFile(filepath.Join(moved, "plugin.json"))
if err != nil {
t.Fatal(err)
}
if tc.changes != (string(movedRaw) != demoManifest(tc.pinned)) {
t.Fatalf("pinned manifest edit = %v, want %v: %q", !tc.changes, tc.changes, movedRaw)
}
})
}
}
func TestSetEnabledByIDRespectsUserOnlyFilter(t *testing.T) {
dir := t.TempDir()
userRoot := filepath.Join(dir, "user")
projectRoot := filepath.Join(dir, "project")
writePluginManifest(t, filepath.Join(userRoot, "demo"), map[string]any{
"schemaVersion": 1,
"id": "zero.demo",
"name": "User Demo",
"version": "0.1.0",
"enabled": true,
})
writePluginManifest(t, filepath.Join(projectRoot, "demo"), map[string]any{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Project Demo",
"version": "0.2.0",
"enabled": true,
})
result, err := SetEnabledByID(LoadOptions{
Roots: []Root{
{Source: SourceUser, Path: userRoot},
{Source: SourceProject, Path: projectRoot},
},
}, "zero.demo", false)
if err != nil {
t.Fatalf("SetEnabledByID: %v", err)
}
if result.Source != SourceProject {
t.Fatalf("source = %s, want project (precedence)", result.Source)
}
if !result.Changed || result.Enabled {
t.Fatalf("unexpected result: %#v", result)
}
userOnly, err := SetEnabledByID(LoadOptions{
Roots: []Root{
{Source: SourceUser, Path: userRoot},
{Source: SourceProject, Path: projectRoot},
},
ExcludeProject: true,
}, "zero.demo", false)
if err != nil {
t.Fatalf("SetEnabledByID user-only: %v", err)
}
if userOnly.Source != SourceUser {
t.Fatalf("user-only source = %s, want user", userOnly.Source)
}
}
func TestSetEnabledPreservesManifestMode(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("Windows does not preserve chmod bits in os.Stat mode")
}
root := t.TempDir()
writePluginManifest(t, filepath.Join(root, "demo"), map[string]any{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "1.0.0",
})
manifestPath := filepath.Join(root, "demo", "plugin.json")
if err := os.Chmod(manifestPath, 0o644); err != nil {
t.Fatal(err)
}
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
t.Fatalf("SetEnabledByID: %v", err)
}
info, err := os.Stat(manifestPath)
if err != nil {
t.Fatal(err)
}
if got := info.Mode().Perm(); got != 0o644 {
t.Fatalf("mode = %04o, want 0644", got)
}
}
func TestSetEnabledByIDMissingPlugin(t *testing.T) {
_, err := SetEnabledByID(LoadOptions{Roots: []Root{{Source: SourceUser, Path: t.TempDir()}}}, "missing", false)
if err == nil {
t.Fatal("expected missing plugin error")
}
if !errors.Is(err, ErrNotInstalled) {
t.Fatalf("error = %v, want ErrNotInstalled", err)
}
}
func TestSetEnabledPublishFailureLeavesManifestIntact(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("Windows directory permissions do not block writes the way this test needs")
}
if os.Geteuid() == 0 {
t.Skip("root bypasses directory permission checks")
}
root := t.TempDir()
raw := demoManifest(true)
manifestPath := writeRawManifest(t, root, "demo", raw)
pluginDir := filepath.Dir(manifestPath)
if err := os.Chmod(pluginDir, 0o555); err != nil {
t.Fatal(err)
}
defer func() { _ = os.Chmod(pluginDir, 0o755) }()
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err == nil {
t.Fatal("expected the publish to fail in a read-only plugin directory")
}
after, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if string(after) != raw {
t.Fatalf("failed publish changed the manifest:\n got %q\nwant %q", after, raw)
}
}
// TestSetEnabledByIDWaitsForInstallRootLock proves the toggle takes the same
// cross-process lock Install and Remove take on the plugin's install root: it
// must not complete while an install holds that lock.
func TestSetEnabledByIDWaitsForInstallRootLock(t *testing.T) {
root := t.TempDir()
manifestPath := writeRawManifest(t, root, "demo", demoManifest(true))
unlock, err := installtxn.Lock(root)
if err != nil {
t.Fatal(err)
}
results := make(chan SetEnabledResult, 1)
failures := make(chan error, 1)
go func() {
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
if err != nil {
failures <- err
return
}
results <- result
}()
select {
case result := <-results:
unlock()
t.Fatalf("toggle completed while the install lock was held: %#v", result)
case err := <-failures:
unlock()
t.Fatalf("toggle failed while the install lock was held: %v", err)
case <-time.After(200 * time.Millisecond):
}
unlock()
select {
case result := <-results:
if !result.Changed {
t.Fatalf("toggle did not report a change: %#v", result)
}
case err := <-failures:
t.Fatalf("toggle failed after the lock was released: %v", err)
case <-time.After(10 * time.Second):
t.Fatal("toggle did not finish after the install lock was released")
}
raw, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(raw), `"enabled": false`) {
t.Fatalf("manifest not disabled: %s", raw)
}
}
// TestSetEnabledByIDSynchronizesWithStagedInstallCommit pauses an install
// between staging and commit, lets the toggle try to run, then commits the
// install. The toggle must apply to the newly installed manifest instead of
// overwriting it with bytes read before the install, and the lockfile entry
// must describe the tree actually on disk.
func TestSetEnabledByIDSynchronizesWithStagedInstallCommit(t *testing.T) {
root := t.TempDir()
oldRaw := demoManifest(true)
pluginDir := filepath.Dir(writeRawManifest(t, root, "zero.demo", oldRaw))
// Build the replacement tree with the same staging primitives Install
// uses and leave it staged but uncommitted.
src := t.TempDir()
newRaw := `{
"schemaVersion": 1,
"id": "zero.demo",
"name": "Demo",
"version": "2.0.0",
"description": "reinstalled",
"enabled": true
}
`
srcPlugin := filepath.Join(src, "zero.demo")
if err := os.MkdirAll(srcPlugin, 0o700); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(srcPlugin, "plugin.json"), []byte(newRaw), 0o600); err != nil {
t.Fatal(err)
}
staged, cleanup, err := installtxn.StageDir(root)
if err != nil {
t.Fatal(err)
}
defer cleanup()
if err := copyTree(srcPlugin, staged); err != nil {
t.Fatal(err)
}
stagedHash, err := hashTree(staged)
if err != nil {
t.Fatal(err)
}
// An install holds the root lock from before its commit until after it.
unlock, err := installtxn.Lock(root)
if err != nil {
t.Fatal(err)
}
results := make(chan SetEnabledResult, 1)
failures := make(chan error, 1)
go func() {
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
if err != nil {
failures <- err
return
}
results <- result
}()
select {
case result := <-results:
unlock()
t.Fatalf("toggle completed while the install lock was held: %#v", result)
case err := <-failures:
unlock()
t.Fatalf("toggle failed while the install lock was held: %v", err)
case <-time.After(200 * time.Millisecond):
}
// Commit the staged install exactly like Install does: swap the tree and
// publish the lockfile entry while still holding the lock.
lock, err := ReadLock(root)
if err != nil {
unlock()
t.Fatal(err)
}
lock["zero.demo"] = LockEntry{Source: string(SourceUser), Hash: stagedHash}
if err := installtxn.CommitDir(pluginDir, staged, func() error { return writeLock(root, lock) }); err != nil {
unlock()
t.Fatal(err)
}
unlock()
var result SetEnabledResult
select {
case result = <-results:
case err := <-failures:
t.Fatalf("toggle failed after the install committed: %v", err)
case <-time.After(10 * time.Second):
t.Fatal("toggle did not finish after the install committed")
}
if !result.Changed {
t.Fatalf("toggle result = %#v, want changed", result)
}
// The toggle must have applied to the newly installed manifest, not to the
// bytes it first read.
final, err := os.ReadFile(filepath.Join(pluginDir, "plugin.json"))
if err != nil {
t.Fatal(err)
}
want := strings.Replace(newRaw, `"enabled": true`, `"enabled": false`, 1)
if string(final) != want {
t.Fatalf("manifest after concurrent install + disable:\n got %q\nwant %q", final, want)
}
// The lockfile entry must describe the tree that is actually on disk.
entries, err := ReadLock(root)
if err != nil {
t.Fatal(err)
}
current, err := hashTree(pluginDir)
if err != nil {
t.Fatal(err)
}
if entries["zero.demo"].Hash != current {
t.Fatalf("plugins.lock hash = %q, want current tree hash %q", entries["zero.demo"].Hash, current)
}
}
func TestSetEnabledByIDRefreshesCleanLockEntry(t *testing.T) {
root := t.TempDir()
manifestPath := writeRawManifest(t, root, "zero.demo", demoManifest(true))
pluginDir := filepath.Dir(manifestPath)
current, err := hashTree(pluginDir)
if err != nil {
t.Fatal(err)
}
if err := writeLock(root, map[string]LockEntry{"zero.demo": {Source: string(SourceUser), Hash: current}}); err != nil {
t.Fatal(err)
}
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
t.Fatalf("SetEnabledByID: %v", err)
}
entries, err := ReadLock(root)
if err != nil {
t.Fatal(err)
}
afterToggle, err := hashTree(pluginDir)
if err != nil {
t.Fatal(err)
}
if entries["zero.demo"].Hash == current {
t.Fatal("lock hash was not refreshed for a clean install")
}
if entries["zero.demo"].Hash != afterToggle {
t.Fatalf("lock hash = %q, want refreshed tree hash %q", entries["zero.demo"].Hash, afterToggle)
}
info, err := Info(InfoOptions{LoadOptions: enableTestOptions(root)}, "zero.demo")
if err != nil {
t.Fatal(err)
}
if info.HashDrift {
t.Fatal("toggle left hash drift behind for a clean install")
}
// Pre-existing drift must survive the next toggle instead of being
// absorbed into the lockfile.
if err := os.WriteFile(filepath.Join(pluginDir, "extra.txt"), []byte("edited\n"), 0o600); err != nil {
t.Fatal(err)
}
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", true); err != nil {
t.Fatalf("SetEnabledByID(true): %v", err)
}
entries, err = ReadLock(root)
if err != nil {
t.Fatal(err)
}
if entries["zero.demo"].Hash != afterToggle {
t.Fatalf("toggle absorbed pre-existing drift: hash = %q, want %q", entries["zero.demo"].Hash, afterToggle)
}
}
// TestSetEnabledByIDReportsAPluginRemovedWhileWaiting removes the plugin while
// the toggle is blocked on the install-root lock, exactly as Remove does while
// it holds that lock. The toggle must report the plugin as not installed
// instead of editing, or recreating, a tree that no longer exists.
func TestSetEnabledByIDReportsAPluginRemovedWhileWaiting(t *testing.T) {
root := t.TempDir()
pluginDir := filepath.Dir(writeRawManifest(t, root, "zero.demo", demoManifest(true)))
unlock, err := installtxn.Lock(root)
if err != nil {
t.Fatal(err)
}
failures := make(chan error, 1)
go func() {
_, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
failures <- err
}()
select {
case err := <-failures:
unlock()
t.Fatalf("toggle returned while the install lock was held: %v", err)
case <-time.After(200 * time.Millisecond):
}
if err := installtxn.RemoveDir(pluginDir, func() error { return nil }); err != nil {
unlock()
t.Fatal(err)
}
unlock()
select {
case err := <-failures:
if !errors.Is(err, ErrNotInstalled) {
t.Fatalf("toggle after a concurrent removal: error = %v, want ErrNotInstalled", err)
}
case <-time.After(10 * time.Second):
t.Fatal("toggle did not finish after the removal released the lock")
}
if _, err := os.Stat(pluginDir); !os.IsNotExist(err) {
t.Fatalf("toggle recreated the removed plugin: %v", err)
}
}
// TestSetEnabledByIDResolvesInterruptedUpdatesFirst covers every point an
// update can be killed at that leaves the plugin discoverable. The toggle
// holds the install-root lock, so it has to
// run the same recovery pass Install and Remove run before it reads the tree:
// otherwise it edits, or fails to find, a tree that is still owed a restore,
// and the next install or removal puts the backup back over the user's edit.
func TestSetEnabledByIDResolvesInterruptedUpdatesFirst(t *testing.T) {
for _, state := range interruptedStates() {
t.Run(state.name, func(t *testing.T) {
u := seedInterruptedUpdate(t)
workspace := state.plant(t, u)
want := state.want(u)
if _, err := os.Lstat(u.target()); os.IsNotExist(err) {
// No root lists a plugin whose only copy is in a workspace, so the
// toggle reports it not installed without taking any lock, just as
// list and info do; the next install or removal restores it.
t.Skip("nothing is at the target, so the toggle never takes the lock")
}
result, err := SetEnabledByID(enableTestOptions(u.dir), "zero.demo", false)
if err != nil {
t.Fatalf("toggle after the interrupted update: %v", err)
}
if !result.Changed || result.Enabled {
t.Fatalf("toggle result = %#v, want changed and disabled", result)
}
assertDisabledAfterRecovery(t, u, workspace, want)
// The next lock holder runs recovery again. The disable has to survive
// it rather than be replaced by a backup the toggle never saw.
if err := installOther(t, u.dir, "zero.other"); err != nil {
t.Fatalf("install after the toggle: %v", err)
}
assertDisabledAfterRecovery(t, u, workspace, want)
})
}
}
func assertDisabledAfterRecovery(t *testing.T, u interruptedUpdate, workspace string, want recoveryWant) {
t.Helper()
data, err := os.ReadFile(filepath.Join(u.target(), "generation.txt"))
if err != nil {
t.Fatalf("reading the live tree: %v", err)
}
if string(data) != want.generation {
t.Errorf("the live tree is the %q generation, want %q", data, want.generation)
}
plugin, ok, err := FindByID(enableTestOptions(u.dir), "zero.demo")
if err != nil {
t.Fatal(err)
}
if !ok {
t.Fatal("Load does not see the toggled plugin")
}
if plugin.Version != want.version {
t.Errorf("Load sees version %q, want %q", plugin.Version, want.version)
}
if plugin.Enabled {
t.Errorf("the disable did not stick: Load sees the plugin enabled")
}
_, err = os.Stat(workspace)
if want.workspace && err != nil {
t.Errorf("the workspace should be left alone, got %v", err)
}
if !want.workspace && !os.IsNotExist(err) {
t.Errorf("the resolved workspace should be cleared, got %v", err)
}
}
// TestSetEnabledByIDAbortsOnAnUnresolvedInterruptedUpdate leaves a workspace
// whose trees match neither lockfile hash. Recovery cannot pick one, so the
// toggle must fail without editing the live manifest or the lockfile.
func TestSetEnabledByIDAbortsOnAnUnresolvedInterruptedUpdate(t *testing.T) {
u := seedInterruptedUpdate(t)
workspace := u.plantWorkspace(t)
u.mustRename(t, u.target(), filepath.Join(workspace, "previous"))
u.mustRename(t, filepath.Join(workspace, "staged"), u.target())
unmatched := LockEntry{Source: u.oldSource, Hash: "sha256:0000000000000000000000000000000000000000000000000000000000000000"}
lock, err := ReadLock(u.dir)
if err != nil {
t.Fatal(err)
}
lock["zero.demo"] = unmatched
if err := writeLock(u.dir, lock); err != nil {
t.Fatal(err)
}
manifestPath := filepath.Join(u.target(), "plugin.json")
before, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if _, err := SetEnabledByID(enableTestOptions(u.dir), "zero.demo", false); err == nil {
t.Fatal("toggle did not report the unresolved transaction")
}
after, err := os.ReadFile(manifestPath)
if err != nil {
t.Fatal(err)
}
if string(after) != string(before) {
t.Errorf("the aborted toggle edited the manifest:\n got %q\nwant %q", after, before)
}
assertUntouched(t, u, "new", unmatched)
if _, err := os.Stat(filepath.Join(workspace, "previous")); err != nil {
t.Errorf("the backup should still be there for a hand rescue, got %v", err)
}
}
+2 -1
View File
@@ -7,7 +7,8 @@ import (
"strings"
)
// ErrNotInstalled is returned when Info cannot find a plugin with the requested id.
// ErrNotInstalled is returned when a plugin id cannot be found under the given
// LoadOptions (for example by Info or SetEnabledByID).
var ErrNotInstalled = errors.New("plugin is not installed")
// PluginInfo combines a loaded plugin with optional lockfile metadata.
+37 -11
View File
@@ -567,15 +567,32 @@ func validInstallID(id string) bool {
// sorted order. Each file contributes its plugin-relative path, executable bit,
// and bytes, so renames, mode flips, and content edits all change the hash.
func hashTree(root string) (string, error) {
handle, err := os.OpenRoot(root)
if err != nil {
return "", err
}
defer handle.Close()
return hashRootedTree(handle)
}
// hashRootedTree is hashTree over a directory that is already open. Walking
// relative to the handle hashes the directory it names even if its path is
// replaced meanwhile, and keeps the walk inside it.
//
// The walk calls the handle's own methods rather than handle.FS(): the fs.FS
// adapter only accepts fs.ValidPath names, which must be UTF-8, so it would
// refuse file names that the filesystem and copyTree accept.
func hashRootedTree(handle *os.Root) (string, error) {
hasher := sha256.New()
if err := hashTreeInto(hasher, root, root); err != nil {
if err := hashTreeInto(hasher, handle, "."); err != nil {
return "", err
}
return "sha256:" + hex.EncodeToString(hasher.Sum(nil)), nil
}
func hashTreeInto(hasher io.Writer, root string, dir string) error {
entries, err := os.ReadDir(dir)
// hashTreeInto hashes dir, a path relative to handle, into hasher.
func hashTreeInto(hasher io.Writer, handle *os.Root, dir string) error {
entries, err := readRootedDir(handle, dir)
if err != nil {
return err
}
@@ -588,8 +605,8 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
if name == ".git" {
continue
}
path := filepath.Join(dir, name)
info, err := os.Lstat(path)
rel := filepath.Join(dir, name)
info, err := handle.Lstat(rel)
if err != nil {
return err
}
@@ -598,14 +615,10 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
// Skipped by copyTree, so excluded from the hash too.
continue
case info.IsDir():
if err := hashTreeInto(hasher, root, path); err != nil {
if err := hashTreeInto(hasher, handle, rel); err != nil {
return err
}
case info.Mode().IsRegular():
rel, err := filepath.Rel(root, path)
if err != nil {
return err
}
executable := 0
if info.Mode().Perm()&0o111 != 0 {
executable = 1
@@ -616,7 +629,7 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
if _, err := io.WriteString(hasher, header); err != nil {
return err
}
file, err := os.Open(path)
file, err := handle.Open(rel)
if err != nil {
return err
}
@@ -634,3 +647,16 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
}
return nil
}
// readRootedDir lists dir, a path relative to handle, like os.ReadDir.
func readRootedDir(handle *os.Root, dir string) ([]os.DirEntry, error) {
file, err := handle.Open(dir)
if err != nil {
return nil, err
}
entries, err := file.ReadDir(-1)
if closeErr := file.Close(); err == nil {
err = closeErr
}
return entries, err
}
+59
View File
@@ -1181,3 +1181,62 @@ func TestRecoveryLeavesAPrefixCollidingDirectoryAlone(t *testing.T) {
t.Errorf("recovery published content it does not own: %v", err)
}
}
// TestNonUTF8FileNamesAreInstalledHashedAndToggled covers a plugin whose tree
// holds a file and a directory named with bytes that are not valid UTF-8, which
// Linux allows. copyTree copies them, so hashTree must hash them too: Install
// records the hash, the toggle refreshes it, and Info sees an edit to that file
// as drift. A walk that rejects such names breaks all three.
func TestNonUTF8FileNamesAreInstalledHashedAndToggled(t *testing.T) {
src := writeSourcePlugin(t, filepath.Join(t.TempDir(), "src"), validManifest())
nested := filepath.Join("assets\xff", "data\xfe.bin")
if err := os.Mkdir(filepath.Join(src, "assets\xff"), 0o755); err != nil {
t.Skipf("this filesystem rejects non-UTF-8 names: %v", err)
}
if err := os.WriteFile(filepath.Join(src, nested), []byte("original\n"), 0o644); err != nil {
t.Skipf("this filesystem rejects non-UTF-8 names: %v", err)
}
destDir := t.TempDir()
if _, err := Install(context.Background(), InstallOptions{Source: src, Dir: destDir}); err != nil {
t.Fatalf("Install a plugin with a non-UTF-8 file name: %v", err)
}
pluginDir := filepath.Join(destDir, "zero.demo")
if data, err := os.ReadFile(filepath.Join(pluginDir, nested)); err != nil || string(data) != "original\n" {
t.Fatalf("installed copy of the non-UTF-8 file = %q, %v", data, err)
}
options := LoadOptions{Roots: []Root{{Source: SourceUser, Path: destDir}}}
before, err := ReadLock(destDir)
if err != nil {
t.Fatal(err)
}
if _, err := SetEnabledByID(options, "zero.demo", false); err != nil {
t.Fatalf("SetEnabledByID on a plugin with a non-UTF-8 file name: %v", err)
}
after, err := ReadLock(destDir)
if err != nil {
t.Fatal(err)
}
if after["zero.demo"].Hash == before["zero.demo"].Hash {
t.Fatal("the toggle did not refresh the lock hash")
}
info, err := Info(InfoOptions{LoadOptions: options}, "zero.demo")
if err != nil {
t.Fatal(err)
}
if info.HashDrift {
t.Fatal("the toggle left hash drift behind")
}
if err := os.WriteFile(filepath.Join(pluginDir, nested), []byte("edited\n"), 0o644); err != nil {
t.Fatal(err)
}
info, err = Info(InfoOptions{LoadOptions: options}, "zero.demo")
if err != nil {
t.Fatal(err)
}
if !info.HashDrift {
t.Fatal("an edit to the non-UTF-8 file was not reported as drift")
}
}
+3 -1
View File
@@ -132,7 +132,9 @@ var descriptors = []Descriptor{
// way in; RequiresAuth is forced true so catalog consumers know the provider
// needs an interactive login before a request will succeed.
func() Descriptor {
d := openAICompat("chatgpt", "ChatGPT", "https://chatgpt.com/backend-api/codex", "gpt-5.5", nil)
// New ChatGPT profiles follow the vendor's GPT-5.5 replacement guidance:
// https://learn.chatgpt.com/docs/models#gpt-55-retirement
d := openAICompat("chatgpt", "ChatGPT", "https://chatgpt.com/backend-api/codex", "gpt-6.1-sol", nil)
d.RequiresAuth = true
return oauthProvider(d, false, false)
}(),
+20 -4
View File
@@ -44,6 +44,20 @@ var minimaxCuratedModels = []Model{
OutputModalities: []string{"text"},
Source: minimaxModelSource,
},
{
ID: "MiniMax-M3.1-Flash",
Description: "fast reasoning model",
ContextWindow: 1_000_000,
ToolCall: true,
Reasoning: true,
ReasoningEfforts: []string{"low", "medium", "high", "xhigh", "max"},
DefaultReasoningEffort: "max",
InputModalities: []string{"text", "image", "video"},
OutputModalities: []string{"text"},
InputCost: 0.15,
OutputCost: 0.60,
Source: minimaxModelSource,
},
{
ID: "MiniMax-M2.7",
Description: "agentic coding model",
@@ -113,10 +127,12 @@ var curatedModels = map[string][]Model{
{ID: "deepseek/deepseek-chat", Description: "coding model"},
},
"chatgpt": {
{ID: "gpt-5.5", Description: "recommended Codex model"},
{ID: "gpt-5.4", Description: "strong Codex model"},
{ID: "gpt-5.4-mini", Description: "fast Codex model"},
{ID: "gpt-5.3-codex-spark", Description: "research preview fast Codex model"},
// Suggestions only: availability comes from the authenticated Codex catalog.
// https://learn.chatgpt.com/docs/models
{ID: "gpt-6.1-sol", Description: "balanced Codex model"},
{ID: "gpt-6.1-terra", Description: "fast model for writing, visual understanding, and computer use"},
{ID: "gpt-6-astra", Description: "advanced reasoning model"},
{ID: "gpt-6-luna", Description: "efficient coding model"},
},
"deepseek": {
{ID: "deepseek-chat", Description: "catalog default"},
+22 -6
View File
@@ -35,8 +35,8 @@ func TestModelsAreProviderScoped(t *testing.T) {
},
{
provider: "chatgpt",
want: []string{"gpt-5.5", "gpt-5.4", "gpt-5.4-mini", "gpt-5.3-codex-spark"},
notWant: []string{"gpt-5", "gpt-4.1", "openai/gpt-4.1"},
want: []string{"gpt-6.1-sol", "gpt-6.1-terra", "gpt-6-astra", "gpt-6-luna"},
notWant: []string{"gpt-5.5", "gpt-5.4", "gpt-5.4-mini", "gpt-5.3-codex-spark", "openai/gpt-4.1"},
},
{
provider: "aimlapi",
@@ -65,7 +65,7 @@ func TestModelsAreProviderScoped(t *testing.T) {
},
{
provider: "minimaxi-cn",
want: []string{"MiniMax-M3", "MiniMax-M2.7", "MiniMax-M2.1"},
want: []string{"MiniMax-M3", "MiniMax-M3.1-Flash", "MiniMax-M2.7", "MiniMax-M2.1"},
notWant: []string{"gpt-4.1", "claude-sonnet-4.5"},
},
{
@@ -165,8 +165,8 @@ func TestMiniMaxModelsExposeCurrentCapabilities(t *testing.T) {
t.Fatal("provider minimax missing from catalog")
}
models := Models(descriptor)
if len(models) < 2 || models[0].ID != "MiniMax-M3" || models[1].ID != "MiniMax-M2.7" {
t.Fatalf("MiniMax current models = %#v, want MiniMax-M3 followed by MiniMax-M2.7", modelIDs(models))
if len(models) < 3 || models[0].ID != "MiniMax-M3" || models[1].ID != "MiniMax-M3.1-Flash" || models[2].ID != "MiniMax-M2.7" {
t.Fatalf("MiniMax current models = %#v, want MiniMax-M3, MiniMax-M3.1-Flash, then MiniMax-M2.7", modelIDs(models))
}
m3 := models[0]
@@ -178,7 +178,23 @@ func TestMiniMaxModelsExposeCurrentCapabilities(t *testing.T) {
t.Fatalf("MiniMax-M3 modalities = input:%#v output:%#v", m3.InputModalities, m3.OutputModalities)
}
m27 := models[1]
m31Flash := models[1]
if m31Flash.ContextWindow != 1_000_000 || !m31Flash.ToolCall || !m31Flash.Reasoning {
t.Fatalf("MiniMax-M3.1-Flash capabilities = %#v, want 1M context with tools and reasoning", m31Flash)
}
if !reflect.DeepEqual(m31Flash.ReasoningEfforts, []string{"low", "medium", "high", "xhigh", "max"}) ||
m31Flash.DefaultReasoningEffort != "max" {
t.Fatalf("MiniMax-M3.1-Flash reasoning efforts = %#v default %q", m31Flash.ReasoningEfforts, m31Flash.DefaultReasoningEffort)
}
if !reflect.DeepEqual(m31Flash.InputModalities, []string{"text", "image", "video"}) ||
!reflect.DeepEqual(m31Flash.OutputModalities, []string{"text"}) {
t.Fatalf("MiniMax-M3.1-Flash modalities = input:%#v output:%#v", m31Flash.InputModalities, m31Flash.OutputModalities)
}
if m31Flash.InputCost != 0.15 || m31Flash.OutputCost != 0.60 {
t.Fatalf("MiniMax-M3.1-Flash costs = %f/%f, want 0.15/0.60", m31Flash.InputCost, m31Flash.OutputCost)
}
m27 := models[2]
if m27.ContextWindow != 204_800 || !m27.ToolCall || !m27.Reasoning {
t.Fatalf("MiniMax-M2.7 capabilities = %#v, want 204.8K context with tools and reasoning", m27)
}
@@ -0,0 +1,31 @@
package providermodeldiscovery
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"github.com/Twigpine/zero/internal/config"
)
func TestChatGPTDiscoveryUsesCurrentCompatibilityVersion(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if got := r.URL.Query().Get("client_version"); got != "0.159.0" {
t.Errorf("client_version = %q, want documented GPT-6.1 Terra compatibility floor 0.159.0", got)
}
_, _ = w.Write([]byte(`{"models":[{"slug":"gpt-future-coder"}]}`))
}))
defer server.Close()
profile := config.ProviderProfile{
CatalogID: "chatgpt", ProviderKind: config.ProviderKindOpenAICompatible,
BaseURL: server.URL + "/backend-api/codex", APIKey: "test-bearer",
}
models, err := Discover(context.Background(), profile, Options{HTTPClient: server.Client()})
if err != nil {
t.Fatal(err)
}
if len(models) != 1 || models[0].ID != "gpt-future-coder" {
t.Fatalf("models = %#v, want only gpt-future-coder", models)
}
}
+3 -2
View File
@@ -20,8 +20,9 @@ import (
)
const (
anthropicVersion = "2023-06-01"
chatGPTModelsProtocolVersion = "0.146.0"
anthropicVersion = "2023-06-01"
// GPT-6.1 Terra's documented Codex client floor: https://learn.chatgpt.com/docs/models.
chatGPTModelsProtocolVersion = "0.159.0"
)
type Model struct {
+22 -5
View File
@@ -49,6 +49,19 @@ func thinkingBudgetForEffort(effort string) int {
}
}
func miniMaxM31FlashEffort(model, effort string) string {
if !strings.EqualFold(strings.TrimSpace(model), "MiniMax-M3.1-Flash") {
return ""
}
effort = strings.ToLower(strings.TrimSpace(effort))
switch effort {
case "low", "medium", "high", "xhigh", "max":
return effort
default:
return ""
}
}
// resolveThinking returns the thinking budget and the max_tokens to send. When a
// budget is requested, max_tokens is raised if needed so the budget plus a
// minimum response both fit (Anthropic rejects budget >= max_tokens). enabled is
@@ -377,11 +390,15 @@ func (provider *Provider) anthropicRequest(request zeroruntime.CompletionRequest
Messages: messages,
Stream: true,
}
// Extended thinking: enable when a reasoning effort was requested. The budget
// is counted against max_tokens, so raise max_tokens to leave room for the
// response. Temperature is intentionally left unset (Anthropic requires the
// default when thinking is on).
if budget, effectiveMax, enabled := resolveThinking(request.ReasoningEffort, provider.maxTokens); enabled {
// MiniMax-M3.1-Flash uses adaptive thinking with output_config.effort instead
// of Anthropic's token budget. For other models, the budget is counted against
// max_tokens, so raise max_tokens to leave room for the response. Temperature
// is intentionally left unset (Anthropic requires the default when thinking
// is on).
if effort := miniMaxM31FlashEffort(provider.model, request.ReasoningEffort); effort != "" {
mapped.Thinking = &thinkingConfig{Type: "adaptive"}
mapped.OutputConfig = &outputConfig{Effort: effort}
} else if budget, effectiveMax, enabled := resolveThinking(request.ReasoningEffort, provider.maxTokens); enabled {
mapped.MaxTokens = effectiveMax
mapped.Thinking = &thinkingConfig{Type: "enabled", BudgetTokens: budget}
}
@@ -266,6 +266,53 @@ func TestStreamCompletionEnablesThinkingWhenEffortRequested(t *testing.T) {
}
}
func TestStreamCompletionUsesMiniMaxAdaptiveEffort(t *testing.T) {
t.Parallel()
for _, effort := range []string{"low", "xhigh"} {
effort := effort
t.Run(effort, func(t *testing.T) {
t.Parallel()
var gotBody map[string]any
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if err := json.NewDecoder(r.Body).Decode(&gotBody); err != nil {
t.Fatalf("decode request body: %v", err)
}
writeSSEEvent(w, "message_stop", `{"type":"message_stop"}`)
}))
defer server.Close()
provider, err := New(Options{BaseURL: server.URL + "/", Model: "MiniMax-M3.1-Flash", MaxTokens: 64_000})
if err != nil {
t.Fatalf("New returned error: %v", err)
}
stream, err := provider.StreamCompletion(context.Background(), zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "hi"}},
ReasoningEffort: effort,
})
if err != nil {
t.Fatalf("StreamCompletion returned error: %v", err)
}
drain(stream)
thinking, ok := gotBody["thinking"].(map[string]any)
if !ok || thinking["type"] != "adaptive" {
t.Fatalf("thinking = %#v, want adaptive", gotBody["thinking"])
}
if _, ok := thinking["budget_tokens"]; ok {
t.Fatalf("MiniMax thinking must not include budget_tokens: %#v", thinking)
}
outputConfig, ok := gotBody["output_config"].(map[string]any)
if !ok || outputConfig["effort"] != effort {
t.Fatalf("output_config = %#v, want effort %q", gotBody["output_config"], effort)
}
if mt, _ := gotBody["max_tokens"].(float64); int(mt) != 64_000 {
t.Fatalf("max_tokens = %#v, want unchanged 64000", gotBody["max_tokens"])
}
})
}
}
func TestStreamCompletionOmitsThinkingWithoutEffort(t *testing.T) {
t.Parallel()
var gotBody map[string]any
+11 -7
View File
@@ -10,17 +10,21 @@ type messagesRequest struct {
System any `json:"system,omitempty"`
// Thinking enables extended thinking when a reasoning effort was requested.
// Omitted (nil) for normal requests so behavior is unchanged.
Thinking *thinkingConfig `json:"thinking,omitempty"`
Tools []anthropicTool `json:"tools,omitempty"`
Stream bool `json:"stream"`
Thinking *thinkingConfig `json:"thinking,omitempty"`
OutputConfig *outputConfig `json:"output_config,omitempty"`
Tools []anthropicTool `json:"tools,omitempty"`
Stream bool `json:"stream"`
}
// thinkingConfig requests extended thinking with a token budget. Type is always
// "enabled"; BudgetTokens is how many tokens the model may spend thinking (it is
// counted against max_tokens).
// thinkingConfig requests extended thinking. Anthropic uses "enabled" with a
// token budget; MiniMax-M3.1-Flash uses "adaptive" with output_config.effort.
type thinkingConfig struct {
Type string `json:"type"`
BudgetTokens int `json:"budget_tokens"`
BudgetTokens int `json:"budget_tokens,omitempty"`
}
type outputConfig struct {
Effort string `json:"effort"`
}
type anthropicMessage struct {
@@ -288,10 +288,11 @@ func TestScanSSEDataWithContextContentResetsOnData(t *testing.T) {
pr, pw := io.Pipe()
go func() {
// One data line every 30ms for ~300ms (well past the 120ms content window,
// with comfortable slack under the 100ms idle timeout), so the content
// watchdog keeps resetting, then close cleanly.
for i := 0; i < 10; i++ {
// One data line every 30ms for ~750ms (well past the 600ms content window,
// with ~470ms of slack under the 500ms idle timeout, so a scheduling stall
// on a busy shared runner does not trip it), so the content watchdog keeps
// resetting, then close cleanly.
for i := 0; i < 25; i++ {
if _, err := io.WriteString(pw, "data: chunk\n\n"); err != nil {
return
}
@@ -303,7 +304,7 @@ func TestScanSSEDataWithContextContentResetsOnData(t *testing.T) {
n := 0
done := make(chan error, 1)
go func() {
done <- ScanSSEDataWithContext(context.Background(), func() {}, pr, 100*time.Millisecond, func(string) bool { n++; return true })
done <- ScanSSEDataWithContext(context.Background(), func() {}, pr, 500*time.Millisecond, func(string) bool { n++; return true })
}()
select {
@@ -314,8 +315,8 @@ func TestScanSSEDataWithContextContentResetsOnData(t *testing.T) {
case <-time.After(3 * time.Second):
t.Fatal("ScanSSEDataWithContext hung on a producing stream")
}
if n != 10 {
t.Fatalf("handled %d data lines, want 10", n)
if n != 25 {
t.Fatalf("handled %d data lines, want 25", n)
}
}
+127
View File
@@ -63,6 +63,14 @@ import (
// rest of a key from prose written after a whole one: that prose would be eaten,
// and a following JWT would lose its header.
//
// CALLER-SUPPLIED SECRETS (Options.ExtraSecretValues) get the same treatment
// from redactExtraSecretValues. Such a value is often opaque, matching no shape
// here, so without its own gap-tolerant match a separator inside it left both
// fragments in the output. Unlike a shape, the value is known exactly and has no
// word boundary to judge, so it is found in a copy with the separators removed
// and mapped back. RedactString runs it first, before any shape matcher, so a
// caller that clips or reformats the result afterwards never sees a fragment.
//
// WHAT THIS DELIBERATELY DOES NOT TREAT AS A SEPARATOR: tab, newline and
// carriage return. Those three are real text structure rather than invisible
// filler, so a credential broken across a line boundary is still two visible
@@ -435,6 +443,125 @@ func openAIShapeIsSecret(match string) bool {
return !strings.Contains(strings.TrimPrefix(match, "sk-"), "-")
}
// redactExtraSecretValues replaces every occurrence of every caller-supplied
// secret, whole or with runs of invisible separators between its characters.
//
// All occurrences, overlapping ones included, are found in the same text and
// overlapping spans are merged before anything is replaced. Replacing one at a
// time let each replacement consume bytes another occurrence needed: a shorter
// value inside a split longer one, a periodic value ("abcabc" in
// "abc<NUL>abcabc"), a value whose first and last characters match, or two
// values that overlap all left a fragment visible. Occurrences that only touch
// stay separate replacements, as they always were.
//
// Separators inside a match are covered by its replacement; a split match never
// starts or ends on one, so those beside it stay, unless the value as written
// starts or ends with them. Tab, newline and carriage return are not separators
// (see above).
func redactExtraSecretValues(value string, secrets []string, replacement string) string {
haystack, offsets := value, []int(nil)
if containsSplitSeparator(value) {
haystack, offsets = compactSplitSeparators(value)
}
var spans [][2]int
for _, secret := range secrets {
if strings.TrimSpace(secret) == "" {
continue
}
needle := secret
if offsets != nil {
// The value as written always matches, as it did before split
// matching existed. The compacted match cannot stand in for it: a
// value made only of separators compacts to nothing, and separators
// at either end of a value fall outside its compacted match.
if strings.Contains(value, secret) {
for _, at := range overlappingIndexes(value, secret) {
spans = append(spans, [2]int{at, at + len(secret)})
}
}
needle = stripSplitSeparators(secret)
}
if needle == "" || !strings.Contains(haystack, needle) {
continue
}
for _, at := range overlappingIndexes(haystack, needle) {
start, end := at, at+len(needle)
if offsets != nil {
start, end = offsets[start], offsets[end-1]+1
}
spans = append(spans, [2]int{start, end})
}
}
if len(spans) == 0 {
return value
}
sort.Slice(spans, func(i, j int) bool { return spans[i][0] < spans[j][0] })
var out strings.Builder
out.Grow(len(value))
written := 0
for i := 0; i < len(spans); {
start, end := spans[i][0], spans[i][1]
for i++; i < len(spans) && spans[i][0] < end; i++ {
end = max(end, spans[i][1])
}
out.WriteString(value[written:start])
out.WriteString(replacement)
written = end
}
out.WriteString(value[written:])
return out.String()
}
// compactSplitSeparators returns value with every splitSecretSeparator removed,
// and for each byte of the result the offset of that byte in value.
func compactSplitSeparators(value string) (string, []int) {
var builder strings.Builder
builder.Grow(len(value))
offsets := make([]int, 0, len(value))
for offset := 0; offset < len(value); {
r, width := utf8.DecodeRuneInString(value[offset:])
if !splitSecretSeparator(r) {
builder.WriteString(value[offset : offset+width])
for i := 0; i < width; i++ {
offsets = append(offsets, offset+i)
}
}
offset += width
}
return builder.String(), offsets
}
// overlappingIndexes returns the start of every occurrence of needle in
// haystack, overlapping ones included, in one linear Knuth-Morris-Pratt pass:
// restarting a search one byte after each match is quadratic for a periodic
// value over text full of it.
func overlappingIndexes(haystack, needle string) []int {
fallback := make([]int, len(needle))
for i, k := 1, 0; i < len(needle); i++ {
for k > 0 && needle[i] != needle[k] {
k = fallback[k-1]
}
if needle[i] == needle[k] {
k++
}
fallback[i] = k
}
var starts []int
for i, k := 0, 0; i < len(haystack); i++ {
for k > 0 && haystack[i] != needle[k] {
k = fallback[k-1]
}
if haystack[i] == needle[k] {
k++
}
if k == len(needle) {
starts = append(starts, i-len(needle)+1)
k = fallback[k-1]
}
}
return starts
}
// redactControlSplitSecrets replaces credentials whose body is interrupted by
// invisible characters. It is a no-op for text containing none of them, which is
// nearly all text, so the contiguous path keeps its behavior exactly.
+108
View File
@@ -105,6 +105,114 @@ func TestRedactStringCoversTheSeparatorsInsideASplitSecret(t *testing.T) {
}
}
// A CALLER-SUPPLIED SECRET IS A CREDENTIAL TOO.
//
// ExtraSecretValues names a value the caller knows is secret, whatever its
// shape: an opaque provider key matches none of the shape matchers, so the
// split pass above never saw it, and a separator inside it left both fragments
// in the output. It has to be matched across the same separators, cover them in
// its replacement, and leave the separators beside it where they were.
func TestRedactStringRedactsExtraSecretValuesSplitByInvisibleCharacters(t *testing.T) {
const secret = "zq7Mv2Lp9Xr4Tn8Wk3"
options := Options{ExtraSecretValues: []string{secret}}
if out := RedactString("key "+secret, Options{}); !strings.Contains(out, secret) {
t.Fatalf("SETUP INVALID: a shape matcher already redacts %q, so this would not reach the explicit-value pass: %q", secret, out)
}
for _, testCase := range []struct {
name string
value string
want string
}{
{"NUL", "key " + secret[:6] + "\x00" + secret[6:], "key [REDACTED]"},
{"zero width space", "key " + secret[:1] + "\u200b" + secret[1:], "key [REDACTED]"},
{"soft hyphen near the end", "key " + secret[:len(secret)-1] + "\u00ad" + secret[len(secret)-1:], "key [REDACTED]"},
{"every gap", "key " + strings.Join(strings.Split(secret, ""), "\x1b"), "key [REDACTED]"},
{"adjacent separators", "key " + secret[:9] + "\x00\x1b\u2060" + secret[9:], "key [REDACTED]"},
{"twice", secret[:3] + "\x00" + secret[3:] + " and " + secret[:12] + "\ufeff" + secret[12:], "[REDACTED] and [REDACTED]"},
{"separators beside it stay", "\x00" + secret[:5] + "\u200b" + secret[5:] + "\x00", "\x00[REDACTED]\x00"},
{"a line break is not a gap", secret[:5] + "\n" + secret[5:], secret[:5] + "\n" + secret[5:]},
} {
t.Run(testCase.name, func(t *testing.T) {
if got := RedactString(testCase.value, options); got != testCase.want {
t.Errorf("RedactString(%q) = %q, want %q", testCase.value, got, testCase.want)
}
})
}
// A SHORTER SECRET MUST NOT CLAIM PART OF A SPLIT LONGER ONE. Longest first
// has to hold across the contiguous and split matches together: replacing
// a contiguous shorter value first left the rest of the split longer one
// ("[REDACTED]\x00Lp9Xr4Tn8Wk3") visible.
for _, testCase := range []struct {
name string
secrets []string
value string
want string
}{
{"prefix", []string{secret[:6], secret}, "key " + secret[:6] + "\x00" + secret[6:], "key [REDACTED]"},
{"suffix", []string{secret[9:], secret}, "key " + secret[:9] + "\u200b" + secret[9:], "key [REDACTED]"},
{"inside", []string{secret[4:10], secret}, "key " + secret[:2] + "\x00" + secret[2:], "key [REDACTED]"},
{"shorter alone too", []string{secret, secret[:6]}, secret[:6] + " and " + secret[:6] + "\x00" + secret[6:], "[REDACTED] and [REDACTED]"},
} {
t.Run("overlap "+testCase.name, func(t *testing.T) {
if got := RedactString(testCase.value, Options{ExtraSecretValues: testCase.secrets}); got != testCase.want {
t.Errorf("RedactString(%q, %q) = %q, want %q", testCase.value, testCase.secrets, got, testCase.want)
}
})
}
// OVERLAPPING OCCURRENCES ARE ONE SPAN. Replacing one occurrence (either
// form) and then looking for the next consumed bytes the other needed:
// "abc\x00abcabc" holds a split "abc\x00abc" overlapping a whole "abcabc",
// and replacing the whole one first left "abc\x00" in front of it. Every
// occurrence of every value, in either form, has to be found in the same
// text and overlapping ones replaced together.
for _, testCase := range []struct {
name string
secrets []string
value string
want string
}{
{"split then whole, periodic secret", []string{"abcabc"}, "abc\x00abcabc", "[REDACTED]"},
{"whole then split, periodic secret", []string{"abcabc"}, "abcabc\x00abc", "[REDACTED]"},
{"shared first and last character", []string{"xY7qx"}, "key xY7q\u200bxY7qx", "key [REDACTED]"},
{"two secrets overlapping across a split", []string{"zq7Mv2Lp", "Lp9Xr4Tn"}, "key zq7Mv2L\x00p9Xr4Tn", "key [REDACTED]"},
{"whole occurrences overlapping", []string{"xY7qx"}, "xY7qxY7qx", "[REDACTED]"},
{"adjacent occurrences stay two", []string{"abc"}, "abc\x00abc", "[REDACTED]\x00[REDACTED]"},
} {
t.Run("overlapping occurrences "+testCase.name, func(t *testing.T) {
if got := RedactString(testCase.value, Options{ExtraSecretValues: testCase.secrets}); got != testCase.want {
t.Errorf("RedactString(%q, %q) = %q, want %q", testCase.value, testCase.secrets, got, testCase.want)
}
})
}
// A SECRET AS WRITTEN IS ALWAYS REDACTED. The split match compares the text
// and the value with separators removed; it adds to the whole match and
// must never replace it. Once the text held any separator, a value made only
// of separators compacted to nothing and was skipped, and separators at
// either end of a value fell outside its compacted match, so both stayed
// visible where a plain replacement had always caught them.
for _, testCase := range []struct {
name string
secrets []string
value string
want string
}{
{"separator-only secret", []string{"\x00"}, "a\x00b", "a[REDACTED]b"},
{"separator-only secret beside another separator", []string{"\u200b"}, "a\u200bb\x00", "a[REDACTED]b\x00"},
{"separators at both ends", []string{"\u200babc\x00"}, "x\u200babc\x00y", "x[REDACTED]y"},
} {
t.Run("as written "+testCase.name, func(t *testing.T) {
if got := RedactString(testCase.value, Options{ExtraSecretValues: testCase.secrets}); got != testCase.want {
t.Errorf("RedactString(%q, %q) = %q, want %q", testCase.value, testCase.secrets, got, testCase.want)
}
})
}
// A value the caller names that is NOT in the text leaves the text alone,
// separators and all.
if value := "unrelated\x00text \u200bhere"; RedactString(value, options) != value {
t.Errorf("RedactString rewrote text without the secret in it: %q", RedactString(value, options))
}
}
// NOTHING ELSE MOVES. The compaction exists to find a match; it must never
// reach text that has no secret in it, and text with no invisible character at
// all must take exactly the path it took before.
+1 -9
View File
@@ -463,15 +463,7 @@ func RedactString(value string, options Options) string {
replacement := replacement(options)
redacted := value
if len(options.ExtraSecretValues) > 0 {
secrets := append([]string{}, options.ExtraSecretValues...)
sort.SliceStable(secrets, func(i, j int) bool {
return len(secrets[i]) > len(secrets[j])
})
for _, secret := range secrets {
if strings.TrimSpace(secret) != "" {
redacted = strings.ReplaceAll(redacted, secret, replacement)
}
}
redacted = redactExtraSecretValues(redacted, options.ExtraSecretValues, replacement)
}
if strings.Contains(redacted, "-----BEGIN ") && strings.Contains(redacted, "PRIVATE KEY") {
-35
View File
@@ -1,35 +0,0 @@
package sandbox
import (
"os"
"testing"
)
// TestMain points the sandbox runtime's user-cache root at test-owned storage
// for the whole package.
//
// PLAN CONSTRUCTION CREATES DIRECTORIES, which is easy to miss because it reads
// like naming. windowsSandboxProfileWithProvisionedRuntime provisions the root
// it selects, and the simulated-Windows tests run on every platform, so a test
// that supplies an explicit child environment but leaves the cache alone writes
// into the developer's real one. Not hypothetical: the machine this was found on
// had accumulated thousands of entries under the real runtime root from exactly
// these runs.
//
// Done once for the package rather than per test, because the leak is in the
// DEFAULT: any new test that builds a Windows plan is affected unless its author
// remembers, and remembering is what failed here. A test that needs a specific
// cache root still overrides sandboxUserCacheDir itself.
func TestMain(m *testing.M) {
root, err := os.MkdirTemp("", "zero-sandbox-testcache-")
if err != nil {
// Fail loudly rather than silently falling back to the real cache.
panic("sandbox tests: create the test cache root: " + err.Error())
}
sandboxUserCacheDir = func() (string, error) { return root, nil }
code := m.Run()
_ = os.RemoveAll(root)
os.Exit(code)
}
+1 -12
View File
@@ -189,18 +189,7 @@ func (engine *Engine) BuildCommandPlan(spec CommandSpec) (CommandPlan, error) {
}
var runtimeCleanup func()
if preference != SandboxPreferenceForbid && policy.Mode != ModeDisabled {
// The home THIS command asked for. Windows planning resolves
// ZERO_WINDOWS_SANDBOX_HOME out of spec.Env and hands it to the runner for
// marker validation, so selection has to read the same environment or the
// two disagree about which marker describes the tree. See
// pinnedSandboxRuntimeRoot.
commandSandboxHome := ""
if spec.Env != nil {
if resolved, err := ResolveWindowsSandboxHome(envListToMap(spec.Env)); err == nil {
commandSandboxHome = resolved
}
}
runtimeState, cleanup, runtimeErr := prepareSandboxRuntime(workspaceRoot, commandSandboxHome)
runtimeState, cleanup, runtimeErr := prepareSandboxRuntime(workspaceRoot)
if runtimeErr != nil {
return CommandPlan{}, runtimeErr
}
@@ -393,11 +393,10 @@ func realSmokeExecutable(t *testing.T, envKey string, fallbackName string) strin
func runWindowsRealSmokeSetup(t *testing.T, setupExe string, options WindowsSandboxSetupArgsOptions) {
t.Helper()
setupPlan, err := BuildWindowsSandboxSetupArgs(options)
args, err := BuildWindowsSandboxSetupArgs(options)
if err != nil {
t.Fatalf("BuildWindowsSandboxSetupArgs: %v", err)
}
args := setupPlan.Args
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, setupExe, args...)
@@ -407,30 +406,8 @@ func runWindowsRealSmokeSetup(t *testing.T, setupExe string, options WindowsSand
}
}
// prepareWindowsRealSmokeCommand gives an elevated-tier smoke command the profile
// the planner would have given it. The elevated runner validates the command
// against the setup marker, and setup fingerprinted a profile with the selected
// runtime folded in, so a command built from the bare profile can only be
// refused. The unelevated tier applies its own plan and has no marker to agree
// with, so it is left exactly as the test wrote it.
func prepareWindowsRealSmokeCommand(t *testing.T, base WindowsSandboxCommandArgsOptions) (WindowsSandboxCommandArgsOptions, func()) {
t.Helper()
if base.SandboxLevel != WindowsSandboxLevelRestrictedToken || base.PermissionProfile.Runtime != nil {
return base, func() {}
}
workspace := firstNonEmpty(base.WorkspaceRoots...)
prepared, release, err := plannerPreparedWindowsProfile(workspace, base.SandboxHome, base.PermissionProfile)
if err != nil {
t.Fatalf("prepare the smoke command the way the planner does: %v", err)
}
base.PermissionProfile = prepared
return base, release
}
func runWindowsRealSmokeCommand(t *testing.T, runnerExe string, base WindowsSandboxCommandArgsOptions, command []string, wantCode int) {
t.Helper()
base, release := prepareWindowsRealSmokeCommand(t, base)
defer release()
base.Command = command
args, err := BuildWindowsSandboxCommandArgs(base)
if err != nil {
@@ -456,8 +433,6 @@ func runWindowsRealSmokeCommand(t *testing.T, runnerExe string, base WindowsSand
// explicit unsupported-mode rejections rather than sandboxed command failures).
func runWindowsRealSmokeCommandExpectError(t *testing.T, runnerExe string, base WindowsSandboxCommandArgsOptions, command []string, wantSubstr ...string) {
t.Helper()
base, release := prepareWindowsRealSmokeCommand(t, base)
defer release()
base.Command = command
args, err := BuildWindowsSandboxCommandArgs(base)
if err != nil {
@@ -1,88 +0,0 @@
package sandbox
import (
"os"
"path/filepath"
"testing"
)
// THE TWO HALVES OF A ROLLBACK RECORD MUST DESCRIBE ONE OBJECT.
//
// The snapshot used to read the identity through a handle, close it, and then
// re-resolve the pathname to read the stamp. A rename between those pairs one
// directory's identity with another's bytes, and a rollback that correctly
// proves it holds the first then writes the second's contents into it.
//
// The window itself needs an elevated installer racing an unelevated renamer and
// is not reproducible here, so this pins the contract the binding provides: both
// facts come back together, and they agree with the object actually at the path.
func TestStampSnapshotPairsIdentityWithItsOwnBytes(t *testing.T) {
root := filepath.Join(t.TempDir(), "root")
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
const contents = "the stamp that belongs to this directory"
if err := os.WriteFile(windowsSandboxRuntimeStampPath(root, testStampPlanHash), []byte(contents), 0o600); err != nil {
t.Fatal(err)
}
identity, identified, prior, state, err := snapshotRuntimeStampBound(root, windowsSandboxRuntimeStampName(testStampPlanHash))
if err != nil {
t.Fatalf("snapshot: %v", err)
}
existed := state == runtimeStampPresent
if !identified {
t.Fatal("the snapshot established no identity for a directory that exists")
}
if !existed || string(prior) != contents {
t.Fatalf("prior stamp = %q existed=%v, want %q", string(prior), existed, contents)
}
if direct, ok := runtimeDirIdentity(root); !ok || direct != identity {
t.Errorf("snapshot identity %q does not describe the directory at the path (%q)", identity, direct)
}
}
// An absent stamp still establishes the identity, because that came from the
// directory handle and not from the stamp read.
func TestStampSnapshotIdentifiesARootWithNoStamp(t *testing.T) {
root := filepath.Join(t.TempDir(), "root")
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
identity, identified, _, state, err := snapshotRuntimeStampBound(root, windowsSandboxRuntimeStampName(testStampPlanHash))
if err != nil {
t.Fatalf("snapshot: %v", err)
}
existed := state == runtimeStampPresent
if !identified || identity == "" {
t.Error("a root with no stamp established no identity")
}
if existed {
t.Error("reported a prior stamp that does not exist")
}
}
// A created directory's identity must come from the creation, so the ledger
// cannot record an object this run did not make.
func TestCreatedDirectoryIdentityDescribesWhatWasCreated(t *testing.T) {
path := filepath.Join(t.TempDir(), "created")
identity, identified, err := createRuntimeDirIdentified(path)
if err != nil {
t.Fatalf("create: %v", err)
}
if !identified || identity == "" {
t.Fatal("creation established no identity")
}
if info, statErr := os.Stat(path); statErr != nil || !info.IsDir() {
t.Fatalf("the directory was not created: %v", statErr)
}
if direct, ok := runtimeDirIdentity(path); !ok || direct != identity {
t.Errorf("creation identity %q does not describe the directory now at the path (%q)", identity, direct)
}
// Creating over something that exists is the caller's already-handled signal.
if _, _, again := createRuntimeDirIdentified(path); !os.IsExist(again) {
t.Errorf("creating over an existing directory returned %v, want an IsExist error", again)
}
}
@@ -1,97 +0,0 @@
//go:build windows
package sandbox
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
)
// ABSENCE IS CLASSIFIED IN THE PRODUCER'S DOMAIN.
//
// The stamp is opened through NtCreateFile, so a stamp that is not there comes
// back as STATUS_OBJECT_NAME_NOT_FOUND. Compensation compared it with the Win32
// codes only. An NTStatus is a different type and errors.Is does not convert it,
// so removing a stamp that never existed was reported as a rollback failure.
//
// Native, against real directories: the question is what the API returns at this
// boundary, and a synthetic error value would only test the comparison.
func runtimeRootForCompensation(t *testing.T) (string, string) {
t.Helper()
root := filepath.Join(t.TempDir(), "zero", "runtime", "v1", "abcd")
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatalf("create the runtime root: %v", err)
}
identity, ok := runtimeDirIdentity(root)
if !ok {
t.Fatal("identify the runtime root")
}
return root, identity
}
// A runtime directory that exists and holds no stamp: removal has nothing to do
// and says so by succeeding.
func TestRemovingAStampThatIsNotThereSucceeds(t *testing.T) {
root, identity := runtimeRootForCompensation(t)
name := windowsSandboxRuntimeStampName(testStampPlanHash)
if err := compensateRuntimeStampBound(root, identity, name, nil, false); err != nil {
t.Fatalf("removing a stamp that was never written reported a failure: %v", err)
}
if _, err := os.Stat(root); err != nil {
t.Errorf("the runtime directory itself was disturbed: %v", err)
}
}
// A fresh setup that fails BEFORE its stamp is written, composed the way the
// setup transaction composes it: a real snapshot of the untouched tree, and the
// compensation run with the original failure as its cause. The cause has to come
// back alone. A second error about the stamp sends the operator looking for
// residue that does not exist.
func TestAFreshSetupFailureBeforeTheStampReportsOnlyItsCause(t *testing.T) {
root, _ := runtimeRootForCompensation(t)
snapshot, err := snapshotWindowsSandboxRuntimeStamp(root, testStampPlanHash)
if err != nil {
t.Fatalf("snapshot: %v", err)
}
if snapshot.priorState != runtimeStampAbsent || !snapshot.rootIdentified {
t.Fatalf("SETUP INVALID: the snapshot did not record an identified root with no stamp: %+v", snapshot)
}
cause := errors.New("windows ACL target does not exist: an earlier entry in the plan")
reported := runWindowsSandboxSetupCompensations(cause, nil, windowsRuntimeRootRollback{stamp: snapshot})
if reported == nil || !errors.Is(reported, cause) {
t.Fatalf("the original failure is no longer visible: %v", reported)
}
if text := reported.Error(); strings.Contains(text, "rollback failed") || strings.Contains(text, "setup stamp") {
t.Errorf("compensation added a failure of its own for a stamp that never existed:\n%s", text)
}
}
// AND ONLY ABSENCE. Something at the stamp's name that cannot be opened as the
// stamp is not "already removed", and treating every failed open as success
// would report a clean undo over residue nobody looked at. A directory at that
// name is a failure the real API produces on demand.
func TestAnUninspectableStampIsStillReported(t *testing.T) {
root, identity := runtimeRootForCompensation(t)
name := windowsSandboxRuntimeStampName(testStampPlanHash)
if err := os.Mkdir(filepath.Join(root, name), 0o700); err != nil {
t.Fatalf("put a directory where the stamp would be: %v", err)
}
err := compensateRuntimeStampBound(root, identity, name, nil, false)
if err == nil {
t.Fatal("compensation reported success over something it could not open as the stamp")
}
if isWindowsNotFound(err) {
t.Errorf("the failure classifies as not-found, so it would be swallowed: %v", err)
}
if _, statErr := os.Stat(filepath.Join(root, name)); statErr != nil {
t.Errorf("the object it could not inspect was removed anyway: %v", statErr)
}
}
@@ -1,112 +0,0 @@
package sandbox
import (
"os"
"path/filepath"
"strings"
"testing"
)
// COMPENSATION MUST PROVE IT HOLDS THE OBJECT IT CHANGED.
//
// The forward apply and its stamp go through one handle, so they are provably
// about one object. Compensation runs later, after those handles have closed,
// and used to resolve the pathname again. Rename the original aside, put an
// ordinary directory at the name, and a pathname-only undo strips a stamp from
// the substitute, or writes bytes snapshotted from another object onto it, and
// then removes it as though this run had created it, while the moved original
// keeps this run's grant and stamp.
func TestStampCompensationRefusesAReplacementDirectory(t *testing.T) {
parent := t.TempDir()
root := filepath.Join(parent, "runtime-root")
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
stampPath := windowsSandboxRuntimeStampPath(root, testStampPlanHash)
if err := os.WriteFile(stampPath, []byte("previous-plan-hash"), 0o600); err != nil {
t.Fatal(err)
}
snapshot, err := snapshotWindowsSandboxRuntimeStamp(root, testStampPlanHash)
if err != nil {
t.Fatalf("snapshot: %v", err)
}
if snapshot.priorState != runtimeStampPresent {
t.Fatal("SETUP INVALID: the snapshot did not record the pre-existing stamp")
}
// The original is moved aside and an ordinary directory takes the name.
moved := filepath.Join(parent, "moved-aside")
if err := os.Rename(root, moved); err != nil {
t.Skipf("cannot rename the runtime root on this filesystem: %v", err)
}
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
substituteStamp := filepath.Join(root, filepath.Base(stampPath))
if err := os.WriteFile(substituteStamp, []byte("not-ours"), 0o600); err != nil {
t.Fatal(err)
}
err = snapshot.restore()
if err == nil {
t.Fatal("compensation mutated a directory it never touched and reported success")
}
for _, want := range []string{"no longer the directory", "replacement"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the error does not say what was left behind (%q): %v", want, err)
}
}
if body, readErr := os.ReadFile(substituteStamp); readErr != nil || string(body) != "not-ours" {
t.Errorf("the substitute's stamp was overwritten: %q %v", string(body), readErr)
}
if body, readErr := os.ReadFile(filepath.Join(moved, filepath.Base(stampPath))); readErr != nil || string(body) != "previous-plan-hash" {
t.Errorf("the original lost its stamp: %q %v", string(body), readErr)
}
}
// And the created-directory ledger applies the same rule, or the stamp check
// just moves the damage one line down.
func TestCreatedDirectoryCompensationRefusesAReplacement(t *testing.T) {
parent := t.TempDir()
root := filepath.Join(parent, "created-root")
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
rollback := windowsRuntimeRootRollback{created: createdRuntimeDirsForTest(root)}
moved := filepath.Join(parent, "moved")
if err := os.Rename(root, moved); err != nil {
t.Skipf("cannot rename on this filesystem: %v", err)
}
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
err := rollback.run()
if err == nil {
t.Fatal("a substitute directory was removed as though this run had created it")
}
if !strings.Contains(err.Error(), "no longer the directory this run created") {
t.Errorf("the error does not name the replacement: %v", err)
}
if _, statErr := os.Stat(root); statErr != nil {
t.Errorf("the substitute was removed: %v", statErr)
}
}
// The ordinary case still cleans up, or the guard would be refusing everything.
func TestCompensationStillRemovesWhatThisRunCreated(t *testing.T) {
parent := t.TempDir()
root := filepath.Join(parent, "mine")
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
rollback := windowsRuntimeRootRollback{created: createdRuntimeDirsForTest(root)}
if err := rollback.run(); err != nil {
t.Fatalf("compensation refused a directory it really did create: %v", err)
}
if _, err := os.Stat(root); !os.IsNotExist(err) {
t.Errorf("the directory this run created survived compensation: %v", err)
}
}
@@ -1,67 +0,0 @@
//go:build !windows
package sandbox
import (
"errors"
"fmt"
"os"
"path/filepath"
)
// runtimeCompensationSwapSeam exists so the shared compensation code compiles
// everywhere. Only the Windows build closes a check-then-mutate window, because
// only there does setup run elevated against a tree an unelevated process can
// rename.
var runtimeCompensationSwapSeam func()
func compensateRuntimeStampBound(root string, identity string, name string, prior []byte, existed bool) error {
current, ok := runtimeDirIdentity(root)
if !ok {
return fmt.Errorf("identify the sandbox runtime root %s for stamp compensation", root)
}
if current != identity {
return fmt.Errorf("sandbox runtime root %s is no longer the directory this setup stamped; "+
"leaving the replacement untouched, and the original still carries this run's stamp", root)
}
if runtimeCompensationSwapSeam != nil {
runtimeCompensationSwapSeam()
}
path := filepath.Join(root, name)
if !existed {
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("remove sandbox runtime setup stamp written by this run: %w", err)
}
return nil
}
if err := os.WriteFile(path, prior, 0o600); err != nil {
return fmt.Errorf("restore the previous sandbox runtime setup stamp: %w", err)
}
return nil
}
func removeCreatedRuntimeDirBound(path string, identity string) error {
current, ok := runtimeDirIdentity(path)
if !ok {
// ONLY ABSENCE MEANS THERE IS NOTHING TO UNDO. Any lookup failure used to
// return here, so a directory that exists but could not be identified was
// reported as cleanly removed while it stayed on disk. The Windows build
// already draws this line, and the rollback it serves promises to report
// what it could not remove. Reported by CodeRabbit.
if _, err := os.Lstat(path); errors.Is(err, os.ErrNotExist) {
return nil
}
return fmt.Errorf("sandbox runtime root %s was created by this run but could not be identified for removal; leaving it in place", path)
}
if current != identity {
return fmt.Errorf("sandbox runtime root %s is no longer the directory this run created; "+
"leaving the replacement in place", path)
}
if runtimeCompensationSwapSeam != nil {
runtimeCompensationSwapSeam()
}
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("remove sandbox runtime root %s created by this run: %w", path, err)
}
return nil
}
@@ -1,74 +0,0 @@
//go:build !windows
package sandbox
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
)
// ONLY ABSENCE IS NOTHING TO UNDO.
//
// removeCreatedRuntimeDirBound returned success on any failure to identify the
// directory, so one that was still there, but could not be looked at, was
// reported as removed. Here the parent loses search permission, which makes the
// directory exist and be unidentifiable at the same time.
func TestRemovingAnUnidentifiableCreatedDirectoryIsReported(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("root bypasses directory search permission, so the lookup cannot be made to fail")
}
parent := filepath.Join(t.TempDir(), "zero")
created := filepath.Join(parent, "runtime")
if err := os.MkdirAll(created, 0o700); err != nil {
t.Fatalf("create the directory: %v", err)
}
identity, ok := runtimeDirIdentity(created)
if !ok {
t.Fatal("identify the created directory")
}
if err := os.Chmod(parent, 0o600); err != nil {
t.Fatalf("withdraw search permission: %v", err)
}
t.Cleanup(func() { _ = os.Chmod(parent, 0o700) })
if _, err := os.Lstat(created); err == nil || errors.Is(err, os.ErrNotExist) {
t.Fatalf("SETUP INVALID: the directory should exist and be unidentifiable, Lstat said %v", err)
}
err := removeCreatedRuntimeDirBound(created, identity)
if err == nil {
t.Fatal("a created directory that could not be identified was reported as removed")
}
if !strings.Contains(err.Error(), "could not be identified") {
t.Errorf("the error does not say why the directory was left: %v", err)
}
// And it is still there, which is why the error is owed.
if err := os.Chmod(parent, 0o700); err != nil {
t.Fatalf("restore search permission: %v", err)
}
if _, err := os.Lstat(created); err != nil {
t.Fatalf("the directory was gone after all, so the error was not owed: %v", err)
}
}
// Absence stays a clean undo, or the test above would be satisfied by a function
// that reports every lookup failure.
func TestRemovingACreatedDirectoryThatIsAlreadyGoneSucceeds(t *testing.T) {
created := filepath.Join(t.TempDir(), "runtime")
if err := os.Mkdir(created, 0o700); err != nil {
t.Fatalf("create the directory: %v", err)
}
identity, ok := runtimeDirIdentity(created)
if !ok {
t.Fatal("identify the created directory")
}
if err := os.Remove(created); err != nil {
t.Fatalf("remove the directory: %v", err)
}
if err := removeCreatedRuntimeDirBound(created, identity); err != nil {
t.Fatalf("a created directory that is already gone was reported as a failed undo: %v", err)
}
}
@@ -1,100 +0,0 @@
//go:build windows
package sandbox
import (
"os"
"path/filepath"
"testing"
)
// swapDuringCompensation installs a seam that runs once, between the identity
// check and the mutation, and replaces root with a fresh directory carrying a
// file of its own.
func swapDuringCompensation(t *testing.T, root string, aside string) *string {
t.Helper()
fired := false
previous := runtimeCompensationSwapSeam
t.Cleanup(func() { runtimeCompensationSwapSeam = previous })
runtimeCompensationSwapSeam = func() {
if fired {
return
}
fired = true
if err := os.Rename(root, aside); err != nil {
t.Logf("could not rename the original aside: %v", err)
return
}
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatalf("create the substitute: %v", err)
}
if err := os.WriteFile(filepath.Join(root, "substitute.txt"), []byte("belongs to whoever put it here"), 0o600); err != nil {
t.Fatalf("seed the substitute: %v", err)
}
}
return &aside
}
// A REPLACEMENT AFTER THE CHECK MUST NOT REACH THE SUBSTITUTE.
//
// Compensation used to read the identity through a handle, close it, and then
// resolve the pathname again for the write or the delete. A rename plus a
// replacement in that interval made the comparison true about one object while
// the mutation landed on another, under elevation. Holding the handle across
// both means the mutation follows the object that was verified, and whatever now
// answers to the name is untouched.
func TestStampCompensationDoesNotReachASubstitute(t *testing.T) {
parent := t.TempDir()
root := filepath.Join(parent, "zero", "runtime", "v1", "abcd")
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatalf("create the runtime root: %v", err)
}
if err := writeWindowsSandboxRuntimeStamp(root, "planhash"); err != nil {
t.Fatalf("write the stamp: %v", err)
}
identity, ok := runtimeDirIdentity(root)
if !ok {
t.Fatal("identify the runtime root")
}
aside := filepath.Join(parent, "renamed-aside")
swapDuringCompensation(t, root, aside)
// Removal of a stamp this run wrote, i.e. the fresh-setup rollback.
_ = compensateRuntimeStampBound(root, identity, windowsSandboxRuntimeStampName(testStampPlanHash), nil, false)
// Whatever the outcome, the substitute is not this run's business.
substitute := filepath.Join(root, "substitute.txt")
if _, err := os.Stat(substitute); err != nil {
t.Errorf("compensation removed a file from the substitute directory: %v", err)
}
if _, err := os.Stat(windowsSandboxRuntimeStampPath(root, testStampPlanHash)); err == nil {
t.Error("compensation created a stamp inside the substitute directory")
}
// And the original, which is the object that was verified, is the one that
// lost the stamp this run wrote.
if _, err := os.Stat(windowsSandboxRuntimeStampPath(aside, testStampPlanHash)); err == nil {
t.Error("the stamp this run wrote is still on the original object, so compensation followed the name instead")
}
}
// The same for the delete, where following the name would remove a directory
// this run never created.
func TestDirectoryCompensationDoesNotRemoveASubstitute(t *testing.T) {
parent := t.TempDir()
created := filepath.Join(parent, "created")
if err := os.MkdirAll(created, 0o700); err != nil {
t.Fatalf("create the directory: %v", err)
}
identity, ok := runtimeDirIdentity(created)
if !ok {
t.Fatal("identify the created directory")
}
aside := filepath.Join(parent, "renamed-aside")
swapDuringCompensation(t, created, aside)
_ = removeCreatedRuntimeDirBound(created, identity)
if _, err := os.Stat(filepath.Join(created, "substitute.txt")); err != nil {
t.Errorf("compensation removed the substitute directory's contents: %v", err)
}
}
@@ -1,78 +0,0 @@
//go:build windows
package sandbox
import (
"errors"
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
)
// ONLY NOT-FOUND IS EVIDENCE OF REMOVAL.
//
// The verification probe used to read any Lstat error as absence, so a sharing
// violation, an access denial, or an entry left delete-pending by another
// process that holds a share-delete handle all reported complete compensation
// for a directory that is still there. A holder able to clear the disposition
// could then make the "removed" object visible again, and setup would already
// have said the rollback finished.
//
// The three outcomes are distinguished with a probe seam, because a real
// filesystem will not produce the third on demand.
func TestDeletionVerificationDistinguishesAllThreeOutcomes(t *testing.T) {
previous := runtimeCompensationStat
t.Cleanup(func() { runtimeCompensationStat = previous })
newDir := func() (string, string) {
t.Helper()
dir := filepath.Join(t.TempDir(), "created")
if err := os.MkdirAll(dir, 0o700); err != nil {
t.Fatal(err)
}
identity, ok := runtimeDirIdentity(dir)
if !ok {
t.Fatal("identify the created directory")
}
return dir, identity
}
t.Run("absence is proven and reported as success", func(t *testing.T) {
dir, identity := newDir()
runtimeCompensationStat = func(string) (fs.FileInfo, error) {
return nil, &os.PathError{Op: "lstat", Path: dir, Err: os.ErrNotExist}
}
if err := removeCreatedRuntimeDirBound(dir, identity); err != nil {
t.Errorf("a proven-absent directory was reported as a failure: %v", err)
}
})
t.Run("still present is reported", func(t *testing.T) {
dir, identity := newDir()
runtimeCompensationStat = func(path string) (fs.FileInfo, error) { return os.Stat(".") }
err := removeCreatedRuntimeDirBound(dir, identity)
if err == nil {
t.Fatal("a directory still present after deletion was reported as removed")
}
if !strings.Contains(err.Error(), "still present") {
t.Errorf("unexpected error: %v", err)
}
})
t.Run("an unverifiable probe is residue, not success", func(t *testing.T) {
dir, identity := newDir()
// What a share-delete holder produces: neither absence nor presence.
runtimeCompensationStat = func(path string) (fs.FileInfo, error) {
return nil, &os.PathError{Op: "lstat", Path: path, Err: errors.New("Access is denied.")}
}
err := removeCreatedRuntimeDirBound(dir, identity)
if err == nil {
t.Fatal("an unverifiable removal was reported as complete compensation")
}
if !strings.Contains(err.Error(), "could not be verified") {
t.Errorf("the error does not say the removal is unproven: %v", err)
}
})
}
@@ -1,195 +0,0 @@
//go:build windows
package sandbox
import (
"errors"
"fmt"
"os"
"unsafe"
"golang.org/x/sys/windows"
)
// THE HANDLE IS THE OBJECT. A pathname is not.
//
// Compensation used to read the directory identity through a handle, close it,
// and then resolve the pathname again for the mutation. A rename followed by a
// replacement between those two steps makes the comparison true about one
// object while the write or delete lands on another, and this runs elevated, so
// a redirected pathname gives the mutation reach the replacer does not have
// directly.
//
// Opening once and keeping the handle open across BOTH the identity check and
// the mutation removes the interval. The child operations are relative to that
// handle, so no ancestor is re-resolved either.
// runtimeCompensationSwapSeam runs between the identity check and the mutation.
// Nil in production; a test installs one to replace the directory in exactly the
// window this design closes.
var runtimeCompensationSwapSeam func()
// runtimeCompensationStat is the post-deletion existence probe. A var so a test
// can produce the third outcome, an inspection that neither proves absence nor
// presence, which no real filesystem produces on demand.
var runtimeCompensationStat = os.Lstat
type fileDispositionInfo struct {
DeleteFile byte
}
func handleRuntimeIdentity(handle windows.Handle) (string, error) {
var info windows.ByHandleFileInformation
if err := windows.GetFileInformationByHandle(handle, &info); err != nil {
return "", err
}
return fmt.Sprintf("%d:%d:%d", info.VolumeSerialNumber, info.FileIndexHigh, info.FileIndexLow), nil
}
// openVerifiedRuntimeDirectory opens path without following a link at the final
// component and returns it only if it is still the object identity names.
func openVerifiedRuntimeDirectory(path string, identity string, access uint32, what string) (windows.Handle, error) {
utf16Path, err := windows.UTF16PtrFromString(path)
if err != nil {
return 0, err
}
handle, err := windows.CreateFile(
utf16Path,
access|windows.FILE_READ_ATTRIBUTES,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
nil,
windows.OPEN_EXISTING,
windows.FILE_FLAG_BACKUP_SEMANTICS|windows.FILE_FLAG_OPEN_REPARSE_POINT,
0,
)
if err != nil {
return 0, err
}
current, err := handleRuntimeIdentity(handle)
if err != nil {
_ = windows.CloseHandle(handle)
return 0, fmt.Errorf("identify the sandbox runtime directory %s for compensation: %w", path, err)
}
if current != identity {
_ = windows.CloseHandle(handle)
return 0, fmt.Errorf("sandbox runtime root %s is no longer the directory this run %s; "+
"leaving the replacement in place, and the original still carries this run's changes", path, what)
}
return handle, nil
}
// markForDeletion queues the object the handle names for removal. It never
// resolves a pathname, so it can only reach what this process already holds.
func markForDeletion(handle windows.Handle) error {
info := fileDispositionInfo{DeleteFile: 1}
return windows.SetFileInformationByHandle(
handle,
windows.FileDispositionInfo,
(*byte)(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
)
}
// compensateRuntimeStampBound restores or removes the stamp through a handle on
// the directory whose identity still matches.
//
// RESTORED WITHOUT A GAP. This used to delete the stamp and then recreate the
// previous one, so a command validating in between, or a process stopping
// there, found no stamp at all where the previous setup had left a valid one.
// The ordinary writer replaces the stamp in one step and puts the protected
// DACL back, which a raw write would not, so a restore is just a publish of the
// previous bytes. Only the branch that had no stamp before deletes, because
// absent is the state it restores. Reported by @jatmn.
func compensateRuntimeStampBound(root string, identity string, name string, prior []byte, existed bool) error {
directory, err := openVerifiedRuntimeDirectory(root, identity,
windows.FILE_TRAVERSE|windowsFileAddFile|windows.READ_CONTROL|windows.SYNCHRONIZE, "stamped")
if err != nil {
return err
}
defer windows.CloseHandle(directory)
if runtimeCompensationSwapSeam != nil {
runtimeCompensationSwapSeam()
}
if !existed {
return deleteRuntimeStampChild(directory, name)
}
// Through the ordinary writer so it is protected again, and so the reader ACE
// is resolved the same way a fresh setup resolves it.
if err := writeWindowsRuntimeStampToDirectoryHandle(directory, name, string(prior)); err != nil {
return fmt.Errorf("restore the previous sandbox runtime setup stamp: %w", err)
}
return nil
}
// deleteRuntimeStampChild removes the stamp relative to an already verified
// directory handle. A stamp that is not there is the desired end state; anything
// else is reported rather than swallowed.
func deleteRuntimeStampChild(directory windows.Handle, name string) error {
stamp, err := openWindowsChildNoFollow(directory, name,
windows.DELETE|windows.FILE_READ_ATTRIBUTES, windows.FILE_NON_DIRECTORY_FILE)
if err != nil {
// ABSENCE IS CLASSIFIED BY THE PRODUCER'S DOMAIN. openWindowsChildNoFollow
// goes through NtCreateFile, so a stamp that is not there arrives as
// STATUS_OBJECT_NAME_NOT_FOUND. This compared against the Win32 codes only,
// and an NTStatus is a different type that errors.Is does not convert, so a
// fresh setup that failed before writing any stamp reported a second,
// spurious rollback failure for removing a file that never existed. The
// snapshot path already answers this question for both domains, and using
// the same answer keeps snapshot and compensation agreeing about what
// absence means. Reported by @jatmn.
if isWindowsNotFound(err) {
return nil
}
return fmt.Errorf("remove sandbox runtime setup stamp written by this run: %w", err)
}
if err := markForDeletion(stamp); err != nil {
_ = windows.CloseHandle(stamp)
return fmt.Errorf("remove sandbox runtime setup stamp written by this run: %w", err)
}
// The entry goes when the last handle closes.
_ = windows.CloseHandle(stamp)
return nil
}
// removeCreatedRuntimeDirBound removes a directory this run created, through a
// handle on the object identity names.
func removeCreatedRuntimeDirBound(path string, identity string) error {
handle, err := openVerifiedRuntimeDirectory(path, identity, windows.DELETE, "created")
if err != nil {
if isWindowsNotFound(err) {
return nil
}
return err
}
if runtimeCompensationSwapSeam != nil {
runtimeCompensationSwapSeam()
}
if err := markForDeletion(handle); err != nil {
_ = windows.CloseHandle(handle)
return fmt.Errorf("remove sandbox runtime root %s created by this run: %w", path, err)
}
// The entry goes when the last handle closes, so releasing ours is what makes
// the removal observable.
_ = windows.CloseHandle(handle)
// REPORTED SUCCESS IS NOT PROOF. A handle-bound operation accepting the call
// has been seen not to take effect (PR #751, the promote rename), so the
// outcome is checked rather than assumed.
//
// THREE OUTCOMES, NOT TWO. This used to read any Lstat error as absence, so a
// sharing violation, an access denial, or a delete-pending entry held open by
// another process all reported complete compensation for a directory that is
// still there. A holder that clears the disposition can then make the
// "removed" object visible again. Only not-found is evidence of removal;
// everything else is at best unproven and has to be reported as residue.
_, statErr := runtimeCompensationStat(path)
switch {
case errors.Is(statErr, os.ErrNotExist):
return nil
case statErr == nil:
return fmt.Errorf("remove sandbox runtime root %s created by this run: it is still present after the deletion was accepted", path)
default:
return fmt.Errorf("remove sandbox runtime root %s created by this run: its removal could not be verified, so it may still be present: %w", path, statErr)
}
}
-15
View File
@@ -1,15 +0,0 @@
package sandbox
// runtimeIdentityAfterCreate resolves a just-created runtime directory's
// identity BY PATHNAME, which is a second resolution of the same name and
// therefore a window: the directory created can be renamed away and an ordinary
// one substituted before this runs, and the ledger then records the substitute
// for a directory the run never made.
//
// It is the honest implementation off Windows, where the elevated-installer
// versus unelevated-renamer split this closes does not apply. On Windows the
// creation returns the handle and identity is read from that, so nothing there
// may call this: a Windows test asserts the count is zero, which is what makes
// "the creation establishes the identity" a checked property rather than a
// comment.
var runtimeIdentityAfterCreate = runtimeDirIdentity
-15
View File
@@ -1,15 +0,0 @@
//go:build !windows
package sandbox
import "os"
// createRuntimeDirIdentified keeps the pathname form off Windows, where the
// elevated-installer-versus-unelevated-renamer split this closes does not apply.
func createRuntimeDirIdentified(path string) (string, bool, error) {
if err := os.Mkdir(path, 0o700); err != nil {
return "", false, err
}
identity, ok := runtimeIdentityAfterCreate(path)
return identity, ok, nil
}
-131
View File
@@ -1,131 +0,0 @@
//go:build windows
package sandbox
import (
"errors"
"fmt"
"os"
"path/filepath"
"unsafe"
"golang.org/x/sys/windows"
)
// createRuntimeDirIdentified creates path and returns the identity of the
// directory it created, read from the handle the creation itself returned.
//
// THE CREATION MUST BE THE THING THAT ESTABLISHES IDENTITY. os.Mkdir followed by
// runtimeDirIdentity(path) is two resolutions of one name: Mkdir creates A, and
// the reopen can land on a B substituted in between, so the rollback ledger
// records B's identity for a directory this run never made. Compensation then
// correctly proves it holds B and deletes it, while A keeps this run's ACL and
// stamp under a name nothing is tracking. The runtime parent belongs to the
// ordinary user, so that substitution needs no privilege.
//
// Win32 CreateFile cannot create a directory at all, whatever disposition it is
// given, so an earlier attempt to do this with CREATE_NEW fell through to the
// Mkdir-plus-reopen path on EVERY real creation and the documented handle
// contract never once held. NtCreateFile with FILE_CREATE and
// FILE_DIRECTORY_FILE does create one, and returns the handle to it.
//
// If the atomic create cannot be completed, this returns an error rather than
// manufacturing an ownership record from a reopen: an unidentified create must
// stop setup before privileged state is applied, not enter the ledger as though
// it were proven.
func createRuntimeDirIdentified(path string) (string, bool, error) {
clean := filepath.Clean(path)
parentPath, leaf := filepath.Split(clean)
leaf = filepath.Clean(leaf)
parentPath = filepath.Clean(parentPath)
if leaf == "" || leaf == "." || parentPath == clean {
return "", false, fmt.Errorf("sandbox runtime path %s has no component to create", path)
}
// The parent is either a directory that already existed when the missing
// components were computed, or one this same loop created a moment ago.
parent, err := openWindowsDirectoryByName(parentPath)
if err != nil {
return "", false, fmt.Errorf("open sandbox runtime parent %s: %w", parentPath, err)
}
defer windows.CloseHandle(parent)
handle, err := createWindowsChildDirectory(parent, leaf)
if err != nil {
// A collision is the same signal os.Mkdir gives for a component another
// process won the race to create, and the caller already treats that as
// "not ours" (windows_setup.go). Returned as a *PathError so BOTH
// os.IsExist, which the caller uses and which does not unwrap %w, and
// errors.Is recognize it; wrapping with %w alone would have turned a
// benign race into a hard setup failure.
if errors.Is(err, windows.STATUS_OBJECT_NAME_COLLISION) {
return "", false, &os.PathError{Op: "mkdir", Path: clean, Err: os.ErrExist}
}
return "", false, fmt.Errorf("create sandbox runtime directory %s: %w", clean, err)
}
defer windows.CloseHandle(handle)
identity, idErr := handleRuntimeIdentity(handle)
if idErr != nil {
return "", false, fmt.Errorf("identify the sandbox runtime directory created at %s: %w", clean, idErr)
}
return identity, true, nil
}
// removeWindowsObjectByHandle deletes the object this handle is open on.
//
// THROUGH THE HANDLE, NOT THE NAME. Every caller is on a failure path partway
// through a creation, and the one object it may remove is the one it just made
// and is still holding. Re-resolving the name to delete it is the substitution
// hole this whole descent exists to close, and here it would be a pathname
// authoritative delete of somebody else's directory.
//
// The handle needs DELETE access, which the creating opens above ask for.
func removeWindowsObjectByHandle(handle windows.Handle) error {
var iosb windows.IO_STATUS_BLOCK
deleteIt := byte(1)
return windows.NtSetInformationFile(handle, &iosb, &deleteIt, 1, windows.FileDispositionInformation)
}
// createWindowsChildDirectory creates exactly one directory component beneath
// parent and returns the handle to the object it created.
//
// Relative to a handle, so no ancestor is re-resolved and there is no interval
// for a swap to land in. FILE_CREATE fails rather than opening anything that is
// already there, so the returned handle cannot describe a pre-existing object,
// and FILE_OPEN_REPARSE_POINT keeps the failure honest if one is.
func createWindowsChildDirectory(parent windows.Handle, name string) (windows.Handle, error) {
objectName, err := windows.NewNTUnicodeString(name)
if err != nil {
return 0, fmt.Errorf("encode sandbox runtime component %s: %w", name, err)
}
attributes := windows.OBJECT_ATTRIBUTES{
RootDirectory: parent,
ObjectName: objectName,
Attributes: windows.OBJ_CASE_INSENSITIVE,
}
attributes.Length = uint32(unsafe.Sizeof(attributes))
var handle windows.Handle
var iosb windows.IO_STATUS_BLOCK
// DELETE because the caller has to be able to undo this creation through the
// handle it just got, without going back through the name. See
// removeWindowsObjectByHandle.
err = windows.NtCreateFile(
&handle,
windows.DELETE|windows.FILE_READ_ATTRIBUTES|windows.FILE_TRAVERSE|windows.SYNCHRONIZE,
&attributes,
&iosb,
nil,
windows.FILE_ATTRIBUTE_DIRECTORY,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
windows.FILE_CREATE,
windows.FILE_DIRECTORY_FILE|windows.FILE_SYNCHRONOUS_IO_NONALERT|windows.FILE_OPEN_REPARSE_POINT,
0,
0,
)
if err != nil {
return 0, err
}
return handle, nil
}
@@ -1,158 +0,0 @@
package sandbox
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
)
// CREATION IS THE OWNERSHIP BOUNDARY, NOT THE STEP AFTER IT.
//
// Acquisition learns it created a directory or a lease file at the create
// itself, but used to publish that fact only once every later step had also
// succeeded: the identity read for a directory, and the inspection, wrapping and
// locking for a lease. A failure in between returned an error while leaving the
// object on disk with nothing above holding a record of it. Setup then reported
// failure and left invocation-owned state behind, and where an ancestor HAD been
// recorded, that ancestor's compensation failed afterwards on a child it could
// not account for.
//
// Both platforms are driven through acquireRuntimeLeaseForPlatform, the call
// production makes, rather than through either descent directly.
var errInjectedRuntimeFailure = errors.New("injected failure after the create")
// failRuntimeCreationAt makes the step after the creation of the named object
// fail, which is the only way to reach the paths under test.
func failRuntimeCreationAt(t *testing.T, match string) {
t.Helper()
runtimeCreationFailure = func(created string) error {
if filepath.Base(created) == match {
return errInjectedRuntimeFailure
}
return nil
}
t.Cleanup(func() { runtimeCreationFailure = nil })
}
// runtimeRootForAcquisition returns a root whose owned tail does not exist yet,
// so acquisition has to create it.
func runtimeRootForAcquisition(t *testing.T) string {
t.Helper()
base := t.TempDir()
if err := os.MkdirAll(filepath.Join(base, "zero", "runtime"), 0o700); err != nil {
t.Fatal(err)
}
return filepath.Join(base, "zero", "runtime", "v1", "digest")
}
// A directory whose identity could not be read is removed, not abandoned.
func TestAcquisitionUndoesADirectoryItCreatedButCouldNotIdentify(t *testing.T) {
root := runtimeRootForAcquisition(t)
version := filepath.Dir(root)
failRuntimeCreationAt(t, filepath.Base(version))
lease, created, err := acquireRuntimeLeaseForPlatform(root)
if err == nil {
lease.release()
t.Fatal("SETUP INVALID: acquisition succeeded, so the injected failure never fired")
}
if !errors.Is(err, errInjectedRuntimeFailure) {
t.Fatalf("acquisition failed for some other reason: %v", err)
}
if len(created) != 0 {
t.Errorf("acquisition returned a ledger for a failed create: %v", created)
}
if _, statErr := os.Stat(version); !os.IsNotExist(statErr) {
t.Errorf("%s was created by this run and left behind after the failure, with nothing recording it (stat: %v)", version, statErr)
}
// And what was already there is untouched.
if _, statErr := os.Stat(filepath.Dir(version)); statErr != nil {
t.Errorf("a directory that existed before the run was removed: %v", statErr)
}
}
// So is a lease file this run created but never got to lock.
func TestAcquisitionUndoesALeaseFileItCreatedButCouldNotLock(t *testing.T) {
root := runtimeRootForAcquisition(t)
leasePath := sandboxRuntimeLeasePath(root)
failRuntimeCreationAt(t, filepath.Base(leasePath))
lease, _, err := acquireRuntimeLeaseForPlatform(root)
if err == nil {
lease.release()
t.Fatal("SETUP INVALID: acquisition succeeded, so the injected failure never fired")
}
if !errors.Is(err, errInjectedRuntimeFailure) {
t.Fatalf("acquisition failed for some other reason: %v", err)
}
if _, statErr := os.Stat(leasePath); !os.IsNotExist(statErr) {
t.Errorf("%s was created by this run and left behind after the failure, so its directory can never be compensated (stat: %v)", leasePath, statErr)
}
}
// A LEASE FILE THAT WAS ALREADY THERE IS SOMEBODY ELSE'S.
//
// The undo is keyed on this call having created the object, so an acquisition
// that merely opened an existing lease must not remove it on its way out.
func TestAcquisitionLeavesALeaseFileItDidNotCreate(t *testing.T) {
root := runtimeRootForAcquisition(t)
if err := os.MkdirAll(filepath.Dir(root), 0o700); err != nil {
t.Fatal(err)
}
leasePath := sandboxRuntimeLeasePath(root)
if err := os.WriteFile(leasePath, nil, 0o600); err != nil {
t.Fatal(err)
}
failRuntimeCreationAt(t, filepath.Base(leasePath))
lease, _, err := acquireRuntimeLeaseForPlatform(root)
if err == nil {
lease.release()
t.Fatal("SETUP INVALID: acquisition succeeded, so the injected failure never fired")
}
if _, statErr := os.Stat(leasePath); statErr != nil {
t.Errorf("acquisition removed a lease file it did not create: %v", statErr)
}
}
// AND THE LEDGER STILL COVERS WHAT SURVIVES A LATER FAILURE.
//
// Undoing the component that failed must not undo the ones before it: those are
// recorded, and compensation is what removes them.
func TestAcquisitionKeepsTheLedgerForComponentsItFinished(t *testing.T) {
base := t.TempDir()
root := filepath.Join(base, "zero", "runtime", "v1", "digest")
failRuntimeCreationAt(t, "v1")
lease, created, err := acquireRuntimeLeaseForPlatform(root)
if err == nil {
lease.release()
t.Fatal("SETUP INVALID: acquisition succeeded, so the injected failure never fired")
}
var recorded []string
for _, entry := range created {
recorded = append(recorded, filepath.Base(entry.path))
}
// "zero" and "runtime" were created and identified before the injected
// failure, so they belong to the caller to compensate.
if len(recorded) != 2 || recorded[0] != "zero" || recorded[1] != "runtime" {
t.Errorf("the completed components are not on the ledger the caller compensates from: %v", recorded)
}
if _, statErr := os.Stat(filepath.Join(base, "zero", "runtime")); statErr != nil {
t.Errorf("a component that was created and recorded was removed anyway: %v", statErr)
}
if failed := filepath.Join(base, "zero", "runtime", "v1"); !isMissing(failed) {
t.Errorf("%s failed after being created and was left behind", failed)
}
if !strings.Contains(err.Error(), "v1") {
t.Errorf("the error does not name the component that failed: %v", err)
}
}
func isMissing(path string) bool {
_, err := os.Stat(path)
return os.IsNotExist(err)
}
-24
View File
@@ -1,24 +0,0 @@
//go:build !windows
package sandbox
import "golang.org/x/sys/unix"
// createRuntimeTailHandleRelative creates the owned tail through the same
// no-follow boundary the lease uses.
//
// This was an os.Mkdir loop over full pathnames, on the reasoning that the
// elevated-installer-versus-unelevated-renamer split the Windows descent closes
// does not apply here. That is true of the elevation asymmetry and false of the
// substitution: the fallback root is a predictable name under a temp directory
// every local account can write to, so another user can put a link at an owned
// component and have this loop create the rest of the tree inside it. Same
// descent, same refusal, one implementation per platform rather than one
// protected platform.
func createRuntimeTailHandleRelative(base string, tail []string) ([]windowsCreatedRuntimeDir, error) {
created, parent, err := createRuntimeTailRetainingFD(base, tail)
if parent >= 0 {
_ = unix.Close(parent)
}
return created, err
}
-40
View File
@@ -1,40 +0,0 @@
package sandbox
// The two seams the rooted descent exposes, shared by both platform
// implementations so a test asserts the same property on either.
//
// They live apart from the Windows descent they started in because the POSIX
// fallback now takes the same shape, and a seam defined next to one of two
// implementations is how the other one quietly ends up untested.
// runtimeDescentBarrier, when set, runs after the base directory has been opened
// and before the first owned component is touched. It exists so a test can swap
// an owned component for a link at exactly the point the old pathname walk was
// vulnerable, and prove the redirected target is never created or granted. Nil in
// production.
var runtimeDescentBarrier func()
// runtimeCreationFailure, when set, is consulted immediately after this run
// creates a runtime object and before the step that would publish it: a
// directory component's identity read, or the lease file's inspection, wrapping
// and locking. Returning an error stands in for that step failing.
//
// It is the only way to reach those paths, and they are the ones where the
// transaction knows it created something that nothing above it knows about yet.
// The argument is the created component's path, or the lease file's name. Nil in
// production.
var runtimeCreationFailure func(string) error
// runtimeCleanupExclusivityBarrier, when set, runs during compensation while the
// exclusive cleanup lease is held: after it has been taken and before the first
// thing is removed. It exists so a test can put a contender into that window and
// prove the exclusion still covers it, which is precisely where the old shape had
// already handed the lease back. Nil in production.
var runtimeCleanupExclusivityBarrier func()
// runtimeBaseOpenedByName, when set, receives the ONE path this descent opens by
// name. The whole security property is which path that is: the fixed cache or
// temp directory above the owned tail, never a predictable component Zero owns. A
// test can assert it directly instead of inferring it from whether a swap
// happened to be caught, which is not discriminating. Nil in production.
var runtimeBaseOpenedByName func(string)
-171
View File
@@ -1,171 +0,0 @@
//go:build !windows
package sandbox
import (
"errors"
"fmt"
"path/filepath"
"strconv"
"golang.org/x/sys/unix"
)
// createRuntimeTailRetainingFD creates the owned tail of the runtime root
// beneath base, one component at a time, from retained descriptors.
//
// THE NAME IS NOT AUTHORIZATION, ON THIS SIDE EITHER.
//
// The fallback root moved from an atomically minted os.MkdirTemp parent to a
// predictable /tmp/zero-u<uid>/runtime/v1/<digest>. The stable name is required,
// because setup and every later process have to agree on one root without
// talking to each other, but it also means another local account can name the
// first owned component before this one does. refuseAliasedRuntimeComponents
// answers about a component that is ABSENT by saying there is nothing to alias,
// and the code then called os.MkdirAll on the parent and opened <root>.lease by
// full pathname. Both follow a link planted after that answer, so the guard was
// authorizing writes it could not see the destination of.
//
// So the base is opened by name exactly once, and every component below it is
// created or opened relative to a retained descriptor with O_NOFOLLOW. A link at
// an owned component is then an ELOOP or ENOTDIR from the kernel rather than a
// redirection nobody noticed, and no owned name is ever resolved twice.
//
// The base itself is opened WITHOUT O_NOFOLLOW. It belongs to the operator and is
// legitimately a link on macOS, where /tmp resolves through /private. Only the
// tail Zero creates is held to the stricter rule.
func createRuntimeTailRetainingFD(base string, tail []string) ([]windowsCreatedRuntimeDir, int, error) {
if runtimeBaseOpenedByName != nil {
runtimeBaseOpenedByName(base)
}
parent, err := unix.Open(base, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
if err != nil {
return nil, -1, fmt.Errorf("open sandbox runtime base %s: %w", base, err)
}
if runtimeDescentBarrier != nil {
runtimeDescentBarrier()
}
var created []windowsCreatedRuntimeDir
for _, path := range tail {
name := filepath.Base(path)
child, madeIt, openErr := openOrCreateRuntimeChildNoFollow(parent, name)
if openErr != nil {
_ = unix.Close(parent)
return created, -1, fmt.Errorf("open sandbox runtime component %s: %w", path, openErr)
}
// CREATED, AND ABOUT TO BE FORGOTTEN. Both checks below run before the
// ledger entry is appended, so failing either used to return a directory
// this run had made with nothing above holding a record of it: setup
// reported failure and left it behind, and a recorded ancestor's
// compensation then failed on a child it could not explain.
//
// Undone relative to the parent descriptor rather than published to the
// ledger, because on the identity path the identity the ledger needs is the
// thing that just failed, and the descriptor is the only authority here
// that is not a name.
undoCreation := func(err error) error {
if !madeIt {
return err
}
if undoErr := unix.Unlinkat(parent, name, unix.AT_REMOVEDIR); undoErr != nil {
return fmt.Errorf("%w; and the directory this run created there could not be removed: %w", err, undoErr)
}
return err
}
if err := refuseForeignRuntimeDirectory(child, path); err != nil {
err = undoCreation(err)
_ = unix.Close(child)
_ = unix.Close(parent)
return created, -1, err
}
if madeIt {
identity, idErr := runtimeDirectoryIdentity(child)
if idErr == nil && runtimeCreationFailure != nil {
idErr = runtimeCreationFailure(path)
}
if idErr != nil {
err := undoCreation(fmt.Errorf("identify the sandbox runtime directory created at %s: %w", path, idErr))
_ = unix.Close(child)
_ = unix.Close(parent)
return created, -1, err
}
created = append(created, windowsCreatedRuntimeDir{path: path, identity: identity, identified: true})
}
_ = unix.Close(parent)
parent = child
}
return created, parent, nil
}
// openOrCreateRuntimeChildNoFollow opens name under parent, creating it if it is
// not there, and never follows a link at that name.
//
// madeIt distinguishes the directory this process created from one that was
// already there, because only the former belongs on the rollback ledger.
func openOrCreateRuntimeChildNoFollow(parent int, name string) (fd int, madeIt bool, err error) {
const flags = unix.O_RDONLY | unix.O_DIRECTORY | unix.O_NOFOLLOW | unix.O_CLOEXEC
child, openErr := unix.Openat(parent, name, flags, 0)
if openErr == nil {
return child, false, nil
}
if !errors.Is(openErr, unix.ENOENT) {
// ELOOP or ENOTDIR here IS the finding: a link sits where a directory of
// ours should be. Returned as-is rather than translated, because the two
// platforms spell it differently and the caller only needs it to fail.
return -1, false, openErr
}
if mkErr := unix.Mkdirat(parent, name, 0o700); mkErr != nil {
if !errors.Is(mkErr, unix.EEXIST) {
return -1, false, mkErr
}
// Somebody created it between the open and the create. That is an ordinary
// race with another Zero process, and the reopen below is still no-follow,
// so a link that arrived in the same window is refused rather than used.
child, openErr = unix.Openat(parent, name, flags, 0)
if openErr != nil {
return -1, false, openErr
}
return child, false, nil
}
child, openErr = unix.Openat(parent, name, flags, 0)
if openErr != nil {
return -1, false, openErr
}
return child, true, nil
}
// refuseForeignRuntimeDirectory proves the directory reached belongs to this
// user and is not group- or world-writable.
//
// Asked of the DESCRIPTOR, so it describes the object the descent is holding
// rather than whatever the name resolves to now. Ownership is the part that
// matters on a shared temp root: a component another account created first is
// theirs, and continuing into it would put the sandbox runtime inside a tree they
// control even though nothing about it is a link.
func refuseForeignRuntimeDirectory(fd int, path string) error {
var stat unix.Stat_t
if err := unix.Fstat(fd, &stat); err != nil {
return fmt.Errorf("inspect sandbox runtime component %s: %w", path, err)
}
if uid := unix.Getuid(); uid >= 0 && stat.Uid != uint32(uid) {
return fmt.Errorf("refusing to use the sandbox runtime component %s: it is owned by uid %d, not %d, so another account chose what the sandbox writes into",
path, stat.Uid, uid)
}
if stat.Mode&(unix.S_IWGRP|unix.S_IWOTH) != 0 {
return fmt.Errorf("refusing to use the sandbox runtime component %s: mode %#o lets another account replace what is inside it",
path, stat.Mode&0o7777)
}
return nil
}
// runtimeDirectoryIdentity names the object behind a descriptor, so rollback can
// tell the directory it created from one that replaced it under the same name.
func runtimeDirectoryIdentity(fd int) (string, error) {
var stat unix.Stat_t
if err := unix.Fstat(fd, &stat); err != nil {
return "", err
}
return strconv.FormatUint(uint64(stat.Dev), 10) + ":" + strconv.FormatUint(uint64(stat.Ino), 10), nil
}
-124
View File
@@ -1,124 +0,0 @@
//go:build windows
package sandbox
import (
"errors"
"fmt"
"path/filepath"
"golang.org/x/sys/windows"
)
// createRuntimeTailHandleRelative creates the owned tail of the runtime root
// beneath base, one component at a time, from retained handles.
//
// THE NAME IS NOT AUTHORIZATION. The previous shape found the deepest existing
// ancestor with os.Stat and then created each missing component by opening its
// parent BY NAME. Both follow a junction, so a local user could replace the
// predictable owned "zero" component with a junction after the pre-check,
// point it anywhere, let elevated setup create runtime\v1\<hash> beneath that
// target, and put the original back before the post-check. The post-check then
// saw an ordinary path, the plan applied normally, and rollback reported the
// separately created target only as identity-mismatched residue.
//
// So the base is opened by name exactly once, and everything below it is
// addressed relative to a handle with FILE_OPEN_REPARSE_POINT: an existing
// component is opened no-follow and refused if it is a link, and a missing one
// is created relative to its parent's handle and identified from the handle the
// create returned. No component name is resolved twice, so there is no interval
// for a swap to land in, and a junction placed at any owned component is seen
// as the link it is rather than followed.
//
// Redirected cache and TEMP locations ABOVE the owned tail are still allowed:
// they are part of base, which is the operator's business. The restriction is
// on the zero/runtime/v1/<hash> components Zero itself owns.
func createRuntimeTailHandleRelative(base string, tail []string) ([]windowsCreatedRuntimeDir, error) {
created, parent, err := createRuntimeTailRetainingHandle(base, tail)
if parent != 0 {
_ = windows.CloseHandle(parent)
}
return created, err
}
// createRuntimeTailRetainingHandle is the same descent, but it hands the caller
// the retained handle to the deepest component instead of closing it.
//
// Lease acquisition needs that handle: the lease file is a sibling of the
// runtime root, so creating it by pathname would re-resolve the owned components
// the descent just verified, which is the whole interval this design removes.
// The caller owns the returned handle and must close it, including when an error
// is returned alongside a non-zero handle.
func createRuntimeTailRetainingHandle(base string, tail []string) ([]windowsCreatedRuntimeDir, windows.Handle, error) {
if runtimeBaseOpenedByName != nil {
runtimeBaseOpenedByName(base)
}
parent, err := openWindowsDirectoryByName(base)
if err != nil {
return nil, 0, fmt.Errorf("open sandbox runtime base %s: %w", base, err)
}
if runtimeDescentBarrier != nil {
runtimeDescentBarrier()
}
var created []windowsCreatedRuntimeDir
for _, path := range tail {
name := filepath.Base(path)
// Exists already: another process won the race, or it was there before
// setup began. Opened no-follow, so a junction here is refused rather than
// descended, and it is not ours to record.
existing, openErr := openWindowsChildNoFollow(parent, name,
windows.FILE_READ_ATTRIBUTES|windows.FILE_TRAVERSE, windows.FILE_DIRECTORY_FILE)
if openErr == nil {
_ = windows.CloseHandle(parent)
parent = existing
continue
}
if !isWindowsNotFound(openErr) {
return created, parent, fmt.Errorf("inspect sandbox runtime component %s: %w", path, openErr)
}
handle, createErr := createWindowsChildDirectory(parent, name)
if createErr != nil {
if errors.Is(createErr, windows.STATUS_OBJECT_NAME_COLLISION) {
// Created between the open and the create by someone else. Same
// answer as the os.IsExist branch the caller already had: not ours,
// but not a failure either. Reopen no-follow so the descent
// continues through a verified object rather than a name.
existing, reopenErr := openWindowsChildNoFollow(parent, name,
windows.FILE_READ_ATTRIBUTES|windows.FILE_TRAVERSE, windows.FILE_DIRECTORY_FILE)
if reopenErr != nil {
return created, parent, fmt.Errorf("inspect sandbox runtime component %s after a concurrent create: %w", path, reopenErr)
}
_ = windows.CloseHandle(parent)
parent = existing
continue
}
return created, parent, fmt.Errorf("create sandbox runtime root %s: %w", path, createErr)
}
identity, idErr := handleRuntimeIdentity(handle)
if idErr == nil && runtimeCreationFailure != nil {
idErr = runtimeCreationFailure(path)
}
if idErr != nil {
// CREATED, AND ABOUT TO BE FORGOTTEN. The ledger entry is appended on
// the next line, so returning here left a directory this run had made
// with nothing above holding a record of it: setup reports failure and
// leaves it behind, and if an ancestor WAS recorded, that ancestor's
// compensation then fails on a directory it cannot explain.
//
// Undone through the handle rather than published to the ledger,
// because the identity the ledger needs is the thing that just failed.
err := fmt.Errorf("identify the sandbox runtime directory created at %s: %w", path, idErr)
if undoErr := removeWindowsObjectByHandle(handle); undoErr != nil {
err = fmt.Errorf("%w; and the directory this run created there could not be removed: %w", err, undoErr)
}
_ = windows.CloseHandle(handle)
return created, parent, err
}
created = append(created, windowsCreatedRuntimeDir{path: path, identity: identity, identified: true})
_ = windows.CloseHandle(parent)
parent = handle
}
return created, parent, nil
}
@@ -1,93 +0,0 @@
//go:build windows
package sandbox
import (
"os"
"os/exec"
"path/filepath"
"testing"
)
// THE SWAP THE OLD WALK COULD NOT SEE.
//
// createRuntimeDirRecording used to find the deepest existing ancestor with
// os.Stat and then create each missing component by opening its parent BY
// NAME. Both follow a junction. So between the pre-check and the create, a
// local user could replace the predictable owned "zero" component with a
// junction into a directory of their choosing, and elevated setup would create
// runtime\v1\<hash> beneath that target, apply its ACL plan there, and then
// find an ordinary path at the post-check once the original was put back.
//
// This plants the junction at the deterministic barrier between opening the
// base and touching the first owned component, which is that exact interval,
// and proves two things: nothing is created beneath the redirected target, and
// the descent refuses rather than continuing through the link. A junction
// needs no privilege on Windows, which is why it is the shape used here rather
// than a symlink that would skip on an unelevated box.
func TestRuntimeDescentRefusesAJunctionSwappedIntoAnOwnedComponent(t *testing.T) {
base := t.TempDir()
target := t.TempDir()
zero := filepath.Join(base, "zero")
tail := []string{zero, filepath.Join(zero, "runtime")}
previous := runtimeDescentBarrier
runtimeDescentBarrier = func() {
out, err := exec.Command("cmd", "/c", "mklink", "/J", zero, target).CombinedOutput()
if err != nil {
t.Fatalf("mklink /J: %v\n%s", err, out)
}
}
t.Cleanup(func() { runtimeDescentBarrier = previous })
created, err := createRuntimeTailHandleRelative(base, tail)
if err == nil {
t.Fatalf("the descent continued through a junction and reported success: created=%v", created)
}
if len(created) != 0 {
t.Errorf("a junction-swapped component still produced ownership records: %v", created)
}
// The whole point: the redirected target must be untouched.
entries, readErr := os.ReadDir(target)
if readErr != nil {
t.Fatal(readErr)
}
if len(entries) != 0 {
names := make([]string, 0, len(entries))
for _, entry := range entries {
names = append(names, entry.Name())
}
t.Errorf("setup created %v beneath the junction target, which is somebody else's directory", names)
}
}
// And the honest control: with no swap, the same descent creates the tail,
// records an identity for each component from the creation handle, and the
// result matches what the name resolves to afterwards.
func TestRuntimeDescentCreatesTheOwnedTailFromHandles(t *testing.T) {
base := t.TempDir()
zero := filepath.Join(base, "zero")
tail := []string{zero, filepath.Join(zero, "runtime"), filepath.Join(zero, "runtime", "v1")}
created, err := createRuntimeTailHandleRelative(base, tail)
if err != nil {
t.Fatalf("descent: %v", err)
}
if len(created) != len(tail) {
t.Fatalf("created %d components, want %d: %v", len(created), len(tail), created)
}
for i, record := range created {
if record.path != tail[i] {
t.Errorf("record %d path = %q, want %q", i, record.path, tail[i])
}
if !record.identified {
t.Errorf("record %d for %s carries no identity", i, record.path)
continue
}
if now, ok := runtimeDirIdentity(record.path); !ok || now != record.identity {
t.Errorf("record %d identity %q does not describe the directory at %s (%q)", i, record.identity, record.path, now)
}
}
}
@@ -1,27 +0,0 @@
//go:build !windows
package sandbox
import (
"fmt"
"os"
"syscall"
)
// runtimeDirIdentity identifies the directory currently at path.
//
// device plus inode, read with Lstat so a link substituted at the final
// component is identified as the link rather than followed. The Windows build
// cannot use os.SameFile for this and explains why; here the same eager capture
// keeps both platforms on one rule.
func runtimeDirIdentity(path string) (string, bool) {
info, err := os.Lstat(path)
if err != nil {
return "", false
}
stat, ok := info.Sys().(*syscall.Stat_t)
if !ok {
return "", false
}
return fmt.Sprintf("%d:%d", stat.Dev, stat.Ino), true
}
@@ -1,45 +0,0 @@
//go:build windows
package sandbox
import (
"fmt"
"golang.org/x/sys/windows"
)
// runtimeDirIdentity identifies the directory currently at path, EAGERLY.
//
// os.SameFile cannot be used for this on Windows. A Windows fileStat loads its
// volume serial and file index lazily, by PATHNAME, at comparison time, so an
// identity captured before a replacement and compared afterwards reports the
// substitute as the same object and the original as a different one: exactly
// backwards, and silently. Measured, not reasoned about.
//
// Reading the identity through a handle at capture time removes the lazy step.
// Opened with FILE_FLAG_OPEN_REPARSE_POINT so a link substituted at the final
// component is identified as the link it is rather than followed.
func runtimeDirIdentity(path string) (string, bool) {
utf16Path, err := windows.UTF16PtrFromString(path)
if err != nil {
return "", false
}
handle, err := windows.CreateFile(
utf16Path,
windows.FILE_READ_ATTRIBUTES,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
nil,
windows.OPEN_EXISTING,
windows.FILE_FLAG_BACKUP_SEMANTICS|windows.FILE_FLAG_OPEN_REPARSE_POINT,
0,
)
if err != nil {
return "", false
}
defer windows.CloseHandle(handle)
var info windows.ByHandleFileInformation
if err := windows.GetFileInformationByHandle(handle, &info); err != nil {
return "", false
}
return fmt.Sprintf("%d:%d:%d", info.VolumeSerialNumber, info.FileIndexHigh, info.FileIndexLow), true
}
@@ -1,44 +0,0 @@
//go:build windows
package sandbox
import "testing"
// THE COMMAND HAS TO RECOGNISE THE ROOT SETUP RECORDED, WHATEVER ITS
// ENVIRONMENT SAYS.
//
// Setup derives the fallback root in an elevated terminal and records it; a
// command re-derives it in an ordinary one to decide whether the record belongs
// to this workspace. The user scope in that derivation used to come from
// USERNAME, so a value that differed between the two terminals made the command
// reject the record and select a root with no capability ACE on it.
func TestRecordedFallbackRootSurvivesADifferentUsernameVariable(t *testing.T) {
// Canonical, as selection hands it over. The record check canonicalizes on
// its own, so a raw t.TempDir() spelling digests differently wherever TEMP is
// an 8.3 short path, as it is on the CI runners.
workspace := canonicalSandboxWorkspaceRoot(t.TempDir())
tempHome := t.TempDir()
t.Setenv("TMP", tempHome)
t.Setenv("TEMP", tempHome)
t.Setenv("USERNAME", "setup-terminal")
recorded, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatalf("derive the fallback root as setup: %v", err)
}
if !ownedFallbackRuntimeRecord(workspace, recorded) {
t.Fatalf("SETUP INVALID: the fallback root is not recognised even in the environment that derived it: %s", recorded)
}
t.Setenv("USERNAME", "command-terminal")
if !ownedFallbackRuntimeRecord(workspace, recorded) {
t.Fatalf("once USERNAME changed, the command no longer recognises the fallback root setup recorded: %s", recorded)
}
derived, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatalf("derive the fallback root as the command: %v", err)
}
if !sameWindowsRuntimeRootPath(derived, recorded) {
t.Fatalf("setup and the command derived different fallback roots for one workspace:\n setup: %s\n command: %s", recorded, derived)
}
}
@@ -1,106 +0,0 @@
package sandbox
import (
"path/filepath"
"runtime"
"strings"
"testing"
)
// EVERY OWNERSHIP-CHECKED ANCESTOR MUST ALREADY BE INSIDE ONE USER'S NAMESPACE.
//
// The temp-derived fallback lives under a directory that is shared on Unix
// whenever TMPDIR is unset. Runtime preparation creates and ownership-checks
// each component of the tail at 0700, so a fixed first component meant the
// first account to use the fallback created a private directory every other
// account was then refused at: traversal fails on the mode, and relaxing the
// mode fails the ownership guard instead. The per-workspace digest is the leaf,
// so it never got the chance to separate them.
//
// The invariant is therefore about the SHALLOWEST checked component, not the
// leaf: it has to carry the user scope, or the guards below it are guarding a
// namespace two users share.
func TestFallbackRuntimeRootScopesItsShallowestOwnedComponent(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("os.TempDir() resolves inside the user profile on Windows, so the shared-temp collision cannot arise")
}
workspace := t.TempDir()
tempHome := t.TempDir()
t.Setenv("TMPDIR", tempHome)
root, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatalf("fallbackSandboxRuntimeRoot: %v", err)
}
components := ownedRuntimeComponents(root)
if len(components) == 0 {
t.Fatal("no ownership-checked components derived for the fallback root")
}
// ownedRuntimeComponents walks upward, so the last entry is the shallowest
// directory the guards create and validate.
shallowest := filepath.Base(components[len(components)-1])
scope := sandboxRuntimeUserScope()
if !strings.Contains(shallowest, scope) {
t.Errorf("the shallowest ownership-checked component is %q and does not carry the user scope %q; "+
"two accounts on a shared temp would contend for it", shallowest, scope)
}
}
// The workspace still decides the leaf, so setup and the command derive the same
// root for the same workspace. Scoping the top must not have moved that.
func TestFallbackRuntimeRootStaysStableForOneWorkspace(t *testing.T) {
workspace := t.TempDir()
tempHome := t.TempDir()
t.Setenv("TMPDIR", tempHome)
t.Setenv("TMP", tempHome)
t.Setenv("TEMP", tempHome)
first, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatal(err)
}
second, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatal(err)
}
if first != second {
t.Errorf("the fallback root is not stable for one workspace:\n %s\n %s", first, second)
}
other, err := fallbackSandboxRuntimeRoot(t.TempDir())
if err != nil {
t.Fatal(err)
}
if other == first {
t.Error("two different workspaces derived the same fallback root")
}
if filepath.Dir(other) != filepath.Dir(first) {
t.Errorf("two workspaces for one user should differ only in the leaf:\n %s\n %s", first, other)
}
}
// And the Windows side is a deliberate choice, not an omission. Its temp root
// is already inside the user profile, and the names have to stay fixed so
// windowsSandboxRuntimeOwnedTail can still recognise a root that an elevated
// setup running as a different account built.
func TestFallbackOwnedNamesAreUnscopedOnWindowsOnly(t *testing.T) {
names := sandboxRuntimeFallbackOwnedNames()
if len(names) != len(windowsSandboxRuntimeOwnedNames) {
t.Fatalf("fallback names = %v, want the same depth as %v", names, windowsSandboxRuntimeOwnedNames)
}
// The components below the first are shared by both platforms, so a change
// to them would break the tail matcher on Windows.
for index := 1; index < len(names); index++ {
if names[index] != windowsSandboxRuntimeOwnedNames[index] {
t.Errorf("component %d = %q, want %q", index, names[index], windowsSandboxRuntimeOwnedNames[index])
}
}
scoped := names[0] != windowsSandboxRuntimeOwnedNames[0]
if runtime.GOOS == "windows" && scoped {
t.Errorf("the Windows fallback scoped its first component to %q; the tail matcher compares fixed names", names[0])
}
if runtime.GOOS != "windows" && !scoped {
t.Errorf("the first component is %q on %s, where the temp root can be shared between accounts",
names[0], runtime.GOOS)
}
}
@@ -1,74 +0,0 @@
package sandbox
import (
"encoding/json"
"os"
"path/filepath"
"testing"
)
// writeRecordedRoot puts a setup marker naming root under sandboxHome.
func writeRecordedRoot(t *testing.T, sandboxHome, root string) {
t.Helper()
path := WindowsSandboxSetupMarkerPath(sandboxHome)
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
t.Fatal(err)
}
blob, err := json.Marshal(WindowsSandboxSetupMarker{SchemaVersion: windowsSandboxSetupMarkerSchemaVersion, RuntimeRoot: root})
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, blob, 0o600); err != nil {
t.Fatal(err)
}
}
// ONE COMMAND, ONE SANDBOX HOME.
//
// Runtime preparation happens before Windows platform planning, and it used to
// resolve the sandbox home from the AMBIENT environment while the planner
// resolves ZERO_WINDOWS_SANDBOX_HOME out of the command's own spec.Env and hands
// that one to the runner for marker validation. So a command that explicitly
// selects home B, while the parent process still points at home A, pinned A's
// recorded root into the profile; the runner then loaded B's marker, saw a
// different root, and rejected the command as out of date even though setup for
// B was valid. The two homes need no different derivation rules to disagree,
// only different valid selections from the same preferred/fallback pair.
func TestThePinnedRootComesFromTheCommandsOwnSandboxHome(t *testing.T) {
homeA := t.TempDir()
homeB := t.TempDir()
preferred := filepath.Join(t.TempDir(), "preferred")
fallback := filepath.Join(t.TempDir(), "fallback")
writeRecordedRoot(t, homeA, preferred)
writeRecordedRoot(t, homeB, fallback)
if got := pinnedSandboxRuntimeRoot(t.TempDir(), preferred, fallback, homeB); got != fallback {
t.Errorf("pinned %q for a command that selected home B, want %q: the ambient home decided instead of the command's", got, fallback)
}
if got := pinnedSandboxRuntimeRoot(t.TempDir(), preferred, fallback, homeA); got != preferred {
t.Errorf("pinned %q for home A, want %q", got, preferred)
}
}
// A recorded root that this workspace could not select is still refused, which
// is the protection that stops one workspace's tree being pinned into another.
func TestARecordedRootFromAnotherWorkspaceIsStillRefused(t *testing.T) {
home := t.TempDir()
writeRecordedRoot(t, home, filepath.Join(t.TempDir(), "someone-elses-tree"))
if got := pinnedSandboxRuntimeRoot(t.TempDir(), filepath.Join(t.TempDir(), "preferred"), filepath.Join(t.TempDir(), "fallback"), home); got != "" {
t.Errorf("pinned %q, want none: it matches neither candidate this workspace derives", got)
}
}
// No command context means the ambient environment is the only authority there
// is, so an empty home must still resolve rather than refusing outright.
func TestAnEmptyCommandHomeFallsBackToTheAmbientEnvironment(t *testing.T) {
home := t.TempDir()
t.Setenv("ZERO_WINDOWS_SANDBOX_HOME", home)
preferred := filepath.Join(t.TempDir(), "preferred")
writeRecordedRoot(t, home, preferred)
if got := pinnedSandboxRuntimeRoot(t.TempDir(), preferred, filepath.Join(t.TempDir(), "fallback"), ""); got != preferred {
t.Errorf("pinned %q with no command home, want the ambient one to decide (%q)", got, preferred)
}
}
+7 -64
View File
@@ -1,88 +1,31 @@
package sandbox
import (
"errors"
"fmt"
"sync"
)
const sandboxRuntimeLeaseSuffix = ".lease"
// errRuntimeLeaseReplaced says a lock was taken on a lease object that the lease
// name no longer resolves to: cleanup unlinked or replaced it while this call was
// waiting. Both acquisitions retry from their open on it, and surface it only
// once the retries are spent.
var errRuntimeLeaseReplaced = errors.New("the sandbox runtime lease was removed or replaced while it was being taken")
// runtimeLeaseAcquireAttempts bounds those retries. Each stale lock means one
// cleanup ran to completion in the window, and cleanups do not chain without
// limit; a third stale object in a row is reported rather than chased.
const runtimeLeaseAcquireAttempts = 3
// sandboxRuntimeLease is one holder's grip on a runtime root.
//
// createdFile says this acquisition is the one that brought the lease file into
// existence. Compensation needs it: rollback refuses a non-empty directory, and
// the lease sits beside the leaf inside a directory a failed setup has to be able
// to remove, so the file has to go with it. It may only go if it was ours, which
// is a fact only the create knows.
type sandboxRuntimeLease struct {
handle runtimeLeaseHandle
once sync.Once
// root is the runtime root this lease is for, kept so compensation can name
// the lease file without deriving it again.
root string
// createdFile says THIS acquisition brought the lease file into existence.
createdFile bool
}
// createdLeaseFile reports whether compensation may remove the lease file.
//
// Only the acquisition that created it may, and only while nothing else holds
// it. A lease that was already there belongs to whoever made it, and removing it
// would take the coordination object out from under a live command.
func (lease *sandboxRuntimeLease) createdLeaseFile() (string, bool) {
if lease == nil || !lease.createdFile {
return "", false
}
return sandboxRuntimeLeasePath(lease.root), true
}
func sandboxRuntimeLeasePath(root string) string {
return root + sandboxRuntimeLeaseSuffix
}
// The pathname-based shared acquisition that used to live here is gone with its
// two platform halves. It opened the lease by full pathname, so it followed a
// link planted at that name and, on Windows, opened without FILE_SHARE_DELETE,
// which alone stops cleanup from ever removing the lease it is holding. Every
// caller now goes through acquireRuntimeLeaseForPlatform, which descends from a
// retained parent handle no-follow. Leaving the weaker door defined next to the
// stronger one is how a later change quietly takes it.
// appendCreatedRuntimeDirs merges one acquisition attempt's ledger into the
// running one. A path seen again was re-created after cleanup removed the
// earlier object, so the later identity supersedes; rollback verifies identity
// before it removes anything, and a stale one would make it refuse the directory
// this run actually made.
func appendCreatedRuntimeDirs(existing, made []windowsCreatedRuntimeDir) []windowsCreatedRuntimeDir {
for _, record := range made {
replaced := false
for index := range existing {
if existing[index].path == record.path {
existing[index] = record
replaced = true
break
}
}
if !replaced {
existing = append(existing, record)
}
func acquireSandboxRuntimeLease(root string) (*sandboxRuntimeLease, error) {
handle, err := acquireSharedRuntimeLease(sandboxRuntimeLeasePath(root))
if err != nil {
return nil, fmt.Errorf("acquire sandbox runtime lease: %w", err)
}
return existing
return &sandboxRuntimeLease{handle: handle}, nil
}
func tryAcquireSandboxRuntimeCleanupLease(root string) (*sandboxRuntimeLease, bool, error) {
handle, inUse, err := tryAcquireExclusiveRuntimeLease(root)
handle, inUse, err := tryAcquireExclusiveRuntimeLease(sandboxRuntimeLeasePath(root))
if err != nil || inUse {
return nil, inUse, err
}
-15
View File
@@ -1,15 +0,0 @@
package sandbox
// runtimeLeasePreCreateBarrier, when set, runs after lease acquisition has
// decided what it is going to do and before it creates anything.
//
// It exists so a test can reproduce the CHECK-THEN-USE RACE rather than a
// junction that was already there when acquisition started. The two are not the
// same finding: a junction present from the outset was already refused by the
// alias pre-check, while one planted in this window defeated it, because
// os.MkdirAll and the lease open both resolved the component again and followed
// it. A test that only plants the junction up front passes against the defective
// implementation and proves nothing.
//
// Nil in production.
var runtimeLeasePreCreateBarrier func()
@@ -1,150 +0,0 @@
//go:build windows
package sandbox
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// leaseRootUnder builds a production-shaped runtime root beneath a test-owned
// cache directory, so the owned tail is real rather than assumed.
func leaseRootUnder(t *testing.T, cacheRoot string) string {
t.Helper()
workspace := t.TempDir()
root, ok := deterministicSandboxRuntimeRoot(canonicalSandboxWorkspaceRoot(workspace), canonicalSandboxWorkspaceRoot(cacheRoot))
if !ok {
t.Fatalf("SETUP INVALID: no deterministic runtime root under %s", cacheRoot)
}
return root
}
// THE LEASE IS THE FIRST WRITE SETUP MAKES, SO IT IS THE ONE THAT MATTERS MOST.
//
// prepareSandboxRuntimeLease checked the owned components for aliases and then
// created the parent with os.MkdirAll and opened "<root>.lease" by pathname.
// Both follow. An ordinary same-account process could replace "zero", "runtime"
// or "v1" with a junction after the check and before either call, and elevated
// setup would then build the tree and the lease file inside somebody else's
// target. Putting the component back afterwards left the later handle-relative
// provisioning working on the legitimate tree, so no post-check ever saw it, and
// the rollback could not compensate what it had no record of.
//
// A junction needs no privilege on Windows, so the caller's half of this runs on
// an ordinary unelevated box.
func TestRuntimeLeaseRefusesAJunctionPlantedAfterTheCheck(t *testing.T) {
cacheRoot := t.TempDir()
target := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
owned := filepath.Join(canonicalSandboxWorkspaceRoot(cacheRoot), "zero")
if !strings.HasPrefix(strings.ToLower(root), strings.ToLower(owned)) {
t.Fatalf("SETUP INVALID: %s does not sit under the owned component %s", root, owned)
}
// THE JUNCTION ARRIVES IN THE WINDOW, WHICH IS THE WHOLE POINT.
//
// One that is already there when acquisition starts was refused by the old
// alias pre-check too, so a test that plants it up front passes against the
// defective implementation and proves nothing. This one lands after the
// decision and before the first create, which is exactly where os.MkdirAll and
// the pathname lease open used to follow it.
previous := runtimeLeasePreCreateBarrier
t.Cleanup(func() { runtimeLeasePreCreateBarrier = previous })
attempted, planted := false, false
runtimeLeasePreCreateBarrier = func() {
if attempted {
return
}
attempted = true
if out, err := exec.Command("cmd", "/c", "mklink", "/J", owned, target).CombinedOutput(); err != nil {
t.Logf("mklink /J unavailable: %v: %s", err, out)
return
}
planted = true
}
lease, _, err := prepareSandboxRuntimeLeaseRecording(root)
if lease != nil {
lease.release()
}
if !attempted {
t.Skip("the pre-create barrier never ran, so the race was not reproduced")
}
// Bound to mklink's result, not to the attempt. When it fails, acquisition
// goes on to create an ordinary directory at the same name, the check below
// finds it, and a correct acquisition would be failed as a junction bypass.
if !planted {
t.Skip("mklink /J could not create the junction here, so the race was not reproduced")
}
if _, statErr := os.Lstat(owned); statErr != nil {
t.Skipf("the junction could not be created here: %v", statErr)
}
entries, readErr := os.ReadDir(target)
if readErr != nil {
t.Fatal(readErr)
}
if len(entries) != 0 {
names := make([]string, 0, len(entries))
for _, entry := range entries {
names = append(names, entry.Name())
}
t.Fatalf("lease acquisition wrote %v beneath a junction planted after its own check; err=%v", names, err)
}
if err == nil {
t.Fatal("lease acquisition reported success through a junction planted after its check")
}
}
// And an ordinary cache tree still gets its lease, or the refusal above would be
// satisfied by an acquirer that refuses everything.
func TestRuntimeLeaseStillAcquiresOnAnOrdinaryTree(t *testing.T) {
cacheRoot := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
lease, created, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Fatalf("an ordinary cache tree was refused: %v", err)
}
defer lease.release()
if _, statErr := os.Stat(sandboxRuntimeLeasePath(root)); statErr != nil {
t.Fatalf("the lease file was not created: %v", statErr)
}
// The components above the leaf are recorded, so a later failure can undo
// them. The leaf itself belongs to provisioning and must NOT appear here.
for _, entry := range created {
if strings.EqualFold(filepath.Clean(entry.path), filepath.Clean(root)) {
t.Fatalf("lease acquisition claimed the runtime leaf %s, which provisioning owns", root)
}
}
if len(created) == 0 {
t.Fatal("SETUP INVALID: nothing was recorded as created, so the accounting assertion above is vacuous")
}
}
// A second acquisition over an existing tree records nothing: it created nothing,
// so it owns nothing, and rollback must not remove a tree it found.
func TestRuntimeLeaseRecordsOnlyWhatItCreated(t *testing.T) {
cacheRoot := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
first, _, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Fatalf("first acquisition: %v", err)
}
first.release()
second, created, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Fatalf("second acquisition: %v", err)
}
defer second.release()
if len(created) != 0 {
t.Fatalf("the second acquisition claimed %d directories it did not create; rollback would delete a tree it found", len(created))
}
}
@@ -1,216 +0,0 @@
//go:build !windows
package sandbox
import (
"errors"
"os"
"path/filepath"
"testing"
"golang.org/x/sys/unix"
)
// unixLeaseRootUnder builds a production-shaped runtime root beneath a
// test-owned base, so the owned tail is real rather than assumed.
func unixLeaseRootUnder(t *testing.T, base string) string {
t.Helper()
workspace := t.TempDir()
root, ok := deterministicSandboxRuntimeRoot(canonicalSandboxWorkspaceRoot(workspace), canonicalSandboxWorkspaceRoot(base))
if !ok {
t.Fatalf("SETUP INVALID: no deterministic runtime root under %s", base)
}
return root
}
// countEntriesUnder reports how many entries exist anywhere beneath dir, so a
// test can say "nothing was written here" rather than checking one name.
func countEntriesUnder(t *testing.T, dir string) []string {
t.Helper()
var found []string
err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if path == dir {
return nil
}
found = append(found, path)
return nil
})
if err != nil {
t.Fatalf("walk %s: %v", dir, err)
}
return found
}
// A PRE-CHECK CANNOT AUTHORIZE A PATHNAME WRITE THAT COMES AFTER IT.
//
// The fallback root moved from an atomically minted MkdirTemp parent to a
// predictable /tmp/zero-u<uid>/runtime/v1/<digest>. The stable name is required,
// because setup and every later process have to agree on one root without talking
// to each other, but it also means another local account can name the first owned
// component first. refuseAliasedRuntimeComponents answers about an ABSENT
// component by saying there is nothing to alias, which is the state a fresh
// fallback is in, and os.MkdirAll plus the pathname lease open then followed a
// link planted after that answer.
//
// The link arrives after the last validation and before the first create, which
// is the only window that matters: one planted earlier was refused by the old
// pre-check too, so a test that plants it up front passes against the defect and
// proves nothing.
func TestRuntimeLeaseRefusesALinkPlantedAfterTheCheck(t *testing.T) {
base := t.TempDir()
target := t.TempDir()
root := unixLeaseRootUnder(t, base)
owned := filepath.Join(canonicalSandboxWorkspaceRoot(base), "zero")
if !pathWithinRoot(owned, root) {
t.Fatalf("SETUP INVALID: %s does not sit under the owned component %s", root, owned)
}
previous := runtimeDescentBarrier
t.Cleanup(func() { runtimeDescentBarrier = previous })
planted := false
runtimeDescentBarrier = func() {
if planted {
return
}
planted = true
if err := os.Symlink(target, owned); err != nil {
t.Errorf("plant the link: %v", err)
}
}
lease, _, err := prepareSandboxRuntimeLeaseRecording(root)
if lease != nil {
lease.release()
}
// SETUP: the swap really happened in the window, or nothing was under test.
if !planted {
t.Fatal("SETUP INVALID: the descent never reached the barrier, so no link was planted")
}
if err == nil {
t.Fatal("lease acquisition succeeded through a link planted at the first owned component")
}
// ELOOP on Linux, and either that or ENOTDIR elsewhere. Asserted as a class
// rather than one kernel's spelling.
if !errors.Is(err, unix.ELOOP) && !errors.Is(err, unix.ENOTDIR) {
t.Errorf("refused for the wrong reason, want a no-follow refusal: %v", err)
}
// THE POINT IS WHERE THE WRITES DID NOT GO. An error alone would also be
// produced by a later guard noticing the link after the tree was built inside
// somebody else's directory.
if leaked := countEntriesUnder(t, target); len(leaked) != 0 {
t.Fatalf("the redirected target received %v; the refusal happened after the writes rather than instead of them", leaked)
}
}
// CONTROL: an ordinary tree still gets its lease, and the one path opened by name
// is the operator's base rather than a component Zero owns.
//
// Refusing everything would satisfy the test above. Asserting the base directly
// is what makes this discriminating: inferring it from whether a swap was caught
// says nothing about which name was trusted.
func TestRuntimeLeaseOpensOnlyTheBaseByName(t *testing.T) {
base := t.TempDir()
root := unixLeaseRootUnder(t, base)
previousBase := runtimeBaseOpenedByName
t.Cleanup(func() { runtimeBaseOpenedByName = previousBase })
var byName []string
runtimeBaseOpenedByName = func(path string) { byName = append(byName, path) }
lease, created, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Fatalf("lease acquisition failed on an ordinary tree: %v", err)
}
lease.release()
if len(byName) != 1 {
t.Fatalf("opened %v by name, want exactly the base", byName)
}
if got := byName[0]; got != canonicalSandboxWorkspaceRoot(base) && got != base {
t.Fatalf("opened %q by name, want the base %q", got, base)
}
if len(created) == 0 {
t.Fatal("the descent created the owned components but recorded none of them, so rollback has nothing to undo")
}
if _, err := os.Lstat(sandboxRuntimeLeasePath(root)); err != nil {
t.Fatalf("no lease file was created on an ordinary tree: %v", err)
}
}
// And a link at the LEASE NAME itself is refused by both holders, so the shared
// and exclusive sides cannot end up locking different objects.
func TestRuntimeLeaseRefusesALinkAtTheLeaseName(t *testing.T) {
base := t.TempDir()
root := unixLeaseRootUnder(t, base)
lease, _, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Fatalf("SETUP: cannot seed a lease: %v", err)
}
lease.release()
leasePath := sandboxRuntimeLeasePath(root)
if err := os.Remove(leasePath); err != nil {
t.Fatalf("SETUP INVALID: cannot clear the seeded lease: %v", err)
}
target := filepath.Join(t.TempDir(), "target.lease")
if err := os.WriteFile(target, nil, 0o600); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, leasePath); err != nil {
t.Fatalf("SETUP: cannot plant the link: %v", err)
}
shared, _, sharedErr := prepareSandboxRuntimeLeaseRecording(root)
if shared != nil {
shared.release()
}
if sharedErr == nil {
t.Error("shared acquisition accepted a link at the lease name")
}
cleanup, inUse, cleanupErr := tryAcquireSandboxRuntimeCleanupLease(root)
if cleanup != nil {
cleanup.release()
}
if cleanupErr == nil {
t.Errorf("cleanup accepted a link at the lease name (inUse=%t); it would treat a lock on %s as proof the root is free to delete", inUse, target)
}
}
// CONTROL: the two sides still coordinate on an ordinary lease.
func TestUnixSharedLeaseMakesCleanupReportInUse(t *testing.T) {
base := t.TempDir()
root := unixLeaseRootUnder(t, base)
held, _, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Fatalf("cannot acquire a runtime lease: %v", err)
}
lease, inUse, err := tryAcquireSandboxRuntimeCleanupLease(root)
if lease != nil {
lease.release()
}
if err != nil {
t.Fatalf("cleanup failed against a legitimately held lease: %v", err)
}
if !inUse {
t.Fatal("cleanup reported the runtime root free while a shared lease was held; it would delete a tree a live command is using")
}
held.release()
lease, inUse, err = tryAcquireSandboxRuntimeCleanupLease(root)
if err != nil {
t.Fatalf("cleanup failed after the shared lease was released: %v", err)
}
if inUse {
t.Fatal("cleanup still reported the root in use after the only holder released it, so reclamation never happens")
}
lease.release()
}
@@ -1,93 +0,0 @@
package sandbox
import (
"os"
"path/filepath"
"testing"
)
// runtimeRootUnderTest builds a root with the shape production uses, so the
// components refuseAliasedRuntimeComponents walks stay inside test-owned
// storage.
//
// Owned depth is the fixed names plus the digest, so a root placed directly in
// t.TempDir() puts an owned component on /tmp, which the Unix ownership guard
// correctly refuses because /tmp belongs to root. That is the guard working, not
// a test environment problem, and it only shows up off Windows.
func runtimeRootUnderTest(t *testing.T, leaf string) string {
t.Helper()
root := filepath.Join(t.TempDir(), "zero", "runtime", "v1", leaf)
if err := os.MkdirAll(root, 0o700); err != nil {
t.Fatal(err)
}
return root
}
// A LEASE IS OWNERSHIP FOR A TRANSACTION, NOT A SELECTION PROBE.
//
// Setup took a lease only to learn which root won and released it at once, so
// nothing owned the selected root while the elevated helper provisioned the
// tree, applied the ACL and stamp, installed network state and wrote the marker.
// A command for another workspace scans the same runtime parent and excludes
// only its own current root, so it can take this root's cleanup lease and
// RemoveAll it mid-transaction; setup then publishes success for a pathname that
// is gone.
//
// This pins the mechanism the fix relies on: a held lease is what makes the
// cleanup's exclusive acquire fail.
func TestAHeldRuntimeLeaseStopsConcurrentCleanup(t *testing.T) {
root := runtimeRootUnderTest(t, "deadbeefdeadbeef")
lease, err := prepareSandboxRuntimeLease(root)
if err != nil {
t.Fatalf("acquire the runtime lease: %v", err)
}
removeSandboxRuntimeRootIfUnused(root)
if _, err := os.Stat(root); err != nil {
t.Fatalf("cleanup removed a root that setup was holding: %v", err)
}
lease.release()
// And once nothing holds it, cleanup does its job: without this half the test
// would pass against a cleanup that never removes anything.
removeSandboxRuntimeRootIfUnused(root)
if _, err := os.Stat(root); !os.IsNotExist(err) {
t.Fatalf("an unheld root survived cleanup: %v", err)
}
}
// The lease is on the root the caller names, so holding one root does not
// protect a sibling: setup reserving its own selection must not stop the
// cleanup doing its work elsewhere.
func TestAHeldLeaseDoesNotProtectASiblingRoot(t *testing.T) {
held := runtimeRootUnderTest(t, "aaaaaaaaaaaaaaaa")
sibling := filepath.Join(filepath.Dir(held), "bbbbbbbbbbbbbbbb")
if err := os.MkdirAll(sibling, 0o700); err != nil {
t.Fatal(err)
}
lease, err := prepareSandboxRuntimeLease(held)
if err != nil {
t.Fatal(err)
}
defer lease.release()
removeSandboxRuntimeRootIfUnused(sibling)
if _, err := os.Stat(sibling); !os.IsNotExist(err) {
t.Errorf("holding one root blocked cleanup of an unrelated sibling: %v", err)
}
if _, err := os.Stat(held); err != nil {
t.Errorf("the held root was removed: %v", err)
}
}
// createdRuntimeDirsForTest builds identity-bound rollback records for paths a
// test made itself, so a test can express "these are the directories this run
// created" without restating the identity capture.
func createdRuntimeDirsForTest(paths ...string) []windowsCreatedRuntimeDir {
records := make([]windowsCreatedRuntimeDir, 0, len(paths))
for _, path := range paths {
identity, identified := runtimeDirIdentity(path)
records = append(records, windowsCreatedRuntimeDir{path: path, identity: identity, identified: identified})
}
return records
}
@@ -1,21 +0,0 @@
//go:build !windows
package sandbox
// acquireRuntimeLeaseForPlatform creates and locks the lease through the same
// rooted no-follow boundary the Windows side uses.
//
// A PRE-CHECK IS NOT A BOUNDARY. This used to call refuseAliasedRuntimeComponents
// and then os.MkdirAll plus a pathname lease open. The guard answers about an
// ABSENT component by saying there is nothing to alias, which is exactly the
// state a fresh fallback root is in, and both calls that followed resolve the
// name again. A link planted in between was therefore created through and
// written into, and the next check could only report it after the writes.
//
// The alias guard is gone from this path rather than kept alongside: retaining it
// would suggest the two together are the protection, when the descent is the
// protection and the guard is the thing that could not be one. Other callers
// still use it where a pathname really is all there is.
func acquireRuntimeLeaseForPlatform(root string) (*sandboxRuntimeLease, []windowsCreatedRuntimeDir, error) {
return acquireRuntimeLeaseRootedUnix(root)
}
@@ -1,10 +0,0 @@
//go:build windows
package sandbox
// acquireRuntimeLeaseForPlatform routes Windows through the rooted, no-follow
// acquisition. See acquireRuntimeLeaseRooted for why the pathname walk it
// replaces was unsafe.
func acquireRuntimeLeaseForPlatform(root string) (*sandboxRuntimeLease, []windowsCreatedRuntimeDir, error) {
return acquireRuntimeLeaseRooted(root)
}
@@ -1,299 +0,0 @@
//go:build windows
package sandbox
import (
"os"
"path/filepath"
"strings"
"testing"
"unsafe"
"golang.org/x/sys/windows"
)
// seedRuntimeLeaseTree creates the owned tail and the lease the way a first run
// does, then removes just the lease file so a test can put something else at that
// name. The tree has to exist, or cleanup's descent has nothing to open and the
// test would pass for the wrong reason.
func seedRuntimeLeaseTree(t *testing.T, root string) string {
t.Helper()
lease, _, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Skipf("cannot seed a runtime lease here: %v", err)
}
lease.release()
leasePath := sandboxRuntimeLeasePath(root)
if err := os.Remove(leasePath); err != nil {
t.Fatalf("SETUP INVALID: cannot clear the seeded lease: %v", err)
}
return leasePath
}
// plantFileLink puts a FILE symbolic link at leasePath pointing at an ordinary
// file, and returns the target.
//
// A file link, not a junction: a junction is a directory and FILE_NON_DIRECTORY_FILE
// already refuses it, so a junction here would pass against the defect and prove
// nothing. Creating one needs SeCreateSymbolicLinkPrivilege, which an ordinary
// unelevated account does not hold, so this skips there and runs on CI.
func plantFileLink(t *testing.T, leasePath string) string {
t.Helper()
target := filepath.Join(t.TempDir(), "target.lease")
if err := os.WriteFile(target, nil, 0o600); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, leasePath); err != nil {
t.Skipf("cannot create a file symbolic link here, which needs SeCreateSymbolicLinkPrivilege: %v", err)
}
// SETUP: it really is a reparse point, or nothing below is about links.
info, err := os.Lstat(leasePath)
if err != nil || info.Mode()&os.ModeSymlink == 0 {
t.Fatalf("SETUP INVALID: %s is not a link (%v, %v)", leasePath, info, err)
}
return target
}
// assertTargetUnlocked proves nobody holds a lock on the link's target, which is
// the object the defect made the two sides fight over.
func assertTargetUnlocked(t *testing.T, target string) {
t.Helper()
file, err := os.OpenFile(target, os.O_RDWR, 0o600)
if err != nil {
t.Fatalf("open the link target: %v", err)
}
defer func() { _ = file.Close() }()
var overlapped windows.Overlapped
flags := uint32(windows.LOCKFILE_EXCLUSIVE_LOCK | windows.LOCKFILE_FAIL_IMMEDIATELY)
if err := windows.LockFileEx(windows.Handle(file.Fd()), flags, 0, 1, 0, &overlapped); err != nil {
t.Fatalf("the link target is locked, so the refusal happened after the lease had already been taken on it: %v", err)
}
_ = windows.UnlockFileEx(windows.Handle(file.Fd()), 0, 1, 0, &overlapped)
}
// FILE_OPEN_REPARSE_POINT SAYS DO NOT FOLLOW, NOT REFUSE.
//
// The rooted opener passed that flag and FILE_NON_DIRECTORY_FILE and treated the
// pair as a guarantee. It is not: the flag returns a handle to the LINK, and
// FILE_NON_DIRECTORY_FILE excludes directories rather than non-directory reparse
// objects. A file symbolic link at <digest>.lease was opened, wrapped and locked
// as though it were the lease.
//
// That matters because cleanup opened the same name by pathname with no
// no-follow flag at all, so it locked the link's TARGET. Two holders, two
// objects, both calls succeeding, and cleanup free to RemoveAll a runtime root a
// live command was using.
func TestSharedLeaseRefusesAFileLinkAtTheLeaseName(t *testing.T) {
cacheRoot := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
leasePath := seedRuntimeLeaseTree(t, root)
target := plantFileLink(t, leasePath)
lease, _, err := prepareSandboxRuntimeLeaseRecording(root)
if lease != nil {
lease.release()
}
if err == nil {
t.Fatal("shared acquisition accepted a file link at the lease name, so it is holding a lock on an object cleanup does not see")
}
if !strings.Contains(err.Error(), "reparse point") {
t.Errorf("refused for the wrong reason: %v", err)
}
assertTargetUnlocked(t, target)
}
// And cleanup refuses the same object, so it cannot decide the root is free by
// locking something else.
func TestCleanupLeaseRefusesAFileLinkAtTheLeaseName(t *testing.T) {
cacheRoot := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
leasePath := seedRuntimeLeaseTree(t, root)
target := plantFileLink(t, leasePath)
lease, inUse, err := tryAcquireSandboxRuntimeCleanupLease(root)
if lease != nil {
lease.release()
}
if err == nil {
t.Fatalf("cleanup accepted a file link at the lease name (inUse=%t); it would treat a lock on %s as proof the runtime root is free to delete", inUse, target)
}
if !strings.Contains(err.Error(), "reparse point") {
t.Errorf("refused for the wrong reason: %v", err)
}
assertTargetUnlocked(t, target)
}
// CONTROL: THE TWO SIDES REALLY DO COORDINATE ON AN ORDINARY LEASE.
//
// Without this, refusing everything would satisfy the two tests above. This one
// runs everywhere, including on an unelevated box with no symbolic-link
// privilege, so the mutual-exclusion contract stays pinned even where the link
// cases skip.
func TestASharedLeaseMakesCleanupReportInUse(t *testing.T) {
cacheRoot := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
held, _, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Skipf("cannot acquire a runtime lease here: %v", err)
}
lease, inUse, err := tryAcquireSandboxRuntimeCleanupLease(root)
if lease != nil {
lease.release()
}
if err != nil {
t.Fatalf("cleanup failed against a legitimately held lease: %v", err)
}
if !inUse {
t.Fatal("cleanup reported the runtime root free while a shared lease was held; it would delete a tree a live command is using")
}
held.release()
lease, inUse, err = tryAcquireSandboxRuntimeCleanupLease(root)
if err != nil {
t.Fatalf("cleanup failed after the shared lease was released: %v", err)
}
if inUse {
t.Fatal("cleanup still reported the runtime root in use after the only holder released it, so reclamation never happens")
}
if lease == nil {
t.Fatal("cleanup reported the root free but handed back no lease")
}
lease.release()
}
// CONTROL: a legitimate lease is still SHARED between processes.
//
// The classification must not turn the shared lease into an exclusive one, or
// two concurrent commands on one workspace would stop working.
func TestAnOrdinaryLeaseIsStillSharedByTwoHolders(t *testing.T) {
cacheRoot := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
first, _, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Skipf("cannot acquire a runtime lease here: %v", err)
}
defer first.release()
second, _, err := prepareSandboxRuntimeLeaseRecording(root)
if err != nil {
t.Fatalf("a second holder was refused an existing ordinary lease: %v", err)
}
second.release()
}
// reparseGUIDDataBuffer is the non-Microsoft form of REPARSE_GUID_DATA_BUFFER.
//
// Setting a tag with the Microsoft bit clear needs only write access to the file,
// unlike a symbolic link, so this runs on an ordinary unelevated account and the
// classification stays pinned where the link cases skip.
type reparseGUIDDataBuffer struct {
ReparseTag uint32
ReparseDataLength uint16
Reserved uint16
ReparseGUID windows.GUID
Data [16]byte
}
// plantGenericReparsePoint turns an ordinary file at path into a reparse object.
func plantGenericReparsePoint(t *testing.T, path string) {
t.Helper()
if err := os.WriteFile(path, nil, 0o600); err != nil {
t.Fatal(err)
}
name, err := windows.UTF16PtrFromString(path)
if err != nil {
t.Fatal(err)
}
handle, err := windows.CreateFile(name,
windows.GENERIC_WRITE|windows.FILE_WRITE_ATTRIBUTES, 0, nil,
windows.OPEN_EXISTING,
windows.FILE_FLAG_OPEN_REPARSE_POINT|windows.FILE_FLAG_BACKUP_SEMANTICS, 0)
if err != nil {
t.Skipf("cannot open %s to set a reparse tag: %v", path, err)
}
defer func() { _ = windows.CloseHandle(handle) }()
buffer := reparseGUIDDataBuffer{
// Bit 31 clear, so this is a third-party tag and needs no privilege.
ReparseTag: 0x00000042,
ReparseDataLength: 16,
ReparseGUID: windows.GUID{Data1: 0x5ee0e5f1, Data2: 0x1a11, Data3: 0x4d3b, Data4: [8]byte{0x9a, 0x77, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06}},
}
size := uint32(unsafe.Sizeof(buffer))
var returned uint32
if err := windows.DeviceIoControl(handle, windowsFSCTLSetReparsePoint,
(*byte)(unsafe.Pointer(&buffer)), size, nil, 0, &returned, nil); err != nil {
t.Skipf("cannot set a generic reparse tag here: %v", err)
}
}
const windowsFSCTLSetReparsePoint = 0x000900A4
// ANY REPARSE OBJECT AT THE LEASE NAME IS REFUSED, NOT ONLY A SYMBOLIC LINK.
//
// The requirement is that the name denotes an ordinary file, because that is what
// makes the shared holder and cleanup lock the same thing. Keying on the link
// shape instead would leave every other reparse tag accepted, and it would leave
// this untested on any machine without the symbolic-link privilege.
//
// WHAT THIS CASE CANNOT SHOW, so nobody reads a local pass as more than it is:
// an unknown third-party tag is unresolvable, so the OLD pathname cleanup fails
// on it too, with ERROR_CANT_ACCESS_FILE rather than by classifying anything. It
// therefore pins that both sites refuse, and the reason check below is what
// separates a refusal from an accident. Only the symbolic-link cases above show
// the half that matters most, a pathname open SUCCEEDING on the target, and those
// need the privilege. Read their result in CI, not here.
func TestLeaseRefusesAnyReparseObjectAtTheLeaseName(t *testing.T) {
for name, acquire := range map[string]func(string) error{
"shared": func(root string) error {
lease, _, err := prepareSandboxRuntimeLeaseRecording(root)
if lease != nil {
lease.release()
}
return err
},
"cleanup": func(root string) error {
lease, _, err := tryAcquireSandboxRuntimeCleanupLease(root)
if lease != nil {
lease.release()
}
return err
},
} {
t.Run(name, func(t *testing.T) {
cacheRoot := t.TempDir()
root := leaseRootUnder(t, cacheRoot)
leasePath := seedRuntimeLeaseTree(t, root)
plantGenericReparsePoint(t, leasePath)
// SETUP: the attribute really is set, or this asserts nothing.
info, err := os.Lstat(leasePath)
if err != nil || info.Mode()&os.ModeIrregular == 0 && info.Mode()&os.ModeSymlink == 0 {
attrs, statErr := windowsFileAttributes(leasePath)
if statErr != nil || attrs&windows.FILE_ATTRIBUTE_REPARSE_POINT == 0 {
t.Fatalf("SETUP INVALID: %s is not a reparse object (attrs=%#x err=%v)", leasePath, attrs, statErr)
}
}
err = acquire(root)
if err == nil {
t.Fatalf("%s acquisition accepted a reparse object at the lease name", name)
}
if !strings.Contains(err.Error(), "reparse point") {
t.Errorf("refused for the wrong reason: %v", err)
}
})
}
}
func windowsFileAttributes(path string) (uint32, error) {
name, err := windows.UTF16PtrFromString(path)
if err != nil {
return 0, err
}
return windows.GetFileAttributes(name)
}
@@ -1,282 +0,0 @@
//go:build !windows
package sandbox
import (
"errors"
"fmt"
"os"
"path/filepath"
"golang.org/x/sys/unix"
)
// acquireRuntimeLeaseRootedUnix creates and locks the lease for root without
// resolving an owned component by name.
//
// The lease is the FIRST thing setup writes, so it is the write a redirection has
// to be caught before, not after. It is a sibling of the runtime root rather than
// one of its owned components, which is why the alias guard never looked at it at
// all: it walks the components, and the lease is not one of them.
//
// Only the components above the leaf are created here. The leaf belongs to
// provisioning, which records it for rollback, and creating it here would mean
// two owners for one directory.
func acquireRuntimeLeaseRootedUnix(root string) (*sandboxRuntimeLease, []windowsCreatedRuntimeDir, error) {
base, components, owned := windowsSandboxRuntimeOwnedTail(root)
if !owned || len(components) == 0 {
// Fail rather than fall back to the pathname walk: the walk is the defect.
return nil, nil, fmt.Errorf("acquire sandbox runtime lease for %s: %w", root, errRuntimeTailNotOwned)
}
if runtimeLeasePreCreateBarrier != nil {
runtimeLeasePreCreateBarrier()
}
// The base is the operator's, and may legitimately be a redirected or linked
// temp location, so it is created and opened by name exactly as before.
// Everything below it is Zero's and is addressed by descriptor.
if err := os.MkdirAll(base, 0o700); err != nil {
return nil, nil, fmt.Errorf("create sandbox runtime base: %w", err)
}
// The lease sits beside the leaf, so the deepest directory needed here is the
// leaf's parent.
tail := make([]string, 0, len(components)-1)
current := base
for _, component := range components[:len(components)-1] {
current = filepath.Join(current, component)
tail = append(tail, current)
}
// THE WHOLE ACQUISITION RETRIES, NOT JUST THE LOCK. A contender that waited
// on the lease while cleanup held it exclusively can wake on an object cleanup
// has since unlinked, and by then cleanup has compensated the tail this run
// created as well. Re-taking the lock alone would lock a fresh file beside a
// leaf that is gone, so a stale lock sends the caller back through tail
// creation. The ledger accumulates across attempts: a directory this run made
// on an earlier pass and cleanup did not remove is still this run's to
// compensate.
var created []windowsCreatedRuntimeDir
for attempt := 1; ; attempt++ {
made, parent, err := createRuntimeTailRetainingFD(base, tail)
created = appendCreatedRuntimeDirs(created, made)
if err != nil {
return nil, created, err
}
handle, madeLease, err := acquireSharedRuntimeLeaseAtFD(parent, filepath.Base(sandboxRuntimeLeasePath(root)))
_ = unix.Close(parent)
if err == nil {
return &sandboxRuntimeLease{handle: handle, root: root, createdFile: madeLease}, created, nil
}
if errors.Is(err, errRuntimeLeaseReplaced) && attempt < runtimeLeaseAcquireAttempts {
continue
}
return nil, created, fmt.Errorf("acquire sandbox runtime lease: %w", err)
}
}
// acquireSharedRuntimeLeaseAtFD opens the lease relative to a verified parent and
// takes the shared lock on it.
func acquireSharedRuntimeLeaseAtFD(parent int, name string) (runtimeLeaseHandle, bool, error) {
file, created, err := openRuntimeLeaseAtFD(parent, name)
if err != nil {
return runtimeLeaseHandle{}, false, err
}
if err := unix.Flock(int(file.Fd()), unix.LOCK_SH); err != nil {
// The lock is the last step, and until now the created fact went out only
// alongside a lease object. Failing here returned no object, so this
// invocation's own lease file stayed on disk with nothing recording it,
// and its directory could never be compensated afterwards.
err = undoCreatedRuntimeLeaseAtFD(created, parent, name, err)
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
// THE LOCK IS ON AN INODE, THE COORDINATION IS ON A NAME. Flock blocks while
// cleanup holds the exclusive lock, and cleanup unlinks the lease name before
// it lets go. A waiter then acquires the old, unlinked inode and returns a
// lease that coordinates with nobody: the next cleanup creates a fresh file at
// the name, locks it unopposed, and reports the root unused while this holder
// is still running in it. So the lock is only a lease once the locked object
// is still what the name resolves to. Reported by @gnanam1990.
current, err := runtimeLeaseIsCurrentAtFD(parent, name, int(file.Fd()))
if err != nil {
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
if !current {
// Not undone by name, whatever created says: the name is no longer this
// object's, so an unlink there would take somebody else's lease.
_ = file.Close()
return runtimeLeaseHandle{}, false, errRuntimeLeaseReplaced
}
return runtimeLeaseHandle{file: file}, created, nil
}
// runtimeLeaseIsCurrentAtFD reports whether the locked lease is still the object
// the lease name resolves to under parent.
//
// Both answers come from the descriptor side: the locked file's own identity and
// the directory entry read relative to the retained parent, no-follow. A missing
// entry, a different inode at the name, or a locked inode with no links left all
// mean the coordination moved on while this call was waiting.
func runtimeLeaseIsCurrentAtFD(parent int, name string, fd int) (bool, error) {
var locked, entry unix.Stat_t
if err := unix.Fstat(fd, &locked); err != nil {
return false, fmt.Errorf("inspect the locked sandbox runtime lease %s: %w", name, err)
}
if locked.Nlink == 0 {
return false, nil
}
if err := unix.Fstatat(parent, name, &entry, unix.AT_SYMLINK_NOFOLLOW); err != nil {
if errors.Is(err, unix.ENOENT) {
return false, nil
}
return false, fmt.Errorf("inspect the sandbox runtime lease entry %s: %w", name, err)
}
return locked.Dev == entry.Dev && locked.Ino == entry.Ino, nil
}
// undoCreatedRuntimeLeaseAtFD removes a lease file this call created, when a
// later step means no lease object will be returned to carry that fact.
//
// Relative to the verified parent descriptor, and only when this call is the one
// that created the file: a lease that was already there belongs to whoever is
// holding it.
func undoCreatedRuntimeLeaseAtFD(created bool, parent int, name string, cause error) error {
if !created {
return cause
}
if undoErr := unix.Unlinkat(parent, name, 0); undoErr != nil {
return fmt.Errorf("%w; and the lease file this run created could not be removed: %w", cause, undoErr)
}
return cause
}
// openRuntimeLeaseAtFD opens or creates the lease under parent and proves it is
// an ordinary file.
//
// O_NOFOLLOW so a symlink planted at the lease name is an ELOOP rather than an
// open of its target, and a regular-file check besides, because O_NOFOLLOW says
// nothing about a fifo or a device a local user can also create there. Both
// holders have to end up on the same object or the lock protects nothing.
func openRuntimeLeaseAtFD(parent int, name string) (*os.File, bool, error) {
const flags = unix.O_RDWR | unix.O_NOFOLLOW | unix.O_CLOEXEC
// O_EXCL FIRST, so the create is what reports the create. A Stat beforehand
// answers about a moment that has passed by the time the open runs, and
// compensation would then delete a lease another process had just made.
created := true
fd, err := unix.Openat(parent, name, flags|unix.O_CREAT|unix.O_EXCL, 0o600)
if errors.Is(err, unix.EEXIST) {
created = false
fd, err = unix.Openat(parent, name, flags, 0)
}
if err != nil {
if errors.Is(err, unix.ELOOP) {
return nil, false, fmt.Errorf("refusing to use the sandbox runtime lease at %s: it is a link, so the holders would lock different objects: %w", name, err)
}
return nil, false, err
}
// Every failure from here on has already created the file when created is
// true, and returns no lease object to carry that fact, so each one undoes
// its own creation rather than leaving an unrecorded artifact behind.
if runtimeCreationFailure != nil {
if injected := runtimeCreationFailure(name); injected != nil {
injected = undoCreatedRuntimeLeaseAtFD(created, parent, name, injected)
_ = unix.Close(fd)
return nil, false, injected
}
}
var stat unix.Stat_t
if err := unix.Fstat(fd, &stat); err != nil {
err = undoCreatedRuntimeLeaseAtFD(created, parent, name, fmt.Errorf("inspect the sandbox runtime lease %s: %w", name, err))
_ = unix.Close(fd)
return nil, false, err
}
if stat.Mode&unix.S_IFMT != unix.S_IFREG {
err := undoCreatedRuntimeLeaseAtFD(created, parent, name, fmt.Errorf("refusing to use the sandbox runtime lease at %s: it is not an ordinary file (mode %#o)", name, stat.Mode&unix.S_IFMT))
_ = unix.Close(fd)
return nil, false, err
}
if uid := unix.Getuid(); uid >= 0 && stat.Uid != uint32(uid) {
err := undoCreatedRuntimeLeaseAtFD(created, parent, name, fmt.Errorf("refusing to use the sandbox runtime lease at %s: it is owned by uid %d, not %d", name, stat.Uid, uid))
_ = unix.Close(fd)
return nil, false, err
}
file := os.NewFile(uintptr(fd), name)
if file == nil {
err := undoCreatedRuntimeLeaseAtFD(created, parent, name, fmt.Errorf("wrap the sandbox runtime lease handle for %s", name))
_ = unix.Close(fd)
return nil, false, err
}
return file, created, nil
}
// tryAcquireExclusiveRuntimeLeaseRootedUnix is cleanup's acquisition, resolved
// the way acquisition resolves it.
//
// It opens and never creates the tree: a runtime root that is not there has no
// lease to take, and cleanup rebuilding it in order to lock it would be inventing
// the thing it is about to remove.
func tryAcquireExclusiveRuntimeLeaseRootedUnix(root string) (runtimeLeaseHandle, bool, error) {
base, components, owned := windowsSandboxRuntimeOwnedTail(root)
if !owned || len(components) == 0 {
return runtimeLeaseHandle{}, false, fmt.Errorf("open the sandbox runtime lease parent for %s: %w", root, errRuntimeTailNotOwned)
}
parent, err := unix.Open(base, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
if err != nil {
return runtimeLeaseHandle{}, false, fmt.Errorf("open sandbox runtime base %s: %w", base, err)
}
path := base
for _, name := range components[:len(components)-1] {
path = filepath.Join(path, name)
child, openErr := unix.Openat(parent, name, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_NOFOLLOW|unix.O_CLOEXEC, 0)
if openErr != nil {
_ = unix.Close(parent)
return runtimeLeaseHandle{}, false, fmt.Errorf("open sandbox runtime component %s: %w", path, openErr)
}
if err := refuseForeignRuntimeDirectory(child, path); err != nil {
_ = unix.Close(child)
_ = unix.Close(parent)
return runtimeLeaseHandle{}, false, err
}
_ = unix.Close(parent)
parent = child
}
defer func() { _ = unix.Close(parent) }()
name := filepath.Base(sandboxRuntimeLeasePath(root))
for attempt := 1; ; attempt++ {
file, created, err := openRuntimeLeaseAtFD(parent, name)
if err != nil {
return runtimeLeaseHandle{}, false, err
}
if err := unix.Flock(int(file.Fd()), unix.LOCK_EX|unix.LOCK_NB); err != nil {
if errors.Is(err, unix.EWOULDBLOCK) || errors.Is(err, unix.EAGAIN) {
// Somebody holds it, so it is not this call's to remove, whatever the
// open reported. A file this call created cannot reach here anyway.
_ = file.Close()
return runtimeLeaseHandle{}, true, nil
}
err = undoCreatedRuntimeLeaseAtFD(created, parent, name, err)
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
// The exclusive side has the same window in miniature: between its open
// and its lock another cleanup can unlink and replace the name, and an
// exclusive lock on the old inode would then remove a tree a holder of the
// new one is entering. Same check, same answer: only the current object
// counts, and a stale one is reopened rather than reasoned about.
current, err := runtimeLeaseIsCurrentAtFD(parent, name, int(file.Fd()))
if err != nil {
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
if !current {
_ = file.Close()
if attempt < runtimeLeaseAcquireAttempts {
continue
}
return runtimeLeaseHandle{}, false, errRuntimeLeaseReplaced
}
return runtimeLeaseHandle{file: file}, false, nil
}
}
@@ -1,408 +0,0 @@
//go:build windows
package sandbox
import (
"errors"
"fmt"
"os"
"path/filepath"
"unsafe"
"golang.org/x/sys/windows"
)
// acquireRuntimeLeaseRooted creates and locks the lease file for root without
// ever resolving an owned component by name.
//
// THE LEASE IS ACQUIRED BEFORE ANYTHING ELSE, SO IT CANNOT BE THE WEAK LINK.
//
// Provisioning already descends from the fixed cache or TEMP base through
// retained no-follow handles, because a predictable owned component is exactly
// what an ordinary same-account process can replace with a junction. Lease
// acquisition ran first and did neither: it checked the components for aliases,
// then called os.MkdirAll on the parent by pathname and opened
// "<root>.lease" by pathname, both of which follow. A junction dropped on
// "zero", "runtime" or "v1" between the check and either call put elevated
// setup's first writes inside somebody else's tree, and restoring the component
// afterwards left the later handle-relative provisioning working on the
// legitimate tree so no post-check ever saw it.
//
// The lease file is a SIBLING of the runtime root rather than one of its owned
// components, so refuseAliasedRuntimeComponents never inspected it at all. Here
// it is created relative to the retained handle for the directory that contains
// it, which is the deepest owned component, so its name is resolved exactly once
// and relative to a verified object.
//
// Only the components above the leaf are created. The leaf itself belongs to
// provisioning, which records it for rollback; creating it here would mean two
// owners for one directory.
func acquireRuntimeLeaseRooted(root string) (*sandboxRuntimeLease, []windowsCreatedRuntimeDir, error) {
base, components, owned := windowsSandboxRuntimeOwnedTail(root)
if !owned {
// Fail rather than fall back to the pathname walk: the walk is the defect.
return nil, nil, fmt.Errorf("acquire sandbox runtime lease for %s: %w", root, errRuntimeTailNotOwned)
}
if len(components) == 0 {
return nil, nil, fmt.Errorf("acquire sandbox runtime lease for %s: %w", root, errRuntimeTailNotOwned)
}
// The base is the operator's, and may legitimately be a redirected cache or
// TEMP location, so it is created and opened by name exactly as provisioning
// does. Everything below it is Zero's and is addressed by handle.
if runtimeLeasePreCreateBarrier != nil {
runtimeLeasePreCreateBarrier()
}
if err := os.MkdirAll(base, 0o700); err != nil {
return nil, nil, fmt.Errorf("create sandbox runtime base: %w", err)
}
// The lease sits beside the leaf, so the deepest directory needed here is the
// leaf's parent.
parents := components[:len(components)-1]
tail := make([]string, 0, len(parents))
current := base
for _, component := range parents {
current = filepath.Join(current, component)
tail = append(tail, current)
}
// THE WHOLE ACQUISITION RETRIES, NOT JUST THE LOCK. A contender that waited
// on the lease while cleanup held it exclusively can wake on an object cleanup
// has since marked for deletion, and by then cleanup has compensated the tail
// this run created as well. Re-taking the lock alone would lock a fresh file
// beside a leaf that is gone, so a stale lock sends the caller back through
// tail creation. The ledger accumulates across attempts: a directory this run
// made on an earlier pass and cleanup did not remove is still this run's to
// compensate.
var created []windowsCreatedRuntimeDir
for attempt := 1; ; attempt++ {
made, parent, err := createRuntimeTailRetainingHandle(base, tail)
created = appendCreatedRuntimeDirs(created, made)
if err != nil {
if parent != 0 {
_ = windows.CloseHandle(parent)
}
return nil, created, err
}
handle, madeLease, err := acquireSharedRuntimeLeaseAt(parent, filepath.Base(sandboxRuntimeLeasePath(root)))
_ = windows.CloseHandle(parent)
if err == nil {
return &sandboxRuntimeLease{handle: handle, root: root, createdFile: madeLease}, created, nil
}
if errors.Is(err, errRuntimeLeaseReplaced) && attempt < runtimeLeaseAcquireAttempts {
continue
}
return nil, created, fmt.Errorf("acquire sandbox runtime lease: %w", err)
}
}
// acquireSharedRuntimeLeaseAt is acquireSharedRuntimeLease with the file named
// relative to a directory handle instead of by full path.
func acquireSharedRuntimeLeaseAt(parent windows.Handle, name string) (runtimeLeaseHandle, bool, error) {
objectName, err := windows.NewNTUnicodeString(name)
if err != nil {
return runtimeLeaseHandle{}, false, fmt.Errorf("encode sandbox runtime lease name %s: %w", name, err)
}
attributes := windows.OBJECT_ATTRIBUTES{
RootDirectory: parent,
ObjectName: objectName,
Attributes: windows.OBJ_CASE_INSENSITIVE,
}
attributes.Length = uint32(unsafe.Sizeof(attributes))
var handle windows.Handle
var iosb windows.IO_STATUS_BLOCK
// FILE_OPEN_IF because the lease is shared: whoever gets there first creates
// it. FILE_NON_DIRECTORY_FILE so a directory under that name is refused.
//
// FILE_OPEN_REPARSE_POINT IS NOT A REFUSAL. It says do not follow, so the call
// returns a handle to the LINK, and FILE_NON_DIRECTORY_FILE excludes
// directories rather than non-directory reparse objects. A file symbolic link
// planted at <digest>.lease was therefore opened and locked as if it were the
// lease. No-follow and classification are two requirements; the flag is only
// the first, and the handle is asked for the second below.
err = windows.NtCreateFile(
&handle,
windows.DELETE|windows.GENERIC_READ|windows.GENERIC_WRITE|windows.SYNCHRONIZE,
&attributes,
&iosb,
nil,
windows.FILE_ATTRIBUTE_NORMAL,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
windows.FILE_OPEN_IF,
windows.FILE_NON_DIRECTORY_FILE|windows.FILE_SYNCHRONOUS_IO_NONALERT|windows.FILE_OPEN_REPARSE_POINT,
0,
0,
)
if err != nil {
return runtimeLeaseHandle{}, false, err
}
// FILE_CREATED here, rather than a Stat before the call, because only the
// create itself can distinguish the file it made from one that arrived a
// moment earlier.
created := iosb.Information == windowsFileCreatedDisposition
// And every failure below has already created the file when that is true,
// while returning no lease object to carry the fact, so each undoes its own
// creation instead of leaving an unrecorded artifact in a directory
// compensation then cannot empty.
if runtimeCreationFailure != nil {
if injected := runtimeCreationFailure(name); injected != nil {
injected = undoCreatedRuntimeLease(created, handle, injected)
_ = windows.CloseHandle(handle)
return runtimeLeaseHandle{}, false, injected
}
}
if err := refuseReparseRuntimeLeaseHandle(handle, name); err != nil {
err = undoCreatedRuntimeLease(created, handle, err)
_ = windows.CloseHandle(handle)
return runtimeLeaseHandle{}, false, err
}
file := os.NewFile(uintptr(handle), name)
if file == nil {
err := undoCreatedRuntimeLease(created, handle, fmt.Errorf("wrap the sandbox runtime lease handle for %s", name))
_ = windows.CloseHandle(handle)
return runtimeLeaseHandle{}, false, err
}
lease := runtimeLeaseHandle{file: file}
if err := windows.LockFileEx(windows.Handle(file.Fd()), 0, 0, 1, 0, &lease.overlapped); err != nil {
err = undoCreatedRuntimeLease(created, windows.Handle(file.Fd()), err)
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
// THE LOCK IS ON AN OBJECT, THE COORDINATION IS ON A NAME. LockFileEx waits
// while cleanup holds the exclusive lock, and cleanup marks the lease for
// deletion before it lets go; this handle was opened with FILE_SHARE_DELETE
// precisely so cleanup could. A waiter then locks a delete-pending object and
// returns a lease that coordinates with nobody: once this handle closes the
// name is free, the next cleanup creates a fresh file there, locks it
// unopposed, and reports the root unused while this holder is still running
// in it. So the lock is only a lease once the locked object is still what the
// name resolves to. Reported by @gnanam1990.
current, err := runtimeLeaseIsCurrentAt(parent, name, windows.Handle(file.Fd()))
if err != nil {
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
if !current {
// Not undone, whatever created says: the object is already going, and the
// name now belongs to whoever comes next. Closing this handle is what lets
// the name go so the retry can take a current one.
_ = file.Close()
return runtimeLeaseHandle{}, false, errRuntimeLeaseReplaced
}
return lease, created, nil
}
// runtimeLeaseIsCurrentAt reports whether the locked lease is still the object
// the lease name resolves to under parent.
//
// Both answers come from handles: the locked file's own identity, and a fresh
// no-follow open of the name relative to the retained parent, asked for nothing
// but attributes. A name that is gone or delete-pending, or that resolves to a
// different volume and file index, means the coordination moved on while this
// call was waiting.
func runtimeLeaseIsCurrentAt(parent windows.Handle, name string, locked windows.Handle) (bool, error) {
var lockedInfo windows.ByHandleFileInformation
if err := windows.GetFileInformationByHandle(locked, &lockedInfo); err != nil {
return false, fmt.Errorf("inspect the locked sandbox runtime lease %s: %w", name, err)
}
objectName, err := windows.NewNTUnicodeString(name)
if err != nil {
return false, fmt.Errorf("encode sandbox runtime lease name %s: %w", name, err)
}
attributes := windows.OBJECT_ATTRIBUTES{
RootDirectory: parent,
ObjectName: objectName,
Attributes: windows.OBJ_CASE_INSENSITIVE,
}
attributes.Length = uint32(unsafe.Sizeof(attributes))
var probe windows.Handle
var iosb windows.IO_STATUS_BLOCK
err = windows.NtCreateFile(
&probe,
windows.FILE_READ_ATTRIBUTES|windows.SYNCHRONIZE,
&attributes,
&iosb,
nil,
0,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
windows.FILE_OPEN,
windows.FILE_NON_DIRECTORY_FILE|windows.FILE_SYNCHRONOUS_IO_NONALERT|windows.FILE_OPEN_REPARSE_POINT,
0,
0,
)
if err != nil {
if errors.Is(err, windows.STATUS_OBJECT_NAME_NOT_FOUND) || errors.Is(err, windows.STATUS_DELETE_PENDING) {
return false, nil
}
return false, fmt.Errorf("inspect the sandbox runtime lease entry %s: %w", name, err)
}
defer func() { _ = windows.CloseHandle(probe) }()
var entryInfo windows.ByHandleFileInformation
if err := windows.GetFileInformationByHandle(probe, &entryInfo); err != nil {
return false, fmt.Errorf("inspect the sandbox runtime lease entry %s: %w", name, err)
}
return lockedInfo.VolumeSerialNumber == entryInfo.VolumeSerialNumber &&
lockedInfo.FileIndexHigh == entryInfo.FileIndexHigh &&
lockedInfo.FileIndexLow == entryInfo.FileIndexLow, nil
}
// undoCreatedRuntimeLease removes a lease file this call created, when a later
// step means no lease object will be returned to carry that fact.
//
// Through the handle, and only when this call is the one that created the file:
// a lease that was already there belongs to whoever is holding it.
func undoCreatedRuntimeLease(created bool, handle windows.Handle, cause error) error {
if !created {
return cause
}
if undoErr := removeWindowsObjectByHandle(handle); undoErr != nil {
return fmt.Errorf("%w; and the lease file this run created could not be removed: %w", cause, undoErr)
}
return cause
}
// refuseReparseRuntimeLeaseHandle proves the opened lease is an ordinary file.
//
// Asked of the HANDLE, not the name, so there is no second resolution for a
// substitution to land in. This is the check the directory descent already makes
// in openWindowsChildNoFollow; the lease carried the no-follow flag and not the
// classification that has to go with it.
func refuseReparseRuntimeLeaseHandle(handle windows.Handle, name string) error {
var info windows.ByHandleFileInformation
if err := windows.GetFileInformationByHandle(handle, &info); err != nil {
return fmt.Errorf("inspect the sandbox runtime lease %s: %w", name, err)
}
if info.FileAttributes&windows.FILE_ATTRIBUTE_REPARSE_POINT != 0 {
return fmt.Errorf("refusing to use the sandbox runtime lease at %s: a reparse point here means the shared and exclusive holders would lock different objects", name)
}
return nil
}
// openRuntimeLeaseParentRooted opens, and never creates, the directory holding
// the lease for root.
//
// Cleanup's side of the contract. It descends the same owned components from the
// same base with the same no-follow opens acquisition uses, so both sides resolve
// the lease name relative to a verified directory rather than by pathname.
//
// It creates nothing. A runtime tree that is not there has no lease to take, and
// cleanup rebuilding the tree in order to lock it would be inventing the thing it
// is about to remove.
func openRuntimeLeaseParentRooted(root string) (windows.Handle, error) {
base, components, owned := windowsSandboxRuntimeOwnedTail(root)
if !owned || len(components) == 0 {
return 0, fmt.Errorf("open the sandbox runtime lease parent for %s: %w", root, errRuntimeTailNotOwned)
}
parent, err := openWindowsDirectoryByName(base)
if err != nil {
return 0, fmt.Errorf("open sandbox runtime base %s: %w", base, err)
}
// The lease is a SIBLING of the leaf, so the deepest directory needed here is
// the leaf's parent, exactly as in acquisition.
for _, name := range components[:len(components)-1] {
child, openErr := openWindowsChildNoFollow(parent, name,
windows.FILE_READ_ATTRIBUTES|windows.FILE_TRAVERSE, windows.FILE_DIRECTORY_FILE)
if openErr != nil {
_ = windows.CloseHandle(parent)
return 0, openErr
}
_ = windows.CloseHandle(parent)
parent = child
}
return parent, nil
}
// tryAcquireExclusiveRuntimeLeaseRooted is cleanup's acquisition, resolved the
// way acquisition resolves it.
func tryAcquireExclusiveRuntimeLeaseRooted(root string) (runtimeLeaseHandle, bool, error) {
parent, err := openRuntimeLeaseParentRooted(root)
if err != nil {
return runtimeLeaseHandle{}, false, err
}
defer func() { _ = windows.CloseHandle(parent) }()
name := filepath.Base(sandboxRuntimeLeasePath(root))
objectName, err := windows.NewNTUnicodeString(name)
if err != nil {
return runtimeLeaseHandle{}, false, fmt.Errorf("encode sandbox runtime lease name %s: %w", name, err)
}
attributes := windows.OBJECT_ATTRIBUTES{
RootDirectory: parent,
ObjectName: objectName,
Attributes: windows.OBJ_CASE_INSENSITIVE,
}
attributes.Length = uint32(unsafe.Sizeof(attributes))
for attempt := 1; ; attempt++ {
var handle windows.Handle
var iosb windows.IO_STATUS_BLOCK
// FILE_OPEN_IF matches what cleanup did by pathname with O_CREATE: a
// runtime root whose lease file is gone is held by nobody, and creating the
// empty lease is how that is expressed.
err = windows.NtCreateFile(
&handle,
windows.DELETE|windows.GENERIC_READ|windows.GENERIC_WRITE|windows.SYNCHRONIZE,
&attributes,
&iosb,
nil,
windows.FILE_ATTRIBUTE_NORMAL,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
windows.FILE_OPEN_IF,
windows.FILE_NON_DIRECTORY_FILE|windows.FILE_SYNCHRONOUS_IO_NONALERT|windows.FILE_OPEN_REPARSE_POINT,
0,
0,
)
if err != nil {
return runtimeLeaseHandle{}, false, err
}
created := iosb.Information == windowsFileCreatedDisposition
if err := refuseReparseRuntimeLeaseHandle(handle, name); err != nil {
err = undoCreatedRuntimeLease(created, handle, err)
_ = windows.CloseHandle(handle)
return runtimeLeaseHandle{}, false, err
}
file := os.NewFile(uintptr(handle), name)
if file == nil {
err := undoCreatedRuntimeLease(created, handle, fmt.Errorf("wrap the sandbox runtime lease handle for %s", name))
_ = windows.CloseHandle(handle)
return runtimeLeaseHandle{}, false, err
}
lease := runtimeLeaseHandle{file: file}
flags := uint32(windows.LOCKFILE_EXCLUSIVE_LOCK | windows.LOCKFILE_FAIL_IMMEDIATELY)
if err := windows.LockFileEx(windows.Handle(file.Fd()), flags, 0, 1, 0, &lease.overlapped); err != nil {
if errors.Is(err, windows.ERROR_LOCK_VIOLATION) {
// Somebody holds it, so it is not this call's to remove, whatever the
// open reported. A file this call created cannot reach here anyway.
_ = file.Close()
return runtimeLeaseHandle{}, true, nil
}
err = undoCreatedRuntimeLease(created, windows.Handle(file.Fd()), err)
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
// The exclusive side has the same window in miniature: between its open
// and its lock another cleanup can mark the object for deletion, and an
// exclusive lock on that object would then remove a tree a holder of the
// replacement is entering. Same check, same answer: only the current
// object counts, and a stale one is reopened rather than reasoned about.
current, err := runtimeLeaseIsCurrentAt(parent, name, windows.Handle(file.Fd()))
if err != nil {
_ = file.Close()
return runtimeLeaseHandle{}, false, err
}
if !current {
_ = file.Close()
if attempt < runtimeLeaseAcquireAttempts {
continue
}
return runtimeLeaseHandle{}, false, errRuntimeLeaseReplaced
}
return lease, false, nil
}
}
// windowsFileCreatedDisposition is the IO_STATUS_BLOCK Information value that
// says NtCreateFile made the file rather than opening one that was there.
// x/sys/windows does not export it.
const windowsFileCreatedDisposition = 2
+26 -9
View File
@@ -3,6 +3,7 @@
package sandbox
import (
"errors"
"os"
"golang.org/x/sys/unix"
@@ -12,15 +13,31 @@ type runtimeLeaseHandle struct {
file *os.File
}
// BOTH SIDES HAVE TO MEAN THE SAME OBJECT.
//
// Cleanup opened the lease by full pathname while acquisition opened it relative
// to a verified parent, so a symlink at <digest>.lease gave the two of them
// different files: a live command held a shared lock on one, cleanup took an
// exclusive lock on the other, both succeeded, and cleanup went on to remove a
// runtime root that was still in use.
func tryAcquireExclusiveRuntimeLease(root string) (runtimeLeaseHandle, bool, error) {
return tryAcquireExclusiveRuntimeLeaseRootedUnix(root)
func acquireSharedRuntimeLease(path string) (runtimeLeaseHandle, error) {
file, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
if err != nil {
return runtimeLeaseHandle{}, err
}
if err := unix.Flock(int(file.Fd()), unix.LOCK_SH); err != nil {
_ = file.Close()
return runtimeLeaseHandle{}, err
}
return runtimeLeaseHandle{file: file}, nil
}
func tryAcquireExclusiveRuntimeLease(path string) (runtimeLeaseHandle, bool, error) {
file, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
if err != nil {
return runtimeLeaseHandle{}, false, err
}
if err := unix.Flock(int(file.Fd()), unix.LOCK_EX|unix.LOCK_NB); err != nil {
_ = file.Close()
if errors.Is(err, unix.EWOULDBLOCK) || errors.Is(err, unix.EAGAIN) {
return runtimeLeaseHandle{}, true, nil
}
return runtimeLeaseHandle{}, false, err
}
return runtimeLeaseHandle{file: file}, false, nil
}
func (lease runtimeLeaseHandle) release() {
Loaded 100 of 226 files, more files were not shown because too many files have changed in this diff. Show more