mirror of
https://github.com/Twigpine/zero.git
synced 2026-11-09 18:08:31 -10:00
Compare commits
21
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f317a79a02 | ||
|
|
90143736a8 | ||
|
|
c8144df06d | ||
|
|
dd763d9921 | ||
|
|
f4df51cc48 | ||
|
|
197026c5e9 | ||
|
|
fa1dbced03 | ||
|
|
88716c4fb1 | ||
|
|
91a25079ce | ||
|
|
b32f7ce4d6 | ||
|
|
8e9c3ec4ce | ||
|
|
ef1ca39b09 | ||
|
|
bbf779e691 | ||
|
|
47f78c6624 | ||
|
|
e034a93815 | ||
|
|
2ca84cc127 | ||
|
|
05c1b5e160 | ||
|
|
16ccd78765 | ||
|
|
c9f179482a | ||
|
|
98736457bd | ||
|
|
e59632406b |
No files matched your search
@@ -1 +1 @@
|
||||
{".":"0.12.0"}
|
||||
{".":"0.13.0"}
|
||||
@@ -6,6 +6,28 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once the first release is
|
||||
tagged. Until then, source builds report the version `dev`.
|
||||
|
||||
## [0.13.0](https://github.com/Twigpine/zero/compare/v0.12.0...v0.13.0) (2026-11-09)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **mcp:** add AI/ML API as an opt-in built-in server ([#1193](https://github.com/Twigpine/zero/issues/1193)) ([16ccd78](https://github.com/Twigpine/zero/commit/16ccd78765b6a2a56b7370e3031710d718aa4757))
|
||||
* **plugins:** add CLI enable/disable for plugin.json ([#1202](https://github.com/Twigpine/zero/issues/1202)) ([05c1b5e](https://github.com/Twigpine/zero/commit/05c1b5e16098e1c60212316d20ae8e055a27b5ae))
|
||||
* **providers:** point OpenGateway at api.twigpine.com and trust its live model list ([#1208](https://github.com/Twigpine/zero/issues/1208)) ([47f78c6](https://github.com/Twigpine/zero/commit/47f78c66247aa96c8753d6bb95e935126ff1761e))
|
||||
* **web-fetch:** add an opt-in AI/ML API Toolbox scraping backend ([#1192](https://github.com/Twigpine/zero/issues/1192)) ([c9f1794](https://github.com/Twigpine/zero/commit/c9f179482a7f6d7111ee64fa9cb8ffa2517c2397))
|
||||
* **web-search:** add AI/ML API Toolbox as an opt-in backend ([#1189](https://github.com/Twigpine/zero/issues/1189)) ([9873645](https://github.com/Twigpine/zero/commit/98736457bdac6b0272e032728a5485cf557780fe))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **hooks:** dispatch specialistStart/Stop lifecycle hooks ([#1204](https://github.com/Twigpine/zero/issues/1204)) ([2ca84cc](https://github.com/Twigpine/zero/commit/2ca84cc127d9369765004b72cf95049fecc6c28b))
|
||||
* **tui:** make reasoning stream rendering linear (coalesced deltas, O(1) append, tail-bounded preview) ([#1198](https://github.com/Twigpine/zero/issues/1198)) ([bbf779e](https://github.com/Twigpine/zero/commit/bbf779e69168d62bb1103aefdd1b10e53433209e))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* **tui:** bounded file read, LRU rendering cache, and size caps for full-file view ([#1203](https://github.com/Twigpine/zero/issues/1203)) ([e034a93](https://github.com/Twigpine/zero/commit/e034a93815629c68976dcdab7dcc08bbe9b03693))
|
||||
|
||||
## [0.12.0](https://github.com/Twigpine/zero/compare/v0.11.0...v0.12.0) (2026-11-02)
|
||||
|
||||
|
||||
|
||||
@@ -397,6 +397,76 @@ and can rewrite every request through it, so this is the trust you accept by
|
||||
configuring one. Leave the variables unset for the checks to be enforced end to
|
||||
end.
|
||||
|
||||
The built-in `web_search` tool is registered only when a search backend is
|
||||
configured: `ZERO_WEBSEARCH_BASE_URL` (with an optional `ZERO_WEBSEARCH_API_KEY`)
|
||||
for a generic JSON endpoint, `ZERO_WEBSEARCH_PROVIDER=searxng` with a SearXNG
|
||||
base URL, or the AI/ML API Toolbox:
|
||||
|
||||
```bash
|
||||
export ZERO_WEBSEARCH_PROVIDER=aimlapi
|
||||
```
|
||||
|
||||
The AI/ML API backend runs the Toolbox tool `web/search`, and every search is
|
||||
billed to your AI/ML API balance: about $0.0013 per result, with a documented
|
||||
minimum of $0.0091 per search. Each search is sent with a $0.025 spending cap,
|
||||
which the Toolbox enforces. It is used only when
|
||||
`ZERO_WEBSEARCH_PROVIDER` is set to `aimlapi`; an AI/ML API key alone never
|
||||
enables it. The key is read from your user configuration only; provider
|
||||
profiles in a project `.zero` config are not used. It is the aimlapi profile's
|
||||
key (the active profile when it is aimlapi, otherwise the first aimlapi
|
||||
profile), including a key in the credential store, or `AIMLAPI_API_KEY` when no
|
||||
aimlapi profile exists. It is resolved again on every search;
|
||||
`ZERO_WEBSEARCH_API_KEY` is not used. When any aimlapi profile, or any profile
|
||||
that reads `AIMLAPI_API_KEY`, points at a base URL other than
|
||||
`https://api.aimlapi.com/v1`, or `AIMLAPI_INFERENCE_URL` is overridden, the key
|
||||
may belong to a proxy, so it is not sent and the search fails without a request.
|
||||
Requests go only to
|
||||
`https://tools.aimlapi.com` (`ZERO_WEBSEARCH_BASE_URL` is ignored and redirects
|
||||
are refused) and carry `X-AIMLAPI-Source: agent/zero` and an
|
||||
`X-AIMLAPI-Partner-ID` header. The `domains` argument is sent to the Toolbox as
|
||||
`include_domains`, and results are still filtered locally.
|
||||
|
||||
### AI/ML API Toolbox backend for `web_fetch`
|
||||
|
||||
`web_fetch` can fetch pages through the AI/ML API Toolbox tool
|
||||
`search1api/crawl` instead of a direct HTTP GET. It is off unless you choose it:
|
||||
|
||||
```bash
|
||||
export ZERO_WEBFETCH_PROVIDER=aimlapi
|
||||
```
|
||||
|
||||
Having an AI/ML API key never switches it on by itself, and any other value (or
|
||||
none) keeps the direct fetch. The variable is read on every fetch.
|
||||
|
||||
- **Billing:** Toolbox runs are billed to your AI/ML API balance, about
|
||||
$0.0013 per run. Each run is sent with `max_cost_usd` 0.005, so a run
|
||||
priced above that is refused before it starts and not charged. Rate-limited
|
||||
or gateway-failed requests are retried with the same idempotency key, so a
|
||||
retry is never billed twice.
|
||||
- **Key:** the key comes from your `aimlapi` provider profile (including a key
|
||||
in the credential store), or from `AIMLAPI_API_KEY` when there is no such
|
||||
profile. It is withheld, and `web_fetch` returns an error, when any `aimlapi`
|
||||
profile or any profile using `AIMLAPI_API_KEY` points at a base URL other than
|
||||
`https://api.aimlapi.com/v1`, or when `AIMLAPI_INFERENCE_URL` is overridden:
|
||||
the key may then belong to a proxy.
|
||||
- **Destination:** requests go only to the fixed `https://tools.aimlapi.com`
|
||||
endpoint, with `Authorization: Bearer <key>` and the attribution headers
|
||||
`X-AIMLAPI-Source: agent/zero` and `X-AIMLAPI-Partner-ID`. Redirects are
|
||||
refused. `HTTP_PROXY`/`HTTPS_PROXY` apply to these requests as to Zero's other
|
||||
aimlapi.com calls.
|
||||
- **No fallback:** without a usable key, or when the run fails, `web_fetch`
|
||||
returns an error and does not fetch the page directly. Unset the variable to
|
||||
go back to direct fetches.
|
||||
- **Remote fetch:** the page is fetched by a remote service, which may serve a
|
||||
cached copy, and the full URL (query string included) is sent to it. The URL
|
||||
checks above still run first, so loopback, private and metadata URLs are
|
||||
never handed to it. The result is Markdown with a
|
||||
`Backend: aimlapi (search1api/crawl)` line; `max_bytes` cuts the Markdown,
|
||||
and `format: "raw"` is refused because the original response is not
|
||||
available. The complete Toolbox run response is limited to 8 MiB before JSON
|
||||
decoding; larger responses return an error even if `max_bytes` is smaller.
|
||||
The remote run may already have been billed when this limit is detected.
|
||||
|
||||
For local dev servers, use shell commands such as `curl` through `exec_command`
|
||||
so the normal sandbox and permission policy applies. Long-running commands stay
|
||||
attached to a background terminal session and can be listed or stopped from the
|
||||
|
||||
+13
-1
@@ -33,4 +33,16 @@ Please include, where possible:
|
||||
depend on a user explicitly disabling the sandbox (`--skip-permissions-unsafe`), are generally
|
||||
out of scope — but please report anything you are unsure about.
|
||||
- Zero sends no telemetry. Network calls go to the model/tool providers you configure (and, for
|
||||
keyless web search, to Exa's hosted MCP endpoint at `https://mcp.exa.ai/mcp`).
|
||||
keyless web search, to Exa's hosted MCP endpoint at `https://mcp.exa.ai/mcp`). The built-in
|
||||
AI/ML API MCP server (`https://mcp.aimlapi.com/mcp`) is off until you run
|
||||
`zero mcp enable aimlapi`. Once enabled, the Bearer key follows the aimlapi
|
||||
profile resolution order (inline key, then the profile's `apiKeyEnv` — default
|
||||
`AIMLAPI_API_KEY` — then the credential store), or `AIMLAPI_API_KEY` alone when
|
||||
no aimlapi profile exists; a non-empty `AIMLAPI_API_KEY` can override a stored
|
||||
profile key for MCP billing. The automatic key is withheld if an aimlapi profile
|
||||
(or a profile reading `AIMLAPI_API_KEY`) uses a base URL other than
|
||||
`https://api.aimlapi.com/v1`, `AIMLAPI_INFERENCE_URL` points elsewhere, a
|
||||
non-empty user Authorization header is configured, OAuth is selected, project
|
||||
config has touched the server, or the key is missing or not a single token
|
||||
(including Unicode whitespace). See
|
||||
[docs/EXTENDING.md](docs/EXTENDING.md#built-in-servers).
|
||||
+58
-2
@@ -203,7 +203,11 @@ Events the agent emits (in dispatch order):
|
||||
| `specialistStart` | A sub-agent is spawned | yes (specialist name) |
|
||||
| `specialistStop` | A sub-agent ends | yes (specialist name) |
|
||||
|
||||
A hook's exit code decides what happens next: `0` continues, non-zero blocks the tool call (`beforeTool`) or surfaces an error (`afterTool`). Hook execution is recorded in the audit log; the audit is reachable from the agent's view of past actions, not from a dedicated `zero doctor` check.
|
||||
For swarm members the matcher subject is the **agent type** (`subagent`, `teammate`, …), not a registered specialist profile name — swarm members run from an inline manifest keyed by agent type.
|
||||
|
||||
`specialistStart` fires once the child process has actually started; a launch that fails before exec (missing binary, spawn error) records no public lifecycle hooks. `specialistStop` fires once per terminal stop: completion, error, explicit stop of a recovered task, and a background task reaped by a later `TaskOutput` all share one durable claim, so a hook that notifies or cleans up does not run twice for the same sub-agent run.
|
||||
|
||||
A hook's exit code decides what happens next: `0` continues, non-zero blocks the tool call (`beforeTool`) or surfaces an error (`afterTool`). `sessionStart` / `sessionEnd` / `specialistStart` / `specialistStop` hooks are advisory (failures are audited but do not interrupt the run). Hook execution is recorded in the audit log; the audit is reachable from the agent's view of past actions, not from a dedicated `zero doctor` check.
|
||||
|
||||
> **Roadmap.** An in-UI hooks manager is on the backlog. Today you edit the JSON directly.
|
||||
|
||||
@@ -257,6 +261,56 @@ or use a new server name. OAuth tokens are bound to the resolved server identity
|
||||
so changing a server target can require `zero mcp oauth login <server>` again.
|
||||
OAuth server names ending in `.<32 hex chars>` are reserved for token storage.
|
||||
|
||||
#### Built-in servers
|
||||
|
||||
Zero seeds two MCP servers before your config is merged. Both show up in
|
||||
`zero mcp list`, and any field can be overridden in your user config.
|
||||
|
||||
- `exa` (`https://mcp.exa.ai/mcp`) is on by default for keyless web search.
|
||||
`zero mcp disable exa` turns it off.
|
||||
- `aimlapi` (`https://mcp.aimlapi.com/mcp`, the AI/ML API Toolbox, model
|
||||
discovery, image/video/audio generation, chat, embeddings and balance tools)
|
||||
is **off by default**, because its tool runs and generations are billed to
|
||||
your AI/ML API balance. Turn it on with `zero mcp enable aimlapi`, which
|
||||
writes only `"disabled": false` to your user config, and off again with
|
||||
`zero mcp disable aimlapi`. A project config cannot turn it on.
|
||||
|
||||
When `aimlapi` connects, Zero sends your AI/ML API key as
|
||||
`Authorization: Bearer <key>`. The key is resolved again on every request and
|
||||
is never written to `config.json`. With an aimlapi.com provider profile
|
||||
(`zero providers setup aimlapi`), the key is chosen in this order:
|
||||
|
||||
1. An inline key on the profile (if set in config).
|
||||
2. Otherwise the environment variable named by the profile's `apiKeyEnv`
|
||||
(the catalog default is `AIMLAPI_API_KEY`).
|
||||
3. Otherwise the key held in the credential store when the profile uses
|
||||
`apiKeyStored`.
|
||||
|
||||
When there is no aimlapi profile, Zero uses `AIMLAPI_API_KEY` alone. If both
|
||||
`AIMLAPI_API_KEY` and a stored profile key exist, step 2 wins over the store
|
||||
whenever the inline key is empty — MCP requests may bill the environment key's
|
||||
account, not the stored key.
|
||||
|
||||
No automatic key is sent when an aimlapi profile, or any profile reading
|
||||
`AIMLAPI_API_KEY`, uses a base URL other than `https://api.aimlapi.com/v1`, or
|
||||
when `AIMLAPI_INFERENCE_URL` is overridden, because that credential may belong
|
||||
to a proxy. No key is sent when the resolved value is not a single token (for
|
||||
example a comma- or newline-separated pool), when you set your own
|
||||
non-empty `Authorization` header, when the server uses `"auth": "oauth"`, or
|
||||
when project config touched the server. Without a key the server answers 401.
|
||||
To sign in with your aimlapi.com account instead, set `"auth": "oauth"` on the
|
||||
server and run `zero mcp oauth login aimlapi`.
|
||||
|
||||
Every request to exactly `https://mcp.aimlapi.com` (https, default port,
|
||||
whatever the server is named, including one you add by hand) also carries the
|
||||
attribution headers `X-AIMLAPI-Source: agent/zero` and `X-AIMLAPI-Partner-ID`,
|
||||
and gets the key under the rules above. Other MCP servers never receive these
|
||||
headers or the key.
|
||||
|
||||
If the server echoes back the key Zero sent it, Zero replaces the key with
|
||||
`[REDACTED]` in that server's error messages and tool results before anything
|
||||
reads them, and reports an HTTP error status without the response body.
|
||||
|
||||
### As a server — expose Zero's tools to another agent
|
||||
|
||||
```bash
|
||||
@@ -301,10 +355,12 @@ Install and manage:
|
||||
zero plugins add ./github-pr-review # copy into ~/.config/zero/plugins/ or ./.zero/plugins/
|
||||
zero plugins list
|
||||
zero plugins info github-pr-review # show manifest path, enabled state, lock metadata
|
||||
zero plugins disable github-pr-review # sets "enabled": false in plugin.json
|
||||
zero plugins enable github-pr-review # sets "enabled": true in plugin.json
|
||||
zero plugins remove github-pr-review # alias: rm
|
||||
```
|
||||
|
||||
A plugin is enabled by being present in the plugins directory and disabled by removing it (or by the user setting `"enabled": false` in its `plugin.json`). Plugins are not enabled or disabled by a CLI subcommand today.
|
||||
A plugin is active when it is present under a plugins directory and its `plugin.json` has `"enabled": true` (the default when the field is omitted). Use `zero plugins disable <id>` / `zero plugins enable <id>` to toggle that field without removing the plugin. Pass `--user` to target only user plugins and ignore `./.zero/plugins`; without it, a project plugin with the same id takes precedence. The toggle edits the `enabled` value in place — the rest of the manifest keeps its formatting, field order, and numeric literals — and serializes with `plugins add` / `plugins remove` on the same plugins root. Zero reads plugins when it starts, so the change applies to the next `zero exec` run or interactive session; an interactive session that is already running keeps the plugin tools, hooks, and skills it started with until you restart it.
|
||||
|
||||
Plugin commands run with the plugin directory as their working directory. Use relative paths; the loader resolves them at activation time.
|
||||
|
||||
|
||||
@@ -634,6 +634,10 @@ Core tool groups include:
|
||||
`HTTP_PROXY`/`HTTPS_PROXY` set, the proxy is dialed instead and resolves the
|
||||
target itself, so destination enforcement ends at the proxy; see the note in
|
||||
the README.
|
||||
With `ZERO_WEBFETCH_PROVIDER=aimlapi` (opt-in; a key alone does not select
|
||||
it), the same URL checks run and the page is then fetched as Markdown by the
|
||||
billed AI/ML API Toolbox tool `search1api/crawl`, with no fallback to a
|
||||
direct fetch; see the README.
|
||||
- **Local-control tools**: config-gated wrappers for browser, terminal, desktop,
|
||||
and artifact helpers backed by packages such as `internal/localcontrol` and
|
||||
`internal/browser`.
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// A CONFIGURATION REFUSAL IS NOT A RETRIABLE FAILURE, EVEN WHEN IT ARRIVES
|
||||
// EARLY.
|
||||
//
|
||||
// capture_artifact rejects in RejectBeforePermission, which the registry returns
|
||||
// straight back before any of the gates that attach provenance. Its
|
||||
// valid-but-unavailable calls therefore reached the classifier with no denial
|
||||
// category, no permission metadata and no refusal marker, and were read as
|
||||
// ordinary retriable failures: the model got the schema hint and the call could
|
||||
// consume the profile failure-streak escalation, for a tool that never executed
|
||||
// and that no argument change can enable.
|
||||
func TestDisabledCaptureArtifactIsAPolicyRefusalNotARetriableFailure(t *testing.T) {
|
||||
// No artifacts directory and no enabled driver: valid arguments, unavailable
|
||||
// tool. This is the shape an operator produces by configuration alone.
|
||||
registry := tools.NewRegistry()
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{}) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "browser_screenshot",
|
||||
"name": "shot",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: expected the disabled tool to refuse, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if !tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("the refusal carries no provenance, so nothing downstream can tell it from a failed command: %#v", result.Meta)
|
||||
}
|
||||
|
||||
// Through the conversion the loop performs, which is where the marker has to
|
||||
// survive to be worth anything.
|
||||
converted := ToolResult{
|
||||
Status: result.Status,
|
||||
Output: result.Output,
|
||||
Meta: result.Meta,
|
||||
}
|
||||
if !isPolicyRefusal(converted) {
|
||||
t.Error("the marker did not survive into the agent-facing result")
|
||||
}
|
||||
if isRetriableToolError(converted) {
|
||||
t.Error("a tool that never executed, and that no argument change can enable, was marked retriable: the model gets a schema hint telling it to fix arguments that were already valid")
|
||||
}
|
||||
}
|
||||
|
||||
// The malformed-argument branch must stay retriable. That one IS fixable by
|
||||
// trying again differently, which is what the hint exists for, so marking every
|
||||
// early rejection would trade one wrong answer for another.
|
||||
func TestMalformedCaptureArtifactArgumentsStayRetriable(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{}) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "not_a_real_action",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: expected invalid arguments to fail, got %s", result.Status)
|
||||
}
|
||||
if tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("a malformed-argument error was marked a policy refusal, so the model is denied the hint that would let it fix the call: %q", result.Output)
|
||||
}
|
||||
converted := ToolResult{Status: result.Status, Output: result.Output, Meta: result.Meta}
|
||||
if !isRetriableToolError(converted) {
|
||||
t.Error("invalid arguments should stay retriable")
|
||||
}
|
||||
}
|
||||
|
||||
// alwaysRefusingCaptureTool stands in for the disabled tool at Run level, so the
|
||||
// loop consequence can be observed rather than inferred.
|
||||
type alwaysRefusingCaptureTool struct{ ran int }
|
||||
|
||||
func (tool *alwaysRefusingCaptureTool) Name() string { return "capture_artifact" }
|
||||
func (tool *alwaysRefusingCaptureTool) Description() string { return "test capture tool" }
|
||||
func (tool *alwaysRefusingCaptureTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"action": {Type: "string"}},
|
||||
Required: []string{"action"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *alwaysRefusingCaptureTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "captures artifacts"}
|
||||
}
|
||||
func (tool *alwaysRefusingCaptureTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{}
|
||||
}
|
||||
func (tool *alwaysRefusingCaptureTool) RejectBeforePermission(map[string]any) (tools.Result, bool) {
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
Output: "Error: capture_artifact is disabled because no artifact directory is configured.",
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalToolNotEnabled},
|
||||
}, true
|
||||
}
|
||||
|
||||
// THE LOOP CONSEQUENCE. A refusal must not draw the retry hint, because the hint
|
||||
// tells the model to fix arguments that were already valid and the tool will
|
||||
// refuse identically next time.
|
||||
func TestRunDoesNotHintARefusedCaptureArtifact(t *testing.T) {
|
||||
tool := &alwaysRefusingCaptureTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
calls := toolFailureHintAt + 1
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "capture_artifact", `{"action":"browser_screenshot"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up on the screenshot"))
|
||||
|
||||
result, err := Run(context.Background(), "take a screenshot", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Fatalf("SETUP INVALID: the tool executed %d times; it must be refused before Run", tool.ran)
|
||||
}
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, toolFailureHintMarker) {
|
||||
t.Fatal("a refused, never-executed tool drew the retry hint, which tells the model to fix arguments that were already valid")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// alternatingRefusalCaptureTool refuses every call with the same CATEGORY and a
|
||||
// different MESSAGE, which is what a real disabled driver does: the refusal
|
||||
// names the action, and the model is free to alternate valid actions.
|
||||
type alternatingRefusalCaptureTool struct{ ran int }
|
||||
|
||||
func (tool *alternatingRefusalCaptureTool) Name() string { return "capture_artifact" }
|
||||
func (tool *alternatingRefusalCaptureTool) Description() string { return "test capture tool" }
|
||||
func (tool *alternatingRefusalCaptureTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"action": {Type: "string"}},
|
||||
Required: []string{"action"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *alternatingRefusalCaptureTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "captures artifacts"}
|
||||
}
|
||||
func (tool *alternatingRefusalCaptureTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{}
|
||||
}
|
||||
func (tool *alternatingRefusalCaptureTool) RejectBeforePermission(args map[string]any) (tools.Result, bool) {
|
||||
action, _ := args["action"].(string)
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
// Action-specific wording, same category. This is the whole point: the
|
||||
// prose differs on every call while the refusal never changes.
|
||||
Output: "Error: Local control driver for " + action + " is disabled.",
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalToolNotEnabled},
|
||||
}, true
|
||||
}
|
||||
|
||||
// A REFUSAL KEYS ON ITS CATEGORY, INCLUDING WHEN THE CATEGORY IS ONLY A MARKER.
|
||||
//
|
||||
// The registry marks pre-execution refusals in metadata, and isPolicyRefusal
|
||||
// read that marker while observeToolResult keyed on DenialReason, which those
|
||||
// paths leave empty. The guard fell back to errorSignature(output), so two
|
||||
// refusals of the same category with different wording looked like two
|
||||
// different failures and the streak restarted at 1 every call.
|
||||
//
|
||||
// A model alternating capture_artifact's browser_screenshot and browser_pdf
|
||||
// against a disabled driver is refused identically each time and never tripped
|
||||
// the six-call halt. Only the generic twelve-error fallback stopped the run,
|
||||
// reporting varied errors rather than a repeated refusal.
|
||||
func TestAlternatingRefusedActionsStillTripTheRefusalHalt(t *testing.T) {
|
||||
tool := &alternatingRefusalCaptureTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// More calls than the refusal halt but FEWER than the generic any-error
|
||||
// fallback, so only the category-keyed streak can stop this.
|
||||
calls := toolFailureAnyErrorStopAt - 1
|
||||
if calls <= toolFailureStopAt {
|
||||
t.Fatalf("SETUP INVALID: %d calls cannot distinguish the refusal halt from the generic fallback", calls)
|
||||
}
|
||||
actions := []string{"browser_screenshot", "browser_pdf"}
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
action := actions[i%len(actions)]
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "capture_artifact", `{"action":"`+action+`"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up"))
|
||||
|
||||
result, err := Run(context.Background(), "capture something", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Fatalf("SETUP INVALID: the tool executed %d times; it must be refused before Run", tool.ran)
|
||||
}
|
||||
|
||||
refusals := 0
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, "is disabled") {
|
||||
refusals++
|
||||
}
|
||||
}
|
||||
if refusals > toolFailureStopAt {
|
||||
t.Errorf("the run made %d refused calls; the six-call refusal halt never tripped because the streak re-keyed on each action's wording", refusals)
|
||||
}
|
||||
|
||||
// And the halt has to read as a repeated refusal, not as varied errors.
|
||||
stop := strings.ToLower(strings.Join(messageContents(result.Messages), "\n"))
|
||||
if strings.Contains(stop, toolFailureHintMarker) {
|
||||
t.Error("a refused, never-executed tool drew the retry hint")
|
||||
}
|
||||
}
|
||||
|
||||
func messageContents(messages []zeroruntime.Message) []string {
|
||||
out := make([]string, 0, len(messages))
|
||||
for _, message := range messages {
|
||||
out = append(out, message.Content)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/localcontrol"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// THE DISABLED-DRIVER BRANCH, REACHED FOR REAL.
|
||||
//
|
||||
// RejectBeforePermission refuses on two separate conditions, and its neighbour
|
||||
// test constructs empty options, so it returns at the FIRST one (no artifact
|
||||
// directory configured) and the second is never evaluated. Reverting the
|
||||
// disabled-driver branch in local_capture.go to a plain errorResult therefore
|
||||
// left the whole agent and tools suites green.
|
||||
//
|
||||
// This is the shape an operator actually produces: an artifact root IS
|
||||
// configured and one driver IS enabled, and the model asks for an action owned
|
||||
// by a different, disabled driver. Without provenance that call is read as a
|
||||
// retriable failure, so it collects a schema hint and can spend the profile's
|
||||
// failure-streak escalation, for a tool no argument change can enable.
|
||||
func TestCaptureArtifactDisabledDriverIsAPolicyRefusal(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
options := tools.LocalControlArtifactOptions{
|
||||
ArtifactsDir: t.TempDir(),
|
||||
// Configured and enabled, so the tool as a whole is available and the
|
||||
// missing-directory branch cannot fire.
|
||||
Browser: localcontrol.BrowserOptions{Enabled: true, Driver: "test-browser", HelperPath: "browser-helper"},
|
||||
// Terminal deliberately left disabled.
|
||||
}
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(options) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
// An enabled driver's action must still be accepted, or the test below would
|
||||
// pass for a tool that refuses everything.
|
||||
if result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "browser_screenshot", "name": "shot",
|
||||
}, tools.RunOptions{PermissionGranted: true}); tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("SETUP INVALID: the enabled browser driver was refused as policy: %s", result.Output)
|
||||
}
|
||||
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "terminal_snapshot", "name": "snap", "session": "test-session",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: the disabled terminal driver did not refuse: %s / %s", result.Status, result.Output)
|
||||
}
|
||||
if !tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("a driver disabled by configuration refuses without provenance, so it reads as a retriable failure: %#v", result.Meta)
|
||||
}
|
||||
if got := result.Meta["policy_refusal"]; got != tools.PolicyRefusalToolNotEnabled {
|
||||
t.Errorf("policy_refusal = %q, want %q", got, tools.PolicyRefusalToolNotEnabled)
|
||||
}
|
||||
if isRetriableToolError(ToolResult{Status: result.Status, Output: result.ModelOutput(), Meta: result.Meta}) {
|
||||
t.Error("the refusal is still classified as retriable, so the model gets a schema hint for a configuration decision")
|
||||
}
|
||||
}
|
||||
|
||||
// AND THE EARLY REJECTIONS MUST NOT COLLAPSE INTO ONE ANSWER.
|
||||
//
|
||||
// RejectBeforePermission refuses on configuration; argument validation refuses
|
||||
// on a fixable mistake, and it runs FIRST, which is how the disabled-driver
|
||||
// branch stayed uncovered. A malformed call must stay an ordinary retriable
|
||||
// error, because trying again differently is exactly the right response to it.
|
||||
func TestCaptureArtifactMalformedArgumentsStayRetriable(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{
|
||||
ArtifactsDir: t.TempDir(),
|
||||
Terminal: localcontrol.TerminalOptions{Enabled: true, Driver: "test-terminal", HelperPath: "terminal-helper"},
|
||||
}) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
// The driver IS enabled, so nothing here is a configuration decision. The
|
||||
// call is simply missing the session the action requires.
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "terminal_snapshot", "name": "snap",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: the malformed call did not fail: %s", result.Output)
|
||||
}
|
||||
if tools.IsPolicyRefusalResult(result) {
|
||||
t.Errorf("a fixable argument mistake was marked a policy refusal, so the model loses the schema hint that would fix it: %s", result.Output)
|
||||
}
|
||||
if !isRetriableToolError(ToolResult{Status: result.Status, Output: result.ModelOutput(), Meta: result.Meta}) {
|
||||
t.Errorf("a fixable argument mistake is not retriable: %s", result.Output)
|
||||
}
|
||||
}
|
||||
+192
-13
@@ -55,6 +55,22 @@ const (
|
||||
// error, so this only affects true same-error loops.
|
||||
toolFailureStopAt = 6
|
||||
|
||||
// toolFailureAnyErrorStopAt halts a tool that keeps failing with DIFFERENT
|
||||
// errors. The streak above cannot see that case by construction: a changed
|
||||
// signature rebuilds the record at 1, so a tool whose error text varies every
|
||||
// call never reaches toolFailureStopAt however long it loops.
|
||||
//
|
||||
// That is not hypothetical. A headless run made 384 denied calls over 26
|
||||
// minutes without tripping a halt set to 6, because each denial named a
|
||||
// different path. Keying denials on their category (see observeToolResult)
|
||||
// fixes that case; this bound covers every error with no category to key on.
|
||||
//
|
||||
// Deliberately well above toolFailureStopAt. A model iterating on a genuinely
|
||||
// tricky edit legitimately fails several times with different errors while it
|
||||
// converges, and this must not cut those runs short — the same reasoning that
|
||||
// moved toolFailureStopAt from 4 to 6. Both counters reset on success.
|
||||
toolFailureAnyErrorStopAt = 12
|
||||
|
||||
// maxContinueNudges bounds how many times the headless completion gate
|
||||
// (Options.RequireCompletionSignal) re-prompts a model that stopped without a
|
||||
// tool call while work clearly remained. Once spent, the run finalizes as
|
||||
@@ -324,15 +340,91 @@ func acceptanceVerificationNudge(objective string) string {
|
||||
const toolFailureHintMarker = "kept failing with the same error"
|
||||
|
||||
type toolFailureRecord struct {
|
||||
count int
|
||||
errSig string
|
||||
hintShown bool
|
||||
count int
|
||||
identity failureIdentity
|
||||
// sawExecuted and sawRefused are the AGGREGATE over the whole streak, not
|
||||
// the current identity. The content-blind bound needs them because it fires
|
||||
// exactly when no identity repeated, so the last one is unrepresentative.
|
||||
sawExecuted bool
|
||||
sawRefused bool
|
||||
hintShown bool
|
||||
// anyErrorCount counts consecutive failures of this tool REGARDLESS of the
|
||||
// error, and is cleared only by a success. count above restarts whenever the
|
||||
// signature changes, which is exactly what a varying error message defeats;
|
||||
// this one cannot be reset by changing the text.
|
||||
anyErrorCount int
|
||||
}
|
||||
|
||||
// failureKind separates a failure the TOOL produced from one POLICY produced.
|
||||
//
|
||||
// The two used to share one string namespace, with provenance recovered
|
||||
// afterwards by testing that string for a "denial:" prefix. Trusted provenance
|
||||
// and untrusted content in one namespace is a namespace the untrusted side can
|
||||
// write into: a command that printed exactly "denial:permission_denied" and
|
||||
// exited non-zero acquired the identity of a real permission refusal, merged
|
||||
// its streak with one, and had the run report it as refused although it ran
|
||||
// every time. The kind is carried as its own field so no output can spell it.
|
||||
type failureKind uint8
|
||||
|
||||
const (
|
||||
// The zero value: no failure recorded.
|
||||
_ failureKind = iota
|
||||
// failureKindExecuted: the tool ran and returned an error. The key is the
|
||||
// normalized error signature, which is tool-controlled text.
|
||||
failureKindExecuted
|
||||
// failureKindRefused: policy refused the call before the tool ran. The key is
|
||||
// the denial category, a small closed enum the loop sets.
|
||||
failureKindRefused
|
||||
)
|
||||
|
||||
// failureIdentity is what the same-identity streak counts. Comparing the pair
|
||||
// means an executed error can never equal a refusal however it is spelled.
|
||||
type failureIdentity struct {
|
||||
kind failureKind
|
||||
key string
|
||||
}
|
||||
|
||||
// toolFailureCause is why the guard stopped, decided where the facts are rather
|
||||
// than reconstructed by the caller from overlapping booleans.
|
||||
type toolFailureCause uint8
|
||||
|
||||
const (
|
||||
// The zero value: the guard did not stop.
|
||||
_ toolFailureCause = iota
|
||||
// toolFailureCauseSameError: the same executed error signature, six times.
|
||||
toolFailureCauseSameError
|
||||
// toolFailureCauseSameRefusal: the same denial category, six times. The prose
|
||||
// differs on every call because it names what was refused, so this is NOT the
|
||||
// same-error case.
|
||||
toolFailureCauseSameRefusal
|
||||
// toolFailureCauseVariedExecuted: the content-blind bound, and every failure
|
||||
// was the tool executing and failing.
|
||||
toolFailureCauseVariedExecuted
|
||||
// toolFailureCauseVariedRefused: the content-blind bound, and every failure
|
||||
// was a policy refusal. Alternating two categories reaches twelve without
|
||||
// either reaching six, and the tool never ran once.
|
||||
toolFailureCauseVariedRefused
|
||||
// toolFailureCauseVariedMixed: the content-blind bound over both kinds.
|
||||
// Named deliberately rather than left to whichever field a switch tested
|
||||
// first.
|
||||
toolFailureCauseVariedMixed
|
||||
)
|
||||
|
||||
type toolFailureOutcome struct {
|
||||
InjectHint bool
|
||||
Stop bool
|
||||
Count int
|
||||
// Cause is set only when Stop is. It carries the aggregate fact about the
|
||||
// sequence that tripped a bound, which the last record alone cannot supply:
|
||||
// the content-blind bound is reached precisely when no single identity
|
||||
// repeated enough, so the identity present at the end says nothing about the
|
||||
// eleven before it.
|
||||
Cause toolFailureCause
|
||||
}
|
||||
|
||||
// Refused reports whether policy, rather than the tool, produced the failures.
|
||||
func (cause toolFailureCause) Refused() bool {
|
||||
return cause == toolFailureCauseSameRefusal || cause == toolFailureCauseVariedRefused
|
||||
}
|
||||
|
||||
// errorSignature normalizes a tool error to a short, comparable signature so
|
||||
@@ -358,12 +450,47 @@ func toolFailureHint(toolName, schemaJSON, errOutput string) string {
|
||||
|
||||
// toolFailureStopAnswer is the final answer when the repeated-failure guard halts
|
||||
// a run.
|
||||
func toolFailureStopAnswer(toolName string, count int) string {
|
||||
return "Agent stopped: the `" + toolName + "` tool failed " + strconv.Itoa(count) +
|
||||
" times in a row with the same error, so I halted instead of looping further. " +
|
||||
func toolFailureStopAnswer(toolName string, count int, cause toolFailureCause) string {
|
||||
// Each branch claims only what its counter established, and the mixed case is
|
||||
// spelled out rather than decided by whichever field a switch happened to
|
||||
// test first. The wording used to overclaim in both directions: the
|
||||
// content-blind bound said every failure differed, which it does not track,
|
||||
// and it described a run of policy refusals as the tool failing, when the
|
||||
// tool had not run at all.
|
||||
verb, tail := " tool failed ", " times in a row with the same error, "
|
||||
switch cause {
|
||||
case toolFailureCauseSameRefusal:
|
||||
verb, tail = " tool was refused ", " times in a row, "
|
||||
case toolFailureCauseVariedExecuted:
|
||||
tail = " times in a row with varying errors, "
|
||||
case toolFailureCauseVariedRefused:
|
||||
verb, tail = " tool was refused ", " times in a row for different reasons, "
|
||||
case toolFailureCauseVariedMixed:
|
||||
verb, tail = " tool failed or was refused ", " times in a row, "
|
||||
}
|
||||
return "Agent stopped: the `" + toolName + "`" + verb + strconv.Itoa(count) +
|
||||
tail + "so I halted instead of looping further. " +
|
||||
"Please check the request or adjust the tool arguments."
|
||||
}
|
||||
|
||||
// toolFailureIncompleteReason describes a guard halt for the headless
|
||||
// completion gate. A guard halt is BY DEFINITION unfinished work: the run
|
||||
// stopped because a tool would not stop failing, not because the task was done.
|
||||
//
|
||||
// It makes the same claim as toolFailureStopAnswer for the same halt, including
|
||||
// the mixed case: a headless caller reads this reason and an interactive one
|
||||
// reads the stop answer, and the two must not describe one halt differently.
|
||||
// Reported by @jatmn.
|
||||
func toolFailureIncompleteReason(toolName string, cause toolFailureCause) string {
|
||||
switch {
|
||||
case cause.Refused():
|
||||
return "halted after `" + toolName + "` was refused repeatedly, without completing the task"
|
||||
case cause == toolFailureCauseVariedMixed:
|
||||
return "halted after `" + toolName + "` failed or was refused repeatedly, without completing the task"
|
||||
}
|
||||
return "halted after `" + toolName + "` failed repeatedly, without completing the task"
|
||||
}
|
||||
|
||||
// The no-output stop answer is assembled from these fixed parts (only the turn
|
||||
// count varies). IsNoProgressStop matches all three so a legitimate message that
|
||||
// merely quotes the marker substring is not misclassified as a failed empty run.
|
||||
@@ -472,28 +599,68 @@ func newGuardState() *guardState {
|
||||
// observeToolResult tracks repeated identical failures of a tool. A successful
|
||||
// result clears that tool's failure streak. Returns whether to inject a one-shot
|
||||
// corrective hint and/or stop the run.
|
||||
func (state *guardState) observeToolResult(name string, failed bool, output string) toolFailureOutcome {
|
||||
// hintable is separate from failed on purpose. A categorized denial MUST count
|
||||
// toward the streaks — that is the whole point of keying on the category — but a
|
||||
// schema hint is the wrong response to a policy refusal: the call shape is fine,
|
||||
// the answer was no. Collapsing the two is what made the earlier version of this
|
||||
// fix a no-op, since the caller could only pass a flag that excluded denials.
|
||||
func (state *guardState) observeToolResult(name string, failed bool, hintable bool, output string, denial DenialCategory) toolFailureOutcome {
|
||||
if state.toolFailures == nil {
|
||||
state.toolFailures = map[string]*toolFailureRecord{}
|
||||
}
|
||||
if !failed {
|
||||
delete(state.toolFailures, name) // success resets the streak
|
||||
delete(state.toolFailures, name) // success resets both counters
|
||||
return toolFailureOutcome{}
|
||||
}
|
||||
sig := errorSignature(output)
|
||||
// A denial keys on its CATEGORY, not its prose. The message embeds the path
|
||||
// or command that was refused, so it differs on every call while describing
|
||||
// the same unchanging refusal — which rebuilt the record at 1 each time and
|
||||
// let a denied tool loop indefinitely under a halt set to 6. The category is
|
||||
// a small closed enum the loop already sets on the result.
|
||||
identity := failureIdentity{kind: failureKindExecuted, key: errorSignature(output)}
|
||||
if denial != DenialNone {
|
||||
identity = failureIdentity{kind: failureKindRefused, key: string(denial)}
|
||||
}
|
||||
record := state.toolFailures[name]
|
||||
if record == nil || record.errSig != sig {
|
||||
record = &toolFailureRecord{count: 1, errSig: sig}
|
||||
if record == nil {
|
||||
record = &toolFailureRecord{identity: identity}
|
||||
state.toolFailures[name] = record
|
||||
}
|
||||
if record.identity != identity {
|
||||
// A different failure restarts the same-identity streak but NOT the
|
||||
// content-blind one: changing how a tool fails is not progress.
|
||||
record.count = 1
|
||||
record.identity = identity
|
||||
record.hintShown = false
|
||||
} else {
|
||||
record.count++
|
||||
}
|
||||
record.anyErrorCount++
|
||||
if identity.kind == failureKindRefused {
|
||||
record.sawRefused = true
|
||||
} else {
|
||||
record.sawExecuted = true
|
||||
}
|
||||
|
||||
outcome := toolFailureOutcome{Count: record.count}
|
||||
if record.count >= toolFailureStopAt {
|
||||
switch {
|
||||
case record.count >= toolFailureStopAt:
|
||||
outcome.Stop = true
|
||||
outcome.Cause = toolFailureCauseSameError
|
||||
if identity.kind == failureKindRefused {
|
||||
outcome.Cause = toolFailureCauseSameRefusal
|
||||
}
|
||||
return outcome
|
||||
case record.anyErrorCount >= toolFailureAnyErrorStopAt:
|
||||
// Report the counter that actually tripped. record.count is the
|
||||
// same-identity streak and is often 1 here, which would describe a tool
|
||||
// that failed a dozen different ways as having failed once.
|
||||
outcome.Stop = true
|
||||
outcome.Count = record.anyErrorCount
|
||||
outcome.Cause = record.variedCause()
|
||||
return outcome
|
||||
}
|
||||
if record.count >= toolFailureHintAt && !record.hintShown {
|
||||
if hintable && record.count >= toolFailureHintAt && !record.hintShown {
|
||||
record.hintShown = true
|
||||
outcome.InjectHint = true
|
||||
}
|
||||
@@ -603,3 +770,15 @@ func (state *guardState) planReminder(turn int) string {
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
// variedCause classifies a content-blind stop by what the whole streak held.
|
||||
func (record *toolFailureRecord) variedCause() toolFailureCause {
|
||||
switch {
|
||||
case record.sawRefused && record.sawExecuted:
|
||||
return toolFailureCauseVariedMixed
|
||||
case record.sawRefused:
|
||||
return toolFailureCauseVariedRefused
|
||||
default:
|
||||
return toolFailureCauseVariedExecuted
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,8 @@ package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -192,7 +194,7 @@ func TestUnknownExecSessionProbingTripsFailureHalt(t *testing.T) {
|
||||
var state guardState
|
||||
var stoppedAt int
|
||||
for i := 1; i <= toolFailureStopAt; i++ {
|
||||
out := state.observeToolResult(tools.WriteStdinToolName, true, tools.UnknownExecSessionError(i))
|
||||
out := state.observeToolResult(tools.WriteStdinToolName, true, true, tools.UnknownExecSessionError(i), "")
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
@@ -203,6 +205,223 @@ func TestUnknownExecSessionProbingTripsFailureHalt(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A permission denial repeats forever when the streak is keyed on the error
|
||||
// TEXT, because the denial message embeds the path or command that varies per
|
||||
// call.
|
||||
//
|
||||
// Observed, not theorised: a headless run made 384 denied calls over 26 minutes
|
||||
// without tripping a halt that stops at 6. Every denial carried the same typed
|
||||
// category and a different reason string, so errorSignature differed each time
|
||||
// and the record was rebuilt at count 1 on every call.
|
||||
//
|
||||
// TestUnknownExecSessionErrorSignatureIsIDInvariant fixed one message this way.
|
||||
// Keying on the category fixes the class, without needing every future denial
|
||||
// message to remember to be invariant.
|
||||
func TestPermissionDenialStreakSurvivesVaryingReasonText(t *testing.T) {
|
||||
var state guardState
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureStopAt; i++ {
|
||||
// The shape the loop actually produces: same tool, same category, a
|
||||
// different path every time.
|
||||
output := "Error: Permission denied for write_file: cannot write " +
|
||||
filepath.Join("C:", "ws", "pkg", "file"+strconv.Itoa(i)+".go")
|
||||
out := state.observeToolResult("write_file", true, true, output, DenialPermissionDenied)
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureStopAt {
|
||||
t.Fatalf("denials with varying reason text stopped at %d, want %d", stoppedAt, toolFailureStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// The content-blind bound. A tool failing over and over with genuinely
|
||||
// DIFFERENT errors and no denial category is still a tool that is not working,
|
||||
// and the same-signature streak can never see it.
|
||||
//
|
||||
// Bounded well above toolFailureStopAt on purpose: a model iterating on a
|
||||
// tricky edit legitimately fails a few times with different errors while it
|
||||
// converges, and cutting that short is the regression
|
||||
// TestSuccessResetsBothFailureCounters guards.
|
||||
func TestToolFailingWithDifferentErrorsEveryTimeStillStops(t *testing.T) {
|
||||
var state guardState
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
|
||||
out := state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), "")
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureAnyErrorStopAt {
|
||||
t.Fatalf("a tool failing with a new error each call stopped at %d, want %d", stoppedAt, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// Neither counter may outlive a success, or a long run that fails occasionally
|
||||
// and recovers would eventually halt for no reason. This is the property that
|
||||
// keeps the content-blind bound safe to add.
|
||||
func TestSuccessResetsBothFailureCounters(t *testing.T) {
|
||||
var state guardState
|
||||
for i := 1; i < toolFailureAnyErrorStopAt; i++ {
|
||||
if out := state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), ""); out.Stop {
|
||||
t.Fatalf("stopped at %d before the success that should reset it", i)
|
||||
}
|
||||
}
|
||||
state.observeToolResult("bash", false, false, "ok", "")
|
||||
|
||||
// Same again from zero. Reaching the bound a second time proves the counter
|
||||
// restarted rather than merely paused.
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
|
||||
if out := state.observeToolResult("bash", true, true, "later failure "+strconv.Itoa(i), ""); out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureAnyErrorStopAt {
|
||||
t.Fatalf("after a success the tool stopped at %d, want a full fresh %d", stoppedAt, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// Records are keyed per tool, so ANOTHER tool succeeding in between must not
|
||||
// clear the failing tool's streak.
|
||||
//
|
||||
// This is the realistic shape of the run that motivated the fix: the model kept
|
||||
// making progress elsewhere — reading files, updating its plan — while one tool
|
||||
// was refused over and over. A reset keyed on "something succeeded" rather than
|
||||
// "this tool succeeded" would make the halt unreachable in exactly the runs that
|
||||
// need it.
|
||||
func TestAnotherToolSucceedingDoesNotClearAFailingToolsStreak(t *testing.T) {
|
||||
var state guardState
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureStopAt; i++ {
|
||||
state.observeToolResult("read_file", false, false, "ok", "")
|
||||
out := state.observeToolResult("write_file", true, false,
|
||||
"Error: Permission denied for write_file: "+strconv.Itoa(i), DenialPermissionDenied)
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureStopAt {
|
||||
t.Fatalf("denials interleaved with another tool's successes stopped at %d, want %d", stoppedAt, toolFailureStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// alwaysPromptingTool is never allowed to run: it exists so a Run-level test can
|
||||
// drive real permission denials through the loop.
|
||||
type alwaysPromptingTool struct{ ran int }
|
||||
|
||||
func (tool *alwaysPromptingTool) Name() string { return "bash" }
|
||||
func (tool *alwaysPromptingTool) Description() string { return "test shell tool" }
|
||||
func (tool *alwaysPromptingTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *alwaysPromptingTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs shell commands"}
|
||||
}
|
||||
func (tool *alwaysPromptingTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusOK, Output: "should never run"}
|
||||
}
|
||||
|
||||
// The regression for the gap this whole change existed to close, driven through
|
||||
// Run rather than through the guard helper.
|
||||
//
|
||||
// The first version of this fix was a no-op in production and every one of its
|
||||
// unit tests passed, because they called observeToolResult directly with
|
||||
// failed=true. The loop passes isRetriableToolError, which returns false for a
|
||||
// categorized denial, so the guard took its success branch and deleted the
|
||||
// record before it could key on the category. A denied tool still looped to the
|
||||
// turn limit.
|
||||
//
|
||||
// This drives the real path: a tool that always prompts, an approver that always
|
||||
// denies, and a different command each turn so the denial REASON — and therefore
|
||||
// the error text — varies exactly as it does in a real run.
|
||||
func TestRunStopsARepeatedlyDeniedToolAtTheFailureBound(t *testing.T) {
|
||||
tool := &alwaysPromptingTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// Comfortably more turns than the bound, so reaching the bound is what stops
|
||||
// the run rather than exhausting the provider or MaxTurns.
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
|
||||
for i := range toolFailureStopAt + 4 {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
|
||||
`{"command":"touch /etc/file`+strconv.Itoa(i)+`"}`))
|
||||
}
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
denials := 0
|
||||
result, err := Run(context.Background(), "do the thing", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnPermissionRequest: func(_ context.Context, request PermissionRequest) (PermissionDecision, error) {
|
||||
denials++
|
||||
// The varying half. In production this is the refused path or command;
|
||||
// here it is the command, which lands in the denial message the guard
|
||||
// used to key on.
|
||||
return PermissionDecision{
|
||||
Action: PermissionDecisionDeny,
|
||||
Reason: "refused " + request.ToolName + " call " + strconv.Itoa(denials),
|
||||
}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if denials != toolFailureStopAt {
|
||||
t.Errorf("the run made %d denied calls, want it halted at %d", denials, toolFailureStopAt)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Errorf("the denied tool executed %d times; the denial must precede execution", tool.ran)
|
||||
}
|
||||
want := toolFailureStopAnswer("bash", toolFailureStopAt, toolFailureCauseSameRefusal)
|
||||
if result.FinalAnswer != want {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The stop message must describe the bound that actually tripped. When the
|
||||
// content-blind counter is what halts the run, the same-signature streak is
|
||||
// usually 1, and reporting that would tell the user a tool failed once after it
|
||||
// failed a dozen different ways.
|
||||
func TestVariedFailureStopAnswerReportsTheRightCounter(t *testing.T) {
|
||||
var state guardState
|
||||
var outcome toolFailureOutcome
|
||||
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
|
||||
outcome = state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), "")
|
||||
if outcome.Stop {
|
||||
break
|
||||
}
|
||||
}
|
||||
if !outcome.Stop {
|
||||
t.Fatal("never stopped")
|
||||
}
|
||||
if outcome.Cause != toolFailureCauseVariedExecuted {
|
||||
t.Errorf("Cause = %v for a stop driven by the content-blind counter over executed failures", outcome.Cause)
|
||||
}
|
||||
if outcome.Count != toolFailureAnyErrorStopAt {
|
||||
t.Errorf("Count = %d, want the counter that tripped (%d)", outcome.Count, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
answer := toolFailureStopAnswer("bash", outcome.Count, outcome.Cause)
|
||||
if !strings.Contains(answer, "varying errors") {
|
||||
t.Errorf("stop answer describes the wrong cause: %q", answer)
|
||||
}
|
||||
if strings.Contains(answer, "with the same error") {
|
||||
t.Errorf("stop answer claims a same-error loop after distinct failures: %q", answer)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGuardStateResetsToolOnlyStreakOnEmptyNonToolTurn(t *testing.T) {
|
||||
var state guardState
|
||||
toolOnly := zeroruntime.CollectedStream{
|
||||
|
||||
+214
-27
@@ -413,7 +413,8 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// collected and a non-nil stop error when the run must end now.
|
||||
recoverStreamError := func(collected zeroruntime.CollectedStream) (zeroruntime.CollectedStream, error) {
|
||||
if isImageRejectionError(errors.New(collected.Error)) {
|
||||
return collected, fmt.Errorf("model %s rejected the image: %s. The model may not support image input — try switching to a vision-capable model (claude, gpt-4o, gemini)", options.Model, collected.Error)
|
||||
cause := &zeroruntime.StreamError{Message: collected.Error, StatusCode: collected.ErrorStatusCode, Cause: collected.ErrorCause}
|
||||
return collected, fmt.Errorf("model %s rejected the image: %w. The model may not support image input — try switching to a vision-capable model (claude, gpt-4o, gemini)", options.Model, cause)
|
||||
}
|
||||
// REACTIVE compaction: the streamed error may also be a context limit
|
||||
// (some providers surface it mid-stream). Compact and retry once.
|
||||
@@ -542,7 +543,7 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
}
|
||||
if collected.Error != "" {
|
||||
result.Messages = copyMessages(messages)
|
||||
return result, errors.New(collected.Error)
|
||||
return result, &zeroruntime.StreamError{Message: collected.Error, StatusCode: collected.ErrorStatusCode, Cause: collected.ErrorCause}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -719,6 +720,60 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// executed concurrently, consumed strictly in order below.
|
||||
var precomputed []precomputedToolResult
|
||||
precomputedStart, precomputedEnd := 0, 0
|
||||
// closeOutRemaining terminates every advertised call from `from` onward
|
||||
// TRUTHFULLY, and every early return in this loop goes through it.
|
||||
//
|
||||
// Read-ahead broke the assumption the abort placeholders were written
|
||||
// under. executeParallelReadBatch runs an entire eligible run before the
|
||||
// loop consumes any of it, so "not consumed yet" stopped meaning "not
|
||||
// executed": a sibling can already have run, committed its side effects
|
||||
// and any authorization credit they carry, and still sit past the index
|
||||
// where a terminal branch fires. Recording it as aborted threw that result
|
||||
// away, so the transcript disagreed with what had actually happened, and
|
||||
// its callbacks, trace counter, task observation and images went with it.
|
||||
//
|
||||
// So each remaining call is put in the one state that is true of it:
|
||||
// completed, and finalized exactly once with the same bookkeeping the main
|
||||
// path performs, or unstarted, and aborted. The guard is deliberately NOT
|
||||
// consulted for a drained sibling: these results cannot reverse a stop
|
||||
// decision that has already been made, they are only owed an honest record.
|
||||
closeOutRemaining := func(from int) {
|
||||
for next := from; next < len(collected.ToolCalls); next++ {
|
||||
sibling, ran := precomputedResultFor(precomputed, precomputedStart, precomputedEnd, next)
|
||||
if !ran {
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[next:next+1])
|
||||
continue
|
||||
}
|
||||
nextCall := collected.ToolCalls[next]
|
||||
if options.OnToolCall != nil {
|
||||
options.OnToolCall(nextCall)
|
||||
}
|
||||
options.Trace.Counter(trace.CounterToolCalls, 1)
|
||||
recordOutputBudgetTrace(options.Trace, sibling)
|
||||
task.observe(taskStateEvent{kind: taskStateEventToolResult, arguments: nextCall.Arguments, toolResult: sibling})
|
||||
if options.OnToolResult != nil {
|
||||
options.OnToolResult(sibling)
|
||||
}
|
||||
for _, name := range sibling.LoadedTools {
|
||||
loaded[name] = true
|
||||
}
|
||||
// The same fields the consumption path below records, provider call
|
||||
// ID included: a strict replay pairs the result to its tool_use by
|
||||
// that ID, and a finalized sibling without it was the one result in
|
||||
// the turn that could not be paired. Reported by @jatmn.
|
||||
messages = append(messages, zeroruntime.Message{
|
||||
Role: zeroruntime.MessageRoleTool,
|
||||
Content: sibling.ModelOutput(),
|
||||
ToolCallID: sibling.ToolCallID,
|
||||
ToolCallProviderID: nextCall.ProviderCallID,
|
||||
IsError: sibling.Status == tools.StatusError,
|
||||
ChangedFiles: append([]string(nil), sibling.ChangedFiles...),
|
||||
})
|
||||
if len(sibling.Images) > 0 {
|
||||
toolResultsWithImages = append(toolResultsWithImages, sibling)
|
||||
}
|
||||
}
|
||||
}
|
||||
for index, call := range collected.ToolCalls {
|
||||
// When this call starts a consecutive run of >= 2 auto-allowed read-only
|
||||
// calls, execute the whole run concurrently now (see parallel_tools.go).
|
||||
@@ -783,13 +838,13 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
abortErr = ctx.Err()
|
||||
}
|
||||
if abortErr != nil {
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
|
||||
closeOutRemaining(index + 1)
|
||||
messages = append(messages, buildToolImageMessages()...)
|
||||
result.Messages = copyMessages(messages)
|
||||
return result, abortErr
|
||||
}
|
||||
if stopReason := stopReasonFromToolResult(toolResult); stopReason != "" {
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
|
||||
closeOutRemaining(index + 1)
|
||||
messages = append(messages, buildToolImageMessages()...)
|
||||
result.FinalAnswer = toolResult.ModelOutput()
|
||||
result.StopReason = stopReason
|
||||
@@ -804,8 +859,29 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// aren't fixed by reformatting the call, so a "match this schema" hint
|
||||
// would misdirect the model toward JSON shape or blocked behavior.
|
||||
retriableFailure := isRetriableToolError(toolResult)
|
||||
outcome := guards.observeToolResult(call.Name, retriableFailure, toolResult.ModelOutput())
|
||||
posture.observeToolOutcome(outcome, toolResult)
|
||||
// A categorized denial is NOT retriable — retrying it verbatim is
|
||||
// pointless — but it is still a failure the streaks must count, or a
|
||||
// refused tool loops until the turn limit. Passing retriableFailure for
|
||||
// both is what let that happen: observeToolResult took its success
|
||||
// branch and deleted the record before it could key on the category.
|
||||
// isPolicyRefusal rather than DenialReason alone: the categories are
|
||||
// only attached where a typed denial is built, so a headless prompt
|
||||
// refusal and an uncategorized sandbox block were counted as SUCCESS
|
||||
// and cleared the record they were supposed to accumulate.
|
||||
policyRefusal := isPolicyRefusal(toolResult)
|
||||
countedFailure := retriableFailure || policyRefusal
|
||||
outcome := guards.observeToolResult(call.Name, countedFailure, retriableFailure, toolResult.ModelOutput(), toolResult.DenialReason)
|
||||
// The profile's failure-streak trigger is for RETRIABLE failures: it
|
||||
// restores the displaced turn budget and reasoning effort on the theory
|
||||
// that the tool is struggling and needs room. A policy refusal is not a
|
||||
// struggling tool, it is an answer, and spending the one-shot
|
||||
// escalation on one contradicts that trigger's own contract. Denials
|
||||
// still count for the halt above; they just do not buy more budget.
|
||||
if policyRefusal {
|
||||
posture.observeToolOutcome(toolFailureOutcome{}, toolResult)
|
||||
} else {
|
||||
posture.observeToolOutcome(outcome, toolResult)
|
||||
}
|
||||
if outcome.Stop {
|
||||
// The assistant message advertised EVERY collected tool call, but
|
||||
// the guard halts mid-turn so the calls after this one never run.
|
||||
@@ -813,17 +889,22 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// every tool_use has a matching tool_result and the recorded
|
||||
// messages stay valid for a strict provider replay (Anthropic
|
||||
// rejects a tool_use with no answering tool_result).
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
|
||||
closeOutRemaining(index + 1)
|
||||
messages = append(messages, buildToolImageMessages()...)
|
||||
result.FinalAnswer = toolFailureStopAnswer(call.Name, outcome.Count)
|
||||
// Same headless contract as the empty-turn guard: halting on a
|
||||
// repeated failure is an unfinished run, not success. Gate-off
|
||||
// (interactive / --no-completion-gate) keeps the old behavior.
|
||||
result.FinalAnswer = toolFailureStopAnswer(call.Name, outcome.Count, outcome.Cause)
|
||||
result.Messages = copyMessages(messages)
|
||||
// THE HALT HAS TO CROSS THE TERMINAL-STATUS BOUNDARY TOO.
|
||||
//
|
||||
// This branch returns straight out, so it never reached the
|
||||
// completion gate that the max-turns paths below go through. Under
|
||||
// RequireCompletionSignal `zero exec` treats only Incomplete as exit
|
||||
// 4 and otherwise emits run_end("success", 0), so a task that was
|
||||
// denied six times, or failed through the twelve-call bound, became
|
||||
// a successful automation result having done none of the work.
|
||||
if options.RequireCompletionSignal {
|
||||
result.Incomplete = true
|
||||
result.IncompleteReason = "the " + call.Name + " tool failed " + strconv.Itoa(outcome.Count) + " times in a row with the same error"
|
||||
result.IncompleteReason = toolFailureIncompleteReason(call.Name, outcome.Cause)
|
||||
}
|
||||
result.Messages = copyMessages(messages)
|
||||
return result, nil
|
||||
}
|
||||
if outcome.InjectHint && failureHint == "" {
|
||||
@@ -1605,9 +1686,41 @@ func executeToolCall(ctx context.Context, registry *tools.Registry, call ToolCal
|
||||
// the Run turn loop performs the actual provider switch. Empty for every
|
||||
// ordinary tool result.
|
||||
RequestedModel: result.Meta["escalate_to_model"],
|
||||
// ONE REFUSAL IDENTITY, derived here so classification and streak
|
||||
// accounting cannot disagree.
|
||||
//
|
||||
// The registry marks its pre-execution refusals in metadata, and
|
||||
// isPolicyRefusal read that marker while observeToolResult keyed on
|
||||
// DenialReason, which those paths leave empty. The guard therefore fell
|
||||
// back to errorSignature(output), and two refusals of the same category
|
||||
// with different wording looked like two different failures. A model
|
||||
// alternating capture_artifact's browser_screenshot and browser_pdf against
|
||||
// a disabled driver is refused identically each time, and never tripped the
|
||||
// six-call refusal halt: only the generic twelve-error fallback stopped it,
|
||||
// reporting varied errors rather than a repeated refusal.
|
||||
DenialReason: denialCategoryForResult(result),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// denialCategoryForResult gives every pre-execution refusal a stable category,
|
||||
// whether it was built as a typed denial or only marked in metadata.
|
||||
//
|
||||
// The mapping lives here, at the one boundary tools.Result becomes ToolResult,
|
||||
// rather than at each producer. A marker without a category is the shape that
|
||||
// caused this: it classified as a refusal and keyed as an error signature.
|
||||
func denialCategoryForResult(result tools.Result) DenialCategory {
|
||||
switch result.Meta[tools.PolicyRefusalMeta] {
|
||||
case tools.PolicyRefusalToolNotEnabled:
|
||||
return DenialFiltered
|
||||
case tools.PolicyRefusalPermissionDenied, tools.PolicyRefusalPermissionRequired:
|
||||
return DenialPermissionDenied
|
||||
case tools.PolicyRefusalSandboxDenied, tools.PolicyRefusalSandboxApproval:
|
||||
return DenialSandboxBlock
|
||||
default:
|
||||
return DenialNone
|
||||
}
|
||||
}
|
||||
|
||||
const sandboxNamespaceLimitedReason = "sandbox output is limited to the sandbox PID namespace; host/global state requires approval"
|
||||
|
||||
func maybeRetryUnsandboxedAfterSandboxRestriction(ctx context.Context, registry *tools.Registry, call ToolCall, tool tools.Tool, args map[string]any, result tools.Result, permissionMode PermissionMode, options Options, progressCallback func(streamjson.Event)) (tools.Result, *ToolResult, bool, PermissionDecisionAction, string, []string, error) {
|
||||
@@ -1938,6 +2051,10 @@ func toolResultFromPrePermissionReject(call ToolCall, result tools.Result) ToolR
|
||||
Display: display,
|
||||
LoadedTools: loadedToolsFromResult(meta),
|
||||
RequestedModel: meta["escalate_to_model"],
|
||||
// The SAME identity the executed path derives. This is the route a
|
||||
// RejectBeforePermission refusal takes, so leaving it empty here is what
|
||||
// made the marker classify as a refusal and key as an error signature.
|
||||
DenialReason: denialCategoryForResult(tools.Result{Meta: meta}),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2109,21 +2226,64 @@ func isRetriableToolError(result ToolResult) bool {
|
||||
// A categorized denial (filtered / permission / sandbox) is a policy decision,
|
||||
// not a transient failure — never retry it. This is robust to message wording
|
||||
// (the text checks below remain as a fallback for results lacking the field).
|
||||
return !isPolicyRefusal(result)
|
||||
}
|
||||
|
||||
// isPolicyRefusal reports a result the run refused on policy grounds: a
|
||||
// permission gate, a filter, a sandbox preflight, or a hook.
|
||||
//
|
||||
// Split out of isRetriableToolError so the two questions cannot drift apart,
|
||||
// which is exactly what they had done. The halt guard asked
|
||||
// `DenialReason != DenialNone`, and a category is only attached on the paths
|
||||
// that build a TYPED denial. A headless run leaves OnPermissionRequest nil, so
|
||||
// the loop never reaches its typed-denial branch and the registry returns a bare
|
||||
// `Error: Permission required ...` carrying no category. A sandbox preflight
|
||||
// denial on a non-shell tool loses its SandboxDecision converting to ToolResult
|
||||
// and arrives as an uncategorized `Sandbox block`.
|
||||
//
|
||||
// Both are refusals, and both were counted as SUCCESS, so observeToolResult
|
||||
// cleared the record and the same refused call could repeat to MaxTurns. That is
|
||||
// the loop this guard exists to stop. The text checks below already enumerated
|
||||
// these outcomes for the retriable question; they simply were not reachable from
|
||||
// the counting one.
|
||||
func isPolicyRefusal(result ToolResult) bool {
|
||||
// A REFUSAL IS A FAILURE. Everything below inspects metadata and then output
|
||||
// text, and neither is meaningful on a result the tool completed.
|
||||
//
|
||||
// isRetriableToolError gates on this before it calls here, so the boundary
|
||||
// held while that was the only caller. Extracting this helper and calling it
|
||||
// straight from the counting path dropped the gate: an allowed `bash`
|
||||
// printing "Sandbox block", or a read_file returning a document that quotes
|
||||
// one of these phrases, was counted as a denial. Six such successes tripped
|
||||
// the same-signature stop and ended a healthy run with a refusal answer.
|
||||
//
|
||||
// The gate belongs here rather than at each caller, because the next caller
|
||||
// will forget it too.
|
||||
if result.Status != tools.StatusError {
|
||||
return false
|
||||
}
|
||||
if result.DenialReason != DenialNone {
|
||||
return false
|
||||
return true
|
||||
}
|
||||
if result.Meta["permission_action"] == string(PermissionActionDeny) {
|
||||
return false
|
||||
if result.Meta[tools.PermissionActionMeta] == string(PermissionActionDeny) {
|
||||
return true
|
||||
}
|
||||
switch {
|
||||
case strings.Contains(result.Output, "is not enabled for this run"),
|
||||
strings.Contains(result.Output, "Permission denied for "),
|
||||
strings.Contains(result.Output, "Permission required for "),
|
||||
strings.Contains(result.Output, "Sandbox block"),
|
||||
strings.Contains(result.Output, "Sandbox approval required for "):
|
||||
return false
|
||||
}
|
||||
return true
|
||||
// STRUCTURED ONLY. This used to fall back to matching phrases in Output
|
||||
// ("Sandbox block", "Permission denied for ", and so on), which cannot work:
|
||||
// Output is tool-controlled. `bash` preserves arbitrary stdout and stderr on
|
||||
// a StatusError for any nonzero exit, so an ALLOWED command running
|
||||
// `printf 'Sandbox block\n' >&2; exit 1` has actually executed, carries no
|
||||
// DenialReason, and was still classified as refused. The loop then withheld
|
||||
// the schema hint and the profile failure-streak recovery, counted the
|
||||
// executed failure toward the refusal halt, and a later stop told the user a
|
||||
// tool had been refused when it had run.
|
||||
//
|
||||
// read_file returning a document that quotes one of those phrases did the
|
||||
// same thing, which is the more likely way a real session hits it.
|
||||
//
|
||||
// The registry now marks every path that returns BEFORE the tool runs, so
|
||||
// the question is answerable from provenance. Nothing below reads Output.
|
||||
return tools.IsPolicyRefusalResult(tools.Result{Meta: result.Meta})
|
||||
}
|
||||
|
||||
// scrubInterceptedOutput mirrors the registry's scrubResultSecrets boundary for
|
||||
@@ -2689,7 +2849,7 @@ func deniedPermissionResult(call ToolCall, reason string, requestEvent Permissio
|
||||
Output: "Error: Permission denied for " + call.Name + ": " + reason,
|
||||
DenialReason: denial,
|
||||
Meta: map[string]string{
|
||||
"permission_action": string(event.Action),
|
||||
tools.PermissionActionMeta: string(event.Action),
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -2715,7 +2875,7 @@ func canceledPermissionResult(call ToolCall, reason string, requestEvent Permiss
|
||||
Output: "Error: Permission approval cancelled for " + call.Name + ": " + reason,
|
||||
DenialReason: DenialApprovalCanceled,
|
||||
Meta: map[string]string{
|
||||
"permission_action": string(event.Action),
|
||||
tools.PermissionActionMeta: string(event.Action),
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -3569,6 +3729,33 @@ func toolResultImageMessage(result ToolResult, options Options) (zeroruntime.Mes
|
||||
}, true
|
||||
}
|
||||
|
||||
// precomputedResultFor reports the read-ahead result for call index, and whether
|
||||
// that call actually produced one. A batch entry that never started is still
|
||||
// unstarted as far as the transcript is concerned.
|
||||
//
|
||||
// PRODUCING A RESULT AND ASKING THE RUN TO STOP ARE DIFFERENT FACTS. Treating a
|
||||
// non-nil abortErr as "unstarted" conflated them, and a cancelled permission
|
||||
// request is exactly the case where they disagree: executeToolCall builds the
|
||||
// cancellation result and returns it WITH ErrPermissionApprovalCanceled, so an
|
||||
// earlier sibling reaching a terminal branch discarded a real result and wrote
|
||||
// an aborted placeholder in its place. The transcript then denied that the call
|
||||
// had completed permission handling, and the cancellation was missing from
|
||||
// OnToolResult, the trace counters and the task observation even though the
|
||||
// permission event had already happened.
|
||||
//
|
||||
// The terminal decision is not affected: draining such a sibling only records
|
||||
// what occurred, it cannot reverse a stop already selected.
|
||||
func precomputedResultFor(precomputed []precomputedToolResult, start, end, index int) (ToolResult, bool) {
|
||||
if index < start || index >= end {
|
||||
return ToolResult{}, false
|
||||
}
|
||||
entry := precomputed[index-start]
|
||||
if !entry.completed {
|
||||
return ToolResult{}, false
|
||||
}
|
||||
return entry.result, true
|
||||
}
|
||||
|
||||
func modelAcceptsToolImages(options Options) bool {
|
||||
if options.SupportsVision != nil {
|
||||
return options.SupportsVision(options.Model)
|
||||
|
||||
@@ -685,11 +685,23 @@ func TestIsRetriableToolError(t *testing.T) {
|
||||
{"success", ToolResult{Status: tools.StatusOK}, false},
|
||||
{"bad arguments", ToolResult{Status: tools.StatusError, Output: "Error: Failed to parse arguments for x: bad json"}, true},
|
||||
{"execution failure", ToolResult{Status: tools.StatusError, Output: "Error: read foo.txt: no such file"}, true},
|
||||
{"disabled tool", ToolResult{Status: tools.StatusError, Output: `Error: Tool "x" is not enabled for this run.`}, false},
|
||||
// Each refusal carries the provenance its production path attaches: the
|
||||
// disabled-tool path sets a denial category, the registry gates set the
|
||||
// pre-execution marker. Text is never what decides.
|
||||
{"disabled tool", ToolResult{Status: tools.StatusError, Output: `Error: Tool "x" is not enabled for this run.`, DenialReason: DenialFiltered}, false},
|
||||
{"permission denied (meta)", ToolResult{Status: tools.StatusError, Output: "Error: Permission denied for x: needs approval", Meta: map[string]string{"permission_action": "deny"}}, false},
|
||||
{"permission required", ToolResult{Status: tools.StatusError, Output: "Error: Permission required for x: approve first"}, false},
|
||||
{"sandbox block", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox block: outside_workspace"}, false},
|
||||
{"sandbox approval", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox approval required for x: network"}, false},
|
||||
{"permission required", ToolResult{Status: tools.StatusError, Output: "Error: Permission required for x: approve first", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalPermissionRequired}}, false},
|
||||
{"sandbox block", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox block: outside_workspace", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied}}, false},
|
||||
{"sandbox approval", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox approval required for x: network", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxApproval}}, false},
|
||||
// THE INVERSE, which is the failure this classifier had. An allowed tool
|
||||
// that RAN, failed, and happened to print one of those phrases must stay
|
||||
// retriable: `bash` preserves arbitrary stdout and stderr on any nonzero
|
||||
// exit, and read_file returns whatever the file says.
|
||||
{"executed failure printing the sandbox phrase", ToolResult{Status: tools.StatusError, Output: "Sandbox block\n"}, true},
|
||||
{"executed failure printing a permission denial", ToolResult{Status: tools.StatusError, Output: "cp: Permission denied for /etc/hosts"}, true},
|
||||
{"executed failure printing a permission requirement", ToolResult{Status: tools.StatusError, Output: "Permission required for sudo"}, true},
|
||||
{"executed failure printing the approval phrase", ToolResult{Status: tools.StatusError, Output: "Sandbox approval required for curl"}, true},
|
||||
{"executed failure quoting the disabled-tool phrase", ToolResult{Status: tools.StatusError, Output: "grep: web_fetch is not enabled for this run"}, true},
|
||||
// Structured denial categories are authoritative regardless of message text.
|
||||
{"denial: filtered", ToolResult{Status: tools.StatusError, Output: "anything", DenialReason: DenialFiltered}, false},
|
||||
{"denial: permission", ToolResult{Status: tools.StatusError, Output: "anything", DenialReason: DenialPermissionDenied}, false},
|
||||
@@ -4193,6 +4205,86 @@ func TestRunNilTraceForwardsUsage(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunReturnsStreamErrorWithStatusCodeAndCause covers the collector-to-agent
|
||||
// propagation boundary: a provider stream that terminates with a StreamEventError
|
||||
// carrying StatusCode/Cause must have Run return an error that errors.As can
|
||||
// still recover those fields from — the detail a session's "error" event needs
|
||||
// to actually diagnose why a provider call failed (#674).
|
||||
func TestRunReturnsStreamErrorWithStatusCodeAndCause(t *testing.T) {
|
||||
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
|
||||
{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: "provider error: provider returned error",
|
||||
StatusCode: 500,
|
||||
Cause: "upstream returned an empty completion",
|
||||
},
|
||||
}}}
|
||||
|
||||
_, err := Run(context.Background(), "hi", provider, Options{
|
||||
SessionID: "stream-error-session",
|
||||
Cwd: t.TempDir(),
|
||||
ProviderName: "test-provider",
|
||||
Model: "test-model",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("Run: expected an error, got nil")
|
||||
}
|
||||
|
||||
var streamErr *zeroruntime.StreamError
|
||||
if !errors.As(err, &streamErr) {
|
||||
t.Fatalf("Run error %v (%T) does not unwrap to *zeroruntime.StreamError", err, err)
|
||||
}
|
||||
if streamErr.StatusCode != 500 {
|
||||
t.Fatalf("StatusCode = %d, want 500", streamErr.StatusCode)
|
||||
}
|
||||
if streamErr.Cause != "upstream returned an empty completion" {
|
||||
t.Fatalf("Cause = %q, want %q", streamErr.Cause, "upstream returned an empty completion")
|
||||
}
|
||||
if err.Error() != "provider error: provider returned error" {
|
||||
t.Fatalf("Error() = %q, want the flattened message unchanged", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunImageRejectionKeepsStreamErrorStatusCodeAndCause covers the 400
|
||||
// image-rejection recovery path: the friendly message is kept, but the returned
|
||||
// error must still unwrap to the *StreamError so StatusCode and Cause reach the
|
||||
// session's "error" event (#674).
|
||||
func TestRunImageRejectionKeepsStreamErrorStatusCodeAndCause(t *testing.T) {
|
||||
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
|
||||
{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: "provider error 400: image input is not supported",
|
||||
StatusCode: 400,
|
||||
Cause: "this model does not support image input",
|
||||
},
|
||||
}}}
|
||||
|
||||
_, err := Run(context.Background(), "hi", provider, Options{
|
||||
SessionID: "image-rejection-session",
|
||||
Cwd: t.TempDir(),
|
||||
ProviderName: "test-provider",
|
||||
Model: "test-model",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("Run: expected an error, got nil")
|
||||
}
|
||||
|
||||
want := "model test-model rejected the image: provider error 400: image input is not supported. The model may not support image input — try switching to a vision-capable model (claude, gpt-4o, gemini)"
|
||||
if err.Error() != want {
|
||||
t.Fatalf("Error() = %q, want the friendly message unchanged: %q", err.Error(), want)
|
||||
}
|
||||
var streamErr *zeroruntime.StreamError
|
||||
if !errors.As(err, &streamErr) {
|
||||
t.Fatalf("Run error %v (%T) does not unwrap to *zeroruntime.StreamError", err, err)
|
||||
}
|
||||
if streamErr.StatusCode != 400 {
|
||||
t.Fatalf("StatusCode = %d, want 400", streamErr.StatusCode)
|
||||
}
|
||||
if streamErr.Cause != "this model does not support image input" {
|
||||
t.Fatalf("Cause = %q, want the upstream cause", streamErr.Cause)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunSuppressesAdvisoryHooksInPlanMode: plan mode promises a read-only
|
||||
// turn for advisory hooks (sessionStart/sessionEnd/afterTool), which execute
|
||||
// configured host commands outside the advertised-tool and sandbox gates.
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// mixedOutcomeTool needs permission on every call. The test denies every other
|
||||
// request, and a call that is allowed executes and fails with an error naming
|
||||
// its own item, so no single identity repeats and only the content-blind bound
|
||||
// can stop the run.
|
||||
type mixedOutcomeTool struct{ ran int }
|
||||
|
||||
func (tool *mixedOutcomeTool) Name() string { return "bash" }
|
||||
func (tool *mixedOutcomeTool) Description() string { return "probe" }
|
||||
func (tool *mixedOutcomeTool) Parameters() tools.Schema { return tools.Schema{Type: "object"} }
|
||||
func (tool *mixedOutcomeTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs a command"}
|
||||
}
|
||||
|
||||
func (tool *mixedOutcomeTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusError, Output: "Error: item " + strconv.Itoa(tool.ran) + " failed"}
|
||||
}
|
||||
|
||||
// A MIXED HALT READS THE SAME HEADLESS AS INTERACTIVE.
|
||||
//
|
||||
// The stop answer for the content-blind bound over executed failures and
|
||||
// refusals says the tool "failed or was refused". The headless reason for the
|
||||
// same halt said only that it "failed repeatedly", so zero exec and the
|
||||
// interactive answer disagreed about one stop. Reported by @jatmn.
|
||||
func TestAMixedGuardHaltIsReportedAsMixedUnderTheCompletionGate(t *testing.T) {
|
||||
tool := &mixedOutcomeTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
|
||||
for i := range toolFailureAnyErrorStopAt + 4 {
|
||||
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "bash", `{"item":`+strconv.Itoa(i)+`}`))
|
||||
}
|
||||
asked := 0
|
||||
result, err := Run(context.Background(), "run them", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
RequireCompletionSignal: true,
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
asked++
|
||||
if asked%2 == 0 {
|
||||
return PermissionDecision{Action: PermissionDecisionAllow}, nil
|
||||
}
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran == 0 || asked <= tool.ran {
|
||||
t.Fatalf("the run did not mix executed failures with refusals (ran %d of %d asked), so this proves nothing", tool.ran, asked)
|
||||
}
|
||||
if !strings.Contains(result.FinalAnswer, "failed or was refused") {
|
||||
t.Fatalf("the halt was not the mixed content-blind bound: %q", result.FinalAnswer)
|
||||
}
|
||||
if !result.Incomplete {
|
||||
t.Fatal("a guard halt under the completion gate must be Incomplete")
|
||||
}
|
||||
if !strings.Contains(result.IncompleteReason, "failed or was refused") || !strings.Contains(result.IncompleteReason, "bash") {
|
||||
t.Errorf("IncompleteReason = %q, want it to say the tool failed or was refused, as the stop answer does", result.IncompleteReason)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// readAheadTool is read-only and thread-safe, so a consecutive run of its calls
|
||||
// is executed as one parallel batch before the loop consumes any of it.
|
||||
type readAheadTool struct {
|
||||
mu sync.Mutex
|
||||
ran int
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Name() string { return "read_probe" }
|
||||
func (t *readAheadTool) Description() string { return "read-only probe" }
|
||||
func (t *readAheadTool) Parameters() tools.Schema {
|
||||
return tools.Schema{Type: "object", Properties: map[string]tools.PropertySchema{"id": {Type: "string"}}}
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "test"}
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Capabilities() tools.ToolCapabilities {
|
||||
return tools.ToolCapabilities{Effect: tools.EffectReadOnly, ThreadSafe: true}
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Run(_ context.Context, args map[string]any) tools.Result {
|
||||
t.mu.Lock()
|
||||
t.ran++
|
||||
t.mu.Unlock()
|
||||
id, _ := args["id"].(string)
|
||||
// Varying output, so no identity repeats and only the content-blind counter
|
||||
// can halt the run.
|
||||
return tools.Result{Status: tools.StatusError, Output: "failure for " + id}
|
||||
}
|
||||
|
||||
func readCall(id string) []zeroruntime.StreamEvent {
|
||||
return []zeroruntime.StreamEvent{
|
||||
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: id, ToolCallProviderID: "provider-" + id, ToolName: "read_probe"},
|
||||
{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: id, ArgumentsFragment: `{"id":"` + id + `"}`},
|
||||
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: id},
|
||||
}
|
||||
}
|
||||
|
||||
// READ-AHEAD MEANS "NOT CONSUMED YET" NO LONGER MEANS "NOT EXECUTED".
|
||||
//
|
||||
// executeParallelReadBatch runs an entire eligible run of read calls before the
|
||||
// loop consumes any of them. Every terminal branch in the consumption loop used
|
||||
// to close out the calls after the current index with aborted placeholders, on
|
||||
// the assumption that they had not run. A sibling that had already executed was
|
||||
// therefore recorded as aborted: its real result was discarded, the transcript
|
||||
// disagreed with what had happened, and its callbacks, trace counter, task
|
||||
// observation and images were lost with it. Where such a sibling is a
|
||||
// successful read, execution may already have committed file-observation credit
|
||||
// for content the model never receives.
|
||||
func TestParallelSiblingsAreFinalizedWhenTheGuardHalts(t *testing.T) {
|
||||
tool := &readAheadTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// Eleven single-call turns bring the content-blind counter to one below its
|
||||
// bound, so the first call of the pair below trips it.
|
||||
var turns [][]zeroruntime.StreamEvent
|
||||
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
|
||||
id := "s" + strconv.Itoa(i)
|
||||
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
}
|
||||
// Then one turn advertising two parallel-eligible calls. Both execute in the
|
||||
// batch; the halt fires while consuming the first.
|
||||
pair := append(readCall("pA"), readCall("pB")...)
|
||||
turns = append(turns, append(pair, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
turns = append(turns, append(readCall("unreached"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
|
||||
var reported []ToolResult
|
||||
var announced int
|
||||
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAuto,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnToolCall: func(ToolCall) { announced++ },
|
||||
OnToolResult: func(r ToolResult) { reported = append(reported, r) },
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
|
||||
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
|
||||
}
|
||||
|
||||
// Every execution is reported exactly once, and the announcements match.
|
||||
if tool.ran != len(reported) {
|
||||
t.Errorf("the tool executed %d times but %d results were reported; a completed call was recorded as aborted",
|
||||
tool.ran, len(reported))
|
||||
}
|
||||
if announced != len(reported) {
|
||||
t.Errorf("OnToolCall fired %d times against %d results; the two callbacks disagree", announced, len(reported))
|
||||
}
|
||||
|
||||
// Strict pairing: one tool result per advertised tool call, and the drained
|
||||
// sibling carries its REAL output rather than the aborted placeholder.
|
||||
byID := map[string]int{}
|
||||
var siblingContent string
|
||||
for _, message := range result.Messages {
|
||||
if message.Role != zeroruntime.MessageRoleTool {
|
||||
continue
|
||||
}
|
||||
byID[message.ToolCallID]++
|
||||
if message.ToolCallID == "pB" {
|
||||
siblingContent = message.Content
|
||||
}
|
||||
// A strict replay pairs each result to its tool_use by the provider
|
||||
// call ID, so the drained sibling needs it as much as the consumed call.
|
||||
if (message.ToolCallID == "pA" || message.ToolCallID == "pB") && message.ToolCallProviderID != "provider-"+message.ToolCallID {
|
||||
t.Errorf("tool result %s carries provider call ID %q, want %q", message.ToolCallID, message.ToolCallProviderID, "provider-"+message.ToolCallID)
|
||||
}
|
||||
}
|
||||
for _, id := range []string{"pA", "pB"} {
|
||||
if byID[id] != 1 {
|
||||
t.Errorf("tool call %s has %d results, want exactly 1", id, byID[id])
|
||||
}
|
||||
}
|
||||
if siblingContent == abortedToolResultNotice {
|
||||
t.Error("the sibling that already ran was recorded as aborted")
|
||||
}
|
||||
if !strings.Contains(siblingContent, "failure for pB") {
|
||||
t.Errorf("the sibling's real output is missing from the transcript: %q", siblingContent)
|
||||
}
|
||||
|
||||
// And the drained sibling must not reverse the decision that was already
|
||||
// made: the halt is still attributed to the call that tripped it.
|
||||
if !strings.Contains(result.FinalAnswer, strconv.Itoa(toolFailureAnyErrorStopAt)) {
|
||||
t.Errorf("the stop answer changed after draining siblings: %q", result.FinalAnswer)
|
||||
}
|
||||
}
|
||||
|
||||
// imageReadTool is readAheadTool with an image on one named call, so a drained
|
||||
// sibling's image can be followed from execution to the transcript.
|
||||
type imageReadTool struct {
|
||||
readAheadTool
|
||||
imageFor string
|
||||
}
|
||||
|
||||
func (t *imageReadTool) Run(ctx context.Context, args map[string]any) tools.Result {
|
||||
result := t.readAheadTool.Run(ctx, args)
|
||||
if id, _ := args["id"].(string); id == t.imageFor {
|
||||
result.Images = []zeroruntime.ImageBlock{{MediaType: "image/png", Data: []byte("\x89PNG")}}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// The drained sibling's image is part of what ran, so it follows the tool
|
||||
// results like any other call's image. Images reach the transcript through the
|
||||
// turn's deferred builder, which the halt path calls after closing out the
|
||||
// siblings; a drain that skipped the sibling's image would lose it silently.
|
||||
func TestParallelSiblingImageSurvivesTheGuardHalt(t *testing.T) {
|
||||
tool := &imageReadTool{imageFor: "pB"}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
var turns [][]zeroruntime.StreamEvent
|
||||
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
|
||||
id := "s" + strconv.Itoa(i)
|
||||
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
}
|
||||
pair := append(readCall("pA"), readCall("pB")...)
|
||||
turns = append(turns, append(pair, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
|
||||
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAuto,
|
||||
MaxTurns: len(turns) + 5,
|
||||
SupportsVision: func(string) bool { return true },
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
|
||||
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
|
||||
}
|
||||
images := 0
|
||||
for _, message := range result.Messages {
|
||||
if message.Role == zeroruntime.MessageRoleUser {
|
||||
images += len(message.Images)
|
||||
}
|
||||
}
|
||||
if images != 1 {
|
||||
t.Errorf("the drained sibling's image reached the transcript %d times, want once", images)
|
||||
}
|
||||
}
|
||||
|
||||
// A call that genuinely never ran still gets an aborted placeholder, so the fix
|
||||
// did not simply stop aborting anything.
|
||||
func TestUnstartedCallsStillGetAbortedPlaceholders(t *testing.T) {
|
||||
tool := &readAheadTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
var turns [][]zeroruntime.StreamEvent
|
||||
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
|
||||
id := "s" + strconv.Itoa(i)
|
||||
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
}
|
||||
// One eligible pair, then a THIRD call outside the batch window is not what
|
||||
// happens here: the whole run is eligible, so instead advertise a single call
|
||||
// and let the halt fire on it, leaving nothing precomputed beyond it.
|
||||
turns = append(turns, append(readCall("solo"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
turns = append(turns, append(readCall("unreached"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
|
||||
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAuto,
|
||||
MaxTurns: len(turns) + 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
|
||||
t.Fatalf("the guard did not halt: %q", result.FinalAnswer)
|
||||
}
|
||||
if tool.ran != toolFailureAnyErrorStopAt {
|
||||
t.Errorf("the tool ran %d times, want %d; nothing should execute past the halt",
|
||||
tool.ran, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,7 @@ package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
@@ -12,8 +13,17 @@ import (
|
||||
// them one after another serializes pure I/O waits. A consecutive run of
|
||||
// capability-safe read-only calls is executed concurrently instead; results are
|
||||
// then consumed in the original call order, so guard counters, message
|
||||
// ordering, abort semantics, and the surface's call/result event pairing are
|
||||
// byte-identical to sequential execution. Runs never span a mutating call: a
|
||||
// ordering, abort semantics, and the surface's call/result event pairing match
|
||||
// sequential execution.
|
||||
//
|
||||
// What can differ is how many calls RAN. A run executes before the guard
|
||||
// consumes its first result, so a halt on an earlier call can leave up to
|
||||
// maxParallelReadTools-1 siblings already executed. The guard bounds count
|
||||
// consumed results, not executions, and the extra executions are reads by the
|
||||
// eligibility rule below; closeOutRemaining records them without feeding the
|
||||
// guard. Reported by @jatmn.
|
||||
//
|
||||
// Runs never span a mutating call: a
|
||||
// read that follows a write must observe the write, so eligibility is decided
|
||||
// per consecutive run, not per batch.
|
||||
//
|
||||
@@ -36,6 +46,15 @@ const maxParallelReadTools = 8
|
||||
type precomputedToolResult struct {
|
||||
result ToolResult
|
||||
abortErr error
|
||||
// completed records whether this call PRODUCED a result, which is a
|
||||
// different question from whether it asks the enclosing run to stop.
|
||||
//
|
||||
// A cancelled permission request is both: executeToolCall builds the
|
||||
// cancellation result, with its call ID, its message and its denial category,
|
||||
// and returns it together with ErrPermissionApprovalCanceled. Reading the
|
||||
// error as "never started" throws that result away, and the transcript then
|
||||
// denies that permission handling happened at all.
|
||||
completed bool
|
||||
}
|
||||
|
||||
// parallelSafeToolCall reports whether call may run concurrently with its
|
||||
@@ -189,7 +208,15 @@ func executeParallelReadBatch(ctx context.Context, registry *tools.Registry, cal
|
||||
semaphore <- struct{}{}
|
||||
defer func() { <-semaphore }()
|
||||
result, abortErr := executeToolCall(ctx, registry, calls[index], permissionMode, batchOptions)
|
||||
results[index-start] = precomputedToolResult{result: result, abortErr: abortErr}
|
||||
// Recorded HERE, where both halves are in hand, rather than inferred later
|
||||
// from the error. A populated call ID is what every path that produced a
|
||||
// result sets and what a context cancellation before execution leaves
|
||||
// empty.
|
||||
results[index-start] = precomputedToolResult{
|
||||
result: result,
|
||||
abortErr: abortErr,
|
||||
completed: strings.TrimSpace(result.ToolCallID) != "",
|
||||
}
|
||||
}(index)
|
||||
}
|
||||
waitGroup.Wait()
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// forgedMarkerTool executes, fails, and claims in its metadata that the registry
|
||||
// refused it before it ran.
|
||||
type forgedMarkerTool struct {
|
||||
ran int
|
||||
key string
|
||||
value string
|
||||
}
|
||||
|
||||
func (tool *forgedMarkerTool) Name() string { return "bash" }
|
||||
func (tool *forgedMarkerTool) Description() string { return "test shell tool" }
|
||||
func (tool *forgedMarkerTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *forgedMarkerTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
|
||||
}
|
||||
func (tool *forgedMarkerTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
Output: "Error: the script exited 1",
|
||||
Meta: map[string]string{tool.key: tool.value},
|
||||
}
|
||||
}
|
||||
|
||||
// THE FORGED MARKER MUST NOT CHANGE THE RUN.
|
||||
//
|
||||
// Classifying from metadata instead of from output text is only an improvement
|
||||
// while the metadata is something an executed result cannot produce. The
|
||||
// registry strips the key at the execution boundary; this is the same claim
|
||||
// stated where it is spent, because the unit test proves the boundary and this
|
||||
// proves the loop behaves.
|
||||
//
|
||||
// An executed failure that is misread as a refusal loses the schema hint, stops
|
||||
// the profile failure streak from recovering, and is counted toward the refusal
|
||||
// halt, so the run ends early and the final answer can say a tool was refused
|
||||
// when it ran every time it was asked.
|
||||
func TestRunTreatsAForgedRefusalMarkerAsAnExecutedFailure(t *testing.T) {
|
||||
for _, forged := range []struct{ key, value string }{
|
||||
{tools.PolicyRefusalMeta, tools.PolicyRefusalSandboxDenied},
|
||||
{tools.PolicyRefusalMeta, tools.PolicyRefusalPermissionDenied},
|
||||
{tools.PolicyRefusalMeta, tools.PolicyRefusalToolNotEnabled},
|
||||
{tools.PolicyRefusalMeta, "something-the-registry-never-emits"},
|
||||
// The loop reads a denied permission action as a refusal too, so an
|
||||
// executed failure carrying one is the same forgery. Reported by @jatmn.
|
||||
{tools.PermissionActionMeta, string(PermissionActionDeny)},
|
||||
} {
|
||||
value := forged.value
|
||||
t.Run(forged.key+"="+value, func(t *testing.T) {
|
||||
tool := &forgedMarkerTool{key: forged.key, value: value}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// One more call than the hint threshold, so the hint has to have been
|
||||
// injected by the last one, then a final text turn to end the run.
|
||||
calls := toolFailureHintAt + 1
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash", `{"command":"./flaky.sh"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up on the script"))
|
||||
|
||||
var denials []DenialCategory
|
||||
result, err := Run(context.Background(), "run the script", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnToolResult: func(toolResult ToolResult) {
|
||||
denials = append(denials, toolResult.DenialReason)
|
||||
},
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
t.Error("permission was requested for an allow-safety tool; this test must exercise the executed-failure path")
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != calls {
|
||||
t.Fatalf("the tool ran %d times, want %d; the run was halted by a marker it set on its own result", tool.ran, calls)
|
||||
}
|
||||
for index, denial := range denials {
|
||||
if denial != DenialNone {
|
||||
t.Errorf("result %d was recorded as denial %q; the tool executed", index, denial)
|
||||
}
|
||||
}
|
||||
var hinted bool
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, toolFailureHintMarker) {
|
||||
hinted = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !hinted {
|
||||
t.Errorf("no retry hint was injected for an executed failure carrying %q, so the model got no correction", value)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// THE UNCATEGORIZED REFUSALS ARE THE ONES THIS CHANGE EXISTS FOR, SO THEY ARE THE
|
||||
// ONES THAT MUST BE DRIVEN THROUGH Run.
|
||||
//
|
||||
// A categorized denial is easy to test and was never the gap. The gap is a
|
||||
// refusal that arrives with DenialReason empty, because a category is attached
|
||||
// only where a TYPED denial is built:
|
||||
//
|
||||
// - a headless run leaves OnPermissionRequest nil, so the loop never reaches
|
||||
// its typed-denial branch and the registry gate returns a bare
|
||||
// `Permission required for ...`
|
||||
// - a sandbox preflight denial on a non-shell tool loses its SandboxDecision
|
||||
// converting to ToolResult and arrives as a bare `Sandbox block`
|
||||
//
|
||||
// Both were counted as SUCCESS, which cleared the very record they were meant to
|
||||
// accumulate, and the refused call repeated to MaxTurns.
|
||||
//
|
||||
// Asserting that through the helper alone would prove nothing: the first version
|
||||
// of this fix passed every helper test while being a no-op in production,
|
||||
// because the loop asked a different question than the tests did. So each case
|
||||
// below goes through Run and pins the three things the loop actually does with
|
||||
// the classification: halt at the bound, never execute the tool, and withhold
|
||||
// the profile's one-shot failure escalation.
|
||||
|
||||
// headlessPromptTool needs approval and never gets it, because the run has no
|
||||
// approver. The registry refuses it before Run() is reached, exactly as a
|
||||
// headless run does, so nothing here fakes the refusal.
|
||||
type headlessPromptTool struct{ ran int }
|
||||
|
||||
func (tool *headlessPromptTool) Name() string { return "bash" }
|
||||
func (tool *headlessPromptTool) Description() string { return "test shell tool" }
|
||||
func (tool *headlessPromptTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *headlessPromptTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs shell commands"}
|
||||
}
|
||||
func (tool *headlessPromptTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusOK, Output: "should never run"}
|
||||
}
|
||||
|
||||
// A headless prompt refusal must halt at the same-signature bound rather than
|
||||
// repeat to MaxTurns.
|
||||
//
|
||||
// It used to key on the refusal TEXT, which the registry happens to hold
|
||||
// constant here, and the stop then described a repeated failure. The registry
|
||||
// marks this path, so the category is now derived at the ToolResult boundary and
|
||||
// the streak keys on that instead. The bound is unchanged; the wording is more
|
||||
// honest, because a headless prompt refusal is a refusal and not a tool that
|
||||
// keeps failing.
|
||||
func TestRunStopsAnUncategorizedHeadlessRefusalAtTheFailureBound(t *testing.T) {
|
||||
tool := &headlessPromptTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// Comfortably more turns than the bound, so reaching the bound is what stops
|
||||
// the run rather than exhausting the provider or MaxTurns.
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
|
||||
for i := range toolFailureStopAt + 4 {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
|
||||
`{"command":"touch /etc/file`+strconv.Itoa(i)+`"}`))
|
||||
}
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
recorder := trace.NewRecorder("policy-refusal-session", "run-1", "fast")
|
||||
result, err := Run(context.Background(), "do the thing", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
Trace: recorder,
|
||||
// Armed well below the halt bound, so a refusal counted as a retriable
|
||||
// failure would visibly spend it.
|
||||
Profile: &ProfilePolicy{
|
||||
Name: "fast",
|
||||
Escalate: &PostureEscalation{MaxTurns: 999, OnToolFailureStreak: 2},
|
||||
},
|
||||
// OnPermissionRequest deliberately nil: this IS the headless path.
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if len(provider.requests) != toolFailureStopAt {
|
||||
t.Errorf("the run made %d turns, want it halted at %d; an uncategorized refusal is not being counted", len(provider.requests), toolFailureStopAt)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Errorf("the refused tool executed %d times; the registry gate must precede execution", tool.ran)
|
||||
}
|
||||
// Categorized now, so the stop answer says refused. Previously the loop could
|
||||
// not tell and reported a repeated failure.
|
||||
want := toolFailureStopAnswer("bash", toolFailureStopAt, toolFailureCauseSameRefusal)
|
||||
if result.FinalAnswer != want {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
|
||||
}
|
||||
assertNoPostureEscalation(t, recorder)
|
||||
}
|
||||
|
||||
// uncategorizedSandboxTool returns the shape a sandbox preflight denial has by
|
||||
// the time the loop sees it on a non-shell tool: StatusError, `Sandbox block`
|
||||
// prose, and no category, because the SandboxDecision does not survive the
|
||||
// conversion to ToolResult. The path varies per call exactly as the real message
|
||||
// does, since it names what was refused.
|
||||
type uncategorizedSandboxTool struct{ ran int }
|
||||
|
||||
func (tool *uncategorizedSandboxTool) Name() string { return "write_file" }
|
||||
func (tool *uncategorizedSandboxTool) Description() string { return "test write tool" }
|
||||
func (tool *uncategorizedSandboxTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"path": {Type: "string"}},
|
||||
Required: []string{"path"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *uncategorizedSandboxTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectWrite, Permission: tools.PermissionAllow, Reason: "writes files"}
|
||||
}
|
||||
func (tool *uncategorizedSandboxTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
Output: "Sandbox block: write to /out-" + strconv.Itoa(tool.ran) + ".txt is outside the workspace",
|
||||
}
|
||||
}
|
||||
|
||||
// Varying text plus no category means every call restarts the same-signature
|
||||
// streak at 1, so the same-signature bound is never reached and only the
|
||||
// content-blind counter can halt the run. Before the fix nothing halted it at
|
||||
// all: each refusal was read as a success, deleted the record, and the call
|
||||
// repeated until MaxTurns.
|
||||
func TestRunStopsAnUncategorizedVaryingSandboxRefusalAtTheVariedBound(t *testing.T) {
|
||||
tool := &uncategorizedSandboxTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
|
||||
for i := range toolFailureAnyErrorStopAt + 4 {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "write_file",
|
||||
`{"path":"/out-`+strconv.Itoa(i)+`.txt"}`))
|
||||
}
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
recorder := trace.NewRecorder("policy-refusal-session", "run-2", "fast")
|
||||
result, err := Run(context.Background(), "write the files", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
Trace: recorder,
|
||||
Profile: &ProfilePolicy{
|
||||
Name: "fast",
|
||||
Escalate: &PostureEscalation{MaxTurns: 999, OnToolFailureStreak: 2},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if tool.ran != toolFailureAnyErrorStopAt {
|
||||
t.Errorf("the tool was called %d times, want the run halted at %d", tool.ran, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
want := toolFailureStopAnswer("write_file", toolFailureAnyErrorStopAt, toolFailureCauseVariedExecuted)
|
||||
if result.FinalAnswer != want {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
|
||||
}
|
||||
assertNoPostureEscalation(t, recorder)
|
||||
}
|
||||
|
||||
// The profile's failure-streak trigger restores turn budget and reasoning effort
|
||||
// on the theory that a struggling tool needs room. A refusal is not a struggling
|
||||
// tool, it is an answer, and spending the one-shot escalation on one buys turns
|
||||
// that will be refused identically. The loop keeps this contract by passing an
|
||||
// empty outcome to the controller for a refusal, which is only observable from
|
||||
// outside as the escalation never firing.
|
||||
func assertNoPostureEscalation(t *testing.T, recorder *trace.Recorder) {
|
||||
t.Helper()
|
||||
if got := recorder.Finish().Counter(trace.CounterPostureEscalations); got != 0 {
|
||||
t.Errorf("posture_escalations = %d, want 0; a policy refusal spent the one-shot failure escalation", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,265 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// A REFUSAL IS A FAILURE, AND THE CLASSIFIER MUST SAY SO BEFORE IT READS ANYTHING
|
||||
// ELSE.
|
||||
//
|
||||
// isPolicyRefusal decides on denial category, then permission metadata, then
|
||||
// output text. None of those questions is meaningful about a call the tool
|
||||
// completed, and the last one is answered by content the model does not control:
|
||||
// a `bash` that greps the sandbox docs, a read_file returning this very source
|
||||
// file, any successful command whose output happens to carry one of the phrases.
|
||||
//
|
||||
// isRetriableToolError gated on StatusError before calling in, so the boundary
|
||||
// held while that was the only caller. Extracting the helper and calling it from
|
||||
// the counting path dropped the gate, and every successful result was put to a
|
||||
// question that only failures can answer.
|
||||
//
|
||||
// Each case below is a SUCCESS. None may be read as a refusal.
|
||||
func TestPolicyRefusalNeverClassifiesASuccessfulResult(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
result ToolResult
|
||||
// structured marks a result carrying real provenance: a denial category,
|
||||
// permission metadata, or the registry's pre-execution marker. Only those
|
||||
// may be read as a refusal when they fail. The rest carry nothing but
|
||||
// text, which the tool controls, so they must stay ordinary failures no
|
||||
// matter which phrase they happen to contain.
|
||||
structured bool
|
||||
}{
|
||||
{
|
||||
name: "output quotes the sandbox block phrase",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "grep: docs/sandbox.md: Sandbox block"},
|
||||
},
|
||||
{
|
||||
name: "output quotes the disabled-tool phrase",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "web_fetch is not enabled for this run"},
|
||||
},
|
||||
{
|
||||
name: "output quotes a permission denial",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "log line: Permission denied for bash"},
|
||||
},
|
||||
{
|
||||
name: "output quotes a permission requirement",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "log line: Permission required for bash"},
|
||||
},
|
||||
{
|
||||
name: "output quotes a sandbox approval prompt",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "Sandbox approval required for curl"},
|
||||
},
|
||||
{
|
||||
// Metadata is no more trustworthy than text on a completed call. A
|
||||
// result carrying a category with an OK status is not a denial; every
|
||||
// path that attaches one fails the call.
|
||||
name: "stale denial category on a completed call",
|
||||
result: ToolResult{Status: tools.StatusOK, DenialReason: DenialSandboxBlock, Output: "ok"},
|
||||
structured: true,
|
||||
},
|
||||
{
|
||||
name: "stale permission metadata on a completed call",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusOK,
|
||||
Meta: map[string]string{"permission_action": string(PermissionActionDeny)},
|
||||
Output: "ok",
|
||||
},
|
||||
structured: true,
|
||||
},
|
||||
{
|
||||
name: "registry refusal marker on a completed call",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusOK,
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied},
|
||||
Output: "ok",
|
||||
},
|
||||
structured: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, testCase := range cases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
if isPolicyRefusal(testCase.result) {
|
||||
t.Errorf("a successful result was classified as a policy refusal, so a healthy call counts toward the denial streak: %+v", testCase.result)
|
||||
}
|
||||
failed := testCase.result
|
||||
failed.Status = tools.StatusError
|
||||
switch {
|
||||
case testCase.structured && !isPolicyRefusal(failed):
|
||||
t.Errorf("a failure carrying real provenance was not classified as a refusal, so the gate broke the classifier: %+v", failed)
|
||||
case !testCase.structured && isPolicyRefusal(failed):
|
||||
t.Errorf("a failure carrying only text was classified as a refusal; output is tool-controlled and cannot decide this: %+v", failed)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// blockTextShellTool runs freely and prints text the classifier keys on. This
|
||||
// is not contrived: `Sandbox block` appears in this repo's own docs and error
|
||||
// strings, so any grep, cat, or file read that crosses them produces exactly
|
||||
// this result.
|
||||
type blockTextShellTool struct{ ran int }
|
||||
|
||||
func (tool *blockTextShellTool) Name() string { return "bash" }
|
||||
func (tool *blockTextShellTool) Description() string { return "test shell tool" }
|
||||
func (tool *blockTextShellTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *blockTextShellTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
|
||||
}
|
||||
func (tool *blockTextShellTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{
|
||||
Status: tools.StatusOK,
|
||||
Output: "docs/windows-sandbox.md:41: Sandbox block (write jail) is reported as ...",
|
||||
}
|
||||
}
|
||||
|
||||
// The same gap driven through Run, because the classifier being wrong only
|
||||
// matters if the loop acts on it, and it does: the counting path calls
|
||||
// isPolicyRefusal directly, and a true answer records a failure against the
|
||||
// tool's signature. Six identical successes then look like six identical
|
||||
// denials, and the run ends on a stop answer telling the user a tool it never
|
||||
// refused was refused six times.
|
||||
//
|
||||
// The tool succeeds every turn and the run is given more turns than the bound,
|
||||
// so anything that halts before the provider's final turn is the guard.
|
||||
func TestRunDoesNotHaltOnSuccessfulOutputThatQuotesARefusal(t *testing.T) {
|
||||
tool := &blockTextShellTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
calls := toolFailureStopAt + 4
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
|
||||
`{"command":"grep -n 'Sandbox block' docs/windows-sandbox.md"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("found the docs"))
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
result, err := Run(context.Background(), "search the docs", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
t.Error("the loop asked permission for an allow-safety tool; this test must exercise the success path")
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if tool.ran != calls {
|
||||
t.Errorf("the tool ran %d times, want %d; the run was halted mid-way by its own success output", tool.ran, calls)
|
||||
}
|
||||
if result.FinalAnswer != "found the docs" {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q\n(a stop answer here means successful output was counted as a refusal)",
|
||||
result.FinalAnswer, "found the docs")
|
||||
}
|
||||
}
|
||||
|
||||
// failingPhraseTool RAN and failed. Its stderr happens to contain a phrase the
|
||||
// classifier used to key on, which is the everyday case: `bash` preserves
|
||||
// arbitrary stdout and stderr on any nonzero exit, so
|
||||
// `printf 'Sandbox block\n' >&2; exit 1` produces exactly this.
|
||||
type failingPhraseTool struct {
|
||||
ran int
|
||||
output string
|
||||
}
|
||||
|
||||
func (tool *failingPhraseTool) Name() string { return "bash" }
|
||||
func (tool *failingPhraseTool) Description() string { return "test shell tool" }
|
||||
func (tool *failingPhraseTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *failingPhraseTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
|
||||
}
|
||||
func (tool *failingPhraseTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusError, Output: tool.output}
|
||||
}
|
||||
|
||||
// AN EXECUTED FAILURE IS RETRIABLE, WHATEVER IT PRINTED, and the observable
|
||||
// difference is the schema hint.
|
||||
//
|
||||
// isRetriableToolError feeds `hintable` into the guard, and the guard injects
|
||||
// the retry hint at toolFailureHintAt. Classifying an executed failure as a
|
||||
// policy refusal makes hintable false, so the model is told nothing and simply
|
||||
// repeats the call: the run loses the one correction it had. That is on top of
|
||||
// the failure being accumulated as a denial.
|
||||
//
|
||||
// Each phrase below is one the classifier used to match on. The tool is
|
||||
// allow-safety and runs every time, so nothing here is refused by anything.
|
||||
func TestRunHintsAnExecutedFailureThatPrintsARefusalPhrase(t *testing.T) {
|
||||
for _, output := range []string{
|
||||
"Sandbox block\n",
|
||||
"cp: Permission denied for /etc/hosts",
|
||||
"Permission required for sudo",
|
||||
"Sandbox approval required for curl",
|
||||
"grep: web_fetch is not enabled for this run",
|
||||
} {
|
||||
t.Run(output, func(t *testing.T) {
|
||||
tool := &failingPhraseTool{output: output}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// One more call than the hint threshold, so the hint has to have been
|
||||
// injected by the last one, then a final text turn to end the run.
|
||||
calls := toolFailureHintAt + 1
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash", `{"command":"./flaky.sh"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up on the script"))
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
result, err := Run(context.Background(), "run the script", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
t.Error("permission was requested for an allow-safety tool; this test must exercise the executed-failure path")
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != calls {
|
||||
t.Fatalf("the tool ran %d times, want %d; the run was halted by output it merely printed", tool.ran, calls)
|
||||
}
|
||||
|
||||
var hinted bool
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, toolFailureHintMarker) {
|
||||
hinted = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !hinted {
|
||||
t.Errorf("no retry hint was injected for an executed failure printing %q; it was classified as a policy refusal, so the model got no correction", output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// UNCATEGORIZED REFUSALS MUST STILL COUNT.
|
||||
//
|
||||
// The halt guard asked DenialReason != DenialNone, but a category is only
|
||||
// attached where a TYPED denial is built. A headless run leaves
|
||||
// OnPermissionRequest nil, so the loop never reaches that branch and the
|
||||
// registry returns a bare "Error: Permission required ..." with no category. A
|
||||
// sandbox preflight denial on a non-shell tool loses its SandboxDecision in the
|
||||
// conversion and arrives as an uncategorized "Sandbox block".
|
||||
//
|
||||
// Both were counted as SUCCESS, which cleared the very record the guard
|
||||
// accumulates, so the same refused call could repeat to MaxTurns. That is the
|
||||
// loop this PR exists to stop.
|
||||
func TestUncategorizedPolicyRefusalsAreCounted(t *testing.T) {
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
result ToolResult
|
||||
}{
|
||||
{
|
||||
// Still uncategorized in the DenialReason sense: a headless run never
|
||||
// reaches the branch that builds a typed denial. It carries the
|
||||
// registry's pre-execution marker instead, which is what makes it
|
||||
// classifiable without reading the message.
|
||||
name: "headless prompt refusal carries no denial category",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusError,
|
||||
Output: `Error: Permission required for bash: The tool is marked "prompt" and was not executed.`,
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalPermissionRequired},
|
||||
},
|
||||
},
|
||||
{
|
||||
// The SandboxDecision is still dropped in the conversion for a
|
||||
// non-shell tool; the marker is what survives it.
|
||||
name: "sandbox preflight denial loses its decision in conversion",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusError,
|
||||
Output: "Sandbox block: write outside the workspace",
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "an explicitly denied permission action",
|
||||
result: ToolResult{Status: tools.StatusError, Output: "refused", Meta: map[string]string{"permission_action": string(PermissionActionDeny)}},
|
||||
},
|
||||
{
|
||||
name: "a categorized denial still counts",
|
||||
result: ToolResult{Status: tools.StatusError, Output: "denied", DenialReason: DenialFiltered},
|
||||
},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
if !isPolicyRefusal(testCase.result) {
|
||||
t.Fatal("refusal was treated as a success, so the guard's record is cleared and the call can repeat to MaxTurns")
|
||||
}
|
||||
if isRetriableToolError(testCase.result) {
|
||||
t.Error("a policy refusal is not retriable: repeating it verbatim cannot help")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// An ordinary failure is still retriable and still not a refusal, or the guard
|
||||
// would halt on things the model can legitimately fix by trying again.
|
||||
func TestOrdinaryFailuresAreNotPolicyRefusals(t *testing.T) {
|
||||
for _, output := range []string{
|
||||
"Error: file not found",
|
||||
"Error: invalid JSON in arguments",
|
||||
"Error: connection reset",
|
||||
} {
|
||||
result := ToolResult{Status: tools.StatusError, Output: output}
|
||||
if isPolicyRefusal(result) {
|
||||
t.Errorf("%q was classified as a policy refusal", output)
|
||||
}
|
||||
if !isRetriableToolError(result) {
|
||||
t.Errorf("%q should stay retriable", output)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// PRODUCING A RESULT AND ASKING THE RUN TO STOP ARE DIFFERENT FACTS.
|
||||
//
|
||||
// closeOutRemaining decides between finalizing a completed sibling and writing
|
||||
// an aborted placeholder, and it used to make that decision from abortErr. A
|
||||
// cancelled permission request is exactly where the two facts disagree:
|
||||
// executeToolCall builds the cancellation result, with its call ID, its message
|
||||
// and its denial category, and returns it TOGETHER with
|
||||
// ErrPermissionApprovalCanceled. Reading the error as "never started" discards a
|
||||
// real result and records the opposite of what happened, and the cancellation
|
||||
// then goes missing from OnToolResult, the trace counters and the task
|
||||
// observation even though the permission event already fired.
|
||||
func TestABatchEntryWithAResultAndAnAbortErrorIsDrained(t *testing.T) {
|
||||
canceled := canceledPermissionResult(
|
||||
ToolCall{ID: "call-b", Name: "read_probe"},
|
||||
"cancelled in TUI",
|
||||
PermissionEvent{ToolName: "read_probe"},
|
||||
)
|
||||
precomputed := []precomputedToolResult{
|
||||
{result: ToolResult{ToolCallID: "call-a", Name: "read_probe", Status: tools.StatusOK}, completed: true},
|
||||
{
|
||||
result: canceled,
|
||||
abortErr: fmt.Errorf("%w for read_probe", ErrPermissionApprovalCanceled),
|
||||
completed: true,
|
||||
},
|
||||
}
|
||||
|
||||
sibling, ran := precomputedResultFor(precomputed, 0, 2, 1)
|
||||
if !ran {
|
||||
t.Fatal("a cancelled permission result was treated as a call that never started")
|
||||
}
|
||||
if sibling.ToolCallID != "call-b" {
|
||||
t.Errorf("ToolCallID = %q, want the cancellation result rather than an empty placeholder", sibling.ToolCallID)
|
||||
}
|
||||
if sibling.DenialReason != DenialApprovalCanceled {
|
||||
t.Errorf("DenialReason = %q, want the cancellation preserved", sibling.DenialReason)
|
||||
}
|
||||
}
|
||||
|
||||
// An entry that never produced a result is still unstarted, or the fix would
|
||||
// have stopped aborting anything.
|
||||
func TestABatchEntryWithNoResultIsStillUnstarted(t *testing.T) {
|
||||
precomputed := []precomputedToolResult{
|
||||
{abortErr: errors.New("cancelled before the tool ran")},
|
||||
}
|
||||
if _, ran := precomputedResultFor(precomputed, 0, 1, 0); ran {
|
||||
t.Error("an entry that produced nothing was finalized as if it had run")
|
||||
}
|
||||
}
|
||||
|
||||
// A call outside the batch window never started either.
|
||||
func TestACallOutsideTheBatchWindowIsUnstarted(t *testing.T) {
|
||||
precomputed := []precomputedToolResult{{result: ToolResult{ToolCallID: "call-a"}, completed: true}}
|
||||
if _, ran := precomputedResultFor(precomputed, 0, 1, 1); ran {
|
||||
t.Error("a call past the batch window was reported as executed")
|
||||
}
|
||||
}
|
||||
|
||||
// And the flag is recorded where the truth is known rather than inferred later,
|
||||
// so a real batch marks what it ran.
|
||||
func TestTheBatchRecordsWhatItRan(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(&readAheadTool{})
|
||||
calls := []ToolCall{
|
||||
{ID: "call-a", Name: "read_probe", Arguments: `{"id":"a"}`},
|
||||
{ID: "call-b", Name: "read_probe", Arguments: `{"id":"b"}`},
|
||||
}
|
||||
results := executeParallelReadBatch(context.Background(), registry, calls, 0, 2, PermissionModeAuto, Options{Registry: registry})
|
||||
for index, entry := range results {
|
||||
if !entry.completed {
|
||||
t.Errorf("entry %d executed but was not recorded as completed: %#v", index, entry)
|
||||
}
|
||||
if entry.result.ToolCallID != calls[index].ID {
|
||||
t.Errorf("entry %d is keyed to %q, want %q", index, entry.result.ToolCallID, calls[index].ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// legacyDenialSignaturePrefix is the string provenance used to be encoded in.
|
||||
// The identity was one namespace holding both a normalized error signature and
|
||||
// a synthetic "denial:<category>" key, and provenance was recovered afterwards
|
||||
// with strings.HasPrefix. It survives here, and only here, so the regression
|
||||
// names the exact spelling that used to collide.
|
||||
const legacyDenialSignaturePrefix = "denial:"
|
||||
|
||||
func runUntilGuardHalt(t *testing.T, gate bool) Result {
|
||||
t.Helper()
|
||||
tool := &uncategorizedSandboxTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
|
||||
for i := range toolFailureAnyErrorStopAt + 4 {
|
||||
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "write_file", `{"path":"/o-`+strconv.Itoa(i)+`.txt"}`))
|
||||
}
|
||||
result, err := Run(context.Background(), "write", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
RequireCompletionSignal: gate,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
|
||||
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// A GUARD HALT IS UNFINISHED WORK, AND HEADLESS HAS TO HEAR ABOUT IT.
|
||||
//
|
||||
// The halt returns straight out of the tool loop, so it never reached the
|
||||
// completion gate the max-turns paths go through. zero exec treats only
|
||||
// Result.Incomplete as exit 4 and otherwise reports run_end success with exit
|
||||
// 0, so a task denied six times, or failed through the twelve-call bound, came
|
||||
// back as a successful automation result having done none of the work asked.
|
||||
func TestGuardHaltIsIncompleteUnderTheCompletionGate(t *testing.T) {
|
||||
headless := runUntilGuardHalt(t, true)
|
||||
if !headless.Incomplete {
|
||||
t.Error("a guard halt reports the run as complete; zero exec would call a run that did nothing a success")
|
||||
}
|
||||
if !strings.Contains(headless.IncompleteReason, "write_file") {
|
||||
t.Errorf("the incomplete reason does not name the tool that halted the run: %q", headless.IncompleteReason)
|
||||
}
|
||||
// And the interactive default is untouched: Incomplete exists for the
|
||||
// headless gate, so setting it without one would be inventing a status.
|
||||
if interactive := runUntilGuardHalt(t, false); interactive.Incomplete {
|
||||
t.Error("Incomplete was set without RequireCompletionSignal, changing interactive behaviour")
|
||||
}
|
||||
}
|
||||
|
||||
// executedDenialLookalikeTool runs, fails, and prints exactly the string the
|
||||
// guard used to store for a real permission refusal.
|
||||
type executedDenialLookalikeTool struct{ ran int }
|
||||
|
||||
func (t *executedDenialLookalikeTool) Name() string { return "bash" }
|
||||
func (t *executedDenialLookalikeTool) Description() string { return "probe" }
|
||||
func (t *executedDenialLookalikeTool) Parameters() tools.Schema { return tools.Schema{Type: "object"} }
|
||||
func (t *executedDenialLookalikeTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionAllow}
|
||||
}
|
||||
|
||||
func (t *executedDenialLookalikeTool) Run(context.Context, map[string]any) tools.Result {
|
||||
t.ran++
|
||||
return tools.Result{Status: tools.StatusError, Output: legacyDenialSignaturePrefix + string(DenialPermissionDenied)}
|
||||
}
|
||||
|
||||
// TRUSTED PROVENANCE AND UNTRUSTED CONTENT MUST NOT SHARE A NAMESPACE.
|
||||
//
|
||||
// While the identity was a single string, a command that printed exactly
|
||||
// "denial:permission_denied" and exited non-zero took on the identity of a real
|
||||
// permission refusal: it merged streaks with one, and the run told the user the
|
||||
// tool had been refused when it had executed every time.
|
||||
func TestAnExecutedErrorCannotImpersonateARefusal(t *testing.T) {
|
||||
tool := &executedDenialLookalikeTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
|
||||
for i := range toolFailureStopAt + 4 {
|
||||
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "bash", `{}`))
|
||||
}
|
||||
result, err := Run(context.Background(), "run it", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry, PermissionMode: PermissionModeAsk, MaxTurns: len(turns) + 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran == 0 {
|
||||
t.Fatal("the tool never ran, so this no longer covers an executed failure")
|
||||
}
|
||||
if strings.Contains(result.FinalAnswer, "was refused") {
|
||||
t.Errorf("the tool executed %d times and nothing refused it, but the run says it was refused: %q", tool.ran, result.FinalAnswer)
|
||||
}
|
||||
if !strings.Contains(result.FinalAnswer, "with the same error") {
|
||||
t.Errorf("an executed failure repeating identically should read as the same error: %q", result.FinalAnswer)
|
||||
}
|
||||
}
|
||||
|
||||
// AND THE AGGREGATE IS ITS OWN FACT.
|
||||
//
|
||||
// The content-blind bound fires precisely when no single identity repeated
|
||||
// enough, so the identity present at the end says nothing about the eleven
|
||||
// before it. Alternating two refusal categories reaches twelve without either
|
||||
// reaching six; the tool never ran once, and the answer used to describe that
|
||||
// as the tool failing with varying errors.
|
||||
func TestTheContentBlindBoundKeepsRefusalProvenance(t *testing.T) {
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
kinds []DenialCategory
|
||||
wantCause toolFailureCause
|
||||
wantAnswer string
|
||||
}{
|
||||
{
|
||||
name: "alternating refusal categories",
|
||||
kinds: []DenialCategory{DenialPermissionDenied, DenialSandboxBlock},
|
||||
wantCause: toolFailureCauseVariedRefused,
|
||||
wantAnswer: "was refused",
|
||||
},
|
||||
{
|
||||
// Mixed is DECIDED, not inherited from whichever field a switch
|
||||
// tested first. Neither "failed" alone nor "refused" alone is true of
|
||||
// this sequence, so the wording says both.
|
||||
name: "executed failures mixed with refusals",
|
||||
kinds: []DenialCategory{DenialNone, DenialPermissionDenied},
|
||||
wantCause: toolFailureCauseVariedMixed,
|
||||
wantAnswer: "failed or was refused",
|
||||
},
|
||||
{
|
||||
name: "executed failures only",
|
||||
kinds: []DenialCategory{DenialNone, DenialNone},
|
||||
wantCause: toolFailureCauseVariedExecuted,
|
||||
wantAnswer: "with varying errors",
|
||||
},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
state := newGuardState()
|
||||
var outcome toolFailureOutcome
|
||||
for index := 0; index < toolFailureAnyErrorStopAt; index++ {
|
||||
// Distinct prose every call, exactly as a real refusal or error
|
||||
// naming what it touched would produce.
|
||||
outcome = state.observeToolResult("bash", true, false,
|
||||
"failure on item "+strconv.Itoa(index), testCase.kinds[index%len(testCase.kinds)])
|
||||
if outcome.Stop {
|
||||
break
|
||||
}
|
||||
}
|
||||
if !outcome.Stop {
|
||||
t.Fatal("twelve consecutive failures did not trip the content-blind bound")
|
||||
}
|
||||
if outcome.Count != toolFailureAnyErrorStopAt {
|
||||
t.Errorf("Count = %d, want the counter that tripped (%d)", outcome.Count, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
if outcome.Cause != testCase.wantCause {
|
||||
t.Errorf("Cause = %v, want %v", outcome.Cause, testCase.wantCause)
|
||||
}
|
||||
answer := toolFailureStopAnswer("bash", outcome.Count, outcome.Cause)
|
||||
if !strings.Contains(answer, testCase.wantAnswer) {
|
||||
t.Errorf("stop answer = %q, want it to contain %q", answer, testCase.wantAnswer)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// deniedWriteTool records every execution. Its body must never run: the sandbox
|
||||
// refuses the call at the registry boundary, before the tool is reached.
|
||||
type deniedWriteTool struct{ ran int }
|
||||
|
||||
func (t *deniedWriteTool) Name() string { return "write_file" }
|
||||
func (t *deniedWriteTool) Description() string { return "test write tool" }
|
||||
func (t *deniedWriteTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"path": {Type: "string"}},
|
||||
Required: []string{"path"},
|
||||
}
|
||||
}
|
||||
|
||||
func (t *deniedWriteTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectWrite, Permission: tools.PermissionAllow, Reason: "writes files"}
|
||||
}
|
||||
|
||||
func (t *deniedWriteTool) Run(context.Context, map[string]any) tools.Result {
|
||||
t.ran++
|
||||
return tools.Result{Status: tools.StatusOK, Output: "wrote"}
|
||||
}
|
||||
|
||||
// THROUGH THE REAL SANDBOX, NOT A FAKE STANDING WHERE IT WOULD BE.
|
||||
//
|
||||
// The neighbouring uncategorized test deliberately hand-builds the shape a
|
||||
// refusal has when no category survives, which is a fact about the LOOP. This
|
||||
// one covers the producer: a real sandbox.Engine evaluation denies the call,
|
||||
// the registry turns that into a categorized refusal, and the guard keys on the
|
||||
// category. Without it, reverting refusalResult(..., PolicyRefusalSandboxDenied)
|
||||
// in registry.go to a plain errorResult left both the agent and tools suites
|
||||
// green, so nothing depended on the marker at all.
|
||||
//
|
||||
// Every assertion here fails if a different link breaks: ran proves the body was
|
||||
// skipped, the request count proves the CATEGORY bound halted it rather than the
|
||||
// content-blind one, and the wording proves the provenance survived conversion.
|
||||
func TestRunHaltsARealSandboxDenialOnTheCategoryBound(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
forbidden := t.TempDir()
|
||||
policy := sandbox.DefaultPolicy()
|
||||
// Applies whenever the sandbox is enforcing, so this does not depend on the
|
||||
// workspace boundary being reachable from a synthetic request.
|
||||
policy.DenyWrite = []string{forbidden}
|
||||
|
||||
tool := &deniedWriteTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// A different path every call. Note that errorSignature truncates at 80
|
||||
// characters and the message opens with a fixed 40-character prefix followed
|
||||
// by a long temp root, so the signatures collide here anyway: the request
|
||||
// count below pins WHICH bound halted the run, but it does not by itself
|
||||
// prove the streak was held by the category. The wording assertion is the one
|
||||
// that discriminates, and it is the one that fails when the marker is gone.
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
|
||||
for i := range toolFailureStopAt + 4 {
|
||||
target := filepath.Join(forbidden, "escape-"+strconv.Itoa(i)+".txt")
|
||||
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "write_file", `{"path":`+strconv.Quote(target)+`}`))
|
||||
}
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
result, err := Run(context.Background(), "write the files", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
Cwd: workspace,
|
||||
Sandbox: sandbox.NewEngine(sandbox.EngineOptions{
|
||||
WorkspaceRoot: workspace,
|
||||
Policy: policy,
|
||||
}),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if tool.ran != 0 {
|
||||
t.Errorf("the tool body ran %d times; the sandbox must refuse before execution", tool.ran)
|
||||
}
|
||||
if len(provider.requests) != toolFailureStopAt {
|
||||
t.Errorf("the run made %d turns, want the category bound at %d; a categorized refusal whose prose varies must still halt on its category",
|
||||
len(provider.requests), toolFailureStopAt)
|
||||
}
|
||||
if want := toolFailureStopAnswer("write_file", toolFailureStopAt, toolFailureCauseSameRefusal); result.FinalAnswer != want {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
|
||||
}
|
||||
if !strings.Contains(result.FinalAnswer, "was refused") {
|
||||
t.Errorf("the refusal provenance did not survive to the answer: %q", result.FinalAnswer)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// THE ANSWER MUST CLAIM ONLY WHAT THE COUNTER ESTABLISHED.
|
||||
//
|
||||
// anyErrorCount establishes that the tool failed consecutively without a
|
||||
// success. It does NOT establish that every failure differed: five A, five B and
|
||||
// two C reach the content-blind bound of 12 without any signature repeating six
|
||||
// times, and three of those errors were shared. The old wording called that
|
||||
// "each with a different error".
|
||||
//
|
||||
// The signature bound has the inverse problem. A denial streak keys on the
|
||||
// CATEGORY, and the prose behind it embeds the path or command refused, so it
|
||||
// differs on every call. Calling that "the same error" is false in the other
|
||||
// direction.
|
||||
func TestStopAnswerDoesNotOverclaimTheFailurePattern(t *testing.T) {
|
||||
varied := toolFailureStopAnswer("bash", 12, toolFailureCauseVariedExecuted)
|
||||
if strings.Contains(varied, "each with a different error") {
|
||||
t.Errorf("the content-blind bound does not track that every failure differed: %q", varied)
|
||||
}
|
||||
if !strings.Contains(varied, "varying errors") {
|
||||
t.Errorf("the varied stop should still say the errors varied: %q", varied)
|
||||
}
|
||||
|
||||
refused := toolFailureStopAnswer("bash", 6, toolFailureCauseSameRefusal)
|
||||
if strings.Contains(refused, "same error") {
|
||||
t.Errorf("a denial streak covers refusals of different paths, so it is not the same error: %q", refused)
|
||||
}
|
||||
if !strings.Contains(refused, "refused") {
|
||||
t.Errorf("a denial streak should say it was refused: %q", refused)
|
||||
}
|
||||
|
||||
// The one claim that IS justified: a signature streak really did repeat the
|
||||
// same error signature, so that wording stays.
|
||||
same := toolFailureStopAnswer("bash", 6, toolFailureCauseSameError)
|
||||
if !strings.Contains(same, "same error") {
|
||||
t.Errorf("a signature streak may still be described as the same error: %q", same)
|
||||
}
|
||||
}
|
||||
|
||||
// The mixed-signature case jatmn asked for, driven through the real counter
|
||||
// rather than asserted about the wording in isolation: a run whose errors vary
|
||||
// must trip the content-blind bound and must not be described as all-different.
|
||||
func TestMixedSignatureStreakTripsTheContentBlindBound(t *testing.T) {
|
||||
var state guardState
|
||||
var outcome toolFailureOutcome
|
||||
// Five of one error, five of another, two of a third: twelve failures, no
|
||||
// signature repeating six times in a row.
|
||||
for index, output := range []string{
|
||||
"Error: A", "Error: A", "Error: A", "Error: A", "Error: A",
|
||||
"Error: B", "Error: B", "Error: B", "Error: B", "Error: B",
|
||||
"Error: C", "Error: C",
|
||||
} {
|
||||
outcome = state.observeToolResult("bash", true, true, output, DenialNone)
|
||||
if outcome.Stop && index < 11 {
|
||||
t.Fatalf("halted early at failure %d", index+1)
|
||||
}
|
||||
}
|
||||
if !outcome.Stop {
|
||||
t.Fatal("twelve consecutive failures did not trip the content-blind bound")
|
||||
}
|
||||
if outcome.Cause != toolFailureCauseVariedExecuted {
|
||||
t.Errorf("a mixed-signature streak of executed failures should report varied-executed, got %v", outcome.Cause)
|
||||
}
|
||||
answer := toolFailureStopAnswer("bash", outcome.Count, outcome.Cause)
|
||||
if strings.Contains(answer, "each with a different error") {
|
||||
t.Errorf("three of these errors were shared, so they were not each different: %q", answer)
|
||||
}
|
||||
}
|
||||
@@ -1,14 +1,20 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/testutil"
|
||||
)
|
||||
|
||||
// isolatedUserDirsRoot is where TestMain points every per-user directory.
|
||||
var isolatedUserDirsRoot string
|
||||
|
||||
// TestMain points userConfigDirForPrompt at an empty, package-wide temp
|
||||
// directory for every test in this package. Without this, buildSystemPrompt
|
||||
// falls back to the real config.UserConfigDir, so any developer with a
|
||||
@@ -17,17 +23,56 @@ import (
|
||||
// across contributor machines. Tests that specifically exercise user
|
||||
// guidelines stub userConfigDirForPrompt themselves via
|
||||
// withSystemPromptTestUserConfigDir(Func) and restore this default on cleanup.
|
||||
//
|
||||
// It also points every per-user directory root at the same temp directory:
|
||||
// sandboxed tool runs keep their runtime state and lease files under the user
|
||||
// cache dir, so without this they land in the developer's real cache.
|
||||
func TestMain(m *testing.M) {
|
||||
dir, err := os.MkdirTemp("", "zero-agent-tests-*")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
isolatedUserDirsRoot = dir
|
||||
restore, err := testutil.IsolateUserDirs(dir)
|
||||
if err != nil {
|
||||
os.RemoveAll(dir)
|
||||
panic(err)
|
||||
}
|
||||
userConfigDirForPrompt = func() (string, error) { return dir, nil }
|
||||
code := m.Run()
|
||||
if err := restore(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "agent tests: restore user dirs:", err)
|
||||
code = 1
|
||||
}
|
||||
os.RemoveAll(dir)
|
||||
os.Exit(code)
|
||||
}
|
||||
|
||||
// Guards TestMain: the per-user directories the package can read or write
|
||||
// must resolve inside the isolated root on every OS.
|
||||
func TestUserDirsAreIsolatedFromTheDeveloperHome(t *testing.T) {
|
||||
if isolatedUserDirsRoot == "" {
|
||||
t.Fatal("TestMain did not record an isolated root")
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
userConfig, err := config.UserConfigDir()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cache, err := os.UserCacheDir()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for name, dir := range map[string]string{"home": home, "config": userConfig, "cache": cache} {
|
||||
if !testutil.PathWithin(isolatedUserDirsRoot, dir) {
|
||||
t.Errorf("%s dir %q is outside the isolated root %q", name, dir, isolatedUserDirsRoot)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCoreSystemPromptIncludesCodingQualityRules(t *testing.T) {
|
||||
prompt := strings.Join(strings.Fields(strings.ToLower(buildSystemPrompt(Options{}))), " ")
|
||||
if strings.Contains(prompt, "clickable file paths") {
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
package aimlapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
// SourceHeaderName and SourceValue identify Zero as the calling agent on
|
||||
// requests to aimlapi.com services (the Toolbox and the MCP server).
|
||||
SourceHeaderName = "X-AIMLAPI-Source"
|
||||
SourceValue = "agent/zero"
|
||||
|
||||
// CanonicalInferenceBaseURL is the catalog inference endpoint. A provider
|
||||
// profile whose base URL differs is treated as a proxy: its key is never sent
|
||||
// to other aimlapi.com services.
|
||||
CanonicalInferenceBaseURL = "https://api.aimlapi.com/v1"
|
||||
// ToolboxBaseURL is the fixed AI/ML API Toolbox REST base. It is not
|
||||
// configurable, so the key and attribution can only reach tools.aimlapi.com.
|
||||
ToolboxBaseURL = "https://tools.aimlapi.com/v1/tools"
|
||||
// MCPServerURL is the AI/ML API MCP server (streamable HTTP).
|
||||
MCPServerURL = "https://mcp.aimlapi.com/mcp"
|
||||
)
|
||||
|
||||
// attributedHosts are the only hostnames that receive the AI/ML API key or the
|
||||
// attribution headers. Matching is exact; a suffix or substring match would
|
||||
// also accept hosts such as tools.aimlapi.com.example.net.
|
||||
var attributedHosts = [...]string{"api.aimlapi.com", "tools.aimlapi.com", "mcp.aimlapi.com"}
|
||||
|
||||
// IsAttributedURL reports whether target is an https URL on one of the
|
||||
// aimlapi.com service hosts, on the default port and without userinfo.
|
||||
func IsAttributedURL(target *url.URL) bool {
|
||||
if target == nil || !strings.EqualFold(target.Scheme, "https") || target.User != nil {
|
||||
return false
|
||||
}
|
||||
if port := target.Port(); port != "" && port != "443" {
|
||||
return false
|
||||
}
|
||||
host := strings.ToLower(target.Hostname())
|
||||
for _, candidate := range attributedHosts {
|
||||
if host == candidate {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ApplyAttributionHeaders sets the source and partner headers on header only
|
||||
// when target is an aimlapi.com service URL; any other target is left as is.
|
||||
func ApplyAttributionHeaders(header http.Header, target *url.URL) {
|
||||
if header == nil || !IsAttributedURL(target) {
|
||||
return
|
||||
}
|
||||
header.Set(SourceHeaderName, SourceValue)
|
||||
header.Set(PartnerHeaderName, ResolvePartnerID(""))
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package aimlapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIsAttributedURLMatchesExactServiceHostsOnly(t *testing.T) {
|
||||
cases := map[string]bool{
|
||||
"https://tools.aimlapi.com/v1/tools/run": true,
|
||||
"https://mcp.aimlapi.com/mcp": true,
|
||||
"https://api.aimlapi.com/v1": true,
|
||||
"https://TOOLS.aimlapi.com:443/v1/tools": true,
|
||||
"http://tools.aimlapi.com/v1/tools/run": false,
|
||||
"https://tools.aimlapi.com:8443/v1/tools": false,
|
||||
"https://[email protected]/v1/tools": false,
|
||||
"https://tools.aimlapi.com.example.net/run": false,
|
||||
"https://eviltools.aimlapi.com/run": false,
|
||||
"https://aimlapi.com/app": false,
|
||||
"https://auth.aimlapi.com/": false,
|
||||
"https://example.com/?h=tools.aimlapi.com": false,
|
||||
}
|
||||
for raw, want := range cases {
|
||||
target, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("parse %q: %v", raw, err)
|
||||
}
|
||||
if got := IsAttributedURL(target); got != want {
|
||||
t.Errorf("IsAttributedURL(%q) = %v, want %v", raw, got, want)
|
||||
}
|
||||
}
|
||||
if IsAttributedURL(nil) {
|
||||
t.Fatal("IsAttributedURL(nil) = true")
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyAttributionHeadersSetsSourceAndPartnerOnServiceHosts(t *testing.T) {
|
||||
t.Setenv("AIMLAPI_PARTNER_ID", "")
|
||||
header := http.Header{}
|
||||
ApplyAttributionHeaders(header, &url.URL{Scheme: "https", Host: "tools.aimlapi.com", Path: "/v1/tools/run"})
|
||||
if got := header.Get(SourceHeaderName); got != "agent/zero" {
|
||||
t.Fatalf("%s = %q, want agent/zero", SourceHeaderName, got)
|
||||
}
|
||||
if got := header.Get(PartnerHeaderName); got != DefaultPartnerID {
|
||||
t.Fatalf("%s = %q, want %q", PartnerHeaderName, got, DefaultPartnerID)
|
||||
}
|
||||
|
||||
other := http.Header{}
|
||||
ApplyAttributionHeaders(other, &url.URL{Scheme: "https", Host: "tools.aimlapi.com.example.net"})
|
||||
if len(other) != 0 {
|
||||
t.Fatalf("attribution reached a non-aimlapi host: %#v", other)
|
||||
}
|
||||
}
|
||||
+25
-2
@@ -964,6 +964,7 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
|
||||
// report can be combined with the plugin activation's, and emit at most one
|
||||
// notice when project hooks/plugins were dropped for an untrusted workspace.
|
||||
hookDispatcher, hookSkip := newHookDispatcherWithExtra(workspaceRoot, pluginActivation.hooks, trustRoot, executionRunner)
|
||||
specialistRuntime.attachLifecycleHooks(hookDispatcher)
|
||||
emitTrustNotice(stderr, hookSkip, pluginActivation.trustSkip, mcpSkip)
|
||||
// From the engine the session's commands run through, so the notice
|
||||
// describes the sandbox they actually get.
|
||||
@@ -1221,6 +1222,9 @@ type agentToolRuntime struct {
|
||||
// the orchestrator's system-prompt delegation section. Populated alongside the
|
||||
// tools, so it is present exactly when the Task tool is.
|
||||
specialists []agent.SpecialistInfo
|
||||
// lifecycleHooks is shared with the specialist executor so the hooks
|
||||
// dispatcher (built later in startup) can be attached before any specialist runs.
|
||||
lifecycleHooks *specialist.LifecycleHooks
|
||||
}
|
||||
|
||||
// specialistInfos returns the runtime's specialist summaries, nil-safe so the
|
||||
@@ -1232,12 +1236,26 @@ func (r *agentToolRuntime) specialistInfos() []agent.SpecialistInfo {
|
||||
return r.specialists
|
||||
}
|
||||
|
||||
func (r *agentToolRuntime) attachLifecycleHooks(dispatcher *hooks.Dispatcher) {
|
||||
if r == nil || r.lifecycleHooks == nil || dispatcher == nil {
|
||||
return
|
||||
}
|
||||
r.lifecycleHooks.Dispatch = func(ctx context.Context, event string, specialistName string, payload map[string]any) {
|
||||
dispatcher.Dispatch(ctx, hooks.DispatchInput{
|
||||
Event: hooks.Event(event),
|
||||
ToolName: specialistName,
|
||||
Payload: payload,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func registerSpecialistTools(registry *tools.Registry, workspaceRoot string, maxTeamSize int) (*agentToolRuntime, error) {
|
||||
paths, err := specialist.DefaultPaths(workspaceRoot)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
executor := specialist.Executor{Paths: paths}
|
||||
lifecycleHooks := &specialist.LifecycleHooks{}
|
||||
executor := specialist.Executor{Paths: paths, LifecycleHooks: lifecycleHooks}
|
||||
runtime, err := specialist.RegisterTools(registry, executor)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -1256,7 +1274,12 @@ func registerSpecialistTools(registry *tools.Registry, workspaceRoot string, max
|
||||
return nil, err
|
||||
}
|
||||
swarm.RegisterTools(registry, sw)
|
||||
return &agentToolRuntime{specialist: runtime, swarm: sw, specialists: specialistSummaries(paths)}, nil
|
||||
return &agentToolRuntime{
|
||||
specialist: runtime,
|
||||
swarm: sw,
|
||||
specialists: specialistSummaries(paths),
|
||||
lifecycleHooks: lifecycleHooks,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// specialistSummaries loads the available specialists (built-ins + user/project
|
||||
|
||||
@@ -655,6 +655,7 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
|
||||
// can be combined with the plugin activation's, and emit at most one notice when
|
||||
// project hooks/plugins were dropped for an untrusted workspace.
|
||||
hookDispatcher, hookSkip := newHookDispatcherWithExtra(workspaceRoot, pluginActivation.hooks, trustRoot, executionRunner)
|
||||
specialistRuntime.attachLifecycleHooks(hookDispatcher)
|
||||
emitTrustNotice(stderr, hookSkip, pluginActivation.trustSkip, mcpSkip)
|
||||
result, err := agent.Run(runCtx, agentPrompt, provider, agent.Options{
|
||||
MaxTurns: resolved.MaxTurns,
|
||||
@@ -798,7 +799,7 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
|
||||
}
|
||||
return exitInterrupted
|
||||
}
|
||||
sessionRecorder.append(sessions.EventError, map[string]any{"message": err.Error()})
|
||||
sessionRecorder.append(sessions.EventError, sessions.ErrorEventPayload(err))
|
||||
if options.outputFormat == execOutputStreamJSON {
|
||||
writer.errorEvent("provider_error", err.Error(), false)
|
||||
writer.runEnd("error", exitProvider)
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// streamErrorExecProvider fails every request with one provider error event.
|
||||
type streamErrorExecProvider struct {
|
||||
event zeroruntime.StreamEvent
|
||||
}
|
||||
|
||||
func (provider streamErrorExecProvider) StreamCompletion(ctx context.Context, _ zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
|
||||
ch := make(chan zeroruntime.StreamEvent, 1)
|
||||
ch <- provider.event
|
||||
close(ch)
|
||||
return ch, nil
|
||||
}
|
||||
|
||||
func providerErrorExecDeps(t *testing.T, store *sessions.Store, workspaceRoot string) appDeps {
|
||||
t.Helper()
|
||||
return appDeps{
|
||||
getwd: func() (string, error) { return workspaceRoot, nil },
|
||||
resolveConfig: func(_ string, _ config.Overrides) (config.ResolvedConfig, error) {
|
||||
return config.ResolvedConfig{
|
||||
ActiveProvider: "test",
|
||||
Provider: config.ProviderProfile{
|
||||
Name: "test",
|
||||
ProviderKind: config.ProviderKindOpenAICompatible,
|
||||
BaseURL: "http://127.0.0.1/v1",
|
||||
Model: "test-model",
|
||||
},
|
||||
MaxTurns: 3,
|
||||
}, nil
|
||||
},
|
||||
resolveMCPConfig: func(string, bool) (config.MCPConfig, error) { return config.MCPConfig{}, nil },
|
||||
newProvider: func(config.ProviderProfile) (zeroruntime.Provider, error) {
|
||||
return streamErrorExecProvider{event: zeroruntime.StreamEvent{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: "provider request error: model not found",
|
||||
StatusCode: 404,
|
||||
Cause: "upstream said: no such model",
|
||||
}}, nil
|
||||
},
|
||||
newSessionStore: func() *sessions.Store { return store },
|
||||
newSandboxStore: func() (*sandbox.GrantStore, error) {
|
||||
return sandbox.NewGrantStore(sandbox.StoreOptions{FilePath: t.TempDir() + "/grants.json"})
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// recordedProviderError returns the payload of the one error event the run
|
||||
// recorded, failing unless there is exactly one session with exactly one.
|
||||
func recordedProviderError(t *testing.T, store *sessions.Store) map[string]any {
|
||||
t.Helper()
|
||||
items, err := store.List()
|
||||
if err != nil || len(items) != 1 {
|
||||
t.Fatalf("sessions = %d, %v; want one", len(items), err)
|
||||
}
|
||||
events, err := store.ReadEvents(items[0].SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
var found []map[string]any
|
||||
for _, event := range events {
|
||||
if event.Type != sessions.EventError {
|
||||
continue
|
||||
}
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(event.Payload, &payload); err != nil {
|
||||
t.Fatalf("unmarshal error payload: %v", err)
|
||||
}
|
||||
found = append(found, payload)
|
||||
}
|
||||
if len(found) != 1 {
|
||||
t.Fatalf("recorded %d error events, want 1: %#v", len(found), found)
|
||||
}
|
||||
return found[0]
|
||||
}
|
||||
|
||||
func assertProviderErrorDetail(t *testing.T, payload map[string]any) {
|
||||
t.Helper()
|
||||
if got, want := payload["message"], "provider request error: model not found"; got != want {
|
||||
t.Errorf("message = %#v, want %q", got, want)
|
||||
}
|
||||
if got, want := payload["statusCode"], float64(404); got != want {
|
||||
t.Errorf("statusCode = %#v, want %v: the run recorded only the flattened message", got, want)
|
||||
}
|
||||
if got, want := payload["cause"], "upstream said: no such model"; got != want {
|
||||
t.Errorf("cause = %#v, want %q: the run recorded only the flattened message", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A provider error reaches the exec session log with its HTTP status and the
|
||||
// upstream cause, not only the flattened message (#674).
|
||||
func TestRunExecRecordsTheProviderErrorStatusAndCause(t *testing.T) {
|
||||
isolateCLIUserState(t)
|
||||
workspaceRoot := t.TempDir()
|
||||
// exec records through the default store, which isolateCLIUserState has
|
||||
// moved under the test's temp root; this opens the same one.
|
||||
store := sessions.NewStore(sessions.StoreOptions{})
|
||||
var stdout, stderr bytes.Buffer
|
||||
|
||||
exitCode := runWithDeps([]string{"exec", "--cwd", workspaceRoot, "--init-session-id", "provider_error_run", "hello"}, &stdout, &stderr,
|
||||
providerErrorExecDeps(t, store, workspaceRoot))
|
||||
|
||||
if exitCode != exitProvider {
|
||||
t.Fatalf("exit = %d, want %d; stderr=%s", exitCode, exitProvider, stderr.String())
|
||||
}
|
||||
assertProviderErrorDetail(t, recordedProviderError(t, store))
|
||||
}
|
||||
|
||||
// The spec draft run records its provider error through its own call site, so
|
||||
// it is checked on its own.
|
||||
func TestRunExecUseSpecRecordsTheProviderErrorStatusAndCause(t *testing.T) {
|
||||
isolateCLIUserState(t)
|
||||
workspaceRoot := t.TempDir()
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
var stdout, stderr bytes.Buffer
|
||||
|
||||
exitCode := runWithDeps([]string{"exec", "--cwd", workspaceRoot, "--use-spec", "-o", "json", "plan the review flow"}, &stdout, &stderr,
|
||||
providerErrorExecDeps(t, store, workspaceRoot))
|
||||
|
||||
if exitCode != exitProvider {
|
||||
t.Fatalf("exit = %d, want %d; stderr=%s stdout=%s", exitCode, exitProvider, stderr.String(), stdout.String())
|
||||
}
|
||||
assertProviderErrorDetail(t, recordedProviderError(t, store))
|
||||
}
|
||||
@@ -201,7 +201,7 @@ func runExecSpecDraft(run execSpecDraftRun) int {
|
||||
}
|
||||
return exitInterrupted
|
||||
}
|
||||
sessionRecorder.append(sessions.EventError, map[string]any{"message": err.Error()})
|
||||
sessionRecorder.append(sessions.EventError, sessions.ErrorEventPayload(err))
|
||||
return writeExecSpecDraftProviderError(&writer, run.options.outputFormat, err.Error())
|
||||
}
|
||||
if result.StopReason != agent.StopReasonSpecReviewRequired || draftInfo.SpecID == "" || draftInfo.SpecFilePath == "" {
|
||||
|
||||
@@ -2,6 +2,7 @@ package cli
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
@@ -82,11 +83,105 @@ func runPlugins(args []string, stdout io.Writer, stderr io.Writer, deps appDeps)
|
||||
return runPluginInfo(args[1:], stdout, stderr, deps)
|
||||
case "remove", "rm":
|
||||
return runPluginRemove(args[1:], deps.pluginsDir(), stdout, stderr)
|
||||
case "enable":
|
||||
return runPluginsToggle(args[1:], stdout, stderr, deps, false)
|
||||
case "disable":
|
||||
return runPluginsToggle(args[1:], stdout, stderr, deps, true)
|
||||
default:
|
||||
return writeExecUsageError(stderr, fmt.Sprintf("unknown plugins subcommand %q", args[0]))
|
||||
}
|
||||
}
|
||||
|
||||
type pluginToggleOptions struct {
|
||||
json bool
|
||||
user bool
|
||||
}
|
||||
|
||||
// runPluginsToggle implements `zero plugins enable|disable <id>`.
|
||||
//
|
||||
// Every message it writes, including usage errors that echo the caller's
|
||||
// arguments, passes through the redaction pipeline: an argument is user input
|
||||
// and can be a mistyped credential.
|
||||
func runPluginsToggle(args []string, stdout io.Writer, stderr io.Writer, deps appDeps, disabled bool) int {
|
||||
commandName := "enable"
|
||||
if disabled {
|
||||
commandName = "disable"
|
||||
}
|
||||
usageError := func(message string) int {
|
||||
return writeExecUsageError(stderr, redaction.RedactString(message, redaction.Options{}))
|
||||
}
|
||||
options, positional, help, err := parsePluginToggleArgs(args, commandName)
|
||||
if err != nil {
|
||||
return usageError(err.Error())
|
||||
}
|
||||
if help {
|
||||
if err := writePluginsToggleHelp(stdout, commandName); err != nil {
|
||||
return exitCrash
|
||||
}
|
||||
return exitSuccess
|
||||
}
|
||||
if len(positional) != 1 {
|
||||
return usageError(fmt.Sprintf("usage: zero plugins %s <id> [--user] [--json]", commandName))
|
||||
}
|
||||
pluginID := strings.TrimSpace(positional[0])
|
||||
if pluginID == "" {
|
||||
return usageError(fmt.Sprintf("usage: zero plugins %s <id> [--user] [--json]", commandName))
|
||||
}
|
||||
|
||||
cwd, err := deps.getwd()
|
||||
if err != nil {
|
||||
return writeAppError(stderr, "failed to resolve workspace: "+redaction.ErrorMessage(err, redaction.Options{}), exitCrash)
|
||||
}
|
||||
loadOptions := plugins.LoadOptions{Cwd: cwd, ExcludeProject: options.user}
|
||||
result, err := plugins.SetEnabledByID(loadOptions, pluginID, !disabled)
|
||||
if err != nil {
|
||||
if errors.Is(err, plugins.ErrNotInstalled) {
|
||||
return usageError(err.Error())
|
||||
}
|
||||
return writeAppError(stderr, redaction.ErrorMessage(err, redaction.Options{}), exitCrash)
|
||||
}
|
||||
|
||||
state := "enabled"
|
||||
if disabled {
|
||||
state = "disabled"
|
||||
}
|
||||
if options.json {
|
||||
if err := writePrettyJSON(stdout, redaction.RedactValue(result, redaction.Options{})); err != nil {
|
||||
return exitCrash
|
||||
}
|
||||
return exitSuccess
|
||||
}
|
||||
message := fmt.Sprintf("Plugin %s is now %s in %s.\n", result.ID, state, result.ManifestPath)
|
||||
if !result.Changed {
|
||||
message = fmt.Sprintf("Plugin %s was already %s in %s.\n", result.ID, state, result.ManifestPath)
|
||||
}
|
||||
if _, err := fmt.Fprint(stdout, redaction.RedactString(message, redaction.Options{})); err != nil {
|
||||
return exitCrash
|
||||
}
|
||||
return exitSuccess
|
||||
}
|
||||
|
||||
func parsePluginToggleArgs(args []string, command string) (pluginToggleOptions, []string, bool, error) {
|
||||
options := pluginToggleOptions{}
|
||||
positional := []string{}
|
||||
for _, arg := range args {
|
||||
switch arg {
|
||||
case "-h", "--help", "help":
|
||||
return options, positional, true, nil
|
||||
case "--json":
|
||||
options.json = true
|
||||
case "--user":
|
||||
options.user = true
|
||||
default:
|
||||
if strings.HasPrefix(arg, "-") {
|
||||
return options, positional, false, execUsageError{fmt.Sprintf("unknown plugins %s flag %q", command, arg)}
|
||||
}
|
||||
positional = append(positional, arg)
|
||||
}
|
||||
}
|
||||
return options, positional, false, nil
|
||||
}
|
||||
|
||||
func runHooks(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) int {
|
||||
if len(args) == 0 {
|
||||
return writeExecUsageError(stderr, "hooks subcommand required. Use `zero hooks list`.")
|
||||
@@ -566,10 +661,31 @@ Commands:
|
||||
info <id> Show plugin details and lockfile metadata
|
||||
add <git-url|path> Install a plugin (manifest-validated, pinned in plugins.lock)
|
||||
remove <id> Remove an installed plugin and its lockfile entry
|
||||
enable <id> Enable a plugin via its plugin.json
|
||||
disable <id> Disable a plugin via its plugin.json
|
||||
`)
|
||||
return err
|
||||
}
|
||||
|
||||
func writePluginsToggleHelp(w io.Writer, command string) error {
|
||||
_, err := fmt.Fprintf(w, `Usage:
|
||||
zero plugins %s <id> [flags]
|
||||
|
||||
Flags:
|
||||
--user Target only user plugins (ignore ./.zero/plugins)
|
||||
--json Print command result as JSON
|
||||
-h, --help Show this help
|
||||
|
||||
Without --user, a project plugin with the same id takes precedence over a user
|
||||
plugin, matching the loader's discovery order.
|
||||
|
||||
Zero reads plugins when it starts, so the change applies to the next zero exec
|
||||
run or interactive session. A running interactive session keeps the plugins it
|
||||
started with until you restart it.
|
||||
`, command)
|
||||
return err
|
||||
}
|
||||
|
||||
func writePluginsListHelp(w io.Writer) error {
|
||||
_, err := fmt.Fprint(w, `Usage:
|
||||
zero plugins list [flags]
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
@@ -333,3 +334,234 @@ func TestRunMCPPermissionsHelpDoesNotOpenStore(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunPluginsEnableDisable(t *testing.T) {
|
||||
cwd := t.TempDir()
|
||||
home := t.TempDir()
|
||||
t.Setenv("XDG_CONFIG_HOME", home)
|
||||
t.Setenv("HOME", home)
|
||||
pluginDir := filepath.Join(cwd, ".zero", "plugins", "demo")
|
||||
if err := os.MkdirAll(pluginDir, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
manifestPath := filepath.Join(pluginDir, "plugin.json")
|
||||
if err := os.WriteFile(manifestPath, []byte(`{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0"
|
||||
}
|
||||
`), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
deps := appDeps{getwd: func() (string, error) { return cwd, nil }}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
exitCode := runWithDeps([]string{"plugins", "disable", "zero.demo", "--json"}, &stdout, &stderr, deps)
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("disable exit=%d stderr=%s", exitCode, stderr.String())
|
||||
}
|
||||
var disablePayload plugins.SetEnabledResult
|
||||
if err := json.Unmarshal(stdout.Bytes(), &disablePayload); err != nil {
|
||||
t.Fatalf("decode disable JSON: %v\n%s", err, stdout.String())
|
||||
}
|
||||
if disablePayload.ID != "zero.demo" || disablePayload.Enabled || !disablePayload.Changed {
|
||||
t.Fatalf("unexpected disable payload: %#v", disablePayload)
|
||||
}
|
||||
|
||||
raw, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(raw), `"enabled": false`) {
|
||||
t.Fatalf("manifest not disabled: %s", raw)
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
exitCode = runWithDeps([]string{"plugins", "disable", "zero.demo"}, &stdout, &stderr, deps)
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("noop disable exit=%d stderr=%s", exitCode, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "was already disabled") {
|
||||
t.Fatalf("noop disable output: %s", stdout.String())
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
exitCode = runWithDeps([]string{"plugins", "enable", "zero.demo"}, &stdout, &stderr, deps)
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("enable exit=%d stderr=%s", exitCode, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "is now enabled") {
|
||||
t.Fatalf("enable output: %s", stdout.String())
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
exitCode = runWithDeps([]string{"plugins", "enable", "missing-plugin"}, &stdout, &stderr, deps)
|
||||
if exitCode != exitUsage {
|
||||
t.Fatalf("missing plugin exit=%d want %d stderr=%s", exitCode, exitUsage, stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
// The text-mode toggle output runs through the redaction pipeline, so a
|
||||
// manifest path that contains something secret-shaped is masked.
|
||||
func TestRunPluginsToggleTextOutputIsRedacted(t *testing.T) {
|
||||
cwd := t.TempDir()
|
||||
home := t.TempDir()
|
||||
t.Setenv("XDG_CONFIG_HOME", home)
|
||||
t.Setenv("HOME", home)
|
||||
secretDir := "sk-proj-abcdefghijklmnopqrstuvwxyz"
|
||||
pluginDir := filepath.Join(cwd, ".zero", "plugins", secretDir)
|
||||
if err := os.MkdirAll(pluginDir, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
manifestPath := filepath.Join(pluginDir, "plugin.json")
|
||||
if err := os.WriteFile(manifestPath, []byte(`{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0"
|
||||
}
|
||||
`), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
deps := appDeps{getwd: func() (string, error) { return cwd, nil }}
|
||||
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
exitCode := runWithDeps([]string{"plugins", "disable", "zero.demo"}, &stdout, &stderr, deps)
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("disable exit=%d stderr=%s", exitCode, stderr.String())
|
||||
}
|
||||
if strings.Contains(stdout.String(), secretDir) {
|
||||
t.Fatalf("toggle text leaked a secret-shaped manifest path: %q", stdout.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "[REDACTED]") {
|
||||
t.Fatalf("toggle text was not redacted: %q", stdout.String())
|
||||
}
|
||||
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
exitCode = runWithDeps([]string{"plugins", "disable", "zero.demo"}, &stdout, &stderr, deps)
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("noop disable exit=%d stderr=%s", exitCode, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "was already disabled") {
|
||||
t.Fatalf("noop toggle output: %q", stdout.String())
|
||||
}
|
||||
if strings.Contains(stdout.String(), secretDir) {
|
||||
t.Fatalf("noop toggle text leaked a secret-shaped manifest path: %q", stdout.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunPluginsToggleRejectsEmptyID(t *testing.T) {
|
||||
deps := appDeps{getwd: func() (string, error) { return t.TempDir(), nil }}
|
||||
|
||||
for _, subcommand := range []string{"enable", "disable"} {
|
||||
t.Run(subcommand, func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
exitCode := runWithDeps([]string{"plugins", subcommand, ""}, &stdout, &stderr, deps)
|
||||
if exitCode != exitUsage {
|
||||
t.Fatalf("exit=%d want %d stderr=%s", exitCode, exitUsage, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stderr.String(), "usage: zero plugins "+subcommand) {
|
||||
t.Fatalf("stderr = %q", stderr.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunPluginsDisableUserIgnoresProject(t *testing.T) {
|
||||
home := t.TempDir()
|
||||
cwd := t.TempDir()
|
||||
t.Setenv("XDG_CONFIG_HOME", home)
|
||||
t.Setenv("HOME", home)
|
||||
|
||||
userManifest := filepath.Join(home, "zero", "plugins", "demo", "plugin.json")
|
||||
projectManifest := filepath.Join(cwd, ".zero", "plugins", "demo", "plugin.json")
|
||||
for _, path := range []string{userManifest, projectManifest} {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, []byte(`{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
"enabled": true
|
||||
}
|
||||
`), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
deps := appDeps{getwd: func() (string, error) { return cwd, nil }}
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
exitCode := runWithDeps([]string{"plugins", "disable", "zero.demo", "--user"}, &stdout, &stderr, deps)
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("disable --user exit=%d stderr=%s", exitCode, stderr.String())
|
||||
}
|
||||
if !strings.Contains(stdout.String(), "is now disabled") {
|
||||
t.Fatalf("disable --user output: %s", stdout.String())
|
||||
}
|
||||
|
||||
userRaw, err := os.ReadFile(userManifest)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(userRaw), `"enabled": false`) {
|
||||
t.Fatalf("user manifest not disabled: %s", userRaw)
|
||||
}
|
||||
projectRaw, err := os.ReadFile(projectManifest)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(projectRaw), `"enabled": false`) {
|
||||
t.Fatalf("project manifest should stay enabled: %s", projectRaw)
|
||||
}
|
||||
}
|
||||
|
||||
// The toggle's usage errors echo the caller's arguments, which can be a
|
||||
// mistyped credential, so they pass through redaction like its other output.
|
||||
func TestRunPluginsToggleUsageErrorsAreRedacted(t *testing.T) {
|
||||
home := t.TempDir()
|
||||
t.Setenv("XDG_CONFIG_HOME", home)
|
||||
t.Setenv("HOME", home)
|
||||
deps := appDeps{getwd: func() (string, error) { return t.TempDir(), nil }}
|
||||
const secret = "sk-proj-abcdefghijklmnopqrstuvwxyz"
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
args []string
|
||||
want string
|
||||
}{
|
||||
{"unknown flag", []string{"zero.demo", "--token=" + secret}, "unknown plugins"},
|
||||
{"missing plugin", []string{secret}, "plugin is not installed"},
|
||||
}
|
||||
for _, subcommand := range []string{"enable", "disable"} {
|
||||
for _, tc := range cases {
|
||||
t.Run(subcommand+" "+tc.name, func(t *testing.T) {
|
||||
var stdout bytes.Buffer
|
||||
var stderr bytes.Buffer
|
||||
args := append([]string{"plugins", subcommand}, tc.args...)
|
||||
exitCode := runWithDeps(args, &stdout, &stderr, deps)
|
||||
if exitCode != exitUsage {
|
||||
t.Fatalf("exit=%d want %d stderr=%s", exitCode, exitUsage, stderr.String())
|
||||
}
|
||||
if strings.Contains(stderr.String(), secret) {
|
||||
t.Fatalf("usage error leaked the argument: %q", stderr.String())
|
||||
}
|
||||
if !strings.Contains(stderr.String(), tc.want) || !strings.Contains(stderr.String(), "[REDACTED]") {
|
||||
t.Fatalf("stderr = %q, want a redacted %q error", stderr.String(), tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -319,7 +319,7 @@ Flags:
|
||||
--command <cmd> Command to run (required)
|
||||
--name <name> Human-readable hook name
|
||||
--description <text> Hook description
|
||||
--matcher <pattern> Tool matcher (beforeTool/afterTool only)
|
||||
--matcher <pattern> Tool/specialist matcher (beforeTool/afterTool/specialistStart/specialistStop)
|
||||
--arg <value> Command argument (repeatable)
|
||||
--user Write to user config instead of the project
|
||||
--json Print command result as JSON
|
||||
|
||||
@@ -741,6 +741,13 @@ func (cfg *mcpWritableConfig) upsertServer(name string, server config.MCPServerC
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if !mergedServer.Disabled && config.IsDefaultMCPServerDisabled(name) {
|
||||
// Adding a server under the name of a default that ships disabled means
|
||||
// the user wants it on; the omitted flag would otherwise keep it off.
|
||||
if data, err = setMCPServerRawDisabledFalse(data); err != nil {
|
||||
return false, err
|
||||
}
|
||||
}
|
||||
cfg.serverRaw[name] = data
|
||||
if _, err := cfg.removeLegacyServer(name); err != nil {
|
||||
return false, err
|
||||
@@ -800,11 +807,29 @@ func mergeMCPServerRaw(existing json.RawMessage, server config.MCPServerConfig)
|
||||
if key == "oauth" && preserveRawOAuth {
|
||||
continue
|
||||
}
|
||||
// `mcp add` replaces the entry: it writes only a disable, never a stale
|
||||
// explicit false carried over from the old entry (upsertServer adds the
|
||||
// false a default that ships disabled needs).
|
||||
if key == "disabled" && string(value) == "false" {
|
||||
continue
|
||||
}
|
||||
merged[key] = value
|
||||
}
|
||||
return json.Marshal(merged)
|
||||
}
|
||||
|
||||
func setMCPServerRawDisabledFalse(data json.RawMessage) (json.RawMessage, error) {
|
||||
var server map[string]json.RawMessage
|
||||
if err := json.Unmarshal(data, &server); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if server == nil {
|
||||
server = map[string]json.RawMessage{}
|
||||
}
|
||||
server["disabled"] = json.RawMessage("false")
|
||||
return json.Marshal(server)
|
||||
}
|
||||
|
||||
func (cfg *mcpWritableConfig) removeServer(name string) (bool, error) {
|
||||
cfg.ensureRaw()
|
||||
removed := false
|
||||
@@ -856,15 +881,19 @@ func (cfg *mcpWritableConfig) setServerDisabled(name string, disabled bool) (boo
|
||||
server = map[string]json.RawMessage{}
|
||||
}
|
||||
|
||||
current := false
|
||||
// A default that ships disabled (aimlapi) stays off until an entry says
|
||||
// otherwise, so an absent flag reads as disabled and enabling it must be
|
||||
// written out explicitly: dropping the flag would leave it off.
|
||||
shippedDisabled := config.IsDefaultMCPServerDisabled(name)
|
||||
current := shippedDisabled
|
||||
if rawDisabled, ok := server["disabled"]; ok && len(rawDisabled) > 0 && string(rawDisabled) != "null" {
|
||||
if err := json.Unmarshal(rawDisabled, ¤t); err != nil {
|
||||
return false, false, err
|
||||
}
|
||||
}
|
||||
changed := current != disabled
|
||||
if disabled {
|
||||
data, err := json.Marshal(true)
|
||||
if disabled || shippedDisabled {
|
||||
data, err := json.Marshal(disabled)
|
||||
if err != nil {
|
||||
return false, false, err
|
||||
}
|
||||
|
||||
@@ -2,10 +2,18 @@ package cli
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
)
|
||||
|
||||
@@ -44,3 +52,255 @@ func TestRunMCPDisableSeededExaDefault(t *testing.T) {
|
||||
t.Fatal("expected `mcp disable exa` to turn the seeded default off in the resolved config")
|
||||
}
|
||||
}
|
||||
|
||||
// aimlapiWire stands in for https://mcp.aimlapi.com/mcp at the transport layer:
|
||||
// requests keep their real URL, nothing is dialed, and each one is recorded.
|
||||
type aimlapiWire struct {
|
||||
mu sync.Mutex
|
||||
requests []*http.Request
|
||||
}
|
||||
|
||||
func (wire *aimlapiWire) RoundTrip(request *http.Request) (*http.Response, error) {
|
||||
var body []byte
|
||||
if request.Body != nil {
|
||||
body, _ = io.ReadAll(request.Body)
|
||||
_ = request.Body.Close()
|
||||
}
|
||||
wire.mu.Lock()
|
||||
wire.requests = append(wire.requests, request.Clone(context.Background()))
|
||||
wire.mu.Unlock()
|
||||
var message struct {
|
||||
ID json.RawMessage `json:"id"`
|
||||
Method string `json:"method"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &message)
|
||||
respond := func(status int, payload string) *http.Response {
|
||||
return &http.Response{StatusCode: status, Header: http.Header{"Content-Type": {"application/json"}}, Body: io.NopCloser(strings.NewReader(payload)), Request: request}
|
||||
}
|
||||
if request.URL.String() != aimlapi.MCPServerURL {
|
||||
return respond(http.StatusBadGateway, `{}`), nil
|
||||
}
|
||||
if len(message.ID) == 0 {
|
||||
return respond(http.StatusAccepted, ""), nil
|
||||
}
|
||||
result := `{"protocolVersion":"2024-11-05","capabilities":{},"serverInfo":{"name":"fake","version":"1"}}`
|
||||
if message.Method == "tools/list" {
|
||||
result = `{"tools":[{"name":"tools_search","description":"Search the Toolbox","inputSchema":{"type":"object"}}]}`
|
||||
}
|
||||
return respond(http.StatusOK, fmt.Sprintf(`{"jsonrpc":"2.0","id":%s,"result":%s}`, message.ID, result)), nil
|
||||
}
|
||||
|
||||
func (wire *aimlapiWire) recorded() []*http.Request {
|
||||
wire.mu.Lock()
|
||||
defer wire.mu.Unlock()
|
||||
return append([]*http.Request(nil), wire.requests...)
|
||||
}
|
||||
|
||||
// isolateAIMLAPIMCPCommand redirects every root the real resolve path reads to
|
||||
// a temporary home, sets an AI/ML API key in the environment, and routes the
|
||||
// default HTTP transport to a recorder. It returns the user config path.
|
||||
func isolateAIMLAPIMCPCommand(t *testing.T, key string) (string, *aimlapiWire, appDeps) {
|
||||
t.Helper()
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("XDG_STATE_HOME", filepath.Join(home, ".state"))
|
||||
t.Setenv("XDG_CACHE_HOME", filepath.Join(home, ".cache"))
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData", "Roaming"))
|
||||
t.Setenv("LOCALAPPDATA", filepath.Join(home, "AppData", "Local"))
|
||||
t.Setenv("ZERO_CRED_STORAGE", "file")
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_PARTNER_ID", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", key)
|
||||
configPath, err := config.DefaultUserConfigPath()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeMCPCommandRawConfig(t, configPath, `{}`)
|
||||
|
||||
wire := &aimlapiWire{}
|
||||
previous := http.DefaultTransport
|
||||
http.DefaultTransport = wire
|
||||
t.Cleanup(func() { http.DefaultTransport = previous })
|
||||
|
||||
workspace := t.TempDir()
|
||||
return configPath, wire, appDeps{getwd: func() (string, error) { return workspace, nil }}
|
||||
}
|
||||
|
||||
func TestRunMCPEnableAIMLAPIDefaultWritesOnlyTheToggle(t *testing.T) {
|
||||
configPath, wire, deps := isolateAIMLAPIMCPCommand(t, "aiml-secret-key")
|
||||
|
||||
var out, errBuf bytes.Buffer
|
||||
if code := runWithDeps([]string{"mcp", "disable", "aimlapi", "--json"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("disable exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
if !strings.Contains(out.String(), `"changed": false`) {
|
||||
t.Fatalf("disabling the off-by-default server should be a no-op: %s", out.String())
|
||||
}
|
||||
|
||||
out.Reset()
|
||||
errBuf.Reset()
|
||||
if code := runWithDeps([]string{"mcp", "enable", "aimlapi", "--json"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("enable exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
if !strings.Contains(out.String(), `"changed": true`) {
|
||||
t.Fatalf("enable should report a change: %s", out.String())
|
||||
}
|
||||
data, err := os.ReadFile(configPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var written struct {
|
||||
MCP struct {
|
||||
Servers map[string]map[string]any `json:"servers"`
|
||||
} `json:"mcp"`
|
||||
}
|
||||
if err := json.Unmarshal(data, &written); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entry := written.MCP.Servers["aimlapi"]; len(entry) != 1 || entry["disabled"] != false {
|
||||
t.Fatalf("enable must write only the toggle, got %#v in %s", entry, data)
|
||||
}
|
||||
if strings.Contains(string(data), "aiml-secret-key") || strings.Contains(string(data), "Authorization") {
|
||||
t.Fatalf("config holds credential material: %s", data)
|
||||
}
|
||||
|
||||
cfg, err := config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if server := cfg.Servers["aimlapi"]; server.Disabled || config.IsUnconfiguredDefault("aimlapi", server) {
|
||||
t.Fatalf("enabled aimlapi = %#v, want enabled and user-configured", server)
|
||||
}
|
||||
// An unrelated save rewrites the whole file through FileConfig; the
|
||||
// explicit enable has to survive it or the shipped disable comes back.
|
||||
if _, err := config.SetTheme(configPath, "dark"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err = config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Servers["aimlapi"].Disabled {
|
||||
data, _ := os.ReadFile(configPath)
|
||||
t.Fatalf("an unrelated config save turned aimlapi off again: %s", data)
|
||||
}
|
||||
if len(wire.recorded()) != 0 {
|
||||
t.Fatal("enable must not connect")
|
||||
}
|
||||
|
||||
out.Reset()
|
||||
if code := runWithDeps([]string{"mcp", "disable", "aimlapi"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("disable exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
cfg, err = config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !cfg.Servers["aimlapi"].Disabled {
|
||||
t.Fatal("disable after enable must turn aimlapi off again")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunMCPAddUnderAIMLAPIDefaultNameEnablesIt(t *testing.T) {
|
||||
configPath, _, deps := isolateAIMLAPIMCPCommand(t, "")
|
||||
var out, errBuf bytes.Buffer
|
||||
if code := runWithDeps([]string{"mcp", "add", "aimlapi", "--type", "http", "--url", aimlapi.MCPServerURL, "--auth", "oauth"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("add exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
cfg, err := config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if server := cfg.Servers["aimlapi"]; server.Disabled || server.Auth != "oauth" {
|
||||
t.Fatalf("added aimlapi = %#v, want enabled with oauth", server)
|
||||
}
|
||||
if code := runWithDeps([]string{"mcp", "add", "aimlapi", "--type", "http", "--url", aimlapi.MCPServerURL, "--disabled"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("add --disabled exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
if cfg, err = config.ResolveMCP(config.ResolveOptions{UserConfigPath: configPath}); err != nil || !cfg.Servers["aimlapi"].Disabled {
|
||||
t.Fatalf("add --disabled left aimlapi enabled: %#v, %v", cfg.Servers["aimlapi"], err)
|
||||
}
|
||||
}
|
||||
|
||||
// Through the real resolve path (no injected resolveMCPConfig): the default is
|
||||
// listed as disabled, never connects, and once enabled it reaches session
|
||||
// startup and `zero mcp check`, which connects with the key on the wire.
|
||||
func TestAIMLAPIMCPDefaultThroughRealResolvePath(t *testing.T) {
|
||||
_, wire, deps := isolateAIMLAPIMCPCommand(t, "aiml-secret-key")
|
||||
|
||||
var out, errBuf bytes.Buffer
|
||||
if code := runWithDeps([]string{"mcp", "list"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("list exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
if !strings.Contains(out.String(), "aimlapi [http] disabled https://mcp.aimlapi.com/mcp") || strings.Contains(out.String(), "aiml-secret-key") {
|
||||
t.Fatalf("list output should show the disabled aimlapi default and no key:\n%s", out.String())
|
||||
}
|
||||
out.Reset()
|
||||
if code := runWithDeps([]string{"mcp", "list", "--json"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("list --json exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
if !strings.Contains(out.String(), `"aimlapi"`) || strings.Contains(out.String(), "aiml-secret-key") {
|
||||
t.Fatalf("list --json = %s", out.String())
|
||||
}
|
||||
out.Reset()
|
||||
errBuf.Reset()
|
||||
if code := runWithDeps([]string{"mcp", "check", "aimlapi"}, &out, &errBuf, deps); code == exitSuccess || !strings.Contains(errBuf.String(), "is disabled") {
|
||||
t.Fatalf("check of the disabled default: exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
resolved, err := fillAppDeps(deps).resolveMCPConfig(t.TempDir(), true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
critical, optional := splitMCPStartupConfig(resolved)
|
||||
if _, ok := critical.Servers["aimlapi"]; ok {
|
||||
t.Fatal("the disabled default must not start with a session")
|
||||
}
|
||||
if _, ok := optional.Servers["aimlapi"]; ok {
|
||||
t.Fatal("the disabled default must not start with a session")
|
||||
}
|
||||
if len(wire.recorded()) != 0 {
|
||||
t.Fatalf("the disabled default connected: %d requests", len(wire.recorded()))
|
||||
}
|
||||
|
||||
if code := runWithDeps([]string{"mcp", "enable", "aimlapi"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("enable exit=%d stderr=%s", code, errBuf.String())
|
||||
}
|
||||
resolved, err = fillAppDeps(deps).resolveMCPConfig(t.TempDir(), true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
critical, _ = splitMCPStartupConfig(resolved)
|
||||
if _, ok := critical.Servers["aimlapi"]; !ok {
|
||||
t.Fatal("an enabled aimlapi must start with the session as a user-configured server")
|
||||
}
|
||||
|
||||
out.Reset()
|
||||
errBuf.Reset()
|
||||
if code := runWithDeps([]string{"mcp", "check", "aimlapi", "--json"}, &out, &errBuf, deps); code != exitSuccess {
|
||||
t.Fatalf("check exit=%d stdout=%s stderr=%s", code, out.String(), errBuf.String())
|
||||
}
|
||||
if !strings.Contains(out.String(), `"status": "ok"`) || !strings.Contains(out.String(), `"toolCount": 1`) {
|
||||
t.Fatalf("check output = %s", out.String())
|
||||
}
|
||||
if strings.Contains(out.String()+errBuf.String(), "aiml-secret-key") {
|
||||
t.Fatalf("check output leaks the key: %s %s", out.String(), errBuf.String())
|
||||
}
|
||||
requests := wire.recorded()
|
||||
if len(requests) == 0 {
|
||||
t.Fatal("check did not connect")
|
||||
}
|
||||
for _, request := range requests {
|
||||
if request.URL.String() != aimlapi.MCPServerURL {
|
||||
t.Fatalf("request went to %s", request.URL)
|
||||
}
|
||||
if got := request.Header.Get("Authorization"); got != "Bearer aiml-secret-key" {
|
||||
t.Fatalf("Authorization = %q, want the AI/ML API key", got)
|
||||
}
|
||||
if request.Header.Get(aimlapi.SourceHeaderName) != aimlapi.SourceValue || request.Header.Get(aimlapi.PartnerHeaderName) == "" {
|
||||
t.Fatalf("attribution headers missing: %v", request.Header)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"io"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/hooks"
|
||||
"github.com/Twigpine/zero/internal/mcp"
|
||||
"github.com/Twigpine/zero/internal/specialist"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/tui"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestRegisterSpecialistToolsWiresLifecycleHooksIntoTaskOutput(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
runtime, err := registerSpecialistTools(registry, t.TempDir(), 0)
|
||||
if err != nil {
|
||||
t.Fatalf("registerSpecialistTools returned error: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { closeSpecialistRuntime(io.Discard, runtime) })
|
||||
|
||||
if runtime.lifecycleHooks == nil {
|
||||
t.Fatal("lifecycleHooks is nil after registerSpecialistTools")
|
||||
}
|
||||
raw, ok := registry.Get("TaskOutput")
|
||||
if !ok {
|
||||
t.Fatal("TaskOutput tool not registered")
|
||||
}
|
||||
outputTool, ok := raw.(*specialist.OutputTool)
|
||||
if !ok {
|
||||
t.Fatalf("TaskOutput type = %T, want *specialist.OutputTool", raw)
|
||||
}
|
||||
if outputTool.LifecycleHooks != runtime.lifecycleHooks {
|
||||
t.Fatal("TaskOutput LifecycleHooks is not the shared registerSpecialistTools bridge")
|
||||
}
|
||||
|
||||
dispatcher := hooks.NewDispatcher(hooks.DispatcherOptions{})
|
||||
runtime.attachLifecycleHooks(dispatcher)
|
||||
if runtime.lifecycleHooks.Dispatch == nil {
|
||||
t.Fatal("attachLifecycleHooks left Dispatch nil")
|
||||
}
|
||||
if outputTool.LifecycleHooks.Dispatch == nil {
|
||||
t.Fatal("TaskOutput did not observe Dispatch after attachLifecycleHooks")
|
||||
}
|
||||
}
|
||||
|
||||
// TestRealEntryPointsAttachLifecycleHooks closes the gap the registration-level
|
||||
// test above cannot see: wiring the bridge when tools register is not enough
|
||||
// unless the real startup paths actually call attachLifecycleHooks before any
|
||||
// run. Both surfaces that can launch specialists must do so — the interactive
|
||||
// TUI and headless exec. attachLifecycleHooks mutates the same
|
||||
// *specialist.LifecycleHooks the TaskOutput tool holds, so a nil Dispatch on
|
||||
// that tool after a full startup proves the call never ran and the
|
||||
// specialistStart/specialistStop hooks would silently never fire.
|
||||
func TestRealEntryPointsAttachLifecycleHooks(t *testing.T) {
|
||||
requireTaskOutputDispatchAttached := func(t *testing.T, registry *tools.Registry) {
|
||||
t.Helper()
|
||||
if registry == nil {
|
||||
t.Fatal("startup did not reach the run registry")
|
||||
}
|
||||
raw, ok := registry.Get("TaskOutput")
|
||||
if !ok {
|
||||
t.Fatal("TaskOutput tool not registered by startup")
|
||||
}
|
||||
outputTool, ok := raw.(*specialist.OutputTool)
|
||||
if !ok {
|
||||
t.Fatalf("TaskOutput type = %T, want *specialist.OutputTool", raw)
|
||||
}
|
||||
if outputTool.LifecycleHooks == nil || outputTool.LifecycleHooks.Dispatch == nil {
|
||||
t.Fatal("startup left the specialist lifecycle bridge unattached: specialistStart/specialistStop hooks would never fire")
|
||||
}
|
||||
}
|
||||
|
||||
t.Run("interactive TUI", func(t *testing.T) {
|
||||
isolateCLIUserState(t)
|
||||
cwd := t.TempDir()
|
||||
var registry *tools.Registry
|
||||
|
||||
// Empty args route runWithDeps to the interactive TUI path; the injected
|
||||
// runTUI receives the same registry specialist tools registered into.
|
||||
exitCode := runWithDeps([]string{}, io.Discard, io.Discard, appDeps{
|
||||
getwd: func() (string, error) { return cwd, nil },
|
||||
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
|
||||
return config.ResolvedConfig{
|
||||
Provider: config.ProviderProfile{
|
||||
Name: "p",
|
||||
ProviderKind: config.ProviderKindOpenAICompatible,
|
||||
BaseURL: "http://127.0.0.1/v1",
|
||||
Model: "m",
|
||||
},
|
||||
}, nil
|
||||
},
|
||||
resolveMCPConfig: func(string, bool) (config.MCPConfig, error) { return config.MCPConfig{}, nil },
|
||||
runTUI: func(_ context.Context, options tui.Options) int {
|
||||
registry = options.Registry
|
||||
return exitSuccess
|
||||
},
|
||||
})
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("runWithDeps exit = %d, want %d", exitCode, exitSuccess)
|
||||
}
|
||||
requireTaskOutputDispatchAttached(t, registry)
|
||||
})
|
||||
|
||||
t.Run("headless exec", func(t *testing.T) {
|
||||
isolateCLIUserState(t)
|
||||
cwd := t.TempDir()
|
||||
var registry *tools.Registry
|
||||
var stdout, stderr bytes.Buffer
|
||||
|
||||
// --auto high is what registers specialist tools on the exec surface.
|
||||
// MCP registration is the one startup chokepoint that sees the run
|
||||
// registry before attachLifecycleHooks runs, so capture it there and
|
||||
// assert after the run completes.
|
||||
exitCode := runWithDeps([]string{
|
||||
"exec", "--auto", "high", "--cwd", cwd, "--init-session-id", "hook_wireup", "hello",
|
||||
}, &stdout, &stderr, appDeps{
|
||||
getwd: func() (string, error) { return cwd, nil },
|
||||
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
|
||||
return execResolvedConfig(), nil
|
||||
},
|
||||
newProvider: func(config.ProviderProfile) (zeroruntime.Provider, error) {
|
||||
return echoExecProvider{}, nil
|
||||
},
|
||||
resolveMCPConfig: func(string, bool) (config.MCPConfig, error) {
|
||||
return config.MCPConfig{Servers: map[string]config.MCPServerConfig{
|
||||
"wireup": {Type: "stdio", Command: "wireup-mcp"},
|
||||
}}, nil
|
||||
},
|
||||
newMCPStore: func() (*mcp.PermissionStore, error) { return nil, nil },
|
||||
registerMCPTools: func(_ context.Context, captured *tools.Registry, _ config.MCPConfig, _ mcp.RegisterOptions) (mcpToolRuntime, error) {
|
||||
registry = captured
|
||||
return closeFunc(func() error { return nil }), nil
|
||||
},
|
||||
})
|
||||
if exitCode != exitSuccess {
|
||||
t.Fatalf("runWithDeps exit = %d, want %d (stderr=%q)", exitCode, exitSuccess, stderr.String())
|
||||
}
|
||||
requireTaskOutputDispatchAttached(t, registry)
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/providercatalog"
|
||||
)
|
||||
|
||||
const aimlapiKeyEnv = "AIMLAPI_API_KEY"
|
||||
|
||||
var (
|
||||
// ErrAIMLAPIKeyMissing means no AI/ML API key is configured.
|
||||
ErrAIMLAPIKeyMissing = errors.New("no AI/ML API key is configured; set up aimlapi.com with /provider or set AIMLAPI_API_KEY")
|
||||
// ErrAIMLAPIProxyProfile means the aimlapi profile points at a proxy, so its
|
||||
// key belongs to that proxy and is withheld from aimlapi.com services.
|
||||
ErrAIMLAPIProxyProfile = errors.New("the aimlapi provider points at a proxy base URL; its key is not sent to aimlapi.com services")
|
||||
// ErrAIMLAPIKeyInvalid means the key cannot be sent as a single Bearer token.
|
||||
ErrAIMLAPIKeyInvalid = errors.New("the AI/ML API key contains whitespace, a comma or a control character")
|
||||
)
|
||||
|
||||
// AIMLAPIServiceKey returns the AI/ML API key that may be sent to the
|
||||
// aimlapi.com Toolbox and MCP server. It is resolved again on every call, so a
|
||||
// provider change is seen by the next request.
|
||||
//
|
||||
// The key comes from the aimlapi provider profile (the active one when it is
|
||||
// aimlapi, otherwise the first), including a key held in the credential store,
|
||||
// or from AIMLAPI_API_KEY when no aimlapi profile exists. The key is withheld
|
||||
// when that profile, any aimlapi profile, or any profile reading
|
||||
// AIMLAPI_API_KEY points at a base URL other than the canonical
|
||||
// https://api.aimlapi.com/v1, and when AIMLAPI_INFERENCE_URL is overridden:
|
||||
// in those cases the credential may belong to a proxy.
|
||||
func AIMLAPIServiceKey(resolved ResolvedConfig, store APIKeyGetter, getenv func(string) string) (string, error) {
|
||||
if getenv == nil {
|
||||
getenv = os.Getenv
|
||||
}
|
||||
if override := strings.TrimSpace(getenv("AIMLAPI_INFERENCE_URL")); override != "" && !sameBaseURL(override, aimlapi.CanonicalInferenceBaseURL) {
|
||||
return "", ErrAIMLAPIProxyProfile
|
||||
}
|
||||
for _, profile := range resolved.Providers {
|
||||
if aimlapiKeyedProfile(profile) && !sameBaseURL(profile.BaseURL, aimlapi.CanonicalInferenceBaseURL) {
|
||||
return "", ErrAIMLAPIProxyProfile
|
||||
}
|
||||
}
|
||||
if profile, ok := aimlapiServiceProfile(resolved); ok {
|
||||
if !sameBaseURL(profile.BaseURL, aimlapi.CanonicalInferenceBaseURL) {
|
||||
return "", ErrAIMLAPIProxyProfile
|
||||
}
|
||||
if store != nil && profile.APIKeyStored && strings.TrimSpace(profile.APIKey) == "" && strings.TrimSpace(profile.Name) != "" {
|
||||
key, found, err := store.Get(strings.TrimSpace(profile.Name))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("read AI/ML API credential store: %w", err)
|
||||
}
|
||||
if found {
|
||||
profile.APIKey = key
|
||||
}
|
||||
}
|
||||
return validAIMLAPIServiceKey(profile.APIKey)
|
||||
}
|
||||
return validAIMLAPIServiceKey(getenv(aimlapiKeyEnv))
|
||||
}
|
||||
|
||||
// LoadAIMLAPIServiceKey resolves the user configuration, the environment and
|
||||
// the user credential store, then applies AIMLAPIServiceKey. Project config is
|
||||
// not read: provider keys are user-scoped. A config with no active provider
|
||||
// still resolves its profiles (ErrNoActiveProvider), so the key is looked up in
|
||||
// that partial config; every other resolve error is returned. Resolve drops a
|
||||
// non-active profile it cannot normalize, so the configured profiles are
|
||||
// checked as well: such a profile still withholds the key when it carries it.
|
||||
func LoadAIMLAPIServiceKey() (string, error) {
|
||||
userConfigPath, err := DefaultUserConfigPath()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
existing, err := existingConfigFile(userConfigPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
resolved, err := Resolve(ResolveOptions{UserConfigPath: existing, ExcludeProject: true})
|
||||
if err != nil && !errors.Is(err, ErrNoActiveProvider) {
|
||||
return "", err
|
||||
}
|
||||
if existing != "" {
|
||||
fileConfig, err := loadConfigFile(existing)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
var configured FileConfig
|
||||
mergeConfig(&configured, fileConfig)
|
||||
if droppedAIMLAPIProxyProfile(configured.Providers, resolved.Providers) {
|
||||
return "", ErrAIMLAPIProxyProfile
|
||||
}
|
||||
}
|
||||
// Check proxy eligibility and already-resolved keys before opening a store.
|
||||
key, keyErr := AIMLAPIServiceKey(resolved, nil, os.Getenv)
|
||||
if !errors.Is(keyErr, ErrAIMLAPIKeyMissing) {
|
||||
return key, keyErr
|
||||
}
|
||||
profile, ok := aimlapiServiceProfile(resolved)
|
||||
if !ok || !profile.APIKeyStored {
|
||||
return key, keyErr
|
||||
}
|
||||
store, err := ProviderKeyStoreAt(filepath.Dir(userConfigPath))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("open AI/ML API credential store: %w", err)
|
||||
}
|
||||
return AIMLAPIServiceKey(resolved, store, os.Getenv)
|
||||
}
|
||||
|
||||
func isAIMLAPIProfile(profile ProviderProfile) bool {
|
||||
return strings.EqualFold(providercatalog.NormalizeID(profile.CatalogID), providercatalog.AIMLAPIID)
|
||||
}
|
||||
|
||||
// aimlapiKeyedProfile reports whether profile carries an AI/ML API credential:
|
||||
// an aimlapi catalog profile, or any profile that reads AIMLAPI_API_KEY.
|
||||
func aimlapiKeyedProfile(profile ProviderProfile) bool {
|
||||
return isAIMLAPIProfile(profile) || strings.EqualFold(strings.TrimSpace(profile.APIKeyEnv), aimlapiKeyEnv)
|
||||
}
|
||||
|
||||
// droppedAIMLAPIProxyProfile reports whether a configured profile missing from
|
||||
// the resolved list (Resolve could not normalize it) carries an AI/ML API
|
||||
// credential and is not explicitly on the canonical base URL. Its endpoint was
|
||||
// never resolved, so an empty base URL counts as a proxy.
|
||||
func droppedAIMLAPIProxyProfile(configured, resolved []ProviderProfile) bool {
|
||||
kept := make(map[string]bool, len(resolved))
|
||||
for _, profile := range resolved {
|
||||
kept[profile.Name] = true
|
||||
}
|
||||
for _, profile := range configured {
|
||||
if kept[strings.TrimSpace(profile.Name)] || !aimlapiKeyedProfile(profile) {
|
||||
continue
|
||||
}
|
||||
if !sameBaseURL(profile.BaseURL, aimlapi.CanonicalInferenceBaseURL) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func aimlapiServiceProfile(resolved ResolvedConfig) (ProviderProfile, bool) {
|
||||
if isAIMLAPIProfile(resolved.Provider) {
|
||||
return resolved.Provider, true
|
||||
}
|
||||
for _, profile := range resolved.Providers {
|
||||
if isAIMLAPIProfile(profile) {
|
||||
return profile, true
|
||||
}
|
||||
}
|
||||
return ProviderProfile{}, false
|
||||
}
|
||||
|
||||
func validAIMLAPIServiceKey(raw string) (string, error) {
|
||||
key := strings.TrimSpace(raw)
|
||||
if key == "" {
|
||||
return "", ErrAIMLAPIKeyMissing
|
||||
}
|
||||
for _, r := range key {
|
||||
if unicode.IsSpace(r) || r <= ' ' || r == 0x7f || r == ',' || (r >= 0x80 && r <= 0x9f) {
|
||||
return "", ErrAIMLAPIKeyInvalid
|
||||
}
|
||||
}
|
||||
return key, nil
|
||||
}
|
||||
@@ -0,0 +1,282 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func aimlapiServiceKeyEnv(values map[string]string) func(string) string {
|
||||
return func(name string) string { return values[name] }
|
||||
}
|
||||
|
||||
func TestAIMLAPIServiceKeyUsesCanonicalProfileKey(t *testing.T) {
|
||||
resolved := ResolvedConfig{Providers: []ProviderProfile{
|
||||
{Name: "openai", BaseURL: "https://api.openai.com/v1", APIKey: "sk-openai"},
|
||||
{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1/", APIKey: " aiml-key "},
|
||||
}}
|
||||
key, err := AIMLAPIServiceKey(resolved, nil, aimlapiServiceKeyEnv(nil))
|
||||
if err != nil || key != "aiml-key" {
|
||||
t.Fatalf("AIMLAPIServiceKey = %q, %v; want aiml-key", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIServiceKeyReadsCredentialStore(t *testing.T) {
|
||||
resolved := ResolvedConfig{Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKeyStored: true}}
|
||||
store := fakeKeyGetter{keys: map[string]string{"aimlapi": "stored-key"}}
|
||||
key, err := AIMLAPIServiceKey(resolved, store, aimlapiServiceKeyEnv(nil))
|
||||
if err != nil || key != "stored-key" {
|
||||
t.Fatalf("AIMLAPIServiceKey = %q, %v; want stored-key", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIServiceKeyPrefersResolvedProfileKeyOverCredentialStore(t *testing.T) {
|
||||
resolved := ResolvedConfig{Provider: ProviderProfile{
|
||||
Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1",
|
||||
APIKey: "env-key", APIKeyStored: true,
|
||||
}}
|
||||
store := fakeKeyGetter{keys: map[string]string{"aimlapi": "stored-key"}}
|
||||
key, err := AIMLAPIServiceKey(resolved, store, aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "env-key"}))
|
||||
if err != nil || key != "env-key" {
|
||||
t.Fatalf("AIMLAPIServiceKey = %q, %v; want env-key from resolved profile, not stored-key", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIServiceKeyFallsBackToEnvironmentWithoutProfile(t *testing.T) {
|
||||
key, err := AIMLAPIServiceKey(ResolvedConfig{}, nil, aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "env-key"}))
|
||||
if err != nil || key != "env-key" {
|
||||
t.Fatalf("AIMLAPIServiceKey = %q, %v; want env-key", key, err)
|
||||
}
|
||||
if _, err := AIMLAPIServiceKey(ResolvedConfig{}, nil, aimlapiServiceKeyEnv(nil)); !errors.Is(err, ErrAIMLAPIKeyMissing) {
|
||||
t.Fatalf("missing key error = %v, want ErrAIMLAPIKeyMissing", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIServiceKeyWithholdsProxyCredentials(t *testing.T) {
|
||||
env := aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "proxy-key"})
|
||||
cases := map[string]ResolvedConfig{
|
||||
"active aimlapi profile on a proxy": {Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://proxy.example/v1", APIKey: "proxy-key"}},
|
||||
"a second aimlapi profile on a proxy": {Providers: []ProviderProfile{
|
||||
{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKey: "proxy-key"},
|
||||
{Name: "aimlapi-proxy", CatalogID: "aimlapi", BaseURL: "https://proxy.example/v1", APIKey: "proxy-key"},
|
||||
}},
|
||||
"a custom profile reading AIMLAPI_API_KEY": {Providers: []ProviderProfile{
|
||||
{Name: "gateway", BaseURL: "https://proxy.example/v1", APIKeyEnv: "AIMLAPI_API_KEY", APIKey: "proxy-key"},
|
||||
}},
|
||||
}
|
||||
for name, resolved := range cases {
|
||||
key, err := AIMLAPIServiceKey(resolved, nil, env)
|
||||
if !errors.Is(err, ErrAIMLAPIProxyProfile) || key != "" {
|
||||
t.Errorf("%s: AIMLAPIServiceKey = %q, %v; want ErrAIMLAPIProxyProfile", name, key, err)
|
||||
}
|
||||
}
|
||||
|
||||
overridden := aimlapiServiceKeyEnv(map[string]string{"AIMLAPI_API_KEY": "proxy-key", "AIMLAPI_INFERENCE_URL": "https://staging.example/v1"})
|
||||
if key, err := AIMLAPIServiceKey(ResolvedConfig{}, nil, overridden); !errors.Is(err, ErrAIMLAPIProxyProfile) || key != "" {
|
||||
t.Fatalf("inference URL override: AIMLAPIServiceKey = %q, %v; want ErrAIMLAPIProxyProfile", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIServiceKeyRejectsKeysThatAreNotOneToken(t *testing.T) {
|
||||
for _, raw := range []string{"first\u00a0second", "first\u2003second", "first,second", "first\nsecond", "first\r\nsecond", "a b", "key\x00tail", "key\u0085tail"} {
|
||||
resolved := ResolvedConfig{Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKey: raw}}
|
||||
if key, err := AIMLAPIServiceKey(resolved, nil, aimlapiServiceKeyEnv(nil)); !errors.Is(err, ErrAIMLAPIKeyInvalid) || key != "" {
|
||||
t.Errorf("key %q: AIMLAPIServiceKey = %q, %v; want ErrAIMLAPIKeyInvalid", raw, key, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadAIMLAPIServiceKeyResolvesUserConfig(t *testing.T) {
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("ZERO_CRED_STORAGE", "file")
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", "env-key")
|
||||
|
||||
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "env-key" {
|
||||
t.Fatalf("without config: LoadAIMLAPIServiceKey = %q, %v; want env-key", key, err)
|
||||
}
|
||||
|
||||
path, err := DefaultUserConfigPath()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
proxy := `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi","baseURL":"https://proxy.example/v1","model":"m"}]}`
|
||||
if err := os.WriteFile(path, []byte(proxy), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if key, err := LoadAIMLAPIServiceKey(); !errors.Is(err, ErrAIMLAPIProxyProfile) || key != "" {
|
||||
t.Fatalf("proxy profile: LoadAIMLAPIServiceKey = %q, %v; want ErrAIMLAPIProxyProfile", key, err)
|
||||
}
|
||||
|
||||
canonical := `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi"}]}`
|
||||
if err := os.WriteFile(path, []byte(canonical), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "env-key" {
|
||||
t.Fatalf("canonical profile: LoadAIMLAPIServiceKey = %q, %v; want env-key", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Several profiles and no activeProvider make Resolve return
|
||||
// ErrNoActiveProvider with the normalized profiles; the interactive app keeps
|
||||
// running on that partial config, so the service key must still resolve.
|
||||
func TestLoadAIMLAPIServiceKeyWithoutActiveProvider(t *testing.T) {
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("ZERO_CRED_STORAGE", "file")
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", "env-key")
|
||||
path, err := DefaultUserConfigPath()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other := `{"name":"other","provider":"openai","model":"m","apiKey":"other-key"}`
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
active string
|
||||
profile string
|
||||
want string
|
||||
wantErr error
|
||||
}{
|
||||
{name: "environment key", profile: `{"name":"second","provider":"openai","model":"m","apiKey":"second-key"}`, want: "env-key"},
|
||||
{name: "stale active provider", active: "gone", profile: `{"name":"second","provider":"openai","model":"m","apiKey":"second-key"}`, want: "env-key"},
|
||||
{name: "aimlapi profile key", profile: `{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKey":"profile-key"}`, want: "profile-key"},
|
||||
{name: "proxy profile", profile: `{"name":"aimlapi","catalogID":"aimlapi","baseURL":"https://proxy.example/v1","model":"m"}`, wantErr: ErrAIMLAPIProxyProfile},
|
||||
} {
|
||||
data := `{"activeProvider":"` + tc.active + `","providers":[` + other + `,` + tc.profile + `]}`
|
||||
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
key, err := LoadAIMLAPIServiceKey()
|
||||
if tc.wantErr != nil {
|
||||
if !errors.Is(err, tc.wantErr) || key != "" {
|
||||
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %v", tc.name, key, err, tc.wantErr)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err != nil || key != tc.want {
|
||||
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %q", tc.name, key, err, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Resolve drops a non-active profile it cannot normalize (here an unknown
|
||||
// catalog entry). Such a profile still shows that AIMLAPI_API_KEY belongs to a
|
||||
// proxy, so the key is withheld whether or not a provider is active.
|
||||
func TestLoadAIMLAPIServiceKeyWithholdsForDroppedProxyProfile(t *testing.T) {
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("ZERO_CRED_STORAGE", "file")
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", "env-key")
|
||||
path, err := DefaultUserConfigPath()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
canonical := `{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKey":"profile-key"}`
|
||||
other := `{"name":"other","provider":"openai","model":"m","apiKey":"other-key"}`
|
||||
dropped := `{"name":"future","catalogID":"no-such-catalog","baseURL":"https://proxy.example/v1","apiKeyEnv":"AIMLAPI_API_KEY","model":"m"}`
|
||||
unrelated := `{"name":"future","catalogID":"no-such-catalog","baseURL":"https://proxy.example/v1","apiKeyEnv":"OTHER_API_KEY","model":"m"}`
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
data string
|
||||
want string
|
||||
wantErr error
|
||||
}{
|
||||
{name: "active aimlapi profile", data: `{"activeProvider":"aimlapi","providers":[` + canonical + `,` + dropped + `]}`, wantErr: ErrAIMLAPIProxyProfile},
|
||||
{name: "no active provider", data: `{"providers":[` + other + `,` + dropped + `]}`, wantErr: ErrAIMLAPIProxyProfile},
|
||||
{name: "dropped profile without the key", data: `{"activeProvider":"aimlapi","providers":[` + canonical + `,` + unrelated + `]}`, want: "profile-key"},
|
||||
} {
|
||||
if err := os.WriteFile(path, []byte(tc.data), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
key, err := LoadAIMLAPIServiceKey()
|
||||
if tc.wantErr != nil {
|
||||
if !errors.Is(err, tc.wantErr) || key != "" {
|
||||
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %v", tc.name, key, err, tc.wantErr)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err != nil || key != tc.want {
|
||||
t.Fatalf("%s: LoadAIMLAPIServiceKey = %q, %v; want %q", tc.name, key, err, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadAIMLAPIServiceKeyStoreOpenFailure(t *testing.T) {
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", "env-key")
|
||||
t.Setenv("ZERO_CRED_STORAGE", "invalid-test-storage")
|
||||
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "env-key" {
|
||||
t.Fatalf("environment key must not require a store: %q, %v", key, err)
|
||||
}
|
||||
path, err := DefaultUserConfigPath()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("AIMLAPI_API_KEY", "")
|
||||
data := `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKeyStored":true}]}`
|
||||
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if key, err := LoadAIMLAPIServiceKey(); key != "" || err == nil || !strings.Contains(err.Error(), "unknown storage") {
|
||||
t.Fatalf("stored key must report store failure: %q, %v", key, err)
|
||||
}
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "https://proxy.example/v1")
|
||||
if key, err := LoadAIMLAPIServiceKey(); key != "" || !errors.Is(err, ErrAIMLAPIProxyProfile) {
|
||||
t.Fatalf("proxy rejection must precede store open: %q, %v", key, err)
|
||||
}
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
data = `{"activeProvider":"aimlapi","providers":[{"name":"aimlapi","catalogID":"aimlapi","model":"m","apiKeyStored":true,"apiKey":"inline-key"}]}`
|
||||
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if key, err := LoadAIMLAPIServiceKey(); err != nil || key != "inline-key" {
|
||||
t.Fatalf("inline key must not require a store: %q, %v", key, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIServiceKeyReportsStoreReadFailure(t *testing.T) {
|
||||
want := errors.New("test credential read failure")
|
||||
resolved := ResolvedConfig{Provider: ProviderProfile{Name: "aimlapi", CatalogID: "aimlapi", BaseURL: "https://api.aimlapi.com/v1", APIKeyStored: true}}
|
||||
if key, err := AIMLAPIServiceKey(resolved, fakeKeyGetter{err: want}, aimlapiServiceKeyEnv(nil)); key != "" || !errors.Is(err, want) {
|
||||
t.Fatalf("read failure was hidden: %q, %v", key, err)
|
||||
}
|
||||
resolved.Provider.BaseURL = "https://proxy.example/v1"
|
||||
if _, err := AIMLAPIServiceKey(resolved, fakeKeyGetter{err: want}, aimlapiServiceKeyEnv(nil)); !errors.Is(err, ErrAIMLAPIProxyProfile) {
|
||||
t.Fatalf("proxy check must precede store read: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -3,26 +3,48 @@ package config
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
)
|
||||
|
||||
// DefaultMCPServers returns the MCP servers Zero ships ENABLED by default so
|
||||
// web search and page fetching work out of the box with no setup and no API
|
||||
// key. They are seeded before user/project config is merged (see ResolveMCP),
|
||||
// so a user can override any field — for example add an API-key header to lift
|
||||
// Exa's anonymous rate limit — or disable it entirely with
|
||||
// DefaultMCPServers returns the MCP servers Zero ships built in. Exa is ENABLED
|
||||
// by default so web search and page fetching work out of the box with no setup
|
||||
// and no API key. They are seeded before user/project config is merged (see
|
||||
// ResolveMCP), so a user can override any field — for example add an API-key
|
||||
// header to lift Exa's anonymous rate limit — or disable it entirely with
|
||||
// `zero mcp disable <name>` (which writes `"disabled": true`).
|
||||
//
|
||||
// Exa's hosted MCP server works anonymously with rate limits. Users can add an
|
||||
// Exa API key for higher limits.
|
||||
//
|
||||
// The AI/ML API server ships DISABLED: its tool runs and generations are billed
|
||||
// to the user's AI/ML API balance, so it only connects after
|
||||
// `zero mcp enable aimlapi` (which writes `"disabled": false`). The disable is
|
||||
// marked as explicitly set so the lower-trust project layer cannot lift it
|
||||
// (see mergeMCPServer); only the user and CLI-override scopes can. The key is
|
||||
// attached per request by the MCP client and is never part of this config.
|
||||
func DefaultMCPServers() map[string]MCPServerConfig {
|
||||
return map[string]MCPServerConfig{
|
||||
"exa": {
|
||||
Type: "http",
|
||||
URL: "https://mcp.exa.ai/mcp",
|
||||
},
|
||||
"aimlapi": {
|
||||
Type: "http",
|
||||
URL: aimlapi.MCPServerURL,
|
||||
Disabled: true,
|
||||
disabledSet: true,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// IsDefaultMCPServerDisabled reports whether name is a built-in default that
|
||||
// ships disabled, so enabling it has to be written out as `"disabled": false`
|
||||
// rather than by dropping the flag.
|
||||
func IsDefaultMCPServerDisabled(name string) bool {
|
||||
return DefaultMCPServers()[strings.TrimSpace(name)].Disabled
|
||||
}
|
||||
|
||||
// IsDefaultMCPServer reports whether name is one of Zero's built-in default MCP
|
||||
// servers. The config commands use it so a default can be disabled/enabled even
|
||||
// though it is not written to the user's config file until overridden.
|
||||
|
||||
@@ -291,3 +291,92 @@ func TestResolveMCPDisabledRetiredEntryStaysReEnableable(t *testing.T) {
|
||||
t.Fatal("the disable must still carry to the successor")
|
||||
}
|
||||
}
|
||||
|
||||
func writeMCPDefaultsTestConfig(t *testing.T, data string) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(t.TempDir(), "config.json")
|
||||
if err := os.WriteFile(path, []byte(data), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
func TestResolveMCPSeedsDisabledAIMLAPIDefault(t *testing.T) {
|
||||
if !IsDefaultMCPServer("aimlapi") || !IsDefaultMCPServerDisabled("aimlapi") || IsDefaultMCPServerDisabled("exa") {
|
||||
t.Fatal("aimlapi must be a built-in default that ships disabled, exa one that ships enabled")
|
||||
}
|
||||
// No user config at all, and a user config that never mentions aimlapi.
|
||||
for name, options := range map[string]ResolveOptions{
|
||||
"no config": {},
|
||||
"unrelated user file": {UserConfigPath: writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"exa":{"disabled":true}}}}`)},
|
||||
"user entry with only headers": {UserConfigPath: writeMCPDefaultsTestConfig(t,
|
||||
`{"mcp":{"servers":{"aimlapi":{"headers":{"X-Trace":"1"}}}}}`)},
|
||||
} {
|
||||
cfg, err := ResolveMCP(options)
|
||||
if err != nil {
|
||||
t.Fatalf("%s: ResolveMCP: %v", name, err)
|
||||
}
|
||||
server, ok := cfg.Servers["aimlapi"]
|
||||
if !ok {
|
||||
t.Fatalf("%s: the aimlapi default must be visible in the resolved config", name)
|
||||
}
|
||||
if server.Type != "http" || server.URL != "https://mcp.aimlapi.com/mcp" || !server.Disabled {
|
||||
t.Fatalf("%s: aimlapi default = %#v, want a disabled http server at the MCP URL", name, server)
|
||||
}
|
||||
}
|
||||
cfg, err := ResolveMCP(ResolveOptions{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !IsUnconfiguredDefault("aimlapi", cfg.Servers["aimlapi"]) {
|
||||
t.Fatal("the untouched aimlapi default should be reported as unconfigured")
|
||||
}
|
||||
if exa := cfg.Servers["exa"]; exa.Disabled || !IsUnconfiguredDefault("exa", exa) {
|
||||
t.Fatalf("exa default changed: %#v", exa)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveMCPUserEnablesAIMLAPIDefault(t *testing.T) {
|
||||
path := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":false}}}}`)
|
||||
cfg, err := ResolveMCP(ResolveOptions{UserConfigPath: path})
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveMCP: %v", err)
|
||||
}
|
||||
server := cfg.Servers["aimlapi"]
|
||||
if server.Disabled || server.URL != "https://mcp.aimlapi.com/mcp" || server.Type != "http" {
|
||||
t.Fatalf("user enable did not turn the default on: %#v", server)
|
||||
}
|
||||
if IsUnconfiguredDefault("aimlapi", server) {
|
||||
t.Fatal("an enabled aimlapi is user-configured, so its failures must be reported")
|
||||
}
|
||||
|
||||
enable := MCPServerConfig{Disabled: false, disabledSet: true}
|
||||
cfg, err = ResolveMCP(ResolveOptions{Overrides: Overrides{MCP: MCPConfig{Servers: map[string]MCPServerConfig{"aimlapi": enable}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Servers["aimlapi"].Disabled {
|
||||
t.Fatal("the CLI override scope must be able to enable the aimlapi default")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveMCPProjectCannotEnableAIMLAPIDefault(t *testing.T) {
|
||||
project := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":false}}}}`)
|
||||
cfg, err := ResolveMCP(ResolveOptions{ProjectConfigPath: project})
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveMCP: %v", err)
|
||||
}
|
||||
if !cfg.Servers["aimlapi"].Disabled {
|
||||
t.Fatal("a project config must not turn on the billed aimlapi default")
|
||||
}
|
||||
|
||||
user := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":false}}}}`)
|
||||
projectDisable := writeMCPDefaultsTestConfig(t, `{"mcp":{"servers":{"aimlapi":{"disabled":true}}}}`)
|
||||
cfg, err = ResolveMCP(ResolveOptions{UserConfigPath: user, ProjectConfigPath: projectDisable})
|
||||
if err != nil {
|
||||
t.Fatalf("ResolveMCP: %v", err)
|
||||
}
|
||||
if cfg.Servers["aimlapi"].Disabled {
|
||||
t.Fatal("the user's explicit enable wins over the project layer, as for any server")
|
||||
}
|
||||
}
|
||||
@@ -455,7 +455,7 @@ func projectAPIKeyEnvAllowed(profile ProviderProfile, envName string) bool {
|
||||
if baseURL == "" {
|
||||
baseURL = descriptor.DefaultBaseURL
|
||||
}
|
||||
if !sameBaseURL(baseURL, descriptor.DefaultBaseURL) {
|
||||
if !providercatalog.IsCatalogBaseURL(descriptor, baseURL) {
|
||||
return false
|
||||
}
|
||||
return containsStringFold(descriptor.AuthEnvVars, envName)
|
||||
@@ -463,7 +463,7 @@ func projectAPIKeyEnvAllowed(profile ProviderProfile, envName string) bool {
|
||||
|
||||
func projectBaseURLAllowed(profile ProviderProfile) bool {
|
||||
descriptor, ok := catalogDescriptorForProfile(profile)
|
||||
return ok && sameBaseURL(profile.BaseURL, descriptor.DefaultBaseURL)
|
||||
return ok && providercatalog.IsCatalogBaseURL(descriptor, profile.BaseURL)
|
||||
}
|
||||
|
||||
func hasInheritedProviderCredentialMaterial(project ProviderProfile, candidate ProviderProfile) bool {
|
||||
@@ -1128,10 +1128,13 @@ func applyCatalogDescriptor(profile *ProviderProfile, descriptor providercatalog
|
||||
if profile.Model == "" {
|
||||
profile.Model = descriptor.DefaultModel
|
||||
}
|
||||
if profile.APIKeyEnv == "" && len(descriptor.AuthEnvVars) > 0 && (!explicitBaseURL || sameBaseURL(profile.BaseURL, descriptor.DefaultBaseURL)) {
|
||||
// A legacy catalog base URL (an earlier default that still serves the
|
||||
// provider, e.g. a profile saved before a host move) counts as the catalog
|
||||
// endpoint, so it keeps the catalog env var binding.
|
||||
canonicalCatalogEndpoint := !explicitBaseURL || providercatalog.IsCatalogBaseURL(descriptor, profile.BaseURL)
|
||||
if profile.APIKeyEnv == "" && len(descriptor.AuthEnvVars) > 0 && canonicalCatalogEndpoint {
|
||||
profile.APIKeyEnv = descriptor.AuthEnvVars[0]
|
||||
}
|
||||
canonicalCatalogEndpoint := !explicitBaseURL || sameBaseURL(profile.BaseURL, descriptor.DefaultBaseURL)
|
||||
if len(descriptor.CustomHeaders) > 0 && canonicalCatalogEndpoint {
|
||||
catalogHeaders := descriptor.CustomHeaders
|
||||
if strings.EqualFold(strings.TrimSpace(descriptor.ID), "aimlapi") {
|
||||
|
||||
@@ -1357,6 +1357,74 @@ func TestResolveAppliesProviderCatalogDefaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveAppliesOpenGatewayCatalogDefaults(t *testing.T) {
|
||||
path := writeConfig(t, `{
|
||||
"activeProvider": "gitlawb-opengateway",
|
||||
"providers": [{
|
||||
"name": "gitlawb-opengateway",
|
||||
"catalogID": "gitlawb-opengateway"
|
||||
}]
|
||||
}`)
|
||||
|
||||
resolved, err := Resolve(ResolveOptions{
|
||||
UserConfigPath: path,
|
||||
Env: map[string]string{
|
||||
"GITLAWB_OPENGATEWAY_API_KEY": "ogw_live_test",
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
}
|
||||
if resolved.Provider.BaseURL != "https://api.twigpine.com/v1" {
|
||||
t.Fatalf("BaseURL = %q, want https://api.twigpine.com/v1", resolved.Provider.BaseURL)
|
||||
}
|
||||
if resolved.Provider.Model != "xiaomi/mimo-v2.6-pro" {
|
||||
t.Fatalf("Model = %q, want xiaomi/mimo-v2.6-pro", resolved.Provider.Model)
|
||||
}
|
||||
if resolved.Provider.APIKeyEnv != "GITLAWB_OPENGATEWAY_API_KEY" || resolved.Provider.APIKey != "ogw_live_test" {
|
||||
t.Fatalf("credential = (%q, %q), want GITLAWB_OPENGATEWAY_API_KEY env-resolved key", resolved.Provider.APIKeyEnv, resolved.Provider.APIKey)
|
||||
}
|
||||
}
|
||||
|
||||
// Profiles saved before the Twigpine rename persist the catalog's old base URL
|
||||
// (https://opengateway.gitlawb.com/v1) explicitly. That host still serves the
|
||||
// gateway, so those configs must keep resolving exactly as before: same host,
|
||||
// same model, catalog env var still bound, credential still picked up.
|
||||
func TestResolveKeepsLegacyOpenGatewayHostProfiles(t *testing.T) {
|
||||
path := writeConfig(t, `{
|
||||
"activeProvider": "gitlawb-opengateway",
|
||||
"providers": [{
|
||||
"name": "gitlawb-opengateway",
|
||||
"catalogID": "gitlawb-opengateway",
|
||||
"providerKind": "openai-compatible",
|
||||
"baseURL": "https://opengateway.gitlawb.com/v1",
|
||||
"model": "mimo-v2.5-pro"
|
||||
}]
|
||||
}`)
|
||||
|
||||
resolved, err := Resolve(ResolveOptions{
|
||||
UserConfigPath: path,
|
||||
Env: map[string]string{
|
||||
"GITLAWB_OPENGATEWAY_API_KEY": "ogw_live_test",
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Resolve() error = %v", err)
|
||||
}
|
||||
if resolved.Provider.BaseURL != "https://opengateway.gitlawb.com/v1" {
|
||||
t.Fatalf("BaseURL = %q, want the saved legacy host kept", resolved.Provider.BaseURL)
|
||||
}
|
||||
if resolved.Provider.Model != "mimo-v2.5-pro" {
|
||||
t.Fatalf("Model = %q, want the saved model kept", resolved.Provider.Model)
|
||||
}
|
||||
if resolved.Provider.APIKeyEnv != "GITLAWB_OPENGATEWAY_API_KEY" {
|
||||
t.Fatalf("APIKeyEnv = %q, want GITLAWB_OPENGATEWAY_API_KEY", resolved.Provider.APIKeyEnv)
|
||||
}
|
||||
if resolved.Provider.APIKey != "ogw_live_test" {
|
||||
t.Fatalf("APIKey = %q, want env-resolved key", resolved.Provider.APIKey)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveAppliesMiniMaxCNCatalogDefaults(t *testing.T) {
|
||||
path := writeConfig(t, `{
|
||||
"activeProvider": "mini-cn",
|
||||
|
||||
@@ -724,6 +724,39 @@ func (cfg LocalControlDriverConfig) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(raw)
|
||||
}
|
||||
|
||||
// MarshalJSON writes "disabled" whenever it was explicitly set, not only when
|
||||
// true. A read-modify-write of config.json (SetTheme, SetActiveProvider, ...)
|
||||
// would otherwise drop `"disabled": false`, which turns a default that ships
|
||||
// disabled (aimlapi) back off and lets project config override a user's
|
||||
// explicit enable (see mergeMCPServer).
|
||||
func (server MCPServerConfig) MarshalJSON() ([]byte, error) {
|
||||
type rawServer struct {
|
||||
Type string `json:"type,omitempty"`
|
||||
Command string `json:"command,omitempty"`
|
||||
Args []string `json:"args,omitempty"`
|
||||
Env map[string]string `json:"env,omitempty"`
|
||||
URL string `json:"url,omitempty"`
|
||||
Headers map[string]string `json:"headers,omitempty"`
|
||||
Auth string `json:"auth,omitempty"`
|
||||
OAuth *MCPOAuthConfig `json:"oauth,omitempty"`
|
||||
Disabled *bool `json:"disabled,omitempty"`
|
||||
}
|
||||
raw := rawServer{
|
||||
Type: server.Type,
|
||||
Command: server.Command,
|
||||
Args: server.Args,
|
||||
Env: server.Env,
|
||||
URL: server.URL,
|
||||
Headers: server.Headers,
|
||||
Auth: server.Auth,
|
||||
OAuth: server.OAuth,
|
||||
}
|
||||
if server.disabledSet || server.Disabled {
|
||||
raw.Disabled = &server.Disabled
|
||||
}
|
||||
return json.Marshal(raw)
|
||||
}
|
||||
|
||||
func (server *MCPServerConfig) UnmarshalJSON(data []byte) error {
|
||||
type rawServer struct {
|
||||
Type string `json:"type"`
|
||||
|
||||
@@ -779,7 +779,7 @@ func normalizeDefinition(raw any, field string) (Definition, error) {
|
||||
return Definition{}, err
|
||||
}
|
||||
if matcher != "" && !eventSupportsMatcher(event) {
|
||||
return Definition{}, manifestError{fieldPath: field + ".matcher", message: "matcher is only supported for beforeTool and afterTool hooks."}
|
||||
return Definition{}, manifestError{fieldPath: field + ".matcher", message: "matcher is only supported for beforeTool, afterTool, specialistStart, and specialistStop hooks."}
|
||||
}
|
||||
command, err := requiredString(obj, field+".command")
|
||||
if err != nil {
|
||||
@@ -843,7 +843,7 @@ func matchesHookMatcher(matcher string, toolName string) bool {
|
||||
}
|
||||
|
||||
func eventSupportsMatcher(event Event) bool {
|
||||
return event == EventBeforeTool || event == EventAfterTool
|
||||
return event == EventBeforeTool || event == EventAfterTool || event == EventSpecialistStart || event == EventSpecialistStop
|
||||
}
|
||||
|
||||
func toDiagnostic(err error, source ConfigSource, path string) Diagnostic {
|
||||
|
||||
@@ -180,7 +180,7 @@ func TestLoadConfigPreservesUserDisabledStateWhenProjectOmitsEnabled(t *testing.
|
||||
}
|
||||
|
||||
func TestLoadConfigRejectsMatchersOnLifecycleHooks(t *testing.T) {
|
||||
for _, event := range []string{"sessionStart", "specialistStart"} {
|
||||
for _, event := range []string{"sessionStart", "sessionEnd"} {
|
||||
t.Run(event, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
projectConfigPath := filepath.Join(dir, "hooks.json")
|
||||
@@ -210,6 +210,46 @@ func TestLoadConfigRejectsMatchersOnLifecycleHooks(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadConfigAcceptsMatchersOnSpecialistHooks(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
event Event
|
||||
id string
|
||||
}{
|
||||
{event: EventSpecialistStart, id: "zero.explorer-start"},
|
||||
{event: EventSpecialistStop, id: "zero.explorer-stop"},
|
||||
} {
|
||||
t.Run(string(tc.event), func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
projectConfigPath := filepath.Join(dir, "hooks.json")
|
||||
writeHookJSON(t, projectConfigPath, map[string]any{
|
||||
"hooks": []any{map[string]any{
|
||||
"id": tc.id,
|
||||
"event": string(tc.event),
|
||||
"matcher": "explorer",
|
||||
"command": "node",
|
||||
}},
|
||||
})
|
||||
|
||||
result, err := LoadConfig(LoadOptions{
|
||||
UserConfigPath: filepath.Join(dir, "missing-user-hooks.json"),
|
||||
ProjectConfigPath: projectConfigPath,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("LoadConfig returned error: %v", err)
|
||||
}
|
||||
if len(result.Config.Hooks) != 1 || result.Config.Hooks[0].Matcher != "explorer" {
|
||||
t.Fatalf("specialist matcher hook = %#v", result.Config.Hooks)
|
||||
}
|
||||
if got := hookIDs(Select(result.Config, SelectInput{Event: tc.event, ToolName: "explorer"})); !reflect.DeepEqual(got, []string{tc.id}) {
|
||||
t.Fatalf("matched selection = %#v", got)
|
||||
}
|
||||
if got := Select(result.Config, SelectInput{Event: tc.event, ToolName: "worker"}); len(got) != 0 {
|
||||
t.Fatalf("unmatched specialist should be empty, got %#v", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfigStorePersistsUpdates(t *testing.T) {
|
||||
configPath := filepath.Join(t.TempDir(), "hooks.json")
|
||||
store, err := NewConfigStore(StoreOptions{ConfigPath: configPath})
|
||||
|
||||
@@ -0,0 +1,287 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/redaction"
|
||||
)
|
||||
|
||||
// The AI/ML API MCP server authenticates with the user's AI/ML API key.
|
||||
//
|
||||
// Invariant: the key and the attribution headers are attached only to an
|
||||
// outgoing request whose own URL is https://mcp.aimlapi.com on the default port
|
||||
// with no userinfo (exact hostname, see aimlapi.IsAttributedURL), whatever the
|
||||
// server is named. The key is resolved again for every request, redirect hops
|
||||
// included (see reapplyAIMLAPIRedirectHeaders), through
|
||||
// config.LoadAIMLAPIServiceKey, which in the same call decides whether it may
|
||||
// be sent (no proxy profile, a single token), so a provider change applies to
|
||||
// the next request and a withheld or invalid key means no Authorization
|
||||
// header. It lives only in that request's headers, and for the length of the
|
||||
// call in the collector its context carries (aimlapiSentKeys), so the reply can
|
||||
// be scrubbed: never in Server, its Identity (OAuth token binding), config.json,
|
||||
// list output, errors or tool output. Every key the call sent, redirect hops
|
||||
// included, and on the SSE transport the keys the stream request sent, is
|
||||
// scrubbed from the response's error text, method and result before anything
|
||||
// reads them (scrubAIMLAPIKeys) and from any error the call returns, a
|
||||
// transport error quoting a URL the server chose among them
|
||||
// (scrubAIMLAPIKeyError). An error status to such a request is also reported
|
||||
// without its body. A user-configured Authorization header, OAuth, or a
|
||||
// project-touched server always goes without it.
|
||||
var loadAIMLAPIServiceKey = config.LoadAIMLAPIServiceKey
|
||||
|
||||
var aimlapiMCPHost = func() string {
|
||||
parsed, err := url.Parse(aimlapi.MCPServerURL)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return parsed.Hostname()
|
||||
}()
|
||||
|
||||
func isAIMLAPIMCPURL(target *url.URL) bool {
|
||||
return aimlapi.IsAttributedURL(target) && strings.EqualFold(target.Hostname(), aimlapiMCPHost)
|
||||
}
|
||||
|
||||
// aimlapiAutomaticHeaders are the headers applyAIMLAPIRequestHeaders may add.
|
||||
var aimlapiAutomaticHeaders = [...]string{"Authorization", aimlapi.SourceHeaderName, aimlapi.PartnerHeaderName}
|
||||
|
||||
// applyAIMLAPIRequestHeaders runs after the server's own headers are set.
|
||||
func applyAIMLAPIRequestHeaders(server Server, request *http.Request) {
|
||||
if request == nil || !isAIMLAPIMCPURL(request.URL) {
|
||||
return
|
||||
}
|
||||
aimlapi.ApplyAttributionHeaders(request.Header, request.URL)
|
||||
if request.Header.Get("Authorization") != "" || strings.EqualFold(server.Auth, ServerAuthOAuth) || server.ProjectConfigured {
|
||||
return
|
||||
}
|
||||
key, err := loadAIMLAPIServiceKey()
|
||||
if err != nil || key == "" {
|
||||
return
|
||||
}
|
||||
request.Header.Set("Authorization", "Bearer "+key)
|
||||
if sent, ok := request.Context().Value(aimlapiSentKeysContextKey{}).(*aimlapiSentKeys); ok {
|
||||
sent.add(key)
|
||||
}
|
||||
}
|
||||
|
||||
// reapplyAIMLAPIRedirectHeaders gives a redirect hop the automatic headers its
|
||||
// own URL earns. net/http copies the first request's headers onto every hop
|
||||
// without calling the header hooks, so without this a key or attribution added
|
||||
// for the first URL would follow a same-origin hop that fails isAIMLAPIMCPURL
|
||||
// (userinfo, say), and an eligible hop would reuse the first hop's key instead
|
||||
// of resolving it again. The hop gets back the server's own values for these
|
||||
// headers, then the same rule as any other request.
|
||||
func reapplyAIMLAPIRedirectHeaders(server Server, request *http.Request, first *http.Request) {
|
||||
if request == nil || first == nil || !isAIMLAPIMCPURL(first.URL) {
|
||||
return
|
||||
}
|
||||
for _, name := range aimlapiAutomaticHeaders {
|
||||
request.Header.Del(name)
|
||||
}
|
||||
for name, value := range server.Headers {
|
||||
for _, automatic := range aimlapiAutomaticHeaders {
|
||||
if strings.EqualFold(name, automatic) {
|
||||
request.Header.Set(name, value)
|
||||
}
|
||||
}
|
||||
}
|
||||
applyAIMLAPIRequestHeaders(server, request)
|
||||
}
|
||||
|
||||
// sentAIMLAPIServiceKey reports whether request went out carrying the key
|
||||
// applyAIMLAPIRequestHeaders injects, as opposed to the server's own
|
||||
// Authorization header or OAuth.
|
||||
func sentAIMLAPIServiceKey(server Server, request *http.Request) bool {
|
||||
if request == nil || !isAIMLAPIMCPURL(request.URL) || strings.EqualFold(server.Auth, ServerAuthOAuth) || server.ProjectConfigured {
|
||||
return false
|
||||
}
|
||||
return request.Header.Get("Authorization") != "" && !hasUserAuthorizationHeader(server)
|
||||
}
|
||||
|
||||
func hasUserAuthorizationHeader(server Server) bool {
|
||||
for name, value := range server.Headers {
|
||||
if strings.EqualFold(name, "Authorization") && value != "" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// aimlapiSentKeys collects every key applyAIMLAPIRequestHeaders puts on the
|
||||
// requests of one call. The first request's context carries it, and net/http
|
||||
// gives each redirect hop that same context, so a key resolved again for a later
|
||||
// hop is collected too. The final hop's key alone is not enough: a same-origin
|
||||
// server answering the last hop has seen every earlier one.
|
||||
type aimlapiSentKeys struct {
|
||||
mu sync.Mutex
|
||||
keys []string
|
||||
}
|
||||
|
||||
type aimlapiSentKeysContextKey struct{}
|
||||
|
||||
// withAIMLAPISentKeys returns ctx carrying a fresh collector for one call.
|
||||
func withAIMLAPISentKeys(ctx context.Context) (context.Context, *aimlapiSentKeys) {
|
||||
sent := &aimlapiSentKeys{}
|
||||
return context.WithValue(ctx, aimlapiSentKeysContextKey{}, sent), sent
|
||||
}
|
||||
|
||||
func (sent *aimlapiSentKeys) add(key string) {
|
||||
sent.mu.Lock()
|
||||
defer sent.mu.Unlock()
|
||||
for _, existing := range sent.keys {
|
||||
if existing == key {
|
||||
return
|
||||
}
|
||||
}
|
||||
sent.keys = append(sent.keys, key)
|
||||
}
|
||||
|
||||
// list returns a copy of the keys collected so far.
|
||||
func (sent *aimlapiSentKeys) list() []string {
|
||||
sent.mu.Lock()
|
||||
defer sent.mu.Unlock()
|
||||
return append([]string(nil), sent.keys...)
|
||||
}
|
||||
|
||||
// scrubAIMLAPIKeys removes the injected keys from a response before any of it
|
||||
// is read: the error text, the method name and every string in the result,
|
||||
// object member names included. A server can echo the credential it was sent,
|
||||
// and MCP reports a failed tool call as a result rather than an error, so the
|
||||
// result needs it as much as the error text. Only the known values go, split
|
||||
// forms included (redaction.RedactSecretValues); a result that holds none of
|
||||
// them keeps its original bytes.
|
||||
func scrubAIMLAPIKeys(message rpcMessage, keys ...string) rpcMessage {
|
||||
var sent []string
|
||||
for _, key := range keys {
|
||||
if strings.TrimSpace(key) != "" {
|
||||
sent = append(sent, key)
|
||||
}
|
||||
}
|
||||
if len(sent) == 0 {
|
||||
return message
|
||||
}
|
||||
if message.Error != nil {
|
||||
scrubbed := *message.Error
|
||||
scrubbed.Message = redaction.RedactSecretValues(scrubbed.Message, sent...)
|
||||
message.Error = &scrubbed
|
||||
}
|
||||
message.Method = redaction.RedactSecretValues(message.Method, sent...)
|
||||
message.Result = scrubAIMLAPIKeysFromJSON(message.Result, sent)
|
||||
return message
|
||||
}
|
||||
|
||||
func scrubAIMLAPIKeysFromJSON(raw json.RawMessage, keys []string) json.RawMessage {
|
||||
if len(raw) == 0 {
|
||||
return raw
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(raw))
|
||||
decoder.UseNumber()
|
||||
var value any
|
||||
if err := decoder.Decode(&value); err != nil {
|
||||
// Decoding it into the caller's type fails the same way, and that
|
||||
// error does not quote the text.
|
||||
return raw
|
||||
}
|
||||
scrubbed, changed := scrubAIMLAPIKeysFromValue(value, keys)
|
||||
if !changed {
|
||||
return raw
|
||||
}
|
||||
encoded, err := json.Marshal(scrubbed)
|
||||
if err != nil {
|
||||
// Never hand back the unscrubbed bytes: an empty result reads as
|
||||
// nothing rather than as the key.
|
||||
return nil
|
||||
}
|
||||
return encoded
|
||||
}
|
||||
|
||||
func scrubAIMLAPIKeysFromValue(value any, keys []string) (any, bool) {
|
||||
switch typed := value.(type) {
|
||||
case string:
|
||||
scrubbed := redaction.RedactSecretValues(typed, keys...)
|
||||
return scrubbed, scrubbed != typed
|
||||
case []any:
|
||||
changed := false
|
||||
for index, item := range typed {
|
||||
if scrubbed, itemChanged := scrubAIMLAPIKeysFromValue(item, keys); itemChanged {
|
||||
typed[index] = scrubbed
|
||||
changed = true
|
||||
}
|
||||
}
|
||||
return typed, changed
|
||||
case map[string]any:
|
||||
changed := false
|
||||
scrubbed := make(map[string]any, len(typed))
|
||||
for name, item := range typed {
|
||||
scrubbedName := redaction.RedactSecretValues(name, keys...)
|
||||
scrubbedItem, itemChanged := scrubAIMLAPIKeysFromValue(item, keys)
|
||||
if itemChanged || scrubbedName != name {
|
||||
changed = true
|
||||
}
|
||||
scrubbed[scrubbedName] = scrubbedItem
|
||||
}
|
||||
return scrubbed, changed
|
||||
}
|
||||
return value, false
|
||||
}
|
||||
|
||||
// scrubAIMLAPIKeyError returns err with the injected keys removed from its text.
|
||||
// An error the keys do not appear in keeps its chain. One they do appear in is
|
||||
// replaced by its scrubbed text, because its chain still holds the key (a
|
||||
// *url.Error keeps the URL the server chose); the replacement unwraps only to
|
||||
// the context sentinel the original matched, so a cancellation still reads as
|
||||
// one.
|
||||
func scrubAIMLAPIKeyError(err error, keys ...string) error {
|
||||
if err == nil || len(keys) == 0 {
|
||||
return err
|
||||
}
|
||||
text := err.Error()
|
||||
scrubbed := redaction.RedactSecretValues(text, keys...)
|
||||
if scrubbed == text {
|
||||
return err
|
||||
}
|
||||
replacement := &aimlapiScrubbedError{text: scrubbed}
|
||||
for _, sentinel := range []error{context.Canceled, context.DeadlineExceeded} {
|
||||
if errors.Is(err, sentinel) {
|
||||
replacement.cause = sentinel
|
||||
break
|
||||
}
|
||||
}
|
||||
return replacement
|
||||
}
|
||||
|
||||
// aimlapiScrubbedError is an error whose text had an injected key removed.
|
||||
type aimlapiScrubbedError struct {
|
||||
text string
|
||||
cause error
|
||||
}
|
||||
|
||||
func (scrubbed *aimlapiScrubbedError) Error() string { return scrubbed.text }
|
||||
|
||||
func (scrubbed *aimlapiScrubbedError) Unwrap() error { return scrubbed.cause }
|
||||
|
||||
// aimlapiUnauthorizedHint explains how to authenticate when the AI/ML API MCP
|
||||
// server rejects a request without OAuth. It names no key material.
|
||||
func aimlapiUnauthorizedHint(server Server, status int) string {
|
||||
if status != http.StatusUnauthorized || strings.EqualFold(server.Auth, ServerAuthOAuth) {
|
||||
return ""
|
||||
}
|
||||
target, err := url.Parse(server.URL)
|
||||
if err != nil || !isAIMLAPIMCPURL(target) {
|
||||
return ""
|
||||
}
|
||||
oauthHint := fmt.Sprintf("To sign in instead, set \"auth\": \"oauth\" on this server and run `zero mcp oauth login %s`", server.Name)
|
||||
if server.ProjectConfigured || hasUserAuthorizationHeader(server) {
|
||||
return " (" + oauthHint + ")"
|
||||
}
|
||||
return fmt.Sprintf(" (set up the aimlapi.com provider with `zero providers setup aimlapi` or set AIMLAPI_API_KEY; automatic injection is also disabled for proxy profiles, OAuth, project config, and non-empty Authorization headers — see docs/EXTENDING.md#built-in-servers. To sign in instead, set \"auth\": \"oauth\" on this server and run `zero mcp oauth login %s`)", server.Name)
|
||||
}
|
||||
@@ -0,0 +1,658 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
)
|
||||
|
||||
// scrubTestKey has no credential shape, so only the known-value scrub can find
|
||||
// it; a shape pattern would leave it alone.
|
||||
const scrubTestKey = "aiml0123456789abcdef0123456789abcd"
|
||||
|
||||
// aimlapiEcho stands in for mcp.aimlapi.com over streamable HTTP. It answers
|
||||
// each JSON-RPC request with reply(method, id), as a JSON body or as one SSE
|
||||
// event, and records the Authorization header every request carried.
|
||||
type aimlapiEcho struct {
|
||||
sse bool
|
||||
reply func(method string, id json.RawMessage) string
|
||||
mu sync.Mutex
|
||||
auth []string
|
||||
}
|
||||
|
||||
func (echo *aimlapiEcho) RoundTrip(request *http.Request) (*http.Response, error) {
|
||||
var body []byte
|
||||
if request.Body != nil {
|
||||
body, _ = io.ReadAll(request.Body)
|
||||
_ = request.Body.Close()
|
||||
}
|
||||
echo.mu.Lock()
|
||||
echo.auth = append(echo.auth, request.Header.Get("Authorization"))
|
||||
echo.mu.Unlock()
|
||||
var message struct {
|
||||
ID json.RawMessage `json:"id"`
|
||||
Method string `json:"method"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &message)
|
||||
if len(message.ID) == 0 {
|
||||
return fakeHTTPResponse(request, http.StatusAccepted, ""), nil
|
||||
}
|
||||
reply := echo.reply(message.Method, message.ID)
|
||||
response := fakeHTTPResponse(request, http.StatusOK, reply)
|
||||
if echo.sse {
|
||||
response.Header.Set("Content-Type", "text/event-stream")
|
||||
response.Body = io.NopCloser(strings.NewReader("event: message\ndata: " + reply + "\n\n"))
|
||||
}
|
||||
return response, nil
|
||||
}
|
||||
|
||||
// assertSent fails unless every request carried want as its Authorization, so a
|
||||
// test cannot pass by never reaching the injected-key path.
|
||||
func (echo *aimlapiEcho) assertSent(t *testing.T, want string) {
|
||||
t.Helper()
|
||||
echo.mu.Lock()
|
||||
defer echo.mu.Unlock()
|
||||
if len(echo.auth) == 0 {
|
||||
t.Fatal("the fake server was never called")
|
||||
}
|
||||
for _, got := range echo.auth {
|
||||
if got != want {
|
||||
t.Fatalf("Authorization = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func rpcReply(id json.RawMessage, field string, value any) string {
|
||||
encoded, err := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": id, field: value})
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return string(encoded)
|
||||
}
|
||||
|
||||
// echoedKeys are the spellings of the key a server echoes back: whole, and
|
||||
// split by a NUL and by a zero-width space, which a plain replace would miss.
|
||||
func echoedKeys() []string {
|
||||
return []string{
|
||||
scrubTestKey,
|
||||
scrubTestKey[:10] + string(rune(0)) + scrubTestKey[10:],
|
||||
scrubTestKey[:20] + string(rune(0x200b)) + scrubTestKey[20:],
|
||||
}
|
||||
}
|
||||
|
||||
// assertNoPartOf fails if any eight-byte run of key survives in text.
|
||||
func assertNoPartOf(t *testing.T, text string, key string) {
|
||||
t.Helper()
|
||||
for start := 0; start+8 <= len(key); start++ {
|
||||
if strings.Contains(text, key[start:start+8]) {
|
||||
t.Fatalf("text still holds part of the key (%q): %q", key[start:start+8], text)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// assertKeyScrubbed fails if any part of scrubTestKey survives in text, or if
|
||||
// text does not show the replacement.
|
||||
func assertKeyScrubbed(t *testing.T, text string) {
|
||||
t.Helper()
|
||||
assertNoPartOf(t, text, scrubTestKey)
|
||||
if !strings.Contains(text, "[REDACTED]") {
|
||||
t.Fatalf("text = %q, want the key replaced by [REDACTED]", text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestScrubAIMLAPIKeysKeepsAResultWithoutTheKeyByteForByte(t *testing.T) {
|
||||
raw := json.RawMessage(`{"tools": [ {"name": "b", "description": "password=hunter2"} ], "a": 1.50}`)
|
||||
if got := scrubAIMLAPIKeysFromJSON(raw, []string{scrubTestKey}); string(got) != string(raw) {
|
||||
t.Fatalf("result without the key = %s, want the original bytes %s", got, raw)
|
||||
}
|
||||
|
||||
withKey := json.RawMessage(`{"content": [ {"type": "text", "text": "key ` + scrubTestKey + `"} ], "n": 1.50}`)
|
||||
got := scrubAIMLAPIKeysFromJSON(withKey, []string{scrubTestKey})
|
||||
assertKeyScrubbed(t, string(got))
|
||||
var decoded struct {
|
||||
Content []Content `json:"content"`
|
||||
N json.Number `json:"n"`
|
||||
}
|
||||
if err := json.Unmarshal(got, &decoded); err != nil || len(decoded.Content) != 1 || decoded.Content[0].Text != "key [REDACTED]" || decoded.N != "1.50" {
|
||||
t.Fatalf("scrubbed result = %s (%v), want valid JSON with only the key replaced and the number as written", got, err)
|
||||
}
|
||||
}
|
||||
|
||||
func newEchoNetworkClient(server Server, echo *aimlapiEcho) *networkClient {
|
||||
return &networkClient{server: server, client: mcpHTTPClient(server, echo), nextID: 1}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPScrubsTheInjectedKeyFromResponses(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, scrubTestKey)
|
||||
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
for _, sse := range []bool{false, true} {
|
||||
transport := "json"
|
||||
if sse {
|
||||
transport = "sse"
|
||||
}
|
||||
t.Run(transport+"/error message", func(t *testing.T) {
|
||||
for _, echoed := range echoedKeys() {
|
||||
echo := &aimlapiEcho{sse: sse, reply: func(_ string, id json.RawMessage) string {
|
||||
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "invalid credential " + echoed + " for this account"})
|
||||
}}
|
||||
err := newEchoNetworkClient(server, echo).initialize(context.Background())
|
||||
if err == nil {
|
||||
t.Fatal("initialize succeeded against a server that answered with an error")
|
||||
}
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey)
|
||||
assertKeyScrubbed(t, err.Error())
|
||||
if !strings.Contains(err.Error(), "invalid credential [REDACTED] for this account") {
|
||||
t.Fatalf("error = %q, want the rest of the server text kept", err)
|
||||
}
|
||||
}
|
||||
})
|
||||
t.Run(transport+"/tool result", func(t *testing.T) {
|
||||
const image = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="
|
||||
for _, echoed := range echoedKeys() {
|
||||
echo := &aimlapiEcho{sse: sse, reply: func(_ string, id json.RawMessage) string {
|
||||
return rpcReply(id, "result", map[string]any{
|
||||
"isError": true,
|
||||
"content": []any{
|
||||
map[string]any{"type": "text", "text": "upstream rejected key " + echoed},
|
||||
map[string]any{"type": "image", "mimeType": "image/png", "data": image},
|
||||
},
|
||||
})
|
||||
}}
|
||||
result, err := newEchoNetworkClient(server, echo).CallTool(context.Background(), "generate", map[string]any{"prompt": "x"})
|
||||
if err != nil {
|
||||
t.Fatalf("CallTool: %v", err)
|
||||
}
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey)
|
||||
if len(result.Content) != 2 || !result.IsError {
|
||||
t.Fatalf("result = %+v, want the error result with both blocks", result)
|
||||
}
|
||||
assertKeyScrubbed(t, result.Content[0].Text)
|
||||
if result.Content[0].Text != "upstream rejected key [REDACTED]" {
|
||||
t.Fatalf("text block = %q, want only the key replaced", result.Content[0].Text)
|
||||
}
|
||||
if result.Content[1].Data != image {
|
||||
t.Fatalf("image data changed: %q", result.Content[1].Data)
|
||||
}
|
||||
}
|
||||
})
|
||||
t.Run(transport+"/result without the key", func(t *testing.T) {
|
||||
// Shapes stay as the server wrote them: only the known value goes.
|
||||
const description = "Set password=hunter2 or pass Authorization: Bearer other-token"
|
||||
echo := &aimlapiEcho{sse: sse, reply: func(_ string, id json.RawMessage) string {
|
||||
return rpcReply(id, "result", map[string]any{"tools": []any{map[string]any{
|
||||
"name": "generate",
|
||||
"description": description,
|
||||
"inputSchema": map[string]any{"type": "object", "properties": map[string]any{"api_key": map[string]any{"type": "string"}}},
|
||||
}}})
|
||||
}}
|
||||
tools, err := newEchoNetworkClient(server, echo).ListTools(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("ListTools: %v", err)
|
||||
}
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey)
|
||||
if len(tools) != 1 || tools[0].Description != description {
|
||||
t.Fatalf("tools = %+v, want the description unchanged", tools)
|
||||
}
|
||||
properties, _ := tools[0].InputSchema["properties"].(map[string]any)
|
||||
if _, ok := properties["api_key"]; !ok {
|
||||
t.Fatalf("inputSchema = %v, want the api_key property kept", tools[0].InputSchema)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("json/unexpected method", func(t *testing.T) {
|
||||
echo := &aimlapiEcho{reply: func(string, json.RawMessage) string {
|
||||
return `{"jsonrpc":"2.0","method":"notifications/` + scrubTestKey + `"}`
|
||||
}}
|
||||
err := newEchoNetworkClient(server, echo).initialize(context.Background())
|
||||
if err == nil || !strings.Contains(err.Error(), "got method") {
|
||||
t.Fatalf("initialize error = %v, want the unexpected method reported", err)
|
||||
}
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey)
|
||||
assertKeyScrubbed(t, err.Error())
|
||||
})
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPLeavesResponsesAloneWithoutTheInjectedKey(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, scrubTestKey)
|
||||
// A user-configured Authorization header means nothing was injected, so the
|
||||
// server text is reported as written; this change scrubs only what Zero
|
||||
// itself attached.
|
||||
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
server.Headers = map[string]string{"Authorization": "Bearer " + scrubTestKey}
|
||||
echo := &aimlapiEcho{reply: func(_ string, id json.RawMessage) string {
|
||||
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "invalid credential " + scrubTestKey})
|
||||
}}
|
||||
err := newEchoNetworkClient(server, echo).initialize(context.Background())
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey)
|
||||
if err == nil || !strings.Contains(err.Error(), "invalid credential "+scrubTestKey) {
|
||||
t.Fatalf("error = %v, want the server text unchanged when no key was injected", err)
|
||||
}
|
||||
}
|
||||
|
||||
// aimlapiSSEEcho stands in for mcp.aimlapi.com over the legacy SSE transport:
|
||||
// the GET opens a stream that starts with the endpoint event, and each POST is
|
||||
// answered on that stream with reply(method, id).
|
||||
type aimlapiSSEEcho struct {
|
||||
endpoint string
|
||||
reply func(method string, id json.RawMessage) string
|
||||
// postErr, when set, fails every POST at the transport.
|
||||
postErr error
|
||||
// rawEvent, when set, writes its whole event to the stream instead of a
|
||||
// message event built from reply.
|
||||
rawEvent func(method string, id json.RawMessage) string
|
||||
events chan string
|
||||
mu sync.Mutex
|
||||
auth []string
|
||||
}
|
||||
|
||||
func newAIMLAPISSEEcho(endpoint string, reply func(string, json.RawMessage) string) *aimlapiSSEEcho {
|
||||
return &aimlapiSSEEcho{endpoint: endpoint, reply: reply, events: make(chan string, 8)}
|
||||
}
|
||||
|
||||
func (echo *aimlapiSSEEcho) RoundTrip(request *http.Request) (*http.Response, error) {
|
||||
echo.mu.Lock()
|
||||
echo.auth = append(echo.auth, request.Header.Get("Authorization"))
|
||||
echo.mu.Unlock()
|
||||
if request.Method == http.MethodGet {
|
||||
reader, writer := io.Pipe()
|
||||
go func() {
|
||||
defer writer.Close()
|
||||
if _, err := io.WriteString(writer, "event: endpoint\ndata: "+echo.endpoint+"\n\n"); err != nil {
|
||||
return
|
||||
}
|
||||
for {
|
||||
select {
|
||||
case event := <-echo.events:
|
||||
if _, err := io.WriteString(writer, event); err != nil {
|
||||
return
|
||||
}
|
||||
case <-request.Context().Done():
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
response := fakeHTTPResponse(request, http.StatusOK, "")
|
||||
response.Header.Set("Content-Type", "text/event-stream")
|
||||
response.Body = reader
|
||||
return response, nil
|
||||
}
|
||||
if echo.postErr != nil {
|
||||
return nil, echo.postErr
|
||||
}
|
||||
var body []byte
|
||||
if request.Body != nil {
|
||||
body, _ = io.ReadAll(request.Body)
|
||||
_ = request.Body.Close()
|
||||
}
|
||||
var message struct {
|
||||
ID json.RawMessage `json:"id"`
|
||||
Method string `json:"method"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &message)
|
||||
if len(message.ID) > 0 {
|
||||
if echo.rawEvent != nil {
|
||||
echo.events <- echo.rawEvent(message.Method, message.ID)
|
||||
} else {
|
||||
echo.events <- "event: message\ndata: " + echo.reply(message.Method, message.ID) + "\n\n"
|
||||
}
|
||||
}
|
||||
return fakeHTTPResponse(request, http.StatusAccepted, ""), nil
|
||||
}
|
||||
|
||||
// rotateAIMLAPIKeys makes the key resolver hand out keys in order, repeating the
|
||||
// last, the way a provider change between two requests of one call would.
|
||||
func rotateAIMLAPIKeys(t *testing.T, keys ...string) {
|
||||
t.Helper()
|
||||
var mu sync.Mutex
|
||||
previous := loadAIMLAPIServiceKey
|
||||
loadAIMLAPIServiceKey = func() (string, error) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
key := keys[0]
|
||||
if len(keys) > 1 {
|
||||
keys = keys[1:]
|
||||
}
|
||||
return key, nil
|
||||
}
|
||||
t.Cleanup(func() { loadAIMLAPIServiceKey = previous })
|
||||
}
|
||||
|
||||
func (echo *aimlapiSSEEcho) assertSent(t *testing.T, want string, requests int) {
|
||||
t.Helper()
|
||||
echo.mu.Lock()
|
||||
defer echo.mu.Unlock()
|
||||
if len(echo.auth) < requests {
|
||||
t.Fatalf("the fake server saw %d requests, want at least %d", len(echo.auth), requests)
|
||||
}
|
||||
for _, got := range echo.auth {
|
||||
if got != want {
|
||||
t.Fatalf("Authorization = %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func newEchoSSEClient(echo *aimlapiSSEEcho) *remoteSSEClient {
|
||||
server := Server{Name: "aimlapi", Type: ServerTypeSSE, URL: aimlapi.MCPServerURL}
|
||||
return &remoteSSEClient{
|
||||
server: server,
|
||||
client: mcpHTTPClient(server, echo),
|
||||
nextID: 1,
|
||||
pending: map[string]chan ssePendingResponse{},
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPSSEScrubsTheInjectedKeyFromStreamAnswers(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, scrubTestKey)
|
||||
for _, echoed := range echoedKeys() {
|
||||
echo := newAIMLAPISSEEcho("/messages", func(method string, id json.RawMessage) string {
|
||||
if method == "tools/call" {
|
||||
return rpcReply(id, "result", map[string]any{"isError": true, "content": []any{map[string]any{"type": "text", "text": "rejected " + echoed}}})
|
||||
}
|
||||
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "invalid credential " + echoed})
|
||||
})
|
||||
client := newEchoSSEClient(echo)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
if err := client.openStream(ctx); err != nil {
|
||||
cancel()
|
||||
t.Fatalf("openStream: %v", err)
|
||||
}
|
||||
_, listErr := client.ListTools(ctx)
|
||||
result, callErr := client.CallTool(ctx, "generate", map[string]any{})
|
||||
_ = client.Close()
|
||||
cancel()
|
||||
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey, 3)
|
||||
if listErr == nil {
|
||||
t.Fatal("ListTools succeeded against a server that answered with an error")
|
||||
}
|
||||
assertKeyScrubbed(t, listErr.Error())
|
||||
if callErr != nil || len(result.Content) != 1 {
|
||||
t.Fatalf("CallTool = %+v, %v, want the error result", result, callErr)
|
||||
}
|
||||
assertKeyScrubbed(t, result.Content[0].Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPSSEScrubsTheStreamKeyAndThePostKey(t *testing.T) {
|
||||
// The key is resolved for every request, so the stream and a later POST can
|
||||
// carry different keys when the provider changes in between. An answer on
|
||||
// the stream comes from a server that was sent both.
|
||||
const streamKey = "strm9f8e7d6c5b4a39281706f5e4d3c2b1"
|
||||
const postKey = "post1a2b3c4d5e6f708192a3b4c5d6e7f8"
|
||||
rotateAIMLAPIKeys(t, streamKey, postKey)
|
||||
|
||||
echo := newAIMLAPISSEEcho("/messages", func(_ string, id json.RawMessage) string {
|
||||
return rpcReply(id, "error", map[string]any{"code": -32001, "message": "stream key " + streamKey + ", post key " + postKey})
|
||||
})
|
||||
client := newEchoSSEClient(echo)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := client.openStream(ctx); err != nil {
|
||||
t.Fatalf("openStream: %v", err)
|
||||
}
|
||||
_, err := client.ListTools(ctx)
|
||||
_ = client.Close()
|
||||
|
||||
echo.mu.Lock()
|
||||
auth := append([]string(nil), echo.auth...)
|
||||
echo.mu.Unlock()
|
||||
if len(auth) != 2 || auth[0] != "Bearer "+streamKey || auth[1] != "Bearer "+postKey {
|
||||
t.Fatalf("Authorization headers = %q, want the stream key and then the post key", auth)
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("ListTools succeeded against a server that answered with an error")
|
||||
}
|
||||
assertNoPartOf(t, err.Error(), streamKey)
|
||||
assertNoPartOf(t, err.Error(), postKey)
|
||||
if !strings.Contains(err.Error(), "stream key [REDACTED], post key [REDACTED]") {
|
||||
t.Fatalf("error = %q, want both keys replaced", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPSSEScrubsTheInjectedKeyFromAnEndpointError(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, scrubTestKey)
|
||||
// An invalid escape makes the endpoint fail to parse, and the parse error
|
||||
// quotes the endpoint the server sent.
|
||||
echo := newAIMLAPISSEEcho("/messages/"+scrubTestKey+"%zz", func(string, json.RawMessage) string { return "" })
|
||||
client := newEchoSSEClient(echo)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
err := client.openStream(ctx)
|
||||
_ = client.Close()
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey, 1)
|
||||
if err == nil || !strings.Contains(err.Error(), "endpoint") {
|
||||
t.Fatalf("openStream error = %v, want the endpoint parse failure", err)
|
||||
}
|
||||
assertKeyScrubbed(t, err.Error())
|
||||
}
|
||||
|
||||
// aimlapiHopEcho answers the first request with a 307 to location and hands the
|
||||
// second hop to second, recording the Authorization each hop carried.
|
||||
type aimlapiHopEcho struct {
|
||||
location string
|
||||
second func(request *http.Request, id json.RawMessage) (*http.Response, error)
|
||||
mu sync.Mutex
|
||||
auth []string
|
||||
}
|
||||
|
||||
func (echo *aimlapiHopEcho) RoundTrip(request *http.Request) (*http.Response, error) {
|
||||
var body []byte
|
||||
if request.Body != nil {
|
||||
body, _ = io.ReadAll(request.Body)
|
||||
_ = request.Body.Close()
|
||||
}
|
||||
echo.mu.Lock()
|
||||
echo.auth = append(echo.auth, request.Header.Get("Authorization"))
|
||||
hop := len(echo.auth)
|
||||
echo.mu.Unlock()
|
||||
if hop == 1 {
|
||||
response := fakeHTTPResponse(request, http.StatusTemporaryRedirect, "")
|
||||
response.Header.Set("Location", echo.location)
|
||||
return response, nil
|
||||
}
|
||||
var message struct {
|
||||
ID json.RawMessage `json:"id"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &message)
|
||||
return echo.second(request, message.ID)
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPScrubsEveryKeySentAcrossRedirectHops(t *testing.T) {
|
||||
// The key is resolved again for a redirect hop, so the hops of one call can
|
||||
// carry different keys, and the same-origin server answering the last hop
|
||||
// has seen them all.
|
||||
const first = "aimlfirst0123456789abcdef01234567"
|
||||
const second = "aimlsecond0123456789abcdef0123456"
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
location string
|
||||
// hopKey is the key the redirect hop must carry, "" for none.
|
||||
hopKey string
|
||||
second func(*http.Request, json.RawMessage) (*http.Response, error)
|
||||
}{
|
||||
{
|
||||
name: "error echoes the first hop's key",
|
||||
location: "https://mcp.aimlapi.com/mcp/",
|
||||
hopKey: second,
|
||||
second: func(request *http.Request, id json.RawMessage) (*http.Response, error) {
|
||||
return fakeHTTPResponse(request, http.StatusOK, rpcReply(id, "error", map[string]any{"code": -32001, "message": "stale " + first})), nil
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "status body after a hop that dropped the key",
|
||||
location: "https://[email protected]/mcp",
|
||||
second: func(request *http.Request, _ json.RawMessage) (*http.Response, error) {
|
||||
return fakeHTTPResponse(request, http.StatusPaymentRequired, `{"error":"invalid credential `+first+`"}`), nil
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "transport error quoting the redirect target",
|
||||
location: "https://mcp.aimlapi.com/mcp?session=" + first,
|
||||
hopKey: second,
|
||||
second: func(*http.Request, json.RawMessage) (*http.Response, error) {
|
||||
return nil, errors.New("connection reset by peer")
|
||||
},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
rotateAIMLAPIKeys(t, first, second)
|
||||
echo := &aimlapiHopEcho{location: tc.location, second: tc.second}
|
||||
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
client := &networkClient{server: server, client: mcpHTTPClient(server, echo), nextID: 1}
|
||||
err := client.initialize(context.Background())
|
||||
|
||||
echo.mu.Lock()
|
||||
auth := append([]string(nil), echo.auth...)
|
||||
echo.mu.Unlock()
|
||||
if len(auth) != 2 || auth[0] != "Bearer "+first {
|
||||
t.Fatalf("Authorization per hop = %q, want the first key on the first request and a redirect hop", auth)
|
||||
}
|
||||
if tc.hopKey != "" && auth[1] != "Bearer "+tc.hopKey {
|
||||
t.Fatalf("redirect hop Authorization = %q, want the key resolved again for it", auth[1])
|
||||
}
|
||||
// With no key, net/http fills Basic auth from the hop's own userinfo.
|
||||
if tc.hopKey == "" && (strings.Contains(auth[1], first) || strings.Contains(auth[1], second)) {
|
||||
t.Fatalf("redirect hop Authorization = %q, want no automatic key", auth[1])
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("initialize succeeded against a failing server")
|
||||
}
|
||||
assertNoPartOf(t, err.Error(), first)
|
||||
assertNoPartOf(t, err.Error(), second)
|
||||
if !strings.Contains(err.Error(), "[REDACTED]") {
|
||||
t.Fatalf("error = %q, want the echoed key replaced", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPSSEScrubsTheStreamKeyFromAFailedPost(t *testing.T) {
|
||||
// The server chose an endpoint holding the key its stream request carried.
|
||||
// A POST to it that fails at the transport quotes that URL, and the POST
|
||||
// itself carried a different key, so only the stream's keys can scrub it.
|
||||
const streamKey = "strm0a1b2c3d4e5f60718293a4b5c6d7e8"
|
||||
const postKey = "post9e8d7c6b5a4f30291807f6e5d4c3b2"
|
||||
rotateAIMLAPIKeys(t, streamKey, postKey)
|
||||
echo := newAIMLAPISSEEcho("/messages/"+streamKey, func(string, json.RawMessage) string { return "" })
|
||||
echo.postErr = errors.New("connection reset by peer")
|
||||
client := newEchoSSEClient(echo)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := client.openStream(ctx); err != nil {
|
||||
t.Fatalf("openStream: %v", err)
|
||||
}
|
||||
_, err := client.ListTools(ctx)
|
||||
_ = client.Close()
|
||||
|
||||
echo.mu.Lock()
|
||||
auth := append([]string(nil), echo.auth...)
|
||||
echo.mu.Unlock()
|
||||
if len(auth) != 2 || auth[0] != "Bearer "+streamKey || auth[1] != "Bearer "+postKey {
|
||||
t.Fatalf("Authorization headers = %q, want the stream key and then the post key", auth)
|
||||
}
|
||||
if err == nil || !strings.Contains(err.Error(), "post MCP SSE message") {
|
||||
t.Fatalf("ListTools error = %v, want the failed POST", err)
|
||||
}
|
||||
assertNoPartOf(t, err.Error(), streamKey)
|
||||
assertNoPartOf(t, err.Error(), postKey)
|
||||
if !strings.Contains(err.Error(), "/messages/[REDACTED]") {
|
||||
t.Fatalf("error = %q, want the endpoint quoted with the key replaced", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPSSEScrubsTheKeyFromALaterEndpointError(t *testing.T) {
|
||||
// An endpoint event can arrive after the stream is open. Its parse error
|
||||
// then fails the pending requests directly rather than returning from
|
||||
// openStream, so it has to be scrubbed where it is made.
|
||||
isolateAIMLAPIKeyLookup(t, scrubTestKey)
|
||||
echo := newAIMLAPISSEEcho("/messages", nil)
|
||||
echo.rawEvent = func(string, json.RawMessage) string {
|
||||
return "event: endpoint\ndata: /messages/" + scrubTestKey + "%zz\n\n"
|
||||
}
|
||||
client := newEchoSSEClient(echo)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := client.openStream(ctx); err != nil {
|
||||
t.Fatalf("openStream: %v", err)
|
||||
}
|
||||
_, err := client.ListTools(ctx)
|
||||
_ = client.Close()
|
||||
echo.assertSent(t, "Bearer "+scrubTestKey, 2)
|
||||
if err == nil || !strings.Contains(err.Error(), "endpoint") {
|
||||
t.Fatalf("ListTools error = %v, want the later endpoint's parse failure", err)
|
||||
}
|
||||
assertKeyScrubbed(t, err.Error())
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPSSEScrubsKeysFromAFailedStreamRedirect(t *testing.T) {
|
||||
// The stream request follows redirects like any other, so a hop that fails
|
||||
// at the transport quotes the target the server chose, which can hold the
|
||||
// key the first hop carried.
|
||||
const first = "strmfirst0123456789abcdef01234567"
|
||||
const second = "strmsecond0123456789abcdef0123456"
|
||||
rotateAIMLAPIKeys(t, first, second)
|
||||
echo := &aimlapiHopEcho{
|
||||
location: "https://mcp.aimlapi.com/mcp?session=" + first,
|
||||
second: func(*http.Request, json.RawMessage) (*http.Response, error) {
|
||||
return nil, errors.New("connection reset by peer")
|
||||
},
|
||||
}
|
||||
server := Server{Name: "aimlapi", Type: ServerTypeSSE, URL: aimlapi.MCPServerURL}
|
||||
client := &remoteSSEClient{server: server, client: mcpHTTPClient(server, echo), nextID: 1, pending: map[string]chan ssePendingResponse{}}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
err := client.openStream(ctx)
|
||||
_ = client.Close()
|
||||
|
||||
echo.mu.Lock()
|
||||
auth := append([]string(nil), echo.auth...)
|
||||
echo.mu.Unlock()
|
||||
if len(auth) != 2 || auth[0] != "Bearer "+first || auth[1] != "Bearer "+second {
|
||||
t.Fatalf("Authorization per hop = %q, want the first key and then the second", auth)
|
||||
}
|
||||
if err == nil || !strings.Contains(err.Error(), "open MCP SSE stream") {
|
||||
t.Fatalf("openStream error = %v, want the failed stream request", err)
|
||||
}
|
||||
assertNoPartOf(t, err.Error(), first)
|
||||
assertNoPartOf(t, err.Error(), second)
|
||||
if !strings.Contains(err.Error(), "session=[REDACTED]") {
|
||||
t.Fatalf("error = %q, want the redirect target quoted with the key replaced", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestScrubAIMLAPIKeyErrorKeepsOnlyTheContextSentinel(t *testing.T) {
|
||||
for _, sentinel := range []error{context.Canceled, context.DeadlineExceeded} {
|
||||
original := fmt.Errorf("post MCP SSE message to aimlapi: %w", &url.Error{Op: "Post", URL: "https://mcp.aimlapi.com/messages/" + scrubTestKey, Err: sentinel})
|
||||
scrubbed := scrubAIMLAPIKeyError(original, scrubTestKey)
|
||||
assertKeyScrubbed(t, scrubbed.Error())
|
||||
if !errors.Is(scrubbed, sentinel) {
|
||||
t.Fatalf("scrubbed error %q no longer reads as %v", scrubbed, sentinel)
|
||||
}
|
||||
if unwrapped := errors.Unwrap(scrubbed); unwrapped != sentinel {
|
||||
t.Fatalf("scrubbed error unwraps to %v, want only %v", unwrapped, sentinel)
|
||||
}
|
||||
var urlErr *url.Error
|
||||
if errors.As(scrubbed, &urlErr) {
|
||||
t.Fatalf("scrubbed error still exposes the *url.Error holding the key: %v", urlErr.URL)
|
||||
}
|
||||
}
|
||||
|
||||
if scrubbed := scrubAIMLAPIKeyError(fmt.Errorf("decode: %s", scrubTestKey), scrubTestKey); errors.Unwrap(scrubbed) != nil {
|
||||
t.Fatalf("scrubbed error without a context sentinel unwraps to %v, want nothing", errors.Unwrap(scrubbed))
|
||||
}
|
||||
untouched := fmt.Errorf("call MCP: %w", context.Canceled)
|
||||
if got := scrubAIMLAPIKeyError(untouched, scrubTestKey); got != untouched {
|
||||
t.Fatalf("an error without the key was replaced: %v", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,537 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
)
|
||||
|
||||
// aimlapiRecorder stands in for mcp.aimlapi.com: it records every request it
|
||||
// is handed (the URL stays https://mcp.aimlapi.com/mcp, nothing is dialed) and
|
||||
// answers JSON-RPC, or 401 with the server's OAuth challenge when unauthorized.
|
||||
type aimlapiRecorder struct {
|
||||
mu sync.Mutex
|
||||
headers []http.Header
|
||||
urls []string
|
||||
unauthorized bool
|
||||
// redirectFirstTo, when set, answers the first request with a 307 to it.
|
||||
redirectFirstTo string
|
||||
// failStatus, when set, answers every request with it and failBody.
|
||||
failStatus int
|
||||
failBody string
|
||||
}
|
||||
|
||||
func (recorder *aimlapiRecorder) RoundTrip(request *http.Request) (*http.Response, error) {
|
||||
var body []byte
|
||||
if request.Body != nil {
|
||||
body, _ = io.ReadAll(request.Body)
|
||||
_ = request.Body.Close()
|
||||
}
|
||||
recorder.mu.Lock()
|
||||
recorder.headers = append(recorder.headers, request.Header.Clone())
|
||||
recorder.urls = append(recorder.urls, request.URL.String())
|
||||
unauthorized := recorder.unauthorized
|
||||
redirect := recorder.redirectFirstTo != "" && len(recorder.headers) == 1
|
||||
recorder.mu.Unlock()
|
||||
|
||||
if redirect {
|
||||
response := fakeHTTPResponse(request, http.StatusTemporaryRedirect, "")
|
||||
response.Header.Set("Location", recorder.redirectFirstTo)
|
||||
return response, nil
|
||||
}
|
||||
if recorder.failStatus != 0 {
|
||||
return fakeHTTPResponse(request, recorder.failStatus, recorder.failBody), nil
|
||||
}
|
||||
if unauthorized {
|
||||
response := fakeHTTPResponse(request, http.StatusUnauthorized, `{"error":"invalid_token"}`)
|
||||
response.Header.Set("WWW-Authenticate", `Bearer resource_metadata="https://mcp.aimlapi.com/.well-known/oauth-protected-resource/mcp"`)
|
||||
return response, nil
|
||||
}
|
||||
var message struct {
|
||||
ID json.RawMessage `json:"id"`
|
||||
Method string `json:"method"`
|
||||
}
|
||||
_ = json.Unmarshal(body, &message)
|
||||
if len(message.ID) == 0 {
|
||||
return fakeHTTPResponse(request, http.StatusAccepted, ""), nil
|
||||
}
|
||||
result := `{}`
|
||||
switch message.Method {
|
||||
case "initialize":
|
||||
result = `{"protocolVersion":"2024-11-05","capabilities":{},"serverInfo":{"name":"fake","version":"1"}}`
|
||||
case "tools/list":
|
||||
result = `{"tools":[{"name":"tools_search","inputSchema":{"type":"object"}}]}`
|
||||
}
|
||||
return fakeHTTPResponse(request, http.StatusOK, fmt.Sprintf(`{"jsonrpc":"2.0","id":%s,"result":%s}`, message.ID, result)), nil
|
||||
}
|
||||
|
||||
func (recorder *aimlapiRecorder) recorded() []http.Header {
|
||||
recorder.mu.Lock()
|
||||
defer recorder.mu.Unlock()
|
||||
return append([]http.Header(nil), recorder.headers...)
|
||||
}
|
||||
|
||||
func fakeHTTPResponse(request *http.Request, status int, body string) *http.Response {
|
||||
header := http.Header{}
|
||||
if body != "" {
|
||||
header.Set("Content-Type", "application/json")
|
||||
}
|
||||
return &http.Response{
|
||||
StatusCode: status,
|
||||
Header: header,
|
||||
Body: io.NopCloser(bytes.NewBufferString(body)),
|
||||
Request: request,
|
||||
}
|
||||
}
|
||||
|
||||
// isolateAIMLAPIKeyLookup points the real key resolver at an empty temporary
|
||||
// home and returns the user config path it reads.
|
||||
func isolateAIMLAPIKeyLookup(t *testing.T, envKey string) string {
|
||||
t.Helper()
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData"))
|
||||
t.Setenv("ZERO_CRED_STORAGE", "file")
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_PARTNER_ID", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", envKey)
|
||||
path, err := config.DefaultUserConfigPath()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
func writeAIMLAPIProviderConfig(t *testing.T, path string, baseURL string, apiKey string) {
|
||||
t.Helper()
|
||||
profile := map[string]any{"name": "aimlapi", "catalogID": "aimlapi", "model": "m", "apiKey": apiKey}
|
||||
if baseURL != "" {
|
||||
profile["baseURL"] = baseURL
|
||||
}
|
||||
data, err := json.Marshal(map[string]any{"activeProvider": "aimlapi", "providers": []any{profile}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, data, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func newRecordedNetworkClient(server Server, recorder *aimlapiRecorder) *networkClient {
|
||||
return &networkClient{server: server, client: mcpHTTPClient(server, recorder), nextID: 1}
|
||||
}
|
||||
|
||||
func aimlapiTestServer(name string, rawURL string) Server {
|
||||
return Server{Name: name, Type: ServerTypeHTTP, URL: rawURL}
|
||||
}
|
||||
|
||||
func assertAIMLAPIAttribution(t *testing.T, header http.Header) {
|
||||
t.Helper()
|
||||
if got := header.Get(aimlapi.SourceHeaderName); got != aimlapi.SourceValue {
|
||||
t.Fatalf("%s = %q, want %q", aimlapi.SourceHeaderName, got, aimlapi.SourceValue)
|
||||
}
|
||||
if got, want := header.Get(aimlapi.PartnerHeaderName), aimlapi.ResolvePartnerID(""); got == "" || got != want {
|
||||
t.Fatalf("%s = %q, want %q", aimlapi.PartnerHeaderName, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPRequestsCarryServiceKeyAndAttribution(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, "aiml-secret")
|
||||
// A hand-added server on the exact host is treated like the built-in one.
|
||||
for _, name := range []string{"aimlapi", "media"} {
|
||||
recorder := &aimlapiRecorder{}
|
||||
client := newRecordedNetworkClient(aimlapiTestServer(name, aimlapi.MCPServerURL), recorder)
|
||||
if err := client.initialize(context.Background()); err != nil {
|
||||
t.Fatalf("%s: initialize: %v", name, err)
|
||||
}
|
||||
if _, err := client.ListTools(context.Background()); err != nil {
|
||||
t.Fatalf("%s: ListTools: %v", name, err)
|
||||
}
|
||||
requests := recorder.recorded()
|
||||
if len(requests) != 3 {
|
||||
t.Fatalf("%s: recorded %d requests, want initialize, initialized and tools/list", name, len(requests))
|
||||
}
|
||||
for _, header := range requests {
|
||||
if got := header.Get("Authorization"); got != "Bearer aiml-secret" {
|
||||
t.Fatalf("%s: Authorization = %q, want the AI/ML API key", name, got)
|
||||
}
|
||||
assertAIMLAPIAttribution(t, header)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPHeadersOnlyForExactHost(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, "aiml-secret")
|
||||
for _, rawURL := range []string{
|
||||
"https://mcp.aimlapi.com.example.net/mcp",
|
||||
"http://mcp.aimlapi.com/mcp",
|
||||
"https://mcp.aimlapi.com:8443/mcp",
|
||||
"https://[email protected]/mcp",
|
||||
"https://api.aimlapi.com/mcp",
|
||||
"https://evilmcp.aimlapi.com/mcp",
|
||||
"https://mcp.exa.ai/mcp",
|
||||
} {
|
||||
recorder := &aimlapiRecorder{}
|
||||
server := aimlapiTestServer("other", rawURL)
|
||||
server.Headers = map[string]string{"X-Custom": "kept"}
|
||||
client := newRecordedNetworkClient(server, recorder)
|
||||
if err := client.initialize(context.Background()); err != nil {
|
||||
t.Fatalf("%s: initialize: %v", rawURL, err)
|
||||
}
|
||||
for _, header := range recorder.recorded() {
|
||||
if strings.Contains(rawURL, "user@") && strings.HasPrefix(header.Get("Authorization"), "Basic ") {
|
||||
// net/http turns URL userinfo into Basic auth on its own.
|
||||
header.Del("Authorization")
|
||||
}
|
||||
// Byte-for-byte what any other MCP server gets.
|
||||
want := http.Header{
|
||||
"X-Custom": {"kept"},
|
||||
"Content-Type": {"application/json"},
|
||||
"Accept": {"application/json, text/event-stream"},
|
||||
}
|
||||
if fmt.Sprint(header) != fmt.Sprint(want) {
|
||||
t.Fatalf("%s: headers = %v, want %v", rawURL, header, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Port 443 spelled out and a different host case are still the exact host.
|
||||
for _, rawURL := range []string{"https://mcp.aimlapi.com:443/mcp", "https://MCP.AIMLAPI.COM/mcp"} {
|
||||
recorder := &aimlapiRecorder{}
|
||||
client := newRecordedNetworkClient(aimlapiTestServer("aimlapi", rawURL), recorder)
|
||||
if err := client.initialize(context.Background()); err != nil {
|
||||
t.Fatalf("%s: initialize: %v", rawURL, err)
|
||||
}
|
||||
header := recorder.recorded()[0]
|
||||
if header.Get("Authorization") != "Bearer aiml-secret" {
|
||||
t.Fatalf("%s: Authorization = %q, want the key", rawURL, header.Get("Authorization"))
|
||||
}
|
||||
assertAIMLAPIAttribution(t, header)
|
||||
}
|
||||
}
|
||||
|
||||
// The key and its eligibility are resolved for every request, so switching the
|
||||
// aimlapi profile to a proxy keeps the proxy's key off mcp.aimlapi.com on the
|
||||
// very next request of an already-connected client and on new connections.
|
||||
func TestAIMLAPIMCPKeyIsResolvedPerRequestAcrossProviderSwitch(t *testing.T) {
|
||||
path := isolateAIMLAPIKeyLookup(t, "")
|
||||
writeAIMLAPIProviderConfig(t, path, "", "canonical-key")
|
||||
|
||||
recorder := &aimlapiRecorder{}
|
||||
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
client := newRecordedNetworkClient(server, recorder)
|
||||
if err := client.initialize(context.Background()); err != nil {
|
||||
t.Fatalf("initialize: %v", err)
|
||||
}
|
||||
if got := recorder.recorded()[0].Get("Authorization"); got != "Bearer canonical-key" {
|
||||
t.Fatalf("canonical profile: Authorization = %q, want Bearer canonical-key", got)
|
||||
}
|
||||
|
||||
writeAIMLAPIProviderConfig(t, path, "https://proxy.example/v1", "proxy-key")
|
||||
if _, err := client.ListTools(context.Background()); err != nil {
|
||||
t.Fatalf("ListTools after switch: %v", err)
|
||||
}
|
||||
fresh := newRecordedNetworkClient(server, recorder)
|
||||
if err := fresh.initialize(context.Background()); err != nil {
|
||||
t.Fatalf("initialize after switch: %v", err)
|
||||
}
|
||||
requests := recorder.recorded()
|
||||
for _, header := range requests[2:] {
|
||||
if got := header.Get("Authorization"); got != "" {
|
||||
t.Fatalf("proxy profile: Authorization = %q, want none", got)
|
||||
}
|
||||
assertAIMLAPIAttribution(t, header)
|
||||
}
|
||||
for _, header := range requests {
|
||||
if strings.Contains(fmt.Sprint(header), "proxy-key") {
|
||||
t.Fatalf("proxy key reached mcp.aimlapi.com: %v", header)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPUnusableKeySendsNoAuthorization(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"multi-line key pool": "first-key\nsecond-key",
|
||||
"comma key pool": "first-key,second-key",
|
||||
"no key": "",
|
||||
}
|
||||
for name, apiKey := range cases {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
path := isolateAIMLAPIKeyLookup(t, "")
|
||||
writeAIMLAPIProviderConfig(t, path, "", apiKey)
|
||||
recorder := &aimlapiRecorder{unauthorized: true}
|
||||
client := newRecordedNetworkClient(aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), recorder)
|
||||
err := client.initialize(context.Background())
|
||||
if err == nil {
|
||||
t.Fatal("initialize succeeded against a 401")
|
||||
}
|
||||
requests := recorder.recorded()
|
||||
if len(requests) != 1 {
|
||||
t.Fatalf("recorded %d requests, want one intact request", len(requests))
|
||||
}
|
||||
header := requests[0]
|
||||
if got := header.Get("Authorization"); got != "" {
|
||||
t.Fatalf("Authorization = %q, want none", got)
|
||||
}
|
||||
if header.Get("Content-Type") != "application/json" {
|
||||
t.Fatalf("request headers broken: %v", header)
|
||||
}
|
||||
assertAIMLAPIAttribution(t, header)
|
||||
if !strings.Contains(err.Error(), "returned HTTP 401") || !strings.Contains(err.Error(), "zero mcp oauth login aimlapi") || !strings.Contains(err.Error(), "AIMLAPI_API_KEY") {
|
||||
t.Fatalf("401 error lacks the authentication hint: %v", err)
|
||||
}
|
||||
if strings.Contains(err.Error(), "first-key") {
|
||||
t.Fatalf("error leaks key material: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPUnauthorizedHintOnlyForAIMLAPIWithoutOAuth(t *testing.T) {
|
||||
if hint := aimlapiUnauthorizedHint(aimlapiTestServer("docs", "https://mcp.example.com/mcp"), http.StatusUnauthorized); hint != "" {
|
||||
t.Fatalf("hint for another server = %q", hint)
|
||||
}
|
||||
oauth := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
oauth.Auth = ServerAuthOAuth
|
||||
if hint := aimlapiUnauthorizedHint(oauth, http.StatusUnauthorized); hint != "" {
|
||||
t.Fatalf("hint for an OAuth server = %q", hint)
|
||||
}
|
||||
if hint := aimlapiUnauthorizedHint(aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), http.StatusForbidden); hint != "" {
|
||||
t.Fatalf("hint for 403 = %q", hint)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPKeyDefersToUserAuthorizationOAuthAndProjectConfig(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, "aiml-secret")
|
||||
|
||||
userHeader := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
userHeader.Headers = map[string]string{"authorization": "Bearer user-token"}
|
||||
oauth := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
oauth.Auth = ServerAuthOAuth
|
||||
project := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
project.ProjectConfigured = true
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
server Server
|
||||
want string
|
||||
}{
|
||||
{"user Authorization header", userHeader, "Bearer user-token"},
|
||||
{"OAuth server", oauth, ""},
|
||||
{"project-configured server", project, ""},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
recorder := &aimlapiRecorder{}
|
||||
client := newRecordedNetworkClient(tc.server, recorder)
|
||||
if err := client.initialize(context.Background()); err != nil {
|
||||
t.Fatalf("%s: initialize: %v", tc.name, err)
|
||||
}
|
||||
header := recorder.recorded()[0]
|
||||
if got := header.Get("Authorization"); got != tc.want {
|
||||
t.Fatalf("%s: Authorization = %q, want %q", tc.name, got, tc.want)
|
||||
}
|
||||
assertAIMLAPIAttribution(t, header)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPSSERequestsCarryKeyAndAttribution(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, "aiml-secret")
|
||||
server := Server{Name: "aimlapi", Type: ServerTypeSSE, URL: aimlapi.MCPServerURL}
|
||||
client := &remoteSSEClient{server: server}
|
||||
apply := map[string]func(*http.Request){
|
||||
"stream": client.applyStreamHeaders,
|
||||
"post": client.applyPostHeaders,
|
||||
}
|
||||
for name, applyHeaders := range apply {
|
||||
request, err := http.NewRequest(http.MethodPost, aimlapi.MCPServerURL, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
applyHeaders(request)
|
||||
if got := request.Header.Get("Authorization"); got != "Bearer aiml-secret" {
|
||||
t.Fatalf("%s: Authorization = %q, want the key", name, got)
|
||||
}
|
||||
assertAIMLAPIAttribution(t, request.Header)
|
||||
|
||||
other, err := http.NewRequest(http.MethodPost, "https://mcp.aimlapi.com.example.net/mcp", nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
applyHeaders(other)
|
||||
if other.Header.Get("Authorization") != "" || other.Header.Get(aimlapi.SourceHeaderName) != "" || other.Header.Get(aimlapi.PartnerHeaderName) != "" {
|
||||
t.Fatalf("%s: lookalike host got AI/ML API headers: %v", name, other.Header)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPKeyStaysOutOfServerConfigAndIdentity(t *testing.T) {
|
||||
enabled := config.MCPConfig{Servers: map[string]config.MCPServerConfig{
|
||||
"aimlapi": {Type: "http", URL: aimlapi.MCPServerURL},
|
||||
}}
|
||||
isolateAIMLAPIKeyLookup(t, "")
|
||||
withoutKey, err := NormalizeConfig(enabled)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
isolateAIMLAPIKeyLookup(t, "aiml-secret")
|
||||
withKey, err := NormalizeConfig(enabled)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(withKey) != 1 || withKey[0].Identity != withoutKey[0].Identity {
|
||||
t.Fatalf("server identity depends on the key: %#v vs %#v", withKey, withoutKey)
|
||||
}
|
||||
if strings.Contains(fmt.Sprintf("%#v", withKey), "aiml-secret") {
|
||||
t.Fatalf("normalized server carries the key: %#v", withKey)
|
||||
}
|
||||
|
||||
// The shipped default is off, so it is not even normalized for a session.
|
||||
defaults, err := NormalizeConfig(config.MCPConfig{Servers: config.DefaultMCPServers()})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, server := range defaults {
|
||||
if server.Name == "aimlapi" {
|
||||
t.Fatalf("disabled aimlapi default was normalized for connection: %#v", server)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIMCPUnauthorizedHintRespectsCredentialEligibility(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
headers map[string]string
|
||||
project bool
|
||||
wantSetup bool
|
||||
}{
|
||||
{"default", nil, false, true},
|
||||
{"project", nil, true, false},
|
||||
{"custom bearer", map[string]string{"Authorization": "Bearer manual"}, false, false},
|
||||
{"mixed case header", map[string]string{"aUtHoRiZaTiOn": "Bearer manual"}, false, false},
|
||||
{"empty header", map[string]string{"authorization": ""}, false, true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
server := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
server.Headers = tc.headers
|
||||
server.ProjectConfigured = tc.project
|
||||
hint := aimlapiUnauthorizedHint(server, http.StatusUnauthorized)
|
||||
if strings.Contains(hint, "zero providers setup aimlapi") != tc.wantSetup || !strings.Contains(hint, "zero mcp oauth login aimlapi") {
|
||||
t.Fatalf("incorrect hint: %s", hint)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// net/http copies the first request's headers onto every redirect hop without
|
||||
// calling the header hooks, so the redirect guard has to apply the per-request
|
||||
// rule again: a same-origin hop that fails the exact-URL rule (userinfo) gets
|
||||
// neither the key nor attribution, an eligible hop gets a freshly resolved key,
|
||||
// and a user's own Authorization header still follows as before.
|
||||
func TestAIMLAPIMCPRedirectHopsReapplyAutomaticHeaders(t *testing.T) {
|
||||
isolateAIMLAPIKeyLookup(t, "")
|
||||
userHeader := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
userHeader.Headers = map[string]string{"authorization": "Bearer user-token"}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
server Server
|
||||
location string
|
||||
wantFirst string
|
||||
wantHop string
|
||||
wantAttribution bool
|
||||
}{
|
||||
{"userinfo hop", aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), "https://[email protected]/mcp", "Bearer aiml-first", "", false},
|
||||
{"eligible hop", aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), "https://mcp.aimlapi.com/mcp/", "Bearer aiml-first", "Bearer aiml-second", true},
|
||||
{"user header hop", userHeader, "https://[email protected]/mcp", "Bearer user-token", "Bearer user-token", false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
keys := []string{"aiml-first", "aiml-second"}
|
||||
previous := loadAIMLAPIServiceKey
|
||||
loadAIMLAPIServiceKey = func() (string, error) {
|
||||
key := keys[0]
|
||||
if len(keys) > 1 {
|
||||
keys = keys[1:]
|
||||
}
|
||||
return key, nil
|
||||
}
|
||||
t.Cleanup(func() { loadAIMLAPIServiceKey = previous })
|
||||
|
||||
recorder := &aimlapiRecorder{redirectFirstTo: tc.location}
|
||||
client := newRecordedNetworkClient(tc.server, recorder)
|
||||
if err := client.initialize(context.Background()); err != nil {
|
||||
t.Fatalf("initialize: %v", err)
|
||||
}
|
||||
requests := recorder.recorded()
|
||||
if len(requests) < 2 || recorder.urls[1] != tc.location {
|
||||
t.Fatalf("redirect was not followed: %v", recorder.urls)
|
||||
}
|
||||
if got := requests[0].Get("Authorization"); got != tc.wantFirst {
|
||||
t.Fatalf("first hop Authorization = %q, want %q", got, tc.wantFirst)
|
||||
}
|
||||
hop := requests[1]
|
||||
// With no key, net/http fills Basic auth from the hop's own userinfo;
|
||||
// only the automatic key must be absent.
|
||||
if got := hop.Get("Authorization"); (tc.wantHop == "" && strings.Contains(got, "aiml-")) || (tc.wantHop != "" && got != tc.wantHop) {
|
||||
t.Fatalf("redirect hop Authorization = %q, want %q", got, tc.wantHop)
|
||||
}
|
||||
if tc.wantAttribution {
|
||||
assertAIMLAPIAttribution(t, hop)
|
||||
} else if hop.Get(aimlapi.SourceHeaderName) != "" || hop.Get(aimlapi.PartnerHeaderName) != "" {
|
||||
t.Fatalf("redirect hop kept attribution headers: %v", hop)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// An error body can echo the key Zero injected; the body is left out of the
|
||||
// error rather than matched, so a key cut by the 1024-byte read limit cannot
|
||||
// leak its prefix either. Requests without the automatic key keep the detail.
|
||||
func TestAIMLAPIMCPStatusErrorOmitsBodyWhenKeyWasInjected(t *testing.T) {
|
||||
const key = "sk-echo-0123456789abcdef"
|
||||
isolateAIMLAPIKeyLookup(t, key)
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
body string
|
||||
}{
|
||||
{"echoed key", `{"error":"invalid credential ` + key + `"}`},
|
||||
{"key across the read limit", strings.Repeat("x", 1020) + key},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
recorder := &aimlapiRecorder{failStatus: http.StatusPaymentRequired, failBody: tc.body}
|
||||
err := newRecordedNetworkClient(aimlapiTestServer("aimlapi", aimlapi.MCPServerURL), recorder).initialize(context.Background())
|
||||
if err == nil {
|
||||
t.Fatal("initialize succeeded against a failing server")
|
||||
}
|
||||
if got := recorder.recorded()[0].Get("Authorization"); got != "Bearer "+key {
|
||||
t.Fatalf("request did not carry the injected key: %q", got)
|
||||
}
|
||||
if strings.Contains(err.Error(), key[:4]) || !strings.Contains(err.Error(), "HTTP 402") {
|
||||
t.Fatalf("error = %q, want HTTP 402 without the echoed key", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
userHeader := aimlapiTestServer("aimlapi", aimlapi.MCPServerURL)
|
||||
userHeader.Headers = map[string]string{"Authorization": "Bearer user-token"}
|
||||
recorder := &aimlapiRecorder{failStatus: http.StatusPaymentRequired, failBody: `{"error":"insufficient balance"}`}
|
||||
err := newRecordedNetworkClient(userHeader, recorder).initialize(context.Background())
|
||||
if err == nil || !strings.Contains(err.Error(), "insufficient balance") {
|
||||
t.Fatalf("error without the automatic key = %v, want the response detail", err)
|
||||
}
|
||||
}
|
||||
@@ -35,6 +35,11 @@ type remoteSSEClient struct {
|
||||
pending map[string]chan ssePendingResponse
|
||||
streamErr error
|
||||
closed bool
|
||||
// streamKeys are the injected AI/ML API keys the stream request carried,
|
||||
// redirect hops included. openStream sets them once, before the reader
|
||||
// starts; the endpoint and the messages on the stream come from a server
|
||||
// that was sent them.
|
||||
streamKeys []string
|
||||
}
|
||||
|
||||
type ssePendingResponse struct {
|
||||
@@ -259,12 +264,13 @@ func (client *remoteSSEClient) request(ctx context.Context, method string, param
|
||||
client.pending[key] = responseChannel
|
||||
client.mu.Unlock()
|
||||
|
||||
if err := client.post(ctx, rpcMessage{
|
||||
sentKeys, err := client.post(ctx, rpcMessage{
|
||||
JSONRPC: "2.0",
|
||||
ID: id,
|
||||
Method: method,
|
||||
Params: rawParams,
|
||||
}); err != nil {
|
||||
})
|
||||
if err != nil {
|
||||
client.removePending(key)
|
||||
return err
|
||||
}
|
||||
@@ -274,14 +280,15 @@ func (client *remoteSSEClient) request(ctx context.Context, method string, param
|
||||
if response.err != nil {
|
||||
return response.err
|
||||
}
|
||||
if !rpcIDMatches(response.message.ID, id) {
|
||||
message := scrubAIMLAPIKeys(response.message, append(append([]string(nil), client.streamKeys...), sentKeys...)...)
|
||||
if !rpcIDMatches(message.ID, id) {
|
||||
return fmt.Errorf("MCP %s response id mismatch for server %s", method, client.server.Name)
|
||||
}
|
||||
if response.message.Error != nil {
|
||||
return fmt.Errorf("MCP %s failed: %s", method, response.message.Error.Message)
|
||||
if message.Error != nil {
|
||||
return fmt.Errorf("MCP %s failed: %s", method, message.Error.Message)
|
||||
}
|
||||
if target != nil && len(response.message.Result) > 0 {
|
||||
if err := json.Unmarshal(response.message.Result, target); err != nil {
|
||||
if target != nil && len(message.Result) > 0 {
|
||||
if err := json.Unmarshal(message.Result, target); err != nil {
|
||||
return fmt.Errorf("decode MCP %s result: %w", method, err)
|
||||
}
|
||||
}
|
||||
@@ -320,11 +327,12 @@ func (client *remoteSSEClient) notify(ctx context.Context, method string, params
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return client.post(ctx, rpcMessage{
|
||||
_, err = client.post(ctx, rpcMessage{
|
||||
JSONRPC: "2.0",
|
||||
Method: method,
|
||||
Params: rawParams,
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func (client *networkClient) post(ctx context.Context, message rpcMessage, expectResponse bool) (result rpcMessage, err error) {
|
||||
@@ -332,6 +340,11 @@ func (client *networkClient) post(ctx context.Context, message rpcMessage, expec
|
||||
if err != nil {
|
||||
return rpcMessage{}, err
|
||||
}
|
||||
ctx, sent := withAIMLAPISentKeys(ctx)
|
||||
// From here an error can quote the server or a URL it chose, a redirect
|
||||
// target among them, so every key the call sent, on any hop, is scrubbed
|
||||
// from it. This runs last, after the body is closed.
|
||||
defer func() { err = scrubAIMLAPIKeyError(err, sent.list()...) }()
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodPost, client.server.URL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return rpcMessage{}, fmt.Errorf("create MCP %s request for %s: %w", client.server.Type, client.server.Name, err)
|
||||
@@ -357,12 +370,18 @@ func (client *networkClient) post(ctx context.Context, message rpcMessage, expec
|
||||
if response.StatusCode == http.StatusNoContent {
|
||||
return rpcMessage{}, fmt.Errorf("MCP %s server %s returned no response body", client.server.Type, client.server.Name)
|
||||
}
|
||||
return client.decodeResponse(response)
|
||||
decoded, err := client.decodeResponse(response)
|
||||
if err != nil {
|
||||
return rpcMessage{}, err
|
||||
}
|
||||
return scrubAIMLAPIKeys(decoded, sent.list()...), nil
|
||||
}
|
||||
|
||||
func (client *remoteSSEClient) openStream(ctx context.Context) error {
|
||||
func (client *remoteSSEClient) openStream(ctx context.Context) (err error) {
|
||||
streamCtx, cancel := context.WithCancel(ctx)
|
||||
client.streamCancel = cancel
|
||||
streamCtx, sent := withAIMLAPISentKeys(streamCtx)
|
||||
defer func() { err = scrubAIMLAPIKeyError(err, sent.list()...) }()
|
||||
|
||||
request, err := http.NewRequestWithContext(streamCtx, http.MethodGet, client.server.URL, nil)
|
||||
if err != nil {
|
||||
@@ -380,6 +399,7 @@ func (client *remoteSSEClient) openStream(ctx context.Context) error {
|
||||
return err
|
||||
}
|
||||
client.streamBody = response.Body
|
||||
client.streamKeys = sent.list()
|
||||
|
||||
ready := make(chan error, 1)
|
||||
go client.readStream(response.Body, ready)
|
||||
@@ -391,31 +411,41 @@ func (client *remoteSSEClient) openStream(ctx context.Context) error {
|
||||
}
|
||||
}
|
||||
|
||||
func (client *remoteSSEClient) post(ctx context.Context, message rpcMessage) (err error) {
|
||||
// post sends message to the endpoint and returns the injected AI/ML API keys the
|
||||
// request carried, redirect hops included, since the answer arrives on the
|
||||
// stream from a server that was sent them. The endpoint is the server's own
|
||||
// choice and can hold one of the stream's keys, and a transport error quotes it,
|
||||
// so the stream's keys are scrubbed from any error as well as the POST's.
|
||||
func (client *remoteSSEClient) post(ctx context.Context, message rpcMessage) (sentKeys []string, err error) {
|
||||
ctx, sent := withAIMLAPISentKeys(ctx)
|
||||
defer func() {
|
||||
sentKeys = sent.list()
|
||||
err = scrubAIMLAPIKeyError(err, append(append([]string(nil), client.streamKeys...), sentKeys...)...)
|
||||
}()
|
||||
endpointURL, err := client.currentEndpoint()
|
||||
if err != nil {
|
||||
return err
|
||||
return nil, err
|
||||
}
|
||||
body, err := json.Marshal(message)
|
||||
if err != nil {
|
||||
return err
|
||||
return nil, err
|
||||
}
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodPost, endpointURL, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return fmt.Errorf("create MCP SSE post for %s: %w", client.server.Name, err)
|
||||
return nil, fmt.Errorf("create MCP SSE post for %s: %w", client.server.Name, err)
|
||||
}
|
||||
client.applyPostHeaders(request)
|
||||
|
||||
response, err := client.client.Do(request)
|
||||
if err != nil {
|
||||
return fmt.Errorf("post MCP SSE message to %s: %w", client.server.Name, err)
|
||||
return nil, fmt.Errorf("post MCP SSE message to %s: %w", client.server.Name, err)
|
||||
}
|
||||
defer closeResponseBody(&err, client.server, response.Body)
|
||||
if response.StatusCode < http.StatusOK || response.StatusCode >= http.StatusMultipleChoices {
|
||||
return httpStatusError(client.server, response)
|
||||
return nil, httpStatusError(client.server, response)
|
||||
}
|
||||
_, _ = io.Copy(io.Discard, response.Body)
|
||||
return nil
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (client *networkClient) applyHeaders(request *http.Request) {
|
||||
@@ -427,6 +457,7 @@ func (client *networkClient) applyHeaders(request *http.Request) {
|
||||
if client.sessionID != "" {
|
||||
request.Header.Set("Mcp-Session-Id", client.sessionID)
|
||||
}
|
||||
applyAIMLAPIRequestHeaders(client.server, request)
|
||||
}
|
||||
|
||||
func (client *remoteSSEClient) applyStreamHeaders(request *http.Request) {
|
||||
@@ -434,6 +465,7 @@ func (client *remoteSSEClient) applyStreamHeaders(request *http.Request) {
|
||||
request.Header.Set(key, value)
|
||||
}
|
||||
request.Header.Set("Accept", "text/event-stream")
|
||||
applyAIMLAPIRequestHeaders(client.server, request)
|
||||
}
|
||||
|
||||
func (client *remoteSSEClient) applyPostHeaders(request *http.Request) {
|
||||
@@ -442,6 +474,7 @@ func (client *remoteSSEClient) applyPostHeaders(request *http.Request) {
|
||||
}
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
request.Header.Set("Accept", "application/json")
|
||||
applyAIMLAPIRequestHeaders(client.server, request)
|
||||
}
|
||||
|
||||
func closeResponseBody(errp *error, server Server, body io.Closer) {
|
||||
@@ -523,7 +556,8 @@ func (client *remoteSSEClient) readStream(reader io.Reader, ready chan<- error)
|
||||
func (client *remoteSSEClient) setEndpoint(value string) error {
|
||||
endpointURL, err := resolveSSEEndpointURL(client.server.URL, strings.TrimSpace(value))
|
||||
if err != nil {
|
||||
return err
|
||||
// A URL parse error quotes the endpoint, which is the server's text.
|
||||
return scrubAIMLAPIKeyError(err, client.streamKeys...)
|
||||
}
|
||||
client.mu.Lock()
|
||||
client.endpointURL = endpointURL
|
||||
@@ -607,12 +641,18 @@ func (client *remoteSSEClient) failPending(err error) {
|
||||
}
|
||||
|
||||
func httpStatusError(server Server, response *http.Response) error {
|
||||
hint := aimlapiUnauthorizedHint(server, response.StatusCode)
|
||||
if sentAIMLAPIServiceKey(server, response.Request) {
|
||||
// The body is server text that may echo the key Zero injected, whole or
|
||||
// cut by the read limit, so it is left out rather than matched.
|
||||
return fmt.Errorf("MCP %s server %s returned HTTP %d (response body omitted: the request carried your AI/ML API key)%s", server.Type, server.Name, response.StatusCode, hint)
|
||||
}
|
||||
body, _ := io.ReadAll(io.LimitReader(response.Body, 1024))
|
||||
detail := strings.TrimSpace(string(body))
|
||||
if detail == "" {
|
||||
return fmt.Errorf("MCP %s server %s returned HTTP %d", server.Type, server.Name, response.StatusCode)
|
||||
return fmt.Errorf("MCP %s server %s returned HTTP %d%s", server.Type, server.Name, response.StatusCode, hint)
|
||||
}
|
||||
return fmt.Errorf("MCP %s server %s returned HTTP %d: %s", server.Type, server.Name, response.StatusCode, detail)
|
||||
return fmt.Errorf("MCP %s server %s returned HTTP %d: %s%s", server.Type, server.Name, response.StatusCode, detail, hint)
|
||||
}
|
||||
|
||||
func decodeSSERPCMessage(reader io.Reader) (rpcMessage, error) {
|
||||
|
||||
@@ -28,6 +28,7 @@ func checkMCPRedirect(server Server) func(*http.Request, []*http.Request) error
|
||||
if !sameMCPOrigin(via[0].URL, request.URL) {
|
||||
return fmt.Errorf("MCP %s server %s refused cross-origin redirect to %s", server.Type, server.Name, mcpOrigin(request.URL))
|
||||
}
|
||||
reapplyAIMLAPIRedirectHeaders(server, request, via[0])
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,552 @@
|
||||
package plugins
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/Twigpine/zero/internal/installtxn"
|
||||
"github.com/Twigpine/zero/internal/pathjail"
|
||||
)
|
||||
|
||||
// SetEnabledResult describes a successful enable/disable of one plugin manifest.
|
||||
type SetEnabledResult struct {
|
||||
ID string `json:"id"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Changed bool `json:"changed"`
|
||||
Source Source `json:"source"`
|
||||
ManifestPath string `json:"manifestPath"`
|
||||
}
|
||||
|
||||
// maxEnabledRelockAttempts bounds how many times SetEnabledByID re-resolves the
|
||||
// plugin after it moves to another install root while waiting for the lock.
|
||||
const maxEnabledRelockAttempts = 4
|
||||
|
||||
// FindByID returns the loaded plugin with the given id after normal discovery
|
||||
// and precedence (project overrides user). A missing id returns ok=false.
|
||||
func FindByID(options LoadOptions, id string) (LoadedPlugin, bool, error) {
|
||||
id = strings.TrimSpace(id)
|
||||
if id == "" {
|
||||
return LoadedPlugin{}, false, errors.New("plugin id is required")
|
||||
}
|
||||
result, err := Load(options)
|
||||
if err != nil {
|
||||
return LoadedPlugin{}, false, err
|
||||
}
|
||||
for _, plugin := range result.Plugins {
|
||||
if plugin.ID == id {
|
||||
return plugin, true, nil
|
||||
}
|
||||
}
|
||||
return LoadedPlugin{}, false, nil
|
||||
}
|
||||
|
||||
// SetEnabledByID finds a plugin by id (respecting LoadOptions) and toggles the
|
||||
// top-level "enabled" field of its plugin.json.
|
||||
//
|
||||
// The read-modify-write is serialized with plugin installs and removals: it
|
||||
// takes the same cross-process installtxn lock they take on the plugin's
|
||||
// install root, runs the same interrupted-transaction recovery they run under
|
||||
// it, re-resolves the plugin under that lock, and publishes the
|
||||
// manifest with an atomic rename while still holding it, so a concurrent
|
||||
// install can never be interleaved between the read and the write. Only the
|
||||
// bytes of the "enabled" value change, so field order, formatting, and numeric
|
||||
// literals in the rest of the manifest survive the toggle.
|
||||
func SetEnabledByID(options LoadOptions, id string, enabled bool) (SetEnabledResult, error) {
|
||||
for attempt := 0; ; attempt++ {
|
||||
plugin, ok, err := FindByID(options, id)
|
||||
if err != nil {
|
||||
return SetEnabledResult{}, err
|
||||
}
|
||||
if !ok {
|
||||
return SetEnabledResult{}, fmt.Errorf("%w: %q", ErrNotInstalled, id)
|
||||
}
|
||||
// Take the install root lock before the final lookup and read so an
|
||||
// install or remove cannot swap the tree between them.
|
||||
unlock, err := installtxn.Lock(plugin.Root)
|
||||
if err != nil {
|
||||
return SetEnabledResult{}, err
|
||||
}
|
||||
// Put back anything an earlier install or removal was killed mid-commit,
|
||||
// and abort if a transaction could not be resolved: editing a tree that is
|
||||
// still owed a restore loses the edit when the next install or removal
|
||||
// puts the backup back over it. See installtxn.Recover.
|
||||
if err := installtxn.Recover(plugin.Root, reconcileInterrupted(plugin.Root)); err != nil {
|
||||
unlock()
|
||||
return SetEnabledResult{}, err
|
||||
}
|
||||
final, ok, err := FindByID(options, id)
|
||||
if err != nil {
|
||||
unlock()
|
||||
return SetEnabledResult{}, err
|
||||
}
|
||||
if !ok {
|
||||
unlock()
|
||||
return SetEnabledResult{}, fmt.Errorf("%w: %q", ErrNotInstalled, id)
|
||||
}
|
||||
if final.Root != plugin.Root {
|
||||
// Precedence moved to another root while this lock was pending;
|
||||
// retry against the root that wins now so the edit still serializes
|
||||
// with that root's installs.
|
||||
unlock()
|
||||
if attempt >= maxEnabledRelockAttempts {
|
||||
return SetEnabledResult{}, fmt.Errorf("plugin %q kept changing roots while waiting for the install lock", id)
|
||||
}
|
||||
continue
|
||||
}
|
||||
changed, err := setEnabledLocked(final, enabled)
|
||||
unlock()
|
||||
if err != nil {
|
||||
return SetEnabledResult{}, err
|
||||
}
|
||||
return SetEnabledResult{
|
||||
ID: final.ID,
|
||||
Enabled: enabled,
|
||||
Changed: changed,
|
||||
Source: final.Source,
|
||||
ManifestPath: final.ManifestPath,
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
|
||||
// setEnabledLocked performs the manifest read-modify-write while the caller
|
||||
// holds the installtxn lock for the plugin's install root.
|
||||
//
|
||||
// The lock only serializes cooperating installers; it does not stop the plugin
|
||||
// directory from being swapped after discovery found it. So the whole update
|
||||
// works on one directory, pinned by openPluginDir: the manifest is read and
|
||||
// published, and the tree hashed for the lockfile, through that handle, never
|
||||
// by re-resolving the path.
|
||||
func setEnabledLocked(plugin LoadedPlugin, enabled bool) (bool, error) {
|
||||
pinned, err := openPluginDir(plugin)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
defer pinned.Close()
|
||||
return setEnabledPinned(plugin, pinned, enabled)
|
||||
}
|
||||
|
||||
// setEnabledPinned is setEnabledLocked on an already pinned plugin directory.
|
||||
//
|
||||
// The edit lands in the pinned directory even if the path now names another
|
||||
// one, so before refreshing the lockfile or reporting any result it confirms
|
||||
// that the directory entry still is the pinned directory. Otherwise the
|
||||
// command would report a state that the installed plugin does not have.
|
||||
func setEnabledPinned(plugin LoadedPlugin, pinned *pinnedPluginDir, enabled bool) (bool, error) {
|
||||
manifestName := filepath.Base(plugin.ManifestPath)
|
||||
raw, perm, err := readManifest(pinned.dir, manifestName)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
data, changed, err := toggleEnabledManifest(raw, enabled)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if !changed {
|
||||
if err := pinned.verifyLive(); err != nil {
|
||||
return false, err
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// Read the lockfile before the edit so the refresh below only runs when the
|
||||
// installed tree still matched its recorded hash; a toggle must never mask
|
||||
// pre-existing drift.
|
||||
lock, entry, hashClean := cleanLockEntry(plugin, pinned.dir)
|
||||
|
||||
if err := publishManifest(pinned.dir, manifestName, data, perm); err != nil {
|
||||
return false, err
|
||||
}
|
||||
if !hashClean {
|
||||
if err := pinned.verifyLive(); err != nil {
|
||||
return false, err
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
after, err := hashRootedTree(pinned.dir)
|
||||
if err != nil {
|
||||
return true, fmt.Errorf("plugin manifest updated but %s hash refresh failed: %w", LockFileName, err)
|
||||
}
|
||||
if err := pinned.verifyLive(); err != nil {
|
||||
return false, err
|
||||
}
|
||||
entry.Hash = after
|
||||
lock[plugin.ID] = entry
|
||||
if err := writeLock(plugin.Root, lock); err != nil {
|
||||
return true, fmt.Errorf("plugin manifest updated but %s hash refresh failed: %w", LockFileName, err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// errPluginDirReplaced reports that the plugin's directory entry no longer
|
||||
// names the directory the toggle pinned and edited.
|
||||
var errPluginDirReplaced = errors.New("plugin directory was replaced while it was being updated; the plugin installed there now was not changed")
|
||||
|
||||
// pinnedPluginDir is the plugin directory discovery found, held open for the
|
||||
// length of one toggle.
|
||||
type pinnedPluginDir struct {
|
||||
root *os.Root // the plugin's install root
|
||||
name string // the plugin directory's entry name in root
|
||||
path string // the plugin directory's path, for messages
|
||||
dir *os.Root // the plugin directory itself
|
||||
}
|
||||
|
||||
// Close releases both handles.
|
||||
func (p *pinnedPluginDir) Close() {
|
||||
_ = p.dir.Close()
|
||||
_ = p.root.Close()
|
||||
}
|
||||
|
||||
// verifyLive confirms that the plugin's entry in its install root is still a
|
||||
// real directory, not a symlink, junction, or other reparse point, and is the
|
||||
// very directory p.dir holds open.
|
||||
func (p *pinnedPluginDir) verifyLive() error {
|
||||
opened, err := p.dir.Stat(".")
|
||||
if err != nil {
|
||||
return fmt.Errorf("inspect plugin directory: %w", err)
|
||||
}
|
||||
entry, err := p.root.Lstat(p.name)
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return fmt.Errorf("%w: %s", errPluginDirReplaced, p.path)
|
||||
}
|
||||
if err != nil {
|
||||
return fmt.Errorf("inspect plugin directory: %w", err)
|
||||
}
|
||||
if entry.Mode()&(os.ModeSymlink|os.ModeIrregular) != 0 {
|
||||
return fmt.Errorf("%w: plugin directory %s", pathjail.ErrReparse, p.path)
|
||||
}
|
||||
if !os.SameFile(entry, opened) {
|
||||
return fmt.Errorf("%w: %s", errPluginDirReplaced, p.path)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// openPluginDir pins the plugin directory discovery found, confined to the
|
||||
// plugin's install root.
|
||||
//
|
||||
// A link swapped in before the open is refused here. One swapped in after it
|
||||
// cannot redirect anything, because every later read, write, and hash is
|
||||
// relative to the returned handle rather than to a path; verifyLive then
|
||||
// detects that the path no longer names the edited directory.
|
||||
func openPluginDir(plugin LoadedPlugin) (*pinnedPluginDir, error) {
|
||||
root, name, err := pathjail.Open(plugin.Root, plugin.PluginDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open plugin directory: %w", err)
|
||||
}
|
||||
if name == "." || filepath.Base(name) != name {
|
||||
_ = root.Close()
|
||||
return nil, fmt.Errorf("%w: plugin directory %s is not an entry of %s", pathjail.ErrEscapes, plugin.PluginDir, plugin.Root)
|
||||
}
|
||||
if err := pathjail.RefuseReparse(root, name); err != nil {
|
||||
_ = root.Close()
|
||||
return nil, fmt.Errorf("open plugin directory: %w", err)
|
||||
}
|
||||
dir, err := root.OpenRoot(name)
|
||||
if err != nil {
|
||||
_ = root.Close()
|
||||
return nil, fmt.Errorf("open plugin directory: %w", err)
|
||||
}
|
||||
pinned := &pinnedPluginDir{root: root, name: name, path: plugin.PluginDir, dir: dir}
|
||||
if err := pinned.verifyLive(); err != nil {
|
||||
pinned.Close()
|
||||
return nil, err
|
||||
}
|
||||
return pinned, nil
|
||||
}
|
||||
|
||||
// readManifest reads name from dir and returns its bytes and permission bits.
|
||||
// Failing on a stat error keeps the publish from widening the permissions.
|
||||
func readManifest(dir *os.Root, name string) ([]byte, os.FileMode, error) {
|
||||
file, err := dir.Open(name)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("read plugin manifest: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
info, err := file.Stat()
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("inspect plugin manifest: %w", err)
|
||||
}
|
||||
raw, err := io.ReadAll(file)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("read plugin manifest: %w", err)
|
||||
}
|
||||
return raw, info.Mode().Perm(), nil
|
||||
}
|
||||
|
||||
// publishManifest replaces name in dir with data by renaming a complete
|
||||
// temporary sibling over it, so a concurrent reader sees either the old
|
||||
// manifest or the new one, never a partial write.
|
||||
func publishManifest(dir *os.Root, name string, data []byte, perm os.FileMode) error {
|
||||
temp, tempName, err := pathjail.CreateTemp(dir, ".", ".zero-manifest", "")
|
||||
if err != nil {
|
||||
return fmt.Errorf("write plugin manifest: %w", err)
|
||||
}
|
||||
defer func() { _ = dir.Remove(tempName) }()
|
||||
if err := temp.Chmod(perm); err != nil {
|
||||
_ = temp.Close()
|
||||
return fmt.Errorf("write plugin manifest: %w", err)
|
||||
}
|
||||
if _, err := temp.Write(data); err != nil {
|
||||
_ = temp.Close()
|
||||
return fmt.Errorf("write plugin manifest: %w", err)
|
||||
}
|
||||
if err := temp.Sync(); err != nil {
|
||||
_ = temp.Close()
|
||||
return fmt.Errorf("write plugin manifest: %w", err)
|
||||
}
|
||||
if err := temp.Close(); err != nil {
|
||||
return fmt.Errorf("write plugin manifest: %w", err)
|
||||
}
|
||||
if err := dir.Rename(tempName, name); err != nil {
|
||||
return fmt.Errorf("write plugin manifest: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// cleanLockEntry reports whether the pinned plugin directory dir still matches
|
||||
// the lockfile entry recorded for the plugin and returns both for a refresh.
|
||||
func cleanLockEntry(plugin LoadedPlugin, dir *os.Root) (map[string]LockEntry, LockEntry, bool) {
|
||||
lock, err := ReadLock(plugin.Root)
|
||||
if err != nil {
|
||||
return nil, LockEntry{}, false
|
||||
}
|
||||
entry, ok := lock[plugin.ID]
|
||||
if !ok {
|
||||
return nil, LockEntry{}, false
|
||||
}
|
||||
current, err := hashRootedTree(dir)
|
||||
if err != nil || current != entry.Hash {
|
||||
return nil, LockEntry{}, false
|
||||
}
|
||||
return lock, entry, true
|
||||
}
|
||||
|
||||
// toggleEnabledManifest returns raw with the top-level "enabled" member set to
|
||||
// enabled. A missing or null value counts as enabled, matching the loader's
|
||||
// optionalBool fallback. changed reports whether the result differs from raw.
|
||||
func toggleEnabledManifest(raw []byte, enabled bool) (out []byte, changed bool, err error) {
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
openTok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
|
||||
}
|
||||
if delim, ok := openTok.(json.Delim); !ok || delim != '{' {
|
||||
return nil, false, errors.New("plugin manifest must be a JSON object")
|
||||
}
|
||||
openEnd := dec.InputOffset()
|
||||
|
||||
current := true
|
||||
lastInvalid := false
|
||||
var valueStart, valueEnd int64 = -1, -1
|
||||
for dec.More() {
|
||||
keyTok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
|
||||
}
|
||||
key, ok := keyTok.(string)
|
||||
if !ok {
|
||||
return nil, false, errors.New("plugin manifest must be a JSON object")
|
||||
}
|
||||
keyEnd := dec.InputOffset()
|
||||
if key != "enabled" {
|
||||
if err := skipManifestValue(dec); err != nil {
|
||||
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
valueTok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
|
||||
}
|
||||
var literal string
|
||||
switch value := valueTok.(type) {
|
||||
case bool:
|
||||
current = value
|
||||
literal = "false"
|
||||
if value {
|
||||
literal = "true"
|
||||
}
|
||||
case nil:
|
||||
current = true
|
||||
literal = "null"
|
||||
default:
|
||||
// Only the last occurrence of a duplicate key counts, as in the
|
||||
// loader's json.Unmarshal, so an earlier non-boolean value that a
|
||||
// later one overwrites is not an error. Remember it and decide after
|
||||
// the last member.
|
||||
if err := skipManifestComposite(dec, valueTok); err != nil {
|
||||
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
|
||||
}
|
||||
lastInvalid = true
|
||||
continue
|
||||
}
|
||||
lastInvalid = false
|
||||
start, err := enabledValueOffset(raw, keyEnd)
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
if start+int64(len(literal)) > int64(len(raw)) ||
|
||||
!bytes.Equal(raw[start:start+int64(len(literal))], []byte(literal)) {
|
||||
return nil, false, errors.New("parse plugin manifest: cannot locate the enabled value")
|
||||
}
|
||||
// A duplicate key keeps the last occurrence, matching json.Unmarshal.
|
||||
valueStart, valueEnd = start, start+int64(len(literal))
|
||||
}
|
||||
closingTok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("parse plugin manifest: %w", err)
|
||||
}
|
||||
if delim, ok := closingTok.(json.Delim); !ok || delim != '}' {
|
||||
return nil, false, errors.New("plugin manifest must be a JSON object")
|
||||
}
|
||||
closingEnd := dec.InputOffset()
|
||||
if _, err := dec.Token(); err != io.EOF {
|
||||
return nil, false, errors.New("parse plugin manifest: unexpected data after the top-level object")
|
||||
}
|
||||
if lastInvalid {
|
||||
return nil, false, ManifestError{FieldPath: "enabled", Message: "Expected a boolean."}
|
||||
}
|
||||
|
||||
if current == enabled {
|
||||
return nil, false, nil
|
||||
}
|
||||
newLiteral := "false"
|
||||
if enabled {
|
||||
newLiteral = "true"
|
||||
}
|
||||
if valueStart >= 0 {
|
||||
out := make([]byte, 0, len(raw)-int(valueEnd-valueStart)+len(newLiteral))
|
||||
out = append(out, raw[:valueStart]...)
|
||||
out = append(out, newLiteral...)
|
||||
out = append(out, raw[valueEnd:]...)
|
||||
return out, true, nil
|
||||
}
|
||||
return insertEnabledMember(raw, openEnd, closingEnd-1, newLiteral), true, nil
|
||||
}
|
||||
|
||||
// enabledValueOffset returns the byte offset of the value that follows the
|
||||
// colon after an "enabled" key token that ended at keyEnd.
|
||||
func enabledValueOffset(raw []byte, keyEnd int64) (int64, error) {
|
||||
i := keyEnd
|
||||
for i < int64(len(raw)) && isJSONSpace(raw[i]) {
|
||||
i++
|
||||
}
|
||||
if i >= int64(len(raw)) || raw[i] != ':' {
|
||||
return 0, errors.New("parse plugin manifest: missing colon after key")
|
||||
}
|
||||
i++
|
||||
for i < int64(len(raw)) && isJSONSpace(raw[i]) {
|
||||
i++
|
||||
}
|
||||
return i, nil
|
||||
}
|
||||
|
||||
// insertEnabledMember adds a top-level "enabled" member to a manifest that has
|
||||
// none, mirroring the document's own whitespace style where it can. openEnd is
|
||||
// the byte offset just after the opening brace and bracePos is the byte index
|
||||
// of the closing brace.
|
||||
func insertEnabledMember(raw []byte, openEnd, bracePos int64, literal string) []byte {
|
||||
insertPos := bracePos
|
||||
for insertPos > openEnd && isJSONSpace(raw[insertPos-1]) {
|
||||
insertPos--
|
||||
}
|
||||
|
||||
newline := "\n"
|
||||
if bytes.Contains(raw, []byte("\r\n")) {
|
||||
newline = "\r\n"
|
||||
}
|
||||
|
||||
first := openEnd
|
||||
for first < bracePos && isJSONSpace(raw[first]) {
|
||||
first++
|
||||
}
|
||||
empty := first >= bracePos
|
||||
multiline := bytes.IndexByte(raw[openEnd:bracePos], '\n') >= 0
|
||||
|
||||
member := `"enabled": ` + literal
|
||||
var insert string
|
||||
switch {
|
||||
case empty && !multiline:
|
||||
insert = member
|
||||
case empty && multiline:
|
||||
insert = newline + closingLineIndent(raw, openEnd, bracePos) + " " + member
|
||||
case !empty && !multiline:
|
||||
insert = "," + member
|
||||
default:
|
||||
insert = "," + newline + memberLineIndent(raw, openEnd, first) + member
|
||||
}
|
||||
|
||||
out := make([]byte, 0, len(raw)+len(insert))
|
||||
out = append(out, raw[:insertPos]...)
|
||||
out = append(out, insert...)
|
||||
out = append(out, raw[insertPos:]...)
|
||||
return out
|
||||
}
|
||||
|
||||
// memberLineIndent returns the indentation of the line the first member of a
|
||||
// JSON object starts on. first is the byte index of that member's first token.
|
||||
func memberLineIndent(raw []byte, openEnd, first int64) string {
|
||||
lineStart := first
|
||||
for lineStart > openEnd && raw[lineStart-1] != '\n' {
|
||||
lineStart--
|
||||
}
|
||||
end := lineStart
|
||||
for end < first && (raw[end] == ' ' || raw[end] == '\t') {
|
||||
end++
|
||||
}
|
||||
return string(raw[lineStart:end])
|
||||
}
|
||||
|
||||
// closingLineIndent returns the indentation before an object's closing brace.
|
||||
func closingLineIndent(raw []byte, openEnd, bracePos int64) string {
|
||||
start := bracePos
|
||||
for start > openEnd && (raw[start-1] == ' ' || raw[start-1] == '\t') {
|
||||
start--
|
||||
}
|
||||
return string(raw[start:bracePos])
|
||||
}
|
||||
|
||||
// skipManifestValue consumes the next JSON value from dec.
|
||||
func skipManifestValue(dec *json.Decoder) error {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return skipManifestComposite(dec, tok)
|
||||
}
|
||||
|
||||
// skipManifestComposite consumes the rest of the value whose first token tok
|
||||
// was just read from dec. A scalar is already complete; an object or array is
|
||||
// consumed through its matching closing delimiter.
|
||||
func skipManifestComposite(dec *json.Decoder, tok json.Token) error {
|
||||
delim, ok := tok.(json.Delim)
|
||||
if !ok || (delim != '{' && delim != '[') {
|
||||
return nil
|
||||
}
|
||||
depth := 1
|
||||
for depth > 0 {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if inner, ok := tok.(json.Delim); ok {
|
||||
switch inner {
|
||||
case '{', '[':
|
||||
depth++
|
||||
case '}', ']':
|
||||
depth--
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func isJSONSpace(b byte) bool {
|
||||
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
|
||||
}
|
||||
@@ -0,0 +1,999 @@
|
||||
package plugins
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Twigpine/zero/internal/installtxn"
|
||||
"github.com/Twigpine/zero/internal/pathjail"
|
||||
)
|
||||
|
||||
func enableTestOptions(root string) LoadOptions {
|
||||
return LoadOptions{Roots: []Root{{Source: SourceUser, Path: root}}}
|
||||
}
|
||||
|
||||
func writeRawManifest(t *testing.T, root, dir, raw string) string {
|
||||
t.Helper()
|
||||
pluginDir := filepath.Join(root, dir)
|
||||
if err := os.MkdirAll(pluginDir, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
manifestPath := filepath.Join(pluginDir, "plugin.json")
|
||||
if err := os.WriteFile(manifestPath, []byte(raw), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return manifestPath
|
||||
}
|
||||
|
||||
func demoManifest(enabled bool) string {
|
||||
state := "false"
|
||||
if enabled {
|
||||
state = "true"
|
||||
}
|
||||
return `{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
"enabled": ` + state + `
|
||||
}
|
||||
`
|
||||
}
|
||||
|
||||
func TestSetEnabledByIDPreservesManifestBytes(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
raw := `{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
"limits": {
|
||||
"maxBytes": 9007199254740993
|
||||
},
|
||||
"description": "keep me",
|
||||
"enabled": true,
|
||||
"tools": []
|
||||
}
|
||||
`
|
||||
manifestPath := writeRawManifest(t, root, "demo", raw)
|
||||
|
||||
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID(false): %v", err)
|
||||
}
|
||||
if !result.Changed || result.Enabled {
|
||||
t.Fatalf("unexpected result: %#v", result)
|
||||
}
|
||||
|
||||
disabled, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := strings.Replace(raw, `"enabled": true`, `"enabled": false`, 1)
|
||||
if string(disabled) != want {
|
||||
t.Fatalf("toggle changed more than the enabled value:\n got %q\nwant %q", disabled, want)
|
||||
}
|
||||
if !strings.Contains(string(disabled), "9007199254740993") {
|
||||
t.Fatal("large integer literal was rounded by the toggle")
|
||||
}
|
||||
|
||||
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("FindByID after disable: ok=%v err=%v", ok, err)
|
||||
}
|
||||
if plugin.Enabled {
|
||||
t.Fatal("loader still reports the plugin enabled after disable")
|
||||
}
|
||||
|
||||
result, err = SetEnabledByID(enableTestOptions(root), "zero.demo", true)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID(true): %v", err)
|
||||
}
|
||||
if !result.Changed || !result.Enabled {
|
||||
t.Fatalf("unexpected enable result: %#v", result)
|
||||
}
|
||||
enabled, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(enabled) != raw {
|
||||
t.Fatalf("round trip did not restore the original bytes:\n got %q\nwant %q", enabled, raw)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetEnabledByIDTogglesManifestAndIsIdempotent(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writePluginManifest(t, filepath.Join(root, "demo"), map[string]any{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
"description": "keep me",
|
||||
"tools": []any{},
|
||||
})
|
||||
manifestPath := filepath.Join(root, "demo", "plugin.json")
|
||||
|
||||
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID(false): %v", err)
|
||||
}
|
||||
if !result.Changed {
|
||||
t.Fatal("expected first disable to change the manifest")
|
||||
}
|
||||
raw, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(raw), `"enabled": false`) {
|
||||
t.Fatalf("manifest not disabled: %s", raw)
|
||||
}
|
||||
if !strings.Contains(string(raw), `"keep me"`) {
|
||||
t.Fatalf("description was not preserved: %s", raw)
|
||||
}
|
||||
|
||||
result, err = SetEnabledByID(enableTestOptions(root), "zero.demo", false)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID(false) again: %v", err)
|
||||
}
|
||||
if result.Changed {
|
||||
t.Fatal("expected second disable to be a no-op")
|
||||
}
|
||||
|
||||
result, err = SetEnabledByID(enableTestOptions(root), "zero.demo", true)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID(true): %v", err)
|
||||
}
|
||||
if !result.Changed {
|
||||
t.Fatal("expected enable to change the manifest")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetEnabledByIDInsertsMissingField(t *testing.T) {
|
||||
t.Run("pretty manifest", func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
raw := `{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0"
|
||||
}
|
||||
`
|
||||
manifestPath := writeRawManifest(t, root, "demo", raw)
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
|
||||
t.Fatalf("SetEnabledByID(false): %v", err)
|
||||
}
|
||||
disabled, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantDisabled := `{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
"enabled": false
|
||||
}
|
||||
`
|
||||
if string(disabled) != wantDisabled {
|
||||
t.Fatalf("after insert:\n got %q\nwant %q", disabled, wantDisabled)
|
||||
}
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", true); err != nil {
|
||||
t.Fatalf("SetEnabledByID(true): %v", err)
|
||||
}
|
||||
enabled, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantEnabled := strings.Replace(wantDisabled, `"enabled": false`, `"enabled": true`, 1)
|
||||
if string(enabled) != wantEnabled {
|
||||
t.Fatalf("after replace:\n got %q\nwant %q", enabled, wantEnabled)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("compact manifest", func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
raw := `{"schemaVersion":1,"id":"zero.demo","name":"Demo","version":"1.0.0"}`
|
||||
manifestPath := writeRawManifest(t, root, "demo", raw)
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
|
||||
t.Fatalf("SetEnabledByID(false): %v", err)
|
||||
}
|
||||
disabled, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := `{"schemaVersion":1,"id":"zero.demo","name":"Demo","version":"1.0.0","enabled": false}`
|
||||
if string(disabled) != want {
|
||||
t.Fatalf("compact insert:\n got %q\nwant %q", disabled, want)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("null value", func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
raw := `{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
"enabled": null
|
||||
}
|
||||
`
|
||||
manifestPath := writeRawManifest(t, root, "demo", raw)
|
||||
|
||||
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", true)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID(true): %v", err)
|
||||
}
|
||||
if result.Changed {
|
||||
t.Fatal("null counts as enabled, so enabling must be a no-op")
|
||||
}
|
||||
unchanged, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(unchanged) != raw {
|
||||
t.Fatalf("no-op toggle rewrote the manifest: %q", unchanged)
|
||||
}
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
|
||||
t.Fatalf("SetEnabledByID(false): %v", err)
|
||||
}
|
||||
disabled, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := strings.Replace(raw, `"enabled": null`, `"enabled": false`, 1)
|
||||
if string(disabled) != want {
|
||||
t.Fatalf("null replacement:\n got %q\nwant %q", disabled, want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestToggleEnabledManifestFormattingEdges(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
in string
|
||||
enabled bool
|
||||
want string
|
||||
changed bool
|
||||
}{
|
||||
{"empty object", `{}`, false, `{"enabled": false}`, true},
|
||||
{"empty object with space", `{ }`, false, `{"enabled": false }`, true},
|
||||
{"empty multiline", "{\n}", false, "{\n \"enabled\": false\n}", true},
|
||||
{"compact members", `{"a":1}`, false, `{"a":1,"enabled": false}`, true},
|
||||
{"crlf members", "{\r\n \"a\": 1\r\n}", false, "{\r\n \"a\": 1,\r\n \"enabled\": false\r\n}", true},
|
||||
{"nested enabled untouched", `{"nested":{"enabled":true},"a":2}`, false, `{"nested":{"enabled":true},"a":2,"enabled": false}`, true},
|
||||
{"replace with true", `{"enabled": false}`, true, `{"enabled": true}`, true},
|
||||
{"replace with false", `{"enabled":true}`, false, `{"enabled":false}`, true},
|
||||
{"null to false", `{"enabled": null}`, false, `{"enabled": false}`, true},
|
||||
{"null to true is a no-op", `{"enabled": null}`, true, `{"enabled": null}`, false},
|
||||
{"already enabled is a no-op", `{"enabled": true}`, true, `{"enabled": true}`, false},
|
||||
// The loader keeps the last duplicate, so an overwritten non-boolean
|
||||
// value is not an error and only the last value is rewritten.
|
||||
{"overwritten string", `{"enabled":"obsolete","enabled":true}`, false, `{"enabled":"obsolete","enabled":false}`, true},
|
||||
{"overwritten composite", `{"enabled":{"a":[1,{"enabled":2}]},"enabled":null}`, false, `{"enabled":{"a":[1,{"enabled":2}]},"enabled":false}`, true},
|
||||
{"overwritten number is a no-op", `{"enabled":1,"enabled":false}`, false, `{"enabled":1,"enabled":false}`, false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
out, changed, err := toggleEnabledManifest([]byte(tc.in), tc.enabled)
|
||||
if err != nil {
|
||||
t.Fatalf("toggleEnabledManifest: %v", err)
|
||||
}
|
||||
if changed != tc.changed {
|
||||
t.Fatalf("changed = %v, want %v", changed, tc.changed)
|
||||
}
|
||||
if !tc.changed {
|
||||
if out != nil {
|
||||
t.Fatalf("no-op returned bytes: %q", out)
|
||||
}
|
||||
return
|
||||
}
|
||||
if string(out) != tc.want {
|
||||
t.Fatalf("got %q, want %q", out, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestToggleEnabledManifestRejectsMalformedInput(t *testing.T) {
|
||||
for _, raw := range []string{`{"enabled": "yes"}`, `{"enabled": 1}`, `{"enabled": []}`, `{"enabled": {}}`, `{"enabled": true, "enabled": "yes"}`} {
|
||||
t.Run("value "+raw, func(t *testing.T) {
|
||||
_, _, err := toggleEnabledManifest([]byte(raw), false)
|
||||
if err == nil {
|
||||
t.Fatal("expected error")
|
||||
}
|
||||
var manifestErr ManifestError
|
||||
if !errors.As(err, &manifestErr) {
|
||||
t.Fatalf("error = %v, want ManifestError", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "enabled") {
|
||||
t.Fatalf("error = %v, want it to name the enabled field", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, raw := range []string{``, `{`, `[]`, `null`, `"str"`, `{} {}`, `{}{}`} {
|
||||
t.Run("document "+raw, func(t *testing.T) {
|
||||
if _, _, err := toggleEnabledManifest([]byte(raw), false); err == nil {
|
||||
t.Fatalf("toggleEnabledManifest(%q) succeeded, want error", raw)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledByIDHonorsTheLastDuplicateEnabledValue keeps the toggle in step
|
||||
// with the loader on a manifest whose earlier "enabled" value is overwritten:
|
||||
// the loader discovers the plugin from the last value, so the toggle must be
|
||||
// able to change it too.
|
||||
func TestSetEnabledByIDHonorsTheLastDuplicateEnabledValue(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeRawManifest(t, root, "demo", `{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
"enabled": "obsolete",
|
||||
"enabled": true
|
||||
}
|
||||
`)
|
||||
loadedEnabled := func() bool {
|
||||
t.Helper()
|
||||
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("FindByID: ok=%v err=%v", ok, err)
|
||||
}
|
||||
return plugin.Enabled
|
||||
}
|
||||
if !loadedEnabled() {
|
||||
t.Fatal("the loader should discover the plugin as enabled from the last value")
|
||||
}
|
||||
for _, enabled := range []bool{false, true} {
|
||||
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", enabled)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID(%v): %v", enabled, err)
|
||||
}
|
||||
if !result.Changed {
|
||||
t.Fatalf("SetEnabledByID(%v) reported no change", enabled)
|
||||
}
|
||||
if got := loadedEnabled(); got != enabled {
|
||||
t.Fatalf("after SetEnabledByID(%v) the loader sees enabled=%v", enabled, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledRefusesASwappedPluginDirectory swaps the plugin directory that
|
||||
// discovery found for a link between the lookup and the edit. The install lock
|
||||
// does not cover that window, because it only serializes cooperating
|
||||
// installers. The edit must fail without following the link: neither the
|
||||
// link's target nor the moved-away original may change.
|
||||
func TestSetEnabledRefusesASwappedPluginDirectory(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
// link returns the symlink target to plant and the manifest it leads to.
|
||||
link func(t *testing.T, root string) (target string, manifest string)
|
||||
}{
|
||||
{"to a directory outside the plugins root", func(t *testing.T, root string) (string, string) {
|
||||
manifest := writeRawManifest(t, t.TempDir(), "elsewhere", demoManifest(true))
|
||||
return filepath.Dir(manifest), manifest
|
||||
}},
|
||||
{"to another plugin in the same root", func(t *testing.T, root string) (string, string) {
|
||||
// A relative link stays inside the root, so confinement to the
|
||||
// root alone would follow it.
|
||||
manifest := writeRawManifest(t, root, "other", strings.Replace(demoManifest(true), "zero.demo", "zero.other", 1))
|
||||
return "other", manifest
|
||||
}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
raw := demoManifest(true)
|
||||
writeRawManifest(t, root, "demo", raw)
|
||||
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("FindByID: ok=%v err=%v", ok, err)
|
||||
}
|
||||
target, targetManifest := tc.link(t, root)
|
||||
targetBefore, err := os.ReadFile(targetManifest)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
moved := filepath.Join(t.TempDir(), "moved-away")
|
||||
if err := os.Rename(plugin.PluginDir, moved); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(target, plugin.PluginDir); err != nil {
|
||||
t.Skipf("cannot create a directory symlink here: %v", err)
|
||||
}
|
||||
|
||||
if _, err := setEnabledLocked(plugin, false); !errors.Is(err, pathjail.ErrReparse) {
|
||||
t.Fatalf("setEnabledLocked through a swapped directory: err = %v, want %v", err, pathjail.ErrReparse)
|
||||
}
|
||||
if after, err := os.ReadFile(targetManifest); err != nil || string(after) != string(targetBefore) {
|
||||
t.Fatalf("the edit followed the link and changed %s (err %v):\n got %q\nwant %q", targetManifest, err, after, targetBefore)
|
||||
}
|
||||
if after, err := os.ReadFile(filepath.Join(moved, "plugin.json")); err != nil || string(after) != raw {
|
||||
t.Fatalf("the moved-away manifest changed (err %v): %q", err, after)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledDetectsAPluginDirectoryReplacedAfterOpening replaces the live
|
||||
// plugin directory after the toggle pinned it. The edit then lands in the
|
||||
// moved-away directory, so the toggle must not refresh the lockfile from either
|
||||
// tree and must not report success: the plugin installed at the path now is
|
||||
// unchanged.
|
||||
func TestSetEnabledDetectsAPluginDirectoryReplacedAfterOpening(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
pinned bool // the pinned manifest's enabled value
|
||||
changes bool // whether disabling it is a real edit
|
||||
}{
|
||||
{"edit", true, true},
|
||||
{"no-op", false, false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
manifestPath := writeRawManifest(t, root, "demo", demoManifest(tc.pinned))
|
||||
pluginDir := filepath.Dir(manifestPath)
|
||||
hash, err := hashTree(pluginDir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := writeLock(root, map[string]LockEntry{"zero.demo": {Source: "./demo", Hash: hash}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lockPath := filepath.Join(root, LockFileName)
|
||||
lockBefore, err := os.ReadFile(lockPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
plugin, ok, err := FindByID(enableTestOptions(root), "zero.demo")
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("FindByID: ok=%v err=%v", ok, err)
|
||||
}
|
||||
pinned, err := openPluginDir(plugin)
|
||||
if err != nil {
|
||||
t.Fatalf("openPluginDir: %v", err)
|
||||
}
|
||||
defer pinned.Close()
|
||||
|
||||
moved := filepath.Join(t.TempDir(), "moved-away")
|
||||
if err := os.Rename(pluginDir, moved); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
replacement := strings.Replace(demoManifest(true), `"version": "1.0.0"`, `"version": "2.0.0"`, 1)
|
||||
writeRawManifest(t, root, "demo", replacement)
|
||||
|
||||
// The lockfile check hashes the pinned tree, which still matches the
|
||||
// recorded hash, not the replacement now at the path.
|
||||
if _, _, clean := cleanLockEntry(plugin, pinned.dir); !clean {
|
||||
t.Error("cleanLockEntry hashed the replacement at the path instead of the pinned directory")
|
||||
}
|
||||
|
||||
if _, err := setEnabledPinned(plugin, pinned, false); !errors.Is(err, errPluginDirReplaced) {
|
||||
t.Fatalf("setEnabledPinned after the directory was replaced: err = %v, want %v", err, errPluginDirReplaced)
|
||||
}
|
||||
if after, err := os.ReadFile(manifestPath); err != nil || string(after) != replacement {
|
||||
t.Fatalf("the replacement manifest changed (err %v): %q", err, after)
|
||||
}
|
||||
if after, err := os.ReadFile(lockPath); err != nil || string(after) != string(lockBefore) {
|
||||
t.Fatalf("the lockfile changed (err %v):\n got %s\nwant %s", err, after, lockBefore)
|
||||
}
|
||||
movedRaw, err := os.ReadFile(filepath.Join(moved, "plugin.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tc.changes != (string(movedRaw) != demoManifest(tc.pinned)) {
|
||||
t.Fatalf("pinned manifest edit = %v, want %v: %q", !tc.changes, tc.changes, movedRaw)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetEnabledByIDRespectsUserOnlyFilter(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
userRoot := filepath.Join(dir, "user")
|
||||
projectRoot := filepath.Join(dir, "project")
|
||||
writePluginManifest(t, filepath.Join(userRoot, "demo"), map[string]any{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "User Demo",
|
||||
"version": "0.1.0",
|
||||
"enabled": true,
|
||||
})
|
||||
writePluginManifest(t, filepath.Join(projectRoot, "demo"), map[string]any{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Project Demo",
|
||||
"version": "0.2.0",
|
||||
"enabled": true,
|
||||
})
|
||||
|
||||
result, err := SetEnabledByID(LoadOptions{
|
||||
Roots: []Root{
|
||||
{Source: SourceUser, Path: userRoot},
|
||||
{Source: SourceProject, Path: projectRoot},
|
||||
},
|
||||
}, "zero.demo", false)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID: %v", err)
|
||||
}
|
||||
if result.Source != SourceProject {
|
||||
t.Fatalf("source = %s, want project (precedence)", result.Source)
|
||||
}
|
||||
if !result.Changed || result.Enabled {
|
||||
t.Fatalf("unexpected result: %#v", result)
|
||||
}
|
||||
|
||||
userOnly, err := SetEnabledByID(LoadOptions{
|
||||
Roots: []Root{
|
||||
{Source: SourceUser, Path: userRoot},
|
||||
{Source: SourceProject, Path: projectRoot},
|
||||
},
|
||||
ExcludeProject: true,
|
||||
}, "zero.demo", false)
|
||||
if err != nil {
|
||||
t.Fatalf("SetEnabledByID user-only: %v", err)
|
||||
}
|
||||
if userOnly.Source != SourceUser {
|
||||
t.Fatalf("user-only source = %s, want user", userOnly.Source)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetEnabledPreservesManifestMode(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("Windows does not preserve chmod bits in os.Stat mode")
|
||||
}
|
||||
root := t.TempDir()
|
||||
writePluginManifest(t, filepath.Join(root, "demo"), map[string]any{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "1.0.0",
|
||||
})
|
||||
manifestPath := filepath.Join(root, "demo", "plugin.json")
|
||||
if err := os.Chmod(manifestPath, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
|
||||
t.Fatalf("SetEnabledByID: %v", err)
|
||||
}
|
||||
info, err := os.Stat(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := info.Mode().Perm(); got != 0o644 {
|
||||
t.Fatalf("mode = %04o, want 0644", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetEnabledByIDMissingPlugin(t *testing.T) {
|
||||
_, err := SetEnabledByID(LoadOptions{Roots: []Root{{Source: SourceUser, Path: t.TempDir()}}}, "missing", false)
|
||||
if err == nil {
|
||||
t.Fatal("expected missing plugin error")
|
||||
}
|
||||
if !errors.Is(err, ErrNotInstalled) {
|
||||
t.Fatalf("error = %v, want ErrNotInstalled", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetEnabledPublishFailureLeavesManifestIntact(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("Windows directory permissions do not block writes the way this test needs")
|
||||
}
|
||||
if os.Geteuid() == 0 {
|
||||
t.Skip("root bypasses directory permission checks")
|
||||
}
|
||||
root := t.TempDir()
|
||||
raw := demoManifest(true)
|
||||
manifestPath := writeRawManifest(t, root, "demo", raw)
|
||||
pluginDir := filepath.Dir(manifestPath)
|
||||
if err := os.Chmod(pluginDir, 0o555); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { _ = os.Chmod(pluginDir, 0o755) }()
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err == nil {
|
||||
t.Fatal("expected the publish to fail in a read-only plugin directory")
|
||||
}
|
||||
after, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(after) != raw {
|
||||
t.Fatalf("failed publish changed the manifest:\n got %q\nwant %q", after, raw)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledByIDWaitsForInstallRootLock proves the toggle takes the same
|
||||
// cross-process lock Install and Remove take on the plugin's install root: it
|
||||
// must not complete while an install holds that lock.
|
||||
func TestSetEnabledByIDWaitsForInstallRootLock(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
manifestPath := writeRawManifest(t, root, "demo", demoManifest(true))
|
||||
|
||||
unlock, err := installtxn.Lock(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
results := make(chan SetEnabledResult, 1)
|
||||
failures := make(chan error, 1)
|
||||
go func() {
|
||||
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
|
||||
if err != nil {
|
||||
failures <- err
|
||||
return
|
||||
}
|
||||
results <- result
|
||||
}()
|
||||
|
||||
select {
|
||||
case result := <-results:
|
||||
unlock()
|
||||
t.Fatalf("toggle completed while the install lock was held: %#v", result)
|
||||
case err := <-failures:
|
||||
unlock()
|
||||
t.Fatalf("toggle failed while the install lock was held: %v", err)
|
||||
case <-time.After(200 * time.Millisecond):
|
||||
}
|
||||
|
||||
unlock()
|
||||
|
||||
select {
|
||||
case result := <-results:
|
||||
if !result.Changed {
|
||||
t.Fatalf("toggle did not report a change: %#v", result)
|
||||
}
|
||||
case err := <-failures:
|
||||
t.Fatalf("toggle failed after the lock was released: %v", err)
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("toggle did not finish after the install lock was released")
|
||||
}
|
||||
|
||||
raw, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(raw), `"enabled": false`) {
|
||||
t.Fatalf("manifest not disabled: %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledByIDSynchronizesWithStagedInstallCommit pauses an install
|
||||
// between staging and commit, lets the toggle try to run, then commits the
|
||||
// install. The toggle must apply to the newly installed manifest instead of
|
||||
// overwriting it with bytes read before the install, and the lockfile entry
|
||||
// must describe the tree actually on disk.
|
||||
func TestSetEnabledByIDSynchronizesWithStagedInstallCommit(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
oldRaw := demoManifest(true)
|
||||
pluginDir := filepath.Dir(writeRawManifest(t, root, "zero.demo", oldRaw))
|
||||
|
||||
// Build the replacement tree with the same staging primitives Install
|
||||
// uses and leave it staged but uncommitted.
|
||||
src := t.TempDir()
|
||||
newRaw := `{
|
||||
"schemaVersion": 1,
|
||||
"id": "zero.demo",
|
||||
"name": "Demo",
|
||||
"version": "2.0.0",
|
||||
"description": "reinstalled",
|
||||
"enabled": true
|
||||
}
|
||||
`
|
||||
srcPlugin := filepath.Join(src, "zero.demo")
|
||||
if err := os.MkdirAll(srcPlugin, 0o700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(srcPlugin, "plugin.json"), []byte(newRaw), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
staged, cleanup, err := installtxn.StageDir(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer cleanup()
|
||||
if err := copyTree(srcPlugin, staged); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stagedHash, err := hashTree(staged)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// An install holds the root lock from before its commit until after it.
|
||||
unlock, err := installtxn.Lock(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
results := make(chan SetEnabledResult, 1)
|
||||
failures := make(chan error, 1)
|
||||
go func() {
|
||||
result, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
|
||||
if err != nil {
|
||||
failures <- err
|
||||
return
|
||||
}
|
||||
results <- result
|
||||
}()
|
||||
|
||||
select {
|
||||
case result := <-results:
|
||||
unlock()
|
||||
t.Fatalf("toggle completed while the install lock was held: %#v", result)
|
||||
case err := <-failures:
|
||||
unlock()
|
||||
t.Fatalf("toggle failed while the install lock was held: %v", err)
|
||||
case <-time.After(200 * time.Millisecond):
|
||||
}
|
||||
|
||||
// Commit the staged install exactly like Install does: swap the tree and
|
||||
// publish the lockfile entry while still holding the lock.
|
||||
lock, err := ReadLock(root)
|
||||
if err != nil {
|
||||
unlock()
|
||||
t.Fatal(err)
|
||||
}
|
||||
lock["zero.demo"] = LockEntry{Source: string(SourceUser), Hash: stagedHash}
|
||||
if err := installtxn.CommitDir(pluginDir, staged, func() error { return writeLock(root, lock) }); err != nil {
|
||||
unlock()
|
||||
t.Fatal(err)
|
||||
}
|
||||
unlock()
|
||||
|
||||
var result SetEnabledResult
|
||||
select {
|
||||
case result = <-results:
|
||||
case err := <-failures:
|
||||
t.Fatalf("toggle failed after the install committed: %v", err)
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("toggle did not finish after the install committed")
|
||||
}
|
||||
if !result.Changed {
|
||||
t.Fatalf("toggle result = %#v, want changed", result)
|
||||
}
|
||||
|
||||
// The toggle must have applied to the newly installed manifest, not to the
|
||||
// bytes it first read.
|
||||
final, err := os.ReadFile(filepath.Join(pluginDir, "plugin.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := strings.Replace(newRaw, `"enabled": true`, `"enabled": false`, 1)
|
||||
if string(final) != want {
|
||||
t.Fatalf("manifest after concurrent install + disable:\n got %q\nwant %q", final, want)
|
||||
}
|
||||
|
||||
// The lockfile entry must describe the tree that is actually on disk.
|
||||
entries, err := ReadLock(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
current, err := hashTree(pluginDir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entries["zero.demo"].Hash != current {
|
||||
t.Fatalf("plugins.lock hash = %q, want current tree hash %q", entries["zero.demo"].Hash, current)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSetEnabledByIDRefreshesCleanLockEntry(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
manifestPath := writeRawManifest(t, root, "zero.demo", demoManifest(true))
|
||||
pluginDir := filepath.Dir(manifestPath)
|
||||
|
||||
current, err := hashTree(pluginDir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := writeLock(root, map[string]LockEntry{"zero.demo": {Source: string(SourceUser), Hash: current}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false); err != nil {
|
||||
t.Fatalf("SetEnabledByID: %v", err)
|
||||
}
|
||||
entries, err := ReadLock(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
afterToggle, err := hashTree(pluginDir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entries["zero.demo"].Hash == current {
|
||||
t.Fatal("lock hash was not refreshed for a clean install")
|
||||
}
|
||||
if entries["zero.demo"].Hash != afterToggle {
|
||||
t.Fatalf("lock hash = %q, want refreshed tree hash %q", entries["zero.demo"].Hash, afterToggle)
|
||||
}
|
||||
|
||||
info, err := Info(InfoOptions{LoadOptions: enableTestOptions(root)}, "zero.demo")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.HashDrift {
|
||||
t.Fatal("toggle left hash drift behind for a clean install")
|
||||
}
|
||||
|
||||
// Pre-existing drift must survive the next toggle instead of being
|
||||
// absorbed into the lockfile.
|
||||
if err := os.WriteFile(filepath.Join(pluginDir, "extra.txt"), []byte("edited\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := SetEnabledByID(enableTestOptions(root), "zero.demo", true); err != nil {
|
||||
t.Fatalf("SetEnabledByID(true): %v", err)
|
||||
}
|
||||
entries, err = ReadLock(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entries["zero.demo"].Hash != afterToggle {
|
||||
t.Fatalf("toggle absorbed pre-existing drift: hash = %q, want %q", entries["zero.demo"].Hash, afterToggle)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledByIDReportsAPluginRemovedWhileWaiting removes the plugin while
|
||||
// the toggle is blocked on the install-root lock, exactly as Remove does while
|
||||
// it holds that lock. The toggle must report the plugin as not installed
|
||||
// instead of editing, or recreating, a tree that no longer exists.
|
||||
func TestSetEnabledByIDReportsAPluginRemovedWhileWaiting(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
pluginDir := filepath.Dir(writeRawManifest(t, root, "zero.demo", demoManifest(true)))
|
||||
|
||||
unlock, err := installtxn.Lock(root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
failures := make(chan error, 1)
|
||||
go func() {
|
||||
_, err := SetEnabledByID(enableTestOptions(root), "zero.demo", false)
|
||||
failures <- err
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-failures:
|
||||
unlock()
|
||||
t.Fatalf("toggle returned while the install lock was held: %v", err)
|
||||
case <-time.After(200 * time.Millisecond):
|
||||
}
|
||||
|
||||
if err := installtxn.RemoveDir(pluginDir, func() error { return nil }); err != nil {
|
||||
unlock()
|
||||
t.Fatal(err)
|
||||
}
|
||||
unlock()
|
||||
|
||||
select {
|
||||
case err := <-failures:
|
||||
if !errors.Is(err, ErrNotInstalled) {
|
||||
t.Fatalf("toggle after a concurrent removal: error = %v, want ErrNotInstalled", err)
|
||||
}
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("toggle did not finish after the removal released the lock")
|
||||
}
|
||||
if _, err := os.Stat(pluginDir); !os.IsNotExist(err) {
|
||||
t.Fatalf("toggle recreated the removed plugin: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledByIDResolvesInterruptedUpdatesFirst covers every point an
|
||||
// update can be killed at that leaves the plugin discoverable. The toggle
|
||||
// holds the install-root lock, so it has to
|
||||
// run the same recovery pass Install and Remove run before it reads the tree:
|
||||
// otherwise it edits, or fails to find, a tree that is still owed a restore,
|
||||
// and the next install or removal puts the backup back over the user's edit.
|
||||
func TestSetEnabledByIDResolvesInterruptedUpdatesFirst(t *testing.T) {
|
||||
for _, state := range interruptedStates() {
|
||||
t.Run(state.name, func(t *testing.T) {
|
||||
u := seedInterruptedUpdate(t)
|
||||
workspace := state.plant(t, u)
|
||||
want := state.want(u)
|
||||
if _, err := os.Lstat(u.target()); os.IsNotExist(err) {
|
||||
// No root lists a plugin whose only copy is in a workspace, so the
|
||||
// toggle reports it not installed without taking any lock, just as
|
||||
// list and info do; the next install or removal restores it.
|
||||
t.Skip("nothing is at the target, so the toggle never takes the lock")
|
||||
}
|
||||
|
||||
result, err := SetEnabledByID(enableTestOptions(u.dir), "zero.demo", false)
|
||||
if err != nil {
|
||||
t.Fatalf("toggle after the interrupted update: %v", err)
|
||||
}
|
||||
if !result.Changed || result.Enabled {
|
||||
t.Fatalf("toggle result = %#v, want changed and disabled", result)
|
||||
}
|
||||
assertDisabledAfterRecovery(t, u, workspace, want)
|
||||
|
||||
// The next lock holder runs recovery again. The disable has to survive
|
||||
// it rather than be replaced by a backup the toggle never saw.
|
||||
if err := installOther(t, u.dir, "zero.other"); err != nil {
|
||||
t.Fatalf("install after the toggle: %v", err)
|
||||
}
|
||||
assertDisabledAfterRecovery(t, u, workspace, want)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func assertDisabledAfterRecovery(t *testing.T, u interruptedUpdate, workspace string, want recoveryWant) {
|
||||
t.Helper()
|
||||
data, err := os.ReadFile(filepath.Join(u.target(), "generation.txt"))
|
||||
if err != nil {
|
||||
t.Fatalf("reading the live tree: %v", err)
|
||||
}
|
||||
if string(data) != want.generation {
|
||||
t.Errorf("the live tree is the %q generation, want %q", data, want.generation)
|
||||
}
|
||||
plugin, ok, err := FindByID(enableTestOptions(u.dir), "zero.demo")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !ok {
|
||||
t.Fatal("Load does not see the toggled plugin")
|
||||
}
|
||||
if plugin.Version != want.version {
|
||||
t.Errorf("Load sees version %q, want %q", plugin.Version, want.version)
|
||||
}
|
||||
if plugin.Enabled {
|
||||
t.Errorf("the disable did not stick: Load sees the plugin enabled")
|
||||
}
|
||||
_, err = os.Stat(workspace)
|
||||
if want.workspace && err != nil {
|
||||
t.Errorf("the workspace should be left alone, got %v", err)
|
||||
}
|
||||
if !want.workspace && !os.IsNotExist(err) {
|
||||
t.Errorf("the resolved workspace should be cleared, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSetEnabledByIDAbortsOnAnUnresolvedInterruptedUpdate leaves a workspace
|
||||
// whose trees match neither lockfile hash. Recovery cannot pick one, so the
|
||||
// toggle must fail without editing the live manifest or the lockfile.
|
||||
func TestSetEnabledByIDAbortsOnAnUnresolvedInterruptedUpdate(t *testing.T) {
|
||||
u := seedInterruptedUpdate(t)
|
||||
workspace := u.plantWorkspace(t)
|
||||
u.mustRename(t, u.target(), filepath.Join(workspace, "previous"))
|
||||
u.mustRename(t, filepath.Join(workspace, "staged"), u.target())
|
||||
unmatched := LockEntry{Source: u.oldSource, Hash: "sha256:0000000000000000000000000000000000000000000000000000000000000000"}
|
||||
lock, err := ReadLock(u.dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lock["zero.demo"] = unmatched
|
||||
if err := writeLock(u.dir, lock); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
manifestPath := filepath.Join(u.target(), "plugin.json")
|
||||
before, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if _, err := SetEnabledByID(enableTestOptions(u.dir), "zero.demo", false); err == nil {
|
||||
t.Fatal("toggle did not report the unresolved transaction")
|
||||
}
|
||||
after, err := os.ReadFile(manifestPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(after) != string(before) {
|
||||
t.Errorf("the aborted toggle edited the manifest:\n got %q\nwant %q", after, before)
|
||||
}
|
||||
assertUntouched(t, u, "new", unmatched)
|
||||
if _, err := os.Stat(filepath.Join(workspace, "previous")); err != nil {
|
||||
t.Errorf("the backup should still be there for a hand rescue, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -7,7 +7,8 @@ import (
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ErrNotInstalled is returned when Info cannot find a plugin with the requested id.
|
||||
// ErrNotInstalled is returned when a plugin id cannot be found under the given
|
||||
// LoadOptions (for example by Info or SetEnabledByID).
|
||||
var ErrNotInstalled = errors.New("plugin is not installed")
|
||||
|
||||
// PluginInfo combines a loaded plugin with optional lockfile metadata.
|
||||
|
||||
+37
-11
@@ -567,15 +567,32 @@ func validInstallID(id string) bool {
|
||||
// sorted order. Each file contributes its plugin-relative path, executable bit,
|
||||
// and bytes, so renames, mode flips, and content edits all change the hash.
|
||||
func hashTree(root string) (string, error) {
|
||||
handle, err := os.OpenRoot(root)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer handle.Close()
|
||||
return hashRootedTree(handle)
|
||||
}
|
||||
|
||||
// hashRootedTree is hashTree over a directory that is already open. Walking
|
||||
// relative to the handle hashes the directory it names even if its path is
|
||||
// replaced meanwhile, and keeps the walk inside it.
|
||||
//
|
||||
// The walk calls the handle's own methods rather than handle.FS(): the fs.FS
|
||||
// adapter only accepts fs.ValidPath names, which must be UTF-8, so it would
|
||||
// refuse file names that the filesystem and copyTree accept.
|
||||
func hashRootedTree(handle *os.Root) (string, error) {
|
||||
hasher := sha256.New()
|
||||
if err := hashTreeInto(hasher, root, root); err != nil {
|
||||
if err := hashTreeInto(hasher, handle, "."); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return "sha256:" + hex.EncodeToString(hasher.Sum(nil)), nil
|
||||
}
|
||||
|
||||
func hashTreeInto(hasher io.Writer, root string, dir string) error {
|
||||
entries, err := os.ReadDir(dir)
|
||||
// hashTreeInto hashes dir, a path relative to handle, into hasher.
|
||||
func hashTreeInto(hasher io.Writer, handle *os.Root, dir string) error {
|
||||
entries, err := readRootedDir(handle, dir)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -588,8 +605,8 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
|
||||
if name == ".git" {
|
||||
continue
|
||||
}
|
||||
path := filepath.Join(dir, name)
|
||||
info, err := os.Lstat(path)
|
||||
rel := filepath.Join(dir, name)
|
||||
info, err := handle.Lstat(rel)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -598,14 +615,10 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
|
||||
// Skipped by copyTree, so excluded from the hash too.
|
||||
continue
|
||||
case info.IsDir():
|
||||
if err := hashTreeInto(hasher, root, path); err != nil {
|
||||
if err := hashTreeInto(hasher, handle, rel); err != nil {
|
||||
return err
|
||||
}
|
||||
case info.Mode().IsRegular():
|
||||
rel, err := filepath.Rel(root, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
executable := 0
|
||||
if info.Mode().Perm()&0o111 != 0 {
|
||||
executable = 1
|
||||
@@ -616,7 +629,7 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
|
||||
if _, err := io.WriteString(hasher, header); err != nil {
|
||||
return err
|
||||
}
|
||||
file, err := os.Open(path)
|
||||
file, err := handle.Open(rel)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -634,3 +647,16 @@ func hashTreeInto(hasher io.Writer, root string, dir string) error {
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// readRootedDir lists dir, a path relative to handle, like os.ReadDir.
|
||||
func readRootedDir(handle *os.Root, dir string) ([]os.DirEntry, error) {
|
||||
file, err := handle.Open(dir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entries, err := file.ReadDir(-1)
|
||||
if closeErr := file.Close(); err == nil {
|
||||
err = closeErr
|
||||
}
|
||||
return entries, err
|
||||
}
|
||||
@@ -1181,3 +1181,62 @@ func TestRecoveryLeavesAPrefixCollidingDirectoryAlone(t *testing.T) {
|
||||
t.Errorf("recovery published content it does not own: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestNonUTF8FileNamesAreInstalledHashedAndToggled covers a plugin whose tree
|
||||
// holds a file and a directory named with bytes that are not valid UTF-8, which
|
||||
// Linux allows. copyTree copies them, so hashTree must hash them too: Install
|
||||
// records the hash, the toggle refreshes it, and Info sees an edit to that file
|
||||
// as drift. A walk that rejects such names breaks all three.
|
||||
func TestNonUTF8FileNamesAreInstalledHashedAndToggled(t *testing.T) {
|
||||
src := writeSourcePlugin(t, filepath.Join(t.TempDir(), "src"), validManifest())
|
||||
nested := filepath.Join("assets\xff", "data\xfe.bin")
|
||||
if err := os.Mkdir(filepath.Join(src, "assets\xff"), 0o755); err != nil {
|
||||
t.Skipf("this filesystem rejects non-UTF-8 names: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(src, nested), []byte("original\n"), 0o644); err != nil {
|
||||
t.Skipf("this filesystem rejects non-UTF-8 names: %v", err)
|
||||
}
|
||||
|
||||
destDir := t.TempDir()
|
||||
if _, err := Install(context.Background(), InstallOptions{Source: src, Dir: destDir}); err != nil {
|
||||
t.Fatalf("Install a plugin with a non-UTF-8 file name: %v", err)
|
||||
}
|
||||
pluginDir := filepath.Join(destDir, "zero.demo")
|
||||
if data, err := os.ReadFile(filepath.Join(pluginDir, nested)); err != nil || string(data) != "original\n" {
|
||||
t.Fatalf("installed copy of the non-UTF-8 file = %q, %v", data, err)
|
||||
}
|
||||
|
||||
options := LoadOptions{Roots: []Root{{Source: SourceUser, Path: destDir}}}
|
||||
before, err := ReadLock(destDir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := SetEnabledByID(options, "zero.demo", false); err != nil {
|
||||
t.Fatalf("SetEnabledByID on a plugin with a non-UTF-8 file name: %v", err)
|
||||
}
|
||||
after, err := ReadLock(destDir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after["zero.demo"].Hash == before["zero.demo"].Hash {
|
||||
t.Fatal("the toggle did not refresh the lock hash")
|
||||
}
|
||||
info, err := Info(InfoOptions{LoadOptions: options}, "zero.demo")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if info.HashDrift {
|
||||
t.Fatal("the toggle left hash drift behind")
|
||||
}
|
||||
|
||||
if err := os.WriteFile(filepath.Join(pluginDir, nested), []byte("edited\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
info, err = Info(InfoOptions{LoadOptions: options}, "zero.demo")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !info.HashDrift {
|
||||
t.Fatal("an edit to the non-UTF-8 file was not reported as drift")
|
||||
}
|
||||
}
|
||||
@@ -75,6 +75,12 @@ type Descriptor struct {
|
||||
// catalog-ordered lists and pickers. The recommended descriptors are the first
|
||||
// entries in the catalog, in order.
|
||||
Recommended bool
|
||||
|
||||
// LegacyBaseURLs lists earlier DefaultBaseURL values that still address
|
||||
// this provider's own endpoint. Profiles saved against one keep catalog
|
||||
// endpoint treatment (catalog env var binding, base-URL matching) after the
|
||||
// default moves; see IsCatalogBaseURL.
|
||||
LegacyBaseURLs []string
|
||||
}
|
||||
|
||||
func RuntimeSupported(descriptor Descriptor) bool {
|
||||
@@ -99,11 +105,17 @@ func RuntimeUnsupportedReason(descriptor Descriptor) string {
|
||||
}
|
||||
|
||||
var descriptors = []Descriptor{
|
||||
// GitLawb OpenGateway — the recommended default. An OpenAI-compatible gateway
|
||||
// that smart-routes by model id across upstream providers (xiaomi-mimo,
|
||||
// minimax, qwen, google, nvidia, tencent, z-ai). Flat /v1/chat/completions
|
||||
// with a Bearer ogw_live_… key; listed first and badged in every picker.
|
||||
recommended(openAICompat("gitlawb-opengateway", "GitLawb OpenGateway", "https://opengateway.gitlawb.com/v1", "mimo-v2.5-pro", []string{"GITLAWB_OPENGATEWAY_API_KEY"}, "gitlawb opengateway", "opengateway")),
|
||||
// Twigpine OpenGateway (formerly GitLawb OpenGateway) — the recommended
|
||||
// default. An OpenAI-compatible gateway at api.twigpine.com that smart-routes
|
||||
// by model id across upstream providers. Flat /v1/chat/completions with a
|
||||
// Bearer ogw_live_… key; listed first and badged in every picker. The public
|
||||
// GET /v1/models lists only active models and is what pickers show when
|
||||
// reachable. DefaultModel must be an exact id from that list (not an alias)
|
||||
// so onboarding keeps it selected after live discovery. The id and env var
|
||||
// keep their gitlawb names so saved configs keep resolving; the old host
|
||||
// (opengateway.gitlawb.com) still serves clients and stays a legacy catalog
|
||||
// endpoint so profiles saved against it resolve unchanged.
|
||||
recommended(withLegacyBaseURLs(openAICompat("gitlawb-opengateway", "Twigpine OpenGateway", "https://api.twigpine.com/v1", "xiaomi/mimo-v2.6-pro", []string{"GITLAWB_OPENGATEWAY_API_KEY"}, "gitlawb opengateway", "opengateway", "twigpine opengateway", "twigpine"), "https://opengateway.gitlawb.com/v1")),
|
||||
// aimlapi.com — OpenAI-compatible aggregating gateway, also badged as
|
||||
// recommended (second, right after the OpenGateway default). The built-in TUI
|
||||
// onboarding can register/top up an account and save the issued key; the partner
|
||||
@@ -307,6 +319,37 @@ func recommended(descriptor Descriptor) Descriptor {
|
||||
return descriptor
|
||||
}
|
||||
|
||||
// withLegacyBaseURLs records earlier default base URLs that still serve the
|
||||
// provider, so profiles saved before a host move keep resolving as the catalog
|
||||
// endpoint.
|
||||
func withLegacyBaseURLs(descriptor Descriptor, baseURLs ...string) Descriptor {
|
||||
descriptor.LegacyBaseURLs = append(descriptor.LegacyBaseURLs, baseURLs...)
|
||||
return descriptor
|
||||
}
|
||||
|
||||
// IsCatalogBaseURL reports whether baseURL addresses the descriptor's own
|
||||
// endpoint: its DefaultBaseURL or one of its LegacyBaseURLs, ignoring case,
|
||||
// surrounding space, and a trailing slash. An empty baseURL never matches.
|
||||
func IsCatalogBaseURL(descriptor Descriptor, baseURL string) bool {
|
||||
normalized := normalizeBaseURL(baseURL)
|
||||
if normalized == "" {
|
||||
return false
|
||||
}
|
||||
if normalized == normalizeBaseURL(descriptor.DefaultBaseURL) {
|
||||
return true
|
||||
}
|
||||
for _, legacy := range descriptor.LegacyBaseURLs {
|
||||
if normalized == normalizeBaseURL(legacy) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func normalizeBaseURL(baseURL string) string {
|
||||
return strings.ToLower(strings.TrimRight(strings.TrimSpace(baseURL), "/"))
|
||||
}
|
||||
|
||||
// oauthProvider marks a descriptor as OAuth-capable. mintsKey => the flow returns
|
||||
// an API key (OpenRouter); deviceFlow => RFC 8628 device code is also supported.
|
||||
func oauthProvider(descriptor Descriptor, mintsKey bool, deviceFlow bool) Descriptor {
|
||||
@@ -382,6 +425,9 @@ func cloneDescriptor(descriptor Descriptor) Descriptor {
|
||||
descriptor.AuthEnvVars = append([]string{}, descriptor.AuthEnvVars...)
|
||||
descriptor.SupportedAPIFormats = append([]APIFormat{}, descriptor.SupportedAPIFormats...)
|
||||
descriptor.Aliases = append([]string{}, descriptor.Aliases...)
|
||||
if descriptor.LegacyBaseURLs != nil {
|
||||
descriptor.LegacyBaseURLs = append([]string{}, descriptor.LegacyBaseURLs...)
|
||||
}
|
||||
if descriptor.CustomHeaders != nil {
|
||||
descriptor.CustomHeaders = copyStringMap(descriptor.CustomHeaders)
|
||||
}
|
||||
|
||||
@@ -77,7 +77,8 @@ func TestAllHasStableUniqueIDs(t *testing.T) {
|
||||
func TestRecommendedProvidersAreTopOfCatalog(t *testing.T) {
|
||||
descriptors := All()
|
||||
// The recommended providers are badged and pinned to the top of the catalog,
|
||||
// in this order (OpenGateway remains the default; aimlapi.com is also badged).
|
||||
// in this order (Twigpine OpenGateway remains the default; aimlapi.com is also
|
||||
// badged).
|
||||
wantTop := []string{"gitlawb-opengateway", "aimlapi"}
|
||||
if len(descriptors) < len(wantTop) {
|
||||
t.Fatalf("All() returned %d descriptors, want at least %d", len(descriptors), len(wantTop))
|
||||
@@ -107,12 +108,66 @@ func TestRecommendedProviderEndpoint(t *testing.T) {
|
||||
if !ok {
|
||||
t.Fatal("gitlawb-opengateway not found in catalog")
|
||||
}
|
||||
if descriptor.DefaultBaseURL != "https://opengateway.gitlawb.com/v1" {
|
||||
t.Fatalf("OpenGateway base URL = %q, want %q", descriptor.DefaultBaseURL, "https://opengateway.gitlawb.com/v1")
|
||||
if descriptor.Name != "Twigpine OpenGateway" {
|
||||
t.Fatalf("OpenGateway name = %q, want %q", descriptor.Name, "Twigpine OpenGateway")
|
||||
}
|
||||
if descriptor.DefaultBaseURL != "https://api.twigpine.com/v1" {
|
||||
t.Fatalf("OpenGateway base URL = %q, want %q", descriptor.DefaultBaseURL, "https://api.twigpine.com/v1")
|
||||
}
|
||||
// The default must be an exact id from the live /v1/models list (not a
|
||||
// gateway alias such as the bare "mimo-v2.6-pro"): onboarding keeps the
|
||||
// default selected after live discovery only on an exact id match.
|
||||
if descriptor.DefaultModel != "xiaomi/mimo-v2.6-pro" {
|
||||
t.Fatalf("OpenGateway default model = %q, want %q", descriptor.DefaultModel, "xiaomi/mimo-v2.6-pro")
|
||||
}
|
||||
if descriptor.Transport != TransportOpenAICompatible {
|
||||
t.Fatalf("OpenGateway transport = %q, want %q", descriptor.Transport, TransportOpenAICompatible)
|
||||
}
|
||||
// Saved configs keep the original credential env var.
|
||||
if !reflect.DeepEqual(descriptor.AuthEnvVars, []string{"GITLAWB_OPENGATEWAY_API_KEY"}) {
|
||||
t.Fatalf("OpenGateway auth env vars = %#v, want GITLAWB_OPENGATEWAY_API_KEY", descriptor.AuthEnvVars)
|
||||
}
|
||||
// Old and new names both resolve to the same stable catalog id.
|
||||
for _, name := range []string{"gitlawb-opengateway", "GitLawb OpenGateway", "opengateway", "Twigpine OpenGateway", "twigpine"} {
|
||||
got, ok := Get(name)
|
||||
if !ok || got.ID != "gitlawb-opengateway" {
|
||||
t.Fatalf("Get(%q) = (%q, %v), want gitlawb-opengateway", name, got.ID, ok)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsCatalogBaseURLAcceptsDefaultAndLegacyHosts(t *testing.T) {
|
||||
gateway, ok := Get("gitlawb-opengateway")
|
||||
if !ok {
|
||||
t.Fatal("gitlawb-opengateway not found in catalog")
|
||||
}
|
||||
for _, baseURL := range []string{
|
||||
"https://api.twigpine.com/v1",
|
||||
"https://API.twigpine.com/v1/ ",
|
||||
"https://opengateway.gitlawb.com/v1",
|
||||
"https://opengateway.gitlawb.com/v1/",
|
||||
} {
|
||||
if !IsCatalogBaseURL(gateway, baseURL) {
|
||||
t.Fatalf("IsCatalogBaseURL(gateway, %q) = false, want true", baseURL)
|
||||
}
|
||||
}
|
||||
for _, baseURL := range []string{"", "https://gateway.example.com/v1", "https://opengateway.gitlawb.com/v2"} {
|
||||
if IsCatalogBaseURL(gateway, baseURL) {
|
||||
t.Fatalf("IsCatalogBaseURL(gateway, %q) = true, want false", baseURL)
|
||||
}
|
||||
}
|
||||
openrouter, ok := Get("openrouter")
|
||||
if !ok {
|
||||
t.Fatal("openrouter not found in catalog")
|
||||
}
|
||||
if !IsCatalogBaseURL(openrouter, "https://openrouter.ai/api/v1") || IsCatalogBaseURL(openrouter, "https://opengateway.gitlawb.com/v1") {
|
||||
t.Fatal("legacy hosts must only match their own descriptor")
|
||||
}
|
||||
// Get returns a copy: mutating it must not leak into the catalog.
|
||||
gateway.LegacyBaseURLs[0] = "https://mutated.example.com/v1"
|
||||
if again, _ := Get("gitlawb-opengateway"); again.LegacyBaseURLs[0] != "https://opengateway.gitlawb.com/v1" {
|
||||
t.Fatalf("LegacyBaseURLs aliased the catalog: %#v", again.LegacyBaseURLs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIDescriptor(t *testing.T) {
|
||||
@@ -385,6 +440,8 @@ func TestLookupNormalizesIDsAndAliases(t *testing.T) {
|
||||
"custom_openai_compatible": "custom-openai-compatible",
|
||||
"custom--anthropic compatible": "custom-anthropic-compatible",
|
||||
"GitLawb OpenGateway": "gitlawb-opengateway",
|
||||
"Twigpine OpenGateway": "gitlawb-opengateway",
|
||||
"twigpine": "gitlawb-opengateway",
|
||||
}
|
||||
for input, want := range cases {
|
||||
descriptor, ok := Get(input)
|
||||
|
||||
@@ -675,10 +675,12 @@ func blockedAddrReason(addr netip.Addr) string {
|
||||
}
|
||||
|
||||
// overrideHealthEndpoint returns a provider-specific connectivity probe URL when
|
||||
// the default {baseURL}+/models path does not exist. GitLawb OpenGateway is a
|
||||
// smart-routing gateway whose flat /v1/models endpoint 404s by design ("Use
|
||||
// /v1/<provider>/<path>"), so probe its public /health endpoint at the host root
|
||||
// instead, which reports real reachability without a per-model call.
|
||||
// the default {baseURL}+/models path is not the right reachability check.
|
||||
// Twigpine OpenGateway (formerly GitLawb OpenGateway) is a smart-routing gateway
|
||||
// whose flat /v1/models 404'd on older deployments, so probe its public /health
|
||||
// endpoint at the host root instead (served by api.twigpine.com and the legacy
|
||||
// opengateway.gitlawb.com host), which reports real reachability without a
|
||||
// per-model call.
|
||||
func overrideHealthEndpoint(profile config.ProviderProfile, baseURL string) (string, bool) {
|
||||
if profile.CatalogID != "gitlawb-opengateway" {
|
||||
return "", false
|
||||
|
||||
@@ -179,6 +179,10 @@ func TestOverrideHealthEndpointOpenGateway(t *testing.T) {
|
||||
if got != "https://opengateway.gitlawb.com/health" {
|
||||
t.Fatalf("OpenGateway health endpoint = %q, want host-root /health", got)
|
||||
}
|
||||
got, ok = overrideHealthEndpoint(config.ProviderProfile{CatalogID: "gitlawb-opengateway"}, "https://api.twigpine.com/v1")
|
||||
if !ok || got != "https://api.twigpine.com/health" {
|
||||
t.Fatalf("Twigpine OpenGateway health endpoint = (%q, %v), want https://api.twigpine.com/health", got, ok)
|
||||
}
|
||||
|
||||
if _, ok := overrideHealthEndpoint(config.ProviderProfile{CatalogID: "openai"}, "https://api.openai.com/v1"); ok {
|
||||
t.Fatal("non-OpenGateway providers must not override the health endpoint")
|
||||
|
||||
@@ -225,26 +225,17 @@ var curatedModels = map[string][]Model{
|
||||
},
|
||||
"zai": zaiCuratedModels,
|
||||
"zai-cn": zaiCuratedModels,
|
||||
// OpenGateway smart-routes by model id across its upstream providers
|
||||
// (see /health: xiaomi-mimo, minimax, qwen, google, nvidia, tencent, z-ai).
|
||||
// These are the curated coding defaults; the gateway accepts any model its
|
||||
// upstreams expose, so users can also type an id the picker doesn't list.
|
||||
// Twigpine OpenGateway publishes its live, active-only model list at
|
||||
// GET /v1/models; pickers and onboarding show that list whenever it is
|
||||
// reachable (see providermodeldiscovery.DiscoverCatalog). This is only the
|
||||
// offline fallback, so keep it to ids the gateway is expected to keep
|
||||
// serving: the smart router, the catalog default, and its fast sibling. Do
|
||||
// not add promo or time-limited models here. Users can still type any id
|
||||
// the gateway accepts.
|
||||
"gitlawb-opengateway": {
|
||||
{ID: "mimo-v2.5-pro", Description: "catalog default (Xiaomi MiMo)"},
|
||||
{ID: "xiaomi/mimo-v2.5-pro", Description: "Xiaomi MiMo V2.5 Pro"},
|
||||
{ID: "mimo-v2.5-pro-ultraspeed", Description: "fast model (Xiaomi MiMo)"},
|
||||
{ID: "xiaomi/mimo-v2.5", Description: "multimodal model (Xiaomi)"},
|
||||
{ID: "tencent/hy3", Description: "free Tencent HY3 model"},
|
||||
{ID: "MiniMax-M3", Description: "MiniMax model"},
|
||||
{ID: "minimax/minimax-m3", Description: "MiniMax M3 model"},
|
||||
{ID: "qwen-plus", Description: "Qwen model"},
|
||||
{ID: "qwen/qwen3.7-max", Description: "Qwen flagship coding model"},
|
||||
{ID: "gemini-2.5-pro", Description: "long-context model (Google)"},
|
||||
{ID: "google/gemini-3.1-flash-lite", Description: "Google Gemini 3.1 Flash Lite"},
|
||||
{ID: "glm-4.6", Description: "Z.ai model"},
|
||||
{ID: "z-ai/glm-5.2", Description: "GLM coding & reasoning model"},
|
||||
{ID: "nvidia/llama-3.1-nemotron-70b-instruct", Description: "NVIDIA NIM model"},
|
||||
{ID: "nvidia/nemotron-3-ultra-550b-a55b:free", Description: "free Nemotron 3 Ultra reasoning MoE"},
|
||||
{ID: "xiaomi/mimo-v2.6-pro", Description: "catalog default", ContextWindow: 262_144},
|
||||
{ID: "auto", Description: "smart routing (picks a capable model per request)"},
|
||||
{ID: "xiaomi/mimo-v2.6-flash", Description: "fast model (Xiaomi MiMo V2.6 Flash)", ContextWindow: 262_144},
|
||||
},
|
||||
"atomic-chat": {
|
||||
{ID: "gpt-4.1", Description: "catalog default"},
|
||||
|
||||
@@ -74,13 +74,15 @@ func TestModelsAreProviderScoped(t *testing.T) {
|
||||
notWant: []string{"gpt-4.1", "claude-sonnet-4.5"},
|
||||
},
|
||||
{
|
||||
// Offline fallback only; the live /v1/models list is shown when reachable.
|
||||
provider: "gitlawb-opengateway",
|
||||
want: []string{
|
||||
"mimo-v2.5-pro", "tencent/hy3",
|
||||
"xiaomi/mimo-v2.5-pro", "xiaomi/mimo-v2.5", "minimax/minimax-m3", "qwen/qwen3.7-max",
|
||||
"google/gemini-3.1-flash-lite", "z-ai/glm-5.2", "nvidia/nemotron-3-ultra-550b-a55b:free",
|
||||
want: []string{"xiaomi/mimo-v2.6-pro", "auto", "xiaomi/mimo-v2.6-flash"},
|
||||
notWant: []string{
|
||||
"openai/gpt-4.1", "claude-sonnet-4.5",
|
||||
"mimo-v2.5-pro", "xiaomi/mimo-v2.5-pro", "qwen-plus", "glm-4.6", "gemini-2.5-pro",
|
||||
"MiniMax-M3", "nvidia/llama-3.1-nemotron-70b-instruct",
|
||||
"qwen/qwen3.8-max", "qwen/qwen3.8-flash",
|
||||
},
|
||||
notWant: []string{"openai/gpt-4.1", "claude-sonnet-4.5"},
|
||||
},
|
||||
{
|
||||
provider: "opencode-go-anthropic-compatible",
|
||||
|
||||
@@ -54,9 +54,22 @@ func TestDefaultedOpenGatewayURL(t *testing.T) {
|
||||
if got := defaultedOpenGatewayURL(providercatalog.Descriptor{DefaultBaseURL: "https://gw.example.com/v1"}, ""); got != "https://gw.example.com/v1/models" {
|
||||
t.Fatalf("derived = %q", got)
|
||||
}
|
||||
if got := defaultedOpenGatewayURL(providercatalog.Descriptor{DefaultBaseURL: "::not a url"}, ""); got != "https://opengateway.gitlawb.com/v1/models" {
|
||||
if got := defaultedOpenGatewayURL(providercatalog.Descriptor{DefaultBaseURL: "::not a url"}, ""); got != "https://api.twigpine.com/v1/models" {
|
||||
t.Fatalf("fallback = %q", got)
|
||||
}
|
||||
// The catalog descriptor points at the Twigpine API host.
|
||||
descriptor, ok := providercatalog.Get("gitlawb-opengateway")
|
||||
if !ok {
|
||||
t.Fatal("gitlawb-opengateway missing from catalog")
|
||||
}
|
||||
if got := defaultedOpenGatewayURL(descriptor, ""); got != "https://api.twigpine.com/v1/models" {
|
||||
t.Fatalf("catalog descriptor = %q, want https://api.twigpine.com/v1/models", got)
|
||||
}
|
||||
// A descriptor still carrying the legacy host keeps deriving from it, so
|
||||
// old-host configs resolve unchanged (the old host still serves clients).
|
||||
if got := defaultedOpenGatewayURL(providercatalog.Descriptor{DefaultBaseURL: "https://opengateway.gitlawb.com/v1"}, ""); got != "https://opengateway.gitlawb.com/v1/models" {
|
||||
t.Fatalf("legacy host = %q, want https://opengateway.gitlawb.com/v1/models", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDefaultedOpenRouterURL(t *testing.T) {
|
||||
|
||||
@@ -19,7 +19,7 @@ import (
|
||||
|
||||
const (
|
||||
DefaultModelsDevURL = "https://models.dev/api.json"
|
||||
defaultOpenGatewayURL = "https://opengateway.gitlawb.com/v1/models"
|
||||
defaultOpenGatewayURL = "https://api.twigpine.com/v1/models"
|
||||
defaultOpenRouterURL = "https://openrouter.ai/api/v1/models"
|
||||
modelsDevSource = "models.dev"
|
||||
openGatewaySource = "opengateway"
|
||||
@@ -120,6 +120,8 @@ func ParseModelsDevProvider(body []byte, providerID string) ([]Model, error) {
|
||||
// ParseOpenGatewayCatalog parses OpenGateway's live GET /v1/models payload (or the
|
||||
// older {models:[...]} shape). The gateway already curates what it exposes, so
|
||||
// every non-empty id is accepted rather than applying the coding-model heuristic.
|
||||
// Retired entries (status "retired", listed by ?status=all) are skipped so a
|
||||
// picker never offers a model the gateway no longer serves.
|
||||
func ParseOpenGatewayCatalog(body []byte) ([]Model, error) {
|
||||
items, err := parseOpenAIStyleModelList(body)
|
||||
if err != nil {
|
||||
@@ -127,6 +129,9 @@ func ParseOpenGatewayCatalog(body []byte) ([]Model, error) {
|
||||
}
|
||||
models := make([]Model, 0, len(items))
|
||||
for _, item := range items {
|
||||
if item.retired() {
|
||||
continue
|
||||
}
|
||||
model := item.toModel("", openGatewaySource)
|
||||
if model.ID == "" {
|
||||
continue
|
||||
@@ -224,6 +229,7 @@ func PublicLiveCatalog(providerID string) bool {
|
||||
type remoteModel struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Status string `json:"status"` // OpenGateway: "active" | "retired"; absent means active
|
||||
Description string `json:"description"`
|
||||
ContextWindow int `json:"context_window"`
|
||||
ContextWindowCamel int `json:"contextWindow"`
|
||||
@@ -348,6 +354,11 @@ func (model remoteModel) toModel(key string, source string) Model {
|
||||
}
|
||||
}
|
||||
|
||||
// retired reports whether the catalog marks this entry as retired.
|
||||
func (model remoteModel) retired() bool {
|
||||
return strings.EqualFold(strings.TrimSpace(model.Status), "retired")
|
||||
}
|
||||
|
||||
func normalizedServiceTiers(tiers []string, legacy []string) []string {
|
||||
values := append(append([]string{}, tiers...), legacy...)
|
||||
seen := map[string]bool{}
|
||||
|
||||
@@ -67,6 +67,58 @@ func TestParseModelsDevProviderScopesAndMapsMetadata(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseOpenGatewayCatalogSkipsRetiredModels(t *testing.T) {
|
||||
// Mirrors GET https://api.twigpine.com/v1/models?status=all once the gateway
|
||||
// retires models: an active entry plus a retired one carrying a successor.
|
||||
body := []byte(`{
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "xiaomi/mimo-v2.6-pro",
|
||||
"object": "model",
|
||||
"name": "MiMo V2.6-Pro",
|
||||
"description": "flagship omnimodal agent model",
|
||||
"aliases": ["xiaomi/mimo-v2.5-pro", "mimo-v2.5-pro"],
|
||||
"context_window": 262144,
|
||||
"pricing": {"prompt": "0.000001", "completion": "0.000003"},
|
||||
"supports_tools": true,
|
||||
"status": "active"
|
||||
},
|
||||
{
|
||||
"id": "qwen/qwen-plus",
|
||||
"object": "model",
|
||||
"name": "Qwen Plus",
|
||||
"description": "retired general model",
|
||||
"context_window": 131072,
|
||||
"status": "retired",
|
||||
"successor": "qwen/qwen3.7-max"
|
||||
}
|
||||
]
|
||||
}`)
|
||||
|
||||
models, err := ParseOpenGatewayCatalog(body)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseOpenGatewayCatalog returned error: %v", err)
|
||||
}
|
||||
if got := strings.Join(modelIDs(models), ","); got != "xiaomi/mimo-v2.6-pro" {
|
||||
t.Fatalf("models = %q, want only the active entry", got)
|
||||
}
|
||||
descriptor, ok := providercatalog.Get("gitlawb-opengateway")
|
||||
if !ok {
|
||||
t.Fatal("gitlawb-opengateway missing from catalog")
|
||||
}
|
||||
// Onboarding keeps the default selected only on an exact id match against
|
||||
// the live list, so the default must be a real active id, not an alias.
|
||||
if !containsModelID(models, descriptor.DefaultModel) {
|
||||
t.Fatalf("default model %q missing from live list %v", descriptor.DefaultModel, modelIDs(models))
|
||||
}
|
||||
|
||||
// A list made only of retired entries is treated as empty, not offered.
|
||||
if _, err := ParseOpenGatewayCatalog([]byte(`{"data":[{"id":"old/model","status":"retired"}]}`)); err == nil {
|
||||
t.Fatal("expected an error when every entry is retired")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseOpenGatewayCatalogSupportsRichModelJSON(t *testing.T) {
|
||||
body := []byte(`{
|
||||
"models": [
|
||||
|
||||
@@ -355,6 +355,7 @@ type modelsResponseItem struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
Visibility string `json:"visibility"`
|
||||
Status string `json:"status"`
|
||||
ContextWindow int `json:"context_window"`
|
||||
ContextWindowAlt int `json:"contextWindow"`
|
||||
ContextLength int `json:"context_length"`
|
||||
@@ -412,6 +413,11 @@ func parseModelsResponse(body []byte) ([]Model, error) {
|
||||
if visibility := strings.ToLower(strings.TrimSpace(item.Visibility)); visibility != "" && visibility != "list" {
|
||||
continue
|
||||
}
|
||||
// OpenGateway marks lifecycle with status "active" | "retired" (retired
|
||||
// entries only appear with ?status=all). Never offer a retired model.
|
||||
if strings.EqualFold(strings.TrimSpace(item.Status), "retired") {
|
||||
continue
|
||||
}
|
||||
seen[id] = true
|
||||
description := firstNonEmptyDiscovery(
|
||||
item.DisplayName,
|
||||
|
||||
@@ -346,6 +346,70 @@ func TestDiscoverCatalogOpenGatewayUsesLiveListWithoutKey(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverCatalogOpenGatewayDropsRetiredAndKeepsDefault(t *testing.T) {
|
||||
// Both the catalog fetch and the live /v1/models probe return one active and
|
||||
// one retired entry. Users must only ever see the active one, and the
|
||||
// catalog's default model must be among the ids discovery returns (onboarding
|
||||
// keeps the default selected only on an exact id match).
|
||||
const body = `{"object":"list","data":[
|
||||
{"id":"xiaomi/mimo-v2.6-pro","name":"MiMo V2.6-Pro","context_window":262144,"supports_tools":true,"status":"active"},
|
||||
{"id":"qwen/qwen-plus","name":"Qwen Plus","context_window":131072,"status":"retired","successor":"qwen/qwen3.7-max"}
|
||||
]}`
|
||||
var paths []string
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
paths = append(paths, r.URL.Path)
|
||||
switch r.URL.Path {
|
||||
case "/catalog/v1/models", "/v1/models":
|
||||
_, _ = w.Write([]byte(body))
|
||||
default:
|
||||
t.Fatalf("unexpected path %q", r.URL.Path)
|
||||
}
|
||||
}))
|
||||
defer server.Close()
|
||||
|
||||
descriptor, ok := providercatalog.Get("gitlawb-opengateway")
|
||||
if !ok {
|
||||
t.Fatal("gitlawb-opengateway missing from catalog")
|
||||
}
|
||||
provider := descriptor
|
||||
provider.DefaultBaseURL = server.URL + "/v1"
|
||||
models, err := DiscoverCatalog(context.Background(), provider, config.ProviderProfile{
|
||||
CatalogID: "gitlawb-opengateway",
|
||||
ProviderKind: config.ProviderKindOpenAICompatible,
|
||||
BaseURL: server.URL + "/v1",
|
||||
Model: descriptor.DefaultModel,
|
||||
}, Options{
|
||||
HTTPClient: server.Client(),
|
||||
OpenGatewayURL: server.URL + "/catalog/v1/models",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("DiscoverCatalog: %v", err)
|
||||
}
|
||||
if got := strings.Join(modelIDs(models), ","); got != "xiaomi/mimo-v2.6-pro" {
|
||||
t.Fatalf("models = %q, want only the active entry", got)
|
||||
}
|
||||
if models[0].ID != descriptor.DefaultModel {
|
||||
t.Fatalf("default model %q missing from discovered %v", descriptor.DefaultModel, modelIDs(models))
|
||||
}
|
||||
if strings.Join(paths, ",") != "/catalog/v1/models,/v1/models" {
|
||||
t.Fatalf("requests = %v, want catalog fetch then live probe", paths)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseModelsResponseSkipsRetiredStatus(t *testing.T) {
|
||||
models, err := parseModelsResponse([]byte(`{"data":[
|
||||
{"id":"active-model","status":"active"},
|
||||
{"id":"retired-model","status":"Retired"},
|
||||
{"id":"no-status-model"}
|
||||
]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("parseModelsResponse: %v", err)
|
||||
}
|
||||
if got := strings.Join(modelIDs(models), ","); got != "active-model,no-status-model" {
|
||||
t.Fatalf("models = %q, want retired entry dropped", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiscoverCatalogOpenRouterKeepsLiveOnlyModels(t *testing.T) {
|
||||
// Catalog omits anthropic/claude-sonnet-4.5 and the generic tools-only id;
|
||||
// live retains both so preferLive keeps coding-capable live-only entries
|
||||
|
||||
@@ -160,7 +160,7 @@ func TestNewPassesOpenGatewayHY3ModelThrough(t *testing.T) {
|
||||
for range stream {
|
||||
}
|
||||
|
||||
if transport.request.URL.String() != "https://opengateway.gitlawb.com/v1/chat/completions" {
|
||||
if transport.request.URL.String() != "https://api.twigpine.com/v1/chat/completions" {
|
||||
t.Fatalf("request URL = %q, want OpenGateway chat completions", transport.request.URL.String())
|
||||
}
|
||||
var body map[string]any
|
||||
|
||||
@@ -298,26 +298,32 @@ func (provider *Provider) emitPayload(ctx context.Context, data string, state *t
|
||||
if chunk.Error != nil {
|
||||
state.flushContent(ctx, events)
|
||||
state.closeOpen(ctx, events)
|
||||
// statusCode buckets the error for classification only. The HTTP response
|
||||
// was 200, so observedStatus stays 0 unless the payload's code maps to a
|
||||
// known status; a bucket default must not be persisted as a real status.
|
||||
statusCode := http.StatusInternalServerError
|
||||
observedStatus := 0
|
||||
if chunk.Error.Code != nil {
|
||||
switch c := chunk.Error.Code.(type) {
|
||||
case string:
|
||||
if code, ok := openAIStreamErrorStatusByCode[c]; ok {
|
||||
statusCode = code
|
||||
statusCode, observedStatus = code, code
|
||||
}
|
||||
case float64:
|
||||
if code, ok := openAIStreamErrorStatusByCode[strconv.Itoa(int(c))]; ok {
|
||||
statusCode = code
|
||||
statusCode, observedStatus = code, code
|
||||
}
|
||||
case int:
|
||||
if code, ok := openAIStreamErrorStatusByCode[strconv.Itoa(c)]; ok {
|
||||
statusCode = code
|
||||
statusCode, observedStatus = code, code
|
||||
}
|
||||
}
|
||||
}
|
||||
sendEvent(ctx, events, zeroruntime.StreamEvent{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: provider.classifiedError(statusCode, chunk.Error.Message),
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: provider.classifiedError(statusCode, chunk.Error.Message),
|
||||
StatusCode: observedStatus,
|
||||
Cause: provider.redact(chunk.Error.Message),
|
||||
})
|
||||
state.done = true
|
||||
return false
|
||||
@@ -404,12 +410,19 @@ func (provider *Provider) emitHTTPError(ctx context.Context, response *http.Resp
|
||||
// localhost but returns a gateway error when it cannot reach its own backend.
|
||||
// Surface that as a clear connectivity message instead of the raw proxied body.
|
||||
if humanized, ok := providerio.UpstreamUnreachable(message); ok {
|
||||
sendEvent(ctx, events, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventError, Error: provider.redact(humanized)})
|
||||
sendEvent(ctx, events, zeroruntime.StreamEvent{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: provider.redact(humanized),
|
||||
StatusCode: response.StatusCode,
|
||||
Cause: provider.redact(message),
|
||||
})
|
||||
return
|
||||
}
|
||||
sendEvent(ctx, events, zeroruntime.StreamEvent{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: provider.classifiedError(response.StatusCode, message),
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: provider.classifiedError(response.StatusCode, message),
|
||||
StatusCode: response.StatusCode,
|
||||
Cause: provider.redact(message),
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -500,6 +500,35 @@ func TestStreamCompletionClassifiesHTTPErrorsAndRedactsToken(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestStreamCompletionHTTPErrorCarriesStatusCodeAndCause verifies the StreamEvent
|
||||
// for an HTTP-level provider failure carries the raw status code and a redacted
|
||||
// cause separately from the classified, user-facing Error string — the detail
|
||||
// a session's "error" event needs to actually diagnose why a provider call
|
||||
// failed instead of only storing the flattened message (#674).
|
||||
func TestStreamCompletionHTTPErrorCarriesStatusCodeAndCause(t *testing.T) {
|
||||
provider := newTestProviderWithKey(t, "sk-secret", func(w http.ResponseWriter, r *http.Request) {
|
||||
http.Error(w, `{"error":{"message":"upstream saw Bearer sk-secret"}}`, http.StatusInternalServerError)
|
||||
})
|
||||
stream, err := provider.StreamCompletion(context.Background(), zeroruntime.CompletionRequest{})
|
||||
if err != nil {
|
||||
t.Fatalf("StreamCompletion returned setup error: %v", err)
|
||||
}
|
||||
events := readAll(stream)
|
||||
if len(events) != 1 || events[0].Type != zeroruntime.StreamEventError {
|
||||
t.Fatalf("events = %#v, want one error", events)
|
||||
}
|
||||
event := events[0]
|
||||
if event.StatusCode != http.StatusInternalServerError {
|
||||
t.Fatalf("StatusCode = %d, want %d", event.StatusCode, http.StatusInternalServerError)
|
||||
}
|
||||
if !strings.Contains(event.Cause, "upstream saw Bearer") {
|
||||
t.Fatalf("Cause = %q, want it to contain the upstream detail", event.Cause)
|
||||
}
|
||||
if strings.Contains(event.Cause, "sk-secret") {
|
||||
t.Fatalf("Cause leaked token: %q", event.Cause)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStreamCompletionHumanizesUpstreamUnreachableGatewayError(t *testing.T) {
|
||||
t.Parallel()
|
||||
// A local Ollama daemon serving a "-cloud" model answers on localhost but
|
||||
@@ -526,6 +555,15 @@ func TestStreamCompletionHumanizesUpstreamUnreachableGatewayError(t *testing.T)
|
||||
t.Fatalf("error = %q, want it to contain %q", got, want)
|
||||
}
|
||||
}
|
||||
// The humanized "upstream unreachable" message must not drop the response
|
||||
// metadata a session error event needs — StatusCode/Cause must still reflect
|
||||
// the real HTTP response, matching the classified-error path below it (#674).
|
||||
if events[0].StatusCode != http.StatusBadGateway {
|
||||
t.Fatalf("StatusCode = %d, want %d", events[0].StatusCode, http.StatusBadGateway)
|
||||
}
|
||||
if !strings.Contains(events[0].Cause, "TLS handshake timeout") {
|
||||
t.Fatalf("Cause = %q, want it to contain the raw upstream detail", events[0].Cause)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStreamCompletionEmitsStreamErrorObject(t *testing.T) {
|
||||
@@ -584,6 +622,50 @@ func TestStreamCompletionClassifiesStreamErrorCode(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestStreamCompletionStreamedErrorCarriesStatusCodeAndCause mirrors
|
||||
// TestStreamCompletionHTTPErrorCarriesStatusCodeAndCause for the other error
|
||||
// path: an error arriving inside a 200 OK SSE payload's "code" field rather
|
||||
// than the HTTP status (#674).
|
||||
func TestStreamCompletionStreamedErrorCarriesStatusCodeAndCause(t *testing.T) {
|
||||
provider := newTestProviderWithKey(t, "sk-secret", func(w http.ResponseWriter, r *http.Request) {
|
||||
writeSSE(w, `{"error":{"message":"rate limited sk-secret, see Bearer token docs","code":"429"}}`)
|
||||
})
|
||||
|
||||
events := collectProviderEvents(t, provider)
|
||||
if len(events) != 1 || events[0].Type != zeroruntime.StreamEventError {
|
||||
t.Fatalf("events = %#v, want one error", events)
|
||||
}
|
||||
event := events[0]
|
||||
if event.StatusCode != http.StatusTooManyRequests {
|
||||
t.Fatalf("StatusCode = %d, want %d", event.StatusCode, http.StatusTooManyRequests)
|
||||
}
|
||||
if !strings.Contains(event.Cause, "rate limited") {
|
||||
t.Fatalf("Cause = %q, want it to contain the upstream detail", event.Cause)
|
||||
}
|
||||
if strings.Contains(event.Cause, "sk-secret") {
|
||||
t.Fatalf("Cause leaked token: %q", event.Cause)
|
||||
}
|
||||
}
|
||||
|
||||
// An error inside a 200 OK SSE payload whose code is not a known status was
|
||||
// never an observed HTTP status, so none must be reported (#674 review).
|
||||
func TestStreamCompletionStreamedErrorWithUnknownCodeReportsNoStatus(t *testing.T) {
|
||||
provider := newTestProvider(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
writeSSE(w, `{"error":{"message":"model overloaded","code":"server_error"}}`)
|
||||
})
|
||||
|
||||
events := collectProviderEvents(t, provider)
|
||||
if len(events) != 1 || events[0].Type != zeroruntime.StreamEventError {
|
||||
t.Fatalf("events = %#v, want one error", events)
|
||||
}
|
||||
if events[0].StatusCode != 0 {
|
||||
t.Fatalf("StatusCode = %d, want 0 (no HTTP status was observed)", events[0].StatusCode)
|
||||
}
|
||||
if !strings.Contains(events[0].Cause, "model overloaded") {
|
||||
t.Fatalf("Cause = %q, want the upstream detail", events[0].Cause)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStreamCompletionEmitsErrorForMalformedJSON(t *testing.T) {
|
||||
provider := newTestProvider(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
writeSSE(w, `{"choices":`)
|
||||
|
||||
@@ -63,6 +63,14 @@ import (
|
||||
// rest of a key from prose written after a whole one: that prose would be eaten,
|
||||
// and a following JWT would lose its header.
|
||||
//
|
||||
// CALLER-SUPPLIED SECRETS (Options.ExtraSecretValues) get the same treatment
|
||||
// from redactExtraSecretValues. Such a value is often opaque, matching no shape
|
||||
// here, so without its own gap-tolerant match a separator inside it left both
|
||||
// fragments in the output. Unlike a shape, the value is known exactly and has no
|
||||
// word boundary to judge, so it is found in a copy with the separators removed
|
||||
// and mapped back. RedactString runs it first, before any shape matcher, so a
|
||||
// caller that clips or reformats the result afterwards never sees a fragment.
|
||||
//
|
||||
// WHAT THIS DELIBERATELY DOES NOT TREAT AS A SEPARATOR: tab, newline and
|
||||
// carriage return. Those three are real text structure rather than invisible
|
||||
// filler, so a credential broken across a line boundary is still two visible
|
||||
@@ -435,6 +443,125 @@ func openAIShapeIsSecret(match string) bool {
|
||||
return !strings.Contains(strings.TrimPrefix(match, "sk-"), "-")
|
||||
}
|
||||
|
||||
// redactExtraSecretValues replaces every occurrence of every caller-supplied
|
||||
// secret, whole or with runs of invisible separators between its characters.
|
||||
//
|
||||
// All occurrences, overlapping ones included, are found in the same text and
|
||||
// overlapping spans are merged before anything is replaced. Replacing one at a
|
||||
// time let each replacement consume bytes another occurrence needed: a shorter
|
||||
// value inside a split longer one, a periodic value ("abcabc" in
|
||||
// "abc<NUL>abcabc"), a value whose first and last characters match, or two
|
||||
// values that overlap all left a fragment visible. Occurrences that only touch
|
||||
// stay separate replacements, as they always were.
|
||||
//
|
||||
// Separators inside a match are covered by its replacement; a split match never
|
||||
// starts or ends on one, so those beside it stay, unless the value as written
|
||||
// starts or ends with them. Tab, newline and carriage return are not separators
|
||||
// (see above).
|
||||
func redactExtraSecretValues(value string, secrets []string, replacement string) string {
|
||||
haystack, offsets := value, []int(nil)
|
||||
if containsSplitSeparator(value) {
|
||||
haystack, offsets = compactSplitSeparators(value)
|
||||
}
|
||||
var spans [][2]int
|
||||
for _, secret := range secrets {
|
||||
if strings.TrimSpace(secret) == "" {
|
||||
continue
|
||||
}
|
||||
needle := secret
|
||||
if offsets != nil {
|
||||
// The value as written always matches, as it did before split
|
||||
// matching existed. The compacted match cannot stand in for it: a
|
||||
// value made only of separators compacts to nothing, and separators
|
||||
// at either end of a value fall outside its compacted match.
|
||||
if strings.Contains(value, secret) {
|
||||
for _, at := range overlappingIndexes(value, secret) {
|
||||
spans = append(spans, [2]int{at, at + len(secret)})
|
||||
}
|
||||
}
|
||||
needle = stripSplitSeparators(secret)
|
||||
}
|
||||
if needle == "" || !strings.Contains(haystack, needle) {
|
||||
continue
|
||||
}
|
||||
for _, at := range overlappingIndexes(haystack, needle) {
|
||||
start, end := at, at+len(needle)
|
||||
if offsets != nil {
|
||||
start, end = offsets[start], offsets[end-1]+1
|
||||
}
|
||||
spans = append(spans, [2]int{start, end})
|
||||
}
|
||||
}
|
||||
if len(spans) == 0 {
|
||||
return value
|
||||
}
|
||||
sort.Slice(spans, func(i, j int) bool { return spans[i][0] < spans[j][0] })
|
||||
var out strings.Builder
|
||||
out.Grow(len(value))
|
||||
written := 0
|
||||
for i := 0; i < len(spans); {
|
||||
start, end := spans[i][0], spans[i][1]
|
||||
for i++; i < len(spans) && spans[i][0] < end; i++ {
|
||||
end = max(end, spans[i][1])
|
||||
}
|
||||
out.WriteString(value[written:start])
|
||||
out.WriteString(replacement)
|
||||
written = end
|
||||
}
|
||||
out.WriteString(value[written:])
|
||||
return out.String()
|
||||
}
|
||||
|
||||
// compactSplitSeparators returns value with every splitSecretSeparator removed,
|
||||
// and for each byte of the result the offset of that byte in value.
|
||||
func compactSplitSeparators(value string) (string, []int) {
|
||||
var builder strings.Builder
|
||||
builder.Grow(len(value))
|
||||
offsets := make([]int, 0, len(value))
|
||||
for offset := 0; offset < len(value); {
|
||||
r, width := utf8.DecodeRuneInString(value[offset:])
|
||||
if !splitSecretSeparator(r) {
|
||||
builder.WriteString(value[offset : offset+width])
|
||||
for i := 0; i < width; i++ {
|
||||
offsets = append(offsets, offset+i)
|
||||
}
|
||||
}
|
||||
offset += width
|
||||
}
|
||||
return builder.String(), offsets
|
||||
}
|
||||
|
||||
// overlappingIndexes returns the start of every occurrence of needle in
|
||||
// haystack, overlapping ones included, in one linear Knuth-Morris-Pratt pass:
|
||||
// restarting a search one byte after each match is quadratic for a periodic
|
||||
// value over text full of it.
|
||||
func overlappingIndexes(haystack, needle string) []int {
|
||||
fallback := make([]int, len(needle))
|
||||
for i, k := 1, 0; i < len(needle); i++ {
|
||||
for k > 0 && needle[i] != needle[k] {
|
||||
k = fallback[k-1]
|
||||
}
|
||||
if needle[i] == needle[k] {
|
||||
k++
|
||||
}
|
||||
fallback[i] = k
|
||||
}
|
||||
var starts []int
|
||||
for i, k := 0, 0; i < len(haystack); i++ {
|
||||
for k > 0 && haystack[i] != needle[k] {
|
||||
k = fallback[k-1]
|
||||
}
|
||||
if haystack[i] == needle[k] {
|
||||
k++
|
||||
}
|
||||
if k == len(needle) {
|
||||
starts = append(starts, i-len(needle)+1)
|
||||
k = fallback[k-1]
|
||||
}
|
||||
}
|
||||
return starts
|
||||
}
|
||||
|
||||
// redactControlSplitSecrets replaces credentials whose body is interrupted by
|
||||
// invisible characters. It is a no-op for text containing none of them, which is
|
||||
// nearly all text, so the contiguous path keeps its behavior exactly.
|
||||
|
||||
@@ -105,6 +105,114 @@ func TestRedactStringCoversTheSeparatorsInsideASplitSecret(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A CALLER-SUPPLIED SECRET IS A CREDENTIAL TOO.
|
||||
//
|
||||
// ExtraSecretValues names a value the caller knows is secret, whatever its
|
||||
// shape: an opaque provider key matches none of the shape matchers, so the
|
||||
// split pass above never saw it, and a separator inside it left both fragments
|
||||
// in the output. It has to be matched across the same separators, cover them in
|
||||
// its replacement, and leave the separators beside it where they were.
|
||||
func TestRedactStringRedactsExtraSecretValuesSplitByInvisibleCharacters(t *testing.T) {
|
||||
const secret = "zq7Mv2Lp9Xr4Tn8Wk3"
|
||||
options := Options{ExtraSecretValues: []string{secret}}
|
||||
if out := RedactString("key "+secret, Options{}); !strings.Contains(out, secret) {
|
||||
t.Fatalf("SETUP INVALID: a shape matcher already redacts %q, so this would not reach the explicit-value pass: %q", secret, out)
|
||||
}
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
value string
|
||||
want string
|
||||
}{
|
||||
{"NUL", "key " + secret[:6] + "\x00" + secret[6:], "key [REDACTED]"},
|
||||
{"zero width space", "key " + secret[:1] + "\u200b" + secret[1:], "key [REDACTED]"},
|
||||
{"soft hyphen near the end", "key " + secret[:len(secret)-1] + "\u00ad" + secret[len(secret)-1:], "key [REDACTED]"},
|
||||
{"every gap", "key " + strings.Join(strings.Split(secret, ""), "\x1b"), "key [REDACTED]"},
|
||||
{"adjacent separators", "key " + secret[:9] + "\x00\x1b\u2060" + secret[9:], "key [REDACTED]"},
|
||||
{"twice", secret[:3] + "\x00" + secret[3:] + " and " + secret[:12] + "\ufeff" + secret[12:], "[REDACTED] and [REDACTED]"},
|
||||
{"separators beside it stay", "\x00" + secret[:5] + "\u200b" + secret[5:] + "\x00", "\x00[REDACTED]\x00"},
|
||||
{"a line break is not a gap", secret[:5] + "\n" + secret[5:], secret[:5] + "\n" + secret[5:]},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
if got := RedactString(testCase.value, options); got != testCase.want {
|
||||
t.Errorf("RedactString(%q) = %q, want %q", testCase.value, got, testCase.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
// A SHORTER SECRET MUST NOT CLAIM PART OF A SPLIT LONGER ONE. Longest first
|
||||
// has to hold across the contiguous and split matches together: replacing
|
||||
// a contiguous shorter value first left the rest of the split longer one
|
||||
// ("[REDACTED]\x00Lp9Xr4Tn8Wk3") visible.
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
secrets []string
|
||||
value string
|
||||
want string
|
||||
}{
|
||||
{"prefix", []string{secret[:6], secret}, "key " + secret[:6] + "\x00" + secret[6:], "key [REDACTED]"},
|
||||
{"suffix", []string{secret[9:], secret}, "key " + secret[:9] + "\u200b" + secret[9:], "key [REDACTED]"},
|
||||
{"inside", []string{secret[4:10], secret}, "key " + secret[:2] + "\x00" + secret[2:], "key [REDACTED]"},
|
||||
{"shorter alone too", []string{secret, secret[:6]}, secret[:6] + " and " + secret[:6] + "\x00" + secret[6:], "[REDACTED] and [REDACTED]"},
|
||||
} {
|
||||
t.Run("overlap "+testCase.name, func(t *testing.T) {
|
||||
if got := RedactString(testCase.value, Options{ExtraSecretValues: testCase.secrets}); got != testCase.want {
|
||||
t.Errorf("RedactString(%q, %q) = %q, want %q", testCase.value, testCase.secrets, got, testCase.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
// OVERLAPPING OCCURRENCES ARE ONE SPAN. Replacing one occurrence (either
|
||||
// form) and then looking for the next consumed bytes the other needed:
|
||||
// "abc\x00abcabc" holds a split "abc\x00abc" overlapping a whole "abcabc",
|
||||
// and replacing the whole one first left "abc\x00" in front of it. Every
|
||||
// occurrence of every value, in either form, has to be found in the same
|
||||
// text and overlapping ones replaced together.
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
secrets []string
|
||||
value string
|
||||
want string
|
||||
}{
|
||||
{"split then whole, periodic secret", []string{"abcabc"}, "abc\x00abcabc", "[REDACTED]"},
|
||||
{"whole then split, periodic secret", []string{"abcabc"}, "abcabc\x00abc", "[REDACTED]"},
|
||||
{"shared first and last character", []string{"xY7qx"}, "key xY7q\u200bxY7qx", "key [REDACTED]"},
|
||||
{"two secrets overlapping across a split", []string{"zq7Mv2Lp", "Lp9Xr4Tn"}, "key zq7Mv2L\x00p9Xr4Tn", "key [REDACTED]"},
|
||||
{"whole occurrences overlapping", []string{"xY7qx"}, "xY7qxY7qx", "[REDACTED]"},
|
||||
{"adjacent occurrences stay two", []string{"abc"}, "abc\x00abc", "[REDACTED]\x00[REDACTED]"},
|
||||
} {
|
||||
t.Run("overlapping occurrences "+testCase.name, func(t *testing.T) {
|
||||
if got := RedactString(testCase.value, Options{ExtraSecretValues: testCase.secrets}); got != testCase.want {
|
||||
t.Errorf("RedactString(%q, %q) = %q, want %q", testCase.value, testCase.secrets, got, testCase.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
// A SECRET AS WRITTEN IS ALWAYS REDACTED. The split match compares the text
|
||||
// and the value with separators removed; it adds to the whole match and
|
||||
// must never replace it. Once the text held any separator, a value made only
|
||||
// of separators compacted to nothing and was skipped, and separators at
|
||||
// either end of a value fell outside its compacted match, so both stayed
|
||||
// visible where a plain replacement had always caught them.
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
secrets []string
|
||||
value string
|
||||
want string
|
||||
}{
|
||||
{"separator-only secret", []string{"\x00"}, "a\x00b", "a[REDACTED]b"},
|
||||
{"separator-only secret beside another separator", []string{"\u200b"}, "a\u200bb\x00", "a[REDACTED]b\x00"},
|
||||
{"separators at both ends", []string{"\u200babc\x00"}, "x\u200babc\x00y", "x[REDACTED]y"},
|
||||
} {
|
||||
t.Run("as written "+testCase.name, func(t *testing.T) {
|
||||
if got := RedactString(testCase.value, Options{ExtraSecretValues: testCase.secrets}); got != testCase.want {
|
||||
t.Errorf("RedactString(%q, %q) = %q, want %q", testCase.value, testCase.secrets, got, testCase.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
// A value the caller names that is NOT in the text leaves the text alone,
|
||||
// separators and all.
|
||||
if value := "unrelated\x00text \u200bhere"; RedactString(value, options) != value {
|
||||
t.Errorf("RedactString rewrote text without the secret in it: %q", RedactString(value, options))
|
||||
}
|
||||
}
|
||||
|
||||
// NOTHING ELSE MOVES. The compaction exists to find a match; it must never
|
||||
// reach text that has no secret in it, and text with no invisible character at
|
||||
// all must take exactly the path it took before.
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
package redaction
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestRedactSecretValuesRemovesOnlyTheKnownValues(t *testing.T) {
|
||||
const key = "aiml-0123456789abcdef0123456789abcdef"
|
||||
nul := string(rune(0))
|
||||
zeroWidthSpace := string(rune(0x200b))
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
value string
|
||||
want string
|
||||
// shaped marks text RedactString would rewrite by shape alone, so the
|
||||
// case shows that RedactSecretValues leaves shapes to RedactString.
|
||||
shaped bool
|
||||
}{
|
||||
{name: "whole", value: "invalid key " + key + " for this account", want: "invalid key " + RedactedSecret + " for this account"},
|
||||
{name: "bearer form", value: "got Bearer " + key, want: "got Bearer " + RedactedSecret},
|
||||
{name: "split by NUL", value: "invalid key " + key[:9] + nul + key[9:], want: "invalid key " + RedactedSecret},
|
||||
{name: "split by a zero-width space", value: "value " + key[:20] + zeroWidthSpace + key[20:] + ".", want: "value " + RedactedSecret + "."},
|
||||
{name: "password assignment", value: "password=hunter2", want: "password=hunter2", shaped: true},
|
||||
{name: "another bearer token", value: "Authorization: Bearer some-other-token", want: "Authorization: Bearer some-other-token", shaped: true},
|
||||
{name: "openai shaped key", value: "use sk-proj-abcdefghijklmnopqrstuvwxyz0123456789", want: "use sk-proj-abcdefghijklmnopqrstuvwxyz0123456789", shaped: true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := RedactSecretValues(tc.value, key); got != tc.want {
|
||||
t.Fatalf("RedactSecretValues(%q) = %q, want %q", tc.value, got, tc.want)
|
||||
}
|
||||
if tc.shaped && RedactString(tc.value, Options{}) == tc.value {
|
||||
t.Fatalf("RedactString(%q) left it unchanged, so this case does not show the shape patterns are off", tc.value)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
for _, secrets := range [][]string{nil, {""}, {" "}} {
|
||||
if got := RedactSecretValues("token "+key, secrets...); got != "token "+key {
|
||||
t.Fatalf("RedactSecretValues with secrets %q = %q, want the text unchanged", secrets, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -463,15 +463,7 @@ func RedactString(value string, options Options) string {
|
||||
replacement := replacement(options)
|
||||
redacted := value
|
||||
if len(options.ExtraSecretValues) > 0 {
|
||||
secrets := append([]string{}, options.ExtraSecretValues...)
|
||||
sort.SliceStable(secrets, func(i, j int) bool {
|
||||
return len(secrets[i]) > len(secrets[j])
|
||||
})
|
||||
for _, secret := range secrets {
|
||||
if strings.TrimSpace(secret) != "" {
|
||||
redacted = strings.ReplaceAll(redacted, secret, replacement)
|
||||
}
|
||||
}
|
||||
redacted = redactExtraSecretValues(redacted, options.ExtraSecretValues, replacement)
|
||||
}
|
||||
|
||||
if strings.Contains(redacted, "-----BEGIN ") && strings.Contains(redacted, "PRIVATE KEY") {
|
||||
@@ -636,6 +628,19 @@ func secretMatchHasDigit(s string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// RedactSecretValues replaces every occurrence of each of secrets in value with
|
||||
// RedactedSecret, whole or split by invisible separators, exactly as
|
||||
// RedactString treats Options.ExtraSecretValues. Unlike RedactString it runs
|
||||
// none of the shape patterns, so text that does not contain one of the values
|
||||
// comes back unchanged. That suits content where only known values must go,
|
||||
// such as tool output, which a shape pattern could otherwise rewrite.
|
||||
func RedactSecretValues(value string, secrets ...string) string {
|
||||
if len(secrets) == 0 {
|
||||
return value
|
||||
}
|
||||
return redactExtraSecretValues(value, secrets, RedactedSecret)
|
||||
}
|
||||
|
||||
func RedactValue(value any, options Options) any {
|
||||
return redactReflect(reflect.ValueOf(value), redactionContext{
|
||||
options: options,
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
package sessions
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// maxErrorCauseBytes bounds the persisted cause. Provider error bodies are read
|
||||
// up to 64 KiB, which is far more than a diagnostic needs and would otherwise be
|
||||
// copied whole into every error event in events.jsonl.
|
||||
const maxErrorCauseBytes = 8 << 10
|
||||
|
||||
const errorCauseTruncationMarker = "… [truncated]"
|
||||
|
||||
// ErrorEventPayload builds the payload for an EventError session event. It
|
||||
// always includes the flattened message (unchanged, for backward
|
||||
// compatibility with existing consumers of events.jsonl), and additionally
|
||||
// persists the HTTP status code and upstream cause when err is (or wraps) a
|
||||
// *zeroruntime.StreamError — the structured error a failed provider stream
|
||||
// returns. Without this, a provider error left no way to diagnose *why* a
|
||||
// call failed from the CLI or its logs — only the generic top-level message
|
||||
// was ever recorded (#674). Cause has already been redacted for secrets by
|
||||
// the provider before reaching here (see zeroruntime.StreamEvent.Cause) — it
|
||||
// is never re-scrubbed or stored raw at this layer. Invalid UTF-8 in the cause
|
||||
// is replaced with U+FFFD, and a longer cause is cut on a rune boundary and
|
||||
// marked; the marked result, marker included, stays within maxErrorCauseBytes
|
||||
// as it reads back from the store.
|
||||
func ErrorEventPayload(err error) map[string]any {
|
||||
payload := map[string]any{"message": err.Error()}
|
||||
var streamErr *zeroruntime.StreamError
|
||||
if errors.As(err, &streamErr) {
|
||||
if streamErr.StatusCode != 0 {
|
||||
payload["statusCode"] = streamErr.StatusCode
|
||||
}
|
||||
if streamErr.Cause != "" {
|
||||
payload["cause"] = boundErrorCause(streamErr.Cause)
|
||||
}
|
||||
}
|
||||
return payload
|
||||
}
|
||||
|
||||
func boundErrorCause(cause string) string {
|
||||
// Made valid first, so the bound applies to what the store keeps. The
|
||||
// payload is written as JSON, which turns every invalid byte into a
|
||||
// three-byte U+FFFD, so a body of invalid bytes cut to the limit here came
|
||||
// back from events.jsonl at up to three times it.
|
||||
cause = strings.ToValidUTF8(cause, "\uFFFD")
|
||||
if len(cause) <= maxErrorCauseBytes {
|
||||
return cause
|
||||
}
|
||||
// The marker counts toward the bound, so the cut leaves room for it.
|
||||
return truncateUTF8(cause, maxErrorCauseBytes-len(errorCauseTruncationMarker)) + errorCauseTruncationMarker
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
package sessions
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestErrorEventPayloadPlainError(t *testing.T) {
|
||||
payload := ErrorEventPayload(errors.New("provider error: provider returned error"))
|
||||
|
||||
if got, want := payload["message"], "provider error: provider returned error"; got != want {
|
||||
t.Fatalf("message = %v, want %v", got, want)
|
||||
}
|
||||
if _, ok := payload["statusCode"]; ok {
|
||||
t.Fatalf("statusCode should be absent for a plain error, got %v", payload["statusCode"])
|
||||
}
|
||||
if _, ok := payload["cause"]; ok {
|
||||
t.Fatalf("cause should be absent for a plain error, got %v", payload["cause"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrorEventPayloadStreamError(t *testing.T) {
|
||||
err := &zeroruntime.StreamError{
|
||||
Message: "provider error: provider returned error",
|
||||
StatusCode: 500,
|
||||
Cause: "upstream returned an empty completion",
|
||||
}
|
||||
|
||||
payload := ErrorEventPayload(err)
|
||||
|
||||
if got, want := payload["message"], "provider error: provider returned error"; got != want {
|
||||
t.Fatalf("message = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := payload["statusCode"], 500; got != want {
|
||||
t.Fatalf("statusCode = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := payload["cause"], "upstream returned an empty completion"; got != want {
|
||||
t.Fatalf("cause = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrorEventPayloadWrappedStreamError(t *testing.T) {
|
||||
// A StreamError wrapped by another layer (e.g. fmt.Errorf("...: %w", err))
|
||||
// must still surface its status/cause via errors.As, not just a direct type
|
||||
// assertion — this is the shape #674's real repro chain produces.
|
||||
inner := &zeroruntime.StreamError{Message: "rate limit error: too many requests", StatusCode: 429, Cause: "retry after 30s"}
|
||||
wrapped := fmt.Errorf("turn failed: %w", inner)
|
||||
|
||||
payload := ErrorEventPayload(wrapped)
|
||||
|
||||
if got, want := payload["statusCode"], 429; got != want {
|
||||
t.Fatalf("statusCode = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := payload["cause"], "retry after 30s"; got != want {
|
||||
t.Fatalf("cause = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := payload["message"], "turn failed: rate limit error: too many requests"; got != want {
|
||||
t.Fatalf("message = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrorEventPayloadZeroStatusCodeOmitted(t *testing.T) {
|
||||
// A transport-level failure (e.g. context cancellation) never got an HTTP
|
||||
// response, so StatusCode is legitimately 0 — that must stay omitted rather
|
||||
// than be persisted as a misleading "statusCode": 0.
|
||||
err := &zeroruntime.StreamError{Message: "context deadline exceeded"}
|
||||
|
||||
payload := ErrorEventPayload(err)
|
||||
|
||||
if _, ok := payload["statusCode"]; ok {
|
||||
t.Fatalf("statusCode should be omitted when zero, got %v", payload["statusCode"])
|
||||
}
|
||||
if _, ok := payload["cause"]; ok {
|
||||
t.Fatalf("cause should be omitted when empty, got %v", payload["cause"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestStoreAppendEventPersistsErrorEventStatusCodeAndCause(t *testing.T) {
|
||||
// Closes the seam between ErrorEventPayload (unit tested above) and Run
|
||||
// returning a *StreamError (tested in internal/agent): this asserts the
|
||||
// two actually meet at the Store, i.e. a *StreamError survives a real
|
||||
// AppendEvent/ReadEvents round trip with statusCode/cause intact.
|
||||
store := NewStore(StoreOptions{RootDir: t.TempDir(), Now: fixedClock("2026-08-10T12:00:00Z")})
|
||||
session, err := store.Create(CreateInput{SessionID: "error_join"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create returned error: %v", err)
|
||||
}
|
||||
|
||||
streamErr := &zeroruntime.StreamError{
|
||||
Message: "provider error: provider returned error",
|
||||
StatusCode: 503,
|
||||
Cause: "upstream returned an empty completion",
|
||||
}
|
||||
|
||||
if _, err := store.AppendEvent(session.SessionID, AppendEventInput{
|
||||
Type: EventError,
|
||||
Payload: ErrorEventPayload(streamErr),
|
||||
}); err != nil {
|
||||
t.Fatalf("AppendEvent returned error: %v", err)
|
||||
}
|
||||
|
||||
events, err := store.ReadEvents(session.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
if len(events) != 1 || events[0].Type != EventError {
|
||||
t.Fatalf("expected a single error event, got %#v", events)
|
||||
}
|
||||
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(events[0].Payload, &payload); err != nil {
|
||||
t.Fatalf("failed to unmarshal persisted payload: %v", err)
|
||||
}
|
||||
if got, want := payload["message"], "provider error: provider returned error"; got != want {
|
||||
t.Fatalf("persisted message = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := payload["statusCode"], float64(503); got != want {
|
||||
t.Fatalf("persisted statusCode = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := payload["cause"], "upstream returned an empty completion"; got != want {
|
||||
t.Fatalf("persisted cause = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrorEventPayloadBoundsCause(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
cause string
|
||||
}{
|
||||
// "é" is two bytes; an odd cap boundary must not split it.
|
||||
{"two-byte runes", strings.Repeat("é", maxErrorCauseBytes)},
|
||||
{"one byte over the cap", strings.Repeat("a", maxErrorCauseBytes+1)},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
payload := ErrorEventPayload(&zeroruntime.StreamError{Message: "m", StatusCode: 500, Cause: tc.cause})
|
||||
|
||||
got, ok := payload["cause"].(string)
|
||||
if !ok {
|
||||
t.Fatalf("cause = %#v, want string", payload["cause"])
|
||||
}
|
||||
if !strings.HasSuffix(got, errorCauseTruncationMarker) {
|
||||
t.Fatalf("cause missing truncation marker: ...%q", got[len(got)-20:])
|
||||
}
|
||||
// The marker counts: the persisted cause as a whole stays within the cap.
|
||||
if len(got) > maxErrorCauseBytes {
|
||||
t.Fatalf("bounded cause is %d bytes with its marker, want <= %d", len(got), maxErrorCauseBytes)
|
||||
}
|
||||
if !utf8.ValidString(got) {
|
||||
t.Fatal("bounded cause is not valid UTF-8")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrorEventPayloadKeepsCauseAtCap(t *testing.T) {
|
||||
cause := strings.Repeat("a", maxErrorCauseBytes)
|
||||
payload := ErrorEventPayload(&zeroruntime.StreamError{Message: "m", Cause: cause})
|
||||
|
||||
if got := payload["cause"]; got != cause {
|
||||
t.Fatalf("cause at exactly the cap was altered (len %d)", len(got.(string)))
|
||||
}
|
||||
}
|
||||
|
||||
// The bound holds for the cause as it reads back from events.jsonl, not only
|
||||
// for the string handed to AppendEvent. JSON turns each invalid byte into a
|
||||
// three-byte U+FFFD, so an error body of invalid bytes cut to the limit came
|
||||
// back at up to three times it.
|
||||
func TestAnInvalidUTF8CauseStaysWithinTheBoundAfterTheStoreRoundTrip(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
cause string
|
||||
}{
|
||||
{"invalid bytes between ASCII", strings.Repeat("\xffa", maxErrorCauseBytes)},
|
||||
{"a run of invalid bytes", strings.Repeat("\xfe", maxErrorCauseBytes) + "tail"},
|
||||
{"short and invalid", "status \xff\xfe body"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
store := NewStore(StoreOptions{RootDir: t.TempDir(), Now: fixedClock("2026-11-09T12:00:00Z")})
|
||||
session, err := store.Create(CreateInput{SessionID: "invalid_cause"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create returned error: %v", err)
|
||||
}
|
||||
streamErr := &zeroruntime.StreamError{Message: "provider error", StatusCode: 502, Cause: tc.cause}
|
||||
if _, err := store.AppendEvent(session.SessionID, AppendEventInput{Type: EventError, Payload: ErrorEventPayload(streamErr)}); err != nil {
|
||||
t.Fatalf("AppendEvent returned error: %v", err)
|
||||
}
|
||||
events, err := store.ReadEvents(session.SessionID)
|
||||
if err != nil || len(events) != 1 {
|
||||
t.Fatalf("ReadEvents = %d events, %v", len(events), err)
|
||||
}
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(events[0].Payload, &payload); err != nil {
|
||||
t.Fatalf("unmarshal persisted payload: %v", err)
|
||||
}
|
||||
got, ok := payload["cause"].(string)
|
||||
if !ok {
|
||||
t.Fatalf("persisted cause = %#v, want a string", payload["cause"])
|
||||
}
|
||||
if len(got) > maxErrorCauseBytes {
|
||||
t.Fatalf("persisted cause reads back at %d bytes, want <= %d", len(got), maxErrorCauseBytes)
|
||||
}
|
||||
if !utf8.ValidString(got) {
|
||||
t.Fatal("persisted cause is not valid UTF-8")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,9 @@
|
||||
package specialist
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
@@ -31,18 +33,41 @@ type specialistAccountingInput struct {
|
||||
Mode string
|
||||
Background bool
|
||||
PID int
|
||||
// LaunchFailed marks a child that never started (e.g. the command failed to
|
||||
// exec). The internal start/stop accounting still records the attempt, but no
|
||||
// public specialistStart/specialistStop hook fires for it: there was no
|
||||
// sub-agent lifecycle to observe. The zero value is "started", so a caller
|
||||
// that forgets to set it never silently suppresses hooks.
|
||||
LaunchFailed bool
|
||||
}
|
||||
|
||||
// recordSpecialistStart records the internal attempt accounting. It deliberately
|
||||
// does NOT dispatch the public specialistStart hook: the hook fires only once
|
||||
// the child has actually started (dispatchSpecialistStartHook), so a failed
|
||||
// launch cannot announce a sub-agent that never existed.
|
||||
func (executor Executor) recordSpecialistStart(input specialistAccountingInput) {
|
||||
payload := baseSpecialistPayload(input)
|
||||
_, _ = appendSpecialistSessionEvent(executor.SessionStore, input.ParentSessionID, sessions.EventSpecialistStart, payload)
|
||||
}
|
||||
|
||||
// dispatchSpecialistStartHook publishes the specialistStart hook for a child
|
||||
// whose startup was confirmed. Callers pass the same accounting input used for
|
||||
// the attempt record so payload and matcher subject stay identical.
|
||||
func (executor Executor) dispatchSpecialistStartHook(input specialistAccountingInput) {
|
||||
if input.LaunchFailed || executor.LifecycleHooks == nil || executor.LifecycleHooks.Dispatch == nil {
|
||||
return
|
||||
}
|
||||
executor.dispatchLifecycleHook("specialistStart", input, baseSpecialistPayload(input))
|
||||
}
|
||||
|
||||
func (executor Executor) recordSpecialistStop(input specialistAccountingInput, summary StreamResult, status string, exitCode int, runErr error, usageRolledUp bool) {
|
||||
store := accountingStore(executor.SessionStore)
|
||||
accountingMu.Lock()
|
||||
defer accountingMu.Unlock()
|
||||
payload := baseSpecialistPayload(input)
|
||||
// childStarted is the durable eligibility flag for the public stop hook: a
|
||||
// later reaper (TaskOutput poll, recovered TaskStop, onExit in another
|
||||
// process) derives whether the terminal transition deserves a hook from the
|
||||
// recorded stop event, since it cannot know whether the child ever started.
|
||||
payload["childStarted"] = !input.LaunchFailed
|
||||
if summary.RunID != "" {
|
||||
payload["runId"] = summary.RunID
|
||||
}
|
||||
@@ -55,9 +80,127 @@ func (executor Executor) recordSpecialistStop(input specialistAccountingInput, s
|
||||
if len(summary.Errors) > 0 {
|
||||
payload["errors"] = append([]string(nil), summary.Errors...)
|
||||
}
|
||||
// Atomic check+append under the session lock so a concurrent stop path cannot
|
||||
// also pass the existence check and write a duplicate stop event.
|
||||
// Hold accountingMu only for the deduped append so a slow lifecycle hook cannot
|
||||
// stall concurrent specialist stop/usage accounting.
|
||||
accountingMu.Lock()
|
||||
_, _ = appendSpecialistEventOnce(store, input.ParentSessionID, sessions.EventSpecialistStop, payload, input.ChildSessionID, summary.RunID)
|
||||
accountingMu.Unlock()
|
||||
executor.dispatchSpecialistStopHook(store, input, payload, summary.RunID)
|
||||
}
|
||||
|
||||
// specialistStopHookEvent durably records that the specialistStop hook dispatch
|
||||
// decision for one child/run was settled. It lives in the parent session's event
|
||||
// log - the same cross-process, lock-guarded ledger the accounting dedup uses -
|
||||
// so a later Zero process reaping the same task (fresh LifecycleHooks bridge)
|
||||
// does not re-run the hook. This is internal bookkeeping, not a lifecycle event.
|
||||
const specialistStopHookEvent sessions.EventType = "specialist_stop_hook"
|
||||
|
||||
// dispatchSpecialistStopHook runs the specialistStop hook at most once per
|
||||
// child/run. The claim follows the accounting equivalence rules (an empty runId
|
||||
// is a catch-all for the same child), so concurrent or sequential finishers -
|
||||
// Task.onExit, a TaskOutput poll in another process, a recovered TaskStop - share
|
||||
// one durable claim. Swarm members have no parent session (nothing durable to
|
||||
// key on), so they fall back to this bridge's in-process claim, which mirrors
|
||||
// the same run-ID equivalence.
|
||||
//
|
||||
// Hooks are external side effects, so on a store error the durable claim cannot
|
||||
// be established; the hook then falls back to the in-process claim rather than
|
||||
// running unchecked, keeping at-most-once within this process. A won durable
|
||||
// claim also takes the in-process claim, so a transient store error before or
|
||||
// after it on the same bridge cannot dispatch the same stop a second time.
|
||||
func (executor Executor) dispatchSpecialistStopHook(store *sessions.Store, input specialistAccountingInput, payload map[string]any, runID string) {
|
||||
hooks := executor.LifecycleHooks
|
||||
if hooks == nil || hooks.Dispatch == nil {
|
||||
return
|
||||
}
|
||||
started := !input.LaunchFailed
|
||||
parentSessionID := strings.TrimSpace(input.ParentSessionID)
|
||||
if parentSessionID == "" || !sessions.ValidSessionID(parentSessionID) {
|
||||
if started && hooks.claimInProcessStopDispatch(input.ChildSessionID, runID) {
|
||||
executor.dispatchLifecycleHook("specialistStop", input, payload)
|
||||
}
|
||||
return
|
||||
}
|
||||
if recorded, ok := recordedStopChildStarted(store, parentSessionID, input.ChildSessionID, runID); ok {
|
||||
started = recorded
|
||||
}
|
||||
claimed, err := appendSpecialistEventOnce(store, parentSessionID, specialistStopHookEvent, stopHookClaimPayload(input, runID, started), input.ChildSessionID, runID)
|
||||
if err != nil {
|
||||
// The durable ledger is unavailable. Keep the hook advisory but bounded to
|
||||
// a single dispatch in this process rather than dropping it or risking an
|
||||
// unbounded duplicate.
|
||||
if started && hooks.claimInProcessStopDispatch(input.ChildSessionID, runID) {
|
||||
executor.dispatchLifecycleHook("specialistStop", input, payload)
|
||||
}
|
||||
return
|
||||
}
|
||||
if !claimed {
|
||||
return
|
||||
}
|
||||
// Take the in-process claim even for an ineligible stop: this bridge has
|
||||
// settled the decision, and a later fallback must not re-decide it.
|
||||
if hooks.claimInProcessStopDispatch(input.ChildSessionID, runID) && started {
|
||||
executor.dispatchLifecycleHook("specialistStop", input, payload)
|
||||
}
|
||||
}
|
||||
|
||||
// recordedStopChildStarted reads the first recorded stop event matching the
|
||||
// child/run equivalence rules and reports its childStarted eligibility flag.
|
||||
// ok is false when no stop event exists yet or it predates the flag, in which
|
||||
// case the caller keeps its locally known eligibility.
|
||||
func recordedStopChildStarted(store *sessions.Store, parentSessionID, childSessionID, runID string) (started bool, ok bool) {
|
||||
store = accountingStore(store)
|
||||
childSessionID = strings.TrimSpace(childSessionID)
|
||||
if childSessionID == "" {
|
||||
return false, false
|
||||
}
|
||||
runID = strings.TrimSpace(runID)
|
||||
events, err := store.ReadEvents(parentSessionID)
|
||||
if err != nil {
|
||||
return false, false
|
||||
}
|
||||
for _, event := range events {
|
||||
if event.Type != sessions.EventSpecialistStop {
|
||||
continue
|
||||
}
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(event.Payload, &payload); err != nil {
|
||||
continue
|
||||
}
|
||||
if !specialistPayloadMatches(payload, childSessionID, runID) {
|
||||
continue
|
||||
}
|
||||
if started, ok := payload["childStarted"].(bool); ok {
|
||||
return started, true
|
||||
}
|
||||
return false, false
|
||||
}
|
||||
return false, false
|
||||
}
|
||||
|
||||
func stopHookClaimPayload(input specialistAccountingInput, runID string, started bool) map[string]any {
|
||||
payload := baseSpecialistPayload(input)
|
||||
payload["hook"] = "specialistStop"
|
||||
// childStarted is the resolved eligibility (locally known or read back from
|
||||
// the durable stop event); dispatched records that a hook was run for it.
|
||||
payload["childStarted"] = started
|
||||
payload["dispatched"] = started
|
||||
if runID = strings.TrimSpace(runID); runID != "" {
|
||||
payload["runId"] = runID
|
||||
}
|
||||
return payload
|
||||
}
|
||||
|
||||
func (executor Executor) dispatchLifecycleHook(event string, input specialistAccountingInput, payload map[string]any) {
|
||||
if executor.LifecycleHooks == nil || executor.LifecycleHooks.Dispatch == nil {
|
||||
return
|
||||
}
|
||||
hookPayload := make(map[string]any, len(payload)+1)
|
||||
for key, value := range payload {
|
||||
hookPayload[key] = value
|
||||
}
|
||||
hookPayload["event"] = event
|
||||
executor.LifecycleHooks.Dispatch(context.Background(), event, strings.TrimSpace(input.SpecialistName), hookPayload)
|
||||
}
|
||||
|
||||
func (executor Executor) rollUpSpecialistUsage(input specialistAccountingInput, summary StreamResult) bool {
|
||||
@@ -82,6 +225,18 @@ func (executor Executor) recordBackgroundTaskAccounting(task background.Task, su
|
||||
executor.recordSpecialistStop(input, summary, string(task.Status), task.ExitCode, nil, rolledUp)
|
||||
}
|
||||
|
||||
// taskStreamSummary builds the stream summary for a finished background task
|
||||
// from its output file. Every reaper (Task.onExit, TaskOutput, TaskStop) derives
|
||||
// its runId this way, so the durable stop event and stop-hook claim agree on the
|
||||
// run identity regardless of which path observed the task first.
|
||||
func taskStreamSummary(task background.Task) StreamResult {
|
||||
summary := StreamResult{ExitCode: task.ExitCode}
|
||||
if data, err := os.ReadFile(task.OutputFile); err == nil {
|
||||
summary, _ = summarizeTaskData(string(data), task.ExitCode)
|
||||
}
|
||||
return summary
|
||||
}
|
||||
|
||||
func appendSpecialistUsageRollup(store *sessions.Store, input specialistAccountingInput, summary StreamResult) (bool, error) {
|
||||
if !summary.Usage.HasUsage() {
|
||||
return false, nil
|
||||
@@ -132,19 +287,7 @@ func specialistEventMatcher(eventType sessions.EventType, childSessionID, runID
|
||||
if err := json.Unmarshal(event.Payload, &payload); err != nil {
|
||||
continue
|
||||
}
|
||||
if specialistPayloadString(payload, "source") != specialistAccountingSource {
|
||||
continue
|
||||
}
|
||||
if specialistPayloadString(payload, "childSessionId") != childSessionID && specialistPayloadString(payload, "taskId") != childSessionID {
|
||||
continue
|
||||
}
|
||||
// An already-recorded event with NO runId is a catch-all for this child
|
||||
// (e.g. the immediate stop written when a PID couldn't be registered) and
|
||||
// must match a later event that does carry a runId — otherwise the same
|
||||
// child gets two stop/usage events. We match when: querying without a runId,
|
||||
// the existing event has none, or the runIds are equal.
|
||||
existingRunID := specialistPayloadString(payload, "runId")
|
||||
if runID == "" || existingRunID == "" || existingRunID == runID {
|
||||
if specialistPayloadMatches(payload, childSessionID, runID) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -152,6 +295,25 @@ func specialistEventMatcher(eventType sessions.EventType, childSessionID, runID
|
||||
}
|
||||
}
|
||||
|
||||
// specialistPayloadMatches applies the shared "same specialist stop" identity to
|
||||
// one event payload. The accounting dedup (specialistEventMatcher) and the
|
||||
// durable stop-hook claim both use it, so the audit event and the hook side
|
||||
// effect can never disagree about what counts as a repeat. An already-recorded
|
||||
// event with NO runId is a catch-all for this child (e.g. the immediate stop
|
||||
// written when a PID couldn't be registered) and matches a later event that
|
||||
// carries a runId - otherwise the same child gets two stop/usage events. A query
|
||||
// without a runId likewise matches any recorded run.
|
||||
func specialistPayloadMatches(payload map[string]any, childSessionID, runID string) bool {
|
||||
if specialistPayloadString(payload, "source") != specialistAccountingSource {
|
||||
return false
|
||||
}
|
||||
if specialistPayloadString(payload, "childSessionId") != childSessionID && specialistPayloadString(payload, "taskId") != childSessionID {
|
||||
return false
|
||||
}
|
||||
existingRunID := specialistPayloadString(payload, "runId")
|
||||
return runID == "" || existingRunID == "" || existingRunID == runID
|
||||
}
|
||||
|
||||
// appendSpecialistEventOnce atomically records a "record once" specialist event:
|
||||
// it appends payload only if no matching event already exists, under the session
|
||||
// store's lock so concurrent stop/usage callers (even cross-process) cannot both
|
||||
|
||||
@@ -3,10 +3,15 @@ package specialist
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Twigpine/zero/internal/background"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
@@ -21,10 +26,21 @@ func TestExecutorRecordsForegroundLifecycleAndUsageRollup(t *testing.T) {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
zero := 0
|
||||
var hookEvents []string
|
||||
var hookNames []string
|
||||
executor := Executor{
|
||||
BinaryPath: "/usr/local/bin/zero",
|
||||
SessionStore: store,
|
||||
NewSessionID: func() (string, error) { return "child_task", nil },
|
||||
LifecycleHooks: &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, specialistName string, payload map[string]any) {
|
||||
hookEvents = append(hookEvents, event)
|
||||
hookNames = append(hookNames, specialistName)
|
||||
if payload["event"] != event {
|
||||
t.Errorf("payload event = %#v, want %q", payload["event"], event)
|
||||
}
|
||||
},
|
||||
},
|
||||
Load: func(LoadOptions) (LoadResult, error) {
|
||||
return LoadResult{Specialists: []Manifest{{
|
||||
Metadata: Metadata{Name: "worker", Description: "Does focused work"},
|
||||
@@ -59,12 +75,18 @@ func TestExecutorRecordsForegroundLifecycleAndUsageRollup(t *testing.T) {
|
||||
if result.Result.Status != tools.StatusOK {
|
||||
t.Fatalf("Run status = %s, output=%s", result.Result.Status, result.Result.Output)
|
||||
}
|
||||
if got, want := strings.Join(hookEvents, ","), "specialistStart,specialistStop"; got != want {
|
||||
t.Fatalf("lifecycle hooks = %q, want %q", got, want)
|
||||
}
|
||||
if got, want := strings.Join(hookNames, ","), "worker,worker"; got != want {
|
||||
t.Fatalf("lifecycle specialist names = %q, want %q", got, want)
|
||||
}
|
||||
|
||||
events, err := store.ReadEvents(parent.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
if got, want := eventTypes(events), []sessions.EventType{sessions.EventSpecialistStart, sessions.EventUsage, sessions.EventSpecialistStop}; strings.Join(eventTypesString(got), ",") != strings.Join(eventTypesString(want), ",") {
|
||||
if got, want := eventTypes(events), []sessions.EventType{sessions.EventSpecialistStart, sessions.EventUsage, sessions.EventSpecialistStop, specialistStopHookEvent}; strings.Join(eventTypesString(got), ",") != strings.Join(eventTypesString(want), ",") {
|
||||
t.Fatalf("event types = %#v, want %#v", got, want)
|
||||
}
|
||||
startPayload := eventPayload(t, events[0])
|
||||
@@ -88,6 +110,13 @@ func TestExecutorRecordsForegroundLifecycleAndUsageRollup(t *testing.T) {
|
||||
requirePayloadString(t, stopPayload, "status", "success")
|
||||
requirePayloadInt(t, stopPayload, "exitCode", 0)
|
||||
requirePayloadBool(t, stopPayload, "usageRolledUp", true)
|
||||
requirePayloadBool(t, stopPayload, "childStarted", true)
|
||||
|
||||
claimPayload := eventPayload(t, events[3])
|
||||
requirePayloadString(t, claimPayload, "hook", "specialistStop")
|
||||
requirePayloadString(t, claimPayload, "runId", "run_1")
|
||||
requirePayloadBool(t, claimPayload, "dispatched", true)
|
||||
requirePayloadBool(t, claimPayload, "childStarted", true)
|
||||
}
|
||||
|
||||
func TestExecutorRecordsStartedChildErrorExitCode(t *testing.T) {
|
||||
@@ -286,7 +315,17 @@ func TestRecordSpecialistStopDedupesUnderConcurrency(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
executor := Executor{SessionStore: store}
|
||||
var hookStops atomic.Int32
|
||||
executor := Executor{
|
||||
SessionStore: store,
|
||||
LifecycleHooks: &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
if event == "specialistStop" {
|
||||
hookStops.Add(1)
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
input := specialistAccountingInput{
|
||||
ParentSessionID: parent.SessionID,
|
||||
ChildSessionID: "child_task",
|
||||
@@ -321,4 +360,799 @@ func TestRecordSpecialistStopDedupesUnderConcurrency(t *testing.T) {
|
||||
if stops != 1 {
|
||||
t.Fatalf("expected exactly 1 stop event under concurrency, got %d", stops)
|
||||
}
|
||||
if got := hookStops.Load(); got != 1 {
|
||||
t.Fatalf("specialistStop hooks = %d, want 1 under concurrency", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordSpecialistStopFiresWithoutParentSession(t *testing.T) {
|
||||
var hookStops atomic.Int32
|
||||
executor := Executor{
|
||||
LifecycleHooks: &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, name string, _ map[string]any) {
|
||||
if event == "specialistStop" {
|
||||
hookStops.Add(1)
|
||||
if name != "subagent" {
|
||||
t.Errorf("specialist name = %q, want subagent", name)
|
||||
}
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
executor.recordSpecialistStop(specialistAccountingInput{
|
||||
// Swarm members omit ParentSessionID; append never succeeds.
|
||||
ChildSessionID: "member_child",
|
||||
SpecialistName: "subagent",
|
||||
Mode: "foreground",
|
||||
}, StreamResult{RunID: "run_swarm"}, "success", 0, nil, false)
|
||||
|
||||
if got := hookStops.Load(); got != 1 {
|
||||
t.Fatalf("specialistStop hooks = %d, want 1 with empty parent", got)
|
||||
}
|
||||
// Second observation of the same child/run must not re-fire.
|
||||
executor.recordSpecialistStop(specialistAccountingInput{
|
||||
ChildSessionID: "member_child",
|
||||
SpecialistName: "subagent",
|
||||
Mode: "foreground",
|
||||
}, StreamResult{RunID: "run_swarm"}, "success", 0, nil, false)
|
||||
if got := hookStops.Load(); got != 1 {
|
||||
t.Fatalf("specialistStop hooks = %d after duplicate, want 1", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordSpecialistStopHookLessWinnerDoesNotSuppressLaterHooks(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
input := specialistAccountingInput{
|
||||
ParentSessionID: parent.SessionID,
|
||||
ChildSessionID: "child_task",
|
||||
SpecialistName: "worker",
|
||||
Mode: "background",
|
||||
Background: true,
|
||||
}
|
||||
summary := StreamResult{RunID: "run_1"}
|
||||
|
||||
// Hook-less TaskOutput path wins the session append race.
|
||||
Executor{SessionStore: store}.recordSpecialistStop(input, summary, "success", 0, nil, true)
|
||||
|
||||
var hookStops atomic.Int32
|
||||
hooks := &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
if event == "specialistStop" {
|
||||
hookStops.Add(1)
|
||||
}
|
||||
},
|
||||
}
|
||||
Executor{SessionStore: store, LifecycleHooks: hooks}.recordSpecialistStop(input, summary, "success", 0, nil, true)
|
||||
if got := hookStops.Load(); got != 1 {
|
||||
t.Fatalf("specialistStop hooks = %d, want 1 after hook-less winner", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A durable stop-hook claim must survive a process boundary: a later Zero
|
||||
// process (fresh LifecycleHooks bridge over the same session store) reaping the
|
||||
// same task must not re-run the hook even though its session-event append is
|
||||
// rejected as a duplicate.
|
||||
func TestRecordSpecialistStopHookClaimDurableAcrossBridges(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
input := specialistAccountingInput{
|
||||
ParentSessionID: parent.SessionID,
|
||||
ChildSessionID: "child_task",
|
||||
SpecialistName: "worker",
|
||||
Mode: "background",
|
||||
Background: true,
|
||||
}
|
||||
summary := StreamResult{RunID: "run_1"}
|
||||
|
||||
var firstBridge atomic.Int32
|
||||
Executor{SessionStore: store, LifecycleHooks: stopCounterHooks(&firstBridge)}.recordSpecialistStop(input, summary, "success", 0, nil, true)
|
||||
if got := firstBridge.Load(); got != 1 {
|
||||
t.Fatalf("first bridge stop hooks = %d, want 1", got)
|
||||
}
|
||||
|
||||
var secondBridge atomic.Int32
|
||||
Executor{SessionStore: store, LifecycleHooks: stopCounterHooks(&secondBridge)}.recordSpecialistStop(input, summary, "success", 0, nil, true)
|
||||
if got := secondBridge.Load(); got != 0 {
|
||||
t.Fatalf("second bridge stop hooks = %d, want 0: the durable claim must suppress a re-dispatch", got)
|
||||
}
|
||||
|
||||
events, err := store.ReadEvents(parent.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
if claims := eventsOfType(events, specialistStopHookEvent); len(claims) != 1 {
|
||||
t.Fatalf("stop-hook claim events = %d, want 1", len(claims))
|
||||
}
|
||||
if stops := eventsOfType(events, sessions.EventSpecialistStop); len(stops) != 1 {
|
||||
t.Fatalf("stop events = %d, want 1", len(stops))
|
||||
}
|
||||
}
|
||||
|
||||
// The durable claim must follow the accounting run-ID equivalence: an empty
|
||||
// runId is a catch-all in both directions, while genuinely distinct runs stay
|
||||
// distinct stop transitions.
|
||||
func TestRecordSpecialistStopHookClaimRunIDEquivalence(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
first string
|
||||
second string
|
||||
want int32
|
||||
}{
|
||||
{name: "empty run id is a catch-all for a later run", first: "", second: "run_1", want: 1},
|
||||
{name: "a recorded run covers a later empty query", first: "run_1", second: "", want: 1},
|
||||
{name: "distinct runs are distinct stops", first: "run_1", second: "run_2", want: 2},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
var stops atomic.Int32
|
||||
executor := Executor{SessionStore: store, LifecycleHooks: stopCounterHooks(&stops)}
|
||||
input := specialistAccountingInput{
|
||||
ParentSessionID: parent.SessionID,
|
||||
ChildSessionID: "child_task",
|
||||
SpecialistName: "worker",
|
||||
Mode: "background",
|
||||
Background: true,
|
||||
}
|
||||
executor.recordSpecialistStop(input, StreamResult{RunID: tc.first}, "success", 0, nil, false)
|
||||
executor.recordSpecialistStop(input, StreamResult{RunID: tc.second}, "success", 0, nil, false)
|
||||
if got := stops.Load(); got != tc.want {
|
||||
t.Fatalf("stop hooks = %d, want %d", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The in-process fallback (swarm members have no parent session) applies the
|
||||
// same run-ID equivalence, so a later observation carrying a runId cannot
|
||||
// re-fire a stop that was already claimed without one.
|
||||
func TestRecordSpecialistStopInProcessClaimRunIDEquivalence(t *testing.T) {
|
||||
var stops atomic.Int32
|
||||
executor := Executor{LifecycleHooks: stopCounterHooks(&stops)}
|
||||
input := specialistAccountingInput{ChildSessionID: "member_child", SpecialistName: "subagent", Mode: "foreground"}
|
||||
executor.recordSpecialistStop(input, StreamResult{}, "success", 0, nil, false)
|
||||
executor.recordSpecialistStop(input, StreamResult{RunID: "run_later"}, "success", 0, nil, false)
|
||||
if got := stops.Load(); got != 1 {
|
||||
t.Fatalf("stop hooks = %d, want 1", got)
|
||||
}
|
||||
}
|
||||
|
||||
// When the durable ledger is briefly unavailable, the in-process fallback and a
|
||||
// durable claim on the same bridge must make one dispatch decision between
|
||||
// them, whichever order they happen in.
|
||||
func TestRecordSpecialistStopDurableAndFallbackClaimsShareOneDispatch(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
fallbackFirst bool
|
||||
}{
|
||||
{name: "fallback then durable", fallbackFirst: true},
|
||||
{name: "durable then fallback", fallbackFirst: false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
eventsFile := filepath.Join(store.RootDir, parent.SessionID, sessions.EventsFile)
|
||||
asideFile := eventsFile + ".aside"
|
||||
// A directory where the event log lives makes every read and append
|
||||
// fail on all platforms, the way a transient store error would.
|
||||
breakLedger := func() {
|
||||
t.Helper()
|
||||
if err := os.Rename(eventsFile, asideFile); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatalf("move event log aside: %v", err)
|
||||
}
|
||||
if err := os.Mkdir(eventsFile, 0o700); err != nil {
|
||||
t.Fatalf("block event log: %v", err)
|
||||
}
|
||||
}
|
||||
restoreLedger := func() {
|
||||
t.Helper()
|
||||
if err := os.Remove(eventsFile); err != nil {
|
||||
t.Fatalf("unblock event log: %v", err)
|
||||
}
|
||||
if err := os.Rename(asideFile, eventsFile); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatalf("restore event log: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
var stops atomic.Int32
|
||||
executor := Executor{SessionStore: store, LifecycleHooks: stopCounterHooks(&stops)}
|
||||
input := specialistAccountingInput{
|
||||
ParentSessionID: parent.SessionID,
|
||||
ChildSessionID: "child_task",
|
||||
SpecialistName: "worker",
|
||||
Mode: "background",
|
||||
Background: true,
|
||||
}
|
||||
summary := StreamResult{RunID: "run_1"}
|
||||
|
||||
if tc.fallbackFirst {
|
||||
breakLedger()
|
||||
executor.recordSpecialistStop(input, summary, "success", 0, nil, false)
|
||||
restoreLedger()
|
||||
executor.recordSpecialistStop(input, summary, "success", 0, nil, false)
|
||||
} else {
|
||||
executor.recordSpecialistStop(input, summary, "success", 0, nil, false)
|
||||
breakLedger()
|
||||
executor.recordSpecialistStop(input, summary, "success", 0, nil, false)
|
||||
restoreLedger()
|
||||
}
|
||||
if got := stops.Load(); got != 1 {
|
||||
t.Fatalf("stop hooks = %d, want 1 across durable and fallback claims", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A child that never started must not publish public lifecycle hooks, while the
|
||||
// internal attempt/stop accounting is retained and durably marks the stop as
|
||||
// ineligible so a later reaper cannot fire the hook either.
|
||||
func TestExecutorSkipsHooksWhenChildNeverStarts(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
var hooks []string
|
||||
executor := Executor{
|
||||
BinaryPath: "/usr/local/bin/zero",
|
||||
SessionStore: store,
|
||||
NewSessionID: func() (string, error) { return "child_task", nil },
|
||||
LifecycleHooks: &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
hooks = append(hooks, event)
|
||||
},
|
||||
},
|
||||
Load: func(LoadOptions) (LoadResult, error) {
|
||||
return LoadResult{Specialists: []Manifest{{
|
||||
Metadata: Metadata{Name: "worker", Description: "Does focused work"},
|
||||
SystemPrompt: "Work carefully.",
|
||||
ResolvedTools: []string{"read_file"},
|
||||
}}}, nil
|
||||
},
|
||||
RunChild: func(context.Context, string, []string, func(streamjson.Event)) (ChildRunResult, error) {
|
||||
return ChildRunResult{}, errors.New("exec: no such file or directory")
|
||||
},
|
||||
}
|
||||
|
||||
if _, err := executor.Run(context.Background(), TaskParameters{Name: "worker", Prompt: "inspect auth"}, TaskRunOptions{ParentSessionID: parent.SessionID}); err == nil {
|
||||
t.Fatal("Run returned nil error for a failed child start")
|
||||
}
|
||||
if len(hooks) != 0 {
|
||||
t.Fatalf("public lifecycle hooks fired for a child that never started: %#v", hooks)
|
||||
}
|
||||
|
||||
events, err := store.ReadEvents(parent.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
if starts := eventsOfType(events, sessions.EventSpecialistStart); len(starts) != 1 {
|
||||
t.Fatalf("attempt start events = %d, want 1", len(starts))
|
||||
}
|
||||
stopEvents := eventsOfType(events, sessions.EventSpecialistStop)
|
||||
if len(stopEvents) != 1 {
|
||||
t.Fatalf("stop events = %#v", events)
|
||||
}
|
||||
requirePayloadBool(t, eventPayload(t, stopEvents[0]), "childStarted", false)
|
||||
|
||||
claims := eventsOfType(events, specialistStopHookEvent)
|
||||
if len(claims) != 1 {
|
||||
t.Fatalf("stop-hook claim events = %d, want 1", len(claims))
|
||||
}
|
||||
requirePayloadBool(t, eventPayload(t, claims[0]), "dispatched", false)
|
||||
}
|
||||
|
||||
// A never-started background child leaves an errored task behind; a later
|
||||
// TaskOutput reaper must not fire the stop hook from the recorded eligibility.
|
||||
func TestBackgroundLaunchFailureDoesNotFireStopHookOnReap(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
manager, err := background.NewManager(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("NewManager returned error: %v", err)
|
||||
}
|
||||
var hooks []string
|
||||
executor := Executor{
|
||||
BinaryPath: "/usr/local/bin/zero",
|
||||
SessionStore: store,
|
||||
BackgroundManager: manager,
|
||||
NewSessionID: func() (string, error) { return "child_task", nil },
|
||||
LifecycleHooks: &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
hooks = append(hooks, event)
|
||||
},
|
||||
},
|
||||
Load: func(LoadOptions) (LoadResult, error) {
|
||||
return LoadResult{Specialists: []Manifest{{
|
||||
Metadata: Metadata{Name: "worker", Description: "Does focused work"},
|
||||
SystemPrompt: "Work carefully.",
|
||||
ResolvedTools: []string{"read_file"},
|
||||
}}}, nil
|
||||
},
|
||||
LaunchBackground: func(string, []string, string, func(int)) (int, error) {
|
||||
return 0, errors.New("launch failed")
|
||||
},
|
||||
}
|
||||
|
||||
_, err = executor.Run(context.Background(), TaskParameters{Name: "worker", Prompt: "inspect auth", RunInBackground: true}, TaskRunOptions{ParentSessionID: parent.SessionID})
|
||||
if err == nil {
|
||||
t.Fatal("Run returned nil error for a failed background launch")
|
||||
}
|
||||
if len(hooks) != 0 {
|
||||
t.Fatalf("public lifecycle hooks fired for a child that never started: %#v", hooks)
|
||||
}
|
||||
|
||||
// A separate process reaping the errored task must stay silent too.
|
||||
var reapHooks atomic.Int32
|
||||
tool := NewOutputTool(manager)
|
||||
tool.SessionStore = store
|
||||
tool.LifecycleHooks = stopCounterHooks(&reapHooks)
|
||||
if result := tool.Run(context.Background(), map[string]any{"task_id": "child_task"}); result.Status != tools.StatusOK {
|
||||
t.Fatalf("TaskOutput status = %s output=%s", result.Status, result.Output)
|
||||
}
|
||||
if got := reapHooks.Load(); got != 0 {
|
||||
t.Fatalf("stop hooks from reaping a never-started task = %d, want 0", got)
|
||||
}
|
||||
|
||||
events, err := store.ReadEvents(parent.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
if stopEvents := eventsOfType(events, sessions.EventSpecialistStop); len(stopEvents) != 1 {
|
||||
t.Fatalf("stop events = %d, want 1", len(stopEvents))
|
||||
}
|
||||
if claims := eventsOfType(events, specialistStopHookEvent); len(claims) != 1 {
|
||||
t.Fatalf("stop-hook claim events = %d, want 1", len(claims))
|
||||
}
|
||||
}
|
||||
|
||||
// A background child that exits before the launcher returns must not run its
|
||||
// terminal accounting - and therefore must not fire the specialistStop hook -
|
||||
// until the start decision has completed. Otherwise a fast-exiting child
|
||||
// publishes specialistStop before specialistStart, and the recorded stop can
|
||||
// miss the pid that SetPID has not stored yet. The synchronous launcher below
|
||||
// also pins the deadlock requirement: the exit callback may not wait inline,
|
||||
// because onExit runs before the launcher (and the start decision) can return.
|
||||
func TestRunBackgroundDefersExitUntilStartDecisionCompletes(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
manager, err := background.NewManager(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("NewManager returned error: %v", err)
|
||||
}
|
||||
var hookMu sync.Mutex
|
||||
var hookEvents []string
|
||||
stopDispatched := make(chan struct{}, 1)
|
||||
executor := Executor{
|
||||
BinaryPath: "/usr/local/bin/zero",
|
||||
SessionStore: store,
|
||||
BackgroundManager: manager,
|
||||
NewSessionID: func() (string, error) { return "child_task", nil },
|
||||
LifecycleHooks: &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
hookMu.Lock()
|
||||
hookEvents = append(hookEvents, event)
|
||||
hookMu.Unlock()
|
||||
if event == "specialistStop" {
|
||||
select {
|
||||
case stopDispatched <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
},
|
||||
},
|
||||
Load: func(LoadOptions) (LoadResult, error) {
|
||||
return LoadResult{Specialists: []Manifest{{
|
||||
Metadata: Metadata{Name: "worker", Description: "Does focused work"},
|
||||
SystemPrompt: "Work carefully.",
|
||||
ResolvedTools: []string{"read_file"},
|
||||
}}}, nil
|
||||
},
|
||||
// The child exits before the launcher returns, so the exit callback runs
|
||||
// synchronously - exactly the ordering hazard this test guards.
|
||||
LaunchBackground: func(_ string, _ []string, _ string, onExit func(exitCode int)) (int, error) {
|
||||
onExit(0)
|
||||
return 1234, nil
|
||||
},
|
||||
}
|
||||
|
||||
runErr := make(chan error, 1)
|
||||
go func() {
|
||||
_, err := executor.Run(context.Background(), TaskParameters{
|
||||
Name: "worker",
|
||||
Prompt: "inspect auth",
|
||||
RunInBackground: true,
|
||||
}, TaskRunOptions{ParentSessionID: parent.SessionID})
|
||||
runErr <- err
|
||||
}()
|
||||
select {
|
||||
case err := <-runErr:
|
||||
if err != nil {
|
||||
t.Fatalf("Run returned error: %v", err)
|
||||
}
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("synchronous launcher deadlocked the background start decision")
|
||||
}
|
||||
|
||||
// Run publishes specialistStart before it returns; the deferred exit handling
|
||||
// publishes specialistStop afterwards.
|
||||
select {
|
||||
case <-stopDispatched:
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("deferred exit accounting never dispatched specialistStop")
|
||||
}
|
||||
hookMu.Lock()
|
||||
gotHooks := strings.Join(hookEvents, ",")
|
||||
hookMu.Unlock()
|
||||
if gotHooks != "specialistStart,specialistStop" {
|
||||
t.Fatalf("lifecycle hooks = %q, want start before stop", gotHooks)
|
||||
}
|
||||
|
||||
events, err := store.ReadEvents(parent.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
stopEvents := eventsOfType(events, sessions.EventSpecialistStop)
|
||||
if len(stopEvents) != 1 {
|
||||
t.Fatalf("stop events = %#v", events)
|
||||
}
|
||||
// The deferred stop must observe the task after SetPID; a stop recorded
|
||||
// before the start decision completes would omit the pid.
|
||||
requirePayloadInt(t, eventPayload(t, stopEvents[0]), "pid", 1234)
|
||||
}
|
||||
|
||||
// When SetPID fails after the child started, Run kills the unrecorded child. Only
|
||||
// a successful kill is a terminal stop. A failed kill may leave the child
|
||||
// running, so Run must keep the task running (no reaper may claim it) and must
|
||||
// not record the stop or take the one-shot specialistStop claim; the launcher's
|
||||
// exit callback does both when the child really ends.
|
||||
func TestRunBackgroundSetPIDFailureRecordsStopOnlyAfterTermination(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
killErr error
|
||||
}{
|
||||
{name: "kill succeeds"},
|
||||
{name: "kill fails", killErr: errors.New("operation not permitted")},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
manager, err := background.NewManager(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("NewManager returned error: %v", err)
|
||||
}
|
||||
var killed []int
|
||||
originalTerminate := terminateOrphanedProcess
|
||||
terminateOrphanedProcess = func(pid int) error {
|
||||
killed = append(killed, pid)
|
||||
return tc.killErr
|
||||
}
|
||||
t.Cleanup(func() { terminateOrphanedProcess = originalTerminate })
|
||||
|
||||
var hookMu sync.Mutex
|
||||
var hookEvents []string
|
||||
hooksSoFar := func() string {
|
||||
hookMu.Lock()
|
||||
defer hookMu.Unlock()
|
||||
return strings.Join(hookEvents, ",")
|
||||
}
|
||||
stopDispatched := make(chan struct{}, 1)
|
||||
metadataFile := filepath.Join(manager.RootDir(), "child_task.json")
|
||||
// The child reads its prompt from this file, so it must outlive any
|
||||
// child that may still be running.
|
||||
promptFile := filepath.Join(t.TempDir(), "prompt.md")
|
||||
promptExists := func() bool {
|
||||
_, err := os.Stat(promptFile)
|
||||
return err == nil
|
||||
}
|
||||
var onExitChild func(exitCode int)
|
||||
executor := Executor{
|
||||
BinaryPath: "/usr/local/bin/zero",
|
||||
SessionStore: store,
|
||||
BackgroundManager: manager,
|
||||
NewSessionID: func() (string, error) { return "child_task", nil },
|
||||
PromptFileMaxSize: 1,
|
||||
WritePromptFile: func(prompt string) (string, error) {
|
||||
return promptFile, os.WriteFile(promptFile, []byte(prompt), 0o600)
|
||||
},
|
||||
LifecycleHooks: &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
hookMu.Lock()
|
||||
hookEvents = append(hookEvents, event)
|
||||
hookMu.Unlock()
|
||||
if event == "specialistStop" {
|
||||
select {
|
||||
case stopDispatched <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
},
|
||||
},
|
||||
Load: func(LoadOptions) (LoadResult, error) {
|
||||
return LoadResult{Specialists: []Manifest{{
|
||||
Metadata: Metadata{Name: "worker", Description: "Does focused work"},
|
||||
SystemPrompt: "Work carefully.",
|
||||
ResolvedTools: []string{"read_file"},
|
||||
}}}, nil
|
||||
},
|
||||
LaunchBackground: func(_ string, _ []string, _ string, onExit func(exitCode int)) (int, error) {
|
||||
onExitChild = onExit
|
||||
// The child has started. Put a non-empty directory where the
|
||||
// task metadata lives so SetPID's atomic replace fails.
|
||||
if err := os.Remove(metadataFile); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Join(metadataFile, "blocker"), 0o700); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return 1234, nil
|
||||
},
|
||||
}
|
||||
|
||||
_, runErr := executor.Run(context.Background(), TaskParameters{
|
||||
Name: "worker",
|
||||
Prompt: "inspect auth",
|
||||
RunInBackground: true,
|
||||
}, TaskRunOptions{ParentSessionID: parent.SessionID})
|
||||
if runErr == nil {
|
||||
t.Fatal("Run returned nil error after SetPID failed")
|
||||
}
|
||||
if !reflect.DeepEqual(killed, []int{1234}) {
|
||||
t.Fatalf("terminated pids = %v, want [1234]", killed)
|
||||
}
|
||||
readStops := func() ([]sessions.Event, []sessions.Event) {
|
||||
t.Helper()
|
||||
events, err := store.ReadEvents(parent.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
return eventsOfType(events, sessions.EventSpecialistStop), eventsOfType(events, specialistStopHookEvent)
|
||||
}
|
||||
|
||||
if tc.killErr == nil {
|
||||
// The child is gone, so the stop is terminal and recorded now.
|
||||
stops, claims := readStops()
|
||||
if len(stops) != 1 || len(claims) != 1 {
|
||||
t.Fatalf("after successful kill: stop events = %d, claims = %d, want 1 and 1", len(stops), len(claims))
|
||||
}
|
||||
if got := hooksSoFar(); got != "specialistStart,specialistStop" {
|
||||
t.Fatalf("lifecycle hooks = %q, want specialistStart,specialistStop", got)
|
||||
}
|
||||
if promptExists() {
|
||||
t.Fatal("prompt file survived after the orphan was killed")
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if !strings.Contains(runErr.Error(), "terminate orphaned pid 1234") {
|
||||
t.Fatalf("Run error = %v, want the failed kill surfaced", runErr)
|
||||
}
|
||||
if !promptExists() {
|
||||
t.Fatal("prompt file removed while the child may still be running")
|
||||
}
|
||||
if got := hooksSoFar(); got != "specialistStart" {
|
||||
t.Fatalf("lifecycle hooks after failed kill = %q, want only specialistStart", got)
|
||||
}
|
||||
if stops, claims := readStops(); len(stops) != 0 || len(claims) != 0 {
|
||||
t.Fatalf("after failed kill: stop events = %d, claims = %d, want none while the child may run", len(stops), len(claims))
|
||||
}
|
||||
task, ok := manager.Get("child_task")
|
||||
if !ok || task.Status != background.StatusRunning {
|
||||
t.Fatalf("task after failed kill = %#v (found %v), want still running", task, ok)
|
||||
}
|
||||
// A TaskOutput or TaskStop reap in the meantime must not claim it.
|
||||
executor.recordBackgroundTaskAccounting(task, taskStreamSummary(task))
|
||||
if stops, claims := readStops(); len(stops) != 0 || len(claims) != 0 {
|
||||
t.Fatalf("reap of running task recorded stop events = %d, claims = %d", len(stops), len(claims))
|
||||
}
|
||||
|
||||
// The surviving child finally exits: its exit callback records the stop
|
||||
// and fires the hook exactly once.
|
||||
if err := os.RemoveAll(metadataFile); err != nil {
|
||||
t.Fatalf("RemoveAll metadata blocker returned error: %v", err)
|
||||
}
|
||||
onExitChild(1)
|
||||
select {
|
||||
case <-stopDispatched:
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("exit callback never dispatched specialistStop")
|
||||
}
|
||||
if got := hooksSoFar(); got != "specialistStart,specialistStop" {
|
||||
t.Fatalf("lifecycle hooks = %q, want specialistStart,specialistStop", got)
|
||||
}
|
||||
stops, claims := readStops()
|
||||
if len(stops) != 1 || len(claims) != 1 {
|
||||
t.Fatalf("after child exit: stop events = %d, claims = %d, want 1 and 1", len(stops), len(claims))
|
||||
}
|
||||
requirePayloadInt(t, eventPayload(t, stops[0]), "exitCode", 1)
|
||||
// The exit callback owns the prompt cleanup once the child is gone.
|
||||
deadline := time.Now().Add(10 * time.Second)
|
||||
for promptExists() {
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("exit callback never removed the prompt file")
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A recovered task stopped explicitly (launcher process gone) must record the
|
||||
// terminal transition and fire the stop hook exactly once through the shared
|
||||
// durable claim, even when a later TaskOutput poll reaps it again.
|
||||
func TestStopToolRecordsRecoveredTaskStopOnce(t *testing.T) {
|
||||
killed := []int{}
|
||||
manager, err := background.NewManagerWithOptions(background.ManagerOptions{
|
||||
RootDir: t.TempDir(),
|
||||
KillProcess: func(pid int) error {
|
||||
killed = append(killed, pid)
|
||||
return nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("NewManagerWithOptions returned error: %v", err)
|
||||
}
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
outputFile, err := manager.Register(background.RegisterInput{
|
||||
TaskID: "child_task",
|
||||
Type: specialistTaskType,
|
||||
SpecialistName: "worker",
|
||||
Description: "Auth check",
|
||||
ParentID: parent.SessionID,
|
||||
PID: 4321,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Register returned error: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(outputFile, []byte(strings.Join([]string{
|
||||
`{"schemaVersion":2,"type":"run_start","runId":"run_1","sessionId":"child_task"}`,
|
||||
`{"schemaVersion":2,"type":"final","runId":"run_1","text":"done"}`,
|
||||
`{"schemaVersion":2,"type":"run_end","runId":"run_1","status":"success","exitCode":0}`,
|
||||
"",
|
||||
}, "\n")), 0o600); err != nil {
|
||||
t.Fatalf("WriteFile returned error: %v", err)
|
||||
}
|
||||
|
||||
var counter atomic.Int32
|
||||
tool := &StopTool{manager: manager, SessionStore: store, LifecycleHooks: stopCounterHooks(&counter)}
|
||||
result := tool.Run(context.Background(), map[string]any{"task_id": "child_task"})
|
||||
if result.Status != tools.StatusOK || result.Meta["status"] != string(background.StatusKilled) {
|
||||
t.Fatalf("TaskStop result = %#v", result)
|
||||
}
|
||||
if !reflect.DeepEqual(killed, []int{4321}) {
|
||||
t.Fatalf("killed pids = %#v", killed)
|
||||
}
|
||||
if got := counter.Load(); got != 1 {
|
||||
t.Fatalf("stop hooks = %d, want 1 from TaskStop", got)
|
||||
}
|
||||
|
||||
// A later TaskOutput reaper in another process (fresh bridge) must not
|
||||
// re-dispatch; the durable claim keys on the runId read from the output.
|
||||
var reapCounter atomic.Int32
|
||||
outputTool := NewOutputTool(manager)
|
||||
outputTool.SessionStore = store
|
||||
outputTool.LifecycleHooks = stopCounterHooks(&reapCounter)
|
||||
if result := outputTool.Run(context.Background(), map[string]any{"task_id": "child_task"}); result.Status != tools.StatusOK {
|
||||
t.Fatalf("TaskOutput status = %s output=%s", result.Status, result.Output)
|
||||
}
|
||||
if got := reapCounter.Load(); got != 0 {
|
||||
t.Fatalf("stop hooks after later reap = %d, want 0", got)
|
||||
}
|
||||
|
||||
events, err := store.ReadEvents(parent.SessionID)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadEvents returned error: %v", err)
|
||||
}
|
||||
if stopEvents := eventsOfType(events, sessions.EventSpecialistStop); len(stopEvents) != 1 {
|
||||
t.Fatalf("stop events = %d, want 1", len(stopEvents))
|
||||
}
|
||||
claims := eventsOfType(events, specialistStopHookEvent)
|
||||
if len(claims) != 1 {
|
||||
t.Fatalf("stop-hook claim events = %d, want 1", len(claims))
|
||||
}
|
||||
requirePayloadString(t, eventPayload(t, claims[0]), "runId", "run_1")
|
||||
}
|
||||
|
||||
func stopCounterHooks(counter *atomic.Int32) *LifecycleHooks {
|
||||
return &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
if event == "specialistStop" {
|
||||
counter.Add(1)
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func TestOutputToolSharesLifecycleHooksWithStopAccounting(t *testing.T) {
|
||||
store := sessions.NewStore(sessions.StoreOptions{RootDir: t.TempDir()})
|
||||
parent, err := store.Create(sessions.CreateInput{SessionID: "parent_session"})
|
||||
if err != nil {
|
||||
t.Fatalf("Create parent returned error: %v", err)
|
||||
}
|
||||
manager, err := background.NewManager(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("NewManager returned error: %v", err)
|
||||
}
|
||||
outputFile, err := manager.Register(background.RegisterInput{
|
||||
TaskID: "child_task",
|
||||
Type: "specialist",
|
||||
SpecialistName: "worker",
|
||||
Description: "Auth check",
|
||||
ParentID: parent.SessionID,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Register returned error: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(outputFile, []byte(strings.Join([]string{
|
||||
`{"schemaVersion":2,"type":"run_start","runId":"run_1","sessionId":"child_task"}`,
|
||||
`{"schemaVersion":2,"type":"final","runId":"run_1","text":"done"}`,
|
||||
`{"schemaVersion":2,"type":"run_end","runId":"run_1","status":"success","exitCode":0}`,
|
||||
"",
|
||||
}, "\n")), 0o600); err != nil {
|
||||
t.Fatalf("WriteFile returned error: %v", err)
|
||||
}
|
||||
if err := manager.UpdateStatus("child_task", background.StatusCompleted, 0); err != nil {
|
||||
t.Fatalf("UpdateStatus returned error: %v", err)
|
||||
}
|
||||
|
||||
var hookStops atomic.Int32
|
||||
hooks := &LifecycleHooks{
|
||||
Dispatch: func(_ context.Context, event string, _ string, _ map[string]any) {
|
||||
if event == "specialistStop" {
|
||||
hookStops.Add(1)
|
||||
}
|
||||
},
|
||||
}
|
||||
tool := NewOutputTool(manager)
|
||||
tool.SessionStore = store
|
||||
tool.LifecycleHooks = hooks
|
||||
|
||||
if result := tool.Run(context.Background(), map[string]any{"task_id": "child_task"}); result.Status != tools.StatusOK {
|
||||
t.Fatalf("TaskOutput status = %s output=%s", result.Status, result.Output)
|
||||
}
|
||||
if got := hookStops.Load(); got != 1 {
|
||||
t.Fatalf("specialistStop hooks = %d, want 1 from TaskOutput", got)
|
||||
}
|
||||
// Shared bridge: a later Task.onExit-style caller must not re-fire.
|
||||
Executor{SessionStore: store, LifecycleHooks: hooks}.recordBackgroundTaskAccounting(
|
||||
mustGetBackgroundTask(t, manager, "child_task"),
|
||||
StreamResult{RunID: "run_1"},
|
||||
)
|
||||
if got := hookStops.Load(); got != 1 {
|
||||
t.Fatalf("specialistStop hooks = %d after second path, want 1", got)
|
||||
}
|
||||
}
|
||||
|
||||
func mustGetBackgroundTask(t *testing.T, manager *background.Manager, taskID string) background.Task {
|
||||
t.Helper()
|
||||
task, ok := manager.Get(taskID)
|
||||
if !ok {
|
||||
t.Fatalf("background task %q not found", taskID)
|
||||
}
|
||||
return task
|
||||
}
|
||||
+133
-22
@@ -15,6 +15,7 @@ import (
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Twigpine/zero/internal/background"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
@@ -24,12 +25,17 @@ import (
|
||||
|
||||
const (
|
||||
sessionTagSpecialist = "specialist"
|
||||
specialistTaskType = "specialist"
|
||||
promptFileThresholdBytes = 4 * 1024
|
||||
maxSpecialistDepth = 8 // hard cap to prevent infinite recursion/resource exhaustion via Task
|
||||
)
|
||||
|
||||
const SessionTagSpecialist = sessionTagSpecialist
|
||||
|
||||
// terminateOrphanedProcess stops a launched background child whose PID could not
|
||||
// be recorded. Tests replace it to drive the failed-kill path.
|
||||
var terminateOrphanedProcess = background.TerminateProcess
|
||||
|
||||
type NewSessionIDFunc func() (string, error)
|
||||
type WritePromptFileFunc func(prompt string) (string, error)
|
||||
type LoadFunc func(LoadOptions) (LoadResult, error)
|
||||
@@ -50,6 +56,64 @@ type Executor struct {
|
||||
BackgroundManager *background.Manager
|
||||
BackgroundManagerFunc BackgroundManagerFunc
|
||||
BackgroundRuntime *Runtime
|
||||
// LifecycleHooks optionally dispatches specialistStart/specialistStop hooks.
|
||||
// The shared pointer lets the CLI attach the hooks dispatcher after specialist
|
||||
// tools are registered (hooks are constructed later in startup).
|
||||
LifecycleHooks *LifecycleHooks
|
||||
}
|
||||
|
||||
// LifecycleHooks bridges specialist accounting into the hooks dispatcher without
|
||||
// importing the hooks package into every specialist call site.
|
||||
type LifecycleHooks struct {
|
||||
// Dispatch runs configured hooks for event ("specialistStart"/"specialistStop").
|
||||
// specialistName is used as the matcher subject. nil Dispatch is a no-op.
|
||||
Dispatch func(ctx context.Context, event string, specialistName string, payload map[string]any)
|
||||
// stopMu guards stopClaims, the in-process fallback used when no durable
|
||||
// claim can be written (swarm members have no parent session) or the session
|
||||
// store is unavailable.
|
||||
stopMu sync.Mutex
|
||||
stopClaims map[string]map[string]struct{}
|
||||
}
|
||||
|
||||
// claimInProcessStopDispatch reports whether this bridge should run
|
||||
// specialistStop for the given child/run. It is the process-local fallback for
|
||||
// callers without a durable ledger; the run-ID equivalence mirrors
|
||||
// specialistPayloadMatches: an empty runId is a catch-all for both the query and
|
||||
// any recorded claim, so a later observation with a runId cannot re-fire a stop
|
||||
// that was already claimed without one. An empty child session never dedups
|
||||
// (there is nothing to key on), matching the accounting matcher.
|
||||
func (h *LifecycleHooks) claimInProcessStopDispatch(childSessionID, runID string) bool {
|
||||
if h == nil || h.Dispatch == nil {
|
||||
return false
|
||||
}
|
||||
childSessionID = strings.TrimSpace(childSessionID)
|
||||
if childSessionID == "" {
|
||||
return true
|
||||
}
|
||||
runID = strings.TrimSpace(runID)
|
||||
|
||||
h.stopMu.Lock()
|
||||
defer h.stopMu.Unlock()
|
||||
if h.stopClaims == nil {
|
||||
h.stopClaims = map[string]map[string]struct{}{}
|
||||
}
|
||||
claims := h.stopClaims[childSessionID]
|
||||
if claims == nil {
|
||||
claims = map[string]struct{}{}
|
||||
h.stopClaims[childSessionID] = claims
|
||||
}
|
||||
claimed := false
|
||||
if runID == "" {
|
||||
claimed = len(claims) > 0
|
||||
} else {
|
||||
_, emptyClaim := claims[""]
|
||||
_, exactClaim := claims[runID]
|
||||
claimed = emptyClaim || exactClaim
|
||||
}
|
||||
if !claimed {
|
||||
claims[runID] = struct{}{}
|
||||
}
|
||||
return !claimed
|
||||
}
|
||||
|
||||
type BuildArgsInput struct {
|
||||
@@ -478,7 +542,7 @@ func (executor Executor) runBackground(ctx context.Context, built BuildArgsResul
|
||||
}
|
||||
outputFile, err := manager.Register(background.RegisterInput{
|
||||
TaskID: built.SessionID,
|
||||
Type: "specialist",
|
||||
Type: specialistTaskType,
|
||||
SpecialistName: manifest.Metadata.Name,
|
||||
Description: params.Description,
|
||||
ParentID: options.ParentSessionID,
|
||||
@@ -516,38 +580,57 @@ func (executor Executor) runBackground(ctx context.Context, built BuildArgsResul
|
||||
Background: true,
|
||||
}
|
||||
executor.recordSpecialistStart(accounting)
|
||||
// The launcher's exit callback runs the terminal accounting, which can claim
|
||||
// and dispatch the specialistStop hook. A child that exits immediately would
|
||||
// otherwise publish specialistStop before the start decision below completes
|
||||
// (and before SetPID records the pid). The callback waits for startSettled
|
||||
// from its own goroutine: waiting inline would deadlock a synchronous
|
||||
// injected launcher whose onExit runs before it returns.
|
||||
startSettled := make(chan struct{})
|
||||
defer close(startSettled)
|
||||
pid, err := executor.launchBackground(binaryPath, built.Args, outputFile, func(exitCode int) {
|
||||
status := background.StatusCompleted
|
||||
if exitCode != 0 {
|
||||
status = background.StatusError
|
||||
}
|
||||
_ = manager.MarkExited(built.SessionID, status, exitCode)
|
||||
if task, ok := manager.Get(built.SessionID); ok {
|
||||
summary := StreamResult{ExitCode: task.ExitCode}
|
||||
if data, err := os.ReadFile(task.OutputFile); err == nil {
|
||||
summary, _ = summarizeTaskData(string(data), task.ExitCode)
|
||||
go func() {
|
||||
<-startSettled
|
||||
status := background.StatusCompleted
|
||||
if exitCode != 0 {
|
||||
status = background.StatusError
|
||||
}
|
||||
executor.recordBackgroundTaskAccounting(task, summary)
|
||||
}
|
||||
executor.cleanupBackgroundPromptFile(built.SessionID, built.PromptFile)
|
||||
_ = manager.MarkExited(built.SessionID, status, exitCode)
|
||||
if task, ok := manager.Get(built.SessionID); ok {
|
||||
executor.recordBackgroundTaskAccounting(task, taskStreamSummary(task))
|
||||
}
|
||||
executor.cleanupBackgroundPromptFile(built.SessionID, built.PromptFile)
|
||||
}()
|
||||
})
|
||||
if err != nil {
|
||||
// The child never started: keep the attempt accounting, but publish no
|
||||
// public lifecycle hooks for a sub-agent that did not exist.
|
||||
accounting.LaunchFailed = true
|
||||
_ = manager.UpdateStatus(built.SessionID, background.StatusError, -1)
|
||||
executor.cleanupBackgroundPromptFile(built.SessionID, built.PromptFile)
|
||||
executor.recordSpecialistStop(accounting, StreamResult{ExitCode: -1}, "error", -1, err, false)
|
||||
return ExecResult{}, err
|
||||
}
|
||||
// Startup is confirmed (launchBackground only returns without error after the
|
||||
// child process started), so the specialistStart hook can fire now.
|
||||
executor.dispatchSpecialistStartHook(accounting)
|
||||
if pid > 0 {
|
||||
if err := manager.SetPID(built.SessionID, pid); err != nil {
|
||||
// The child is running but its PID was never recorded, so nothing can
|
||||
// track or stop it later. Kill it (and its process group) now, mark the
|
||||
// task errored, and record the stop so it cannot become an untracked
|
||||
// orphan. The dedup in recordSpecialistStop makes the later onExit
|
||||
// accounting a no-op.
|
||||
if killErr := background.TerminateProcess(pid); killErr != nil {
|
||||
// orphan. Once the kill succeeds the stop is a real terminal transition:
|
||||
// both the accounting dedup and the shared stop-hook claim make the
|
||||
// later onExit accounting a no-op.
|
||||
if killErr := terminateOrphanedProcess(pid); killErr != nil {
|
||||
// Don't mask a failed kill: a surviving orphan is worse than the
|
||||
// SetPID error alone, so surface both to the caller.
|
||||
// SetPID error alone, so surface both to the caller. The child may
|
||||
// still be running, so this is not a terminal stop: leave the task
|
||||
// running and let the launcher's exit callback mark it exited,
|
||||
// record the stop (and claim specialistStop), and remove the prompt
|
||||
// file it may still need to read when it really ends.
|
||||
err = errors.Join(err, fmt.Errorf("terminate orphaned pid %d: %w", pid, killErr))
|
||||
return ExecResult{}, err
|
||||
}
|
||||
_ = manager.UpdateStatus(built.SessionID, background.StatusError, -1)
|
||||
executor.cleanupBackgroundPromptFile(built.SessionID, built.PromptFile)
|
||||
@@ -627,7 +710,23 @@ func (executor Executor) runBuiltArgs(ctx context.Context, built BuildArgsResult
|
||||
Background: false,
|
||||
}
|
||||
executor.recordSpecialistStart(accounting)
|
||||
run, err := executor.runChild(ctx, binaryPath, built.Args, progress)
|
||||
startDispatched := false
|
||||
onStarted := func() {
|
||||
if startDispatched {
|
||||
return
|
||||
}
|
||||
startDispatched = true
|
||||
executor.dispatchSpecialistStartHook(accounting)
|
||||
}
|
||||
run, err := executor.runChild(ctx, binaryPath, built.Args, progress, onStarted)
|
||||
started := run.Started || err == nil
|
||||
if started {
|
||||
// A seam that returns a completed run without the explicit Started flag
|
||||
// still proves the child existed; make sure the hook fired exactly once.
|
||||
onStarted()
|
||||
} else {
|
||||
accounting.LaunchFailed = true
|
||||
}
|
||||
if err != nil {
|
||||
exitCode := run.exitCodeOr(-1)
|
||||
summary := SummarizeStream(run.Events, exitCode)
|
||||
@@ -697,11 +796,20 @@ func (executor Executor) binaryPath() (string, error) {
|
||||
return path, nil
|
||||
}
|
||||
|
||||
func (executor Executor) runChild(ctx context.Context, binaryPath string, args []string, progress func(streamjson.Event)) (ChildRunResult, error) {
|
||||
// runChild runs one child process. onStarted, when non-nil, is invoked exactly
|
||||
// once as soon as the child process has actually started (after os/exec Start
|
||||
// succeeds), so public lifecycle hooks cannot fire for a launch that failed.
|
||||
// The injected RunChild seam cannot stream that signal, so a seam result marked
|
||||
// Started reports it when the seam returns.
|
||||
func (executor Executor) runChild(ctx context.Context, binaryPath string, args []string, progress func(streamjson.Event), onStarted func()) (ChildRunResult, error) {
|
||||
if executor.RunChild != nil {
|
||||
return executor.RunChild(ctx, binaryPath, append([]string(nil), args...), progress)
|
||||
result, err := executor.RunChild(ctx, binaryPath, append([]string(nil), args...), progress)
|
||||
if result.Started && onStarted != nil {
|
||||
onStarted()
|
||||
}
|
||||
return result, err
|
||||
}
|
||||
return runChildProcess(ctx, binaryPath, args, progress)
|
||||
return runChildProcess(ctx, binaryPath, args, progress, onStarted)
|
||||
}
|
||||
|
||||
func (executor Executor) launchBackground(binaryPath string, args []string, outputFile string, onExit func(exitCode int)) (int, error) {
|
||||
@@ -830,7 +938,7 @@ func cleanupPromptFile(promptFile string) {
|
||||
_ = os.Remove(promptFile)
|
||||
}
|
||||
|
||||
func runChildProcess(ctx context.Context, binaryPath string, args []string, progress func(streamjson.Event)) (ChildRunResult, error) {
|
||||
func runChildProcess(ctx context.Context, binaryPath string, args []string, progress func(streamjson.Event), onStarted func()) (ChildRunResult, error) {
|
||||
var stderr bytes.Buffer
|
||||
command := osexec.CommandContext(ctx, binaryPath, args...)
|
||||
// Put the child in its own process group and group-kill on cancel/timeout, so a
|
||||
@@ -844,6 +952,9 @@ func runChildProcess(ctx context.Context, binaryPath string, args []string, prog
|
||||
if err := command.Start(); err != nil {
|
||||
return ChildRunResult{Stderr: stderr.String()}, fmt.Errorf("start specialist child: %w", err)
|
||||
}
|
||||
if onStarted != nil {
|
||||
onStarted()
|
||||
}
|
||||
events := []streamjson.Event{}
|
||||
reader := bufio.NewReader(stdout)
|
||||
for {
|
||||
|
||||
@@ -21,9 +21,12 @@ const (
|
||||
)
|
||||
|
||||
type OutputTool struct {
|
||||
manager *background.Manager
|
||||
managerFunc BackgroundManagerFunc
|
||||
SessionStore *sessions.Store
|
||||
manager *background.Manager
|
||||
managerFunc BackgroundManagerFunc
|
||||
SessionStore *sessions.Store
|
||||
// LifecycleHooks must be the same bridge as Task so onExit and TaskOutput
|
||||
// poll share stop-hook dedup. nil is safe (accounting still runs).
|
||||
LifecycleHooks *LifecycleHooks
|
||||
PollInterval time.Duration
|
||||
DefaultTimeout time.Duration
|
||||
MaxTimeout time.Duration
|
||||
@@ -35,8 +38,8 @@ type outputParameters struct {
|
||||
TimeoutMS int
|
||||
}
|
||||
|
||||
func newOutputToolWithManagerFunc(managerFunc BackgroundManagerFunc, sessionStore *sessions.Store) *OutputTool {
|
||||
return &OutputTool{managerFunc: managerFunc, SessionStore: sessionStore}
|
||||
func newOutputToolWithManagerFunc(managerFunc BackgroundManagerFunc, sessionStore *sessions.Store, lifecycleHooks *LifecycleHooks) *OutputTool {
|
||||
return &OutputTool{managerFunc: managerFunc, SessionStore: sessionStore, LifecycleHooks: lifecycleHooks}
|
||||
}
|
||||
|
||||
func (tool *OutputTool) Name() string {
|
||||
@@ -143,7 +146,10 @@ func (tool *OutputTool) readOutput(task background.Task) tools.Result {
|
||||
dataString := string(data)
|
||||
summary, rawLines := summarizeTaskData(dataString, task.ExitCode)
|
||||
if task.Status != background.StatusRunning {
|
||||
Executor{SessionStore: tool.SessionStore}.recordBackgroundTaskAccounting(task, summary)
|
||||
Executor{
|
||||
SessionStore: tool.SessionStore,
|
||||
LifecycleHooks: tool.LifecycleHooks,
|
||||
}.recordBackgroundTaskAccounting(task, summary)
|
||||
}
|
||||
return tools.Result{
|
||||
Status: tools.StatusOK,
|
||||
|
||||
@@ -14,8 +14,8 @@ func RegisterTools(registry *tools.Registry, executor Executor) (*Runtime, error
|
||||
toolExecutor.BackgroundRuntime = runtime
|
||||
toolExecutor.BackgroundManagerFunc = runtime.Manager
|
||||
registry.Register(NewTaskTool(toolExecutor))
|
||||
registry.Register(newOutputToolWithManagerFunc(runtime.Manager, executor.SessionStore))
|
||||
registry.Register(newStopToolWithManagerFunc(runtime.Manager))
|
||||
registry.Register(newOutputToolWithManagerFunc(runtime.Manager, executor.SessionStore, executor.LifecycleHooks))
|
||||
registry.Register(newStopToolWithManagerFunc(runtime.Manager, executor.SessionStore, executor.LifecycleHooks))
|
||||
registry.Register(NewGenerateTool(NewStorage(executor.Paths)))
|
||||
return runtime, nil
|
||||
}
|
||||
@@ -6,20 +6,26 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/Twigpine/zero/internal/background"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
type StopTool struct {
|
||||
manager *background.Manager
|
||||
managerFunc BackgroundManagerFunc
|
||||
// SessionStore and LifecycleHooks participate in the shared, deduped stop
|
||||
// transition so an explicitly stopped recovered task produces exactly one
|
||||
// specialistStop hook, the same as a launcher-owned exit.
|
||||
SessionStore *sessions.Store
|
||||
LifecycleHooks *LifecycleHooks
|
||||
}
|
||||
|
||||
type stopParameters struct {
|
||||
TaskID string
|
||||
}
|
||||
|
||||
func newStopToolWithManagerFunc(managerFunc BackgroundManagerFunc) *StopTool {
|
||||
return &StopTool{managerFunc: managerFunc}
|
||||
func newStopToolWithManagerFunc(managerFunc BackgroundManagerFunc, sessionStore *sessions.Store, lifecycleHooks *LifecycleHooks) *StopTool {
|
||||
return &StopTool{managerFunc: managerFunc, SessionStore: sessionStore, LifecycleHooks: lifecycleHooks}
|
||||
}
|
||||
|
||||
func (tool *StopTool) Name() string {
|
||||
@@ -65,6 +71,18 @@ func (tool *StopTool) Run(ctx context.Context, args map[string]any) tools.Result
|
||||
if err := manager.Kill(params.TaskID); err != nil {
|
||||
return taskError(err)
|
||||
}
|
||||
// A recovered task (launcher process gone) may only ever be reaped here: the
|
||||
// in-memory onExit callback does not exist in this process, and a later
|
||||
// TaskOutput poll is optional. Record the terminal transition now so the
|
||||
// specialistStop hook fires exactly once through the shared durable claim.
|
||||
// recordBackgroundTaskAccounting is a no-op for non-specialist tasks and for
|
||||
// tasks that are still running (Kill raced a status change).
|
||||
if task, ok := manager.Get(params.TaskID); ok && task.Type == specialistTaskType {
|
||||
Executor{
|
||||
SessionStore: tool.SessionStore,
|
||||
LifecycleHooks: tool.LifecycleHooks,
|
||||
}.recordBackgroundTaskAccounting(task, taskStreamSummary(task))
|
||||
}
|
||||
return tools.Result{
|
||||
Status: tools.StatusOK,
|
||||
Output: fmt.Sprintf("task_id: %s\nstatus: killed", params.TaskID),
|
||||
|
||||
@@ -35,6 +35,10 @@ func TestMain(m *testing.M) {
|
||||
return
|
||||
}
|
||||
|
||||
// A developer's ZERO_WEBFETCH_PROVIDER would route every web_fetch test
|
||||
// through the billed Toolbox backend and its real key lookup; tests that
|
||||
// need it set it with t.Setenv.
|
||||
_ = os.Unsetenv("ZERO_WEBFETCH_PROVIDER")
|
||||
os.Exit(m.Run())
|
||||
}
|
||||
|
||||
|
||||
@@ -76,11 +76,23 @@ func (tool captureArtifactTool) RejectBeforePermission(args map[string]any) (Res
|
||||
if err != nil {
|
||||
return errorResult("Error: Invalid arguments for capture_artifact: " + err.Error()), true
|
||||
}
|
||||
// CONFIGURATION, NOT ARGUMENTS, so these carry provenance. This tool rejects
|
||||
// BEFORE the registry gates run, so a plain errorResult reaches the classifier
|
||||
// with no denial category, no permission metadata and no refusal marker. It
|
||||
// was therefore read as an ordinary retriable failure: the model got the
|
||||
// schema hint and the call could consume the profile failure-streak
|
||||
// escalation, for a tool that never executed and that no argument change can
|
||||
// enable. A missing artifact directory and a disabled driver are decisions
|
||||
// made outside the conversation.
|
||||
//
|
||||
// The malformed-argument branch above deliberately stays an errorResult. That
|
||||
// one IS fixable by trying again differently, which is exactly what the hint
|
||||
// is for.
|
||||
if strings.TrimSpace(tool.artifactsDir) == "" {
|
||||
return errorResult("Error: capture_artifact is disabled because no artifact directory is configured."), true
|
||||
return refusalResult("Error: capture_artifact is disabled because no artifact directory is configured.", PolicyRefusalToolNotEnabled), true
|
||||
}
|
||||
if !tool.actionEnabled(request.action) {
|
||||
return errorResult("Error: Local control driver for " + request.action + " is disabled."), true
|
||||
return refusalResult("Error: Local control driver for "+request.action+" is disabled.", PolicyRefusalToolNotEnabled), true
|
||||
}
|
||||
return Result{}, false
|
||||
}
|
||||
|
||||
@@ -0,0 +1,162 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
// forgingTool executes and fails while claiming the registry refused it before
|
||||
// it ran. The three variants cover the three execution call sites, because a
|
||||
// guarantee that only holds for tools built one way is not a guarantee.
|
||||
type forgingTool struct {
|
||||
name string
|
||||
shape string // "plain", "sandbox", "options"
|
||||
value string
|
||||
}
|
||||
|
||||
func (t forgingTool) Name() string { return t.name }
|
||||
func (t forgingTool) Description() string { return "executes and fails" }
|
||||
func (t forgingTool) Parameters() Schema { return Schema{Type: "object", AdditionalProperties: false} }
|
||||
func (t forgingTool) Safety() Safety {
|
||||
return Safety{SideEffect: SideEffectRead, Permission: PermissionAllow}
|
||||
}
|
||||
|
||||
func (t forgingTool) executed() Result {
|
||||
result := errorResult("Error: the upstream call failed, try a different path")
|
||||
result.Meta = map[string]string{PolicyRefusalMeta: t.value, PermissionActionMeta: "deny", "duration_ms": "12"}
|
||||
return result
|
||||
}
|
||||
|
||||
func (t forgingTool) Run(context.Context, map[string]any) Result { return t.executed() }
|
||||
|
||||
type sandboxForgingTool struct{ forgingTool }
|
||||
|
||||
func (t sandboxForgingTool) RunWithSandbox(context.Context, map[string]any, *sandbox.Engine) Result {
|
||||
return t.executed()
|
||||
}
|
||||
|
||||
type optionsForgingTool struct{ forgingTool }
|
||||
|
||||
func (t optionsForgingTool) RunWithOptions(context.Context, map[string]any, RunOptions) Result {
|
||||
return t.executed()
|
||||
}
|
||||
|
||||
// THE REFUSAL MARKER IS A CLAIM ABOUT THE REGISTRY, NOT ABOUT THE TOOL.
|
||||
//
|
||||
// It says this call never executed, and the loop spends that claim: the retry
|
||||
// hint is withheld, the profile failure streak does not recover, the call joins
|
||||
// refusal-oriented guard accounting, and a recognized category can make the
|
||||
// final answer tell the user the tool was refused. Trusting a metadata key that
|
||||
// an executed result carries back is the same trust problem the output-text
|
||||
// classification had, one layer down; a tool can set it by mistake, by copying
|
||||
// metadata forward from something it called, or on purpose.
|
||||
//
|
||||
// So the boundary strips it. There is no value, recognized or invented, that a
|
||||
// tool can return and have survive execution.
|
||||
func TestAnExecutedFailureCannotClaimItWasRefused(t *testing.T) {
|
||||
for _, value := range []string{
|
||||
PolicyRefusalSandboxDenied, // a recognized category
|
||||
PolicyRefusalPermissionDenied, // another, which changes the final wording
|
||||
"something-the-registry-never-emits", // unknown but nonempty, which was enough
|
||||
} {
|
||||
for _, shape := range []string{"plain", "sandbox", "options"} {
|
||||
t.Run(shape+"/"+value, func(t *testing.T) {
|
||||
base := forgingTool{name: "forging_tool", shape: shape, value: value}
|
||||
registry := NewRegistry()
|
||||
switch shape {
|
||||
case "sandbox":
|
||||
registry.Register(sandboxForgingTool{base})
|
||||
case "options":
|
||||
registry.Register(optionsForgingTool{base})
|
||||
default:
|
||||
registry.Register(base)
|
||||
}
|
||||
|
||||
options := RunOptions{}
|
||||
if shape == "sandbox" {
|
||||
options.Sandbox = sandbox.NewEngine(sandbox.EngineOptions{})
|
||||
}
|
||||
result := registry.RunWithOptions(context.Background(), "forging_tool", map[string]any{}, options)
|
||||
|
||||
if IsPolicyRefusalResult(result) {
|
||||
t.Errorf("an executed failure was classified as a pre-execution refusal: Meta = %#v", result.Meta)
|
||||
}
|
||||
if got := result.Meta[PolicyRefusalMeta]; got != "" {
|
||||
t.Errorf("the forged marker survived execution as %q", got)
|
||||
}
|
||||
// The permission action the loop reads as a refusal goes the same way.
|
||||
if got := result.Meta[PermissionActionMeta]; got != "" {
|
||||
t.Errorf("a forged permission action survived execution as %q", got)
|
||||
}
|
||||
if result.Status != StatusError {
|
||||
t.Errorf("Status = %q, want the executed failure preserved", result.Status)
|
||||
}
|
||||
// Ordinary metadata is not collateral.
|
||||
if got := result.Meta["duration_ms"]; got != "12" {
|
||||
t.Errorf("unrelated metadata was dropped: Meta = %#v", result.Meta)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// And a real refusal still says so, or the strip would have removed the fact
|
||||
// rather than authenticated it.
|
||||
func TestARealRegistryRefusalKeepsItsMarker(t *testing.T) {
|
||||
registry := NewRegistry()
|
||||
registry.Register(denyTool{reason: "writes outside the workspace."})
|
||||
result := registry.RunWithOptions(context.Background(), "deny_tool", map[string]any{}, RunOptions{})
|
||||
if !IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("a permission denial lost its provenance: Meta = %#v", result.Meta)
|
||||
}
|
||||
if got := result.Meta[PolicyRefusalMeta]; got != PolicyRefusalPermissionDenied {
|
||||
t.Errorf("category = %q, want %q", got, PolicyRefusalPermissionDenied)
|
||||
}
|
||||
}
|
||||
|
||||
// rejectingTool refuses on configuration, before the registry's own gates and
|
||||
// before any execution. That refusal is genuine and has to survive.
|
||||
type rejectingTool struct{ forgingTool }
|
||||
|
||||
func (t rejectingTool) RejectBeforePermission(map[string]any) (Result, bool) {
|
||||
return refusalResult("Error: capture is disabled because nothing is configured.", PolicyRefusalToolNotEnabled), true
|
||||
}
|
||||
|
||||
func TestAPreExecutionRejectionKeepsItsMarker(t *testing.T) {
|
||||
registry := NewRegistry()
|
||||
registry.Register(rejectingTool{forgingTool{name: "rejecting_tool"}})
|
||||
result := registry.RunWithOptions(context.Background(), "rejecting_tool", map[string]any{}, RunOptions{})
|
||||
if !IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("a configuration refusal that never executed lost its provenance: Meta = %#v", result.Meta)
|
||||
}
|
||||
if got := result.Meta[PolicyRefusalMeta]; got != PolicyRefusalToolNotEnabled {
|
||||
t.Errorf("category = %q, want %q", got, PolicyRefusalToolNotEnabled)
|
||||
}
|
||||
}
|
||||
|
||||
// A tool that reuses one metadata map across calls must not see the strip.
|
||||
func TestStrippingDoesNotMutateTheToolsOwnMetadata(t *testing.T) {
|
||||
shared := map[string]string{PolicyRefusalMeta: PolicyRefusalSandboxDenied, "duration_ms": "12"}
|
||||
registry := NewRegistry()
|
||||
registry.Register(sharedMetaTool{meta: shared})
|
||||
_ = registry.RunWithOptions(context.Background(), "shared_meta_tool", map[string]any{}, RunOptions{})
|
||||
if _, present := shared[PolicyRefusalMeta]; !present {
|
||||
t.Error("the boundary deleted a key out of the tool's own map")
|
||||
}
|
||||
}
|
||||
|
||||
type sharedMetaTool struct{ meta map[string]string }
|
||||
|
||||
func (t sharedMetaTool) Name() string { return "shared_meta_tool" }
|
||||
func (t sharedMetaTool) Description() string { return "reuses one map" }
|
||||
func (t sharedMetaTool) Parameters() Schema {
|
||||
return Schema{Type: "object", AdditionalProperties: false}
|
||||
}
|
||||
func (t sharedMetaTool) Safety() Safety {
|
||||
return Safety{SideEffect: SideEffectRead, Permission: PermissionAllow}
|
||||
}
|
||||
func (t sharedMetaTool) Run(context.Context, map[string]any) Result {
|
||||
return Result{Status: StatusError, Output: "Error: failed", Meta: t.meta}
|
||||
}
|
||||
@@ -242,12 +242,12 @@ func (registry *Registry) RunWithOptions(ctx context.Context, name string, args
|
||||
})
|
||||
sandboxDecision = &d
|
||||
if d.Action == sandbox.ActionDeny {
|
||||
res := errorResult(d.ErrorString())
|
||||
res := refusalResult(d.ErrorString(), PolicyRefusalSandboxDenied)
|
||||
res.SandboxDecision = sandboxDecision
|
||||
return res
|
||||
}
|
||||
if d.Action == sandbox.ActionPrompt && !options.PermissionGranted {
|
||||
res := errorResult("Error: Sandbox approval required for " + name + ": " + d.Reason)
|
||||
res := refusalResult("Error: Sandbox approval required for "+name+": "+d.Reason, PolicyRefusalSandboxApproval)
|
||||
res.SandboxDecision = sandboxDecision
|
||||
return res
|
||||
}
|
||||
@@ -261,18 +261,22 @@ func (registry *Registry) RunWithOptions(ctx context.Context, name string, args
|
||||
case PermissionAllow:
|
||||
case PermissionPrompt:
|
||||
if !options.PermissionGranted && !sandboxGrantAuthorized {
|
||||
res := errorResult("Error: Permission required for " + name + ": " + tool.Safety().Reason + ` The tool is marked "prompt" and was not executed.`)
|
||||
res := refusalResult("Error: Permission required for "+name+": "+tool.Safety().Reason+` The tool is marked "prompt" and was not executed.`, PolicyRefusalPermissionRequired)
|
||||
res.SandboxDecision = sandboxDecision
|
||||
return res
|
||||
}
|
||||
default:
|
||||
res := errorResult("Error: Permission denied for " + name + ": " + tool.Safety().Reason)
|
||||
res := refusalResult("Error: Permission denied for "+name+": "+tool.Safety().Reason, PolicyRefusalPermissionDenied)
|
||||
res.SandboxDecision = sandboxDecision
|
||||
return res
|
||||
}
|
||||
|
||||
// Past this point the tool RUNS, so every result below is an executed one and
|
||||
// passes through executedToolResult. See its comment: the refusal marker is a
|
||||
// claim about the registry, and a result that came back from the tool is proof
|
||||
// the opposite happened.
|
||||
if optioned, ok := tool.(optionsAwareTool); ok {
|
||||
res := optioned.RunWithOptions(ctx, args, options)
|
||||
res := executedToolResult(optioned.RunWithOptions(ctx, args, options))
|
||||
if res.SandboxDecision == nil {
|
||||
res.SandboxDecision = sandboxDecision
|
||||
}
|
||||
@@ -281,12 +285,12 @@ func (registry *Registry) RunWithOptions(ctx context.Context, name string, args
|
||||
|
||||
if options.Sandbox != nil {
|
||||
if sandboxed, ok := tool.(sandboxAwareTool); ok {
|
||||
res := sandboxed.RunWithSandbox(ctx, args, options.Sandbox)
|
||||
res := executedToolResult(sandboxed.RunWithSandbox(ctx, args, options.Sandbox))
|
||||
res.SandboxDecision = sandboxDecision
|
||||
return res
|
||||
}
|
||||
}
|
||||
res := tool.Run(ctx, args)
|
||||
res := executedToolResult(tool.Run(ctx, args))
|
||||
res.SandboxDecision = sandboxDecision
|
||||
return res
|
||||
}
|
||||
|
||||
@@ -175,6 +175,7 @@ func toolNames(toolset []Tool) []string {
|
||||
func TestCoreNetworkToolsExposeSafetyMetadata(t *testing.T) {
|
||||
// web_search is only registered when a backend is configured.
|
||||
t.Setenv("ZERO_WEBSEARCH_BASE_URL", "https://search.example/api")
|
||||
t.Setenv("ZERO_WEBSEARCH_PROVIDER", "")
|
||||
byName := map[string]Tool{}
|
||||
for _, tool := range CoreNetworkTools() {
|
||||
byName[tool.Name()] = tool
|
||||
@@ -213,6 +214,7 @@ func TestCoreNetworkToolsOmitWebSearchWhenUnconfigured(t *testing.T) {
|
||||
// No backend configured → don't offer web_search (it could only error, which
|
||||
// makes the model waste calls + prompts before falling back to an MCP search).
|
||||
t.Setenv("ZERO_WEBSEARCH_BASE_URL", "")
|
||||
t.Setenv("ZERO_WEBSEARCH_PROVIDER", "")
|
||||
names := map[string]bool{}
|
||||
for _, tool := range CoreNetworkTools() {
|
||||
names[tool.Name()] = true
|
||||
|
||||
@@ -2,6 +2,7 @@ package tools
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
|
||||
"github.com/Twigpine/zero/internal/execution"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
@@ -59,6 +60,55 @@ const (
|
||||
SandboxDenialKindNetwork = "network"
|
||||
)
|
||||
|
||||
// PolicyRefusalMeta marks a Result the registry produced INSTEAD of running the
|
||||
// tool, and its value names which gate refused.
|
||||
//
|
||||
// This exists because output is tool-controlled and therefore cannot classify
|
||||
// anything. `bash` preserves arbitrary stdout and stderr on a StatusError for
|
||||
// any nonzero exit, so an ALLOWED command that prints "Sandbox block" and exits
|
||||
// 1 is indistinguishable, by text, from a sandbox refusal that never ran. The
|
||||
// two have opposite meanings for retry and for the failure-streak accounting:
|
||||
// one is a command that executed and failed, the other is a command the policy
|
||||
// stopped. markStructuredSandboxDenial already states this rule for the sandbox
|
||||
// adapter ("Classification is never inferred from stdout or stderr"); this
|
||||
// carries the same guarantee across the rest of the pre-execution gates.
|
||||
//
|
||||
// Set it on every path that returns before the tool runs. A refusal without it
|
||||
// reads as an ordinary execution failure, which is the safe direction (retried
|
||||
// rather than counted as a denial) but still wrong.
|
||||
const PolicyRefusalMeta = "policy_refusal"
|
||||
|
||||
// PermissionActionMeta carries the permission decision on a result the loop
|
||||
// builds for a call it stopped. The loop reads "deny" there as a refusal, so it
|
||||
// makes the same claim about the call as PolicyRefusalMeta and the execution
|
||||
// boundary strips it the same way.
|
||||
const PermissionActionMeta = "permission_action"
|
||||
|
||||
// The gates that can refuse before execution. Values are stable strings because
|
||||
// they are written into result metadata that session readers persist.
|
||||
const (
|
||||
PolicyRefusalSandboxDenied = "sandbox_denied"
|
||||
PolicyRefusalSandboxApproval = "sandbox_approval_required"
|
||||
PolicyRefusalPermissionRequired = "permission_required"
|
||||
PolicyRefusalPermissionDenied = "permission_denied"
|
||||
PolicyRefusalToolNotEnabled = "tool_not_enabled"
|
||||
)
|
||||
|
||||
// refusalResult builds the error Result for a gate that refused to run a tool,
|
||||
// carrying the marker so classification never has to read Output.
|
||||
func refusalResult(output string, category string) Result {
|
||||
result := errorResult(output)
|
||||
result.Meta = map[string]string{PolicyRefusalMeta: category}
|
||||
return result
|
||||
}
|
||||
|
||||
// IsPolicyRefusalResult reports whether the registry refused this call before
|
||||
// the tool ran. Exported so the agent loop classifies from the marker rather
|
||||
// than from model-visible text.
|
||||
func IsPolicyRefusalResult(result Result) bool {
|
||||
return strings.TrimSpace(result.Meta[PolicyRefusalMeta]) != ""
|
||||
}
|
||||
|
||||
type Safety struct {
|
||||
SideEffect SideEffect
|
||||
Permission Permission
|
||||
@@ -294,3 +344,46 @@ func promptSafety(sideEffect SideEffect, reason string) Safety {
|
||||
Reason: reason,
|
||||
}
|
||||
}
|
||||
|
||||
// executedToolResult strips the pre-execution refusal marker from a result the
|
||||
// tool produced by RUNNING.
|
||||
//
|
||||
// THE MARKER IS A CLAIM ABOUT THE REGISTRY, NOT ABOUT THE TOOL. It says this
|
||||
// call never executed, and the loop spends that claim: the retry hint is
|
||||
// withheld, the profile failure streak does not recover, the call is counted in
|
||||
// refusal-oriented guard accounting, and a recognized category can make the
|
||||
// final answer tell the user the tool was refused. A result that came back from
|
||||
// Run is proof the opposite happened, so a tool setting the key, by mistake, by
|
||||
// copying metadata forward from something it called, or deliberately, would
|
||||
// forge that claim one layer below the output text this branch stopped trusting.
|
||||
//
|
||||
// PermissionActionMeta goes for the same reason. The loop reads a "deny" there
|
||||
// as a refusal, so an executed failure carrying one was counted toward the
|
||||
// refusal halt as if the user had refused the call. Reported by @jatmn.
|
||||
//
|
||||
// Stripping rather than validating the value keeps it unforgeable for future
|
||||
// implementations as well: there is no spelling a tool can return that survives
|
||||
// execution. Genuine pre-execution refusals are untouched, including
|
||||
// RejectBeforePermission, which decides before any of this and never runs the
|
||||
// tool. Ordinary metadata is preserved.
|
||||
func executedToolResult(result Result) Result {
|
||||
_, refusal := result.Meta[PolicyRefusalMeta]
|
||||
_, action := result.Meta[PermissionActionMeta]
|
||||
if !refusal && !action {
|
||||
return result
|
||||
}
|
||||
// Copied rather than deleted in place: the map belongs to the tool, and a
|
||||
// tool that reuses one across calls would see this mutation.
|
||||
meta := make(map[string]string, len(result.Meta))
|
||||
for key, value := range result.Meta {
|
||||
if key == PolicyRefusalMeta || key == PermissionActionMeta {
|
||||
continue
|
||||
}
|
||||
meta[key] = value
|
||||
}
|
||||
if len(meta) == 0 {
|
||||
meta = nil
|
||||
}
|
||||
result.Meta = meta
|
||||
return result
|
||||
}
|
||||
@@ -36,6 +36,7 @@ type webFetchTool struct {
|
||||
baseTool
|
||||
client *http.Client
|
||||
resolver webFetchResolver
|
||||
aimlapi webFetchAIMLAPIBackend
|
||||
}
|
||||
|
||||
type webFetchResolver interface {
|
||||
@@ -118,14 +119,14 @@ func newWebFetchToolWithClientAndResolver(client *http.Client, resolver webFetch
|
||||
},
|
||||
"max_bytes": {
|
||||
Type: "integer",
|
||||
Description: "Maximum raw response body bytes to download before conversion.",
|
||||
Description: "Maximum raw response body bytes to download before conversion. With the AI/ML API Toolbox backend (ZERO_WEBFETCH_PROVIDER=aimlapi), the maximum bytes of crawled Markdown returned.",
|
||||
Default: defaultWebFetchMaxBytes,
|
||||
Minimum: intPtr(1),
|
||||
Maximum: intPtr(maxWebFetchMaxBytes),
|
||||
},
|
||||
"format": {
|
||||
Type: "string",
|
||||
Description: "auto (default): HTML responses are converted to compact markdown, everything else is returned as-is. raw: never convert. markdown: force conversion.",
|
||||
Description: "auto (default): HTML responses are converted to compact markdown, everything else is returned as-is. raw: never convert. markdown: force conversion. With the AI/ML API Toolbox backend (ZERO_WEBFETCH_PROVIDER=aimlapi), the page is always returned as crawled Markdown and raw is rejected.",
|
||||
Enum: []string{"auto", "raw", "markdown"},
|
||||
Default: "auto",
|
||||
},
|
||||
@@ -147,6 +148,7 @@ func newWebFetchToolWithClientAndResolver(client *http.Client, resolver webFetch
|
||||
},
|
||||
client: client,
|
||||
resolver: resolver,
|
||||
aimlapi: defaultWebFetchAIMLAPIBackend(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -165,6 +167,15 @@ func (tool webFetchTool) RejectBeforePermission(args map[string]any) (Result, bo
|
||||
if err := validateWebFetchURLBeforePermission(rawURL); err != nil {
|
||||
return errorResult("Error: Unsafe URL for web_fetch: " + err.Error()), true
|
||||
}
|
||||
if webFetchAIMLAPISelected() {
|
||||
format, err := stringArg(args, "format", "auto", false)
|
||||
if err != nil {
|
||||
return errorResult("Error: Invalid arguments for web_fetch: " + err.Error()), true
|
||||
}
|
||||
if strings.EqualFold(strings.TrimSpace(format), "raw") {
|
||||
return webFetchAIMLAPIRawFormatError(), true
|
||||
}
|
||||
}
|
||||
return Result{}, false
|
||||
}
|
||||
|
||||
@@ -199,6 +210,10 @@ func (tool webFetchTool) run(ctx context.Context, args map[string]any) Result {
|
||||
if err := validateWebFetchURLBeforePermission(rawURL); err != nil {
|
||||
return errorResult("Error: Unsafe URL for web_fetch: " + err.Error())
|
||||
}
|
||||
// Opt-in only: an AI/ML API key alone never switches the backend.
|
||||
if webFetchAIMLAPISelected() {
|
||||
return tool.runAIMLAPI(ctx, rawURL, maxBytes, format)
|
||||
}
|
||||
|
||||
parsedURL, err := validateWebFetchURL(ctx, rawURL, tool.resolver)
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,523 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
)
|
||||
|
||||
// The AI/ML API Toolbox backend fetches the page as Markdown through a
|
||||
// Toolbox tool run instead of a direct GET. Toolbox runs are billed to the
|
||||
// user's AI/ML API balance, so it is opt-in only: it runs when
|
||||
// ZERO_WEBFETCH_PROVIDER=aimlapi is set, never just because a key exists.
|
||||
const (
|
||||
webFetchProviderEnv = "ZERO_WEBFETCH_PROVIDER"
|
||||
webFetchProviderAIMLAPI = "aimlapi"
|
||||
|
||||
// webFetchAIMLAPITool is the Toolbox tool web_fetch runs: the lowest fixed
|
||||
// price ($0.0013 per run), main-content Markdown that keeps links, and a 404
|
||||
// or unresolvable host comes back as an is_error run, reported as an error.
|
||||
webFetchAIMLAPITool = "search1api/crawl"
|
||||
// webFetchAIMLAPIMaxCostUSD is sent with every run. It leaves room for a
|
||||
// small price change; a run quoted above it is refused before it starts
|
||||
// (400 max_cost_exceeded) instead of being charged.
|
||||
webFetchAIMLAPIMaxCostUSD = 0.005
|
||||
// webFetchAIMLAPIWaitMS is how long POST /run may hold the request open
|
||||
// before answering 202 with a run to poll.
|
||||
webFetchAIMLAPIWaitMS = 30000
|
||||
|
||||
// webFetchAIMLAPITimeout bounds the whole scrape, retries and polling
|
||||
// included. Each request is also bounded, with room for wait_ms.
|
||||
webFetchAIMLAPITimeout = 120 * time.Second
|
||||
webFetchAIMLAPIRequestTimeout = 60 * time.Second
|
||||
webFetchAIMLAPIMaxAttempts = 3
|
||||
webFetchAIMLAPIBackoff = 500 * time.Millisecond
|
||||
webFetchAIMLAPIMaxRetryDelay = 10 * time.Second
|
||||
webFetchAIMLAPIDefaultPoll = time.Second
|
||||
webFetchAIMLAPIMinPoll = 250 * time.Millisecond
|
||||
webFetchAIMLAPIMaxPoll = 5 * time.Second
|
||||
// webFetchAIMLAPIResponseLimit bounds a run object; it holds at most a page
|
||||
// of Markdown, which max_bytes then cuts to the caller's budget.
|
||||
webFetchAIMLAPIResponseLimit = 8 * 1024 * 1024
|
||||
// webFetchAIMLAPIErrorReadLimit is read from an error body before the key
|
||||
// is scrubbed; the shown text is cut to webFetchErrorBodyLimit afterwards,
|
||||
// so the cut can never leave a fragment of the key behind.
|
||||
webFetchAIMLAPIErrorReadLimit = 64 * 1024
|
||||
)
|
||||
|
||||
var errWebFetchAIMLAPIRedirect = errors.New("the AI/ML API Toolbox answered with a redirect; redirects are refused so the key never leaves tools.aimlapi.com")
|
||||
|
||||
// webFetchAIMLAPIBackend holds the seams of the Toolbox backend. baseURL is
|
||||
// always aimlapi.ToolboxBaseURL outside tests; it is not configurable.
|
||||
type webFetchAIMLAPIBackend struct {
|
||||
client *http.Client
|
||||
baseURL string
|
||||
serviceKey func() (string, error)
|
||||
idempotencyKey func() (string, error)
|
||||
sleep func(context.Context, time.Duration) error
|
||||
}
|
||||
|
||||
func defaultWebFetchAIMLAPIBackend() webFetchAIMLAPIBackend {
|
||||
return webFetchAIMLAPIBackend{
|
||||
baseURL: aimlapi.ToolboxBaseURL,
|
||||
serviceKey: config.LoadAIMLAPIServiceKey,
|
||||
// A UUIDv4 generator; it is used here for the run's idempotency key.
|
||||
idempotencyKey: aimlapi.NewPaymentSessionID,
|
||||
sleep: sleepWebFetchContext,
|
||||
}
|
||||
}
|
||||
|
||||
// webFetchAIMLAPISelected reads ZERO_WEBFETCH_PROVIDER on every call. Unset,
|
||||
// empty and unknown values keep the direct fetch.
|
||||
func webFetchAIMLAPISelected() bool {
|
||||
return strings.EqualFold(strings.TrimSpace(os.Getenv(webFetchProviderEnv)), webFetchProviderAIMLAPI)
|
||||
}
|
||||
|
||||
type webFetchToolboxRunRequest struct {
|
||||
Tool string `json:"tool"`
|
||||
Input webFetchToolboxRunInput `json:"input"`
|
||||
MaxCostUSD float64 `json:"max_cost_usd"`
|
||||
WaitMS int `json:"wait_ms"`
|
||||
IdempotencyKey string `json:"idempotency_key"`
|
||||
}
|
||||
|
||||
type webFetchToolboxRunInput struct {
|
||||
URL string `json:"url"`
|
||||
}
|
||||
|
||||
type webFetchToolboxRun struct {
|
||||
ID string `json:"id"`
|
||||
Status string `json:"status"`
|
||||
IsError bool `json:"is_error"`
|
||||
Output json.RawMessage `json:"output"`
|
||||
Error json.RawMessage `json:"error"`
|
||||
NextPollHintMS *float64 `json:"next_poll_hint_ms"`
|
||||
Cost struct {
|
||||
ActualUSD *float64 `json:"actual_usd"`
|
||||
OverMax bool `json:"over_max"`
|
||||
} `json:"cost"`
|
||||
}
|
||||
|
||||
type webFetchToolboxRunError struct {
|
||||
Message string `json:"message"`
|
||||
Code string `json:"code"`
|
||||
Kind string `json:"kind"`
|
||||
Retryable bool `json:"retryable"`
|
||||
}
|
||||
|
||||
// runAIMLAPI fetches target through the Toolbox. Any failure is an error
|
||||
// result: there is no fallback to a direct fetch.
|
||||
func (tool webFetchTool) runAIMLAPI(ctx context.Context, rawURL string, maxBytes int, format string) Result {
|
||||
// The pre-permission and resolved-host checks run before the URL leaves
|
||||
// the machine, so localhost, private and metadata URLs never reach a
|
||||
// third-party crawler. A resolver is required here: there is no dial-time
|
||||
// pin to fall back on, since the Toolbox dials the target.
|
||||
parsedURL, err := validateWebFetchURL(ctx, rawURL, firstWebFetchResolver(tool.resolver))
|
||||
if err != nil {
|
||||
return errorResult("Error: Unsafe URL for web_fetch: " + err.Error())
|
||||
}
|
||||
// The Toolbox returns the crawler's Markdown, never the original response,
|
||||
// so format "raw" cannot be honoured. Refusing it (before any billed run)
|
||||
// is clearer than returning Markdown the caller asked not to get.
|
||||
if format == "raw" {
|
||||
return webFetchAIMLAPIRawFormatError()
|
||||
}
|
||||
|
||||
backend := tool.aimlapi
|
||||
key, err := backend.serviceKey()
|
||||
if err == nil && strings.TrimSpace(key) == "" {
|
||||
err = config.ErrAIMLAPIKeyMissing
|
||||
}
|
||||
if err != nil {
|
||||
return errorResult("Error: ZERO_WEBFETCH_PROVIDER=aimlapi needs a usable AI/ML API key: " + redactWebFetchText(scrubWebFetchKey(err.Error(), key)) + ". web_fetch does not fall back to a direct fetch; unset ZERO_WEBFETCH_PROVIDER to fetch pages directly.")
|
||||
}
|
||||
scrub := func(value string) string {
|
||||
return redactWebFetchText(scrubWebFetchKey(value, key))
|
||||
}
|
||||
|
||||
run, statusCode, err := backend.scrape(ctx, key, parsedURL.String())
|
||||
if err != nil {
|
||||
return errorResult("Error fetching URL via AI/ML API Toolbox (" + webFetchAIMLAPITool + "): " + scrub(err.Error()))
|
||||
}
|
||||
capNote := ""
|
||||
if run.Cost.OverMax {
|
||||
capNote = fmt.Sprintf(" The run hit the $%g max_cost_usd cap.", webFetchAIMLAPIMaxCostUSD)
|
||||
}
|
||||
if run.IsError || run.Status != "completed" {
|
||||
reason := "run ended with status " + firstNonEmptyString(run.Status, "unknown")
|
||||
if runError := decodeWebFetchToolboxRunError(run.Error); runError.Message != "" {
|
||||
reason = runError.Message
|
||||
if runError.Retryable {
|
||||
reason += " (retryable)"
|
||||
}
|
||||
}
|
||||
return errorResult("Error fetching URL via AI/ML API Toolbox (" + webFetchAIMLAPITool + "): " + scrub(reason) + "." + capNote)
|
||||
}
|
||||
content := webFetchToolboxContent(run.Output)
|
||||
if strings.TrimSpace(content) == "" {
|
||||
return errorResult("Error fetching URL via AI/ML API Toolbox (" + webFetchAIMLAPITool + "): the run returned no page content." + capNote)
|
||||
}
|
||||
|
||||
// Redact before cutting to max_bytes, so the cut cannot split a secret
|
||||
// into a fragment the patterns no longer match.
|
||||
body := scrub(content)
|
||||
truncated := false
|
||||
if len(body) > maxBytes {
|
||||
cut := maxBytes
|
||||
for cut > 0 && !utf8.RuneStart(body[cut]) {
|
||||
cut--
|
||||
}
|
||||
body = body[:cut]
|
||||
truncated = true
|
||||
}
|
||||
|
||||
finalURL := redactWebFetchURL(parsedURL)
|
||||
headers := []string{
|
||||
"URL: " + finalURL,
|
||||
"Status: completed (AI/ML API Toolbox run)",
|
||||
"Content-Type: text/markdown",
|
||||
"Bytes: " + strconv.Itoa(len(body)),
|
||||
"Backend: " + webFetchProviderAIMLAPI + " (" + webFetchAIMLAPITool + ")",
|
||||
}
|
||||
meta := map[string]string{
|
||||
"url": finalURL,
|
||||
"status_code": strconv.Itoa(statusCode),
|
||||
"content_type": "text/markdown",
|
||||
"bytes": strconv.Itoa(len(body)),
|
||||
"truncated": strconv.FormatBool(truncated),
|
||||
"converted": "false",
|
||||
"backend": webFetchProviderAIMLAPI,
|
||||
"tool": webFetchAIMLAPITool,
|
||||
}
|
||||
if run.ID != "" {
|
||||
meta["run_id"] = scrub(run.ID)
|
||||
}
|
||||
if run.Cost.ActualUSD != nil {
|
||||
meta["cost_usd"] = strconv.FormatFloat(*run.Cost.ActualUSD, 'f', -1, 64)
|
||||
}
|
||||
return Result{
|
||||
Status: StatusOK,
|
||||
Output: strings.Join(append(headers, "", body), "\n"),
|
||||
Truncated: truncated,
|
||||
Meta: meta,
|
||||
}
|
||||
}
|
||||
|
||||
// scrape starts one run and waits for it to finish. It returns the terminal
|
||||
// run and the HTTP status of the response that carried it.
|
||||
func (backend webFetchAIMLAPIBackend) scrape(ctx context.Context, key string, target string) (webFetchToolboxRun, int, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, webFetchAIMLAPITimeout)
|
||||
defer cancel()
|
||||
|
||||
idempotencyKey, err := backend.idempotencyKey()
|
||||
if err != nil {
|
||||
return webFetchToolboxRun{}, 0, fmt.Errorf("create idempotency key: %w", err)
|
||||
}
|
||||
// One body for every attempt: a retry carries the same idempotency key, so
|
||||
// a run that did start is returned again instead of being billed twice.
|
||||
body, err := json.Marshal(webFetchToolboxRunRequest{
|
||||
Tool: webFetchAIMLAPITool,
|
||||
Input: webFetchToolboxRunInput{URL: target},
|
||||
MaxCostUSD: webFetchAIMLAPIMaxCostUSD,
|
||||
WaitMS: webFetchAIMLAPIWaitMS,
|
||||
IdempotencyKey: idempotencyKey,
|
||||
})
|
||||
if err != nil {
|
||||
return webFetchToolboxRun{}, 0, err
|
||||
}
|
||||
client := backend.httpClient()
|
||||
|
||||
var run webFetchToolboxRun
|
||||
statusCode := 0
|
||||
for attempt := 1; ; attempt++ {
|
||||
response, err := backend.send(ctx, &client, http.MethodPost, backend.baseURL+"/run", body, key)
|
||||
if err != nil {
|
||||
return webFetchToolboxRun{}, 0, err
|
||||
}
|
||||
if response.StatusCode >= 200 && response.StatusCode < 300 {
|
||||
run, err = decodeWebFetchToolboxRun(response)
|
||||
if err != nil {
|
||||
return webFetchToolboxRun{}, 0, err
|
||||
}
|
||||
statusCode = response.StatusCode
|
||||
break
|
||||
}
|
||||
// 400 max_cost_exceeded and other refusals are final; only rate limits
|
||||
// and gateway errors are retried.
|
||||
if !webFetchAIMLAPIRetryable(response.StatusCode) || attempt >= webFetchAIMLAPIMaxAttempts {
|
||||
return webFetchToolboxRun{}, 0, webFetchToolboxRequestError(response, "run", key)
|
||||
}
|
||||
delay := webFetchAIMLAPIRetryDelay(response.Header.Get("Retry-After"), attempt)
|
||||
discardWebFetchBody(response)
|
||||
if err := backend.sleep(ctx, delay); err != nil {
|
||||
return webFetchToolboxRun{}, 0, err
|
||||
}
|
||||
}
|
||||
|
||||
pollRetryDelay := time.Duration(0)
|
||||
for !webFetchToolboxTerminal(run.Status) {
|
||||
if run.ID == "" {
|
||||
return webFetchToolboxRun{}, 0, errors.New("the run is still pending but has no run id")
|
||||
}
|
||||
if err := backend.sleep(ctx, max(webFetchAIMLAPIPollDelay(run.NextPollHintMS), pollRetryDelay)); err != nil {
|
||||
return webFetchToolboxRun{}, 0, err
|
||||
}
|
||||
response, err := backend.send(ctx, &client, http.MethodGet, backend.baseURL+"/runs/"+url.PathEscape(run.ID), nil, key)
|
||||
if err != nil {
|
||||
return webFetchToolboxRun{}, 0, err
|
||||
}
|
||||
pollRetryDelay = 0
|
||||
if webFetchAIMLAPIRetryable(response.StatusCode) {
|
||||
if retryAfter := strings.TrimSpace(response.Header.Get("Retry-After")); retryAfter != "" {
|
||||
pollRetryDelay = webFetchAIMLAPIRetryDelay(retryAfter, 1)
|
||||
}
|
||||
discardWebFetchBody(response)
|
||||
continue
|
||||
}
|
||||
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
||||
return webFetchToolboxRun{}, 0, webFetchToolboxRequestError(response, "run status", key)
|
||||
}
|
||||
next, err := decodeWebFetchToolboxRun(response)
|
||||
if err != nil {
|
||||
return webFetchToolboxRun{}, 0, err
|
||||
}
|
||||
if next.ID == "" {
|
||||
next.ID = run.ID
|
||||
}
|
||||
run, statusCode = next, response.StatusCode
|
||||
}
|
||||
return run, statusCode, nil
|
||||
}
|
||||
|
||||
// httpClient is a plain client for the fixed Toolbox endpoint. It does not use
|
||||
// the SSRF-pinning transport meant for target sites, and like Zero's other
|
||||
// aimlapi.com clients it uses the default transport, which honours
|
||||
// HTTP_PROXY/HTTPS_PROXY/NO_PROXY.
|
||||
func (backend webFetchAIMLAPIBackend) httpClient() http.Client {
|
||||
client := http.Client{Timeout: webFetchAIMLAPIRequestTimeout}
|
||||
if backend.client != nil {
|
||||
client = *backend.client
|
||||
}
|
||||
// Never follow a redirect: the key and attribution headers must not reach
|
||||
// any host but the Toolbox.
|
||||
client.CheckRedirect = func(*http.Request, []*http.Request) error {
|
||||
return errWebFetchAIMLAPIRedirect
|
||||
}
|
||||
return client
|
||||
}
|
||||
|
||||
func (backend webFetchAIMLAPIBackend) send(ctx context.Context, client *http.Client, method string, endpoint string, body []byte, key string) (*http.Response, error) {
|
||||
var reader io.Reader
|
||||
if body != nil {
|
||||
reader = bytes.NewReader(body)
|
||||
}
|
||||
request, err := http.NewRequestWithContext(ctx, method, endpoint, reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Fail closed: the key and attribution go only to an aimlapi.com service.
|
||||
if !aimlapi.IsAttributedURL(request.URL) {
|
||||
return nil, errors.New("refusing to send the AI/ML API key to a host other than tools.aimlapi.com")
|
||||
}
|
||||
request.Header.Set("Authorization", "Bearer "+key)
|
||||
aimlapi.ApplyAttributionHeaders(request.Header, request.URL)
|
||||
request.Header.Set("User-Agent", "zero-web-fetch/0.1")
|
||||
request.Header.Set("Accept", "application/json")
|
||||
if body != nil {
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
response, err := client.Do(request)
|
||||
if err != nil {
|
||||
// Report the cause without the *url.Error around it, as web_search
|
||||
// does: that quotes the request URL, and a polling URL carries the run
|
||||
// id the Toolbox returned percent-escaped, a form of any key echoed in
|
||||
// it that scrubbing cannot match.
|
||||
var urlErr *url.Error
|
||||
if errors.As(err, &urlErr) {
|
||||
return nil, urlErr.Err
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if response.Body == nil {
|
||||
response.Body = io.NopCloser(strings.NewReader(""))
|
||||
}
|
||||
return response, nil
|
||||
}
|
||||
|
||||
func decodeWebFetchToolboxRun(response *http.Response) (webFetchToolboxRun, error) {
|
||||
defer response.Body.Close()
|
||||
raw, truncated, err := readWebFetchBody(response.Body, webFetchAIMLAPIResponseLimit)
|
||||
if err != nil {
|
||||
return webFetchToolboxRun{}, fmt.Errorf("read the run: %w", err)
|
||||
}
|
||||
if truncated {
|
||||
return webFetchToolboxRun{}, fmt.Errorf("the run response is larger than %d bytes", webFetchAIMLAPIResponseLimit)
|
||||
}
|
||||
var run webFetchToolboxRun
|
||||
if err := json.Unmarshal([]byte(raw), &run); err != nil {
|
||||
return webFetchToolboxRun{}, errors.New("the run response is not valid JSON")
|
||||
}
|
||||
return run, nil
|
||||
}
|
||||
|
||||
// webFetchToolboxRequestError reports a non-2xx answer with the error envelope
|
||||
// the Toolbox sends ({"error":{"code","message","request_id"}}). The key is
|
||||
// scrubbed from the decoded text before it is cut to size.
|
||||
func webFetchToolboxRequestError(response *http.Response, action string, key string) error {
|
||||
defer response.Body.Close()
|
||||
raw, _, _ := readWebFetchBody(response.Body, webFetchAIMLAPIErrorReadLimit)
|
||||
var envelope struct {
|
||||
Error struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
RequestID string `json:"request_id"`
|
||||
} `json:"error"`
|
||||
}
|
||||
detail := ""
|
||||
if json.Unmarshal([]byte(raw), &envelope) == nil && (envelope.Error.Code != "" || envelope.Error.Message != "") {
|
||||
if envelope.Error.Code != "" {
|
||||
detail += " (" + envelope.Error.Code + ")"
|
||||
}
|
||||
if envelope.Error.Message != "" {
|
||||
detail += ": " + envelope.Error.Message
|
||||
}
|
||||
if envelope.Error.RequestID != "" {
|
||||
detail += " [request_id " + envelope.Error.RequestID + "]"
|
||||
}
|
||||
} else if text := strings.TrimSpace(raw); text != "" {
|
||||
detail = ": " + text
|
||||
}
|
||||
detail = scrubWebFetchKey(detail, key)
|
||||
if len(detail) > webFetchErrorBodyLimit {
|
||||
detail = detail[:webFetchErrorBodyLimit]
|
||||
}
|
||||
message := "AI/ML API Toolbox " + action + " returned HTTP " + webFetchStatusLine(response) + detail
|
||||
if envelope.Error.Code == "max_cost_exceeded" {
|
||||
message += fmt.Sprintf(". The run was refused before it started (cap $%g) and was not billed", webFetchAIMLAPIMaxCostUSD)
|
||||
}
|
||||
return errors.New(message)
|
||||
}
|
||||
|
||||
func decodeWebFetchToolboxRunError(raw json.RawMessage) webFetchToolboxRunError {
|
||||
var runError webFetchToolboxRunError
|
||||
if len(raw) == 0 {
|
||||
return runError
|
||||
}
|
||||
if json.Unmarshal(raw, &runError) == nil {
|
||||
if runError.Message == "" {
|
||||
runError.Message = firstNonEmptyString(runError.Code, runError.Kind)
|
||||
}
|
||||
return runError
|
||||
}
|
||||
var text string
|
||||
if json.Unmarshal(raw, &text) == nil {
|
||||
runError.Message = text
|
||||
}
|
||||
return runError
|
||||
}
|
||||
|
||||
// webFetchToolboxContent returns output.results.content, the crawler's
|
||||
// Markdown, or "" when the output has another shape.
|
||||
func webFetchToolboxContent(raw json.RawMessage) string {
|
||||
var output struct {
|
||||
Results struct {
|
||||
Content json.RawMessage `json:"content"`
|
||||
} `json:"results"`
|
||||
}
|
||||
if len(raw) == 0 || json.Unmarshal(raw, &output) != nil {
|
||||
return ""
|
||||
}
|
||||
var content string
|
||||
if json.Unmarshal(output.Results.Content, &content) != nil {
|
||||
return ""
|
||||
}
|
||||
return content
|
||||
}
|
||||
|
||||
func webFetchToolboxTerminal(status string) bool {
|
||||
switch status {
|
||||
case "completed", "failed", "stopped", "timed_out":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func webFetchAIMLAPIRetryable(statusCode int) bool {
|
||||
switch statusCode {
|
||||
case http.StatusTooManyRequests, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// webFetchAIMLAPIRetryDelay honours Retry-After (seconds or an HTTP date) and
|
||||
// otherwise backs off exponentially; either way it is capped.
|
||||
func webFetchAIMLAPIRetryDelay(retryAfter string, attempt int) time.Duration {
|
||||
delay := webFetchAIMLAPIBackoff << (attempt - 1)
|
||||
if value := strings.TrimSpace(retryAfter); value != "" {
|
||||
if seconds, err := strconv.Atoi(value); err == nil && seconds >= 0 {
|
||||
// Capped before the multiplication, so a huge value cannot overflow.
|
||||
delay = time.Duration(min(seconds, int(webFetchAIMLAPIMaxRetryDelay/time.Second))) * time.Second
|
||||
} else if when, err := http.ParseTime(value); err == nil {
|
||||
delay = time.Until(when)
|
||||
}
|
||||
}
|
||||
return max(0, min(delay, webFetchAIMLAPIMaxRetryDelay))
|
||||
}
|
||||
|
||||
// webFetchAIMLAPIPollDelay honours next_poll_hint_ms, clamped to 250 ms..5 s.
|
||||
func webFetchAIMLAPIPollDelay(hintMS *float64) time.Duration {
|
||||
if hintMS == nil {
|
||||
return webFetchAIMLAPIDefaultPoll
|
||||
}
|
||||
// Clamp in milliseconds first, so a huge hint cannot overflow a Duration.
|
||||
ms := max(float64(webFetchAIMLAPIMinPoll/time.Millisecond), min(*hintMS, float64(webFetchAIMLAPIMaxPoll/time.Millisecond)))
|
||||
return time.Duration(ms) * time.Millisecond
|
||||
}
|
||||
|
||||
// scrubWebFetchKey removes an echoed AI/ML API key from text shown to the
|
||||
// model, whole or split by invisible characters, as web_search does. An empty
|
||||
// key is no key: replacing "" would rewrite every byte gap.
|
||||
func scrubWebFetchKey(value string, key string) string {
|
||||
if key == "" {
|
||||
return value
|
||||
}
|
||||
return scrubAIMLAPIResult(value, key)
|
||||
}
|
||||
|
||||
func discardWebFetchBody(response *http.Response) {
|
||||
_, _ = io.Copy(io.Discard, io.LimitReader(response.Body, webFetchErrorBodyLimit))
|
||||
_ = response.Body.Close()
|
||||
}
|
||||
|
||||
func sleepWebFetchContext(ctx context.Context, delay time.Duration) error {
|
||||
if delay <= 0 {
|
||||
return ctx.Err()
|
||||
}
|
||||
timer := time.NewTimer(delay)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-timer.C:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// webFetchAIMLAPIRawFormatError is shared by preflight and execution.
|
||||
func webFetchAIMLAPIRawFormatError() Result {
|
||||
return errorResult(`Error: format "raw" is not available with ZERO_WEBFETCH_PROVIDER=aimlapi: the AI/ML API Toolbox returns the page as Markdown, not the original response. Use format "auto" or "markdown", or unset ZERO_WEBFETCH_PROVIDER to fetch the page directly.`)
|
||||
}
|
||||
@@ -0,0 +1,842 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"net/url"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
)
|
||||
|
||||
const webFetchAIMLAPITestKey = "aiml-test-key-0042"
|
||||
|
||||
// Trimmed from a real search1api/crawl run.
|
||||
const webFetchAIMLAPICompletedRun = `{"id":"5SJmhg3ePaXxHh_BLdlS9","tool":"search1api/crawl","status":"completed","is_error":false,"untrusted":true,` +
|
||||
`"output":{"crawlParameters":{"url":"https://example.com/"},"results":{"title":"Example Domain","link":"https://example.com/",` +
|
||||
`"content":"This domain is for use in documentation examples without needing permission.","metadata":{"sourceUrl":"https://example.com/"}}},` +
|
||||
`"cost":{"quoted_usd":0.0013,"actual_usd":0.0013,"credits":2600,"over_max":false}}`
|
||||
|
||||
// Trimmed from a real 400 max_cost_exceeded refusal (same envelope for every tool).
|
||||
const webFetchAIMLAPIMaxCostBody = `{"error":{"message":"The cheapest source costs 0.0091 USD, above the limit of 0.001 USD.","type":"invalid_request_error",` +
|
||||
`"code":"max_cost_exceeded","request_id":"hnbQUvsseM1EX9RrgPjBT","refused_by":"request","estimated_cost_usd":0.0091,"max_cost_usd":0.001}}`
|
||||
|
||||
var webFetchUUIDPattern = regexp.MustCompile(`^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$`)
|
||||
|
||||
type recordedToolboxRequest struct {
|
||||
method string
|
||||
url string
|
||||
header http.Header
|
||||
body []byte
|
||||
}
|
||||
|
||||
// webFetchFakeToolbox records every request that reaches the Toolbox client and
|
||||
// answers from respond, indexed by request order.
|
||||
type webFetchFakeToolbox struct {
|
||||
t *testing.T
|
||||
mu sync.Mutex
|
||||
requests []recordedToolboxRequest
|
||||
respond func(index int, request *http.Request) *http.Response
|
||||
}
|
||||
|
||||
func (fake *webFetchFakeToolbox) client() *http.Client {
|
||||
return &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
||||
var body []byte
|
||||
if request.Body != nil {
|
||||
body, _ = io.ReadAll(request.Body)
|
||||
}
|
||||
fake.mu.Lock()
|
||||
index := len(fake.requests)
|
||||
fake.requests = append(fake.requests, recordedToolboxRequest{
|
||||
method: request.Method,
|
||||
url: request.URL.String(),
|
||||
header: request.Header.Clone(),
|
||||
body: body,
|
||||
})
|
||||
fake.mu.Unlock()
|
||||
if fake.respond == nil {
|
||||
fake.t.Errorf("unexpected Toolbox request %s %s", request.Method, request.URL)
|
||||
return webFetchTestResponse(request, http.StatusInternalServerError, "", ""), nil
|
||||
}
|
||||
return fake.respond(index, request), nil
|
||||
})}
|
||||
}
|
||||
|
||||
func (fake *webFetchFakeToolbox) recorded() []recordedToolboxRequest {
|
||||
fake.mu.Lock()
|
||||
defer fake.mu.Unlock()
|
||||
return append([]recordedToolboxRequest(nil), fake.requests...)
|
||||
}
|
||||
|
||||
type webFetchAIMLAPIHarness struct {
|
||||
tool webFetchTool
|
||||
toolbox *webFetchFakeToolbox
|
||||
direct *[]string
|
||||
sleeps *[]time.Duration
|
||||
}
|
||||
|
||||
// hermeticWebFetchAIMLAPIEnv redirects every root the real key lookup reads.
|
||||
func hermeticWebFetchAIMLAPIEnv(t *testing.T) {
|
||||
t.Helper()
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData", "Roaming"))
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("ZERO_CRED_STORAGE", "file")
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_PARTNER_ID", "")
|
||||
}
|
||||
|
||||
// newWebFetchAIMLAPIHarness builds web_fetch with provider set to provider, a
|
||||
// recording direct client, a fake Toolbox, a fake resolver, a fixed key and a
|
||||
// sleep that only records.
|
||||
func newWebFetchAIMLAPIHarness(t *testing.T, provider string, respond func(int, *http.Request) *http.Response) webFetchAIMLAPIHarness {
|
||||
t.Helper()
|
||||
hermeticWebFetchAIMLAPIEnv(t)
|
||||
t.Setenv(webFetchProviderEnv, provider)
|
||||
|
||||
var mu sync.Mutex
|
||||
direct := []string{}
|
||||
sleeps := []time.Duration{}
|
||||
directClient := webFetchTestClient(func(request *http.Request) (*http.Response, error) {
|
||||
mu.Lock()
|
||||
direct = append(direct, request.URL.String())
|
||||
mu.Unlock()
|
||||
return webFetchTestResponse(request, http.StatusOK, "text/plain", "direct body"), nil
|
||||
})
|
||||
resolver := webFetchResolverFunc(func(_ context.Context, _ string, host string) ([]netip.Addr, error) {
|
||||
if host == "private.example" {
|
||||
return []netip.Addr{netip.MustParseAddr("10.0.0.5")}, nil
|
||||
}
|
||||
return []netip.Addr{netip.MustParseAddr("93.184.216.34")}, nil
|
||||
})
|
||||
tool := newWebFetchToolWithClientAndResolver(directClient, resolver).(webFetchTool)
|
||||
toolbox := &webFetchFakeToolbox{t: t, respond: respond}
|
||||
tool.aimlapi.client = toolbox.client()
|
||||
tool.aimlapi.serviceKey = func() (string, error) { return webFetchAIMLAPITestKey, nil }
|
||||
tool.aimlapi.sleep = func(ctx context.Context, delay time.Duration) error {
|
||||
mu.Lock()
|
||||
sleeps = append(sleeps, delay)
|
||||
mu.Unlock()
|
||||
return ctx.Err()
|
||||
}
|
||||
return webFetchAIMLAPIHarness{tool: tool, toolbox: toolbox, direct: &direct, sleeps: &sleeps}
|
||||
}
|
||||
|
||||
func toolboxJSON(request *http.Request, statusCode int, body string) *http.Response {
|
||||
return webFetchTestResponse(request, statusCode, "application/json", body)
|
||||
}
|
||||
|
||||
func completedToolboxRun(content string) string {
|
||||
encoded, _ := json.Marshal(content)
|
||||
return `{"id":"run-1","status":"completed","is_error":false,"output":{"results":{"title":"T","link":"https://example.com/","content":` +
|
||||
string(encoded) + `}},"cost":{"quoted_usd":0.0013,"actual_usd":0.0013,"over_max":false}}`
|
||||
}
|
||||
|
||||
func decodeToolboxBody(t *testing.T, body []byte) map[string]any {
|
||||
t.Helper()
|
||||
var decoded map[string]any
|
||||
if err := json.Unmarshal(body, &decoded); err != nil {
|
||||
t.Fatalf("Toolbox body is not JSON: %v: %s", err, body)
|
||||
}
|
||||
return decoded
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIKeyAloneKeepsDirectFetch(t *testing.T) {
|
||||
for _, provider := range []string{"", " ", "firecrawl", "aimlapi-x"} {
|
||||
t.Run("provider="+provider, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, provider, nil)
|
||||
// A usable key from every source must not switch the backend.
|
||||
t.Setenv("AIMLAPI_API_KEY", webFetchAIMLAPITestKey)
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/page"})
|
||||
|
||||
if result.Status != StatusOK || !strings.Contains(result.Output, "direct body") {
|
||||
t.Fatalf("expected the direct fetch, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if strings.Contains(result.Output, "Backend:") || result.Meta["backend"] != "" {
|
||||
t.Fatalf("direct fetch must not report a backend: %q %#v", result.Output, result.Meta)
|
||||
}
|
||||
if len(*h.direct) != 1 || len(h.toolbox.recorded()) != 0 {
|
||||
t.Fatalf("direct=%v toolbox=%d, want one direct request and no Toolbox request", *h.direct, len(h.toolbox.recorded()))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPISelectedRunsToolboxCrawl(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, " AIMLAPI ", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, webFetchAIMLAPICompletedRun)
|
||||
})
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/guide?page=2&q=go"})
|
||||
|
||||
if result.Status != StatusOK {
|
||||
t.Fatalf("expected ok, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if len(*h.direct) != 0 {
|
||||
t.Fatalf("the direct fetch must not run, got %v", *h.direct)
|
||||
}
|
||||
requests := h.toolbox.recorded()
|
||||
if len(requests) != 1 {
|
||||
t.Fatalf("Toolbox requests = %d, want 1", len(requests))
|
||||
}
|
||||
request := requests[0]
|
||||
if request.method != http.MethodPost || request.url != "https://tools.aimlapi.com/v1/tools/run" {
|
||||
t.Fatalf("request = %s %s, want POST https://tools.aimlapi.com/v1/tools/run", request.method, request.url)
|
||||
}
|
||||
body := decodeToolboxBody(t, request.body)
|
||||
idempotencyKey, _ := body["idempotency_key"].(string)
|
||||
if !webFetchUUIDPattern.MatchString(idempotencyKey) {
|
||||
t.Fatalf("idempotency_key = %q, want a UUIDv4", idempotencyKey)
|
||||
}
|
||||
want := map[string]any{
|
||||
"tool": "search1api/crawl",
|
||||
"input": map[string]any{"url": "https://example.com/guide?page=2&q=go"},
|
||||
"max_cost_usd": 0.005,
|
||||
"wait_ms": float64(30000),
|
||||
"idempotency_key": idempotencyKey,
|
||||
}
|
||||
if !reflect.DeepEqual(body, want) {
|
||||
t.Fatalf("body = %#v\nwant %#v", body, want)
|
||||
}
|
||||
for name, value := range map[string]string{
|
||||
"Authorization": "Bearer " + webFetchAIMLAPITestKey,
|
||||
"X-AIMLAPI-Source": "agent/zero",
|
||||
"X-AIMLAPI-Partner-ID": aimlapi.DefaultPartnerID,
|
||||
"User-Agent": "zero-web-fetch/0.1",
|
||||
"Content-Type": "application/json",
|
||||
} {
|
||||
if got := request.header.Get(name); got != value {
|
||||
t.Fatalf("header %s = %q, want %q", name, got, value)
|
||||
}
|
||||
}
|
||||
|
||||
content := "This domain is for use in documentation examples without needing permission."
|
||||
for _, line := range []string{
|
||||
"URL: https://example.com/guide?page=2&q=go",
|
||||
"Status: completed (AI/ML API Toolbox run)",
|
||||
"Content-Type: text/markdown",
|
||||
"Bytes: 76",
|
||||
"Backend: aimlapi (search1api/crawl)",
|
||||
"\n\n" + content,
|
||||
} {
|
||||
if !strings.Contains(result.Output, line) {
|
||||
t.Fatalf("output missing %q:\n%s", line, result.Output)
|
||||
}
|
||||
}
|
||||
if result.Truncated {
|
||||
t.Fatal("result must not be truncated")
|
||||
}
|
||||
for key, value := range map[string]string{
|
||||
"status_code": "200", "content_type": "text/markdown", "bytes": "76", "truncated": "false",
|
||||
"backend": "aimlapi", "tool": "search1api/crawl", "run_id": "5SJmhg3ePaXxHh_BLdlS9", "cost_usd": "0.0013",
|
||||
} {
|
||||
if result.Meta[key] != value {
|
||||
t.Fatalf("meta[%s] = %q, want %q (meta %#v)", key, result.Meta[key], value, result.Meta)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIResolvesTheKeyOnEveryCall(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, webFetchAIMLAPICompletedRun)
|
||||
})
|
||||
keys := []string{"first-key", "second-key"}
|
||||
calls := 0
|
||||
h.tool.aimlapi.serviceKey = func() (string, error) {
|
||||
key := keys[calls]
|
||||
calls++
|
||||
return key, nil
|
||||
}
|
||||
|
||||
for range keys {
|
||||
if result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"}); result.Status != StatusOK {
|
||||
t.Fatalf("expected ok, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
}
|
||||
|
||||
requests := h.toolbox.recorded()
|
||||
if len(requests) != 2 || requests[0].header.Get("Authorization") != "Bearer first-key" || requests[1].header.Get("Authorization") != "Bearer second-key" {
|
||||
t.Fatalf("each call must resolve the key again, got %d requests", len(requests))
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIWithoutUsableKeySendsNothing(t *testing.T) {
|
||||
for name, keyErr := range map[string]error{
|
||||
"missing": config.ErrAIMLAPIKeyMissing,
|
||||
"proxy": config.ErrAIMLAPIProxyProfile,
|
||||
"invalid": config.ErrAIMLAPIKeyInvalid,
|
||||
"empty key": nil,
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", nil)
|
||||
h.tool.aimlapi.serviceKey = func() (string, error) { return "", keyErr }
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
assertWebFetchAIMLAPIKeyRefusal(t, h, result)
|
||||
if keyErr != nil && !strings.Contains(result.Output, keyErr.Error()) {
|
||||
t.Fatalf("expected the key error in the output, got %q", result.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// The real lookup, against an empty hermetic home.
|
||||
t.Run("real lookup without a key", func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", nil)
|
||||
h.tool.aimlapi.serviceKey = defaultWebFetchAIMLAPIBackend().serviceKey
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
assertWebFetchAIMLAPIKeyRefusal(t, h, result)
|
||||
if !strings.Contains(result.Output, config.ErrAIMLAPIKeyMissing.Error()) {
|
||||
t.Fatalf("expected the missing-key error, got %q", result.Output)
|
||||
}
|
||||
})
|
||||
|
||||
// The real lookup withholds an environment key once the inference URL
|
||||
// points at a proxy, and web_fetch then sends nothing.
|
||||
t.Run("real lookup with a proxied key", func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", nil)
|
||||
h.tool.aimlapi.serviceKey = defaultWebFetchAIMLAPIBackend().serviceKey
|
||||
t.Setenv("AIMLAPI_API_KEY", webFetchAIMLAPITestKey)
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "https://proxy.example/v1")
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
assertWebFetchAIMLAPIKeyRefusal(t, h, result)
|
||||
if !strings.Contains(result.Output, config.ErrAIMLAPIProxyProfile.Error()) {
|
||||
t.Fatalf("expected the proxy-profile error, got %q", result.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func assertWebFetchAIMLAPIKeyRefusal(t *testing.T, h webFetchAIMLAPIHarness, result Result) {
|
||||
t.Helper()
|
||||
if result.Status != StatusError {
|
||||
t.Fatalf("expected an error result, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if !strings.Contains(result.Output, "needs a usable AI/ML API key") || !strings.Contains(result.Output, "does not fall back") {
|
||||
t.Fatalf("unexpected refusal: %q", result.Output)
|
||||
}
|
||||
if len(h.toolbox.recorded()) != 0 || len(*h.direct) != 0 {
|
||||
t.Fatalf("no request may be sent: toolbox=%d direct=%v", len(h.toolbox.recorded()), *h.direct)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIRefusesUnsafeURLsBeforeTheToolbox(t *testing.T) {
|
||||
for _, rawURL := range []string{
|
||||
"http://127.0.0.1/admin",
|
||||
"http://localhost/status",
|
||||
"http://169.254.169.254/latest/meta-data",
|
||||
"http://metadata.google.internal/computeMetadata/v1/",
|
||||
"https://private.example/internal",
|
||||
"ftp://example.com/file",
|
||||
"https://example.com:8443/",
|
||||
} {
|
||||
t.Run(rawURL, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", nil)
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": rawURL})
|
||||
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "Unsafe URL") {
|
||||
t.Fatalf("expected an unsafe URL error, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if len(h.toolbox.recorded()) != 0 || len(*h.direct) != 0 {
|
||||
t.Fatalf("an unsafe URL must not reach anyone: toolbox=%d direct=%v", len(h.toolbox.recorded()), *h.direct)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", nil)
|
||||
if _, rejected := h.tool.RejectBeforePermission(map[string]any{"url": "http://localhost/"}); !rejected {
|
||||
t.Fatal("RejectBeforePermission must still refuse localhost with the Toolbox backend")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIRefusesNonToolboxHosts(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", nil)
|
||||
h.tool.aimlapi.baseURL = "https://tools.aimlapi.com.evil.example/v1/tools"
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "refusing to send the AI/ML API key") {
|
||||
t.Fatalf("expected a fail-closed refusal, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if len(h.toolbox.recorded()) != 0 {
|
||||
t.Fatalf("the key must not leave for a non-Toolbox host, got %d requests", len(h.toolbox.recorded()))
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIDefaultClientIsPlainAndRefusesRedirects(t *testing.T) {
|
||||
client := defaultWebFetchAIMLAPIBackend().httpClient()
|
||||
// A nil transport is http.DefaultTransport: the proxy environment applies
|
||||
// and the target-site SSRF pin does not.
|
||||
if client.Transport != nil {
|
||||
t.Fatalf("Toolbox client transport = %T, want the default transport", client.Transport)
|
||||
}
|
||||
if client.CheckRedirect == nil || client.CheckRedirect(nil, nil) == nil {
|
||||
t.Fatal("Toolbox client must refuse redirects")
|
||||
}
|
||||
if defaultWebFetchAIMLAPIBackend().baseURL != aimlapi.ToolboxBaseURL {
|
||||
t.Fatalf("baseURL = %q, want %q", defaultWebFetchAIMLAPIBackend().baseURL, aimlapi.ToolboxBaseURL)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIRefusesRedirects(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(index int, request *http.Request) *http.Response {
|
||||
if index == 0 {
|
||||
response := toolboxJSON(request, http.StatusFound, "")
|
||||
response.Header.Set("Location", "https://collector.example/steal")
|
||||
return response
|
||||
}
|
||||
return toolboxJSON(request, http.StatusOK, webFetchAIMLAPICompletedRun)
|
||||
})
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "redirects are refused") {
|
||||
t.Fatalf("expected a redirect refusal, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if requests := h.toolbox.recorded(); len(requests) != 1 {
|
||||
t.Fatalf("the redirect must not be followed, got %d requests (last %s)", len(requests), requests[len(requests)-1].url)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIPollsAQueuedRun(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(index int, request *http.Request) *http.Response {
|
||||
switch index {
|
||||
case 0:
|
||||
return toolboxJSON(request, http.StatusAccepted, `{"id":"run/7","status":"queued","next_poll_hint_ms":10}`)
|
||||
case 1:
|
||||
return toolboxJSON(request, http.StatusServiceUnavailable, `{"error":{"code":"unavailable"}}`)
|
||||
case 2:
|
||||
return toolboxJSON(request, http.StatusOK, `{"id":"run/7","status":"running","next_poll_hint_ms":60000}`)
|
||||
default:
|
||||
return toolboxJSON(request, http.StatusOK, completedToolboxRun("polled page"))
|
||||
}
|
||||
})
|
||||
deadlineChecked := false
|
||||
h.tool.aimlapi.sleep = func(ctx context.Context, delay time.Duration) error {
|
||||
deadline, ok := ctx.Deadline()
|
||||
if !ok || time.Until(deadline) > webFetchAIMLAPITimeout {
|
||||
t.Errorf("polling must run under the %s overall deadline (deadline %v, set %v)", webFetchAIMLAPITimeout, deadline, ok)
|
||||
}
|
||||
deadlineChecked = true
|
||||
*h.sleeps = append(*h.sleeps, delay)
|
||||
return nil
|
||||
}
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if result.Status != StatusOK || !strings.Contains(result.Output, "polled page") {
|
||||
t.Fatalf("expected the polled content, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
requests := h.toolbox.recorded()
|
||||
if len(requests) != 4 {
|
||||
t.Fatalf("requests = %d, want POST and three polls", len(requests))
|
||||
}
|
||||
for _, request := range requests[1:] {
|
||||
if request.method != http.MethodGet || request.url != "https://tools.aimlapi.com/v1/tools/runs/run%2F7" {
|
||||
t.Fatalf("poll = %s %s, want GET of the escaped run id", request.method, request.url)
|
||||
}
|
||||
if request.header.Get("Authorization") != "Bearer "+webFetchAIMLAPITestKey || request.header.Get("X-AIMLAPI-Source") != "agent/zero" {
|
||||
t.Fatalf("poll must carry auth and attribution, got %v", request.header)
|
||||
}
|
||||
}
|
||||
// Hints are clamped to 250 ms..5 s; the 503 poll reuses the queued run's hint.
|
||||
wantSleeps := []time.Duration{250 * time.Millisecond, 250 * time.Millisecond, 5 * time.Second}
|
||||
if !deadlineChecked || !reflect.DeepEqual(*h.sleeps, wantSleeps) {
|
||||
t.Fatalf("sleeps = %v, want %v", *h.sleeps, wantSleeps)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIStopsPollingWhenTheContextEnds(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
if request.Method == http.MethodGet {
|
||||
t.Error("poll GET sent after canceled sleep")
|
||||
return toolboxJSON(request, http.StatusOK, `{"id":"run-1","status":"failed"}`)
|
||||
}
|
||||
return toolboxJSON(request, http.StatusAccepted, `{"id":"run-1","status":"queued"}`)
|
||||
})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
h.tool.aimlapi.sleep = func(sleepCtx context.Context, delay time.Duration) error {
|
||||
cancel()
|
||||
return sleepWebFetchContext(sleepCtx, delay)
|
||||
}
|
||||
|
||||
result := h.tool.Run(ctx, map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "context canceled") {
|
||||
t.Fatalf("expected a cancellation error, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIReportsFailedRuns(t *testing.T) {
|
||||
for name, body := range map[string]string{
|
||||
"is_error with completed": `{"id":"run-1","status":"completed","is_error":true,"error":{"message":"Target returned HTTP 404","kind":"provider_error"},` +
|
||||
`"output":{"results":{"content":"stale page"}}}`,
|
||||
"failed status": `{"id":"run-1","status":"failed","is_error":false,"output":{"results":{"content":"stale page"}}}`,
|
||||
"no content": `{"id":"run-1","status":"completed","is_error":false,"output":{"results":{"content":""}}}`,
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, body)
|
||||
})
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/missing"})
|
||||
|
||||
if result.Status != StatusError || strings.Contains(result.Output, "stale page") {
|
||||
t.Fatalf("expected an error without content, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if len(*h.direct) != 0 {
|
||||
t.Fatalf("a failed scrape must not fall back to a direct fetch, got %v", *h.direct)
|
||||
}
|
||||
if name == "is_error with completed" && !strings.Contains(result.Output, "Target returned HTTP 404") {
|
||||
t.Fatalf("expected the run error message, got %q", result.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIDoesNotRetryMaxCostExceeded(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusBadRequest, webFetchAIMLAPIMaxCostBody)
|
||||
})
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if result.Status != StatusError {
|
||||
t.Fatalf("expected an error, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
for _, want := range []string{"HTTP 400 Bad Request", "(max_cost_exceeded)", "request_id hnbQUvsseM1EX9RrgPjBT", "not billed"} {
|
||||
if !strings.Contains(result.Output, want) {
|
||||
t.Fatalf("output missing %q: %q", want, result.Output)
|
||||
}
|
||||
}
|
||||
if len(h.toolbox.recorded()) != 1 || len(*h.sleeps) != 0 {
|
||||
t.Fatalf("max_cost_exceeded must not be retried: requests=%d sleeps=%v", len(h.toolbox.recorded()), *h.sleeps)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIRetriesWithTheSameRunRequest(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(index int, request *http.Request) *http.Response {
|
||||
switch index {
|
||||
case 0:
|
||||
response := toolboxJSON(request, http.StatusTooManyRequests, `{"error":{"code":"rate_limited","message":"slow down"}}`)
|
||||
response.Header.Set("Retry-After", "2")
|
||||
return response
|
||||
case 1:
|
||||
return toolboxJSON(request, http.StatusBadGateway, `{"error":{"code":"bad_gateway"}}`)
|
||||
default:
|
||||
return toolboxJSON(request, http.StatusOK, webFetchAIMLAPICompletedRun)
|
||||
}
|
||||
})
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if result.Status != StatusOK {
|
||||
t.Fatalf("expected success after retries, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
requests := h.toolbox.recorded()
|
||||
if len(requests) != 3 {
|
||||
t.Fatalf("requests = %d, want 3", len(requests))
|
||||
}
|
||||
for _, request := range requests[1:] {
|
||||
if request.method != http.MethodPost || string(request.body) != string(requests[0].body) {
|
||||
t.Fatalf("a retry must resend the identical body:\n%s\n%s", requests[0].body, request.body)
|
||||
}
|
||||
}
|
||||
// Retry-After: 2 is honoured; the 502 without one backs off 500 ms << 1.
|
||||
if want := []time.Duration{2 * time.Second, time.Second}; !reflect.DeepEqual(*h.sleeps, want) {
|
||||
t.Fatalf("sleeps = %v, want %v", *h.sleeps, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIGivesUpAfterTheLastAttempt(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
response := toolboxJSON(request, http.StatusTooManyRequests, `{"error":{"code":"rate_limited","message":"too many concurrent runs"}}`)
|
||||
response.Header.Set("Retry-After", "3600")
|
||||
return response
|
||||
})
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "(rate_limited): too many concurrent runs") {
|
||||
t.Fatalf("expected the rate-limit error, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if len(h.toolbox.recorded()) != 3 {
|
||||
t.Fatalf("requests = %d, want 3 attempts", len(h.toolbox.recorded()))
|
||||
}
|
||||
// A huge Retry-After is capped.
|
||||
if want := []time.Duration{10 * time.Second, 10 * time.Second}; !reflect.DeepEqual(*h.sleeps, want) {
|
||||
t.Fatalf("sleeps = %v, want %v", *h.sleeps, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIUsesAFreshIdempotencyKeyPerCall(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, webFetchAIMLAPICompletedRun)
|
||||
})
|
||||
|
||||
for range 2 {
|
||||
if result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"}); result.Status != StatusOK {
|
||||
t.Fatalf("expected ok, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
}
|
||||
|
||||
requests := h.toolbox.recorded()
|
||||
first, _ := decodeToolboxBody(t, requests[0].body)["idempotency_key"].(string)
|
||||
second, _ := decodeToolboxBody(t, requests[1].body)["idempotency_key"].(string)
|
||||
if !webFetchUUIDPattern.MatchString(first) || !webFetchUUIDPattern.MatchString(second) || first == second {
|
||||
t.Fatalf("each web_fetch call needs its own UUID idempotency key, got %q and %q", first, second)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPITruncatesAtMaxBytes(t *testing.T) {
|
||||
for name, tc := range map[string]struct {
|
||||
content string
|
||||
maxBytes int
|
||||
want string
|
||||
}{
|
||||
"ascii": {content: "abcdefghij", maxBytes: 4, want: "abcd"},
|
||||
"rune boundary": {content: "aé-tail", maxBytes: 2, want: "a"},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, completedToolboxRun(tc.content))
|
||||
})
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/", "max_bytes": tc.maxBytes})
|
||||
|
||||
if result.Status != StatusOK || !result.Truncated || result.Meta["truncated"] != "true" {
|
||||
t.Fatalf("expected a truncated result, got %s truncated=%v meta=%#v", result.Status, result.Truncated, result.Meta)
|
||||
}
|
||||
wantBytes := len(tc.want)
|
||||
if result.Meta["bytes"] != strconv.Itoa(wantBytes) || !strings.Contains(result.Output, "Bytes: "+strconv.Itoa(wantBytes)+"\n") {
|
||||
t.Fatalf("byte count must describe the cut body, meta=%#v output=%q", result.Meta, result.Output)
|
||||
}
|
||||
if !strings.HasSuffix(result.Output, "\n\n"+tc.want) {
|
||||
t.Fatalf("output must end with %q, got %q", tc.want, result.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIFormats(t *testing.T) {
|
||||
content := "# Title\n\n<b>kept</b> as the crawler sent it"
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, completedToolboxRun(content))
|
||||
})
|
||||
|
||||
raw := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/", "format": "raw"})
|
||||
if raw.Status != StatusError || !strings.Contains(raw.Output, `format "raw" is not available`) {
|
||||
t.Fatalf("format raw must be refused, got %s: %s", raw.Status, raw.Output)
|
||||
}
|
||||
if len(h.toolbox.recorded()) != 0 {
|
||||
t.Fatal("format raw must be refused before a billed run")
|
||||
}
|
||||
|
||||
for _, format := range []string{"auto", "markdown"} {
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/", "format": format})
|
||||
if result.Status != StatusOK || !strings.HasSuffix(result.Output, "\n\n"+content) {
|
||||
t.Fatalf("format %s must return the crawler's Markdown unchanged, got %s: %q", format, result.Status, result.Output)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIRedactsTheKey(t *testing.T) {
|
||||
// The key contains "/", which a JSON encoder may write as "\/": the key is
|
||||
// scrubbed from the decoded text, so the escaped echo is caught as well.
|
||||
const key = "aiml/Zx9-echo-key"
|
||||
for name, respond := range map[string]func(*http.Request) *http.Response{
|
||||
"error envelope": func(request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusUnauthorized, `{"error":{"code":"invalid_api_key","message":"key aiml\/Zx9-echo-key is not valid","request_id":"r-1"}}`)
|
||||
},
|
||||
"plain error body": func(request *http.Request) *http.Response {
|
||||
return webFetchTestResponse(request, http.StatusForbidden, "text/plain", "denied for Bearer "+key)
|
||||
},
|
||||
"run error": func(request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, `{"id":"r","status":"completed","is_error":true,"error":{"message":"bad key `+key+`"}}`)
|
||||
},
|
||||
"content": func(request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, completedToolboxRun("page echoing "+key))
|
||||
},
|
||||
// The shown body is cut to 4 KiB; the key straddles that cut, so it
|
||||
// must be scrubbed before the cut or a fragment of it survives.
|
||||
"key at the cut": func(request *http.Request) *http.Response {
|
||||
return webFetchTestResponse(request, http.StatusForbidden, "text/plain", strings.Repeat("x", webFetchErrorBodyLimit-8)+key)
|
||||
},
|
||||
// A key split by an invisible character reads whole once a terminal or
|
||||
// log viewer drops that character, so it must be scrubbed as well.
|
||||
"split key in error body": func(request *http.Request) *http.Response {
|
||||
return webFetchTestResponse(request, http.StatusForbidden, "text/plain", "denied for Bearer aiml/Zx9\u200b-echo-key")
|
||||
},
|
||||
"split key in run error": func(request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, `{"id":"r","status":"completed","is_error":true,"error":{"message":"bad key aiml/Zx9\u0000-echo-key"}}`)
|
||||
},
|
||||
"split key in content": func(request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, completedToolboxRun("page echoing aiml/Zx9\u200b-echo-key"))
|
||||
},
|
||||
"split key in run id": func(request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, strings.Replace(completedToolboxRun("page"), `"id":"run-1"`, `"id":"run aiml/Zx9\u0000-echo-key"`, 1))
|
||||
},
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response { return respond(request) })
|
||||
h.tool.aimlapi.serviceKey = func() (string, error) { return key, nil }
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if output := withoutInvisibleRunes(result.Output); strings.Contains(output, key) || strings.Contains(output, "Zx9-echo-key") || strings.Contains(output, "aiml/Z") {
|
||||
t.Fatalf("the key leaked: %q", result.Output)
|
||||
}
|
||||
for metaKey, value := range result.Meta {
|
||||
if strings.Contains(withoutInvisibleRunes(value), "Zx9-echo-key") {
|
||||
t.Fatalf("the key leaked in meta[%s]: %q", metaKey, value)
|
||||
}
|
||||
}
|
||||
// The cut case keeps only the start of the marker.
|
||||
if !strings.Contains(result.Output, "[REDAC") && !strings.Contains(result.Meta["run_id"], "[REDAC") {
|
||||
t.Fatalf("expected a redaction marker, got %q", result.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A transport failure is reported by its cause, not by the *url.Error around
|
||||
// it: that quotes the polling URL, whose run id the Toolbox chose and which
|
||||
// carries any key echoed in it percent-escaped, past the key scrubbing.
|
||||
func TestWebFetchAIMLAPITransportErrorOmitsPollingURL(t *testing.T) {
|
||||
for name, separator := range map[string]string{"NUL": `\u0000`, "ZWSP": `\u200b`} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", nil)
|
||||
queued := `{"id":"run_` + splitAIMLAPIKey(webFetchAIMLAPITestKey, separator) + `","status":"queued","next_poll_hint_ms":1}`
|
||||
polls := 0
|
||||
h.tool.aimlapi.client = &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
||||
if request.Method == http.MethodPost {
|
||||
return toolboxJSON(request, http.StatusAccepted, queued), nil
|
||||
}
|
||||
polls++
|
||||
return nil, errors.New("connection reset by peer")
|
||||
})}
|
||||
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
|
||||
if polls != 1 {
|
||||
t.Fatalf("polls = %d, want the failing poll to be sent once", polls)
|
||||
}
|
||||
decoded, err := url.PathUnescape(result.Output)
|
||||
if err != nil {
|
||||
decoded = result.Output
|
||||
}
|
||||
if strings.Contains(withoutInvisibleRunes(decoded), webFetchAIMLAPITestKey) {
|
||||
t.Fatalf("the AI/ML API key leaked into the output: %q", result.Output)
|
||||
}
|
||||
if strings.Contains(result.Output, "/runs/") {
|
||||
t.Fatalf("the polling URL leaked into the output: %q", result.Output)
|
||||
}
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "connection reset by peer") {
|
||||
t.Fatalf("status = %v, output = %q, want the transport failure", result.Status, result.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIRetryDelay(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
header string
|
||||
attempt int
|
||||
want time.Duration
|
||||
}{
|
||||
{"", 1, 500 * time.Millisecond},
|
||||
{"", 2, time.Second},
|
||||
{"3", 1, 3 * time.Second},
|
||||
{"0", 1, 0},
|
||||
{"99999999999999999999", 1, 500 * time.Millisecond},
|
||||
{"99999999999999999", 1, 10 * time.Second},
|
||||
{"86400", 1, 10 * time.Second},
|
||||
{"soon", 2, time.Second},
|
||||
} {
|
||||
if got := webFetchAIMLAPIRetryDelay(tc.header, tc.attempt); got != tc.want {
|
||||
t.Errorf("retry delay(%q, %d) = %v, want %v", tc.header, tc.attempt, got, tc.want)
|
||||
}
|
||||
}
|
||||
if got := webFetchAIMLAPIRetryDelay(time.Now().Add(time.Hour).UTC().Format(http.TimeFormat), 1); got != 10*time.Second {
|
||||
t.Errorf("an HTTP-date Retry-After an hour out = %v, want the 10s cap", got)
|
||||
}
|
||||
if err := sleepWebFetchContext(context.Background(), 0); err != nil {
|
||||
t.Fatalf("zero sleep = %v", err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
if err := sleepWebFetchContext(ctx, time.Hour); !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("cancelled sleep = %v, want context.Canceled", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebFetchAIMLAPIRawRejectedBeforePermission(t *testing.T) {
|
||||
tool := newWebFetchToolWithClientAndResolver(nil, nil)
|
||||
for _, provider := range []string{"aimlapi", ""} {
|
||||
t.Setenv("ZERO_WEBFETCH_PROVIDER", provider)
|
||||
for _, format := range []string{"raw", " RAW ", "markdown", "auto"} {
|
||||
result, rejected := tool.(interface {
|
||||
RejectBeforePermission(map[string]any) (Result, bool)
|
||||
}).RejectBeforePermission(map[string]any{"url": "https://example.com/", "format": format})
|
||||
want := provider == "aimlapi" && strings.EqualFold(strings.TrimSpace(format), "raw")
|
||||
if rejected != want {
|
||||
t.Fatalf("provider=%q format=%q: rejected=%v result=%s", provider, format, rejected, result.Output)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestWebFetchAIMLAPIPollHonorsRetryAfter(t *testing.T) {
|
||||
for _, header := range []string{"2", time.Now().Add(4 * time.Second).UTC().Format(http.TimeFormat)} {
|
||||
t.Run(header, func(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(index int, request *http.Request) *http.Response {
|
||||
if index == 0 {
|
||||
return toolboxJSON(request, http.StatusAccepted, `{"id":"run-1","status":"queued","next_poll_hint_ms":250}`)
|
||||
}
|
||||
if index == 1 {
|
||||
response := toolboxJSON(request, http.StatusTooManyRequests, `{}`)
|
||||
response.Header.Set("Retry-After", header)
|
||||
return response
|
||||
}
|
||||
return toolboxJSON(request, http.StatusOK, webFetchAIMLAPICompletedRun)
|
||||
})
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/"})
|
||||
if result.Status != StatusOK {
|
||||
t.Fatalf("fetch failed: %s", result.Output)
|
||||
}
|
||||
if sleeps := *h.sleeps; len(sleeps) != 2 || sleeps[1] < 2*time.Second {
|
||||
t.Fatalf("Retry-After not honored: %v", sleeps)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestWebFetchAIMLAPIOversizedRunIsRejected(t *testing.T) {
|
||||
h := newWebFetchAIMLAPIHarness(t, "aimlapi", func(_ int, request *http.Request) *http.Response {
|
||||
return toolboxJSON(request, http.StatusOK, completedToolboxRun(strings.Repeat("x", webFetchAIMLAPIResponseLimit)))
|
||||
})
|
||||
result := h.tool.Run(context.Background(), map[string]any{"url": "https://example.com/", "max_bytes": 32})
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "larger than") {
|
||||
t.Fatalf("oversized envelope accepted: %s", result.Output)
|
||||
}
|
||||
}
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/redaction"
|
||||
zeroSandbox "github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
@@ -43,6 +44,18 @@ type searchBackend interface {
|
||||
Search(ctx context.Context, query string, limit int) ([]searchResult, error)
|
||||
}
|
||||
|
||||
// timedSearchBackend is implemented by a backend that needs a time budget other
|
||||
// than webSearchTimeout.
|
||||
type timedSearchBackend interface {
|
||||
searchTimeout() time.Duration
|
||||
}
|
||||
|
||||
// domainScopedSearchBackend is implemented by a backend that can restrict a
|
||||
// search to an allowlist of hostnames itself. Run still filters the results.
|
||||
type domainScopedSearchBackend interface {
|
||||
searchDomains(ctx context.Context, query string, limit int, domains []string) ([]searchResult, error)
|
||||
}
|
||||
|
||||
type webSearchTool struct {
|
||||
baseTool
|
||||
backend searchBackend
|
||||
@@ -132,20 +145,33 @@ func (tool webSearchTool) Run(ctx context.Context, args map[string]any) Result {
|
||||
}
|
||||
|
||||
if tool.backend == nil {
|
||||
return errorResult("Error: no search backend configured. Set ZERO_WEBSEARCH_BASE_URL (and ZERO_WEBSEARCH_API_KEY) to enable web_search.")
|
||||
return errorResult("Error: no search backend configured. Set ZERO_WEBSEARCH_BASE_URL (and ZERO_WEBSEARCH_API_KEY), or ZERO_WEBSEARCH_PROVIDER=aimlapi to use the AI/ML API Toolbox (billed per search), to enable web_search.")
|
||||
}
|
||||
|
||||
runCtx, cancel := context.WithTimeout(ctx, webSearchTimeout)
|
||||
timeout := webSearchTimeout
|
||||
if timed, ok := tool.backend.(timedSearchBackend); ok {
|
||||
timeout = timed.searchTimeout()
|
||||
}
|
||||
runCtx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
|
||||
results, err := tool.backend.Search(runCtx, query, limit)
|
||||
var results []searchResult
|
||||
scopedSearch := false
|
||||
if scoped, ok := tool.backend.(domainScopedSearchBackend); ok && len(domains) > 0 {
|
||||
scopedSearch = true
|
||||
results, err = scoped.searchDomains(runCtx, query, limit, domains)
|
||||
} else {
|
||||
results, err = tool.backend.Search(runCtx, query, limit)
|
||||
}
|
||||
if err != nil {
|
||||
return errorResult("Error performing web search: " + redactWebSearchText(err.Error()))
|
||||
}
|
||||
// Filter by domains BEFORE the empty-result short-circuit so a non-empty
|
||||
// allowlist that ate every hit surfaces as a clear "no results matched
|
||||
// domains" error rather than a misleading "no results" message.
|
||||
if len(results) > 0 && len(domains) > 0 {
|
||||
// domains" error rather than a misleading "no results" message. A scoped
|
||||
// backend applied the allowlist itself, so its empty result gets the same
|
||||
// error; an unscoped backend's empty result is an ordinary "no results".
|
||||
if len(domains) > 0 && (len(results) > 0 || scopedSearch) {
|
||||
filtered, err := filterWebSearchByDomains(results, domains)
|
||||
if err != nil {
|
||||
return errorResult("Error: " + redactWebSearchText(err.Error()))
|
||||
@@ -188,9 +214,18 @@ func formatSearchResults(results []searchResult) string {
|
||||
return strings.Join(lines, "\n")
|
||||
}
|
||||
|
||||
// defaultSearchBackend returns the env-configured generic backend, or nil when
|
||||
// ZERO_WEBSEARCH_BASE_URL is unset (the tool then reports it as unconfigured).
|
||||
// defaultSearchBackend returns the env-configured backend, or nil when none is
|
||||
// configured (the tool then reports it as unconfigured).
|
||||
//
|
||||
// ZERO_WEBSEARCH_PROVIDER=aimlapi selects the AI/ML API Toolbox. It ignores
|
||||
// ZERO_WEBSEARCH_BASE_URL and ZERO_WEBSEARCH_API_KEY: the Toolbox host is fixed
|
||||
// and the key is the AI/ML API service key. Otherwise the generic backend is
|
||||
// used when ZERO_WEBSEARCH_BASE_URL is set.
|
||||
func defaultSearchBackend() searchBackend {
|
||||
provider := strings.TrimSpace(os.Getenv("ZERO_WEBSEARCH_PROVIDER"))
|
||||
if strings.EqualFold(provider, aimlapiSearchProvider) {
|
||||
return newAIMLAPISearchBackend(nil, config.LoadAIMLAPIServiceKey)
|
||||
}
|
||||
baseURL := strings.TrimSpace(os.Getenv("ZERO_WEBSEARCH_BASE_URL"))
|
||||
if baseURL == "" {
|
||||
return nil
|
||||
@@ -202,7 +237,7 @@ func defaultSearchBackend() searchBackend {
|
||||
},
|
||||
baseURL: baseURL,
|
||||
apiKey: strings.TrimSpace(os.Getenv("ZERO_WEBSEARCH_API_KEY")),
|
||||
provider: strings.TrimSpace(os.Getenv("ZERO_WEBSEARCH_PROVIDER")),
|
||||
provider: provider,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,325 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/redaction"
|
||||
)
|
||||
|
||||
const (
|
||||
// aimlapiSearchProvider is the ZERO_WEBSEARCH_PROVIDER value that selects the
|
||||
// AI/ML API Toolbox. It is the only way to select it: every run is billed to
|
||||
// the user, so a configured AI/ML API key never enables it by itself.
|
||||
aimlapiSearchProvider = "aimlapi"
|
||||
aimlapiSearchTool = "web/search"
|
||||
// aimlapiSearchMaxCostUSD caps the spend of one search. web/search costs
|
||||
// $0.0013 per result, at least $0.0091 and at most $0.0221 (20 results), so
|
||||
// the cap always covers a run; the Toolbox refuses a run it would exceed
|
||||
// before billing it.
|
||||
aimlapiSearchMaxCostUSD = 0.025
|
||||
|
||||
// Toolbox searches typically finish in about 2-3 s, but a run can queue.
|
||||
// wait_ms asks the Toolbox to hold the POST open until the run ends or that
|
||||
// time passes, so most searches need no polling; the rest of the budget is
|
||||
// left for polling a run that is still queued. The generic backends keep
|
||||
// webSearchTimeout.
|
||||
aimlapiSearchTimeout = 30 * time.Second
|
||||
aimlapiSearchWaitMS = 20_000
|
||||
|
||||
aimlapiDefaultPollDelay = time.Second
|
||||
aimlapiMinPollDelay = 250 * time.Millisecond
|
||||
aimlapiMaxPollDelay = 5 * time.Second
|
||||
aimlapiMaxErrorTextRunes = 300
|
||||
)
|
||||
|
||||
// aimlapiSearchBackend runs the Toolbox tool web/search. The destination is
|
||||
// the fixed aimlapi.ToolboxBaseURL (ZERO_WEBSEARCH_BASE_URL is ignored), and
|
||||
// the key is the AI/ML API service key, resolved again on every search so a
|
||||
// provider change is seen by the next call.
|
||||
type aimlapiSearchBackend struct {
|
||||
client *http.Client
|
||||
loadKey func() (string, error)
|
||||
}
|
||||
|
||||
// newAIMLAPISearchBackend builds the backend; a nil transport means
|
||||
// http.DefaultTransport.
|
||||
func newAIMLAPISearchBackend(transport http.RoundTripper, loadKey func() (string, error)) *aimlapiSearchBackend {
|
||||
return &aimlapiSearchBackend{
|
||||
client: &http.Client{
|
||||
Transport: transport,
|
||||
Timeout: aimlapiSearchTimeout,
|
||||
// Never follow a redirect: the key and the attribution headers must
|
||||
// reach no host but the Toolbox.
|
||||
CheckRedirect: func(*http.Request, []*http.Request) error {
|
||||
return errors.New("refusing redirect from the AI/ML API Toolbox")
|
||||
},
|
||||
},
|
||||
loadKey: loadKey,
|
||||
}
|
||||
}
|
||||
|
||||
func (backend *aimlapiSearchBackend) searchTimeout() time.Duration {
|
||||
return aimlapiSearchTimeout
|
||||
}
|
||||
|
||||
func (backend *aimlapiSearchBackend) Search(ctx context.Context, query string, limit int) ([]searchResult, error) {
|
||||
return backend.searchDomains(ctx, query, limit, nil)
|
||||
}
|
||||
|
||||
// searchDomains passes the normalized allowlist as include_domains so the
|
||||
// Toolbox filters server-side; Run still applies filterWebSearchByDomains.
|
||||
func (backend *aimlapiSearchBackend) searchDomains(ctx context.Context, query string, limit int, domains []string) ([]searchResult, error) {
|
||||
key, err := backend.loadKey()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("AI/ML API search is unavailable: %w", err)
|
||||
}
|
||||
idempotencyKey, err := aimlapi.NewPaymentSessionID()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("generate idempotency key: %w", err)
|
||||
}
|
||||
input := map[string]any{"query": query, "max_results": limit}
|
||||
if len(domains) > 0 {
|
||||
input["include_domains"] = domains
|
||||
}
|
||||
payload, err := json.Marshal(map[string]any{
|
||||
"tool": aimlapiSearchTool,
|
||||
"input": input,
|
||||
"max_cost_usd": aimlapiSearchMaxCostUSD,
|
||||
"wait_ms": aimlapiSearchWaitMS,
|
||||
// A fresh key per search: the Toolbox coalesces a retried POST onto the
|
||||
// same run instead of billing a second one.
|
||||
"idempotency_key": idempotencyKey,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("encode AI/ML API search request: %w", err)
|
||||
}
|
||||
|
||||
run, err := backend.do(ctx, http.MethodPost, aimlapi.ToolboxBaseURL+"/run", payload, key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for !aimlapiTerminalRunStatus(run.Status) && !run.IsError {
|
||||
if strings.TrimSpace(run.ID) == "" {
|
||||
return nil, errors.New("AI/ML API search returned a queued run without an id")
|
||||
}
|
||||
timer := time.NewTimer(aimlapiPollDelay(run.NextPollHintMS))
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
timer.Stop()
|
||||
return nil, fmt.Errorf("AI/ML API search run %s did not finish: %w", aimlapiErrorText(run.ID, key), ctx.Err())
|
||||
case <-timer.C:
|
||||
}
|
||||
run, err = backend.do(ctx, http.MethodGet, aimlapi.ToolboxBaseURL+"/runs/"+url.PathEscape(run.ID), nil, key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return aimlapiRunResults(run, key)
|
||||
}
|
||||
|
||||
type aimlapiErrorBody struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
RequestID string `json:"request_id"`
|
||||
}
|
||||
|
||||
type aimlapiRun struct {
|
||||
ID string `json:"id"`
|
||||
Status string `json:"status"`
|
||||
IsError bool `json:"is_error"`
|
||||
NextPollHintMS *float64 `json:"next_poll_hint_ms"`
|
||||
Error *aimlapiErrorBody `json:"error"`
|
||||
Output *struct {
|
||||
Results []struct {
|
||||
URL string `json:"url"`
|
||||
Title string `json:"title"`
|
||||
Snippet string `json:"snippet"`
|
||||
} `json:"results"`
|
||||
} `json:"output"`
|
||||
}
|
||||
|
||||
// do sends one Toolbox request and decodes the run object it returns.
|
||||
func (backend *aimlapiSearchBackend) do(ctx context.Context, method, target string, payload []byte, key string) (aimlapiRun, error) {
|
||||
var body io.Reader
|
||||
if payload != nil {
|
||||
body = bytes.NewReader(payload)
|
||||
}
|
||||
request, err := newAIMLAPIToolboxRequest(ctx, method, target, body, key)
|
||||
if err != nil {
|
||||
return aimlapiRun{}, err
|
||||
}
|
||||
response, err := backend.client.Do(request)
|
||||
if err != nil {
|
||||
// Report the cause without the *url.Error around it: that quotes the
|
||||
// request URL, and a polling URL carries the run id the Toolbox returned
|
||||
// percent-escaped, a form of any key echoed in it that scrubbing cannot
|
||||
// match.
|
||||
cause := err
|
||||
var urlErr *url.Error
|
||||
if errors.As(err, &urlErr) {
|
||||
cause = urlErr.Err
|
||||
}
|
||||
return aimlapiRun{}, fmt.Errorf("AI/ML API search request failed: %s", aimlapiErrorText(cause.Error(), key))
|
||||
}
|
||||
defer response.Body.Close()
|
||||
raw, err := io.ReadAll(io.LimitReader(response.Body, webSearchBodyLimit))
|
||||
if err != nil {
|
||||
return aimlapiRun{}, fmt.Errorf("read AI/ML API search response: %s", aimlapiErrorText(err.Error(), key))
|
||||
}
|
||||
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
||||
return aimlapiRun{}, aimlapiHTTPError(response, raw, key)
|
||||
}
|
||||
var run aimlapiRun
|
||||
if err := json.Unmarshal(raw, &run); err != nil {
|
||||
return aimlapiRun{}, fmt.Errorf("AI/ML API search returned an invalid response (HTTP %d)", response.StatusCode)
|
||||
}
|
||||
return run, nil
|
||||
}
|
||||
|
||||
// newAIMLAPIToolboxRequest builds a Toolbox request and attaches the key and
|
||||
// the attribution headers only after checking that target is an aimlapi.com
|
||||
// service URL; any other target is refused before anything is sent.
|
||||
func newAIMLAPIToolboxRequest(ctx context.Context, method, target string, body io.Reader, key string) (*http.Request, error) {
|
||||
request, err := http.NewRequestWithContext(ctx, method, target, body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build AI/ML API search request: %w", err)
|
||||
}
|
||||
if !aimlapi.IsAttributedURL(request.URL) {
|
||||
return nil, fmt.Errorf("AI/ML API search refused to send credentials to %q", request.URL.Redacted())
|
||||
}
|
||||
request.Header.Set("Accept", "application/json")
|
||||
if body != nil {
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
request.Header.Set("User-Agent", "zero-web-search/0.1")
|
||||
request.Header.Set("Authorization", "Bearer "+key)
|
||||
aimlapi.ApplyAttributionHeaders(request.Header, request.URL)
|
||||
return request, nil
|
||||
}
|
||||
|
||||
// aimlapiHTTPError reports a refused request by status, error code, message
|
||||
// and request id. The raw body is never surfaced, and the key is scrubbed from
|
||||
// every field taken from it.
|
||||
func aimlapiHTTPError(response *http.Response, raw []byte, key string) error {
|
||||
var envelope struct {
|
||||
Error *aimlapiErrorBody `json:"error"`
|
||||
}
|
||||
_ = json.Unmarshal(raw, &envelope)
|
||||
var parts aimlapiErrorBody
|
||||
if envelope.Error != nil {
|
||||
parts = *envelope.Error
|
||||
}
|
||||
text := fmt.Sprintf("AI/ML API search error HTTP %d", response.StatusCode)
|
||||
if code := aimlapiErrorText(parts.Code, key); code != "" {
|
||||
text += " " + code
|
||||
}
|
||||
switch {
|
||||
case response.StatusCode == http.StatusUnauthorized:
|
||||
text += " (check the AI/ML API key)"
|
||||
case parts.Code == "insufficient_funds":
|
||||
text += " (top up your AI/ML API balance)"
|
||||
case response.StatusCode == http.StatusTooManyRequests:
|
||||
if seconds, err := strconv.Atoi(strings.TrimSpace(response.Header.Get("Retry-After"))); err == nil && seconds > 0 {
|
||||
text += fmt.Sprintf(" (rate limited; retry after %ds)", seconds)
|
||||
} else {
|
||||
text += " (rate limited)"
|
||||
}
|
||||
}
|
||||
if message := aimlapiErrorText(parts.Message, key); message != "" {
|
||||
text += ": " + message
|
||||
}
|
||||
if requestID := aimlapiErrorText(parts.RequestID, key); requestID != "" {
|
||||
text += " (request_id: " + requestID + ")"
|
||||
}
|
||||
return errors.New(text)
|
||||
}
|
||||
|
||||
// aimlapiRunResults maps a finished run onto search results. A run that ended
|
||||
// in any status but completed, or reports is_error (which a completed run can),
|
||||
// is a failure.
|
||||
func aimlapiRunResults(run aimlapiRun, key string) ([]searchResult, error) {
|
||||
status := strings.TrimSpace(run.Status)
|
||||
if status == "" {
|
||||
status = "unknown"
|
||||
}
|
||||
if run.IsError || status != "completed" {
|
||||
text := fmt.Sprintf("AI/ML API %s run %s", aimlapiSearchTool, aimlapiErrorText(status, key))
|
||||
if run.IsError && status == "completed" {
|
||||
text += " with an error"
|
||||
}
|
||||
if run.Error != nil {
|
||||
if code := aimlapiErrorText(run.Error.Code, key); code != "" {
|
||||
text += " " + code
|
||||
}
|
||||
if message := aimlapiErrorText(run.Error.Message, key); message != "" {
|
||||
text += ": " + message
|
||||
}
|
||||
}
|
||||
if id := aimlapiErrorText(run.ID, key); id != "" {
|
||||
text += " (run_id: " + id + ")"
|
||||
}
|
||||
return nil, errors.New(text)
|
||||
}
|
||||
if run.Output == nil || run.Output.Results == nil {
|
||||
return nil, fmt.Errorf("AI/ML API %s returned no results list", aimlapiSearchTool)
|
||||
}
|
||||
out := make([]searchResult, 0, len(run.Output.Results))
|
||||
for _, item := range run.Output.Results {
|
||||
if strings.TrimSpace(item.URL) == "" {
|
||||
continue
|
||||
}
|
||||
out = append(out, searchResult{Title: scrubAIMLAPIResult(item.Title, key), URL: scrubAIMLAPIResult(item.URL, key), Snippet: scrubAIMLAPIResult(item.Snippet, key)})
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func aimlapiTerminalRunStatus(status string) bool {
|
||||
switch strings.TrimSpace(status) {
|
||||
case "completed", "failed", "stopped", "timed_out":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// aimlapiPollDelay honours next_poll_hint_ms, clamped to 250 ms..5 s.
|
||||
func aimlapiPollDelay(hintMS *float64) time.Duration {
|
||||
if hintMS == nil {
|
||||
return aimlapiDefaultPollDelay
|
||||
}
|
||||
delay := time.Duration(*hintMS * float64(time.Millisecond))
|
||||
if delay < aimlapiMinPollDelay {
|
||||
return aimlapiMinPollDelay
|
||||
}
|
||||
if delay > aimlapiMaxPollDelay {
|
||||
return aimlapiMaxPollDelay
|
||||
}
|
||||
return delay
|
||||
}
|
||||
|
||||
// aimlapiErrorText scrubs the key from a field taken from a response, then
|
||||
// collapses whitespace and clips it. Scrubbing runs first so clipping can never
|
||||
// leave a fragment of the key behind.
|
||||
func aimlapiErrorText(value, key string) string {
|
||||
text := redaction.RedactString(value, redaction.Options{ExtraSecretValues: []string{key}})
|
||||
text = strings.Join(strings.Fields(text), " ")
|
||||
if runes := []rune(text); len(runes) > aimlapiMaxErrorTextRunes {
|
||||
text = string(runes[:aimlapiMaxErrorTextRunes]) + "…"
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
// scrubAIMLAPIResult redacts the active credential without clipping page content.
|
||||
func scrubAIMLAPIResult(value, key string) string {
|
||||
return redaction.RedactString(value, redaction.Options{ExtraSecretValues: []string{key}})
|
||||
}
|
||||
@@ -0,0 +1,702 @@
|
||||
package tools
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"github.com/Twigpine/zero/internal/aimlapi"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
)
|
||||
|
||||
// Trimmed from a real 200 web/search run.
|
||||
const aimlapiCompletedRun = `{"id":"h3X2expTh3D7vd4pwyxiY","tool":"web/search","status":"completed","is_error":false,"untrusted":true,
|
||||
"output":{"results":[
|
||||
{"url":"https://pkg.go.dev/net/http","title":"http package - net/http - Go Packages","snippet":"For control over proxies, TLS configuration, keep-alives..."},
|
||||
{"url":"https://go.dev/src/net/http/client.go?m=text","title":"go.dev","snippet":"// Timeout specifies a time limit for requests made by this Client."},
|
||||
{"url":"https://stackoverflow.com/questions/16895294/how-to-set-timeout-for-http-get-requests-in-golang","title":"How to set timeout for http.Get() requests in Golang?","snippet":"Apparently in Go 1.3 http.Client has Timeout field","published_at":"2013-11-02T00:00:00.000Z"}
|
||||
]},
|
||||
"cost":{"quoted_usd":0.0091,"actual_usd":0.0091,"over_max":false}}`
|
||||
|
||||
// A real 400 upfront refusal (not billed).
|
||||
const aimlapiMaxCostExceeded = `{"error":{"message":"The cheapest source costs 0.0091 USD, above the limit of 0.001 USD.","type":"invalid_request_error","code":"max_cost_exceeded","request_id":"hnbQUvsseM1EX9RrgPjBT","refused_by":"request","estimated_cost_usd":0.0091,"max_cost_usd":0.001}}`
|
||||
|
||||
type toolboxCall struct {
|
||||
method string
|
||||
url string
|
||||
header http.Header
|
||||
body map[string]any
|
||||
remaining time.Duration
|
||||
}
|
||||
|
||||
// fakeToolbox is an http.RoundTripper that records every request and answers
|
||||
// from respond, so no request leaves the process.
|
||||
type fakeToolbox struct {
|
||||
mu sync.Mutex
|
||||
calls []toolboxCall
|
||||
respond func(index int, request *http.Request) *http.Response
|
||||
}
|
||||
|
||||
func (fake *fakeToolbox) RoundTrip(request *http.Request) (*http.Response, error) {
|
||||
call := toolboxCall{method: request.Method, url: request.URL.String(), header: request.Header.Clone()}
|
||||
if deadline, ok := request.Context().Deadline(); ok {
|
||||
call.remaining = time.Until(deadline)
|
||||
}
|
||||
if request.Body != nil {
|
||||
raw, _ := io.ReadAll(request.Body)
|
||||
_ = request.Body.Close()
|
||||
if len(raw) > 0 {
|
||||
_ = json.Unmarshal(raw, &call.body)
|
||||
}
|
||||
}
|
||||
fake.mu.Lock()
|
||||
index := len(fake.calls)
|
||||
fake.calls = append(fake.calls, call)
|
||||
fake.mu.Unlock()
|
||||
// Like a real transport, refuse a request whose context has already ended.
|
||||
if err := request.Context().Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
response := fake.respond(index, request)
|
||||
response.Request = request
|
||||
return response, nil
|
||||
}
|
||||
|
||||
func (fake *fakeToolbox) recorded() []toolboxCall {
|
||||
fake.mu.Lock()
|
||||
defer fake.mu.Unlock()
|
||||
return append([]toolboxCall(nil), fake.calls...)
|
||||
}
|
||||
|
||||
func toolboxResponse(status int, body string) *http.Response {
|
||||
return &http.Response{
|
||||
StatusCode: status,
|
||||
Header: http.Header{"Content-Type": []string{"application/json"}},
|
||||
Body: io.NopCloser(strings.NewReader(body)),
|
||||
}
|
||||
}
|
||||
|
||||
func alwaysRespond(status int, body string) func(int, *http.Request) *http.Response {
|
||||
return func(int, *http.Request) *http.Response { return toolboxResponse(status, body) }
|
||||
}
|
||||
|
||||
func staticKey(key string) func() (string, error) {
|
||||
return func() (string, error) { return key, nil }
|
||||
}
|
||||
|
||||
// isolateAIMLAPISearchEnv points every config, cache and credential root at a
|
||||
// temporary home and clears the variables the backend selection and the key
|
||||
// resolution read.
|
||||
func isolateAIMLAPISearchEnv(t *testing.T) {
|
||||
t.Helper()
|
||||
home := t.TempDir()
|
||||
t.Setenv("HOME", home)
|
||||
t.Setenv("USERPROFILE", home)
|
||||
t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config"))
|
||||
t.Setenv("XDG_CACHE_HOME", filepath.Join(home, ".cache"))
|
||||
t.Setenv("APPDATA", filepath.Join(home, "AppData", "Roaming"))
|
||||
t.Setenv("LOCALAPPDATA", filepath.Join(home, "AppData", "Local"))
|
||||
t.Setenv("ZERO_CRED_STORAGE", "file")
|
||||
t.Setenv("ZERO_PROVIDER", "")
|
||||
t.Setenv("AIMLAPI_API_KEY", "")
|
||||
t.Setenv("AIMLAPI_INFERENCE_URL", "")
|
||||
t.Setenv("AIMLAPI_PARTNER_ID", "")
|
||||
t.Setenv("ZERO_WEBSEARCH_PROVIDER", "")
|
||||
t.Setenv("ZERO_WEBSEARCH_BASE_URL", "")
|
||||
t.Setenv("ZERO_WEBSEARCH_API_KEY", "")
|
||||
}
|
||||
|
||||
func hasWebSearchTool(tools []Tool) bool {
|
||||
for _, tool := range tools {
|
||||
if tool.Name() == "web_search" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func TestWebSearchAIMLAPISelectedOnlyByExplicitProvider(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
provider string
|
||||
baseURL string
|
||||
want bool
|
||||
}{
|
||||
{name: "key alone", provider: "", want: false},
|
||||
{name: "other provider", provider: "searxng", want: false},
|
||||
{name: "explicit", provider: "aimlapi", want: true},
|
||||
{name: "case-insensitive", provider: " AIMLAPI ", want: true},
|
||||
{name: "base URL ignored", provider: "aimlapi", baseURL: "https://search.example/api", want: true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
t.Setenv("AIMLAPI_API_KEY", "aiml-env-key")
|
||||
t.Setenv("ZERO_WEBSEARCH_PROVIDER", tc.provider)
|
||||
t.Setenv("ZERO_WEBSEARCH_BASE_URL", tc.baseURL)
|
||||
|
||||
_, isAIMLAPI := defaultSearchBackend().(*aimlapiSearchBackend)
|
||||
if isAIMLAPI != tc.want {
|
||||
t.Fatalf("defaultSearchBackend() is aimlapi = %v, want %v", isAIMLAPI, tc.want)
|
||||
}
|
||||
if registered := hasWebSearchTool(CoreNetworkTools()); registered != tc.want && tc.baseURL == "" {
|
||||
t.Fatalf("web_search registered = %v, want %v", registered, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebSearchAIMLAPIDefaultBackendUsesServiceKeyAndFixedHost(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
t.Setenv("ZERO_WEBSEARCH_PROVIDER", "aimlapi")
|
||||
t.Setenv("ZERO_WEBSEARCH_BASE_URL", "https://search.example/api")
|
||||
t.Setenv("ZERO_WEBSEARCH_API_KEY", "zws-generic-key")
|
||||
t.Setenv("AIMLAPI_API_KEY", "aiml-env-key")
|
||||
|
||||
backend, ok := defaultSearchBackend().(*aimlapiSearchBackend)
|
||||
if !ok {
|
||||
t.Fatalf("defaultSearchBackend() = %T, want *aimlapiSearchBackend", defaultSearchBackend())
|
||||
}
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusOK, aimlapiCompletedRun)}
|
||||
backend.client.Transport = fake // keeps the production redirect policy and key loader
|
||||
|
||||
res := newWebSearchToolWithBackend(backend).Run(context.Background(), map[string]any{"query": "go http timeout"})
|
||||
if res.Status != StatusOK {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
calls := fake.recorded()
|
||||
if len(calls) != 1 {
|
||||
t.Fatalf("requests = %d, want 1", len(calls))
|
||||
}
|
||||
if calls[0].url != aimlapi.ToolboxBaseURL+"/run" {
|
||||
t.Fatalf("request URL = %q, want the fixed Toolbox URL", calls[0].url)
|
||||
}
|
||||
if got := calls[0].header.Get("Authorization"); got != "Bearer aiml-env-key" {
|
||||
t.Fatalf("Authorization = %q, want the AI/ML API service key", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchSendsExactRequest(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusOK, aimlapiCompletedRun)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
for range 2 {
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "go http timeout", "limit": 3})
|
||||
if res.Status != StatusOK {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
for _, want := range []string{
|
||||
"1. http package - net/http - Go Packages — https://pkg.go.dev/net/http",
|
||||
" For control over proxies",
|
||||
"3. How to set timeout for http.Get() requests in Golang?",
|
||||
} {
|
||||
if !strings.Contains(res.Output, want) {
|
||||
t.Fatalf("output missing %q:\n%s", want, res.Output)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
calls := fake.recorded()
|
||||
if len(calls) != 2 {
|
||||
t.Fatalf("requests = %d, want 2", len(calls))
|
||||
}
|
||||
uuid := regexp.MustCompile(`^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$`)
|
||||
for _, call := range calls {
|
||||
if call.method != http.MethodPost || call.url != "https://tools.aimlapi.com/v1/tools/run" {
|
||||
t.Fatalf("request = %s %s, want POST https://tools.aimlapi.com/v1/tools/run", call.method, call.url)
|
||||
}
|
||||
for header, want := range map[string]string{
|
||||
"Authorization": "Bearer test-aiml-key",
|
||||
aimlapi.SourceHeaderName: "agent/zero",
|
||||
aimlapi.PartnerHeaderName: aimlapi.DefaultPartnerID,
|
||||
"User-Agent": "zero-web-search/0.1",
|
||||
"Content-Type": "application/json",
|
||||
} {
|
||||
if got := call.header.Get(header); got != want {
|
||||
t.Fatalf("%s = %q, want %q", header, got, want)
|
||||
}
|
||||
}
|
||||
body := call.body
|
||||
if body["tool"] != "web/search" {
|
||||
t.Fatalf("tool = %v, want web/search", body["tool"])
|
||||
}
|
||||
if body["max_cost_usd"] != 0.025 {
|
||||
t.Fatalf("max_cost_usd = %v, want 0.025", body["max_cost_usd"])
|
||||
}
|
||||
if body["wait_ms"] != float64(20000) {
|
||||
t.Fatalf("wait_ms = %v, want 20000", body["wait_ms"])
|
||||
}
|
||||
input, _ := body["input"].(map[string]any)
|
||||
if input["query"] != "go http timeout" || input["max_results"] != float64(3) {
|
||||
t.Fatalf("input = %v, want query and max_results 3", input)
|
||||
}
|
||||
if _, ok := input["include_domains"]; ok {
|
||||
t.Fatalf("include_domains sent without a domains argument: %v", input)
|
||||
}
|
||||
if key, _ := body["idempotency_key"].(string); !uuid.MatchString(key) {
|
||||
t.Fatalf("idempotency_key = %q, want a UUIDv4", key)
|
||||
}
|
||||
// The aimlapi budget replaces the generic 10 s one, and the server-side
|
||||
// wait stays inside it.
|
||||
if call.remaining <= webSearchTimeout || call.remaining > aimlapiSearchTimeout {
|
||||
t.Fatalf("request deadline in %v, want within (%v, %v]", call.remaining, webSearchTimeout, aimlapiSearchTimeout)
|
||||
}
|
||||
}
|
||||
if calls[0].body["idempotency_key"] == calls[1].body["idempotency_key"] {
|
||||
t.Fatalf("idempotency_key reused across searches: %v", calls[0].body["idempotency_key"])
|
||||
}
|
||||
if time.Duration(aimlapiSearchWaitMS)*time.Millisecond >= aimlapiSearchTimeout {
|
||||
t.Fatalf("wait_ms %d must stay below the %v budget", aimlapiSearchWaitMS, aimlapiSearchTimeout)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchResolvesKeyOnEveryCall(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusOK, aimlapiCompletedRun)}
|
||||
loads := 0
|
||||
backend := newAIMLAPISearchBackend(fake, func() (string, error) {
|
||||
loads++
|
||||
if loads == 1 {
|
||||
return "first-key", nil
|
||||
}
|
||||
return "", config.ErrAIMLAPIKeyMissing
|
||||
})
|
||||
tool := newWebSearchToolWithBackend(backend)
|
||||
|
||||
if res := tool.Run(context.Background(), map[string]any{"query": "q"}); res.Status != StatusOK {
|
||||
t.Fatalf("first search: status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
// The key was removed after the first search: the next one must see that.
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, "no AI/ML API key is configured") {
|
||||
t.Fatalf("second search: status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
if calls := fake.recorded(); len(calls) != 1 || calls[0].header.Get("Authorization") != "Bearer first-key" {
|
||||
t.Fatalf("requests = %+v, want exactly the first search", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchSendsNothingWithoutUsableKey(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
env map[string]string
|
||||
want string
|
||||
}{
|
||||
{name: "missing", want: config.ErrAIMLAPIKeyMissing.Error()},
|
||||
{name: "proxy", env: map[string]string{"AIMLAPI_API_KEY": "proxy-key", "AIMLAPI_INFERENCE_URL": "https://proxy.example/v1"}, want: config.ErrAIMLAPIProxyProfile.Error()},
|
||||
{name: "invalid", env: map[string]string{"AIMLAPI_API_KEY": "two words"}, want: config.ErrAIMLAPIKeyInvalid.Error()},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
for name, value := range tc.env {
|
||||
t.Setenv(name, value)
|
||||
}
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusOK, aimlapiCompletedRun)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, config.LoadAIMLAPIServiceKey))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, tc.want) {
|
||||
t.Fatalf("status = %v, output = %q, want an error mentioning %q", res.Status, res.Output, tc.want)
|
||||
}
|
||||
if calls := fake.recorded(); len(calls) != 0 {
|
||||
t.Fatalf("sent %d requests without a usable key", len(calls))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchPollsQueuedRun(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: func(index int, _ *http.Request) *http.Response {
|
||||
switch index {
|
||||
case 0:
|
||||
return toolboxResponse(http.StatusAccepted, `{"id":"run/1","status":"queued","next_poll_hint_ms":1}`)
|
||||
case 1:
|
||||
return toolboxResponse(http.StatusOK, `{"id":"run/1","status":"running","is_error":false,"next_poll_hint_ms":1}`)
|
||||
default:
|
||||
return toolboxResponse(http.StatusOK, aimlapiCompletedRun)
|
||||
}
|
||||
}}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusOK || !strings.Contains(res.Output, "https://pkg.go.dev/net/http") {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
calls := fake.recorded()
|
||||
if len(calls) != 3 {
|
||||
t.Fatalf("requests = %d, want POST then two polls", len(calls))
|
||||
}
|
||||
for _, call := range calls[1:] {
|
||||
if call.method != http.MethodGet || call.url != "https://tools.aimlapi.com/v1/tools/runs/run%2F1" {
|
||||
t.Fatalf("poll = %s %s, want GET of the run status", call.method, call.url)
|
||||
}
|
||||
if call.header.Get("Authorization") != "Bearer test-aiml-key" || call.header.Get(aimlapi.SourceHeaderName) != "agent/zero" {
|
||||
t.Fatalf("poll headers = %v, want the key and attribution", call.header)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIPollDelayIsClamped(t *testing.T) {
|
||||
ms := func(value float64) *float64 { return &value }
|
||||
cases := []struct {
|
||||
hint *float64
|
||||
want time.Duration
|
||||
}{
|
||||
{hint: nil, want: time.Second},
|
||||
{hint: ms(1), want: 250 * time.Millisecond},
|
||||
{hint: ms(1200), want: 1200 * time.Millisecond},
|
||||
{hint: ms(60_000), want: 5 * time.Second},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := aimlapiPollDelay(tc.hint); got != tc.want {
|
||||
t.Fatalf("aimlapiPollDelay(%v) = %v, want %v", tc.hint, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchStopsPollingWhenContextEnds(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusAccepted, `{"id":"run_1","status":"queued","next_poll_hint_ms":5000}`)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
|
||||
defer cancel()
|
||||
|
||||
started := time.Now()
|
||||
res := tool.Run(ctx, map[string]any{"query": "q"})
|
||||
if elapsed := time.Since(started); elapsed > 2*time.Second {
|
||||
t.Fatalf("search took %v after the context ended", elapsed)
|
||||
}
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, "did not finish") {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchCompletedRunWithIsErrorFails(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusOK, `{"id":"run_9","status":"completed","is_error":true,
|
||||
"error":{"code":"source_failed","message":"upstream search failed"},
|
||||
"output":{"results":[{"url":"https://example.com/partial","title":"partial"}]}}`)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError {
|
||||
t.Fatalf("status = %v, output = %q, want an error", res.Status, res.Output)
|
||||
}
|
||||
for _, want := range []string{"source_failed", "upstream search failed", "run_9"} {
|
||||
if !strings.Contains(res.Output, want) {
|
||||
t.Fatalf("output missing %q: %q", want, res.Output)
|
||||
}
|
||||
}
|
||||
if strings.Contains(res.Output, "example.com/partial") {
|
||||
t.Fatalf("results of a failed run were shown: %q", res.Output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchTerminalFailureStatuses(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
for _, status := range []string{"failed", "stopped", "timed_out"} {
|
||||
t.Run(status, func(t *testing.T) {
|
||||
fake := &fakeToolbox{respond: func(index int, _ *http.Request) *http.Response {
|
||||
if index > 0 { // a poll would mean the status was not treated as terminal
|
||||
return toolboxResponse(http.StatusInternalServerError, `{}`)
|
||||
}
|
||||
return toolboxResponse(http.StatusOK, `{"id":"run_2","status":"`+status+`","is_error":false}`)
|
||||
}}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, "run "+status) {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
if calls := fake.recorded(); len(calls) != 1 {
|
||||
t.Fatalf("requests = %d, want no polling after a terminal status", len(calls))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchMalformedResponses(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
cases := map[string]struct{ body, want string }{
|
||||
"not json": {body: `<html>oops</html>`, want: "invalid response"},
|
||||
"no results": {body: `{"id":"run_3","status":"completed","is_error":false,"output":{}}`, want: "no results list"},
|
||||
}
|
||||
for name, tc := range cases {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusOK, tc.body)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, tc.want) {
|
||||
t.Fatalf("status = %v, output = %q, want an error mentioning %q", res.Status, res.Output, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchMaxCostExceededReportsCodeAndRequestID(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusBadRequest, aimlapiMaxCostExceeded)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
for _, want := range []string{"HTTP 400", "max_cost_exceeded", "cheapest source costs", "request_id: hnbQUvsseM1EX9RrgPjBT"} {
|
||||
if !strings.Contains(res.Output, want) {
|
||||
t.Fatalf("output missing %q: %q", want, res.Output)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchRateLimitedReportsRetryAfter(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: func(int, *http.Request) *http.Response {
|
||||
response := toolboxResponse(http.StatusTooManyRequests, `{"error":{"code":"rate_limited","message":"slow down","request_id":"req_429"}}`)
|
||||
response.Header.Set("Retry-After", "7")
|
||||
return response
|
||||
}}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
for _, want := range []string{"HTTP 429 rate_limited", "retry after 7s", "request_id: req_429"} {
|
||||
if !strings.Contains(res.Output, want) {
|
||||
t.Fatalf("output missing %q: %q", want, res.Output)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchRefusesRedirect(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
for _, location := range []string{"https://collector.example/run", "https://tools.aimlapi.com/v1/tools/run"} {
|
||||
t.Run(location, func(t *testing.T) {
|
||||
fake := &fakeToolbox{respond: func(index int, _ *http.Request) *http.Response {
|
||||
if index == 0 {
|
||||
response := toolboxResponse(http.StatusTemporaryRedirect, ``)
|
||||
response.Header.Set("Location", location)
|
||||
return response
|
||||
}
|
||||
return toolboxResponse(http.StatusOK, aimlapiCompletedRun)
|
||||
}}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, "refusing redirect") {
|
||||
t.Fatalf("status = %v, output = %q, want a refused redirect", res.Status, res.Output)
|
||||
}
|
||||
if calls := fake.recorded(); len(calls) != 1 {
|
||||
t.Fatalf("requests = %d, want the redirect not followed", len(calls))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchRedactsKeyEchoedInErrors(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
const key = "zq7Mv2Lp9Xr4Tn8Wk3"
|
||||
if redacted := redactWebSearchText("key " + key); !strings.Contains(redacted, key) {
|
||||
t.Fatalf("precondition: the generic redactor already hides this key shape (%q); pick another", redacted)
|
||||
}
|
||||
bodies := map[string]struct {
|
||||
status int
|
||||
body string
|
||||
}{
|
||||
"http error": {status: http.StatusUnauthorized, body: `{"error":{"code":"invalid_api_key","message":"API key ` + key + ` is not valid","request_id":"req_` + key + `"}}`},
|
||||
"run error": {status: http.StatusOK, body: `{"id":"run_` + key + `","status":"failed","is_error":true,"error":{"code":"bad_auth","message":"rejected ` + key + `"}}`},
|
||||
// The same key split by characters nobody sees: a reader that drops
|
||||
// them gets the key back, so the split form has to be redacted too.
|
||||
"http error split by NUL": {status: http.StatusUnauthorized, body: `{"error":{"code":"invalid_api_key","message":"API key ` + splitAIMLAPIKey(key, `\u0000`) + ` is not valid","request_id":"req_` + splitAIMLAPIKey(key, `\u200b`) + `"}}`},
|
||||
"run error split by ZWSP": {status: http.StatusOK, body: `{"id":"run_` + splitAIMLAPIKey(key, `\u200b`) + `","status":"failed","is_error":true,"error":{"code":"bad_auth","message":"rejected ` + splitAIMLAPIKey(key, `\u0000`) + `"}}`},
|
||||
}
|
||||
for name, tc := range bodies {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
fake := &fakeToolbox{respond: alwaysRespond(tc.status, tc.body)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey(key)))
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
if strings.Contains(withoutInvisibleRunes(res.Output), key) {
|
||||
t.Fatalf("the AI/ML API key leaked into the output: %q", res.Output)
|
||||
}
|
||||
if !strings.Contains(res.Output, "req_") && !strings.Contains(res.Output, "run_") {
|
||||
t.Fatalf("output lost the request or run id: %q", res.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A failed request must not echo its URL. The polling URL carries the run id
|
||||
// the Toolbox returned, percent-escaped, and an escaped key ("%00",
|
||||
// "%E2%80%8B") is not the text the key scrubber matches.
|
||||
func TestAIMLAPISearchTransportErrorOmitsPollingURL(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
const key = "zq7Mv2Lp9Xr4Tn8Wk3"
|
||||
for name, separator := range map[string]string{"NUL": `\u0000`, "ZWSP": `\u200b`} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
queued := `{"id":"run_` + splitAIMLAPIKey(key, separator) + `","status":"queued","next_poll_hint_ms":1}`
|
||||
var polls int
|
||||
transport := roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
||||
if request.Method == http.MethodPost {
|
||||
response := toolboxResponse(http.StatusAccepted, queued)
|
||||
response.Request = request
|
||||
return response, nil
|
||||
}
|
||||
polls++
|
||||
return nil, errors.New("connection reset by peer")
|
||||
})
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(transport, staticKey(key)))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if polls != 1 {
|
||||
t.Fatalf("polls = %d, want the failing poll to be sent once", polls)
|
||||
}
|
||||
decoded, err := url.PathUnescape(res.Output)
|
||||
if err != nil {
|
||||
decoded = res.Output
|
||||
}
|
||||
if strings.Contains(withoutInvisibleRunes(decoded), key) {
|
||||
t.Fatalf("the AI/ML API key leaked into the output: %q", res.Output)
|
||||
}
|
||||
if strings.Contains(res.Output, "/runs/") {
|
||||
t.Fatalf("the polling URL leaked into the output: %q", res.Output)
|
||||
}
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, "AI/ML API search request failed: connection reset by peer") {
|
||||
t.Fatalf("status = %v, output = %q, want the transport failure", res.Status, res.Output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPIToolboxRequestRefusesOtherDestinations(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
for _, target := range []string{
|
||||
"https://tools.aimlapi.com.evil.example/v1/tools/run",
|
||||
"http://tools.aimlapi.com/v1/tools/run",
|
||||
"https://tools.aimlapi.com:8443/v1/tools/run",
|
||||
"https://[email protected]/v1/tools/run",
|
||||
"https://search.example/api",
|
||||
} {
|
||||
request, err := newAIMLAPIToolboxRequest(context.Background(), http.MethodPost, target, strings.NewReader(`{}`), "test-aiml-key")
|
||||
if err == nil || request != nil {
|
||||
t.Fatalf("newAIMLAPIToolboxRequest(%q) = %v, %v; want a refusal", target, request, err)
|
||||
}
|
||||
if strings.Contains(err.Error(), "test-aiml-key") {
|
||||
t.Fatalf("refusal leaked the key: %v", err)
|
||||
}
|
||||
}
|
||||
request, err := newAIMLAPIToolboxRequest(context.Background(), http.MethodGet, aimlapi.ToolboxBaseURL+"/runs/x", nil, "test-aiml-key")
|
||||
if err != nil {
|
||||
t.Fatalf("Toolbox URL refused: %v", err)
|
||||
}
|
||||
if request.Header.Get("Authorization") != "Bearer test-aiml-key" || request.Header.Get(aimlapi.SourceHeaderName) != "agent/zero" {
|
||||
t.Fatalf("headers = %v, want key and attribution on the Toolbox URL", request.Header)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchDomainsAreSentAndStillFiltered(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: alwaysRespond(http.StatusOK, aimlapiCompletedRun)}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-aiml-key")))
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{
|
||||
"query": "go http timeout",
|
||||
"domains": []any{"https://WWW.Go.dev/doc", "pkg.go.dev", "go.dev"},
|
||||
})
|
||||
if res.Status != StatusOK {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
calls := fake.recorded()
|
||||
if len(calls) != 1 {
|
||||
t.Fatalf("requests = %d, want 1", len(calls))
|
||||
}
|
||||
input, _ := calls[0].body["input"].(map[string]any)
|
||||
if got, _ := json.Marshal(input["include_domains"]); string(got) != `["go.dev","pkg.go.dev"]` {
|
||||
t.Fatalf("include_domains = %s, want the normalized allowlist", got)
|
||||
}
|
||||
// The Toolbox filter is not trusted alone: a result outside the allowlist
|
||||
// is still dropped.
|
||||
if strings.Contains(res.Output, "stackoverflow.com") {
|
||||
t.Fatalf("result outside the allowlist was shown: %q", res.Output)
|
||||
}
|
||||
if !strings.Contains(res.Output, "https://pkg.go.dev/net/http") || !strings.Contains(res.Output, "https://go.dev/src/net/http/client.go") {
|
||||
t.Fatalf("allowed results missing: %q", res.Output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebSearchUnconfiguredHintNamesAIMLAPI(t *testing.T) {
|
||||
res := newWebSearchToolWithBackend(nil).Run(context.Background(), map[string]any{"query": "q"})
|
||||
if res.Status != StatusError || !strings.Contains(res.Output, "ZERO_WEBSEARCH_PROVIDER=aimlapi") {
|
||||
t.Fatalf("status = %v, output = %q", res.Status, res.Output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchRedactsKeyInSuccessfulResults(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
const key = "zq7Mv2Lp9Xr4Tn8Wk3"
|
||||
fake := &fakeToolbox{respond: func(_ int, _ *http.Request) *http.Response {
|
||||
return toolboxResponse(http.StatusOK, `{"status":"completed","output":{"results":[{"title":"`+key+`","url":"https://example.com/`+key+`","snippet":"echo `+key+`"}]}}`)
|
||||
}}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey(key)))
|
||||
result := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if result.Status != StatusOK || strings.Contains(result.Output, key) {
|
||||
t.Fatalf("successful result leaked key: %s", result.Output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIMLAPISearchRedactsSplitKeyInSuccessfulResults(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
const key = "zq7Mv2Lp9Xr4Tn8Wk3"
|
||||
fake := &fakeToolbox{respond: func(_ int, _ *http.Request) *http.Response {
|
||||
return toolboxResponse(http.StatusOK, `{"status":"completed","output":{"results":[{"title":"`+splitAIMLAPIKey(key, `\u0000`)+`","url":"https://example.com/`+splitAIMLAPIKey(key, `\u200b`)+`","snippet":"echo `+splitAIMLAPIKey(key, `\u2060`)+`"}]}}`)
|
||||
}}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey(key)))
|
||||
result := tool.Run(context.Background(), map[string]any{"query": "q"})
|
||||
if result.Status != StatusOK {
|
||||
t.Fatalf("status = %v, output = %q", result.Status, result.Output)
|
||||
}
|
||||
if strings.Contains(withoutInvisibleRunes(result.Output), key) {
|
||||
t.Fatalf("successful result leaked the split key: %q", result.Output)
|
||||
}
|
||||
}
|
||||
|
||||
// splitAIMLAPIKey puts a JSON-escaped invisible character into the middle of
|
||||
// key, as a response that echoes it that way would.
|
||||
func splitAIMLAPIKey(key, escapedSeparator string) string {
|
||||
return key[:len(key)/2] + escapedSeparator + key[len(key)/2:]
|
||||
}
|
||||
|
||||
// withoutInvisibleRunes drops the control and format characters a terminal,
|
||||
// log viewer or copy-paste would never show, so a key split by one reads whole.
|
||||
func withoutInvisibleRunes(value string) string {
|
||||
return strings.Map(func(r rune) rune {
|
||||
if r != '\t' && r != '\n' && r != '\r' && (unicode.Is(unicode.Cc, r) || unicode.Is(unicode.Cf, r)) {
|
||||
return -1
|
||||
}
|
||||
return r
|
||||
}, value)
|
||||
}
|
||||
func TestAIMLAPISearchEmptyScopedResultsReportAllowlist(t *testing.T) {
|
||||
isolateAIMLAPISearchEnv(t)
|
||||
fake := &fakeToolbox{respond: func(_ int, _ *http.Request) *http.Response {
|
||||
return toolboxResponse(http.StatusOK, `{"status":"completed","output":{"results":[]}}`)
|
||||
}}
|
||||
tool := newWebSearchToolWithBackend(newAIMLAPISearchBackend(fake, staticKey("test-key")))
|
||||
result := tool.Run(context.Background(), map[string]any{"query": "q", "domains": []any{"example.com"}})
|
||||
if result.Status != StatusError || !strings.Contains(result.Output, "loosen the allowlist") {
|
||||
t.Fatalf("missing allowlist guidance: %s", result.Output)
|
||||
}
|
||||
}
|
||||
@@ -105,6 +105,7 @@ func TestWebSearchRedactsBackendError(t *testing.T) {
|
||||
func TestWebSearchRegisteredInCoreNetworkTools(t *testing.T) {
|
||||
// web_search is registered only when a backend is configured.
|
||||
t.Setenv("ZERO_WEBSEARCH_BASE_URL", "https://search.example/api")
|
||||
t.Setenv("ZERO_WEBSEARCH_PROVIDER", "")
|
||||
found := false
|
||||
for _, tool := range CoreNetworkTools() {
|
||||
if tool.Name() == "web_search" {
|
||||
@@ -300,6 +301,20 @@ func TestWebSearchDomainsFilterEmptyResultIsError(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebSearchDomainsUnscopedEmptyResultIsNoResults(t *testing.T) {
|
||||
// A backend that cannot scope by domain never saw the allowlist, so an empty
|
||||
// result is an ordinary "no results", not an allowlist that ate every hit.
|
||||
tool := newWebSearchToolWithBackend(&fakeSearchBackend{})
|
||||
|
||||
res := tool.Run(context.Background(), map[string]any{
|
||||
"query": "x",
|
||||
"domains": []any{"react.dev"},
|
||||
})
|
||||
if res.Status != StatusOK || !strings.Contains(res.Output, "No results for query") {
|
||||
t.Fatalf("want ordinary no-results for an unscoped backend, got %v: %s", res.Status, res.Output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWebSearchDomainsFilterToleratesStringSlice(t *testing.T) {
|
||||
// Some providers send []string instead of []any. The tool must accept both.
|
||||
backend := &fakeSearchBackend{results: []searchResult{
|
||||
|
||||
+77
-9
@@ -91,6 +91,20 @@ func (m model) handleBTWCommand(question string) (model, tea.Cmd) {
|
||||
parent.flushQueue = nil
|
||||
|
||||
side := parent
|
||||
// The side surface must own its file-view lifecycle. Because `side` is a
|
||||
// value copy of `parent`, its liveSeq pointer would otherwise alias the
|
||||
// hidden parent's: closing or switching the view in BTW would revoke the
|
||||
// parent's in-flight load. Detach it and give the side a fresh lifetime so
|
||||
// inherited markers cannot masquerade as the parent's snapshot.
|
||||
side.fileView.liveSeq = nil
|
||||
if side.fileView.active {
|
||||
side.fileView.lifetimeToken = nextFileViewLifetimeToken()
|
||||
side.fileView.loadedToken = [16]byte{}
|
||||
side.fileView.loadedSeq = 0
|
||||
side.fileView.loadedRev = 0
|
||||
side.fileView.snapshotReady = false
|
||||
side.fileView.renderedContent = ""
|
||||
}
|
||||
side.activeSession = fork
|
||||
side.sessionEvents = events
|
||||
side.transcript = initialTranscript()
|
||||
@@ -146,7 +160,7 @@ func (m model) handleBTWCommand(question string) (model, tea.Cmd) {
|
||||
side.plan.clear()
|
||||
side.planDetailGen++
|
||||
side.streamingText = nil
|
||||
side.streamingReasoning = ""
|
||||
side.streamingReasoning = nil
|
||||
side.streamingReasoningExpanded = false
|
||||
side.clearStreamingToolCall()
|
||||
side.resetStreamingFade()
|
||||
@@ -156,10 +170,16 @@ func (m model) handleBTWCommand(question string) (model, tea.Cmd) {
|
||||
sideRunIDBase: side.runID,
|
||||
}
|
||||
|
||||
if question == "" {
|
||||
return side, nil
|
||||
var btwFileViewCmd tea.Cmd
|
||||
if side.fileView.active && side.fileView.mode == fileViewFull {
|
||||
side, btwFileViewCmd = side.startFileViewLoadCmd(side.chatColumnWidth())
|
||||
}
|
||||
return side.launchPrompt(question)
|
||||
|
||||
if question == "" {
|
||||
return side, btwFileViewCmd
|
||||
}
|
||||
next, launchCmd := side.launchPrompt(question)
|
||||
return next, batchCommands(btwFileViewCmd, launchCmd)
|
||||
}
|
||||
|
||||
func (m model) leaveBTW() (model, tea.Cmd) {
|
||||
@@ -175,6 +195,10 @@ func (m model) leaveBTW() (model, tea.Cmd) {
|
||||
if m.compactInFlight {
|
||||
return m.appendSystemNotice("BTW compaction is still running. Wait for it to finish before returning."), nil
|
||||
}
|
||||
// The side surface is being discarded. Revoke its file-view lifetime token so
|
||||
// a load still queued behind BTW cannot run or land after the view closes.
|
||||
// The restored parent keeps its own request and snapshot untouched.
|
||||
m.revokeFileViewRequest()
|
||||
m, _ = m.clearLoopsForSessionSwitch()
|
||||
parent := *m.btw.parent
|
||||
parent.goalContinuationsSuspended = false
|
||||
@@ -201,7 +225,9 @@ func (m model) leaveBTW() (model, tea.Cmd) {
|
||||
parent.resetFlushFrontier("· returned from btw ·")
|
||||
var goalCmd tea.Cmd
|
||||
parent, goalCmd = parent.launchGoalContinuationIfReady()
|
||||
return parent, batchCommands(sweepCmd, spinnerCmd, goalCmd)
|
||||
var fileCmd tea.Cmd
|
||||
parent, fileCmd = parent.recoverInvalidatedFileView()
|
||||
return parent, batchCommands(sweepCmd, spinnerCmd, goalCmd, fileCmd)
|
||||
}
|
||||
|
||||
func btwCommandUnavailable(command parsedCommand) bool {
|
||||
@@ -236,9 +262,9 @@ func btwCommandUnavailable(command parsedCommand) bool {
|
||||
// messages have no run ID, so normal BTW routing intentionally leaves them on
|
||||
// the visible side surface; copying the layout fields prevents stale wrapping
|
||||
// after the parent is restored.
|
||||
func (m model) resizeBTWParent(msg tea.WindowSizeMsg) model {
|
||||
func (m model) resizeBTWParent(msg tea.WindowSizeMsg) (model, tea.Cmd) {
|
||||
if !m.btw.active || m.btw.parent == nil {
|
||||
return m
|
||||
return m, nil
|
||||
}
|
||||
parent := *m.btw.parent
|
||||
parent.width = msg.Width
|
||||
@@ -247,8 +273,15 @@ func (m model) resizeBTWParent(msg tea.WindowSizeMsg) model {
|
||||
parent.lineAges = nil
|
||||
parent.lastStreamActivity = parent.now()
|
||||
parent.input.SetWidth(maxInt(20, chatWidth(msg.Width)-14))
|
||||
// The parent's full-file view wrapped at the old width; reload it at the new
|
||||
// one. Its completion has no run ID, so routeBTWParentMessage matches it by
|
||||
// lifetime token and delivers it to the hidden parent.
|
||||
var cmd tea.Cmd
|
||||
if parent.fileView.active && parent.fileView.mode == fileViewFull {
|
||||
parent, cmd = parent.startFileViewLoadCmd(parent.chatColumnWidth())
|
||||
}
|
||||
m.btw.parent = &parent
|
||||
return m
|
||||
return m, cmd
|
||||
}
|
||||
|
||||
func btwTitle(parent string) string {
|
||||
@@ -272,6 +305,22 @@ func (m model) routeBTWParentMessage(msg tea.Msg) (model, tea.Cmd, bool) {
|
||||
}
|
||||
return m.routeBTWMessageToParent(msg)
|
||||
}
|
||||
if loaded, ok := msg.(fileViewLoadedMsg); ok {
|
||||
// File-view completions carry no run ID. Deliver the hidden parent's
|
||||
// load (matched by lifetime token) so it does not stay stuck on
|
||||
// "Loading…" while BTW is active; side-owned loads fall through.
|
||||
if m.btw.parent.fileView.active && m.btw.parent.fileView.lifetimeToken == loaded.lifetimeToken {
|
||||
return m.routeBTWMessageToParent(msg)
|
||||
}
|
||||
return m, nil, false
|
||||
}
|
||||
if revalidated, ok := msg.(fileViewRevalidatedMsg); ok {
|
||||
// A revalidation the parent started before /btw is routed the same way.
|
||||
if m.btw.parent.fileView.active && m.btw.parent.fileView.lifetimeToken == revalidated.lifetimeToken {
|
||||
return m.routeBTWMessageToParent(msg)
|
||||
}
|
||||
return m, nil, false
|
||||
}
|
||||
runID, ok := btwMessageRunID(msg)
|
||||
if !ok || runID <= 0 || runID >= m.btw.sideRunIDBase {
|
||||
return m, nil, false
|
||||
@@ -279,6 +328,21 @@ func (m model) routeBTWParentMessage(msg tea.Msg) (model, tea.Cmd, bool) {
|
||||
return m.routeBTWMessageToParent(msg)
|
||||
}
|
||||
|
||||
// refreshHiddenParentFileView reloads the hidden parent's full-file view when a
|
||||
// side-surface mutation advanced the shared path revision. The load completion
|
||||
// carries the parent's lifetime token, so routeBTWParentMessage delivers it
|
||||
// back to the hidden model without waiting for a git sweep or for /btw return.
|
||||
func (m model) refreshHiddenParentFileView() (model, tea.Cmd) {
|
||||
if !m.btw.active || m.btw.parent == nil {
|
||||
return m, nil
|
||||
}
|
||||
parent := *m.btw.parent
|
||||
var cmd tea.Cmd
|
||||
parent, cmd = parent.recoverInvalidatedFileView()
|
||||
m.btw.parent = &parent
|
||||
return m, cmd
|
||||
}
|
||||
|
||||
func (m model) routeBTWMessageToParent(msg tea.Msg) (model, tea.Cmd, bool) {
|
||||
parentNext, cmd := m.btw.parent.updateModel(msg)
|
||||
parent, ok := parentNext.(model)
|
||||
@@ -296,7 +360,9 @@ func (m model) routeBTWMessageToParent(msg tea.Msg) (model, tea.Cmd, bool) {
|
||||
case agentResponseMsg:
|
||||
m.btw.parentNeedsInput = parent.pendingPermission != nil || parent.pendingAskUser != nil
|
||||
}
|
||||
return m, cmd, true
|
||||
var fileCmd tea.Cmd
|
||||
m, fileCmd = m.recoverInvalidatedFileView()
|
||||
return m, batchCommands(cmd, fileCmd), true
|
||||
}
|
||||
|
||||
func btwMessageRunID(msg tea.Msg) (int, bool) {
|
||||
@@ -327,6 +393,8 @@ func btwMessageRunID(msg tea.Msg) (int, bool) {
|
||||
return typed.runID, true
|
||||
case specialistProgressMsg:
|
||||
return typed.runID, true
|
||||
case unknownScopeMutationMsg:
|
||||
return typed.runID, true
|
||||
case swarmSessionsMsg:
|
||||
return typed.runID, true
|
||||
case permissionRequestMsg:
|
||||
|
||||
@@ -3,6 +3,7 @@ package tui
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
@@ -535,3 +536,112 @@ func TestBTWCtrlCDuringRunDoesNotClearDraft(t *testing.T) {
|
||||
t.Fatalf("missing in-flight return guidance: %#v", got.transcript)
|
||||
}
|
||||
}
|
||||
|
||||
// TestBTWLeaveRevokesSideFileLoad verifies that leaving BTW revokes the side
|
||||
// surface's file-view lifetime so a still-queued side load cannot run or land
|
||||
// after the view closes, without cancelling the restored parent's own load.
|
||||
func TestBTWLeaveRevokesSideFileLoad(t *testing.T) {
|
||||
resetFileViewCacheForTest()
|
||||
m := newBTWTestModel(t)
|
||||
dir := t.TempDir()
|
||||
m.cwd = dir
|
||||
name := "side_view.go"
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte("package side\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Open a full view so the side surface inherits an active file view.
|
||||
m, parentLoadCmd := m.openFileView(name)
|
||||
if parentLoadCmd == nil {
|
||||
t.Fatal("expected a pending parent load command")
|
||||
}
|
||||
parentLiveSeq := m.fileView.liveSeq
|
||||
if parentLiveSeq == nil {
|
||||
t.Fatal("parent load token missing")
|
||||
}
|
||||
parentSeq := parentLiveSeq.Load()
|
||||
|
||||
// Enter BTW: the side surface detaches and schedules its own load.
|
||||
side, sideLoadCmd := m.handleBTWCommand("")
|
||||
if !side.btw.active || side.btw.parent == nil {
|
||||
t.Fatal("expected an active BTW conversation")
|
||||
}
|
||||
if sideLoadCmd == nil {
|
||||
t.Fatal("side surface should schedule its own file load")
|
||||
}
|
||||
sideLiveSeq := side.fileView.liveSeq
|
||||
if sideLiveSeq == nil || sideLiveSeq == parentLiveSeq {
|
||||
t.Fatal("side load must own a detached lifetime token")
|
||||
}
|
||||
sideSeq := sideLiveSeq.Load()
|
||||
|
||||
// Return while the side load is still queued.
|
||||
parent, _ := side.leaveBTW()
|
||||
if sideLiveSeq.Load() == sideSeq {
|
||||
t.Fatal("leaveBTW must revoke the side file-view request")
|
||||
}
|
||||
if parentLiveSeq.Load() != parentSeq || parentLiveSeq == sideLiveSeq {
|
||||
t.Fatal("leaveBTW must not revoke the restored parent's request")
|
||||
}
|
||||
|
||||
// The held side command must now be a superseded no-op.
|
||||
held := sideLoadCmd()
|
||||
loaded, ok := held.(fileViewLoadedMsg)
|
||||
if !ok {
|
||||
t.Fatalf("held side command produced %T, want fileViewLoadedMsg", held)
|
||||
}
|
||||
if !errors.Is(loaded.err, errFileViewSuperseded) {
|
||||
t.Fatalf("queued side load must be superseded after leaving BTW, got %v", loaded.err)
|
||||
}
|
||||
|
||||
// The parent's own load still completes and settles its snapshot.
|
||||
parent = deliverCommandMessages(t, parent, parentLoadCmd)
|
||||
if parent.fileView.loading {
|
||||
t.Fatal("parent load must remain valid after the side revocation")
|
||||
}
|
||||
if !strings.Contains(plainRender(t, parent.renderFileViewFull(80)), "package side") {
|
||||
t.Fatal("restored parent must keep its snapshot")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBTWForkOfErroredFileViewShowsLoadingPlaceholder verifies that a side
|
||||
// surface forked from a full view whose last snapshot was a read error does
|
||||
// not render an empty body while its own load is pending. The inherited error
|
||||
// snapshot is invalidated by the fork, so the side must show the loading
|
||||
// placeholder until its load lands, then the side's own error snapshot.
|
||||
func TestBTWForkOfErroredFileViewShowsLoadingPlaceholder(t *testing.T) {
|
||||
resetFileViewCacheForTest()
|
||||
m := newBTWTestModel(t)
|
||||
m.cwd = t.TempDir()
|
||||
|
||||
// A missing file opens straight in full mode and settles on a read error.
|
||||
m, loadCmd := m.openFileView("missing.go")
|
||||
if loadCmd == nil || m.fileView.mode != fileViewFull {
|
||||
t.Fatal("expected a full-view load for a file without recorded edits")
|
||||
}
|
||||
m = deliverCommandMessages(t, m, loadCmd)
|
||||
if !m.fileView.hasError {
|
||||
t.Fatal("parent full view should settle on a read error")
|
||||
}
|
||||
if got := plainRender(t, m.renderFileViewFull(80)); !strings.Contains(got, "Could not read file") {
|
||||
t.Fatalf("parent should show the read error, got %q", got)
|
||||
}
|
||||
|
||||
side, sideLoadCmd := m.handleBTWCommand("")
|
||||
if !side.btw.active || sideLoadCmd == nil {
|
||||
t.Fatal("expected an active BTW surface with its own pending file load")
|
||||
}
|
||||
if got := plainRender(t, side.renderFileViewFull(80)); !strings.Contains(got, fileViewLoadingPlaceholder) {
|
||||
t.Fatalf("pending side load must show the loading placeholder, got %q", got)
|
||||
}
|
||||
if got := plainRender(t, side.btw.parent.renderFileViewFull(80)); !strings.Contains(got, "Could not read file") {
|
||||
t.Fatalf("hidden parent must keep its own error snapshot, got %q", got)
|
||||
}
|
||||
|
||||
side = deliverCommandMessages(t, side, sideLoadCmd)
|
||||
if side.fileView.loading {
|
||||
t.Fatal("side load should settle")
|
||||
}
|
||||
if got := plainRender(t, side.renderFileViewFull(80)); !strings.Contains(got, "Could not read file") {
|
||||
t.Fatalf("settled side load should show its read error, got %q", got)
|
||||
}
|
||||
}
|
||||
+67
-33
@@ -7,19 +7,27 @@ import (
|
||||
tea "charm.land/bubbletea/v2"
|
||||
)
|
||||
|
||||
// streamCoalesceInterval is roughly one 60fps frame. Assistant-text deltas that
|
||||
// arrive within this window are merged into a single agentTextMsg, so the render
|
||||
// rate decouples from the token rate: a fast local model (100+ tok/s) no longer
|
||||
// forces 100+ full Update→View cycles (each re-parsing the growing markdown) per
|
||||
// second. Rendering stays smooth regardless of provider speed.
|
||||
// streamCoalesceInterval is roughly one 60fps frame. Assistant-text and
|
||||
// reasoning deltas that arrive within this window are merged into a single
|
||||
// message, so the render rate decouples from the token rate: a fast local
|
||||
// model (100+ tok/s) no longer forces 100+ full Update→View cycles (each
|
||||
// re-rendering the growing markdown or thinking block) per second. Reasoning
|
||||
// matters as much as text here: interimBlock re-renders the ENTIRE accumulated
|
||||
// thinking string every frame, so uncoalesced reasoning deltas are O(n) work
|
||||
// per token (O(n²) per segment) and the message queue backs up faster than it
|
||||
// drains — the UI keeps replaying "thoughts" long after the provider stopped
|
||||
// thinking. Rendering stays smooth regardless of provider speed.
|
||||
const streamCoalesceInterval = 16 * time.Millisecond
|
||||
|
||||
// textCoalescer batches agentTextMsg deltas before forwarding them to the Bubble
|
||||
// Tea program. Any OTHER message flushes the pending text first, so ordering
|
||||
// between streamed prose and tool-call / reasoning / row / usage messages is
|
||||
// preserved. The turn's final agentResponseMsg does not pass through here (it is
|
||||
// a tea.Cmd return, not a sink message), but the model drops deltas whose runID
|
||||
// is no longer active, so a flush that races just past end-of-turn is harmless.
|
||||
// textCoalescer batches agentTextMsg and agentReasoningMsg deltas before
|
||||
// forwarding them to the Bubble Tea program. Any OTHER message flushes the
|
||||
// pending stream first, so ordering between streamed prose/reasoning and
|
||||
// tool-call / row / usage messages is preserved. A switch between text and
|
||||
// reasoning (or between runs) also flushes the buffered kind first, so
|
||||
// interleaved thinking and prose never reorder. The turn's final
|
||||
// agentResponseMsg does not pass through here (it is a tea.Cmd return, not a
|
||||
// sink message), but the model drops deltas whose runID is no longer active,
|
||||
// so a flush that races just past end-of-turn is harmless.
|
||||
//
|
||||
// Sink messages originate from the single agent goroutine and so arrive
|
||||
// serially; the only concurrent caller is the flush timer. The mutex guards the
|
||||
@@ -37,7 +45,11 @@ type textCoalescer struct {
|
||||
mu sync.Mutex
|
||||
buf []byte
|
||||
runID int
|
||||
timer coalesceTimer
|
||||
// reasoning records whether the buffered deltas are reasoning (true) or
|
||||
// prose (false). Only one kind is ever buffered at a time: a kind switch
|
||||
// flushes first, which preserves interleaved order.
|
||||
reasoning bool
|
||||
timer coalesceTimer
|
||||
}
|
||||
|
||||
// coalesceTimer is the subset of *time.Timer the coalescer needs. Abstracted
|
||||
@@ -60,50 +72,72 @@ func (c *textCoalescer) send(msg tea.Msg) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
|
||||
text, ok := msg.(agentTextMsg)
|
||||
if !ok {
|
||||
// Non-text message: flush buffered text first (preserving order), then
|
||||
var (
|
||||
delta string
|
||||
runID int
|
||||
reasoning bool
|
||||
streaming bool
|
||||
)
|
||||
switch m := msg.(type) {
|
||||
case agentTextMsg:
|
||||
delta, runID, streaming = m.delta, m.runID, true
|
||||
case agentReasoningMsg:
|
||||
delta, runID, reasoning, streaming = m.delta, m.runID, true, true
|
||||
}
|
||||
if !streaming {
|
||||
// Non-stream message: flush buffered stream first (preserving order), then
|
||||
// forward it — both under the lock so nothing can interleave between them.
|
||||
c.drainAndForwardLocked()
|
||||
c.forward(msg)
|
||||
return
|
||||
}
|
||||
|
||||
// A delta for a different run than the one buffered: flush the old run's text
|
||||
// before buffering the new run's. In practice runs are sequential (the prior
|
||||
// run's end already flushed via a non-text message), so this is belt-and-braces.
|
||||
if len(c.buf) > 0 && text.runID != c.runID {
|
||||
// A kind or run switch: flush the buffered kind before buffering the new
|
||||
// one, so interleaved reasoning/prose is delivered in arrival order. In
|
||||
// practice runs are sequential (the prior run's end already flushed via a
|
||||
// non-stream message), so the run switch is belt-and-braces.
|
||||
if len(c.buf) > 0 && (runID != c.runID || reasoning != c.reasoning) {
|
||||
c.drainAndForwardLocked()
|
||||
}
|
||||
c.runID = text.runID
|
||||
c.buf = append(c.buf, text.delta...)
|
||||
c.runID = runID
|
||||
c.reasoning = reasoning
|
||||
c.buf = append(c.buf, delta...)
|
||||
if c.timer == nil {
|
||||
c.timer = c.afterFunc(c.flush)
|
||||
}
|
||||
}
|
||||
|
||||
// flush forwards any buffered text as one agentTextMsg. Runs on the timer
|
||||
// goroutine; the lock it takes serializes it against send so its output can't be
|
||||
// reordered around a concurrent non-text message.
|
||||
// flush forwards any buffered stream deltas as one coalesced message. Runs on
|
||||
// the timer goroutine; the lock it takes serializes it against send so its
|
||||
// output can't be reordered around a concurrent non-stream message.
|
||||
func (c *textCoalescer) flush() {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.drainAndForwardLocked()
|
||||
}
|
||||
|
||||
// drainAndForwardLocked forwards any buffered text as one agentTextMsg and stops
|
||||
// the timer, all while the caller holds c.mu — so a text flush and any non-text
|
||||
// forward are strictly ordered and never interleave. A no-op when nothing is
|
||||
// buffered. string(c.buf) copies, so reusing the backing array via [:0] is safe.
|
||||
// drainAndForwardLocked forwards any buffered deltas as one coalesced
|
||||
// agentTextMsg or agentReasoningMsg (whichever kind was buffered) and stops the
|
||||
// timer, all while the caller holds c.mu — so a flush and any concurrent
|
||||
// forward are strictly ordered and never interleave. Nothing is forwarded when
|
||||
// nothing is buffered, but the timer is still cleared: an empty delta can arm
|
||||
// a timer that fires against an empty buffer, and leaving that timer non-nil
|
||||
// would make send() skip arming a new one, stalling all later deltas until a
|
||||
// kind switch or non-stream message. string(c.buf) copies, so reusing the
|
||||
// backing array via [:0] is safe.
|
||||
func (c *textCoalescer) drainAndForwardLocked() {
|
||||
if len(c.buf) == 0 {
|
||||
return
|
||||
}
|
||||
if c.timer != nil {
|
||||
c.timer.Stop()
|
||||
c.timer = nil
|
||||
}
|
||||
msg := agentTextMsg{runID: c.runID, delta: string(c.buf)}
|
||||
if len(c.buf) == 0 {
|
||||
return
|
||||
}
|
||||
text := string(c.buf)
|
||||
c.buf = c.buf[:0]
|
||||
c.forward(msg)
|
||||
if c.reasoning {
|
||||
c.forward(agentReasoningMsg{runID: c.runID, delta: text})
|
||||
return
|
||||
}
|
||||
c.forward(agentTextMsg{runID: c.runID, delta: text})
|
||||
}
|
||||
@@ -180,3 +180,110 @@ func TestCoalescerTimerFlushes(t *testing.T) {
|
||||
t.Fatalf("timer-flushed message = %#v", got[0])
|
||||
}
|
||||
}
|
||||
|
||||
// An empty delta can arm a timer whose flush fires against an empty buffer.
|
||||
// That fired timer must be cleared, or send() sees a non-nil timer and skips
|
||||
// arming a new one, stalling all later deltas until a kind switch or non-stream
|
||||
// message happens to flush them.
|
||||
func TestCoalescerEmptyDeltaDoesNotStrandTimer(t *testing.T) {
|
||||
rec := &recorder{}
|
||||
c := newTextCoalescer(rec.forward)
|
||||
|
||||
var (
|
||||
mu sync.Mutex
|
||||
armCount int
|
||||
fireLatest func()
|
||||
)
|
||||
c.afterFunc = func(fn func()) coalesceTimer {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
armCount++
|
||||
fireLatest = fn
|
||||
return &manualTimer{}
|
||||
}
|
||||
|
||||
// Empty delta arms a timer; it fires against an empty buffer.
|
||||
c.send(agentTextMsg{runID: 1, delta: ""})
|
||||
mu.Lock()
|
||||
first := armCount
|
||||
mu.Unlock()
|
||||
if first != 1 {
|
||||
t.Fatalf("empty delta should arm exactly one timer, armed %d", first)
|
||||
}
|
||||
fireLatest()
|
||||
|
||||
// A real delta after the fired timer must arm a NEW timer, so it can be
|
||||
// delivered on schedule instead of stalling forever.
|
||||
c.send(agentTextMsg{runID: 1, delta: "after stall"})
|
||||
mu.Lock()
|
||||
second := armCount
|
||||
mu.Unlock()
|
||||
if second != 2 {
|
||||
t.Fatalf("delta after empty-buffer timer fire should re-arm the timer (arm count %d), so later deltas stall", second)
|
||||
}
|
||||
fireLatest()
|
||||
|
||||
got := rec.snapshot()
|
||||
if len(got) != 1 {
|
||||
t.Fatalf("expected exactly one forwarded message, got %d: %#v", len(got), got)
|
||||
}
|
||||
if text, ok := got[0].(agentTextMsg); !ok || text.delta != "after stall" {
|
||||
t.Fatalf("forwarded message = %#v, want agentTextMsg{\"after stall\"}", got[0])
|
||||
}
|
||||
}
|
||||
|
||||
// Rapid reasoning deltas within one frame collapse into a single
|
||||
// agentReasoningMsg, exactly like text deltas. Without this, every reasoning
|
||||
// token forces a full Update→View cycle that re-renders the whole growing
|
||||
// thinking block, so the UI falls behind the stream and keeps "streaming"
|
||||
// thoughts long after the provider finished.
|
||||
func TestCoalescerBatchesReasoningDeltas(t *testing.T) {
|
||||
rec := &recorder{}
|
||||
c := newTextCoalescer(rec.forward)
|
||||
c.afterFunc = func(func()) coalesceTimer { return &manualTimer{} }
|
||||
|
||||
c.send(agentReasoningMsg{runID: 1, delta: "think"})
|
||||
c.send(agentReasoningMsg{runID: 1, delta: "ing…"})
|
||||
|
||||
if got := rec.snapshot(); len(got) != 0 {
|
||||
t.Fatalf("reasoning deltas should buffer, forwarded %d early: %#v", len(got), got)
|
||||
}
|
||||
|
||||
c.flush()
|
||||
|
||||
got := rec.snapshot()
|
||||
if len(got) != 1 {
|
||||
t.Fatalf("expected 1 coalesced message, got %d: %#v", len(got), got)
|
||||
}
|
||||
reasoning, ok := got[0].(agentReasoningMsg)
|
||||
if !ok || reasoning.delta != "thinking…" || reasoning.runID != 1 {
|
||||
t.Fatalf("coalesced message = %#v, want agentReasoningMsg{1, \"thinking…\"}", got[0])
|
||||
}
|
||||
}
|
||||
|
||||
// A text delta following buffered reasoning flushes the reasoning first, and
|
||||
// vice versa, so interleaved thinking/prose never reorders.
|
||||
func TestCoalescerPreservesReasoningTextOrder(t *testing.T) {
|
||||
rec := &recorder{}
|
||||
c := newTextCoalescer(rec.forward)
|
||||
c.afterFunc = func(func()) coalesceTimer { return &manualTimer{} }
|
||||
|
||||
c.send(agentReasoningMsg{runID: 1, delta: "thought"})
|
||||
c.send(agentTextMsg{runID: 1, delta: "answer"})
|
||||
c.send(agentReasoningMsg{runID: 1, delta: "more thought"})
|
||||
c.flush()
|
||||
|
||||
got := rec.snapshot()
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("expected reasoning, text, reasoning in order, got %d: %#v", len(got), got)
|
||||
}
|
||||
if r, ok := got[0].(agentReasoningMsg); !ok || r.delta != "thought" {
|
||||
t.Fatalf("first message must be the flushed reasoning, got %#v", got[0])
|
||||
}
|
||||
if tx, ok := got[1].(agentTextMsg); !ok || tx.delta != "answer" {
|
||||
t.Fatalf("second message must be the flushed text, got %#v", got[1])
|
||||
}
|
||||
if r, ok := got[2].(agentReasoningMsg); !ok || r.delta != "more thought" {
|
||||
t.Fatalf("third message must be the final reasoning, got %#v", got[2])
|
||||
}
|
||||
}
|
||||
@@ -128,6 +128,15 @@ func (c *staticRenderCache) stats() renderCacheStats {
|
||||
return c.statsData
|
||||
}
|
||||
|
||||
func fileViewCacheStatsForTest() fileViewCacheStats {
|
||||
return defaultFileViewCache.stats()
|
||||
}
|
||||
|
||||
func resetFileViewCacheForTest() {
|
||||
defaultFileViewCache.clear()
|
||||
defaultFileViewCache.resetStats()
|
||||
}
|
||||
|
||||
func renderSelectableList(options selectableListOptions) string {
|
||||
if len(options.Items) == 0 {
|
||||
return ""
|
||||
|
||||
Loaded 100 of 129 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user