mirror of
https://github.com/Twigpine/zero.git
synced 2026-11-09 18:08:31 -10:00
Compare commits
45
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f317a79a02 | ||
|
|
90143736a8 | ||
|
|
c8144df06d | ||
|
|
dd763d9921 | ||
|
|
f4df51cc48 | ||
|
|
197026c5e9 | ||
|
|
fa1dbced03 | ||
|
|
88716c4fb1 | ||
|
|
91a25079ce | ||
|
|
b32f7ce4d6 | ||
|
|
8e9c3ec4ce | ||
|
|
ef1ca39b09 | ||
|
|
bbf779e691 | ||
|
|
47f78c6624 | ||
|
|
e034a93815 | ||
|
|
2ca84cc127 | ||
|
|
05c1b5e160 | ||
|
|
16ccd78765 | ||
|
|
c9f179482a | ||
|
|
98736457bd | ||
|
|
e59632406b | ||
|
|
7c55a09c07 | ||
|
|
d99f92b81a | ||
|
|
5aa54fe1d2 | ||
|
|
f912f80d00 | ||
|
|
9c4e3693e7 | ||
|
|
63962081cf | ||
|
|
2e10a4b60b | ||
|
|
4247a07aba | ||
|
|
b837448dd4 | ||
|
|
a0b036a3c7 | ||
|
|
164150633a | ||
|
|
5d080ef46b | ||
|
|
1d233c0a9b | ||
|
|
de1443b2ed | ||
|
|
84c1b15540 | ||
|
|
6393a90ed6 | ||
|
|
cd3e0b0a9f | ||
|
|
aa9ba8412e | ||
|
|
a97db8832f | ||
|
|
ef4da1323f | ||
|
|
1e848492e5 | ||
|
|
2ffe1831b0 | ||
|
|
13e2596b2e | ||
|
|
8c4e759798 |
No files matched your search
@@ -1 +1 @@
|
||||
{".":"0.11.0"}
|
||||
{".":"0.13.0"}
|
||||
@@ -6,6 +6,52 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once the first release is
|
||||
tagged. Until then, source builds report the version `dev`.
|
||||
|
||||
## [0.13.0](https://github.com/Twigpine/zero/compare/v0.12.0...v0.13.0) (2026-11-09)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **mcp:** add AI/ML API as an opt-in built-in server ([#1193](https://github.com/Twigpine/zero/issues/1193)) ([16ccd78](https://github.com/Twigpine/zero/commit/16ccd78765b6a2a56b7370e3031710d718aa4757))
|
||||
* **plugins:** add CLI enable/disable for plugin.json ([#1202](https://github.com/Twigpine/zero/issues/1202)) ([05c1b5e](https://github.com/Twigpine/zero/commit/05c1b5e16098e1c60212316d20ae8e055a27b5ae))
|
||||
* **providers:** point OpenGateway at api.twigpine.com and trust its live model list ([#1208](https://github.com/Twigpine/zero/issues/1208)) ([47f78c6](https://github.com/Twigpine/zero/commit/47f78c66247aa96c8753d6bb95e935126ff1761e))
|
||||
* **web-fetch:** add an opt-in AI/ML API Toolbox scraping backend ([#1192](https://github.com/Twigpine/zero/issues/1192)) ([c9f1794](https://github.com/Twigpine/zero/commit/c9f179482a7f6d7111ee64fa9cb8ffa2517c2397))
|
||||
* **web-search:** add AI/ML API Toolbox as an opt-in backend ([#1189](https://github.com/Twigpine/zero/issues/1189)) ([9873645](https://github.com/Twigpine/zero/commit/98736457bdac6b0272e032728a5485cf557780fe))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **hooks:** dispatch specialistStart/Stop lifecycle hooks ([#1204](https://github.com/Twigpine/zero/issues/1204)) ([2ca84cc](https://github.com/Twigpine/zero/commit/2ca84cc127d9369765004b72cf95049fecc6c28b))
|
||||
* **tui:** make reasoning stream rendering linear (coalesced deltas, O(1) append, tail-bounded preview) ([#1198](https://github.com/Twigpine/zero/issues/1198)) ([bbf779e](https://github.com/Twigpine/zero/commit/bbf779e69168d62bb1103aefdd1b10e53433209e))
|
||||
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
* **tui:** bounded file read, LRU rendering cache, and size caps for full-file view ([#1203](https://github.com/Twigpine/zero/issues/1203)) ([e034a93](https://github.com/Twigpine/zero/commit/e034a93815629c68976dcdab7dcc08bbe9b03693))
|
||||
|
||||
## [0.12.0](https://github.com/Twigpine/zero/compare/v0.11.0...v0.12.0) (2026-11-02)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **providers:** add MiniMax-M3.1-Flash to catalog ([#1180](https://github.com/Twigpine/zero/issues/1180)) ([d99f92b](https://github.com/Twigpine/zero/commit/d99f92b81a671f186cede81614c89b8e8a7f9451))
|
||||
* **sandbox:** support self-exec pattern to eliminate separate helper binary ([#946](https://github.com/Twigpine/zero/issues/946)) ([8c4e759](https://github.com/Twigpine/zero/commit/8c4e7597989ef9b3ca67b7703dfd15d92372aed2))
|
||||
* **skills:** install a skill's bundled files and point the skill tool at them ([#1167](https://github.com/Twigpine/zero/issues/1167)) ([1641506](https://github.com/Twigpine/zero/commit/164150633a8b2e6ae864ae02fd7105b201eeb68e))
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **agent:** mark headless guardrail stops incomplete ([#1182](https://github.com/Twigpine/zero/issues/1182)) ([5aa54fe](https://github.com/Twigpine/zero/commit/5aa54fe1d2b419c9bbef73605a77919c9007cf7b))
|
||||
* bump Go to 1.26.10 for GO-2026-6809 ([#1168](https://github.com/Twigpine/zero/issues/1168)) ([6393a90](https://github.com/Twigpine/zero/commit/6393a90ed618a3536e09c02852f4ec829b97545b))
|
||||
* **chatgpt:** refresh subscription defaults and compatibility version ([#1179](https://github.com/Twigpine/zero/issues/1179)) ([f912f80](https://github.com/Twigpine/zero/commit/f912f80d002602e11ba0ef8548ecee58442874aa))
|
||||
* **deps:** refresh the lockfile to hono 4.13.13 ([#1171](https://github.com/Twigpine/zero/issues/1171)) ([de1443b](https://github.com/Twigpine/zero/commit/de1443b2edb44fd3747cc77200d54dc8cffcd0a7))
|
||||
* **oauth:** reuse the crash-safe secret store and sync encrypted publications ([#1165](https://github.com/Twigpine/zero/issues/1165)) ([84c1b15](https://github.com/Twigpine/zero/commit/84c1b1554076679a342e9cac61f8f8cdbaf3a429))
|
||||
* **redaction:** redact credentials whose trigger a greedy span swallowed ([#1158](https://github.com/Twigpine/zero/issues/1158)) ([a97db88](https://github.com/Twigpine/zero/commit/a97db8832fdba077f2d31bdf03ecef8d7c774fe1))
|
||||
* **sessions:** capture freeform apply_patch checkpoints ([#1183](https://github.com/Twigpine/zero/issues/1183)) ([6396208](https://github.com/Twigpine/zero/commit/63962081cf130c2171407936ce3beb7021d238b8))
|
||||
* **sessions:** preserve image attachment summaries on resume ([#1152](https://github.com/Twigpine/zero/issues/1152)) ([1e84849](https://github.com/Twigpine/zero/commit/1e848492e5c1919d187fb5e1eb5e7ef9bc6140dc)), closes [#1135](https://github.com/Twigpine/zero/issues/1135)
|
||||
* **tools:** grep stops where a file's line breaks stop ([#1160](https://github.com/Twigpine/zero/issues/1160)) ([aa9ba84](https://github.com/Twigpine/zero/commit/aa9ba8412eb8917daa2ca5cb3fdbd0a39276fa88))
|
||||
* **workspaceindex:** import gitsafe under the Twigpine module path ([#1153](https://github.com/Twigpine/zero/issues/1153)) ([ef4da13](https://github.com/Twigpine/zero/commit/ef4da1323f1097863ae278224b5f304c2fff909a))
|
||||
* **workspaceindex:** run Scan's git calls through gitsafe ([#1149](https://github.com/Twigpine/zero/issues/1149)) ([2ffe183](https://github.com/Twigpine/zero/commit/2ffe1831b0950b1463ec46d487d03a5fcb679cc6))
|
||||
|
||||
## [0.11.0](https://github.com/Twigpine/zero/compare/v0.10.0...v0.11.0) (2026-10-15)
|
||||
|
||||
|
||||
|
||||
@@ -12,9 +12,10 @@
|
||||
|
||||
<p align="center">
|
||||
<a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-MIT-blue"></a>
|
||||
<img alt="Go 1.26.9+" src="https://img.shields.io/badge/Go-1.26.9+-00ADD8?logo=go&logoColor=white">
|
||||
<img alt="Go 1.26.10+" src="https://img.shields.io/badge/Go-1.26.10+-00ADD8?logo=go&logoColor=white">
|
||||
<img alt="25+ providers" src="https://img.shields.io/badge/providers-25+-34E2EA">
|
||||
<a href="https://discord.gg/CaQDS6wdFn"><img alt="Discord" src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white"></a>
|
||||
<a href="https://vercel.com/open-source-program"><img alt="Vercel OSS Program" src="https://vercel.com/oss/program-badge-2026.svg"></a>
|
||||
<br>
|
||||
<strong>English</strong> | <a href="README_ZH.md">中文</a>
|
||||
</p>
|
||||
@@ -87,9 +88,18 @@ zero exec --output-format stream-json < turns.jsonl
|
||||
</picture>
|
||||
</a>
|
||||
</td>
|
||||
<td align="center" width="150" height="80">
|
||||
<a href="https://vercel.com/">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/vercel-logo-dark.svg">
|
||||
<img src="docs/assets/vercel-logo.svg" alt="Vercel logo" width="120">
|
||||
</picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center"><a href="https://www.gmicloud.ai/"><strong>GMI Cloud</strong></a></td>
|
||||
<td align="center"><a href="https://vercel.com/"><strong>Vercel</strong></a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
@@ -158,7 +168,7 @@ irm https://raw.githubusercontent.com/Twigpine/zero/main/scripts/install.ps1 | i
|
||||
|
||||
### From source
|
||||
|
||||
Source builds require Go 1.26.9+.
|
||||
Source builds require Go 1.26.10+.
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Twigpine/zero.git
|
||||
@@ -173,17 +183,18 @@ If you are testing before the first public release, build from source:
|
||||
go build -o zero ./cmd/zero
|
||||
```
|
||||
|
||||
On Linux, build the sandbox helper too if you want native sandboxing:
|
||||
On Linux, native sandboxing needs Bubblewrap. The main `zero` binary re-executes
|
||||
itself as the helper when `zero-linux-sandbox` is absent. A separate helper next
|
||||
to `zero` or on `PATH` still takes precedence:
|
||||
|
||||
```bash
|
||||
go build -o zero-linux-sandbox ./cmd/zero-linux-sandbox
|
||||
go build -o zero-seccomp ./cmd/zero-seccomp # optional compatibility wrapper
|
||||
go build -o zero-linux-sandbox ./cmd/zero-linux-sandbox # optional
|
||||
go build -o zero-seccomp ./cmd/zero-seccomp # optional compatibility wrapper
|
||||
```
|
||||
|
||||
Put `zero` and `zero-linux-sandbox` in the same directory on `PATH`
|
||||
(`~/.local/bin` is a good default). macOS does not need an extra helper binary.
|
||||
Windows source builds can use the main `zero.exe` as their sandbox helper; release
|
||||
archives still ship standalone Windows helper executables.
|
||||
Put `zero` on `PATH` (`~/.local/bin` is a good default). macOS does not need an
|
||||
extra helper binary. Windows source builds can use the main `zero.exe` as their
|
||||
sandbox helper; release archives still ship standalone Windows helper executables.
|
||||
|
||||
More install details: [docs/INSTALL.md](docs/INSTALL.md).
|
||||
|
||||
@@ -386,6 +397,76 @@ and can rewrite every request through it, so this is the trust you accept by
|
||||
configuring one. Leave the variables unset for the checks to be enforced end to
|
||||
end.
|
||||
|
||||
The built-in `web_search` tool is registered only when a search backend is
|
||||
configured: `ZERO_WEBSEARCH_BASE_URL` (with an optional `ZERO_WEBSEARCH_API_KEY`)
|
||||
for a generic JSON endpoint, `ZERO_WEBSEARCH_PROVIDER=searxng` with a SearXNG
|
||||
base URL, or the AI/ML API Toolbox:
|
||||
|
||||
```bash
|
||||
export ZERO_WEBSEARCH_PROVIDER=aimlapi
|
||||
```
|
||||
|
||||
The AI/ML API backend runs the Toolbox tool `web/search`, and every search is
|
||||
billed to your AI/ML API balance: about $0.0013 per result, with a documented
|
||||
minimum of $0.0091 per search. Each search is sent with a $0.025 spending cap,
|
||||
which the Toolbox enforces. It is used only when
|
||||
`ZERO_WEBSEARCH_PROVIDER` is set to `aimlapi`; an AI/ML API key alone never
|
||||
enables it. The key is read from your user configuration only; provider
|
||||
profiles in a project `.zero` config are not used. It is the aimlapi profile's
|
||||
key (the active profile when it is aimlapi, otherwise the first aimlapi
|
||||
profile), including a key in the credential store, or `AIMLAPI_API_KEY` when no
|
||||
aimlapi profile exists. It is resolved again on every search;
|
||||
`ZERO_WEBSEARCH_API_KEY` is not used. When any aimlapi profile, or any profile
|
||||
that reads `AIMLAPI_API_KEY`, points at a base URL other than
|
||||
`https://api.aimlapi.com/v1`, or `AIMLAPI_INFERENCE_URL` is overridden, the key
|
||||
may belong to a proxy, so it is not sent and the search fails without a request.
|
||||
Requests go only to
|
||||
`https://tools.aimlapi.com` (`ZERO_WEBSEARCH_BASE_URL` is ignored and redirects
|
||||
are refused) and carry `X-AIMLAPI-Source: agent/zero` and an
|
||||
`X-AIMLAPI-Partner-ID` header. The `domains` argument is sent to the Toolbox as
|
||||
`include_domains`, and results are still filtered locally.
|
||||
|
||||
### AI/ML API Toolbox backend for `web_fetch`
|
||||
|
||||
`web_fetch` can fetch pages through the AI/ML API Toolbox tool
|
||||
`search1api/crawl` instead of a direct HTTP GET. It is off unless you choose it:
|
||||
|
||||
```bash
|
||||
export ZERO_WEBFETCH_PROVIDER=aimlapi
|
||||
```
|
||||
|
||||
Having an AI/ML API key never switches it on by itself, and any other value (or
|
||||
none) keeps the direct fetch. The variable is read on every fetch.
|
||||
|
||||
- **Billing:** Toolbox runs are billed to your AI/ML API balance, about
|
||||
$0.0013 per run. Each run is sent with `max_cost_usd` 0.005, so a run
|
||||
priced above that is refused before it starts and not charged. Rate-limited
|
||||
or gateway-failed requests are retried with the same idempotency key, so a
|
||||
retry is never billed twice.
|
||||
- **Key:** the key comes from your `aimlapi` provider profile (including a key
|
||||
in the credential store), or from `AIMLAPI_API_KEY` when there is no such
|
||||
profile. It is withheld, and `web_fetch` returns an error, when any `aimlapi`
|
||||
profile or any profile using `AIMLAPI_API_KEY` points at a base URL other than
|
||||
`https://api.aimlapi.com/v1`, or when `AIMLAPI_INFERENCE_URL` is overridden:
|
||||
the key may then belong to a proxy.
|
||||
- **Destination:** requests go only to the fixed `https://tools.aimlapi.com`
|
||||
endpoint, with `Authorization: Bearer <key>` and the attribution headers
|
||||
`X-AIMLAPI-Source: agent/zero` and `X-AIMLAPI-Partner-ID`. Redirects are
|
||||
refused. `HTTP_PROXY`/`HTTPS_PROXY` apply to these requests as to Zero's other
|
||||
aimlapi.com calls.
|
||||
- **No fallback:** without a usable key, or when the run fails, `web_fetch`
|
||||
returns an error and does not fetch the page directly. Unset the variable to
|
||||
go back to direct fetches.
|
||||
- **Remote fetch:** the page is fetched by a remote service, which may serve a
|
||||
cached copy, and the full URL (query string included) is sent to it. The URL
|
||||
checks above still run first, so loopback, private and metadata URLs are
|
||||
never handed to it. The result is Markdown with a
|
||||
`Backend: aimlapi (search1api/crawl)` line; `max_bytes` cuts the Markdown,
|
||||
and `format: "raw"` is refused because the original response is not
|
||||
available. The complete Toolbox run response is limited to 8 MiB before JSON
|
||||
decoding; larger responses return an error even if `max_bytes` is smaller.
|
||||
The remote run may already have been billed when this limit is detected.
|
||||
|
||||
For local dev servers, use shell commands such as `curl` through `exec_command`
|
||||
so the normal sandbox and permission policy applies. Long-running commands stay
|
||||
attached to a background terminal session and can be listed or stopped from the
|
||||
|
||||
+16
-6
@@ -12,9 +12,10 @@
|
||||
|
||||
<p align="center">
|
||||
<a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-MIT-blue"></a>
|
||||
<img alt="Go 1.26.9+" src="https://img.shields.io/badge/Go-1.26.9+-00ADD8?logo=go&logoColor=white">
|
||||
<img alt="Go 1.26.10+" src="https://img.shields.io/badge/Go-1.26.10+-00ADD8?logo=go&logoColor=white">
|
||||
<img alt="25+ providers" src="https://img.shields.io/badge/providers-25+-34E2EA">
|
||||
<a href="https://discord.gg/CaQDS6wdFn"><img alt="Discord" src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white"></a>
|
||||
<a href="https://vercel.com/open-source-program"><img alt="Vercel OSS Program" src="https://vercel.com/oss/program-badge-2026.svg"></a>
|
||||
<br>
|
||||
<a href="README.md">English</a> | <strong>中文</strong>
|
||||
</p>
|
||||
@@ -85,9 +86,18 @@ zero exec --output-format stream-json < turns.jsonl
|
||||
</picture>
|
||||
</a>
|
||||
</td>
|
||||
<td align="center" width="150" height="80">
|
||||
<a href="https://vercel.com/">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/vercel-logo-dark.svg">
|
||||
<img src="docs/assets/vercel-logo.svg" alt="Vercel logo" width="120">
|
||||
</picture>
|
||||
</a>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center"><a href="https://www.gmicloud.ai/"><strong>GMI Cloud</strong></a></td>
|
||||
<td align="center"><a href="https://vercel.com/"><strong>Vercel</strong></a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
@@ -136,7 +146,7 @@ irm https://raw.githubusercontent.com/Twigpine/zero/main/scripts/install.ps1 | i
|
||||
|
||||
### 从源码构建
|
||||
|
||||
源码构建需要 Go 1.26.9+。
|
||||
源码构建需要 Go 1.26.10+。
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Twigpine/zero.git
|
||||
@@ -150,14 +160,14 @@ go run ./cmd/zero
|
||||
go build -o zero ./cmd/zero
|
||||
```
|
||||
|
||||
在 Linux 上,如果你需要原生沙箱,还需要构建沙箱辅助程序:
|
||||
在 Linux 上,原生沙箱需要 Bubblewrap。主程序 `zero` 在找不到 `zero-linux-sandbox` 时会以自身作为辅助进程重新执行。同目录或 PATH 上的独立 helper 仍优先:
|
||||
|
||||
```bash
|
||||
go build -o zero-linux-sandbox ./cmd/zero-linux-sandbox
|
||||
go build -o zero-seccomp ./cmd/zero-seccomp # 可选的兼容性包装器
|
||||
go build -o zero-linux-sandbox ./cmd/zero-linux-sandbox # 可选
|
||||
go build -o zero-seccomp ./cmd/zero-seccomp # 可选的兼容性包装器
|
||||
```
|
||||
|
||||
将 `zero` 和 `zero-linux-sandbox` 放在 `PATH` 上的同一目录中(`~/.local/bin` 是一个好的默认选择)。macOS 不需要额外的辅助二进制文件。Windows 源码构建可以使用主 `zero.exe` 作为沙箱辅助程序;发布包仍然附带独立的 Windows 辅助可执行文件。
|
||||
将 `zero` 放在 `PATH` 上(`~/.local/bin` 是一个好的默认选择)。macOS 不需要额外的辅助二进制文件。Windows 源码构建可以使用主 `zero.exe` 作为沙箱辅助程序;发布包仍然附带独立的 Windows 辅助可执行文件。
|
||||
|
||||
更多安装细节:[docs/INSTALL.md](docs/INSTALL.md)。
|
||||
|
||||
|
||||
+13
-1
@@ -33,4 +33,16 @@ Please include, where possible:
|
||||
depend on a user explicitly disabling the sandbox (`--skip-permissions-unsafe`), are generally
|
||||
out of scope — but please report anything you are unsure about.
|
||||
- Zero sends no telemetry. Network calls go to the model/tool providers you configure (and, for
|
||||
keyless web search, to Exa's hosted MCP endpoint at `https://mcp.exa.ai/mcp`).
|
||||
keyless web search, to Exa's hosted MCP endpoint at `https://mcp.exa.ai/mcp`). The built-in
|
||||
AI/ML API MCP server (`https://mcp.aimlapi.com/mcp`) is off until you run
|
||||
`zero mcp enable aimlapi`. Once enabled, the Bearer key follows the aimlapi
|
||||
profile resolution order (inline key, then the profile's `apiKeyEnv` — default
|
||||
`AIMLAPI_API_KEY` — then the credential store), or `AIMLAPI_API_KEY` alone when
|
||||
no aimlapi profile exists; a non-empty `AIMLAPI_API_KEY` can override a stored
|
||||
profile key for MCP billing. The automatic key is withheld if an aimlapi profile
|
||||
(or a profile reading `AIMLAPI_API_KEY`) uses a base URL other than
|
||||
`https://api.aimlapi.com/v1`, `AIMLAPI_INFERENCE_URL` points elsewhere, a
|
||||
non-empty user Authorization header is configured, OAuth is selected, project
|
||||
config has touched the server, or the key is missing or not a single token
|
||||
(including Unicode whitespace). See
|
||||
[docs/EXTENDING.md](docs/EXTENDING.md#built-in-servers).
|
||||
+5
-5
@@ -2,8 +2,8 @@
|
||||
|
||||
## Questions, setup help, configuration
|
||||
|
||||
Ask in [GitHub Discussions](https://github.com/Gitlawb/zero/discussions). Use
|
||||
[Q&A](https://github.com/Gitlawb/zero/discussions/categories/q-a) so answers can
|
||||
Ask in [GitHub Discussions](https://github.com/Twigpine/zero/discussions). Use
|
||||
[Q&A](https://github.com/Twigpine/zero/discussions/categories/q-a) so answers can
|
||||
be marked and found by the next person.
|
||||
|
||||
The fastest way to a useful answer is to include:
|
||||
@@ -18,19 +18,19 @@ The fastest way to a useful answer is to include:
|
||||
## Bugs
|
||||
|
||||
If Zero does something wrong (crashes, wrong output, broken behavior), file a
|
||||
[bug report](https://github.com/Gitlawb/zero/issues/new/choose). If you are not
|
||||
[bug report](https://github.com/Twigpine/zero/issues/new/choose). If you are not
|
||||
sure whether it is a bug or a setup problem, start in Discussions and we will
|
||||
promote it to an issue if it is one.
|
||||
|
||||
## Feature ideas
|
||||
|
||||
Start a thread in
|
||||
[Ideas](https://github.com/Gitlawb/zero/discussions/categories/ideas). Zero is
|
||||
[Ideas](https://github.com/Twigpine/zero/discussions/categories/ideas). Zero is
|
||||
in controlled development (see CONTRIBUTING.md), so discussing an idea first is
|
||||
also the path to getting a PR accepted.
|
||||
|
||||
## Security issues
|
||||
|
||||
Never file a public issue or discussion. Report privately via
|
||||
[GitHub Security Advisories](https://github.com/Gitlawb/zero/security/advisories/new)
|
||||
[GitHub Security Advisories](https://github.com/Twigpine/zero/security/advisories/new)
|
||||
(see SECURITY.md).
|
||||
@@ -5,7 +5,7 @@ package main
|
||||
import (
|
||||
"os"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
@@ -11,7 +11,7 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/perfbench"
|
||||
"github.com/Twigpine/zero/internal/perfbench"
|
||||
)
|
||||
|
||||
const (
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/perfbench"
|
||||
"github.com/Twigpine/zero/internal/perfbench"
|
||||
)
|
||||
|
||||
func TestParseArgsUsesCliAndEnvOverrides(t *testing.T) {
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/perfbench"
|
||||
"github.com/Twigpine/zero/internal/perfbench"
|
||||
)
|
||||
|
||||
// taskOptions configures the `zero-perf-bench tasks` subcommand: the reproducible
|
||||
|
||||
@@ -9,8 +9,8 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/execprofile"
|
||||
"github.com/Gitlawb/zero/internal/perfbench"
|
||||
"github.com/Twigpine/zero/internal/execprofile"
|
||||
"github.com/Twigpine/zero/internal/perfbench"
|
||||
)
|
||||
|
||||
// turnOptions configures the `zero-perf-bench turn` subcommand: the per-turn
|
||||
|
||||
@@ -6,7 +6,7 @@ import (
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/review"
|
||||
"github.com/Twigpine/zero/internal/review"
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/release"
|
||||
"github.com/Twigpine/zero/internal/release"
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
@@ -12,7 +12,7 @@ import (
|
||||
"os/exec"
|
||||
"syscall"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
@@ -3,7 +3,7 @@ package main
|
||||
import (
|
||||
"os"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
@@ -3,7 +3,7 @@ package main
|
||||
import (
|
||||
"os"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
+5
-1
@@ -3,9 +3,13 @@ package main
|
||||
import (
|
||||
"os"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/cli"
|
||||
"github.com/Twigpine/zero/internal/cli"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 && os.Args[1] == sandbox.LinuxSandboxHelperSubcommand {
|
||||
os.Exit(sandbox.RunLinuxSandboxHelper(os.Args[1:], os.Stderr))
|
||||
}
|
||||
os.Exit(cli.Run(os.Args[1:], os.Stdout, os.Stderr))
|
||||
}
|
||||
+69
-2
@@ -110,6 +110,8 @@ Discovery roots (earlier wins on name collisions):
|
||||
|
||||
A missing directory is fine — Zero just omits it. Management commands (`zero skill add` / `zero skill remove`) always write to the primary Zero dir only; `list` / `info` search primary + `~/.agents/skills`. `skill info` reports lock source/hash when present and whether the on-disk `SKILL.md` has drifted from the recorded hash.
|
||||
|
||||
For a skill installed since Zero started installing bundled files, `skill info` also reports drift in any file installed with it.
|
||||
|
||||
```text
|
||||
~/.local/share/zero/skills/ # primary (install/remove target)
|
||||
run-benchmarks/
|
||||
@@ -140,6 +142,15 @@ Only `name` and `description` are recognized in the frontmatter today. The `name
|
||||
|
||||
The `skill` tool lets the agent load any discovered skill by name (primary, agents, or plugin).
|
||||
|
||||
A skill can ship files beside its `SKILL.md`, in the [agentskills.io](https://agentskills.io/specification) layout: `scripts/`, `references/`, `assets/` or anything else the instructions refer to. `zero skill add` installs the whole skill directory, leaving out hidden entries such as `.git` and not following or copying symlinks (it lists any it skipped). A skill larger than 1,000 files or 32 MiB is refused. When the `skill` tool loads a skill that has such files, its output ends with the skill's directory and a list of the files, so instructions like "run `scripts/fill.py`" can be followed. Running a bundled script goes through the normal tool permissions; installing or loading a skill never runs anything.
|
||||
|
||||
```text
|
||||
pdf/
|
||||
SKILL.md
|
||||
scripts/fill.py
|
||||
references/form-fields.md
|
||||
```
|
||||
|
||||
## 4. Hooks
|
||||
|
||||
Hooks fire shell commands on lifecycle events. Configure them in JSON:
|
||||
@@ -192,7 +203,11 @@ Events the agent emits (in dispatch order):
|
||||
| `specialistStart` | A sub-agent is spawned | yes (specialist name) |
|
||||
| `specialistStop` | A sub-agent ends | yes (specialist name) |
|
||||
|
||||
A hook's exit code decides what happens next: `0` continues, non-zero blocks the tool call (`beforeTool`) or surfaces an error (`afterTool`). Hook execution is recorded in the audit log; the audit is reachable from the agent's view of past actions, not from a dedicated `zero doctor` check.
|
||||
For swarm members the matcher subject is the **agent type** (`subagent`, `teammate`, …), not a registered specialist profile name — swarm members run from an inline manifest keyed by agent type.
|
||||
|
||||
`specialistStart` fires once the child process has actually started; a launch that fails before exec (missing binary, spawn error) records no public lifecycle hooks. `specialistStop` fires once per terminal stop: completion, error, explicit stop of a recovered task, and a background task reaped by a later `TaskOutput` all share one durable claim, so a hook that notifies or cleans up does not run twice for the same sub-agent run.
|
||||
|
||||
A hook's exit code decides what happens next: `0` continues, non-zero blocks the tool call (`beforeTool`) or surfaces an error (`afterTool`). `sessionStart` / `sessionEnd` / `specialistStart` / `specialistStop` hooks are advisory (failures are audited but do not interrupt the run). Hook execution is recorded in the audit log; the audit is reachable from the agent's view of past actions, not from a dedicated `zero doctor` check.
|
||||
|
||||
> **Roadmap.** An in-UI hooks manager is on the backlog. Today you edit the JSON directly.
|
||||
|
||||
@@ -246,6 +261,56 @@ or use a new server name. OAuth tokens are bound to the resolved server identity
|
||||
so changing a server target can require `zero mcp oauth login <server>` again.
|
||||
OAuth server names ending in `.<32 hex chars>` are reserved for token storage.
|
||||
|
||||
#### Built-in servers
|
||||
|
||||
Zero seeds two MCP servers before your config is merged. Both show up in
|
||||
`zero mcp list`, and any field can be overridden in your user config.
|
||||
|
||||
- `exa` (`https://mcp.exa.ai/mcp`) is on by default for keyless web search.
|
||||
`zero mcp disable exa` turns it off.
|
||||
- `aimlapi` (`https://mcp.aimlapi.com/mcp`, the AI/ML API Toolbox, model
|
||||
discovery, image/video/audio generation, chat, embeddings and balance tools)
|
||||
is **off by default**, because its tool runs and generations are billed to
|
||||
your AI/ML API balance. Turn it on with `zero mcp enable aimlapi`, which
|
||||
writes only `"disabled": false` to your user config, and off again with
|
||||
`zero mcp disable aimlapi`. A project config cannot turn it on.
|
||||
|
||||
When `aimlapi` connects, Zero sends your AI/ML API key as
|
||||
`Authorization: Bearer <key>`. The key is resolved again on every request and
|
||||
is never written to `config.json`. With an aimlapi.com provider profile
|
||||
(`zero providers setup aimlapi`), the key is chosen in this order:
|
||||
|
||||
1. An inline key on the profile (if set in config).
|
||||
2. Otherwise the environment variable named by the profile's `apiKeyEnv`
|
||||
(the catalog default is `AIMLAPI_API_KEY`).
|
||||
3. Otherwise the key held in the credential store when the profile uses
|
||||
`apiKeyStored`.
|
||||
|
||||
When there is no aimlapi profile, Zero uses `AIMLAPI_API_KEY` alone. If both
|
||||
`AIMLAPI_API_KEY` and a stored profile key exist, step 2 wins over the store
|
||||
whenever the inline key is empty — MCP requests may bill the environment key's
|
||||
account, not the stored key.
|
||||
|
||||
No automatic key is sent when an aimlapi profile, or any profile reading
|
||||
`AIMLAPI_API_KEY`, uses a base URL other than `https://api.aimlapi.com/v1`, or
|
||||
when `AIMLAPI_INFERENCE_URL` is overridden, because that credential may belong
|
||||
to a proxy. No key is sent when the resolved value is not a single token (for
|
||||
example a comma- or newline-separated pool), when you set your own
|
||||
non-empty `Authorization` header, when the server uses `"auth": "oauth"`, or
|
||||
when project config touched the server. Without a key the server answers 401.
|
||||
To sign in with your aimlapi.com account instead, set `"auth": "oauth"` on the
|
||||
server and run `zero mcp oauth login aimlapi`.
|
||||
|
||||
Every request to exactly `https://mcp.aimlapi.com` (https, default port,
|
||||
whatever the server is named, including one you add by hand) also carries the
|
||||
attribution headers `X-AIMLAPI-Source: agent/zero` and `X-AIMLAPI-Partner-ID`,
|
||||
and gets the key under the rules above. Other MCP servers never receive these
|
||||
headers or the key.
|
||||
|
||||
If the server echoes back the key Zero sent it, Zero replaces the key with
|
||||
`[REDACTED]` in that server's error messages and tool results before anything
|
||||
reads them, and reports an HTTP error status without the response body.
|
||||
|
||||
### As a server — expose Zero's tools to another agent
|
||||
|
||||
```bash
|
||||
@@ -290,10 +355,12 @@ Install and manage:
|
||||
zero plugins add ./github-pr-review # copy into ~/.config/zero/plugins/ or ./.zero/plugins/
|
||||
zero plugins list
|
||||
zero plugins info github-pr-review # show manifest path, enabled state, lock metadata
|
||||
zero plugins disable github-pr-review # sets "enabled": false in plugin.json
|
||||
zero plugins enable github-pr-review # sets "enabled": true in plugin.json
|
||||
zero plugins remove github-pr-review # alias: rm
|
||||
```
|
||||
|
||||
A plugin is enabled by being present in the plugins directory and disabled by removing it (or by the user setting `"enabled": false` in its `plugin.json`). Plugins are not enabled or disabled by a CLI subcommand today.
|
||||
A plugin is active when it is present under a plugins directory and its `plugin.json` has `"enabled": true` (the default when the field is omitted). Use `zero plugins disable <id>` / `zero plugins enable <id>` to toggle that field without removing the plugin. Pass `--user` to target only user plugins and ignore `./.zero/plugins`; without it, a project plugin with the same id takes precedence. The toggle edits the `enabled` value in place — the rest of the manifest keeps its formatting, field order, and numeric literals — and serializes with `plugins add` / `plugins remove` on the same plugins root. Zero reads plugins when it starts, so the change applies to the next `zero exec` run or interactive session; an interactive session that is already running keeps the plugin tools, hooks, and skills it started with until you restart it.
|
||||
|
||||
Plugin commands run with the plugin directory as their working directory. Use relative paths; the loader resolves them at activation time.
|
||||
|
||||
|
||||
@@ -634,6 +634,10 @@ Core tool groups include:
|
||||
`HTTP_PROXY`/`HTTPS_PROXY` set, the proxy is dialed instead and resolves the
|
||||
target itself, so destination enforcement ends at the proxy; see the note in
|
||||
the README.
|
||||
With `ZERO_WEBFETCH_PROVIDER=aimlapi` (opt-in; a key alone does not select
|
||||
it), the same URL checks run and the page is then fetched as Markdown by the
|
||||
billed AI/ML API Toolbox tool `search1api/crawl`, with no fallback to a
|
||||
direct fetch; see the README.
|
||||
- **Local-control tools**: config-gated wrappers for browser, terminal, desktop,
|
||||
and artifact helpers backed by packages such as `internal/localcontrol` and
|
||||
`internal/browser`.
|
||||
|
||||
+27
-14
@@ -185,13 +185,19 @@ describe a version that is no longer on disk. Prefer `mise upgrade`.
|
||||
go install github.com/Gitlawb/zero/cmd/zero@latest
|
||||
```
|
||||
|
||||
This builds from source, so it needs Go 1.26.9+ and it does not go through the
|
||||
Every release so far declares the module path `github.com/Gitlawb/zero`, so
|
||||
`go install` has to use that path; `github.com/Twigpine/zero/cmd/zero@latest`
|
||||
fails with "module declares its path as" until a release is tagged with the new
|
||||
module path.
|
||||
|
||||
This builds from source, so it needs Go 1.26.10+ and it does not go through the
|
||||
release archives. Two consequences worth knowing before you pick it:
|
||||
|
||||
- On Linux you also need the sandbox helper, which is a separate binary and is
|
||||
not installed by this command. See
|
||||
- On Linux, Bubblewrap is still required for native sandboxing. Zero resolves
|
||||
the helper in this order: an executable `zero-linux-sandbox` next to `zero`,
|
||||
then `PATH`, then self-exec of the main binary. Standalone helpers remain
|
||||
optional and preferred. See
|
||||
[Sandbox Helpers For Source Builds](#sandbox-helpers-for-source-builds).
|
||||
Without it, native sandboxing is unavailable.
|
||||
- `zero upgrade` treats the result as a standalone install and will replace the
|
||||
binary with a release build rather than rebuilding from source. If you chose
|
||||
`go install` deliberately, rerun it instead.
|
||||
@@ -212,25 +218,32 @@ Build a local binary:
|
||||
go build -o zero ./cmd/zero
|
||||
```
|
||||
|
||||
Source builds require Go 1.26.9+.
|
||||
Source builds require Go 1.26.10+.
|
||||
|
||||
### Sandbox Helpers For Source Builds
|
||||
|
||||
Release archives include the platform sandbox helpers. If you build directly
|
||||
from source, build the helpers you need:
|
||||
Release archives include the platform sandbox helpers. A source build of the
|
||||
main `zero` binary is enough for Linux native sandboxing. Zero resolves the
|
||||
helper in this order:
|
||||
|
||||
Linux:
|
||||
1. an executable `zero-linux-sandbox` next to `zero`
|
||||
2. then `PATH`
|
||||
3. then self-exec of the main binary
|
||||
|
||||
Bubblewrap must still be installed. Standalone helpers remain optional and
|
||||
preferred.
|
||||
|
||||
A separate helper remains supported if it sits next to `zero` or on `PATH`:
|
||||
|
||||
```bash
|
||||
go build -o zero ./cmd/zero
|
||||
go build -o zero-linux-sandbox ./cmd/zero-linux-sandbox
|
||||
go build -o zero-seccomp ./cmd/zero-seccomp
|
||||
go build -o zero-linux-sandbox ./cmd/zero-linux-sandbox # optional
|
||||
go build -o zero-seccomp ./cmd/zero-seccomp # optional compatibility wrapper
|
||||
```
|
||||
|
||||
Put `zero` and `zero-linux-sandbox` in the same directory on `PATH`, for example
|
||||
`~/.local/bin`. `zero-seccomp` is kept as a compatibility wrapper; the sandbox
|
||||
helper applies the Unix-socket filter itself when that sandbox option is enabled.
|
||||
Linux native sandboxing also requires Bubblewrap to be installed.
|
||||
Put `zero` on `PATH`, for example `~/.local/bin`. `zero-seccomp` is kept as a
|
||||
compatibility wrapper; the sandbox helper applies the Unix-socket filter itself
|
||||
when that sandbox option is enabled.
|
||||
|
||||
macOS uses the system sandbox and does not need an extra helper binary.
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="261" height="52" fill="none" viewBox="0 0 261 52"><path fill="white" d="M59.8 52H0L29.9 0zm67.82-38.45q4.9 0 8.81 2.13a15.5 15.5 0 0 1 6.22 6.32q2.3 4.2 2.38 10.26v2.06h-26.35q.27 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.76-4.93l9.16.67q-1.68 4.98-6.29 7.99t-10.63 3q-5.53 0-9.64-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m62.55 0q4.47 0 8.18 1.66a15.3 15.3 0 0 1 6.15 4.6q2.38 3 2.87 7.05l-9.23.47a8 8 0 0 0-2.8-5 7.7 7.7 0 0 0-5.17-1.86q-4.33 0-6.7 3-2.4 3-2.39 8.52t2.38 8.52q2.37 3 6.71 3 3.15 0 5.39-1.87 2.24-1.92 2.72-5.46l9.3.4a14.7 14.7 0 0 1-2.87 7.33 16 16 0 0 1-6.15 4.86 21 21 0 0 1-8.39 1.66q-5.53 0-9.64-2.26a16 16 0 0 1-6.44-6.46 20 20 0 0 1-2.3-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m38.66 0q4.9 0 8.8 2.13a15.5 15.5 0 0 1 6.22 6.32q2.31 4.2 2.38 10.26v2.06h-26.35q.28 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.77-4.93l9.15.67q-1.68 5-6.29 7.99t-10.62 3q-5.53 0-9.65-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26M86.9 36.69l17.24-34.33h10.8L89.96 49.63h-6.12L58.85 2.36h10.81zm71.62-15.55a11 11 0 0 1 2.47-4.48q2.28-2.31 6.38-2.31h3.4v7.26h-3.47q-2.91 0-4.79.8a5.8 5.8 0 0 0-2.77 2.5q-.9 1.72-.9 4.37v20.35h-8.89V14.35h8.33zm101.73 28.5h-8.95V2.35h8.95zM127.62 20.26q-3.7 0-6 2.2-2.32 2.2-2.87 6.2h17.05q-.48-4.34-2.72-6.33a7.8 7.8 0 0 0-5.46-2.07m101.2 0q-3.7 0-6 2.2-2.31 2.2-2.87 6.2H237q-.5-4.34-2.73-6.33a7.8 7.8 0 0 0-5.46-2.07" style="fill:white;fill-opacity:1;"/></svg>
|
||||
|
After Width: | Height: | Size: 1.6 KiB |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="261" height="52" fill="none" viewBox="0 0 261 52"><path fill="black" d="M59.8 52H0L29.9 0zm67.82-38.45q4.9 0 8.81 2.13a15.5 15.5 0 0 1 6.22 6.32q2.3 4.2 2.38 10.26v2.06h-26.35q.27 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.76-4.93l9.16.67q-1.68 4.98-6.29 7.99t-10.63 3q-5.53 0-9.64-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m62.55 0q4.47 0 8.18 1.66a15.3 15.3 0 0 1 6.15 4.6q2.38 3 2.87 7.05l-9.23.47a8 8 0 0 0-2.8-5 7.7 7.7 0 0 0-5.17-1.86q-4.33 0-6.7 3-2.4 3-2.39 8.52t2.38 8.52q2.37 3 6.71 3 3.15 0 5.39-1.87 2.24-1.92 2.72-5.46l9.3.4a14.7 14.7 0 0 1-2.87 7.33 16 16 0 0 1-6.15 4.86 21 21 0 0 1-8.39 1.66q-5.53 0-9.64-2.26a16 16 0 0 1-6.44-6.46 20 20 0 0 1-2.3-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26m38.66 0q4.9 0 8.8 2.13a15.5 15.5 0 0 1 6.22 6.32q2.31 4.2 2.38 10.26v2.06h-26.35q.28 4.4 2.58 6.92 2.38 2.47 6.36 2.47a8.4 8.4 0 0 0 7.77-4.93l9.15.67q-1.68 5-6.29 7.99t-10.62 3q-5.53 0-9.65-2.27a16 16 0 0 1-6.43-6.46 20 20 0 0 1-2.31-9.72q0-5.52 2.3-9.72a16 16 0 0 1 6.44-6.46 20 20 0 0 1 9.64-2.26M86.9 36.69l17.24-34.33h10.8L89.96 49.63h-6.12L58.85 2.36h10.81zm71.62-15.55a11 11 0 0 1 2.47-4.48q2.28-2.31 6.38-2.31h3.4v7.26h-3.47q-2.91 0-4.79.8a5.8 5.8 0 0 0-2.77 2.5q-.9 1.72-.9 4.37v20.35h-8.89V14.35h8.33zm101.73 28.5h-8.95V2.35h8.95zM127.62 20.26q-3.7 0-6 2.2-2.32 2.2-2.87 6.2h17.05q-.48-4.34-2.72-6.33a7.8 7.8 0 0 0-5.46-2.07m101.2 0q-3.7 0-6 2.2-2.31 2.2-2.87 6.2H237q-.5-4.34-2.73-6.33a7.8 7.8 0 0 0-5.46-2.07" style="fill:black;fill-opacity:1;"/></svg>
|
||||
|
After Width: | Height: | Size: 1.6 KiB |
@@ -1,6 +1,6 @@
|
||||
module github.com/Gitlawb/zero
|
||||
module github.com/Twigpine/zero
|
||||
|
||||
go 1.26.9
|
||||
go 1.26.10
|
||||
|
||||
require (
|
||||
charm.land/bubbles/v2 v2.1.1
|
||||
|
||||
+27
-20
@@ -13,18 +13,18 @@ import (
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Gitlawb/zero/internal/config"
|
||||
"github.com/Gitlawb/zero/internal/modelregistry"
|
||||
"github.com/Gitlawb/zero/internal/providercatalog"
|
||||
"github.com/Gitlawb/zero/internal/providermodelcatalog"
|
||||
"github.com/Gitlawb/zero/internal/providermodeldiscovery"
|
||||
"github.com/Gitlawb/zero/internal/redaction"
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/sessions"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/usage"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/modelregistry"
|
||||
"github.com/Twigpine/zero/internal/providercatalog"
|
||||
"github.com/Twigpine/zero/internal/providermodelcatalog"
|
||||
"github.com/Twigpine/zero/internal/providermodeldiscovery"
|
||||
"github.com/Twigpine/zero/internal/redaction"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/usage"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// Deps are the ZERO capabilities the ACP Agent drives. The CLI fills these with
|
||||
@@ -528,8 +528,6 @@ func (a *Agent) runTurn(ctx context.Context, sess *acpSession, userText string,
|
||||
queue := func(input sessions.AppendEventInput) {
|
||||
pendingEvents = append(pendingEvents, input)
|
||||
}
|
||||
queue(messageEvent("user", userText))
|
||||
|
||||
visionCache := make(map[string]bool)
|
||||
var visionCacheMu sync.Mutex
|
||||
supportsVision := func(modelID string) bool {
|
||||
@@ -553,6 +551,9 @@ func (a *Agent) runTurn(ctx context.Context, sess *acpSession, userText string,
|
||||
effectivePrompt = fmt.Sprintf("[Note: %s]\n\n%s", msg, userText)
|
||||
images = nil
|
||||
}
|
||||
// Match the TUI: only images actually sent to the model enter history.
|
||||
attachments := sessions.MessageAttachments{Images: len(images)}
|
||||
queue(messageEvent("user", userText, attachments))
|
||||
|
||||
opts := agent.Options{
|
||||
Cwd: sess.cwd,
|
||||
@@ -593,7 +594,7 @@ func (a *Agent) runTurn(ctx context.Context, sess *acpSession, userText string,
|
||||
agentPrompt := buildPrompt(sess.snapshotHistory(), effectivePrompt)
|
||||
result, runErr := a.deps.RunAgent(ctx, agentPrompt, provider, opts)
|
||||
if result.FinalAnswer != "" {
|
||||
queue(messageEvent("assistant", result.FinalAnswer))
|
||||
queue(messageEvent("assistant", result.FinalAnswer, sessions.MessageAttachments{}))
|
||||
}
|
||||
reason, stopErr := stopReasonFor(result, runErr)
|
||||
if stopErr != nil {
|
||||
@@ -614,7 +615,7 @@ func (a *Agent) runTurn(ctx context.Context, sess *acpSession, userText string,
|
||||
// one file write, which is the contiguity restored here — not a disk
|
||||
// transaction, and not a promise about crashes mid-fsync. Reported by @jatmn.
|
||||
persistenceErr := a.persistTurn(sess.id, pendingEvents)
|
||||
sess.appendHistory(turnRecord{user: userText, assistant: result.FinalAnswer})
|
||||
sess.appendHistory(turnRecord{user: attachments.WithContent(userText), assistant: result.FinalAnswer})
|
||||
a.warnPersistence(
|
||||
note,
|
||||
"save session history",
|
||||
@@ -930,10 +931,14 @@ func (a *Agent) persistTurn(sessionID string, inputs []sessions.AppendEventInput
|
||||
return err
|
||||
}
|
||||
|
||||
func messageEvent(role, content string) sessions.AppendEventInput {
|
||||
func messageEvent(role, content string, attachments sessions.MessageAttachments) sessions.AppendEventInput {
|
||||
payload := map[string]any{"role": role, "content": content}
|
||||
if role == "user" && (attachments.Images > 0 || attachments.Documents > 0) {
|
||||
payload["attachments"] = attachments
|
||||
}
|
||||
return sessions.AppendEventInput{
|
||||
Type: sessions.EventMessage,
|
||||
Payload: map[string]any{"role": role, "content": content},
|
||||
Payload: payload,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1094,14 +1099,16 @@ func (a *Agent) loadHistory(sessionID string, requireHistoryLog bool) ([]turnRec
|
||||
continue
|
||||
}
|
||||
var msg struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
Attachments sessions.MessageAttachments `json:"attachments"`
|
||||
}
|
||||
if json.Unmarshal(raw, &msg) != nil {
|
||||
continue
|
||||
}
|
||||
switch msg.Role {
|
||||
case "user":
|
||||
msg.Content = msg.Attachments.WithContent(msg.Content)
|
||||
messages = append(messages, persistedMessage{eventID: persistedMessageIdentity(sessionID, e), role: msg.Role, content: msg.Content})
|
||||
if havePending {
|
||||
records = append(records, turnRecord{user: pendingUser})
|
||||
|
||||
@@ -19,13 +19,13 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Gitlawb/zero/internal/config"
|
||||
"github.com/Gitlawb/zero/internal/providermodeldiscovery"
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/sessions"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/providermodeldiscovery"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// fakeProvider streams a canned assistant message and ends the turn — enough to
|
||||
@@ -960,7 +960,11 @@ func TestACPHardTurnFailureDoesNotCommitHistory(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := h.client.Call(ctx, MethodSessionPrompt, PromptParams{
|
||||
SessionID: created.SessionID, Prompt: []ContentBlock{TextBlock("failed question")},
|
||||
SessionID: created.SessionID,
|
||||
Prompt: []ContentBlock{
|
||||
TextBlock("failed question"),
|
||||
ImageBlock(base64.StdEncoding.EncodeToString([]byte("failed image")), "image/png"),
|
||||
},
|
||||
}, &PromptResult{}); err == nil {
|
||||
t.Fatal("hard RunAgent failure was reported as a successful turn")
|
||||
}
|
||||
|
||||
@@ -15,7 +15,7 @@ import (
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
)
|
||||
|
||||
// ---- an offered option is an acceptable answer ----
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
package acp
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/config"
|
||||
"github.com/Twigpine/zero/internal/providermodeldiscovery"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestACPImageAttachmentHistory(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
text string
|
||||
images int
|
||||
withoutVision bool
|
||||
want string
|
||||
}{
|
||||
{name: "image only", images: 1, want: "[1 image]"},
|
||||
{name: "text and images", text: "compare these", images: 2, want: "compare these [2 images]"},
|
||||
{name: "ignored by model", images: 1, withoutVision: true, want: ""},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
deps := testDeps(t)
|
||||
deps.DiscoverModels = func(context.Context, config.ProviderProfile) ([]providermodeldiscovery.Model, error) {
|
||||
modalities := []string{"text", "image"}
|
||||
if tc.withoutVision {
|
||||
modalities = []string{"text"}
|
||||
}
|
||||
return []providermodeldiscovery.Model{{ID: "fake-model", InputModalities: modalities}}, nil
|
||||
}
|
||||
type observedTurn struct {
|
||||
prompt string
|
||||
images int
|
||||
}
|
||||
turns := make(chan observedTurn, 8)
|
||||
deps.RunAgent = func(_ context.Context, prompt string, _ zeroruntime.Provider, opts agent.Options) (agent.Result, error) {
|
||||
turns <- observedTurn{prompt: prompt, images: len(opts.Images)}
|
||||
return agent.Result{FinalAnswer: "received"}, nil
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
workspace := t.TempDir()
|
||||
h := newHarness(t, deps)
|
||||
defer h.stop()
|
||||
var created NewSessionResult
|
||||
if err := h.client.Call(ctx, MethodSessionNew, NewSessionParams{Cwd: workspace}, &created); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const imageData = "image-history-bytes-not-for-storage"
|
||||
encoded := base64.StdEncoding.EncodeToString([]byte(imageData))
|
||||
var blocks []ContentBlock
|
||||
if tc.text != "" {
|
||||
blocks = append(blocks, TextBlock(tc.text))
|
||||
}
|
||||
for range tc.images {
|
||||
blocks = append(blocks, ImageBlock(encoded, "image/png"))
|
||||
}
|
||||
if err := h.client.Call(ctx, MethodSessionPrompt, PromptParams{SessionID: created.SessionID, Prompt: blocks}, &PromptResult{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
live := <-turns
|
||||
wantLiveImages := tc.images
|
||||
if tc.withoutVision {
|
||||
wantLiveImages = 0
|
||||
}
|
||||
if live.images != wantLiveImages {
|
||||
t.Fatalf("live images = %d, want %d", live.images, wantLiveImages)
|
||||
}
|
||||
events, err := deps.Store.ReadEvents(created.SessionID)
|
||||
if err != nil || len(events) < 2 {
|
||||
t.Fatalf("read completed turn: events=%+v err=%v", events, err)
|
||||
}
|
||||
var user struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
Attachments struct {
|
||||
Images int `json:"images"`
|
||||
} `json:"attachments"`
|
||||
}
|
||||
if err := json.Unmarshal(events[0].Payload, &user); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if user.Role != "user" || user.Content != tc.text || user.Attachments.Images != wantLiveImages {
|
||||
t.Errorf("stored user event = %s, want original text and attachments.images=%d", events[0].Payload, wantLiveImages)
|
||||
}
|
||||
for _, event := range events {
|
||||
if bytes.Contains(event.Payload, []byte(encoded)) || bytes.Contains(event.Payload, []byte(imageData)) {
|
||||
t.Errorf("image data leaked into session event: %s", event.Payload)
|
||||
}
|
||||
}
|
||||
followUp := func(t *testing.T, client *clientHarness) {
|
||||
t.Helper()
|
||||
if err := client.client.Call(ctx, MethodSessionPrompt, PromptParams{
|
||||
SessionID: created.SessionID, Prompt: []ContentBlock{TextBlock("what did I send?")},
|
||||
}, &PromptResult{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := <-turns
|
||||
if want := "User: " + tc.want + "\n"; !strings.Contains(got.prompt, want) {
|
||||
t.Errorf("seeded prompt missing user history %q: %q", want, got.prompt)
|
||||
}
|
||||
if got.images != 0 {
|
||||
t.Errorf("historical images reattached: %d", got.images)
|
||||
}
|
||||
}
|
||||
followUp(t, h)
|
||||
h.stop()
|
||||
// Fresh agents and stores reproduce a process restart, with no shared
|
||||
// in-memory history to hide a missing attachment in the event log.
|
||||
for _, method := range []string{MethodSessionLoad, MethodSessionResume} {
|
||||
t.Run(method, func(t *testing.T) {
|
||||
restoredDeps := deps
|
||||
restoredDeps.Store = sessions.NewStore(sessions.StoreOptions{RootDir: deps.Store.RootDir})
|
||||
loader := newHarness(t, restoredDeps)
|
||||
defer loader.stop()
|
||||
if err := loader.client.Call(ctx, method, LoadSessionParams{SessionID: created.SessionID, Cwd: workspace}, &LoadSessionResult{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if method == MethodSessionLoad {
|
||||
select {
|
||||
case update := <-loader.notifications:
|
||||
if update.SessionUpdate != UpdateUserMessageChunk || update.Content.Text != tc.want {
|
||||
t.Errorf("replayed user = %+v, want %q", update, tc.want)
|
||||
}
|
||||
case <-ctx.Done():
|
||||
t.Fatal("user turn was not replayed")
|
||||
}
|
||||
} else {
|
||||
select {
|
||||
case update := <-loader.notifications:
|
||||
t.Errorf("resume replayed transcript: %+v", update)
|
||||
default:
|
||||
}
|
||||
}
|
||||
followUp(t, loader)
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestACPAttachmentHistoryReadsExistingSessionMessages(t *testing.T) {
|
||||
deps := testDeps(t)
|
||||
meta, err := deps.Store.Create(sessions.CreateInput{SessionID: "attachment-compatibility", Cwd: t.TempDir()})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for i, payload := range []string{
|
||||
`{"role":"user","content":"old text"}`,
|
||||
`{"role":"user","content":"","attachments":{"images":1,"documents":2}}`,
|
||||
`{"role":"user","content":"still text","attachments":{"images":-1,"documents":1.5}}`,
|
||||
`{"role":"assistant","content":"answer","attachments":{"images":1}}`,
|
||||
} {
|
||||
if _, err := deps.Store.AppendEvent(meta.SessionID, sessions.AppendEventInput{Type: sessions.EventMessage, Payload: json.RawMessage(payload)}); err != nil {
|
||||
t.Fatalf("append message %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
a := &Agent{deps: deps}
|
||||
_, messages, _, err := a.loadHistory(meta.SessionID, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := []string{"old text", "[1 image] [2 documents]", "still text", "answer"}
|
||||
if len(messages) != len(want) {
|
||||
t.Fatalf("messages = %+v", messages)
|
||||
}
|
||||
for i, text := range want {
|
||||
if messages[i].content != text {
|
||||
t.Errorf("message %d = %q, want %q", i, messages[i].content, text)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -8,9 +8,9 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Gitlawb/zero/internal/sessions"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// captureAgentOptions makes every turn record the agent.Options it was built
|
||||
|
||||
@@ -3,7 +3,7 @@ package acp
|
||||
import (
|
||||
"encoding/json"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
)
|
||||
|
||||
// permission.go maps ZERO's permission prompt model onto ACP's
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
)
|
||||
|
||||
func escalationRequest(escalates bool) agent.PermissionRequest {
|
||||
|
||||
@@ -3,7 +3,7 @@ package acp
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
)
|
||||
|
||||
func TestBuildPermissionOptions(t *testing.T) {
|
||||
|
||||
@@ -8,8 +8,8 @@ import (
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// translate.go maps ZERO's agent events onto ACP session/update payloads. The
|
||||
@@ -83,7 +83,7 @@ func browserToolDetails(name string) (*BrowserToolDetails, bool) {
|
||||
}
|
||||
}
|
||||
|
||||
const zeroBrowserMetaKey = "github.com/Gitlawb/zero/browser"
|
||||
const zeroBrowserMetaKey = "github.com/Twigpine/zero/browser"
|
||||
|
||||
// attachBrowserToolDetails stores ZERO's browser descriptor in ACP's reserved
|
||||
// extension channel. Keeping this in one helper prevents start, result, and
|
||||
|
||||
@@ -8,8 +8,8 @@ import (
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
func browserDescriptor(t *testing.T, update ToolCallUpdate) BrowserToolDetails {
|
||||
|
||||
@@ -8,9 +8,9 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/agent"
|
||||
"github.com/Gitlawb/zero/internal/sessions"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/agent"
|
||||
"github.com/Twigpine/zero/internal/sessions"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// `zero exec` persists every provider usage report as a session usage event, so
|
||||
|
||||
@@ -5,8 +5,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func enabledSelfCorrector(t *testing.T) *SelfCorrector {
|
||||
|
||||
@@ -5,8 +5,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
type normalizationRecordingShellTool struct {
|
||||
|
||||
@@ -6,8 +6,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// providerCallingAskUserThenAnswer returns a mock provider whose first turn calls
|
||||
|
||||
@@ -8,9 +8,9 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/testutil"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/testutil"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestAsyncDiagnosticsNilCollectorNoOps(t *testing.T) {
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// A CONFIGURATION REFUSAL IS NOT A RETRIABLE FAILURE, EVEN WHEN IT ARRIVES
|
||||
// EARLY.
|
||||
//
|
||||
// capture_artifact rejects in RejectBeforePermission, which the registry returns
|
||||
// straight back before any of the gates that attach provenance. Its
|
||||
// valid-but-unavailable calls therefore reached the classifier with no denial
|
||||
// category, no permission metadata and no refusal marker, and were read as
|
||||
// ordinary retriable failures: the model got the schema hint and the call could
|
||||
// consume the profile failure-streak escalation, for a tool that never executed
|
||||
// and that no argument change can enable.
|
||||
func TestDisabledCaptureArtifactIsAPolicyRefusalNotARetriableFailure(t *testing.T) {
|
||||
// No artifacts directory and no enabled driver: valid arguments, unavailable
|
||||
// tool. This is the shape an operator produces by configuration alone.
|
||||
registry := tools.NewRegistry()
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{}) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "browser_screenshot",
|
||||
"name": "shot",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: expected the disabled tool to refuse, got %s: %s", result.Status, result.Output)
|
||||
}
|
||||
if !tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("the refusal carries no provenance, so nothing downstream can tell it from a failed command: %#v", result.Meta)
|
||||
}
|
||||
|
||||
// Through the conversion the loop performs, which is where the marker has to
|
||||
// survive to be worth anything.
|
||||
converted := ToolResult{
|
||||
Status: result.Status,
|
||||
Output: result.Output,
|
||||
Meta: result.Meta,
|
||||
}
|
||||
if !isPolicyRefusal(converted) {
|
||||
t.Error("the marker did not survive into the agent-facing result")
|
||||
}
|
||||
if isRetriableToolError(converted) {
|
||||
t.Error("a tool that never executed, and that no argument change can enable, was marked retriable: the model gets a schema hint telling it to fix arguments that were already valid")
|
||||
}
|
||||
}
|
||||
|
||||
// The malformed-argument branch must stay retriable. That one IS fixable by
|
||||
// trying again differently, which is what the hint exists for, so marking every
|
||||
// early rejection would trade one wrong answer for another.
|
||||
func TestMalformedCaptureArtifactArgumentsStayRetriable(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{}) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "not_a_real_action",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: expected invalid arguments to fail, got %s", result.Status)
|
||||
}
|
||||
if tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("a malformed-argument error was marked a policy refusal, so the model is denied the hint that would let it fix the call: %q", result.Output)
|
||||
}
|
||||
converted := ToolResult{Status: result.Status, Output: result.Output, Meta: result.Meta}
|
||||
if !isRetriableToolError(converted) {
|
||||
t.Error("invalid arguments should stay retriable")
|
||||
}
|
||||
}
|
||||
|
||||
// alwaysRefusingCaptureTool stands in for the disabled tool at Run level, so the
|
||||
// loop consequence can be observed rather than inferred.
|
||||
type alwaysRefusingCaptureTool struct{ ran int }
|
||||
|
||||
func (tool *alwaysRefusingCaptureTool) Name() string { return "capture_artifact" }
|
||||
func (tool *alwaysRefusingCaptureTool) Description() string { return "test capture tool" }
|
||||
func (tool *alwaysRefusingCaptureTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"action": {Type: "string"}},
|
||||
Required: []string{"action"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *alwaysRefusingCaptureTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "captures artifacts"}
|
||||
}
|
||||
func (tool *alwaysRefusingCaptureTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{}
|
||||
}
|
||||
func (tool *alwaysRefusingCaptureTool) RejectBeforePermission(map[string]any) (tools.Result, bool) {
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
Output: "Error: capture_artifact is disabled because no artifact directory is configured.",
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalToolNotEnabled},
|
||||
}, true
|
||||
}
|
||||
|
||||
// THE LOOP CONSEQUENCE. A refusal must not draw the retry hint, because the hint
|
||||
// tells the model to fix arguments that were already valid and the tool will
|
||||
// refuse identically next time.
|
||||
func TestRunDoesNotHintARefusedCaptureArtifact(t *testing.T) {
|
||||
tool := &alwaysRefusingCaptureTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
calls := toolFailureHintAt + 1
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "capture_artifact", `{"action":"browser_screenshot"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up on the screenshot"))
|
||||
|
||||
result, err := Run(context.Background(), "take a screenshot", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Fatalf("SETUP INVALID: the tool executed %d times; it must be refused before Run", tool.ran)
|
||||
}
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, toolFailureHintMarker) {
|
||||
t.Fatal("a refused, never-executed tool drew the retry hint, which tells the model to fix arguments that were already valid")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// alternatingRefusalCaptureTool refuses every call with the same CATEGORY and a
|
||||
// different MESSAGE, which is what a real disabled driver does: the refusal
|
||||
// names the action, and the model is free to alternate valid actions.
|
||||
type alternatingRefusalCaptureTool struct{ ran int }
|
||||
|
||||
func (tool *alternatingRefusalCaptureTool) Name() string { return "capture_artifact" }
|
||||
func (tool *alternatingRefusalCaptureTool) Description() string { return "test capture tool" }
|
||||
func (tool *alternatingRefusalCaptureTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"action": {Type: "string"}},
|
||||
Required: []string{"action"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *alternatingRefusalCaptureTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "captures artifacts"}
|
||||
}
|
||||
func (tool *alternatingRefusalCaptureTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{}
|
||||
}
|
||||
func (tool *alternatingRefusalCaptureTool) RejectBeforePermission(args map[string]any) (tools.Result, bool) {
|
||||
action, _ := args["action"].(string)
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
// Action-specific wording, same category. This is the whole point: the
|
||||
// prose differs on every call while the refusal never changes.
|
||||
Output: "Error: Local control driver for " + action + " is disabled.",
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalToolNotEnabled},
|
||||
}, true
|
||||
}
|
||||
|
||||
// A REFUSAL KEYS ON ITS CATEGORY, INCLUDING WHEN THE CATEGORY IS ONLY A MARKER.
|
||||
//
|
||||
// The registry marks pre-execution refusals in metadata, and isPolicyRefusal
|
||||
// read that marker while observeToolResult keyed on DenialReason, which those
|
||||
// paths leave empty. The guard fell back to errorSignature(output), so two
|
||||
// refusals of the same category with different wording looked like two
|
||||
// different failures and the streak restarted at 1 every call.
|
||||
//
|
||||
// A model alternating capture_artifact's browser_screenshot and browser_pdf
|
||||
// against a disabled driver is refused identically each time and never tripped
|
||||
// the six-call halt. Only the generic twelve-error fallback stopped the run,
|
||||
// reporting varied errors rather than a repeated refusal.
|
||||
func TestAlternatingRefusedActionsStillTripTheRefusalHalt(t *testing.T) {
|
||||
tool := &alternatingRefusalCaptureTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// More calls than the refusal halt but FEWER than the generic any-error
|
||||
// fallback, so only the category-keyed streak can stop this.
|
||||
calls := toolFailureAnyErrorStopAt - 1
|
||||
if calls <= toolFailureStopAt {
|
||||
t.Fatalf("SETUP INVALID: %d calls cannot distinguish the refusal halt from the generic fallback", calls)
|
||||
}
|
||||
actions := []string{"browser_screenshot", "browser_pdf"}
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
action := actions[i%len(actions)]
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "capture_artifact", `{"action":"`+action+`"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up"))
|
||||
|
||||
result, err := Run(context.Background(), "capture something", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Fatalf("SETUP INVALID: the tool executed %d times; it must be refused before Run", tool.ran)
|
||||
}
|
||||
|
||||
refusals := 0
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, "is disabled") {
|
||||
refusals++
|
||||
}
|
||||
}
|
||||
if refusals > toolFailureStopAt {
|
||||
t.Errorf("the run made %d refused calls; the six-call refusal halt never tripped because the streak re-keyed on each action's wording", refusals)
|
||||
}
|
||||
|
||||
// And the halt has to read as a repeated refusal, not as varied errors.
|
||||
stop := strings.ToLower(strings.Join(messageContents(result.Messages), "\n"))
|
||||
if strings.Contains(stop, toolFailureHintMarker) {
|
||||
t.Error("a refused, never-executed tool drew the retry hint")
|
||||
}
|
||||
}
|
||||
|
||||
func messageContents(messages []zeroruntime.Message) []string {
|
||||
out := make([]string, 0, len(messages))
|
||||
for _, message := range messages {
|
||||
out = append(out, message.Content)
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/localcontrol"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// THE DISABLED-DRIVER BRANCH, REACHED FOR REAL.
|
||||
//
|
||||
// RejectBeforePermission refuses on two separate conditions, and its neighbour
|
||||
// test constructs empty options, so it returns at the FIRST one (no artifact
|
||||
// directory configured) and the second is never evaluated. Reverting the
|
||||
// disabled-driver branch in local_capture.go to a plain errorResult therefore
|
||||
// left the whole agent and tools suites green.
|
||||
//
|
||||
// This is the shape an operator actually produces: an artifact root IS
|
||||
// configured and one driver IS enabled, and the model asks for an action owned
|
||||
// by a different, disabled driver. Without provenance that call is read as a
|
||||
// retriable failure, so it collects a schema hint and can spend the profile's
|
||||
// failure-streak escalation, for a tool no argument change can enable.
|
||||
func TestCaptureArtifactDisabledDriverIsAPolicyRefusal(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
options := tools.LocalControlArtifactOptions{
|
||||
ArtifactsDir: t.TempDir(),
|
||||
// Configured and enabled, so the tool as a whole is available and the
|
||||
// missing-directory branch cannot fire.
|
||||
Browser: localcontrol.BrowserOptions{Enabled: true, Driver: "test-browser", HelperPath: "browser-helper"},
|
||||
// Terminal deliberately left disabled.
|
||||
}
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(options) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
// An enabled driver's action must still be accepted, or the test below would
|
||||
// pass for a tool that refuses everything.
|
||||
if result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "browser_screenshot", "name": "shot",
|
||||
}, tools.RunOptions{PermissionGranted: true}); tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("SETUP INVALID: the enabled browser driver was refused as policy: %s", result.Output)
|
||||
}
|
||||
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "terminal_snapshot", "name": "snap", "session": "test-session",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: the disabled terminal driver did not refuse: %s / %s", result.Status, result.Output)
|
||||
}
|
||||
if !tools.IsPolicyRefusalResult(result) {
|
||||
t.Fatalf("a driver disabled by configuration refuses without provenance, so it reads as a retriable failure: %#v", result.Meta)
|
||||
}
|
||||
if got := result.Meta["policy_refusal"]; got != tools.PolicyRefusalToolNotEnabled {
|
||||
t.Errorf("policy_refusal = %q, want %q", got, tools.PolicyRefusalToolNotEnabled)
|
||||
}
|
||||
if isRetriableToolError(ToolResult{Status: result.Status, Output: result.ModelOutput(), Meta: result.Meta}) {
|
||||
t.Error("the refusal is still classified as retriable, so the model gets a schema hint for a configuration decision")
|
||||
}
|
||||
}
|
||||
|
||||
// AND THE EARLY REJECTIONS MUST NOT COLLAPSE INTO ONE ANSWER.
|
||||
//
|
||||
// RejectBeforePermission refuses on configuration; argument validation refuses
|
||||
// on a fixable mistake, and it runs FIRST, which is how the disabled-driver
|
||||
// branch stayed uncovered. A malformed call must stay an ordinary retriable
|
||||
// error, because trying again differently is exactly the right response to it.
|
||||
func TestCaptureArtifactMalformedArgumentsStayRetriable(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
for _, tool := range tools.NewLocalControlArtifactTools(tools.LocalControlArtifactOptions{
|
||||
ArtifactsDir: t.TempDir(),
|
||||
Terminal: localcontrol.TerminalOptions{Enabled: true, Driver: "test-terminal", HelperPath: "terminal-helper"},
|
||||
}) {
|
||||
registry.Register(tool)
|
||||
}
|
||||
|
||||
// The driver IS enabled, so nothing here is a configuration decision. The
|
||||
// call is simply missing the session the action requires.
|
||||
result := registry.RunWithOptions(context.Background(), "capture_artifact", map[string]any{
|
||||
"action": "terminal_snapshot", "name": "snap",
|
||||
}, tools.RunOptions{PermissionGranted: true})
|
||||
|
||||
if result.Status != tools.StatusError {
|
||||
t.Fatalf("SETUP INVALID: the malformed call did not fail: %s", result.Output)
|
||||
}
|
||||
if tools.IsPolicyRefusalResult(result) {
|
||||
t.Errorf("a fixable argument mistake was marked a policy refusal, so the model loses the schema hint that would fix it: %s", result.Output)
|
||||
}
|
||||
if !isRetriableToolError(ToolResult{Status: result.Status, Output: result.ModelOutput(), Meta: result.Meta}) {
|
||||
t.Errorf("a fixable argument mistake is not retriable: %s", result.Output)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package agent
|
||||
|
||||
import "github.com/Twigpine/zero/internal/tools"
|
||||
|
||||
// CheckpointTargets returns the workspace-relative paths a tool call will mutate,
|
||||
// resolving its arguments exactly as executeToolCall does. Freeform apply_patch
|
||||
// input is normalized through the registered built-in tool, so absolute headers
|
||||
// are rewritten relative to the selected write root and cwd is set: checkpoint
|
||||
// capture and execution therefore agree on the files at stake.
|
||||
//
|
||||
// Best-effort: nil when the call has no capture targets or its arguments cannot
|
||||
// be resolved. Checkpoint capture never fails a run.
|
||||
func CheckpointTargets(workspaceRoot string, registry *tools.Registry, call ToolCall) []string {
|
||||
if registry == nil {
|
||||
return nil
|
||||
}
|
||||
args := map[string]any{}
|
||||
if call.Freeform {
|
||||
tool, ok := registry.Get(call.Name)
|
||||
if !ok || !tools.IsBuiltInApplyPatch(tool) {
|
||||
return nil
|
||||
}
|
||||
prepared, err := tools.PrepareFreeformApplyPatchArguments(tool, call.Arguments)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
args = prepared
|
||||
} else if call.Arguments != "" {
|
||||
if err := decodeToolArguments(call.Arguments, &args); err != nil {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return tools.MutationTargets(workspaceRoot, call.Name, args)
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
func TestCheckpointTargets(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
// Absolute headers must be compared against the physical root: t.TempDir
|
||||
// differs from it on macOS (/var -> /private/var) and via Windows short names.
|
||||
if resolved, err := filepath.EvalSymlinks(root); err == nil {
|
||||
root = resolved
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "main.txt"), []byte("before\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
relativePatch := "*** Begin Patch\n*** Update File: main.txt\n@@\n-before\n+after\n*** End Patch\n"
|
||||
absolutePatch := "*** Begin Patch\n*** Update File: " + filepath.Join(root, "main.txt") + "\n@@\n-before\n+after\n*** End Patch\n"
|
||||
multiFilePatch := "*** Begin Patch\n*** Update File: main.txt\n@@\n-before\n+after\n*** Add File: notes.txt\n+new\n*** End Patch\n"
|
||||
jsonArgs, err := json.Marshal(map[string]any{"patch": relativePatch})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tools.NewScopedApplyPatchTool(root, nil))
|
||||
unbuiltinRegistry := tools.NewRegistry()
|
||||
unbuiltinRegistry.Register(&customApplyPatchTool{})
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
registry *tools.Registry
|
||||
call ToolCall
|
||||
want []string
|
||||
}{
|
||||
{
|
||||
name: "freeform relative header",
|
||||
call: ToolCall{Name: "apply_patch", Arguments: relativePatch, Freeform: true},
|
||||
want: []string{"main.txt"},
|
||||
},
|
||||
{
|
||||
name: "freeform absolute header stays workspace-relative",
|
||||
call: ToolCall{Name: "apply_patch", Arguments: absolutePatch, Freeform: true},
|
||||
want: []string{"main.txt"},
|
||||
},
|
||||
{
|
||||
name: "freeform multi-file patch",
|
||||
call: ToolCall{Name: "apply_patch", Arguments: multiFilePatch, Freeform: true},
|
||||
want: []string{"main.txt", "notes.txt"},
|
||||
},
|
||||
{
|
||||
name: "json arguments",
|
||||
call: ToolCall{Name: "apply_patch", Arguments: string(jsonArgs)},
|
||||
want: []string{"main.txt"},
|
||||
},
|
||||
{
|
||||
name: "freeform traversal is rejected",
|
||||
call: ToolCall{Name: "apply_patch", Arguments: "*** Begin Patch\n*** Update File: ../escape.txt\n@@\n-x\n+y\n*** End Patch\n", Freeform: true},
|
||||
want: nil,
|
||||
},
|
||||
{
|
||||
name: "freeform custom apply_patch is not prepared",
|
||||
registry: unbuiltinRegistry,
|
||||
call: ToolCall{Name: "apply_patch", Arguments: relativePatch, Freeform: true},
|
||||
want: nil,
|
||||
},
|
||||
{
|
||||
name: "unknown freeform tool",
|
||||
call: ToolCall{Name: "read_file", Arguments: "raw", Freeform: true},
|
||||
want: nil,
|
||||
},
|
||||
{
|
||||
name: "malformed json arguments",
|
||||
call: ToolCall{Name: "apply_patch", Arguments: "{"},
|
||||
want: nil,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
reg := registry
|
||||
if tc.registry != nil {
|
||||
reg = tc.registry
|
||||
}
|
||||
got := CheckpointTargets(root, reg, tc.call)
|
||||
if !slices.Equal(got, tc.want) {
|
||||
t.Fatalf("CheckpointTargets() = %#v, want %#v", got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
if got := CheckpointTargets(root, nil, ToolCall{Name: "apply_patch", Arguments: relativePatch}); got != nil {
|
||||
t.Fatalf("CheckpointTargets(nil registry) = %#v, want nil", got)
|
||||
}
|
||||
}
|
||||
@@ -5,8 +5,8 @@ import (
|
||||
"runtime"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"mvdan.cc/sh/v3/syntax"
|
||||
)
|
||||
|
||||
|
||||
@@ -4,8 +4,8 @@ import (
|
||||
"runtime"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
func TestProposedCommandPrefixUsesSafeSimpleCommands(t *testing.T) {
|
||||
|
||||
@@ -8,8 +8,8 @@ import (
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// Session compaction.
|
||||
|
||||
@@ -6,8 +6,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// compactableHistory returns a history whose middle is large enough to trip
|
||||
|
||||
@@ -6,7 +6,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestCompactMessagesReturnsMetadataForManualCompaction(t *testing.T) {
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// Compaction preservation.
|
||||
|
||||
@@ -6,8 +6,8 @@ import (
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// stateConversation is a long enough conversation that Compact elides a middle
|
||||
|
||||
@@ -6,7 +6,7 @@ import (
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
const (
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestClipBytesHandlesTinyLimitsOnRuneBoundaries(t *testing.T) {
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// recentEdits extracts each mutated file's path and a one-line note from the
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
var markerPattern = regexp.MustCompile(`msg-\d+`)
|
||||
|
||||
@@ -6,9 +6,9 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// --- Pure Compact() tests -------------------------------------------------
|
||||
|
||||
@@ -5,8 +5,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// someRequestContains reports whether any message sent to the provider across all
|
||||
|
||||
@@ -3,7 +3,7 @@ package agent
|
||||
import (
|
||||
"encoding/json"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// Context budget.
|
||||
|
||||
@@ -3,7 +3,7 @@ package agent
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestMeasureContextSplitsByCategory(t *testing.T) {
|
||||
|
||||
@@ -3,7 +3,7 @@ package agent
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
type contextPlannerConfig struct {
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestContextPlannerPreservesProviderRequest(t *testing.T) {
|
||||
|
||||
@@ -6,7 +6,7 @@ import (
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// richMCPTool mimics a real MCP tool's schema (Exa/Firecrawl-style search): an
|
||||
|
||||
@@ -8,8 +8,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestOptionsDeferThresholdFieldExists(t *testing.T) {
|
||||
|
||||
@@ -4,8 +4,8 @@ import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
func TestExecuteToolCallCategorizesFilteredDenial(t *testing.T) {
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// argsPermTool implements tools.ArgsPermissioner; plainTool does not.
|
||||
|
||||
@@ -6,8 +6,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// TestOptionsModelSwitcherFieldExists asserts the escalation hook is an
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func equalStringSlices(left []string, right []string) bool {
|
||||
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/lsp"
|
||||
"github.com/Twigpine/zero/internal/lsp"
|
||||
)
|
||||
|
||||
// fileDiagnosticsTimeout bounds one inline post-edit diagnostics check so a
|
||||
|
||||
@@ -8,9 +8,9 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
type customApplyPatchTool struct {
|
||||
|
||||
+193
-14
@@ -4,7 +4,7 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// Guardrail thresholds for the agent loop. These keep a runaway model from
|
||||
@@ -55,6 +55,22 @@ const (
|
||||
// error, so this only affects true same-error loops.
|
||||
toolFailureStopAt = 6
|
||||
|
||||
// toolFailureAnyErrorStopAt halts a tool that keeps failing with DIFFERENT
|
||||
// errors. The streak above cannot see that case by construction: a changed
|
||||
// signature rebuilds the record at 1, so a tool whose error text varies every
|
||||
// call never reaches toolFailureStopAt however long it loops.
|
||||
//
|
||||
// That is not hypothetical. A headless run made 384 denied calls over 26
|
||||
// minutes without tripping a halt set to 6, because each denial named a
|
||||
// different path. Keying denials on their category (see observeToolResult)
|
||||
// fixes that case; this bound covers every error with no category to key on.
|
||||
//
|
||||
// Deliberately well above toolFailureStopAt. A model iterating on a genuinely
|
||||
// tricky edit legitimately fails several times with different errors while it
|
||||
// converges, and this must not cut those runs short — the same reasoning that
|
||||
// moved toolFailureStopAt from 4 to 6. Both counters reset on success.
|
||||
toolFailureAnyErrorStopAt = 12
|
||||
|
||||
// maxContinueNudges bounds how many times the headless completion gate
|
||||
// (Options.RequireCompletionSignal) re-prompts a model that stopped without a
|
||||
// tool call while work clearly remained. Once spent, the run finalizes as
|
||||
@@ -324,15 +340,91 @@ func acceptanceVerificationNudge(objective string) string {
|
||||
const toolFailureHintMarker = "kept failing with the same error"
|
||||
|
||||
type toolFailureRecord struct {
|
||||
count int
|
||||
errSig string
|
||||
hintShown bool
|
||||
count int
|
||||
identity failureIdentity
|
||||
// sawExecuted and sawRefused are the AGGREGATE over the whole streak, not
|
||||
// the current identity. The content-blind bound needs them because it fires
|
||||
// exactly when no identity repeated, so the last one is unrepresentative.
|
||||
sawExecuted bool
|
||||
sawRefused bool
|
||||
hintShown bool
|
||||
// anyErrorCount counts consecutive failures of this tool REGARDLESS of the
|
||||
// error, and is cleared only by a success. count above restarts whenever the
|
||||
// signature changes, which is exactly what a varying error message defeats;
|
||||
// this one cannot be reset by changing the text.
|
||||
anyErrorCount int
|
||||
}
|
||||
|
||||
// failureKind separates a failure the TOOL produced from one POLICY produced.
|
||||
//
|
||||
// The two used to share one string namespace, with provenance recovered
|
||||
// afterwards by testing that string for a "denial:" prefix. Trusted provenance
|
||||
// and untrusted content in one namespace is a namespace the untrusted side can
|
||||
// write into: a command that printed exactly "denial:permission_denied" and
|
||||
// exited non-zero acquired the identity of a real permission refusal, merged
|
||||
// its streak with one, and had the run report it as refused although it ran
|
||||
// every time. The kind is carried as its own field so no output can spell it.
|
||||
type failureKind uint8
|
||||
|
||||
const (
|
||||
// The zero value: no failure recorded.
|
||||
_ failureKind = iota
|
||||
// failureKindExecuted: the tool ran and returned an error. The key is the
|
||||
// normalized error signature, which is tool-controlled text.
|
||||
failureKindExecuted
|
||||
// failureKindRefused: policy refused the call before the tool ran. The key is
|
||||
// the denial category, a small closed enum the loop sets.
|
||||
failureKindRefused
|
||||
)
|
||||
|
||||
// failureIdentity is what the same-identity streak counts. Comparing the pair
|
||||
// means an executed error can never equal a refusal however it is spelled.
|
||||
type failureIdentity struct {
|
||||
kind failureKind
|
||||
key string
|
||||
}
|
||||
|
||||
// toolFailureCause is why the guard stopped, decided where the facts are rather
|
||||
// than reconstructed by the caller from overlapping booleans.
|
||||
type toolFailureCause uint8
|
||||
|
||||
const (
|
||||
// The zero value: the guard did not stop.
|
||||
_ toolFailureCause = iota
|
||||
// toolFailureCauseSameError: the same executed error signature, six times.
|
||||
toolFailureCauseSameError
|
||||
// toolFailureCauseSameRefusal: the same denial category, six times. The prose
|
||||
// differs on every call because it names what was refused, so this is NOT the
|
||||
// same-error case.
|
||||
toolFailureCauseSameRefusal
|
||||
// toolFailureCauseVariedExecuted: the content-blind bound, and every failure
|
||||
// was the tool executing and failing.
|
||||
toolFailureCauseVariedExecuted
|
||||
// toolFailureCauseVariedRefused: the content-blind bound, and every failure
|
||||
// was a policy refusal. Alternating two categories reaches twelve without
|
||||
// either reaching six, and the tool never ran once.
|
||||
toolFailureCauseVariedRefused
|
||||
// toolFailureCauseVariedMixed: the content-blind bound over both kinds.
|
||||
// Named deliberately rather than left to whichever field a switch tested
|
||||
// first.
|
||||
toolFailureCauseVariedMixed
|
||||
)
|
||||
|
||||
type toolFailureOutcome struct {
|
||||
InjectHint bool
|
||||
Stop bool
|
||||
Count int
|
||||
// Cause is set only when Stop is. It carries the aggregate fact about the
|
||||
// sequence that tripped a bound, which the last record alone cannot supply:
|
||||
// the content-blind bound is reached precisely when no single identity
|
||||
// repeated enough, so the identity present at the end says nothing about the
|
||||
// eleven before it.
|
||||
Cause toolFailureCause
|
||||
}
|
||||
|
||||
// Refused reports whether policy, rather than the tool, produced the failures.
|
||||
func (cause toolFailureCause) Refused() bool {
|
||||
return cause == toolFailureCauseSameRefusal || cause == toolFailureCauseVariedRefused
|
||||
}
|
||||
|
||||
// errorSignature normalizes a tool error to a short, comparable signature so
|
||||
@@ -358,12 +450,47 @@ func toolFailureHint(toolName, schemaJSON, errOutput string) string {
|
||||
|
||||
// toolFailureStopAnswer is the final answer when the repeated-failure guard halts
|
||||
// a run.
|
||||
func toolFailureStopAnswer(toolName string, count int) string {
|
||||
return "Agent stopped: the `" + toolName + "` tool failed " + strconv.Itoa(count) +
|
||||
" times in a row with the same error, so I halted instead of looping further. " +
|
||||
func toolFailureStopAnswer(toolName string, count int, cause toolFailureCause) string {
|
||||
// Each branch claims only what its counter established, and the mixed case is
|
||||
// spelled out rather than decided by whichever field a switch happened to
|
||||
// test first. The wording used to overclaim in both directions: the
|
||||
// content-blind bound said every failure differed, which it does not track,
|
||||
// and it described a run of policy refusals as the tool failing, when the
|
||||
// tool had not run at all.
|
||||
verb, tail := " tool failed ", " times in a row with the same error, "
|
||||
switch cause {
|
||||
case toolFailureCauseSameRefusal:
|
||||
verb, tail = " tool was refused ", " times in a row, "
|
||||
case toolFailureCauseVariedExecuted:
|
||||
tail = " times in a row with varying errors, "
|
||||
case toolFailureCauseVariedRefused:
|
||||
verb, tail = " tool was refused ", " times in a row for different reasons, "
|
||||
case toolFailureCauseVariedMixed:
|
||||
verb, tail = " tool failed or was refused ", " times in a row, "
|
||||
}
|
||||
return "Agent stopped: the `" + toolName + "`" + verb + strconv.Itoa(count) +
|
||||
tail + "so I halted instead of looping further. " +
|
||||
"Please check the request or adjust the tool arguments."
|
||||
}
|
||||
|
||||
// toolFailureIncompleteReason describes a guard halt for the headless
|
||||
// completion gate. A guard halt is BY DEFINITION unfinished work: the run
|
||||
// stopped because a tool would not stop failing, not because the task was done.
|
||||
//
|
||||
// It makes the same claim as toolFailureStopAnswer for the same halt, including
|
||||
// the mixed case: a headless caller reads this reason and an interactive one
|
||||
// reads the stop answer, and the two must not describe one halt differently.
|
||||
// Reported by @jatmn.
|
||||
func toolFailureIncompleteReason(toolName string, cause toolFailureCause) string {
|
||||
switch {
|
||||
case cause.Refused():
|
||||
return "halted after `" + toolName + "` was refused repeatedly, without completing the task"
|
||||
case cause == toolFailureCauseVariedMixed:
|
||||
return "halted after `" + toolName + "` failed or was refused repeatedly, without completing the task"
|
||||
}
|
||||
return "halted after `" + toolName + "` failed repeatedly, without completing the task"
|
||||
}
|
||||
|
||||
// The no-output stop answer is assembled from these fixed parts (only the turn
|
||||
// count varies). IsNoProgressStop matches all three so a legitimate message that
|
||||
// merely quotes the marker substring is not misclassified as a failed empty run.
|
||||
@@ -472,28 +599,68 @@ func newGuardState() *guardState {
|
||||
// observeToolResult tracks repeated identical failures of a tool. A successful
|
||||
// result clears that tool's failure streak. Returns whether to inject a one-shot
|
||||
// corrective hint and/or stop the run.
|
||||
func (state *guardState) observeToolResult(name string, failed bool, output string) toolFailureOutcome {
|
||||
// hintable is separate from failed on purpose. A categorized denial MUST count
|
||||
// toward the streaks — that is the whole point of keying on the category — but a
|
||||
// schema hint is the wrong response to a policy refusal: the call shape is fine,
|
||||
// the answer was no. Collapsing the two is what made the earlier version of this
|
||||
// fix a no-op, since the caller could only pass a flag that excluded denials.
|
||||
func (state *guardState) observeToolResult(name string, failed bool, hintable bool, output string, denial DenialCategory) toolFailureOutcome {
|
||||
if state.toolFailures == nil {
|
||||
state.toolFailures = map[string]*toolFailureRecord{}
|
||||
}
|
||||
if !failed {
|
||||
delete(state.toolFailures, name) // success resets the streak
|
||||
delete(state.toolFailures, name) // success resets both counters
|
||||
return toolFailureOutcome{}
|
||||
}
|
||||
sig := errorSignature(output)
|
||||
// A denial keys on its CATEGORY, not its prose. The message embeds the path
|
||||
// or command that was refused, so it differs on every call while describing
|
||||
// the same unchanging refusal — which rebuilt the record at 1 each time and
|
||||
// let a denied tool loop indefinitely under a halt set to 6. The category is
|
||||
// a small closed enum the loop already sets on the result.
|
||||
identity := failureIdentity{kind: failureKindExecuted, key: errorSignature(output)}
|
||||
if denial != DenialNone {
|
||||
identity = failureIdentity{kind: failureKindRefused, key: string(denial)}
|
||||
}
|
||||
record := state.toolFailures[name]
|
||||
if record == nil || record.errSig != sig {
|
||||
record = &toolFailureRecord{count: 1, errSig: sig}
|
||||
if record == nil {
|
||||
record = &toolFailureRecord{identity: identity}
|
||||
state.toolFailures[name] = record
|
||||
}
|
||||
if record.identity != identity {
|
||||
// A different failure restarts the same-identity streak but NOT the
|
||||
// content-blind one: changing how a tool fails is not progress.
|
||||
record.count = 1
|
||||
record.identity = identity
|
||||
record.hintShown = false
|
||||
} else {
|
||||
record.count++
|
||||
}
|
||||
record.anyErrorCount++
|
||||
if identity.kind == failureKindRefused {
|
||||
record.sawRefused = true
|
||||
} else {
|
||||
record.sawExecuted = true
|
||||
}
|
||||
|
||||
outcome := toolFailureOutcome{Count: record.count}
|
||||
if record.count >= toolFailureStopAt {
|
||||
switch {
|
||||
case record.count >= toolFailureStopAt:
|
||||
outcome.Stop = true
|
||||
outcome.Cause = toolFailureCauseSameError
|
||||
if identity.kind == failureKindRefused {
|
||||
outcome.Cause = toolFailureCauseSameRefusal
|
||||
}
|
||||
return outcome
|
||||
case record.anyErrorCount >= toolFailureAnyErrorStopAt:
|
||||
// Report the counter that actually tripped. record.count is the
|
||||
// same-identity streak and is often 1 here, which would describe a tool
|
||||
// that failed a dozen different ways as having failed once.
|
||||
outcome.Stop = true
|
||||
outcome.Count = record.anyErrorCount
|
||||
outcome.Cause = record.variedCause()
|
||||
return outcome
|
||||
}
|
||||
if record.count >= toolFailureHintAt && !record.hintShown {
|
||||
if hintable && record.count >= toolFailureHintAt && !record.hintShown {
|
||||
record.hintShown = true
|
||||
outcome.InjectHint = true
|
||||
}
|
||||
@@ -603,3 +770,15 @@ func (state *guardState) planReminder(turn int) string {
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
// variedCause classifies a content-blind stop by what the whole streak held.
|
||||
func (record *toolFailureRecord) variedCause() toolFailureCause {
|
||||
switch {
|
||||
case record.sawRefused && record.sawExecuted:
|
||||
return toolFailureCauseVariedMixed
|
||||
case record.sawRefused:
|
||||
return toolFailureCauseVariedRefused
|
||||
default:
|
||||
return toolFailureCauseVariedExecuted
|
||||
}
|
||||
}
|
||||
@@ -2,11 +2,13 @@ package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// emptyTurn is a stream that produces no visible text and no tool calls.
|
||||
@@ -80,6 +82,9 @@ func TestRunStopsAfterConsecutiveEmptyTurns(t *testing.T) {
|
||||
if result.FinalAnswer == maxTurnsAnswer {
|
||||
t.Fatalf("no-output guard must stop before reaching maxTurns, got max-turns answer")
|
||||
}
|
||||
if result.Incomplete {
|
||||
t.Fatalf("interactive run must not be marked Incomplete; reason=%q", result.IncompleteReason)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunResetsEmptyTurnCounterOnVisibleOutput(t *testing.T) {
|
||||
@@ -189,7 +194,7 @@ func TestUnknownExecSessionProbingTripsFailureHalt(t *testing.T) {
|
||||
var state guardState
|
||||
var stoppedAt int
|
||||
for i := 1; i <= toolFailureStopAt; i++ {
|
||||
out := state.observeToolResult(tools.WriteStdinToolName, true, tools.UnknownExecSessionError(i))
|
||||
out := state.observeToolResult(tools.WriteStdinToolName, true, true, tools.UnknownExecSessionError(i), "")
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
@@ -200,6 +205,223 @@ func TestUnknownExecSessionProbingTripsFailureHalt(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A permission denial repeats forever when the streak is keyed on the error
|
||||
// TEXT, because the denial message embeds the path or command that varies per
|
||||
// call.
|
||||
//
|
||||
// Observed, not theorised: a headless run made 384 denied calls over 26 minutes
|
||||
// without tripping a halt that stops at 6. Every denial carried the same typed
|
||||
// category and a different reason string, so errorSignature differed each time
|
||||
// and the record was rebuilt at count 1 on every call.
|
||||
//
|
||||
// TestUnknownExecSessionErrorSignatureIsIDInvariant fixed one message this way.
|
||||
// Keying on the category fixes the class, without needing every future denial
|
||||
// message to remember to be invariant.
|
||||
func TestPermissionDenialStreakSurvivesVaryingReasonText(t *testing.T) {
|
||||
var state guardState
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureStopAt; i++ {
|
||||
// The shape the loop actually produces: same tool, same category, a
|
||||
// different path every time.
|
||||
output := "Error: Permission denied for write_file: cannot write " +
|
||||
filepath.Join("C:", "ws", "pkg", "file"+strconv.Itoa(i)+".go")
|
||||
out := state.observeToolResult("write_file", true, true, output, DenialPermissionDenied)
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureStopAt {
|
||||
t.Fatalf("denials with varying reason text stopped at %d, want %d", stoppedAt, toolFailureStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// The content-blind bound. A tool failing over and over with genuinely
|
||||
// DIFFERENT errors and no denial category is still a tool that is not working,
|
||||
// and the same-signature streak can never see it.
|
||||
//
|
||||
// Bounded well above toolFailureStopAt on purpose: a model iterating on a
|
||||
// tricky edit legitimately fails a few times with different errors while it
|
||||
// converges, and cutting that short is the regression
|
||||
// TestSuccessResetsBothFailureCounters guards.
|
||||
func TestToolFailingWithDifferentErrorsEveryTimeStillStops(t *testing.T) {
|
||||
var state guardState
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
|
||||
out := state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), "")
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureAnyErrorStopAt {
|
||||
t.Fatalf("a tool failing with a new error each call stopped at %d, want %d", stoppedAt, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// Neither counter may outlive a success, or a long run that fails occasionally
|
||||
// and recovers would eventually halt for no reason. This is the property that
|
||||
// keeps the content-blind bound safe to add.
|
||||
func TestSuccessResetsBothFailureCounters(t *testing.T) {
|
||||
var state guardState
|
||||
for i := 1; i < toolFailureAnyErrorStopAt; i++ {
|
||||
if out := state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), ""); out.Stop {
|
||||
t.Fatalf("stopped at %d before the success that should reset it", i)
|
||||
}
|
||||
}
|
||||
state.observeToolResult("bash", false, false, "ok", "")
|
||||
|
||||
// Same again from zero. Reaching the bound a second time proves the counter
|
||||
// restarted rather than merely paused.
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
|
||||
if out := state.observeToolResult("bash", true, true, "later failure "+strconv.Itoa(i), ""); out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureAnyErrorStopAt {
|
||||
t.Fatalf("after a success the tool stopped at %d, want a full fresh %d", stoppedAt, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// Records are keyed per tool, so ANOTHER tool succeeding in between must not
|
||||
// clear the failing tool's streak.
|
||||
//
|
||||
// This is the realistic shape of the run that motivated the fix: the model kept
|
||||
// making progress elsewhere — reading files, updating its plan — while one tool
|
||||
// was refused over and over. A reset keyed on "something succeeded" rather than
|
||||
// "this tool succeeded" would make the halt unreachable in exactly the runs that
|
||||
// need it.
|
||||
func TestAnotherToolSucceedingDoesNotClearAFailingToolsStreak(t *testing.T) {
|
||||
var state guardState
|
||||
stoppedAt := 0
|
||||
for i := 1; i <= toolFailureStopAt; i++ {
|
||||
state.observeToolResult("read_file", false, false, "ok", "")
|
||||
out := state.observeToolResult("write_file", true, false,
|
||||
"Error: Permission denied for write_file: "+strconv.Itoa(i), DenialPermissionDenied)
|
||||
if out.Stop {
|
||||
stoppedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if stoppedAt != toolFailureStopAt {
|
||||
t.Fatalf("denials interleaved with another tool's successes stopped at %d, want %d", stoppedAt, toolFailureStopAt)
|
||||
}
|
||||
}
|
||||
|
||||
// alwaysPromptingTool is never allowed to run: it exists so a Run-level test can
|
||||
// drive real permission denials through the loop.
|
||||
type alwaysPromptingTool struct{ ran int }
|
||||
|
||||
func (tool *alwaysPromptingTool) Name() string { return "bash" }
|
||||
func (tool *alwaysPromptingTool) Description() string { return "test shell tool" }
|
||||
func (tool *alwaysPromptingTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *alwaysPromptingTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs shell commands"}
|
||||
}
|
||||
func (tool *alwaysPromptingTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusOK, Output: "should never run"}
|
||||
}
|
||||
|
||||
// The regression for the gap this whole change existed to close, driven through
|
||||
// Run rather than through the guard helper.
|
||||
//
|
||||
// The first version of this fix was a no-op in production and every one of its
|
||||
// unit tests passed, because they called observeToolResult directly with
|
||||
// failed=true. The loop passes isRetriableToolError, which returns false for a
|
||||
// categorized denial, so the guard took its success branch and deleted the
|
||||
// record before it could key on the category. A denied tool still looped to the
|
||||
// turn limit.
|
||||
//
|
||||
// This drives the real path: a tool that always prompts, an approver that always
|
||||
// denies, and a different command each turn so the denial REASON — and therefore
|
||||
// the error text — varies exactly as it does in a real run.
|
||||
func TestRunStopsARepeatedlyDeniedToolAtTheFailureBound(t *testing.T) {
|
||||
tool := &alwaysPromptingTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// Comfortably more turns than the bound, so reaching the bound is what stops
|
||||
// the run rather than exhausting the provider or MaxTurns.
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
|
||||
for i := range toolFailureStopAt + 4 {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
|
||||
`{"command":"touch /etc/file`+strconv.Itoa(i)+`"}`))
|
||||
}
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
denials := 0
|
||||
result, err := Run(context.Background(), "do the thing", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnPermissionRequest: func(_ context.Context, request PermissionRequest) (PermissionDecision, error) {
|
||||
denials++
|
||||
// The varying half. In production this is the refused path or command;
|
||||
// here it is the command, which lands in the denial message the guard
|
||||
// used to key on.
|
||||
return PermissionDecision{
|
||||
Action: PermissionDecisionDeny,
|
||||
Reason: "refused " + request.ToolName + " call " + strconv.Itoa(denials),
|
||||
}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if denials != toolFailureStopAt {
|
||||
t.Errorf("the run made %d denied calls, want it halted at %d", denials, toolFailureStopAt)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Errorf("the denied tool executed %d times; the denial must precede execution", tool.ran)
|
||||
}
|
||||
want := toolFailureStopAnswer("bash", toolFailureStopAt, toolFailureCauseSameRefusal)
|
||||
if result.FinalAnswer != want {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The stop message must describe the bound that actually tripped. When the
|
||||
// content-blind counter is what halts the run, the same-signature streak is
|
||||
// usually 1, and reporting that would tell the user a tool failed once after it
|
||||
// failed a dozen different ways.
|
||||
func TestVariedFailureStopAnswerReportsTheRightCounter(t *testing.T) {
|
||||
var state guardState
|
||||
var outcome toolFailureOutcome
|
||||
for i := 1; i <= toolFailureAnyErrorStopAt; i++ {
|
||||
outcome = state.observeToolResult("bash", true, true, "distinct failure "+strconv.Itoa(i), "")
|
||||
if outcome.Stop {
|
||||
break
|
||||
}
|
||||
}
|
||||
if !outcome.Stop {
|
||||
t.Fatal("never stopped")
|
||||
}
|
||||
if outcome.Cause != toolFailureCauseVariedExecuted {
|
||||
t.Errorf("Cause = %v for a stop driven by the content-blind counter over executed failures", outcome.Cause)
|
||||
}
|
||||
if outcome.Count != toolFailureAnyErrorStopAt {
|
||||
t.Errorf("Count = %d, want the counter that tripped (%d)", outcome.Count, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
answer := toolFailureStopAnswer("bash", outcome.Count, outcome.Cause)
|
||||
if !strings.Contains(answer, "varying errors") {
|
||||
t.Errorf("stop answer describes the wrong cause: %q", answer)
|
||||
}
|
||||
if strings.Contains(answer, "with the same error") {
|
||||
t.Errorf("stop answer claims a same-error loop after distinct failures: %q", answer)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGuardStateResetsToolOnlyStreakOnEmptyNonToolTurn(t *testing.T) {
|
||||
var state guardState
|
||||
toolOnly := zeroruntime.CollectedStream{
|
||||
@@ -513,6 +735,22 @@ func repeatedFlakyTurns(n int) [][]zeroruntime.StreamEvent {
|
||||
return turns
|
||||
}
|
||||
|
||||
// alwaysSucceedingTool is the successful counterpart to alwaysFailingTool, used
|
||||
// to put a productive tool turn before an empty-turn streak.
|
||||
type alwaysSucceedingTool struct{}
|
||||
|
||||
func (alwaysSucceedingTool) Name() string { return "ok" }
|
||||
func (alwaysSucceedingTool) Description() string { return "always succeeds for testing" }
|
||||
func (alwaysSucceedingTool) Parameters() tools.Schema {
|
||||
return tools.Schema{Type: "object", AdditionalProperties: false}
|
||||
}
|
||||
func (alwaysSucceedingTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow}
|
||||
}
|
||||
func (alwaysSucceedingTool) Run(context.Context, map[string]any) tools.Result {
|
||||
return tools.Result{Status: tools.StatusOK, Output: "ok"}
|
||||
}
|
||||
|
||||
func TestRunStopsAfterRepeatedToolFailures(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(alwaysFailingTool{})
|
||||
@@ -529,6 +767,51 @@ func TestRunStopsAfterRepeatedToolFailures(t *testing.T) {
|
||||
if len(provider.requests) != toolFailureStopAt {
|
||||
t.Fatalf("expected stop at %d failures, made %d requests", toolFailureStopAt, len(provider.requests))
|
||||
}
|
||||
if result.Incomplete {
|
||||
t.Fatalf("interactive run must not be marked Incomplete; reason=%q", result.IncompleteReason)
|
||||
}
|
||||
}
|
||||
|
||||
// Under the headless completion gate both guardrail stops are unfinished runs,
|
||||
// so they must set Result.Incomplete and a reason; exec maps that to exit 4.
|
||||
// Thresholds and stop answers stay pinned by the two tests above.
|
||||
func TestHeadlessGuardStopsAreIncomplete(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
turns [][]zeroruntime.StreamEvent
|
||||
wantTurns int
|
||||
wantReason string
|
||||
}{
|
||||
{"empty_turns", [][]zeroruntime.StreamEvent{emptyTurn(), emptyTurn(), emptyTurn()}, maxEmptyTurns, "3 consecutive turns with no output"},
|
||||
{"empty_turns_after_progress", [][]zeroruntime.StreamEvent{
|
||||
toolTurn("ok-1", "ok", "{}"),
|
||||
emptyTurn(), emptyTurn(), emptyTurn(),
|
||||
}, maxEmptyTurns + 1, "3 consecutive turns with no output"},
|
||||
{"repeated_tool_failures", repeatedFlakyTurns(10), toolFailureStopAt, "flaky"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(alwaysFailingTool{})
|
||||
registry.Register(alwaysSucceedingTool{})
|
||||
result, err := Run(context.Background(), "go", &mockProvider{turns: tc.turns}, Options{
|
||||
Registry: registry,
|
||||
MaxTurns: 12,
|
||||
RequireCompletionSignal: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if result.Turns != tc.wantTurns {
|
||||
t.Fatalf("guard did not trigger: turns=%d want=%d", result.Turns, tc.wantTurns)
|
||||
}
|
||||
if !result.Incomplete {
|
||||
t.Fatalf("guardrail stop under the headless gate must be Incomplete; final=%q", result.FinalAnswer)
|
||||
}
|
||||
if !strings.Contains(result.IncompleteReason, tc.wantReason) {
|
||||
t.Fatalf("IncompleteReason = %q, want it to mention %q", result.IncompleteReason, tc.wantReason)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunInjectsToolFailureHintWithSchema(t *testing.T) {
|
||||
|
||||
@@ -5,8 +5,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/hooks"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/hooks"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
func TestAppendHookFeedbackFormatsOutput(t *testing.T) {
|
||||
|
||||
+233
-31
@@ -11,15 +11,15 @@ import (
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/execution"
|
||||
"github.com/Gitlawb/zero/internal/hooks"
|
||||
"github.com/Gitlawb/zero/internal/modelregistry"
|
||||
"github.com/Gitlawb/zero/internal/redaction"
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/streamjson"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/execution"
|
||||
"github.com/Twigpine/zero/internal/hooks"
|
||||
"github.com/Twigpine/zero/internal/modelregistry"
|
||||
"github.com/Twigpine/zero/internal/redaction"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/streamjson"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
const maxTurnsAnswer = "Agent reached maximum number of turns without a final answer."
|
||||
@@ -413,7 +413,8 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// collected and a non-nil stop error when the run must end now.
|
||||
recoverStreamError := func(collected zeroruntime.CollectedStream) (zeroruntime.CollectedStream, error) {
|
||||
if isImageRejectionError(errors.New(collected.Error)) {
|
||||
return collected, fmt.Errorf("model %s rejected the image: %s. The model may not support image input — try switching to a vision-capable model (claude, gpt-4o, gemini)", options.Model, collected.Error)
|
||||
cause := &zeroruntime.StreamError{Message: collected.Error, StatusCode: collected.ErrorStatusCode, Cause: collected.ErrorCause}
|
||||
return collected, fmt.Errorf("model %s rejected the image: %w. The model may not support image input — try switching to a vision-capable model (claude, gpt-4o, gemini)", options.Model, cause)
|
||||
}
|
||||
// REACTIVE compaction: the streamed error may also be a context limit
|
||||
// (some providers surface it mid-stream). Compact and retry once.
|
||||
@@ -542,7 +543,7 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
}
|
||||
if collected.Error != "" {
|
||||
result.Messages = copyMessages(messages)
|
||||
return result, errors.New(collected.Error)
|
||||
return result, &zeroruntime.StreamError{Message: collected.Error, StatusCode: collected.ErrorStatusCode, Cause: collected.ErrorCause}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -587,6 +588,14 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// counted toward the runaway cap so we stop before burning maxTurns.
|
||||
if guards.observeTurn(collected) {
|
||||
result.FinalAnswer = noOutputStopAnswer(result.Turns)
|
||||
// Under the headless completion gate this stop is unfinished work,
|
||||
// not a successful final answer: the guard exists precisely because
|
||||
// the model made no progress. Interactive runs keep the gate off, so
|
||||
// their behavior is unchanged.
|
||||
if options.RequireCompletionSignal {
|
||||
result.Incomplete = true
|
||||
result.IncompleteReason = "stopped after " + strconv.Itoa(guards.emptyTurns) + " consecutive turns with no output"
|
||||
}
|
||||
result.Messages = copyMessages(messages)
|
||||
return result, nil
|
||||
}
|
||||
@@ -711,6 +720,60 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// executed concurrently, consumed strictly in order below.
|
||||
var precomputed []precomputedToolResult
|
||||
precomputedStart, precomputedEnd := 0, 0
|
||||
// closeOutRemaining terminates every advertised call from `from` onward
|
||||
// TRUTHFULLY, and every early return in this loop goes through it.
|
||||
//
|
||||
// Read-ahead broke the assumption the abort placeholders were written
|
||||
// under. executeParallelReadBatch runs an entire eligible run before the
|
||||
// loop consumes any of it, so "not consumed yet" stopped meaning "not
|
||||
// executed": a sibling can already have run, committed its side effects
|
||||
// and any authorization credit they carry, and still sit past the index
|
||||
// where a terminal branch fires. Recording it as aborted threw that result
|
||||
// away, so the transcript disagreed with what had actually happened, and
|
||||
// its callbacks, trace counter, task observation and images went with it.
|
||||
//
|
||||
// So each remaining call is put in the one state that is true of it:
|
||||
// completed, and finalized exactly once with the same bookkeeping the main
|
||||
// path performs, or unstarted, and aborted. The guard is deliberately NOT
|
||||
// consulted for a drained sibling: these results cannot reverse a stop
|
||||
// decision that has already been made, they are only owed an honest record.
|
||||
closeOutRemaining := func(from int) {
|
||||
for next := from; next < len(collected.ToolCalls); next++ {
|
||||
sibling, ran := precomputedResultFor(precomputed, precomputedStart, precomputedEnd, next)
|
||||
if !ran {
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[next:next+1])
|
||||
continue
|
||||
}
|
||||
nextCall := collected.ToolCalls[next]
|
||||
if options.OnToolCall != nil {
|
||||
options.OnToolCall(nextCall)
|
||||
}
|
||||
options.Trace.Counter(trace.CounterToolCalls, 1)
|
||||
recordOutputBudgetTrace(options.Trace, sibling)
|
||||
task.observe(taskStateEvent{kind: taskStateEventToolResult, arguments: nextCall.Arguments, toolResult: sibling})
|
||||
if options.OnToolResult != nil {
|
||||
options.OnToolResult(sibling)
|
||||
}
|
||||
for _, name := range sibling.LoadedTools {
|
||||
loaded[name] = true
|
||||
}
|
||||
// The same fields the consumption path below records, provider call
|
||||
// ID included: a strict replay pairs the result to its tool_use by
|
||||
// that ID, and a finalized sibling without it was the one result in
|
||||
// the turn that could not be paired. Reported by @jatmn.
|
||||
messages = append(messages, zeroruntime.Message{
|
||||
Role: zeroruntime.MessageRoleTool,
|
||||
Content: sibling.ModelOutput(),
|
||||
ToolCallID: sibling.ToolCallID,
|
||||
ToolCallProviderID: nextCall.ProviderCallID,
|
||||
IsError: sibling.Status == tools.StatusError,
|
||||
ChangedFiles: append([]string(nil), sibling.ChangedFiles...),
|
||||
})
|
||||
if len(sibling.Images) > 0 {
|
||||
toolResultsWithImages = append(toolResultsWithImages, sibling)
|
||||
}
|
||||
}
|
||||
}
|
||||
for index, call := range collected.ToolCalls {
|
||||
// When this call starts a consecutive run of >= 2 auto-allowed read-only
|
||||
// calls, execute the whole run concurrently now (see parallel_tools.go).
|
||||
@@ -775,13 +838,13 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
abortErr = ctx.Err()
|
||||
}
|
||||
if abortErr != nil {
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
|
||||
closeOutRemaining(index + 1)
|
||||
messages = append(messages, buildToolImageMessages()...)
|
||||
result.Messages = copyMessages(messages)
|
||||
return result, abortErr
|
||||
}
|
||||
if stopReason := stopReasonFromToolResult(toolResult); stopReason != "" {
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
|
||||
closeOutRemaining(index + 1)
|
||||
messages = append(messages, buildToolImageMessages()...)
|
||||
result.FinalAnswer = toolResult.ModelOutput()
|
||||
result.StopReason = stopReason
|
||||
@@ -796,8 +859,29 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// aren't fixed by reformatting the call, so a "match this schema" hint
|
||||
// would misdirect the model toward JSON shape or blocked behavior.
|
||||
retriableFailure := isRetriableToolError(toolResult)
|
||||
outcome := guards.observeToolResult(call.Name, retriableFailure, toolResult.ModelOutput())
|
||||
posture.observeToolOutcome(outcome, toolResult)
|
||||
// A categorized denial is NOT retriable — retrying it verbatim is
|
||||
// pointless — but it is still a failure the streaks must count, or a
|
||||
// refused tool loops until the turn limit. Passing retriableFailure for
|
||||
// both is what let that happen: observeToolResult took its success
|
||||
// branch and deleted the record before it could key on the category.
|
||||
// isPolicyRefusal rather than DenialReason alone: the categories are
|
||||
// only attached where a typed denial is built, so a headless prompt
|
||||
// refusal and an uncategorized sandbox block were counted as SUCCESS
|
||||
// and cleared the record they were supposed to accumulate.
|
||||
policyRefusal := isPolicyRefusal(toolResult)
|
||||
countedFailure := retriableFailure || policyRefusal
|
||||
outcome := guards.observeToolResult(call.Name, countedFailure, retriableFailure, toolResult.ModelOutput(), toolResult.DenialReason)
|
||||
// The profile's failure-streak trigger is for RETRIABLE failures: it
|
||||
// restores the displaced turn budget and reasoning effort on the theory
|
||||
// that the tool is struggling and needs room. A policy refusal is not a
|
||||
// struggling tool, it is an answer, and spending the one-shot
|
||||
// escalation on one contradicts that trigger's own contract. Denials
|
||||
// still count for the halt above; they just do not buy more budget.
|
||||
if policyRefusal {
|
||||
posture.observeToolOutcome(toolFailureOutcome{}, toolResult)
|
||||
} else {
|
||||
posture.observeToolOutcome(outcome, toolResult)
|
||||
}
|
||||
if outcome.Stop {
|
||||
// The assistant message advertised EVERY collected tool call, but
|
||||
// the guard halts mid-turn so the calls after this one never run.
|
||||
@@ -805,10 +889,22 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
|
||||
// every tool_use has a matching tool_result and the recorded
|
||||
// messages stay valid for a strict provider replay (Anthropic
|
||||
// rejects a tool_use with no answering tool_result).
|
||||
messages = appendAbortedToolResults(messages, collected.ToolCalls[index+1:])
|
||||
closeOutRemaining(index + 1)
|
||||
messages = append(messages, buildToolImageMessages()...)
|
||||
result.FinalAnswer = toolFailureStopAnswer(call.Name, outcome.Count)
|
||||
result.FinalAnswer = toolFailureStopAnswer(call.Name, outcome.Count, outcome.Cause)
|
||||
result.Messages = copyMessages(messages)
|
||||
// THE HALT HAS TO CROSS THE TERMINAL-STATUS BOUNDARY TOO.
|
||||
//
|
||||
// This branch returns straight out, so it never reached the
|
||||
// completion gate that the max-turns paths below go through. Under
|
||||
// RequireCompletionSignal `zero exec` treats only Incomplete as exit
|
||||
// 4 and otherwise emits run_end("success", 0), so a task that was
|
||||
// denied six times, or failed through the twelve-call bound, became
|
||||
// a successful automation result having done none of the work.
|
||||
if options.RequireCompletionSignal {
|
||||
result.Incomplete = true
|
||||
result.IncompleteReason = toolFailureIncompleteReason(call.Name, outcome.Cause)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
if outcome.InjectHint && failureHint == "" {
|
||||
@@ -1590,9 +1686,41 @@ func executeToolCall(ctx context.Context, registry *tools.Registry, call ToolCal
|
||||
// the Run turn loop performs the actual provider switch. Empty for every
|
||||
// ordinary tool result.
|
||||
RequestedModel: result.Meta["escalate_to_model"],
|
||||
// ONE REFUSAL IDENTITY, derived here so classification and streak
|
||||
// accounting cannot disagree.
|
||||
//
|
||||
// The registry marks its pre-execution refusals in metadata, and
|
||||
// isPolicyRefusal read that marker while observeToolResult keyed on
|
||||
// DenialReason, which those paths leave empty. The guard therefore fell
|
||||
// back to errorSignature(output), and two refusals of the same category
|
||||
// with different wording looked like two different failures. A model
|
||||
// alternating capture_artifact's browser_screenshot and browser_pdf against
|
||||
// a disabled driver is refused identically each time, and never tripped the
|
||||
// six-call refusal halt: only the generic twelve-error fallback stopped it,
|
||||
// reporting varied errors rather than a repeated refusal.
|
||||
DenialReason: denialCategoryForResult(result),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// denialCategoryForResult gives every pre-execution refusal a stable category,
|
||||
// whether it was built as a typed denial or only marked in metadata.
|
||||
//
|
||||
// The mapping lives here, at the one boundary tools.Result becomes ToolResult,
|
||||
// rather than at each producer. A marker without a category is the shape that
|
||||
// caused this: it classified as a refusal and keyed as an error signature.
|
||||
func denialCategoryForResult(result tools.Result) DenialCategory {
|
||||
switch result.Meta[tools.PolicyRefusalMeta] {
|
||||
case tools.PolicyRefusalToolNotEnabled:
|
||||
return DenialFiltered
|
||||
case tools.PolicyRefusalPermissionDenied, tools.PolicyRefusalPermissionRequired:
|
||||
return DenialPermissionDenied
|
||||
case tools.PolicyRefusalSandboxDenied, tools.PolicyRefusalSandboxApproval:
|
||||
return DenialSandboxBlock
|
||||
default:
|
||||
return DenialNone
|
||||
}
|
||||
}
|
||||
|
||||
const sandboxNamespaceLimitedReason = "sandbox output is limited to the sandbox PID namespace; host/global state requires approval"
|
||||
|
||||
func maybeRetryUnsandboxedAfterSandboxRestriction(ctx context.Context, registry *tools.Registry, call ToolCall, tool tools.Tool, args map[string]any, result tools.Result, permissionMode PermissionMode, options Options, progressCallback func(streamjson.Event)) (tools.Result, *ToolResult, bool, PermissionDecisionAction, string, []string, error) {
|
||||
@@ -1923,6 +2051,10 @@ func toolResultFromPrePermissionReject(call ToolCall, result tools.Result) ToolR
|
||||
Display: display,
|
||||
LoadedTools: loadedToolsFromResult(meta),
|
||||
RequestedModel: meta["escalate_to_model"],
|
||||
// The SAME identity the executed path derives. This is the route a
|
||||
// RejectBeforePermission refusal takes, so leaving it empty here is what
|
||||
// made the marker classify as a refusal and key as an error signature.
|
||||
DenialReason: denialCategoryForResult(tools.Result{Meta: meta}),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2094,21 +2226,64 @@ func isRetriableToolError(result ToolResult) bool {
|
||||
// A categorized denial (filtered / permission / sandbox) is a policy decision,
|
||||
// not a transient failure — never retry it. This is robust to message wording
|
||||
// (the text checks below remain as a fallback for results lacking the field).
|
||||
return !isPolicyRefusal(result)
|
||||
}
|
||||
|
||||
// isPolicyRefusal reports a result the run refused on policy grounds: a
|
||||
// permission gate, a filter, a sandbox preflight, or a hook.
|
||||
//
|
||||
// Split out of isRetriableToolError so the two questions cannot drift apart,
|
||||
// which is exactly what they had done. The halt guard asked
|
||||
// `DenialReason != DenialNone`, and a category is only attached on the paths
|
||||
// that build a TYPED denial. A headless run leaves OnPermissionRequest nil, so
|
||||
// the loop never reaches its typed-denial branch and the registry returns a bare
|
||||
// `Error: Permission required ...` carrying no category. A sandbox preflight
|
||||
// denial on a non-shell tool loses its SandboxDecision converting to ToolResult
|
||||
// and arrives as an uncategorized `Sandbox block`.
|
||||
//
|
||||
// Both are refusals, and both were counted as SUCCESS, so observeToolResult
|
||||
// cleared the record and the same refused call could repeat to MaxTurns. That is
|
||||
// the loop this guard exists to stop. The text checks below already enumerated
|
||||
// these outcomes for the retriable question; they simply were not reachable from
|
||||
// the counting one.
|
||||
func isPolicyRefusal(result ToolResult) bool {
|
||||
// A REFUSAL IS A FAILURE. Everything below inspects metadata and then output
|
||||
// text, and neither is meaningful on a result the tool completed.
|
||||
//
|
||||
// isRetriableToolError gates on this before it calls here, so the boundary
|
||||
// held while that was the only caller. Extracting this helper and calling it
|
||||
// straight from the counting path dropped the gate: an allowed `bash`
|
||||
// printing "Sandbox block", or a read_file returning a document that quotes
|
||||
// one of these phrases, was counted as a denial. Six such successes tripped
|
||||
// the same-signature stop and ended a healthy run with a refusal answer.
|
||||
//
|
||||
// The gate belongs here rather than at each caller, because the next caller
|
||||
// will forget it too.
|
||||
if result.Status != tools.StatusError {
|
||||
return false
|
||||
}
|
||||
if result.DenialReason != DenialNone {
|
||||
return false
|
||||
return true
|
||||
}
|
||||
if result.Meta["permission_action"] == string(PermissionActionDeny) {
|
||||
return false
|
||||
if result.Meta[tools.PermissionActionMeta] == string(PermissionActionDeny) {
|
||||
return true
|
||||
}
|
||||
switch {
|
||||
case strings.Contains(result.Output, "is not enabled for this run"),
|
||||
strings.Contains(result.Output, "Permission denied for "),
|
||||
strings.Contains(result.Output, "Permission required for "),
|
||||
strings.Contains(result.Output, "Sandbox block"),
|
||||
strings.Contains(result.Output, "Sandbox approval required for "):
|
||||
return false
|
||||
}
|
||||
return true
|
||||
// STRUCTURED ONLY. This used to fall back to matching phrases in Output
|
||||
// ("Sandbox block", "Permission denied for ", and so on), which cannot work:
|
||||
// Output is tool-controlled. `bash` preserves arbitrary stdout and stderr on
|
||||
// a StatusError for any nonzero exit, so an ALLOWED command running
|
||||
// `printf 'Sandbox block\n' >&2; exit 1` has actually executed, carries no
|
||||
// DenialReason, and was still classified as refused. The loop then withheld
|
||||
// the schema hint and the profile failure-streak recovery, counted the
|
||||
// executed failure toward the refusal halt, and a later stop told the user a
|
||||
// tool had been refused when it had run.
|
||||
//
|
||||
// read_file returning a document that quotes one of those phrases did the
|
||||
// same thing, which is the more likely way a real session hits it.
|
||||
//
|
||||
// The registry now marks every path that returns BEFORE the tool runs, so
|
||||
// the question is answerable from provenance. Nothing below reads Output.
|
||||
return tools.IsPolicyRefusalResult(tools.Result{Meta: result.Meta})
|
||||
}
|
||||
|
||||
// scrubInterceptedOutput mirrors the registry's scrubResultSecrets boundary for
|
||||
@@ -2674,7 +2849,7 @@ func deniedPermissionResult(call ToolCall, reason string, requestEvent Permissio
|
||||
Output: "Error: Permission denied for " + call.Name + ": " + reason,
|
||||
DenialReason: denial,
|
||||
Meta: map[string]string{
|
||||
"permission_action": string(event.Action),
|
||||
tools.PermissionActionMeta: string(event.Action),
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -2700,7 +2875,7 @@ func canceledPermissionResult(call ToolCall, reason string, requestEvent Permiss
|
||||
Output: "Error: Permission approval cancelled for " + call.Name + ": " + reason,
|
||||
DenialReason: DenialApprovalCanceled,
|
||||
Meta: map[string]string{
|
||||
"permission_action": string(event.Action),
|
||||
tools.PermissionActionMeta: string(event.Action),
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -3554,6 +3729,33 @@ func toolResultImageMessage(result ToolResult, options Options) (zeroruntime.Mes
|
||||
}, true
|
||||
}
|
||||
|
||||
// precomputedResultFor reports the read-ahead result for call index, and whether
|
||||
// that call actually produced one. A batch entry that never started is still
|
||||
// unstarted as far as the transcript is concerned.
|
||||
//
|
||||
// PRODUCING A RESULT AND ASKING THE RUN TO STOP ARE DIFFERENT FACTS. Treating a
|
||||
// non-nil abortErr as "unstarted" conflated them, and a cancelled permission
|
||||
// request is exactly the case where they disagree: executeToolCall builds the
|
||||
// cancellation result and returns it WITH ErrPermissionApprovalCanceled, so an
|
||||
// earlier sibling reaching a terminal branch discarded a real result and wrote
|
||||
// an aborted placeholder in its place. The transcript then denied that the call
|
||||
// had completed permission handling, and the cancellation was missing from
|
||||
// OnToolResult, the trace counters and the task observation even though the
|
||||
// permission event had already happened.
|
||||
//
|
||||
// The terminal decision is not affected: draining such a sibling only records
|
||||
// what occurred, it cannot reverse a stop already selected.
|
||||
func precomputedResultFor(precomputed []precomputedToolResult, start, end, index int) (ToolResult, bool) {
|
||||
if index < start || index >= end {
|
||||
return ToolResult{}, false
|
||||
}
|
||||
entry := precomputed[index-start]
|
||||
if !entry.completed {
|
||||
return ToolResult{}, false
|
||||
}
|
||||
return entry.result, true
|
||||
}
|
||||
|
||||
func modelAcceptsToolImages(options Options) bool {
|
||||
if options.SupportsVision != nil {
|
||||
return options.SupportsVision(options.Model)
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// imageEchoProvider records the messages of the first request it receives, then
|
||||
|
||||
+103
-11
@@ -13,13 +13,13 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/execution"
|
||||
"github.com/Gitlawb/zero/internal/hooks"
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/specmode"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/execution"
|
||||
"github.com/Twigpine/zero/internal/hooks"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/specmode"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
type mockProvider struct {
|
||||
@@ -685,11 +685,23 @@ func TestIsRetriableToolError(t *testing.T) {
|
||||
{"success", ToolResult{Status: tools.StatusOK}, false},
|
||||
{"bad arguments", ToolResult{Status: tools.StatusError, Output: "Error: Failed to parse arguments for x: bad json"}, true},
|
||||
{"execution failure", ToolResult{Status: tools.StatusError, Output: "Error: read foo.txt: no such file"}, true},
|
||||
{"disabled tool", ToolResult{Status: tools.StatusError, Output: `Error: Tool "x" is not enabled for this run.`}, false},
|
||||
// Each refusal carries the provenance its production path attaches: the
|
||||
// disabled-tool path sets a denial category, the registry gates set the
|
||||
// pre-execution marker. Text is never what decides.
|
||||
{"disabled tool", ToolResult{Status: tools.StatusError, Output: `Error: Tool "x" is not enabled for this run.`, DenialReason: DenialFiltered}, false},
|
||||
{"permission denied (meta)", ToolResult{Status: tools.StatusError, Output: "Error: Permission denied for x: needs approval", Meta: map[string]string{"permission_action": "deny"}}, false},
|
||||
{"permission required", ToolResult{Status: tools.StatusError, Output: "Error: Permission required for x: approve first"}, false},
|
||||
{"sandbox block", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox block: outside_workspace"}, false},
|
||||
{"sandbox approval", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox approval required for x: network"}, false},
|
||||
{"permission required", ToolResult{Status: tools.StatusError, Output: "Error: Permission required for x: approve first", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalPermissionRequired}}, false},
|
||||
{"sandbox block", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox block: outside_workspace", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied}}, false},
|
||||
{"sandbox approval", ToolResult{Status: tools.StatusError, Output: "Error: Sandbox approval required for x: network", Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxApproval}}, false},
|
||||
// THE INVERSE, which is the failure this classifier had. An allowed tool
|
||||
// that RAN, failed, and happened to print one of those phrases must stay
|
||||
// retriable: `bash` preserves arbitrary stdout and stderr on any nonzero
|
||||
// exit, and read_file returns whatever the file says.
|
||||
{"executed failure printing the sandbox phrase", ToolResult{Status: tools.StatusError, Output: "Sandbox block\n"}, true},
|
||||
{"executed failure printing a permission denial", ToolResult{Status: tools.StatusError, Output: "cp: Permission denied for /etc/hosts"}, true},
|
||||
{"executed failure printing a permission requirement", ToolResult{Status: tools.StatusError, Output: "Permission required for sudo"}, true},
|
||||
{"executed failure printing the approval phrase", ToolResult{Status: tools.StatusError, Output: "Sandbox approval required for curl"}, true},
|
||||
{"executed failure quoting the disabled-tool phrase", ToolResult{Status: tools.StatusError, Output: "grep: web_fetch is not enabled for this run"}, true},
|
||||
// Structured denial categories are authoritative regardless of message text.
|
||||
{"denial: filtered", ToolResult{Status: tools.StatusError, Output: "anything", DenialReason: DenialFiltered}, false},
|
||||
{"denial: permission", ToolResult{Status: tools.StatusError, Output: "anything", DenialReason: DenialPermissionDenied}, false},
|
||||
@@ -4193,6 +4205,86 @@ func TestRunNilTraceForwardsUsage(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunReturnsStreamErrorWithStatusCodeAndCause covers the collector-to-agent
|
||||
// propagation boundary: a provider stream that terminates with a StreamEventError
|
||||
// carrying StatusCode/Cause must have Run return an error that errors.As can
|
||||
// still recover those fields from — the detail a session's "error" event needs
|
||||
// to actually diagnose why a provider call failed (#674).
|
||||
func TestRunReturnsStreamErrorWithStatusCodeAndCause(t *testing.T) {
|
||||
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
|
||||
{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: "provider error: provider returned error",
|
||||
StatusCode: 500,
|
||||
Cause: "upstream returned an empty completion",
|
||||
},
|
||||
}}}
|
||||
|
||||
_, err := Run(context.Background(), "hi", provider, Options{
|
||||
SessionID: "stream-error-session",
|
||||
Cwd: t.TempDir(),
|
||||
ProviderName: "test-provider",
|
||||
Model: "test-model",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("Run: expected an error, got nil")
|
||||
}
|
||||
|
||||
var streamErr *zeroruntime.StreamError
|
||||
if !errors.As(err, &streamErr) {
|
||||
t.Fatalf("Run error %v (%T) does not unwrap to *zeroruntime.StreamError", err, err)
|
||||
}
|
||||
if streamErr.StatusCode != 500 {
|
||||
t.Fatalf("StatusCode = %d, want 500", streamErr.StatusCode)
|
||||
}
|
||||
if streamErr.Cause != "upstream returned an empty completion" {
|
||||
t.Fatalf("Cause = %q, want %q", streamErr.Cause, "upstream returned an empty completion")
|
||||
}
|
||||
if err.Error() != "provider error: provider returned error" {
|
||||
t.Fatalf("Error() = %q, want the flattened message unchanged", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunImageRejectionKeepsStreamErrorStatusCodeAndCause covers the 400
|
||||
// image-rejection recovery path: the friendly message is kept, but the returned
|
||||
// error must still unwrap to the *StreamError so StatusCode and Cause reach the
|
||||
// session's "error" event (#674).
|
||||
func TestRunImageRejectionKeepsStreamErrorStatusCodeAndCause(t *testing.T) {
|
||||
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
|
||||
{
|
||||
Type: zeroruntime.StreamEventError,
|
||||
Error: "provider error 400: image input is not supported",
|
||||
StatusCode: 400,
|
||||
Cause: "this model does not support image input",
|
||||
},
|
||||
}}}
|
||||
|
||||
_, err := Run(context.Background(), "hi", provider, Options{
|
||||
SessionID: "image-rejection-session",
|
||||
Cwd: t.TempDir(),
|
||||
ProviderName: "test-provider",
|
||||
Model: "test-model",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("Run: expected an error, got nil")
|
||||
}
|
||||
|
||||
want := "model test-model rejected the image: provider error 400: image input is not supported. The model may not support image input — try switching to a vision-capable model (claude, gpt-4o, gemini)"
|
||||
if err.Error() != want {
|
||||
t.Fatalf("Error() = %q, want the friendly message unchanged: %q", err.Error(), want)
|
||||
}
|
||||
var streamErr *zeroruntime.StreamError
|
||||
if !errors.As(err, &streamErr) {
|
||||
t.Fatalf("Run error %v (%T) does not unwrap to *zeroruntime.StreamError", err, err)
|
||||
}
|
||||
if streamErr.StatusCode != 400 {
|
||||
t.Fatalf("StatusCode = %d, want 400", streamErr.StatusCode)
|
||||
}
|
||||
if streamErr.Cause != "this model does not support image input" {
|
||||
t.Fatalf("Cause = %q, want the upstream cause", streamErr.Cause)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunSuppressesAdvisoryHooksInPlanMode: plan mode promises a read-only
|
||||
// turn for advisory hooks (sessionStart/sessionEnd/afterTool), which execute
|
||||
// configured host commands outside the advertised-tool and sandbox gates.
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
type safetyTool struct {
|
||||
|
||||
@@ -8,9 +8,9 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// midStreamAbortProvider connects successfully but emits a transport-abort
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// mixedOutcomeTool needs permission on every call. The test denies every other
|
||||
// request, and a call that is allowed executes and fails with an error naming
|
||||
// its own item, so no single identity repeats and only the content-blind bound
|
||||
// can stop the run.
|
||||
type mixedOutcomeTool struct{ ran int }
|
||||
|
||||
func (tool *mixedOutcomeTool) Name() string { return "bash" }
|
||||
func (tool *mixedOutcomeTool) Description() string { return "probe" }
|
||||
func (tool *mixedOutcomeTool) Parameters() tools.Schema { return tools.Schema{Type: "object"} }
|
||||
func (tool *mixedOutcomeTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs a command"}
|
||||
}
|
||||
|
||||
func (tool *mixedOutcomeTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusError, Output: "Error: item " + strconv.Itoa(tool.ran) + " failed"}
|
||||
}
|
||||
|
||||
// A MIXED HALT READS THE SAME HEADLESS AS INTERACTIVE.
|
||||
//
|
||||
// The stop answer for the content-blind bound over executed failures and
|
||||
// refusals says the tool "failed or was refused". The headless reason for the
|
||||
// same halt said only that it "failed repeatedly", so zero exec and the
|
||||
// interactive answer disagreed about one stop. Reported by @jatmn.
|
||||
func TestAMixedGuardHaltIsReportedAsMixedUnderTheCompletionGate(t *testing.T) {
|
||||
tool := &mixedOutcomeTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
|
||||
for i := range toolFailureAnyErrorStopAt + 4 {
|
||||
turns = append(turns, toolTurn("c"+strconv.Itoa(i), "bash", `{"item":`+strconv.Itoa(i)+`}`))
|
||||
}
|
||||
asked := 0
|
||||
result, err := Run(context.Background(), "run them", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
RequireCompletionSignal: true,
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
asked++
|
||||
if asked%2 == 0 {
|
||||
return PermissionDecision{Action: PermissionDecisionAllow}, nil
|
||||
}
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran == 0 || asked <= tool.ran {
|
||||
t.Fatalf("the run did not mix executed failures with refusals (ran %d of %d asked), so this proves nothing", tool.ran, asked)
|
||||
}
|
||||
if !strings.Contains(result.FinalAnswer, "failed or was refused") {
|
||||
t.Fatalf("the halt was not the mixed content-blind bound: %q", result.FinalAnswer)
|
||||
}
|
||||
if !result.Incomplete {
|
||||
t.Fatal("a guard halt under the completion gate must be Incomplete")
|
||||
}
|
||||
if !strings.Contains(result.IncompleteReason, "failed or was refused") || !strings.Contains(result.IncompleteReason, "bash") {
|
||||
t.Errorf("IncompleteReason = %q, want it to say the tool failed or was refused, as the stop answer does", result.IncompleteReason)
|
||||
}
|
||||
}
|
||||
@@ -8,10 +8,10 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/hooks"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/hooks"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
type propagationOutputTool struct {
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// readAheadTool is read-only and thread-safe, so a consecutive run of its calls
|
||||
// is executed as one parallel batch before the loop consumes any of it.
|
||||
type readAheadTool struct {
|
||||
mu sync.Mutex
|
||||
ran int
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Name() string { return "read_probe" }
|
||||
func (t *readAheadTool) Description() string { return "read-only probe" }
|
||||
func (t *readAheadTool) Parameters() tools.Schema {
|
||||
return tools.Schema{Type: "object", Properties: map[string]tools.PropertySchema{"id": {Type: "string"}}}
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "test"}
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Capabilities() tools.ToolCapabilities {
|
||||
return tools.ToolCapabilities{Effect: tools.EffectReadOnly, ThreadSafe: true}
|
||||
}
|
||||
|
||||
func (t *readAheadTool) Run(_ context.Context, args map[string]any) tools.Result {
|
||||
t.mu.Lock()
|
||||
t.ran++
|
||||
t.mu.Unlock()
|
||||
id, _ := args["id"].(string)
|
||||
// Varying output, so no identity repeats and only the content-blind counter
|
||||
// can halt the run.
|
||||
return tools.Result{Status: tools.StatusError, Output: "failure for " + id}
|
||||
}
|
||||
|
||||
func readCall(id string) []zeroruntime.StreamEvent {
|
||||
return []zeroruntime.StreamEvent{
|
||||
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: id, ToolCallProviderID: "provider-" + id, ToolName: "read_probe"},
|
||||
{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: id, ArgumentsFragment: `{"id":"` + id + `"}`},
|
||||
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: id},
|
||||
}
|
||||
}
|
||||
|
||||
// READ-AHEAD MEANS "NOT CONSUMED YET" NO LONGER MEANS "NOT EXECUTED".
|
||||
//
|
||||
// executeParallelReadBatch runs an entire eligible run of read calls before the
|
||||
// loop consumes any of them. Every terminal branch in the consumption loop used
|
||||
// to close out the calls after the current index with aborted placeholders, on
|
||||
// the assumption that they had not run. A sibling that had already executed was
|
||||
// therefore recorded as aborted: its real result was discarded, the transcript
|
||||
// disagreed with what had happened, and its callbacks, trace counter, task
|
||||
// observation and images were lost with it. Where such a sibling is a
|
||||
// successful read, execution may already have committed file-observation credit
|
||||
// for content the model never receives.
|
||||
func TestParallelSiblingsAreFinalizedWhenTheGuardHalts(t *testing.T) {
|
||||
tool := &readAheadTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// Eleven single-call turns bring the content-blind counter to one below its
|
||||
// bound, so the first call of the pair below trips it.
|
||||
var turns [][]zeroruntime.StreamEvent
|
||||
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
|
||||
id := "s" + strconv.Itoa(i)
|
||||
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
}
|
||||
// Then one turn advertising two parallel-eligible calls. Both execute in the
|
||||
// batch; the halt fires while consuming the first.
|
||||
pair := append(readCall("pA"), readCall("pB")...)
|
||||
turns = append(turns, append(pair, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
turns = append(turns, append(readCall("unreached"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
|
||||
var reported []ToolResult
|
||||
var announced int
|
||||
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAuto,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnToolCall: func(ToolCall) { announced++ },
|
||||
OnToolResult: func(r ToolResult) { reported = append(reported, r) },
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
|
||||
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
|
||||
}
|
||||
|
||||
// Every execution is reported exactly once, and the announcements match.
|
||||
if tool.ran != len(reported) {
|
||||
t.Errorf("the tool executed %d times but %d results were reported; a completed call was recorded as aborted",
|
||||
tool.ran, len(reported))
|
||||
}
|
||||
if announced != len(reported) {
|
||||
t.Errorf("OnToolCall fired %d times against %d results; the two callbacks disagree", announced, len(reported))
|
||||
}
|
||||
|
||||
// Strict pairing: one tool result per advertised tool call, and the drained
|
||||
// sibling carries its REAL output rather than the aborted placeholder.
|
||||
byID := map[string]int{}
|
||||
var siblingContent string
|
||||
for _, message := range result.Messages {
|
||||
if message.Role != zeroruntime.MessageRoleTool {
|
||||
continue
|
||||
}
|
||||
byID[message.ToolCallID]++
|
||||
if message.ToolCallID == "pB" {
|
||||
siblingContent = message.Content
|
||||
}
|
||||
// A strict replay pairs each result to its tool_use by the provider
|
||||
// call ID, so the drained sibling needs it as much as the consumed call.
|
||||
if (message.ToolCallID == "pA" || message.ToolCallID == "pB") && message.ToolCallProviderID != "provider-"+message.ToolCallID {
|
||||
t.Errorf("tool result %s carries provider call ID %q, want %q", message.ToolCallID, message.ToolCallProviderID, "provider-"+message.ToolCallID)
|
||||
}
|
||||
}
|
||||
for _, id := range []string{"pA", "pB"} {
|
||||
if byID[id] != 1 {
|
||||
t.Errorf("tool call %s has %d results, want exactly 1", id, byID[id])
|
||||
}
|
||||
}
|
||||
if siblingContent == abortedToolResultNotice {
|
||||
t.Error("the sibling that already ran was recorded as aborted")
|
||||
}
|
||||
if !strings.Contains(siblingContent, "failure for pB") {
|
||||
t.Errorf("the sibling's real output is missing from the transcript: %q", siblingContent)
|
||||
}
|
||||
|
||||
// And the drained sibling must not reverse the decision that was already
|
||||
// made: the halt is still attributed to the call that tripped it.
|
||||
if !strings.Contains(result.FinalAnswer, strconv.Itoa(toolFailureAnyErrorStopAt)) {
|
||||
t.Errorf("the stop answer changed after draining siblings: %q", result.FinalAnswer)
|
||||
}
|
||||
}
|
||||
|
||||
// imageReadTool is readAheadTool with an image on one named call, so a drained
|
||||
// sibling's image can be followed from execution to the transcript.
|
||||
type imageReadTool struct {
|
||||
readAheadTool
|
||||
imageFor string
|
||||
}
|
||||
|
||||
func (t *imageReadTool) Run(ctx context.Context, args map[string]any) tools.Result {
|
||||
result := t.readAheadTool.Run(ctx, args)
|
||||
if id, _ := args["id"].(string); id == t.imageFor {
|
||||
result.Images = []zeroruntime.ImageBlock{{MediaType: "image/png", Data: []byte("\x89PNG")}}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// The drained sibling's image is part of what ran, so it follows the tool
|
||||
// results like any other call's image. Images reach the transcript through the
|
||||
// turn's deferred builder, which the halt path calls after closing out the
|
||||
// siblings; a drain that skipped the sibling's image would lose it silently.
|
||||
func TestParallelSiblingImageSurvivesTheGuardHalt(t *testing.T) {
|
||||
tool := &imageReadTool{imageFor: "pB"}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
var turns [][]zeroruntime.StreamEvent
|
||||
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
|
||||
id := "s" + strconv.Itoa(i)
|
||||
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
}
|
||||
pair := append(readCall("pA"), readCall("pB")...)
|
||||
turns = append(turns, append(pair, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
|
||||
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAuto,
|
||||
MaxTurns: len(turns) + 5,
|
||||
SupportsVision: func(string) bool { return true },
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
|
||||
t.Fatalf("the guard did not halt this run: %q", result.FinalAnswer)
|
||||
}
|
||||
images := 0
|
||||
for _, message := range result.Messages {
|
||||
if message.Role == zeroruntime.MessageRoleUser {
|
||||
images += len(message.Images)
|
||||
}
|
||||
}
|
||||
if images != 1 {
|
||||
t.Errorf("the drained sibling's image reached the transcript %d times, want once", images)
|
||||
}
|
||||
}
|
||||
|
||||
// A call that genuinely never ran still gets an aborted placeholder, so the fix
|
||||
// did not simply stop aborting anything.
|
||||
func TestUnstartedCallsStillGetAbortedPlaceholders(t *testing.T) {
|
||||
tool := &readAheadTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
var turns [][]zeroruntime.StreamEvent
|
||||
for i := 0; i < toolFailureAnyErrorStopAt-1; i++ {
|
||||
id := "s" + strconv.Itoa(i)
|
||||
turns = append(turns, append(readCall(id), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
}
|
||||
// One eligible pair, then a THIRD call outside the batch window is not what
|
||||
// happens here: the whole run is eligible, so instead advertise a single call
|
||||
// and let the halt fire on it, leaving nothing precomputed beyond it.
|
||||
turns = append(turns, append(readCall("solo"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
turns = append(turns, append(readCall("unreached"), zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone}))
|
||||
|
||||
result, err := Run(context.Background(), "read things", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAuto,
|
||||
MaxTurns: len(turns) + 5,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(result.FinalAnswer, "Agent stopped:") {
|
||||
t.Fatalf("the guard did not halt: %q", result.FinalAnswer)
|
||||
}
|
||||
if tool.ran != toolFailureAnyErrorStopAt {
|
||||
t.Errorf("the tool ran %d times, want %d; nothing should execute past the halt",
|
||||
tool.ran, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
}
|
||||
@@ -2,9 +2,10 @@ package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// Parallel read-ahead for tool batches. When a turn requests several
|
||||
@@ -12,8 +13,17 @@ import (
|
||||
// them one after another serializes pure I/O waits. A consecutive run of
|
||||
// capability-safe read-only calls is executed concurrently instead; results are
|
||||
// then consumed in the original call order, so guard counters, message
|
||||
// ordering, abort semantics, and the surface's call/result event pairing are
|
||||
// byte-identical to sequential execution. Runs never span a mutating call: a
|
||||
// ordering, abort semantics, and the surface's call/result event pairing match
|
||||
// sequential execution.
|
||||
//
|
||||
// What can differ is how many calls RAN. A run executes before the guard
|
||||
// consumes its first result, so a halt on an earlier call can leave up to
|
||||
// maxParallelReadTools-1 siblings already executed. The guard bounds count
|
||||
// consumed results, not executions, and the extra executions are reads by the
|
||||
// eligibility rule below; closeOutRemaining records them without feeding the
|
||||
// guard. Reported by @jatmn.
|
||||
//
|
||||
// Runs never span a mutating call: a
|
||||
// read that follows a write must observe the write, so eligibility is decided
|
||||
// per consecutive run, not per batch.
|
||||
//
|
||||
@@ -36,6 +46,15 @@ const maxParallelReadTools = 8
|
||||
type precomputedToolResult struct {
|
||||
result ToolResult
|
||||
abortErr error
|
||||
// completed records whether this call PRODUCED a result, which is a
|
||||
// different question from whether it asks the enclosing run to stop.
|
||||
//
|
||||
// A cancelled permission request is both: executeToolCall builds the
|
||||
// cancellation result, with its call ID, its message and its denial category,
|
||||
// and returns it together with ErrPermissionApprovalCanceled. Reading the
|
||||
// error as "never started" throws that result away, and the transcript then
|
||||
// denies that permission handling happened at all.
|
||||
completed bool
|
||||
}
|
||||
|
||||
// parallelSafeToolCall reports whether call may run concurrently with its
|
||||
@@ -189,7 +208,15 @@ func executeParallelReadBatch(ctx context.Context, registry *tools.Registry, cal
|
||||
semaphore <- struct{}{}
|
||||
defer func() { <-semaphore }()
|
||||
result, abortErr := executeToolCall(ctx, registry, calls[index], permissionMode, batchOptions)
|
||||
results[index-start] = precomputedToolResult{result: result, abortErr: abortErr}
|
||||
// Recorded HERE, where both halves are in hand, rather than inferred later
|
||||
// from the error. A populated call ID is what every path that produced a
|
||||
// result sets and what a context cancellation before execution leaves
|
||||
// empty.
|
||||
results[index-start] = precomputedToolResult{
|
||||
result: result,
|
||||
abortErr: abortErr,
|
||||
completed: strings.TrimSpace(result.ToolCallID) != "",
|
||||
}
|
||||
}(index)
|
||||
}
|
||||
waitGroup.Wait()
|
||||
|
||||
@@ -7,8 +7,8 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// probeTool records execution overlap and ordering so tests can assert what
|
||||
|
||||
@@ -3,7 +3,7 @@ package agent
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
func TestPartitionToolsActiveAppendsLoadedToolAfterEagerBlock(t *testing.T) {
|
||||
|
||||
@@ -5,8 +5,8 @@ import (
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// countingSchemaTool records how many times its Parameters() schema is read, so a
|
||||
|
||||
@@ -4,8 +4,8 @@ import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// promptShellTool is a minimal shell-style tool whose Safety requires a prompt,
|
||||
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
func TestPermissionScope(t *testing.T) {
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func planUpdateTurn(status string) zeroruntime.CollectedStream {
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// forgedMarkerTool executes, fails, and claims in its metadata that the registry
|
||||
// refused it before it ran.
|
||||
type forgedMarkerTool struct {
|
||||
ran int
|
||||
key string
|
||||
value string
|
||||
}
|
||||
|
||||
func (tool *forgedMarkerTool) Name() string { return "bash" }
|
||||
func (tool *forgedMarkerTool) Description() string { return "test shell tool" }
|
||||
func (tool *forgedMarkerTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *forgedMarkerTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
|
||||
}
|
||||
func (tool *forgedMarkerTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
Output: "Error: the script exited 1",
|
||||
Meta: map[string]string{tool.key: tool.value},
|
||||
}
|
||||
}
|
||||
|
||||
// THE FORGED MARKER MUST NOT CHANGE THE RUN.
|
||||
//
|
||||
// Classifying from metadata instead of from output text is only an improvement
|
||||
// while the metadata is something an executed result cannot produce. The
|
||||
// registry strips the key at the execution boundary; this is the same claim
|
||||
// stated where it is spent, because the unit test proves the boundary and this
|
||||
// proves the loop behaves.
|
||||
//
|
||||
// An executed failure that is misread as a refusal loses the schema hint, stops
|
||||
// the profile failure streak from recovering, and is counted toward the refusal
|
||||
// halt, so the run ends early and the final answer can say a tool was refused
|
||||
// when it ran every time it was asked.
|
||||
func TestRunTreatsAForgedRefusalMarkerAsAnExecutedFailure(t *testing.T) {
|
||||
for _, forged := range []struct{ key, value string }{
|
||||
{tools.PolicyRefusalMeta, tools.PolicyRefusalSandboxDenied},
|
||||
{tools.PolicyRefusalMeta, tools.PolicyRefusalPermissionDenied},
|
||||
{tools.PolicyRefusalMeta, tools.PolicyRefusalToolNotEnabled},
|
||||
{tools.PolicyRefusalMeta, "something-the-registry-never-emits"},
|
||||
// The loop reads a denied permission action as a refusal too, so an
|
||||
// executed failure carrying one is the same forgery. Reported by @jatmn.
|
||||
{tools.PermissionActionMeta, string(PermissionActionDeny)},
|
||||
} {
|
||||
value := forged.value
|
||||
t.Run(forged.key+"="+value, func(t *testing.T) {
|
||||
tool := &forgedMarkerTool{key: forged.key, value: value}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// One more call than the hint threshold, so the hint has to have been
|
||||
// injected by the last one, then a final text turn to end the run.
|
||||
calls := toolFailureHintAt + 1
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash", `{"command":"./flaky.sh"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up on the script"))
|
||||
|
||||
var denials []DenialCategory
|
||||
result, err := Run(context.Background(), "run the script", &mockProvider{turns: turns}, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnToolResult: func(toolResult ToolResult) {
|
||||
denials = append(denials, toolResult.DenialReason)
|
||||
},
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
t.Error("permission was requested for an allow-safety tool; this test must exercise the executed-failure path")
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != calls {
|
||||
t.Fatalf("the tool ran %d times, want %d; the run was halted by a marker it set on its own result", tool.ran, calls)
|
||||
}
|
||||
for index, denial := range denials {
|
||||
if denial != DenialNone {
|
||||
t.Errorf("result %d was recorded as denial %q; the tool executed", index, denial)
|
||||
}
|
||||
}
|
||||
var hinted bool
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, toolFailureHintMarker) {
|
||||
hinted = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !hinted {
|
||||
t.Errorf("no retry hint was injected for an executed failure carrying %q, so the model got no correction", value)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// THE UNCATEGORIZED REFUSALS ARE THE ONES THIS CHANGE EXISTS FOR, SO THEY ARE THE
|
||||
// ONES THAT MUST BE DRIVEN THROUGH Run.
|
||||
//
|
||||
// A categorized denial is easy to test and was never the gap. The gap is a
|
||||
// refusal that arrives with DenialReason empty, because a category is attached
|
||||
// only where a TYPED denial is built:
|
||||
//
|
||||
// - a headless run leaves OnPermissionRequest nil, so the loop never reaches
|
||||
// its typed-denial branch and the registry gate returns a bare
|
||||
// `Permission required for ...`
|
||||
// - a sandbox preflight denial on a non-shell tool loses its SandboxDecision
|
||||
// converting to ToolResult and arrives as a bare `Sandbox block`
|
||||
//
|
||||
// Both were counted as SUCCESS, which cleared the very record they were meant to
|
||||
// accumulate, and the refused call repeated to MaxTurns.
|
||||
//
|
||||
// Asserting that through the helper alone would prove nothing: the first version
|
||||
// of this fix passed every helper test while being a no-op in production,
|
||||
// because the loop asked a different question than the tests did. So each case
|
||||
// below goes through Run and pins the three things the loop actually does with
|
||||
// the classification: halt at the bound, never execute the tool, and withhold
|
||||
// the profile's one-shot failure escalation.
|
||||
|
||||
// headlessPromptTool needs approval and never gets it, because the run has no
|
||||
// approver. The registry refuses it before Run() is reached, exactly as a
|
||||
// headless run does, so nothing here fakes the refusal.
|
||||
type headlessPromptTool struct{ ran int }
|
||||
|
||||
func (tool *headlessPromptTool) Name() string { return "bash" }
|
||||
func (tool *headlessPromptTool) Description() string { return "test shell tool" }
|
||||
func (tool *headlessPromptTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *headlessPromptTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectShell, Permission: tools.PermissionPrompt, Reason: "runs shell commands"}
|
||||
}
|
||||
func (tool *headlessPromptTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusOK, Output: "should never run"}
|
||||
}
|
||||
|
||||
// A headless prompt refusal must halt at the same-signature bound rather than
|
||||
// repeat to MaxTurns.
|
||||
//
|
||||
// It used to key on the refusal TEXT, which the registry happens to hold
|
||||
// constant here, and the stop then described a repeated failure. The registry
|
||||
// marks this path, so the category is now derived at the ToolResult boundary and
|
||||
// the streak keys on that instead. The bound is unchanged; the wording is more
|
||||
// honest, because a headless prompt refusal is a refusal and not a tool that
|
||||
// keeps failing.
|
||||
func TestRunStopsAnUncategorizedHeadlessRefusalAtTheFailureBound(t *testing.T) {
|
||||
tool := &headlessPromptTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// Comfortably more turns than the bound, so reaching the bound is what stops
|
||||
// the run rather than exhausting the provider or MaxTurns.
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureStopAt+4)
|
||||
for i := range toolFailureStopAt + 4 {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
|
||||
`{"command":"touch /etc/file`+strconv.Itoa(i)+`"}`))
|
||||
}
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
recorder := trace.NewRecorder("policy-refusal-session", "run-1", "fast")
|
||||
result, err := Run(context.Background(), "do the thing", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
Trace: recorder,
|
||||
// Armed well below the halt bound, so a refusal counted as a retriable
|
||||
// failure would visibly spend it.
|
||||
Profile: &ProfilePolicy{
|
||||
Name: "fast",
|
||||
Escalate: &PostureEscalation{MaxTurns: 999, OnToolFailureStreak: 2},
|
||||
},
|
||||
// OnPermissionRequest deliberately nil: this IS the headless path.
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if len(provider.requests) != toolFailureStopAt {
|
||||
t.Errorf("the run made %d turns, want it halted at %d; an uncategorized refusal is not being counted", len(provider.requests), toolFailureStopAt)
|
||||
}
|
||||
if tool.ran != 0 {
|
||||
t.Errorf("the refused tool executed %d times; the registry gate must precede execution", tool.ran)
|
||||
}
|
||||
// Categorized now, so the stop answer says refused. Previously the loop could
|
||||
// not tell and reported a repeated failure.
|
||||
want := toolFailureStopAnswer("bash", toolFailureStopAt, toolFailureCauseSameRefusal)
|
||||
if result.FinalAnswer != want {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
|
||||
}
|
||||
assertNoPostureEscalation(t, recorder)
|
||||
}
|
||||
|
||||
// uncategorizedSandboxTool returns the shape a sandbox preflight denial has by
|
||||
// the time the loop sees it on a non-shell tool: StatusError, `Sandbox block`
|
||||
// prose, and no category, because the SandboxDecision does not survive the
|
||||
// conversion to ToolResult. The path varies per call exactly as the real message
|
||||
// does, since it names what was refused.
|
||||
type uncategorizedSandboxTool struct{ ran int }
|
||||
|
||||
func (tool *uncategorizedSandboxTool) Name() string { return "write_file" }
|
||||
func (tool *uncategorizedSandboxTool) Description() string { return "test write tool" }
|
||||
func (tool *uncategorizedSandboxTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"path": {Type: "string"}},
|
||||
Required: []string{"path"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *uncategorizedSandboxTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectWrite, Permission: tools.PermissionAllow, Reason: "writes files"}
|
||||
}
|
||||
func (tool *uncategorizedSandboxTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{
|
||||
Status: tools.StatusError,
|
||||
Output: "Sandbox block: write to /out-" + strconv.Itoa(tool.ran) + ".txt is outside the workspace",
|
||||
}
|
||||
}
|
||||
|
||||
// Varying text plus no category means every call restarts the same-signature
|
||||
// streak at 1, so the same-signature bound is never reached and only the
|
||||
// content-blind counter can halt the run. Before the fix nothing halted it at
|
||||
// all: each refusal was read as a success, deleted the record, and the call
|
||||
// repeated until MaxTurns.
|
||||
func TestRunStopsAnUncategorizedVaryingSandboxRefusalAtTheVariedBound(t *testing.T) {
|
||||
tool := &uncategorizedSandboxTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, toolFailureAnyErrorStopAt+4)
|
||||
for i := range toolFailureAnyErrorStopAt + 4 {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "write_file",
|
||||
`{"path":"/out-`+strconv.Itoa(i)+`.txt"}`))
|
||||
}
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
recorder := trace.NewRecorder("policy-refusal-session", "run-2", "fast")
|
||||
result, err := Run(context.Background(), "write the files", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
Trace: recorder,
|
||||
Profile: &ProfilePolicy{
|
||||
Name: "fast",
|
||||
Escalate: &PostureEscalation{MaxTurns: 999, OnToolFailureStreak: 2},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if tool.ran != toolFailureAnyErrorStopAt {
|
||||
t.Errorf("the tool was called %d times, want the run halted at %d", tool.ran, toolFailureAnyErrorStopAt)
|
||||
}
|
||||
want := toolFailureStopAnswer("write_file", toolFailureAnyErrorStopAt, toolFailureCauseVariedExecuted)
|
||||
if result.FinalAnswer != want {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q", result.FinalAnswer, want)
|
||||
}
|
||||
assertNoPostureEscalation(t, recorder)
|
||||
}
|
||||
|
||||
// The profile's failure-streak trigger restores turn budget and reasoning effort
|
||||
// on the theory that a struggling tool needs room. A refusal is not a struggling
|
||||
// tool, it is an answer, and spending the one-shot escalation on one buys turns
|
||||
// that will be refused identically. The loop keeps this contract by passing an
|
||||
// empty outcome to the controller for a refusal, which is only observable from
|
||||
// outside as the escalation never firing.
|
||||
func assertNoPostureEscalation(t *testing.T, recorder *trace.Recorder) {
|
||||
t.Helper()
|
||||
if got := recorder.Finish().Counter(trace.CounterPostureEscalations); got != 0 {
|
||||
t.Errorf("posture_escalations = %d, want 0; a policy refusal spent the one-shot failure escalation", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,265 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// A REFUSAL IS A FAILURE, AND THE CLASSIFIER MUST SAY SO BEFORE IT READS ANYTHING
|
||||
// ELSE.
|
||||
//
|
||||
// isPolicyRefusal decides on denial category, then permission metadata, then
|
||||
// output text. None of those questions is meaningful about a call the tool
|
||||
// completed, and the last one is answered by content the model does not control:
|
||||
// a `bash` that greps the sandbox docs, a read_file returning this very source
|
||||
// file, any successful command whose output happens to carry one of the phrases.
|
||||
//
|
||||
// isRetriableToolError gated on StatusError before calling in, so the boundary
|
||||
// held while that was the only caller. Extracting the helper and calling it from
|
||||
// the counting path dropped the gate, and every successful result was put to a
|
||||
// question that only failures can answer.
|
||||
//
|
||||
// Each case below is a SUCCESS. None may be read as a refusal.
|
||||
func TestPolicyRefusalNeverClassifiesASuccessfulResult(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
result ToolResult
|
||||
// structured marks a result carrying real provenance: a denial category,
|
||||
// permission metadata, or the registry's pre-execution marker. Only those
|
||||
// may be read as a refusal when they fail. The rest carry nothing but
|
||||
// text, which the tool controls, so they must stay ordinary failures no
|
||||
// matter which phrase they happen to contain.
|
||||
structured bool
|
||||
}{
|
||||
{
|
||||
name: "output quotes the sandbox block phrase",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "grep: docs/sandbox.md: Sandbox block"},
|
||||
},
|
||||
{
|
||||
name: "output quotes the disabled-tool phrase",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "web_fetch is not enabled for this run"},
|
||||
},
|
||||
{
|
||||
name: "output quotes a permission denial",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "log line: Permission denied for bash"},
|
||||
},
|
||||
{
|
||||
name: "output quotes a permission requirement",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "log line: Permission required for bash"},
|
||||
},
|
||||
{
|
||||
name: "output quotes a sandbox approval prompt",
|
||||
result: ToolResult{Status: tools.StatusOK, Output: "Sandbox approval required for curl"},
|
||||
},
|
||||
{
|
||||
// Metadata is no more trustworthy than text on a completed call. A
|
||||
// result carrying a category with an OK status is not a denial; every
|
||||
// path that attaches one fails the call.
|
||||
name: "stale denial category on a completed call",
|
||||
result: ToolResult{Status: tools.StatusOK, DenialReason: DenialSandboxBlock, Output: "ok"},
|
||||
structured: true,
|
||||
},
|
||||
{
|
||||
name: "stale permission metadata on a completed call",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusOK,
|
||||
Meta: map[string]string{"permission_action": string(PermissionActionDeny)},
|
||||
Output: "ok",
|
||||
},
|
||||
structured: true,
|
||||
},
|
||||
{
|
||||
name: "registry refusal marker on a completed call",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusOK,
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied},
|
||||
Output: "ok",
|
||||
},
|
||||
structured: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, testCase := range cases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
if isPolicyRefusal(testCase.result) {
|
||||
t.Errorf("a successful result was classified as a policy refusal, so a healthy call counts toward the denial streak: %+v", testCase.result)
|
||||
}
|
||||
failed := testCase.result
|
||||
failed.Status = tools.StatusError
|
||||
switch {
|
||||
case testCase.structured && !isPolicyRefusal(failed):
|
||||
t.Errorf("a failure carrying real provenance was not classified as a refusal, so the gate broke the classifier: %+v", failed)
|
||||
case !testCase.structured && isPolicyRefusal(failed):
|
||||
t.Errorf("a failure carrying only text was classified as a refusal; output is tool-controlled and cannot decide this: %+v", failed)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// blockTextShellTool runs freely and prints text the classifier keys on. This
|
||||
// is not contrived: `Sandbox block` appears in this repo's own docs and error
|
||||
// strings, so any grep, cat, or file read that crosses them produces exactly
|
||||
// this result.
|
||||
type blockTextShellTool struct{ ran int }
|
||||
|
||||
func (tool *blockTextShellTool) Name() string { return "bash" }
|
||||
func (tool *blockTextShellTool) Description() string { return "test shell tool" }
|
||||
func (tool *blockTextShellTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *blockTextShellTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
|
||||
}
|
||||
func (tool *blockTextShellTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{
|
||||
Status: tools.StatusOK,
|
||||
Output: "docs/windows-sandbox.md:41: Sandbox block (write jail) is reported as ...",
|
||||
}
|
||||
}
|
||||
|
||||
// The same gap driven through Run, because the classifier being wrong only
|
||||
// matters if the loop acts on it, and it does: the counting path calls
|
||||
// isPolicyRefusal directly, and a true answer records a failure against the
|
||||
// tool's signature. Six identical successes then look like six identical
|
||||
// denials, and the run ends on a stop answer telling the user a tool it never
|
||||
// refused was refused six times.
|
||||
//
|
||||
// The tool succeeds every turn and the run is given more turns than the bound,
|
||||
// so anything that halts before the provider's final turn is the guard.
|
||||
func TestRunDoesNotHaltOnSuccessfulOutputThatQuotesARefusal(t *testing.T) {
|
||||
tool := &blockTextShellTool{}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
calls := toolFailureStopAt + 4
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash",
|
||||
`{"command":"grep -n 'Sandbox block' docs/windows-sandbox.md"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("found the docs"))
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
result, err := Run(context.Background(), "search the docs", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
t.Error("the loop asked permission for an allow-safety tool; this test must exercise the success path")
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if tool.ran != calls {
|
||||
t.Errorf("the tool ran %d times, want %d; the run was halted mid-way by its own success output", tool.ran, calls)
|
||||
}
|
||||
if result.FinalAnswer != "found the docs" {
|
||||
t.Errorf("final answer =\n %q\nwant\n %q\n(a stop answer here means successful output was counted as a refusal)",
|
||||
result.FinalAnswer, "found the docs")
|
||||
}
|
||||
}
|
||||
|
||||
// failingPhraseTool RAN and failed. Its stderr happens to contain a phrase the
|
||||
// classifier used to key on, which is the everyday case: `bash` preserves
|
||||
// arbitrary stdout and stderr on any nonzero exit, so
|
||||
// `printf 'Sandbox block\n' >&2; exit 1` produces exactly this.
|
||||
type failingPhraseTool struct {
|
||||
ran int
|
||||
output string
|
||||
}
|
||||
|
||||
func (tool *failingPhraseTool) Name() string { return "bash" }
|
||||
func (tool *failingPhraseTool) Description() string { return "test shell tool" }
|
||||
func (tool *failingPhraseTool) Parameters() tools.Schema {
|
||||
return tools.Schema{
|
||||
Type: "object",
|
||||
Properties: map[string]tools.PropertySchema{"command": {Type: "string"}},
|
||||
Required: []string{"command"},
|
||||
AdditionalProperties: false,
|
||||
}
|
||||
}
|
||||
func (tool *failingPhraseTool) Safety() tools.Safety {
|
||||
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "reads files"}
|
||||
}
|
||||
func (tool *failingPhraseTool) Run(context.Context, map[string]any) tools.Result {
|
||||
tool.ran++
|
||||
return tools.Result{Status: tools.StatusError, Output: tool.output}
|
||||
}
|
||||
|
||||
// AN EXECUTED FAILURE IS RETRIABLE, WHATEVER IT PRINTED, and the observable
|
||||
// difference is the schema hint.
|
||||
//
|
||||
// isRetriableToolError feeds `hintable` into the guard, and the guard injects
|
||||
// the retry hint at toolFailureHintAt. Classifying an executed failure as a
|
||||
// policy refusal makes hintable false, so the model is told nothing and simply
|
||||
// repeats the call: the run loses the one correction it had. That is on top of
|
||||
// the failure being accumulated as a denial.
|
||||
//
|
||||
// Each phrase below is one the classifier used to match on. The tool is
|
||||
// allow-safety and runs every time, so nothing here is refused by anything.
|
||||
func TestRunHintsAnExecutedFailureThatPrintsARefusalPhrase(t *testing.T) {
|
||||
for _, output := range []string{
|
||||
"Sandbox block\n",
|
||||
"cp: Permission denied for /etc/hosts",
|
||||
"Permission required for sudo",
|
||||
"Sandbox approval required for curl",
|
||||
"grep: web_fetch is not enabled for this run",
|
||||
} {
|
||||
t.Run(output, func(t *testing.T) {
|
||||
tool := &failingPhraseTool{output: output}
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(tool)
|
||||
|
||||
// One more call than the hint threshold, so the hint has to have been
|
||||
// injected by the last one, then a final text turn to end the run.
|
||||
calls := toolFailureHintAt + 1
|
||||
turns := make([][]zeroruntime.StreamEvent, 0, calls+1)
|
||||
for i := range calls {
|
||||
turns = append(turns, toolTurn("call-"+strconv.Itoa(i), "bash", `{"command":"./flaky.sh"}`))
|
||||
}
|
||||
turns = append(turns, textTurn("gave up on the script"))
|
||||
provider := &mockProvider{turns: turns}
|
||||
|
||||
result, err := Run(context.Background(), "run the script", provider, Options{
|
||||
Registry: registry,
|
||||
PermissionMode: PermissionModeAsk,
|
||||
MaxTurns: len(turns) + 5,
|
||||
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
|
||||
t.Error("permission was requested for an allow-safety tool; this test must exercise the executed-failure path")
|
||||
return PermissionDecision{Action: PermissionDecisionDeny}, nil
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tool.ran != calls {
|
||||
t.Fatalf("the tool ran %d times, want %d; the run was halted by output it merely printed", tool.ran, calls)
|
||||
}
|
||||
|
||||
var hinted bool
|
||||
for _, message := range result.Messages {
|
||||
if strings.Contains(message.Content, toolFailureHintMarker) {
|
||||
hinted = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !hinted {
|
||||
t.Errorf("no retry hint was injected for an executed failure printing %q; it was classified as a policy refusal, so the model got no correction", output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// UNCATEGORIZED REFUSALS MUST STILL COUNT.
|
||||
//
|
||||
// The halt guard asked DenialReason != DenialNone, but a category is only
|
||||
// attached where a TYPED denial is built. A headless run leaves
|
||||
// OnPermissionRequest nil, so the loop never reaches that branch and the
|
||||
// registry returns a bare "Error: Permission required ..." with no category. A
|
||||
// sandbox preflight denial on a non-shell tool loses its SandboxDecision in the
|
||||
// conversion and arrives as an uncategorized "Sandbox block".
|
||||
//
|
||||
// Both were counted as SUCCESS, which cleared the very record the guard
|
||||
// accumulates, so the same refused call could repeat to MaxTurns. That is the
|
||||
// loop this PR exists to stop.
|
||||
func TestUncategorizedPolicyRefusalsAreCounted(t *testing.T) {
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
result ToolResult
|
||||
}{
|
||||
{
|
||||
// Still uncategorized in the DenialReason sense: a headless run never
|
||||
// reaches the branch that builds a typed denial. It carries the
|
||||
// registry's pre-execution marker instead, which is what makes it
|
||||
// classifiable without reading the message.
|
||||
name: "headless prompt refusal carries no denial category",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusError,
|
||||
Output: `Error: Permission required for bash: The tool is marked "prompt" and was not executed.`,
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalPermissionRequired},
|
||||
},
|
||||
},
|
||||
{
|
||||
// The SandboxDecision is still dropped in the conversion for a
|
||||
// non-shell tool; the marker is what survives it.
|
||||
name: "sandbox preflight denial loses its decision in conversion",
|
||||
result: ToolResult{
|
||||
Status: tools.StatusError,
|
||||
Output: "Sandbox block: write outside the workspace",
|
||||
Meta: map[string]string{tools.PolicyRefusalMeta: tools.PolicyRefusalSandboxDenied},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "an explicitly denied permission action",
|
||||
result: ToolResult{Status: tools.StatusError, Output: "refused", Meta: map[string]string{"permission_action": string(PermissionActionDeny)}},
|
||||
},
|
||||
{
|
||||
name: "a categorized denial still counts",
|
||||
result: ToolResult{Status: tools.StatusError, Output: "denied", DenialReason: DenialFiltered},
|
||||
},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
if !isPolicyRefusal(testCase.result) {
|
||||
t.Fatal("refusal was treated as a success, so the guard's record is cleared and the call can repeat to MaxTurns")
|
||||
}
|
||||
if isRetriableToolError(testCase.result) {
|
||||
t.Error("a policy refusal is not retriable: repeating it verbatim cannot help")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// An ordinary failure is still retriable and still not a refusal, or the guard
|
||||
// would halt on things the model can legitimately fix by trying again.
|
||||
func TestOrdinaryFailuresAreNotPolicyRefusals(t *testing.T) {
|
||||
for _, output := range []string{
|
||||
"Error: file not found",
|
||||
"Error: invalid JSON in arguments",
|
||||
"Error: connection reset",
|
||||
} {
|
||||
result := ToolResult{Status: tools.StatusError, Output: output}
|
||||
if isPolicyRefusal(result) {
|
||||
t.Errorf("%q was classified as a policy refusal", output)
|
||||
}
|
||||
if !isRetriableToolError(result) {
|
||||
t.Errorf("%q should stay retriable", output)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// PRODUCING A RESULT AND ASKING THE RUN TO STOP ARE DIFFERENT FACTS.
|
||||
//
|
||||
// closeOutRemaining decides between finalizing a completed sibling and writing
|
||||
// an aborted placeholder, and it used to make that decision from abortErr. A
|
||||
// cancelled permission request is exactly where the two facts disagree:
|
||||
// executeToolCall builds the cancellation result, with its call ID, its message
|
||||
// and its denial category, and returns it TOGETHER with
|
||||
// ErrPermissionApprovalCanceled. Reading the error as "never started" discards a
|
||||
// real result and records the opposite of what happened, and the cancellation
|
||||
// then goes missing from OnToolResult, the trace counters and the task
|
||||
// observation even though the permission event already fired.
|
||||
func TestABatchEntryWithAResultAndAnAbortErrorIsDrained(t *testing.T) {
|
||||
canceled := canceledPermissionResult(
|
||||
ToolCall{ID: "call-b", Name: "read_probe"},
|
||||
"cancelled in TUI",
|
||||
PermissionEvent{ToolName: "read_probe"},
|
||||
)
|
||||
precomputed := []precomputedToolResult{
|
||||
{result: ToolResult{ToolCallID: "call-a", Name: "read_probe", Status: tools.StatusOK}, completed: true},
|
||||
{
|
||||
result: canceled,
|
||||
abortErr: fmt.Errorf("%w for read_probe", ErrPermissionApprovalCanceled),
|
||||
completed: true,
|
||||
},
|
||||
}
|
||||
|
||||
sibling, ran := precomputedResultFor(precomputed, 0, 2, 1)
|
||||
if !ran {
|
||||
t.Fatal("a cancelled permission result was treated as a call that never started")
|
||||
}
|
||||
if sibling.ToolCallID != "call-b" {
|
||||
t.Errorf("ToolCallID = %q, want the cancellation result rather than an empty placeholder", sibling.ToolCallID)
|
||||
}
|
||||
if sibling.DenialReason != DenialApprovalCanceled {
|
||||
t.Errorf("DenialReason = %q, want the cancellation preserved", sibling.DenialReason)
|
||||
}
|
||||
}
|
||||
|
||||
// An entry that never produced a result is still unstarted, or the fix would
|
||||
// have stopped aborting anything.
|
||||
func TestABatchEntryWithNoResultIsStillUnstarted(t *testing.T) {
|
||||
precomputed := []precomputedToolResult{
|
||||
{abortErr: errors.New("cancelled before the tool ran")},
|
||||
}
|
||||
if _, ran := precomputedResultFor(precomputed, 0, 1, 0); ran {
|
||||
t.Error("an entry that produced nothing was finalized as if it had run")
|
||||
}
|
||||
}
|
||||
|
||||
// A call outside the batch window never started either.
|
||||
func TestACallOutsideTheBatchWindowIsUnstarted(t *testing.T) {
|
||||
precomputed := []precomputedToolResult{{result: ToolResult{ToolCallID: "call-a"}, completed: true}}
|
||||
if _, ran := precomputedResultFor(precomputed, 0, 1, 1); ran {
|
||||
t.Error("a call past the batch window was reported as executed")
|
||||
}
|
||||
}
|
||||
|
||||
// And the flag is recorded where the truth is known rather than inferred later,
|
||||
// so a real batch marks what it ran.
|
||||
func TestTheBatchRecordsWhatItRan(t *testing.T) {
|
||||
registry := tools.NewRegistry()
|
||||
registry.Register(&readAheadTool{})
|
||||
calls := []ToolCall{
|
||||
{ID: "call-a", Name: "read_probe", Arguments: `{"id":"a"}`},
|
||||
{ID: "call-b", Name: "read_probe", Arguments: `{"id":"b"}`},
|
||||
}
|
||||
results := executeParallelReadBatch(context.Background(), registry, calls, 0, 2, PermissionModeAuto, Options{Registry: registry})
|
||||
for index, entry := range results {
|
||||
if !entry.completed {
|
||||
t.Errorf("entry %d executed but was not recorded as completed: %#v", index, entry)
|
||||
}
|
||||
if entry.result.ToolCallID != calls[index].ID {
|
||||
t.Errorf("entry %d is keyed to %q, want %q", index, entry.result.ToolCallID, calls[index].ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,8 +4,8 @@ import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
)
|
||||
|
||||
// escalatingEngine allows unsandboxed escalation: no denied reads, so
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
)
|
||||
|
||||
// profileController observes per-turn run signals and decides at most one
|
||||
|
||||
@@ -5,10 +5,10 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/sandbox"
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/sandbox"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
func TestProfileControllerNilPolicyIsNoOp(t *testing.T) {
|
||||
|
||||
@@ -4,8 +4,8 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/tools"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/tools"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// These ceilings are a deliberate ratchet on Zero's fixed per-turn overhead — the
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// promptSubstrings are the cacheable components of the prompt a run sends to a
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// TestComputePrefixFingerprintStableAcrossCalls asserts the headline property
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// Stale tool-output pruning reclaims context at ZERO token/latency cost before
|
||||
|
||||
@@ -4,7 +4,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// bigToolResult builds a tool-result message with approximately tokens worth of
|
||||
|
||||
@@ -7,9 +7,9 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/Gitlawb/zero/internal/errhint"
|
||||
"github.com/Gitlawb/zero/internal/trace"
|
||||
"github.com/Gitlawb/zero/internal/zeroruntime"
|
||||
"github.com/Twigpine/zero/internal/errhint"
|
||||
"github.com/Twigpine/zero/internal/trace"
|
||||
"github.com/Twigpine/zero/internal/zeroruntime"
|
||||
)
|
||||
|
||||
// Mid-stream reconnect: a long autonomous task (a big refactor, a swarm member,
|
||||
|
||||
Loaded 100 of 763 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user