Compare commits

..
Author SHA1 Message Date
KRATOS 7bce9ad9ff perf(agent): concurrent read-only tool batches via capability gate
Upgrade the parallel tool planner from SideEffectRead probes to the PR5
CapabilitiesOf contract: EffectReadOnly + ThreadSafe + auto-allowed, with
resource-key conflict boundaries so same-path reads stay sequential.

Mark audited pure reads ThreadSafe (read_file, read_minified_file,
list_directory, glob, grep, skill, web_fetch). Glob/grep use scopedScan
directory keys (nil for workspace-wide "." so scans do not false-conflict).

Tests cover the capability gate, key conflicts, extendParallelRun windows,
and catalog ThreadSafe audit for concurrent-safe reads.
2026-07-17 17:34:16 +05:30
141 changed files with 687 additions and 11784 deletions

No files matched your search

-6
View File
@@ -1,11 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: Questions and setup help
url: https://github.com/Gitlawb/zero/discussions/categories/q-a
about: Ask in Discussions Q&A. Include zero --version, your OS, provider/model, and zero doctor output for setup problems.
- name: Feature ideas
url: https://github.com/Gitlawb/zero/discussions/categories/ideas
about: Propose and discuss ideas in Discussions before opening a PR (Zero is in controlled development, see CONTRIBUTING.md).
- name: Security vulnerability
url: https://github.com/Gitlawb/zero/security/advisories/new
about: Report security issues privately via GitHub Security Advisories (see SECURITY.md). Do not file a public issue.
+3 -13
View File
@@ -1,22 +1,12 @@
name: Feature request (previously discussed)
description: File only after your idea gained traction in Discussions Ideas or a maintainer asked for a tracking issue
name: Feature request
description: Propose a new capability or change
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: |
New feature ideas start in [Discussions Ideas](https://github.com/Gitlawb/zero/discussions/categories/ideas),
not here. This form is for the step after that: an idea that gained traction in Discussions,
or one a maintainer asked to have tracked as an issue. Please link the discussion below.
Do not begin implementation until the issue has been reviewed and marked with the
Please do not begin implementation until the issue has been reviewed and marked with the
`issue-approved` label (see CONTRIBUTING.md).
- type: input
id: discussion
attributes:
label: Ideas discussion
description: Link to the Discussions Ideas thread (or note which maintainer asked for this issue).
validations:
required: true
- type: textarea
id: problem
attributes:
-3
View File
@@ -55,6 +55,3 @@ docs/superpowers/
zero-termux-arm64
zero-linux-sandbox
zero-seccomp
# Generated benchmark reports are evidence for one configuration, never tree state
internal/perfbench/reports/*.json
+19 -63
View File
@@ -1,71 +1,27 @@
# Repository Guidelines for Zero
---
description: Project guidelines for the open-source CLI coding agent (zero) repository.
globs: "*.go, *.js, *.md, *.json, *.toml, *.yaml, *.yml"
alwaysApply: false
---
These instructions apply to all work in this repository. For the user-facing
guide to extending Zero with specialists, hooks, plugins, MCP, and skills, see
[docs/EXTENDING.md](docs/EXTENDING.md).
# Repository Conventions for Zero
## 1. Contribution and Pull Request Rules
This file outlines the project conventions and repository guidelines for coding agents when working on the `zero` repository. For the general guide on how to extend Zero (write specialist sub-agents, hooks, plugins, MCP, skills), see [docs/EXTENDING.md](docs/EXTENDING.md).
- Before opening any pull request, **all contributors**—including maintainers,
community contributors, and coding agents—must read and follow
[CONTRIBUTING.md](CONTRIBUTING.md).
- Community pull requests require an existing parent issue with the
`issue-approved` label. Team members may open pull requests through the
internal development process described in `CONTRIBUTING.md`.
- Keep each change focused on the approved or assigned scope. Do not include
unrelated fixes, refactors, formatting churn, generated output, or existing
local changes in the same commit or pull request.
- Discuss new implementation languages, runtimes, major dependency changes,
and broad architectural rewrites with maintainers before implementation.
- Pull request descriptions must explain what changed and why, link the parent
issue when required, and list the tests or verification performed. Include
screenshots or a short recording for user-visible UI changes when practical.
## 1. Project Conventions
## 2. Repository and Implementation Conventions
- Build with `make`, not `go build` directly.
- Tests live next to the source file (`foo_test.go` next to `foo.go`).
- Run `make lint` before opening a PR.
- Never edit files under `third_party/` — those are vendored.
- Unify functions/methods where possible to prevent codebase inflation. Prefer a single cross-platform function with conditional checks over duplicating helpers per platform.
- Use the Go version declared in `go.mod`. Do not hardcode a different local
toolchain version in scripts or documentation.
- Use the repository build and release commands (`make` and
`go run ./cmd/zero-release ...`) instead of inventing parallel build flows.
- Keep tests beside their source files (`foo_test.go` next to `foo.go`). Add a
regression test for behavior changes and run affected concurrent code under
the race detector.
- Never edit files under `third_party/`; they are vendored.
- Prefer one cross-platform function with small conditional checks over
duplicated platform-specific helpers when the behavior can remain unified.
- Do not commit generated benchmark reports from
`internal/perfbench/reports/*.json`; reports are configuration-specific
evidence, not repository state.
- Preserve the user's working tree. Do not overwrite, delete, stage, or commit
unrelated tracked or untracked files.
## 2. Guidelines for Coding Agents
## 3. Required Validation
If you are an AI coding agent executing tasks in this repository, you **MUST** run all Go code quality and security checks before committing code or completing your task:
Run validation from the repository root before committing, opening a pull
request, or completing an implementation task:
1. **Format & Vet**: Run `go fmt ./...` and `go vet ./...`.
2. **Lint**: Run `go run github.com/golangci/golangci-lint/v2/cmd/[email protected] run --enable-only unused,ineffassign,staticcheck ./...`.
3. **Security**: Run `go run golang.org/x/vuln/cmd/[email protected] ./...`.
1. **Formatting check**: `make fmt-check`. If it fails, format with
`go fmt ./...` (or `make fmt`) and run the check again.
2. **Vet**: `go vet ./...` (or `make vet`).
3. **Tests**: `go test ./...`. Use `make test` for the full race-enabled suite,
or run focused tests with `-race`, when concurrency is affected.
4. **Build**: `go run ./cmd/zero-release build`.
5. **Smoke test**: `go run ./cmd/zero-release smoke`.
6. **Advisory lint**:
`go run github.com/golangci/golangci-lint/v2/cmd/[email protected] run --enable-only unused,ineffassign,staticcheck ./...`.
7. **Security**:
`go run golang.org/x/vuln/cmd/[email protected] ./...`.
8. **Diff hygiene**: `git diff HEAD --check` (covers staged and unstaged
tracked changes).
`make lint` currently runs the formatting check and `go vet`; it does **not**
run golangci-lint. The pinned golangci-lint job is advisory in CI while the
existing repository-wide backlog is cleaned up. Fix findings introduced by or
related to the current change. Unrelated pre-existing advisory findings do not
justify expanding a focused pull request; report them separately.
Formatting, vet, tests, build, smoke, diff hygiene, and govulncheck are hard
requirements. If a related check fails, fix it. If a required check cannot run
because of the environment or fails for an unrelated external reason, report
the exact failure and obtain maintainer direction rather than silently ignoring
it.
If any check fails or cannot be run, do not ignore it. Prompt the user for instructions or setup assistance before proceeding.
+5 -48
View File
@@ -8,13 +8,6 @@ stabilizes. Community participation is still welcome, especially through bug
reports, feature requests, reproduction details, logs, and discussion on existing
issues.
Questions and setup help belong in
[GitHub Discussions](https://github.com/Gitlawb/zero/discussions) (see
[SUPPORT.md](SUPPORT.md)), not in the issue tracker. Feature ideas start in the
[Ideas](https://github.com/Gitlawb/zero/discussions/categories/ideas) category;
an idea that gains traction there is the path to an approved issue and,
eventually, an accepted PR.
Please read this policy before opening a pull request. Pull requests that do not
follow it may be closed without review.
@@ -40,11 +33,7 @@ unless the issue has been explicitly approved for outside contribution.
There are three primary ways to contribute:
1. Open a bug report with clear reproduction steps.
2. Propose a feature in
[Discussions Ideas](https://github.com/Gitlawb/zero/discussions/categories/ideas),
describing the problem, use case, and expected behavior. A feature-request
issue is filed only after the idea gains traction there or a maintainer asks
for one.
2. Open a feature request describing the problem, use case, and expected behavior.
3. Add useful context to an existing issue, such as logs, screenshots, examples,
affected versions, or reproduction cases.
@@ -57,8 +46,7 @@ All community pull requests require an approved parent issue.
Before starting implementation work:
1. For a bug, open an issue using the bug-report template. For a feature, start
in Discussions Ideas; the feature-request issue comes after the discussion.
1. Open an issue using the appropriate issue template.
2. Describe the bug, feature request, or functional change clearly.
3. Wait for the core team to review the issue.
4. Do not open a pull request unless the issue has the `issue-approved` label.
@@ -67,28 +55,6 @@ An approved issue means only that the specific issue has been accepted for work.
It does not approve unrelated future work, future pull requests, or future issues
from the same author. Each issue is reviewed case by case.
## Contribution Volume and Batching
To keep triage and review sustainable while the project stabilizes, please keep
your contribution flow to a size the core team can realistically work through:
- One issue per bug. File each defect as its own issue with its own
reproduction. Please do not bundle several unrelated findings into a single
"audit" issue; separate issues are far easier to triage, approve, and close
individually.
- Keep only a few items open at a time. If you already have several issues or
pull requests awaiting review, let those move through the queue before opening
more. A large simultaneous burst from one contributor slows review for
everyone, even when each item is valid.
- AI-assisted contributions are welcome, held to the same bar. Automated audits
and generated patches are fine, but each issue and pull request must be
individually verified, scoped, and reproducible before you open it. Please do
not post bulk machine-generated findings unfiltered.
These apply to all contributors and are about review bandwidth, not the value of
any single report. Maintainers may ask you to consolidate, split, or hold
submissions to keep the queue manageable.
## Pull Request Requirements
Before opening a community pull request, make sure all of the following are true:
@@ -167,12 +133,7 @@ Clear bug reports are the most useful way to help the project during this phase.
## Submitting a Feature Request
Feature requests start as a thread in
[Discussions Ideas](https://github.com/Gitlawb/zero/discussions/categories/ideas),
not as an issue. The feature-request issue form exists for the step AFTER that:
when an idea has gained traction in Discussions or a maintainer has asked for an
issue to track it. Whether in the discussion or the eventual issue, please
include:
When filing a feature request, please include:
- The problem you are trying to solve.
- The behavior or functionality you are requesting.
@@ -195,9 +156,5 @@ contribution phase.
## Getting Help
General questions, setup help, and "is this a bug or my config" uncertainty all
go to
[Discussions Q&A](https://github.com/Gitlawb/zero/discussions/categories/q-a).
If you are unsure whether a proposed code change is appropriate, raise it in
Discussions Ideas or on the relevant issue first and ask. Please keep discussion
focused, specific, and respectful of maintainer time.
If you are unsure whether a change is appropriate, open an issue first and ask.
Please keep discussion focused, specific, and respectful of maintainer time.
+1 -35
View File
@@ -195,7 +195,7 @@ Useful controls:
| `Ctrl+P` / `Ctrl+N` | previous / next item in menus (arrows still work) |
| `Shift+Tab` | cycle permission mode |
| `Ctrl+B` | show/hide the sidebar |
| `Ctrl+C` | cancel, exit, or return from a `/btw` conversation |
| `Ctrl+C` | cancel or exit |
Common slash commands:
@@ -205,7 +205,6 @@ Common slash commands:
| `/spec`, `/plan` | draft and review a plan before building |
| `/image` | attach an image for vision-capable models |
| `/resume`, `/rewind` | continue or roll back local sessions |
| `/btw [question]` | ask in an isolated fork without adding the side conversation to the main session |
| `/loop` | repeat a prompt or custom `/command` on an interval (`/loop 5m /babysit-prs`) or self-paced |
| `/compact`, `/context` | manage context usage |
| `/permissions`, `/tools` | inspect available tools and policy |
@@ -359,12 +358,6 @@ go run ./cmd/zero-release smoke
go run ./cmd/zero-perf-bench
```
Experimental: `ZERO_OPENAI_TURN_SESSION=1` enables the optimized OpenAI turn
session (background connection prewarm + request-prefix telemetry) for headless
`zero exec` runs against official OpenAI profiles. Off by default; `0`/`false`
disable. A/B-benchmark it by running the same `zero-perf-bench` suite with the
variable unset and set.
### Code Quality and Security Checks
Before committing any changes, ensure all Go code quality and security checks pass. Pinned `go run` commands matching CI constraints can be used directly without prior installation:
@@ -384,15 +377,6 @@ go install github.com/golangci/golangci-lint/v2/cmd/[email protected]
go install golang.org/x/vuln/cmd/[email protected]
```
The installed binaries land in `$GOBIN` when it is set, otherwise in
`$GOPATH/bin` (default `~/go/bin`). That directory must be on your `PATH` to
run them directly. If it isn't, add it:
```bash
gobin="$(go env GOBIN)"; [ -z "$gobin" ] && gobin="$(go env GOPATH)/bin"
case ":$PATH:" in *":$gobin:"*) ;; *) export PATH="$PATH:$gobin" ;; esac
```
### Cross-Compile Examples
```bash
@@ -412,24 +396,6 @@ go run ./cmd/zero-release build --goos windows --goarch amd64 --output dist/zero
- [Performance](docs/PERFORMANCE.md)
- [Agent evals](docs/AGENT_EVALS.md)
## Community
Questions, setup help, ideas, and sharing all live in
[GitHub Discussions](https://github.com/Gitlawb/zero/discussions):
| Category | Use it for |
|---|---|
| [Q&A](https://github.com/Gitlawb/zero/discussions/categories/q-a) | Setup help, provider/model configuration, "how do I" questions |
| [Ideas](https://github.com/Gitlawb/zero/discussions/categories/ideas) | Feature proposals and design discussion before any PR |
| [Show and tell](https://github.com/Gitlawb/zero/discussions/categories/show-and-tell) | Your skills, plugins, MCP setups, themes, and workflows |
| [Announcements](https://github.com/Gitlawb/zero/discussions/categories/announcements) | Releases and project news from the maintainers |
For a good Q&A answer fast, include `zero --version`, your OS and install
method, the provider/model in use, and `zero doctor` output. See
[SUPPORT.md](SUPPORT.md). Bugs belong in
[issues](https://github.com/Gitlawb/zero/issues/new/choose); security reports
follow [SECURITY.md](SECURITY.md), never a public thread.
## Contributing
Contributions are welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md), run the
-36
View File
@@ -1,36 +0,0 @@
# Getting help with Zero
## Questions, setup help, configuration
Ask in [GitHub Discussions](https://github.com/Gitlawb/zero/discussions). Use
[Q&A](https://github.com/Gitlawb/zero/discussions/categories/q-a) so answers can
be marked and found by the next person.
The fastest way to a useful answer is to include:
- `zero --version` output
- your OS (Windows, macOS, Linux) and how you installed Zero (npm, install
script, source)
- the provider and model you are using (`zero providers list`)
- `zero doctor` output for setup problems
- the exact command you ran and the exact error text
## Bugs
If Zero does something wrong (crashes, wrong output, broken behavior), file a
[bug report](https://github.com/Gitlawb/zero/issues/new/choose). If you are not
sure whether it is a bug or a setup problem, start in Discussions and we will
promote it to an issue if it is one.
## Feature ideas
Start a thread in
[Ideas](https://github.com/Gitlawb/zero/discussions/categories/ideas). Zero is
in controlled development (see CONTRIBUTING.md), so discussing an idea first is
also the path to getting a PR accepted.
## Security issues
Never file a public issue or discussion. Report privately via
[GitHub Security Advisories](https://github.com/Gitlawb/zero/security/advisories/new)
(see SECURITY.md).
+15 -159
View File
@@ -55,133 +55,6 @@ function localControlHelperManifest(packageRoot) {
return JSON.stringify({ version: 1, helpers });
}
// Mirrors internal/cli/observability.go parseDoctorArgs + writeDoctorHelp so the
// wrapper's missing-binary doctor fallback matches the Go doctor CLI surface.
const DOCTOR_HELP = `Usage:
zero doctor [flags]
Runs Go backend health checks for config and provider setup.
Flags:
--json Print JSON report
--connectivity Include provider endpoint connectivity probe when available
-h, --help Show this help
`;
const EXIT_USAGE = 2;
function installedByBun() {
if (process.env.ZERO_WRAPPER_SIMULATE_BUN === '1') {
return true;
}
return process.execPath.includes('bun') || !!process.versions?.bun;
}
function bunRecoveryParagraph() {
return (
'You installed with Bun, which does not run dependency lifecycle scripts\n' +
'by default. Trust the package to run the blocked postinstall:\n' +
' bun pm trust @gitlawb/zero (project install)\n' +
' bun pm -g trust @gitlawb/zero (global install)\n' +
'On Bun versions without `bun pm trust`, add\n' +
' "trustedDependencies": ["@gitlawb/zero"]\n' +
'to your project package.json and reinstall.\n' +
'\n'
);
}
function buildFromSourceParagraph() {
return (
'If this platform has no prebuilt binary, build from source:\n' +
'https://github.com/Gitlawb/zero (go run ./cmd/zero, requires Go 1.26+).'
);
}
function missingBinaryContextParagraph() {
return (
'Normally npm installs it as an optional dependency of @gitlawb/zero\n' +
`(@gitlawb/zero-${process.platform}-${process.arch}), and the wrapper can\n` +
'also download it from the GitHub Release when it is missing.'
);
}
function missingNativeRecoveryParagraphs(postinstallScript) {
const ranByBun = installedByBun();
return (
missingBinaryContextParagraph() +
'\n' +
'\n' +
'Things to try:\n' +
' - reinstall without omitting optional dependencies:\n' +
' npm install -g @gitlawb/zero\n' +
' - run the downloader manually (needs write access to the package dir):\n' +
` node "${postinstallScript}"\n` +
'\n' +
(ranByBun ? bunRecoveryParagraph() : '') +
buildFromSourceParagraph()
);
}
function formatGenericMissingNativeBinaryMessage(postinstallScript) {
return (
'[zero] No native binary is available for this install.\n' +
missingNativeRecoveryParagraphs(postinstallScript)
);
}
function parseDoctorArgs(args) {
let json = false;
for (const arg of args) {
switch (arg) {
case '-h':
case '--help':
case 'help':
return { kind: 'help' };
case '--json':
json = true;
break;
case '--connectivity':
break;
default:
return { kind: 'error', message: `unknown doctor flag ${JSON.stringify(arg)}` };
}
}
return { kind: 'run', json };
}
function missingNativeDoctorJSONReport(postinstallScript) {
return {
generatedAt: new Date().toISOString().replace(/\.\d{3}Z$/, 'Z'),
ok: false,
checks: [
{
id: 'runtime.go',
label: 'Go runtime',
status: 'fail',
message: 'Native zero binary is missing next to the npm wrapper.',
details: {
remedy: `node "${postinstallScript}"`,
},
},
],
};
}
function missingNativeDoctorTextReport(postinstallScript) {
return (
'Zero doctor report (' +
new Date().toISOString() +
')\n' +
'Overall: fail\n' +
'[fail] runtime.go - Native zero binary is missing next to the npm wrapper.\n' +
' remedy: node "' +
postinstallScript +
'"\n' +
'\n' +
missingNativeRecoveryParagraphs(postinstallScript)
);
}
// The platform payload is a version of @gitlawb/zero itself, installed under
// an npm: alias (see docs/NPM_PACKAGING.md). The alias name is derived from
// process.platform/process.arch, so an unsupported platform simply fails to
@@ -241,38 +114,21 @@ function resolveNativeBinary() {
const nativePath = resolveNativeBinary();
if (!nativePath) {
const postinstallScript = join(packageRoot, 'scripts', 'postinstall.mjs');
// `zero doctor` is the diagnostic command: when the native binary is missing
// it's the one invocation that must NOT bail with the generic wrapper error,
// because that's exactly the blind alley issue #405 reports. Instead, surface
// a doctor-shaped FAIL line for the runtime so the user's diagnostic finds the
// real cause. We branch on a literal 'doctor' subcommand only (matching `zero
// doctor` and `zero doctor --connectivity`), preserving the existing bail for
// every other invocation (exec, providers list, TUI, --version, etc.).
const argv = process.argv.slice(2);
const isDoctor = argv.length > 0 && argv[0] === 'doctor';
if (isDoctor) {
const parsed = parseDoctorArgs(argv.slice(1));
if (parsed.kind === 'help') {
process.stdout.write(DOCTOR_HELP);
process.exit(0);
}
if (parsed.kind === 'error') {
process.stderr.write(`[zero] ${parsed.message}\n`);
process.exit(EXIT_USAGE);
}
if (parsed.json) {
process.stdout.write(JSON.stringify(missingNativeDoctorJSONReport(postinstallScript), null, 2) + '\n');
process.exit(1);
}
process.stdout.write(missingNativeDoctorTextReport(postinstallScript) + '\n');
process.exit(1);
}
console.error(formatGenericMissingNativeBinaryMessage(postinstallScript));
console.error(
'[zero] No native binary is available for this install.\n' +
'Normally npm installs it as an optional dependency of @gitlawb/zero\n' +
`(@gitlawb/zero-${process.platform}-${process.arch}), and the wrapper can\n` +
'also download it from the GitHub Release when it is missing.\n' +
'\n' +
'Things to try:\n' +
' - reinstall without omitting optional dependencies:\n' +
' npm install -g @gitlawb/zero\n' +
' - run the downloader manually (needs write access to the package dir):\n' +
` node "${join(packageRoot, 'scripts', 'postinstall.mjs')}"\n` +
'\n' +
'If this platform has no prebuilt binary, build from source:\n' +
'https://github.com/Gitlawb/zero (go run ./cmd/zero, requires Go 1.26+).',
);
process.exit(1);
}
-17
View File
@@ -4,8 +4,6 @@ import (
"bytes"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/perfbench"
)
func TestParseArgsUsesCliAndEnvOverrides(t *testing.T) {
@@ -94,18 +92,3 @@ func TestHelpText(t *testing.T) {
func emptyEnv(string) string {
return ""
}
func TestTurnExitCodeFailsWhenEveryTaskErrored(t *testing.T) {
var stderr bytes.Buffer
allErrored := perfbench.TurnBenchResult{TasksAttempted: 3, TasksErrored: 3}
if code := turnExitCode(allErrored, &stderr); code != 1 {
t.Fatalf("exit=%d, want 1 when every task errored", code)
}
if !strings.Contains(stderr.String(), "no accepted benchmark sample") {
t.Fatalf("stderr missing explanation: %q", stderr.String())
}
partial := perfbench.TurnBenchResult{TasksAttempted: 3, TasksErrored: 2}
if code := turnExitCode(partial, &stderr); code != 0 {
t.Fatalf("exit=%d, want 0 for partial errors (summary surfaces them)", code)
}
}
+1 -42
View File
@@ -9,7 +9,6 @@ import (
"path/filepath"
"strings"
"github.com/Gitlawb/zero/internal/execprofile"
"github.com/Gitlawb/zero/internal/perfbench"
)
@@ -21,7 +20,6 @@ type turnOptions struct {
SuitePath string
Model string
Mode string
ExecProfile string
SelfCorrect bool
Binary string
Iterations int
@@ -66,7 +64,6 @@ func runTurnCommand(args []string, getenv func(string) string, stdout io.Writer,
result, err := perfbench.RunTurnBench(context.Background(), set, perfbench.TurnBenchConfig{
Model: options.Model,
Mode: options.Mode,
ExecProfile: options.ExecProfile,
SelfCorrect: options.SelfCorrect,
Version: options.Version,
Commit: options.Commit,
@@ -89,23 +86,9 @@ func runTurnCommand(args []string, getenv func(string) string, stdout io.Writer,
_, _ = fmt.Fprintln(stderr, "[zero] Turn benchmark failed: "+err.Error())
return 1
}
return turnExitCode(result, stderr)
return 0
}
_, _ = fmt.Fprintln(stdout, perfbench.FormatTurnBenchSummary(result))
return turnExitCode(result, stderr)
}
// turnExitCode fails the command when every attempted task errored (no
// iteration produced an accepted benchmark sample): such a report measures
// nothing, and exiting 0 would let a broken configuration (missing binary, bad
// path, failing harness step) pass as a clean baseline. Partial errors keep
// exit 0 — the summary surfaces them loudly and the surviving samples are
// still valid measurements.
func turnExitCode(result perfbench.TurnBenchResult, stderr io.Writer) int {
if result.TasksAttempted > 0 && result.TasksErrored == result.TasksAttempted {
_, _ = fmt.Fprintln(stderr, "[zero] Turn benchmark failed: every task errored with no accepted benchmark sample (see warnings); the report contains no valid measurements")
return 1
}
return 0
}
@@ -140,13 +123,6 @@ func parseTurnArgs(args []string, getenv func(string) string) (turnOptions, erro
}
options.Mode = value
index = next
case "--exec-profile":
value, next, err := readOptionValue(args, inlineValue, index, flag)
if err != nil {
return options, err
}
options.ExecProfile = value
index = next
case "--binary":
value, next, err := readOptionValue(args, inlineValue, index, flag)
if err != nil {
@@ -219,20 +195,6 @@ func parseTurnArgs(args []string, getenv func(string) string) (turnOptions, erro
if strings.TrimSpace(options.Model) == "" && !options.DryRun {
return options, fmt.Errorf("--model is required (or pass --dry-run)")
}
// Validate the profile here, before anything spawns: an unknown name would
// otherwise make every child exit with a usage error in milliseconds, and
// those near-zero walls would be recorded as valid latency samples in an
// exit-0 report — exactly the misread-as-improvement trap the harness
// closed for spawn failures. Normalizing to the catalog name also keeps
// the stamped execProfile canonical (Lookup is case-insensitive), so two
// captures of the same posture always compare equal.
if raw := strings.TrimSpace(options.ExecProfile); raw != "" {
profile, ok := execprofile.Lookup(raw)
if !ok {
return options, fmt.Errorf("unknown execution profile %q for --exec-profile. Valid profiles: %s", raw, strings.Join(execprofile.Names(), ", "))
}
options.ExecProfile = profile.Name
}
return options, nil
}
@@ -263,9 +225,6 @@ func turnHelpText() string {
" --suite <path> Task set JSON file (required)",
" --model <model> Model to run (required unless --dry-run)",
" --mode <name> Exec mode preset to apply",
" --exec-profile <name>",
" Execution profile for every task (fast|balanced|thorough);",
" forwarded to zero exec and stamped into the result",
" --self-correct Enable the post-edit verify-and-correct loop",
" --binary <path> Path to the `zero` binary (default: zero on PATH / repo root)",
" --iterations <n> Times to run each task (default: 1)",
-65
View File
@@ -1,65 +0,0 @@
package main
import (
"strings"
"testing"
)
func TestParseTurnArgsExecProfile(t *testing.T) {
getenv := func(string) string { return "" }
options, err := parseTurnArgs([]string{
"--suite", "suite.json",
"--model", "m",
"--exec-profile", "fast",
}, getenv)
if err != nil {
t.Fatalf("parseTurnArgs: %v", err)
}
if options.ExecProfile != "fast" {
t.Fatalf("ExecProfile = %q, want fast", options.ExecProfile)
}
// Inline form.
options, err = parseTurnArgs([]string{
"--suite=suite.json",
"--model=m",
"--exec-profile=thorough",
}, getenv)
if err != nil {
t.Fatalf("parseTurnArgs inline: %v", err)
}
if options.ExecProfile != "thorough" {
t.Fatalf("ExecProfile = %q, want thorough", options.ExecProfile)
}
}
// The bench validates the profile BEFORE spawning anything: an unknown name
// would otherwise make every child exit with a usage error in milliseconds
// and record those near-zero walls as valid latency samples in an exit-0
// report. Lookup normalization also keeps the stamped name canonical so two
// captures of the same posture compare equal.
func TestParseTurnArgsExecProfileValidatesAndNormalizes(t *testing.T) {
getenv := func(string) string { return "" }
options, err := parseTurnArgs([]string{
"--suite", "suite.json", "--model", "m", "--exec-profile", "FAST",
}, getenv)
if err != nil {
t.Fatalf("parseTurnArgs: %v", err)
}
if options.ExecProfile != "fast" {
t.Fatalf("ExecProfile = %q, want the canonical fast", options.ExecProfile)
}
_, err = parseTurnArgs([]string{
"--suite", "suite.json", "--model", "m", "--exec-profile", "blanced",
}, getenv)
if err == nil {
t.Fatal("an unknown profile must fail parse, before anything spawns")
}
for _, name := range []string{"balanced", "fast", "thorough"} {
if !strings.Contains(err.Error(), name) {
t.Fatalf("error must list %q, got %q", name, err.Error())
}
}
}
+3 -16
View File
@@ -272,8 +272,7 @@ sequenceDiagram
Agent->>Tool: execute(ctx, args)
Tool-->>Agent: output, error, metadata
Agent->>Hooks: afterTool hooks
Agent->>Agent: redact secrets, apply semantic output budget, enforce byte ceiling
Agent->>Agent: classify success/failure
Agent->>Agent: redact, truncate, classify success/failure
Agent-->>Surface: OnToolResult callback
Agent->>Transcript: append tool result message
end
@@ -605,22 +604,10 @@ flowchart TD
Prompt -- Yes --> Decision[Permission callback]
Prompt -- No --> Run[Registry.RunWithOptions]
Decision --> Run
Run --> Redact[Redact secrets]
Redact --> Budget[Token-aware semantic output budget]
Budget --> Ceiling[Existing hard byte ceiling]
Ceiling --> Message[Return one tool result to model]
Run --> Redact[Redact secrets + enforce output ceiling]
Redact --> Message[Return tool result to model]
```
Oversized results use deterministic, provider-neutral estimated-token budgets.
Policies retain useful structure for files, search matches, tests, process logs,
diffs, and worker conclusions; tools without a declared category use a UTF-8-safe
head/tail fallback. The estimate is intentionally conservative for non-ASCII
text and is not exact provider tokenization. Existing byte ceilings remain the
authoritative safety limit. When the existing spill mechanism can persist the
complete redacted text received by the budgeting layer, the result includes its
safe spill reference. This does not imply capture of subprocess bytes already
discarded by a tool's established internal buffer.
Core tool groups include:
- **Read-only tools**: file reads, directory listing, glob, grep, LSP navigation,
+1 -6
View File
@@ -141,11 +141,9 @@ func TestRunAskUserCancellationAbortsRun(t *testing.T) {
registry := registryWithAskUser()
args := `{"questions":[{"question":"Which framework?"}]}`
provider := providerCallingAskUserThenAnswer(args, "done")
var toolResults []ToolResult
result, err := Run(context.Background(), "clarify", provider, Options{
Registry: registry,
OnToolResult: func(result ToolResult) { toolResults = append(toolResults, result) },
Registry: registry,
OnAskUser: func(_ context.Context, _ AskUserRequest) (AskUserResponse, error) {
return AskUserResponse{}, context.Canceled
},
@@ -160,9 +158,6 @@ func TestRunAskUserCancellationAbortsRun(t *testing.T) {
if len(provider.requests) != 1 {
t.Fatalf("expected the run to stop after the canceled ask_user (1 turn), got %d", len(provider.requests))
}
if len(toolResults) != 1 || toolResults[0].ToolCallID == "" {
t.Fatalf("cancellation must emit exactly one result for the call, got %#v", toolResults)
}
// The recorded tool result must reflect cancellation, not a synthetic answer.
var toolMsg string
for _, m := range result.Messages {
+3 -12
View File
@@ -56,10 +56,6 @@ type CompactionOptions struct {
// injected so Compact stays pure and testable; the agent loop wires it to a
// real provider call.
Summarize func(toSummarize []zeroruntime.Message) (string, error)
// taskState is a snapshot supplied by the running agent. Its immutable
// objective is always preserved; mutable fields are admitted only when its
// plan projection still matches the transcript.
taskState *taskStateSnapshot
}
// CompactionResult is the metadata-bearing result returned by CompactMessages.
@@ -224,7 +220,7 @@ func CompactMessages(messages []zeroruntime.Message, opts CompactionOptions) (Co
// Preserve structured state (active plan + loaded skills) from the elided
// middle verbatim, so it is not lost or paraphrased away by the prose summary.
content := appendPreservedState(summaryLabel+"\n"+summary, middle, opts.taskState)
content := appendPreservedState(summaryLabel+"\n"+summary, middle)
compacted := make([]zeroruntime.Message, 0, systemEnd+1+(len(messages)-boundary))
compacted = append(compacted, messages[:systemEnd]...)
@@ -330,7 +326,6 @@ type compactionState struct {
// OnText is deliberately NOT forwarded (compaction stays invisible to the user),
// but its token COST must still be counted so usage reports and budgets include it.
onUsage func(Usage)
task *taskState
// calibrationRatio scales the raw byte/4 token estimate toward the provider's
// real prompt-token count. ApproxTextTokens over-counts code-heavy content by
@@ -370,15 +365,13 @@ func (state *compactionState) calibratedTokens(raw int) int {
return int(float64(raw) * state.calibrationRatio)
}
func newCompactionState(options Options, task *taskState) *compactionState {
state := &compactionState{
func newCompactionState(options Options) *compactionState {
return &compactionState{
enabled: options.ContextWindow > 0,
threshold: compactionThreshold(options.ContextWindow),
preserveLast: options.CompactionPreserveLast,
onUsage: options.OnUsage,
task: task,
}
return state
}
// maybeCompact runs proactive compaction at the top of a turn. It returns the
@@ -425,7 +418,6 @@ func (state *compactionState) maybeCompact(
compacted, err := Compact(messages, CompactionOptions{
PreserveLast: state.preserveLast,
Summarize: summarizeClosure(ctx, provider, state.onUsage),
taskState: state.task.snapshotForCompaction(messages),
})
if err != nil {
// Summarizer failed: keep the original history. The reactive path (or a
@@ -476,7 +468,6 @@ func (state *compactionState) recover(
result, compactErr := Compact(messages, CompactionOptions{
PreserveLast: state.preserveLast,
Summarize: summarizeClosure(ctx, provider, state.onUsage),
taskState: state.task.snapshotForCompaction(messages),
})
if compactErr != nil {
// A genuine compaction attempt was made (and failed): the budget is spent
+29 -60
View File
@@ -30,10 +30,6 @@ const (
const (
maxRecentEdits = 20
maxEditNoteBytes = 160
// maxTaskObjectiveBytes keeps the original objective recognizable after
// compaction without allowing an unusually large prompt to recreate the
// context pressure the compaction just removed.
maxTaskObjectiveBytes = 512
)
const (
@@ -80,14 +76,32 @@ func extractLatestPlan(messages []zeroruntime.Message) string {
// ({"plan":[{content,status,...}]}) as terse status-tagged bullet lines. Returns
// "" on malformed arguments or an empty plan.
func formatPlanArguments(arguments string) string {
plan, ok := parseTaskPlan(arguments)
if !ok {
var parsed struct {
Plan []struct {
Content string `json:"content"`
Step string `json:"step"`
Status string `json:"status"`
Notes string `json:"notes"`
} `json:"plan"`
}
if err := json.Unmarshal([]byte(strings.TrimSpace(arguments)), &parsed); err != nil {
return ""
}
lines := make([]string, 0, len(plan))
for _, item := range plan {
line := "- [" + item.Status + "] " + item.Content
if notes := item.Notes; notes != "" {
lines := make([]string, 0, len(parsed.Plan))
for _, item := range parsed.Plan {
content := strings.TrimSpace(item.Content)
if content == "" {
content = strings.TrimSpace(item.Step)
}
if content == "" {
continue
}
status := strings.TrimSpace(item.Status)
if status == "" {
status = "pending"
}
line := "- [" + status + "] " + content
if notes := strings.TrimSpace(item.Notes); notes != "" {
line += "\n Notes: " + notes
}
lines = append(lines, line)
@@ -417,25 +431,12 @@ func capBody(body string) string {
// preservedState is the JSON shape of the carried-across-compaction block.
type preservedState struct {
Plan string `json:"plan,omitempty"`
Task *preservedTaskState `json:"task,omitempty"`
RecentEdits []preservedEdit `json:"recent_edits,omitempty"`
Tools []preservedTool `json:"tools,omitempty"`
Skills []preservedSkill `json:"skills,omitempty"`
ProjectInstructions []preservedInstruction `json:"project_instructions,omitempty"`
}
type preservedTaskState struct {
Objective string `json:"objective"`
Status taskStatus `json:"status,omitempty"`
Pending int `json:"pending,omitempty"`
InProgress int `json:"in_progress,omitempty"`
Completed int `json:"completed,omitempty"`
Failed int `json:"failed,omitempty"`
VerificationPassed int `json:"verification_passed,omitempty"`
VerificationFailed int `json:"verification_failed,omitempty"`
VerificationOutcome Outcome `json:"verification_outcome,omitempty"`
}
type preservedEdit struct {
Path string `json:"path"`
Note string `json:"note,omitempty"`
@@ -463,27 +464,8 @@ type preservedInstruction struct {
// may live only inside the injected summary message, which on a later compaction
// lands in middle with no real tool calls left to extract. Fresh tool calls and
// instruction blocks override the carried-forward copy by name/source.
func appendPreservedState(summary string, middle []zeroruntime.Message, taskSnapshot *taskStateSnapshot) string {
func appendPreservedState(summary string, middle []zeroruntime.Message) string {
priorState := parsePreservedStateBlock(latestSummaryContent(middle))
task := priorState.Task
if taskSnapshot != nil {
task = &preservedTaskState{
Objective: capTaskObjective(taskSnapshot.Objective),
}
// Plan parity corroborates only the mutable task projection. The objective
// comes directly from the run prompt and is immutable, so it must survive
// even after compaction removes the plan tool call needed for comparison.
if taskSnapshot.PlanParity == taskPlanParityMatch {
task.Status = taskSnapshot.Status
task.Pending = taskSnapshot.Plan.Pending
task.InProgress = taskSnapshot.Plan.InProgress
task.Completed = taskSnapshot.Plan.Completed
task.Failed = taskSnapshot.Plan.Failed
task.VerificationPassed = taskSnapshot.Verification.Passed
task.VerificationFailed = taskSnapshot.Verification.Failed
task.VerificationOutcome = taskSnapshot.Verification.LastOutcome
}
}
// Plan: a fresh update_plan in middle is authoritative; otherwise carry the
// plan preserved by an earlier compaction.
@@ -511,25 +493,12 @@ func appendPreservedState(summary string, middle []zeroruntime.Message, taskSnap
projectInstructionEntries(middle),
)
if block := formatPreservedState(plan, task, edits, tools, skills, instructions); block != "" {
if block := formatPreservedState(plan, edits, tools, skills, instructions); block != "" {
summary += "\n\n" + block
}
return summary
}
func capTaskObjective(objective string) string {
objective = strings.TrimSpace(objective)
if len(objective) <= maxTaskObjectiveBytes {
return objective
}
const suffix = "…"
limit := maxTaskObjectiveBytes - len(suffix)
for limit > 0 && !utf8.RuneStart(objective[limit]) {
limit--
}
return strings.TrimSpace(objective[:limit]) + suffix
}
// mergeRecentEdits overlays fresh edits onto edits preserved by an earlier
// compaction. Unlike mergeSkillEntries (which keeps refreshed entries in their
// original slot), a path touched again by a fresh edit MOVES to the newest
@@ -586,11 +555,11 @@ func preservedEditsToEntries(edits []preservedEdit) []skillEntry {
// formatPreservedState renders state as the labelled, single-line
// JSON block. Returns "" when there is nothing to preserve.
func formatPreservedState(plan string, task *preservedTaskState, edits, tools, skills, instructions []skillEntry) string {
if plan == "" && task == nil && len(edits) == 0 && len(tools) == 0 && len(skills) == 0 && len(instructions) == 0 {
func formatPreservedState(plan string, edits, tools, skills, instructions []skillEntry) string {
if plan == "" && len(edits) == 0 && len(tools) == 0 && len(skills) == 0 && len(instructions) == 0 {
return ""
}
state := preservedState{Plan: plan, Task: task}
state := preservedState{Plan: plan}
for _, e := range edits {
state.RecentEdits = append(state.RecentEdits, preservedEdit{Path: e.name, Note: e.body})
}
+5 -101
View File
@@ -1,7 +1,6 @@
package agent
import (
"encoding/json"
"strings"
"testing"
"unicode/utf8"
@@ -60,103 +59,6 @@ func TestCompactPreservesActivePlan(t *testing.T) {
}
}
func TestCompactPreservesBoundedTaskContext(t *testing.T) {
objective := strings.Repeat("世", maxTaskObjectiveBytes)
task := newTaskState(objective, nil)
task.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"write code","status":"in_progress"},{"content":"add tests","status":"pending"}]}`})
messages := stateConversation()
compacted, err := Compact(messages, CompactionOptions{
PreserveLast: 2,
Summarize: func([]zeroruntime.Message) (string, error) { return "SUMMARY", nil },
taskState: task.snapshotForCompaction(messages),
})
if err != nil {
t.Fatalf("Compact: %v", err)
}
state := parsePreservedStateBlock(compacted[1].Content)
if state.Task == nil || state.Task.Status != taskStatusActive || state.Task.InProgress != 1 || state.Task.Pending != 1 {
t.Fatalf("unexpected compact task state: %#v", state.Task)
}
if len(state.Task.Objective) > maxTaskObjectiveBytes || !utf8.ValidString(state.Task.Objective) {
t.Fatalf("objective was not safely bounded: %d bytes %q", len(state.Task.Objective), state.Task.Objective)
}
}
func TestCompactPreservesObjectiveAfterPlanParityMismatch(t *testing.T) {
prior := preservedState{Task: &preservedTaskState{Objective: "stale objective", Status: taskStatusActive, Pending: 1}}
encoded, err := json.Marshal(prior)
if err != nil {
t.Fatal(err)
}
messages := []zeroruntime.Message{
{Role: zeroruntime.MessageRoleSystem, Content: "system"},
{Role: zeroruntime.MessageRoleUser, Content: summaryLabel + "\nold\n\n" + preservedStateLabel + "\n" + string(encoded)},
{Role: zeroruntime.MessageRoleAssistant, Content: "continuing"},
{Role: zeroruntime.MessageRoleUser, Content: "more"},
{Role: zeroruntime.MessageRoleAssistant, Content: "working"},
{Role: zeroruntime.MessageRoleUser, Content: "again"},
{Role: zeroruntime.MessageRoleAssistant, Content: "done"},
}
compacted, err := Compact(messages, CompactionOptions{
PreserveLast: 2,
Summarize: func([]zeroruntime.Message) (string, error) { return "SUMMARY", nil },
taskState: &taskStateSnapshot{
Objective: "current objective",
PlanParity: taskPlanParityMismatch,
},
})
if err != nil {
t.Fatalf("Compact: %v", err)
}
state := parsePreservedStateBlock(compacted[1].Content)
if state.Task == nil || state.Task.Objective != "current objective" {
t.Fatalf("immutable objective was lost on plan mismatch: %#v", state.Task)
}
if state.Task.Status != "" || state.Task.Pending != 0 {
t.Fatalf("uncorroborated mutable task fields survived plan mismatch: %#v", state.Task)
}
}
func TestTaskObjectiveSurvivesRepeatedCompactionWithoutPlanRefresh(t *testing.T) {
task := newTaskState("keep this objective", nil)
task.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"write code","status":"in_progress"},{"content":"add tests","status":"pending"}]}`})
messages := stateConversation()
first, err := Compact(messages, CompactionOptions{
PreserveLast: 2,
Summarize: func([]zeroruntime.Message) (string, error) { return "FIRST", nil },
taskState: task.snapshotForCompaction(messages),
})
if err != nil {
t.Fatalf("first Compact: %v", err)
}
secondInput := append(append([]zeroruntime.Message{}, first...),
zeroruntime.Message{Role: zeroruntime.MessageRoleUser, Content: "more"},
zeroruntime.Message{Role: zeroruntime.MessageRoleAssistant, Content: "working"},
zeroruntime.Message{Role: zeroruntime.MessageRoleUser, Content: "again"},
zeroruntime.Message{Role: zeroruntime.MessageRoleAssistant, Content: "done"},
)
snapshot := task.snapshotForCompaction(secondInput)
if snapshot.PlanParity != taskPlanParityMismatch {
t.Fatalf("plan call should be absent after first compaction, parity=%q", snapshot.PlanParity)
}
second, err := Compact(secondInput, CompactionOptions{
PreserveLast: 2,
Summarize: func([]zeroruntime.Message) (string, error) { return "SECOND", nil },
taskState: snapshot,
})
if err != nil {
t.Fatalf("second Compact: %v", err)
}
state := parsePreservedStateBlock(second[1].Content)
if state.Task == nil || state.Task.Objective != "keep this objective" {
t.Fatalf("objective lost after second compaction: %#v", state.Task)
}
if state.Task.Status != "" || state.Task.InProgress != 0 {
t.Fatalf("uncorroborated mutable fields survived second compaction: %#v", state.Task)
}
}
func TestCompactPreservesLoadedSkills(t *testing.T) {
summary := compactStateConversation(t, stateConversation())
if !strings.Contains(summary, preservedStateLabel) {
@@ -381,10 +283,12 @@ func TestExtractLatestPlanReturnsMostRecent(t *testing.T) {
}
}
func TestFormatPlanArgumentsRejectsUnsupportedStepAlias(t *testing.T) {
func TestFormatPlanArgumentsAcceptsStepAlias(t *testing.T) {
got := formatPlanArguments(`{"plan":[{"step":"write failing test","status":"in_progress"},{"content":"keep existing shape","status":"pending"}]}`)
if got != "" {
t.Fatalf("unsupported alias should reject the whole plan like update_plan, got %q", got)
for _, want := range []string{"- [in_progress] write failing test", "- [pending] keep existing shape"} {
if !strings.Contains(got, want) {
t.Fatalf("expected %q in formatted plan, got %q", want, got)
}
}
}
+2 -2
View File
@@ -581,7 +581,7 @@ func TestCompactNeverProducesConsecutiveUserMessages(t *testing.T) {
}
func TestRecoverNoopDoesNotConsumeReactiveBudget(t *testing.T) {
st := newCompactionState(Options{ContextWindow: 1000, CompactionPreserveLast: 2}, nil)
st := newCompactionState(Options{ContextWindow: 1000, CompactionPreserveLast: 2})
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventText, Content: "SUMMARY"}, {Type: zeroruntime.StreamEventDone},
}}}
@@ -628,7 +628,7 @@ func TestRecoverNoopDoesNotConsumeReactiveBudget(t *testing.T) {
}
func TestRecoverDisabledIsNoop(t *testing.T) {
st := newCompactionState(Options{ContextWindow: 0}, nil)
st := newCompactionState(Options{ContextWindow: 0})
msgs := []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "x"}}
called := false
// recover must not invoke the provider/summarize when disabled, even on a
-86
View File
@@ -1,86 +0,0 @@
package agent
// CompletionDecision is the deterministic outcome of evaluating a text-only
// assistant turn. Uncertain means the loop needs one bounded follow-up action
// before it can finalize the run.
type CompletionDecision string
const (
CompletionUncertain CompletionDecision = "uncertain"
CompletionComplete CompletionDecision = "complete"
CompletionIncomplete CompletionDecision = "incomplete"
)
type completionAction string
const (
completionActionNone completionAction = ""
completionActionContinue completionAction = "continue"
completionActionSemanticCheck completionAction = "semantic_check"
)
type completionEvaluation struct {
Decision CompletionDecision
Action completionAction
Reason string
}
// completionPolicy owns the small amount of state needed to keep uncertain
// decisions bounded. requireSemanticCheck is set only for run profiles that
// already opted into self-correction; ordinary completion remains entirely
// local and adds no model call.
type completionPolicy struct {
continueNudges int
requireSemanticCheck bool
semanticCheckRequested bool
}
func newCompletionPolicy(requireSemanticCheck bool) *completionPolicy {
return &completionPolicy{requireSemanticCheck: requireSemanticCheck}
}
func (policy *completionPolicy) evaluate(text string, context completionContext) completionEvaluation {
// A direct admission is strong local evidence and takes precedence over all
// weaker signals, avoiding both continuation nudges and semantic checks.
if reason := selfReportedIncompletion(text); reason != "" {
return completionEvaluation{Decision: CompletionIncomplete, Reason: reason}
}
// A continuation cue is strong unfinished evidence; a pending plan is only
// weak evidence because bookkeeping can be stale. Both get a bounded chance
// to continue, but only a persisted cue is ultimately incomplete.
cue := endsWithContinuationCue(text)
if cue || context.PlanPending {
if policy.continueNudges < maxContinueNudges {
policy.continueNudges++
reason := "your message ended mid-step"
if !cue {
reason = "pending plan items remain — finish them, or mark them complete with update_plan if you are done"
}
return completionEvaluation{
Decision: CompletionUncertain,
Action: completionActionContinue,
Reason: reason,
}
}
if cue {
return completionEvaluation{
Decision: CompletionIncomplete,
Reason: "your message ended mid-step",
}
}
}
// Profiles with self-correction enabled require one task-grounded semantic
// check. It is requested at most once per run; the next locally complete turn
// is accepted without another model call.
if policy.requireSemanticCheck && !policy.semanticCheckRequested {
policy.semanticCheckRequested = true
return completionEvaluation{
Decision: CompletionUncertain,
Action: completionActionSemanticCheck,
}
}
return completionEvaluation{Decision: CompletionComplete, Action: completionActionNone}
}
-75
View File
@@ -1,75 +0,0 @@
package agent
import (
"strings"
"testing"
)
func TestCompletionPolicyLocalEvidenceDecidesWithoutSemanticCheck(t *testing.T) {
policy := newCompletionPolicy(false)
complete := policy.evaluate("Done. All required checks pass.", completionContext{})
if complete.Decision != CompletionComplete {
t.Fatalf("confident completion decision = %q, want %q", complete.Decision, CompletionComplete)
}
incomplete := policy.evaluate("I couldn't verify the result, so this is my best guess.", completionContext{})
if incomplete.Decision != CompletionIncomplete {
t.Fatalf("admitted failure decision = %q, want %q", incomplete.Decision, CompletionIncomplete)
}
}
func TestCompletionPolicyPreservesBoundedPlanStallProtection(t *testing.T) {
policy := newCompletionPolicy(false)
for attempt := 0; attempt < maxContinueNudges; attempt++ {
got := policy.evaluate("Let me inspect the remaining configuration:", completionContext{PlanPending: true})
if got.Decision != CompletionUncertain || got.Action != completionActionContinue {
t.Fatalf("attempt %d = (%q, %q), want uncertain continue", attempt+1, got.Decision, got.Action)
}
}
got := policy.evaluate("Let me inspect the remaining configuration:", completionContext{PlanPending: true})
if got.Decision != CompletionIncomplete {
t.Fatalf("decision after nudge budget = %q, want %q", got.Decision, CompletionIncomplete)
}
}
func TestCompletionPolicyTreatsPendingPlanAsWeakEvidence(t *testing.T) {
policy := newCompletionPolicy(false)
for attempt := 0; attempt < maxContinueNudges; attempt++ {
got := policy.evaluate("All set.", completionContext{PlanPending: true})
if got.Decision != CompletionUncertain || got.Action != completionActionContinue {
t.Fatalf("attempt %d = (%q, %q), want uncertain continue", attempt+1, got.Decision, got.Action)
}
}
got := policy.evaluate("All set.", completionContext{PlanPending: true})
if got.Decision != CompletionComplete {
t.Fatalf("stale plan decision after nudge budget = %q, want %q", got.Decision, CompletionComplete)
}
}
func TestCompletionPolicyAllowsExactlyOneRequiredSemanticCheck(t *testing.T) {
policy := newCompletionPolicy(true)
first := policy.evaluate("Implemented and tested.", completionContext{})
if first.Decision != CompletionUncertain || first.Action != completionActionSemanticCheck {
t.Fatalf("first decision = (%q, %q), want uncertain semantic check", first.Decision, first.Action)
}
second := policy.evaluate("PASS. The result meets the task criterion.", completionContext{})
if second.Decision != CompletionComplete {
t.Fatalf("post-check decision = %q, want %q", second.Decision, CompletionComplete)
}
if second.Action == completionActionSemanticCheck {
t.Fatal("semantic check was requested more than once")
}
}
func TestAcceptanceVerificationNudgeIncludesBoundedObjective(t *testing.T) {
objective := "make the requested behavior observable"
nudge := acceptanceVerificationNudge(objective)
if !strings.Contains(nudge, "Task objective: "+objective) {
t.Fatalf("semantic check was not grounded in the objective: %q", nudge)
}
}
+13 -11
View File
@@ -1,6 +1,7 @@
package agent
import (
"encoding/json"
"strconv"
"strings"
@@ -126,8 +127,9 @@ func endsWithContinuationCue(text string) bool {
// which the update_plan tool coerces to "pending") counts as remaining, matching
// that tool's own status normalization.
func planStatusRemaining(status string) bool {
switch normalizeTaskPlanStatus(status) {
case "completed", "failed":
switch strings.ToLower(strings.TrimSpace(status)) {
case "completed", "complete", "done", "finished", "resolved", "✓", "x", "[x]",
"failed", "fail", "error", "errored", "blocked", "cancelled", "canceled", "abandoned", "skipped":
return false
default:
return true
@@ -305,8 +307,8 @@ func hasAnyPrefix(s string, prefixes []string) bool {
// the bug-hunt surfaced: well-formed output treated as correct; pre-existing tests
// passing treated as the objective being met; and a result that merely matches a
// baseline the task asked to beat or improve. General — no task-specific content.
func acceptanceVerificationNudge(objective string) string {
nudge := "Before this task can be marked complete, " + acceptanceNudgeMarker + " — " +
func acceptanceVerificationNudge() string {
return "Before this task can be marked complete, " + acceptanceNudgeMarker + " — " +
"NOT the shape or format of your output, NOT that pre-existing tests pass, and NOT that your " +
"result merely matches a baseline you were asked to beat or improve. " +
"Re-read the original task, then run a concrete check that exercises the actual requirement: " +
@@ -314,10 +316,6 @@ func acceptanceVerificationNudge(objective string) string {
"thing the task asked you to produce, recover, fix, or optimize. " +
"If that check passes, reply PASS and cite the evidence. " +
"If it does not pass — or you cannot run such a check — say so plainly and keep working; do not claim success."
if objective = capTaskObjective(objective); objective != "" {
nudge += "\n\nTask objective: " + objective
}
return nudge
}
// toolFailureHintMarker is a stable substring for tests.
@@ -555,12 +553,16 @@ func (state *guardState) pendingPlanItems() bool {
// many items are still remaining. Malformed arguments leave the prior count
// unchanged (best-effort — the plan panel itself tolerates the same).
func (state *guardState) observePlanUpdate(arguments string) {
plan, ok := parseTaskPlan(arguments)
if !ok {
var parsed struct {
Plan []struct {
Status string `json:"status"`
} `json:"plan"`
}
if json.Unmarshal([]byte(arguments), &parsed) != nil {
return
}
pending := 0
for _, item := range plan {
for _, item := range parsed.Plan {
if planStatusRemaining(item.Status) {
pending++
}
-31
View File
@@ -169,37 +169,6 @@ func TestRunResetsEmptyTurnCounterOnToolCall(t *testing.T) {
}
}
// The whole point of #702's fix: the unknown-session error's normalized
// signature must not vary with the session id, so a model probing ids 1, 2,
// 3, … produces one repeated signature the failure guard can count. If the id
// leaked into the first 80 normalized chars, each probe would reset the streak
// and the halt would never fire.
func TestUnknownExecSessionErrorSignatureIsIDInvariant(t *testing.T) {
a := errorSignature(tools.UnknownExecSessionError(1))
b := errorSignature(tools.UnknownExecSessionError(999999))
if a != b {
t.Fatalf("unknown-session signature varies with id:\n %q\n %q", a, b)
}
}
// End to end: with an id-invariant signature, probing a different unknown id
// each turn now trips the repeated-failure halt at toolFailureStopAt, where
// before the fix it never would.
func TestUnknownExecSessionProbingTripsFailureHalt(t *testing.T) {
var state guardState
var stoppedAt int
for i := 1; i <= toolFailureStopAt; i++ {
out := state.observeToolResult(tools.WriteStdinToolName, true, tools.UnknownExecSessionError(i))
if out.Stop {
stoppedAt = i
break
}
}
if stoppedAt != toolFailureStopAt {
t.Fatalf("probing distinct unknown ids stopped at %d, want %d", stoppedAt, toolFailureStopAt)
}
}
func TestGuardStateResetsToolOnlyStreakOnEmptyNonToolTurn(t *testing.T) {
var state guardState
toolOnly := zeroruntime.CollectedStream{
+90 -209
View File
@@ -7,7 +7,6 @@ import (
"fmt"
"io"
"sort"
"strconv"
"strings"
"sync"
@@ -136,33 +135,6 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
}
}
// Establish the run's turn session — the seam an optimized provider session
// (connection reuse, prewarm, native compaction) plugs into without touching
// this loop. Default: wrap the passed provider so the session's Stream IS
// provider.StreamCompletion, byte-identical to calling it directly. Opened
// before dispatchSessionStart so a failed open is a clean run-start error
// with no session hooks fired and nothing to unwind.
turnSessions := options.TurnSessionProvider
if turnSessions == nil {
turnSessions = zeroruntime.NewProviderTurnSessionProvider(provider, zeroruntime.ProviderCapabilities{})
}
session, sessionErr := turnSessions.OpenTurnSession(ctx)
if sessionErr != nil {
return Result{}, fmt.Errorf("agent: open turn session: %w", sessionErr)
}
// Closed via closure (not `defer session.Close()`) so after a mid-run model
// swap the CURRENT session is the one closed at teardown — the swap closes
// the old one inline. Close errors are advisory and never fail the run.
defer func() { _ = session.Close() }()
// Best-effort prewarm: the default adapter no-ops; a real session may prime
// a connection. Failure is ignored by contract (never fatal).
_ = session.Prewarm(ctx)
// Route ALL provider I/O — per-turn streams, mid-stream retries, the
// compaction summarizer, and the final-answer request — through the session
// by reassigning the loop's provider local. Every existing call site reads
// this local, so no signatures change anywhere downstream.
provider = sessionProvider{session: session}
maxTurns := options.MaxTurns
if maxTurns <= 0 {
maxTurns = 12
@@ -181,46 +153,10 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
options.runPermissions = runPermissions
defer runPermissions.cleanup()
promptParts := buildSystemPromptParts(options)
messages := zeroruntime.SeedMessagesWithImages(promptParts.prompt, prompt, options.Images)
messages := zeroruntime.SeedMessagesWithImages(buildSystemPrompt(options), prompt, options.Images)
guards := newGuardState()
task := newTaskState(prompt, options.Trace)
compactor := newCompactionState(options, task)
defer func() {
// A final transcript comparison is observational. It records drift but
// never rewrites the result or changes execution after the fact.
task.observePlanParity(result.Messages)
// Tool-result observations are intentionally coalesced; this final event
// guarantees their aggregate counts are present even on an early return.
task.emit()
}()
// The execution-profile posture controller. nil Options.Profile (every
// existing caller) makes every observe/decide call a no-op, keeping the
// loop byte-identical for unprofiled runs.
posture := newProfileController(options.Profile)
// applyPosture applies at most one posture escalation when an armed trigger
// has fired: raise the hoisted turn ceiling, restore effort and the
// completion gate, stamp the counter. Shared by the end-of-turn tail and
// the completion-gate uncertain path (which continues the loop before the
// tail runs). No-op forever after the first escalation and for unprofiled
// runs.
applyPosture := func() {
if target, fired := posture.maybeEscalate(); fired {
if target.MaxTurns > maxTurns {
maxTurns = target.MaxTurns
}
if target.ReasoningEffort != "" {
options.ReasoningEffort = target.ReasoningEffort
} else if target.RestoreDefaultEffort {
options.ReasoningEffort = ""
}
if target.RestoreCompletionGate {
options.RequireCompletionSignal = true
}
options.Trace.Counter(trace.CounterPostureEscalations, 1)
}
}
compactor := newCompactionState(options)
// Background post-edit diagnostics: files changed by mutating tools are
// checked off the tool-call critical path and any errors are appended as a
@@ -233,10 +169,13 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// loaded tool's full schema; it lives only for the run (v1 within-run scope).
loaded := map[string]bool{}
// The feature-gated completion policy keeps local completion decisions and its
// bounded follow-up state out of the central loop. Self-correcting profiles
// opt into the one permitted task-grounded semantic check.
completionPolicy := newCompletionPolicy(options.SelfCorrect != nil)
// continueNudges counts how many times the headless completion gate
// (Options.RequireCompletionSignal) has re-prompted a no-tool-call turn that
// stopped with work still unfinished. Bounded by maxContinueNudges.
continueNudges := 0
// acceptanceRequested records that the one-time task-grounded acceptance check
// has already been demanded this run, so it fires at most once.
acceptanceRequested := false
// toolDefCache memoizes each tool's rendered JSON-schema definition across
// turns (a tool's advertised schema is stable for the run), so partitionTools
@@ -278,14 +217,16 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
exposed, _ := partitionToolsCached(registry, permissionMode, options, loaded, toolDefCache)
toolPartitionSpan.End()
// Fingerprint the exact system prompt built at run start plus this turn's
// provider-visible tool definitions in their emitted order. Reusing the
// retained prompt parts avoids rebuilding workspace context while ensuring
// the trace describes the same bytes carried by the request.
// Prompt-prefix fingerprint: hash the seven cacheable sub-components
// of this turn's request (system prompt sections, project context,
// skills, partitioned tool list, tool schemas) and emit one trace
// event. Stable across turns => cacheable. Drift in any sub-hash
// names the sub-component that broke the cache for this turn. The
// computation is pure and cheap (seven SHA-256s of small strings)
// and is a no-op when tracing is off.
if options.Trace != nil {
fp := computePrefixFingerprint(buildPromptSubstringsFromParts(promptParts, exposed))
fp := ComputePrefixFingerprint(options, exposed)
options.Trace.EmitPrefixHash(trace.PrefixHash{
SystemPromptHash: fp.SystemPromptHash,
BaseInstructionsHash: fp.BaseInstructionsHash,
ConfirmationPolicyHash: fp.ConfirmationPolicyHash,
ProjectContextHash: fp.ProjectContextHash,
@@ -579,42 +520,69 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// model's final answer ONLY when the work is actually done. Default off
// (RequireCompletionSignal), so interactive runs stay byte-identical.
if options.RequireCompletionSignal {
completionContext := task.completionContext(messages, guards.pendingPlanItems())
evaluation := completionPolicy.evaluate(collected.Text, completionContext)
task.observe(taskStateEvent{kind: taskStateEventCompletion, completion: evaluation})
switch evaluation.Decision {
case CompletionIncomplete:
// (1) Self-report downgrade (strongest, unambiguous): the model's own
// final message admits it guessed / could not meet the objective. Checked
// FIRST so an admitted-impossible task is downgraded immediately (no wasted
// continue-nudges) and reports the accurate reason.
if reason := selfReportedIncompletion(collected.Text); reason != "" {
result.Incomplete = true
result.IncompleteReason = evaluation.Reason
result.IncompleteReason = reason
result.FinalAnswer = collected.Text
result.Messages = copyMessages(messages)
return result, nil
case CompletionUncertain:
// Observe the uncertainty signal and apply any escalation NOW:
// this branch continues the turn loop before the end-of-turn
// act point, so deferring would skip escalation entirely.
posture.observeUncertain()
applyPosture()
switch evaluation.Action {
case completionActionContinue:
options.Trace.Counter(trace.CounterCompletionNudges, 1)
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleUser,
Content: continueNudge(evaluation.Reason),
})
case completionActionSemanticCheck:
options.Trace.Counter(trace.CounterAcceptanceChecks, 1)
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleUser,
Content: acceptanceVerificationNudge(completionContext.Objective),
})
}
continue
case CompletionComplete:
// Local evidence is sufficient; proceed to final diagnostics.
}
} else {
task.observe(taskStateEvent{kind: taskStateEventCompletion, completion: completionEvaluation{Decision: CompletionComplete}})
// (2) The model stopped without a tool call while work may be unfinished:
// - a continuation cue ("…Let me check the config:") is an unambiguous
// mid-step stop;
// - pending update_plan items are a WEAK, ambiguous signal (the model may
// have finished without re-marking the last step).
// Nudge to continue (bounded). After the budget: a persisted continuation
// cue finalizes INCOMPLETE; pending-plan WITHOUT a cue does NOT (that would
// false-fail a completed run with stale bookkeeping) — fall through to the
// acceptance check / success.
cue := endsWithContinuationCue(collected.Text)
planPending := guards.pendingPlanItems()
if cue || planPending {
if continueNudges < maxContinueNudges {
continueNudges++
options.Trace.Counter(trace.CounterCompletionNudges, 1)
reason := "your message ended mid-step"
if !cue {
reason = "pending plan items remain — finish them, or mark them complete with update_plan if you are done"
}
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleUser,
Content: continueNudge(reason),
})
continue
}
if cue {
result.Incomplete = true
result.IncompleteReason = "your message ended mid-step"
result.FinalAnswer = collected.Text
result.Messages = copyMessages(messages)
return result, nil
}
// pending-plan only, budget spent: trust the model's completion claim
// over stale plan bookkeeping; fall through.
}
// (3) Task-grounded acceptance: before accepting a "done" turn as success,
// require ONE acceptance check grounded in the task's stated criterion
// (only when self-correct is on). Rejects "well-formed == correct",
// "existing-tests-pass == objective met", and "result == baseline" false
// successes. Bounded to a single pass; a genuine post-check completion
// (no admission, no cue) then finalizes as success on the next turn.
if options.SelfCorrect != nil && !acceptanceRequested {
acceptanceRequested = true
options.Trace.Counter(trace.CounterAcceptanceChecks, 1)
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleUser,
Content: acceptanceVerificationNudge(),
})
continue
}
}
// Finalization diagnostics gate: edits from this run may still have
// checks in flight — the per-turn drain waits only briefly and defers
@@ -638,11 +606,6 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// executing so the empty-turn counter resets and plan-tracking signals
// stay current.
guards.observeTurn(collected)
for _, call := range collected.ToolCalls {
if call.Name == planToolName {
task.observe(taskStateEvent{kind: taskStateEventPlan, arguments: call.Arguments})
}
}
failureHint := ""
// turnRequestedModel records the FIRST mid-run escalation target requested
@@ -693,8 +656,6 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
toolSpan.End()
}
options.Trace.Counter(trace.CounterToolCalls, 1)
recordOutputBudgetTrace(options.Trace, toolResult)
task.observe(taskStateEvent{kind: taskStateEventToolResult, toolResult: toolResult})
if options.OnToolResult != nil {
options.OnToolResult(toolResult)
}
@@ -741,7 +702,6 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// would misdirect the model toward JSON shape or blocked behavior.
retriableFailure := isRetriableToolError(toolResult)
outcome := guards.observeToolResult(call.Name, retriableFailure, toolResult.Output)
posture.observeToolOutcome(outcome, toolResult)
if outcome.Stop {
// The assistant message advertised EVERY collected tool call, but
// the guard halts mid-turn so the calls after this one never run.
@@ -774,10 +734,7 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// the assistant's tool_results stay contiguous (a user message between
// tool_results breaks strict provider replay). nil SelfCorrect is a no-op.
if options.SelfCorrect != nil && len(changedFilesThisBatch) > 0 {
feedback, selfCorrectOutcome := options.SelfCorrect.AfterEdit(ctx, dedupeStrings(changedFilesThisBatch))
posture.observeSelfCorrect(selfCorrectOutcome)
task.observe(taskStateEvent{kind: taskStateEventVerification, verification: selfCorrectOutcome})
if feedback != "" {
if feedback, _ := options.SelfCorrect.AfterEdit(ctx, dedupeStrings(changedFilesThisBatch)); feedback != "" {
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleUser,
Content: feedback,
@@ -790,53 +747,25 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
// of the run. At most one switch per turn (first request wins). A switcher
// error is NON-FATAL — record a brief note and continue on the current
// model. nil switcher ⇒ requests are ignored entirely (escalation off).
if turnRequestedModel != "" && (options.ModelSessionSwitcher != nil || options.ModelSwitcher != nil) {
// Resolve the escalated model's session source. The target-aware
// ModelSessionSwitcher is preferred: its TurnSessionProvider carries an
// optimized session (and capabilities) across the swap. The legacy
// ModelSwitcher fallback returns a bare Provider, wrapped in the
// default no-op session — identical to pre-seam behavior.
var newSessions zeroruntime.TurnSessionProvider
var switchErr error
if options.ModelSessionSwitcher != nil {
newSessions, switchErr = options.ModelSessionSwitcher(ctx, turnRequestedModel)
} else {
var newProvider Provider
newProvider, switchErr = options.ModelSwitcher(ctx, turnRequestedModel)
if switchErr == nil && newProvider != nil {
newSessions = zeroruntime.NewProviderTurnSessionProvider(newProvider, zeroruntime.ProviderCapabilities{})
}
}
if turnRequestedModel != "" && options.ModelSwitcher != nil {
newProvider, switchErr := options.ModelSwitcher(ctx, turnRequestedModel)
if switchErr != nil {
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleUser,
Content: escalationFailedNoticePrefix + " (" + turnRequestedModel + "): " + switchErr.Error() + ". Continuing on " + options.Model + ".",
})
} else if newSessions != nil {
// Open the new session and close the old one, so the next turn's
// streams and compaction use the new model. For the default adapter
// the open never errors and Close is a no-op, so this stays
// byte-identical to a direct provider reassignment; the guarded open
// and prewarm matter once a session holds real state.
newSession, openErr := newSessions.OpenTurnSession(ctx)
if openErr != nil {
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleUser,
Content: escalationFailedNoticePrefix + " (" + turnRequestedModel + "): " + openErr.Error() + ". Continuing on " + options.Model + ".",
})
} else {
_ = session.Close()
session = newSession
_ = session.Prewarm(ctx)
provider = sessionProvider{session: session}
options.Model = turnRequestedModel
options.Trace.Counter(trace.CounterModelSwitches, 1)
// KNOWN LIMITATION (deferred): the compactor's context-window budget
// is fixed at run start from options.ContextWindow and is NOT updated
// here, so a switch to a model with a different window keeps compacting
// against the original budget. Fixing it needs the switcher to also
// report the new window — out of scope for this change.
}
} else if newProvider != nil {
// Reassign the local provider so the next turn's StreamCompletion and
// compaction use it; update options.Model so subsequent RunOptions.Model,
// context-window sizing, and usage attribution follow the new model.
provider = newProvider
options.Model = turnRequestedModel
options.Trace.Counter(trace.CounterModelSwitches, 1)
// KNOWN LIMITATION (deferred): the compactor's context-window budget
// is fixed at run start from options.ContextWindow and is NOT updated
// here, so a switch to a model with a different window keeps compacting
// against the original budget. Fixing it needs a ModelSwitcher contract
// change (return the new window) — out of scope for this change.
}
}
@@ -870,13 +799,6 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
Content: reminder,
})
}
// One-shot posture escalation: when an armed profile trigger fired this
// turn, restore the stricter knob values the profile displaced. Mutates
// only per-turn-read policy (the hoisted turn ceiling, reasoning effort,
// completion gate); never messages, model, or session. No-op forever
// after the first escalation and for every unprofiled run.
applyPosture()
}
if ctx.Err() != nil {
@@ -919,28 +841,6 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
return result, nil
}
func recordOutputBudgetTrace(recorder *trace.Recorder, result ToolResult) {
if recorder == nil || result.Meta["output_budget_category"] == "" {
return
}
parseInt := func(key string) int {
value, _ := strconv.Atoi(result.Meta[key])
return value
}
spillCreated, _ := strconv.ParseBool(result.Meta["output_budget_spill_created"])
recorder.EmitOutputBudget(trace.OutputBudgetEvent{
Tool: result.Name,
Category: result.Meta["output_budget_category"],
OriginalBytes: parseInt("output_budget_original_bytes"),
RetainedBytes: parseInt("output_budget_retained_bytes"),
EstimatedOriginalTokens: parseInt("output_budget_estimated_original_tokens"),
EstimatedRetainedTokens: parseInt("output_budget_estimated_retained_tokens"),
Truncated: result.Truncated,
Reason: result.Meta["output_budget_reason"],
SpillCreated: spillCreated,
})
}
func finalAnswerAfterMaxTurns(ctx context.Context, provider Provider, messages []zeroruntime.Message, toolDefs []zeroruntime.ToolDefinition, options Options) (string, []zeroruntime.Message, string) {
finalMessages := copyMessages(messages)
finalMessages = append(finalMessages, zeroruntime.Message{
@@ -1395,33 +1295,16 @@ func executeToolCall(ctx context.Context, registry *tools.Registry, call ToolCal
if didRedact {
result.Redacted = true
}
result = registry.RebudgetAfterHook(call.Name, args, result)
}
}
// Stamp the sandbox risk classification for this EXECUTED call so run-policy
// observers can see the risk of an allowed mutation. Prefer the preflight
// decision's classification when the sandbox evaluated the call; otherwise
// run the same pure classifier the permission path uses. Denied/canceled
// results returned earlier keep the zero value.
executedRisk := sandbox.Risk{}
if preflightDecision != nil {
executedRisk = preflightDecision.Risk
} else if toolFound {
// Unknown-tool results fall through here with a nil tool; they carry a
// denial and keep the zero risk value.
executedRisk = sandbox.Classify(sandboxRequest(call.Name, tool, args, permissionGranted, permissionMode, options))
}
// Secret scrubbing happens at the registry boundary (the single point both
// the agent loop and the MCP server pass through), so result.Output is
// already redacted here and result.Redacted reflects whether it changed.
return ToolResult{
Risk: executedRisk,
ToolCallID: call.ID,
Name: call.Name,
Status: result.Status,
Output: result.Output,
Truncated: result.Truncated,
Meta: result.Meta,
Redacted: result.Redacted,
ChangedFiles: result.ChangedFiles,
@@ -1732,7 +1615,6 @@ func toolResultFromPrePermissionReject(call ToolCall, result tools.Result) ToolR
Name: call.Name,
Status: result.Status,
Output: output,
Truncated: result.Truncated,
Meta: meta,
Redacted: result.Redacted || outputRedacted || summaryRedacted || metaRedacted,
ChangedFiles: result.ChangedFiles,
@@ -2002,7 +1884,6 @@ func askUserFallbackResult(ctx context.Context, registry *tools.Registry, call T
Name: call.Name,
Status: result.Status,
Output: result.Output,
Truncated: result.Truncated,
Meta: result.Meta,
Redacted: result.Redacted,
ChangedFiles: result.ChangedFiles,
-41
View File
@@ -8,7 +8,6 @@ import (
"os"
"os/exec"
"path/filepath"
"reflect"
"runtime"
"strings"
"testing"
@@ -1259,46 +1258,6 @@ func TestRunExecutesToolCallThroughRegistry(t *testing.T) {
}
}
func TestRunPreservesRequestPrefixAcrossTurns(t *testing.T) {
root := t.TempDir()
writeAgentTestFile(t, filepath.Join(root, "notes.txt"), "alpha\n")
registry := tools.NewRegistry()
registry.Register(tools.NewReadFileTool(root))
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{
{
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "call-1", ToolName: "read_file"},
{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: "call-1", ArgumentsFragment: `{"path":"notes.txt"}`},
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "call-1"},
{Type: zeroruntime.StreamEventDone},
},
{
{Type: zeroruntime.StreamEventText, Content: "done"},
{Type: zeroruntime.StreamEventDone},
},
}}
if _, err := Run(context.Background(), "read notes", provider, Options{
Cwd: root,
Registry: registry,
SessionID: "session-stable-prefix",
}); err != nil {
t.Fatal(err)
}
if len(provider.requests) != 2 {
t.Fatalf("provider requests = %d, want 2", len(provider.requests))
}
first, second := provider.requests[0], provider.requests[1]
if len(second.Messages) < len(first.Messages) || !reflect.DeepEqual(second.Messages[:len(first.Messages)], first.Messages) {
t.Fatalf("first request messages must be an exact prefix of the second:\nfirst=%#v\nsecond=%#v", first.Messages, second.Messages)
}
if !reflect.DeepEqual(first.Tools, second.Tools) {
t.Fatalf("tool definitions drifted between turns:\nfirst=%#v\nsecond=%#v", first.Tools, second.Tools)
}
if first.PromptCacheKey != "session-stable-prefix" || second.PromptCacheKey != first.PromptCacheKey {
t.Fatalf("prompt cache key must remain stable: first=%q second=%q", first.PromptCacheKey, second.PromptCacheKey)
}
}
func TestRunSanitizesMalformedToolCallArgumentsBeforeRetry(t *testing.T) {
root := t.TempDir()
registry := tools.NewRegistry()
@@ -1,167 +0,0 @@
package agent
import (
"context"
"os"
"strconv"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/hooks"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
type propagationOutputTool struct {
output string
}
func TestOutputBudgetHookHelperProcess(t *testing.T) {
for index, arg := range os.Args {
if arg != "--zero-output-budget-hook" || index+1 >= len(os.Args) {
continue
}
if _, err := os.Stdout.WriteString(os.Args[index+1]); err != nil {
os.Exit(2)
}
os.Exit(0)
}
}
func largeOutputBudgetHookDispatcher() *hooks.Dispatcher {
feedback := strings.Repeat("hook feedback ", 200)
return hooks.NewDispatcher(hooks.DispatcherOptions{Config: hooks.Config{
Enabled: true,
Hooks: []hooks.Definition{{
ID: "large-feedback",
Event: hooks.EventAfterTool,
Matcher: "propagation_output",
Command: os.Args[0],
Args: []string{
"-test.run=TestOutputBudgetHookHelperProcess",
"--",
"--zero-output-budget-hook",
feedback,
},
Enabled: true,
}},
}})
}
func (tool propagationOutputTool) Name() string { return "propagation_output" }
func (tool propagationOutputTool) Description() string { return "returns output for propagation tests" }
func (tool propagationOutputTool) Parameters() tools.Schema {
return tools.Schema{Type: "object", AdditionalProperties: false}
}
func (tool propagationOutputTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow, Reason: "test read"}
}
func (tool propagationOutputTool) Run(context.Context, map[string]any) tools.Result {
return tools.Result{Status: tools.StatusOK, Output: tool.output}
}
func TestExecuteToolCallPropagatesOutputTruncation(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
t.Setenv("ZERO_TOOL_OUTPUT_CEILING_TOKENS", "80")
registry := tools.NewRegistry()
registry.Register(propagationOutputTool{output: strings.Repeat("large output\n", 1000)})
result, abortErr := executeToolCall(context.Background(), registry, ToolCall{
ID: "call-budget",
Name: "propagation_output",
Arguments: `{}`,
}, PermissionModeAuto, Options{Cwd: t.TempDir()})
if abortErr != nil {
t.Fatalf("executeToolCall abort error: %v", abortErr)
}
if !result.Truncated {
t.Fatalf("agent ToolResult lost tools.Result.Truncated: %#v", result)
}
if result.Meta["spill_path"] == "" {
t.Fatalf("agent ToolResult lost spill metadata: %#v", result.Meta)
}
}
func TestRecordOutputBudgetTraceUsesOnlyCompactMetadata(t *testing.T) {
recorder := trace.NewRecorder("session", "run", "")
recorder.Start()
recordOutputBudgetTrace(recorder, ToolResult{
Name: "grep",
Truncated: true,
Output: "SECRET OUTPUT MUST NOT ENTER TRACE",
Meta: map[string]string{
"output_budget_category": "search",
"output_budget_original_bytes": "1000",
"output_budget_retained_bytes": "100",
"output_budget_estimated_original_tokens": "250",
"output_budget_estimated_retained_tokens": "25",
"output_budget_reason": "semantic_search_budget",
"output_budget_spill_created": "true",
"spill_path": "/secret/path/not-for-trace",
},
})
events := recorder.Finish().OutputBudgets
if len(events) != 1 {
t.Fatalf("events = %#v", events)
}
event := events[0]
if event.Tool != "grep" || event.Category != "search" || event.OriginalBytes != 1000 || event.RetainedBytes != 100 || !event.SpillCreated {
t.Fatalf("unexpected trace event: %#v", event)
}
}
func TestExecuteToolCallRebudgetsOversizedAfterToolFeedback(t *testing.T) {
t.Setenv("ZERO_TOOL_OUTPUT_CEILING_TOKENS", "80")
registry := tools.NewRegistry()
registry.Register(propagationOutputTool{output: "tool output"})
dispatcher := largeOutputBudgetHookDispatcher()
result, abortErr := executeToolCall(context.Background(), registry, ToolCall{
ID: "call-hook-budget",
Name: "propagation_output",
Arguments: `{}`,
}, PermissionModeAuto, Options{Hooks: dispatcher})
if abortErr != nil {
t.Fatalf("executeToolCall abort error: %v", abortErr)
}
if !result.Truncated || len(result.Output) > 80*4 {
t.Fatalf("afterTool feedback bypassed output budget: truncated=%t bytes=%d meta=%#v", result.Truncated, len(result.Output), result.Meta)
}
if result.Meta["output_budget_category"] == "" || result.Meta["output_budget_retained_bytes"] != strconv.Itoa(len(result.Output)) {
t.Fatalf("post-hook budget metadata does not describe final output: %#v", result.Meta)
}
}
func TestRunTraceReflectsPostHookBudget(t *testing.T) {
t.Setenv("ZERO_TOOL_OUTPUT_CEILING_TOKENS", "80")
registry := tools.NewRegistry()
registry.Register(propagationOutputTool{output: "tool output"})
dispatcher := largeOutputBudgetHookDispatcher()
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{
{
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "call-hook-trace", ToolName: "propagation_output"},
{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: "call-hook-trace", ArgumentsFragment: `{}`},
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "call-hook-trace"},
{Type: zeroruntime.StreamEventDone},
},
{{Type: zeroruntime.StreamEventText, Content: "done"}, {Type: zeroruntime.StreamEventDone}},
}}
recorder := trace.NewRecorder("session", "run", "")
var toolResults []ToolResult
if _, err := Run(context.Background(), "budget hook", provider, Options{
Registry: registry,
Hooks: dispatcher,
Trace: recorder,
OnToolResult: func(result ToolResult) { toolResults = append(toolResults, result) },
}); err != nil {
t.Fatalf("Run: %v", err)
}
if len(toolResults) != 1 || !toolResults[0].Truncated {
t.Fatalf("tool result = %#v, want one truncated post-hook result", toolResults)
}
events := recorder.Finish().OutputBudgets
if len(events) != 1 || !events[0].Truncated || events[0].RetainedBytes != len(toolResults[0].Output) {
t.Fatalf("trace does not describe final post-hook output: events=%#v result=%#v", events, toolResults[0])
}
}
-118
View File
@@ -1,118 +0,0 @@
package agent
import (
"github.com/Gitlawb/zero/internal/sandbox"
)
// profileController observes per-turn run signals and decides at most one
// posture escalation per run for the armed execution profile. A nil policy (or
// a policy without Escalate) makes every method a no-op, so runs without a
// profile are byte-identical to before — the same opt-in convention as Trace
// and SelfCorrect.
//
// The controller only ever CHANGES loop policy knobs (turn ceiling, reasoning
// effort, completion gate) through the values the loop applies at its act
// point; it never injects messages, never touches the model or session, and
// never interacts with the model-initiated escalate_model path.
type profileController struct {
policy *PostureEscalation
escalated bool
uncertainSeen int
failureTripped bool
riskTripped bool
scTripped bool
uncertainTrip bool
}
// newProfileController is nil-safe: a nil ProfilePolicy (the default for every
// existing caller) or a nil Escalate yields a controller that observes and
// decides nothing.
func newProfileController(policy *ProfilePolicy) *profileController {
if policy == nil {
return &profileController{}
}
return &profileController{policy: policy.Escalate}
}
// observeToolOutcome watches two signals from an executed tool call: the
// repeated-failure guard's streak (reusing the guard's own counting, not a
// second failure model) and the sandbox risk classification of an executed
// result.
func (c *profileController) observeToolOutcome(outcome toolFailureOutcome, result ToolResult) {
if c.policy == nil || c.escalated {
return
}
if c.policy.OnToolFailureStreak > 0 && outcome.Count >= c.policy.OnToolFailureStreak {
c.failureTripped = true
}
if threshold := riskRank(c.policy.OnRiskyMutation); threshold > 0 &&
result.DenialReason == DenialNone &&
riskRank(result.Risk.Level) >= threshold {
// The call EXECUTED (was not denied): partial failures count too, since
// the mutation ran. An unrecognized threshold ranks 0 and disables the
// signal instead of matching everything.
c.riskTripped = true
}
}
// observeUncertain counts uncertain completion evaluations (continue nudges and
// semantic checks). Headless-only by construction: the completion gate never
// runs interactively.
func (c *profileController) observeUncertain() {
if c.policy == nil || c.escalated || c.policy.OnCompletionUncertain <= 0 {
return
}
c.uncertainSeen++
if c.uncertainSeen >= c.policy.OnCompletionUncertain {
c.uncertainTrip = true
}
}
// observeSelfCorrect watches the post-edit verification outcome. Any failing
// outcome (correcting, reported, aborted) is a signal; passed and disabled are
// not.
func (c *profileController) observeSelfCorrect(outcome Outcome) {
if c.policy == nil || c.escalated || !c.policy.OnSelfCorrectFailure {
return
}
switch outcome {
case OutcomeCorrecting, OutcomeReported, OutcomeAborted:
c.scTripped = true
}
}
// maybeEscalate reports the escalation target exactly once, the first time any
// armed trigger has fired. Subsequent calls return false for the rest of the
// run: escalation is one-shot and never de-escalates, so no cooldown state is
// needed.
func (c *profileController) maybeEscalate() (PostureEscalation, bool) {
if c.policy == nil || c.escalated {
return PostureEscalation{}, false
}
if !(c.failureTripped || c.riskTripped || c.scTripped || c.uncertainTrip) {
return PostureEscalation{}, false
}
c.escalated = true
return *c.policy, true
}
// riskRank orders sandbox risk levels for threshold comparison. Unknown levels
// rank 0: an unrecognized RESULT level can never meet a valid threshold, and an
// unrecognized THRESHOLD is rejected before comparison (rank 0 disables the
// signal) so a typo in a profile can never make every result match.
func riskRank(level sandbox.RiskLevel) int {
switch level {
case sandbox.RiskLow:
return 1
case sandbox.RiskMedium:
return 2
case sandbox.RiskHigh:
return 3
case sandbox.RiskCritical:
return 4
default:
return 0
}
}
-345
View File
@@ -1,345 +0,0 @@
package agent
import (
"context"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/sandbox"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
func TestProfileControllerNilPolicyIsNoOp(t *testing.T) {
for _, controller := range []*profileController{
newProfileController(nil),
newProfileController(&ProfilePolicy{Name: "balanced"}), // Escalate nil
} {
controller.observeToolOutcome(toolFailureOutcome{Count: 99}, ToolResult{Status: tools.StatusOK, Risk: sandbox.Risk{Level: sandbox.RiskCritical}})
controller.observeUncertain()
controller.observeSelfCorrect(OutcomeAborted)
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("nil policy must never escalate")
}
}
}
func TestProfileControllerEscalatesOnFailureStreak(t *testing.T) {
controller := newProfileController(&ProfilePolicy{Escalate: &PostureEscalation{OnToolFailureStreak: 2, MaxTurns: 80}})
controller.observeToolOutcome(toolFailureOutcome{Count: 1}, ToolResult{Status: tools.StatusError})
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("streak below threshold must not escalate")
}
controller.observeToolOutcome(toolFailureOutcome{Count: 2}, ToolResult{Status: tools.StatusError})
target, fired := controller.maybeEscalate()
if !fired || target.MaxTurns != 80 {
t.Fatalf("expected escalation with the policy target, got fired=%t target=%+v", fired, target)
}
}
func TestProfileControllerEscalatesOnRiskyMutation(t *testing.T) {
controller := newProfileController(&ProfilePolicy{Escalate: &PostureEscalation{OnRiskyMutation: sandbox.RiskHigh}})
// Medium risk on an executed result: below threshold.
controller.observeToolOutcome(toolFailureOutcome{}, ToolResult{Status: tools.StatusOK, Risk: sandbox.Risk{Level: sandbox.RiskMedium}})
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("medium risk must not trip a high threshold")
}
// Critical risk on a DENIED result: never counts (the mutation never ran).
controller.observeToolOutcome(toolFailureOutcome{}, ToolResult{Status: tools.StatusError, DenialReason: DenialSandboxBlock, Risk: sandbox.Risk{Level: sandbox.RiskCritical}})
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("a denied result must not trip the risky-mutation signal")
}
// Critical risk on an executed PARTIAL FAILURE (error status, no denial):
// the mutation ran, so it counts.
controller.observeToolOutcome(toolFailureOutcome{}, ToolResult{Status: tools.StatusError, Risk: sandbox.Risk{Level: sandbox.RiskCritical}})
if _, fired := controller.maybeEscalate(); !fired {
t.Fatal("an executed partial failure at critical risk must trip a high threshold")
}
}
func TestProfileControllerIgnoresInvalidRiskThreshold(t *testing.T) {
controller := newProfileController(&ProfilePolicy{Escalate: &PostureEscalation{OnRiskyMutation: sandbox.RiskLevel("hgih")}})
controller.observeToolOutcome(toolFailureOutcome{}, ToolResult{Status: tools.StatusOK, Risk: sandbox.Risk{Level: sandbox.RiskCritical}})
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("an unrecognized threshold must disable the signal, not match every result")
}
}
func TestProfileControllerEscalatesOnSelfCorrectFailure(t *testing.T) {
controller := newProfileController(&ProfilePolicy{Escalate: &PostureEscalation{OnSelfCorrectFailure: true}})
controller.observeSelfCorrect(OutcomePassed)
controller.observeSelfCorrect(OutcomeDisabled)
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("passing/disabled outcomes must not escalate")
}
controller.observeSelfCorrect(OutcomeReported)
if _, fired := controller.maybeEscalate(); !fired {
t.Fatal("a failing verification outcome must escalate")
}
}
func TestProfileControllerEscalatesOnUncertainCompletion(t *testing.T) {
controller := newProfileController(&ProfilePolicy{Escalate: &PostureEscalation{OnCompletionUncertain: 2}})
controller.observeUncertain()
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("first uncertain evaluation must not trip a threshold of 2")
}
controller.observeUncertain()
if _, fired := controller.maybeEscalate(); !fired {
t.Fatal("second uncertain evaluation must escalate")
}
}
func TestProfileControllerEscalatesAtMostOnce(t *testing.T) {
controller := newProfileController(&ProfilePolicy{Escalate: &PostureEscalation{OnToolFailureStreak: 1}})
controller.observeToolOutcome(toolFailureOutcome{Count: 1}, ToolResult{Status: tools.StatusError})
if _, fired := controller.maybeEscalate(); !fired {
t.Fatal("expected first escalation")
}
controller.observeToolOutcome(toolFailureOutcome{Count: 5}, ToolResult{Status: tools.StatusOK, Risk: sandbox.Risk{Level: sandbox.RiskCritical}})
controller.observeUncertain()
if _, fired := controller.maybeEscalate(); fired {
t.Fatal("escalation must be one-shot per run")
}
}
// TestPostureEscalationOnUncertainCompletion proves the uncertain-path act
// point: the completion gate's continue branch escapes the end-of-turn tail,
// so escalation must apply before the continue. A cue turn under the gate
// counts as uncertain, escalates, and the very next request carries the target
// effort.
func TestPostureEscalationOnUncertainCompletion(t *testing.T) {
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{
{
{Type: zeroruntime.StreamEventText, Content: "Let me read the file:"},
{Type: zeroruntime.StreamEventDone},
},
{
{Type: zeroruntime.StreamEventText, Content: "Done. All set."},
{Type: zeroruntime.StreamEventDone},
},
}}
recorder := trace.NewRecorder("posture-session", "run-2", "fast")
result, err := Run(context.Background(), "go", provider, Options{
Registry: tools.NewRegistry(),
MaxTurns: 10,
RequireCompletionSignal: true,
Trace: recorder,
Profile: &ProfilePolicy{
Name: "fast",
Escalate: &PostureEscalation{
ReasoningEffort: "high",
OnCompletionUncertain: 1,
},
},
})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "Done. All set." {
t.Fatalf("expected the completed answer, got %q", result.FinalAnswer)
}
if len(provider.requests) != 2 {
t.Fatalf("expected 2 requests, got %d", len(provider.requests))
}
if provider.requests[1].ReasoningEffort != "high" {
t.Fatalf("expected the post-escalation request to carry the target effort, got %q", provider.requests[1].ReasoningEffort)
}
tr := recorder.Finish()
if got := tr.Counter(trace.CounterPostureEscalations); got != 1 {
t.Fatalf("posture_escalations = %d, want 1", got)
}
}
// A profile that filled the run's effort displaced the provider default ("");
// RestoreDefaultEffort lets escalation restore that default, which a plain ""
// ReasoningEffort target cannot express (it means "leave untouched").
func TestPostureEscalationRestoresDefaultEffort(t *testing.T) {
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{
{
{Type: zeroruntime.StreamEventText, Content: "Let me read the file:"},
{Type: zeroruntime.StreamEventDone},
},
{
{Type: zeroruntime.StreamEventText, Content: "Done. All set."},
{Type: zeroruntime.StreamEventDone},
},
}}
result, err := Run(context.Background(), "go", provider, Options{
Registry: tools.NewRegistry(),
MaxTurns: 10,
ReasoningEffort: "low",
RequireCompletionSignal: true,
Profile: &ProfilePolicy{
Name: "fast",
Escalate: &PostureEscalation{
RestoreDefaultEffort: true,
OnCompletionUncertain: 1,
},
},
})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "Done. All set." {
t.Fatalf("expected the completed answer, got %q", result.FinalAnswer)
}
if len(provider.requests) != 2 {
t.Fatalf("expected 2 requests, got %d", len(provider.requests))
}
if provider.requests[0].ReasoningEffort != "low" {
t.Fatalf("pre-escalation request effort = %q, want the profile's low", provider.requests[0].ReasoningEffort)
}
if provider.requests[1].ReasoningEffort != "" {
t.Fatalf("post-escalation request effort = %q, want the restored provider default (empty)", provider.requests[1].ReasoningEffort)
}
}
// failingProfileTool always errors with the same signature so the repeated-
// failure guard builds a streak.
type failingProfileTool struct{}
func (failingProfileTool) Name() string { return "flaky_probe" }
func (failingProfileTool) Description() string { return "always fails identically (test)" }
func (failingProfileTool) Parameters() tools.Schema {
return tools.Schema{Type: "object", AdditionalProperties: false}
}
func (failingProfileTool) Safety() tools.Safety {
return tools.Safety{SideEffect: tools.SideEffectRead, Permission: tools.PermissionAllow}
}
func (failingProfileTool) Run(context.Context, map[string]any) tools.Result {
return tools.Result{Status: tools.StatusError, Output: "probe exploded: same signature"}
}
// TestPostureEscalationRaisesTurnCeilingMidRun proves the end-to-end act path:
// a Fast-style run with a 2-turn ceiling hits a failure streak, escalates, and
// finishes on a turn the original ceiling would have cut off, with the effort
// target applied to the post-escalation request and the counter emitted.
func TestPostureEscalationRaisesTurnCeilingMidRun(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(failingProfileTool{})
toolTurn := []zeroruntime.StreamEvent{
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "c1", ToolName: "flaky_probe"},
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "c1"},
{Type: zeroruntime.StreamEventDone},
}
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{
toolTurn,
toolTurn,
{
{Type: zeroruntime.StreamEventText, Content: "recovered after escalation"},
{Type: zeroruntime.StreamEventDone},
},
}}
recorder := trace.NewRecorder("posture-session", "run-1", "fast")
result, err := Run(context.Background(), "go", provider, Options{
Registry: registry,
MaxTurns: 2,
Trace: recorder,
Profile: &ProfilePolicy{
Name: "fast",
Escalate: &PostureEscalation{
MaxTurns: 4,
ReasoningEffort: "high",
OnToolFailureStreak: 2,
},
},
})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "recovered after escalation" {
t.Fatalf("expected the run to continue past the original 2-turn ceiling, got %q (turns=%d)", result.FinalAnswer, result.Turns)
}
if len(provider.requests) != 3 {
t.Fatalf("expected 3 requests (ceiling raised from 2 to 4), got %d", len(provider.requests))
}
if provider.requests[2].ReasoningEffort != "high" {
t.Fatalf("expected the post-escalation request to carry the target effort, got %q", provider.requests[2].ReasoningEffort)
}
tr := recorder.Finish()
if got := tr.Counter(trace.CounterPostureEscalations); got != 1 {
t.Fatalf("posture_escalations = %d, want 1", got)
}
}
// TestPostureEscalationAbsentWithoutProfile pins the no-regression invariant at
// the loop level: the identical failing script without a profile ends at the
// original ceiling with the max-turns answer.
func TestPostureEscalationAbsentWithoutProfile(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(failingProfileTool{})
toolTurn := []zeroruntime.StreamEvent{
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "c1", ToolName: "flaky_probe"},
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "c1"},
{Type: zeroruntime.StreamEventDone},
}
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{toolTurn, toolTurn}}
result, err := Run(context.Background(), "go", provider, Options{
Registry: registry,
MaxTurns: 2,
})
if err != nil {
t.Fatal(err)
}
if len(provider.requests) > 3 {
t.Fatalf("unprofiled run must not extend the ceiling, got %d requests", len(provider.requests))
}
if !strings.Contains(result.FinalAnswer, "maximum number of turns") {
t.Fatalf("unprofiled run must end with the max-turns answer, got %q", result.FinalAnswer)
}
}
// TestExecuteToolCallClassifiesRiskWithoutSandbox verifies the executed-risk
// stamp on the unsandboxed path: a read-only tool classifies low.
func TestExecuteToolCallClassifiesRiskWithoutSandbox(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(failingProfileTool{})
result, abortErr := executeToolCall(
context.Background(),
registry,
ToolCall{ID: "c1", Name: "flaky_probe"},
PermissionModeAuto,
Options{},
)
if abortErr != nil {
t.Fatal(abortErr)
}
if result.Risk.Level == "" {
t.Fatal("executed result must carry a risk classification")
}
if riskRank(result.Risk.Level) > riskRank(sandbox.RiskMedium) {
t.Fatalf("read-only tool classified %q, want at most medium", result.Risk.Level)
}
}
// TestUnknownToolResultCarriesZeroRisk verifies a call that never executed a
// tool (unknown name, nil tool) keeps the zero risk value instead of panicking
// or inventing a classification.
func TestUnknownToolResultCarriesZeroRisk(t *testing.T) {
registry := tools.NewRegistry()
result, abortErr := executeToolCall(
context.Background(),
registry,
ToolCall{ID: "c1", Name: "no_such_tool"},
PermissionModeAuto,
Options{},
)
if abortErr != nil {
t.Fatal(abortErr)
}
if result.Status == tools.StatusOK {
t.Fatal("expected an error result for an unknown tool")
}
if result.Risk.Level != "" {
t.Fatalf("not-executed result must keep the zero risk value, got %q", result.Risk.Level)
}
}
+98 -42
View File
@@ -11,11 +11,12 @@ import (
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// promptSubstrings are the cacheable components of the prompt a run sends to a
// model. systemPrompt is the exact joined system-message content. The remaining
// fields retain useful diagnostic boundaries within that prompt and tool list.
// promptSubstrings are the seven cacheable sub-components of the prompt a run
// sends to a model. Each is the literal content the sub-component would emit
// into the system prompt or tool list, hashed so a downstream observer can
// detect drift turn-over-turn. The substrings are produced by
// buildPromptSubstrings and consumed by ComputePrefixFingerprint.
type promptSubstrings struct {
systemPrompt string
baseInstructions string
confirmationPolicy string
projectContext string
@@ -24,55 +25,92 @@ type promptSubstrings struct {
schema string
}
// buildPromptSubstrings builds the prompt and retains both its exact joined
// form and narrower diagnostic components. Callers that already built the
// prompt should use buildPromptSubstringsFromParts to avoid repeating workspace
// reads.
// buildPromptSubstrings assembles the seven cacheable sub-components of the
// prompt without joining them. The system prompt sections are produced by the
// same private builders that buildSystemPrompt uses, so the substrings are
// byte-identical to what would have appeared in the joined prompt. The tools
// and schema substrings are derived from the partitioned tool list (caller-
// provided) so this helper does not need to re-run the partition.
//
// This is the seam ComputePrefixFingerprint reads from. It exists as a
// separate function (rather than returning both prompt and substrings from
// buildSystemPrompt) so existing callers of buildSystemPrompt are unaffected
// and the substrings helper is independently testable.
func buildPromptSubstrings(options Options, exposed []zeroruntime.ToolDefinition) promptSubstrings {
return buildPromptSubstringsFromParts(buildSystemPromptParts(options), exposed)
}
core := strings.TrimSpace(options.SystemPrompt)
if core == "" {
core = strings.TrimSpace(coreSystemPrompt)
}
if core == "" {
core = fallbackSystemPrompt
}
func buildPromptSubstringsFromParts(parts systemPromptParts, exposed []zeroruntime.ToolDefinition) promptSubstrings {
// The confirmation policy is appended unconditionally in buildSystemPrompt;
// the substring here is the same constant.
policy := strings.TrimSpace(confirmationPolicy)
// projectContext is the joined output of workspaceContext, which itself
// walks the AGENTS.md / ZERO.md / .zero/AGENTS.md chain. Hashing the
// joined string captures every guideline file's contribution and the
// ordering between them.
project := workspaceContext(options.Cwd)
// skills is the joined output of skillsContext, with each skill on its own
// line. The substring is the literal text the model would see.
skills := skillsContext(options)
// tools and schema are derived from the partitioned tool list. The list
// passed in is already in stable order (partitionToolsCached is the
// canonical stable partitioner) so concatenating in slice order is
// sufficient — we sort names again defensively in case a future caller
// hands us a list from a different partitioner.
toolsSubstr, schemaSubstr := toolSubstrings(exposed)
return promptSubstrings{
systemPrompt: parts.prompt,
baseInstructions: parts.baseInstructions,
confirmationPolicy: parts.confirmationPolicy,
projectContext: parts.projectContext,
skills: parts.skills,
baseInstructions: core,
confirmationPolicy: policy,
projectContext: project,
skills: skills,
tools: toolsSubstr,
schema: schemaSubstr,
}
}
// toolSubstrings preserves the emitted tool order. Names and descriptions are
// fingerprinted separately from schemas so either kind of drift is identifiable.
// Order is part of request identity: silently sorting here would hide a provider-
// visible reordering and could report a false cache hit.
// toolSubstrings extracts a stable, name-sorted digest of the partitioned tool
// list. The tools substring is "name1\nname2\n..." (sorted) so a reordering of
// the partition produces a different hash. The schema substring is the
// concatenation of each tool's JSON schema in name order, so a schema change
// is detected independently of the tool ordering.
func toolSubstrings(exposed []zeroruntime.ToolDefinition) (toolsSubstr, schemaSubstr string) {
if len(exposed) == 0 {
return "", ""
}
var toolsSB, schemaSB strings.Builder
names := make([]string, 0, len(exposed))
byName := make(map[string]zeroruntime.ToolDefinition, len(exposed))
for _, def := range exposed {
writeLengthPrefixed(&toolsSB, def.Name)
writeLengthPrefixed(&toolsSB, def.Description)
names = append(names, def.Name)
byName[def.Name] = def
}
sort.Strings(names)
var toolsSB, schemaSB strings.Builder
for i, name := range names {
if i > 0 {
toolsSB.WriteByte('\n')
}
toolsSB.WriteString(name)
def := byName[name]
// Parameters is rendered to a canonical JSON string per tool. The
// schema-render cache in internal/agent (used by partitionToolsCached)
// guarantees the same render is byte-identical across turns for the
// same tool, so this substring is a stable hash input.
writeLengthPrefixed(&schemaSB, def.Name)
writeLengthPrefixed(&schemaSB, canonicalSchemaString(def.Parameters))
schemaSB.WriteString(name)
schemaSB.WriteByte('\n')
schemaSB.WriteString(canonicalSchemaString(def.Parameters))
schemaSB.WriteByte('\n')
}
return toolsSB.String(), schemaSB.String()
}
func writeLengthPrefixed(sb *strings.Builder, value string) {
fmt.Fprintf(sb, "%d:", len(value))
sb.WriteString(value)
}
// canonicalSchemaString renders a tool's parameter schema to a stable string.
// tools.ToolDef.Parameters is a map[string]any in the common case; Go's
// fmt.Sprintf("%v", m) iterates a map in random order, which would produce
@@ -189,16 +227,36 @@ func writeStable(sb *strings.Builder, v any) {
}
}
// ComputePrefixFingerprint builds and fingerprints a prompt for callers that do
// not already have one. The agent loop uses computePrefixFingerprint with the
// prompt parts it built once at run start, ensuring tracing observes the exact
// request content without repeating workspace I/O.
func ComputePrefixFingerprint(options Options, exposed []zeroruntime.ToolDefinition) prefixFingerprint {
return computePrefixFingerprint(buildPromptSubstrings(options, exposed))
}
// ComputePrefixFingerprint returns a trace.PrefixHash (a 7-field fingerprint
// of the prompt prefix) for one turn of a run. The seven sub-hashes are
// independent SHA-256s of the corresponding sub-component; the complete-prefix
// hash is a SHA-256 of the canonical concatenation of the other six, so any
// sub-component drift is observable both individually and in aggregate.
//
// I/O side effects: buildPromptSubstrings calls workspaceContext, which runs
// git (gitBranchForPrompt, FindProjectGitRoot) and reads the AGENTS.md /
// ZERO.md / .zero/AGENTS.md chain plus the repo map. The same workspace-
// context work is performed separately by buildSystemPrompt a few lines
// later in the request-build path, so on every traced turn the file and
// git reads happen twice. The duplication is intentional for this PR (a
// follow-up will pass substrings through) and is a known perf cost only on
// the opt-in trace path.
//
// A stable CompletePrefixHash across turns means the four captured
// sub-components (baseInstructions, confirmationPolicy, projectContext,
// skills) are byte-identical. It does NOT rule out drift in the seven
// uncaptured sections of buildSystemPrompt (modelPromptAddendum,
// sessionRuntimeContext, approvedCommandPrefixContext, workspaceSeedContext,
// userGuidelines, specialistDelegationContext, responseStyleContext). For
// default Options the four captured substrings are the full prompt; for
// non-default Options the seven uncaptured sections may contribute, and a
// consumer correlating CompletePrefix stability with cached_input_tokens
// must cross-check the model_switches counter (modelPromptAddendum changes
// on a model switch) to disambiguate "no drift" from "drift in an
// uncaptured section."
func computePrefixFingerprint(subs promptSubstrings) prefixFingerprint {
systemPrompt := sha256hex(subs.systemPrompt)
func ComputePrefixFingerprint(options Options, exposed []zeroruntime.ToolDefinition) prefixFingerprint {
subs := buildPromptSubstrings(options, exposed)
base := sha256hex(subs.baseInstructions)
policy := sha256hex(subs.confirmationPolicy)
project := sha256hex(subs.projectContext)
@@ -206,10 +264,9 @@ func computePrefixFingerprint(subs promptSubstrings) prefixFingerprint {
toolsH := sha256hex(subs.tools)
schema := sha256hex(subs.schema)
complete := sha256hex(strings.Join([]string{
systemPrompt, toolsH, schema,
base, policy, project, skills, toolsH, schema,
}, "|"))
return prefixFingerprint{
SystemPromptHash: systemPrompt,
BaseInstructionsHash: base,
ConfirmationPolicyHash: policy,
ProjectContextHash: project,
@@ -225,7 +282,6 @@ func computePrefixFingerprint(subs promptSubstrings) prefixFingerprint {
// in loop.go) so the trace package does not need to import this one. The field
// names match the trace.PrefixHash JSON tags 1:1.
type prefixFingerprint struct {
SystemPromptHash string
BaseInstructionsHash string
ConfirmationPolicyHash string
ProjectContextHash string
+31 -37
View File
@@ -28,28 +28,38 @@ func TestComputePrefixFingerprintStableAcrossCalls(t *testing.T) {
}
}
// Tool order and descriptions are provider-visible and therefore part of the
// cacheable prefix identity.
func TestComputePrefixFingerprintToolsHashMatchesEmittedOrderAndDescriptions(t *testing.T) {
// TestComputePrefixFingerprintToolsHashIsNameSetSensitive asserts the
// name-set sensitivity of the ToolsHash. The test name used to read as
// "reorder changes hash" but the assertion is the opposite: the hash is
// name-set sensitive, not name-order sensitive (toolSubstrings sorts
// names before hashing). The partitioner is expected to produce a stable
// order, so a name-order change is not a hash signal — a name-set change
// is. The test asserts both: permutation of the same tools produces the
// same hash, and adding a tool produces a different hash.
func TestComputePrefixFingerprintToolsHashIsNameSetSensitive(t *testing.T) {
opts := Options{Cwd: t.TempDir(), SystemPrompt: "core"}
a := []zeroruntime.ToolDefinition{
{Name: "read_file", Description: "read", Parameters: map[string]any{"schema": "schema-read"}},
{Name: "grep", Description: "search", Parameters: map[string]any{"schema": "schema-grep"}},
{Name: "read_file", Parameters: map[string]any{"schema": "schema-read"}},
{Name: "grep", Parameters: map[string]any{"schema": "schema-grep"}},
}
b := []zeroruntime.ToolDefinition{
{Name: "grep", Description: "search", Parameters: map[string]any{"schema": "schema-grep"}},
{Name: "read_file", Description: "read", Parameters: map[string]any{"schema": "schema-read"}},
{Name: "grep", Parameters: map[string]any{"schema": "schema-grep"}},
{Name: "read_file", Parameters: map[string]any{"schema": "schema-read"}},
}
// toolSubstrings sorts names internally, so the ToolsHash is identical
// for a permutation of the same tools. We assert that here to document
// the contract: name-order independence, name-set sensitivity.
fpA := ComputePrefixFingerprint(opts, a)
fpB := ComputePrefixFingerprint(opts, b)
if fpA.ToolsHash == fpB.ToolsHash {
t.Fatalf("ToolsHash must change when emitted order changes: a=%s b=%s", fpA.ToolsHash, fpB.ToolsHash)
if fpA.ToolsHash != fpB.ToolsHash {
t.Fatalf("ToolsHash must be name-set sensitive, not name-order sensitive: a=%s b=%s", fpA.ToolsHash, fpB.ToolsHash)
}
c := append([]zeroruntime.ToolDefinition(nil), a...)
c[0].Description = "read a file"
// A different tool set must produce a different ToolsHash.
c := append([]zeroruntime.ToolDefinition{}, a...)
c = append(c, zeroruntime.ToolDefinition{Name: "bash", Parameters: map[string]any{"schema": "schema-bash"}})
fpC := ComputePrefixFingerprint(opts, c)
if fpA.ToolsHash == fpC.ToolsHash {
t.Fatalf("ToolsHash must change when a description changes: a=%s c=%s", fpA.ToolsHash, fpC.ToolsHash)
t.Fatalf("ToolsHash must change when the tool set changes: a=%s c=%s", fpA.ToolsHash, fpC.ToolsHash)
}
}
@@ -91,35 +101,25 @@ func TestComputePrefixFingerprintSystemPromptChangeChangesBaseHash(t *testing.T)
}
}
// CompletePrefixHash represents the exact joined system prompt and the ordered
// tool definitions. Narrower system-section hashes remain diagnostic only.
// TestComputePrefixFingerprintCompleteIsAggregateOfSubHashes asserts the
// canonical-join property: CompletePrefixHash must depend on every sub-hash.
// This is the regression catch for "did anyone reorder or drop a sub-hash
// from the canonical join without updating CompletePrefixHash."
func TestComputePrefixFingerprintCompleteIsAggregateOfSubHashes(t *testing.T) {
opts := Options{Cwd: t.TempDir(), SystemPrompt: "core"}
exposed := []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"k": "v"}}}
fp := ComputePrefixFingerprint(opts, exposed)
// Manually compute what the canonical join should be.
expected := sha256hex(strings.Join([]string{
fp.SystemPromptHash,
fp.BaseInstructionsHash,
fp.ConfirmationPolicyHash,
fp.ProjectContextHash,
fp.SkillsHash,
fp.ToolsHash,
fp.SchemaHash,
}, "|"))
if fp.CompletePrefixHash != expected {
t.Fatalf("CompletePrefixHash must equal sha256hex of the exact system, tools, and schema hashes:\n got: %s\n expected: %s", fp.CompletePrefixHash, expected)
}
}
func TestComputePrefixFingerprintCoversEverySystemPromptSection(t *testing.T) {
base := Options{Cwd: t.TempDir(), SystemPrompt: "core", Model: "gpt-5"}
changed := base
changed.ResponseStyle = "concise"
fpBase := ComputePrefixFingerprint(base, nil)
fpChanged := ComputePrefixFingerprint(changed, nil)
if fpBase.SystemPromptHash == fpChanged.SystemPromptHash {
t.Fatal("SystemPromptHash must change when any rendered system section changes")
}
if fpBase.CompletePrefixHash == fpChanged.CompletePrefixHash {
t.Fatal("CompletePrefixHash must include the exact full system prompt")
t.Fatalf("CompletePrefixHash must equal sha256hex of the canonical join of the other six:\n got: %s\n expected: %s", fp.CompletePrefixHash, expected)
}
}
@@ -141,9 +141,6 @@ func TestBuildPromptSubstringsDefaultOptions(t *testing.T) {
if subs.baseInstructions != "test core" {
t.Fatalf("baseInstructions substring must equal the core system prompt: got %q want %q", subs.baseInstructions, "test core")
}
if subs.systemPrompt != buildSystemPrompt(opts) {
t.Fatal("systemPrompt substring must equal the exact joined system prompt")
}
if subs.confirmationPolicy == "" {
t.Fatalf("confirmationPolicy substring must be non-empty (the embedded policy is always present)")
}
@@ -154,9 +151,6 @@ func TestBuildPromptSubstringsDefaultOptions(t *testing.T) {
// sha256hex of the substrings, with no truncation or reformatting
// between the two layers.
fp := ComputePrefixFingerprint(opts, nil)
if fp.SystemPromptHash != sha256hex(subs.systemPrompt) {
t.Fatalf("SystemPromptHash in fingerprint must equal sha256hex of the joined prompt")
}
if fp.BaseInstructionsHash != sha256hex(subs.baseInstructions) {
t.Fatalf("BaseInstructionsHash in fingerprint must equal sha256hex of the substring")
}
+5 -30
View File
@@ -70,22 +70,6 @@ const (
// guidelines), and the safety confirmation policy. It is built once per run so
// every turn shares one (cacheable) system turn.
func buildSystemPrompt(options Options) string {
return buildSystemPromptParts(options).prompt
}
// systemPromptParts retains both the exact joined prompt and the diagnostic
// sections used by prompt-prefix tracing. Building these together prevents the
// tracing path from re-reading workspace state and guarantees that the hash is
// computed from the same prompt bytes placed in the request.
type systemPromptParts struct {
prompt string
baseInstructions string
confirmationPolicy string
projectContext string
skills string
}
func buildSystemPromptParts(options Options) systemPromptParts {
core := strings.TrimSpace(options.SystemPrompt)
if core == "" {
core = strings.TrimSpace(coreSystemPrompt)
@@ -113,31 +97,22 @@ func buildSystemPromptParts(options Options) systemPromptParts {
if user := userGuidelines(); user != "" {
sections = append(sections, user)
}
project := workspaceContext(options.Cwd)
if project != "" {
sections = append(sections, project)
if ws := workspaceContext(options.Cwd); ws != "" {
sections = append(sections, ws)
}
if delegation := specialistDelegationContext(options); delegation != "" {
sections = append(sections, delegation)
}
skillsBlock := skillsContext(options)
if skillsBlock != "" {
if skillsBlock := skillsContext(options); skillsBlock != "" {
sections = append(sections, skillsBlock)
}
if style := responseStyleContext(options); style != "" {
sections = append(sections, style)
}
policy := strings.TrimSpace(confirmationPolicy)
if policy != "" {
if policy := strings.TrimSpace(confirmationPolicy); policy != "" {
sections = append(sections, policy)
}
return systemPromptParts{
prompt: strings.Join(sections, "\n\n"),
baseInstructions: core,
confirmationPolicy: policy,
projectContext: project,
skills: skillsBlock,
}
return strings.Join(sections, "\n\n")
}
// responseStyleContext renders the operator-selected reply style (TUI /style) as
-3
View File
@@ -100,9 +100,6 @@ work.
- Use exec_command with `tty: true` for interactive terminal-style commands that
need stdin beyond Ctrl-C. `/ps` and `/stop` are user-facing TUI commands; when
you need to clean up a running foreground command yourself, use write_stdin.
- write_stdin's session_id is only ever an id returned by a still-running
exec_command; never guess or probe ids. If you have no such session, start one
with exec_command, or use write_file/edit_file/apply_patch for file changes.
- write_stdin with empty input polls an existing exec_command session, and
`\u0003` interrupts it. Sending other stdin bytes may require approval because
it can drive the running process beyond the original command. Non-tty sessions
-359
View File
@@ -1,359 +0,0 @@
package agent
import (
"encoding/json"
"reflect"
"sort"
"strings"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
type taskStatus string
const (
taskStatusActive taskStatus = "active"
taskStatusComplete taskStatus = "complete"
taskStatusIncomplete taskStatus = "incomplete"
)
type taskPlanParity string
const (
taskPlanParityUnknown taskPlanParity = "unknown"
taskPlanParityMatch taskPlanParity = "match"
taskPlanParityMismatch taskPlanParity = "mismatch"
)
type taskPlanItem struct {
Content string `json:"content"`
Status string `json:"status"`
Notes string `json:"notes,omitempty"`
}
type taskPlanState struct {
Items []taskPlanItem `json:"items,omitempty"`
Pending int `json:"pending"`
InProgress int `json:"in_progress"`
Completed int `json:"completed"`
Failed int `json:"failed"`
}
type taskToolState struct {
Succeeded int `json:"succeeded"`
Failed int `json:"failed"`
}
type taskVerificationState struct {
Passed int `json:"passed"`
Failed int `json:"failed"`
LastOutcome Outcome `json:"last_outcome,omitempty"`
}
type taskStateSnapshot struct {
Revision int `json:"revision"`
Objective string `json:"objective"`
Status taskStatus `json:"status"`
Plan taskPlanState `json:"plan"`
Tools taskToolState `json:"tools"`
Verification taskVerificationState `json:"verification"`
ChangedFiles []string `json:"changed_files,omitempty"`
CompletionDecision CompletionDecision `json:"completion_decision,omitempty"`
CompletionReason string `json:"completion_reason,omitempty"`
PlanParity taskPlanParity `json:"plan_parity"`
}
type taskStateEventKind int
const (
taskStateEventPlan taskStateEventKind = iota + 1
taskStateEventToolResult
taskStateEventVerification
taskStateEventCompletion
)
type taskStateEvent struct {
kind taskStateEventKind
arguments string
toolResult ToolResult
verification Outcome
completion completionEvaluation
}
// taskState is a deterministic projection of facts the loop already observes.
// It does not initiate work or replace the transcript; consumers must check
// transcript parity before using its compact representation.
type taskState struct {
snapshotValue taskStateSnapshot
changedFiles map[string]struct{}
recorder *trace.Recorder
planObserved bool
}
func newTaskState(objective string, recorder *trace.Recorder) *taskState {
state := &taskState{
snapshotValue: taskStateSnapshot{
Objective: strings.TrimSpace(objective),
Status: taskStatusActive,
PlanParity: taskPlanParityUnknown,
},
changedFiles: map[string]struct{}{},
}
state.recorder = recorder
return state
}
func (state *taskState) observe(event taskStateEvent) bool {
if state == nil {
return false
}
changed := false
switch event.kind {
case taskStateEventPlan:
plan, ok := parseTaskPlan(event.arguments)
if !ok {
return false
}
state.snapshotValue.Plan = summarizeTaskPlan(plan)
state.planObserved = true
state.markActive()
changed = true
case taskStateEventToolResult:
state.markActive()
if event.toolResult.Status == tools.StatusOK {
state.snapshotValue.Tools.Succeeded++
} else {
state.snapshotValue.Tools.Failed++
}
for _, path := range event.toolResult.ChangedFiles {
path = strings.TrimSpace(path)
if path != "" {
state.changedFiles[path] = struct{}{}
}
}
state.snapshotValue.ChangedFiles = sortedKeys(state.changedFiles)
changed = true
case taskStateEventVerification:
if event.verification == OutcomeDisabled {
return false
}
state.markActive()
state.snapshotValue.Verification.LastOutcome = event.verification
switch event.verification {
case OutcomePassed:
state.snapshotValue.Verification.Passed++
case OutcomeCorrecting, OutcomeReported, OutcomeAborted:
state.snapshotValue.Verification.Failed++
}
changed = true
case taskStateEventCompletion:
state.snapshotValue.CompletionDecision = event.completion.Decision
state.snapshotValue.CompletionReason = event.completion.Reason
switch event.completion.Decision {
case CompletionComplete:
state.snapshotValue.Status = taskStatusComplete
case CompletionIncomplete:
state.snapshotValue.Status = taskStatusIncomplete
default:
state.snapshotValue.Status = taskStatusActive
}
changed = true
}
if changed {
state.snapshotValue.Revision++
if event.kind != taskStateEventToolResult {
state.emit()
}
}
return changed
}
func (state *taskState) markActive() {
state.snapshotValue.Status = taskStatusActive
state.snapshotValue.CompletionDecision = ""
state.snapshotValue.CompletionReason = ""
}
func (state *taskState) snapshot() taskStateSnapshot {
if state == nil {
return taskStateSnapshot{}
}
snapshot := state.snapshotValue
snapshot.Plan.Items = append([]taskPlanItem(nil), state.snapshotValue.Plan.Items...)
snapshot.ChangedFiles = append([]string(nil), state.snapshotValue.ChangedFiles...)
return snapshot
}
// observePlanParity compares only the latest plan projection with the plan tool
// calls still present in messages. It mutates the snapshot and emits when the
// parity value changes; objective, tool, and verification fields are not part of
// this comparison.
func (state *taskState) observePlanParity(messages []zeroruntime.Message) taskPlanParity {
if state == nil {
return taskPlanParityUnknown
}
transcriptPlan, found, valid := latestTaskPlan(messages)
tracked := state.snapshotValue.Plan.Items
parity := taskPlanParityMatch
if !valid || found != state.planObserved || (found && !reflect.DeepEqual(transcriptPlan, tracked)) {
parity = taskPlanParityMismatch
}
if state.snapshotValue.PlanParity != parity {
state.snapshotValue.PlanParity = parity
state.snapshotValue.Revision++
state.emit()
}
return parity
}
type completionContext struct {
Objective string
PlanPending bool
PlanMatchesTranscript bool
}
func (state *taskState) completionContext(messages []zeroruntime.Message, transcriptPlanPending bool) completionContext {
context := completionContext{PlanPending: transcriptPlanPending}
if state == nil {
return context
}
context.Objective = state.snapshotValue.Objective
context.PlanMatchesTranscript = state.observePlanParity(messages) == taskPlanParityMatch
if context.PlanMatchesTranscript {
context.PlanPending = state.snapshotValue.Plan.Pending+state.snapshotValue.Plan.InProgress > 0
}
return context
}
func (state *taskState) snapshotForCompaction(messages []zeroruntime.Message) *taskStateSnapshot {
if state == nil {
return nil
}
state.observePlanParity(messages)
snapshot := state.snapshot()
return &snapshot
}
func (state *taskState) emit() {
if state == nil || state.recorder == nil {
return
}
snapshot := state.snapshotValue
state.recorder.EmitTaskState(trace.TaskStateEvent{
Revision: snapshot.Revision,
Status: string(snapshot.Status),
PlanPending: snapshot.Plan.Pending,
PlanInProgress: snapshot.Plan.InProgress,
PlanCompleted: snapshot.Plan.Completed,
PlanFailed: snapshot.Plan.Failed,
ToolsSucceeded: snapshot.Tools.Succeeded,
ToolsFailed: snapshot.Tools.Failed,
VerificationPassed: snapshot.Verification.Passed,
VerificationFailed: snapshot.Verification.Failed,
VerificationOutcome: string(snapshot.Verification.LastOutcome),
ChangedFileCount: len(snapshot.ChangedFiles),
CompletionDecision: string(snapshot.CompletionDecision),
PlanParity: string(snapshot.PlanParity),
})
}
func parseTaskPlan(arguments string) ([]taskPlanItem, bool) {
var object map[string]json.RawMessage
if json.Unmarshal([]byte(strings.TrimSpace(arguments)), &object) != nil {
return nil, false
}
rawPlan, ok := object["plan"]
if !ok || string(rawPlan) == "null" {
return nil, false
}
var parsed []struct {
ID *string `json:"id"`
Content string `json:"content"`
Status string `json:"status"`
Notes string `json:"notes"`
}
if json.Unmarshal(rawPlan, &parsed) != nil {
return nil, false
}
plan := make([]taskPlanItem, 0, len(parsed))
for _, raw := range parsed {
if raw.Content == "" {
return nil, false
}
plan = append(plan, taskPlanItem{
Content: raw.Content,
Status: normalizeTaskPlanStatus(raw.Status),
Notes: raw.Notes,
})
}
lastInProgress := -1
for index := range plan {
if plan[index].Status == "in_progress" {
if lastInProgress >= 0 {
plan[lastInProgress].Status = "completed"
}
lastInProgress = index
}
}
if len(plan) == 0 {
return nil, true
}
return plan, true
}
func normalizeTaskPlanStatus(status string) string {
switch strings.ToLower(strings.TrimSpace(status)) {
case "completed", "complete", "done", "finished", "resolved", "✓", "x", "[x]":
return "completed"
case "in_progress", "in-progress", "inprogress", "in progress", "active", "doing", "started", "current", "wip", "ongoing", "running":
return "in_progress"
case "failed", "fail", "error", "errored", "blocked", "cancelled", "canceled", "abandoned", "skipped":
return "failed"
default:
return "pending"
}
}
func summarizeTaskPlan(items []taskPlanItem) taskPlanState {
plan := taskPlanState{Items: append([]taskPlanItem(nil), items...)}
for _, item := range items {
switch item.Status {
case "completed":
plan.Completed++
case "in_progress":
plan.InProgress++
case "failed":
plan.Failed++
default:
plan.Pending++
}
}
return plan
}
func latestTaskPlan(messages []zeroruntime.Message) ([]taskPlanItem, bool, bool) {
for i := len(messages) - 1; i >= 0; i-- {
for j := len(messages[i].ToolCalls) - 1; j >= 0; j-- {
call := messages[i].ToolCalls[j]
if call.Name != planToolName {
continue
}
plan, ok := parseTaskPlan(call.Arguments)
return plan, true, ok
}
}
return nil, false, true
}
func sortedKeys(values map[string]struct{}) []string {
keys := make([]string, 0, len(values))
for value := range values {
keys = append(keys, value)
}
// Paths are evidence, not chronology. Stable ordering makes replay snapshots
// deterministic even if events originate from parallel read batches later.
sort.Strings(keys)
return keys
}
-204
View File
@@ -1,204 +0,0 @@
package agent
import (
"context"
"reflect"
"testing"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
func TestTaskStateReplayIsDeterministic(t *testing.T) {
events := []taskStateEvent{
{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"inspect","status":"done"},{"content":"implement","status":"in progress"},{"content":"verify"}]}`},
{kind: taskStateEventToolResult, toolResult: ToolResult{Name: "apply_patch", Status: tools.StatusOK, ChangedFiles: []string{"a.go", "a.go", "b.go"}}},
{kind: taskStateEventToolResult, toolResult: ToolResult{Name: "go test", Status: tools.StatusError}},
{kind: taskStateEventVerification, verification: OutcomePassed},
{kind: taskStateEventCompletion, completion: completionEvaluation{Decision: CompletionUncertain, Reason: "pending work"}},
}
first := newTaskState("ship the change", nil)
second := newTaskState("ship the change", nil)
for _, event := range events {
first.observe(event)
second.observe(event)
}
if !reflect.DeepEqual(first.snapshot(), second.snapshot()) {
t.Fatalf("replaying the same events produced different snapshots:\nfirst: %#v\nsecond: %#v", first.snapshot(), second.snapshot())
}
got := first.snapshot()
if got.Status != taskStatusActive || got.Plan.Pending != 1 || got.Plan.InProgress != 1 || got.Plan.Completed != 1 {
t.Fatalf("unexpected plan snapshot: %#v", got)
}
if got.Tools.Succeeded != 1 || got.Tools.Failed != 1 || len(got.ChangedFiles) != 2 {
t.Fatalf("unexpected evidence snapshot: %#v", got)
}
if got.Verification.Passed != 1 || got.Verification.Failed != 0 || got.Verification.LastOutcome != OutcomePassed {
t.Fatalf("unexpected verification snapshot: %#v", got.Verification)
}
}
func TestTaskStateCoalescesToolResultsIntoNextTraceEvent(t *testing.T) {
recorder := trace.NewRecorder("session", "run", "")
state := newTaskState("objective", recorder)
for range 20 {
state.observe(taskStateEvent{kind: taskStateEventToolResult, toolResult: ToolResult{Status: tools.StatusOK}})
}
if events := recorder.Finish().TaskStates; len(events) != 0 {
t.Fatalf("tool results should be coalesced, got %d trace events", len(events))
}
recorder = trace.NewRecorder("session", "run-2", "")
state = newTaskState("objective", recorder)
for range 20 {
state.observe(taskStateEvent{kind: taskStateEventToolResult, toolResult: ToolResult{Status: tools.StatusOK}})
}
state.observe(taskStateEvent{kind: taskStateEventCompletion, completion: completionEvaluation{Decision: CompletionComplete}})
events := recorder.Finish().TaskStates
if len(events) != 1 || events[0].ToolsSucceeded != 20 {
t.Fatalf("coalesced tool total missing from completion snapshot: %#v", events)
}
}
func TestRunEmitsTaskStateFromExistingLoopEvents(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(tools.NewUpdatePlanTool())
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{
{
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "plan-1", ToolName: planToolName},
{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: "plan-1", ArgumentsFragment: `{"plan":[{"content":"implement","status":"completed"}]}`},
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "plan-1"},
{Type: zeroruntime.StreamEventDone},
},
{
{Type: zeroruntime.StreamEventText, Content: "done"},
{Type: zeroruntime.StreamEventDone},
},
}}
recorder := trace.NewRecorder("session", "run", "")
result, err := Run(context.Background(), "implement the change", provider, Options{
Registry: registry,
Trace: recorder,
})
if err != nil {
t.Fatalf("Run: %v", err)
}
if result.FinalAnswer != "done" {
t.Fatalf("final answer = %q, want done", result.FinalAnswer)
}
events := recorder.Finish().TaskStates
if len(events) < 3 {
t.Fatalf("expected plan, tool, completion, and parity snapshots, got %#v", events)
}
last := events[len(events)-1]
if last.Status != string(taskStatusComplete) || last.PlanCompleted != 1 || last.ToolsSucceeded != 1 || last.PlanParity != string(taskPlanParityMatch) {
t.Fatalf("unexpected final task snapshot: %#v", last)
}
}
func TestTaskStateSnapshotIsImmutable(t *testing.T) {
state := newTaskState("objective", nil)
state.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"one","status":"pending"}]}`})
state.observe(taskStateEvent{kind: taskStateEventToolResult, toolResult: ToolResult{Status: tools.StatusOK, ChangedFiles: []string{"one.go"}}})
snapshot := state.snapshot()
snapshot.Plan.Items[0].Content = "mutated"
snapshot.ChangedFiles[0] = "mutated.go"
fresh := state.snapshot()
if fresh.Plan.Items[0].Content != "one" || fresh.ChangedFiles[0] != "one.go" {
t.Fatalf("snapshot mutation leaked into task state: %#v", fresh)
}
}
func TestTaskStatePlanParityUsesLatestPlan(t *testing.T) {
state := newTaskState("objective", nil)
state.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"old","status":"completed"}]}`})
state.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"new","status":"in_progress"}]}`})
messages := []zeroruntime.Message{{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{
{Name: planToolName, Arguments: `{"plan":[{"content":"new","status":"in_progress"}]}`},
}}}
if parity := state.observePlanParity(messages); parity != taskPlanParityMatch {
t.Fatalf("parity = %q, want match", parity)
}
messages[0].ToolCalls[0].Arguments = `{"plan":[{"content":"different","status":"pending"}]}`
if parity := state.observePlanParity(messages); parity != taskPlanParityMismatch {
t.Fatalf("parity = %q, want mismatch", parity)
}
}
func TestTaskStateMatchesPlanToolNormalization(t *testing.T) {
state := newTaskState("objective", nil)
arguments := `{"plan":[{"content":"first","status":"in_progress"},{"content":"second","status":"in_progress"}]}`
state.observe(taskStateEvent{kind: taskStateEventPlan, arguments: arguments})
snapshot := state.snapshot()
if snapshot.Plan.Completed != 1 || snapshot.Plan.InProgress != 1 {
t.Fatalf("multiple active items were not normalized like the plan tool: %#v", snapshot.Plan)
}
messages := []zeroruntime.Message{{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{{Name: planToolName, Arguments: arguments}}}}
if parity := state.observePlanParity(messages); parity != taskPlanParityMatch {
t.Fatalf("normalized state should still match its transcript event, got %q", parity)
}
empty := newTaskState("objective", nil)
empty.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[]}`})
emptyMessages := []zeroruntime.Message{{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{{Name: planToolName, Arguments: `{"plan":[]}`}}}}
if parity := empty.observePlanParity(emptyMessages); parity != taskPlanParityMatch {
t.Fatalf("explicit empty plan should match transcript, got %q", parity)
}
}
func TestTaskStateContextFallsBackWhenTranscriptDiffers(t *testing.T) {
state := newTaskState("objective", nil)
state.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"tracked","status":"completed"}]}`})
messages := []zeroruntime.Message{{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{
{Name: planToolName, Arguments: `{"plan":[{"content":"transcript","status":"pending"}]}`},
}}}
context := state.completionContext(messages, true)
if !context.PlanPending || context.PlanMatchesTranscript {
t.Fatalf("completion context must retain transcript truth on mismatch: %#v", context)
}
if context.Objective != "objective" {
t.Fatalf("objective lost from completion context: %#v", context)
}
}
func TestTaskStateCompactionSnapshotRetainsObjectiveOnPlanMismatch(t *testing.T) {
state := newTaskState("objective", nil)
state.observe(taskStateEvent{kind: taskStateEventPlan, arguments: `{"plan":[{"content":"verify","status":"pending"}]}`})
matching := []zeroruntime.Message{{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{
{Name: planToolName, Arguments: `{"plan":[{"content":"verify","status":"pending"}]}`},
}}}
if snapshot := state.snapshotForCompaction(matching); snapshot == nil || snapshot.Objective != "objective" {
t.Fatalf("matching transcript should produce compact state, got %#v", snapshot)
}
mismatching := append([]zeroruntime.Message(nil), matching...)
mismatching[0].ToolCalls = []zeroruntime.ToolCall{{Name: planToolName, Arguments: `{"plan":[{"content":"other","status":"pending"}]}`}}
if snapshot := state.snapshotForCompaction(mismatching); snapshot == nil || snapshot.Objective != "objective" || snapshot.PlanParity != taskPlanParityMismatch {
t.Fatalf("plan mismatch must retain immutable objective and mark mutable state uncorroborated, got %#v", snapshot)
}
}
func TestParseTaskPlanRejectsArgumentsTheToolWouldReject(t *testing.T) {
for _, arguments := range []string{
`{}`,
`{"plan":null}`,
`{"plan":[{"step":"alias is not accepted"}]}`,
`{"plan":[{"content":""}]}`,
`{"plan":[{"content":"valid"},{"content":""}]}`,
`{"plan":[{"id":4,"content":"valid"}]}`,
} {
if plan, ok := parseTaskPlan(arguments); ok {
t.Fatalf("parseTaskPlan(%s) = %#v, true; want rejected", arguments, plan)
}
}
}
-21
View File
@@ -1,21 +0,0 @@
package agent
import (
"context"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// sessionProvider adapts a TurnSession back to the Provider interface so the
// loop's existing Provider-shaped call sites (streamWithReconnect, the
// compaction summarizer, finalAnswerAfterMaxTurns) route their stream I/O
// through the session without any signature change. For the default adapter
// this reduces to the wrapped provider's StreamCompletion, so behavior is
// byte-identical; an optimized session (PR8) takes effect here transparently.
type sessionProvider struct {
session zeroruntime.TurnSession
}
func (s sessionProvider) StreamCompletion(ctx context.Context, request zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
return s.session.Stream(ctx, request)
}
-367
View File
@@ -1,367 +0,0 @@
package agent
import (
"context"
"errors"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// fakeTurnSession wraps an inner provider for streaming while counting the
// session lifecycle, so tests can observe exactly how the loop drives it.
type fakeTurnSession struct {
inner Provider
prewarms int
streams int
closes int
closeErr error
}
func (s *fakeTurnSession) Prewarm(context.Context) error { s.prewarms++; return nil }
func (s *fakeTurnSession) Stream(ctx context.Context, request zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
s.streams++
return s.inner.StreamCompletion(ctx, request)
}
func (s *fakeTurnSession) Compact(context.Context, zeroruntime.CompletionRequest) ([]zeroruntime.Message, error) {
return nil, zeroruntime.ErrCompactionUnsupported
}
func (s *fakeTurnSession) Close() error { s.closes++; return s.closeErr }
// fakeTurnSessionProvider opens a scripted session (or fails to).
type fakeTurnSessionProvider struct {
session *fakeTurnSession
openErr error
opens int
}
func (p *fakeTurnSessionProvider) OpenTurnSession(context.Context) (zeroruntime.TurnSession, error) {
p.opens++
if p.openErr != nil {
return nil, p.openErr
}
return p.session, nil
}
func (p *fakeTurnSessionProvider) Capabilities() zeroruntime.ProviderCapabilities {
return zeroruntime.ProviderCapabilities{}
}
func singleAnswerTurns(answer string) [][]zeroruntime.StreamEvent {
return [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventText, Content: answer},
{Type: zeroruntime.StreamEventDone},
}}
}
// TestRunOpenTurnSessionFailureSurfaces verifies a failed session open is a
// clean run-start error: no panic, no stream ever issued, nothing to close.
func TestRunOpenTurnSessionFailureSurfaces(t *testing.T) {
positional := &mockProvider{turns: singleAnswerTurns("never reached")}
tsp := &fakeTurnSessionProvider{openErr: errors.New("handshake refused")}
_, err := Run(context.Background(), "go", positional, Options{
MaxTurns: 2,
TurnSessionProvider: tsp,
})
if err == nil {
t.Fatal("expected an error from a failed session open")
}
if !strings.Contains(err.Error(), "open turn session") || !strings.Contains(err.Error(), "handshake refused") {
t.Fatalf("expected a wrapped open-turn-session error, got %v", err)
}
if len(positional.requests) != 0 {
t.Fatalf("expected no provider request after a failed open, got %d", len(positional.requests))
}
if tsp.opens != 1 {
t.Fatalf("expected exactly one open attempt, got %d", tsp.opens)
}
}
// TestRunCloseFailureIsSafe verifies a Close error at teardown is swallowed:
// the run still returns its normal result.
func TestRunCloseFailureIsSafe(t *testing.T) {
inner := &mockProvider{turns: singleAnswerTurns("done")}
session := &fakeTurnSession{inner: inner, closeErr: errors.New("close blew up")}
tsp := &fakeTurnSessionProvider{session: session}
result, err := Run(context.Background(), "go", inner, Options{
MaxTurns: 2,
TurnSessionProvider: tsp,
})
if err != nil {
t.Fatalf("expected Close failure to be swallowed, got %v", err)
}
if result.FinalAnswer != "done" {
t.Fatalf("expected normal result despite Close error, got %q", result.FinalAnswer)
}
if session.closes != 1 {
t.Fatalf("expected exactly one Close at teardown, got %d", session.closes)
}
}
// TestRunStreamsThroughTurnSession verifies that when a TurnSessionProvider is
// wired, every model request of the run flows through the session (and the
// session is prewarmed once and closed once).
func TestRunStreamsThroughTurnSession(t *testing.T) {
inner := &mockProvider{turns: singleAnswerTurns("via session")}
session := &fakeTurnSession{inner: inner}
tsp := &fakeTurnSessionProvider{session: session}
// The positional provider is a DIFFERENT mock: it must stay untouched,
// proving the session (not the positional provider) carried the run.
positional := &mockProvider{turns: singleAnswerTurns("wrong path")}
result, err := Run(context.Background(), "go", positional, Options{
MaxTurns: 2,
TurnSessionProvider: tsp,
})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "via session" {
t.Fatalf("expected the session-backed answer, got %q", result.FinalAnswer)
}
if len(positional.requests) != 0 {
t.Fatalf("expected the positional provider to be bypassed, got %d requests", len(positional.requests))
}
if session.streams == 0 || session.streams != len(inner.requests) {
t.Fatalf("expected every request to flow through the session: streams=%d inner=%d", session.streams, len(inner.requests))
}
if session.prewarms != 1 {
t.Fatalf("expected exactly one prewarm, got %d", session.prewarms)
}
if session.closes != 1 {
t.Fatalf("expected exactly one close, got %d", session.closes)
}
}
// TestRunDefaultTurnSessionMatchesRawProvider verifies the default (nil
// TurnSessionProvider) path is behavior-identical to passing an explicit
// default adapter over the same provider: same result, same request count.
func TestRunDefaultTurnSessionMatchesRawProvider(t *testing.T) {
run := func(explicit bool) (Result, *mockProvider) {
provider := &mockProvider{turns: escalateThenAnswerTurns("identical")}
registry := tools.NewRegistry()
registry.Register(escalatingTool{target: ""})
options := Options{Registry: registry, MaxTurns: 4}
if explicit {
options.TurnSessionProvider = zeroruntime.NewProviderTurnSessionProvider(provider, zeroruntime.ProviderCapabilities{})
}
result, err := Run(context.Background(), "go", provider, options)
if err != nil {
t.Fatal(err)
}
return result, provider
}
defaultResult, defaultProvider := run(false)
explicitResult, explicitProvider := run(true)
if defaultResult.FinalAnswer != explicitResult.FinalAnswer {
t.Fatalf("final answers diverged: default=%q explicit=%q", defaultResult.FinalAnswer, explicitResult.FinalAnswer)
}
if defaultResult.Turns != explicitResult.Turns {
t.Fatalf("turn counts diverged: default=%d explicit=%d", defaultResult.Turns, explicitResult.Turns)
}
if len(defaultProvider.requests) != len(explicitProvider.requests) {
t.Fatalf("request counts diverged: default=%d explicit=%d", len(defaultProvider.requests), len(explicitProvider.requests))
}
}
// TestRunModelSwitchClosesSessionAndContinues verifies a mid-run model switch
// closes the active session exactly once and streams the rest of the run on
// the new provider.
func TestRunModelSwitchClosesSessionAndContinues(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(escalatingTool{target: "claude-opus-4.1"})
firstInner := &mockProvider{turns: escalateThenAnswerTurns("unused")}
firstSession := &fakeTurnSession{inner: firstInner}
tsp := &fakeTurnSessionProvider{session: firstSession}
secondProvider := &mockProvider{turns: singleAnswerTurns("answered after switch")}
result, err := Run(context.Background(), "go", firstInner, Options{
Registry: registry,
Model: "claude-sonnet-4.5",
MaxTurns: 4,
TurnSessionProvider: tsp,
ModelSwitcher: func(_ context.Context, _ string) (Provider, error) {
return secondProvider, nil
},
})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "answered after switch" {
t.Fatalf("expected the post-switch answer, got %q", result.FinalAnswer)
}
// The original session handled exactly the escalation turn, then was closed
// once by the swap (the teardown defer closes the NEW session, not this one).
if firstSession.streams != 1 {
t.Fatalf("expected the original session to stream exactly the escalation turn, got %d", firstSession.streams)
}
if firstSession.closes != 1 {
t.Fatalf("expected the swap to close the original session exactly once, got %d", firstSession.closes)
}
if len(secondProvider.requests) != 1 {
t.Fatalf("expected the post-switch turn on the new provider, got %d requests", len(secondProvider.requests))
}
}
// TestRunModelSessionSwitcherCarriesOptimizedSession verifies the target-aware
// switcher: the switched-to TurnSessionProvider's DISTINCT session receives the
// post-switch streams, is prewarmed exactly once, and is closed exactly once at
// teardown — and the legacy ModelSwitcher is never consulted when both are set.
func TestRunModelSessionSwitcherCarriesOptimizedSession(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(escalatingTool{target: "claude-opus-4.1"})
firstInner := &mockProvider{turns: escalateThenAnswerTurns("unused")}
firstSession := &fakeTurnSession{inner: firstInner}
initial := &fakeTurnSessionProvider{session: firstSession}
switchedInner := &mockProvider{turns: singleAnswerTurns("answered on the switched session")}
switchedSession := &fakeTurnSession{inner: switchedInner}
switched := &fakeTurnSessionProvider{session: switchedSession}
var switchedTo string
legacyCalled := false
result, err := Run(context.Background(), "go", firstInner, Options{
Registry: registry,
Model: "claude-sonnet-4.5",
MaxTurns: 4,
TurnSessionProvider: initial,
ModelSessionSwitcher: func(_ context.Context, modelID string) (zeroruntime.TurnSessionProvider, error) {
switchedTo = modelID
return switched, nil
},
ModelSwitcher: func(_ context.Context, _ string) (Provider, error) {
legacyCalled = true
return nil, errors.New("legacy switcher must not be consulted")
},
})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "answered on the switched session" {
t.Fatalf("expected the switched session's answer, got %q", result.FinalAnswer)
}
if switchedTo != "claude-opus-4.1" {
t.Fatalf("expected switch to claude-opus-4.1, got %q", switchedTo)
}
if legacyCalled {
t.Fatal("legacy ModelSwitcher was consulted despite ModelSessionSwitcher being set")
}
if switched.opens != 1 {
t.Fatalf("expected exactly one open of the switched provider, got %d", switched.opens)
}
// The switched session carries the rest of the run: it streams the answer
// turn, is prewarmed exactly once at the swap, and is closed exactly once by
// the teardown defer.
if switchedSession.streams != 1 || len(switchedInner.requests) != 1 {
t.Fatalf("expected the switched session to stream the post-switch turn, streams=%d inner=%d", switchedSession.streams, len(switchedInner.requests))
}
if switchedSession.prewarms != 1 {
t.Fatalf("expected the switched session to be prewarmed exactly once, got %d", switchedSession.prewarms)
}
if switchedSession.closes != 1 {
t.Fatalf("expected the switched session to be closed exactly once at teardown, got %d", switchedSession.closes)
}
// The original session ends with the swap: one stream, one close.
if firstSession.streams != 1 || firstSession.closes != 1 {
t.Fatalf("expected the original session to end at the swap (streams=%d closes=%d)", firstSession.streams, firstSession.closes)
}
}
// TestRunModelSessionSwitcherOpenFailureIsNonFatal verifies the swap-time open
// failure: the switcher returns a TurnSessionProvider whose OpenTurnSession
// fails. The run stays non-fatal on the ORIGINAL session (closed only at
// teardown), and the switch-failure note reaches the next request.
func TestRunModelSessionSwitcherOpenFailureIsNonFatal(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(escalatingTool{target: "claude-opus-4.1"})
inner := &mockProvider{turns: escalateThenAnswerTurns("recovered after failed open")}
session := &fakeTurnSession{inner: inner}
failingOpen := &fakeTurnSessionProvider{openErr: errors.New("switched handshake refused")}
result, err := Run(context.Background(), "go", inner, Options{
Registry: registry,
Model: "claude-sonnet-4.5",
MaxTurns: 4,
TurnSessionProvider: &fakeTurnSessionProvider{session: session},
ModelSessionSwitcher: func(_ context.Context, _ string) (zeroruntime.TurnSessionProvider, error) {
return failingOpen, nil
},
})
if err != nil {
t.Fatalf("expected a failed swap open to be non-fatal, got %v", err)
}
if result.FinalAnswer != "recovered after failed open" {
t.Fatalf("expected the run to recover on the original session, got %q", result.FinalAnswer)
}
if failingOpen.opens != 1 {
t.Fatalf("expected exactly one open attempt on the switched provider, got %d", failingOpen.opens)
}
// The original session serves both turns and is closed only at teardown —
// never by the failed swap.
if session.streams != 2 || session.closes != 1 {
t.Fatalf("expected the original session to serve both turns and close once (streams=%d closes=%d)", session.streams, session.closes)
}
var sawNote bool
for _, message := range inner.requests[1].Messages {
if message.Role == zeroruntime.MessageRoleUser && strings.Contains(strings.ToLower(message.Content), "could not switch") {
sawNote = true
}
}
if !sawNote {
t.Fatalf("expected a switch-failure note on the next turn, messages: %+v", inner.requests[1].Messages)
}
}
// TestRunModelSessionSwitcherErrorIsNonFatal verifies a ModelSessionSwitcher
// error keeps the run on the current session with a transcript note — the same
// non-fatal contract as ModelSwitcher.
func TestRunModelSessionSwitcherErrorIsNonFatal(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(escalatingTool{target: "claude-opus-4.1"})
inner := &mockProvider{turns: escalateThenAnswerTurns("recovered on original session")}
session := &fakeTurnSession{inner: inner}
result, err := Run(context.Background(), "go", inner, Options{
Registry: registry,
Model: "claude-sonnet-4.5",
MaxTurns: 4,
TurnSessionProvider: &fakeTurnSessionProvider{session: session},
ModelSessionSwitcher: func(_ context.Context, _ string) (zeroruntime.TurnSessionProvider, error) {
return nil, errors.New("session build blew up")
},
})
if err != nil {
t.Fatalf("expected session-switcher error to be non-fatal, got %v", err)
}
if result.FinalAnswer != "recovered on original session" {
t.Fatalf("expected the run to recover on the original session, got %q", result.FinalAnswer)
}
// Both turns stream through the original session; it closes once at teardown.
if session.streams != 2 || session.closes != 1 {
t.Fatalf("expected the original session to serve both turns and close once (streams=%d closes=%d)", session.streams, session.closes)
}
var sawNote bool
for _, request := range inner.requests[1].Messages {
if request.Role == zeroruntime.MessageRoleUser && strings.Contains(strings.ToLower(request.Content), "could not switch") {
sawNote = true
}
}
if !sawNote {
t.Fatalf("expected a non-fatal switch-failure note on the next turn, messages: %+v", inner.requests[1].Messages)
}
}
+6 -87
View File
@@ -61,13 +61,10 @@ const (
)
type ToolResult struct {
ToolCallID string
Name string
Status tools.Status
Output string
// Truncated reports that the tool's model-visible output omitted content.
// The full result may be recoverable through Meta["spill_path"].
Truncated bool
ToolCallID string
Name string
Status tools.Status
Output string
Meta map[string]string
Redacted bool
ChangedFiles []string
@@ -75,12 +72,6 @@ type ToolResult struct {
// DenialReason categorizes why a tool call was blocked (empty when it ran).
// It lets a surface distinguish the cause precisely instead of parsing Output.
DenialReason DenialCategory
// Risk is the sandbox risk classification of this call, stamped for EXECUTED
// results so run-policy observers (the execution-profile controller) can see
// the risk level of an allowed mutation. It mirrors the classification the
// permission path already computes; denied or canceled results keep the zero
// value. Pure observation: nothing about permissions or sandboxing changes.
Risk sandbox.Risk
// LoadedTools carries the deferred-tool names a tool_search call asked the
// loop to expose next turn (lifted from Meta["load_tools"]). nil for every
// ordinary tool result; only tool_search populates it.
@@ -104,54 +95,6 @@ const (
DenialHookBlocked DenialCategory = "hook_blocked" // vetoed by a beforeTool hook
)
// ProfilePolicy is the loop-facing slice of a selected execution profile.
// The surface (exec flag, TUI command) resolves a named profile into this
// policy; the loop only ever sees the policy, never the catalog.
type ProfilePolicy struct {
// Name labels trace counters and diagnostics (e.g. "fast"). The trace
// recorder's profile label is set by the caller at recorder construction.
Name string
// Escalate, when non-nil, arms one-shot in-run posture escalation.
Escalate *PostureEscalation
}
// PostureEscalation describes a one-shot escalation to stricter knob values,
// applied mid-run when an armed trigger fires. Targets are the values the
// selected profile DISPLACED at run start (i.e. "restore the balanced
// posture"), so escalation can never introduce a value that was not already
// valid for this run and model. Zero-valued targets leave that knob untouched.
type PostureEscalation struct {
// MaxTurns raises the turn ceiling to this value when greater than the
// ceiling in effect. 0 leaves the ceiling untouched.
MaxTurns int
// ReasoningEffort replaces the run's effort when non-empty.
ReasoningEffort string
// RestoreDefaultEffort clears the run's effort override (back to the
// provider/model default) when true. It exists because the displaced value
// of a profile-filled effort is "" — which as a ReasoningEffort target
// means "leave untouched" — so restoring the default needs its own signal.
// Ignored when ReasoningEffort is non-empty.
RestoreDefaultEffort bool
// RestoreCompletionGate re-enables RequireCompletionSignal (headless
// completion semantics) when true.
RestoreCompletionGate bool
// Triggers. A zero value disables that signal entirely.
// OnToolFailureStreak fires when the repeated-failure guard observes a
// same-tool retriable-failure streak of at least this length.
OnToolFailureStreak int
// OnCompletionUncertain fires on the Nth uncertain completion evaluation
// (continue nudge or semantic check). Headless only: the completion gate
// never runs interactively.
OnCompletionUncertain int
// OnSelfCorrectFailure fires when a post-edit verification cycle reports a
// failing outcome (correcting, reported, or aborted).
OnSelfCorrectFailure bool
// OnRiskyMutation fires when an EXECUTED tool result carries a sandbox risk
// level at or above this threshold. Empty disables the signal.
OnRiskyMutation sandbox.RiskLevel
}
type PermissionRequest struct {
ToolCallID string `json:"toolCallId"`
ToolName string `json:"name"`
@@ -337,27 +280,6 @@ type Options struct {
// request), so every existing caller is unaffected. A returned error is
// non-fatal: the run continues on the current model.
ModelSwitcher func(ctx context.Context, modelID string) (Provider, error)
// TurnSessionProvider, when set, supplies the turn session the run streams
// through — the seam an optimized provider session (connection reuse,
// prewarm, native compaction) plugs into without touching the loop. nil
// keeps the default: the loop wraps the passed provider in a no-op session
// whose Stream IS provider.StreamCompletion, so behavior is byte-identical
// and every existing caller is unaffected.
TurnSessionProvider zeroruntime.TurnSessionProvider
// ModelSessionSwitcher, when set, is the target-aware escalation hook: the
// loop prefers it over ModelSwitcher, and its TurnSessionProvider keeps an
// optimized session (and its capabilities) across a mid-run model switch.
// nil falls back to ModelSwitcher, whose bare Provider is wrapped in the
// default no-op session — today's behavior, unchanged. Same non-fatal error
// contract as ModelSwitcher: a returned error records a note and the run
// continues on the current model and session.
ModelSessionSwitcher func(ctx context.Context, modelID string) (zeroruntime.TurnSessionProvider, error)
// Profile, when set, arms the execution-profile posture controller for this
// run (auto-escalation to stricter knob values on failure/uncertainty/risky
// mutation signals). nil — the default everywhere today — leaves the loop
// byte-identical: no observation, no escalation, no counters. Same opt-in
// convention as Trace and SelfCorrect.
Profile *ProfilePolicy
// Trace, when set, records per-turn timing for the run: the loop stamps
// spans (prompt build, generation, tool execution, permission wait,
// compaction, provider connect) and counters (model requests, tool calls,
@@ -390,11 +312,8 @@ type Options struct {
// continuation cue ("…Let me check the config:"). The loop then nudges the
// model to continue instead, bounded by maxContinueNudges (and still by
// MaxTurns and the run deadline); if the model keeps stalling, the run
// finalizes as INCOMPLETE (Result.Incomplete) rather than success. When the
// run's profile also enables SelfCorrect, an otherwise-complete turn gets one
// task-grounded semantic check before success; profiles without SelfCorrect
// add no model call. Default false leaves the loop byte-identical, so the
// interactive TUI is unaffected.
// finalizes as INCOMPLETE (Result.Incomplete) rather than success. Default
// false leaves the loop byte-identical, so the interactive TUI is unaffected.
RequireCompletionSignal bool
runPermissions *permissionRunState
-3
View File
@@ -1341,9 +1341,6 @@ Flags:
--spec-reasoning-effort <effort>
Override draft reasoning effort when --use-spec is set
--max-turns <number> Override the maximum agent loop turns
--exec-profile <name> Apply an execution profile (balanced, fast, thorough): loop
posture only (turn budget, effort, self-correction, escalation);
composes with --mode (which picks the model) and explicit flags win
--auto <low|medium|high> Set exec autonomy; high enables unsafe tools
--enabled-tools <tools> Only expose these comma or space separated tools
--disabled-tools <tools> Hide these comma or space separated tools
+25 -153
View File
@@ -13,7 +13,6 @@ import (
"github.com/Gitlawb/zero/internal/agent"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/errhint"
"github.com/Gitlawb/zero/internal/execprofile"
"github.com/Gitlawb/zero/internal/imageinput"
"github.com/Gitlawb/zero/internal/lsp"
"github.com/Gitlawb/zero/internal/modelregistry"
@@ -70,14 +69,7 @@ type execOptions struct {
// intentionally inert: nothing consumes it. Model selection is driven by
// --model / --mode instead. See writeExecHelp ("Accept legacy model profile
// selection") and TestRunExecAcceptsLegacyModelProfileFlags.
modelProfile string
// execProfile selects a named execution profile (balanced, fast, thorough):
// a loop-posture bundle (turn budget, reasoning effort, self-correction,
// escalation triggers) applied AFTER --mode with the same
// fill-only-if-unset rule, so precedence is explicit flag > mode > profile.
// Distinct from the mode preset also named "fast" (which picks a model) and
// from the legacy inert --profile above. See internal/execprofile.
execProfile string
modelProfile string
reasoningEffort string
useSpec bool
specModel string
@@ -175,14 +167,6 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
if err := applyExecMode(&options); err != nil {
return writeExecFormatUsageError(stdout, stderr, options.outputFormat, err.Error())
}
// A profile tunes loop posture the way a mode tunes model selection, and
// runs after it so the mode's fills count as "set" and win. MaxTurns is
// deferred to config resolution below, where the displaced resolved budget
// is known and becomes the escalation restore target.
execProfile, execProfileFilledEffort, err := applyExecProfile(&options)
if err != nil {
return writeExecFormatUsageError(stdout, stderr, options.outputFormat, err.Error())
}
workspaceRoot, err := resolveWorkspaceRoot(options.cwd, deps)
if err != nil {
@@ -288,8 +272,6 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
}
return writeExecProviderError(stdout, stderr, options.outputFormat, "provider_error", err.Error())
}
var displacedMaxTurns int
resolved.MaxTurns, displacedMaxTurns = applyProfileTurnBudget(execProfile, options.maxTurns, resolved.MaxTurns)
registerLocalControlTools(registry, workspaceRoot, resolved.LocalControl)
if err := validateExecToolFilters(options, registry); err != nil {
return writeExecFormatUsageError(stdout, stderr, options.outputFormat, err.Error())
@@ -430,36 +412,6 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
}
}
// Optimized OpenAI turn sessions (ZERO_OPENAI_TURN_SESSION, default off).
// nil when gated off or the profile is ineligible: agent.Run then wraps the
// provider in its default adapter — the exact code path of today. The
// session switcher is installed only when the run START is optimized, so
// the legacy ModelSwitcher path above stays untouched otherwise.
turnSessions, _ := providers.OptimizedTurnSessions(resolved.Provider, provider, providers.Options{})
var modelSessionSwitcher func(context.Context, string) (zeroruntime.TurnSessionProvider, error)
if options.allowEscalation && turnSessions != nil {
modelSessionSwitcher = func(_ context.Context, modelID string) (zeroruntime.TurnSessionProvider, error) {
switchedProfile := resolved.Provider
switchedProfile.Model = modelID
switchedProvider, err := deps.newProvider(switchedProfile)
if err != nil {
return nil, err
}
if switchedProvider == nil {
// The loop treats a nil session source as "no swap" — mirror the
// legacy closure's (nil, nil) contract.
return nil, nil
}
currentModel = modelID
if optimized, ok := providers.OptimizedTurnSessions(switchedProfile, switchedProvider, providers.Options{}); ok {
return optimized, nil
}
// Ineligible switch target: default adapter, but with the switched
// model's resolved capability projection preserved.
return providers.DefaultTurnSessions(switchedProfile, switchedProvider, providers.Options{}), nil
}
}
runMetadata, err := resolveExecRunMetadata(resolved.Provider)
if err != nil {
return writeExecProviderError(stdout, stderr, options.outputFormat, "provider_error", err.Error())
@@ -504,13 +456,6 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
reasoningEffort: forwardEffort,
specPermissionMode: permissionMode,
notifier: notifier,
// The profile displaced resolved.MaxTurns above, so the spec-draft
// run arms the same escalation policy as the main run (in practice
// only the failure-streak and risky-mutation triggers can fire
// here: the draft runs without the completion gate or a
// self-corrector).
profilePolicy: execProfile.Policy(displacedMaxTurns,
specProfileEffortFilled(execProfileFilledEffort, options.specReasoningEffort)),
})
}
@@ -553,7 +498,7 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
var traceRecorder *trace.Recorder
var traceSnapshot *trace.TurnTrace
if tracePath != "" {
traceRecorder = trace.NewRecorder(preparedSession.Session.SessionID, runID, execProfile.Name)
traceRecorder = trace.NewRecorder(preparedSession.Session.SessionID, runID, "")
defer func() {
if err := writeTraceSnapshot(traceSnapshot, tracePath, stderr); err != nil {
fmt.Fprintf(stderr, "[zero] failed to write trace: %s\n", err)
@@ -627,32 +572,29 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
hookDispatcher, hookSkip := newHookDispatcherWithExtra(workspaceRoot, pluginActivation.hooks, trustRoot)
emitTrustNotice(stderr, hookSkip, pluginActivation.trustSkip, mcpSkip)
result, err := agent.Run(runCtx, agentPrompt, provider, agent.Options{
MaxTurns: resolved.MaxTurns,
ContextWindow: resolveAgentContextWindow(runCtx, modelRegistry, resolved.Provider),
DeferThreshold: effectiveDeferThreshold,
Specialists: specialistRuntime.specialistInfos(),
Skills: pluginActivation.skillInfos(deps.skillsDir()),
SessionID: preparedSession.Session.SessionID,
CallingSessionID: options.callingSessionID,
CallingToolUseID: options.callingToolUseID,
Tag: options.tag,
Depth: options.depth,
SessionTitle: sessionTitle,
ProviderName: resolved.Provider.Name,
Model: resolved.Provider.Model,
ModelSwitcher: modelSwitcher,
TurnSessionProvider: turnSessions,
ModelSessionSwitcher: modelSessionSwitcher,
ReasoningEffort: forwardEffort,
Trace: traceRecorder,
Cwd: workspaceRoot,
Images: images,
Registry: registry,
PermissionMode: permissionMode,
Autonomy: options.autonomy,
SelfCorrect: selfCorrector,
FileDiagnostics: fileDiagnostics,
Profile: execProfile.Policy(displacedMaxTurns, execProfileFilledEffort),
MaxTurns: resolved.MaxTurns,
ContextWindow: resolveAgentContextWindow(runCtx, modelRegistry, resolved.Provider),
DeferThreshold: effectiveDeferThreshold,
Specialists: specialistRuntime.specialistInfos(),
Skills: pluginActivation.skillInfos(deps.skillsDir()),
SessionID: preparedSession.Session.SessionID,
CallingSessionID: options.callingSessionID,
CallingToolUseID: options.callingToolUseID,
Tag: options.tag,
Depth: options.depth,
SessionTitle: sessionTitle,
ProviderName: resolved.Provider.Name,
Model: resolved.Provider.Model,
ModelSwitcher: modelSwitcher,
ReasoningEffort: forwardEffort,
Trace: traceRecorder,
Cwd: workspaceRoot,
Images: images,
Registry: registry,
PermissionMode: permissionMode,
Autonomy: options.autonomy,
SelfCorrect: selfCorrector,
FileDiagnostics: fileDiagnostics,
// Headless exec: don't accept a no-tool-call turn as "done" while work
// clearly remains (pending plan items / a mid-step continuation cue) —
// nudge to continue, and finalize as INCOMPLETE rather than false success
@@ -694,9 +636,6 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
if len(result.Meta) > 0 {
payload["meta"] = result.Meta
}
if result.Truncated {
payload["truncated"] = true
}
if result.Redacted {
payload["redacted"] = true
}
@@ -939,12 +878,6 @@ func providerSensitiveEnvKeys(resolved config.ResolvedConfig) []string {
// applyConfiguredSandboxPolicy overlays every config-sourced sandbox knob onto
// the default policy.
func applyConfiguredSandboxPolicy(policy sandbox.Policy, cfg config.SandboxConfig) sandbox.Policy {
// An explicit `"sandbox": {"enabled": false}` disables the sandbox: every
// request is allowed and commands run unwrapped. Only reachable from global
// config / CLI — project config never sets Enabled (see mergeProjectConfig).
if cfg.Enabled != nil && !*cfg.Enabled {
policy.Mode = sandbox.ModeDisabled
}
if network := strings.TrimSpace(cfg.Network); network != "" {
switch sandbox.NetworkMode(network) {
case sandbox.NetworkAllow, sandbox.NetworkDeny:
@@ -1165,67 +1098,6 @@ func applyExecMode(options *execOptions) error {
return nil
}
// applyExecProfile expands an --exec-profile selection onto the exec options.
// Like applyExecMode it only fills fields the caller left unset, and it runs
// AFTER the mode so a mode-filled field counts as set: precedence is explicit
// flag > --mode > --exec-profile. Only the options-level knobs are applied here
// (reasoning effort still flows through the model-supported gating downstream;
// self-correct is presence-only so a profile can only turn it on). MaxTurns is
// intentionally NOT applied here — the profile displaces the resolved budget
// after config resolution, where the displaced value is known and becomes the
// escalation restore target. An unknown profile is a usage error listing the
// valid names. Not to be confused with the legacy inert --profile flag.
// The second return reports whether the profile actually filled the reasoning
// effort (false when the user set one explicitly), so the escalation policy
// knows the displaced effort was the provider default and can restore it.
func applyExecProfile(options *execOptions) (execprofile.Profile, bool, error) {
name := strings.TrimSpace(options.execProfile)
if name == "" {
return execprofile.Profile{}, false, nil
}
profile, ok := execprofile.Lookup(name)
if !ok {
return execprofile.Profile{}, false, execUsageError{fmt.Sprintf("unknown execution profile %q. Valid profiles: %s.", options.execProfile, strings.Join(execprofile.Names(), ", "))}
}
effortFilled := false
if options.reasoningEffort == "" && profile.ReasoningEffort != "" {
options.reasoningEffort = profile.ReasoningEffort
effortFilled = true
}
// Spec-safe projection: the spec-draft path wires no self-corrector (an
// explicit --self-correct --use-spec is rejected at parse time, before
// profiles apply), so a profile's self-correct knob is meaningless there.
// Project it away rather than erroring: the user asked for a thorough
// DRAFT, and the profile's other knobs (budget, effort) apply to it fine.
if profile.SelfCorrect && !options.useSpec {
options.selfCorrect = true
}
return profile, effortFilled, nil
}
// specProfileEffortFilled reports whether the profile's effort fill actually
// governs the spec-draft run. An explicit --spec-reasoning-effort replaces the
// filled effort for the draft, so the escalation's effort restore must not arm
// there: escalation must never clear an effort the user pinned by hand.
func specProfileEffortFilled(effortFilled bool, specReasoningEffort string) bool {
return effortFilled && strings.TrimSpace(specReasoningEffort) == ""
}
// applyProfileTurnBudget decides the run's turn budget once config is resolved.
// The profile displaces the RESOLVED value, not the flag: a pinned budget
// always wins and the profile backs off entirely, while an env/config budget
// is the "balanced posture" a mid-run escalation can restore. pinnedMaxTurns
// is any caller-pinned budget — an explicit --max-turns flag OR a mode
// preset's fill (both land in options.maxTurns, and a mode outranks a profile
// just like the flag does). displaced is 0 when nothing was displaced, so an
// escalation leaves the ceiling untouched.
func applyProfileTurnBudget(profile execprofile.Profile, pinnedMaxTurns int, resolvedMaxTurns int) (effective int, displaced int) {
if profile.MaxTurns > 0 && pinnedMaxTurns == 0 {
return profile.MaxTurns, resolvedMaxTurns
}
return resolvedMaxTurns, 0
}
// resolveSelectedModel routes a user-supplied --model value through the model
// registry so that fuzzy aliases (e.g. "sonnet 4.5") resolve to canonical ids
// and deprecated models auto-redirect to their fallback. It returns the model id
-13
View File
@@ -147,19 +147,6 @@ func parseExecArgs(args []string) (execOptions, bool, error) {
index = next
case strings.HasPrefix(arg, "--profile="):
options.modelProfile = strings.TrimSpace(strings.TrimPrefix(arg, "--profile="))
case arg == "--exec-profile":
value, next, err := nextFlagValue(args, index, arg)
if err != nil {
return options, false, err
}
options.execProfile = strings.TrimSpace(value)
index = next
case strings.HasPrefix(arg, "--exec-profile="):
value, err := requiredInlineFlagValue(arg, "--exec-profile")
if err != nil {
return options, false, err
}
options.execProfile = value
case arg == "-r" || arg == "--reasoning-effort":
value, next, err := nextFlagValue(args, index, arg)
if err != nil {
-224
View File
@@ -1,224 +0,0 @@
package cli
import (
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/execprofile"
)
func TestApplyExecProfileFillsOnlyUnset(t *testing.T) {
// Unset effort: the profile fills it and reports the fill, so the
// escalation policy knows the displaced effort was the provider default.
options := execOptions{execProfile: "fast"}
profile, effortFilled, err := applyExecProfile(&options)
if err != nil {
t.Fatalf("applyExecProfile: %v", err)
}
if profile.Name != "fast" {
t.Fatalf("profile.Name = %q, want fast", profile.Name)
}
if options.reasoningEffort != "low" {
t.Fatalf("reasoningEffort = %q, want the profile's low", options.reasoningEffort)
}
if !effortFilled {
t.Fatal("the profile filled the effort, so effortFilled must report it")
}
// Set effort (explicit flag or a mode's fill): the profile backs off.
options = execOptions{execProfile: "fast", reasoningEffort: "high"}
_, effortFilled, err = applyExecProfile(&options)
if err != nil {
t.Fatalf("applyExecProfile: %v", err)
}
if options.reasoningEffort != "high" {
t.Fatalf("reasoningEffort = %q, an explicit value must win over the profile", options.reasoningEffort)
}
if effortFilled {
t.Fatal("the profile backed off, so effortFilled must be false")
}
}
func TestApplyExecProfileThoroughArmsSelfCorrect(t *testing.T) {
options := execOptions{execProfile: "thorough"}
if _, _, err := applyExecProfile(&options); err != nil {
t.Fatalf("applyExecProfile: %v", err)
}
if !options.selfCorrect {
t.Fatal("thorough must arm the post-edit self-correct loop")
}
if options.reasoningEffort != "high" {
t.Fatalf("reasoningEffort = %q, want high", options.reasoningEffort)
}
}
// The spec-draft path wires no self-corrector (explicit --self-correct
// --use-spec is rejected at parse time, before profiles apply), so a profile's
// self-correct knob is projected away under --use-spec instead of silently
// arming a knob the draft runner ignores. The other knobs still apply.
func TestApplyExecProfileSpecProjectsAwaySelfCorrect(t *testing.T) {
options := execOptions{execProfile: "thorough", useSpec: true}
if _, _, err := applyExecProfile(&options); err != nil {
t.Fatalf("applyExecProfile: %v", err)
}
if options.selfCorrect {
t.Fatal("thorough under --use-spec must not arm self-correct (the draft runner has none)")
}
if options.reasoningEffort != "high" {
t.Fatalf("reasoningEffort = %q, the profile's other knobs must still apply", options.reasoningEffort)
}
}
func TestApplyExecProfileUnknownIsUsageError(t *testing.T) {
options := execOptions{execProfile: "turbo"}
_, _, err := applyExecProfile(&options)
if err == nil {
t.Fatal("expected a usage error for an unknown profile")
}
if _, ok := err.(execUsageError); !ok {
t.Fatalf("expected execUsageError, got %T: %v", err, err)
}
for _, name := range execprofile.Names() {
if !strings.Contains(err.Error(), name) {
t.Fatalf("usage error must list %q, got %q", name, err.Error())
}
}
}
// Precedence: explicit flag > --mode > --exec-profile. The deep mode fills
// effort=high and max-turns=160; the fast profile must not override either,
// because a mode-filled field counts as set by the time the profile runs.
func TestExecProfilePrecedenceFlagOverModeOverProfile(t *testing.T) {
options := execOptions{mode: "deep", execProfile: "fast"}
if err := applyExecMode(&options); err != nil {
t.Fatalf("applyExecMode: %v", err)
}
profile, _, err := applyExecProfile(&options)
if err != nil {
t.Fatalf("applyExecProfile: %v", err)
}
if options.reasoningEffort != "high" {
t.Fatalf("reasoningEffort = %q, the mode's high must win over the profile's low", options.reasoningEffort)
}
// The mode filled options.maxTurns, so the turn budget must not be displaced.
effective, displaced := applyProfileTurnBudget(profile, options.maxTurns, options.maxTurns)
if effective != 160 || displaced != 0 {
t.Fatalf("turn budget = (%d, displaced %d), the mode's 160 must win with nothing displaced", effective, displaced)
}
}
func TestApplyProfileTurnBudget(t *testing.T) {
fast, _ := execprofile.Lookup("fast")
balanced, _ := execprofile.Lookup("balanced")
// No explicit flag: the profile displaces the resolved budget.
if effective, displaced := applyProfileTurnBudget(fast, 0, 80); effective != 30 || displaced != 80 {
t.Fatalf("fast over resolved 80 = (%d, %d), want (30, 80)", effective, displaced)
}
// Explicit --max-turns pins the budget; the profile backs off entirely.
if effective, displaced := applyProfileTurnBudget(fast, 50, 50); effective != 50 || displaced != 0 {
t.Fatalf("fast with explicit 50 = (%d, %d), want (50, 0)", effective, displaced)
}
// Balanced never displaces anything.
if effective, displaced := applyProfileTurnBudget(balanced, 0, 80); effective != 80 || displaced != 0 {
t.Fatalf("balanced over resolved 80 = (%d, %d), want (80, 0)", effective, displaced)
}
}
// The no-regression invariant at the options level: selecting balanced leaves
// the options byte-identical to not selecting a profile at all, and produces
// no loop policy.
func TestExecProfileBalancedLeavesOptionsUntouched(t *testing.T) {
options := execOptions{execProfile: "balanced"}
reference := options
profile, _, err := applyExecProfile(&options)
if err != nil {
t.Fatalf("applyExecProfile: %v", err)
}
if !reflect.DeepEqual(options, reference) {
t.Fatalf("balanced changed the options: %+v vs %+v", options, reference)
}
if policy := profile.Policy(80, false); policy != nil {
t.Fatalf("balanced must produce a nil policy, got %+v", policy)
}
}
// --exec-profile= with an inline empty value must error like its two-token
// form and the sibling --mode=, not silently run the default posture (the
// classic shell hazard: an unset variable expanding into the inline form).
func TestParseExecProfileInlineEmptyIsError(t *testing.T) {
_, _, err := parseExecArgs([]string{"--exec-profile=", "hello"})
if err == nil || !strings.Contains(err.Error(), "--exec-profile requires a value") {
t.Fatalf("expected a required-value error, got %v", err)
}
}
// Full echo-provider run: the selected profile must be stamped into the
// per-turn trace so benchmark attribution can group runs by posture.
func TestRunExecTraceRecordsSelectedProfile(t *testing.T) {
tracePath := filepath.Join(t.TempDir(), "trace.ndjson")
exitCode, _, stderr := runExecWithEcho(t, []string{
"exec", "--exec-profile", "fast", "--trace", tracePath, "hello",
})
if exitCode != exitSuccess {
t.Fatalf("expected exit %d, got %d: %s", exitSuccess, exitCode, stderr)
}
raw, err := os.ReadFile(tracePath)
if err != nil {
t.Fatalf("read trace: %v", err)
}
if !strings.Contains(string(raw), `"profile":"fast"`) {
t.Fatalf("trace must record the selected profile, got %s", raw)
}
}
// An explicit --spec-reasoning-effort governs the spec draft's effort, so the
// escalation effort-restore must not arm there even when the profile filled
// the (unused) main effort.
func TestSpecProfileEffortFilled(t *testing.T) {
if !specProfileEffortFilled(true, "") {
t.Fatal("no explicit spec effort: the profile's fill governs, restore must arm")
}
if specProfileEffortFilled(true, "high") {
t.Fatal("an explicit spec effort must disarm the effort restore")
}
if specProfileEffortFilled(false, "") {
t.Fatal("nothing filled, nothing to restore")
}
}
// The trace label deliberately records the SELECTED profile, balanced
// included, so captures are self-describing. Run behavior stays identical
// (balanced fills nothing and arms no policy); the label is the one intended
// difference in the opt-in trace artifact.
func TestRunExecTraceRecordsBalancedSelection(t *testing.T) {
tracePath := filepath.Join(t.TempDir(), "trace.ndjson")
exitCode, _, stderr := runExecWithEcho(t, []string{
"exec", "--exec-profile", "balanced", "--trace", tracePath, "hello",
})
if exitCode != exitSuccess {
t.Fatalf("expected exit %d, got %d: %s", exitSuccess, exitCode, stderr)
}
raw, err := os.ReadFile(tracePath)
if err != nil {
t.Fatalf("read trace: %v", err)
}
if !strings.Contains(string(raw), `"profile":"balanced"`) {
t.Fatalf("trace must record the balanced selection, got %s", raw)
}
}
func TestRunExecUnknownExecProfileIsUsageError(t *testing.T) {
exitCode, _, stderr := runExecWithEcho(t, []string{
"exec", "--exec-profile", "turbo", "hello",
})
if exitCode != exitUsage {
t.Fatalf("expected usage exit %d, got %d", exitUsage, exitCode)
}
if !strings.Contains(stderr, "unknown execution profile") {
t.Fatalf("expected an unknown-profile usage error, got %q", stderr)
}
}
+1 -4
View File
@@ -72,11 +72,8 @@ func preflightExecSession(options execOptions) error {
if session == nil {
return execUsageError{"Zero session not found: " + options.resume}
}
if !sessions.IsResumableKind(session.SessionKind) {
return execUsageError{"Zero session is not resumable: " + options.resume}
}
case options.resumeLatest:
latest, err := store.LatestResumable()
latest, err := store.Latest()
if err != nil {
return err
}
-26
View File
@@ -1,26 +0,0 @@
package cli
import (
"strings"
"testing"
"github.com/Gitlawb/zero/internal/sessions"
)
func TestPreflightRejectsSideSessions(t *testing.T) {
t.Setenv("XDG_DATA_HOME", t.TempDir())
store := sessions.NewStore(sessions.StoreOptions{})
if _, err := store.Create(sessions.CreateInput{SessionID: "side", SessionKind: sessions.SessionKindSide}); err != nil {
t.Fatalf("create side session: %v", err)
}
err := preflightExecSession(execOptions{resumeLatest: true})
if err == nil || !strings.Contains(err.Error(), "No Zero sessions available to resume") {
t.Fatalf("preflightExecSession error = %v, want no resumable sessions", err)
}
err = preflightExecSession(execOptions{resume: "side"})
if err == nil || !strings.Contains(err.Error(), "Zero session is not resumable: side") {
t.Fatalf("explicit side-session preflight error = %v, want non-resumable rejection", err)
}
}
-6
View File
@@ -46,11 +46,6 @@ type execSpecDraftRun struct {
reasoningEffort string
specPermissionMode agent.PermissionMode
notifier *notify.Notifier
// profilePolicy carries the selected execution profile's escalation policy
// (nil when none): the profile displaces resolved.MaxTurns before this
// path branches off, so the spec-draft run needs the same safety net as
// the main run.
profilePolicy *agent.ProfilePolicy
}
type execSpecDraftInfo struct {
@@ -124,7 +119,6 @@ func runExecSpecDraft(run execSpecDraftRun) int {
ProviderName: run.resolved.Provider.Name,
Model: run.resolved.Provider.Model,
ReasoningEffort: run.reasoningEffort,
Profile: run.profilePolicy,
Cwd: run.workspaceRoot,
SystemPrompt: specmode.DraftSystemPrompt,
Images: run.images,
-4
View File
@@ -549,14 +549,10 @@ func TestExecMemberAutoToolListIncludesMutators(t *testing.T) {
}
func TestRunExecAcceptsLegacyModelProfileFlags(t *testing.T) {
// --profile (legacy, inert model profile) and --exec-profile (execution
// profile) are distinct flags and must coexist on one invocation.
exitCode, stdout, stderr := runExecWithEcho(t, []string{
"exec",
"--profile",
"fast",
"--exec-profile",
"balanced",
"--reasoning-effort",
"low",
"hello",
-125
View File
@@ -1,125 +0,0 @@
package cli
import (
"bytes"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/providers/openai"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// TestRunExecOptimizedSessionUnderGate proves the end-to-end wiring: with
// ZERO_OPENAI_TURN_SESSION on and a real official-OpenAI provider, a headless
// run streams through the optimized session — the server sees the prewarm HEAD
// probe in addition to the turn's POST.
func TestRunExecOptimizedSessionUnderGate(t *testing.T) {
t.Setenv("XDG_DATA_HOME", t.TempDir())
t.Setenv("ZERO_OPENAI_TURN_SESSION", "1")
cwd := t.TempDir()
var heads, posts atomic.Int64
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case r.Method == http.MethodHead:
heads.Add(1)
w.WriteHeader(http.StatusMethodNotAllowed)
case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/chat/completions"):
posts.Add(1)
w.Header().Set("Content-Type", "text/event-stream")
_, _ = w.Write([]byte("data: {\"choices\":[{\"delta\":{\"content\":\"done\"}}]}\n\n"))
_, _ = w.Write([]byte("data: [DONE]\n\n"))
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer server.Close()
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"exec", "say done"}, &stdout, &stderr, appDeps{
getwd: func() (string, error) { return cwd, nil },
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return config.ResolvedConfig{
ActiveProvider: "official-openai",
Provider: config.ProviderProfile{
Name: "official-openai",
ProviderKind: config.ProviderKindOpenAI,
BaseURL: server.URL,
APIKey: "sk-exec-test",
Model: "pr8-exec-model",
},
}, nil
},
newProvider: func(profile config.ProviderProfile) (zeroruntime.Provider, error) {
// Pin a keep-alive transport so the probe fires on every platform:
// the shared default transport disables keep-alives on macOS, where
// the session correctly skips the prewarm.
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.DisableKeepAlives = false
return openai.New(openai.Options{
APIKey: profile.APIKey,
BaseURL: profile.BaseURL,
Model: profile.Model,
HTTPClient: &http.Client{Transport: transport},
})
},
})
if exitCode != exitSuccess {
t.Fatalf("exitCode = %d stdout=%s stderr=%s", exitCode, stdout.String(), stderr.String())
}
if got := posts.Load(); got < 1 {
t.Fatalf("server saw %d chat-completions POSTs, want >= 1", got)
}
// The prewarm probe is asynchronous; it fires well before the model turn
// finishes, but poll rather than assuming scheduling order. The deadline
// exceeds the production prewarmTimeout (3s) so a slow probe cannot flake
// this assertion.
deadline := time.Now().Add(5 * time.Second)
for heads.Load() == 0 && time.Now().Before(deadline) {
time.Sleep(10 * time.Millisecond)
}
if got := heads.Load(); got != 1 {
t.Fatalf("server saw %d prewarm HEAD probes, want exactly 1", got)
}
}
// TestRunExecGateOnFallsBackForFakeProvider proves fallback safety: with the
// gate on but a provider that is not the concrete *openai.Provider, the run
// proceeds on the default path exactly as today.
func TestRunExecGateOnFallsBackForFakeProvider(t *testing.T) {
t.Setenv("XDG_DATA_HOME", t.TempDir())
t.Setenv("ZERO_OPENAI_TURN_SESSION", "1")
cwd := t.TempDir()
var builds int
var stdout, stderr bytes.Buffer
exitCode := runWithDeps([]string{"exec", "--model", "claude-haiku-4.5", "hello"}, &stdout, &stderr, appDeps{
getwd: func() (string, error) { return cwd, nil },
resolveConfig: func(_ string, overrides config.Overrides) (config.ResolvedConfig, error) {
model := "claude-haiku-4.5"
if overrides.Provider.Model != "" {
model = overrides.Provider.Model
}
cfg := execResolvedConfig()
cfg.Provider.ProviderKind = config.ProviderKindAnthropic
cfg.Provider.Model = model
cfg.MaxTurns = 3
return cfg, nil
},
newProvider: func(config.ProviderProfile) (zeroruntime.Provider, error) {
builds++
return &escalatingExecProvider{}, nil
},
})
if exitCode != exitSuccess {
t.Fatalf("exitCode = %d stdout=%s stderr=%s", exitCode, stdout.String(), stderr.String())
}
if builds != 1 {
t.Fatalf("newProvider called %d times, want 1 (default path, no optimized session)", builds)
}
}
+1 -5
View File
@@ -153,9 +153,6 @@ func (writer *execEventWriter) toolResult(result agent.ToolResult) {
if len(result.Meta) > 0 {
payload["meta"] = result.Meta
}
if result.Truncated {
payload["truncated"] = true
}
if result.Redacted {
payload["redacted"] = true
}
@@ -169,8 +166,7 @@ func (writer *execEventWriter) toolResult(result agent.ToolResult) {
return
}
if writer.format == execOutputStreamJSON {
output, surfaceTruncated := truncateForStreamJSONOutput(result.Output)
truncated := result.Truncated || surfaceTruncated
output, truncated := truncateForStreamJSONOutput(result.Output)
event := streamjson.Event{
Type: streamjson.EventToolResult,
RunID: writer.runID,
-36
View File
@@ -1,12 +1,8 @@
package cli
import (
"bytes"
"encoding/json"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/agent"
"github.com/Gitlawb/zero/internal/tools"
)
@@ -20,35 +16,3 @@ func TestStreamJSONSideEffectReportsNoneForControlTool(t *testing.T) {
t.Fatalf("streamJSONSideEffect(escalate_model) = %q, want none", got)
}
}
func TestExecWriterPropagatesToolResultTruncation(t *testing.T) {
for _, format := range []execOutputFormat{execOutputJSON, execOutputStreamJSON} {
t.Run(string(format), func(t *testing.T) {
var stdout, stderr bytes.Buffer
writer := execEventWriter{
stdout: &stdout,
stderr: &stderr,
format: format,
runID: "run_budget",
streamedText: &strings.Builder{},
}
writer.toolResult(agent.ToolResult{
ToolCallID: "call_budget",
Name: "read_file",
Status: tools.StatusOK,
Output: "bounded output",
Truncated: true,
})
if writer.err != nil {
t.Fatalf("toolResult: %v", writer.err)
}
var payload map[string]any
if err := json.Unmarshal(bytes.TrimSpace(stdout.Bytes()), &payload); err != nil {
t.Fatalf("decode output %q: %v", stdout.String(), err)
}
if payload["truncated"] != true {
t.Fatalf("truncated = %#v, want true; payload=%#v", payload["truncated"], payload)
}
})
}
}
-159
View File
@@ -53,22 +53,6 @@ func runProvidersUse(args []string, stdout io.Writer, stderr io.Writer, deps app
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
// SetActiveProvider only ever matches profiles persisted in config.json
// (see config.ProviderPersisted), but a provider can be visible in
// `zero providers list`/the TUI picker purely because Resolve()
// synthesized it in-memory from an ambient env var (e.g. OPENAI_API_KEY)
// without ever writing a row to disk. Without this check, switching to
// that provider by name always fails with a confusing "not found" even
// though it is genuinely usable this session (issue #707).
persisted, err := config.ProviderPersisted(configPath, options.name)
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
if !persisted {
if exit, handled := reportUnpersistedProviderUse(stdout, stderr, deps, options, configPath); handled {
return exit
}
}
cfg, err := config.SetActiveProvider(configPath, options.name)
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
@@ -368,22 +352,6 @@ func runProvidersRemove(args []string, stdout io.Writer, stderr io.Writer, deps
return writeAppError(stderr, err.Error(), exitCrash)
}
name := options.names[0]
// RemoveProvider only ever matches profiles persisted in config.json (see
// config.ProviderPersisted), but a provider can be visible in
// `zero providers list`/the TUI picker purely because Resolve()
// synthesized it in-memory from an ambient env var (e.g. OPENAI_API_KEY)
// without ever writing a row to disk. Without this check, deleting that
// provider by name always fails with a confusing "not found" even though
// it is genuinely visible/usable this session (issue #707).
persisted, err := config.ProviderPersisted(configPath, name)
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
if !persisted {
if exit, handled := reportUnpersistedProviderRemove(stdout, stderr, deps, name, options.json, configPath); handled {
return exit
}
}
cfg, err := config.RemoveProvider(configPath, name)
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
@@ -460,16 +428,6 @@ func runProvidersRename(args []string, stdout io.Writer, stderr io.Writer, deps
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
oldName := options.names[0]
persisted, err := config.ProviderPersisted(configPath, oldName)
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
}
if !persisted {
if exit, handled := reportUnpersistedProviderRename(stdout, stderr, deps, oldName, options.json, configPath); handled {
return exit
}
}
cfg, err := config.RenameProvider(configPath, options.names[0], options.names[1])
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
@@ -489,120 +447,3 @@ func runProvidersRename(args []string, stdout io.Writer, stderr io.Writer, deps
}
return exitSuccess
}
// providerResolvedByName reports whether name matches a provider in a
// resolved provider list — used to tell a genuinely unknown name (a typo)
// apart from a real, env-derived provider that just has no config.json row.
func providerResolvedByName(providers []config.ProviderProfile, name string) bool {
name = strings.TrimSpace(name)
for _, provider := range providers {
if strings.EqualFold(strings.TrimSpace(provider.Name), name) {
return true
}
}
return false
}
// reportUnpersistedProviderUse handles `zero providers use <name>` for a
// provider that is not persisted in config.json. If it's not resolvable at
// all (an unknown/misspelled name), it returns handled=false so the caller
// falls through to SetActiveProvider's real "not found" error. If it IS
// resolvable — an env-derived profile (e.g. ambient OPENAI_API_KEY) —
// SetActiveProvider would only ever fail "not found" against it, so this
// reports the situation plainly instead of that confusing error (issue
// #707).
func reportUnpersistedProviderUse(stdout, stderr io.Writer, deps appDeps, options providerUseOptions, configPath string) (int, bool) {
resolved, exitCode := resolveCommandCenterConfig(stderr, deps)
if exitCode != exitSuccess {
// resolveCommandCenterConfig already wrote its own error to stderr;
// stop here instead of letting the caller try SetActiveProvider too.
return exitCode, true
}
if !providerResolvedByName(resolved.Providers, options.name) {
return exitCode, false
}
message := fmt.Sprintf(
"Provider %q is not saved in config.json (likely set via an environment variable), so there is no saved profile to switch to.\nIt is available whenever its environment variable is set, but is only active when selected (for example via ZERO_PROVIDER); unset its environment variable to stop Zero from detecting it automatically.",
options.name,
)
if options.json {
if err := writePrettyJSON(stdout, map[string]any{
"activeProvider": resolved.ActiveProvider,
"configPath": configPath,
"persisted": false,
"message": message,
}); err != nil {
return exitCrash, true
}
return exitSuccess, true
}
if _, err := fmt.Fprintln(stdout, message); err != nil {
return exitCrash, true
}
return exitSuccess, true
}
// reportUnpersistedProviderRemove handles `zero providers remove <name>` for
// a provider that is not persisted in config.json, mirroring the TUI
// provider manager's delete handling (internal/tui/provider_manager.go). If
// name isn't resolvable at all, it returns handled=false so the caller falls
// through to RemoveProvider's real "not found" error.
func reportUnpersistedProviderRemove(stdout, stderr io.Writer, deps appDeps, name string, jsonOutput bool, configPath string) (int, bool) {
resolved, exitCode := resolveCommandCenterConfig(stderr, deps)
if exitCode != exitSuccess {
return exitCode, true
}
if !providerResolvedByName(resolved.Providers, name) {
return exitCode, false
}
message := fmt.Sprintf(
"Provider %q is not saved in config.json (likely set via an environment variable) — nothing to remove there.\nUnset its environment variable to stop Zero from detecting it automatically.",
name,
)
if jsonOutput {
if err := writePrettyJSON(stdout, map[string]any{
"removed": "",
"keyRemoved": false,
"activeProvider": resolved.ActiveProvider,
"configPath": configPath,
"persisted": false,
"message": message,
}); err != nil {
return exitCrash, true
}
return exitSuccess, true
}
if _, err := fmt.Fprintln(stdout, message); err != nil {
return exitCrash, true
}
return exitSuccess, true
}
func reportUnpersistedProviderRename(stdout, stderr io.Writer, deps appDeps, name string, jsonOutput bool, configPath string) (int, bool) {
resolved, exitCode := resolveCommandCenterConfig(stderr, deps)
if exitCode != exitSuccess {
return exitCode, true
}
if !providerResolvedByName(resolved.Providers, name) {
return exitCode, false
}
message := fmt.Sprintf(
"Provider %q is not saved in config.json (likely set via an environment variable), so there is no saved profile to rename.",
name,
)
if jsonOutput {
if err := writePrettyJSON(stdout, map[string]any{
"renamed": nil,
"configPath": configPath,
"persisted": false,
"message": message,
}); err != nil {
return exitCrash, true
}
return exitSuccess, true
}
if _, err := fmt.Fprintln(stdout, message); err != nil {
return exitCrash, true
}
return exitSuccess, true
}
-105
View File
@@ -75,111 +75,6 @@ func TestRunProvidersUseJSONIncludesActiveProviderAndConfigPath(t *testing.T) {
}
}
func TestRunProvidersUseSurfacesMalformedConfig(t *testing.T) {
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
if err := os.WriteFile(configPath, []byte("{"), 0o600); err != nil {
t.Fatalf("write malformed config: %v", err)
}
exitCode := runWithDeps([]string{"providers", "use", "openai"}, &stdout, &stderr, providerSetupDeps(configPath))
if exitCode != exitCrash {
t.Fatalf("exit code = %d, want %d", exitCode, exitCrash)
}
if !strings.Contains(stderr.String(), "invalid config JSON") {
t.Fatalf("stderr = %q, want malformed-config error", stderr.String())
}
}
func TestRunProvidersUseEnvDerivedJSONIncludesConfigPath(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "use", "openai", "--json"}, &stdout, &stderr, providerSetupDeps(configPath))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
var payload struct {
ConfigPath string `json:"configPath"`
Persisted bool `json:"persisted"`
}
if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil {
t.Fatalf("decode JSON: %v\n%s", err, stdout.String())
}
if payload.ConfigPath != configPath || payload.Persisted {
t.Fatalf("unexpected payload: %+v", payload)
}
}
func TestRunProvidersRemoveEnvDerivedJSONKeepsSchema(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "remove", "openai", "--json"}, &stdout, &stderr, providerSetupDeps(configPath))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
var payload struct {
Removed string `json:"removed"`
KeyRemoved bool `json:"keyRemoved"`
Persisted bool `json:"persisted"`
ConfigPath string `json:"configPath"`
}
if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil {
t.Fatalf("decode JSON: %v\n%s", err, stdout.String())
}
if payload.Removed != "" || payload.KeyRemoved || payload.Persisted || payload.ConfigPath != configPath {
t.Fatalf("unexpected payload: %+v", payload)
}
}
func TestRunProvidersRenameEnvDerivedExplainsNoSavedProfile(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "rename", "openai", "renamed"}, &stdout, &stderr, providerSetupDeps(configPath))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
if !strings.Contains(stdout.String(), "no saved profile to rename") {
t.Fatalf("stdout = %q, want unpersisted explanation", stdout.String())
}
}
func TestRunProvidersRenameEnvDerivedJSONKeepsSchema(t *testing.T) {
t.Setenv("OPENAI_API_KEY", "sk-env")
var stdout, stderr bytes.Buffer
configPath := filepath.Join(t.TempDir(), "config.json")
writeProviderOnboardingConfig(t, configPath, config.FileConfig{})
exitCode := runWithDeps([]string{"providers", "rename", "openai", "renamed", "--json"}, &stdout, &stderr, providerSetupDeps(configPath))
if exitCode != exitSuccess {
t.Fatalf("exit code = %d, want %d: %s", exitCode, exitSuccess, stderr.String())
}
var payload struct {
Renamed *struct{} `json:"renamed"`
ConfigPath string `json:"configPath"`
Persisted bool `json:"persisted"`
}
if err := json.Unmarshal(stdout.Bytes(), &payload); err != nil {
t.Fatalf("decode JSON: %v\n%s", err, stdout.String())
}
if payload.Renamed != nil || payload.ConfigPath != configPath || payload.Persisted {
t.Fatalf("unexpected payload: %+v", payload)
}
}
func TestRunProvidersUseRejectsUsageErrors(t *testing.T) {
cases := []struct {
name string
-15
View File
@@ -1001,18 +1001,3 @@ func sandboxPolicyCapabilityStatus(capabilities []sandbox.BackendCapability, key
}
return ""
}
func TestApplyConfiguredSandboxPolicyEnabledFalseDisables(t *testing.T) {
disabled := false
if got := applyConfiguredSandboxPolicy(sandbox.DefaultPolicy(), config.SandboxConfig{Enabled: &disabled}); got.Mode != sandbox.ModeDisabled {
t.Fatalf("enabled:false must yield ModeDisabled, got %q", got.Mode)
}
// Omitted (nil) and explicit true keep the default enforcing mode.
if got := applyConfiguredSandboxPolicy(sandbox.DefaultPolicy(), config.SandboxConfig{}); got.Mode == sandbox.ModeDisabled {
t.Fatal("omitted enabled must not disable the sandbox")
}
enabled := true
if got := applyConfiguredSandboxPolicy(sandbox.DefaultPolicy(), config.SandboxConfig{Enabled: &enabled}); got.Mode == sandbox.ModeDisabled {
t.Fatal("enabled:true must not disable the sandbox")
}
}
+3 -3
View File
@@ -216,10 +216,10 @@ func parseNonEmptySessionsFlag(flag string, value string) (string, error) {
func parseSessionKindFlag(value string) (sessions.SessionKind, error) {
kind := sessions.SessionKind(strings.ToLower(strings.TrimSpace(value)))
switch kind {
case sessions.SessionKindFork, sessions.SessionKindChild, sessions.SessionKindSide, sessions.SessionKindSpecDraft, sessions.SessionKindSpecImpl:
case sessions.SessionKindFork, sessions.SessionKindChild, sessions.SessionKindSpecDraft, sessions.SessionKindSpecImpl:
return kind, nil
default:
return "", execUsageError{fmt.Sprintf("invalid --kind %q. Expected fork, child, side, spec-draft, or spec-impl.", value)}
return "", execUsageError{fmt.Sprintf("invalid --kind %q. Expected fork, child, spec-draft, or spec-impl.", value)}
}
}
@@ -535,7 +535,7 @@ Commands:
Flags:
--json Print JSON output
--kind <kind> Filter list by fork, child, side, spec-draft, or spec-impl
--kind <kind> Filter list by fork, child, spec-draft, or spec-impl
--sequence <n> Rewind target sequence (rewind-plan, rewind)
--event <id> Rewind target event id (rewind-plan, rewind)
--exclude-target Drop the target event (rewind-plan, rewind)
-14
View File
@@ -233,9 +233,6 @@ func TestRunSessionsListFiltersByKind(t *testing.T) {
}); err != nil {
t.Fatal(err)
}
if _, err := store.Create(sessions.CreateInput{SessionID: "side", SessionKind: sessions.SessionKindSide, Title: "BTW side"}); err != nil {
t.Fatal(err)
}
var stdout bytes.Buffer
var stderr bytes.Buffer
@@ -252,17 +249,6 @@ func TestRunSessionsListFiltersByKind(t *testing.T) {
t.Fatalf("filtered sessions output = %q", output)
}
stdout.Reset()
stderr.Reset()
exitCode = runWithDeps([]string{"sessions", "list", "--kind", "side"}, &stdout, &stderr, appDeps{
newSessionStore: func() *sessions.Store {
return store
},
})
if exitCode != exitSuccess || !strings.Contains(stdout.String(), "side") || strings.Contains(stdout.String(), "regular") {
t.Fatalf("sessions list --kind side exit=%d stdout=%q stderr=%q", exitCode, stdout.String(), stderr.String())
}
stdout.Reset()
stderr.Reset()
exitCode = runWithDeps([]string{"sessions", "tree", "draft", "--kind", "spec-draft"}, &stdout, &stderr, appDeps{
-20
View File
@@ -93,16 +93,6 @@ func Resolve(options ResolveOptions) (ResolvedConfig, error) {
if err != nil {
return ResolvedConfig{}, err
}
// Sandbox.Enabled is NOT accepted from a provider command, for the same
// reason project config cannot set it (see mergeProjectConfig): only
// global config and the CLI may turn the sandbox off. A provider command
// is an arbitrary executable named in config, and LoadProviderCommand
// parses its stdout into a full FileConfig — so without this, a command
// returning a valid provider plus {"sandbox":{"enabled":false}} would
// disable the very sandbox meant to constrain what it can do. Cleared
// here rather than in mergeConfig because that helper is shared with the
// trusted global-config merge, which must keep honouring the setting.
commandConfig.Sandbox.Enabled = nil
mergeConfig(&cfg, commandConfig)
}
@@ -226,9 +216,6 @@ func mergeConfig(dst *FileConfig, src FileConfig) {
mergeProvider(dst, provider)
}
mergeMCPConfig(&dst.MCP, src.MCP, true)
if src.Sandbox.Enabled != nil {
dst.Sandbox.Enabled = src.Sandbox.Enabled
}
if network := strings.TrimSpace(src.Sandbox.Network); network != "" {
dst.Sandbox.Network = network
}
@@ -286,10 +273,6 @@ func mergeProjectConfig(dst *FileConfig, src FileConfig) error {
if err := mergeProjectMCPConfig(&dst.MCP, src.MCP); err != nil {
return err
}
// Sandbox.Enabled is intentionally NOT merged from project config: a cloned
// repo's .zero/config.json must not be able to disable the sandbox that
// constrains it. Only global config and CLI can turn the sandbox off.
//
// Sandbox.AdditionalWriteRoots is intentionally NOT merged from project
// config: a cloned repo's .zero/config.json must not be able to grant
// itself write access outside the workspace. Global config and CLI flags
@@ -705,9 +688,6 @@ func applyOverrides(cfg *FileConfig, overrides Overrides) {
if overrides.MaxTurns > 0 {
cfg.MaxTurns = overrides.MaxTurns
}
if overrides.Sandbox.Enabled != nil {
cfg.Sandbox.Enabled = overrides.Sandbox.Enabled
}
if overrides.Sandbox.BlockUnixSockets {
cfg.Sandbox.BlockUnixSockets = true
}
-79
View File
@@ -1744,85 +1744,6 @@ func TestResolveSandboxNetworkProjectConfigCannotWeaken(t *testing.T) {
}
}
func TestResolveSandboxEnabledUserConfigDisables(t *testing.T) {
userPath := writeConfig(t, `{"sandbox": {"enabled": false}}`)
resolved, err := Resolve(ResolveOptions{UserConfigPath: userPath, Env: map[string]string{}})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if resolved.Sandbox.Enabled == nil || *resolved.Sandbox.Enabled {
t.Fatalf("user config enabled:false must resolve to a false pointer, got %v", resolved.Sandbox.Enabled)
}
}
func TestResolveSandboxDisableIgnoredFromProjectConfig(t *testing.T) {
// Security: a cloned repo's project config must NOT be able to disable the
// sandbox that constrains it. Only global config / CLI can turn it off.
userPath := writeConfig(t, `{}`)
projectPath := writeConfig(t, `{"sandbox": {"enabled": false}}`)
resolved, err := Resolve(ResolveOptions{UserConfigPath: userPath, ProjectConfigPath: projectPath, Env: map[string]string{}})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if resolved.Sandbox.Enabled != nil {
t.Fatalf("project config enabled:false must be ignored, got %v — a repo could disable the sandbox", *resolved.Sandbox.Enabled)
}
}
func TestResolveSandboxEnabledIgnoredFromProviderCommand(t *testing.T) {
// Security: a provider command is an arbitrary executable named in config, and
// its stdout is parsed into a full FileConfig — so it must NOT be able to reach
// Sandbox.Enabled, in either direction. Same rule as project config: only global
// config / CLI may turn the sandbox off.
for _, tc := range []struct {
name string
value string
}{
{name: "cannot disable", value: "false"},
{name: "cannot enable", value: "true"},
} {
t.Run(tc.name, func(t *testing.T) {
command := writeCommand(t, commandScript{
Stdout: `{"activeProvider":"cmd","providers":[{"name":"cmd","provider":"openai","apiKey":"sk-command","model":"gpt-command"}],"sandbox":{"enabled":` + tc.value + `}}`,
})
resolved, err := Resolve(ResolveOptions{ProviderCommand: command, Env: map[string]string{}})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
// The provider itself must still be applied — only the sandbox toggle is dropped.
if resolved.ActiveProvider != "cmd" {
t.Fatalf("ActiveProvider = %q, want cmd (provider command must still apply)", resolved.ActiveProvider)
}
if resolved.Sandbox.Enabled != nil {
t.Fatalf("provider command set sandbox.enabled=%s, got %v — a provider command could toggle the sandbox", tc.value, *resolved.Sandbox.Enabled)
}
})
}
}
func TestResolveSandboxEnabledCLIOverride(t *testing.T) {
// A CLI/programmatic override (applied after every config source) must be able
// to disable the sandbox, even with no config file.
disabled := false
resolved, err := Resolve(ResolveOptions{Env: map[string]string{}, Overrides: Overrides{Sandbox: SandboxConfig{Enabled: &disabled}}})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if resolved.Sandbox.Enabled == nil || *resolved.Sandbox.Enabled {
t.Fatalf("CLI override enabled:false must disable the sandbox, got %v", resolved.Sandbox.Enabled)
}
// An explicit true override must win over a lower-precedence global-config false.
userPath := writeConfig(t, `{"sandbox": {"enabled": false}}`)
enabled := true
resolved2, err := Resolve(ResolveOptions{UserConfigPath: userPath, Env: map[string]string{}, Overrides: Overrides{Sandbox: SandboxConfig{Enabled: &enabled}}})
if err != nil {
t.Fatalf("Resolve: %v", err)
}
if resolved2.Sandbox.Enabled == nil || !*resolved2.Sandbox.Enabled {
t.Fatalf("CLI override enabled:true must win over config false, got %v", resolved2.Sandbox.Enabled)
}
}
func TestResolveNotifyValid(t *testing.T) {
path := writeConfig(t, `{"notify":{"mode":"both","focusMode":"always"}}`)
resolved, err := Resolve(ResolveOptions{UserConfigPath: path, Env: map[string]string{}})
-6
View File
@@ -62,12 +62,6 @@ func HasProviderProfile(profile ProviderProfile) bool {
}
type SandboxConfig struct {
// Enabled turns the sandbox off when set to false (`"sandbox": {"enabled":
// false}`). A pointer distinguishes an explicit false (disable) from an
// omitted key (keep the default: enabled). Honored from the GLOBAL user
// config and CLI only — deliberately NOT project config, so a cloned repo
// cannot disable the sandbox that constrains it. nil / true keep enforcement.
Enabled *bool `json:"enabled,omitempty"`
// Network controls whether shell commands classified as network-touching
// (curl, git push, package installs, …) are allowed: "allow" or "deny".
// Empty keeps the built-in default (deny). Without this knob the engine's
-27
View File
@@ -177,33 +177,6 @@ func SetActiveProvider(path string, name string) (FileConfig, error) {
return FileConfig{}, fmt.Errorf("provider %q not found", name)
}
// ProviderPersisted reports whether a provider profile named name actually has
// a row in the config file at path. A provider can appear in the resolved/
// in-memory provider list without ever being written to config.json — e.g.
// applyProviderEnv synthesizes an "openai" profile purely from an ambient
// OPENAI_API_KEY environment variable on every Resolve() call, without ever
// persisting it. RemoveProvider/SetActiveProvider/SetProviderModel only ever
// look at what's on disk, so a caller offering to mutate a provider by name
// should check this first: "not on disk" needs different handling (nothing to
// persist/remove there) than a name that doesn't exist anywhere at all.
func ProviderPersisted(path string, name string) (bool, error) {
path = strings.TrimSpace(path)
name = strings.TrimSpace(name)
if path == "" || name == "" {
return false, nil
}
cfg, err := loadConfigFile(path)
if err != nil {
return false, err
}
for _, provider := range cfg.Providers {
if strings.EqualFold(strings.TrimSpace(provider.Name), name) {
return true, nil
}
}
return false, nil
}
// RemoveProvider deletes the named provider profile from the config at path.
// When the removed profile was active, activeProvider hands off to the first
// remaining provider (or clears when none remain) so the config never points at
-153
View File
@@ -1,153 +0,0 @@
// Package execprofile defines named execution profiles: bundles of loop-posture
// knobs (turn budget, reasoning effort, self-correction, escalation triggers)
// that tune how hard the agent loop tries, independent of which model runs.
// Profiles COMPOSE with --mode: a mode answers "what runs" (model, tools),
// a profile answers "how hard the loop tries". Selection precedence is
// explicit flag > --mode > profile, enforced by the callers applying profiles
// last with the same fill-only-if-unset rule modes use.
//
// The catalog is deliberately tiny and value-stable: balanced is the empty
// profile (selecting it leaves the run's options, budget, and loop behavior
// identical to an unflagged run — asserted by test; only the opt-in trace and
// bench artifacts record the selected profile name, by design, so captures are
// self-describing), fast trades turn budget and effort for latency with a
// one-shot escalation back to the displaced posture as the safety net, and
// thorough raises the budget and arms full self-correction.
package execprofile
import (
"sort"
"strings"
"github.com/Gitlawb/zero/internal/agent"
"github.com/Gitlawb/zero/internal/sandbox"
)
// Profile is a named execution posture. Zero-valued fields inherit the run's
// existing value (flag, mode, config, or built-in default) — a profile only
// ever fills knobs the caller left unset, except MaxTurns which REPLACES the
// resolved budget (that displacement is what escalation restores).
type Profile struct {
// Name identifies the profile in traces, help text, and diagnostics.
Name string
// MaxTurns replaces the resolved per-run tool-turn budget when > 0 and the
// user did not pass an explicit --max-turns. The displaced resolved value
// becomes the escalation target (see Policy).
MaxTurns int
// ReasoningEffort fills the run's effort when the user left it unset. It
// still flows through the model-supported gating at the call site, so an
// unsupported level degrades exactly like an explicit flag would.
ReasoningEffort string
// SelfCorrect arms the post-edit verify-and-correct loop. Profiles can only
// turn it ON (the flag is presence-only, so false is indistinguishable from
// unset and must never override an explicit opt-in).
SelfCorrect bool
// Escalation triggers. All zero means the profile never escalates and
// Policy returns nil (the loop stays byte-identical to a nil-profile run).
// Targets are NOT part of the catalog: escalation restores the values the
// profile displaced at selection time, never invented ones, so targets are
// stamped by Policy from what the caller measured.
EscalateOnToolFailureStreak int
EscalateOnCompletionUncertain int
EscalateOnSelfCorrectFailure bool
EscalateOnRiskyMutation sandbox.RiskLevel
}
// The catalog. Values marked provisional are floors/ceilings derived from the
// Phase 0 baseline's read-class evidence (successful nav runs used single-digit
// turns; the mutating classes produced no successful samples to tune from) and
// are expected to be re-tuned from the post-oracle-fix re-capture. The
// escalation safety net is what makes shipping provisional Fast values safe:
// a run that hits trouble gets its displaced budget back mid-run.
var (
// Balanced is the default posture: empty on purpose. Selecting it changes
// nothing — the no-regression invariant of the whole feature.
Balanced = Profile{Name: "balanced"}
// Fast starts cheap (30 turns, low effort) and arms every escalation
// trigger so a struggling run restores its displaced posture: two
// same-tool retriable failures in a row, a second uncertain completion
// evaluation, a failing self-correction cycle, or a critical-risk
// mutation. The risk threshold is deliberately Critical, not High: the
// sandbox classifies EVERY shell command as at least high risk before any
// command analysis, so a High trigger would fire on the first `go test`
// of virtually any coding task and spend the one-shot on turn one.
// Critical marks the genuinely scary categories (destructive commands,
// piped installers, out-of-workspace writes, network mutations).
Fast = Profile{
Name: "fast",
MaxTurns: 30,
ReasoningEffort: "low",
EscalateOnToolFailureStreak: 2,
EscalateOnCompletionUncertain: 2,
EscalateOnSelfCorrectFailure: true,
EscalateOnRiskyMutation: sandbox.RiskCritical,
}
// Thorough doubles the default budget, asks for high effort, and arms the
// full post-edit verify-and-correct loop. Already the maximum posture, so
// it has nothing to escalate to.
Thorough = Profile{
Name: "thorough",
MaxTurns: 160,
ReasoningEffort: "high",
SelfCorrect: true,
}
)
var catalog = map[string]Profile{
Balanced.Name: Balanced,
Fast.Name: Fast,
Thorough.Name: Thorough,
}
// Lookup resolves a profile by name, case-insensitively and ignoring
// surrounding whitespace.
func Lookup(name string) (Profile, bool) {
profile, ok := catalog[strings.ToLower(strings.TrimSpace(name))]
return profile, ok
}
// Names returns the catalog's profile names, sorted, for usage errors and help.
func Names() []string {
names := make([]string, 0, len(catalog))
for name := range catalog {
names = append(names, name)
}
sort.Strings(names)
return names
}
// Policy projects the profile into the loop-facing ProfilePolicy.
//
// displacedMaxTurns is the turn budget this profile displaced at selection time
// (what the run would have used without it) and becomes the escalation's
// restore target; pass 0 when the profile did not displace the budget (e.g.
// the user pinned --max-turns explicitly) so escalation leaves the ceiling
// untouched. effortFilled reports whether the profile actually filled the
// run's reasoning effort (it backs off when the user set one explicitly); the
// displaced effort is then "" (the provider default) by construction, which
// the escalation restores via RestoreDefaultEffort — a plain "" target would
// mean "leave untouched".
//
// Profiles with no armed triggers return nil: the loop treats a nil policy as
// "no profile" (no observation, no counters), which keeps balanced and
// thorough runs byte-identical to the same knob values set by hand.
func (p Profile) Policy(displacedMaxTurns int, effortFilled bool) *agent.ProfilePolicy {
if p.EscalateOnToolFailureStreak == 0 && p.EscalateOnCompletionUncertain == 0 &&
!p.EscalateOnSelfCorrectFailure && p.EscalateOnRiskyMutation == "" {
return nil
}
return &agent.ProfilePolicy{
Name: p.Name,
Escalate: &agent.PostureEscalation{
MaxTurns: displacedMaxTurns,
RestoreDefaultEffort: effortFilled,
OnToolFailureStreak: p.EscalateOnToolFailureStreak,
OnCompletionUncertain: p.EscalateOnCompletionUncertain,
OnSelfCorrectFailure: p.EscalateOnSelfCorrectFailure,
OnRiskyMutation: p.EscalateOnRiskyMutation,
},
}
}
-131
View File
@@ -1,131 +0,0 @@
package execprofile
import (
"reflect"
"testing"
"github.com/Gitlawb/zero/internal/sandbox"
)
// The no-regression invariant of the whole feature: balanced must be the empty
// profile (nothing but a name), so selecting it cannot change a run.
func TestBalancedProfileIsEmpty(t *testing.T) {
profile, ok := Lookup("balanced")
if !ok {
t.Fatal("balanced must exist in the catalog")
}
if profile.Name != "balanced" {
t.Fatalf("Name = %q, want %q", profile.Name, "balanced")
}
profile.Name = ""
if !reflect.DeepEqual(profile, Profile{}) {
t.Fatalf("balanced must be zero-valued apart from its name, got %+v", profile)
}
if policy := Balanced.Policy(80, false); policy != nil {
t.Fatalf("balanced Policy must be nil (byte-identical loop), got %+v", policy)
}
}
func TestLookupIsCaseAndSpaceInsensitive(t *testing.T) {
for _, name := range []string{"fast", "Fast", " FAST "} {
if _, ok := Lookup(name); !ok {
t.Fatalf("Lookup(%q) should resolve", name)
}
}
if _, ok := Lookup("turbo"); ok {
t.Fatal("Lookup(turbo) should not resolve")
}
}
func TestNamesAreSorted(t *testing.T) {
want := []string{"balanced", "fast", "thorough"}
if got := Names(); !reflect.DeepEqual(got, want) {
t.Fatalf("Names() = %v, want %v", got, want)
}
}
// Escalation targets must be the displaced values only — the catalog carries
// triggers, the call site supplies what was displaced, and nothing else may
// appear in the escalation.
func TestFastPolicyTargetsDisplacedValuesOnly(t *testing.T) {
policy := Fast.Policy(80, true)
if policy == nil {
t.Fatal("fast must arm an escalation policy")
}
if policy.Name != "fast" {
t.Fatalf("policy.Name = %q, want fast", policy.Name)
}
esc := policy.Escalate
if esc == nil {
t.Fatal("fast policy must carry an escalation")
}
if esc.MaxTurns != 80 {
t.Fatalf("Escalate.MaxTurns = %d, want the displaced 80", esc.MaxTurns)
}
if esc.ReasoningEffort != "" {
t.Fatalf("Escalate.ReasoningEffort = %q, want empty (displaced effort is the provider default by construction)", esc.ReasoningEffort)
}
if !esc.RestoreDefaultEffort {
t.Fatal("effort was profile-filled, so escalation must restore the provider default")
}
if esc.RestoreCompletionGate {
t.Fatal("fast does not touch the completion gate, so escalation must not either")
}
if esc.OnToolFailureStreak != Fast.EscalateOnToolFailureStreak ||
esc.OnCompletionUncertain != Fast.EscalateOnCompletionUncertain ||
esc.OnSelfCorrectFailure != Fast.EscalateOnSelfCorrectFailure ||
esc.OnRiskyMutation != Fast.EscalateOnRiskyMutation {
t.Fatalf("escalation triggers must mirror the catalog, got %+v", esc)
}
}
// The effort restore arms only when the profile actually filled the effort:
// with an explicit user effort the profile backed off, so escalation must not
// clear the user's choice.
func TestFastPolicyExplicitEffortIsNeverRestored(t *testing.T) {
policy := Fast.Policy(80, false)
if policy == nil || policy.Escalate == nil {
t.Fatal("fast must still arm its triggers")
}
if policy.Escalate.RestoreDefaultEffort {
t.Fatal("the profile did not fill the effort, so escalation must leave it alone")
}
}
// A profile that did not displace the budget (explicit --max-turns pinned it)
// must not let escalation move the ceiling at all.
func TestFastPolicyZeroDisplacedLeavesCeilingUntouched(t *testing.T) {
policy := Fast.Policy(0, false)
if policy == nil || policy.Escalate == nil {
t.Fatal("fast must still arm its triggers with a zero displaced budget")
}
if policy.Escalate.MaxTurns != 0 {
t.Fatalf("Escalate.MaxTurns = %d, want 0 (no displaced value to restore)", policy.Escalate.MaxTurns)
}
}
// Thorough is already the maximum posture: no triggers, so no policy — the
// loop must stay byte-identical to the same knobs set by hand.
func TestThoroughPolicyIsNil(t *testing.T) {
if policy := Thorough.Policy(80, true); policy != nil {
t.Fatalf("thorough Policy must be nil, got %+v", policy)
}
}
// Pin the catalog's provisional values so a retune is a deliberate,
// test-visible diff (these floors came from the Phase 0 baseline's read-class
// evidence and are expected to move after the post-oracle-fix re-capture).
func TestCatalogProvisionalValues(t *testing.T) {
if Fast.MaxTurns != 30 || Fast.ReasoningEffort != "low" || Fast.SelfCorrect {
t.Fatalf("fast knobs changed: %+v", Fast)
}
// Critical, not High: the sandbox classifies every shell command as at
// least high risk, so a High trigger would spend the one-shot escalation
// on the first benign `go test` of any coding task.
if Fast.EscalateOnRiskyMutation != sandbox.RiskCritical {
t.Fatalf("fast risky-mutation trigger = %q, want %q", Fast.EscalateOnRiskyMutation, sandbox.RiskCritical)
}
if Thorough.MaxTurns != 160 || Thorough.ReasoningEffort != "high" || !Thorough.SelfCorrect {
t.Fatalf("thorough knobs changed: %+v", Thorough)
}
}
-26
View File
@@ -133,26 +133,6 @@ func discoverAuthorizationServer(ctx context.Context, client *http.Client, baseU
}, nil
}
// validateResolvedEndpoints applies the shared https/loopback endpoint rule to
// every non-empty endpoint in the resolved metadata — configured or discovered
// alike — so discovery metadata can never downgrade an MCP login to an insecure
// or attacker-controlled authorization, token, or registration endpoint.
func validateResolvedEndpoints(metadata authServerMetadata) error {
for _, ep := range []struct{ kind, url string }{
{"authorization", metadata.AuthorizationEndpoint},
{"token", metadata.TokenEndpoint},
{"registration", metadata.RegistrationEndpoint},
} {
if strings.TrimSpace(ep.url) == "" {
continue
}
if err := oauth.ValidateEndpointURL(ep.url); err != nil {
return fmt.Errorf("mcp oauth: %s endpoint: %w", ep.kind, err)
}
}
return nil
}
// resolveAuthorizationServer discovers metadata and applies explicit config
// overrides. Configured endpoints take precedence over discovered ones, and act
// as a fallback when discovery fails or omits a value.
@@ -168,9 +148,6 @@ func resolveAuthorizationServer(ctx context.Context, client *http.Client, baseUR
TokenEndpoint: strings.TrimSpace(cfg.TokenEndpoint),
RegistrationEndpoint: strings.TrimSpace(cfg.RegistrationEndpoint),
}
if err := validateResolvedEndpoints(metadata); err != nil {
return authServerMetadata{}, err
}
return metadata, nil
}
@@ -202,9 +179,6 @@ func resolveAuthorizationServer(ctx context.Context, client *http.Client, baseUR
if strings.TrimSpace(metadata.TokenEndpoint) == "" {
return authServerMetadata{}, errors.New("no token endpoint discovered or configured")
}
if err := validateResolvedEndpoints(metadata); err != nil {
return authServerMetadata{}, err
}
return metadata, nil
}
-16
View File
@@ -73,22 +73,6 @@ func TestResolveEndpointsFallsBackToConfig(t *testing.T) {
}
}
// Discovered MCP metadata that serves an insecure endpoint must be rejected
// before use, so discovery cannot downgrade the MCP login to an
// attacker-controlled origin (issue #511).
func TestResolveAuthorizationServerRejectsInsecureDiscovered(t *testing.T) {
mux := http.NewServeMux()
server := httptest.NewServer(mux)
defer server.Close()
mux.HandleFunc("/.well-known/oauth-authorization-server", func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"issuer":"` + server.URL + `","token_endpoint":"` + server.URL + `/token","authorization_endpoint":"http://evil.example/authorize"}`))
})
_, err := resolveAuthorizationServer(context.Background(), server.Client(), server.URL, OAuthConfig{})
if err == nil || !strings.Contains(err.Error(), "authorization endpoint") {
t.Fatalf("err = %v, want insecure authorization endpoint rejection", err)
}
}
// roundTripperFunc adapts a function to http.RoundTripper so a test can
// observe every outbound request.
type roundTripperFunc func(*http.Request) (*http.Response, error)
-263
View File
@@ -12,54 +12,6 @@ import (
"time"
)
// Mirror internal/cli/exec.go exit codes so wrapper doctor fallback tests assert
// the same CLI contract as the Go doctor command.
const (
wrapperExitSuccess = 0
wrapperExitUsage = 2
wrapperExitDoctor = 1
)
// Canonical Bun recovery copy from bin/zero.js bunRecoveryParagraph() — shared
// by the generic missing-binary path and the doctor text fallback.
const (
bunRecoveryLead = "You installed with Bun, which does not run dependency lifecycle scripts"
bunPmTrustProject = "bun pm trust @gitlawb/zero"
bunPmTrustGlobal = "bun pm -g trust @gitlawb/zero"
bunRecoveryTrustedDeps = `"trustedDependencies": ["@gitlawb/zero"]`
buildFromSourceLead = "If this platform has no prebuilt binary, build from source:"
)
func runWrapperFixture(t *testing.T, wrapperPath string, args ...string) (stdout string, stderr string, exitCode int) {
return runWrapperFixtureWithEnv(t, wrapperPath, nil, args...)
}
func runWrapperFixtureWithEnv(t *testing.T, wrapperPath string, extraEnv []string, args ...string) (stdout string, stderr string, exitCode int) {
t.Helper()
node := requireNode(t)
ctx, cancel := context.WithTimeout(context.Background(), nodeWrapperTimeout())
defer cancel()
command := nodeWrapperCommand(ctx, node, wrapperPath, args...)
command.Env = append(withoutEnvKey(command.Env, "ZERO_LOCAL_CONTROL_HELPERS"), "ZERO_LOCAL_CONTROL_HELPERS=")
command.Env = append(withoutEnvKey(command.Env, "ZERO_WRAPPER_SIMULATE_BUN"), extraEnv...)
var stdoutBuf, stderrBuf strings.Builder
command.Stdout = &stdoutBuf
command.Stderr = &stderrBuf
err := command.Run()
if ctx.Err() != nil {
t.Fatalf("wrapper timed out: %v; stdout: %s stderr: %s", ctx.Err(), stdoutBuf.String(), stderrBuf.String())
}
exitCode = wrapperExitSuccess
if err != nil {
exitErr, ok := err.(*exec.ExitError)
if !ok {
t.Fatalf("wrapper err = %v, want *exec.ExitError; stdout=%s stderr=%s", err, stdoutBuf.String(), stderrBuf.String())
}
exitCode = exitErr.ExitCode()
}
return stdoutBuf.String(), stderrBuf.String(), exitCode
}
func TestPackageBinPointsToNodeWrapper(t *testing.T) {
root := repoRoot(t)
bytes, err := os.ReadFile(filepath.Join(root, "package.json"))
@@ -441,221 +393,6 @@ func nodeArchName() string {
}
}
// Issue #405: `zero doctor` is the diagnostic command, so when the native
// binary is the thing that's broken it must NOT bail with the generic wrapper
// error; that's exactly the blind alley the bug report calls out. Instead it
// emits a doctor-shaped FAIL line for the runtime so the user's own diagnostic
// surfaces the real cause. We assert on the doctor-report shape (so the doctor
// UX matches what the Go-side doctor.Format produces) and on exit 1 (a missing
// binary is still a hard failure, not a pass).
func TestNodeWrapperDoctorReportsMissingNativeBinaryAsDoctorFail(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
stdout, stderr, exitCode := runWrapperFixture(t, wrapperPath, "doctor")
if exitCode != wrapperExitDoctor {
t.Fatalf("doctor exit = %d, want %d; stdout=%s stderr=%s", exitCode, wrapperExitDoctor, stdout, stderr)
}
if strings.TrimSpace(stderr) != "" {
t.Fatalf("doctor text report must go to stdout only, got stderr=%q", stderr)
}
// Doctor-shaped report, not the generic wrapper bail. Matches the shape the
// Go-side doctor.Format emits: a header, "Overall: <pass/fail>", then
// "[<status>] <id> - <message>" lines.
if !strings.Contains(stdout, "Zero doctor report (") {
t.Fatalf("doctor output should start with a doctor report header, got stdout=%q", stdout)
}
if !strings.Contains(stdout, "Overall: fail") {
t.Fatalf("doctor overall must be fail when the native binary is missing, got stdout=%q", stdout)
}
if !strings.Contains(stdout, "[fail] runtime.go") {
t.Fatalf("doctor must report a failing runtime.go check, got stdout=%q", stdout)
}
if !strings.Contains(stdout, "Native zero binary is missing next to the npm wrapper") {
t.Fatalf("doctor must name the actual cause (missing native binary), got stdout=%q", stdout)
}
// The actionable remedy must point at the postinstall script that would fix
// the install, not just "build from source".
if !strings.Contains(stdout, "postinstall.mjs") {
t.Fatalf("doctor remedy should name the postinstall script, got stdout=%q", stdout)
}
// Regression guard for the original blind-alley bug: the doctor path must
// NOT emit the generic wrapper bail (that's what sent users debugging the
// wrong thing).
if strings.Contains(stdout, "[zero] No native binary is available for this install") {
t.Fatalf("doctor must not emit the generic wrapper bail, got stdout=%q", stdout)
}
}
func TestNodeWrapperDoctorJSONReportsMissingNativeBinaryAsJSONFail(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
stdout, stderr, exitCode := runWrapperFixture(t, wrapperPath, "doctor", "--json")
if exitCode != wrapperExitDoctor {
t.Fatalf("doctor --json exit = %d, want %d; stdout=%s stderr=%s", exitCode, wrapperExitDoctor, stdout, stderr)
}
if strings.TrimSpace(stderr) != "" {
t.Fatalf("doctor --json should write machine-readable output to stdout only, got stderr=%q", stderr)
}
var report struct {
GeneratedAt string `json:"generatedAt"`
OK bool `json:"ok"`
Checks []struct {
ID string `json:"id"`
Label string `json:"label"`
Status string `json:"status"`
Message string `json:"message"`
Details map[string]any `json:"details"`
} `json:"checks"`
}
if err := json.Unmarshal([]byte(stdout), &report); err != nil {
t.Fatalf("doctor --json stdout should be valid JSON, got %q: %v", stdout, err)
}
if report.GeneratedAt == "" {
t.Fatalf("doctor --json report should include generatedAt: %#v", report)
}
if report.OK {
t.Fatalf("doctor --json ok = true, want false: %#v", report)
}
if len(report.Checks) != 1 {
t.Fatalf("doctor --json checks length = %d, want 1: %#v", len(report.Checks), report.Checks)
}
check := report.Checks[0]
if check.ID != "runtime.go" || check.Label != "Go runtime" || check.Status != "fail" {
t.Fatalf("doctor --json runtime check = %#v, want failing runtime.go check", check)
}
if !strings.Contains(check.Message, "Native zero binary is missing next to the npm wrapper") {
t.Fatalf("doctor --json must name the actual cause, got %#v", check)
}
remedy, _ := check.Details["remedy"].(string)
if !strings.Contains(remedy, "postinstall.mjs") {
t.Fatalf("doctor --json remedy should name the postinstall script, got %#v", check.Details)
}
}
func TestNodeWrapperDoctorHelpShowsUsage(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
for _, args := range [][]string{{"doctor", "--help"}, {"doctor", "help"}, {"doctor", "-h"}} {
stdout, stderr, exitCode := runWrapperFixture(t, wrapperPath, args...)
if exitCode != wrapperExitSuccess {
t.Fatalf("%v exit = %d, want %d; stdout=%s stderr=%s", args, exitCode, wrapperExitSuccess, stdout, stderr)
}
if strings.TrimSpace(stderr) != "" {
t.Fatalf("%v help must write to stdout only, got stderr=%q", args, stderr)
}
if !strings.Contains(stdout, "Usage:") || !strings.Contains(stdout, "zero doctor [flags]") {
t.Fatalf("%v help output = %q, want doctor usage text", args, stdout)
}
}
}
func TestNodeWrapperDoctorRejectsUnknownFlag(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
stdout, stderr, exitCode := runWrapperFixture(t, wrapperPath, "doctor", "--bogus")
if exitCode != wrapperExitUsage {
t.Fatalf("doctor --bogus exit = %d, want %d; stdout=%s stderr=%s", exitCode, wrapperExitUsage, stdout, stderr)
}
if strings.TrimSpace(stdout) != "" {
t.Fatalf("invalid doctor flag should not write stdout, got %q", stdout)
}
if stderr != "[zero] unknown doctor flag \"--bogus\"\n" {
t.Fatalf("invalid doctor flag stderr = %q", stderr)
}
}
// `doctor --connectivity` (a valid doctor invocation with a trailing flag) must
// take the same doctor-shaped path: parseDoctorArgs accepts --connectivity and
// the missing-binary fallback still reports the runtime failure.
func TestNodeWrapperDoctorWithFlagsStillReportsDoctorFail(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
stdout, stderr, exitCode := runWrapperFixture(t, wrapperPath, "doctor", "--connectivity")
if exitCode != wrapperExitDoctor {
t.Fatalf("doctor --connectivity exit = %d, want %d; stdout=%s stderr=%s", exitCode, wrapperExitDoctor, stdout, stderr)
}
if strings.TrimSpace(stderr) != "" {
t.Fatalf("doctor --connectivity must write to stdout only, got stderr=%q", stderr)
}
if !strings.Contains(stdout, "[fail] runtime.go") {
t.Fatalf("doctor --connectivity must still emit the failing runtime.go line, got stdout=%q", stdout)
}
if strings.Contains(stdout, "[zero] No native binary is available for this install") {
t.Fatalf("doctor --connectivity must not fall back to the generic bail, got stdout=%q", stdout)
}
}
func TestNodeWrapperGenericReportsBunRecoveryWhenInstalledByBun(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
_, stderr, exitCode := runWrapperFixtureWithEnv(t, wrapperPath, []string{"ZERO_WRAPPER_SIMULATE_BUN=1"}, "--version")
if exitCode != wrapperExitDoctor {
t.Fatalf("generic missing-binary exit = %d, want %d; stderr=%s", exitCode, wrapperExitDoctor, stderr)
}
assertBunRecoveryMessage(t, stderr)
if !strings.Contains(stderr, buildFromSourceLead) {
t.Fatalf("generic bun stderr should include build-from-source guidance, got %q", stderr)
}
}
func TestNodeWrapperDoctorReportsBunRecoveryWhenInstalledByBun(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
stdout, stderr, exitCode := runWrapperFixtureWithEnv(t, wrapperPath, []string{"ZERO_WRAPPER_SIMULATE_BUN=1"}, "doctor")
if exitCode != wrapperExitDoctor {
t.Fatalf("doctor bun exit = %d, want %d; stdout=%s stderr=%s", exitCode, wrapperExitDoctor, stdout, stderr)
}
if strings.TrimSpace(stderr) != "" {
t.Fatalf("doctor bun report must go to stdout only, got stderr=%q", stderr)
}
if !strings.Contains(stdout, "[fail] runtime.go") {
t.Fatalf("doctor bun report must still fail runtime.go, got stdout=%q", stdout)
}
assertBunRecoveryMessage(t, stdout)
if !strings.Contains(stdout, buildFromSourceLead) {
t.Fatalf("doctor bun report should reuse generic build-from-source guidance, got stdout=%q", stdout)
}
if strings.Contains(stdout, "If reinstall fails") {
t.Fatalf("doctor bun report must not use a separate build-from-source string, got stdout=%q", stdout)
}
}
func TestNodeWrapperDoctorAndGenericShareBunRecoveryCopy(t *testing.T) {
wrapperPath := copyWrapperFixture(t)
simulateBun := []string{"ZERO_WRAPPER_SIMULATE_BUN=1"}
_, genericStderr, _ := runWrapperFixtureWithEnv(t, wrapperPath, simulateBun, "--version")
doctorStdout, _, _ := runWrapperFixtureWithEnv(t, wrapperPath, simulateBun, "doctor")
genericBun := extractBunRecoveryBlock(genericStderr)
doctorBun := extractBunRecoveryBlock(doctorStdout)
if genericBun != doctorBun {
t.Fatalf("bun recovery copy drift:\ngeneric=%q\ndoctor=%q", genericBun, doctorBun)
}
}
func assertBunRecoveryMessage(t *testing.T, output string) {
t.Helper()
if !strings.Contains(output, bunRecoveryLead) {
t.Fatalf("output should include bun recovery lead, got %q", output)
}
if !strings.Contains(output, bunPmTrustProject) {
t.Fatalf("output should include project bun pm trust guidance, got %q", output)
}
if !strings.Contains(output, bunPmTrustGlobal) {
t.Fatalf("output should include global bun pm trust guidance, got %q", output)
}
if !strings.Contains(output, bunRecoveryTrustedDeps) {
t.Fatalf("output should include trustedDependencies fallback guidance, got %q", output)
}
if !strings.Contains(output, "to your project package.json and reinstall.") {
t.Fatalf("output should mention package.json reinstall fallback, got %q", output)
}
}
func extractBunRecoveryBlock(output string) string {
start := strings.Index(output, bunRecoveryLead)
if start < 0 {
return ""
}
end := strings.Index(output[start:], buildFromSourceLead)
if end < 0 {
return strings.TrimSpace(output[start:])
}
return strings.TrimSpace(output[start : start+end])
}
func TestNodeWrapperLaunchesNativeBinary(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("mock executable fixture uses a POSIX shell script")
+6 -6
View File
@@ -49,9 +49,9 @@ func RequestDeviceCode(ctx context.Context, client *http.Client, cfg Config, now
if err := validateTokenEndpoint(endpoint); err != nil {
return DeviceAuth{}, err
}
// Refuse redirects so a device-authorization redirect can't replay the
// client_id/client_secret POST body to an unvalidated origin.
client = withoutRedirects(client)
if client == nil {
client = http.DefaultClient
}
if now == nil {
now = time.Now
}
@@ -157,9 +157,9 @@ func pollDeviceOnce(ctx context.Context, client *http.Client, cfg Config, device
if err := validateTokenEndpoint(cfg.TokenEndpoint); err != nil {
return Token{}, err
}
// Refuse redirects so a poll redirect can't replay the device_code/secret
// POST body to an unvalidated origin.
client = withoutRedirects(client)
if client == nil {
client = http.DefaultClient
}
form := url.Values{}
form.Set("grant_type", deviceGrantType)
form.Set("device_code", deviceCode)
-25
View File
@@ -2,7 +2,6 @@ package oauth
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"strings"
@@ -29,30 +28,6 @@ func TestRequestDeviceCodeSendsClientSecret(t *testing.T) {
}
}
func TestRequestDeviceCodeRefusesRedirect(t *testing.T) {
endpoint, hits := redirectTrap(t, http.StatusTemporaryRedirect) // 307
_, err := RequestDeviceCode(context.Background(), http.DefaultClient,
Config{ClientID: "c", ClientSecret: "shh", DeviceAuthorizationEndpoint: endpoint}, nil)
if !errors.Is(err, ErrUnsafeRedirect) {
t.Fatalf("RequestDeviceCode err = %v, want ErrUnsafeRedirect", err)
}
if n := hits(); n != 0 {
t.Fatalf("attacker received %d request(s) — client_secret replayed", n)
}
}
func TestPollDeviceOnceRefusesRedirect(t *testing.T) {
endpoint, hits := redirectTrap(t, http.StatusPermanentRedirect) // 308
_, err := pollDeviceOnce(context.Background(), http.DefaultClient,
Config{ClientID: "c", ClientSecret: "shh", TokenEndpoint: endpoint}, "device-code", nil)
if !errors.Is(err, ErrUnsafeRedirect) {
t.Fatalf("pollDeviceOnce err = %v, want ErrUnsafeRedirect", err)
}
if n := hits(); n != 0 {
t.Fatalf("attacker received %d request(s) — device_code/secret replayed", n)
}
}
func TestRequestDeviceCode(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"device_code":"dc","user_code":"WXYZ-1234","verification_uri":"https://x/dev","verification_uri_complete":"https://x/dev?c=WXYZ","expires_in":600,"interval":3}`))
+8 -41
View File
@@ -39,13 +39,6 @@ func BuildAuthorizationURL(cfg Config, pkce PKCE, state, redirectURI string, ext
if endpoint == "" {
return "", errors.New("oauth: no authorization endpoint configured")
}
// Backstop: validate the authorization endpoint at the shared choke point
// both the provider and MCP flows build their browser URL through, so a
// discovery-downgraded endpoint can never open in the browser even if a
// merge site missed it.
if err := ValidateEndpointURL(endpoint); err != nil {
return "", err
}
parsed, err := url.Parse(endpoint)
if err != nil {
return "", fmt.Errorf("oauth: parse authorization endpoint: %w", err)
@@ -90,15 +83,13 @@ func isReservedAuthParam(key string) bool {
}
}
// ValidateEndpointURL refuses an OAuth endpoint that is not https, unless it is
// a loopback host (mirrors oauth2-client.js validateTokenEndpointURL). Every
// credential-bearing endpoint — configured OR learned from discovery — must
// pass this single rule, so discovery metadata can never downgrade a login to
// cleartext or redirect it to an attacker-controlled http origin.
func ValidateEndpointURL(endpoint string) error {
// validateTokenEndpoint refuses to send a credential to a token endpoint that is
// not https, unless it is a loopback host (mirrors oauth2-client.js
// validateTokenEndpointURL). This prevents leaking codes/tokens over cleartext.
func validateTokenEndpoint(endpoint string) error {
parsed, err := url.Parse(trimmed(endpoint))
if err != nil || parsed.Host == "" {
return fmt.Errorf("oauth: invalid endpoint %q", endpoint)
return fmt.Errorf("oauth: invalid token endpoint %q", endpoint)
}
if parsed.Scheme == "https" {
return nil
@@ -109,30 +100,6 @@ func ValidateEndpointURL(endpoint string) error {
return fmt.Errorf("%w: %s", ErrInsecureTokenEndpoint, parsed.Scheme+"://"+parsed.Host)
}
// validateTokenEndpoint checks a token endpoint against the shared endpoint rule
// (kept as a named helper for the token-exchange and device call sites). This
// prevents leaking codes/tokens over cleartext.
func validateTokenEndpoint(endpoint string) error {
return ValidateEndpointURL(endpoint)
}
// withoutRedirects returns a shallow copy of client whose redirect policy
// refuses to follow ANY redirect. A credential-bearing OAuth POST (token
// exchange/refresh, device authorization, device-token poll) must never let a
// 307/308 replay its form body — codes, PKCE verifiers, refresh tokens, client
// secrets — to a target that was never endpoint-validated. The caller's client
// is left unmodified.
func withoutRedirects(client *http.Client) *http.Client {
if client == nil {
client = http.DefaultClient
}
copied := *client
copied.CheckRedirect = func(*http.Request, []*http.Request) error {
return ErrUnsafeRedirect
}
return &copied
}
func isLoopbackHost(host string) bool {
if host == "localhost" {
return true
@@ -193,9 +160,9 @@ func PostToken(ctx context.Context, client *http.Client, tokenEndpoint string, f
if err := validateTokenEndpoint(tokenEndpoint); err != nil {
return Token{}, err
}
// Refuse redirects so a 307/308 from the token endpoint can't replay this
// credential-bearing POST body to another (unvalidated) origin.
client = withoutRedirects(client)
if client == nil {
client = http.DefaultClient
}
if now == nil {
now = time.Now
}
-74
View File
@@ -5,12 +5,10 @@ import (
"crypto/sha256"
"encoding/base64"
"errors"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"sync/atomic"
"testing"
"time"
)
@@ -114,78 +112,6 @@ func TestBuildAuthorizationURLRejectsPlain(t *testing.T) {
}
}
// The shared choke point both flows build their browser URL through must refuse
// an insecure authorization endpoint, so a discovery-downgraded endpoint can
// never open in the browser even if a merge site missed it (issue #511).
func TestBuildAuthorizationURLRejectsInsecureEndpoint(t *testing.T) {
_, err := BuildAuthorizationURL(
Config{AuthorizationEndpoint: "http://evil.example/authorize", ClientID: "c"},
PKCE{Method: MethodS256, Challenge: "chal"},
"state", "http://127.0.0.1/cb", nil,
)
if !errors.Is(err, ErrInsecureTokenEndpoint) {
t.Fatalf("err = %v, want ErrInsecureTokenEndpoint", err)
}
}
// redirectTrap starts an "attacker" server that counts any hit, and a credential
// endpoint that redirects to it with the given status. It returns the endpoint
// URL and a func reporting how many times the attacker was contacted — a
// non-zero count means the credential-bearing POST body was replayed (#729).
func redirectTrap(t *testing.T, status int) (endpoint string, attackerHits func() int32) {
t.Helper()
var hits int32
attacker := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
atomic.AddInt32(&hits, 1)
_, _ = io.WriteString(w, `{"access_token":"leaked","refresh_token":"leaked"}`)
}))
t.Cleanup(attacker.Close)
endp := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, attacker.URL, status)
}))
t.Cleanup(endp.Close)
return endp.URL, func() int32 { return atomic.LoadInt32(&hits) }
}
func TestPostTokenRefusesRedirect(t *testing.T) {
endpoint, hits := redirectTrap(t, http.StatusTemporaryRedirect) // 307
form := url.Values{}
form.Set("grant_type", "authorization_code")
form.Set("code", "the-code")
form.Set("client_secret", "shh")
_, err := PostToken(context.Background(), http.DefaultClient, endpoint, form, Token{}, nil)
if !errors.Is(err, ErrUnsafeRedirect) {
t.Fatalf("PostToken err = %v, want ErrUnsafeRedirect", err)
}
if n := hits(); n != 0 {
t.Fatalf("attacker received %d request(s) — credential body was replayed", n)
}
}
func TestExchangeCodeRefusesRedirect(t *testing.T) {
endpoint, hits := redirectTrap(t, http.StatusPermanentRedirect) // 308
_, err := ExchangeCode(context.Background(), http.DefaultClient,
Config{ClientID: "c", TokenEndpoint: endpoint}, "the-code", "verifier", "http://127.0.0.1/cb", nil)
if !errors.Is(err, ErrUnsafeRedirect) {
t.Fatalf("ExchangeCode err = %v, want ErrUnsafeRedirect", err)
}
if n := hits(); n != 0 {
t.Fatalf("attacker received %d request(s) — code/verifier replayed", n)
}
}
func TestRefreshRefusesRedirect(t *testing.T) {
endpoint, hits := redirectTrap(t, http.StatusTemporaryRedirect) // 307
_, err := Refresh(context.Background(), http.DefaultClient,
Config{ClientID: "c", TokenEndpoint: endpoint}, Token{RefreshToken: "rt"}, nil)
if !errors.Is(err, ErrUnsafeRedirect) {
t.Fatalf("Refresh err = %v, want ErrUnsafeRedirect", err)
}
if n := hits(); n != 0 {
t.Fatalf("attacker received %d request(s) — refresh_token replayed", n)
}
}
func TestValidateTokenEndpoint(t *testing.T) {
cases := map[string]bool{
"https://auth.example.com/token": true,
+3 -16
View File
@@ -218,26 +218,13 @@ func (m *Manager) resolveEndpoints(ctx context.Context, cfg Config) (Config, err
// may already be sufficient for the chosen flow. Only merge on success (and
// never return from the error branch, which would trip the nilerr linter).
if meta, err := m.discover(ctx, cfg.IssuerURL); err == nil {
// A discovered endpoint must pass the same https/loopback rule as a
// configured one before it is merged, so discovery metadata can never
// silently downgrade the login to an insecure or attacker-controlled
// endpoint. Fail closed on a bad endpoint rather than merging it.
if cfg.AuthorizationEndpoint == "" && meta.AuthorizationEndpoint != "" {
if err := ValidateEndpointURL(meta.AuthorizationEndpoint); err != nil {
return cfg, fmt.Errorf("oauth: discovered authorization endpoint: %w", err)
}
if cfg.AuthorizationEndpoint == "" {
cfg.AuthorizationEndpoint = meta.AuthorizationEndpoint
}
if cfg.TokenEndpoint == "" && meta.TokenEndpoint != "" {
if err := ValidateEndpointURL(meta.TokenEndpoint); err != nil {
return cfg, fmt.Errorf("oauth: discovered token endpoint: %w", err)
}
if cfg.TokenEndpoint == "" {
cfg.TokenEndpoint = meta.TokenEndpoint
}
if cfg.DeviceAuthorizationEndpoint == "" && meta.DeviceAuthorizationEndpoint != "" {
if err := ValidateEndpointURL(meta.DeviceAuthorizationEndpoint); err != nil {
return cfg, fmt.Errorf("oauth: discovered device authorization endpoint: %w", err)
}
if cfg.DeviceAuthorizationEndpoint == "" {
cfg.DeviceAuthorizationEndpoint = meta.DeviceAuthorizationEndpoint
}
}
-19
View File
@@ -2,7 +2,6 @@ package oauth
import (
"context"
"errors"
"io"
"net/http"
"net/http/httptest"
@@ -201,24 +200,6 @@ func TestManagerGetFreshDiscoversIssuerEndpoints(t *testing.T) {
}
}
// A discovery document that serves an insecure (non-https, non-loopback)
// endpoint must be rejected before it is merged, so discovery metadata cannot
// silently downgrade the login to an attacker-controlled origin (issue #511).
func TestResolveEndpointsRejectsInsecureDiscoveredEndpoint(t *testing.T) {
mux := http.NewServeMux()
server := httptest.NewServer(mux)
defer server.Close()
mux.HandleFunc("/.well-known/oauth-authorization-server", func(w http.ResponseWriter, _ *http.Request) {
// Loopback issuer, but an attacker-controlled http authorization endpoint.
_, _ = io.WriteString(w, `{"issuer":"`+server.URL+`","token_endpoint":"`+server.URL+`/token","authorization_endpoint":"http://evil.example/authorize"}`)
})
m := managerFor(t, map[string]string{}, nil)
_, err := m.resolveEndpoints(context.Background(), Config{IssuerURL: server.URL})
if !errors.Is(err, ErrInsecureTokenEndpoint) {
t.Fatalf("resolveEndpoints err = %v, want ErrInsecureTokenEndpoint", err)
}
}
func TestManagerGetFreshRefreshesExpired(t *testing.T) {
fp := newFakeProvider(t, `{"access_token":"fresh-at","expires_in":3600}`)
env := map[string]string{
-5
View File
@@ -90,11 +90,6 @@ var (
// ErrInsecureTokenEndpoint is returned when a credential would be sent over a
// non-https, non-loopback endpoint.
ErrInsecureTokenEndpoint = errors.New("oauth: refusing to send credential to a non-https token endpoint")
// ErrUnsafeRedirect is returned when a credential-bearing OAuth POST (token
// exchange/refresh, device authorization, or device-token poll) is redirected.
// Following it would replay the form body — codes, PKCE verifiers, refresh
// tokens, client secrets — to a target that never passed endpoint validation.
ErrUnsafeRedirect = errors.New("oauth: refusing to follow a redirect from a credential endpoint")
// ErrNoRefreshToken is returned when a refresh is attempted without one.
ErrNoRefreshToken = errors.New("oauth: no refresh token available")
// ErrAuthorizationPending is the RFC 8628 "keep polling" signal.
+4 -4
View File
@@ -133,7 +133,7 @@
"verificationCommand": [
"bash",
"-c",
"go test ./... && ! grep --exclude=.zero-answer.txt -RIn 'MaxRetries =' ."
"go test ./... && ! grep -RIn 'MaxRetries =' ."
],
"oracleTest": "package edit\n\nvar _ = RetryLimit\n"
},
@@ -146,7 +146,7 @@
"verificationCommand": [
"bash",
"-c",
"go test ./... && ! grep --exclude=.zero-answer.txt -R receieves . && grep --exclude=.zero-answer.txt -R receives ."
"go test ./... && ! grep -R receieves . && grep -R receives ."
]
},
{
@@ -400,7 +400,7 @@
"verificationCommand": [
"bash",
"-c",
"go test ./... && ! grep --exclude=.zero-answer.txt -R 'package refactor' . && grep --exclude=.zero-answer.txt -R 'package zeroapp' ."
"go test ./... && ! grep -R 'package refactor' . && grep -R 'package zeroapp' ."
],
"oracleTest": "package zeroapp\n\nvar _ = Config{}\n"
},
@@ -426,7 +426,7 @@
"verificationCommand": [
"bash",
"-c",
"go test ./... && ! grep --exclude=.zero-answer.txt -R 'func Wrapper' ."
"go test ./... && ! grep -R 'func Wrapper' ."
],
"oracleTest": "package refactor\n\nimport \"testing\"\n\n// TestGreetWrapped proves greetWrapped still exists and behaves as the inlined\n// Wrapper did: deleting both Wrapper and greetWrapped (or stubbing greetWrapped\n// to return \"\") fails to compile or run, so the oracle can't be passed by\n// removing the target along with its caller. Paired with the ! grep 'func\n// Wrapper' verificationCommand that proves Wrapper itself is gone.\nfunc TestGreetWrapped(t *testing.T) {\n\tif got := greetWrapped(\"x\"); got != \"hello, x\" {\n\t\tt.Fatalf(\"greetWrapped(x) = %q, want %q\", got, \"hello, x\")\n\t}\n}\n"
},
-4
View File
@@ -85,10 +85,6 @@ type RunContext struct {
Model string
Mode string
SelfCorrect bool
// ExecProfile, when non-empty, is forwarded to the agent invocation as
// --exec-profile (turn benchmark runner; the task benchmark leaves it
// empty).
ExecProfile string
}
// TaskRunner runs a single benchmark task and returns its outcome.
+20 -152
View File
@@ -15,7 +15,6 @@ import (
"strings"
"time"
"github.com/Gitlawb/zero/internal/execprofile"
"github.com/Gitlawb/zero/internal/trace"
)
@@ -41,25 +40,7 @@ import (
// prevents a v2→v2 cross-version comparison from misreading the jump as a model
// improvement. New counts: correctness 34 (edit 10 + fix 8 + nav 10 + refactor
// 6), build 0, latency 14 (longproc 4 + longctx 4 + parallel 6).
//
// v4 adds tasksErrored: the number of tasks whose every iteration produced no
// accepted benchmark sample. That covers pre-run failures (missing binary,
// spawn error, process crash) and harness errors after a successful agent run
// (e.g. oracle stamping failed) alike — in every case the iteration yields no
// usable measurement. Errored tasks were previously visible only in warnings,
// so a run where nothing was actually measured still printed a clean-looking
// 0% pass summary and exited 0. tasksErrored makes that state first-class:
// the summary surfaces it, and the turn command exits nonzero when every
// attempted task errored. Accounting: an errored oracle task stays in its
// tier denominator as a failure; an errored latency-only task counts in
// latencyOnlyTasks and, as always, in no pass rate.
//
// v5 is ADDITIVE ONLY: execProfile records the execution profile the run was
// benchmarked under (omitted when none was selected), so profile A/B captures
// are self-describing and two reports can't be compared without noticing they
// ran different postures. No existing field changed shape or meaning; a v4
// consumer reading a v5 report misses only the new field.
const TurnSchemaVersion = 5
const TurnSchemaVersion = 3
// TurnRunner runs one benchmark task and reports its outcome plus the captured
// per-turn trace. A non-nil Err means the run failed to execute (process crash);
@@ -86,10 +67,6 @@ type TurnBenchConfig struct {
Model string
Mode string
SelfCorrect bool
// ExecProfile, when non-empty, runs every task under the named execution
// profile (zero exec --exec-profile) and stamps it into the result so the
// report is self-describing for profile A/B comparisons.
ExecProfile string
Version string
Commit string
// Iterations is how many times each task is run. The per-process `zero exec`
@@ -162,13 +139,11 @@ type TurnBenchResult struct {
Suite string `json:"suite"`
Model string `json:"model"`
Mode string `json:"mode,omitempty"`
ExecProfile string `json:"execProfile,omitempty"`
SelfCorrect bool `json:"selfCorrect"`
Version string `json:"version,omitempty"`
Commit string `json:"commit,omitempty"`
Date string `json:"date"`
TasksAttempted int `json:"tasksAttempted"`
TasksErrored int `json:"tasksErrored"`
TasksVerified int `json:"tasksVerified"`
TasksPassed int `json:"tasksPassed"`
LatencyOnlyTasks int `json:"latencyOnlyTasks"`
@@ -214,19 +189,6 @@ func RunTurnBench(ctx context.Context, set TaskSet, cfg TurnBenchConfig) (TurnBe
if cfg.Runner == nil {
return TurnBenchResult{}, errors.New("turn benchmark requires a runner")
}
// Canonicalize the profile once at the boundary so every consumer — the
// child exec args, the stamped result, the summary — carries the same
// catalog name regardless of the caller's casing/whitespace, and a direct
// library caller cannot slip an unknown name into a report the way the CLI
// (which validates at parse time) cannot.
benchProfile := strings.TrimSpace(cfg.ExecProfile)
if benchProfile != "" {
profile, ok := execprofile.Lookup(benchProfile)
if !ok {
return TurnBenchResult{}, fmt.Errorf("unknown execution profile %q (valid: %s)", cfg.ExecProfile, strings.Join(execprofile.Names(), ", "))
}
benchProfile = profile.Name
}
iterations := cfg.Iterations
if iterations < 1 {
iterations = 1
@@ -235,7 +197,7 @@ func RunTurnBench(ctx context.Context, set TaskSet, cfg TurnBenchConfig) (TurnBe
if now == nil {
now = time.Now
}
rc := RunContext{Model: cfg.Model, Mode: cfg.Mode, SelfCorrect: cfg.SelfCorrect, ExecProfile: benchProfile}
rc := RunContext{Model: cfg.Model, Mode: cfg.Mode, SelfCorrect: cfg.SelfCorrect}
perSpanSamples := map[string][]float64{}
classWalls := map[string][]float64{}
@@ -265,7 +227,6 @@ func RunTurnBench(ctx context.Context, set TaskSet, cfg TurnBenchConfig) (TurnBe
Suite: strings.TrimSpace(set.ID),
Model: strings.TrimSpace(cfg.Model),
Mode: strings.TrimSpace(cfg.Mode),
ExecProfile: benchProfile,
SelfCorrect: cfg.SelfCorrect,
Version: strings.TrimSpace(cfg.Version),
Commit: strings.TrimSpace(cfg.Commit),
@@ -288,7 +249,6 @@ func RunTurnBench(ctx context.Context, set TaskSet, cfg TurnBenchConfig) (TurnBe
// runner is cold-start, so a flaky pass on one iteration and a fail on
// another is a real regression signal, not noise to average away.
passedForTask := true
erroredIterations := 0
for iter := 0; iter < iterations; iter++ {
outcome := cfg.Runner(ctx, task, rc)
if outcome.TraceIssue != "" {
@@ -306,7 +266,6 @@ func RunTurnBench(ctx context.Context, set TaskSet, cfg TurnBenchConfig) (TurnBe
Message: fmt.Sprintf("task %s: %v", task.ID, outcome.Err),
})
passedForTask = false
erroredIterations++
continue
}
if !outcome.Passed {
@@ -347,14 +306,6 @@ func RunTurnBench(ctx context.Context, set TaskSet, cfg TurnBenchConfig) (TurnBe
// counters. A latency-only task (no verificationCommand) is never counted
// in any pass rate even when the runner reports Passed — an exit-0
// read-only run proves the turn ran, not that the answer was right.
if erroredIterations == iterations {
// Every iteration produced no accepted sample (spawn failure or a
// harness error after the run). The task still counts in its tier
// below — an oracle task as a failure, a latency-only task in no
// pass rate as always — but the errored state is first-class so a
// broken run can never print a clean-looking summary.
result.TasksErrored++
}
hasOracle := len(task.VerificationCommand) > 0
switch {
case !hasOracle:
@@ -505,26 +456,9 @@ func FormatTurnBenchSummary(result TurnBenchResult) string {
result.BuildPassedTasks, result.BuildCheckedTasks, result.BuildPassRate*100,
result.LatencyOnlyTasks, result.Iterations),
}
if result.TasksErrored > 0 {
lines = append(lines, fmt.Sprintf("ERRORED: %d task(s) produced no accepted benchmark sample (spawn/crash or harness error) — errored oracle tasks count as failures in their tier's pass rate; latency-only tasks stay out of pass rates as always", result.TasksErrored))
shown := 0
for _, warning := range result.Warnings {
if warning.Metric != "run" {
continue
}
lines = append(lines, " "+warning.Message)
shown++
if shown == 3 {
break
}
}
}
if result.Mode != "" {
lines = append(lines, "mode: "+result.Mode)
}
if result.ExecProfile != "" {
lines = append(lines, "exec-profile: "+result.ExecProfile)
}
if len(result.TopLatency) > 0 {
lines = append(lines, "top latency sources:")
for _, src := range result.TopLatency {
@@ -578,54 +512,28 @@ func WriteTurnBenchJSON(w io.Writer, result TurnBenchResult) error {
return encoder.Encode(result)
}
// execExitIncomplete mirrors internal/cli.exitIncomplete (the `zero exec` exit
// code, 4) for a headless run the completion gate marked INCOMPLETE: the run
// stopped without a completion signal (e.g. it couldn't self-verify because its
// sandboxed shell was unavailable under --auto member), which is distinct from a
// crash (1), usage error (2), provider failure (3), or interruption (130). It is
// the ONLY nonzero exit an oracle-bearing task may defer to the oracle for; every
// other nonzero exit stays authoritative. Kept as a local literal so perfbench
// doesn't import the cli package; must stay in sync with that constant.
const execExitIncomplete = 4
// NewTurnExecRunner builds the production turn-benchmark runner: it invokes
// headless `zero exec` with stream-json output AND `--trace <tmpfile>`, then
// parses the emitted NDJSON trace into a *trace.TurnTrace. binary is the path to
// the `zero` binary; extraArgs are appended to every invocation. Pass/fail is
// decided by the task's oracle when it has one. The oracle that gates the verdict
// is the VerificationCommand (which compiles and runs the stamped OracleTest via
// `go test`, plus any greps against the fixture); it is ground truth, so an
// oracle-bearing task that applied the right edit but exited INCOMPLETE (exit 4,
// its sandboxed self-verification couldn't run) still passes. Any OTHER nonzero
// exit (crash/usage/provider/interrupt) stays authoritative and fails even an
// oracle-bearing task. A latency-only task has no VerificationCommand, so any
// nonzero exit is its only signal and fails it.
// decided from the stream-json run_end exit code (and the task's
// VerificationCommand when present), exactly like NewExecRunner.
func NewTurnExecRunner(binary string, extraArgs ...string) TurnRunner {
return func(ctx context.Context, task BenchTask, rc RunContext) TurnTaskOutcome {
// buildTurnExecArgs grants the write + sandboxed-shell tool set to EVERY
// invocation, so a task MUST run inside an isolated fixture copy — never the
// caller's cwd. The grant is unconditional, so the isolation can't be
// optional: a fixtureless task would turn an agent with write/shell tools
// loose in the caller's working directory (their real repo). Enforce the
// invariant instead of trusting it — reject a fixtureless task up front,
// before anything is launched, so the grant and the guard can't drift apart.
// Every shipped task declares a fixture, so this only fires on a malformed or
// newly added task, and it fails loudly rather than silently.
if strings.TrimSpace(task.WorkspaceFixture) == "" {
return TurnTaskOutcome{Err: errors.New("benchmark task has no workspaceFixture: the run grants write/shell tools and must execute in an isolated fixture copy, not the caller's cwd")}
}
// Isolate the workspace: copy the fixture into a fresh temp dir so a
// mutating task (edit/fix/refactor) can't dirty the shared, checked-in
// fixture or bleed into a later iteration of the same task. The guard above
// guarantees WorkspaceFixture is set, so this always runs.
copyDir, parent, cerr := copyFixture(strings.TrimSpace(task.WorkspaceFixture))
if cerr != nil {
return TurnTaskOutcome{Err: fmt.Errorf("isolate fixture: %w", cerr)}
// fixture or bleed into a later iteration of the same task. When no
// fixture is configured the agent runs in the caller's cwd as before.
if fixture := strings.TrimSpace(task.WorkspaceFixture); fixture != "" {
copyDir, parent, cerr := copyFixture(fixture)
if cerr != nil {
return TurnTaskOutcome{Err: fmt.Errorf("isolate fixture: %w", cerr)}
}
// Clean the whole unique parent (which owns copyDir) so the
// per-invocation scratch dir never leaks.
defer os.RemoveAll(parent)
task.WorkspaceFixture = copyDir
}
// Clean the whole unique parent (which owns copyDir) so the per-invocation
// scratch dir never leaks.
defer os.RemoveAll(parent)
task.WorkspaceFixture = copyDir
traceFile, err := os.CreateTemp("", "zero-turn-trace-*.ndjson")
if err != nil {
@@ -682,23 +590,10 @@ func NewTurnExecRunner(binary string, extraArgs ...string) TurnRunner {
}
}
// Decide what a nonzero exit means. We defer to the oracle for exactly ONE
// case: an oracle-bearing task that exited INCOMPLETE (exit 4). That is the
// completion gate downgrading a run whose edits may be correct but that
// couldn't emit a completion signal (e.g. its sandboxed self-verify couldn't
// run under --auto member) — the oracle below (the compiled test / grep run
// against the actual fixture files) is ground truth, so if the edit really
// landed it passes, and if the run abandoned the task the oracle fails it.
// Every OTHER nonzero exit stays authoritative: a crash (1), usage error
// (2), provider failure (3), or interruption (130) is a genuine failure, so
// a partial edit that happens to satisfy the oracle can't launder it into a
// pass. A latency-only task has no oracle to defer to, so any nonzero exit
// fails it.
// A nonzero agent exit already decided failure; don't run verification or
// mark the task passed.
if outcome.VerifyErr != "" {
if len(task.VerificationCommand) == 0 || exitCode != execExitIncomplete {
return outcome
}
outcome.VerifyErr = ""
return outcome
}
// Stamp the compiler-backed oracle and capture the agent's final answer
// BEFORE running the verification command. Both write into the fixture
@@ -829,23 +724,7 @@ func stampOracleAndAnswer(task BenchTask, outBuf []byte) error {
}
func buildTurnExecArgs(task BenchTask, rc RunContext, tracePath string, extraArgs []string) []string {
// --auto member is REQUIRED, not optional: without a permission grant `zero
// exec` runs in its default read-only posture, which exposes no write or shell
// tools (no edit_file/apply_patch/write_file/exec_command/bash). The mutating
// task classes (edit/fix/refactor) then cannot apply any change, so every run
// grinds to the turn ceiling or reports a no-tool blocker, and the only tasks
// that "pass" are ones whose oracle grep matches the stamped answer text rather
// than a real edit. member-auto grants the write + sandboxed-shell tools the
// benchmark needs while keeping the workspace/network/destructive safeguards, so
// it is preferred over the broader --skip-permissions-unsafe (which also drops
// those guards and auto-retries blocked commands unsandboxed). Note the shell it
// grants is sandboxed: on a host without sandbox setup the agent's own
// self-verification (go test/build) can't run and the turn exits INCOMPLETE even
// after a correct edit — the runner treats the stamped oracle, not that exit
// code, as ground truth for oracle-bearing tasks (see NewTurnExecRunner), so a
// correct edit still passes. Each task also runs in an isolated, throwaway
// fixture copy, so the granted tools have nothing outside the task to harm.
args := []string{"exec", "--auto", "member", "--output-format", "stream-json", "--trace", tracePath}
args := []string{"exec", "--output-format", "stream-json", "--trace", tracePath}
if model := strings.TrimSpace(rc.Model); model != "" {
args = append(args, "--model", model)
}
@@ -855,9 +734,6 @@ func buildTurnExecArgs(task BenchTask, rc RunContext, tracePath string, extraArg
if rc.SelfCorrect {
args = append(args, "--self-correct")
}
if profile := strings.TrimSpace(rc.ExecProfile); profile != "" {
args = append(args, "--exec-profile", profile)
}
args = append(args, extraArgs...)
args = append(args, task.Prompt)
return args
@@ -871,15 +747,7 @@ func ResolveBinary(explicit string) (string, error) {
if _, err := os.Stat(v); err != nil {
return "", fmt.Errorf("trace binary not found: %w", err)
}
// Pin the explicit path to an absolute one: the turn runner sets each
// child's cmd.Dir to a per-task fixture copy, so a relative path
// (./zero, the Makefile default) that stats fine here would fail to
// spawn from inside every fixture dir, silently erroring all tasks.
absolute, err := filepath.Abs(v)
if err != nil {
return "", fmt.Errorf("resolve trace binary path: %w", err)
}
return absolute, nil
return v, nil
}
if path, err := exec.LookPath("zero"); err == nil {
return path, nil
+8 -334
View File
@@ -4,8 +4,6 @@ import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
@@ -532,17 +530,14 @@ func TestRunTurnBenchBuildOnlyMechanism(t *testing.T) {
}
}
// TestTurnSchemaVersion pins the schema bump so a shape change is a conscious
// decision. v3 recorded the tier reclassification (refactor structural-positive
// and nav answer-oracles moved into correctnessPassRate). v4 adds tasksErrored:
// tasks whose every iteration died before the agent produced a run are now
// first-class in the report instead of visible only in warnings, so a
// spawn-broken run cannot print a clean-looking summary or exit 0. v5 is
// additive only: execProfile stamps the execution profile the run was
// benchmarked under so profile A/B reports are self-describing.
func TestTurnSchemaVersion(t *testing.T) {
if TurnSchemaVersion != 5 {
t.Fatalf("TurnSchemaVersion = %d, want 5", TurnSchemaVersion)
// TestTurnSchemaVersion3 pins the schema bump. v3 records the tier
// reclassification (refactor structural-positive and nav answer-oracles moved
// into correctnessPassRate), so a v2->v3 cross-version comparison cannot
// silently misread the jump as a model improvement — exactly the misread the
// tier system exists to prevent.
func TestTurnSchemaVersion3(t *testing.T) {
if TurnSchemaVersion != 3 {
t.Fatalf("TurnSchemaVersion = %d, want 3", TurnSchemaVersion)
}
}
@@ -751,92 +746,6 @@ echo '{"type":"run_end","exitCode":0}'
}
}
// TestOracleAuthoritativeOnIncompleteExit is the companion to the --auto member
// switch: under member-auto the agent's shell is sandboxed, so on a host without
// sandbox setup its self-verification (go test/build) can't run and the turn
// exits INCOMPLETE (exit 4) even after a correct edit. For an ORACLE-bearing task
// the runner must treat the stamped oracle — not that INCOMPLETE exit — as ground
// truth: here the stub applies the real edit-01 rename but reports exitCode 4, and
// the task must still pass because the fixture is correct. This defers ONLY for
// exit 4; TestNonIncompleteExitStaysAuthoritative pins the other side.
func TestOracleAuthoritativeOnIncompleteExit(t *testing.T) {
task := loadBaselineTask(t, "edit-01")
outcome := runTurnStub(t, task, `sed 's/const MaxRetries = 3/const RetryLimit = 3/' main.go > .zero-tmp && mv .zero-tmp main.go
echo '{"type":"run_end","exitCode":4}'
`)
if outcome.Err != nil {
t.Fatalf("incomplete-exit with a correct edit should pass, got harness error: %v", outcome.Err)
}
if !outcome.Passed {
t.Fatalf("a correct edit that exited INCOMPLETE (exit 4) must still pass its oracle: %+v", outcome)
}
if strings.TrimSpace(outcome.VerifyErr) != "" {
t.Fatalf("a passing oracle must clear the exit-code VerifyErr, got %q", outcome.VerifyErr)
}
}
// TestNonIncompleteExitStaysAuthoritative is the guard the oracle-authoritative
// change MUST NOT weaken: only INCOMPLETE (exit 4) defers to the oracle. A crash
// (1), provider failure (3), or interruption (130) is a genuine failure, so it
// stays authoritative even for an oracle-bearing task — otherwise a partial edit
// that happens to satisfy the oracle would launder a crashed or interrupted run
// into a pass. Each case applies the CORRECT edit-01 rename (so the oracle WOULD
// pass) but reports the failing exit code; the task must still fail, and the
// failure must be attributed to the exit code, not the oracle.
func TestNonIncompleteExitStaysAuthoritative(t *testing.T) {
for _, code := range []int{1, 3, 130} {
t.Run(fmt.Sprintf("exit%d", code), func(t *testing.T) {
task := loadBaselineTask(t, "edit-01")
outcome := runTurnStub(t, task, fmt.Sprintf(`sed 's/const MaxRetries = 3/const RetryLimit = 3/' main.go > .zero-tmp && mv .zero-tmp main.go
echo '{"type":"run_end","exitCode":%d}'
`, code))
if outcome.Err != nil {
t.Fatalf("a nonzero exit should be a task fail, not a harness error: %v", outcome.Err)
}
if outcome.Passed {
t.Fatalf("a run that exited %d must not pass even when the edit satisfies the oracle: %+v", code, outcome)
}
if want := fmt.Sprintf("exit code %d", code); !strings.Contains(outcome.VerifyErr, want) {
t.Fatalf("a non-INCOMPLETE nonzero exit must stay authoritative and surface its code, got VerifyErr=%q", outcome.VerifyErr)
}
})
}
}
// TestIncompleteExitStillFailsWhenOracleFails pins the OTHER half of the exit-4
// deferral: deferring to the oracle on INCOMPLETE must still MEAN the oracle is
// consulted, not a blanket pass. Here the stub does no real work and exits
// INCOMPLETE (4); the edit-01 rename never happened, so the oracle fails and the
// task fails. Without this, a regression that turned exit 4 into an unconditional
// pass would slip through, because its sibling TestOracleAuthoritativeOnIncomplete-
// Exit applies a CORRECT edit and would pass either way.
func TestIncompleteExitStillFailsWhenOracleFails(t *testing.T) {
task := loadBaselineTask(t, "edit-01")
outcome := runTurnStub(t, task, `echo '{"type":"run_end","exitCode":4}'
`)
assertVerifyFailed(t, "incomplete exit with no edit applied", outcome)
}
// TestNonzeroExitStillFailsLatencyOnly is the guard on the other side of that
// change: a latency-only task has no oracle to appeal to, so the exit code is
// the ONLY correctness signal and a nonzero exit must still fail it. Without
// this, dropping the exit-code gate for oracle tasks could be misread as
// dropping it everywhere.
func TestNonzeroExitStillFailsLatencyOnly(t *testing.T) {
task := loadBaselineTask(t, "longproc-01")
outcome := runTurnStub(t, task, `echo '{"type":"run_end","exitCode":4}'
`)
if outcome.Err != nil {
t.Fatalf("latency-only nonzero exit should be a verify fail, not a harness error: %v", outcome.Err)
}
if outcome.Passed {
t.Fatalf("a latency-only task that exited nonzero must not pass: %+v", outcome)
}
if strings.TrimSpace(outcome.VerifyErr) == "" {
t.Fatalf("a latency-only nonzero exit must surface a VerifyErr, got none: %+v", outcome)
}
}
// --- Satisfiable tests: the oracle PASSES the right thing (real fix applied) ---
// TestStampedOraclePassesWhenRefactorHappened proves the refactor-01 oracle is
@@ -1158,238 +1067,3 @@ printf '%s\n' '{"type":"run_end","exitCode":0}'
`)
assertVerifyFailed(t, "nav-08 fmt-only-no-testing", outcome)
}
func TestResolveBinaryAbsolutizesExplicitPath(t *testing.T) {
dir := t.TempDir()
name := "zero-probe.exe"
if err := os.WriteFile(filepath.Join(dir, name), []byte("x"), 0o755); err != nil {
t.Fatal(err)
}
cwd, err := os.Getwd()
if err != nil {
t.Fatal(err)
}
if err := os.Chdir(dir); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = os.Chdir(cwd) })
resolved, err := ResolveBinary("./" + name)
if err != nil {
t.Fatalf("ResolveBinary: %v", err)
}
if !filepath.IsAbs(resolved) {
t.Fatalf("ResolveBinary returned relative path %q; the turn runner sets cmd.Dir per task, so a relative binary fails to spawn from every fixture copy", resolved)
}
}
func TestRunTurnBenchCountsErroredTasks(t *testing.T) {
set := TaskSet{
ID: "errored-suite",
Tasks: []BenchTask{
{ID: "t1", Class: "nav", Prompt: "p1", VerificationCommand: []string{"true"}},
{ID: "t2", Class: "longproc", Prompt: "p2"},
},
}
cfg := TurnBenchConfig{
Model: "fake-model",
Iterations: 1,
Runner: func(context.Context, BenchTask, RunContext) TurnTaskOutcome {
return TurnTaskOutcome{Err: errors.New("fork/exec ./zero: file does not exist")}
},
Now: func() time.Time { return time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC) },
}
result, err := RunTurnBench(context.Background(), set, cfg)
if err != nil {
t.Fatalf("RunTurnBench: %v", err)
}
if result.TasksErrored != 2 {
t.Fatalf("TasksErrored=%d, want 2 (every iteration of both tasks errored)", result.TasksErrored)
}
if len(result.Warnings) != 2 {
t.Fatalf("warnings=%d, want 2 run warnings", len(result.Warnings))
}
summary := FormatTurnBenchSummary(result)
if !strings.Contains(summary, "ERRORED: 2 task(s)") {
t.Fatalf("summary does not surface the errored tasks:\n%s", summary)
}
if !strings.Contains(summary, "fork/exec ./zero") {
t.Fatalf("summary does not echo the underlying run error:\n%s", summary)
}
}
func TestRunTurnBenchPartialErrorStillCounts(t *testing.T) {
set := TaskSet{
ID: "partial-suite",
Tasks: []BenchTask{
{ID: "ok", Class: "nav", Prompt: "p", VerificationCommand: []string{"true"}},
{ID: "dead", Class: "nav", Prompt: "p", VerificationCommand: []string{"true"}},
},
}
canned := map[string]*trace.TurnTrace{"ok": cannedTrace(100, 10, 1000)}
inner := fakeTurnRunner(canned)
cfg := TurnBenchConfig{
Model: "fake-model",
Iterations: 1,
Runner: func(ctx context.Context, task BenchTask, rc RunContext) TurnTaskOutcome {
if task.ID == "dead" {
return TurnTaskOutcome{Err: errors.New("spawn failed")}
}
return inner(ctx, task, rc)
},
Now: func() time.Time { return time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC) },
}
result, err := RunTurnBench(context.Background(), set, cfg)
if err != nil {
t.Fatalf("RunTurnBench: %v", err)
}
if result.TasksErrored != 1 {
t.Fatalf("TasksErrored=%d, want 1", result.TasksErrored)
}
if result.TasksAttempted != 2 {
t.Fatalf("TasksAttempted=%d, want 2", result.TasksAttempted)
}
}
// The exec-profile passthrough: the profile must reach every task's zero exec
// invocation as --exec-profile (with the prompt staying last) and stay out of
// the args entirely when unset.
func TestBuildTurnExecArgsIncludesExecProfile(t *testing.T) {
task := BenchTask{ID: "t", Prompt: "do the thing"}
args := buildTurnExecArgs(task, RunContext{Model: "m", ExecProfile: "fast"}, "trace.ndjson", nil)
found := false
for i := 0; i < len(args)-1; i++ {
if args[i] == "--exec-profile" && args[i+1] == "fast" {
found = true
break
}
}
if !found {
t.Fatalf("args must carry --exec-profile fast, got %v", args)
}
if args[len(args)-1] != "do the thing" {
t.Fatalf("prompt must stay the last argument, got %v", args)
}
args = buildTurnExecArgs(task, RunContext{Model: "m"}, "trace.ndjson", nil)
for _, arg := range args {
if arg == "--exec-profile" {
t.Fatalf("no profile configured, but args carry --exec-profile: %v", args)
}
}
}
// Every benchmark invocation MUST grant the write/shell tool set via --auto
// member. Without a permission grant the agent runs read-only and cannot apply
// any edit, so the mutating classes measure nothing but oracle/answer
// contamination. member-auto is the RIGHT grant: it exposes the write +
// sandboxed-shell tools the benchmark needs while keeping the workspace/network/
// destructive safeguards, so the args must NOT reach for the broader
// --skip-permissions-unsafe (which drops those guards). This is a correctness
// contract, not a preference.
func TestBuildTurnExecArgsGrantsWriteTools(t *testing.T) {
args := buildTurnExecArgs(BenchTask{ID: "t", Prompt: "edit the file"}, RunContext{Model: "m"}, "trace.ndjson", nil)
autoMember := false
for i := 0; i < len(args)-1; i++ {
if args[i] == "--auto" && args[i+1] == "member" {
autoMember = true
break
}
}
if !autoMember {
t.Fatalf("benchmark exec args must include --auto member so the agent can apply edits, got %v", args)
}
for _, arg := range args {
if arg == "--skip-permissions-unsafe" {
t.Fatalf("benchmark exec args must NOT use --skip-permissions-unsafe (member-auto keeps the sandbox/network guards), got %v", args)
}
}
if args[len(args)-1] != "edit the file" {
t.Fatalf("prompt must stay the last argument, got %v", args)
}
}
// TestTurnRunnerRejectsFixturelessTask is the other half of the write-tools
// contract: because buildTurnExecArgs grants the write + sandboxed-shell tool
// set to every invocation, a task with no fixture would run an agent with those
// tools in the caller's cwd. The runner must reject such a task BEFORE launching
// zero exec. Passing a binary path that does not exist proves the rejection is
// pre-launch — the outcome carries the fixture error, not a spawn error — so this
// runs on every OS (it never reaches the POSIX-only exec stub).
func TestTurnRunnerRejectsFixturelessTask(t *testing.T) {
task := BenchTask{ID: "no-fixture", Prompt: "do a thing"} // no WorkspaceFixture set
outcome := NewTurnExecRunner(filepath.Join(t.TempDir(), "does-not-exist-zero"))(context.Background(), task, RunContext{Model: "m"})
if outcome.Err == nil {
t.Fatalf("a fixtureless task must be rejected before launch, got %+v", outcome)
}
if !strings.Contains(outcome.Err.Error(), "workspaceFixture") {
t.Fatalf("rejection must name the missing workspaceFixture, got: %v", outcome.Err)
}
if outcome.Passed {
t.Fatalf("a rejected task must not pass, got %+v", outcome)
}
}
// The configured profile must reach the runner's RunContext and be stamped
// into the result, so a profile A/B report is self-describing. The boundary
// canonicalizes (case/whitespace) so equivalent postures always carry the same
// label, and rejects unknown names so a direct library caller cannot slip an
// unvalidated profile into a report the way the CLI (parse-time check) cannot.
func TestRunTurnBenchStampsExecProfile(t *testing.T) {
set := TaskSet{
ID: "profile-suite",
Tasks: []BenchTask{{ID: "t1", Class: "longproc", Prompt: "p"}},
}
var gotProfile string
cfg := TurnBenchConfig{
Model: "fake-model",
ExecProfile: " FAST ",
Iterations: 1,
Runner: func(_ context.Context, _ BenchTask, rc RunContext) TurnTaskOutcome {
gotProfile = rc.ExecProfile
return TurnTaskOutcome{Passed: true, WallMs: 10, Trace: cannedTrace(10, 1, 100)}
},
Now: func() time.Time { return time.Date(2026, 1, 2, 3, 4, 5, 0, time.UTC) },
}
result, err := RunTurnBench(context.Background(), set, cfg)
if err != nil {
t.Fatalf("RunTurnBench: %v", err)
}
if gotProfile != "fast" {
t.Fatalf("runner RunContext.ExecProfile = %q, want the canonical fast", gotProfile)
}
if result.ExecProfile != "fast" {
t.Fatalf("result.ExecProfile = %q, want the canonical fast", result.ExecProfile)
}
raw, err := json.Marshal(result)
if err != nil {
t.Fatalf("marshal result: %v", err)
}
if !strings.Contains(string(raw), `"execProfile":"fast"`) {
t.Fatal("published JSON must carry execProfile")
}
}
func TestRunTurnBenchRejectsUnknownExecProfile(t *testing.T) {
set := TaskSet{
ID: "profile-suite",
Tasks: []BenchTask{{ID: "t1", Class: "longproc", Prompt: "p"}},
}
ran := false
cfg := TurnBenchConfig{
Model: "fake-model",
ExecProfile: "blanced",
Iterations: 1,
Runner: func(context.Context, BenchTask, RunContext) TurnTaskOutcome {
ran = true
return TurnTaskOutcome{Passed: true, WallMs: 10, Trace: cannedTrace(10, 1, 100)}
},
}
_, err := RunTurnBench(context.Background(), set, cfg)
if err == nil || !strings.Contains(err.Error(), "unknown execution profile") {
t.Fatalf("err = %v, want an unknown-profile rejection", err)
}
if ran {
t.Fatal("nothing may run under an unknown profile")
}
}
-51
View File
@@ -108,57 +108,6 @@ func New(profile config.ProviderProfile, options Options) (zeroruntime.Provider,
}
}
// NewTurnSessionProvider builds the default TurnSessionProvider for a resolved
// profile: it constructs the provider exactly as New does, then wraps it with a
// ProviderCapabilities projection computed from the same model-registry entry
// New already resolves. The default session's Stream is the provider's
// StreamCompletion, so runtime behavior is identical to using New directly.
func NewTurnSessionProvider(profile config.ProviderProfile, options Options) (zeroruntime.TurnSessionProvider, error) {
provider, err := New(profile, options)
if err != nil {
return nil, err
}
caps, err := resolveCapabilities(profile, options)
if err != nil {
return nil, err
}
return zeroruntime.NewProviderTurnSessionProvider(provider, caps), nil
}
// resolveCapabilities projects the resolved profile's model-registry entry into
// the flat zeroruntime.ProviderCapabilities (zeroruntime cannot reference
// modelregistry types — modelregistry imports zeroruntime, so a typed field
// would form an import cycle). A model absent from the registry yields only the
// resolved API model id and max-output tokens; the rest stays zero (unknown).
func resolveCapabilities(profile config.ProviderProfile, options Options) (zeroruntime.ProviderCapabilities, error) {
resolved, err := resolveProfile(profile, options)
if err != nil {
return zeroruntime.ProviderCapabilities{}, err
}
caps := zeroruntime.ProviderCapabilities{
Model: resolved.apiModel,
MaxOutputTokens: resolved.maxOutputTokens,
}
registry, err := defaultRegistry(options.ModelRegistry)
if err != nil {
return zeroruntime.ProviderCapabilities{}, err
}
if entry, ok := registry.Get(strings.TrimSpace(profile.Model)); ok {
caps.ContextWindow = entry.ContextLimits.ContextWindow
caps.SupportsVision = entry.Supports(modelregistry.ModelCapabilityVision)
caps.SupportsReasoning = entry.Supports(modelregistry.ModelCapabilityReasoning)
caps.SupportsPromptCache = entry.Supports(modelregistry.ModelCapabilityPromptCache)
// Read efforts through Registry.ReasoningEfforts (not the raw entry) so
// the projection matches what the /effort picker and the run-time
// resolver advertise — including the name-based fallback for catalog
// entries that enumerate no efforts of their own.
for _, effort := range registry.ReasoningEfforts(entry.ID) {
caps.ReasoningEfforts = append(caps.ReasoningEfforts, string(effort))
}
}
return caps, nil
}
func parseThinkTagsForProfile(profile config.ProviderProfile, resolved resolvedProfile) bool {
if profile.ParseThinkTags != nil {
return *profile.ParseThinkTags
@@ -1,194 +0,0 @@
package providers
import (
"context"
"errors"
"testing"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/modelregistry"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// TestNewTurnSessionProviderForEveryKind asserts every current provider kind
// wraps into the default TurnSessionProvider: construction succeeds, a session
// opens, and Compact reports unsupported (the default adapter contract).
func TestNewTurnSessionProviderForEveryKind(t *testing.T) {
kinds := []config.ProviderKind{
config.ProviderKindOpenAI,
config.ProviderKindOpenAICompatible,
config.ProviderKindAnthropic,
config.ProviderKindAnthropicCompat,
config.ProviderKindGoogle,
}
for _, kind := range kinds {
t.Run(string(kind), func(t *testing.T) {
tsp, err := NewTurnSessionProvider(config.ProviderProfile{
Name: "pr7-" + string(kind),
ProviderKind: kind,
BaseURL: "https://provider.example/v1",
APIKey: "sk-turn-session-test",
// A model absent from the registry: resolution falls back to the
// raw id, so this test stays hermetic across catalog changes.
Model: "pr7-unregistered-model",
}, Options{UserAgent: "zero-turn-session-test"})
if err != nil {
t.Fatalf("NewTurnSessionProvider(%s): %v", kind, err)
}
session, err := tsp.OpenTurnSession(context.Background())
if err != nil {
t.Fatalf("OpenTurnSession(%s): %v", kind, err)
}
defer func() {
if closeErr := session.Close(); closeErr != nil {
t.Fatalf("Close(%s): %v", kind, closeErr)
}
}()
if err := session.Prewarm(context.Background()); err != nil {
t.Fatalf("Prewarm(%s): %v", kind, err)
}
if _, compactErr := session.Compact(context.Background(), zeroruntime.CompletionRequest{}); !errors.Is(compactErr, zeroruntime.ErrCompactionUnsupported) {
t.Fatalf("Compact(%s) = %v, want ErrCompactionUnsupported", kind, compactErr)
}
caps := tsp.Capabilities()
if caps.Model != "pr7-unregistered-model" {
t.Fatalf("Capabilities(%s).Model = %q, want the resolved api model", kind, caps.Model)
}
if caps.NativeCompaction {
t.Fatalf("Capabilities(%s).NativeCompaction = true, want false for the default adapter", kind)
}
})
}
}
// TestNewTurnSessionProviderProjectsRegistryCapabilities asserts the factory
// projects the model-registry entry (context limits, capability flags,
// reasoning efforts) into the flat ProviderCapabilities.
func TestNewTurnSessionProviderProjectsRegistryCapabilities(t *testing.T) {
registry, err := modelregistry.NewRegistry([]modelregistry.ModelEntry{{
ID: "pr7-caps-model",
DisplayName: "PR7 Capability Probe",
APIModel: "pr7-caps-api-model",
Provider: modelregistry.ProviderOpenAI,
ContextLimits: modelregistry.ContextLimits{
ContextWindow: 200_000,
MaxOutputTokens: 64_000,
},
ReasoningEfforts: []modelregistry.ReasoningEffort{
modelregistry.ReasoningEffortLow,
modelregistry.ReasoningEffortMedium,
modelregistry.ReasoningEffortHigh,
},
Capabilities: modelregistry.ModelCapabilities{
modelregistry.ModelCapabilityChat,
modelregistry.ModelCapabilityVision,
modelregistry.ModelCapabilityReasoning,
modelregistry.ModelCapabilityPromptCache,
},
Cost: modelregistry.ModelCost{
Currency: "USD",
Unit: "per_1m_tokens",
InputPerMillion: 1,
OutputPerMillion: 2,
Source: "https://provider.example/pricing (test fixture)",
SourceLastVerified: "2026-07-18",
},
Status: modelregistry.ModelStatusActive,
Aliases: []string{"pr7-caps"},
}})
if err != nil {
t.Fatalf("NewRegistry: %v", err)
}
tsp, err := NewTurnSessionProvider(config.ProviderProfile{
Name: "pr7-caps",
ProviderKind: config.ProviderKindOpenAI,
BaseURL: "https://provider.example/v1",
APIKey: "sk-turn-session-test",
Model: "pr7-caps-model",
}, Options{
UserAgent: "zero-turn-session-test",
ModelRegistry: &registry,
})
if err != nil {
t.Fatalf("NewTurnSessionProvider: %v", err)
}
caps := tsp.Capabilities()
if caps.Model != "pr7-caps-api-model" {
t.Fatalf("Model = %q, want the registry APIModel", caps.Model)
}
if caps.ContextWindow != 200_000 || caps.MaxOutputTokens != 64_000 {
t.Fatalf("limits = (%d, %d), want (200000, 64000)", caps.ContextWindow, caps.MaxOutputTokens)
}
if !caps.SupportsVision || !caps.SupportsReasoning || !caps.SupportsPromptCache {
t.Fatalf("capability flags = %+v, want vision/reasoning/prompt-cache all true", caps)
}
if len(caps.ReasoningEfforts) != 3 || caps.ReasoningEfforts[0] != "low" || caps.ReasoningEfforts[2] != "high" {
t.Fatalf("ReasoningEfforts = %v, want [low medium high]", caps.ReasoningEfforts)
}
if caps.NativeCompaction {
t.Fatal("NativeCompaction = true, want false for the default adapter")
}
}
// TestNewTurnSessionProviderUsesEffectiveReasoningEfforts asserts the
// projection reads efforts through Registry.ReasoningEfforts, so a catalog
// entry that enumerates no efforts of its own still reports the name-inferred
// effective tiers the /effort picker and run-time resolver advertise.
func TestNewTurnSessionProviderUsesEffectiveReasoningEfforts(t *testing.T) {
registry, err := modelregistry.NewRegistry([]modelregistry.ModelEntry{{
// A gpt-5-family id with NO ReasoningEfforts listed: the effective
// efforts come from name inference, differing from the raw entry.
ID: "gpt-5-pr7-probe",
DisplayName: "PR7 Effective Efforts Probe",
APIModel: "gpt-5-pr7-probe-api",
Provider: modelregistry.ProviderOpenAI,
ContextLimits: modelregistry.ContextLimits{
ContextWindow: 400_000,
MaxOutputTokens: 128_000,
},
Capabilities: modelregistry.ModelCapabilities{
modelregistry.ModelCapabilityChat,
modelregistry.ModelCapabilityReasoning,
},
Cost: modelregistry.ModelCost{
Currency: "USD",
Unit: "per_1m_tokens",
InputPerMillion: 1,
OutputPerMillion: 2,
Source: "https://provider.example/pricing (test fixture)",
SourceLastVerified: "2026-07-18",
},
Status: modelregistry.ModelStatusActive,
Aliases: []string{"pr7-effective-efforts"},
}})
if err != nil {
t.Fatalf("NewRegistry: %v", err)
}
tsp, err := NewTurnSessionProvider(config.ProviderProfile{
Name: "pr7-effective",
ProviderKind: config.ProviderKindOpenAI,
BaseURL: "https://provider.example/v1",
APIKey: "sk-turn-session-test",
Model: "gpt-5-pr7-probe",
}, Options{
UserAgent: "zero-turn-session-test",
ModelRegistry: &registry,
})
if err != nil {
t.Fatalf("NewTurnSessionProvider: %v", err)
}
got := tsp.Capabilities().ReasoningEfforts
want := []string{"minimal", "low", "medium", "high"}
if len(got) != len(want) {
t.Fatalf("ReasoningEfforts = %v, want %v (name-inferred fallback)", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("ReasoningEfforts[%d] = %q, want %q (full: %v)", i, got[i], want[i], got)
}
}
}
@@ -5,7 +5,6 @@ import (
"encoding/json"
"net/http"
"net/http/httptest"
"reflect"
"strings"
"testing"
"time"
@@ -1376,61 +1375,3 @@ func TestOpenAIRequestPromptCacheKey(t *testing.T) {
}
}
}
func TestOpenAIRequestPreservesCacheablePrefixAcrossTurns(t *testing.T) {
provider, err := New(Options{Model: "gpt-test"})
if err != nil {
t.Fatalf("New returned error: %v", err)
}
tools := []zeroruntime.ToolDefinition{{
Name: "read_file",
Description: "Read a file.",
Parameters: map[string]any{
"type": "object",
"properties": map[string]any{
"path": map[string]any{"type": "string"},
},
},
}}
firstMessages := []zeroruntime.Message{
{Role: zeroruntime.MessageRoleSystem, Content: "stable system prompt"},
{Role: zeroruntime.MessageRoleUser, Content: "first turn"},
}
secondMessages := append([]zeroruntime.Message(nil), firstMessages...)
secondMessages = append(secondMessages,
zeroruntime.Message{Role: zeroruntime.MessageRoleAssistant, Content: "first response"},
zeroruntime.Message{Role: zeroruntime.MessageRoleUser, Content: "second turn"},
)
marshalBody := func(messages []zeroruntime.Message) map[string]any {
t.Helper()
mapped := provider.openAIRequest(zeroruntime.CompletionRequest{
Messages: messages,
Tools: tools,
PromptCacheKey: "session-stable-prefix",
})
data, marshalErr := json.Marshal(mapped)
if marshalErr != nil {
t.Fatalf("marshal request: %v", marshalErr)
}
var body map[string]any
if unmarshalErr := json.Unmarshal(data, &body); unmarshalErr != nil {
t.Fatalf("unmarshal request: %v", unmarshalErr)
}
return body
}
first := marshalBody(firstMessages)
second := marshalBody(secondMessages)
firstWireMessages := first["messages"].([]any)
secondWireMessages := second["messages"].([]any)
if !reflect.DeepEqual(secondWireMessages[:len(firstWireMessages)], firstWireMessages) {
t.Fatalf("first wire messages must be an exact prefix of the second:\nfirst=%#v\nsecond=%#v", firstWireMessages, secondWireMessages)
}
if !reflect.DeepEqual(first["tools"], second["tools"]) {
t.Fatalf("wire tool definitions drifted:\nfirst=%#v\nsecond=%#v", first["tools"], second["tools"])
}
if first["prompt_cache_key"] != second["prompt_cache_key"] || first["prompt_cache_key"] != "session-stable-prefix" {
t.Fatalf("wire prompt cache key must remain stable: first=%#v second=%#v", first["prompt_cache_key"], second["prompt_cache_key"])
}
}
-205
View File
@@ -1,205 +0,0 @@
package openai
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"strings"
"sync"
"time"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// prewarmTimeout bounds the best-effort prewarm probe. The probe runs in the
// background, so this cap only limits how long the goroutine may linger — it
// can never delay the first turn.
const prewarmTimeout = 3 * time.Second
// NewTurnSessionProvider wraps an already-constructed *Provider in the
// optimized OpenAI turn session: a best-effort connection prewarm at run start
// plus request-prefix stability telemetry. Stream is Provider.StreamCompletion
// verbatim, so runtime request behavior is identical to the default adapter —
// the session adds observation and pool priming, never a different request.
func NewTurnSessionProvider(provider *Provider, caps zeroruntime.ProviderCapabilities) zeroruntime.TurnSessionProvider {
return &turnSessionProvider{provider: provider, caps: caps}
}
type turnSessionProvider struct {
provider *Provider
caps zeroruntime.ProviderCapabilities
}
func (p *turnSessionProvider) OpenTurnSession(context.Context) (zeroruntime.TurnSession, error) {
return &turnSession{provider: p.provider}, nil
}
func (p *turnSessionProvider) Capabilities() zeroruntime.ProviderCapabilities {
return p.caps
}
// turnSession is one run's optimized OpenAI session. The agent loop serializes
// all provider I/O through its session shim, so fields need no mutex. The
// fingerprint counters exist so a future stateful-reuse session (Responses API)
// inherits a proven prefix-stability detector; on chat completions there is no
// server-side response state to invalidate, so drift is telemetry only.
type turnSession struct {
provider *Provider
lastFingerprint string
prewarmOnce sync.Once
// prewarmDone closes when the background probe settles; tests wait on it.
prewarmDone chan struct{}
}
// Prewarm launches one bounded, unauthenticated HEAD probe to the provider's
// base URL in the background so the TCP+TLS handshake lands in the shared
// connection pool while the loop assembles the first prompt. One attempt, no
// retries, no bearer token — the goal is the handshake, not a 2xx (a 401/404/
// 405 response still primes the pool). Always returns nil: prewarm is advisory
// by contract and never required for correctness.
//
// The probe is issued directly through the client (NOT providerio.SendWithRetry)
// so it stamps ONLY the provider_prewarm span — a nested provider_connect stamp
// would contaminate the exact A/B metric this feature is measured by.
//
// When the client's transport cannot retain idle connections (macOS disables
// keep-alives on the shared transport because degraded pooled connections are
// indistinguishable from backend slowness there), the probe is skipped entirely
// rather than spending a request that can never be reused. The pool's 30s idle
// timeout bounds the warm window; a slower start simply falls back to a cold
// dial, never worse than today.
func (s *turnSession) Prewarm(ctx context.Context) error {
s.prewarmOnce.Do(func() {
done := make(chan struct{})
s.prewarmDone = done
provider := s.provider
if !transportRetainsIdleConns(provider.httpClient) {
close(done)
return
}
recorder := trace.FromContext(ctx)
go func() {
defer close(done)
probeCtx, cancel := context.WithTimeout(ctx, prewarmTimeout)
defer cancel()
span := recorder.Span(trace.SpanProviderPrewarm)
defer span.End()
recorder.Counter(trace.CounterPrewarmAttempts, 1)
request, err := http.NewRequestWithContext(probeCtx, http.MethodHead, provider.baseURL, nil)
if err != nil {
return
}
if provider.userAgent != "" {
request.Header.Set("User-Agent", provider.userAgent)
}
response, err := provider.httpClient.Do(request)
if err == nil {
_ = response.Body.Close()
}
}()
})
return nil
}
// transportRetainsIdleConns reports whether a warmed connection can outlive the
// probe. A *http.Transport with DisableKeepAlives (the shared transport on
// macOS) closes every connection after its request, so a prewarm there is pure
// overhead. A nil transport is net/http's default (keep-alives on); an unknown
// custom transport is assumed reusable — the probe stays a harmless best-effort.
func transportRetainsIdleConns(client *http.Client) bool {
if client == nil {
return false
}
if transport, ok := client.Transport.(*http.Transport); ok {
return !transport.DisableKeepAlives
}
return true
}
func (s *turnSession) Stream(ctx context.Context, request zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
s.observePrefix(ctx, request)
return s.provider.StreamCompletion(ctx, request)
}
func (s *turnSession) Compact(context.Context, zeroruntime.CompletionRequest) ([]zeroruntime.Message, error) {
return nil, zeroruntime.ErrCompactionUnsupported
}
// Close is an idempotent no-op: the shared transport pool owns the connections.
func (s *turnSession) Close() error { return nil }
// observePrefix tracks whether the request-prefix parameters stayed stable
// between this session's streams. The first stream seeds the fingerprint and
// counts as neither stable nor drift.
func (s *turnSession) observePrefix(ctx context.Context, request zeroruntime.CompletionRequest) {
fingerprint := s.computeFingerprint(request)
if s.lastFingerprint == "" {
s.lastFingerprint = fingerprint
return
}
recorder := trace.FromContext(ctx)
if fingerprint == s.lastFingerprint {
recorder.Counter(trace.CounterPrefixStable, 1)
return
}
recorder.Counter(trace.CounterPrefixDrift, 1)
s.lastFingerprint = fingerprint
}
// computeFingerprint digests the wire-affecting request parameters: the
// session's model and max-tokens (fixed at construction — CompletionRequest
// carries no model field), the normalized reasoning effort as it would appear
// on the wire, the prompt-cache key (constant within a session, and part of
// the request the provider serializes), and an order-preserving digest of the
// advertised tools (request serialization preserves tool order, so a reorder
// changes the wire bytes and counts as drift). Messages are excluded — they
// grow every turn; this fingerprints request parameters, not conversation
// content.
//
// SCOPE: this is per-provider request-parameter TELEMETRY, not a complete
// compatibility fingerprint. It deliberately does not incorporate the prompt
// prefix hashes (base instructions / project context / skills — the
// complete_prefix signal the agent loop already emits per turn). Any future
// stateful response reuse MUST combine both signals plus the remaining
// wire-affecting fields before reusing provider-side state; these counters
// alone are not sufficient compatibility protection.
func (s *turnSession) computeFingerprint(request zeroruntime.CompletionRequest) string {
var builder strings.Builder
builder.WriteString(s.provider.model)
builder.WriteByte('|')
fmt.Fprintf(&builder, "%d", s.provider.maxTokens)
builder.WriteByte('|')
builder.WriteString(openAIReasoningEffort(request.ReasoningEffort))
builder.WriteByte('|')
builder.WriteString(request.PromptCacheKey)
builder.WriteByte('|')
builder.WriteString(wireToolsDigest(request.Tools))
sum := sha256.Sum256([]byte(builder.String()))
return hex.EncodeToString(sum[:])
}
// wireToolsDigest renders the tool set in WIRE ORDER: each tool as name +
// description + its JSON-marshaled parameters (encoding/json sorts map keys,
// so each render is stable). Order is preserved deliberately — the OpenAI
// request serializes tools in the order given, so a reorder changes the
// request bytes and must count as drift. Descriptions are model-visible
// request bytes too. A schema that fails to marshal is recorded under a
// stable sentinel — this digest feeds telemetry counters, so cruder handling
// of that pathological case is acceptable.
func wireToolsDigest(tools []zeroruntime.ToolDefinition) string {
rendered := make([]string, 0, len(tools))
for _, tool := range tools {
schema, err := json.Marshal(tool.Parameters)
if err != nil {
rendered = append(rendered, tool.Name+"\n"+tool.Description+"\n__non_json:"+tool.Name)
continue
}
rendered = append(rendered, tool.Name+"\n"+tool.Description+"\n"+string(schema))
}
sum := sha256.Sum256([]byte(strings.Join(rendered, "\x00")))
return hex.EncodeToString(sum[:])
}
-387
View File
@@ -1,387 +0,0 @@
package openai
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// keepAliveClient returns a client whose transport retains idle connections on
// EVERY platform. The shared default transport disables keep-alives on macOS,
// where the session correctly skips the probe — tests that assert the probe
// fires must therefore pin a reusable transport to stay platform-deterministic.
func keepAliveClient() *http.Client {
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.DisableKeepAlives = false
return &http.Client{Transport: transport}
}
// newPrewarmTestProvider builds a provider whose transport always allows the
// prewarm probe, regardless of host platform.
func newPrewarmTestProvider(t *testing.T, handler http.HandlerFunc) *Provider {
t.Helper()
return newTestProviderWithOptions(t, Options{APIKey: "sk-test", HTTPClient: keepAliveClient()}, handler)
}
func waitPrewarmDone(t *testing.T, session *turnSession) {
t.Helper()
if session.prewarmDone == nil {
t.Fatal("prewarm was never launched")
}
select {
case <-session.prewarmDone:
case <-time.After(10 * time.Second):
t.Fatal("prewarm probe did not settle in time")
}
}
func openOptimizedSession(t *testing.T, provider *Provider) *turnSession {
t.Helper()
session, err := NewTurnSessionProvider(provider, zeroruntime.ProviderCapabilities{}).OpenTurnSession(context.Background())
if err != nil {
t.Fatalf("OpenTurnSession: %v", err)
}
return session.(*turnSession)
}
func TestTurnSessionPrewarmSendsSingleHEAD(t *testing.T) {
var requests atomic.Int64
var lastMethod atomic.Value
provider := newPrewarmTestProvider(t, func(w http.ResponseWriter, r *http.Request) {
requests.Add(1)
lastMethod.Store(r.Method)
w.WriteHeader(http.StatusMethodNotAllowed)
})
session := openOptimizedSession(t, provider)
if err := session.Prewarm(context.Background()); err != nil {
t.Fatalf("Prewarm: %v", err)
}
waitPrewarmDone(t, session)
if got := requests.Load(); got != 1 {
t.Fatalf("prewarm sent %d requests, want exactly 1 (no retries even on 405)", got)
}
if method, _ := lastMethod.Load().(string); method != http.MethodHead {
t.Fatalf("prewarm method = %q, want HEAD", method)
}
// A second Prewarm (e.g. after a mid-run swap re-open) must not re-probe.
if err := session.Prewarm(context.Background()); err != nil {
t.Fatalf("second Prewarm: %v", err)
}
waitPrewarmDone(t, session)
if got := requests.Load(); got != 1 {
t.Fatalf("second Prewarm re-probed: %d requests, want 1", got)
}
}
func TestTurnSessionPrewarmNonFatalOnConnectError(t *testing.T) {
// A server that is immediately closed leaves a port that refuses connections.
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
deadURL := server.URL
server.Close()
provider, err := New(Options{APIKey: "sk-test", BaseURL: deadURL, Model: "gpt-test", HTTPClient: keepAliveClient()})
if err != nil {
t.Fatalf("New: %v", err)
}
session := openOptimizedSession(t, provider)
if err := session.Prewarm(context.Background()); err != nil {
t.Fatalf("Prewarm must be advisory even when the probe cannot connect, got %v", err)
}
waitPrewarmDone(t, session)
}
func TestTurnSessionPrewarmTraceStamps(t *testing.T) {
provider := newPrewarmTestProvider(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusMethodNotAllowed)
})
recorder := trace.NewRecorder("session-test", "run-1", "test")
recorder.Start()
ctx := trace.WithContext(context.Background(), recorder)
session := openOptimizedSession(t, provider)
if err := session.Prewarm(ctx); err != nil {
t.Fatalf("Prewarm: %v", err)
}
waitPrewarmDone(t, session)
tr := recorder.Finish()
if got := tr.Counter(trace.CounterPrewarmAttempts); got != 1 {
t.Fatalf("prewarm_attempts = %d, want 1", got)
}
var sawSpan bool
for _, span := range tr.Spans {
if span.Name == trace.SpanProviderPrewarm {
sawSpan = true
}
}
if !sawSpan {
t.Fatalf("missing %s span in %+v", trace.SpanProviderPrewarm, tr.Spans)
}
}
func TestTurnSessionStreamDelegatesToProvider(t *testing.T) {
handler := func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
_, _ = w.Write([]byte("data: {\"choices\":[{\"delta\":{\"content\":\"hello\"}}]}\n\n"))
_, _ = w.Write([]byte("data: [DONE]\n\n"))
}
direct := newTestProvider(t, handler)
viaSession := newTestProvider(t, handler)
session := openOptimizedSession(t, viaSession)
request := zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "hi"}},
}
collect := func(stream <-chan zeroruntime.StreamEvent) []zeroruntime.StreamEvent {
var events []zeroruntime.StreamEvent
for event := range stream {
events = append(events, event)
}
return events
}
directStream, err := direct.StreamCompletion(context.Background(), request)
if err != nil {
t.Fatalf("direct StreamCompletion: %v", err)
}
sessionStream, err := session.Stream(context.Background(), request)
if err != nil {
t.Fatalf("session Stream: %v", err)
}
directEvents := collect(directStream)
sessionEvents := collect(sessionStream)
if len(directEvents) != len(sessionEvents) {
t.Fatalf("event counts diverged: direct=%d session=%d", len(directEvents), len(sessionEvents))
}
for i := range directEvents {
if directEvents[i].Type != sessionEvents[i].Type || directEvents[i].Content != sessionEvents[i].Content {
t.Fatalf("event %d diverged: direct=%+v session=%+v", i, directEvents[i], sessionEvents[i])
}
}
}
func TestTurnSessionFingerprintIgnoresMessages(t *testing.T) {
provider := newTestProvider(t, func(http.ResponseWriter, *http.Request) {})
session := openOptimizedSession(t, provider)
tools := []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}}
base := session.computeFingerprint(zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "turn one"}},
Tools: tools,
PromptCacheKey: "session-a",
})
grown := session.computeFingerprint(zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{
{Role: zeroruntime.MessageRoleUser, Content: "turn one"},
{Role: zeroruntime.MessageRoleAssistant, Content: "reply"},
{Role: zeroruntime.MessageRoleUser, Content: "turn two"},
},
Tools: tools,
PromptCacheKey: "session-a",
})
if base != grown {
t.Fatal("fingerprint changed when only messages grew — conversation content must not be an input")
}
}
func TestTurnSessionFingerprintDriftOnCacheKey(t *testing.T) {
provider := newTestProvider(t, func(http.ResponseWriter, *http.Request) {})
session := openOptimizedSession(t, provider)
tools := []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}}
base := session.computeFingerprint(zeroruntime.CompletionRequest{Tools: tools, PromptCacheKey: "session-a"})
changed := session.computeFingerprint(zeroruntime.CompletionRequest{Tools: tools, PromptCacheKey: "session-b"})
if base == changed {
t.Fatal("fingerprint did not drift on a changed prompt-cache key — it is a serialized request field")
}
}
func TestTurnSessionFingerprintDriftOnToolsAndEffort(t *testing.T) {
provider := newTestProvider(t, func(http.ResponseWriter, *http.Request) {})
session := openOptimizedSession(t, provider)
base := session.computeFingerprint(zeroruntime.CompletionRequest{
Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}},
})
changedTools := session.computeFingerprint(zeroruntime.CompletionRequest{
Tools: []zeroruntime.ToolDefinition{{Name: "write_file", Parameters: map[string]any{"type": "object"}}},
})
if base == changedTools {
t.Fatal("fingerprint did not drift on a changed tool set")
}
changedEffort := session.computeFingerprint(zeroruntime.CompletionRequest{
Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}},
ReasoningEffort: "high",
})
if base == changedEffort {
t.Fatal("fingerprint did not drift on a changed reasoning effort")
}
// Unrecognized efforts normalize to omitted — same as empty.
droppedEffort := session.computeFingerprint(zeroruntime.CompletionRequest{
Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}},
ReasoningEffort: "xhigh",
})
if base != droppedEffort {
t.Fatal("an effort the wire would omit must fingerprint like no effort")
}
}
func TestTurnSessionFingerprintDriftOnToolDescription(t *testing.T) {
provider := newTestProvider(t, func(http.ResponseWriter, *http.Request) {})
session := openOptimizedSession(t, provider)
base := session.computeFingerprint(zeroruntime.CompletionRequest{
Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Description: "Reads a file.", Parameters: map[string]any{"type": "object"}}},
})
changedDescription := session.computeFingerprint(zeroruntime.CompletionRequest{
Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Description: "Reads a file, now with ranges.", Parameters: map[string]any{"type": "object"}}},
})
if base == changedDescription {
t.Fatal("fingerprint did not drift on a description-only tool change — descriptions are model-visible request bytes")
}
}
func TestTurnSessionFingerprintDriftOnToolReorder(t *testing.T) {
provider := newTestProvider(t, func(http.ResponseWriter, *http.Request) {})
session := openOptimizedSession(t, provider)
forward := session.computeFingerprint(zeroruntime.CompletionRequest{Tools: []zeroruntime.ToolDefinition{
{Name: "alpha", Parameters: map[string]any{"type": "object"}},
{Name: "beta", Parameters: map[string]any{"type": "object"}},
}})
reversed := session.computeFingerprint(zeroruntime.CompletionRequest{Tools: []zeroruntime.ToolDefinition{
{Name: "beta", Parameters: map[string]any{"type": "object"}},
{Name: "alpha", Parameters: map[string]any{"type": "object"}},
}})
if forward == reversed {
t.Fatal("fingerprint did not drift on a tool reorder — request serialization preserves tool order, so a reorder changes the wire bytes")
}
}
// TestTurnSessionPrewarmDoesNotStampProviderConnect guards the A/B metric: the
// prewarm probe must appear ONLY under provider_prewarm — a provider_connect
// stamp from the probe would contaminate the exact span the benchmark compares.
func TestTurnSessionPrewarmDoesNotStampProviderConnect(t *testing.T) {
provider := newPrewarmTestProvider(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusMethodNotAllowed)
})
recorder := trace.NewRecorder("session-test", "run-1", "test")
recorder.Start()
ctx := trace.WithContext(context.Background(), recorder)
session := openOptimizedSession(t, provider)
if err := session.Prewarm(ctx); err != nil {
t.Fatalf("Prewarm: %v", err)
}
waitPrewarmDone(t, session)
tr := recorder.Finish()
for _, span := range tr.Spans {
if span.Name == trace.SpanProviderConnect {
t.Fatalf("prewarm stamped a %s span — it must stay solely under %s", trace.SpanProviderConnect, trace.SpanProviderPrewarm)
}
}
}
// TestTurnSessionPrewarmSkippedWhenKeepAlivesDisabled verifies the probe is not
// spent when the transport cannot retain idle connections (the macOS shared
// transport): no request, no trace stamps, and the done channel still closes.
func TestTurnSessionPrewarmSkippedWhenKeepAlivesDisabled(t *testing.T) {
var requests atomic.Int64
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
requests.Add(1)
w.WriteHeader(http.StatusMethodNotAllowed)
}))
defer server.Close()
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.DisableKeepAlives = true
provider, err := New(Options{
APIKey: "sk-test",
BaseURL: server.URL,
Model: "gpt-test",
HTTPClient: &http.Client{Transport: transport},
})
if err != nil {
t.Fatalf("New: %v", err)
}
recorder := trace.NewRecorder("session-test", "run-1", "test")
recorder.Start()
ctx := trace.WithContext(context.Background(), recorder)
session := openOptimizedSession(t, provider)
if err := session.Prewarm(ctx); err != nil {
t.Fatalf("Prewarm: %v", err)
}
waitPrewarmDone(t, session)
if got := requests.Load(); got != 0 {
t.Fatalf("keep-alive-disabled transport still received %d probe requests, want 0", got)
}
tr := recorder.Finish()
if got := tr.Counter(trace.CounterPrewarmAttempts); got != 0 {
t.Fatalf("prewarm_attempts = %d, want 0 for a skipped probe", got)
}
}
func TestTurnSessionPrefixCounters(t *testing.T) {
handler := func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
_, _ = w.Write([]byte("data: [DONE]\n\n"))
}
provider := newTestProvider(t, handler)
session := openOptimizedSession(t, provider)
recorder := trace.NewRecorder("session-test", "run-1", "test")
recorder.Start()
ctx := trace.WithContext(context.Background(), recorder)
stable := zeroruntime.CompletionRequest{Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}}}
drain := func(request zeroruntime.CompletionRequest) {
stream, err := session.Stream(ctx, request)
if err != nil {
t.Fatalf("Stream: %v", err)
}
for range stream {
}
}
drain(stable) // seeds — neither stable nor drift
drain(stable) // stable
drain(zeroruntime.CompletionRequest{Tools: []zeroruntime.ToolDefinition{{Name: "write_file", Parameters: map[string]any{"type": "object"}}}}) // drift
tr := recorder.Finish()
if got := tr.Counter(trace.CounterPrefixStable); got != 1 {
t.Fatalf("prefix_stable = %d, want 1", got)
}
if got := tr.Counter(trace.CounterPrefixDrift); got != 1 {
t.Fatalf("prefix_drift = %d, want 1", got)
}
}
func TestTurnSessionCloseIdempotentAndCompactUnsupported(t *testing.T) {
provider := newTestProvider(t, func(http.ResponseWriter, *http.Request) {})
session := openOptimizedSession(t, provider)
if err := session.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
if err := session.Close(); err != nil {
t.Fatalf("second Close: %v", err)
}
if _, err := session.Compact(context.Background(), zeroruntime.CompletionRequest{}); !errors.Is(err, zeroruntime.ErrCompactionUnsupported) {
t.Fatalf("Compact error = %v, want ErrCompactionUnsupported", err)
}
}
-72
View File
@@ -1,72 +0,0 @@
package providers
import (
"os"
"strings"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/providers/openai"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// openaiTurnSessionEnv gates the optimized OpenAI turn session (prewarm +
// prefix telemetry). Default OFF: unset, "0", or "false" (any case) leave the
// agent loop's default adapter path untouched. Set to any other non-empty
// value (e.g. "1") to enable — the same boolean idiom as ZERO_FORMAT_ON_WRITE.
const openaiTurnSessionEnv = "ZERO_OPENAI_TURN_SESSION"
func openaiTurnSessionEnabled() bool {
value := strings.TrimSpace(os.Getenv(openaiTurnSessionEnv))
return value != "" && value != "0" && !strings.EqualFold(value, "false")
}
// OptimizedTurnSessions returns the gated optimized TurnSessionProvider for an
// already-built provider, or (nil, false) when the gate is off or the profile
// is ineligible. Callers leave agent Options.TurnSessionProvider nil on false,
// so the loop's default wrap keeps today's behavior byte-identical.
//
// Eligibility is deliberately narrow: the resolved provider kind must be
// official OpenAI (NOT openai-compatible gateways — the constructor merges the
// two, so this branches on the resolved kind explicitly), not the ChatGPT
// Codex catalog, and the provider value must be the concrete *openai.Provider
// (a fake, a Codex provider, or nil falls back to the default path safely).
// No base-URL check on top of the kind: prewarm and fingerprint telemetry are
// harmless against any host, and kind==openai mirrors the existing
// official-OpenAI precedent used for prompt_cache_key.
func OptimizedTurnSessions(profile config.ProviderProfile, provider zeroruntime.Provider, options Options) (zeroruntime.TurnSessionProvider, bool) {
if !openaiTurnSessionEnabled() {
return nil, false
}
resolved, err := resolveProfile(profile, options)
if err != nil || resolved.providerKind != config.ProviderKindOpenAI || isCodexCatalog(profile, resolved) {
return nil, false
}
concrete, ok := provider.(*openai.Provider)
if !ok {
return nil, false
}
caps, err := resolveCapabilities(profile, options)
if err != nil {
// resolveProfile above already succeeded, so this is effectively
// unreachable — but never ship an optimized session with unknown
// capabilities; the default path is the safe answer.
return nil, false
}
return openai.NewTurnSessionProvider(concrete, caps), true
}
// DefaultTurnSessions wraps an already-built provider in the DEFAULT (no-op)
// turn-session adapter with the profile's resolved capability projection. It is
// the non-optimized sibling of OptimizedTurnSessions for callers that still
// want capabilities populated — e.g. the mid-run model-switch fallback when the
// switched model is not eligible for the optimized session. Capability
// resolution failure degrades to an unknown (zero) projection rather than
// failing the wrap: the default session has no behavior that depends on
// capabilities, so a swap must never be blocked by a projection error.
func DefaultTurnSessions(profile config.ProviderProfile, provider zeroruntime.Provider, options Options) zeroruntime.TurnSessionProvider {
caps, err := resolveCapabilities(profile, options)
if err != nil {
caps = zeroruntime.ProviderCapabilities{}
}
return zeroruntime.NewProviderTurnSessionProvider(provider, caps)
}
@@ -1,128 +0,0 @@
package providers
import (
"context"
"testing"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
func openaiEligibleProfile() config.ProviderProfile {
return config.ProviderProfile{
Name: "official-openai",
ProviderKind: config.ProviderKindOpenAI,
BaseURL: "https://provider.example/v1",
APIKey: "sk-gate-test",
Model: "pr8-unregistered-model",
}
}
func buildGateProvider(t *testing.T, profile config.ProviderProfile) zeroruntime.Provider {
t.Helper()
provider, err := New(profile, Options{UserAgent: "zero-gate-test"})
if err != nil {
t.Fatalf("New: %v", err)
}
return provider
}
type fakeGateProvider struct{}
func (fakeGateProvider) StreamCompletion(context.Context, zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
return nil, nil
}
func TestOptimizedTurnSessionsDefaultOff(t *testing.T) {
// No env set: the gate must be off even for a fully eligible profile.
profile := openaiEligibleProfile()
tsp, ok := OptimizedTurnSessions(profile, buildGateProvider(t, profile), Options{})
if ok || tsp != nil {
t.Fatal("optimized turn sessions must be disabled by default")
}
}
func TestOptimizedTurnSessionsFalseyValues(t *testing.T) {
profile := openaiEligibleProfile()
provider := buildGateProvider(t, profile)
for _, value := range []string{"0", "false", "FALSE", "False", " ", ""} {
t.Setenv(openaiTurnSessionEnv, value)
if _, ok := OptimizedTurnSessions(profile, provider, Options{}); ok {
t.Fatalf("gate enabled for falsey value %q", value)
}
}
for _, value := range []string{"1", "true", "on"} {
t.Setenv(openaiTurnSessionEnv, value)
if _, ok := OptimizedTurnSessions(profile, provider, Options{}); !ok {
t.Fatalf("gate disabled for truthy value %q", value)
}
}
}
func TestOptimizedTurnSessionsOpenAIEligible(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, "1")
profile := openaiEligibleProfile()
tsp, ok := OptimizedTurnSessions(profile, buildGateProvider(t, profile), Options{})
if !ok || tsp == nil {
t.Fatal("expected the optimized session for an official-OpenAI profile with the gate on")
}
session, err := tsp.OpenTurnSession(context.Background())
if err != nil {
t.Fatalf("OpenTurnSession: %v", err)
}
if err := session.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
if caps := tsp.Capabilities(); caps.Model != "pr8-unregistered-model" {
t.Fatalf("Capabilities().Model = %q, want the resolved api model", caps.Model)
}
}
func TestOptimizedTurnSessionsRejectsCompatible(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, "1")
profile := openaiEligibleProfile()
profile.ProviderKind = config.ProviderKindOpenAICompatible
if _, ok := OptimizedTurnSessions(profile, buildGateProvider(t, profile), Options{}); ok {
t.Fatal("openai-compatible gateways must not get the optimized session")
}
}
func TestOptimizedTurnSessionsRejectsCodexCatalog(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, "1")
profile := openaiEligibleProfile()
profile.CatalogID = "chatgpt"
// Build the provider from the base profile: the point under test is the
// catalog check, not Codex construction (which needs OAuth wiring).
if _, ok := OptimizedTurnSessions(profile, buildGateProvider(t, openaiEligibleProfile()), Options{}); ok {
t.Fatal("the ChatGPT Codex catalog must not get the optimized session")
}
}
func TestDefaultTurnSessionsPreservesResolvedCapabilities(t *testing.T) {
profile := openaiEligibleProfile()
tsp := DefaultTurnSessions(profile, buildGateProvider(t, profile), Options{})
if tsp == nil {
t.Fatal("DefaultTurnSessions returned nil")
}
caps := tsp.Capabilities()
if caps.Model != "pr8-unregistered-model" {
t.Fatalf("Capabilities().Model = %q, want the resolved api model", caps.Model)
}
session, err := tsp.OpenTurnSession(context.Background())
if err != nil {
t.Fatalf("OpenTurnSession: %v", err)
}
if err := session.Close(); err != nil {
t.Fatalf("Close: %v", err)
}
}
func TestOptimizedTurnSessionsRejectsForeignProviderValue(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, "1")
if _, ok := OptimizedTurnSessions(openaiEligibleProfile(), fakeGateProvider{}, Options{}); ok {
t.Fatal("a non-*openai.Provider value must fall back to the default path")
}
if _, ok := OptimizedTurnSessions(openaiEligibleProfile(), nil, Options{}); ok {
t.Fatal("a nil provider must fall back to the default path")
}
}
-70
View File
@@ -65,76 +65,6 @@ func AnalyzeCommand(script string) AnalysisResult {
return result
}
// astCommandFields parses command with the shell parser and returns each simple
// command as its literal field slice (program + args as text), resolving the
// real command positions across quoting, command substitution, subshells, and
// newline separators — the constructs the hand-written splitter in
// safe_command.go mis-handles (issue #473). It returns nil when the command
// cannot be parsed (e.g. a Windows cmd.exe string), so callers fall through to
// the regex path rather than hard-blocking.
func astCommandFields(command string) [][]string {
file, err := syntax.NewParser().Parse(strings.NewReader(command), "")
if err != nil {
return nil
}
var commands [][]string
syntax.Walk(file, func(node syntax.Node) bool {
call, ok := node.(*syntax.CallExpr)
if !ok || len(call.Args) == 0 {
return true
}
// EVERY word must be a static literal, not just the program name.
// wordText keeps only the literal/quoted parts of a word and silently
// drops expansions, so a dynamic word anywhere in the call reconstructs
// to something the shell will never run: `$(printf foo)vim` (runs as
// `foovim`) collapses to "vim", and `git $(printf foo)rebase -i` (runs
// as `git foorebase -i`, non-interactive) collapses to `git rebase -i`
// and fabricates an interactive match. Since the runtime value of an
// expansion is unknowable here, skip the whole call rather than classify
// a lossy reconstruction. Skipping is the safe direction: the
// hand-written passes above already ran, and a missed detection falls
// through to the normal permission prompt instead of hard-blocking a
// command the user never wrote.
for _, word := range call.Args {
if !isLiteralWord(word) {
return true
}
}
fields := make([]string, 0, len(call.Args))
for _, word := range call.Args {
fields = append(fields, wordText(word))
}
commands = append(commands, fields)
return true
})
return commands
}
// isLiteralWord reports whether every part of word is a static literal (bare or
// quoted). A word containing a command substitution, parameter/arithmetic
// expansion, process substitution, etc. is dynamic — its runtime value is
// unknown, so its wordText (a partial literal) must not be trusted as a program
// name.
func isLiteralWord(word *syntax.Word) bool {
if word == nil {
return false
}
for _, part := range word.Parts {
switch typed := part.(type) {
case *syntax.Lit, *syntax.SglQuoted:
case *syntax.DblQuoted:
for _, inner := range typed.Parts {
if _, ok := inner.(*syntax.Lit); !ok {
return false
}
}
default:
return false
}
}
return true
}
// analyzeInto parses script and folds its interactive/destructive/network usage
// into result, sharing seen so program names are de-duplicated across recursion.
func analyzeInto(script string, result *AnalysisResult, seen map[string]bool, depth int) {
-34
View File
@@ -93,40 +93,6 @@ func TestSandboxedBashAutoAllowDoesNotAffectNonShell(t *testing.T) {
}
}
// Forged re-entrancy markers (ambient ZERO_SANDBOXED=1 + ZERO_SANDBOX_BACKEND at
// an unsandboxed process boundary) must NOT let a shell command combine an
// auto-allow decision with an unwrapped pass-through plan (issue #727). The
// runner skips wrapping when the markers are present, so the policy must stop
// auto-allowing on the basis of an "active sandbox" — the command takes the
// normal prompt instead. Without the markers the identical setup auto-allows
// (TestSandboxedBashAutoAllowedWhenSandboxActive), so this is the before/after.
func TestForgedReentrancyMarkersDoNotAutoAllowUnwrappedShell(t *testing.T) {
t.Setenv(EnvSandboxed, "1")
t.Setenv(EnvSandboxBackend, string(BackendLinuxBwrap))
root := t.TempDir()
engine := NewEngine(EngineOptions{
WorkspaceRoot: root,
Policy: DefaultPolicy(),
Backend: nativeWrappingBackend,
})
// Policy side: the shell command must NOT be auto-allowed.
decision := engine.Evaluate(context.Background(), bashRequest())
if decision.Action != ActionPrompt || decision.AutoAllowed {
t.Fatalf("forged markers must not auto-allow the shell command, got %#v", decision)
}
// Planning side: the same command is an unwrapped pass-through. Together with
// the prompt above, this proves auto-allow and unwrapped can no longer combine.
plan, err := engine.BuildCommandPlan(CommandSpec{Name: "/bin/sh", Args: []string{"-c", "echo hi"}, Dir: root})
if err != nil {
t.Fatalf("BuildCommandPlan: %v", err)
}
if plan.Wrapped {
t.Fatalf("re-entrancy plan must be an unwrapped pass-through, got wrapped=%v", plan.Wrapped)
}
}
// TestShellSandboxActive reports correctly across backends and policy modes.
func TestShellSandboxActive(t *testing.T) {
root := t.TempDir()
-9
View File
@@ -232,15 +232,6 @@ func (engine *Engine) shellSandboxActive(policy Policy) bool {
if policy.Mode == ModeDisabled {
return false
}
// Re-entrancy skips wrapping: when the process carries the sandbox markers,
// the runner returns a pass-through (unwrapped) plan, so the native sandbox
// is NOT the boundary for THIS command. Because the markers are ambient and
// forgeable at an unsandboxed process boundary (issue #727), auto-allowing a
// shell command on that basis would run it unwrapped with no prompt. Mirror
// the runner: report inactive so the command takes the normal approval path.
if IsAlreadySandboxed() {
return false
}
backend := engine.backend
if !(backend.Available && backend.Executable != "" && backend.CommandWrapping && backend.NativeIsolation) {
return false
-56
View File
@@ -219,65 +219,9 @@ func DetectInteractiveCommand(command string, goos string) InteractiveCommandRes
}
}
// AST second opinion (issue #473): the hand-written segment split above
// mis-handles interactive programs hidden by unusual quoting, command
// substitution, subshells, or newline separators. Re-extract the real simple
// commands from the parsed shell tree and apply the SAME full pipeline
// (interactive segments, sh -c payload recursion, per-program lookup), so a
// bypass is caught while every program stays classified exactly as before
// (ssh with a trailing command, python -c, etc. remain allowed). A command
// the parser cannot handle (Windows cmd.exe, obfuscation) yields no commands
// and falls through unchanged — the guard never hard-blocks on a parse error.
for _, fields := range astCommandFields(command) {
if result, ok := inspectCommandFields(fields, goos); ok {
return result
}
}
return InteractiveCommandResult{}
}
// inspectCommandFields applies the full interactive-detection pipeline to one
// already-split command's fields: the multi-word interactive segments, then the
// `sh -c <payload>` recursion, then the per-program lookup with its
// non-interactive suppressions (hasNonInteractiveFlag covers REPL flags and
// trailing-command clients like ssh). It mirrors the hand-written passes above
// so an AST-extracted command is classified identically to a plainly-split one.
func inspectCommandFields(fields []string, goos string) (InteractiveCommandResult, bool) {
body := strings.ToLower(commandBody(fields))
for _, seg := range interactiveSegments {
if body == seg.match || strings.HasPrefix(body, seg.match+" ") {
suggestion := seg.suggestion
if goos == "windows" && seg.windowsSuggestion != "" {
suggestion = seg.windowsSuggestion
}
return InteractiveCommandResult{Interactive: true, Command: seg.command, Reason: seg.reason, Suggestion: suggestion}, true
}
}
first := firstProgram(fields)
if first == "" {
return InteractiveCommandResult{}, false
}
if payload := shellDashCPayload(first, fields); payload != "" {
if inner := DetectInteractiveCommand(payload, goos); inner.Interactive {
return inner, true
}
return InteractiveCommandResult{}, false
}
program, ok := interactivePrograms[first]
if !ok || (program.windowsOnly && goos != "windows") {
return InteractiveCommandResult{}, false
}
if hasNonInteractiveFlag(first, fields) {
return InteractiveCommandResult{}, false
}
suggestion := program.suggestion
if goos == "windows" && program.windowsSuggestion != "" {
suggestion = program.windowsSuggestion
}
return InteractiveCommandResult{Interactive: true, Command: first, Reason: program.reason, Suggestion: suggestion}, true
}
// wrapperPrograms are launcher prefixes that precede the real program. After
// one of these we keep scanning for the actual executable.
var wrapperPrograms = map[string]bool{
-84
View File
@@ -406,87 +406,3 @@ func TestDetectInteractiveMongoEvalAndFullPaths(t *testing.T) {
}
}
}
// The hand-written segment splitter misses interactive programs hidden by
// constructs only a real shell parser resolves (a newline separator collapsed
// to a space; a brace group that shifts the real command position). The AST
// second opinion must catch them (issue #473).
func TestDetectInteractiveCommandCatchesParserBypasses(t *testing.T) {
for _, cmd := range []string{
"echo hi\nvim file.txt", // newline separator collapsed to a space by the regex path
"{ vim file.txt; }", // brace group hides the real program position
} {
got := DetectInteractiveCommand(cmd, "linux")
if !got.Interactive {
t.Errorf("DetectInteractiveCommand(%q) = not interactive, want caught (parser bypass)", cmd)
continue
}
if got.Command != "vim" {
t.Errorf("DetectInteractiveCommand(%q).Command = %q, want vim", cmd, got.Command)
}
}
}
// The AST second opinion must not flag an interactive program NAME that appears
// only inside a quoted argument (not a real command position) — the parser
// distinguishes program from argument, so this must stay non-interactive.
func TestDetectInteractiveCommandNoFalsePositiveOnQuotedArgument(t *testing.T) {
got := DetectInteractiveCommand(`echo "please run vim later"`, "linux")
if got.Interactive {
t.Fatalf("interactive name inside a quoted argument must not be flagged, got %#v", got)
}
}
// The AST second opinion must run the FULL pipeline — multi-word interactive
// segments and sh -c payload recursion, not just the per-program lookup — so a
// grouped/nested bypass is caught; and it must not fabricate a program name from
// a command substitution concatenated with a literal (issue #473, review).
func TestDetectInteractiveCommandASTPipelineBoundaries(t *testing.T) {
interactive := []struct{ cmd, wantCmd string }{
{"{ git rebase -i HEAD~1; }", "git rebase -i"}, // multi-word segment in a brace group
{"{ sh -c 'vim file.txt'; }", "vim"}, // sh -c payload in a brace group
}
for _, tc := range interactive {
got := DetectInteractiveCommand(tc.cmd, "linux")
if !got.Interactive || got.Command != tc.wantCmd {
t.Errorf("DetectInteractiveCommand(%q) = %+v, want interactive Command=%q", tc.cmd, got, tc.wantCmd)
}
}
}
// The AST pass must not fabricate a program name from a dynamic (non-literal)
// program word: `$(printf '%s' foo)vim` runs as `foovim`, not vim. Tested at the
// extractor so the hand-written pass (which pre-empts DetectInteractiveCommand
// with its own, separate handling of that construct) does not mask the guard.
func TestAstCommandFieldsSkipsDynamicProgram(t *testing.T) {
for _, fields := range astCommandFields("$(printf '%s' foo)vim file.txt") {
if firstProgram(fields) == "vim" {
t.Fatalf("astCommandFields fabricated a vim command from a substitution: %v", fields)
}
}
}
// The literalness guard covers every word, not just the program name. A dynamic
// ARGUMENT is dropped by wordText the same way a dynamic program word is, so
// `git $(printf foo)rebase -i HEAD~1` (which runs as `git foorebase -i`, a
// non-existent and non-interactive subcommand) would otherwise be reconstructed
// as `git rebase -i` and blocked — a false positive on a command the user never
// wrote. Asserted through DetectInteractiveCommand as well as the extractor,
// because the whole point is what the caller ends up blocking.
func TestAstCommandFieldsSkipsDynamicArgument(t *testing.T) {
const dynamic = "{ git $(printf foo)rebase -i HEAD~1 ; }"
for _, fields := range astCommandFields(dynamic) {
if firstProgram(fields) == "git" {
t.Fatalf("astCommandFields reconstructed a git command from a dynamic argument: %v", fields)
}
}
if got := DetectInteractiveCommand(dynamic, "linux"); got.Interactive {
t.Errorf("DetectInteractiveCommand(%q) = %+v, want not interactive", dynamic, got)
}
// The genuine form must still be detected — the guard skips lossy
// reconstructions, it does not weaken real detection.
const genuine = "{ git rebase -i HEAD~1 ; }"
if got := DetectInteractiveCommand(genuine, "linux"); !got.Interactive || got.Command != "git rebase -i" {
t.Errorf("DetectInteractiveCommand(%q) = %+v, want interactive Command=%q", genuine, got, "git rebase -i")
}
}
+1 -4
View File
@@ -89,7 +89,7 @@ func PrepareExec(options PrepareExecOptions) (PreparedExec, error) {
if resumeID != "" || options.ResumeLatest {
sessionID := resumeID
if sessionID == "" && options.ResumeLatest {
latest, err := store.LatestResumable()
latest, err := store.Latest()
if err != nil {
return PreparedExec{}, err
}
@@ -105,9 +105,6 @@ func PrepareExec(options PrepareExecOptions) (PreparedExec, error) {
if session == nil {
return PreparedExec{}, ExecError{"Zero session not found: " + sessionID}
}
if !IsResumableKind(session.SessionKind) {
return PreparedExec{}, ExecError{"Zero session is not resumable: " + sessionID}
}
contextEvents, err := readExecContextEvents(store, session.SessionID)
if err != nil {
return PreparedExec{}, err
+10 -20
View File
@@ -52,7 +52,6 @@ type SessionKind string
const (
SessionKindFork SessionKind = "fork"
SessionKindChild SessionKind = "child"
SessionKindSide SessionKind = "side"
SessionKindSpecDraft SessionKind = "spec-draft"
SessionKindSpecImpl SessionKind = "spec-impl"
)
@@ -126,13 +125,11 @@ type CreateInput struct {
}
type ForkInput struct {
SessionID string
SessionKind SessionKind
Title string
Cwd string
ModelID string
Provider string
Tag string
SessionID string
Title string
Cwd string
ModelID string
Provider string
}
type ChildInput struct {
@@ -357,9 +354,10 @@ func (store *Store) Latest() (*Metadata, error) {
}
// IsResumableKind reports whether a session kind represents a standalone,
// user-resumable conversation rather than a transient or agent-owned sub-run.
// Regular ("") and user fork sessions are resumable; side, child, and spec
// sessions are not, so they do not flood the resume picker.
// user-resumable conversation rather than an agent sub-run. Regular ("") and
// user fork sessions are resumable; child (specialist/sub-agent) and spec
// draft/impl sessions are not — each agent task and /spec run creates one, so
// listing them in the resume picker floods it with non-conversation entries.
func IsResumableKind(kind SessionKind) bool {
switch kind {
case "", SessionKindFork:
@@ -420,21 +418,13 @@ func (store *Store) Fork(parentSessionID string, input ForkInput) (Metadata, err
if title == "" && parent.Title != "" {
title = parent.Title + " (fork)"
}
kind := input.SessionKind
if kind == "" {
kind = SessionKindFork
}
if kind != SessionKindFork && kind != SessionKindSide {
return Metadata{}, fmt.Errorf("invalid zero fork session kind %q", kind)
}
fork, err := store.Create(CreateInput{
SessionID: input.SessionID,
SessionKind: kind,
SessionKind: SessionKindFork,
Title: title,
Cwd: firstNonEmpty(input.Cwd, parent.Cwd),
ModelID: firstNonEmpty(input.ModelID, parent.ModelID),
Provider: firstNonEmpty(input.Provider, parent.Provider),
Tag: input.Tag,
ParentSessionID: parent.SessionID,
RootSessionID: firstNonEmpty(parent.RootSessionID, parent.SessionID),
ForkedFromEventID: last.ID,
+1 -34
View File
@@ -181,32 +181,6 @@ func TestStoreForkCopiesEventsAndLineage(t *testing.T) {
}
}
func TestStoreForkSupportsNonResumableSideSession(t *testing.T) {
store := NewStore(StoreOptions{RootDir: t.TempDir()})
parent, err := store.Create(CreateInput{SessionID: "parent", Title: "Parent"})
if err != nil {
t.Fatalf("Create parent: %v", err)
}
if _, err := store.AppendEvent(parent.SessionID, AppendEventInput{Type: EventMessage, Payload: map[string]string{"content": "context"}}); err != nil {
t.Fatalf("AppendEvent: %v", err)
}
side, err := store.Fork(parent.SessionID, ForkInput{
SessionID: "side",
SessionKind: SessionKindSide,
Tag: "btw",
})
if err != nil {
t.Fatalf("Fork side: %v", err)
}
if side.SessionKind != SessionKindSide || side.Tag != "btw" || side.ParentSessionID != parent.SessionID {
t.Fatalf("side metadata = %#v", side)
}
if IsResumableKind(side.SessionKind) {
t.Fatal("side sessions must not appear as resumable conversations")
}
}
func TestStoreForkSkipsUsageEvents(t *testing.T) {
store := NewStore(StoreOptions{RootDir: t.TempDir(), Now: fixedClock("2026-06-04T11:00:00Z")})
parent, err := store.Create(CreateInput{SessionID: "parent", Title: "Parent"})
@@ -407,7 +381,6 @@ func TestListAndLatestResumableExcludeSubRuns(t *testing.T) {
mk(SessionKindSpecImpl, "spec-impl-1")
newestResumable := mk("", "conversation-2") // newest standalone conversation
mk(SessionKindChild, "child-2") // newer overall, but a sub-run
mk(SessionKindSide, "side-1") // newer overall, but non-resumable
resumable, err := store.ListResumable()
if err != nil {
@@ -431,7 +404,7 @@ func TestListAndLatestResumableExcludeSubRuns(t *testing.T) {
if latest != nil {
got = latest.SessionID
}
t.Fatalf("LatestResumable = %s, want newest resumable %s (must skip newer child and side sessions)", got, newestResumable.SessionID)
t.Fatalf("LatestResumable = %s, want newest resumable %s (must skip the newer child)", got, newestResumable.SessionID)
}
}
@@ -562,9 +535,6 @@ func TestPrepareExecSessionResolvesResumeAndFork(t *testing.T) {
if _, err := store.AppendEvent("latest", AppendEventInput{Type: EventMessage, Payload: map[string]any{"content": "previous answer"}}); err != nil {
t.Fatal(err)
}
if _, err := store.Create(CreateInput{SessionID: "newer-side", SessionKind: SessionKindSide}); err != nil {
t.Fatal(err)
}
prepared, err := PrepareExec(PrepareExecOptions{Store: store, ResumeLatest: true})
if err != nil {
@@ -576,9 +546,6 @@ func TestPrepareExecSessionResolvesResumeAndFork(t *testing.T) {
if got := FormatExecPrompt("continue", prepared); got == "continue" || !strings.Contains(got, "previous answer") {
t.Fatalf("expected session context in prompt, got %q", got)
}
if _, err := PrepareExec(PrepareExecOptions{Store: store, Resume: "newer-side"}); err == nil || !strings.Contains(err.Error(), "Zero session is not resumable: newer-side") {
t.Fatalf("explicit side-session resume error = %v, want non-resumable rejection", err)
}
forked, err := PrepareExec(PrepareExecOptions{Store: store, Fork: "latest", SessionID: "forked"})
if err != nil {
+6 -41
View File
@@ -23,15 +23,6 @@ type bashTool struct {
scope PathScope
}
func (bashTool) outputCategory(args map[string]any) outputCategory {
command, _ := bashCommandArg(args)
return shellOutputCategory(command)
}
func bashCommandArg(args map[string]any) (string, error) {
return aliasedStringArg(args, []string{"command", "cmd", "script", "shell"}, "", true, false)
}
func NewBashTool(workspaceRoot string) Tool {
return NewScopedBashTool(workspaceRoot, nil)
}
@@ -72,19 +63,15 @@ func NewScopedBashTool(workspaceRoot string, scope PathScope) Tool {
}
func (tool bashTool) Run(ctx context.Context, args map[string]any) Result {
return tool.run(ctx, args, nil, true)
return tool.run(ctx, args, nil)
}
func (tool bashTool) RunWithSandbox(ctx context.Context, args map[string]any, engine *zeroSandbox.Engine) Result {
return tool.run(ctx, args, engine, true)
return tool.run(ctx, args, engine)
}
func (tool bashTool) RunWithOptions(ctx context.Context, args map[string]any, options RunOptions) Result {
return tool.run(ctx, args, options.Sandbox, false)
}
func (tool bashTool) run(ctx context.Context, args map[string]any, engine *zeroSandbox.Engine, directBudget bool) Result {
commandText, err := bashCommandArg(args)
func (tool bashTool) run(ctx context.Context, args map[string]any, engine *zeroSandbox.Engine) Result {
commandText, err := aliasedStringArg(args, []string{"command", "cmd", "script", "shell"}, "", true, false)
if err != nil {
return errorResult("Error: Invalid arguments for bash: " + err.Error())
}
@@ -185,7 +172,7 @@ func (tool bashTool) run(ctx context.Context, args map[string]any, engine *zeroS
}
}
markLikelySandboxDenial(meta, plan, exitCode, stdoutText, stderrText)
outText, errText, truncated := prepareBashOutput(stdoutText, stdout.total, stderrText, stderrTotal, meta, directBudget)
outText, errText, truncated := budgetBashCapture(stdoutText, stdout.total, stderrText, stderrTotal, meta)
return Result{
Status: StatusError,
Output: formatBashOutputWithShellHint(outText, errText, exitCode, meta),
@@ -195,7 +182,7 @@ func (tool bashTool) run(ctx context.Context, args map[string]any, engine *zeroS
}
markLikelySandboxDenial(meta, plan, exitCode, stdoutText, stderrText)
outText, errText, truncated := prepareBashOutput(stdoutText, stdout.total, stderrText, stderrTotal, meta, directBudget)
outText, errText, truncated := budgetBashCapture(stdoutText, stdout.total, stderrText, stderrTotal, meta)
if meta[SandboxLikelyDeniedMeta] == "true" {
return Result{
Status: StatusError,
@@ -428,28 +415,6 @@ func budgetBashOutput(stdout string, stderr string, meta map[string]string) (str
return budgetBashCapture(stdout, len(stdout), stderr, len(stderr), meta)
}
// prepareBashOutput keeps direct callers on the established positional budget.
// Registry calls retain the existing bounded capture but leave final semantic
// reduction and spill creation to the post-redaction registry boundary.
func prepareBashOutput(out string, outTotal int, errStr string, errTotal int, meta map[string]string, directBudget bool) (string, string, bool) {
if directBudget {
return budgetBashCapture(out, outTotal, errStr, errTotal, meta)
}
outText := sectionWithCaptureGap(out, outTotal)
errText := sectionWithCaptureGap(errStr, errTotal)
truncated := outTotal > len(out) || errTotal > len(errStr)
if meta != nil {
meta["raw_bytes"] = strconv.Itoa(outTotal + errTotal)
meta["emitted_bytes"] = strconv.Itoa(len(outText) + len(errText))
meta["estimated_tokens"] = strconv.Itoa(estimatedTokensFromBytes(len(outText) + len(errText)))
if truncated {
meta["truncated"] = "true"
meta["truncation_reason"] = "capture_budget"
}
}
return outText, errText, truncated
}
// budgetBashCapture is budgetBashOutput for the streaming-capture path: outTotal
// and errTotal are the true byte counts (from boundedBuffer.total), which may
// exceed the retained strings when the middle was dropped during capture. Meta's
-12
View File
@@ -37,18 +37,6 @@ func TestBudgetBashOutputSmallPassesThrough(t *testing.T) {
}
}
func TestPrepareBashOutputDefersRegistrySemanticBudget(t *testing.T) {
output := "START\n" + strings.Repeat("progress\n", 8_000) + "ERROR: retained\nEND"
registryOutput, _, registryTruncated := prepareBashOutput(output, len(output), "", 0, map[string]string{}, false)
if registryTruncated || registryOutput != output {
t.Fatalf("registry preparation must preserve capture-bounded output for semantic reduction: truncated=%t bytes=%d", registryTruncated, len(registryOutput))
}
directOutput, _, directTruncated := prepareBashOutput(output, len(output), "", 0, map[string]string{}, true)
if !directTruncated || len(directOutput) > bashOutputBudgetBytes+512 {
t.Fatalf("direct path must preserve legacy positional budget: truncated=%t bytes=%d", directTruncated, len(directOutput))
}
}
// Oversized stdout is truncated head+tail: both the first and last lines survive,
// the middle is dropped behind a marker, meta is flagged, and the captured text
// is spilled to a re-readable file.
+15 -60
View File
@@ -427,15 +427,6 @@ type execCommandTool struct {
manager *execSessionManager
}
func (execCommandTool) outputCategory(args map[string]any) outputCategory {
command, _ := execCommandArg(args)
return shellOutputCategory(command)
}
func execCommandArg(args map[string]any) (string, error) {
return aliasedStringArg(args, []string{"cmd", "command", "script", "shell"}, "", true, false)
}
func NewExecCommandTool(workspaceRoot string, manager *execSessionManager) Tool {
return NewScopedExecCommandTool(workspaceRoot, nil, manager)
}
@@ -484,15 +475,11 @@ func NewScopedExecCommandTool(workspaceRoot string, scope PathScope, manager *ex
}
func (tool execCommandTool) Run(ctx context.Context, args map[string]any) Result {
return tool.run(ctx, args, nil, true)
return tool.run(ctx, args, nil)
}
func (tool execCommandTool) RunWithSandbox(ctx context.Context, args map[string]any, engine *zeroSandbox.Engine) Result {
return tool.run(ctx, args, engine, true)
}
func (tool execCommandTool) RunWithOptions(ctx context.Context, args map[string]any, options RunOptions) Result {
return tool.run(ctx, args, options.Sandbox, false)
return tool.run(ctx, args, engine)
}
func (tool execCommandTool) ExecSessions() []ExecSessionSnapshot {
@@ -507,8 +494,8 @@ func (tool execCommandTool) StopAllExecSessions() []int {
return tool.manager.stopAll()
}
func (tool execCommandTool) run(ctx context.Context, args map[string]any, engine *zeroSandbox.Engine, directBudget bool) Result {
commandText, err := execCommandArg(args)
func (tool execCommandTool) run(ctx context.Context, args map[string]any, engine *zeroSandbox.Engine) Result {
commandText, err := aliasedStringArg(args, []string{"cmd", "command", "script", "shell"}, "", true, false)
if err != nil {
return errorResult("Error: Invalid arguments for exec_command: " + err.Error())
}
@@ -563,7 +550,7 @@ func (tool execCommandTool) run(ctx context.Context, args map[string]any, engine
if exited {
tool.manager.remove(session.id)
}
return execToolResultWithBudget(execToolResultInput{
return execToolResult(execToolResultInput{
commandText: commandText,
output: output,
outputBufferTruncated: outputTruncated,
@@ -574,7 +561,7 @@ func (tool execCommandTool) run(ctx context.Context, args map[string]any, engine
tty: session.tty,
plan: session.plan,
maxOutputTokens: maxOutputTokens,
}, directBudget)
})
}
func (tool execCommandTool) startSession(commandText string, absoluteCwd string, relativeCwd string, ttyRequested bool, engine *zeroSandbox.Engine, sandboxPermissions SandboxPermissionOverride) (*execSession, error) {
@@ -707,22 +694,6 @@ type writeStdinTool struct {
manager *execSessionManager
}
// UnknownExecSessionError is the result returned when write_stdin targets a
// session_id with no live exec session. The stable recovery guidance leads and
// the numeric id trails on purpose: the agent's repeated-failure guard keys on
// a normalized, truncated prefix of the error string, so keeping the id out of
// that prefix makes a model probing ids 1, 2, 3, … produce ONE signature that
// finally trips the halt instead of a fresh signature per id — while also
// telling the model how to actually recover. See TestUnknownExecSessionErrorSignatureIsIDInvariant
// in internal/agent, which pins this invariant against the real guard.
func UnknownExecSessionError(sessionID int) string {
return fmt.Sprintf("Error: write_stdin needs a session_id returned by a still-running exec_command; do not guess or probe session ids. Start a process with exec_command, or use write_file/edit_file/apply_patch for file changes. (no live session %d)", sessionID)
}
func (writeStdinTool) outputCategory(map[string]any) outputCategory {
return outputCategoryProcess
}
func NewWriteStdinTool(manager *execSessionManager) Tool {
if manager == nil {
manager = defaultExecSessionManager
@@ -734,7 +705,7 @@ func NewWriteStdinTool(manager *execSessionManager) Tool {
parameters: Schema{
Type: "object",
Properties: map[string]PropertySchema{
"session_id": {Type: "integer", Description: "Identifier of a running unified exec session, as returned by exec_command. Never guess or probe ids.", Minimum: intPtr(1)},
"session_id": {Type: "integer", Description: "Identifier of the running unified exec session."},
"chars": {Type: "string", Description: "Bytes to write to stdin. Defaults to empty, which polls without writing.", Default: ""},
"yield_time_ms": {Type: "integer", Description: "Wait before yielding output. Non-empty writes default to 250 ms and cap at 30000 ms; empty polls wait 5000-300000 ms by default.", Default: defaultPollYieldTimeMS, Minimum: intPtr(1), Maximum: intPtr(maxPollYieldTimeMS)},
"max_output_tokens": {Type: "integer", Description: "Output token budget. Defaults to 10000 tokens; larger requests may be capped by policy.", Default: defaultMaxOutputTokens, Minimum: intPtr(1), Maximum: intPtr(maxExecOutputTokenRequest)},
@@ -775,20 +746,12 @@ func (tool writeStdinTool) Run(ctx context.Context, args map[string]any) Result
}
func (tool writeStdinTool) RunWithOptions(ctx context.Context, args map[string]any, _ RunOptions) Result {
// A missing, non-integer, or < 1 session_id all mean the same thing: the model
// has no live session to write to. Route them to the SAME recovery guidance the
// no-live-session case uses (UnknownExecSessionError) instead of a terse
// "session_id must be at least 1". The terse error gives no way forward and, by
// naming the minimum, nudges the model to try 1, 2, 3... — the exact id-probing
// #749 works to suppress. The recovery message instead tells it to start a
// session or edit files directly, and because that message is id-invariant, the
// missing/zero/non-integer entry points and id-probing collapse to ONE
// repeated-failure signature, so any mix of them accumulates toward the halt
// rather than resetting the streak on each class of mistake.
value, present := args["session_id"]
sessionID, sessionErr := intArg(args, "session_id", 0, 1, 0)
if !present || value == nil || sessionErr != nil {
return errorResult(UnknownExecSessionError(sessionID))
if value, ok := args["session_id"]; !ok || value == nil {
return errorResult("Error: Invalid arguments for write_stdin: session_id is required")
}
sessionID, err := intArg(args, "session_id", 0, 1, 0)
if err != nil {
return errorResult("Error: Invalid arguments for write_stdin: " + err.Error())
}
chars, err := stringArgWithEmpty(args, "chars", "", false, true)
if err != nil {
@@ -804,7 +767,7 @@ func (tool writeStdinTool) RunWithOptions(ctx context.Context, args map[string]a
}
session, ok := tool.manager.get(sessionID)
if !ok {
return errorResult(UnknownExecSessionError(sessionID))
return errorResult(fmt.Sprintf("Error: Unknown exec session_id %d.", sessionID))
}
session.touch()
interrupted := false
@@ -880,15 +843,7 @@ type execToolResultInput struct {
}
func execToolResult(input execToolResultInput) Result {
return execToolResultWithBudget(input, true)
}
func execToolResultWithBudget(input execToolResultInput, directBudget bool) Result {
output := input.output
truncated := false
if directBudget {
output, truncated = truncateExecOutput(input.output, input.maxOutputTokens)
}
output, truncated := truncateExecOutput(input.output, input.maxOutputTokens)
meta := map[string]string{
"cwd": input.relativeCwd,
"tty": strconv.FormatBool(input.tty),
+13 -46
View File
@@ -881,7 +881,7 @@ func TestRegistryHonorsWriteStdinArgumentPermission(t *testing.T) {
registry.Register(NewWriteStdinTool(newExecSessionManager()))
poll := registry.Run(context.Background(), WriteStdinToolName, map[string]any{"session_id": 9999})
if poll.Status != StatusError || !strings.Contains(poll.Output, "still-running exec_command") {
if poll.Status != StatusError || !strings.Contains(poll.Output, "Unknown exec session_id") {
t.Fatalf("empty poll should reach tool without permission prompt, got status=%s output=%q", poll.Status, poll.Output)
}
@@ -901,57 +901,24 @@ func TestWriteStdinReportsUnknownSession(t *testing.T) {
if result.Status != StatusError {
t.Fatalf("status = %s, want error", result.Status)
}
// The message must lead with recovery guidance and still identify the id.
if !strings.Contains(result.Output, "still-running exec_command") {
t.Fatalf("unknown-session error must carry recovery guidance, got: %q", result.Output)
}
if !strings.Contains(result.Output, "1234") {
t.Fatalf("unknown-session error must name the offending id, got: %q", result.Output)
if !strings.Contains(result.Output, "Unknown exec session_id 1234") {
t.Fatalf("unexpected output: %q", result.Output)
}
}
// The runtime rejects session_id < 1 (intArg min 1); the advertised schema must
// say the same so a model sees the constraint before it probes id 0.
func TestWriteStdinSchemaPinsSessionIDMinimum(t *testing.T) {
schema := NewWriteStdinTool(nil).(writeStdinTool).parameters
prop, ok := schema.Properties["session_id"]
if !ok {
t.Fatal("session_id property missing from write_stdin schema")
}
if prop.Minimum == nil || *prop.Minimum != 1 {
t.Fatalf("session_id schema Minimum = %v, want 1 to match the runtime floor", prop.Minimum)
}
}
// A missing, zero, negative, or non-integer session_id all mean the model has no
// live session, so write_stdin returns the SAME recovery guidance the no-live-
// session case uses (start a session with exec_command, or edit files directly),
// not a terse "session_id must be at least 1" that gives no way forward and names
// a minimum that nudges the model to probe ids 1, 2, 3... Sharing one id-invariant
// message also keeps a single repeated-failure signature, so any mix of these
// mistakes accumulates toward the halt instead of resetting the streak (#749).
func TestWriteStdinRequiresPositiveSessionID(t *testing.T) {
tool := NewWriteStdinTool(newExecSessionManager())
want := UnknownExecSessionError(0)
for name, args := range map[string]map[string]any{
"missing": {},
"nil": {"session_id": nil},
"zero": {"session_id": 0},
"negative": {"session_id": -3},
"non-integer": {"session_id": "abc"},
for _, args := range []map[string]any{
{},
{"session_id": 0},
} {
t.Run(name, func(t *testing.T) {
result := tool.Run(context.Background(), args)
if result.Status != StatusError {
t.Fatalf("status = %s, want error", result.Status)
}
if result.Output != want {
t.Fatalf("output = %q,\n want the recovery message %q", result.Output, want)
}
if strings.Contains(result.Output, "must be at least 1") {
t.Fatalf("still returns the terse minimum error: %q", result.Output)
}
})
result := tool.Run(context.Background(), args)
if result.Status != StatusError {
t.Fatalf("Run(%#v) status = %s, want error", args, result.Status)
}
if !strings.Contains(result.Output, "Invalid arguments for write_stdin") {
t.Fatalf("Run(%#v) output = %q, want invalid arguments", args, result.Output)
}
}
}
-58
View File
@@ -829,61 +829,3 @@ func TestGlobSkipsWorkspaceExcludedDirectoriesAndBinaryFiles(t *testing.T) {
t.Fatalf("expected only keep.txt, got:\n%s", res.Output)
}
}
// A backwards range (end_line < start_line) is a caller-arithmetic slip, not a
// fatal error: read_file recovers by reading just start_line and surfaces a note
// explaining the adjustment, matching how an end_line past EOF is auto-clamped.
// It must never hard-error the way it used to.
func TestReadFileToolRecoversBackwardsRange(t *testing.T) {
root := t.TempDir()
writeTestFile(t, filepath.Join(root, "notes.txt"), "alpha\nbeta\ngamma\ndelta\nepsilon")
result := NewReadFileTool(root).Run(context.Background(), map[string]any{
"path": "notes.txt",
"start_line": 3,
"end_line": 2, // backwards
})
if result.Status != StatusOK {
t.Fatalf("backwards range must recover, not error; got %s: %s", result.Status, result.Output)
}
if strings.Contains(result.Output, "must be greater than or equal to start_line") {
t.Fatalf("backwards range still hard-errors: %q", result.Output)
}
for _, want := range []string{
"end_line 2 was before start_line 3", // the note
"only line 3 was read",
"3 | gamma",
} {
if !strings.Contains(result.Output, want) {
t.Fatalf("expected %q in output, got %q", want, result.Output)
}
}
// It read exactly start_line, nothing adjacent.
if strings.Contains(result.Output, "beta") || strings.Contains(result.Output, "delta") {
t.Fatalf("recovery read the wrong lines: %q", result.Output)
}
}
// A valid range with end_line >= start_line is byte-for-byte unchanged: the
// recovery path must not touch normal reads (no stray note, exact same window).
func TestReadFileToolValidRangeUnchangedByRecovery(t *testing.T) {
root := t.TempDir()
writeTestFile(t, filepath.Join(root, "notes.txt"), "alpha\nbeta\ngamma\ndelta\nepsilon")
result := NewReadFileTool(root).Run(context.Background(), map[string]any{
"path": "notes.txt",
"start_line": 2,
"end_line": 4,
})
if result.Status != StatusOK {
t.Fatalf("valid range failed: %s: %s", result.Status, result.Output)
}
// Byte-for-byte: header, blank separator, the exact selected lines, and no
// recovery note or trailing content. An altered header/separator or a stray
// out-of-range line would fail this where a substring check would not.
const want = "File: notes.txt (lines 2-4 of 5)\n\n2 | beta\n3 | gamma\n4 | delta"
if result.Output != want {
t.Fatalf("valid-range output changed by the recovery path:\n got: %q\nwant: %q", result.Output, want)
}
}
+14 -23
View File
@@ -18,8 +18,6 @@ type globTool struct {
scope PathScope
}
func (globTool) outputCategory(map[string]any) outputCategory { return outputCategorySearch }
func NewGlobTool(workspaceRoot string) Tool {
return NewScopedGlobTool(workspaceRoot, nil)
}
@@ -49,25 +47,17 @@ func NewScopedGlobTool(workspaceRoot string, scope PathScope) Tool {
}
func (tool globTool) Run(ctx context.Context, args map[string]any) Result {
return tool.runWith(ctx, args, readExcluder{}, true)
}
func (tool globTool) RunWithOptions(ctx context.Context, args map[string]any, options RunOptions) Result {
exclude := readExcluder{}
if options.Sandbox != nil {
exclude = sandboxReadExcluder(options.Sandbox)
}
return tool.runWith(ctx, args, exclude, false)
return tool.runWith(ctx, args, readExcluder{})
}
// RunWithSandbox runs glob while skipping subtrees the sandbox policy denies
// reads to (DenyRead). With no DenyRead configured the excluder is a no-op and
// behavior is unchanged.
func (tool globTool) RunWithSandbox(ctx context.Context, args map[string]any, engine *sandbox.Engine) Result {
return tool.runWith(ctx, args, sandboxReadExcluder(engine), true)
return tool.runWith(ctx, args, sandboxReadExcluder(engine))
}
func (tool globTool) runWith(ctx context.Context, args map[string]any, exclude readExcluder, directBudget bool) Result {
func (tool globTool) runWith(ctx context.Context, args map[string]any, exclude readExcluder) Result {
pattern, err := aliasedStringArg(args, []string{"pattern", "glob", "match", "query", "expression"}, "", true, false)
if err != nil {
return errorResult("Error: Invalid arguments for glob: " + err.Error())
@@ -120,23 +110,24 @@ func (tool globTool) runWith(ctx context.Context, args map[string]any, exclude r
if truncated {
output += fmt.Sprintf("\n\n[truncated: showing first %d of %d matches; increase limit or narrow cwd/pattern]", len(matches), totalMatches)
}
meta := map[string]string{}
budgeted := applyOutputBudget(output, searchOutputBudgetBytes, "increase limit or narrow cwd/pattern")
meta := outputBudgetMeta(budgeted)
meta["pattern"] = pattern
if truncated {
if truncated || budgeted.Truncated {
meta["truncated"] = "true"
meta["truncation_reason"] = "limit"
if budgeted.Truncated {
meta["truncation_reason"] = "byte_budget"
} else {
meta["truncation_reason"] = "limit"
}
}
result := Result{
return Result{
Status: StatusOK,
Output: output,
Truncated: truncated,
Output: budgeted.Output,
Truncated: truncated || budgeted.Truncated,
Meta: meta,
}
if directBudget {
return applyLegacyByteBudgetToResult(result, searchOutputBudgetBytes, "increase limit or narrow cwd/pattern")
}
return result
}
func scanGlob(ctx context.Context, root string, displayRoot string, matcher *regexp.Regexp, includeDirs bool, exclude readExcluder) ([]string, error) {
+23 -29
View File
@@ -21,8 +21,6 @@ type grepTool struct {
scope PathScope
}
func (grepTool) outputCategory(map[string]any) outputCategory { return outputCategorySearch }
type grepMatch struct {
file string
line int
@@ -64,15 +62,7 @@ func NewScopedGrepTool(workspaceRoot string, scope PathScope) Tool {
}
func (tool grepTool) Run(ctx context.Context, args map[string]any) Result {
return tool.runWith(ctx, args, readExcluder{}, true)
}
func (tool grepTool) RunWithOptions(ctx context.Context, args map[string]any, options RunOptions) Result {
exclude := readExcluder{}
if options.Sandbox != nil {
exclude = sandboxReadExcluder(options.Sandbox)
}
return tool.runWith(ctx, args, exclude, false)
return tool.runWith(ctx, args, readExcluder{})
}
// RunWithSandbox runs the search while skipping subtrees the sandbox policy
@@ -80,10 +70,10 @@ func (tool grepTool) RunWithOptions(ctx context.Context, args map[string]any, op
// path. With no DenyRead configured the excluder is a no-op and behavior is
// unchanged.
func (tool grepTool) RunWithSandbox(ctx context.Context, args map[string]any, engine *sandbox.Engine) Result {
return tool.runWith(ctx, args, sandboxReadExcluder(engine), true)
return tool.runWith(ctx, args, sandboxReadExcluder(engine))
}
func (tool grepTool) runWith(ctx context.Context, args map[string]any, exclude readExcluder, directBudget bool) Result {
func (tool grepTool) runWith(ctx context.Context, args map[string]any, exclude readExcluder) Result {
pattern, err := aliasedStringArg(args, []string{"pattern", "query", "regex", "search", "expression"}, "", true, false)
if err != nil {
return errorResult("Error: Invalid arguments for grep: " + err.Error())
@@ -164,7 +154,7 @@ func (tool grepTool) runWith(ctx context.Context, args map[string]any, exclude r
}
return errorResult("Error running grep: " + err.Error())
}
return applyDirectSearchBudget(collector.result(), directBudget, "narrow path/glob/pattern to continue")
return collector.result()
case "files_with_matches":
collector := &grepFileListCollector{}
if err := scanGrepMatches(ctx, resolvedRoot, target, globMatcher, exclude, absolutePaths, presenceGrepLineMatcher(compiled), collector.collect); err != nil {
@@ -173,7 +163,7 @@ func (tool grepTool) runWith(ctx context.Context, args map[string]any, exclude r
}
return errorResult("Error running grep: " + err.Error())
}
return applyDirectSearchBudget(collector.result(), directBudget, "narrow path/glob/pattern to continue")
return collector.result()
default:
collector := &grepContentCollector{headLimit: headLimit}
if err := scanGrepMatches(ctx, resolvedRoot, target, globMatcher, exclude, absolutePaths, presenceGrepLineMatcher(compiled), collector.collect); err != nil {
@@ -182,17 +172,10 @@ func (tool grepTool) runWith(ctx context.Context, args map[string]any, exclude r
}
return errorResult("Error running grep: " + err.Error())
}
return applyDirectSearchBudget(collector.result(), directBudget, "narrow path/glob/pattern or increase head_limit")
return collector.result()
}
}
func applyDirectSearchBudget(result Result, directBudget bool, hint string) Result {
if !directBudget {
return result
}
return applyLegacyByteBudgetToResult(result, searchOutputBudgetBytes, hint)
}
// resolveGrepRoot picks the scope root whose EvalSymlinks-resolved path contains
// the already-resolved target, so that confineGrepFile computes correct
// workspace-relative paths even when the target lives in an extra root.
@@ -472,7 +455,13 @@ func (collector *grepFileListCollector) result() Result {
return okResult("No matches found.")
}
sort.Strings(collector.files)
return Result{Status: StatusOK, Output: strings.Join(collector.files, "\n")}
budgeted := applyOutputBudget(strings.Join(collector.files, "\n"), searchOutputBudgetBytes, "narrow path/glob/pattern to continue")
meta := outputBudgetMeta(budgeted)
if budgeted.Truncated {
meta["truncated"] = "true"
meta["truncation_reason"] = "byte_budget"
}
return Result{Status: StatusOK, Output: budgeted.Output, Truncated: budgeted.Truncated, Meta: meta}
}
type grepContentCollector struct {
@@ -505,15 +494,20 @@ func (collector *grepContentCollector) result() Result {
if truncated {
output += fmt.Sprintf("\n\n[truncated: showing first %d matches; narrow path/glob/pattern or increase head_limit]", len(lines))
}
meta := map[string]string{}
if truncated {
budgeted := applyOutputBudget(output, searchOutputBudgetBytes, "narrow path/glob/pattern or increase head_limit")
meta := outputBudgetMeta(budgeted)
if truncated || budgeted.Truncated {
meta["truncated"] = "true"
meta["truncation_reason"] = "head_limit"
if budgeted.Truncated {
meta["truncation_reason"] = "byte_budget"
} else {
meta["truncation_reason"] = "head_limit"
}
}
return Result{
Status: StatusOK,
Output: output,
Truncated: truncated,
Output: budgeted.Output,
Truncated: truncated || budgeted.Truncated,
Meta: meta,
}
}
+6 -16
View File
@@ -15,10 +15,6 @@ type listDirectoryTool struct {
scope PathScope
}
func (listDirectoryTool) outputCategory(map[string]any) outputCategory {
return outputCategorySearch
}
func NewListDirectoryTool(workspaceRoot string) Tool {
return NewScopedListDirectoryTool(workspaceRoot, nil)
}
@@ -46,14 +42,6 @@ func NewScopedListDirectoryTool(workspaceRoot string, scope PathScope) Tool {
}
func (tool listDirectoryTool) Run(_ context.Context, args map[string]any) Result {
return tool.run(args, true)
}
func (tool listDirectoryTool) RunWithOptions(_ context.Context, args map[string]any, _ RunOptions) Result {
return tool.run(args, false)
}
func (tool listDirectoryTool) run(args map[string]any, directBudget bool) Result {
// Optional with a "." default: treat an explicit empty path (a common
// weak-model quirk) the same as the key being absent rather than erroring.
requestedPath, err := aliasedStringArg(args, []string{"path", "directory", "dir"}, ".", false, true)
@@ -88,11 +76,13 @@ func (tool listDirectoryTool) run(args map[string]any, directBudget bool) Result
return okResult("Directory is empty: " + relativePath)
}
output := "Contents of " + relativePath + ":\n\n" + strings.Join(entries, "\n")
result := Result{Status: StatusOK, Output: output}
if directBudget {
return applyLegacyByteBudgetToResult(result, searchOutputBudgetBytes, "use path, recursive=false, or a smaller max_depth to narrow the listing")
budgeted := applyOutputBudget(output, searchOutputBudgetBytes, "use path, recursive=false, or a smaller max_depth to narrow the listing")
meta := outputBudgetMeta(budgeted)
if budgeted.Truncated {
meta["truncated"] = "true"
meta["truncation_reason"] = "byte_budget"
}
return result
return Result{Status: StatusOK, Output: budgeted.Output, Truncated: budgeted.Truncated, Meta: meta}
}
func listDirectoryEntries(path string, depth int, maxDepth int) ([]string, error) {
-254
View File
@@ -1,254 +0,0 @@
package tools
import (
"strconv"
"strings"
)
const (
outputBudgetCategoryMeta = "output_budget_category"
outputBudgetOriginalBytesMeta = "output_budget_original_bytes"
outputBudgetRetainedBytesMeta = "output_budget_retained_bytes"
outputBudgetEstimatedOriginalTokensMeta = "output_budget_estimated_original_tokens"
outputBudgetEstimatedRetainedTokensMeta = "output_budget_estimated_retained_tokens"
outputBudgetReasonMeta = "output_budget_reason"
outputBudgetSpillCreatedMeta = "output_budget_spill_created"
)
// applyRegistryOutputBudget is the common post-redaction semantic budgeting
// boundary for tools that do not already own a deliberate output budget.
func applyRegistryOutputBudget(tool Tool, toolName string, args map[string]any, result Result) Result {
budget := registryOutputBudget(toolName)
if budget.maxEstimatedTokens <= 0 && budget.hardMaxBytes <= 0 {
return result // preserve ZERO_TOOL_OUTPUT_CEILING_TOKENS=0 semantics
}
category := resolveOutputCategory(tool, toolName, args)
budgeted := budgetSemanticOutput(result.Output, category, budget)
if budgeted.truncated {
budgeted = attachExistingSpill(toolName, result.Output, budget, budgeted)
}
if result.Truncated {
budgeted.truncated = true
if budgeted.reason == "" {
budgeted.reason = result.Meta["truncation_reason"]
if budgeted.reason == "" {
budgeted.reason = "upstream_tool_budget"
}
}
if budgeted.spillPath == "" {
budgeted.spillPath = result.Meta["spill_path"]
}
}
result.Output = budgeted.text
result.Truncated = result.Truncated || budgeted.truncated
result.Meta = addOutputBudgetMetadata(result.Meta, budgeted)
return result
}
// applySelfManagedOutputBudget applies semantic retention to tools that retain
// output with their own capture-aware limits. Their explicit budgets remain
// authoritative, rather than being replaced by the registry default ceiling.
func applySelfManagedOutputBudget(tool Tool, toolName string, args map[string]any, result Result) Result {
budget := selfManagedOutputBudget(toolName, args)
if budget.maxEstimatedTokens <= 0 && budget.hardMaxBytes <= 0 {
return annotateSelfBudgetedOutput(tool, toolName, args, result)
}
category := resolveOutputCategory(tool, toolName, args)
budgeted := budgetSemanticOutput(result.Output, category, budget)
if result.Truncated && !budgeted.truncated {
budgeted.truncated = true
budgeted.reason = result.Meta["truncation_reason"]
if budgeted.reason == "" {
budgeted.reason = "upstream_tool_budget"
}
}
if budgeted.truncated {
budgeted = attachExistingSpill(toolName, result.Output, budget, budgeted)
}
result.Output = budgeted.text
result.Truncated = result.Truncated || budgeted.truncated
result.Meta = addOutputBudgetMetadata(result.Meta, budgeted)
return result
}
func selfManagedOutputBudget(toolName string, args map[string]any) outputBudget {
switch toolName {
case "bash":
// bash previously exposed at most 32 KiB from each stream. Keep that
// combined 64 KiB ceiling while selecting its contents semantically.
return outputBudget{maxEstimatedTokens: (bashOutputBudgetBytes * 2) / 4, hardMaxBytes: bashOutputBudgetBytes * 2}
case ExecCommandToolName:
maxOutputTokens := defaultMaxOutputTokens
if parsed, err := intArg(args, "max_output_tokens", defaultMaxOutputTokens, 1, maxExecOutputTokenRequest); err == nil {
maxOutputTokens = parsed
}
return outputBudget{maxEstimatedTokens: maxOutputTokens, hardMaxBytes: maxOutputTokens * 4}
default:
return outputBudget{}
}
}
// RebudgetAfterHook reapplies the registry's redaction and output limits after
// an afterTool hook appends model-visible feedback to a completed result. The
// initial registry pass has already run; this second pass is limited to the
// newly combined result so hooks cannot bypass the established safety ceiling.
func (registry *Registry) RebudgetAfterHook(toolName string, args map[string]any, result Result) Result {
result = scrubResultSecrets(result)
tool, _ := registry.Get(toolName)
if _, ok := tool.(selfBudgeting); ok {
// Match the primary registry boundary: self-managed tools keep their
// call-specific capture/output budget instead of being tightened or
// loosened to the generic registry ceiling after hook feedback.
return applySelfManagedOutputBudget(tool, toolName, args, result)
}
result = applyRegistryOutputBudget(tool, toolName, args, result)
return enforceOutputCeiling(toolName, result)
}
func registryOutputBudget(toolName string) outputBudget {
switch toolName {
case "read_file", "read_minified_file":
return outputBudget{maxEstimatedTokens: readOutputBudgetBytes / 4, hardMaxBytes: readOutputBudgetBytes}
case "grep", "glob", "list_directory":
return outputBudget{maxEstimatedTokens: searchOutputBudgetBytes / 4, hardMaxBytes: searchOutputBudgetBytes}
default:
ceilingTokens := resolveOutputCeilingTokens()
if ceilingTokens <= 0 {
return outputBudget{}
}
return outputBudget{maxEstimatedTokens: ceilingTokens, hardMaxBytes: ceilingTokens * 4}
}
}
func resolveOutputCategory(tool Tool, toolName string, args map[string]any) outputCategory {
if provider, ok := tool.(outputPolicyProvider); ok {
if category := provider.outputCategory(args); category != "" {
return category
}
}
switch toolName {
case "Task", "swarm_collect":
return outputCategoryWorker
case "apply_patch":
return outputCategoryDiff
case "write_stdin":
return outputCategoryProcess
default:
return outputCategoryDefault
}
}
// annotateSelfBudgetedOutput records the same compact metadata for tools whose
// existing capture/budget implementation remains authoritative in PR11. It
// does not re-budget their text or claim that raw_bytes represents every byte a
// subprocess produced beyond its established capture limits.
func annotateSelfBudgetedOutput(tool Tool, toolName string, args map[string]any, result Result) Result {
retainedBytes := len(result.Output)
originalBytes := retainedBytes
if parsed, err := strconv.Atoi(result.Meta["raw_bytes"]); err == nil && parsed > originalBytes {
originalBytes = parsed
}
retainedTokens := estimateOutputTokens(result.Output)
originalTokens := retainedTokens
if originalBytes > retainedBytes {
originalTokens = max(originalTokens, estimatedTokensFromBytes(originalBytes))
}
reason := result.Meta["truncation_reason"]
if result.Truncated && reason == "" {
reason = "upstream_tool_budget"
}
observed := budgetedOutput{
text: result.Output,
originalBytes: originalBytes,
retainedBytes: retainedBytes,
estimatedOriginalTokens: originalTokens,
estimatedRetainedTokens: retainedTokens,
truncated: result.Truncated,
category: resolveOutputCategory(tool, toolName, args),
reason: reason,
spillPath: result.Meta["spill_path"],
}
result.Meta = addOutputBudgetMetadata(result.Meta, observed)
return result
}
func shellOutputCategory(command string) outputCategory {
normalized := strings.ToLower(strings.TrimSpace(command))
if containsAny(normalized,
"go test", "pytest", "python -m pytest", "cargo test", "npm test", "npm run test",
"pnpm test", "yarn test", "bun test", "dotnet test", "mvn test", "gradle test", "phpunit") {
return outputCategoryTest
}
if containsAny(normalized, "git diff", "git show", "git format-patch", " diff -u", "diff --git") || strings.HasPrefix(normalized, "diff ") {
return outputCategoryDiff
}
return outputCategoryProcess
}
// attachExistingSpill reuses Zero's hardened per-user spill directory. output
// is the already-redacted text received by this layer; it may itself be a
// capture-bounded view produced by a subprocess tool.
func attachExistingSpill(toolName, output string, budget outputBudget, current budgetedOutput) budgetedOutput {
path := spillTruncatedOutput(toolName, output)
if path == "" {
return current
}
notice := "[zero] full output received by budgeting layer saved to " + path + " (grep or read_file it instead of re-running)"
reduced := outputBudget{
maxEstimatedTokens: max(1, budget.maxEstimatedTokens-estimateOutputTokens("\n"+notice)),
hardMaxBytes: max(1, budget.hardMaxBytes-len("\n"+notice)),
}
base := budgetSemanticOutput(output, current.category, reduced)
text := strings.TrimRight(base.text, "\n") + "\n" + notice
if !fitsOutputBudget(text, budget) {
// An unusually long temp path or tiny configured ceiling can leave no
// room for the full notice. Keep the safe bounded result; the spill still
// exists but is intentionally not advertised with a chopped reference.
current.spillPath = path
return current
}
base.text = text
base.retainedBytes = len(text)
base.estimatedRetainedTokens = estimateOutputTokens(text)
base.truncated = true
if base.reason == "" {
base.reason = current.reason
}
base.category = current.category
base.spillPath = path
return base
}
func addOutputBudgetMetadata(meta map[string]string, output budgetedOutput) map[string]string {
if meta == nil {
meta = map[string]string{}
}
meta[outputBudgetCategoryMeta] = string(output.category)
meta[outputBudgetOriginalBytesMeta] = strconv.Itoa(output.originalBytes)
meta[outputBudgetRetainedBytesMeta] = strconv.Itoa(output.retainedBytes)
meta[outputBudgetEstimatedOriginalTokensMeta] = strconv.Itoa(output.estimatedOriginalTokens)
meta[outputBudgetEstimatedRetainedTokensMeta] = strconv.Itoa(output.estimatedRetainedTokens)
meta[outputBudgetSpillCreatedMeta] = strconv.FormatBool(output.spillPath != "")
// These established fields describe the final model-visible output. Refresh
// them on every pass because redaction and after-tool hooks may change the
// text even when this pass does not newly truncate it.
meta["emitted_bytes"] = strconv.Itoa(output.retainedBytes)
meta["estimated_tokens"] = strconv.Itoa(output.estimatedRetainedTokens)
if output.reason != "" {
meta[outputBudgetReasonMeta] = output.reason
}
if output.truncated {
// Preserve the existing metadata vocabulary used by callers and tests.
if _, exists := meta["raw_bytes"]; !exists {
meta["raw_bytes"] = strconv.Itoa(output.originalBytes)
}
meta["truncated"] = "true"
meta["truncation_reason"] = output.reason
}
if output.spillPath != "" {
meta["spill_path"] = output.spillPath
}
return meta
}
-381
View File
@@ -1,381 +0,0 @@
package tools
import (
"context"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"unicode/utf8"
)
func TestRegistryBudgetRunsAfterRedactionAndSpillsRedactedOutput(t *testing.T) {
setTestTempDir(t)
secret := "ghp_" + strings.Repeat("a", 36)
big := "HEAD\n" + secret + "\n" + strings.Repeat("🙂 noisy output\n", 20_000) + "TAIL"
registry := NewRegistry()
registry.Register(newCeilingFakeTool("redacted_big", big))
result := registry.Run(context.Background(), "redacted_big", map[string]any{})
if !result.Truncated || result.Meta[outputBudgetSpillCreatedMeta] != "true" {
t.Fatalf("unexpected budget result: truncated=%t meta=%#v", result.Truncated, result.Meta)
}
if strings.Contains(result.Output, secret) || !utf8.ValidString(result.Output) {
t.Fatalf("exposed output leaked secret or invalid UTF-8: %q", result.Output)
}
spillPath := result.Meta["spill_path"]
content, err := os.ReadFile(spillPath)
if err != nil {
t.Fatalf("read spill: %v", err)
}
if strings.Contains(string(content), secret) {
t.Fatal("spill contains unredacted secret")
}
if !strings.Contains(string(content), "HEAD") || !strings.Contains(string(content), "TAIL") {
t.Fatal("spill does not contain the complete redacted output received by the budget layer")
}
}
func TestRegistryBudgetSpillFailureFallsBackToBoundedOutput(t *testing.T) {
temp := t.TempDir()
blockedTemp := filepath.Join(temp, "not-a-directory")
if err := os.WriteFile(blockedTemp, []byte("x"), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("TMPDIR", blockedTemp)
t.Setenv("TMP", blockedTemp)
t.Setenv("TEMP", blockedTemp)
t.Setenv(outputCeilingEnv, "100")
registry := NewRegistry()
registry.Register(newCeilingFakeTool("spill_failure", strings.Repeat("large output\n", 1000)))
result := registry.Run(context.Background(), "spill_failure", map[string]any{})
if !result.Truncated || len(result.Output) > 400 {
t.Fatalf("fallback is not bounded: truncated=%t bytes=%d", result.Truncated, len(result.Output))
}
if result.Meta[outputBudgetSpillCreatedMeta] != "false" || result.Meta["spill_path"] != "" {
t.Fatalf("spill failure incorrectly advertised: %#v", result.Meta)
}
}
func TestRegistryMissingPolicyUsesDefaultAndExactHardCeiling(t *testing.T) {
setTestTempDir(t)
t.Setenv(outputCeilingEnv, "80")
input := "HEAD\n" + strings.Repeat("x", 3000) + "\nTAIL"
registry := NewRegistry()
registry.Register(newCeilingFakeTool("no_policy", input))
result := registry.Run(context.Background(), "no_policy", map[string]any{})
if got := result.Meta[outputBudgetCategoryMeta]; got != string(outputCategoryDefault) {
t.Fatalf("category = %q, want default", got)
}
if len(result.Output) > 80*4 {
t.Fatalf("output = %d bytes, hard ceiling %d", len(result.Output), 80*4)
}
if result.Meta[outputBudgetOriginalBytesMeta] != strconv.Itoa(len(input)) {
t.Fatalf("original size metadata = %q", result.Meta[outputBudgetOriginalBytesMeta])
}
}
func TestRegistrySmallOutputRemainsByteIdentical(t *testing.T) {
registry := NewRegistry()
registry.Register(newCeilingFakeTool("small_identity", "hello\nworld\n"))
result := registry.Run(context.Background(), "small_identity", map[string]any{})
if result.Output != "hello\nworld\n" || result.Truncated {
t.Fatalf("small output changed: %#v", result)
}
}
func TestDirectReadAndSearchToolsKeepLegacyByteBudgets(t *testing.T) {
root := t.TempDir()
if err := os.WriteFile(filepath.Join(root, "large-read.txt"), []byte(strings.Repeat("source line with enough content\n", 7000)), 0o600); err != nil {
t.Fatal(err)
}
if result := NewReadFileTool(root).Run(context.Background(), map[string]any{"path": "large-read.txt"}); !result.Truncated || len(result.Output) > readOutputBudgetBytes {
t.Fatalf("direct read_file output is not bounded: %#v", result)
}
for index := 0; index < 800; index++ {
name := fmt.Sprintf("%04d-%s.txt", index, strings.Repeat("n", 80))
if err := os.WriteFile(filepath.Join(root, name), []byte("match\n"), 0o600); err != nil {
t.Fatal(err)
}
}
if err := os.WriteFile(filepath.Join(root, "large-content.txt"), []byte(strings.Repeat("match "+strings.Repeat("x", 2000)+"\n", 50)), 0o600); err != nil {
t.Fatal(err)
}
for name, result := range map[string]Result{
"list_directory": NewListDirectoryTool(root).Run(context.Background(), map[string]any{"recursive": false}),
"glob": NewGlobTool(root).Run(context.Background(), map[string]any{"pattern": "*.txt", "limit": 1000}),
"grep files": NewGrepTool(root).Run(context.Background(), map[string]any{"pattern": "match", "output_mode": "files_with_matches"}),
"grep content": NewGrepTool(root).Run(context.Background(), map[string]any{"pattern": "match", "path": "large-content.txt", "head_limit": 50}),
} {
if !result.Truncated || len(result.Output) > searchOutputBudgetBytes {
t.Fatalf("direct %s output is not bounded: truncated=%t bytes=%d meta=%#v", name, result.Truncated, len(result.Output), result.Meta)
}
}
}
func TestRegistryLargeFileUsesSemanticFilePolicy(t *testing.T) {
setTestTempDir(t)
root := t.TempDir()
var content strings.Builder
content.WriteString("HEAD_MARK\n")
for line := 0; line < 7000; line++ {
content.WriteString("ordinary source line with enough text to fill the output budget\n")
}
content.WriteString("TAIL_MARK\n")
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte(content.String()), 0o600); err != nil {
t.Fatal(err)
}
registry := NewRegistry()
registry.Register(NewReadFileTool(root))
result := registry.Run(context.Background(), "read_file", map[string]any{"path": "large.txt"})
if !result.Truncated || result.Meta[outputBudgetCategoryMeta] != string(outputCategoryFile) {
t.Fatalf("large file was not semantically budgeted: truncated=%t meta=%#v", result.Truncated, result.Meta)
}
if len(result.Output) > readOutputBudgetBytes {
t.Fatalf("large file output = %d bytes, ceiling %d", len(result.Output), readOutputBudgetBytes)
}
for _, want := range []string{"File: large.txt", "HEAD_MARK", "TAIL_MARK"} {
if !strings.Contains(result.Output, want) {
t.Fatalf("large file output missing %q", want)
}
}
}
func TestRegistryLargeSingleLineFileRetainsHeadAndTail(t *testing.T) {
setTestTempDir(t)
root := t.TempDir()
content := "HEAD_SINGLE_LINE_MARK" + strings.Repeat("x", readOutputBudgetBytes*2) + "TAIL_SINGLE_LINE_MARK"
if err := os.WriteFile(filepath.Join(root, "large.min.js"), []byte(content), 0o600); err != nil {
t.Fatal(err)
}
registry := NewRegistry()
registry.Register(NewReadFileTool(root))
result := registry.Run(context.Background(), "read_file", map[string]any{"path": "large.min.js"})
if !result.Truncated || result.Meta[outputBudgetCategoryMeta] != string(outputCategoryFile) {
t.Fatalf("single-line file was not semantically budgeted: truncated=%t meta=%#v", result.Truncated, result.Meta)
}
if len(result.Output) > readOutputBudgetBytes || !utf8.ValidString(result.Output) {
t.Fatalf("single-line file output is not safely bounded: bytes=%d valid=%t", len(result.Output), utf8.ValidString(result.Output))
}
if rawBytes, err := strconv.Atoi(result.Meta["raw_bytes"]); err != nil || rawBytes <= len(result.Output) {
t.Fatalf("single-line file capture was not bounded head/tail output: raw=%q emitted=%d", result.Meta["raw_bytes"], len(result.Output))
}
for _, want := range []string{"File: large.min.js", "HEAD_SINGLE_LINE_MARK", "TAIL_SINGLE_LINE_MARK"} {
if !strings.Contains(result.Output, want) {
t.Fatalf("single-line file output missing %q", want)
}
}
}
func TestRegistryMinifiedFileUpdatesEmittedBytesAfterBudgeting(t *testing.T) {
setTestTempDir(t)
root := t.TempDir()
content := "HEAD_MINIFIED_MARK" + strings.Repeat("x", readOutputBudgetBytes*2) + "TAIL_MINIFIED_MARK"
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte(content), 0o600); err != nil {
t.Fatal(err)
}
registry := NewRegistry()
registry.Register(NewReadMinifiedFileTool(root))
result := registry.Run(context.Background(), "read_minified_file", map[string]any{"path": "large.txt"})
if !result.Truncated || result.Meta["emitted_bytes"] != strconv.Itoa(len(result.Output)) {
t.Fatalf("minified metadata does not describe final output: truncated=%t emitted=%q bytes=%d", result.Truncated, result.Meta["emitted_bytes"], len(result.Output))
}
for _, want := range []string{"HEAD_MINIFIED_MARK", "TAIL_MINIFIED_MARK"} {
if !strings.Contains(result.Output, want) {
t.Fatalf("minified single-line file output missing %q", want)
}
}
}
func TestRegistryGrepUsesSemanticMultiFileCoverage(t *testing.T) {
setTestTempDir(t)
root := t.TempDir()
for _, name := range []string{"a.txt", "b.txt", "c.txt", "d.txt"} {
body := strings.Repeat("needle "+strings.Repeat(name, 15)+"\n", 350)
if err := os.WriteFile(filepath.Join(root, name), []byte(body), 0o600); err != nil {
t.Fatal(err)
}
}
registry := NewRegistry()
registry.Register(NewGrepTool(root))
result := registry.Run(context.Background(), "grep", map[string]any{
"pattern": "needle",
"path": ".",
"head_limit": 1400,
})
if !result.Truncated || result.Meta[outputBudgetCategoryMeta] != string(outputCategorySearch) {
t.Fatalf("grep was not semantically budgeted: truncated=%t meta=%#v", result.Truncated, result.Meta)
}
for _, name := range []string{"a.txt:", "b.txt:", "c.txt:", "d.txt:"} {
if !strings.Contains(result.Output, name) {
t.Fatalf("search output lost %s coverage", name)
}
}
}
func TestShellOutputCategoryClassification(t *testing.T) {
tests := []struct {
command string
want outputCategory
}{
{"go test ./...", outputCategoryTest},
{"python -m pytest -q", outputCategoryTest},
{"git diff --stat main...HEAD", outputCategoryDiff},
{"git show HEAD", outputCategoryDiff},
{"make build", outputCategoryProcess},
}
for _, test := range tests {
if got := shellOutputCategory(test.command); got != test.want {
t.Errorf("shellOutputCategory(%q) = %q, want %q", test.command, got, test.want)
}
}
}
func TestShellToolOutputCategoryUsesAllCommandAliases(t *testing.T) {
providers := []struct {
name string
provider outputPolicyProvider
}{
{name: "bash", provider: NewBashTool(t.TempDir()).(outputPolicyProvider)},
{name: "exec_command", provider: NewExecCommandTool(t.TempDir(), newExecSessionManager()).(outputPolicyProvider)},
}
for _, provider := range providers {
for _, alias := range []string{"command", "cmd", "script", "shell"} {
t.Run(provider.name+"/"+alias, func(t *testing.T) {
if got := provider.provider.outputCategory(map[string]any{alias: "go test ./..."}); got != outputCategoryTest {
t.Fatalf("category = %q, want test", got)
}
})
}
}
if got := providers[0].provider.outputCategory(map[string]any{"command": "make build", "cmd": "go test ./..."}); got != outputCategoryProcess {
t.Fatalf("bash alias precedence category = %q, want process", got)
}
if got := providers[1].provider.outputCategory(map[string]any{"cmd": "go test ./...", "command": "make build"}); got != outputCategoryTest {
t.Fatalf("exec_command alias precedence category = %q, want test", got)
}
}
func TestRegistryBudgetPreservesExistingTruncationAndRefreshesMetadata(t *testing.T) {
output := "post-hook output"
result := applyRegistryOutputBudget(newCeilingFakeTool("existing_truncation", output), "existing_truncation", map[string]any{}, Result{
Status: StatusOK,
Output: output,
Truncated: true,
Meta: map[string]string{
"emitted_bytes": "999",
"estimated_tokens": "999",
"truncation_reason": "capture_budget",
},
})
if !result.Truncated || result.Meta["truncated"] != "true" || result.Meta[outputBudgetReasonMeta] != "capture_budget" {
t.Fatalf("prior truncation state was lost: %#v", result)
}
if result.Meta["emitted_bytes"] != strconv.Itoa(len(output)) || result.Meta["estimated_tokens"] != strconv.Itoa(estimateOutputTokens(output)) {
t.Fatalf("final output metadata is stale: %#v", result.Meta)
}
if result.Meta[outputBudgetSpillCreatedMeta] != "false" {
t.Fatalf("existing truncation unexpectedly created a new spill: %#v", result.Meta)
}
}
func TestSelfManagedOutputBudgetUsesSemanticTestPolicy(t *testing.T) {
setTestTempDir(t)
input := "stdout:\n" + strings.Repeat("PASS progress\n", 8_000) +
"--- FAIL: TestImportant (0.01s)\nthing_test.go:42: expected 7, got 9\nFAIL\nexit_code: 1"
tests := []struct {
name string
tool Tool
args map[string]any
}{
{name: "bash", tool: NewBashTool(t.TempDir()), args: map[string]any{"command": "go test ./..."}},
{name: "exec", tool: NewExecCommandTool(t.TempDir(), newExecSessionManager()), args: map[string]any{"cmd": "go test ./...", "max_output_tokens": 16_000}},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
got := applySelfManagedOutputBudget(test.tool, test.tool.Name(), test.args, Result{Status: StatusError, Output: input, Meta: map[string]string{}})
if !got.Truncated || got.Meta[outputBudgetCategoryMeta] != string(outputCategoryTest) {
t.Fatalf("semantic budget was not applied: %#v", got)
}
for _, want := range []string{"TestImportant", "expected 7", "FAIL", "exit_code: 1"} {
if !strings.Contains(got.Output, want) {
t.Fatalf("semantic test output missing %q:\n%s", want, got.Output)
}
}
})
}
}
type semanticBashFakeTool struct{ ceilingFakeTool }
func (semanticBashFakeTool) managesOutputBudget() {}
func (semanticBashFakeTool) outputCategory(args map[string]any) outputCategory {
command, _ := args["command"].(string)
return shellOutputCategory(command)
}
func TestRegistryAppliesSemanticBudgetToSelfManagedShellOutput(t *testing.T) {
setTestTempDir(t)
input := "startup\n" + strings.Repeat("PASS progress\n", 8_000) +
"--- FAIL: TestRegistryImportant (0.01s)\nexpected 7, got 9\nFAIL\nexit_code: 1"
registry := NewRegistry()
registry.Register(semanticBashFakeTool{newCeilingFakeTool("bash", input)})
result := registry.Run(context.Background(), "bash", map[string]any{"command": "go test ./..."})
if !result.Truncated || result.Meta[outputBudgetReasonMeta] != "semantic_test_budget" {
t.Fatalf("registry did not apply semantic test budget: %#v", result.Meta)
}
for _, want := range []string{"TestRegistryImportant", "expected 7", "FAIL", "exit_code: 1"} {
if !strings.Contains(result.Output, want) {
t.Fatalf("registry semantic test output missing %q:\n%s", want, result.Output)
}
}
}
func TestRegistryAppliesSemanticBudgetToSelfManagedProcessOutput(t *testing.T) {
setTestTempDir(t)
input := "starting build\n" + strings.Repeat("progress\n", 8_000) +
"WARNING: cache is cold\nERROR: compilation failed\nexit_code: 1"
registry := NewRegistry()
registry.Register(semanticBashFakeTool{newCeilingFakeTool("bash", input)})
result := registry.Run(context.Background(), "bash", map[string]any{"command": "make build"})
if !result.Truncated || result.Meta[outputBudgetReasonMeta] != "semantic_process_budget" {
t.Fatalf("registry did not apply semantic process budget: %#v", result.Meta)
}
for _, want := range []string{"starting build", "WARNING", "ERROR", "exit_code: 1"} {
if !strings.Contains(result.Output, want) {
t.Fatalf("registry semantic process output missing %q:\n%s", want, result.Output)
}
}
}
func TestRebudgetAfterHookPreservesSelfManagedExecBudget(t *testing.T) {
setTestTempDir(t)
registry := NewRegistry()
registry.Register(NewExecCommandTool(t.TempDir(), newExecSessionManager()))
t.Run("tight call budget remains authoritative", func(t *testing.T) {
args := map[string]any{"cmd": "make build", "max_output_tokens": 10}
result := registry.RebudgetAfterHook(ExecCommandToolName, args, Result{Status: StatusOK, Output: strings.Repeat("x", 1_000)})
if !result.Truncated || len(result.Output) > 40 {
t.Fatalf("post-hook output escaped call budget: truncated=%t bytes=%d meta=%#v", result.Truncated, len(result.Output), result.Meta)
}
})
t.Run("raised call budget is not replaced by generic ceiling", func(t *testing.T) {
args := map[string]any{"cmd": "make build", "max_output_tokens": 20_000}
output := strings.Repeat("x", 70_000)
result := registry.RebudgetAfterHook(ExecCommandToolName, args, Result{Status: StatusOK, Output: output})
if result.Truncated || result.Output != output {
t.Fatalf("post-hook output was incorrectly limited by generic ceiling: truncated=%t bytes=%d", result.Truncated, len(result.Output))
}
})
}
+2 -213
View File
@@ -4,7 +4,6 @@ import (
"fmt"
"strconv"
"strings"
"unicode/utf8"
)
const (
@@ -12,167 +11,6 @@ const (
searchOutputBudgetBytes = 64 * 1024
)
// outputCategory selects the deterministic retention policy used for a tool
// result. It is deliberately internal: categories are an implementation detail
// of the tools boundary, not part of Zero's public tool protocol.
type outputCategory string
const (
outputCategoryDefault outputCategory = "default"
outputCategoryFile outputCategory = "file"
outputCategorySearch outputCategory = "search"
outputCategoryTest outputCategory = "test"
outputCategoryProcess outputCategory = "process"
outputCategoryDiff outputCategory = "diff"
outputCategoryWorker outputCategory = "worker"
)
// outputBudget combines a provider-neutral estimated-token target with the
// existing byte ceiling. The byte ceiling is always authoritative: semantic
// retention may use fewer bytes, but can never use more.
type outputBudget struct {
maxEstimatedTokens int
hardMaxBytes int
}
// budgetedOutput describes the text retained by a semantic output policy. The
// original size is the output received by this layer; it does not claim to be
// every byte an underlying subprocess may have produced before its own capture
// limits were applied.
type budgetedOutput struct {
text string
originalBytes int
retainedBytes int
estimatedOriginalTokens int
estimatedRetainedTokens int
truncated bool
category outputCategory
reason string
spillPath string
}
// outputPolicyProvider optionally assigns a semantic category to a tool call.
// Tools that do not implement it use outputCategoryDefault.
type outputPolicyProvider interface {
outputCategory(args map[string]any) outputCategory
}
// estimateOutputTokens is a deterministic, provider-neutral estimate used only
// for output budgeting. It is not exact provider tokenization. ASCII non-space
// text uses the repository's established four-bytes-per-token approximation;
// each byte of non-ASCII UTF-8 is counted as a token, intentionally
// overestimating multilingual text and emoji rather than letting them bypass a
// budget. Existing hard byte ceilings remain the final safety limit.
func estimateOutputTokens(value string) int {
asciiNonSpace, nonASCIIBytes := outputTokenComponents(value)
return (asciiNonSpace+3)/4 + nonASCIIBytes
}
func outputTokenComponents(value string) (asciiNonSpace int, nonASCIIBytes int) {
for index := 0; index < len(value); {
r, size := utf8.DecodeRuneInString(value[index:])
if r == utf8.RuneError && size == 1 {
// Invalid input is charged conservatively one token per byte. Budget
// slicing itself remains rune-safe for valid UTF-8 tool output.
nonASCIIBytes++
index++
continue
}
if r < utf8.RuneSelf {
switch r {
case ' ', '\t', '\n', '\r', '\f', '\v':
default:
asciiNonSpace++
}
} else {
nonASCIIBytes += size
}
index += size
}
return asciiNonSpace, nonASCIIBytes
}
const defaultSemanticTruncationNotice = "\n[zero] output truncated\n"
// budgetDefaultOutput applies the safe fallback policy: retain a rune-safe head
// and tail around a stable truncation notice. Small output is returned
// byte-identically. Spill creation is intentionally handled by the shared
// boundary integration so every semantic policy uses the existing spill path.
func budgetDefaultOutput(output string, budget outputBudget) budgetedOutput {
result := budgetedOutput{
text: output,
originalBytes: len(output),
retainedBytes: len(output),
estimatedOriginalTokens: estimateOutputTokens(output),
estimatedRetainedTokens: estimateOutputTokens(output),
category: outputCategoryDefault,
}
if fitsOutputBudget(output, budget) {
return result
}
reason := outputBudgetReason(result.originalBytes, result.estimatedOriginalTokens, budget)
maxContentBytes := len(output)
if budget.hardMaxBytes > 0 {
maxContentBytes = min(maxContentBytes, max(0, budget.hardMaxBytes-len(defaultSemanticTruncationNotice)))
}
// Find the largest deterministic head+tail window that satisfies both the
// estimate and the hard byte ceiling. fitsOutputBudget is monotonic as this
// window grows, so binary search avoids repeatedly trimming one rune at a time.
low, high := 0, maxContentBytes
best := ""
if fitsOutputBudget(defaultSemanticTruncationNotice, budget) {
best = defaultSemanticTruncationNotice
}
for low <= high {
window := low + (high-low)/2
candidate := defaultHeadTail(output, window)
if fitsOutputBudget(candidate, budget) {
best = candidate
low = window + 1
} else {
high = window - 1
}
}
result.text = best
result.retainedBytes = len(best)
result.estimatedRetainedTokens = estimateOutputTokens(best)
result.truncated = true
result.reason = reason
return result
}
func defaultHeadTail(output string, contentBytes int) string {
if contentBytes <= 0 {
return defaultSemanticTruncationNotice
}
headBytes := contentBytes / 2
tailBytes := contentBytes - headBytes
return utf8Prefix(output, headBytes) + defaultSemanticTruncationNotice + utf8Suffix(output, tailBytes)
}
func fitsOutputBudget(output string, budget outputBudget) bool {
if budget.hardMaxBytes > 0 && len(output) > budget.hardMaxBytes {
return false
}
return budget.maxEstimatedTokens <= 0 || estimateOutputTokens(output) <= budget.maxEstimatedTokens
}
func outputBudgetReason(originalBytes int, estimatedTokens int, budget outputBudget) string {
overTokens := budget.maxEstimatedTokens > 0 && estimatedTokens > budget.maxEstimatedTokens
overBytes := budget.hardMaxBytes > 0 && originalBytes > budget.hardMaxBytes
switch {
case overTokens && overBytes:
return "token_and_byte_budget"
case overTokens:
return "estimated_token_budget"
default:
return "hard_byte_ceiling"
}
}
type outputBudgetResult struct {
Output string
Truncated bool
@@ -218,7 +56,6 @@ func estimatedTokensFromBytes(bytes int) int {
type outputBudgetBuilder struct {
builder strings.Builder
capture *boundedBuffer
rawBytes int
maxBytes int
hint string
@@ -228,60 +65,17 @@ func newOutputBudgetBuilder(maxBytes int, hint string) *outputBudgetBuilder {
return &outputBudgetBuilder{maxBytes: maxBytes, hint: hint}
}
// newHeadTailOutputBudgetBuilder bounds capture while retaining both ends of
// an input for a later semantic policy. It is used for registry-driven file
// reads; direct Tool.Run calls keep the legacy prefix-only builder above.
func newHeadTailOutputBudgetBuilder(maxBytes int, hint string) *outputBudgetBuilder {
headBytes := maxBytes / 2
return &outputBudgetBuilder{
capture: newBoundedBuffer(headBytes, maxBytes-headBytes),
maxBytes: maxBytes,
hint: hint,
}
}
func (builder *outputBudgetBuilder) WriteString(value string) {
builder.rawBytes += len(value)
if builder.capture != nil {
_, _ = builder.capture.Write([]byte(value))
return
}
if builder.maxBytes <= 0 {
builder.builder.WriteString(value)
return
}
if builder.builder.Len() >= builder.maxBytes {
if builder.maxBytes <= 0 || builder.builder.Len() >= builder.maxBytes {
return
}
remaining := builder.maxBytes - builder.builder.Len()
builder.builder.WriteString(utf8Prefix(value, remaining))
}
// applyLegacyByteBudgetToResult preserves direct Tool.Run behavior for callers
// that intentionally bypass Registry.RunWithOptions. Agent/MCP execution uses
// the registry's post-redaction semantic boundary instead.
func applyLegacyByteBudgetToResult(result Result, maxBytes int, hint string) Result {
budgeted := applyOutputBudget(result.Output, maxBytes, hint)
result.Output = budgeted.Output
result.Truncated = result.Truncated || budgeted.Truncated
if result.Meta == nil {
result.Meta = map[string]string{}
}
for key, value := range outputBudgetMeta(budgeted) {
result.Meta[key] = value
}
if budgeted.Truncated {
result.Meta["truncated"] = "true"
result.Meta["truncation_reason"] = "byte_budget"
}
return result
}
func (builder *outputBudgetBuilder) Result() outputBudgetResult {
output := builder.builder.String()
if builder.capture != nil {
output = builder.capture.retained()
}
result := outputBudgetResult{
Output: output,
RawBytes: builder.rawBytes,
@@ -296,12 +90,7 @@ func (builder *outputBudgetBuilder) Result() outputBudgetResult {
if budget < 0 {
budget = 0
}
if builder.capture != nil {
headBytes := budget / 2
result.Output = utf8Prefix(output, headBytes) + marker + utf8Suffix(output, budget-headBytes)
} else {
result.Output = utf8Prefix(output, budget) + marker
}
result.Output = utf8Prefix(output, budget) + marker
result.Truncated = true
result.EmittedBytes = len(result.Output)
return result
-90
View File
@@ -1,90 +0,0 @@
package tools
import (
"strings"
"testing"
"unicode/utf8"
)
func TestEstimateOutputTokensASCIIAndUnicode(t *testing.T) {
if got := estimateOutputTokens("abcd efgh"); got != 2 {
t.Fatalf("ASCII estimate = %d, want 2", got)
}
unicodeText := "🙂界"
if got := estimateOutputTokens(unicodeText); got < len([]byte(unicodeText)) {
t.Fatalf("Unicode estimate = %d, want conservative estimate >= %d", got, len([]byte(unicodeText)))
}
if first, second := estimateOutputTokens(unicodeText), estimateOutputTokens(unicodeText); first != second {
t.Fatalf("estimator is not deterministic: %d != %d", first, second)
}
if input := strings.Repeat("\u0080", 4); estimateOutputTokens(input) < len([]byte(input)) {
t.Fatalf("U+0080 estimate = %d, want conservative estimate >= %d", estimateOutputTokens(input), len([]byte(input)))
}
}
func TestBudgetDefaultOutputLeavesSmallOutputByteIdentical(t *testing.T) {
input := "small output\nwith spacing\n"
got := budgetDefaultOutput(input, outputBudget{maxEstimatedTokens: 100, hardMaxBytes: 1024})
if got.text != input {
t.Fatalf("small output changed: got %q want %q", got.text, input)
}
if got.truncated || got.originalBytes != len(input) || got.retainedBytes != len(input) {
t.Fatalf("unexpected small-output metadata: %#v", got)
}
}
func TestBudgetDefaultOutputKeepsUTF8HeadTailWithinHardCeiling(t *testing.T) {
input := "HEAD🙂\n" + strings.Repeat("界", 200) + "\nTAIL🙂"
const hardMax = 120
got := budgetDefaultOutput(input, outputBudget{maxEstimatedTokens: 10_000, hardMaxBytes: hardMax})
if !got.truncated {
t.Fatal("large output was not truncated")
}
if len(got.text) > hardMax {
t.Fatalf("retained %d bytes, hard ceiling %d", len(got.text), hardMax)
}
if !utf8.ValidString(got.text) {
t.Fatalf("budgeted output is invalid UTF-8: %q", got.text)
}
for _, want := range []string{"HEAD", "TAIL", "output truncated"} {
if !strings.Contains(got.text, want) {
t.Fatalf("budgeted output missing %q: %q", want, got.text)
}
}
}
func TestBudgetDefaultOutputHonorsEstimatedTokenBudget(t *testing.T) {
input := strings.Repeat("🙂", 100)
got := budgetDefaultOutput(input, outputBudget{maxEstimatedTokens: 60, hardMaxBytes: 1024})
if !got.truncated || got.reason != "estimated_token_budget" {
t.Fatalf("unexpected token-budget result: %#v", got)
}
if got.estimatedRetainedTokens > 60 {
t.Fatalf("retained estimate = %d, budget 60", got.estimatedRetainedTokens)
}
}
func TestBudgetDefaultOutputDeterministic(t *testing.T) {
input := "head\n" + strings.Repeat("same line\n", 1000) + "tail\n"
budget := outputBudget{maxEstimatedTokens: 80, hardMaxBytes: 512}
want := budgetDefaultOutput(input, budget)
for iteration := 0; iteration < 20; iteration++ {
got := budgetDefaultOutput(input, budget)
if got != want {
t.Fatalf("iteration %d differs:\n got %#v\nwant %#v", iteration, got, want)
}
}
}
func TestBudgetDefaultOutputTinyBudgetsNeverExceedTheirCeilings(t *testing.T) {
input := strings.Repeat("oversized output ", 100)
for _, budget := range []outputBudget{
{hardMaxBytes: len(defaultSemanticTruncationNotice) - 1},
{maxEstimatedTokens: estimateOutputTokens(defaultSemanticTruncationNotice) - 1},
} {
got := budgetDefaultOutput(input, budget)
if !got.truncated || !fitsOutputBudget(got.text, budget) {
t.Fatalf("tiny budget result does not fit: budget=%#v result=%#v", budget, got)
}
}
}
+15 -17
View File
@@ -25,19 +25,23 @@ const defaultOutputCeilingTokens = 16_000
// disables the ceiling entirely; unset or unparsable keeps the default.
const outputCeilingEnv = "ZERO_TOOL_OUTPUT_CEILING_TOKENS"
// selfBudgeting marks a tool with a deliberate capture-aware output budget —
// possibly model-raisable (exec_command). The registry applies semantic
// retention within that explicit budget rather than replacing it with the
// default ceiling. The method is unexported so an MCP-served tool cannot opt
// into this path.
// selfBudgeting marks a tool that enforces its own deliberate output budget —
// possibly model-raisable (exec_command) — which the registry ceiling must not
// second-guess. The method is unexported on purpose: only tools in this
// package can opt out, so an MCP-served tool can never exempt itself.
type selfBudgeting interface{ managesOutputBudget() }
// The list is kept in one place. Shell/process tools retain their established
// capture-aware budgets: bash (per-stream bounded capture) and exec_command
// (model-raisable bounded session output). File/search tools use the standard
// shared post-redaction boundary.
func (bashTool) managesOutputBudget() {}
func (execCommandTool) managesOutputBudget() {}
// The exemption list, kept in one place. Each of these applies its own budget
// before returning: bash (bashOutputBudgetBytes per stream + spill),
// exec_command (model-raisable token budget + spill), read tools (128 KiB),
// search tools (64 KiB).
func (bashTool) managesOutputBudget() {}
func (execCommandTool) managesOutputBudget() {}
func (readFileTool) managesOutputBudget() {}
func (readMinifiedFileTool) managesOutputBudget() {}
func (grepTool) managesOutputBudget() {}
func (globTool) managesOutputBudget() {}
func (listDirectoryTool) managesOutputBudget() {}
func resolveOutputCeilingTokens() int {
raw := strings.TrimSpace(os.Getenv(outputCeilingEnv))
@@ -57,12 +61,6 @@ func resolveOutputCeilingTokens() int {
// and the spill file agree on what was hidden.
func enforceOutputCeiling(toolName string, result Result) Result {
ceiling := resolveOutputCeilingTokens()
switch toolName {
case "read_file", "read_minified_file":
ceiling = readOutputBudgetBytes / 4
case "grep", "glob", "list_directory":
ceiling = searchOutputBudgetBytes / 4
}
if ceiling <= 0 {
return result
}
+7 -7
View File
@@ -55,10 +55,10 @@ func TestOutputCeilingCapsUnbudgetedTool(t *testing.T) {
if result.Meta["raw_bytes"] != strconv.Itoa(len(big)) {
t.Fatalf("raw_bytes = %s, want %d", result.Meta["raw_bytes"], len(big))
}
if !strings.Contains(result.Output, "full output received by budgeting layer saved to ") {
if !strings.Contains(result.Output, "full output saved to ") {
t.Fatalf("ceiling truncation must include a spill hint: %q", result.Output[:200])
}
start := strings.Index(result.Output, "full output received by budgeting layer saved to ") + len("full output received by budgeting layer saved to ")
start := strings.Index(result.Output, "full output saved to ") + len("full output saved to ")
end := strings.Index(result.Output[start:], " (grep")
content, err := os.ReadFile(result.Output[start : start+end])
if err != nil {
@@ -123,6 +123,11 @@ func TestSelfBudgetingExemptionList(t *testing.T) {
exempt := []Tool{
NewBashTool(dir),
NewExecCommandTool(dir, newExecSessionManager()),
NewReadFileTool(dir),
NewReadMinifiedFileTool(dir),
NewGrepTool(dir),
NewGlobTool(dir),
NewListDirectoryTool(dir),
}
for _, tool := range exempt {
if _, ok := tool.(selfBudgeting); !ok {
@@ -132,9 +137,4 @@ func TestSelfBudgetingExemptionList(t *testing.T) {
if _, ok := NewWebFetchTool().(selfBudgeting); ok {
t.Error("web_fetch must NOT be exempt — the ceiling is its only budget")
}
for _, tool := range []Tool{NewReadFileTool(dir), NewReadMinifiedFileTool(dir), NewGrepTool(dir), NewGlobTool(dir), NewListDirectoryTool(dir)} {
if _, ok := tool.(selfBudgeting); ok {
t.Errorf("%s must use the shared post-redaction budget boundary", tool.Name())
}
}
}
Loaded 100 of 141 files, more files were not shown because too many files have changed in this diff. Show more