Files
zero/internal/tools/spill.go
Kevin Codex 95ccd5bc32 perf: universal tool-output ceiling with spill + async post-edit diagnostics (#518)
* feat(tools): universal token-based output ceiling with spill-to-disk

Cap every tool result at the registry boundary (16k estimated tokens,
ZERO_TOOL_OUTPUT_CEILING_TOKENS-tunable, 0 disables) unless the tool
manages its own deliberate budget. Head+tail truncation, full output
spilled to the existing redacted spill dir and re-readable via
grep/read_file, so nothing is lost — only deferred.

This closes the three unbounded paths a single call could flood the
context window through (and then re-bill every following turn):
- web_fetch: up to 256KiB default / 2MiB max of non-HTML body
- skill: full SKILL.md returned uncapped
- MCP-served tools: no budget at all

Self-budgeting tools opt out via an unexported marker interface (an MCP
server can never exempt itself): bash, exec_command (model-raisable),
read_file, read_minified_file, grep, glob, list_directory. Browser
snapshots deliberately stay under the net.

bash itself is tightened to match the unified token model: emit budget
96KiB -> 32KiB per stream (~24k -> ~8k tokens worst case per stream),
while capture retention stays at 96KiB head+tail so the new spill file
covers 6x more of the output than the transcript shows.

* perf(agent): collect post-edit LSP diagnostics off the tool-call path

Inline post-edit diagnostics blocked every edit_file/write_file result on
the language server: a >=300ms quiet-period debounce that reset on each
publish, seconds on the first edit per language, capped at 10s — the
largest recurring latency tax in an edit-heavy session.

Diagnostics are now collected in the background: a file is enqueued the
moment a mutating tool finishes, a single worker checks it while the rest
of the batch executes, and the loop drains completed results just before
building the NEXT provider request, appending errors as a user nudge.
The model always issues another request after an edit turn (to read its
tool results), so it still sees introduced errors at the same decision
point — no tool call ever stalls on the language server.

A file re-edited before its check runs is checked once in its final
state (the checker re-reads the file); re-edited after, it is re-queued
and the newer result wins. A drain that finds the worker still busy
gives up after 3s and delivers on a later turn — results stay accurate
because any further edit re-enqueues the file.

The tools-side inline mechanism (tools.RunOptions.Diagnostics) is kept
for direct API callers; the loop just no longer wires it.

* fix: address review — max-turns diagnostics drain, capture-gap marker, test hardening

- Drain background diagnostics before the max-turns final-answer request
  so an error introduced by the LAST turn's edit is reported in the
  summary instead of silently dropped (+ regression test).
- spillBashStreams marks the head/tail junction with a capture-gap
  marker when boundedBuffer dropped the middle of a stream, so a spilled
  log never reads as contiguous when it is not; junction position is
  asserted in the test.
- Tests override TMP/TEMP alongside TMPDIR (os.TempDir reads TMP/TEMP on
  Windows) via a shared setTestTempDir helper.
- TestBudgetBashCaptureReportsTrueTotal now asserts spill_path meta and
  spill file content; exemption-list test gains read_minified_file.

* fix(test): use a real workspace root in async diagnostics tests

filepath.IsAbs("/ws") is false on Windows (no drive letter), so the
collector test's workspace-absolute assertion failed on the windows
smoke job. Root the fake workspace at t.TempDir() — absolute on every
platform — and assert against it.

* fix: pattern-scrub spill files and gate finalization on pending diagnostics

Review follow-ups (Vasanthdev2004):

- Spill files now pass the pattern-based secret scanner (AWS keys,
  GitHub/Slack/OpenAI tokens, PEM blocks, JWTs) in addition to the
  configured-key scrub. A bash spill runs before formatBashOutput's
  scan, so without this a spilled file held in cleartext exactly the
  credentials the transcript redacts. Applied centrally in
  spillTruncatedOutput, covering bash, exec_command, and ceiling spills.

- Finalization diagnostics gate: a no-tool-call final answer now drains
  pending post-edit diagnostics with the full inline-era budget (10s,
  vs the 3s per-turn wait) before the run is accepted; introduced
  errors append the nudge and give the model one more turn instead of
  being silently dropped when no later turn exists. The max-turns
  summary path uses the same finalization budget. Free for runs that
  never edited — an idle collector returns immediately.
2026-07-05 23:50:31 +08:00

146 lines
4.8 KiB
Go

package tools
import (
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/Gitlawb/zero/internal/redaction"
"github.com/Gitlawb/zero/internal/secrets"
)
// Spill-to-disk for truncated tool output. When a command produces more than
// its context budget, the tail was previously simply gone — the model's only
// recourse was re-running the (possibly expensive or non-idempotent) command
// with a bigger budget. Instead, the full output is written to a cache file
// whose path is included in the truncation notice, so the model can grep or
// read_file the remainder. Spilling is best-effort: on any error truncation
// behaves exactly as before, just without the file hint.
// spillRetention is how long spilled outputs are kept; files older than this
// are opportunistically removed whenever a new spill happens.
const spillRetention = 7 * 24 * time.Hour
// spillRootPath returns the spill directory path without creating or checking
// it. Used by the read-path resolver to recognize spill files as readable.
func spillRootPath() string {
name := "zero-tool-output"
if uid := os.Getuid(); uid >= 0 {
name = fmt.Sprintf("zero-tool-output-%d", uid)
}
return filepath.Join(os.TempDir(), name)
}
// resolveSpillReadPath reports whether requestedPath is a file inside the
// spill directory and, if so, returns its verified absolute path. Spill files
// are zero-created, per-uid owned, and already secret-redacted, so letting the
// scoped read tools open them is what makes the truncation notice's
// "read_file it" recovery actually work. Symlinks are resolved and the result
// must still be inside the spill dir, so a planted link cannot smuggle an
// out-of-scope file through this gate.
func resolveSpillReadPath(requestedPath string) (string, bool) {
if requestedPath == "" || !filepath.IsAbs(requestedPath) {
return "", false
}
root := spillRootPath()
cleaned := filepath.Clean(requestedPath)
if cleaned == root || !strings.HasPrefix(cleaned, root+string(filepath.Separator)) {
return "", false
}
resolved, err := filepath.EvalSymlinks(cleaned)
if err != nil {
return "", false
}
resolvedRoot, err := filepath.EvalSymlinks(root)
if err != nil {
return "", false
}
if !strings.HasPrefix(resolved, resolvedRoot+string(filepath.Separator)) {
return "", false
}
return resolved, true
}
// spillDir returns the per-user spill directory, creating it on first use.
// Hardened for shared temp dirs (Linux /tmp): the name carries the uid so
// users cannot collide, and a pre-existing path is only accepted when it is a
// real directory (not a symlink that would redirect spills) owned by the
// current user. Any doubt fails the spill — it is best-effort anyway.
func spillDir() (string, error) {
dir := spillRootPath()
if err := os.MkdirAll(dir, 0o700); err != nil {
return "", err
}
// MkdirAll follows symlinks and leaves an existing directory untouched, so
// verify what is actually at the path.
info, err := os.Lstat(dir)
if err != nil {
return "", err
}
if !info.IsDir() {
return "", fmt.Errorf("spill path %s is not a directory", dir)
}
if err := checkSpillDirOwner(info); err != nil {
return "", err
}
return dir, nil
}
// spillTruncatedOutput writes the full pre-truncation output to the spill
// directory and returns the file path, or "" when spilling fails. Output is
// scrubbed with the same configured-key redaction the registry applies at the
// tool boundary PLUS the pattern-based secret scanner (AWS keys, tokens, PEM
// blocks, JWTs) that bash applies to its model-visible output — a spill runs
// before that formatter, so without the scan here a spilled file would hold
// pattern-matched credentials in cleartext that the transcript hides.
func spillTruncatedOutput(toolName, output string) string {
dir, err := spillDir()
if err != nil {
return ""
}
sweepSpillDir(dir)
prefix := strings.Map(func(r rune) rune {
if r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' || r == '_' || r == '-' {
return r
}
return '-'
}, toolName)
file, err := os.CreateTemp(dir, prefix+"-*.txt")
if err != nil {
return ""
}
defer file.Close()
scrubbed := redaction.RedactString(output, redaction.Options{})
scrubbed, _ = secrets.Redact(scrubbed)
if _, err := file.WriteString(scrubbed); err != nil {
_ = os.Remove(file.Name())
return ""
}
return file.Name()
}
// sweepSpillDir removes spill files older than spillRetention. Best-effort:
// errors are ignored, the directory is small, and a sweep runs only when a new
// spill is about to happen anyway.
func sweepSpillDir(dir string) {
entries, err := os.ReadDir(dir)
if err != nil {
return
}
cutoff := time.Now().Add(-spillRetention)
for _, entry := range entries {
if entry.IsDir() {
continue
}
info, err := entry.Info()
if err != nil {
continue
}
if info.ModTime().Before(cutoff) {
_ = os.Remove(filepath.Join(dir, entry.Name()))
}
}
}