Files
zero/internal/tools/structured_patch.go
Kevin CodexandClaude Fable 5 2749447cdc fix(tools): make apply_patch tolerant and re-expose edit_file (#956)
* fix(tools): make apply_patch tolerant and re-expose edit_file

Head-to-head benchmarking against Grok Build on grok-4.6 (3 tasks × 3 runs)
showed Zero solving every task but spending 2.7× the tokens and 2.3× the
time on the multi-file feature task. Every one of the 20 apply_patch calls
in those runs failed, after which the model degraded to whole-file
write_file rewrites and read→write verification loops.

Root causes and fixes:

- The sandbox and the tool both rejected an absolute path that lies inside
  the workspace ("must stay inside the workspace"). Models routinely echo
  the absolute path read_file showed them. Absolute paths now flow through
  the normal workspace-scope validation (inside → allowed, outside → still
  denied); unified-diff headers are rewritten to a/ b/ relative form before
  git apply, and platform symlink prefixes (/var → /private/var) are
  normalised the same way the other file tools do.
- The structured parser demanded a byte-exact "*** Begin Patch" /
  "*** End Patch"; grok-4.6 writes "*** Begin Patch ***". Decorated marker
  spellings are accepted, and a unified range after "@@" ("-12,4 +12,6",
  optionally with a heading) no longer becomes a literal context search.
- Parse errors now carry a one-line format reminder so a failed call is
  recoverable in one retry instead of a format-guessing loop.
- edit_file (exact search/replace with uniqueness guard, CRLF and fuzzy
  fallbacks) is advertised to the model again alongside apply_patch, and
  the apply_patch description shows the structured format.
- Writers (edit_file, write_file, apply_patch) recorded a file's line total
  with lineCount, which is one higher than read_file's count for a file
  ending in a newline. RecordSeenRange resets every observation when the
  total changes, so a partial read_file after a successful edit silently
  discarded whole-file knowledge and the next edit was refused with
  "content that has not been read exactly in this session". All writers
  now record the same total read_file reports (trackedLineTotal).
- read_file's padded "  N | " line prefix was ~19% of every read (about
  1.5k tokens per 800-line file) and was re-sent on every later call. It is
  now the compact "N→" prefix models already know; nothing parsed the old
  form.
- The editing-discipline prompt steers: edit_file for a targeted change,
  apply_patch for multi-hunk/multi-file changes, write_file only for new
  files or full rewrites; a successful edit result is confirmation, so the
  file is not re-read to verify it.

Tests: regression tests for decorated markers, writer/reader line-total
agreement (edit → partial read → edit far away still succeeds), range hunk headers,
absolute in-workspace paths (structured + unified), outside-workspace
rejection, header-line-only relativisation, and tool visibility; the
prompt and tool-budget tests updated to the new contract.

Benchmark (grok-4.6, 3 tasks x 3 runs, harness in demo/bench-zero-vs-grok):
Zero 0.8.0 617 s / 1,527k tokens -> with this change 346 s / 493k tokens;
Grok Build 407 s / 713k. Every run of every configuration solved its task.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(tools): keep CRLF line endings when relativizing patch headers

relativizeUnifiedPatchPaths split on a normalized "\n" and rejoined with
"\n", so any CRLF patch that needed a header rewrite lost its line endings
everywhere; git apply could then fail to match a CRLF file. Split on "\n"
only and carry each line's trailing "\r" through, including on rewritten
headers.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(sandbox): share the structured-patch classifier with apply_patch

Review follow-ups for #956:

- The tool accepted decorated structured markers ("***Begin Patch") that
  the sandbox's applyPatchPaths did not recognise, so for that spelling the
  boundary scanned the patch as a unified diff, extracted no targets and
  validated nothing. The marker classifier now lives in the sandbox package
  (StructuredPatchMarker / IsStructuredPatch) and the tool delegates to it,
  so both sides accept exactly the same set; tests cover inside, traversal
  and outside targets for the no-space spelling at the boundary.
- Document the containment model at the apply site: structured patches are
  applied through an opened os.Root (descriptor-relative, no-follow);
  unified diffs keep the pre-existing git-apply pathname flow, which the
  absolute-path rewrite does not widen (they become the same root-relative
  form relative paths always used).
- OpenAI addendum: the one-file-per-call rule names read_file, edit_file and
  write_file explicitly and allows a single multi-file apply_patch.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* test(sandbox): compare slash-normalised patch paths so the check passes on Windows

structuredPatchHeaderPaths normalises separators to "/", so the absolute
outside-workspace path in the boundary test must be compared in its slash
form; on Windows the backslash original never matched and the Smoke job
failed.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* test(sandbox): validate the parsed patch path at the scope boundary

For each accepted marker spelling, pass the exact path applyPatchRequestPaths
produced to scope.validate: an outside path must be denied and an inside
path accepted, so a separator-handling regression between the parser and the
scope cannot slip past the test.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(tools): apply unified diffs in-process through the workspace root

Review follow-up for #956. Unified diffs were validated by pathname and then
handed to git apply, which reopened the targets by name — a check-to-use
window a symlink swap could exploit — and the absolute-path rewrite added in
this PR normalised `diff --git` headers with strings.Fields, which splits a
git C-quoted path containing whitespace.

Both go away by not shelling out at all: parseUnifiedPatch translates a
unified diff into the same operations a structured patch produces, and both
formats are applied by applyPatchOperations through os.OpenRoot — every
stat, read, create and write is descriptor-relative and refuses to follow a
link out of the root. git is no longer required by apply_patch.

Unified-diff support: modifications, creations (--- /dev/null), deletions
(+++ /dev/null), multiple hunks per file, git "rename from/to" (a move,
hunks optional) and "copy from/to" (a new copy operation whose destination
inherits only the tracker credit the source had), "\ No newline at end of
file" on either side, a/ b/ prefixes, C-quoted paths, CRLF patches, and
git's diff/index/mode lines. Hunks are placed at their "@@ -a,b" range when
the expected lines match there and located by context otherwise (so a
stale range still applies and duplicate blocks are disambiguated by the
range). Binary patches are rejected.

Removed: the git apply flow, relativizeUnifiedPatchPaths,
recheckPatchWriteTargets, missingPatchTargets, completeCreatedPatchTargets
and recordCreatedPatchTargets. validatePatchPaths/changedFilesFromPatch stay
for mutation-target listing.

Tests: the existing unified-diff suite now runs through the in-process
applier unchanged; new coverage for quoted whitespace paths (inside applies,
outside is refused untouched), a workspace path swapped for a symlink that
escapes the root (both formats refused, outside file unmodified), create +
delete + no-newline markers, range hint vs context fallback, rename with a
hunk, and binary/empty rejection.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(tools): harden unified-diff parsing and cover the copy operation

Review follow-ups for #956 (in-process unified applier):

- A hunk that ends before its declared line counts is reported as
  malformed instead of silently absorbing the next file's ---/+++ header
  pair (which then failed later against the wrong file). Both shapes are
  caught: leftover counts at a header or at end of input, and a --- line
  directly followed by +++ inside a hunk. The element after a patch's final
  newline is no longer counted as an empty context line, so a truncated
  hunk cannot look complete.
- "\ No newline at end of file" may only follow a file's final hunk; a
  marker before another hunk is rejected rather than leaking into that
  hunk's result.
- Zero-context insertions (git diff -U0) are documented as
  position-trusting by design, matching git apply without fuzz; hunks with
  context are verified at the range before it is used.

Tests: copy from/to (source kept, destination created with the hunk
applied and tracked with the source's whole-file state; existing
destination refused untouched; copy onto itself refused), over-declared and
truncated hunk counts, and the no-newline marker before vs after the final
hunk.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(tools): require an @@ line before treating ---/+++ inside a hunk as a file boundary

A removed "-- x" line directly followed by an added "++ y" line is valid
hunk content that also matches the ---/+++ header shape; the boundary check
now additionally requires the following line to be a hunk header, and a
regression test keeps such a pair intact. End-of-input strictness for a hunk
cut short is documented as deliberate: applying a truncated hunk would make
a partial change (removals without their replacements).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* refactor(tools): share one line-count implementation between reader and writer

sourceLineCount and trackedLineTotal computed the same count; keeping two
copies invites exactly the reader/writer drift this PR fixed. The reader now
delegates to trackedLineTotal, which carries the tracker contract.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(tools): verify a unified deletion's expected content before removing the file

A unified diff's "+++ /dev/null" hunks state the content being removed, but
the in-process applier dropped them and deleted unconditionally, so a stale
deletion removed a file whose current content the patch never described.
Unified deletions now keep their hunks, require at least one, and run them
through the same verifier as updates: the hunks must match the current file
and leave nothing behind, otherwise the deletion is refused and the file is
untouched. A structured "*** Delete File" keeps its unconditional meaning.

Hardening from the same review: every non-create change re-reads its source
through the opened root immediately before commit and refuses to proceed if
the bytes differ from what was planned.

Tests: matching deletion, stale deletion (file byte-for-byte unchanged),
partial deletion, header-only deletion, multi-hunk deletion with one stale
hunk (nothing removed) and all-good hunks, structured delete unchanged, and
a source changed between planning and commit for update and delete.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* test(tools): a move with a missing source is refused before publishing

The pre-commit recheck now refuses a move whose source vanished after
planning instead of publishing the destination and then failing; update the
test to the safer contract (error names the recheck, nothing is created).

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(tools): byte-exact unified deletions and git's header-only empty-file forms

Review follow-ups for #956:

- A unified deletion is verified byte for byte: the removed lines (with the
  removed side's "\ No newline at end of file" honoured) must equal the
  current file exactly, with only the file's own line-ending style
  normalised. None of the whitespace tolerance used to locate update hunks
  applies, and a whitespace-only remainder no longer passes; a deletion is
  not recoverable, so it fails closed.
- git describes an empty file's deletion or creation with header lines only
  ("diff --git", "deleted file mode"/"new file mode", "index") and no
  hunks. Both forms are recognised: the deletion is applied only when the
  current file is empty, the creation writes an empty file, and they compose
  with hunk-carrying files in the same patch.
- The pre-commit recheck test also covers the copy operation and asserts no
  destination is created on refusal.

Tests: whitespace/indentation/blank-line/space-only/missing-newline
mismatches all refused with the file unchanged, matching no-newline marker
and CRLF file accepted, header-only deletion of a non-empty file refused,
multi-file patch with header-only delete + create + a normal hunk.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* test(tools): remove the move source after planning via a deterministic hook

The refused-move test now creates source.txt, plans the move, and removes
the file through a test-only pre-commit hook (structuredPatchBeforeCommit)
so it exercises the post-planning disappearance path rather than a source
that was never there. Assertions unchanged: refused before publishing, no
destination created.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* test(tui): read_file fixtures use the compact N→ prefix

Three TUI fixtures still built fake read_file output in the legacy
"  N | text" shape; they pass either way but pinned an outdated contract.
They now mirror what appendReadFileRange emits, and the shape comment and
the expanded-card assertion are updated.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

* fix(tools): name the committed files when a patch is partially applied

Review follow-up for #956. When a later change in a multi-file patch fails
(for example the pre-commit recheck refuses a file that changed after
planning), the error now lists exactly which files were already committed
and states that the rest are unchanged, instead of a generic "partially
applied" warning. The model can then re-read only what actually changed.

Test: a three-file structured patch whose second file is altered after
planning fails with "already committed: first.txt", first.txt holds the
patched content, and the other two files are untouched.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-25 20:23:48 +08:00

960 lines
33 KiB
Go

package tools
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/Gitlawb/zero/internal/pathjail"
"github.com/Gitlawb/zero/internal/sandbox"
)
const (
structuredPatchBegin = "*** Begin Patch"
structuredPatchEnd = "*** End Patch"
structuredAddFile = "*** Add File: "
structuredDeleteFile = "*** Delete File: "
structuredUpdateFile = "*** Update File: "
structuredMoveTo = "*** Move to: "
structuredEndOfFile = "*** End of File"
structuredHunkContext = "@@ "
)
type structuredPatchKind uint8
const (
structuredPatchAdd structuredPatchKind = iota
structuredPatchDelete
structuredPatchUpdate
// structuredPatchCopy creates movePath from path's content (plus any
// hunks) and keeps path; produced by a unified diff's "copy from/to".
structuredPatchCopy
)
type structuredPatchOperation struct {
kind structuredPatchKind
path string
movePath string
contents string
chunks []structuredPatchChunk
line int
// eofNewline lets a unified diff's "\ No newline at end of file" marker
// force the trailing-newline state of the result; structured patches keep
// the file's existing state.
eofNewline eofNewlineMode
// verifyDelete marks a deletion that carries the expected old content
// (a unified diff's "+++ /dev/null" hunks, or git's header-only form for
// an empty file): the removed lines must equal the current file byte for
// byte, otherwise the deletion is stale and refused. A structured
// "*** Delete File" means "delete this path" and has no chunks.
verifyDelete bool
// oldNoNewline records a unified diff's "\ No newline at end of file"
// on the removed side, i.e. the old content did not end with a newline.
oldNoNewline bool
}
type eofNewlineMode uint8
const (
eofNewlineKeep eofNewlineMode = iota
eofNewlinePresent
eofNewlineAbsent
)
type structuredPatchChunk struct {
context string
hasContext bool
old []string
new []string
newSourceOffsets []int
endOfFile bool
// hint is the 0-based line the hunk is expected at (from a unified diff's
// "@@ -a,b" range). When the expected lines match there it is used as-is;
// otherwise the hunk is located by content like a structured hunk.
hint int
hasHint bool
}
type structuredPatchTarget struct {
requested string
absolute string
relative string
}
type structuredPatchChange struct {
kind structuredPatchKind
from structuredPatchTarget
to structuredPatchTarget
before string
after string
mode os.FileMode
}
// unifiedHunkRangePattern recognises a unified-diff range header ("-12,4 +12,6",
// optionally followed by "@@ heading") written inside a structured hunk marker.
// The line numbers carry no information for the context-anchored grammar, so
// the range is dropped and only an explicit heading is kept as the anchor.
var unifiedHunkRangePattern = regexp.MustCompile(`^-\d+(?:,\d+)?\s+\+\d+(?:,\d+)?\s*(?:@@\s*(.*))?$`)
const structuredPatchFormatHint = ` (format: "*** Begin Patch", then "*** Update File: path" / "*** Add File: path" / "*** Delete File: path" sections whose hunks start with "@@" and use " " context, "-" removed and "+" added lines, then "*** End Patch")`
// structuredPatchMarker classifies a line as the "begin" or "end" marker of a
// structured patch, or "" when it is neither. The classifier is owned by the
// sandbox package so the boundary check and the tool accept exactly the same
// spellings; a strict byte-equal match here once made every patch from some
// models fail on line 1 and pushed them into whole-file rewrites.
func structuredPatchMarker(line string) string {
return sandbox.StructuredPatchMarker(line)
}
// structuredHunkAnchor normalises the text after a hunk's "@@ " marker. A
// unified-diff range is dropped (its optional heading survives as the anchor);
// anything else is used verbatim.
func structuredHunkAnchor(context string) string {
context = strings.TrimSpace(context)
if match := unifiedHunkRangePattern.FindStringSubmatch(context); match != nil {
return strings.TrimSpace(match[1])
}
return context
}
func isStructuredPatch(patch string) bool {
return sandbox.IsStructuredPatch(patch)
}
func (tool applyPatchTool) runStructuredPatch(applyRoot, relativeRoot, patch string, options RunOptions) Result {
operations, err := parseStructuredPatch(patch)
if err != nil {
return errorResult("Error applying patch: " + err.Error())
}
return applyPatchOperations(applyRoot, relativeRoot, operations, options)
}
// applyPatchOperations applies parsed operations (from either patch format)
// through an opened workspace root: every stat, read, create and write is
// descriptor-relative and refuses to follow a link out of the root, so there
// is no pathname check-to-use window between validation and write.
func applyPatchOperations(applyRoot, relativeRoot string, operations []structuredPatchOperation, options RunOptions) Result {
workspace, err := os.OpenRoot(applyRoot)
if err != nil {
return errorResult("Error applying patch: " + err.Error())
}
defer workspace.Close()
changes, err := planStructuredPatch(workspace, operations, options.FileTracker)
if err != nil {
return errorResult("Error applying patch: " + err.Error())
}
wholeBefore := make(map[string]bool, len(changes))
if options.FileTracker != nil {
for _, change := range changes {
if change.kind == structuredPatchUpdate || change.kind == structuredPatchCopy {
wholeBefore[change.from.absolute] = options.FileTracker.SeenWhole(change.from.absolute)
}
}
}
if err := applyStructuredPatchChanges(workspace, changes, options.FileTracker); err != nil {
return errorResult("Error applying patch: " + err.Error())
}
for _, change := range changes {
switch change.kind {
case structuredPatchDelete:
options.FileTracker.Forget(change.from.absolute)
case structuredPatchAdd:
recordStructuredPatchFile(options.FileTracker, change.to.absolute, true, true)
case structuredPatchUpdate:
wasWhole := wholeBefore[change.from.absolute]
options.FileTracker.Forget(change.from.absolute)
recordStructuredPatchFile(options.FileTracker, change.to.absolute, false, wasWhole)
case structuredPatchCopy:
// The source is untouched; the destination inherits only what the
// model had actually seen of the source.
recordStructuredPatchFile(options.FileTracker, change.to.absolute, true, wholeBefore[change.from.absolute])
}
}
summary := "Patch applied successfully."
if relativeRoot != "." {
summary = "Patch applied successfully in " + relativeRoot + "."
}
result := okResult(summary)
result.ChangedFiles = changedFilesFromStructuredPatch(relativeRoot, changes)
result.Display = Display{Summary: summary, Kind: "diff", Preview: structuredPatchPreview(changes)}
return result
}
func parseStructuredPatch(patch string) ([]structuredPatchOperation, error) {
normalized := strings.TrimSpace(strings.TrimPrefix(strings.ReplaceAll(patch, "\r\n", "\n"), "\ufeff"))
if normalized == "" {
return nil, fmt.Errorf("structured patch is empty")
}
lines := strings.Split(normalized, "\n")
if structuredPatchMarker(lines[0]) != "begin" {
return nil, fmt.Errorf("the first line of a structured patch must be %q%s", structuredPatchBegin, structuredPatchFormatHint)
}
if structuredPatchMarker(lines[len(lines)-1]) != "end" {
return nil, fmt.Errorf("the last line of a structured patch must be %q%s", structuredPatchEnd, structuredPatchFormatHint)
}
var operations []structuredPatchOperation
for index := 1; index < len(lines)-1; {
header := strings.TrimSpace(lines[index])
lineNumber := index + 1
if header == "" {
index++
continue
}
switch {
case strings.HasPrefix(header, structuredAddFile):
path, err := structuredPatchPath(header, structuredAddFile, lineNumber)
if err != nil {
return nil, err
}
op := structuredPatchOperation{kind: structuredPatchAdd, path: path, line: lineNumber}
index++
var content []string
for index < len(lines)-1 && !isStructuredPatchHeader(lines[index]) {
if !strings.HasPrefix(lines[index], "+") {
return nil, structuredPatchLineError(index+1, "added-file contents must start with '+'")
}
content = append(content, strings.TrimPrefix(lines[index], "+"))
index++
}
if len(content) > 0 {
op.contents = strings.Join(content, "\n") + "\n"
}
operations = append(operations, op)
case strings.HasPrefix(header, structuredDeleteFile):
path, err := structuredPatchPath(header, structuredDeleteFile, lineNumber)
if err != nil {
return nil, err
}
operations = append(operations, structuredPatchOperation{kind: structuredPatchDelete, path: path, line: lineNumber})
index++
case strings.HasPrefix(header, structuredUpdateFile):
path, err := structuredPatchPath(header, structuredUpdateFile, lineNumber)
if err != nil {
return nil, err
}
op := structuredPatchOperation{kind: structuredPatchUpdate, path: path, line: lineNumber}
index++
if index < len(lines)-1 && strings.HasPrefix(strings.TrimSpace(lines[index]), structuredMoveTo) {
op.movePath, err = structuredPatchPath(strings.TrimSpace(lines[index]), structuredMoveTo, index+1)
if err != nil {
return nil, err
}
index++
}
for index < len(lines)-1 && !isStructuredPatchHeader(lines[index]) {
line := lines[index]
trimmed := strings.TrimSpace(line)
if trimmed == structuredEndOfFile {
if len(op.chunks) == 0 || (!hasStructuredPatchContent(op.chunks[len(op.chunks)-1])) {
return nil, structuredPatchLineError(index+1, "*** End of File must follow an update hunk")
}
op.chunks[len(op.chunks)-1].endOfFile = true
index++
continue
}
if trimmed == "@@" || strings.HasPrefix(trimmed, structuredHunkContext) {
chunk := structuredPatchChunk{}
rest := ""
if trimmed != "@@" {
rest = strings.TrimPrefix(trimmed, structuredHunkContext)
}
if anchor := structuredHunkAnchor(rest); anchor != "" {
chunk.context = anchor
chunk.hasContext = true
}
op.chunks = append(op.chunks, chunk)
index++
continue
}
if len(op.chunks) == 0 {
op.chunks = append(op.chunks, structuredPatchChunk{})
}
chunk := &op.chunks[len(op.chunks)-1]
switch {
case line == "":
sourceOffset := len(chunk.old)
chunk.old = append(chunk.old, "")
chunk.new = append(chunk.new, "")
chunk.newSourceOffsets = append(chunk.newSourceOffsets, sourceOffset)
case strings.HasPrefix(line, " "):
content := strings.TrimPrefix(line, " ")
sourceOffset := len(chunk.old)
chunk.old = append(chunk.old, content)
chunk.new = append(chunk.new, content)
chunk.newSourceOffsets = append(chunk.newSourceOffsets, sourceOffset)
case strings.HasPrefix(line, "+"):
chunk.new = append(chunk.new, strings.TrimPrefix(line, "+"))
chunk.newSourceOffsets = append(chunk.newSourceOffsets, -1)
case strings.HasPrefix(line, "-"):
chunk.old = append(chunk.old, strings.TrimPrefix(line, "-"))
default:
return nil, structuredPatchLineError(index+1, "update lines must start with ' ', '+', or '-'")
}
index++
}
if len(op.chunks) == 0 || !allStructuredPatchChunksHaveContent(op.chunks) {
return nil, structuredPatchLineError(lineNumber, "update file hunk is empty")
}
operations = append(operations, op)
default:
return nil, structuredPatchLineError(lineNumber, "expected an Add File, Delete File, or Update File header")
}
}
if len(operations) == 0 {
return nil, fmt.Errorf("structured patch contains no file operations")
}
return operations, nil
}
func structuredPatchPath(header, prefix string, line int) (string, error) {
path := strings.TrimSpace(strings.TrimPrefix(header, prefix))
if path == "" {
return "", structuredPatchLineError(line, "file path is required")
}
return path, nil
}
func isStructuredPatchHeader(line string) bool {
return structuredPatchMarker(line) == "end" || strings.HasPrefix(line, structuredAddFile) || strings.HasPrefix(line, structuredDeleteFile) || strings.HasPrefix(line, structuredUpdateFile)
}
func structuredPatchLineError(line int, message string) error {
return fmt.Errorf("invalid structured patch at line %d: %s", line, message)
}
func hasStructuredPatchContent(chunk structuredPatchChunk) bool {
return len(chunk.old) > 0 || len(chunk.new) > 0
}
func allStructuredPatchChunksHaveContent(chunks []structuredPatchChunk) bool {
for _, chunk := range chunks {
if !hasStructuredPatchContent(chunk) {
return false
}
}
return true
}
func structuredPatchOperationPaths(operations []structuredPatchOperation) []string {
seen := make(map[string]bool)
var paths []string
for _, operation := range operations {
for _, path := range []string{operation.path, operation.movePath} {
if path == "" || seen[path] {
continue
}
seen[path] = true
paths = append(paths, path)
}
}
return paths
}
func planStructuredPatch(root *os.Root, operations []structuredPatchOperation, tracker *FileTracker) ([]structuredPatchChange, error) {
seen := make(map[string]struct{})
var changes []structuredPatchChange
for _, operation := range operations {
from, err := resolveStructuredPatchTarget(root.Name(), operation.path)
if err != nil {
return nil, err
}
to := from
if operation.movePath != "" {
to, err = resolveStructuredPatchTarget(root.Name(), operation.movePath)
if err != nil {
return nil, err
}
}
targets := []structuredPatchTarget{from}
if to.absolute != from.absolute {
targets = append(targets, to)
}
for _, target := range targets {
if _, exists := seen[target.absolute]; exists {
return nil, fmt.Errorf("structured patch changes %q more than once", target.relative)
}
seen[target.absolute] = struct{}{}
}
change := structuredPatchChange{kind: operation.kind, from: from, to: to, mode: 0o644}
switch operation.kind {
case structuredPatchAdd:
if _, err := root.Lstat(to.relative); err == nil {
return nil, fmt.Errorf("cannot add %s because it already exists", to.relative)
} else if !os.IsNotExist(err) {
return nil, err
}
change.after = operation.contents
case structuredPatchDelete, structuredPatchUpdate, structuredPatchCopy:
info, err := root.Stat(from.relative)
if err != nil {
return nil, fmt.Errorf("stating %s: %w", from.relative, err)
}
change.mode = info.Mode()
content, err := root.ReadFile(from.relative)
if err != nil {
return nil, fmt.Errorf("reading %s: %w", from.relative, err)
}
if err := tracker.CheckConflict(from.absolute, content); err != nil {
return nil, fmt.Errorf("%s", fileConflictMessage(from.relative))
}
change.before = string(content)
if operation.kind == structuredPatchDelete && operation.verifyDelete {
if err := verifyUnifiedDeletion(change.before, from.relative, operation); err != nil {
return nil, err
}
}
if operation.kind == structuredPatchUpdate || operation.kind == structuredPatchCopy {
updated, err := applyStructuredPatchUpdate(change.before, from.relative, operation.chunks)
if err != nil {
return nil, err
}
change.after = applyEOFNewline(updated, operation.eofNewline)
if operation.kind == structuredPatchCopy && from.absolute == to.absolute {
return nil, fmt.Errorf("cannot copy %s onto itself", from.relative)
}
if from.absolute != to.absolute {
if _, err := root.Lstat(to.relative); err == nil {
return nil, fmt.Errorf("cannot move %s to %s because the destination already exists", from.relative, to.relative)
} else if !os.IsNotExist(err) {
return nil, err
}
}
}
}
changes = append(changes, change)
}
return changes, nil
}
func resolveStructuredPatchTarget(root, path string) (structuredPatchTarget, error) {
// Relative traversal is rejected outright. An absolute path is allowed when
// it resolves inside the apply root (models routinely echo the absolute path
// they were shown by read_file); resolveWorkspaceTargetPath rejects anything
// that lands outside.
if path == ".." || strings.HasPrefix(filepath.ToSlash(path), "../") {
return structuredPatchTarget{}, fmt.Errorf("patch path %q must stay inside the workspace", path)
}
absolute, relative, err := resolveWorkspaceTargetPath(root, normalizePatchPathForRoot(root, path))
if err != nil {
return structuredPatchTarget{}, err
}
return structuredPatchTarget{requested: path, absolute: absolute, relative: relative}, nil
}
func applyStructuredPatchUpdate(content, path string, chunks []structuredPatchChunk) (string, error) {
lineEnding := structuredPatchLineEnding(content)
if lineEnding == "\r\n" {
content = strings.ReplaceAll(content, "\r\n", "\n")
}
lines := strings.Split(content, "\n")
trailingNewline := content != "" && len(lines) > 0 && lines[len(lines)-1] == ""
if len(lines) > 0 && lines[len(lines)-1] == "" {
lines = lines[:len(lines)-1]
}
replacements := make([]structuredReplacement, 0, len(chunks))
lineIndex := 0
for _, chunk := range chunks {
if chunk.hasContext {
index, ambiguous := findStructuredPatchSequence(lines, []string{chunk.context}, lineIndex, false)
if ambiguous {
return "", fmt.Errorf("context %q is ambiguous in %s; provide a more specific hunk header", chunk.context, path)
}
if index < 0 {
return "", fmt.Errorf("could not find context %q in %s", chunk.context, path)
}
lineIndex = index + 1
}
if len(chunk.old) == 0 {
// A context anchor identifies the line immediately before a pure
// insertion. Only an explicit end-of-file hunk (or an unanchored
// insertion) belongs at the file end. A unified diff names the
// insertion point in its range instead; a zero-context hunk (as
// from `git diff -U0`) has nothing to verify the position against,
// so it is position-trusting by design — the same semantics git
// apply uses without fuzz. Hunks that carry context are verified.
start := lineIndex
if chunk.hasHint {
start = max(chunk.hint, lineIndex)
} else if chunk.endOfFile || !chunk.hasContext {
start = len(lines)
}
if start > len(lines) {
start = len(lines)
}
replacement, err := structuredPatchReplacement(chunk, lines, start)
if err != nil {
return "", fmt.Errorf("building replacement for %s: %w", path, err)
}
replacements = append(replacements, structuredReplacement{start: start, replacement: replacement})
lineIndex = start
continue
}
var index int
ambiguous := false
if chunk.hasHint && chunk.hint >= lineIndex && sequenceMatchesAt(lines, chunk.old, chunk.hint) {
index = chunk.hint
} else {
index, ambiguous = findStructuredPatchSequence(lines, chunk.old, lineIndex, chunk.endOfFile)
}
if ambiguous {
return "", fmt.Errorf("expected lines are ambiguous in %s; provide more surrounding context:\n%s", path, strings.Join(chunk.old, "\n"))
}
if index < 0 {
return "", fmt.Errorf("could not find expected lines in %s:\n%s", path, strings.Join(chunk.old, "\n"))
}
replacement, err := structuredPatchReplacement(chunk, lines, index)
if err != nil {
return "", fmt.Errorf("building replacement for %s: %w", path, err)
}
replacements = append(replacements, structuredReplacement{start: index, length: len(chunk.old), replacement: replacement})
lineIndex = index + len(chunk.old)
}
for index := 1; index < len(replacements); index++ {
previous := replacements[index-1]
current := replacements[index]
if current.start < previous.start+previous.length {
return "", fmt.Errorf("overlapping or out-of-order hunks in %s; provide non-overlapping context", path)
}
}
for index := len(replacements) - 1; index >= 0; index-- {
replacement := replacements[index]
lines = append(lines[:replacement.start], append(replacement.replacement, lines[replacement.start+replacement.length:]...)...)
}
if len(lines) == 0 {
return "", nil
}
updated := strings.Join(lines, lineEnding)
if trailingNewline {
updated += lineEnding
}
return updated, nil
}
// verifyUnifiedDeletion checks, byte for byte, that the lines a unified
// deletion removes are exactly the file's current content. A deletion is not
// recoverable, so none of the whitespace tolerance used to locate update
// hunks applies here; only the file's own line-ending style is normalised.
// With no hunks (git's header-only form) the file must be empty.
func verifyUnifiedDeletion(current, path string, operation structuredPatchOperation) error {
var removed []string
for _, chunk := range operation.chunks {
if len(chunk.new) > 0 {
return fmt.Errorf("deletion of %s must not add lines", path)
}
removed = append(removed, chunk.old...)
}
expected := strings.Join(removed, "\n")
if len(removed) > 0 && !operation.oldNoNewline {
expected += "\n"
}
actual := current
if structuredPatchLineEnding(current) == "\r\n" {
actual = strings.ReplaceAll(current, "\r\n", "\n")
}
if actual != expected {
if len(removed) == 0 {
return fmt.Errorf("deletion of %s expects an empty file but it has content; include the removed lines in the patch", path)
}
return fmt.Errorf("deletion of %s does not match its current content; the removed lines must equal the whole file", path)
}
return nil
}
// sequenceMatchesAt reports whether wanted appears verbatim at lines[start:].
func sequenceMatchesAt(lines, wanted []string, start int) bool {
if start < 0 || start+len(wanted) > len(lines) {
return false
}
for offset, line := range wanted {
if lines[start+offset] != line {
return false
}
}
return true
}
// applyEOFNewline forces or strips a single trailing line ending when a
// unified diff's end-of-file marker asked for it.
func applyEOFNewline(content string, mode eofNewlineMode) string {
switch mode {
case eofNewlinePresent:
if content != "" && !strings.HasSuffix(content, "\n") {
return content + structuredPatchLineEnding(content)
}
case eofNewlineAbsent:
content = strings.TrimSuffix(content, "\n")
content = strings.TrimSuffix(content, "\r")
}
return content
}
func structuredPatchReplacement(chunk structuredPatchChunk, source []string, start int) ([]string, error) {
if len(chunk.newSourceOffsets) != len(chunk.new) {
return nil, fmt.Errorf("context mapping has %d entries for %d output lines", len(chunk.newSourceOffsets), len(chunk.new))
}
replacement := append([]string(nil), chunk.new...)
for outputIndex, sourceOffset := range chunk.newSourceOffsets {
if sourceOffset < 0 {
continue
}
sourceIndex := start + sourceOffset
if sourceIndex < 0 || sourceIndex >= len(source) {
return nil, fmt.Errorf("context source index %d is outside the matched file", sourceIndex)
}
replacement[outputIndex] = source[sourceIndex]
}
return replacement, nil
}
func structuredPatchLineEnding(content string) string {
if strings.Contains(content, "\r\n") && !strings.Contains(strings.ReplaceAll(content, "\r\n", ""), "\n") {
return "\r\n"
}
return "\n"
}
type structuredReplacement struct {
start int
length int
replacement []string
}
func findStructuredPatchSequence(lines, wanted []string, start int, endOfFile bool) (int, bool) {
if len(wanted) == 0 {
return start, false
}
if len(wanted) > len(lines) {
return -1, false
}
searchStart := start
if endOfFile {
searchStart = len(lines) - len(wanted)
if searchStart < start {
searchStart = start
}
}
for _, equal := range []func(string, string) bool{
func(left, right string) bool { return left == right },
func(left, right string) bool {
return strings.TrimRight(left, " \t") == strings.TrimRight(right, " \t")
},
func(left, right string) bool { return strings.TrimSpace(left) == strings.TrimSpace(right) },
} {
match := -1
for index := searchStart; index+len(wanted) <= len(lines); index++ {
matched := true
for offset, line := range wanted {
if !equal(lines[index+offset], line) {
matched = false
break
}
}
if matched {
if match >= 0 {
return -1, true
}
match = index
}
}
if match >= 0 {
return match, false
}
}
return -1, false
}
func applyStructuredPatchChanges(root *os.Root, changes []structuredPatchChange, tracker *FileTracker) error {
// committed lists, in order, the paths whose change reached disk before a
// later change failed, so the caller (and the model) knows exactly which
// files now hold the patched content and which were never touched.
var committed []string
for _, change := range changes {
done, err := applyStructuredPatchChange(root, change)
if done {
committed = append(committed, structuredPatchChangePaths(change)...)
}
if err != nil {
forgetStructuredPatchFiles(tracker, changes)
if len(committed) > 0 {
return fmt.Errorf("%w; patch was partially applied — already committed: %s; the remaining files are unchanged; re-read the committed files before retrying", err, strings.Join(committed, ", "))
}
return err
}
}
return nil
}
// structuredPatchChangePaths names the workspace-relative paths a committed
// change touched: the destination, plus the source of a move or copy.
func structuredPatchChangePaths(change structuredPatchChange) []string {
if change.kind == structuredPatchDelete {
return []string{change.from.relative}
}
paths := []string{change.to.relative}
if change.from.absolute != change.to.absolute && change.from.relative != "" {
paths = append([]string{change.from.relative}, paths...)
}
return paths
}
func forgetStructuredPatchFiles(tracker *FileTracker, changes []structuredPatchChange) {
if tracker == nil {
return
}
for _, change := range changes {
tracker.Forget(change.from.absolute)
if change.to.absolute != change.from.absolute {
tracker.Forget(change.to.absolute)
}
}
}
// structuredPatchBeforeCommit, when set, runs before each change's pre-commit
// recheck. Tests use it to alter the filesystem between planning and commit
// deterministically; it is nil in production.
var structuredPatchBeforeCommit func(change structuredPatchChange)
func applyStructuredPatchChange(root *os.Root, change structuredPatchChange) (bool, error) {
if structuredPatchBeforeCommit != nil {
structuredPatchBeforeCommit(change)
}
// The planner validated the source against the bytes it read; re-read
// through the same root immediately before committing so a change made
// by another process in between is refused rather than overwritten or
// removed.
if change.kind != structuredPatchAdd {
current, err := root.ReadFile(change.from.relative)
if err != nil {
return false, fmt.Errorf("re-reading %s before commit: %w", change.from.relative, err)
}
if string(current) != change.before {
return false, fmt.Errorf("%s changed on disk between planning and commit; re-read it and retry", change.from.relative)
}
}
switch change.kind {
case structuredPatchDelete:
if err := root.Remove(change.from.relative); err != nil {
return false, fmt.Errorf("deleting %s: %w", change.from.relative, err)
}
return true, nil
case structuredPatchAdd, structuredPatchCopy:
return writeStructuredPatchFile(root, change.to, change.after, change.mode, true)
case structuredPatchUpdate:
moving := change.from.absolute != change.to.absolute
committed, err := writeStructuredPatchFile(root, change.to, change.after, change.mode, moving)
if err != nil {
return committed, err
}
if moving {
if err := root.Remove(change.from.relative); err != nil {
return true, fmt.Errorf("removing moved source %s: %w", change.from.relative, err)
}
}
return true, nil
}
return false, fmt.Errorf("unsupported structured patch operation")
}
func writeStructuredPatchFile(root *os.Root, target structuredPatchTarget, content string, mode os.FileMode, createOnly bool) (bool, error) {
parent := filepath.Dir(target.relative)
if err := root.MkdirAll(parent, 0o755); err != nil {
return false, fmt.Errorf("creating parent directory for %s: %w", target.relative, err)
}
temp, tempName, err := pathjail.CreateTemp(root, parent, "zero-patch", ".tmp")
if err != nil {
return false, fmt.Errorf("writing %s: %w", target.relative, err)
}
defer func() { _ = removeStructuredPatchTemp(root, tempName) }()
if _, err := temp.WriteString(content); err != nil {
_ = temp.Close()
return false, fmt.Errorf("writing %s: %w", target.relative, err)
}
if err := temp.Chmod(mode.Perm()); err != nil {
_ = temp.Close()
return false, fmt.Errorf("writing %s: %w", target.relative, err)
}
if err := temp.Close(); err != nil {
return false, fmt.Errorf("writing %s: %w", target.relative, err)
}
if createOnly {
return publishStructuredPatchNoReplace(root, tempName, target.relative, mode)
}
if err := root.Rename(tempName, target.relative); err != nil {
return false, fmt.Errorf("writing %s: %w", target.relative, err)
}
return true, nil
}
func publishStructuredPatchNoReplace(root *os.Root, source, target string, mode os.FileMode) (bool, error) {
return publishStructuredPatchNoReplaceWith(root, source, target, mode, root.Link)
}
func publishStructuredPatchNoReplaceWith(root *os.Root, source, target string, mode os.FileMode, link func(string, string) error) (bool, error) {
if linkErr := link(source, target); linkErr == nil {
cleanupErr := removeStructuredPatchTemp(root, source)
chmodErr := root.Chmod(target, mode.Perm())
if cleanupErr != nil || chmodErr != nil {
return true, fmt.Errorf("publishing %s: %w", target, errors.Join(cleanupErr, chmodErr))
}
return true, nil
} else if errors.Is(linkErr, os.ErrExist) {
return false, fmt.Errorf("creating %s: %w", target, linkErr)
} else {
committed, copyErr := copyStructuredPatchNoReplace(root, source, target, mode)
if copyErr != nil {
return committed, fmt.Errorf("creating %s without overwrite: hard link failed: %w; exclusive-copy fallback failed: %w", target, linkErr, copyErr)
}
return committed, nil
}
}
func copyStructuredPatchNoReplace(root *os.Root, source, target string, mode os.FileMode) (bool, error) {
return copyStructuredPatchNoReplaceWith(root, source, target, mode, io.Copy, removeStructuredPatchTemp)
}
func copyStructuredPatchNoReplaceWith(
root *os.Root,
source, target string,
mode os.FileMode,
copyFile func(io.Writer, io.Reader) (int64, error),
cleanup func(*os.Root, string) error,
) (bool, error) {
sourceFile, err := root.Open(source)
if err != nil {
return false, err
}
sourceClosed := false
defer func() {
if !sourceClosed {
_ = sourceFile.Close()
}
}()
targetFile, err := root.OpenFile(target, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
if err != nil {
return false, err
}
committed := true
cleanupTarget := func(cause error) (bool, error) {
_ = targetFile.Close()
if cleanupErr := cleanup(root, target); cleanupErr == nil {
return false, cause
} else {
return committed, errors.Join(cause, fmt.Errorf("cleaning partial target: %w", cleanupErr))
}
}
if _, err := copyFile(targetFile, sourceFile); err != nil {
return cleanupTarget(err)
}
if err := sourceFile.Close(); err != nil {
sourceClosed = true
return cleanupTarget(err)
}
sourceClosed = true
if err := targetFile.Chmod(mode.Perm()); err != nil {
return cleanupTarget(err)
}
if err := targetFile.Close(); err != nil {
if cleanupErr := cleanup(root, target); cleanupErr == nil {
return false, err
} else {
return true, errors.Join(err, fmt.Errorf("cleaning target after close failure: %w", cleanupErr))
}
}
if err := cleanup(root, source); err != nil {
return true, fmt.Errorf("removing published source: %w", err)
}
return true, nil
}
func removeStructuredPatchTemp(root *os.Root, name string) error {
chmodErr := root.Chmod(name, 0o600)
if os.IsNotExist(chmodErr) {
return nil
}
removeErr := root.Remove(name)
if os.IsNotExist(removeErr) {
removeErr = nil
}
if removeErr == nil {
return nil
}
return errors.Join(chmodErr, removeErr)
}
func recordStructuredPatchFile(tracker *FileTracker, absolute string, created, seenWhole bool) {
if tracker == nil {
return
}
content, err := os.ReadFile(absolute)
if err != nil {
tracker.Forget(absolute)
return
}
info, _ := os.Stat(absolute)
tracker.Record(absolute, content, info)
if seenWhole {
lines := trackedLineTotal(string(content))
tracker.RecordSeenRange(absolute, 1, lines, lines)
}
if created {
tracker.RecordCreated(absolute)
}
}
func changedFilesFromStructuredPatch(relativeRoot string, changes []structuredPatchChange) []string {
seen := make(map[string]bool)
var paths []string
for _, change := range changes {
targets := []structuredPatchTarget{change.to}
if change.from.absolute != change.to.absolute {
targets = append([]structuredPatchTarget{change.from}, targets...)
}
for _, target := range targets {
path := target.relative
if relativeRoot != "" && relativeRoot != "." {
path = filepath.ToSlash(filepath.Join(relativeRoot, path))
}
if !seen[path] {
seen[path] = true
paths = append(paths, path)
}
}
}
return paths
}
func structuredPatchPreview(changes []structuredPatchChange) string {
var preview strings.Builder
for _, change := range changes {
path := change.to.relative
if change.kind == structuredPatchDelete {
path = change.from.relative
}
before, after := change.before, change.after
if change.kind == structuredPatchDelete {
after = ""
}
diff := boundedUnifiedDiff(path, before, after)
if diff == "" {
continue
}
if preview.Len()+len(diff)+1 > maxToolPreviewBytes {
break
}
if preview.Len() > 0 {
preview.WriteByte('\n')
}
preview.WriteString(diff)
}
return capPreviewDiff(preview.String())
}