Files
zero/internal/tools/file_tracker_test.go
KRATOS 1649008506 fix(tools): an edit keeps the reads it did not disturb (#908)
* fix(tools): an edit keeps the reads it did not disturb

Split out of #829 as an independent fix, per @Vasanthdev2004's review asking for
the small self-contained pieces to arrive separately.

A successful edit re-baselined the file through RecordHash, which drops EVERY
recorded read range. A file read in three pieces therefore lost all three to a
single two-line edit, and the next six edits into regions that had already been
read were refused as unseen — the model had to re-read what it had just read in
order to keep working.

RecordEdit replaces that on the edit path. Because the edit is ours, the changed
span is known exactly, so ranges the edit did not touch are carried across,
ranges after it are shifted by the line delta, and a range the edit spans is
split around it.

An EXTERNAL change still drops everything, deliberately: once a formatter has
rewritten the file we no longer know which line holds what was read, and the
conservative drop is the only honest answer there.

"Seen whole" is now decided in ONE place. SeenWhole derives the answer from the
ranges because the raw flag is only ever set by a single covering read, but
RecordEdit branched on the flag — so a file read whole in two chunks reported
seen-whole true, then false after one edit, putting back the write_file refusal
the derived answer exists to prevent.

countLines now reports the count a READER would give. strings.Split leaves an
empty final element for content ending in a newline, so it counted 4 lines for a
3-line file; since SeenWhole asks whether the ranges cover 1..total, that
inflated total made full coverage unreachable for a file just read in full.
Nearly every text file ends in a newline, so this was the common case.

Origin-Session: local-de382f | Claude Code | 9 prompts
Origin-Snapshot: 92ae33c95cd1
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: ae653221fa15

* chore(tools): re-request an automated review of this head

No code change. CodeRabbit's verdict on this PR is pinned to 6f0cd6c4, a commit
that was force-pushed away five days ago, and a stale changes-requested keeps the
PR at mergeStateStatus=BLOCKED exactly as a human one would. It re-reviews on
push rather than on request, and repeated @coderabbitai triggers over five days
produced nothing, so this is the only lever available without maintainer rights.

The cost is that this dismisses @Vasanthdev2004's and @anandh8x's approvals of
0b6fcf54, which is an unhappy trade for an empty commit — the tree they approved
is byte-identical to this one.

Origin-Session: local-8cd239 | Claude Code | 6 prompts
Origin-Snapshot: 149ab98453ad
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: ae653221fa15

* perf(tools): derive the changed-line span from bytes, not from per-line slices

Reported by @jatmn.

changedLineSpan split both file versions into []string, so it allocated one
string header per line for EACH version — work that scales with the size of the
file rather than the size of the edit. countLines then split a third time just to
take a length. All of it ran before RecordEdit had checked whether there was a
tracked observation to update at all.

He measured 32,036,640 bytes for two 2 MB versions of a million short lines, and
extrapolated roughly 1.6 GB of headers for a 100 MB file before counting the
content, the updated bytes and the hashes already live beside them.

The memory was not the sharp end. edit_file writes the updated bytes BEFORE
calling RecordEdit, so an out-of-memory kill lands after the user's file has
changed and before the tracker baseline catches up: the file and the record of it
disagree, and nothing says so. His instruction was to fix the allocation source
rather than add a size guard, and that a guard — if used at all — would have to
reject before os.WriteFile rather than during post-write re-baselining. Taken as
written: there is no new limit, and the same edits are accepted as before.

Two identical byte prefixes share every line that ends inside them, so counting
newlines in the common prefix counts the unchanged leading lines directly. Same
for the suffix, with one rule that the line-array version got for free: the line
a shared tail STARTS in is only shared when the tail begins on a line boundary in
both versions, or its opening bytes differ and it is a changed line that merely
ends the same way.

Measured on his own scenario, two 2 MB versions:

  changedLineSpan   32,036,640 bytes  ->  0 allocations
  countLines        per-line slice    ->  0 allocations

## The equivalence is tested, not asserted

A rewrite of a diff primitive is only safe if it is behaviour-preserving, so the
replaced implementation is kept in the test file as an oracle and both are asked
the same questions. That caught 206 mismatches in my first attempt at the byte
scan — empty versions, and trailing lines that are only partially shared. Neither
appears in the hand-written cases, and I would have shipped it.

Two regressions: the oracle comparison, and an absolute allocation ceiling rather
than a ratio, so a future rewrite that reintroduces per-line slices fails rather
than merely getting slower. Mutation: restoring strings.Split in either function
allocates 3.2 MB and 3.6 MB against a 4 KB bound.

splitLinesForTracking lost its last caller and is removed — the same obsolete-
helper lint failure that broke Windows CI on #911, caught here by running
make lint-static before pushing rather than after. 0 issues.

Rebased onto ad34dc8d, 0 behind. go test -race ./internal/tools/: clean.
Pre-existing here and on main: TestRunDoctorFormatsRedactedProviderDiagnostics
and TestRunDoctorConnectivityProbesProvider exit 3 in this environment.

Origin-Session: local-c962d7 | Claude Code | 17 prompts
Origin-Snapshot: a599377c09e0
2026-08-26 13:20:48 +08:00

239 lines
8.3 KiB
Go

package tools
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestFileTrackerRecordsAndReadsBackVersion(t *testing.T) {
tracker := NewFileTracker()
path := filepath.Join(t.TempDir(), "a.txt")
content := []byte("hello world")
if err := os.WriteFile(path, content, 0o644); err != nil {
t.Fatal(err)
}
info, err := os.Stat(path)
if err != nil {
t.Fatal(err)
}
tracker.Record(path, content, info)
version, ok := tracker.Version(path)
if !ok {
t.Fatal("expected a recorded version")
}
if version.Hash != HashContent(content) {
t.Fatalf("hash = %q, want %q", version.Hash, HashContent(content))
}
if version.Size != int64(len(content)) {
t.Fatalf("size = %d, want %d", version.Size, len(content))
}
if version.MTime.IsZero() {
t.Fatal("expected mtime to be recorded from stat info")
}
}
func TestFileTrackerRecordHashMatchesRecord(t *testing.T) {
tracker := NewFileTracker()
content := []byte("streamed content")
tracker.RecordHash("/repo/x.txt", HashContent(content), nil)
if err := tracker.CheckConflict("/repo/x.txt", content); err != nil {
t.Fatalf("recorded hash should match content, got %v", err)
}
if err := tracker.CheckConflict("/repo/x.txt", []byte("changed")); err != ErrFileChangedOnDisk {
t.Fatalf("changed content should conflict, got %v", err)
}
}
func TestCheckConflictAllowsUntrackedPath(t *testing.T) {
tracker := NewFileTracker()
// No Record call: a first-touch write has no baseline to conflict against.
if err := tracker.CheckConflict("/nowhere/x.txt", []byte("anything")); err != nil {
t.Fatalf("untracked path should not conflict, got %v", err)
}
}
func TestCheckConflictAllowsMatchingContent(t *testing.T) {
tracker := NewFileTracker()
content := []byte("stable content")
tracker.Record("/repo/x.txt", content, nil)
if err := tracker.CheckConflict("/repo/x.txt", content); err != nil {
t.Fatalf("unchanged content should not conflict, got %v", err)
}
}
func TestCheckConflictBlocksDriftedContent(t *testing.T) {
tracker := NewFileTracker()
tracker.Record("/repo/x.txt", []byte("version one"), nil)
if err := tracker.CheckConflict("/repo/x.txt", []byte("version two — changed underneath us")); err != ErrFileChangedOnDisk {
t.Fatalf("drifted content should report ErrFileChangedOnDisk, got %v", err)
}
}
func TestForgetClearsBaselineSoNextWriteIsAllowed(t *testing.T) {
tracker := NewFileTracker()
tracker.Record("/repo/x.txt", []byte("version one"), nil)
tracker.Forget("/repo/x.txt")
if _, ok := tracker.Version("/repo/x.txt"); ok {
t.Fatal("Forget should drop the recorded version")
}
if err := tracker.CheckConflict("/repo/x.txt", []byte("version two")); err != nil {
t.Fatalf("forgotten path should behave as untracked, got %v", err)
}
}
func TestRecordOverwritesBaselineWithNewContent(t *testing.T) {
tracker := NewFileTracker()
tracker.Record("/repo/x.txt", []byte("old"), nil)
tracker.Record("/repo/x.txt", []byte("new"), nil)
// After re-recording, the new content is the baseline and matches.
if err := tracker.CheckConflict("/repo/x.txt", []byte("new")); err != nil {
t.Fatalf("re-recorded content should be the new baseline, got %v", err)
}
if err := tracker.CheckConflict("/repo/x.txt", []byte("old")); err != ErrFileChangedOnDisk {
t.Fatal("the superseded content should now conflict")
}
}
func TestNilFileTrackerIsANoop(t *testing.T) {
var tracker *FileTracker
tracker.Record("/repo/x.txt", []byte("x"), nil) // must not panic
tracker.Forget("/repo/x.txt")
if _, ok := tracker.Version("/repo/x.txt"); ok {
t.Fatal("nil tracker should report no version")
}
if err := tracker.CheckConflict("/repo/x.txt", []byte("x")); err != nil {
t.Fatalf("nil tracker should never conflict, got %v", err)
}
}
func TestHashContentIsStableAndDistinguishing(t *testing.T) {
// Store results of separate calls before comparing so the stability check is
// not a same-expression comparison (staticcheck SA4000).
first := HashContent([]byte("a"))
second := HashContent([]byte("a"))
if first != second {
t.Fatal("hash must be stable for identical content")
}
if first == HashContent([]byte("b")) {
t.Fatal("hash must differ for different content")
}
}
func TestRecordCreatedTracksNewFilesInOrder(t *testing.T) {
tracker := NewFileTracker()
tracker.RecordCreated("/repo/b.txt")
tracker.RecordCreated("/repo/a.txt")
got := tracker.CreatedFiles()
want := []string{"/repo/b.txt", "/repo/a.txt"}
if len(got) != len(want) {
t.Fatalf("CreatedFiles() = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("CreatedFiles() = %v, want %v", got, want)
}
}
}
func TestRecordCreatedDeduplicates(t *testing.T) {
tracker := NewFileTracker()
tracker.RecordCreated("/repo/a.txt")
tracker.RecordCreated("/repo/a.txt")
if got := tracker.CreatedFiles(); len(got) != 1 {
t.Fatalf("CreatedFiles() = %v, want a single entry", got)
}
}
func TestNilFileTrackerCreatedFilesIsANoop(t *testing.T) {
var tracker *FileTracker
tracker.RecordCreated("/repo/a.txt") // must not panic
if got := tracker.CreatedFiles(); got != nil {
t.Fatalf("CreatedFiles() on nil tracker = %v, want nil", got)
}
}
// THE SPAN IS DERIVED IN BOUNDED MEMORY, whatever the file size.
//
// changedLineSpan used to split both versions into []string, so it allocated one
// string header per line for each — work scaling with the SIZE OF THE FILE rather
// than the size of the edit, and it ran before RecordEdit had even checked whether
// there was an observation to update. @jatmn measured 32,036,640 bytes for two
// 2 MB versions of a million short lines.
//
// The cost was not only memory. edit_file writes the updated bytes BEFORE calling
// RecordEdit, so an out-of-memory kill lands after the user's file has changed and
// before the tracker baseline catches up — file and record disagree, silently.
//
// This asserts an absolute ceiling rather than a ratio, so a future rewrite that
// reintroduces per-line slices fails here rather than merely getting slower.
func TestTheChangedSpanDoesNotAllocatePerLine(t *testing.T) {
before := strings.Repeat("x\n", 200_000)
after := before[:len(before)-2] + "y\n"
spanBytes := testing.Benchmark(func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
changedLineSpan(before, after)
}
}).AllocedBytesPerOp()
if spanBytes > 4096 {
t.Errorf("changedLineSpan allocated %d bytes for a %d-line pair; per-line slices are back", spanBytes, 200_000)
}
content := []byte(before)
countBytes := testing.Benchmark(func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
countLines(content)
}
}).AllocedBytesPerOp()
if countBytes > 4096 {
t.Errorf("countLines allocated %d bytes to return a number", countBytes)
}
}
// AND IT GIVES THE SAME ANSWERS THE LINE-ARRAY VERSION DID. The rewrite is only
// safe if it is behaviour-preserving, so the replaced implementation is kept here
// as an oracle and both are asked the same questions. This caught 206 mismatches
// in the first attempt at the byte scan — empty versions and partially shared
// trailing lines, neither of which the hand-written cases below would have found.
func TestTheByteScanAgreesWithTheLineSplit(t *testing.T) {
oracle := func(before, after string) (int, int, int) {
split := func(text string) []string {
if text == "" {
return nil
}
return strings.Split(text, "\n")
}
beforeLines, afterLines := split(before), split(after)
prefix := 0
for prefix < len(beforeLines) && prefix < len(afterLines) && beforeLines[prefix] == afterLines[prefix] {
prefix++
}
suffix := 0
for suffix < len(beforeLines)-prefix && suffix < len(afterLines)-prefix &&
beforeLines[len(beforeLines)-1-suffix] == afterLines[len(afterLines)-1-suffix] {
suffix++
}
return prefix + 1, len(beforeLines) - suffix, len(afterLines) - len(beforeLines)
}
versions := []string{
"", "a", "\n", "a\n", "a\nb", "a\nb\n", "a\nb\nc", "a\nb\nc\n",
"\n\n", "x\n\ny", "a\nb\nc\nd\ne", "same\nsame\nsame\n", "ab\nc", "b\nc",
}
for _, before := range versions {
for _, after := range versions {
wantFirst, wantLast, wantDelta := oracle(before, after)
gotFirst, gotLast, gotDelta := changedLineSpan(before, after)
if wantFirst != gotFirst || wantLast != gotLast || wantDelta != gotDelta {
t.Errorf("changedLineSpan(%q, %q) = (%d, %d, %d), want (%d, %d, %d)",
before, after, gotFirst, gotLast, gotDelta, wantFirst, wantLast, wantDelta)
}
}
}
}