Compare commits

...
Author SHA1 Message Date
Vasanthdev2004 7a94518f82 fix: bump Go to 1.26.10 for GO-2026-6809
govulncheck now reports GO-2026-6809 (crypto/x509 distrusts compromised
intermediate certificates) against Go 1.26.9, reachable through
daemon.WriteFrame, so Security & Code Health fails on main and on every
open PR. It is fixed in 1.26.10.

CI and release builds read the toolchain from go.mod. The stated
source-build minimum in README.md, README_ZH.md and docs/INSTALL.md
moves with it.
2026-10-21 12:23:35 +05:30
Vasanth T cd3e0b0a9f test(cli): isolate the exec image tests from the models.dev refresh (#1161)
runExec starts a background models.dev download unless
ZERO_DISABLE_MODELS_FETCH is set. TestRunExecImageAttachmentHistory moved
the user directories into a TempDir by hand but left the fetch on, so a
5.7 MB download landed in the TempDir while its cleanup ran, and the
macOS runner failed with "unlinkat ... directory not empty". The two
older image tests had no isolation and fetched into the real user cache.

Use isolateCLIUserState in all three, which also points the cache path
into the fixture and turns the fetch off.
2026-10-18 10:08:09 +08:00
Felix-Ayush aa9ba8412e fix(tools): grep stops where a file's line breaks stop (#1160)
* fix(tools): grep stops where a file's line breaks stop

scanGrepFile treated EOF after a trailing line break - and an empty file -
as the START of another line, and handed the matcher an empty line one
number past the real content. Every pattern that matches an empty string
then reported a match on a line no reader sees:

  a.txt holds "x\n"; grep pattern "^$"
    -> a.txt:2:           (grep -c '' GNU-style reports no such line)
  same file, pattern ".*", count mode
    -> "2 matches found"  (one real line, one phantom)
  an empty file, pattern "^$"
    -> a.txt:1:           (the file has no lines at all)

read_file and the file tracker's countLines already state the rule the
scan violated: "the trailing newline does not open a line", and an empty
file has no lines. The scan now stops there too, so anchored and
empty-matching patterns can only match the lines the content carries.

Real matches, numbering, count-per-hit accounting for non-empty patterns,
file mode, glob and symlink handling are unchanged.

* test(tools): make the grep line-number parity test catch the phantom

The parity test matched only "needle", a pattern that cannot match the
empty line the old EOF branch invented, so its absence-of-line-4
assertion passed on the code being fixed. The pattern now also matches an
empty line, and the expected output is asserted exactly: on the pre-fix
scanner the same run reported

    grep = "a.txt:2: needle two\na.txt:4: ",
        want exactly the line read_file numbers 2

so the phantom is visible right beside the real line, and the test is
about the numbering claim it states. Also gofmt-aligns the case table
that made the CI formatting check fail.
2026-10-18 10:07:34 +08:00
Felix-Ayush a97db8832f fix(redaction): redact credentials whose trigger a greedy span swallowed (#1158)
Two credentials glued end to end share one recovering property: the second
one's trigger prefix is where redaction can re-anchor. For fixed-length
bodies that anchor is exact (AKIA{16} stops at the next key), which is the
case the anchored end-chaining always covered. For unbounded bodies it
isn't: github's [A-Za-z0-9]{36,} body eats the literal "ghp" of the token
glued behind it, so the first match ends PAST the second trigger, the scan
that re-anchors at the span end finds nothing there, and the second
credential leaves the output as +_bbbb... / .eyJzdWIi... plaintext:

  RedactString("ghp_aaa..36a..ghp_bbb..36b..")
    -> "[REDACTED]_bbbb......"   (second token's body printed in full)
  RedactString(jwtA+jwtB)
    -> "[REDACTED].eyJzdWIiOiI5ODc2...SA" (second token's payload+sig)

The same holds across shapes (a github key directly feeding "sk-ant-..."
through its body) and for three tokens in one run.

redactAdjacent now follows the run a match starts: at each boundary it
re-binds the match at the EARLIEST trigger occurrence inside the span that
stills validates as its shape (the head before the trigger must match the
anchored pattern exactly, and the run chained from the trigger must reach
past the span, so a mid-body lookalike and a mid-span cut that recovers
nothing leave the span alone), and continues under whichever shape matched
the swallowed trigger. Candidate examination is capped at
maxTriggerOccurrences per span so body spam of trigger lookalikes costs
what the span costs, and the first validating cut wins.

End-chaining for fixed-length bodies, single-credential shapes, custom
replacement templates, and the split path's announced no-op-for-contiguous
text guarantee are all unchanged; the regression tests pin the glued pairs
that previously leaked plus the AWS chaining and lone-token cases.
2026-10-18 10:07:04 +08:00
9 changed files with 514 additions and 49 deletions

No files matched your search

+2 -2
View File
@@ -12,7 +12,7 @@
<p align="center">
<a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-MIT-blue"></a>
<img alt="Go 1.26.9+" src="https://img.shields.io/badge/Go-1.26.9+-00ADD8?logo=go&logoColor=white">
<img alt="Go 1.26.10+" src="https://img.shields.io/badge/Go-1.26.10+-00ADD8?logo=go&logoColor=white">
<img alt="25+ providers" src="https://img.shields.io/badge/providers-25+-34E2EA">
<a href="https://discord.gg/CaQDS6wdFn"><img alt="Discord" src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white"></a>
<br>
@@ -158,7 +158,7 @@ irm https://raw.githubusercontent.com/Twigpine/zero/main/scripts/install.ps1 | i
### From source
Source builds require Go 1.26.9+.
Source builds require Go 1.26.10+.
```bash
git clone https://github.com/Twigpine/zero.git
+2 -2
View File
@@ -12,7 +12,7 @@
<p align="center">
<a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-MIT-blue"></a>
<img alt="Go 1.26.9+" src="https://img.shields.io/badge/Go-1.26.9+-00ADD8?logo=go&logoColor=white">
<img alt="Go 1.26.10+" src="https://img.shields.io/badge/Go-1.26.10+-00ADD8?logo=go&logoColor=white">
<img alt="25+ providers" src="https://img.shields.io/badge/providers-25+-34E2EA">
<a href="https://discord.gg/CaQDS6wdFn"><img alt="Discord" src="https://img.shields.io/badge/Discord-join-5865F2?logo=discord&logoColor=white"></a>
<br>
@@ -136,7 +136,7 @@ irm https://raw.githubusercontent.com/Twigpine/zero/main/scripts/install.ps1 | i
### 从源码构建
源码构建需要 Go 1.26.9+。
源码构建需要 Go 1.26.10+。
```bash
git clone https://github.com/Twigpine/zero.git
+2 -2
View File
@@ -190,7 +190,7 @@ Every release so far declares the module path `github.com/Gitlawb/zero`, so
fails with "module declares its path as" until a release is tagged with the new
module path.
This builds from source, so it needs Go 1.26.9+ and it does not go through the
This builds from source, so it needs Go 1.26.10+ and it does not go through the
release archives. Two consequences worth knowing before you pick it:
- On Linux, Bubblewrap is still required for native sandboxing. Zero resolves
@@ -218,7 +218,7 @@ Build a local binary:
go build -o zero ./cmd/zero
```
Source builds require Go 1.26.9+.
Source builds require Go 1.26.10+.
### Sandbox Helpers For Source Builds
+1 -1
View File
@@ -1,6 +1,6 @@
module github.com/Twigpine/zero
go 1.26.9
go 1.26.10
require (
charm.land/bubbles/v2 v2.1.1
+6 -4
View File
@@ -54,10 +54,10 @@ func TestRunExecImageAttachmentHistory(t *testing.T) {
{name: "stream and file images", model: "gpt-4.1", fileImage: true},
} {
t.Run(tc.name, func(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"HOME", "USERPROFILE", "APPDATA", "LOCALAPPDATA", "XDG_CONFIG_HOME", "XDG_CACHE_HOME", "XDG_DATA_HOME", "XDG_STATE_HOME"} {
t.Setenv(name, root)
}
// The shared isolation also turns off the models.dev refresh that
// runExec starts in the background. Without it, the refresh downloads
// the catalog into this HOME and races the TempDir cleanup.
isolateCLIUserState(t)
cwd := t.TempDir()
png := []byte{0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0, 0, 0, 0x0d, 0x49, 0x48, 0x44, 0x52}
encoded := base64.StdEncoding.EncodeToString(png)
@@ -148,6 +148,7 @@ func TestRunExecImageAttachmentHistory(t *testing.T) {
// user turn). Before the fix, stream-json images were parsed and validated but
// silently dropped — never threaded into the agent run.
func TestRunExecStreamJSONImageReachesAgent(t *testing.T) {
isolateCLIUserState(t)
cwd := t.TempDir()
// A minimal PNG, base64-encoded for the stream-json image payload.
@@ -217,6 +218,7 @@ func TestRunExecStreamJSONImageReachesAgent(t *testing.T) {
// the fix, ResolvePrompt's "must include at least one prompt or user message
// event" error was returned before images were considered, rejecting the turn.
func TestRunExecStreamJSONImageOnlyMessageProceeds(t *testing.T) {
isolateCLIUserState(t)
cwd := t.TempDir()
png := []byte{
+241 -31
View File
@@ -112,6 +112,42 @@ var textSecretPrefixes = [][]string{
{"eyJ"},
}
type secretTrigger struct {
literal string
// shape indexes the anchoredSecretPatterns entry the trigger belongs to,
// so a cut can chain the bytes after it against the shape the trigger
// actually starts.
shape int
}
// secretTriggers flattens textSecretPrefixes into one sorted list for the
// swallowed-trigger search below. Sorted so cuts are discovered earliest-first
// regardless of which pattern list they came from.
var secretTriggers = func() []secretTrigger {
pairs := []secretTrigger{}
for shape, prefixes := range textSecretPrefixes {
for _, prefix := range prefixes {
pairs = append(pairs, secretTrigger{literal: prefix, shape: shape})
}
}
sort.Slice(pairs, func(i, j int) bool { return pairs[i].literal < pairs[j].literal })
return pairs
}()
// longestTrigger bounds how far past a swallowed-scored span the trigger
// inspection may need to look: an occurrence whose last bytes run past the
// span end (a greedy body stops at a byte outside its class, and several
// triggers end in one).
var longestTrigger = func() int {
longest := 0
for _, trigger := range secretTriggers {
if len(trigger.literal) > longest {
longest = len(trigger.literal)
}
}
return longest
}()
// anchoredSecretPatterns mirrors textSecretPatterns with the leading \b
// replaced by \A, so each shape can be tested at an exact byte offset. A
// credential that starts exactly where a redacted span ends (two AWS keys
@@ -129,52 +165,226 @@ var anchoredSecretPatterns = func() []*regexp.Regexp {
return out
}()
// redactAdjacent redacts every match of pattern, plus any match of anchored
// that begins exactly where a redacted span ends. This catches credentials
// glued end to end (AKIA...AKIA...) without weakening the leading-boundary
// rule elsewhere: a mid-word occurrence that does not abut a redacted span
// still does not match. Chained runs (three or more glued credentials) are
// followed to the end.
// A credential's body can run into the trigger prefix of the NEXT one when
// both are glued end to end. For fixed-length bodies this cannot happen — the
// `{16}` stops the body exactly where the next AKIA...AKIA... key begins,
// which is the case the anchored end-chaining always covered. An unbounded
// body ({36,} on a github key, {20,} on an API key) swallows whichever trigger
// bytes sit inside its class — "ghp" after a github key is all lowercase
// letters — so the first match ends PAST the next credential's trigger, and a
// scan that re-anchors at the span end finds nothing: the second credential
// is never redacted.
//
// redactedSpan is one region of a redaction: the slice s[start:end] and the
// bytes to write in its place. Chained runs can link shapes together, so a
// span says which shape produced it rather than assuming the caller's pattern
// covered it.
type redactedSpan struct {
start int
end int
// text replaces s[start:end]. Every entry in textSecretPatterns captures
// nothing, so this is the replacement itself unless a custom replacement
// template references a group; anchoring the expansion to the shape that
// matched keeps that working for cross-shape chains.
text string
}
// redactAdjacent redacts every match of pattern, plus every credential glued
// into the runs those matches start. Runs follow three loss-of-boundary
// mechanisms: end-chaining (a credential glued exactly at the previous span's
// end, the AKIA...AKIA... case), all-shape chaining (a glued credential of a
// different shape), and the swallowed-trigger cut (one credential's greedy
// body runs into the next one's trigger prefix, hiding it from any scan that
// re-anchors at a span end). A run ends at the first byte boundary where no
// shape matches; a mid-word lookalike never matches and never starts one.
func redactAdjacent(s string, pattern, anchored *regexp.Regexp, replacement string) string {
spans := pattern.FindAllStringIndex(s, -1)
// Bound the outer loop to the original match count. The inner loop
// appends chained spans to the same slice; re-reading len(spans) would
// re-chain every appended span, growing the span count (and the
// FindStringIndex calls) exponentially on long glued runs.
n := len(spans)
for i := 0; i < n; i++ {
end := spans[i][1]
for end < len(s) {
loc := anchored.FindStringIndex(s[end:])
if loc == nil || loc[0] != 0 || loc[1] <= 0 {
break
}
newEnd := end + loc[1]
spans = append(spans, []int{end, newEnd})
end = newEnd
}
plain := pattern.FindAllStringIndex(s, -1)
if len(plain) == 0 {
return s
}
// Bound the outer loop to the original match count. Every span collects
// its own run; re-reading len(plain) after appends would re-run every
// appended credential and grow the span count (with the anchored scans
// each drags) exponentially on long glued runs.
n := len(plain)
spans := make([]redactedSpan, 0, n+4)
for i := 0; i < n; i++ {
spans = appendCredentialRun(s, plain[i][0], plain[i][1], replacement, spans)
}
return writeRedactedSpans(s, spans)
}
func writeRedactedSpans(s string, spans []redactedSpan) string {
if len(spans) == 0 {
return s
}
sort.Slice(spans, func(i, j int) bool { return spans[i][0] < spans[j][0] })
sort.Slice(spans, func(i, j int) bool { return spans[i].start < spans[j].start })
var out strings.Builder
pos := 0
for _, sp := range spans {
if sp[0] < pos {
if sp.start < pos {
continue // overlapping span, already covered
}
out.WriteString(s[pos:sp[0]])
// s[sp[0]:sp[1]] is a match (anchored spans match at \b when taken
// alone), so ReplaceAllString expands $ references exactly as the
// plain loop below would.
out.WriteString(pattern.ReplaceAllString(s[sp[0]:sp[1]], replacement))
pos = sp[1]
out.WriteString(s[pos:sp.start])
out.WriteString(sp.text)
pos = sp.end
}
out.WriteString(s[pos:])
return out.String()
}
// appendCredentialRun follows the path that begins exactly at start, whose
// first plain match ended at end. Every step finds the shape that matches at
// the current boundary, tries to re-bind that match at a swallowed trigger,
// and otherwise consumes the whole match; the run ends where no shape does.
func appendCredentialRun(s string, start, end int, replacement string, spans []redactedSpan) []redactedSpan {
cur := start
for cur < len(s) {
shape, matchEnd := firstShapeAt(s, cur)
if shape < 0 || matchEnd == cur {
break
}
if cutStart, cut := earliestSwallowedCut(s, cur, matchEnd, shape); cut {
// The bytes before the trigger are the credential that matched;
// from the trigger on the run continues under the next shape.
spans = append(spans, redactedSpan{start: cur, end: cutStart, text: shapeReplacement(shape, s[cur:cutStart], replacement)})
cur = cutStart
continue
}
spans = append(spans, redactedSpan{start: cur, end: matchEnd, text: shapeReplacement(shape, s[cur:matchEnd], replacement)})
cur = matchEnd
}
return spans
}
func shapeReplacement(shape int, slice, replacement string) string {
return anchoredSecretPatterns[shape].ReplaceAllString(slice, replacement)
}
// firstShapeAt returns the first anchored shape that matches exactly at off,
// with the end of its match. Shapes are tried in declaration order, so the
// strict JWT form wins over the loose one; both redact the same bytes, so the
// tie-break only decides which regex expanded the replacement.
func firstShapeAt(s string, off int) (shape int, end int) {
for shape, anchored := range anchoredSecretPatterns {
loc := anchored.FindStringIndex(s[off:])
if loc != nil && loc[0] == 0 && loc[1] > 0 {
return shape, off + loc[1]
}
}
return -1, off
}
// earliestSwallowedCut searches the matched span [start, end) of shape for a
// trigger literal that begins a valid credential right there: the head (the
// bytes before it) still matches the shape as a whole, and the run chained
// from the trigger reaches past the span. The earliest such trigger is the
// cut a boundary at that byte would have produced.
//
// The head requirement and the past-the-end requirement keep the cut honest:
// a trigger spelled mid-body that does not begin a credential leaves the span
// alone, and a cut that stayed inside the original span recovers no bytes —
// it would only split one redaction in two.
//
// The candidate set is the trigger occurrences inside the span, bounded by
// the span plus one trigger length; every occurrence of every trigger in that
// window is a candidate, earliest-first.
func earliestSwallowedCut(s string, start, end, shape int) (int, bool) {
if end-start <= 1 {
return 0, false
}
lookahead := end + longestTrigger
if lookahead > len(s) {
lookahead = len(s)
}
window := s[start+1 : lookahead]
// A trigger occurrence whose bytes begin inside the span may extend past
// its end: the greedy body stops at one byte outside its class, and
// several triggers end in exactly such a byte ("ghp_"'s underscore,
// "sk-ant-" plus a hyphen).
anchored := anchoredSecretPatterns[shape]
examined := 0
for _, q := range triggerOccurrences(window, start+1, end) {
if examined >= maxTriggerOccurrences {
// Occurrences are earliest-first, so the ones past the cap are the
// farthest into the span; a cut among them is missed and the
// greedy span stands, as it did before the cut existed.
break
}
examined++
// The bytes before the trigger must still be a complete match of this
// shape: a trigger spelling inside a body whose head does not
// validate is prose-shaped, not a swallowed credential.
loc := anchored.FindStringIndex(s[start:q])
if loc == nil || loc[0] != 0 || loc[1] != q-start {
continue
}
if runReach(s, q) > end {
return q, true
}
}
return 0, false
}
// maxTriggerOccurrences bounds how many trigger occurrences one span is
// examined for, after sorting. Real glued credentials announce their trigger
// within reach of the span's own start, so a distant lookalike run does not
// get walked to its end; the split path makes the same trade ("at most two
// passes and no more"). A cut found past the cap is missed and the greedy
// span stands, as it did before the cut existed.
const maxTriggerOccurrences = 64
// triggerOccurrences returns every occurrence of every trigger literal that
// begins inside [windowStart, windowEnd), where window is the slice rooted at
// windowStart that the caller bounded so occurrences could reach one trigger
// length past its end. Earliest first, duplicates collapsed.
func triggerOccurrences(window string, windowStart, windowEnd int) []int {
occurrences := make([]int, 0, 4)
for _, trigger := range secretTriggers {
at := 0
for {
found := strings.Index(window[at:], trigger.literal)
if found < 0 {
break
}
q := windowStart + at + found
if q >= windowEnd {
break // later occurrences for this trigger start even later
}
occurrences = append(occurrences, q)
at += found + 1
}
}
sort.Ints(occurrences)
// In-place deduplicate: duplicates come only from triggers shared between
// shapes ("eyJ"), and zero is never an occurrence — the scan starts past it.
unique := occurrences[:0]
for i, q := range occurrences {
if i > 0 && q == occurrences[i-1] {
continue
}
unique = append(unique, q)
}
return unique
}
// runReach reports how far the credential run beginning exactly at off
// extends: each boundary picks the first shape that matches there, until one
// does not. The run is followed, not redacted; this is the shared-size check
// that keeps a cut honest.
func runReach(s string, off int) int {
cur := off
for cur < len(s) {
_, matchEnd := firstShapeAt(s, cur)
if matchEnd == cur {
break
}
cur = matchEnd
}
return cur
}
var (
privateKeyPattern = regexp.MustCompile(`(?s)-----BEGIN [A-Z ]*PRIVATE KEY-----.*?-----END [A-Z ]*PRIVATE KEY-----`)
jsonStringPattern = regexp.MustCompile(`("([^"\\]*(?:\\.[^"\\]*)*)"\s*:\s*)"([^"\\]*(?:\\.[^"\\]*)*)"`)
+171
View File
@@ -0,0 +1,171 @@
package redaction
import (
"strings"
"testing"
)
// A GREEDY BODY SWALLOWS THE NEXT CREDENTIAL'S TRIGGER, SO THE END-CHAIN
// NEVER SEES IT.
//
// The anchored chain re-binds a redacted span at its END; it can only catch a
// credential that begins exactly where the previous one stops. That is true
// for fixed-length bodies (AKIA{16} stops at the next key) and false for
// unbounded ones: github's body class [A-Za-z0-9] happily eats the literal
// "ghp" of the token glued behind it, so the first match ends past the second
// trigger and the plain chain has nothing left to match. Every unbounded shape
// below leaked its second credential before the swallowed-trigger cut.
func TestRedactStringRedactsCredentialsSwallowedByAGluedRun(t *testing.T) {
// Assembled so no secret-shaped literal appears in source.
// gitleaks:allow -- synthetic redaction fixtures below
firstGitHub := "ghp_" + strings.Repeat("a", 36)
secondGitHub := "ghp_" + strings.Repeat("b", 36)
t.Run("two github tokens glued end to end", func(t *testing.T) {
got := RedactString(firstGitHub+secondGitHub, Options{})
if got != RedactedSecret+RedactedSecret {
t.Fatalf("second token survived a glued github pair: got %q", got)
}
})
t.Run("two JWTs glued end to end", func(t *testing.T) {
got := RedactString(jwtToken+otherJWT, Options{})
if got != RedactedSecret+RedactedSecret {
t.Fatalf("second token survived a glued JWT pair: got %q", got)
}
})
t.Run("third github token in the same run", func(t *testing.T) {
thirdGitHub := "ghp_" + strings.Repeat("c", 36)
got := RedactString(firstGitHub+secondGitHub+thirdGitHub, Options{})
if got != strings.Repeat(RedactedSecret, 3) {
t.Fatalf("glued github run left residue between markers: got %q", got)
}
})
}
// THE SWALLOWED TRIGGER CAN BELONG TO A DIFFERENT SHAPE. The chained run from
// a cut must try every shape at each boundary, not only the one that matched
// the swallowed span.
//
// Two outcomes are both fully redacted and the tests accept either. A github
// body stops at the hyphen an anthropic trigger carries, so the cut recovers
// the boundary and each key becomes its own marker. An anthropic body class
// spans [A-Za-z0-9_-], wide enough to cover the next trigger and the
// credential behind it whole, so the single match already consumed them and
// the requirement is only that nothing of either key survives.
func TestRedactStringRedactsACrossShapedCredentialGluedOntoARun(t *testing.T) {
// gitleaks:allow -- synthetic redaction fixtures below
github := "ghp_" + strings.Repeat("a", 36)
anthropic := "sk-ant-" + strings.Repeat("b", 20)
secondAnthropic := "sk-ant-" + strings.Repeat("d", 20)
// keptOut fails when any trigger or credential-shaped residue survives the
// redaction, the one property a glued run can never relax.
keptOut := func(t *testing.T, got string, pieces ...string) {
t.Helper()
for _, mark := range []string{"sk-ant-", "ghp_"} {
if strings.Contains(got, mark) {
t.Fatalf("a trigger survived a glued run: %q in %q", mark, got)
}
}
for _, piece := range pieces {
if strings.Contains(got, piece) {
t.Fatalf("a credential survived a glued run: parts of %q… in %q", piece[:8], got)
}
}
}
t.Run("anthropic glued behind github", func(t *testing.T) {
value := github + anthropic
got := RedactString(value, Options{})
if got != RedactedSecret+RedactedSecret {
t.Fatalf("anthropic key survived behind a github key: got %q", got)
}
keptOut(t, got, github, anthropic)
})
t.Run("anthropic glued behind anthropic, then github", func(t *testing.T) {
value := anthropic + secondAnthropic + github
got := RedactString(value, Options{})
if got == value {
t.Fatal("SETUP INVALID: the glued run is not redacted at all")
}
keptOut(t, got, anthropic, secondAnthropic, github)
})
t.Run("github glued behind anthropic", func(t *testing.T) {
value := anthropic + github
got := RedactString(value, Options{})
if got == value {
t.Fatal("SETUP INVALID: the glued run is not redacted at all")
}
keptOut(t, got, anthropic, github)
})
}
// THE CUT MUST NOT WEAKEN WHAT ALREADY WORKED.
//
// A trigger spelled inside a matched span that starts no credential there
// leaves the span whole: the bytes before it do not re-validate as their
// shape, and the run chained from it stays inside the span. The end-chaining
// for fixed-length bodies and every single-credential behavior the suite
// already pins must be untouched.
func TestRedactStringSwallowedTriggerCutKeepsExistingBehavior(t *testing.T) {
// gitleaks:allow -- synthetic redaction fixtures below
github := "ghp_" + strings.Repeat("a", 36)
t.Run("two fixed-length AWS keys still chain at the end", func(t *testing.T) {
got := RedactString("AKIAIOSFODNN7EXAMPLE"+"ASIAIOSFODNN7EXAMPLE", Options{})
if got != RedactedSecret+RedactedSecret {
t.Fatalf("glued AWS keys regressed: got %q", got)
}
})
t.Run("an interior trigger spelling starts no credential of its own", func(t *testing.T) {
// A JWT payload can contain "eyJ" away from a segment start. The strict
// form needs the segment to open with it and the loose form needs the
// three-segment skeleton, so a payload fragment is prose to both and the
// token stays one marker.
token := "eyJhbGciOiJIUzI1NiJ9." + "aaaaeyJaaaaaa" + ".QWxpY2VCb2JDYXJvbA"
if got := RedactString(token, Options{}); got != RedactedSecret {
t.Fatalf("a JWT with an interior trigger spelling changed shape: got %q", got)
}
})
t.Run("a single token is one marker, not two", func(t *testing.T) {
got := RedactString(github, Options{})
if got != RedactedSecret {
t.Fatalf("a lone token changed shape: got %q", got)
}
})
}
// A SPAM OF TRIGGER SPELLINGS INSIDE ONE BODY MUST COST WHAT THE SPAN COSTS.
//
// Every trigger occurrence inside a matched span is a cut candidate, but only
// the first maxTriggerOccurrences are examined, so the scan stays linear in
// the span: a body stuffed with lookalikes must not walk the span once per
// candidate. Measured as allocation against the same bar the split path's
// linearity test uses, the one an index-per-byte or a walk-per-credential
// explosion offends.
func TestRedactStringTriggerLookalikeSpamStaysBounded(t *testing.T) {
// No digit appears anywhere in the fixture, so the openai shape's filter
// rejects it as kebab-case prose and the anthropic shape still matches it
// whole: every "sk-ant-" inside the span is a cut candidate, and none
// validates because the run chained from any of them reaches only to the
// span's own end.
// gitleaks:allow -- synthetic redaction fixture below
value := "sk-ant-" + strings.Repeat("sk-ant-", (64<<10)/len("sk-ant-"))
if got := RedactString(value, Options{}); got != RedactedSecret {
t.Fatalf("SETUP INVALID: the lookalike span was not redacted whole: got %q", got)
}
// Sixty-four candidates, each re-checked against the span, cost a small
// constant multiple of the span. A walk over every occurrence quadruples
// the cost when the span doubles, and the split path measured that walk
// at seventeen gigabytes on a one-megabyte chain.
limit := int64(200 * len(value))
if used := redactAllocBytes(value); used > limit {
t.Fatalf("redacting a %d byte lookalike span allocated %d bytes, over the %d byte limit", len(value), used, limit)
}
}
+7 -7
View File
@@ -387,24 +387,24 @@ func scanGrepFile(ctx context.Context, resolvedRoot string, absolutePaths bool,
reader := bufio.NewReader(handle)
lineNumber := 1
sawLine := false
lastEnded := false
for {
if err := ctx.Err(); err != nil {
return err
}
raw, ended, err := readRawLine(reader)
// At EOF the scan stops where the line breaks stopped. A trailing
// line break is the boundary that CLOSED the line it ends — the rule
// the file tracker's countLines states for everything a reader
// reports, and an empty file has no lines at all. Emitting at EOF
// anyway handed every pattern that matches an empty string (anchors
// among them: ^$, \b, x*) a phantom match on a line that does not
// exist, numbered one past the real content.
if err == io.EOF {
if !sawLine || lastEnded {
return emitGrepLine(matcher, fileLabel, lineNumber, nil, emit)
}
return nil
}
if err != nil {
return nil
}
sawLine = true
lastEnded = ended
line := trimTrailingCarriageReturns(trimLineBreak(raw, ended))
if err := emitGrepLine(matcher, fileLabel, lineNumber, line, emit); err != nil {
return err
+82
View File
@@ -0,0 +1,82 @@
package tools
import (
"context"
"path/filepath"
"strings"
"testing"
)
// A LINE BREAK ENDS THE LINE IT TRAILS; IT DOES NOT OPEN THE ONE AFTER IT.
//
// The scan used to hand the matcher an empty line past the file's real content
// at EOF, so every pattern that matches an empty string — the anchored ones ^$
// and \b among them — reported a match on a line no reader sees, numbered one
// past the real content, and count mode charged that phantom to its total.
// read_file and the file tracker's countLines both report "x\n" as one line
// and an empty file as none; the grep scan now stops with them.
func TestGrepStopsAtTheLineBreaksTheFileHas(t *testing.T) {
for name, testCase := range map[string]struct {
content string
pattern string
want string
}{
"a trailing newline closes the line": {"x\n", `^$`, "content:No matches found."},
"an unterminated last line stands alone": {"x", `^$`, "content:No matches found."},
"an empty file has no lines": {"", `^$`, "content:No matches found."},
"the middle empty line is the only empty one": {"x\n\nz\n", `^$`, "content:a.txt:2: "},
"dot star does not run past the break": {"x\n", ".*", "content:a.txt:1: x"},
"count stops at the trailing break": {"x\n", `^$`, "count:0 matches found"},
"count charges only real lines": {"x\n\nz\n", `^$`, "count:1 matches found"},
"count takes the trailing empty lines": {"z\n\n\n", `^$`, "count:2 matches found"},
"an empty file counts nothing": {"", `^$`, "count:0 matches found"},
} {
t.Run(name, func(t *testing.T) {
root := t.TempDir()
writeTestFile(t, filepath.Join(root, "a.txt"), testCase.content)
mode, want, found := strings.Cut(testCase.want, ":")
if !found {
t.Fatalf("SETUP INVALID: %s carries no mode", testCase.want)
}
result := NewScopedGrepTool(root, nil).Run(context.Background(), map[string]any{
"pattern": testCase.pattern,
"output_mode": mode,
})
if result.Status != StatusOK {
t.Fatalf("grep status = %s: %s", result.Status, result.Output)
}
if result.Output != want {
t.Fatalf("grep = %q, want %q", result.Output, want)
}
})
}
}
// THE LINE NUMBERS THE SCAN REPORTS ARE THE LINE NUMBERS read_file USES, and
// neither of them runs past the breaks the content carries. The pattern also
// matches an empty line, so the line the scan used to invent at EOF would
// appear right beside the real one: numbering a real line wrong in the same
// file would fail the read side first, and the phantom would fail as a match
// the content does not carry.
func TestGrepLineNumbersAgreeWithReadFile(t *testing.T) {
root := t.TempDir()
writeTestFile(t, filepath.Join(root, "a.txt"), "one\nneedle two\nthree\n")
read := NewScopedReadFileTool(root, nil).Run(context.Background(), map[string]any{
"path": "a.txt",
})
if read.Status != StatusOK || !strings.Contains(read.Output, "2→needle two") {
t.Fatalf("SETUP INVALID: read_file does not number the needle line 2: %s", read.Output)
}
result := NewScopedGrepTool(root, nil).Run(context.Background(), map[string]any{
"pattern": "needle|^$",
})
if result.Status != StatusOK {
t.Fatalf("grep status = %s: %s", result.Status, result.Output)
}
if result.Output != "a.txt:2: needle two" {
t.Fatalf("grep = %q, want exactly the line read_file numbers 2; an extra empty match is a line the content does not have, so the scan opened one past the trailing break", result.Output)
}
}