Compare commits

...
Author SHA1 Message Date
Vasanthdev2004 759133cdd3 test(sandbox): keep the SID helper from colliding with #886 2026-09-03 18:19:59 +05:30
Vasanthdev2004 cf40b8680b fix(sandbox): take Everyone out of the restricted-SID list
A profile that sets denyRead drops WRITE_RESTRICTED, and the token that
replaces it carried the World SID. Every principal carries Everyone, so the
restricted-SID check passed for free on any path whose DACL grants Everyone
write, and the workspace write jail fell back to the caller's own permissions,
which is the boundary the token exists to be stricter than. No privilege, no
symlink and no race: an Everyone-writable directory was enough, and share roots
opened Everyone:F and loose installer trees supply them. #865 closed this for
the WRITE_RESTRICTED token and left this half open.

It could not simply be dropped. Without WRITE_RESTRICTED the restricted-SID
check covers reads as well, and default Windows DACLs grant BUILTIN\Users
rather than anything in the list, so a token without Everyone cannot open
cmd.exe and dies at launch with a bare access denial. Confirmed by
impersonating both token shapes against C:\Windows\System32\cmd.exe.

So the read capability takes its place. BuildWindowsACLPlan already grants that
SID on every read root and denies it on every denyRead path under exactly the
same condition, so the read allowance and the restriction become one decision,
and it names only what setup granted rather than every principal on the
machine. The principal path already worked this way.

Regressions drive a real restricted token through a real impersonated write:
an Everyone-only DACL is refused, a directory the capability grants is written,
and the token's own restricted-SID list is read back from the token. A separate
test pins the plan and the token to the same answer, because the two halves are
decided in different files off the same field.

Closes #869
2026-09-03 18:17:48 +05:30
Vasanthdev2004 88dc53dd98 fix(sandbox): check the runtime root still carries its grant, not just its pathname
cleanupSandboxRuntimeRoots reclaims inactive sibling workspace roots on an age
and count policy, treating them as disposable cache state. Setup and its marker
treat their DACL as durable provisioned state. When the reclaimed workspace runs
again, command-side preparation recreates the same deterministic pathname as the
ordinary caller, and the new directory inherits from its parent without the
capability SID ACE elevated setup applied to the object that used to be there.

ValidateWindowsSandboxSetupMarker still passed, because it fingerprints pathnames
and actions rather than the identity of the ACL-bearing object. So the restricted
child launched and then failed its cache, temp and package-cache writes with a
bare access-denied and nothing pointing at setup.

The command now asks the object, not the pathname: before the token is minted it
confirms the runtime root carries an allow ACE for its capability SID, and
refuses with the setup remedy when it does not. That reconciles the two owners
without making cleanup preserve trees it is meant to reclaim.

The regression applies a real plan, reclaims the directory the way cleanup does,
recreates the pathname the way an ordinary run does, and asserts the refusal
names the root and the remedy; accepting a missing grant fails it on that. Not
covered, and said plainly: a write performed by a real capability token, which
needs a provisioned sandbox an unelevated box cannot create. The ACE presence is
the fact the command consumes, and that is what is pinned.
2026-09-03 13:57:05 +05:30
Vasanthdev2004 b203ad1201 fix(sandbox): validate runtime candidates before the elevated create, not after
ensureWindowsSandboxRuntimeCandidates ran as Administrator and used os.MkdirAll,
which follows links. Both candidate ancestries are prepared by the invoking user,
so a junction planted at a not-yet-created component below the cache directory,
or at the fallback anchor, made elevated setup build the runtime tree beneath a
redirected, Administrator-writable target. The ACL applier's no-follow check runs
afterwards and does spot the redirection, but the privileged create has already
happened by then and sits outside its rollback boundary. A leaf check after
MkdirAll would leave the same gap.

Candidates are now created by peermsg.EnsurePrivateDir, which descends
handle-relative with no-follow, refuses any component that is a link, and refuses
a leaf this user does not own. It is handed a physically resolved base first,
because it walks from the volume root and a redirected cache or TEMP above the
owned tail is the operator's business: the same pairing the fallback anchor
already uses, and the same mistake that refused every fallback on macOS when the
parent was left unresolved. physicalTempDir is generalised to physicalDir for
that, keeping GetFinalPathNameByHandle on Windows because EvalSymlinks does not
traverse a junction.

The regression plants a junction at the first owned component and asserts the
redirected target stays empty; letting the creator follow links again fails it on
that assertion.
2026-09-03 13:52:44 +05:30
Vasanthdev2004 e81ed96af3 fix(sandbox,cli): plan the grant and its guard together, and state the tier boundary
The .git rename guard belonged to one planner rather than to the grant. The
allow-write mask both backends share includes DELETE, and it inherits from each
write root onto .git; the carveouts that actually protect git are attached to
.git/config and .git/hooks as objects, so renaming .git aside and recreating it
discards them and hands back credential.helper and core.hooksPath. The principal
planner denied DELETE there and the capability planner did not, and the
capability backend is the default. Both now derive that object from one place, so
a change to the shared mask cannot update one consumer and miss the other. The
regression reads the ACE back off the applied object rather than asserting plan
shape, because the plan is what was wrong.

The unelevated tier now states its boundary before it mutates anything. A profile
carrying denyRead runs on a strict token, which applies the restricted-SID check
to reads, so the read capability must be granted at the volume root that
permissionProfileReadRoots seeds. Elevated setup can write that DACL; an ordinary
user cannot, and the common opener asks for WRITE_DAC on every entry, so this
tier failed on that one root on every command. Dropping the root ACE instead is
not available, because the strict token would then fail its own read check for
the executable. So it is refused up front, naming the root, the denyRead cause
and elevated setup as the remedy. That last part matters: the previous
after-the-fact diagnostic told the reader that running setup elevated would NOT
fix it, which is the opposite of the truth for this case.

Separately, `sandbox exec` no longer reports a signaled child as exit 255. On
Unix ExitCode() answers -1 for signal termination and os.Exit truncates that, so
a child killed by SIGTERM was indistinguishable from one that chose to exit 255.
It now returns the conventional 128+signal, read from the ProcessState, with
Windows keeping the exit code it already reports.
2026-09-03 13:45:34 +05:30
Vasanthdev2004 31da8a9abf fix(sandbox): resolve the temp dir physically before proving the fallback anchor
peermsg.EnsurePrivateDir walks every component from the root and refuses
any link. That is the right rule for the directory Zero owns and the wrong
rule for the operator's temp location above it: on macOS os.TempDir sits
under /var, a symlink to /private/var, so 8af05997 refused every fallback
on every Mac with "refusing non-directory or symlink runtime path component
var".

Anchor the fallback under the physical temp dir instead, so only the anchor
itself is a new component. EvalSymlinks off Windows; GetFinalPathNameByHandle
on Windows, where EvalSymlinks does not traverse a junction and a redirected
TEMP would trip the same way. The fixture compares against the resolved
temp dir rather than its spelling, since t.TempDir can come back as an 8.3
short name.
2026-09-02 12:56:38 +05:30
Vasanthdev2004 8af05997e1 fix(sandbox): root the fallback runtime tree beneath a validated private anchor
The fallback runtime root moved from a fresh os.MkdirTemp parent to the
predictable <temp>/zero/runtime/v1/<hash> so that elevated setup and the later
command process agree on it. That agreement is required and stays. But a
predictable path directly under the shared system temp is not a private
allocation. On an ordinary shared /tmp another local user can create /tmp/zero
with mode 0700 first, and every affected user then fails before the sandbox
command starts; a pre-existing redirected component makes host-side
preparation lease, create, chmod and clean somewhere other than the derived
tree. Determinism was being used as if it were ownership.

The tree now lives beneath <temp>/zero-runtime-<user>, and that anchor is
created and validated by peermsg.EnsurePrivateDir before anything is built
under it: handle-relative descent, no-follow, refused if any component is a
link or the leaf is not owned by the current user, mode 0700. That primitive
already existed with exactly the semantics jatmn described, so it is exported
and reused rather than written a fourth time. On Unix the uid is in the name
because /tmp is shared; on Windows os.TempDir is already per-user and the
private descriptor plus owner check do the rest.

Validation runs on both paths that can hand back a fallback root:
sandboxRuntimeRootFor returns one itself when the cache lands inside the
workspace, and prepareSandboxRuntime asks for one when the cache root cannot be
leased. Both are covered, each with a junction planted at the anchor, and both
refuse with the junction target untouched; a control shows a clean fallback
still prepares beneath the anchor.

Two things went wrong on the way and are worth recording. The first draft
called pathWithinRoot with its arguments reversed, which asked whether the
anchor was beneath the root, answered false every time, and skipped the
validation entirely while every dependent test reported success through the
junction. And the first lease-failure trigger planted its file at the derived
leaf, so preparation failed on that mkdir before ever reaching the fallback
branch. Both fixed; with validation disabled the two refusal tests now die on
"reported success" while the control stays green.
2026-09-02 12:36:42 +05:30
Vasanthdev2004 41590d74fd test(sandbox): stop the launch preflight test asserting a machine's token
TestPrincipalLaunchPreflightExplainsAMissingPrivilege ran the real preflight and
then asserted SeAssignPrimaryTokenPrivilege appeared in the message. Which
privilege is missing is a property of the account running the test, not of the
code: an unelevated user often holds neither, but this box holds
SeAssignPrimaryTokenPrivilege and not SeIncreaseQuotaPrivilege, so only the
other name appeared and the test failed for a reason that was never a defect.

Its own comment said it asserted "the shape of the answer rather than a fixed
verdict, because the verdict legitimately differs by machine", and then it
hardcoded one of the two names. The comment was right and the assertion did not
match it.

principalLaunchPrivilegeError splits the rendering out from the token work, so
each combination is driven directly: neither held, only assign-primary-token,
only increase-quota, and nothing missing. Each asserts the message names what IS
missing and does NOT name the one that is held, since sending an operator after
a privilege they already have is its own failure. Making the message always name
both fails the two single-privilege cases.

The real preflight keeps a test, reduced to what does not vary: if it refuses,
the refusal names at least one of the two rather than something the operator
cannot act on. Stability across calls is unchanged.
2026-08-31 15:32:01 +05:30
Vasanthdev2004 55b8e2699d test(sandbox): keep the runtime-root candidates inside test-owned storage
runtimeRootTestConfig put the workspace and sandbox home under t.TempDir, but
windowsSandboxRuntimeCandidates still derived a candidate through the
production sandboxUserCacheDir seam, and the test creates every candidate and
registers os.RemoveAll cleanup for each.

So the test reached into the real user cache: it fails outright on a read-only
home, and on an ordinary developer or CI account it creates and then deletes a
path outside its own storage. The workspace hash makes a collision unlikely; it
does not make somebody else's directory test-owned.

Redirected before any candidate is derived, and restored with t.Cleanup.
Production derivation is unchanged, and the assertion that setup materializes
every granted write root still stands.

A before/after count of the real cache cannot observe this, since the test
creates and removes in the same run. The mechanism is the evidence:
prepareSandboxRuntime reads sandboxUserCacheDir, and the test RemoveAlls every
candidate that derivation produces.
2026-08-31 15:26:25 +05:30
Vasanthdev2004 16fea97904 fix(sandbox): keep principal provisioning closed until a launch path exists
The gate asked whether the CALLING process held SeAssignPrimaryTokenPrivilege
and SeIncreaseQuotaPrivilege. That is the wrong lifetime, and it got the answer
backwards where it mattered.

The process that needs those privileges is the later, ordinary command process.
Setup does not run it, and nothing carries launch authority across the UAC
boundary. So the caller-token check refused unelevated setup, which was never
the dangerous case, and PASSED elevated setup, which is precisely the case that
lands a local account, its password, its logon-right assignments, workspace
ACEs, the recovery ledger and network filter state for a backend no later
command can use. The check existed to prevent durable machine state serving
something that cannot run, and in the one configuration that creates that state
it allowed it.

Requiring every sandboxed command to run elevated would line the privileges up
and is deliberately not done: it is a worse boundary than the restricted token
it would replace, and it is not the lifecycle this is for. Until a reviewed
bootstrap or broker exists, provisioning is closed rather than half-working.

windowsPrincipalLaunchAvailable replaces the preflight var and takes no
argument, because the answer is about which mechanisms exist rather than about
the current process. It is the one place that learns to say yes when a broker
lands, and everything below it -- the plans, the ledger, the ACL and filter
work -- still builds and is still tested against it.

The runner keeps its own privilege check at launch time, which is the correct
lifetime for one; only the setup wiring is removed.

The regression test asserts the property the old shape could not express: the
refusal is stable regardless of what the calling process holds, and it does not
tell the operator to re-run from an elevated terminal, which is the advice that
produced the unusable state. Restoring the caller-passes behaviour fails both
new tests.

Refs #662, which this is foundation for and does not close.
2026-08-31 15:25:00 +05:30
Vasanthdev2004 a40b50fe1f test(sandbox): assert nothing was created, not that a count held still
TestTeardownPathDerivationCreatesNothing compared the number of entries in the
SHARED temp directory before and after deriving the teardown paths. The
assertion it wants is that deriving a path creates nothing, and a count cannot
tell creation from removal: that root is also used by every other test binary
running at the same time and by the OS, so a concurrent cleanup made the count
fall and the failure read "temp directory gained -1 entries".

It compares the entry names now and reports anything that APPEARED, which is the
question actually being asked and is indifferent to whatever else disappears.
The failure also names the entry rather than a delta, so the next person sees
what was created instead of a number.
2026-08-27 14:23:38 +05:30
Vasanthdev2004 ef8f5c9003 fix(sandbox): refuse to provision a principal this caller could never launch
Measured on an ordinary unelevated session: the token holds neither
SeAssignPrimaryTokenPrivilege nor SeIncreaseQuotaPrivilege, so the principal
launch path can never engage for the process the runner is designed to be
called from. Elevated setup does not change that, because the command runs
later from the caller rather than from setup.

Until now setup succeeded completely in that situation. It created a local
account, its password, its logon-right assignments, the workspace ACEs, the
recovery ledger and the network filter state, and then every principal-mode
command refused before opening its executable. The operator was left with
durable machine state serving a backend that cannot run, and nothing said so at
the point they could still act on it.

The check runs in the caller's own process, which is the one whose privileges
decide the answer, and before anything crosses the UAC boundary. It is wired to
the same function the launch path uses, so a change to what a launch requires
cannot leave setup provisioning for a capability that no longer exists.

Three existing tests asserted argument plumbing with the opt-in on and passed
only because nothing checked; they stub the preflight now, so they no longer
depend on the privileges of whoever runs the suite.

This does not give the principal backend a working launch path. That needs a
different architecture and is not in this change.
2026-08-27 13:27:17 +05:30
Vasanthdev2004 746b6af602 fix(sandbox): name the two role inventories, and fingerprint the whole plan
Three follow-ups from going back over the review findings. None was raised
directly; two are the same shape as things that were.

windowsPrincipalPlanFingerprint was the one of three buildWindowsPrincipalACLPlan
call sites that did not pass DenyWrite. Apply and teardown both did, so the hash
the marker carries described a different plan than the one that actually gets
applied, and a change to the policy's deny-write paths moved the applied plan
while leaving the marker where it was. It is not a live hole, because the
capability ACLPlanHash covers the same paths and moves the marker anyway. That
is also exactly what would have kept it invisible until somebody changed the
capability plan's shape.

The role list was spelled out in three places and two of them are meant to
differ, which is why writing them out by hand kept going wrong.
windowsSandboxPrincipalIsInstalled asked only the offline and online roles while
teardown retires the legacy account too, so a machine still holding the untagged
pre-split account was reported clean and the opted-out marker claimed a teardown
that had not happened. Provisioning has the opposite constraint: legacy must
never appear there or setup would recreate that account on every run of an
already-upgraded machine. Both are named now, windowsSandboxLiveRoles and
windowsSandboxRetirableRoles, with a test pinning the legacy role into exactly
one of them.

And the opt-out error said "retire the principal" when a workspace has two plus
the legacy one, all of which that re-run retires.
2026-08-27 13:27:16 +05:30
Vasanthdev2004 3239565241 fix(sandbox): bind the ACL restore to the object it snapshotted
jatmn's P1 asked for creation AND cleanup to be handle-bound rather than
resolved from a pathname. Creation and the materialization unwind both are.
The DACL restore was not, and its comment read as though it were.

It re-opened the target with a no-follow open, which rules out a reparse point
swapped in since apply and nothing else. The other substitution passes it
untouched: rename the target aside and put an ordinary directory of the same
name in its place. Nothing there is a link, so the open succeeds, the pre-apply
DACL lands on the decoy, and the real object keeps the ACEs from the setup that
just aborted. The snapshot now records the volume serial and file index of the
object it read the DACL from, the restore proves it is writing back to that same
object, and a mismatch is refused rather than forced through. Leaving the real
object with the aborted setup's ACEs is the safe direction, since the caller is
failing anyway.

Three things nothing was pinning, all of which could be deleted with a green
suite. This repository has already had a fix silently reverted by a later
change, so these are worth more than their size.

rollbackWindowsACLSnapshots documented its reverse iteration as pinned by
TestRollbackUnwindsDescendantsBeforeAncestors. That test did not exist anywhere
in the repo; the only match for the name was the sentence claiming it. The
ordering is load-bearing twice over, because a materialized directory must be
empty before its own removal and SetSecurityInfo propagates inheritable ACEs
downward, so the ancestor has to go last. It exists now.

The principal ACL rollback restores the ledger alongside the DACLs, and the
neighbouring test asserted only the order of the two ACL reverts and never read
the ledger. Deleting the restore left the suite green while the paths it put
back were unnamed, so cleanup could not find them.

windowsSandboxUserIsManaged promises an account carrying the legacy bare comment
gets the workspace key stamped on. The probe for the legacy comment was seamed
and the upgrade itself was not, so nothing could observe the call. It is seamed
now, with the negative case covered too so the assertion cannot be satisfied by
an unconditional rewrite.
2026-08-27 13:27:16 +05:30
Vasanth T d9b72ba9d0 feat(sandbox): give each workspace an offline and an online principal (#812)
* feat(sandbox): give each workspace an offline and an online principal

The principal backend stood down whenever the network was denied, which is the
default, so opting into it left the restricted-token path doing all the work in
normal use. The reason was that network denial is enforced by block filters
keyed to the offline-marker SID, and a principal token cannot carry it:
LogonUser builds a token from an account's real group memberships, and the
marker is a synthetic capability SID.

A real local group closes that gap. ZeroSandboxOffline is created by setup, the
block filters name its SID alongside the marker, and a principal is denied the
network by being a member. Each workspace therefore gets two accounts that
differ only in that membership, and the command's network mode selects between
them. A group rather than each principal's own SID because principals are per
workspace: one filter set covers every offline principal on the machine instead
of needing a filter per workspace.

Both principals are provisioned together even though a given setup run sees one
profile, because setup needs elevation and commands do not. Provisioning lazily
would mean an unelevated command discovering it needs an account it cannot
create. They also get identical filesystem access, so an approved network
command sees the same filesystem as an ordinary one.

Two orderings are load bearing. The network plan is now built AFTER
provisioning, because the group it keys to is created there; planning first
installed filters naming only the marker and left every offline principal with
an open network while looking correctly set up. And the role tag sits before
the workspace hash in the account name, so truncation at the 20 character limit
eats hash characters rather than the tag, which would otherwise collide the two
roles onto one account on exactly the workspaces most likely to truncate.

Anything that is not an explicit allow maps to the offline principal, so an
unrecognised mode loses the network rather than keeping it.

The filter identity set is resolved rather than assumed, and stays absent until
the group exists, so a machine that never provisions principals computes the
same plan as before. That matters because the plan is hashed into the setup
marker and re-derived on every command; an identity set that differed between
setup and the command path would fail every command as out of date.

Cost worth stating: this doubles the sandbox accounts on a machine, to two per
workspace.

* fix(sandbox): prove ownership before deleting a sandbox account

removeWindowsSandboxIdentity is called with a DERIVED name, so it could be
pointed at a name that happens to belong to somebody else's local account.
Deleting a user is not a recoverable mistake, and the only thing standing
between the two cases was the name matching a pattern we generate ourselves.
Raised by CodeRabbit against the test fixtures, but the production teardown path
had the same hazard, so the guard belongs there rather than in the tests.

The ownership check itself now lives on the base branch, which grew the same
helper to stop provisioning ADOPTING a squatted account. This applies it to the
other end: an account that is not ours is left alone rather than deleted. The
gated tests get the protection for free, since their pre-clean goes through the
same helper.

Also appends the trimmed offline-group SID rather than the raw one. Worth noting
the reported consequence does not hold: newWindowsWFPUserCondition canonicalises
before converting, so a padded value would have been trimmed before reaching
StringToSid. The resolver returns SID.String(), which never carries whitespace,
so this is defensive tidying rather than a fix.

* fix(sandbox): assert the filters cover principals, and report a retained account

Two follow-ups from review, both on the same theme: a control that quietly does
nothing looks identical to one that works.

The network plan must be built AFTER provisioning, because provisioning creates
the group the block filters name. Built first, the filters name only the
offline marker and every offline principal has an open network while setup
reports success. That ordering is invisible at the call site, so setup now
checks the plan actually names the offline group before installing anything and
refuses if it does not. A later refactor that moves the plan build back fails
loudly instead of producing a security control that enforces nothing.

The predicate is separate so it can be asserted directly: a plan carrying only
the marker must read as uncovered, one carrying the group as covered, case
differences must not read as missing, and a host with no group provisioned has
no principal to miss and must not be refused. Making coverage always report
true fails that test.

Removal also reported plain success when it declined to delete an account Zero
did not create. Leaving it alone is right, but telling an operator cleanup
completed when a name they may care about was deliberately retained is not.
That case is now a distinguishable sentinel, and teardown treats it as success,
since "no principal of ours under this name" is the goal state either way.

* fix(sandbox): spare adopted principals when dual-role setup rolls back

Provisioning already declined to delete an account it had adopted, and the
outer setup rollback then appended an unconditional removal for every role
that got that far. With two roles that is the common case rather than an
unlucky one: the offline role usually succeeds, so a failure in the online
role or in ACL application destroyed a principal that was working before
the run started. The removal closure is now only appended for a principal
this run created.

Threads the workspace key into the delete path as well. Ownership was
proven from the account comment alone, which on a name collision belongs
to a DIFFERENT workspace, so deleting it would have been the same
unrecoverable mistake the check exists to prevent.

Policy DenyWrite now reaches the principal ACL plan here too, matching the
single-principal path.

Fixes the mode-independence test, which required exactly one identity SID
and so failed on any Windows host that already had ZeroSandboxOffline,
where the plan legitimately carries two. CI never saw it because the Linux
and macOS jobs leave the hook nil and a fresh Windows runner has no group.
The hook is now pinned, and the test additionally asserts the property it
is named for in the group-present case, including that the infra hash
changes when the group appears, which is the cross-workspace coupling
raised for a maintainer decision.

* test(sandbox): stub the password reset and pin the resolved group SIDs

Two review points on the tests added in the previous commit.

The provisioning stub left resetWindowsSandboxUserPassword as a real
call. Nothing under test reaches it any more, because rotation moved to
the caller, but a test that resets a real managed account's password if
the code ever moves back is not a risk worth carrying. It is now stubbed
to fail the test instead, which also states the contract.

The group-present assertion checked only that two identity SIDs were
present. A duplicated offline marker or an unrelated SID would satisfy
that while meaning something quite different, so it now pins both
positions.

Also drops a duplicated stub assignment left by the rebase.

* fix(sandbox): refuse an offline group zero does not own

ensureWindowsLocalGroup accepted NERR_GroupExists and ERROR_ALIAS_EXISTS as
success without inspecting the group it was about to reuse. Setup then resolved
that group's SID and installed it on the persistent WFP deny filters, and made
the sandbox principal a member of it.

If anything else on the machine already owns a group named ZeroSandboxOffline —
another tool, a policy, a prior unrelated convention — that is not a no-op. Every
existing member abruptly loses outbound access, because the filters now name
their group. In the other direction the sandbox principal inherits whatever
permissions that group carries, which is the opposite of what an offline
principal is for.

The add now reports its raw status and the already-exists branch verifies the
group carries this setup's managed marker before adopting it, failing with an
actionable message otherwise. A lookup error fails closed rather than adopting.

NetLocalGroupAdd and the ownership lookup sit behind seams so the branch is
reachable in tests without an elevated machine; the marker compared is the
group's own, so the principals group and the offline group cannot be confused.

Reported by jatmn on #812.

* fix(sandbox): recheck offline group membership before minting a token

Network denial does not follow from picking the offline account. The WFP block
filters match the offline GROUP'S SID, and LogonUser builds a token from the
account's real memberships — so membership is the whole enforcement, and the
command path never revalidated it.

An account that drifts out of ZeroSandboxOffline through local policy, an
administrator, or a re-setup that could not re-add it still resolves, still has
its stored secret, and still logs on. Its token no longer satisfies the filter
condition, so a NetworkDeny command gets full egress under a profile that asked
for none. The stale setup marker keeps the whole path looking healthy.

The offline role now confirms the membership its mode depends on before the
secret is read, and falls back to the restricted token when it is absent. That
direction is deliberate: the restricted token carries the offline marker the
same filters match, so egress stays blocked, and only read confinement is lost.
A failed lookup surfaces rather than downgrading silently. The online role is
not checked, since it is not in that group by design.

Reported by jatmn on #812.

* fix(sandbox): derive setup's runtime root deterministically or not at all

windowsSandboxRuntimeRootPath resolved through sandboxRuntimeRootFor, which
falls back to os.MkdirTemp when the user cache lives inside the workspace and
memoizes that only in-process.

Elevated setup is its own process. It granted the principals an ACE on temp root
A; the next command, being a new process, derived temp root B, where the
principal has no ACE, and failed ordinary cache writes with a bare ACCESS_DENIED
and nothing pointing at the sandbox. Teardown, a third process, cleaned a third
directory. The three callers that have to agree exactly could not agree at all.

Setup now uses the same side-effect-free derivation teardown already used, and
reports no runtime root when that derivation is unusable rather than inventing
one. A root only the granting process can name is worse than no root: the
principal loses the runtime tree, which is a degraded sandbox, instead of the
sandbox appearing provisioned while every command fails.

TestTeardownPathDerivationCreatesNothing asserted the opposite — that setup
"should still fall back to a usable tree" — so it is inverted here, with the
reasoning recorded in the test. That assertion encoded the assumption this
finding overturns: a per-process temp tree is not usable. Restoring the fallback
fails it with the invented path in the message, and a new
TestSetupAndTeardownDeriveTheSameRuntimeRoot pins the ordinary case, so
"report none" cannot quietly become the answer everywhere.

Reported by jatmn on #812.

* style(sandbox): separate the two doc paragraphs the rebase ran together

Adapting the ACL-record test to dual roles left #808's fail-open rationale
and #812's per-role rationale as one unbroken block. Both are worth
keeping; they are two points, not one.

* fix(sandbox): fail closed when offline-group coverage cannot be verified

The post-provisioning assertion ran inside `if groupErr == nil`, so a
failed lookup skipped it and setup carried on to install filters and
write a success marker. The comment directly above it says what that
costs: a machine reporting a successful setup while every offline
principal has an open network.

An empty SID was the same hole by a different route. Resolving to
("", nil) means the group does not exist, which is the ordinary state
before provisioning and an impossible one after it, and
WindowsNetworkPlanCoversPrincipals answers true for an empty SID
(correctly, for the pre-provisioning callers that ask it). So the
assertion passed vacuously in exactly the case where the group setup
had just created was missing.

Move the check into assertWindowsNetworkPlanCoversOfflineGroup, which
takes the resolver as a parameter and fails closed on every answer that
is not a definite yes: lookup error (wrapped, so the Win32 reason still
reaches the operator), empty SID, plan omitting the group, and a nil
resolver. Setup rolls back and exits 1 on each.

Taking the resolver as a parameter is what makes the error paths
testable, which is the regression the review asked for.

Reported by @anandh8x on #812.

* fix(sandbox): scope the offline-group assert to provisioned runs

c404ebd2 made the coverage assert reject an empty group SID, closing the
vacuous pass where a missing group counted as covered. It ran the assert
unconditionally, and the offline group is only created inside
provisionWindowsSandboxIdentity, which runs only under the
ZERO_WINDOWS_SANDBOX_IDENTITY opt-in.

So on a default machine with principals opted out, the resolver reports
("", nil) exactly as it should, and setup died with "the sandbox offline
group does not exist after provisioning" on a path that worked before
c404ebd2. The empty-SID rejection is correct after provisioning and wrong
before it.

Pass provisioned to the assert and return early when it is false, gated
at the call site on the same windowsSandboxIdentityEnabled check that
decides whether principals are provisioned at all. The fail-closed
behaviour anandh8x asked for is unchanged whenever provisioning ran.

Reported by @jatmn on #812.

* fix(sandbox): keep opted-out setup markers valid, and refuse a foreign offline group

Two of jatmn's findings on this PR.

Existing markers stay compatible (maintainer decision). The offline group is
machine-global, and the plan included its SID whenever the group existed. So the
first workspace to opt in changed the computed NetworkInfraHash for every OTHER
sandbox home on the machine, and those homes rejected their own stored markers
until each was re-run from an elevated terminal, having opted into nothing.

The inclusion is now gated on THIS home's opt-in rather than on the group
existing, so an opted-out home computes exactly the plan it computed before any
of this existed. Setup and the command path read the flag from the same
environment, so they agree. Opting in after setup does invalidate that home's
marker, which is correct: it has no principals yet.

Do not install filters for an unowned offline group (P1). The ownership check
only ran through principal provisioning, so an opt-out setup reached the resolver
and adopted any local alias carrying the name. applyWindowsNetworkPlan turns
every SID in the plan into an allowed-to-match WFP descriptor, so a foreign group
meant global deny filters against every one of ITS members: anyone with a local
group by that name loses the network for those accounts because we ran setup.
The resolver now requires the managed comment, and refuses rather than skipping,
because a plan whose filters cover no principal while setup reports success is
the failure this backend exists to prevent.

Three existing tests exercised the group path without the opt-in and now set it.
The new test asserts the other direction, that an opted-out home's hash is
unchanged when another workspace creates the group, since that is the property
the decision turns on. Verified both ways: disabling the gate fails the existing
tests, making it unconditional fails the new one.

* fix(doctor): report the principal that dual-role setup actually uses

jatmn's P2. This branch made the offline principal work under NetworkDeny, but
the doctor helper still described the old restricted-token standdown, so
`zero doctor` reported active:false and told operators reads were unconfined for
a correctly provisioned offline principal, recommending they enable network or
drop the opt-in to fix something that was not broken.

WindowsSandboxPrincipalInactiveReason is removed rather than reworded. Its only
condition was the deny-mode standdown, so after this branch it could never return
anything, and a check that cannot fire is worse than no check.

What replaced it matters more than what it said. That helper existed to be the
SINGLE rule doctor and the runtime both read, precisely so they could not drift,
and drift is what happened anyway when dual-role changed the behaviour under one
of them. WindowsSandboxPrincipalRoleForNetwork is now that shared rule:
windowsSandboxRoleForNetwork delegates to it and doctor calls it, so the reported
account and the used account cannot disagree. Doctor now names which principal a
command runs as instead of asserting a standdown.

One thing the existing tests caught. Routing the shared rule through
NormalizeNetworkMode case-folds, so "ALLOW" selected the ONLINE principal where
the runtime required an exact match and failed closed to offline. Sharing a rule
is only an improvement if it shares the stricter one, so the comparison is exact
and a test pins the casing.

* fix(sandbox): converge the dual-role branch with the rebased identity work

Rebasing #812 onto the new #808 needed real resolution rather than taking a
side, and this records what each conflict actually decided.

The account key. #808 made the principal key caller-scoped so elevated setup
provisions the account the caller will later look for. #812 derived usernames
from the workspace key alone. Every username derivation now uses the caller
scoped key, including the two inline call sites a blanket substitution missed:
the identity lookup in the unrecorded-retire path and the ledger read in
windowsPrincipalRevocationPaths. That second one is why teardown could not find
a recorded root the current policy no longer named. The setup LOCK stays keyed
to the workspace on purpose, because two users setting up one shared workspace
still write DACLs on the same paths and must serialize against each other.

The network plan stays where #812 put it, after provisioning, because the block
filters are keyed to the offline group that provisioning creates. Building it
earlier, as #808 does, would install filters naming only the marker and leave
every offline principal with an open network while looking correctly set up.

Group ownership converged on #812's implementation, not mine. #808 grew a check
hardcoded to the users group; #812 already had the general
ensureWindowsLocalGroup plus windowsLocalGroupOwnedByZero, which covers both
managed groups. The narrower version was removed and its test rewritten against
the general seams, so the users group keeps the coverage anandh8x asked for
while the offline group keeps its own.

Two functions the resolution dropped and the compiler caught:
windowsSandboxPrincipalKey and windowsCurrentUserSID. Worth naming because the
previous attempt at this convergence lost the same first function silently.

Also closes jatmn's remaining findings on this branch. The opt-out installed
check now asks about BOTH role accounts rather than one, since retiring one
while the other survives is exactly the half-done teardown an opted-out marker
must not report as success. And the post-provisioning filter-coverage assert now
resolves the offline group through the existing hook rather than the concrete
function, so it is stubbable like everything else around it.

The SensitiveEnvKeys omission jatmn reported on sandbox_exec.go arrives with the
rebase; it was fixed on #808.

* fix(sandbox): keep offline coverage and grant the fallback runtime root

Two ways the dual-role split left a workspace worse off than it looked.

The block filters are machine-global and every setup installs them by
deleting and recreating one fixed set, but the plan a home builds names
the offline group only when THAT home opted in. So an ordinary opted-out
setup for a second workspace replaced the filters without the group SID,
and the first workspace's offline principal, still in the group, still
passing the runtime membership check and still holding a valid marker,
was no longer matched by any filter. A NetworkDeny command there gained
egress silently, because the second setup did exactly what it was asked.

The gate itself is left alone, because it is load bearing for a different
reason: the plan is hashed into each home's marker, and keying it on the
group's existence made the first workspace to opt in invalidate every
other home's marker on the machine. What a home RECORDS is about its own
configuration; what setup INSTALLS is about the machine. Answering both
from one plan is what forced a choice between stale markers and a silent
hole, so WindowsNetworkPlanForApply answers the second question only, at
the apply call site, leaving the fingerprinted plan untouched.

Setup also granted the principals an ACE on the cache-derived runtime
root alone, and none at all when the cache sat inside the workspace. That
was correct when the other branch minted a random per-process directory
through MkdirTemp, but fallbackSandboxRuntimeRoot now derives its path by
hashing the workspace and creates nothing, so every process agrees on it.
Commands in that layout therefore DO select it and redirect TMP, GOCACHE
and the package caches into it, against a tree neither principal could
write. Setup now grants the same candidate set the capability plan
already covers, and creates each one, since applyWindowsACLPlan fails on
a target that does not exist.

Reverting either fix fails its regression: the opted-out plan installs
filters naming only the marker SID, and setup grants nothing while
commands write to Temp\zero\runtime\v1\<hash>.

Two existing assertions had to be inverted rather than adapted, and both
were asserting the old bug. One required setup to report NO runtime root
in the cache-inside-workspace layout; the other compared setup's single
root for equality against the command's choice. Setup covers the whole
candidate set now precisely because that choice is made per process, so
the contract is membership.

* fix(sandbox): treat an unreadable offline group as a failure, and retire the pre-split principal

Three findings from review.

A resolution failure in WindowsNetworkPlanForApply returned a marker-only plan,
and the machine's WFP filters are replaced wholesale from that plan. So an
opted-out setup whose group lookup failed transiently removed the offline group
SID another workspace's principals depend on, and that workspace's NetworkDeny
commands silently regained egress while its marker and its direct membership
check both still passed. The old reasoning was that refusing to install would
trade a partial denial for no denial; that is wrong, because the alternative to
installing is leaving the existing filters alone. It is fatal now, which is only
reachable from an opted-out home since assertWindowsNetworkPlanCoversOfflineGroup
is gated on provisioned.

Splitting the single principal into offline and online roles changed both
account names without changing the marker schema, so an installation made by the
previous version kept a valid marker, setup was never re-run, and the runner
found neither zero-sbx-d<key> nor zero-sbx-n<key> and fell back to the
restricted-token backend with no read confinement. The schema version is bumped
so that installation reports as out of date, and a legacy role derives the old
untagged name so the ordered retirement can remove the account, its secret, its
logon rights, its ACEs and its ledger. It is retired, never provisioned, and
there is a test for that because the two lists are one line apart.

Doctor reported that commands run as the selected principal whenever the marker
validated, but marker validation compares serialized plans and hashes and names
no account: deleting the account or its secret, or dropping the offline account
out of its group, leaves the marker valid while the runtime falls back or fails.
Verifying liveness needs Windows-only queries internal/doctor cannot make, so
the claim is narrowed to what the marker actually proves. The role is still
reported, since that part is derived rather than assumed.
2026-08-27 13:27:16 +05:30
Vasanthdev2004 e26e292d1a test(sandbox): compare normalized paths, not the raw input
TestACLComparablePathDoesNotResolveAReparsePoint failed on Windows CI
while passing locally. The test was wrong, not the code.

It asserted that windowsACLComparablePath returns a string equal to the
path passed in. GetLongPathName legitimately rewrites that string: a CI
runner's temp directory is an 8.3 short name, so expanding RUNNER~1 to
runneradmin produces a different string naming exactly the same object,
and the assertion failed on the one property the function is supposed to
have.

Both sides are now normalized before comparison, which states the real
property: a path THROUGH the reparse point must not normalize to the
target's normalization, or the guard would compare a redirected handle
against a redirected expectation and match itself. A second assertion
keeps the link component present so a resolution to something else
entirely still fails.

The test's documented limit is unchanged and still honest: a junction
cannot separate the old basis from the new one, because EvalSymlinks does
not resolve junctions either. It remains an invariant test.
2026-08-27 13:27:16 +05:30
Vasanthdev2004 695ef641f9 fix(sandbox): stop the redirect check resolving what it checks for
verifyWindowsACLTargetNotRedirected asks GetFinalPathNameByHandle where
the handle landed, then compared that against
canonicalSandboxWorkspaceRoot(path), which runs filepath.EvalSymlinks.
That is the same resolution the kernel had just performed, so for a
directory symlink the two sides agreed precisely BECAUSE the redirect
happened: elevated setup went on to rewrite the DACL of an object outside
the workspace while the check reported success.

Junctions were rejected, but by accident rather than by design. Go reports
a junction as ModeIrregular rather than ModeSymlink, so EvalSymlinks
refuses it, the canonicalization falls back to the lexical path, and the
mismatch surfaces. That is a property of the standard library's mode bits,
not of this guard, and a Go release that resolved mount points would
silently disarm the one case it was known to catch.

The expected side is now normalized without resolving anything.
GetLongPathName expands an 8.3 short name by reading directory entries and
does not follow a link to its target, and EqualFold still covers casing,
so the two spellings that legitimately name the same object still compare
equal. Every failure degrades to the lexically cleaned path, which can
only produce a spurious refusal, never a spurious match.

A target deliberately spelled through a symlink is now refused. That is
the intended direction: the question here is whether the object the handle
landed on is the object that was named.

On the tests, plainly: the directory-symlink regression is the one that
separates the old basis from the new, and it needs Developer Mode or
SeCreateSymbolicLinkPrivilege, so it skips on a machine without either and
skipped on mine. The junction test beside it passes against both bases and
says so in its own comment; it is an invariant test guarding the Go
behaviour the old code accidentally depended on, not a regression for this
change.
2026-08-27 13:27:15 +05:30
Vasanthdev2004 aecd614305 fix(sandbox): drop the unfollowable sandbox override from ACL failures
Both unelevated ACL failures told the reader to re-run with `--sandbox
forbid`. There is no such option: SandboxPreferenceForbid is an internal
engine state with no flag behind it, so acting on it produced an unknown
option and left them stuck on the failure they had just been told how to
clear. Advice that does not work costs more than none, because finding
that out takes the reader's time.

Name the real way out instead, the user config key, which is honored
from global config only so a cloned repo cannot set it. The
elevated-setup remedy beside it was already correct and stays.

Reported by jatmn against the same string on #640. It predates this
branch, having arrived with the unelevated fallback tier in #427, and
the copy on #886 is fixed separately in 1b304e13.

Also covers the secret write with the junction regression it was owed:
the caller owns the sandbox home, so they can put a reparse point where
the secret directory is expected, and the pathname version followed it
in an elevated process. The test asserts the refusal names the reparse
point and that nothing survives on the far side, since refusing while
still creating the file would leave the caller holding it.
2026-08-27 13:27:15 +05:30
Vasanthdev2004 7cad854012 fix(sandbox): stop the principal's environment naming the caller
The child environment starts as the invoking user's, and the deliberate
sandbox redirects only replace HOME, the temp variables and the per-tool
cache dirs. Everything identifying the account survived, so a command
running as the principal read USERPROFILE, APPDATA, LOCALAPPDATA,
HOMEDRIVE, HOMEPATH, USERNAME and USERDOMAIN describing the CALLER,
whose profile the principal deliberately cannot open. Native tools
resolve per-user state through exactly those, so they fail during
startup or quietly look somewhere they have no business reading.

Point them into the sandbox runtime tree, which is already granted to
the principal and already holds its caches, so the paths are writable by
construction. Naming the real Windows profile would need LoadUserProfile
to have run, and a variable pointing at a directory that does not exist
yet is a worse answer than one pointing somewhere usable.

Layered under the deliberate redirects rather than over them:
sandboxRuntimeEnvironment stays the single owner of HOME, TMPDIR, TMP
and TEMP, and a regression pins that so the two cannot drift into
setting the same variables from two places.

This is the environment half of the finding. Loading the principal's
profile and known folders is left until the launch mechanism is settled,
since LOGON_WITH_PROFILE would do it as a side effect.
2026-08-27 13:27:15 +05:30
Vasanthdev2004 1efc1aff4b fix(sandbox): report the privileges a principal launch actually needs
CreateProcessAsUser exempts only a restricted version of the caller's
own primary token from SE_ASSIGNPRIMARYTOKEN_NAME, which is precisely
why the ordinary restricted-token path works while holding nothing
special. A principal token comes from LogonUser against a separate local
account, so the exemption does not apply and both that privilege and
SE_INCREASE_QUOTA_NAME are required. Nothing enabled or checked either,
and a token measured on an ordinary unelevated process holds neither, so
the failure arrived as a bare "Access is denied" from inside process
creation, before the command's executable was ever opened, and read as
the command being rejected.

Enable them where they are held, since present-but-disabled still fails
the access check and that is where an elevated administrator lands, and
refuse with the specific names and a way out where they are not.

Detected by ENUMERATING the token. AdjustTokenPrivileges reports an
unheld privilege by returning success with ERROR_NOT_ALL_ASSIGNED, which
this binding does not surface: it returns nil for SeTcbPrivilege on an
ordinary process. A check built on its error passed everywhere, which is
worse than no check, since it would call the sandbox ready in exactly
the case it cannot run.

This does not make the principal launchable. It makes the reason legible
while the launch mechanism itself is settled.
2026-08-27 13:27:15 +05:30
Vasanthdev2004 ab76cfeb1e fix(sandbox): write the principal secret through one pinned handle
The elevated secret write resolved its path four separate times:
MkdirAll, an O_TRUNC open, SetNamedSecurityInfo by name, then WriteFile
by name. The sandbox home belongs to the invoking user, who is the party
this sandbox contains, so each resolution was a place to swap a
component. A symlink leaf lets an Administrator truncate a file of the
caller's choosing and then rewrite its DACL; a junction alone is enough
to plant the deterministic secret somewhere the caller controls.

Create the leaf relative to a pinned no-follow parent handle, refuse it
if it is a reparse point, and apply both the DACL and the bytes to that
handle. The name is never resolved again after the create.

The payload is now sealed before the file exists, so a failure there
leaves nothing on disk rather than an empty file for someone to race.
Cleanup on failure stays by name, which is safe in the direction that
matters: at worst it misses and leaves a locked-down file, never deletes
something it did not create.
2026-08-27 13:27:15 +05:30
Vasanthdev2004 3944704131 fix(sandbox): protect the gitdir pointer in a linked worktree
A linked worktree or submodule has .git as a FILE holding a `gitdir:`
pointer, not a directory. The principal plan names .git/config and
.git/hooks and materializes both, and the Windows materializer gets
there by descending through .git as a directory. A regular file cannot
have children, so opted-in elevated setup aborted and the sandbox could
not be used in a worktree at all. Zero's own development worktrees are
this shape, which is how it went unnoticed.

Deny the pointer file itself there instead. That is the stronger
protection rather than a fallback: a principal able to rewrite `gitdir:`
repoints the repository at a control directory of its choosing, which
subsumes editing config or planting a hook. The real control directory
sits outside the write root, so nothing is inherited there and no
carveout is needed.

Decided by an Lstat rather than a lexical guess, since the layout is a
property of the checkout. An absent .git keeps the directory-shaped
carveouts so they are still created before git first runs.
2026-08-27 13:27:15 +05:30
Vasanthdev2004 1e29354a0e test(sandbox): fix the workspace-root literal in the setup-args tests
A bulk edit dropped the separator, leaving the drive-relative
`C:workspace` where `C:\workspace` was meant. Windows resolves the
first well enough that the two halves still agreed; on Linux a backslash
is an ordinary character, so the command half derived no runtime
candidates at all and the marker carried two the command never saw.

The setup half no longer pre-augments either: BuildWindowsSandboxSetupArgs
folds the runtime roots in itself, and passing them in hid that from the
one test that covers it.
2026-08-27 13:27:15 +05:30
Vasanthdev2004 51976966aa fix(sandbox): give the principal's read roots a restricting SID
A profile with DenyRead drops WRITE_RESTRICTED, so Windows runs the
restricted-SID check over reads as well as writes. Read roots were
granted only to the principal's account SID, and the write jail removes
that SID from the restricting set on purpose, so reads passed the normal
check and matched nothing on the restricted one. The result was not a
narrower sandbox but an unusable one: no read root readable, including
the executable the command was trying to start.

Mint a read capability SID, grant it on every read root in the
capability plan, and carry it in the strict token's restricting set, so
the read allow-list and the restriction come from one value instead of
two that can drift.

Deny it on every DenyRead path too. The read roots begin at the
filesystem root, so without that the carveouts stay readable through the
new grant and the deny list stops meaning anything, which is the whole
reason the strict token is chosen. The no-write-roots case keeps its
ReadOnly deny as well: both SIDs the token can carry must be denied, not
only the newest.

Gated on DenyRead, which is what selects the strict token. Elsewhere
reads never reach the restricted check and the grant would be ACEs on
the filesystem root that buy nothing. Both halves decide from the
profile alone, so they cannot disagree about whether the entries exist.
2026-08-27 13:27:14 +05:30
Vasanthdev2004 1dd1709bff fix(sandbox): derive the runtime roots where TEMP is the operator's
Every sandboxed command failed marker validation with two plans of the
same size and different hashes:

  marker ...\windows-setup.json has 12 entries, hash 76fda2032e66;
  this command expects 12 entries, hash 4be1dbf8b642

The temp-derived runtime candidate reads os.TempDir(), and the sandbox
points TMPDIR, TMP and TEMP at its own runtime temp for everything it
launches. The command runner inherits that env, so when it derived the
candidate set it produced a root under the runtime tree while setup,
whose TEMP is untouched, produced one under the real temp. Same count,
different path, and no command could run.

A fingerprint both halves compare cannot be a function of the caller's
environment. Derive it only where TEMP is still the operator's: the
setup args builder, the Windows command plan, and doctor. The runner now
takes the profile it is handed, and commandConfig no longer re-derives
on behalf of whoever happens to call it.

The regression settles the profile first and redirects TEMP afterwards,
in that order, so it reproduces the mismatch against the old behaviour.
2026-08-27 13:27:14 +05:30
Vasanthdev2004 622371f73f fix(sandbox): say what changed when the setup marker is rejected
"permission roots or deny lists changed" told an operator that every
sandboxed command would now refuse to run, and nothing else. Not which
side is stale, not by how much, not even whether the marker belongs to
this workspace. Debugging it meant reading the source and guessing,
which is what happened.

Report the marker path, both entry counts and both hashes. The counts
separate the two shapes this takes: equal counts mean the same roots
spelled differently, unequal counts mean one side has roots the other
has never heard of.
2026-08-27 13:27:14 +05:30
Vasanthdev2004 2d755dba96 fix(sandbox): create the runtime roots setup grants
Elevated setup stopped working entirely:

  zero-windows-sandbox-setup.exe: windows ACL target does not exist:
  ...\AppData\Local\Temp\zero\runtime\v1\67b1b01412f588b9

Putting both runtime candidates into the capability plan added a write
root that nothing creates. The capability plan deliberately refuses to
materialize a write root, since an absent path is a typo or a stale
config and inventing the tree would grant write on a directory nobody
asked for, so the whole run fails on a path that is merely missing. Only
the selected root was ever created, and only on the principal path.

Create every candidate on the setup side, before the plan that grants
them is built. The regression walks the plan and fails on any granted
write root that does not exist, so the two halves cannot drift apart
again: today one function chooses the candidates and another creates
them, and nothing else couples them.

Found by the elevated end-to-end run, which is the only thing that
executes this path. No unit test reached it and CI does not run it.
2026-08-27 13:27:14 +05:30
Vasanthdev2004 16d7512e2a test(sandbox): make the jail aliasing assertion able to fail
The aliasing test appended a sentinel to the returned jail and then
ranged over the caller's slice looking for it. That can never fail:
append returns a new header, so the caller's length never grows and the
range never reaches the slot the write landed in. It passed against an
implementation that returns the caller's slice verbatim, which is the
exact thing it claims to rule out. Lint noticed the symptom as an
ineffectual assignment.

Assert through the backing array instead: element storage must not be
shared, and an append must not write into the caller's spare capacity.
Both fail against an aliasing implementation.

Also drop windowsSandboxDeterministicRuntimeRootPath, which lost its
last caller to the shared runtime-candidate helper and duplicated its
derivation.
2026-08-27 13:27:14 +05:30
Vasanthdev2004 c736616526 fix(sandbox): cover both runtime roots in the setup contract
A fresh setup marker rejected the very command it had just been written
for. Engine.run augments the profile with the selected runtime root
before the Windows runner sees it, but setup fingerprinted the bare
profile, so the extra write root changed the ACL plan hash and every
command on a restricted filesystem failed with "permission roots or
deny lists changed".

The runtime root also reached the principal ACL plan only. A principal
command runs on a WRITE_RESTRICTED token, where a write needs the
normal check and the restricting-SID check to both pass, so a root
carrying just the account ACE was still unwritable once the marker
agreed.

Derive the runtime candidates once and present them on both sides of
the setup protocol. Setup covers the cache-derived root and the
temp-derived fallback rather than whichever one it happened to select,
since selection is per process and a command that fell back would
otherwise land on an unprovisioned tree. Both are pure functions of the
workspace root, so setup can cover the set and a later process can
select from it.

Regressions fail without this: the marker one reproduces the exact
rejection, the capability-plan one shows the missing restricting-SID
grant.
2026-08-27 13:27:14 +05:30
Vasanthdev2004 1826326ed0 chore: drop a stray probe artifact from the branch
probe-inside.txt is a leftover from the elevated end-to-end probe and was never
meant to be committed. A `git add -A` in the write-jail commit swept it in.

It is what the automated review's diff-hygiene check was failing on: the file
carries trailing whitespace, so `git diff --check` reported a blocker while
tests, build and smoke all passed.
2026-08-27 13:27:13 +05:30
Vasanthdev2004 c981c3cb9a fix(sandbox): derive the fallback runtime root instead of minting one per process
Reported by jatmn. sandboxRuntimeRootFor falls back to a private tree when the
cache-derived runtime root would land inside the workspace, and that fallback
called os.MkdirTemp and cached the answer in a process-global map.

Separate processes therefore got separate answers. Elevated setup granted the
sandbox principal write access to the directory it created, and every later
__windows-command-runner process created a different one and pointed TMP,
GOCACHE, npm and the rest at it. Those directories are made by the calling user
and carry no ACE for the principal, so ordinary cache and temp writes failed
with a bare ACCESS_DENIED and nothing naming the sandbox. sandboxRuntimeRootFor
already documented that both callers must agree exactly; the fallback was the
branch where that could not hold.

It now hashes the workspace under os.TempDir, the same shape as the
cache-derived root, so every process reaches the same path with no shared state.
When even that lands inside the workspace it returns an error rather than
picking somewhere arbitrary: a runtime tree governed by the workspace's own
policy makes the sandbox's cache writes indistinguishable from the work it is
confining.

Two consequences worth naming.

It creates nothing now, so the split between deterministicSandboxRuntimeRoot and
the resolver is no longer about avoiding a side effect. The comments that
justified that split on those grounds were rewritten rather than left describing
behaviour the code no longer has.

Teardown can name the fallback tree for the first time. windowsPrincipalTeardown
Paths used the deterministic resolver precisely because the fallback was random
and unnameable, which meant an opted-out machine kept principal ACEs on whatever
tree the fallback had produced. It now goes through the shared resolver and
revokes what commands actually used.

Tests pin the property the defect turned on: a repeated call, which is what a
second process looks like with no shared state, must return the same root. The
old implementation could not have passed that, since only the in-process map
made repeat calls agree. Also pinned: per-workspace separation, that naming the
tree creates nothing so teardown leaves no directory behind, and that the root
stays outside the workspace.
2026-08-27 13:27:13 +05:30
Vasanthdev2004 c914d762fc fix(sandbox): stop the principal's own SID from defeating its write jail
Reported by jatmn. The principal path appended the account's own SID to the
restricting-SID list before building a WRITE_RESTRICTED token.

That token allows a write only when BOTH the normal token and the restricting
list allow it. The account SID is already enabled in the normal token, so
listing it as a restricting SID makes the second check a formality for anything
granted to that account: every path carrying a direct principal ACE passes both
halves and is writable wherever it sits. The principal's own profile directory,
which Windows creates on first logon with exactly such an ACE, is outside every
configured write root and was writable for that reason.

It is the same defect as the World SID in the restricted list (#865), with the
account SID in place of Everyone. A SID already carried by the normal token
cannot also restrict it.

The previous reasoning, recorded in the comment this replaces, was that the ACL
plan grants the workspace to the principal SID so removing it would jail the
principal out of its own tree. That is not so, because setup applies
BuildWindowsACLPlan on every path, not only the restricted-token one, so each
configured write root already carries a capability ACE as well as the principal
ACE. Confining to the capability SIDs therefore restores the intersection the
jail is supposed to be: a write root satisfies the normal token through its
principal ACE and the restriction through its capability ACE, while a path
holding only a principal ACE now fails the restricted check.

The account SID is passed to windowsPrincipalJailSIDs and excluded there rather
than simply not passed. Naming it makes the exclusion the function's contract
instead of an omission a later edit could undo silently, and it also strips the
SID should it ever arrive through the capability list, which is the route the
World SID took.

On the test jatmn asked for: the existing jail test grants WinBuiltinGuestsSid,
a GROUP, so it exercised a configuration the product never ships and could not
have caught this. The new test plants the account SID INSIDE the capability list
and asserts it is gone, which is deliberately falsifiable: a list that never
contained it would pass against any implementation, including one that appends
the SID straight back. Verified by mutation, disabling the filter fails it with
the reported symptom. A second test pins that the capability SIDs survive, so
the fix cannot degenerate into jailing the principal out of its workspace.

Still outstanding from the same review and not addressed here: the runtime-root
fallback that hands setup and each command runner a different directory, and the
elevated end-to-end evidence.
2026-08-27 13:27:13 +05:30
Vasanthdev2004 67f3d6145a fix(sandbox): refuse to adopt a ZeroSandboxUsers group Zero did not create
Reported by anandh8x as P2 #3. ensureWindowsSandboxGroup treated NERR_GroupExists
and ERROR_ALIAS_EXISTS as plain success, so any local group that happened to
carry our name was adopted. Its members, and every grant already keyed to it,
silently became part of the sandbox's identity.

A name is not proof of provenance, which is the same reasoning
windowsSandboxUserIsManaged already applies to an ACCOUNT of our name. The group
half was missing. An unprivileged user cannot create a local group, but an
administrator, an installer or an earlier build can, and the principal would
then inherit whatever it grants.

An existing group is now adopted only when it carries the managed comment, read
back with NetLocalGroupGetInfo. Anything else is refused by name rather than
adopted, renamed around or deleted: removing somebody else's group would be
destructive, and provisioning into it is the hole being closed.

The decision is split from the syscall into resolveWindowsSandboxGroupAdd so it
can be tested without Administrator and without leaving a real local group on
the machine running the suite. Tests cover both "already exists" statuses,
refusal for a foreign group, adoption of our own so re-running setup still
converges, no ownership probe when we just created it ourselves, an unreadable
probe surfacing rather than being guessed either way, and a real API failure
still failing.

Also rebased onto main, 17 commits behind, under the fresh-base rule. All 51
commits replayed with no conflicts.
2026-08-27 13:27:13 +05:30
Vasanthdev2004 0fda33eee2 fix(sandbox): fail opt-out only when the principal actually survived
Three corrections, the first to my own previous commit.

Making the opt-out retirement failure fatal was too broad.
removeWindowsSandboxPrincipalForSetup deliberately continues past a secret file
owned by another administrator's setup, reporting it at the end through
errors.Join, so that one unlinkable file cannot strand the account, its logon
rights and its ACEs. The account is already gone when that error is returned.
Treating it as fatal made `zero sandbox setup` fail permanently on such a
machine and rolled back its ACL work, taking the DEFAULT restricted-token
sandbox down over inert residue.

The rule is now the narrower one that was always meant: fatal when the ACCOUNT
survived, because that is exactly what makes an opted-out marker a lie. Asked of
the account rather than inferred from the error, since the error is a join whose
parts are not separable at the call site, and the lookup fails closed so an
undeterminable state counts as still installed. A retirement that left something
inert behind still completes, and still names what it left.

Second, assertWindowsSetupRunsAsCaller cannot fire on any path Zero takes, and
said otherwise. Zero never elevates the helper: runSandboxSetupHelper is a plain
exec.Command and runWindowsSandboxSetup refuses unless already elevated, so the
operator supplies elevation by opening an elevated terminal and BOTH halves run
there. The caller SID is resolved from the process that becomes the helper, so
the two are equal by construction. The guard is kept, because it is a
correctness assertion that becomes load-bearing the moment a ShellExecute
"runas" path is added and already covers a helper invoked directly, but its doc
now says plainly what it does not do. The residual hazard has a different shape
and is loud: elevating as another administrator provisions into that account's
sandbox home, and the operator's own session finds no marker and is told to run
setup.

Third, parseSandboxExecArgs claimed to scan for the separator up front. Every
branch of its loop body returned, so it only ever examined index 0 and the scan
did not exist. Rewritten as the straight-line decision it actually was, which is
also the rule we want: only the first token is the wrapper's.

sandbox exec also built its engine without SensitiveEnvKeys, so a key named by
apiKeyEnv in the user's config was scrubbed for every sandboxed tool call and
handed to the one command whose purpose is to reproduce that environment.

Tests: the opt-out branch now covers both sides of the rule, and
parseSandboxExecArgs gets the coverage it never had, including the reported
`-- cmd --help` case and an invariant that the returned command is always a
trailing slice of the input.
2026-08-27 13:27:13 +05:30
Vasanthdev2004 d608a3c087 fix(sandbox): keep elevated Windows setup tied to the caller's identity
Two failures on the elevated setup path, both reported by anandh8x.

The first is caller identity. windowsSandboxPrincipalKey sampled the CURRENT
process token, and elevated setup is not the caller's process: under
over-the-shoulder UAC or `runas` it belongs to whichever administrator typed the
credentials. Setup therefore provisioned an account, a secret, an ACL ledger and
a set of grants named after that administrator, while every later command
derived the ordinary user's key, found nothing, and fell back to the restricted
token. The admin-keyed leftovers were unreachable and nothing would ever reclaim
them.

The caller's SID now crosses the UAC boundary in the setup args, exactly as the
principal opt-in already does and for exactly the same reason, and the key is
derived from it. That fixes everything merely NAMED after the invoking user.

It cannot fix the secret. CryptProtectData derives its key from the user that
stores the blob, so a password sealed by another administrator could not be
unsealed by the caller no matter how the account were named. Provisioning is
refused up front in that case, before an account or a grant exists, with a
message that says how to clear it. The refusal is scoped to the principal
opt-in: the default restricted-token sandbox stores no secret and works fine
across that boundary.

The second is the opt-out path. Retiring an existing principal could fail, and
setup printed the error and carried on to write an opted-out marker and exit 0.
Teardown being idempotent was the argument for continuing, but it is the reason
this must fail instead: a machine with nothing to retire passes straight
through, so an error here means a real account, secret, logon right, ACE set and
ledger are still installed, and the marker now claims none of them exist. Setup
returns non-zero and rolls back its ACL work.

runWindowsSandboxSetup had no coverage at all, since every step wants an
elevated token, a machine-global mutex, real DACLs, the WFP engine or a local
account. The external effects are now seams in the idiom the package already
used for two of them, and the tests drive the step order and each failure path,
including the marker never being written.
2026-08-27 13:27:13 +05:30
Vasanthdev2004 09cc534e92 fix(sandbox): address jatmn's review on the principal backend and sandbox exec
Four of the five findings were real. Taking them in order of what they cost.

Cross-user principal collision (P1). The account name was keyed on the workspace
alone while its DPAPI secret and ACL ledger both live under the invoking user's
sandbox home, so two Windows users sharing one workspace derived the same account
with private bookkeeping. The second user found no ledger of their own and either
retired the first user's working principal or adopted it and rotated its password
while storing only their own secret; either way the first user's marker still
validated and their next command could not log on.

windowsSandboxPrincipalKey now folds the invoking user's SID into the key. The
setup LOCK deliberately keeps the workspace-only key: two users setting up one
shared workspace still write DACLs on the same paths, so they must keep
serializing even though they now provision separate accounts. Splitting the two
keys is the point, and there is a test asserting they differ.

Ledger not restored on rollback (P1). The narrowed ledger is written only after
everything succeeds, but the rollback closure restored the old ACEs and left it
narrowed. A later setup or teardown reads the ledger to decide what to revoke, so
those restored paths would never be revisited: access outside the current policy
held by an account nothing knows to clean up. The closure now writes back the
union recorded before any DACL changed. Over-recording is the safe direction.

sandbox exec skipped the production planning path (P2). It built a SandboxManager
directly, so it never reached prepareSandboxRuntime and ran without the runtime
write root that a real sandboxed command gets. It now goes through
Engine.BuildCommandPlan. This matters beyond tidiness: the command exists to
prove enforcement, and it was proving a path no tool call takes.

Plan resources leaked (P2). No plan.Cleanup(), so every invocation left a
/tmp/zero-sandbox-report-* behind on Linux. Deferred immediately after the build.

The `--` separator (P2). The reported case, `exec -- cmd --help`, already worked:
the loop hits the separator first and returns. The real break was the tolerated
separator-less form, where `exec mycmd --help` printed Zero's help and never ran
mycmd. The separator is now located before any help flag is interpreted, and help
is only read from the wrapper's own arguments.

Two existing ledger tests seeded state under a username derived from the
workspace key and had to move to the principal key. Worth noting for anyone
running this branch: an account provisioned before this commit will not be found
by the new key, and setup will retire it as unrecorded. That is correct behaviour
for an unmerged branch with no released accounts, but it is not a no-op locally.
2026-08-27 13:27:13 +05:30
Vasanthdev2004 ff077553bf fix(sandbox): serialize elevated setup with a per-workspace lock
anandh8x's finding #3, the last of the twelve.

Setup is a transaction, not a write. It rotates the principal's password,
stores the secret, read-modify-writes the ACL ledger, installs WFP filters and
writes the marker. Two setups for the same workspace at once interleave those
steps: the account ends up with one process's password while the stored secret
is the other's, so every later command fails to log on, and the ledger's
read-modify-write silently loses a path set so ACEs that should have been
revoked stay on disk with nothing recording them.

Atomic individual writes do not address this. Each step being all-or-nothing is
orthogonal to two processes interleaving BETWEEN steps, which is the actual
failure, so the lock spans from the elevation check through the marker write.

Nothing to port here: the reference implementation has the identical bug. Its
password rotation and secret write hold no lock, its named mutex guards only an
unrelated read-ACL-only mode and is machine-singleton, and the singleflight
that looks like a fix is process-local.

Decisions worth the reviewer's attention.

Global rather than Local namespace, because the protected state is not
session-scoped: the local account, its LSA logon rights and the WFP filters are
machine state, so setups in different logon sessions must still exclude each
other. Per-workspace rather than machine-wide, because every artifact the lock
protects is per-workspace and a global lock would serialize unrelated setups.

The DACL is Administrators and SYSTEM only, protected. A sandbox principal able
to open this object could hold it and stall every future setup, or create it
first and squat the name so real setup believes it holds a lock it does not.

The OS thread is pinned. Win32 mutex ownership is thread-affine, so a goroutine
that migrated between the wait and the release would call ReleaseMutex from a
thread that does not own the object, leaving the mutex held until process exit,
which is a deadlock for every later setup on the machine.

WAIT_ABANDONED is treated as acquired and reported. The previous holder died
inside the transaction, so the machine may be half set up; setup is idempotent
and re-running it is the repair, whereas refusing would leave no way forward.
A bounded wait rather than skip-and-succeed, because skipping here means not
setting the sandbox up at all.

On the tests: the production DACL means an unelevated process cannot open an
existing lock, so the contention test could not reach the mutex wait anywhere
that matters, including CI. The SDDL is therefore a var the wait tests relax
for themselves, and a separate test pins the production value so loosening it
takes an explicit change. Writing the test first is what surfaced this at all.
2026-08-27 13:27:12 +05:30
Vasanthdev2004 84709257a8 fix(sandbox): retire the principal on opt-out, and finish teardown past a foreign secret
anandh8x's findings #6 and #8. They are one commit because #6 calls straight
into the path #8 breaks, so fixing #6 alone would have shipped an opt-out that
aborts on exactly the machines #8 describes.

#8 first. readWindowsSandboxSecret already treats permission-denied as
unavailability rather than breakage, and documents why: an operator who
elevated with a separate administrative account ends up with a secret their
ordinary account cannot open. Removal did not agree. os.Remove returning a
permission error aborted teardown at its very first step, so the account, its
logon rights, its ACEs and its ledger all stayed installed because one file
could not be unlinked. The operator asked for removal, got an error, and kept a
working principal.

Removal now classifies that case distinctly and teardown carries on, reporting
the leftover secret at the end rather than swallowing it. An ordinary removal
failure is still fatal, and there is a test for the difference: misclassifying
one as the other would turn a real failure into a shrug.

#6. Opting out has to actually retire the principal, because that is what we
tell people it does. ValidateWindowsSandboxSetupMarker sends an operator here
in as many words. There was no opt-out branch at all, so the marker flipped to
opted-out while everything stayed exactly where it was: the instruction was a
lie, and the leftovers were invisible, because nothing afterwards looks for a
principal it believes was never provisioned.

Setup now runs teardown on the opt-out path. Not fatal if it does not complete:
teardown is idempotent and a machine that never had a principal passes straight
through, so a failure means genuine residue rather than a missing account. Say
so and finish a setup whose sandbox is otherwise fine.
2026-08-27 13:27:12 +05:30
Vasanthdev2004 4e1f4831de fix(sandbox): report an inactive principal in doctor, and finish the comment upgrade
anandh8x's findings #7 and #10.

#7. windowsSandboxPrincipalEligible stands the principal down under
network-deny, which is the DEFAULT policy, so an operator who set the opt-in to
confine reads silently gets the same-user restricted token and no read
confinement at all. Nothing told them. The runner cannot: it is re-exec'd per
command, so the notice would land on the stderr of essentially every tool call,
and it is not per-command actionable anyway. The setup marker validates happily
because setup really did provision the account. The runtime comment already
said this belonged on `zero doctor`, and doctor never carried it.

Doctor now warns, and the rule has one definition rather than two. The
eligibility check asks the same exported predicate doctor does, so the two
cannot drift into disagreeing about whether a principal is in play. Drift here
would mean doctor telling someone reads are confined while commands run on the
restricted token, which is worse than saying nothing.

A warning rather than a failure: commands work and the network is still
enforced. What is wrong is the operator's picture of what they have.

#10. windowsSandboxUserIsManaged has always claimed a legacy account is
"adopted and its comment rewritten on the way through", and it never was:
provisioning only ever set the password (USER_INFO_1003 is documented in this
file as the password-only form). So a pre-key account stayed permanently
unattributable, and every later run had to keep accepting the bare comment to
avoid orphaning it.

Adoption now stamps the workspace key via USER_INFO_1007, which ends that and
lets the bare form eventually be retired rather than accepted forever. Best
effort: the account is adopted and working either way, and failing provisioning
over a cosmetic stamp would strand a usable sandbox for bookkeeping. Reported
rather than swallowed, so a run that keeps re-adopting the same legacy account
is visible.

The legacy-comment probe is a separate function rather than folded into
windowsSandboxUserIsManaged, so that keeps its (bool, error) shape and the test
seam around it does not have to change.
2026-08-27 13:27:12 +05:30
Vasanthdev2004 8bb5546b49 fix(sandbox): fingerprint principal grants in the setup marker
anandh8x's finding #4. The marker hashed BuildWindowsACLPlan, the
capability-SID plan, while principal grants are built separately by
buildWindowsPrincipalACLPlan from the same profile. So the marker was blind to
them: narrowing or removing a principal read root left setup looking current
and the old AllowRead ACE sitting on disk. Narrowing a policy has to be able
to take access away, and this was the one path where it could not.

The fingerprint hashes the plan with a fixed placeholder trustee rather than
the real principal SID. The account is recreated with a fresh SID whenever it
is reprovisioned, so hashing the real one would move the fingerprint on every
rebuild even when the granted paths were identical, and every command would
then rerun setup. What must invalidate the marker is the set of paths and
actions, which is what this captures.

Empty when the principal backend is not opted into, so the default install's
marker does not churn. Schema version goes 5 to 6 so existing markers are
treated as stale rather than read as having no principal grants.

Validation refuses a mismatch with the action to take, since the remedy is
re-running elevated setup so the stale grants are actually revoked rather than
merely re-planned.
2026-08-27 13:27:12 +05:30
Vasanthdev2004 c11e350d2c feat(sandbox): add zero sandbox exec to run one command through the sandbox
Until now the sandbox could only be exercised through a full agent turn with a
model in the loop. `zero sandbox policy` reports the posture it would take and
`zero sandbox check` evaluates a hypothetical decision, but nothing ran a
command and let anyone look at what happened on disk afterwards.

That is the root of the verification gap on this branch. Enforcement is covered
almost entirely by tests asserting the shape of an ACL plan, and almost not at
all by tests asserting that a write was refused. The two are not the same
thing, and this branch has already produced the proof: the .git rename guard
was emitted correctly by the planner and silently dropped by the applier, and
four tests covering the plan passed while the ACE was absent from the
directory.

It goes through SandboxManager.BuildCommandPlan, the same path a shell tool
takes, so what it demonstrates is what users get rather than what a test double
does. Backend, enforcement level and workspace are written to stderr before the
command runs, and a downgrade is printed explicitly, so a harness can assert
the sandbox was actually engaged instead of passing because it quietly stood
down. Exit status is the command's own, which a harness asserting a refusal
needs.

Everything after `--` is the command, so its flags are never parsed as ours.

Verified by hand on Windows: a write inside the workspace succeeds and the file
exists; the same write to C:\Windows\Temp fails with UnauthorizedAccessException
and the file is not created. Both halves matter, since a command can fail for
an unrelated reason while the write still lands.

This is the prerequisite for a real smoke harness, which is the actual goal.
2026-08-27 13:27:12 +05:30
Vasanthdev2004 3cb40ce849 fix(sandbox): name the root that blocks unelevated ACL setup
Found while testing the default sandbox path, not the principal path this PR
is about. One write root that the current user cannot re-DACL fails the entire
unelevated ACL plan, and the success marker is only recorded on success, so
the identical failure repeats on every later command until that root leaves
the plan. The workspace is effectively unusable in the meantime.

Refusing to run is correct and is left alone: without those ACEs there is no
write jail, so continuing would run a command that believes it is sandboxed
and is not. The defect was the diagnosis. Every failure got the same guess,
"the workspace may be on a filesystem the current user does not own", and
recommended elevated setup, which does nothing when the real problem is a
system directory sitting in the root set.

An access denial now names the exact path, says the sandbox cannot enforce a
boundary there, points at TEMP and TMP as the usual way such a path gets in,
and states plainly that elevated setup will NOT help. Other failures keep the
old message.

The extractor reads the path back out of an error string produced two
functions away, which the compiler cannot check, so a test drives the real
producer rather than hand-writing the message: if openWindowsACLTarget rewords
its error the test fails instead of the diagnostic quietly going blank. That
test earned itself immediately by catching the first version splitting on the
first colon and returning "C", since on Windows the path opens with a drive
colon. It splits on colon-space now, which a drive colon can never be.

The helper sits behind the Windows build tag beside its only caller, for the
same reason windowsACLPlanPaths was moved earlier in this branch.
2026-08-27 13:27:12 +05:30
Vasanthdev2004 b86bfc17e9 docs(sandbox): attribute the Users-group premise the volume-root fix relies on
Cross-checking against another Windows sandbox implementation showed it joins
its account to the built-in Users group explicitly, and then verifies at apply
time whether Users already hold read before granting. Ours gets that membership
implicitly from NetUserAdd with USER_PRIV_USER and asserts the consequence in a
comment. Same conclusion, weaker footing, so the comment now says where the
membership comes from and that a hardened image which strips Users read would
need an explicit bounded read set instead.
2026-08-27 13:27:12 +05:30
Vasanthdev2004 37629a2fa5 fix(sandbox): stop granting a principal read at the volume root, and report failed ACE revocation
Two of anandh8x's findings on this PR.

The read grant. permissionProfileReadRoots seeds its list with the bare
separator, because the workspace-write posture is read-all with a write jail.
That costs nothing for the capability backend, whose child runs as the caller
and could read those paths anyway. For a principal it became a real,
persistent, inheritable allow-read ACE for a separate local account at the root
of the current drive, reaching every directory that does not block inheritance.
Verified against the shipped profile rather than argued: with the filter
removed the plan emits AllowRead on the drive root.

Dropping it does not remove the reads a principal needs. It is a member of
Users, and the machine's own ACLs already grant Users read on the system and
program directories. What the grant added was read access to the places Users
are deliberately kept out of, which is the opposite of what a sandbox is for.
The volume root is detected structurally, since filepath.Dir of a root is that
same root, so drive-qualified paths, a bare separator and UNC roots are all
covered on either build host.

The teardown. removeWindowsSandboxPrincipalForSetup discarded the result of
revokeWindowsPrincipalACEs, deleted the account, deleted the ledger and
returned success. A failed revocation therefore left ACEs naming that SID on
the user's tree while the only record of which paths they sat on was removed
moments later: residue nothing could find again.

The existing reasoning for not failing hard was right and is kept, since
refusing to remove the account would strand the principal and its logon rights
permanently. So teardown still completes, but the error is now remembered, the
ledger is kept when revocation failed, and the returned error says the ledger
was kept. A record that outlives its principal is a smaller problem than
unfindable residue.

The new read-root test is built from the production profile, and skips loudly
rather than passing silently if that profile ever stops carrying a volume root.
It also asserts the workspace itself stays reachable, so the fix cannot trade a
real grant for a broken sandbox.
2026-08-27 13:27:11 +05:30
Vasanthdev2004 279c7bb6e8 fix(sandbox): apply the .git rename guard on a workspace that had no .git
Found by gnanam1990 and anandh8x independently, and confirmed on disk before
fixing: the deny-delete ACE that stops a principal replacing .git was silently
absent on exactly the workspace shape that matters most.

.git deliberately carries no Materialize, because creating an empty one breaks
`git init`. Groups are applied in ascending path order, so the .git group ran
while .git did not yet exist, was skipped as a non-materializing missing
target, and nothing revisited it when the .git\config carveout created .git as
its parent chain moments later. windowsACLGroupRequiresExistingTarget did not
catch it either, since it only treats AllowWrite as requiring an existing
target. Net effect: the principal could rename .git aside and recreate it
without the config and hooks carveouts, which is the escape the guard exists
to stop.

Groups skipped for a missing target are now retried once after the pass, since
a later group can create what an earlier one needed. Groups that are genuinely
absent simply skip again. Ordering is otherwise untouched, so the reverse-order
unwind that rollback depends on still holds.

Fixing the gate to demand an existing target for deny-delete would have been
wrong: it turns a fresh clone into a setup failure. Materializing .git is
likewise ruled out by git init.

The four existing guard tests assert the mask and what the planner emits, and
all four pass with the guard absent from disk, which is why a green suite said
nothing. The new tests apply the plan and read the actual DACL back through
SDDL. Verified the fresh-workspace case fails without this change and passes
with it, while the already-has-.git case passes either way, so the new coverage
is pinned to the defect rather than to the code.

Also moves windowsACLPlanPaths behind the Windows build tag. Every caller is
Windows-tagged, so defining it in the portable file made it dead code on Linux
and macOS and failed make lint-static, as anandh8x reported.
2026-08-27 13:27:11 +05:30
Vasanthdev2004 0a80db48fe fix(sandbox): reject protected metadata names that escape the write root
ProtectedMetadataNames is documented as names, but it is joined onto the write
root to place a deny ACE and to materialize the directory that ACE protects. A
value containing ".." or a separator puts both outside the workspace: the deny
lands on a directory the sandbox does not own, and elevated setup creates it
there. Raised by CodeRabbit on this PR.

Not reachable today, since the only caller passes a package constant. It is
guarded so that stays true if a future caller sources these from config, which
is the kind of change that would not obviously be a security decision.

The component check moves to the portable file so both users share it. The
handle-relative primitives need it because NtCreateFile resolves a relative
name containing separators the ordinary way, following any junction inside it;
the plan builder needs it for the escape above. Separators are matched
explicitly rather than through filepath.Base, because these are Windows paths
whatever the build host is, and on Linux filepath.Base leaves a
backslash-joined name untouched and would wave it through.

Rejection tests are in the portable test file, so they run on all three
platforms alongside the rest of the plan-builder coverage.
2026-08-27 13:27:11 +05:30
Vasanthdev2004 9b9937bc3d fix(sandbox): bind windows ACL materialization and rollback to handles
Closes the two path-swap findings on this PR. Both had the same cause: an
object was addressed by a pathname that the kernel re-resolves at the moment
the call runs, so a workspace owner could change what that name meant in the
gap between checking it and using it. Junctions need no privilege, so this was
reachable by exactly the unprivileged user the sandbox exists to contain.

Materialization verified a component, closed the handle, then handed the same
string to os.Mkdir. Swapping the verified ancestor for a junction inside that
gap got the component created outside the approved tree, as Administrator, and
verifying again afterwards cannot un-create it. It now opens the deepest
existing ancestor once and creates each missing component relative to the
handle of the level above, so the tree it descends is pinned to objects rather
than named by strings.

Rollback called os.RemoveAll on the target pathname, so an ancestor swapped to
a junction after creation sent a recursive elevated delete into an unrelated
tree. It now records what it created as single components under an anchor and
unwinds them handle-relative, deepest first. It also removes the whole chain
rather than only the final component, which the old comment claimed was fixed
and was not, and it removes only what the kernel confirmed this run created,
so a racing creator no longer gets its directory deleted on teardown.

Three things this turned up that were not in the original findings:

The delete primitive used FILE_DELETE_ON_CLOSE, which defers the removal to
cleanup and reports a non-empty directory to nobody: open succeeded, close
succeeded, directory still there. Wired into rollback that would have been
worse than the os.RemoveAll it replaced, since rollback would report success
while leaving state on disk. It now sets the disposition explicitly, so
STATUS_DIRECTORY_NOT_EMPTY comes back to the caller. Residue is preferable to
recursing through a path someone else controls; lying about it is not.

The child create never inspected the handle it returned, so an existing
junction was handed back as the next parent in the walk and every deeper
create landed on the far side of it. Component names are now validated as
single components too: NtCreateFile resolves a relative name containing
separators the ordinary way, which would have walked straight through an
intermediate junction.

Rollback re-opens the anchor by pathname and checks its volume and file index
against what materialization saw, because replacing a directory with another
real directory of the same name needs no reparse point at all and passes every
no-follow check there is. Holding the handle instead would be stronger, but
three call sites deliberately discard the rollback closure and would leak.

Testing. makeWindowsACLDirChainNoFollow carries a seam that fires between
verifying the anchor and creating anything, because a race nobody can trigger
on demand is not a regression test. The existing junction test plants its
junction before the walk starts and so never reached this. The new test has a
control arm that performs the identical swap and creates by pathname, and
asserts the object does escape, so the fixed arm proves the hole was open
rather than proving some code ran. Also covers the file target, rollback
through the closure callers actually hold, and the non-empty case that the
old empty-directory-only delete test hid.

Validation now runs before the filesystem is touched, so a malformed entry no
longer creates a chain and then fails.
2026-08-27 13:27:11 +05:30
Vasanthdev2004 52d1a31657 fix(sandbox): keep the git rename guard tests portable
The two rename-guard tests built their expectations from a hardcoded
C:\work\repo, but buildWindowsPrincipalACLPlan normalizes every write
root before it names an ACE. On Windows that root is already absolute,
so normalizing is a no-op and both tests passed locally. On Linux and
macOS it is not absolute, so the plan named a different path and both
tests failed. The plan builder is portable code with no build tag, so
these tests run on every platform.

Use an OS-neutral root and normalize the expected path the same way the
plan does, which is what the other untagged test in this package
already does. The Windows-only assertions about the ACE mask and its
inheritance stay where they are, in the file that is tagged for it.
2026-08-27 13:27:11 +05:30
Vasanthdev2004 9b54fb48b5 feat(sandbox): handle-relative directory create and delete on Windows
Groundwork for the two remaining materialization P1s. No call sites yet;
the walk and the rollback move onto these next.

Every pathname-based call re-resolves the whole path inside the kernel
when it runs, so verifying a component and then creating through it are
two separate resolutions of the same string. A workspace owner can swap
an ancestor for a junction in that gap. Demonstrated, not theorised: with
the same swap performed between verify and create, os.Mkdir put the new
directory OUTSIDE the approved tree ("landed inside approved tree: false,
ESCAPED outside approved tree: true"), and the post-create check cannot
un-create it.

A handle pins the object rather than the name, so a create resolved
against it cannot be redirected however the path is later rearranged.
os.Mkdir, os.OpenFile and os.RemoveAll are pathname-based by construction
with no relative form on Windows, hence NtCreateFile with
OBJECT_ATTRIBUTES.RootDirectory. x/sys/windows already exposes every
piece, so this adds no hand-rolled syscall bindings.

createWindowsACLChildDirectory reports whether it created or opened,
because rollback must delete only what it made; removing a directory that
already existed would destroy a user's data over an unrelated failure.
deleteWindowsACLChildDirectory is the counterpart that rollback needs:
os.RemoveAll on a pathname whose ancestor has since become a junction
recurses outside the workspace.

The fix also makes the race testable. Against the old code the swap had
to be threaded into the middle of a function; here it is three ordinary
lines between an open and a create, because the handle is held across
them. Both tests perform the real swap and assert the operation stayed
inside the verified directory, and the delete test leaves a bystander
under the decoy whose survival proves the delete never resolved by path.

Refs #808.
2026-08-27 13:27:11 +05:30
Vasanthdev2004 66724c6e0e fix(sandbox): stop a principal replacing .git to shed its carveouts
The write-denied carveouts guarding .git are attached to .git/config and
.git/hooks as OBJECTS. The workspace allow grant is inheritable and
carries DELETE, and nothing denied DELETE on .git itself, so a principal
could rename .git aside, recreate it, and create fresh config and hooks
that inherit the allow with no deny of their own. That restores
credential.helper and core.hooksPath, and with them arbitrary code
execution on the next git command.

.git could not simply join sandboxFullyProtectedMetadataNames next to
.zero and .agents: that list emits DenyWrite, whose mask includes
FILE_GENERIC_WRITE, and git has to write index, objects and refs. Its
absence from that list was correct and was also the hole.

Add WindowsACLDenyDelete: DELETE, WRITE_DAC and WRITE_OWNER only.
Renaming a directory needs DELETE on that directory, so denying it is
what closes the replacement. WRITE_DAC and WRITE_OWNER come along because
a guard the principal can rewrite is not a guard. FILE_GENERIC_WRITE and
FILE_DELETE_CHILD stay out so git keeps working.

The ACE is applied uninherited, which is why the narrow mask is safe:
inherited onto .git's children it would deny DELETE on every file inside
and git could not remove a lock file or a ref. windowsExplicitAccessEntries
hardcoded SUB_CONTAINERS_AND_OBJECTS_INHERIT for every directory entry,
so inheritance is now decided per action.

The entry is not materialized. git creates .git, and an empty one made by
setup breaks git init.

Reported by @jatmn on #808.
2026-08-27 13:27:10 +05:30
Vasanthdev2004 03e556bce2 test(sandbox): compare ACL record paths the way the plans do
The two new record tests failed on the Windows runner and passed
everywhere else. The plan builder runs every root through
normalizeProfilePath, whose EvalSymlinks expands the 8.3 short name
GitHub's runners hand out for TEMP, so the record held
C:\Users\runneradmin\... while t.TempDir() had returned
C:\Users\RUNNER~1\... and an EqualFold on the raw spellings called two
spellings of one directory different paths.

Production was never affected: the recorded paths and the newly planned
paths both go through that same normalization, so setup and teardown agree
with each other. This was only the test being naive about what "same path"
means on Windows, and a developer whose volume has 8.3 name generation
disabled cannot reproduce it — which is how it shipped.

The comparison now normalizes both sides through normalizeProfilePath and
keys them with windowsCapabilityPathKey, which is what the ACL plans
themselves use. Re-checked against the mutation that matters: reverting the
revocation to the new plan's paths alone still fails
TestReSetupRevokesARootTheNarrowedPolicyDropped, so the looser-looking
comparison has not defanged the assertion.
2026-08-27 13:27:10 +05:30
Vasanthdev2004 542728a7da fix(sandbox): revoke principal ACEs on roots that left the policy
applyWindowsPrincipalACLs revoked the trustee only from the paths of the
plan it was about to apply. A root that LEFT the policy is by definition
absent from that plan, so its ACE survived the re-setup marker validation
forces and the principal kept write access the current policy no longer
grants -- the sandbox widened as a result of being tightened. Teardown
repeated the same current-plan-only calculation, so retiring the principal
cleaned every path except that one, and then deleted the account, leaving
the ACE naming a SID nothing could resolve.

Nothing on Windows can answer "which paths hold an ACE for this SID"
without walking every volume, so the grants are now written down as they
are made: a per-principal record beside the secret, keyed the same way
because one sandbox home serves every workspace on the machine. Setup
revokes over the union of the recorded paths and the new plan's; teardown
revokes over the same union.

The record is written as that union BEFORE any DACL changes and narrowed
to the granted set after, so a crash in between leaves a superset rather
than a record missing paths the run granted. A superset is the safe
direction: revoking a path that holds no ACE for the trustee is a no-op.

The interesting case is a principal from an earlier setup with no record.
Proceeding with an empty prior set would be the fail-open, on the one path
where the prior set is not empty but unenumerable. Setup retires that
account before provisioning instead: Windows never reuses a deleted local
account's RID, so whatever ACEs cannot be found end up naming a principal
that no longer exists, and the SID minted next is one no DACL on the
machine can already carry. It needs no new operator action, which matters
-- there is no `zero sandbox teardown` to send anyone to.

Every guard is mutation-checked. Reverting the union to the new plan's
paths fails the end-to-end test against real DACLs with "the principal
kept its grant on a root the narrowed policy removed"; moving the record
after the grant, dropping the retirement, reverting teardown, and trusting
an unknown schema each turn a different test red.

Two existing tests move to the new applyWindowsPrincipalACLs signature.
2026-08-27 13:27:10 +05:30
Vasanthdev2004 c13799d56a fix(sandbox): carry the principal opt-in through the setup protocol
Elevated setup and the commands that later use a principal each read
ZERO_WINDOWS_SANDBOX_IDENTITY from their own process environment, and setup runs
in a separate, UAC-elevated process whose environment is not the caller's.
commandConfig() returned a nil Env, so the gate in windows_setup_windows.go fell
through to os.Getenv in the elevated helper. The two halves could disagree and
nothing detected it.

Direction B is the dangerous one: command opted in, setup did not. Marker
validation passed (the marker recorded nothing about the opt-in), the principal
lookup then declined with a nil error, and the command ran on the same-user
restricted token — which by its own comment does not confine reads — while the
operator believed an account boundary was isolating it. No warning fired.
Direction A left an orphaned account holding batch-logon rights and workspace
ACEs that teardown never retires, because teardown sits inside the same opt-in
branch.

The opt-in is now resolved in the shell the user typed `zero sandbox setup` into
and serialized across the UAC boundary as --sandbox-principal 0|1. The marker
records it (schema 4 -> 5) and validation refuses on mismatch, before the ACL and
network checks. Both directions refuse rather than fall back silently: an
unreadable value is rejected outright, since guessing "off" would provision a
weaker sandbox than asked for and report success.

PrincipalOptIn is a *bool, not a bool. Unset means "consult the environment"
rather than "opted out", so the existing smoke-test callers that do not set it
keep working instead of serializing --sandbox-principal 0 while the command half
still consults os.Getenv.

Dropped the deny-mode fallback warning added alongside this. Announcing it looked
right, since deny is the default mode and an opted-in operator therefore never
gets a principal for ordinary commands. But this runner is re-exec'd per command,
so its sync.Once is once per COMMAND: the notice would print on nearly every tool
call, and noise that repeats gets filtered rather than acted on. It is also not
actionable per command. `zero doctor` carries the opt-in now and is the right
surface for a standing configuration fact. The deny-mode behaviour stays pinned
by TestPrincipalBackendDefersToRestrictedTokenWhenNetworkDenied.

Reported by jatmn on #808.
2026-08-27 13:27:10 +05:30
Vasanthdev2004 53fb2e8daa fix(sandbox): keep the principal inside the Windows write jail
The principal branch handed the raw LogonUser token to CreateProcessAsUser. That
token is a full token for the account, so the sandboxed child kept every write
its ambient memberships grant. The ACL plan can add grants and denies at named
paths, but it cannot revoke what BUILTIN\Users, Authenticated Users or
NT AUTHORITY\BATCH already allow elsewhere — so an opted-in command whose
profile permitted writes only to the workspace and runtime roots could still
write C:\Users\Public\Documents, which grants BATCH modify and which a batch
logon therefore satisfies.

The principal now gets its own identity AND the restricted token, not one or the
other: reads stay confined by its ACEs, writes by the restricted-SID check.

The principal's own SID joins the capability SIDs deliberately.
applyWindowsPrincipalACLs grants the workspace to identity.SID rather than to a
capability SID, so omitting it would leave the workspace grant matching nothing
in the restricted list — a jail that locks out the inmate and no one else. The
SID is read back from the token itself rather than threaded through the call, so
it cannot drift from the identity actually running.

Not fixed here, and separate from this finding: worldSID is unconditionally in
the restricted-SID list, so any path whose DACL grants Everyone still satisfies
the restricted check on both this path and the pre-existing fallback. That
predates the principal work and is raised with the maintainers separately.

Reported by jatmn on #808.
2026-08-27 13:27:09 +05:30
Vasanthdev2004 11534a4f1f fix(sandbox): reject reparse ancestors before creating an ACL target
FILE_FLAG_OPEN_REPARSE_POINT only stops the FINAL path component being followed.
materializeWindowsACLTarget built its target with os.MkdirAll and os.OpenFile on
the pathname, and both resolve ancestors — so the ancestor reparse check on the
subsequent no-follow re-open ran only after the objects already existed.

An ordinary workspace owner needs no privilege to create a junction. Turning
.git into one before elevated setup runs, with config absent, had setup create
the target at a location of the attacker's choosing as Administrator. Rejecting
it afterwards does not undo that, and the failure path removes only the final
component, so every intermediate directory MkdirAll created outside the
workspace survived permanently.

Creation now goes through makeWindowsACLDirChainNoFollow, which walks up to the
deepest existing ancestor and verifies it no-follow first. One check suffices for
the whole chain above it because GetFinalPathNameByHandle answers for the entire
resolved path. Missing components are then created one at a time, each
re-verified immediately after creation, so a component swapped for a junction
mid-walk is caught before anything lands underneath it.

Taken over the relative-handle NtCreateFile route because x/sys/windows offers no
ergonomic relative-create primitive, and this leaves a window of one component
with an immediate post-create check rather than create-everything-then-verify.

Reported by jatmn on #808.
2026-08-27 13:27:09 +05:30
Vasanthdev2004 2d5dcfa259 fix(sandbox): surface a failed stale-secret cleanup after rotation
The provisioning rollback removes the stored secret when this run created the
account or rotated an adopted one's password, because in both cases what is on
disk cannot authenticate and absent beats stale: the command path treats a
missing secret as "not provisioned" and falls back, while a stale one fails the
logon and reports a broken sandbox.

That removal ignored its own error. When it failed, the invariant it exists to
keep was not restored — a credential for a password that no longer works stayed
on disk — and setup said nothing. The operator met a provisioned-but-unusable
principal on the next command instead of hearing it from the run that broke it.

undo now returns that one error and the five failure paths join it onto the
error they were already returning, so the original cause and the cleanup failure
both surface. The message names the file and what to do about it. The other undo
steps still swallow: they leave residue, while this one leaves a credential.

Reported by jatmn on #808.
2026-08-27 13:27:09 +05:30
Vasanthdev2004andClaude Opus 5 7681a95656 fix(sandbox): stop teardown creating a directory while naming one
windowsPrincipalTeardownPaths said the runtime root was "resolved without
creating it, since teardown has no business making directories on its way
out". That was my comment and it was false: it went through
sandboxRuntimeRootFor, whose fallback calls os.MkdirTemp when the
cache-derived root would land inside the workspace. So cleanup created a
fresh temp directory, and a useless one — the fallback root is random per
process and could never match the tree the commands actually used.

sandboxRuntimeRootFor is split: deterministicSandboxRuntimeRoot computes the
cache-derived path and says whether it is usable, creating nothing, and the
existing resolver keeps the fallback on top of it. Teardown takes the pure
one and simply has no runtime tree to revoke when it reports unusable, which
is correct — there is no way to name the random root from here anyway.

The first version of the test called the pure resolver directly. It passed,
and reverting the call site to the creating one left it passing. It now
drives windowsPrincipalTeardownPaths and counts temp-directory entries
across the call, and that mutation fails it.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:09 +05:30
Vasanthdev2004andClaude Opus 5 1767a08872 fix(sandbox): close the delete-through-parent, junction-ancestor and rollback gaps
Three findings from jatmn's review, all reachable only under the opt-in
principal backend but all real.

FILE_DELETE_CHILD is no longer granted. On a parent it authorises deleting a
child whatever the child's own DACL says, so granting it on a write root
handed back the carve-outs underneath: delete .git/config, recreate it, and
the replacement inherits the grant with no deny of its own — restoring the
credential.helper and core.hooksPath control the carve-out exists to
prevent. It was granted to keep the mask symmetric with the deny mask, which
is the wrong instinct: denying a capability is not a reason to grant it. The
comment two lines up already made that argument for WRITE_DAC and
WRITE_OWNER. DELETE alone still covers removing and renaming files inside
the roots, which is what the grant is actually for — verified before
removing it.

Note this does NOT close the second route jatmn described: .git itself
carries no ACE, so renaming the whole directory aside needs only DELETE.
That needs a guard on .git and is not in this commit.

ACL targets are now rejected when a PARENT is a reparse point. CreateFile
resolves ancestors even with FILE_FLAG_OPEN_REPARSE_POINT, so the
final-component check passed while elevated setup rewrote the DACL of an
object outside the workspace. Junctions need no privilege to create, unlike
symlinks, so this was reachable by exactly the unprivileged user the sandbox
contains. GetFinalPathNameByHandle answers where the handle really landed,
covering every component in one call instead of walking the path and racing
between checks. The comparison is against the path's own resolved form, so a
differently-cased or 8.3 spelling is still accepted.

The revocation's rollback is returned instead of discarded. Discarding it
was justified on the grounds that the only failure path removes the
principal outright — true for a principal this run CREATED, false for one it
ADOPTED, which #812 keeps alive on failure rather than destroying someone
else's working account. The account survived with its previous ACEs stripped
and the new ones rolled back: logged on, and unable to reach its own
workspace. Teardown still discards it deliberately, since putting ACEs back
on an account about to be deleted is the opposite of the point.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:09 +05:30
Vasanthdev2004andClaude Opus 5 1a6b08fe5f test(sandbox): assert the ancestor walk, not the old all-or-nothing contract
TestCanonicalWorkspaceRootFallsBackWhenResolutionFails asserted that a path
with missing segments came back as the plain cleaned path. That was the
behaviour before the ancestor walk, and Windows CI failed it correctly:

    canonical("C:\Users\RUNNER~1\...\001\never-created\deeper")
      = "C:\Users\runneradmin\...\001\never-created\deeper", want the cleaned path

The existing ancestor resolved and the missing remainder was re-appended,
which is precisely what the walk exists to do. The assertion now says that:
the result equals the canonical parent joined with the segments that do not
exist, and those segments survive rather than collapsing to the ancestor.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:09 +05:30
Vasanthdev2004andClaude Opus 5 e1c57564a0 fix(sandbox): resolve through path segments that do not exist yet
The previous fix normalized both the workspace root and the cache root, and
macOS CI still failed the same way. EvalSymlinks fails outright when the
LEAF does not exist, and a cache root has not been created at the point it
is first normalized — so the workspace resolved (/var to /private/var)
while the cache root did not, and the containment check that decides
whether the runtime tree must move out of the workspace compared the two
anyway.

canonicalSandboxWorkspaceRoot now resolves the longest existing ancestor
and re-appends the remainder, so a path normalizes the same way whether or
not its final segments exist:

    /var/.../001/.cache        leaf missing, walk up
    /var/.../001               resolves
    /private/var/.../001/.cache

Terminates at the filesystem root, where it falls back to the cleaned
absolute path, and a path with no symlink anywhere along it is unchanged.

The regression test needs a symlink to produce a spelling only resolution
reconciles, so it skips on Windows — where creating one needs privilege —
and runs on the platforms that caught this. I could not reproduce either CI
spelling locally: this box's temp paths are already canonical and 8.3 alias
creation is disabled on the volume.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:09 +05:30
Vasanthdev2004andClaude Opus 5 b1fb0ab42a fix(sandbox): normalize the cache root too, not just the workspace root
sandboxRuntimeRootFor compares the workspace root against the runtime root
it derives from the cache root, and falls back to a private temp tree when
the derived root would land inside the workspace. The previous commit
canonicalized only the workspace root, so that comparison ran on two
different spellings of the same path and the containment check missed:

    macOS:   /var/folders/... vs /private/var/folders/...
    Windows: C:\Users\RUNNER~1\... vs C:\Users\runneradmin\...

The fallback never fired and the runtime tree was placed inside the
workspace it exists to stay out of. Both CI runners caught it; my box did
not, because its temp paths are already canonical and 8.3 alias creation is
disabled on the volume, so I could not reproduce either spelling locally.

Both inputs now go through canonicalSandboxWorkspaceRoot, on the
cross-platform path and the Windows setup path.

The regression test uses a symlink, which is the portable way to produce a
spelling only resolution reconciles — Clean cannot see through one. It
skips on Windows, where creating one needs privilege, and runs on the
platforms that caught the bug.

Two things about that test are deliberate. My first version used a
redundant-segment path, which Clean already normalizes, so reverting the
fix left it passing. My second resolved nothing before asserting, and
through the link the runtime root shares no textual prefix with the
workspace — it would have called a root sitting physically inside the
workspace "outside" and passed against the exact bug it exists for. It now
resolves before comparing.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:08 +05:30
Vasanthdev2004andClaude Opus 5 1945d47ef8 fix(sandbox): canonicalize the workspace root on both sides, not just setup
Windows CI caught two faults in the previous commit. Both came from
canonicalizing one side of a pair.

setupWindowsSandboxRuntimeRoot resolved the workspace root while
prepareSandboxRuntime still only cleaned it, so the two disagreed exactly
where they had to agree. It passed locally because my temp paths were
already canonical; a Windows runner's TEMP is an 8.3 short path that
resolution expands:

    setup granted ...\runtime\v1\5e2d212300ccdfba
    commands use  ...\runtime\v1\92c31f8cf536dfde

The canonicalization moves to canonicalSandboxWorkspaceRoot in
runtime_state.go and both sides call it, which is what the original fix
should have done.

The carveout shape was rebuilt from the RESOLVED write root and compared
against subpaths that cannot resolve, since .git/config does not exist at
setup and normalizeProfilePath falls back to Clean when EvalSymlinks fails.
Two spellings of the same path therefore missed the lookup and .git/config
went back to being created as a directory — the original bug, reintroduced
quietly by its own fix. gitMetadataCarveoutIsFile now matches on the
trailing segments, derived from the spec list so it cannot drift from it,
and no reconstructed absolute path is compared at all.

Both failures now have regression tests that reproduce the non-canonical
root by lowercasing, which needs no short name and no privilege. Reverting
either fix fails them.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:08 +05:30
Vasanthdev2004andClaude Opus 5 d0693ffa8d fix(sandbox): revoke ACEs on teardown and key setup off the resolved root
Two ways setup and the command path disagreed.

Teardown removed the secret, the LSA rights and the account, but never the
ACEs. Once the account is gone its SID stops resolving and every ACE naming
it becomes an orphaned raw-SID entry on the user's own tree — precisely the
residue the capability-SID model left behind and this one exists to avoid.
Revocation now runs while the SID still resolves, by trustee so it also
clears grants written by older versions. A revoke failure is deliberately
not fatal: a path the user has since deleted cannot be cleaned, and
refusing to remove the account over it would strand the principal and its
logon rights permanently, which is worse than a leftover ACE.

The runtime root was derived from filepath.Clean(WorkspaceRoots[0]) at
setup while Engine.resolveCommandDir cleans, absolutizes and then
EvalSymlinks it. That needs no symlink to diverge — Windows opens a path in
any casing and EvalSymlinks canonicalizes it:

    setup sees   c:\users\me\myworkspace
    command sees C:\Users\me\MyWorkspace

so setup granted the principal one runtime tree and every command used
another. The grant that exists to make npm/go/pip caches writable landed
where nothing reads, surfacing as a bare ACCESS_DENIED on a cache write.
Both now go through canonicalWindowsSandboxWorkspaceRoot. An unresolvable
root falls back to the cleaned absolute path, matching the command path
rather than failing.

setupWindowsSandboxRuntimeRoot is split into derivation and creation so
teardown can name the tree without making directories on its way out.

The first version of the divergence test called the canonicalization helper
directly. It passed, and reverting setup to filepath.Clean — the actual bug
— left it passing. It now drives windowsSandboxRuntimeRootPath, and that
mutation fails it.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:08 +05:30
Vasanthdev2004andClaude Opus 5 dfe7fa3eb1 fix(sandbox): revoke stale principal ACEs before re-applying the plan
windowsPrincipalRevokePlan was implemented and tested but had no production
caller, so nothing ever used it. applyWindowsACLPlan merges into the
existing DACL, which means a re-run after narrowing a write root or
shortening a deny list left the previous, wider ACEs sitting beside the new
ones: the principal kept access the current policy no longer granted, and
the sandbox silently widened as a result of being tightened.

Setup does get the chance to notice — marker validation already refuses
commands with "permission roots or deny lists changed" until setup runs
again — so the re-apply is exactly where this belongs.

The ACL step is extracted into applyWindowsPrincipalACLs: build the plan,
revoke every ACE naming this trustee on the paths it touches, then apply.
Revocation is by trustee rather than by remembered path, so it also clears
grants written by an older version of Zero. Its rollback is discarded on
purpose — the only failure path from here removes the principal outright,
and restoring stale ACEs for an account about to be deleted is the residue
this exists to prevent.

Extracting it also makes the ordering testable without new provisioning
seams, which #812 already adds with a different signature; adding them here
would have collided on its rebase.

Three tests: revocation actually drops a grant on a root that left the
policy while keeping the one that stayed (asserted against the real DACL,
counting deny ACEs as well as allow, since trustee revocation drops both);
revoking a path that was never created is a no-op rather than an error; and
the production path revokes BEFORE it applies. That last one is the one
that matters — the first two pass just as happily with the call site
deleted, and deleting it kills only the third.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:08 +05:30
Vasanthdev2004andClaude Opus 5 6315181bbf fix(sandbox): re-check principal privilege when minting a command token
Provisioning refuses an account that is already in Administrators, Power
Users or Backup Operators, but group membership is not frozen at setup. An
account provisioned clean can be added afterwards — by an operator, or by
an attacker who already has that access and would like the sandbox to hand
it back. Every command after that minted a token for a privileged account.

The re-check goes on the path that mints the token, not inside
lookupWindowsSandboxIdentity. Teardown resolves the same identity to revoke
its logon rights before deleting the account, so refusing there would leave
the very account this guards against permanently undeletable by Zero. The
command path already propagates anything that is not the not-provisioned
sentinel, so this surfaces to the operator instead of silently dropping
back to the restricted token.

Also make the gating test hermetic. Its "absent" case passed an empty map,
which falls through to os.Getenv, so a developer with the opt-in exported
saw a different result from CI:

    ZERO_WINDOWS_SANDBOX_IDENTITY=1 go test ./internal/sandbox/
    --- FAIL: TestWindowsSandboxIdentityGating/absent
        enabled = true, want false for ""

Every case there supplies an explicit map entry, so the process variable is
now pinned to prove none of them consult it. The os.Getenv fallback is what
elevated setup actually runs on — it passes no Env — so it gets its own
table rather than riding on a case that also has a map entry.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:08 +05:30
Vasanthdev2004andClaude Opus 5 1391376f9d fix(sandbox): materialize .git/config as a file, not a directory
Materializing the git control-plane carveouts creates a missing target so
the deny-write ACE is in place before git first runs. It did that with
os.MkdirAll on the full path, which is right for .git/hooks and wrong for
.git/config: git wants a file there.

The consequence is worse than a mis-ACL'd path. On a fresh workspace
neither carveout exists — which is exactly the case materialization was
added for, so this is the common path rather than a corner — and elevated
setup would leave a directory where git's config file belongs:

    warning: unable to access '<ws>/.git/config': Permission denied
    fatal: unknown error occurred while reading the configuration files

git init then fails outright and the workspace is unusable.

Materialization now takes the shape from the carveout definition:
gitMetadataWriteCarveoutSpecs is the single source of truth and
gitMetadataWriteCarveouts derives its list from it, so a carveout cannot be
added in one place and have its shape forgotten in the other. A file target
gets its parent chain created and then an empty file; a directory target is
unchanged. A racing creator winning the O_EXCL is treated as success, since
the target existing is all materialization needed.

The regression test runs a real `git init` over the applied plan. It names
Guests as the principal rather than Everyone — with Everyone the deny ACE
also denies the test process and git fails for an unrelated reason, which
would have made the test pass for the wrong reason once the shape was fixed.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-27 13:27:08 +05:30
Vasanthdev2004 bf7a530b33 fix(sandbox): refuse to adopt a principal in a privileged group
Adoption takes over an account whose name and ownership comment match,
resets its password and hands it to the sandbox. An account that is also
in Administrators, Power Users or Backup Operators would give the sandbox
the rights it exists to withhold: rewriting the ACLs confining it, reading
the secret locked to the invoking user, and stopping Zero. The name is
derived rather than discovered, so an account can match without anyone
intending it to.

Membership is resolved by well-known SID rather than by group name, so a
localised install where the group is Administratoren or Administrateurs is
still recognised.

Raised as a non-blocking follow-up in review; it is cheap enough to do now
rather than track.
2026-08-27 13:27:08 +05:30
Vasanthdev2004 6d9b437028 fix(sandbox): grant the principal the runtime tree commands write to
Four review findings on the elevated setup path.

The principal had no access to the sandbox runtime root.
permissionProfileWithRuntime appends that root to WriteRoots on every
command and redirects HOME, GOCACHE, npm_config_cache and similar into it,
but it lives under the user cache rather than the workspace, so the
profile setup builds its ACL plan from never contains it. On the
restricted-token path that costs nothing, since the child still runs as
the caller. A principal is a separate local account with none of those
rights, so every npm install, go build or pip install would have failed on
a cache write with a bare ACCESS_DENIED and nothing naming the sandbox as
the cause. Setup now resolves the same root and grants it.

The derivation is extracted so both callers share it. If setup and
prepareSandboxRuntime ever disagreed, the ACE would land on one directory
while commands used another, which is the same failure with a harder
diagnosis, so a test asserts the two agree.

The git control-plane carveouts are materialized. .git/config and
.git/hooks arrive as ReadOnlySubpaths, and applyWindowsACLPlan skips an
absent target, so on a workspace where git had not run yet the deny ACEs
were never written and the principal kept inherited write access once git
created them.

Command-time lookup verifies workspace ownership. The account name carries
only 11 characters of the workspace digest; the comment carries all of it.
Provisioning already refused a foreign account, but the command path
resolved the name straight to a SID, so the workspace that lost a
collision would have run as the other one's principal. SID resolution
still runs first, so an absent account stays the unavailable sentinel
rather than becoming a collision error.

The gated round-trip test asserted a logon with the password from a second
provisioning call. Rotation moved to the setup path, so that value is a
fresh string the account never held. It now exercises the guarantee the
setup path actually makes: the stored secret logs the principal on.
2026-08-27 13:27:07 +05:30
Vasanthdev2004 239894103e fix(sandbox): keep an adopted principal's logon rights on rollback
Second round of review findings, both on the elevated setup path.

The rollback revoked logon rights whenever they had been attempted,
without regard to whether this run created the account.
revokeWindowsSandboxLogonRights passes AllRights, which drops every right
the account holds and deletes its LSA object outright. On an adopted
principal that is not a rollback but destruction: a transient grant,
secret-path or secret-write failure during a re-run stripped the
SeBatchLogonRight and deny-logon rights an earlier setup had established,
leaving exactly the broken-but-present principal this path exists to
avoid. Revocation is now scoped to accounts this run created. The rights
granted to an adopted account are the ones it is supposed to hold, so
leaving them is the safe direction.

A secret the current user cannot read now falls back instead of failing
the command. The secret's DACL names whoever ran setup, so an operator who
elevated with a separate administrative account, through runas or an
over-the-shoulder UAC prompt, leaves a secret their ordinary account
cannot open. That is the documented fail-soft case, and treating it as a
hard error made every sandboxed command fail on a machine that was merely
set up by a different admin. Permission errors from the removal path are
deliberately still reported, since incomplete teardown is worth knowing
about.

Both are covered by injected-failure tests and fail if the guard is
removed. The secret read is seamed to inject the permission error, because
producing a real ERROR_ACCESS_DENIED needs DACL surgery and would test the
platform rather than the mapping.
2026-08-27 13:27:07 +05:30
Vasanthdev2004 815d220a2f fix(sandbox): stop setup destroying a principal it did not create
Six findings from review, all on the elevated setup path.

Teardown was not scoped to what the run created. provisionWindowsSandbox
PrincipalForSetup was careful never to delete an account it had adopted,
and then setupWindowsSandboxPrincipal called removePrincipal on any ACL
failure with no such guard. Re-running elevated setup on a working machine
and hitting one transient ACL error therefore deleted the local account,
its secret and its logon rights. It now returns whether it created the
principal and the outer teardown honours it; ACEs are still reverted,
since this run applied them.

Password rotation moved to immediately before the secret is committed.
Resetting an adopted account's password at the top of provisioning meant
every later step ran against an account whose password had been replaced
with no copy stored. Any failure there left a live account authenticated
by a password nothing on disk knew, and since the account pre-existed the
rollback correctly declined to delete it, so the command path read the
absent secret as "not provisioned" and fell back to the weaker backend for
good. The two operations are now adjacent. The rollback also stops
removing the secret when this run neither created the account nor rotated
it, because that secret still works.

Policy DenyWrite now reaches the principal ACL plan. The capability plan
has always emitted these; the principal plan denied write only on
protected metadata and read-only subpaths, so once the runner used a
principal token a policy deny elsewhere was not enforced at all.

Principal deny-read entries are materialized, matching the capability
plan, so a path created after setup still gets a deny ACE.

Logon-right revocation is keyed to the attempt rather than to success.
Rights are added one at a time and the grant returns on first failure, so
a partial grant left LSA entries behind pointing at a SID that deleting
the account then made unresolvable.

The ownership comment now carries the full workspace key. The account name
holds only 11 characters of the digest, so two workspaces could derive one
name and silently share an account, a secret and an ACL identity; a
mismatch is now refused. Accounts provisioned before the key was recorded
are still adopted.

Also warns once on stderr when the opt-in is set and a provisioned
principal cannot be used, rather than downgrading in silence.
2026-08-27 13:27:07 +05:30
Vasanthdev2004 0b6b37eca5 fix(sandbox): grant delete to the principal and keep rollback able to find it
Two findings from review, both consequences of the principal being a
separate account rather than the calling user.

WindowsACLAllowWrite granted FILE_GENERIC_WRITE, which covers creating and
modifying but not removing or renaming, and a rename needs delete on the
source. Under the old same-user token this was invisible because the caller
already held inherited rights on its own tree. A principal inherits nothing,
so it could write files it could never delete, which fails ordinary editing
and most git operations rather than an edge case. DELETE and
FILE_DELETE_CHILD are now part of the grant, matching WindowsACLDenyWrite,
which already treats delete as part of write. WRITE_DAC and WRITE_OWNER stay
out: they are denied so the principal cannot rewrite its own restrictions.

provisionWindowsSandboxIdentity returned a zero identity alongside
created=true when group attachment or SID resolution failed after NetUserAdd
had already created the account. The caller's rollback deletes by
identity.Username, so it was asked to delete the empty string and left the
account behind. Group attachment is the case that matters, being both the
enforcement boundary and something local policy can refuse. The name now
comes back with the error.

The four provisioning calls are indirected so the failure paths are
reachable in a test. Seaming only the post-creation pair would not have been
enough: every step needs an elevated caller, so the test would have stopped
at the group check and passed without reaching what it names.

Also seeds the empty-secret test with a genuinely empty file. The previous
whitespace seed was several bytes, so it never reached the length check and
failed later in DPAPI instead, which another test already covers.
2026-08-27 13:27:07 +05:30
Vasanthdev2004 324d9f32fb test(sandbox): check the ACE type before decoding its trustee
GetAce returns a generic ACE_HEADER and the helper reinterprets it as an
ACCESS_ALLOWED_ACE. That holds for the fixed-layout types, but an object
ACE carries Flags and two GUIDs ahead of the trustee, so SidStart would
land mid-structure and Copy would read whatever bytes follow. The caller
asserts that no unexpected trustee appears in the DACL, and on such an ACE
it would print a nonsense SID rather than name the entry that does not
belong.

Nothing under test builds anything but allowed ACEs today, so this changes
no current outcome. It keeps the failure legible if that ever changes.
2026-08-27 13:27:07 +05:30
Vasanthdev2004 706e1a4bee fix(sandbox): drop the stored secret whenever provisioning fails
The cleanup added for partial provisioning left the window it existed for
uncovered. It only removed the on-disk secret when this run had written one,
and it derived the secret path after the logon-rights grant, so a failure
before that point had nothing to remove.

That is exactly the case that matters. Provisioning ALWAYS sets the account's
password, including resetting a pre-existing account's, so from the moment it
returns the stored secret is already stale. A failure in the rights grant then
left that stale secret on disk against a password that had just changed, and
the next command failed the logon and reported a broken sandbox instead of
falling back.

The path is now resolved from the account name before anything can fail, and
removal is unconditional rather than gated on having written one. Absent beats
stale: the command path treats a missing secret as "not provisioned" and falls
back to the restricted token, which is the outcome a failed setup should leave
behind.

Raised by CodeRabbit, twice from different angles, on the commit that added the
cleanup.
2026-08-27 13:27:07 +05:30
Vasanthdev2004 b46d7d19c6 fix(sandbox): refuse a squatted account name and clean up partial provisioning
Two problems in the provisioning path, both raised in review.

The account name is derived from a workspace hash rather than discovered, so it
can be occupied by a local account that has nothing to do with Zero, whether by
coincidence or because somebody put it there. Provisioning treated "NetUserAdd
says it exists" as "this is ours", reset the account's password, added it to the
managed group and adopted it. That is a stranger's account taken over during an
elevated setup, on the strength of a name matching a pattern we generate
ourselves.

Ownership is now proven from the comment provisioning stamps before anything is
touched, and a name held by an account Zero did not create fails with a typed
collision error instead of being adopted.

Second, a failure anywhere after the account existed left it behind. The
rollback the setup path installs is only built once provisioning has returned
successfully, so nothing could undo a failure between creating the account and
storing its secret; the account, and possibly its granted logon rights, simply
stayed. Provisioning now unwinds what the run actually did, in reverse, on every
failure path.

Scoped to what THIS run created, deliberately. An account that already existed
and belongs to Zero is a working principal from an earlier setup, so deleting it
because a later run failed would turn a partial failure into a total one. For
that case the repair is dropping the stored secret instead: this run reset the
password, so the secret no longer matches, and absent beats stale because the
command path treats a missing secret as "not provisioned" and falls back rather
than failing.

The ownership gate is asserted against real accounts every Windows install
carries, which needs no privilege because it only has to establish that they are
not ours. Classifying everything as managed makes it fail.
2026-08-27 13:27:06 +05:30
Vasanthdev2004 ae294d8f98 fix(sandbox): revoke logon rights before deleting a principal
Retiring a principal deleted the account but left its LSA account rights
behind, keyed to a SID that no longer resolves. That is the orphaned residue
this model is supposed to avoid, and the reason ACE revocation is keyed to the
trustee rather than to a record of what was granted; the logon-rights half was
simply missing. CodeRabbit spotted it as a test-cleanup gap, but the production
teardown path had the same hole.

revokeWindowsSandboxLogonRights drops every right held by the principal and
removes its LSA entry, and setup teardown now calls it BEFORE deleting the
account, while the SID still resolves. Removing all rights rather than naming
them is deliberate: the principal is being retired, so rights granted by an
older setup that this one no longer knows about should go too.

An account that holds no rights is not an error, since that is the state
teardown wants. That tolerance depends on STATUS_OBJECT_NAME_NOT_FOUND
surviving LsaNtStatusToWinError as something errors.Is can still match, which
is the sort of Windows errno assumption that is often wrong, so there is now an
unprivileged test asserting it, including that the tolerance does not also
swallow access-denied.

Both gated tests now clean up rights and account, in that order. The
provisioning round trip had no cleanup at all and, since it started granting a
batch logon right, was leaving both behind on whatever machine ran it.
2026-08-27 13:27:06 +05:30
Vasanthdev2004 01dcfdc782 fix(sandbox): reset the password when the account already exists
NetUserAdd leaves a pre-existing account completely untouched, password
included, and ensureWindowsSandboxUser treated that status as success. So a
second setup run generated a fresh random password, stored it as the secret,
and left the account still authenticating with the old one. Every later command
then failed to log on with a principal that looked correctly provisioned. Two
comments claimed the caller reset the password in that case; nothing did.
Caught by CodeRabbit.

ensureWindowsSandboxUser now reports whether the account already existed, and
provisioning resets the password through NetUserSetInfo when it did, so the
value it returns is always the account's real password. The comments now
describe what the code does.

The gated provisioning test provisions twice and then logs on with the password
from the SECOND run, which is the only honest assertion here: a stale password
is indistinguishable from a correct one until something tries to authenticate
with it.

Also makes the syscall keep-alives explicit. The LSA and LogonUser call sites
borrow Go memory that was either not kept alive at all (the policy attributes,
the rights descriptor, the three logon strings) or kept alive only after the
error check, so the failure path returned with it already collectable. The two
netapi32 sites that used a deferred no-op closure now use runtime.KeepAlive as
well, so one idiom is used throughout.
2026-08-27 13:27:06 +05:30
Vasanthdev2004 38bac64c50 fix(sandbox): surface a squatted principal name instead of falling back
lookupWindowsSandboxIdentity collapsed every SID-resolution failure into the
"no principal is provisioned" sentinel, which threw away the check
resolveWindowsSandboxSID deliberately makes: a name that resolves to a group or
alias rather than a user account. The command path treats that sentinel as
permission to fall back quietly, so an account name squatted by something that
is not a user reached the operator as silence and a downgrade to the restricted
token. Caught by gnanam in review.

Only ERROR_NONE_MAPPED now means setup has not run. Anything else is a
principal that exists but cannot be used, and the runtime path propagates it
rather than swallowing it, which is where the description already said the line
should sit.

The decision lives in its own function because the lookup derives its account
name from a workspace key, so a test cannot hand it a name that resolves to a
group. The test drives the classifier with a real error from a well-known local
group, needs no privilege, and fails if the old collapse-everything behaviour
is restored.

Also corrects a comment pointing at sandboxRuntimeKey, which does not exist.
The function is windowsSandboxWorkspaceKey.
2026-08-27 13:27:06 +05:30
Vasanthdev2004 672f407d6b feat(sandbox): encrypt the stored principal password to the invoking user
The file ACL stays the primary control and is what keeps the sandbox
principal from reading its own credential. It only binds while the filesystem
is the one being asked, though, so a backup or a mounted image hands over the
password in the clear. CryptProtectData ties the ciphertext to the invoking
user's logon secret, which covers exactly that gap.

The principal name is passed as entropy, so a blob copied onto another
principal's path fails to decrypt rather than authenticating the wrong
account. A secret written by an older build reads as unavailable and falls
back to the restricted token; the next elevated setup rewrites it.

The round-trip test needs no privilege, so it runs everywhere rather than
joining the gated set, and it asserts the password does not appear verbatim
in the stored bytes.
2026-08-27 13:27:06 +05:30
Vasanthdev2004 9bdfa4c618 fix(sandbox): keep the restricted token when the network is denied
Network denial is enforced by WFP filters keyed to the offline-marker SID.
The restricted token carries that SID; a token from LogonUser cannot, because
it names the account rather than a synthetic capability SID. Routing a
denied-network command through a sandbox principal therefore left the block
filters matching nothing and dropped egress enforcement altogether, and deny
is the default mode.

The principal now stands down whenever the network is denied and the
restricted-token backend runs instead, so read confinement is never traded
for a silent loss of network denial. Keying the filters to the principal's
own SID is the follow-up that lifts the restriction.

The decision sits in its own predicate rather than inline: on a machine with
nothing provisioned the lookup declines for its own reasons, so a test that
called through it would have passed with the guard removed.

Also names the opt-out variable when a provisioned principal cannot be used,
since the backend is opt-in and the operator needs a way back.
2026-08-27 13:27:05 +05:30
Vasanthdev2004 39e0e02cc8 feat(sandbox): provision a sandbox principal during elevated setup
Completes the chain. Until now the principal entry points had no non-test
callers, so `zero sandbox setup` created no account and the runner seam always
fell back: the feature was inert end to end.

Setup now provisions this workspace's principal, grants it the batch logon
right, stores its password locked to the invoking user, and applies the ACL plan
that gives it read+write on the workspace and read on the declared read roots. A
principal is a separate account with no inherent access to the caller's tree, so
those grants are what make the sandbox able to run at all, and their absence
elsewhere is what puts credential stores out of reach.

Provisioning is folded into the existing rollback rather than each later failure
path having to remember it, and the rollback revokes ACEs before deleting the
account: removing the account first would leave ACEs naming a SID that no longer
resolves, which is the orphaned residue this model exists to avoid.

Gated on the same opt-in as the runner. Account creation is visible in
`net user` and is exactly what endpoint protection and enterprise policy object
to, so it happens only when asked for; without the opt-in the capability-SID
backend remains the whole of setup, unchanged.
2026-08-27 13:27:05 +05:30
Vasanthdev2004 a56e6f86c9 test(sandbox): cover the logon-rights and token-minting half
Provisioning is now validated on real Windows, but LsaAddAccountRights and
LogonUser had still never executed, so the principal was proven to exist without
being proven usable.

The batch logon doubles as the assertion that the rights grant worked: a
LOGON32_LOGON_BATCH logon fails with ERROR_LOGON_TYPE_NOT_GRANTED unless
SeBatchLogonRight is actually held, so a token coming back is evidence the grant
landed rather than merely that the call returned success. Granting twice is
exercised too, since setup re-runs must not fail on rights already held.

The token's user SID is compared against the principal's. If a token came back
belonging to the caller the identity boundary would be an illusion and reads
would still run as the user, which is the whole thing this model exists to stop.

Removes any leftover account first and cleans up after itself, because an
interrupted earlier run would leave an account whose password no longer matches
a freshly generated one.
2026-08-27 13:27:05 +05:30
Vasanthdev2004 f855802c85 test(sandbox): name the per-shell env syntax in the provisioning skip
The skip fired on an unset environment variable but read as though elevation
was missing. `set VAR=1` is cmd syntax and sets a shell variable rather than an
environment variable in PowerShell, so the test skipped silently after the
operator believed they had enabled it. Spell out all three shells.
2026-08-27 13:27:05 +05:30
Vasanthdev2004 718e064711 feat(sandbox): use a sandbox principal for Windows command execution
Wires the principal model into the runner and settles where the account's
password lives.

The secret is stored under the sandbox home with an explicit,
inheritance-protected DACL naming only the invoking user and SYSTEM. The sandbox
principal is deliberately absent from it: a principal that could read the file
could mint its own token and the identity boundary would be decorative. The ACL
is applied to an empty file before the password is written, so the bytes never
exist under the config directory's inherited permissions, and PROTECTED drops any
inherited ACE outright.

At command time the runner asks for a principal token first and uses it in place
of the restricted token, because a separate account has reads denied by the
filesystem rather than left open the way a same-user restricted token must leave
them. The lookup is fail-soft: opt-out, no provisioned account, or no stored
secret all report "not available" and the existing restricted-token path runs
unchanged. Only a provisioned-but-unusable identity surfaces an error, since
that means setup ran and the sandbox is broken rather than absent.

The backend stays behind ZERO_WINDOWS_SANDBOX_IDENTITY=1 while the privileged
paths are unvalidated, so no existing install changes behaviour.
2026-08-27 13:27:05 +05:30
Vasanthdev2004 2e2ea27dcf feat(sandbox): add Windows sandbox principals
Groundwork for closing the Windows half of #662 and #675, where
credentialDenyReadPaths is a no-op today.

Every Windows backend currently derives its token from the calling user via
CreateRestrictedToken, so the sandbox can constrain writes but not reads: a deny
ACE that would stop the sandboxed child reading a credential store names the
same account Zero runs as, and would lock Zero out too. That is why deny-read is
skipped on Windows rather than merely unimplemented.

This adds a separate local account per workspace, held in one managed group, so
the sandbox has an identity of its own:

- provisioning: managed group, stable per-workspace account name inside the
  20-character limit, crypto/rand password meeting complexity policy, SID
  resolution, idempotent so setup re-runs converge
- logon rights: grants only SeBatchLogonRight and explicitly denies interactive,
  network, remote-interactive and service logon, then mints a token with
  LogonUser pinned to the local machine
- ACLs keyed to the principal: denies emitted before allows so carve-outs
  survive, workspace granted read+write, read roots granted read, protected
  metadata denied write and materialized
- removal: revocation by trustee, so retiring a principal drops every ACE naming
  it without needing a record of what was granted

The inversion is the point. A separate account has no access to the caller's
profile at all, so credential stores are unreachable by construction rather than
by enumerating deny rules, and the same SID is what a write grant or a firewall
rule can be keyed to.

Nothing is wired into command execution yet: these paths are additive and no
existing behavior changes. See the pull request for the open question about
where the principal's password lives.
2026-08-27 13:27:05 +05:30
PierrunoYTandClaude Opus 5 27b319ca88 test(execution): fail loudly when ps cannot answer instead of skipping (#961)
* test(execution): fail loudly when ps cannot answer instead of skipping

Every test in process_unix_test.go proves something about zombie detection, and
every one of those proofs is read out of ps. When ps could not answer, the
tests skipped — which is indistinguishable from success in CI output — and one
assertion passed vacuously for the same reason. A regression in ps parsing or
availability would have shown up as green.

requireProcessState probes BOTH ps pipelines against this test process, which
is by definition alive and not a zombie, so a working ps must report a live
state for it. When it cannot:

- Under CI (or ZERO_REQUIRE_PS=1) this is a hard failure. ps is present on every
  image used today, so its absence means the pipeline regressed and that has to
  surface immediately.
- Otherwise it stays a skip, so a contributor on an unsupported box is not
  blocked.

Both pipelines are probed because they are deliberately separate: the tests
assert what the production process-table parser concludes, so this file keeps
its own per-PID ps oracle rather than borrowing that parser to establish its own
preconditions. One regression must not be able to silence both sides at once.

Once the probe passes, failing to observe an EXPECTED state is a defect rather
than an environment problem, so the remaining escapes become failures:

- The zombie waits (awaitZombie) fail instead of skipping. The caller never
  waits on the child, so its zombie row persists until reaped — reaching the
  deadline means the child never exited or ps stopped answering, and the two
  are reported distinctly.
- A zombie group that is not signalable fails instead of skipping: a zombie
  still occupies the process table, so if kill(-pid, 0) fails the case under
  test is not present and the assertions after it would prove nothing.
- processSignalTarget returning anything but the individual PID fails instead
  of skipping: the child inherited this process's group, so its PGID is this
  process's PID and can never equal its own PID.

The vacuous assertion in TestTerminateProcessTreeStopsRunningGroup now requires
ok. That is sound rather than merely stricter: the test never waits on the
child, so the group's zombie row must still be present after termination, and
an unknown state means ps stopped answering rather than that the group is
clean.

Verified on Linux (WSL, go1.26.6) rather than by cross-compilation alone:

- all three tests pass unmodified
- with a stub ps that exits 1: CI unset skips with the reason named; CI=true
  fails, naming which pipeline could not answer
- the same broken-ps run against the ORIGINAL file produced two silent skips
  plus one unrelated failure after 12s, versus an immediate, explicit failure now

Windows is unaffected (the file is !windows); go vet passes for linux, darwin
and windows.

Fixes #863

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01U389qmQUhoZB3YtXkFmiTq

* test(execution): treat an unsignalable zombie group as a kernel difference

The macOS Smoke job failed:

    the zombie group 5931 is not signalable (operation not permitted)

That was my error. I converted this skip along with the ps-availability ones,
but it never had anything to do with ps: whether kill(2) may signal a group
whose only member is a zombie is a KERNEL difference. Linux permits it, so
signalTargetRunning goes on to consult ps — the branch this test was written
for. Darwin returns EPERM, so signalTargetRunning short-circuits on the kill and
never reaches ps.

Rather than restore the blanket skip, the test now records which branch answers:

- Signal permitted: assert processGroupHasRunningMember directly, so a ps-side
  regression cannot hide behind the kill result. This is new coverage; the old
  test only checked signalTargetRunning, which conflates the two paths.
- Signal refused: log it and continue.

The outcome assertion — a zombie-only group must not count as running — now runs
on EVERY platform. Previously Darwin skipped before reaching it, so this is
strictly more coverage there, not less.

Any errno other than EPERM still fails: it means the group is not in the state
the test needs, and nothing after it would be meaningful.

TerminateProcessTree is asserted only where the signal is permitted, because
terminateTarget's own kill(2) would hit the same EPERM and return it as a
cleanup error. Worth a maintainer's eye: that may be a real gap in
TerminateProcessTree on such a platform — a zombie-only tree is already exited
and arguably should terminate cleanly rather than report failure, which is the
same spurious-cleanup-failure class #774 set out to remove. Confirming that
needs a Darwin host and is a production change, so it is flagged rather than
guessed at here.

Verified on Linux (WSL, go1.26.6): all three tests pass, including the new
direct ps assertion. The Darwin branch was exercised by forcing the EPERM path
on Linux — it logs both explanations and passes with the outcome assertion
still enforced. go vet passes for linux, darwin and windows.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01U389qmQUhoZB3YtXkFmiTq

---------

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
2026-08-26 13:22:43 +08:00
Anandan 1fde941803 perf: reduce startup and turn overhead (#932)
* perf(tui): disable duplicate composer cursor blink

* perf(tui): stop composer wakeups after idle

* refactor(tools): publish immutable registry snapshots

* perf(startup): defer optional tool discovery

* perf(provider): reuse compatible response sessions

* feat(provider): add native patch tool input

* fix(provider): recognize saved chat sessions

* perf(tui): render only on screen changes

* fix(runtime): address performance review findings

* fix(usage): expose cache efficiency details

* perf(agent): reduce redundant bounded-task turns

* fix: harden runtime review edge cases

* fix: address runtime review findings

* build: restore official Bubble Tea dependency

* fix(cli): redact MCP registration errors

* fix: harden turn session regressions

* fix: keep run tools and patch roots consistent

* fix: bind freeform patch writes to scoped roots

* test: keep patch fixtures out of worktree

* fix(tools): reject mixed-root freeform patches
2026-08-26 13:22:05 +08:00
KRATOS 7acac7acee fix(sandbox): refcount temporary roots so one holder's cleanup cannot revoke another's (#911)
* fix(sandbox): a temporary root is refcounted, so one holder's cleanup cannot revoke another's

Split out of #829 as an independent fix, per @Vasanthdev2004's review asking for
the self-contained pieces to arrive separately.

A temporary grant was add-then-remove with no notion of who still needed the
root. The second caller to ask for a root already present got a NO-OP undo, and
the first caller's cleanup then removed the root out from under it — so a live
grant lost its access while its holder still believed it had it.

Two read-only tools in the same parallel batch, both blocked on the same
directory, is exactly that shape, and read-only tools are precisely the ones the
batch runs concurrently.

tempReads/tempWrites now count live holders, and the root is only stripped when
the last one releases.

The release also performs both mutations under ONE hold of the lock. Dropping it
between them left a window where the root was still in readRoots but no longer in
tempReads: a concurrent AddTemporaryRead landing there reads it as a PERMANENT
root, hands its caller a no-op undo, and this call then strips the root — so that
caller believes it holds access it has already silently lost.

Independent of the remaining #829 work; builds and tests against current main on
its own.

Origin-Session: local-13d543 | Claude Code | 5 prompts
Origin-Snapshot: 6b5eb8ba4e5b
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: d2b8f44a9abc

* fix(sandbox): a read covered by a TEMPORARY write root takes a reference too

Found by a self-review of this branch: the refcount closed the defect within
reads and within writes, but not across them.

AddTemporaryRead returned a no-op undo whenever writeRootCoversLocked said a
write root already covered the path, and the comment there claimed that cover was
permanent. It is not: writeRootCoversLocked scans extraRoots, and
AddTemporaryWrite appends TEMPORARY write roots into that same slice. So a read
covered by another holder's temporary write took no reference, and that holder's
cleanup silently revoked the reader's access — the very defect the refcount
exists to close, reached across the read/write boundary instead of within one
side of it.

Demonstrated before fixing, and the regression fails without it:

  ReadRoots before release = [ws, …/outer]
  ReadRoots after  release = [ws]
  the write holder's cleanup revoked a live read grant on …/outer/inner

The read now takes a reference on the covering temporary write root, and the
genuinely permanent case — the workspace root, or a session-scoped grant — still
returns a no-op undo.

The regression also asserts the other direction: once the reader releases too,
the root is gone, so the reference does not leak the grant past its last holder.

Origin-Session: local-0484ba | Claude Code | 6 prompts
Origin-Snapshot: e6d90ff3ce46
Origin-Session: local-13d543 | Claude Code | 5 prompts
Origin-Snapshot: 6b5eb8ba4e5b
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: d2b8f44a9abc

* test(sandbox): make the concurrency test concurrent, and portable off macOS

Both raised by CodeRabbit. The third finding on this PR — a read covered by a
TEMPORARY write root taking no reference — was already fixed in 87a722cb with the
regression it asked for; this covers the two that were still live, and both are
cases of a test reporting more than it checked.

THE CONCURRENCY TEST WAS NOT CONCURRENT. The gate spun until hasReadRoot saw
somebody holding the root, which is satisfied by the FIRST holder, then opened
the release. The spin wins that race against goroutines still being scheduled:
instrumented over 200 runs, 194 peaked at a single simultaneous holder and not
one ever reached eight. A test named for concurrent holders was exercising one
holder at a time. Against a deliberately broken refcount, where the first release
strips the root, it passed 33 times in 40 — it was a flaky test that mostly
agreed with broken code. Counting all eight acquisitions makes the overlap real:
200 of 200 runs now peak at eight, and the same broken refcount is caught 40
times in 40. Holders signal the barrier on every path including failure, so a
holder that returns early fails the test rather than hanging it.

THE HELPER SKIPPED ON WINDOWS. scopeOutsideRoots reached for /Users/Shared and
fell back to /var/empty, neither of which exists there, so every test built on it
skipped and reported green having asserted nothing. It now uses the home
directory, which no platform lists as a default write root, matching what
scope_extra_read_test.go already does — and then PROVES the choice against
defaultTempWriteRoots rather than assuming it, so a future default root turns
these tests into an honest skip instead of a silent no-op.

Origin-Session: local-abff1c | Claude Code | 2 prompts
Origin-Snapshot: d2f269b81f33
Origin-Session: local-13d543 | Claude Code | 5 prompts
Origin-Snapshot: 6b5eb8ba4e5b
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: d2b8f44a9abc

* fix(sandbox): a permanent grant outlives the temporary one, and an undo is per holder

Both P1s from @anandh8x, both reproduced before changing anything, and both are
the same mistake in different places: treating "something covers this path" as
though it answered "and will it still be there afterwards".

A PERMANENT GRANT WAS REVOKED BY A TEMPORARY HOLDER. extraRoots holds temporary
write roots alongside permanent ones, so Add asked only whether the path was
already covered. A session-scoped grant made while a temporary holder happened to
cover it recorded nothing at all, and disappeared the moment that holder
released. Outliving the request that prompted it is the entire difference between
Add and AddTemporaryWrite, and it was not happening. AddRead had it too,
including across the boundary: a permanent read covered only by a temporary WRITE
root died with that root.

Permanent coverage is now looked for first and across every root, so a temporary
cover cannot decide the answer when a permanent one is also present. The same
root held temporarily is promoted in place — its holders' releases become no-ops,
which is what permanence means here. A BROADER temporary root over a narrower
permanent grant cannot be fixed by promotion, so the narrower root is recorded in
its own right; that is the case he specifically called out, and it is the one a
promotion-only fix would have missed.

ONE HOLDER'S UNDO IS ONE HOLDER'S REFERENCE. The comment on releaseTemporaryRead
claimed each undo is called exactly once and that flooring the count at zero
covered the rest. Flooring stops the count going negative; it does nothing about
a holder's second call consuming a DIFFERENT holder's reference. With two
readers, calling the first's undo twice took the count 2 -> 1 -> 0 and removed a
root the second was still using — the exact revocation this refcount exists to
prevent, reached through a duplicate call rather than a sibling's cleanup, and a
deferred cleanup in a retry path is how a caller does that by accident. Every
undo handed out is now wrapped so it drops its own reference at most once, and
the comment says what is actually true.

Both mutation-checked: unwrapping the undo, and ignoring permanence in Add, each
fail the test that covers them. Race detector clean.

Origin-Session: local-abff1c | Claude Code | 3 prompts
Origin-Snapshot: 07900397c62e
Origin-Session: local-13d543 | Claude Code | 5 prompts
Origin-Snapshot: 6b5eb8ba4e5b
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: d2b8f44a9abc

* chore(sandbox): re-request an automated review of this head

No code change. CodeRabbit's verdict on this PR is pinned to ff85dbcc, a commit
that no longer exists on the branch, and a stale changes-requested keeps the PR
at mergeStateStatus=BLOCKED exactly as a human one would. It re-reviews on push
rather than on request, and repeated @coderabbitai triggers over five days
produced nothing, so this is the only lever available without maintainer rights.

The cost is that this dismisses @Vasanthdev2004's and @anandh8x's approvals of
4a7063aa. The tree they approved is byte-identical to this one.

Origin-Session: local-8cd239 | Claude Code | 6 prompts
Origin-Snapshot: 149ab98453ad
Origin-Session: local-13d543 | Claude Code | 5 prompts
Origin-Snapshot: 6b5eb8ba4e5b
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: d2b8f44a9abc

* test(sandbox): a fixture failure fails, a report cannot deadlock, and AddRead's
own branches are covered

All raised by CodeRabbit on the full review. Every one verified before changing
anything.

## The concurrent test could hang instead of failing

failures was buffered at one slot per holder, but each goroutine can report at
BOTH hasReadRoot checks — its own grant going missing while held, and again
after the release gate. Nothing drains the channel until after wg.Wait(), so the
ninth report blocks forever, wg.Done() is never reached and the wait hangs. The
test would deadlock in exactly the case it exists to catch: a refcount bug
severe enough to fail more than eight of eight holders. Buffered at 2*holders.

## A broken fixture reported itself as a pass

TestANarrowerRequestHoldsTheCoveringGrant called t.Skip when os.MkdirAll failed.
The directory is the fixture, not a precondition the environment may reasonably
withhold, and the sibling test above it already uses t.Fatal for the same call.

## Two tests assumed something they never checked

Both tests in scope_extra_read_test.go place their grant under $HOME precisely
because NewScope seeds /tmp and $TMPDIR as write roots — a grant that collapsed
into one of those would be writable for a reason unrelated to the property under
test. $HOME is normally clear of them, but a sandboxed or CI environment that
points HOME at a temp directory turns both tests into something that proves
nothing while still reporting PASS.

homeGrantOutsideTempRoots now builds the fixture for both and skips with the
reason when the assumption does not hold. Verified by running with HOME set to a
temp directory: both tests SKIP naming the covering root, where before they
passed. The path is resolved before comparing, because the temp roots are
resolved and /var is a symlink to /private/var on macOS.

## AddRead's branches were only covered from the write side

Two subtests added: a permanent read under a temporary WRITE root, and a narrow
permanent read under a BROAD temporary read root. Both mutation-checked by
removing the corresponding temporary-coverage guard in AddRead — each fails.

The first mutation I ran did not fail its test, and the finding looked wrong: the
pattern being edited occurs in Add, AddRead and AddTemporaryWrite alike, and a
first-match edit had changed Add. Re-run against the line inside AddRead, it
fails as claimed. Recorded because a mutation that misses its target is
indistinguishable from a test that holds.

## Also asserted: the refcount table is empty, not merely invisible

hasReadRoot answers what a caller can see. An entry left at zero in tempReads is
not visible that way and would still leak one map entry per acquire/release
cycle, so the contention test now checks the terminal state directly.

go test -race ./internal/sandbox/ -count=5: clean.
Unrelated and pre-existing here: TestRunDoctorFormatsRedactedProviderDiagnostics
and TestRunDoctorConnectivityProbesProvider exit 3 in this environment.

Origin-Session: local-8cd239 | Claude Code | 12 prompts
Origin-Snapshot: dd397730a138
Origin-Session: local-13d543 | Claude Code | 5 prompts
Origin-Snapshot: 6b5eb8ba4e5b
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: d2b8f44a9abc

* fix(sandbox): a reader outliving its writer keeps read authority, not write

Reported by @jatmn, and it is a privilege escalation rather than a lifetime bug.

AddTemporaryRead, given a path already covered by a temporary WRITE root, took a
reference on that write root and returned an undo that released it. The lifetime
was then correct — the reader no longer lost its access when the writer cleaned
up — but the capability was wrong. extraRoots feeds validate(), which authorises
WRITES, so once the write holder released first the path stayed in extraRoots for
as long as the read holder lived. A caller that asked only to READ could write
anywhere below the former write root after the write grant had ended.

His diagnosis names the root cause exactly: one write-capability reference was
doing two jobs, tracking a lifetime and standing in for a read-only dependent.

The lifetime is now kept as what the caller actually asked for — a temporary READ
root on its own path. AddTemporaryRead consults permanentWriteRootCoversLocked
rather than writeRootCoversLocked, so a merely temporary write cover no longer
short-circuits it into borrowing write authority.

Mutation-checked: restoring writeRootCoversLocked in that branch compiles and
fails the regression on three separate assertions — the reader keeps no read root
of its own, ReadRoots() loses the path, and the writer's cleanup revokes a live
read grant.

Two things a verification pass turned up that are worth recording:

The comment justifying this design said readRoots "feeds ReadRoots() and nothing
else". That was false — it also feeds ExtraReadRoots(). The security argument
survives, because neither confers write authority and validate() reads
workspaceRoot plus extraRoots, which this path never enters. But that sentence IS
the written justification for the whole choice, so it now lists both and says why
neither grants write.

The "the reader releases first" subtest passes on the unfixed tree: the
escalation needs the WRITER to go first. It is a real no-regression control for
the opposite ordering, not evidence the finding is closed, and it is renamed to
say so rather than padding the count.

Rebased onto ad34dc8d as requested. go test -race ./internal/sandbox/ -count=5:
clean. Pre-existing here and on main, unrelated:
TestRunDoctorFormatsRedactedProviderDiagnostics and
TestRunDoctorConnectivityProbesProvider exit 3 in this environment.

Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: d2b8f44a9abc

* fix(sandbox): a nested write holder keeps only the subtree it asked for

Reported by CodeRabbit, and it is the write-side twin of the read-side
escalation @jatmn found — I had disclosed it on this PR as still open, so this
closes it rather than deferring it again.

AddTemporaryWrite, given a path already covered by a broader TEMPORARY write
root, took a reference on that covering root. The lifetime was right and the
capability was wrong, exactly as on the read side: extraRoots is what validate()
authorises writes from, so keeping the covering root alive handed the nested
holder everything its neighbour had asked for.

Reproduced:

  both held           tempWrites={outer:2}                extraRoots=[... outer]
  after outer release tempWrites={outer:1}                extraRoots=[... outer]
  -> the inner-only holder can WRITE outer/sibling/not-mine.txt

A narrower request now records its own root. Each holder gets exactly the
authority it asked for and a lifetime of its own:

  both held           tempWrites={outer:1, inner:1}       extraRoots=[... outer inner]
  after outer release tempWrites={inner:1}                extraRoots=[... inner]
  -> sibling denied, inner retained

Only an EXACT match shares a refcount entry; a narrower one never does, which is
the distinction the escalation turned on. The nesting costs a redundant
extraRoots entry while both holders are live, which is not wrong — the broader
root already permits everything the narrower one does.

Three regressions: the escalation itself (broad holder releases first), the
opposite order as a control, and two holders of the same root sharing one entry.

Mutation: restoring the covering-root borrow compiles and fails the escalation
test on both the sibling and the covering root itself.

Honest note on a second mutation. Removing the permanent-cover early return
compiles and fails NOTHING. That is correct rather than a coverage gap: a
permanently covered request would then record its own temporary root, which is
redundant but harmless, since the permanent root outlives it and already permits
the writes. The early return is a cleanliness guard, not a correctness one, and
I would rather say so than write a test that pretends otherwise.

go test -race ./internal/sandbox/ -count=5: clean. Pre-existing here and on main:
TestRunDoctorFormatsRedactedProviderDiagnostics and
TestRunDoctorConnectivityProbesProvider exit 3 in this environment.

Origin-Session: local-c962d7 | Claude Code | 5 prompts
Origin-Snapshot: 3bb420a0a97b

* test(sandbox): the outside-roots fixture fails loudly instead of skipping

Raised by CodeRabbit, and the helper had already written the argument against
itself: its own comment says "A skip is not a pass" while three of its four exits
were skips.

Every caller of scopeOutsideRoots asserts an AUTHORIZATION boundary — who may
write where, and whose release revokes it. A skipped fixture means that assertion
never ran and the build reported green anyway, which is precisely the failure the
helper was written to close. It closed it for one case (a home directory that
resolves nowhere on Windows) and left the others open.

Three exits become fatal:

  os.UserHomeDir failing        — resolves on every platform Zero supports
  os.MkdirTemp(home) failing    — a home that will not take a temp directory is a
                                  broken machine, not a limited one
  os.MkdirAll under base        — base was just created successfully, so a
                                  subdirectory failing under it is the same

If any fires, the right outcome is a red build somebody fixes, not silent
non-coverage of the sandbox's write rules.

ONE SKIP STAYS, and it is a different kind. When the grant path is already inside
a default write root, the fixture built correctly and the machine is fine — the
grant simply cannot demonstrate anything, because that path is writable before
any temporary grant exists. Running on would assert a permission that was never
in question. The comment now says which kind it is, so the distinction survives
the next reader.

Verified the fatal path fires: forcing the MkdirTemp error compiles and fails
TestANestedWriteHolderKeepsOnlyItsOwnSubtree at the fixture with the reason,
where it previously reported a pass.

go test -race ./internal/sandbox/ -count=3: clean. Pre-existing here and on main:
TestRunDoctorFormatsRedactedProviderDiagnostics and
TestRunDoctorConnectivityProbesProvider exit 3 in this environment.

Origin-Session: local-c962d7 | Claude Code | 13 prompts
Origin-Snapshot: a599377c09e0

* fix(sandbox): least authority for nested readers, and a hermetic grant fixture

@jatmn's three findings, taken as one pass over the grant construct rather than
as three local patches — his point that the same invariant kept not being
reapplied to the symmetric branch was fair, and this is the symmetric branch.

## The nested reader kept its neighbour's tree

AddTemporaryRead had exactly the shape AddTemporaryWrite was fixed for one
commit earlier: reader B asking for outer/inner took a reference on reader A's
covering root. Lifetime correct, capability wrong. readRoots feeds validateRead
and the native sandbox profile, so once A released, B could read outer/sibling.

Reproduced: tempReads={outer:2}; after A releases, the inner-only reader reads a
sibling it never asked for.

The rule is now the same on both sides: permanent coverage decides first, an
EXACT temporary root shares one refcount, and a narrower request records its own
root. Mutation: restoring the borrow fires both escalation assertions.

## Slice order no longer decides permanent coverage

permanentReadRootCoversLocked asks the whole question over both lists before any
insertion order can answer it, and AddRead now uses the same predicate instead of
its own scan, so the two cannot drift.

Honest note: with the borrow gone, this mutation is no longer catchable. Making
the scan return at the first covering entry compiles and fails nothing, because
AddTemporaryRead no longer borrows anything — it either no-ops or records its own
root, and neither escalates. The guard now prevents a redundant refcount entry
rather than an escalation. Verified by measurement, not assumed.

## A test that asserted the defect

TestANarrowerRequestHoldsTheCoveringGrant proved a lifetime property by
asserting the BROAD root was still present after the broad holder released —
which is the escalation written down as a requirement. It is replaced by tests
that assert lifetime and authority separately: the requested path stays readable,
the sibling and the covering root do not, plus the reverse release order,
exact-root sharing, and a permanent narrow root not borrowing a broad temporary
one.

## The obsolete predicate that broke CI

writeRootCoversLocked lost its last caller when the three coverage decisions
moved to permanentWriteRootCoversLocked and exact-root logic. make lint-static
reproduced the Windows `unused` failure locally. Removed rather than suppressed.
This is on me: I did not run lint-static before the previous push, which is a
required step, and CI caught what I should have.

## Fixtures out of the real home

scopeOutsideRoots and homeGrantOutsideTempRoots both built fixtures under
os.UserHomeDir(). Fixed at the cause jatmn identified rather than by relaxing the
setup checks: NewScope seeds the system temp directory as a permanent write root,
which is what made a t.TempDir() grant vacuous and sent the fixture to HOME in the
first place. Both helpers now build the Scope DIRECTLY — the pattern
scopeOutsideDefaults and temporaryWriteFixture already used — so no defaults are
seeded, t.TempDir() is outside by construction, and nothing touches HOME.

Each helper now asserts the NEGATIVE against its own scope before returning: the
target must be unwritable, and unreadable where relevant, before any grant
exists. That is stronger than the old check against defaultTempWriteRoots(),
which asked about a scope these tests no longer build.

Verified with HOME readable but not writable (caches redirected): the sandbox
suite passes. The one remaining failure there is
TestSeatbeltEnforcesExtraWriteRoots in seatbelt_integration_darwin_test.go, which
is NOT in this PR's diff — pre-existing, same shape, and worth its own issue
rather than riding along here.

gofmt, go vet, lint-static (0 issues), go test -race ./internal/sandbox/ -count=5:
all clean. Pre-existing here and on main:
TestRunDoctorFormatsRedactedProviderDiagnostics and
TestRunDoctorConnectivityProbesProvider exit 3 in this environment.

Origin-Session: local-c962d7 | Claude Code | 14 prompts
Origin-Snapshot: a599377c09e0

* refactor(sandbox): drop the child-grant accessors this PR does not consume

Reported by @jatmn.

ExtraRoots and ExtraReadRoots have no production caller anywhere in the tree —
only their own definitions and this PR's tests. They arrived with the initial
split because the extraction was made at the file level: the temporary-root
implementation came across while the CLI and runner wiring that consumes them
stayed behind in #829's child read-grant propagation area. That leaves unrelated
production surface in a branch scoped to refcounting.

Both accessors are removed, along with the getter-only propagation assertion that
existed to exercise them. The refcount regressions that used ExtraRoots only to
inspect beyond-workspace write state now use a test-only helper over Roots(),
because that inspection is a test concern rather than an API this branch needs.

Nothing about the refcount, permanence, least-authority, atomic-release or
hermetic-fixture work changes, and no child propagation is wired here — that
would widen the scope rather than fix the extraction boundary.

One comment needed correcting rather than deleting. The justification for keeping
a reader's lifetime as a READ root named the lists readRoots feeds, and that
sentence has now been wrong in both directions: an earlier draft said "ReadRoots
and nothing else" while ExtraReadRoots also read it, and the accessor it was
corrected to mention is the one being removed here. It now describes what ships
and says why the list has to stay exhaustive as accessors change.

Rebased onto 6fe0d1ed, 0 behind. lint-static: 0 issues.
go test -race ./internal/sandbox/ -count=3: clean. Pre-existing here and on main:
TestRunDoctorFormatsRedactedProviderDiagnostics and
TestRunDoctorConnectivityProbesProvider exit 3 in this environment.
2026-08-26 13:21:25 +08:00
KRATOS 1649008506 fix(tools): an edit keeps the reads it did not disturb (#908)
* fix(tools): an edit keeps the reads it did not disturb

Split out of #829 as an independent fix, per @Vasanthdev2004's review asking for
the small self-contained pieces to arrive separately.

A successful edit re-baselined the file through RecordHash, which drops EVERY
recorded read range. A file read in three pieces therefore lost all three to a
single two-line edit, and the next six edits into regions that had already been
read were refused as unseen — the model had to re-read what it had just read in
order to keep working.

RecordEdit replaces that on the edit path. Because the edit is ours, the changed
span is known exactly, so ranges the edit did not touch are carried across,
ranges after it are shifted by the line delta, and a range the edit spans is
split around it.

An EXTERNAL change still drops everything, deliberately: once a formatter has
rewritten the file we no longer know which line holds what was read, and the
conservative drop is the only honest answer there.

"Seen whole" is now decided in ONE place. SeenWhole derives the answer from the
ranges because the raw flag is only ever set by a single covering read, but
RecordEdit branched on the flag — so a file read whole in two chunks reported
seen-whole true, then false after one edit, putting back the write_file refusal
the derived answer exists to prevent.

countLines now reports the count a READER would give. strings.Split leaves an
empty final element for content ending in a newline, so it counted 4 lines for a
3-line file; since SeenWhole asks whether the ranges cover 1..total, that
inflated total made full coverage unreachable for a file just read in full.
Nearly every text file ends in a newline, so this was the common case.

Origin-Session: local-de382f | Claude Code | 9 prompts
Origin-Snapshot: 92ae33c95cd1
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: ae653221fa15

* chore(tools): re-request an automated review of this head

No code change. CodeRabbit's verdict on this PR is pinned to 6f0cd6c4, a commit
that was force-pushed away five days ago, and a stale changes-requested keeps the
PR at mergeStateStatus=BLOCKED exactly as a human one would. It re-reviews on
push rather than on request, and repeated @coderabbitai triggers over five days
produced nothing, so this is the only lever available without maintainer rights.

The cost is that this dismisses @Vasanthdev2004's and @anandh8x's approvals of
0b6fcf54, which is an unhappy trade for an empty commit — the tree they approved
is byte-identical to this one.

Origin-Session: local-8cd239 | Claude Code | 6 prompts
Origin-Snapshot: 149ab98453ad
Origin-Session: local-c962d7 | Claude Code | 3 prompts
Origin-Snapshot: ae653221fa15

* perf(tools): derive the changed-line span from bytes, not from per-line slices

Reported by @jatmn.

changedLineSpan split both file versions into []string, so it allocated one
string header per line for EACH version — work that scales with the size of the
file rather than the size of the edit. countLines then split a third time just to
take a length. All of it ran before RecordEdit had checked whether there was a
tracked observation to update at all.

He measured 32,036,640 bytes for two 2 MB versions of a million short lines, and
extrapolated roughly 1.6 GB of headers for a 100 MB file before counting the
content, the updated bytes and the hashes already live beside them.

The memory was not the sharp end. edit_file writes the updated bytes BEFORE
calling RecordEdit, so an out-of-memory kill lands after the user's file has
changed and before the tracker baseline catches up: the file and the record of it
disagree, and nothing says so. His instruction was to fix the allocation source
rather than add a size guard, and that a guard — if used at all — would have to
reject before os.WriteFile rather than during post-write re-baselining. Taken as
written: there is no new limit, and the same edits are accepted as before.

Two identical byte prefixes share every line that ends inside them, so counting
newlines in the common prefix counts the unchanged leading lines directly. Same
for the suffix, with one rule that the line-array version got for free: the line
a shared tail STARTS in is only shared when the tail begins on a line boundary in
both versions, or its opening bytes differ and it is a changed line that merely
ends the same way.

Measured on his own scenario, two 2 MB versions:

  changedLineSpan   32,036,640 bytes  ->  0 allocations
  countLines        per-line slice    ->  0 allocations

## The equivalence is tested, not asserted

A rewrite of a diff primitive is only safe if it is behaviour-preserving, so the
replaced implementation is kept in the test file as an oracle and both are asked
the same questions. That caught 206 mismatches in my first attempt at the byte
scan — empty versions, and trailing lines that are only partially shared. Neither
appears in the hand-written cases, and I would have shipped it.

Two regressions: the oracle comparison, and an absolute allocation ceiling rather
than a ratio, so a future rewrite that reintroduces per-line slices fails rather
than merely getting slower. Mutation: restoring strings.Split in either function
allocates 3.2 MB and 3.6 MB against a 4 KB bound.

splitLinesForTracking lost its last caller and is removed — the same obsolete-
helper lint failure that broke Windows CI on #911, caught here by running
make lint-static before pushing rather than after. 0 issues.

Rebased onto ad34dc8d, 0 behind. go test -race ./internal/tools/: clean.
Pre-existing here and on main: TestRunDoctorFormatsRedactedProviderDiagnostics
and TestRunDoctorConnectivityProbesProvider exit 3 in this environment.

Origin-Session: local-c962d7 | Claude Code | 17 prompts
Origin-Snapshot: a599377c09e0
2026-08-26 13:20:48 +08:00
KRATOS 6fe0d1edcd fix(lockutil): use kernel-held advisory locks (#950)
* fix(lockutil): use kernel-held advisory locks

* fix(lockutil): harden cross-platform lock files

* test(lockutil): require one concurrent holder
2026-08-25 21:20:18 +08:00
e59bf78cab fix(sandbox): Make WSL backend tests deterministic. (#948)
Co-authored-by: euxaristia <[email protected]>
Co-authored-by: cairn-code <[email protected]>
2026-08-25 21:19:31 +08:00
Kevin CodexandClaude Fable 5 6645f9744b perf(agent): cheap compaction — dedicated summarizer model, input cap… (#532)
* perf(agent): cheap compaction — dedicated summarizer model, input caps, free reactive prune, window re-derive

Compaction summarization was the single most expensive recurring event
in a long run: the full elided middle — verbatim tool bodies included —
went to the session's main model at full price (a ~100k-token call near
a 200k window, recursively multiplied on overflow).

Four changes, each independently safe:

- Dedicated summarizer model. Compaction summaries route to a cheap
  model resolved as ZERO_COMPACTION_MODEL > preferences.compactionModel
  > a curated default on official endpoints (Haiku 4.5 / Gemini
  Flash-Lite / gpt-4.1-mini; custom-endpoint catalogs are unknowable so
  they get none). Built lazily on the first paid compaction; ANY
  failure permanently falls back to the main provider for the run, so a
  misconfigured summarizer can never break compaction. "main" at any
  level forces the old behavior.

- Summarizer input caps. renderTranscript clamps tool-result bodies to
  2,000 chars and tool-call arguments to 500 (head+tail, rune-safe) —
  the summary needs what was done and decided, not raw payloads. Cuts
  summarizer input 5-10x on tool-heavy sessions.

- Free reactive prune. recover() now runs the same prune-first stage as
  the proactive path: a context-limit error is first answered by
  pruning stale tool bodies at zero cost, without consuming the
  one-shot reactive budget; the paid summarizer only runs when a later
  error finds nothing left to prune.

- Window re-derivation on model switch. Options.ContextWindowFor
  (wired to the model registry in exec and the TUI) lets the loop
  re-derive the compaction threshold after escalate_model, closing the
  documented KNOWN LIMITATION where a switch to a smaller-window model
  could overflow and a larger one over-compacted.

## Rebase notes (2026-08-25, onto 2749447c)

Rebased across 214 commits. main now summarizes through the context planner
(#879) and switches models via turn sessions, so the agent side was reworked
rather than merged textually:

- Summarizer selection lives on compactionState and is resolved lazily per
  main model through summarizeWithPlanner (planner + trace preserved); a
  failing dedicated summarizer is marked broken and the same slice is
  retried once on the main provider.
- Options.Summarizer now receives the main model in force:
  `func(ctx, mainModelID string) (Provider, error)`, and may return
  (nil, nil) when no dedicated model applies. After a successful
  escalate_model switch the loop calls compactor.switchModel, which
  re-derives the window and re-resolves the summarizer — a run that started
  on the cheap model and escalated regains cheap summaries (review P2).
- recover()'s free prune stage runs before the one-shot reactive guard
  (pruning is free and cannot loop) and mirrors maybeCompact's two prune
  passes; the free retry is taken only when the pruned history fits the
  window the provider enforces (with a 5% estimate margin), otherwise the
  same call falls through to the paid summarizer. Gating on the 70%
  proactive threshold would have sent histories in the 70-95% band to the
  summarizer unnecessarily (review P1 and the follow-up question).
- ContextWindowFor is wired through the turn-session switch path; the
  planner's window is updated alongside the compactor's.

Tests: the PR's suite is ported to the new seams, plus insufficient-prune
fall-through, free prune after the budget is spent, window-vs-threshold
gating, and summarizer re-resolution across a model switch.

* fix(agent): branch on a dedicated-summarizer flag instead of comparing providers

Review follow-ups for #532:

- summarizeProvider now reports whether the dedicated summarizer was
  selected and summarizeClosure branches on that flag; the previous
  `chosen == provider` compared Provider interface values, which panics
  when the dynamic type is not comparable. Regression test uses a provider
  with a slice field for both the summarizer and the main provider.
- CompactionModelID's curated-default test covers the official OpenAI
  profile alongside Anthropic and Google.

Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq

---------

Co-authored-by: Claude Fable 5 <[email protected]>
2026-08-25 21:18:04 +08:00
214 changed files with 24041 additions and 1625 deletions

No files matched your search

+7 -3
View File
@@ -371,9 +371,13 @@ go run ./cmd/zero-perf-bench
Experimental: `ZERO_OPENAI_TURN_SESSION=1` enables the optimized OpenAI turn
session (background connection prewarm + request-prefix telemetry) for headless
`zero exec` runs against official OpenAI profiles. Off by default; `0`/`false`
disable. A/B-benchmark it by running the same `zero-perf-bench` suite with the
variable unset and set.
`zero exec` runs against official OpenAI profiles. Off by default; `0`, `false`,
or `off` disable it. A/B-benchmark it by running the same `zero-perf-bench` suite
with the variable unset and set.
Native ChatGPT Responses sessions are enabled by default. Set
`ZERO_CHATGPT_TURN_SESSION=0`, `false`, or `off` to restore stateless HTTP/SSE
transport.
### Code Quality and Security Checks
+1 -1
View File
@@ -4,7 +4,7 @@ go 1.26.6
require (
charm.land/bubbles/v2 v2.1.1
charm.land/bubbletea/v2 v2.0.8
charm.land/bubbletea/v2 v2.0.9
charm.land/lipgloss/v2 v2.0.5
github.com/Microsoft/go-winio v0.6.2
github.com/alecthomas/chroma/v2 v2.27.0
+2 -4
View File
@@ -1,7 +1,7 @@
charm.land/bubbles/v2 v2.1.1 h1:7r55WzBxpo/R3z98hGmY7KKPd3ET6vsf0Fb9sDHOV60=
charm.land/bubbles/v2 v2.1.1/go.mod h1:GE6M31gaWZVXzGw73OeuTTgy4lX+OtkH0E5ymnNsHxo=
charm.land/bubbletea/v2 v2.0.8 h1:SxTJMhCAI3lbPmy4SgX5LWZ24AdINr4I6UEqzZvYJuY=
charm.land/bubbletea/v2 v2.0.8/go.mod h1:2SkdgoTXluXJHOUwAoRlRXF/28vklb1rFl6GcgV1/ss=
charm.land/bubbletea/v2 v2.0.9 h1:DpJCMWKgzQK8SJv4zbKKFHAI10ymWy/evClPFk0k0f8=
charm.land/bubbletea/v2 v2.0.9/go.mod h1:2SkdgoTXluXJHOUwAoRlRXF/28vklb1rFl6GcgV1/ss=
charm.land/lipgloss/v2 v2.0.5 h1:kbNxgeeUOYv5J0YdpxFjfvf3dFvqH8Aci4zB6xqFtrY=
charm.land/lipgloss/v2 v2.0.5/go.mod h1:9oqhxt4yxIMe6q5A4kHr44DremZk7J9UNh74GlWa5nc=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
@@ -64,8 +64,6 @@ github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavM
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
golang.org/x/exp v0.0.0-20260611194520-c48552f49976 h1:X8Hz2ImujgbmetVuW+w2YkyZChE3cBpZi2P158rTG9M=
golang.org/x/exp v0.0.0-20260611194520-c48552f49976/go.mod h1:vnf4pv9iKZXY58sQE1L86zmNWJ4159e1RkcWiLCkeEY=
golang.org/x/image v0.44.0 h1:+tDekMZED9+LrtB3G5xzRggpVh9CARjZqROla3R3R+I=
golang.org/x/image v0.44.0/go.mod h1:V8K3KE9KKKE+pLpQDOeN18w9oacNSvy1tDOirTu4xtY=
golang.org/x/image v0.45.0 h1:FMb1nTbH5H9vF55SriQHgFw5GnNL9Jg6L25BwXKzhB0=
golang.org/x/image v0.45.0/go.mod h1:n62x/7RqlwXDvGsSU4u6IUTUf6KghUZ9Bt7cG/T9Fx4=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
+178 -11
View File
@@ -4,7 +4,9 @@ import (
"context"
"encoding/json"
"errors"
"strconv"
"strings"
"unicode/utf8"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
@@ -144,11 +146,17 @@ func estimateToolDefTokens(tools []zeroruntime.ToolDefinition) int {
for _, tool := range tools {
total += ApproxTextTokens(tool.Name)
total += ApproxTextTokens(tool.Description)
total += ApproxTextTokens(string(tool.Type))
if len(tool.Parameters) > 0 {
if encoded, err := json.Marshal(tool.Parameters); err == nil {
total += ApproxTextTokens(string(encoded))
}
}
if tool.Format != nil {
total += ApproxTextTokens(tool.Format.Type)
total += ApproxTextTokens(tool.Format.Syntax)
total += ApproxTextTokens(tool.Format.Definition)
}
total += 4 // per-tool overhead
}
return total
@@ -390,6 +398,27 @@ type compactionState struct {
// starts at 1.0 and converges via an EMA as each turn reports actual usage, so
// later turns compact nearer to real capacity. Zero is treated as 1.0.
calibrationRatio float64
// summarizerFactory lazily builds the dedicated (cheap) summarizer provider
// (Options.Summarizer) for the current main model. summarizerProvider
// memoizes the built provider and summarizerResolved records that the
// factory ran (it may legitimately return no provider: the main model is
// already the cheap one). summarizerBroken is the sticky fallback: once a
// build or call fails, the run stays on the main provider until the next
// model switch — a misconfigured cheap model must never break (or
// repeatedly delay) compaction.
summarizerFactory func(ctx context.Context, mainModelID string) (Provider, error)
summarizerProvider Provider
summarizerResolved bool
summarizerBroken bool
// window is the context window the threshold was derived from. The
// reactive free-prune stage compares against it (the limit the provider
// enforces), not against the 70% proactive threshold — a pruned history
// between the two would still succeed on retry.
window int
// mainModel is the model the run currently streams on; the summarizer
// choice is made against it and remade after a mid-run switch.
mainModel string
}
// calibrate folds one turn's (rawEstimate, actualPromptTokens) sample into the
@@ -424,17 +453,88 @@ func (state *compactionState) calibratedTokens(raw int) int {
func newCompactionState(options Options, task *taskState) *compactionState {
state := &compactionState{
enabled: options.ContextWindow > 0,
threshold: compactionThreshold(options.ContextWindow),
preserveLast: options.CompactionPreserveLast,
onUsage: options.OnUsage,
task: task,
planner: newContextPlanner(contextPlannerConfig{contextWindow: options.ContextWindow, serviceTier: options.ServiceTier}),
trace: options.Trace,
enabled: options.ContextWindow > 0,
threshold: compactionThreshold(options.ContextWindow),
window: options.ContextWindow,
mainModel: options.Model,
preserveLast: options.CompactionPreserveLast,
onUsage: options.OnUsage,
task: task,
planner: newContextPlanner(contextPlannerConfig{contextWindow: options.ContextWindow, serviceTier: options.ServiceTier}),
trace: options.Trace,
summarizerFactory: options.Summarizer,
}
return state
}
// updateWindow re-derives the compaction threshold after a mid-run model
// switch. The low-water mark is reset: it was measured against the old
// threshold, and keeping it could suppress a compaction the smaller window
// now requires. <= 0 (unknown model) leaves everything unchanged.
func (state *compactionState) updateWindow(window int) {
if window <= 0 {
return
}
state.enabled = true
state.window = window
state.threshold = compactionThreshold(window)
state.lowWaterMark = 0
if state.planner != nil {
state.planner.config.contextWindow = window
}
}
// switchModel records a mid-run model switch: the window is re-derived
// (window <= 0 keeps the old one) and the summarizer choice is forgotten so
// the next compaction re-resolves it against the new main model. A run that
// started on the cheap model (no dedicated summarizer) and escalated to an
// expensive one therefore regains cheap summaries — the exact workflow a
// dedicated summarizer exists for.
func (state *compactionState) switchModel(modelID string, window int) {
state.mainModel = modelID
state.updateWindow(window)
state.summarizerProvider = nil
state.summarizerResolved = false
state.summarizerBroken = false
}
// summarizeProvider returns the provider summarization calls should use and
// whether it is the dedicated summarizer: that one when it applies to the
// current main model and is healthy, else the main provider. The factory runs
// at most once per main model; a build error marks the summarizer broken for
// that model. Callers branch on the boolean rather than comparing Provider
// values — interface comparison panics on a non-comparable dynamic type.
func (state *compactionState) summarizeProvider(ctx context.Context, main Provider) (Provider, bool) {
if state.summarizerFactory == nil || state.summarizerBroken {
return main, false
}
if !state.summarizerResolved {
state.summarizerResolved = true
built, err := state.summarizerFactory(ctx, state.mainModel)
if err != nil {
state.summarizerBroken = true
return main, false
}
state.summarizerProvider = built
}
if state.summarizerProvider == nil {
return main, false
}
return state.summarizerProvider, true
}
// fitsWindow reports whether a calibrated size is safely under the limit the
// provider enforces. The proactive threshold (70% of the window) is a trigger
// for the paid summarizer, not the hard limit; the reactive path compares
// against the window itself, with a small margin for estimate error, so a
// history pruned to, say, 85% of the window takes the free retry.
func (state *compactionState) fitsWindow(size int) bool {
if state.window <= 0 {
return size <= state.threshold
}
return size <= state.window-state.window/20
}
// maybeCompact runs proactive compaction at the top of a turn. It returns the
// (possibly compacted) message slice. It is a no-op when compaction is disabled,
// when the history is under threshold, or when the history has not grown past
@@ -528,10 +628,34 @@ func (state *compactionState) recover(
// context-limit error never triggers an unexpected summarization call.
return messages, false, nil
}
if state.reactiveAttempted {
if !isContextLimitError(errorMessage) {
return messages, false, nil
}
if !isContextLimitError(errorMessage) {
// CHEAP FIRST STAGE (mirrors maybeCompact): a context-limit failure can
// often be fixed for free by dropping superseded reads and pruning old
// tool-result bodies. It runs regardless of the one-shot reactive budget
// (it costs nothing and cannot loop: a second pass finds nothing left to
// prune). The free retry is taken only when the pruned history fits the
// window the provider enforces — otherwise a retry would fail the same
// way, so fall through to the paid summarizer in this same call.
toolTokens := estimateToolDefTokens(tools)
prunedAny := false
if pruned, reclaimed := pruneSupersededReadResults(messages); reclaimed > 0 {
messages, prunedAny = pruned, true
}
if pruned, reclaimed := pruneStaleToolOutput(messages, state.preserveLast); reclaimed > 0 {
messages, prunedAny = pruned, true
}
if prunedAny {
size := state.calibratedTokens(estimateTokens(messages) + toolTokens)
if state.fitsWindow(size) {
state.lowWaterMark = size
return messages, true, nil
}
}
if state.reactiveAttempted {
return messages, false, nil
}
@@ -571,11 +695,21 @@ func (state *compactionState) recover(
// provider call. The summary stream intentionally does NOT forward OnText (so
// compaction stays invisible on the user-facing surface), but it DOES forward
// OnUsage so the summarizer's token cost is still counted by usage/budgeting.
// The call goes to the dedicated (cheap) summarizer when one applies; if that
// call fails the summarizer is marked broken for the current model and the
// same slice is retried once on the main provider, so the caller sees at most
// one (main-provider-quality) error.
func (state *compactionState) summarizeClosure(ctx context.Context, provider Provider) func([]zeroruntime.Message) (string, error) {
if state.planner == nil {
state.planner = newContextPlanner(contextPlannerConfig{})
}
return func(toSummarize []zeroruntime.Message) (string, error) {
chosen, dedicated := state.summarizeProvider(ctx, provider)
summary, err := summarizeWithPlanner(ctx, chosen, toSummarize, state.onUsage, state.planner, state.trace)
if err == nil || !dedicated {
return summary, err
}
state.summarizerBroken = true
return summarizeWithPlanner(ctx, provider, toSummarize, state.onUsage, state.planner, state.trace)
}
}
@@ -659,8 +793,41 @@ func summarizeMessagesOnce(ctx context.Context, provider Provider, messages []ze
return summary, nil
}
// summaryToolResultClamp bounds each tool-result body in the summarizer input.
// Tool outputs dominate the elided middle's bulk (build logs, file reads,
// search results), yet the summary instructions only need their gist — the
// model that ACTED on the full output already turned it into decisions the
// surrounding messages record. 2,000 chars (~500 tokens) keeps the useful
// head+tail of a result while cutting the summarizer input by 5-10x on
// tool-heavy sessions; the summarizer's own cost is the price being cut.
const summaryToolResultClamp = 2000
// summaryToolArgsClamp bounds each tool-call argument blob in the transcript:
// a write_file call carries the whole file body in its arguments.
const summaryToolArgsClamp = 500
// clampForSummary head+tail-truncates text for the summarizer transcript,
// snapping to rune boundaries so the clamp never splits a UTF-8 sequence.
func clampForSummary(text string, max int) string {
if max <= 0 || len(text) <= max {
return text
}
head := max * 3 / 4
tail := max - head
for head > 0 && !utf8.RuneStart(text[head]) {
head--
}
start := len(text) - tail
for start < len(text) && !utf8.RuneStart(text[start]) {
start++
}
return text[:head] + "\n…[" + strconv.Itoa(len(text)-head-(len(text)-start)) + " chars omitted for summarization]…\n" + text[start:]
}
// renderTranscript flattens messages into a plain-text transcript for the
// summarizer. Secret scrubbing already happened upstream at the tool boundary.
// Tool-result bodies and tool-call arguments are clamped (see the constants
// above): the summary needs what was done and decided, not the raw payloads.
func renderTranscript(messages []zeroruntime.Message) string {
lines := make([]string, 0, len(messages))
for _, message := range messages {
@@ -670,13 +837,13 @@ func renderTranscript(messages []zeroruntime.Message) string {
if len(message.ToolCalls) > 0 {
calls := make([]string, 0, len(message.ToolCalls))
for _, call := range message.ToolCalls {
calls = append(calls, call.Name+"("+call.Arguments+")")
calls = append(calls, call.Name+"("+clampForSummary(call.Arguments, summaryToolArgsClamp)+")")
}
line += "\n[tool calls: " + strings.Join(calls, "; ") + "]"
}
lines = append(lines, line)
case zeroruntime.MessageRoleTool:
lines = append(lines, "tool result: "+message.Content)
lines = append(lines, "tool result: "+clampForSummary(message.Content, summaryToolResultClamp))
default:
lines = append(lines, string(message.Role)+": "+message.Content)
}
+469
View File
@@ -0,0 +1,469 @@
package agent
import (
"context"
"errors"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// compactableHistory returns a history whose middle is large enough to trip
// the proactive threshold for a 1000-token window.
func compactableHistory() []zeroruntime.Message {
return []zeroruntime.Message{
{Role: zeroruntime.MessageRoleSystem, Content: "sys"},
{Role: zeroruntime.MessageRoleUser, Content: strings.Repeat("u", 4000)},
{Role: zeroruntime.MessageRoleAssistant, Content: strings.Repeat("a", 4000)},
{Role: zeroruntime.MessageRoleUser, Content: "u2"},
{Role: zeroruntime.MessageRoleAssistant, Content: "a2"},
{Role: zeroruntime.MessageRoleUser, Content: "u3"},
}
}
// Summarization must run on the dedicated (cheap) summarizer when one is
// configured — the main provider sees no summarize traffic at all.
func TestMaybeCompactUsesDedicatedSummarizer(t *testing.T) {
cheap := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventText, Content: "CHEAP SUMMARY"},
{Type: zeroruntime.StreamEventDone},
}}}
main := &mockProvider{}
factoryCalls := 0
state := newCompactionState(Options{
ContextWindow: 1000,
CompactionPreserveLast: 2,
Summarizer: func(context.Context, string) (Provider, error) {
factoryCalls++
return cheap, nil
},
}, nil)
compacted := state.maybeCompact(context.Background(), main, compactableHistory(), nil)
if len(cheap.requests) == 0 {
t.Fatal("dedicated summarizer never received the summarize call")
}
if len(main.requests) != 0 {
t.Fatalf("main provider must see no summarize traffic, got %d requests", len(main.requests))
}
if factoryCalls != 1 {
t.Fatalf("summarizer factory calls = %d, want 1 (lazy, memoized)", factoryCalls)
}
if estimateTokens(compacted) >= estimateTokens(compactableHistory()) {
t.Fatal("compaction did not shrink the history")
}
found := false
for _, message := range compacted {
if strings.Contains(message.Content, "CHEAP SUMMARY") {
found = true
}
}
if !found {
t.Fatal("cheap summarizer's summary missing from the compacted history")
}
}
// A failing summarizer falls back to the main provider for the SAME slice and
// stays broken for the rest of the run — no repeated failed cheap calls.
func TestSummarizerFailureFallsBackToMainAndSticks(t *testing.T) {
broken := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventError, Error: "boom: model not found"},
}}}
main := &mockProvider{turns: [][]zeroruntime.StreamEvent{
{{Type: zeroruntime.StreamEventText, Content: "MAIN SUMMARY"}, {Type: zeroruntime.StreamEventDone}},
{{Type: zeroruntime.StreamEventText, Content: "MAIN SUMMARY 2"}, {Type: zeroruntime.StreamEventDone}},
}}
state := newCompactionState(Options{
ContextWindow: 1000,
CompactionPreserveLast: 2,
Summarizer: func(context.Context, string) (Provider, error) { return broken, nil },
}, nil)
summarize := state.summarizeClosure(context.Background(), main)
input := []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "history"}}
summary, err := summarize(input)
if err != nil {
t.Fatalf("fallback must succeed via the main provider: %v", err)
}
if summary != "MAIN SUMMARY" {
t.Fatalf("summary = %q, want the main provider's", summary)
}
if len(broken.requests) != 1 {
t.Fatalf("broken summarizer requests = %d, want exactly 1", len(broken.requests))
}
if !state.summarizerBroken {
t.Fatal("a failed summarizer must be marked broken for the run")
}
// Second summarization: straight to main, no cheap retry.
if _, err := summarize(input); err != nil {
t.Fatal(err)
}
if len(broken.requests) != 1 {
t.Fatalf("broken summarizer was retried (%d requests)", len(broken.requests))
}
if len(main.requests) != 2 {
t.Fatalf("main provider requests = %d, want 2", len(main.requests))
}
}
// A factory that errors (misconfigured model) is equivalent to no summarizer:
// main provider used, factory not retried.
func TestSummarizerFactoryErrorFallsBack(t *testing.T) {
main := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventText, Content: "MAIN SUMMARY"},
{Type: zeroruntime.StreamEventDone},
}}}
factoryCalls := 0
state := newCompactionState(Options{
ContextWindow: 1000,
CompactionPreserveLast: 2,
Summarizer: func(context.Context, string) (Provider, error) {
factoryCalls++
return nil, errors.New("unknown model")
},
}, nil)
summarize := state.summarizeClosure(context.Background(), main)
summary, err := summarize([]zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "history"}})
if err != nil || summary != "MAIN SUMMARY" {
t.Fatalf("summary = %q, err = %v; want main provider fallback", summary, err)
}
if factoryCalls != 1 {
t.Fatalf("factory calls = %d, want 1 (sticky failure)", factoryCalls)
}
if _, dedicated := state.summarizeProvider(context.Background(), main); dedicated {
t.Fatal("after a factory failure the run must stay on the main provider")
}
if factoryCalls != 1 {
t.Fatalf("factory retried after failure (%d calls)", factoryCalls)
}
}
// Tool payloads are clamped in the summarizer transcript: results to
// summaryToolResultClamp, call arguments to summaryToolArgsClamp.
func TestRenderTranscriptClampsToolPayloads(t *testing.T) {
bigResult := strings.Repeat("r", 50_000)
bigArgs := strings.Repeat("w", 20_000)
transcript := renderTranscript([]zeroruntime.Message{
{Role: zeroruntime.MessageRoleAssistant, Content: "editing", ToolCalls: []zeroruntime.ToolCall{
{ID: "c1", Name: "write_file", Arguments: bigArgs},
}},
{Role: zeroruntime.MessageRoleTool, ToolCallID: "c1", Content: bigResult},
{Role: zeroruntime.MessageRoleUser, Content: "keep going"},
})
if len(transcript) > summaryToolResultClamp+summaryToolArgsClamp+1024 {
t.Fatalf("transcript is %d bytes; clamps did not apply", len(transcript))
}
if !strings.Contains(transcript, "chars omitted for summarization") {
t.Fatal("clamped transcript must carry the omission marker")
}
if !strings.Contains(transcript, "keep going") {
t.Fatal("non-tool content must survive unclamped")
}
// Head AND tail of the clamped payloads survive.
if !strings.HasPrefix(transcript[strings.Index(transcript, "tool result: ")+len("tool result: "):], "r") ||
!strings.Contains(transcript, "rrr\n\nuser: keep going") {
t.Fatalf("head/tail of the tool result lost:\n%s", transcript[:200])
}
}
// bigToolHistory returns a history with n large tool results (~40k estimated
// tokens each): the trailing protection window keeps the newest verbatim,
// the older ones are prunable.
func bigToolHistory(n int) []zeroruntime.Message {
bigBody := strings.Repeat("x", 160_000)
messages := []zeroruntime.Message{{Role: zeroruntime.MessageRoleSystem, Content: "sys"}}
for i := 0; i < n; i++ {
id := string(rune('a' + i))
messages = append(messages,
zeroruntime.Message{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{{ID: id, Name: "read_file", Arguments: "{\"path\":\"" + id + "\"}"}}},
zeroruntime.Message{Role: zeroruntime.MessageRoleTool, ToolCallID: id, Content: bigBody},
)
}
return append(messages, zeroruntime.Message{Role: zeroruntime.MessageRoleUser, Content: "go"})
}
func summaryProvider(text string) *mockProvider {
return &mockProvider{turns: [][]zeroruntime.StreamEvent{
{{Type: zeroruntime.StreamEventText, Content: text}, {Type: zeroruntime.StreamEventDone}},
{{Type: zeroruntime.StreamEventText, Content: text}, {Type: zeroruntime.StreamEventDone}},
}}
}
// A context-limit error is first answered with the free prune stage; when the
// pruned history fits the window the retry is free (no summarizer call) and
// the one-shot reactive budget is untouched. The paid summarizer only runs
// when a later error finds nothing left to prune.
func TestRecoverPrunesBeforeSummarizing(t *testing.T) {
provider := summaryProvider("SUMMARY")
// The protected trailing tool output (~40k tokens) must fit the window.
state := newCompactionState(Options{ContextWindow: 200_000, CompactionPreserveLast: 2}, nil)
messages := bigToolHistory(5)
pruned, retried, err := state.recover(context.Background(), provider, messages, nil, "maximum context length exceeded")
if err != nil {
t.Fatal(err)
}
if !retried {
t.Fatal("prunable history that fits must produce a free retry")
}
if len(provider.requests) != 0 {
t.Fatalf("free prune stage must not call the summarizer (%d requests)", len(provider.requests))
}
if state.reactiveAttempted {
t.Fatal("a free prune retry must not consume the reactive budget")
}
if estimateTokens(pruned) >= estimateTokens(messages) {
t.Fatal("prune did not shrink the history")
}
// Second context-limit error on the pruned history: nothing meaningfully
// prunable remains, so the paid summarizer fires and consumes the budget.
_, retried, err = state.recover(context.Background(), provider, pruned, nil, "maximum context length exceeded")
if err != nil {
t.Fatal(err)
}
if !retried {
t.Fatal("second recover must compact and retry")
}
if len(provider.requests) == 0 {
t.Fatal("second recover must reach the summarizer")
}
if !state.reactiveAttempted {
t.Fatal("a paid recover must consume the reactive budget")
}
}
// When pruning reclaims something but the pruned history still exceeds the
// window, recover falls through to the paid summarizer in the SAME call
// instead of spending a retry that would fail the same way.
func TestRecoverFallsThroughToSummarizerWhenPruneIsInsufficient(t *testing.T) {
provider := summaryProvider("SUMMARY")
// A 1,000-token window: even the protected trailing output overflows it.
state := newCompactionState(Options{ContextWindow: 1000, CompactionPreserveLast: 2}, nil)
messages := bigToolHistory(5)
compacted, retried, err := state.recover(context.Background(), provider, messages, nil, "maximum context length exceeded")
if err != nil {
t.Fatal(err)
}
if !retried {
t.Fatal("insufficient prune must still end in a retry via summarization")
}
if len(provider.requests) == 0 {
t.Fatal("insufficient prune must reach the summarizer in the same recover call")
}
if !state.reactiveAttempted {
t.Fatal("the paid summarizer consumes the reactive budget")
}
if estimateTokens(compacted) >= estimateTokens(messages) {
t.Fatal("recover did not shrink the history")
}
}
// The free prune stage is not gated behind the one-shot budget: after a paid
// reactive compaction, a later context-limit error still gets a free retry
// when pruning makes the history fit.
func TestRecoverFreePruneRunsAfterBudgetIsSpent(t *testing.T) {
provider := summaryProvider("SUMMARY")
state := newCompactionState(Options{ContextWindow: 200_000, CompactionPreserveLast: 2}, nil)
state.reactiveAttempted = true
messages := bigToolHistory(5)
pruned, retried, err := state.recover(context.Background(), provider, messages, nil, "maximum context length exceeded")
if err != nil {
t.Fatal(err)
}
if !retried || len(provider.requests) != 0 {
t.Fatalf("free prune must retry without the summarizer even when the budget is spent (retried=%v, requests=%d)", retried, len(provider.requests))
}
if estimateTokens(pruned) >= estimateTokens(messages) {
t.Fatal("prune did not shrink the history")
}
// With nothing left to prune and the budget spent, the caller keeps its
// original error — recover neither loops nor summarizes.
_, retried, err = state.recover(context.Background(), provider, pruned, nil, "maximum context length exceeded")
if err != nil || retried || len(provider.requests) != 0 {
t.Fatalf("spent budget must not summarize again (retried=%v, requests=%d, err=%v)", retried, len(provider.requests), err)
}
}
// The free-retry gate compares against the window the provider enforces, not
// the 70% proactive threshold: a history pruned to ~80% of the window still
// takes the free retry.
func TestRecoverFreeRetryGateUsesWindowNotThreshold(t *testing.T) {
provider := summaryProvider("SUMMARY")
messages := bigToolHistory(5)
pruned, _ := pruneStaleToolOutput(messages, 2)
prunedTokens := estimateTokens(pruned)
// Choose a window so the pruned size sits between 70% and 95% of it.
window := prunedTokens * 100 / 80
state := newCompactionState(Options{ContextWindow: window, CompactionPreserveLast: 2}, nil)
if prunedTokens <= state.threshold {
t.Fatalf("test setup: pruned size %d must exceed the proactive threshold %d", prunedTokens, state.threshold)
}
_, retried, err := state.recover(context.Background(), provider, messages, nil, "maximum context length exceeded")
if err != nil {
t.Fatal(err)
}
if !retried || len(provider.requests) != 0 {
t.Fatalf("a history that fits the window must take the free retry (retried=%v, requests=%d)", retried, len(provider.requests))
}
}
// After a mid-run model switch the summarizer is re-resolved against the new
// main model: a run that started on the cheap model (no dedicated summarizer)
// and escalated gets a cheap summarizer, and a previously broken one is
// retried for the new model.
func TestSwitchModelReresolvesSummarizer(t *testing.T) {
cheap := summaryProvider("CHEAP SUMMARY")
main := summaryProvider("MAIN SUMMARY")
var factoryModels []string
state := newCompactionState(Options{
ContextWindow: 1000,
CompactionPreserveLast: 2,
Model: "cheap-model",
Summarizer: func(_ context.Context, mainModelID string) (Provider, error) {
factoryModels = append(factoryModels, mainModelID)
if mainModelID == "cheap-model" {
return nil, nil // main already is the cheap model
}
return cheap, nil
},
}, nil)
if _, dedicated := state.summarizeProvider(context.Background(), main); dedicated {
t.Fatal("on the cheap model the main provider must summarize")
}
if _, dedicated := state.summarizeProvider(context.Background(), main); dedicated || len(factoryModels) != 1 {
t.Fatalf("factory must run once per main model, calls=%v", factoryModels)
}
state.switchModel("big-model", 400_000)
if state.threshold != compactionThreshold(400_000) || state.window != 400_000 {
t.Fatalf("switchModel must re-derive the window, got threshold=%d window=%d", state.threshold, state.window)
}
if got, dedicated := state.summarizeProvider(context.Background(), main); !dedicated || got != Provider(cheap) {
t.Fatal("after escalating to an expensive model the dedicated cheap summarizer must be used")
}
if len(factoryModels) != 2 || factoryModels[1] != "big-model" {
t.Fatalf("factory must be re-invoked with the new main model, calls=%v", factoryModels)
}
// A broken summarizer is retried after the next switch.
state.summarizerBroken = true
if _, dedicated := state.summarizeProvider(context.Background(), main); dedicated {
t.Fatal("a broken summarizer must fall back to main")
}
state.switchModel("other-model", 0)
if got, dedicated := state.summarizeProvider(context.Background(), main); !dedicated || got != Provider(cheap) {
t.Fatal("a model switch must clear the broken flag and rebuild the summarizer")
}
}
func TestCompactionUpdateWindow(t *testing.T) {
state := newCompactionState(Options{ContextWindow: 100_000, CompactionPreserveLast: 2}, nil)
state.lowWaterMark = 50_000
before := state.threshold
state.updateWindow(0) // unknown model: unchanged
if state.threshold != before || state.lowWaterMark != 50_000 {
t.Fatal("updateWindow(0) must be a no-op")
}
state.updateWindow(10_000)
if state.threshold >= before {
t.Fatalf("threshold = %d, want re-derived below %d", state.threshold, before)
}
if state.lowWaterMark != 0 {
t.Fatal("low-water mark must reset on a window change")
}
if !state.enabled {
t.Fatal("a positive window must keep compaction enabled")
}
}
// End-to-end: after escalate_model switches providers, the loop re-derives the
// context window via ContextWindowFor, visible in the OnContext report.
func TestRunEscalationRederivesContextWindow(t *testing.T) {
registry := tools.NewRegistry()
registry.Register(escalatingTool{target: "big-model"})
switched := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventText, Content: "done"},
{Type: zeroruntime.StreamEventDone},
}}}
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventToolCallStart, ToolCallID: "call-1", ToolName: "escalate"},
{Type: zeroruntime.StreamEventToolCallDelta, ToolCallID: "call-1", ArgumentsFragment: `{}`},
{Type: zeroruntime.StreamEventToolCallEnd, ToolCallID: "call-1"},
{Type: zeroruntime.StreamEventDone},
}}}
var windows []int
result, err := Run(context.Background(), "go", provider, Options{
Registry: registry,
ContextWindow: 100_000,
Model: "small-model",
ModelSwitcher: func(context.Context, string) (Provider, error) { return switched, nil },
ContextWindowFor: func(modelID string) int {
if modelID == "big-model" {
return 400_000
}
return 0
},
OnContext: func(breakdown ContextBreakdown) { windows = append(windows, breakdown.ContextWindow) },
})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "done" {
t.Fatalf("final answer = %q", result.FinalAnswer)
}
if len(windows) < 2 {
t.Fatalf("expected at least 2 OnContext reports, got %d", len(windows))
}
if windows[0] != 100_000 {
t.Fatalf("turn 1 window = %d, want the original 100000", windows[0])
}
if windows[len(windows)-1] != 400_000 {
t.Fatalf("post-escalation window = %d, want 400000", windows[len(windows)-1])
}
}
// nonComparableProvider has a slice field, so comparing two Provider interface
// values holding it would panic; the summarizer path must never compare them.
type nonComparableProvider struct {
inner *mockProvider
tags []string
}
func (p nonComparableProvider) StreamCompletion(ctx context.Context, request zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
return p.inner.StreamCompletion(ctx, request)
}
func TestSummarizeClosureNeverComparesProviderValues(t *testing.T) {
broken := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventError, Error: "boom: model not found"},
}}}
main := nonComparableProvider{inner: summaryProvider("MAIN SUMMARY"), tags: []string{"x"}}
state := newCompactionState(Options{
ContextWindow: 1000,
CompactionPreserveLast: 2,
Summarizer: func(context.Context, string) (Provider, error) { return nonComparableProvider{inner: broken}, nil },
}, nil)
summary, err := state.summarizeClosure(context.Background(), main)([]zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "history"}})
if err != nil || summary != "MAIN SUMMARY" {
t.Fatalf("fallback with non-comparable providers must not panic and must use main, got %q err=%v", summary, err)
}
// And with no dedicated summarizer at all the closure still works.
state = newCompactionState(Options{ContextWindow: 1000, CompactionPreserveLast: 2}, nil)
if _, err := state.summarizeClosure(context.Background(), main)([]zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "history"}}); err != nil {
t.Fatal(err)
}
}
+4 -1
View File
@@ -126,8 +126,11 @@ func capCompactionBrief(brief string) (string, bool) {
func compactionToolCallLine(call zeroruntime.ToolCall) string {
detail := ""
if call.Freeform && call.Name == "apply_patch" {
detail = strings.Join(patchPaths(call.Arguments), ", ")
}
var arguments map[string]any
if json.Unmarshal([]byte(strings.TrimSpace(call.Arguments)), &arguments) == nil {
if detail == "" && json.Unmarshal([]byte(strings.TrimSpace(call.Arguments)), &arguments) == nil {
for _, key := range []string{"file_path", "path", "url", "query", "pattern", "prompt", "description", "question"} {
if value, ok := arguments[key].(string); ok && strings.TrimSpace(value) != "" {
detail = value
+2 -1
View File
@@ -73,6 +73,7 @@ func TestProjectCompactionInputRetainsAskUserExchangeAndSemanticArguments(t *tes
{Role: zeroruntime.MessageRoleTool, ToolCallID: "ask", Content: "Which database should remain?: Postgres only; do not touch MySQL."},
{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{
{ID: "patch", Name: "apply_patch", Arguments: `{"patch":"*** Begin Patch\n*** Update File: internal/db.go\n*** End Patch"}`},
{ID: "raw-patch", Name: "apply_patch", Freeform: true, Arguments: "*** Begin Patch\n*** Update File: internal/api.go\n*** Add File: internal/config.go\n*** End Patch\n"},
{ID: "fetch", Name: "web_fetch", Arguments: `{"url":"https://example.com/spec"}`},
{ID: "task", Name: "task", Arguments: `{"prompt":"Inspect the Windows implementation"}`},
}},
@@ -80,7 +81,7 @@ func TestProjectCompactionInputRetainsAskUserExchangeAndSemanticArguments(t *tes
}
content := projectCompactionInput(messages).messages[0].Content
for _, want := range []string{"Which database should remain?", "Postgres only; do not touch MySQL", "internal/db.go", "https://example.com/spec", "Inspect the Windows implementation"} {
for _, want := range []string{"Which database should remain?", "Postgres only; do not touch MySQL", "internal/db.go", "internal/api.go", "internal/config.go", "https://example.com/spec", "Inspect the Windows implementation"} {
if !strings.Contains(content, want) {
t.Fatalf("projection missing %q: %q", want, content)
}
+21
View File
@@ -275,6 +275,27 @@ func TestEstimateToolDefTokensCountsDefinitions(t *testing.T) {
if estimateToolDefTokens(one) <= 0 {
t.Fatal("a tool definition (name + description + schema) should estimate > 0 tokens")
}
typed := append([]zeroruntime.ToolDefinition(nil), one...)
typed[0].Type = zeroruntime.ToolDefinitionFreeform
if estimateToolDefTokens(typed) <= estimateToolDefTokens(one) {
t.Fatal("a non-empty tool type must increase the estimated request cost")
}
for _, test := range []struct {
name string
format zeroruntime.ToolDefinitionFormat
}{
{name: "type", format: zeroruntime.ToolDefinitionFormat{Type: "grammar"}},
{name: "syntax", format: zeroruntime.ToolDefinitionFormat{Syntax: "lark"}},
{name: "definition", format: zeroruntime.ToolDefinitionFormat{Definition: "start: WORD"}},
} {
t.Run("format "+test.name, func(t *testing.T) {
formatted := append([]zeroruntime.ToolDefinition(nil), one...)
formatted[0].Format = &test.format
if estimateToolDefTokens(formatted) <= estimateToolDefTokens(one) {
t.Fatalf("a non-empty format %s must increase the estimated request cost", test.name)
}
})
}
two := append(append([]zeroruntime.ToolDefinition{}, one...), zeroruntime.ToolDefinition{
Name: "write_file",
Description: "Write contents to a file in the workspace.",
+4
View File
@@ -102,6 +102,10 @@ func copyToolDefinitions(toolDefs []zeroruntime.ToolDefinition) []zeroruntime.To
for index, definition := range toolDefs {
copied[index] = definition
copied[index].Parameters = copySchemaMap(definition.Parameters)
if definition.Format != nil {
format := *definition.Format
copied[index].Format = &format
}
}
return copied
}
+3
View File
@@ -15,6 +15,7 @@ func TestContextPlannerPreservesProviderRequest(t *testing.T) {
}
toolDefs := []zeroruntime.ToolDefinition{{
Name: "read_file", Description: "Read a file",
Format: &zeroruntime.ToolDefinitionFormat{Type: "grammar", Syntax: "lark", Definition: "start: PATH"},
Parameters: map[string]any{
"type": "object",
"required": []string{"path"},
@@ -53,10 +54,12 @@ func TestContextPlannerPreservesProviderRequest(t *testing.T) {
toolDefs[0].Parameters["required"].([]string)[0] = "other"
toolDefs[0].Parameters["properties"].(map[string]any)["path"].(map[string]any)["type"] = "integer"
toolDefs[0].Parameters["anyOf"].([]any)[0].(map[string]any)["type"] = "array"
toolDefs[0].Format.Definition = "changed"
if plan.Request.Messages[1].Content != "inspect this" || plan.Request.Messages[1].Images[0].Data[0] != 1 {
t.Fatalf("planned messages alias caller state: %#v", plan.Request.Messages[1])
}
if plan.Request.Tools[0].Name != "read_file" ||
plan.Request.Tools[0].Format == nil || plan.Request.Tools[0].Format.Definition != "start: PATH" ||
plan.Request.Tools[0].Parameters["required"].([]string)[0] != "path" ||
plan.Request.Tools[0].Parameters["properties"].(map[string]any)["path"].(map[string]any)["type"] != "string" ||
plan.Request.Tools[0].Parameters["anyOf"].([]any)[0].(map[string]any)["type"] != "object" {
+378
View File
@@ -0,0 +1,378 @@
package agent
import (
"context"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/sandbox"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
type customApplyPatchTool struct {
calls int
}
type fixedPathScope struct {
roots []string
}
func (scope fixedPathScope) Roots() []string {
return append([]string(nil), scope.roots...)
}
func newFixedPathScope(t *testing.T, roots ...string) fixedPathScope {
t.Helper()
resolved := make([]string, 0, len(roots))
for _, root := range roots {
absolute, err := filepath.Abs(root)
if err != nil {
t.Fatalf("Abs(%q): %v", root, err)
}
if physical, err := filepath.EvalSymlinks(absolute); err == nil {
absolute = physical
}
resolved = append(resolved, absolute)
}
return fixedPathScope{roots: resolved}
}
func (*customApplyPatchTool) Name() string { return "apply_patch" }
func (*customApplyPatchTool) Description() string { return "custom JSON tool" }
func (*customApplyPatchTool) Parameters() tools.Schema {
return tools.Schema{Type: "object", Properties: map[string]tools.PropertySchema{"value": {Type: "string"}}}
}
func (*customApplyPatchTool) Safety() tools.Safety {
return tools.Safety{Permission: tools.PermissionAllow, SideEffect: tools.SideEffectRead}
}
func (tool *customApplyPatchTool) Run(context.Context, map[string]any) tools.Result {
tool.calls++
return tools.Result{Status: tools.StatusOK}
}
func TestFreeformApplyPatchUsesExistingToolExecutionPath(t *testing.T) {
root := t.TempDir()
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(root, nil))
patch := "*** Begin Patch\n*** Add File: hello.txt\n+hello\n*** End Patch\n"
result, err := executeToolCall(context.Background(), registry, zeroruntime.ToolCall{
ID: "call-1",
Name: "apply_patch",
Arguments: patch,
Freeform: true,
}, PermissionModeUnsafe, Options{Cwd: root, FileTracker: tools.NewFileTracker()})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusOK {
t.Fatalf("freeform apply_patch status = %s: %s", result.Status, result.Output)
}
content, err := os.ReadFile(filepath.Join(root, "hello.txt"))
if err != nil {
t.Fatalf("read patched file: %v", err)
}
if string(content) != "hello\n" {
t.Fatalf("patched content = %q, want hello newline", content)
}
}
func TestFreeformApplyPatchSupportsGrantedExtraRoot(t *testing.T) {
root := t.TempDir()
extra := t.TempDir()
scope, err := sandbox.NewScope(root, []string{extra})
if err != nil {
t.Fatalf("NewScope: %v", err)
}
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(root, scope))
target := filepath.Join(extra, "hello.txt")
patch := "*** Begin Patch\n*** Add File: " + target + "\n+hello\n*** End Patch\n"
result, err := executeToolCall(context.Background(), registry, zeroruntime.ToolCall{
ID: "call-1", Name: "apply_patch", Arguments: patch, Freeform: true,
}, PermissionModeUnsafe, Options{Cwd: root, FileTracker: tools.NewFileTracker()})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusOK {
t.Fatalf("freeform extra-root apply_patch status = %s: %s", result.Status, result.Output)
}
content, err := os.ReadFile(target)
if err != nil {
t.Fatalf("read extra-root patched file: %v", err)
}
if string(content) != "hello\n" {
t.Fatalf("patched content = %q, want hello newline", content)
}
}
func TestFreeformApplyPatchRejectsMixedWorkspaceAndExtraRoots(t *testing.T) {
for _, extraHasMatchingPath := range []bool{false, true} {
name := "extra target missing"
if extraHasMatchingPath {
name = "extra target exists"
}
t.Run(name, func(t *testing.T) {
workspace := t.TempDir()
extra := t.TempDir()
if err := os.MkdirAll(filepath.Join(workspace, "src"), 0o755); err != nil {
t.Fatal(err)
}
workspaceTarget := filepath.Join(workspace, "src", "a.go")
if err := os.WriteFile(workspaceTarget, []byte("old\n"), 0o644); err != nil {
t.Fatal(err)
}
extraTarget := filepath.Join(extra, "src", "a.go")
if extraHasMatchingPath {
if err := os.MkdirAll(filepath.Dir(extraTarget), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(extraTarget, []byte("old\n"), 0o644); err != nil {
t.Fatal(err)
}
}
scope, err := sandbox.NewScope(workspace, []string{extra})
if err != nil {
t.Fatal(err)
}
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(workspace, scope))
extraCreated := filepath.Join(extra, "b.go")
patch := strings.Join([]string{
"*** Begin Patch",
"*** Update File: src/a.go",
"@@",
"-old",
"+new",
"*** Add File: " + extraCreated,
"+extra",
"*** End Patch",
}, "\n")
result, err := executeToolCall(context.Background(), registry, zeroruntime.ToolCall{
ID: "call-1", Name: "apply_patch", Arguments: patch, Freeform: true,
}, PermissionModeUnsafe, Options{Cwd: workspace, FileTracker: tools.NewFileTracker()})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusError || !strings.Contains(result.Output, "mixes workspace-relative paths with an extra write root") {
t.Fatalf("mixed-root patch result = %s: %s", result.Status, result.Output)
}
if content, err := os.ReadFile(workspaceTarget); err != nil || string(content) != "old\n" {
t.Fatalf("workspace target changed: content=%q err=%v", content, err)
}
if extraHasMatchingPath {
if content, err := os.ReadFile(extraTarget); err != nil || string(content) != "old\n" {
t.Fatalf("extra-root target changed: content=%q err=%v", content, err)
}
}
if _, err := os.Stat(extraCreated); !os.IsNotExist(err) {
t.Fatalf("mixed-root patch created extra file: %v", err)
}
})
}
}
func TestFreeformApplyPatchAcceptsOneSemanticRoot(t *testing.T) {
for _, test := range []struct {
name string
useExtraRoot bool
}{
{name: "workspace"},
{name: "extra root", useExtraRoot: true},
} {
t.Run(test.name, func(t *testing.T) {
workspace := t.TempDir()
extra := t.TempDir()
scope, err := sandbox.NewScope(workspace, []string{extra})
if err != nil {
t.Fatal(err)
}
root := workspace
if test.useExtraRoot {
root = extra
}
if err := os.MkdirAll(filepath.Join(root, "src"), 0o755); err != nil {
t.Fatal(err)
}
target := filepath.Join(root, "src", "a.go")
if err := os.WriteFile(target, []byte("old\n"), 0o644); err != nil {
t.Fatal(err)
}
updatePath := target
if !test.useExtraRoot {
updatePath = filepath.Join("src", "a.go")
}
created := filepath.Join(root, "b.go")
patch := strings.Join([]string{
"*** Begin Patch",
"*** Update File: " + updatePath,
"@@",
"-old",
"+new",
"*** Add File: " + created,
"+created",
"*** End Patch",
}, "\n")
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(workspace, scope))
result, err := executeToolCall(context.Background(), registry, zeroruntime.ToolCall{
ID: "call-1", Name: "apply_patch", Arguments: patch, Freeform: true,
}, PermissionModeUnsafe, Options{Cwd: workspace, FileTracker: tools.NewFileTracker()})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusOK {
t.Fatalf("single-root patch status = %s: %s", result.Status, result.Output)
}
if content, err := os.ReadFile(target); err != nil || string(content) != "new\n" {
t.Fatalf("updated content=%q err=%v", content, err)
}
if content, err := os.ReadFile(created); err != nil || string(content) != "created\n" {
t.Fatalf("created content=%q err=%v", content, err)
}
})
}
}
func TestFreeformApplyPatchRejectsAbsolutePathOutsideGrantedRoots(t *testing.T) {
root := t.TempDir()
granted := t.TempDir()
outside := t.TempDir()
scope := newFixedPathScope(t, root, granted)
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(root, scope))
target := filepath.Join(outside, "blocked.txt")
patch := "*** Begin Patch\n*** Add File: " + target + "\n+blocked\n*** End Patch\n"
result, err := executeToolCall(context.Background(), registry, zeroruntime.ToolCall{
ID: "call-1", Name: "apply_patch", Arguments: patch, Freeform: true,
}, PermissionModeUnsafe, Options{Cwd: root})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusError {
t.Fatalf("ungranted absolute patch status = %s, want error: %s", result.Status, result.Output)
}
if _, err := os.Stat(target); !os.IsNotExist(err) {
t.Fatalf("ungranted absolute patch created target: %v", err)
}
}
func TestPreparedFreeformPatchRejectsIntermediateSymlinkSwap(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("creating directory symlinks requires privileges on Windows")
}
root := t.TempDir()
extra := t.TempDir()
outside := t.TempDir()
intermediate := filepath.Join(extra, "nested")
if err := os.Mkdir(intermediate, 0o755); err != nil {
t.Fatalf("Mkdir: %v", err)
}
scope := newFixedPathScope(t, root, extra)
tool := tools.NewScopedApplyPatchTool(root, scope)
target := filepath.Join(intermediate, "blocked.txt")
patch := "*** Begin Patch\n*** Add File: " + target + "\n+blocked\n*** End Patch\n"
args, err := tools.PrepareFreeformApplyPatchArguments(tool, patch)
if err != nil {
t.Fatalf("PrepareFreeformApplyPatchArguments: %v", err)
}
if err := os.Remove(intermediate); err != nil {
t.Fatalf("Remove intermediate: %v", err)
}
if err := os.Symlink(outside, intermediate); err != nil {
t.Fatalf("Symlink: %v", err)
}
result := tool.Run(context.Background(), args)
if result.Status != tools.StatusError {
t.Fatalf("symlink-swapped patch status = %s, want error: %s", result.Status, result.Output)
}
if _, err := os.Stat(filepath.Join(outside, "blocked.txt")); !os.IsNotExist(err) {
t.Fatalf("symlink-swapped patch wrote outside root: %v", err)
}
}
func TestUnknownFreeformToolFailsClosed(t *testing.T) {
result, err := executeToolCall(context.Background(), tools.NewRegistry(), zeroruntime.ToolCall{
ID: "call-1", Name: "unknown", Arguments: "raw", Freeform: true,
}, PermissionModeUnsafe, Options{})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusError {
t.Fatalf("unknown freeform status = %s, want error", result.Status)
}
}
func TestCustomApplyPatchCannotUseFreeformContract(t *testing.T) {
custom := &customApplyPatchTool{}
registry := tools.NewRegistry()
registry.Register(custom)
result, err := executeToolCall(context.Background(), registry, zeroruntime.ToolCall{
ID: "call-1", Name: "apply_patch", Arguments: "raw patch", Freeform: true,
}, PermissionModeUnsafe, Options{})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusError || !strings.Contains(result.Output, "Unsupported freeform tool call") {
t.Fatalf("custom freeform apply_patch result = %#v, want unsupported error", result)
}
if custom.calls != 0 {
t.Fatalf("custom apply_patch executed %d times", custom.calls)
}
}
func TestDeniedFreeformApplyPatchDoesNotWrite(t *testing.T) {
root := t.TempDir()
registry := tools.NewRegistry()
registry.Register(tools.NewScopedApplyPatchTool(root, nil))
patch := "*** Begin Patch\n*** Add File: denied.txt\n+must not exist\n*** End Patch\n"
result, err := executeToolCall(context.Background(), registry, zeroruntime.ToolCall{
ID: "call-1",
Name: "apply_patch",
Arguments: patch,
Freeform: true,
}, PermissionModeAsk, Options{
Cwd: root,
OnPermissionRequest: func(context.Context, PermissionRequest) (PermissionDecision, error) {
return PermissionDecision{Action: PermissionDecisionDeny, Reason: "test denial"}, nil
},
})
if err != nil {
t.Fatalf("executeToolCall: %v", err)
}
if result.Status != tools.StatusError {
t.Fatalf("denied freeform status = %s, want error", result.Status)
}
if _, err := os.Stat(filepath.Join(root, "denied.txt")); !os.IsNotExist(err) {
t.Fatalf("denied freeform patch changed the workspace: %v", err)
}
}
func TestHistorySafeToolCallsPreservesRawFreeformArguments(t *testing.T) {
raw := "*** Begin Patch\n*** Update File: internal/main.go\n@@\n-old\n+new\n*** End Patch\n"
safe := historySafeToolCalls([]ToolCall{{
ID: "call-1", ProviderCallID: "provider-1", Name: "apply_patch", Arguments: raw, Freeform: true,
}})
if len(safe) != 1 || safe[0].Arguments != raw || safe[0].ProviderCallID != "provider-1" || !safe[0].Freeform {
t.Fatalf("freeform history changed raw call: %#v", safe)
}
}
func TestAbortedToolResultPreservesProviderCallID(t *testing.T) {
messages := appendAbortedToolResults(nil, []ToolCall{{
ID: "call-1", ProviderCallID: "provider-1", Name: "read_file",
}})
if len(messages) != 1 || messages[0].ToolCallID != "call-1" || messages[0].ToolCallProviderID != "provider-1" || !messages[0].IsError {
t.Fatalf("aborted tool result lost provider identity: %#v", messages)
}
}
+8 -9
View File
@@ -33,11 +33,11 @@ const (
// knows before spending more tool turns.
toolOnlyProgressReminderAt = 6
// planReminderTurn is the turn (1-based) by the end of which a multi-step
// task should have called update_plan; if it hasn't, a one-time reminder is
// injected. Set to 3 (not 2) so short, legitimate two-step tasks finish
// without a spurious planning nag.
planReminderTurn = 3
// planReminderTurn and planReminderToolThreshold keep bounded implementation
// work plan-free while still nudging a genuinely sustained run that has not
// established any structured progress. Both thresholds must be met.
planReminderTurn = 7
planReminderToolThreshold = 7
// planToolName is the planning tool the loop watches for by name.
planToolName = "update_plan"
@@ -591,13 +591,12 @@ func (state *guardState) planReminder(turn int) string {
return planStaleReminder(state.toolCallsSincePlanUpdate)
}
// NOT-CALLED reminder: by the end of planReminderTurn the model should have
// called update_plan if it's doing a multi-step task (>=1 other tool call).
// One-shot for the whole run.
// NOT-CALLED reminder: once both the turn and tool-call thresholds are met,
// sustained work should have called update_plan. One-shot for the whole run.
if !state.notCalledReminderSent &&
!state.planEverCalled &&
turn >= planReminderTurn &&
state.totalToolCalls >= 1 {
state.totalToolCalls >= planReminderToolThreshold {
state.notCalledReminderSent = true
return planNotCalledReminder()
}
+43 -8
View File
@@ -257,19 +257,16 @@ func TestRunDoesNotCountDroppedToolCallTurnsAsEmpty(t *testing.T) {
}
func TestRunInjectsPlanNotCalledReminderForMultiStepTask(t *testing.T) {
const expectedThreshold = 7
if planReminderToolThreshold != expectedThreshold {
t.Fatalf("plan reminder tool threshold = %d, want %d", planReminderToolThreshold, expectedThreshold)
}
root := t.TempDir()
writeAgentTestFile(t, root+"/notes.txt", "alpha")
registry := tools.NewRegistry()
registry.Register(tools.NewScopedReadFileTool(root, nil))
provider := &mockProvider{
turns: [][]zeroruntime.StreamEvent{
toolTurn("call-1", "read_file", `{"path":"notes.txt"}`), // turn 1: other tool call
toolTurn("call-2", "read_file", `{"path":"notes.txt"}`), // turn 2: still no update_plan
toolTurn("call-3", "read_file", `{"path":"notes.txt"}`), // turn 3: reminder fires here
textTurn("done"),
},
}
provider := &mockProvider{turns: append(repeatedReadTurns(expectedThreshold), textTurn("done"))}
result, err := Run(context.Background(), "go", provider, Options{
Registry: registry,
@@ -287,6 +284,44 @@ func TestRunInjectsPlanNotCalledReminderForMultiStepTask(t *testing.T) {
}
}
func TestRunDoesNotInjectPlanReminderForBoundedToolSequence(t *testing.T) {
const expectedThreshold = 7
root := t.TempDir()
writeAgentTestFile(t, root+"/notes.txt", "alpha")
registry := tools.NewRegistry()
registry.Register(tools.NewScopedReadFileTool(root, nil))
provider := &mockProvider{turns: append(repeatedReadTurns(expectedThreshold-1), textTurn("done"))}
result, err := Run(context.Background(), "go", provider, Options{Registry: registry, MaxTurns: 12})
if err != nil {
t.Fatal(err)
}
if result.FinalAnswer != "done" {
t.Fatalf("expected final answer, got %q", result.FinalAnswer)
}
if count := countUserMessagesContaining(result.Messages, planNotCalledReminderMarker); count != 0 {
t.Fatalf("bounded tool sequence must not draw a plan reminder, got %d", count)
}
}
func TestPlanReminderWaitsForTurnThresholdAfterEnoughToolCalls(t *testing.T) {
state := guardState{totalToolCalls: 7}
if got := state.planReminder(6); got != "" {
t.Fatalf("planReminder(6) = %q, want no reminder before turn threshold", got)
}
if state.notCalledReminderSent {
t.Fatal("early turn marked the not-called reminder as sent")
}
}
func repeatedReadTurns(count int) [][]zeroruntime.StreamEvent {
turns := make([][]zeroruntime.StreamEvent, 0, count)
for range count {
turns = append(turns, toolTurn("call", "read_file", `{"path":"notes.txt"}`))
}
return turns
}
func TestRunDoesNotInjectPlanReminderForTrivialTask(t *testing.T) {
root := t.TempDir()
writeAgentTestFile(t, root+"/notes.txt", "alpha")
+56 -17
View File
@@ -141,6 +141,7 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
}
options.Trace.Counter(trace.CounterInputTokens, int64(usage.InputTokens))
options.Trace.Counter(trace.CounterCachedInputTokens, int64(usage.CachedInputTokens))
options.Trace.Counter(trace.CounterCacheWriteTokens, int64(usage.CacheWriteTokens))
options.Trace.Counter(trace.CounterOutputTokens, int64(usage.OutputTokens))
}
}
@@ -697,11 +698,12 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
turnRequestedModel = toolResult.RequestedModel
}
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleTool,
Content: toolResult.ModelOutput(),
ToolCallID: toolResult.ToolCallID,
IsError: toolResult.Status == tools.StatusError,
ChangedFiles: append([]string(nil), toolResult.ChangedFiles...),
Role: zeroruntime.MessageRoleTool,
Content: toolResult.ModelOutput(),
ToolCallID: toolResult.ToolCallID,
ToolCallProviderID: call.ProviderCallID,
IsError: toolResult.Status == tools.StatusError,
ChangedFiles: append([]string(nil), toolResult.ChangedFiles...),
})
// Images ride a following USER message rather than the tool result
// above. Every provider drops images on a tool-role message —
@@ -844,11 +846,21 @@ func Run(ctx context.Context, prompt string, provider Provider, options Options)
options.ServiceTier = ""
planner.config.serviceTier = ""
options.Trace.Counter(trace.CounterModelSwitches, 1)
// KNOWN LIMITATION (deferred): the compactor's context-window budget
// is fixed at run start from options.ContextWindow and is NOT updated
// here, so a switch to a model with a different window keeps compacting
// against the original budget. Fixing it needs the switcher to also
// report the new window — out of scope for this change.
// Re-derive the context window for the new model so compaction and
// the OnContext budget report track the model actually in force —
// without this a switch to a smaller-window model can overflow the
// target, and a switch to a larger one over-compacts. An unknown
// model (<= 0) keeps the original window. The compactor also
// re-resolves its dedicated summarizer against the new model.
window := 0
if options.ContextWindowFor != nil {
window = options.ContextWindowFor(turnRequestedModel)
}
if window > 0 {
options.ContextWindow = window
planner.config.contextWindow = window
}
compactor.switchModel(turnRequestedModel, window)
}
}
}
@@ -1035,6 +1047,9 @@ func historySafeToolCalls(calls []ToolCall) []ToolCall {
safe := make([]ToolCall, len(calls))
for i, call := range calls {
safe[i] = call
if call.Freeform {
continue
}
args := strings.TrimSpace(call.Arguments)
switch {
case args == "":
@@ -1118,8 +1133,28 @@ func decodeToolArguments(arguments string, v any) error {
}
func executeToolCall(ctx context.Context, registry *tools.Registry, call ToolCall, permissionMode PermissionMode, options Options) (ToolResult, error) {
tool, toolFound := registry.Get(call.Name)
args := map[string]any{}
if call.Arguments != "" {
if call.Freeform {
if !toolFound || !tools.IsBuiltInApplyPatch(tool) {
return ToolResult{
ToolCallID: call.ID,
Name: call.Name,
Status: tools.StatusError,
Output: "Error: Unsupported freeform tool call for " + call.Name + ".",
}, nil
}
prepared, err := tools.PrepareFreeformApplyPatchArguments(tool, call.Arguments)
if err != nil {
return ToolResult{
ToolCallID: call.ID,
Name: call.Name,
Status: tools.StatusError,
Output: "Error: Invalid freeform apply_patch input: " + err.Error(),
}, nil
}
args = prepared
} else if call.Arguments != "" {
if err := decodeToolArguments(call.Arguments, &args); err != nil {
return ToolResult{
ToolCallID: call.ID,
@@ -1147,7 +1182,6 @@ func executeToolCall(ctx context.Context, registry *tools.Registry, call ToolCal
DenialReason: DenialFiltered,
}, nil
}
tool, toolFound := registry.Get(call.Name)
if (permissionMode == PermissionModeSpecDraft || permissionMode == PermissionModePlan) && toolFound && !ToolAdvertised(tool, permissionMode) {
modeName := string(permissionMode)
return ToolResult{
@@ -3210,11 +3244,15 @@ func cachedRuntimeToolDefinition(defCache map[string]zeroruntime.ToolDefinition,
// runtimeToolDefinition renders a tool's advertised definition (name, description,
// JSON-schema parameters) as sent to the provider.
func runtimeToolDefinition(tool tools.Tool) zeroruntime.ToolDefinition {
return zeroruntime.ToolDefinition{
definition := zeroruntime.ToolDefinition{
Name: tool.Name(),
Description: tool.Description(),
Parameters: schemaToRuntimeMap(tool.Parameters()),
}
if tools.IsBuiltInApplyPatch(tool) {
definition.Type = zeroruntime.ToolDefinitionFreeform
}
return definition
}
func ToolVisible(tool tools.Tool, permissionMode PermissionMode, enabledTools []string, disabledTools []string) bool {
@@ -3369,10 +3407,11 @@ func loadedToolsFromResult(meta map[string]string) []string {
func appendAbortedToolResults(messages []Message, remaining []ToolCall) []Message {
for _, call := range remaining {
messages = append(messages, zeroruntime.Message{
Role: zeroruntime.MessageRoleTool,
Content: abortedToolResultNotice,
ToolCallID: call.ID,
IsError: true,
Role: zeroruntime.MessageRoleTool,
Content: abortedToolResultNotice,
ToolCallID: call.ID,
ToolCallProviderID: call.ProviderCallID,
IsError: true,
})
}
return messages
+4 -1
View File
@@ -4096,7 +4096,7 @@ func TestRunDoesNotFlagCleanToolOutput(t *testing.T) {
// caller's callback AND stamps token counters, and the run completes.
func TestRunTracingWrapperStampsUsage(t *testing.T) {
provider := &mockProvider{turns: [][]zeroruntime.StreamEvent{{
{Type: zeroruntime.StreamEventUsage, Usage: zeroruntime.Usage{InputTokens: 100, CachedInputTokens: 20, OutputTokens: 40}},
{Type: zeroruntime.StreamEventUsage, Usage: zeroruntime.Usage{PromptTokens: 100, CachedInputTokens: 20, CacheWriteTokens: 10, CompletionTokens: 40}},
{Type: zeroruntime.StreamEventText, Content: "done"},
{Type: zeroruntime.StreamEventDone},
}}}
@@ -4135,6 +4135,9 @@ func TestRunTracingWrapperStampsUsage(t *testing.T) {
if got := tr.Counter(trace.CounterCachedInputTokens); got != 20 {
t.Fatalf("cached input token counter = %d, want 20", got)
}
if got := tr.Counter(trace.CounterCacheWriteTokens); got != 10 {
t.Fatalf("cache-write token counter = %d, want 10", got)
}
if got := tr.Counter(trace.CounterOutputTokens); got != 40 {
t.Fatalf("output token counter = %d, want 40", got)
}
+13
View File
@@ -31,6 +31,19 @@ func (t countingSchemaTool) Run(_ context.Context, _ map[string]any) tools.Resul
return tools.Result{Status: tools.StatusOK}
}
func TestRuntimeToolDefinitionMarksOnlyBuiltInApplyPatchFreeform(t *testing.T) {
builtIn := runtimeToolDefinition(tools.NewScopedApplyPatchTool(t.TempDir(), nil))
if builtIn.Type != zeroruntime.ToolDefinitionFreeform {
t.Fatalf("built-in apply_patch type = %q, want freeform", builtIn.Type)
}
calls := 0
custom := runtimeToolDefinition(countingSchemaTool{name: "apply_patch", calls: &calls})
if custom.Type != "" {
t.Fatalf("custom apply_patch type = %q, want legacy function shape", custom.Type)
}
}
// The cache renders a tool's schema once and reuses it across calls, and its
// output is identical to the uncached path.
func TestPartitionToolsCacheRendersOnceAndMatchesUncached(t *testing.T) {
+6
View File
@@ -58,12 +58,18 @@ func toolSubstrings(exposed []zeroruntime.ToolDefinition) (toolsSubstr, schemaSu
for _, def := range exposed {
writeLengthPrefixed(&toolsSB, def.Name)
writeLengthPrefixed(&toolsSB, def.Description)
writeLengthPrefixed(&toolsSB, string(def.Type))
// Parameters is rendered to a canonical JSON string per tool. The
// schema-render cache in internal/agent (used by partitionToolsCached)
// guarantees the same render is byte-identical across turns for the
// same tool, so this substring is a stable hash input.
writeLengthPrefixed(&schemaSB, def.Name)
writeLengthPrefixed(&schemaSB, canonicalSchemaString(def.Parameters))
if def.Format != nil {
writeLengthPrefixed(&schemaSB, def.Format.Type)
writeLengthPrefixed(&schemaSB, def.Format.Syntax)
writeLengthPrefixed(&schemaSB, def.Format.Definition)
}
}
return toolsSB.String(), schemaSB.String()
}
+32
View File
@@ -73,6 +73,38 @@ func TestComputePrefixFingerprintSchemaChangeChangesSchemaHash(t *testing.T) {
}
}
func TestComputePrefixFingerprintIncludesToolTypeAndFormat(t *testing.T) {
opts := Options{Cwd: t.TempDir(), SystemPrompt: "core"}
base := zeroruntime.ToolDefinition{Name: "apply_patch", Description: "Patch files"}
baseFingerprint := ComputePrefixFingerprint(opts, []zeroruntime.ToolDefinition{base})
typed := base
typed.Type = zeroruntime.ToolDefinitionFreeform
if got := ComputePrefixFingerprint(opts, []zeroruntime.ToolDefinition{typed}); got.ToolsHash == baseFingerprint.ToolsHash {
t.Fatal("ToolsHash must change when tool Type changes")
}
formatCases := []struct {
name string
format zeroruntime.ToolDefinitionFormat
}{
{name: "type", format: zeroruntime.ToolDefinitionFormat{Type: "grammar"}},
{name: "syntax", format: zeroruntime.ToolDefinitionFormat{Syntax: "lark"}},
{name: "definition", format: zeroruntime.ToolDefinitionFormat{Definition: "start: PATCH"}},
{name: "empty", format: zeroruntime.ToolDefinitionFormat{}},
}
for _, test := range formatCases {
t.Run(test.name, func(t *testing.T) {
withFormat := base
withFormat.Format = &test.format
got := ComputePrefixFingerprint(opts, []zeroruntime.ToolDefinition{withFormat})
if got.SchemaHash == baseFingerprint.SchemaHash {
t.Fatalf("SchemaHash must change for %s format input", test.name)
}
})
}
}
// TestComputePrefixFingerprintSystemPromptChangeChangesBaseHash asserts the
// most common drift path: a system-prompt edit (e.g. an agent.md update, a
// model addendum change) must move the base-instructions hash, which moves
+19 -18
View File
@@ -30,13 +30,10 @@ work.
read-before-edit discipline: inspect the target file and nearby callers,
tests, or config before you modify behavior. Never edit a file you have not
read.
2. **Plan.** Use update_plan for implementation or investigation that has at
least three meaningful dependent steps, then keep it current at milestones.
Mark completed work and the next in-progress step together when practical;
do not spend separate turns updating it after every file or command. Keep at
most one item in_progress. Skip plans for simple lookups, explanations, code
navigation, and other short tasks. Never create a plan after the work is
already complete merely to describe what you did.
2. **Plan.** Use update_plan for multi-component, sequencing-uncertain, or long
work. Skip it for bounded changes in one component. Update only when the
overall phase changes, keep at most one item in_progress, and never create a
plan after the work is complete.
3. **Implement.** Make focused changes that match the surrounding code's style,
naming, and conventions. Prefer the smallest change that fully solves the
problem. Avoid broad refactors, unrelated rewrites, dependency churn, and
@@ -55,9 +52,9 @@ work.
edit_file, apply_patch, write_file - over shelling out to
cat/sed/awk/python for file operations.
They are safer, reviewable, and produce clean diffs.
- Prefer read_minified_file when initially exploring source code; it preserves
code while removing comments and redundant whitespace. Use read_file when
exact text, comments, or line numbers are needed.
- Use read_minified_file only to explore large or unfamiliar source. Use
read_file for small or likely edit targets and exact text, comments, or line
numbers. Do not read both unless a new need for exact content appears.
- Keep edits focused and reviewable. A single patch may update several related
files when they form one coherent change; do not hide unrelated edits in a
bulk shell or script rewrite.
@@ -76,21 +73,24 @@ work.
- Solve the problem as posed, not a more general version of it. Add no
speculative abstraction, configurability, or handling for cases that cannot
occur, and nothing the user did not ask for. A small diff can still be
over-built; if a 200-line solution could be 50, rewrite it.
over-built; if a 200-line solution could be 50, rewrite it. Once the required
behavior passes validation, stop instead of adding optional cleanup.
- Preserve behavior you were not asked to change. Do not delete or rewrite code
you did not author unless the task requires it; if you must, say so.
## Testing gate (mandatory)
- After any change to code, verify after edits by running the project's
validators before you summarize or commit: tests, type-checks, linters, and/or
the build, as appropriate. Scope them to the change while iterating; reserve
full-suite runs for milestones.
- If you are unsure which validators apply, search the repo (Makefile, package
manifests, CI config) to find them.
- After code changes, run the project's documented appropriate validators.
Scope iteration checks, combine compatible commands when diagnostics stay
useful, and run the final set once after the last edit. Rerun only after a
change or failure.
- If unsure which entry points apply, inspect the Makefile, manifests, or CI;
do not invent a parallel flow.
- Never claim a task is done, and never commit, while validators are failing. If
they fail, fix the cause and rerun; do not paper over it. If you could not run
a validator, say so explicitly rather than implying success.
- A successful native patch confirms the edit. Reread only when the next change
needs exact content or the result was ambiguous.
## Tool use
@@ -148,6 +148,7 @@ work.
- Default to concise, skimmable output. Lead with the answer or the result.
- Use GitHub-flavored Markdown: headings to structure longer replies, fenced
code blocks for code, and `inline code` for file paths, commands, symbols, and
short snippets. Reference code as `file:line` so it is clickable.
short snippets. Reference code with file paths and include line numbers when
already known; do not look them up solely for the final summary.
- Report outcomes faithfully: if tests failed, show it; if a step was skipped,
say so; when something is done and verified, state it plainly without hedging.
+5 -6
View File
@@ -60,12 +60,11 @@ const openAIPromptAddendum = `<model_guidance>
longer answers, fenced code blocks for code, and ` + "`inline code`" + ` for paths,
commands, and symbols.
- Strongly prefer the native file tools (read_file, list_directory, grep, glob,
edit_file, apply_patch, write_file) over shelling out to cat/sed/awk/python
for file work. read_file, edit_file and write_file take one file per call;
a single apply_patch may span several files when they form one coherent
change. Do not batch file writes into a script. Independent tool calls
(several reads, several edit_file calls to different files) belong in the
same turn.
edit_file, apply_patch, write_file) over shelling out to cat/sed/awk/python for
file work. Use them directly; never run apply_patch through exec_command.
read_file, edit_file and write_file take one file per call, while one coherent
apply_patch may span several files. Keep independent native calls in the same
turn, and do not batch file writes into a script.
- Persist until the task is fully handled this turn: gather context, implement,
run the validators, and report — do not stop at a partial result.
</model_guidance>`
@@ -29,6 +29,9 @@ func TestBuildSystemPromptAppendsModelAddendum(t *testing.T) {
if got := buildSystemPrompt(Options{Model: "gpt-5"}); !strings.Contains(got, openAIPromptAddendum) {
t.Fatalf("expected the OpenAI addendum in the gpt-5 prompt")
}
if got := strings.Join(strings.Fields(strings.ToLower(buildSystemPrompt(Options{Model: "gpt-5"}))), " "); !strings.Contains(got, "never run apply_patch through exec_command") {
t.Fatalf("OpenAI prompt must route edits through the native patch tool:\n%s", got)
}
// Claude is aligned with the core prompt and gets no family addendum now that
// comment discipline is universal; it must not pick up another family's block.
claude := buildSystemPrompt(Options{Model: "claude-opus-4-6"})
+11 -1
View File
@@ -29,7 +29,10 @@ func TestMain(m *testing.M) {
}
func TestCoreSystemPromptIncludesCodingQualityRules(t *testing.T) {
prompt := strings.ToLower(buildSystemPrompt(Options{}))
prompt := strings.Join(strings.Fields(strings.ToLower(buildSystemPrompt(Options{}))), " ")
if strings.Contains(prompt, "clickable file paths") {
t.Fatalf("system prompt must not claim inline-code file paths are clickable: %s", prompt)
}
for _, want := range []string{
"read-before-edit",
@@ -44,6 +47,13 @@ func TestCoreSystemPromptIncludesCodingQualityRules(t *testing.T) {
"do not recognize",
"scaled to the work",
"comment density",
"skip it for bounded changes in one component",
"do not read both unless a new need for exact content appears",
"combine compatible commands when diagnostics stay useful",
"do not invent a parallel flow",
"a successful native patch confirms the edit",
"once the required behavior passes validation, stop",
"do not look them up solely for the final summary",
} {
if !strings.Contains(prompt, want) {
t.Fatalf("expected core system prompt to include %q, got:\n%s", want, buildSystemPrompt(Options{}))
+20 -4
View File
@@ -348,10 +348,26 @@ type Options struct {
// CompactionPreserveLast is how many trailing messages compaction keeps
// verbatim. <= 0 falls back to defaultCompactionPreserveLast.
CompactionPreserveLast int
Registry *tools.Registry
PermissionMode PermissionMode
Autonomy string
Sandbox *sandbox.Engine
// Summarizer, when set, lazily builds the provider used for compaction
// summarization calls — typically a cheap/fast model, since summaries at
// main-model prices are the single most expensive recurring event in a
// long run. It receives the main model currently in force and may return
// (nil, nil) when no dedicated summarizer applies to it (the main model is
// already the cheap one). Built on the first paid compaction and again
// after a mid-run model switch. Any failure (build or call) falls back to
// the run's main provider until the next switch, so a misconfigured
// summarizer can never break compaction. nil keeps today's behavior.
Summarizer func(ctx context.Context, mainModelID string) (Provider, error)
// ContextWindowFor, when set, resolves a model ID to its context window so
// the compactor can re-derive its threshold after a mid-run model switch
// (escalate_model). Without it a switch keeps compacting against the
// original model's window — overflowing a smaller target or over-compacting
// a larger one. Return <= 0 when the model is unknown (window unchanged).
ContextWindowFor func(modelID string) int
Registry *tools.Registry
PermissionMode PermissionMode
Autonomy string
Sandbox *sandbox.Engine
// FileTracker records per-session file read/write versions so the write tools
// can detect a file changed on disk outside Zero since it was last read. nil
// disables the check. Created once per session and threaded into every tool run.
+49 -12
View File
@@ -839,18 +839,20 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
mcpTokenStore = nil
err = nil
}
criticalMCPConfig, optionalMCPConfig := splitMCPStartupConfig(mcpConfig)
mcpRuntime := mcpToolRuntime(noopMCPRuntime{})
if len(mcpConfig.Servers) > 0 {
mcpRuntime, err = deps.registerMCPTools(context.Background(), registry, mcpConfig, mcp.RegisterOptions{
if len(criticalMCPConfig.Servers) > 0 {
runtime, registerErr := deps.registerMCPTools(context.Background(), registry, criticalMCPConfig, mcp.RegisterOptions{
PermissionStore: mcpPermissionStore,
Autonomy: mcp.AutonomyLow,
Execution: executionRunner,
WorkspaceRoot: workspaceRoot,
})
}
if err != nil {
closeMCPRuntime(stderr, mcpRuntime)
return writeAppError(stderr, err.Error(), 1)
if registerErr != nil {
closeMCPRuntime(stderr, runtime)
return writeAppError(stderr, redaction.ErrorMessage(registerErr, redaction.Options{}), 1)
}
mcpRuntime = runtime
}
defer closeMCPRuntime(stderr, mcpRuntime)
// A server that could not be reached or validated is skipped, not fatal (one
@@ -909,11 +911,35 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
}
// Activate deferred MCP-tool loading for the interactive run only when the
// VISIBLE deferred-eligible count meets the resolved threshold, matching exec.
// The registry is complete (core + specialist + MCP + plugins) here, so the
// count is accurate; below threshold this is a no-op and the surface is
// unchanged. The interactive surface applies no operator tool filters, so
// enabled/disabled are nil — matching the AgentOptions below.
// This first pass covers every prompt-critical tool. Optional built-in MCP
// defaults re-run the same gate after publishing their atomic batch, so a
// later catalog generation cannot miss its loader. The interactive surface
// applies no operator tool filters, so enabled/disabled are nil — matching the
// AgentOptions below.
registerToolSearchIfEligible(registry, resolved.Tools.DeferThreshold, permissionMode, nil, nil)
var optionalMCPRuntime *optionalMCPStartup
var awaitToolReadiness func(context.Context)
if len(optionalMCPConfig.Servers) > 0 {
optionalMCPRuntime = startOptionalMCP(
context.Background(),
registry,
optionalMCPConfig,
mcp.RegisterOptions{
PermissionStore: mcpPermissionStore,
Autonomy: mcp.AutonomyLow,
Execution: executionRunner,
WorkspaceRoot: workspaceRoot,
},
deps.registerMCPTools,
func() {
registerToolSearchIfEligible(registry, resolved.Tools.DeferThreshold, permissionMode, nil, nil)
},
)
defer closeMCPRuntime(stderr, optionalMCPRuntime)
awaitToolReadiness = func(ctx context.Context) {
optionalMCPRuntime.Await(ctx, optionalMCPPromptGrace)
}
}
lastKnownMCPConfig := mcpConfig
fileTracker := tools.NewFileTracker()
var scratchBaseline scratchFileBaseline
@@ -948,10 +974,20 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
FavoriteModels: resolved.Preferences.FavoriteModels,
RecentModels: resolved.Preferences.RecentModels,
RecapsEnabled: resolved.Preferences.RecapsEnabled(),
CompactionModel: resolved.Preferences.CompactionModel,
Provider: provider,
NewProvider: deps.newProvider,
ProbeProviderHealth: deps.probeProviderHealth,
UserAgent: userAgent(),
NewTurnSessionProvider: func(profile config.ProviderProfile, provider zeroruntime.Provider) zeroruntime.TurnSessionProvider {
if provider == nil {
return nil
}
if optimized, ok := providers.OptimizedTurnSessions(profile, provider, providers.Options{}); ok {
return optimized
}
return providers.DefaultTurnSessions(profile, provider, providers.Options{})
},
ProbeProviderHealth: deps.probeProviderHealth,
UserAgent: userAgent(),
PrepareRunCompletionWarning: func() {
scratchBaseline = scratchFileSnapshot(workspaceRoot)
},
@@ -959,6 +995,7 @@ func runInteractiveTUIWithSetup(stderr io.Writer, deps appDeps, permissionMode a
return scratchFileWarning(workspaceRoot, scratchBaseline)
},
Registry: registry,
AwaitToolReadiness: awaitToolReadiness,
SessionStore: sessionStore,
PeerService: peerService,
SandboxStore: sandboxStore,
+77 -1
View File
@@ -17,6 +17,7 @@ import (
"github.com/Gitlawb/zero/internal/agent"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/mcp"
"github.com/Gitlawb/zero/internal/redaction"
"github.com/Gitlawb/zero/internal/tools"
"github.com/Gitlawb/zero/internal/tui"
"github.com/Gitlawb/zero/internal/update"
@@ -486,6 +487,74 @@ func TestRunNoArgsLaunchesTUIWithMCPState(t *testing.T) {
}
}
func TestRunNoArgsPaintsBeforeOptionalDefaultMCPIsReady(t *testing.T) {
var stdout bytes.Buffer
var stderr bytes.Buffer
cwd := t.TempDir()
userConfigPath := filepath.Join(t.TempDir(), "zero", "config.json")
permissionStore, err := mcp.NewPermissionStore(mcp.StoreOptions{FilePath: filepath.Join(t.TempDir(), "mcp-permissions.json")})
if err != nil {
t.Fatalf("NewPermissionStore() error = %v", err)
}
tokenStore, err := mcp.NewTokenStore(mcp.TokenStoreOptions{FilePath: filepath.Join(t.TempDir(), "mcp-oauth.json")})
if err != nil {
t.Fatalf("NewTokenStore() error = %v", err)
}
started := make(chan struct{})
release := make(chan struct{})
configResult := make(chan error, 1)
exitCode := runWithDeps([]string{}, &stdout, &stderr, appDeps{
getwd: func() (string, error) { return cwd, nil },
userConfigPath: func() (string, error) { return userConfigPath, nil },
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return config.ResolvedConfig{MaxTurns: 8}, nil
},
resolveMCPConfig: func(string, bool) (config.MCPConfig, error) {
return config.MCPConfig{Servers: config.DefaultMCPServers()}, nil
},
newMCPStore: func() (*mcp.PermissionStore, error) { return permissionStore, nil },
newMCPTokenStore: func() (*mcp.TokenStore, error) { return tokenStore, nil },
registerMCPTools: func(_ context.Context, registry *tools.Registry, cfg config.MCPConfig, _ mcp.RegisterOptions) (mcpToolRuntime, error) {
var configErr error
if _, ok := cfg.Servers["exa"]; !ok || len(cfg.Servers) != 1 {
configErr = fmt.Errorf("optional MCP config = %#v, want only exa", cfg.Servers)
}
configResult <- configErr
close(started)
<-release
registry.Register(cliFakeMCPRegistryTool{})
return noopMCPRuntime{}, nil
},
runTUI: func(ctx context.Context, options tui.Options) int {
select {
case <-started:
case <-time.After(time.Second):
t.Fatal("optional MCP initialization did not start")
}
if options.AwaitToolReadiness == nil {
t.Fatal("TUI did not receive optional tool readiness barrier")
}
if _, ok := options.Registry.Get("mcp_docs_lookup"); ok {
t.Fatal("optional MCP tool was published before registration completed")
}
close(release)
options.AwaitToolReadiness(ctx)
if _, ok := options.Registry.Get("mcp_docs_lookup"); !ok {
t.Fatal("optional MCP tool was not visible after readiness completed")
}
return 0
},
})
if exitCode != 0 {
t.Fatalf("exitCode = %d stderr=%s", exitCode, stderr.String())
}
if configErr := <-configResult; configErr != nil {
t.Fatal(configErr)
}
}
func TestTUIMCPCommandUsesLastGoodConfigOnRefreshError(t *testing.T) {
var stdout bytes.Buffer
var stderr bytes.Buffer
@@ -565,6 +634,7 @@ func TestRunNoArgsClosesPartialMCPRuntimeWhenRegistrationFails(t *testing.T) {
t.Fatalf("NewTokenStore() error = %v", err)
}
runtimeClosed := false
secret := "sk-proj-" + strings.Repeat("a", 23) + "0"
exitCode := runWithDeps([]string{}, &stdout, &stderr, appDeps{
getwd: func() (string, error) {
@@ -591,7 +661,7 @@ func TestRunNoArgsClosesPartialMCPRuntimeWhenRegistrationFails(t *testing.T) {
return closeFunc(func() error {
runtimeClosed = true
return nil
}), errors.New("register mcp tools failed")
}), fmt.Errorf("register mcp tools failed: %s", secret)
},
runTUI: func(ctx context.Context, options tui.Options) int {
t.Fatal("TUI should not launch after MCP registration fails")
@@ -605,6 +675,12 @@ func TestRunNoArgsClosesPartialMCPRuntimeWhenRegistrationFails(t *testing.T) {
if !strings.Contains(stderr.String(), "register mcp tools failed") {
t.Fatalf("stderr missing registration error: %s", stderr.String())
}
if strings.Contains(stderr.String(), secret) {
t.Fatalf("stderr leaked MCP registration secret: %s", stderr.String())
}
if !strings.Contains(stderr.String(), redaction.RedactedSecret) {
t.Fatalf("stderr missing redaction marker: %s", stderr.String())
}
if !runtimeClosed {
t.Fatal("partial MCP runtime was not closed after registration error")
}
+17 -10
View File
@@ -682,16 +682,20 @@ func runExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) in
ModelSwitcher: modelSwitcher,
TurnSessionProvider: turnSessions,
ModelSessionSwitcher: modelSessionSwitcher,
ReasoningEffort: forwardEffort,
Trace: traceRecorder,
Cwd: workspaceRoot,
Images: images,
Registry: registry,
PermissionMode: permissionMode,
Autonomy: options.autonomy,
SelfCorrect: selfCorrector,
FileDiagnostics: fileDiagnostics,
Profile: execProfile.Policy(displacedMaxTurns, execProfileFilledEffort),
Summarizer: summarizerFactory(resolved, deps.newProvider),
ContextWindowFor: func(modelID string) int {
return modelregistry.AgentContextWindow(modelContextWindow(modelRegistry, modelID))
},
ReasoningEffort: forwardEffort,
Trace: traceRecorder,
Cwd: workspaceRoot,
Images: images,
Registry: registry,
PermissionMode: permissionMode,
Autonomy: options.autonomy,
SelfCorrect: selfCorrector,
FileDiagnostics: fileDiagnostics,
Profile: execProfile.Policy(displacedMaxTurns, execProfileFilledEffort),
// Headless exec: don't accept a no-tool-call turn as "done" while work
// clearly remains (pending plan items / a mid-step continuation cue) —
// nudge to continue, and finalize as INCOMPLETE rather than false success
@@ -939,6 +943,9 @@ func registerToolSearchIfEligible(registry *tools.Registry, deferThreshold int,
if deferThreshold <= 0 {
return
}
if _, registered := registry.Get(tools.ToolSearchToolName); registered {
return
}
if deferredEligibleCount(registry, permissionMode, enabledTools, disabledTools) < deferThreshold {
return
}
+15 -9
View File
@@ -267,23 +267,29 @@ func streamJSONPermissionEventType(event agent.PermissionEvent) streamjson.Event
func (writer *execEventWriter) usage(usage agent.Usage) {
if writer.format == execOutputJSON {
writer.writeJSON(map[string]any{
"type": "usage",
"prompt_tokens": usage.PromptTokens,
"completion_tokens": usage.CompletionTokens,
"total_tokens": usage.TotalTokens(),
"type": "usage",
"prompt_tokens": usage.PromptTokens,
"completion_tokens": usage.CompletionTokens,
"cached_input_tokens": usage.CachedInputTokens,
"cache_write_tokens": usage.CacheWriteTokens,
"total_tokens": usage.TotalTokens(),
})
return
}
if writer.format == execOutputStreamJSON {
promptTokens := usage.EffectiveInputTokens()
completionTokens := usage.EffectiveOutputTokens()
cachedInputTokens := usage.CachedInputTokens
cacheWriteTokens := usage.CacheWriteTokens
totalTokens := usage.TotalTokens()
writer.writeStreamJSON(streamjson.Event{
Type: streamjson.EventUsage,
RunID: writer.runID,
PromptTokens: &promptTokens,
CompletionTokens: &completionTokens,
TotalTokens: &totalTokens,
Type: streamjson.EventUsage,
RunID: writer.runID,
PromptTokens: &promptTokens,
CompletionTokens: &completionTokens,
CachedInputTokens: &cachedInputTokens,
CacheWriteTokens: &cacheWriteTokens,
TotalTokens: &totalTokens,
})
}
}
+36
View File
@@ -52,3 +52,39 @@ func TestExecWriterPropagatesToolResultTruncation(t *testing.T) {
})
}
}
func TestExecWriterPropagatesCacheUsage(t *testing.T) {
for _, format := range []execOutputFormat{execOutputJSON, execOutputStreamJSON} {
t.Run(string(format), func(t *testing.T) {
var stdout bytes.Buffer
writer := execEventWriter{
stdout: &stdout,
format: format,
runID: "run_usage",
streamedText: &strings.Builder{},
}
writer.usage(agent.Usage{
InputTokens: 100,
OutputTokens: 20,
PromptTokens: 100,
CompletionTokens: 20,
CachedInputTokens: 60,
CacheWriteTokens: 10,
})
if writer.err != nil {
t.Fatalf("usage: %v", writer.err)
}
var payload map[string]any
if err := json.Unmarshal(bytes.TrimSpace(stdout.Bytes()), &payload); err != nil {
t.Fatalf("decode output %q: %v", stdout.String(), err)
}
cachedKey, writeKey := "cached_input_tokens", "cache_write_tokens"
if format == execOutputStreamJSON {
cachedKey, writeKey = "cachedInputTokens", "cacheWriteTokens"
}
if payload[cachedKey] != float64(60) || payload[writeKey] != float64(10) {
t.Fatalf("cache usage missing from payload: %#v", payload)
}
})
}
}
+147
View File
@@ -0,0 +1,147 @@
package cli
import (
"context"
"sync"
"time"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/mcp"
"github.com/Gitlawb/zero/internal/tools"
)
const (
optionalMCPPromptGrace = time.Second
optionalMCPCloseGrace = 250 * time.Millisecond
)
// splitMCPStartupConfig keeps explicitly configured servers on the
// prompt-critical path. Only unchanged built-in defaults may initialize after
// first paint; their failure was already intentionally silent.
func splitMCPStartupConfig(cfg config.MCPConfig) (critical config.MCPConfig, optional config.MCPConfig) {
critical.Servers = map[string]config.MCPServerConfig{}
optional.Servers = map[string]config.MCPServerConfig{}
for name, server := range cfg.Servers {
if server.Disabled {
continue
}
if config.IsUnconfiguredDefault(name, server) {
optional.Servers[name] = server
continue
}
critical.Servers[name] = server
}
return critical, optional
}
type optionalMCPStartup struct {
cancel context.CancelFunc
done chan struct{}
mu sync.Mutex
runtime mcpToolRuntime
closeOnce sync.Once
runtimeCloseOnce sync.Once
closeErr error
}
func startOptionalMCP(
parent context.Context,
registry *tools.Registry,
cfg config.MCPConfig,
options mcp.RegisterOptions,
register func(context.Context, *tools.Registry, config.MCPConfig, mcp.RegisterOptions) (mcpToolRuntime, error),
onReady func(),
) *optionalMCPStartup {
ctx, cancel := context.WithCancel(parent)
startup := &optionalMCPStartup{cancel: cancel, done: make(chan struct{})}
go func() {
runtime, err := register(ctx, registry, cfg, options)
if err == nil && onReady != nil {
onReady()
}
startup.mu.Lock()
startup.runtime = runtime
startup.mu.Unlock()
close(startup.done)
}()
return startup
}
// Await gives optional tools one bounded opportunity to join the next turn's
// immutable registry snapshot. A timeout leaves this turn on the existing core
// tool set; a later turn observes the completed generation.
func (startup *optionalMCPStartup) Await(ctx context.Context, timeout time.Duration) bool {
if startup == nil {
return true
}
select {
case <-startup.done:
return true
default:
}
if timeout <= 0 {
select {
case <-startup.done:
return true
case <-ctx.Done():
return false
}
}
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
case <-startup.done:
return true
case <-timer.C:
return false
case <-ctx.Done():
return false
}
}
func (startup *optionalMCPStartup) Close() error {
if startup == nil {
return nil
}
startup.closeOnce.Do(func() {
startup.cancel()
reaped := make(chan struct{})
go func() {
defer close(reaped)
<-startup.done
startup.closeRuntime()
}()
timer := time.NewTimer(optionalMCPCloseGrace)
defer timer.Stop()
select {
case <-reaped:
case <-timer.C:
}
})
startup.mu.Lock()
defer startup.mu.Unlock()
return startup.closeErr
}
func (startup *optionalMCPStartup) closeRuntime() {
startup.runtimeCloseOnce.Do(func() {
startup.mu.Lock()
runtime := startup.runtime
startup.mu.Unlock()
if runtime == nil {
return
}
err := runtime.Close()
startup.mu.Lock()
if err != nil && startup.closeErr == nil {
startup.closeErr = err
}
startup.mu.Unlock()
})
}
func (startup *optionalMCPStartup) Skipped() []mcp.SkippedServer {
return nil
}
+187
View File
@@ -0,0 +1,187 @@
package cli
import (
"context"
"errors"
"testing"
"time"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/mcp"
"github.com/Gitlawb/zero/internal/tools"
)
type closeTrackingMCPRuntime struct {
closed chan struct{}
err error
}
func (runtime *closeTrackingMCPRuntime) Close() error {
close(runtime.closed)
return runtime.err
}
func (*closeTrackingMCPRuntime) Skipped() []mcp.SkippedServer { return nil }
func TestSplitMCPStartupConfigDefersOnlyUnconfiguredDefaults(t *testing.T) {
defaults := config.DefaultMCPServers()
cfg := config.MCPConfig{Servers: map[string]config.MCPServerConfig{
"exa": defaults["exa"],
"docs": {Type: "http", URL: "https://docs.example.test/mcp"},
"disabled": {Type: "http", URL: "https://disabled.example.test/mcp", Disabled: true},
}}
critical, optional := splitMCPStartupConfig(cfg)
if _, ok := critical.Servers["docs"]; !ok || len(critical.Servers) != 1 {
t.Fatalf("critical servers = %#v, want only docs", critical.Servers)
}
if _, ok := optional.Servers["exa"]; !ok || len(optional.Servers) != 1 {
t.Fatalf("optional servers = %#v, want only exa", optional.Servers)
}
}
func TestOptionalMCPAwaitTimesOutThenObservesPublishedTools(t *testing.T) {
registry := tools.NewRegistry()
release := make(chan struct{})
started := make(chan struct{})
startup := startOptionalMCP(
context.Background(),
registry,
config.MCPConfig{Servers: config.DefaultMCPServers()},
mcp.RegisterOptions{},
func(ctx context.Context, registry *tools.Registry, _ config.MCPConfig, _ mcp.RegisterOptions) (mcpToolRuntime, error) {
close(started)
select {
case <-release:
registry.RegisterBatch([]tools.Tool{tools.NewScopedReadFileTool(t.TempDir(), nil)})
return noopMCPRuntime{}, nil
case <-ctx.Done():
return noopMCPRuntime{}, ctx.Err()
}
},
nil,
)
defer func() { _ = startup.Close() }()
<-started
if startup.Await(t.Context(), time.Millisecond) {
t.Fatal("optional startup unexpectedly completed before release")
}
if got := len(registry.All()); got != 0 {
t.Fatalf("registry published tools before readiness: %d", got)
}
close(release)
if !startup.Await(t.Context(), time.Second) {
t.Fatal("optional startup did not complete after release")
}
if _, ok := registry.Get("read_file"); !ok {
t.Fatal("completed optional startup did not publish its tool batch")
}
}
func TestOptionalMCPFailedRegistrationSkipsReadinessCallback(t *testing.T) {
readyCalls := 0
startup := startOptionalMCP(
context.Background(),
tools.NewRegistry(),
config.MCPConfig{Servers: config.DefaultMCPServers()},
mcp.RegisterOptions{},
func(context.Context, *tools.Registry, config.MCPConfig, mcp.RegisterOptions) (mcpToolRuntime, error) {
return noopMCPRuntime{}, errors.New("connect failed")
},
func() { readyCalls++ },
)
defer func() { _ = startup.Close() }()
if !startup.Await(t.Context(), time.Second) {
t.Fatal("failed optional startup did not settle")
}
if readyCalls != 0 {
t.Fatalf("readiness callback ran after failed registration: %d", readyCalls)
}
}
func TestOptionalMCPCloseCancelsBlockedStartup(t *testing.T) {
started := make(chan struct{})
startup := startOptionalMCP(
context.Background(),
tools.NewRegistry(),
config.MCPConfig{Servers: config.DefaultMCPServers()},
mcp.RegisterOptions{},
func(ctx context.Context, _ *tools.Registry, _ config.MCPConfig, _ mcp.RegisterOptions) (mcpToolRuntime, error) {
close(started)
<-ctx.Done()
return noopMCPRuntime{}, nil
},
nil,
)
<-started
if err := startup.Close(); err != nil {
t.Fatalf("Close() error = %v", err)
}
}
func TestOptionalMCPCloseReturnsRuntimeErrorWithinGrace(t *testing.T) {
closeErr := errors.New("close failed")
runtime := &closeTrackingMCPRuntime{closed: make(chan struct{}), err: closeErr}
startup := startOptionalMCP(
context.Background(),
tools.NewRegistry(),
config.MCPConfig{Servers: config.DefaultMCPServers()},
mcp.RegisterOptions{},
func(context.Context, *tools.Registry, config.MCPConfig, mcp.RegisterOptions) (mcpToolRuntime, error) {
return runtime, nil
},
nil,
)
if !startup.Await(t.Context(), time.Second) {
t.Fatal("optional startup did not settle")
}
if err := startup.Close(); !errors.Is(err, closeErr) {
t.Fatalf("Close() error = %v, want %v", err, closeErr)
}
}
func TestOptionalMCPCloseBoundsRegistrationThatIgnoresCancellation(t *testing.T) {
started := make(chan struct{})
release := make(chan struct{})
released := false
defer func() {
if !released {
close(release)
}
}()
runtime := &closeTrackingMCPRuntime{closed: make(chan struct{})}
startup := startOptionalMCP(
context.Background(),
tools.NewRegistry(),
config.MCPConfig{Servers: config.DefaultMCPServers()},
mcp.RegisterOptions{},
func(context.Context, *tools.Registry, config.MCPConfig, mcp.RegisterOptions) (mcpToolRuntime, error) {
close(started)
<-release
return runtime, nil
},
nil,
)
<-started
closeResult := make(chan error, 1)
go func() { closeResult <- startup.Close() }()
select {
case err := <-closeResult:
if err != nil {
t.Fatalf("Close() error = %v", err)
}
case <-time.After(time.Second):
t.Fatal("Close blocked on registration after cancellation")
}
close(release)
released = true
select {
case <-runtime.closed:
case <-time.After(time.Second):
t.Fatal("late optional MCP runtime was not closed")
}
}
+11 -1
View File
@@ -22,7 +22,7 @@ const permissionProfileScopeNote = "permissionProfile is derived from this proce
func runSandbox(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) int {
if len(args) == 0 {
return writeExecUsageError(stderr, "sandbox subcommand required. Use `zero sandbox policy` or `zero sandbox grants list`.")
return writeExecUsageError(stderr, "sandbox subcommand required. Use `zero sandbox policy`, `zero sandbox exec`, or `zero sandbox grants list`.")
}
switch args[0] {
case "-h", "--help", "help":
@@ -36,6 +36,8 @@ func runSandbox(args []string, stdout io.Writer, stderr io.Writer, deps appDeps)
return runSandboxSetup(args[1:], stdout, stderr, deps)
case "check":
return runSandboxCheck(args[1:], stdout, stderr, deps)
case "exec":
return runSandboxExec(args[1:], stdout, stderr, deps)
case "grants":
return runSandboxGrants(args[1:], stdout, stderr, deps)
default:
@@ -174,10 +176,17 @@ func runSandboxSetup(args []string, stdout io.Writer, stderr io.Writer, deps app
if !setupHelper.Available() {
return writeAppError(stderr, "Windows sandbox setup helper is not available", exitProvider)
}
// Resolved here, in the shell the user typed `zero sandbox setup` into, and
// carried in the args. The helper may be launched elevated, and an elevated
// process does not inherit this shell's environment. Stated explicitly rather
// than left nil (which resolves the same way) because this is the call site
// the opt-in is about.
principalOptIn := zeroSandbox.WindowsSandboxPrincipalOptIn(nil)
setupArgs, err := zeroSandbox.BuildWindowsSandboxSetupArgs(zeroSandbox.WindowsSandboxSetupArgsOptions{
CommandCWD: workspaceRoot,
WorkspaceRoots: []string{workspaceRoot},
PermissionProfile: profile,
PrincipalOptIn: &principalOptIn,
})
if err != nil {
return writeAppError(stderr, err.Error(), exitCrash)
@@ -646,6 +655,7 @@ Commands:
policy Inspect active sandbox policy and platform backend
setup Run native platform sandbox setup
check Evaluate the sandbox decision for a hypothetical tool action
exec Run one command through the real sandbox
grants Manage persistent sandbox grants
`)
+204
View File
@@ -0,0 +1,204 @@
package cli
import (
"errors"
"fmt"
"io"
"os"
"os/exec"
"strings"
"github.com/Gitlawb/zero/internal/config"
zeroSandbox "github.com/Gitlawb/zero/internal/sandbox"
)
// runSandboxExec runs ONE command through the real sandbox and exits with its
// status.
//
// This exists because until now the sandbox could only be exercised through a
// full agent turn with a model in the loop. `zero sandbox policy` reports what
// the posture would be and `zero sandbox check` evaluates a hypothetical
// decision, but nothing actually ran a command and let you look at what
// happened on disk afterwards. The practical result is that enforcement is
// covered almost entirely by tests asserting the shape of an ACL plan, and
// almost not at all by tests asserting a write was refused.
//
// A plan can be perfectly correct and never reach the filesystem. That is not
// hypothetical here: the .git rename guard was emitted correctly by the planner
// and silently skipped by the applier, and four tests covering the plan all
// passed while the ACE was absent from disk. Something that runs the real
// binary and then stats the file is the only thing that catches that class.
//
// Deliberately NOT a debug curiosity: it takes the same path a shell tool
// takes, through SandboxManager.BuildCommandPlan, so what it proves is what
// users get. It prints the resolved backend and enforcement level to stderr
// before running, so a harness can assert the sandbox was actually engaged
// rather than quietly downgraded.
func runSandboxExec(args []string, stdout io.Writer, stderr io.Writer, deps appDeps) int {
command, err := parseSandboxExecArgs(args)
if err != nil {
if errors.Is(err, errSandboxExecHelp) {
if writeErr := writeSandboxExecHelp(stdout); writeErr != nil {
return exitCrash
}
return exitSuccess
}
return writeExecUsageError(stderr, err.Error())
}
workspaceRoot, err := resolveWorkspaceRoot("", deps)
if err != nil {
return writeExecUsageError(stderr, err.Error())
}
resolved, err := deps.resolveConfig(workspaceRoot, config.Overrides{})
if err != nil {
return writeAppError(stderr, err.Error(), exitProvider)
}
policy := applyConfiguredSandboxPolicy(zeroSandbox.DefaultPolicy(), resolved.Sandbox)
scope, err := zeroSandbox.NewScope(workspaceRoot, resolved.Sandbox.AdditionalWriteRoots)
if err != nil {
return writeAppError(stderr, fmt.Sprintf("resolve sandbox write roots: %v", err), exitCrash)
}
// Through the ENGINE, not a SandboxManager built here.
//
// The engine is what a real tool call goes through, and it does more than
// hand the manager a request: it resolves the permission profile, calls
// prepareSandboxRuntime, and folds the runtime state into that profile
// before planning. Building a manager directly skipped all of it, so the
// command ran without the runtime write root, and cache or temp writes could
// pass or fail differently from the sandboxed command this exists to imitate.
// A harness that exercises the wrong path is worse than no harness, because
// its result still reads as evidence.
engine := zeroSandbox.NewEngine(zeroSandbox.EngineOptions{
WorkspaceRoot: workspaceRoot,
Policy: policy,
Scope: scope,
Backend: deps.selectSandboxBackend(zeroSandbox.BackendOptions{}),
// The same scrub list a real tool call gets. Without it the engine keeps an
// empty set, and only the hardcoded names plus the provider catalog's own
// AuthEnvVars are removed from the child's environment: a key named by
// `apiKeyEnv` in the user's config would be scrubbed for every sandboxed
// tool call and handed to this one. Reproducing the production environment
// is the entire point of the command, and a credential is the last part of
// it that may differ.
SensitiveEnvKeys: providerSensitiveEnvKeys(resolved),
})
plan, err := engine.BuildCommandPlan(zeroSandbox.CommandSpec{
Name: command[0],
Args: command[1:],
Dir: workspaceRoot,
Env: os.Environ(),
})
if err != nil {
return writeAppError(stderr, fmt.Sprintf("build sandbox command plan: %v", err), exitCrash)
}
// The plan owns resources beyond its construction: on Linux it allocates a
// policy-report file and registers the removal here, so without this every
// invocation leaves a /tmp/zero-sandbox-report-* behind.
defer plan.Cleanup()
// Printed before the command runs and on stderr, so it survives a command
// that writes to stdout and stays greppable by a test harness. A downgrade
// is reported loudly for the same reason: a smoke test that passes because
// the sandbox quietly stood down is worse than no smoke test.
fmt.Fprintf(stderr, "sandbox: backend=%s enforcement=%s wrapped=%t workspace=%s\n",
plan.Backend.Name, plan.EnforcementLevel, plan.Wrapped, plan.WorkspaceRoot)
if strings.TrimSpace(plan.DowngradeReason) != "" {
fmt.Fprintf(stderr, "sandbox: DOWNGRADED: %s\n", plan.DowngradeReason)
}
return runSandboxPlannedCommand(plan, stdout, stderr)
}
func runSandboxPlannedCommand(plan zeroSandbox.CommandPlan, stdout io.Writer, stderr io.Writer) int {
process := exec.Command(plan.Name, plan.Args...)
process.Dir = plan.Dir
if process.Dir == "" {
process.Dir = plan.WorkspaceRoot
}
if len(plan.Env) > 0 {
process.Env = plan.Env
}
process.Stdin = os.Stdin
process.Stdout = stdout
process.Stderr = stderr
if err := process.Run(); err != nil {
var exitErr *exec.ExitError
if errors.As(err, &exitErr) {
// The command's own status, not ours. A harness asserting "the write
// was refused" needs the refusal's exit code, not a wrapper's.
// A SIGNALED CHILD HAS NO EXIT CODE TO REPORT. ExitCode() answers -1
// there, and os.Exit truncates that to 255, so a child that took SIGTERM
// became indistinguishable from one that chose to exit 255. Fold the
// signal into the conventional 128+n a shell would report, which needs
// the ProcessState rather than the integer.
if status, signaled := signaledExitStatus(exitErr.ProcessState); signaled {
return status
}
return exitErr.ExitCode()
}
fmt.Fprintf(stderr, "sandbox exec: %v\n", err)
return exitCrash
}
return exitSuccess
}
var errSandboxExecHelp = errors.New("help requested")
// parseSandboxExecArgs takes everything after `--` as the command, so the
// command's own flags are never mistaken for ours.
func parseSandboxExecArgs(args []string) ([]string, error) {
if len(args) == 0 {
return nil, errors.New("usage: zero sandbox exec -- <command> [args...]")
}
// Only the FIRST token is ours to interpret. Everything from the second
// onwards belongs to the child, help flags included.
//
// Written as a straight-line decision on args[0] rather than a loop, because
// a loop here is a lie. An earlier version scanned for `--` "up front" and
// said so in its comment, but every branch of its body returned, so it only
// ever examined index 0 and the promised scan did not exist. The rule below
// is what that code actually implemented, and it is the rule we want: it is
// the wrapper's own prefix that can ask for the wrapper's help, and the
// prefix is at most one token long.
switch args[0] {
case "--":
// Everything after the separator is the command, verbatim, including
// `--help`. That is the documented contract: `zero sandbox exec -- cmd
// --help` must run cmd's help, not ours.
command := args[1:]
if len(command) == 0 {
return nil, errors.New("usage: zero sandbox exec -- <command> [args...]")
}
return command, nil
case "-h", "--help", "help":
return nil, errSandboxExecHelp
}
// The separator-less form, tolerated for interactive use: the first token is
// the command, so nothing after it is ours to read. Without this,
// `zero sandbox exec mycmd --help` printed OUR help and never ran mycmd,
// which is the same contract break as reading past `--`. The help text still
// shows the separator, because anything with a leading dash needs it.
return args, nil
}
func writeSandboxExecHelp(w io.Writer) error {
_, err := fmt.Fprint(w, `Usage:
zero sandbox exec -- <command> [args...]
Runs one command through the real sandbox and exits with its status.
Everything after the -- separator is the command, so its own flags are not
parsed as Zero's. The resolved backend and enforcement level are written to
stderr before the command runs, and a downgrade is reported there explicitly.
Examples:
zero sandbox exec -- cmd /c echo hello
zero sandbox exec -- powershell -Command "Set-Content out.txt x"
`)
return err
}
+28
View File
@@ -0,0 +1,28 @@
//go:build !windows
package cli
import (
"os"
"syscall"
)
// signaledExitStatus reports the conventional shell status for a child that was
// terminated by a signal, which exec.ExitError cannot represent.
//
// ExitCode() returns -1 for a signaled child, and the top level hands that to
// os.Exit, which truncates it to 255. A child that takes SIGTERM is then
// indistinguishable from one that exited 255 of its own accord, which breaks the
// documented status contract for a command whose whole job is to report the
// child's own status faithfully. 128+signal is what a shell reports and what a
// harness comparing statuses expects.
func signaledExitStatus(state *os.ProcessState) (int, bool) {
if state == nil {
return 0, false
}
status, ok := state.Sys().(syscall.WaitStatus)
if !ok || !status.Signaled() {
return 0, false
}
return 128 + int(status.Signal()), true
}
@@ -0,0 +1,65 @@
//go:build !windows
package cli
import (
"errors"
"os/exec"
"syscall"
"testing"
)
// A SIGNALED CHILD MUST NOT LOOK LIKE ONE THAT CHOSE TO EXIT 255.
//
// `sandbox exec` exists to report the child's own status faithfully, and
// exec.ExitError cannot represent signal termination: ExitCode() answers -1, and
// the top level hands that to os.Exit, which truncates it to 255. So a child
// killed by SIGTERM was indistinguishable from an ordinary exit of 255, and a
// harness comparing statuses could not tell a refusal from a kill.
//
// Driven with a real subprocess and a real signal rather than a synthetic
// WaitStatus, because the mapping is only correct if the OS agrees with it.
func TestSignaledChildReportsTheConventionalStatus(t *testing.T) {
for _, testCase := range []struct {
name string
script string
want int
}{
{name: "SIGTERM", script: "kill -TERM $$; sleep 5", want: 128 + int(syscall.SIGTERM)},
{name: "SIGINT", script: "kill -INT $$; sleep 5", want: 128 + int(syscall.SIGINT)},
} {
t.Run(testCase.name, func(t *testing.T) {
err := exec.Command("/bin/sh", "-c", testCase.script).Run()
var exitErr *exec.ExitError
if !errors.As(err, &exitErr) {
t.Fatalf("SETUP INVALID: the child did not end with an ExitError: %v", err)
}
// The behaviour being corrected: the integer alone cannot say this.
if code := exitErr.ExitCode(); code != -1 {
t.Fatalf("SETUP INVALID: a signaled child reported exit code %d, expected -1 on this platform", code)
}
status, signaled := signaledExitStatus(exitErr.ProcessState)
if !signaled {
t.Fatal("a signaled child was not recognised as signaled, so it would be reported as exit 255")
}
if status != testCase.want {
t.Fatalf("status = %d, want %d (128 + signal), which is what a shell reports", status, testCase.want)
}
})
}
}
// An ordinary non-zero exit is untouched: it has a real code and must keep it.
func TestOrdinaryExitIsNotTreatedAsSignaled(t *testing.T) {
err := exec.Command("/bin/sh", "-c", "exit 3").Run()
var exitErr *exec.ExitError
if !errors.As(err, &exitErr) {
t.Fatalf("SETUP INVALID: the child did not end with an ExitError: %v", err)
}
if _, signaled := signaledExitStatus(exitErr.ProcessState); signaled {
t.Fatal("an ordinary exit was reported as signaled, which would rewrite its status")
}
if code := exitErr.ExitCode(); code != 3 {
t.Fatalf("exit code = %d, want the child's own 3", code)
}
}
@@ -0,0 +1,12 @@
//go:build windows
package cli
import "os"
// signaledExitStatus has no Windows counterpart: a process there ends with an
// exit code, and there is no signal to fold into one. Windows keeps the exit
// code exec.ExitError already reports.
func signaledExitStatus(*os.ProcessState) (int, bool) {
return 0, false
}
+108
View File
@@ -0,0 +1,108 @@
package cli
import (
"errors"
"strings"
"testing"
)
// parseSandboxExecArgs sits on the boundary between Zero's flags and a child
// command's, and it had no tests at all: not for the reported defect, not even
// for the happy case. The contract it has to keep is that everything after the
// separator belongs to the child, help flags included, so the wrapper can never
// answer for a command it was supposed to be running.
func TestParseSandboxExecArgs(t *testing.T) {
for _, testCase := range []struct {
name string
args []string
command []string
help bool
usage bool
}{
{
// The reported defect: --help after the separator is the CHILD's.
name: "help after the separator belongs to the child",
args: []string{"--", "cmd", "--help"},
command: []string{"cmd", "--help"},
},
{
name: "a second separator is the child's too",
args: []string{"--", "cmd", "--", "inner"},
command: []string{"cmd", "--", "inner"},
},
{
name: "a flag-looking command survives the separator",
args: []string{"--", "--weird-binary"},
command: []string{"--weird-binary"},
},
{
// The separator-less form. The first token is the command, so its own
// flags are not ours to read either.
name: "help after a separatorless command belongs to the child",
args: []string{"cmd", "--help"},
command: []string{"cmd", "--help"},
},
{
// A separator arriving AFTER the command has already started is part of
// the child's argv, not a delimiter we get a second go at.
name: "a separator after the command is the child's argument",
args: []string{"ls", "--", "foo"},
command: []string{"ls", "--", "foo"},
},
{name: "bare help", args: []string{"--help"}, help: true},
{name: "short help", args: []string{"-h"}, help: true},
{name: "help subcommand", args: []string{"help"}, help: true},
{name: "help in the prefix wins", args: []string{"--help", "--", "cmd"}, help: true},
{name: "no arguments", args: nil, usage: true},
{name: "separator with nothing after it", args: []string{"--"}, usage: true},
} {
t.Run(testCase.name, func(t *testing.T) {
command, err := parseSandboxExecArgs(testCase.args)
switch {
case testCase.help:
if !errors.Is(err, errSandboxExecHelp) {
t.Fatalf("parseSandboxExecArgs(%q) = %q, %v; want the help sentinel", testCase.args, command, err)
}
case testCase.usage:
if err == nil || errors.Is(err, errSandboxExecHelp) {
t.Fatalf("parseSandboxExecArgs(%q) = %q, %v; want a usage error", testCase.args, command, err)
}
if !strings.Contains(err.Error(), "usage:") {
t.Errorf("a usage error must show the usage, got %q", err.Error())
}
default:
if err != nil {
t.Fatalf("parseSandboxExecArgs(%q) returned %v", testCase.args, err)
}
if strings.Join(command, "\x00") != strings.Join(testCase.command, "\x00") {
t.Errorf("parseSandboxExecArgs(%q) = %q, want %q", testCase.args, command, testCase.command)
}
}
})
}
}
// The wrapper must never consume a token it then fails to hand over. Whatever it
// returns as the command has to be a trailing slice of what it was given, with
// at most a leading separator removed, or an argument has gone missing on its
// way to the child.
func TestParseSandboxExecArgsNeverDropsAChildArgument(t *testing.T) {
for _, args := range [][]string{
{"--", "cmd", "--help", "-x", "--"},
{"cmd", "--help"},
{"ls", "--", "foo"},
{"--", "--weird-binary", "arg"},
} {
command, err := parseSandboxExecArgs(args)
if err != nil {
t.Fatalf("parseSandboxExecArgs(%q) returned %v", args, err)
}
if len(command) == 0 {
t.Fatalf("parseSandboxExecArgs(%q) returned an empty command", args)
}
suffix := strings.Join(args[len(args)-len(command):], "\x00")
if strings.Join(command, "\x00") != suffix {
t.Errorf("parseSandboxExecArgs(%q) = %q, which is not a trailing slice of the input; an argument was dropped or reordered", args, command)
}
}
}
+17
View File
@@ -0,0 +1,17 @@
package cli
import (
"context"
"github.com/Gitlawb/zero/internal/agent"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/providers"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// summarizerFactory adapts the resolved profile and the authenticated provider
// builder into agent.Options.Summarizer; the selection rules live in
// providers.CompactionSummarizerFactory, shared with the TUI.
func summarizerFactory(resolved config.ResolvedConfig, newProvider func(config.ProviderProfile) (zeroruntime.Provider, error)) func(context.Context, string) (agent.Provider, error) {
return providers.CompactionSummarizerFactory(resolved.Provider, resolved.Preferences.CompactionModel, newProvider)
}
+6
View File
@@ -119,6 +119,12 @@ type PreferencesConfig struct {
// the user turned idle recaps off. A *bool is its own tri-state, so no
// custom unmarshal is needed (unlike ToolsConfig.DeferThreshold's int).
Recaps *bool `json:"recaps,omitempty"`
// CompactionModel routes compaction summarization calls to this model
// instead of the session's main model (summaries at main-model prices are
// the most expensive recurring event in long runs). Empty = automatic: a
// curated cheap model on official endpoints, the main model elsewhere.
// "main" forces the main model. ZERO_COMPACTION_MODEL overrides.
CompactionModel string `json:"compactionModel,omitempty"`
}
// RecentModelEntry is one provider-qualified model selection recorded in
+9 -66
View File
@@ -5,31 +5,23 @@ import (
"fmt"
"os"
"path/filepath"
"sync/atomic"
"time"
"github.com/Gitlawb/zero/internal/lockutil"
)
// Cross-process lock tuning. The lock is held only for a single metadata
// read-modify-write (milliseconds), never across a job's exec, so the timeout is
// generous and the stale threshold sits far above any real hold.
// read-modify-write (milliseconds), never across a job's exec, so the timeout
// is generous relative to any real hold.
const (
cronLockTimeout = 10 * time.Second
cronLockStaleAfter = 60 * time.Second
cronLockRetryDelay = 20 * time.Millisecond
)
var cronLockSeq atomic.Uint64
// lockJob takes a cross-process exclusive lock for one job by O_EXCL-creating a
// sibling "<id>.lock" file next to the job directory (so Remove's RemoveAll of
// the job dir never deletes a live lock). It serializes the read-modify-write of
// a job's metadata across concurrent schedulers and commands. The lock file is
// removed on release; a stale lock from a crashed holder (older than
// cronLockStaleAfter) is reclaimed. Wall-clock time.Now is used deliberately
// (not the injectable Store.now) so lock timing never depends on a frozen test
// clock and the stale check compares against real file mtimes.
// lockJob takes a cross-process advisory lock for one job on a stable sibling
// "<id>.lock" file next to the job directory. It serializes the read-modify-write
// of a job's metadata across concurrent schedulers and commands. The kernel
// releases the lock when a holder exits; the file is never renamed or removed.
func (s *Store) lockJob(id string) (func(), error) {
if !validID(id) {
return nil, fmt.Errorf("invalid cron job id %q", id)
@@ -38,64 +30,15 @@ func (s *Store) lockJob(id string) (func(), error) {
if err := os.MkdirAll(filepath.Dir(lockPath), 0o700); err != nil {
return nil, err
}
token := fmt.Sprintf("%d-%d-%d", os.Getpid(), time.Now().UnixNano(), cronLockSeq.Add(1))
deadline := time.Now().Add(cronLockTimeout)
for {
f, err := os.OpenFile(lockPath, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
lock, err := lockutil.TryAcquireFileLock(lockPath)
if err == nil {
// A partial write would leave a lock file without our token, so the
// releaser could never delete it — stranding the lock. Fail closed.
if _, werr := f.WriteString(token); werr != nil {
_ = f.Close()
_ = lockutil.RemoveLockFile(lockPath)
return nil, fmt.Errorf("cron: write job lock: %w", werr)
}
if cerr := f.Close(); cerr != nil {
_ = lockutil.RemoveLockFile(lockPath)
return nil, fmt.Errorf("cron: close job lock: %w", cerr)
}
var released bool
return func() {
if released {
return
}
released = true
if data, rerr := os.ReadFile(lockPath); rerr == nil && string(data) == token {
_ = lockutil.RemoveLockFile(lockPath)
}
}, nil
return func() { _ = lock.Release() }, nil
}
// On Windows a concurrent holder's os.Remove leaves the lock file in a
// "delete pending" state, so an O_EXCL create races it with
// ERROR_ACCESS_DENIED (os.ErrPermission) rather than ErrExist. Treat that
// as contention and retry, exactly like ErrExist — otherwise the lock
// spuriously fails under concurrency on Windows.
if !errors.Is(err, os.ErrExist) && !errors.Is(err, os.ErrPermission) {
if !errors.Is(err, lockutil.ErrLockHeld) {
return nil, fmt.Errorf("cron: acquire job lock: %w", err)
}
// Reclaim a stale lock left by a crashed holder — atomically (H3). A blind
// Remove lets two racers both "reclaim" and recreate, so both hold the lock;
// reclaimStaleLock renames the file aside (only one rename of a given file
// wins) and restores it if it turns out fresh, so a live lock is never deleted
// out from under its holder.
if info, statErr := os.Stat(lockPath); statErr == nil && time.Since(info.ModTime()) > cronLockStaleAfter {
cleared, rerr := lockutil.ReclaimStaleLock(lockPath, token, func(reclaimedPath string) bool {
info, err := os.Stat(reclaimedPath)
return err == nil && time.Since(info.ModTime()) <= cronLockStaleAfter
})
if rerr != nil {
// Reclaim hit a hard failure: the rename aside failed outright, or a
// live holder's lock could not be put back (the lock path may be
// missing, so re-acquiring would break mutual exclusion). Fail closed
// instead of spinning to the deadline.
return nil, fmt.Errorf("cron: reclaim stale job lock: %w", rerr)
}
if cleared {
continue // cleared a genuinely stale lock; retry the O_EXCL create now
}
// Lost the reclaim race (or it was actually fresh) — fall through to the
// bounded wait instead of hot-spinning on a reclaim that never wins (L13).
}
if !time.Now().Before(deadline) {
return nil, fmt.Errorf("cron: timed out acquiring job lock for %q", id)
}
+1 -1
View File
@@ -64,7 +64,7 @@ func TestMutateRemovedJobReturnsNotFound(t *testing.T) {
if !errors.Is(err, ErrJobNotFound) {
t.Fatalf("Mutate on removed job = %v, want ErrJobNotFound", err)
}
// And the lock file left no litter / did not resurrect the job dir.
// The stable advisory-lock file does not resurrect the removed job dir.
if _, gerr := store.Get(job.ID); !errors.Is(gerr, ErrJobNotFound) {
t.Fatalf("job should remain removed, Get = %v", gerr)
}
+35 -67
View File
@@ -3,107 +3,60 @@ package daemon
import (
"errors"
"fmt"
"io/fs"
"os"
"strconv"
"strings"
"sync/atomic"
"github.com/Gitlawb/zero/internal/lockutil"
)
// Single-instance lock. Mirrors reference-daemon-code-agent-js/supervisor.js's
// lock file: a PID file created with O_EXCL. A second start fails; a STALE lock
// left by a dead daemon (the recorded PID is no longer alive) is reclaimed so the
// daemon recovers from an unclean shutdown without manual cleanup.
// Single-instance lock. The stable PID file carries a kernel advisory lock for
// the daemon's full lifetime. Process exit releases the lock automatically, so
// stale metadata needs no rename/remove recovery protocol.
// ErrAlreadyRunning is returned when a live daemon already holds the lock.
var ErrAlreadyRunning = errors.New("daemon: another instance is already running")
// fileLock is an acquired single-instance lock.
type fileLock struct {
path string
lock *lockutil.FileLock
}
// processAlive reports whether pid is a live process. Implemented per-platform
// (lock_posix.go / lock_windows.go). It is a package var so tests can stub it.
var processAlive = osProcessAlive
// acquireLock takes the single-instance lock at path, reclaiming a stale lock
// whose recorded PID is dead. isAlive overrides the liveness check (tests pass a
// stub); nil uses the real processAlive.
// acquireLock takes the single-instance advisory lock at path. isAlive is used
// only to enrich a contention error with the current holder's PID; the kernel
// lock, not PID metadata, is authoritative.
func acquireLock(path string, isAlive func(pid int) bool) (*fileLock, error) {
if isAlive == nil {
isAlive = processAlive
}
// At most two passes: create, or detect-stale-then-retry once.
for attempt := 0; attempt < 2; attempt++ {
f, err := os.OpenFile(path, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
if err == nil {
// A failed PID write would leave a malformed lock file that another
// process reads as stale (unparsable PID) and wrongly reclaims, breaking
// the single-instance guarantee — so on write failure, remove it and fail.
if _, werr := fmt.Fprintf(f, "%d\n", os.Getpid()); werr != nil {
_ = f.Close()
_ = lockutil.RemoveLockFile(path)
return nil, werr
}
if cerr := f.Close(); cerr != nil {
_ = lockutil.RemoveLockFile(path)
return nil, cerr
}
return &fileLock{path: path}, nil
}
if !errors.Is(err, fs.ErrExist) && !errors.Is(err, os.ErrPermission) {
lock, err := lockutil.TryAcquireFileLock(path)
if err != nil {
if !errors.Is(err, lockutil.ErrLockHeld) {
return nil, err
}
pid, perr := readPidFile(path)
if perr == nil && pid > 0 && isAlive(pid) {
return nil, fmt.Errorf("%w (pid %d)", ErrAlreadyRunning, pid)
}
// Stale lock (dead PID or unreadable) — reclaim it atomically, then retry the
// O_EXCL create. A blind Remove here races: two daemons starting at once could
// both read the stale PID, both Remove, and then one Removes the OTHER's
// freshly-created lock — leaving both "holding" the single-instance lock.
// reclaimStaleLock renames the file aside so only one racer wins the rename,
// and restores it if a live holder reacquired in the gap (D6).
if _, rerr := reclaimStaleLock(path, isAlive); rerr != nil {
// Reclaim hit a hard failure: the rename aside failed outright, or a
// live holder's lock could not be put back (the lock path may be
// missing, so re-acquiring would break the single-instance guarantee).
// Fail closed instead of spinning to the deadline.
return nil, fmt.Errorf("daemon: reclaim stale lock: %w", rerr)
}
return nil, ErrAlreadyRunning
}
return nil, ErrAlreadyRunning
if err := lock.WriteMetadata([]byte(fmt.Sprintf("%d\n", os.Getpid()))); err != nil {
return nil, errors.Join(err, lock.Release())
}
return &fileLock{lock: lock}, nil
}
// daemonLockSeq makes each reclaim attempt's sidelined filename unique per process.
var daemonLockSeq atomic.Uint64
// reclaimStaleLock atomically reclaims a single-instance lock whose recorded
// PID is dead, via lockutil.ReclaimStaleLock: the lock file is renamed aside
// (only one racer can win the rename) and stolen only if the moved file's PID
// is still dead; a holder that reacquired the lock in the gap between the
// stale check and the rename carries a LIVE pid and is restored rather than
// stolen. Returns true only when a genuinely stale lock was removed. A
// non-nil error means the caller must fail closed instead of re-acquiring
// (see lockutil.ReclaimStaleLock).
func reclaimStaleLock(path string, isAlive func(pid int) bool) (bool, error) {
suffix := fmt.Sprintf("%d-%d", os.Getpid(), daemonLockSeq.Add(1))
return lockutil.ReclaimStaleLock(path, suffix, func(reclaimedPath string) bool {
pid, err := readPidFile(reclaimedPath)
return err == nil && pid > 0 && isAlive(pid)
})
}
// release removes the lock file. Safe to call once. An already-missing lock
// file is not an error (lockutil.RemoveLockFile swallows it on every platform).
// release drops the kernel lock and closes its handle. The stable PID file is
// deliberately retained so contenders always address the same inode.
func (l *fileLock) release() error {
if l == nil || l.path == "" {
if l == nil || l.lock == nil {
return nil
}
return lockutil.RemoveLockFile(l.path)
return l.lock.Release()
}
// readPidFile reads and parses the PID recorded in a lock file.
@@ -112,5 +65,20 @@ func readPidFile(path string) (int, error) {
if err != nil {
return 0, err
}
return strconv.Atoi(strings.TrimSpace(string(data)))
metadata := strings.TrimSpace(string(data))
if pid, err := strconv.Atoi(metadata); err == nil {
return pid, nil
}
pidText, sequenceText, ok := strings.Cut(metadata, "-")
if !ok || pidText == "" || sequenceText == "" || strings.Contains(sequenceText, "-") {
return 0, fmt.Errorf("daemon: parse lock PID metadata %q", metadata)
}
if _, err := strconv.ParseUint(sequenceText, 10, 64); err != nil {
return 0, fmt.Errorf("daemon: parse lock holder sequence: %w", err)
}
pid, err := strconv.Atoi(pidText)
if err != nil {
return 0, fmt.Errorf("daemon: parse lock PID: %w", err)
}
return pid, nil
}
+54 -42
View File
@@ -32,27 +32,48 @@ func TestLockSingleInstance(t *testing.T) {
_ = l2.release()
}
func TestLockStaleRecovery(t *testing.T) {
// A PID liveness probe is diagnostic only. Even if it reports the recorded PID
// dead, a contended kernel lock proves that the holder is active and must win.
// The old rename-aside protocol trusted this stale observation and admitted a
// second daemon while the first one was still inside its critical section.
func TestLockDoesNotOverrideKernelHolderWithStalePIDProbe(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.lock")
// Simulate a stale lock from a crashed daemon: a PID file whose process is
// dead.
first, err := acquireLock(path, func(int) bool { return true })
if err != nil {
t.Fatalf("first acquireLock: %v", err)
}
defer first.release()
second, err := acquireLock(path, func(int) bool { return false })
if second != nil {
_ = second.release()
}
if !errors.Is(err, ErrAlreadyRunning) {
t.Fatalf("contended acquire with stale PID probe = %v, want ErrAlreadyRunning", err)
}
}
func TestLockIgnoresStaleMetadataWhenKernelLockIsFree(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.lock")
// A crashed daemon may leave PID metadata behind, but its kernel lock is
// released automatically, so the next daemon can acquire without moving it.
if err := os.WriteFile(path, []byte("4242\n"), 0o600); err != nil {
t.Fatalf("write stale lock: %v", err)
}
dead := func(int) bool { return false }
l, err := acquireLock(path, dead)
if err != nil {
t.Fatalf("stale-lock recovery failed: %v", err)
t.Fatalf("acquire over stale metadata: %v", err)
}
// The lock now records OUR pid, not the stale one.
data, _ := os.ReadFile(path)
if strings.TrimSpace(string(data)) != strconv.Itoa(os.Getpid()) {
t.Fatalf("reclaimed lock pid = %q, want %d", strings.TrimSpace(string(data)), os.Getpid())
t.Fatalf("lock pid = %q, want %d", strings.TrimSpace(string(data)), os.Getpid())
}
_ = l.release()
}
func TestLockReleaseRemovesFile(t *testing.T) {
func TestLockReleaseKeepsStableFile(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.lock")
l, err := acquireLock(path, func(int) bool { return true })
if err != nil {
@@ -61,46 +82,37 @@ func TestLockReleaseRemovesFile(t *testing.T) {
if err := l.release(); err != nil {
t.Fatalf("release: %v", err)
}
if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("lock file still present after release: %v", err)
if _, err := os.Stat(path); err != nil {
t.Fatalf("release removed stable lock file: %v", err)
}
}
func TestReclaimStaleLockRemovesDeadHolder(t *testing.T) {
func TestReadPidFileMetadataFormats(t *testing.T) {
path := filepath.Join(t.TempDir(), "daemon.lock")
if err := os.WriteFile(path, []byte("4242\n"), 0o600); err != nil {
t.Fatalf("write stale lock: %v", err)
}
if ok, err := reclaimStaleLock(path, func(int) bool { return false }); err != nil || !ok {
t.Fatalf("reclaimStaleLock must report a genuinely stale (dead-PID) lock reclaimed (ok=%v err=%v)", ok, err)
}
if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("a reclaimed stale lock must be removed, stat = %v", err)
}
}
func TestReclaimStaleLockRestoresLiveHolder(t *testing.T) {
// If a holder reacquires the lock in the gap between the stale check and the
// rename, the moved file carries a LIVE pid; reclaim must restore it untouched
// rather than steal it — otherwise two daemons both "hold" the lock (D6).
path := filepath.Join(t.TempDir(), "daemon.lock")
if err := os.WriteFile(path, []byte("4242\n"), 0o600); err != nil {
t.Fatalf("write lock: %v", err)
}
if ok, err := reclaimStaleLock(path, func(int) bool { return true }); err != nil || ok {
t.Fatalf("reclaimStaleLock must NOT report a live-PID lock reclaimed (ok=%v err=%v)", ok, err)
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("live lock must be restored in place, read = %v", err)
}
if strings.TrimSpace(string(data)) != "4242" {
t.Fatalf("restored lock content = %q, want %q (unchanged)", strings.TrimSpace(string(data)), "4242")
}
// No sidelined ".stale" leftovers.
matches, _ := filepath.Glob(path + ".stale.*")
if len(matches) != 0 {
t.Fatalf("reclaim left sidelined files: %v", matches)
for _, test := range []struct {
name string
metadata string
wantPID int
wantErr bool
}{
{name: "legacy PID", metadata: "4242\n", wantPID: 4242},
{name: "PID and holder sequence", metadata: "4242-7\n", wantPID: 4242},
{name: "missing sequence", metadata: "4242-", wantErr: true},
{name: "non-numeric sequence", metadata: "4242-next", wantErr: true},
{name: "extra component", metadata: "4242-7-8", wantErr: true},
} {
t.Run(test.name, func(t *testing.T) {
if err := os.WriteFile(path, []byte(test.metadata), 0o600); err != nil {
t.Fatal(err)
}
got, err := readPidFile(path)
if (err != nil) != test.wantErr {
t.Fatalf("readPidFile() error = %v, wantErr %v", err, test.wantErr)
}
if !test.wantErr && got != test.wantPID {
t.Fatalf("readPidFile() = %d, want %d", got, test.wantPID)
}
})
}
}
+6 -2
View File
@@ -124,12 +124,16 @@ func TestServerEndToEnd(t *testing.T) {
case <-time.After(3 * time.Second):
t.Fatal("Serve did not return after shutdown")
}
// Socket, status, and lock files are removed on exit.
for _, p := range []string{paths.Socket, paths.Status, paths.Lock} {
// Runtime endpoints are removed on exit. The advisory lock file remains as
// a stable inode, but the kernel lock on it has been released.
for _, p := range []string{paths.Socket, paths.Status} {
if _, err := os.Stat(p); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("file %s not cleaned up after shutdown: %v", p, err)
}
}
if _, err := os.Stat(paths.Lock); err != nil {
t.Fatalf("stable lock file missing after shutdown: %v", err)
}
}
func TestServerSecondInstanceFails(t *testing.T) {
+58 -4
View File
@@ -100,10 +100,17 @@ func windowsSandboxSetupCheck(goos string, backend sandbox.Backend, workspaceRoo
}
profile := sandbox.PermissionProfileFromPolicy(workspaceRoot, doctorSandboxPolicy(sandboxConfig), scope)
setupConfig := sandbox.WindowsSandboxSetupConfig{
SandboxHome: sandboxHome,
CommandCWD: workspaceRoot,
WorkspaceRoots: []string{workspaceRoot},
PermissionProfile: profile,
SandboxHome: sandboxHome,
CommandCWD: workspaceRoot,
WorkspaceRoots: []string{workspaceRoot},
// Same augmentation the setup args and the command plan apply, so doctor
// fingerprints what a real command fingerprints. Safe here: doctor runs in
// the operator's shell, not behind the sandbox's TEMP redirection.
PermissionProfile: sandbox.WindowsSandboxProfileWithRuntimeRoots(profile, []string{workspaceRoot}),
// Same opt-in a command would resolve, so doctor reports the principal
// mismatch as out-of-date setup instead of passing a check the next command
// will fail.
PrincipalOptIn: sandbox.WindowsSandboxPrincipalOptIn(nil),
}
if err := sandbox.ValidateWindowsSandboxSetupMarker(setupConfig); err != nil {
result := check("sandbox.backend", "Sandbox backend", StatusWarn, fmt.Sprintf("Native sandbox backend %s is installed, but Windows sandbox setup is missing or out of date: %v.", backend.Name, err), map[string]any{
@@ -115,6 +122,53 @@ func windowsSandboxSetupCheck(goos string, backend sandbox.Backend, workspaceRoo
})
return &result
}
// Setup is valid and the opt-in is set, so report which principal a command
// will actually run as.
//
// This used to warn that the principal stood down under the DEFAULT
// network-deny policy, which was true while network denial could only be
// expressed on the restricted token: a principal token cannot carry the
// synthetic offline-marker SID the WFP filters matched. Dual-role provisioning
// removed that limit by giving each workspace an offline account whose real
// group membership the same filters match, so deny mode now runs on a
// principal exactly as allow mode does. Leaving the warning would have doctor
// telling operators their reads are unconfined while they are confined.
//
// Naming the role rather than just saying "active" is the useful part. The two
// accounts differ only in that membership, so an operator debugging network
// behaviour needs to know which one this workspace picks.
if setupConfig.PrincipalOptIn {
// Asked of the shared rule rather than re-derived here, so doctor and the
// runtime cannot disagree about which account a command runs as. Two
// copies drifting is what produced the wrong report this replaces.
role := sandbox.WindowsSandboxPrincipalRoleForNetwork(profile.Network.Mode)
// SETUP IS CURRENT IS NOT THE SAME AS THE PRINCIPAL IS LIVE, and this used
// to claim the second from the first. Marker validation compares serialized
// plans and hashes; it names no account. Delete the selected account or its
// secret, or drop the offline account out of ZeroSandboxOffline, and the
// marker still validates while the runtime either falls back to the weaker
// restricted-token path or fails outright. Doctor asserting read
// confinement in that state is worse than saying nothing, because it is the
// surface an operator checks precisely when they are unsure.
//
// Verifying liveness needs the account, a usable secret and (for offline)
// the group membership, all Windows-only live queries this package cannot
// make. So the claim is narrowed to what the marker actually proves, and
// the role is still named because that part IS derived rather than assumed
// and is what an operator debugging network behaviour needs.
result := check("sandbox.principal", "Sandbox principal", StatusPass,
fmt.Sprintf("Sandbox principal setup is current; a command in this workspace selects the %s principal. Setup state does not confirm the account is usable.", role), map[string]any{
"backend": string(backend.Name),
"platform": goos,
"optIn": true,
// Deliberately not "active": nothing here checked the account, the
// secret or the group membership.
"setupCurrent": true,
"role": role,
"networkMode": string(profile.Network.Mode),
})
return &result
}
return nil
}
@@ -0,0 +1,72 @@
package doctor
import (
"strings"
"testing"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/sandbox"
)
// A VALID MARKER IS NOT A LIVE PRINCIPAL. Marker validation compares serialized
// plans and hashes and names no account, so it stays valid after the selected
// account is deleted, its secret is removed, or the offline account is dropped
// out of ZeroSandboxOffline. In each of those states the runtime falls back to
// the restricted-token path with no read confinement, or fails outright.
//
// Doctor is the surface an operator checks precisely when they are unsure, so
// claiming read confinement it has not verified is worse than reporting less.
// This pins the narrower claim: setup is current, the role is named because it
// is derived rather than assumed, and nothing asserts the account is usable.
func TestPrincipalCheckDoesNotClaimAnUnverifiedPrincipalIsActive(t *testing.T) {
sandboxHome := t.TempDir()
workspaceRoot := t.TempDir()
t.Setenv("ZERO_WINDOWS_SANDBOX_HOME", sandboxHome)
t.Setenv("ZERO_WINDOWS_SANDBOX_IDENTITY", "1")
sandboxConfig := config.SandboxConfig{}
scope, err := sandbox.NewScope(workspaceRoot, sandboxConfig.AdditionalWriteRoots)
if err != nil {
t.Fatalf("NewScope: %v", err)
}
profile := sandbox.PermissionProfileFromPolicy(workspaceRoot, doctorSandboxPolicy(sandboxConfig), scope)
// Mirrors what windowsSandboxSetupCheck derives, so the marker it writes is
// the one the check will validate against. Nothing here provisions an
// account, which is the whole point: the marker is valid and the principal
// does not exist.
if _, err := sandbox.WriteWindowsSandboxSetupMarker(sandbox.WindowsSandboxSetupConfig{
SandboxHome: sandboxHome,
CommandCWD: workspaceRoot,
WorkspaceRoots: []string{workspaceRoot},
PermissionProfile: sandbox.WindowsSandboxProfileWithRuntimeRoots(profile, []string{workspaceRoot}),
PrincipalOptIn: true,
}); err != nil {
t.Fatalf("WriteWindowsSandboxSetupMarker: %v", err)
}
result := windowsSandboxSetupCheck("windows", sandbox.Backend{Name: sandbox.BackendWindowsRestrictedToken}, workspaceRoot, sandboxConfig)
if result == nil {
t.Fatal("expected a principal check once the marker validates and the opt-in is set")
}
if result.ID != "sandbox.principal" {
t.Fatalf("check = %q (%s): the marker did not validate, so this test is not exercising the principal branch", result.ID, result.Message)
}
lowered := strings.ToLower(result.Message)
for _, claim := range []string{"is active", "commands run as"} {
if strings.Contains(lowered, claim) {
t.Errorf("doctor asserts %q about a principal it never checked: %q", claim, result.Message)
}
}
if active, ok := result.Details["active"]; ok {
t.Errorf("details carry active=%v, which nothing here verified", active)
}
// The role is still worth reporting: it is derived from the network mode by
// the shared rule rather than assumed, and it is what an operator debugging
// network behaviour needs.
if role, ok := result.Details["role"]; !ok || strings.TrimSpace(role.(string)) == "" {
t.Errorf("the selected role should still be reported: %#v", result.Details)
}
}
+157 -29
View File
@@ -3,13 +3,93 @@
package execution
import (
"errors"
"os"
"os/exec"
"strconv"
"strings"
"syscall"
"testing"
"time"
)
// requireProcessState refuses to let a broken ps pipeline masquerade as passing
// coverage. Every test in this file proves something about zombie detection,
// and every one of those proofs is read out of ps — so if ps cannot report
// state, the tests verify nothing whatever they do next.
//
// The old code answered that with t.Skip, which is indistinguishable from
// success in CI output. This probes BOTH ps pipelines against this test
// process, which is by definition alive and not a zombie, so a working ps must
// report a live state for it. Failing that:
//
// - In CI (or with ZERO_REQUIRE_PS=1) it is a hard failure. ps is present on
// every image used today, so its absence means the pipeline regressed and
// that has to surface immediately rather than as skipped-green.
// - Elsewhere it stays a skip, so a contributor on an exotic box is not
// blocked by an environment the project does not claim to support.
//
// This only covers "ps cannot report state at all". Once it passes, a later
// failure to observe an EXPECTED state is a real defect, and the tests below
// treat it as one.
func requireProcessState(t *testing.T) {
t.Helper()
self := os.Getpid()
var reasons []string
// The production path: ps -A -o pid=,pgid=,stat=, parsed by
// processTableStateMatches. This is what signalTargetRunning consults.
if running, ok := processIsRunning(self); !ok || !running {
reasons = append(reasons, "the process table read used by signalTargetRunning reported no live state for this test process")
}
// This file's independent oracle: ps -o stat= -p <pid>. It is deliberately
// NOT the production parser, so a regression in that parser cannot hide by
// breaking the test's own precondition check in the same way.
if zombie, ok := processIsZombie(self); !ok || zombie {
reasons = append(reasons, "the per-PID state read used by this test file reported no live state for this test process")
}
if len(reasons) == 0 {
return
}
reason := "cannot read process state, so ps-based zombie detection is unverifiable here: " + strings.Join(reasons, "; ")
if requireWorkingPS() {
t.Fatal(reason + " — failing rather than skipping because CI (or ZERO_REQUIRE_PS) is set, where ps is expected to work")
}
t.Skip(reason)
}
// requireWorkingPS reports whether an unusable ps must fail the run instead of
// skipping it. ZERO_REQUIRE_PS lets a developer opt into the strict behavior
// locally to reproduce what CI does.
func requireWorkingPS() bool {
if strings.TrimSpace(os.Getenv("ZERO_REQUIRE_PS")) != "" {
return true
}
return strings.TrimSpace(os.Getenv("CI")) != ""
}
// awaitZombie blocks until pid is an unreaped zombie. The caller must not have
// waited on the child, so the zombie is a stable state rather than a race: it
// persists until something reaps it. Failing to reach it therefore means the
// child never exited or ps stopped reporting its state, and both are defects
// rather than reasons to skip.
func awaitZombie(t *testing.T, pid int) {
t.Helper()
deadline := time.Now().Add(5 * time.Second)
for {
zombie, ok := processIsZombie(pid)
if ok && zombie {
return
}
if time.Now().After(deadline) {
if !ok {
t.Fatalf("pid %d left the process table without being reaped; this test never waited on it, so its zombie row must persist", pid)
}
t.Fatalf("pid %d did not reach the zombie state within 5s", pid)
}
time.Sleep(10 * time.Millisecond)
}
}
// TestTerminateProcessTreeTreatsZombieGroupAsExited is the regression test for
// counting unreaped processes as alive: kill(2) with signal 0 succeeds for a
// zombie, so a group whose only remaining member is waiting to be reaped used to
@@ -17,6 +97,8 @@ import (
// as "did not exit after SIGKILL" — a spurious cleanup failure whose timing
// depended on when the reaper ran.
func TestTerminateProcessTreeTreatsZombieGroupAsExited(t *testing.T) {
requireProcessState(t)
cmd := exec.Command("sh", "-c", "exit 0")
ConfigureProcessGroup(cmd)
if err := cmd.Start(); err != nil {
@@ -27,23 +109,59 @@ func TestTerminateProcessTreeTreatsZombieGroupAsExited(t *testing.T) {
// is exactly the state a terminated tree passes through.
t.Cleanup(func() { _ = cmd.Wait() })
deadline := time.Now().Add(5 * time.Second)
for {
if zombie, ok := processIsZombie(pid); ok && zombie {
break
awaitZombie(t, pid)
// Whether a zombie-only group can be signalled is a KERNEL difference, not a
// ps one. On Linux the zombie still occupies the process table and
// kill(-pid, 0) succeeds, so signalTargetRunning goes on to consult ps —
// which is the branch this test was written for. Darwin returns EPERM for
// the same call, so signalTargetRunning short-circuits on the kill and never
// reaches the ps check.
//
// The OUTCOME is identical and is asserted on every platform below. Only the
// branch that produces it differs, so record that instead of skipping the
// whole test (which would drop Darwin's coverage of the outcome) or failing
// it (which would report a supported platform as broken). Any errno other
// than EPERM means the group is not in the state this test needs.
psPathReachable := true
if err := syscall.Kill(-pid, syscall.Signal(0)); err != nil {
if !errors.Is(err, syscall.EPERM) {
t.Fatalf("zombie group %d is unsignalable with an unexpected error (%v); the zombie case is not present, so nothing below would be meaningful", pid, err)
}
if time.Now().After(deadline) {
t.Skip("child did not reach the zombie state; ps output is unavailable in this environment")
}
time.Sleep(10 * time.Millisecond)
}
if syscall.Kill(-pid, syscall.Signal(0)) != nil {
t.Skip("the zombie group is no longer signalable; cannot exercise the case")
psPathReachable = false
t.Logf("kernel refuses to signal zombie-only group %d (EPERM); signalTargetRunning answers from kill(2) rather than ps here, so the ps-based zombie check is asserted directly on platforms that allow the signal", pid)
}
// Where the signal IS permitted, pin the ps-based check itself. This is the
// regression #774 was about, and asserting only signalTargetRunning would
// let a ps-side regression hide behind the kill result.
if psPathReachable {
running, ok := processGroupHasRunningMember(pid)
if !ok {
t.Fatalf("process group %d has no rows while its zombie is unreaped; ps stopped reporting the group rather than the group being gone", pid)
}
if running {
t.Fatal("the ps-based group check must not report a zombie-only group as running")
}
}
// Valid on every platform: whichever branch answers, a zombie-only group is
// not running. On a kernel that refuses the signal this is the whole of the
// coverage, and it is more than the previous skip provided.
if signalTargetRunning(-pid) {
t.Fatal("a group holding only a zombie must not count as running")
}
if !psPathReachable {
// terminateTarget's own kill(2) would hit the same EPERM and be returned
// as a cleanup error, so this assertion cannot be made where the kernel
// refuses the signal. That may be a real gap in TerminateProcessTree on
// such a platform rather than a test limitation — a zombie-only tree is
// already exited and should terminate cleanly — but confirming and
// changing that is a production fix, not this test's business.
t.Log("skipping the TerminateProcessTree assertion: its kill(2) would hit the same EPERM as the probe above")
return
}
if err := TerminateProcessTree(pid, 50*time.Millisecond, 10*time.Millisecond); err != nil {
t.Fatalf("TerminateProcessTree on an already-exited tree: %v", err)
}
@@ -53,6 +171,8 @@ func TestTerminateProcessTreeTreatsZombieGroupAsExited(t *testing.T) {
// a live member must still be seen as running and then actually stopped, so the
// zombie tolerance above cannot degrade into ignoring real processes.
func TestTerminateProcessTreeStopsRunningGroup(t *testing.T) {
requireProcessState(t)
cmd := exec.Command("sh", "-c", "sleep 30")
ConfigureProcessGroup(cmd)
if err := cmd.Start(); err != nil {
@@ -67,7 +187,15 @@ func TestTerminateProcessTreeStopsRunningGroup(t *testing.T) {
if err := TerminateProcessTree(pid, time.Second, 10*time.Millisecond); err != nil {
t.Fatalf("TerminateProcessTree: %v", err)
}
if running, ok := processGroupHasRunningMember(pid); ok && running {
// ok reports whether ANY row for this group was found. This test never waits
// on the child, so its zombie row must still be there — an unknown state
// means ps stopped answering, not that the group is gone. Treating that as a
// pass is what let this assertion succeed without checking anything.
running, ok := processGroupHasRunningMember(pid)
if !ok {
t.Fatalf("process group %d has no rows after termination; the unreaped child's zombie row must persist, so the group's state is undeterminable rather than clean", pid)
}
if running {
t.Fatal("the group still has a running member after termination")
}
}
@@ -82,6 +210,8 @@ func TestTerminateProcessTreeStopsRunningGroup(t *testing.T) {
// "unknown" resulted every time, and signalTargetRunning conservatively (and
// incorrectly) treated a zombie individual-PID target as still running.
func TestSignalTargetRunningTreatsZombieIndividualPIDAsExited(t *testing.T) {
requireProcessState(t)
cmd := exec.Command("sh", "-c", "exit 0")
// Deliberately do NOT call ConfigureProcessGroup: the child inherits this
// test process's group, so it is not its own leader and processSignalTarget
@@ -92,40 +222,38 @@ func TestSignalTargetRunningTreatsZombieIndividualPIDAsExited(t *testing.T) {
pid := cmd.Process.Pid
t.Cleanup(func() { _ = cmd.Wait() })
deadline := time.Now().Add(5 * time.Second)
for {
if zombie, ok := processIsZombie(pid); ok && zombie {
break
}
if time.Now().After(deadline) {
t.Skip("child did not reach the zombie state; ps output is unavailable in this environment")
}
time.Sleep(10 * time.Millisecond)
}
awaitZombie(t, pid)
target, err := processSignalTarget(pid)
if err != nil {
t.Fatalf("processSignalTarget: %v", err)
}
// The child inherited this process's group, so its PGID is this process's
// PID and can never equal its own PID. A negative target here would mean
// processSignalTarget misread the group, which is the very thing the
// individual-PID fallback exists to handle.
if target != pid {
t.Skip("child unexpectedly became its own group leader; cannot exercise the individual-PID fallback")
t.Fatalf("processSignalTarget(%d) = %d, want the individual PID: the child inherited this process's group, so it is not its own leader", pid, target)
}
if signalTargetRunning(target) {
t.Fatal("a zombie individual-PID target must not count as running")
}
}
// processIsZombie reports a process's zombie state via ps. ok is false when the
// state could not be read.
// processIsZombie reports a process's zombie state via ps. It is deliberately a
// separate, per-PID ps invocation rather than a reuse of the production
// process-table parser: these tests assert what that parser concludes, so
// borrowing it to establish their own preconditions would let one regression
// silence both sides at once.
//
// ok is false when no state could be read, which after requireProcessState has
// passed means the process left the table rather than that ps is broken.
func processIsZombie(pid int) (zombie bool, ok bool) {
output, err := exec.Command("ps", "-o", "stat=", "-p", strconv.Itoa(pid)).Output()
if err != nil {
return false, false
}
state := string(output)
for len(state) > 0 && (state[0] == ' ' || state[0] == '\n' || state[0] == '\t') {
state = state[1:]
}
state := strings.TrimLeft(string(output), " \n\t")
if state == "" {
return false, false
}
+9 -63
View File
@@ -5,93 +5,39 @@ import (
"fmt"
"os"
"path/filepath"
"sync/atomic"
"time"
"github.com/Gitlawb/zero/internal/lockutil"
)
// Cross-process lock tuning for the audit log. The lock is held only across a
// single read-then-append (milliseconds), so the timeout is generous and the
// stale threshold sits far above any real hold.
// single read-then-append (milliseconds), so the timeout is generous relative
// to any real hold.
const (
auditLockTimeout = 10 * time.Second
auditLockStaleAfter = 60 * time.Second
auditLockRetryDelay = 20 * time.Millisecond
)
var auditLockSeq atomic.Uint64
// lockAudit takes a cross-process exclusive lock on the audit log by
// O_EXCL-creating a sibling "<auditPath>.lock" file (removed on release). It makes
// lockAudit takes a cross-process advisory lock on the audit log through a
// stable sibling "<auditPath>.lock" file. It makes
// the lastSequence()+append in append() atomic across processes; the audit log is
// a shared global file, so without it two processes can read the same last
// sequence and write duplicate numbers. A stale lock from a crashed holder (older
// than auditLockStaleAfter) is reclaimed. Wall-clock time is used deliberately so
// lock timing never depends on the store's injectable clock and the stale check
// compares against real file mtimes. This mirrors internal/cron/lock.go and
// internal/oauth/lock.go. Uses internal/lockutil for lock restore and reclaim.
// sequence and write duplicate numbers. Kernel lock lifetime provides crash
// recovery without moving or deleting the lock path.
func (store *AuditStore) lockAudit() (func(), error) {
lockPath := store.auditPath + ".lock"
if err := os.MkdirAll(filepath.Dir(lockPath), 0o700); err != nil {
return nil, err
}
token := fmt.Sprintf("%d-%d-%d", os.Getpid(), time.Now().UnixNano(), auditLockSeq.Add(1))
deadline := time.Now().Add(auditLockTimeout)
for {
f, err := os.OpenFile(lockPath, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
lock, err := lockutil.TryAcquireFileLock(lockPath)
if err == nil {
// A partial write would leave a lock file without our token, so the
// releaser could never delete it — stranding the lock. Fail closed.
if _, werr := f.WriteString(token); werr != nil {
_ = f.Close()
_ = lockutil.RemoveLockFile(lockPath)
return nil, fmt.Errorf("hooks: write audit lock: %w", werr)
}
if cerr := f.Close(); cerr != nil {
_ = lockutil.RemoveLockFile(lockPath)
return nil, fmt.Errorf("hooks: close audit lock: %w", cerr)
}
var released bool
return func() {
if released {
return
}
released = true
if data, rerr := os.ReadFile(lockPath); rerr == nil && string(data) == token {
_ = lockutil.RemoveLockFile(lockPath)
}
}, nil
return func() { _ = lock.Release() }, nil
}
// On Windows a concurrent holder's os.Remove leaves the lock file in a
// "delete pending" state, so an O_EXCL create races it with
// ERROR_ACCESS_DENIED (os.ErrPermission) rather than ErrExist. Treat both
// as contention and retry.
if !errors.Is(err, os.ErrExist) && !errors.Is(err, os.ErrPermission) {
if !errors.Is(err, lockutil.ErrLockHeld) {
return nil, fmt.Errorf("hooks: acquire audit lock: %w", err)
}
// Reclaim a stale lock left by a crashed holder — atomically (H3). A blind
// Remove lets two racers both reclaim + recreate and so both hold the lock;
// reclaimStaleLock renames the file aside (only one rename wins) and restores
// it if it turns out fresh, so a live lock is never deleted out from under it.
if info, statErr := os.Stat(lockPath); statErr == nil && time.Since(info.ModTime()) > auditLockStaleAfter {
cleared, rerr := lockutil.ReclaimStaleLock(lockPath, token, func(reclaimedPath string) bool {
info, err := os.Stat(reclaimedPath)
return err == nil && time.Since(info.ModTime()) <= auditLockStaleAfter
})
if rerr != nil {
// Reclaim hit a hard failure: the rename aside failed outright, or a
// live holder's lock could not be put back (the lock path may be
// missing, so re-acquiring would break mutual exclusion). Fail closed
// instead of spinning to the deadline.
return nil, fmt.Errorf("hooks: reclaim stale audit lock: %w", rerr)
}
if cleared {
continue
}
// Lost the reclaim race (or it was actually fresh) — fall through to the
// bounded wait rather than hot-spinning on a reclaim that never wins.
}
if !time.Now().Before(deadline) {
return nil, fmt.Errorf("hooks: timed out acquiring audit lock")
}
+111 -48
View File
@@ -1,64 +1,127 @@
// Package lockutil provides the platform-specific file primitives behind the
// O_EXCL lock files in cron, daemon, hooks, oauth, and swarm: a no-overwrite
// restore for locks that were sidelined during a stale-reclaim attempt, and a
// lock file remover with one cross-platform contract (missing files are a
// no-op; Windows retries transient sharing violations).
// Package lockutil provides cross-process advisory file locks used by cron,
// daemon, hooks, oauth, and swarm. Locks are held by the kernel for the lifetime
// of an open file handle, so process exit releases them without a stale-file
// reclaim protocol. The lock file itself is stable: holders never rename or
// remove it while another process may still have its inode locked.
package lockutil
import (
"io"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
)
// restoreByCopy restores reclaimed to path without overwriting an existing
// path, as a fallback for when the platform's primary no-replace primitive
// (hard link on POSIX, MoveFileEx on Windows) fails for a reason other than
// the destination existing. It stages a full copy under a private name next
// to reclaimed and publishes it to path with publish (the same no-replace
// primitive the caller's platform uses for the primary restore), so path
// never appears in a partially-copied state: a crash between staging and
// publish leaves path exactly as it was (missing or, if a competing holder
// created it, untouched), never a truncated file that a PID/content-based
// liveness check could mistake for dead. Leaving path missing here would let
// the next O_EXCL create succeed while the sidelined holder is still live,
// breaking mutual exclusion. publish keeps the no-overwrite guarantee: a new
// holder that appeared in the meantime wins and this returns os.ErrExist. The
// copy resets the lock's mtime to now, which only makes the restored lock
// look fresher; that is safe, since it is being handed back to a live holder.
func restoreByCopy(reclaimed, path string, publish func(from, to string) error) error {
staged := reclaimed + ".copy"
src, err := os.Open(reclaimed)
var (
// ErrLockHeld reports that another process or goroutine currently holds the
// advisory lock. Callers may wait and retry until their own deadline.
ErrLockHeld = errors.New("lockutil: lock is held")
// ErrLockReleased reports an attempt to update metadata after Release.
ErrLockReleased = errors.New("lockutil: lock is released")
)
var holderSequence atomic.Uint64
// FileLock is an exclusive advisory lock held through an open file handle.
// Release is idempotent. The lock path remains on disk after release; deleting
// or replacing it would let different processes lock different inodes.
type FileLock struct {
mu sync.Mutex
file *os.File
state platformLockState
released bool
releaseErr error
}
// TryAcquireFileLock attempts to take an exclusive advisory lock without
// waiting. A contended lock returns ErrLockHeld. The kernel releases a lock if
// its process exits, so an old unlocked file never needs stale reclamation.
func TryAcquireFileLock(path string) (*FileLock, error) {
return TryAcquireFileLockAt(filepath.Dir(path), path)
}
// TryAcquireFileLockAt attempts to lock path while confining every component
// below root to a handle-relative traversal. root is the caller's trusted
// boundary and may itself be reached through a legitimate link; links and
// reparse points below it are rejected before metadata can be written.
func TryAcquireFileLockAt(root, path string) (*FileLock, error) {
if root == "" {
return nil, errors.New("lockutil: lock root is empty")
}
relative, err := filepath.Rel(root, path)
if err != nil {
return err
return nil, fmt.Errorf("lockutil: resolve lock path relative to root: %w", err)
}
dst, err := os.OpenFile(staged, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
relative = filepath.Clean(relative)
if relative == "." || relative == ".." || filepath.IsAbs(relative) || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
return nil, fmt.Errorf("lockutil: lock path %q escapes root %q", path, root)
}
file, err := openLockFileAt(root, relative, path)
if err != nil {
_ = src.Close()
return err
return nil, fmt.Errorf("lockutil: open lock file: %w", err)
}
_, err = io.Copy(dst, src)
// Close the source before removing anything below: Go opens files without
// FILE_SHARE_DELETE on Windows, so deleting reclaimed or staged while src
// is open would fail with a sharing violation.
_ = src.Close()
state, contended, err := tryLockFile(file)
if err != nil {
_ = dst.Close()
_ = os.Remove(staged)
return err
_ = file.Close()
return nil, fmt.Errorf("lockutil: acquire advisory lock: %w", err)
}
if err := dst.Close(); err != nil {
_ = os.Remove(staged)
return err
if contended {
_ = file.Close()
return nil, ErrLockHeld
}
if err := publish(staged, path); err != nil {
_ = os.Remove(staged)
return err
lock := &FileLock{file: file, state: state}
metadata := []byte(fmt.Sprintf("%d-%d\n", os.Getpid(), holderSequence.Add(1)))
if err := lock.WriteMetadata(metadata); err != nil {
return nil, errors.Join(err, lock.Release())
}
return lock, nil
}
// WriteMetadata replaces the diagnostic contents of the lock file while the
// caller holds it. Metadata does not establish ownership; the kernel lock does.
func (lock *FileLock) WriteMetadata(data []byte) error {
if lock == nil {
return ErrLockReleased
}
lock.mu.Lock()
defer lock.mu.Unlock()
if lock.released || lock.file == nil {
return ErrLockReleased
}
if err := lock.file.Truncate(0); err != nil {
return fmt.Errorf("lockutil: truncate lock metadata: %w", err)
}
if _, err := lock.file.Seek(0, 0); err != nil {
return fmt.Errorf("lockutil: seek lock metadata: %w", err)
}
if _, err := lock.file.Write(data); err != nil {
return fmt.Errorf("lockutil: write lock metadata: %w", err)
}
// publish may leave staged behind (a hard-link publish keeps both names),
// so remove it explicitly; a move-based publish already made this a no-op.
_ = os.Remove(staged)
// The lock is back at path, so the restore has succeeded; failing to clean
// up the sidelined name must not be reported as a failed restore.
_ = RemoveLockFile(reclaimed)
return nil
}
// Release unlocks and closes the held file handle. It deliberately leaves the
// stable lock file in place. Repeated calls return the first release result.
func (lock *FileLock) Release() error {
if lock == nil {
return nil
}
lock.mu.Lock()
defer lock.mu.Unlock()
if lock.released {
return lock.releaseErr
}
lock.released = true
unlockErr := unlockFile(lock.file, &lock.state)
closeErr := lock.file.Close()
lock.file = nil
if err := errors.Join(unlockErr, closeErr); err != nil {
lock.releaseErr = fmt.Errorf("lockutil: release advisory lock: %w", err)
}
return lock.releaseErr
}
+82 -33
View File
@@ -4,43 +4,92 @@ package lockutil
import (
"errors"
"io/fs"
"fmt"
"os"
"path/filepath"
"strings"
"golang.org/x/sys/unix"
)
// RestoreLockFile restores a sidelined lock file on non-Windows platforms. It
// uses os.Link so a competing lock created at path in the meantime wins: the
// link fails with os.ErrExist instead of overwriting it. If the link fails for
// any other reason (hardlink-incapable filesystems such as FAT or some
// FUSE/network mounts, ENOSPC, EPERM), it falls back to an O_EXCL copy rather
// than leaving path missing and the live holder's lock stranded in reclaimed.
// Once the lock is back at path the restore has succeeded, so a failed cleanup
// of the sidelined name is not reported as an error; the leftover file is
// invisible to the lock protocol.
func RestoreLockFile(reclaimed, path string) error {
err := os.Link(reclaimed, path)
type platformLockState struct{}
func openLockFileAt(root, relative, displayPath string) (_ *os.File, resultErr error) {
parentFD, err := unix.Open(root, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
if err != nil {
return nil, fmt.Errorf("open lock root: %w", err)
}
defer func() {
if parentFD >= 0 {
resultErr = errors.Join(resultErr, unix.Close(parentFD))
}
}()
components := strings.Split(filepath.Clean(relative), string(filepath.Separator))
for _, component := range components[:len(components)-1] {
nextFD, openErr := unix.Openat(parentFD, component, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC|unix.O_NOFOLLOW, 0)
if openErr != nil {
return nil, fmt.Errorf("open lock path component %q: %w", component, openErr)
}
if closeErr := unix.Close(parentFD); closeErr != nil {
_ = unix.Close(nextFD)
parentFD = -1
return nil, fmt.Errorf("close lock path component: %w", closeErr)
}
parentFD = nextFD
}
name := components[len(components)-1]
fd := -1
// Darwin can transiently return ENOENT when many callers race to create the
// same O_NOFOLLOW entry. The bound parent handle cannot be redirected, so a
// bounded retry preserves first-use contention without weakening no-follow.
for attempt := 0; attempt < 10; attempt++ {
fd, err = unix.Openat(parentFD, name, unix.O_CREAT|unix.O_RDWR|unix.O_CLOEXEC|unix.O_NOFOLLOW, 0o600)
if !errors.Is(err, unix.ENOENT) {
break
}
}
if err != nil {
return nil, err
}
file := os.NewFile(uintptr(fd), displayPath)
if file == nil {
_ = unix.Close(fd)
return nil, errors.New("convert lock handle to file")
}
var stat unix.Stat_t
if err := unix.Fstat(fd, &stat); err != nil {
return nil, errors.Join(fmt.Errorf("inspect lock file: %w", err), file.Close())
}
if stat.Mode&unix.S_IFMT != unix.S_IFREG {
return nil, errors.Join(errors.New("refusing non-regular lock file"), file.Close())
}
if stat.Nlink != 1 {
return nil, errors.Join(errors.New("refusing multiply-linked lock file"), file.Close())
}
if err := unix.Close(parentFD); err != nil {
parentFD = -1
return nil, errors.Join(fmt.Errorf("close lock parent: %w", err), file.Close())
}
parentFD = -1
return file, nil
}
func tryLockFile(file *os.File) (platformLockState, bool, error) {
err := unix.Flock(int(file.Fd()), unix.LOCK_EX|unix.LOCK_NB)
if err == nil {
_ = RemoveLockFile(reclaimed)
return platformLockState{}, false, nil
}
if errors.Is(err, unix.EWOULDBLOCK) || errors.Is(err, unix.EAGAIN) {
return platformLockState{}, true, nil
}
return platformLockState{}, false, err
}
func unlockFile(file *os.File, _ *platformLockState) error {
if file == nil {
return nil
}
if errors.Is(err, os.ErrExist) {
return err
}
return restoreByCopy(reclaimed, path, os.Link)
}
// isReclaimContended reports whether a failed rename-aside of a suspected
// stale lock means it was lost to a racer rather than a hard failure. POSIX
// rename has no contention errno (a lost race surfaces only as ENOENT, which
// ReclaimStaleLock already treats as benign), so nothing extra is benign here.
func isReclaimContended(error) bool { return false }
// RemoveLockFile removes a lock file on non-Windows platforms. Removing an
// already-missing file reports nil, matching the Windows implementation, so
// callers see one cross-platform contract.
func RemoveLockFile(path string) error {
if err := os.Remove(path); err != nil && !errors.Is(err, fs.ErrNotExist) {
return err
}
return nil
return unix.Flock(int(file.Fd()), unix.LOCK_UN)
}
+205 -101
View File
@@ -1,116 +1,220 @@
package lockutil
import (
"bufio"
"errors"
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"sync"
"testing"
)
func TestRestoreLockFile(t *testing.T) {
tempDir := t.TempDir()
path := filepath.Join(tempDir, "lock")
reclaimed := filepath.Join(tempDir, "lock.stale.token")
// 1. Successful restore when target does not exist
if err := os.WriteFile(reclaimed, []byte("token"), 0o600); err != nil {
t.Fatal(err)
}
if err := RestoreLockFile(reclaimed, path); err != nil {
t.Fatalf("RestoreLockFile failed: %v", err)
}
if data, err := os.ReadFile(path); err != nil || string(data) != "token" {
t.Fatalf("restored lock content = %q, err %v; want %q intact", data, err, "token")
}
if _, err := os.Stat(reclaimed); !os.IsNotExist(err) {
t.Fatalf("expected sidelined lock to be cleaned up/removed: %v", err)
}
// 2. Fail when target already exists (prevent overwrite)
if err := os.WriteFile(reclaimed, []byte("token"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("competing"), 0o600); err != nil {
t.Fatal(err)
}
err := RestoreLockFile(reclaimed, path)
if err == nil {
t.Fatal("expected RestoreLockFile to fail when target exists")
}
if !errors.Is(err, os.ErrExist) {
t.Fatalf("expected os.ErrExist, got: %v", err)
}
// The sidelined lock must still exist on failure, and the competing lock
// must be untouched.
if _, err := os.Stat(reclaimed); err != nil {
t.Fatalf("expected sidelined lock to still exist: %v", err)
}
if data, _ := os.ReadFile(path); string(data) != "competing" {
t.Fatalf("competing lock was clobbered: %q", data)
}
}
func TestRestoreByCopy(t *testing.T) {
tempDir := t.TempDir()
path := filepath.Join(tempDir, "lock")
reclaimed := filepath.Join(tempDir, "lock.stale.token")
// 1. Restores content and cleans up the sidelined file when the target is
// free (the fallback taken when the platform's no-replace primitive fails
// for a reason other than the destination existing).
if err := os.WriteFile(reclaimed, []byte("token"), 0o600); err != nil {
t.Fatal(err)
}
if err := restoreByCopy(reclaimed, path, os.Link); err != nil {
t.Fatalf("restoreByCopy failed: %v", err)
}
if data, err := os.ReadFile(path); err != nil || string(data) != "token" {
t.Fatalf("restored lock content = %q, err %v; want %q intact", data, err, "token")
}
if _, err := os.Stat(reclaimed); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("expected sidelined lock to be cleaned up after copy: %v", err)
}
if _, err := os.Stat(reclaimed + ".copy"); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("expected staged copy to be cleaned up: %v", err)
}
// 2. Keeps the no-overwrite guarantee: a competing lock at the target wins
// and the sidelined lock survives for the caller's ErrExist cleanup.
if err := os.WriteFile(reclaimed, []byte("token"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("competing"), 0o600); err != nil {
t.Fatal(err)
}
if err := restoreByCopy(reclaimed, path, os.Link); !errors.Is(err, os.ErrExist) {
t.Fatalf("expected os.ErrExist when target exists, got: %v", err)
}
if data, _ := os.ReadFile(path); string(data) != "competing" {
t.Fatalf("competing lock was clobbered: %q", data)
}
if _, err := os.Stat(reclaimed); err != nil {
t.Fatalf("expected sidelined lock to still exist: %v", err)
}
if _, err := os.Stat(reclaimed + ".copy"); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("expected staged copy to be cleaned up after a failed publish: %v", err)
}
}
func TestRemoveLockFile(t *testing.T) {
func TestFileLockSerializesAndKeepsStablePath(t *testing.T) {
path := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(path, []byte("token"), 0o600); err != nil {
t.Fatal(err)
first, err := TryAcquireFileLock(path)
if err != nil {
t.Fatalf("first acquire: %v", err)
}
if err := RemoveLockFile(path); err != nil {
t.Fatalf("RemoveLockFile failed: %v", err)
firstInfo, err := os.Stat(path)
if err != nil {
t.Fatalf("stat first lock path: %v", err)
}
if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("expected lock file to be removed: %v", err)
if _, err := TryAcquireFileLock(path); !errors.Is(err, ErrLockHeld) {
t.Fatalf("contended acquire = %v, want ErrLockHeld", err)
}
// Removing an already-missing lock file is a no-op on every platform.
if err := RemoveLockFile(path); err != nil {
t.Fatalf("RemoveLockFile on a missing file = %v, want nil", err)
if err := first.Release(); err != nil {
t.Fatalf("release first: %v", err)
}
releasedInfo, err := os.Stat(path)
if err != nil {
t.Fatalf("release removed stable lock path: %v", err)
}
if !os.SameFile(firstInfo, releasedInfo) {
t.Fatal("release replaced the stable lock file")
}
second, err := TryAcquireFileLock(path)
if err != nil {
t.Fatalf("acquire after release: %v", err)
}
secondInfo, err := os.Stat(path)
if err != nil {
t.Fatalf("stat second lock path: %v", err)
}
if !os.SameFile(firstInfo, secondInfo) {
t.Fatal("reacquire used a different lock file")
}
if err := second.Release(); err != nil {
t.Fatalf("release second: %v", err)
}
if err := second.Release(); err != nil {
t.Fatalf("idempotent release: %v", err)
}
}
func TestFileLockMetadata(t *testing.T) {
path := filepath.Join(t.TempDir(), "lock")
lock, err := TryAcquireFileLock(path)
if err != nil {
t.Fatal(err)
}
if err := lock.WriteMetadata([]byte("holder\n")); err != nil {
t.Fatalf("write metadata: %v", err)
}
data, err := os.ReadFile(path)
if err != nil || string(data) != "holder\n" {
t.Fatalf("metadata = %q, err %v", data, err)
}
if err := lock.Release(); err != nil {
t.Fatal(err)
}
if err := lock.WriteMetadata(nil); !errors.Is(err, ErrLockReleased) {
t.Fatalf("write after release = %v, want ErrLockReleased", err)
}
}
func TestFileLockConcurrentFirstAcquisition(t *testing.T) {
path := filepath.Join(t.TempDir(), "lock")
const contenders = 200
start := make(chan struct{})
results := make(chan error, contenders)
locks := make(chan *FileLock, contenders)
var ready sync.WaitGroup
ready.Add(contenders)
for range contenders {
go func() {
ready.Done()
<-start
lock, err := TryAcquireFileLock(path)
if lock != nil {
locks <- lock
}
results <- err
}()
}
ready.Wait()
close(start)
acquired := 0
for range contenders {
err := <-results
switch {
case err == nil:
acquired++
case !errors.Is(err, ErrLockHeld):
t.Errorf("concurrent first acquire: %v", err)
}
}
close(locks)
for lock := range locks {
if err := lock.Release(); err != nil {
t.Error(err)
}
}
if acquired != 1 {
t.Fatalf("concurrent first acquire: got %d holders, want 1", acquired)
}
}
func TestFileLockRefusesMultiplyLinkedEntry(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "target")
lockPath := filepath.Join(dir, "lock")
const sentinel = "do not overwrite"
if err := os.WriteFile(target, []byte(sentinel), 0o600); err != nil {
t.Fatal(err)
}
if err := os.Link(target, lockPath); err != nil {
t.Fatalf("create redirected hard link: %v", err)
}
if lock, err := TryAcquireFileLock(lockPath); err == nil {
_ = lock.Release()
t.Fatal("TryAcquireFileLock accepted a multiply-linked entry")
}
data, err := os.ReadFile(target)
if err != nil {
t.Fatal(err)
}
if string(data) != sentinel {
t.Fatalf("redirected target was modified: %q", data)
}
}
func TestFileLockAtRejectsEscapingPath(t *testing.T) {
base := t.TempDir()
outside := filepath.Join(filepath.Dir(base), "outside.lock")
if lock, err := TryAcquireFileLockAt(base, outside); err == nil {
_ = lock.Release()
t.Fatal("TryAcquireFileLockAt accepted a path outside its root")
}
if _, err := os.Stat(outside); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("escaping path was created: %v", err)
}
}
// TestFileLockReleasedByProcessExit proves crash recovery is provided by the
// kernel lock lifetime. The child exits without calling Release; the next
// process can then lock the same persistent file without moving or deleting it.
func TestFileLockReleasedByProcessExit(t *testing.T) {
path := filepath.Join(t.TempDir(), "lock")
cmd := exec.Command(os.Args[0], "-test.run=^TestFileLockHelperProcess$")
cmd.Env = append(os.Environ(), "ZERO_LOCKUTIL_HELPER=1", "ZERO_LOCKUTIL_PATH="+path)
stdout, err := cmd.StdoutPipe()
if err != nil {
t.Fatal(err)
}
stdin, err := cmd.StdinPipe()
if err != nil {
t.Fatal(err)
}
cmd.Stderr = os.Stderr
if err := cmd.Start(); err != nil {
t.Fatal(err)
}
ready, err := bufio.NewReader(stdout).ReadString('\n')
if err != nil || strings.TrimSpace(ready) != "ready" {
_ = cmd.Process.Kill()
_ = cmd.Wait()
t.Fatalf("helper readiness = %q, err %v", ready, err)
}
if _, err := TryAcquireFileLock(path); !errors.Is(err, ErrLockHeld) {
_ = cmd.Process.Kill()
_ = cmd.Wait()
t.Fatalf("acquire while helper holds lock = %v, want ErrLockHeld", err)
}
if err := stdin.Close(); err != nil {
t.Fatal(err)
}
if err := cmd.Wait(); err != nil {
t.Fatalf("helper exit: %v", err)
}
lock, err := TryAcquireFileLock(path)
if err != nil {
t.Fatalf("acquire after helper exit: %v", err)
}
if err := lock.Release(); err != nil {
t.Fatal(err)
}
}
func TestFileLockHelperProcess(t *testing.T) {
if os.Getenv("ZERO_LOCKUTIL_HELPER") != "1" {
return
}
lock, err := TryAcquireFileLock(os.Getenv("ZERO_LOCKUTIL_PATH"))
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(2)
}
if _, err := fmt.Fprintln(os.Stdout, "ready"); err != nil {
os.Exit(3)
}
_, _ = io.Copy(io.Discard, os.Stdin)
runtime.KeepAlive(lock) // deliberately exit without Release
}
+183 -58
View File
@@ -4,77 +4,202 @@ package lockutil
import (
"errors"
"fmt"
"os"
"syscall"
"time"
"path/filepath"
"strings"
"unsafe"
"golang.org/x/sys/windows"
)
// RestoreLockFile restores a sidelined lock file on Windows. It moves the file
// with a no-replace MoveFileEx so a competing lock created at path in the
// meantime wins: the move fails with os.ErrExist instead of overwriting it. If
// the move fails for any other reason (ERROR_SHARING_VIOLATION or
// ERROR_ACCESS_DENIED under concurrent access to the file), it falls back to
// an O_EXCL copy rather than leaving path missing and the live holder's lock
// stranded in reclaimed.
func RestoreLockFile(reclaimed, path string) error {
err := moveFileNoReplace(reclaimed, path)
if err == nil || errors.Is(err, os.ErrExist) {
return err
}
return restoreByCopy(reclaimed, path, moveFileNoReplace)
type platformLockState struct {
overlapped windows.Overlapped
}
// moveFileNoReplace renames from to to, failing if to already exists. It calls
// MoveFileExW directly via golang.org/x/sys/windows (the standard library's
// syscall package does not export MoveFileEx on this platform) with no flags,
// so an existing destination fails with ERROR_ALREADY_EXISTS, which satisfies
// errors.Is(err, os.ErrExist) (pinned by
// TestMoveFileNoReplaceMapsAlreadyExistsToErrExist), instead of overwriting it
// the way os.Rename does on Windows (it passes MOVEFILE_REPLACE_EXISTING to
// match POSIX rename semantics cross-platform).
func moveFileNoReplace(from, to string) error {
fromPtr, err := windows.UTF16PtrFromString(from)
const lockByteOffsetHigh = 1
func openLockFileAt(root, relative, displayPath string) (_ *os.File, resultErr error) {
rootPtr, err := windows.UTF16PtrFromString(root)
if err != nil {
return err
return nil, err
}
toPtr, err := windows.UTF16PtrFromString(to)
parent, err := windows.CreateFile(
rootPtr,
windows.FILE_READ_ATTRIBUTES|windows.FILE_TRAVERSE|windows.SYNCHRONIZE,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
nil,
windows.OPEN_EXISTING,
windows.FILE_FLAG_BACKUP_SEMANTICS,
0,
)
if err != nil {
return err
return nil, fmt.Errorf("open lock root: %w", err)
}
return windows.MoveFileEx(fromPtr, toPtr, 0)
}
// isReclaimContended reports whether a failed rename-aside of a suspected
// stale lock means the file was concurrently open or pending deletion (which
// surfaces as ERROR_SHARING_VIOLATION or ERROR_ACCESS_DENIED on Windows)
// rather than a hard failure, so ReclaimStaleLock reports a lost race instead
// of an error. Neither errno satisfies errors.Is(err, os.ErrExist), and
// ERROR_SHARING_VIOLATION does not satisfy errors.Is(err, os.ErrPermission)
// either, so the classification compares raw errnos like RemoveLockFile below.
func isReclaimContended(err error) bool {
var errno syscall.Errno
return errors.As(err, &errno) && (errno == windows.ERROR_SHARING_VIOLATION || errno == windows.ERROR_ACCESS_DENIED)
}
// RemoveLockFile removes a lock file on Windows, retrying on the sharing
// violation or access denied errors that are common under heavy concurrent
// contention. Removing an already-missing file reports nil, matching the
// non-Windows implementation, so callers see one cross-platform contract.
func RemoveLockFile(path string) error {
var err error
for i := 0; i < 15; i++ {
err = os.Remove(path)
if err == nil || os.IsNotExist(err) {
return nil
defer func() {
if parent != 0 {
resultErr = errors.Join(resultErr, windows.CloseHandle(parent))
}
var errno syscall.Errno
if errors.As(err, &errno) && (errno == windows.ERROR_SHARING_VIOLATION || errno == windows.ERROR_ACCESS_DENIED) {
time.Sleep(5 * time.Millisecond)
}()
components := strings.Split(filepath.Clean(relative), string(filepath.Separator))
for _, component := range components[:len(components)-1] {
next, openErr := openLockDirectoryAt(parent, component)
if openErr != nil {
return nil, fmt.Errorf("open lock path component %q: %w", component, openErr)
}
if closeErr := windows.CloseHandle(parent); closeErr != nil {
_ = windows.CloseHandle(next)
parent = 0
return nil, fmt.Errorf("close lock path component: %w", closeErr)
}
parent = next
}
handle, err := openLockFileHandleAt(parent, components[len(components)-1])
if err != nil {
return nil, err
}
file := os.NewFile(uintptr(handle), displayPath)
if file == nil {
_ = windows.CloseHandle(handle)
return nil, errors.New("convert lock handle to file")
}
if err := windows.CloseHandle(parent); err != nil {
parent = 0
return nil, errors.Join(fmt.Errorf("close lock parent: %w", err), file.Close())
}
parent = 0
return file, nil
}
func openLockDirectoryAt(parent windows.Handle, name string) (windows.Handle, error) {
handle, err := openLockPathAt(
parent,
name,
windows.FILE_READ_ATTRIBUTES|windows.FILE_TRAVERSE|windows.SYNCHRONIZE,
windows.FILE_OPEN,
windows.FILE_ATTRIBUTE_DIRECTORY,
windows.FILE_DIRECTORY_FILE|windows.FILE_SYNCHRONOUS_IO_NONALERT|windows.FILE_OPEN_REPARSE_POINT,
)
if err != nil {
return 0, err
}
if err := validateWindowsLockHandle(handle, true); err != nil {
return 0, errors.Join(err, windows.CloseHandle(handle))
}
return handle, nil
}
func openLockFileHandleAt(parent windows.Handle, name string) (windows.Handle, error) {
for attempt := 0; attempt < 10; attempt++ {
handle, err := openLockPathAt(
parent,
name,
windows.GENERIC_READ|windows.GENERIC_WRITE|windows.SYNCHRONIZE,
windows.FILE_OPEN,
windows.FILE_ATTRIBUTE_NORMAL,
windows.FILE_NON_DIRECTORY_FILE|windows.FILE_SYNCHRONOUS_IO_NONALERT|windows.FILE_OPEN_REPARSE_POINT,
)
if err == nil {
if err := validateWindowsLockHandle(handle, false); err != nil {
return 0, errors.Join(err, windows.CloseHandle(handle))
}
return handle, nil
}
if !windowsLockPathMissing(err) {
return 0, err
}
handle, err = openLockPathAt(
parent,
name,
windows.GENERIC_READ|windows.GENERIC_WRITE|windows.SYNCHRONIZE,
windows.FILE_CREATE,
windows.FILE_ATTRIBUTE_NORMAL,
windows.FILE_NON_DIRECTORY_FILE|windows.FILE_SYNCHRONOUS_IO_NONALERT|windows.FILE_OPEN_REPARSE_POINT,
)
if err == windows.STATUS_OBJECT_NAME_COLLISION || errors.Is(err, windows.ERROR_ALREADY_EXISTS) {
continue
}
return err
if err != nil {
return 0, err
}
if err := validateWindowsLockHandle(handle, false); err != nil {
return 0, errors.Join(err, windows.CloseHandle(handle))
}
return handle, nil
}
return err
return 0, errors.New("lock file creation did not stabilize")
}
func windowsLockPathMissing(err error) bool {
return err == windows.STATUS_NO_SUCH_FILE ||
err == windows.STATUS_OBJECT_NAME_NOT_FOUND ||
err == windows.STATUS_OBJECT_PATH_NOT_FOUND ||
errors.Is(err, windows.ERROR_FILE_NOT_FOUND) ||
errors.Is(err, windows.ERROR_PATH_NOT_FOUND)
}
func openLockPathAt(parent windows.Handle, name string, access, disposition, attributes, options uint32) (windows.Handle, error) {
objectName, err := windows.NewNTUnicodeString(name)
if err != nil {
return 0, err
}
objectAttributes := &windows.OBJECT_ATTRIBUTES{
Length: uint32(unsafe.Sizeof(windows.OBJECT_ATTRIBUTES{})),
RootDirectory: parent,
ObjectName: objectName,
Attributes: windows.OBJ_CASE_INSENSITIVE | windows.OBJ_DONT_REPARSE,
}
var handle windows.Handle
err = windows.NtCreateFile(
&handle,
access,
objectAttributes,
&windows.IO_STATUS_BLOCK{},
nil,
attributes,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
disposition,
options,
0,
0,
)
return handle, err
}
func validateWindowsLockHandle(handle windows.Handle, wantDirectory bool) error {
var info windows.ByHandleFileInformation
if err := windows.GetFileInformationByHandle(handle, &info); err != nil {
return fmt.Errorf("inspect lock path handle: %w", err)
}
isDirectory := info.FileAttributes&windows.FILE_ATTRIBUTE_DIRECTORY != 0
if info.FileAttributes&windows.FILE_ATTRIBUTE_REPARSE_POINT != 0 || isDirectory != wantDirectory {
return errors.New("refusing non-file or reparse-point lock path")
}
if !wantDirectory && info.NumberOfLinks != 1 {
return errors.New("refusing multiply-linked lock file")
}
return nil
}
func tryLockFile(file *os.File) (platformLockState, bool, error) {
state := platformLockState{overlapped: windows.Overlapped{OffsetHigh: lockByteOffsetHigh}}
flags := uint32(windows.LOCKFILE_EXCLUSIVE_LOCK | windows.LOCKFILE_FAIL_IMMEDIATELY)
err := windows.LockFileEx(windows.Handle(file.Fd()), flags, 0, 1, 0, &state.overlapped)
if err == nil {
return state, false, nil
}
if errors.Is(err, windows.ERROR_LOCK_VIOLATION) || errors.Is(err, windows.ERROR_SHARING_VIOLATION) {
return platformLockState{}, true, nil
}
return platformLockState{}, false, err
}
func unlockFile(file *os.File, state *platformLockState) error {
if file == nil {
return nil
}
return windows.UnlockFileEx(windows.Handle(file.Fd()), 0, 1, 0, &state.overlapped)
}
@@ -1,83 +0,0 @@
//go:build windows
package lockutil
import (
"errors"
"os"
"path/filepath"
"testing"
"time"
"golang.org/x/sys/windows"
)
// TestMoveFileNoReplaceMapsAlreadyExistsToErrExist pins the errno coupling
// every caller keys its cleanup on: with no flags, MoveFileEx fails with
// ERROR_ALREADY_EXISTS when the destination exists, and that errno must
// satisfy errors.Is(err, os.ErrExist).
func TestMoveFileNoReplaceMapsAlreadyExistsToErrExist(t *testing.T) {
dir := t.TempDir()
from := filepath.Join(dir, "from")
to := filepath.Join(dir, "to")
if err := os.WriteFile(from, []byte("sidelined"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(to, []byte("competing"), 0o600); err != nil {
t.Fatal(err)
}
err := moveFileNoReplace(from, to)
if !errors.Is(err, windows.ERROR_ALREADY_EXISTS) {
t.Fatalf("MoveFileEx with an existing destination = %v, want ERROR_ALREADY_EXISTS", err)
}
if !errors.Is(err, os.ErrExist) {
t.Fatalf("ERROR_ALREADY_EXISTS must satisfy errors.Is(err, os.ErrExist), got: %v", err)
}
if data, _ := os.ReadFile(to); string(data) != "competing" {
t.Fatalf("destination was overwritten: %q", data)
}
}
// TestRemoveLockFileRetriesSharingViolation holds the file open briefly (Go
// opens files without FILE_SHARE_DELETE on Windows, so a concurrent DeleteFile
// fails with ERROR_SHARING_VIOLATION while the handle is open); RemoveLockFile
// must ride its retry loop (15 x 5ms) past the window and succeed once the
// handle closes.
func TestRemoveLockFileRetriesSharingViolation(t *testing.T) {
path := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(path, []byte("token"), 0o600); err != nil {
t.Fatal(err)
}
f, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
go func() {
time.Sleep(10 * time.Millisecond)
_ = f.Close()
}()
if err := RemoveLockFile(path); err != nil {
t.Fatalf("RemoveLockFile should succeed once the contending handle closes: %v", err)
}
if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("expected lock file to be removed: %v", err)
}
}
// TestRemoveLockFileSharingViolationExhausts verifies the retry loop is
// bounded: with the contending handle held for the whole call, RemoveLockFile
// surfaces the sharing violation instead of retrying forever.
func TestRemoveLockFileSharingViolationExhausts(t *testing.T) {
path := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(path, []byte("token"), 0o600); err != nil {
t.Fatal(err)
}
f, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer f.Close()
if err := RemoveLockFile(path); !errors.Is(err, windows.ERROR_SHARING_VIOLATION) {
t.Fatalf("RemoveLockFile with a held handle = %v, want ERROR_SHARING_VIOLATION", err)
}
}
-82
View File
@@ -1,82 +0,0 @@
package lockutil
import (
"errors"
"os"
)
// restoreLockFile is swappable so tests can force the fail-closed path of
// ReclaimStaleLock, which requires both the fast restore and its no-replace
// fallback (with its own copy fallback) to fail; that cannot be provoked
// portably on a healthy filesystem.
var restoreLockFile = restoreLiveLock
// restoreLiveLock puts a lock that turned out to be live back at path after
// ReclaimStaleLock moved it aside to inspect it. It first tries a fast,
// replacing rename straight from reclaimed to path: a single syscall, which
// keeps the window during which path does not exist (and so is open to a
// fourth process's unrelated O_EXCL create landing on it) as short as
// possible. RestoreLockFile's no-replace restore (with its slower copy
// fallback on some failures) is a correctness-preserving fallback for when
// the fast path itself fails (e.g. a cross-device sidelined name): it is a
// much longer version of the identical race, but still detects rather than
// silently clobbers a competing lock, which the fast path's replacing rename
// cannot do. Neither path makes the race impossible, only unlikely; see
// ReclaimStaleLock's doc comment.
func restoreLiveLock(reclaimed, path string) error {
if err := os.Rename(reclaimed, path); err == nil {
return nil
}
return RestoreLockFile(reclaimed, path)
}
// ReclaimStaleLock atomically reclaims a suspected-stale lock file. It renames
// lockPath aside to "<lockPath>.stale.<suffix>" (only one racer can win the
// rename of a given file, so two racers can never both reclaim the same lock),
// then consults isLive on the moved file; if the lock turns out to be live (a
// holder reacquired it in the gap between the caller's stale check and the
// rename) it is restored rather than stolen. The suffix must be unique per
// acquirer attempt. Returns true only when a genuinely stale lock was removed,
// so the caller knows it is safe to retry its exclusive create immediately; on
// a lost race it returns false. A non-nil error means either the rename aside
// failed for a reason that is not contention (so retrying cannot help and the
// caller should fail fast instead of spinning to its deadline), or a live
// holder's lock could not be restored (both restoreLiveLock's fast path and
// its no-replace fallback failed), so lockPath may be missing; callers must
// fail closed instead of re-acquiring. The sidelined file is removed on every
// restore failure: once the restore has failed it has no protocol function
// (release only consults the lock path), so keeping it would only leak files.
//
// The live-restore path has an inherent, unclosed race: between the rename
// aside above and restoreLiveLock putting the lock back, lockPath does not
// exist, so an unrelated caller's O_EXCL create can legitimately succeed
// there. restoreLiveLock's fast path then silently overwrites that new
// claimant's lock file, which does not corrupt release (it is
// ownership-aware, so the new claimant's later release safely no-ops against
// content it no longer owns) but does mean the new claimant can still run
// its critical section concurrently with the original live holder. Making
// this race actually impossible would need an OS-level advisory lock (flock
// / LockFileEx) held for a holder's whole critical section, checked
// non-destructively instead of by moving the file; restoreLiveLock only
// shrinks the window to roughly one syscall, it does not close it.
func ReclaimStaleLock(lockPath, suffix string, isLive func(reclaimedPath string) bool) (bool, error) {
reclaimed := lockPath + ".stale." + suffix
if err := os.Rename(lockPath, reclaimed); err != nil {
if errors.Is(err, os.ErrNotExist) || isReclaimContended(err) {
return false, nil // another racer already moved/removed it, or it vanished
}
return false, err
}
if isLive(reclaimed) {
// Put the live lock back instead of stealing it, and let the caller wait.
if rerr := restoreLockFile(reclaimed, lockPath); rerr != nil {
_ = RemoveLockFile(reclaimed)
if !errors.Is(rerr, os.ErrExist) {
return false, rerr
}
}
return false, nil
}
_ = RemoveLockFile(reclaimed)
return true, nil
}
-31
View File
@@ -1,31 +0,0 @@
//go:build !windows
package lockutil
import (
"os"
"path/filepath"
"testing"
)
// TestReclaimStaleLockPropagatesRenameFailure pins the fail-fast contract of
// the rename aside: a failure that is not a lost race must reach the caller.
// Mapping it to the benign lost-race result would leave acquirers spinning
// until their deadline and reporting a timeout instead of the real cause. A
// directory squatting on the sidelined name makes os.Rename fail with EISDIR,
// which matches neither os.ErrNotExist nor a contention errno.
func TestReclaimStaleLockPropagatesRenameFailure(t *testing.T) {
lockPath := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(lockPath, []byte("stale-holder"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.Mkdir(lockPath+".stale.tok", 0o755); err != nil {
t.Fatal(err)
}
if ok, err := ReclaimStaleLock(lockPath, "tok", func(string) bool { return false }); err == nil || ok {
t.Fatalf("a hard rename failure must surface an error (ok=%v err=%v)", ok, err)
}
if _, err := os.Stat(lockPath); err != nil {
t.Fatalf("the lock file must be left in place on a failed rename: %v", err)
}
}
-112
View File
@@ -1,112 +0,0 @@
package lockutil
import (
"errors"
"os"
"path/filepath"
"testing"
)
func TestReclaimStaleLock(t *testing.T) {
lockPath := filepath.Join(t.TempDir(), "lock")
dead := func(string) bool { return false }
live := func(string) bool { return true }
// A lock the predicate reports dead is reclaimed and removed.
if err := os.WriteFile(lockPath, []byte("crashed-holder"), 0o600); err != nil {
t.Fatal(err)
}
if ok, err := ReclaimStaleLock(lockPath, "tok-a", dead); err != nil || !ok {
t.Fatalf("a dead lock should be reclaimed (ok=%v err=%v)", ok, err)
}
if _, err := os.Stat(lockPath); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("reclaimed dead lock should be gone, stat err=%v", err)
}
// A LIVE lock (a holder reacquired in the gap) must be RESTORED intact.
if err := os.WriteFile(lockPath, []byte("live-holder"), 0o600); err != nil {
t.Fatal(err)
}
if ok, err := ReclaimStaleLock(lockPath, "tok-b", live); err != nil || ok {
t.Fatalf("a live lock must not be reclaimed (ok=%v err=%v)", ok, err)
}
if data, err := os.ReadFile(lockPath); err != nil || string(data) != "live-holder" {
t.Fatalf("live lock must be left intact, got %q err %v", data, err)
}
// A missing lock reports no reclaim (nothing to steal).
_ = os.Remove(lockPath)
if ok, err := ReclaimStaleLock(lockPath, "tok-c", live); err != nil || ok {
t.Fatalf("a missing lock should not report a reclaim (ok=%v err=%v)", ok, err)
}
}
// restoreLiveLock's fast path is a single replacing rename, chosen to
// minimize (not eliminate) the window during which lockPath is absent and
// open to an unrelated O_EXCL create; see ReclaimStaleLock's doc comment for
// why that residual race exists. This test pins the resulting behavior: the
// fast path overwrites a competing file at path rather than detecting it,
// unlike RestoreLockFile's own no-replace contract.
func TestRestoreLiveLockFastPathOverwritesCompetingFile(t *testing.T) {
dir := t.TempDir()
reclaimed := filepath.Join(dir, "lock.stale.tok")
path := filepath.Join(dir, "lock")
if err := os.WriteFile(reclaimed, []byte("original-holder"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("new-claimant"), 0o600); err != nil {
t.Fatal(err)
}
if err := restoreLiveLock(reclaimed, path); err != nil {
t.Fatalf("restoreLiveLock failed: %v", err)
}
data, err := os.ReadFile(path)
if err != nil || string(data) != "original-holder" {
t.Fatalf("path = %q, err %v; want the original holder's content restored", data, err)
}
if _, err := os.Stat(reclaimed); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("expected reclaimed to be consumed by the rename: %v", err)
}
}
func TestReclaimStaleLockFailsClosedOnRestoreError(t *testing.T) {
// When both the no-replace restore and its copy fallback fail (only
// provokable via the seam; a healthy filesystem cannot produce it), the
// caller must receive an error so it fails closed instead of re-acquiring a
// missing lock path, and the sidelined file must not leak.
restoreLockFile = func(reclaimed, path string) error { return errors.New("restore failed") }
defer func() { restoreLockFile = restoreLiveLock }()
lockPath := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(lockPath, []byte("live-holder"), 0o600); err != nil {
t.Fatal(err)
}
ok, err := ReclaimStaleLock(lockPath, "tok", func(string) bool { return true })
if err == nil || ok {
t.Fatalf("a failed restore must surface an error (ok=%v err=%v)", ok, err)
}
if matches, _ := filepath.Glob(lockPath + ".stale.*"); len(matches) != 0 {
t.Fatalf("a failed restore must not leak sidelined files: %v", matches)
}
}
func TestReclaimStaleLockDropsSidelinedWhenNewHolderWins(t *testing.T) {
// An os.ErrExist restore failure means a new holder recreated the lock
// path; that is not an error for the caller, and the sidelined file is
// dropped rather than leaked.
restoreLockFile = func(reclaimed, path string) error { return os.ErrExist }
defer func() { restoreLockFile = restoreLiveLock }()
lockPath := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(lockPath, []byte("live-holder"), 0o600); err != nil {
t.Fatal(err)
}
ok, err := ReclaimStaleLock(lockPath, "tok", func(string) bool { return true })
if err != nil || ok {
t.Fatalf("losing to a new holder is not an error (ok=%v err=%v)", ok, err)
}
if matches, _ := filepath.Glob(lockPath + ".stale.*"); len(matches) != 0 {
t.Fatalf("the sidelined file must be dropped when a new holder wins: %v", matches)
}
}
-50
View File
@@ -1,50 +0,0 @@
//go:build windows
package lockutil
import (
"os"
"path/filepath"
"testing"
)
// TestReclaimStaleLockTreatsSharingViolationAsLostRace pins the errno
// classification the Windows acquire loops depend on: a rename aside that
// fails because the lock file is concurrently open (Go opens files without
// FILE_SHARE_DELETE, so the rename fails with ERROR_SHARING_VIOLATION or
// ERROR_ACCESS_DENIED) is contention, not a hard failure, and must report a
// benign lost race so the caller keeps waiting.
func TestReclaimStaleLockTreatsSharingViolationAsLostRace(t *testing.T) {
lockPath := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(lockPath, []byte("held"), 0o600); err != nil {
t.Fatal(err)
}
f, err := os.Open(lockPath)
if err != nil {
t.Fatal(err)
}
defer f.Close()
if ok, rerr := ReclaimStaleLock(lockPath, "tok", func(string) bool { return true }); rerr != nil || ok {
t.Fatalf("a sharing-violation rename must report a benign lost race (ok=%v err=%v)", ok, rerr)
}
if _, err := os.Stat(lockPath); err != nil {
t.Fatalf("the lock file must be left in place: %v", err)
}
}
// TestReclaimStaleLockPropagatesRenameFailure pins the fail-fast contract of
// the rename aside on Windows: a failure that is not contention must reach
// the caller instead of masquerading as a lost race. A quote is an invalid
// filename character, so the sidelined name fails with ERROR_INVALID_NAME.
func TestReclaimStaleLockPropagatesRenameFailure(t *testing.T) {
lockPath := filepath.Join(t.TempDir(), "lock")
if err := os.WriteFile(lockPath, []byte("stale-holder"), 0o600); err != nil {
t.Fatal(err)
}
if ok, err := ReclaimStaleLock(lockPath, `bad"name`, func(string) bool { return false }); err == nil || ok {
t.Fatalf("a hard rename failure must surface an error (ok=%v err=%v)", ok, err)
}
if _, err := os.Stat(lockPath); err != nil {
t.Fatalf("the lock file must be left in place on a failed rename: %v", err)
}
}
+4 -2
View File
@@ -176,9 +176,11 @@ func RegisterTools(ctx context.Context, registry *tools.Registry, cfg config.MCP
staged = append(staged, tool)
}
}
for _, tool := range staged {
registry.Register(tool)
registered := make([]tools.Tool, 0, len(staged))
for index := range staged {
registered = append(registered, staged[index])
}
registry.RegisterBatch(registered)
return runtime, nil
}
+56
View File
@@ -70,6 +70,62 @@ func TestRegisterToolsAddsPromptGatedMCPTools(t *testing.T) {
}
}
func TestRegisterToolsPublishesServerToolsInOneGeneration(t *testing.T) {
registry := tools.NewRegistry()
before := registry.Snapshot().Generation
client := &fakeToolClient{listed: []RemoteTool{
{Name: "lookup", Description: "Lookup documentation"},
{Name: "search", Description: "Search documentation"},
}}
runtime, err := RegisterTools(context.Background(), registry, config.MCPConfig{Servers: map[string]config.MCPServerConfig{
"docs": {Type: "stdio", Command: "docs-mcp"},
}}, RegisterOptions{ClientFactory: func(context.Context, Server) (ToolClient, error) {
return client, nil
}})
if err != nil {
t.Fatalf("RegisterTools() error = %v", err)
}
defer runtime.Close()
after := registry.Snapshot()
if after.Generation != before+1 {
t.Fatalf("registry generation = %d, want %d", after.Generation, before+1)
}
for _, name := range []string{"mcp_docs_lookup", "mcp_docs_search"} {
if _, ok := registry.Get(name); !ok {
t.Fatalf("expected %s to be published in the batch", name)
}
}
}
func TestRegisterToolsPublishesNoneWhenServerToolValidationFails(t *testing.T) {
registry := tools.NewRegistry()
before := registry.Snapshot().Generation
client := &fakeToolClient{listed: []RemoteTool{
{Name: "lookup", Description: "Lookup documentation"},
{Name: "", Description: "Invalid nameless tool"},
}}
runtime, err := RegisterTools(context.Background(), registry, config.MCPConfig{Servers: map[string]config.MCPServerConfig{
"docs": {Type: "stdio", Command: "docs-mcp"},
}}, RegisterOptions{ClientFactory: func(context.Context, Server) (ToolClient, error) {
return client, nil
}})
if err != nil {
t.Fatalf("RegisterTools() error = %v", err)
}
defer runtime.Close()
if got := registry.Snapshot().Generation; got != before {
t.Fatalf("registry generation = %d, want unchanged %d", got, before)
}
if _, ok := registry.Get("mcp_docs_lookup"); ok {
t.Fatal("valid prefix of rejected server batch was published")
}
if skipped := runtime.Skipped(); len(skipped) != 1 || skipped[0].Name != "docs" || skipped[0].Err == nil {
t.Fatalf("Skipped() = %#v, want one validation failure for docs", skipped)
}
}
func TestRegisterToolsMarksPersistentlyApprovedToolsAllow(t *testing.T) {
store, err := NewPermissionStore(StoreOptions{
FilePath: filepath.Join(t.TempDir(), "permissions.json"),
+7 -64
View File
@@ -5,24 +5,16 @@ import (
"fmt"
"os"
"path/filepath"
"sync/atomic"
"time"
"github.com/Gitlawb/zero/internal/lockutil"
)
const (
fileLockTimeout = 5 * time.Second
fileLockStaleAfter = 30 * time.Second
)
const fileLockTimeout = 5 * time.Second
var lockSeq atomic.Uint64
// acquireFileLock takes a cross-process exclusive lock by creating lockPath with
// O_EXCL. It retries with a short backoff until a timeout, breaking a lock whose
// file is older than fileLockStaleAfter (so a crashed holder cannot deadlock the
// store). Release is ownership-aware: it removes the lock only if it still holds
// our token, so a stale-broken holder cannot delete a newer holder's lock.
// acquireFileLock takes a cross-process advisory lock through a stable lock
// file. It retries with a short backoff until a timeout. Kernel lock lifetime
// provides crash recovery, so the path is never renamed or removed.
func acquireFileLock(lockPath string, now func() time.Time) (func(), error) {
if now == nil {
now = time.Now
@@ -30,64 +22,15 @@ func acquireFileLock(lockPath string, now func() time.Time) (func(), error) {
if err := os.MkdirAll(filepath.Dir(lockPath), 0o700); err != nil {
return nil, err
}
token := fmt.Sprintf("%d-%d-%d", os.Getpid(), now().UnixNano(), lockSeq.Add(1))
deadline := now().Add(fileLockTimeout)
for {
f, err := os.OpenFile(lockPath, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
lock, err := lockutil.TryAcquireFileLock(lockPath)
if err == nil {
// A partial/failed write would leave a lock file without our token, so
// the releaser below could never delete it — stranding the lock for
// other processes. Fail closed: remove the file and surface the error.
if _, werr := f.WriteString(token); werr != nil {
_ = f.Close()
_ = lockutil.RemoveLockFile(lockPath)
return nil, fmt.Errorf("oauth: write token lock: %w", werr)
}
if cerr := f.Close(); cerr != nil {
_ = lockutil.RemoveLockFile(lockPath)
return nil, fmt.Errorf("oauth: close token lock: %w", cerr)
}
var released bool
return func() {
if released {
return
}
released = true
if data, rerr := os.ReadFile(lockPath); rerr == nil && string(data) == token {
_ = lockutil.RemoveLockFile(lockPath)
}
}, nil
return func() { _ = lock.Release() }, nil
}
// On Windows a concurrent holder's os.Remove leaves the lock file in a
// "delete pending" state, so an O_EXCL create races it with
// ERROR_ACCESS_DENIED (os.ErrPermission) rather than ErrExist. Treat that
// as contention and retry, exactly like ErrExist — otherwise the lock
// spuriously fails under concurrency on Windows.
if !errors.Is(err, os.ErrExist) && !errors.Is(err, os.ErrPermission) {
if !errors.Is(err, lockutil.ErrLockHeld) {
return nil, fmt.Errorf("oauth: acquire token lock: %w", err)
}
// Reclaim a stale lock left by a crashed holder — atomically (H3). A blind
// Remove lets two racers both reclaim + recreate and so both hold the lock;
// reclaimStaleLock renames the file aside (only one rename wins) and restores
// it if it turns out fresh, so a live lock is never deleted out from under it.
if info, statErr := os.Stat(lockPath); statErr == nil && time.Since(info.ModTime()) > fileLockStaleAfter {
cleared, rerr := lockutil.ReclaimStaleLock(lockPath, token, func(reclaimedPath string) bool {
info, err := os.Stat(reclaimedPath)
return err == nil && time.Since(info.ModTime()) <= fileLockStaleAfter
})
if rerr != nil {
// Reclaim hit a hard failure: the rename aside failed outright, or a
// live holder's lock could not be put back (the lock path may be
// missing, so re-acquiring would break mutual exclusion). Fail closed
// instead of spinning to the deadline.
return nil, fmt.Errorf("oauth: reclaim stale token lock: %w", rerr)
}
if cleared {
continue
}
// Lost the reclaim race (or it was actually fresh) — fall through to the
// bounded wait rather than hot-spinning on a reclaim that never wins.
}
if now().After(deadline) {
return nil, fmt.Errorf("oauth: timed out acquiring token lock %s", filepath.Base(lockPath))
}
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"strings"
)
func ensurePrivateDir(path string) error {
func EnsurePrivateDir(path string) error {
abs, err := filepath.Abs(path)
if err != nil {
return err
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"golang.org/x/sys/unix"
)
func ensurePrivateDir(path string) error {
func EnsurePrivateDir(path string) error {
abs, err := filepath.Abs(path)
if err != nil {
return err
+2 -2
View File
@@ -18,7 +18,7 @@ func TestEnsurePrivateDirRejectsSymlink(t *testing.T) {
if err := os.Symlink(target, link); err != nil {
t.Fatal(err)
}
if err := ensurePrivateDir(link); err == nil {
if err := EnsurePrivateDir(link); err == nil {
t.Fatal("expected symlink runtime directory to be rejected")
}
}
@@ -33,7 +33,7 @@ func TestEnsurePrivateDirRejectsSymlinkedParent(t *testing.T) {
if err := os.Symlink(target, link); err != nil {
t.Fatal(err)
}
if err := ensurePrivateDir(filepath.Join(link, "peers")); err == nil {
if err := EnsurePrivateDir(filepath.Join(link, "peers")); err == nil {
t.Fatal("expected symlinked parent to be rejected")
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"golang.org/x/sys/windows"
)
func ensurePrivateDir(path string) (resultErr error) {
func EnsurePrivateDir(path string) (resultErr error) {
abs, err := filepath.Abs(path)
if err != nil {
return err
+5 -5
View File
@@ -54,7 +54,7 @@ func TestEnsurePrivateDirAppliesOwnerOnlyProtectedDACL(t *testing.T) {
if !windowsDirectoryDACLContains(t, path, worldSID) {
t.Fatal("test setup did not grant the broad Everyone ACE")
}
if err := ensurePrivateDir(path); err != nil {
if err := EnsurePrivateDir(path); err != nil {
t.Fatal(err)
}
descriptor, err := windows.GetNamedSecurityInfo(
@@ -149,7 +149,7 @@ func windowsDirectoryDACLContains(t *testing.T, path string, wanted *windows.SID
func TestSecurePrivateDirectoryRejectsOwnerMismatch(t *testing.T) {
path := filepath.Join(t.TempDir(), "private")
if err := ensurePrivateDir(path); err != nil {
if err := EnsurePrivateDir(path); err != nil {
t.Fatal(err)
}
handle, err := openWindowsDirectory(path, windows.FILE_LIST_DIRECTORY|windows.FILE_TRAVERSE|windows.SYNCHRONIZE|windows.READ_CONTROL|windows.WRITE_DAC)
@@ -172,7 +172,7 @@ func TestSecurePrivateDirectoryRejectsOwnerMismatch(t *testing.T) {
func TestSecurePrivateDirectoryReportsDACLWriteFailure(t *testing.T) {
path := filepath.Join(t.TempDir(), "private")
if err := ensurePrivateDir(path); err != nil {
if err := EnsurePrivateDir(path); err != nil {
t.Fatal(err)
}
handle, err := openWindowsDirectory(path, windows.FILE_LIST_DIRECTORY|windows.FILE_TRAVERSE|windows.SYNCHRONIZE|windows.READ_CONTROL)
@@ -206,7 +206,7 @@ func TestEnsurePrivateDirRejectsWindowsReparseParent(t *testing.T) {
if err := os.Symlink(target, link); err != nil {
t.Skipf("cannot create Windows directory symlink: %v", err)
}
if err := ensurePrivateDir(filepath.Join(link, "peers")); err == nil {
if err := EnsurePrivateDir(filepath.Join(link, "peers")); err == nil {
t.Fatal("expected reparse-point parent to be rejected")
}
}
@@ -217,7 +217,7 @@ func TestEnsurePrivateDirRejectsWindowsFileComponent(t *testing.T) {
if err := os.WriteFile(file, []byte("not a directory"), 0o600); err != nil {
t.Fatal(err)
}
if err := ensurePrivateDir(filepath.Join(file, "peers")); err == nil {
if err := EnsurePrivateDir(filepath.Join(file, "peers")); err == nil {
t.Fatal("expected file path component to be rejected")
}
}
+3 -3
View File
@@ -158,7 +158,7 @@ func New(options Options) (*Service, error) {
}
// canonicalRuntimePath normalizes aliases in the existing prefix. It does not
// establish a security boundary; ensurePrivateDir validates that boundary when
// establish a security boundary; EnsurePrivateDir validates that boundary when
// the service starts.
func canonicalRuntimePath(path string) (string, error) {
missing := make([]string, 0, 4)
@@ -219,10 +219,10 @@ func (service *Service) Start(handler Handler) error {
if service.closed {
return errors.New("peer messaging: service is closed")
}
if err := ensurePrivateDir(service.root); err != nil {
if err := EnsurePrivateDir(service.root); err != nil {
return fmt.Errorf("peer messaging: create runtime directory: %w", err)
}
if err := ensurePrivateDir(service.registryDir()); err != nil {
if err := EnsurePrivateDir(service.registryDir()); err != nil {
return fmt.Errorf("peer messaging: create registry: %w", err)
}
endpoint, err := service.transport.Endpoint(service.root, service.nonce, service.pid)
+1 -1
View File
@@ -44,7 +44,7 @@ func canonicalPrivateDir(path string) (string, error) {
if err != nil {
return "", err
}
if err := ensurePrivateDir(canonical); err != nil {
if err := EnsurePrivateDir(canonical); err != nil {
return "", err
}
return canonical, nil
+22 -2
View File
@@ -191,6 +191,7 @@ type TurnBenchResult struct {
type TurnBenchTotals struct {
InputTokens int64 `json:"inputTokens"`
CachedInputTokens int64 `json:"cachedInputTokens"`
CacheWriteTokens int64 `json:"cacheWriteTokens"`
OutputTokens int64 `json:"outputTokens"`
ModelRequests int64 `json:"modelRequests"`
ToolCalls int64 `json:"toolCalls"`
@@ -479,6 +480,7 @@ func topLatencySources(perSpan map[string]SpanStats, top int) []LatencySource {
func aggregateTotals(totals *TurnBenchTotals, tr *trace.TurnTrace) {
totals.InputTokens += tr.Counter(trace.CounterInputTokens)
totals.CachedInputTokens += tr.Counter(trace.CounterCachedInputTokens)
totals.CacheWriteTokens += tr.Counter(trace.CounterCacheWriteTokens)
totals.OutputTokens += tr.Counter(trace.CounterOutputTokens)
totals.ModelRequests += tr.Counter(trace.CounterModelRequests)
totals.ToolCalls += tr.Counter(trace.CounterToolCalls)
@@ -531,8 +533,9 @@ func FormatTurnBenchSummary(result TurnBenchResult) string {
lines = append(lines, fmt.Sprintf(" %-18s %10s %5.1f%%", src.Span, FormatMetric(src.TotalMs, "ms"), src.Share*100))
}
}
lines = append(lines, fmt.Sprintf("totals: in=%d (cached %d) out=%d | requests=%d tools=%d retries=%d reconnects=%d compactions=%d",
result.Totals.InputTokens, result.Totals.CachedInputTokens, result.Totals.OutputTokens,
lines = append(lines, fmt.Sprintf("totals: in=%d (cache-read %d, cache-write %d, uncached %d) out=%d | requests=%d tools=%d retries=%d reconnects=%d compactions=%d",
result.Totals.InputTokens, result.Totals.CachedInputTokens, result.Totals.CacheWriteTokens,
uncachedInputTokens(result.Totals), result.Totals.OutputTokens,
result.Totals.ModelRequests, result.Totals.ToolCalls, result.Totals.Retries,
result.Totals.Reconnects, result.Totals.Compactions))
for _, class := range sortedClasses(result.PerClass) {
@@ -545,6 +548,23 @@ func FormatTurnBenchSummary(result TurnBenchResult) string {
return strings.Join(lines, "\n")
}
func uncachedInputTokens(totals TurnBenchTotals) int64 {
remaining := totals.InputTokens
if remaining <= 0 {
return 0
}
for _, cached := range []int64{totals.CachedInputTokens, totals.CacheWriteTokens} {
if cached <= 0 {
continue
}
if cached >= remaining {
return 0
}
remaining -= cached
}
return remaining
}
// classTier returns the oracle tier label a class was classified into, so the
// per-class roll-up states explicitly which oracle (if any) its "passed" count
// is measured against.
+33
View File
@@ -74,6 +74,7 @@ func TestRunTurnBenchAggregation(t *testing.T) {
"t3": cannedTrace(150, 20, 1500),
"t4": cannedTrace(300, 10, 1000),
}
canned["t1"].Counters = append(canned["t1"].Counters, trace.Counter{Name: trace.CounterCacheWriteTokens, Value: 125})
cfg := TurnBenchConfig{
Model: "fake-model",
Iterations: 1,
@@ -155,6 +156,9 @@ func TestRunTurnBenchAggregation(t *testing.T) {
if result.Totals.InputTokens != 5500 {
t.Fatalf("inputTokens = %d, want 5500", result.Totals.InputTokens)
}
if result.Totals.CacheWriteTokens != 125 {
t.Fatalf("cacheWriteTokens = %d, want 125", result.Totals.CacheWriteTokens)
}
if result.Totals.OutputTokens != 2750 {
t.Fatalf("outputTokens = %d, want 2750", result.Totals.OutputTokens)
}
@@ -334,6 +338,35 @@ func TestFormatTurnBenchSummaryNamesTopSources(t *testing.T) {
}
}
func TestUncachedInputTokensClampsInconsistentTotals(t *testing.T) {
tests := []struct {
name string
totals TurnBenchTotals
want int64
}{
{name: "normal", totals: TurnBenchTotals{InputTokens: 100, CachedInputTokens: 30, CacheWriteTokens: 20}, want: 50},
{name: "cached exceeds input", totals: TurnBenchTotals{InputTokens: 100, CachedInputTokens: 120}, want: 0},
{name: "write exceeds remainder", totals: TurnBenchTotals{InputTokens: 100, CachedInputTokens: 20, CacheWriteTokens: 90}, want: 0},
{name: "overflowing inconsistent counters", totals: TurnBenchTotals{CachedInputTokens: 1<<63 - 1, CacheWriteTokens: 1<<63 - 1}, want: 0},
{name: "overflowing counters with input", totals: TurnBenchTotals{InputTokens: 100, CachedInputTokens: 1<<63 - 1, CacheWriteTokens: 1<<63 - 1}, want: 0},
{name: "negative cache counters ignored", totals: TurnBenchTotals{InputTokens: 100, CachedInputTokens: -20, CacheWriteTokens: -10}, want: 100},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if got := uncachedInputTokens(test.totals); got != test.want {
t.Fatalf("uncachedInputTokens(%+v) = %d, want %d", test.totals, got, test.want)
}
})
}
summary := FormatTurnBenchSummary(TurnBenchResult{Totals: TurnBenchTotals{
CachedInputTokens: 1<<63 - 1,
CacheWriteTokens: 1<<63 - 1,
}})
if !strings.Contains(summary, "uncached 0") {
t.Fatalf("summary did not clamp inconsistent cache totals:\n%s", summary)
}
}
func TestLoadBaselineManifest(t *testing.T) {
path := filepath.Join("manifests", "baseline.json")
set, err := LoadTaskSet(path)
+108
View File
@@ -0,0 +1,108 @@
package providers
import (
"context"
"os"
"strings"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
// Compaction summarization model selection.
//
// Compaction summaries used to run on the session's main model — near a 200k
// window one summarization is a ~100k-token input call at full price, the
// single most expensive recurring event in a long run. A cheap model handles
// the "dense factual summary" task fine, so summarization is routed to one
// whenever the target is knowable.
//
// Resolution order: ZERO_COMPACTION_MODEL env > preferences.compactionModel
// config > a curated per-kind default (OFFICIAL endpoints only) > "" (use the
// main model). The literal value "main" at any level forces the main model.
// A wrong choice is never fatal: the agent falls back to the main provider on
// the first summarizer failure for the rest of the run.
// CompactionModelEnv overrides the compaction summarization model.
const CompactionModelEnv = "ZERO_COMPACTION_MODEL"
// compactionMainSentinel disables the dedicated summarizer explicitly.
const compactionMainSentinel = "main"
// Curated cheap summarizer defaults, applied only for provider kinds whose
// model catalog is guaranteed (official endpoints). Custom / *-compatible
// endpoints get NO default — their catalogs are unknowable, and a guaranteed
// failed call per run is worse than main-model prices.
const (
defaultAnthropicCompactionModel = "claude-haiku-4-5-20251001"
defaultGoogleCompactionModel = "gemini-2.5-flash-lite"
defaultOpenAICompactionModel = "gpt-4.1-mini"
)
// CompactionModelID returns the model that compaction summarization calls
// should use for the given profile, or "" to use the session's main model.
// preference is the resolved config value (preferences.compactionModel).
func CompactionModelID(profile config.ProviderProfile, preference string) string {
value := strings.TrimSpace(os.Getenv(CompactionModelEnv))
if value == "" {
value = strings.TrimSpace(preference)
}
if value != "" {
if strings.EqualFold(value, compactionMainSentinel) {
return ""
}
return value
}
return defaultCompactionModel(profile)
}
func defaultCompactionModel(profile config.ProviderProfile) string {
metadata, err := ResolveRuntimeMetadata(profile, Options{})
if err != nil {
return ""
}
var candidate string
switch metadata.ProviderKind {
case config.ProviderKindAnthropic:
candidate = defaultAnthropicCompactionModel
case config.ProviderKindGoogle:
candidate = defaultGoogleCompactionModel
case config.ProviderKindOpenAI:
candidate = defaultOpenAICompactionModel
default:
return ""
}
// Session already runs on the cheap model: a dedicated summarizer buys
// nothing and would only add a second provider instance.
if strings.EqualFold(strings.TrimSpace(metadata.APIModel), candidate) {
return ""
}
return candidate
}
// CompactionSummarizerFactory returns the lazy constructor the agent loop
// calls for its dedicated compaction summarizer (agent.Options.Summarizer).
// The compactor passes the main model in force at that moment — on the first
// paid compaction and again after a mid-run model switch — so the choice
// tracks escalations: a run that starts on the cheap model gets no dedicated
// summarizer, and gains one once it escalates to a pricier main model.
// Returning (nil, nil) means "summarize on the main provider". The factory
// itself is nil only when no provider builder exists. Shared by the exec and
// TUI front ends so the selection rules cannot drift between them.
func CompactionSummarizerFactory(base config.ProviderProfile, preference string, newProvider func(config.ProviderProfile) (zeroruntime.Provider, error)) func(context.Context, string) (zeroruntime.Provider, error) {
if newProvider == nil {
return nil
}
return func(_ context.Context, mainModelID string) (zeroruntime.Provider, error) {
profile := base
if strings.TrimSpace(mainModelID) != "" {
profile.Model = mainModelID
}
modelID := CompactionModelID(profile, preference)
if modelID == "" {
return nil, nil
}
profile.Model = modelID
return newProvider(profile)
}
}
+113
View File
@@ -0,0 +1,113 @@
package providers
import (
"context"
"errors"
"testing"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
func TestCompactionModelIDResolutionOrder(t *testing.T) {
anthropic := config.ProviderProfile{
Name: "anthropic",
ProviderKind: config.ProviderKindAnthropic,
Model: "claude-sonnet-4.5",
}
t.Setenv(CompactionModelEnv, "my-env-model")
if got := CompactionModelID(anthropic, "my-config-model"); got != "my-env-model" {
t.Fatalf("env must win, got %q", got)
}
t.Setenv(CompactionModelEnv, "main")
if got := CompactionModelID(anthropic, "my-config-model"); got != "" {
t.Fatalf("env 'main' must force the main model, got %q", got)
}
t.Setenv(CompactionModelEnv, "")
if got := CompactionModelID(anthropic, "my-config-model"); got != "my-config-model" {
t.Fatalf("config preference must apply when env unset, got %q", got)
}
if got := CompactionModelID(anthropic, "main"); got != "" {
t.Fatalf("config 'main' must force the main model, got %q", got)
}
}
func TestCompactionModelIDCuratedDefaults(t *testing.T) {
t.Setenv(CompactionModelEnv, "")
anthropic := config.ProviderProfile{Name: "anthropic", ProviderKind: config.ProviderKindAnthropic, Model: "claude-sonnet-4.5"}
if got := CompactionModelID(anthropic, ""); got != defaultAnthropicCompactionModel {
t.Fatalf("anthropic default = %q, want %q", got, defaultAnthropicCompactionModel)
}
google := config.ProviderProfile{Name: "google", ProviderKind: config.ProviderKindGoogle, Model: "gemini-2.5-pro"}
if got := CompactionModelID(google, ""); got != defaultGoogleCompactionModel {
t.Fatalf("google default = %q, want %q", got, defaultGoogleCompactionModel)
}
openai := config.ProviderProfile{Name: "openai", ProviderKind: config.ProviderKindOpenAI, Model: "gpt-4.1"}
if got := CompactionModelID(openai, ""); got != defaultOpenAICompactionModel {
t.Fatalf("openai default = %q, want %q", got, defaultOpenAICompactionModel)
}
// Custom/compatible endpoints: catalog unknowable, no default.
compatible := config.ProviderProfile{
Name: "opengateway",
ProviderKind: config.ProviderKindOpenAICompatible,
BaseURL: "https://opengateway.gitlawb.com/v1",
Model: "some-model",
}
if got := CompactionModelID(compatible, ""); got != "" {
t.Fatalf("openai-compatible must have no default, got %q", got)
}
}
func TestCompactionModelIDSkipsWhenMainIsAlreadyCheap(t *testing.T) {
t.Setenv(CompactionModelEnv, "")
haiku := config.ProviderProfile{
Name: "anthropic",
ProviderKind: config.ProviderKindAnthropic,
Model: defaultAnthropicCompactionModel,
}
if got := CompactionModelID(haiku, ""); got != "" {
t.Fatalf("already-cheap session must not get a dedicated summarizer, got %q", got)
}
}
func TestCompactionSummarizerFactoryTracksMainModel(t *testing.T) {
t.Setenv(CompactionModelEnv, "")
base := config.ProviderProfile{Name: "anthropic", ProviderKind: config.ProviderKindAnthropic, Model: defaultAnthropicCompactionModel}
var built []string
newProvider := func(profile config.ProviderProfile) (zeroruntime.Provider, error) {
built = append(built, profile.Model)
return nil, errors.New("not needed")
}
factory := CompactionSummarizerFactory(base, "", newProvider)
if factory == nil {
t.Fatal("factory must exist when a provider builder exists")
}
// Main model is the cheap default: no dedicated summarizer, nothing built.
if provider, err := factory(context.Background(), defaultAnthropicCompactionModel); provider != nil || err != nil || len(built) != 0 {
t.Fatalf("cheap main model must yield (nil, nil) without building, got %v %v built=%v", provider, err, built)
}
// After escalating to an expensive model the cheap default is built.
if _, err := factory(context.Background(), "claude-sonnet-4-5"); err == nil || len(built) != 1 || built[0] != defaultAnthropicCompactionModel {
t.Fatalf("escalated main model must build the cheap summarizer, built=%v err=%v", built, err)
}
// An empty main model falls back to the base profile's model — the cheap
// default here, so again nothing is built.
built = nil
if provider, err := factory(context.Background(), ""); provider != nil || err != nil || len(built) != 0 {
t.Fatalf("empty main model must use the base profile, got %v %v built=%v", provider, err, built)
}
// The explicit "main" preference disables the dedicated summarizer.
if provider, err := CompactionSummarizerFactory(base, "main", newProvider)(context.Background(), "claude-sonnet-4-5"); provider != nil || err != nil {
t.Fatalf("preference main must disable the summarizer, got %v %v", provider, err)
}
if CompactionSummarizerFactory(base, "", nil) != nil {
t.Fatal("no provider builder means no factory")
}
}
+238 -60
View File
@@ -62,6 +62,7 @@ const (
responsesEventReasoningDelta = "response.reasoning_summary_text.delta"
responsesEventOutputTextDone = "response.output_text.done"
responsesEventFunctionArgsDelta = "response.function_call_arguments.delta"
responsesEventCustomInputDelta = "response.custom_tool_call_input.delta"
responsesEventContentPartDone = "response.content_part.done"
responsesEventOutputItemDone = "response.output_item.done"
responsesEventCompleted = "response.completed"
@@ -70,17 +71,37 @@ const (
responsesEventIncomplete = "response.incomplete"
)
const applyPatchLarkGrammar = `start: begin_patch hunk+ end_patch
begin_patch: "*** Begin Patch" LF
end_patch: "*** End Patch" LF?
hunk: add_hunk | delete_hunk | update_hunk
add_hunk: "*** Add File: " filename LF add_line+
delete_hunk: "*** Delete File: " filename LF
update_hunk: "*** Update File: " filename LF change_move? change?
filename: /(.+)/
add_line: "+" /(.*)/ LF -> line
change_move: "*** Move to: " filename LF
change: (change_context | change_line)+ eof_line?
change_context: ("@@" | "@@ " /(.+)/) LF
change_line: ("+" | "-" | " ") /(.*)/ LF
eof_line: "*** End of File" LF
%import common.LF`
// responsesRequest is the wire shape POSTed to {baseURL}/responses.
type responsesRequest struct {
Model string `json:"model"`
Instructions string `json:"instructions"`
Input []inputItem `json:"input"`
Stream bool `json:"stream"`
Store bool `json:"store"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
Tools []responsesTool `json:"tools,omitempty"`
Reasoning *responsesReasoning `json:"reasoning,omitempty"`
ServiceTier string `json:"service_tier,omitempty"`
Type string `json:"type,omitempty"`
Model string `json:"model"`
Instructions string `json:"instructions"`
PreviousResponseID string `json:"previous_response_id,omitempty"`
Input []inputItem `json:"input"`
Stream bool `json:"stream"`
Store bool `json:"store"`
ParallelToolCalls bool `json:"parallel_tool_calls"`
MaxOutputTokens int `json:"max_output_tokens,omitempty"`
Tools []responsesTool `json:"tools,omitempty"`
Reasoning *responsesReasoning `json:"reasoning,omitempty"`
ServiceTier string `json:"service_tier,omitempty"`
PromptCacheKey string `json:"prompt_cache_key,omitempty"`
}
// responsesReasoning carries the reasoning controls for the Responses API. The
@@ -105,6 +126,7 @@ type inputItem struct {
CallID string `json:"call_id,omitempty"`
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
Input string `json:"input,omitempty"`
// function_call_output fields
Output any `json:"output,omitempty"`
}
@@ -118,10 +140,11 @@ type contentItem struct {
}
type responsesTool struct {
Type string `json:"type"`
Name string `json:"name"`
Description string `json:"description,omitempty"`
Parameters map[string]any `json:"parameters,omitempty"`
Type string `json:"type"`
Name string `json:"name"`
Description string `json:"description,omitempty"`
Parameters map[string]any `json:"parameters,omitempty"`
Format *zeroruntime.ToolDefinitionFormat `json:"format,omitempty"`
}
// responsesEvent is the decoded form of one Responses SSE data payload.
@@ -132,6 +155,7 @@ type responsesEvent struct {
Delta string `json:"delta,omitempty"`
// item payloads (response.output_item.added / done)
ItemID string `json:"item_id,omitempty"`
CallID string `json:"call_id,omitempty"`
// OutputIndex is a *int so a real 0 (the first output) is distinguishable from
// "absent" — a plain int defaulting to 0 dropped a no-item_id call's args (M1).
OutputIndex *int `json:"output_index,omitempty"`
@@ -150,6 +174,7 @@ type itemPayload struct {
CallID string `json:"call_id,omitempty"`
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
Input string `json:"input,omitempty"`
Status string `json:"status,omitempty"`
}
@@ -173,7 +198,8 @@ type outputDetails struct {
}
type inputDetails struct {
CachedTokens int `json:"cached_tokens"`
CachedTokens int `json:"cached_tokens"`
CacheWriteTokens int `json:"cache_write_tokens"`
}
type errorPayload struct {
@@ -187,8 +213,10 @@ type errorPayload struct {
// response.output_item.done.
type toolCallBuilder struct {
ID string
CallID string
Name string
Arguments strings.Builder
Freeform bool
started bool
ended bool
}
@@ -197,15 +225,57 @@ type toolCallBuilder struct {
// event has been emitted. It is the Responses-API equivalent of the
// chat-completions toolState in provider.go.
type responsesState struct {
toolCalls map[string]*toolCallBuilder
usage *usagePayload
done bool
toolCalls map[string]*toolCallBuilder
toolOrder []string
text strings.Builder
responseID string
usage *usagePayload
emitted bool
done bool
}
func newResponsesState() *responsesState {
return &responsesState{toolCalls: map[string]*toolCallBuilder{}}
}
func (state *responsesState) outputInputItems() []inputItem {
items := make([]inputItem, 0, len(state.toolOrder)+1)
if state.text.Len() > 0 {
items = append(items, inputItem{
Type: "message",
Role: "assistant",
Content: []contentItem{{Type: "output_text", Text: state.text.String()}},
})
}
for _, key := range state.toolOrder {
builder := state.toolCalls[key]
if builder == nil || builder.ID == "" || builder.Name == "" {
continue
}
callID := builder.CallID
if callID == "" {
callID = builder.ID
}
itemType := "function_call"
if builder.Freeform {
itemType = "custom_tool_call"
}
item := inputItem{
Type: itemType,
ID: builder.ID,
CallID: callID,
Name: builder.Name,
}
if builder.Freeform {
item.Input = builder.Arguments.String()
} else {
item.Arguments = builder.Arguments.String()
}
items = append(items, item)
}
return items
}
// buildResponsesRequest converts a runtime CompletionRequest into the
// Responses API wire format. System messages fold into a top-level
// `instructions` field; everything else (user/assistant turns, tool
@@ -215,11 +285,14 @@ func (p *CodexProvider) buildResponsesRequest(request zeroruntime.CompletionRequ
return nil, errors.New("codex provider: model is required")
}
req := &responsesRequest{
Model: p.inner.model,
Stream: true,
MaxOutputTokens: p.inner.maxTokens,
Model: p.inner.model,
Stream: true,
ParallelToolCalls: true,
MaxOutputTokens: p.inner.maxTokens,
PromptCacheKey: request.PromptCacheKey,
}
instructions := []string{}
customCallIDs := map[string]string{}
for _, msg := range request.Messages {
switch msg.Role {
case zeroruntime.MessageRoleSystem:
@@ -234,13 +307,31 @@ func (p *CodexProvider) buildResponsesRequest(request zeroruntime.CompletionRequ
return nil, err
}
req.Input = append(req.Input, items...)
for _, call := range msg.ToolCalls {
if call.Freeform && call.ID != "" {
callID := call.ProviderCallID
if callID == "" {
callID = call.ID
}
customCallIDs[call.ID] = callID
}
}
case zeroruntime.MessageRoleTool:
if msg.ToolCallID == "" {
return nil, fmt.Errorf("codex provider: tool message missing tool call id")
}
outputType := "function_call_output"
callID := msg.ToolCallProviderID
if callID == "" {
callID = msg.ToolCallID
}
if customID, custom := customCallIDs[msg.ToolCallID]; custom {
outputType = "custom_tool_call_output"
callID = customID
}
req.Input = append(req.Input, inputItem{
Type: "function_call_output",
CallID: msg.ToolCallID,
Type: outputType,
CallID: callID,
Output: msg.Content,
})
default:
@@ -249,12 +340,25 @@ func (p *CodexProvider) buildResponsesRequest(request zeroruntime.CompletionRequ
}
req.Instructions = strings.Join(instructions, "\n\n")
for _, tool := range request.Tools {
req.Tools = append(req.Tools, responsesTool{
Type: "function",
Name: tool.Name,
Description: tool.Description,
Parameters: tool.Parameters,
})
definition := tool
if definition.Name == "apply_patch" && definition.Type == zeroruntime.ToolDefinitionFreeform {
definition.Description = "The apply_patch tool edits files from a raw structured patch. Use absolute file paths in patch headers to target a granted extra write root. Do not wrap the patch in JSON."
definition.Format = &zeroruntime.ToolDefinitionFormat{Type: "grammar", Syntax: "lark", Definition: applyPatchLarkGrammar}
}
wireTool := responsesTool{
Type: string(definition.Type),
Name: definition.Name,
Description: definition.Description,
Parameters: definition.Parameters,
Format: definition.Format,
}
if wireTool.Type == "" {
wireTool.Type = string(zeroruntime.ToolDefinitionFunction)
}
if wireTool.Type == string(zeroruntime.ToolDefinitionFreeform) {
wireTool.Parameters = nil
}
req.Tools = append(req.Tools, wireTool)
}
// Codex / o-series reasoning models take the effort tier nested under
// `reasoning` on the Responses API (the chat-completions `reasoning_effort`
@@ -312,13 +416,26 @@ func (p *CodexProvider) assistantInputItems(msg zeroruntime.Message) ([]inputIte
if tc.Name == "" {
return nil, errors.New("codex provider: assistant tool call missing name")
}
items = append(items, inputItem{
Type: "function_call",
ID: tc.ID,
CallID: tc.ID,
Name: tc.Name,
Arguments: tc.Arguments,
})
callID := tc.ProviderCallID
if callID == "" {
callID = tc.ID
}
itemType := "function_call"
if tc.Freeform {
itemType = "custom_tool_call"
}
item := inputItem{
Type: itemType,
ID: tc.ID,
CallID: callID,
Name: tc.Name,
}
if tc.Freeform {
item.Input = tc.Arguments
} else {
item.Arguments = tc.Arguments
}
items = append(items, item)
}
return items, nil
}
@@ -479,6 +596,18 @@ func (p *CodexProvider) emitResponsesEvent(
state.done = true
return false
}
return p.emitParsedResponsesEvent(ctx, &event, state, events)
}
// emitParsedResponsesEvent converts one already-decoded Responses event into
// zero or more runtime events. WebSocket sessions use this path to avoid
// decoding each frame twice; the SSE path retains emitResponsesEvent above.
func (p *CodexProvider) emitParsedResponsesEvent(
ctx context.Context,
event *responsesEvent,
state *responsesState,
events chan<- zeroruntime.StreamEvent,
) bool {
if event.Type == "" {
// A payload that parses as JSON but carries no `type` field is a
// malformed Responses event — the type is the discriminator that
@@ -500,12 +629,17 @@ func (p *CodexProvider) emitResponsesEvent(
// the runtime. response.incomplete is folded into a length finish at
// scan-end (state.done stays false so the wrapper emits the
// StreamEventDone with FinishReasonLength).
if event.Response != nil && event.Response.ID != "" {
state.responseID = event.Response.ID
}
return true
case responsesEventOutputItemAdded:
p.handleOutputItemAdded(ctx, &event, state, events)
p.handleOutputItemAdded(ctx, event, state, events)
return true
case responsesEventOutputTextDelta:
if event.Delta != "" {
state.emitted = true
state.text.WriteString(event.Delta)
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventText,
Content: event.Delta,
@@ -517,6 +651,7 @@ func (p *CodexProvider) emitResponsesEvent(
// phase shows progress (and keeps the activity clock fresh) instead of
// looking like a hang. Requested via reasoning.summary="auto".
if event.Delta != "" {
state.emitted = true
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventReasoning,
Content: event.Delta,
@@ -524,13 +659,16 @@ func (p *CodexProvider) emitResponsesEvent(
}
return true
case responsesEventFunctionArgsDelta:
p.handleFunctionArgsDelta(ctx, &event, state, events)
p.handleToolInputDelta(ctx, event, state, events, false)
return true
case responsesEventCustomInputDelta:
p.handleToolInputDelta(ctx, event, state, events, true)
return true
case responsesEventOutputItemDone:
p.handleOutputItemDone(ctx, &event, state, events)
p.handleOutputItemDone(ctx, event, state, events)
return true
case responsesEventCompleted, responsesEventFailed:
return p.handleTerminalResponse(ctx, &event, state, events)
return p.handleTerminalResponse(ctx, event, state, events)
case responsesEventError:
message := event.Message
if event.Code != "" {
@@ -570,22 +708,38 @@ func (p *CodexProvider) handleOutputItemAdded(
return
}
switch event.Item.Type {
case "function_call":
case "function_call", "custom_tool_call":
key := p.toolCallKey(event)
if key == "" {
return
}
builder := &toolCallBuilder{
ID: key,
Name: event.Item.Name,
builder, exists := state.toolCalls[key]
if !exists {
builder = &toolCallBuilder{ID: key, Freeform: event.Item.Type == "custom_tool_call"}
state.toolCalls[key] = builder
state.toolOrder = append(state.toolOrder, key)
}
if event.Item.CallID != "" {
builder.CallID = event.Item.CallID
}
if event.Item.Name != "" {
builder.Name = event.Item.Name
}
builder.Freeform = builder.Freeform || event.Item.Type == "custom_tool_call"
if builder.Freeform && event.Item.Input != "" && builder.Arguments.Len() == 0 {
builder.Arguments.WriteString(event.Item.Input)
}
state.emitted = true
if !builder.started {
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventToolCallStart,
ToolCallID: key,
ToolCallProviderID: builder.CallID,
ToolName: builder.Name,
ToolCallFreeform: builder.Freeform,
})
builder.started = true
}
state.toolCalls[key] = builder
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventToolCallStart,
ToolCallID: key,
ToolName: event.Item.Name,
})
builder.started = true
}
}
@@ -593,11 +747,12 @@ func (p *CodexProvider) handleOutputItemAdded(
// to the in-flight builder and emits StreamEventToolCallDelta. The
// Codex backend uses either `item_id` or `output_index` to attribute
// the delta; we honor both.
func (p *CodexProvider) handleFunctionArgsDelta(
func (p *CodexProvider) handleToolInputDelta(
ctx context.Context,
event *responsesEvent,
state *responsesState,
events chan<- zeroruntime.StreamEvent,
freeform bool,
) {
key := p.toolCallKey(event)
if key == "" {
@@ -610,10 +765,16 @@ func (p *CodexProvider) handleFunctionArgsDelta(
// an intermediate proxy). Treat the delta as the start of a new
// tool call with no known name yet — the eventual
// response.output_item.done carries the final name.
builder = &toolCallBuilder{ID: key}
builder = &toolCallBuilder{ID: key, Freeform: freeform}
state.toolCalls[key] = builder
state.toolOrder = append(state.toolOrder, key)
}
builder.Freeform = builder.Freeform || freeform
if event.CallID != "" && builder.CallID == "" {
builder.CallID = event.CallID
}
builder.Arguments.WriteString(event.Delta)
state.emitted = true
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventToolCallDelta,
ToolCallID: key,
@@ -632,7 +793,7 @@ func (p *CodexProvider) handleOutputItemDone(
state *responsesState,
events chan<- zeroruntime.StreamEvent,
) {
if event.Item == nil || event.Item.Type != "function_call" {
if event.Item == nil || (event.Item.Type != "function_call" && event.Item.Type != "custom_tool_call") {
return
}
key := p.toolCallKey(event)
@@ -641,20 +802,31 @@ func (p *CodexProvider) handleOutputItemDone(
}
builder, ok := state.toolCalls[key]
if !ok {
builder = &toolCallBuilder{ID: key}
builder = &toolCallBuilder{ID: key, Freeform: event.Item.Type == "custom_tool_call"}
state.toolCalls[key] = builder
state.toolOrder = append(state.toolOrder, key)
}
if event.Item.Name != "" {
builder.Name = event.Item.Name
}
if event.Item.Arguments != "" && builder.Arguments.Len() == 0 {
builder.Arguments.WriteString(event.Item.Arguments)
if event.Item.CallID != "" {
builder.CallID = event.Item.CallID
}
builder.Freeform = builder.Freeform || event.Item.Type == "custom_tool_call"
finalInput := event.Item.Arguments
if builder.Freeform {
finalInput = event.Item.Input
}
if finalInput != "" && builder.Arguments.Len() == 0 {
builder.Arguments.WriteString(finalInput)
}
if !builder.started {
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventToolCallStart,
ToolCallID: key,
ToolName: builder.Name,
Type: zeroruntime.StreamEventToolCallStart,
ToolCallID: key,
ToolCallProviderID: builder.CallID,
ToolName: builder.Name,
ToolCallFreeform: builder.Freeform,
})
builder.started = true
}
@@ -664,6 +836,7 @@ func (p *CodexProvider) handleOutputItemDone(
ToolName: builder.Name,
})
builder.ended = true
state.emitted = true
}
// handleTerminalResponse emits the final usage + done event for a
@@ -693,6 +866,7 @@ func (p *CodexProvider) handleTerminalResponse(
return false
}
state.usage = event.Response.Usage
state.responseID = event.Response.ID
if event.Response.Usage != nil {
usage := zeroruntime.Usage{
InputTokens: event.Response.Usage.InputTokens,
@@ -700,6 +874,7 @@ func (p *CodexProvider) handleTerminalResponse(
}
if event.Response.Usage.InputTokensDetails != nil {
usage.CachedInputTokens = event.Response.Usage.InputTokensDetails.CachedTokens
usage.CacheWriteTokens = event.Response.Usage.InputTokensDetails.CacheWriteTokens
}
if event.Response.Usage.OutputTokensDetails != nil {
usage.ReasoningTokens = event.Response.Usage.OutputTokensDetails.ReasoningTokens
@@ -735,7 +910,7 @@ func (p *CodexProvider) handleTerminalResponse(
state.done = true
return false
}
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone})
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{Type: zeroruntime.StreamEventDone, ResponseID: event.Response.ID})
state.done = true
return false
}
@@ -754,6 +929,9 @@ func (p *CodexProvider) toolCallKey(event *responsesEvent) string {
if event.Item != nil && event.Item.CallID != "" {
return event.Item.CallID
}
if event.CallID != "" {
return event.CallID
}
if event.OutputIndex != nil {
// output_index is present (a *int distinguishes a real 0 — the first output
// in the response — from "absent"), so key on it even when 0 instead of
+421
View File
@@ -0,0 +1,421 @@
package openai
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"reflect"
"strings"
"sync"
"time"
"github.com/coder/websocket"
"github.com/Gitlawb/zero/internal/providers/providerio"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
const (
responsesWebSocketBetaHeader = "OpenAI-Beta"
responsesWebSocketBetaValue = "responses_websockets=2026-02-06"
responsesWebSocketRequest = "response.create"
previousResponseNotFoundCode = "previous_response_not_found"
webSocketConnectionLimitCode = "websocket_connection_limit_reached"
responsesWebSocketReadLimit = 16 << 20
)
var errCodexTurnSessionBusy = errors.New("codex turn session already has an active stream")
// NewCodexTurnSessionProvider enables one Responses WebSocket connection for
// each agent run. The session falls back to the existing HTTP/SSE provider when
// the endpoint cannot upgrade or loses its response chain.
func NewCodexTurnSessionProvider(provider *CodexProvider, caps zeroruntime.ProviderCapabilities) zeroruntime.TurnSessionProvider {
return &codexTurnSessionProvider{provider: provider, caps: caps}
}
type codexTurnSessionProvider struct {
provider *CodexProvider
caps zeroruntime.ProviderCapabilities
}
func (provider *codexTurnSessionProvider) OpenTurnSession(ctx context.Context) (zeroruntime.TurnSession, error) {
if provider == nil || provider.provider == nil || provider.provider.inner == nil {
return nil, errors.New("codex turn session requires a provider")
}
sessionCtx, cancel := context.WithCancel(ctx)
return &codexTurnSession{
ctx: sessionCtx,
cancel: cancel,
provider: provider.provider,
prewarmDone: make(chan struct{}),
}, nil
}
func (provider *codexTurnSessionProvider) Capabilities() zeroruntime.ProviderCapabilities {
return provider.caps
}
type codexTurnSession struct {
ctx context.Context
cancel context.CancelFunc
provider *CodexProvider
mu sync.Mutex
connection *websocket.Conn
webSocketOff bool
lastRequest *responsesRequest
lastResponseID string
lastOutput []inputItem
streamActive bool
prewarmOnce sync.Once
prewarmDone chan struct{}
closeOnce sync.Once
}
// Prewarm starts the WebSocket handshake without blocking prompt assembly. The
// first Stream waits for this bounded attempt; failure simply selects HTTP/SSE.
func (session *codexTurnSession) Prewarm(ctx context.Context) error {
session.prewarmOnce.Do(func() {
go func() {
defer close(session.prewarmDone)
prewarmCtx, cancel := context.WithTimeout(session.ctx, prewarmTimeout)
defer cancel()
recorder := trace.FromContext(ctx)
span := recorder.Span(trace.SpanProviderPrewarm)
defer span.End()
recorder.Counter(trace.CounterPrewarmAttempts, 1)
connection, err := session.provider.dialResponsesWebSocket(prewarmCtx)
session.mu.Lock()
defer session.mu.Unlock()
if err != nil {
session.webSocketOff = true
return
}
session.connection = connection
}()
})
return nil
}
func (session *codexTurnSession) Stream(ctx context.Context, request zeroruntime.CompletionRequest) (<-chan zeroruntime.StreamEvent, error) {
_ = session.Prewarm(ctx)
select {
case <-session.prewarmDone:
case <-ctx.Done():
return nil, ctx.Err()
}
fullRequest, err := session.provider.buildResponsesRequest(request)
if err != nil {
return nil, err
}
session.mu.Lock()
if session.streamActive {
session.mu.Unlock()
return nil, errCodexTurnSessionBusy
}
if session.webSocketOff || session.connection == nil {
session.mu.Unlock()
trace.FromContext(ctx).Counter(trace.CounterResponsesHTTPFallback, 1)
return session.provider.StreamCompletion(ctx, request)
}
session.streamActive = true
connection := session.connection
wireRequest := *fullRequest
wireRequest.Type = responsesWebSocketRequest
if delta, ok := responseInputDelta(session.lastRequest, session.lastOutput, fullRequest); ok && session.lastResponseID != "" {
wireRequest.PreviousResponseID = session.lastResponseID
wireRequest.Input = delta
trace.FromContext(ctx).Counter(trace.CounterResponseChainReused, 1)
} else if session.lastRequest != nil {
trace.FromContext(ctx).Counter(trace.CounterResponseChainReset, 1)
session.lastRequest = nil
session.lastResponseID = ""
session.lastOutput = nil
}
session.mu.Unlock()
body, err := json.Marshal(wireRequest)
if err != nil {
session.releaseStream()
return nil, fmt.Errorf("encode codex websocket request: %w", err)
}
events := make(chan zeroruntime.StreamEvent, 16)
go func() {
defer close(events)
defer session.releaseStream()
session.streamWebSocket(ctx, connection, body, fullRequest, request, events)
}()
return events, nil
}
func (session *codexTurnSession) releaseStream() {
session.mu.Lock()
session.streamActive = false
session.mu.Unlock()
}
func (session *codexTurnSession) Compact(context.Context, zeroruntime.CompletionRequest) ([]zeroruntime.Message, error) {
return nil, zeroruntime.ErrCompactionUnsupported
}
func (session *codexTurnSession) Close() error {
session.closeOnce.Do(func() {
session.cancel()
session.prewarmOnce.Do(func() { close(session.prewarmDone) })
select {
case <-session.prewarmDone:
case <-time.After(prewarmTimeout):
}
session.mu.Lock()
connection := session.connection
session.connection = nil
session.webSocketOff = true
session.mu.Unlock()
if connection != nil {
_ = connection.CloseNow()
}
})
return nil
}
func responseInputDelta(previous *responsesRequest, output []inputItem, current *responsesRequest) ([]inputItem, bool) {
if previous == nil || current == nil || !responsesRequestPropertiesEqual(previous, current) {
return nil, false
}
prefix := make([]inputItem, 0, len(previous.Input)+len(output))
prefix = append(prefix, previous.Input...)
prefix = append(prefix, output...)
if len(current.Input) <= len(prefix) || !reflect.DeepEqual(current.Input[:len(prefix)], prefix) {
return nil, false
}
return append([]inputItem(nil), current.Input[len(prefix):]...), true
}
func responsesRequestPropertiesEqual(left, right *responsesRequest) bool {
return left.Model == right.Model &&
left.Instructions == right.Instructions &&
left.Stream == right.Stream &&
left.Store == right.Store &&
left.ParallelToolCalls == right.ParallelToolCalls &&
left.MaxOutputTokens == right.MaxOutputTokens &&
reflect.DeepEqual(left.Tools, right.Tools) &&
reflect.DeepEqual(left.Reasoning, right.Reasoning) &&
left.ServiceTier == right.ServiceTier &&
left.PromptCacheKey == right.PromptCacheKey
}
func (session *codexTurnSession) streamWebSocket(
ctx context.Context,
connection *websocket.Conn,
body []byte,
fullRequest *responsesRequest,
runtimeRequest zeroruntime.CompletionRequest,
events chan<- zeroruntime.StreamEvent,
) {
if err := connection.Write(ctx, websocket.MessageText, body); err != nil {
session.disableWebSocket(connection)
session.forwardHTTP(ctx, runtimeRequest, events)
return
}
state := newResponsesState()
for {
readCtx := ctx
cancel := func() {}
idleTimeout := session.provider.inner.streamIdleTimeout
if idleTimeout > 0 {
readCtx, cancel = context.WithTimeout(ctx, idleTimeout)
}
messageType, data, err := connection.Read(readCtx)
readTimedOut := errors.Is(readCtx.Err(), context.DeadlineExceeded) && ctx.Err() == nil
cancel()
if err != nil {
session.disableWebSocket(connection)
if ctx.Err() != nil {
return
}
if !state.emitted {
session.forwardHTTP(ctx, runtimeRequest, events)
return
}
errorMessage := err.Error()
if readTimedOut {
errorMessage = providerio.StreamTimeoutMessage(providerio.ErrStreamIdle, idleTimeout)
}
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventError,
Error: session.provider.redact("provider stream error: " + errorMessage),
})
return
}
if messageType != websocket.MessageText {
continue
}
var responseEvent responsesEvent
decodeErr := json.Unmarshal(data, &responseEvent)
if decodeErr == nil {
if (responseEvent.Code == previousResponseNotFoundCode || responseEvent.Code == webSocketConnectionLimitCode) &&
!state.emitted {
session.disableWebSocket(connection)
session.forwardHTTP(ctx, runtimeRequest, events)
return
}
if responseEvent.Type == responsesEventIncomplete {
session.resetResponseChain(connection)
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventDone,
FinishReason: zeroruntime.FinishReasonLength,
})
return
}
}
keepReading := false
if decodeErr == nil {
keepReading = session.provider.emitParsedResponsesEvent(ctx, &responseEvent, state, events)
} else {
keepReading = session.provider.emitResponsesEvent(ctx, string(data), state, events)
}
if keepReading {
continue
}
if responseEvent.Type == responsesEventCompleted && responseEvent.Response != nil &&
responseEvent.Response.Error == nil && responseEvent.Response.Status != "failed" && state.responseID != "" {
session.mu.Lock()
if session.connection == connection && !session.webSocketOff {
requestCopy := *fullRequest
requestCopy.Input = append([]inputItem(nil), fullRequest.Input...)
session.lastRequest = &requestCopy
session.lastResponseID = state.responseID
session.lastOutput = state.outputInputItems()
}
session.mu.Unlock()
}
return
}
}
func (session *codexTurnSession) resetResponseChain(connection *websocket.Conn) {
session.mu.Lock()
defer session.mu.Unlock()
if session.connection != connection {
return
}
session.lastRequest = nil
session.lastResponseID = ""
session.lastOutput = nil
}
func (session *codexTurnSession) forwardHTTP(ctx context.Context, request zeroruntime.CompletionRequest, events chan<- zeroruntime.StreamEvent) {
trace.FromContext(ctx).Counter(trace.CounterResponsesHTTPFallback, 1)
stream, err := session.provider.StreamCompletion(ctx, request)
if err != nil {
providerio.SendEvent(ctx, events, zeroruntime.StreamEvent{
Type: zeroruntime.StreamEventError,
Error: session.provider.redact(err.Error()),
})
return
}
for event := range stream {
providerio.SendEvent(ctx, events, event)
}
}
func (session *codexTurnSession) disableWebSocket(connection *websocket.Conn) {
session.mu.Lock()
if session.connection == connection {
session.connection = nil
session.webSocketOff = true
session.lastRequest = nil
session.lastResponseID = ""
session.lastOutput = nil
}
session.mu.Unlock()
_ = connection.CloseNow()
}
func (provider *CodexProvider) dialResponsesWebSocket(ctx context.Context) (*websocket.Conn, error) {
endpoint, err := responsesWebSocketURL(provider.inner.endpoint)
if err != nil {
return nil, err
}
for authTry := 0; authTry < 2; authTry++ {
headers, err := provider.responsesWebSocketHeaders(ctx, authTry == 1)
if err != nil {
return nil, err
}
connection, response, err := websocket.Dial(ctx, endpoint, &websocket.DialOptions{
HTTPClient: provider.inner.httpClient,
HTTPHeader: headers,
CompressionMode: websocket.CompressionNoContextTakeover,
})
if err == nil {
connection.SetReadLimit(responsesWebSocketReadLimit)
return connection, nil
}
if response == nil || response.StatusCode != http.StatusUnauthorized || provider.inner.oauthResolver == nil || authTry == 1 {
return nil, err
}
}
return nil, errors.New("codex websocket authentication failed")
}
func (provider *CodexProvider) responsesWebSocketHeaders(ctx context.Context, forceRefresh bool) (http.Header, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodGet, provider.inner.endpoint, nil)
if err != nil {
return nil, err
}
base := providerio.AuthHeaders{
APIKey: provider.inner.apiKey,
DefaultAuthHeader: "Authorization",
DefaultAuthScheme: "Bearer",
AuthHeader: provider.inner.authHeader,
AuthScheme: provider.inner.authScheme,
AuthHeaderValue: provider.inner.authHeaderValue,
CustomHeaders: provider.inner.customHeaders,
}
if resolver := provider.inner.oauthResolver; resolver != nil {
header, value, ok, err := resolver(ctx, forceRefresh)
if err != nil {
return nil, err
}
if ok {
providerio.ApplyAuthHeaders(request, providerio.AuthHeaders{CustomHeaders: provider.inner.customHeaders})
if strings.TrimSpace(header) == "" {
header = "Authorization"
}
request.Header.Set(header, value)
} else {
providerio.ApplyAuthHeaders(request, base)
}
} else {
providerio.ApplyAuthHeaders(request, base)
}
provider.injectCodexHeaders(request)
request.Header.Set(responsesWebSocketBetaHeader, responsesWebSocketBetaValue)
return request.Header.Clone(), nil
}
func responsesWebSocketURL(endpoint string) (string, error) {
parsed, err := url.Parse(strings.TrimSpace(endpoint))
if err != nil {
return "", err
}
switch parsed.Scheme {
case "http":
parsed.Scheme = "ws"
case "https":
parsed.Scheme = "wss"
default:
return "", fmt.Errorf("unsupported Responses websocket URL scheme %q", parsed.Scheme)
}
return parsed.String(), nil
}
@@ -0,0 +1,590 @@
package openai
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
"github.com/coder/websocket"
"github.com/Gitlawb/zero/internal/trace"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
func TestCodexTurnSessionChainsOnlyAppendOnlyRequests(t *testing.T) {
var mu sync.Mutex
requests := []map[string]any{}
betaHeader := ""
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
if !strings.EqualFold(request.Header.Get("Upgrade"), "websocket") {
http.Error(writer, "websocket required", http.StatusUpgradeRequired)
return
}
mu.Lock()
betaHeader = request.Header.Get(responsesWebSocketBetaHeader)
mu.Unlock()
connection, err := websocket.Accept(writer, request, nil)
if err != nil {
return
}
defer connection.CloseNow()
for index := 0; index < 3; index++ {
_, body, err := connection.Read(request.Context())
if err != nil {
return
}
var payload map[string]any
if err := json.Unmarshal(body, &payload); err != nil {
return
}
mu.Lock()
requests = append(requests, payload)
mu.Unlock()
switch index {
case 0:
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.created","response":{"id":"resp-1","status":"in_progress"}}`,
`{"type":"response.output_item.added","item_id":"call-1","item":{"type":"function_call","id":"call-1","call_id":"call-1","name":"read_file"}}`,
`{"type":"response.function_call_arguments.delta","item_id":"call-1","delta":"{}"}`,
`{"type":"response.output_item.done","item_id":"call-1","item":{"type":"function_call","id":"call-1","call_id":"call-1","name":"read_file","arguments":"{}"}}`,
`{"type":"response.completed","response":{"id":"resp-1","status":"completed"}}`,
)
case 1:
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.output_text.delta","delta":"done"}`,
`{"type":"response.completed","response":{"id":"resp-2","status":"completed"}}`,
)
case 2:
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.completed","response":{"id":"resp-3","status":"completed"}}`,
)
}
}
}))
defer server.Close()
provider := newCodexSessionTestProvider(t, server)
// A disabled stream watchdog must use the parent context rather than creating
// an immediately-expired read deadline.
provider.inner.streamIdleTimeout = 0
session := openCodexSession(t, provider)
defer session.Close()
recorder := trace.NewRecorder("session-1", "run-1", "test")
recorder.Start()
streamContext := trace.WithContext(t.Context(), recorder)
base := zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{
{Role: zeroruntime.MessageRoleSystem, Content: "Use tools carefully."},
{Role: zeroruntime.MessageRoleUser, Content: "Read the file."},
},
Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}},
PromptCacheKey: "session-1",
}
first := collectCodexSessionEvents(t, session, base)
if !hasToolCallEnd(first, "call-1") {
t.Fatalf("first stream did not produce the tool call: %#v", first)
}
second := base
second.Messages = append(append([]zeroruntime.Message(nil), base.Messages...),
zeroruntime.Message{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{{ID: "call-1", Name: "read_file", Arguments: "{}"}}},
zeroruntime.Message{Role: zeroruntime.MessageRoleTool, ToolCallID: "call-1", Content: "contents"},
)
secondEvents := collectCodexSessionEventsContext(t, streamContext, session, second)
if got := joinedText(secondEvents); got != "done" {
t.Fatalf("second stream text = %q, want done", got)
}
third := second
third.ReasoningEffort = "high"
third.Messages = append(append([]zeroruntime.Message(nil), second.Messages...),
zeroruntime.Message{Role: zeroruntime.MessageRoleAssistant, Content: "done"},
zeroruntime.Message{Role: zeroruntime.MessageRoleUser, Content: "Continue."},
)
collectCodexSessionEventsContext(t, streamContext, session, third)
finishedTrace := recorder.Finish()
if got := finishedTrace.Counter(trace.CounterResponseChainReused); got != 1 {
t.Fatalf("response-chain reuse count = %d, want 1", got)
}
if got := finishedTrace.Counter(trace.CounterResponseChainReset); got != 1 {
t.Fatalf("response-chain reset count = %d, want 1", got)
}
mu.Lock()
defer mu.Unlock()
if len(requests) != 3 {
t.Fatalf("websocket requests = %d, want 3", len(requests))
}
if betaHeader != responsesWebSocketBetaValue {
t.Fatalf("OpenAI-Beta = %q, want %q", betaHeader, responsesWebSocketBetaValue)
}
if requests[0]["type"] != responsesWebSocketRequest || requests[0]["previous_response_id"] != nil {
t.Fatalf("first request chaining fields = %#v", requests[0])
}
if requests[0]["parallel_tool_calls"] != true || requests[0]["prompt_cache_key"] != "session-1" {
t.Fatalf("first request stable fields = %#v", requests[0])
}
if requests[1]["previous_response_id"] != "resp-1" {
t.Fatalf("second previous_response_id = %#v", requests[1]["previous_response_id"])
}
secondInput, _ := requests[1]["input"].([]any)
if len(secondInput) != 1 || secondInput[0].(map[string]any)["type"] != "function_call_output" {
t.Fatalf("second incremental input = %#v", requests[1]["input"])
}
if requests[2]["previous_response_id"] != nil {
t.Fatalf("incompatible request reused response chain: %#v", requests[2])
}
thirdInput, _ := requests[2]["input"].([]any)
if len(thirdInput) <= 1 {
t.Fatalf("incompatible request did not send full input: %#v", thirdInput)
}
}
func TestCodexTurnSessionFallsBackWhenPreviousResponseIsMissing(t *testing.T) {
var mu sync.Mutex
webSocketRequests := 0
httpRequests := 0
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
if strings.EqualFold(request.Header.Get("Upgrade"), "websocket") {
connection, err := websocket.Accept(writer, request, nil)
if err != nil {
return
}
defer connection.CloseNow()
for index := 0; index < 2; index++ {
if _, _, err := connection.Read(request.Context()); err != nil {
return
}
mu.Lock()
webSocketRequests++
mu.Unlock()
if index == 0 {
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.output_item.added","item_id":"call-1","item":{"type":"function_call","id":"call-1","call_id":"call-1","name":"read_file"}}`,
`{"type":"response.output_item.done","item_id":"call-1","item":{"type":"function_call","id":"call-1","call_id":"call-1","name":"read_file","arguments":"{}"}}`,
`{"type":"response.completed","response":{"id":"resp-1","status":"completed"}}`,
)
continue
}
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.error","code":"previous_response_not_found","message":"missing"}`,
)
}
return
}
mu.Lock()
httpRequests++
mu.Unlock()
writer.Header().Set("Content-Type", "text/event-stream")
_, _ = fmt.Fprint(writer, "data: {\"type\":\"response.output_text.delta\",\"delta\":\"fallback\"}\n\n")
_, _ = fmt.Fprint(writer, "data: {\"type\":\"response.completed\",\"response\":{\"id\":\"resp-http\",\"status\":\"completed\"}}\n\n")
}))
defer server.Close()
provider := newCodexSessionTestProvider(t, server)
session := openCodexSession(t, provider)
defer session.Close()
base := zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "Read it."}},
Tools: []zeroruntime.ToolDefinition{{Name: "read_file", Parameters: map[string]any{"type": "object"}}},
}
collectCodexSessionEvents(t, session, base)
second := base
second.Messages = append(append([]zeroruntime.Message(nil), base.Messages...),
zeroruntime.Message{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{{ID: "call-1", Name: "read_file", Arguments: "{}"}}},
zeroruntime.Message{Role: zeroruntime.MessageRoleTool, ToolCallID: "call-1", Content: "contents"},
)
events := collectCodexSessionEvents(t, session, second)
if got := joinedText(events); got != "fallback" {
t.Fatalf("fallback stream text = %q, want fallback; events=%#v", got, events)
}
for _, event := range events {
if event.Type == zeroruntime.StreamEventError {
t.Fatalf("response-chain recovery leaked an error: %#v", event)
}
}
mu.Lock()
defer mu.Unlock()
if webSocketRequests != 2 || httpRequests != 1 {
t.Fatalf("requests = websocket:%d http:%d, want 2/1", webSocketRequests, httpRequests)
}
}
func TestCodexTurnSessionFallsBackWhenSocketClosesBeforeOutput(t *testing.T) {
var mu sync.Mutex
httpRequests := 0
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
if strings.EqualFold(request.Header.Get("Upgrade"), "websocket") {
connection, err := websocket.Accept(writer, request, nil)
if err != nil {
return
}
if _, _, err := connection.Read(request.Context()); err == nil {
_ = connection.CloseNow()
}
return
}
mu.Lock()
httpRequests++
mu.Unlock()
writer.Header().Set("Content-Type", "text/event-stream")
_, _ = fmt.Fprint(writer, "data: {\"type\":\"response.output_text.delta\",\"delta\":\"recovered\"}\n\n")
_, _ = fmt.Fprint(writer, "data: {\"type\":\"response.completed\",\"response\":{\"id\":\"resp-http\",\"status\":\"completed\"}}\n\n")
}))
defer server.Close()
provider := newCodexSessionTestProvider(t, server)
session := openCodexSession(t, provider)
defer session.Close()
recorder := trace.NewRecorder("session-fallback", "run-fallback", "test")
recorder.Start()
events := collectCodexSessionEventsContext(t, trace.WithContext(t.Context(), recorder), session, zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "Hello"}},
})
if got := joinedText(events); got != "recovered" {
t.Fatalf("fallback text = %q, want recovered", got)
}
mu.Lock()
defer mu.Unlock()
if httpRequests != 1 {
t.Fatalf("HTTP fallback requests = %d, want 1", httpRequests)
}
if got := recorder.Finish().Counter(trace.CounterResponsesHTTPFallback); got != 1 {
t.Fatalf("responses HTTP fallback counter = %d, want 1", got)
}
}
func TestCodexTurnSessionKeepsWebSocketAfterIncompleteResponse(t *testing.T) {
var mu sync.Mutex
webSocketRequests := 0
httpRequests := 0
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
if strings.EqualFold(request.Header.Get("Upgrade"), "websocket") {
connection, err := websocket.Accept(writer, request, nil)
if err != nil {
return
}
defer connection.CloseNow()
for index := 0; index < 2; index++ {
if _, _, err := connection.Read(request.Context()); err != nil {
return
}
mu.Lock()
webSocketRequests++
mu.Unlock()
if index == 0 {
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.incomplete","response":{"id":"resp-1","status":"incomplete"}}`,
)
continue
}
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.output_text.delta","delta":"websocket"}`,
`{"type":"response.completed","response":{"id":"resp-2","status":"completed"}}`,
)
}
return
}
mu.Lock()
httpRequests++
mu.Unlock()
writer.Header().Set("Content-Type", "text/event-stream")
_, _ = fmt.Fprint(writer, "data: {\"type\":\"response.output_text.delta\",\"delta\":\"fallback\"}\n\n")
_, _ = fmt.Fprint(writer, "data: {\"type\":\"response.completed\",\"response\":{\"id\":\"resp-http\",\"status\":\"completed\"}}\n\n")
}))
defer server.Close()
provider := newCodexSessionTestProvider(t, server)
session := openCodexSession(t, provider)
defer session.Close()
request := zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "Continue."}},
}
first := collectCodexSessionEvents(t, session, request)
if len(first) != 1 || first[0].Type != zeroruntime.StreamEventDone || first[0].FinishReason != zeroruntime.FinishReasonLength {
t.Fatalf("incomplete response events = %#v, want length completion", first)
}
second := collectCodexSessionEvents(t, session, request)
if got := joinedText(second); got != "websocket" {
t.Fatalf("second turn text = %q, want websocket; events=%#v", got, second)
}
mu.Lock()
defer mu.Unlock()
if webSocketRequests != 2 || httpRequests != 0 {
t.Fatalf("requests = websocket:%d http:%d, want 2/0", webSocketRequests, httpRequests)
}
}
func TestCodexTurnSessionDoesNotReplayAfterVisibleOutput(t *testing.T) {
var mu sync.Mutex
httpRequests := 0
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
if strings.EqualFold(request.Header.Get("Upgrade"), "websocket") {
connection, err := websocket.Accept(writer, request, nil)
if err != nil {
return
}
if _, _, err := connection.Read(request.Context()); err == nil {
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.output_text.delta","delta":"visible"}`,
)
}
_ = connection.Close(websocket.StatusInternalError, "interrupted")
return
}
mu.Lock()
httpRequests++
mu.Unlock()
}))
defer server.Close()
provider := newCodexSessionTestProvider(t, server)
session := openCodexSession(t, provider)
defer session.Close()
stream, err := session.Stream(t.Context(), zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "Hello"}},
})
if err != nil {
t.Fatalf("Stream: %v", err)
}
sawText, sawError := false, false
for event := range stream {
sawText = sawText || event.Type == zeroruntime.StreamEventText
sawError = sawError || event.Type == zeroruntime.StreamEventError
}
if !sawText || !sawError {
t.Fatalf("visible interrupted stream = text:%v error:%v", sawText, sawError)
}
mu.Lock()
defer mu.Unlock()
if httpRequests != 0 {
t.Fatalf("visible interrupted stream was replayed over HTTP %d time(s)", httpRequests)
}
}
func TestCodexTurnSessionReportsWebSocketIdleTimeout(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
connection, err := websocket.Accept(writer, request, nil)
if err != nil {
return
}
defer connection.CloseNow()
if _, _, err := connection.Read(request.Context()); err != nil {
return
}
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.output_text.delta","delta":"visible"}`,
)
<-request.Context().Done()
}))
defer server.Close()
provider := newCodexSessionTestProvider(t, server)
provider.inner.streamIdleTimeout = 200 * time.Millisecond
session := openCodexSession(t, provider)
defer session.Close()
stream, err := session.Stream(t.Context(), zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "Hello"}},
})
if err != nil {
t.Fatalf("Stream: %v", err)
}
var streamError string
for event := range stream {
if event.Type == zeroruntime.StreamEventError {
streamError = event.Error
break
}
}
if !strings.Contains(streamError, "idle timeout after 200ms") {
t.Fatalf("stream error = %q, want idle-timeout detail", streamError)
}
}
func TestCodexTurnSessionRejectsOverlappingStreams(t *testing.T) {
started := make(chan struct{})
release := make(chan struct{})
var releaseOnce sync.Once
releaseStream := func() { releaseOnce.Do(func() { close(release) }) }
t.Cleanup(releaseStream)
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
connection, err := websocket.Accept(writer, request, nil)
if err != nil {
return
}
defer connection.CloseNow()
if _, _, err := connection.Read(request.Context()); err != nil {
return
}
close(started)
<-release
writeWebSocketEvents(request.Context(), connection,
`{"type":"response.completed","response":{"id":"resp-1","status":"completed"}}`,
)
}))
defer server.Close()
provider := newCodexSessionTestProvider(t, server)
session := openCodexSession(t, provider)
defer session.Close()
request := zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "Hello"}},
}
first, err := session.Stream(t.Context(), request)
if err != nil {
t.Fatalf("first Stream: %v", err)
}
select {
case <-started:
case <-time.After(time.Second):
t.Fatal("first WebSocket stream did not start")
}
if _, err := session.Stream(t.Context(), request); !errors.Is(err, errCodexTurnSessionBusy) {
t.Fatalf("overlapping Stream error = %v, want %v", err, errCodexTurnSessionBusy)
}
releaseStream()
for range first {
}
}
func TestCodexTurnSessionHTTPFallbackRedactsSetupError(t *testing.T) {
const secret = "sk-secret-fallback"
provider, err := NewCodexProvider(CodexOptions{Options: Options{
APIKey: secret,
BaseURL: "https://chatgpt.example/backend-api/codex",
Model: "gpt-test",
}})
if err != nil {
t.Fatalf("NewCodexProvider: %v", err)
}
session := &codexTurnSession{provider: provider}
events := make(chan zeroruntime.StreamEvent, 1)
session.forwardHTTP(t.Context(), zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRole(secret), Content: "invalid"}},
}, events)
select {
case event := <-events:
if event.Type != zeroruntime.StreamEventError {
t.Fatalf("fallback event type = %s, want error", event.Type)
}
if strings.Contains(event.Error, secret) {
t.Fatalf("fallback error leaked credential: %q", event.Error)
}
case <-time.After(time.Second):
t.Fatal("fallback did not emit setup error")
}
}
func TestCodexTurnSessionCloseDoesNotStartPrewarm(t *testing.T) {
provider, err := NewCodexProvider(CodexOptions{Options: Options{
APIKey: "test-token",
BaseURL: "https://chatgpt.example/backend-api/codex",
Model: "gpt-test",
}})
if err != nil {
t.Fatalf("NewCodexProvider: %v", err)
}
session, err := NewCodexTurnSessionProvider(provider, zeroruntime.ProviderCapabilities{}).OpenTurnSession(t.Context())
if err != nil {
t.Fatalf("OpenTurnSession: %v", err)
}
done := make(chan error, 1)
go func() { done <- session.Close() }()
select {
case err := <-done:
if err != nil {
t.Fatalf("Close() error = %v", err)
}
case <-time.After(200 * time.Millisecond):
t.Fatal("Close blocked without a prewarm attempt")
}
}
func newCodexSessionTestProvider(t *testing.T, server *httptest.Server) *CodexProvider {
t.Helper()
provider, err := NewCodexProvider(CodexOptions{
Options: Options{
APIKey: "test-token",
BaseURL: server.URL,
Model: "gpt-test",
HTTPClient: server.Client(),
},
AccountID: "account-test",
})
if err != nil {
t.Fatalf("NewCodexProvider: %v", err)
}
return provider
}
func openCodexSession(t *testing.T, provider *CodexProvider) zeroruntime.TurnSession {
t.Helper()
session, err := NewCodexTurnSessionProvider(provider, zeroruntime.ProviderCapabilities{}).OpenTurnSession(t.Context())
if err != nil {
t.Fatalf("OpenTurnSession: %v", err)
}
if err := session.Prewarm(t.Context()); err != nil {
t.Fatalf("Prewarm: %v", err)
}
return session
}
func collectCodexSessionEvents(t *testing.T, session zeroruntime.TurnSession, request zeroruntime.CompletionRequest) []zeroruntime.StreamEvent {
t.Helper()
return collectCodexSessionEventsContext(t, t.Context(), session, request)
}
func collectCodexSessionEventsContext(t *testing.T, ctx context.Context, session zeroruntime.TurnSession, request zeroruntime.CompletionRequest) []zeroruntime.StreamEvent {
t.Helper()
stream, err := session.Stream(ctx, request)
if err != nil {
t.Fatalf("Stream: %v", err)
}
events := []zeroruntime.StreamEvent{}
for event := range stream {
events = append(events, event)
if event.Type == zeroruntime.StreamEventError {
t.Fatalf("stream error: %s", event.Error)
}
}
return events
}
func writeWebSocketEvents(ctx context.Context, connection *websocket.Conn, events ...string) {
for _, event := range events {
if err := connection.Write(ctx, websocket.MessageText, []byte(event)); err != nil {
return
}
}
}
func hasToolCallEnd(events []zeroruntime.StreamEvent, id string) bool {
for _, event := range events {
if event.Type == zeroruntime.StreamEventToolCallEnd && event.ToolCallID == id {
return true
}
}
return false
}
func joinedText(events []zeroruntime.StreamEvent) string {
var builder strings.Builder
for _, event := range events {
if event.Type == zeroruntime.StreamEventText {
builder.WriteString(event.Content)
}
}
return builder.String()
}
@@ -2,6 +2,8 @@ package openai
import (
"context"
"reflect"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/zeroruntime"
@@ -25,6 +27,126 @@ func TestToolCallKeyOutputIndexZero(t *testing.T) {
}
}
func TestResponsesStatePreservesDistinctCallID(t *testing.T) {
provider := &CodexProvider{}
state := newResponsesState()
events := make(chan zeroruntime.StreamEvent, 4)
provider.handleOutputItemAdded(context.Background(), &responsesEvent{
ItemID: "item-1",
Item: &itemPayload{
Type: "function_call",
ID: "item-1",
CallID: "call-1",
Name: "read_file",
},
}, state, events)
items := state.outputInputItems()
if len(items) != 1 || items[0].ID != "item-1" || items[0].CallID != "call-1" {
t.Fatalf("captured function call = %#v, want item id item-1 and call id call-1", items)
}
}
func TestCodexCustomApplyPatchStreamProducesFreeformCall(t *testing.T) {
provider := &CodexProvider{}
state := newResponsesState()
events := make(chan zeroruntime.StreamEvent, 16)
patch := "*** Begin Patch\n*** Add File: hello.txt\n+hello\n*** End Patch\n"
for _, payload := range []string{
`{"type":"response.output_item.added","item_id":"item-1","item":{"type":"custom_tool_call","call_id":"call-1","name":"apply_patch"}}`,
`{"type":"response.custom_tool_call_input.delta","item_id":"item-1","call_id":"call-1","delta":"*** Begin Patch\n*** Add File: hello.txt\n"}`,
`{"type":"response.custom_tool_call_input.delta","item_id":"item-1","call_id":"call-1","delta":"+hello\n*** End Patch\n"}`,
`{"type":"response.output_item.done","item_id":"item-1","item":{"type":"custom_tool_call","call_id":"call-1","name":"apply_patch","input":"*** Begin Patch\n*** Add File: hello.txt\n+hello\n*** End Patch\n"}}`,
`{"type":"response.completed","response":{"id":"resp-1","status":"completed"}}`,
} {
if keepReading := provider.emitResponsesEvent(context.Background(), payload, state, events); !keepReading && !state.done {
t.Fatalf("stream stopped before terminal event for %s", payload)
}
}
close(events)
collected := zeroruntime.CollectStream(context.Background(), events)
if len(collected.ToolCalls) != 1 {
t.Fatalf("tool calls = %#v, want one", collected.ToolCalls)
}
call := collected.ToolCalls[0]
if call.ID != "item-1" || call.ProviderCallID != "call-1" || call.Name != "apply_patch" || !call.Freeform || call.Arguments != patch {
t.Fatalf("custom apply_patch call = %#v", call)
}
}
func TestCodexApplyPatchUsesFreeformWireContract(t *testing.T) {
provider, err := NewCodexProvider(CodexOptions{Options: Options{
APIKey: "test-token",
BaseURL: "https://chatgpt.example/backend-api/codex",
Model: "gpt-test",
}})
if err != nil {
t.Fatalf("NewCodexProvider: %v", err)
}
patch := "*** Begin Patch\n*** Add File: hello.txt\n+hello\n*** End Patch\n"
request, err := provider.buildResponsesRequest(zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{
{Role: zeroruntime.MessageRoleUser, Content: "Add the file."},
{Role: zeroruntime.MessageRoleAssistant, ToolCalls: []zeroruntime.ToolCall{{
ID: "item-1", ProviderCallID: "call-1", Name: "apply_patch", Arguments: patch, Freeform: true,
}}},
{Role: zeroruntime.MessageRoleTool, ToolCallID: "item-1", ToolCallProviderID: "call-1", Content: "Done!"},
},
Tools: []zeroruntime.ToolDefinition{{
Name: "apply_patch", Description: "Apply a patch.", Type: zeroruntime.ToolDefinitionFreeform,
}},
})
if err != nil {
t.Fatalf("buildResponsesRequest: %v", err)
}
if len(request.Tools) != 1 || request.Tools[0].Type != string(zeroruntime.ToolDefinitionFreeform) ||
request.Tools[0].Format == nil || request.Tools[0].Format.Syntax != "lark" || request.Tools[0].Parameters != nil {
t.Fatalf("apply_patch tool wire shape = %#v", request.Tools)
}
if !strings.Contains(request.Tools[0].Format.Definition, "*** Begin Patch") {
t.Fatalf("apply_patch grammar = %q", request.Tools[0].Format.Definition)
}
if !strings.Contains(request.Tools[0].Description, "absolute file paths") || !strings.Contains(request.Tools[0].Description, "granted extra write root") {
t.Fatalf("apply_patch description does not explain extra-root paths: %q", request.Tools[0].Description)
}
if len(request.Input) != 3 || request.Input[1].Type != "custom_tool_call" || request.Input[1].ID != "item-1" ||
request.Input[1].CallID != "call-1" || request.Input[1].Input != patch ||
request.Input[2].Type != "custom_tool_call_output" || request.Input[2].CallID != "call-1" {
t.Fatalf("custom apply_patch replay input = %#v", request.Input)
}
}
func TestCodexPreservesCustomApplyPatchFunction(t *testing.T) {
provider, err := NewCodexProvider(CodexOptions{Options: Options{
APIKey: "test-token",
BaseURL: "https://chatgpt.example/backend-api/codex",
Model: "gpt-test",
}})
if err != nil {
t.Fatalf("NewCodexProvider: %v", err)
}
parameters := map[string]any{
"type": "object",
"properties": map[string]any{"value": map[string]any{"type": "string"}},
}
request, err := provider.buildResponsesRequest(zeroruntime.CompletionRequest{
Messages: []zeroruntime.Message{{Role: zeroruntime.MessageRoleUser, Content: "Call the custom tool."}},
Tools: []zeroruntime.ToolDefinition{{
Name: "apply_patch", Description: "Custom integration tool.", Parameters: parameters,
}},
})
if err != nil {
t.Fatalf("buildResponsesRequest: %v", err)
}
if len(request.Tools) != 1 {
t.Fatalf("tools = %#v, want one", request.Tools)
}
tool := request.Tools[0]
if tool.Type != string(zeroruntime.ToolDefinitionFunction) || tool.Format != nil ||
tool.Description != "Custom integration tool." || !reflect.DeepEqual(tool.Parameters, parameters) {
t.Fatalf("custom apply_patch function was rewritten: %#v", tool)
}
}
func TestHandleTerminalResponseNilPayload(t *testing.T) {
p := &CodexProvider{}
@@ -90,11 +212,14 @@ func TestHandleTerminalResponseFailedPayloadWithoutError(t *testing.T) {
// A response.completed with a payload and no error remains a clean done.
ok := make(chan zeroruntime.StreamEvent, 8)
p.handleTerminalResponse(context.Background(),
&responsesEvent{Type: responsesEventCompleted, Response: &responsePayload{Status: "completed"}}, &responsesState{}, ok)
&responsesEvent{Type: responsesEventCompleted, Response: &responsePayload{ID: "resp-1", Status: "completed"}}, &responsesState{}, ok)
close(ok)
for ev := range ok {
if ev.Type == zeroruntime.StreamEventError {
t.Error("response.completed with a non-error payload must NOT emit an error")
}
if ev.Type == zeroruntime.StreamEventDone && ev.ResponseID != "resp-1" {
t.Errorf("done response id = %q, want resp-1", ev.ResponseID)
}
}
}
+3 -3
View File
@@ -771,7 +771,7 @@ func TestCodexProviderParsesResponsesTextDeltas(t *testing.T) {
`{"type":"response.output_text.delta","delta":"hello "}`,
`{"type":"response.output_text.delta","delta":"world"}`,
`{"type":"response.output_text.delta","delta":"!"}`,
`{"type":"response.completed","response":{"id":"resp-1","status":"completed","usage":{"input_tokens":7,"output_tokens":3,"output_tokens_details":{"reasoning_tokens":1},"input_tokens_details":{"cached_tokens":2}}}}`,
`{"type":"response.completed","response":{"id":"resp-1","status":"completed","usage":{"input_tokens":7,"output_tokens":3,"output_tokens_details":{"reasoning_tokens":1},"input_tokens_details":{"cached_tokens":2,"cache_write_tokens":1}}}}`,
)
defer srv.Close()
@@ -809,8 +809,8 @@ func TestCodexProviderParsesResponsesTextDeltas(t *testing.T) {
if usage == nil {
t.Fatal("expected a usage event, got none")
}
if usage.InputTokens != 7 || usage.OutputTokens != 3 || usage.CachedInputTokens != 2 || usage.ReasoningTokens != 1 {
t.Fatalf("usage = %+v, want input=7 output=3 cached=2 reasoning=1", *usage)
if usage.InputTokens != 7 || usage.OutputTokens != 3 || usage.CachedInputTokens != 2 || usage.CacheWriteTokens != 1 || usage.ReasoningTokens != 1 {
t.Fatalf("usage = %+v, want input=7 output=3 cached=2 cache-write=1 reasoning=1", *usage)
}
if !gotDone {
t.Fatal("expected a done event, got none")
+1
View File
@@ -357,6 +357,7 @@ func (provider *Provider) emitChunk(
PromptTokens: chunk.Usage.PromptTokens,
CompletionTokens: chunk.Usage.CompletionTokens,
CachedInputTokens: chunk.Usage.PromptTokensDetails.CachedTokens,
CacheWriteTokens: chunk.Usage.PromptTokensDetails.CacheWriteTokens,
ReasoningTokens: chunk.Usage.CompletionTokensDetails.ReasoningTokens,
},
})
+2 -2
View File
@@ -260,14 +260,14 @@ func TestStreamCompletionAppliesCustomAuthAndHeaders(t *testing.T) {
func TestStreamCompletionEmitsTextUsageAndDone(t *testing.T) {
provider := newTestProvider(t, func(w http.ResponseWriter, r *http.Request) {
writeSSE(w, `{"choices":[{"delta":{"content":"hello "}}]}`)
writeSSE(w, `{"choices":[{"delta":{"content":"zero"}}],"usage":{"prompt_tokens":12,"completion_tokens":5,"prompt_tokens_details":{"cached_tokens":3}}}`)
writeSSE(w, `{"choices":[{"delta":{"content":"zero"}}],"usage":{"prompt_tokens":12,"completion_tokens":5,"prompt_tokens_details":{"cached_tokens":3,"cache_write_tokens":2}}}`)
writeSSE(w, `[DONE]`)
})
events := collectProviderEvents(t, provider)
assertEvent(t, events[0], zeroruntime.StreamEventText, "hello ")
assertEvent(t, events[1], zeroruntime.StreamEventText, "zero")
if events[2].Type != zeroruntime.StreamEventUsage || events[2].Usage.PromptTokens != 12 || events[2].Usage.CompletionTokens != 5 || events[2].Usage.CachedInputTokens != 3 {
if events[2].Type != zeroruntime.StreamEventUsage || events[2].Usage.PromptTokens != 12 || events[2].Usage.CompletionTokens != 5 || events[2].Usage.CachedInputTokens != 3 || events[2].Usage.CacheWriteTokens != 2 {
t.Fatalf("unexpected usage event: %#v", events[2])
}
if events[3].Type != zeroruntime.StreamEventDone {
+6 -1
View File
@@ -195,7 +195,12 @@ func (s *turnSession) computeFingerprint(request zeroruntime.CompletionRequest)
func wireToolsDigest(tools []zeroruntime.ToolDefinition) string {
rendered := make([]string, 0, len(tools))
for _, tool := range tools {
schema, err := json.Marshal(tool.Parameters)
wireShape := struct {
Type zeroruntime.ToolDefinitionType `json:"type,omitempty"`
Parameters map[string]any `json:"parameters,omitempty"`
Format *zeroruntime.ToolDefinitionFormat `json:"format,omitempty"`
}{Type: tool.Type, Parameters: tool.Parameters, Format: tool.Format}
schema, err := json.Marshal(wireShape)
if err != nil {
rendered = append(rendered, tool.Name+"\n"+tool.Description+"\n__non_json:"+tool.Name)
continue
+2 -1
View File
@@ -109,7 +109,8 @@ type usage struct {
}
type promptTokenDetails struct {
CachedTokens int `json:"cached_tokens"`
CachedTokens int `json:"cached_tokens"`
CacheWriteTokens int `json:"cache_write_tokens"`
}
type completionTokenDetails struct {
+43 -12
View File
@@ -15,9 +15,27 @@ import (
// value (e.g. "1") to enable — the same boolean idiom as ZERO_FORMAT_ON_WRITE.
const openaiTurnSessionEnv = "ZERO_OPENAI_TURN_SESSION"
// ChatGPT Responses sessions are enabled by default for their native provider.
// Setting this to 0/false/off restores the stateless HTTP/SSE transport.
const chatGPTTurnSessionEnv = "ZERO_CHATGPT_TURN_SESSION"
func openaiTurnSessionEnabled() bool {
value := strings.TrimSpace(os.Getenv(openaiTurnSessionEnv))
return value != "" && value != "0" && !strings.EqualFold(value, "false")
return value != "" && !explicitEnvFalse(value)
}
func chatGPTTurnSessionEnabled() bool {
value := strings.TrimSpace(os.Getenv(chatGPTTurnSessionEnv))
return value == "" || !explicitEnvFalse(value)
}
func explicitEnvFalse(value string) bool {
switch strings.ToLower(strings.TrimSpace(value)) {
case "0", "false", "off":
return true
default:
return false
}
}
// OptimizedTurnSessions returns the gated optimized TurnSessionProvider for an
@@ -25,20 +43,33 @@ func openaiTurnSessionEnabled() bool {
// is ineligible. Callers leave agent Options.TurnSessionProvider nil on false,
// so the loop's default wrap keeps today's behavior byte-identical.
//
// Eligibility is deliberately narrow: the resolved provider kind must be
// official OpenAI (NOT openai-compatible gateways — the constructor merges the
// two, so this branches on the resolved kind explicitly), not the ChatGPT
// Codex catalog, and the provider value must be the concrete *openai.Provider
// (a fake, a Codex provider, or nil falls back to the default path safely).
// No base-URL check on top of the kind: prewarm and fingerprint telemetry are
// harmless against any host, and kind==openai mirrors the existing
// official-OpenAI precedent used for prompt_cache_key.
// Eligibility is deliberately narrow: native ChatGPT Responses uses its
// concrete Codex provider and defaults on with a kill switch; official OpenAI
// chat-completions keeps its existing opt-in prewarm session. Compatible
// gateways and foreign provider values always retain the default adapter.
func OptimizedTurnSessions(profile config.ProviderProfile, provider zeroruntime.Provider, options Options) (zeroruntime.TurnSessionProvider, bool) {
if !openaiTurnSessionEnabled() {
resolved, err := resolveProfile(profile, options)
if err != nil {
return nil, false
}
resolved, err := resolveProfile(profile, options)
if err != nil || resolved.providerKind != config.ProviderKindOpenAI || isCodexCatalog(profile, resolved) {
if isCodexCatalog(profile, resolved) {
if !chatGPTTurnSessionEnabled() {
return nil, false
}
concrete, ok := provider.(*openai.CodexProvider)
if !ok {
return nil, false
}
caps, err := resolveCapabilities(profile, options)
if err != nil {
return nil, false
}
return openai.NewCodexTurnSessionProvider(concrete, caps), true
}
if resolved.providerKind != config.ProviderKindOpenAI {
return nil, false
}
if !openaiTurnSessionEnabled() {
return nil, false
}
concrete, ok := provider.(*openai.Provider)
+72 -5
View File
@@ -5,6 +5,7 @@ import (
"testing"
"github.com/Gitlawb/zero/internal/config"
"github.com/Gitlawb/zero/internal/providers/openai"
"github.com/Gitlawb/zero/internal/zeroruntime"
)
@@ -45,7 +46,7 @@ func TestOptimizedTurnSessionsDefaultOff(t *testing.T) {
func TestOptimizedTurnSessionsFalseyValues(t *testing.T) {
profile := openaiEligibleProfile()
provider := buildGateProvider(t, profile)
for _, value := range []string{"0", "false", "FALSE", "False", " ", ""} {
for _, value := range []string{"0", "false", "FALSE", "False", "off", "OFF", " ", ""} {
t.Setenv(openaiTurnSessionEnv, value)
if _, ok := OptimizedTurnSessions(profile, provider, Options{}); ok {
t.Fatalf("gate enabled for falsey value %q", value)
@@ -59,6 +60,21 @@ func TestOptimizedTurnSessionsFalseyValues(t *testing.T) {
}
}
func TestChatGPTTurnSessionGateValues(t *testing.T) {
for _, value := range []string{"0", "false", "FALSE", "off", "OFF"} {
t.Setenv(chatGPTTurnSessionEnv, value)
if chatGPTTurnSessionEnabled() {
t.Fatalf("ChatGPT gate enabled for falsey value %q", value)
}
}
for _, value := range []string{"", " ", "1", "true", "on"} {
t.Setenv(chatGPTTurnSessionEnv, value)
if !chatGPTTurnSessionEnabled() {
t.Fatalf("ChatGPT gate disabled for default/truthy value %q", value)
}
}
}
func TestOptimizedTurnSessionsOpenAIEligible(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, "1")
profile := openaiEligibleProfile()
@@ -88,13 +104,64 @@ func TestOptimizedTurnSessionsRejectsCompatible(t *testing.T) {
}
func TestOptimizedTurnSessionsRejectsCodexCatalog(t *testing.T) {
t.Setenv(openaiTurnSessionEnv, "1")
t.Setenv(chatGPTTurnSessionEnv, "0")
profile := openaiEligibleProfile()
profile.Name = "chatgpt"
profile.CatalogID = "chatgpt"
if _, ok := OptimizedTurnSessions(profile, buildChatGPTGateProvider(t, profile), Options{}); ok {
t.Fatal("the ChatGPT Codex catalog must honor the turn-session kill switch")
}
}
func TestOptimizedTurnSessionsEnablesChatGPTResponsesByDefault(t *testing.T) {
t.Setenv(chatGPTTurnSessionEnv, "")
profile := openaiEligibleProfile()
profile.Name = "chatgpt"
profile.CatalogID = "chatgpt"
provider := buildChatGPTGateProvider(t, profile)
turnSessions, ok := OptimizedTurnSessions(profile, provider, Options{})
if !ok || turnSessions == nil {
t.Fatal("ChatGPT Responses provider did not receive its default turn session")
}
if got := turnSessions.Capabilities().Model; got != profile.Model {
t.Fatalf("Capabilities().Model = %q, want %q", got, profile.Model)
}
}
func TestOptimizedTurnSessionsRecognizesCompatibleLabeledChatGPTProfile(t *testing.T) {
t.Setenv(chatGPTTurnSessionEnv, "")
profile := openaiEligibleProfile()
profile.Name = "chatgpt"
profile.CatalogID = "chatgpt"
profile.ProviderKind = config.ProviderKindOpenAICompatible
if turnSessions, ok := OptimizedTurnSessions(profile, buildChatGPTGateProvider(t, profile), Options{}); !ok || turnSessions == nil {
t.Fatal("ChatGPT catalog profile must use its native session regardless of the stored compatibility label")
}
}
func buildChatGPTGateProvider(t *testing.T, profile config.ProviderProfile) *openai.CodexProvider {
t.Helper()
provider, err := openai.NewCodexProvider(openai.CodexOptions{
Options: openai.Options{
APIKey: "test-token",
BaseURL: "https://chatgpt.example/backend-api/codex",
Model: profile.Model,
},
AccountID: "account-test",
})
if err != nil {
t.Fatalf("NewCodexProvider: %v", err)
}
return provider
}
func TestOptimizedTurnSessionsRejectsNonCodexProviderForChatGPT(t *testing.T) {
t.Setenv(chatGPTTurnSessionEnv, "1")
profile := openaiEligibleProfile()
profile.CatalogID = "chatgpt"
// Build the provider from the base profile: the point under test is the
// catalog check, not Codex construction (which needs OAuth wiring).
if _, ok := OptimizedTurnSessions(profile, buildGateProvider(t, openaiEligibleProfile()), Options{}); ok {
t.Fatal("the ChatGPT Codex catalog must not get the optimized session")
t.Fatal("a ChatGPT catalog profile with a non-Codex provider must use the default session")
}
}
+2
View File
@@ -7,6 +7,7 @@ import (
type BackendOptions struct {
GOOS string
LookupExecutable func(string) (string, error)
DetectWSL func() WSLInfo
}
type Backend struct {
@@ -53,6 +54,7 @@ func SelectBackend(options BackendOptions) Backend {
return NewSandboxManager(SandboxManagerOptions{
GOOS: options.GOOS,
LookupExecutable: options.LookupExecutable,
DetectWSL: options.DetectWSL,
}).Backend()
}
+15
View File
@@ -45,6 +45,7 @@ func TestSelectBackendChoosesPlatformAdapterWithFallback(t *testing.T) {
}
return "", errors.New("missing")
},
DetectWSL: func() WSLInfo { return WSLInfo{} },
})
if backend.Name != BackendUnavailable || backend.Available {
t.Fatalf("linux backend = %#v, want native sandbox unavailable without Linux helper", backend)
@@ -54,6 +55,20 @@ func TestSelectBackendChoosesPlatformAdapterWithFallback(t *testing.T) {
}
})
t.Run("linux helper missing under WSL reports WSL fallback", func(t *testing.T) {
backend := SelectBackend(BackendOptions{
GOOS: "linux",
LookupExecutable: func(string) (string, error) { return "", errors.New("missing") },
DetectWSL: func() WSLInfo { return WSLInfo{IsWSL: true, IsWSL2: true} },
})
if backend.Name != BackendWSL || backend.Available || !backend.Fallback {
t.Fatalf("linux backend = %#v, want explicit WSL fallback", backend)
}
if !strings.Contains(backend.Message, "WSL2") {
t.Fatalf("linux fallback message = %q, want WSL2 diagnostic", backend.Message)
}
})
t.Run("darwin sandbox exec available", func(t *testing.T) {
backend := SelectBackend(BackendOptions{
GOOS: "darwin",
+7 -3
View File
@@ -51,6 +51,7 @@ const (
type SandboxManagerOptions struct {
GOOS string
LookupExecutable func(string) (string, error)
DetectWSL func() WSLInfo
Backend Backend
}
@@ -96,7 +97,7 @@ func NewSandboxManager(options SandboxManagerOptions) SandboxManager {
goos = runtime.GOOS
}
if backend.Name == "" {
backend = selectPlatformBackend(goos, options.LookupExecutable)
backend = selectPlatformBackend(goos, options.LookupExecutable, options.DetectWSL)
}
if backend.Platform == "" {
backend.Platform = goos
@@ -168,10 +169,13 @@ func lookupExecutable(name string) (string, error) {
return "", errors.New("executable file not found")
}
func selectPlatformBackend(goos string, lookup func(string) (string, error)) Backend {
func selectPlatformBackend(goos string, lookup func(string) (string, error), detect func() WSLInfo) Backend {
if lookup == nil {
lookup = lookupExecutable
}
if detect == nil {
detect = detectWSL
}
switch goos {
case "linux":
if helper, err := lookup(LinuxSandboxHelperName); err == nil && helper != "" {
@@ -180,7 +184,7 @@ func selectPlatformBackend(goos string, lookup func(string) (string, error)) Bac
}
return nativeBackend(goos, BackendLinuxBwrap, helper, "Linux sandbox helper available")
}
if info := detectWSL(); info.IsWSL {
if info := detect(); info.IsWSL {
return wslBackend(goos, info)
}
return unavailableBackend(goos, "Linux sandbox helper is not available")
+92 -3
View File
@@ -96,15 +96,104 @@ var protectedMetadataNames = []string{".git", ".zero", ".agents"}
// gitMetadataWriteCarveouts below.
var sandboxFullyProtectedMetadataNames = []string{".zero", ".agents"}
// sandboxRenameProtectedMetadataName is the metadata directory that cannot be
// fully write-protected but must still not be REPLACEABLE.
//
// It is deliberately not in the list above. That list denies write, and git has
// to write index, objects and refs. But the carveouts guarding it are attached
// to .git/config and .git/hooks as objects, so a principal that renames .git and
// recreates it gets fresh paths inheriting the workspace allow with no denies,
// which restores credential.helper and core.hooksPath. The Windows ACL plan
// therefore denies DELETE on this directory alone, uninherited.
const sandboxRenameProtectedMetadataName = ".git"
// gitMetadataWriteCarveouts returns the .git subpaths that stay write-denied
// under the OS-level sandbox even though the rest of .git is writable to git
// subprocesses. Nonexistent paths are harmless no-ops in every backend's
// enforcement (seatbelt regex, bwrap ro-bind, Windows ACL deny entry).
func gitMetadataWriteCarveouts(root string) []string {
return []string{
filepath.Join(root, ".git", "hooks"),
filepath.Join(root, ".git", "config"),
specs := gitMetadataWriteCarveoutSpecs(root)
out := make([]string, 0, len(specs))
for _, spec := range specs {
out = append(out, spec.Path)
}
return out
}
// gitMetadataCarveout is a write-denied .git path together with the shape git
// expects it to have. The shape matters to exactly one backend: the Windows ACL
// plan creates a missing carveout so the deny ACE is in place before git first
// runs, and creating .git/config as a directory makes `git init` fail outright.
// Every other backend only ever names the path, so it can ignore IsFile.
type gitMetadataCarveout struct {
Path string
IsFile bool
}
// gitMetadataWriteCarveoutSpecs is the single source of truth for the carveout
// set. gitMetadataWriteCarveouts derives its list from this so a path can never
// be added in one place and have its shape forgotten in the other.
func gitMetadataWriteCarveoutSpecs(root string) []gitMetadataCarveout {
gitPath := filepath.Join(root, ".git")
// A LINKED WORKTREE OR SUBMODULE HAS .git AS A FILE, NOT A DIRECTORY.
//
// It holds a `gitdir:` pointer to the real control directory, which lives
// outside this write root. Naming .git/config and .git/hooks there asks the
// Windows plan to materialize them by descending through .git as a directory,
// which cannot open a child beneath a regular file, so opted-in elevated setup
// aborts and the sandbox is unusable in any worktree. Zero's own development
// worktrees are exactly this shape.
//
// Deny the pointer file itself instead. It is the right protection rather than
// a lesser one: a principal that can rewrite `gitdir:` repoints the whole
// repository at a control directory it chooses, which subsumes editing config
// or dropping a hook. The real control directory is outside the write root, so
// the principal has no inherited access to it and needs no carveout there.
//
// Stat, not a lexical guess. Whether .git is a file is a property of the
// checkout, and the two layouts want different ACEs. A missing .git (git has
// not run yet) keeps the directory-shaped carveouts, which is what makes them
// materialize before git first runs.
if info, err := os.Lstat(gitPath); err == nil && !info.IsDir() {
return []gitMetadataCarveout{{Path: gitPath, IsFile: true}}
}
return []gitMetadataCarveout{
{Path: filepath.Join(root, ".git", "hooks")},
{Path: filepath.Join(root, ".git", "config"), IsFile: true},
}
}
// gitMetadataCarveoutSuffixBase is a sentinel root used only to recover the
// trailing segments of the carveout specs. It is never touched on disk.
const gitMetadataCarveoutSuffixBase = string(filepath.Separator) + "zero-carveout-base"
// gitMetadataCarveoutIsFile reports whether path names a carveout git expects
// to be a file.
//
// It matches on the trailing segments rather than on a whole reconstructed
// path. The subpaths reaching the ACL plan are already normalized — resolved
// through EvalSymlinks where that succeeds — while a rebuilt spec path cannot
// be, because .git/config does not exist yet at setup and resolution falls back
// to a plain Clean. On a host where two spellings of the same path differ (an
// 8.3 short name, different casing) a whole-path equality check silently misses
// and the carveout is created as a directory again, which is the original bug
// reintroduced quietly. The suffix cannot drift from the spec list because it
// is derived from it.
func gitMetadataCarveoutIsFile(path string) bool {
candidate := strings.ToLower(filepath.Clean(strings.TrimSpace(path)))
if candidate == "" {
return false
}
for _, spec := range gitMetadataWriteCarveoutSpecs(gitMetadataCarveoutSuffixBase) {
if !spec.IsFile {
continue
}
suffix := strings.ToLower(strings.TrimPrefix(spec.Path, gitMetadataCarveoutSuffixBase))
if suffix != "" && strings.HasSuffix(candidate, suffix) {
return true
}
}
return false
}
func PermissionProfileFromPolicy(workspaceRoot string, policy Policy, scope *Scope) PermissionProfile {
+79
View File
@@ -0,0 +1,79 @@
package sandbox
import (
"os"
"path/filepath"
"strings"
"testing"
)
// A LINKED WORKTREE HAS .git AS A FILE.
//
// The Windows principal plan materializes every carveout, and it does so by
// descending through .git as a directory. Against a `gitdir:` pointer file that
// walk cannot open a child, so opted-in elevated setup aborts and the sandbox
// cannot be used in a worktree at all. Zero's own development worktrees have
// exactly this layout, so this is the common case for anyone working on the
// sandbox itself.
func TestGitCarveoutsHandleAWorktreeGitfile(t *testing.T) {
root := t.TempDir()
gitFile := filepath.Join(root, ".git")
// The real shape git writes for a linked worktree.
if err := os.WriteFile(gitFile, []byte("gitdir: "+filepath.Join(root, "..", "main", ".git", "worktrees", "wt")+"\n"), 0o600); err != nil {
t.Fatalf("WriteFile: %v", err)
}
specs := gitMetadataWriteCarveoutSpecs(root)
if len(specs) != 1 {
t.Fatalf("carveout specs = %#v, want only the .git pointer file", specs)
}
if filepath.Clean(specs[0].Path) != filepath.Clean(gitFile) {
t.Fatalf("carveout path = %s, want the .git pointer file %s", specs[0].Path, gitFile)
}
if !specs[0].IsFile {
t.Fatal("the .git pointer carveout is not marked as a file, so materialization would create a directory where git needs a file")
}
// Nothing may name a path BENEATH the pointer file, which is what forces the
// directory descent that aborts setup.
for _, path := range gitMetadataWriteCarveouts(root) {
rest := strings.TrimPrefix(filepath.Clean(path), filepath.Clean(gitFile))
if rest != "" {
t.Errorf("carveout %s descends through the .git pointer file, which cannot have children", path)
}
}
}
// The ordinary repository layout must keep the config and hooks carveouts. They
// are the whole reason the set exists: without them a principal reinstates
// credential.helper or core.hooksPath through inherited workspace write access.
func TestGitCarveoutsKeepConfigAndHooksForARealGitDir(t *testing.T) {
root := t.TempDir()
if err := os.MkdirAll(filepath.Join(root, ".git"), 0o700); err != nil {
t.Fatalf("MkdirAll: %v", err)
}
paths := gitMetadataWriteCarveouts(root)
for _, want := range []string{filepath.Join(root, ".git", "hooks"), filepath.Join(root, ".git", "config")} {
found := false
for _, got := range paths {
if filepath.Clean(got) == filepath.Clean(want) {
found = true
break
}
}
if !found {
t.Errorf("carveout %s missing for an ordinary .git directory: %#v", want, paths)
}
}
}
// A workspace where git has never run keeps the directory-shaped carveouts. That
// is what lets the Windows plan create them before git first runs, so the deny
// ACEs are already in place rather than being applied to paths git will create
// later with inherited write access.
func TestGitCarveoutsKeepConfigAndHooksWhenGitIsAbsent(t *testing.T) {
root := t.TempDir()
paths := gitMetadataWriteCarveouts(root)
if len(paths) != 2 {
t.Fatalf("carveouts = %#v, want the config and hooks pair when .git is absent", paths)
}
}
@@ -352,6 +352,15 @@ func realSmokeExecutable(t *testing.T, envKey string, fallbackName string) strin
func runWindowsRealSmokeSetup(t *testing.T, setupExe string, options WindowsSandboxSetupArgsOptions) {
t.Helper()
// options.PrincipalOptIn is deliberately left nil by both call sites, which
// makes BuildWindowsSandboxSetupArgs resolve the opt-in from this process's
// environment — the same value the command half resolves, since the smoke
// WindowsSandboxCommandArgsOptions carries no explicit entry either. Do not
// "fix" this by setting it to false: anyone running this suite with
// ZERO_WINDOWS_SANDBOX_IDENTITY=1 (the only way to exercise the principal
// backend) would then serialize `--sandbox-principal 0`, disagree with the
// command half, and fail every command at marker validation instead of
// testing the sandbox.
args, err := BuildWindowsSandboxSetupArgs(options)
if err != nil {
t.Fatalf("BuildWindowsSandboxSetupArgs: %v", err)
@@ -0,0 +1,33 @@
//go:build !windows
package sandbox
import (
"os"
"path/filepath"
)
// physicalTempDir returns os.TempDir with every symlink in its ancestry
// resolved, so the fallback anchor's parent chain is physical and only the
// anchor itself is a new component.
//
// peermsg.EnsurePrivateDir walks from the root with O_NOFOLLOW and refuses any
// component that is a link. That is right for the owned tail and wrong for the
// ancestors above it: on macOS os.TempDir lives under /var, which is a symlink
// to /private/var, so the first version of this fix refused every fallback on
// every Mac with "refusing non-directory or symlink runtime path component
// var". The constraint jatmn stated for #901 applies here identically:
// redirected cache and TEMP locations above the owned tail are the operator's
// business and must keep working; the restriction is on what Zero owns.
//
// EvalSymlinks is the correct resolver off Windows, where the only reparse
// shape is a symlink and it traverses them.
func physicalTempDir() (string, error) {
return physicalDir(os.TempDir())
}
// physicalDir resolves any directory the same way, for the other places that
// have to hand EnsurePrivateDir a physical parent.
func physicalDir(path string) (string, error) {
return filepath.EvalSymlinks(path)
}
@@ -0,0 +1,63 @@
//go:build windows
package sandbox
import (
"fmt"
"os"
"golang.org/x/sys/windows"
)
// physicalTempDir returns os.TempDir as the object it actually names, so the
// fallback anchor's parent chain is physical and only the anchor itself is a
// new component.
//
// peermsg.EnsurePrivateDir walks from the volume root with OBJ_DONT_REPARSE and
// refuses any component that is a reparse point. That is right for the owned
// tail and wrong for the ancestors above it: a redirected %LOCALAPPDATA% is an
// ordinary Windows configuration, and TEMP sits beneath it. The constraint
// jatmn stated for #901 applies here identically: redirected cache and TEMP
// locations above the owned tail must keep working; the restriction is on what
// Zero owns.
//
// GetFinalPathNameByHandle rather than filepath.EvalSymlinks, because
// EvalSymlinks does not traverse a junction on Windows: it returns the
// junction's own path and the walker then refuses it. The handle answers where
// the directory actually is. Same recipe as verifyWindowsACLTargetNotRedirected,
// which is why the flag constants and the prefix trim are shared.
func physicalTempDir() (string, error) {
return physicalDir(os.TempDir())
}
// physicalDir resolves any directory through a handle, for the other places
// that have to hand EnsurePrivateDir a physical parent. GetFinalPathNameByHandle
// rather than EvalSymlinks, because EvalSymlinks does not traverse a junction.
func physicalDir(path string) (string, error) {
utf16Path, err := windows.UTF16PtrFromString(path)
if err != nil {
return "", fmt.Errorf("encode directory %s: %w", path, err)
}
handle, err := windows.CreateFile(
utf16Path,
windows.FILE_READ_ATTRIBUTES|windows.SYNCHRONIZE,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
nil,
windows.OPEN_EXISTING,
windows.FILE_FLAG_BACKUP_SEMANTICS,
0,
)
if err != nil {
return "", fmt.Errorf("open directory %s: %w", path, err)
}
defer windows.CloseHandle(handle)
buffer := make([]uint16, windows.MAX_LONG_PATH)
n, err := windows.GetFinalPathNameByHandle(handle, &buffer[0], uint32(len(buffer)), windowsFileNameNormalized|windowsVolumeNameDOS)
if err != nil {
return "", fmt.Errorf("resolve directory %s: %w", path, err)
}
if int(n) < len(buffer) {
buffer = buffer[:n]
}
return trimWindowsExtendedPathPrefix(windows.UTF16ToString(buffer)), nil
}
@@ -0,0 +1,155 @@
//go:build windows
package sandbox
import (
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// DETERMINISM IS NOT OWNERSHIP.
//
// The fallback runtime root moved from a fresh os.MkdirTemp parent to a
// predictable name so that elevated setup and the later command agree on it.
// That is required. But a predictable path beneath the shared system temp is
// not a private allocation: whatever another local user, or a stray earlier
// process, has already placed at that name is what preparation then leases,
// creates, chmods and cleans through. The anchor beneath which the tree lives
// must therefore be proven to be this user's private directory before anything
// is built under it, on BOTH paths that can hand back a fallback root.
//
// Junctions need no privilege on Windows, so the "attacker-precreated parent"
// jatmn described is modelled as a junction at the anchor pointing into a
// directory the test owns and watches. Either trigger must refuse, and the
// watched directory must stay empty.
// fallbackAnchorFixture redirects TEMP so the anchor lands inside a directory
// this test owns, plants a junction at the anchor into a watched target, and
// returns the target. The junction is the shape that defeats a pathname walk:
// it is not a symlink to os.ModeSymlink, and EvalSymlinks will not traverse it.
func fallbackAnchorFixture(t *testing.T) (target string) {
t.Helper()
tempRoot := t.TempDir()
target = t.TempDir()
t.Setenv("TMP", tempRoot)
t.Setenv("TEMP", tempRoot)
// The anchor's parent is the PHYSICAL temp dir, so compare against the
// resolved form of the redirect rather than its spelling: t.TempDir can
// come back as an 8.3 short name or in a different case from what the
// handle reports, and a prefix check on the raw string would fail for a
// correct anchor.
anchor := fallbackRuntimeAnchor()
resolvedTempRoot, err := physicalTempDir()
if err != nil {
t.Fatalf("SETUP INVALID: cannot resolve the redirected TEMP %s: %v", tempRoot, err)
}
if !pathWithinRoot(resolvedTempRoot, anchor) {
t.Fatalf("SETUP INVALID: anchor %s is not under the redirected TEMP %s (resolved %s)", anchor, tempRoot, resolvedTempRoot)
}
if out, err := exec.Command("cmd", "/c", "mklink", "/J", anchor, target).CombinedOutput(); err != nil {
t.Fatalf("mklink /J: %v\n%s", err, out)
}
return target
}
func assertNothingUnder(t *testing.T, target string) {
t.Helper()
entries, err := os.ReadDir(target)
if err != nil {
t.Fatal(err)
}
if len(entries) != 0 {
names := make([]string, 0, len(entries))
for _, entry := range entries {
names = append(names, entry.Name())
}
t.Errorf("preparation built %v beneath the junction target, which is somebody else's directory", names)
}
}
// Trigger one: the user cache resolves INSIDE the workspace, so
// sandboxRuntimeRootFor itself returns the fallback root and the first lease
// attempt is already against it.
func TestFallbackAnchorIsRefusedWhenTheCacheLandsInsideTheWorkspace(t *testing.T) {
target := fallbackAnchorFixture(t)
workspace := t.TempDir()
previous := sandboxUserCacheDir
t.Cleanup(func() { sandboxUserCacheDir = previous })
sandboxUserCacheDir = func() (string, error) { return filepath.Join(workspace, ".cache"), nil }
_, release, err := prepareSandboxRuntime(workspace)
if err == nil {
release()
t.Fatal("preparation went through a junction at the fallback anchor and reported success")
}
if !strings.Contains(err.Error(), "fallback anchor") {
t.Errorf("the refusal does not name the anchor, so the operator cannot act on it: %v", err)
}
assertNothingUnder(t, target)
}
// Trigger two: the cache root cannot be leased (a FILE sits where the cache
// runtime tree would go), so preparation falls back explicitly.
func TestFallbackAnchorIsRefusedWhenTheCacheRootCannotBeLeased(t *testing.T) {
target := fallbackAnchorFixture(t)
workspace := t.TempDir()
// Make the cache-derived root unleasable at its ROOT: the user cache is a
// regular file, so nothing beneath it can be created and the first lease
// fails, which is the path that asks for the fallback explicitly. The
// first draft planted the file at the derived leaf instead, and the
// preparation failed on that mkdir before ever reaching the fallback
// branch, so the trigger under test never fired.
cache := filepath.Join(t.TempDir(), "cache-is-a-file")
if err := os.WriteFile(cache, []byte("not a directory"), 0o600); err != nil {
t.Fatal(err)
}
previous := sandboxUserCacheDir
t.Cleanup(func() { sandboxUserCacheDir = previous })
sandboxUserCacheDir = func() (string, error) { return cache, nil }
_, release, err := prepareSandboxRuntime(workspace)
if err == nil {
release()
t.Fatal("preparation fell back through a junction at the anchor and reported success")
}
if !strings.Contains(err.Error(), "fallback anchor") {
t.Errorf("the refusal does not name the anchor: %v", err)
}
assertNothingUnder(t, target)
}
// And the honest control: with nothing planted, the fallback prepares, the
// anchor is a real directory this test can see, and the runtime tree sits
// beneath it.
func TestFallbackAnchorIsCreatedPrivatelyWhenNothingIsInTheWay(t *testing.T) {
tempRoot := t.TempDir()
t.Setenv("TMP", tempRoot)
t.Setenv("TEMP", tempRoot)
workspace := t.TempDir()
previous := sandboxUserCacheDir
t.Cleanup(func() { sandboxUserCacheDir = previous })
sandboxUserCacheDir = func() (string, error) { return filepath.Join(workspace, ".cache"), nil }
runtimeState, release, err := prepareSandboxRuntime(workspace)
if err != nil {
t.Fatalf("a clean fallback was refused: %v", err)
}
defer release()
anchor := fallbackRuntimeAnchor()
info, err := os.Lstat(anchor)
if err != nil || !info.IsDir() {
t.Fatalf("the anchor was not created as a directory: err=%v", err)
}
if !pathWithinRoot(anchor, runtimeState.Root) {
t.Errorf("runtime root %s is not beneath the anchor %s", runtimeState.Root, anchor)
}
}
+85
View File
@@ -0,0 +1,85 @@
package sandbox
import (
"path/filepath"
"strings"
"testing"
)
// SETUP AND EVERY COMMAND RUNNER MUST DERIVE THE SAME FALLBACK ROOT.
//
// They are separate processes. The fallback used to call os.MkdirTemp and cache
// the answer in a process-global map, so elevated setup granted the sandbox
// principal write access to the directory IT made, and each later
// __windows-command-runner made a different one and pointed TMP, GOCACHE and npm
// there. Those directories are created by the calling user and carry no ACE for
// the principal, so ordinary cache writes failed with a bare ACCESS_DENIED.
//
// A repeated call is what a second process looks like from here: no shared
// state, same inputs. The old implementation could not pass this, because the
// only thing that made repeat calls agree was the in-process map.
func TestFallbackRuntimeRootIsTheSameForEveryProcess(t *testing.T) {
workspace := filepath.Join(t.TempDir(), "ws")
first, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatalf("fallbackSandboxRuntimeRoot: %v", err)
}
second, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatalf("fallbackSandboxRuntimeRoot (second process): %v", err)
}
if first != second {
t.Fatalf("two processes derived different runtime roots, so setup grants one and commands write to another:\n %s\n %s", first, second)
}
if first == "" {
t.Fatal("no runtime root derived")
}
}
// Different workspaces must not share a runtime tree, or one workspace's
// principal would hold an ACE on another's caches.
func TestFallbackRuntimeRootIsPerWorkspace(t *testing.T) {
base := t.TempDir()
first, err := fallbackSandboxRuntimeRoot(filepath.Join(base, "alpha"))
if err != nil {
t.Fatal(err)
}
second, err := fallbackSandboxRuntimeRoot(filepath.Join(base, "beta"))
if err != nil {
t.Fatal(err)
}
if first == second {
t.Fatalf("two workspaces share the runtime root %q", first)
}
}
// It must NAME the tree without creating it. Teardown asks for this path on its
// way out, and materializing a directory there would leave one behind on every
// teardown.
func TestFallbackRuntimeRootCreatesNothing(t *testing.T) {
workspace := filepath.Join(t.TempDir(), "ws")
root, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatal(err)
}
if pathExists(root) {
t.Errorf("naming the runtime root created %s", root)
}
}
// The root must land outside the workspace, which is the entire reason this
// branch exists.
func TestFallbackRuntimeRootStaysOutsideTheWorkspace(t *testing.T) {
workspace := filepath.Join(t.TempDir(), "ws")
root, err := fallbackSandboxRuntimeRoot(workspace)
if err != nil {
t.Fatal(err)
}
if pathWithinRoot(workspace, root) {
t.Errorf("runtime root %q is inside workspace %q", root, workspace)
}
if !strings.Contains(root, "zero") {
t.Errorf("runtime root %q does not look like a zero-owned tree", root)
}
}
+214 -27
View File
@@ -7,10 +7,14 @@ import (
"fmt"
"os"
"path/filepath"
"runtime"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/Gitlawb/zero/internal/peermsg"
)
var sandboxUserCacheDir = os.UserCacheDir
@@ -21,11 +25,6 @@ const (
sandboxRuntimeMaxRoots = 64
)
var fallbackSandboxRuntimes = struct {
sync.Mutex
roots map[string]string
}{roots: make(map[string]string)}
type SandboxRuntime struct {
Root string `json:"root,omitempty"`
Cache string `json:"cache,omitempty"`
@@ -33,8 +32,37 @@ type SandboxRuntime struct {
Temp string `json:"temp,omitempty"`
}
// sandboxRuntimeRootFor derives the per-workspace runtime root. It is separated
// from prepareSandboxRuntime because the elevated Windows setup path needs the
// same answer WITHOUT taking a lease or creating anything: a sandbox principal
// is a separate account with no inherited rights under the user cache, so setup
// has to grant it write access to this tree before any command runs.
//
// Both callers must agree exactly. If they ever drift, setup grants the ACE on
// one directory while commands write to another, and the failure is a bare
// ACCESS_DENIED from npm or go build with nothing pointing at the sandbox.
func sandboxRuntimeRootFor(workspaceRoot string, cacheRoot string) (string, error) {
if root, ok := deterministicSandboxRuntimeRoot(workspaceRoot, cacheRoot); ok {
return root, nil
}
return fallbackSandboxRuntimeRoot(workspaceRoot)
}
// deterministicSandboxRuntimeRoot returns the cache-derived runtime root and
// whether it is usable, meaning it lands outside the workspace.
//
// Neither this nor the fallback creates anything now, so a caller that only
// needs to NAME the tree can safely go through sandboxRuntimeRootFor and get the
// answer commands will actually use. This remains separate for callers that need
// to distinguish the cache-derived root from the temp-derived one.
func deterministicSandboxRuntimeRoot(workspaceRoot string, cacheRoot string) (string, bool) {
digest := sha256.Sum256([]byte(workspaceRoot))
root := filepath.Join(cacheRoot, "zero", "runtime", "v1", hex.EncodeToString(digest[:8]))
return root, !pathWithinRoot(workspaceRoot, root)
}
func prepareSandboxRuntime(workspaceRoot string) (SandboxRuntime, func(), error) {
workspaceRoot = filepath.Clean(strings.TrimSpace(workspaceRoot))
workspaceRoot = canonicalSandboxWorkspaceRoot(workspaceRoot)
if workspaceRoot == "" || workspaceRoot == "." {
return SandboxRuntime{}, nil, errors.New("sandbox runtime requires a workspace root")
}
@@ -42,17 +70,30 @@ func prepareSandboxRuntime(workspaceRoot string) (SandboxRuntime, func(), error)
if err != nil {
return SandboxRuntime{}, nil, fmt.Errorf("resolve user cache directory: %w", err)
}
cacheRoot = filepath.Clean(strings.TrimSpace(cacheRoot))
// Canonicalized the SAME way as the workspace root, because
// sandboxRuntimeRootFor compares the two: it falls back to a private temp
// tree when the derived runtime root would land inside the workspace.
// Normalizing only one side made that comparison run on two different
// spellings of the same path — /var vs /private/var on macOS, an 8.3 short
// name vs its long form on Windows — so the containment check missed and the
// fallback never fired.
cacheRoot = canonicalSandboxWorkspaceRoot(cacheRoot)
if cacheRoot == "" || cacheRoot == "." {
return SandboxRuntime{}, nil, errors.New("user cache directory is unavailable")
}
digest := sha256.Sum256([]byte(workspaceRoot))
root := filepath.Join(cacheRoot, "zero", "runtime", "v1", hex.EncodeToString(digest[:8]))
if pathWithinRoot(workspaceRoot, root) {
root, err = fallbackSandboxRuntimeRoot(workspaceRoot)
if err != nil {
return SandboxRuntime{}, nil, err
}
root, err := sandboxRuntimeRootFor(workspaceRoot, cacheRoot)
if err != nil {
return SandboxRuntime{}, nil, err
}
// THE ANCHOR IS PROVEN BEFORE THE TREE IS BUILT, on both paths that can
// hand back a fallback root: sandboxRuntimeRootFor returns one itself when
// the cache lands inside the workspace, and the branch below asks for one
// when the cache root cannot be leased. A deterministic name under the
// shared temp is only safe beneath a directory this user owns; if the anchor
// is a link, or exists and belongs to someone else, this refuses rather than
// leasing, creating and chmod-ing through it.
if err := ensureFallbackRuntimeAnchor(root); err != nil {
return SandboxRuntime{}, nil, err
}
lease, err := prepareSandboxRuntimeLease(root)
if err != nil {
@@ -60,6 +101,9 @@ func prepareSandboxRuntime(workspaceRoot string) (SandboxRuntime, func(), error)
if err != nil {
return SandboxRuntime{}, nil, err
}
if err := ensureFallbackRuntimeAnchor(root); err != nil {
return SandboxRuntime{}, nil, err
}
lease, err = prepareSandboxRuntimeLease(root)
if err != nil {
return SandboxRuntime{}, nil, err
@@ -174,22 +218,109 @@ func combineSandboxCleanups(cleanups ...func()) func() {
}
}
// fallbackRuntimeAnchor is the per-user directory the deterministic fallback
// runtime tree lives beneath.
//
// DETERMINISM IS NOT OWNERSHIP. The fallback moved from a fresh, private
// os.MkdirTemp parent to a predictable name so that elevated setup and the
// later command process agree on the runtime root. That agreement is required
// and must stay. But a predictable path directly under the shared system temp
// is not a private allocation: on an ordinary shared /tmp another local user
// can create /tmp/zero with mode 0700 first, and every affected user then fails
// before the sandbox command starts; a pre-existing redirected component makes
// host-side preparation operate somewhere other than the derived tree.
//
// So the tree is rooted beneath an anchor that names THIS user, and the anchor
// is created and validated by peermsg.EnsurePrivateDir before anything is
// built under it: handle-relative descent, no-follow, refused if any component
// is a link or the leaf is not owned by the current user, mode 0700. On Unix
// the uid is in the name because /tmp is shared between users; on Windows
// os.TempDir is already per-user, and the private descriptor plus the owner
// check do the rest.
func fallbackRuntimeAnchor() string {
tag := "user"
if runtime.GOOS != "windows" {
tag = strconv.Itoa(os.Getuid())
}
// PHYSICAL PARENT, OWNED LEAF. The validator walks every component no-follow
// and refuses links, which is right for the anchor Zero owns and wrong for
// the operator's ancestors above it: macOS puts TempDir under /var, a
// symlink to /private/var, and a redirected %LOCALAPPDATA% is ordinary on
// Windows. The first version handed the validator the unresolved path and
// refused every fallback on every Mac. Resolving the parent first means the
// only new component the validator sees is the anchor itself.
//
// On a resolve error the unresolved path is used, deliberately: the
// validator is still the fail-closed check, and refusing there names the
// real component rather than hiding it behind a resolver failure.
base := os.TempDir()
if physical, err := physicalTempDir(); err == nil && physical != "" {
base = physical
}
return filepath.Join(base, "zero-runtime-"+tag)
}
// isFallbackRuntimeRoot reports whether root was derived by
// fallbackSandboxRuntimeRoot, so the caller knows the anchor must be validated
// before the tree beneath it is prepared. Both candidates a command derives can
// be fallback roots: sandboxRuntimeRootFor returns one itself when the cache
// lands inside the workspace, and prepareSandboxRuntime asks for one when the
// cache root cannot be leased.
func isFallbackRuntimeRoot(root string) bool {
// pathWithinRoot(parent, child): is root beneath the anchor. The first
// draft had these reversed, which asked whether the anchor was beneath the
// root, answered false every time, and skipped the validation entirely
// while every test that depended on it reported success through a
// junction. Argument order is the whole function.
return pathWithinRoot(fallbackRuntimeAnchor(), root)
}
// ensureFallbackRuntimeAnchor validates the anchor beneath which a fallback
// runtime tree is about to be built, and fails closed. Called only when the
// root is a fallback root; the cache-derived root lives under the user cache,
// which is already the user's own directory.
func ensureFallbackRuntimeAnchor(root string) error {
if !isFallbackRuntimeRoot(root) {
return nil
}
anchor := fallbackRuntimeAnchor()
if err := peermsg.EnsurePrivateDir(anchor); err != nil {
return fmt.Errorf("sandbox runtime fallback anchor %s is not a private directory owned by this user: %w", anchor, err)
}
return nil
}
// fallbackSandboxRuntimeRoot returns the runtime root for a workspace whose
// cache-derived root would land inside itself.
//
// DERIVED, not minted, and that is the whole of the fix. It used to call
// os.MkdirTemp and remember the answer in a process-global map, which made the
// result private to whichever process asked first. Elevated Windows setup
// granted the sandbox principal write access to the directory IT created, then
// every later __windows-command-runner process created a DIFFERENT one and
// pointed TMP, GOCACHE, npm and the rest at it. Those directories are created by
// the calling user and carry no ACE for the principal, so ordinary cache and
// temp writes failed with a bare ACCESS_DENIED and nothing naming the sandbox.
//
// sandboxRuntimeRootFor already documents that both callers must agree exactly.
// Hashing the workspace, the same way the cache-derived root does, is what makes
// that true for this branch too: every process reaches the same path without
// having to share any state.
//
// It creates nothing, so deterministicSandboxRuntimeRoot's promise about naming
// a tree without materializing it now holds for the fallback as well.
func fallbackSandboxRuntimeRoot(workspaceRoot string) (string, error) {
fallbackSandboxRuntimes.Lock()
defer fallbackSandboxRuntimes.Unlock()
if root := fallbackSandboxRuntimes.roots[workspaceRoot]; root != "" {
return root, nil
}
parent, err := os.MkdirTemp("", "zero-runtime-")
if err != nil {
return "", fmt.Errorf("create fallback sandbox runtime: %w", err)
}
root := filepath.Join(parent, "runtime")
digest := sha256.Sum256([]byte(workspaceRoot))
root := filepath.Join(fallbackRuntimeAnchor(), "v1", hex.EncodeToString(digest[:8]))
if pathWithinRoot(workspaceRoot, root) {
_ = os.RemoveAll(parent)
return "", fmt.Errorf("fallback sandbox runtime root %q must be outside workspace %q", root, workspaceRoot)
// Both candidates land inside the workspace, so there is nowhere left to
// put a runtime tree the workspace's own policy does not govern. Refused
// rather than pointed somewhere arbitrary: a runtime root inside the
// workspace makes the sandbox's own cache writes indistinguishable from
// the work it is meant to be confining.
return "", fmt.Errorf("sandbox runtime root %q would fall inside workspace %q; "+
"open the workspace somewhere other than the cache or temp directory", root, workspaceRoot)
}
fallbackSandboxRuntimes.roots[workspaceRoot] = root
return root, nil
}
@@ -226,3 +357,59 @@ func permissionProfileWithRuntime(profile PermissionProfile, runtimeState Sandbo
profile.FileSystem.WriteRoots = append(profile.FileSystem.WriteRoots, WritableRoot{Root: runtimeState.Root})
return profile
}
// canonicalSandboxWorkspaceRoot normalizes a workspace root the way
// Engine.resolveCommandDir already does — clean, absolutize, then resolve
// symlinks — so every derivation keyed to a workspace agrees on the string.
//
// The runtime root is a hash of this, and the elevated Windows setup grants the
// principal that tree while commands derive it again. Cleaning alone was not
// enough for the two to agree, and it does not take a symlink for them to
// differ: a path opened in different casing, or through an 8.3 short name (what
// a Windows CI runner's TEMP looks like), resolves to a different spelling.
// Setup then granted one tree and every command used another, so the grant that
// makes npm/go/pip caches writable landed where nothing reads and surfaced as a
// bare ACCESS_DENIED.
//
// Resolution failing is not an error: an unresolvable root still needs a stable
// key, and falling back to the cleaned absolute path is what the command path
// does too.
func canonicalSandboxWorkspaceRoot(root string) string {
cleaned := filepath.Clean(strings.TrimSpace(root))
if cleaned == "" || cleaned == "." {
return ""
}
if !filepath.IsAbs(cleaned) {
if absolute, err := filepath.Abs(cleaned); err == nil {
cleaned = absolute
}
}
// EvalSymlinks fails outright when the LEAF does not exist, which is the
// normal case for a cache or runtime root that has not been created yet. A
// plain call therefore resolved an existing workspace while leaving a
// not-yet-created cache root unresolved, and the two were compared against
// each other — the containment check that decides whether the runtime tree
// must move out of the workspace then ran on /private/var/... versus
// /var/..., missed, and left the tree inside the workspace.
//
// Resolve the longest existing ancestor and re-append the rest, so a path
// normalizes the same way whether or not its final segments exist yet.
remainder := ""
current := cleaned
for {
if resolved, err := filepath.EvalSymlinks(current); err == nil {
if remainder == "" {
return resolved
}
return filepath.Join(resolved, remainder)
}
parent := filepath.Dir(current)
if parent == current {
// Nothing along the path resolved; the cleaned absolute form is the
// best stable key available.
return cleaned
}
remainder = filepath.Join(filepath.Base(current), remainder)
current = parent
}
}
+76
View File
@@ -329,3 +329,79 @@ func TestEngineCommandPlanCarriesManagedRuntime(t *testing.T) {
}
cleanupLease.release()
}
// sandboxRuntimeRootFor compares the workspace root against the derived runtime
// root to decide whether to fall back to a private temp tree. Both sides
// therefore have to be the same spelling of the same path.
//
// Canonicalizing only the workspace root broke this on CI: the workspace
// resolved (/var to /private/var on macOS, an 8.3 short name to its long form
// on Windows) while the cache root kept its original spelling, so the
// containment check compared two different strings, the fallback never fired,
// and the runtime tree was placed inside the workspace it exists to stay out of.
//
// A symlink is the portable way to produce a spelling that only resolution
// reconciles — Clean cannot see through one. Windows refuses to create symlinks
// without privilege, so this skips there; the platforms that CI caught the bug
// on are the ones that run it.
func TestPrepareSandboxRuntimeNormalizesTheCacheRootBeforeComparingIt(t *testing.T) {
workspace := t.TempDir()
link := filepath.Join(t.TempDir(), "workspace-link")
if err := os.Symlink(workspace, link); err != nil {
t.Skipf("cannot create a symlink here: %v", err)
}
// The cache root reaches us spelled through the symlink; the workspace does
// not. Resolved, it is plainly inside the workspace and the fallback must
// fire. Unresolved, the two strings share no prefix and it does not.
original := sandboxUserCacheDir
sandboxUserCacheDir = func() (string, error) { return filepath.Join(link, ".cache"), nil }
t.Cleanup(func() { sandboxUserCacheDir = original })
runtimeState, release, err := prepareSandboxRuntime(workspace)
if err != nil {
t.Fatalf("prepareSandboxRuntime: %v", err)
}
if release != nil {
defer release()
}
// Resolve before comparing. Spelled through the link the runtime root shares
// no textual prefix with the workspace, so an unresolved comparison would
// call it "outside" while it sits physically inside — the test would pass
// against the very bug it exists for.
resolved := runtimeState.Root
if actual, err := filepath.EvalSymlinks(runtimeState.Root); err == nil {
resolved = actual
}
if pathWithinRoot(workspace, resolved) {
t.Fatalf("runtime root %q resolves to %q, inside workspace %q; the containment check did not see through the cache root's spelling", runtimeState.Root, resolved, workspace)
}
}
// A path whose final segments do not exist yet must still normalize the same
// way as one that does. This is the shape macOS CI hit: t.TempDir() sits under
// /var, a symlink to /private/var, and the cache root it derives has not been
// created when it is first normalized. Resolving only the workspace left the
// two sides of the containment check spelled differently.
func TestCanonicalSandboxWorkspaceRootResolvesThroughAMissingLeaf(t *testing.T) {
real := t.TempDir()
link := filepath.Join(t.TempDir(), "link")
if err := os.Symlink(real, link); err != nil {
t.Skipf("cannot create a symlink here: %v", err)
}
existing := canonicalSandboxWorkspaceRoot(link)
if existing != canonicalSandboxWorkspaceRoot(real) {
t.Fatalf("an existing symlinked dir did not resolve: %q vs %q", existing, canonicalSandboxWorkspaceRoot(real))
}
// The leaf, and its parent, do not exist.
missing := filepath.Join(link, ".cache", "zero")
got := canonicalSandboxWorkspaceRoot(missing)
want := filepath.Join(existing, ".cache", "zero")
if got != want {
t.Errorf("missing leaf normalized to %q, want %q — the ancestor was not resolved", got, want)
}
if !pathWithinRoot(existing, got) {
t.Errorf("%q should be inside %q once both are canonical", got, existing)
}
}
+239 -25
View File
@@ -19,6 +19,17 @@ type Scope struct {
workspaceRoot string
readRoots []string
extraRoots []string
// tempReads / tempWrites count how many LIVE temporary grants depend on a
// root, so one holder's cleanup cannot revoke another's access.
//
// Without them a temporary grant was add-then-remove with no notion of who
// still needed it: the second caller to ask for a root already present got a
// NO-OP undo, and the first caller's cleanup removed the root out from under
// it. Two read-only tools in the same parallel batch, both blocked on the
// same directory, is exactly that shape — and read-only tools are precisely
// the ones the batch runs concurrently.
tempReads map[string]int
tempWrites map[string]int
}
// NewScope builds a scope for workspaceRoot plus the given extra roots. The
@@ -81,15 +92,88 @@ func (s *Scope) Add(path string) (string, error) {
}
s.mu.Lock()
defer s.mu.Unlock()
for _, existing := range append([]string{s.workspaceRoot}, s.extraRoots...) {
if pathWithinRoot(existing, root) {
// COVERED BY WHAT. extraRoots holds temporary write roots alongside
// permanent ones, so "something already covers this" was not the question
// worth asking: a permanent grant made over a path a temporary holder
// happened to cover recorded nothing, and vanished the moment that holder
// released. A session-scoped grant outliving the request that prompted it is
// the entire difference between Add and AddTemporaryWrite.
//
// Permanent coverage is looked for FIRST and across every root, because a
// path can be covered twice and the temporary cover must not decide the
// answer when a permanent one is also present.
if s.permanentWriteRootCoversLocked(root) {
return root, nil
}
for _, existing := range s.extraRoots {
// The same root, held temporarily: promote it in place. Its holders'
// releases become no-ops, which is what permanence means here.
if existing == root && s.temporaryWriteLocked(existing) {
delete(s.tempWrites, root)
return root, nil
}
}
// Either uncovered, or covered only by a BROADER temporary root that is
// going to be released. Recording the narrower root in its own right is what
// survives that release.
s.extraRoots = append(s.extraRoots, root)
return root, nil
}
// temporaryWriteLocked reports whether root is held by a temporary write grant
// rather than a session-scoped one. Callers must hold the lock.
func (s *Scope) temporaryWriteLocked(root string) bool {
_, temporary := s.tempWrites[root]
return temporary
}
// permanentWriteRootCoversLocked reports whether root sits under write authority
// that outlives every temporary holder: the workspace root, or a session-scoped
// grant. Coverage by a TEMPORARY write root deliberately does not count —
// it is going to be released, so nothing recorded on the strength of it
// survives. Callers must hold the lock.
//
// One helper rather than a copy per caller because Add, AddRead and
// AddTemporaryRead all turn on the same question, and three copies of "covered,
// but by what" is how one of them ends up answering it differently.
// permanentReadRootCoversLocked reports whether root is already readable by a
// grant that outlives every temporary holder.
//
// SLICE ORDER MUST NOT DECIDE THIS. The old check walked readRoots and stopped at
// the first entry covering the path, so a BROAD TEMPORARY root sitting earlier in
// the slice shadowed a narrower PERMANENT one later — and the caller was handed a
// reference on somebody else's grant when it needed no reference at all. Asking
// the whole question first, over both lists, makes the answer independent of
// insertion order.
//
// Write roots count: a grant that permits writing permits reading the same tree.
func (s *Scope) permanentReadRootCoversLocked(root string) bool {
if s.permanentWriteRootCoversLocked(root) {
return true
}
for _, existing := range s.readRoots {
if pathWithinRoot(existing, root) && !s.temporaryReadLocked(existing) {
return true
}
}
return false
}
func (s *Scope) permanentWriteRootCoversLocked(root string) bool {
for _, existing := range append([]string{s.workspaceRoot}, s.extraRoots...) {
if pathWithinRoot(existing, root) && !s.temporaryWriteLocked(existing) {
return true
}
}
return false
}
// temporaryReadLocked is temporaryWriteLocked for read roots.
func (s *Scope) temporaryReadLocked(root string) bool {
_, temporary := s.tempReads[root]
return temporary
}
// AddRead grants read-only access under path. If the path is already covered by
// a writable root, no separate read root is stored.
func (s *Scope) AddRead(path string) (string, error) {
@@ -99,11 +183,19 @@ func (s *Scope) AddRead(path string) (string, error) {
}
s.mu.Lock()
defer s.mu.Unlock()
if s.writeRootCoversLocked(root) {
// Same rule as Add: coverage that is going to be released is not coverage a
// permanent grant can rely on, whichever side of the read/write boundary it
// sits on. A permanent read covered only by a temporary WRITE root died with
// that root just as surely.
if s.permanentWriteRootCoversLocked(root) {
return root, nil
}
if s.permanentReadRootCoversLocked(root) {
return root, nil
}
for _, existing := range s.readRoots {
if pathWithinRoot(existing, root) {
if existing == root && s.temporaryReadLocked(existing) {
delete(s.tempReads, root)
return root, nil
}
}
@@ -118,16 +210,63 @@ func (s *Scope) AddTemporaryRead(path string) (string, func(), error) {
}
s.mu.Lock()
defer s.mu.Unlock()
if s.writeRootCoversLocked(root) {
// Genuinely permanent — the workspace root, or a session-scoped grant. It
// outlives this reader, so there is nothing to release.
if s.permanentWriteRootCoversLocked(root) {
return root, func() {}, nil
}
for _, existing := range s.readRoots {
if pathWithinRoot(existing, root) {
return root, func() {}, nil
}
// A TEMPORARY write root covering this path is NOT borrowed, and this is the
// one place the distinction bites hardest.
//
// The dependency on that root is real — the write holder's cleanup must not
// revoke this reader, which is the defect the refcount exists to close,
// reached across the read/write boundary. But taking a reference on the
// WRITE root pays for the lifetime with the authority: the reference keeps
// the root in extraRoots, extraRoots is what Roots() feeds validate(), and
// validate() is WRITE authorization. So a reader outliving its writer went on
// writing anywhere under a write grant that had already ended — not merely
// under the path it asked to read.
//
// One reference cannot be both the lifetime and the capability. The lifetime
// is kept below, as a temporary READ root for the path this caller actually
// asked for. readRoots feeds ReadRoots(), which confers no write authority —
// validate() authorises writes from workspaceRoot plus extraRoots, and this
// path never enters that list. So the reader survives its writer with read
// authority and only read authority.
//
// That claim is the written justification for the whole design, so it has to
// stay exhaustive as the accessors change. It has already been wrong once, in
// both directions: an earlier draft said readRoots fed ReadRoots() and nothing
// else while ExtraReadRoots() also read it, and that accessor has since been
// removed as belonging to a different change.
if s.permanentReadRootCoversLocked(root) {
return root, func() {}, nil
}
if s.tempReads == nil {
s.tempReads = map[string]int{}
}
// THE SAME ROOT, ANOTHER HOLDER: one entry, two references. Only an EXACT
// match shares — a narrower request must not, which is the whole point below.
if _, held := s.tempReads[root]; held {
s.tempReads[root]++
return root, oncePerHolder(func() { s.releaseTemporaryRead(root) }), nil
}
// A NARROWER REQUEST RECORDS ITS OWN ROOT, even under a broader temporary
// read that currently covers it.
//
// Borrowing the covering root's reference kept this reader alive by keeping
// the BROAD ROOT alive, and readRoots feeds validateRead and the native
// sandbox profile. So once the broad reader released, a caller that had asked
// to read one subdirectory was left able to read the whole tree its neighbour
// had asked for, siblings included. Reported by @jatmn, and it is the read
// twin of the write escalation fixed above: the refcount key decides not only
// WHEN an entry disappears but WHICH path stays authorised while it lives.
s.readRoots = append(s.readRoots, root)
return root, func() { s.removeReadRoot(root) }, nil
if s.tempReads == nil {
s.tempReads = map[string]int{}
}
s.tempReads[root] = 1
return root, oncePerHolder(func() { s.releaseTemporaryRead(root) }), nil
}
func (s *Scope) AddTemporaryWrite(path string) (string, func(), error) {
@@ -137,32 +276,107 @@ func (s *Scope) AddTemporaryWrite(path string) (string, func(), error) {
}
s.mu.Lock()
defer s.mu.Unlock()
if s.writeRootCoversLocked(root) {
// A PERMANENT grant is the only cover that needs no bookkeeping: it outlives
// every temporary holder, so there is nothing to keep alive and nothing to
// release.
if s.permanentWriteRootCoversLocked(root) {
return root, func() {}, nil
}
s.extraRoots = append(s.extraRoots, root)
return root, func() { s.removeWriteRoot(root) }, nil
}
func (s *Scope) writeRootCoversLocked(root string) bool {
for _, existing := range append([]string{s.workspaceRoot}, s.extraRoots...) {
if pathWithinRoot(existing, root) {
return true
}
if s.tempWrites == nil {
s.tempWrites = map[string]int{}
}
return false
// THE SAME ROOT, ANOTHER HOLDER: one entry, two references. Only an exact
// match shares an entry — a NARROWER request must not, which is the whole
// point below.
if _, held := s.tempWrites[root]; held {
s.tempWrites[root]++
return root, oncePerHolder(func() { s.releaseTemporaryWrite(root) }), nil
}
// A NARROWER REQUEST RECORDS ITS OWN ROOT, even when a broader temporary
// grant currently covers it.
//
// Borrowing a reference on the covering root kept this holder alive, which
// was the point, but it kept the COVERING ROOT alive to do it — and
// extraRoots is what validate() authorises writes from. So once the broad
// holder released, a caller that had asked to write one subdirectory was
// left holding write authority over the whole tree its neighbour had asked
// for, including siblings it never named. Reported by CodeRabbit, and it is
// the write-side twin of the read-side escalation @jatmn found: one
// reference cannot be both a lifetime and a capability, whichever side of
// the read/write boundary it sits on.
//
// Recording the requested root separately gives each holder exactly the
// authority it asked for and a lifetime of its own. The nesting costs an
// extra entry in extraRoots while both are live, which is redundant but not
// wrong — the broader root already permits everything the narrower one does.
s.extraRoots = append(s.extraRoots, root)
s.tempWrites[root] = 1
return root, oncePerHolder(func() { s.releaseTemporaryWrite(root) }), nil
}
func (s *Scope) removeReadRoot(root string) {
// releaseTemporaryRead drops ONE holder's reference and removes the root only
// when the last one is gone.
//
// The count alone cannot make this safe, and an earlier comment here claimed it
// could — that each undo is called exactly once, and flooring at zero handled
// the rest. Flooring stops the count going negative; it does not stop one
// holder's second call consuming a DIFFERENT holder's reference. With two
// readers, calling the first's undo twice took the count 2 -> 1 -> 0 and removed
// a root the second was still using. Idempotency belongs to the closure, which
// is the thing that knows whose reference it is, so every undo handed out is
// wrapped in oncePerHolder.
func (s *Scope) releaseTemporaryRead(root string) {
s.mu.Lock()
defer s.mu.Unlock()
remaining, tracked := s.tempReads[root]
if !tracked {
s.mu.Unlock()
return
}
remaining--
if remaining > 0 {
s.tempReads[root] = remaining
s.mu.Unlock()
return
}
// Both mutations under ONE hold. Dropping the lock between them opened a
// window where the root was still in readRoots but no longer in tempReads:
// a concurrent AddTemporaryRead landing there reads it as a PERMANENT root,
// hands its caller a no-op undo, and then this call strips the root — so
// that caller believes it holds access it has already silently lost.
delete(s.tempReads, root)
s.readRoots = removeScopeRoot(s.readRoots, root)
s.mu.Unlock()
}
func (s *Scope) removeWriteRoot(root string) {
// releaseTemporaryWrite is releaseTemporaryRead for write roots. Two functions
// rather than one generic helper because they guard different slices and the
// generic version would take the slice by name — which is how the wrong one
// gets passed.
func (s *Scope) releaseTemporaryWrite(root string) {
s.mu.Lock()
defer s.mu.Unlock()
remaining, tracked := s.tempWrites[root]
if !tracked {
s.mu.Unlock()
return
}
remaining--
if remaining > 0 {
s.tempWrites[root] = remaining
s.mu.Unlock()
return
}
// Same single-hold rule as releaseTemporaryRead above.
delete(s.tempWrites, root)
s.extraRoots = removeScopeRoot(s.extraRoots, root)
s.mu.Unlock()
}
// oncePerHolder makes one holder's undo safe to call more than once. The
// reference it drops is that holder's own, so a second call must do nothing
// rather than reach into the shared count and take somebody else's.
func oncePerHolder(release func()) func() {
var once sync.Once
return func() { once.Do(release) }
}
func removeScopeRoot(roots []string, root string) []string {
+56
View File
@@ -0,0 +1,56 @@
package sandbox
import (
"path/filepath"
"testing"
)
// THE ANTI-ESCALATION PROPERTY. A read grant (AddRead, where --add-read-dir lands)
// is READABLE but NOT WRITABLE. This is what makes propagating a parent's
// request_permissions read grant to a child safe: the child can audit the granted
// path but cannot modify it. Emitting the grant as a WRITE root (--add-dir) would
// break exactly this.
func TestAReadGrantIsReadableButNotWritable(t *testing.T) {
scope, granted := grantOutsideDefaults(t)
root, err := scope.AddRead(granted)
if err != nil {
t.Fatalf("AddRead: %v", err)
}
target := filepath.Join(root, "audit-me.go")
if block := scope.validateRead(target); block != nil {
t.Fatalf("a read grant did not allow READING %q: %v — the audit would fail 'outside the workspace'", target, block)
}
if block := scope.validate(target); block == nil {
t.Fatalf("a read grant ALLOWED WRITING %q — a read grant escalated to write", target)
}
}
// grantOutsideDefaults returns a scope and a directory that scope does not
// already cover, both under test-owned storage.
//
// THE CAUSE, NOT THE SYMPTOM. NewScope seeds the system temporary directory as a
// permanent write root, so a plain t.TempDir() grant is writable before any
// grant exists and the anti-escalation assertions below prove nothing. The first
// fix placed the fixture under os.UserHomeDir() to find ground the defaults do
// not own. That worked and bought two problems: a hermetic runner may expose
// HOME read-only, so these tests failed at MkdirTemp before asserting anything,
// and on an ordinary machine they wrote into the developer's real home and left
// zero-scope-* debris whenever a run was interrupted. Reported by @jatmn.
//
// Building the Scope directly removes the seeded defaults, which is what made
// HOME necessary in the first place. t.TempDir() is then outside by
// construction, the fixture is hermetic, and nothing touches HOME.
func grantOutsideDefaults(t *testing.T) (*Scope, string) {
t.Helper()
workspace := resolvedFixturePath(t, t.TempDir())
granted := resolvedFixturePath(t, t.TempDir())
scope := &Scope{workspaceRoot: workspace}
// PROVED AGAINST THIS SCOPE. The point of the helper is that the grant is
// what changes access, so the target has to begin unauthorised — asserted
// here once rather than trusted in every caller.
if scope.validate(filepath.Join(granted, "probe.txt")) == nil {
t.Fatalf("%s is writable before any grant exists; the assertions below would prove nothing", granted)
}
return scope, granted
}
@@ -0,0 +1,595 @@
package sandbox
import (
"os"
"path/filepath"
"slices"
"sync"
"testing"
)
// scopeOutsideDefaults builds a Scope directly instead of through NewScope.
//
// NewScope adds the system temp directory as a write root, so a target under
// t.TempDir() is already covered and AddTemporaryRead returns before it ever
// touches the refcount. A test built that way exercises none of this and passes
// against the bug — which is how the first version of this test passed.
func scopeOutsideDefaults(t *testing.T) (*Scope, string) {
t.Helper()
workspace := t.TempDir()
target := filepath.Join(t.TempDir(), "shared")
if err := os.MkdirAll(target, 0o700); err != nil {
t.Fatalf("mkdir target: %v", err)
}
// Built directly, NOT via NewScope: real directories are needed because
// normalizeScopeRoot requires them to exist, but NewScope's default temp
// write root would then cover the target and short-circuit the path.
return &Scope{workspaceRoot: workspace}, target
}
// The refcount and the root list have to move together.
//
// Release used to delete the refcount, drop the lock, then strip the root in a
// second acquisition. In that window the root was still in readRoots but no
// longer in tempReads, so a concurrent AddTemporaryRead read it as a PERMANENT
// root and handed its caller a no-op undo — then the release stripped it and
// that caller silently lost access it believed it held.
func TestReleasingATemporaryReadCannotStripALiveGrant(t *testing.T) {
scope, target := scopeOutsideDefaults(t)
// First holder establishes the root.
first, releaseFirst, err := scope.AddTemporaryRead(target)
if err != nil {
t.Fatalf("AddTemporaryRead: %v", err)
}
if len(scope.tempReads) == 0 {
t.Fatal("precondition: the grant should be refcounted, not covered by a default root")
}
// Second holder takes a reference on the same root.
second, releaseSecond, err := scope.AddTemporaryRead(target)
if err != nil {
t.Fatalf("AddTemporaryRead (second): %v", err)
}
// The first holder leaving must not revoke the second's access.
releaseFirst()
if block := scope.validateRead(second); block != nil {
t.Fatalf("the second holder lost its grant when the first released: %v", block)
}
// Only the last release retires the root.
releaseSecond()
if block := scope.validateRead(first); block == nil {
t.Error("the root outlived its last holder")
}
}
// Under contention the same invariant has to hold: while a grant is live, its
// root is readable.
func TestTemporaryReadGrantsSurviveConcurrentReleases(t *testing.T) {
scope, target := scopeOutsideDefaults(t)
var wg sync.WaitGroup
var mu sync.Mutex
var failures []string
for i := 0; i < 24; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < 200; j++ {
granted, undo, err := scope.AddTemporaryRead(target)
if err != nil {
mu.Lock()
failures = append(failures, "AddTemporaryRead: "+err.Error())
mu.Unlock()
return
}
if block := scope.validateRead(granted); block != nil {
mu.Lock()
failures = append(failures, "root not readable while a grant was live")
mu.Unlock()
}
undo()
}
}()
}
wg.Wait()
if len(failures) > 0 {
t.Fatalf("%d failures, first: %s", len(failures), failures[0])
}
}
// A read covered by a TEMPORARY WRITE root must take a reference too.
//
// permanentWriteRootCoversLocked scans extraRoots, and AddTemporaryWrite appends
// temporary write roots there — so "covered by a write root" does not mean
// "covered permanently". Without a reference the write holder's cleanup silently
// revoked the reader's access: the same defect the refcount closes, reached
// across the read/write boundary instead of within one side of it.
func TestAReadCoveredByATemporaryWriteSurvivesItsRelease(t *testing.T) {
workspace := t.TempDir()
outer := filepath.Join(t.TempDir(), "outer")
inner := filepath.Join(outer, "inner")
if err := os.MkdirAll(inner, 0o700); err != nil {
t.Fatal(err)
}
// Built directly for the same reason scopeOutsideDefaults is: NewScope's
// default temp write root would cover the fixture and short-circuit this.
scope := &Scope{workspaceRoot: workspace}
_, releaseWrite, err := scope.AddTemporaryWrite(outer)
if err != nil {
t.Fatalf("temporary write: %v", err)
}
readRoot, releaseRead, err := scope.AddTemporaryRead(inner)
if err != nil {
t.Fatalf("temporary read: %v", err)
}
covered := func() bool {
for _, root := range scope.ReadRoots() {
if pathWithinRoot(root, readRoot) {
return true
}
}
return false
}
if !covered() {
t.Fatal("the reader did not hold access before any release")
}
// The WRITE holder finishes first; the reader has not released.
releaseWrite()
if !covered() {
t.Errorf("the write holder's cleanup revoked a live read grant on %s", readRoot)
}
// And once the reader releases too, the root is genuinely gone.
releaseRead()
if covered() {
t.Errorf("the root outlived its last holder: %v", scope.ReadRoots())
}
}
func hasExtraRoot(scope *Scope, root string) bool {
for _, existing := range writeRootsBeyondWorkspace(scope) {
if existing == root {
return true
}
}
return false
}
// A PERMANENT GRANT MUST OUTLIVE THE TEMPORARY ONE IT LANDED ON TOP OF.
//
// extraRoots holds temporary write roots alongside permanent ones, so Add asked
// only "is this already covered" — and a session-scoped grant made while a
// temporary holder happened to cover the path recorded nothing, then vanished
// when that holder released. Outliving the request that prompted it is the
// entire difference between Add and AddTemporaryWrite.
func TestAPermanentGrantSurvivesTheTemporaryOneItCovered(t *testing.T) {
t.Run("write, same root", func(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
_, releaseTemp, err := scope.AddTemporaryWrite(outside)
if err != nil {
t.Fatal(err)
}
if _, err := scope.Add(outside); err != nil {
t.Fatal(err)
}
releaseTemp()
if !hasExtraRoot(scope, outside) {
t.Error("the temporary holder's release revoked a permanent write grant")
}
})
t.Run("read, same root", func(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
_, releaseTemp, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
if _, err := scope.AddRead(outside); err != nil {
t.Fatal(err)
}
releaseTemp()
if !hasReadRoot(scope, outside) {
t.Error("the temporary holder's release revoked a permanent read grant")
}
})
// The nested shape: a BROAD temporary root covering a NARROW permanent one.
// Promoting in place cannot help here — the narrow root has to be recorded
// in its own right, or it goes when the broad one does.
t.Run("broad temporary over narrow permanent", func(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
inner := filepath.Join(outside, "inner")
if err := os.MkdirAll(inner, 0o700); err != nil {
t.Fatal(err)
}
_, releaseBroad, err := scope.AddTemporaryWrite(outside)
if err != nil {
t.Fatal(err)
}
if _, err := scope.Add(inner); err != nil {
t.Fatal(err)
}
releaseBroad()
covered := false
for _, existing := range writeRootsBeyondWorkspace(scope) {
if pathWithinRoot(existing, inner) {
covered = true
}
}
if !covered {
t.Error("a narrower permanent grant died with the broader temporary root it sat under")
}
})
// THE READ MIRROR OF BOTH SHAPES ABOVE. AddRead makes two claims that the
// subtests so far only made for Add: the loop over extraRoots skips coverage
// that is a temporary WRITE, and the loop over readRoots skips coverage that
// is a temporary READ. Each claim is a separate branch, and neither was
// exercised from the read side.
t.Run("permanent read under a temporary write", func(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
_, releaseWrite, err := scope.AddTemporaryWrite(outside)
if err != nil {
t.Fatal(err)
}
if _, err := scope.AddRead(outside); err != nil {
t.Fatal(err)
}
releaseWrite()
if !hasReadRoot(scope, outside) {
t.Error("a temporary write holder's release revoked a permanent read grant")
}
})
t.Run("broad temporary read over narrow permanent read", func(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
inner := filepath.Join(outside, "inner")
if err := os.MkdirAll(inner, 0o700); err != nil {
t.Fatal(err)
}
_, releaseBroad, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
if _, err := scope.AddRead(inner); err != nil {
t.Fatal(err)
}
releaseBroad()
if block := scope.validateRead(filepath.Join(inner, "audit-me.go")); block != nil {
t.Errorf("a narrower permanent read died with the broader temporary read it sat under: %v", block)
}
})
}
// ONE HOLDER'S UNDO DROPS ONE HOLDER'S REFERENCE, HOWEVER OFTEN IT IS CALLED.
//
// The count flooring at zero stops it going negative; it does not stop a
// holder's second call consuming somebody else's reference. With two readers,
// calling the first's undo twice took the count 2 -> 1 -> 0 and removed a root
// the second was still using — the exact revocation this refcount exists to
// prevent, reached through a duplicate call rather than a sibling's cleanup.
// Deferred cleanups in a retry path are how a real caller does this by accident.
func TestOneHoldersUndoIsIdempotent(t *testing.T) {
scope, workspace, outside := scopeOutsideRoots(t)
_, undoFirst, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
_, undoSecond, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
undoFirst()
undoFirst()
undoFirst()
if !hasReadRoot(scope, outside) {
t.Fatal("repeated calls to one holder's undo revoked a live holder's access")
}
undoSecond()
if hasReadRoot(scope, outside) {
t.Error("the root outlived its last holder")
}
// The write side takes the same rule. Built directly for the same reason the
// helper is: NewScope would seed the system temp directory as a permanent
// write root, which already covers this fixture and makes the grant vacuous.
writeScope := &Scope{workspaceRoot: workspace}
_, undoWriteFirst, err := writeScope.AddTemporaryWrite(outside)
if err != nil {
t.Fatal(err)
}
_, undoWriteSecond, err := writeScope.AddTemporaryWrite(outside)
if err != nil {
t.Fatal(err)
}
undoWriteFirst()
undoWriteFirst()
if !hasExtraRoot(writeScope, outside) {
t.Fatal("repeated calls to one write holder's undo revoked a live holder's access")
}
undoWriteSecond()
if hasExtraRoot(writeScope, outside) {
t.Error("the write root outlived its last holder")
}
}
// temporaryWriteFixture builds a nested pair of real directories and a scope
// that covers neither: an OUTER directory a temporary write grant will take,
// and an INNER one below it that a temporary read grant will ask for.
//
// Built directly rather than through NewScope, for the reason
// scopeOutsideDefaults documents: NewScope seeds the system temp directory as a
// PERMANENT write root, which would cover the whole fixture and send every
// branch under test down the "genuinely permanent" path instead. Both paths are
// returned symlink-resolved because normalizeScopeRoot resolves what it stores,
// and on macOS the fixture's /var spelling is a symlink to /private/var.
func temporaryWriteFixture(t *testing.T) (scope *Scope, workspace, outer, inner string) {
t.Helper()
outer = filepath.Join(t.TempDir(), "outer")
inner = filepath.Join(outer, "inner")
if err := os.MkdirAll(inner, 0o700); err != nil {
t.Fatal(err)
}
workspace = resolvedFixturePath(t, t.TempDir())
return &Scope{workspaceRoot: workspace}, workspace, resolvedFixturePath(t, outer), resolvedFixturePath(t, inner)
}
func resolvedFixturePath(t *testing.T, path string) string {
t.Helper()
resolved, err := filepath.EvalSymlinks(path)
if err != nil {
t.Fatalf("resolve fixture %s: %v", path, err)
}
return resolved
}
// A READER MUST NOT INHERIT ITS WRITER'S AUTHORITY.
//
// The reader's dependency on a covering TEMPORARY WRITE root was recorded as a
// reference on that root — one reference standing for both the lifetime and the
// capability. Release the WRITER first and the reference kept the path in
// extraRoots, which is the list Roots() feeds validate(), so the surviving
// read-only holder could WRITE anywhere below a write grant that had already
// ended — including paths it never asked to read.
//
// This is TestAReadGrantIsReadableButNotWritable's property for temporary
// grants: read authority stays read authority, whichever holder leaves first.
func TestAReaderOutlivingItsWriterKeepsReadAuthorityOnly(t *testing.T) {
t.Run("the writer releases first", func(t *testing.T) {
scope, workspace, outer, inner := temporaryWriteFixture(t)
writeRoot, releaseWrite, err := scope.AddTemporaryWrite(outer)
if err != nil {
t.Fatalf("temporary write: %v", err)
}
if writeRoot != outer {
t.Fatalf("temporary write granted %q, want %q", writeRoot, outer)
}
readRoot, releaseRead, err := scope.AddTemporaryRead(inner)
if err != nil {
t.Fatalf("temporary read: %v", err)
}
if readRoot != inner {
t.Fatalf("temporary read granted %q, want %q", readRoot, inner)
}
// The reader holds a read root of its own and NO reference on the write
// root. Both halves matter: the first is what survives the writer, the
// second is what stops the write root surviving with it.
//
// Errorf, not Fatalf: these say WHY the behavior below breaks, and a
// bookkeeping check that aborts the run hides the behavior it explains.
scope.mu.RLock()
writeHolders := scope.tempWrites[outer]
readHolders := scope.tempReads[inner]
scope.mu.RUnlock()
if writeHolders != 1 {
t.Errorf("tempWrites[%s] = %d, want 1: the reader took a reference on the WRITE root", outer, writeHolders)
}
if readHolders != 1 {
t.Errorf("tempReads[%s] = %d, want 1: the reader kept no read root of its own", inner, readHolders)
}
// The WRITE holder finishes; only the read-only holder is left.
releaseWrite()
if got := writeRootsBeyondWorkspace(scope); len(got) != 0 {
t.Errorf("ExtraRoots() = %v, want none: the reader held the write root open", got)
}
if got, want := scope.ReadRoots(), []string{workspace, inner}; !slices.Equal(got, want) {
t.Errorf("ReadRoots() = %v, want %v", got, want)
}
target := filepath.Join(inner, "audit-me.go")
if block := scope.validateRead(target); block != nil {
t.Errorf("the writer's cleanup revoked a live read grant on %q: %v", target, block)
}
block := scope.validate(target)
if block == nil {
t.Fatalf("a read-only holder may still WRITE %q after the write grant ended", target)
}
if block.Code != BlockOutsideWorkspace {
t.Errorf("write block code = %q, want %q", block.Code, BlockOutsideWorkspace)
}
if block.Path != target {
t.Errorf("write block path = %q, want %q", block.Path, target)
}
// The escalation was never confined to the path the reader asked for —
// a borrowed reference carries the whole covering root — so a sibling
// the reader never named has to be denied too.
sibling := filepath.Join(outer, "sibling.txt")
if block := scope.validate(sibling); block == nil {
t.Errorf("a read-only holder of %q may still WRITE %q, which it never asked to read", inner, sibling)
}
releaseRead()
if got, want := scope.ReadRoots(), []string{workspace}; !slices.Equal(got, want) {
t.Errorf("ReadRoots() = %v, want %v: the read root outlived its last holder", got, want)
}
})
// A CONTROL, NOT COVERAGE FOR THE FINDING. This subtest passes on the
// unfixed tree too — the escalation needs the WRITER to go first, which the
// subtest above exercises. It earns its place by pinning that the opposite
// release order was not broken while fixing the dangerous one, and it is
// labelled so nobody counts it as evidence the finding is closed.
t.Run("the reader releases first (control: passes unfixed)", func(t *testing.T) {
scope, _, outer, inner := temporaryWriteFixture(t)
_, releaseWrite, err := scope.AddTemporaryWrite(outer)
if err != nil {
t.Fatalf("temporary write: %v", err)
}
_, releaseRead, err := scope.AddTemporaryRead(inner)
if err != nil {
t.Fatalf("temporary read: %v", err)
}
// The other order, which the fix must not break: the writer keeps FULL
// authority for as long as it holds the grant.
releaseRead()
buildLog := filepath.Join(outer, "build.log")
if block := scope.validate(buildLog); block != nil {
t.Fatalf("the reader's cleanup revoked the live write grant on %q: %v", buildLog, block)
}
if got, want := writeRootsBeyondWorkspace(scope), []string{outer}; !slices.Equal(got, want) {
t.Errorf("ExtraRoots() = %v, want %v", got, want)
}
releaseWrite()
if block := scope.validate(buildLog); block == nil {
t.Errorf("the write root %q outlived its only holder", outer)
}
if block := scope.validateRead(filepath.Join(inner, "audit-me.go")); block == nil {
t.Errorf("the read root %q outlived its only holder", inner)
}
})
}
// A NARROWER WRITE HOLDER MUST NOT INHERIT ITS NEIGHBOUR'S TREE.
//
// The write-side twin of the read-side escalation above, and the same root
// cause: a nested holder used to take a reference on the BROADER temporary root
// that happened to cover it. That kept the nested holder alive, which was the
// point, but it kept the covering root alive to do it — and extraRoots is what
// validate() authorises writes from. Once the broad holder released, a caller
// that had asked for one subdirectory held write authority over the whole tree,
// siblings included. Reported by CodeRabbit.
func TestANestedWriteHolderKeepsOnlyItsOwnSubtree(t *testing.T) {
scope, _, outsideBase := scopeOutsideRoots(t)
outer := filepath.Join(outsideBase, "outer")
inner := filepath.Join(outer, "inner")
sibling := filepath.Join(outer, "sibling")
for _, dir := range []string{outer, inner, sibling} {
if err := os.MkdirAll(dir, 0o700); err != nil {
t.Fatal(err)
}
}
_, releaseOuter, err := scope.AddTemporaryWrite(outer)
if err != nil {
t.Fatalf("temporary write on the broad root: %v", err)
}
_, releaseInner, err := scope.AddTemporaryWrite(inner)
if err != nil {
t.Fatalf("temporary write on the nested root: %v", err)
}
// THE BROAD HOLDER GOES FIRST — the nested holder outliving it is what
// exposes whose authority it is actually holding.
releaseOuter()
if block := scope.validate(filepath.Join(inner, "mine.txt")); block != nil {
t.Errorf("the nested holder lost the subtree it asked for: %v", block.Reason)
}
if block := scope.validate(filepath.Join(sibling, "not-mine.txt")); block == nil {
t.Error("the nested holder can write a SIBLING it never asked for: the broad holder's authority outlived the broad holder")
}
if block := scope.validate(filepath.Join(outer, "not-mine.txt")); block == nil {
t.Error("the nested holder can write the covering root itself after its holder released")
}
releaseInner()
if block := scope.validate(filepath.Join(inner, "mine.txt")); block == nil {
t.Error("the nested root outlived its only holder")
}
}
// The opposite order, as a control: the nested holder releasing first must not
// disturb the broad holder that is still live.
func TestANestedWriteHolderReleasingFirstLeavesTheBroadGrant(t *testing.T) {
scope, _, outsideBase := scopeOutsideRoots(t)
outer := filepath.Join(outsideBase, "outer")
inner := filepath.Join(outer, "inner")
for _, dir := range []string{outer, inner} {
if err := os.MkdirAll(dir, 0o700); err != nil {
t.Fatal(err)
}
}
_, releaseOuter, err := scope.AddTemporaryWrite(outer)
if err != nil {
t.Fatal(err)
}
_, releaseInner, err := scope.AddTemporaryWrite(inner)
if err != nil {
t.Fatal(err)
}
releaseInner()
if block := scope.validate(filepath.Join(inner, "still-ok.txt")); block != nil {
t.Errorf("the nested holder's release revoked the broad holder's coverage: %v", block.Reason)
}
releaseOuter()
if block := scope.validate(filepath.Join(inner, "gone.txt")); block == nil {
t.Error("the tree outlived every holder")
}
}
// TWO HOLDERS OF THE SAME ROOT SHARE ONE ENTRY. Only an exact match shares —
// a narrower request must not, which is what the escalation test above pins.
func TestTwoHoldersOfTheSameWriteRootShareOneEntry(t *testing.T) {
scope, _, outsideBase := scopeOutsideRoots(t)
target := filepath.Join(outsideBase, "shared")
if err := os.MkdirAll(target, 0o700); err != nil {
t.Fatal(err)
}
_, releaseFirst, err := scope.AddTemporaryWrite(target)
if err != nil {
t.Fatal(err)
}
_, releaseSecond, err := scope.AddTemporaryWrite(target)
if err != nil {
t.Fatal(err)
}
releaseFirst()
if block := scope.validate(filepath.Join(target, "still-held.txt")); block != nil {
t.Errorf("one holder's release revoked the other's grant on the same root: %v", block.Reason)
}
releaseSecond()
if block := scope.validate(filepath.Join(target, "gone.txt")); block == nil {
t.Error("the shared root outlived both holders")
}
}
// writeRootsBeyondWorkspace is Roots() without the workspace entry.
//
// These tests need to see which BEYOND-WORKSPACE write roots a scope currently
// holds. There was an ExtraRoots() accessor that answered exactly this, but it
// had no production caller — it belongs to #829's child-grant propagation, whose
// consumer side is not on this branch, and @jatmn was right that carrying it here
// leaves unrelated production surface in a PR scoped to refcounting. The
// inspection is a test concern, so it lives in the tests.
func writeRootsBeyondWorkspace(scope *Scope) []string {
all := scope.Roots()
if len(all) == 0 {
return nil
}
return append([]string(nil), all[1:]...)
}
+366
View File
@@ -0,0 +1,366 @@
package sandbox
import (
"os"
"path/filepath"
"sync"
"testing"
)
// scopeOutsideRoots builds a workspace and an unrelated directory that no
// default write root already covers.
//
// NOT t.TempDir(). /tmp and $TMPDIR are default write roots, so a temporary
// grant for a path under one is a no-op — every assertion here would pass
// against code that grants nothing at all. That is RULES §2.3 in its exact
// form, and it caught the first version of this test.
// scopeOutsideRoots builds a workspace and a directory outside it, both under
// test-owned storage, and returns a Scope that treats the second as genuinely
// outside.
//
// THE FIXTURE PROBLEM AND ITS ACTUAL CAUSE. NewScope seeds the system temporary
// directory as a permanent write root, so an ordinary t.TempDir() grant is
// covered before any temporary grant exists and every assertion built on it is
// vacuous. The first fix for that reached into os.UserHomeDir() to find ground
// the defaults do not already own — which worked, and bought two new problems:
// a hermetic runner may expose HOME read-only, so the tests failed at MkdirTemp
// before asserting anything; and on an ordinary machine they wrote fixtures into
// the developer's real home and left zeromax-scope-* debris behind whenever a
// run was interrupted. Reported by @jatmn.
//
// The cause is the seeded defaults, not the location, so the Scope is built
// DIRECTLY here — the same pattern scopeOutsideDefaults and temporaryWriteFixture
// already use. With no defaults seeded, t.TempDir() is outside by construction,
// the fixture is hermetic, and nothing touches HOME.
//
// Paths are symlink-resolved because the scope stores resolved roots and macOS
// hands out /var/... temp directories that resolve to /private/var/...; an
// unresolved fixture path compares unequal to the root the scope just recorded.
func scopeOutsideRoots(t *testing.T) (scope *Scope, workspace string, outside string) {
t.Helper()
workspace = resolvedFixturePath(t, t.TempDir())
outside = filepath.Join(resolvedFixturePath(t, t.TempDir()), "outside")
// FATAL, NOT SKIP. Every caller of this helper asserts an AUTHORIZATION
// boundary, so a skipped fixture means that assertion never ran and the build
// reported green anyway. t.TempDir() already failed the test if it could not
// provide storage, so a subdirectory failing under it is a broken machine.
if err := os.MkdirAll(outside, 0o700); err != nil {
t.Fatalf("cannot prepare %s: %v", outside, err)
}
scope = &Scope{workspaceRoot: workspace}
// PROVED AGAINST THIS SCOPE, not against production's default list. The point
// of the helper is that a grant here changes something, and the only thing
// that can answer that is the scope the test will actually use. Checking
// defaultTempWriteRoots() instead asks about a scope nobody builds here — and
// says "prove nothing" for a t.TempDir() path that this scope, seeded with no
// defaults, genuinely does not cover.
//
// Asserting the NEGATIVE up front is also what keeps every caller honest: it
// establishes that the target begins unauthorised, so a later success is the
// grant's doing and not the fixture's placement.
if scope.validate(filepath.Join(outside, "probe.txt")) == nil {
t.Fatalf("%s is writable before any grant exists, so a temporary grant here would prove nothing", outside)
}
if scope.validateRead(filepath.Join(outside, "probe.txt")) == nil {
t.Fatalf("%s is readable before any grant exists, so a temporary read grant here would prove nothing", outside)
}
return scope, workspace, outside
}
func hasReadRoot(scope *Scope, root string) bool {
for _, existing := range scope.ReadRoots() {
if existing == root {
return true
}
}
return false
}
// THE DEFECT: one holder's cleanup revoked another's access. Two read-only
// tools in the same parallel batch, both blocked on the same directory, is
// exactly this — and read-only tools are the ones the batch runs concurrently.
func TestATemporaryReadSurvivesASiblingsCleanup(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
// The probe that the fix must make unnecessary: without it this reads
// true -> false.
if hasReadRoot(scope, outside) {
t.Fatal("the outside root is already covered; this test proves nothing")
}
_, undoA, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
_, undoB, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
if !hasReadRoot(scope, outside) {
t.Fatal("the grant did not take effect")
}
undoA()
if !hasReadRoot(scope, outside) {
t.Fatal("A's cleanup revoked the root while B still held a grant")
}
undoB()
if hasReadRoot(scope, outside) {
t.Fatal("the root outlived its last holder")
}
}
// A NARROWER REQUEST KEEPS ITS OWN SUBTREE AND NOTHING WIDER.
//
// THIS TEST USED TO ASSERT THE OPPOSITE, and the earlier version is worth
// recording because it is how the escalation survived a review round. It was
// written to prove a LIFETIME property — the broad holder's cleanup must not
// revoke a live narrow holder — and it proved that by asserting the BROAD root
// was still present afterwards. That is the defect stated as a requirement: the
// narrow holder was keeping its neighbour's whole tree alive in order to keep
// itself alive, so the surviving reader could read siblings it never asked for.
//
// Lifetime and capability are two properties and one reference cannot carry
// both. The assertions below name them separately: the requested path stays
// readable while its holder lives, AND a sibling under the released broad root
// is denied as soon as the holder that authorised it exits. Reported by @jatmn.
func TestANarrowerReaderKeepsOnlyItsOwnSubtree(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
nested := filepath.Join(outside, "nested")
sibling := filepath.Join(outside, "sibling")
for _, dir := range []string{nested, sibling} {
// FATAL, NOT SKIP. These directories are the fixture, not a precondition
// the environment might reasonably withhold — skipping on them reports a
// broken setup as a passing run.
if err := os.MkdirAll(dir, 0o700); err != nil {
t.Fatal(err)
}
}
_, undoBroad, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
_, undoNarrow, err := scope.AddTemporaryRead(nested)
if err != nil {
t.Fatal(err)
}
// THE BROAD HOLDER GOES FIRST. The narrow holder outliving it is what exposes
// whose authority it is actually holding.
undoBroad()
if block := scope.validateRead(filepath.Join(nested, "mine.txt")); block != nil {
t.Errorf("the broad holder's cleanup revoked the subtree the narrow holder asked for: %v", block.Reason)
}
if block := scope.validateRead(filepath.Join(sibling, "not-mine.txt")); block == nil {
t.Error("the narrow reader can read a SIBLING it never asked for: the broad holder's authority outlived the broad holder")
}
if block := scope.validateRead(filepath.Join(outside, "not-mine.txt")); block == nil {
t.Error("the narrow reader can read the covering root itself after its holder released")
}
undoNarrow()
if block := scope.validateRead(filepath.Join(nested, "mine.txt")); block == nil {
t.Error("the nested root outlived its only holder")
}
}
// The opposite order, as a control: the narrow reader releasing first must not
// disturb the broad reader that is still live.
func TestANarrowerReaderReleasingFirstLeavesTheBroadGrant(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
nested := filepath.Join(outside, "nested")
if err := os.MkdirAll(nested, 0o700); err != nil {
t.Fatal(err)
}
_, undoBroad, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
_, undoNarrow, err := scope.AddTemporaryRead(nested)
if err != nil {
t.Fatal(err)
}
undoNarrow()
if block := scope.validateRead(filepath.Join(nested, "still-ok.txt")); block != nil {
t.Errorf("the narrow holder's release revoked the broad holder's coverage: %v", block.Reason)
}
undoBroad()
if block := scope.validateRead(filepath.Join(nested, "gone.txt")); block == nil {
t.Error("the tree outlived every holder")
}
}
// TWO READERS OF THE SAME ROOT SHARE ONE ENTRY. Only an exact match shares; a
// narrower one never does, which is what the escalation test above pins.
func TestTwoReadersOfTheSameRootShareOneEntry(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
_, undoFirst, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
_, undoSecond, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
undoFirst()
if block := scope.validateRead(filepath.Join(outside, "still-held.txt")); block != nil {
t.Errorf("one reader's release revoked the other's grant on the same root: %v", block.Reason)
}
undoSecond()
if block := scope.validateRead(filepath.Join(outside, "gone.txt")); block == nil {
t.Error("the shared root outlived both holders")
}
}
// A NARROW PERMANENT READ IS NOT BORROWED FROM A BROAD TEMPORARY ONE. The old
// scan walked readRoots in order and stopped at the first entry covering the
// path, so a broad TEMPORARY root earlier in the slice shadowed a narrower
// PERMANENT one later — and the caller took a reference it did not need on a
// grant that was about to end.
func TestAPermanentReadIsNotShadowedByABroadTemporaryOne(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
nested := filepath.Join(outside, "nested")
if err := os.MkdirAll(nested, 0o700); err != nil {
t.Fatal(err)
}
// The broad TEMPORARY grant is taken first, so it sits earlier in readRoots.
_, undoBroad, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
if _, err := scope.AddRead(nested); err != nil {
t.Fatal(err)
}
undoBroad()
if block := scope.validateRead(filepath.Join(nested, "mine.txt")); block != nil {
t.Errorf("a PERMANENT read grant died with a broad temporary one that merely preceded it: %v", block.Reason)
}
}
// A PERMANENT root is not refcounted and must never be removed by a temporary
// holder's cleanup — the undo for a request it already covers is genuinely
// nothing.
func TestATemporaryGrantNeverRevokesAPermanentRoot(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
if _, err := scope.AddRead(outside); err != nil {
t.Fatal(err)
}
_, undo, err := scope.AddTemporaryRead(outside)
if err != nil {
t.Fatal(err)
}
undo()
if !hasReadRoot(scope, outside) {
t.Fatal("a temporary holder's cleanup removed a permanent read root")
}
}
// The same rule for WRITE roots, or a write grant keeps the bug reads no longer
// have.
func TestATemporaryWriteSurvivesASiblingsCleanup(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
covered := func() bool {
for _, existing := range scope.Roots() {
if existing == outside {
return true
}
}
return false
}
_, undoA, err := scope.AddTemporaryWrite(outside)
if err != nil {
t.Fatal(err)
}
_, undoB, err := scope.AddTemporaryWrite(outside)
if err != nil {
t.Fatal(err)
}
if !covered() {
t.Fatal("the write grant did not take effect")
}
undoA()
if !covered() {
t.Fatal("A's cleanup revoked the write root while B still held a grant")
}
undoB()
if covered() {
t.Fatal("the write root outlived its last holder")
}
}
// UNDER REAL CONCURRENCY, which is the shape the parallel tool batch produces:
// eight holders taking and releasing the same root in overlapping windows. The
// root must be present the whole time at least one holder has it, and gone once
// the last releases.
func TestConcurrentHoldersOfOneRoot(t *testing.T) {
scope, _, outside := scopeOutsideRoots(t)
const holders = 8
var wg sync.WaitGroup
start := make(chan struct{})
release := make(chan struct{})
// TWO PER HOLDER, because each one can report at both hasReadRoot checks. At
// one slot per holder the channel fills after eight reports, the ninth send
// blocks forever, wg.Done() is never reached and wg.Wait() hangs — so the
// test would DEADLOCK rather than fail in exactly the case it exists to
// catch, since nothing drains failures until after the wait.
failures := make(chan string, 2*holders)
// Signalled by EVERY holder once its own AddTemporaryRead has returned,
// failure included: the gate below waits for all of them, so a holder that
// returned early without signalling would hang this test rather than fail it.
acquired := make(chan struct{}, holders)
for i := 0; i < holders; i++ {
wg.Add(1)
go func() {
defer wg.Done()
<-start
_, undo, err := scope.AddTemporaryRead(outside)
acquired <- struct{}{}
if err != nil {
failures <- err.Error()
return
}
// Every holder must SEE its own grant for as long as it holds it.
if !hasReadRoot(scope, outside) {
failures <- "a holder could not see the root it had just been granted"
}
<-release
if !hasReadRoot(scope, outside) {
failures <- "a holder lost the root while still holding it"
}
undo()
}()
}
close(start)
// EVERY holder, not the first one. Spinning on hasReadRoot only waited for
// somebody to hold the root, and the spin is tight enough to win that race
// against the goroutines still being scheduled: measured over 200 runs, 194
// of them peaked at a single simultaneous holder and none ever reached
// eight. A test named for concurrent holders was testing one holder at a
// time, and would have passed against an implementation with no refcount at
// all. Counting the acquisitions makes the overlap real instead of hoped for.
for i := 0; i < holders; i++ {
<-acquired
}
close(release)
wg.Wait()
close(failures)
for failure := range failures {
t.Fatal(failure)
}
if hasReadRoot(scope, outside) {
t.Fatal("the root outlived every holder")
}
// AND THE BOOKKEEPING IS EMPTY, not merely invisible. hasReadRoot answers
// what a caller can see; a refcount entry left behind at zero is not visible
// that way and would still leak, one map entry per acquire/release cycle.
scope.mu.Lock()
leftover, stillCounted := scope.tempReads[outside]
scope.mu.Unlock()
if stillCounted {
t.Fatalf("the refcount table kept an entry for %q after every holder released: %v", outside, leftover)
}
}
+169 -5
View File
@@ -2,16 +2,77 @@ package sandbox
import (
"errors"
"fmt"
"path/filepath"
"strings"
)
// isWindowsVolumeRoot reports whether a cleaned path is the top of a volume,
// with nothing above it: `C:\`, a bare separator, or a UNC share root.
//
// Detected structurally rather than by pattern matching drive letters, because
// filepath.Dir of a root is that same root and of anything else is strictly
// shorter. That holds for drive-qualified paths, for the separator alone, and
// for UNC roots, on either build host.
func isWindowsVolumeRoot(path string) bool {
cleaned := filepath.Clean(strings.TrimSpace(path))
if cleaned == "" || cleaned == "." {
return false
}
return filepath.Dir(cleaned) == cleaned
}
// validateWindowsACLComponent rejects anything that is not a single path
// component.
//
// This is load-bearing in two places, which is why it lives in the portable file
// rather than beside either of them.
//
// At apply time NtCreateFile happily resolves a RELATIVE name containing
// separators, and it resolves it the ordinary way, so an intermediate junction
// inside that name is followed and the object lands outside the pinned parent.
// A name with a separator reopens exactly the hole the parent handle exists to
// close.
//
// At plan time the same shape escapes the write root: a name is joined onto the
// root to place a deny ACE, so ".." or a separator puts that ACE on a directory
// outside the workspace entirely.
//
// The separators are checked explicitly rather than via filepath.Base, because
// these are Windows paths whatever the build host is, and on Linux
// filepath.Base leaves a backslash-joined name untouched and would wave it
// through. A colon is rejected too: it names an alternate data stream or a
// drive, neither of which is a child.
func validateWindowsACLComponent(name string) error {
switch {
case name == "":
return errors.New("windows ACL path component is empty")
case name == "." || name == "..":
return fmt.Errorf("windows ACL path component %q is a relative reference, not a child", name)
case strings.ContainsAny(name, `\/`):
return fmt.Errorf("windows ACL path component %q contains a separator, so it would resolve through intermediate directories instead of staying a child", name)
case strings.Contains(name, ":"):
return fmt.Errorf("windows ACL path component %q contains a colon, which names a stream or a drive rather than a child", name)
}
return nil
}
type WindowsACLAction string
const (
WindowsACLAllowWrite WindowsACLAction = "allow-write"
WindowsACLDenyRead WindowsACLAction = "deny-read"
WindowsACLDenyWrite WindowsACLAction = "deny-write"
// WindowsACLDenyDelete denies removing or renaming the object it names,
// WITHOUT denying writes to it or inside it, and without inheriting.
//
// It exists for .git. The write-denied carveouts live on .git/config and
// .git/hooks as objects, so replacing the .git directory discards them: the
// recreated config and hooks inherit the workspace allow with no deny, which
// restores credential.helper and core.hooksPath. .git cannot simply join
// sandboxFullyProtectedMetadataNames, because DenyWrite's mask includes
// FILE_GENERIC_WRITE and git must write index, objects and refs.
WindowsACLDenyDelete WindowsACLAction = "deny-delete"
)
type WindowsACLEntry struct {
@@ -19,6 +80,11 @@ type WindowsACLEntry struct {
Path string `json:"path"`
Capability string `json:"capability"`
Materialize bool `json:"materialize,omitempty"`
// MaterializeFile makes Materialize create an empty FILE instead of a
// directory. Only meaningful with Materialize. .git/config is the case that
// forces the distinction: created as a directory it does not merely carry
// the wrong ACL, it makes `git init` fail outright.
MaterializeFile bool `json:"materializeFile,omitempty"`
}
type WindowsACLPlan struct {
@@ -40,6 +106,15 @@ func BuildWindowsACLPlan(config WindowsSandboxCommandConfig) (WindowsACLPlan, er
Path: capability.Root,
Capability: capability.SID,
})
// The grant above carries DELETE, so the same object guard the principal
// plan has must come with it: not DenyWrite (git writes index, objects and
// refs), not materialized (git creates .git, and an empty one breaks
// git init), and not inherited, so everything underneath stays writable.
entries = append(entries, WindowsACLEntry{
Action: WindowsACLDenyDelete,
Path: windowsRenameProtectedObject(capability.Root),
Capability: capability.SID,
})
for _, path := range capability.ProtectedWriteDenyPaths {
entries = append(entries, WindowsACLEntry{
Action: WindowsACLDenyWrite,
@@ -48,6 +123,31 @@ func BuildWindowsACLPlan(config WindowsSandboxCommandConfig) (WindowsACLPlan, er
})
}
}
// Read roots, granted to the read-capability SID.
//
// A profile carrying DenyRead runs the command on a strict token rather than a
// WRITE_RESTRICTED one, and the strict token applies the restricted-SID check
// to reads. Read roots reached that check granted only to the principal's own
// account SID, which the write jail keeps out of the restricting set on
// purpose, so every read failed it — including opening the executable. The
// principal plan still grants the account SID; this is the other half of the
// same grant, so the allow-list and the restriction now come from one place.
readSID, err := windowsReadAllowCapabilitySID(config)
if err != nil {
return WindowsACLPlan{}, err
}
if readSID != "" {
for _, path := range config.PermissionProfile.FileSystem.ReadRoots {
if path = strings.TrimSpace(path); path == "" {
continue
}
entries = append(entries, WindowsACLEntry{
Action: WindowsACLAllowRead,
Path: path,
Capability: readSID,
})
}
}
writeSIDs := windowsWriteCapabilitySIDs(writeCapabilities)
for _, path := range config.PermissionProfile.FileSystem.DenyWrite {
path = strings.TrimSpace(path)
@@ -79,6 +179,39 @@ func BuildWindowsACLPlan(config WindowsSandboxCommandConfig) (WindowsACLPlan, er
return WindowsACLPlan{Entries: dedupeWindowsACLEntries(entries)}, nil
}
// windowsPlanVolumeRootGrant returns the first volume root the plan would have
// to change permissions on, or empty when it needs none.
//
// Only elevated setup can write a volume-root DACL, so this is the one entry
// that decides whether a plan is applicable by the unelevated tier at all. The
// check is on the PLAN rather than on the profile because the plan is what gets
// applied: a future entry that lands at a volume root for some other reason is
// caught by the same test.
func windowsPlanVolumeRootGrant(plan WindowsACLPlan) string {
for _, entry := range plan.Entries {
if isWindowsVolumeRoot(normalizeProfilePath(entry.Path)) {
return entry.Path
}
}
return ""
}
// windowsRenameProtectedObject names the object whose DELETE must be denied on
// a write root, whatever trustee holds the grant.
//
// ONE DERIVATION, BECAUSE TWO PLANNERS CONSUME IT. The allow-write mask both
// backends share includes DELETE, and it inherits from the write root onto .git.
// The carveouts that actually protect git live on .git/config and .git/hooks as
// OBJECTS, so renaming .git aside and recreating it discards them: the fresh
// config and hooks inherit the workspace allow with no deny of their own, which
// hands back credential.helper and core.hooksPath. The principal planner denied
// DELETE here and the capability planner did not, so the default restricted-token
// backend was missing the guard entirely. Deriving it in one place is what stops
// a change to the shared mask from updating one consumer and not the other.
func windowsRenameProtectedObject(root string) string {
return filepath.Join(root, sandboxRenameProtectedMetadataName)
}
type windowsWriteRootCapability struct {
Root string
SID string
@@ -140,17 +273,32 @@ func windowsWriteCapabilitySIDs(capabilities []windowsWriteRootCapability) []str
}
func windowsReadDenyCapabilitySIDs(config WindowsSandboxCommandConfig, writeSIDs []string) ([]string, error) {
if len(writeSIDs) > 0 {
return writeSIDs, nil
}
if len(config.PermissionProfile.FileSystem.DenyRead) == 0 {
return nil, nil
}
caps, err := LoadOrCreateWindowsCapabilitySIDs(config.SandboxHome)
// The read-capability SID must be denied here as well as granted above. It
// holds read on the read roots, and those start at the filesystem root, so a
// carveout sitting inside one of them would stay readable through that grant
// and the deny list would quietly stop meaning anything.
// Every SID the token can carry has to be denied, not just the newest one.
// With no write roots the token's only capability is ReadOnly, so dropping it
// here would leave the deny naming a SID the command never holds.
denySIDs := append([]string{}, writeSIDs...)
if len(denySIDs) == 0 {
caps, err := LoadOrCreateWindowsCapabilitySIDs(config.SandboxHome)
if err != nil {
return nil, err
}
denySIDs = append(denySIDs, caps.ReadOnly)
}
readSID, err := windowsReadAllowCapabilitySID(config)
if err != nil {
return nil, err
}
return []string{caps.ReadOnly}, nil
if readSID != "" {
denySIDs = append(denySIDs, readSID)
}
return denySIDs, nil
}
func planWindowsDenyReadPaths(paths []string) []string {
@@ -193,3 +341,19 @@ func dedupeWindowsACLEntries(entries []WindowsACLEntry) []WindowsACLEntry {
}
return out
}
// windowsReadAllowCapabilitySID returns the read-capability SID when this profile
// will run on a strict token, and "" otherwise.
//
// Gated on DenyRead because that is exactly what selects the strict token in the
// runner: with no DenyRead the token stays WRITE_RESTRICTED, reads skip the
// restricted-SID check entirely, and granting read on roots as broad as the
// filesystem root would add ACEs that buy nothing. Derived purely from the
// profile, so the setup half and the command half reach the same answer without
// consulting anything ambient.
func windowsReadAllowCapabilitySID(config WindowsSandboxCommandConfig) (string, error) {
if len(config.PermissionProfile.FileSystem.DenyRead) == 0 {
return "", nil
}
return WindowsReadAllowSID(config.SandboxHome)
}
+469 -38
View File
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
@@ -15,28 +16,121 @@ import (
const windowsFileDeleteChild windows.ACCESS_MASK = 0x00000040
type windowsACLPathGroup struct {
Path string
Entries []WindowsACLEntry
Materialize bool
Path string
Entries []WindowsACLEntry
Materialize bool
MaterializeFile bool
}
// windowsACLChainStep is one directory component beneath the anchor, and
// whether THIS run created it. The flag comes from the kernel rather than from
// "it was missing when we looked", because something else can win the gap
// between the probe and the create, and removing a directory the sandbox did not
// make is how a rollback deletes a user's data.
type windowsACLChainStep struct {
Name string
Made bool
}
// windowsACLMaterialization is exactly what materialization created, recorded in
// the shape rollback needs to undo it without resolving a single pathname below
// the anchor.
//
// The anchor is the deepest directory that already existed, and it is the ONLY
// pathname rollback re-resolves. Everything under it is a list of single
// components walked one handle at a time, because a name containing a separator
// is resolved the ordinary way by the kernel and would follow an intermediate
// junction straight out of the approved tree.
type windowsACLMaterialization struct {
AnchorPath string
AnchorID windowsFileIdentity
// Chain is every component between the anchor and the target, shallow to
// deep. All of them are needed to descend at rollback time; only the ones
// with Made set are removed.
Chain []windowsACLChainStep
// File is the leaf file component created inside the deepest Chain entry,
// for the .git/config carveout. Empty when the target is a directory.
File string
FileMade bool
}
func (materialization windowsACLMaterialization) createdAnything() bool {
if materialization.FileMade {
return true
}
for _, step := range materialization.Chain {
if step.Made {
return true
}
}
return false
}
// windowsACLRestoreHook observes restore order. Nil in production; the ordering
// it exposes is load-bearing and was previously pinned by nothing at all,
// despite a comment claiming otherwise.
var windowsACLRestoreHook func(path string)
type windowsACLSnapshot struct {
Path string
Descriptor *windows.SECURITY_DESCRIPTOR
Materialized bool
Path string
Descriptor *windows.SECURITY_DESCRIPTOR
Created windowsACLMaterialization
// TargetID is the object the DACL was read from, so the restore can prove it
// is writing back to that same object rather than to whatever the pathname
// resolves to at rollback time.
//
// A no-follow re-open catches a REPARSE POINT swapped in since apply, and
// that is what the restore relied on. It cannot catch the other
// substitution: rename the target aside and put an ordinary directory of the
// same name in its place. Nothing about that decoy is a reparse point, so the
// re-open succeeds and the old DACL is written onto the attacker's object
// while the real one keeps the aborted setup's ACEs.
//
// Volume serial plus file index is the same identity the materialization
// unwind already anchors on. See reopenWindowsACLDirectoryAsIdentity.
TargetID windowsFileIdentity
}
func applyWindowsACLPlan(plan WindowsACLPlan) (func() error, error) {
groups := groupWindowsACLPlanByPath(plan)
snapshots := make([]windowsACLSnapshot, 0, len(groups))
abort := func(err error) (func() error, error) {
if rollbackErr := rollbackWindowsACLSnapshots(snapshots); rollbackErr != nil {
return nil, fmt.Errorf("%w; rollback failed: %v", err, rollbackErr)
}
return nil, err
}
var deferred []windowsACLPathGroup
for _, group := range groups {
snapshot, applied, err := applyWindowsACLPathGroup(group)
if err != nil {
rollbackErr := rollbackWindowsACLSnapshots(snapshots)
if rollbackErr != nil {
return nil, fmt.Errorf("%w; rollback failed: %v", err, rollbackErr)
}
return nil, err
return abort(err)
}
if applied {
snapshots = append(snapshots, snapshot)
continue
}
deferred = append(deferred, group)
}
// A group that applied to nothing was skipped because its target did not
// exist and the group does not materialize one. A LATER group can still
// create it, so the skip has to be retried rather than treated as final.
//
// The .git rename guard is exactly this shape and was silently absent
// because of it. .git must NOT be materialized, since an empty one breaks
// `git init`, so its deny-delete group carries no Materialize. But .git does
// get created, as the parent chain of the .git\config carveout, and that
// group sorts AFTER it. So on every workspace that did not already have a
// .git, the guard was skipped, the directory appeared moments later, and
// nothing went back for it: the principal could then rename .git aside and
// shed the config and hooks carveouts, which is the escape the guard exists
// to stop. Groups that are genuinely absent simply skip again here.
for _, group := range deferred {
snapshot, applied, err := applyWindowsACLPathGroup(group)
if err != nil {
return abort(err)
}
if applied {
snapshots = append(snapshots, snapshot)
@@ -61,6 +155,7 @@ func groupWindowsACLPlanByPath(plan WindowsACLPlan) []windowsACLPathGroup {
}
group.Entries = append(group.Entries, entry)
group.Materialize = group.Materialize || entry.Materialize
group.MaterializeFile = group.MaterializeFile || entry.MaterializeFile
}
out := make([]windowsACLPathGroup, 0, len(byPath))
for _, group := range byPath {
@@ -84,8 +179,40 @@ func applyWindowsACLPathGroup(group windowsACLPathGroup) (windowsACLSnapshot, bo
// elevated setup a lower-privileged local user could swap the target for a
// symlink/junction between operations and redirect the ACL change onto a
// system object it never validated (issue #728, a TOCTOU privilege boundary).
materialized := false
handle, isDir, err := openWindowsACLTarget(path)
// Reject a malformed group BEFORE touching the filesystem. This validation
// used to run after materialization, so a bad SID or an unknown action
// created a directory chain and only then failed, leaving the error path to
// unwind work that never needed doing. Nothing here depends on isDir: that
// argument only selects the inheritance flag, while the errors come from the
// action lookup and the SID parse.
if _, err := windowsExplicitAccessEntries(group.Entries, false); err != nil {
return windowsACLSnapshot{}, false, err
}
var created windowsACLMaterialization
var handle windows.Handle
// Every exit from here goes through one closure, because there are now two
// things to undo rather than one: the open handle, and whatever
// materialization created. Both are captured by reference and both start
// zero, so calling this before either is set is safe and does nothing.
//
// The unwind is handle-relative. It must never fall back to a pathname
// delete: the failure being cleaned up here can BE the path swap, and
// os.RemoveAll on a swapped ancestor is precisely the recursive elevated
// delete outside the workspace that this cleanup is supposed to prevent.
fail := func(err error) (windowsACLSnapshot, bool, error) {
if handle != 0 {
_ = windows.CloseHandle(handle)
}
if unwindErr := rollbackWindowsACLMaterialization(created); unwindErr != nil {
return windowsACLSnapshot{}, false, fmt.Errorf("%w; cleanup failed: %v", err, unwindErr)
}
return windowsACLSnapshot{}, false, err
}
var isDir bool
var err error
handle, isDir, err = openWindowsACLTarget(path)
if err != nil {
if !errors.Is(err, os.ErrNotExist) {
return windowsACLSnapshot{}, false, err
@@ -96,25 +223,19 @@ func applyWindowsACLPathGroup(group windowsACLPathGroup) (windowsACLSnapshot, bo
}
return windowsACLSnapshot{}, false, nil
}
if err := os.MkdirAll(path, 0o700); err != nil {
return windowsACLSnapshot{}, false, fmt.Errorf("materialize windows ACL target %s: %w", path, err)
// created is assigned even on failure: materialization reports what it
// managed to make before it stopped, and fail() unwinds exactly that.
created, err = materializeWindowsACLTarget(path, group.MaterializeFile)
if err != nil {
return fail(fmt.Errorf("materialize windows ACL target %s: %w", path, err))
}
materialized = true
handle, isDir, err = openWindowsACLTarget(path)
if err != nil {
_ = os.RemoveAll(path)
return windowsACLSnapshot{}, false, fmt.Errorf("open materialized windows ACL target %s: %w", path, err)
// This is the branch that fires when the post-create verify catches a
// swap, so it is the single most important cleanup in the file.
return fail(fmt.Errorf("open materialized windows ACL target %s: %w", path, err))
}
}
// From here the handle is open; every early return must close it first (and
// remove a freshly materialized target) so a failure leaks neither.
fail := func(err error) (windowsACLSnapshot, bool, error) {
_ = windows.CloseHandle(handle)
if materialized {
_ = os.RemoveAll(path)
}
return windowsACLSnapshot{}, false, err
}
descriptor, err := windows.GetSecurityInfo(handle, windows.SE_FILE_OBJECT, windows.DACL_SECURITY_INFORMATION)
if err != nil {
return fail(fmt.Errorf("read windows ACL for %s: %w", path, err))
@@ -138,8 +259,14 @@ func applyWindowsACLPathGroup(group windowsACLPathGroup) (windowsACLSnapshot, bo
// The handle has served its purpose (read+write bound to one object) and is
// closed now — rollback re-opens no-follow rather than holding a handle for
// the whole sandbox lifetime, since one caller discards the rollback closure.
// Captured BEFORE the handle closes, because that handle is the only thing
// that names the object rather than the path.
targetID, err := windowsIdentityOfHandle(handle)
if err != nil {
return fail(fmt.Errorf("read windows ACL target identity for %s: %w", path, err))
}
_ = windows.CloseHandle(handle)
return windowsACLSnapshot{Path: path, Descriptor: descriptor, Materialized: materialized}, true, nil
return windowsACLSnapshot{Path: path, Descriptor: descriptor, Created: created, TargetID: targetID}, true, nil
}
// openWindowsACLTarget opens path for reading and rewriting its DACL without
@@ -178,6 +305,12 @@ func openWindowsACLTarget(path string) (windows.Handle, bool, error) {
_ = windows.CloseHandle(handle)
return 0, false, fmt.Errorf("refusing to apply ACL to reparse-point target %s: possible path swap during elevated setup", path)
}
// Ancestors are resolved by CreateFile even with FILE_FLAG_OPEN_REPARSE_POINT,
// so the check above is not enough on its own.
if err := verifyWindowsACLTargetNotRedirected(handle, path); err != nil {
_ = windows.CloseHandle(handle)
return 0, false, err
}
isDir := info.FileAttributes&windows.FILE_ATTRIBUTE_DIRECTORY != 0
return handle, isDir, nil
}
@@ -206,10 +339,19 @@ func windowsExplicitAccessEntries(entries []WindowsACLEntry, isDir bool) ([]wind
if err != nil {
return nil, err
}
entryInheritance := inheritance
// DenyDelete governs the object it names and nothing beneath it. Inherited
// onto .git's children it would deny DELETE on every file inside, so git
// could not remove a lock file, a ref, or anything else it rewrites, and
// the guard would read as a broken repository rather than as a blocked
// rename.
if entry.Action == WindowsACLDenyDelete {
entryInheritance = windows.NO_INHERITANCE
}
out = append(out, windows.EXPLICIT_ACCESS{
AccessPermissions: permissions,
AccessMode: accessMode,
Inheritance: inheritance,
Inheritance: entryInheritance,
Trustee: windows.TRUSTEE{
TrusteeForm: windows.TRUSTEE_IS_SID,
TrusteeType: windows.TRUSTEE_IS_GROUP,
@@ -223,11 +365,59 @@ func windowsExplicitAccessEntries(entries []WindowsACLEntry, isDir bool) ([]wind
func windowsACLAccess(action WindowsACLAction) (windows.ACCESS_MODE, windows.ACCESS_MASK, error) {
switch action {
case WindowsACLAllowWrite:
return windows.GRANT_ACCESS, windows.FILE_GENERIC_READ | windows.FILE_GENERIC_WRITE | windows.FILE_GENERIC_EXECUTE, nil
// DELETE is part of the grant, not an extra. FILE_GENERIC_WRITE covers
// creating and modifying but not removing or renaming, and a rename needs
// delete access on the source. Under the old same-user token that gap was
// invisible, because the caller already held inherited rights on its own
// tree; a sandbox principal is a separate account with no such
// inheritance, so without DELETE it can write a file it can never delete.
// Ordinary editing and most git operations rewrite files by replacing
// them, so the omission fails normal work rather than an edge case.
//
// FILE_DELETE_CHILD is deliberately NOT granted, for the same reason
// WRITE_DAC and WRITE_OWNER are not. On a parent it authorises deleting a
// child whatever the child's own DACL says, so granting it on a write root
// hands back the write-denied carve-outs underneath it: a principal could
// delete .git/config and recreate it, and the replacement inherits this
// grant with no deny of its own — restoring exactly the credential.helper
// and core.hooksPath control the carve-out exists to prevent.
//
// It was granted here originally to keep the mask symmetric with
// WindowsACLDenyWrite, which does treat FILE_DELETE_CHILD as part of
// write. Symmetry is the wrong goal: denying a capability is not a reason
// to grant it. DELETE alone covers removing and renaming files the
// principal owns inside its roots, which is what the grant is for.
return windows.GRANT_ACCESS, windows.FILE_GENERIC_READ | windows.FILE_GENERIC_WRITE | windows.FILE_GENERIC_EXECUTE | windows.DELETE, nil
case WindowsACLAllowRead:
// Read and traverse without write. A sandbox principal is a separate
// account with no inherent access to the caller's tree, so a read-only
// root has to be granted rather than assumed. Deliberately omits
// FILE_GENERIC_WRITE, DELETE and WRITE_DAC.
return windows.GRANT_ACCESS, windows.FILE_GENERIC_READ | windows.FILE_GENERIC_EXECUTE, nil
case windowsACLRevoke:
// REVOKE_ACCESS drops every ACE naming the trustee regardless of the mask,
// so the mask is ignored here. Used to retire a principal without having
// to remember which access each path was granted.
return windows.REVOKE_ACCESS, 0, nil
case WindowsACLDenyRead:
return windows.DENY_ACCESS, windows.FILE_GENERIC_READ | windows.FILE_GENERIC_EXECUTE, nil
case WindowsACLDenyWrite:
return windows.DENY_ACCESS, windows.FILE_GENERIC_WRITE | windows.DELETE | windowsFileDeleteChild | windows.WRITE_DAC | windows.WRITE_OWNER, nil
case WindowsACLDenyDelete:
// Deny removing or RENAMING the object itself, nothing more. Renaming a
// directory needs DELETE on that directory, so denying DELETE is what
// stops .git being moved aside and recreated without its carveouts.
//
// WRITE_DAC and WRITE_OWNER come along because a guard the principal can
// rewrite, or take ownership of and then rewrite, is not a guard.
//
// FILE_GENERIC_WRITE is deliberately absent: git writes index, objects and
// refs constantly, and denying it would break every commit rather than the
// rename. FILE_DELETE_CHILD is absent for the same reason one level down,
// since git deletes its own lock files and refs. Neither is needed here:
// this ACE does not inherit (see windowsExplicitAccessEntries), so it
// governs the .git directory object alone.
return windows.DENY_ACCESS, windows.DELETE | windows.WRITE_DAC | windows.WRITE_OWNER, nil
default:
return 0, 0, fmt.Errorf("unsupported windows ACL action %q", action)
}
@@ -235,11 +425,18 @@ func windowsACLAccess(action WindowsACLAction) (windows.ACCESS_MODE, windows.ACC
func rollbackWindowsACLSnapshots(snapshots []windowsACLSnapshot) error {
var errs []error
// Reverse order is load-bearing twice over. Groups are sorted ascending by
// path key and an ancestor key is always a proper prefix of its descendants,
// so walking backwards unwinds descendant before ancestor: a materialized
// directory is therefore empty by the time its own removal is attempted.
// Restoring ACLs in the same order is independently right, because
// SetSecurityInfo propagates inheritable ACEs down, so the ancestor must go
// last. TestRollbackUnwindsDescendantsBeforeAncestors pins it.
for index := len(snapshots) - 1; index >= 0; index-- {
snapshot := snapshots[index]
if snapshot.Materialized {
if err := os.RemoveAll(snapshot.Path); err != nil {
errs = append(errs, fmt.Errorf("remove materialized windows ACL target %s: %w", snapshot.Path, err))
if snapshot.Created.createdAnything() {
if err := rollbackWindowsACLMaterialization(snapshot.Created); err != nil {
errs = append(errs, err)
}
continue
}
@@ -248,16 +445,36 @@ func rollbackWindowsACLSnapshots(snapshots []windowsACLSnapshot) error {
errs = append(errs, fmt.Errorf("read rollback windows DACL for %s: %w", snapshot.Path, err))
continue
}
// Re-open no-follow rather than restoring by pathname: the restore must
// land on the real object, not a reparse point swapped in since apply. The
// residual window is small because the target is ACL-restricted by now, but
// a handle keeps the restore honest. On a materialized-target rollback we
// remove it above, so only the restore-existing path opens here.
// Re-open no-follow AND prove it is the same object. The no-follow open
// alone only rules out a reparse point swapped in since apply; a target
// renamed aside and replaced by an ordinary directory of the same name
// passes it, and the old DACL would land on the decoy while the real
// object kept the aborted setup's ACEs. See windowsACLSnapshot.TargetID.
handle, _, err := openWindowsACLTarget(snapshot.Path)
if err != nil {
errs = append(errs, fmt.Errorf("re-open windows ACL target %s for rollback: %w", snapshot.Path, err))
continue
}
if !snapshot.TargetID.empty() {
got, identityErr := windowsIdentityOfHandle(handle)
if identityErr != nil {
_ = windows.CloseHandle(handle)
errs = append(errs, fmt.Errorf("read rollback identity for %s: %w", snapshot.Path, identityErr))
continue
}
if got != snapshot.TargetID {
// REFUSED, not forced through. Leaving the real object with the
// aborted setup's ACEs is the safe direction: the caller is failing
// anyway, and writing a stale DACL onto an object setup never
// touched is the one outcome rollback must not produce.
_ = windows.CloseHandle(handle)
errs = append(errs, fmt.Errorf("refusing to restore windows ACL for %s: it is no longer the object setup applied to", snapshot.Path))
continue
}
}
if windowsACLRestoreHook != nil {
windowsACLRestoreHook(snapshot.Path)
}
if err := windows.SetSecurityInfo(handle, windows.SE_FILE_OBJECT, windows.DACL_SECURITY_INFORMATION, nil, nil, dacl, nil); err != nil {
errs = append(errs, fmt.Errorf("rollback windows ACL for %s: %w", snapshot.Path, err))
}
@@ -265,3 +482,217 @@ func rollbackWindowsACLSnapshots(snapshots []windowsACLSnapshot) error {
}
return errors.Join(errs...)
}
// materializeWindowsACLTarget creates a missing ACL target with the shape the
// owning tool expects. A directory target is created whole; a file target gets
// its parent chain created and then an empty file, because creating it as a
// directory would break the tool that owns it rather than just mis-ACL it.
// The returned record is meaningful even when the error is non-nil: a chain that
// got three levels deep and then failed still has three levels to unwind.
func materializeWindowsACLTarget(path string, asFile bool) (windowsACLMaterialization, error) {
directory := path
leaf := ""
if asFile {
directory = filepath.Dir(path)
leaf = filepath.Base(path)
}
created, parent, err := makeWindowsACLDirChainNoFollow(directory)
if err != nil {
return created, err
}
defer func() { _ = windows.CloseHandle(parent) }()
if !asFile {
return created, nil
}
// A racing creator winning is still fine: the target exists, which is all
// materialization needed. createWindowsACLChildFile reports that as
// created=false, so rollback will not delete a file the sandbox did not make.
created.File = leaf
madeFile, err := createWindowsACLChildFile(parent, leaf)
created.FileMade = madeFile
return created, err
}
// rollbackWindowsACLMaterialization removes exactly what materialization
// created, deepest first, without resolving any pathname below the anchor.
//
// This is the other half of the pathname problem. The old cleanup called
// os.RemoveAll on the target pathname, which re-resolves every ancestor at the
// moment it runs, so an ancestor swapped to a junction after the object was
// created sent a recursive elevated delete into an unrelated tree. It also only
// ever removed the final component, leaving every intermediate directory the
// chain had created behind.
//
// The anchor is the one pathname that has to be resolved again, and it is
// checked by file identity rather than by name: replacing a directory with
// another REAL directory of the same name needs no reparse point at all and
// would otherwise pass every no-follow check there is.
//
// Residue is preferable to over-deletion throughout. When something cannot be
// removed safely this reports it and leaves it, and never falls back to a
// pathname delete.
func rollbackWindowsACLMaterialization(materialization windowsACLMaterialization) error {
if !materialization.createdAnything() {
return nil
}
anchor, err := reopenWindowsACLDirectoryAsIdentity(materialization.AnchorPath, materialization.AnchorID)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
// The anchor is gone, so everything created beneath it is gone too.
// Nothing to undo, and no way to undo it if there were.
return nil
}
return fmt.Errorf("unwind windows ACL materialization under %s: %w", materialization.AnchorPath, err)
}
// One handle per level: handles[i] is the parent of Chain[i], which is what
// deleting Chain[i] relative to a pinned parent requires.
handles := []windows.Handle{anchor}
defer func() {
for _, handle := range handles {
if handle != 0 {
_ = windows.CloseHandle(handle)
}
}
}()
// Descend only as far as is actually required. Removing a directory needs its
// PARENT's handle, not its own, so the deepest component is opened only when
// a file leaf lives inside it. This is not just economy: the deepest
// component is usually the ACL target itself, so it may already carry the
// deny-read ACE this rollback is undoing, and opening it would be refused by
// the very ACL being unwound.
needed := len(materialization.Chain)
if !materialization.FileMade && needed > 0 {
needed--
}
depth := 0
for ; depth < needed; depth++ {
child, err := openWindowsACLChildDirectory(handles[depth], materialization.Chain[depth].Name)
if err != nil {
// Already removed by something else. Stop descending; whatever is
// below it is gone with it.
if isWindowsNotExist(err) {
break
}
return fmt.Errorf("unwind windows ACL materialization under %s: %w", materialization.AnchorPath, err)
}
handles = append(handles, child)
}
var errs []error
// The file leaf lives inside the deepest chain directory, so it goes first
// and only if the descent actually reached that far.
if materialization.FileMade && depth == needed {
if err := deleteWindowsACLChildFile(handles[len(handles)-1], materialization.File); err != nil {
errs = append(errs, fmt.Errorf("remove materialized windows ACL file %s: %w", materialization.File, err))
}
}
// Chain[i] is removed through handles[i], so the deepest one that can be
// removed is bounded by how far the descent actually got.
deepest := len(handles) - 1
if last := len(materialization.Chain) - 1; deepest > last {
deepest = last
}
for index := deepest; index >= 0; index-- {
// Close the child's own handle, if the descent opened one, before asking
// its parent to remove it. A directory with a live handle open still
// counts as present, so the parent's delete would come back as non-empty.
if index+1 < len(handles) && handles[index+1] != 0 {
_ = windows.CloseHandle(handles[index+1])
handles[index+1] = 0
}
if !materialization.Chain[index].Made {
continue
}
if err := deleteWindowsACLChildDirectory(handles[index], materialization.Chain[index].Name); err != nil {
errs = append(errs, fmt.Errorf("remove materialized windows ACL directory %s: %w", materialization.Chain[index].Name, err))
}
}
return errors.Join(errs...)
}
// windowsACLMaterializeSwapHook is a test seam and nothing else. It fires inside
// makeWindowsACLDirChainNoFollow at the exact instant the race used to be
// exploitable: the anchor is verified and pinned, and nothing has been created
// yet. A race reproducible only by luck is not a regression test, so the instant
// is made addressable rather than hoped for. Always nil in production.
var windowsACLMaterializeSwapHook func(anchor string)
// makeWindowsACLDirChainNoFollow is an os.MkdirAll that never resolves a
// pathname below its anchor.
//
// It walks UP to the deepest ancestor that already exists and opens it
// no-follow. Because GetFinalPathNameByHandle answers for the whole resolved
// path, that single check clears every ancestor above it too. Then it walks back
// DOWN, creating one component at a time relative to the HANDLE of the level
// above, so the tree it descends is pinned to objects rather than named by
// strings.
//
// That is the difference that matters. This used to verify a component by
// pathname, close the handle, and then hand the same string to os.Mkdir: two
// independent kernel resolutions with a gap between them. A workspace owner who
// swapped the verified ancestor for a junction inside that gap got the component
// created outside the approved tree, as Administrator, and verifying again
// afterwards cannot un-create it. Junctions need no privilege, so this was
// reachable by exactly the unprivileged user the sandbox exists to contain.
//
// It deliberately does NOT re-verify each created component by pathname. A child
// created relative to a pinned parent is in the right place by construction, so
// comparing pathnames afterwards would add nothing and would reject correct
// creates whenever the tree was legitimately renamed mid-setup.
//
// Returns what it created, plus an open handle to the deepest directory which
// the caller must close.
func makeWindowsACLDirChainNoFollow(dir string) (windowsACLMaterialization, windows.Handle, error) {
cleaned := filepath.Clean(strings.TrimSpace(dir))
if cleaned == "" || cleaned == "." {
return windowsACLMaterialization{}, 0, fmt.Errorf("materialize windows ACL target: empty directory path %q", dir)
}
// Walk up to the deepest ancestor that exists, collecting the component
// NAMES that are missing. Names, not paths: everything below the anchor is
// addressed relative to a handle from here on.
var missing []string
current := cleaned
var anchor windows.Handle
var anchorID windowsFileIdentity
for {
handle, identity, err := openWindowsACLDirectoryNoFollowWithIdentity(current)
if err == nil {
anchor, anchorID = handle, identity
break
}
if !errors.Is(err, os.ErrNotExist) {
return windowsACLMaterialization{}, 0, err
}
parent := filepath.Dir(current)
if parent == current {
return windowsACLMaterialization{}, 0, fmt.Errorf("materialize windows ACL target %s: no existing ancestor to anchor on", dir)
}
missing = append(missing, filepath.Base(current))
current = parent
}
created := windowsACLMaterialization{AnchorPath: current, AnchorID: anchorID}
if hook := windowsACLMaterializeSwapHook; hook != nil {
hook(current)
}
// Walk back down, one component per handle. The anchor handle is released as
// soon as its child is open, so at most two levels are held at once.
parent := anchor
for index := len(missing) - 1; index >= 0; index-- {
name := missing[index]
child, madeNow, err := createWindowsACLChildDirectory(parent, name)
if err != nil {
_ = windows.CloseHandle(parent)
return created, 0, err
}
created.Chain = append(created.Chain, windowsACLChainStep{Name: name, Made: madeNow})
_ = windows.CloseHandle(parent)
parent = child
}
return created, parent, nil
}
@@ -37,8 +37,15 @@ func TestApplyWindowsACLPathGroupHandleBasedRoundTrip(t *testing.T) {
if !applied {
t.Fatal("applied = false, want true for an existing directory target")
}
if snapshot.Path != dir || snapshot.Materialized {
t.Fatalf("snapshot = %#v, want Path=%q Materialized=false", snapshot, dir)
if snapshot.Path != dir {
t.Fatalf("snapshot.Path = %q, want %q", snapshot.Path, dir)
}
// The target already existed, so nothing was created and rollback must have
// nothing to remove. Asserting the chain rather than a bool matters: a
// rewiring that recorded the walked components instead of only the created
// ones would make rollback delete a directory the sandbox never made.
if snapshot.Created.createdAnything() {
t.Fatalf("snapshot recorded %#v as created for a target that already existed", snapshot.Created)
}
if snapshot.Descriptor == nil {
t.Fatal("snapshot has no captured descriptor to roll back to")
@@ -67,8 +74,16 @@ func TestApplyWindowsACLPathGroupMaterializes(t *testing.T) {
if err != nil {
t.Fatalf("applyWindowsACLPathGroup: %v", err)
}
if !applied || !snapshot.Materialized {
t.Fatalf("applied=%v materialized=%v, want both true", applied, snapshot.Materialized)
if !applied {
t.Fatal("applied = false, want true for a materialized target")
}
// Exactly one component was missing, so exactly one must be recorded as
// created, and it must be the leaf's own name rather than a path.
if len(snapshot.Created.Chain) != 1 || snapshot.Created.Chain[0] != (windowsACLChainStep{Name: "created", Made: true}) {
t.Fatalf("created chain = %#v, want one step {created true}", snapshot.Created.Chain)
}
if snapshot.Created.AnchorPath != filepath.Dir(target) {
t.Fatalf("anchor = %q, want the existing parent %q", snapshot.Created.AnchorPath, filepath.Dir(target))
}
if _, err := os.Stat(target); err != nil {
t.Fatalf("materialized target not created: %v", err)
Loaded 100 of 214 files, more files were not shown because too many files have changed in this diff. Show more