Files
zero/internal/cli/exec_plan_test.go
0176ee49ff feat(agent): add PermissionModePlan for interactive read-only planning (#853)
* feat(agent): add PermissionModePlan for interactive read-only planning

* fix(agent): drop redundant modeName branch, add plan mode regression tests

string(permissionMode) already yields "plan" / "spec-draft", so the
if/else recomputing modeName in the denial message was dead branching
on the same values. Also add plan-mode coverage mirroring three of the
four existing spec-draft regression tests: advertised tool set, and
denied write_file/bash calls. The fourth (submit-and-stop review
control) has no plan-mode analog, since plan mode has no submit tool.

* fix(agent): deny request_permissions in plan/spec-draft even when the registry omits it

request_permissions is dispatched by name in executeToolCall before the
registry-based ToolAdvertised gate runs, so that gate only helps when
the tool happens to be present in the caller's registry. A plan- or
spec-draft-mode registry that simply omits the tool (rather than
registering it as denied) let the call fall through to a real
turn/session-scoped permission grant, defeating the read-only
boundary. Deny it unconditionally at the top of
executeRequestPermissions for both read-only modes instead.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

* fix(agent): suppress executable hooks while plan/spec-draft mode is active

Plan mode promises a read-only turn, but sessionStart/sessionEnd fire on
every run and beforeTool/afterTool fire around allowed read calls, and
all four execute configured host commands outside the advertised-tool
and sandbox gates — so a project hook could mutate the workspace or
spawn a process from a session that advertises it cannot. Gate all four
dispatch points on the run's permission mode, with a regression test
asserting no hook command launches during a plan-mode run.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(agent): close plan-mode tool advertisement bypass

toolAdvertisedInPlan whitelisted ask_user and update_plan by name
alone, so a caller could register a mutating tool under either name
and have it advertised and executed in plan mode. Validate every tool
against its Safety() instead. Also exclude lsp_navigate, which is
marked SideEffectRead but lazily spawns a real language-server
process, contradicting plan mode's read-only guarantee. Add
regression tests for both.

* fix(agent): keep trust-gated hooks in spec-draft mode

Plan mode still suppresses executable hooks so a read-only planning turn
cannot spawn host processes via session or tool hooks. Spec-draft keeps
the existing trust model: project hooks fire when the workspace (or its
worktree trust root) is trusted. Unconditionally suppressing hooks in
spec-draft broke TestExecSpecWorktreeInheritsTrustEndToEnd for trusted
worktrees under --use-spec --worktree.

* fix(agent): cover plan-mode spoof and lsp_navigate denials

Expand the name-only spoof regression to both update_plan and ask_user,
and add an execution-path denial for lsp_navigate so plan mode cannot
spawn language servers even when a call still arrives.

* fix(agent): filter tool_search deferred candidates by plan/spec-draft visibility

tool_search resolved and ranked deferred tools by EnabledTools/DisabledTools
only, never by the run's permission-mode visibility. tool_search itself is
already denied at dispatch in plan/spec-draft (its Safety carries no side
effect, so it fails the same SideEffect==Read advertisement gate direct calls
use), so this was not reachable through the normal Run() path today. But it
is a real landmine: if that outer gate is ever loosened independently (e.g.
tool_search's no-side-effect Safety is judged advertisable), the loader had
no gate of its own and would hand a deferred write/mutator tool's name,
description, and full schema straight to a plan/spec-draft model.

Mirror agent.ToolAdvertised's plan/spec-draft branches inside the tools
package (toolAdvertisedForPermissionMode, next to the existing
toolAllowedByFilters mirror that avoids the same import cycle) and apply it
alongside the operator filters in visibleDeferredTools and
visibleEagerToolNames.

Added unit tests in tool_search_test.go for both modes, and an end-to-end
agent test that force-calls tool_search in plan mode and asserts no schema
leaks. Verified by reverting tool_search.go and confirming the new tests fail
(one shows load_tools resolving to the mutator's name); restored and
confirmed they pass. Also confirmed via a temporary probe that if plan mode's
outer advertisement gate is loosened, this filter is what actually stops the
leak.

* fix(agent): require Safety for spec-draft ask_user/submit_spec

Do not advertise or load re-registered control tools by name alone in
spec-draft mode. ask_user must be SideEffectRead+Allow and submit_spec
must be SideEffectWrite+Allow, matching the real tools. Apply the same
filter in tool_search and add spoof regression tests.

Refs Gitlawb/zero#642

* fix(agent): wire plan mode into TUI, CLI, and ACP entry points

Address the P1 finding that PermissionModePlan was documented but never
selected by /plan, zero exec, or ACP mode selectors. /plan on|off now
toggles the session permission mode (restoring the prior mode on off),
zero exec --plan selects plan for a run, and ACP advertises plan as a
client-selectable mode. Integration coverage for each entry path.

* fix(cli): reject --plan combined with --worktree

Worktree preparation runs in runExec before the plan permission mode is
assigned, so `zero exec --plan --worktree` could still trigger workspace
mutation ahead of the read-only gate. Reject the combination during
option validation, alongside the existing --use-spec/--skip-permissions-
unsafe conflict checks, so no worktree prep can occur.

Addresses a coderabbitai finding on PR #642.

* test(tools): assert spoofed tool schema doesn't leak via tool_search

The spoofed-control-tool regression only asserted on the description
string; Parameters() exposed no distinctive schema marker, so a
regression that leaked the schema without the description would still
have passed. Add a spoofed_secret property to the test tool's schema
and assert it's absent from result.Output alongside the description.

Addresses a coderabbitai finding on PR #642.

* fix(tui): gate local mutating commands behind plan mode

/plan on only flips the agent permission mode, which gates agent tool
calls. Local TUI commands that run entirely inside the TUI process
bypass that gate: /rewind restores workspace files from a checkpoint,
/export writes a transcript to disk, and /sandbox-setup spawns a
native host process. Add a shared plan-mode guard at the start of
dispatchCommand (mirroring the existing BTW-unavailable guard) that
rejects these three commands while permissionMode is
agent.PermissionModePlan, with regression coverage proving each is
blocked with no mutation/process spawn in plan mode and unaffected
outside it.

Addresses a coderabbitai finding on PR #642.

* fix(agent,specialist): layer plan mode system prompt and enforce read-only subagent mode

* Fix jatmn review findings for PR 642

* fix(agent,cli,tui): resolve CodeRabbit review comments on active turn model field and --plan permission mode conflict

* fix(agent): propagate permission mode to exec options and serialize ACP mode changes

Parse permissionMode in resolveExecPermissionMode, acquire turnMu.Lock in ACP handleSetMode to serialize mode changes with active turns, and block MCP subcommands in TUI plan mode.

Refs #642

* fix(agent,tui): update tests off removed tools.CoreTools/NewWriteFileTool/NewLSPNavigateTool wrappers

Those were thin unscoped wrappers around the Scoped variants, deleted
upstream in #706 since nothing else called them directly. Only these
tests still did; switch to the Scoped calls main's own tests already
use.

* fix(agent): fail-closed beforeTool vetoes and permission-mode plan guards

Keep beforeTool deny gates active under plan mode so hooksSuppressed no longer
fails open, and stop propagating --permission-mode for auto/ask/member children
so swarm members keep write tools. Apply --plan combination rejects to
--permission-mode plan as well.

Refs #853

* fix(agent): address CodeRabbit findings for plan-mode advertisement and entry paths

Unify plan/spec-draft tool advertisement in tools.ToolAdvertisedForPermissionMode
(with PermissionDeny short-circuit), cover ACP config and --permission-mode plan
list-tools paths, and allow bare /mcp while blocking mutating MCP subcommands.

Refs #853

---------

Co-authored-by: Claude Sonnet 5 <[email protected]>
Co-authored-by: euxaristia <[email protected]>
2026-08-10 10:15:29 +08:00

178 lines
6.9 KiB
Go

package cli
import (
"bytes"
"strings"
"testing"
"github.com/Gitlawb/zero/internal/agent"
"github.com/Gitlawb/zero/internal/config"
)
func TestParseExecArgsRecognizesPlanFlag(t *testing.T) {
options, _, err := parseExecArgs([]string{"--plan", "draft a plan"})
if err != nil {
t.Fatalf("parseExecArgs: %v", err)
}
if !options.plan {
t.Fatal("expected --plan to set options.plan")
}
}
func TestParseExecArgsRejectsPlanWithUseSpec(t *testing.T) {
_, _, err := parseExecArgs([]string{"--plan", "--use-spec", "draft a plan"})
if err == nil || !strings.Contains(err.Error(), "not both") {
t.Fatalf("expected --plan/--use-spec validation, got %v", err)
}
}
func TestParseExecArgsRejectsPlanWithSkipPermissionsUnsafe(t *testing.T) {
_, _, err := parseExecArgs([]string{"--plan", "--skip-permissions-unsafe", "draft a plan"})
if err == nil || !strings.Contains(err.Error(), "not both") {
t.Fatalf("expected --plan/--skip-permissions-unsafe validation, got %v", err)
}
}
// TestParseExecArgsRejectsPlanWithWorktree guards against worktree
// preparation (a filesystem mutation) running ahead of the plan mode gate:
// options.worktree is processed in runExec before the plan permission mode is
// assigned, so the combination must be rejected during option validation,
// before any worktree prep can occur.
func TestParseExecArgsRejectsPlanWithWorktree(t *testing.T) {
_, _, err := parseExecArgs([]string{"--plan", "--worktree", "draft a plan"})
if err == nil || !strings.Contains(err.Error(), "--worktree") {
t.Fatalf("expected --plan/--worktree validation, got %v", err)
}
}
func TestParseExecArgsRejectsPlanWithNonPlanPermissionMode(t *testing.T) {
_, _, err := parseExecArgs([]string{"--plan", "--permission-mode=ask", "draft a plan"})
if err == nil || !strings.Contains(err.Error(), "--permission-mode") {
t.Fatalf("expected --plan/--permission-mode validation, got %v", err)
}
options, _, err := parseExecArgs([]string{"--plan", "--permission-mode=plan", "draft a plan"})
if err != nil {
t.Fatalf("expected --plan with --permission-mode=plan to succeed, got %v", err)
}
if !options.plan || options.permissionMode != "plan" {
t.Fatalf("expected options.plan=true and permissionMode=plan, got plan=%v mode=%q", options.plan, options.permissionMode)
}
}
// TestParseExecArgsPermissionModePlanSharesCombinationGuards ensures the
// combination rejects that key off --plan also fire for --permission-mode plan,
// which reaches PermissionModePlan without setting options.plan.
func TestParseExecArgsPermissionModePlanSharesCombinationGuards(t *testing.T) {
cases := []struct {
name string
args []string
want string
}{
{"worktree", []string{"--permission-mode", "plan", "--worktree", "draft a plan"}, "--worktree"},
{"use-spec", []string{"--permission-mode=plan", "--use-spec", "draft a plan"}, "not both"},
{"skip-permissions-unsafe", []string{"--permission-mode", "plan", "--skip-permissions-unsafe", "draft a plan"}, "not both"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
_, _, err := parseExecArgs(tc.args)
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected --permission-mode plan / %s validation containing %q, got %v", tc.name, tc.want, err)
}
})
}
// Bare --permission-mode plan (no conflicting flags) still parses.
options, _, err := parseExecArgs([]string{"--permission-mode", "plan", "draft a plan"})
if err != nil {
t.Fatalf("expected bare --permission-mode plan to succeed, got %v", err)
}
if options.permissionMode != "plan" || options.plan {
t.Fatalf("expected permissionMode=plan and plan=false, got mode=%q plan=%v", options.permissionMode, options.plan)
}
}
// TestRunExecPlanHidesWriteAndShellToolsFromListing drives the real --plan
// flag through runExec (via --list-tools, so no provider is needed) and
// confirms write_file and bash — advertised under every other mode covered by
// TestRunExecListToolsAppliesModeBeforeListing-style tests — are hidden,
// mirroring the TUI /plan on gating end to end from the CLI entry point.
func TestRunExecPlanHidesWriteAndShellToolsFromListing(t *testing.T) {
cwd := t.TempDir()
var stdout bytes.Buffer
var stderr bytes.Buffer
exitCode := runWithDeps([]string{"exec", "--plan", "--list-tools"}, &stdout, &stderr, appDeps{
getwd: func() (string, error) {
return cwd, nil
},
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return execResolvedConfig(), nil
},
})
if exitCode != exitSuccess {
t.Fatalf("expected exit code %d, got %d: %s", exitSuccess, exitCode, stderr.String())
}
listing := stdout.String()
if !strings.Contains(listing, "Tools visible to model") {
t.Fatalf("expected --plan --list-tools to list tools, got %q", listing)
}
if !strings.Contains(listing, "read_file") {
t.Fatalf("expected plan mode to still list read_file, got %q", listing)
}
for _, hidden := range []string{"write_file", "edit_file", "apply_patch", "bash"} {
if strings.Contains(listing, hidden) {
t.Fatalf("expected --plan to hide %q from the tool listing, got %q", hidden, listing)
}
}
}
// TestRunExecPermissionModePlanHidesWriteAndShellToolsFromListing covers the
// --permission-mode plan entry path, which does not set options.plan and must
// still hide write and shell tools the same way --plan does.
func TestRunExecPermissionModePlanHidesWriteAndShellToolsFromListing(t *testing.T) {
cwd := t.TempDir()
var stdout bytes.Buffer
var stderr bytes.Buffer
exitCode := runWithDeps([]string{"exec", "--permission-mode", "plan", "--list-tools"}, &stdout, &stderr, appDeps{
getwd: func() (string, error) {
return cwd, nil
},
resolveConfig: func(string, config.Overrides) (config.ResolvedConfig, error) {
return execResolvedConfig(), nil
},
})
if exitCode != exitSuccess {
t.Fatalf("expected exit code %d, got %d: %s", exitSuccess, exitCode, stderr.String())
}
listing := stdout.String()
if !strings.Contains(listing, "Tools visible to model") {
t.Fatalf("expected --permission-mode plan --list-tools to list tools, got %q", listing)
}
if !strings.Contains(listing, "read_file") {
t.Fatalf("expected plan mode to still list read_file, got %q", listing)
}
for _, hidden := range []string{"write_file", "edit_file", "apply_patch", "bash"} {
if strings.Contains(listing, hidden) {
t.Fatalf("expected --permission-mode plan to hide %q from the tool listing, got %q", hidden, listing)
}
}
}
func TestResolveExecPermissionModePlanOverride(t *testing.T) {
options := execOptions{autonomy: "low", plan: true}
mode, err := resolveExecPermissionMode(options)
if err != nil {
t.Fatalf("resolveExecPermissionMode: %v", err)
}
// resolveExecPermissionMode itself only resolves --auto; the --plan override
// is applied by the caller (runExec) afterward, same as --use-spec. This
// pins the precondition: --plan must not interfere with autonomy resolution.
if mode != agent.PermissionModeAuto {
t.Fatalf("resolveExecPermissionMode with --plan = %q, want auto (override applied by the caller)", mode)
}
}