Files
zero/internal/execprofile/profile.go
Vasanth T 378d538e24 perf(agent): execution profiles with one-shot posture escalation (PR10b+PR10c) (#740)
* perf(agent): execution profiles, --exec-profile, bench passthrough, TUI /profile (PR10b+10c)

* fix(execprofile): harden profiles against review findings

- fast escalates on critical-risk mutations, not high: the sandbox marks
  every shell command high before command analysis, so a high trigger
  spent the one-shot on the first go test of any task
- escalation can now restore the provider-default effort a profile
  displaced (PostureEscalation.RestoreDefaultEffort; a plain empty
  target means leave untouched)
- TUI /turns while a profile is active pins the budget: disarms the
  escalation turn target (mirroring exec's explicit --max-turns rule)
  and profile reverts use touched bits instead of value equality, so
  explicit choices survive even when they coincide with profile values
- spec-draft runs get the same escalation policy as the main run
  instead of the displaced budget with no safety net
- bench validates and canonicalizes --exec-profile before spawning:
  an unknown name previously produced an exit-0 report full of
  millisecond usage-error walls
- turn summary prints exec-profile alongside mode; zero exec rejects
  an inline-empty --exec-profile=; balanced trace label documented and
  pinned as deliberate attribution

* fix(execprofile): explicit effort choices disarm the escalation effort restore

An explicit effort set AFTER profile selection could be cleared by a
mid-run escalation (the effort analog of the /turns pin, introduced by
RestoreDefaultEffort): TUI /effort and Ctrl+T now disarm the restore on
a policy clone, and the spec-draft path only arms it when no explicit
--spec-reasoning-effort governs the draft.

* fix(execprofile): address review findings on the profile surfaces

- --use-spec projects away a profile's self-correct knob instead of
  silently arming an option the draft runner ignores (the explicit
  --self-correct --use-spec combination is already a parse-time error)
- RunTurnBench canonicalizes and validates ExecProfile at the library
  boundary, so a direct caller's casing/whitespace cannot fork report
  labels and an unknown name fails before anything runs
- reverting a profile whose displaced budget was zero clears
  ZERO_MAX_TURNS (SetMaxTurnsEnv ignores zero, so sub-agents would have
  kept the removed profile's budget)
- the TUI effort-disarm test fails instead of skipping when the catalog
  stops filling low effort

* fix(tui): reconcile an active profile's effort fill on model switch

The fill is per-model, not a session preference: fast selected on a
model without low fills nothing, and switching to a model that supports
it should behave like selecting the profile there; the reverse switch
must not keep sending a level the destination does not support. Both
switch paths (the /model command and the cross-provider picker) now
re-derive the fill, leaving explicitly touched efforts alone and
keeping the escalation's effort restore in step. Also: the revert-env
test asserts key removal via LookupEnv, and applyProfileTurnBudget's
pinned-budget parameter is named and documented as covering mode
presets, not just the flag.

* fix(tui): void profile effort bookkeeping when the model-switch drop erases it

The /model path's pre-existing unsupported-effort drop ran before the
profile reconciliation and left contradictory state behind: a dropped
touched override kept blocking the profile from governing, and a
dropped profile fill left a stale applied value with the effort restore
still armed. The drop itself is load-bearing (the TUI run path forwards
the effort to the provider unfiltered), so instead of preserving a
known-unsupported level, the voided choice now resets the profile's
effort bookkeeping and the reconcile re-derives from scratch. The whole
sequence lives in one ring-parameterized helper driven by the switch
target's authoritative ring, unit-tested against synthetic rings for
both the void-then-refill arc and the supported-override survival.

* fix(tui): only drop an explicit effort when the target ring is authoritative

A nil ring meant both a catalog model with no effort controls and a
live-discovered custom model the catalog cannot vouch for. The drop now
requires an authoritative ring (a resolved catalog entry, whose ring may
legitimately be empty); an unknown ring preserves an explicit preference,
matching the cross-provider picker's behavior, while the profile's own
fill still applies only where support is known.
2026-07-19 00:53:17 +05:30

154 lines
6.8 KiB
Go

// Package execprofile defines named execution profiles: bundles of loop-posture
// knobs (turn budget, reasoning effort, self-correction, escalation triggers)
// that tune how hard the agent loop tries, independent of which model runs.
// Profiles COMPOSE with --mode: a mode answers "what runs" (model, tools),
// a profile answers "how hard the loop tries". Selection precedence is
// explicit flag > --mode > profile, enforced by the callers applying profiles
// last with the same fill-only-if-unset rule modes use.
//
// The catalog is deliberately tiny and value-stable: balanced is the empty
// profile (selecting it leaves the run's options, budget, and loop behavior
// identical to an unflagged run — asserted by test; only the opt-in trace and
// bench artifacts record the selected profile name, by design, so captures are
// self-describing), fast trades turn budget and effort for latency with a
// one-shot escalation back to the displaced posture as the safety net, and
// thorough raises the budget and arms full self-correction.
package execprofile
import (
"sort"
"strings"
"github.com/Gitlawb/zero/internal/agent"
"github.com/Gitlawb/zero/internal/sandbox"
)
// Profile is a named execution posture. Zero-valued fields inherit the run's
// existing value (flag, mode, config, or built-in default) — a profile only
// ever fills knobs the caller left unset, except MaxTurns which REPLACES the
// resolved budget (that displacement is what escalation restores).
type Profile struct {
// Name identifies the profile in traces, help text, and diagnostics.
Name string
// MaxTurns replaces the resolved per-run tool-turn budget when > 0 and the
// user did not pass an explicit --max-turns. The displaced resolved value
// becomes the escalation target (see Policy).
MaxTurns int
// ReasoningEffort fills the run's effort when the user left it unset. It
// still flows through the model-supported gating at the call site, so an
// unsupported level degrades exactly like an explicit flag would.
ReasoningEffort string
// SelfCorrect arms the post-edit verify-and-correct loop. Profiles can only
// turn it ON (the flag is presence-only, so false is indistinguishable from
// unset and must never override an explicit opt-in).
SelfCorrect bool
// Escalation triggers. All zero means the profile never escalates and
// Policy returns nil (the loop stays byte-identical to a nil-profile run).
// Targets are NOT part of the catalog: escalation restores the values the
// profile displaced at selection time, never invented ones, so targets are
// stamped by Policy from what the caller measured.
EscalateOnToolFailureStreak int
EscalateOnCompletionUncertain int
EscalateOnSelfCorrectFailure bool
EscalateOnRiskyMutation sandbox.RiskLevel
}
// The catalog. Values marked provisional are floors/ceilings derived from the
// Phase 0 baseline's read-class evidence (successful nav runs used single-digit
// turns; the mutating classes produced no successful samples to tune from) and
// are expected to be re-tuned from the post-oracle-fix re-capture. The
// escalation safety net is what makes shipping provisional Fast values safe:
// a run that hits trouble gets its displaced budget back mid-run.
var (
// Balanced is the default posture: empty on purpose. Selecting it changes
// nothing — the no-regression invariant of the whole feature.
Balanced = Profile{Name: "balanced"}
// Fast starts cheap (30 turns, low effort) and arms every escalation
// trigger so a struggling run restores its displaced posture: two
// same-tool retriable failures in a row, a second uncertain completion
// evaluation, a failing self-correction cycle, or a critical-risk
// mutation. The risk threshold is deliberately Critical, not High: the
// sandbox classifies EVERY shell command as at least high risk before any
// command analysis, so a High trigger would fire on the first `go test`
// of virtually any coding task and spend the one-shot on turn one.
// Critical marks the genuinely scary categories (destructive commands,
// piped installers, out-of-workspace writes, network mutations).
Fast = Profile{
Name: "fast",
MaxTurns: 30,
ReasoningEffort: "low",
EscalateOnToolFailureStreak: 2,
EscalateOnCompletionUncertain: 2,
EscalateOnSelfCorrectFailure: true,
EscalateOnRiskyMutation: sandbox.RiskCritical,
}
// Thorough doubles the default budget, asks for high effort, and arms the
// full post-edit verify-and-correct loop. Already the maximum posture, so
// it has nothing to escalate to.
Thorough = Profile{
Name: "thorough",
MaxTurns: 160,
ReasoningEffort: "high",
SelfCorrect: true,
}
)
var catalog = map[string]Profile{
Balanced.Name: Balanced,
Fast.Name: Fast,
Thorough.Name: Thorough,
}
// Lookup resolves a profile by name, case-insensitively and ignoring
// surrounding whitespace.
func Lookup(name string) (Profile, bool) {
profile, ok := catalog[strings.ToLower(strings.TrimSpace(name))]
return profile, ok
}
// Names returns the catalog's profile names, sorted, for usage errors and help.
func Names() []string {
names := make([]string, 0, len(catalog))
for name := range catalog {
names = append(names, name)
}
sort.Strings(names)
return names
}
// Policy projects the profile into the loop-facing ProfilePolicy.
//
// displacedMaxTurns is the turn budget this profile displaced at selection time
// (what the run would have used without it) and becomes the escalation's
// restore target; pass 0 when the profile did not displace the budget (e.g.
// the user pinned --max-turns explicitly) so escalation leaves the ceiling
// untouched. effortFilled reports whether the profile actually filled the
// run's reasoning effort (it backs off when the user set one explicitly); the
// displaced effort is then "" (the provider default) by construction, which
// the escalation restores via RestoreDefaultEffort — a plain "" target would
// mean "leave untouched".
//
// Profiles with no armed triggers return nil: the loop treats a nil policy as
// "no profile" (no observation, no counters), which keeps balanced and
// thorough runs byte-identical to the same knob values set by hand.
func (p Profile) Policy(displacedMaxTurns int, effortFilled bool) *agent.ProfilePolicy {
if p.EscalateOnToolFailureStreak == 0 && p.EscalateOnCompletionUncertain == 0 &&
!p.EscalateOnSelfCorrectFailure && p.EscalateOnRiskyMutation == "" {
return nil
}
return &agent.ProfilePolicy{
Name: p.Name,
Escalate: &agent.PostureEscalation{
MaxTurns: displacedMaxTurns,
RestoreDefaultEffort: effortFilled,
OnToolFailureStreak: p.EscalateOnToolFailureStreak,
OnCompletionUncertain: p.EscalateOnCompletionUncertain,
OnSelfCorrectFailure: p.EscalateOnSelfCorrectFailure,
OnRiskyMutation: p.EscalateOnRiskyMutation,
},
}
}