mirror of
https://github.com/Twigpine/zero.git
synced 2026-11-12 08:41:32 -10:00
Vasanth T
378d538e24
* perf(agent): execution profiles, --exec-profile, bench passthrough, TUI /profile (PR10b+10c) * fix(execprofile): harden profiles against review findings - fast escalates on critical-risk mutations, not high: the sandbox marks every shell command high before command analysis, so a high trigger spent the one-shot on the first go test of any task - escalation can now restore the provider-default effort a profile displaced (PostureEscalation.RestoreDefaultEffort; a plain empty target means leave untouched) - TUI /turns while a profile is active pins the budget: disarms the escalation turn target (mirroring exec's explicit --max-turns rule) and profile reverts use touched bits instead of value equality, so explicit choices survive even when they coincide with profile values - spec-draft runs get the same escalation policy as the main run instead of the displaced budget with no safety net - bench validates and canonicalizes --exec-profile before spawning: an unknown name previously produced an exit-0 report full of millisecond usage-error walls - turn summary prints exec-profile alongside mode; zero exec rejects an inline-empty --exec-profile=; balanced trace label documented and pinned as deliberate attribution * fix(execprofile): explicit effort choices disarm the escalation effort restore An explicit effort set AFTER profile selection could be cleared by a mid-run escalation (the effort analog of the /turns pin, introduced by RestoreDefaultEffort): TUI /effort and Ctrl+T now disarm the restore on a policy clone, and the spec-draft path only arms it when no explicit --spec-reasoning-effort governs the draft. * fix(execprofile): address review findings on the profile surfaces - --use-spec projects away a profile's self-correct knob instead of silently arming an option the draft runner ignores (the explicit --self-correct --use-spec combination is already a parse-time error) - RunTurnBench canonicalizes and validates ExecProfile at the library boundary, so a direct caller's casing/whitespace cannot fork report labels and an unknown name fails before anything runs - reverting a profile whose displaced budget was zero clears ZERO_MAX_TURNS (SetMaxTurnsEnv ignores zero, so sub-agents would have kept the removed profile's budget) - the TUI effort-disarm test fails instead of skipping when the catalog stops filling low effort * fix(tui): reconcile an active profile's effort fill on model switch The fill is per-model, not a session preference: fast selected on a model without low fills nothing, and switching to a model that supports it should behave like selecting the profile there; the reverse switch must not keep sending a level the destination does not support. Both switch paths (the /model command and the cross-provider picker) now re-derive the fill, leaving explicitly touched efforts alone and keeping the escalation's effort restore in step. Also: the revert-env test asserts key removal via LookupEnv, and applyProfileTurnBudget's pinned-budget parameter is named and documented as covering mode presets, not just the flag. * fix(tui): void profile effort bookkeeping when the model-switch drop erases it The /model path's pre-existing unsupported-effort drop ran before the profile reconciliation and left contradictory state behind: a dropped touched override kept blocking the profile from governing, and a dropped profile fill left a stale applied value with the effort restore still armed. The drop itself is load-bearing (the TUI run path forwards the effort to the provider unfiltered), so instead of preserving a known-unsupported level, the voided choice now resets the profile's effort bookkeeping and the reconcile re-derives from scratch. The whole sequence lives in one ring-parameterized helper driven by the switch target's authoritative ring, unit-tested against synthetic rings for both the void-then-refill arc and the supported-override survival. * fix(tui): only drop an explicit effort when the target ring is authoritative A nil ring meant both a catalog model with no effort controls and a live-discovered custom model the catalog cannot vouch for. The drop now requires an authoritative ring (a resolved catalog entry, whose ring may legitimately be empty); an unknown ring preserves an explicit preference, matching the cross-provider picker's behavior, while the profile's own fill still applies only where support is known.
66 lines
1.9 KiB
Go
66 lines
1.9 KiB
Go
package main
|
|
|
|
import (
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
func TestParseTurnArgsExecProfile(t *testing.T) {
|
|
getenv := func(string) string { return "" }
|
|
options, err := parseTurnArgs([]string{
|
|
"--suite", "suite.json",
|
|
"--model", "m",
|
|
"--exec-profile", "fast",
|
|
}, getenv)
|
|
if err != nil {
|
|
t.Fatalf("parseTurnArgs: %v", err)
|
|
}
|
|
if options.ExecProfile != "fast" {
|
|
t.Fatalf("ExecProfile = %q, want fast", options.ExecProfile)
|
|
}
|
|
|
|
// Inline form.
|
|
options, err = parseTurnArgs([]string{
|
|
"--suite=suite.json",
|
|
"--model=m",
|
|
"--exec-profile=thorough",
|
|
}, getenv)
|
|
if err != nil {
|
|
t.Fatalf("parseTurnArgs inline: %v", err)
|
|
}
|
|
if options.ExecProfile != "thorough" {
|
|
t.Fatalf("ExecProfile = %q, want thorough", options.ExecProfile)
|
|
}
|
|
}
|
|
|
|
// The bench validates the profile BEFORE spawning anything: an unknown name
|
|
// would otherwise make every child exit with a usage error in milliseconds
|
|
// and record those near-zero walls as valid latency samples in an exit-0
|
|
// report. Lookup normalization also keeps the stamped name canonical so two
|
|
// captures of the same posture compare equal.
|
|
func TestParseTurnArgsExecProfileValidatesAndNormalizes(t *testing.T) {
|
|
getenv := func(string) string { return "" }
|
|
|
|
options, err := parseTurnArgs([]string{
|
|
"--suite", "suite.json", "--model", "m", "--exec-profile", "FAST",
|
|
}, getenv)
|
|
if err != nil {
|
|
t.Fatalf("parseTurnArgs: %v", err)
|
|
}
|
|
if options.ExecProfile != "fast" {
|
|
t.Fatalf("ExecProfile = %q, want the canonical fast", options.ExecProfile)
|
|
}
|
|
|
|
_, err = parseTurnArgs([]string{
|
|
"--suite", "suite.json", "--model", "m", "--exec-profile", "blanced",
|
|
}, getenv)
|
|
if err == nil {
|
|
t.Fatal("an unknown profile must fail parse, before anything spawns")
|
|
}
|
|
for _, name := range []string{"balanced", "fast", "thorough"} {
|
|
if !strings.Contains(err.Error(), name) {
|
|
t.Fatalf("error must list %q, got %q", name, err.Error())
|
|
}
|
|
}
|
|
}
|