Files
zero/internal/tools/escalate_model.go
KRATOSandGnanam 69810d5ab0 Mid-run model escalation: escalate_model tool + --allow-escalation (#146)
* modelregistry: add UpgradeTargetID and Registry.UpgradeTarget escalation lookup

* modelregistry: seed escalation chains in catalog decoration

* modelregistry: verification gate for escalation upgrade map

* agent: add ModelSwitcher option and ToolResult.RequestedModel field

* agent: lift escalate_to_model meta into ToolResult.RequestedModel

* agent: perform at-most-one mid-run model switch in the turn loop

* tools: add SideEffectNone for control-only tools

* tools: add escalate_model control tool (metadata + args)

* cli: parse --allow-escalation exec flag

* cli: document --allow-escalation in exec help

* cli: register escalate_model tool under --allow-escalation

* cli: wire model switcher for exec escalation

* cli: guard byte-identical exec run without escalation flag

* cli: attribute post-switch usage to the escalated model

* tools: accept an explicit empty escalate_model reason

reason is an optional, informational-only arg, but the previous arg
parsing rejected an explicit reason:"" (non-empty-string check), failing
the whole escalation. Switch to stringArgWithEmpty so an absent OR empty
reason is accepted while a non-string value is still rejected. Add a test
covering both the empty and absent forms.

* agent: test (nil,nil) escalation switcher keeps original provider

When a ModelSwitcher returns (nil, nil) — no error, no replacement — the
loop's `else if newProvider != nil` guard already keeps the active
provider, so the run continues on the original model. Add an agent-layer
test that proves this (it panics if the nil guard is dropped), closing the
gap the adversarial review flagged.

Also note the deferred compaction limitation near the switch: the
compactor's context-window budget is fixed at run start and not updated on
a mid-run switch (needs a ModelSwitcher contract change to fix).

* cli: gate usage model attribution on the escalation flag and harden tests

FIX A (back-compat): the OnUsage closure wrote a "model" key into every
persisted EventUsage payload unconditionally, diverging a non-escalation
run from the byte-identical origin/main baseline. The model can only
change mid-run under --allow-escalation, so include the "model" key only
then. A new flag-OFF test asserts the persisted usage payload carries no
"model" key (mutation-proven).

Test hardening for the CLI escalation boundary:
- Wire test: each newProvider build returns a DISTINCT provider with its
  own turn counter; assert the source provider served exactly the
  escalation turn and the escalated provider served exactly the
  post-switch answer. Fails if `provider = newProvider` is dropped.
- Mixed usage attribution: emit usage pre- and post-switch and assert the
  first attributes to the original model, the second to the escalated
  target (the loop's ordering guarantee).
- Switch-error path: a failed rebuild is non-fatal and post-error usage
  stays attributed to the original model (currentModel unchanged).
- Top-tier decline end-to-end: a flag-on run that escalates while already
  top-tier performs no switch (newProvider built exactly once) and
  succeeds.

* modelregistry: validate UpgradeTargetID resolves at registry construction

* cli: update currentModel only when the switched provider is non-nil

* sandbox: make 'none' a first-class side effect (control-only tools)

---------

Co-authored-by: Gnanam <[email protected]>
2026-06-08 20:50:32 +05:30

99 lines
3.8 KiB
Go

package tools
import (
"context"
"fmt"
"github.com/Gitlawb/zero/internal/modelregistry"
)
// escalateToModelMetaKey is the result-metadata key the agent loop reads to
// perform a mid-run model switch. The tool itself stays pure: it validates the
// upgrade target and signals it here; the loop owns the provider swap.
const escalateToModelMetaKey = "escalate_to_model"
// escalateModelTool lets the agent escalate the current model to its configured
// stronger upgrade target for the rest of the run. It is a control action: it
// performs no read/write/shell/network effect and only signals the target via
// result metadata. The agent loop wires the actual provider switch.
type escalateModelTool struct {
baseTool
registry modelregistry.Registry
}
// NewEscalateModelTool builds the tool with the default model registry. If the
// default registry cannot be built the tool degrades gracefully: every run then
// reports "no escalation available" rather than failing.
func NewEscalateModelTool() Tool {
registry, err := modelregistry.DefaultRegistry()
if err != nil {
registry = modelregistry.Registry{}
}
return newEscalateModelTool(registry)
}
// newEscalateModelTool builds the tool with an explicit registry (used by the
// exec wiring and tests so escalation resolves against the same catalog the run
// uses).
func newEscalateModelTool(registry modelregistry.Registry) escalateModelTool {
return escalateModelTool{
baseTool: baseTool{
name: "escalate_model",
description: "Escalate the current model to its stronger upgrade target for the rest of this run. Use when the task is harder than expected. No effect if already on the strongest available model.",
parameters: Schema{
Type: "object",
Properties: map[string]PropertySchema{
"reason": {
Type: "string",
Description: "Optional short note on why a stronger model is needed (recorded for transcript context only).",
},
},
Required: []string{},
AdditionalProperties: false,
},
safety: Safety{
SideEffect: SideEffectNone,
Permission: PermissionAllow,
Reason: "Requests a mid-run switch to a stronger model; the agent loop performs the switch.",
AdvertiseInAuto: true,
},
},
registry: registry,
}
}
// Run satisfies the Tool interface for callers that do not supply RunOptions; it
// has no current model to escalate from, so it reports no escalation. The real
// path is RunWithOptions, which the registry invokes when RunOptions are present.
func (tool escalateModelTool) Run(ctx context.Context, args map[string]any) Result {
return tool.RunWithOptions(ctx, args, RunOptions{})
}
// RunWithOptions reads the current model from options.Model, looks up its upgrade
// target, and either signals the target via Meta or returns an informational
// result. Status is OK either way (a "no escalation" outcome is not an error).
func (tool escalateModelTool) RunWithOptions(_ context.Context, args map[string]any, options RunOptions) Result {
// reason is optional and used only for transcript context; reject a non-string
// value but accept an absent OR explicitly empty string (allowEmpty) — reason
// is informational, so an empty reason must never fail the escalation.
if _, err := stringArgWithEmpty(args, "reason", "", false, true); err != nil {
return errorResult("Error: Invalid arguments for escalate_model: " + err.Error())
}
current := options.Model
target, ok := tool.registry.UpgradeTarget(current)
if !ok {
label := current
if label == "" {
label = "current model"
}
return okResult(fmt.Sprintf("Already using the strongest available model (%s); no escalation performed.", label))
}
return Result{
Status: StatusOK,
Output: fmt.Sprintf("Escalating from %s to %s for the rest of this run.", current, target.ID),
Meta: map[string]string{escalateToModelMetaKey: target.ID},
}
}