mirror of
https://github.com/Twigpine/zero.git
synced 2026-11-10 13:11:31 -10:00
* perf(agent): cheap compaction — dedicated summarizer model, input caps, free reactive prune, window re-derive
Compaction summarization was the single most expensive recurring event
in a long run: the full elided middle — verbatim tool bodies included —
went to the session's main model at full price (a ~100k-token call near
a 200k window, recursively multiplied on overflow).
Four changes, each independently safe:
- Dedicated summarizer model. Compaction summaries route to a cheap
model resolved as ZERO_COMPACTION_MODEL > preferences.compactionModel
> a curated default on official endpoints (Haiku 4.5 / Gemini
Flash-Lite / gpt-4.1-mini; custom-endpoint catalogs are unknowable so
they get none). Built lazily on the first paid compaction; ANY
failure permanently falls back to the main provider for the run, so a
misconfigured summarizer can never break compaction. "main" at any
level forces the old behavior.
- Summarizer input caps. renderTranscript clamps tool-result bodies to
2,000 chars and tool-call arguments to 500 (head+tail, rune-safe) —
the summary needs what was done and decided, not raw payloads. Cuts
summarizer input 5-10x on tool-heavy sessions.
- Free reactive prune. recover() now runs the same prune-first stage as
the proactive path: a context-limit error is first answered by
pruning stale tool bodies at zero cost, without consuming the
one-shot reactive budget; the paid summarizer only runs when a later
error finds nothing left to prune.
- Window re-derivation on model switch. Options.ContextWindowFor
(wired to the model registry in exec and the TUI) lets the loop
re-derive the compaction threshold after escalate_model, closing the
documented KNOWN LIMITATION where a switch to a smaller-window model
could overflow and a larger one over-compacted.
## Rebase notes (2026-08-25, onto 2749447c)
Rebased across 214 commits. main now summarizes through the context planner
(#879) and switches models via turn sessions, so the agent side was reworked
rather than merged textually:
- Summarizer selection lives on compactionState and is resolved lazily per
main model through summarizeWithPlanner (planner + trace preserved); a
failing dedicated summarizer is marked broken and the same slice is
retried once on the main provider.
- Options.Summarizer now receives the main model in force:
`func(ctx, mainModelID string) (Provider, error)`, and may return
(nil, nil) when no dedicated model applies. After a successful
escalate_model switch the loop calls compactor.switchModel, which
re-derives the window and re-resolves the summarizer — a run that started
on the cheap model and escalated regains cheap summaries (review P2).
- recover()'s free prune stage runs before the one-shot reactive guard
(pruning is free and cannot loop) and mirrors maybeCompact's two prune
passes; the free retry is taken only when the pruned history fits the
window the provider enforces (with a 5% estimate margin), otherwise the
same call falls through to the paid summarizer. Gating on the 70%
proactive threshold would have sent histories in the 70-95% band to the
summarizer unnecessarily (review P1 and the follow-up question).
- ContextWindowFor is wired through the turn-session switch path; the
planner's window is updated alongside the compactor's.
Tests: the PR's suite is ported to the new seams, plus insufficient-prune
fall-through, free prune after the budget is spent, window-vs-threshold
gating, and summarizer re-resolution across a model switch.
* fix(agent): branch on a dedicated-summarizer flag instead of comparing providers
Review follow-ups for #532:
- summarizeProvider now reports whether the dedicated summarizer was
selected and summarizeClosure branches on that flag; the previous
`chosen == provider` compared Provider interface values, which panics
when the dynamic type is not comparable. Regression test uses a provider
with a slice field for both the summarizer and the main provider.
- CompactionModelID's curated-default test covers the official OpenAI
profile alongside Anthropic and Google.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012JqvAf7zsP1qYFVZayKXHq
---------
Co-authored-by: Claude Fable 5 <[email protected]>
114 lines
4.7 KiB
Go
114 lines
4.7 KiB
Go
package providers
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"testing"
|
|
|
|
"github.com/Gitlawb/zero/internal/config"
|
|
"github.com/Gitlawb/zero/internal/zeroruntime"
|
|
)
|
|
|
|
func TestCompactionModelIDResolutionOrder(t *testing.T) {
|
|
anthropic := config.ProviderProfile{
|
|
Name: "anthropic",
|
|
ProviderKind: config.ProviderKindAnthropic,
|
|
Model: "claude-sonnet-4.5",
|
|
}
|
|
|
|
t.Setenv(CompactionModelEnv, "my-env-model")
|
|
if got := CompactionModelID(anthropic, "my-config-model"); got != "my-env-model" {
|
|
t.Fatalf("env must win, got %q", got)
|
|
}
|
|
|
|
t.Setenv(CompactionModelEnv, "main")
|
|
if got := CompactionModelID(anthropic, "my-config-model"); got != "" {
|
|
t.Fatalf("env 'main' must force the main model, got %q", got)
|
|
}
|
|
|
|
t.Setenv(CompactionModelEnv, "")
|
|
if got := CompactionModelID(anthropic, "my-config-model"); got != "my-config-model" {
|
|
t.Fatalf("config preference must apply when env unset, got %q", got)
|
|
}
|
|
if got := CompactionModelID(anthropic, "main"); got != "" {
|
|
t.Fatalf("config 'main' must force the main model, got %q", got)
|
|
}
|
|
}
|
|
|
|
func TestCompactionModelIDCuratedDefaults(t *testing.T) {
|
|
t.Setenv(CompactionModelEnv, "")
|
|
|
|
anthropic := config.ProviderProfile{Name: "anthropic", ProviderKind: config.ProviderKindAnthropic, Model: "claude-sonnet-4.5"}
|
|
if got := CompactionModelID(anthropic, ""); got != defaultAnthropicCompactionModel {
|
|
t.Fatalf("anthropic default = %q, want %q", got, defaultAnthropicCompactionModel)
|
|
}
|
|
|
|
google := config.ProviderProfile{Name: "google", ProviderKind: config.ProviderKindGoogle, Model: "gemini-2.5-pro"}
|
|
if got := CompactionModelID(google, ""); got != defaultGoogleCompactionModel {
|
|
t.Fatalf("google default = %q, want %q", got, defaultGoogleCompactionModel)
|
|
}
|
|
|
|
openai := config.ProviderProfile{Name: "openai", ProviderKind: config.ProviderKindOpenAI, Model: "gpt-4.1"}
|
|
if got := CompactionModelID(openai, ""); got != defaultOpenAICompactionModel {
|
|
t.Fatalf("openai default = %q, want %q", got, defaultOpenAICompactionModel)
|
|
}
|
|
|
|
// Custom/compatible endpoints: catalog unknowable, no default.
|
|
compatible := config.ProviderProfile{
|
|
Name: "opengateway",
|
|
ProviderKind: config.ProviderKindOpenAICompatible,
|
|
BaseURL: "https://opengateway.gitlawb.com/v1",
|
|
Model: "some-model",
|
|
}
|
|
if got := CompactionModelID(compatible, ""); got != "" {
|
|
t.Fatalf("openai-compatible must have no default, got %q", got)
|
|
}
|
|
}
|
|
|
|
func TestCompactionModelIDSkipsWhenMainIsAlreadyCheap(t *testing.T) {
|
|
t.Setenv(CompactionModelEnv, "")
|
|
haiku := config.ProviderProfile{
|
|
Name: "anthropic",
|
|
ProviderKind: config.ProviderKindAnthropic,
|
|
Model: defaultAnthropicCompactionModel,
|
|
}
|
|
if got := CompactionModelID(haiku, ""); got != "" {
|
|
t.Fatalf("already-cheap session must not get a dedicated summarizer, got %q", got)
|
|
}
|
|
}
|
|
|
|
func TestCompactionSummarizerFactoryTracksMainModel(t *testing.T) {
|
|
t.Setenv(CompactionModelEnv, "")
|
|
base := config.ProviderProfile{Name: "anthropic", ProviderKind: config.ProviderKindAnthropic, Model: defaultAnthropicCompactionModel}
|
|
var built []string
|
|
newProvider := func(profile config.ProviderProfile) (zeroruntime.Provider, error) {
|
|
built = append(built, profile.Model)
|
|
return nil, errors.New("not needed")
|
|
}
|
|
factory := CompactionSummarizerFactory(base, "", newProvider)
|
|
if factory == nil {
|
|
t.Fatal("factory must exist when a provider builder exists")
|
|
}
|
|
// Main model is the cheap default: no dedicated summarizer, nothing built.
|
|
if provider, err := factory(context.Background(), defaultAnthropicCompactionModel); provider != nil || err != nil || len(built) != 0 {
|
|
t.Fatalf("cheap main model must yield (nil, nil) without building, got %v %v built=%v", provider, err, built)
|
|
}
|
|
// After escalating to an expensive model the cheap default is built.
|
|
if _, err := factory(context.Background(), "claude-sonnet-4-5"); err == nil || len(built) != 1 || built[0] != defaultAnthropicCompactionModel {
|
|
t.Fatalf("escalated main model must build the cheap summarizer, built=%v err=%v", built, err)
|
|
}
|
|
// An empty main model falls back to the base profile's model — the cheap
|
|
// default here, so again nothing is built.
|
|
built = nil
|
|
if provider, err := factory(context.Background(), ""); provider != nil || err != nil || len(built) != 0 {
|
|
t.Fatalf("empty main model must use the base profile, got %v %v built=%v", provider, err, built)
|
|
}
|
|
// The explicit "main" preference disables the dedicated summarizer.
|
|
if provider, err := CompactionSummarizerFactory(base, "main", newProvider)(context.Background(), "claude-sonnet-4-5"); provider != nil || err != nil {
|
|
t.Fatalf("preference main must disable the summarizer, got %v %v", provider, err)
|
|
}
|
|
if CompactionSummarizerFactory(base, "", nil) != nil {
|
|
t.Fatal("no provider builder means no factory")
|
|
}
|
|
}
|