Files
zero/internal/agent/decode_args_test.go
KRATOS 7fb0092de7 fix: agent runtime resilience — stream stalls, weak-model tool args, and sub-agent reliability (#349)
* fix(local): defeat the macOS stale-pooled-connection hang (transport + stall-retry)

Root cause (you confirmed it: both chatgpt AND ollama hung, only on macOS): providers
fell back to http.DefaultClient, whose pooled keep-alive connections get reused on a
later turn after the server/NAT silently dropped them. The model call is a POST
(non-idempotent), so Go won't auto-retry it on a fresh conn — it blocks forever.
macOS keeps dead pooled conns around longer than Linux, so it reproduced on macOS only.

1. providerio.HTTPClient now returns a shared, tuned transport (ResponseHeaderTimeout
   60s bounds a hung POST that gets no response; IdleConnTimeout 30s stops dead conns
   from lingering to be reused). Routed openai/codex through it too (anthropic/gemini
   already used HTTPClient). Slow first tokens / long reasoning are unaffected — the
   header timeout only bounds time-to-first-response-header, not the stream body.
2. Agent loop now retries a stream idle/stall timeout that produced NO output yet, on
   a fresh connection (capped + backoff). A partial-output turn is NOT retried (would
   duplicate). Recovers any stall that slips past the transport timeout.

Local only — not pushed.

* fix(local): tolerate concatenated tool-arg JSON + widen response-header timeout

1) Weak models (minimax-m3, glm) pack MULTIPLE concatenated top-level JSON objects
   into one tool-call arguments slot ({A}{B}), which json.Unmarshal rejects with
   'invalid character {' after top-level value' (~9 failed tool calls/session).
   New recoverableToolArguments returns the first JSON value ONLY when the payload is
   one value optionally followed by more WHOLE JSON values; it rejects a bad/truncated
   first value OR trailing NON-JSON garbage ({"x":1}xyz) so genuine corruption still
   errors (adversarial-review fix). decodeToolArguments uses it at the central dispatch
   (executeToolCall) so {A}{B} runs the first object; historySafeToolCalls uses the
   SAME helper so the replayed transcript records the recovered first object (not {}),
   keeping dispatch and history consistent. Truly-malformed JSON still hits the
   existing sanitize-to-{} + retry path. Trailing objects are dropped (the model
   re-issues on a later turn).

2) Raise the shared transport ResponseHeaderTimeout 60s -> 120s so a slow cloud proxy
   (ollama *:cloud) that withholds its 200 header until first token isn't wrongly
   aborted; still bounds a truly dead reused connection.

Local only — not pushed.

* feat(local): add /turns command + raise default maxTurns 30->50

Agentic multi-step runs (build + delegate + swarm + MCP + self-verify) routinely hit
the 30-turn ceiling mid-task and never reached the later steps; removing /mode earlier
also left NO interactive way to raise the budget.

- /turns [n]: per-session tool-turn budget setter (commandGroupSession), mirroring
  /selfcorrect. Empty/status shows the current budget; a positive int sets
  agentOptions.MaxTurns (read by the run at model.go); clamped to maxTurnsCeiling=500
  to guard against typos; invalid/non-positive shows usage without changing it.
- defaultMaxTurns 30 -> 50 (matches the old 'deep' preset). Raise per-run with /turns
  for long tasks (e.g. /turns 150 for a full acceptance run).

Local only — not pushed.

* fix(local): swarm reports failed members as failed + sub-agents inherit /turns budget

Issue B — a swarm member whose child exited non-zero (e.g. exit 4 / max-turns) was
recorded [done] (swarm_status '0 failed'), so the orchestrator — which can't see the
AGENTS panel — trusted incomplete work. The specialist launcher ignored
res.Result.Status; it now surfaces a StatusError result as a member FAILURE
(errors.New(report)) so the swarm marks it [failed]. Added Coordinator.FailWithSession
and switched the member watch path to it so a failed member keeps its session id
(still drillable) and its report rides along as the failure message.

Issue A — a member hit its OWN turn limit because /turns raised only the parent's
budget; children re-resolved config.json's default. New config.SetMaxTurnsEnv exports
ZERO_MAX_TURNS, read in applyEnv -> cfg.MaxTurns; /turns now exports it so spawned
sub-agents / swarm members (which inherit the env) run with the same budget. A
non-numeric/zero env value never clobbers the configured budget.

Tests: launcher non-zero-exit -> failed (+ session kept); SetMaxTurnsEnv round-trip
+ garbage-env guard; /turns exports + clamps the env. Full gate green (73 pkg).

Local only — not pushed.

* fix(local): corrective error for unknown specialist name + clamp ZERO_MAX_TURNS

1) A model that invents a specialist name (validator-runner, file-creator,
   file-writer — none registered) got an opaque 'specialist not found' and kept
   spawning doomed sub-agents. loadManifest now lists the available specialists and
   notes that 'worker' has shell/edit while explorer/code-review are read-only, so the
   model self-corrects to a capable specialist. New availableSpecialistList helper.

2) Defense-in-depth (review nit): applyEnv now clamps an inherited ZERO_MAX_TURNS to
   MaxTurnsCeiling, so a raw shell 'export ZERO_MAX_TURNS=999999' can't bypass the
   /turns ceiling. Ceiling promoted to config.MaxTurnsCeiling (single source of truth,
   referenced by the TUI /turns clamp).

Tests: unknown-name error lists worker/explorer/code-review; applyEnv clamps an
over-ceiling env value. Full gate green (73 pkg).

Local only — not pushed.

* fix(local): surface sub-agent kill signal instead of opaque exit -1

When a sub-agent child is terminated by a signal, command.Wait returns an ExitError
whose ExitCode() is -1, losing the cause. Capture ProcessState.String() (portable,
e.g. "signal: killed") into ChildRunResult.Signal and have BuildFinalResult render
it — "Subagent terminated by a signal (signal: killed) — killed before it finished.
Likely causes: the OS out-of-memory killer ... a timeout, or cancellation." — instead
of an opaque "Subagent failed (exit -1)". The message lists causes rather than
asserting OOM, since timeout/cancellation also SIGKILL the child. A captured signal
always surfaces even if a late run_end reported a clean exit. The background task
path (which can't carry the signal name) emits the same hint for a negative exit code.

* address CodeRabbit review on #349

- loop.go: gate stall-retry on a turn-level forwardedAnything flag (wrapped
  callbacks), so a retry never re-streams on top of output already forwarded earlier
  in the turn (e.g. across the reactive-compaction retry). New test covers a
  reasoning-then-stall (collected.Text empty but output forwarded).
- loop.go: route a reissued stream's non-stall error through the SAME recovery as
  the initial stream (image-rejection wrapping / context-limit compaction) via a
  shared recoverStreamError closure, instead of returning it raw.
- exec.go: drop the hardcoded worker/explorer/code-review guidance from the
  unknown-specialist error; rely on the dynamically-rendered available list (a
  custom registry may not have those names).
- streamer.go: kill-signal message lists OOM/timeout/cancellation evenhandedly
  instead of asserting OOM (the branch also covers timeouts/cancellations).
- launcher_specialist.go + exec.go: preserve the child SessionID on a post-start
  failure too, so FailWithSession keeps those failed members drillable.
- model.go: block /turns while a run is active (mid-run change would make the
  inherited ZERO_MAX_TURNS budget inconsistent for later-spawned sub-agents).
2026-07-01 09:15:08 +05:30

100 lines
3.5 KiB
Go

package agent
import "testing"
// decodeToolArguments must (a) decode a normal single-object payload unchanged,
// (b) tolerate a weak model packing multiple concatenated top-level JSON objects
// into one arguments string by decoding the FIRST object and ignoring the rest
// (the minimax-m3 "invalid character '{' after top-level value" failure), and
// (c) still reject a genuinely malformed/truncated first object.
func TestDecodeToolArguments(t *testing.T) {
t.Run("single object decodes normally", func(t *testing.T) {
var a map[string]any
if err := decodeToolArguments(`{"path":"x"}`, &a); err != nil {
t.Fatalf("err = %v, want nil", err)
}
if a["path"] != "x" {
t.Fatalf("decoded %v, want path=x", a)
}
})
t.Run("concatenated objects decode the first, ignore trailing", func(t *testing.T) {
var b map[string]any
if err := decodeToolArguments(`{"path":"first"}{"path":"second"}`, &b); err != nil {
t.Fatalf("multi-object should decode the first, got err = %v", err)
}
if b["path"] != "first" {
t.Fatalf("decoded %v, want the first object (path=first)", b)
}
})
t.Run("distinct tools' args concatenated (real minimax case): first object used", func(t *testing.T) {
var c map[string]any
if err := decodeToolArguments(`{"plan":[1,2]}{"cmd":"go version"}`, &c); err != nil {
t.Fatalf("err = %v, want nil", err)
}
if _, ok := c["plan"]; !ok || c["cmd"] != nil {
t.Fatalf("decoded %v, want only the first object's keys", c)
}
})
t.Run("truncated first object still errors", func(t *testing.T) {
var d map[string]any
if err := decodeToolArguments(`{"path":`, &d); err == nil {
t.Fatal("truncated JSON must still error")
}
})
t.Run("non-JSON trailing garbage still errors (not masked)", func(t *testing.T) {
var g map[string]any
if err := decodeToolArguments(`{"x":1}xyz`, &g); err == nil {
t.Fatal("a valid first object followed by non-JSON garbage must still error")
}
})
t.Run("recoverableToolArguments: only whole-JSON trailing is tolerated", func(t *testing.T) {
cases := []struct {
in string
ok bool
first string
}{
{`{"a":1}`, true, `{"a":1}`},
{`{"a":1}{"b":2}`, true, `{"a":1}`},
{`{"a":1} {"b":2} `, true, `{"a":1}`},
{`{"a":1}xyz`, false, ""},
{`{"a":`, false, ""},
{` `, false, ""},
}
for _, c := range cases {
first, ok := recoverableToolArguments(c.in)
if ok != c.ok || (ok && first != c.first) {
t.Fatalf("recoverableToolArguments(%q) = (%q,%v), want (%q,%v)", c.in, first, ok, c.first, c.ok)
}
}
})
t.Run("historySafeToolCalls recovers the first object, not {}", func(t *testing.T) {
out := historySafeToolCalls([]ToolCall{
{ID: "c1", Name: "read_file", Arguments: `{"path":"README.md"}{"path":"AGENTS.md"}`}, // concatenated -> first
{ID: "c2", Name: "read_file", Arguments: `{"path":"x"}xyz`}, // garbage -> {}
{ID: "c3", Name: "read_file", Arguments: `{"path":"ok"}`}, // valid -> unchanged
})
if out[0].Arguments != `{"path":"README.md"}` {
t.Fatalf("concatenated args should record the first object, got %q", out[0].Arguments)
}
if out[1].Arguments != "{}" {
t.Fatalf("garbage-trailing args should record {}, got %q", out[1].Arguments)
}
if out[2].Arguments != `{"path":"ok"}` {
t.Fatalf("valid args should be unchanged, got %q", out[2].Arguments)
}
})
t.Run("whitespace-only errors (caller guards empty)", func(t *testing.T) {
var e map[string]any
if err := decodeToolArguments(" ", &e); err == nil {
t.Fatal("whitespace-only must error")
}
})
}