Files
DeepSeek-Reasonix/cmd
zumacandClaude Opus 5.5 938b20e7a3 fix(e2ebench): count script calls inside their round; keep one trajectory per trial
model_rounds and round_outcomes treated every call run_script made as a
top-level dispatch, so each inner result reopened a model gap and each inner
dispatch closed one. One recorded code_mode run with 24 executor requests
reported a median of 87.5 rounds. Nested calls carry no ParentID — only an id
derived from the caller's, inside its dispatch→result window — so the digest
had nothing structural to tell them apart.

The nested id is now declared in internal/contract/tool (NestedCallID /
IsNestedCallOf) and the agent's nestedInvoker builds ids through it, so the
producer and the reader share one definition. e2ebench's callNesting tracks
open top-level calls and treats subagent calls (ParentID) and invoker calls
(derived id) alike: neither cuts a round, joins a batch or adds tool time.
roundEnds and splitAtCorrect use the same tracker. NestedCalls reports how
many there were, and ModelRounds' doc now says what it counts.

With -trials N every trial wrote <task>.trajectory.jsonl, so only the last
survived and every trial's digest read it. Trials now write
<task>.trial-N.trajectory.jsonl (legs: <task>.trial-N.segM...); a single
trial keeps the old name. sft mode resolves the task id through
trajectoryTaskID, which strips trial and leg marks; traj, serve and barrier
modes glob *.trajectory.jsonl and label each file by its stem, so trials
read as distinct entries.

Cause: the digest had no identity for tool-made calls; the trajectory path
had no trial in it.
Blast radius: cmd/e2ebench digest and file naming; nestedInvoker's id string
is unchanged (same "<parent>.<n>" format, now built by the contract helper).
Neighbouring behaviours tested: subagent ParentID exclusion, segment paths,
answer-isolation audit across legs, existing round decomposition tests.
Why this layer: the id convention is owned where nested calls are created;
the digest reads it rather than restating the format.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 10:56:50 -07:00
..