mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
feat(core): add a convergence reminder for read-only exploration (#13601)
* feat(core): add a convergence reminder for read-only exploration
Share an advisory exploration allowance across core and daemon tool loops without adding a halt or changing tool permissions.
Co-authored-by: Qwen-Coder <[email protected]>
* fix(cli): keep halted tool batches out of the exploration budget
recordDaemonToolCalls fed the exploration budget at the top of its
per-call loop, before the per-turn cap and global-duplicate checks. A
batch that those checks halt is skipped whole and never executes, yet
its calls still counted toward the budget and still resolved every tool
name through the registry. Move the recording after both halt checks so
only calls the turn actually runs feed the budget, and a capped turn
performs no registry lookups.
Also complete the client-goal Config double with getMaxToolCallsPerTurn
and getToolRegistry, which the new budget call sites require.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-conflict/jmuy5teh2cj
* fix(core): roll back the exploration budget and turn cap on ModelFallback
A model fallback restarts the attempt from scratch exactly like a retry
(Turn clears pendingToolCalls for both), but the exploration budget's
rollback and LoopDetectionService's per-turn cap rollback both fired on
Retry only. The discarded attempt's read calls stayed counted, inflating
the phase counter by every failed hop of the fallback chain, so the
single reminder (and the cap) could fire before the executed evidence
reached the allowance — the same drift the daemon path avoids by
recording only the surviving batch.
Wire both rollbacks to ModelFallback beside Retry, and pin the core-side
budget wiring (record/commit/rollback/reset, the session-disable gate,
and the duplicate-call-id dedup) with client integration tests that
drive real tool-result continuations; each guard was verified to fail
under its targeted mutation.
Co-authored-by: Qwen-Coder <[email protected]>
* style(core): apply prettier to the new convergence-reminder tests
The Lint lane's whole-repo prettier check was short-circuited by the stale
freshness gate, so these lines were never formatted. Formatting only.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-conflict/jmv2f2dk80w
* fix(core): keep guard evidence across ModelFallback and close review gaps
The review-pr round on 69f2e313 filed one Critical and ten Suggestions;
this addresses the Critical fully, seven Suggestions, and declines two
with rationale.
Critical: the widened per-turn cap rollback cleared the loop detector's
accumulated guard evidence on ModelFallback. A production fallback can
only precede its own attempt's calls (the chain is gated on the failed
attempt yielding no candidate output), so there was nothing to roll back
— but the evidence clears were not inert: the chain is per-request, so a
capacity-limited primary can prefix every round-trip of one turn with
ModelFallback, zeroing the consecutive-identical streak, the
shell-inspection streak and the adaptive cap's repeat tracker on each
round-trip, silently disabling two always-on guards until the 1000-call
backstop. ModelFallback now returns early without touching accumulated
state; the Retry branch keeps its full rollback. Three new
loopDetectionService tests pin the reachable shape (per-round-trip
fallbacks with committed evidence) and redden under the widened branch.
Also: the client budget's lifecycle moves behind a shared
isAttemptRestartEvent predicate (R1-7); its block moves below the
statement the pre-existing comment documents (R1-10); a concurrent side
query no longer resets the running turn's exploration phase (R1-11);
lsp and tool_search classify as exploration reads without retyping
their Kind.Other registration (R1-2a); the unreachable model_fallback
fixture is replaced with a cross-round-trip shape (R1-1 note 3); reset()
gains a commit-floor unit test (R1-4); bridged tool_call args wiring is
pinned in both runtimes (R1-5); the design docs record the threshold
boundary, the subagent-loop exclusion, the budget-local rollback split,
the todo-write cadence consequence and the discovery-tool
classification, in both languages (R1-3, R1-2b); settings.md documents
the reminder at the maxToolCallsPerTurn row (R1-9).
Declined: insertion-site debug logging (R1-6) — the nearest reminder
peer (activeTodoReminder) logs nothing either, and the injected text
already persists in the transcript; reusing the executor's
ensureTool-resolved name for daemon classification (R1-8) — the
asymmetry is self-healing (ensureTool materialises factory targets into
the map the classifier reads), the only divergent shape found is inert
in fork registries, and the perf half is noise by the reviewer's own
measurement. Both are recorded as follow-up candidates.
Co-authored-by: Qwen-Coder <[email protected]>
* style(core): apply prettier to the convergence-reminder files
Run Prettier reported style issues in 5 files touched by this PR
(docs/users/configuration/settings.md, client.test.ts,
loopDetectionService.ts, loopDetectionService.test.ts,
tool-exploration-budget.test.ts), which failed the Lint & Static job.
Formatting only, no behavior change.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-conflict/jmv2xa8pg1q
* fix(core): isolate the exploration budget per interaction and close review gaps
* fix(core): commit the exploration floor on an in-round-trip phase reset
`record()`'s non-read branch zeroed `calls`/`reminded` without committing the
new floor, so a `rollback()` later in the same round-trip resurrected the
previous phase's `committedCalls`/`committedReminded`. The retried attempt was
then nagged after a single read, and the resurrected reminder latch was
re-committed by every later `Finished`, silencing the reminder for the rest of
the interaction. Commit the floor here, as `reset()` already does.
Also stop crediting a bridge-refused discovery target as a read: `tool_call`
unconditionally refuses `tool_search` and itself, so a bridged call to either
performs no read and must stay unclassified like a name the registry refuses.
A bridged `lsp` still classifies as read.
The strengthened rollback assertion is red on the previous code and green with
the commit; the new bridge case is red without the refusal guard.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmv41akk23g
* fix(core): drop the duplicate Goal-branch exploration budget reset
The Goal branch's toolExplorationBudget.reset() was the only budget
entrypoint with no ownership or side-query gate, contradicting the
invariant the design docs state for every other one. Goal is a member of
startsInteraction, so the interaction-start block already reset the budget
and claimed ownership for every non-side-query Goal submission, and nothing
records into the budget between the two points - making this a second no-op
reset. For a Goal submitted as a concurrent side query the
interaction-start reset is correctly skipped, but this one still fired and
zeroed the running turn's read-only phase, so that turn never converged.
Deleting the duplicate removes the ungated entrypoint entirely rather than
adding a second copy of the guard.
Also pin the Stop-hook restart reset, which no test exercised: the nested
restart is submitted with SendMessageType.Hook, which is not an
interaction-starting type, so that reset is the only thing clearing the
read-only phase across a hook restart.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmv45kw3b3o
* fix(core): drop the inert ModelFallback arm from checkAlwaysOnSafeties
The arm returned false with no mutation between it and the fall-through
`if (event.type !== LlmEventType.ToolCallRequest) return false;`, and
ModelFallback is not a ToolCallRequest, so it already returned false at the
same point with zero mutations. It could not change behaviour.
Delete it and re-point the two references that cited it as a detector-side
pin: the isAttemptRestartEvent docstring (now names the non-ToolCallRequest
early return that actually carries the no-op) and the client.test.ts fallback
comment. The three loopDetectionService.test.ts ModelFallback cases stay: the
invariant they assert (a per-round-trip fallback must not clear committed
streak evidence) is real and documented in the design doc, and they now pin it
at the tier that carries it instead of at dead code.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmv49v7nr3w
* refactor(core): delegate the exploration-budget phase reset to reset()
record()'s non-read branch was a byte-identical copy of reset()'s body,
with a comment asserting the equivalence instead of the code expressing
it. The two copies already diverged once: reset() committed the zeroed
floor while the in-round-trip copy did not, and that divergence was
R3-1, a Critical in which a rollback after a mid-round-trip phase change
resurrected the previous phase's count and its delivered-reminder latch.
Delegate so a future phase field cannot be added to one site only.
Addresses review finding R4-1.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmv585s4o5m
* test(core): cover the goal-permitted Stop-hook exploration-phase reset
The goal-permit branch restart reset at client.ts:5354 had zero test
coverage. Its !goalPermit sibling at :5463 is pinned by "gives a blocking
Stop hook continuation a fresh exploration phase", but no test installed a
goal runtime alongside a blocking Stop hook, so deleting this line changed
nothing any test observed. The nested restart is submitted with
type: SendMessageType.Hook, which is not an interaction-starting type, so
the in-branch reset is the only thing that clears the phase.
Extends the existing fixture rather than building a new one. Verified
discriminating: deleting client.ts:5354 turns this case red
(expected 2 to be +0) while the !goalPermit sibling stays green.
Addresses review finding R3-5.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmv585s4o5m
---------
Co-authored-by: Qwen-Coder <[email protected]>
This commit is contained in:
1 parent
d8558bba01
commit
26c3fb7a5a
12 files changed
+1376
-7
No files matched your search
@@ -0,0 +1,31 @@
|
||||
# Read-only exploration convergence reminder
|
||||
|
||||
[English](tool-exploration-convergence.md) | [简体中文](tool-exploration-convergence.zh-CN.md)
|
||||
|
||||
## Problem and scope
|
||||
|
||||
Issue #13321 requests an actionable convergence point when diverse discovery calls succeed without advancing a deliverable. The adaptive default tool-call cap permits useful diverse calls past its soft threshold. This proposal adds a reminder at that threshold; it does not add a new halt, infer task intent, force writes, or change permissions. Repeated execution failures and their operator rollout policy in #10887 remain separate.
|
||||
|
||||
The threshold equals `model.maxToolCallsPerTurn` (default 100) and counts one continuous read-only phase within a logical turn, so the motivating incident's observed call count (about 92 mixed calls, whose read phases were repeatedly reset by interleaved non-read calls) does not by itself guarantee a reminder.
|
||||
|
||||
## Behavior
|
||||
|
||||
Track a continuous phase of calls whose registered kind is `Read`, `Search`, or `Fetch`; `lsp` and `tool_search` are counted as exploration reads despite registering `Kind.Other` (retyping them would change scheduler concurrency semantics). Resolve bridged `tool_call` targets through the registry, so declared read-only MCP tools do not depend on their names. Planning, implementation, delegation and unknown kinds end the phase. At `model.maxToolCallsPerTurn` calls, insert one reminder after the tool results asking the model to use gathered information, proceed toward the requested deliverable or explain a blocker. For read-only requests, ask for findings and remaining questions. The reminder grants no new tool permission or plan approval.
|
||||
|
||||
Keep the existing adaptive and explicit hard-cap semantics. A disabled/infinite allowance produces no reminder. Continued legitimate investigation remains allowed; the reminder does not assert that every read is stagnation. Another reminder is possible only after a non-read phase or a new logical turn. The interactive loop-detector disable also suppresses the core reminder.
|
||||
|
||||
## Ownership and compatibility
|
||||
|
||||
Core owns one budget alongside its existing logical-turn loop detector, and binds it to the prompt id of the interaction that opened it: the phase belongs to that interaction, and every budget entrypoint (`record`, `commit`, `rollback`, `takeReminder` and the per-turn resets) is a no-op for a stream belonging to another one. A concurrent side query (`/btw`) therefore neither zeroes, inflates nor drains the running turn's phase: it reaches the same client, and its tool-result continuation is submitted without the side-query flag, so the prompt-id binding rather than the flag is what isolates it. Retry and model fallback roll back uncommitted call observations, provider duplicate call IDs are counted once, and existing user/Goal/Stop turn resets clear the budget. ACP owns a budget in each daemon tool-loop state, shared by that loop's tool-result continuations. Foreground, channel, Stop/todo, cron and notification loops instantiate that state. The subagent reasoning loop is excluded: it drives `LlmChat` directly and never enters the client or daemon tool loops that record and inject, so a runaway subagent remains bounded by its existing per-turn cap only. Older callers constructing the exported state shape without the optional budget keep their previous behavior. No state is shared between sessions and no arguments, result contents or identifiers are retained in the budget.
|
||||
|
||||
The rollback extension is budget-local. The loop detector's own per-turn cap rollback stays Retry-only: a production model fallback can only precede its own attempt's calls (the chain runs only when the failed attempt yielded no candidate output), so there is nothing to roll back, and clearing the detector's accumulated streak evidence per round-trip would silently disable the consecutive-identical and stagnation guards for turns served through repeated fallbacks.
|
||||
|
||||
The counter observes emitted/accepted call kinds rather than semantically proving successful progress. A non-read request resets the phase even if its execution later fails; the existing failure protections retain responsibility for that case. Three consequences are accepted: bookkeeping tools (`todo_write`, memory) reset the phase; the runtime's own active-todo nudge re-injects every 3 tool turns, so a todo-maintaining session effectively never accumulates 100 consecutive read-kind calls — that variant of #13321 is out of this feature's coverage; and in `tools.codeModeOnly` sessions `exec` is the only model-visible tool outside the direct-only set, it is constructed `Kind.Other` (so every call resets the phase) while the reads a script performs through `tools.read_file(...)` never surface as `ToolCallRequest` events at all, so that mode effectively never accumulates a full read-only phase — recording an exec script's inner calls from the code-mode runtime is a larger change and is out of scope here. A model may ignore the reminder, so a controlled provider can verify insertion and runtime behavior but cannot establish production-model convergence or token savings.
|
||||
|
||||
## Validation and acceptance
|
||||
|
||||
Verify threshold, one-shot behavior, implementation/planning resets, disabled allowance and retry rollback with focused tests. Verify arbitrary and bridged registered MCP kinds. Use actual CLI/tmux with distinct real file reads and capture the outgoing reminder, continued read-only tool availability, natural completion and any auxiliary model requests separately. Native evidence must be pinned to executed source/build identities. Existing cap and permission checks remain authoritative. Keep a productive phase and a legitimate broad read-only investigation valid without forced writes.
|
||||
|
||||
## Open boundary
|
||||
|
||||
The reminder is a deterministic insertion with nondeterministic model response. A stronger semantic completion gate would require a separate declared task/progress contract; this proposal does not invent one.
|
||||
@@ -0,0 +1,31 @@
|
||||
# 只读探索收敛提醒
|
||||
|
||||
[English](tool-exploration-convergence.md) | [简体中文](tool-exploration-convergence.zh-CN.md)
|
||||
|
||||
## 问题与范围
|
||||
|
||||
#13321 要求在多样化发现调用成功、却没有推进交付物时提供可执行的收敛点。现有自适应默认工具调用上限允许有用的多样调用超过软阈值。本提案在该阈值追加提醒,不增加新的停止条件、不推断任务意图、不强迫写入,也不改变权限。#10887 的重复执行失败及其运维灰度策略继续单独跟进。
|
||||
|
||||
阈值等于 `model.maxToolCallsPerTurn`(默认 100),统计的是单个逻辑轮次内的一段连续只读阶段,因此动机事件观测到的调用数(约 92 次混合调用,其只读阶段被穿插的非只读调用反复重置)并不保证触发提醒。
|
||||
|
||||
## 行为
|
||||
|
||||
记录注册 kind 为 `Read`、`Search` 或 `Fetch` 的连续调用阶段;`lsp` 与 `tool_search` 虽注册为 `Kind.Other`,仍按探索读取计数(重新标注类型会改变调度器并发语义)。通过 registry 解析 `tool_call` 的桥接目标,因此声明为只读的 MCP 工具不依赖名称匹配。规划、实现、委派及未知 kind 结束该阶段。达到 `model.maxToolCallsPerTurn` 次后,在工具结果之后插入一次提醒:使用已收集信息,推进请求的交付物或解释具体阻塞。对于只读请求,应给出发现和剩余问题。提醒不授予工具新权限,也不替代规划批准。
|
||||
|
||||
保留现有自适应和显式硬上限语义。禁用或无限 allowance 不产生提醒。合法的进一步调查仍可继续,提醒不把每次读取都判定为无进展。只有进入非只读阶段或新的逻辑轮次后,才可能再次提醒。交互式 loop-detector 的会话禁用选择也会抑制 core 提醒。
|
||||
|
||||
## 归属与兼容
|
||||
|
||||
core 在现有逻辑轮次 loop detector 旁持有一份 budget,并将其绑定到开启该交互的 prompt id:这一阶段属于该交互,`record`、`commit`、`rollback`、`takeReminder` 以及各轮次 reset 等所有 budget 入口,对于属于其他交互的流一律为空操作。因此并发的插入式提问(`/btw`)既不会清零、也不会虚增或消耗正在运行轮次的阶段:它走到同一个 client,而它的工具结果续轮提交时不带旁路标记,所以真正起隔离作用的是 prompt id 绑定,而不是该标记。Retry 与 model fallback 回滚未提交的调用观测,provider 重复 call ID 只计一次,现有 user/Goal/Stop 轮次 reset 同时清除 budget。ACP 在每个 daemon tool-loop state 内持有 budget,该循环的工具结果续跑共享它。foreground、channel、Stop/todo、cron、notification 循环均会创建该 state。subagent 推理循环被排除在外:它直接驱动 `LlmChat`,不经过记录与注入 budget 的 client 或 daemon 工具循环,因此失控的 subagent 仍只受其既有每轮上限约束。旧调用方自行构造导出的 state、没有可选 budget 时保留原行为。不同 session 不共享状态,budget 不保存参数、结果正文或标识符。
|
||||
|
||||
该回滚扩展仅作用于 budget。loop detector 自身的每轮上限回滚仍只针对 Retry:生产环境的 model fallback 只可能先于自己尝试的调用出现(链路仅在失败尝试未产生任何 candidate output 时启动),没有可回滚的内容,而按轮次清空检测器累积的连击证据,会让经由反复 fallback 服务的轮次静默失去连续相同调用与停滞保护。
|
||||
|
||||
计数观测发出或接纳的调用 kind,不从语义上证明成功推进。非只读请求即使后来执行失败,也会结束该阶段;此情况仍由现有失败保护负责。三个后果是被接受的:记账类工具(`todo_write`、memory)会重置阶段;运行时自身的 active-todo 提示每 3 个工具轮次重新注入一次,因此维护待办清单的会话实际上无法累积 100 次连续读取类调用——#13321 的这一变体不在本功能覆盖范围内;而在 `tools.codeModeOnly` 会话中,除 direct-only 集合外对模型可见的工具只有 `exec`,它被构造为 `Kind.Other`(因此每次调用都会重置阶段),脚本内部通过 `tools.read_file(...)` 执行的读取根本不会以 `ToolCallRequest` 事件形式出现,所以该模式几乎不可能累积出一个完整的只读阶段——若要让它可达,需要由 code-mode 运行时记录 exec 脚本内部的调用,那是更大的改动,不在本 PR 范围内。模型可能忽略提醒,因此受控 provider 能验证插入与运行行为,不能证明生产模型已经收敛或节省 token。
|
||||
|
||||
## 验证与验收
|
||||
|
||||
使用定向测试验证阈值、一次性行为、实现/规划重置、禁用 allowance 以及 Retry 回滚。验证任意名称与桥接的注册 MCP kind。实际 CLI/tmux 使用不同文件的真实读取,捕获模型请求中的提醒、仍可用的只读工具、自然结束,并单独识别附加模型请求。原生证据必须绑定执行源码和构建产物身份。现有调用上限和权限检查仍有最终约束力。保持有效实现阶段和合法的大范围只读调查可继续,不强制写入。
|
||||
|
||||
## 开放边界
|
||||
|
||||
提醒插入是确定性的,模型回应不是。更强的语义完成门槛需要单独声明任务/进展契约,本提案不自行创造该契约。
|
||||
@@ -203,7 +203,7 @@ These settings are read from operator scopes only (User, System, and SystemDefau
|
||||
| `model.chatCompression.screenshotTriggerThreshold` | number | Tool-returned image count at or above which the screenshot trigger fires (only when `enableScreenshotTrigger`). Compaction resets the count — surviving images are re-embedded as top-level parts, which the trigger doesn't count — so it won't immediately re-fire. Env override: `QWEN_COMPACT_SCREENSHOT_THRESHOLD`. | `20` |
|
||||
| `model.skipNextSpeakerCheck` | boolean | Skip the next speaker check. | `true` |
|
||||
| `model.skipLoopDetection` | boolean | Disables streaming loop detection checks. Defaults to `true` (loop detection is skipped) to avoid false positives interrupting legitimate workflows. Set to `false` to re-enable streaming loop detection — useful as a guardrail in headless / non-interactive runs where stuck repetition can otherwise waste budget. In daemon/ACP sessions, which run none of the other streaming detectors, re-enabling also activates a global-duplicate tool-call halt; the always-on per-turn tool-call cap and an invalid-tool-params stagnation guard run there regardless of this setting. | `true` |
|
||||
| `model.maxToolCallsPerTurn` | integer | Per-turn tool-call cap (one model turn plus its tool-result continuations; blocking Stop-hook continuations and runtime-scheduled Goal turns each start a fresh budget). When set explicitly, this value is a hard cap: the turn halts on the next tool call after it is reached (the released behavior). When left unset (default 100), the cap is adaptive: once the turn exceeds 100 it halts only when the model keeps repeating the same call (a stuck loop); a productive turn (diverse calls) continues up to a hard backstop of 1000, which always halts. The adaptive default applies to the interactive TUI, non-interactive (`-p` / JSON / stream-JSON) core-client runs, and daemon/ACP sessions alike. Daemon/ACP sessions evaluate the cap once per tool batch, before execution: a batch that would cross an explicit cap or the hard backstop is skipped whole, so a turn never executes past either (it can halt up to one batch short), while the adaptive soft cap is exceeded by design, up to the backstop. They also have no in-session disable. Always-on circuit breaker against runaway turns, independent of `model.skipLoopDetection`. Set to `0` or a negative value to disable the cap. Choosing "Disable loop detection for this session" in the loop-detected dialog also suppresses it for the rest of the session. | `100` |
|
||||
| `model.maxToolCallsPerTurn` | integer | Per-turn tool-call cap (one model turn plus its tool-result continuations; blocking Stop-hook continuations and runtime-scheduled Goal turns each start a fresh budget). When set explicitly, this value is a hard cap: the turn halts on the next tool call after it is reached (the released behavior). When left unset (default 100), the cap is adaptive: once the turn exceeds 100 it halts only when the model keeps repeating the same call (a stuck loop); a productive turn (diverse calls) continues up to a hard backstop of 1000, which always halts. The adaptive default applies to the interactive TUI, non-interactive (`-p` / JSON / stream-JSON) core-client runs, and daemon/ACP sessions alike. Daemon/ACP sessions evaluate the cap once per tool batch, before execution: a batch that would cross an explicit cap or the hard backstop is skipped whole, so a turn never executes past either (it can halt up to one batch short), while the adaptive soft cap is exceeded by design, up to the backstop. They also have no in-session disable. Always-on circuit breaker against runaway turns, independent of `model.skipLoopDetection`. At this threshold a long continuous read-only phase also receives a one-shot convergence reminder (asking the model to consolidate gathered information or state a blocker); it grants no new permission, and only the first non-read tool or a new turn can make another one possible. Set to `0` or a negative value to disable the cap. Choosing "Disable loop detection for this session" in the loop-detected dialog also suppresses both the cap and the reminder for the rest of the session. | `100` |
|
||||
| `model.skipStartupContext` | boolean | Skips sending the startup workspace context (environment summary and acknowledgement) at the beginning of each session. Enable this if you prefer to provide context manually or want to save tokens on startup. | `false` |
|
||||
| `model.enableOpenAILogging` | boolean | Enables logging of OpenAI API calls for debugging and analysis. When enabled, API requests and responses are logged to JSON files. | `false` |
|
||||
| `model.openAILoggingDir` | string | Custom directory path for OpenAI API logs. If not specified, defaults to `logs/openai` in the current working directory. Supports absolute paths, relative paths (resolved from current working directory), and `~` expansion (home directory). | `undefined` |
|
||||
|
||||
@@ -21335,6 +21335,167 @@ describe('Session', () => {
|
||||
.mockResolvedValueOnce(createEmptyStream());
|
||||
};
|
||||
|
||||
it('adds one exploration reminder after read results and permits further investigation', async () => {
|
||||
const execute = installFailingTool();
|
||||
execute.mockResolvedValue({
|
||||
llmContent: 'file data',
|
||||
returnDisplay: 'file data',
|
||||
});
|
||||
mockToolRegistry.getTool.mockReturnValue({
|
||||
...mockToolRegistry.getTool('failing_tool'),
|
||||
kind: core.Kind.Read,
|
||||
});
|
||||
mockToolRegistry.getAllToolNames.mockReturnValue(['failing_tool']);
|
||||
mockConfig.getMaxToolCallsPerTurn = vi.fn().mockReturnValue(2);
|
||||
mockConfig.isMaxToolCallsPerTurnExplicit = vi
|
||||
.fn()
|
||||
.mockReturnValue(false);
|
||||
mockChat.sendMessageStream = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(streamForBatch(1, 2))
|
||||
.mockResolvedValueOnce(streamForBatch(2, 1))
|
||||
.mockResolvedValueOnce(createEmptyStream());
|
||||
await expect(
|
||||
session.prompt({
|
||||
sessionId: 'test-session-id',
|
||||
prompt: [
|
||||
{ type: 'text', text: 'investigate the files without writing' },
|
||||
],
|
||||
}),
|
||||
).resolves.toMatchObject({ stopReason: 'end_turn' });
|
||||
expect(execute).toHaveBeenCalledTimes(3);
|
||||
expect(
|
||||
sentText().filter((text) =>
|
||||
text.includes('read-only exploration phase'),
|
||||
),
|
||||
).toHaveLength(1);
|
||||
const followUp = vi.mocked(mockChat.sendMessageStream).mock
|
||||
.calls[1]?.[1] as { message: Part[] };
|
||||
expect(
|
||||
followUp.message.filter((part) => part.functionResponse),
|
||||
).toHaveLength(2);
|
||||
expect(followUp.message.at(-1)?.text).toContain(
|
||||
'does not authorize writes',
|
||||
);
|
||||
});
|
||||
|
||||
it('counts a bridged read-only MCP tool toward the exploration budget', async () => {
|
||||
// A bridged call arrives as `tool_call` with the real target in
|
||||
// `args.name`; the classification must resolve the target through the
|
||||
// registry, not fall back to the bridge tool itself (Kind.Other),
|
||||
// or every MCP call would reset the phase.
|
||||
const execute = installFailingTool();
|
||||
execute.mockResolvedValue({
|
||||
llmContent: 'dataset schema',
|
||||
returnDisplay: 'dataset schema',
|
||||
});
|
||||
const bridgeBuild = (envelope: Record<string, unknown>) => ({
|
||||
params: {
|
||||
name: envelope['name'],
|
||||
arguments: envelope['arguments'] ?? {},
|
||||
},
|
||||
execute,
|
||||
getDefaultPermission: vi.fn().mockResolvedValue('allow'),
|
||||
getDescription: vi.fn().mockReturnValue('bridge'),
|
||||
toolLocations: vi.fn().mockReturnValue([]),
|
||||
});
|
||||
const targetTool = {
|
||||
name: 'mcp__srv__describe',
|
||||
kind: core.Kind.Read,
|
||||
displayName: 'Describe Dataset',
|
||||
description: 'describes a dataset schema',
|
||||
build: vi.fn().mockReturnValue({
|
||||
params: {},
|
||||
execute,
|
||||
getDefaultPermission: vi.fn().mockResolvedValue('allow'),
|
||||
getDescription: vi.fn().mockReturnValue('Describe Dataset'),
|
||||
toolLocations: vi.fn().mockReturnValue([]),
|
||||
}),
|
||||
canUpdateOutput: false,
|
||||
isOutputMarkdown: true,
|
||||
};
|
||||
const bridgeTool = {
|
||||
name: 'tool_call',
|
||||
kind: core.Kind.Other,
|
||||
displayName: 'Tool Call',
|
||||
description: 'bridge',
|
||||
build: vi.fn(bridgeBuild),
|
||||
canUpdateOutput: false,
|
||||
isOutputMarkdown: true,
|
||||
};
|
||||
mockToolRegistry.getTool.mockImplementation((name: string) =>
|
||||
name === 'tool_call'
|
||||
? bridgeTool
|
||||
: name === 'mcp__srv__describe'
|
||||
? targetTool
|
||||
: name === 'tool_search'
|
||||
? ({ name: 'tool_search' } as never)
|
||||
: undefined,
|
||||
);
|
||||
mockToolRegistry.getAllToolNames.mockReturnValue([
|
||||
'tool_call',
|
||||
'tool_search',
|
||||
'mcp__srv__describe',
|
||||
]);
|
||||
mockToolRegistry.isDeferredAndHidden.mockImplementation(
|
||||
(name: string) => name === 'mcp__srv__describe',
|
||||
);
|
||||
mockToolRegistry.ensureTool.mockImplementation(async (name: string) =>
|
||||
name === 'tool_call'
|
||||
? (bridgeTool as never)
|
||||
: name === 'mcp__srv__describe'
|
||||
? (targetTool as never)
|
||||
: undefined,
|
||||
);
|
||||
mockConfig.getMaxToolCallsPerTurn = vi.fn().mockReturnValue(2);
|
||||
mockConfig.isMaxToolCallsPerTurnExplicit = vi
|
||||
.fn()
|
||||
.mockReturnValue(false);
|
||||
const bridgedCall = (batch: number, index: number): FunctionCall => ({
|
||||
id: `bridge_${batch}_${index}`,
|
||||
name: 'tool_call',
|
||||
args: { name: 'mcp__srv__describe', arguments: { dataset: index } },
|
||||
});
|
||||
mockChat.sendMessageStream = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
createStreamWithChunks([
|
||||
{
|
||||
type: core.StreamEventType.CHUNK,
|
||||
value: {
|
||||
functionCalls: [bridgedCall(1, 0), bridgedCall(1, 1)],
|
||||
},
|
||||
},
|
||||
]),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
createStreamWithChunks([
|
||||
{
|
||||
type: core.StreamEventType.CHUNK,
|
||||
value: { functionCalls: [bridgedCall(2, 0)] },
|
||||
},
|
||||
]),
|
||||
)
|
||||
.mockResolvedValueOnce(createEmptyStream());
|
||||
await expect(
|
||||
session.prompt({
|
||||
sessionId: 'test-session-id',
|
||||
prompt: [
|
||||
{
|
||||
type: 'text',
|
||||
text: 'inspect the datasets without writing',
|
||||
},
|
||||
],
|
||||
}),
|
||||
).resolves.toMatchObject({ stopReason: 'end_turn' });
|
||||
expect(execute).toHaveBeenCalledTimes(3);
|
||||
expect(
|
||||
sentText().filter((text) =>
|
||||
text.includes('read-only exploration phase'),
|
||||
),
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('defaults an invalid operator mode to shadow and records a warning', () => {
|
||||
debugLoggerWarnSpy.mockClear();
|
||||
const previous = process.env[guardModeEnv];
|
||||
|
||||
@@ -489,6 +489,11 @@ import {
|
||||
type RepeatedToolFailureGuardDecision,
|
||||
type RepeatedToolFailureGuardState,
|
||||
} from './repeated-tool-failure-guard.js';
|
||||
import {
|
||||
getToolExplorationKind,
|
||||
TOOL_EXPLORATION_REMINDER,
|
||||
ToolExplorationBudget,
|
||||
} from '@qwen-code/qwen-code-core/services/tool-exploration-budget.js';
|
||||
|
||||
const debugLogger = createDebugLogger('SESSION');
|
||||
const MAX_RETAINED_SESSION_ROUTE_COUNTS = 8;
|
||||
@@ -842,6 +847,7 @@ export type DaemonToolLoopState = {
|
||||
loopType?: LoopType;
|
||||
repeatedToolFailureMode: RepeatedToolFailureGuardMode;
|
||||
repeatedToolFailureState: RepeatedToolFailureGuardState;
|
||||
explorationBudget?: ToolExplorationBudget;
|
||||
};
|
||||
|
||||
const DAEMON_INVALID_TOOL_PARAMS_THRESHOLD = 3;
|
||||
@@ -923,6 +929,7 @@ function createDaemonToolLoopState(
|
||||
loopDetected: false,
|
||||
repeatedToolFailureMode,
|
||||
repeatedToolFailureState: createRepeatedToolFailureGuardState(),
|
||||
explorationBudget: new ToolExplorationBudget(),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1146,6 +1153,18 @@ function recordDaemonToolCalls(
|
||||
loopState,
|
||||
);
|
||||
}
|
||||
// Deliberately after both halt checks: a halted batch is skipped whole and
|
||||
// never executes, so its calls must not feed the exploration budget (nor
|
||||
// trigger the registry lookups classifying them needs).
|
||||
for (const call of calls) {
|
||||
loopState.explorationBudget?.record(
|
||||
getToolExplorationKind(
|
||||
config.getToolRegistry(),
|
||||
call.name ?? '',
|
||||
call.args ?? {},
|
||||
),
|
||||
);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
@@ -9454,6 +9473,11 @@ export class Session implements SessionContext {
|
||||
const parts = [
|
||||
...toolRun.parts,
|
||||
...(activeTodoReminder ? [{ text: activeTodoReminder }] : []),
|
||||
...(toolLoopState.explorationBudget?.takeReminder(
|
||||
this.config.getMaxToolCallsPerTurn(),
|
||||
)
|
||||
? [{ text: TOOL_EXPLORATION_REMINDER }]
|
||||
: []),
|
||||
...(repeatedToolFailureDecision.kind === 'warn'
|
||||
? [{ text: REPEATED_TOOL_FAILURE_REMINDER }]
|
||||
: []),
|
||||
|
||||
@@ -273,6 +273,11 @@ function setupGoalClient() {
|
||||
getModel: vi.fn(() => 'test-model'),
|
||||
getSkipNextSpeakerCheck: vi.fn(() => false),
|
||||
getSkipLoopDetection: vi.fn(() => false),
|
||||
getMaxToolCallsPerTurn: vi.fn(() => Number.POSITIVE_INFINITY),
|
||||
getToolRegistry: vi.fn(() => ({
|
||||
getAllToolNames: () => [],
|
||||
getTool: () => undefined,
|
||||
})),
|
||||
startActiveTodoWorkChain: vi.fn(),
|
||||
startAutomaticActiveTodoWorkChain: vi.fn(),
|
||||
endAutomaticActiveTodoWorkChain: vi.fn(),
|
||||
|
||||
@@ -124,6 +124,7 @@ import {
|
||||
import { collectAvailableSkillEntries } from '../tools/skill-utils.js';
|
||||
import type { AvailableSkillEntry } from '../tools/skill-utils.js';
|
||||
import { ToolNames } from '../tools/tool-names.js';
|
||||
import { Kind } from '../tools/tools.js';
|
||||
import {
|
||||
DEFERRED_TOOL_CALL_CANCELLATION_PREFIX,
|
||||
DEFERRED_TOOL_CALL_REFUSAL_PREFIX,
|
||||
@@ -684,6 +685,7 @@ describe('Gemini Client (client.ts)', () => {
|
||||
| 'getDeferredToolSummary'
|
||||
| 'getMcpServerInstructions'
|
||||
| 'getTool'
|
||||
| 'getAllToolNames'
|
||||
| 'isDeferredToolRevealed'
|
||||
| 'isPermissionDeferred'
|
||||
| 'revealDeferredTool'
|
||||
@@ -8384,6 +8386,524 @@ Other open files:
|
||||
// The pending queue is fully cleared on halt, same as the turn cap.
|
||||
expect(returnedTurn?.pendingToolCalls).toHaveLength(0);
|
||||
});
|
||||
/** Text parts in `request` carrying the exploration reminder. */
|
||||
const explorationReminders = (request: unknown[]) =>
|
||||
request.filter(
|
||||
(part) =>
|
||||
typeof part === 'object' &&
|
||||
part !== null &&
|
||||
'text' in part &&
|
||||
typeof part.text === 'string' &&
|
||||
part.text.includes('read-only exploration phase'),
|
||||
);
|
||||
|
||||
it('delivers one exploration reminder per read-only phase across tool-result continuations', async () => {
|
||||
// Drives the real client-side wiring of ToolExplorationBudget:
|
||||
// record on ToolCallRequest, commit on Finished, takeReminder on the
|
||||
// next ToolResult continuation, and reset on a fresh interaction.
|
||||
const promptId = 'prompt-exploration-budget';
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(2);
|
||||
// The production default: an unset cap is a soft allowance with the
|
||||
// adaptive backstop, not the suite's explicit hard cap. Under the hard
|
||||
// cap the continuations below are halted, so the reminder would never
|
||||
// reach a request that streams.
|
||||
vi.mocked(mockConfig.isMaxToolCallsPerTurnExplicit).mockReturnValue(
|
||||
false,
|
||||
);
|
||||
// The default registry mock knows no tools, so read_file would
|
||||
// classify as unknown and reset the phase on every call.
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file' ? ({ kind: Kind.Read } as never) : null,
|
||||
);
|
||||
// Distinct args so the consecutive-identical guard never fires.
|
||||
const readTurn = (n: number, count: number) => {
|
||||
const events: unknown[] = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
events.push(
|
||||
toolCallRequest(`call-${n}-${i}`, 'read_file', {
|
||||
path: `f${n}-${i}`,
|
||||
}),
|
||||
);
|
||||
}
|
||||
events.push(stopped());
|
||||
mockTurnRunFn.mockReturnValueOnce(turnStream(...events));
|
||||
};
|
||||
|
||||
installChat();
|
||||
// First interaction: 2 reads reach the allowance of 2.
|
||||
readTurn(1, 2);
|
||||
await run([{ text: 'explore' }], promptId);
|
||||
|
||||
// Continuation 1: the reminder rides after the functionResponse parts.
|
||||
readTurn(2, 1);
|
||||
const cont1Events = await run(
|
||||
[fnResponse('read_file', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
const reminderRequest = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
const reminderIndex = reminderRequest.findIndex(
|
||||
(part) =>
|
||||
typeof part === 'object' &&
|
||||
part !== null &&
|
||||
'text' in part &&
|
||||
typeof part.text === 'string' &&
|
||||
part.text.includes('read-only exploration phase'),
|
||||
);
|
||||
expect(reminderIndex).toBeGreaterThanOrEqual(0);
|
||||
const responseIndex = reminderRequest.findIndex(
|
||||
(part) =>
|
||||
typeof part === 'object' &&
|
||||
part !== null &&
|
||||
'functionResponse' in part,
|
||||
);
|
||||
expect(reminderIndex).toBeGreaterThan(responseIndex);
|
||||
|
||||
// Continuation 2: still the same phase and already reminded — no
|
||||
// second reminder even though the call count keeps growing.
|
||||
readTurn(3, 2);
|
||||
const cont2Events = await run(
|
||||
[fnResponse('read_file', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
const afterReminderRequest = mockTurnRunFn.mock
|
||||
.lastCall?.[1] as unknown[];
|
||||
expect(explorationReminders(afterReminderRequest)).toHaveLength(0);
|
||||
// Both continuations streamed under the adaptive cap instead of being
|
||||
// halted: with the explicit hard cap at 2 the observed stream of
|
||||
// continuation 1 is a single loop_detected event, so the assertions
|
||||
// above would ride a turn that never reached the model.
|
||||
expect(
|
||||
cont1Events.filter((event) => event.type === LlmEventType.LoopDetected),
|
||||
).toHaveLength(0);
|
||||
expect(
|
||||
cont2Events.filter((event) => event.type === LlmEventType.LoopDetected),
|
||||
).toHaveLength(0);
|
||||
|
||||
// A new user interaction resets the phase: the count starts over, so
|
||||
// one read does not re-trigger the reminder at allowance 2.
|
||||
readTurn(4, 1);
|
||||
await run([{ text: 'new question' }], promptId);
|
||||
const newPhaseRequest = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
expect(explorationReminders(newPhaseRequest)).toHaveLength(0);
|
||||
// The reset also cleared the latched one-shot flag: without it the
|
||||
// stale count (5) and reminded flag would leak into the new turn.
|
||||
expect(client['toolExplorationBudget']['calls']).toBe(1);
|
||||
expect(client['toolExplorationBudget']['reminded']).toBe(false);
|
||||
});
|
||||
|
||||
it('suppresses the exploration reminder when loop detection is disabled for the session', async () => {
|
||||
const promptId = 'prompt-exploration-disabled';
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(1);
|
||||
vi.mocked(mockConfig.isMaxToolCallsPerTurnExplicit).mockReturnValue(
|
||||
false,
|
||||
);
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file' ? ({ kind: Kind.Read } as never) : null,
|
||||
);
|
||||
client['loopDetector'].disableForSession();
|
||||
|
||||
installChat();
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-d-1', 'read_file', { path: 'a' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([{ text: 'explore' }], promptId);
|
||||
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-d-2', 'read_file', { path: 'b' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
const disabledEvents = await run(
|
||||
[fnResponse('read_file', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
|
||||
const request = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
expect(explorationReminders(request)).toHaveLength(0);
|
||||
// The continuation streamed under the adaptive cap: an explicit hard
|
||||
// cap of 1 halts it, which would leave the assertion above riding a
|
||||
// turn that never reached the model.
|
||||
expect(
|
||||
disabledEvents.filter(
|
||||
(event) => event.type === LlmEventType.LoopDetected,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('rolls the exploration budget back on a retry so a discarded attempt is not double-counted', async () => {
|
||||
const promptId = 'prompt-exploration-retry';
|
||||
// Allowance 4 with a committed floor of 1: the first round-trip's
|
||||
// read commits; the failed attempt adds 2 more but the retry rolls
|
||||
// back to the floor, and the restart's two reads land at 3 — below
|
||||
// the allowance. Without the rollback the count would be 5 and the
|
||||
// reminder would fire on a turn that only executed 3 reads; without
|
||||
// the commit the floor would be 0 and the restart would land at 2.
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(4);
|
||||
vi.mocked(mockConfig.isMaxToolCallsPerTurnExplicit).mockReturnValue(
|
||||
false,
|
||||
);
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file' ? ({ kind: Kind.Read } as never) : null,
|
||||
);
|
||||
// A retry is reachable mid-stream after counted calls: the provider
|
||||
// can stream tool calls and still fail with a retryable error.
|
||||
const replayedAttempt = turnStream(
|
||||
toolCallRequest('call-a-0', 'read_file', { path: 'a0' }),
|
||||
stopped(),
|
||||
toolCallRequest('call-f-0', 'read_file', { path: 'f0' }),
|
||||
toolCallRequest('call-f-1', 'read_file', { path: 'f1' }),
|
||||
{ type: LlmEventType.Retry, isContinuation: false },
|
||||
toolCallRequest('call-b-0', 'read_file', { path: 'b0' }),
|
||||
toolCallRequest('call-b-1', 'read_file', { path: 'b1' }),
|
||||
stopped(),
|
||||
);
|
||||
|
||||
installChat();
|
||||
mockTurnRunFn.mockReturnValueOnce(replayedAttempt);
|
||||
const firstEvents = await run([{ text: 'explore' }], promptId);
|
||||
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-c-0', 'read_file', { path: 'c0' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([fnResponse('read_file', { ok: true })], promptId, {
|
||||
type: SendMessageType.ToolResult,
|
||||
});
|
||||
|
||||
const request = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
const reminders = explorationReminders(request);
|
||||
// Rolled back to the committed floor of 1, the restart's two reads
|
||||
// land at 3 — below the allowance of 4, so no reminder.
|
||||
expect(reminders).toHaveLength(0);
|
||||
// The surviving count is exactly the committed floor, the restart's
|
||||
// reads, and this continuation's own read: dropping the rollback
|
||||
// leaves the discarded attempt counted (6), dropping the commit
|
||||
// loses the floor (3).
|
||||
expect(client['toolExplorationBudget']['calls']).toBe(4);
|
||||
// The retried attempt ran to completion without tripping the
|
||||
// per-turn cap: the discarded attempt's calls were rolled back.
|
||||
expect(
|
||||
firstEvents.filter((event) => event.type === LlmEventType.LoopDetected),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('does not reset the exploration budget on a per-request model fallback', async () => {
|
||||
const promptId = 'prompt-exploration-fallback';
|
||||
// Production can only emit ModelFallback before its own attempt's
|
||||
// calls: the fallback chain is gated on the failed attempt having
|
||||
// yielded no candidate output, and a streamed functionCall counts as
|
||||
// candidate output (llm-chat). So the reachable shape spans two
|
||||
// round-trips: the first commits its reads, the second opens with
|
||||
// the fallback event and then streams its own reads on top of the
|
||||
// surviving floor.
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(4);
|
||||
vi.mocked(mockConfig.isMaxToolCallsPerTurnExplicit).mockReturnValue(
|
||||
false,
|
||||
);
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file' ? ({ kind: Kind.Read } as never) : null,
|
||||
);
|
||||
|
||||
installChat();
|
||||
// Round-trip 1: one read commits the floor of 1.
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-a-0', 'read_file', { path: 'a0' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([{ text: 'explore' }], promptId);
|
||||
|
||||
// Round-trip 2 (the ToolResult continuation): the fallback event
|
||||
// arrives before this attempt's own reads, which then count on top
|
||||
// of the surviving floor.
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
{
|
||||
type: LlmEventType.ModelFallback,
|
||||
fromModel: 'test-model',
|
||||
toModel: 'fallback-model',
|
||||
fallbackIndex: 1,
|
||||
},
|
||||
toolCallRequest('call-b-0', 'read_file', { path: 'b0' }),
|
||||
toolCallRequest('call-b-1', 'read_file', { path: 'b1' }),
|
||||
toolCallRequest('call-b-2', 'read_file', { path: 'b2' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
const fallbackEvents = await run(
|
||||
[fnResponse('read_file', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
|
||||
// Round-trip 3: the reminder rides this request — the surviving
|
||||
// count (floor 1 + round-trip 2's three reads) reached the allowance.
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-c-0', 'read_file', { path: 'c0' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
const continuationEvents = await run(
|
||||
[fnResponse('read_file', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
|
||||
const request = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
expect(explorationReminders(request)).toHaveLength(1);
|
||||
// Floor 1 + three fallback-attempt reads = 4 at the reminder, then
|
||||
// this continuation's own read lands on top.
|
||||
expect(client['toolExplorationBudget']['calls']).toBe(5);
|
||||
// The fallback round-trip and the continuation after it ran to
|
||||
// completion rather than being halted by the per-turn cap, so the
|
||||
// counts above describe a turn the model actually served. No fixture
|
||||
// can make the ModelFallback rollback arm load-bearing: a fallback
|
||||
// cannot arrive after its own attempt's calls (llm-chat only runs the
|
||||
// chain when the failed attempt yielded no candidate output and a
|
||||
// streamed functionCall counts as candidate output), so here the
|
||||
// rollback is a no-op either way. The detector needs no matching arm:
|
||||
// checkAlwaysOnSafeties ignores every non-ToolCallRequest event, which
|
||||
// the ModelFallback cases in loopDetectionService.test.ts pin.
|
||||
expect(
|
||||
fallbackEvents.filter(
|
||||
(event) => event.type === LlmEventType.LoopDetected,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
expect(
|
||||
continuationEvents.filter(
|
||||
(event) => event.type === LlmEventType.LoopDetected,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('counts a provider-duplicate call id once toward the exploration budget', async () => {
|
||||
const promptId = 'prompt-exploration-duplicate';
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(2);
|
||||
vi.mocked(mockConfig.isMaxToolCallsPerTurnExplicit).mockReturnValue(
|
||||
false,
|
||||
);
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file' ? ({ kind: Kind.Read } as never) : null,
|
||||
);
|
||||
// The provider re-emits the same call id: the scheduler executes it
|
||||
// once, so the budget must count it once, not twice.
|
||||
installChat();
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('dup', 'read_file', { path: 'a' }),
|
||||
toolCallRequest('dup', 'read_file', { path: 'a' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([{ text: 'explore' }], promptId);
|
||||
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-2', 'read_file', { path: 'b' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
const duplicateEvents = await run(
|
||||
[fnResponse('read_file', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
// 1 distinct call < allowance 2: no reminder yet, and the continuation
|
||||
// streamed under the adaptive cap rather than being halted.
|
||||
const request = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
expect(explorationReminders(request)).toHaveLength(0);
|
||||
expect(
|
||||
duplicateEvents.filter(
|
||||
(event) => event.type === LlmEventType.LoopDetected,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('counts a bridged read-only MCP tool toward the exploration budget', async () => {
|
||||
const promptId = 'prompt-exploration-bridged';
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(2);
|
||||
vi.mocked(mockConfig.isMaxToolCallsPerTurnExplicit).mockReturnValue(
|
||||
false,
|
||||
);
|
||||
// The bridge tool itself is Kind.Other; the deferred MCP target is
|
||||
// Kind.Read. A bridged call arrives as `tool_call` with the target in
|
||||
// `args.name` and must classify through the target, not the bridge.
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['tool_call', 'mcp__srv__describe']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'tool_call'
|
||||
? ({ kind: Kind.Other } as never)
|
||||
: name === 'mcp__srv__describe'
|
||||
? ({ kind: Kind.Read } as never)
|
||||
: null,
|
||||
);
|
||||
|
||||
installChat();
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-br-0', 'tool_call', {
|
||||
name: 'mcp__srv__describe',
|
||||
arguments: { dataset: 0 },
|
||||
}),
|
||||
toolCallRequest('call-br-1', 'tool_call', {
|
||||
name: 'mcp__srv__describe',
|
||||
arguments: { dataset: 1 },
|
||||
}),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([{ text: 'inspect the datasets' }], promptId);
|
||||
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-br-2', 'tool_call', {
|
||||
name: 'mcp__srv__describe',
|
||||
arguments: { dataset: 2 },
|
||||
}),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
const bridgedEvents = await run(
|
||||
[fnResponse('tool_call', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
|
||||
const request = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
expect(explorationReminders(request)).toHaveLength(1);
|
||||
// The bridged target's read landed in the same phase through the
|
||||
// adaptive cap: the continuation was not halted.
|
||||
expect(
|
||||
bridgedEvents.filter(
|
||||
(event) => event.type === LlmEventType.LoopDetected,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('does not reset the running exploration phase for a concurrent side query', async () => {
|
||||
const promptId = 'prompt-exploration-side-query';
|
||||
// Production gives the side question its own prompt id (the submission
|
||||
// counter advances), and its tool-result continuation is submitted
|
||||
// WITHOUT `isConcurrentSideQuery` — the prompt id, not the flag, is what
|
||||
// separates the side turn from the interaction that owns the phase.
|
||||
const sidePromptId = 'prompt-exploration-side-query-btw';
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(2);
|
||||
vi.mocked(mockConfig.isMaxToolCallsPerTurnExplicit).mockReturnValue(
|
||||
false,
|
||||
);
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file', 'write_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file'
|
||||
? ({ kind: Kind.Read } as never)
|
||||
: name === 'write_file'
|
||||
? ({ kind: Kind.Edit } as never)
|
||||
: null,
|
||||
);
|
||||
|
||||
installChat();
|
||||
// The running turn reaches the allowance of 2.
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-s-0', 'read_file', { path: 'a' }),
|
||||
toolCallRequest('call-s-1', 'read_file', { path: 'b' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([{ text: 'explore' }], promptId);
|
||||
|
||||
// A /btw side question arrives while the turn is running. It streams a
|
||||
// mutating call, so an ungated budget would zero the phase here.
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-side-0', 'write_file', { path: 'x' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([{ text: '/btw unrelated question' }], sidePromptId, {
|
||||
type: SendMessageType.UserQuery,
|
||||
isConcurrentSideQuery: true,
|
||||
});
|
||||
// The side turn's mutating call must not reset the running phase.
|
||||
expect(client['toolExplorationBudget']['calls']).toBe(2);
|
||||
|
||||
// The side turn's own continuation carries no flag at all — the /btw
|
||||
// submission is the only place the flag appears — so the prompt id is
|
||||
// what keeps a continuation, the shape production actually submits,
|
||||
// from being treated as the owner of this phase. Its read must not be
|
||||
// recorded and must not drain the pending reminder either.
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-side-1', 'read_file', { path: 'y' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
await run([fnResponse('write_file', { ok: true })], sidePromptId, {
|
||||
type: SendMessageType.ToolResult,
|
||||
});
|
||||
expect(client['toolExplorationBudget']['calls']).toBe(2);
|
||||
expect(
|
||||
explorationReminders(mockTurnRunFn.mock.lastCall?.[1] as unknown[]),
|
||||
).toHaveLength(0);
|
||||
|
||||
// The exploration's own continuation still carries the reminder: the
|
||||
// side query did not consume or suppress it.
|
||||
mockTurnRunFn.mockReturnValueOnce(
|
||||
turnStream(
|
||||
toolCallRequest('call-s-2', 'read_file', { path: 'c' }),
|
||||
stopped(),
|
||||
),
|
||||
);
|
||||
const continuationEvents = await run(
|
||||
[fnResponse('read_file', { ok: true })],
|
||||
promptId,
|
||||
{
|
||||
type: SendMessageType.ToolResult,
|
||||
},
|
||||
);
|
||||
const request = mockTurnRunFn.mock.lastCall?.[1] as unknown[];
|
||||
expect(explorationReminders(request)).toHaveLength(1);
|
||||
expect(
|
||||
continuationEvents.filter(
|
||||
(event) => event.type === LlmEventType.LoopDetected,
|
||||
),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('should PRESERVE the pending prefetch when next-speaker continueTurn returns', async () => {
|
||||
// Self-inflicted-regression guard for the round-4 finding: the
|
||||
@@ -10981,6 +11501,90 @@ Other open files:
|
||||
});
|
||||
});
|
||||
|
||||
it('gives a blocking Stop hook continuation a fresh exploration phase', async () => {
|
||||
// The nested restart is submitted with `type: SendMessageType.Hook`,
|
||||
// which is not an interaction-starting type, so the reset beside the
|
||||
// Stop-hook block is the only thing that clears the read-only phase
|
||||
// across the restart.
|
||||
installMessageBus(blockStopOnce(), 'Stop');
|
||||
notDoneChat();
|
||||
// The default registry mock knows no tools, so read_file would
|
||||
// classify as unknown and zero the phase on every call.
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file' ? ({ kind: Kind.Read } as never) : null,
|
||||
);
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(100);
|
||||
let turnIndex = 0;
|
||||
mockTurnRunFn.mockImplementation(() => {
|
||||
const turnNo = turnIndex++;
|
||||
return (async function* () {
|
||||
// Only the first turn reads; the continuation just answers.
|
||||
if (turnNo === 0) {
|
||||
for (let i = 0; i < 2; i++) {
|
||||
yield toolCallRequest(
|
||||
`call-${i}`,
|
||||
'read_file',
|
||||
{ path: `f${i}` },
|
||||
'prompt-stop-hook-exploration',
|
||||
);
|
||||
}
|
||||
}
|
||||
yield { type: LlmEventType.Content, value: 'not done' };
|
||||
})();
|
||||
});
|
||||
|
||||
await run([{ text: 'Hi' }], 'prompt-stop-hook-exploration');
|
||||
|
||||
expect(mockTurnRunFn).toHaveBeenCalledTimes(2);
|
||||
// The first turn's two reads were cleared by the restart instead of
|
||||
// being carried into the continuation.
|
||||
expect(client['toolExplorationBudget']['calls']).toBe(0);
|
||||
});
|
||||
|
||||
it('gives a goal-permitted blocking Stop hook continuation a fresh exploration phase', async () => {
|
||||
// A goal-permitted restart takes a separate branch from the case
|
||||
// above, and its nested turn is likewise submitted with
|
||||
// `type: SendMessageType.Hook`, so the reset beside that branch is
|
||||
// again the only thing that clears the phase across the restart.
|
||||
const { permit } = installGoalRuntime();
|
||||
installMessageBus(blockStopOnce(), 'Stop');
|
||||
notDoneChat();
|
||||
const reg = registryMock();
|
||||
reg.getAllToolNames.mockReturnValue(['read_file']);
|
||||
reg.getTool.mockImplementation((name: string) =>
|
||||
name === 'read_file' ? ({ kind: Kind.Read } as never) : null,
|
||||
);
|
||||
vi.mocked(mockConfig.getMaxToolCallsPerTurn).mockReturnValue(100);
|
||||
let turnIndex = 0;
|
||||
mockTurnRunFn.mockImplementation(() => {
|
||||
const turnNo = turnIndex++;
|
||||
return (async function* () {
|
||||
if (turnNo === 0) {
|
||||
for (let i = 0; i < 2; i++) {
|
||||
yield toolCallRequest(
|
||||
`call-${i}`,
|
||||
'read_file',
|
||||
{ path: `f${i}` },
|
||||
'prompt-goal-stop-hook-exploration',
|
||||
);
|
||||
}
|
||||
}
|
||||
yield { type: LlmEventType.Content, value: 'not done' };
|
||||
})();
|
||||
});
|
||||
|
||||
await run([{ text: 'Hi' }], 'prompt-goal-stop-hook-exploration', {
|
||||
type: SendMessageType.UserQuery,
|
||||
goalPermit: permit,
|
||||
goalTurnKey: 'goal-runtime:turn-1',
|
||||
});
|
||||
|
||||
expect(mockTurnRunFn).toHaveBeenCalledTimes(2);
|
||||
expect(client['toolExplorationBudget']['calls']).toBe(0);
|
||||
});
|
||||
|
||||
it('reports stop_hook_active only when a Stop hook blocked the previous turn', async () => {
|
||||
const mockMessageBus = installMessageBus(blockStopOnce(), 'Stop');
|
||||
notDoneChat();
|
||||
|
||||
@@ -93,13 +93,20 @@ import {
|
||||
} from './turn.js';
|
||||
|
||||
// Services
|
||||
import { LoopDetectionService } from '../services/loopDetectionService.js';
|
||||
import {
|
||||
isAttemptRestartEvent,
|
||||
LoopDetectionService,
|
||||
} from '../services/loopDetectionService.js';
|
||||
import { CommitAttributionService } from '../services/commitAttribution.js';
|
||||
import type { UserPromptRecordPayload } from '../services/chatRecordingService.js';
|
||||
|
||||
// Tools
|
||||
import type { RelevantAutoMemoryPromptResult } from '../memory/manager.js';
|
||||
import { AUTO_SKILL_THRESHOLD } from '../memory/manager.js';
|
||||
import {
|
||||
getToolExplorationKind,
|
||||
ToolExplorationBudget,
|
||||
} from '../services/tool-exploration-budget.js';
|
||||
import {
|
||||
renderAutoMemoryFocusedSubtree,
|
||||
toAutoMemoryRef,
|
||||
@@ -500,6 +507,12 @@ export class LlmClient {
|
||||
private readonly interactionStartTypes = new Map<string, SendMessageType>();
|
||||
|
||||
private readonly loopDetector: LoopDetectionService;
|
||||
private readonly toolExplorationBudget = new ToolExplorationBudget();
|
||||
// The prompt/turn that owns the exploration phase. A concurrent `/btw`
|
||||
// side query reaches this same client with the full tool set, so the phase
|
||||
// is bound to its owning interaction: every budget entrypoint below is a
|
||||
// no-op for an invocation that does not own it.
|
||||
private toolExplorationBudgetPromptId: string | undefined;
|
||||
private lastPromptId: string | undefined = undefined;
|
||||
private activeTodoWorkChainPromptId: string | undefined;
|
||||
private readonly activeAutomaticTodoWorkChainPromptIds = new Set<string>();
|
||||
@@ -3810,8 +3823,13 @@ export class LlmClient {
|
||||
this.loopDetector.reset(prompt_id);
|
||||
this.lastPromptId = prompt_id;
|
||||
// A side question asked while a turn is running is not a new turn: it
|
||||
// must not move the running turn's starting point or drop its target.
|
||||
// must not move the running turn's starting point or drop its target —
|
||||
// nor take over (or clear) the running turn's exploration phase, which
|
||||
// the ownership check below every budget entrypoint enforces for the
|
||||
// side turn's tool results too.
|
||||
if (!options?.isConcurrentSideQuery) {
|
||||
this.toolExplorationBudget.reset();
|
||||
this.toolExplorationBudgetPromptId = prompt_id;
|
||||
this.beginTurnBudget(messageType, request, prompt_id);
|
||||
}
|
||||
// New input starts this interaction, so its first Stop is not
|
||||
@@ -4178,6 +4196,14 @@ export class LlmClient {
|
||||
let memoryDeliveryToCommit: MemoryDeliveryResult | null = null;
|
||||
let memoryDeliveryStateInvalidated = false;
|
||||
let modelRequestAccepted = false;
|
||||
// A concurrent `/btw` side query runs through this same client and its
|
||||
// tool-result continuation is submitted without the flag, so ownership is
|
||||
// carried by the prompt id the interaction claimed at its start: a
|
||||
// non-owner invocation leaves the running turn's phase untouched (it
|
||||
// neither records into it nor drains its pending reminder).
|
||||
const ownsExplorationBudget =
|
||||
!options?.isConcurrentSideQuery &&
|
||||
prompt_id === this.toolExplorationBudgetPromptId;
|
||||
// The delivery was prepared before the request went out, so commit vs
|
||||
// discard rides on whether the model accepted the request (any streamed
|
||||
// event). A loop-detection halt after acceptance must commit: the memory
|
||||
@@ -4801,6 +4827,14 @@ export class LlmClient {
|
||||
const activeTodoReminder = carriesAgentToolResult
|
||||
? this.config.takeActiveTodoReminder(prompt_id, true)
|
||||
: this.config.takeActiveTodoReminder(prompt_id);
|
||||
const explorationReminder =
|
||||
this.loopDetector.isDisabledForSession() || !ownsExplorationBudget
|
||||
? undefined
|
||||
: this.toolExplorationBudget.takeReminder(
|
||||
this.config.getMaxToolCallsPerTurn(),
|
||||
);
|
||||
if (explorationReminder)
|
||||
requestToSend.push({ text: explorationReminder });
|
||||
if (activeTodoReminder) {
|
||||
const insertAt = requestToSend.findIndex(
|
||||
(part) =>
|
||||
@@ -4919,10 +4953,7 @@ export class LlmClient {
|
||||
}
|
||||
if (event.type === LlmEventType.ToolCallRequest) {
|
||||
hasToolCalls = true;
|
||||
} else if (
|
||||
event.type === LlmEventType.Retry ||
|
||||
event.type === LlmEventType.ModelFallback
|
||||
) {
|
||||
} else if (isAttemptRestartEvent(event)) {
|
||||
modelRequestAccepted = false;
|
||||
hasToolCalls = false;
|
||||
loopGuardFedCallIds.clear();
|
||||
@@ -4967,6 +4998,38 @@ export class LlmClient {
|
||||
const alwaysOnLoop =
|
||||
!duplicateLoopGuardRequest &&
|
||||
this.loopDetector.checkAlwaysOnSafeties(event);
|
||||
// Exploration-budget lifecycle, recorded before the halt handling
|
||||
// below takes effect: a halted batch's over-count is harmless only
|
||||
// because every core halt is terminal (the halt branch returns the
|
||||
// turn, and the next interaction resets the budget). Keep any
|
||||
// future non-terminal core halt from silently counting calls that
|
||||
// never executed. Ownership-gated like every other entrypoint: a
|
||||
// concurrent side turn's stream neither feeds this turn's phase nor
|
||||
// rolls it back.
|
||||
if (ownsExplorationBudget) {
|
||||
if (
|
||||
event.type === LlmEventType.ToolCallRequest &&
|
||||
!duplicateLoopGuardRequest
|
||||
) {
|
||||
this.toolExplorationBudget.record(
|
||||
getToolExplorationKind(
|
||||
this.config.getToolRegistry(),
|
||||
event.value.name,
|
||||
event.value.args,
|
||||
),
|
||||
);
|
||||
} else if (event.type === LlmEventType.Finished) {
|
||||
this.toolExplorationBudget.commit();
|
||||
} else if (isAttemptRestartEvent(event)) {
|
||||
// A retry re-streams the failed attempt's calls, so its reads
|
||||
// must not stay counted. A fallback attempt cannot have
|
||||
// streamed calls (the chain only runs when the failed attempt
|
||||
// yielded no candidate output), so its rollback is a no-op kept
|
||||
// for symmetry — and harmless, because a rollback never goes
|
||||
// below the committed floor.
|
||||
this.toolExplorationBudget.rollback();
|
||||
}
|
||||
}
|
||||
if (alwaysOnLoop) {
|
||||
// Drop every tool call collected before the guard fired so the run
|
||||
// halts here instead of spawning a continuation that re-trips it.
|
||||
@@ -5292,6 +5355,7 @@ export class LlmClient {
|
||||
};
|
||||
|
||||
this.loopDetector.reset(prompt_id);
|
||||
if (ownsExplorationBudget) this.toolExplorationBudget.reset();
|
||||
const hookTurnBudget = boundedTurns - 1;
|
||||
const pendingSteer = await takeSteerInput(hookTurnBudget);
|
||||
for (const goalEvent of takePendingGoalEvents()) {
|
||||
@@ -5400,6 +5464,7 @@ export class LlmClient {
|
||||
// protection is preserved: the cap still bounds each iteration, and
|
||||
// the chain itself is bounded by stopHookBlockingCap.
|
||||
this.loopDetector.reset(prompt_id);
|
||||
if (ownsExplorationBudget) this.toolExplorationBudget.reset();
|
||||
|
||||
const hookTurnBudget = boundedTurns - 1;
|
||||
const pendingSteer = await takeSteerInput(hookTurnBudget);
|
||||
|
||||
@@ -1051,6 +1051,127 @@ describe('LoopDetectionService', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('ModelFallback must not clear committed guard evidence', () => {
|
||||
it('keeps the consecutive-identical streak across per-round-trip ModelFallback', () => {
|
||||
service.reset('');
|
||||
// Production can only emit ModelFallback before its own attempt's
|
||||
// calls (the fallback chain is gated on !streamYieldedAnyChunk, and a
|
||||
// streamed functionCall counts as candidate output), and the chain is
|
||||
// per-request, so a capacity-limited primary can prefix every round-trip
|
||||
// of one turn with ModelFallback. The streak evidence accumulated by
|
||||
// earlier committed round-trips must survive the switch, or this guard
|
||||
// never fires again.
|
||||
const finished = {
|
||||
type: LlmEventType.Finished,
|
||||
value: { reason: 'STOP' },
|
||||
} as unknown as ServerLlmStreamEvent;
|
||||
const fallback = {
|
||||
type: LlmEventType.ModelFallback,
|
||||
fromModel: 'primary-model',
|
||||
toModel: 'fallback-model',
|
||||
fallbackIndex: 1,
|
||||
} satisfies ServerLlmModelFallbackEvent as ServerLlmStreamEvent;
|
||||
// TOOL_CALL_LOOP_THRESHOLD - 1 identical calls, each in its own
|
||||
// round-trip separated by Finished + ModelFallback.
|
||||
for (let i = 0; i < TOOL_CALL_LOOP_THRESHOLD - 1; i++) {
|
||||
expect(guardTool('read_file', { absolute_path: '/repo/a.ts' })).toBe(
|
||||
false,
|
||||
);
|
||||
guard(finished);
|
||||
guard(fallback);
|
||||
}
|
||||
// The threshold call must still fire: the streak reached 5 across the
|
||||
// committed round-trips instead of being zeroed by each fallback.
|
||||
expect(guardTool('read_file', { absolute_path: '/repo/a.ts' })).toBe(
|
||||
true,
|
||||
);
|
||||
expect(service.getLastLoopType()).toBe(
|
||||
LoopType.CONSECUTIVE_IDENTICAL_TOOL_CALLS,
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps the shell-inspection stagnation streak across per-round-trip ModelFallback', () => {
|
||||
service.reset('');
|
||||
// Same reachable shape as above but with varied git-inspection
|
||||
// wording, which only the stagnation guard (keyed on command family)
|
||||
// can see — the consecutive-identical guard never fires on it.
|
||||
const finished = {
|
||||
type: LlmEventType.Finished,
|
||||
value: { reason: 'STOP' },
|
||||
} as unknown as ServerLlmStreamEvent;
|
||||
const fallback = {
|
||||
type: LlmEventType.ModelFallback,
|
||||
fromModel: 'primary-model',
|
||||
toModel: 'fallback-model',
|
||||
fallbackIndex: 1,
|
||||
} satisfies ServerLlmModelFallbackEvent as ServerLlmStreamEvent;
|
||||
const VARIANTS = [
|
||||
'git status --short',
|
||||
'git diff --stat',
|
||||
'git ls-files --modified',
|
||||
'git status --porcelain=v1',
|
||||
'git diff --name-only HEAD',
|
||||
'git -C . status --short',
|
||||
'git --no-pager diff --stat',
|
||||
];
|
||||
for (let i = 0; i < SHELL_COMMAND_STAGNATION_THRESHOLD - 1; i++) {
|
||||
expect(
|
||||
guardTool('run_shell_command', {
|
||||
command: VARIANTS[i % VARIANTS.length],
|
||||
description: 'Inspect repository changes',
|
||||
}),
|
||||
).toBe(false);
|
||||
guard(finished);
|
||||
guard(fallback);
|
||||
}
|
||||
expect(
|
||||
guardTool('run_shell_command', {
|
||||
command: 'git status --short',
|
||||
description: 'Inspect repository changes',
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(service.getLastLoopType()).toBe(LoopType.SHELL_COMMAND_STAGNATION);
|
||||
});
|
||||
|
||||
it('keeps capMaxKeyRepeat accumulating across per-round-trip ModelFallback', () => {
|
||||
service.reset('');
|
||||
// The adaptive cap's stuck signal (capMaxKeyRepeat >=
|
||||
// GLOBAL_DUPLICATE_THRESHOLD) is what halts a productive-looking but
|
||||
// stuck diverse turn past the soft cap; per-round-trip fallbacks must
|
||||
// not zero it either. Diverse calls keep every other guard quiet.
|
||||
const finished = {
|
||||
type: LlmEventType.Finished,
|
||||
value: { reason: 'STOP' },
|
||||
} as unknown as ServerLlmStreamEvent;
|
||||
const fallback = {
|
||||
type: LlmEventType.ModelFallback,
|
||||
fromModel: 'primary-model',
|
||||
toModel: 'fallback-model',
|
||||
fallbackIndex: 1,
|
||||
} satisfies ServerLlmModelFallbackEvent as ServerLlmStreamEvent;
|
||||
// Diverse calls past DEFAULT_MAX_TOOL_CALLS_PER_TURN (the soft cap,
|
||||
// where the adaptive cap starts halting on the stuck signal), then
|
||||
// round-trips of alternating stuck keys, each separated by Finished +
|
||||
// ModelFallback: the repeat evidence must accumulate ACROSS the
|
||||
// fallback boundaries, or the adaptive cap's stuck signal never
|
||||
// reaches its threshold and the turn runs to the 1000-call backstop.
|
||||
const SOFT_CAP = DEFAULT_MAX_TOOL_CALLS_PER_TURN;
|
||||
for (let i = 0; i < SOFT_CAP; i++) {
|
||||
expect(guardTool('any_tool', { i })).toBe(false);
|
||||
guard(finished);
|
||||
guard(fallback);
|
||||
}
|
||||
for (let i = 0; i < GLOBAL_DUPLICATE_THRESHOLD - 1; i++) {
|
||||
expect(guardTool('any_tool', { stuck: 'a' })).toBe(false);
|
||||
expect(guardTool('any_tool', { stuck: 'b' })).toBe(false);
|
||||
guard(finished);
|
||||
guard(fallback);
|
||||
}
|
||||
expect(guardTool('any_tool', { stuck: 'a' })).toBe(true);
|
||||
expect(service.getLastLoopType()).toBe(LoopType.TURN_TOOL_CALL_CAP);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Truncation hysteresis', () => {
|
||||
// The physical trim walks the whole contentStats map (one entry per
|
||||
// window position at saturation): Θ(window) synchronous CPU per event.
|
||||
|
||||
@@ -198,6 +198,31 @@ export const DEFAULT_MAX_TOOL_CALLS_PER_TURN = 100;
|
||||
// false-positived, while still bounding a pathological runaway.
|
||||
const ADAPTIVE_CAP_HARD_MULTIPLIER = 10;
|
||||
|
||||
/**
|
||||
* Whether a stream event restarts the current attempt from scratch (the
|
||||
* failed attempt's partial output is discarded and re-streamed), so any
|
||||
* attempt-scoped observations recorded after the last committed round-trip
|
||||
* must be rolled back. The client's stream loop consumes it for both of its
|
||||
* attempt-scoped states — the exploration budget's phase and the loop-guard
|
||||
* feed set (`modelRequestAccepted`/`hasToolCalls`/`loopGuardFedCallIds`) — so
|
||||
* a new restart-style event lands in both of them or in neither.
|
||||
*
|
||||
* It is deliberately not consumed by the loop detector, which has no call
|
||||
* site here: that module's own per-turn cap rollback stays `Retry`-only, and
|
||||
* `checkAlwaysOnSafeties` already treats a fallback as a no-op through its
|
||||
* non-`ToolCallRequest` early return (a fallback cannot have streamed calls,
|
||||
* and clearing the detector's streak evidence per round-trip would silence
|
||||
* the consecutive-identical and stagnation guards). A restart-style event
|
||||
* added here does NOT propagate to the detector; add it there separately and
|
||||
* justify it.
|
||||
*/
|
||||
export function isAttemptRestartEvent(event: ServerLlmStreamEvent): boolean {
|
||||
return (
|
||||
event.type === LlmEventType.Retry ||
|
||||
event.type === LlmEventType.ModelFallback
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Halt predicate of the per-turn tool-call cap, shared with the daemon's
|
||||
* turn-loop guard (ACP Session's recordDaemonToolCalls) so both runtimes
|
||||
@@ -452,6 +477,10 @@ export class LoopDetectionService {
|
||||
);
|
||||
}
|
||||
|
||||
isDisabledForSession(): boolean {
|
||||
return this.disabledForSession;
|
||||
}
|
||||
|
||||
/**
|
||||
* Records the executed result of a tool call so the guards can treat
|
||||
* stateful read tools (see STATEFUL_READ_TOOLS) result-aware: identical
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2025 Qwen
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { Kind } from '../tools/tools.js';
|
||||
import type { ToolRegistry } from '../tools/tool-registry.js';
|
||||
import {
|
||||
getToolExplorationKind,
|
||||
TOOL_EXPLORATION_REMINDER,
|
||||
ToolExplorationBudget,
|
||||
} from './tool-exploration-budget.js';
|
||||
|
||||
describe('ToolExplorationBudget', () => {
|
||||
it('reminds once at the allowance and allows a legitimate investigation to continue', () => {
|
||||
const budget = new ToolExplorationBudget();
|
||||
for (let i = 0; i < 99; i++) budget.record(Kind.Read);
|
||||
expect(budget.takeReminder(100)).toBeUndefined();
|
||||
budget.record(Kind.Search);
|
||||
expect(budget.takeReminder(100)).toBe(TOOL_EXPLORATION_REMINDER);
|
||||
for (let i = 0; i < 50; i++) budget.record(Kind.Fetch);
|
||||
expect(budget.takeReminder(100)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('resets a phase on implementation, planning or unknown tools', () => {
|
||||
const budget = new ToolExplorationBudget();
|
||||
for (const kind of [Kind.Edit, Kind.Think, undefined]) {
|
||||
budget.record(Kind.Read);
|
||||
budget.record(kind);
|
||||
budget.record(Kind.Read);
|
||||
expect(budget.takeReminder(2)).toBeUndefined();
|
||||
budget.record(Kind.Read);
|
||||
expect(budget.takeReminder(2)).toBe(TOOL_EXPLORATION_REMINDER);
|
||||
budget.reset();
|
||||
}
|
||||
});
|
||||
|
||||
it('commits the zeroed floor on reset so a later rollback cannot resurrect a prior turn', () => {
|
||||
const budget = new ToolExplorationBudget();
|
||||
// Turn 1 reads past the allowance and commits its count.
|
||||
budget.record(Kind.Read);
|
||||
budget.record(Kind.Read);
|
||||
budget.commit();
|
||||
// Turn 2 resets (zeroing state AND committing the zero floor), streams
|
||||
// one read, then the attempt fails with a retry: the rollback must land
|
||||
// on turn 2's own floor of 0, not turn 1's committed count of 2.
|
||||
budget.reset();
|
||||
budget.record(Kind.Read);
|
||||
budget.rollback();
|
||||
expect(budget.takeReminder(2)).toBeUndefined();
|
||||
// A fresh phase accumulates normally after the reset.
|
||||
budget.record(Kind.Read);
|
||||
budget.record(Kind.Read);
|
||||
expect(budget.takeReminder(2)).toBe(TOOL_EXPLORATION_REMINDER);
|
||||
});
|
||||
|
||||
it('rolls back a replayed attempt without repeating an already sent reminder', () => {
|
||||
const budget = new ToolExplorationBudget();
|
||||
budget.record(Kind.Read);
|
||||
budget.commit();
|
||||
budget.record(Kind.Read);
|
||||
budget.rollback();
|
||||
expect(budget.takeReminder(2)).toBeUndefined();
|
||||
budget.record(Kind.Read);
|
||||
budget.commit();
|
||||
expect(budget.takeReminder(2)).toBe(TOOL_EXPLORATION_REMINDER);
|
||||
budget.record(Kind.Edit);
|
||||
budget.rollback();
|
||||
expect(budget.takeReminder(2)).toBeUndefined();
|
||||
// The phase reset above must commit its own zero floor, so the rollback
|
||||
// cannot carry the delivered reminder's latch into the phase that
|
||||
// follows it: a fresh phase earns its own reminder.
|
||||
budget.record(Kind.Read);
|
||||
budget.record(Kind.Read);
|
||||
expect(budget.takeReminder(2)).toBe(TOOL_EXPLORATION_REMINDER);
|
||||
});
|
||||
|
||||
it('honors a disabled allowance', () => {
|
||||
const budget = new ToolExplorationBudget();
|
||||
budget.record(Kind.Read);
|
||||
expect(budget.takeReminder(Infinity)).toBeUndefined();
|
||||
expect(budget.takeReminder(0)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('counts read-only discovery tools registered outside the read kinds', () => {
|
||||
// `lsp` and `tool_search` are registered Kind.Other, but interleaving
|
||||
// them into a read-only investigation must not wipe the phase: LSP's
|
||||
// own description steers the model into exactly that alternation.
|
||||
const registry = {
|
||||
getAllToolNames: () => ['read_file', 'lsp', 'tool_search', 'tool_call'],
|
||||
getTool: (name: string) =>
|
||||
name === 'read_file'
|
||||
? { kind: Kind.Read }
|
||||
: name === 'tool_call'
|
||||
? { kind: Kind.Other }
|
||||
: undefined,
|
||||
} as unknown as ToolRegistry;
|
||||
expect(getToolExplorationKind(registry, 'lsp', {})).toBe(Kind.Read);
|
||||
expect(getToolExplorationKind(registry, 'tool_search', {})).toBe(Kind.Read);
|
||||
// A bridged read-only MCP target still classifies as read (covered for
|
||||
// the registry path below); the bridge tool itself still ends a phase.
|
||||
expect(
|
||||
getToolExplorationKind(registry, 'tool_call', {
|
||||
name: 'lsp',
|
||||
arguments: {},
|
||||
}),
|
||||
).toBe(Kind.Read);
|
||||
// The phase survives the interleaving.
|
||||
const budget = new ToolExplorationBudget();
|
||||
budget.record(getToolExplorationKind(registry, 'read_file', {}));
|
||||
budget.record(getToolExplorationKind(registry, 'lsp', {}));
|
||||
budget.record(getToolExplorationKind(registry, 'read_file', {}));
|
||||
expect(budget.takeReminder(2)).toBe(TOOL_EXPLORATION_REMINDER);
|
||||
});
|
||||
|
||||
it('leaves an unregistered read-only discovery name unclassified', () => {
|
||||
// `lsp` is registered only when LSP is enabled and `tool_search` can be
|
||||
// removed with `--exclude-tools`; a refused call still reaches record(),
|
||||
// so a name the session never registered must not be credited as a read.
|
||||
const registry = {
|
||||
getAllToolNames: () => ['read_file'],
|
||||
getTool: (name: string) =>
|
||||
name === 'read_file' ? { kind: Kind.Read } : undefined,
|
||||
} as unknown as ToolRegistry;
|
||||
expect(getToolExplorationKind(registry, 'tool_search', {})).toBeUndefined();
|
||||
expect(getToolExplorationKind(registry, 'lsp', {})).toBeUndefined();
|
||||
expect(
|
||||
getToolExplorationKind(registry, 'tool_call', {
|
||||
name: 'tool_search',
|
||||
arguments: {},
|
||||
}),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('leaves a bridge-refused discovery target unclassified', () => {
|
||||
// `tool_call` unconditionally refuses `tool_search` and itself
|
||||
// (tools/tool-call.ts), so a bridged call to either performs no read
|
||||
// even though the registry knows the name.
|
||||
const registry = {
|
||||
getAllToolNames: () => ['read_file', 'lsp', 'tool_search', 'tool_call'],
|
||||
getTool: (name: string) =>
|
||||
name === 'tool_call' ? { kind: Kind.Other } : undefined,
|
||||
} as unknown as ToolRegistry;
|
||||
expect(
|
||||
getToolExplorationKind(registry, 'tool_call', {
|
||||
name: 'tool_search',
|
||||
arguments: {},
|
||||
}),
|
||||
).toBeUndefined();
|
||||
// A bridged target the bridge does accept keeps its classification.
|
||||
expect(
|
||||
getToolExplorationKind(registry, 'tool_call', {
|
||||
name: 'lsp',
|
||||
arguments: {},
|
||||
}),
|
||||
).toBe(Kind.Read);
|
||||
});
|
||||
|
||||
it('leaves ambiguous case-insensitive registrations unclassified', () => {
|
||||
const registry = {
|
||||
getAllToolNames: () => ['READ_FILE', 'Read_File'],
|
||||
getTool: () => ({ kind: Kind.Read }),
|
||||
} as unknown as ToolRegistry;
|
||||
expect(getToolExplorationKind(registry, 'read_file', {})).toBeUndefined();
|
||||
});
|
||||
|
||||
it('uses the registered kind for arbitrary and bridged MCP tool names', () => {
|
||||
const registry = {
|
||||
getAllToolNames: () => ['mcp__anonymous__describe_dataset'],
|
||||
getTool: (name: string) =>
|
||||
name === 'mcp__anonymous__describe_dataset'
|
||||
? { kind: Kind.Read }
|
||||
: undefined,
|
||||
} as unknown as ToolRegistry;
|
||||
expect(
|
||||
getToolExplorationKind(registry, 'tool_call', {
|
||||
name: 'mcp__anonymous__describe_dataset',
|
||||
arguments: {},
|
||||
}),
|
||||
).toBe(Kind.Read);
|
||||
expect(
|
||||
getToolExplorationKind(registry, 'unregistered', {}),
|
||||
).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,111 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2025 Qwen
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { Kind } from '../tools/tools.js';
|
||||
import type { ToolRegistry } from '../tools/tool-registry.js';
|
||||
import {
|
||||
canonicalToolName,
|
||||
resolveRegisteredToolName,
|
||||
ToolNames,
|
||||
} from '../tools/tool-names.js';
|
||||
|
||||
// Read-only discovery tools registered outside the Read/Search/Fetch kinds.
|
||||
// `lsp` and `tool_search` are `Kind.Other` — retyping them would also enrol
|
||||
// them in CONCURRENCY_SAFE_KINDS, a scheduler behaviour change this feature
|
||||
// must not make — yet both are first-party read-only discovery surfaces the
|
||||
// model is explicitly steered toward (LSP's own description says to prefer it
|
||||
// over grep/glob), and classifying them as phase resets makes the reminder
|
||||
// structurally unreachable for `lsp → read_file → lsp` investigations.
|
||||
const READ_ONLY_DISCOVERY_TOOLS = new Set<string>([
|
||||
ToolNames.LSP,
|
||||
ToolNames.TOOL_SEARCH,
|
||||
]);
|
||||
|
||||
export const TOOL_EXPLORATION_REMINDER =
|
||||
'System: this read-only exploration phase has reached the configured tool-call allowance. Review the information already gathered before continuing discovery. For an implementation request, proceed to the requested deliverable or explain the concrete blocker. For a read-only investigation, summarize the findings and identify any remaining question. Continue tools only to resolve a specific remaining gap or perform necessary verification. This reminder does not authorize writes or bypass plan approval or tool permissions.';
|
||||
|
||||
export function getToolExplorationKind(
|
||||
registry: ToolRegistry,
|
||||
name: string,
|
||||
args: object,
|
||||
): Kind | undefined {
|
||||
const bridged =
|
||||
name === ToolNames.TOOL_CALL &&
|
||||
typeof (args as Record<string, unknown>)['name'] === 'string';
|
||||
const target = bridged
|
||||
? ((args as Record<string, unknown>)['name'] as string)
|
||||
: name;
|
||||
const canonical = canonicalToolName(target);
|
||||
const resolved = resolveRegisteredToolName(
|
||||
canonical,
|
||||
registry.getAllToolNames(),
|
||||
);
|
||||
if (Array.isArray(resolved)) return undefined;
|
||||
// The bridge unconditionally refuses these two targets
|
||||
// (tools/tool-call.ts), so a bridged call to either performs no read and
|
||||
// must stay unclassified like a name the registry refuses.
|
||||
if (
|
||||
bridged &&
|
||||
(resolved === ToolNames.TOOL_SEARCH || resolved === ToolNames.TOOL_CALL)
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
// Keyed on the registry's answer: `resolved` is `undefined` when the
|
||||
// session never registered this name (LSP disabled, `--exclude-tools`
|
||||
// removed `tool_search`), and a name the registry refuses must stay
|
||||
// unclassified like every other unregistered name instead of being
|
||||
// credited as progress.
|
||||
if (resolved !== undefined && READ_ONLY_DISCOVERY_TOOLS.has(resolved)) {
|
||||
return Kind.Read;
|
||||
}
|
||||
return registry.getTool(resolved ?? canonical)?.kind;
|
||||
}
|
||||
|
||||
export class ToolExplorationBudget {
|
||||
private calls = 0;
|
||||
private reminded = false;
|
||||
private committedCalls = 0;
|
||||
private committedReminded = false;
|
||||
|
||||
record(kind: Kind | undefined): void {
|
||||
if (kind === Kind.Read || kind === Kind.Search || kind === Kind.Fetch) {
|
||||
this.calls++;
|
||||
} else {
|
||||
// A non-read ends the phase, and reset() commits the zeroed floor so a
|
||||
// later rollback cannot resurrect the previous phase's count or latch.
|
||||
this.reset();
|
||||
}
|
||||
}
|
||||
|
||||
takeReminder(allowance: number): string | undefined {
|
||||
if (
|
||||
!Number.isFinite(allowance) ||
|
||||
allowance <= 0 ||
|
||||
this.calls < allowance ||
|
||||
this.reminded
|
||||
)
|
||||
return undefined;
|
||||
this.reminded = true;
|
||||
this.committedReminded = true;
|
||||
return TOOL_EXPLORATION_REMINDER;
|
||||
}
|
||||
|
||||
commit(): void {
|
||||
this.committedCalls = this.calls;
|
||||
this.committedReminded = this.reminded;
|
||||
}
|
||||
|
||||
rollback(): void {
|
||||
this.calls = this.committedCalls;
|
||||
this.reminded = this.committedReminded;
|
||||
}
|
||||
|
||||
reset(): void {
|
||||
this.calls = 0;
|
||||
this.reminded = false;
|
||||
this.commit();
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user