mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-11 17:43:43 -10:00
fix(memory): Narrow selector experiment and withdraw unsafe cooldown
This commit is contained in:
1 parent
a4568de26e
commit
b4e0bba5b8
36 files changed
+295
-3106
No files matched your search
@@ -1,35 +0,0 @@
|
||||
# Auto-memory failed extraction write recovery and window suspension
|
||||
|
||||
[English](auto-memory-failed-extraction.md) | [简体中文](auto-memory-failed-extraction.zh-CN.md)
|
||||
|
||||
Status: implemented write recovery and an opt-in failed-window attempt bound for #13158 R1-3; full cost and backlog acceptance remain open.
|
||||
|
||||
## Problem and scope
|
||||
|
||||
A fork can write a valid memory file before failing or being cancelled. Its result retains successful `filesWritten`, but the extraction planner throws before mapping them to memory topics and scopes. The document remains on disk without a rebuilt index, and the manager does not record user-memory mutation. A successful fork followed by an index or cursor-write error similarly loses its known write scope.
|
||||
|
||||
Recover reported writes without advancing a failed extraction's cursor. Apply the same durability rule to the default and experimental window paths. Keep existing window selection, per-fork budgets, cooldown, queue and drain behavior. With the existing cooldown experiment enabled, suspend the same selected window after three consecutive failed extraction attempts, including trailing queued requests.
|
||||
|
||||
An established cursor keeps selecting the oldest pending window after failure. Growing history can therefore repeatedly pay for the same failed selection. A fresh cursor without an attested history identity can instead select a newer real user turn; a whole-session failure cap would block that existing recovery.
|
||||
|
||||
## Decision
|
||||
|
||||
Use a typed Error carrying the existing mapped execution result. Reuse the current memory-root classifier and successful-write accounting; reads and denied writes are not mutations. Extract rebuilds affected indices and refreshes the live memory instruction before rejecting a failed fork. It does not record successful extraction metadata or write the cursor for that failure.
|
||||
|
||||
Reuse the existing project-index error propagation and best-effort user-index policy. Preserve the execution result through post-fork failures. When recovery itself fails, retain both the fork failure and recovery failure with `AggregateError`. The manager records actual user-memory mutation and touched topics while keeping the task and telemetry status failed. User-mutation bookkeeping retains its existing best-effort behavior.
|
||||
|
||||
Reuse the selector's pre-await start and end history hashes as the window identity. The manager supplies one window-eligibility callback through the shared extraction entry and captures that identity for failure accounting. A process-local project/session counter permits three consecutive failures of an unchanged selection. The fourth request returns `failure_limit` with the persisted cursor and no planner call; it does not write a cursor or mark successful extraction. Skips preserve the streak, a changed selection remains eligible, and non-skipped success or a default-off operation clears that tuple. Other project/session results cannot clear it. The existing Error and write-result contracts remain unchanged, including throws without write evidence.
|
||||
|
||||
## Constraints and risks
|
||||
|
||||
Only reported successful writes can be attributed to the fork. A thrown fork without a result carries no write evidence. Indexing uses the existing document validation; a malformed partial file does not become valid merely because the fork reported it. User-index rebuild failure remains best-effort and does not prove user recall availability. Live memory instruction refresh also remains best-effort: its existing helper catches and warns on underlying refresh errors, so those errors do not reach the typed failure context. Index recovery does not prove live prompt refresh succeeded. The original failed cursor is retained, so the same history can be retried.
|
||||
|
||||
The attempt bound applies to one unchanged selection within one manager while the experiment is enabled. Restarting or switching off the experiment permits retries; a changed window starts its own consecutive streak. It is not a limit on total session calls, tokens or provider billing. Existing configurable per-fork budgets remain in effect. A suspended oldest window can still leave later facts pending: neither this bound nor write recovery moves past that window, schedules remaining history, replays discarded history across sessions, or establishes model quality or token savings. Those requirements remain open. Skipping a failed window on the assumption that no file was written is unsafe.
|
||||
|
||||
## Validation and acceptance
|
||||
|
||||
Verify mapped project/user writes on a failed fork, read-only exclusions, affected index rebuilds and instruction refresh, unchanged cursor and successful-extraction metadata, preserved failure causes and scope when recovery fails, and manager user-mutation bookkeeping with failed status. Retain the existing success, default/opt-in, user-index isolation, cursor holdback and queue/drain regressions.
|
||||
|
||||
Verify three same-window failures followed by a skipped fourth request, including one genuinely queued before the third fails. Check unchanged cursor bytes, explicit skipped task/telemetry, fresh-bootstrap eligibility for a new user window, successful reset, project/session isolation and default-off retries. A matched controlled compiled case first reads its owned index to establish a native cursor, then reproduces four failed forks before the fix and three after it. Retain a later raw user fact beyond the window without claiming it was extracted. This observes attempt counts, not real model cost.
|
||||
|
||||
Build, bundle, typecheck and run affected tests. The completed controlled write-recovery case actually wrote valid files in owned project and user roots, failed, and showed regenerated indices and user mutation without cursor advancement. Do not rerun it merely to relabel the source. Provider execution, representative memory quality, total retry cost and later-history reachability require separate acceptance.
|
||||
@@ -1,35 +0,0 @@
|
||||
# 自动记忆提取失败后的写入恢复与窗口暂停
|
||||
|
||||
[English](auto-memory-failed-extraction.md) | [简体中文](auto-memory-failed-extraction.zh-CN.md)
|
||||
|
||||
状态:#13158 R1-3 的写入恢复和默认关闭的失败窗口尝试次数限制已实现;完整成本和积压历史验收仍未完成。
|
||||
|
||||
## 问题与范围
|
||||
|
||||
fork 可能先写入有效记忆文件,再失败或被取消。结果保留了成功的 `filesWritten`,但提取 planner 在映射记忆主题和作用域之前就抛错。文件仍在磁盘上,索引却没有重建,manager 也没有记录用户记忆变更。成功 fork 后发生索引或游标写入错误,同样会丢失已知的写入作用域。
|
||||
|
||||
恢复已报告的写入,同时不推进失败提取的游标。默认路径和实验窗口路径使用相同的持久性规则。保留现有窗口选择、每次 fork 的预算、冷却、队列和 drain 行为。开启现有冷却实验时,同一选中窗口连续失败三次后暂停,包括后续排队请求。
|
||||
|
||||
已建立的游标在失败后仍选择最早的待处理窗口,因此历史增长可能反复为同一失败选择付费。没有有效历史身份的初始游标则可能选择更新的真实用户轮;整个会话的失败上限会阻断这种已有恢复路径。
|
||||
|
||||
## 决策
|
||||
|
||||
用携带现有映射执行结果的类型化 Error,复用当前记忆根目录分类和成功写入统计;读取及被拒绝的写入不算变更。extract 在拒绝失败 fork 之前,重建受影响的索引并刷新当前记忆指令。此次失败不记录成功提取元数据,也不写入游标。
|
||||
|
||||
复用项目索引错误向上传播、用户索引尽力执行的现有策略。在 fork 之后的失败中保留执行结果。如果恢复本身失败,用 `AggregateError` 同时保留 fork 失败和恢复失败。manager 记录实际用户记忆变更和主题,同时保持任务及遥测状态为失败。用户变更记录沿用现有尽力执行策略。
|
||||
|
||||
复用选择器在 await 前计算的起点和终点历史哈希作为窗口身份。manager 通过共同提取入口提供一个窗口可运行回调,并捕获该身份用于失败记数。按项目及会话保存的进程内计数器允许同一选择连续失败三次。第四次请求返回 `failure_limit` 和持久化游标,不调用 planner、不写游标,也不标记提取成功。跳过保留失败次数,实际选择改变后仍可运行,未跳过的成功操作或关闭实验后的操作清除该元组。其他项目或会话的结果不能清除它。现有 Error 和写入结果契约不变,包括没有写入证据的抛错。
|
||||
|
||||
## 约束与风险
|
||||
|
||||
只有报告成功的写入才能归属于 fork。没有返回结果就抛错的 fork 不携带写入证据。索引沿用现有文档验证;部分文件格式无效时,不会因为 fork 报告写入而变得有效。用户索引重建失败仍采用尽力执行策略,不能证明用户记忆可检索。当前记忆指令刷新也沿用尽力执行策略:现有 helper 捕获底层刷新错误并记录警告,这些错误不会进入类型化失败上下文。索引恢复不能证明实时提示刷新成功。原失败游标保留,因此同一历史仍可能重试。
|
||||
|
||||
尝试次数限制仅适用于开启实验期间、一个 manager 内保持不变的选择。重启或关闭实验允许重试;选择改变后开始自己的连续失败计数。它不限制整个会话的调用次数、token 或提供方账单。现有可配置的每次 fork 预算仍然生效。暂停最早窗口仍可能让后续事实等待处理:次数限制和写入恢复都不越过该窗口、不调度剩余历史、不跨会话回放已丢弃历史,也不证明模型质量或 token 节省。这些要求仍未完成。假设失败没有写入文件而跳过窗口是不安全的。
|
||||
|
||||
## 验证与验收
|
||||
|
||||
验证失败 fork 的项目及用户写入映射、排除纯读取、重建受影响索引和刷新指令、保持游标及成功提取元数据不变、恢复失败时保留两个失败原因与作用域,以及 manager 在失败状态下记录用户变更。保留成功路径、默认及实验路径、用户索引隔离、游标保留和队列及 drain 回归测试。
|
||||
|
||||
验证同一窗口三次失败后第四次跳过,包括第三次失败前真实进入队列的请求。检查游标字节不变、明确的跳过任务和遥测、新用户窗口的初始游标恢复、成功后重置、项目及会话隔离,以及关闭实验时继续重试。匹配的受控编译用例先实际读取自有索引以建立原生游标,再复现修复前四次失败 fork、修复后三次失败 fork。保留窗口后的原始用户事实,不宣称该事实已被提取。此处观测尝试次数,不测真实模型成本。
|
||||
|
||||
执行 build、bundle、typecheck 和相关测试。已经完成的受控写入恢复用例在自有隔离的项目及用户根目录实际写入有效文件后失败,证明索引重建和用户变更记录发生,同时游标未推进。不要为了换源码标签而重跑它。提供方执行、代表性记忆质量、总重试成本及后续历史可达性需要分别验收。
|
||||
@@ -1,45 +0,0 @@
|
||||
# Auto-memory extraction history identity
|
||||
|
||||
[English](auto-memory-history-identity.md) | [简体中文](auto-memory-history-identity.zh-CN.md)
|
||||
|
||||
Status: implemented history-identity and ineligible-tail fixes for the default-off window experiment in #13158; controlled input verification remains separate from model acceptance.
|
||||
|
||||
## Problem and scope
|
||||
|
||||
The project cursor's `processedOffset` becomes a content selector when `preserveUnprocessedHistory` is enabled. A foreign session, compression, or history replacement can make that index select unrelated content. Comparing only the current length cannot detect compression followed by regrowth or a same-length replacement. Starting at the latest 40 entries also loses the user preference at the beginning of a long tool turn.
|
||||
|
||||
Keep the default extraction path and the 40-entry window cap unchanged. Flush pending windows before compaction in the opt-in experiment. Shutdown replay, partial-write retries, and model quality and token acceptance remain separate.
|
||||
|
||||
## Decision
|
||||
|
||||
Add an optional `processedHistoryHash` to the cursor. Only the window experiment writes or reads it. Hash the processed raw-history prefix with Node's SHA-256, including the structural-prefix length but excluding its synthetic content. Feed one entry at a time to avoid allocating a full-history JSON string. An offset is reusable only when its session, bounds, and prefix hash match.
|
||||
|
||||
Without an identity, or in a different session, start at the latest genuine user prompt and retain the whole pending tool turn across bounded windows. Reuse the existing user-prompt classifier, excluding delivered task notifications. A same-session identity mismatch restarts at the first real entry because the changed position cannot be recovered from a hash. A compressed prefix also starts after the synthetic entries, including for legacy cursors, so all subsequent real facts remain reachable.
|
||||
|
||||
Use `getStartupContextLength` with `includeCompressed: true` before history curation. Preserve a trailing function call belonging to the compressed prefix together with its response. Exclude summaries and full-file attachments from the selected raw history. Do not trim a curated prefix: curation can merge an attachment with the next genuine prompt.
|
||||
|
||||
Compute the start and end hashes before invoking the asynchronous extractor. Persist the hash at the offset actually written, including the live-end zero-tool holdback and the no-user early return. Default-path writes omit the field, so enabling the experiment after a default run bootstraps rather than trusts an unattested index.
|
||||
|
||||
Keep the fork's selected window capped, but consume its remaining tail in the same execution when that entire tail has no eligible user text and the selected window made genuine progress. Reuse the no-user gate's per-part classifier: tool responses, model text, runtime reminders, and hidden reasoning alone do not require another fork. The no-user early return may likewise consume such a tail. If any eligible user text remains, keep the existing window boundary and leave the entire remainder pending. A zero-tool completion retains its existing capped-window advancement or live-end holdback; the free tail must not convert it into a cooldown-arming no-op. Attest the consumable tail before the asynchronous fork, so changes to that content invalidate the cursor on the next execution.
|
||||
|
||||
### Pre-compaction flush
|
||||
|
||||
Before the PreCompact hook and summary, the shared compression entry flushes the main session's raw shallow history when the cooldown experiment and managed memory are enabled. Reuse the existing extractor, history identity, and 40-entry windows; do not retain another history queue or add a scheduler. Exclude subagent identity contexts so memory forks cannot flush recursively. The existing main-session notification boundary exits both the agent frame and its separate identity/name contexts, preventing an inherited worker identity from bypassing this flush.
|
||||
|
||||
All windows share a two-minute cancellation deadline and the caller's cancellation signal. An existing extraction, a suspended window, failure, or no advancement leaves the original history intact and returns a memory-flush compression failure. Existing compression failure limits apply. A later attempt resumes from the last committed window; a zero-tool live-end completion retains its existing holdback.
|
||||
|
||||
Forward cancellation to the extraction fork and check it at the atomic cursor commit. Recovery of valid writes from a failed fork remains in place; filesystem recovery can finish after model cancellation. This boundary does not implement shutdown replay or prove memory quality and cost.
|
||||
|
||||
## Constraints and risks
|
||||
|
||||
Prefix validation costs CPU proportional to the processed raw content in the opt-in experiment. Structural startup refreshes do not invalidate the index, but changes to processed conversation content intentionally restart extraction and may repeat facts. The project cursor remains shared between sessions; session changes bootstrap the current user turn. Facts already discarded by compression cannot be recovered. Media-only prompt classification and synthetic-prefix recognition retain the existing helpers' limits.
|
||||
|
||||
Early tail consumption changes future extraction input: a later user prompt can no longer bring that old tool or model context into the next pending window, whereas it could before a subsequent no-user execution consumed the tail. User-text eligibility alone does not prove unchanged memory quality. Representative quality and cost acceptance remains required before enabling the experiment.
|
||||
|
||||
## Validation and acceptance
|
||||
|
||||
Regression tests must cover a legal new session inheriting long history, a fresh long tool turn, compressed summaries and attachments, shrink followed by regrowth, same-length replacement with an unchanged boundary entry, startup-reminder refresh, asynchronous mutation, and preserved call/response pairs. Verify ineligible-tail consumption with and without a fork, pre-fork tail attestation, retained later user facts, and no extra fork or larger agent budget. Existing empty-window and zero-tool holdback tests must still pass with an attested cursor.
|
||||
|
||||
Build, bundle, typecheck, and run the affected extraction, planner, manager, and client tests. Repeat the controlled native-composer input-selection reproduction against the fixed source. Acceptance requires real post-compression facts to remain reachable, synthetic user context to stay out of the selected window, and default extraction behavior to remain unchanged. Controlled forks prove input selection only, not provider execution, memory quality, or cost reduction.
|
||||
|
||||
Pre-compaction validation must cover raw versus curated history, consecutive cooled backlog windows, extraction failure and holdback, an existing running extraction, cancellation preventing cursor commit, and memory-fork exclusion. Build, bundle, typecheck, and affected failure-status consumer tests must pass. Controlled offline verification still does not replace real-provider quality and cost acceptance.
|
||||
@@ -1,45 +0,0 @@
|
||||
# 自动记忆提取的历史身份
|
||||
|
||||
[English](auto-memory-history-identity.md) | [简体中文](auto-memory-history-identity.zh-CN.md)
|
||||
|
||||
状态:已实现 #13158 默认关闭窗口实验的历史身份和无可提取文本尾部修复;受控输入验证与模型验收分开。
|
||||
|
||||
## 问题与范围
|
||||
|
||||
启用 `preserveUnprocessedHistory` 后,项目游标的 `processedOffset` 成为内容选择器。其他会话、压缩或历史替换会使它指向无关内容。仅比较当前长度无法检测压缩后重新增长或同长替换。直接从最后 40 项开始也会遗漏长工具轮开头的用户偏好。
|
||||
|
||||
保持默认提取路径和 40 项窗口上限不变。在可选实验中,压缩前冲刷待处理窗口。退出重放、部分写入重试,以及模型质量与 token 成本验收仍需单独完成。
|
||||
|
||||
## 决策
|
||||
|
||||
为游标添加可选的 `processedHistoryHash`,仅窗口实验读写它。使用 Node 的 SHA-256 对已处理原始历史前缀计算指纹,包含结构前缀长度,但排除其中的合成内容。逐项输入,避免分配完整历史的 JSON 字符串。只有会话、边界和前缀指纹都匹配时才能复用偏移量。
|
||||
|
||||
缺少身份或切换会话时,从最近真实用户提示开始,通过有界窗口保留整个待处理工具轮。复用现有用户提示分类器,排除已投递的任务通知。同会话身份失配时从首条真实历史重新开始,因为指纹无法恢复具体变更位置。遇到压缩前缀时,即使是旧版游标,也从合成条目之后开始,让后续全部真实事实仍可到达。
|
||||
|
||||
在历史整理之前调用带 `includeCompressed: true` 的 `getStartupContextLength`。保留压缩前缀末尾的函数调用,使它与响应一起处理。从所选原始历史中排除摘要和完整文件附件。不要裁剪已整理的前缀:整理可能把附件与下一条真实提示合并。
|
||||
|
||||
在调用异步提取器前计算起点和终点指纹。持久化实际写入偏移量对应的指纹,包括末尾零工具调用时保留起点、无用户消息时提前返回这两条路径。默认路径写入时省略此字段,因此在默认提取后启用实验会重新初始化,而不是信任未经认证的索引。
|
||||
|
||||
保持 fork 所选窗口的上限;当整个剩余尾部都没有可提取的用户文本,且所选窗口取得真实进展时,在同次执行中消费尾部。复用无用户消息门槛的逐 part 分类:工具响应、模型文本、运行时提醒和隐藏推理本身不需要再次 fork。无用户消息的提前返回路径也可以消费这种尾部。只要尾部还有任何可提取的用户文本,就保留原有窗口边界,让整个剩余部分继续待处理。零工具调用完成时仍按原规则推进有上限的历史窗口,或保留实时末尾的起点;免费消费尾部不能把它变成会触发冷却的空操作。在异步 fork 之前认证可消费尾部,使这些内容的变化能在下次执行时使游标失效。
|
||||
|
||||
### 压缩前冲刷
|
||||
|
||||
启用冷却实验和托管记忆时,共用压缩入口在 PreCompact 钩子和摘要之前冲刷主会话的原始浅拷贝历史。复用现有提取器、历史身份和 40 项窗口,不额外保留历史队列或新增调度器。排除子代理身份上下文,避免记忆 fork 递归冲刷。现有主会话通知边界同时退出 agent frame 和独立的身份/名称上下文,防止继承的 worker 身份绕过冲刷。
|
||||
|
||||
全部窗口共用两分钟取消截止时间和调用者的取消信号。已有提取任务、窗口暂停、失败或未推进时,保留原历史并返回记忆冲刷导致的压缩失败。沿用现有压缩失败限制。后续尝试从最后已提交窗口继续;实时末尾的零工具调用完成仍保留原有游标暂缓规则。
|
||||
|
||||
把取消信号传入提取 fork,并在原子游标提交时检查。保留失败 fork 有效写入的恢复;文件系统恢复可能在模型取消后完成。此边界不实现退出重放,也不证明记忆质量和成本。
|
||||
|
||||
## 约束与风险
|
||||
|
||||
可选实验的前缀校验 CPU 成本与已处理原始内容量成正比。结构化启动信息刷新不会使索引失效,但已处理会话内容的变化会有意重启提取,可能重复事实。项目游标仍由多个会话共享,切换会话时从当前用户轮初始化。已经被压缩丢弃的事实无法恢复。纯媒体提示分类和合成前缀识别保留现有辅助函数的限制。
|
||||
|
||||
提前消费尾部会改变未来提取的输入:后续用户提示无法再把这些旧工具或模型上下文带入下一个待处理窗口,而此前在下一次无用户消息执行消费尾部之前仍可能带入。仅凭用户文本的提取门槛,不能证明记忆质量不变。启用实验之前仍必须完成代表性质量与成本验收。
|
||||
|
||||
## 验证与验收
|
||||
|
||||
回归测试必须覆盖合法新会话继承长历史、首次长工具轮、压缩摘要和附件、缩短后重新增长、边界条目不变的同长替换、启动提醒刷新、异步期间历史变化,以及调用与响应配对。验证有 fork 和无 fork 时的无可提取文本尾部消费、fork 前的尾部认证、后续用户事实仍待处理,以及没有增加 fork 或扩大 agent 预算。原有空窗口和零工具调用保留游标测试在已认证游标下必须继续通过。
|
||||
|
||||
执行构建、打包、类型检查,以及受影响的提取、规划器、管理器和客户端测试。在修复后的源码上重复受控原生 composer 输入选择复现。验收要求真实压缩后事实仍可到达、合成用户上下文不进入所选窗口,且默认提取行为不变。受控 fork 只证明输入选择,不证明提供商执行、记忆质量或成本降低。
|
||||
|
||||
压缩前验证必须覆盖原始与整理后历史的区别、冷却积压的连续窗口、提取失败与暂缓、已有运行任务、取消后的游标不提交,以及记忆 fork 排除。构建、打包、类型检查和受影响的失败状态消费者测试必须通过。离线受控验证仍不能替代真实提供商质量与成本验收。
|
||||
@@ -193,9 +193,19 @@ The fast scorer and selector read the same snapshot but have different roles:
|
||||
|
||||
Steps 2 and 4 are conditional while the experimental skip-selector knob
|
||||
(`QWEN_CODE_MEMORY_RECALL_SKIP_SELECTOR_ON_UNIQUE_STRONG_HIT`, off by default)
|
||||
is set: a single fast hit that is strong and current suppresses the selector
|
||||
for that recall, so no selector pass runs and no refined result arrives. The
|
||||
knob is an ablation experiment; the default contract above is unchanged.
|
||||
is set: a single fast hit matching the query by title or keyword, whose body
|
||||
is not already in context, suppresses the selector. Metadata-only substring
|
||||
matches and already-present or stale bodies keep the selector. Count fast
|
||||
candidates before prompt trimming so a second match cannot disappear from the
|
||||
guard. No refined result arrives after a skip; the shared client delivers it
|
||||
as the fast phase. This is an ablation experiment; default behavior is unchanged.
|
||||
|
||||
Extraction cadence is unchanged. The #13004 cooldown implementation was removed
|
||||
from #13158: its shutdown tail and suspended oldest windows require durable
|
||||
replay before skipped facts can be preserved. Its raw-window, timestamp,
|
||||
failure-limit and compaction changes are also removed. Historical code remains
|
||||
at `a4568de26e49`; #13004 stays open. Selector-only validation must not be cited
|
||||
as extraction, quality, or token-savings acceptance.
|
||||
|
||||
Short Latin keywords require token boundaries, so `ai` does not match
|
||||
`explain`. Han, Hiragana, Katakana, and Hangul use a shared CJK tokenizer. Body
|
||||
|
||||
@@ -176,9 +176,15 @@ ref、标题、描述与适用场景。聚焦提示词有固定的字符预算
|
||||
|
||||
当实验性的跳过选择器开关
|
||||
(`QWEN_CODE_MEMORY_RECALL_SKIP_SELECTOR_ON_UNIQUE_STRONG_HIT`,默认关闭)
|
||||
开启时,第 2、4 步是有条件的:唯一且强、且内容未过时的快速命中会抑制本次召回的
|
||||
选择器,因此不会发生选择器调用,也不会有精修结果到达。该开关是消融实验;
|
||||
上述默认契约不变。
|
||||
开启时,第 2、4 步是有条件的:快速命中唯一、查询匹配其标题或关键词,且正文尚未
|
||||
存在于上下文中时,才跳过选择器。仅描述等元数据的子串匹配、正文已存在或过时的
|
||||
情况仍运行选择器。候选数量在提示裁剪前计算,不能因第二个候选被裁掉而触发跳过。
|
||||
跳过后没有精修结果,共享 client 按快速阶段投递。该开关仍是消融实验,默认行为不变。
|
||||
|
||||
提取频率保持原样。#13158 已撤下 #13004 冷却实现:退出尾部和暂停的最旧窗口
|
||||
需要持久化重放才能保全跳过的事实。相关原始窗口、时间戳、失败次数限制与压缩改动
|
||||
也已移除。历史代码保留在 `a4568de26e49`,#13004 继续开放。仅选择器的验证不能
|
||||
冒充提取、质量或 token 节省验收。
|
||||
|
||||
短拉丁关键词要求 token 边界,因此 `ai` 不会匹配 `explain`。汉字、平假名、
|
||||
片假名与谚文使用共享的 CJK 分词器。正文文本是低权重兜底,
|
||||
|
||||
@@ -752,7 +752,7 @@ The following events are logged:
|
||||
- **Attributes**: `trigger` ("auto"/"manual"), `status` ("updated"/"noop"/"failed"/"cancelled"), `deduped_entries` (int), `touched_topics_count` (int), `touched_topics` (string), `duration_ms` (int)
|
||||
|
||||
- `qwen-code.memory.recall`: Memory recall operation completed.
|
||||
- **Attributes**: `query_length` (int), `docs_scanned` (int), `docs_selected` (int), `strategy` ("none"/"heuristic"/"model"), `duration_ms` (int), `selector_skipped` (bool, optional — set only for structured-mode recalls while the #13003 skip-selector knob is on)
|
||||
- **Attributes**: `query_length` (int), `docs_scanned` (int), `docs_selected` (int), `strategy` ("none"/"heuristic"/"model"), `duration_ms` (int), `selector_skipped` (bool, optional — set for structured-mode recalls; false when the selector runs, including when the #13003 knob is off)
|
||||
|
||||
#### Prompt Suggestion & Speculation Events
|
||||
|
||||
|
||||
@@ -343,6 +343,7 @@ export const legacyCoreBarrelImports = [
|
||||
'packages/cli/src/ui/utils/backgroundWorkUtils.ts',
|
||||
'packages/cli/src/ui/utils/clipboardUtils.ts',
|
||||
'packages/cli/src/ui/utils/commandUtils.ts',
|
||||
'packages/cli/src/ui/utils/compression-text.ts',
|
||||
'packages/cli/src/ui/utils/export/export-transcript-document.ts',
|
||||
'packages/cli/src/ui/utils/export/normalize.ts',
|
||||
'packages/cli/src/ui/utils/formatters.ts',
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
*/
|
||||
|
||||
import {
|
||||
CompressionStatus,
|
||||
findProviderByCredentials,
|
||||
getExtensionDisplayName,
|
||||
getMCPServerStatus,
|
||||
@@ -26,7 +27,6 @@ import {
|
||||
resolveMetadataKey,
|
||||
uiTelemetryService,
|
||||
} from '@qwen-code/qwen-code-core';
|
||||
import { CompressionStatus } from '@qwen-code/qwen-code-core/core/turn.js';
|
||||
import type {
|
||||
Config,
|
||||
SessionMetrics,
|
||||
@@ -860,8 +860,6 @@ export function projectCompression(compression: {
|
||||
: 'Chat history compression did not reduce size. This may indicate issues with the compression prompt.';
|
||||
case CompressionStatus.COMPRESSION_FAILED_TOKEN_COUNT_ERROR:
|
||||
return 'Could not compress chat history due to a token counting error.';
|
||||
case CompressionStatus.COMPRESSION_FAILED_MEMORY_FLUSH:
|
||||
return 'Failed to compress chat history.';
|
||||
case CompressionStatus.NOOP:
|
||||
return 'Nothing to compress.';
|
||||
default:
|
||||
|
||||
@@ -70,8 +70,6 @@ export function compactionText(props: CompactionViewProps): string {
|
||||
);
|
||||
case CompressionStatus.COMPRESSION_FAILED_API_ERROR:
|
||||
return t('Could not compress chat history due to an API error.');
|
||||
case CompressionStatus.COMPRESSION_FAILED_MEMORY_FLUSH:
|
||||
return t('Failed to compress chat history.');
|
||||
case CompressionStatus.NOOP:
|
||||
return 'Nothing to compress.';
|
||||
default:
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
import {
|
||||
CompressionStatus,
|
||||
isCompressionFailureStatus,
|
||||
} from '@qwen-code/qwen-code-core/core/turn.js';
|
||||
} from '@qwen-code/qwen-code-core';
|
||||
import { t } from '../../i18n/index.js';
|
||||
|
||||
export const formatCompressionTokenCount = (
|
||||
@@ -76,8 +76,6 @@ export function getCompressionStatusText({
|
||||
);
|
||||
case CompressionStatus.COMPRESSION_FAILED_API_ERROR:
|
||||
return t('Could not compress chat history due to an API error.');
|
||||
case CompressionStatus.COMPRESSION_FAILED_MEMORY_FLUSH:
|
||||
return t('Failed to compress chat history.');
|
||||
case CompressionStatus.NOOP:
|
||||
return 'Nothing to compress.';
|
||||
default:
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
/**
|
||||
* How many trailing curated history entries the cache-safe snapshot keeps.
|
||||
* A forked agent — the managed auto-memory extractor among them — sees only
|
||||
* this tail of the conversation, not all of it.
|
||||
*
|
||||
* Kept in a module with no imports so the memory manager can derive its own
|
||||
* bound from it at load time without joining the forkedAgent → config cycle.
|
||||
*/
|
||||
export const CACHE_SAFE_HISTORY_TAIL_ENTRIES = 40;
|
||||
@@ -47,7 +47,6 @@ import { FunctionCallingConfigMode } from '../core/genai-compat.js';
|
||||
import { createRuntimeContentGeneratorView } from '../models/content-generator-config.js';
|
||||
import { createApprovalModeOverride } from '../tools/agent/agent.js';
|
||||
import { createDebugLogger } from '../utils/debugLogger.js';
|
||||
import { CACHE_SAFE_HISTORY_TAIL_ENTRIES } from './cache-safe-history.js';
|
||||
import {
|
||||
AgentHeadless,
|
||||
AgentEventEmitter,
|
||||
@@ -222,7 +221,7 @@ export function createForkedChat(
|
||||
config: Config,
|
||||
params: CacheSafeParams,
|
||||
): LlmChat {
|
||||
const maxHistoryEntries = CACHE_SAFE_HISTORY_TAIL_ENTRIES;
|
||||
const maxHistoryEntries = 40;
|
||||
const history =
|
||||
params.history.length > maxHistoryEntries
|
||||
? params.history.slice(-maxHistoryEntries)
|
||||
|
||||
@@ -13,7 +13,6 @@ import {
|
||||
getCurrentAgentId,
|
||||
getRuntimeContentGenerator,
|
||||
isTopLevelSession,
|
||||
runOutsideAgentContext,
|
||||
runWithAgentContext,
|
||||
runWithAgentDisallowedTools,
|
||||
runWithRuntimeContentGenerator,
|
||||
@@ -22,10 +21,6 @@ import {
|
||||
} from './agent-context.js';
|
||||
import { runWithTeammateIdentity } from '../team/identity.js';
|
||||
import { runInForkContext } from '../../tools/agent/fork-subagent.js';
|
||||
import {
|
||||
subagentIdentityContext,
|
||||
subagentNameContext,
|
||||
} from '../../utils/subagentNameContext.js';
|
||||
import {
|
||||
AuthType,
|
||||
type ContentGenerator,
|
||||
@@ -113,27 +108,6 @@ describe('agent-context (agentId)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('runOutsideAgentContext', () => {
|
||||
it('clears subagent identity for main-session continuations and restores the caller', async () => {
|
||||
const identity = { type: 'worker', id: 'worker' };
|
||||
await runWithAgentContext('worker', () =>
|
||||
subagentIdentityContext.run(identity, () =>
|
||||
subagentNameContext.run('worker', async () => {
|
||||
await runOutsideAgentContext(async () => {
|
||||
await Promise.resolve();
|
||||
expect(getCurrentAgentId()).toBeNull();
|
||||
expect(subagentIdentityContext.getStore()).toBeUndefined();
|
||||
expect(subagentNameContext.getStore()).toBeUndefined();
|
||||
});
|
||||
expect(getCurrentAgentId()).toBe('worker');
|
||||
expect(subagentIdentityContext.getStore()).toBe(identity);
|
||||
expect(subagentNameContext.getStore()).toBe('worker');
|
||||
}),
|
||||
),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('agent-context (runtimeView)', () => {
|
||||
it('returns undefined outside any frame', () => {
|
||||
expect(getRuntimeContentGenerator()).toBeUndefined();
|
||||
|
||||
@@ -19,10 +19,6 @@
|
||||
*/
|
||||
|
||||
import { AsyncLocalStorage } from 'node:async_hooks';
|
||||
import {
|
||||
subagentIdentityContext,
|
||||
subagentNameContext,
|
||||
} from '../../utils/subagentNameContext.js';
|
||||
import { runOutsideHookExecutionOwner } from '../../hooks/hook-execution-context.js';
|
||||
import type { LlmChat } from '../../core/llm-chat.js';
|
||||
import type {
|
||||
@@ -189,11 +185,7 @@ export function getRuntimeContentGenerator():
|
||||
* with this helper.
|
||||
*/
|
||||
export function runOutsideAgentContext<T>(fn: () => T): T {
|
||||
return storage.exit(() =>
|
||||
subagentIdentityContext.exit(() =>
|
||||
subagentNameContext.exit(() => runOutsideHookExecutionOwner(fn)),
|
||||
),
|
||||
);
|
||||
return storage.exit(() => runOutsideHookExecutionOwner(fn));
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -47,7 +47,6 @@ import { MemoryManager } from '../memory/manager.js';
|
||||
import { buildAgentContentGeneratorConfig } from '../models/content-generator-config.js';
|
||||
import { LlmChat, userContentPushSnapshotKey } from './llm-chat.js';
|
||||
import { DEFAULT_TOKEN_LIMIT } from './tokenLimits.js';
|
||||
import { computeThresholds } from '../services/chatCompressionService.js';
|
||||
import type { Config } from '../config/config.js';
|
||||
import { ApprovalMode } from '../config/config.js';
|
||||
import type { RelevantAutoMemoryPromptResult } from '../memory/manager.js';
|
||||
@@ -921,7 +920,6 @@ describe('Gemini Client (client.ts)', () => {
|
||||
}),
|
||||
getCliVersion: vi.fn().mockReturnValue('1.0.0'),
|
||||
getChatCompression: vi.fn().mockReturnValue(undefined),
|
||||
getAutoCompactThreshold: vi.fn().mockReturnValue(undefined),
|
||||
getSkipNextSpeakerCheck: vi.fn().mockReturnValue(false),
|
||||
getUseModelRouter: vi.fn().mockReturnValue(false),
|
||||
getProjectRoot: vi.fn().mockReturnValue('/test/project/root'),
|
||||
@@ -8045,7 +8043,6 @@ Other open files:
|
||||
|
||||
mockTurnRunFn.mockReturnValue(textTurn('Done'));
|
||||
|
||||
const tokenCount = vi.fn();
|
||||
const mockChat = installChat({
|
||||
getHistory: vi
|
||||
.fn()
|
||||
@@ -8053,7 +8050,6 @@ Other open files:
|
||||
userText('I prefer terse responses.'),
|
||||
modelText('Done'),
|
||||
]),
|
||||
getLastPromptTokenCount: tokenCount,
|
||||
});
|
||||
|
||||
const events = await run(
|
||||
@@ -8068,27 +8064,7 @@ Other open files:
|
||||
sessionId: 'test-session-id',
|
||||
history: recordedHistory,
|
||||
config: mockConfig,
|
||||
// Read only while a no-op cooldown is active (#13004).
|
||||
isBelowCompactionWarn: expect.any(Function),
|
||||
});
|
||||
const cooldownPosition =
|
||||
mockMemoryManager.scheduleExtract.mock.calls.at(-1)?.[0]
|
||||
.isBelowCompactionWarn;
|
||||
const window = 100_000;
|
||||
vi.mocked(mockConfig.getContentGeneratorConfig).mockReturnValue({
|
||||
...mockConfig.getContentGeneratorConfig()!,
|
||||
contextWindowSize: window,
|
||||
});
|
||||
const { warn } = computeThresholds(
|
||||
window,
|
||||
mockConfig.getAutoCompactThreshold(),
|
||||
);
|
||||
tokenCount.mockReturnValue(0);
|
||||
expect(cooldownPosition?.()).toBe(false);
|
||||
tokenCount.mockReturnValue(warn - 1);
|
||||
expect(cooldownPosition?.()).toBe(true);
|
||||
tokenCount.mockReturnValue(warn);
|
||||
expect(cooldownPosition?.()).toBe(false);
|
||||
expect(mockMemoryManager.scheduleMetadataMigration).toHaveBeenCalledTimes(
|
||||
2,
|
||||
);
|
||||
|
||||
@@ -57,7 +57,6 @@ import {
|
||||
import { formatStopHookBlockingCapWarning } from '../hooks/stopHookCap.js';
|
||||
import { buildContextUsage } from '../hooks/context-usage.js';
|
||||
import { DEFAULT_TOKEN_LIMIT, tokenLimit } from './tokenLimits.js';
|
||||
import { computeThresholds } from '../services/chatCompressionService.js';
|
||||
import { createSessionStartProfiler } from './session-start-profiler.js';
|
||||
|
||||
const debugLogger = createDebugLogger('CLIENT');
|
||||
@@ -142,7 +141,6 @@ import {
|
||||
saveCacheSafeParams,
|
||||
clearCacheSafeParams,
|
||||
} from '../agents/forkedAgent.js';
|
||||
import { CACHE_SAFE_HISTORY_TAIL_ENTRIES } from '../agents/cache-safe-history.js';
|
||||
|
||||
// Utilities
|
||||
import {
|
||||
@@ -1439,10 +1437,7 @@ export class LlmClient {
|
||||
captureCacheSafeParams(): void {
|
||||
try {
|
||||
const chat = this.getChat();
|
||||
const historyForCache = this.getHistoryTailShallow(
|
||||
CACHE_SAFE_HISTORY_TAIL_ENTRIES,
|
||||
true,
|
||||
);
|
||||
const historyForCache = this.getHistoryTailShallow(40, true);
|
||||
const cachedHistory = slimCompactionInput(
|
||||
historyForCache,
|
||||
this.config.getEffectiveInputModalities(),
|
||||
@@ -1496,32 +1491,6 @@ export class LlmClient {
|
||||
this.pendingMemoryPrefetch?.fastDeliveredRefs.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the last prompt sits below the compaction warn threshold (#13004).
|
||||
* This is a cooldown heuristic, not a guarantee against compaction on the
|
||||
* next send (which can add a large input). Unknown counts as not below.
|
||||
*/
|
||||
private isBelowCompactionWarn(): boolean {
|
||||
// Match the gate that decides the next send: estimatePromptTokens adds
|
||||
// the previous response's output tokens on top of the prompt count, so
|
||||
// this guard must too — with a thinking-heavy turn the output alone can
|
||||
// consume the whole warn→auto buffer and the cooldown would skip turns
|
||||
// straight across a compaction. Optional-chained: installChat mocks
|
||||
// supply only getLastPromptTokenCount.
|
||||
const promptTokens =
|
||||
(this.chat?.getLastPromptTokenCount() ?? 0) +
|
||||
(this.chat?.getLastOutputTokenCount?.() ?? 0);
|
||||
if (promptTokens <= 0) return false;
|
||||
const window =
|
||||
this.config.getContentGeneratorConfig()?.contextWindowSize ??
|
||||
DEFAULT_TOKEN_LIMIT;
|
||||
const { warn } = computeThresholds(
|
||||
window,
|
||||
this.config.getAutoCompactThreshold(),
|
||||
);
|
||||
return promptTokens < warn;
|
||||
}
|
||||
|
||||
/** @internal */
|
||||
resetManagedAutoMemoryAfterCompression(): void {
|
||||
this.resetManagedAutoMemoryDeliveryState();
|
||||
@@ -3171,7 +3140,6 @@ export class LlmClient {
|
||||
sessionId,
|
||||
history,
|
||||
config: this.config,
|
||||
isBelowCompactionWarn: () => this.isBelowCompactionWarn(),
|
||||
})
|
||||
.then((result) => result.touchedTopics.length)
|
||||
.catch((error: unknown) => {
|
||||
|
||||
@@ -1595,9 +1595,7 @@ function validateHistory(history: Content[]) {
|
||||
* filters or recitation). Extracting valid turns from the history
|
||||
* ensures that subsequent requests could be accepted by the model.
|
||||
*/
|
||||
export function extractCuratedHistory(
|
||||
comprehensiveHistory: Content[],
|
||||
): Content[] {
|
||||
function extractCuratedHistory(comprehensiveHistory: Content[]): Content[] {
|
||||
if (comprehensiveHistory === undefined || comprehensiveHistory.length === 0) {
|
||||
return [];
|
||||
}
|
||||
|
||||
@@ -67,7 +67,6 @@ describe('isCompressionFailureStatus', () => {
|
||||
CompressionStatus.COMPRESSION_FAILED_EMPTY_SUMMARY,
|
||||
CompressionStatus.COMPRESSION_FAILED_OUTPUT_TRUNCATED,
|
||||
CompressionStatus.COMPRESSION_FAILED_API_ERROR,
|
||||
CompressionStatus.COMPRESSION_FAILED_MEMORY_FLUSH,
|
||||
]) {
|
||||
expect(isCompressionFailureStatus(status)).toBe(true);
|
||||
}
|
||||
|
||||
@@ -467,9 +467,6 @@ export enum CompressionStatus {
|
||||
* apart from model output quality failures.
|
||||
*/
|
||||
COMPRESSION_FAILED_API_ERROR,
|
||||
|
||||
/** Pending memory could not be extracted before replacing the history. */
|
||||
COMPRESSION_FAILED_MEMORY_FLUSH,
|
||||
}
|
||||
|
||||
export function isCompressionFailureStatus(
|
||||
@@ -480,8 +477,7 @@ export function isCompressionFailureStatus(
|
||||
status === CompressionStatus.COMPRESSION_FAILED_TOKEN_COUNT_ERROR ||
|
||||
status === CompressionStatus.COMPRESSION_FAILED_EMPTY_SUMMARY ||
|
||||
status === CompressionStatus.COMPRESSION_FAILED_OUTPUT_TRUNCATED ||
|
||||
status === CompressionStatus.COMPRESSION_FAILED_API_ERROR ||
|
||||
status === CompressionStatus.COMPRESSION_FAILED_MEMORY_FLUSH
|
||||
status === CompressionStatus.COMPRESSION_FAILED_API_ERROR
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -5,16 +5,11 @@
|
||||
*/
|
||||
|
||||
import * as fs from 'node:fs/promises';
|
||||
import { createHash } from 'node:crypto';
|
||||
import type { Content } from '@google/genai';
|
||||
import type { Config } from '../config/config.js';
|
||||
import { atomicWriteFile } from '../utils/atomicFileWrite.js';
|
||||
import { createDebugLogger } from '../utils/debugLogger.js';
|
||||
import {
|
||||
getStartupContextLength,
|
||||
stripSystemReminderBlocks,
|
||||
} from '../core/environmentContext.js';
|
||||
import { isApiUserPrompt } from '../services/api-user-prompt.js';
|
||||
import { partToString } from '../utils/partUtils.js';
|
||||
import {
|
||||
getAutoMemoryExtractCursorPath,
|
||||
getAutoMemoryMetadataPath,
|
||||
@@ -23,16 +18,12 @@ import {
|
||||
ensureAutoMemoryScaffold,
|
||||
ensureUserAutoMemoryScaffold,
|
||||
} from './store.js';
|
||||
import {
|
||||
AutoMemoryExtractionError,
|
||||
runAutoMemoryExtractionByAgent,
|
||||
} from './extractionAgentPlanner.js';
|
||||
import { runAutoMemoryExtractionByAgent } from './extractionAgentPlanner.js';
|
||||
import {
|
||||
rebuildManagedAutoMemoryIndex,
|
||||
rebuildUserAutoMemoryIndex,
|
||||
} from './indexer.js';
|
||||
import { getCacheSafeParamsSessionId } from '../agents/forkedAgent.js';
|
||||
import { CACHE_SAFE_HISTORY_TAIL_ENTRIES } from '../agents/cache-safe-history.js';
|
||||
import { refreshMemoryInstruction } from './refresh.js';
|
||||
import {
|
||||
type AutoMemoryExtractCursor,
|
||||
@@ -50,28 +41,9 @@ export interface AutoMemoryExtractResult {
|
||||
| 'queued'
|
||||
| 'memory_tool'
|
||||
| 'memory_pressure'
|
||||
| 'session_mismatch'
|
||||
| 'cooldown'
|
||||
| 'failure_limit'
|
||||
| 'no_user_text';
|
||||
| 'session_mismatch';
|
||||
systemMessage?: string;
|
||||
cursor: AutoMemoryExtractCursor;
|
||||
/**
|
||||
* True when the extraction agent actually ran and completed. Absent on every
|
||||
* early return (no new user messages, session mismatch), which otherwise
|
||||
* share a no-op's shape; the cooldown in `MemoryManager` must not treat
|
||||
* those as a completed no-op (#13004).
|
||||
*/
|
||||
extractorRan?: true;
|
||||
/**
|
||||
* True when this run's cursor advanced past the offset it started from.
|
||||
* The cooldown arms on a completed no-op that consumed its window; a cursor
|
||||
* merely held at the live end by the zero-tool-call guard must not arm it,
|
||||
* and a windowed no-op that stops short of `history.length` must be able
|
||||
* to (#13004's tool-heavy-session case, where the window caps below the
|
||||
* live end every turn).
|
||||
*/
|
||||
cursorAdvanced?: true;
|
||||
}
|
||||
|
||||
function getSessionMismatchResult(
|
||||
@@ -94,68 +66,6 @@ function getSessionMismatchResult(
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Cut-safety predicates for the pending-window boundary. A cut is unsafe when
|
||||
* the last included entry is a model entry carrying a functionCall (its
|
||||
* response falls outside the window and the trailing repair would fabricate
|
||||
* one reusing the real call's id), or when the next entry is a user entry
|
||||
* that is all functionResponse parts (the next window would open on an
|
||||
* orphaned response whose call sits in already-processed history, and no
|
||||
* extraction run would ever see it — the cursor only advances).
|
||||
*/
|
||||
function endsOnOpenCall(
|
||||
history: readonly Content[],
|
||||
endOffset: number,
|
||||
): boolean {
|
||||
const last = history[endOffset - 1];
|
||||
return (
|
||||
last?.role === 'model' &&
|
||||
(last.parts ?? []).some((part) => part.functionCall)
|
||||
);
|
||||
}
|
||||
|
||||
function opensOnOrphanedResponse(
|
||||
history: readonly Content[],
|
||||
endOffset: number,
|
||||
): boolean {
|
||||
const next = history[endOffset];
|
||||
return (
|
||||
next?.role === 'user' &&
|
||||
(next.parts ?? []).length > 0 &&
|
||||
(next.parts ?? []).every((part) => part.functionResponse)
|
||||
);
|
||||
}
|
||||
|
||||
function hasUserText(content: Content): boolean {
|
||||
return (
|
||||
content.role === 'user' &&
|
||||
(content.parts ?? []).some(
|
||||
(part) =>
|
||||
!part.thought &&
|
||||
(typeof part.text === 'string'
|
||||
? stripSystemReminderBlocks(part.text).trim().length > 0
|
||||
: // A media-only user turn (image/file) has no text but is still
|
||||
// user content — counting it as empty would advance the cursor
|
||||
// past a turn no extractor ever sees, and the cursor only moves
|
||||
// forward.
|
||||
part.inlineData !== undefined || part.fileData !== undefined),
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
function hashProcessedHistory(
|
||||
history: Content[],
|
||||
floor: number,
|
||||
offset: number,
|
||||
): string {
|
||||
// ponytail: O(processed history); use a chat revision if hashing becomes costly.
|
||||
const hash = createHash('sha256').update(`${floor}\n`);
|
||||
for (let i = floor; i < offset; i++) {
|
||||
hash.update(JSON.stringify(history[i])).update('\n');
|
||||
}
|
||||
return hash.digest('hex');
|
||||
}
|
||||
|
||||
async function readExtractCursor(
|
||||
projectRoot: string,
|
||||
): Promise<AutoMemoryExtractCursor> {
|
||||
@@ -177,15 +87,11 @@ async function readExtractCursor(
|
||||
async function writeExtractCursor(
|
||||
projectRoot: string,
|
||||
cursor: AutoMemoryExtractCursor,
|
||||
abortSignal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
await atomicWriteFile(
|
||||
getAutoMemoryExtractCursorPath(projectRoot),
|
||||
`${JSON.stringify(cursor, null, 2)}\n`,
|
||||
{
|
||||
encoding: 'utf-8',
|
||||
assertCanCommit: () => abortSignal?.throwIfAborted(),
|
||||
},
|
||||
{ encoding: 'utf-8' },
|
||||
);
|
||||
}
|
||||
|
||||
@@ -223,11 +129,7 @@ export async function runAutoMemoryExtract(params: {
|
||||
history: Content[];
|
||||
now?: Date;
|
||||
config?: Config;
|
||||
abortSignal?: AbortSignal;
|
||||
preserveUnprocessedHistory?: boolean;
|
||||
canExtractWindow?: (windowIdentity: string) => boolean;
|
||||
}): Promise<AutoMemoryExtractResult> {
|
||||
params.abortSignal?.throwIfAborted();
|
||||
const now = params.now ?? new Date();
|
||||
if (!params.config) {
|
||||
throw new Error(
|
||||
@@ -266,104 +168,25 @@ export async function runAutoMemoryExtract(params: {
|
||||
: 0;
|
||||
// History may shrink between extract calls (compression). Clamp to length
|
||||
// so new messages after compression are not permanently skipped.
|
||||
let startOffset = rawOffset > params.history.length ? 0 : rawOffset;
|
||||
let historyFloor = 0;
|
||||
if (params.preserveUnprocessedHistory) {
|
||||
const prefixLength = getStartupContextLength(params.history, {
|
||||
includeCompressed: true,
|
||||
});
|
||||
// Compression can preserve an open call as the last prefix entry. Keep
|
||||
// it with its real response, while skipping synthetic user context.
|
||||
historyFloor =
|
||||
prefixLength - (endsOnOpenCall(params.history, prefixLength) ? 1 : 0);
|
||||
const sameSession = currentCursor.sessionId === params.sessionId;
|
||||
const matchesHistory =
|
||||
sameSession &&
|
||||
Number.isInteger(rawOffset) &&
|
||||
rawOffset >= historyFloor &&
|
||||
rawOffset <= params.history.length &&
|
||||
currentCursor.processedHistoryHash ===
|
||||
hashProcessedHistory(params.history, historyFloor, rawOffset);
|
||||
if (!matchesHistory) {
|
||||
const compressed = prefixLength > getStartupContextLength(params.history);
|
||||
startOffset =
|
||||
(sameSession && currentCursor.processedHistoryHash) || compressed
|
||||
? historyFloor
|
||||
: Math.max(
|
||||
historyFloor,
|
||||
params.history.findLastIndex(
|
||||
(content, i) =>
|
||||
i >= historyFloor &&
|
||||
hasUserText(content) &&
|
||||
isApiUserPrompt(content, { excludeTaskNotifications: true }),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
// With turn-skipping enabled, a large ending turn can evict skipped facts
|
||||
// from the usual tail. Process the oldest pending window instead, and never
|
||||
// mark its unseen user text as processed.
|
||||
let endOffset = params.preserveUnprocessedHistory
|
||||
? Math.min(
|
||||
params.history.length,
|
||||
startOffset + CACHE_SAFE_HISTORY_TAIL_ENTRIES,
|
||||
)
|
||||
: params.history.length;
|
||||
// A plain index can split a model functionCall from its functionResponse:
|
||||
// the trailing repair would fabricate a response reusing the real call's
|
||||
// id, and the next window would open on the orphaned true output, which no
|
||||
// extraction run ever sees (the cursor only advances). Back the cut off to
|
||||
// a real turn boundary — but only while the window does not already reach
|
||||
// the end of history (a session ending on an open call must still be able
|
||||
// to arm the no-op cooldown at processedOffset === history.length), and
|
||||
// never past startOffset (a zero-length window still writes the cursor).
|
||||
if (params.preserveUnprocessedHistory) {
|
||||
while (
|
||||
endOffset < params.history.length &&
|
||||
endOffset > startOffset &&
|
||||
(endsOnOpenCall(params.history, endOffset) ||
|
||||
opensOnOrphanedResponse(params.history, endOffset))
|
||||
) {
|
||||
endOffset--;
|
||||
}
|
||||
}
|
||||
const pendingHistory = params.history.slice(startOffset, endOffset);
|
||||
const consumableEndOffset =
|
||||
params.preserveUnprocessedHistory &&
|
||||
!params.history.some((content, i) => i >= endOffset && hasUserText(content))
|
||||
? params.history.length
|
||||
: endOffset;
|
||||
// Attest the selection before the fork awaits: the parent may mutate its
|
||||
// history while extraction runs. Holdback must retain the start identity.
|
||||
const startHistoryHash = params.preserveUnprocessedHistory
|
||||
? hashProcessedHistory(params.history, historyFloor, startOffset)
|
||||
: undefined;
|
||||
const endHistoryHash = params.preserveUnprocessedHistory
|
||||
? hashProcessedHistory(params.history, historyFloor, endOffset)
|
||||
: undefined;
|
||||
const consumableHistoryHash =
|
||||
consumableEndOffset === endOffset
|
||||
? endHistoryHash
|
||||
: hashProcessedHistory(params.history, historyFloor, consumableEndOffset);
|
||||
const startOffset = rawOffset > params.history.length ? 0 : rawOffset;
|
||||
|
||||
// Skip if there are no new, non-empty user messages in the unprocessed
|
||||
// slice. Strip runtime reminders per part as the planner does, so a
|
||||
// reminder-only tool response cannot trigger extraction after its text is removed.
|
||||
const hasNewUserMessages = pendingHistory.some(hasUserText);
|
||||
// slice. partToString runs only on this small slice and without the
|
||||
// global whitespace regex — the .trim().length check preserves the old
|
||||
// behaviour of ignoring empty-text user turns.
|
||||
const hasNewUserMessages = params.history
|
||||
.slice(startOffset)
|
||||
.some(
|
||||
(m) => m.role === 'user' && partToString(m.parts ?? []).trim().length > 0,
|
||||
);
|
||||
if (!hasNewUserMessages) {
|
||||
const cursor: AutoMemoryExtractCursor = {
|
||||
sessionId: params.sessionId,
|
||||
processedOffset: consumableEndOffset,
|
||||
...(consumableHistoryHash && {
|
||||
processedHistoryHash: consumableHistoryHash,
|
||||
}),
|
||||
processedOffset: params.history.length,
|
||||
updatedAt: now.toISOString(),
|
||||
};
|
||||
await writeExtractCursor(params.projectRoot, cursor, params.abortSignal);
|
||||
// No fork ever ran: name the skip so the task list and telemetry do not
|
||||
// report this turn as a completed no-op extraction (extractorRan stays
|
||||
// absent either way, which is what the cooldown gate keys on).
|
||||
return { touchedTopics: [], skippedReason: 'no_user_text', cursor };
|
||||
await writeExtractCursor(params.projectRoot, cursor);
|
||||
return { touchedTopics: [], cursor };
|
||||
}
|
||||
|
||||
const lateMismatch = getSessionMismatchResult(
|
||||
@@ -373,149 +196,67 @@ export async function runAutoMemoryExtract(params: {
|
||||
);
|
||||
if (lateMismatch) return lateMismatch;
|
||||
|
||||
if (
|
||||
startHistoryHash &&
|
||||
endHistoryHash &&
|
||||
params.canExtractWindow?.(
|
||||
JSON.stringify([startHistoryHash, endHistoryHash]),
|
||||
) === false
|
||||
) {
|
||||
return {
|
||||
touchedTopics: [],
|
||||
skippedReason: 'failure_limit',
|
||||
cursor: currentCursor,
|
||||
};
|
||||
}
|
||||
|
||||
let extractionFailure: AutoMemoryExtractionError | undefined;
|
||||
const agentResult = await runAutoMemoryExtractionByAgent(
|
||||
params.config,
|
||||
params.projectRoot,
|
||||
params.preserveUnprocessedHistory ? pendingHistory : undefined,
|
||||
// A pending window that does not reach the end of history is a
|
||||
// historical segment: the planner drops the recency wording and the
|
||||
// anchor-to-today date claim, which would otherwise convert the
|
||||
// segment's relative dates against the drain day. The previous cursor's
|
||||
// write time is the nearest proxy for the segment's own date.
|
||||
endOffset < params.history.length &&
|
||||
// The stamp proxies the window's own date with the previous cursor's
|
||||
// write time — only meaningful for the session that wrote it. A cursor
|
||||
// from another session (or an unattested one) must not relabel today's
|
||||
// opening turns as historical. A session-less cursor (the scaffold's
|
||||
// bootstrap write) is unknown, not a mismatch.
|
||||
(currentCursor.sessionId === undefined ||
|
||||
currentCursor.sessionId === params.sessionId)
|
||||
? {
|
||||
windowAsOf: currentCursor.updatedAt,
|
||||
...(params.abortSignal && { abortSignal: params.abortSignal }),
|
||||
}
|
||||
: params.abortSignal
|
||||
? { abortSignal: params.abortSignal }
|
||||
: undefined,
|
||||
).catch((error: unknown) => {
|
||||
if (!(error instanceof AutoMemoryExtractionError)) throw error;
|
||||
extractionFailure = error;
|
||||
return error.result;
|
||||
});
|
||||
);
|
||||
|
||||
try {
|
||||
if (
|
||||
agentResult.touchedTopics.length > 0 ||
|
||||
(extractionFailure &&
|
||||
(agentResult.touchedProjectScope || agentResult.touchedUserScope))
|
||||
) {
|
||||
if (!extractionFailure && agentResult.touchedTopics.length > 0) {
|
||||
await bumpMetadata(
|
||||
params.projectRoot,
|
||||
now,
|
||||
params.sessionId,
|
||||
agentResult.touchedTopics,
|
||||
);
|
||||
}
|
||||
// Asymmetric failure isolation:
|
||||
// * project-level rebuild MUST bubble its error up. The cursor advances
|
||||
// only after rebuilds complete; a project rebuild failure that gets
|
||||
// silently swallowed would leave the memory file written, the index
|
||||
// stale, AND the cursor advanced — the memory becomes un-recallable
|
||||
// until some later session happens to trigger another rebuild. The
|
||||
// pre-existing `Promise.all` contract (throw → cursor stays → retry
|
||||
// on next session) is the durability guarantee we must preserve.
|
||||
// * user-level rebuild is best-effort. A read-only `~/.qwen/memories/`
|
||||
// (EACCES) must not poison the project-level rebuild or block the
|
||||
// cursor. Catch + warn, same shape as the user-level scaffold above.
|
||||
const projectRebuild =
|
||||
agentResult.touchedProjectScope || !agentResult.touchedUserScope
|
||||
? // Either explicitly touched, or the defensive fallback when both
|
||||
// scope flags were unset (e.g. older planner) — both paths must
|
||||
// surface project-level rebuild failures.
|
||||
rebuildManagedAutoMemoryIndex(params.projectRoot)
|
||||
: Promise.resolve();
|
||||
const userRebuild = agentResult.touchedUserScope
|
||||
? rebuildUserAutoMemoryIndex().catch((error: unknown) => {
|
||||
debugLogger.warn(
|
||||
`Auto-memory user-level index rebuild failed (non-critical, project-level rebuild unaffected): ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
})
|
||||
: Promise.resolve();
|
||||
await Promise.all([projectRebuild, userRebuild]);
|
||||
await refreshMemoryInstruction(params.config, {
|
||||
logContext: 'managed auto-memory extraction',
|
||||
});
|
||||
}
|
||||
|
||||
if (extractionFailure) throw extractionFailure;
|
||||
|
||||
const madeGenuineProgress =
|
||||
agentResult.touchedTopics.length > 0 || agentResult.hasToolActivity;
|
||||
const advances = madeGenuineProgress || endOffset < params.history.length;
|
||||
|
||||
const cursor: AutoMemoryExtractCursor = {
|
||||
sessionId: params.sessionId,
|
||||
// A capped window must advance even without genuine progress, or the
|
||||
// same slice freezes next turn. At the live end, keep the #6311 hold-back:
|
||||
// new turns grow this slice, and a zero-tool completion must not consume
|
||||
// it or arm the no-op cooldown.
|
||||
// Consume a tail that the no-user gate would skip without another fork,
|
||||
// but don't let it turn a zero-tool completion into a live-end no-op.
|
||||
processedOffset: madeGenuineProgress
|
||||
? consumableEndOffset
|
||||
: advances
|
||||
? endOffset
|
||||
: startOffset,
|
||||
...(endHistoryHash && {
|
||||
processedHistoryHash: madeGenuineProgress
|
||||
? consumableHistoryHash
|
||||
: advances
|
||||
? endHistoryHash
|
||||
: startHistoryHash,
|
||||
}),
|
||||
updatedAt: now.toISOString(),
|
||||
};
|
||||
await writeExtractCursor(params.projectRoot, cursor, params.abortSignal);
|
||||
|
||||
debugLogger.debug(
|
||||
`Managed auto-memory extract completed with ${agentResult.touchedTopics.length} touched topic(s).`,
|
||||
if (agentResult.touchedTopics.length > 0) {
|
||||
await bumpMetadata(
|
||||
params.projectRoot,
|
||||
now,
|
||||
params.sessionId,
|
||||
agentResult.touchedTopics,
|
||||
);
|
||||
|
||||
return {
|
||||
touchedTopics: agentResult.touchedTopics,
|
||||
touchedUserScope: agentResult.touchedUserScope,
|
||||
cursor,
|
||||
systemMessage: agentResult.systemMessage,
|
||||
extractorRan: true,
|
||||
...(cursor.processedOffset !== undefined &&
|
||||
cursor.processedOffset > startOffset
|
||||
? { cursorAdvanced: true as const }
|
||||
: {}),
|
||||
};
|
||||
} catch (error) {
|
||||
if (extractionFailure && error === extractionFailure) throw error;
|
||||
const cause = extractionFailure
|
||||
? new AggregateError(
|
||||
[extractionFailure, error],
|
||||
`${extractionFailure.message}; memory recovery failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
)
|
||||
: error;
|
||||
throw new AutoMemoryExtractionError(cause, agentResult);
|
||||
// Asymmetric failure isolation:
|
||||
// * project-level rebuild MUST bubble its error up. The cursor advances
|
||||
// only after rebuilds complete; a project rebuild failure that gets
|
||||
// silently swallowed would leave the memory file written, the index
|
||||
// stale, AND the cursor advanced — the memory becomes un-recallable
|
||||
// until some later session happens to trigger another rebuild. The
|
||||
// pre-existing `Promise.all` contract (throw → cursor stays → retry
|
||||
// on next session) is the durability guarantee we must preserve.
|
||||
// * user-level rebuild is best-effort. A read-only `~/.qwen/memories/`
|
||||
// (EACCES) must not poison the project-level rebuild or block the
|
||||
// cursor. Catch + warn, same shape as the user-level scaffold above.
|
||||
const projectRebuild =
|
||||
agentResult.touchedProjectScope || !agentResult.touchedUserScope
|
||||
? // Either explicitly touched, or the defensive fallback when both
|
||||
// scope flags were unset (e.g. older planner) — both paths must
|
||||
// surface project-level rebuild failures.
|
||||
rebuildManagedAutoMemoryIndex(params.projectRoot)
|
||||
: Promise.resolve();
|
||||
const userRebuild = agentResult.touchedUserScope
|
||||
? rebuildUserAutoMemoryIndex().catch((error: unknown) => {
|
||||
debugLogger.warn(
|
||||
`Auto-memory user-level index rebuild failed (non-critical, project-level rebuild unaffected): ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
})
|
||||
: Promise.resolve();
|
||||
await Promise.all([projectRebuild, userRebuild]);
|
||||
await refreshMemoryInstruction(params.config, {
|
||||
logContext: 'managed auto-memory extraction',
|
||||
});
|
||||
}
|
||||
|
||||
const madeGenuineProgress =
|
||||
agentResult.touchedTopics.length > 0 || agentResult.hasToolActivity;
|
||||
|
||||
const cursor: AutoMemoryExtractCursor = {
|
||||
sessionId: params.sessionId,
|
||||
processedOffset: madeGenuineProgress ? params.history.length : startOffset,
|
||||
updatedAt: now.toISOString(),
|
||||
};
|
||||
await writeExtractCursor(params.projectRoot, cursor);
|
||||
|
||||
debugLogger.debug(
|
||||
`Managed auto-memory extract completed with ${agentResult.touchedTopics.length} touched topic(s).`,
|
||||
);
|
||||
|
||||
return {
|
||||
touchedTopics: agentResult.touchedTopics,
|
||||
touchedUserScope: agentResult.touchedUserScope,
|
||||
cursor,
|
||||
systemMessage: agentResult.systemMessage,
|
||||
};
|
||||
}
|
||||
@@ -87,13 +87,9 @@ describe('auto-memory extraction with agent planner', () => {
|
||||
});
|
||||
|
||||
expect(result.touchedTopics).toEqual(['user']);
|
||||
// The cached-tail path passes no window stamp: history stays undefined
|
||||
// and the planner keeps the current-tail prompt wording.
|
||||
expect(runAutoMemoryExtractionByAgent).toHaveBeenCalledWith(
|
||||
mockConfig,
|
||||
projectRoot,
|
||||
undefined,
|
||||
undefined,
|
||||
);
|
||||
|
||||
const docs = await scanAutoMemoryTopicDocuments(projectRoot);
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { Config } from '../config/config.js';
|
||||
import type { Content } from '@google/genai';
|
||||
import { runAutoMemoryExtractionByAgent } from './extractionAgentPlanner.js';
|
||||
import { scanAutoMemoryTopicDocuments } from './structured-scan.js';
|
||||
import {
|
||||
@@ -50,7 +49,6 @@ describe('runAutoMemoryExtractionByAgent', () => {
|
||||
const mockConfig = {
|
||||
getSessionId: vi.fn().mockReturnValue('session-1'),
|
||||
getModel: vi.fn().mockReturnValue('qwen3-coder-plus'),
|
||||
getEffectiveInputModalities: vi.fn().mockReturnValue({ image: false }),
|
||||
getApprovalMode: vi.fn(),
|
||||
getMemoryAgentTimeoutMinutes: vi.fn().mockReturnValue(undefined),
|
||||
getMemoryAgentMaxTurns: vi.fn().mockReturnValue(undefined),
|
||||
@@ -139,101 +137,6 @@ describe('runAutoMemoryExtractionByAgent', () => {
|
||||
expect(systemPrompt).toContain('at most 64 characters');
|
||||
});
|
||||
|
||||
it('passes the lifecycle cancellation signal to the fork', async () => {
|
||||
vi.mocked(runForkedAgent).mockResolvedValue({
|
||||
status: 'completed',
|
||||
finalText: '',
|
||||
filesTouched: [],
|
||||
});
|
||||
const abortSignal = new AbortController().signal;
|
||||
await runAutoMemoryExtractionByAgent(mockConfig, '/tmp', undefined, {
|
||||
abortSignal,
|
||||
});
|
||||
expect(forkedCall()?.abortSignal).toBe(abortSignal);
|
||||
});
|
||||
|
||||
it('uses the pending extraction window with the same media filtering as the cached tail', async () => {
|
||||
vi.mocked(runForkedAgent).mockResolvedValue({
|
||||
status: 'completed',
|
||||
finalText: '',
|
||||
filesTouched: [],
|
||||
filesWritten: [],
|
||||
});
|
||||
const pending: Content[] = [
|
||||
{
|
||||
role: 'user',
|
||||
parts: [
|
||||
{ text: 'Remember the skipped preference.' },
|
||||
{ inlineData: { mimeType: 'image/png', data: 'AAAA' } },
|
||||
],
|
||||
},
|
||||
{ role: 'model', parts: [{ text: 'Acknowledged.' }] },
|
||||
];
|
||||
|
||||
await runAutoMemoryExtractionByAgent(mockConfig, '/tmp', pending);
|
||||
|
||||
expect(vi.mocked(runForkedAgent).mock.calls[0]?.[0].extraHistory).toEqual([
|
||||
{
|
||||
role: 'user',
|
||||
parts: [
|
||||
pending[0]!.parts![0]!,
|
||||
{ text: expect.stringContaining('image/png') },
|
||||
],
|
||||
},
|
||||
pending[1],
|
||||
]);
|
||||
});
|
||||
|
||||
it('curates degraded placeholder turns out of the pending window', async () => {
|
||||
// The window comes from raw history; the cached tail was curated. Without
|
||||
// the same curation the extractor is told the assistant said
|
||||
// "(request timeout)" and may write a durable memory from it.
|
||||
vi.mocked(runForkedAgent).mockResolvedValue({
|
||||
status: 'completed',
|
||||
finalText: '',
|
||||
filesTouched: [],
|
||||
filesWritten: [],
|
||||
});
|
||||
const pending: Content[] = [
|
||||
{ role: 'user', parts: [{ text: 'Remember: I prefer pnpm.' }] },
|
||||
{ role: 'model', parts: [{ text: '(request timeout)' }] },
|
||||
{ role: 'model', parts: [{ text: 'Noted.' }] },
|
||||
];
|
||||
|
||||
await runAutoMemoryExtractionByAgent(mockConfig, '/tmp', pending);
|
||||
|
||||
const extraHistory =
|
||||
vi.mocked(runForkedAgent).mock.calls[0]?.[0].extraHistory;
|
||||
expect(JSON.stringify(extraHistory)).not.toContain('(request timeout)');
|
||||
expect(JSON.stringify(extraHistory)).toContain('Noted.');
|
||||
});
|
||||
|
||||
it('drops the recency and today-anchoring claims for a historical window', async () => {
|
||||
// While the cursor is behind, the window is a historical segment: the
|
||||
// prompt must not claim it is the recent tail nor anchor its relative
|
||||
// dates to today (nothing on this path carries the segment's own date).
|
||||
vi.mocked(runForkedAgent).mockResolvedValue({
|
||||
status: 'completed',
|
||||
finalText: '',
|
||||
filesTouched: [],
|
||||
filesWritten: [],
|
||||
});
|
||||
const pending: Content[] = [
|
||||
{ role: 'user', parts: [{ text: 'We hit this yesterday.' }] },
|
||||
{ role: 'model', parts: [{ text: 'Noted.' }] },
|
||||
];
|
||||
|
||||
await runAutoMemoryExtractionByAgent(mockConfig, '/tmp', pending, {
|
||||
windowAsOf: '2026-09-28T10:00:00.000Z',
|
||||
});
|
||||
|
||||
const prompt = vi.mocked(runForkedAgent).mock.calls[0]?.[0].taskPrompt;
|
||||
expect(prompt).not.toContain("Today's date is");
|
||||
expect(prompt).not.toContain('recent conversation history');
|
||||
expect(prompt).toContain('earlier part of the session');
|
||||
expect(prompt).toContain('2026-09-28T10:00:00.000Z');
|
||||
});
|
||||
|
||||
it('strips runtime reminders and hidden reasoning from inherited history', async () => {
|
||||
vi.mocked(getCacheSafeParams).mockReturnValue({
|
||||
generationConfig: {},
|
||||
@@ -714,35 +617,6 @@ describe('runAutoMemoryExtractionByAgent', () => {
|
||||
).rejects.toThrow('timeout');
|
||||
});
|
||||
|
||||
it('retains reported memory writes when the agent fails to complete', async () => {
|
||||
vi.mocked(runForkedAgent).mockResolvedValue({
|
||||
status: 'failed',
|
||||
terminateReason: 'MAX_TURNS',
|
||||
filesTouched: [
|
||||
'/tmp/auto-memory/project/build.md',
|
||||
'/tmp/user-memory/user/style.md',
|
||||
'/tmp/auto-memory/reference/read-only.md',
|
||||
],
|
||||
filesWritten: [
|
||||
'/tmp/auto-memory/project/build.md',
|
||||
'/tmp/user-memory/user/style.md',
|
||||
'/tmp/auto-memory-other/user/outside.md',
|
||||
],
|
||||
});
|
||||
|
||||
await expect(
|
||||
runAutoMemoryExtractionByAgent(mockConfig, '/tmp/project'),
|
||||
).rejects.toMatchObject({
|
||||
message: 'MAX_TURNS',
|
||||
result: {
|
||||
touchedTopics: ['project', 'user'],
|
||||
touchedProjectScope: true,
|
||||
touchedUserScope: true,
|
||||
hasToolActivity: true,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('ignores non-memory file paths in filesTouched', async () => {
|
||||
const result = await extract([
|
||||
'/tmp/auto-memory/project/arch.md',
|
||||
|
||||
@@ -7,8 +7,6 @@
|
||||
import type { Config } from '../config/config.js';
|
||||
import { createDebugLogger } from '../utils/debugLogger.js';
|
||||
import { runForkedAgent, getCacheSafeParams } from '../agents/forkedAgent.js';
|
||||
import { slimCompactionInput } from '../services/compactionInputSlimming.js';
|
||||
import { extractCuratedHistory } from '../core/llm-chat.js';
|
||||
import { buildFunctionResponseParts } from '../tools/agent/fork-subagent.js';
|
||||
import type { Content } from '@google/genai';
|
||||
import {
|
||||
@@ -44,7 +42,7 @@ const debugLogger = createDebugLogger('AUTO_MEMORY_EXTRACTION_AGENT');
|
||||
const EXTRACTION_AGENT_SYSTEM_PROMPT = [
|
||||
'You are now acting as the managed memory extraction subagent for an AI coding assistant.',
|
||||
'',
|
||||
'The conversation history is already in your context. Analyze only that conversation and use it to update persistent managed memory.',
|
||||
'The recent conversation history is already in your context. Analyze only that recent conversation and use it to update persistent managed memory.',
|
||||
'',
|
||||
'Rules:',
|
||||
'- Read existing memory files first to avoid creating duplicates.',
|
||||
@@ -83,33 +81,6 @@ export interface AutoMemoryExtractionExecutionResult {
|
||||
hasToolActivity: boolean;
|
||||
}
|
||||
|
||||
export class AutoMemoryExtractionError extends Error {
|
||||
constructor(
|
||||
cause: unknown,
|
||||
readonly result: AutoMemoryExtractionExecutionResult,
|
||||
) {
|
||||
super(cause instanceof Error ? cause.message : String(cause), { cause });
|
||||
this.name = 'AutoMemoryExtractionError';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The cached-tail arm's history source. The slot is process-global (another
|
||||
* session's params can sit in it — see forkedAgent.ts), so the lookup is
|
||||
* keyed by session and throws when absent: extraction must run after a
|
||||
* completed main turn. Only called when no complete window was handed in.
|
||||
*/
|
||||
function getCacheSafeHistoryOrThrow(sessionId: string): Content[] {
|
||||
const cacheSafe = getCacheSafeParams(sessionId);
|
||||
if (!cacheSafe) {
|
||||
throw new Error(
|
||||
'runAutoMemoryExtractionByAgent: no cache-safe params available; ' +
|
||||
'extraction must run after a completed main turn.',
|
||||
);
|
||||
}
|
||||
return cacheSafe.history;
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop runtime reminders and hidden reasoning while preserving tool traffic,
|
||||
* which tells the extractor when the turn only read existing memory.
|
||||
@@ -216,35 +187,25 @@ function buildTaskPrompt(
|
||||
userMemoryRoot: string,
|
||||
topicSummaries: string,
|
||||
keywordVocabularySnapshot: string,
|
||||
windowAsOf?: string,
|
||||
): string {
|
||||
return [
|
||||
'Managed memory has TWO directories. Choose which one to write each memory into using the per-type `<scope>` guidance in your system instructions:',
|
||||
`- USER memory (cross-project, durable knowledge about who the user is): \`${userMemoryRoot}\``,
|
||||
`- PROJECT memory (this project only): \`${projectMemoryRoot}\``,
|
||||
'',
|
||||
windowAsOf
|
||||
? 'Scan the conversation history segment in your context and update durable managed memory in whichever directory each memory belongs.'
|
||||
: 'Scan the recent conversation history in your context and update durable managed memory in whichever directory each memory belongs.',
|
||||
'Scan the recent conversation history in your context and update durable managed memory in whichever directory each memory belongs.',
|
||||
'',
|
||||
// Inherited history is scrubbed of `<system-reminder>` blocks above, and
|
||||
// those are the only place a date reaches the model (the startup prelude
|
||||
// and the per-turn refresh). Passing extraHistory also suppresses the
|
||||
// fork's own env bootstrap, so the date has to be stated here or the
|
||||
// "convert relative dates to absolute ones" instruction is unanswerable.
|
||||
// A historical segment (the pending window while the cursor is behind)
|
||||
// must NOT be anchored to today: nothing on this path carries the
|
||||
// window's own date, and `windowAsOf` is the previous cursor's write
|
||||
// time — only a proxy, so the instruction drops the conversion claim
|
||||
// rather than re-anchoring to it.
|
||||
windowAsOf
|
||||
? `This segment is from an earlier part of the session (last processed around ${windowAsOf}) — do not convert its relative dates against today; keep them relative or omit them. The standing type guidance that says to always convert relative dates to absolute ones does not apply to this segment: it has no reliable date to convert against.`
|
||||
: `Today's date is ${formatDateForContext()} — use it to turn any relative date in the history into an absolute one before saving.`,
|
||||
`Today's date is ${formatDateForContext()} — use it to turn any relative date in the history into an absolute one before saving.`,
|
||||
'',
|
||||
'Available tools in this run: `read_file`, `grep_search`, `glob`, and `write_file`/`edit` for paths inside EITHER managed memory directory above.',
|
||||
'- Do not use any other tools.',
|
||||
'- You have a limited turn budget. `edit` requires a prior `read_file` of the same file, so the efficient strategy is: first issue all reads in parallel for every file you might update; then issue all `write_file`/`edit` calls in parallel. Do not interleave reads and writes across multiple turns.',
|
||||
'- You MUST only use content from the conversation history in your context plus the current managed memory files.',
|
||||
'- You MUST only use content from the recent conversation history in your context plus the current managed memory files.',
|
||||
'- Do not inspect repository code, git history, or unrelated files.',
|
||||
`- Treat files under the top-level \`${AUTO_MEMORY_PINNED_DIRNAME}/\` directory in either managed memory root as protected read-only records. You may read them to avoid duplicates, but never modify, overwrite, rename, merge into, or delete them, and do not intentionally remove their valid entries from \`${AUTO_MEMORY_INDEX_FILENAME}\`.`,
|
||||
'- Prefer updating an existing writable memory file over creating a duplicate. Check both directories for an existing entry before creating a new one.',
|
||||
@@ -337,29 +298,15 @@ function touchedTopicsFromFilePaths(
|
||||
export async function runAutoMemoryExtractionByAgent(
|
||||
config: Config,
|
||||
projectRoot: string,
|
||||
history?: Content[],
|
||||
opts?: {
|
||||
abortSignal?: AbortSignal;
|
||||
/**
|
||||
* Set when `history` is the pending window and the cursor is behind: the
|
||||
* previous cursor's write time, the nearest proxy for the window's own
|
||||
* date. Drops the recency wording and the anchor-to-today date claim.
|
||||
*/
|
||||
windowAsOf?: string;
|
||||
},
|
||||
): Promise<AutoMemoryExtractionExecutionResult> {
|
||||
// The pending window comes from raw history: curate it the way the cached
|
||||
// tail is curated, or degraded placeholder turns (e.g. "(request
|
||||
// timeout)") reach the extractor as something the assistant said. Only the
|
||||
// cached-tail arm reads the process-global slot — a caller that handed a
|
||||
// complete window in must neither pay the clone nor be gated on the slot.
|
||||
const inputHistory = history
|
||||
? slimCompactionInput(
|
||||
extractCuratedHistory(history),
|
||||
config.getEffectiveInputModalities(),
|
||||
).slimmedHistory
|
||||
: getCacheSafeHistoryOrThrow(config.getSessionId());
|
||||
const extraHistory = buildAgentHistory(inputHistory);
|
||||
const cacheSafe = getCacheSafeParams(config.getSessionId());
|
||||
if (!cacheSafe) {
|
||||
throw new Error(
|
||||
'runAutoMemoryExtractionByAgent: no cache-safe params available; ' +
|
||||
'extraction must run after a completed main turn.',
|
||||
);
|
||||
}
|
||||
const extraHistory = buildAgentHistory(cacheSafe.history);
|
||||
|
||||
const { topicSummaries, keywordVocabularySnapshot } =
|
||||
await buildExistingMemoryContext(projectRoot);
|
||||
@@ -385,7 +332,6 @@ export async function runAutoMemoryExtractionByAgent(
|
||||
userMemoryRoot,
|
||||
topicSummaries,
|
||||
keywordVocabularySnapshot,
|
||||
opts?.windowAsOf,
|
||||
),
|
||||
systemPrompt: EXTRACTION_AGENT_SYSTEM_PROMPT,
|
||||
maxTurns: config.getMemoryAgentMaxTurns() ?? 5,
|
||||
@@ -398,13 +344,19 @@ export async function runAutoMemoryExtractionByAgent(
|
||||
ToolNames.EDIT,
|
||||
],
|
||||
extraHistory,
|
||||
abortSignal: opts?.abortSignal,
|
||||
});
|
||||
|
||||
if (result.status !== 'completed') {
|
||||
throw new Error(
|
||||
result.terminateReason ||
|
||||
'Extraction agent did not complete successfully',
|
||||
);
|
||||
}
|
||||
|
||||
const { topics, touchedProjectScope, touchedUserScope } =
|
||||
touchedTopicsFromFilePaths(result.filesWritten ?? [], projectRoot);
|
||||
|
||||
const executionResult: AutoMemoryExtractionExecutionResult = {
|
||||
return {
|
||||
touchedTopics: topics,
|
||||
touchedProjectScope,
|
||||
touchedUserScope,
|
||||
@@ -414,12 +366,4 @@ export async function runAutoMemoryExtractionByAgent(
|
||||
? `Managed auto-memory updated: ${topics.map((t) => `${t}.md`).join(', ')}`
|
||||
: undefined,
|
||||
};
|
||||
if (result.status !== 'completed') {
|
||||
throw new AutoMemoryExtractionError(
|
||||
result.terminateReason ||
|
||||
'Extraction agent did not complete successfully',
|
||||
executionResult,
|
||||
);
|
||||
}
|
||||
return executionResult;
|
||||
}
|
||||
@@ -8,16 +8,12 @@ import * as fs from 'node:fs/promises';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
EXTRACT_NOOP_COOLDOWN_TURNS_ENV,
|
||||
globalMemoryManager,
|
||||
MAX_COOLDOWN_PENDING_HISTORY_ENTRIES,
|
||||
MemoryManager,
|
||||
resolveExtractNoopCooldownTurns,
|
||||
type ExtractResult,
|
||||
type MemoryTaskRecord,
|
||||
type ScheduleSkillReviewParams,
|
||||
import type {
|
||||
ExtractResult,
|
||||
ScheduleSkillReviewParams,
|
||||
MemoryTaskRecord,
|
||||
} from './manager.js';
|
||||
import { globalMemoryManager, MemoryManager } from './manager.js';
|
||||
import { ensureAutoMemoryScaffold } from './store.js';
|
||||
import {
|
||||
getAutoMemoryMetadataPath,
|
||||
@@ -32,7 +28,6 @@ import type { Content } from '@google/genai';
|
||||
import type { Config } from '../config/config.js';
|
||||
import * as metadataMigration from './metadata-migration.js';
|
||||
import { ToolNames } from '../tools/tool-names.js';
|
||||
import { CACHE_SAFE_HISTORY_TAIL_ENTRIES } from '../agents/cache-safe-history.js';
|
||||
|
||||
// ─── Mocks ────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -68,7 +63,6 @@ vi.mock('./skillReviewAgentPlanner.js', async (importOriginal) => ({
|
||||
}));
|
||||
|
||||
import { runAutoMemoryExtract } from './extract.js';
|
||||
import { AutoMemoryExtractionError } from './extractionAgentPlanner.js';
|
||||
import { runManagedAutoMemoryDream } from './dream.js';
|
||||
import {
|
||||
recordUserAutoMemoryMutation,
|
||||
@@ -1576,89 +1570,6 @@ describe('MemoryManager', () => {
|
||||
extract.resolve(extractResult('sess'));
|
||||
expect(await mgr.drain()).toBe(true);
|
||||
});
|
||||
|
||||
it.each([undefined, 1_000])(
|
||||
'waits for trailing extraction with timeout %s',
|
||||
async (timeoutMs) => {
|
||||
const mgr = new MemoryManager();
|
||||
const first = deferred<ExtractResult>();
|
||||
const trailing = deferred<ExtractResult>();
|
||||
vi.mocked(runAutoMemoryExtract)
|
||||
.mockReturnValueOnce(first.promise)
|
||||
.mockReturnValueOnce(trailing.promise);
|
||||
const active = mgr.scheduleExtract(extractParams('/project', 'sess'));
|
||||
const queued = await mgr.scheduleExtract(
|
||||
extractParams('/project', 'sess', 'second'),
|
||||
);
|
||||
expect(queued.skippedReason).toBe('queued');
|
||||
const settled = vi.fn();
|
||||
const draining = mgr.drain({ timeoutMs }).then(settled);
|
||||
|
||||
try {
|
||||
first.resolve(extractResult('sess'));
|
||||
await active;
|
||||
await new Promise<void>((resolve) => setImmediate(resolve));
|
||||
|
||||
expect(mgr.listTasksByType('extract')).toContainEqual(
|
||||
expect.objectContaining({ status: 'running' }),
|
||||
);
|
||||
expect(settled).not.toHaveBeenCalled();
|
||||
|
||||
trailing.resolve(extractResult('sess'));
|
||||
await draining;
|
||||
expect(settled).toHaveBeenCalledWith(true);
|
||||
expect(mgr.listTasksByType('extract')).toHaveLength(2);
|
||||
expect(
|
||||
mgr
|
||||
.listTasksByType('extract')
|
||||
.every((t) => t.status === 'completed'),
|
||||
).toBe(true);
|
||||
} finally {
|
||||
first.resolve(extractResult('sess'));
|
||||
trailing.resolve(extractResult('sess'));
|
||||
await active;
|
||||
await draining;
|
||||
await mgr.drain();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('shares one timeout across active and trailing extraction', async () => {
|
||||
vi.useFakeTimers();
|
||||
const mgr = new MemoryManager();
|
||||
const first = deferred<ExtractResult>();
|
||||
const trailing = deferred<ExtractResult>();
|
||||
vi.mocked(runAutoMemoryExtract)
|
||||
.mockReturnValueOnce(first.promise)
|
||||
.mockReturnValueOnce(trailing.promise);
|
||||
const active = mgr.scheduleExtract(extractParams('/project', 'sess'));
|
||||
await mgr.scheduleExtract(extractParams('/project', 'sess', 'second'));
|
||||
const settled = vi.fn();
|
||||
const draining = mgr.drain({ timeoutMs: 20 }).then(settled);
|
||||
|
||||
try {
|
||||
await vi.advanceTimersByTimeAsync(10);
|
||||
first.resolve(extractResult('sess'));
|
||||
await active;
|
||||
await vi.advanceTimersByTimeAsync(10);
|
||||
|
||||
expect(settled).toHaveBeenCalledWith(false);
|
||||
expect(mgr.listTasksByType('extract')).toContainEqual(
|
||||
expect.objectContaining({ status: 'running' }),
|
||||
);
|
||||
|
||||
trailing.resolve(extractResult('sess'));
|
||||
await mgr.drain();
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
first.resolve(extractResult('sess'));
|
||||
trailing.resolve(extractResult('sess'));
|
||||
await active;
|
||||
await draining;
|
||||
await mgr.drain();
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('scheduleExtract()', () => {
|
||||
@@ -1690,588 +1601,6 @@ describe('MemoryManager', () => {
|
||||
}
|
||||
});
|
||||
|
||||
describe('consecutive window failures', () => {
|
||||
const failure = new Error('provider unavailable');
|
||||
let window: string;
|
||||
let attempts: number;
|
||||
let succeed: boolean;
|
||||
let holdThird: ReturnType<typeof deferred<void>> | undefined;
|
||||
const params = (projectRoot = tmp.projectRoot, sessionId = 'sess') =>
|
||||
extractParams(
|
||||
projectRoot,
|
||||
sessionId,
|
||||
'Remember: preserve tenant boundaries.',
|
||||
makeMockConfig({ getSessionId: () => sessionId }),
|
||||
);
|
||||
|
||||
beforeEach(() => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
window = 'selected-window';
|
||||
attempts = 0;
|
||||
succeed = false;
|
||||
holdThird = undefined;
|
||||
vi.mocked(runAutoMemoryExtract).mockImplementation(async (p) => {
|
||||
if (p.canExtractWindow?.(window) === false) {
|
||||
return {
|
||||
...extractResult(p.sessionId),
|
||||
skippedReason: 'failure_limit',
|
||||
};
|
||||
}
|
||||
attempts++;
|
||||
if (attempts === 3 && holdThird) await holdThird.promise;
|
||||
if (succeed) {
|
||||
return { ...extractResult(p.sessionId), extractorRan: true };
|
||||
}
|
||||
throw failure;
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
delete process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV];
|
||||
});
|
||||
|
||||
it('suspends a genuinely queued fourth attempt after the third failure', async () => {
|
||||
const mgr = new MemoryManager();
|
||||
for (let i = 0; i < 2; i++) {
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
}
|
||||
holdThird = deferred<void>();
|
||||
const third = mgr
|
||||
.scheduleExtract(params())
|
||||
.catch((error: unknown) => error);
|
||||
expect(attempts).toBe(3);
|
||||
expect((await mgr.scheduleExtract(params())).skippedReason).toBe(
|
||||
'queued',
|
||||
);
|
||||
holdThird.resolve();
|
||||
expect(await third).toBe(failure);
|
||||
expect(await mgr.drain()).toBe(true);
|
||||
expect(attempts).toBe(3);
|
||||
const tasks = mgr.listTasksByType('extract');
|
||||
expect(tasks).toHaveLength(4);
|
||||
expect(tasks.filter((t) => t.status === 'failed')).toHaveLength(3);
|
||||
expect(tasks).toContainEqual(
|
||||
expect.objectContaining({
|
||||
status: 'skipped',
|
||||
metadata: expect.objectContaining({
|
||||
skippedReason: 'failure_limit',
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(telemetryMocks.logMemoryExtract).toHaveBeenLastCalledWith(
|
||||
expect.anything(),
|
||||
expect.objectContaining({
|
||||
status: 'skipped',
|
||||
skipped_reason: 'failure_limit',
|
||||
}),
|
||||
);
|
||||
expect((await mgr.scheduleExtract(params())).skippedReason).toBe(
|
||||
'failure_limit',
|
||||
);
|
||||
expect(attempts).toBe(3);
|
||||
});
|
||||
|
||||
it('permits a changed selection and resets its streak after success', async () => {
|
||||
const mgr = new MemoryManager();
|
||||
for (let i = 0; i < 3; i++) {
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
}
|
||||
expect((await mgr.scheduleExtract(params())).skippedReason).toBe(
|
||||
'failure_limit',
|
||||
);
|
||||
window = 'new-selected-window';
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
succeed = true;
|
||||
await mgr.scheduleExtract(params());
|
||||
succeed = false;
|
||||
for (let i = 0; i < 3; i++) {
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
}
|
||||
expect((await mgr.scheduleExtract(params())).skippedReason).toBe(
|
||||
'failure_limit',
|
||||
);
|
||||
expect(attempts).toBe(8);
|
||||
});
|
||||
|
||||
it.each(['project', 'session'])(
|
||||
'keeps another %s success from clearing the bound',
|
||||
async (scope) => {
|
||||
const mgr = new MemoryManager();
|
||||
for (let i = 0; i < 3; i++) {
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
}
|
||||
succeed = true;
|
||||
await mgr.scheduleExtract(
|
||||
params(
|
||||
scope === 'project'
|
||||
? path.join(tmp.tempDir, 'other')
|
||||
: tmp.projectRoot,
|
||||
scope === 'session' ? 'other-session' : 'sess',
|
||||
),
|
||||
);
|
||||
succeed = false;
|
||||
expect((await mgr.scheduleExtract(params())).skippedReason).toBe(
|
||||
'failure_limit',
|
||||
);
|
||||
expect(attempts).toBe(4);
|
||||
},
|
||||
);
|
||||
|
||||
it('keeps default-off retries unchanged and clears the old experiment streak', async () => {
|
||||
const mgr = new MemoryManager();
|
||||
for (let i = 0; i < 3; i++) {
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
}
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '0';
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
expect(
|
||||
vi.mocked(runAutoMemoryExtract).mock.lastCall?.[0],
|
||||
).not.toHaveProperty('canExtractWindow');
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
await expect(mgr.scheduleExtract(params())).rejects.toBe(failure);
|
||||
expect(attempts).toBe(5);
|
||||
});
|
||||
});
|
||||
|
||||
describe('no-op cooldown (#13004)', () => {
|
||||
const history: Content[] = [{ role: 'user', parts: [{ text: 'hi' }] }];
|
||||
const completedNoop = (sessionId = 'sess-1') => ({
|
||||
touchedTopics: [],
|
||||
extractorRan: true as const,
|
||||
cursorAdvanced: true as const,
|
||||
cursor: {
|
||||
sessionId,
|
||||
processedOffset: history.length,
|
||||
updatedAt: new Date().toISOString(),
|
||||
},
|
||||
});
|
||||
const turn = (
|
||||
mgr: MemoryManager,
|
||||
extra: {
|
||||
isBelowCompactionWarn?: () => boolean;
|
||||
config?: Config;
|
||||
sessionId?: string;
|
||||
history?: Content[];
|
||||
} = {
|
||||
isBelowCompactionWarn: () => true,
|
||||
},
|
||||
) =>
|
||||
mgr.scheduleExtract({
|
||||
projectRoot: tmp.projectRoot,
|
||||
sessionId: 'sess-1',
|
||||
history,
|
||||
...extra,
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
delete process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV];
|
||||
});
|
||||
|
||||
const grownBy = (entries: number): Content[] => [
|
||||
...history,
|
||||
...Array.from(
|
||||
{ length: entries },
|
||||
(): Content => ({ role: 'model', parts: [{ text: 'step' }] }),
|
||||
),
|
||||
];
|
||||
|
||||
it('still skips while the turns since the no-op fit half the extractor window', async () => {
|
||||
expect(MAX_COOLDOWN_PENDING_HISTORY_ENTRIES * 2).toBeLessThanOrEqual(
|
||||
CACHE_SAFE_HISTORY_TAIL_ENTRIES,
|
||||
);
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await turn(mgr);
|
||||
const next = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
history: grownBy(MAX_COOLDOWN_PENDING_HISTORY_ENTRIES),
|
||||
});
|
||||
|
||||
expect(next.skippedReason).toBe('cooldown');
|
||||
expect(next.cursor.processedOffset).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(1);
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ preserveUnprocessedHistory: true }),
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[MAX_COOLDOWN_PENDING_HISTORY_ENTRIES, 'cooldown'],
|
||||
[MAX_COOLDOWN_PENDING_HISTORY_ENTRIES + 1, undefined],
|
||||
] as const)(
|
||||
'counts the unprocessed window remainder before skipping (%s entries)',
|
||||
async (pendingEntries, skippedReason) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
const windowedHistory = [
|
||||
...grownBy(CACHE_SAFE_HISTORY_TAIL_ENTRIES - history.length),
|
||||
...Array.from({ length: pendingEntries }, () =>
|
||||
userText('Remember the release window is Friday.'),
|
||||
),
|
||||
];
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue({
|
||||
...completedNoop(),
|
||||
cursor: {
|
||||
sessionId: 'sess-1',
|
||||
processedOffset: CACHE_SAFE_HISTORY_TAIL_ENTRIES,
|
||||
updatedAt: new Date().toISOString(),
|
||||
},
|
||||
});
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await turn(mgr, { history: windowedHistory });
|
||||
const next = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
history: windowedHistory,
|
||||
});
|
||||
|
||||
expect(next.skippedReason).toBe(skippedReason);
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(
|
||||
skippedReason ? 1 : 2,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('does not arm the cooldown on a completed no-op whose cursor was held', async () => {
|
||||
// The zero-tool-call guard holds the cursor at its opening offset;
|
||||
// that turn must not arm the cooldown even at the live end.
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
const { cursorAdvanced: _dropped, ...heldNoop } = completedNoop();
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(heldNoop);
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await turn(mgr);
|
||||
const next = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
history: grownBy(1),
|
||||
});
|
||||
|
||||
expect(next.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('flushes history skipped by cooldown through consecutive windows', async () => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValueOnce(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
await turn(mgr);
|
||||
const pending = [...history, userText('Remember Friday.')];
|
||||
expect(
|
||||
(
|
||||
await turn(mgr, {
|
||||
history: pending,
|
||||
isBelowCompactionWarn: () => true,
|
||||
})
|
||||
).skippedReason,
|
||||
).toBe('cooldown');
|
||||
const abortSignal = new AbortController().signal;
|
||||
for (const processedOffset of [40, 80, 85]) {
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValueOnce({
|
||||
...completedNoop(),
|
||||
cursor: { ...completedNoop().cursor, processedOffset },
|
||||
});
|
||||
}
|
||||
const completeHistory = [
|
||||
...pending,
|
||||
...Array.from({ length: 83 }, () =>
|
||||
userText('Another durable fact.'),
|
||||
),
|
||||
];
|
||||
|
||||
expect(
|
||||
await mgr.flushExtract({
|
||||
projectRoot: tmp.projectRoot,
|
||||
sessionId: 'sess-1',
|
||||
history: completeHistory,
|
||||
abortSignal,
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(4);
|
||||
expect(
|
||||
vi
|
||||
.mocked(runAutoMemoryExtract)
|
||||
.mock.calls.slice(1)
|
||||
.every(
|
||||
([params]) =>
|
||||
params.history === completeHistory &&
|
||||
params.abortSignal === abortSignal,
|
||||
),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it.each(['failure_limit', 'held'] as const)(
|
||||
'does not report an incomplete flush as complete (%s)',
|
||||
async (reason) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
const { cursorAdvanced: _unused, ...held } = completedNoop();
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue({
|
||||
...held,
|
||||
...(reason === 'failure_limit' ? { skippedReason: reason } : {}),
|
||||
});
|
||||
const mgr = new MemoryManager();
|
||||
expect(
|
||||
await mgr.flushExtract({
|
||||
projectRoot: tmp.projectRoot,
|
||||
sessionId: 'sess-1',
|
||||
history:
|
||||
reason === 'held' ? [...history, userText('pending')] : history,
|
||||
}),
|
||||
).toBe(false);
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(1);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[
|
||||
'grow past half the extractor window',
|
||||
() => grownBy(MAX_COOLDOWN_PENDING_HISTORY_ENTRIES + 1),
|
||||
],
|
||||
['shrink, as after a compaction', (): Content[] => []],
|
||||
] as const)(
|
||||
'runs once the turns since the no-op %s',
|
||||
async (_label, nextHistory) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await turn(mgr);
|
||||
const next = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
history: nextHistory(),
|
||||
});
|
||||
|
||||
expect(next.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
},
|
||||
);
|
||||
|
||||
it('keeps the once-per-turn cadence when the knob is unset', async () => {
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
for (let i = 0; i < 3; i++) await turn(mgr);
|
||||
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(3);
|
||||
// The knob-OFF half is the PR's core promise ("default sessions keep
|
||||
// the current history.length cursor and cached-tail input"): the flag
|
||||
// must be absent from the call, not merely falsy, so making the
|
||||
// windowed arm unconditional reds this assertion.
|
||||
for (const call of vi.mocked(runAutoMemoryExtract).mock.calls) {
|
||||
expect(call[0]).not.toHaveProperty('preserveUnprocessedHistory');
|
||||
}
|
||||
});
|
||||
|
||||
it('skips the configured turns after a completed no-op, then runs again', async () => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
const config = makeMockConfig();
|
||||
|
||||
await turn(mgr);
|
||||
const first = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
config,
|
||||
});
|
||||
const second = await turn(mgr);
|
||||
await turn(mgr);
|
||||
|
||||
expect(first.skippedReason).toBe('cooldown');
|
||||
expect(second.skippedReason).toBe('cooldown');
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
expect(telemetryMocks.logMemoryExtract).toHaveBeenCalledWith(
|
||||
config,
|
||||
expect.objectContaining({
|
||||
status: 'skipped',
|
||||
skipped_reason: 'cooldown',
|
||||
}),
|
||||
);
|
||||
expect(
|
||||
mgr
|
||||
.listTasksByType('extract', tmp.projectRoot)
|
||||
.filter((t) => t.metadata?.['skippedReason'] === 'cooldown'),
|
||||
).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('accumulates pending entries from the arm point across consecutive skips', async () => {
|
||||
// The arm-point length is written once and never refreshed on a skip:
|
||||
// two skips over a growing history must see 10 then 20 pending — a
|
||||
// one-line rebase to the current length would release the second skip.
|
||||
// The budget is three so the release turn still has cooldown left —
|
||||
// otherwise the release is exhausted-budget, not past-gate.
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '3';
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await turn(mgr);
|
||||
const half = MAX_COOLDOWN_PENDING_HISTORY_ENTRIES / 2;
|
||||
const first = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
history: grownBy(half),
|
||||
});
|
||||
const second = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
history: grownBy(half * 2),
|
||||
});
|
||||
// Past the arm point by more than the gate: runs again even with a
|
||||
// turn of cooldown remaining.
|
||||
const third = await turn(mgr, {
|
||||
isBelowCompactionWarn: () => true,
|
||||
history: grownBy(MAX_COOLDOWN_PENDING_HISTORY_ENTRIES + 1),
|
||||
});
|
||||
|
||||
expect(first.skippedReason).toBe('cooldown');
|
||||
expect(second.skippedReason).toBe('cooldown');
|
||||
expect(third.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
"does not inherit another session's cooldown (completion after switch: %s)",
|
||||
async (completeAfterSwitch) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
let complete!: (result: ReturnType<typeof completedNoop>) => void;
|
||||
vi.mocked(runAutoMemoryExtract)
|
||||
.mockImplementationOnce(
|
||||
() =>
|
||||
new Promise((resolve) => {
|
||||
complete = resolve;
|
||||
}),
|
||||
)
|
||||
.mockResolvedValue(completedNoop('sess-2'));
|
||||
const mgr = new MemoryManager();
|
||||
const config = makeMockConfig({
|
||||
getSessionId: vi.fn().mockReturnValue('sess-1'),
|
||||
});
|
||||
const first = turn(mgr, { config });
|
||||
|
||||
if (!completeAfterSwitch) {
|
||||
complete(completedNoop());
|
||||
await first;
|
||||
}
|
||||
vi.mocked(config.getSessionId).mockReturnValue('sess-2');
|
||||
if (completeAfterSwitch) {
|
||||
complete(completedNoop());
|
||||
await first;
|
||||
}
|
||||
const next = await turn(mgr, {
|
||||
config,
|
||||
sessionId: 'sess-2',
|
||||
isBelowCompactionWarn: () => true,
|
||||
});
|
||||
|
||||
expect(next.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
[
|
||||
'a write',
|
||||
() => ({ ...completedNoop(), touchedTopics: ['user' as const] }),
|
||||
],
|
||||
[
|
||||
'a cursor held back by the zero-tool-call guard',
|
||||
() => {
|
||||
// Held means "not advanced": the flag stays absent, and the
|
||||
// offset alone must not arm.
|
||||
const { cursorAdvanced: _dropped, ...held } = completedNoop();
|
||||
return {
|
||||
...held,
|
||||
cursor: { ...held.cursor, processedOffset: 0 },
|
||||
};
|
||||
},
|
||||
],
|
||||
[
|
||||
'an early return with no extractor run',
|
||||
() => ({ touchedTopics: [], cursor: completedNoop().cursor }),
|
||||
],
|
||||
] as const)('does not arm after %s', async (_label, result) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(result());
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await turn(mgr);
|
||||
const next = await turn(mgr);
|
||||
|
||||
expect(next.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('does not arm after a failed run, so the next turn retries', async () => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
vi.mocked(runAutoMemoryExtract)
|
||||
.mockRejectedValueOnce(new Error('MAX_TURNS'))
|
||||
.mockResolvedValueOnce(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await expect(turn(mgr)).rejects.toThrow('MAX_TURNS');
|
||||
const next = await turn(mgr);
|
||||
|
||||
expect(next.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('clears an armed cooldown when the queued extraction fails', async () => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
let complete!: (result: ReturnType<typeof completedNoop>) => void;
|
||||
vi.mocked(runAutoMemoryExtract)
|
||||
.mockImplementationOnce(
|
||||
() =>
|
||||
new Promise((resolve) => {
|
||||
complete = resolve;
|
||||
}),
|
||||
)
|
||||
.mockRejectedValueOnce(new Error('MAX_TURNS'))
|
||||
.mockResolvedValueOnce(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
const first = turn(mgr);
|
||||
expect((await turn(mgr)).skippedReason).toBe('queued');
|
||||
complete(completedNoop());
|
||||
await first;
|
||||
await mgr.drain();
|
||||
expect(
|
||||
mgr
|
||||
.listTasksByType('extract', tmp.projectRoot)
|
||||
.some((task) => task.status === 'failed'),
|
||||
).toBe(true);
|
||||
|
||||
const next = await turn(mgr);
|
||||
|
||||
expect(next.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['near compaction', { isBelowCompactionWarn: () => false }],
|
||||
['when the context position is unknown', {}],
|
||||
] as const)('runs %s even while cooling down', async (_label, extra) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(completedNoop());
|
||||
const mgr = new MemoryManager();
|
||||
|
||||
await turn(mgr);
|
||||
const next = await turn(mgr, extra);
|
||||
|
||||
expect(next.skippedReason).toBeUndefined();
|
||||
expect(runAutoMemoryExtract).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it('reads the knob as a small non-negative integer, else off', () => {
|
||||
const read = (value: string) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = value;
|
||||
return resolveExtractNoopCooldownTurns();
|
||||
};
|
||||
|
||||
expect(read('2')).toBe(2);
|
||||
expect(read('99')).toBe(5);
|
||||
expect(read('-1')).toBe(0);
|
||||
expect(read('1.5')).toBe(0);
|
||||
expect(read('two')).toBe(0);
|
||||
expect(read('')).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
it('runs extract and records a completed task', async () => {
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue(
|
||||
extractResult('sess-1', ['user']),
|
||||
@@ -2314,46 +1643,6 @@ describe('MemoryManager', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('records user mutation and actual topics while extraction remains failed', async () => {
|
||||
const failure = new AutoMemoryExtractionError('MAX_TURNS', {
|
||||
touchedTopics: ['user'],
|
||||
touchedProjectScope: false,
|
||||
touchedUserScope: true,
|
||||
hasToolActivity: true,
|
||||
});
|
||||
vi.mocked(runAutoMemoryExtract).mockRejectedValueOnce(failure);
|
||||
const mgr = new MemoryManager();
|
||||
const recordUserMutation = vi
|
||||
.spyOn(mgr, 'recordUserMutation')
|
||||
.mockResolvedValue();
|
||||
const config = makeMockConfig();
|
||||
|
||||
await expect(
|
||||
mgr.scheduleExtract(
|
||||
extractParams(tmp.projectRoot, 'sess-1', 'Remember this.', config),
|
||||
),
|
||||
).rejects.toBe(failure);
|
||||
|
||||
expect(recordUserMutation).toHaveBeenCalledWith(
|
||||
tmp.projectRoot,
|
||||
config,
|
||||
expect.any(Date),
|
||||
);
|
||||
expect(mgr.listTasksByType('extract', tmp.projectRoot)[0]).toMatchObject({
|
||||
status: 'failed',
|
||||
error: 'MAX_TURNS',
|
||||
metadata: { touchedTopics: ['user'] },
|
||||
});
|
||||
expect(telemetryMocks.logMemoryExtract).toHaveBeenCalledWith(
|
||||
config,
|
||||
expect.objectContaining({
|
||||
status: 'failed',
|
||||
patches_count: 1,
|
||||
touched_topics: 'user',
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('records a session mismatch as skipped', async () => {
|
||||
vi.mocked(runAutoMemoryExtract).mockResolvedValue({
|
||||
...extractResult('sess-1'),
|
||||
|
||||
@@ -65,8 +65,6 @@ import {
|
||||
} from './paths.js';
|
||||
import { ensureAutoMemoryScaffold } from './store.js';
|
||||
import { runAutoMemoryExtract } from './extract.js';
|
||||
import { AutoMemoryExtractionError } from './extractionAgentPlanner.js';
|
||||
import { CACHE_SAFE_HISTORY_TAIL_ENTRIES } from '../agents/cache-safe-history.js';
|
||||
import {
|
||||
runManagedAutoMemoryDream,
|
||||
type AutoMemoryDreamResult,
|
||||
@@ -170,58 +168,6 @@ export interface ScheduleExtractParams {
|
||||
history: Content[];
|
||||
now?: Date;
|
||||
config?: Config;
|
||||
abortSignal?: AbortSignal;
|
||||
/**
|
||||
* Whether the conversation is still below the compaction warn threshold.
|
||||
* Consulted only while a no-op cooldown is active: a skipped turn is safe
|
||||
* only if the next extraction still sees it, and compaction would replace
|
||||
* it with a summary first. Absent means "unknown", which never skips.
|
||||
*/
|
||||
isBelowCompactionWarn?: () => boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Internal experiment for #13004: after an extraction that ran, completed and
|
||||
* wrote nothing, skip this many following user turns. Enabled runs process the
|
||||
* oldest pending {@link CACHE_SAFE_HISTORY_TAIL_ENTRIES} entries and advance
|
||||
* the cursor only through that window, so a larger ending turn cannot mark
|
||||
* skipped facts unseen in the usual tail as processed. Default 0 keeps today's
|
||||
* once-per-turn cadence; not a user setting until a paired run shows memory
|
||||
* quality is unchanged.
|
||||
*
|
||||
* Coverage note: the skip half engages only where the caller supplies
|
||||
* `isBelowCompactionWarn` (the primary-chat path). The ACP / `qwen serve`
|
||||
* caller passes no such predicate, so with the knob set those sessions get
|
||||
* the windowed extraction without any cooldown skips — do not read an
|
||||
* ablation through that path.
|
||||
*/
|
||||
export const EXTRACT_NOOP_COOLDOWN_TURNS_ENV =
|
||||
'QWEN_CODE_MEMORY_EXTRACT_NOOP_COOLDOWN_TURNS';
|
||||
export const MAX_EXTRACT_NOOP_COOLDOWN_TURNS = 5;
|
||||
/**
|
||||
* Stop deferring extraction once pending history fills half a window. The
|
||||
* extractor's bounded pending slice, not this cadence bound, prevents loss.
|
||||
*/
|
||||
export const MAX_COOLDOWN_PENDING_HISTORY_ENTRIES = Math.floor(
|
||||
CACHE_SAFE_HISTORY_TAIL_ENTRIES / 2,
|
||||
);
|
||||
|
||||
/**
|
||||
* Reads {@link EXTRACT_NOOP_COOLDOWN_TURNS_ENV}: a non-negative integer,
|
||||
* clamped to {@link MAX_EXTRACT_NOOP_COOLDOWN_TURNS}. Anything else is ignored
|
||||
* with a debug warning and falls back to 0 (off).
|
||||
*/
|
||||
export function resolveExtractNoopCooldownTurns(): number {
|
||||
const raw = process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV]?.trim();
|
||||
if (!raw) return 0;
|
||||
const value = Number(raw);
|
||||
if (!Number.isInteger(value) || value < 0) {
|
||||
debugLogger.warn(
|
||||
`Ignoring ${EXTRACT_NOOP_COOLDOWN_TURNS_ENV}=${raw}: expected a non-negative integer.`,
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
return Math.min(value, MAX_EXTRACT_NOOP_COOLDOWN_TURNS);
|
||||
}
|
||||
|
||||
export interface ScheduleSkillReviewParams {
|
||||
@@ -709,17 +655,6 @@ export class MemoryManager {
|
||||
string,
|
||||
{ taskId: string; params: ScheduleExtractParams }
|
||||
>();
|
||||
// User turns still to skip after a completed no-op in this session (#13004).
|
||||
// Process-local on purpose: a restart resets it to "run", the safe side.
|
||||
private readonly extractCooldownRemaining = new Map<
|
||||
string,
|
||||
{ sessionId: string; remaining: number; processedOffset: number }
|
||||
>();
|
||||
// ponytail: process-local suspension; durable recovery needs a separate backlog budget.
|
||||
private readonly extractFailures = new Map<
|
||||
string,
|
||||
{ windowIdentity: string; count: number }
|
||||
>();
|
||||
|
||||
// ── Skill-review in-flight dedup ─────────────────────────────────────────────
|
||||
private readonly skillReviewInFlightByProject = new Map<string, string>();
|
||||
@@ -866,29 +801,18 @@ export class MemoryManager {
|
||||
|
||||
// ─── Drain ────────────────────────────────────────────────────────────────────
|
||||
|
||||
/** Wait for tracked tasks, including trailing work, within one timeout. */
|
||||
/** Wait for all in-flight tasks to settle, with optional timeout. */
|
||||
async drain(options: DrainOptions = {}): Promise<boolean> {
|
||||
if (this.inFlight.size === 0) return true;
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
const timeout =
|
||||
options.timeoutMs && options.timeoutMs > 0
|
||||
? new Promise<boolean>((resolve) => {
|
||||
timer = setTimeout(() => resolve(false), options.timeoutMs);
|
||||
})
|
||||
: undefined;
|
||||
try {
|
||||
while (this.inFlight.size > 0) {
|
||||
const batch = Promise.allSettled([...this.inFlight.values()]).then(
|
||||
() => true,
|
||||
);
|
||||
if (!(await (timeout ? Promise.race([batch, timeout]) : batch))) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
} finally {
|
||||
if (timer !== undefined) clearTimeout(timer);
|
||||
}
|
||||
const promises = [...this.inFlight.values()];
|
||||
if (promises.length === 0) return true;
|
||||
const waitAll = Promise.allSettled(promises).then(() => true);
|
||||
if (!options.timeoutMs || options.timeoutMs <= 0) return waitAll;
|
||||
return Promise.race<boolean>([
|
||||
waitAll,
|
||||
new Promise<boolean>((resolve) =>
|
||||
setTimeout(() => resolve(false), options.timeoutMs),
|
||||
),
|
||||
]);
|
||||
}
|
||||
|
||||
private track<T>(taskId: string, promise: Promise<T>): Promise<T> {
|
||||
@@ -1162,8 +1086,6 @@ export class MemoryManager {
|
||||
* Returns immediately with a skipped result if:
|
||||
* - The last history turn wrote to a memory file (memory_tool)
|
||||
* - Extraction is already running for this project (queues trailing request)
|
||||
* - A no-op cooldown is armed and too little history has grown since
|
||||
* (cooldown; #13004, opt-in)
|
||||
*
|
||||
* The trailing request starts automatically when the active extraction
|
||||
* completes.
|
||||
@@ -1210,28 +1132,6 @@ export class MemoryManager {
|
||||
} as never;
|
||||
}
|
||||
|
||||
const cooldown = this.extractCooldownRemaining.get(params.projectRoot);
|
||||
if (cooldown) {
|
||||
const pendingEntries = params.history.length - cooldown.processedOffset;
|
||||
if (
|
||||
cooldown.sessionId === params.sessionId &&
|
||||
cooldown.remaining > 0 &&
|
||||
pendingEntries >= 0 &&
|
||||
pendingEntries <= MAX_COOLDOWN_PENDING_HISTORY_ENTRIES &&
|
||||
params.isBelowCompactionWarn?.() === true
|
||||
) {
|
||||
cooldown.remaining--;
|
||||
return this.recordExtractCooldownSkip(
|
||||
params,
|
||||
cooldown.remaining,
|
||||
) as never;
|
||||
}
|
||||
// A different session cannot inherit a no-op from the previous history.
|
||||
// Near compaction, with an unknown position, after a history shrink, or
|
||||
// once pending history fills half the extractor's window, run normally.
|
||||
this.extractCooldownRemaining.delete(params.projectRoot);
|
||||
}
|
||||
|
||||
if (this.extractRunning.has(params.projectRoot)) {
|
||||
const currentTaskId = this.extractCurrentTaskId.get(params.projectRoot);
|
||||
if (!currentTaskId) {
|
||||
@@ -1303,38 +1203,6 @@ export class MemoryManager {
|
||||
return this.track(record.id, this.runExtract(record.id, params)) as never;
|
||||
}
|
||||
|
||||
async flushExtract(params: ScheduleExtractParams): Promise<boolean> {
|
||||
if (resolveExtractNoopCooldownTurns() === 0) return true;
|
||||
let previousOffset = -1;
|
||||
while (true) {
|
||||
params.abortSignal?.throwIfAborted();
|
||||
if (this.extractRunning.has(params.projectRoot)) return false;
|
||||
const record = makeTaskRecord(
|
||||
'extract',
|
||||
params.projectRoot,
|
||||
params.sessionId,
|
||||
);
|
||||
this.store(record);
|
||||
const result = await this.track(
|
||||
record.id,
|
||||
this.runExtract(record.id, params),
|
||||
);
|
||||
if (result.skippedReason && result.skippedReason !== 'no_user_text') {
|
||||
return false;
|
||||
}
|
||||
const offset = result.cursor.processedOffset;
|
||||
if (offset === params.history.length) return true;
|
||||
if (
|
||||
offset === undefined ||
|
||||
offset <= previousOffset ||
|
||||
(!result.cursorAdvanced && result.skippedReason !== 'no_user_text')
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
previousOffset = offset;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* True when the runtime is under hard or critical memory pressure, as
|
||||
* reported by the shared MemoryPressureMonitor (#5147). The monitor is
|
||||
@@ -1364,79 +1232,6 @@ export class MemoryManager {
|
||||
);
|
||||
}
|
||||
|
||||
private recordExtractCooldownSkip(
|
||||
params: ScheduleExtractParams,
|
||||
remaining: number,
|
||||
): Awaited<ReturnType<typeof runAutoMemoryExtract>> {
|
||||
const record = makeTaskRecord(
|
||||
'extract',
|
||||
params.projectRoot,
|
||||
params.sessionId,
|
||||
);
|
||||
this.storeWith(record, {
|
||||
status: 'skipped',
|
||||
progressText:
|
||||
'Skipped: the last extraction found nothing durable; this turn is left for the next one.',
|
||||
metadata: {
|
||||
skippedReason: 'cooldown',
|
||||
cooldownRemaining: remaining,
|
||||
historyLength: params.history.length,
|
||||
},
|
||||
});
|
||||
if (params.config) {
|
||||
logMemoryExtract(
|
||||
params.config,
|
||||
new MemoryExtractEvent({
|
||||
trigger: 'auto',
|
||||
status: 'skipped',
|
||||
skipped_reason: 'cooldown',
|
||||
patches_count: 0,
|
||||
touched_topics: [],
|
||||
duration_ms: 0,
|
||||
}),
|
||||
);
|
||||
}
|
||||
return {
|
||||
touchedTopics: [],
|
||||
skippedReason: 'cooldown',
|
||||
cursor: {
|
||||
sessionId: params.sessionId,
|
||||
updatedAt: (params.now ?? new Date()).toISOString(),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Arms the no-op cooldown only after an extraction that ran, completed,
|
||||
* wrote nothing and advanced its cursor. A write, an early return with no
|
||||
* extractor run, or a cursor held back by the zero-tool-call guard all
|
||||
* clear it, so the next turn runs as today. Skipped results leave it as is.
|
||||
* The windowed arm stops short of `history.length` by construction, so the
|
||||
* trigger is an advanced cursor, not reaching the live end — otherwise a
|
||||
* tool-heavy session (every turn outgrows the window cap) never arms the
|
||||
* cooldown at all (#13004's measured waste case).
|
||||
*/
|
||||
private updateExtractCooldown(
|
||||
params: ScheduleExtractParams,
|
||||
result: Awaited<ReturnType<typeof runAutoMemoryExtract>>,
|
||||
): void {
|
||||
if (result.skippedReason) return;
|
||||
const completedNoop =
|
||||
result.extractorRan === true &&
|
||||
result.touchedTopics.length === 0 &&
|
||||
result.cursorAdvanced === true;
|
||||
const turns = completedNoop ? resolveExtractNoopCooldownTurns() : 0;
|
||||
if (turns > 0) {
|
||||
this.extractCooldownRemaining.set(params.projectRoot, {
|
||||
sessionId: params.sessionId,
|
||||
remaining: turns,
|
||||
processedOffset: result.cursor.processedOffset ?? params.history.length,
|
||||
});
|
||||
} else {
|
||||
this.extractCooldownRemaining.delete(params.projectRoot);
|
||||
}
|
||||
}
|
||||
|
||||
private async runExtract(
|
||||
taskId: string,
|
||||
params: ScheduleExtractParams,
|
||||
@@ -1452,10 +1247,6 @@ export class MemoryManager {
|
||||
});
|
||||
|
||||
const t0 = Date.now();
|
||||
const preserveUnprocessedHistory = resolveExtractNoopCooldownTurns() > 0;
|
||||
const failureKey = JSON.stringify([params.projectRoot, params.sessionId]);
|
||||
let selectedWindow: string | undefined;
|
||||
if (!preserveUnprocessedHistory) this.extractFailures.delete(failureKey);
|
||||
try {
|
||||
// Memory-pressure gate. Checked inside try so the finally block
|
||||
// always runs — extractRunning/extractCurrentTaskId are cleaned up
|
||||
@@ -1490,25 +1281,7 @@ export class MemoryManager {
|
||||
};
|
||||
}
|
||||
|
||||
const result = await runAutoMemoryExtract(
|
||||
preserveUnprocessedHistory
|
||||
? {
|
||||
...params,
|
||||
preserveUnprocessedHistory: true,
|
||||
canExtractWindow: (windowIdentity) => {
|
||||
selectedWindow = windowIdentity;
|
||||
const failure = this.extractFailures.get(failureKey);
|
||||
return (
|
||||
!failure ||
|
||||
failure.windowIdentity !== windowIdentity ||
|
||||
failure.count < 3
|
||||
);
|
||||
},
|
||||
}
|
||||
: params,
|
||||
);
|
||||
if (!result.skippedReason) this.extractFailures.delete(failureKey);
|
||||
this.updateExtractCooldown(params, result);
|
||||
const result = await runAutoMemoryExtract(params);
|
||||
if (result.touchedUserScope && params.config) {
|
||||
await this.recordUserMutation(
|
||||
params.projectRoot,
|
||||
@@ -1549,32 +1322,10 @@ export class MemoryManager {
|
||||
}
|
||||
return result;
|
||||
} catch (error) {
|
||||
if (selectedWindow !== undefined) {
|
||||
const previous = this.extractFailures.get(failureKey);
|
||||
this.extractFailures.set(failureKey, {
|
||||
windowIdentity: selectedWindow,
|
||||
count:
|
||||
previous?.windowIdentity === selectedWindow
|
||||
? previous.count + 1
|
||||
: 1,
|
||||
});
|
||||
}
|
||||
this.extractCooldownRemaining.delete(params.projectRoot);
|
||||
const writeResult =
|
||||
error instanceof AutoMemoryExtractionError ? error.result : undefined;
|
||||
if (writeResult?.touchedUserScope && params.config) {
|
||||
await this.recordUserMutation(
|
||||
params.projectRoot,
|
||||
params.config,
|
||||
params.now ?? new Date(),
|
||||
);
|
||||
}
|
||||
const touchedTopics = writeResult?.touchedTopics ?? [];
|
||||
const durationMs = Date.now() - t0;
|
||||
this.update(record, {
|
||||
status: 'failed',
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
metadata: { touchedTopics },
|
||||
});
|
||||
if (params.config) {
|
||||
logMemoryExtract(
|
||||
@@ -1582,8 +1333,8 @@ export class MemoryManager {
|
||||
new MemoryExtractEvent({
|
||||
trigger: 'auto',
|
||||
status: 'failed',
|
||||
patches_count: touchedTopics.length,
|
||||
touched_topics: touchedTopics,
|
||||
patches_count: 0,
|
||||
touched_topics: [],
|
||||
duration_ms: durationMs,
|
||||
}),
|
||||
);
|
||||
@@ -1592,26 +1343,19 @@ export class MemoryManager {
|
||||
} finally {
|
||||
this.extractCurrentTaskId.delete(params.projectRoot);
|
||||
this.extractRunning.delete(params.projectRoot);
|
||||
this.startQueuedExtract(params.projectRoot);
|
||||
void this.startQueuedExtract(params.projectRoot);
|
||||
}
|
||||
}
|
||||
|
||||
private startQueuedExtract(projectRoot: string): void {
|
||||
private async startQueuedExtract(projectRoot: string): Promise<void> {
|
||||
if (this.extractRunning.has(projectRoot)) return;
|
||||
const queued = this.extractQueued.get(projectRoot);
|
||||
if (!queued) return;
|
||||
this.extractQueued.delete(projectRoot);
|
||||
// runExtract rethrows; `track`'s removal handler swallows the rejection
|
||||
// without surfacing it, so observe it here — the trailing path has no
|
||||
// caller left to catch.
|
||||
void this.track(
|
||||
await this.track(
|
||||
queued.taskId,
|
||||
this.runExtract(queued.taskId, queued.params),
|
||||
).catch((error: unknown) => {
|
||||
debugLogger.warn(
|
||||
`Queued managed auto-memory extraction failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
});
|
||||
);
|
||||
}
|
||||
|
||||
// ─── Skill review ─────────────────────────────────────────────────────────────
|
||||
@@ -2806,8 +2550,6 @@ export class MemoryManager {
|
||||
this.extractRunning.clear();
|
||||
this.extractCurrentTaskId.clear();
|
||||
this.extractQueued.clear();
|
||||
this.extractFailures.clear();
|
||||
this.extractCooldownRemaining.clear();
|
||||
}
|
||||
|
||||
/** Reset all dream scheduling state. */
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
/**
|
||||
* Internal experiment for #13003: skip the model selector when the published
|
||||
* fast result is exactly one strong, current match. Structured recall only.
|
||||
* fast result is one title/keyword match with no body in history. Structured only.
|
||||
* Off unless set to `1` or `true`; not a user setting until an ablation shows
|
||||
* recall quality is unchanged.
|
||||
*
|
||||
|
||||
@@ -963,35 +963,39 @@ describe('auto-memory relevant recall', () => {
|
||||
delete process.env[RECALL_SKIP_SELECTOR_ON_UNIQUE_STRONG_HIT_ENV];
|
||||
});
|
||||
|
||||
it('returns the one strong, current fast hit without the selector', async () => {
|
||||
// docs[1] is a weaker candidate the selector could have added; the skip
|
||||
// narrows proactive injection to the unique strong hit on purpose.
|
||||
mockSnapshot([exact, docs[1]!]);
|
||||
const onFastResult = vi.fn();
|
||||
it.each(['1', ' TRUE '])(
|
||||
'skips the selector with flag %s',
|
||||
async (flag) => {
|
||||
process.env[RECALL_SKIP_SELECTOR_ON_UNIQUE_STRONG_HIT_ENV] = flag;
|
||||
// docs[1] is a weaker candidate the selector could have added; the skip
|
||||
// narrows proactive injection to the unique strong hit on purpose.
|
||||
mockSnapshot([exact, docs[1]!]);
|
||||
const onFastResult = vi.fn();
|
||||
|
||||
const result = await resolveRelevantAutoMemoryPromptForQuery(
|
||||
'/tmp/project',
|
||||
query,
|
||||
{ config, onFastResult },
|
||||
);
|
||||
const result = await resolveRelevantAutoMemoryPromptForQuery(
|
||||
'/tmp/project',
|
||||
query,
|
||||
{ config, onFastResult },
|
||||
);
|
||||
|
||||
expect(selectRelevantAutoMemoryDocumentsByModel).not.toHaveBeenCalled();
|
||||
expect(onFastResult.mock.calls[0]?.[0].selectedDocs).toEqual([exact]);
|
||||
expect(result.selectedDocs).toEqual([exact]);
|
||||
expect(result.strategy).toBe('heuristic');
|
||||
expect(result.selectorSkipped).toBe(true);
|
||||
expect(result.treeSnapshot).toBe(
|
||||
onFastResult.mock.calls[0]?.[0].treeSnapshot,
|
||||
);
|
||||
expect(vi.mocked(logMemoryRecall)).toHaveBeenLastCalledWith(
|
||||
config,
|
||||
expect.objectContaining({
|
||||
strategy: 'heuristic',
|
||||
selector_skipped: true,
|
||||
selector_duration_ms: 0,
|
||||
}),
|
||||
);
|
||||
});
|
||||
expect(selectRelevantAutoMemoryDocumentsByModel).not.toHaveBeenCalled();
|
||||
expect(onFastResult.mock.calls[0]?.[0].selectedDocs).toEqual([exact]);
|
||||
expect(result.selectedDocs).toEqual([exact]);
|
||||
expect(result.strategy).toBe('heuristic');
|
||||
expect(result.selectorSkipped).toBe(true);
|
||||
expect(result.treeSnapshot).toBe(
|
||||
onFastResult.mock.calls[0]?.[0].treeSnapshot,
|
||||
);
|
||||
expect(vi.mocked(logMemoryRecall)).toHaveBeenLastCalledWith(
|
||||
config,
|
||||
expect.objectContaining({
|
||||
strategy: 'heuristic',
|
||||
selector_skipped: true,
|
||||
selector_duration_ms: 0,
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('keeps the selector when the knob is off', async () => {
|
||||
delete process.env[RECALL_SKIP_SELECTOR_ON_UNIQUE_STRONG_HIT_ENV];
|
||||
@@ -1049,6 +1053,44 @@ describe('auto-memory relevant recall', () => {
|
||||
expect(result.selectorSkipped).toBeUndefined();
|
||||
});
|
||||
|
||||
it('keeps the selector for a metadata-only substring match', async () => {
|
||||
const weak = {
|
||||
...exact,
|
||||
title: 'Runtime dashboard',
|
||||
keywords: [],
|
||||
description: 'provider fallback',
|
||||
usageScenarios: [],
|
||||
};
|
||||
mockSnapshot([weak]);
|
||||
const onFastResult = vi.fn();
|
||||
const result = await resolveRelevantAutoMemoryPromptForQuery(
|
||||
'/tmp/project',
|
||||
query,
|
||||
{ config, onFastResult },
|
||||
);
|
||||
expect(onFastResult.mock.calls[0]?.[0].selectedDocs).toEqual([weak]);
|
||||
expect(selectRelevantAutoMemoryDocumentsByModel).toHaveBeenCalledOnce();
|
||||
expect(result.selectorSkipped).toBeUndefined();
|
||||
});
|
||||
|
||||
it('keeps the selector when the matching body is already present', async () => {
|
||||
mockSnapshot([exact, docs[1]!]);
|
||||
bodyPresentVersions.set('project:reference.md', exact.mtimeMs);
|
||||
vi.mocked(selectRelevantAutoMemoryDocumentsByModel).mockResolvedValue([
|
||||
docs[1]!,
|
||||
]);
|
||||
const onFastResult = vi.fn();
|
||||
const result = await resolveRelevantAutoMemoryPromptForQuery(
|
||||
'/tmp/project',
|
||||
query,
|
||||
{ config, onFastResult },
|
||||
);
|
||||
expect(onFastResult.mock.calls[0]?.[0].selectedDocs).toEqual([exact]);
|
||||
expect(selectRelevantAutoMemoryDocumentsByModel).toHaveBeenCalledOnce();
|
||||
expect(result.selectedDocs).toContainEqual(docs[1]!);
|
||||
expect(result.selectorSkipped).toBeUndefined();
|
||||
});
|
||||
|
||||
it('keeps the selector when two strong matches compete', async () => {
|
||||
const second = { ...docs[1]!, keywords: ['provider fallback'] };
|
||||
mockSnapshot([exact, second]);
|
||||
|
||||
@@ -307,7 +307,7 @@ function scoreDocument(
|
||||
return lexicalScore + typeBoost;
|
||||
}
|
||||
|
||||
function isStrongFastMatch(
|
||||
function matchesTitleOrKeyword(
|
||||
query: string,
|
||||
doc: ScannedAutoMemoryDocument,
|
||||
): boolean {
|
||||
@@ -326,12 +326,17 @@ function isStrongFastMatch(
|
||||
'u',
|
||||
).test(normalizedQuery);
|
||||
};
|
||||
if (
|
||||
return (
|
||||
(title.length > 0 && normalizedQuery.includes(title)) ||
|
||||
keywords.some(includesKeyword)
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
function isStrongFastMatch(
|
||||
query: string,
|
||||
doc: ScannedAutoMemoryDocument,
|
||||
): boolean {
|
||||
if (matchesTitleOrKeyword(query, doc)) return true;
|
||||
|
||||
const queryTokens = tokenize(query);
|
||||
const metadata = normalizeRecallText(
|
||||
@@ -486,7 +491,7 @@ export interface RelevantAutoMemoryPromptResult {
|
||||
strategy: 'none' | 'heuristic' | 'model';
|
||||
/**
|
||||
* Set only when the model selector was skipped because the fast result was
|
||||
* a unique, strong, current match: this result IS the fast result, so the
|
||||
* a unique title/keyword match with no body in history: this IS the fast result, so the
|
||||
* consumer delivers it as the fast phase rather than as a refined one.
|
||||
*/
|
||||
selectorSkipped?: true;
|
||||
@@ -725,17 +730,8 @@ export async function resolveRelevantAutoMemoryPromptForQuery(
|
||||
}
|
||||
}
|
||||
fastDurationMs = Date.now() - fastStartedAt;
|
||||
// #13003: when the delivered fast result is one strong, current match,
|
||||
// its answer is already in front of the model and the selector round
|
||||
// trip buys nothing. Test strength and freshness, not count: the fast
|
||||
// list is stale-body-first and a stale-body document is published
|
||||
// without passing isStrongFastMatch, so a single delivered document
|
||||
// may be a reread candidate that must keep the selector. One document
|
||||
// that is strong and not stale also means no stale candidate and no
|
||||
// second strong one. The count is taken before rendering, which may
|
||||
// trim trailing documents to fit its budget. Proactive injection
|
||||
// narrows from up to MAX_RELEVANT_DOCS to this one; the router, and
|
||||
// search_memory through it, are unchanged.
|
||||
// Count before rendering: a second candidate trimmed by the prompt
|
||||
// budget must not turn an ambiguous recall into a selector skip.
|
||||
const uniqueStrongHit = publishedFast?.selectedDocs[0];
|
||||
if (
|
||||
!legacy &&
|
||||
@@ -743,8 +739,8 @@ export async function resolveRelevantAutoMemoryPromptForQuery(
|
||||
fastCandidateCount === 1 &&
|
||||
publishedFast?.selectedDocs.length === 1 &&
|
||||
uniqueStrongHit !== undefined &&
|
||||
isStrongFastMatch(query, uniqueStrongHit) &&
|
||||
!hasStaleBodyInHistory(uniqueStrongHit, bodyPresentVersions) &&
|
||||
matchesTitleOrKeyword(query, uniqueStrongHit) &&
|
||||
!bodyPresentVersions?.has(toAutoMemoryRef(uniqueStrongHit)) &&
|
||||
!options.abortSignal?.aborted
|
||||
) {
|
||||
const result: RelevantAutoMemoryPromptResult = {
|
||||
|
||||
@@ -95,6 +95,5 @@ export interface UserAutoMemoryMetadata {
|
||||
export interface AutoMemoryExtractCursor {
|
||||
sessionId?: string;
|
||||
processedOffset?: number;
|
||||
processedHistoryHash?: string;
|
||||
updatedAt: string;
|
||||
}
|
||||
@@ -37,8 +37,6 @@ import * as postCompactModule from './postCompactAttachments.js';
|
||||
import * as slimmingModule from './compactionInputSlimming.js';
|
||||
import { logChatCompression } from '../telemetry/loggers.js';
|
||||
import { estimateContentTokens } from './tokenEstimation.js';
|
||||
import { EXTRACT_NOOP_COOLDOWN_TURNS_ENV } from '../memory/manager.js';
|
||||
import { subagentIdentityContext } from '../utils/subagentNameContext.js';
|
||||
import {
|
||||
content,
|
||||
fnCall,
|
||||
@@ -173,82 +171,6 @@ async function expectGate(
|
||||
}
|
||||
}
|
||||
|
||||
describe('memory flush before compaction', () => {
|
||||
afterEach(() => {
|
||||
delete process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV];
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it.each([false, true])(
|
||||
'flushes memory before hooks and summary (complete=%s)',
|
||||
async (complete) => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
const history = turns('Remember Friday.', 'Understood.');
|
||||
const chat = fakeChat(history);
|
||||
vi.mocked(chat.getHistoryShallow).mockImplementation((curated) =>
|
||||
curated ? msgs(2) : history,
|
||||
);
|
||||
const order: string[] = [];
|
||||
const flushExtract = vi.fn().mockImplementation(async () => {
|
||||
order.push('memory');
|
||||
return complete;
|
||||
});
|
||||
const preCompact = vi.fn().mockImplementation(async () => {
|
||||
order.push('hook');
|
||||
});
|
||||
const query = spySideQuery('summary', usage(10, 5));
|
||||
const config = fakeConfig({
|
||||
hookSystem: {
|
||||
firePreCompactEvent: preCompact,
|
||||
firePostCompactEvent: resolvedFn(),
|
||||
},
|
||||
extra: {
|
||||
getManagedAutoMemoryEnabled: () => true,
|
||||
getMemoryManager: () => ({ flushExtract }),
|
||||
getProjectRoot: () => '/tmp/project',
|
||||
getSessionId: () => 'session',
|
||||
},
|
||||
});
|
||||
const result = await compressWith(chat, config);
|
||||
expect(flushExtract).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
history,
|
||||
projectRoot: '/tmp/project',
|
||||
sessionId: 'session',
|
||||
}),
|
||||
);
|
||||
if (complete) {
|
||||
expect(order).toEqual(['memory', 'hook']);
|
||||
expect(query).toHaveBeenCalled();
|
||||
} else {
|
||||
expectNoNewHistory(
|
||||
result,
|
||||
CompressionStatus.COMPRESSION_FAILED_MEMORY_FLUSH,
|
||||
);
|
||||
expect(order).toEqual(['memory']);
|
||||
expect(query).not.toHaveBeenCalled();
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('does not flush a memory fork through its own manager', async () => {
|
||||
process.env[EXTRACT_NOOP_COOLDOWN_TURNS_ENV] = '2';
|
||||
const flushExtract = vi.fn().mockResolvedValue(true);
|
||||
const config = fakeConfig({
|
||||
extra: {
|
||||
getManagedAutoMemoryEnabled: () => true,
|
||||
getMemoryManager: () => ({ flushExtract }),
|
||||
},
|
||||
});
|
||||
spySideQuery('summary', usage(10, 5));
|
||||
await subagentIdentityContext.run(
|
||||
{ type: 'managed-auto-memory-extractor', id: 'fork' },
|
||||
() => compressWith(fakeChat(msgs(2)), config),
|
||||
);
|
||||
expect(flushExtract).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
const SNAPSHOT_SUMMARY: [string, object] = [
|
||||
'<state_snapshot>summary</state_snapshot>',
|
||||
usage(1000, 500),
|
||||
|
||||
@@ -24,9 +24,6 @@ import { logChatCompression } from '../telemetry/loggers.js';
|
||||
import { makeChatCompressionEvent } from '../telemetry/types.js';
|
||||
import { PreCompactTrigger, PostCompactTrigger } from '../hooks/types.js';
|
||||
import { createDebugLogger } from '../utils/debugLogger.js';
|
||||
import { combineAbortSignals } from '../utils/abortController.js';
|
||||
import { subagentIdentityContext } from '../utils/subagentNameContext.js';
|
||||
import { resolveExtractNoopCooldownTurns } from '../memory/manager.js';
|
||||
import {
|
||||
estimateContentChars,
|
||||
resolveCompactionTuning,
|
||||
@@ -556,43 +553,6 @@ export class ChatCompressionService {
|
||||
};
|
||||
}
|
||||
|
||||
if (
|
||||
resolveExtractNoopCooldownTurns() > 0 &&
|
||||
config.getManagedAutoMemoryEnabled() &&
|
||||
!subagentIdentityContext.getStore()
|
||||
) {
|
||||
// ponytail: one two-minute flush budget; scheduled replay if more is needed.
|
||||
const flush = combineAbortSignals([signal], { timeoutMs: 120_000 });
|
||||
let completed = false;
|
||||
try {
|
||||
completed = await config.getMemoryManager().flushExtract({
|
||||
projectRoot: config.getProjectRoot(),
|
||||
sessionId: config.getSessionId(),
|
||||
history: chat.getHistoryShallow(),
|
||||
config,
|
||||
abortSignal: flush.signal,
|
||||
});
|
||||
} catch (error) {
|
||||
if (signal?.aborted) throw error;
|
||||
config
|
||||
.getDebugLogger()
|
||||
.warn(`Memory flush before compaction failed: ${error}`);
|
||||
} finally {
|
||||
flush.cleanup();
|
||||
}
|
||||
if (!completed) {
|
||||
return {
|
||||
newHistory: null,
|
||||
info: {
|
||||
originalTokenCount,
|
||||
newTokenCount: originalTokenCount,
|
||||
compressionStatus:
|
||||
CompressionStatus.COMPRESSION_FAILED_MEMORY_FLUSH,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// Fire PreCompact hook before compression begins. Pass any user-supplied
|
||||
// `/compress` instructions so hook scripts can read / log / amend them
|
||||
// via `hookSpecificOutput.additionalContext`. The aggregator concatenates
|
||||
|
||||
@@ -369,22 +369,25 @@ describe('loggers', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('attaches selector_skipped to the recall metrics while the #13003 experiment is enabled', () => {
|
||||
process.env[SKIP_SELECTOR_EXPERIMENT_ENV] = '1';
|
||||
const config = makeFakeConfig({ sessionId: 'test-session-id' });
|
||||
it.each([true, false])(
|
||||
'preserves selector_skipped=%s while enabled',
|
||||
(skipped) => {
|
||||
process.env[SKIP_SELECTOR_EXPERIMENT_ENV] = '1';
|
||||
const config = makeFakeConfig({ sessionId: 'test-session-id' });
|
||||
|
||||
logMemoryRecall(config, makeRecallEvent(true));
|
||||
logMemoryRecall(config, makeRecallEvent(skipped));
|
||||
|
||||
expect(metrics.recordMemoryRecallMetrics).toHaveBeenCalledWith(
|
||||
config,
|
||||
42,
|
||||
{
|
||||
strategy: 'heuristic',
|
||||
docs_selected: 1,
|
||||
selector_skipped: true,
|
||||
},
|
||||
);
|
||||
});
|
||||
expect(metrics.recordMemoryRecallMetrics).toHaveBeenCalledWith(
|
||||
config,
|
||||
42,
|
||||
{
|
||||
strategy: 'heuristic',
|
||||
docs_selected: 1,
|
||||
selector_skipped: skipped,
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('drops the selector_skipped metric dimension when the recall had no skip decision', () => {
|
||||
// Legacy-mode recalls never reach the skip guard; stamping a constant
|
||||
|
||||
@@ -1657,10 +1657,7 @@ export class MemoryExtractEvent implements BaseTelemetryEvent {
|
||||
| 'queued'
|
||||
| 'memory_tool'
|
||||
| 'memory_pressure'
|
||||
| 'session_mismatch'
|
||||
| 'cooldown'
|
||||
| 'failure_limit'
|
||||
| 'no_user_text';
|
||||
| 'session_mismatch';
|
||||
patches_count: number;
|
||||
touched_topics: string;
|
||||
duration_ms: number;
|
||||
@@ -1673,10 +1670,7 @@ export class MemoryExtractEvent implements BaseTelemetryEvent {
|
||||
| 'queued'
|
||||
| 'memory_tool'
|
||||
| 'memory_pressure'
|
||||
| 'session_mismatch'
|
||||
| 'cooldown'
|
||||
| 'failure_limit'
|
||||
| 'no_user_text';
|
||||
| 'session_mismatch';
|
||||
patches_count: number;
|
||||
touched_topics: string[];
|
||||
duration_ms: number;
|
||||
@@ -1760,7 +1754,7 @@ export class MemoryRecallEvent implements BaseTelemetryEvent {
|
||||
selector_duration_ms: number;
|
||||
/**
|
||||
* True only when the model selector was skipped because the deterministic
|
||||
* fast result was a unique, strong, current match (#13003). Keeps a
|
||||
* fast result matched a title/keyword and its body was absent (#13003). Keeps a
|
||||
* deliberate skip apart from a selector failure, which also reports
|
||||
* `strategy: 'heuristic'`. Undefined when the recall mode had no skip
|
||||
* decision to make (legacy mode), so the metric dimension stays off a
|
||||
|
||||
@@ -38,9 +38,6 @@ export function buildBtwCacheSafeParams(
|
||||
const chat = llmClient.getChat();
|
||||
const generationConfig = chat.getGenerationConfig();
|
||||
if (!generationConfig) return null;
|
||||
// utils/ is a leaf layer (architecture/no-core-utils-upward-import), so
|
||||
// this mirrors CACHE_SAFE_HISTORY_TAIL_ENTRIES in agents/ rather than
|
||||
// importing it.
|
||||
const maxHistoryEntries = 40;
|
||||
const history = llmClient.getHistoryTail(maxHistoryEntries, true);
|
||||
return {
|
||||
|
||||
Reference in new issue
Block a user