mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-13 05:41:49 -10:00
5374641323
* feat(memory): opt-in extraction cooldown and selector skip experiments Moved out of #13020, where they had been bundled with unrelated deferred-tool work. Content is the #13020 head (ffd9df8928) for these files and hunks, applied onto current main; history and review threads for these changes live on #13020. - #13004: after a no-op auto-memory extraction, an opt-in cooldown (QWEN_CODE_MEMORY_EXTRACT_NOOP_COOLDOWN_TURNS, off by default) skips extraction for a bounded number of turns while the prompt stays below the compaction warn threshold. Skipped turns stay inside the extractor's window (CACHE_SAFE_HISTORY_TAIL_ENTRIES), and a run that makes no progress advances the window instead of freezing it. - #13003: an opt-in flag (recall-experiment.ts, off by default) skips the recall selector when there is a single strong hit; the result is delivered as the fast phase, and a low-cardinality selector_skipped attribute is added to the recall log and metrics. Refs #13004 Refs #13003 * fix(core): Preserve the zero-tool extraction cursor guard * fix(memory): Avoid extraction from runtime reminders * fix(memory): Preserve user text across reminder parts * fix(memory): Bind extraction cursor to history * test(memory): Require exact same-length replacement * fix(memory): Wait for queued tasks during drain * fix(memory): Consume ineligible history tails * docs(memory): Record tail context acceptance limit * fix(memory): Recover reported writes after extraction failure * docs(memory): Clarify best-effort refresh recovery * fix(memory): Suspend repeated failed extraction windows * fix(core): close the cooldown-arm and review-round findings on the memory PR - The no-op cooldown now arms on a completed no-op whose cursor ADVANCED (extract.ts reports `cursorAdvanced`), not on reaching the live end — the windowed arm stops short of history.length by construction, so a tool-heavy session never armed it and the experiment saved nothing exactly where #13004 measured the waste (R1-1). - A media-only user turn counts as user content for the extraction gate; an empty-text read advanced the cursor past a turn no extractor ever sees, and the cursor only moves forward (R1-11). - The queued trailing extraction's rejection is observed with a warn log instead of vanishing into track()'s in-flight removal (R2-9). - `selector_skipped` is stamped only on structured-mode recalls, where a skip decision exists; legacy recalls left the field at a constant false and mixed into the experiment's control series. The metric dimension attaches only when the knob is on AND the field is a boolean (R2-14). - The docs now say so: the structured-recall contract (both languages) marks selector steps conditional under the knob, and the telemetry inventory lists selector_skipped with its gating conditions (R2-11). - Pins: cooldown arm on a windowed no-op + held-cursor non-arm + arm-point accumulation across consecutive skips (manager.test.ts); media-only user turn + cursorAdvanced assertions (extract.test.ts); legacy-unset event + dimension drop (recall.test.ts, loggers.test.ts). Co-authored-by: Qwen-Coder <[email protected]> * fix(memory): Count pending backlog from the extraction cursor * fix(memory): Flush pending extraction before compaction * fix(memory): Narrow selector experiment and withdraw unsafe cooldown * fix(memory): widen the #13003 skip guard, narrow selector_skipped Two review findings on the opt-in skip-selector guard, each reproduced atb4e0bba5b8before fixing. The uniqueness test counted only the published fast list -- the top MAX_RELEVANT_DOCS of the lexical ranking -- while the selector it suppresses is handed modelCandidates, whose lexical half is the top MAX_MODEL_CANDIDATE_DOCS - RECENT_MODEL_CANDIDATE_RESERVE, roughly 36x wider. A second strong title/keyword match ranked 6th or lower was therefore invisible to the guard: an ambiguous recall skipped the selector and still reported selector_skipped: true, landing in the treatment arm as a "unique" strong hit. selectModelCandidateDocuments now also returns the strong-match count over the lexical array it already scored. The count is computed only while the guard is live, so knob-off recalls pay nothing on a path that carries a latency budget, and the flag is now read once per recall so the count and the guard cannot disagree. logRecallResult also stamped selector_skipped: false on the empty-query / empty-corpus / non-positive-limit short circuit, where the selector is never called. Those trivially fast recalls joined the ablation's control arm, which the treatment arm structurally cannot contain, and recordMemoryRecallMetrics puts only strategy and selector_skipped on the counter and histogram, so the samples could not be sliced back out. logRecallResult now takes the caller's decision state and omits the field when no skip decision was made; the heuristic fallback keys off selectorStartedAt so a selector failure still reports false as a genuine control sample. Both fixes are mutation-checked: reverting either one reds exactly the tests added for it (1 failure and 2 failures respectively). Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com> Patrol-Run: qwen-pr-closeout/jmurnh6s41i * style(memory): let Prettier reflow the suppressed-selector assertion Lint & Static failed at step "Run Prettier" on34894db1d6with "[warn] packages/core/src/memory/recall.test.ts / Code style issues found in 1 file". Prettier 3.6.1 reproduces it locally and --write produces this two-line reflow of the `mock.calls[0]?.[2] ?? []` member chain; the expression is unchanged, so this is formatting only. Verified: prettier --check now passes on this file and on all 13 files the branch changes. Test execution is left to CI (no node_modules in this worktree). Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmurrend91n * fix(memory): require a token boundary for a short title in the #13003 skip gate R3-1: the skip gate made matchesTitleOrKeyword the final authority on whether the model selector runs at all, but its title arm is an unbounded substring test while the keyword arm already guards short Latin strings with a token-boundary regex. A memory titled `ai` therefore matched the query `explain provider fallback.`, and as the unique strong hit it suppressed the selector: every other memory the selector would have added was lost, and the recall entered the ablation's treatment arm as a genuine "unique strong hit" when the match was a coincidence. The injected document itself is not what this fixes. isStrongFastMatch keeps the loose arm, so a coincidental short-title substring was and still is published as a fast candidate; the refined selector result is filtered against fastDeliveredRefs (core/client.ts:1656), so it can only add to the fast phase, never retract it. What the gate changed is that the match also cancelled the selector, and that is what is closed here. Reproduced at279c5440eb: the new test fails there with the selector never called, and passes after the fix (88/88). The guard is applied only at the gate, so lexical ranking and the default (knob-off) path are unchanged and the design doc's "default behavior is unchanged" claim still holds. The gate's title test reuses the keyword arm's existing boundary rule instead of adding a second matcher; both design docs now state it. lexicalStrongMatchCount deliberately keeps the loose arm and now says so, because a looser count under-fires the skip, which is the safe direction for an ablation. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmurtwo5e1r * fix(memory): make the #13003 selector-skip gate require a real unique hit The strict title arm delegated to `includesKeyword`, whose token-boundary rule is gated on `/^[a-z0-9]{1,2}$/`, so for any Latin title of three or more characters the strict and loose arms were byte-identical: a memory titled `log` cancelled the model selector for a query containing `catalog`. Apply the boundary rule to any single-word Latin title when `requireTitleBoundary` is set; multi-word and non-Latin titles keep the loose arm, and ranking callers are untouched. The uniqueness count also ran over `lexical`, while the selector the gate suppresses is handed `modelCandidates` — whose `recent` reserve is built from `eligible`. A doc whose only strong-match evidence is a keyword that is a proper substring of a query token scores 0, never enters `lexical`, and so was never counted, yet it sat in the suppressed selector's pool. Count the pool the selector actually receives, and rename the field to say what it counts (title/keyword matches, deliberately not `isStrongFastMatch` — that predicate's loose arm would empty the experiment's treatment arm). Both changes only ever raise the count, so the gate skips strictly less often. State the real gate condition in both design docs, and pin `selectorDecided` in both directions: each constant mutant of the heuristic-fallback argument now fails a named assertion. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmus4mgzr28 * fix(memory): require a token boundary for every non-CJK value in the #13003 skip gate The strict arm keyed its boundary rule on value shape: `^[a-z0-9]+$` for a title and `^[a-z0-9]{1,2}$` for a keyword. That left the coincidental inner substring the gate exists to reject reachable through every other shape: a keyword of three or more characters (`log` inside `catalog`, the collision the gate's own rationale names), a multi-word Latin title (`log conventions` inside `catalog conventions`), and a single-word non-ASCII title (`ток` inside `поток`). Each suppressed the selector, dropped the refined phase over up to 200 candidates, and put a sample that is not a unique strong hit into the ablation's treatment arm as `selector_skipped: true`. Express the strictness once, on the script split this module's own tokenizer already draws: a non-CJK value goes through `includesAtBoundary`, a CJK value keeps `includes`, because a boundary rule needs a `[^\p{L}\p{N}]` neighbour while Han, Kana, and Hangul are `\p{L}`. Both arms share it and it stays behind the gate flag, so ranking callers and knob-off recall are unchanged. Pinned by three tests that are red without the fix. The design-doc sentences scoped the rule to single-word Latin titles; both twins now describe both arms. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmusfc9vi2o * fix(core): preserve memory selector freshness and delivery phase * fix(memory): reject concurrent document changes during reread * fix(memory): tighten selector skip boundaries and log attributes * fix(memory): clarify unstable reads and cover selector edges --------- Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: yiliang114 <[email protected]> Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com>