mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
32d22f950cf6f987eb8f06ac2722a02ca79fed2a
44
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jinyeandQwen-Coder
|
32d22f950c |
feat(managed-agent): support durable structured user questions (#16433)
Connect Hosted question actions, structured answers, durable continuation, and the WebShell question form. Cover canonical response replay and pending-question recovery across Harness and Java context restarts. Co-authored-by: Qwen-Coder <[email protected]> |
||
|
callmeYe
|
17ad9cdb04 | feat(skills): add file-backed archive installation kernel (#16421) | ||
|
Shaojin Wenandverify-round
|
e2d3243863 |
fix(web-shell): express the PR watch events dependency in the channel editor (#16236)
* fix(web-shell): express the PR watch events dependency in the channel editor The GitHub adapter rejects a non-empty prWatchEvents unless deliverToBoundSessions is enabled, but the editor never expressed that dependency: operators could tick watch events, press Save, and receive a raw English daemon refusal that read as "the selection cannot be saved". Group the bound-delivery switch and the watch-event checkboxes into one framed panel, disable the checkboxes with an inline hint until the switch is on, and mirror the runtime rule as a client-side validation backstop for hand-written stored configs that carry events without delivery. * fix(web-shell): keep stored watch selections clearable while delivery is off Review follow-ups on the bound-session panel: - A stored selection stays uncheckable-while-gated no more: with delivery off, selected events (including unknown stored values) can be cleared directly, while new selections remain disabled, so turning delivery off never strands the operator between a dependency error and disabled boxes. - Turning the delivery switch on clears the stale requiresBoundDelivery error instead of leaving the instruction under the now-on switch. - The disabled hint moves into FieldShell's existing hint slot, restoring the "description is the first paragraph" invariant; the hint gets an id, aria-describedby on the checkbox group and a persistent polite live region so screen readers hear why the boxes are gated. - The render gate now carries the same github scope as the validation mirror, and checked-but-disabled rows keep the selected border. - Tests: shutdown path, stale-error retirement, aria association, and a stored delivery-on mount; visuals scenario seeds the bound-session fields and captures the gated and enabled states. * fix(web-shell): scope dependency-error retirement to the dependency message The switch-on clear deleted the whole prWatchEvents error slot, wiping the higher-precedence invalidOption message while an unknown stored value was still ticked; and unticking the last selected event — the other remedy — left the dependency alert over an empty, valid selection. Retire only the dependency message, from both remedies, and pin the live region's persistence (empty region mounted before the hint, same node across the transition) plus the hint-less rendering of the pre-existing hint slots. * fix(web-shell): retire watch-events errors by re-deriving validity Round-4 review follow-ups, all in this PR: - R3-1: replace the string-matched dependency clear with a per-field re-validation on edit, so unticking an unknown stored value retires the invalidOption instruction it satisfies (and a remaining valid event swaps the slot to the still-unsatisfied dependency message). - R3-3: submit now scrolls the first erroring field into view even when it is not an advanced field; a stored watch-events refusal below the fold is no longer invisible. - R3-2: pin the empty-panel guard with EN/zh absence assertions on a non-GitHub descriptor. - R3-4: table-driven parity test feeds the plugin's own descriptor and config through both the editor mirror and the daemon validator (test-only devDependency, never bundled); a lone change on either side now fails the table. Each fix carries a mutation witness; 286 channel tests green. * fix(web-shell): re-derive only shown errors, in both switch directions Round-5 review follow-ups, all in this PR: - R4-1: refreshErrors re-derives only the error keys already on screen, so a remedy retires exactly the instruction it satisfies without surfacing validation the operator never submitted (submit stays the only population site), and the delivery switch re-derives in both directions so a shown instruction is never silently dropped. New tests pin pre-submit silence and shown-message survival across switch toggles. - R4-2: narrow plugin.management once at module scope in the parity test so the optional contract member stays TS18048-clean if test files ever join the package typecheck, with a clear error instead of a load-time TypeError. - Deferred note from the same round: pin the data-disabled styling hook on gated rows (dimmed) versus clearable stored rows (selected styling). Each fix carries a mutation witness; 288 channel tests green. --------- Co-authored-by: verify-round <verify-round@local> |
||
|
|
cca888dfcc |
fix(core): bump MCP client and core to 2.4.0 to drop pre-bundled fast-uri 3.1.0 (#15925)
* fix(core): bump MCP client and core to 2.4.0 to drop pre-bundled fast-uri 3.1.0 The MCP client 2.0.0 pre-bundles ajv with an inlined fast-uri 3.1.0, which dependency audits cannot see and overrides cannot reach. 2.4.0 re-bundles fast-uri 3.1.8, clearing CVE-2026-6321, CVE-2026-6322, CVE-2026-18446, CVE-2026-75975 and CVE-2026-76172 from the published bundle. Both packages stay exact-pinned in lockstep because the client pins core to the same version. Fixes #15919 Co-authored-by: Qwen-Coder <[email protected]> * chore(vscode-ide-companion): regenerate NOTICES for MCP client/core 2.4.0 Co-authored-by: Qwen-Coder <[email protected]> * fix(core): refuse redirects before the SDK, not only inside the fetch 2.4.0 resolves a same-origin 30x in its own layer, against requestInit.redirect, before the compatibility fetch is called. The Agent Plugin transport only set headers, so a configured or bearer credential was replayed to another path on the same origin. Ask the same question in one place and carry the answer into requestInit on both transport paths, and pin it with a test that fails when the option is absent. --------- Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> Co-authored-by: yiliang114 <[email protected]> |
||
|
ChiGao
|
0ac1fcca91 |
feat(cli): align OpenTUI input, editing and status lines with Ink (#14964)
* fix(cli): restore bounded OpenTUI dialogs and add migration gates * fix(cli): preserve OpenTUI input and attachment ownership * feat(cli): align OpenTUI status lines with Ink * fix(cli): align OpenTUI editor and cancellation transactions * fix(tui): harden migration cleanup and evidence after review * test(tui): fail acceptance on unhandled errors * docs(cli): clarify consolidated OpenTUI delivery status * test(cli): isolate system settings in TUI parity fixtures * fix(cli): settle OpenTUI cancellation when replay resets a session * style(cli): format merged OpenTUI test callbacks * test(cli): preserve visible input across session rotation * test(cli): await cancellation before releasing held response * fix(cli): preserve completion and escape precedence and validate status colors * fix(cli): preserve attachment navigation with one completion * fix(cli): display session titles in the OpenTUI composer border * fix(cli): discard generated command work after cancellation |
||
|
顾盼
|
a791e47a54 |
feat(cua): streamline computer use and port native reliability fixes (#14846)
* feat(cua): preload the JavaScript computer use runtime * refactor(cua): remove the native Wayland backend * feat(cua): move app computer use into the driver * feat(cua): preserve background delivery and port platform fixes * feat(cua): add explicit desktop input and browser runtime facade * feat(cua): package the unified 0.26.0 computer use runtime * fix(cua): finalize cross-platform native reliability * fix(cua): isolate runtime releases from CLI versioning * fix(cua): keep Electron out of root workspace installs * fix(cua): synchronize consumers and async cleanup tests * fix(browser): migrate smoke to preloaded cua runtime * docs(cua): document MCP setup for Qwen Code Claude Code and Codex |
||
|
ChiGao
|
6e6598e35c |
fix(cli): restore bounded OpenTUI dialogs and add migration gates (#14933)
* fix(cli): restore bounded OpenTUI dialogs and add migration gates * fix(tui): harden migration cleanup and evidence after review * test(tui): fail acceptance on unhandled errors |
||
|
ytahdnand钉萁
|
b08e41b25e |
feat(web-shell): preview DOCX files across file entry points (#14868)
Co-authored-by: 钉萁 <[email protected]> |
||
|
|
3745654f8a |
fix(channels): preserve Markdown structure when splitting messages (#14043)
* fix(channels): preserve Markdown structure when splitting messages * fix(channels): preserve Markdown regression fixture formatting * fix(channels): preserve Markdown chunks at budget boundaries * fix(channels): preserve deep-list delivery and background code continuations * fix(channels): Preserve Markdown fence source at chunk boundaries * fix(channels): bound markdown message chunk amplification * fix(channels): Replan after nested Markdown budget failures * fix(channels): preserve affordable markdown within bounded plans --------- Co-authored-by: Ethan Ye <[email protected]> Co-authored-by: claudeethan <[email protected]> |
||
|
|
72bbbd9ec2 |
feat(channels): add WeCom application callback support (#13976)
* feat(channels): add WeCom application callback support * fix(channels): allow WeCom callback environment references on save * fix(channels): harden WeCom callback admission and retries --------- Co-authored-by: Ethan Ye <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> Co-authored-by: qwen-code-dev-bot <[email protected]> |
||
|
qqqysandqqqys
|
c4f0acdef8 |
feat(extensions): add bounded generated HTML and JavaScript views (#14338)
Co-authored-by: qqqys <[email protected]> |
||
|
|
a8eb74fdf5 |
fix(web-shell): polish files and trim preview resources (#14143)
* fix(web-shell): polish files and trim preview resources * test(cli): observe pending child Hook promises * fix(web-shell): address Files browser review findings * fix(vscode): sync Pierre modification notices * chore: ignore the repo-local .pnpm-store fallback directory pnpm falls back to a project-local store at .pnpm-store/ when the global store directory is not writable (e.g. sandboxed CI runners). It is a regenerable cache that node_modules hard-links into; its thousands of untracked files otherwise dirty the working tree and fail deterministic workspace-clean verification. Co-authored-by: Qwen-Coder <[email protected]> * fix(web-shell): address Pierre preview and file display review findings - getPreviewLanguage folds case on the extension only, so README.MD and config.YML highlight while whole-name detector keys (Gemfile, BUILD) keep their case-sensitive behavior - formatFileName strips the full Default_Ignorable_Code_Point class so zero-advance Hangul fillers and variation selectors cannot desync the tree search projection from the rendered name - pin the previously unwitnessed pierre-build guards: redirected Shiki subpaths count toward the 4 MiB budget, getModuleInfo may return null for redirected ids, emitted assets are skipped, and entry-module resolveId calls carry no importer - pin the theme name->module mapping with exact token colors and give the formatFileName display sites discriminating coverage with repeated-space names --------- Co-authored-by: 钉萁 <[email protected]> Co-authored-by: qwen-code[bot] <qwen-code[bot]@users.noreply.github.com> Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: qwen-code-dev-bot <[email protected]> |
||
|
ytahdnand钉萁
|
df5470e430 |
feat(web-shell): Add workspace file browser and previews (#13878)
* feat(web-shell): Add workspace file browser and previews * fix(web-shell): Restore source preview CI fixtures and notices * fix(web-shell): Address file pane review blockers * fix(web-shell): Resolve review regressions and strengthen file checks --------- Co-authored-by: 钉萁 <[email protected]> |
||
|
|
ef38ad7753 |
feat(web-shell): add read-only Excel artifact previews (#13664)
* feat(web-shell): add read-only Excel artifact previews * fix(vscode): refresh notices for Excel preview dependencies * fix(web-shell): bound Excel merge loading and preserve preview content * fix(web-shell): bound Excel preview work and preserve recovery * fix(web-shell): harden Excel preview values and recovery * fix(web-shell): include bundled ExcelJS dependency licenses * fix(web-shell): guard column expansion and complete ExcelJS notices * fix(web-shell): complete Excel color fallback and review guards * test(web-shell): follow artifact filename labels in Excel previews --------- Co-authored-by: 钉萁 <[email protected]> Co-authored-by: qwen-code-dev-bot <[email protected]> |
||
|
ytahdnand钉萁
|
f3f5e005a3 |
feat(web-shell): support copying selected reply content (#13833)
* feat(web-shell): support copying selected reply content * fix(web-shell): preserve selected reply data and menu lifecycle * fix(web-shell): keep selection copying faithful to visible content --------- Co-authored-by: 钉萁 <[email protected]> |
||
|
qqqysandqqqys
|
30582105fe |
feat(extensions): add interactive test failure explorer (PR2) (#13916)
* feat(extensions): add interactive test failure explorer app * fix(extensions): lint test explorer build scripts * fix(test-failure-explorer): keep expanded report controls reachable --------- Co-authored-by: qqqys <[email protected]> |
||
|
qqqysandqqqys
|
e6ed740eb1 |
feat(mods): add TypeScript authoring and configuration (PR2a) (#13890)
* feat(mods): add TypeScript authoring and configuration * fix(mods): preserve marketplace dependencies and update notices --------- Co-authored-by: qqqys <[email protected]> |
||
|
qqqysandqqqys
|
6775f24927 |
feat(extensions): add test failure report explorer (#13854)
* feat(extensions): add test failure report explorer * fix(extensions): bound report snapshots before serialization * fix(extensions): bound test report validation --------- Co-authored-by: qqqys <[email protected]> |
||
|
易良andQwen-Coder
|
c468f2ab66 |
fix(web-shell): allow deleting queued prompts during admission (#13594)
* fix(web-shell): allow deleting queued prompts during admission Co-authored-by: Qwen-Coder <[email protected]> * fix(web-shell): keep aborted text-only prompts on the admission path The cancelled-upload cleanup ran an unconditional throwIfAborted after attachment handling, so a text-only submission whose signal was already aborted rejected instead of reaching admission. Those submissions must still be admitted so the accepted prompt id can be removed exactly, which is the contract DaemonSessionProvider covers. Scope the early abort to submissions that actually uploaded attachments. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmuxzdx6dcb --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
|
933dc0a614 |
feat(managed-agent): admit glob in new hosted-workspace /2 profiles (#13166)
* feat(managed-agent): admit glob in new hosted-workspace /2 profiles
Hosted Sessions could not find a file the model was not told about: the
file profile has no search and the Shell profile only falls back to a
full durable dispatch per probe. Add the read-only glob tool behind
hosted-workspace-files/2 and hosted-workspace-shell/2, following the H1
precedent: the profile gates what the model is offered, the worker
admits GlobTool, and the frozen managed-runtime-tools/1 identity is
unchanged, so no Java or Broker change is needed.
The Harness validates the optional path with normalizeWorkspaceRelativePath
before acquisition; the worker pins the search to the Session's installed
context (never the mount-wide include list) and rewrites results to
Workspace-relative paths. An oversized result keeps the longest fitting
line prefix with a narrowing hint instead of the output-omitted path.
Glob is pre-approved under the default and auto-edit approval modes like
read_file.
Resolves #13030
* fix(managed-agent): contain glob pattern, links and error paths
Completes the containment boundary this PR creates by admitting glob into
the hosted-workspace /2 profiles:
- Reject a pattern that is absolute or carries a ".." segment. Glob
resolves ".." against the filesystem and treats an absolute pattern as
absolute, so the pattern was a second, uncontained search root.
Compared by segment so a literal "a/..b/*.ts" stays usable.
- Compare realpaths, not lexical paths, so a symlink inside the Session
context that points outside it is refused like the shell branch already
does via admitsDirectory.
- Rewrite glob's error channel too. A refused or failed glob reaches the
model and the durable record the same way its results do, so it leaks
the Runtime host path unless it is rewritten the same way.
- Dispatch the trimmed pattern that was validated; glob treats an
untrimmed pattern as a literal and the false negative is persisted.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmuq3r0k30o
* fix(managed-agent): anchor glob root relativization and clear prettier debt
relativizeGlobText stripped the Session root with an unanchored
split/join, so a nested directory whose path repeats the root lost its
separators and two real paths fused into one that does not exist
(/s/x/s/y.ts -> "xy.ts"). Strip the root only where it begins a path
token; every privacy-boundary case is byte-identical to before.
Also normalizes the two added hunks prettier --check rejects, which the
lint gate's freshness step was short-circuiting.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmuqegteq18
* fix(serve): contain hosted file reads and glob text rewriting (round 3)
- read_file/write_file/edit now realpath-contain file_path against the
Session directory: the glob admission makes an in-context symlink
enumerable, and without this a files-only Session could read a sibling
Session's content through it (R3-1). The ENOENT fallback keeps a missing
path a tool-level not-found, not a traversal accusation.
- relativizeGlobText stands down at the degenerate root '/' (the prefix is
every path's prefix; rewriting only ate separators, including the echoed
pattern's), and bareRoot gains the same leading boundary tokenPrefix has
so a hit whose tail repeats the root string is no longer truncated
mid-token (R3-2).
- Pin the anchoring invariants at the unit level — a tmpdir-rooted Session
cannot express the repeating-root geometry through the worker route
(R3-3).
Co-authored-by: Qwen-Coder <[email protected]>
* fix(managed-agent): preserve search profiles during continuation
* fix(managed-agent): judge the glob args that get dispatched, not the raw ones
Four round-4 review findings, each pinned by a test that goes red under its
own revert:
R2-3 — the pre-acquisition glob guard validated the raw `pattern` while the
value dispatched next to it was the trimmed one, so a whitespace-masked `..`
or absolute shape cleared the guard whose comment says it exists to refuse it,
and was refused only by the worker after a full durable Runtime round trip
with a message that does not name the argument. Trim first, then validate
exactly what is dispatched. Containment was never breached; the executor
applies the same rule to the dispatched value.
R4-11 — `path: null` is not `undefined`, so it fell into the non-string
refusal and was reported as an absolute-path/traversal attempt. Because
argument validation is batch-wide, that one encoding of an unset optional also
cancelled every valid sibling call in the turn. Treat `null` as the omitted
case alongside the blank string; other non-strings are still refused.
R4-9 — the `file_path` containment was gated only on "not shell", so the glob
this same diff admits inherited a rule for a parameter glob's declaration does
not have. A stray relative `file_path` on a glob call — schema confusion with
the file tools sharing the turn, or a hook that stamps the key on every call —
refused the whole search and accused the model of a traversal it never
attempted, uncorrectably, because glob names no such argument. Glob never
reads `file_path`; its own pattern and output containment is untouched.
R4-6 — the workspace activation envelope was built twice in one test file, so
a change to that request shape had to be made in two places and missing one
left half the suite posting a stale envelope silently. Keep the module-scope
pair as the single definition and alias the local names to it.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmurw1tuo1o
* fix(managed-agent): close the glob-profile containment review round
- R3-1: the file_path containment resolves the deepest existing ancestor,
so `write_file` through an in-context symlink to a sibling Session is
refused instead of creating the file there; a genuinely absent path
keeps the tool's own not-found answer.
- R4-2: the Session-directory boundary now names Sessions, not mount
contents — an escape is refused when the realpath lands in another
INSTALLED Session's directory, while a linked dependency inside the
same mount (node_modules/@acme/ui -> packages/ui) keeps the
pre-containment /1 read-through behavior.
- R1-1: glob containment runs over glob's full collected set
(ToolResult.collectedFilePaths), never the mtime display slice the
count is not computed from; and both gates admit the brace-EXPANDED
alternatives, so `{.,..}/**/*.md` or `{/etc,/zz}/host*` is refused
before the search runs.
- R4-3: a left-to-right saturating-product scan bounds the brace
expansion cost (4096 alternatives) at both gates before minimatch
materializes anything — a ~70-byte pattern can no longer block the
shared worker's event loop for seconds. minimatch is now a declared
cli dependency.
- R4-1: the W1 recovery verifier accepts the hosted /2 profiles (and
/2-shell captureBytes), so one /2 Session no longer aborts recovery of
the whole shared storage; the W1 recovery design doc names them in
both languages.
Co-authored-by: Qwen-Coder <[email protected]>
* fix(managed-agent): contain the glob walk, not just the pattern spelling
The /2 glob gates refused `..` by segment equality, so `[.][.]` and `\.\.`
spellings walked out of the Session and turned the output guard into a
host-filesystem existence oracle (existing outside file refused, missing
one answered "No files found"). The saturating-product budget also bounded
alternatives but not nesting depth or ranges, so a nested pattern overflowed
brace-expansion's recursion and `{1..100000}` stalled the shared worker.
- GlobTool gains an opt-in `containmentRoot`; the managed tool set passes the
Session directory and glob prunes every entry whose lexical path, or whose
parent's realpath, leaves it. Existing and missing outside paths now answer
identically. The ordinary CLI does not set the option.
- `checkHostedGlobPattern` replaces the budget scan at both gates: at most
1024 characters, nesting depth 8 and 64 alternatives, bounded from the
pattern's structure before anything expands it.
- Recovery reads profiles through `isHostedWorkspaceProfile`.
Ported from #13168 (
|
||
|
|
612a552959 |
feat(channels): add IMAP and SMTP email channel (#12939)
* feat(channels): add IMAP and SMTP email channel * test(channels): include email in managed channel catalog expectations --------- Co-authored-by: destire-mio <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> |
||
|
|
356262d8e9 |
feat(agents): add A2A access and sharing for workspace agents (#12851)
* feat(agents): expose the local runtime host * docs(agents): audit the original collaboration goal * docs(agents): define the registered host boundary * docs(agents): stage the remote host protocol * feat(agents): register remote hosts * fix(agents): repair two compile errors the new host work introduced Running the checked-in gates against the 70 commits that landed since |
||
|
qqqysandShaojin Wen
|
23e0a451e6 |
fix(workflow): reject unsupported scripts and oversized batches early (#13012)
* fix(workflow): reject unsupported scripts and oversized batches early A dynamic import() in a workflow script only failed when V8 reached it, after earlier agent() calls had already been dispatched and paid for. The shared compile step now parses the wrapped body and refuses any real import() expression before the body runs, so inline, saved, resumed and nested scripts are all refused without dispatching an agent or writing run state. parallel() thunks, pipeline() items and pipeline() stages are each limited to 4096 entries per call. A longer list rejects the whole call before any thunk or stage runs, as an ordinary rejection that can be caught; run-level failures keep their existing propagation. Each list is copied once after its length check so a Proxy or custom iterator cannot stretch the batch past it. The authoring reference documents both limits with a batching example. * fix(workflow): keep the import check fail-closed with its own cause Run the syntax-tree walk inside the same handler as the parse, and raise a failure of either as WorkflowUnsupportedSyntaxError, so a script the check cannot read is refused without the TypeScript/quoting hint meant for ordinary syntax errors. Pin import() refusal in class static blocks, class fields and parameter defaults, and regenerate the VS Code companion notices for the new acorn and acorn-walk dependencies. --------- Co-authored-by: Shaojin Wen <[email protected]> |
||
|
|
2cfd390650 |
feat(computer-use): let a remote session use your desktop through a node_repl relay (#11799)
* docs(plans): add remote computer use plan for a local Mac Plan for letting a Qwen Code session on a headless Linux dev box operate the user's local Mac through computer use: the principle, the existing pieces, three options, and a Mac companion that joins the reverse tool channel. * docs(plans): add handoff and slice-0 verification brief for remote computer use Add a handoff document for the session or agent that continues this work, and a real-Mac verification brief for slice 0. Update the plan with a newly found pitfall: on macOS the driver's LaunchServices auto-launch does not forward environment variables, so an auto-launched daemon never enables the HTTP MCP endpoint. * docs(plans): replace the driver relay with a node_repl relay for remote computer use Rereading main shows the bundled computer-use skill never talks to the qwen-cua-driver daemon: ComputerUse.create() embeds the driver inside the node_repl process, and a standalone daemon rejects trusted sessions from any connection that is not its embedding parent (serve.rs:787, error 77). That invalidates the v1 plan's option B and makes options A and C relay the wrong layer (the raw tool surface meant for external agents). The plan now relays node_repl itself over the reverse tool channel, keeps the skill's App API intact, and lists the one skill change this needs (read references with read_file instead of inside node_repl). The verification brief is rewritten around a ten-minute fact check on a real Mac plus the slice-1 prototype acceptance; the handoff document follows. * docs(plans): make the Mac relay a qwen subcommand, not a standalone app Maintainer decision: the Mac-side relay for remote computer use is a qwen subcommand (qwen mac-bridge) run from a terminal. That matches how local computer use already works, where qwen launches node_repl from the terminal and the TCC grants are recorded against the terminal, so it adds no install step and no new permission identity. The plan drops the signed-app, bundle-id and menu-bar requirements, makes Ctrl-C the local stop control, and demotes the signed host to an optional slice that is off by default. The handoff document records the decision so the next session does not reopen it, and the verification brief uses the subcommand throughout. * docs(plans): name the relay qwen bridge and drop the macOS-only framing The relay is for any machine with a graphical session, not for macOS: it only moves MCP frames, the SDK reports the connected platform, and the skill already carries per-platform references. macOS is the platform verified in this round because the TCC facts were read from its code path. The subcommand is qwen bridge, one new top-level command in packages/cli/src/commands/bridge.ts; the plan file is renamed from mac-companion to desktop-relay and the handoff and verification brief follow. * feat(computer-use): let a remote session use this desktop through a node_repl relay A Qwen Code session on a headless machine can now drive the desktop the user is sitting at. That computer lends its own node_repl (with @qwen-code/cua-sdk and the embedded driver) to one remote session: - `node-repl-mcp desktop-relay install` puts the runtime under ~/.qwen/desktop-relay and registers a launchd socket on 127.0.0.1:47821 in inetd mode, so nothing runs until something connects. - The Web Shell gains a "Use this computer" footer entry. It hands the daemon URL, session and token to the relay, which asks for approval in a native dialog, starts node_repl and registers it for that session over the daemon's reverse tool channel. A dropped connection ends the relay; control never resumes without a fresh approval. - Terminal sessions reach the same socket through an SSH RemoteForward; their first tools/call asks for approval. One node_repl serves the daemon's several MCP clients: the relay answers repeated initialize requests from the first result, rewrites request ids and turns replies above the 10 MB frame cap into errors. The skill needs no change; main already reads references with read_file when node_repl cannot. Not built, typechecked or tested locally; CI and the real-Mac brief in docs/verification/remote-computer-use/ are the verification. * chore(deps): record ws in the pnpm lockfile for node-repl-mcp The pnpm lockfile is generated from package-lock.json by `corepack pnpm import`; the previous commit updated only the npm one, so frozen installs failed on the node-repl importer's new ws and @types/ws specifiers. Both resolve to the versions the cli importer already records. * fix(node-repl): drop the console override in the desktop relay agent The repo lints packages with no-console, and the override guarded against output nothing on the agent path produces. * fix(node-repl): restore the desktop relay agent entry point The previous commit spliced runtime.ts at stale offsets and left it unparseable. This applies the intended change to the last good copy: no console override and no unused format import. * fix: harden remote computer-use relay * fix(node-repl): forward unambiguous cancellations through the desktop relay Stopping a turn in the remote session sends notifications/cancelled for the running tools/call. The relay dropped every cancellation because the reverse channel does not say which multiplexed MCP client sent it, so a node_repl cell kept driving the desktop after the user had stopped the agent. Forward a cancellation, with its request id rewritten, when exactly one pending request carries that id; still drop it when none or several do. * docs(computer-use): refresh the remote desktop relay handoff for another machine Switch the verification setup to the pnpm install main now uses, add a check that stopping a turn stops a running desktop cell, and add a troubleshooting guide that walks the chain from browser to launchd to daemon to macOS permissions. The handoff records the current state, the two decisions still open, and why SSH workspaces do not replace this. * docs(computer-use): record the Linux-side verification results for the desktop relay The Mac-side end-to-end run is still blocked: this machine has no GUI and no reachable macOS host. What could be verified here is recorded instead - the relay's real HTTP surface over a genuine fd-0 socket with launchd's inetd activation emulated (B.1-B.4 plus the edge cases curl cannot express), and a code-level walk of the cancellation chain the 2026-09-23 fix relies on. Two new findings: a failing osascript is indistinguishable from an explicit Deny and leaves nothing in agent.log, and launchd's default ThrottleInterval may serialize these short-lived per-connection processes, so it should be measured first on the Mac. Co-authored-by: Qwen-Coder <[email protected]> * fix(web-shell): gate desktop relay and surface browser permission * fix(computer-use): keep desktop relay usable after cancellation * fix(computer-use): complete remote desktop relay acceptance * fix(node-repl): harden desktop relay consent, lifecycle, and cancellation paths (#11799) - skill: probe the desktop tool with `in` so the code-mode tools proxy does not abort the script before the fallback runs - agent: bound the raw JSON-RPC buffer (64 KiB, matching the HTTP path) and retire tools/call cancelled while the consent dialog is open - launchd: roll back the plist when bootstrap fails; confirm bootout actually unloaded the agent before deleting its plist - consent: log dialog failures to stderr instead of reporting them as a human denial with no trace - mcp-child-relay: forward an ambiguous-id cancellation to every match instead of dropping it, clamp relayed yield_time_ms under the daemon's 30 s frame budget, and never let a throwing child reject handle() - docs: replace the overlay merge recipe with a three-way merge, scope the tccutil reset guidance, and align the cancellation check with the channel timeout Co-authored-by: Qwen-Coder <[email protected]> * fix(node-repl): bound desktop relay connection failures * fix(web-shell): preserve active desktop relay revocation * docs(computer-use): correct setup and redact verification details * fix(desktop-relay): close review security gaps * fix(desktop-relay): harden process ownership and prewarm * fix(desktop-relay): pin the setup command to a published node-repl version DESKTOP_RELAY_INSTALL_COMMAND, packages/node-repl/README.md and docs/users/features/computer-use.md all handed out `npx -y @qwen-code/node-repl-mcp@latest desktop-relay install`. The published `latest` (0.1.6) has no `desktop-relay` subcommand and its entrypoint never reads argv, so it connects a stdio MCP server and blocks on stdin: zero output, nothing installed, no launchd socket. The Web Shell panel then re-serves that same string behind a Copy button in exactly the `missing` phase the command produced, so the loop has no exit. Pin the spec to the package version the way the repo already does for this package (packages/core/src/skills/bundled/browser-use/skill.test.ts), and bump packages/node-repl 0.1.6 -> 0.1.7 in this PR so the publish carries the relay. Without the bump, cd-cua-driver.yml rejects the release with "already exists with different integrity" and the pinned version would still lack the subcommand. Sync the three version lines the release validate-version gate reads (computer-use SKILL.md, browser-use SKILL.md, computer-use user guide) plus the hard pin in scripts/tests/release-workflow.test.js. desktop-relay-client.test.ts now fails when the constant drifts from packages/node-repl/package.json or reintroduces `@latest`. Co-authored-by: Qwen-Coder <[email protected]> * fix(desktop-relay): stop reporting relay state that was never confirmed Five review findings shared one cause: the Web Shell panel and the launchd teardown each asserted a state they had not actually confirmed. - deriveDesktopRelayStatus let another session's live relay mask this session's connect failure, because input.error was only consulted after the other-session return. The error now wins over other-session; a relay that is live for this session still outranks it, since that is the truth. - awaitingApproval and error were component-global while the probe was scoped by sessionId and daemonUrl, so switching session or daemon showed the previous one's pending dialog or stale error. Both are reset when either changes. - retainLiveDesktopRelayProbe kept a live observation for any non-ready probe without limit, so a relay that vanished stayed "connected" for as long as the panel was mounted. Retention now expires after three inconclusive probes, which still covers a transient blip. - disconnectDesktopRelay resolved void and never read the response status, so a revocation that never reached the relay was indistinguishable from one that did, and disconnect() cleared the probe unconditionally. It now reports success only on an ok response, and the panel keeps its last observation until the relay confirms, leaving Disconnect retryable. - currentUid() returned 0 under sudo, so uninstall queried gui/0 while the agent was loaded in the invoking user's domain. The existing "still loaded" guard in uninstallLaunchAgent therefore could not see the agent and deleted the plist out from under it. SUDO_UID now supplies the domain; Number() rather than parseInt() rejects a malformed value instead of truncating it into a wrong uid. Addresses review findings R3-5, R3-6, R3-7, R2-10 and R3-14. Each has a regression test that was verified to fail when its fix is reverted. Co-authored-by: Qwen-Coder <[email protected]> * fix(desktop-relay): cancel stale approvals and bound registration * fix(desktop-relay): keep revocation available when unreachable * fix(desktop-relay): serialize consent and allow cold runtime startup --------- Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: qwen-code-ci-bot <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> Co-authored-by: qwen-code-dev-bot <[email protected]> |
||
|
Shaojin Wen
|
692a5d7446 |
feat(managed-agent): Add the public API contract and contract tests (#12808)
* feat(managed-agent): Add the public API contract and contract tests Move the reviewed Managed Agent OpenAPI into the server module as the single source for the public and WebShell routes (v1.13), and check the server and the WebShell client against it. - A contract test compares the mapped Spring routes, the API records, and the real requests, responses, declared headers and SSE frames of every partial route with the spec. Current differences are recorded as expected failures that the later Stage D slices remove; a fixed gap that stays listed fails as well. - The WebShell client types are generated from the spec without its planned surface, and a test fails when the committed types are stale. - Rename, unarchive, archive and delete, which the server already serves, are recorded as partial routes. Part of #12793. * test(managed-agent): Cover archived reads and WebShell first input in the contract Address review feedback on the contract test and its callers. - The real-traffic scenario now reads an archived Session through get and list, and creates a WebShell Session with its first message. That records three more gaps: archived Sessions read back with status "archived", which the contract's enum lacks, and the WebShell create request carries the same "text" input type as the other D2 lines. - A property that appears in several oneOf or allOf members counts as planned only when every member marks it planned. - Replace the deprecated JsonNode.fields() calls. - The managed-progress e2e spec builds a Session with agentId and events with data, as the generated types require. - The design note records the gaps that the gap file cannot hold: undeclared error responses, the full-page has_more semantics, workspace-bound Sessions, and the merge order with W0d. * test(managed-agent): Reject unknown gap categories and quiet the validator Address the sandboxed verification of the contract test. - Every line of the known-gaps file must start with a category that a test filters on. A line with any other prefix was never compared, so it could not be reported as resolved; it now fails every contract test and names the line. - The validator treats keywords outside JSON Schema as annotations without logging, which removes the nine "Unknown keyword" warnings that loading the whole OpenAPI document produced on every run. - The design note no longer says that no route changes status: archive and delete move from planned to partial, and none becomes implemented. * docs(managed-agent): State exactly what openapi-typescript reads from supports-color The override comment and the design note said the generator reads only stdout.hasBasic, which 7.2.0 provides. It reads stdout and its hasBasic flag; 7.2.0 exports the same stdout, false without a TTY, and the truthiness check on stdout is what keeps colours off there. |
||
|
易良andQwen-Coder
|
010941d202 |
fix(standalone): pin @lydell/node-pty-linux-arm64 and fail release on missing prebuilds (#12649)
* fix(standalone): pin @lydell/node-pty-linux-arm64 and make missing prebuilds fatal at release The wrapper publishes a linux-arm64 prebuild at the same 1.2.0-beta.10 pin, but the root and core manifests never declared it, so the linux-arm64 standalone archive and the linux-arm64 Desktop runtime both degraded to "PTY not available" while every other target shipped a working web terminal. With the pin in place every shipped target resolves a prebuild package, so the release archive build can stop degrading silently: release.yml now exports QWEN_STANDALONE_REQUIRE_NODE_PTY_PREBUILD=1 next to the existing audio-capture gate, turning a missing prebuild into a hard failure instead of another #11872. Fixes #11872 * fix(standalone): update pnpm lockfile for the linux-arm64 node-pty pin The two package.json manifests gained @lydell/node-pty-linux-arm64 but pnpm-lock.yaml was not regenerated, so every CI job failed at the frozen-lockfile install with ERR_PNPM_OUTDATED_LOCKFILE. * docs(standalone): align the linux-arm64 PTY guide and tripwire with the pin Pinning @lydell/node-pty-linux-arm64 makes linux-arm64 a supported PTY target, but INSTALLATION_GUIDE.md still told users that archive ships without PTY and reports "PTY not available". CI could not catch the drift because install-script.test.js pinned that exact sentence as its tripwire, so the guide and the assertion have to move together. List all five shipped targets in the guide, retarget the tripwire at the new sentence, and refresh the matching stale comment in desktop-shell's test-release.js degrade fixture - that arm now only fires for a target the map does not know. Co-authored-by: Qwen-Coder <[email protected]> * fix(standalone): complete the node-pty linux-arm64 pin in esbuild externals `create-standalone-package.js` documents that "esbuild.config.js keeps every node-pty specifier external" so the bundled CLI resolves the platform prebuild from `lib/node_modules` at runtime. The external list carried five of the six pinned platform packages and omitted `@lydell/node-pty-linux-arm64` — the exact platform this PR pins. Nothing imports that specifier statically today, so the bundle is unaffected; the gap is that the invariant the archive's PTY support rests on was one entry short of complete. Probe (esbuild, `--platform=node --format=esm`, entry importing both specifiers): - without the entry: `Could not resolve "@lydell/node-pty-linux-arm64"` - with the entry: builds clean and emits the import left external, matching the other five platforms Also corrects two comments this PR rewrote in the previous round. Both claimed the desktop degrade arm "fires only for a target the map does not know (a future platform)". That trigger is unreachable: `desktopTarget()` throws `Unsupported desktop target` for anything outside its five-entry allowlist and `NODE_PTY_PREBUILD_PACKAGE` maps exactly those five, so `prebuildPackage` is always a real name. Verified by running the script: $ QWEN_DESKTOP_SKIP_BUILD=1 QWEN_DESKTOP_TARGET=freebsd-x64 \ node packages/desktop-shell/scripts/prepare-runtime.js Error: Unsupported desktop target: freebsd-x64 at desktopTarget (prepare-runtime.js:351:11) The reachable trigger is the one the warn string already names and the one `test-release.js` reproduces four lines below its own comment: a known target whose prebuild package is unpinned in the source root manifest. Comment-only apart from the single external entry. prettier --check and eslint --max-warnings 0 clean on all three files; node --check passes. Co-authored-by: Qwen-Coder <[email protected]> * test(standalone): pin the node-pty release gate and its target map Closes out three review findings on the node-pty pin. Test- and comment-only apart from one new export; no production behavior changes. R1-1 prepare-runtime.js: the nodePtyPackageSpecs() docstring still claimed its null return "is how an unsupported target degrades". desktopTarget() throws for anything outside its five-entry allowlist, and test-release.js:testRuntimeNodePtyTargetMapping() asserts NODE_PTY_PREBUILD_PACKAGE's keys equal exactly that allowlist, so an unknown target can never reach it. A null here is a dropped pin. Verified by re-running that tripwire's extraction against the edited file: both literals still match and keys == allowlist == the same five targets. R1-2 create-standalone-package.js: export TARGET_PREBUILD_DIR and assert every release target maps to a package the manifests actually pin. The guide sentence this PR adds states that invariant while release.yml:707 makes violating it abort the whole archive step, yet the map had zero test references. The expectation is derived from readNodePtyPackageSpecs() so the version list stays single-sourced, and is one-directional because @lydell/node-pty-win32-arm64 is pinned while no target maps to it. R1-3 install-script.test.js: the release.yml assertion matched only the variable name, which also appears in the packager's usage text, so nothing connected producer to consumer and no test executed the fail() branch at create-standalone-package.js:578. Tightened to the full wiring line and added the behavioral twin of the audio-capture pair: gate on with the addon removed must throw, gate off must still build a PTY-less archive. packageFakeStandalone() gained an env option for this; spreading process.env leaves its 15 existing callers unchanged. Mutation-checked, each mutant applied alone then restored and md5 verified: win-x64 mapped to an unpinned name, gate reader renamed to '2', gate polarity flipped to !==, and release.yml no longer evaluating to '1' are each caught by a new or tightened assertion. The polarity flip was previously undetectable on the PR lane, where the linux-x64 prebuild is present so the gate is never reached. Focused: install-script.test.js 130 passed / 11 skipped (141). prettier --check and eslint --max-warnings 0 clean on all three files. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmufv3et42h * docs(standalone): state the real node-pty degrade conditions in comments prepare-runtime.js said the degrade arm fires only "where the repo pins nothing", but nodePtyPackageSpecs returns null at the first unpinned name of the pair, so it also fires when the wrapper pin is the one that went missing while the warning names the prebuild package. Say "either name in the pair" and note which package the warning names. create-standalone-package.js justified the non-fatal default with "a developer archive stays usable on an unmapped future target", which cannot happen: main() calls copyNativeAddon first and it reads the same TARGET_PREBUILD_DIR in an unconditional path.join(), so an unmapped target dies with a TypeError in the audio step before copyNodePtyAddon runs. Replace the rationale with the case that does reach the warn arm (a target whose prebuild the host install does not have). Comment-only; no behaviour change. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmug1iw422v * test(standalone): make the node-pty gate assertions discriminate Pin the `Required ` prefix on the fatal-gate regex. copyNodePtyAddon builds one message and emits it on both paths, so the prefix-less regex was also satisfied by the degrade warning folded into the child's stderr — the suite would have reported the gate as covered after the prefix was dropped from fail(). The sibling clipboard gate assertion already spells the prefix. Assert QWEN_STANDALONE_REQUIRE_NODE_PTY_PREBUILD structurally on the `Build Standalone Archives` step env instead of as a whole-file substring, so commenting the line out or moving it into a neighbouring step's env goes red. build-standalone-release.js spawns the packager with no env override, so that step's env is the only route to the gate. The comment defending the substring pin named a collision with create-standalone-package.js's usage text, which cannot occur: the searched string is release.yml alone. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmug1iw422v * test(standalone): cover the node-pty target map and the win-x64 dir name Two coverage gaps left by the previous round of gate tests: - The map test looped over TARGET_PREBUILD_DIR itself, so a release target with no map entry was never visited, deleting a whole entry merely removed an iteration, and an emptied map passed vacuously. Drive the loop from TARGETS and assert the lookup before the pin. Still one-directional: @lydell/node-pty-win32-arm64 is pinned while no target maps to it. - Both gate tests reached the packager through packageFakeStandalone, which hardcoded --target linux-x64, the one dialect where the prebuild dir name equals the target name. Add a win-x64 arm so the 'win-x64' -> 'win32-x64' translation in copyNodePtyAddon is covered on every lane rather than only on test_windows. Parameterise the staged-modules fixture and the packaging helper to make that reachable without a Windows host. Test-only change; no production code is modified. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmug5t7mp33 --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
jinyeandQwen-Coder
|
c822995d3a |
feat(cli): Add managed runtime attestation contract (#12447)
* feat(cli): Add managed runtime attestation contract Co-authored-by: Qwen-Coder <[email protected]> * fix(cli): Strengthen runtime attestation conformance Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
ChiGao
|
74b5eb9e6d |
chore(cli): bump the OpenTUI renderer to 0.5.10 (#12321)
* chore(cli): bump the OpenTUI renderer to 0.5.11 The upstream release makes wrapped layout byte-accurate, which removes an extra leading space on wrapped paragraph continuations and stops greedy wrapping from breaking a line short. Both were visible divergences from the incumbent renderer. It also adds line breaking between CJK characters. * chore(cli): pin the OpenTUI renderer to 0.5.10 and gate the wrap fix 0.5.11 stops printing the markdown link target text, which is the only thing link-click.ts can match on to implement click-to-open, so the newer release silently breaks that path. 0.5.10 carries the same three wrapping fixes without that regression and declares no engines floor. Add a wrap-parity golden covering the paragraphs whose framing drove the bump, wired into the tui-parity workflow, so re-pinning to 0.5.8 or accepting a later release that regresses wrapping turns CI red. Verified in both directions: green on 0.5.10, all three cases red on 0.5.8 while the pre-existing parity scripts stay green there. Also correct the two places whose claims measurement disproved — the link-click header comment asserted no 0.5.x release emits OSC 8, and the migration design doc still called 0.5.8 the pinned version — and record in the parity sweep that whole-row divergence narrowed rather than closed, since the rows carrying these paragraphs also carry two unresolved divergences. |
||
|
易良andQwen-Coder
|
baf0ff41f7 |
refactor(build,release): Use pnpm workspace builds and publishing (#12400)
* refactor(release): Use pnpm for workspace publishing and versioning * fix(ci): revert desktop-release install to npm ci for Windows compatibility The pnpm migration added a bash conditional to check for pnpm-lock.yaml, but the step did not specify shell: 'bash'. Windows runners default to PowerShell, which cannot parse bash syntax, causing the build to fail with a ParserError. Since desktop-release always checks out the same commit and the root lockfile is now pnpm-lock.yaml, the conditional is unnecessary. Revert to the simple npm ci command that works on all platforms. Fixes the Windows build failure in Desktop release v0.24.2. * refactor(build): Schedule workspace builds with pnpm * fix(ci): Preserve Windows release installation and version filters --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
64dd058c9c |
ci(pnpm): install with pnpm everywhere and retire package-lock.json (#11859)
* ci(pnpm): install dependencies with pnpm in CI and release CI tests the graph release ships only if both install it the same way, so this moves both, keeping every `npm run` script and `npm publish` as they are. - ci.yml (7), release.yml (3) and release-vscode-companion.yml (2) install with `corepack pnpm install --frozen-lockfile`; release keeps --ignore-scripts plus its explicit postinstall and generate steps. - setup-node drops the npm cache and sets package-manager-cache: false, which setup-node v5+ needs once package.json declares a pnpm packageManager. - A composite action caches the pnpm store on the three always-hosted jobs; the ECS pool keeps its store on local disk. - version.js no longer lets npm reify node_modules: --no-workspaces-update on both `npm version` calls, and a package-lock-only refresh in step 9. - The notices generator, prepare-package's sharp pin and the standalone release's native package specs read the installed tree or pnpm-lock.yaml instead of package-lock.json keys. - New size baselines for the three workflows, which were already over them. * chore(vscode): regenerate NOTICES from the pnpm-installed tree Generated in CI (pnpm Worktree Smoke on ubuntu-latest) by the ported generator. Same entry set as before except the five packages npm and pnpm resolve differently: fdir 6.5.0, picomatch 4.0.5, mdurl 2.1.0, @types/node 22.20.1 and mime-db 1.52.0 (form-data's pinned copy). No entry lacks a license text or repository. * fix(pnpm): resolve undeclared zod peers to the repo's zod, as npm does The first CI run on this branch failed the export renderer's size budget: 1,930,015 bytes against 1,930,000, where the same commit built on npm's tree comes to 1,858,186. Comparing both trees' esbuild metafiles and prebuilt bundles put the whole difference in web-shell's transcript.js (+100,903 bytes pre-minify), and the added code is zod 4 plus the @modelcontextprotocol/ext-apps schemas; the sdk daemon bundle is identical. web-shell bundles ext-apps, whose zod and MCP SDK are peers, and declares no zod itself, so pnpm filled the peer with the highest locked zod (4.4.3) while npm serves the hoisted 3.25.76. Declaring zod 3.25.76 as a root devDependency makes pnpm resolve every such peer to it (resolve-peers-from-workspace-root is on by default), which is npm's hoisting behaviour. mobile-mcp keeps its own zod 4.4.3, as under npm; the npm tree does not change. * fix(pnpm): keep npm's root @types/node so root-level typechecks match The no-AK integration gate typechecks integration-tests/, which takes Node's types from the root node_modules. npm hoists @types/node 20.19.1 there, because acp-bridge and core pin it; pnpm, with no root declaration, hoisted 22.20.1, whose spawnSync overloads type process-registry.ts's stdout as string | NonSharedBuffer and fail TS2322. The same code passes on every npm-installed PR. Declaring 20.19.1 as a root devDependency puts npm's copy back where a root-level tsconfig looks. The npm tree does not change. * ci(pnpm): install with pnpm in the remaining workflows The follow-up to switching CI and release: every other workflow that installs the root workspace now uses the pinned pnpm with --frozen-lockfile, so no job tests a graph that release does not ship. - e2e, tui-parity, repo-hygiene, web-shell-visuals, serve-ab, sdk-java, windows-runner-smoke, qwen-autofix, the SDK releases, finalize-release, sync-release-to-oss, the root install in desktop-release and cd-cua-driver, and the sandbox Dockerfile. - setup-node drops the root npm cache (package-manager-cache: false); hosted jobs restore the pnpm store through the shared composite action, and qwen-autofix keeps af-012's rule that the persistent pool restores no remote cache. - serve-ab and web-shell-visuals install a separate base checkout that can predate pnpm-lock.yaml, so those installs fall back to npm ci there. - Out of scope, unchanged: packages with their own lockfile (cua-driver's TypeScript SDK, desktop-shell, live-host, mobile-mcp), and the triage lanes with npm-cache.yml and the npm audit gate, which move with the package-lock.json retirement. - New size baselines for the touched workflows; qwen-autofix stays under the 470000-byte gate. * fix(pnpm): link the workspace packages that root-level code imports npm links every workspace package into the root node_modules, so integration-tests/ and scripts/, which are not workspace packages themselves, can import @qwen-code/* by name. pnpm links a workspace package only where it is a declared dependency, and the root declared none, so the no-AK integration gate could not find @qwen-code/qwen-code-core. The root now declares the seven packages that root-level code imports at runtime as file: devDependencies: acp-bridge, channel-base, the CLI, core, sdk, web-shell and web-templates. npm already links them, so its tree does not change (letting npm re-resolve the lock reproduces the same root entry); .pnpmfile.mjs rewrites them to workspace:*, which pnpm links at the root. * ci(pnpm): move triage, the store producer and the root audit to pnpm - qwen-triage's verify and tmux-testing lanes install the PR with `corepack pnpm install --frozen-lockfile` from a restored pnpm store. The one retry clears node_modules first, because pnpm install, unlike npm ci, keeps it. - npm-cache.yml becomes pnpm-store.yml, keyed on pnpm-lock.yaml. - security-checks audits the root workspace with `pnpm audit`, which reads the lockfile, so the root install step goes. The vendored lockfiles keep npm, and the retry recognises pnpm's endpoint error too. - qwen-autofix drops the hosted-only store cache: its heavy jobs forbid local `uses: './...'` actions, and the file sits at the size gate. - cd-cua-driver: setup-node@v4 has no package-manager-cache input. - package-assets fixtures install sharp rather than listing it in package-lock.json, matching how prepare-package now resolves it. - AGENTS.md, CONTRIBUTING.md and the npm workspace guide install with pnpm. * feat(review): install pnpm repos in qwen review's build/test qwen review installed dependencies only for a repo with package-lock.json and reported a pnpm repo as unsupported, so reviews of qwen-code would lose their build and test once its npm lockfile goes. - lockfileInstaller() picks npm or pnpm from the committed lockfile. With both, `packageManager: pnpm@...` chooses pnpm; anything else keeps npm. - A pnpm repo installs with `corepack pnpm install --frozen-lockfile` through the same disk, budget and sandbox path as `npm ci`, and counts as installed once node_modules/.modules.yaml exists. - Scripts still run through `npm run`, and the npm path is unchanged. * ci(pnpm): retire the root package-lock.json pnpm-lock.yaml is now the only root lockfile. - check-lockfile keeps the pnpm integrity check and Playwright parity, which now reads pnpm importers and snapshots. The npm-vs-pnpm agreement and build-approval checks go: pnpm 11's strictDepBuilds and every CI install's --frozen-lockfile enforce what they covered. - pnpm-lock-freshness.yml goes; it regenerated pnpm-lock.yaml from package-lock.json. - Release: version.js drops its package-lock refresh, the release commit stages pnpm-lock.yaml and the SDK release stages no lockfile, so neither `git add` fails on the missing file. `npm version` in the SDK and mobile-mcp releases passes --no-workspaces-update so it cannot reinstall the root with npm. - cd-mobile-mcp installs from the root pnpm lockfile. setup-node in cd-mobile-mcp, qwen-code-pr-review and desktop-release no longer keys an npm cache on the root lockfile. - check-desktop-isolation reads pnpm importers; build.js, build_sandbox.js and preflight install with pnpm. - pnpm-lock.yaml, pnpm-workspace.yaml and .pnpmfile.mjs join the autofix supply-chain class and repo-hygiene's deny list; pnpm-lock.yaml joins autofix's generated-diff exclusions, sdk-java's path filter and the platform-sensitivity manifests. - .gitignore lists /package-lock.json so an npm install cannot commit it back. Tests and docs follow. * fix(standalone): read the node-pty versions from the pnpm lockfile * chore(vscode-ide-companion): regenerate NOTICES.txt after the lockfile swap * chore(vscode-ide-companion): regenerate NOTICES.txt from the pnpm dependency tree The committed NOTICES.txt was generated from an npm install and does not match the tree produced by the pnpm install the release pipeline now uses (659 resolved dependencies, differing @types/node and mime-db versions). Regenerating with the pinned pnpm install makes the freshness check reproducible. * fix(ci): drop the npm cache from the daily dependency audit actions/setup-node's npm cache keys off the root package-lock.json, which this branch retires, so the scheduled job would fail at Set up Node before the audit ran. No pull_request run reaches this workflow to catch it. The root tree is never installed here either: pnpm audit reads pnpm-lock.yaml directly. Matches the sibling workflows that already set package-manager-cache: false; scripts/tests/security-workflows.test.js now pins it. * chore(ci): stop banking main-side drift in the workflow size ratchet The ratchet exists to turn a workflow's growth into one reviewed line. This branch re-recorded entries for 17 workflows it never touches, several upward by 0.4-2.8 KB - drift that happened on main, banked into the manifest without review. Restore those to their merge-base values, so .size-baseline changes only for the 22 workflows this branch actually edits. The entries were not needed to go green: ci.yml passes the PR's base commit, so a file that drifted on main and is otherwise untouched earns a warning, not a failure. * fix(release): initialize standalone state before main * test(browser-use): read Playwright pin from pnpm lock * fix(ci): address pnpm migration review blockers * fix(desktop): stop reading the retired root lockfile --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
|
b7543aeb1b |
feat(browser-use): add Chrome Native Messaging relay (#11242)
* feat(browser-use): add Playwright browser SDK
* docs(browser-use): document browser use architecture
* fix(browser-use): make browser input actions reliable
* feat(browser-use): preserve screenshot coordinate metadata
* fix(browser-use): recover stale sockets across VM realms
* fix(browser-use): bound locator action retries
* fix(browser-use): keep page input in the background
* fix(browser-use): reject ambiguous input arguments
* fix(browser-use): validate locator composition identity
* fix(browser-use): recover socket locks across VM realms
* fix(browser-use): preserve input results across navigation
* fix(browser-use): capture screenshots through bounded CDP frames
* fix(browser-use): separate input and navigation waits
* feat(browser-use): add Chrome native relay
* fix(chrome-extension): avoid foreground tabs and stale cursor markup
* fix(chrome-extension): require Chrome 125 for flat sessions
* feat(browser-use): add product integration
* test(browser-use): add managed Chrome smoke coverage
* feat(node-repl): expose structured image metadata
* test(browser-use): preserve complete smoke reports
* feat(browser-use): bundle skill and runtime with CLI
* fix(browser-use): install pinned Playwright in fresh workspaces
* feat(browser-use): require explicit skill opt-in
* feat(browser-use): make bundled skill available by default
Keep explicit skill disables authoritative, update documentation and tests, and use the latest Node REPL package in setup instructions.
* fix(ci): build browser-use before Live Host publish
Build the bundled browser runtime before core in the Live Host npm dependency chain, and cover that prerequisite order with a regression test.
* fix(browser-use): output screenshot metadata explicitly
* fix(node-repl): preserve image metadata outside prose budget
* fix(browser-use): address review round on workspace wiring and runtime correctness
- register the workspace in .pnpmfile.mjs, pnpm-lock.yaml, and the
unit-vitest-configs registry, with the shared-pool timeout ternary and
off-Linux unhandled-error exemption (fixes the red Install/Test checks)
- map Playwright's 'warning' console type to the contract's 'warn' level
- reject captures whose pixel dimensions contradict the css-pixels
coordinate contract instead of certifying rescaled images
- keep one unattachable derived tab from failing tabs.list while still
propagating a lost session
- clear the cached dialog when Chrome resolves it out of band or the page
navigates, guarded by dialog identity so a newer dialog survives
- synthesize a parseable Browser.getVersion (numeric Chrome major, host
platform token) so Playwright enables the right keyboard editing commands
- keep YAML-quoted snapshot keys and link props in interactive listings and
stop claiming subtrees that filtering removed
- always release the mouse button when a drag gesture fails
* fix(dev): build browser-use runtime before startup
* fix(browser-use): isolate cleanup and correct smoke checks
* fix(browser-use): resolve managed Chrome through Playwright
* fix(browser-use): harden native relay and tab lifecycle
* fix(browser-use): drain host output and persist restored tab state
* fix(browser-use): scope dialog handles and clean navigation waiters
* fix(browser-use): preserve input outcomes and release modifiers
* fix(browser-use): drain session teardown and release debugger attachments
* fix(browser-use): correct locator validation and evaluation
* fix(dev): reuse prebuilt Browser Use runtime
* fix(browser-use): address review round on locator, bridge, and screenshot correctness
Co-authored-by: Qwen-Coder <[email protected]>
* fix(browser-use): require installed extension before native host setup
* fix(browser-use): address review round on schema probe bound and regex validation
* fix(browser-use): start transport before Chrome preflight
* fix(browser-use): exclude staged runtime from code checks
* fix(core): declare Browser Use build dependency
* fix(browser-use): preserve successful navigation and dialog input
* fix(browser-use): harden input drain and tab registration cleanup
* fix(browser-use): keep sessions bound to one Chrome profile
* fix(browser-use): bound commands and validate evaluation JSON
* fix(browser-use): reuse unchanged Native Host registration
* fix(browser-use): wait for fresh extension installation
* fix(browser-use): address review round on screenshot origin, dialog scope, and bridge privacy
- screenshot: tolerate settle-probe failure (measure the ratio from the
captured pixels, re-capture on HiDPI), measure layout after the settle
wait, and reject screencast frames whose scroll offsets disagree with
the published origin
- session: clear the cached dialog only on main-frame navigation;
abort tab attachment when the tab is removed mid-attach
- errors: classify crashes and timeouts by Playwright's structured error
name instead of page-influenced message text
- locator.press: release chord modifiers when a later token is rejected;
cap locator.type at the 60k characters its per-character budget can
deliver inside the 120s ceiling
- snapshot: keep AI-mode [cursor=pointer] generic nodes in the
interactive listing
- bridge: derive the fallback socket directory as an env-independent
per-user 0700 subdirectory on all POSIX hosts and verify ownership
before binding
* fix(browser-use): close review-critical gaps in error mapping, budgets, and downloads
- errors: classify by Playwright error name first; text phrases match only
the start of the first message line so page-controlled text (log tails,
quoted markup, page-thrown values) cannot pick the reported code, and a
TimeoutError name wins over crash-shaped page text
- transport: cdp.send bridge requests default to 130s so operations up to
the 120s schema ceiling report their own deadline; other requests keep
the 30s default
- transport: the /tmp fallback socket directory is a per-user leaf directly
under the sticky root (no shared intermediate a first mover could own);
ensureSocketDirectory re-verifies the leaf after creation and walks
ancestors up to the sticky root, requiring each to be owned by this user
or root and not group/other-writable
- runtime: withModifiers no longer rewrites a completed action into a
failure when a modifier release fails; chord failure cleanup releases the
chord's own tokens (including non-modifiers) instead of four fixed names
- playwright: enable CDP download events on tab attach (Playwright's
noDefaults connect leaves them off, so waitForEvent('download') could
never resolve); locator.downloadMedia races the caller deadline, aborts
the in-page fetch at it, resolves media contained in a located wrapper
before an ancestor link, restricts URL schemes, and names CORS rejections
distinctly; locator.allTextContents waits for the first match within the
caller budget and still resolves [] when nothing attaches
- screenshot: full-page captures get a 16 Mi-pixel budget (1920x3000 pages
now succeed) and the encoded byte ceiling drops to the 4 MiB model image
limit so captures are not dropped at the model boundary
- snapshot: interactiveOnly promotes only a terminal [cursor=pointer]
attribute (page text quoting the marker no longer promotes the line), and
truncation skips over-long lines instead of discarding the rest
- schemas: reject empty string/regex matchers and exact:true combined with
a regex matcher (Playwright silently drops exact for regexes)
- tests: the SDK evaluate fixture now mirrors production serialization;
POSIX socket tests carry win32 gates; posix.join keeps the derived socket
path a pure function of uid and platform on every host
* fix(browser-use): close review criticals in error mapping, chords, downloads, and snapshots
- errors: classify Playwright's genuine strict-mode violation, which
crosses CDP as the raw exception description and reaches the client
behind an "Error: " layer ("locator.click: Error: strict mode
violation: ... resolved to N elements:"); requiring the element count
keeps a page-thrown lookalike from forging LOCATOR_NOT_UNIQUE
- runtime-helpers: on a failed chord press, release the tokens
Playwright actually derived ('+' separates only after a non-empty
token, so a standalone '+' is a literal key) rather than the caller's
array elements; an element embedding '+' previously released nothing
and left modifiers held. locator.press uses the same tokenizer
- locator.downloadMedia: resolve an ordered candidate list (element,
media descendants, anchor descendants, closest ancestor) and keep the
first candidate that exposes a URL, so a wrapping anchor no longer
beats the image it contains and a source srcset or an unloaded
video's source child can supply the URL
- snapshot: unwrap Playwright's single-quoted YAML key before the
[cursor=pointer] suffix test, so clickable nodes whose accessible
name forces key quoting stay in the interactive listing
- docs: align the Input completion contract with the shipped modifier
cleanup semantics (cleanup failures are discarded; only the action's
own error propagates) in both languages
* fix(browser-use): close round-13 review criticals in errors, srcset, evaluateAll, snapshots, and screenshot origin
* fix(browser-use): point the pnpm importer at a vitest snapshot that exists
Merging main brought in #11625, which regenerated pnpm-lock.yaml and
dropped the [email protected] snapshot keyed with @vitest/browser and
[email protected]. The packages/browser-use importer still referenced
that key, so every Install leg failed with
ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY. Re-key the importer to the snapshot
its sibling packages already use; nothing else in the lockfile changes.
* fix(browser-use): close round-14 criticals in error classification, dialog ordering and screenshot ratio
R13-2: the strict-mode branch of sanitizeOperationError was the only
text-phrase branch not gated on the evaluate channel, so a page-thrown
primitive "strict mode violation: ... resolved to 2 elements:" picked
LOCATOR_NOT_UNIQUE. On that channel only Playwright's "Error: "-wrapped
form classifies now; other channels keep the unwrapped form.
R14-1: Playwright hands a dialog over on a later turn than the bridge
reports its CDP events (a microtask in CRSession plus the in-process
dispatcher's setImmediate), so an open+close pair delivered in one chunk
left a phantom dialog that blocked the tab. Each tab now traces dialog
openings and closes in bridge order and drops a delivered dialog the
bridge already reported closed; a close with no traced opening (a dialog
open before the tab was attached) is never charged to a later dialog, and
the microtask clear stays for the delivered case.
R14-2: window.devicePixelRatio is page script, and a spoofed value became
an unbounded clip.scale while the pixel re-measure only ran when the
probe failed. The probed ratio is now trusted only within 0.25..8, and the
measure-and-recapture correction runs whenever the first fallback capture
disagrees with the requested CSS region, deriving the real ratio from
Chrome's own pixels (measured / (width * scale)).
Also resolve the temp root in the ensureSocketDirectory tests: the
ancestor sweep rejects symlinks, and macOS reaches os.tmpdir() through
/var -> /private/var, so the tests failed locally on macOS.
* fix(browser-use): tear down accepted peers on start failure and pin the transport's untested paths
The esbuild banner declared require/__filename/__dirname/process bindings
the bundle never uses, so any bundled module declaring one of them built
green but failed to load; drop it (and the unused .wasm loader) and import
dist/index.js after bundling so an unloadable bundle fails the build.
When start() fails after listen (chmod, handshake), a peer validated in
between kept server.close() waiting forever; destroy and clear the
accepted sockets first, mirroring stop(). net.Server has no
closeAllConnections(), so that is the whole fix.
Tests now cover the QWEN_BROWSER_USE_SOCKET_PATH override, a hello with a
mismatched protocol version or extension id, onConnectionChange on the
real transport, an in-flight request when the extension disconnects and
the configured request budget, the ok:false error-code mapping,
FrameDecoder split-frame reassembly and its length ceiling, the
start-failure teardown, and all five schema refine predicates.
* fix(browser-use): mirror the contract bounds in the SDK and state the documented limits
nth() now rejects indexes outside the contract's -10000..10000 before
dispatch, and all() fails closed when the plan already has 32 steps or
the match count exceeds what nth() can address, instead of handing back
locators that only fail on first use. Both evaluate script builders
normalise method-shorthand and class-method sources the way Playwright
does (compile-only vm.Script probe, then a function prefix), so those
well-typed callbacks no longer die as a page-side SyntaxError.
providerTab clamps title and url to the claimTab schema's 20000-char cap
so the documented openTabs() -> claimTab() round trip survives long URLs,
and orderOpenTabs() establishes the http(s)-only, newest-first listing
the model documentation promises. downloadMedia derives the file name
from the URL path only for http(s) sources and leaves blob:/data: names
to the browser.
The model-facing documentation now states the 1s read default, that
dom_cua.scroll's x/y are wheel deltas rather than coordinates, and that
a cross-origin resource without CORS headers cannot be downloaded from
the page origin; the design docs record that boundary as follow-up work.
Tests pin every one of these, plus buildLocator's frame, combinator,
filter and positional branches, the file-chooser SDK surface, and the
claimTab argument guards.
* fix(browser-use): keep background tab registration from moving the selection
registerTab selected every tab it registered, and two of its callers are
background paths (derivedTabTracked, syncDerivedTabs via tabs.list), so
a read-only tabs.list could move tabs.selected onto a popup. Selection is
now set only by newTab and claimTab from the info they return.
Also: tabs.get is allowed while a dialog is open, matching tabs.list and
tabs.selected, which already return the dialog-safe TabInfo; the
auxiliary-button click moves Playwright's tracked cursor before sending
the CDP button events so a later wheel lands where the click did; the
dead providerTabId lookup is removed; openTabs applies orderOpenTabs.
Tests: the runtime test file mocks node:timers so vi.useFakeTimers()
actually controls the input drain bound (the 250ms test drops to ~0ms),
and new cases pin the drain clearTimeout, the registration probe's
session detach, promptText pass-through, tabs.get under a dialog, the
openTabs/claimTab fail-closed guards and ordering, popup adoption from
derivedTabTracked, the abandoned navigation waiter's swallowed rejection,
the uploadFiles path guards, and the screencast fallback and byte budget
on the unconstrained capture path.
* test(browser-use): pin the element bindings the runtime gives evaluate scripts
The SDK emits scripts that read the free variables `element` and
`elements`, and the runtime binds them when it compiles the script inside
the page; no test ran either page-side callback, so swapping the binding
name on one side kept the suite green while every locator.evaluate would
throw a ReferenceError in production. Run the real callbacks with fake
elements so the contract is pinned from both ends.
* fix(browser-use): fail closed on page-controlled strict-mode text and prefer contained media over anchor hrefs
* fix(browser-use): bound page title reads and download a located anchor's file over its icon
* fix(browser-use): correct media downloads and frame composition
* fix(browser-use): bound locator.count and allTextContents by the caller deadline
* fix(browser-use): preserve native snapshots for DOM CUA
Co-authored-by: Qwen-Coder <[email protected]>
* fix(browser-use): classify a closed tab from the tab's own state
The pinned playwright-core client renders a closed target as a plain Error:
only TimeoutError sets `name`, so the `name === 'TargetClosedError'` branch
in sanitizeOperationError never fired, and because evaluate-channel text
fails closed by design, a tab that closed or crashed while a page command
ran was reported as OPERATION_FAILED instead of the documented STALE_TAB.
Decide tab-gone in the dispatcher from `tab.stale` and `page.isClosed()`,
which the page observers keep current, before the text classifier runs.
Drop the dead name branch and replace the test that fabricated the name
with one built the way the client builds the error, plus dispatch-level
cases for both signals that fail without the dispatcher check.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
* fix(browser-use): restore the kernel realm process binding and land the round-5 review batch
Criticals from review round 5, each reproduced before fixing:
- The bundled skill imports the staged runtime inside the node_repl kernel,
whose untrusted vm context has no `process` global and denies the
`node:process` specifier.
|
||
|
|
1c193a9a3e |
feat(browser-use): add Playwright-based Browser SDK (#11241)
* feat(browser-use): add Playwright browser SDK * docs(browser-use): document browser use architecture * fix(browser-use): make browser input actions reliable * feat(browser-use): preserve screenshot coordinate metadata * fix(browser-use): recover stale sockets across VM realms * fix(browser-use): bound locator action retries * fix(browser-use): keep page input in the background * fix(browser-use): reject ambiguous input arguments * fix(browser-use): validate locator composition identity * fix(browser-use): recover socket locks across VM realms * fix(browser-use): preserve input results across navigation * fix(browser-use): capture screenshots through bounded CDP frames * fix(browser-use): separate input and navigation waits * fix(browser-use): address review round on workspace wiring and runtime correctness - register the workspace in .pnpmfile.mjs, pnpm-lock.yaml, and the unit-vitest-configs registry, with the shared-pool timeout ternary and off-Linux unhandled-error exemption (fixes the red Install/Test checks) - map Playwright's 'warning' console type to the contract's 'warn' level - reject captures whose pixel dimensions contradict the css-pixels coordinate contract instead of certifying rescaled images - keep one unattachable derived tab from failing tabs.list while still propagating a lost session - clear the cached dialog when Chrome resolves it out of band or the page navigates, guarded by dialog identity so a newer dialog survives - synthesize a parseable Browser.getVersion (numeric Chrome major, host platform token) so Playwright enables the right keyboard editing commands - keep YAML-quoted snapshot keys and link props in interactive listings and stop claiming subtrees that filtering removed - always release the mouse button when a drag gesture fails * fix(browser-use): scope dialog handles and clean navigation waiters * fix(browser-use): preserve input outcomes and release modifiers * fix(browser-use): drain session teardown and release debugger attachments * fix(browser-use): correct locator validation and evaluation * fix(browser-use): address review round on locator, bridge, and screenshot correctness Co-authored-by: Qwen-Coder <[email protected]> * fix(browser-use): address review round on schema probe bound and regex validation * fix(browser-use): preserve successful navigation and dialog input * fix(browser-use): harden input drain and tab registration cleanup * fix(browser-use): bound commands and validate evaluation JSON * fix(browser-use): address review round on screenshot origin, dialog scope, and bridge privacy - screenshot: tolerate settle-probe failure (measure the ratio from the captured pixels, re-capture on HiDPI), measure layout after the settle wait, and reject screencast frames whose scroll offsets disagree with the published origin - session: clear the cached dialog only on main-frame navigation; abort tab attachment when the tab is removed mid-attach - errors: classify crashes and timeouts by Playwright's structured error name instead of page-influenced message text - locator.press: release chord modifiers when a later token is rejected; cap locator.type at the 60k characters its per-character budget can deliver inside the 120s ceiling - snapshot: keep AI-mode [cursor=pointer] generic nodes in the interactive listing - bridge: derive the fallback socket directory as an env-independent per-user 0700 subdirectory on all POSIX hosts and verify ownership before binding * fix(browser-use): close review-critical gaps in error mapping, budgets, and downloads - errors: classify by Playwright error name first; text phrases match only the start of the first message line so page-controlled text (log tails, quoted markup, page-thrown values) cannot pick the reported code, and a TimeoutError name wins over crash-shaped page text - transport: cdp.send bridge requests default to 130s so operations up to the 120s schema ceiling report their own deadline; other requests keep the 30s default - transport: the /tmp fallback socket directory is a per-user leaf directly under the sticky root (no shared intermediate a first mover could own); ensureSocketDirectory re-verifies the leaf after creation and walks ancestors up to the sticky root, requiring each to be owned by this user or root and not group/other-writable - runtime: withModifiers no longer rewrites a completed action into a failure when a modifier release fails; chord failure cleanup releases the chord's own tokens (including non-modifiers) instead of four fixed names - playwright: enable CDP download events on tab attach (Playwright's noDefaults connect leaves them off, so waitForEvent('download') could never resolve); locator.downloadMedia races the caller deadline, aborts the in-page fetch at it, resolves media contained in a located wrapper before an ancestor link, restricts URL schemes, and names CORS rejections distinctly; locator.allTextContents waits for the first match within the caller budget and still resolves [] when nothing attaches - screenshot: full-page captures get a 16 Mi-pixel budget (1920x3000 pages now succeed) and the encoded byte ceiling drops to the 4 MiB model image limit so captures are not dropped at the model boundary - snapshot: interactiveOnly promotes only a terminal [cursor=pointer] attribute (page text quoting the marker no longer promotes the line), and truncation skips over-long lines instead of discarding the rest - schemas: reject empty string/regex matchers and exact:true combined with a regex matcher (Playwright silently drops exact for regexes) - tests: the SDK evaluate fixture now mirrors production serialization; POSIX socket tests carry win32 gates; posix.join keeps the derived socket path a pure function of uid and platform on every host * fix(browser-use): close review criticals in error mapping, chords, downloads, and snapshots - errors: classify Playwright's genuine strict-mode violation, which crosses CDP as the raw exception description and reaches the client behind an "Error: " layer ("locator.click: Error: strict mode violation: ... resolved to N elements:"); requiring the element count keeps a page-thrown lookalike from forging LOCATOR_NOT_UNIQUE - runtime-helpers: on a failed chord press, release the tokens Playwright actually derived ('+' separates only after a non-empty token, so a standalone '+' is a literal key) rather than the caller's array elements; an element embedding '+' previously released nothing and left modifiers held. locator.press uses the same tokenizer - locator.downloadMedia: resolve an ordered candidate list (element, media descendants, anchor descendants, closest ancestor) and keep the first candidate that exposes a URL, so a wrapping anchor no longer beats the image it contains and a source srcset or an unloaded video's source child can supply the URL - snapshot: unwrap Playwright's single-quoted YAML key before the [cursor=pointer] suffix test, so clickable nodes whose accessible name forces key quoting stay in the interactive listing - docs: align the Input completion contract with the shipped modifier cleanup semantics (cleanup failures are discarded; only the action's own error propagates) in both languages * fix(browser-use): close round-13 review criticals in errors, srcset, evaluateAll, snapshots, and screenshot origin * fix(browser-use): point the pnpm importer at a vitest snapshot that exists Merging main brought in #11625, which regenerated pnpm-lock.yaml and dropped the [email protected] snapshot keyed with @vitest/browser and [email protected]. The packages/browser-use importer still referenced that key, so every Install leg failed with ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY. Re-key the importer to the snapshot its sibling packages already use; nothing else in the lockfile changes. * fix(browser-use): close round-14 criticals in error classification, dialog ordering and screenshot ratio R13-2: the strict-mode branch of sanitizeOperationError was the only text-phrase branch not gated on the evaluate channel, so a page-thrown primitive "strict mode violation: ... resolved to 2 elements:" picked LOCATOR_NOT_UNIQUE. On that channel only Playwright's "Error: "-wrapped form classifies now; other channels keep the unwrapped form. R14-1: Playwright hands a dialog over on a later turn than the bridge reports its CDP events (a microtask in CRSession plus the in-process dispatcher's setImmediate), so an open+close pair delivered in one chunk left a phantom dialog that blocked the tab. Each tab now traces dialog openings and closes in bridge order and drops a delivered dialog the bridge already reported closed; a close with no traced opening (a dialog open before the tab was attached) is never charged to a later dialog, and the microtask clear stays for the delivered case. R14-2: window.devicePixelRatio is page script, and a spoofed value became an unbounded clip.scale while the pixel re-measure only ran when the probe failed. The probed ratio is now trusted only within 0.25..8, and the measure-and-recapture correction runs whenever the first fallback capture disagrees with the requested CSS region, deriving the real ratio from Chrome's own pixels (measured / (width * scale)). Also resolve the temp root in the ensureSocketDirectory tests: the ancestor sweep rejects symlinks, and macOS reaches os.tmpdir() through /var -> /private/var, so the tests failed locally on macOS. * fix(browser-use): tear down accepted peers on start failure and pin the transport's untested paths The esbuild banner declared require/__filename/__dirname/process bindings the bundle never uses, so any bundled module declaring one of them built green but failed to load; drop it (and the unused .wasm loader) and import dist/index.js after bundling so an unloadable bundle fails the build. When start() fails after listen (chmod, handshake), a peer validated in between kept server.close() waiting forever; destroy and clear the accepted sockets first, mirroring stop(). net.Server has no closeAllConnections(), so that is the whole fix. Tests now cover the QWEN_BROWSER_USE_SOCKET_PATH override, a hello with a mismatched protocol version or extension id, onConnectionChange on the real transport, an in-flight request when the extension disconnects and the configured request budget, the ok:false error-code mapping, FrameDecoder split-frame reassembly and its length ceiling, the start-failure teardown, and all five schema refine predicates. * fix(browser-use): mirror the contract bounds in the SDK and state the documented limits nth() now rejects indexes outside the contract's -10000..10000 before dispatch, and all() fails closed when the plan already has 32 steps or the match count exceeds what nth() can address, instead of handing back locators that only fail on first use. Both evaluate script builders normalise method-shorthand and class-method sources the way Playwright does (compile-only vm.Script probe, then a function prefix), so those well-typed callbacks no longer die as a page-side SyntaxError. providerTab clamps title and url to the claimTab schema's 20000-char cap so the documented openTabs() -> claimTab() round trip survives long URLs, and orderOpenTabs() establishes the http(s)-only, newest-first listing the model documentation promises. downloadMedia derives the file name from the URL path only for http(s) sources and leaves blob:/data: names to the browser. The model-facing documentation now states the 1s read default, that dom_cua.scroll's x/y are wheel deltas rather than coordinates, and that a cross-origin resource without CORS headers cannot be downloaded from the page origin; the design docs record that boundary as follow-up work. Tests pin every one of these, plus buildLocator's frame, combinator, filter and positional branches, the file-chooser SDK surface, and the claimTab argument guards. * fix(browser-use): keep background tab registration from moving the selection registerTab selected every tab it registered, and two of its callers are background paths (derivedTabTracked, syncDerivedTabs via tabs.list), so a read-only tabs.list could move tabs.selected onto a popup. Selection is now set only by newTab and claimTab from the info they return. Also: tabs.get is allowed while a dialog is open, matching tabs.list and tabs.selected, which already return the dialog-safe TabInfo; the auxiliary-button click moves Playwright's tracked cursor before sending the CDP button events so a later wheel lands where the click did; the dead providerTabId lookup is removed; openTabs applies orderOpenTabs. Tests: the runtime test file mocks node:timers so vi.useFakeTimers() actually controls the input drain bound (the 250ms test drops to ~0ms), and new cases pin the drain clearTimeout, the registration probe's session detach, promptText pass-through, tabs.get under a dialog, the openTabs/claimTab fail-closed guards and ordering, popup adoption from derivedTabTracked, the abandoned navigation waiter's swallowed rejection, the uploadFiles path guards, and the screencast fallback and byte budget on the unconstrained capture path. * test(browser-use): pin the element bindings the runtime gives evaluate scripts The SDK emits scripts that read the free variables `element` and `elements`, and the runtime binds them when it compiles the script inside the page; no test ran either page-side callback, so swapping the binding name on one side kept the suite green while every locator.evaluate would throw a ReferenceError in production. Run the real callbacks with fake elements so the contract is pinned from both ends. * fix(browser-use): fail closed on page-controlled strict-mode text and prefer contained media over anchor hrefs * fix(browser-use): bound page title reads and download a located anchor's file over its icon * fix(browser-use): correct media downloads and frame composition * fix(browser-use): bound locator.count and allTextContents by the caller deadline * fix(browser-use): preserve native snapshots for DOM CUA Co-authored-by: Qwen-Coder <[email protected]> * fix(browser-use): classify a closed tab from the tab's own state The pinned playwright-core client renders a closed target as a plain Error: only TimeoutError sets `name`, so the `name === 'TargetClosedError'` branch in sanitizeOperationError never fired, and because evaluate-channel text fails closed by design, a tab that closed or crashed while a page command ran was reported as OPERATION_FAILED instead of the documented STALE_TAB. Decide tab-gone in the dispatcher from `tab.stale` and `page.isClosed()`, which the page observers keep current, before the text classifier runs. Drop the dead name branch and replace the test that fabricated the name with one built the way the client builds the error, plus dispatch-level cases for both signals that fail without the dispatcher check. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(browser-use): close six fail-open paths in the playwright adapter Round-22 review found six Critical defects in the new package; each is fixed at its root and pinned by a test that goes red without the fix: - locator.evaluateAll opened a second full timeout window after the attach wait had consumed the caller's budget; the wait and the read now share one deadline, and a wait that consumed the budget settles [] without evaluating. - tab.goto passed any string to page.goto, so a file:// URL would have handed local file contents to the tab's evaluate and snapshot reads; goto is gated to http(s) before the navigation runs. - Synthesized CDP error responses carried no code, so a reply orphaned by a target crash tripped Playwright's internal assert and tore down the whole transport; they now carry -32001, the orphan-swallow code. - A dialog already open when a tab attaches is replayed by neither Chrome nor Playwright, so the DIALOG_OPEN gate stayed blind while the modal blocked the renderer; registration now probes the renderer and gates the tab on an unanswered probe, with getJsDialog reporting it under a stable handle whose accept/dismiss run over CDP. - A dialog opening Playwright never delivered stayed in the trace forever and swallowed the charge of every later dialog on the tab; a closed, undelivered entry is retired once the dispatcher turn has passed. - snapshotRefLocator validated only the ref's shape, so a ref from a pre-navigation snapshot could act on whichever element the new document happened to hand the same string; refs are bound to the snapshot that emitted them and cleared on main-frame navigation. Co-Authored-By: Qwen-Coder <[email protected]> --------- Co-authored-by: qwen-code-dev-bot <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> Co-authored-by: qwen-code-ci-bot <[email protected]> Co-authored-by: Claude Fable 5.1 <[email protected]> |
||
|
|
ac505acbfc |
feat(omni): integrate multimodal media pipeline into main (#12019)
* ci: run PR CI for the omni-experiment base branch
PRs targeting omni-experiment need the same CI coverage as main.
Branch-local change only; main's ci.yml is untouched.
* feat(omni): S1 minimal path — local video via DashScope upload delivery (#8422)
* feat(omni): S1 minimal path — local video via DashScope upload delivery
Local @video files are recognized (magic-byte sniff + streaming SHA-256 +
ffprobe metadata), promoted into a content-addressed object store under
.qwen/omni/objects/, uploaded through the DashScope temporary upload
channel (getPolicy + OSS multipart form POST), and delivered to the model
as oss:// fileData parts resolved server-side via the
X-DashScope-OssResourceResolve header.
- omni pipeline gated on omni.enabled (or QWEN_CODE_ENABLE_OMNI=1) AND a
DashScope-compatible endpoint AND an available API key; any other
configuration falls back to the existing inline base64 behavior
- ffmpeg/ffprobe are hard runtime prerequisites when omni is enabled:
Config.initialize fails fast with an actionable FatalConfigError
- upload failures fail closed with an explanatory error instead of
silently downgrading to inline delivery
- per-file ceiling omni.upload.maxFileBytes (default 1 GiB) replaces the
10MB inline cap for omni-delivered video
- object store writes are atomic (.tmp + rename) and content-deduped;
.qwen/omni/ is self-gitignored
Part of the omni-experiment S1 slice (#8183).
* fix(omni): harden S1 gate, storage, and failure paths after review
Review round 1 findings addressed:
- gate: exclude Qwen OAuth (its ContentGeneratorConfig carries the
QWEN_OAUTH_DYNAMIC_TOKEN placeholder, unusable for the uploads
endpoint), require an explicit baseUrl so the configured credential is
never sent to a default origin, and require a trusted workspace before
writing .qwen/omni/ or uploading workspace bytes
- aborts: user cancellation propagates as the original abort error at
every pipeline stage instead of being wrapped into a delivery failure
- size ceiling: enforced from a cheap stat before any hashing/probing,
and invalid (<=0) configured limits fall back to the default
- object store: bytes are re-hashed while copying and verified against
the object key (TOCTOU fail-closed); dedup hits re-hash the existing
object and heal mismatched content; store directories refuse symlinks
- error hygiene: messages that can reach model-visible content carry
basenames only and a structured upstream error summary (status +
code/message) instead of raw response bodies
- getPolicy: validate the OSS ACL fields so a partial policy fails with
a clear error instead of an opaque OSS 403
Part of the omni-experiment S1 slice (#8183).
* refactor(omni): apply simplification review round
- reuse the shared combineAbortSignals helper (with listener cleanup)
instead of a local AbortSignal combinator
- deduplicate the stream-SHA-256 helper: storage now imports
hashFileSha256 from recognition
- collapse probeBinary's callback plumbing into a Map-based
availability cache
- drop the unreachable video/x-matroska extension branch (the sniffer
never emits that MIME type)
- move the omni read-result shaping out of fileUtils into
readVideoViaOmniDelivery so the file-reading hot path only carries
the gate check and a single delegation call
Part of the omni-experiment S1 slice (#8183).
* fix(omni): keep abort/timeout coverage through response body reads
Regression review of the previous two commits found:
- upload.ts: cleanup() of the combined abort signal ran at
headers-arrival time, leaving the subsequent response body read
(getPolicy JSON / failure summary) unabortable — a stalled body would
hang with neither the timeout nor ESC able to interrupt it. Body
handling now happens inside the signal's lifetime.
- storage.ts: the rename-race fallback could convert a user abort into
a dedup success after an unabortable full-file re-hash; aborts now
rethrow before the fallback, and the verify hash carries the signal.
- storage.ts: healing a planted entry now removes directories too, not
just files.
Part of the omni-experiment S1 slice (#8183).
* fix(cli): make Esc cancel @-preprocessing (media uploads) in the TUI
The @-resolution phase of a user query (file reads, and with omni
enabled potentially minutes of media hashing + upload) ran while
streamingState still reported Idle: no loading indicator appeared and
cancelOngoingRequest early-returned, so Esc was silently swallowed for
the whole preprocessing window.
Track an isPreparingQuery state around prepareQueryForGemini for
interactive user queries and feed it into streamingState. Preprocessing
now renders the spinner with the esc-to-cancel hint, and Esc aborts the
turn controller — the signal already flowed through handleAtCommand
into the omni upload pipeline, which cancels cleanly (verified: no
orphan temp files, no lingering sockets, session stays usable).
Scoped to SendMessageType.UserQuery: notification/cron/teammate drains
key off streamingState transitions and keep their existing behavior;
/btw side-questions leave the main stream state untouched.
Part of the omni-experiment S1 slice (#8183).
* docs(omni): add multimodal experiment architecture designs (#8110)
Four interlinked design docs for the omni multimodal experiment:
- file recognition & metadata: unified MediaRecognitionService, two
normalization trigger points, URL localization, raw-resource token
estimation, ffmpeg/ffprobe as hard dependency
- policy orchestration: MediaPolicyTool contract with per-invocation
args + per-tool settings, fixedPolicies with condition DSL, transport
guard on upload-channel limits, DashScope temporary-upload delivery
(all-upload, no inline), lossy-output disclosure contract
- memory: two collection trigger points (FileRecognized /
OmniPolicySucceeded), harness-exclusive writes, in-file graph,
active/side-query recall sharing one protocol
- managed media storage: content-addressed object store, staging /
quarantine lifecycle, upload cache (sha256+model -> oss URL, 48h),
mark-and-sweep GC rooted at memory references
Verified against origin/main and validated end-to-end against the
DashScope uploads API (123KB / 21MB / 424MB video, image, audio).
* feat(omni): S2 input expansion — image/audio/URL sources and token-dimension transport guard (#8512)
* feat(omni): S2 input expansion — three modalities, URL sources, token guard
Extends the S1 video-only upload delivery to the full S2 input surface:
- recognition: generalized to image/audio/video with content sniffing
(magic bytes for png/jpeg/webp/gif, mp3/wav/flac/ogg/m4a, plus the S1
video set) and per-modality ffprobe metadata; modality mismatches
between sniff and reference fail closed
- delivery: all modalities upload ORIGINAL bytes — no resizing or
transcoding on the default path (lossy transforms are the job of omni
policies, which must disclose); images skip renderImageOverview when
omni is active and carry a dimensions+zoom-hint text part instead
- converter: new fileData→input_audio branch passing the bare oss:// URL
(previously audio fileData silently textified as 'Unsupported file
media type' after a successful upload); getAudioFormat widened to
flac/ogg/m4a in lockstep with the recognizer
- URL inputs: @https://… is intercepted ahead of filesystem resolution
(previously silently dropped via the ENOENT skip), streamed to
downloads/ with SSRF/redirect policy mirroring fetchWithPolicy, hashed
while writing, re-recognized from local bytes, and delivered through
the same pipeline; per-URL failure isolation keeps the turn alive
- token estimation: versioned raw-resource estimator (method
raw-resource-v1) attached to read results as the single tokenEstimate
location; formula is a swappable slot pending provider confirmation
- transport guard: byte ceiling (S1) plus omni.transport.maxEstimatedTokens
(default 0 = disabled — observability only until the formula is
confirmed; positive values fail closed before any copy/upload)
- second normalization trigger point: processToolResultOmniMedia converts
inline tool-result media to oss:// fileData at both physical funnels
(CoreToolScheduler terminal sites and ACP Session.runTool), as a
sibling of the vision bridge — converted parts are invisible to
isImagePart so bridge behavior is untouched
Part of the omni-experiment S2 slice (#8184).
* fix(cli): deliver URL-only @ media instead of dropping it after upload
The no-file-paths early return in resolveAtCommandQuery did not count
urlMediaRefs, so a prompt whose only @-reference was a URL ran the full
download → recognize → store → upload pipeline and then discarded both
the delivered fileData parts and their display cards. Found by S2 E2E
verification (the model improvised around the missing image via its own
tools, masking the drop).
Part of the omni-experiment S2 slice (#8184).
* fix(omni): apply S2 adversarial review findings
Security lens:
- tool-result media funnel: per-result upload budget (8 parts /
128 MiB aggregate — a malicious tool result must not fan out
unbounded uploads from the user's account); excess parts stay inline
- modality gate on tool-result parts now keyed to the SNIFFED modality,
not the declared MIME type (a part declared audio whose bytes are a
video container no longer bypasses a video-disabled config)
- fs error messages are path-scrubbed before reaching model-visible
error text (basenames survive, absolute paths do not)
Correctness lens:
- non-sniffable media formats (bmp/tiff/heic images, exotic containers)
fall back to the baseline inline path via a cheap content pre-sniff
instead of failing closed — fail-closed remains for genuine pipeline
failures, not for formats the recognizer simply does not support
- user cancellation during URL media download ends the turn gracefully
instead of rejecting through resolveAtCommandQuery (which has no
throw contract) into an unhandled-rejection banner
Part of the omni-experiment S2 slice (#8184).
* fix(omni): keep the omni graph out of the ACP static bundle closure
The serve fast-path bundle-closure CI check failed: iconv-lite encoding
tables (553KB) became statically reachable from the acpAgent chunk.
Root cause: the CLI-side dynamic imports of the FULL core barrel
(`import('@qwen-code/qwen-code-core')` in atCommandProcessor) request
the whole namespace object, defeating tree-shaking of the barrel — which
statically re-exports sync-file-encoding → iconvHelper → iconv-lite
tables — and esbuild folds a dynamic import of an already-statically-
imported module into a static chunk edge.
Fixes:
- add a `./omni` subpath export to @qwen-code/qwen-code-core (pattern:
existing ./transcriptRecords) and point every CLI-side dynamic import
at it, so only the omni module graph is requested;
- re-export processToolResultOmniMedia from omni/index (circular-safe:
both modules only bind functions) so the subpath serves the
tool-result funnel;
- coreToolScheduler and ACP Session load the funnel via dynamic import
(mirroring fileUtils' existing pattern for the omni module).
Verified: `npm run check:serve-fast-path-bundle` passes; bisected
against base in a clean worktree to isolate the offending edge.
Part of the omni-experiment S2 slice (#8184).
* fix(build): register the core ./omni subpath in test resolver configs
The previous fix added a `./omni` subpath export to
@qwen-code/qwen-code-core, but vitest/tsc in this monorepo resolve the
core package through explicit aliases to TS sources rather than the
package exports map — so every suite loading atCommandProcessor or the
ACP Session failed with "Failed to resolve import
'@qwen-code/qwen-code-core/omni'".
Add the subpath alias alongside the existing transcriptRecords/goalWire
precedents in cli/acp-bridge/sdk-typescript vitest configs and the
cli/acp-bridge tsconfig paths.
Part of the omni-experiment S2 slice (#8184).
* test: isolate runtime-path suites from the runner's QWEN_RUNTIME_DIR
The self-hosted CI runners export QWEN_RUNTIME_DIR for their own qwen
job tooling. Storage.getRuntimeBaseDir gives that env var top priority,
so logger.test.ts (HOME-derived path expectations) and the ACP Session
runtime-pinning tests (Storage.setRuntimeBaseDir-based expectations)
fail whenever a PR lands on such a runner — independent of the change
under test. Reproduced locally with QWEN_RUNTIME_DIR=/tmp/fake-spool.
Clear the variable inside the affected suites (restored after each
test), making them hermetic on any runner.
* fix(omni): gate the omni dynamic import behind isOmniEnabled
The `await import()` of the omni graph inside processSingleFileContent and
resolveAtCommandQuery performs a real filesystem read under vitest (the SSR
transform writes and reads the module under /tmp/RP_*/ssr/), so mock-fs
based suites — pathReader's image cases — failed with ENOENT on a path they
never mocked.
Gate both call sites on the cheap synchronous config.isOmniEnabled() before
reaching for the import. Sessions without omni configured (every existing
test suite, and every user who has not opted in) now never load the module,
which also avoids the wasted module load on the common path. Delivery
behavior when omni IS enabled is unchanged: isOmniDeliveryActive still
performs the full five-condition gate check afterwards.
* fix(omni): resolve DNS before each media fetch and close review gaps
SSRF (review-blocking): downloadMediaUrl's gate called only isPrivateHost,
which classifies hostname text and IP literals but never resolves. A
syntactically public name whose A/AAAA record points at 127.0.0.1,
169.254.169.254 or an RFC1918 address was therefore accepted, and because
this path uploads the fetched bytes to DashScope, an internal endpoint's
response would be exfiltrated.
Add findNonPublicAddress() to utils/fetch.ts: it resolves the hostname via
dns.lookup({all: true}) and rejects if ANY returned address is non-public,
since the connect may pick any of them. IP literals short-circuit without a
lookup. Extract isPrivateAddress() as the shared bare-address classifier so
isPrivateIp keeps one implementation.
download.ts calls it inside the redirect loop, immediately before each
connect, so every accepted hop is re-validated — a permitted same-host
redirect can still re-resolve to a private address between hops.
Two deliberate choices, both recorded in the doc comment: the helper is NOT
wired into fetchWithPolicy, because WebFetch intentionally supports intranet
FQDNs resolving to private addresses, so applying it there would be a silent
breaking change; and DNS failure returns null rather than failing closed,
because the connect that follows reports the real error and failing closed
would turn a resolver blip into something the user cannot act on. The
residual check-then-connect race is documented too: fetch resolves again, so
closing it fully needs an agent that pins the connect to a vetted address,
which undici does not expose portably.
Also fixed: sniffMediaType classified any file starting with 0xFF 0xFE (the
UTF-16 LE BOM) as audio/mpeg, because 0xFE passes the frame-sync mask. Real
MPEG frames never use 0xFE — the layer bits would be 11, reserved — so
excluding it costs no genuine detection and keeps the "null for non-media"
contract honest for callers without a secondary modality gate.
Tests for the previously uncovered branches reviewers flagged:
- download: public-looking host resolving to 127.0.0.1/169.254.169.254/
10.1.2.3; mixed A records where only one is private; per-hop re-check
asserting the redirect is refused before the second fetch; DNS failure;
the ?? fetch fallback via a globalThis spy; and the production DNS path
asserting dns.lookup receives {all: true, verbatim: true}
- readMediaViaOmniDelivery: image gets the resolution + zoom_image hint
parts, audio gets a bare fileData, failure yields an error result and
never inline base64
- probeMediaMetadata: the audio branch reads the audio stream (proven with a
file carrying both h264 and aac), the image branch reports no duration
- sniffFileModality: recognized headers, non-media, and an absent path
- atCommandProcessor URL pipeline: a URL with omni active becomes a fileData
part, the same URL with omni disabled falls through as text without
downloading, and a URL-only prompt does not hit the no-content early
return
* fix(omni): pin URL-media connections to the vetted address, fail closed
Validating the resolved address and then letting fetch re-resolve is
check-then-connect: a low-TTL record answering public once and private
the second time would still be connected to, and the fetched bytes are
uploaded to a third party. Each hop now resolves via
resolveNetworkTarget('public') — which vets every returned address and
returns a lookup pinned to the vetted one — and installs that lookup as
the request dispatcher, so the name is never resolved a second time.
- fail closed: unresolvable/unvettable hosts and targets without a
pinned lookup are refused instead of fetched unpinned
- re-validated and re-pinned at every redirect hop; the previous hop's
agent is replaced and closed
- fetch defaults to undici's own fetch so dispatcher and fetch come
from one undici version (mixing majors rejects the agent)
- Node-only: Bun accepts dispatcher and silently ignores it (verified
on 1.3.11), so non-Node runtimes are refused with a clear message
- https and credential-free URLs enforced with named errors
- utils/fetch.ts: remove findNonPublicAddress/HostResolver (a weaker
duplicate of resolveNetworkTarget) and the doc paragraph wrongly
claiming undici cannot pin connections
* fix(cli): stop the @-path parser truncating URLs at query strings
The @-path terminator set treats '?', '&', ',' and ';' as sentence
punctuation, which is right for filesystem paths but wrong for URL refs:
a presigned OSS/S3 link carries its signature in the query string, so
'@https://…/a.mkv?x-oss-signature=…' was silently cut at the '?' and the
download failed with HTTP 403 naming only the host.
URL refs now run to the first character RFC 3986 does not permit
unencoded (covering whitespace and unspaced CJK prose), with trailing
sentence punctuation trimmed afterwards so it can never cut a URL short
mid-query. Filesystem path parsing is unchanged.
* fix(cli): highlight the full URL ref in the input box
The input-box highlighter had its own @-ref charset that excluded '%',
'?', '&' and '=', so a presigned URL was painted file-colored only up to
the first percent-escape and looked truncated even after the parser was
fixed to consume it whole. URL refs now use the same RFC 3986 charset
and trailing-punctuation trim as parseAllAtCommands, keeping the painted
span identical to what the parser consumes.
* fix(omni): close review round-2 findings
Criticals:
- tool-result-media: staging-dir setup moved inside convertPart's try —
a mkdir failure (ENOSPC, EACCES, ~/.qwen/omni as a regular file) now
degrades that part to inline instead of rejecting the whole tool
result and reporting a succeeded tool as failed
- sanitizeErrorMessage: exact split/join replacement of the known
filePath before the pattern pass (immune to CJK segments, ~-prefixed
and special-character basenames), with separator-based — not
ASCII-word-based — fallback classes and a Windows-path pass
- the fail-closed delivery result now sanitizes BOTH llmContent and the
error field (the scheduler sends 'error', not llmContent, to the model
on READ_CONTENT_FAILURE) and names the file by displayName
- the 100 MB image-source cap only applies when the overview decoder
will actually run: it protects the decoder, and the omni path uploads
original bytes under its own 1 GiB ceiling without decoding
Suggestions:
- malformed redirect Location surfaces as a named OmniDownloadError
instead of a raw TypeError (header value kept out of the message)
- header/idle watchdog timeouts are now identifiable in the surfaced
error instead of a generic 'This operation was aborted'
- effectiveMaxDownloadFileBytes is clamped to the upload cap even when
omni.download.maxFileBytes is configured higher
- estimation rejects degenerate present values (0/negative/non-finite)
as 'unavailable' instead of emitting a 0-token 'ok' estimate
- corrected the 0xFE sniff comment: 0xFF 0xFE is a valid MPEG-1 Layer I
header (reserved encodings are version 01 / layer 00); the exclusion
is a deliberate trade against UTF-16 LE false positives
- SSRF tests now cover private IPv6 (loopback, link-local, ULA,
IPv4-mapped) plus a public-IPv6 acceptance case
Regression tests are mutation-verified: reverting the mkdir placement
or the error-field sanitization makes exactly the new tests fail.
* fix(omni): close review round-3 criticals
- sanitizeErrorMessage: a path segment containing a space defeats the
pattern pass (segment classes break at whitespace), so 'john doe'
survived scrubbing. Call sites now pass every path they know about:
the store wrap adds the store root (covers putFile throwing before
objectPath is assigned), and the upload wrap - previously passing
err.message through verbatim - sanitizes with the object path and
store root. Function exported for direct shape tests.
- estimation: animated images (GIF/APNG/animated WebP) no longer
estimate as a single frame. The image probe branch retains ffprobe's
nb_frames as frameCount, and the estimator applies the design 6.4
visual formula with the real count - a 480x480 300-frame GIF now
estimates ~33,750 tokens instead of 113, so an enabled token guard
actually engages. Static images and containers that do not report a
frame count keep frameCount 1.
- tool-result media: a transport-guard rejection is a policy verdict,
not a transfer failure - the part is now withheld with a text
placeholder instead of being delivered inline as base64, which
silently bypassed an enabled guard at greater request cost than the
upload it was rejecting. Transfer failures keep the inline
degradation; the batch never fails for one part.
- mid-turn URL media: resolveAtCommandQuery runs under a fixed 10s
mid-turn budget that structurally killed every mid-turn @https
reference (the download alone can exceed it, and inside the resolver
the timeout abort is indistinguishable from a user cancel). URL-media
turns are now exempt from that timer - the download path carries its
own 30s header and 60s idle watchdogs - while filesystem resolution
keeps the 10s cap unchanged.
- input-box highlighter: match the URL scheme case-insensitively, like
the parser it mirrors; an uppercase @HTTPS ref no longer half-paints.
* fix(omni): close review round-3 findings
- download: resolve-phase watchdog, malformed-URL and proxy refusal,
resolver-error translation, path scrubbing, cause-chain surfacing,
DownloadedMedia slimmed to partPath; mutation-proven tests for
redirect cap, both watchdogs, byte-cap streaming, abort wiring
- recognition: direct recognizeMediaFile tests, GIF 6-byte signature,
guarded probe-handle close; sha256 moved out of RecognizedMedia
- pipeline: hash after guards (stat -> byte guard -> probe -> token
guard -> hash); displayName override so URL-funnel errors name the
remote file, not the staging path
- fileUtils: omni gating suite incl. 100 MB image-cap bypass pin
- dead S1 shims deleted (video-only probe/delivery/read wrappers)
- URL funnel: post-download modality gate mirroring the local-file gate
* feat(omni): S3 delivery reliability — upload cache, credential reuse, failure semantics, startup recovery (#8632)
* feat(omni): S3 delivery reliability — upload cache, credential reuse, recovery
Third omni slice: repeated deliveries stop re-uploading, transient
server-side failures self-heal, and crash leftovers get swept.
- upload cache: persistent (sha256, model) → oss:// URL map at
.qwen/omni/upload-cache.json with a 47h TTL (48h official validity
minus margin). Atomic writes; corrupt files are backed up and rebuilt;
per-file op serialization prevents same-process load-modify-save
races; TTL configurable via omni.upload.cacheTtlHours (0 disables).
The cache file is the only place omni persists an oss URL — the URL
remains a delivery cache, never an identity.
- credential reuse: module-level getPolicy cache keyed by
origin|model|apiKey-hash, 240s TTL (300s official validity), shared
in-flight fetches, failures never cached.
- failure semantics: provider errors that name the oss scheme or the
media download step invalidate the matching cache entries (throttling
is explicitly exempted), so the user's retry re-uploads instead of
resending a dead reference — no automatic in-pipeline resend.
- startup recovery (lazy, once per process, never throws): expired
downloads/*.part removed (48h resume window), promotion .tmp orphans
older than a 1h grace window removed (younger ones may be another
process's in-flight rename), and a day-rotating sample of ≤3 objects
(≤64MiB each) is hash-verified with corrupt objects deleted and their
cache entries cascaded.
Verified: 89 omni unit tests; 9/9 real-API E2E (cache hit 3.1× faster on
a 20MB image; fake-URL 403 → invalidate → self-heal on retry; corruption
cascade; recovery sweep), adversarial review round applied (6 findings
fixed: cross-process .tmp grace, sampling bias+size cap, throttle-safe
invalidation matching, apiKey-scoped credentials, NaN expiry, in-process
cache races).
Part of the omni-experiment S3 slice (#8185).
* fix(omni): harden S3 cache, credential, and invalidation paths from review
Upload cache (upload-cache.ts):
- reject entries:null/array shapes as corrupt (backup + rebuild) instead
of throwing raw TypeErrors on every subsequent read
- treat only ENOENT/ENOTDIR as an empty cache; other read errors skip the
operation rather than persisting a wiped file over valid entries
- key entries by (sha256, model, scope) where scope fingerprints the
endpoint credential, so URLs never cross accounts or origins
- clamp positive TTLs to the 48h server-side URL validity
- cap .corrupt-* backups at 2; shrink the fileOps serializer map
Credential cache (upload.ts):
- getPolicy no longer bakes the first caller's AbortSignal into the
shared in-flight fetch; each caller races the shared promise against
its own signal, so one caller's abort cannot poison others
Failure semantics (pipeline.ts):
- invalidate oss cache entries on the streaming error path too (the CLI
default) by threading the request into processStreamWithLogging
- tighten /oss/i to /oss:\/\//i so 'connection loss' never nukes
healthy entries; await the invalidation instead of fire-and-forget
Pipeline integration (index.ts):
- check the upload cache BEFORE store promotion: a hit now skips the
full-file copy as well as the upload
- pass the credential-scope fingerprint into the cache
Recovery (recovery.ts):
- key the one-shot latch by omni root; guard a synchronously throwing
store; fix the stride-sampling tail blind spot; make sample budgets
injectable for tests; correct the .part retention comment
Adds mutation-proof suites across all five areas: 89 tests in the omni
package plus a dedicated pipeline invalidation suite.
* fix(omni): keep startup recovery inside the managed omni root
Recovery readdir'd downloads/, the objects root, and each shard without
checking for symlinks, and hashed sampled candidates via lstat without an
isFile() check — so a symlinked shard (or a symlinked downloads/ or
objects/ directory) could redirect the scan outside .qwen/omni, where a
hash-mismatched name would get an external file deleted. The store's own
symlink guard only runs at putFile, after recovery.
- reject symlinked/non-directory managed dirs (lstat isDirectory) before
traversing downloads/, the objects root, and every shard
- reject non-regular-file candidates before hashing in the sampler
- regression tests: symlinked shard (reviewer repro), symlinked
downloads/, symlinked objects root, symlinked candidate — all four
fail against the unfixed code and pin that external files survive
with content intact
* fix(omni): close root-level symlink escape and add wholesale cache TTL sweep
- startup recovery now verifies the intermediate directory chain (omni
root and objects/) with lstat before any sweep runs: the per-directory
guards only check the final path component, so a symlink planted at
.qwen/omni or .qwen/omni/objects redirected the entire scan outside
the managed tree; two containment tests mirror the existing four at
the new levels
- upload-cache put() now sweeps all expired entries while it holds the
serialized write: get() only prunes the queried key, so entries never
read again accumulated in upload-cache.json forever
* feat(omni): S4 policy pipeline — three-modality degradation policies, Stage B tools, modelAccess gating (#8815)
* refactor(omni): migrate settings to design namespace (processing/delivery/ingestion/storage)
- omni.upload.maxFileBytes -> omni.processing.transportGuard.maxUploadFileBytes (<=1GiB)
- omni.transport.maxEstimatedTokens -> omni.processing.transportGuard.maxEstimatedTokens
- omni.upload.cacheTtlHours -> omni.delivery.upload.urlTtlHours
- omni.download.maxFileBytes -> omni.ingestion.localization.url.maxFileBytes
- new: omni.processing.{limits,fixedPolicies,transportGuard.policies,policyTools},
omni.storage.quarantine.{retentionDays,maxBytes}
- core getters renamed to match; error messages cite new key paths
Experimental branch: one-shot rename, no migration shim.
* feat(omni): surface bitRate/sampleRateHz/channels from ffprobe
ffprobe already reports these; the parse branches discarded them. The
policy condition DSL (resource.bitRate / resource.sampleRateHz /
resource.channels) and degradation disclosures both need the real
numbers. Format-level bit_rate is preferred over the stream's.
* feat(omni): add media-policy tool protocol (execution origin, descriptor, modelAccess gate)
Protocol core for the S4 policy pipeline:
- ToolExecutionOrigin (model | client | fixed_policy) on
ToolCallRequestInfo; only settable by in-process callers, never
deserialized from protocol payloads; missing origin fails closed as
model.
- MediaPolicyToolDescriptor as a DeclarativeTool code-registration
getter (default undefined) — config can never turn an ordinary tool
into a policy tool.
- PolicyArtifactBatch channel on ToolCallResponseInfo, capturing raw
successful media-policy artifacts before PostToolUse hook merging;
error/timeout results promote nothing.
- Shared modelAccess resolver + call gate
(omni/policy/model-access.ts): media-policy tools are
fixed-policy-only unless
omni.processing.policyTools.<name>.modelAccess.enabled; model calls
get defaultArguments/lockedArguments merged and are rejected when
they name a locked key; a forged fixed_policy origin on a
non-media-policy tool is rejected.
- Gate enforced on every model surface: registry declaration lists
(incl. includeDeferred for subagents), ToolSearch keyword + select:,
CoreToolScheduler pre-build, and ACP Session.runTool.
- fixed_policy scheduler calls skip the interactive permission flow but
keep PermissionManager tool-enablement and hook execution.
- omni.processing.policyTools threaded through ConfigParameters
(core + cli).
* feat(omni): add staging/quarantine storage areas and recovery sweeps
Storage design §4.3/§4.4/§6.1 for the policy pipeline:
- OmniObjectStore gains staging/ and quarantine/ areas (0o700, symlink
refusal in ensureLayout) plus the invocation lifecycle: exclusive
createStagingDir (16-hex id validation, no silent reuse),
removeStagingDir, and quarantineInvocation — which writes reason.json
(policyId/toolName/reason/failedAt) into the staging directory before
a single atomic rename into quarantine/<invocationId>/.
- Startup recovery deletes everything under staging/ (entries belong to
invocations that never committed) and trims quarantine/ to a
retention window (default 7 days) and size budget (default 5 GiB,
oldest-first), following the existing isRealDirectory containment
convention: symlinked roots and entries are never traversed, sized,
or deleted through.
* feat(omni): add fixed-policy when-condition DSL evaluator and validator
Implement the restricted when-condition DSL from the policy design (§8.3):
recursive all/any combinators over gt|gte|lt|lte|eq comparisons, with
operands drawn from three read-only namespaces (resource.*, request.*,
session.*) or literals. No arbitrary code, no JSONPath.
Evaluation is three-valued. A comparison over an unresolvable field yields
`unavailable` with the missing fields recorded — never a silent false —
so the caller can apply the policy's onConditionUnavailable behavior and
surface the fields in the run record. Combinators use strong Kleene
logic: `all` with a false branch is a determinate no_match regardless of
unavailable siblings; `any` with a true branch is a determinate match.
Vacuous semantics: all [] → match, any [] → no_match.
The evaluator is a total function that never throws — structurally
malformed nodes degrade to `unavailable`, the fail-safe outcome. The
separate structural validator (for the startup config-normalization pass,
policy design §13 #5) rejects malformed conditions with path-annotated
errors: exactly-one-of comparison/all/any, non-empty combinator arrays,
exactly-one-of field/value per operand, known-field membership, and
finite numeric literals for ordering operators.
Condition types are re-exported from omni/policy/types.ts alongside the
rest of the policy protocol surface.
* feat(omni): add three-modality degradation media-policy tools
Commit 6 of the S4 policy pipeline: the three built-in degradation tools
(mapping doc §6), each a real BaseDeclarativeTool carrying a
media_policy descriptor so the orchestrator and the modelAccess gate can
key off code-level facts.
- omni/policy/tools/media-policy-tool.ts: shared base — validates
against the NATIVE parameter schema (never the model-visible
projection Stage B will narrow), async io assertions (lstat: symlink
input refused, real output directory required), per-tool
policyTools.<tool>.runtime.timeoutMs resolution (default 600s),
uniform success/error ToolResults; success emits exactly one lossy
artifact whose metadata.omniDisclosure carries the D8 disclosure and
whose workspacePath is staging-relative.
- omni_downsample_image (sharp, lazy-loaded per D9; load failure is an
execution failure): fit 1568px / JPEG q75, EXIF orientation baked in,
animated inputs refused outright (sharp would silently keep only the
first frame).
- omni_downscale_video (ffmpeg): scale to even target height computed
in JS (no filtergraph expressions), fps 10, x264 crf 28 veryfast;
audio stream-copy with AAC 64k fallback.
- omni_downsample_audio (ffmpeg): AAC 64kbps/16kHz/mono, -vn strips
cover art.
- omni/ffmpeg.ts: runFfmpeg — never rejects, callers branch on the exit
code and must check signal.aborted explicitly.
- Registered lazily behind isOmniEnabled(); the commit-3 modelAccess
gate keeps them hidden from model surfaces by default.
* feat(omni): add degradation result cache keyed by policy fingerprint
Commit 7 of the S4 policy pipeline (decision D2): reusing a previous
transcode instead of re-paying it — a 424MB video downscale is
minutes-long and must not run once per delivery round.
- omni/json-cache-file.ts: the file mechanics extracted verbatim from
upload-cache (per-file serialized load-modify-save, atomic tmp+rename
0600 writes, corrupt backup+rebuild with capped .corrupt-* backups,
unreadable-but-existing file = operation no-op) as a shared
OmniJsonCacheFile — the design doc mandates the degradation cache
mirror these exact semantics, so they now exist once.
- omni/upload-cache.ts: refactored onto the shared file; entry
semantics (model/scope keys, TTL clamp, invalidation) unchanged — all
22 existing tests pass untouched.
- omni/policy/degradation-cache.ts: (originalSha256, policyFingerprint)
→ { degradedSha256, extension, disclosure, mimeType } at
.qwen/omni/policy-cache.json. policyFingerprint = sha256(toolName +
key-sorted tunables + tool version); per-invocation io params
(inputPath/outputDir) are excluded — they are plumbing, not policy
identity. Entries carry no TTL (content-addressed identities never go
stale); removeByOriginalSha256 / removeByDegradedSha256 serve the
GC/corruption cascades. Object existence checks stay with the
orchestrator.
* feat(omni): add fixed-policy orchestrator and reorder the delivery pipeline
Introduce runFixedPolicies: for each recognized media resource it matches
normalized fixed policies (modality, origins, when-conditions), executes the
policy's media tool through the real executeToolCall protocol (fixed_policy
execution origin, recordToolResult:false, isolated staging outputDir),
validates the returned policy artifacts against the tool's declared
descriptor (workspace containment, recognized kind/mime match, mandatory
disclosure for lossy outputs), promotes derivatives into objects/ and records
them in the degradation cache keyed by original sha + policy fingerprint.
Reorder processMediaForOmniDelivery so transport guards judge the FINAL
delivery set instead of the source: recognize -> recovery -> fixed policies
-> guards -> hash -> upload. Degraded deliveries carry a disclosure that is
emitted as a text Part immediately before the media Part everywhere media
surfaces (read tool results, tool-result inline media conversion), and the
OpenAI converter moves the disclosure together with its media part when
splitting tool media into a follow-up user message.
* feat(omni): normalize omni.processing config with system default policies
Startup normalization of the fixed-policy pipeline configuration
(policy design §13 applicable subset):
- omni/policy/config.ts: normalizeOmniProcessingConfig merges user
settings over system defaults (id-merge, whole-entry replacement,
null tombstones for fixedPolicies only) and validates structure,
enums, when-conditions, tool references (registered + media_policy
descriptor + required/lossy-disclosure outputs), fixed arguments
against the io-stripped settingsSchema, reserved io keys, guard
rules (no when, source=omit, mandatory three-modality coverage),
limits (§12.2 defaults), the 1 GiB upload cap and the 48h URL TTL.
Any violation throws OmniPolicyConfigError and aborts startup — a
mis-configured guard must never degrade into sending over-limit
media.
- System defaults (D7 dual registration): the three degradation tools
registered as preprocessing fixedPolicies WITH when-thresholds and
as transportGuard.policies WITHOUT when.
- core Config: thread fixedPolicies / transportGuard.policies /
limits / quarantine settings; normalize in initialize() after tool
warmup; expose getOmniProcessingConfig and quarantine getters.
- cli: thread the new omni.processing / omni.storage.quarantine keys.
* feat(omni): enforce policy budgets, transport-guard passes and quarantine (Stage B)
- runFixedPolicies enforces maxPolicyRunsPerRoot (checked before each
execution), maxArtifactsPerRoot and maxDerivedBytesPerRoot (checked as
derivatives land); exhaustion records outcome 'budget_exhausted' while
committed deliveries stand; maxLineageDepth clamps reprocessing.
- Failed invocations quarantine their staging dir with a reason.json
(policyId, toolName, reason, failedAt); user aborts and quarantine
failures fall back to plain staging removal. Startup recovery receives
quarantine retention/size settings from config.
- Transport-guard violations now run modality-matched guard policies for
up to maxTransportPasses passes on the final delivery; a still-violating
resource is explicitly omitted (【媒体省略】notice replaces the part in
both the read path and tool-result funnel) instead of delivered
oversized; guard-pass failures stay fail-closed. Configs without a
normalized processing config keep the Stage A throw.
- Media-policy tools project a model-visible declaration (D6): locked
arguments removed from properties/required, optional narrowing-only
parameterSchema merge and description override; validation keeps the
native schema.
* fix(omni): declare the disclosure text output in degradation tool descriptors
All three degradation tools emit a disclosure at runtime, but their
descriptors never declared the text output, so the system default
policies failed the lossy-requires-disclosure validation (config §13 #8)
at real startup. Stub-based unit tests missed the drift; config.test.ts
now normalizes the defaults against the real tool instances.
* fix(omni): harden policy pipeline per review (cache validation, permissions, settings, sweep grace, concurrency)
- validate degradation-cache entries and object-store path components
(sha256/extension shape) so a poisoned policy-cache.json cannot traverse
paths or serve malformed derivatives; malformed entries self-heal
- re-hash cache-hit objects before reuse: mismatched bytes trigger a fresh
transcode and heal the object store (D2 integrity)
- default media policy tools to 'ask' permission via a shared
BaseMediaPolicyToolInvocation (model-origin calls confirm outside yolo;
fixed_policy runs are unaffected)
- consume policyTools settings from config: tool-level settings defaults
merge under policy arguments and feed the cache fingerprint so settings
edits invalidate cached derivatives
- deliver @url omission notices and degradation disclosures: omission
replaces the fileData part with the notice text; disclosure text lands
immediately before its fileData part (D8)
- give staging sweep a 1h multi-process grace window (only entries older
than the window are deleted; symlink entries removed regardless of age)
- gate concurrent fixed-policy runs per omni root with a FIFO counting
semaphore honoring maxConcurrentResources
* fix(omni): harden policy pipeline after self-review round
Orchestrator: exclude animated images from policy matching (D9), self-heal
unverifiable degradation-cache entries, key cache fingerprints with the
descriptor version, and warn when reprocess origins exclude 'policy'.
Scheduler: fixed_policy invocations no longer bounce PreToolUse 'ask' to an
unanswerable awaiting_approval (deny stays fail-closed), and
processToolResultImages short-circuits for fixed_policy results at the
method level so all three call sites skip the omni re-delivery / vision
bridge re-entrancy.
Delivery/CLI: chain preprocessing and guard disclosures instead of replacing
(D8), exclude hidden media-policy tools from the /context per-tool breakdown
to keep it aligned with getFunctionDeclarations(), annotate fixed-only
media-policy tools in /tools, and drop the stale maxConcurrency mention from
the policyTools settings description.
Tests: add negative coverage for artifact validation (no artifacts,
undeclared media type, kind mismatch, missing required output) and for the
fixed_policy hook ask/deny and image-funnel paths.
* fix(omni): drop system-default fixedPolicies; guard is the only default
The three degradation tools were registered twice: as transportGuard
policies (design-mandated, always-on, transport-limit triggered) and as
fixedPolicies with invented when-thresholds (1568px/480p/96kbps). The
upstream design gives fixedPolicies pure user-experiment semantics — a
zero-config setup must not trigger any preprocessing below transport
limits. Remove the fixedPolicies-side registration and its threshold
constants; keep the guard defaults. Threshold acceptance for #8186 moves
to explicitly-configured fixedPolicies in E2E.
* feat(omni): add four local stage B policy tools (keyframes/extract-audio/clip/convert)
- omni_extract_keyframes: ffmpeg scene detection (select+showinfo) with
uniform-fps fallback, multi-artifact batch, per-frame disclosures
- omni_extract_audio: audio track extraction to WAV/MP3/M4A, 16kHz mono
WAV default (ASR-recommended shape)
- omni_clip_video: frame-accurate time-axis cut (input-side -ss/-t,
libx264 re-encode, faststart), rejects no-op invocations
- omni_convert_image: sharp re-encode to JPEG/PNG/WEBP with EXIF
orientation baked in, animated inputs refused
- uniform lossy declaration per mapping doc §6.1: every output carries a
disclosure since the representation change itself is lossy
- extract shared sharp loader (sharp-module.ts) and describeChannels
helper; register the four tools lazily behind isOmniEnabled()
* feat(omni): add omni_transcribe_audio with transcript delivery protocol
* feat(omni): fill request/session condition namespaces and wire client origin
The when-condition evaluator has supported the request.* and session.*
namespaces since the initial S4 landing, but no caller populated them, so
every policy referencing those fields resolved to 'unavailable'. This
fills both per policy design section 8.3:
- request.totalEstimatedMediaTokens is computed inside the orchestrator
from the pending-delivery work-item set at each pass start, so it is
recomputed as derivatives enter later passes. If any pending resource
is unestimable the whole namespace reads unavailable - a partial sum
must never pass thresholds as a smaller total. Callers can override
via options.conditionContext.request (future multi-root aggregation).
- session.* is snapshotted once per media delivery by the new
policy/session-context.ts (contextWindowTokens from
contentGeneratorConfig.contextWindowSize, promptTokenCount from the
CURRENT chat's getLastPromptTokenCount, availableContextTokens as a
clamped subtraction only when both inputs are known) and reused across
preprocessing and every transport-guard pass of that delivery.
Also wires the client execution origin: slash commands scheduling a tool
(schedule_tool in useGeminiStream) now stamp executionOrigin
{kind:'client'}, putting the in-process client channel under the same
media-policy modelAccess gate as model calls.
* fix(omni): deliver every media deliverable of multi-output fixed policies
A fixed policy producing multiple media deliverables (omni_extract_keyframes
promoting N frames) previously failed delivery with "exactly one is
supported" after all frames were already atomically promoted to objects/ —
zero frames reached the model, violating the multi-output transaction
visibility acceptance.
Fixed-side contract now: deliveries[0] is the primary (unchanged path:
guard re-check, upload, disclosure); the rest ride along as
OmniMediaDelivery.additionalMedia, each independently judged against the
transport limits and uploaded through the same hash → upload-cache →
objects promotion → DashScope pipeline. Extras are processed lazily after
the primary settles, so the primary uploads first and a fail-closed
primary never wastes extra uploads. An over-limit extra becomes an
explicit omission entry (visible reason, no guard-policy re-derivation —
it is already a policy product). The guard side keeps its strict 1:1
semantics.
All three delivery consumers (readMediaViaOmniDelivery, tool-result
media conversion, the @-command URL funnel) materialize extras through a
shared buildAdditionalMediaParts helper: per-extra [disclosure?,
fileData | omission text] pairs after the primary slot, before
transcripts, preserving disclosure adjacency per pair.
* fix(core): detect mkv/avi/flac/aac media missing from mime/lite standard db
mime/lite's default database carries no mapping for .mkv/.avi/.flac/.aac,
so detectFileType fell through to the binary content sampler and a real
Matroska movie was rejected by the 10MB inline cap instead of entering
media delivery. Generalize the existing .m4v override map to cover these
containers (authoritative types from the full mime db), scoped to
extensions the omni recognizer can sniff-confirm so extension lies still
fall back to the legacy path.
* fix(omni): harden policy pipeline per self-review correctness findings
- Validate operator-configured defaultArguments/lockedArguments against a
tool's NATIVE parameterSchema instead of the model-visible projection,
which strips exactly those keys and rejected every legitimate locked or
operator-only argument at startup; regression test uses real tools.
- Write cache files with noFollow so the atomic rename replaces a symlink
planted at the cache path instead of writing (and chmod-ing) through it.
- Route every .part staging write through prepareOmniDownloadsDir, a
fail-closed symlink guard (mkdir recursive succeeds silently on a link).
- Detect ADTS AAC in the MPEG-audio sniffer (layer bits 00) so .aac
streams stop being labeled audio/mpeg.
- Preserve metadata.omniRole in degradation-cache entries so cached
reruns keep the same artifact matching as fresh derivations.
- Fall back to the default quarantine retention when the configured value
is zero/negative/NaN instead of expiring the whole quarantine.
- Propagate error causes through OmniTransportGuardError and document the
operatorOnlyParams contract on MediaPolicyToolDescriptor.
* refactor(omni): consolidate media-policy tool plumbing
- Single-source tool names from ToolNames across all eight policy tools
and the config validator.
- Share the config view on the base class (protected configView) instead
of eight shadowed per-subclass fields; move the view type and the
isPlainRecord helper to policy/types.
- Memoize the model-visible schema projection on the settings object's
identity (one normalized settings object per initialize()).
- Extract sharpTimeoutSeconds for the duplicated libvips timeout
conversion and route clip-video's io validation through the base
validateToolParamValues.
* refactor(omni): share the transcript materializer across delivery consumers
Extract buildTranscriptParts (§6.2/D8 ordering: each transcript follows
its media part or omission notice, preceded by its own disclosure) and
use it from readMediaViaOmniDelivery, the tool-result funnel, and the
CLI @-command URL funnel instead of three hand-rolled loops. Also
extract the textOnlyDelivery helper, upload additional-media extras
concurrently (content-addressed store and per-file serialized upload
cache make them independent; map preserves deliverable order), and drop
the now-consumerless formatTranscriptText barrel export.
* perf(omni): parallelize orchestrator validation and promotion
Validate a batch's artifacts and promote them into the object store
concurrently — they are independent files in a shared staging dir and
the store is content-addressed with race-safe puts; map keeps the batch
order and a sequential partition preserves derived/derivedFiles order.
File artifacts hash the bytes already in memory for the UTF-8 check
instead of streaming the file a second time.
* fix(omni): add display-name translations for the media-policy tools
The eight omni media-policy tools landed without toolDisplayName locale
entries (cli zh/zh-TW/en) or web-shell wire-name display mappings and
toolName translations, tripping both display-name drift guards
(cli src/i18n/index.test.ts and web-shell toolFormatting tests).
* fix(omni): open the download .part fd before the pipeline starts
createWriteStream opens lazily, so a pipeline failure on an
all-synchronous body (the byte cap tripping on the first chunks) could
reject — and reach the outer .part cleanup — before the file was even
created, resurrecting the .part after its rm. Await the open event
before streaming so creation always precedes cleanup.
* feat(omni): switch when-condition DSL to expression arrays
Replace the {left, operator, right} object form with Mapbox-style
expression arrays: a comparison is [op, operand, operand] with op one of
> >= < <= == !=, an operand is a ["field", "<namespace.name>"]
reference or a bare literal, and combinators nest as ["all", ...],
["any", ...], and the new ["!", expr] negation. The array form is
shorter to write, nests without ceremony, and leaves room for future
operators.
Semantics are unchanged: three-valued evaluation with strong Kleene
combinators, unavailable fields never silently collapse to false, and
negation passes unavailable through rather than laundering unknowns.
Startup validation rejects the retired object form with a pointed
migration hint instead of a generic type error.
* feat(omni): degrade delivered media and retry when the server rejects input as over-limit
The server's real input ceiling is unknowable locally (frame-based video
billing is invisible to the byte/duration estimator), so a 400 like
DashScope's 'Range of input length should be [1, 196608]' previously
aborted the session even though history compression cannot shrink media
tokens.
Now, before reactive compression, the chat loop reverse-maps each
delivered oss:// URL through the upload cache to its stored object,
re-runs the configured transport-guard policy with an escalating
argument ladder (video fps 2 -> 0.5 -> 0.25 is the only lever that
reduces frame-billed tokens), re-uploads the derivative, swaps the
history parts in place with a fresh disclosure, and retries. Bounded by
limits.maxTransportPasses; best-effort throughout - any failure falls
through to the existing compression/error paths.
Also: parse the DashScope input-range upper bound as limitTokens, allow
fractional fps in omni_downscale_video, and add a scope-agnostic
findSha256ByUrl reverse lookup to the upload cache.
* feat(omni): inject progressive media understanding guidance into the system prompt
Disclosure parts state WHAT a delivery lost, but nothing told the model
WHY: degradation is a context-budget-driven progressive-understanding
strategy, the degraded delivery is an overview rather than the complete
content, and fuller evidence can be fetched on demand. Without that
contract the model treats a 600s clip + opening keyframes as the whole
film and extrapolates.
- omni/media-guidance.ts: builds a stable system-prompt section that
explains the three disclosure markers, frames degraded deliveries as
progressive-understanding overviews (never conclude undelivered
content doesn't exist), and directs proactive targeted evidence
fetching via exactly the modelAccess-enabled media policy tools; with
none enabled it instructs stating missing evidence instead of
extrapolating. Returns null when omni delivery is inactive.
- prompts.ts: new stable SystemPromptLayers slot mediaGuidance, placed
right after base so it stays inside the cached static prefix.
- client.ts: wire the section into the main-session stable layers.
- omni/delivery-gate.ts: isOmniDeliveryActive moved out of omni/index.ts
into a leaf module so prompt assembly doesn't statically pull the
whole delivery pipeline; index.ts re-exports it unchanged.
* fix(omni): extract keyframes across the full video duration (bucketed sampling)
The scene-detection pass stopped at the first maxFrames scene changes,
so every keyframe of a long video landed in its opening minutes (an
81-minute film yielded 16 frames all within 0-130s). Split the timeline
into maxFrames equal buckets instead: each bucket contributes one frame
- a scene change from its opening window (input seek, 30s search cap)
when one exists, the bucket midpoint otherwise. Absolute timestamps are
reconstructed as bucketStart + showinfo pts_time. Individual bucket
failures are tolerated and all runs share one wall-clock budget. The
single-pass path remains for unknown duration or maxFrames = 1; the
now-unreachable uniform-sampling fallback is removed.
* fix(omni): chunk long-audio transcription and collapse repetition degeneration
A single ASR request over long audio truncates well before the end and
degenerates into repetition loops (an 81-min film came back as 2114
chars ending in 48 copies of "Hej!"). Audio longer than chunkSeconds
(default 180s) is now split into equal segments, re-encoded to 16kHz
mono AAC, transcribed with bounded concurrency under one shared
wall-clock budget, and assembled into [MM:SS-MM:SS]-labeled lines.
Per-segment failures become inline markers instead of failing the run;
the run only errors when every segment failed. Repetition degeneration
is detected and collapsed in every transcript (chunked and single-shot)
and reported in the disclosure.
* docs(omni): add media policy orchestration usage guide
* fix(omni): key pure-transcript guard resolution on this pass's file deliveries
A transport-guard pass that omitted the source without producing any
deliverable used to short-circuit into textOnlyDelivery whenever an
EARLIER pass had already collected a transcript, silently dropping the
zero-deliverable error. Key the branch on the current pass's
fileDeliveries instead of the cumulative transcript list.
* fix(omni): render the primary disclosure on pure-transcript deliveries
All three delivery consumers (readMediaViaOmniDelivery, tool-result
media, @-command processor) dropped delivery.disclosure whenever the
primary media degraded all the way to a transcript-only delivery. The
disclosure chains every prior lossy step (decision D8) and the
transcript was derived through those steps, so it must still render —
as the leading text part of the delivery.
* fix(omni): prune non-object entry values when loading a JSON cache file
The load-time shape check validated the top-level {version, entries}
envelope but trusted every entry VALUE, so a crafted workspace cache
holding e.g. `"key": null` surfaced as TypeErrors from field accessors
in whole-file scans (removeByDegradedSha256 touches every entry, not
just the requested key). Drop malformed values at load and log the
count; the next put() rebuilds them from verified data.
* fix(omni): harden degradation-cache hits against tampered entries
Two gaps in the workspace-shippable cache's trust boundary:
- disclosure had no length ceiling, leaving an unbounded
prompt-stuffing channel that renders into the conversation on every
cache hit. Entries with a disclosure over 2048 chars are now dropped
like any other malformed field.
- the hit path verified the derivative's hash but not its TYPE: a
swapped extension/mimeType could deliver a media kind the producing
tool never declared. The orchestrator now cross-checks the recognized
MIME type against the tool descriptor's declared media outputs and
re-transcodes on mismatch, dropping the entry.
* fix(omni): include the hours field when a rounded duration reaches 3600s
withHours compared the raw duration while formatClock rounds each
boundary: a 3599.6s audio rounded to 3600s inside the label and, with
hours disabled, rendered as "00:00". Round the duration the same way
formatClock does before deciding the format.
* fix(omni): cap chunked transcription at 512 segments (fail closed)
The claimed duration comes from container metadata, which a crafted
file controls freely: an absurd duration used to fan out into hundreds
of thousands of outcome slots and queued ffmpeg cuts before the time
budget could intervene. Reject anything over 512 segments — at the 30s
chunkSeconds floor that is ~4h16m of audio, far past what the 10MiB
default input cap plausibly holds.
* fix(omni): disable the omni pipeline in bare mode
isOmniEnabled ignored bareMode, so --bare sessions still registered the
media-policy tools, ran content normalization and the ffmpeg startup
assert, and gated deliveries. Bare mode now wins over the env opt-in at
the single choke point every omni entry path already consults.
* fix(omni): degrade media nested in functionResponse.parts on reactive retry
The reactive over-limit fallback only walked top-level content parts,
so media delivered inside a tool result's functionResponse.parts (the
normal carrier for tool media) was invisible to it: the retry loop
resent an unchanged request and stalled. Collection and replacement now
descend one level into functionResponse.parts, and the swap rebuilds
the nested array in place — the disclosure must land immediately before
its media part (D8), which hoisting to the top level would break.
* fix(omni): detect animated images whose headers carry no frame count
Animated WebP/APNG containers report no nb_frames, so the D9 animation
exclusion silently passed them and the image tools kept only the first
frame with no disclosure. Two independent gates now close this:
- the ffprobe path re-probes animation-capable containers
(gif/webp/png/apng) with -count_frames when nb_frames is absent
- both image tools additionally check sharp's metadata().pages before
encoding and refuse multi-page inputs
* docs(omni): align tunable descriptions with the implemented semantics
- maxWidth/maxHeight are same-value aliases of width/height, not
independent axis caps
- request.totalEstimatedMediaTokens is the per-resource estimate
(resource plus its derivatives) within the current scheduling pass,
not a cross-resource total
* fix(omni): keep a raw "__proto__" policy id as an ordinary own key
A JSON settings file can carry "__proto__" as an ordinary key; spreading
it into a plain object routes it through the prototype setter and the
entry silently vanishes. Merge policy maps onto a null-prototype object
so the id reaches normalizePolicy and is rejected by the id pattern.
* fix(omni): validate transportGuard.maxEstimatedTokens at startup
Settings load performs no runtime type checks and guard.ts compares the
threshold with <=/>: a string value makes both comparisons false and
silently disables the token guard (fail-open). Reject anything but a
finite number >= 0 at normalization, and thread the value through
Config so the check actually sees it.
* fix(omni): raise quarantine retention/budget schema minimums to 1
The runtime accessors treat non-positive values as absent and fall back
to the defaults, so a configured 0 never means "delete immediately" /
"no budget" — advertising minimum: 0 in the IDE schema promised a
semantic the code does not honor. Document the fallback in the
descriptions as well.
* docs(omni): correct the fixedPolicies merge/tombstone description
There are no built-in default fixed policies, so "null tombstones a
default policy" described a merge target that does not exist. Describe
what the merge actually does: entries merge by id across settings
scopes and a null entry tombstones a policy from a lower-priority
scope.
* fix(omni): expose io paths to the AUTO-mode permission classifier
Media policy tools override getDefaultPermission but left
toAutoClassifierInput at the empty-string sentinel, so the AUTO-mode
classifier saw "Arguments: {}" and its path-based block rules could
never fire on a model-origin call. Project exactly inputPath and
outputDir — the fields those rules key on, carrying no secrets.
* fix(omni): reject unknown keys in policyTools entries (§13 #1)
A typo like "settigns" or "modelaccess" read as an absent optional
section downstream, so the intended configuration silently never took
effect. Fail startup on unknown keys at the entry, runtime, and
modelAccess levels, and drop the stale maxConcurrency mention from the
runtime doc comment (timeoutMs is the only supported limit).
* fix(omni): reject parameterSchema overrides that loosen native constraints
Design §11.2: a projection may only narrow the tool's native schema —
类型、枚举、范围 included, not just the property set. The declaration
merges each projected property's keys over the native ones, so an
override raising maximum, lowering minimum, widening an enum, changing
the type, or relaxing min/max length/items promised the model a range
the native per-call validation then rejects. Validate the merged
property schema against the native effective bounds at startup
(minimum/exclusiveMinimum interplay included).
* fix(omni): clamp downsample-audio targets to the probed source
A source already below the target bit rate, sample rate, or channel
count re-encoded into a LARGER derivative that the transport guard
counted as progress, under a disclosure claiming 高频细节丢失/声道合并
losses that never happened. Clamp each target to the probed source
(the withoutEnlargement analogue of the image/video tools) and build
the loss clause from the drops that actually occurred (D8).
* fix(omni): reject full-span no-op clips instead of re-encoding
clip_video is a time-axis cut, not a degradation tool: a span covering
the entire video only burns a lossy re-encode while the disclosure
falsely claims content outside the span was discarded. Reject
startSec 0 with no durationSec at the parameter layer, and a
probed-duration-covering span before the transcode starts.
* fix(omni): disclose partial bucket coverage in keyframe extraction
When some buckets yield no frame (scene attempt and midpoint fallback
both empty), the blanket 全片分桶采样 note overstated coverage — the
un-sampled buckets' time ranges were silently invisible to the model.
Disclose 仅覆盖 N/M 个分桶，其余时段未采样 whenever fewer frames than
buckets survive (D8).
* fix(omni): only claim alpha loss for alpha-capable JPEG conversion sources
A JPEG→JPEG re-encode cannot drop a transparency channel that never
existed, yet the disclosure asserted 透明通道丢弃 unconditionally.
Derive the loss clause from the probed input codec: alpha-capable
formats (png/webp/gif/tiff) keep the claim, others get 元数据丢弃, and
an unknown codec hedges with 如有 (D8).
* fix(omni): build the downscale-video loss clause from actual drops
A 360p@8fps input downscaled against the 480p/10fps defaults lowers
neither dimension, yet the disclosure always claimed
分辨率与帧率下降 — contradicting the before/after figures shown right
next to it. Emit 分辨率下降/帧率下降 only for the parameters that
actually dropped, falling back to 重新编码压缩 for a pure re-encode
(D8).
* fix(omni): stop calling a degraded delivery "full resolution" in the zoom hint
On the degradation path delivery.recognized re-recognizes the
DERIVATIVE, so the hint's width/height are the downsampled dimensions.
Calling them "full resolution" contradicted the degradation disclosure
pushed immediately below and steered the model away from zoom_image —
the exact remedy for degradation-stripped detail, since it reads the
original from disk.
* fix(omni): localize the fixed-only tool description suffix
The '[fixed-only: runs via media policies, not the model]' suffix was
appended as a hard-coded English string, so zh/zh-TW tool listings
showed a mixed-language description. Route it through the i18n table
like the display names it accompanies.
* fix(omni): unify the downscale-video display name as 降采样视频
降清视频 was a coined term inconsistent with the 降采样图像/降采样音频
siblings; use 降采样视频 (zh-TW: 降採樣影片) across the CLI locales and
the web-shell tool-name table.
* test(omni): match the actual .tmp suffix in cache-file cleanup assertions
The write-temp files are named <name>.<random>.tmp; filtering on
'.tmp-' could never match anything, so the leak assertions were
vacuously green. Filter on the real endsWith('.tmp') suffix.
* test(omni): pin the repetition-collapse rep threshold independently of the span floor
The 好的， unit (3 chars × 7 reps = 21 chars) sat below the 24-char span
floor, so the test passed for the wrong reason and REPETITION_MIN_REPS
itself was unpinned. Use a 6-char unit so 7 reps already clear the span
floor, and assert the collapse output at 8 reps.
* fix(omni): cascade corrupt-object deletion into the degradation cache
Startup recovery's sample verification deleted a corrupt object and its
upload-cache entry but left policy-cache.json untouched, accumulating
entries that can never be served again. Cascade the deletion both ways
— the object may be a policy SOURCE (its derivatives can no longer be
re-verified) or a policy DERIVATIVE (entries pointing at it are dead) —
mirroring the storage design's upload-cache cascade. The orchestrator's
lazy self-heal only fires on a cache hit; the sweep side is proactive.
* refactor(omni): resolve role selectors via a role→output-spec map
The role existence check consulted a Set while the transcript shape
check re-scanned descriptor.outputs with a find, duplicating the
first-declaration-wins semantics across two structures. A single
Map<role, spec> now serves both.
* test(omni): pin the when-condition round-trip through normalization (D7)
when is preprocessing's ONLY trigger mechanism: a normalization
regression that drops or rewrites it would silently widen every user
condition to ALL matching resources, with no failing test. Pin that a
valid condition tree survives normalization verbatim.
* test(omni): pin the runtime.timeoutMs / staging-grace boundary (recovery §5)
The timeoutMs < STAGING_GRACE_MS cap is a cross-file invariant: a tool
allowed to run for the full grace window could have its live staging
directory swept as crash leftovers mid-run by another process's startup
recovery. Pin BOTH sides of the boundary against the exported constant
so removing, inverting, or relocating the cap fails a test.
* feat(omni): S5 multimodal media memory — two-trigger collection, root-bounded recall, active/sideQuery surfaces (#8949)
* feat(omni): collect multimodal media memory at recognition and policy-success points
Add the media-memory service (packages/core/src/services/media-memory):
a persistent, file-scoped lineage graph over .qwen/omni/memory.json that
records recognized files/versions and policy executions with their
derived media and text outputs.
Collection side (Stage A of the multimodal memory design):
- store.ts: single-JSON v1 backend with atomic writes, corrupt-document
backup/rebuild, and read/transact serialization shared with the omni
JSON cache chain.
- service.ts: recordFileRecognized (idempotent file+version upsert,
CURRENT_VERSION follows disk both ways) and commitPolicySucceeded
(one atomic commit per succeeded policy execution: execution record,
derived versions with DERIVED_FROM/PRODUCED_BY lineage, and
derived_media/policy_result entries with conservative channel and
coverage derivations). executionId is a content-identity key
(source sha256 + policy fingerprint), so replays and degradation-cache
hits converge on the same execution node. findBindingBySha256 gives
the reactive ladder a read-only identity lookup.
- config.ts: omni.memory normalization with fail-fast validation
(unknown keys, budgets, kinds, cross-field ordering).
Wire both trigger points: processMediaForOmniDelivery records the
source file after recognition (hashing the source upfront and threading
sha256 through the policy pipeline), and executePolicy commits after
object promotion on both the success and degradation-cache-hit paths.
Persistence failures are logged and never block delivery; a policy
commit itself stays all-or-nothing.
* fix(omni): label keyframe and clip artifacts with their roles
omni_extract_keyframes and omni_clip_video emitted artifacts without
metadata.omniRole, so role consumers could not tell an excerpt from a
complete derivative: output routing selectors (role:keyframe /
role:clip) never matched, and media-memory coverage derivation fell
through to 'complete' for sampled keyframes and temporal clips.
Label them 'keyframe' and 'clip', matching the transcript labeling
convention.
* feat(omni): add session resource registry and memory recall service
Stage B read side of multimodal media memory (design M §9):
- MediaResourceRegistry: session-scoped opaque handle binder
(fileVersionId <-> resourceId, M §5.2) so recall payloads never
carry real paths or stable identifiers.
- MediaMemoryRecallService: current-version-first recall (§9.5) over
the rootFileId-bounded derivation graph (§8), returning the minimal
protocol shape (§9.4) with synthesized metadata/execution entries,
honest gaps (not_processed / partial_coverage / artifact_unavailable,
D5), and advisor-driven nextPolicyActions. Unknown or empty resource
requests reject as a whole (§9.2); an unreadable store degrades to
a plain miss.
* feat(omni): mount session media resource registry on delivery
Hang a lazy MediaResourceRegistry off Config and bind every
memory-known resource the delivery pipeline puts in front of the
model — the source at recognition, each preprocessing deliverable,
and the transport-guard replacement — so recall can rebind opaque
session handles back to persistent memory identities without ever
exposing a path.
* feat(omni): add active-mode media memory recall tool and session handle disclosure
Stage B recall surface #1 (memory design M §9, D10 mutual exclusion):
- omni_recall_media_memory tool, registered only when omni is enabled AND
omni.memory.recall.mode === 'active'. Read-only (D11): consults the
persistent store through MediaMemoryRecallService and binds session
handles for returned derived artifacts; whole-request rejection maps to
invalid_tool_params so the model can correct and retry.
- Session resource handles are now disclosed to the model: delivery
carries the SOURCE binding's resourceId, and both consumers (file reads
and the tool-result funnel) lead the part group with a 【媒体资源】
annotation — the handle stands in for the path the model never sees
(M §5.2), while disclosures keep their D8 adjacency to the media part.
- Advisor wiring: recall gaps suggest nextPolicyActions only for tools
that are both registered and opened via modelAccess.enabled — recall
never steers the model into calls the media-policy gate would reject.
- omni.memory normalization moved to first use (ensureOmniMemoryConfig):
createToolRegistry needs recall.mode before initialize()'s omni block
runs; invalid settings stay startup-fatal on both paths.
* feat(omni): resolve resourceId to inputPath at the media-policy call gate
Stage B evidence-gathering path (memory design M §5.2): a gated
model/client call of a media-policy tool may now name its source by the
opaque session resourceId announced at delivery or returned by recall,
instead of a filesystem path the model must never see.
- evaluateMediaPolicyToolCall resolves the handle through the session
registry BEFORE the lockedArguments check (a resolved inputPath cannot
sidestep an operator-pinned input); unknown/fabricated handles and
inputPath+resourceId together reject as invalid_params; fixed_policy
origins remain untouched (RESERVED_ARGUMENT_KEYS already bans the key
in policy arguments).
- Shared io schema advertises resourceId as the alternative to
inputPath; required drops inputPath (presence is enforced in
validateMediaPolicyIoParams AFTER gate resolution, with an actionable
either-or message).
- assertMediaPolicyIo input errors now name only the file's basename:
those messages reach the model, and a handle-resolved call must not
leak the locator the handle stands in for — basename matches the
displayName the model already saw at delivery (D5: a deleted source
errors without disclosing its path).
* feat(omni): passive sideQuery media-memory recall before the main request
Stage B recall surface #2 (memory design M §9.3, D10 sideQuery mode):
- MediaMemoryRecallService grows the two sideQuery faces on the same
root-bounded walk recall() uses: candidateSummaries() builds the
bounded selector manifest (structure + capped preview, never raw
media/full text/paths; maxCandidateEntries cap, deterministic order),
and recallSelection() materializes a selection through the unified
protocol — an unknown/over-budget entryId rejects the WHOLE selection
(invalid_selection), never a partial fulfilment.
- runOmniMemorySideQuery orchestrates one passive pass: handles are
parsed from the 【媒体资源】 annotations the request itself carries
(never a project-wide scan), the selector runs on the runSideQuery
JSON face (entryIds only, validate enforces manifest membership and
maxSelectedEntries), and every failure — timeout, selector error,
rejected selection — degrades to an empty recall with a recorded
reason while the main request proceeds.
- client.ts injects the materialized recall as a system reminder on
UserQuery/Cron turns strictly BEFORE the main request is sent (M
§9.3: never retrofitted into a later turn); latency is bounded by
sideQuery.timeoutMs.
- disclosure.ts gains parseResourceHandleText, keying on the
harness-minted handle grammar so displayNames containing the
separator cannot confuse extraction.
* fix(omni): propagate the recall tool display name to every drift-guarded surface
A new core tool name must land in four places, not one: core
ToolNames/ToolDisplayNames, the cli i18n dictionaries, and the web-shell
display-name map plus its own zh dictionary. Two drift tests guard this
(cli i18n/index.test.ts and web-shell toolFormatting) and both were red.
* fix(omni): reject unknown omni.memory root keys, stop advising on deleted sources, keep the selector's view of the question
Three independent correctness fixes in the memory slice:
- Config normalization rejected unknown keys at every nested level but
not at the root, so `{recalll: {...}}` silently discarded the entire
configuration and ran default active mode while the operator believed
sideQuery was configured — the exact silent fallback this module's
startup-fatal stance exists to prevent.
- gapsForVersion emitted `artifact_unavailable` for a deleted source and
then CONTINUED the channel scan, adding `not_processed` siblings. The
advisor only filters `artifact_unavailable`, so recall steered the
model into evidence-gathering calls on a file that no longer exists —
the gate resolves the handle and the io assertion then fails. Nothing
can be gathered from a deleted file: report the loss and stop.
- The sideQuery selector dropped any text part starting with
`<system-reminder>`, but IDE context is PREPENDED INTO the user's own
part before the passive pass runs — so in IDE mode the selector never
saw the question and picked relevance-blind. Strip reminder blocks
instead of discarding the part.
* fix(omni): bump clip/keyframe descriptor versions alongside their role annotations
The role labels these tools gained are what memory maps to honest
coverage (`clip` → partial, `keyframe` → sampled); role-less outputs
derive `complete`. The descriptor version feeds the policy fingerprint,
which keys both the degradation cache and the content-identity execution
id — so leaving it at '1' let pre-role cache entries and recorded
executions converge onto the post-change fingerprint two ways: a cache
hit recommits with the persisted `role: undefined`, and the replay
branch returns without upgrading recorded entries.
Either path makes recall report a temporal excerpt or a sampled frame
set as complete coverage: the model then answers about footage it never
saw and the advisor never proposes closing the gap. Bumping the version
partitions pre- and post-annotation state instead.
* fix(omni): prune malformed memory record values instead of blacking out every read
`load()` validated the document envelope but not the record values inside
it. A single non-object value — one hand edit, bad merge, or truncated
sync — made every read path throw (`indexSnapshot` dereferences
`version.parentVersionId`, `findBindingBySha256` reads `version.sha256`),
and those throws are caught into miss/empty by the never-fatal read
wrappers. The result was a PERMANENT global recall blackout for the whole
project, with no `.corrupt-*` backup and nothing named in the debug log,
because the envelope itself stayed valid.
Prune the bad values individually and name them, matching the sibling
`OmniJsonCacheFile.load()` defense. Dangling references to pruned records
already degrade gracefully — a missing version reads as an
`artifact_unavailable` gap.
* fix(omni): give every file its own policy execution and derivative rows
Two Files with identical bytes shared one execution node: the key was
`sha256 ⊕ configHash`, so the second file took the replay branch and got
ZERO records of its own, while the first file's derivative versions were
handed back stamped with the second file's rootFileId. Recall on the
second file then reported false `not_processed` gaps across every channel
and re-paid the work, and the derivative's lineage pointed into a
different root's tree. M §11.2/§11.3 prescribe the opposite: identical
bytes never merge Files, each File writes its own PolicyExecution and
provenance, and only the computation and the stored bytes are reused.
Three layers, cleanly separated:
- execution identity is now per-File (source VERSION ⊕ config), so a
replay is a genuine same-file replay and stays idempotent;
- the content-identity reuse key (`sha256 ⊕ config`) is derived
separately and spans Files — that is what makes cross-file reuse
possible while provenance stays apart. A match records
`reusedExecutionId` (declared since the schema landed, never assigned
until now) pointing at the original;
- derivative File nodes are keyed by (root, object path) instead of the
object path alone, so each root owns cheap metadata rows over the SAME
content-addressed object rather than sharing a node whose root belongs
to whoever derived it first — bounded traversal (M §8) holds for both.
* fix(omni): anchor tool-result media to the object store, not its staging file
Collection recorded the source's persistent identity as the path it was
handed — correct for a user file, whose bytes stay in place (S §4), but
wrong for tool-result media: that path is a staging `.part` the funnel
deletes in its `finally` the same turn, while this same delivery promotes
the identical bytes into the content-addressed object store.
So the disclosed handle resolved to a file that no longer existed: any
media-policy tool the model pointed at it failed with ENOENT, and recall
reported `artifact_unavailable` across every channel for an artifact that
actually persists. Tool-origin sources now record (and bind) the
content-addressed object location, which is derivable from the hash
before promotion runs. If promotion never happens — the transport guard
omitted the media — the ref dangles and recall says
`artifact_unavailable`: honest, because the bytes were not retained.
* feat(omni): commit model and client policy successes through the same memory boundary
Memory design M §7.1 requires fixed calls, model tool calls and direct
client calls to enter ONE `OmniPolicySucceeded` boundary, and §17 names
both capture sites — the core scheduler and ACP's own executor. Only the
fixed-policy orchestrator ever committed: the scheduler built the
`PolicyArtifactBatch` (with the real origin) and nothing consumed it, and
the "model-call artifact bridge" its comment referred to did not exist.
So evidence gathering never accumulated. Recall reported a gap, the
advisor suggested a tool, the model called it with a session handle, the
tool succeeded and wrote real artifacts — and the next recall reported the
identical gap, re-paying the work every session, forever. That is exactly
the loop the resourceId path in this PR was built to close.
The bridge walks the same gates in the same order as the orchestrator:
validate each artifact against the descriptor (contained in the output
directory the call declared), require every declared output, promote to
the content-addressed store, THEN commit (M §6.4 / S §5 — no record may
reference bytes that are not yet in `objects/`). A handle-driven call
recovers its source identity from the session registry instead of
re-hashing the input; a raw-path call falls back to an idempotent
recognition upsert. Fixed-policy batches are skipped: the orchestrator
already owns them.
`resourceId` also joins the fingerprint's excluded keys. Like
`inputPath`, it is per-invocation plumbing — and it is minted fresh every
session, so leaving it in would make each session re-derive identical
work.
* feat(omni): reuse recorded policy outputs instead of re-running the tool
The degradation cache maps one input to ONE media derivative, so its own
comment admits that multi-output tools and text products are "simply not
cached — re-run instead of guessing". That made #8189's «同文件同 settings
二次触发同一 policy：直接复用，无重复执行» unmet in the cases that cost the
most: re-delivering an 81-minute movie re-ran the whole ASR pass every
time, because a transcript is a text product.
Memory already records every output of every execution, keyed by content
identity — so it can answer the reuse question the cache cannot. The
orchestrator now consults it BEFORE the cache and, on a hit, rebuilds the
deliverables from the recorded objects and skips execution entirely,
covering media, text and multi-output alike.
Reuse is verified exactly like a cache hit, because memory.json is
project-local and hand-editable: every object must still be a regular
file, must still hash to its recorded identity, and a media object must
re-recognize as a type the tool DECLARES producing — otherwise a crafted
record could route arbitrary store content through a policy that never
made it. Text is re-read from the promoted object rather than from the
entry's `inlineText`, which is truncated to
`collection.maxInlineTextBytes` and must never be delivered as the whole
transcript. Any doubt falls through to a fresh derivation.
The reusing file still commits its own execution, stamped with
`reusedExecutionId` (M §11.3): the computation is shared, the provenance
is not.
* docs(omni): reconcile the memory design with the S5 issue acceptance wording
Two acceptance criteria in #8188 read as direct contradictions of this
document, and one of them contradicts #8189 as well. Both are resolved by
naming the layer each statement belongs to, rather than by changing the
model.
- §11.2.1 (new): «内容相同的两个文件不重复建节点» and «不合并不同文件»
live on different layers. Content-keyed reuse covers the physical object
and the derived computation; the File node stays locator-keyed because
identity MUST be stable across content changes — key it on content and
§11.1's version chain and #8189's «修改内容后旧衍生物不默认召回» both
become inexpressible (an edit yields an unrelated node, and editing back
makes history a cycle). Also records why derivatives are keyed by
(root, object path).
- §9.2.1 (new): «按路径/内容 hash 查询» and «返回中永不暴露真实本地路径»
are reconciled per caller — the model gets session handles only
(unforgeable capability; a path parameter would let injected media steer
it into probing other files, whose transcripts recall returns), while
path/hash lookup stays a harness capability that already exists. The
known cost is written down too: a deleted file's memory is currently
unreachable, and the fix belongs in the authorization layer
(@-reference re-anchoring), not in widening what the model may pass.
The issue bodies were updated to match.
* fix(omni): reuse text products too, and fail on unknown omni settings sections
Both found by running the real pipeline rather than the unit suites.
Text-product reuse never fired. `findReusableOutputs` only resolves an
object path for outputs that have a derived VERSION node, which text
products do not have — so `rebuildReusedOutputs` bailed and the tool
re-ran. The comment there even said the caller reconstructs the path from
the content hash; that half was never written. A real run made it obvious:
extract-audio and keyframes logged reuse hits while the transcript re-ran
the entire ASR pass — the single most expensive thing #8189 exists to
avoid, and the headline claim of the reuse commit. The object store is
content-addressed, so the path is derivable; text outputs now reuse like
media. The existing test passed because it used a media artifact, so a
transcript-shaped case was added alongside it.
Unknown sections under `omni` were silently ignored. The settings loader
scans TOP-LEVEL keys only, and only to a debug line, so a nested typo was
caught by nothing: `omni.memoryy.recall.mode = sideQuery` started
cleanly and registered the active-mode recall tool — the operator's whole
configuration discarded without a word. The same hole swallowed
`omni.processingg`, which would silently disable every degradation
policy. `omni`'s own normalizers are startup-fatal by design, so the
namespace now rejects unknown sections with the allowed set named. The
permitted keys are derived from the settings schema, not hardcoded, so
they cannot drift.
* fix(omni): stop policy artifacts colliding, guard duration, record real execution windows
Three defects a real multi-session run surfaced. The first destroyed work.
**Fixed output filenames collided.** Every policy tool wrote a constant
name (`clip.mp4`, `downsampled.jpg`, `transcript.txt`). Under
fixed-policy orchestration that is safe — each invocation gets its own
staging directory — but `modelAccess` lets a caller pick a PERSISTENT
`outputDir`, and there the second call silently destroyed the first
artifact. In the run: a clip cut on day one was overwritten by a different
clip on day three, and the commentary written against the first clip began
describing the wrong footage. Worse, the model noticed something was off,
could not diagnose it, and skipped re-watching — so the defect surfaced as
degraded output quality rather than an error.
Names now carry the axes that distinguish artifacts: the source stem, plus
a natural variant where the operation has one (a clip's span, a frame's
index). Same source and variant still resolve to one name, so re-running
an operation replaces its own output — idempotent, not destructive. The
keyframe lister derives its matcher from the same template, so it can
never pick up a sibling video's frames out of a shared directory.
**The transport guard had no duration dimension.** A 98-minute film
downscaled to 474 MB cleared the 1 GiB byte ceiling, uploaded, and was
then refused by the provider for being too long: the guard reported
success, the caller got an opaque 400, and a 97-second transcode was paid
for nothing. Duration is a transport limit like the others, so
`maxDurationSeconds` now sits beside the byte and token checks with the
same semantics (unset disables it; missing metadata never rejects) and a
message that names duration and says downscaling cannot fix it.
**Model-origin executions recorded the collection window, not the work.**
`startedAt` was taken after the tool had already run, so extracting audio
from a 98-minute film was recorded as 0.6s. The scheduler already tracks
`executionStartTime` (approval wait excluded); it is now threaded through.
* test(omni): keep '+' in generated artifact stems
A derived artifact of a clip re-uses the clip's filename as its source
stem, and the clip's own variant contains '+' (`123s+40s`). Stripping it
made the derived stem drift from its parent (`clip-123s-40s-keyframe-…`).
'+' is portable on every major filesystem, so keep it: names now round-trip
through a derivation chain.
* feat(omni): re-anchor remembered media, and say when recall was truncated
Two friction points the character-analysis run surfaced.
**A remembered file's memory was unreachable once its bytes were gone.** A
handle is minted only at delivery, and delivery needs the bytes — so a
transcript or keyframe set outlived the file it described with no way back
into a session. The same wall hits an audit that does not want to
re-deliver a 2.4 GB film just to ask what work was recorded: with no
handle, recall rejects everything, and the natural next move (passing the
filename as a resourceId) is correctly refused. Observed exactly that: the
model reached for `resourceIds: ["robot-dreams.mkv"]` and got
`not issued in this session`.
An `@`-reference to a missing-but-remembered path now mints a handle from
the recorded identity — the user's reference is the same authorization a
delivery carries, so the fix belongs at the authorization layer rather
than by widening what the model may pass (design M §9.2.1). The model gets
the handle plus an explicit 【媒体缺失】 note, so it recalls instead of
trying to read. Path/locator lookup stays a harness capability.
**A truncated recall page was indistinguishable from an exhaustive one.**
An audit read 6 clips under `limit: 12` and reported "no keyframes were
ever extracted" while the store held 72 of them. The reader was honest
about what it saw — it simply had no way to know it was looking at a page.
`matchedEntries` now states the full match count, and only when the budget
actually cut something, so an exhaustive page stays as small as before
(§9.4 minimal return).
* feat(omni): tell the model what a media handle is and to recall before reprocessing
The handle annotation ships with every memory-known delivery, but nothing
in the model's context explained it: the media-guidance section listed only
the three disclosure markers, and the tool that consumes handles is
deferred, so its description — which does explain them — is not in context
until ToolSearch surfaces it. Meanwhile the same section actively tells the
model to gather missing evidence with the policy tools, i.e. to reprocess
from scratch exactly what memory already holds. The active-recall surface's
whole purpose could silently never fire.
The section now names the handle marker, states that it is the only
identity the model will ever get for that file, and says to consult recall
BEFORE reprocessing. Gated on active recall mode: under sideQuery the
harness injects recalled memory itself and the tool is not registered, so
naming it would invite a guaranteed unknown-tool error (D10).
The tool-display-name drift test now runs over every translating locale
rather than zh alone. `t()` has no cross-locale fallback, so a tool added
to one locale renders a raw English badge beside translated siblings in the
other.
* fix(omni): close the honesty and hygiene gaps found reviewing S5 memory
Fourteen findings from the PR review, verified one at a time. The ones
that could mislead a caller:
- a truncated recall entry now says so. maxTextChars (24k chars) is
smaller than the collection bound (64 KiB), so any long transcript is
cut at READ time while coverage still legitimately reports `complete`
— the model saw a prefix, no gap, and answered about audio it never
read. `contentTruncated` marks the prefix.
- a resourceId handle whose modality the tool does not accept is refused
as a parameter error instead of becoming a spawned ffmpeg that burns
the tool timeout and returns an opaque stderr tail.
- the advisor no longer suggests audio extraction against a video's
speech_text gap: that gap is still open in the very payload that
RETURNS the extracted track, so the suggestion repeated itself. Nor
does it suggest anything against `partial_coverage` — sampled evidence
stays sampled by design, so that advice could never close the gap.
- a derived version keeps the execution that actually produced it. A
second execution landing on byte-identical output was rewriting the
pointer, leaving the version naming an execution whose outputs it is
not.
- `maxAttempts` governs the client's retry loop only; a selection that
parses but names entries outside the manifest is refused without a
retry. The comment and the setting description said otherwise.
- `sideQuery.model: null` means the side-query default (the configured
fast model, falling back to the session model), not "the session's
active model" as three places claimed.
And the hygiene:
- duplicate resourceIds and repeated selector picks collapse instead of
spending the budget twice
- an empty materialization records `materialized_nothing` rather than
injecting a reminder that says nothing
- `candidateSummaries` lets a defect propagate (the caller already
degrades with a recorded reason) instead of swallowing the stack
- `kinds`/`roles` reject `[]`, which read as "restrict to nothing"
- count and millisecond settings are `integer` in the generated schema
- dropped three never-written fields; corrected the `managedId` doc,
which described an extension the key does not carry
* fix(omni): disclose the session handle for URL-delivered media too
The @-command/URL funnel records the file in memory and mints a registry
binding, then never tells the model the handle. Active recall rejects the
handle as never-issued, and the passive selector finds no handles to
consult — so a session could pay for a download, an upload, and a policy
run, and be structurally unable to recall any of it.
The withheld case is where it matters most: the model cannot see the
media, so the handle is its only remaining route to the content.
* docs(omni): record the two recall fields that keep a page honest
`contentTruncated` and `matchedEntries` both exist because a reader that
cannot tell a prefix from the whole, or a page from the set, draws
confident wrong conclusions — and both were shipped in code without the
design doc naming them.
* test(omni): pin the selector gate and the handle disclosure
Mutation probes showed both were free to break silently:
- inverting the selector's manifest-membership check kept the whole suite
green while, with the default single attempt, every passive recall
degrades to `selector_failed` — the feature dead in production. The
mock now forwards responses through `validate` the way production does,
so the happy path exercises the gate too; three tests fail on the
inversion.
- dropping `...handleParts` at any of the five replacement sites left
tool-produced or withheld media with no recall handle, and no test
cared. Each site now fails exactly one case.
Also pinned: the 4000-char selector request cap, the abort/timeout signal
composition (a Ctrl-C in the selector window must cancel it, not leave
the main request waiting out the full timeout), promptId attribution, and
the basename-only input error — whose leak-prevention was satisfied by
the old full-path message too. The manifest no-path assertion moved out
of the mock: a leak used to fail as "result was null" instead of naming
the path.
* test(omni): pin the memory invariants that mutation probes walked through
Sixteen behaviors the design doc states and the code implements, none of
which any test defended. Each was verified by breaking the source, seeing
the new test fail, and restoring it. The ones that matter most:
- gaps must speak for the CURRENT version's processing state, always, and
must be computed from the FULL subgraph. Two mutants — looping gaps over
every consulted version, and threading the request's kinds/roles filter
into the coverage scan — would have made recall report missing work that
exists, purely because the caller narrowed what it asked to see. The
existing history test could not catch either; the new fixtures invert
its direction (unprocessed history, processed current) and kill both.
- the selector-visible manifest: the 200-char description cap was never
binding (longest fixture was 184 chars), so removing it would have sent
full transcripts to the selector; and validation against the CAPPED
manifest let an entry past maxCandidateEntries be materialized though
the selector was never shown it.
- registry rebinding now pins first-binding-wins explicitly. Updating in
place would repoint a handle the model already holds; minting a second
handle for one version would break cross-call correlation.
- the store's rename-based write and newest-wins backup retention, pinned
black-box (inode replacement, no orphan .tmp) rather than by mocking.
- multi-output executions: every call site in the suite committed exactly
one output, so truncating the output loop was invisible.
- the config normalizer's real contract: it throws rather than clamps, its
bounds are non-strict, and it validates the MERGED values — lowering
only maxEntries to 6 is startup-fatal against the default 12.
* fix(omni): keep a withheld resource reachable, and stop losing why recall degraded
Three findings that each end the same way — the session holds a memory it
cannot get back to, and nothing says so:
- A transport-guard rejection threw away the handle it had already minted.
Every throw site sits downstream of the bind, and the omission branch
with the identical "over-limit, withheld" verdict does disclose its
handle — so the rejection was the one path that stranded a recorded
resource: the model can neither see the bytes nor ask memory about them.
The verdict now carries the handle, and the tool-result funnel emits it.
- Passive recall degrades silently by design (the turn proceeds without the
memory it was meant to carry). With a pinned-but-unavailable selector
model that is a permanent outage with nothing to see, so the reason is
now recorded at the injection seam, and the manifest-failure branch —
which had no log at all — gets one.
- Chinese queries never ranked. Splitting on separators yielded one token
per phrase, scored by whole-substring containment: unless an entry
repeated the caller's exact phrasing, everything scored zero and ordering
collapsed to newest-first. Unsegmented scripts now index as overlapping
character bigrams, split at script boundaries so `480p字幕` does not
produce bigrams that straddle them.
Also: `PolicySucceededCommit.created` had no consumer outside tests. It
distinguishes a content-identity replay from a fresh record, which is
exactly what a reader of the debug log cannot otherwise tell — "no new
execution appeared" reads identically to "collection did nothing".
* test(omni): cover the four seams where memory reaches the pipeline
Each was free to disappear without a red test:
- the orchestrator's post-promotion commit, its degradation-cache-hit
commit, and the memoryBinding threaded onto each WorkItem. Disabling the
commit, or always threading the source binding instead of the derived
one, left the suite green while every fixed-policy execution recorded
wrong or no lineage. Covered with a real MediaMemoryService on a tmp
store, reading memory.json back — including a two-stage chain, where
stage two must be EXECUTED_ON stage one's version, not the root.
- the reactive-degradation ladder's memory wiring, which had no coverage
at all beyond its pure helpers. One test pins that a rung's re-derivation
lands on the binding found by content hash; the other that an unknown
hash degrades normally and records NO execution, rather than inventing
provenance on someone else's lineage.
- the client injection seam: moving it after the systemReminders splice —
the exact regression that kills passive recall — now fails.
- the D10 exposure gate at registry-construction level: the recall tool
appears in active mode, is absent in sideQuery mode, and an invalid
omni.memory.recall.mode aborts instead of falling back to defaults.
* fix(omni): keep S5 off the root barrel, which was dragging 550 KB into ACP
`Check serve fast-path bundle closure` went red: the ACP agent's static
import closure had acquired iconv-lite's encoding tables (~553 KB).
The mechanism is worth writing down, because the obvious readings are all
wrong. Nothing imported iconv-lite. What S5 did was add five re-exports to
the ROOT barrel, which is statically imported by ~150 modules in the ACP
closure. esbuild assigns modules to chunks by which entry points reach
them, so every module added to the barrel's graph regroups the split — and
that regrouping moved `sync-file-encoding` (and through it `iconvHelper`,
whose top-level `import 'iconv-lite'` has side effects and cannot be shaken
out) into a chunk the ACP chunk statically imports.
So the fix is not "move the export somewhere else": my first two attempts
relocated it to the `omni` barrel, which is itself statically imported by
the root barrel and by tool-result-media, and stayed red. The barrel
surface itself has to come back to baseline.
None of the five needed to be there. Four had no consumer outside their own
module and tests; the rest are reached by dynamic import, which does not
care where the export lives:
- ACP's model-call collection gets a dedicated `./omniPolicyCollection`
subpath, the pattern `./goalWire` and `./transcriptRecords` already
establish for "the CLI needs one deep function, not a barrel".
- `reanchorRememberedMedia` moves to the `omni` door the `@`-reference
funnel already loads for every other omni call it makes.
- `createMediaMemoryRecallService`, `buildMediaMemoryRecallAdvisor`,
`extractRequestResourceIds`, `runOmniMemorySideQuery`,
`formatOmniMemorySideQueryReminder`, `OmniRecallMediaMemoryTool` and
`formatResourceHandleText` simply leave the root barrel.
Located by bisecting the 28 commits with the check itself, then diffing the
ACP closure module sets between a green and a red metafile: the delta was
exactly three modules, which is what named the mechanism.
* fix(omni): anchor URL media off its staging file, and record ACP calls as model calls
Two findings from the re-review, both real:
- The URL funnel hands its staging download to the delivery pipeline with
no lifetime marker, so memory recorded the `.part` path as the file's
persistent identity — the same defect C9 fixed for tool-result media,
missed on this third funnel. The handle already shown to the model
resolves to ENOENT the moment the funnel's `finally` runs: every later
policy call on it fails, and cross-session recall reports
`artifact_unavailable` for bytes the object store still holds. URL media
now anchors to the content-addressed object path, and the version's
source records the URL itself (`protocol: 'url'` existed in the schema,
unused until now) — the durable identity of downloaded media is where it
came from, not where it briefly landed.
- ACP committed its policy successes with `executionOrigin: 'client'`
while the modelAccess gate forty screens up pins the very same call as
`{ kind: 'model' }` ("Every ACP-originated call is a model call").
Recall provenance contradicted the gate that admitted the call.
The URL-anchor test is mutation-verified: reverting the lifetime marker
fails it.
* docs(omni): reconcile storage lifecycle with the shipped recovery, stake out GC
Section 6.1 described the recovery scan as drafted; the implementation
learned things the draft could not know, and the doc should carry them:
"delete all of staging/" became a grace window because a second CLI
process's recovery must not delete a live invocation's transcode out from
under it (config validation caps tool timeouts below the window for the
same reason), and corrupt-object deletion cascades into the degradation
cache so a cache hit can never point at bytes that are gone.
Section 6.2 is S6's landing point, sharpened by what S5 settled: the GC
root set has TWO sources in the memory snapshot (artifactRef.managedId
AND managed-protocol source locators — tool/URL media anchor their
identity there), the snapshot read is fail-closed (no roots readable, no
deletion at all), and object deletion reuses the recovery scan's existing
cache-cascade rather than growing a second one.
* test(omni): stop asserting an exact timeout the wall clock can undercut
The bucket loop's budget guard calls remainingTimeoutMs() right before
the ffmpeg call reads it again; on a slow runner a millisecond elapses
between the two reads and the exact-equality assertions on the first
call's timeoutMs turn flaky (observed on ubuntu CI: 599999 != 600000).
Freeze Date.now in the full-duration spread test (ffmpeg is mocked,
nothing there needs real time) and tolerate the guard-to-use drift in
the shared-budget test, which does need the real clock for its 50ms
burn. Restore spies in afterEach so the frozen clock never leaks into
neighbouring tests.
* feat(omni): S6 governance — object-store GC with byte budget, JSONL trajectory export (#9081)
* feat(omni): mark-and-sweep GC over the object store, with a byte budget
The store only ever grew: recovery cleans staging, downloads and
quarantine, but a committed object had no way out — one film-commentary
session leaves 38 objects behind, forever. S6 (storage design §6.2).
Two hard rules shape the implementation:
- An unreadable ledger deletes nothing. The root set is the memory
snapshot; if it cannot be read there is no telling "unreferenced" from
"unknown", so the run skips. This interacts with the store's own
corruption recovery (C8) in a way the first test caught: a corrupt
memory.json is backed up and REPLACED WITH AN EMPTY LEDGER — right for
recall, catastrophic for a GC that trusts it, because empty reads as
"nothing is referenced" and one corruption event would erase the whole
store. A fresh .corrupt-* backup therefore blocks the run until it ages
past the retention window.
- A referenced object is never deleted, even over budget. Roots come from
BOTH reference kinds in the snapshot — entries[].artifactRef.managedId
and managed-protocol source locators (tool/URL media anchor their file
identity in the store; missing these would delete exactly the objects
whose store copy is the only copy) — plus the live session registry,
which may hold a binding whose memory commit has not landed yet. Budget
pressure with only referenced objects left suspends NEW derivations
(policy design §8.4) instead: the orchestrator refuses to execute a
policy tool, while reuse and cache hits — which produce no new bytes —
stay allowed.
Sweep: unreferenced and older than omni.storage.retentionDays (14d
default); over omni.storage.maxTotalBytes (20 GiB default), oldest
unreferenced first regardless of age. Deletion cascades into the upload
cache and the degradation cache, same invariant recovery's corrupt-object
path enforces. Symlinked shards are never descended into. Runs once per
store root per process, asynchronously after startup recovery.
* feat(omni): export a session's trajectory as training-consumable JSONL
S6's second deliverable (issue #8190, absorbing #8196): the experiment's
whole point is trajectories — what media came in, what the pipeline did
to it, what the model actually saw, what it answered — and until now they
lived split across the chat transcript and memory.json in shapes no
training script can consume.
The exporter is a pure reader over those two artifacts. No collection
points are added: it cannot affect runtime behavior and works on sessions
recorded before it existed. Three record kinds per line — `turn` (request
text with reminders stripped and any injected passive recall extracted,
the media annotations the model saw, the response and its tool calls),
`execution` (full provenance: arguments, config hash, reuse marker), and
`file` (the durable identity: fileVersionId, sha256, origin, source).
The join is explicit rather than fabricated: model/client executions join
turns on callId ↔ invocationId (both sides record the scheduler's id);
fixed-policy executions join through file identities. No turn↔execution
edge is invented from timestamps — a wrong edge in training data is worse
than an explicit join key.
Privacy carries over from recall: reserved runtime keys (inputPath /
outputDir) are stripped from exported tool args, media bytes are never
embedded (sha256 is the pointer, valid while GC retention lasts), and
every execution line carries omniConfigHash so downstream can split
trajectories by processing configuration instead of mixing distributions.
Deterministic ordering makes re-exports byte-identical. An unreadable
memory store degrades to transcript-only output; an unreadable transcript
is an error — there is no trajectory without it. Wrapper script at
scripts/export-omni-trajectory.js; the function is exported through the
`omni` door (never the root barrel — see
|
||
|
callmeYe
|
8aaaaf2524 |
feat(web-shell): add footnote previews and per-turn sources (#11480)
* feat(web-shell): add grouped footnote previews * fix(web-shell): synchronize footnote type dependency lockfile * feat(web-shell): complete source footnote previews * fix(web-shell): stabilize citation footer previews * fix(web-shell): align citation controls * fix(web-shell): optically align citation icon * feat(web-shell): generalize footnotes and expose host icon resolvers * fix(web-shell): align demo citation artwork with footer icons * feat(web-shell): add framework-neutral footnote preview content slot * feat(web-shell): show turn sources independently of footnotes * test(web-shell): use same-origin footnote thumbnail fixture * fix(web-shell): preserve source focus and nested footnotes * test(web-shell): isolate footnote model-list fixtures * chore: regenerate pnpm lockfile for citation types * fix(web-shell): reuse committed turn sources while streaming * docs(web-shell): clarify panel-only source visibility * fix(web-shell): isolate history sources and restore hover dismissal * fix(web-shell): preserve single formula text in footnote previews * fix(web-shell): restore declined footnote pages on return |
||
|
易良andQwen-Coder
|
c666ec1a0a |
fix(ci): restore Windows test baseline (#11787)
* fix(live): restore Windows filesystem compatibility
* fix(ci): restore Windows test baseline
* fix(ci): keep release-age excludes internal and re-arm two witnesses
- pnpm-workspace.yaml: drop '@tybys/[email protected].4' from
minimumReleaseAgeExclude. scripts/tests/package-scripts.test.js pins
that list to internal, version-independent entries. The exclusion is
also inert: minimumReleaseAge is not configured anywhere in the repo,
and pnpm-lock.yaml records `overrides` but not minimumReleaseAgeExclude
-- so the '@tybys/wasm-util': '0.10.4' override stays, because dropping
it would desync the lockfile and fail --frozen-lockfile.
- web-shell-brand.test.ts: build the swapped-file fake as BigIntStats.
Production compares bigint fields with a bare `!==`, so a number-backed
fake tripped `dev` first and the perturbed inode was never reached;
removing the perturbation now turns the test red again.
- run-qwen-serve.test.ts: '0' is unresolvable on Windows (getaddrinfo
ENOTFOUND 0), so the wildcard case keeps '0.0.0.0' and witnesses the
socket rather than a spelling: the operator typed port 0, so the report
must carry the ephemeral port the listener actually got. Spelling
normalisation stays pinned platform-independently in
remote-quickstart.test.ts.
- artifact-snapshots.ts: revert the readdir hoist to main. It is
behaviourally inert (the catch rethrows ENOENT on the operationId path
either way), has no test and no Windows link.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmu07e78o2w
* fix(ci): restore the load-bearing release-age exclude, keep the witness fixes
The previous commit dropped '@tybys/[email protected].4' from
minimumReleaseAgeExclude to satisfy scripts/tests/package-scripts.test.js:390,
on the reasoning that the entry was inert because minimumReleaseAge is not
configured in this repo. That reasoning was wrong and CI measured it: the gate
is enforced by pnpm itself, not by repo config, so removing the exclude failed
'Install frozen pnpm worktree dependencies' on all three platforms with
Lockfile failed supply-chain policy check (1975 entries)
@tybys/[email protected].4 was published at 2026-09-13T08:00:13.464Z,
within the minimumReleaseAge cutoff (2026-09-12T20:03:50.286Z)
(run 34779568628; Install was green at
|
||
|
易良
|
f71a4e797e |
chore(pnpm): scope build approvals, pin the ACP zod peer, add a freshness lane (#11797)
Follow-ups left open when #11625 merged: its three unresolved review threads, the deferred finding in #11681, and #10524. - allowBuilds decisions now honour the whole key the way pnpm does: a bare name covers every version, an exact `name@version` (or a `||` union) covers only those, and a range or source-like key decides nothing. Reducing a key to its name let an approval scoped to one version cover the next (R1-10). - Pin `@agentclientprotocol/sdk>zod` to 3.25.76 so a pnpm worktree runs the zod the npm tree installs. Under zod 4 the SDK's own `instanceof z.ZodError` test stops matching and a malformed request answers -32602 instead of -32603; both versions are npm-locked, so the agreement gate cannot see the split (R1-1). - Cover web-shell's shipped client sources with the undeclared-import rule; it is the one published package whose sources are not under src/. Measured cost today: zero violations (R2-6). - Add a path-filtered workflow that regenerates pnpm-lock.yaml with `corepack pnpm import` and fails when the committed file differs, which is the freshness half set membership cannot cover (#11681). - setup-worktree: accept only an executable file on PATH, so a corepack left without its exec bit reports the actionable error instead of EACCES (#10524). - Name the esbuild divergence an accepted parity gap, and record what `fsevents: false` costs a macOS worktree. |
||
|
|
1e768979c5 |
chore(pnpm): gate pnpm-lock on package-lock and declare hoisted imports (#11625)
* chore(pnpm): gate pnpm-lock on package-lock and declare hoisted imports - Lockfile check: every pnpm-lock version must be locked in package-lock (one documented npm-lock gap, failing if it goes stale), and every dependency npm runs an install script for needs an explicit allowBuilds decision (fsevents: false records current behaviour). - Regenerate pnpm-lock with `pnpm import`, which is deterministic and drops 18 versions npm never locked; document it as the regeneration step. - Enforce import/no-extraneous-dependencies on shipped sources and declare the 15 imports that only resolved through hoisting, at already-locked versions. - Run the pnpm smoke workflow when package-lock.json changes. - Add the Stage 2 verification brief. Refs #10444 * chore(pnpm): regenerate vscode NOTICES and pin the new smoke path - NOTICES.txt: core's new OpenTelemetry declarations make the generator visit six Apache-2.0 entries in a different order. Same 655 entries, same text; only their order changes (6 header lines). - package-scripts test: expect package-lock.json in the pnpm smoke paths. * fix(pnpm): close lockfile-gate gaps found in review - fail closed when pnpm-lock.yaml has no packages section instead of printing the strongest success line over an unread graph - match git/file: dependency keys against npm's resolved field, whose pnpm key is the source rather than a version - count an allowBuilds entry only when its value is boolean, so pnpm's own 'set this to true or false' placeholder no longer passes the gate - drop the inert package-lock.json trigger from the pnpm smoke workflow; no step in that job reads it, and npm/pnpm agreement is already gated by ci.yml's Check lockfile step on the full profile - stage-2 evidence recipe: cover nested workspace roots in the reflink arm, sample peak worktree count at most 5 minutes apart, run the parity matrix in a throwaway workflow, and give the npm baseline the same skip guards the pnpm arm has - drive every gate branch with fixture-based negative tests * docs(verification): correct pnpm stage-2 measurement recipe Quiesce the df window, name the worktree sampler's working directory and forbid host cron, and register the throwaway workflow's size baseline so the parity run's test:ci does not fail under both installers. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmty6iqs0zv * docs(verification): fix stage-2 recipe's dispatch and macOS fs probe Step 2 now pushes the scratch branch before gh workflow run --ref (the ref is resolved server-side), notes the workflow token scope, and gives the merge-to- default / pull_request fallback when a branch-only dispatch is refused. Step 1 probes the checkout's own volume with diskutil info "$(pwd)" instead of the sealed read-only / system volume. Pushing, dispatching, and posting the results comment are marked operator actions. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmty8nwk9zy * test(lockfile): cover stale-gap and alias arms of check-lockfile Add two failure-branch cases: a knownNpmLockGaps entry that leaves the pnpm graph (the !pnpmVersions.includes arm), and an aliased install script reported under its real package name (the npmPackageName details.name arm). Both were untested and stayed green if the arm was deleted. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmty8nwk9zy * fix(lockfile): close the empty-env seam and name the overrides remedy - `CHECK_LOCKFILE_ROOT` was read with `??`, so an exported-but-empty variable made `root` the empty string and every path in this gate cwd-relative: it would then report on whatever lockfiles sit in that directory as if they were the repository's, and pass quietly when the cwd happens to be the repo root. Truthiness keeps the fixture seam and closes the empty case. - The version-agreement failure named exactly one remedy, `corepack pnpm import`, while pnpm-workspace.yaml carries eight `overrides` (three of them pin typescript) that no npm-side regeneration can reproduce. The message now names that layer, so a still-red gate points at what actually decides the value. Verified: `node scripts/check-lockfile.js` exits 0 on the committed tree, prettier and eslint are clean on the file, and both suites that execute the gate pass. Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: qwen-code-ci-bot <[email protected]> Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: qwen-code-dev-bot <[email protected]> |
||
|
易良andShaojin Wen
|
8a21551599 |
chore: enforce Playwright version parity and close the deferred #11101 follow-ups (#11371)
* chore: enforce Playwright parity and close deferred #11101 follow-ups Works the four items tracked in #11336: - Pin Web Shell's @playwright/test to the same exact version as the root playwright dependency, and fail `npm run check:lockfile` when the two direct manifests drift apart or their resolved root trees stop matching the pin. The check deliberately tolerates the packages/mobile-mcp tree: mobilewright depends on playwright and playwright-core at an exact older revision, so no manifest edit in this repository can dedupe it. - Correct the "Install evidence browser" rationale in qwen-triage.yml and its restatement in the workflow test. The two-trees premise no longer holds now that both manifests are pinned; the resolution behavior is unchanged, since resolving from the harness's own directory still keeps the binary tied to the package it imports. - Cover the ENOENT fallback of the exported isMainModule helper. The existing runner.py assertion takes the realpath success path because that file exists, so the fallback stayed unpinned. - Separate the CLI-only from the daemon-only followup suggestion gates in the documentation instead of pointing daemon clients at the blended CLI-oriented list, keeping the headless/SDK opt-out guidance. * chore: pin the Playwright parity check and correct its scope notes Follow-up to the round-1 review on this PR. The parity block had no test in either direction: the only suite that runs the script asserted the two pre-existing banners, so deleting the block, inverting a comparison, or breaking the importer derivation left both `npm run test:scripts` and CI's `check:lockfile` gate green. That is the defect class this PR exists to remove — the same diff rewrites two rationale comments to lean on the check, so an unpinned guard rots into a comment that lies. Add the third banner to the existing suite and a fixture suite that runs the script against perturbed copies of the real manifests and lockfiles, covering both drift arms and the two shapes that must stay green. Also correct three properties of the block the review found: - The inventory comment claimed two Playwright declarations sit outside the invariant; `packages/mobile-mcp` declares `@playwright/test` as a range directly, in a workspace member, and was not named. It is the only in-workspace range on a name this check pins, so it is what the resolved-version message should point an operator at. - The agreement check destructured exactly two entries while the comment invited maintainers to extend the list, so a third entry would have received the exactness check and silently skipped the comparison the block exists for. Compare every entry against the first and derive the resolved-tree locations from the same list instead of repeating them as a second literal. - The pnpm arm prescribed "regenerate it" for every divergence, but `pnpm-workspace.yaml`'s `overrides:` and `.pnpmfile.mjs` both rewrite the recorded specifier, so regeneration is a no-op for those. Name the deciding layer in the messages and the importer list's source for a missing entry. * docs: stop counting the followup suggestion code paths Merging main brought in the OpenTUI port of ink's AppContainer effect (packages/cli/src/ui/opentui/followup-generation.ts), which is a third non-test `generatePromptSuggestion` call site. The section opener claimed "Two code paths decide this", which a grep now contradicts even though the substance still holds: the OpenTUI hook applies the same CLI gate set as AppContainer — interactive, not SDK mode, the Responding to Idle edge, no error in the last item, no waiting calls, approval mode not plan — and collapses ink's five explicit dialog checks into `waitingCalls.length === 0` because the composer is unmounted while a shell dialog is open. Name the two decision-makers instead of counting implementations, so the sentence stays true when a renderer is added or retired. The CLI/daemon split below is unchanged and was re-verified against the merged tree: the ink gate is byte-identical at AppContainer.tsx:3612-3627, the daemon gate moved only by line number (Session.ts:5080-5095), and the shared MIN_ASSISTANT_TURNS gate is still at suggestionGenerator.ts:83,103. --------- Co-authored-by: Shaojin Wen <[email protected]> |
||
|
Dragon
|
4ec9f63d0a |
feat(core): add CodeModeOnly tool execution (#10607)
* feat(core): add CodeModeOnly tool execution * fix(core): preserve CodeModeOnly execution semantics * fix(core): drop the tool_search reminder in CodeModeOnly CodeModeOnly hides `tool_search` and binds every deferred tool — schema included — into the `exec` description, so progressive discovery has nothing left to do. The session prelude still announced those tools as "reachable via `tool_search`", pointing the model at a tool that is never declared and re-billing the text on every cached prefix. Worse, tools already callable as `tools.web_fetch(...)` looked gated behind a lookup step that does not exist. The subagent and fork-resume callers already opted out, but the main-session prelude sites did not, so the gate now lives where the prelude is built rather than in each caller. * chore: refresh settings schema * chore: refresh companion notices * fix: register exec tool metadata * fix(cli): localize exec tool display name * fix(core): stop offering ToolSearch lookups in CodeModeOnly CodeModeOnly hides tool_search and binds every deferred schema into the exec description, so text that tells the model to look a tool up describes a surface it cannot reach. Return an empty deferred summary from the registry itself - this also covers the fork-resume reminder that bypassed the earlier call-site gate - and point the image zoom hint at tools.zoom_image. * feat(core): teach the system prompt CodeModeOnly's tool surface In CodeModeOnly the model can only call exec, so the tool guidance and the worked examples described a surface it does not have: bare tool names it cannot call, an instruction to issue several tool calls in one response, and examples in a direct tool-call syntax that does not exist there. Give "Using Your Tools" a code-mode branch that routes every tool through tools.<name>, states which tools are direct controls instead, and replaces multi-call parallelism with batching inside one exec program. Swap the four model-family example sets for one shared exec set, since the syntax differences they exist for do not apply. The mechanics of exec itself stay in its tool description; the prompt carries only policy. * test: include code mode host in standalone fixture * fix(core): return code mode media as multimodal output * feat(core): support image tool results in code mode * fix(core): carry deferred tool schemas in the exec description CodeModeOnly hides tool_search and never surfaces a nested call as a history functionCall, so a signature collapsed to Record<string, unknown> could never be filled in later: parameter names were reachable only by guessing, or by reading them off validation errors one at a time. Deferred tools keep their registry state; only the generated signature gains the schema. * fix(node-repl): restore the bin exec bit after rebuild npm chmods dist/index.js when it links the bin at install time, but the build deletes that exact file and tsc re-emits it as 0644. After any rebuild node_modules/.bin/node-repl-mcp therefore points at a non-executable target and spawning it fails with EACCES (exit 126) before the shebang is ever read. OR the exec bits into the emitted mode instead of setting 0o755, so a restrictive umask is preserved rather than widened. * fix(cli): pin patched Ink version * feat(cli): add collapsible tool call details * fix(dev): use CLI tsconfig outside repository * fix(cli): preserve expanded tool details after completion * fix(cli): fail headless runs on model API errors * fix(core): expose shell timeout bounds in code mode * feat(core): add code mode timeout helpers * feat(cli): expose code mode in settings * fix(mcp): preserve structured results for model and exec consumers * fix(core): preserve code mode output and truncate inline * feat(core): port scoped context tools to code mode * fix(build): synchronize pnpm lockfile and capability docs * fix(core): recover from rejected Responses encrypted reasoning |
||
|
|
f649d65d1f |
feat(live): add visual input, proactive assistance and memory (#11369)
* feat(live): add visual input, proactive memory and desktop controls * fix(live): preserve upstream checks and finalize migration validation * feat(live): open active configuration from host settings * fix(live): address lifecycle and proactive review findings * feat(live): add subagent controls and diagnostics with review fixes --------- Co-authored-by: 菲兹 <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> |
||
|
Shaojin WenandQwen-Coder
|
13b69b3313 |
feat(web-shell): make the product name and logo configurable (#11244)
* feat(web-shell): make the product name and logo configurable
White-labelling the Web Shell meant editing six source files and rebuilding: the
product name was a literal in the sidebar, the welcome header, the About panel
and the document title, and the logo existed as two hand-duplicated copies of the
same inline SVG.
Brand is now configuration. A name and a logo path are read from the operator
settings scopes, and a new process-global daemon route resolves the logo into a
data URI for the client. Embedding hosts get the same control through a prop that
replaces the resolved value outright. The terminal banner already had its own
white-label settings, so this follows their sanitization and path-resolution
conventions rather than inventing parallel ones.
Two decisions carry the design. The workspace settings layer is never read, so a
repository's own settings file cannot rename the product or name a file for the
daemon to inline into every connected browser. And a custom logo is only ever
rendered as an image, never injected as markup, which is what makes it safe for
the daemon to skip sanitizing the SVG it read.
An untouched installation renders exactly what it did before: the resolved brand
is empty, every consumer falls back to its own built-in literal, and the served
document keeps its original title and favicon bytes, so the existing visual
baselines stay valid.
* fix(docs): sync the daemon capability tag count with the new brand tag
The daemon index states how many capability tags the registry holds, and a
contract test fails the build when the two disagree. Adding the brand tag made
the registry 158 while the index still said 157.
* chore: re-trigger CI on the current head
The previous push arrived while a re-run of the prior commit was still holding
the workflow's concurrency slot, so GitHub suppressed the new run instead of
queueing it and the event does not replay once the slot frees. This leaves the
branch head with no CI at all. An empty commit re-fires the synchronize event;
the repository squash-merges, so it does not survive into main.
* fix(web-shell): Address brand review feedback across resolver, client and docs
- Expose a brandSettled flag from the workspace provider so a
settled-with-no-brand outcome (older daemon, withdrawn host prop) is
reported and invalidates stale cached chrome, while the in-flight
state still fires nothing
- Fall back to the built-in sidebar mark when a logo data URI fails to
decode, and when a host logo node is falsy; remount the logo image
per data URI so a decode failure cannot stick to the next logo
- Report an empty host brand name as unset in onBrandResolved, and type
the callback payload as the document-actionable subset
- Exclude the favicon from the boot watchdog's fatal-resource
classification and keep its content-free error out of the panel list
- Require a namespaced <svg> root in the resolver, cover the post-decode
size cap, and pin the exact stderr prefix the protocol doc publishes
- Assert GET /brand answers JSON on the real app for any Accept header
- Sync the capability index count (159), settings docs and the design
doc with the above
* fix(web-shell): Address second brand review round — settle semantics, SVG scanner, test fixtures
- Settle the brand fetch only on a definitive outcome: an answer, or a
404 from a route-less daemon. A retryable failure (503 bootstrap, 429,
transport, old SDK) is unknown rather than absent — settling it
cleared the pre-paint cache and reset the tab title mid-session over
a blip nothing retries (review Critical)
- Rewrite the SVG root scanner quote-aware: a quoted `>`, `[` or `]` in
DOCTYPE literals or the root tag no longer rejects valid documents,
and an xmlns-shaped substring inside another attribute's value no
longer certifies a namespace-less document as SVG
- Warn (not reject) when the root svg carries no viewBox and no
width/height: the browser loads such an image successfully but may
paint a blank mark at the sidebar's fixed size, and only the daemon's
stderr can say so
- Treat a nullish host brand prop as absent in the resolution gate,
matching the `??` one line above
- Pin previously unwitnessed guards: the fd identity re-check and both
fs soft-fail branches, the logo key remount, the favicon error-list
exclusion, the unsettled-to-settled transition, whole-object host
takeover against a daemon logo, and the superseded-client reset and
catch-leg disposed guards
- Drive GET /brand on a real app with the SPA fallback mounted and a
token configured, and pin the System settings layer to empty files so
a maintainer's machine-wide brand cannot leak into the assertions
- Round-trip the pre-paint cache through the real inline script instead
of pinning each side against its own copy of the key
- Document the hard-link and decoded-size rules, the namespace
requirement, the viewBox advisory, and the corrected 503 semantics
(bootstrap catch-all, retryable; a draining daemon does not reject
this route); the README's onBrandResolved row now names the narrower
payload type
* fix(web-shell): Close third brand review round — placeholder exclusion, scanner correctness, retry path
- Read brand keys from the pre-substitution snapshot and refuse
environment placeholders with a warning: placeholder substitution
draws from the process-wide environment, which a workspace populates
first at boot, so resolving brand placeholders would let a repository
supply the value — the workspace exclusion through a side door
(review Critical)
- Blank quoted SVG attribute spans per UTF-16 code unit, not per code
point: an astral character collapsed to one space and shifted every
later index, refusing valid namespaced documents (review Critical)
- Accept prefix-bound <svg:svg> roots that bind xmlns:svg to the SVG
namespace (browsers render them), keep refusing <svgfoo> near-misses
and prefix-only bindings on an unprefixed root, and pin the
whitespace-prefixed fake-xmlns shape that actually exercises the
quote-blanking
- Make the scaling-geometry advisory value-aware: an empty viewBox,
zero dimensions, or percentage width/height now warns (loads
successfully but cannot scale into the sidebar box)
- Add refreshBrand beside refreshCapabilities — same 404-only settle
rule, generation-guarded, gated to the genuinely-missing state and
no in-flight fetch — and call it from the connection recovery path,
so a retryable brand failure no longer leaves in-app chrome and tab
chrome disagreeing for the page's lifetime
- Warn on console when the sidebar logo fails to decode (the one logo
failure the daemon's root-tag validation cannot see)
- Pin the O_NONBLOCK FIFO defence with a real mkfifo fixture, the
published 429/drain transport states on the real app, and make the
ordering test's body discriminating with a fixture brand
- Document the placeholder carve-out, prefix-bound roots and the
value-aware advisory across settings docs, schema, protocol reference
and the design record
* fix(web-shell): Close fourth brand review round — env-var side door, retry machinery, scanner class fix
- Add QWEN_CODE_SYSTEM_SETTINGS_PATH / QWEN_CODE_SYSTEM_DEFAULTS_PATH to
the hardcoded project-.env exclusions: they select which file becomes
the System settings layer, so a repository could otherwise promote
its own file above the operator's User layer and rebrand the shell
with literal values (review Critical)
- Refuse brand placeholders on substitution, not syntax: run the real
resolveEnvVarsInString and refuse only when it would change the
value, so a literal name containing `$` survives while anything a
workspace-tainted environment could have supplied is refused
- Emit one stderr line per brand warning instead of a joined line, so
a log rule keyed on one key's prefix keeps firing when another
misconfiguration exists
- Make the SVG root matcher prefix-agnostic and entity-aware: any
prefix bound to the SVG namespace qualifies, character references in
attribute values are decoded before comparison, a degenerate viewBox
no longer suppresses the scaling advisory, and a prefix-bound root
with unprefixed children gets its own advisory
- Retry a retryable brand failure once after a short delay and warn
only when the retry is also refused (404 stays silent); wire
refreshBrand into the recovery path with tests; gate it on
in-flight; attribute retries per client instance; reset the brand in
the render that observes a client change so no committed frame mixes
two connections
- Pin the O_NONBLOCK flag with a flags-capture witness (the FIFO case
alone cannot fail fast: a blocked open wedges the worker's event
loop), skip the FIFO fixture on Windows where the defence does not
exist, and mock the not-stattable path instead of relying on POSIX
ENOTDIR
- Document the startup hold vs terminal 503 semantics correctly and
retire the design record's pre-retry narrative
* fix(web-shell): Close fifth brand review round — real XML parser, docs alignment, hold test
- Parse the logo with the vendored fast-xml-parser instead of the
hand-rolled root-tag scanner: well-formedness (duplicate attributes,
misplaced declarations) is now rejected by a real validator, the
namespace check reads parsed attributes with entity references
decoded, any prefix bound to the SVG namespace qualifies, and the
prefix-children advisory walks the parsed tree — construct-aware by
construction. The DOCTYPE is stripped for the validator with a
quote- and subset-aware scan because the validator's own DOCTYPE
handling is quote-blind (review Suggestion, filed as the class-level
finding)
- Bound entity decoding at Unicode's ceiling: an out-of-range
character reference no longer throws a RangeError out of the
resolver — which the route's catch-all turned into a 200 {} that
dropped the validly configured brand NAME along with the logo
(review Critical)
- Refuse brand placeholders by comparing the layer's substituted value
against its pre-substitution snapshot — the authoritative engine
WITH the home-.env fallback — and keep an unresolvable placeholder
verbatim so a typo'd variable shows as literal text; say so in all
five published artifacts, and hint when a missing logo's path still
contains a placeholder token (review Critical)
- Gate the prefix-children advisory on the default namespace being
absent: with both bindings, unprefixed children render fine
- Split the scaling advisory's message by cause (zero-area viewport,
malformed viewBox, no usable geometry) and document both advisories
- Pin the startup-hold guarantee with a real deferred-runtime test:
GET /brand issued before the runtime is ready is held and answered
200 once it settles
- Drive the two retry tests on fake timers (BRAND_RETRY_DELAY_MS is
exported for them) instead of sleeping through the production
constant, and pin the route's one-stderr-line-per-cause invariant
with a two-misconfiguration case
* fix(web-shell): Close sixth brand review round — saxes parser, lockfiles
Swap fast-xml-parser (subset validator plus pre-decoded attributes) for
saxes, the streaming XML parser jsdom uses: malformed documents are now
refused instead of shipped, attribute values are entity-decoded exactly
once so the xmlns gate cannot be faked by a second decode, and the parse
stream is closed after write so an unclosed root fails at EOF. Regenerate
both lockfiles for the dependency swap — the round-5 fast-xml-parser edge
was never recorded, which is what turned the frozen pnpm installs red —
and pin the refused well-formedness classes and the double-decode fixture
in the resolver tests.
Co-authored-by: Qwen-Coder <[email protected]>
* fix(web-shell): Enable namespace processing in the brand logo parser
saxes was constructed with default options, so an undeclared namespace
prefix on a child (the classic legacy-export xlink:href with no
xmlns:xlink) parsed clean here while browsers fatal on it — the daemon
shipped a data URI no browser renders, with nothing on stderr. Enable
xmlns: true, matching jsdom (browsers always have namespace support),
flatten the object-shaped attribute values that mode returns, and pin
both sides: undeclared child prefixes refuse, implicitly-bound xml: and
properly declared bindings still accept. Mutation-checked: reverting the
option and the flattening turns the new refusal test red.
Co-authored-by: Qwen-Coder <[email protected]>
---------
Co-authored-by: Qwen-Coder <[email protected]>
|
||
|
易良andQwen-Coder
|
b661a2cdb2 |
fix(deps): Bump sharp to 0.35.4 to resolve high-severity libheif CVE (#11435)
* fix(deps): Bump sharp to 0.35.4 to resolve high-severity libheif CVE The dependency CVE audit gates on high-severity findings. sharp 0.35.3 pulls in a vulnerable libheif (GHSA-rgj7-g3m4-5g8c), which fails the audit. Bump sharp (and its @img platform/libvips binaries) to 0.35.4, which ships the fixed libvips 1.3.3. * fix(deps): regenerate pnpm-lock.yaml for sharp 0.35.4 Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmttlhdvjsx * chore(deps): Refresh VS Code companion notices for sharp 0.35.4 The sharp bump changes the generated NOTICES.txt version string; Lint & Static gates on the file being up-to-date. * chore(deps): Clear transitive sharp 0.35.3 from pnpm lockfile @mobilewright/core (mobile-mcp) still resolved sharp 0.35.3 through its ^0.35.3 range, leaving the vulnerable libheif build reachable from the worktree bootstrap. Refresh the resolution so every sharp and @img platform entry in the pnpm lockfile is 0.35.4 / 1.3.3. --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
32f51a330b |
chore: restore regression coverage after WebUI retirement (#11101)
* chore(web-shell): follow up #9812 review findings and deferred suggestions Address the remaining review suggestions from the @qwen-code/webui retirement (#9812) and the deferred items tracked in #11076: - Fix the 01-architecture.md adapter diagram path (missing session/). - Correct the followup-suggestions doc: generation fires on end_turn, and headless/SDK daemon clients should opt out to avoid the per-turn LLM cost. - Pin playwright to 1.61.1 to restore version parity with @playwright/test. - Restore file: scheme rejection coverage in Markdown.test.ts. - Pin the historical sessionStorage key prefix with a literal assertion. - Cover the legacy exported JSONL rejection and ChatRecord happy path. - Restore the shell-substring classifier trap with a live path. - Point useDaemonFollowupSuggestion docs at ChatEditor, not the deleted InputForm. - Replace the theme-toggle timing-race assertion and exercise the unhandledrejection fail-closed path in the document browser gate. Closes #11076 * fix(test): poll the theme class instead of a matcher vitest lacks The browser gate imports expect from vitest and drives playwright's Locator directly, so toHaveClass does not exist on the assertion and typecheck:integration failed with TS2339 on this line. Poll the class attribute through the same expect.poll idiom the neighbouring assertions use, which keeps the retry the assertion was switched to and needs no @playwright/test dependency. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmtooas8sln * fix(test): assert the exported theme class exactly `document-main.tsx` toggles `dark` and `light` on <html> mutually exclusively and nothing else writes `documentElement` classes, so the attribute is always exactly one of them. Matching `/light/` as an unanchored substring would also accept `dark light`, which is the theme-toggle breakage this poll exists to catch. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmtow5pgslz * fix(test): realpath the exporter main check and pin its JSONL guard Node realpath-resolves `import.meta.url` for the ESM main module but leaves `process.argv[1]` as invoked, so comparing the raw spellings made a symlinked invocation skip `main()` and exit 0 having written nothing — a silent no-op the harness reads as success. Compare realpaths instead and export the predicate so both directions are pinned. Also extract the input gate into `assertRenderableJsonl` so the legacy exported-JSONL rejection can be asserted directly: the previous tests only pinned the two predicates feeding it, so deleting the guard and losing the remediation hint kept every test green. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmtow5pgslz * docs(followup): name the real ACP stop reasons, drop "unconditional" The parenthetical this PR added named `interrupted` as a daemon stop reason. The ACP `StopReason` union is `end_turn | max_tokens | max_turn_requests | refusal | cancelled`, so there is no `interrupted`; an interrupted turn reports `cancelled`. Name the four suppressing values the union actually defines so an integrator writing `stopReason === '...'` gets a branch that can fire. "unconditional" was wider than the code and contradicted the condition list on the same page: `Session.ts#maybeEmitFollowupSuggestion` returns early on the stop reason, the todo stop guard, `ui.enableFollowupSuggestions`, PLAN approval mode, a missing chat, and a non-model last history entry. Point at those conditions instead; the cost advice still holds because the `enableFollowupSuggestions === false` return sits before `generatePromptSuggestion` is reached. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmtp6vibomd * test(web-shell): drop the duplicate historical-key assertion `persists under the literal historical key prefix`, added on this branch, became a byte-equivalent twin of `writes under the historical WebUI key`, which reached main with #11107 and arrived here through the merge at this PR's head: same `persistStableClientId('client-a', 'session-a')`, same literal `qwen-code-webui-client-id:session:session-a`, same expectation. One fact was reported under two names, so a prefix rename reddened two tests and an auditor could not tell which copy was load-bearing. The surviving test's comment already carries the rename rationale, so dropping the copy loses no coverage. Verified with `cd packages/web-shell && npx vitest run client/daemon/session/clientLifecycle.test.ts`: 19 passed before, 18 passed after. Mutating SESSION_CLIENT_ID_STORAGE_PREFIX to `qwen-code-webshell-client-id:session:` after the dedupe still reddens 3 tests (15 passed), including the surviving `writes under the historical WebUI key`; reverting the mutation returns to 18 passed. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmtp6vibomd * test(integration): drop the unreachable unhandledrejection arm The block this PR added after the CDN-failure loop could not reach the `window.addEventListener('unhandledrejection', showLoadError)` handler its comment claimed to exercise. `page.route('**/*', route => route.abort('blockedbyclient'))` fails the `<script id="transcript-renderer">` load, the template's capture-phase `error` listener (document-index.html:45-56, matching `event.target.id === 'transcript-renderer'`) calls `showLoadError()` first, and `setContent(..., { waitUntil: 'load' })` only resolves after that subresource has failed - so `data-render-complete='error'` and the alert text were already in place before `page.evaluate` dispatched anything, and showLoadError's one-shot guard (`if (document.body.dataset.renderComplete) return;`, document-index.html:30) made the synthetic rejection a no-op. The block was a strict duplicate of the loop's own `rendererBody === null` iteration: identical all-abort route, identical two assertions. Exercising the handler for real needs a renderer body that passes the document's pinned SRI integrity and then rejects during boot; no body this gate can serve passes SRI, so the arm cannot be made honest here. Deleting it returns the file to origin/main's state for this test - the handler and its missing coverage are both pre-existing on main (packages/web-templates is not in this PR's diff), so no coverage this PR was responsible for is lost. Verified: `npm run typecheck:integration` reports 0 errors in this file (6 remain, all in packages/** against generated/dist paths absent from an unbuilt worktree: channels/feishu, cli/src/generated/git-commit.js, web-templates/src/generated/*); `npx prettier --check` and `npx eslint` clean on the file. The browser gate itself was not executed locally - it needs the full build chain plus Chromium. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmtpfg5ewmp * ci(review-context): route concurrent-runner changes to the helper-tests lane This PR added `integration-tests/concurrent-runner/export-html-from-chatrecord-jsonl.test.mjs`, the first `node:test` file under `integration-tests/`, and wired it into CI through the `HELPER_TESTS` env list in `.github/workflows/ci.yml`. The tracked review-context manifest still mapped the whole `integration-tests/**` path to `recommendedTests: ['integration-tests']`, whose vitest lane collects `**/*.test.ts` only (`integration-tests/vitest.config.ts:21`), so a future PR touching only the exporter and its `.mjs` test is told to run a lane that cannot collect the file - the same silent drop `HELPER_TESTS` exists to prevent. Add a narrower rule for `integration-tests/concurrent-runner/**` recommending `helper-tests`. Matching rules merge rather than first-match, so the lane list for that directory becomes the union and every other `integration-tests/` path is untouched. The pinned `expectedManifest` in `manifest-repository-context.committed.test.ts` moves with it, which is what that pin is for. Verified through the real provider (tsx importing `packages/cli/src/commands/review/lib/manifest-repository-context.ts`) because the packages/cli vitest globalSetup gate refuses to run on this unbuilt worktree: concurrent-runner .mjs -> ["helper-tests","integration-tests"] concurrent-runner .js -> ["helper-tests","integration-tests"] chat-transcript-document.test.ts -> ["integration-tests"] (unchanged) all 10 rules co-matching -> non-null, 80 relatedPaths, and every `paths` probe matched some rule Also checked the committed manifest deep-equals the test's pinned literal (10 rules on both sides), and that dropping the new rule breaks that equality, so the pin is not tautological. `npx prettier --check` and `npx eslint` clean on both files. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmtpfg5ewmp * fix(deps): sync pnpm lockfile for Playwright pin --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
|
3ecfaffdf8 |
perf(dev): add pnpm worktree bootstrap foundation (#10449)
* perf(dev): add pnpm worktree bootstrap foundation Co-authored-by: Qwen-Coder <[email protected]> * fix(dev): harden worktree bootstrap process handling Co-authored-by: Qwen-Coder <[email protected]> * perf(dev): prefer cache-only worktree installs Co-authored-by: Qwen-Coder <[email protected]> * fix(dev): preserve cached install cancellation Co-authored-by: Qwen-Coder <[email protected]> * fix(build): harden pnpm worktree bootstrap Validate pnpm lock updates in releases and exercise real installs and builds across supported hosts. Preserve npm release compatibility and keep dependency-only setup from rewriting npm-layout notices. Co-authored-by: Qwen-Coder <[email protected]> * chore: record merge attribution Record the required attribution for the Stage 1 merge without rewriting published history. Co-authored-by: Qwen-Coder <[email protected]> * fix(dev): keep worktree bootstrap clean Use a bootstrap-scoped notice guard because nested npm lifecycle commands replace npm_lifecycle_event. This preserves explicit notice generation while preventing dependency setup from rewriting tracked output. Co-authored-by: Qwen-Coder <[email protected]> * ci: track pnpm smoke workflow size Register the new workflow in the repository size ratchet as required by the main CI gate. Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): drop stale importers from the new pnpm lockfile The lockfile was generated before the WebShell cutover (#9811) removed @qwen-code/webui from web-shell and the tailwind tooling plus @qwen-code/webui from vscode-ide-companion, so frozen-lockfile installs fail on all three smoke platforms with ERR_PNPM_OUTDATED_LOCKFILE. Drop the five stale importer entries so the lockfile matches the current package.json manifests; verified with pnpm 11.24.0 install --frozen-lockfile --lockfile-only. Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): refresh pnpm lockfile after merging main The merge of origin/main added remend@^1.3.1 to packages/cli/package.json without updating pnpm-lock.yaml, breaking the pnpm Worktree Smoke workflow frozen-lockfile install. Regenerated with pnpm install. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmtid3rse9h * fix(scripts): resolve the path variable case-insensitively on Windows A spread of process.env is an ordinary object, and native Windows shells expose the path variable as `Path`, so `env.PATH` was undefined there and findOnPath never located corepack — the offline-first Corepack bootstrap silently degraded to npx on exactly the hosts it exists for. Co-authored-by: Qwen-Coder <[email protected]> * test(scripts): pin the worktree bootstrap guards against mutation - Split the fallback log on /\r?\n/ like the sibling cmd.exe mocks, so the assertion holds on the Windows lane's CRLF output. - Add a win32 variant bootstrapping with the native `Path` casing to pin the case-insensitive lookup. - Assert the smoke workflow's fail-fast flag, the install-before-clean step order, and the no-build guard via a substring on the raw job text. - Extend the pnpmfile rewrite fixture to devDependencies and optionalDependencies, which the committed lockfile already uses. - Assert the notice-skip guard by effect (writeFile never called) and add the flag-absent companion test with I/O stubbed. Co-authored-by: Qwen-Coder <[email protected]> * chore(deps): sync pnpm-lock.yaml with qwen-live ACP backend deps Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmtjyz3tdco * fix(dev): stop patch-package from corrupting the pnpm store The root postinstall runs patch-package, which rewrites files under node_modules in place. With pnpm's default import method those files are hard links into the content-addressable store, so the patch rewrites the store entry too and its contents stop matching the sha512 it is filed under. Every subsequent fresh worktree then fails its --offline stage with ERR_PNPM_NO_OFFLINE_TARBALL, silently falls back to the registry (exit 0), and the fallback's postinstall corrupts the store again, so the offline path never hits. The advertised offline timing was only reachable on a same-tree reinstall, which needs no relinking. Set packageImportMethod to clone-or-copy: a copy-on-write clone where the filesystem supports it, a plain copy elsewhere. Either way patch-package edits only the worktree's own copy and the store entry stays intact, so later worktrees resolve entirely from the store. Verified on an isolated fixture store: under the default method the node_modules file and its store entry share an inode, and an in-place edit moves the store file's sha512 off its address, after which a fresh offline install fails with ERR_PNPM_NO_OFFLINE_TARBALL; under clone-or-copy the same edit leaves the store hash unchanged and the fresh offline install reports reused 1, downloaded 0. Also add pnpm-lock.yaml to .prettierignore. prettier reflows it into a shape pnpm does not emit, so `npm run format` and `pnpm install` fight over the file; .yamllint.yml already ignores it for the same reason. Both facts are pinned by tests in scripts/tests/package-scripts.test.js. Claude-Session: https://claude.ai/code/session_012797rgiteWJxLT9TLkKq8G * fix(ci): refresh pnpm-lock.yaml for qwen-live prompts deps Main added prompts/@types/prompts (and the ACP sdk) to packages/qwen-live/package.json after this branch generated its lockfile, so the pnpm Worktree Smoke frozen-lockfile install failed with ERR_PNPM_OUTDATED_LOCKFILE. Regenerated with pnpm 11.24.0 (--lockfile-only); frozen-lockfile and supply-chain policy checks pass. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-conflict/jmtlc9sozfg * fix(pnpm): close review findings on worktree bootstrap - Declare @qwen-code/qwen-code-core in vscode-ide-companion so the pnpm linker materializes it; regenerate both lockfiles and assert the link in the smoke workflow (R5-1) - Pin @types/node 20.19.1 in packages/core so the pnpm layout compiles against the same types npm hoists (D6-1) - Accept and pin the corepack +sha512 integrity suffix on packageManager; strip it for the npx fallback spec (D6-2) - Cover every workspace member in the pnpmfile rewrite set, mirror the npm channel list in pnpm-workspace.yaml, and cross-check both in tests - Run setup-worktree.js installs with cwd pinned to the checkout, gate the smoke clean-check on git status --porcelain, stop post-merge runs from cancelling each other, and extend check-lockfile.js to pnpm-lock.yaml * merge(main): realign pnpm workspace set and lockfiles Main removed the webui and cua-driver packages; drop both from the pnpm rewrite set and regenerate the pnpm lockfile against the merged manifests (picks up playwright, react-markdown, and the other main-side dependency changes) so the frozen bootstrap matches what the PR merge ref will run. * style(ci): quote smoke workflow scalars per yamllint * fix(dev): require Corepack for pnpm bootstrap * fix(dev): close pnpm bootstrap review blockers --------- Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: yiliang114 <[email protected]> |