mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
f3af3dd8570e0cd91314ab25c881bfec26879501
199
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jinyeandQwen-Coder
|
f3af3dd857 |
refactor(managed-agent): Split server into Maven modules (#15871)
* refactor(managed-agent): split server into Maven modules Separate contracts, orchestration, persistence, adapters, assembly, and verification while retaining the existing service delivery model. Move tests and build consumers together and document local evidence and pending CI acceptance. Co-authored-by: Qwen-Coder <[email protected]> * refactor(managed-agent): reduce configuration and SDK boilerplate Co-authored-by: Qwen-Coder <[email protected]> * refactor(managed-agent): narrow contracts and simplify test scaffolding Co-authored-by: Qwen-Coder <[email protected]> * fix(managed-agent): repair verification after module migration Resolve shared fixtures and worker paths from the Maven module root, update CI packaging assertions, and align existing test fixtures with the extracted contracts. Restore restricted Workspace constructors and remove redundant test visibility changes without altering inherited MySQL coverage. Co-authored-by: Qwen-Coder <[email protected]> * test(managed-agent): align remaining Harness and OSS fixtures Preserve SDK refusal messages and register the managed OSS adapter at fixture injection points. Keep the existing retry and client shutdown assertions intact. Co-authored-by: Qwen-Coder <[email protected]> * style(managed-agent): order Hosted cleanup Harness imports Co-authored-by: Qwen-Coder <[email protected]> * test(managed-agent): fix recovery smoke fixture and import order Supply the required transcript coverage in the recovery browser fixture and wait for history before testing a complete handoff. Preserve its existing safety and submission assertions. Order the cleanup coordinator test imports for Checkstyle. Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
cc6de38cb5 |
fix(ci): isolate web-shell smoke from shared runner networks (#15886)
* fix(ci): isolate web-shell smoke from shared runner networks Co-authored-by: Qwen-Coder <[email protected]> * docs(ci): clarify hosted smoke font guard behavior Co-authored-by: Qwen-Coder <[email protected]> * docs(ci): explain dormant smoke pool steps Co-authored-by: Qwen-Coder <[email protected]> * docs(ci): stop overstating which dormant smoke guards are pinned The retained comment claimed every self-hosted step's guard is pinned by checkout/preflight tests. Only two of six are: `Recreate runner Git metadata` (scripts/tests/qwen-runner-checkout-workflow.test.js) and `Use pre-installed Node.js (self-hosted)` (scripts/tests/no-ak-integration-ci.test.js). `Check host font baseline (self-hosted)` is named by no test file, and its `runner.environment == 'self-hosted'` guard can never be true under `runs-on: 'ubuntu-latest'` — so it is parked, not pinned. Say which two are pinned and that the font baseline guard is parked, and re-measure the ci.yml size ratchet entry (155625 -> 155708). Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmvugxtx22a --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
84aee5b9b7 |
test(web-shell): include transcript coverage in recovery fixture (#15883)
* test(web-shell): include transcript coverage in recovery fixture Co-authored-by: Qwen-Coder <[email protected]> * test(web-shell): complete recovery event fixtures Remove the unevaluated transcript type guard and supply required event terminal flags without changing assertions. Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): use runner temp for Web Shell browser smoke (#15884) * fix(ci): use runner temp for Web Shell browser smoke Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): resolve runner temp in the browser step context Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): partition the complete macOS test suite into bounded jobs (#15888) * fix(ci): use runner temp for Web Shell browser smoke Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): resolve runner temp in the browser step context Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): partition the complete macOS test suite into bounded jobs Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): account for the macOS aggregate job budget Record the additional five-minute aggregate without changing any test execution timeout. Co-authored-by: Qwen-Coder <[email protected]> * test(install): isolate Windows fixtures from persistent PATH (#15889) Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): write the macOS shard matrix in block style and pin its gates The seven-row matrix used YAML flow mappings, which `.yamllint.yml` forbids (`braces: forbid: 'non-empty'`), so the `Run yamllint` step cannot go green on this branch. Rewrite the rows in the block style every other workflow matrix in this repo already uses. Add the test pins the shard plumbing was missing: - the `cli`/`core` dispatch argv, including the bare `--` that keeps `--shard` forwarded to vitest instead of consumed by npm, and the per-group `NODE_OPTIONS` choice - `id: 'install'` on `test_macos_shards`, which the suite's `steps.install.outcome` gate silently reads as '' if it is renamed - the aggregate `test_macos` gate against the shards' gate, so the preserved check name cannot report `skipped` while the legs fail Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmvu6810z1y * fix(ci): preserve shell fixture gates on macOS system Bash Handle empty issue families under nounset without mapfile and explicitly reject either missing OSS credential. Keep failure assertions and include shell diagnostics. Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): keep macOS aggregate failures eligible for infra retry Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
9acb3a5a1d |
fix(ci): pin PR retries to the triggering head SHA (#15734)
* feat(managed-agent): materialize inert external workspaces * feat(managed-agent): enable external files with verified close (#15685) * feat(managed-agent): add the external-workspace provisioning prerequisite (#15512) Step 3a of the external Hosted Workspace plan: a new `external-workspace` Runtime provisioner kind that places a session-isolated worker on a registered EXTERNAL storage tree and mints the boot document the shipped worker already admits (`managed-context/1` plus the existing `storage` member). The storage identity comes only from the durable registration: a read-only primary-key lookup that admits `REGISTERED` rows, bounds the revision to 2^53-1, joins the owner row on its own primary key, and fails closed when a persisted row disagrees with itself. A registration that cannot be read to completion stays retryable (`runtime_external_unavailable`) so the Broker keeps the placement adoptable instead of burying it. `external-workspace` accepts only the closed evidence set {storageRef, mountRoot, continuity}; `mountNonce` from the design is deferred and explicitly refused, because the registration record cannot persist it yet and an unverifiable claim must not be ignored. Hosted tools, EXECUTION admission, publication consumption, `stopDrained` and close gating are not enabled: `supportsDrainedStop()` stays false, every managed Workspace feature remains refused, and preflight now states the prerequisite in an `external-prerequisite` warning. No Session can be placed yet, and every placement fails closed. `external-scratch` behavior and wording, the shipped worker boot/storage/registration.json contract, the schema and the CLI are unchanged. * test(managed-agent): pin external storage refusal in lock canary * fix(managed-agent): require exact external workspace digest * fix(managed-agent): reject storage evidence for non-running workers * feat(managed-agent): add inert exact-worker external drain * feat(managed-agent): fence external workspace ownership and drain * feat(managed-agent): enable external files with verified close * docs: record verified external files and remaining phase two work * fix(web-shell): avoid recovery warning for closed sessions * test(ci): pin HostedExternalPublicFilesIT in the hosted-process class list #15685 adds HostedExternalPublicFilesIT, so hosted-process-ci.test.js's expected class list must carry it; the meta-test pins which IT classes the hosted-process lane runs and had 35 entries against 36 discovered. * fix(managed-agent): restore admission precedence and honour a LOST drain claim The external-files increment resolved the Workspace above the Agent pin, so a request whose stored definition cannot take effect, or whose revision cannot be selected, was refused on Workspace grounds first: it answered 404 workspace_required and a bare IllegalStateException instead of 409 agent_definition_unsupported / 400 unsupported_feature. Resolve the pin first again and keep the external pin rewrite once the Workspace is known. The close-claim validation admitted only DRAINING, but drainClaimedBinding deliberately keeps a LOST binding in LOST until its stop proof commits, so the arm refused its own claim with runtime_close_claim_pending instead of surfacing the stop's runtime_broker_recovery_blocked. BindingRenewal now records that its claim was taken on a LOST binding and admits that state. The settlement probe reads the storage kind to refuse an EXTERNAL tree the bridge holds no mount for, so its fixture needs the managed catalog rather than an empty database. The lock-order canary gains the new ExternalWork.apply/3 ambiguity, whose callee locks the Session row first and never reads ma_turn. * fix(ci): pin PR retries to the triggering head SHA * docs(ci): correct two comments the head-SHA pin made stale Pinning the four pull_request lanes to github.event.pull_request.head.sha left two comments describing the old checkout: the one above test_macos still cross-referenced the Ubuntu gate's superseded wording, and check-lint-gate-freshness.mjs still said the lint lane checks out refs/pull/N/head. Comment-only. ci.yml shrinks by 5 bytes (156090 -> 156085), so neither the .size-baseline ratchet nor the .ci-budget ledger moves. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmvr2rokbyb * test(ci): pin all four PR checkout lanes to their whole ref expression The whole-expression `toBe` pin covered only `test` and `lint_and_static`. `web_shell_e2e_smoke` and `integration_no_ak` were guarded by `toContain(head.sha)` plus `not.toContain('refs/pull/')`, neither of which can see arm order: moving `|| github.ref` ahead of the pull-request arm keeps both green while the lane checks out the asynchronously rebuilt merge ref this PR exists to get away from, and the ancestor guard still passes because a newer merge tree contains the expected head. Pin both lanes against their own full ref literal, so arm order is part of the contract the same way it already is for the push-arm lanes. Verified one mutant at a time on each of the two lines: the new assertion turns red (1 failed | 79 passed) while the two substring assertions stay green, and the tree is byte-identical after reverting. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmvr7204nyh --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良
|
e24f9c48fd |
fix(ci): deepen until the lint scope's own acceptance holds (#15644)
* fix(ci): deepen until the lint scope's own acceptance holds The deepen loop stopped as soon as `git merge-base --all` exited 0, but scripts/lint-scope.js also rejects an ambiguous merge base and a shallow boundary above it. A branch that merged the base tip proves a merge base immediately while its head-side boundary still sits above it, so the loop broke with zero deepens and lint-scope fell back to a full scan with three unused attempts. Mirror the consumer's acceptance instead: ambiguity stops the loop (deepening cannot fix it), a shallow boundary keeps it going. The two give-up paths — cap reached and an aborted deepen fetch — end in the same downstream warning whose remedy (a larger cap) is wrong for the aborted one. Emit the deepen count and the base depth reached so they are distinguishable in the log and the step summary. Pins now assert the mechanism rather than a bare `--deepen=` substring: the blobless deepen, the merge-base guard, the shared predicate call, `|| break`, the 3-round cap, and the give-up line. * fix(ci): report completed deepens and ambiguous history |
||
|
Shaojin Wen
|
eea44208b8 |
ci: run every suite but CLI and core in the side test lane (#15570)
* ci: run every suite but CLI and core in the side test lane
The Linux test job's main lane ran the CLI suite and then, one after
another, core, about twenty smaller workspace suites and scripts/tests,
while the side lane (web-shell, then the standalone regression) went idle
long before the CLI suite finished. The main lane now runs only the CLI
and core suites; the side lane runs every other workspace's test:ci, the
standalone regression and scripts/tests, each at half the Vitest worker
cap. Every side-lane suite runs even after another fails.
* ci: print the side lane of a test step killed before its replay
The lane script replays the side lane's log only after the main lane
ends, so a step timeout or cancel lost every suite but the CLI and core
ones. The script now writes that log to SIDE_LANE_LOG when the caller
names one; ci.yml names it under runner.temp, outside the TMPDIR the step
removes, and a failure()/cancelled() step prints whatever is left of it.
The lane test also checks that the main lane's output is present before
it checks its order against the replay, and a comment that still placed
scripts/tests in the main lane now names the side lane.
* ci: keep the lane script's log out of a nested run
scripts/tests runs `npm run test:scripts:unit`, which runs the lane script
again, inside the step that names SIDE_LANE_LOG. The nested run inherited
that name and truncated, replayed and removed the outer side lane's log, so
the outer replay had nothing to print. The script now unsets the variable
once it has read it, and the lane test drives a nested run with a named log
to pin that: the nested run must not see the caller's log, and the outer
side lane's own output must still reach its replay.
The print-step test also pins the `if [ -f "${side_log}" ]` guard around the
cat, which is what keeps that step quiet on an ordinary red test job.
|
||
|
ChiGao
|
0539160f35 |
fix(cli): preserve previews and verify installed OpenTUI distribution (#15655)
* fix(cli): preserve previews and verify installed OpenTUI distribution * fix(ci): record installed distribution workflow growth * fix(ci): record installed distribution matrix budget |
||
|
易良
|
22c89204f0 |
fix(ci): deepen the incremental lint history until the merge base is provable (#15525)
A fixed 100-commit window expires in about a day at this repository's merge rate, after which merge-base finds no common commit and every longer-lived PR silently ran the full scan (measured when the finding was filed: 68.7% of recently-updated branches were more than 100 commits from their base, median 196). Keep the fetch bounded: after the depth-100 blobless fetch, deepen in capped steps until Git can prove the merge base. Each deepen transfers trees only, and the existing history-fallback warning still fires if the cap is reached. Verified with the literal step body under bash -e -o pipefail against a local filter-capable server whose fork sits 300 commits behind the base tip: the current single fetch leaves merge-base unprovable, the new loop proves it after one deepen round and yields the expected changed-file list. |
||
|
易良
|
f885f39ee9 |
fix(ci): price ARM64 fleet lanes from the self-hosted label (#15529)
The two Linux lanes that raise their timeout on the shared self-hosted pool -- the merge-gating lint lane and the no-AK integration gate -- decided "reserved pool" by looking for the x64 fleet's name inside the classifier's runner label set. The ARM64 fleet renders ["self-hosted", "linux", "arm64"], which carries no fleet name, so the same contended machines were priced as hosted: lint at 45 minutes and no_ak at 30 instead of 90/60. An ARM run killed by the shorter ceiling reads as an ARM compatibility failure, which is the signal the fleet pilot exists to collect. Scope on the routing property that matters -- self-hosted versus GitHub-hosted -- and model the new key in the routing meta-test with an explicit ARM64 case. |
||
|
Shaojin Wen
|
1df5031e70 |
perf(ci): overlap web-shell and standalone tests with the CLI suite (#15438)
* perf(ci): overlap web-shell and standalone tests with the CLI suite The Linux Test job ran every workspace suite, the standalone runtime regression and scripts/tests one after another. The CLI suite keeps its workers busy for 13-20 minutes, while web-shell's suite spends most of its run on App.test.tsx alone and the standalone regression starts one runtime process at a time, so those two added their mostly idle minutes straight to the job. Run them in a side lane beside the other workspaces instead. .github/scripts/ci/run-unit-test-lanes.sh starts web-shell and the standalone regression in the background, runs every other workspace's test:ci (listed with npm pkg get, in npm's workspace order) and then scripts/tests, and replays the side lane's log after the main lane so the two never interleave. The step runs the script under the same setpriv wrapper, so every test command still runs without the DAC override capabilities. test:scripts now reaches its Vitest run through a new test:scripts:unit script with a trailing --, so the main lane can run it without the standalone regression while test:ci, test:ci:workspaces and test:scripts run what they ran before. * fix(ci): name the cause when the lane script cannot start a lane - Fail with an ::error:: line when mktemp cannot create the side lane's log file, instead of running the main lane alone and dying in tail. - Check the exit status of `npm pkg get` and of the parser. A failing `npm pkg get --json` prints an error object, which used to parse as an empty listing and report "Found no workspaces". - Note in the header that the replay puts side-lane failures after the main lane's in main-failure-signature.mjs's title and search markers. - Lane test: clear the caller's NODE_OPTIONS, assert the main lane's output is present before checking its order, and cover the failed listing and the missing temp dir. - Cross-check npm's workspace resolution against the selector in scripts/workspaces.js. * fix(ci): halve web-shell's worker cap beside the main lane Each Vitest run applies VITEST_MAX_FORKS / VITEST_MAX_THREADS on its own, so running two lanes at once doubled the step's worker cap, and on the hosted runners, where no cap is set, put two all-CPUs pools on 4 vCPU. The lane script now runs web-shell at half the cap, at least 1, and at 2 where the cap is unset; the main lane keeps the full cap. - Refuse a workspace name with whitespace in it: the main lane's arguments are read one per line, so a line break would split a name into other arguments, flags among them. - Lane test: web-shell and the main lane's suites now wait for each other, so a side lane serialized before or after the main lane fails; record each call's worker caps; run the script against npm's real listing and compare its main lane with the selector the release pins use; cover a name with a line break in it. - ci.yml: the standalone runtime cache comment names the lane script's side lane, not test:scripts, as what runs the regression in this job. - ci-variables.md: say the cap applies per Vitest run and what the two-lane step runs at its peak. - release-workflow.test.js: follow test:scripts through test:scripts:unit to vitest; #15102 pinned the shape before the split. |
||
|
易良
|
cc49daa282 | fix(ci): preserve independent validation for main push SHAs (#15508) | ||
|
易良
|
10c036b655 |
fix(ci): keep the incremental lint history fetch blobless and visible (#15511)
Lint & Static can already hold a blobless merge base: Report test growth fetches it with --filter=blob:none. A later plain depth-100 fetch then fails resolving thin-pack deltas against blobs this checkout does not have (fatal: missing blob object + remote did not send all necessary objects), and continue-on-error turns that into a silent full scan: 4 of 8 sampled PR runs after #15381 merged, with 5 of 8 falling back to full lint in total. Fetch blobless too, and drop the recorded filter so later fetches in the checkout stay ordinary. lint-scope reads only commits and trees. Announce a history fallback as ::warning:: plus a step-summary line on workflow runs, deduplicated across the seven lint.js invocations, so the optimization can no longer stop working without a trace. |
||
|
易良andShaojin Wen
|
6abb3676c1 |
fix(ci): skip the font baseline gate when the checkout predates it (#15436)
The web-shell smoke lane checks out the pull request head, not the merge commit, so a branch cut before #15410 landed has no `.github/scripts/ecs-runner/check-font-baseline.sh` in its tree. The gate then fails with `bash: ...: No such file or directory` — a red web-shell lane for every in-flight pull request, which is what happened to five runs between 10:23Z and 11:15Z on 2026-10-24. Skip the step when the script is absent, and say so in the log. Branches cut after the guard landed always carry it, so the gate still runs where it can act. Co-authored-by: Shaojin Wen <[email protected]> |
||
|
易良andqwen-code-dev-bot
|
23ab47c9ab |
feat(ci): fail fast when a pool host drifts off the font baseline (#15410)
The web-shell smoke suite asserts exact text layout, so its verdict depends on the host's default `sans-serif`. The x64 ECS images ship `fonts-dejavu-core` and fontconfig resolves Latin text to DejaVu Sans; the ARM64 pool images (Ubuntu 26.04 minimal) do not, so fontconfig falls back to WenQuanYi Zen Hei for Latin as well as CJK. The same strings then measure ~26 px narrower, the token-usage header stops wrapping at the 320 px panel minimum, and two @smoke tests fail 26 minutes into the lane with nothing in the log naming fonts (#15383). The hosts are provisioned now, but nothing records the requirement, so the next image rebuild reintroduces it silently. This adds: - `install-qwen-font-baseline.sh`: idempotent Debian/Ubuntu installer for fonts-dejavu-core + fonts-dejavu-mono, `fc-cache`, then verification. - `check-font-baseline.sh`: L0 asserts the resolved family; `--with-metrics` (L1) measures the three hero text runs whose combined width decides the wrap, in the host's Playwright Chromium, against the x64 baseline within 0.5 px. Their provenance (mock total, i18n key, fixed clock) is recorded at the constants so the numbers stay re-measurable. - A `Check host font baseline (self-hosted)` step in the web-shell smoke lane, before dependency install, so drift costs seconds instead of a 26-minute suite. L1 stays manual because the lane has no Chromium at that point. - Unit tests with stubbed `fc-match`/`apt-get`/`dpkg-query`/Chromium, and a README section that records the requirement. No host is touched by this PR; both ARM64 hosts already carry the packages. Co-authored-by: qwen-code-dev-bot <[email protected]> |
||
|
易良andQwen-Coder
|
cdd5f1e027 |
ci: retain raw JUnit reports for same-repo runs (#15379)
* ci: retain raw JUnit reports for same-repo runs
* ci: keep the retained-report upload advisory and pin its guards
Review follow-up on
|
||
|
易良
|
412c7c307d |
ci: start the web-shell smoke lane beside Test instead of behind it (#15408)
The lane has depended on the Test job since it was created, for exactly one value: the normalized CI profile. classify_pr produces that value itself (its classifier forces an unknown one to full), so the edge buys ordering rather than trust: Test takes ~34 minutes and the smoke lane ~35, and every full-profile PR pays that queueing twice in series. Gate on the classifier's output instead and both lanes start together. The '|| full' fallback keeps the old behaviour when the classifier emitted nothing, and the meta-test that evaluates this gate now models the term it reads. |
||
|
易良
|
1f7ed6b50a |
perf(ci): scope PR lint and reuse static build outputs (#15381)
* perf(ci): scope PR lint and reuse static build outputs * fix(lint): keep incremental selection a superset of the full lane - Select .mts/.cts: flat-config --ext is additive to the config's own files scope, so lint:ci checks them; the selector dropped them and a PR touching only such a file reported green without opening it. - Stop escalating the whole-repository ShellCheck for terminal-bench paths the lane itself greps out. - Report every failing Prettier directory in one run, since Prettier runs per directory with basename targets and the first failure aborted the rest. - Pin RUNNER_TEMP/GITHUB_* in the subprocess fixture so its setup cannot rmSync the ambient runner linter temp directory. * fix(ci): fetch incremental lint history after the shallow refetches Both 'Check the API contract version advances' and 'Report test growth' refetch shallow history below the old position; a depth-1 fetch of the base ref grafts the commit LINT_BASE_SHA names, so merge-base fails and every contract-document PR silently ran the full scan. Pin the binding and the step order, correct the freshness header that still claimed the lanes run repo-wide, and record the per-file selection backstop. * chore(ci): record the ci.yml size ratchet bump Lint & Static failed on the workflow-size ratchet: ci.yml is 151690 bytes against a recorded 147136 with a 4096-byte allowance. The overage is this PR's opt-in step plus ci.yml growth that landed on main without a same-PR bump, which is the drift the ratchet exists to surface. The comment above the fetch step was cut to four lines with the mechanism moved into the design docs, and the recorded size now matches the file. |
||
|
Shaojin Wen
|
07835d248f |
fix(ci): pin and cache the standalone runtime archives (#15159)
* fix(ci): pin and cache the standalone runtime archives The standalone runtime regression downloaded Node 24.19.0 and Bun 1.3.14 on every run, after first fetching Node's checksum list and Bun's npm metadata, with no retry. That cost minutes per run on the shared runners and turned an npm 502 into a red required check on an unrelated change. Pin the published digests for the four supported platforms so archives are verified without metadata requests, keep verified archives in a cache directory when one is configured, and retry downloads. The required Linux unit-test job points the cache at a host-wide directory on persistent self-hosted runners. * fix(ci): keep a verified runtime download when the cache is unusable If the cache directory, or a directory on its path, cannot be searched, writing the archive fails as intended, but removing the temporary file then throws too (EACCES, or ENOTDIR when the path runs through a file): `force` only ignores a missing file. The verified archive already in memory was discarded, the regression never started, and the script tests chained after it were skipped. Make that cleanup best-effort like the rest of the cache write. The download-and-cache logic moves unchanged into scripts/lib so it can be tested; the tests cover cache reuse, replacing a copy that fails verification, a cache path through a file, and an unsearchable cache directory (skipped as root, where permissions do not apply). * fix(scripts): retry unverified runtime downloads and sweep the host cache Verify each download attempt, so a truncated or stale 200 body is retried like a failed request, and name both digests when the pin never matches. Retry logs now include the undici error cause, and cache hits and writes each log a line. The CI step reports the cache path and size. Entries no run has used for a week (previous pins, partial files left by a killed run) are removed after each successful run; a run refreshes the mtime of the archive it uses, so revisions still in flight keep theirs. Tests cover retries, verification per attempt, the uncached path, the sweep, the CI wiring, and gate the permission case on the shared permission-bits probe so it also runs on root lanes with DAC caps dropped. * fix(ci): sweep the runtime cache only after a run fetched both archives The Node fetch swept every other entry unused for a week, so after a quiet week it removed the Bun archive the same run was about to reuse and Bun was downloaded again. Fetches now only mark their archive as used; ci() sweeps once after both fetches, sparing the archives it used. The sweep skips non-files and keeps going past an entry it cannot remove, and a cached copy that fails verification is logged. Also: - export the same host-wide cache to release.yml's quality_scripts, which runs the same regression on a persistent host - move the pinned versions and digests to scripts/lib/standalone-runtime-pins.js and check their shape - pin the wiring itself: the export step's gate, its GITHUB_ENV write and its position before the consuming step, in both workflows - cover the write-then-rename into the cache |
||
|
易良
|
2b05c8bd99 |
fix(ci): route web shell smoke to ARM64 independently (#15364)
* fix(ci): route web shell smoke to ARM64 independently * test(ci): pin the smoke lane's ARM64 routing in the meta-tests The runs-on change left two scripts/tests assertions pinning the old routing, so the Linux Test check and the nightly macOS/Windows lanes would go red once dependency installation succeeded (AGENTS.md guideline #3). - ci-platform-lanes: evaluate the real runs-on expression and pin self-hosted -> ARM64 plus the hosted and missing-output fallbacks, instead of the old equality with the shared test lane. - no-ak-integration-ci: replace the raw-YAML literal pin with a check that the lane still derives its runner from the classifier; the architecture override is pinned in ci-platform-lanes. - ci.yml: restore the #11250 pool history the routing change deleted, marked as x64-specific rather than a verdict on ARM64. * test: tolerate slow durable I/O in CI synchronization checks |
||
|
|
ac2c7ac93a |
fix(ci): annotate the temp short-alias guard's failing branches (#15366)
Both failing branches of the `Verify temp paths carry no short alias` step emitted a bare console.error, so a real 8.3-alias regression reached the check UI as only "Process completed with exit code 1" with no annotation attached to the run. Prefix each message with '::error::', matching the hosted-process-suite step in this same workflow and configure-windows-runner/action.yml. The casing-only branch keeps its deliberate '::warning::', and process.exitCode = 1 is unchanged. Patrol-Run: qwen-issue-patrol/jmvkv59rei6 Co-authored-by: yiliang114 <[email protected]> Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良
|
a6c98278b2 |
feat(ci): switchable Linux runner fleet (hosted/arm64) via LINUX_RUNNER_FLEET variable + dispatch option (#15329)
* feat(ci): add self-hosted-arm64 dispatch option for Linux routing workflow_dispatch linux_runner gains a 'self-hosted-arm64' choice that routes all Linux lanes to runners labeled [self-hosted, linux, arm64] (the experimental Hangzhou Alfa ECS fleet). Production lanes (push, pull_request, merge_group) keep the existing x64 ecs-qwen routing; the arm64 fleet is only reachable through manual dispatch while ARM64 compatibility is evaluated. * feat(ci): allow fleet switching via LINUX_RUNNER_FLEET repo variable pick_runner now reads vars.LINUX_RUNNER_FLEET for PR/push/merge_group automatic routing: 'hosted' forces GitHub-hosted ubuntu-latest, 'arm64' routes to the experimental ARM64 ECS fleet, any other value keeps the x64 ecs-qwen default. The dispatch linux_runner input still takes precedence over the variable, and MAINTAINER_ECS_RUNNER_DISABLED=true remains the hard hosted override. Unset variable behaves exactly as before. |
||
|
qwen-code-dev-botandqwen-code-dev-bot
|
416bed0e12 |
fix(ci): gate the Windows Hosted suite on a successful install (#15287) (#15290)
Co-authored-by: qwen-code-dev-bot <[email protected]> |
||
|
易良
|
21093a78f1 | ci: keep a green suite green when the JUnit report is unreadable (#15322) | ||
|
易良andQwen-Coder
|
d6abebb40c |
fix(ci): report swap and a df -h fallback in the lint lane failure dump (#15095)
* fix(ci): report swap and a df -h fallback in the lint lane's failure dump The lint lane's state-at-failure dump asked /proc/meminfo for MemTotal and MemAvailable only, and ran df -hT with no retry, while the three sibling dumps in the same workflow grep SwapTotal|SwapFree and fall back to df -h. A swap-exhausted death on the contended pool therefore uploaded an artifact that could not show it, and a runner whose df has no -T uploaded no filesystem table at all. Witness it by executing the step body rather than pinning its text: the new case stubs a df that rejects any option cluster carrying T, so the marker can only come from the -h retry, and asserts both swap fields reach the artifact. Red before (the artifact held MemTotal/MemAvailable only and no filesystem table), green after. Co-authored-by: Qwen-Coder <[email protected]> * test(ci): pin the retained dump fields and the three sibling dumps R1-1: the swap/df-fallback case asserted only what the PR adds, so the halves of the two edited lines it kept were unguarded. Collapsing the retry to `df -h || true`, or reordering it to `df -h || df -hT || true`, stayed green while the artifact permanently lost the filesystem Type column; narrowing the grep to swap alone stayed green while the lane this PR fixes stopped reporting RAM at failure. Give the stub's `-hT` arm its own marker so "retried after -hT failed" is distinguishable from "never tried -hT", and assert both RAM fields beside the swap ones. R1-2: the same dump is replicated at four sites and nothing pinned them equal, so dropping the swap fields or the `|| df -h` retry from test, test_macos or test_windows shipped green. Pin the three siblings textually, extending the sampler pin loop idiom already in this file -- textual rather than executed because test_windows greps /proc/meminfo, which Git Bash lacks. Test-only: .github/workflows/ci.yml is byte-unchanged by this commit (md5 31b18cbbaeb407d9348e1f3f51f1c0b4). Verified with a nine-mutant battery run one mutant at a time against ci.yml, restoring and md5- checking after each: all nine are now killed, each by a named assertion. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmvik9qbdof --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
Shaojin Wen
|
b6195e1bba |
fix(ci): run the trusted web-shell browser smoke on the ECS pool (#14471)
* fix(ci): run the trusted web-shell browser smoke on the ECS pool The web-shell E2E smoke is the second-largest user of the hosted ubuntu pool: on 2026-10-17 it took about 27% of hosted minutes, two thirds of that from same-repo PRs, while the hosted pool sat at its 20-job org cap and the ECS pool idled. #11250 moved the job hosted after four pool attempts failed in run 34081409590. Three of them ran the smoke with Playwright's unpinned default of half the host's cores, 8 workers against hosted's 2, and timed out broadly on visibility and clock waits across long-standing tests. The fourth's transcript gate took 73 s against its 60 s budget while npm install alone took 28 min on that host. Route the job like test's again, so trusted runs use the pool and untrusted fork PRs stay hosted, and pin the smoke at 2 workers, the hosted default, so a pool job no longer scales its browser count to the host. Drop the comments that described the pool steps as retained for a future reroute. * fix(ci): recreate runner Git metadata before the smoke checkout The smoke job now checks out on the reused ECS workspace, so it needs the same pre-checkout .git removal as test, lint_and_static, integration_cli and integration_no_ak. Otherwise actions/checkout reuses whatever the previous job left, including a broken shallow boundary or a live post-checkout hook. Hold the job to that rule in qwen-runner-checkout-workflow.test.js. * fix(ci): install WebKit's system libraries on pool hosts that lack them The first pool run of the web-shell smoke (run 39030597470, ecs-qwen-hk3-11) passed 272 of 286 tests in 10.6 min at 2 workers, and the transcript gate took 17.8 s against its 60 s budget. All 14 failures were mobile-webkit browser launches: the host lacked GTK 4, GStreamer, libwoff2 and about 40 more WebKit libraries, which Playwright's install step had already reported in its host validation. When that validation reports missing libraries, run Playwright's install-deps for Chromium and WebKit through sudo -n. An APT_CONFIG adds a dpkg lock timeout and the hosted step's network timeouts. The packages persist, so each host installs them once and later jobs skip apt. The install step gets the hosted step's 30-minute bound. |
||
|
Shaojin Wen
|
20de6bf12e |
fix(ci): give the hosted Test lane the same 120-minute ceiling as ECS (#14474)
* fix(ci): give the hosted Test lane the same 120-minute ceiling as ECS The full unit suite no longer fits 60 minutes on GitHub-hosted runners. Those runners take fork PRs from untrusted authors and everything when the ECS kill-switch is on. On 2026-10-17, of the hosted full-profile Test runs that ended on their own, 13 passed in 45-59 minutes and 17 were cancelled at exactly 60 while still passing tests. Job 117375307261 is a typical run: install 7.4 min, cli 32.5 min, core 10.2 min, then cancelled six minutes into web-shell with the root scripts tests still to run. Each cancellation burns 60 hosted minutes and usually a manual rerun. Drop the per-pool split and use 120 on both pools. The test step's own 110-minute cap still bounds a genuine hang. * ci: record the Test lane's fixed 120-minute ceiling in the CI budget check-ci-budget sums only fixed timeout-minutes and counts an expression as a job without a fixed timeout. Turning the test job's per-pool expression into a fixed 120 therefore moves ci.yml from 180 minutes with five expression timeouts to 300 with four. The effective ECS ceiling is unchanged; the hosted one rises from 60 to 120 as intended. |
||
|
易良
|
4d835faee5 |
fix(ci): document the shared-group supersede policy and keep main uncancellable (#14286)
* fix(ci): let main pushes supersede under the shared concurrency group A concurrency group holds at most one running plus one pending run, and a newer pending run replaces the older one even while cancel-in-progress reads false. Back-to-back merges to main therefore already dropped each other's run, only silently and one merge behind. Make the intent explicit: only the newest tree matters and it covers every merged change — the same stance e2e.yml already takes (#5859). Release branches stay uncancelled. * fix(ci): keep main uncancellable and document the pending-replacement policy Review on #14286 showed the flip cancels the in-flight main run: merges land every ~18min against this ~30min lane, no merge queue is enabled, so this run is the only fast witness of a merged tree, and its always()-gated coverage upload would publish partial reports under the complete artifact name. e2e.yml already measured that cancelling main starved the suite (67 of 100 push runs cancelled, 25 reported, #5859). Restore the main exclusion and instead make the existing behavior explicit: a newer pending run replaces the older one even with cancel-in-progress false, so back-to-back merges already coalesce onto the newest tree, and the in-flight run always finishes and reports — by design, not accident. |
||
|
易良andQwen-Coder
|
a84dbff23e |
feat(providers): add QwenCloud as the first /auth entry (#14073)
* feat(providers): add 千问AI平台 / QwenCloud as the first /auth entry Adds QwenCloud as a new top-level provider group ahead of Alibaba ModelStudio, following the product ruling in #14025: the two entries coexist rather than one replacing the other, so saved ModelStudio credentials, endpoints and model selections keep resolving to their existing presets and no migration is needed. Two new presets share the ModelStudio wizard shape (Access Method -> Platform -> API Key -> models): - qwencloud-standard (pay-as-you-go, `sk-` / `sk-ws-` keys) - qwencloud-token-plan (Credit subscription, `sk-sp-` keys) Each offers China (千问AI平台) and International (QwenCloud) as an explicit platform choice, so the UI language never decides the endpoint. Coding Plan is not offered here — it is a ModelStudio product. Endpoints were taken from the platform's own documentation and probed before landing (all four return the DashScope `invalid_api_key` error envelope): - https://maas.qianwenaiapi.com/compatible-mode/v1 - https://token-plan.maas.qianwenaiapi.com/compatible-mode/v1 - https://maas.qwencloudapi.com/compatible-mode/v1 - https://token-plan.maas.qwencloudapi.com/compatible-mode/v1 The new hosts are added to the DashScope provider detection and the web-search endpoint allowlist so the built-in `web_search` backend keeps working; both new env keys are added to the no-AK CI gate's cleared-env list. Group wiring is mirrored across the ink and OpenTUI dialogs, the VS Code provider picker, and the serve catalog consumed by Web Shell. Closes #14025 Co-authored-by: Qwen-Coder <[email protected]> * fix(cli): add QwenCloud auth strings to en and zh-TW locales The new auth entries introduced in the previous commit were only added to zh.js. Since en.js is the source of translation keys and both zh and zh-TW enforce strict key parity, the i18n check failed with "Extra key in zh.js" for all eight strings. Add the same keys to en.js as identity mappings and to zh-TW.js with Traditional Chinese values, so all three locale files have an identical key set. * test(cli): follow the QwenCloud row in the OpenTUI auth menu tests The new first /auth entry shifts the main menu by one row, but the tests addressed the sub-menus by position, so most flows landed a row early and the region-height assertions still named the old first row. * feat(providers): localize the QwenCloud /auth entry label per locale The top-level /auth entry was the only mixed CJK/Latin label in the menu ("千问AI平台 / QwenCloud"), and because en/zh mapped that key to itself it was effectively hardcoded mixed in every locale. Route it through i18n instead: en and other locales show "QwenCloud", zh shows "千问AI平台", zh-TW shows "千問AI平台". Locale-neutral surfaces (serve catalog, VS Code provider picker) carry the single Latin brand; the Chinese brand stays visible in the explicit platform step ("China — 千问AI平台") and the docs. The step-title flowTitle now goes through t() so the localized brand also appears in "… · Step 1/3 · …". Co-authored-by: Qwen-Coder <[email protected]> * fix(providers): register QwenCloud hosts in telemetry and reasoning classifiers The QwenCloud MaaS hosts and env keys were registered in the two host gates this PR edits, but two further independent classifiers of the same hosts did not learn about them, and both misclassify the entry this PR makes the first-position default: - telemetry/gen-ai-provider.ts had no providerFromHostname match for maas.qianwenaiapi.com / maas.qwencloudapi.com and no PROVIDER_BY_ENV_KEY entry for QWENCLOUD_API_KEY / QWENCLOUD_TOKEN_PLAN_API_KEY, so resolveGenAiProviderName fell through to its `default: return 'openai'` arm. Every span and log record for a QwenCloud install was attributed to OpenAI, while the identical ModelStudio path resolved to 'dashscope'. - core/reasoning-overrides.ts keeps its own inline hostname regex without the new hosts, so a user-authored partial reasoning override on a qwen* model behind a QwenCloud baseUrl selected 'qwen-chat-template' -> toggleOnly -> invalidReasoning, which captureLatestReasoning catches and warns. The override was silently discarded; the same declaration on dashscope.aliyuncs.com resolves to 'dashscope-effort'. Also widen the SDK-side mirror DaemonAuthProviderCatalog.groups[].id to include 'qwencloud', which packages/cli/src/serve/types.ts and auth-provider-helpers.ts already emit as the first group, and derive web-shell's AuthGroupId from that union instead of re-declaring it, so the SDK type is the single source of truth. A second literal union in web-shell is why nothing reddened in-repo while the daemon emitted a group id its own published wire type declared impossible. The new suffixes go inside the existing anchored (^|\.)...$ group, so a forged maas.qianwenaiapi.com.evil.com cannot match while the legitimate token-plan. subscription subdomains still do. Both env keys map to the existing 'dashscope' provider value rather than a new one. The hosts stay literals in gen-ai-provider.ts rather than importing QWENCLOUD_MAAS_HOSTS: that module is a zero-import telemetry leaf, and dashscope.ts already imports core/reasoning-overrides at runtime, so importing back would cycle. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmv8fwg2z5f * test(providers): pin QwenCloud host classification and registry membership Cover behaviour this PR added that no test observed: - dashscope.test.ts: the QwenCloud MaaS pay-as-you-go and token-plan hosts are DashScope origins, plus dot-boundary lookalike negatives. - web-search.test.ts: the same four hosts pass the web-search gate. - qwencloud-standard/token-plan.test.ts: both presets are registered in ALL_PROVIDERS and QWENCLOUD_PROVIDERS, and the qwencloud access-method group excludes Coding Plan (replaces a constant-true id/label check). Each new assertion is mutation-verified: dropping isQwenCloudOrigin from the dashscope return, weakening its host match, dropping the QWENCLOUD_MAAS_HOSTS spread, removing the two registry entries, or flipping codingPlanProvider uiGroup each turns exactly the intended test red. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmv8mbxgn5t * test(providers,cli): pin QwenCloud preset separation and the auth catalog group Coverage for lines this PR itself added (review round 1: R1-21, R1-26, R1-15): - qwencloudStandardProvider.ownsModel is cross-checked against its QwenCloud Token Plan sibling and against a foreign envKey on its own endpoint. The two presets share uiGroup, catalog and protocol bucket, so that envKey gate is the only discriminator; dropping it kept the suite green while applyModelProvidersPatch would claim a Token Plan install. - buildAuthProviderCatalog gets its first test: the QwenCloud group is first and carries both access methods. A misspelled group filter stayed green. - The zero-row main-menu paint test now names row 0, which this PR re-pointed from Alibaba ModelStudio to QwenCloud while only rows 1 and 2 stayed pinned. Each assertion was confirmed load-bearing by one mutation that the pre-existing tests did not catch. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmv8oh38g5y * docs(users): sweep the /auth menu enumeration in overview and quickstart This PR re-numbered the /auth menu to four options in auth.md and model-providers.md, but overview.md and quickstart.md still described three entries with Alibaba ModelStudio as the recommended one (review round 1: R1-4). The doc set contradicted itself, and a quickstart reader holding a QwenCloud key was routed to the ModelStudio entry, whose endpoints reject it. Both pages now list QwenCloud first and reserve the "recommended setup" claim for it, matching auth.md. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmv8oh38g5y * fix(providers): validate daemon credentials and close QwenCloud follow-ups (#14302) * fix(serve): validate API key format before persisting in daemon install path The CLI, ACP and VS Code surfaces each call provider.validateApiKey before persisting credentials. The daemon route (POST /workspace/auth/provider) did not, because validateApiKey is a function on ProviderConfig and cannot be serialized to the wire — so no client of this route can apply the check either. Without this guard a key the provider's own entry rejects (e.g. a Token Plan sk-sp- key aimed at the Standard API Key entry) gets persisted silently, and the user only discovers the mismatch at the first authenticated call. Fix: call validateApiKey in the route after protocol validation and before invoking installAuthProvider. Return 400 with code 'invalid_api_key' on failure, matching the other three surfaces. Closes #14257 (item S3 / R2-13). * fix(providers): address #14257 deferred Suggestions (D1/D2, T1/T3-T9, I1, P5, S6/S7) Batch fix for actionable items from #14257: Docs: - D1: Add settings.json example to QwenCloud headless setup section - D2: Add QwenCloud rows to web-search.md auto-activation table Tests: - T1: Pin flowTitle locale resolution in both dialog suites - T3/T4: Assert QwenCloud sub-menu contents (Token Plan + Standard, no Coding Plan) - T5: Add missing token-plan.maas.qwencloudapi.com endpoint to dashscope.test.ts - T6: Add QwenCloud host tests to reasoning-overrides.test.ts - T7: Assert providerState in qwencloud-standard preset test - T8: Test ownsModel baseUrl conjunct independently of envKey - T9: Test providerMatchesCredentials URL gate independently of envKey - S7: Assert QwenCloud group in VS Code AuthMessageHandler picker i18n: - I1: Add validateApiKey messages to all 9 locale files Product: - P5: Fix qwen3.7-plus modalities to include video (matches alibaba-token-plan) Code: - S6: Update docstring enumerations in dashscope.ts and web-search.ts Remaining items need human decisions: P1-P4 (wording/labels), S1-S5 (API surface), T2 (ink suite gating is a pre-existing harness pattern). * fix(auth): translate provider titles at render time Co-authored-by: Qwen-Coder <[email protected]> * test(providers): handle optional preset models Co-authored-by: Qwen-Coder <[email protected]> * test(providers): pin Token Plan providerState and ownsModel baseUrl leg Addresses four review Suggestions on this PR: - R1-10 (#14257 T7): the providerState stamp was asserted for the Standard preset only. Token Plan has two baseUrl options and a baseUrl-dependent modelNamePrefix, so stamp both routes. - R1-11 (#14257 T8): the deciding baseUrl negative for ownsModel was added to the Standard suite only. The Token Plan negatives added here differ in envKey, so the baseUrl conjunct was never the deciding factor. - R1-1: the web-search host note contradicted the two QwenCloud rows this PR adds to the table on the same page. Name the Model Studio and QwenCloud MaaS families the code actually accepts. - R1-4: the daemon route comment claimed parity with the CLI/ACP/VS Code copies. Those resolve the endpoint first and validate the resolved URL; this route validates the client-supplied baseUrl. Comment only. Both new Token Plan rows were mutation-checked one mutant at a time: dropping baseUrl from resolveProviderState, stamping the provider default endpoint instead of the requested one, and relaxing ownsModel to envKey-only each turn exactly the intended row red. An additional cross-route "versions must differ" row stayed green under all of them (model configs already embed baseUrl), so it was dropped rather than shipped inert. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmv9zmmcj86 * test: make the QwenCloud witnesses discriminating; trim daemon key once R1-9: the two Token Plan cases and the four lookalike cases never reached the dashscope host regex - both blocks short-circuited in validateReasoningDeclaration/validateReasoningCapabilities before route.baseUrl was parsed, and the block comment misstated the Token Plan endpoints as unregistered. Fold the Token Plan hosts into the QwenCloud host table with a declaration that omits `thinking`/`profile`, so all four registered endpoints now pin the anchored suffixes. R1-5/R1-6: the Ink VIEW_TITLES fix had no CI-executed guard (every keystroke case is skipped under CI=true) and its T1 case asserted only the already-lazy flowTitle path. Add one ungated, keystroke-free render assertion and assert the sub-menu heading between the two keystrokes. R1-3: validate and forward the same trimmed key rather than validating the trimmed value while forwarding the raw one. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmva623ox8i * fix(auth): guard zh locale test cleanup, note hand-written provider caveat R1-7: wrap the QwenCloud zh flow-title test body in try/finally so unmount() and the locale reset still run when an assertion rejects, matching the shape of the opentui twin added in the same change. R1-2: note in the settings.json alternative that a hand-written provider entry carries no capabilities stamp, so /effort offers the generic tiers rather than the QwenCloud curated set. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmvaacf8g8p * fix(providers): address #14073 round-3 deferred Suggestions - docs/design/auth/motivation.md: QwenCloud-first /auth enumeration (S1) - docs/users/quickstart.md: name QwenCloud credentials in Before you begin (S3) - docs/users/configuration/auth.md: scope the Token Plan prefix rule to ModelStudio vs QwenCloud so Option 1 and Option 4 no longer read as contradictory (S4) - qwencloud-standard.test.ts: correct the ownsModel comment — envKey and baseUrl conjuncts each exclude the sibling independently (S5) - auth-provider-helpers.test.ts + alibaba-standard.test.ts: pin every catalog group's providerIds and the alibaba uiGroup (S6) - web-search.test.ts: forged-suffix boundary cases for the QwenCloud host gate (S7) S2 (6-latin-locale backfill) and S8 (web-shell group dict) verified as no-ops: the sibling ModelStudio strings rely on the identical raw-key fallback in the same locales/dictionary. * fix(auth): localize provider API key errors * fix(providers): seed Token Plan only with models its endpoint serves A live GET /models on the China Token Plan route (real subscription key, 2026-10-16) serves neither kimi-k3 nor deepseek-v4-pro-0813 — the endpoint offers kimi-k2.x and deepseek-v4-pro instead, matching the platform doc's personal-edition table. The shared QWENCLOUD_MODELS seed list therefore pre-filled two ids the wizard would install as dead entries whenever discovery succeeded, and offered them in the offline fallback. Filter the shared catalog down to the five confirmed-served models for the Token Plan preset instead of duplicating stamps; the Standard (pay-as-you-go) catalog keeps the full list, which no live data has falsified. Pinned by a new preset test asserting the exact seed ids. --------- Co-authored-by: Qwen-Coder <[email protected]> * fix(auth): preserve explicit models during provider discovery --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良
|
631b86023b |
fix(ci): route post-merge classifier to ECS and yield CLI workers (#14293)
* fix(ci): route post-merge classifier to ECS * fix(ci): let CLI workers drain RPC replies between tests Yield with a captured real timer so synchronous test batches cannot starve progress acknowledgements, even when fake timers are installed. Keep unhandled errors fatal on Linux. * test(cli): clean up mounted replay fixtures |
||
|
易良
|
c3012a523f |
fix(ci): restore a meaningful Windows merge gate (#14298)
* fix(ci): restore a meaningful Windows merge gate Keep the known workspace baseline visible while making Windows installer/script coverage and hosted process checks authoritative again. * fix(ci): gate Windows on installer security coverage Keep the broader failing baselines visible while making the verified Windows installer and Hosted process contracts authoritative. |
||
|
jinyeandQwen-Coder
|
28415ddff3 |
refactor(managed-agent): Consolidate server and broker sources (#14277)
* refactor(managed-agent): Consolidate server and broker sources Move the combined Java application beside sdk-java and remove the standalone Runtime Broker artifact. Update build, development, packaging and CI references while preserving Java package identities, runtime protocols and schema resources. Co-authored-by: Qwen-Coder <[email protected]> * docs(managed-agent): Preserve historical evidence and verification guidance Restore commit-pinned H3 probe inputs and align the Java verification environment contract with the consolidated server layout. Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): Run hosted Broker tests under Bash Keep Maven property arguments intact on Windows and stop before Broker tests when the SDK install fails. Exercise both successful and failed SDK installation in the workflow regression tests. Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): Quote the hosted Broker shell value Follow the repository yamllint single-quote rule without changing the parsed workflow configuration. Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
813840b1ea |
ci: ratchet the PR-stage budget instead of only workflow file size (#14213)
* ci: ratchet the PR-stage budget instead of only workflow file size
The two gates that bound CI growth today measure the wrong dimension.
check-workflow-size.sh ratchets workflow file BYTES and check-test-growth.mjs
counts test LINES, so a new PR-stage lane can be added with nothing objecting.
On main at
|
||
|
易良andQwen-Coder
|
c1b012539b |
fix(ci): repair daily failure reporting and contract guards (#14238)
Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
70407feeb9 |
ci: run Java and Desktop checks daily or on demand (#14205)
* ci: run Java and Desktop checks weekly or on demand Move Java compatibility and integration checks out of PR and push events. Group Desktop Shell checks with the existing packaging dry run, replace daily packaging with weekly validation, and retain manual dispatch. Co-authored-by: Qwen-Coder <[email protected]> * ci: report weekly Java and Desktop failures as issues Include both weekly suites in the existing main failure analyzer and reuse its issue deduplication and autofix routing. Co-authored-by: Qwen-Coder <[email protected]> * ci: run Java and Desktop checks daily Co-authored-by: Qwen-Coder <[email protected]> * ci: format failure reporting workflow list Co-authored-by: Qwen-Coder <[email protected]> * ci: preserve lightweight guards after daily suite migration Co-authored-by: Qwen-Coder <[email protected]> * ci: align lint permission assertion and Java concurrency Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
ytahdnand钉萁
|
df5470e430 |
feat(web-shell): Add workspace file browser and previews (#13878)
* feat(web-shell): Add workspace file browser and previews * fix(web-shell): Restore source preview CI fixtures and notices * fix(web-shell): Address file pane review blockers * fix(web-shell): Resolve review regressions and strengthen file checks --------- Co-authored-by: 钉萁 <[email protected]> |
||
|
易良andQwen-Coder
|
aceb7f1223 |
fix(ci): recover reused runner checkouts and Java lock waits (#14017)
* fix(ci): recover reused runner checkouts and Java lock waits Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): refresh Git metadata across Linux validation lanes Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): refresh checkout metadata for triage jobs Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
|
3f56f74a6a |
feat(managed-agent): implement durable Hosted Hooks (H2) (#13129)
* feat(managed-agent): implement durable Hosted Hooks (H2)
* style(runtime-broker): remove blank line rejected by Checkstyle
* test(managed-hooks): align CI guards with H2 enablement
* fix(managed-agent): select current Turn by admission sequence
* fix(hooks): keep managed command environment out of argv
* fix(serve): close Hosted Hook recovery and capacity gaps
* fix(serve): settle cancelled startup hooks and identify deletions
* fix(hooks): settle cancelled callbacks and retry journal commits
* test(hosted): exhaust identical Store commit retries in fault gates
* fix(hooks): retain HTTP completion evidence after managed cancellation
* fix(hooks): settle cancellation before occurrence planning
* fix(hooks): settle proven-unstarted pre-tool cancellations
* fix(hooks): recover initialization and bounded cancellation results
* fix(hooks): settle cancelled initialization and bounded tool results
* fix(ci): bound hosted browser dependency installation
* fix(ci): record the ci.yml growth in the workflow size baseline
The hosted-browser install step adds two lines to ci.yml, taking it to
147136 bytes: 4110 over the recorded 143026 and 14 past the 4096-byte
allowance, which is why "Check workflow file size" failed. Most of that
growth is main-side drift the allowance had been absorbing, but this PR
touches the file, so the ratchet's stale-baseline leniency does not apply
and the number has to move in the same PR. The growth is real — a bounded
apt/Playwright install — so bump the entry instead of shrinking the
workflow.
Verified locally: .github/scripts/check-workflow-size.sh exits 0 against
base
|
||
|
tanzhenxin
|
a7deb01bcb |
ci: report the test code each pull request adds (#13149)
Nothing counted the test code a PR adds: the size rules count production lines only. Lint & Static now writes each PR's test and production lines for core, cli and web-shell to the job summary, measured from the true merge base, and warns at 2,000 or more net test lines. It reports only; nothing blocks on it. |
||
|
易良andQwen-Coder
|
7443411ba3 |
feat(desktop): add a linux-aarch64 leg to the Desktop release matrix (#12833)
* feat(desktop): add a linux-aarch64 leg to the Desktop release matrix ARM64 Linux has no Desktop artifact: the build matrix stops at x86_64-unknown-linux-gnu, so desktop-latest.json carries four platform keys and an ARM64 installer that trusts the feed falls back to the x86_64 AppImage, which cannot execute (#12806). The matrix row is not the whole change. Both manifest scripts select a Linux AppImage by extension alone and require exactly one match, so a second Linux leg breaks publish *after* every build succeeded: Expected one updater artifact for linux-x86_64, found 2 The Linux selection is now arch-explicit in create-desktop-update-manifest.mjs (which gains the linux-aarch64 key), in create-electron-bridge-manifest.mjs, and in the Electron bridge's release-assets glob. Tauri's AppImage arch tokens are `_amd64`/`_aarch64` while the .deb beside them uses Debian's `_amd64`/`_arm64`, so the two Linux legs cannot be told apart by the Debian token alone. The arm64 leg builds on ubuntu-22.04-arm rather than ubuntu-24.04-arm so both Linux artifacts keep one glibc floor (2.35); 24.04 would need 2.39 and drop Ubuntu 22.04 / Debian 12 arm64, which the x64 artifact supports. Re-mirroring an already-published release through sync-desktop-to-oss.yml has to reproduce the feed that release actually shipped, so that path passes --allow-missing-platform linux-aarch64. The fresh-build path stays strict: a leg that failed to upload must fail the mirror, not be published as missing. test-release.js gains a check that the matrix's rust targets and the manifest's platform keys stay in sync — a leg nobody taught the manifest about ships silently, which is the failure this issue reports. Fixes #12806 Co-authored-by: Qwen-Coder <[email protected]> * fix(desktop): close the round-1 review follow-ups on the arm64 leg All six round-1 suggestions, plus two gaps found while re-checking the change independently: - Warn on stdout when --allow-missing-platform actually drops a key. A tolerant mirror run was byte-identical in the log to a complete one, so a feed published without linux-aarch64 was only discoverable by diffing it against the previous mirror -- and the OSS feed is the first updater endpoint, so arm64 clients would not fall through to GitHub either. - Accumulate repeated flags in parseArguments. The new option's two spellings disagreed: comma worked, repeated flags silently kept only the last value and then failed with an error naming a build leg. - List create-electron-bridge-manifest.mjs in the desktop_shell changed-files filter beside its sibling. test-release.js is the only test either feed script has and it runs in that job, so a PR touching only the bridge script skipped its own test and reported green. That was harmless while one Linux AppImage existed; pinning the selector to `_amd64` is what made the filter load-bearing. - desktop README: the Electron bridge publishes the *x64* Linux AppImage, which stopped being unambiguous once a release carries two. - Tests: pin the manifest_args expansion rather than only its pieces, pin that the publish step never tolerates a missing leg, pin that the filter lists both scripts, and pin both spellings of the multi-valued option. - test-release.js: reuse the runManifest helper for the trailing invocation it subsumes. Every new assertion was mutation-tested: dropping the expansion, making publish tolerant, unlisting the bridge script, reverting the flag accumulation and removing the warning each turn a suite red. Co-authored-by: Qwen-Coder <[email protected]> * docs(desktop): stop the parity-guard comment overclaiming what it catches The comment said a commented-out entry stops counting as published. That holds for the multi-line `[platform, selectArtifact(` entries, including the linux-aarch64 entry this PR adds, but not for a single-line entry behind a `//` prefix: the pattern is unanchored, so `// ['windows-x86_64',` still matches and still counts. Measured both shapes before narrowing the wording rather than the guard: comment out the multi-line linux-aarch64 entry -> published loses linux-aarch64, assert.deepEqual against the matrix fails: "every build matrix leg needs an updater feed entry" comment out the single-line windows-x86_64 entry -> published unchanged, suite stays green Tightening the regex to reject a `//`-prefixed single-line entry would add machinery for an edit to a pre-existing entry this PR does not touch, so the comment now states the guarantee the code actually gives. Co-authored-by: Qwen-Coder <[email protected]> * fix(desktop): report a tolerated platform only once the feed is published Round-2 review follow-ups on the linux-aarch64 release leg: - create-desktop-update-manifest.mjs: emit the ::warning::no updater artifact annotation only after the feed has been written. It fired at selection time, so a run that later threw on a different leg published an annotation claiming an incomplete feed went out while no feed existed at all. - create-desktop-update-manifest.mjs: scope repeat-accumulation to --allow-missing-platform, the only genuinely multi-valued option. Accumulating for every option comma-joined a duplicated single-valued flag straight into the signed feed (version 0.1.0,0.1.0, which no updater client parses as semver) or wrote a file named f,f so no feed existed, both at exit status 0. Single-valued options keep the ordinary last-wins override. - test-release.js: pin tolerated-and-present, the normal case on the only production caller (the OSS re-mirror path), so dropping the matches.length === 0 conjunct cannot silently omit linux-aarch64 from the mirror feed; plus the refused-after-tolerating and duplicated-flag cases. - desktop-isolation.test.js: read the changed-files filter alternatives out of the anchored grep -Eq alternation instead of substring-matching the whole run script, and pin that the lane the filter gates still runs test-release.js. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmujyzhfka1 Co-authored-by: Qwen-Coder <[email protected]> * test(desktop): pin the strictness split by containment and by feed location Two test-discrimination gaps raised in round 3 review. desktop-oss-workflow.test.js pinned the SOURCE env, the guard line, the tolerant assignment and its expansion separately, but nothing pinned that the tolerant assignment sits inside the release guard. Hoisting it into the unconditional declaration leaves every asserted string present and the if block merely redundant, so a SOURCE=artifact fresh build whose arm64 leg failed to upload would publish desktop-latest.json without linux-aarch64 and still exit 0. Assert the guarded block as one contiguous string. The duplicatedOutput case asserted only status 0, which caught a comma-joined --output solely because the fixture path is absolute and the write dies with ENOENT on a parent directory that does not exist. Both production callers spell --output relatively after a cd (sync-desktop-to-oss.yml:135 and desktop-release.yml:721), and there the comma-joined value is a legal filename: the run exits 0, writes desktop-latest.json,desktop-latest.json, and leaves no feed at the path the upload step checksums. Delete the feed before the run and read it back the way the sibling duplicatedVersion case does, then repeat the run with the relative production spelling in its own cwd. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmuk7k4kaah --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
Shaojin Wen
|
0a136f89d7 |
test(managed-agent): Close the deferred Hosted no-tool gate follow-ups (#12864)
The SDK Java integration tests are split by name: every Hosted*IT runs only in the MySQL job, every other *IT only in the MariaDB job. The workflow guard checked that split as text, so a profile <includes>, or -Dit.test passed through MAVEN_ARGS or .mvn/maven.config, could narrow either job while Maven, the job and the guard all stayed green. After its tests, each job now runs scripts/check-failsafe-reports.js, which compares the failsafe reports with the source tree. Every *IT class of the job's family must have run a test case, and no other test class may have run. A report that records -Dit.test, -Dfailsafe.includesFile or -Dfailsafe.excludesFile, or records no properties at all, fails the check too, so a method filter that keeps one test of every class is caught as well. The MySQL job now cleans before it builds, and changes to the script or to any .mvn directory trigger the workflow. The Windows lane ran only the two portable startup cases. It now runs every Hosted case after its unit tests, with a 10-minute budget, and fails unless every collected case passed. The macOS lane keeps the portable subset, since its E2E legs run the whole file. The Hosted helper and the daemon harness each had their own listening-banner match and SIGTERM-then-SIGKILL teardown, with a 3 s grace in one and 5 s in the other. Both now use integration-tests/helpers/daemon-process.ts, which imports only Node built-ins. Teardown waits for the child to exit and returns at once for a child that has already exited, including one killed by a signal; the harness used to wait out the full grace for such a child. A new Hosted case fails if the caller's HOME or QWEN_HOME, or its whole environment, reaches the child. The route-probe comment now states its invariant. The bilingual design doc corrects when the temp-root sweep runs, and records the lane changes, the Java check and its limits, and this change's evidence. The workflow guard pins the new steps. The sdk-java.yml size baseline records the file's new size, since main had already grown it 3,485 bytes past the recorded one. Refs #12823 |
||
|
qwen-code-dev-botandqwen-code-bot
|
ff5148e69e |
fix(ci): retry the helper-tests step once against pool contention (#12772) (#12780)
* fix(ci): retry the helper-tests step once against pool contention (#12772)
The Lint & Static lane failed on main at
|
||
|
Shaojin Wen
|
c7c6def8bc |
test(managed-agent): add Hosted no-tool process gates (#12733)
* test(managed-agent): add Hosted no-tool process gates Refs #12728 * fix(ci): satisfy YAML quoting rules in hosted process gates |
||
|
|
e944a23900 |
feat(managed-agent): admit workspace-bound sessions without execution (#12709)
* feat(managed-agent): admit workspace-bound sessions without execution * fix(managed-agent): preserve terminal SSE delivery * fix(managed-agent): keep workspace lookups indexable * fix(managed-agent): correct workspace lookup bindings * fix(managed-agent): validate workspace identifiers at API edge * fix(managed-agent): address admission review and CI failures * fix(managed-agent): preserve negative stream cursor validation * fix(managed-agent): preserve writer lease precision across JDBC timezones * fix(ci): honor file permissions in root-run unit tests --------- Co-authored-by: jinye.djy <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> Co-authored-by: Shaojin Wen <[email protected]> |
||
|
易良andQwen-Coder
|
73aa65a4b4 |
chore(desktop): rename packages/desktop-shell to packages/desktop (#12653)
* chore(desktop): rename packages/desktop-shell to packages/desktop The Electron-era packages/desktop was removed in #9085 and the OpenWork downstream has forked, so the short name is free and the shell suffix is the only thing left of the transition. Rename the package directory and every load-bearing reference: workspace negations (npm + pnpm), the ci.yml crate filter, desktop-release.yml, the isolation guard, lint/ignore configs, scripts tests, CLI workspace-fixture tests, and living docs. User-facing identity is untouched: the crate stays qwen-code-desktop, productName stays "Qwen Code Desktop", the bundle identifier, updater feed, release tag scheme, and the legacy electron-bridge asset names are all unchanged. check-desktop-isolation now guards both path names so the old one cannot re-enter a workspace set. Dated design docs under docs/design/ keep their historical paths deliberately. Refs #8596 * docs(design): follow the rename in the undated notification design notes The web-shell-browser-notification-details pair is a living doc without a date prefix, so it falls under the PR's update rule rather than the historical-record exception. * docs(desktop): stop citing paths the rename invalidated Three prose sites and one CI comment still described the pre-rename world, or contradicted themselves once `packages/desktop` named both the removed Electron app and the living Tauri shell. - ci.yml `desktop_shell`: the comment this branch added credited the Cargo.toml guard with keeping "branches from either side of the rename" green. It does not. A pre-rename head's files sit under `packages/desktop-shell`, which the changed-files filter no longer matches, so the job skips before the guard ever runs. Say that, and say what the guard does cover (a head the filter matched because ci.yml itself changed, with no crate at the new path). - docs/design/web-shell-user-message-links.md: "desktop-shell routing" / "desktop-shell clicks" -> "desktop-host", the wording this branch already chose in LinkifiedText.tsx for the same rule. - docs/design/telemetry-runtime-client-attribution-design.md: repoint the `runtime.rs` citation. It backs the live `QWEN_CODE_DESKTOP=1` claim, and the other document of record for that invariant -- the comment in acp-channel-fallback.ts -- was already repointed by this branch, so the two disagreed. Only the path token changes; the doc's dated findings are untouched. - brand-create.mjs / SKILL.md / src-tauri/src/main.rs: name the historical referent as historical (the Electron `packages/desktop` removed in PR 9085) instead of letting it read as a self-reference. In main.rs the citation `packages/desktop/packages/shared/src/config/storage.ts` now looked like a subpath of the file's own package. Rewritten, not deleted: the `QWEN_DEFAULT_WORKSPACE_DIR` clause next to it is the only in-code documentation of that override and still resolves. Deliberately not swept: `scripts/check-desktop-isolation.js`'s `packages/desktop-shell` tripwire, ci.yml's #8132 history comment, `docs/design/9152-architecture-invariant-classification.md` (names both spellings and is accurate), and the date-prefixed `docs/design/2026-07-31-desktop-web-shell-release.md`. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmug1iw422v * test(scripts): pin the desktop crate path and the isolation guard's coverage Both invariants fail green today, and this rename is the second time the crate directory has moved. `scripts/check-desktop-isolation.js` gained `'packages/desktop'` in `nativePrefixes` -- the only production-logic line in this branch -- and nothing read it. Deleting that line leaves the guard's output and exit code byte-identical, because `isNativeLocation('packages/desktop')` then matches no remaining prefix; a later PR that drops `!packages/desktop` from the root workspace list would re-admit the package with CI green. The `desktop_shell` job names the crate directory in five places (the changed-files filter, the Cargo.toml guard and its `::notice::` text, the rust-cache `workspaces:`, and two `working-directory:` values). Nothing asserted they agree with each other or with a directory that exists, so the next drift reports a skipped job as a passed one. The new suite parses both instead of re-running them, and is superset-shaped on purpose: `nativePrefixes` carries a fourth entry (the `packages/desktop-shell` tripwire) that nothing negates, and an exact-match assertion would invite deleting it. The npm/pnpm mirror of the negation list is already pinned by package-scripts.test.js, so this reads package.json only. Verified by mutation, each applied alone and then restored: reverting the filter alternative or either `working-directory` to `packages/desktop-shell`, dropping `'packages/desktop'` from `nativePrefixes`, and dropping `!packages/desktop` from package.json all turn this suite red. Today no other test does. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmug1iw422v * test(scripts): pin what the desktop-isolation tests actually claim Four review findings on scripts/tests/desktop-isolation.test.js, each verified by mutating the thing it claims to pin, before and after. - The comment above the local `isNativeLocation` copy claimed a change to the script's rule "has to be made here too". Nothing reads the rule back from the producer -- `nativePrefixesFromSource` parses the array literal only -- and narrowing the script's rule to prefix equality leaves the suite green. Describe the copy instead of claiming coupling that does not exist. - `still carries the Cargo.toml existence guard` matched only the guard's condition text. Flip the body's `changed=false` to `changed=true`, or move it after `fi`, and the suite stayed green at 5/5 while the job runs every gated step against a crate that is not there. Assert the assignment sits inside the guard's own `if ... fi` body. - Both on-disk checks keyed on the bare directory, so git-ignored residue under `packages/desktop-shell` (`node_modules`, which survives a branch switch with an empty `git status`) reddened two tests on a tree with no defect and blamed the root workspace set. Key on `package.json`, which is what `packages/*` requires to make a directory a workspace member; a real re-entry still reddens both. - The `startsWith(filterAlternative)` assertion compared a path built from `crateDir` -- `filterAlternative` with one trailing slash stripped -- back against `filterAlternative`, so it could not fail for any input. Dropping the trailing slash from the changed-files filter in ci.yml left it green. Replace it with the property that makes the alternative a path prefix rather than a bare string prefix. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmugedurp3j --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良
|
97cd5636c9 |
fix(ci): trust the job workspace as a git safe.directory before checkout (#12648)
A main CI run on ecs-qwen-hk4-19 failed in 'Run yamllint' (run 35999787358) because the leftover workspace was owned by the runner user while the job's steps run as root, so every git call failed with 'detected dubious ownership'. The pinned actions/checkout adds safe.directory only to its own temporary global config, leaving later steps uncovered: 'npm ci' prepare tolerated it, shellcheck's pipeline ends in sed so it passed vacuously on an empty file list, and yamllint died on 'xargs yamllint' with zero FILE_OR_DIR arguments. The 'Restore workspace ownership' step only chowns when the step user is non-root, so nothing reconciled the mismatch. Add an idempotent safe.directory trust for $GITHUB_WORKSPACE to that step (all six ci.yml copies, kept byte-identical) so git works for the whole job regardless of leftover ownership. The --get-all guard keeps a persistent runner's gitconfig from growing a duplicate entry per job, and the warning fallback matches the neighbouring chown/chmod ladders. The +5 KB ci.yml growth is recorded in .size-baseline per the ratchet. |
||
|
易良andyiliang114
|
a5a5eb556b |
ci(web-shell): give E2E Smoke 30 minutes (#12593)
* ci(web-shell): give E2E Smoke 30 minutes The job now runs 184 smoke tests after about 7.5 minutes of setup and ends right at its 20-minute limit: on #11206 three runs passed every test and were then cancelled during cleanup or the last test. Raising the limit stops passing runs from reporting red; sharding the suite is the fix at the source. * test(ci): pin the E2E Smoke limit at 30 minutes The lanes test records every job's limit with its reason; update the web-shell E2E Smoke entry to the new value and the runs behind it. --------- Co-authored-by: yiliang114 <[email protected]> |
||
|
|
1cc63cfed4 |
fix(web-shell): restore mobile history navigation (#12362)
* fix(web-shell): restore mobile history navigation * test(ci): update Chromium and WebKit install checks * fix(web-shell): keep browsing snapshot on duplicate history push Address review on #12362: - Move the historyRef write-back below push()'s duplicate-entry early return so a mid-browse duplicate push cannot desync the surviving browse index from the freshly-read array (adds a regression test that was red before the move). - Dispatch-test the remaining three migrated onMouseDown buttons (toolbar history, toolbar shell mode, expanded hide keyboard); each handler revert probe reddens the new test. - Pin the mobile-webkit and mobile-chromium smoke-lane projects with a contract test; deleting the project or re-scoping its grep reddens it. - Correct three Chromium-only comments falsified by the WebKit lane (mobile spec header, smoke spec parenthetical, design doc clause; the Firefox deferral stands). - Pin the unsavedHistoryRef reset on workspace switch after a failed save with the maintainer-provided witness test. Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: qwen-code-dev-bot <[email protected]> Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
64dd058c9c |
ci(pnpm): install with pnpm everywhere and retire package-lock.json (#11859)
* ci(pnpm): install dependencies with pnpm in CI and release CI tests the graph release ships only if both install it the same way, so this moves both, keeping every `npm run` script and `npm publish` as they are. - ci.yml (7), release.yml (3) and release-vscode-companion.yml (2) install with `corepack pnpm install --frozen-lockfile`; release keeps --ignore-scripts plus its explicit postinstall and generate steps. - setup-node drops the npm cache and sets package-manager-cache: false, which setup-node v5+ needs once package.json declares a pnpm packageManager. - A composite action caches the pnpm store on the three always-hosted jobs; the ECS pool keeps its store on local disk. - version.js no longer lets npm reify node_modules: --no-workspaces-update on both `npm version` calls, and a package-lock-only refresh in step 9. - The notices generator, prepare-package's sharp pin and the standalone release's native package specs read the installed tree or pnpm-lock.yaml instead of package-lock.json keys. - New size baselines for the three workflows, which were already over them. * chore(vscode): regenerate NOTICES from the pnpm-installed tree Generated in CI (pnpm Worktree Smoke on ubuntu-latest) by the ported generator. Same entry set as before except the five packages npm and pnpm resolve differently: fdir 6.5.0, picomatch 4.0.5, mdurl 2.1.0, @types/node 22.20.1 and mime-db 1.52.0 (form-data's pinned copy). No entry lacks a license text or repository. * fix(pnpm): resolve undeclared zod peers to the repo's zod, as npm does The first CI run on this branch failed the export renderer's size budget: 1,930,015 bytes against 1,930,000, where the same commit built on npm's tree comes to 1,858,186. Comparing both trees' esbuild metafiles and prebuilt bundles put the whole difference in web-shell's transcript.js (+100,903 bytes pre-minify), and the added code is zod 4 plus the @modelcontextprotocol/ext-apps schemas; the sdk daemon bundle is identical. web-shell bundles ext-apps, whose zod and MCP SDK are peers, and declares no zod itself, so pnpm filled the peer with the highest locked zod (4.4.3) while npm serves the hoisted 3.25.76. Declaring zod 3.25.76 as a root devDependency makes pnpm resolve every such peer to it (resolve-peers-from-workspace-root is on by default), which is npm's hoisting behaviour. mobile-mcp keeps its own zod 4.4.3, as under npm; the npm tree does not change. * fix(pnpm): keep npm's root @types/node so root-level typechecks match The no-AK integration gate typechecks integration-tests/, which takes Node's types from the root node_modules. npm hoists @types/node 20.19.1 there, because acp-bridge and core pin it; pnpm, with no root declaration, hoisted 22.20.1, whose spawnSync overloads type process-registry.ts's stdout as string | NonSharedBuffer and fail TS2322. The same code passes on every npm-installed PR. Declaring 20.19.1 as a root devDependency puts npm's copy back where a root-level tsconfig looks. The npm tree does not change. * ci(pnpm): install with pnpm in the remaining workflows The follow-up to switching CI and release: every other workflow that installs the root workspace now uses the pinned pnpm with --frozen-lockfile, so no job tests a graph that release does not ship. - e2e, tui-parity, repo-hygiene, web-shell-visuals, serve-ab, sdk-java, windows-runner-smoke, qwen-autofix, the SDK releases, finalize-release, sync-release-to-oss, the root install in desktop-release and cd-cua-driver, and the sandbox Dockerfile. - setup-node drops the root npm cache (package-manager-cache: false); hosted jobs restore the pnpm store through the shared composite action, and qwen-autofix keeps af-012's rule that the persistent pool restores no remote cache. - serve-ab and web-shell-visuals install a separate base checkout that can predate pnpm-lock.yaml, so those installs fall back to npm ci there. - Out of scope, unchanged: packages with their own lockfile (cua-driver's TypeScript SDK, desktop-shell, live-host, mobile-mcp), and the triage lanes with npm-cache.yml and the npm audit gate, which move with the package-lock.json retirement. - New size baselines for the touched workflows; qwen-autofix stays under the 470000-byte gate. * fix(pnpm): link the workspace packages that root-level code imports npm links every workspace package into the root node_modules, so integration-tests/ and scripts/, which are not workspace packages themselves, can import @qwen-code/* by name. pnpm links a workspace package only where it is a declared dependency, and the root declared none, so the no-AK integration gate could not find @qwen-code/qwen-code-core. The root now declares the seven packages that root-level code imports at runtime as file: devDependencies: acp-bridge, channel-base, the CLI, core, sdk, web-shell and web-templates. npm already links them, so its tree does not change (letting npm re-resolve the lock reproduces the same root entry); .pnpmfile.mjs rewrites them to workspace:*, which pnpm links at the root. * ci(pnpm): move triage, the store producer and the root audit to pnpm - qwen-triage's verify and tmux-testing lanes install the PR with `corepack pnpm install --frozen-lockfile` from a restored pnpm store. The one retry clears node_modules first, because pnpm install, unlike npm ci, keeps it. - npm-cache.yml becomes pnpm-store.yml, keyed on pnpm-lock.yaml. - security-checks audits the root workspace with `pnpm audit`, which reads the lockfile, so the root install step goes. The vendored lockfiles keep npm, and the retry recognises pnpm's endpoint error too. - qwen-autofix drops the hosted-only store cache: its heavy jobs forbid local `uses: './...'` actions, and the file sits at the size gate. - cd-cua-driver: setup-node@v4 has no package-manager-cache input. - package-assets fixtures install sharp rather than listing it in package-lock.json, matching how prepare-package now resolves it. - AGENTS.md, CONTRIBUTING.md and the npm workspace guide install with pnpm. * feat(review): install pnpm repos in qwen review's build/test qwen review installed dependencies only for a repo with package-lock.json and reported a pnpm repo as unsupported, so reviews of qwen-code would lose their build and test once its npm lockfile goes. - lockfileInstaller() picks npm or pnpm from the committed lockfile. With both, `packageManager: pnpm@...` chooses pnpm; anything else keeps npm. - A pnpm repo installs with `corepack pnpm install --frozen-lockfile` through the same disk, budget and sandbox path as `npm ci`, and counts as installed once node_modules/.modules.yaml exists. - Scripts still run through `npm run`, and the npm path is unchanged. * ci(pnpm): retire the root package-lock.json pnpm-lock.yaml is now the only root lockfile. - check-lockfile keeps the pnpm integrity check and Playwright parity, which now reads pnpm importers and snapshots. The npm-vs-pnpm agreement and build-approval checks go: pnpm 11's strictDepBuilds and every CI install's --frozen-lockfile enforce what they covered. - pnpm-lock-freshness.yml goes; it regenerated pnpm-lock.yaml from package-lock.json. - Release: version.js drops its package-lock refresh, the release commit stages pnpm-lock.yaml and the SDK release stages no lockfile, so neither `git add` fails on the missing file. `npm version` in the SDK and mobile-mcp releases passes --no-workspaces-update so it cannot reinstall the root with npm. - cd-mobile-mcp installs from the root pnpm lockfile. setup-node in cd-mobile-mcp, qwen-code-pr-review and desktop-release no longer keys an npm cache on the root lockfile. - check-desktop-isolation reads pnpm importers; build.js, build_sandbox.js and preflight install with pnpm. - pnpm-lock.yaml, pnpm-workspace.yaml and .pnpmfile.mjs join the autofix supply-chain class and repo-hygiene's deny list; pnpm-lock.yaml joins autofix's generated-diff exclusions, sdk-java's path filter and the platform-sensitivity manifests. - .gitignore lists /package-lock.json so an npm install cannot commit it back. Tests and docs follow. * fix(standalone): read the node-pty versions from the pnpm lockfile * chore(vscode-ide-companion): regenerate NOTICES.txt after the lockfile swap * chore(vscode-ide-companion): regenerate NOTICES.txt from the pnpm dependency tree The committed NOTICES.txt was generated from an npm install and does not match the tree produced by the pnpm install the release pipeline now uses (659 resolved dependencies, differing @types/node and mime-db versions). Regenerating with the pinned pnpm install makes the freshness check reproducible. * fix(ci): drop the npm cache from the daily dependency audit actions/setup-node's npm cache keys off the root package-lock.json, which this branch retires, so the scheduled job would fail at Set up Node before the audit ran. No pull_request run reaches this workflow to catch it. The root tree is never installed here either: pnpm audit reads pnpm-lock.yaml directly. Matches the sibling workflows that already set package-manager-cache: false; scripts/tests/security-workflows.test.js now pins it. * chore(ci): stop banking main-side drift in the workflow size ratchet The ratchet exists to turn a workflow's growth into one reviewed line. This branch re-recorded entries for 17 workflows it never touches, several upward by 0.4-2.8 KB - drift that happened on main, banked into the manifest without review. Restore those to their merge-base values, so .size-baseline changes only for the 22 workflows this branch actually edits. The entries were not needed to go green: ci.yml passes the PR's base commit, so a file that drifted on main and is otherwise untouched earns a warning, not a failure. * fix(release): initialize standalone state before main * test(browser-use): read Playwright pin from pnpm lock * fix(ci): address pnpm migration review blockers * fix(desktop): stop reading the retired root lockfile --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
易良andQwen-Coder
|
c56d60438b |
fix(ci): reclaim Docker cache and clean review scratch directories (#12135)
* fix(ci): reclaim unused ECS Docker build cache * fix(ci): clean job-owned review scratch directories * fix(ci): single-quote the new review-scratch workflow scalars yamllint's quoted-strings rule (quote-type single, required) rejects the four unquoted scalars this PR added to qwen-code-pr-review.yml, which failed the Lint & Static job at the yamllint step. Co-authored-by: Qwen-Coder <[email protected]> Patrol-Run: qwen-pr-closeout/jmu6gg1w62o * fix(ci): resolve ECS cleanup review findings * fix(ci): resolve remaining ECS cleanup blockers --------- Co-authored-by: Qwen-Coder <[email protected]> |
||
|
|
ac505acbfc |
feat(omni): integrate multimodal media pipeline into main (#12019)
* ci: run PR CI for the omni-experiment base branch
PRs targeting omni-experiment need the same CI coverage as main.
Branch-local change only; main's ci.yml is untouched.
* feat(omni): S1 minimal path — local video via DashScope upload delivery (#8422)
* feat(omni): S1 minimal path — local video via DashScope upload delivery
Local @video files are recognized (magic-byte sniff + streaming SHA-256 +
ffprobe metadata), promoted into a content-addressed object store under
.qwen/omni/objects/, uploaded through the DashScope temporary upload
channel (getPolicy + OSS multipart form POST), and delivered to the model
as oss:// fileData parts resolved server-side via the
X-DashScope-OssResourceResolve header.
- omni pipeline gated on omni.enabled (or QWEN_CODE_ENABLE_OMNI=1) AND a
DashScope-compatible endpoint AND an available API key; any other
configuration falls back to the existing inline base64 behavior
- ffmpeg/ffprobe are hard runtime prerequisites when omni is enabled:
Config.initialize fails fast with an actionable FatalConfigError
- upload failures fail closed with an explanatory error instead of
silently downgrading to inline delivery
- per-file ceiling omni.upload.maxFileBytes (default 1 GiB) replaces the
10MB inline cap for omni-delivered video
- object store writes are atomic (.tmp + rename) and content-deduped;
.qwen/omni/ is self-gitignored
Part of the omni-experiment S1 slice (#8183).
* fix(omni): harden S1 gate, storage, and failure paths after review
Review round 1 findings addressed:
- gate: exclude Qwen OAuth (its ContentGeneratorConfig carries the
QWEN_OAUTH_DYNAMIC_TOKEN placeholder, unusable for the uploads
endpoint), require an explicit baseUrl so the configured credential is
never sent to a default origin, and require a trusted workspace before
writing .qwen/omni/ or uploading workspace bytes
- aborts: user cancellation propagates as the original abort error at
every pipeline stage instead of being wrapped into a delivery failure
- size ceiling: enforced from a cheap stat before any hashing/probing,
and invalid (<=0) configured limits fall back to the default
- object store: bytes are re-hashed while copying and verified against
the object key (TOCTOU fail-closed); dedup hits re-hash the existing
object and heal mismatched content; store directories refuse symlinks
- error hygiene: messages that can reach model-visible content carry
basenames only and a structured upstream error summary (status +
code/message) instead of raw response bodies
- getPolicy: validate the OSS ACL fields so a partial policy fails with
a clear error instead of an opaque OSS 403
Part of the omni-experiment S1 slice (#8183).
* refactor(omni): apply simplification review round
- reuse the shared combineAbortSignals helper (with listener cleanup)
instead of a local AbortSignal combinator
- deduplicate the stream-SHA-256 helper: storage now imports
hashFileSha256 from recognition
- collapse probeBinary's callback plumbing into a Map-based
availability cache
- drop the unreachable video/x-matroska extension branch (the sniffer
never emits that MIME type)
- move the omni read-result shaping out of fileUtils into
readVideoViaOmniDelivery so the file-reading hot path only carries
the gate check and a single delegation call
Part of the omni-experiment S1 slice (#8183).
* fix(omni): keep abort/timeout coverage through response body reads
Regression review of the previous two commits found:
- upload.ts: cleanup() of the combined abort signal ran at
headers-arrival time, leaving the subsequent response body read
(getPolicy JSON / failure summary) unabortable — a stalled body would
hang with neither the timeout nor ESC able to interrupt it. Body
handling now happens inside the signal's lifetime.
- storage.ts: the rename-race fallback could convert a user abort into
a dedup success after an unabortable full-file re-hash; aborts now
rethrow before the fallback, and the verify hash carries the signal.
- storage.ts: healing a planted entry now removes directories too, not
just files.
Part of the omni-experiment S1 slice (#8183).
* fix(cli): make Esc cancel @-preprocessing (media uploads) in the TUI
The @-resolution phase of a user query (file reads, and with omni
enabled potentially minutes of media hashing + upload) ran while
streamingState still reported Idle: no loading indicator appeared and
cancelOngoingRequest early-returned, so Esc was silently swallowed for
the whole preprocessing window.
Track an isPreparingQuery state around prepareQueryForGemini for
interactive user queries and feed it into streamingState. Preprocessing
now renders the spinner with the esc-to-cancel hint, and Esc aborts the
turn controller — the signal already flowed through handleAtCommand
into the omni upload pipeline, which cancels cleanly (verified: no
orphan temp files, no lingering sockets, session stays usable).
Scoped to SendMessageType.UserQuery: notification/cron/teammate drains
key off streamingState transitions and keep their existing behavior;
/btw side-questions leave the main stream state untouched.
Part of the omni-experiment S1 slice (#8183).
* docs(omni): add multimodal experiment architecture designs (#8110)
Four interlinked design docs for the omni multimodal experiment:
- file recognition & metadata: unified MediaRecognitionService, two
normalization trigger points, URL localization, raw-resource token
estimation, ffmpeg/ffprobe as hard dependency
- policy orchestration: MediaPolicyTool contract with per-invocation
args + per-tool settings, fixedPolicies with condition DSL, transport
guard on upload-channel limits, DashScope temporary-upload delivery
(all-upload, no inline), lossy-output disclosure contract
- memory: two collection trigger points (FileRecognized /
OmniPolicySucceeded), harness-exclusive writes, in-file graph,
active/side-query recall sharing one protocol
- managed media storage: content-addressed object store, staging /
quarantine lifecycle, upload cache (sha256+model -> oss URL, 48h),
mark-and-sweep GC rooted at memory references
Verified against origin/main and validated end-to-end against the
DashScope uploads API (123KB / 21MB / 424MB video, image, audio).
* feat(omni): S2 input expansion — image/audio/URL sources and token-dimension transport guard (#8512)
* feat(omni): S2 input expansion — three modalities, URL sources, token guard
Extends the S1 video-only upload delivery to the full S2 input surface:
- recognition: generalized to image/audio/video with content sniffing
(magic bytes for png/jpeg/webp/gif, mp3/wav/flac/ogg/m4a, plus the S1
video set) and per-modality ffprobe metadata; modality mismatches
between sniff and reference fail closed
- delivery: all modalities upload ORIGINAL bytes — no resizing or
transcoding on the default path (lossy transforms are the job of omni
policies, which must disclose); images skip renderImageOverview when
omni is active and carry a dimensions+zoom-hint text part instead
- converter: new fileData→input_audio branch passing the bare oss:// URL
(previously audio fileData silently textified as 'Unsupported file
media type' after a successful upload); getAudioFormat widened to
flac/ogg/m4a in lockstep with the recognizer
- URL inputs: @https://… is intercepted ahead of filesystem resolution
(previously silently dropped via the ENOENT skip), streamed to
downloads/ with SSRF/redirect policy mirroring fetchWithPolicy, hashed
while writing, re-recognized from local bytes, and delivered through
the same pipeline; per-URL failure isolation keeps the turn alive
- token estimation: versioned raw-resource estimator (method
raw-resource-v1) attached to read results as the single tokenEstimate
location; formula is a swappable slot pending provider confirmation
- transport guard: byte ceiling (S1) plus omni.transport.maxEstimatedTokens
(default 0 = disabled — observability only until the formula is
confirmed; positive values fail closed before any copy/upload)
- second normalization trigger point: processToolResultOmniMedia converts
inline tool-result media to oss:// fileData at both physical funnels
(CoreToolScheduler terminal sites and ACP Session.runTool), as a
sibling of the vision bridge — converted parts are invisible to
isImagePart so bridge behavior is untouched
Part of the omni-experiment S2 slice (#8184).
* fix(cli): deliver URL-only @ media instead of dropping it after upload
The no-file-paths early return in resolveAtCommandQuery did not count
urlMediaRefs, so a prompt whose only @-reference was a URL ran the full
download → recognize → store → upload pipeline and then discarded both
the delivered fileData parts and their display cards. Found by S2 E2E
verification (the model improvised around the missing image via its own
tools, masking the drop).
Part of the omni-experiment S2 slice (#8184).
* fix(omni): apply S2 adversarial review findings
Security lens:
- tool-result media funnel: per-result upload budget (8 parts /
128 MiB aggregate — a malicious tool result must not fan out
unbounded uploads from the user's account); excess parts stay inline
- modality gate on tool-result parts now keyed to the SNIFFED modality,
not the declared MIME type (a part declared audio whose bytes are a
video container no longer bypasses a video-disabled config)
- fs error messages are path-scrubbed before reaching model-visible
error text (basenames survive, absolute paths do not)
Correctness lens:
- non-sniffable media formats (bmp/tiff/heic images, exotic containers)
fall back to the baseline inline path via a cheap content pre-sniff
instead of failing closed — fail-closed remains for genuine pipeline
failures, not for formats the recognizer simply does not support
- user cancellation during URL media download ends the turn gracefully
instead of rejecting through resolveAtCommandQuery (which has no
throw contract) into an unhandled-rejection banner
Part of the omni-experiment S2 slice (#8184).
* fix(omni): keep the omni graph out of the ACP static bundle closure
The serve fast-path bundle-closure CI check failed: iconv-lite encoding
tables (553KB) became statically reachable from the acpAgent chunk.
Root cause: the CLI-side dynamic imports of the FULL core barrel
(`import('@qwen-code/qwen-code-core')` in atCommandProcessor) request
the whole namespace object, defeating tree-shaking of the barrel — which
statically re-exports sync-file-encoding → iconvHelper → iconv-lite
tables — and esbuild folds a dynamic import of an already-statically-
imported module into a static chunk edge.
Fixes:
- add a `./omni` subpath export to @qwen-code/qwen-code-core (pattern:
existing ./transcriptRecords) and point every CLI-side dynamic import
at it, so only the omni module graph is requested;
- re-export processToolResultOmniMedia from omni/index (circular-safe:
both modules only bind functions) so the subpath serves the
tool-result funnel;
- coreToolScheduler and ACP Session load the funnel via dynamic import
(mirroring fileUtils' existing pattern for the omni module).
Verified: `npm run check:serve-fast-path-bundle` passes; bisected
against base in a clean worktree to isolate the offending edge.
Part of the omni-experiment S2 slice (#8184).
* fix(build): register the core ./omni subpath in test resolver configs
The previous fix added a `./omni` subpath export to
@qwen-code/qwen-code-core, but vitest/tsc in this monorepo resolve the
core package through explicit aliases to TS sources rather than the
package exports map — so every suite loading atCommandProcessor or the
ACP Session failed with "Failed to resolve import
'@qwen-code/qwen-code-core/omni'".
Add the subpath alias alongside the existing transcriptRecords/goalWire
precedents in cli/acp-bridge/sdk-typescript vitest configs and the
cli/acp-bridge tsconfig paths.
Part of the omni-experiment S2 slice (#8184).
* test: isolate runtime-path suites from the runner's QWEN_RUNTIME_DIR
The self-hosted CI runners export QWEN_RUNTIME_DIR for their own qwen
job tooling. Storage.getRuntimeBaseDir gives that env var top priority,
so logger.test.ts (HOME-derived path expectations) and the ACP Session
runtime-pinning tests (Storage.setRuntimeBaseDir-based expectations)
fail whenever a PR lands on such a runner — independent of the change
under test. Reproduced locally with QWEN_RUNTIME_DIR=/tmp/fake-spool.
Clear the variable inside the affected suites (restored after each
test), making them hermetic on any runner.
* fix(omni): gate the omni dynamic import behind isOmniEnabled
The `await import()` of the omni graph inside processSingleFileContent and
resolveAtCommandQuery performs a real filesystem read under vitest (the SSR
transform writes and reads the module under /tmp/RP_*/ssr/), so mock-fs
based suites — pathReader's image cases — failed with ENOENT on a path they
never mocked.
Gate both call sites on the cheap synchronous config.isOmniEnabled() before
reaching for the import. Sessions without omni configured (every existing
test suite, and every user who has not opted in) now never load the module,
which also avoids the wasted module load on the common path. Delivery
behavior when omni IS enabled is unchanged: isOmniDeliveryActive still
performs the full five-condition gate check afterwards.
* fix(omni): resolve DNS before each media fetch and close review gaps
SSRF (review-blocking): downloadMediaUrl's gate called only isPrivateHost,
which classifies hostname text and IP literals but never resolves. A
syntactically public name whose A/AAAA record points at 127.0.0.1,
169.254.169.254 or an RFC1918 address was therefore accepted, and because
this path uploads the fetched bytes to DashScope, an internal endpoint's
response would be exfiltrated.
Add findNonPublicAddress() to utils/fetch.ts: it resolves the hostname via
dns.lookup({all: true}) and rejects if ANY returned address is non-public,
since the connect may pick any of them. IP literals short-circuit without a
lookup. Extract isPrivateAddress() as the shared bare-address classifier so
isPrivateIp keeps one implementation.
download.ts calls it inside the redirect loop, immediately before each
connect, so every accepted hop is re-validated — a permitted same-host
redirect can still re-resolve to a private address between hops.
Two deliberate choices, both recorded in the doc comment: the helper is NOT
wired into fetchWithPolicy, because WebFetch intentionally supports intranet
FQDNs resolving to private addresses, so applying it there would be a silent
breaking change; and DNS failure returns null rather than failing closed,
because the connect that follows reports the real error and failing closed
would turn a resolver blip into something the user cannot act on. The
residual check-then-connect race is documented too: fetch resolves again, so
closing it fully needs an agent that pins the connect to a vetted address,
which undici does not expose portably.
Also fixed: sniffMediaType classified any file starting with 0xFF 0xFE (the
UTF-16 LE BOM) as audio/mpeg, because 0xFE passes the frame-sync mask. Real
MPEG frames never use 0xFE — the layer bits would be 11, reserved — so
excluding it costs no genuine detection and keeps the "null for non-media"
contract honest for callers without a secondary modality gate.
Tests for the previously uncovered branches reviewers flagged:
- download: public-looking host resolving to 127.0.0.1/169.254.169.254/
10.1.2.3; mixed A records where only one is private; per-hop re-check
asserting the redirect is refused before the second fetch; DNS failure;
the ?? fetch fallback via a globalThis spy; and the production DNS path
asserting dns.lookup receives {all: true, verbatim: true}
- readMediaViaOmniDelivery: image gets the resolution + zoom_image hint
parts, audio gets a bare fileData, failure yields an error result and
never inline base64
- probeMediaMetadata: the audio branch reads the audio stream (proven with a
file carrying both h264 and aac), the image branch reports no duration
- sniffFileModality: recognized headers, non-media, and an absent path
- atCommandProcessor URL pipeline: a URL with omni active becomes a fileData
part, the same URL with omni disabled falls through as text without
downloading, and a URL-only prompt does not hit the no-content early
return
* fix(omni): pin URL-media connections to the vetted address, fail closed
Validating the resolved address and then letting fetch re-resolve is
check-then-connect: a low-TTL record answering public once and private
the second time would still be connected to, and the fetched bytes are
uploaded to a third party. Each hop now resolves via
resolveNetworkTarget('public') — which vets every returned address and
returns a lookup pinned to the vetted one — and installs that lookup as
the request dispatcher, so the name is never resolved a second time.
- fail closed: unresolvable/unvettable hosts and targets without a
pinned lookup are refused instead of fetched unpinned
- re-validated and re-pinned at every redirect hop; the previous hop's
agent is replaced and closed
- fetch defaults to undici's own fetch so dispatcher and fetch come
from one undici version (mixing majors rejects the agent)
- Node-only: Bun accepts dispatcher and silently ignores it (verified
on 1.3.11), so non-Node runtimes are refused with a clear message
- https and credential-free URLs enforced with named errors
- utils/fetch.ts: remove findNonPublicAddress/HostResolver (a weaker
duplicate of resolveNetworkTarget) and the doc paragraph wrongly
claiming undici cannot pin connections
* fix(cli): stop the @-path parser truncating URLs at query strings
The @-path terminator set treats '?', '&', ',' and ';' as sentence
punctuation, which is right for filesystem paths but wrong for URL refs:
a presigned OSS/S3 link carries its signature in the query string, so
'@https://…/a.mkv?x-oss-signature=…' was silently cut at the '?' and the
download failed with HTTP 403 naming only the host.
URL refs now run to the first character RFC 3986 does not permit
unencoded (covering whitespace and unspaced CJK prose), with trailing
sentence punctuation trimmed afterwards so it can never cut a URL short
mid-query. Filesystem path parsing is unchanged.
* fix(cli): highlight the full URL ref in the input box
The input-box highlighter had its own @-ref charset that excluded '%',
'?', '&' and '=', so a presigned URL was painted file-colored only up to
the first percent-escape and looked truncated even after the parser was
fixed to consume it whole. URL refs now use the same RFC 3986 charset
and trailing-punctuation trim as parseAllAtCommands, keeping the painted
span identical to what the parser consumes.
* fix(omni): close review round-2 findings
Criticals:
- tool-result-media: staging-dir setup moved inside convertPart's try —
a mkdir failure (ENOSPC, EACCES, ~/.qwen/omni as a regular file) now
degrades that part to inline instead of rejecting the whole tool
result and reporting a succeeded tool as failed
- sanitizeErrorMessage: exact split/join replacement of the known
filePath before the pattern pass (immune to CJK segments, ~-prefixed
and special-character basenames), with separator-based — not
ASCII-word-based — fallback classes and a Windows-path pass
- the fail-closed delivery result now sanitizes BOTH llmContent and the
error field (the scheduler sends 'error', not llmContent, to the model
on READ_CONTENT_FAILURE) and names the file by displayName
- the 100 MB image-source cap only applies when the overview decoder
will actually run: it protects the decoder, and the omni path uploads
original bytes under its own 1 GiB ceiling without decoding
Suggestions:
- malformed redirect Location surfaces as a named OmniDownloadError
instead of a raw TypeError (header value kept out of the message)
- header/idle watchdog timeouts are now identifiable in the surfaced
error instead of a generic 'This operation was aborted'
- effectiveMaxDownloadFileBytes is clamped to the upload cap even when
omni.download.maxFileBytes is configured higher
- estimation rejects degenerate present values (0/negative/non-finite)
as 'unavailable' instead of emitting a 0-token 'ok' estimate
- corrected the 0xFE sniff comment: 0xFF 0xFE is a valid MPEG-1 Layer I
header (reserved encodings are version 01 / layer 00); the exclusion
is a deliberate trade against UTF-16 LE false positives
- SSRF tests now cover private IPv6 (loopback, link-local, ULA,
IPv4-mapped) plus a public-IPv6 acceptance case
Regression tests are mutation-verified: reverting the mkdir placement
or the error-field sanitization makes exactly the new tests fail.
* fix(omni): close review round-3 criticals
- sanitizeErrorMessage: a path segment containing a space defeats the
pattern pass (segment classes break at whitespace), so 'john doe'
survived scrubbing. Call sites now pass every path they know about:
the store wrap adds the store root (covers putFile throwing before
objectPath is assigned), and the upload wrap - previously passing
err.message through verbatim - sanitizes with the object path and
store root. Function exported for direct shape tests.
- estimation: animated images (GIF/APNG/animated WebP) no longer
estimate as a single frame. The image probe branch retains ffprobe's
nb_frames as frameCount, and the estimator applies the design 6.4
visual formula with the real count - a 480x480 300-frame GIF now
estimates ~33,750 tokens instead of 113, so an enabled token guard
actually engages. Static images and containers that do not report a
frame count keep frameCount 1.
- tool-result media: a transport-guard rejection is a policy verdict,
not a transfer failure - the part is now withheld with a text
placeholder instead of being delivered inline as base64, which
silently bypassed an enabled guard at greater request cost than the
upload it was rejecting. Transfer failures keep the inline
degradation; the batch never fails for one part.
- mid-turn URL media: resolveAtCommandQuery runs under a fixed 10s
mid-turn budget that structurally killed every mid-turn @https
reference (the download alone can exceed it, and inside the resolver
the timeout abort is indistinguishable from a user cancel). URL-media
turns are now exempt from that timer - the download path carries its
own 30s header and 60s idle watchdogs - while filesystem resolution
keeps the 10s cap unchanged.
- input-box highlighter: match the URL scheme case-insensitively, like
the parser it mirrors; an uppercase @HTTPS ref no longer half-paints.
* fix(omni): close review round-3 findings
- download: resolve-phase watchdog, malformed-URL and proxy refusal,
resolver-error translation, path scrubbing, cause-chain surfacing,
DownloadedMedia slimmed to partPath; mutation-proven tests for
redirect cap, both watchdogs, byte-cap streaming, abort wiring
- recognition: direct recognizeMediaFile tests, GIF 6-byte signature,
guarded probe-handle close; sha256 moved out of RecognizedMedia
- pipeline: hash after guards (stat -> byte guard -> probe -> token
guard -> hash); displayName override so URL-funnel errors name the
remote file, not the staging path
- fileUtils: omni gating suite incl. 100 MB image-cap bypass pin
- dead S1 shims deleted (video-only probe/delivery/read wrappers)
- URL funnel: post-download modality gate mirroring the local-file gate
* feat(omni): S3 delivery reliability — upload cache, credential reuse, failure semantics, startup recovery (#8632)
* feat(omni): S3 delivery reliability — upload cache, credential reuse, recovery
Third omni slice: repeated deliveries stop re-uploading, transient
server-side failures self-heal, and crash leftovers get swept.
- upload cache: persistent (sha256, model) → oss:// URL map at
.qwen/omni/upload-cache.json with a 47h TTL (48h official validity
minus margin). Atomic writes; corrupt files are backed up and rebuilt;
per-file op serialization prevents same-process load-modify-save
races; TTL configurable via omni.upload.cacheTtlHours (0 disables).
The cache file is the only place omni persists an oss URL — the URL
remains a delivery cache, never an identity.
- credential reuse: module-level getPolicy cache keyed by
origin|model|apiKey-hash, 240s TTL (300s official validity), shared
in-flight fetches, failures never cached.
- failure semantics: provider errors that name the oss scheme or the
media download step invalidate the matching cache entries (throttling
is explicitly exempted), so the user's retry re-uploads instead of
resending a dead reference — no automatic in-pipeline resend.
- startup recovery (lazy, once per process, never throws): expired
downloads/*.part removed (48h resume window), promotion .tmp orphans
older than a 1h grace window removed (younger ones may be another
process's in-flight rename), and a day-rotating sample of ≤3 objects
(≤64MiB each) is hash-verified with corrupt objects deleted and their
cache entries cascaded.
Verified: 89 omni unit tests; 9/9 real-API E2E (cache hit 3.1× faster on
a 20MB image; fake-URL 403 → invalidate → self-heal on retry; corruption
cascade; recovery sweep), adversarial review round applied (6 findings
fixed: cross-process .tmp grace, sampling bias+size cap, throttle-safe
invalidation matching, apiKey-scoped credentials, NaN expiry, in-process
cache races).
Part of the omni-experiment S3 slice (#8185).
* fix(omni): harden S3 cache, credential, and invalidation paths from review
Upload cache (upload-cache.ts):
- reject entries:null/array shapes as corrupt (backup + rebuild) instead
of throwing raw TypeErrors on every subsequent read
- treat only ENOENT/ENOTDIR as an empty cache; other read errors skip the
operation rather than persisting a wiped file over valid entries
- key entries by (sha256, model, scope) where scope fingerprints the
endpoint credential, so URLs never cross accounts or origins
- clamp positive TTLs to the 48h server-side URL validity
- cap .corrupt-* backups at 2; shrink the fileOps serializer map
Credential cache (upload.ts):
- getPolicy no longer bakes the first caller's AbortSignal into the
shared in-flight fetch; each caller races the shared promise against
its own signal, so one caller's abort cannot poison others
Failure semantics (pipeline.ts):
- invalidate oss cache entries on the streaming error path too (the CLI
default) by threading the request into processStreamWithLogging
- tighten /oss/i to /oss:\/\//i so 'connection loss' never nukes
healthy entries; await the invalidation instead of fire-and-forget
Pipeline integration (index.ts):
- check the upload cache BEFORE store promotion: a hit now skips the
full-file copy as well as the upload
- pass the credential-scope fingerprint into the cache
Recovery (recovery.ts):
- key the one-shot latch by omni root; guard a synchronously throwing
store; fix the stride-sampling tail blind spot; make sample budgets
injectable for tests; correct the .part retention comment
Adds mutation-proof suites across all five areas: 89 tests in the omni
package plus a dedicated pipeline invalidation suite.
* fix(omni): keep startup recovery inside the managed omni root
Recovery readdir'd downloads/, the objects root, and each shard without
checking for symlinks, and hashed sampled candidates via lstat without an
isFile() check — so a symlinked shard (or a symlinked downloads/ or
objects/ directory) could redirect the scan outside .qwen/omni, where a
hash-mismatched name would get an external file deleted. The store's own
symlink guard only runs at putFile, after recovery.
- reject symlinked/non-directory managed dirs (lstat isDirectory) before
traversing downloads/, the objects root, and every shard
- reject non-regular-file candidates before hashing in the sampler
- regression tests: symlinked shard (reviewer repro), symlinked
downloads/, symlinked objects root, symlinked candidate — all four
fail against the unfixed code and pin that external files survive
with content intact
* fix(omni): close root-level symlink escape and add wholesale cache TTL sweep
- startup recovery now verifies the intermediate directory chain (omni
root and objects/) with lstat before any sweep runs: the per-directory
guards only check the final path component, so a symlink planted at
.qwen/omni or .qwen/omni/objects redirected the entire scan outside
the managed tree; two containment tests mirror the existing four at
the new levels
- upload-cache put() now sweeps all expired entries while it holds the
serialized write: get() only prunes the queried key, so entries never
read again accumulated in upload-cache.json forever
* feat(omni): S4 policy pipeline — three-modality degradation policies, Stage B tools, modelAccess gating (#8815)
* refactor(omni): migrate settings to design namespace (processing/delivery/ingestion/storage)
- omni.upload.maxFileBytes -> omni.processing.transportGuard.maxUploadFileBytes (<=1GiB)
- omni.transport.maxEstimatedTokens -> omni.processing.transportGuard.maxEstimatedTokens
- omni.upload.cacheTtlHours -> omni.delivery.upload.urlTtlHours
- omni.download.maxFileBytes -> omni.ingestion.localization.url.maxFileBytes
- new: omni.processing.{limits,fixedPolicies,transportGuard.policies,policyTools},
omni.storage.quarantine.{retentionDays,maxBytes}
- core getters renamed to match; error messages cite new key paths
Experimental branch: one-shot rename, no migration shim.
* feat(omni): surface bitRate/sampleRateHz/channels from ffprobe
ffprobe already reports these; the parse branches discarded them. The
policy condition DSL (resource.bitRate / resource.sampleRateHz /
resource.channels) and degradation disclosures both need the real
numbers. Format-level bit_rate is preferred over the stream's.
* feat(omni): add media-policy tool protocol (execution origin, descriptor, modelAccess gate)
Protocol core for the S4 policy pipeline:
- ToolExecutionOrigin (model | client | fixed_policy) on
ToolCallRequestInfo; only settable by in-process callers, never
deserialized from protocol payloads; missing origin fails closed as
model.
- MediaPolicyToolDescriptor as a DeclarativeTool code-registration
getter (default undefined) — config can never turn an ordinary tool
into a policy tool.
- PolicyArtifactBatch channel on ToolCallResponseInfo, capturing raw
successful media-policy artifacts before PostToolUse hook merging;
error/timeout results promote nothing.
- Shared modelAccess resolver + call gate
(omni/policy/model-access.ts): media-policy tools are
fixed-policy-only unless
omni.processing.policyTools.<name>.modelAccess.enabled; model calls
get defaultArguments/lockedArguments merged and are rejected when
they name a locked key; a forged fixed_policy origin on a
non-media-policy tool is rejected.
- Gate enforced on every model surface: registry declaration lists
(incl. includeDeferred for subagents), ToolSearch keyword + select:,
CoreToolScheduler pre-build, and ACP Session.runTool.
- fixed_policy scheduler calls skip the interactive permission flow but
keep PermissionManager tool-enablement and hook execution.
- omni.processing.policyTools threaded through ConfigParameters
(core + cli).
* feat(omni): add staging/quarantine storage areas and recovery sweeps
Storage design §4.3/§4.4/§6.1 for the policy pipeline:
- OmniObjectStore gains staging/ and quarantine/ areas (0o700, symlink
refusal in ensureLayout) plus the invocation lifecycle: exclusive
createStagingDir (16-hex id validation, no silent reuse),
removeStagingDir, and quarantineInvocation — which writes reason.json
(policyId/toolName/reason/failedAt) into the staging directory before
a single atomic rename into quarantine/<invocationId>/.
- Startup recovery deletes everything under staging/ (entries belong to
invocations that never committed) and trims quarantine/ to a
retention window (default 7 days) and size budget (default 5 GiB,
oldest-first), following the existing isRealDirectory containment
convention: symlinked roots and entries are never traversed, sized,
or deleted through.
* feat(omni): add fixed-policy when-condition DSL evaluator and validator
Implement the restricted when-condition DSL from the policy design (§8.3):
recursive all/any combinators over gt|gte|lt|lte|eq comparisons, with
operands drawn from three read-only namespaces (resource.*, request.*,
session.*) or literals. No arbitrary code, no JSONPath.
Evaluation is three-valued. A comparison over an unresolvable field yields
`unavailable` with the missing fields recorded — never a silent false —
so the caller can apply the policy's onConditionUnavailable behavior and
surface the fields in the run record. Combinators use strong Kleene
logic: `all` with a false branch is a determinate no_match regardless of
unavailable siblings; `any` with a true branch is a determinate match.
Vacuous semantics: all [] → match, any [] → no_match.
The evaluator is a total function that never throws — structurally
malformed nodes degrade to `unavailable`, the fail-safe outcome. The
separate structural validator (for the startup config-normalization pass,
policy design §13 #5) rejects malformed conditions with path-annotated
errors: exactly-one-of comparison/all/any, non-empty combinator arrays,
exactly-one-of field/value per operand, known-field membership, and
finite numeric literals for ordering operators.
Condition types are re-exported from omni/policy/types.ts alongside the
rest of the policy protocol surface.
* feat(omni): add three-modality degradation media-policy tools
Commit 6 of the S4 policy pipeline: the three built-in degradation tools
(mapping doc §6), each a real BaseDeclarativeTool carrying a
media_policy descriptor so the orchestrator and the modelAccess gate can
key off code-level facts.
- omni/policy/tools/media-policy-tool.ts: shared base — validates
against the NATIVE parameter schema (never the model-visible
projection Stage B will narrow), async io assertions (lstat: symlink
input refused, real output directory required), per-tool
policyTools.<tool>.runtime.timeoutMs resolution (default 600s),
uniform success/error ToolResults; success emits exactly one lossy
artifact whose metadata.omniDisclosure carries the D8 disclosure and
whose workspacePath is staging-relative.
- omni_downsample_image (sharp, lazy-loaded per D9; load failure is an
execution failure): fit 1568px / JPEG q75, EXIF orientation baked in,
animated inputs refused outright (sharp would silently keep only the
first frame).
- omni_downscale_video (ffmpeg): scale to even target height computed
in JS (no filtergraph expressions), fps 10, x264 crf 28 veryfast;
audio stream-copy with AAC 64k fallback.
- omni_downsample_audio (ffmpeg): AAC 64kbps/16kHz/mono, -vn strips
cover art.
- omni/ffmpeg.ts: runFfmpeg — never rejects, callers branch on the exit
code and must check signal.aborted explicitly.
- Registered lazily behind isOmniEnabled(); the commit-3 modelAccess
gate keeps them hidden from model surfaces by default.
* feat(omni): add degradation result cache keyed by policy fingerprint
Commit 7 of the S4 policy pipeline (decision D2): reusing a previous
transcode instead of re-paying it — a 424MB video downscale is
minutes-long and must not run once per delivery round.
- omni/json-cache-file.ts: the file mechanics extracted verbatim from
upload-cache (per-file serialized load-modify-save, atomic tmp+rename
0600 writes, corrupt backup+rebuild with capped .corrupt-* backups,
unreadable-but-existing file = operation no-op) as a shared
OmniJsonCacheFile — the design doc mandates the degradation cache
mirror these exact semantics, so they now exist once.
- omni/upload-cache.ts: refactored onto the shared file; entry
semantics (model/scope keys, TTL clamp, invalidation) unchanged — all
22 existing tests pass untouched.
- omni/policy/degradation-cache.ts: (originalSha256, policyFingerprint)
→ { degradedSha256, extension, disclosure, mimeType } at
.qwen/omni/policy-cache.json. policyFingerprint = sha256(toolName +
key-sorted tunables + tool version); per-invocation io params
(inputPath/outputDir) are excluded — they are plumbing, not policy
identity. Entries carry no TTL (content-addressed identities never go
stale); removeByOriginalSha256 / removeByDegradedSha256 serve the
GC/corruption cascades. Object existence checks stay with the
orchestrator.
* feat(omni): add fixed-policy orchestrator and reorder the delivery pipeline
Introduce runFixedPolicies: for each recognized media resource it matches
normalized fixed policies (modality, origins, when-conditions), executes the
policy's media tool through the real executeToolCall protocol (fixed_policy
execution origin, recordToolResult:false, isolated staging outputDir),
validates the returned policy artifacts against the tool's declared
descriptor (workspace containment, recognized kind/mime match, mandatory
disclosure for lossy outputs), promotes derivatives into objects/ and records
them in the degradation cache keyed by original sha + policy fingerprint.
Reorder processMediaForOmniDelivery so transport guards judge the FINAL
delivery set instead of the source: recognize -> recovery -> fixed policies
-> guards -> hash -> upload. Degraded deliveries carry a disclosure that is
emitted as a text Part immediately before the media Part everywhere media
surfaces (read tool results, tool-result inline media conversion), and the
OpenAI converter moves the disclosure together with its media part when
splitting tool media into a follow-up user message.
* feat(omni): normalize omni.processing config with system default policies
Startup normalization of the fixed-policy pipeline configuration
(policy design §13 applicable subset):
- omni/policy/config.ts: normalizeOmniProcessingConfig merges user
settings over system defaults (id-merge, whole-entry replacement,
null tombstones for fixedPolicies only) and validates structure,
enums, when-conditions, tool references (registered + media_policy
descriptor + required/lossy-disclosure outputs), fixed arguments
against the io-stripped settingsSchema, reserved io keys, guard
rules (no when, source=omit, mandatory three-modality coverage),
limits (§12.2 defaults), the 1 GiB upload cap and the 48h URL TTL.
Any violation throws OmniPolicyConfigError and aborts startup — a
mis-configured guard must never degrade into sending over-limit
media.
- System defaults (D7 dual registration): the three degradation tools
registered as preprocessing fixedPolicies WITH when-thresholds and
as transportGuard.policies WITHOUT when.
- core Config: thread fixedPolicies / transportGuard.policies /
limits / quarantine settings; normalize in initialize() after tool
warmup; expose getOmniProcessingConfig and quarantine getters.
- cli: thread the new omni.processing / omni.storage.quarantine keys.
* feat(omni): enforce policy budgets, transport-guard passes and quarantine (Stage B)
- runFixedPolicies enforces maxPolicyRunsPerRoot (checked before each
execution), maxArtifactsPerRoot and maxDerivedBytesPerRoot (checked as
derivatives land); exhaustion records outcome 'budget_exhausted' while
committed deliveries stand; maxLineageDepth clamps reprocessing.
- Failed invocations quarantine their staging dir with a reason.json
(policyId, toolName, reason, failedAt); user aborts and quarantine
failures fall back to plain staging removal. Startup recovery receives
quarantine retention/size settings from config.
- Transport-guard violations now run modality-matched guard policies for
up to maxTransportPasses passes on the final delivery; a still-violating
resource is explicitly omitted (【媒体省略】notice replaces the part in
both the read path and tool-result funnel) instead of delivered
oversized; guard-pass failures stay fail-closed. Configs without a
normalized processing config keep the Stage A throw.
- Media-policy tools project a model-visible declaration (D6): locked
arguments removed from properties/required, optional narrowing-only
parameterSchema merge and description override; validation keeps the
native schema.
* fix(omni): declare the disclosure text output in degradation tool descriptors
All three degradation tools emit a disclosure at runtime, but their
descriptors never declared the text output, so the system default
policies failed the lossy-requires-disclosure validation (config §13 #8)
at real startup. Stub-based unit tests missed the drift; config.test.ts
now normalizes the defaults against the real tool instances.
* fix(omni): harden policy pipeline per review (cache validation, permissions, settings, sweep grace, concurrency)
- validate degradation-cache entries and object-store path components
(sha256/extension shape) so a poisoned policy-cache.json cannot traverse
paths or serve malformed derivatives; malformed entries self-heal
- re-hash cache-hit objects before reuse: mismatched bytes trigger a fresh
transcode and heal the object store (D2 integrity)
- default media policy tools to 'ask' permission via a shared
BaseMediaPolicyToolInvocation (model-origin calls confirm outside yolo;
fixed_policy runs are unaffected)
- consume policyTools settings from config: tool-level settings defaults
merge under policy arguments and feed the cache fingerprint so settings
edits invalidate cached derivatives
- deliver @url omission notices and degradation disclosures: omission
replaces the fileData part with the notice text; disclosure text lands
immediately before its fileData part (D8)
- give staging sweep a 1h multi-process grace window (only entries older
than the window are deleted; symlink entries removed regardless of age)
- gate concurrent fixed-policy runs per omni root with a FIFO counting
semaphore honoring maxConcurrentResources
* fix(omni): harden policy pipeline after self-review round
Orchestrator: exclude animated images from policy matching (D9), self-heal
unverifiable degradation-cache entries, key cache fingerprints with the
descriptor version, and warn when reprocess origins exclude 'policy'.
Scheduler: fixed_policy invocations no longer bounce PreToolUse 'ask' to an
unanswerable awaiting_approval (deny stays fail-closed), and
processToolResultImages short-circuits for fixed_policy results at the
method level so all three call sites skip the omni re-delivery / vision
bridge re-entrancy.
Delivery/CLI: chain preprocessing and guard disclosures instead of replacing
(D8), exclude hidden media-policy tools from the /context per-tool breakdown
to keep it aligned with getFunctionDeclarations(), annotate fixed-only
media-policy tools in /tools, and drop the stale maxConcurrency mention from
the policyTools settings description.
Tests: add negative coverage for artifact validation (no artifacts,
undeclared media type, kind mismatch, missing required output) and for the
fixed_policy hook ask/deny and image-funnel paths.
* fix(omni): drop system-default fixedPolicies; guard is the only default
The three degradation tools were registered twice: as transportGuard
policies (design-mandated, always-on, transport-limit triggered) and as
fixedPolicies with invented when-thresholds (1568px/480p/96kbps). The
upstream design gives fixedPolicies pure user-experiment semantics — a
zero-config setup must not trigger any preprocessing below transport
limits. Remove the fixedPolicies-side registration and its threshold
constants; keep the guard defaults. Threshold acceptance for #8186 moves
to explicitly-configured fixedPolicies in E2E.
* feat(omni): add four local stage B policy tools (keyframes/extract-audio/clip/convert)
- omni_extract_keyframes: ffmpeg scene detection (select+showinfo) with
uniform-fps fallback, multi-artifact batch, per-frame disclosures
- omni_extract_audio: audio track extraction to WAV/MP3/M4A, 16kHz mono
WAV default (ASR-recommended shape)
- omni_clip_video: frame-accurate time-axis cut (input-side -ss/-t,
libx264 re-encode, faststart), rejects no-op invocations
- omni_convert_image: sharp re-encode to JPEG/PNG/WEBP with EXIF
orientation baked in, animated inputs refused
- uniform lossy declaration per mapping doc §6.1: every output carries a
disclosure since the representation change itself is lossy
- extract shared sharp loader (sharp-module.ts) and describeChannels
helper; register the four tools lazily behind isOmniEnabled()
* feat(omni): add omni_transcribe_audio with transcript delivery protocol
* feat(omni): fill request/session condition namespaces and wire client origin
The when-condition evaluator has supported the request.* and session.*
namespaces since the initial S4 landing, but no caller populated them, so
every policy referencing those fields resolved to 'unavailable'. This
fills both per policy design section 8.3:
- request.totalEstimatedMediaTokens is computed inside the orchestrator
from the pending-delivery work-item set at each pass start, so it is
recomputed as derivatives enter later passes. If any pending resource
is unestimable the whole namespace reads unavailable - a partial sum
must never pass thresholds as a smaller total. Callers can override
via options.conditionContext.request (future multi-root aggregation).
- session.* is snapshotted once per media delivery by the new
policy/session-context.ts (contextWindowTokens from
contentGeneratorConfig.contextWindowSize, promptTokenCount from the
CURRENT chat's getLastPromptTokenCount, availableContextTokens as a
clamped subtraction only when both inputs are known) and reused across
preprocessing and every transport-guard pass of that delivery.
Also wires the client execution origin: slash commands scheduling a tool
(schedule_tool in useGeminiStream) now stamp executionOrigin
{kind:'client'}, putting the in-process client channel under the same
media-policy modelAccess gate as model calls.
* fix(omni): deliver every media deliverable of multi-output fixed policies
A fixed policy producing multiple media deliverables (omni_extract_keyframes
promoting N frames) previously failed delivery with "exactly one is
supported" after all frames were already atomically promoted to objects/ —
zero frames reached the model, violating the multi-output transaction
visibility acceptance.
Fixed-side contract now: deliveries[0] is the primary (unchanged path:
guard re-check, upload, disclosure); the rest ride along as
OmniMediaDelivery.additionalMedia, each independently judged against the
transport limits and uploaded through the same hash → upload-cache →
objects promotion → DashScope pipeline. Extras are processed lazily after
the primary settles, so the primary uploads first and a fail-closed
primary never wastes extra uploads. An over-limit extra becomes an
explicit omission entry (visible reason, no guard-policy re-derivation —
it is already a policy product). The guard side keeps its strict 1:1
semantics.
All three delivery consumers (readMediaViaOmniDelivery, tool-result
media conversion, the @-command URL funnel) materialize extras through a
shared buildAdditionalMediaParts helper: per-extra [disclosure?,
fileData | omission text] pairs after the primary slot, before
transcripts, preserving disclosure adjacency per pair.
* fix(core): detect mkv/avi/flac/aac media missing from mime/lite standard db
mime/lite's default database carries no mapping for .mkv/.avi/.flac/.aac,
so detectFileType fell through to the binary content sampler and a real
Matroska movie was rejected by the 10MB inline cap instead of entering
media delivery. Generalize the existing .m4v override map to cover these
containers (authoritative types from the full mime db), scoped to
extensions the omni recognizer can sniff-confirm so extension lies still
fall back to the legacy path.
* fix(omni): harden policy pipeline per self-review correctness findings
- Validate operator-configured defaultArguments/lockedArguments against a
tool's NATIVE parameterSchema instead of the model-visible projection,
which strips exactly those keys and rejected every legitimate locked or
operator-only argument at startup; regression test uses real tools.
- Write cache files with noFollow so the atomic rename replaces a symlink
planted at the cache path instead of writing (and chmod-ing) through it.
- Route every .part staging write through prepareOmniDownloadsDir, a
fail-closed symlink guard (mkdir recursive succeeds silently on a link).
- Detect ADTS AAC in the MPEG-audio sniffer (layer bits 00) so .aac
streams stop being labeled audio/mpeg.
- Preserve metadata.omniRole in degradation-cache entries so cached
reruns keep the same artifact matching as fresh derivations.
- Fall back to the default quarantine retention when the configured value
is zero/negative/NaN instead of expiring the whole quarantine.
- Propagate error causes through OmniTransportGuardError and document the
operatorOnlyParams contract on MediaPolicyToolDescriptor.
* refactor(omni): consolidate media-policy tool plumbing
- Single-source tool names from ToolNames across all eight policy tools
and the config validator.
- Share the config view on the base class (protected configView) instead
of eight shadowed per-subclass fields; move the view type and the
isPlainRecord helper to policy/types.
- Memoize the model-visible schema projection on the settings object's
identity (one normalized settings object per initialize()).
- Extract sharpTimeoutSeconds for the duplicated libvips timeout
conversion and route clip-video's io validation through the base
validateToolParamValues.
* refactor(omni): share the transcript materializer across delivery consumers
Extract buildTranscriptParts (§6.2/D8 ordering: each transcript follows
its media part or omission notice, preceded by its own disclosure) and
use it from readMediaViaOmniDelivery, the tool-result funnel, and the
CLI @-command URL funnel instead of three hand-rolled loops. Also
extract the textOnlyDelivery helper, upload additional-media extras
concurrently (content-addressed store and per-file serialized upload
cache make them independent; map preserves deliverable order), and drop
the now-consumerless formatTranscriptText barrel export.
* perf(omni): parallelize orchestrator validation and promotion
Validate a batch's artifacts and promote them into the object store
concurrently — they are independent files in a shared staging dir and
the store is content-addressed with race-safe puts; map keeps the batch
order and a sequential partition preserves derived/derivedFiles order.
File artifacts hash the bytes already in memory for the UTF-8 check
instead of streaming the file a second time.
* fix(omni): add display-name translations for the media-policy tools
The eight omni media-policy tools landed without toolDisplayName locale
entries (cli zh/zh-TW/en) or web-shell wire-name display mappings and
toolName translations, tripping both display-name drift guards
(cli src/i18n/index.test.ts and web-shell toolFormatting tests).
* fix(omni): open the download .part fd before the pipeline starts
createWriteStream opens lazily, so a pipeline failure on an
all-synchronous body (the byte cap tripping on the first chunks) could
reject — and reach the outer .part cleanup — before the file was even
created, resurrecting the .part after its rm. Await the open event
before streaming so creation always precedes cleanup.
* feat(omni): switch when-condition DSL to expression arrays
Replace the {left, operator, right} object form with Mapbox-style
expression arrays: a comparison is [op, operand, operand] with op one of
> >= < <= == !=, an operand is a ["field", "<namespace.name>"]
reference or a bare literal, and combinators nest as ["all", ...],
["any", ...], and the new ["!", expr] negation. The array form is
shorter to write, nests without ceremony, and leaves room for future
operators.
Semantics are unchanged: three-valued evaluation with strong Kleene
combinators, unavailable fields never silently collapse to false, and
negation passes unavailable through rather than laundering unknowns.
Startup validation rejects the retired object form with a pointed
migration hint instead of a generic type error.
* feat(omni): degrade delivered media and retry when the server rejects input as over-limit
The server's real input ceiling is unknowable locally (frame-based video
billing is invisible to the byte/duration estimator), so a 400 like
DashScope's 'Range of input length should be [1, 196608]' previously
aborted the session even though history compression cannot shrink media
tokens.
Now, before reactive compression, the chat loop reverse-maps each
delivered oss:// URL through the upload cache to its stored object,
re-runs the configured transport-guard policy with an escalating
argument ladder (video fps 2 -> 0.5 -> 0.25 is the only lever that
reduces frame-billed tokens), re-uploads the derivative, swaps the
history parts in place with a fresh disclosure, and retries. Bounded by
limits.maxTransportPasses; best-effort throughout - any failure falls
through to the existing compression/error paths.
Also: parse the DashScope input-range upper bound as limitTokens, allow
fractional fps in omni_downscale_video, and add a scope-agnostic
findSha256ByUrl reverse lookup to the upload cache.
* feat(omni): inject progressive media understanding guidance into the system prompt
Disclosure parts state WHAT a delivery lost, but nothing told the model
WHY: degradation is a context-budget-driven progressive-understanding
strategy, the degraded delivery is an overview rather than the complete
content, and fuller evidence can be fetched on demand. Without that
contract the model treats a 600s clip + opening keyframes as the whole
film and extrapolates.
- omni/media-guidance.ts: builds a stable system-prompt section that
explains the three disclosure markers, frames degraded deliveries as
progressive-understanding overviews (never conclude undelivered
content doesn't exist), and directs proactive targeted evidence
fetching via exactly the modelAccess-enabled media policy tools; with
none enabled it instructs stating missing evidence instead of
extrapolating. Returns null when omni delivery is inactive.
- prompts.ts: new stable SystemPromptLayers slot mediaGuidance, placed
right after base so it stays inside the cached static prefix.
- client.ts: wire the section into the main-session stable layers.
- omni/delivery-gate.ts: isOmniDeliveryActive moved out of omni/index.ts
into a leaf module so prompt assembly doesn't statically pull the
whole delivery pipeline; index.ts re-exports it unchanged.
* fix(omni): extract keyframes across the full video duration (bucketed sampling)
The scene-detection pass stopped at the first maxFrames scene changes,
so every keyframe of a long video landed in its opening minutes (an
81-minute film yielded 16 frames all within 0-130s). Split the timeline
into maxFrames equal buckets instead: each bucket contributes one frame
- a scene change from its opening window (input seek, 30s search cap)
when one exists, the bucket midpoint otherwise. Absolute timestamps are
reconstructed as bucketStart + showinfo pts_time. Individual bucket
failures are tolerated and all runs share one wall-clock budget. The
single-pass path remains for unknown duration or maxFrames = 1; the
now-unreachable uniform-sampling fallback is removed.
* fix(omni): chunk long-audio transcription and collapse repetition degeneration
A single ASR request over long audio truncates well before the end and
degenerates into repetition loops (an 81-min film came back as 2114
chars ending in 48 copies of "Hej!"). Audio longer than chunkSeconds
(default 180s) is now split into equal segments, re-encoded to 16kHz
mono AAC, transcribed with bounded concurrency under one shared
wall-clock budget, and assembled into [MM:SS-MM:SS]-labeled lines.
Per-segment failures become inline markers instead of failing the run;
the run only errors when every segment failed. Repetition degeneration
is detected and collapsed in every transcript (chunked and single-shot)
and reported in the disclosure.
* docs(omni): add media policy orchestration usage guide
* fix(omni): key pure-transcript guard resolution on this pass's file deliveries
A transport-guard pass that omitted the source without producing any
deliverable used to short-circuit into textOnlyDelivery whenever an
EARLIER pass had already collected a transcript, silently dropping the
zero-deliverable error. Key the branch on the current pass's
fileDeliveries instead of the cumulative transcript list.
* fix(omni): render the primary disclosure on pure-transcript deliveries
All three delivery consumers (readMediaViaOmniDelivery, tool-result
media, @-command processor) dropped delivery.disclosure whenever the
primary media degraded all the way to a transcript-only delivery. The
disclosure chains every prior lossy step (decision D8) and the
transcript was derived through those steps, so it must still render —
as the leading text part of the delivery.
* fix(omni): prune non-object entry values when loading a JSON cache file
The load-time shape check validated the top-level {version, entries}
envelope but trusted every entry VALUE, so a crafted workspace cache
holding e.g. `"key": null` surfaced as TypeErrors from field accessors
in whole-file scans (removeByDegradedSha256 touches every entry, not
just the requested key). Drop malformed values at load and log the
count; the next put() rebuilds them from verified data.
* fix(omni): harden degradation-cache hits against tampered entries
Two gaps in the workspace-shippable cache's trust boundary:
- disclosure had no length ceiling, leaving an unbounded
prompt-stuffing channel that renders into the conversation on every
cache hit. Entries with a disclosure over 2048 chars are now dropped
like any other malformed field.
- the hit path verified the derivative's hash but not its TYPE: a
swapped extension/mimeType could deliver a media kind the producing
tool never declared. The orchestrator now cross-checks the recognized
MIME type against the tool descriptor's declared media outputs and
re-transcodes on mismatch, dropping the entry.
* fix(omni): include the hours field when a rounded duration reaches 3600s
withHours compared the raw duration while formatClock rounds each
boundary: a 3599.6s audio rounded to 3600s inside the label and, with
hours disabled, rendered as "00:00". Round the duration the same way
formatClock does before deciding the format.
* fix(omni): cap chunked transcription at 512 segments (fail closed)
The claimed duration comes from container metadata, which a crafted
file controls freely: an absurd duration used to fan out into hundreds
of thousands of outcome slots and queued ffmpeg cuts before the time
budget could intervene. Reject anything over 512 segments — at the 30s
chunkSeconds floor that is ~4h16m of audio, far past what the 10MiB
default input cap plausibly holds.
* fix(omni): disable the omni pipeline in bare mode
isOmniEnabled ignored bareMode, so --bare sessions still registered the
media-policy tools, ran content normalization and the ffmpeg startup
assert, and gated deliveries. Bare mode now wins over the env opt-in at
the single choke point every omni entry path already consults.
* fix(omni): degrade media nested in functionResponse.parts on reactive retry
The reactive over-limit fallback only walked top-level content parts,
so media delivered inside a tool result's functionResponse.parts (the
normal carrier for tool media) was invisible to it: the retry loop
resent an unchanged request and stalled. Collection and replacement now
descend one level into functionResponse.parts, and the swap rebuilds
the nested array in place — the disclosure must land immediately before
its media part (D8), which hoisting to the top level would break.
* fix(omni): detect animated images whose headers carry no frame count
Animated WebP/APNG containers report no nb_frames, so the D9 animation
exclusion silently passed them and the image tools kept only the first
frame with no disclosure. Two independent gates now close this:
- the ffprobe path re-probes animation-capable containers
(gif/webp/png/apng) with -count_frames when nb_frames is absent
- both image tools additionally check sharp's metadata().pages before
encoding and refuse multi-page inputs
* docs(omni): align tunable descriptions with the implemented semantics
- maxWidth/maxHeight are same-value aliases of width/height, not
independent axis caps
- request.totalEstimatedMediaTokens is the per-resource estimate
(resource plus its derivatives) within the current scheduling pass,
not a cross-resource total
* fix(omni): keep a raw "__proto__" policy id as an ordinary own key
A JSON settings file can carry "__proto__" as an ordinary key; spreading
it into a plain object routes it through the prototype setter and the
entry silently vanishes. Merge policy maps onto a null-prototype object
so the id reaches normalizePolicy and is rejected by the id pattern.
* fix(omni): validate transportGuard.maxEstimatedTokens at startup
Settings load performs no runtime type checks and guard.ts compares the
threshold with <=/>: a string value makes both comparisons false and
silently disables the token guard (fail-open). Reject anything but a
finite number >= 0 at normalization, and thread the value through
Config so the check actually sees it.
* fix(omni): raise quarantine retention/budget schema minimums to 1
The runtime accessors treat non-positive values as absent and fall back
to the defaults, so a configured 0 never means "delete immediately" /
"no budget" — advertising minimum: 0 in the IDE schema promised a
semantic the code does not honor. Document the fallback in the
descriptions as well.
* docs(omni): correct the fixedPolicies merge/tombstone description
There are no built-in default fixed policies, so "null tombstones a
default policy" described a merge target that does not exist. Describe
what the merge actually does: entries merge by id across settings
scopes and a null entry tombstones a policy from a lower-priority
scope.
* fix(omni): expose io paths to the AUTO-mode permission classifier
Media policy tools override getDefaultPermission but left
toAutoClassifierInput at the empty-string sentinel, so the AUTO-mode
classifier saw "Arguments: {}" and its path-based block rules could
never fire on a model-origin call. Project exactly inputPath and
outputDir — the fields those rules key on, carrying no secrets.
* fix(omni): reject unknown keys in policyTools entries (§13 #1)
A typo like "settigns" or "modelaccess" read as an absent optional
section downstream, so the intended configuration silently never took
effect. Fail startup on unknown keys at the entry, runtime, and
modelAccess levels, and drop the stale maxConcurrency mention from the
runtime doc comment (timeoutMs is the only supported limit).
* fix(omni): reject parameterSchema overrides that loosen native constraints
Design §11.2: a projection may only narrow the tool's native schema —
类型、枚举、范围 included, not just the property set. The declaration
merges each projected property's keys over the native ones, so an
override raising maximum, lowering minimum, widening an enum, changing
the type, or relaxing min/max length/items promised the model a range
the native per-call validation then rejects. Validate the merged
property schema against the native effective bounds at startup
(minimum/exclusiveMinimum interplay included).
* fix(omni): clamp downsample-audio targets to the probed source
A source already below the target bit rate, sample rate, or channel
count re-encoded into a LARGER derivative that the transport guard
counted as progress, under a disclosure claiming 高频细节丢失/声道合并
losses that never happened. Clamp each target to the probed source
(the withoutEnlargement analogue of the image/video tools) and build
the loss clause from the drops that actually occurred (D8).
* fix(omni): reject full-span no-op clips instead of re-encoding
clip_video is a time-axis cut, not a degradation tool: a span covering
the entire video only burns a lossy re-encode while the disclosure
falsely claims content outside the span was discarded. Reject
startSec 0 with no durationSec at the parameter layer, and a
probed-duration-covering span before the transcode starts.
* fix(omni): disclose partial bucket coverage in keyframe extraction
When some buckets yield no frame (scene attempt and midpoint fallback
both empty), the blanket 全片分桶采样 note overstated coverage — the
un-sampled buckets' time ranges were silently invisible to the model.
Disclose 仅覆盖 N/M 个分桶，其余时段未采样 whenever fewer frames than
buckets survive (D8).
* fix(omni): only claim alpha loss for alpha-capable JPEG conversion sources
A JPEG→JPEG re-encode cannot drop a transparency channel that never
existed, yet the disclosure asserted 透明通道丢弃 unconditionally.
Derive the loss clause from the probed input codec: alpha-capable
formats (png/webp/gif/tiff) keep the claim, others get 元数据丢弃, and
an unknown codec hedges with 如有 (D8).
* fix(omni): build the downscale-video loss clause from actual drops
A 360p@8fps input downscaled against the 480p/10fps defaults lowers
neither dimension, yet the disclosure always claimed
分辨率与帧率下降 — contradicting the before/after figures shown right
next to it. Emit 分辨率下降/帧率下降 only for the parameters that
actually dropped, falling back to 重新编码压缩 for a pure re-encode
(D8).
* fix(omni): stop calling a degraded delivery "full resolution" in the zoom hint
On the degradation path delivery.recognized re-recognizes the
DERIVATIVE, so the hint's width/height are the downsampled dimensions.
Calling them "full resolution" contradicted the degradation disclosure
pushed immediately below and steered the model away from zoom_image —
the exact remedy for degradation-stripped detail, since it reads the
original from disk.
* fix(omni): localize the fixed-only tool description suffix
The '[fixed-only: runs via media policies, not the model]' suffix was
appended as a hard-coded English string, so zh/zh-TW tool listings
showed a mixed-language description. Route it through the i18n table
like the display names it accompanies.
* fix(omni): unify the downscale-video display name as 降采样视频
降清视频 was a coined term inconsistent with the 降采样图像/降采样音频
siblings; use 降采样视频 (zh-TW: 降採樣影片) across the CLI locales and
the web-shell tool-name table.
* test(omni): match the actual .tmp suffix in cache-file cleanup assertions
The write-temp files are named <name>.<random>.tmp; filtering on
'.tmp-' could never match anything, so the leak assertions were
vacuously green. Filter on the real endsWith('.tmp') suffix.
* test(omni): pin the repetition-collapse rep threshold independently of the span floor
The 好的， unit (3 chars × 7 reps = 21 chars) sat below the 24-char span
floor, so the test passed for the wrong reason and REPETITION_MIN_REPS
itself was unpinned. Use a 6-char unit so 7 reps already clear the span
floor, and assert the collapse output at 8 reps.
* fix(omni): cascade corrupt-object deletion into the degradation cache
Startup recovery's sample verification deleted a corrupt object and its
upload-cache entry but left policy-cache.json untouched, accumulating
entries that can never be served again. Cascade the deletion both ways
— the object may be a policy SOURCE (its derivatives can no longer be
re-verified) or a policy DERIVATIVE (entries pointing at it are dead) —
mirroring the storage design's upload-cache cascade. The orchestrator's
lazy self-heal only fires on a cache hit; the sweep side is proactive.
* refactor(omni): resolve role selectors via a role→output-spec map
The role existence check consulted a Set while the transcript shape
check re-scanned descriptor.outputs with a find, duplicating the
first-declaration-wins semantics across two structures. A single
Map<role, spec> now serves both.
* test(omni): pin the when-condition round-trip through normalization (D7)
when is preprocessing's ONLY trigger mechanism: a normalization
regression that drops or rewrites it would silently widen every user
condition to ALL matching resources, with no failing test. Pin that a
valid condition tree survives normalization verbatim.
* test(omni): pin the runtime.timeoutMs / staging-grace boundary (recovery §5)
The timeoutMs < STAGING_GRACE_MS cap is a cross-file invariant: a tool
allowed to run for the full grace window could have its live staging
directory swept as crash leftovers mid-run by another process's startup
recovery. Pin BOTH sides of the boundary against the exported constant
so removing, inverting, or relocating the cap fails a test.
* feat(omni): S5 multimodal media memory — two-trigger collection, root-bounded recall, active/sideQuery surfaces (#8949)
* feat(omni): collect multimodal media memory at recognition and policy-success points
Add the media-memory service (packages/core/src/services/media-memory):
a persistent, file-scoped lineage graph over .qwen/omni/memory.json that
records recognized files/versions and policy executions with their
derived media and text outputs.
Collection side (Stage A of the multimodal memory design):
- store.ts: single-JSON v1 backend with atomic writes, corrupt-document
backup/rebuild, and read/transact serialization shared with the omni
JSON cache chain.
- service.ts: recordFileRecognized (idempotent file+version upsert,
CURRENT_VERSION follows disk both ways) and commitPolicySucceeded
(one atomic commit per succeeded policy execution: execution record,
derived versions with DERIVED_FROM/PRODUCED_BY lineage, and
derived_media/policy_result entries with conservative channel and
coverage derivations). executionId is a content-identity key
(source sha256 + policy fingerprint), so replays and degradation-cache
hits converge on the same execution node. findBindingBySha256 gives
the reactive ladder a read-only identity lookup.
- config.ts: omni.memory normalization with fail-fast validation
(unknown keys, budgets, kinds, cross-field ordering).
Wire both trigger points: processMediaForOmniDelivery records the
source file after recognition (hashing the source upfront and threading
sha256 through the policy pipeline), and executePolicy commits after
object promotion on both the success and degradation-cache-hit paths.
Persistence failures are logged and never block delivery; a policy
commit itself stays all-or-nothing.
* fix(omni): label keyframe and clip artifacts with their roles
omni_extract_keyframes and omni_clip_video emitted artifacts without
metadata.omniRole, so role consumers could not tell an excerpt from a
complete derivative: output routing selectors (role:keyframe /
role:clip) never matched, and media-memory coverage derivation fell
through to 'complete' for sampled keyframes and temporal clips.
Label them 'keyframe' and 'clip', matching the transcript labeling
convention.
* feat(omni): add session resource registry and memory recall service
Stage B read side of multimodal media memory (design M §9):
- MediaResourceRegistry: session-scoped opaque handle binder
(fileVersionId <-> resourceId, M §5.2) so recall payloads never
carry real paths or stable identifiers.
- MediaMemoryRecallService: current-version-first recall (§9.5) over
the rootFileId-bounded derivation graph (§8), returning the minimal
protocol shape (§9.4) with synthesized metadata/execution entries,
honest gaps (not_processed / partial_coverage / artifact_unavailable,
D5), and advisor-driven nextPolicyActions. Unknown or empty resource
requests reject as a whole (§9.2); an unreadable store degrades to
a plain miss.
* feat(omni): mount session media resource registry on delivery
Hang a lazy MediaResourceRegistry off Config and bind every
memory-known resource the delivery pipeline puts in front of the
model — the source at recognition, each preprocessing deliverable,
and the transport-guard replacement — so recall can rebind opaque
session handles back to persistent memory identities without ever
exposing a path.
* feat(omni): add active-mode media memory recall tool and session handle disclosure
Stage B recall surface #1 (memory design M §9, D10 mutual exclusion):
- omni_recall_media_memory tool, registered only when omni is enabled AND
omni.memory.recall.mode === 'active'. Read-only (D11): consults the
persistent store through MediaMemoryRecallService and binds session
handles for returned derived artifacts; whole-request rejection maps to
invalid_tool_params so the model can correct and retry.
- Session resource handles are now disclosed to the model: delivery
carries the SOURCE binding's resourceId, and both consumers (file reads
and the tool-result funnel) lead the part group with a 【媒体资源】
annotation — the handle stands in for the path the model never sees
(M §5.2), while disclosures keep their D8 adjacency to the media part.
- Advisor wiring: recall gaps suggest nextPolicyActions only for tools
that are both registered and opened via modelAccess.enabled — recall
never steers the model into calls the media-policy gate would reject.
- omni.memory normalization moved to first use (ensureOmniMemoryConfig):
createToolRegistry needs recall.mode before initialize()'s omni block
runs; invalid settings stay startup-fatal on both paths.
* feat(omni): resolve resourceId to inputPath at the media-policy call gate
Stage B evidence-gathering path (memory design M §5.2): a gated
model/client call of a media-policy tool may now name its source by the
opaque session resourceId announced at delivery or returned by recall,
instead of a filesystem path the model must never see.
- evaluateMediaPolicyToolCall resolves the handle through the session
registry BEFORE the lockedArguments check (a resolved inputPath cannot
sidestep an operator-pinned input); unknown/fabricated handles and
inputPath+resourceId together reject as invalid_params; fixed_policy
origins remain untouched (RESERVED_ARGUMENT_KEYS already bans the key
in policy arguments).
- Shared io schema advertises resourceId as the alternative to
inputPath; required drops inputPath (presence is enforced in
validateMediaPolicyIoParams AFTER gate resolution, with an actionable
either-or message).
- assertMediaPolicyIo input errors now name only the file's basename:
those messages reach the model, and a handle-resolved call must not
leak the locator the handle stands in for — basename matches the
displayName the model already saw at delivery (D5: a deleted source
errors without disclosing its path).
* feat(omni): passive sideQuery media-memory recall before the main request
Stage B recall surface #2 (memory design M §9.3, D10 sideQuery mode):
- MediaMemoryRecallService grows the two sideQuery faces on the same
root-bounded walk recall() uses: candidateSummaries() builds the
bounded selector manifest (structure + capped preview, never raw
media/full text/paths; maxCandidateEntries cap, deterministic order),
and recallSelection() materializes a selection through the unified
protocol — an unknown/over-budget entryId rejects the WHOLE selection
(invalid_selection), never a partial fulfilment.
- runOmniMemorySideQuery orchestrates one passive pass: handles are
parsed from the 【媒体资源】 annotations the request itself carries
(never a project-wide scan), the selector runs on the runSideQuery
JSON face (entryIds only, validate enforces manifest membership and
maxSelectedEntries), and every failure — timeout, selector error,
rejected selection — degrades to an empty recall with a recorded
reason while the main request proceeds.
- client.ts injects the materialized recall as a system reminder on
UserQuery/Cron turns strictly BEFORE the main request is sent (M
§9.3: never retrofitted into a later turn); latency is bounded by
sideQuery.timeoutMs.
- disclosure.ts gains parseResourceHandleText, keying on the
harness-minted handle grammar so displayNames containing the
separator cannot confuse extraction.
* fix(omni): propagate the recall tool display name to every drift-guarded surface
A new core tool name must land in four places, not one: core
ToolNames/ToolDisplayNames, the cli i18n dictionaries, and the web-shell
display-name map plus its own zh dictionary. Two drift tests guard this
(cli i18n/index.test.ts and web-shell toolFormatting) and both were red.
* fix(omni): reject unknown omni.memory root keys, stop advising on deleted sources, keep the selector's view of the question
Three independent correctness fixes in the memory slice:
- Config normalization rejected unknown keys at every nested level but
not at the root, so `{recalll: {...}}` silently discarded the entire
configuration and ran default active mode while the operator believed
sideQuery was configured — the exact silent fallback this module's
startup-fatal stance exists to prevent.
- gapsForVersion emitted `artifact_unavailable` for a deleted source and
then CONTINUED the channel scan, adding `not_processed` siblings. The
advisor only filters `artifact_unavailable`, so recall steered the
model into evidence-gathering calls on a file that no longer exists —
the gate resolves the handle and the io assertion then fails. Nothing
can be gathered from a deleted file: report the loss and stop.
- The sideQuery selector dropped any text part starting with
`<system-reminder>`, but IDE context is PREPENDED INTO the user's own
part before the passive pass runs — so in IDE mode the selector never
saw the question and picked relevance-blind. Strip reminder blocks
instead of discarding the part.
* fix(omni): bump clip/keyframe descriptor versions alongside their role annotations
The role labels these tools gained are what memory maps to honest
coverage (`clip` → partial, `keyframe` → sampled); role-less outputs
derive `complete`. The descriptor version feeds the policy fingerprint,
which keys both the degradation cache and the content-identity execution
id — so leaving it at '1' let pre-role cache entries and recorded
executions converge onto the post-change fingerprint two ways: a cache
hit recommits with the persisted `role: undefined`, and the replay
branch returns without upgrading recorded entries.
Either path makes recall report a temporal excerpt or a sampled frame
set as complete coverage: the model then answers about footage it never
saw and the advisor never proposes closing the gap. Bumping the version
partitions pre- and post-annotation state instead.
* fix(omni): prune malformed memory record values instead of blacking out every read
`load()` validated the document envelope but not the record values inside
it. A single non-object value — one hand edit, bad merge, or truncated
sync — made every read path throw (`indexSnapshot` dereferences
`version.parentVersionId`, `findBindingBySha256` reads `version.sha256`),
and those throws are caught into miss/empty by the never-fatal read
wrappers. The result was a PERMANENT global recall blackout for the whole
project, with no `.corrupt-*` backup and nothing named in the debug log,
because the envelope itself stayed valid.
Prune the bad values individually and name them, matching the sibling
`OmniJsonCacheFile.load()` defense. Dangling references to pruned records
already degrade gracefully — a missing version reads as an
`artifact_unavailable` gap.
* fix(omni): give every file its own policy execution and derivative rows
Two Files with identical bytes shared one execution node: the key was
`sha256 ⊕ configHash`, so the second file took the replay branch and got
ZERO records of its own, while the first file's derivative versions were
handed back stamped with the second file's rootFileId. Recall on the
second file then reported false `not_processed` gaps across every channel
and re-paid the work, and the derivative's lineage pointed into a
different root's tree. M §11.2/§11.3 prescribe the opposite: identical
bytes never merge Files, each File writes its own PolicyExecution and
provenance, and only the computation and the stored bytes are reused.
Three layers, cleanly separated:
- execution identity is now per-File (source VERSION ⊕ config), so a
replay is a genuine same-file replay and stays idempotent;
- the content-identity reuse key (`sha256 ⊕ config`) is derived
separately and spans Files — that is what makes cross-file reuse
possible while provenance stays apart. A match records
`reusedExecutionId` (declared since the schema landed, never assigned
until now) pointing at the original;
- derivative File nodes are keyed by (root, object path) instead of the
object path alone, so each root owns cheap metadata rows over the SAME
content-addressed object rather than sharing a node whose root belongs
to whoever derived it first — bounded traversal (M §8) holds for both.
* fix(omni): anchor tool-result media to the object store, not its staging file
Collection recorded the source's persistent identity as the path it was
handed — correct for a user file, whose bytes stay in place (S §4), but
wrong for tool-result media: that path is a staging `.part` the funnel
deletes in its `finally` the same turn, while this same delivery promotes
the identical bytes into the content-addressed object store.
So the disclosed handle resolved to a file that no longer existed: any
media-policy tool the model pointed at it failed with ENOENT, and recall
reported `artifact_unavailable` across every channel for an artifact that
actually persists. Tool-origin sources now record (and bind) the
content-addressed object location, which is derivable from the hash
before promotion runs. If promotion never happens — the transport guard
omitted the media — the ref dangles and recall says
`artifact_unavailable`: honest, because the bytes were not retained.
* feat(omni): commit model and client policy successes through the same memory boundary
Memory design M §7.1 requires fixed calls, model tool calls and direct
client calls to enter ONE `OmniPolicySucceeded` boundary, and §17 names
both capture sites — the core scheduler and ACP's own executor. Only the
fixed-policy orchestrator ever committed: the scheduler built the
`PolicyArtifactBatch` (with the real origin) and nothing consumed it, and
the "model-call artifact bridge" its comment referred to did not exist.
So evidence gathering never accumulated. Recall reported a gap, the
advisor suggested a tool, the model called it with a session handle, the
tool succeeded and wrote real artifacts — and the next recall reported the
identical gap, re-paying the work every session, forever. That is exactly
the loop the resourceId path in this PR was built to close.
The bridge walks the same gates in the same order as the orchestrator:
validate each artifact against the descriptor (contained in the output
directory the call declared), require every declared output, promote to
the content-addressed store, THEN commit (M §6.4 / S §5 — no record may
reference bytes that are not yet in `objects/`). A handle-driven call
recovers its source identity from the session registry instead of
re-hashing the input; a raw-path call falls back to an idempotent
recognition upsert. Fixed-policy batches are skipped: the orchestrator
already owns them.
`resourceId` also joins the fingerprint's excluded keys. Like
`inputPath`, it is per-invocation plumbing — and it is minted fresh every
session, so leaving it in would make each session re-derive identical
work.
* feat(omni): reuse recorded policy outputs instead of re-running the tool
The degradation cache maps one input to ONE media derivative, so its own
comment admits that multi-output tools and text products are "simply not
cached — re-run instead of guessing". That made #8189's «同文件同 settings
二次触发同一 policy：直接复用，无重复执行» unmet in the cases that cost the
most: re-delivering an 81-minute movie re-ran the whole ASR pass every
time, because a transcript is a text product.
Memory already records every output of every execution, keyed by content
identity — so it can answer the reuse question the cache cannot. The
orchestrator now consults it BEFORE the cache and, on a hit, rebuilds the
deliverables from the recorded objects and skips execution entirely,
covering media, text and multi-output alike.
Reuse is verified exactly like a cache hit, because memory.json is
project-local and hand-editable: every object must still be a regular
file, must still hash to its recorded identity, and a media object must
re-recognize as a type the tool DECLARES producing — otherwise a crafted
record could route arbitrary store content through a policy that never
made it. Text is re-read from the promoted object rather than from the
entry's `inlineText`, which is truncated to
`collection.maxInlineTextBytes` and must never be delivered as the whole
transcript. Any doubt falls through to a fresh derivation.
The reusing file still commits its own execution, stamped with
`reusedExecutionId` (M §11.3): the computation is shared, the provenance
is not.
* docs(omni): reconcile the memory design with the S5 issue acceptance wording
Two acceptance criteria in #8188 read as direct contradictions of this
document, and one of them contradicts #8189 as well. Both are resolved by
naming the layer each statement belongs to, rather than by changing the
model.
- §11.2.1 (new): «内容相同的两个文件不重复建节点» and «不合并不同文件»
live on different layers. Content-keyed reuse covers the physical object
and the derived computation; the File node stays locator-keyed because
identity MUST be stable across content changes — key it on content and
§11.1's version chain and #8189's «修改内容后旧衍生物不默认召回» both
become inexpressible (an edit yields an unrelated node, and editing back
makes history a cycle). Also records why derivatives are keyed by
(root, object path).
- §9.2.1 (new): «按路径/内容 hash 查询» and «返回中永不暴露真实本地路径»
are reconciled per caller — the model gets session handles only
(unforgeable capability; a path parameter would let injected media steer
it into probing other files, whose transcripts recall returns), while
path/hash lookup stays a harness capability that already exists. The
known cost is written down too: a deleted file's memory is currently
unreachable, and the fix belongs in the authorization layer
(@-reference re-anchoring), not in widening what the model may pass.
The issue bodies were updated to match.
* fix(omni): reuse text products too, and fail on unknown omni settings sections
Both found by running the real pipeline rather than the unit suites.
Text-product reuse never fired. `findReusableOutputs` only resolves an
object path for outputs that have a derived VERSION node, which text
products do not have — so `rebuildReusedOutputs` bailed and the tool
re-ran. The comment there even said the caller reconstructs the path from
the content hash; that half was never written. A real run made it obvious:
extract-audio and keyframes logged reuse hits while the transcript re-ran
the entire ASR pass — the single most expensive thing #8189 exists to
avoid, and the headline claim of the reuse commit. The object store is
content-addressed, so the path is derivable; text outputs now reuse like
media. The existing test passed because it used a media artifact, so a
transcript-shaped case was added alongside it.
Unknown sections under `omni` were silently ignored. The settings loader
scans TOP-LEVEL keys only, and only to a debug line, so a nested typo was
caught by nothing: `omni.memoryy.recall.mode = sideQuery` started
cleanly and registered the active-mode recall tool — the operator's whole
configuration discarded without a word. The same hole swallowed
`omni.processingg`, which would silently disable every degradation
policy. `omni`'s own normalizers are startup-fatal by design, so the
namespace now rejects unknown sections with the allowed set named. The
permitted keys are derived from the settings schema, not hardcoded, so
they cannot drift.
* fix(omni): stop policy artifacts colliding, guard duration, record real execution windows
Three defects a real multi-session run surfaced. The first destroyed work.
**Fixed output filenames collided.** Every policy tool wrote a constant
name (`clip.mp4`, `downsampled.jpg`, `transcript.txt`). Under
fixed-policy orchestration that is safe — each invocation gets its own
staging directory — but `modelAccess` lets a caller pick a PERSISTENT
`outputDir`, and there the second call silently destroyed the first
artifact. In the run: a clip cut on day one was overwritten by a different
clip on day three, and the commentary written against the first clip began
describing the wrong footage. Worse, the model noticed something was off,
could not diagnose it, and skipped re-watching — so the defect surfaced as
degraded output quality rather than an error.
Names now carry the axes that distinguish artifacts: the source stem, plus
a natural variant where the operation has one (a clip's span, a frame's
index). Same source and variant still resolve to one name, so re-running
an operation replaces its own output — idempotent, not destructive. The
keyframe lister derives its matcher from the same template, so it can
never pick up a sibling video's frames out of a shared directory.
**The transport guard had no duration dimension.** A 98-minute film
downscaled to 474 MB cleared the 1 GiB byte ceiling, uploaded, and was
then refused by the provider for being too long: the guard reported
success, the caller got an opaque 400, and a 97-second transcode was paid
for nothing. Duration is a transport limit like the others, so
`maxDurationSeconds` now sits beside the byte and token checks with the
same semantics (unset disables it; missing metadata never rejects) and a
message that names duration and says downscaling cannot fix it.
**Model-origin executions recorded the collection window, not the work.**
`startedAt` was taken after the tool had already run, so extracting audio
from a 98-minute film was recorded as 0.6s. The scheduler already tracks
`executionStartTime` (approval wait excluded); it is now threaded through.
* test(omni): keep '+' in generated artifact stems
A derived artifact of a clip re-uses the clip's filename as its source
stem, and the clip's own variant contains '+' (`123s+40s`). Stripping it
made the derived stem drift from its parent (`clip-123s-40s-keyframe-…`).
'+' is portable on every major filesystem, so keep it: names now round-trip
through a derivation chain.
* feat(omni): re-anchor remembered media, and say when recall was truncated
Two friction points the character-analysis run surfaced.
**A remembered file's memory was unreachable once its bytes were gone.** A
handle is minted only at delivery, and delivery needs the bytes — so a
transcript or keyframe set outlived the file it described with no way back
into a session. The same wall hits an audit that does not want to
re-deliver a 2.4 GB film just to ask what work was recorded: with no
handle, recall rejects everything, and the natural next move (passing the
filename as a resourceId) is correctly refused. Observed exactly that: the
model reached for `resourceIds: ["robot-dreams.mkv"]` and got
`not issued in this session`.
An `@`-reference to a missing-but-remembered path now mints a handle from
the recorded identity — the user's reference is the same authorization a
delivery carries, so the fix belongs at the authorization layer rather
than by widening what the model may pass (design M §9.2.1). The model gets
the handle plus an explicit 【媒体缺失】 note, so it recalls instead of
trying to read. Path/locator lookup stays a harness capability.
**A truncated recall page was indistinguishable from an exhaustive one.**
An audit read 6 clips under `limit: 12` and reported "no keyframes were
ever extracted" while the store held 72 of them. The reader was honest
about what it saw — it simply had no way to know it was looking at a page.
`matchedEntries` now states the full match count, and only when the budget
actually cut something, so an exhaustive page stays as small as before
(§9.4 minimal return).
* feat(omni): tell the model what a media handle is and to recall before reprocessing
The handle annotation ships with every memory-known delivery, but nothing
in the model's context explained it: the media-guidance section listed only
the three disclosure markers, and the tool that consumes handles is
deferred, so its description — which does explain them — is not in context
until ToolSearch surfaces it. Meanwhile the same section actively tells the
model to gather missing evidence with the policy tools, i.e. to reprocess
from scratch exactly what memory already holds. The active-recall surface's
whole purpose could silently never fire.
The section now names the handle marker, states that it is the only
identity the model will ever get for that file, and says to consult recall
BEFORE reprocessing. Gated on active recall mode: under sideQuery the
harness injects recalled memory itself and the tool is not registered, so
naming it would invite a guaranteed unknown-tool error (D10).
The tool-display-name drift test now runs over every translating locale
rather than zh alone. `t()` has no cross-locale fallback, so a tool added
to one locale renders a raw English badge beside translated siblings in the
other.
* fix(omni): close the honesty and hygiene gaps found reviewing S5 memory
Fourteen findings from the PR review, verified one at a time. The ones
that could mislead a caller:
- a truncated recall entry now says so. maxTextChars (24k chars) is
smaller than the collection bound (64 KiB), so any long transcript is
cut at READ time while coverage still legitimately reports `complete`
— the model saw a prefix, no gap, and answered about audio it never
read. `contentTruncated` marks the prefix.
- a resourceId handle whose modality the tool does not accept is refused
as a parameter error instead of becoming a spawned ffmpeg that burns
the tool timeout and returns an opaque stderr tail.
- the advisor no longer suggests audio extraction against a video's
speech_text gap: that gap is still open in the very payload that
RETURNS the extracted track, so the suggestion repeated itself. Nor
does it suggest anything against `partial_coverage` — sampled evidence
stays sampled by design, so that advice could never close the gap.
- a derived version keeps the execution that actually produced it. A
second execution landing on byte-identical output was rewriting the
pointer, leaving the version naming an execution whose outputs it is
not.
- `maxAttempts` governs the client's retry loop only; a selection that
parses but names entries outside the manifest is refused without a
retry. The comment and the setting description said otherwise.
- `sideQuery.model: null` means the side-query default (the configured
fast model, falling back to the session model), not "the session's
active model" as three places claimed.
And the hygiene:
- duplicate resourceIds and repeated selector picks collapse instead of
spending the budget twice
- an empty materialization records `materialized_nothing` rather than
injecting a reminder that says nothing
- `candidateSummaries` lets a defect propagate (the caller already
degrades with a recorded reason) instead of swallowing the stack
- `kinds`/`roles` reject `[]`, which read as "restrict to nothing"
- count and millisecond settings are `integer` in the generated schema
- dropped three never-written fields; corrected the `managedId` doc,
which described an extension the key does not carry
* fix(omni): disclose the session handle for URL-delivered media too
The @-command/URL funnel records the file in memory and mints a registry
binding, then never tells the model the handle. Active recall rejects the
handle as never-issued, and the passive selector finds no handles to
consult — so a session could pay for a download, an upload, and a policy
run, and be structurally unable to recall any of it.
The withheld case is where it matters most: the model cannot see the
media, so the handle is its only remaining route to the content.
* docs(omni): record the two recall fields that keep a page honest
`contentTruncated` and `matchedEntries` both exist because a reader that
cannot tell a prefix from the whole, or a page from the set, draws
confident wrong conclusions — and both were shipped in code without the
design doc naming them.
* test(omni): pin the selector gate and the handle disclosure
Mutation probes showed both were free to break silently:
- inverting the selector's manifest-membership check kept the whole suite
green while, with the default single attempt, every passive recall
degrades to `selector_failed` — the feature dead in production. The
mock now forwards responses through `validate` the way production does,
so the happy path exercises the gate too; three tests fail on the
inversion.
- dropping `...handleParts` at any of the five replacement sites left
tool-produced or withheld media with no recall handle, and no test
cared. Each site now fails exactly one case.
Also pinned: the 4000-char selector request cap, the abort/timeout signal
composition (a Ctrl-C in the selector window must cancel it, not leave
the main request waiting out the full timeout), promptId attribution, and
the basename-only input error — whose leak-prevention was satisfied by
the old full-path message too. The manifest no-path assertion moved out
of the mock: a leak used to fail as "result was null" instead of naming
the path.
* test(omni): pin the memory invariants that mutation probes walked through
Sixteen behaviors the design doc states and the code implements, none of
which any test defended. Each was verified by breaking the source, seeing
the new test fail, and restoring it. The ones that matter most:
- gaps must speak for the CURRENT version's processing state, always, and
must be computed from the FULL subgraph. Two mutants — looping gaps over
every consulted version, and threading the request's kinds/roles filter
into the coverage scan — would have made recall report missing work that
exists, purely because the caller narrowed what it asked to see. The
existing history test could not catch either; the new fixtures invert
its direction (unprocessed history, processed current) and kill both.
- the selector-visible manifest: the 200-char description cap was never
binding (longest fixture was 184 chars), so removing it would have sent
full transcripts to the selector; and validation against the CAPPED
manifest let an entry past maxCandidateEntries be materialized though
the selector was never shown it.
- registry rebinding now pins first-binding-wins explicitly. Updating in
place would repoint a handle the model already holds; minting a second
handle for one version would break cross-call correlation.
- the store's rename-based write and newest-wins backup retention, pinned
black-box (inode replacement, no orphan .tmp) rather than by mocking.
- multi-output executions: every call site in the suite committed exactly
one output, so truncating the output loop was invisible.
- the config normalizer's real contract: it throws rather than clamps, its
bounds are non-strict, and it validates the MERGED values — lowering
only maxEntries to 6 is startup-fatal against the default 12.
* fix(omni): keep a withheld resource reachable, and stop losing why recall degraded
Three findings that each end the same way — the session holds a memory it
cannot get back to, and nothing says so:
- A transport-guard rejection threw away the handle it had already minted.
Every throw site sits downstream of the bind, and the omission branch
with the identical "over-limit, withheld" verdict does disclose its
handle — so the rejection was the one path that stranded a recorded
resource: the model can neither see the bytes nor ask memory about them.
The verdict now carries the handle, and the tool-result funnel emits it.
- Passive recall degrades silently by design (the turn proceeds without the
memory it was meant to carry). With a pinned-but-unavailable selector
model that is a permanent outage with nothing to see, so the reason is
now recorded at the injection seam, and the manifest-failure branch —
which had no log at all — gets one.
- Chinese queries never ranked. Splitting on separators yielded one token
per phrase, scored by whole-substring containment: unless an entry
repeated the caller's exact phrasing, everything scored zero and ordering
collapsed to newest-first. Unsegmented scripts now index as overlapping
character bigrams, split at script boundaries so `480p字幕` does not
produce bigrams that straddle them.
Also: `PolicySucceededCommit.created` had no consumer outside tests. It
distinguishes a content-identity replay from a fresh record, which is
exactly what a reader of the debug log cannot otherwise tell — "no new
execution appeared" reads identically to "collection did nothing".
* test(omni): cover the four seams where memory reaches the pipeline
Each was free to disappear without a red test:
- the orchestrator's post-promotion commit, its degradation-cache-hit
commit, and the memoryBinding threaded onto each WorkItem. Disabling the
commit, or always threading the source binding instead of the derived
one, left the suite green while every fixed-policy execution recorded
wrong or no lineage. Covered with a real MediaMemoryService on a tmp
store, reading memory.json back — including a two-stage chain, where
stage two must be EXECUTED_ON stage one's version, not the root.
- the reactive-degradation ladder's memory wiring, which had no coverage
at all beyond its pure helpers. One test pins that a rung's re-derivation
lands on the binding found by content hash; the other that an unknown
hash degrades normally and records NO execution, rather than inventing
provenance on someone else's lineage.
- the client injection seam: moving it after the systemReminders splice —
the exact regression that kills passive recall — now fails.
- the D10 exposure gate at registry-construction level: the recall tool
appears in active mode, is absent in sideQuery mode, and an invalid
omni.memory.recall.mode aborts instead of falling back to defaults.
* fix(omni): keep S5 off the root barrel, which was dragging 550 KB into ACP
`Check serve fast-path bundle closure` went red: the ACP agent's static
import closure had acquired iconv-lite's encoding tables (~553 KB).
The mechanism is worth writing down, because the obvious readings are all
wrong. Nothing imported iconv-lite. What S5 did was add five re-exports to
the ROOT barrel, which is statically imported by ~150 modules in the ACP
closure. esbuild assigns modules to chunks by which entry points reach
them, so every module added to the barrel's graph regroups the split — and
that regrouping moved `sync-file-encoding` (and through it `iconvHelper`,
whose top-level `import 'iconv-lite'` has side effects and cannot be shaken
out) into a chunk the ACP chunk statically imports.
So the fix is not "move the export somewhere else": my first two attempts
relocated it to the `omni` barrel, which is itself statically imported by
the root barrel and by tool-result-media, and stayed red. The barrel
surface itself has to come back to baseline.
None of the five needed to be there. Four had no consumer outside their own
module and tests; the rest are reached by dynamic import, which does not
care where the export lives:
- ACP's model-call collection gets a dedicated `./omniPolicyCollection`
subpath, the pattern `./goalWire` and `./transcriptRecords` already
establish for "the CLI needs one deep function, not a barrel".
- `reanchorRememberedMedia` moves to the `omni` door the `@`-reference
funnel already loads for every other omni call it makes.
- `createMediaMemoryRecallService`, `buildMediaMemoryRecallAdvisor`,
`extractRequestResourceIds`, `runOmniMemorySideQuery`,
`formatOmniMemorySideQueryReminder`, `OmniRecallMediaMemoryTool` and
`formatResourceHandleText` simply leave the root barrel.
Located by bisecting the 28 commits with the check itself, then diffing the
ACP closure module sets between a green and a red metafile: the delta was
exactly three modules, which is what named the mechanism.
* fix(omni): anchor URL media off its staging file, and record ACP calls as model calls
Two findings from the re-review, both real:
- The URL funnel hands its staging download to the delivery pipeline with
no lifetime marker, so memory recorded the `.part` path as the file's
persistent identity — the same defect C9 fixed for tool-result media,
missed on this third funnel. The handle already shown to the model
resolves to ENOENT the moment the funnel's `finally` runs: every later
policy call on it fails, and cross-session recall reports
`artifact_unavailable` for bytes the object store still holds. URL media
now anchors to the content-addressed object path, and the version's
source records the URL itself (`protocol: 'url'` existed in the schema,
unused until now) — the durable identity of downloaded media is where it
came from, not where it briefly landed.
- ACP committed its policy successes with `executionOrigin: 'client'`
while the modelAccess gate forty screens up pins the very same call as
`{ kind: 'model' }` ("Every ACP-originated call is a model call").
Recall provenance contradicted the gate that admitted the call.
The URL-anchor test is mutation-verified: reverting the lifetime marker
fails it.
* docs(omni): reconcile storage lifecycle with the shipped recovery, stake out GC
Section 6.1 described the recovery scan as drafted; the implementation
learned things the draft could not know, and the doc should carry them:
"delete all of staging/" became a grace window because a second CLI
process's recovery must not delete a live invocation's transcode out from
under it (config validation caps tool timeouts below the window for the
same reason), and corrupt-object deletion cascades into the degradation
cache so a cache hit can never point at bytes that are gone.
Section 6.2 is S6's landing point, sharpened by what S5 settled: the GC
root set has TWO sources in the memory snapshot (artifactRef.managedId
AND managed-protocol source locators — tool/URL media anchor their
identity there), the snapshot read is fail-closed (no roots readable, no
deletion at all), and object deletion reuses the recovery scan's existing
cache-cascade rather than growing a second one.
* test(omni): stop asserting an exact timeout the wall clock can undercut
The bucket loop's budget guard calls remainingTimeoutMs() right before
the ffmpeg call reads it again; on a slow runner a millisecond elapses
between the two reads and the exact-equality assertions on the first
call's timeoutMs turn flaky (observed on ubuntu CI: 599999 != 600000).
Freeze Date.now in the full-duration spread test (ffmpeg is mocked,
nothing there needs real time) and tolerate the guard-to-use drift in
the shared-budget test, which does need the real clock for its 50ms
burn. Restore spies in afterEach so the frozen clock never leaks into
neighbouring tests.
* feat(omni): S6 governance — object-store GC with byte budget, JSONL trajectory export (#9081)
* feat(omni): mark-and-sweep GC over the object store, with a byte budget
The store only ever grew: recovery cleans staging, downloads and
quarantine, but a committed object had no way out — one film-commentary
session leaves 38 objects behind, forever. S6 (storage design §6.2).
Two hard rules shape the implementation:
- An unreadable ledger deletes nothing. The root set is the memory
snapshot; if it cannot be read there is no telling "unreferenced" from
"unknown", so the run skips. This interacts with the store's own
corruption recovery (C8) in a way the first test caught: a corrupt
memory.json is backed up and REPLACED WITH AN EMPTY LEDGER — right for
recall, catastrophic for a GC that trusts it, because empty reads as
"nothing is referenced" and one corruption event would erase the whole
store. A fresh .corrupt-* backup therefore blocks the run until it ages
past the retention window.
- A referenced object is never deleted, even over budget. Roots come from
BOTH reference kinds in the snapshot — entries[].artifactRef.managedId
and managed-protocol source locators (tool/URL media anchor their file
identity in the store; missing these would delete exactly the objects
whose store copy is the only copy) — plus the live session registry,
which may hold a binding whose memory commit has not landed yet. Budget
pressure with only referenced objects left suspends NEW derivations
(policy design §8.4) instead: the orchestrator refuses to execute a
policy tool, while reuse and cache hits — which produce no new bytes —
stay allowed.
Sweep: unreferenced and older than omni.storage.retentionDays (14d
default); over omni.storage.maxTotalBytes (20 GiB default), oldest
unreferenced first regardless of age. Deletion cascades into the upload
cache and the degradation cache, same invariant recovery's corrupt-object
path enforces. Symlinked shards are never descended into. Runs once per
store root per process, asynchronously after startup recovery.
* feat(omni): export a session's trajectory as training-consumable JSONL
S6's second deliverable (issue #8190, absorbing #8196): the experiment's
whole point is trajectories — what media came in, what the pipeline did
to it, what the model actually saw, what it answered — and until now they
lived split across the chat transcript and memory.json in shapes no
training script can consume.
The exporter is a pure reader over those two artifacts. No collection
points are added: it cannot affect runtime behavior and works on sessions
recorded before it existed. Three record kinds per line — `turn` (request
text with reminders stripped and any injected passive recall extracted,
the media annotations the model saw, the response and its tool calls),
`execution` (full provenance: arguments, config hash, reuse marker), and
`file` (the durable identity: fileVersionId, sha256, origin, source).
The join is explicit rather than fabricated: model/client executions join
turns on callId ↔ invocationId (both sides record the scheduler's id);
fixed-policy executions join through file identities. No turn↔execution
edge is invented from timestamps — a wrong edge in training data is worse
than an explicit join key.
Privacy carries over from recall: reserved runtime keys (inputPath /
outputDir) are stripped from exported tool args, media bytes are never
embedded (sha256 is the pointer, valid while GC retention lasts), and
every execution line carries omniConfigHash so downstream can split
trajectories by processing configuration instead of mixing distributions.
Deterministic ordering makes re-exports byte-identical. An unreadable
memory store degrades to transcript-only output; an unreadable transcript
is an error — there is no trajectory without it. Wrapper script at
scripts/export-omni-trajectory.js; the function is exported through the
`omni` door (never the root barrel — see
|