From 94448d368db0c4748914c883f9185046933264b1 Mon Sep 17 00:00:00 2001 From: Shaojin Wen Date: Sat, 31 Oct 2026 21:08:39 +0800 Subject: [PATCH] fix(verify-pr): keep the mutation matrix off the tree live cells measure on (#15999) * fix(verify-pr): keep the mutation matrix off the tree live cells measure on A local round interleaved its mutation matrix with 80 live daemon cells on one tree. A build inside a mutation row propagated the mutant into dist/, and the hand-patched repair still carried the mutant expression, so every live cell and the whole matrix were withdrawn and the round shipped as inconclusive. Require a dedicated tree for the mutation matrix, gate every daemon or dist-level harness launch on a scripted dist-versus-source anchor check with a mismatch-reporting control, and forbid hand-patching dist to repair a contaminated build. Pin the new rule in the workflow meta-test. * test(verify-pr): pin the launch gate's coverage and control clauses Review on 0d06209b2d showed the four rig-isolation pins left two clauses of the launch gate unasserted: deleting "covers every PR anchor" or the mismatch-reporting control still passed 4/4. Pin both so a future edit cannot silently weaken the gate. Also name the clean-dist launch gate in the mutation-runner pointer, so the section that proves a mutant reached dist and the rule that proves the measurement tree's dist is clean cross-reference each other. * fix(verify-pr): define the launch gate's reference side and isolate per-tree artifacts Review on 21014c30d6 (R1-1/R1-2/R1-3, RC1-RC5): the gate compared dist against an unnamed source, so a self-consistent mutant build passed it; "PR anchor" was undefined and collided with the mutation-anchor sense; the control proved liveness, not refusal; the gate had no carve-out for the producing row's own run; and tree isolation stopped at the checkout while the workspace links, the shared sandbox tag, and the single Maven repository stayed host-global. Name the reference side as a sha256 manifest of each measurement arm's clean dist recorded before the matrix starts, retire the undefined anchor term, require the gate to refuse a known-mutant build, exempt the producing row's own launches, and extend isolation to node_modules, the sandbox image tag, and the Maven repository. Replace the fragment pins with the whole bullet pinned inside its section plus the whole pointer line; a 17-row mutation acceptance goes red on every row, and a real-file M8 run turned the suite red before a byte-identical restore. * fix(verify-pr): qualify the matrix-tree copies and name both sandbox levers Review on ff867ec035: the cp -al recipe dropped the per-package qualifier the install strategy already requires (root marked 17.0.1 vs packages/core marked 15.0.12 shadow for real), hardlinks share inodes so a writing row truncates the round's copy, the sandbox clause named only the launch-side variable while the build reads QWEN_SANDBOX_IMAGE_TAG and --sandbox-image outranks the export, and the pin's section slice failed open when the closing heading was renamed. Carry the qualifier and the never-symlink rule, require real copies from the store for rows that write under node_modules, name the build lever and the flag precedence, and assert the slice bounds. The acceptance matrix grows to 26 rows, all red on mutation including both heading renames; a real-file rename run turned the suite red before a byte-identical restore. * fix(verify-pr): scope the sandbox image per launch and bound the pin slice by the next heading Review on b99f52a5e3: sat where a Docker tag must go and the export's value was never spelled out; a shell-wide export leaks the matrix image into every later cell; the two cp -al copies disagreed on when hardlinks are safe (the base build's postinstall step writes under node_modules); the slice bound was a named heading, so an inserted section could shelter a moved bullet; and the prose named a build variable no test pinned at its read site. Spell a sanitized label and the full image reference, inject the variable per launch instead of exporting it, carry the write caveat in both copies with an explicit copy import method (pnpm's auto falls back to hardlinks without clones), bound the slice on the next heading, and pin process.env.QWEN_SANDBOX_IMAGE_TAG in build_sandbox.js. The acceptance matrix grows to 30 rows, all red on mutation; real-file heading-insertion and variable-rename runs each turned the suite red before byte-identical restores. Merged main (#15988, #16002); the suite now runs 197 tests, all green. --- .qwen/skills/verify-pr/SKILL.md | 44 ++++++++++++++- scripts/tests/qwen-triage-workflow.test.js | 64 ++++++++++++++++++++++ 2 files changed, 106 insertions(+), 2 deletions(-) diff --git a/.qwen/skills/verify-pr/SKILL.md b/.qwen/skills/verify-pr/SKILL.md index 6d6693263d0..aa7fb94e232 100644 --- a/.qwen/skills/verify-pr/SKILL.md +++ b/.qwen/skills/verify-pr/SKILL.md @@ -583,7 +583,11 @@ differs only by the change under test; the verdict is the pair of counts. hardlinks are free, and the relative `@qwen-code/*` symlinks then resolve into the base tree, which the realpath assertion above can confirm. Do not symlink the directory itself — the internal links would resolve through it - back into the head tree and both arms would run head code. + back into the head tree and both arms would run head code. Hardlinks + share inodes, and the base build can write under `node_modules` (its + postinstall patch step), so a base arm that rebuilds needs real copies + from the store instead + (`pnpm install --offline --frozen-lockfile --config.package-import-method=copy`). - Report the cell table: environment per cell, observable oracle per cell (exit code, stderr line, wire request, rendered frame), and `X/Y` at head vs control. "5/9 flip from broken to fixed" is the shape to aim for. @@ -975,7 +979,8 @@ least one test that imports it. (`../escape` has two dots and a slash). Likewise a mode-000 fixture violates read and execute at once, so dropping either check survived 990 unit tests. -- Runner hygiene (time caps, PID-only kills, restoring the tree, syntax +- Runner hygiene (time caps, PID-only kills, restoring the tree, a tree of + its own apart from every live cell, the clean-`dist` launch gate, syntax checks that do not execute) is in **Rig hygiene (local rounds)** below. **A test that can skip is coverage only where it executes.** A test gated on @@ -1267,6 +1272,41 @@ results in both directions. - Run scenarios that kill processes, revoke grants or wedge a shared resource alone on their arm; three runs that overlapped a concurrent revocation probe had to be set aside and repeated. +- Give the mutation matrix a tree of its own — never the tree a live cell + is measured on. Restoring the source after each row is not enough: a + build inside a row propagates the mutant into `dist/`, and a daemon + started mid-row loads it. Isolate every artifact a launch resolves, not + just the checkout: `cp -al` the round's `node_modules` — root plus any + per-package ones — into the matrix tree (never symlink the directory + itself) and assert each `@qwen-code/*` link resolves inside it (a nested + tree without its own `node_modules` walks up to the round's); a row that + writes under `node_modules` needs real copies from the store + (`pnpm install --offline --frozen-lockfile --config.package-import-method=copy`), + because hardlinks share the round's inodes and pnpm's default import + falls back to hardlinks where clones are unavailable. Build the sandbox + image under a per-tree tag — a sanitized tree label, never a path + (`QWEN_SANDBOX_IMAGE_TAG=