mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
易良andQwen-Coder
49056c09e9
* feat(ci): canary-split the PR review model by PR number
The review job read its model from a single repository variable, so
evaluating a second model on real review traffic meant flipping every
review at once and flipping back to undo it.
Read a canary model and a percentage alongside the default, and route
`PR_NUMBER % 100 < percent` to the canary. The bucket is per PR, so a
PR keeps one model across its review rounds. Both variables unset (or a
zero/non-numeric percent) leaves today's behaviour byte-identical, and
reverting a rollout is one variable write.
The run also announces the model it chose: nothing else records it (the
cost ledger carries tokens, not a model name), so a rollout nobody can
attribute afterwards is unmeasurable.
* fix(ci): fail the canary percent safe to off, and pin the clamps that guard it
Out-of-range canary percents clamped to 100 — maximum rollout. Both
out-of-range paths failed open: a 4+ digit value hit the digit-count bound
and became 100, and a 1-3 digit value above 100 hit the >100 clamp and
became 100. Fat-fingering 50 as 500, or pasting a per-mille 1000, would
switch every PR review in the repository to the unproven model at once.
The block's own invariant is "the canary must never be the reason a review
breaks", so out-of-range now lands on off.
That diverges from SALVAGE_ELAPSED_PERCENT deliberately — there 100 means
cede, here it would mean full rollout — and the comment now says so.
Tests, each mutation-verified one mutant at a time:
- The clamp table pinned the old direction ('250' and '1000' both expected
'canary'), so it flipped with the fix. Each row now isolates one clamp
step: '0050' the leading-zero strip, '250' the >100 clamp, and
18446744073709551666 the digit bound — bash wraps that to 50, so
deleting the bound is a silent half rollout rather than off. The '1000'
row discriminated nothing and is gone.
- The "Sticky per PR" assertion compared two identical calls into a pure
function, so it could not fail. Replaced with a sweep of the whole bucket
range at 10% asserting the selected set is exactly 12300..12309. That
dominates the two literal boundary pins it replaces (12309 in-set, 12310
excluded) and additionally catches a nondeterministic bucket.
Mutation matrix (focused run, one mutant each, restored byte-identical):
revert clamp direction / delete strip / delete digit bound / delete >100
clamp / %100->%10 / -lt->-le / RANDOM bucket — all RED. RANDOM killed 3/3,
where the old self-equality assertion was measured at 22.5%.
Full file: 263 passed | 4 skipped (267). check-workflow-size.sh and
check-ci-budget.mjs rc=0; .size-baseline bumped to the file's exact 270879
bytes.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmvo5qtfnv2
* fix(ci): report the canary percent actually applied, and pin the call site
The cohort notice interpolated the raw REVIEW_MODEL_CANARY_PERCENT while
review_model_for_bucket applied a clamped one. Since the clamp-direction
fix, out-of-range values clamp to off, so a repository-variable typo (250,
1000, abc, "10 ") announced `canary 250%` while every PR routed to the
default model — the notice asserted a cohort that never ran, which is the
inverse of the truth in exactly the case the clamps exist to absorb. The
clamp chain moves into canary_percent_effective so the router and the
notice read one normalized value.
Also trims the canary model name before the call. A padded paste reaches
--model verbatim, the provider rejects it, and that failure is terminal
for the whole cohort on every re-review round while reading as flakiness.
Whitespace-only still means "no split" rather than an invented cohort. The
default model's untrimmed path is main-side at two call sites and is left
to its own change.
Adds an executed call-site test: it slices the real block (helper, router,
trim, call site, notice, MODEL_ARGS) and asserts on stdout, so the four
arguments the workflow actually passes are pinned for the first time — the
unit rows above supply their own positional arguments and cannot see them.
Co-authored-by: Qwen-Coder <[email protected]>
Patrol-Run: qwen-pr-closeout/jmvoc6arqva
---------
Co-authored-by: Qwen-Coder <[email protected]>