perf(release): reuse verified native audio prebuilds across releases (#15854)

* perf(release): reuse verified native audio prebuilds across releases

Every release rebuilds the five native audio prebuilds even when nothing
that affects those binaries changed. The build is short; the wait is not.
For v0.26.3 the Linux x64 prebuild queued 74 minutes to execute for 50
seconds, and the collection step queued again after it — and `publish`
depends on that collection, so the whole release sits behind it.

Fingerprint the audio build inputs and reuse an artifact from an earlier
successful official main release when the fingerprint matches. The key
covers the audio package, the dependency lock, the package manager pin and
the contents of the producing workflow and this helper, so the Node pin and
the prebuildify target arguments are part of it. The package version is
excluded deliberately: no part of the native build reads it, and including
it would make reuse impossible between releases.

Reuse means executing bytes an earlier run produced, so provenance is
checked rather than assumed: exact fingerprint name, unexpired, same
repository for both run and head run, and a completed successful
release.yml run on main from workflow_dispatch or schedule. The pinned
download-artifact already verifies the artifact digest.

Retention is checked differently at selection and consumption. Selection
requires 48 hours of headroom so the artifact outlives the release;
consumers require only that it has not expired. Re-applying the margin
against a later clock would reject an artifact the release had already
committed to, after the native matrix was skipped and with no fallback.

Lookup, trial download and payload validation are all non-fatal, and the
reuse ID is published only once a trial download has validated; an empty ID
runs the full matrix, which is today's behaviour. The publisher does not
trust that decision: it recomputes the fingerprint from its own checkout,
re-verifies the source run, and validates the payload again, fatally — by
then the matrix was skipped, so a bad artifact must stop the release.

This does not depend on ECS_RELEASE_HOST_RUNNER_ENABLED and takes effect
with the switch unset.

The collector needed three fixes to be a trustworthy producer. Its
`rm -rf -- "$GITHUB_WORKSPACE/packages/audio-capture/prebuilds"` resolved
through whatever `packages` was on a persistent host: with a leftover
symlink it deleted an external tree and exited 0, and the following
download wrote through the same symlink. It now wipes through a shared
preflight that canonicalises the workspace and refuses paths resolving
outside it. It also ran `node` with no Node contract, inheriting Node from
the hosted image by accident. And its release-only label pinned `x64` while
the runners holding that label are ARM64, which never matches and queues
until timeout.

Contract tests pin the wiring, each confirmed to fail against an injected
regression. Design doc in both languages.

* fix(release): reconcile audio reuse workflow contracts

Share the pre-checkout wipe, preserve the workflow size cap, and let fingerprint failures rebuild native audio.

Co-authored-by: Qwen-Coder <[email protected]>

* fix(ci): use block-style release permissions

Co-authored-by: Qwen-Coder <[email protected]>

* fix(release): drop x64 from publish's release-host expression

The review's unresolved item: collect got the ARM64 label fix but publish kept
["self-hosted", "linux", "x64", "ecs-qwen-release-host"], which can never
match the ARM64 machines holding that label, so the OIDC-carrying job would
queue forever once ECS_RELEASE_HOST_RUNNER_ENABLED is true. Same expression as
the collect fix, same one-word drop.

* docs(release): clarify audio reuse rollout and validation

Co-authored-by: Qwen-Coder <[email protected]>

* test(release): cover artifact overwrite and contained symlink guards

* test(web-shell): include transcript coverage in recovery fixture

Bring the fixture correction from #15883 into this branch to address the observed smoke failure.

Co-authored-by: Qwen-Coder <[email protected]>

* fix(release): make audio cache seeding best effort

Keep the validated same-run artifact usable when retaining a keyed copy fails. Pin the reuse wiring and correct the retention and producer fixtures without changing release trust boundaries.

Co-authored-by: Qwen-Coder <[email protected]>

* chore(test): apply prettier to the audio prebuild fixtures

* docs(release): clarify audio reuse run provenance

Co-authored-by: Qwen-Coder <[email protected]>

---------

Co-authored-by: Qwen-Coder <[email protected]>
Co-authored-by: zzj <[email protected]>
This commit is contained in:
authored and GitHub committed 2026-10-30 11:27:38 +00:00
1 parent bf074c4ad8
commit 7c9e79e70b
11 files changed
+1331 -35

No files matched your search

+180
View File
@@ -0,0 +1,180 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { execFileSync } from 'node:child_process';
import { createHash } from 'node:crypto';
import { lstatSync, readFileSync, readdirSync } from 'node:fs';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
export const AUDIO_TARGETS = [
'darwin-arm64',
'darwin-x64',
'linux-arm64',
'linux-x64',
'win32-x64',
];
export function audioPrebuildKey(root, ref, workflowRoot) {
const git = (...args) => execFileSync('git', args, { cwd: root });
const paths = git(
'ls-tree',
'-r',
'--name-only',
'-z',
ref,
'--',
'packages/audio-capture',
'pnpm-lock.yaml',
'pnpm-workspace.yaml',
'.pnpmfile.mjs',
'.npmrc',
'package.json',
)
.toString()
.split('\0')
.filter(Boolean)
.sort();
// Older release refs have no frozen native dependency graph to reuse.
if (!paths.includes('pnpm-lock.yaml')) return '';
const hash = createHash('sha256');
const add = (name, data) =>
hash.update(JSON.stringify([name, Buffer.from(data).toString('base64')]));
for (const path of paths) {
// Tree entries include the content ID and mode without buffering large headers.
let data = git('ls-tree', ref, '--', path);
if (path === 'packages/audio-capture/package.json') {
const manifest = JSON.parse(git('show', `${ref}:${path}`));
delete manifest.version;
data = JSON.stringify(manifest);
} else if (path === 'package.json') {
data = JSON.stringify({
packageManager: JSON.parse(git('show', `${ref}:${path}`))
.packageManager,
});
}
add(path, data);
}
for (const path of [
'.github/workflows/audio-capture-prebuilds.yml',
'.github/scripts/audio-prebuilds.mjs',
])
add(path, readFileSync(join(workflowRoot, path)));
return hash.digest('hex');
}
export function validateAudioPrebuilds(directory) {
if (!lstatSync(directory).isDirectory())
throw new Error('Invalid prebuild directory');
const targets = readdirSync(directory).sort();
if (JSON.stringify(targets) !== JSON.stringify([...AUDIO_TARGETS].sort())) {
throw new Error(
'Audio prebuilds must contain exactly the five release targets',
);
}
for (const target of targets) {
const targetPath = join(directory, target);
if (!lstatSync(targetPath).isDirectory())
throw new Error(`Invalid target: ${target}`);
const files = readdirSync(targetPath);
if (!files.length) throw new Error(`Missing native binary: ${target}`);
for (const file of files) {
const stat = lstatSync(join(targetPath, file));
if (!file.endsWith('.node') || !stat.isFile() || stat.size === 0) {
throw new Error(`Invalid native binary: ${target}/${file}`);
}
}
}
}
// Selection demands enough retention to outlive the rest of the release;
// consumption only demands that the bytes are still there. Re-applying the
// margin downstream would reject an artifact that prepare already committed
// to, after the fresh-build job was skipped and with no fallback left.
export const AUDIO_SELECTION_MARGIN_MS = 48 * 60 * 60 * 1000;
export async function verifyAudioArtifact(
github,
repo,
artifact,
key,
now = Date.now(),
marginMs = 0,
) {
if (
!key ||
artifact.name !== `audio-capture-prebuilds-${key}` ||
artifact.expired ||
!(Date.parse(artifact.expires_at) >= now + marginMs) ||
artifact.workflow_run?.head_branch !== 'main' ||
artifact.workflow_run.repository_id !==
artifact.workflow_run.head_repository_id
) {
throw new Error(
'Audio artifact has an invalid identity, source, or retention',
);
}
const { data: run } = await github.rest.actions.getWorkflowRun({
...repo,
run_id: artifact.workflow_run.id,
});
const repository = `${repo.owner}/${repo.repo}`;
if (
run.status !== 'completed' ||
run.conclusion !== 'success' ||
run.head_branch !== 'main' ||
run.path !== '.github/workflows/release.yml' ||
!['workflow_dispatch', 'schedule'].includes(run.event) ||
run.repository?.full_name !== repository ||
run.head_repository?.full_name !== repository
) {
throw new Error(
'Audio artifact must come from a successful official main release',
);
}
return { artifact_id: artifact.id, run_id: run.id };
}
export async function findAudioArtifact(github, repo, key, warn) {
if (!key) return undefined;
try {
const { data } = await github.rest.actions.listArtifactsForRepo({
...repo,
name: `audio-capture-prebuilds-${key}`,
per_page: 100,
});
for (const artifact of data.artifacts) {
try {
return await verifyAudioArtifact(
github,
repo,
artifact,
key,
Date.now(),
AUDIO_SELECTION_MARGIN_MS,
);
} catch (error) {
warn(`Skipping audio artifact ${artifact.id}: ${error.message}`);
}
}
} catch (error) {
warn(`Audio reuse lookup failed; rebuilding: ${error.message}`);
}
return undefined;
}
if (process.argv[1] === fileURLToPath(import.meta.url)) {
const [command, ...args] = process.argv.slice(2);
if (command === 'key') {
console.log(
`cache_key=${audioPrebuildKey(process.cwd(), args[0], args[1])}`,
);
} else if (command === 'validate') {
validateAudioPrebuilds(args[0]);
} else {
throw new Error(`Unknown audio prebuild command: ${command}`);
}
}
+2 -2
View File
@@ -14,7 +14,7 @@
# real, bump the number and say why in the PR.
2226 assign-issue-owner.yml
3555 assign-pr-owner.yml
6457 audio-capture-prebuilds.yml
7107 audio-capture-prebuilds.yml
9023 auto-minimize-spam.yml
9551 build-and-publish-image.yml
55276 cd-cua-driver.yml
@@ -55,7 +55,7 @@
22345 release-sdk-python.yml
31270 release-sdk.yml
17941 release-vscode-companion.yml
45269 release.yml
45270 release.yml
44285 repo-hygiene.yml
1079 scorecard-monthly.yml
9751 sdk-java-contract-open-prs.yml
+71 -17
View File
@@ -20,6 +20,10 @@ on:
inputs:
ref:
type: 'string'
required: true
cache_key:
type: 'string'
required: false
default: ''
workflow_dispatch:
@@ -60,36 +64,56 @@ jobs:
ref: '${{ inputs.ref || github.sha }}'
persist-credentials: false
- uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
if: "${{ runner.environment != 'self-hosted' }}"
if: "${{ !startsWith(matrix.os, 'ubuntu') }}"
with:
node-version: '22'
# Install only this package's deps (skip the node-gyp-build install hook;
# prebuildify does its own compile below).
node-version: '22.22.3'
# Use the lock that participates in the reuse fingerprint. Older refs
# without pnpm retain the original build path and cannot seed reuse.
- name: 'Install build deps'
if: "${{ runner.environment != 'self-hosted' }}"
run: 'npm install --no-workspaces --ignore-scripts --no-audit --no-fund'
if: "${{ !startsWith(matrix.os, 'ubuntu') }}"
working-directory: '.'
shell: 'bash'
run: |-
if [ -f pnpm-lock.yaml ]; then
corepack pnpm --filter @qwen-code/audio-capture install --frozen-lockfile --ignore-scripts --reporter=append-only
else
cd packages/audio-capture
npm install --no-workspaces --ignore-scripts --no-audit --no-fund
fi
- name: 'Build prebuild'
if: "${{ runner.environment != 'self-hosted' }}"
if: "${{ !startsWith(matrix.os, 'ubuntu') }}"
shell: 'bash'
run: |
npm run prebuildify
npm run prebuildify -- --target 22.22.3
# Cross-compile the Intel (x64) slice on this arm64 runner instead of
# a separate macos-13 runner (frameworks are universal). See #5642.
if [ "$RUNNER_OS" = 'macOS' ]; then
npm run prebuildify -- --arch x64
npm run prebuildify -- --target 22.22.3 --arch x64
fi
# Keep Linux binaries compatible with older glibc even on newer ECS hosts.
- name: 'Build Linux prebuild on ECS'
if: "${{ runner.environment == 'self-hosted' }}"
# Use the same Linux toolchain on hosted and ECS runners for cache identity.
- name: 'Build Linux prebuild'
if: "${{ startsWith(matrix.os, 'ubuntu') }}"
working-directory: '.'
shell: 'bash'
run: |-
docker run --rm --user "$(id -u):$(id -g)" \
-e HOME=/tmp \
-v "$PWD:/build" -w /build node:22-bookworm \
bash -euc 'npm install --no-workspaces --ignore-scripts --no-audit --no-fund; npm run prebuildify'
-e HOME=/tmp -e COREPACK_HOME=/tmp/corepack \
-v "$PWD:/build" -w /build node:22.22.3-bookworm \
bash -euc '
if [ -f pnpm-lock.yaml ]; then
corepack pnpm --filter @qwen-code/audio-capture install --frozen-lockfile --ignore-scripts --reporter=append-only
else
cd packages/audio-capture
npm install --no-workspaces --ignore-scripts --no-audit --no-fund
cd ../..
fi
cd packages/audio-capture
npm run prebuildify -- --target 22.22.3
'
- name: 'Upload prebuild'
uses: 'actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a' # v7.0.1
with:
overwrite: true
name: 'prebuilds-${{ matrix.os }}-${{ matrix.artifact_suffix || matrix.arch }}'
path: 'packages/audio-capture/prebuilds/'
if-no-files-found: 'error'
@@ -107,11 +131,41 @@ jobs:
pattern: 'prebuilds-*'
merge-multiple: true
path: 'packages/audio-capture/prebuilds'
- name: 'List collected prebuilds'
run: 'find prebuilds -type f'
- name: 'Checkout audio workflow helper'
uses: 'actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10' # v6.0.3
with:
ref: '${{ github.workflow_sha }}'
persist-credentials: false
sparse-checkout: |-
.github/scripts/audio-prebuilds.mjs
.github/actions/self-hosted-node
sparse-checkout-cone-mode: false
path: '.audio-workflow'
- name: 'Use pre-installed Node.js (self-hosted)'
if: "${{ runner.environment == 'self-hosted' }}"
uses: './.audio-workflow/.github/actions/self-hosted-node'
- name: 'Setup Node.js'
if: "${{ runner.environment != 'self-hosted' }}"
uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
with:
node-version: '22.22.3'
- name: 'Validate complete audio prebuilds'
working-directory: '.'
run: 'node .audio-workflow/.github/scripts/audio-prebuilds.mjs validate packages/audio-capture/prebuilds'
- name: 'Upload combined prebuilds'
uses: 'actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a' # v7.0.1
with:
name: 'audio-capture-prebuilds'
overwrite: true
path: 'packages/audio-capture/prebuilds/'
if-no-files-found: 'error'
- name: 'Retain reusable audio prebuilds'
if: "${{ inputs.cache_key != '' }}"
continue-on-error: true
uses: 'actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a' # v7.0.1
with:
name: 'audio-capture-prebuilds-${{ inputs.cache_key }}'
path: 'packages/audio-capture/prebuilds/'
if-no-files-found: 'error'
overwrite: true
+101 -4
View File
@@ -54,8 +54,12 @@ jobs:
${{ github.repository == 'QwenLM/qwen-code' }}
permissions:
contents: 'read'
actions: 'read'
outputs:
release_sha: '${{ steps.source.outputs.release_sha }}'
audio_prebuild_key: '${{ steps.audio_key.outputs.cache_key }}'
audio_artifact_id: '${{ steps.audio_reuse.outputs.artifact_id }}'
audio_run_id: '${{ steps.audio_reuse.outputs.run_id }}'
release_tag: '${{ steps.version.outputs.RELEASE_TAG }}'
release_version: '${{ steps.version.outputs.RELEASE_VERSION }}'
npm_tag: '${{ steps.version.outputs.NPM_TAG }}'
@@ -85,6 +89,7 @@ jobs:
persist-credentials: false
sparse-checkout: |-
/.github/scripts
/.github/workflows/audio-capture-prebuilds.yml
/scripts/assert-release-version.mjs
/scripts/check-release-tarball-exec-bits.mjs
/scripts/release-packages.mjs
@@ -146,6 +151,59 @@ jobs:
- *checkout_release_workflow_scripts
- name: 'Fingerprint audio build inputs'
id: 'audio_key'
continue-on-error: true
env:
RELEASE_SHA: '${{ steps.source.outputs.release_sha }}'
run: |-
directory=$(mktemp -d "${RUNNER_TEMP}/audio-reuse.XXXXXX")
echo "directory=$directory" >> "$GITHUB_OUTPUT"
node .release-workflow/.github/scripts/audio-prebuilds.mjs key "$RELEASE_SHA" .release-workflow >> "$GITHUB_OUTPUT"
- name: 'Find reusable audio prebuilds'
id: 'audio_candidate'
continue-on-error: true
uses: 'actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3' # v9.0.0
env:
AUDIO_KEY: '${{ steps.audio_key.outputs.cache_key }}'
with:
script: |-
const { findAudioArtifact } = await import(process.env.GITHUB_WORKSPACE + '/.release-workflow/.github/scripts/audio-prebuilds.mjs');
const artifact = await findAudioArtifact(github, context.repo, process.env.AUDIO_KEY, core.warning);
for (const [key, value] of Object.entries(artifact ?? {})) core.setOutput(key, value);
- name: 'Try reusable audio prebuilds'
id: 'audio_download'
if: "${{ steps.audio_candidate.outputs.artifact_id != '' }}"
continue-on-error: true
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
with:
artifact-ids: '${{ steps.audio_candidate.outputs.artifact_id }}'
run-id: '${{ steps.audio_candidate.outputs.run_id }}'
github-token: '${{ github.token }}'
merge-multiple: true
path: '${{ steps.audio_key.outputs.directory }}'
- name: 'Accept complete audio prebuilds'
id: 'audio_reuse'
if: "${{ steps.audio_download.outcome == 'success' }}"
continue-on-error: true
env:
AUDIO_DIRECTORY: '${{ steps.audio_key.outputs.directory }}'
AUDIO_ARTIFACT_ID: '${{ steps.audio_candidate.outputs.artifact_id }}'
AUDIO_RUN_ID: '${{ steps.audio_candidate.outputs.run_id }}'
run: |-
node .release-workflow/.github/scripts/audio-prebuilds.mjs validate "$AUDIO_DIRECTORY"
echo "artifact_id=$AUDIO_ARTIFACT_ID" >> "$GITHUB_OUTPUT"
echo "run_id=$AUDIO_RUN_ID" >> "$GITHUB_OUTPUT"
- name: 'Remove trial audio download'
if: "${{ always() && steps.audio_key.outputs.directory != '' }}"
env:
AUDIO_DIRECTORY: '${{ steps.audio_key.outputs.directory }}'
run: 'rm -rf -- "$AUDIO_DIRECTORY"'
- name: 'Get the version'
id: 'version'
env:
@@ -518,10 +576,11 @@ jobs:
name: 'Audio Capture Prebuilds'
needs: 'prepare'
if: |-
${{ github.repository == 'QwenLM/qwen-code' }}
${{ github.repository == 'QwenLM/qwen-code' && needs.prepare.outputs.audio_artifact_id == '' }}
uses: './.github/workflows/audio-capture-prebuilds.yml'
with:
ref: '${{ needs.prepare.outputs.release_sha }}'
cache_key: '${{ needs.prepare.outputs.audio_prebuild_key }}'
build_release:
name: 'Build Release Artifacts'
@@ -542,7 +601,10 @@ jobs:
github.repository == 'QwenLM/qwen-code' &&
vars.ECS_RELEASE_HOST_RUNNER_ENABLED == 'true' &&
needs.prepare.result == 'success' &&
needs.audio_capture_prebuilds.result == 'success' &&
(
needs.audio_capture_prebuilds.result == 'success' ||
(needs.audio_capture_prebuilds.result == 'skipped' && needs.prepare.outputs.audio_artifact_id != '')
) &&
(
github.event.inputs.force_skip_tests == 'true' ||
(
@@ -591,15 +653,45 @@ jobs:
- *reset_release_workflow_scripts
- *checkout_release_workflow_scripts
- &verify_reused_audio
name: 'Verify reusable audio source'
if: "${{ needs.prepare.outputs.audio_artifact_id != '' }}"
uses: 'actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3' # v9.0.0
env:
RELEASE_SHA: '${{ needs.prepare.outputs.release_sha }}'
AUDIO_ARTIFACT_ID: '${{ needs.prepare.outputs.audio_artifact_id }}'
with:
script: |-
const { audioPrebuildKey, verifyAudioArtifact } = await import(process.env.GITHUB_WORKSPACE + '/.release-workflow/.github/scripts/audio-prebuilds.mjs');
const key = audioPrebuildKey(process.env.GITHUB_WORKSPACE, process.env.RELEASE_SHA, process.env.GITHUB_WORKSPACE + '/.release-workflow');
const { data: artifact } = await github.rest.actions.getArtifact({ ...context.repo, artifact_id: Number(process.env.AUDIO_ARTIFACT_ID) });
await verifyAudioArtifact(github, context.repo, artifact, key);
- &download_reused_audio
name: 'Download reusable audio prebuilds'
if: "${{ needs.prepare.outputs.audio_artifact_id != '' }}"
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
with:
artifact-ids: '${{ needs.prepare.outputs.audio_artifact_id }}'
run-id: '${{ needs.prepare.outputs.audio_run_id }}'
github-token: '${{ github.token }}'
merge-multiple: true
path: 'packages/audio-capture/prebuilds'
- &download_fresh_audio
name: 'Download audio capture prebuilds'
if: |-
${{ github.repository == 'QwenLM/qwen-code' }}
${{ github.repository == 'QwenLM/qwen-code' && needs.prepare.outputs.audio_artifact_id == '' }}
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
with:
name: 'audio-capture-prebuilds'
path: 'packages/audio-capture/prebuilds'
- &validate_audio
name: 'Validate complete audio prebuilds'
if: "${{ github.repository == 'QwenLM/qwen-code' }}"
run: 'node .release-workflow/.github/scripts/audio-prebuilds.mjs validate packages/audio-capture/prebuilds'
- *reset_release_workflow_scripts
- *checkout_release_workflow_scripts
- name: 'Build Bundle and Prepare Package'
@@ -660,7 +752,8 @@ jobs:
) &&
(
github.repository != 'QwenLM/qwen-code' ||
needs.audio_capture_prebuilds.result == 'success'
needs.audio_capture_prebuilds.result == 'success' ||
(needs.audio_capture_prebuilds.result == 'skipped' && needs.prepare.outputs.audio_artifact_id != '')
) &&
(
github.event.inputs.force_skip_tests == 'true' ||
@@ -759,8 +852,12 @@ jobs:
- *reset_release_workflow_scripts
- *checkout_release_workflow_scripts
- *verify_reused_audio
- *download_reused_audio
- *download_fresh_audio
- *validate_audio
- *reset_release_workflow_scripts
- *checkout_release_workflow_scripts
@@ -0,0 +1,164 @@
# Audio prebuild reuse
[简体中文](./2026-10-29-audio-prebuild-reuse.zh-CN.md)
## Problem
Every release rebuilds the native audio prebuilds for all five targets, even
when nothing that affects those binaries changed. The build itself is short;
the wait is not. For v0.26.3 the Linux x64 prebuild waited 74 minutes for a
hosted runner to execute for 50 seconds, and the collection step queued again
after it. Because `publish` depends on that collection, the whole release sits
behind it.
Audio prebuilds are a good reuse candidate precisely because they are
expensive to schedule and cheap to verify: five native `.node` files whose
inputs are fully described by the repository at a known commit.
## Design
A release fingerprints its audio build inputs, looks for an artifact from an
earlier successful official `main` release with the same fingerprint, and
reuses it instead of rebuilding.
### What the fingerprint covers
The content identity (git blob OID) of everything that can change the built
binaries: `packages/audio-capture`, `pnpm-lock.yaml`, `pnpm-workspace.yaml`,
`.pnpmfile.mjs`, `.npmrc`, the root `package.json`'s `packageManager` pin, plus
the contents of the producing workflow and this helper. Hashing the workflow is
what makes the Node pin and the `prebuildify --target` arguments part of the
key.
Two deliberate exclusions: `packages/audio-capture/package.json`'s `version`
field, because a version bump is packaging metadata that no part of the native
build reads, and everything in the root `package.json` except `packageManager`.
Without the first exclusion no release could ever reuse another's prebuilds.
A ref with no `pnpm-lock.yaml` produces an empty key and never reuses — older
refs have no frozen native dependency graph to match against.
### What makes an artifact trustworthy
Reuse means executing bytes built by an earlier run, so provenance is checked
rather than assumed. A candidate must be named for the exact fingerprint, be
unexpired, come from a run whose repository and head repository are both this
repository, and come from a _completed, successful_ run of `release.yml` on
`main` triggered by `workflow_dispatch` or `schedule`. That excludes fork PRs,
`workflow_run` producers, and direct dispatches of the audio workflow.
Transport integrity is already covered: the pinned `download-artifact` compares
the artifact digest and fails on mismatch.
### Retention: selection vs consumption
Selection requires at least 48 hours of remaining retention, so a chosen
artifact outlives the rest of the release. The downstream consumer requires
only that the artifact has not expired.
These must differ. Re-applying the selection margin against a later clock
rejects an artifact the release has already committed to: an artifact with
48h1m remaining passes in `prepare` and then fails two minutes later with
47h59m still available — after `prepare` skipped the native matrix, leaving no
fallback.
### Failure handling
Fingerprinting, lookup, trial download and validation are non-fatal, and the
reuse ID is only published after a _trial download and payload validation_
succeed:
- lookup API failure → no candidate
- fingerprint or temporary-directory failure → no key, so the native matrix runs
- download failure → acceptance skipped (keyed on `outcome`, since
`continue-on-error` rewrites `conclusion`)
- payload validation failure → the step fails before writing the output
- reusable upload failure → the current release proceeds; future reuse may miss
An empty reuse ID runs the full native matrix. Fresh builds now pin Node and
the prebuildify target to 22.22.3 and use filtered, frozen pnpm dependencies
when a lockfile exists; only older refs retain the npm install path. Linux on hosted and ECS runners
builds run that same frozen install inside `node:22.22.3-bookworm`, preserving
the existing glibc baseline. The trial bytes are discarded and re-downloaded
by the consumer.
Both the ECS build job and the hosted publisher independently recompute the
fingerprint from their own checkout, re-verify the source run, download by
artifact ID, and validate the payload again. That verification is fatal — by
then the native matrix was skipped, so a bad artifact must stop the release
rather than ship. A download failure or artifact deletion at this stage also
stops the release; it does not restart the skipped matrix.
Validation checks exactly the five target directories, each containing only
nonempty regular `.node` files and no symlinks. It is a shape check, not a
platform check; the provenance and digest checks above carry the trust.
### Collector hygiene
The collector merges per-platform artifacts on a possibly persistent workspace,
where `download-artifact`'s `merge-multiple` merges into existing paths, so a
stale `prebuilds/` tree would leak into the artifact it uploads.
It uses the shared release-runner preflight already on main, which canonicalises
and wipes the workspace and refuses paths resolving through a symlinked
component or outside the runner workspace. This change preserves that action
and its cleanup implementation.
The action fetches the cleanup and disk-floor scripts at the workflow SHA into
the runner's temporary directory before checkout. It isolates Git/npm/Docker/gh
state, provides a job-local npm prefix, asserts Node 22 and checks disk space.
The cleanup script itself does not depend on Node.
The collector uses the existing pre-installed Node preflight on self-hosted
runners and pins hosted `setup-node` to 22.22.3 before the validator. Native
compilation always uses Node 22.22.3. The collector remains architecture-free;
the Linux build legs retain their matching x64 or arm64 release-runner labels.
### Relationship to the release host switch
Audio reuse works with `ECS_RELEASE_HOST_RUNNER_ENABLED` either enabled or unset.
The decision happens in `prepare`. With the switch unset, the existing hosted
publisher builds and consumes the audio payload. When enabled, the ECS producer
consumes it before handing release outputs to the hosted publisher, which also
verifies the audio source and payload. npm publication remains hosted in both
cases, preserving the split already merged in #15832.
This change neither alters runner routing nor enables the switch. A cache hit
skips the native matrix; a miss follows the existing hosted/ECS routing.
## Validation
Fingerprint, provenance, retention and payload behaviour are covered by unit
tests with direct witnesses, including the retention boundary that motivated
the selection/consumption split. The workflow wiring — non-fatal lookup,
`outcome`-keyed acceptance, matrix skip, publisher re-verification, collector
ordering and the architecture-free label — is pinned by contract tests.
The latest review adds regression-sensitive wiring assertions and fixes the
retention test's clock; these additions await CI and were not run locally.
A branch release dry-run can verify the miss path, all five native payloads and
publisher consumption without publishing. Reuse provenance is checked at the
run level: repository, `head_branch`, event, workflow path and successful
conclusion. `head_branch` identifies the dispatch branch, not the built
`inputs.ref`. A run dispatched on a feature branch cannot seed an eligible
candidate; one dispatched on `main` with `inputs.ref` pointing at a feature
branch can. The fingerprint pins the audio build inputs, not the source
commit's membership in `main`. A successful eligible run must first retain a
keyed artifact; a later run with the same key can exercise reuse.
Record the switch state and actual runners used by each dry run. A hosted run
does not verify ECS execution, and a miss does not verify reuse.
## Affected files
`.github/scripts/audio-prebuilds.mjs`, the audio and release workflows, the
shared release-runner preflight it reuses, workflow tests under `scripts/tests`,
and the workflow size baseline.
## Open questions
Runner image updates under unchanged labels do not change the fingerprint.
N-API stabilises the Node ABI, not glibc or toolchain linkage, so this reuses a
verified binary rather than guaranteeing that a fresh build would be
bit-for-bit identical. Linux on hosted and ECS runners uses a versioned Bookworm container tag, not
an immutable image digest; macOS and Windows hosted images also continue to evolve. Pinning those
environments completely is outside this change.
@@ -0,0 +1,140 @@
# 音频预构建复用
[English](./2026-10-29-audio-prebuild-reuse.md)
## 问题
每次发布都会为五个目标重新构建原生音频预构建产物,即使影响这些二进制的输入
没有任何变化。构建本身很短,等待不短。v0.26.3 的 Linux x64 预构建排队 74
分钟,执行只用 50 秒,随后汇总步骤又排了一次。由于 `publish` 依赖该汇总,
整个发布都卡在后面。
音频预构建适合复用,恰恰因为它调度代价高而校验代价低:五个原生 `.node`
文件,其输入可以由某个已知 commit 上的仓库内容完整描述。
## 设计
发布先对音频构建输入取指纹,再查找此前一次成功的官方 `main` 发布产出的、
指纹相同的产物,命中则复用而不是重新构建。
### 指纹覆盖什么
覆盖所有可能改变构建结果的内容身份(git blob OID):
`packages/audio-capture`、`pnpm-lock.yaml`、`pnpm-workspace.yaml`、
`.pnpmfile.mjs`、`.npmrc`、根 `package.json` 的 `packageManager` pin,以及
生产该产物的 workflow 和这个辅助脚本本身的内容。把 workflow 纳入哈希,正是
Node 版本 pin 和 `prebuildify --target` 参数进入 key 的方式。
两处有意排除:`packages/audio-capture/package.json` 的 `version` 字段——版本
号是打包元数据,原生构建的任何环节都不读它;以及根 `package.json` 中除
`packageManager` 以外的全部内容。没有第一条排除,任何发布都无法复用另一次
发布的产物。
没有 `pnpm-lock.yaml` 的 ref 产出空 key,永不复用——较老的 ref 没有可比对的
原生依赖冻结图。
### 什么样的产物才可信
复用意味着执行由此前运行构建出的字节,所以来源要校验而不是假定。候选产物
必须以精确指纹命名、未过期、所属运行的 repository 与 head repository 都是
本仓库,且来自 `main` 上由 `workflow_dispatch` 或 `schedule` 触发的、
**已完成且成功**的 `release.yml` 运行。这排除了 fork PR、`workflow_run`
生产者,以及直接 dispatch 音频 workflow 的情形。
传输完整性已有保障:pin 住的 `download-artifact` 会比对产物摘要,不一致即
失败。
### 保留期:选择 vs 消费
选择时要求至少剩余 48 小时保留期,使选中的产物能活过发布的其余阶段。下游
消费方只要求产物尚未过期。
两者必须不同。按更晚的时钟重复套用选择期余量,会拒绝一个发布已经选定的
产物:剩 48 小时 1 分钟的产物在 `prepare` 通过,两分钟后失败——而此时还剩
47 小时 59 分钟,且 `prepare` 已经跳过原生构建矩阵,没有任何回退。
### 失败处理
指纹计算、查询、试下载与校验都是非致命的,而复用 ID 只在**试下载与载荷
校验**都成功后才发布:
- 查询 API 失败 → 没有候选
- 指纹计算或临时目录创建失败 → 没有 key,执行原生构建矩阵
- 下载失败 → 跳过接受步骤(以 `outcome` 为判据,因为 `continue-on-error`
会改写 `conclusion`)
- 载荷校验失败 → 该步骤在写出 output 之前就失败
- 复用产物上传失败 → 当前发布继续,后续发布可能无法复用
空的复用 ID 会执行完整的原生构建矩阵。重新构建现在把 Node 和 prebuildify
目标固定为 22.22.3,存在锁文件时使用按包筛选、冻结的 pnpm 依赖;只有旧 ref
保留 npm install 路径。hosted 与 ECS 上的 Linux 在 `node:22.22.3-bookworm` 容器中执行
同样的冻结安装,保留已有的 glibc 基线。试下载的字节会被丢弃、由消费方
重新下载。
ECS 构建任务与 hosted 发布任务分别使用自己的 checkout 重算指纹、重新校验
来源运行、按 artifact ID 下载并再次校验载荷。这一校验是致命的——此时原生构建
矩阵已被跳过,所以坏产物必须让发布停止,而不是发出去。此阶段下载失败或
产物被删除也会停止发布,不会重新启动已跳过的构建矩阵。
校验检查恰好五个目标目录,每个目录只包含非空的普通 `.node` 文件且无符号
链接。这是形状校验而不是平台校验;信任由上述来源校验和摘要校验承载。
### 汇总任务的卫生
汇总任务在可能持久的工作区上合并各平台产物,而 `download-artifact` 的
`merge-multiple` 会合并进已存在的路径,因此残留的 `prebuilds/` 目录会泄漏
进它上传的产物。
汇总任务使用 main 已有的共享 release runner 预检:规范化并清空工作区,
拒绝经由符号链接组件解析、或落在 runner 工作区之外的路径。本次保留该
共享 action 及其清理实现。
action 在 checkout 前按 workflow SHA 把清理与磁盘余量脚本下载到 runner
临时目录。它隔离 Git/npm/Docker/gh 状态,提供任务私有 npm 安装目录,
断言 Node 22 并检查磁盘余量。清理脚本本身不依赖 Node。
汇总任务在 self-hosted 上使用现有的预装 Node 预检,在 hosted 上把
`setup-node` 固定为 22.22.3,两者都在载荷校验前执行。原生编译始终使用
Node 22.22.3。汇总任务保留不含架构的 label;Linux 构建任务保留与目标
匹配的 x64 或 arm64 release runner label。
### 与 release host 开关的关系
`ECS_RELEASE_HOST_RUNNER_ENABLED` 启用或未设置时,音频复用都能工作。
决策发生在 `prepare`。开关未设置时,现有 hosted 发布任务构建并消费音频
产物;启用后,ECS 构建任务先消费音频产物,再把发布输出交给 hosted 发布
任务,后者也会校验音频来源和载荷。两种情况下 npm 发布都保持 hosted,
保留 #15832 已合入的构建与发布拆分。
本改动不改变 runner 路由,也不启用开关。命中缓存时跳过原生构建矩阵,
未命中时沿用现有 hosted/ECS 路由。
## 验证
指纹、来源、保留期和载荷行为由单元测试配合直接 witness 覆盖,包括促成
选择/消费分离的那个保留期边界。workflow 的接线——非致命查询、以 `outcome`
为判据的接受步骤、矩阵跳过、发布方重新校验、汇总任务步骤顺序以及不含架构的
label——由契约测试固定。最新 review 补充了针对接线回归的断言,并修正保留期
测试的时钟;这些新增断言等待 CI,未在本地执行。
分支 release dry-run 可以在不发布的情况下验证未命中路径、五种原生载荷与
发布方消费。复用来源在运行层面校验:仓库、`head_branch`、事件、workflow
路径和成功结论。`head_branch` 表示派发分支,而不是实际构建的 `inputs.ref`。
从功能分支派发的运行不能产生合格候选;从 `main` 派发、但将 `inputs.ref`
指向功能分支的运行可以。指纹固定的是音频构建输入,并不证明源码提交属于
`main`。先由一次成功且符合来源条件的运行保存带指纹的产物,后续相同 key 的
运行才可验证复用。每次 dry-run 都应记录开关状态及各任务实际使用的 runner。hosted
运行不能证明 ECS 路径,未命中运行也不能证明复用路径。
## 涉及文件
`.github/scripts/audio-prebuilds.mjs`、音频与发布 workflow、它复用的共享
release runner 预检、`scripts/tests` 下的 workflow 测试,以及 workflow
大小基线。
## 待确认
label 不变而 runner 镜像更新时,指纹不会变化。N-API 稳定的是 Node ABI,
不是 glibc 或工具链链接,所以这里复用的是经过验证的二进制,而不保证重新
构建能产出逐字节一致的结果。hosted 与 ECS 上的 Linux 使用带版本的 Bookworm 容器 tag,
并非不可变镜像摘要;macOS 和 Windows 的 hosted 镜像也会继续更新。完全固定这些环境不在本次范围。
@@ -0,0 +1,270 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { readFileSync } from 'node:fs';
import { describe, expect, it } from 'vitest';
import { parse } from 'yaml';
import { AUDIO_TARGETS } from '../../.github/scripts/audio-prebuilds.mjs';
const workflow = (name) =>
parse(readFileSync(`.github/workflows/${name}.yml`, 'utf8'));
const releaseYaml = workflow('release');
const audioYaml = workflow('audio-capture-prebuilds');
const step = (job, name) =>
releaseYaml.jobs[job].steps.find((entry) => entry.name === name);
describe('audio prebuild reuse wiring', () => {
it('builds every target required by payload validation', () => {
const build = audioYaml.jobs.build;
const targets = build.strategy.matrix.include.flatMap(({ os, arch }) => {
if (os.startsWith('macos')) return [`darwin-${arch}`, 'darwin-x64'];
return [`${os.startsWith('windows') ? 'win32' : 'linux'}-${arch}`];
});
expect(targets.sort()).toEqual([...AUDIO_TARGETS].sort());
expect(
build.steps.find((entry) => entry.name === 'Build prebuild').run,
).toContain('npm run prebuildify -- --target 22.22.3 --arch x64');
});
it('only publishes a reuse ID after a trial download and payload validation', () => {
// The ID comes from the acceptance step, not the lookup: a candidate that
// cannot be downloaded or does not validate must leave the output empty so
// the fresh build still runs.
for (const [output, source] of Object.entries({
audio_prebuild_key: '${{ steps.audio_key.outputs.cache_key }}',
audio_artifact_id: '${{ steps.audio_reuse.outputs.artifact_id }}',
audio_run_id: '${{ steps.audio_reuse.outputs.run_id }}',
}))
expect(releaseYaml.jobs.prepare.outputs[output]).toBe(source);
expect(releaseYaml.jobs.prepare.permissions.actions).toBe('read');
const lookup = step('prepare', 'Find reusable audio prebuilds');
const fingerprint = step('prepare', 'Fingerprint audio build inputs');
const download = step('prepare', 'Try reusable audio prebuilds');
const accept = step('prepare', 'Accept complete audio prebuilds');
expect(fingerprint.id).toBe('audio_key');
expect(lookup.env.AUDIO_KEY).toBe(
'${{ steps.audio_key.outputs.cache_key }}',
);
const steps = releaseYaml.jobs.prepare.steps;
for (const [before, after] of [
[fingerprint, lookup],
[lookup, download],
[download, accept],
])
expect(steps.indexOf(before)).toBeLessThan(steps.indexOf(after));
// Failure before acceptance must leave the fresh-build path available.
for (const entry of [fingerprint, lookup, download, accept])
expect(entry['continue-on-error']).toBe(true);
expect(fingerprint.run).toContain('echo "directory=$directory"');
expect(fingerprint.run.indexOf('echo "directory=$directory"')).toBeLessThan(
fingerprint.run.indexOf('audio-prebuilds.mjs key'),
);
expect(download.if).toBe(
"${{ steps.audio_candidate.outputs.artifact_id != '' }}",
);
// outcome, not conclusion: continue-on-error rewrites conclusion to success.
expect(accept.if).toBe("${{ steps.audio_download.outcome == 'success' }}");
expect(accept.run.trim().split('\n')).toEqual([
'node .release-workflow/.github/scripts/audio-prebuilds.mjs validate "$AUDIO_DIRECTORY"',
'echo "artifact_id=$AUDIO_ARTIFACT_ID" >> "$GITHUB_OUTPUT"',
'echo "run_id=$AUDIO_RUN_ID" >> "$GITHUB_OUTPUT"',
]);
// The trial bytes are always discarded, including when a step failed.
const cleanup = step('prepare', 'Remove trial audio download');
expect(download.with.path).toBe('${{ steps.audio_key.outputs.directory }}');
for (const entry of [accept, cleanup])
expect(entry.env.AUDIO_DIRECTORY).toBe(download.with.path);
expect(cleanup.if).toContain('always()');
expect(cleanup.run).toBe('rm -rf -- "$AUDIO_DIRECTORY"');
// The fingerprint has to see the producing workflow, so it must be
// checked out alongside the helper.
const checkout = releaseYaml.jobs.prepare.steps.find((entry) =>
entry.with?.['sparse-checkout']?.includes('/.github/scripts'),
);
expect(checkout.with['sparse-checkout']).toContain(
'/.github/workflows/audio-capture-prebuilds.yml',
);
});
it('skips the native matrix on a hit and still gates the publisher', () => {
const native = releaseYaml.jobs.audio_capture_prebuilds;
expect(native.if).toContain(
"needs.prepare.outputs.audio_artifact_id == ''",
);
expect(native.with.ref).toBe('${{ needs.prepare.outputs.release_sha }}');
expect(native.with.cache_key).toBe(
'${{ needs.prepare.outputs.audio_prebuild_key }}',
);
// A skipped matrix is only acceptable when a reuse ID was actually
// selected, otherwise the publisher would ship without prebuilds.
const gate = releaseYaml.jobs.publish.if;
expect(gate).toContain("needs.audio_capture_prebuilds.result == 'success'");
expect(gate).toContain(
"needs.audio_capture_prebuilds.result == 'skipped' && needs.prepare.outputs.audio_artifact_id != ''",
);
});
it.each(['build_release', 'publish'])(
'rechecks the reused source and validates either path in %s',
(job) => {
expect(releaseYaml.jobs[job].permissions.actions).toBe('read');
// prepare's decision is not trusted blindly: each consumer recomputes the
// fingerprint from its own checkout and re-verifies the source run.
const verify = step(job, 'Verify reusable audio source');
expect(verify.if).toBe(
"${{ needs.prepare.outputs.audio_artifact_id != '' }}",
);
expect(verify['continue-on-error']).toBeUndefined();
expect(verify.with.script).toContain('audioPrebuildKey');
expect(verify.with.script).toContain('verifyAudioArtifact');
expect(verify.with.script).toContain(
'await verifyAudioArtifact(github, context.repo, artifact, key);',
);
expect(verify.with.script).not.toContain('AUDIO_SELECTION_MARGIN_MS');
const reused = step(job, 'Download reusable audio prebuilds');
const fresh = step(job, 'Download audio capture prebuilds');
expect(reused.if).toBe(
"${{ needs.prepare.outputs.audio_artifact_id != '' }}",
);
expect(fresh.if).toContain(
"needs.prepare.outputs.audio_artifact_id == ''",
);
expect(reused.with['artifact-ids']).toBe(
'${{ needs.prepare.outputs.audio_artifact_id }}',
);
expect(reused.with['run-id']).toBe(
'${{ needs.prepare.outputs.audio_run_id }}',
);
// Exactly one of the two runs, and both land in the same directory.
expect(reused.with.path).toBe(fresh.with.path);
const validateStep = step(job, 'Validate complete audio prebuilds');
expect(validateStep['continue-on-error']).toBeUndefined();
expect(validateStep.if).toBe(
"${{ github.repository == 'QwenLM/qwen-code' }}",
);
expect(validateStep.run).toBe(
`node .release-workflow/.github/scripts/audio-prebuilds.mjs validate ${reused.with.path}`,
);
// Whichever path ran, the payload is validated before the build consumes it.
const names = releaseYaml.jobs[job].steps.map((entry) => entry.name);
const validate = names.indexOf('Validate complete audio prebuilds');
expect(validate).toBeGreaterThan(names.indexOf(reused.name));
expect(validate).toBeGreaterThan(names.indexOf(fresh.name));
expect(validate).toBeLessThan(
names.indexOf('Build Bundle and Prepare Package'),
);
},
);
it('retains a fingerprint-named artifact for later releases', () => {
const collect = audioYaml.jobs.collect;
const retain = collect.steps.find(
(entry) => entry.name === 'Retain reusable audio prebuilds',
);
// Without the keyed upload nothing can ever be reused; the fixed-name
// artifact stays for the same-run consumer.
expect(retain.if).toBe("${{ inputs.cache_key != '' }}");
expect(retain['continue-on-error']).toBe(true);
expect(retain.with.path).toBe('packages/audio-capture/prebuilds/');
expect(retain.with.name).toBe(
'audio-capture-prebuilds-${{ inputs.cache_key }}',
);
expect(retain.with['if-no-files-found']).toBe('error');
expect(retain.with.overwrite).toBe(true);
expect(
collect.steps.find((entry) => entry.name === 'Upload combined prebuilds')
.with.name,
).toBe('audio-capture-prebuilds');
expect(
collect.steps.find((entry) => entry.name === 'Upload combined prebuilds')
.with.overwrite,
).toBe(true);
expect(
collect.steps.find((entry) => entry.name === 'Upload combined prebuilds')[
'continue-on-error'
],
).toBeUndefined();
// The producer builds the ref the release pinned, not the branch tip.
expect(
audioYaml.jobs.build.steps.find((entry) =>
entry.uses?.startsWith('actions/checkout@'),
).with.ref,
).toBe('${{ inputs.ref || github.sha }}');
});
});
describe('prebuild collector hygiene', () => {
it('wipes the workspace through the shared preflight before merging', () => {
// download-artifact's merge-multiple merges into existing paths, so a
// stale prebuilds/ tree on a persistent host would leak into the artifact.
// The shared preflight wipes with path containment; the subtree rm it
// replaced followed a leftover symlinked parent out of the workspace.
const steps = audioYaml.jobs.collect.steps;
expect(steps[0].uses).toBe('$/.github/actions/prepare-release-runner');
expect(steps[1].name).toBe('Merge per-platform prebuilds');
for (const entry of steps)
expect(entry.run ?? '').not.toContain('rm -rf -- "${GITHUB_WORKSPACE}');
});
it('keeps the release-only label architecture-free', () => {
// The release pool includes ARM64 runners. An "x64"
// component does not fall back to hosted — it never matches, and the job
// queues until it times out. Resolve the expression rather than
// substring-matching it, which would pass with the pin present.
const route = (enabled, repository = 'QwenLM/qwen-code') =>
Function(
'github',
'vars',
'fromJSON',
`return (${audioYaml.jobs.collect['runs-on'].slice(3, -2)})`,
)(
{ repository },
{ ECS_RELEASE_HOST_RUNNER_ENABLED: enabled },
JSON.parse,
);
expect(route('true')).toEqual([
'self-hosted',
'linux',
'ecs-qwen-release-host',
]);
expect([route(''), route('false'), route('true', 'fork/repo')]).toEqual([
['ubuntu-latest'],
['ubuntu-latest'],
['ubuntu-latest'],
]);
});
it('has a Node contract on both runner kinds before it runs node', () => {
// The validator shells out to `node`; the collector previously inherited
// Node from the hosted image by accident and asserted nothing.
const steps = audioYaml.jobs.collect.steps;
const node = steps.filter(
(entry) =>
entry.uses?.endsWith('.github/actions/self-hosted-node') ||
entry.uses?.startsWith('actions/setup-node@'),
);
expect(node.map((entry) => entry.if)).toEqual([
"${{ runner.environment == 'self-hosted' }}",
"${{ runner.environment != 'self-hosted' }}",
]);
const validate = steps.findIndex((entry) =>
entry.run?.includes('audio-prebuilds.mjs validate'),
);
for (const entry of node)
expect(steps.indexOf(entry)).toBeLessThan(validate);
// self-hosted-node is resolved from the helper checkout, so that checkout
// has to include it.
expect(
steps.find((entry) => entry.name === 'Checkout audio workflow helper')
.with['sparse-checkout'],
).toContain('.github/actions/self-hosted-node');
});
});
+365
View File
@@ -0,0 +1,365 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { execFileSync } from 'node:child_process';
import {
mkdirSync,
mkdtempSync,
rmSync,
symlinkSync,
writeFileSync,
} from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { afterEach, describe, expect, it, vi } from 'vitest';
import {
AUDIO_SELECTION_MARGIN_MS,
AUDIO_TARGETS,
audioPrebuildKey,
findAudioArtifact,
validateAudioPrebuilds,
verifyAudioArtifact,
} from '../../.github/scripts/audio-prebuilds.mjs';
const directories = [];
const temporary = () => {
const path = mkdtempSync(join(tmpdir(), 'audio-prebuilds-'));
directories.push(path);
return path;
};
afterEach(() => {
vi.restoreAllMocks();
directories
.splice(0)
.forEach((path) => rmSync(path, { recursive: true, force: true }));
});
const write = (root, path, content) => {
mkdirSync(dirname(join(root, path)), { recursive: true });
writeFileSync(join(root, path), content);
};
describe('audio prebuild identity', () => {
function fixture() {
const root = temporary();
const git = (...args) =>
execFileSync('git', args, {
cwd: root,
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
}).trim();
git('init', '-b', 'main');
git('config', 'user.name', 'Test');
git('config', 'user.email', '[email protected]');
for (const [path, content] of Object.entries({
'package.json': '{"version":"1.0.0","packageManager":"[email protected]"}',
'packages/audio-capture/package.json':
'{"name":"audio","version":"1.0.0","scripts":{"prebuildify":"prebuildify --napi"}}',
'packages/audio-capture/native/audio.cc': 'native source',
'packages/audio-capture/binding.gyp': 'binding',
'packages/audio-capture/install.js': 'install',
'pnpm-lock.yaml': 'locked dependencies',
'pnpm-workspace.yaml': 'workspace settings',
'.pnpmfile.mjs': 'dependency hook',
'.npmrc': 'registry=https://registry.npmjs.org',
'.github/workflows/audio-capture-prebuilds.yml': 'native workflow',
'.github/scripts/audio-prebuilds.mjs': 'trusted helper',
}))
write(root, path, content);
const commit = () => {
git('add', '.');
git('commit', '-m', 'fixture');
return git('rev-parse', 'HEAD');
};
const ref = commit();
return {
root,
git,
ref,
commit,
key: () => audioPrebuildKey(root, 'HEAD', root),
};
}
it('ignores release versions and uses the selected commit rather than dirty files', () => {
const f = fixture();
const key = f.key();
write(
f.root,
'packages/audio-capture/package.json',
'{"name":"audio","version":"2.0.0","scripts":{"prebuildify":"prebuildify --napi"}}',
);
write(
f.root,
'package.json',
'{"version":"2.0.0","packageManager":"[email protected]"}',
);
f.commit();
expect(f.key()).toBe(key);
write(f.root, 'packages/audio-capture/native/audio.cc', 'changed');
expect(f.key()).toBe(key);
f.commit();
expect(f.key()).not.toBe(key);
expect(audioPrebuildKey(f.root, f.ref, f.root)).toBe(key);
});
it.each([
'packages/audio-capture/binding.gyp',
'packages/audio-capture/install.js',
'pnpm-lock.yaml',
'pnpm-workspace.yaml',
'.pnpmfile.mjs',
'.npmrc',
'.github/workflows/audio-capture-prebuilds.yml',
'.github/scripts/audio-prebuilds.mjs',
])('invalidates reuse for %s', (path) => {
const f = fixture();
const key = f.key();
write(f.root, path, 'changed input');
f.commit();
expect(f.key()).not.toBe(key);
});
it('invalidates changed package scripts and package manager versions', () => {
const f = fixture();
const key = f.key();
write(
f.root,
'packages/audio-capture/package.json',
'{"name":"audio","version":"1.0.0","scripts":{"prebuildify":"different"}}',
);
f.commit();
const changedScript = f.key();
expect(changedScript).not.toBe(key);
write(f.root, 'package.json', '{"packageManager":"[email protected]"}');
f.commit();
expect(f.key()).not.toBe(changedScript);
});
it('fingerprints the workflow checkout separately from the release ref', () => {
const f = fixture();
const workflowRoot = temporary();
write(
workflowRoot,
'.github/scripts/audio-prebuilds.mjs',
'trusted helper',
);
const path = '.github/workflows/audio-capture-prebuilds.yml';
write(workflowRoot, path, 'workflow at dispatch');
const key = audioPrebuildKey(f.root, f.ref, workflowRoot);
write(f.root, path, 'unrelated release checkout edit');
expect(audioPrebuildKey(f.root, f.ref, workflowRoot)).toBe(key);
write(workflowRoot, path, 'updated native toolchain');
expect(audioPrebuildKey(f.root, f.ref, workflowRoot)).not.toBe(key);
});
it('does not reuse refs without a native dependency lock', () => {
const f = fixture();
f.git('rm', 'pnpm-lock.yaml');
f.commit();
expect(f.key()).toBe('');
});
it('fingerprints large native headers without a child-process buffer limit', () => {
const f = fixture();
const path = 'packages/audio-capture/native/miniaudio.h';
write(f.root, path, 'x'.repeat(4 * 1024 * 1024));
f.commit();
const key = f.key();
write(f.root, path, 'y'.repeat(4 * 1024 * 1024));
f.commit();
expect(f.key()).not.toBe(key);
});
});
describe('audio prebuild payload', () => {
const targets = [
'darwin-arm64',
'darwin-x64',
'linux-arm64',
'linux-x64',
'win32-x64',
];
const fixture = () => {
const root = temporary();
for (const target of targets)
write(root, `${target}/audio.node`, 'native binary');
return root;
};
it('accepts all five nonempty targets', () => {
expect(AUDIO_TARGETS).toEqual(targets);
expect(() => validateAudioPrebuilds(fixture())).not.toThrow();
});
it.each(AUDIO_TARGETS)('rejects missing %s', (target) => {
const root = fixture();
rmSync(join(root, target), { recursive: true });
expect(() => validateAudioPrebuilds(root)).toThrow();
});
it.each(['empty', 'extra', 'symlink'])('rejects %s binaries', (kind) => {
const root = fixture();
if (kind === 'empty') write(root, 'linux-x64/audio.node', '');
if (kind === 'extra') write(root, 'linux-x64/unexpected.js', 'code');
if (kind === 'symlink') {
rmSync(join(root, 'linux-x64/audio.node'));
symlinkSync(
join(root, 'linux-arm64/audio.node'),
join(root, 'linux-x64/audio.node'),
);
}
expect(() => validateAudioPrebuilds(root)).toThrow();
});
it('rejects symlinked platform directories', () => {
const root = fixture();
rmSync(join(root, 'linux-x64'), { recursive: true });
symlinkSync(join(root, 'linux-arm64'), join(root, 'linux-x64'), 'junction');
expect(() => validateAudioPrebuilds(root)).toThrow();
});
});
describe('audio artifact provenance', () => {
const repo = { owner: 'QwenLM', repo: 'qwen-code' };
const fixture = () => {
const artifact = {
id: 12,
name: 'audio-capture-prebuilds-key',
expired: false,
expires_at: '2100-01-01T00:00:00Z',
workflow_run: {
id: 34,
head_branch: 'main',
repository_id: 1,
head_repository_id: 1,
},
};
const run = {
id: 34,
status: 'completed',
conclusion: 'success',
head_branch: 'main',
path: '.github/workflows/release.yml',
event: 'workflow_dispatch',
repository: { full_name: 'QwenLM/qwen-code' },
head_repository: { full_name: 'QwenLM/qwen-code' },
};
const github = {
rest: {
actions: {
getWorkflowRun: vi.fn(async () => ({ data: run })),
listArtifactsForRepo: vi.fn(async () => ({
data: { artifacts: [artifact] },
})),
},
},
};
return { artifact, run, github };
};
it.each(['workflow_dispatch', 'schedule'])(
'selects an immutable artifact ID from a successful main %s release',
async (event) => {
const f = fixture();
f.run.event = event;
await expect(
findAudioArtifact(f.github, repo, 'key', vi.fn()),
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
},
);
it.each([
['expired', true],
['expires_at', 'invalid'],
['expires_at', new Date(Date.now() - 1000).toISOString()],
['name', 'other'],
])('rejects artifact %s=%s', async (field, value) => {
const f = fixture();
f.artifact[field] = value;
await expect(
verifyAudioArtifact(f.github, repo, f.artifact, 'key'),
).rejects.toThrow();
});
it('requires selection headroom but consumes any unexpired artifact', async () => {
const f = fixture();
const now = Date.parse('2026-10-29T00:00:00Z');
vi.spyOn(Date, 'now').mockReturnValue(now);
expect(AUDIO_SELECTION_MARGIN_MS).toBe(48 * 60 * 60 * 1000);
// Just over the selection margin: chosen by prepare, then consumed later.
f.artifact.expires_at = new Date(
now + AUDIO_SELECTION_MARGIN_MS + 60_000,
).toISOString();
await expect(
verifyAudioArtifact(
f.github,
repo,
f.artifact,
'key',
now,
AUDIO_SELECTION_MARGIN_MS,
),
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
await expect(
findAudioArtifact(f.github, repo, 'key', vi.fn()),
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
// The downstream consumers re-verify after the release has already
// committed to reuse and skipped the fresh build, so they must not
// re-apply the margin: the same artifact still has ~48h left.
await expect(
verifyAudioArtifact(f.github, repo, f.artifact, 'key', now + 120_000),
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
// Too thin to survive the release: never selected in the first place.
f.artifact.expires_at = new Date(now + 60_000).toISOString();
await expect(
verifyAudioArtifact(
f.github,
repo,
f.artifact,
'key',
now,
AUDIO_SELECTION_MARGIN_MS,
),
).rejects.toThrow();
await expect(
findAudioArtifact(f.github, repo, 'key', vi.fn()),
).resolves.toBeUndefined();
});
it.each([
['conclusion', 'failure'],
['status', 'in_progress'],
['head_branch', 'feature'],
['path', '.github/workflows/ci.yml'],
['event', 'pull_request'],
['head_repository', { full_name: 'attacker/fork' }],
])('rejects producer %s', async (field, value) => {
const f = fixture();
f.run[field] = value;
await expect(
verifyAudioArtifact(f.github, repo, f.artifact, 'key'),
).rejects.toThrow();
});
it('rejects a fork artifact before requesting its run', async () => {
const f = fixture();
f.artifact.workflow_run.head_repository_id = 2;
await expect(
verifyAudioArtifact(f.github, repo, f.artifact, 'key'),
).rejects.toThrow();
expect(f.github.rest.actions.getWorkflowRun).not.toHaveBeenCalled();
});
it('falls back on API failure and skips rejected candidates', async () => {
const f = fixture();
const warn = vi.fn();
f.github.rest.actions.listArtifactsForRepo.mockRejectedValueOnce(
new Error('offline'),
);
await expect(
findAudioArtifact(f.github, repo, 'key', warn),
).resolves.toBeUndefined();
f.github.rest.actions.listArtifactsForRepo.mockResolvedValueOnce({
data: {
artifacts: [{ ...f.artifact, id: 13, expired: true }, f.artifact],
},
});
await expect(
findAudioArtifact(f.github, repo, 'key', warn),
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
expect(warn).toHaveBeenCalledTimes(2);
});
});
@@ -264,11 +264,11 @@ describe('audio release slots', () => {
expect(route(audio.jobs.collect, {}, '')).toEqual(['ubuntu-latest']);
});
it('builds the selected release commit without a cache dependency', () => {
it('builds the selected release commit', () => {
expect(release.jobs.audio_capture_prebuilds.uses).toBe(
'./.github/workflows/audio-capture-prebuilds.yml',
);
expect(release.jobs.audio_capture_prebuilds.with).toEqual({
expect(release.jobs.audio_capture_prebuilds.with).toMatchObject({
ref: '${{ needs.prepare.outputs.release_sha }}',
});
expect(
@@ -276,9 +276,6 @@ describe('audio release slots', () => {
s.uses?.startsWith('actions/checkout@'),
).with.ref,
).toBe('${{ inputs.ref || github.sha }}');
expect(JSON.stringify(release)).not.toMatch(
/audio_artifact_id|audio_prebuild_key|audio-prebuilds\.mjs/,
);
for (const job of [audio.jobs.build, audio.jobs.collect]) {
expect(job.steps[0].uses).toBe(
'$/.github/actions/prepare-release-runner',
@@ -288,15 +285,16 @@ describe('audio release slots', () => {
it('keeps newer host glibc out of Linux native binaries', () => {
const steps = audio.jobs.build.steps;
const container = steps.find(
(s) => s.name === 'Build Linux prebuild on ECS',
);
expect(container.if).toBe("${{ runner.environment == 'self-hosted' }}");
expect(container.run).toContain('node:22-bookworm');
const container = steps.find((s) => s.name === 'Build Linux prebuild');
expect(container.if).toBe("${{ startsWith(matrix.os, 'ubuntu') }}");
expect(container.run).toContain('node:22.22.3-bookworm');
expect(container['working-directory']).toBe('.');
expect(container.run).toContain('--frozen-lockfile --ignore-scripts');
expect(container.run).toContain('npm run prebuildify -- --target 22.22.3');
expect(container.run).toContain('--user "$(id -u):$(id -g)"');
for (const name of ['Install build deps', 'Build prebuild']) {
expect(steps.find((s) => s.name === name).if).toBe(
"${{ runner.environment != 'self-hosted' }}",
"${{ !startsWith(matrix.os, 'ubuntu') }}",
);
}
});
@@ -28,6 +28,7 @@ it
'normal',
'workspace-leaf-link',
'workspace-parent-link',
'workspace-contained-parent-link',
'runner-link',
'workspace-outside',
'workspace-equals-runner',
@@ -58,6 +59,11 @@ it
symlinkSync(outside, join(runner, 'parent'), 'dir');
mkdirSync(join(outside, 'repo'));
workspaceInput = join(runner, 'parent', 'repo');
} else if (kind === 'workspace-contained-parent-link') {
mkdirSync(workspace);
writeFileSync(join(workspace, 'sentinel'), 'KEEP_INTERNAL');
symlinkSync(runner, join(runner, 'parent'), 'dir');
workspaceInput = join(runner, 'parent', 'repo');
} else if (kind === 'runner-link') {
mkdirSync(workspace);
symlinkSync(runner, join(root, 'runner-link'), 'dir');
@@ -99,6 +105,11 @@ it
'KEEP_EXTERNAL',
);
assert.equal(statSync(join(outside, 'sentinel')).mode & 0o777, 0o444);
if (kind === 'workspace-contained-parent-link')
assert.equal(
readFileSync(join(workspace, 'sentinel'), 'utf8'),
'KEEP_INTERNAL',
);
if (success) {
assert.ok(lstatSync(workspace).isDirectory());
assert.deepEqual(readdirSync(workspace), []);
+18 -1
View File
@@ -577,7 +577,7 @@ describe('release workflow', () => {
// The tarball gate rides this checkout too: run from the selected
// ref's own `scripts/`, it is missing on any ref that predates it.
'sparse-checkout':
'/.github/scripts\n/scripts/assert-release-version.mjs\n/scripts/check-release-tarball-exec-bits.mjs\n/scripts/release-packages.mjs\n/scripts/workspaces.js\n/package.json\n/packages/*/package.json\n/packages/channels/*/package.json\n/integrations/*/package.json',
'/.github/scripts\n/.github/workflows/audio-capture-prebuilds.yml\n/scripts/assert-release-version.mjs\n/scripts/check-release-tarball-exec-bits.mjs\n/scripts/release-packages.mjs\n/scripts/workspaces.js\n/package.json\n/packages/*/package.json\n/packages/channels/*/package.json\n/integrations/*/package.json',
'sparse-checkout-cone-mode': false,
path: '.release-workflow',
});
@@ -3884,6 +3884,22 @@ describe('release artifact handoff', () => {
expectedBuild: true,
expectedPublish: true,
},
{
name: 'enabled, reused audio',
audio: 'skipped',
artifact: '123',
expectedBuild: true,
expectedPublish: true,
},
{
name: 'unset switch, reused audio',
enabled: '',
audio: 'skipped',
artifact: '123',
build: 'skipped',
expectedBuild: false,
expectedPublish: true,
},
{
name: 'missing audio',
audio: 'skipped',
@@ -3944,6 +3960,7 @@ describe('release artifact handoff', () => {
const needs = {
prepare: {
result: scenario.prepare ?? 'success',
outputs: { audio_artifact_id: scenario.artifact ?? '' },
},
quality: { result: scenario.quality ?? 'success' },
integration_none: { result: 'success' },