mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
perf(release): reuse verified native audio prebuilds across releases (#15854)
* perf(release): reuse verified native audio prebuilds across releases Every release rebuilds the five native audio prebuilds even when nothing that affects those binaries changed. The build is short; the wait is not. For v0.26.3 the Linux x64 prebuild queued 74 minutes to execute for 50 seconds, and the collection step queued again after it — and `publish` depends on that collection, so the whole release sits behind it. Fingerprint the audio build inputs and reuse an artifact from an earlier successful official main release when the fingerprint matches. The key covers the audio package, the dependency lock, the package manager pin and the contents of the producing workflow and this helper, so the Node pin and the prebuildify target arguments are part of it. The package version is excluded deliberately: no part of the native build reads it, and including it would make reuse impossible between releases. Reuse means executing bytes an earlier run produced, so provenance is checked rather than assumed: exact fingerprint name, unexpired, same repository for both run and head run, and a completed successful release.yml run on main from workflow_dispatch or schedule. The pinned download-artifact already verifies the artifact digest. Retention is checked differently at selection and consumption. Selection requires 48 hours of headroom so the artifact outlives the release; consumers require only that it has not expired. Re-applying the margin against a later clock would reject an artifact the release had already committed to, after the native matrix was skipped and with no fallback. Lookup, trial download and payload validation are all non-fatal, and the reuse ID is published only once a trial download has validated; an empty ID runs the full matrix, which is today's behaviour. The publisher does not trust that decision: it recomputes the fingerprint from its own checkout, re-verifies the source run, and validates the payload again, fatally — by then the matrix was skipped, so a bad artifact must stop the release. This does not depend on ECS_RELEASE_HOST_RUNNER_ENABLED and takes effect with the switch unset. The collector needed three fixes to be a trustworthy producer. Its `rm -rf -- "$GITHUB_WORKSPACE/packages/audio-capture/prebuilds"` resolved through whatever `packages` was on a persistent host: with a leftover symlink it deleted an external tree and exited 0, and the following download wrote through the same symlink. It now wipes through a shared preflight that canonicalises the workspace and refuses paths resolving outside it. It also ran `node` with no Node contract, inheriting Node from the hosted image by accident. And its release-only label pinned `x64` while the runners holding that label are ARM64, which never matches and queues until timeout. Contract tests pin the wiring, each confirmed to fail against an injected regression. Design doc in both languages. * fix(release): reconcile audio reuse workflow contracts Share the pre-checkout wipe, preserve the workflow size cap, and let fingerprint failures rebuild native audio. Co-authored-by: Qwen-Coder <[email protected]> * fix(ci): use block-style release permissions Co-authored-by: Qwen-Coder <[email protected]> * fix(release): drop x64 from publish's release-host expression The review's unresolved item: collect got the ARM64 label fix but publish kept ["self-hosted", "linux", "x64", "ecs-qwen-release-host"], which can never match the ARM64 machines holding that label, so the OIDC-carrying job would queue forever once ECS_RELEASE_HOST_RUNNER_ENABLED is true. Same expression as the collect fix, same one-word drop. * docs(release): clarify audio reuse rollout and validation Co-authored-by: Qwen-Coder <[email protected]> * test(release): cover artifact overwrite and contained symlink guards * test(web-shell): include transcript coverage in recovery fixture Bring the fixture correction from #15883 into this branch to address the observed smoke failure. Co-authored-by: Qwen-Coder <[email protected]> * fix(release): make audio cache seeding best effort Keep the validated same-run artifact usable when retaining a keyed copy fails. Pin the reuse wiring and correct the retention and producer fixtures without changing release trust boundaries. Co-authored-by: Qwen-Coder <[email protected]> * chore(test): apply prettier to the audio prebuild fixtures * docs(release): clarify audio reuse run provenance Co-authored-by: Qwen-Coder <[email protected]> --------- Co-authored-by: Qwen-Coder <[email protected]> Co-authored-by: zzj <[email protected]>
This commit is contained in:
11 files changed
+1331
-35
No files matched your search
@@ -0,0 +1,180 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { lstatSync, readFileSync, readdirSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
export const AUDIO_TARGETS = [
|
||||
'darwin-arm64',
|
||||
'darwin-x64',
|
||||
'linux-arm64',
|
||||
'linux-x64',
|
||||
'win32-x64',
|
||||
];
|
||||
|
||||
export function audioPrebuildKey(root, ref, workflowRoot) {
|
||||
const git = (...args) => execFileSync('git', args, { cwd: root });
|
||||
const paths = git(
|
||||
'ls-tree',
|
||||
'-r',
|
||||
'--name-only',
|
||||
'-z',
|
||||
ref,
|
||||
'--',
|
||||
'packages/audio-capture',
|
||||
'pnpm-lock.yaml',
|
||||
'pnpm-workspace.yaml',
|
||||
'.pnpmfile.mjs',
|
||||
'.npmrc',
|
||||
'package.json',
|
||||
)
|
||||
.toString()
|
||||
.split('\0')
|
||||
.filter(Boolean)
|
||||
.sort();
|
||||
// Older release refs have no frozen native dependency graph to reuse.
|
||||
if (!paths.includes('pnpm-lock.yaml')) return '';
|
||||
const hash = createHash('sha256');
|
||||
const add = (name, data) =>
|
||||
hash.update(JSON.stringify([name, Buffer.from(data).toString('base64')]));
|
||||
for (const path of paths) {
|
||||
// Tree entries include the content ID and mode without buffering large headers.
|
||||
let data = git('ls-tree', ref, '--', path);
|
||||
if (path === 'packages/audio-capture/package.json') {
|
||||
const manifest = JSON.parse(git('show', `${ref}:${path}`));
|
||||
delete manifest.version;
|
||||
data = JSON.stringify(manifest);
|
||||
} else if (path === 'package.json') {
|
||||
data = JSON.stringify({
|
||||
packageManager: JSON.parse(git('show', `${ref}:${path}`))
|
||||
.packageManager,
|
||||
});
|
||||
}
|
||||
add(path, data);
|
||||
}
|
||||
for (const path of [
|
||||
'.github/workflows/audio-capture-prebuilds.yml',
|
||||
'.github/scripts/audio-prebuilds.mjs',
|
||||
])
|
||||
add(path, readFileSync(join(workflowRoot, path)));
|
||||
return hash.digest('hex');
|
||||
}
|
||||
|
||||
export function validateAudioPrebuilds(directory) {
|
||||
if (!lstatSync(directory).isDirectory())
|
||||
throw new Error('Invalid prebuild directory');
|
||||
const targets = readdirSync(directory).sort();
|
||||
if (JSON.stringify(targets) !== JSON.stringify([...AUDIO_TARGETS].sort())) {
|
||||
throw new Error(
|
||||
'Audio prebuilds must contain exactly the five release targets',
|
||||
);
|
||||
}
|
||||
for (const target of targets) {
|
||||
const targetPath = join(directory, target);
|
||||
if (!lstatSync(targetPath).isDirectory())
|
||||
throw new Error(`Invalid target: ${target}`);
|
||||
const files = readdirSync(targetPath);
|
||||
if (!files.length) throw new Error(`Missing native binary: ${target}`);
|
||||
for (const file of files) {
|
||||
const stat = lstatSync(join(targetPath, file));
|
||||
if (!file.endsWith('.node') || !stat.isFile() || stat.size === 0) {
|
||||
throw new Error(`Invalid native binary: ${target}/${file}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Selection demands enough retention to outlive the rest of the release;
|
||||
// consumption only demands that the bytes are still there. Re-applying the
|
||||
// margin downstream would reject an artifact that prepare already committed
|
||||
// to, after the fresh-build job was skipped and with no fallback left.
|
||||
export const AUDIO_SELECTION_MARGIN_MS = 48 * 60 * 60 * 1000;
|
||||
|
||||
export async function verifyAudioArtifact(
|
||||
github,
|
||||
repo,
|
||||
artifact,
|
||||
key,
|
||||
now = Date.now(),
|
||||
marginMs = 0,
|
||||
) {
|
||||
if (
|
||||
!key ||
|
||||
artifact.name !== `audio-capture-prebuilds-${key}` ||
|
||||
artifact.expired ||
|
||||
!(Date.parse(artifact.expires_at) >= now + marginMs) ||
|
||||
artifact.workflow_run?.head_branch !== 'main' ||
|
||||
artifact.workflow_run.repository_id !==
|
||||
artifact.workflow_run.head_repository_id
|
||||
) {
|
||||
throw new Error(
|
||||
'Audio artifact has an invalid identity, source, or retention',
|
||||
);
|
||||
}
|
||||
const { data: run } = await github.rest.actions.getWorkflowRun({
|
||||
...repo,
|
||||
run_id: artifact.workflow_run.id,
|
||||
});
|
||||
const repository = `${repo.owner}/${repo.repo}`;
|
||||
if (
|
||||
run.status !== 'completed' ||
|
||||
run.conclusion !== 'success' ||
|
||||
run.head_branch !== 'main' ||
|
||||
run.path !== '.github/workflows/release.yml' ||
|
||||
!['workflow_dispatch', 'schedule'].includes(run.event) ||
|
||||
run.repository?.full_name !== repository ||
|
||||
run.head_repository?.full_name !== repository
|
||||
) {
|
||||
throw new Error(
|
||||
'Audio artifact must come from a successful official main release',
|
||||
);
|
||||
}
|
||||
return { artifact_id: artifact.id, run_id: run.id };
|
||||
}
|
||||
|
||||
export async function findAudioArtifact(github, repo, key, warn) {
|
||||
if (!key) return undefined;
|
||||
try {
|
||||
const { data } = await github.rest.actions.listArtifactsForRepo({
|
||||
...repo,
|
||||
name: `audio-capture-prebuilds-${key}`,
|
||||
per_page: 100,
|
||||
});
|
||||
for (const artifact of data.artifacts) {
|
||||
try {
|
||||
return await verifyAudioArtifact(
|
||||
github,
|
||||
repo,
|
||||
artifact,
|
||||
key,
|
||||
Date.now(),
|
||||
AUDIO_SELECTION_MARGIN_MS,
|
||||
);
|
||||
} catch (error) {
|
||||
warn(`Skipping audio artifact ${artifact.id}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
warn(`Audio reuse lookup failed; rebuilding: ${error.message}`);
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
||||
const [command, ...args] = process.argv.slice(2);
|
||||
if (command === 'key') {
|
||||
console.log(
|
||||
`cache_key=${audioPrebuildKey(process.cwd(), args[0], args[1])}`,
|
||||
);
|
||||
} else if (command === 'validate') {
|
||||
validateAudioPrebuilds(args[0]);
|
||||
} else {
|
||||
throw new Error(`Unknown audio prebuild command: ${command}`);
|
||||
}
|
||||
}
|
||||
@@ -14,7 +14,7 @@
|
||||
# real, bump the number and say why in the PR.
|
||||
2226 assign-issue-owner.yml
|
||||
3555 assign-pr-owner.yml
|
||||
6457 audio-capture-prebuilds.yml
|
||||
7107 audio-capture-prebuilds.yml
|
||||
9023 auto-minimize-spam.yml
|
||||
9551 build-and-publish-image.yml
|
||||
55276 cd-cua-driver.yml
|
||||
@@ -55,7 +55,7 @@
|
||||
22345 release-sdk-python.yml
|
||||
31270 release-sdk.yml
|
||||
17941 release-vscode-companion.yml
|
||||
45269 release.yml
|
||||
45270 release.yml
|
||||
44285 repo-hygiene.yml
|
||||
1079 scorecard-monthly.yml
|
||||
9751 sdk-java-contract-open-prs.yml
|
||||
|
||||
@@ -20,6 +20,10 @@ on:
|
||||
inputs:
|
||||
ref:
|
||||
type: 'string'
|
||||
required: true
|
||||
cache_key:
|
||||
type: 'string'
|
||||
required: false
|
||||
default: ''
|
||||
workflow_dispatch:
|
||||
|
||||
@@ -60,36 +64,56 @@ jobs:
|
||||
ref: '${{ inputs.ref || github.sha }}'
|
||||
persist-credentials: false
|
||||
- uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
|
||||
if: "${{ runner.environment != 'self-hosted' }}"
|
||||
if: "${{ !startsWith(matrix.os, 'ubuntu') }}"
|
||||
with:
|
||||
node-version: '22'
|
||||
# Install only this package's deps (skip the node-gyp-build install hook;
|
||||
# prebuildify does its own compile below).
|
||||
node-version: '22.22.3'
|
||||
# Use the lock that participates in the reuse fingerprint. Older refs
|
||||
# without pnpm retain the original build path and cannot seed reuse.
|
||||
- name: 'Install build deps'
|
||||
if: "${{ runner.environment != 'self-hosted' }}"
|
||||
run: 'npm install --no-workspaces --ignore-scripts --no-audit --no-fund'
|
||||
if: "${{ !startsWith(matrix.os, 'ubuntu') }}"
|
||||
working-directory: '.'
|
||||
shell: 'bash'
|
||||
run: |-
|
||||
if [ -f pnpm-lock.yaml ]; then
|
||||
corepack pnpm --filter @qwen-code/audio-capture install --frozen-lockfile --ignore-scripts --reporter=append-only
|
||||
else
|
||||
cd packages/audio-capture
|
||||
npm install --no-workspaces --ignore-scripts --no-audit --no-fund
|
||||
fi
|
||||
- name: 'Build prebuild'
|
||||
if: "${{ runner.environment != 'self-hosted' }}"
|
||||
if: "${{ !startsWith(matrix.os, 'ubuntu') }}"
|
||||
shell: 'bash'
|
||||
run: |
|
||||
npm run prebuildify
|
||||
npm run prebuildify -- --target 22.22.3
|
||||
# Cross-compile the Intel (x64) slice on this arm64 runner instead of
|
||||
# a separate macos-13 runner (frameworks are universal). See #5642.
|
||||
if [ "$RUNNER_OS" = 'macOS' ]; then
|
||||
npm run prebuildify -- --arch x64
|
||||
npm run prebuildify -- --target 22.22.3 --arch x64
|
||||
fi
|
||||
# Keep Linux binaries compatible with older glibc even on newer ECS hosts.
|
||||
- name: 'Build Linux prebuild on ECS'
|
||||
if: "${{ runner.environment == 'self-hosted' }}"
|
||||
# Use the same Linux toolchain on hosted and ECS runners for cache identity.
|
||||
- name: 'Build Linux prebuild'
|
||||
if: "${{ startsWith(matrix.os, 'ubuntu') }}"
|
||||
working-directory: '.'
|
||||
shell: 'bash'
|
||||
run: |-
|
||||
docker run --rm --user "$(id -u):$(id -g)" \
|
||||
-e HOME=/tmp \
|
||||
-v "$PWD:/build" -w /build node:22-bookworm \
|
||||
bash -euc 'npm install --no-workspaces --ignore-scripts --no-audit --no-fund; npm run prebuildify'
|
||||
-e HOME=/tmp -e COREPACK_HOME=/tmp/corepack \
|
||||
-v "$PWD:/build" -w /build node:22.22.3-bookworm \
|
||||
bash -euc '
|
||||
if [ -f pnpm-lock.yaml ]; then
|
||||
corepack pnpm --filter @qwen-code/audio-capture install --frozen-lockfile --ignore-scripts --reporter=append-only
|
||||
else
|
||||
cd packages/audio-capture
|
||||
npm install --no-workspaces --ignore-scripts --no-audit --no-fund
|
||||
cd ../..
|
||||
fi
|
||||
cd packages/audio-capture
|
||||
npm run prebuildify -- --target 22.22.3
|
||||
'
|
||||
- name: 'Upload prebuild'
|
||||
uses: 'actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a' # v7.0.1
|
||||
with:
|
||||
overwrite: true
|
||||
name: 'prebuilds-${{ matrix.os }}-${{ matrix.artifact_suffix || matrix.arch }}'
|
||||
path: 'packages/audio-capture/prebuilds/'
|
||||
if-no-files-found: 'error'
|
||||
@@ -107,11 +131,41 @@ jobs:
|
||||
pattern: 'prebuilds-*'
|
||||
merge-multiple: true
|
||||
path: 'packages/audio-capture/prebuilds'
|
||||
- name: 'List collected prebuilds'
|
||||
run: 'find prebuilds -type f'
|
||||
- name: 'Checkout audio workflow helper'
|
||||
uses: 'actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10' # v6.0.3
|
||||
with:
|
||||
ref: '${{ github.workflow_sha }}'
|
||||
persist-credentials: false
|
||||
sparse-checkout: |-
|
||||
.github/scripts/audio-prebuilds.mjs
|
||||
.github/actions/self-hosted-node
|
||||
sparse-checkout-cone-mode: false
|
||||
path: '.audio-workflow'
|
||||
- name: 'Use pre-installed Node.js (self-hosted)'
|
||||
if: "${{ runner.environment == 'self-hosted' }}"
|
||||
uses: './.audio-workflow/.github/actions/self-hosted-node'
|
||||
- name: 'Setup Node.js'
|
||||
if: "${{ runner.environment != 'self-hosted' }}"
|
||||
uses: 'actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e' # v6.4.0
|
||||
with:
|
||||
node-version: '22.22.3'
|
||||
- name: 'Validate complete audio prebuilds'
|
||||
working-directory: '.'
|
||||
run: 'node .audio-workflow/.github/scripts/audio-prebuilds.mjs validate packages/audio-capture/prebuilds'
|
||||
- name: 'Upload combined prebuilds'
|
||||
uses: 'actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a' # v7.0.1
|
||||
with:
|
||||
name: 'audio-capture-prebuilds'
|
||||
overwrite: true
|
||||
path: 'packages/audio-capture/prebuilds/'
|
||||
if-no-files-found: 'error'
|
||||
|
||||
- name: 'Retain reusable audio prebuilds'
|
||||
if: "${{ inputs.cache_key != '' }}"
|
||||
continue-on-error: true
|
||||
uses: 'actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a' # v7.0.1
|
||||
with:
|
||||
name: 'audio-capture-prebuilds-${{ inputs.cache_key }}'
|
||||
path: 'packages/audio-capture/prebuilds/'
|
||||
if-no-files-found: 'error'
|
||||
overwrite: true
|
||||
@@ -54,8 +54,12 @@ jobs:
|
||||
${{ github.repository == 'QwenLM/qwen-code' }}
|
||||
permissions:
|
||||
contents: 'read'
|
||||
actions: 'read'
|
||||
outputs:
|
||||
release_sha: '${{ steps.source.outputs.release_sha }}'
|
||||
audio_prebuild_key: '${{ steps.audio_key.outputs.cache_key }}'
|
||||
audio_artifact_id: '${{ steps.audio_reuse.outputs.artifact_id }}'
|
||||
audio_run_id: '${{ steps.audio_reuse.outputs.run_id }}'
|
||||
release_tag: '${{ steps.version.outputs.RELEASE_TAG }}'
|
||||
release_version: '${{ steps.version.outputs.RELEASE_VERSION }}'
|
||||
npm_tag: '${{ steps.version.outputs.NPM_TAG }}'
|
||||
@@ -85,6 +89,7 @@ jobs:
|
||||
persist-credentials: false
|
||||
sparse-checkout: |-
|
||||
/.github/scripts
|
||||
/.github/workflows/audio-capture-prebuilds.yml
|
||||
/scripts/assert-release-version.mjs
|
||||
/scripts/check-release-tarball-exec-bits.mjs
|
||||
/scripts/release-packages.mjs
|
||||
@@ -146,6 +151,59 @@ jobs:
|
||||
|
||||
- *checkout_release_workflow_scripts
|
||||
|
||||
- name: 'Fingerprint audio build inputs'
|
||||
id: 'audio_key'
|
||||
continue-on-error: true
|
||||
env:
|
||||
RELEASE_SHA: '${{ steps.source.outputs.release_sha }}'
|
||||
run: |-
|
||||
directory=$(mktemp -d "${RUNNER_TEMP}/audio-reuse.XXXXXX")
|
||||
echo "directory=$directory" >> "$GITHUB_OUTPUT"
|
||||
node .release-workflow/.github/scripts/audio-prebuilds.mjs key "$RELEASE_SHA" .release-workflow >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: 'Find reusable audio prebuilds'
|
||||
id: 'audio_candidate'
|
||||
continue-on-error: true
|
||||
uses: 'actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3' # v9.0.0
|
||||
env:
|
||||
AUDIO_KEY: '${{ steps.audio_key.outputs.cache_key }}'
|
||||
with:
|
||||
script: |-
|
||||
const { findAudioArtifact } = await import(process.env.GITHUB_WORKSPACE + '/.release-workflow/.github/scripts/audio-prebuilds.mjs');
|
||||
const artifact = await findAudioArtifact(github, context.repo, process.env.AUDIO_KEY, core.warning);
|
||||
for (const [key, value] of Object.entries(artifact ?? {})) core.setOutput(key, value);
|
||||
|
||||
- name: 'Try reusable audio prebuilds'
|
||||
id: 'audio_download'
|
||||
if: "${{ steps.audio_candidate.outputs.artifact_id != '' }}"
|
||||
continue-on-error: true
|
||||
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
|
||||
with:
|
||||
artifact-ids: '${{ steps.audio_candidate.outputs.artifact_id }}'
|
||||
run-id: '${{ steps.audio_candidate.outputs.run_id }}'
|
||||
github-token: '${{ github.token }}'
|
||||
merge-multiple: true
|
||||
path: '${{ steps.audio_key.outputs.directory }}'
|
||||
|
||||
- name: 'Accept complete audio prebuilds'
|
||||
id: 'audio_reuse'
|
||||
if: "${{ steps.audio_download.outcome == 'success' }}"
|
||||
continue-on-error: true
|
||||
env:
|
||||
AUDIO_DIRECTORY: '${{ steps.audio_key.outputs.directory }}'
|
||||
AUDIO_ARTIFACT_ID: '${{ steps.audio_candidate.outputs.artifact_id }}'
|
||||
AUDIO_RUN_ID: '${{ steps.audio_candidate.outputs.run_id }}'
|
||||
run: |-
|
||||
node .release-workflow/.github/scripts/audio-prebuilds.mjs validate "$AUDIO_DIRECTORY"
|
||||
echo "artifact_id=$AUDIO_ARTIFACT_ID" >> "$GITHUB_OUTPUT"
|
||||
echo "run_id=$AUDIO_RUN_ID" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: 'Remove trial audio download'
|
||||
if: "${{ always() && steps.audio_key.outputs.directory != '' }}"
|
||||
env:
|
||||
AUDIO_DIRECTORY: '${{ steps.audio_key.outputs.directory }}'
|
||||
run: 'rm -rf -- "$AUDIO_DIRECTORY"'
|
||||
|
||||
- name: 'Get the version'
|
||||
id: 'version'
|
||||
env:
|
||||
@@ -518,10 +576,11 @@ jobs:
|
||||
name: 'Audio Capture Prebuilds'
|
||||
needs: 'prepare'
|
||||
if: |-
|
||||
${{ github.repository == 'QwenLM/qwen-code' }}
|
||||
${{ github.repository == 'QwenLM/qwen-code' && needs.prepare.outputs.audio_artifact_id == '' }}
|
||||
uses: './.github/workflows/audio-capture-prebuilds.yml'
|
||||
with:
|
||||
ref: '${{ needs.prepare.outputs.release_sha }}'
|
||||
cache_key: '${{ needs.prepare.outputs.audio_prebuild_key }}'
|
||||
|
||||
build_release:
|
||||
name: 'Build Release Artifacts'
|
||||
@@ -542,7 +601,10 @@ jobs:
|
||||
github.repository == 'QwenLM/qwen-code' &&
|
||||
vars.ECS_RELEASE_HOST_RUNNER_ENABLED == 'true' &&
|
||||
needs.prepare.result == 'success' &&
|
||||
needs.audio_capture_prebuilds.result == 'success' &&
|
||||
(
|
||||
needs.audio_capture_prebuilds.result == 'success' ||
|
||||
(needs.audio_capture_prebuilds.result == 'skipped' && needs.prepare.outputs.audio_artifact_id != '')
|
||||
) &&
|
||||
(
|
||||
github.event.inputs.force_skip_tests == 'true' ||
|
||||
(
|
||||
@@ -591,15 +653,45 @@ jobs:
|
||||
|
||||
- *reset_release_workflow_scripts
|
||||
- *checkout_release_workflow_scripts
|
||||
- &verify_reused_audio
|
||||
name: 'Verify reusable audio source'
|
||||
if: "${{ needs.prepare.outputs.audio_artifact_id != '' }}"
|
||||
uses: 'actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3' # v9.0.0
|
||||
env:
|
||||
RELEASE_SHA: '${{ needs.prepare.outputs.release_sha }}'
|
||||
AUDIO_ARTIFACT_ID: '${{ needs.prepare.outputs.audio_artifact_id }}'
|
||||
with:
|
||||
script: |-
|
||||
const { audioPrebuildKey, verifyAudioArtifact } = await import(process.env.GITHUB_WORKSPACE + '/.release-workflow/.github/scripts/audio-prebuilds.mjs');
|
||||
const key = audioPrebuildKey(process.env.GITHUB_WORKSPACE, process.env.RELEASE_SHA, process.env.GITHUB_WORKSPACE + '/.release-workflow');
|
||||
const { data: artifact } = await github.rest.actions.getArtifact({ ...context.repo, artifact_id: Number(process.env.AUDIO_ARTIFACT_ID) });
|
||||
await verifyAudioArtifact(github, context.repo, artifact, key);
|
||||
|
||||
- &download_reused_audio
|
||||
name: 'Download reusable audio prebuilds'
|
||||
if: "${{ needs.prepare.outputs.audio_artifact_id != '' }}"
|
||||
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
|
||||
with:
|
||||
artifact-ids: '${{ needs.prepare.outputs.audio_artifact_id }}'
|
||||
run-id: '${{ needs.prepare.outputs.audio_run_id }}'
|
||||
github-token: '${{ github.token }}'
|
||||
merge-multiple: true
|
||||
path: 'packages/audio-capture/prebuilds'
|
||||
|
||||
- &download_fresh_audio
|
||||
name: 'Download audio capture prebuilds'
|
||||
if: |-
|
||||
${{ github.repository == 'QwenLM/qwen-code' }}
|
||||
${{ github.repository == 'QwenLM/qwen-code' && needs.prepare.outputs.audio_artifact_id == '' }}
|
||||
uses: 'actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c' # v8.0.1
|
||||
with:
|
||||
name: 'audio-capture-prebuilds'
|
||||
path: 'packages/audio-capture/prebuilds'
|
||||
|
||||
- &validate_audio
|
||||
name: 'Validate complete audio prebuilds'
|
||||
if: "${{ github.repository == 'QwenLM/qwen-code' }}"
|
||||
run: 'node .release-workflow/.github/scripts/audio-prebuilds.mjs validate packages/audio-capture/prebuilds'
|
||||
|
||||
- *reset_release_workflow_scripts
|
||||
- *checkout_release_workflow_scripts
|
||||
- name: 'Build Bundle and Prepare Package'
|
||||
@@ -660,7 +752,8 @@ jobs:
|
||||
) &&
|
||||
(
|
||||
github.repository != 'QwenLM/qwen-code' ||
|
||||
needs.audio_capture_prebuilds.result == 'success'
|
||||
needs.audio_capture_prebuilds.result == 'success' ||
|
||||
(needs.audio_capture_prebuilds.result == 'skipped' && needs.prepare.outputs.audio_artifact_id != '')
|
||||
) &&
|
||||
(
|
||||
github.event.inputs.force_skip_tests == 'true' ||
|
||||
@@ -759,8 +852,12 @@ jobs:
|
||||
|
||||
- *reset_release_workflow_scripts
|
||||
- *checkout_release_workflow_scripts
|
||||
- *verify_reused_audio
|
||||
- *download_reused_audio
|
||||
- *download_fresh_audio
|
||||
|
||||
- *validate_audio
|
||||
|
||||
- *reset_release_workflow_scripts
|
||||
- *checkout_release_workflow_scripts
|
||||
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
# Audio prebuild reuse
|
||||
|
||||
[简体中文](./2026-10-29-audio-prebuild-reuse.zh-CN.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Every release rebuilds the native audio prebuilds for all five targets, even
|
||||
when nothing that affects those binaries changed. The build itself is short;
|
||||
the wait is not. For v0.26.3 the Linux x64 prebuild waited 74 minutes for a
|
||||
hosted runner to execute for 50 seconds, and the collection step queued again
|
||||
after it. Because `publish` depends on that collection, the whole release sits
|
||||
behind it.
|
||||
|
||||
Audio prebuilds are a good reuse candidate precisely because they are
|
||||
expensive to schedule and cheap to verify: five native `.node` files whose
|
||||
inputs are fully described by the repository at a known commit.
|
||||
|
||||
## Design
|
||||
|
||||
A release fingerprints its audio build inputs, looks for an artifact from an
|
||||
earlier successful official `main` release with the same fingerprint, and
|
||||
reuses it instead of rebuilding.
|
||||
|
||||
### What the fingerprint covers
|
||||
|
||||
The content identity (git blob OID) of everything that can change the built
|
||||
binaries: `packages/audio-capture`, `pnpm-lock.yaml`, `pnpm-workspace.yaml`,
|
||||
`.pnpmfile.mjs`, `.npmrc`, the root `package.json`'s `packageManager` pin, plus
|
||||
the contents of the producing workflow and this helper. Hashing the workflow is
|
||||
what makes the Node pin and the `prebuildify --target` arguments part of the
|
||||
key.
|
||||
|
||||
Two deliberate exclusions: `packages/audio-capture/package.json`'s `version`
|
||||
field, because a version bump is packaging metadata that no part of the native
|
||||
build reads, and everything in the root `package.json` except `packageManager`.
|
||||
Without the first exclusion no release could ever reuse another's prebuilds.
|
||||
|
||||
A ref with no `pnpm-lock.yaml` produces an empty key and never reuses — older
|
||||
refs have no frozen native dependency graph to match against.
|
||||
|
||||
### What makes an artifact trustworthy
|
||||
|
||||
Reuse means executing bytes built by an earlier run, so provenance is checked
|
||||
rather than assumed. A candidate must be named for the exact fingerprint, be
|
||||
unexpired, come from a run whose repository and head repository are both this
|
||||
repository, and come from a _completed, successful_ run of `release.yml` on
|
||||
`main` triggered by `workflow_dispatch` or `schedule`. That excludes fork PRs,
|
||||
`workflow_run` producers, and direct dispatches of the audio workflow.
|
||||
|
||||
Transport integrity is already covered: the pinned `download-artifact` compares
|
||||
the artifact digest and fails on mismatch.
|
||||
|
||||
### Retention: selection vs consumption
|
||||
|
||||
Selection requires at least 48 hours of remaining retention, so a chosen
|
||||
artifact outlives the rest of the release. The downstream consumer requires
|
||||
only that the artifact has not expired.
|
||||
|
||||
These must differ. Re-applying the selection margin against a later clock
|
||||
rejects an artifact the release has already committed to: an artifact with
|
||||
48h1m remaining passes in `prepare` and then fails two minutes later with
|
||||
47h59m still available — after `prepare` skipped the native matrix, leaving no
|
||||
fallback.
|
||||
|
||||
### Failure handling
|
||||
|
||||
Fingerprinting, lookup, trial download and validation are non-fatal, and the
|
||||
reuse ID is only published after a _trial download and payload validation_
|
||||
succeed:
|
||||
|
||||
- lookup API failure → no candidate
|
||||
- fingerprint or temporary-directory failure → no key, so the native matrix runs
|
||||
- download failure → acceptance skipped (keyed on `outcome`, since
|
||||
`continue-on-error` rewrites `conclusion`)
|
||||
- payload validation failure → the step fails before writing the output
|
||||
- reusable upload failure → the current release proceeds; future reuse may miss
|
||||
|
||||
An empty reuse ID runs the full native matrix. Fresh builds now pin Node and
|
||||
the prebuildify target to 22.22.3 and use filtered, frozen pnpm dependencies
|
||||
when a lockfile exists; only older refs retain the npm install path. Linux on hosted and ECS runners
|
||||
builds run that same frozen install inside `node:22.22.3-bookworm`, preserving
|
||||
the existing glibc baseline. The trial bytes are discarded and re-downloaded
|
||||
by the consumer.
|
||||
|
||||
Both the ECS build job and the hosted publisher independently recompute the
|
||||
fingerprint from their own checkout, re-verify the source run, download by
|
||||
artifact ID, and validate the payload again. That verification is fatal — by
|
||||
then the native matrix was skipped, so a bad artifact must stop the release
|
||||
rather than ship. A download failure or artifact deletion at this stage also
|
||||
stops the release; it does not restart the skipped matrix.
|
||||
|
||||
Validation checks exactly the five target directories, each containing only
|
||||
nonempty regular `.node` files and no symlinks. It is a shape check, not a
|
||||
platform check; the provenance and digest checks above carry the trust.
|
||||
|
||||
### Collector hygiene
|
||||
|
||||
The collector merges per-platform artifacts on a possibly persistent workspace,
|
||||
where `download-artifact`'s `merge-multiple` merges into existing paths, so a
|
||||
stale `prebuilds/` tree would leak into the artifact it uploads.
|
||||
|
||||
It uses the shared release-runner preflight already on main, which canonicalises
|
||||
and wipes the workspace and refuses paths resolving through a symlinked
|
||||
component or outside the runner workspace. This change preserves that action
|
||||
and its cleanup implementation.
|
||||
|
||||
The action fetches the cleanup and disk-floor scripts at the workflow SHA into
|
||||
the runner's temporary directory before checkout. It isolates Git/npm/Docker/gh
|
||||
state, provides a job-local npm prefix, asserts Node 22 and checks disk space.
|
||||
The cleanup script itself does not depend on Node.
|
||||
|
||||
The collector uses the existing pre-installed Node preflight on self-hosted
|
||||
runners and pins hosted `setup-node` to 22.22.3 before the validator. Native
|
||||
compilation always uses Node 22.22.3. The collector remains architecture-free;
|
||||
the Linux build legs retain their matching x64 or arm64 release-runner labels.
|
||||
|
||||
### Relationship to the release host switch
|
||||
|
||||
Audio reuse works with `ECS_RELEASE_HOST_RUNNER_ENABLED` either enabled or unset.
|
||||
The decision happens in `prepare`. With the switch unset, the existing hosted
|
||||
publisher builds and consumes the audio payload. When enabled, the ECS producer
|
||||
consumes it before handing release outputs to the hosted publisher, which also
|
||||
verifies the audio source and payload. npm publication remains hosted in both
|
||||
cases, preserving the split already merged in #15832.
|
||||
|
||||
This change neither alters runner routing nor enables the switch. A cache hit
|
||||
skips the native matrix; a miss follows the existing hosted/ECS routing.
|
||||
|
||||
## Validation
|
||||
|
||||
Fingerprint, provenance, retention and payload behaviour are covered by unit
|
||||
tests with direct witnesses, including the retention boundary that motivated
|
||||
the selection/consumption split. The workflow wiring — non-fatal lookup,
|
||||
`outcome`-keyed acceptance, matrix skip, publisher re-verification, collector
|
||||
ordering and the architecture-free label — is pinned by contract tests.
|
||||
The latest review adds regression-sensitive wiring assertions and fixes the
|
||||
retention test's clock; these additions await CI and were not run locally.
|
||||
|
||||
A branch release dry-run can verify the miss path, all five native payloads and
|
||||
publisher consumption without publishing. Reuse provenance is checked at the
|
||||
run level: repository, `head_branch`, event, workflow path and successful
|
||||
conclusion. `head_branch` identifies the dispatch branch, not the built
|
||||
`inputs.ref`. A run dispatched on a feature branch cannot seed an eligible
|
||||
candidate; one dispatched on `main` with `inputs.ref` pointing at a feature
|
||||
branch can. The fingerprint pins the audio build inputs, not the source
|
||||
commit's membership in `main`. A successful eligible run must first retain a
|
||||
keyed artifact; a later run with the same key can exercise reuse.
|
||||
Record the switch state and actual runners used by each dry run. A hosted run
|
||||
does not verify ECS execution, and a miss does not verify reuse.
|
||||
|
||||
## Affected files
|
||||
|
||||
`.github/scripts/audio-prebuilds.mjs`, the audio and release workflows, the
|
||||
shared release-runner preflight it reuses, workflow tests under `scripts/tests`,
|
||||
and the workflow size baseline.
|
||||
|
||||
## Open questions
|
||||
|
||||
Runner image updates under unchanged labels do not change the fingerprint.
|
||||
N-API stabilises the Node ABI, not glibc or toolchain linkage, so this reuses a
|
||||
verified binary rather than guaranteeing that a fresh build would be
|
||||
bit-for-bit identical. Linux on hosted and ECS runners uses a versioned Bookworm container tag, not
|
||||
an immutable image digest; macOS and Windows hosted images also continue to evolve. Pinning those
|
||||
environments completely is outside this change.
|
||||
@@ -0,0 +1,140 @@
|
||||
# 音频预构建复用
|
||||
|
||||
[English](./2026-10-29-audio-prebuild-reuse.md)
|
||||
|
||||
## 问题
|
||||
|
||||
每次发布都会为五个目标重新构建原生音频预构建产物,即使影响这些二进制的输入
|
||||
没有任何变化。构建本身很短,等待不短。v0.26.3 的 Linux x64 预构建排队 74
|
||||
分钟,执行只用 50 秒,随后汇总步骤又排了一次。由于 `publish` 依赖该汇总,
|
||||
整个发布都卡在后面。
|
||||
|
||||
音频预构建适合复用,恰恰因为它调度代价高而校验代价低:五个原生 `.node`
|
||||
文件,其输入可以由某个已知 commit 上的仓库内容完整描述。
|
||||
|
||||
## 设计
|
||||
|
||||
发布先对音频构建输入取指纹,再查找此前一次成功的官方 `main` 发布产出的、
|
||||
指纹相同的产物,命中则复用而不是重新构建。
|
||||
|
||||
### 指纹覆盖什么
|
||||
|
||||
覆盖所有可能改变构建结果的内容身份(git blob OID):
|
||||
`packages/audio-capture`、`pnpm-lock.yaml`、`pnpm-workspace.yaml`、
|
||||
`.pnpmfile.mjs`、`.npmrc`、根 `package.json` 的 `packageManager` pin,以及
|
||||
生产该产物的 workflow 和这个辅助脚本本身的内容。把 workflow 纳入哈希,正是
|
||||
Node 版本 pin 和 `prebuildify --target` 参数进入 key 的方式。
|
||||
|
||||
两处有意排除:`packages/audio-capture/package.json` 的 `version` 字段——版本
|
||||
号是打包元数据,原生构建的任何环节都不读它;以及根 `package.json` 中除
|
||||
`packageManager` 以外的全部内容。没有第一条排除,任何发布都无法复用另一次
|
||||
发布的产物。
|
||||
|
||||
没有 `pnpm-lock.yaml` 的 ref 产出空 key,永不复用——较老的 ref 没有可比对的
|
||||
原生依赖冻结图。
|
||||
|
||||
### 什么样的产物才可信
|
||||
|
||||
复用意味着执行由此前运行构建出的字节,所以来源要校验而不是假定。候选产物
|
||||
必须以精确指纹命名、未过期、所属运行的 repository 与 head repository 都是
|
||||
本仓库,且来自 `main` 上由 `workflow_dispatch` 或 `schedule` 触发的、
|
||||
**已完成且成功**的 `release.yml` 运行。这排除了 fork PR、`workflow_run`
|
||||
生产者,以及直接 dispatch 音频 workflow 的情形。
|
||||
|
||||
传输完整性已有保障:pin 住的 `download-artifact` 会比对产物摘要,不一致即
|
||||
失败。
|
||||
|
||||
### 保留期:选择 vs 消费
|
||||
|
||||
选择时要求至少剩余 48 小时保留期,使选中的产物能活过发布的其余阶段。下游
|
||||
消费方只要求产物尚未过期。
|
||||
|
||||
两者必须不同。按更晚的时钟重复套用选择期余量,会拒绝一个发布已经选定的
|
||||
产物:剩 48 小时 1 分钟的产物在 `prepare` 通过,两分钟后失败——而此时还剩
|
||||
47 小时 59 分钟,且 `prepare` 已经跳过原生构建矩阵,没有任何回退。
|
||||
|
||||
### 失败处理
|
||||
|
||||
指纹计算、查询、试下载与校验都是非致命的,而复用 ID 只在**试下载与载荷
|
||||
校验**都成功后才发布:
|
||||
|
||||
- 查询 API 失败 → 没有候选
|
||||
- 指纹计算或临时目录创建失败 → 没有 key,执行原生构建矩阵
|
||||
- 下载失败 → 跳过接受步骤(以 `outcome` 为判据,因为 `continue-on-error`
|
||||
会改写 `conclusion`)
|
||||
- 载荷校验失败 → 该步骤在写出 output 之前就失败
|
||||
- 复用产物上传失败 → 当前发布继续,后续发布可能无法复用
|
||||
|
||||
空的复用 ID 会执行完整的原生构建矩阵。重新构建现在把 Node 和 prebuildify
|
||||
目标固定为 22.22.3,存在锁文件时使用按包筛选、冻结的 pnpm 依赖;只有旧 ref
|
||||
保留 npm install 路径。hosted 与 ECS 上的 Linux 在 `node:22.22.3-bookworm` 容器中执行
|
||||
同样的冻结安装,保留已有的 glibc 基线。试下载的字节会被丢弃、由消费方
|
||||
重新下载。
|
||||
|
||||
ECS 构建任务与 hosted 发布任务分别使用自己的 checkout 重算指纹、重新校验
|
||||
来源运行、按 artifact ID 下载并再次校验载荷。这一校验是致命的——此时原生构建
|
||||
矩阵已被跳过,所以坏产物必须让发布停止,而不是发出去。此阶段下载失败或
|
||||
产物被删除也会停止发布,不会重新启动已跳过的构建矩阵。
|
||||
|
||||
校验检查恰好五个目标目录,每个目录只包含非空的普通 `.node` 文件且无符号
|
||||
链接。这是形状校验而不是平台校验;信任由上述来源校验和摘要校验承载。
|
||||
|
||||
### 汇总任务的卫生
|
||||
|
||||
汇总任务在可能持久的工作区上合并各平台产物,而 `download-artifact` 的
|
||||
`merge-multiple` 会合并进已存在的路径,因此残留的 `prebuilds/` 目录会泄漏
|
||||
进它上传的产物。
|
||||
|
||||
汇总任务使用 main 已有的共享 release runner 预检:规范化并清空工作区,
|
||||
拒绝经由符号链接组件解析、或落在 runner 工作区之外的路径。本次保留该
|
||||
共享 action 及其清理实现。
|
||||
|
||||
action 在 checkout 前按 workflow SHA 把清理与磁盘余量脚本下载到 runner
|
||||
临时目录。它隔离 Git/npm/Docker/gh 状态,提供任务私有 npm 安装目录,
|
||||
断言 Node 22 并检查磁盘余量。清理脚本本身不依赖 Node。
|
||||
|
||||
汇总任务在 self-hosted 上使用现有的预装 Node 预检,在 hosted 上把
|
||||
`setup-node` 固定为 22.22.3,两者都在载荷校验前执行。原生编译始终使用
|
||||
Node 22.22.3。汇总任务保留不含架构的 label;Linux 构建任务保留与目标
|
||||
匹配的 x64 或 arm64 release runner label。
|
||||
|
||||
### 与 release host 开关的关系
|
||||
|
||||
`ECS_RELEASE_HOST_RUNNER_ENABLED` 启用或未设置时,音频复用都能工作。
|
||||
决策发生在 `prepare`。开关未设置时,现有 hosted 发布任务构建并消费音频
|
||||
产物;启用后,ECS 构建任务先消费音频产物,再把发布输出交给 hosted 发布
|
||||
任务,后者也会校验音频来源和载荷。两种情况下 npm 发布都保持 hosted,
|
||||
保留 #15832 已合入的构建与发布拆分。
|
||||
|
||||
本改动不改变 runner 路由,也不启用开关。命中缓存时跳过原生构建矩阵,
|
||||
未命中时沿用现有 hosted/ECS 路由。
|
||||
|
||||
## 验证
|
||||
|
||||
指纹、来源、保留期和载荷行为由单元测试配合直接 witness 覆盖,包括促成
|
||||
选择/消费分离的那个保留期边界。workflow 的接线——非致命查询、以 `outcome`
|
||||
为判据的接受步骤、矩阵跳过、发布方重新校验、汇总任务步骤顺序以及不含架构的
|
||||
label——由契约测试固定。最新 review 补充了针对接线回归的断言,并修正保留期
|
||||
测试的时钟;这些新增断言等待 CI,未在本地执行。
|
||||
|
||||
分支 release dry-run 可以在不发布的情况下验证未命中路径、五种原生载荷与
|
||||
发布方消费。复用来源在运行层面校验:仓库、`head_branch`、事件、workflow
|
||||
路径和成功结论。`head_branch` 表示派发分支,而不是实际构建的 `inputs.ref`。
|
||||
从功能分支派发的运行不能产生合格候选;从 `main` 派发、但将 `inputs.ref`
|
||||
指向功能分支的运行可以。指纹固定的是音频构建输入,并不证明源码提交属于
|
||||
`main`。先由一次成功且符合来源条件的运行保存带指纹的产物,后续相同 key 的
|
||||
运行才可验证复用。每次 dry-run 都应记录开关状态及各任务实际使用的 runner。hosted
|
||||
运行不能证明 ECS 路径,未命中运行也不能证明复用路径。
|
||||
|
||||
## 涉及文件
|
||||
|
||||
`.github/scripts/audio-prebuilds.mjs`、音频与发布 workflow、它复用的共享
|
||||
release runner 预检、`scripts/tests` 下的 workflow 测试,以及 workflow
|
||||
大小基线。
|
||||
|
||||
## 待确认
|
||||
|
||||
label 不变而 runner 镜像更新时,指纹不会变化。N-API 稳定的是 Node ABI,
|
||||
不是 glibc 或工具链链接,所以这里复用的是经过验证的二进制,而不保证重新
|
||||
构建能产出逐字节一致的结果。hosted 与 ECS 上的 Linux 使用带版本的 Bookworm 容器 tag,
|
||||
并非不可变镜像摘要;macOS 和 Windows 的 hosted 镜像也会继续更新。完全固定这些环境不在本次范围。
|
||||
@@ -0,0 +1,270 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { parse } from 'yaml';
|
||||
import { AUDIO_TARGETS } from '../../.github/scripts/audio-prebuilds.mjs';
|
||||
|
||||
const workflow = (name) =>
|
||||
parse(readFileSync(`.github/workflows/${name}.yml`, 'utf8'));
|
||||
const releaseYaml = workflow('release');
|
||||
const audioYaml = workflow('audio-capture-prebuilds');
|
||||
const step = (job, name) =>
|
||||
releaseYaml.jobs[job].steps.find((entry) => entry.name === name);
|
||||
|
||||
describe('audio prebuild reuse wiring', () => {
|
||||
it('builds every target required by payload validation', () => {
|
||||
const build = audioYaml.jobs.build;
|
||||
const targets = build.strategy.matrix.include.flatMap(({ os, arch }) => {
|
||||
if (os.startsWith('macos')) return [`darwin-${arch}`, 'darwin-x64'];
|
||||
return [`${os.startsWith('windows') ? 'win32' : 'linux'}-${arch}`];
|
||||
});
|
||||
expect(targets.sort()).toEqual([...AUDIO_TARGETS].sort());
|
||||
expect(
|
||||
build.steps.find((entry) => entry.name === 'Build prebuild').run,
|
||||
).toContain('npm run prebuildify -- --target 22.22.3 --arch x64');
|
||||
});
|
||||
|
||||
it('only publishes a reuse ID after a trial download and payload validation', () => {
|
||||
// The ID comes from the acceptance step, not the lookup: a candidate that
|
||||
// cannot be downloaded or does not validate must leave the output empty so
|
||||
// the fresh build still runs.
|
||||
for (const [output, source] of Object.entries({
|
||||
audio_prebuild_key: '${{ steps.audio_key.outputs.cache_key }}',
|
||||
audio_artifact_id: '${{ steps.audio_reuse.outputs.artifact_id }}',
|
||||
audio_run_id: '${{ steps.audio_reuse.outputs.run_id }}',
|
||||
}))
|
||||
expect(releaseYaml.jobs.prepare.outputs[output]).toBe(source);
|
||||
expect(releaseYaml.jobs.prepare.permissions.actions).toBe('read');
|
||||
|
||||
const lookup = step('prepare', 'Find reusable audio prebuilds');
|
||||
const fingerprint = step('prepare', 'Fingerprint audio build inputs');
|
||||
const download = step('prepare', 'Try reusable audio prebuilds');
|
||||
const accept = step('prepare', 'Accept complete audio prebuilds');
|
||||
expect(fingerprint.id).toBe('audio_key');
|
||||
expect(lookup.env.AUDIO_KEY).toBe(
|
||||
'${{ steps.audio_key.outputs.cache_key }}',
|
||||
);
|
||||
const steps = releaseYaml.jobs.prepare.steps;
|
||||
for (const [before, after] of [
|
||||
[fingerprint, lookup],
|
||||
[lookup, download],
|
||||
[download, accept],
|
||||
])
|
||||
expect(steps.indexOf(before)).toBeLessThan(steps.indexOf(after));
|
||||
// Failure before acceptance must leave the fresh-build path available.
|
||||
for (const entry of [fingerprint, lookup, download, accept])
|
||||
expect(entry['continue-on-error']).toBe(true);
|
||||
expect(fingerprint.run).toContain('echo "directory=$directory"');
|
||||
expect(fingerprint.run.indexOf('echo "directory=$directory"')).toBeLessThan(
|
||||
fingerprint.run.indexOf('audio-prebuilds.mjs key'),
|
||||
);
|
||||
expect(download.if).toBe(
|
||||
"${{ steps.audio_candidate.outputs.artifact_id != '' }}",
|
||||
);
|
||||
// outcome, not conclusion: continue-on-error rewrites conclusion to success.
|
||||
expect(accept.if).toBe("${{ steps.audio_download.outcome == 'success' }}");
|
||||
expect(accept.run.trim().split('\n')).toEqual([
|
||||
'node .release-workflow/.github/scripts/audio-prebuilds.mjs validate "$AUDIO_DIRECTORY"',
|
||||
'echo "artifact_id=$AUDIO_ARTIFACT_ID" >> "$GITHUB_OUTPUT"',
|
||||
'echo "run_id=$AUDIO_RUN_ID" >> "$GITHUB_OUTPUT"',
|
||||
]);
|
||||
|
||||
// The trial bytes are always discarded, including when a step failed.
|
||||
const cleanup = step('prepare', 'Remove trial audio download');
|
||||
expect(download.with.path).toBe('${{ steps.audio_key.outputs.directory }}');
|
||||
for (const entry of [accept, cleanup])
|
||||
expect(entry.env.AUDIO_DIRECTORY).toBe(download.with.path);
|
||||
expect(cleanup.if).toContain('always()');
|
||||
expect(cleanup.run).toBe('rm -rf -- "$AUDIO_DIRECTORY"');
|
||||
|
||||
// The fingerprint has to see the producing workflow, so it must be
|
||||
// checked out alongside the helper.
|
||||
const checkout = releaseYaml.jobs.prepare.steps.find((entry) =>
|
||||
entry.with?.['sparse-checkout']?.includes('/.github/scripts'),
|
||||
);
|
||||
expect(checkout.with['sparse-checkout']).toContain(
|
||||
'/.github/workflows/audio-capture-prebuilds.yml',
|
||||
);
|
||||
});
|
||||
|
||||
it('skips the native matrix on a hit and still gates the publisher', () => {
|
||||
const native = releaseYaml.jobs.audio_capture_prebuilds;
|
||||
expect(native.if).toContain(
|
||||
"needs.prepare.outputs.audio_artifact_id == ''",
|
||||
);
|
||||
expect(native.with.ref).toBe('${{ needs.prepare.outputs.release_sha }}');
|
||||
expect(native.with.cache_key).toBe(
|
||||
'${{ needs.prepare.outputs.audio_prebuild_key }}',
|
||||
);
|
||||
|
||||
// A skipped matrix is only acceptable when a reuse ID was actually
|
||||
// selected, otherwise the publisher would ship without prebuilds.
|
||||
const gate = releaseYaml.jobs.publish.if;
|
||||
expect(gate).toContain("needs.audio_capture_prebuilds.result == 'success'");
|
||||
expect(gate).toContain(
|
||||
"needs.audio_capture_prebuilds.result == 'skipped' && needs.prepare.outputs.audio_artifact_id != ''",
|
||||
);
|
||||
});
|
||||
|
||||
it.each(['build_release', 'publish'])(
|
||||
'rechecks the reused source and validates either path in %s',
|
||||
(job) => {
|
||||
expect(releaseYaml.jobs[job].permissions.actions).toBe('read');
|
||||
// prepare's decision is not trusted blindly: each consumer recomputes the
|
||||
// fingerprint from its own checkout and re-verifies the source run.
|
||||
const verify = step(job, 'Verify reusable audio source');
|
||||
expect(verify.if).toBe(
|
||||
"${{ needs.prepare.outputs.audio_artifact_id != '' }}",
|
||||
);
|
||||
expect(verify['continue-on-error']).toBeUndefined();
|
||||
expect(verify.with.script).toContain('audioPrebuildKey');
|
||||
expect(verify.with.script).toContain('verifyAudioArtifact');
|
||||
expect(verify.with.script).toContain(
|
||||
'await verifyAudioArtifact(github, context.repo, artifact, key);',
|
||||
);
|
||||
expect(verify.with.script).not.toContain('AUDIO_SELECTION_MARGIN_MS');
|
||||
|
||||
const reused = step(job, 'Download reusable audio prebuilds');
|
||||
const fresh = step(job, 'Download audio capture prebuilds');
|
||||
expect(reused.if).toBe(
|
||||
"${{ needs.prepare.outputs.audio_artifact_id != '' }}",
|
||||
);
|
||||
expect(fresh.if).toContain(
|
||||
"needs.prepare.outputs.audio_artifact_id == ''",
|
||||
);
|
||||
expect(reused.with['artifact-ids']).toBe(
|
||||
'${{ needs.prepare.outputs.audio_artifact_id }}',
|
||||
);
|
||||
expect(reused.with['run-id']).toBe(
|
||||
'${{ needs.prepare.outputs.audio_run_id }}',
|
||||
);
|
||||
// Exactly one of the two runs, and both land in the same directory.
|
||||
expect(reused.with.path).toBe(fresh.with.path);
|
||||
const validateStep = step(job, 'Validate complete audio prebuilds');
|
||||
expect(validateStep['continue-on-error']).toBeUndefined();
|
||||
expect(validateStep.if).toBe(
|
||||
"${{ github.repository == 'QwenLM/qwen-code' }}",
|
||||
);
|
||||
expect(validateStep.run).toBe(
|
||||
`node .release-workflow/.github/scripts/audio-prebuilds.mjs validate ${reused.with.path}`,
|
||||
);
|
||||
|
||||
// Whichever path ran, the payload is validated before the build consumes it.
|
||||
const names = releaseYaml.jobs[job].steps.map((entry) => entry.name);
|
||||
const validate = names.indexOf('Validate complete audio prebuilds');
|
||||
expect(validate).toBeGreaterThan(names.indexOf(reused.name));
|
||||
expect(validate).toBeGreaterThan(names.indexOf(fresh.name));
|
||||
expect(validate).toBeLessThan(
|
||||
names.indexOf('Build Bundle and Prepare Package'),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('retains a fingerprint-named artifact for later releases', () => {
|
||||
const collect = audioYaml.jobs.collect;
|
||||
const retain = collect.steps.find(
|
||||
(entry) => entry.name === 'Retain reusable audio prebuilds',
|
||||
);
|
||||
// Without the keyed upload nothing can ever be reused; the fixed-name
|
||||
// artifact stays for the same-run consumer.
|
||||
expect(retain.if).toBe("${{ inputs.cache_key != '' }}");
|
||||
expect(retain['continue-on-error']).toBe(true);
|
||||
expect(retain.with.path).toBe('packages/audio-capture/prebuilds/');
|
||||
expect(retain.with.name).toBe(
|
||||
'audio-capture-prebuilds-${{ inputs.cache_key }}',
|
||||
);
|
||||
expect(retain.with['if-no-files-found']).toBe('error');
|
||||
expect(retain.with.overwrite).toBe(true);
|
||||
expect(
|
||||
collect.steps.find((entry) => entry.name === 'Upload combined prebuilds')
|
||||
.with.name,
|
||||
).toBe('audio-capture-prebuilds');
|
||||
expect(
|
||||
collect.steps.find((entry) => entry.name === 'Upload combined prebuilds')
|
||||
.with.overwrite,
|
||||
).toBe(true);
|
||||
expect(
|
||||
collect.steps.find((entry) => entry.name === 'Upload combined prebuilds')[
|
||||
'continue-on-error'
|
||||
],
|
||||
).toBeUndefined();
|
||||
// The producer builds the ref the release pinned, not the branch tip.
|
||||
expect(
|
||||
audioYaml.jobs.build.steps.find((entry) =>
|
||||
entry.uses?.startsWith('actions/checkout@'),
|
||||
).with.ref,
|
||||
).toBe('${{ inputs.ref || github.sha }}');
|
||||
});
|
||||
});
|
||||
|
||||
describe('prebuild collector hygiene', () => {
|
||||
it('wipes the workspace through the shared preflight before merging', () => {
|
||||
// download-artifact's merge-multiple merges into existing paths, so a
|
||||
// stale prebuilds/ tree on a persistent host would leak into the artifact.
|
||||
// The shared preflight wipes with path containment; the subtree rm it
|
||||
// replaced followed a leftover symlinked parent out of the workspace.
|
||||
const steps = audioYaml.jobs.collect.steps;
|
||||
expect(steps[0].uses).toBe('$/.github/actions/prepare-release-runner');
|
||||
expect(steps[1].name).toBe('Merge per-platform prebuilds');
|
||||
for (const entry of steps)
|
||||
expect(entry.run ?? '').not.toContain('rm -rf -- "${GITHUB_WORKSPACE}');
|
||||
});
|
||||
|
||||
it('keeps the release-only label architecture-free', () => {
|
||||
// The release pool includes ARM64 runners. An "x64"
|
||||
// component does not fall back to hosted — it never matches, and the job
|
||||
// queues until it times out. Resolve the expression rather than
|
||||
// substring-matching it, which would pass with the pin present.
|
||||
const route = (enabled, repository = 'QwenLM/qwen-code') =>
|
||||
Function(
|
||||
'github',
|
||||
'vars',
|
||||
'fromJSON',
|
||||
`return (${audioYaml.jobs.collect['runs-on'].slice(3, -2)})`,
|
||||
)(
|
||||
{ repository },
|
||||
{ ECS_RELEASE_HOST_RUNNER_ENABLED: enabled },
|
||||
JSON.parse,
|
||||
);
|
||||
expect(route('true')).toEqual([
|
||||
'self-hosted',
|
||||
'linux',
|
||||
'ecs-qwen-release-host',
|
||||
]);
|
||||
expect([route(''), route('false'), route('true', 'fork/repo')]).toEqual([
|
||||
['ubuntu-latest'],
|
||||
['ubuntu-latest'],
|
||||
['ubuntu-latest'],
|
||||
]);
|
||||
});
|
||||
|
||||
it('has a Node contract on both runner kinds before it runs node', () => {
|
||||
// The validator shells out to `node`; the collector previously inherited
|
||||
// Node from the hosted image by accident and asserted nothing.
|
||||
const steps = audioYaml.jobs.collect.steps;
|
||||
const node = steps.filter(
|
||||
(entry) =>
|
||||
entry.uses?.endsWith('.github/actions/self-hosted-node') ||
|
||||
entry.uses?.startsWith('actions/setup-node@'),
|
||||
);
|
||||
expect(node.map((entry) => entry.if)).toEqual([
|
||||
"${{ runner.environment == 'self-hosted' }}",
|
||||
"${{ runner.environment != 'self-hosted' }}",
|
||||
]);
|
||||
const validate = steps.findIndex((entry) =>
|
||||
entry.run?.includes('audio-prebuilds.mjs validate'),
|
||||
);
|
||||
for (const entry of node)
|
||||
expect(steps.indexOf(entry)).toBeLessThan(validate);
|
||||
// self-hosted-node is resolved from the helper checkout, so that checkout
|
||||
// has to include it.
|
||||
expect(
|
||||
steps.find((entry) => entry.name === 'Checkout audio workflow helper')
|
||||
.with['sparse-checkout'],
|
||||
).toContain('.github/actions/self-hosted-node');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,365 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import {
|
||||
mkdirSync,
|
||||
mkdtempSync,
|
||||
rmSync,
|
||||
symlinkSync,
|
||||
writeFileSync,
|
||||
} from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
AUDIO_SELECTION_MARGIN_MS,
|
||||
AUDIO_TARGETS,
|
||||
audioPrebuildKey,
|
||||
findAudioArtifact,
|
||||
validateAudioPrebuilds,
|
||||
verifyAudioArtifact,
|
||||
} from '../../.github/scripts/audio-prebuilds.mjs';
|
||||
|
||||
const directories = [];
|
||||
const temporary = () => {
|
||||
const path = mkdtempSync(join(tmpdir(), 'audio-prebuilds-'));
|
||||
directories.push(path);
|
||||
return path;
|
||||
};
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
directories
|
||||
.splice(0)
|
||||
.forEach((path) => rmSync(path, { recursive: true, force: true }));
|
||||
});
|
||||
const write = (root, path, content) => {
|
||||
mkdirSync(dirname(join(root, path)), { recursive: true });
|
||||
writeFileSync(join(root, path), content);
|
||||
};
|
||||
|
||||
describe('audio prebuild identity', () => {
|
||||
function fixture() {
|
||||
const root = temporary();
|
||||
const git = (...args) =>
|
||||
execFileSync('git', args, {
|
||||
cwd: root,
|
||||
encoding: 'utf8',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
}).trim();
|
||||
git('init', '-b', 'main');
|
||||
git('config', 'user.name', 'Test');
|
||||
git('config', 'user.email', '[email protected]');
|
||||
for (const [path, content] of Object.entries({
|
||||
'package.json': '{"version":"1.0.0","packageManager":"[email protected]"}',
|
||||
'packages/audio-capture/package.json':
|
||||
'{"name":"audio","version":"1.0.0","scripts":{"prebuildify":"prebuildify --napi"}}',
|
||||
'packages/audio-capture/native/audio.cc': 'native source',
|
||||
'packages/audio-capture/binding.gyp': 'binding',
|
||||
'packages/audio-capture/install.js': 'install',
|
||||
'pnpm-lock.yaml': 'locked dependencies',
|
||||
'pnpm-workspace.yaml': 'workspace settings',
|
||||
'.pnpmfile.mjs': 'dependency hook',
|
||||
'.npmrc': 'registry=https://registry.npmjs.org',
|
||||
'.github/workflows/audio-capture-prebuilds.yml': 'native workflow',
|
||||
'.github/scripts/audio-prebuilds.mjs': 'trusted helper',
|
||||
}))
|
||||
write(root, path, content);
|
||||
const commit = () => {
|
||||
git('add', '.');
|
||||
git('commit', '-m', 'fixture');
|
||||
return git('rev-parse', 'HEAD');
|
||||
};
|
||||
const ref = commit();
|
||||
return {
|
||||
root,
|
||||
git,
|
||||
ref,
|
||||
commit,
|
||||
key: () => audioPrebuildKey(root, 'HEAD', root),
|
||||
};
|
||||
}
|
||||
|
||||
it('ignores release versions and uses the selected commit rather than dirty files', () => {
|
||||
const f = fixture();
|
||||
const key = f.key();
|
||||
write(
|
||||
f.root,
|
||||
'packages/audio-capture/package.json',
|
||||
'{"name":"audio","version":"2.0.0","scripts":{"prebuildify":"prebuildify --napi"}}',
|
||||
);
|
||||
write(
|
||||
f.root,
|
||||
'package.json',
|
||||
'{"version":"2.0.0","packageManager":"[email protected]"}',
|
||||
);
|
||||
f.commit();
|
||||
expect(f.key()).toBe(key);
|
||||
write(f.root, 'packages/audio-capture/native/audio.cc', 'changed');
|
||||
expect(f.key()).toBe(key);
|
||||
f.commit();
|
||||
expect(f.key()).not.toBe(key);
|
||||
expect(audioPrebuildKey(f.root, f.ref, f.root)).toBe(key);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'packages/audio-capture/binding.gyp',
|
||||
'packages/audio-capture/install.js',
|
||||
'pnpm-lock.yaml',
|
||||
'pnpm-workspace.yaml',
|
||||
'.pnpmfile.mjs',
|
||||
'.npmrc',
|
||||
'.github/workflows/audio-capture-prebuilds.yml',
|
||||
'.github/scripts/audio-prebuilds.mjs',
|
||||
])('invalidates reuse for %s', (path) => {
|
||||
const f = fixture();
|
||||
const key = f.key();
|
||||
write(f.root, path, 'changed input');
|
||||
f.commit();
|
||||
expect(f.key()).not.toBe(key);
|
||||
});
|
||||
|
||||
it('invalidates changed package scripts and package manager versions', () => {
|
||||
const f = fixture();
|
||||
const key = f.key();
|
||||
write(
|
||||
f.root,
|
||||
'packages/audio-capture/package.json',
|
||||
'{"name":"audio","version":"1.0.0","scripts":{"prebuildify":"different"}}',
|
||||
);
|
||||
f.commit();
|
||||
const changedScript = f.key();
|
||||
expect(changedScript).not.toBe(key);
|
||||
write(f.root, 'package.json', '{"packageManager":"[email protected]"}');
|
||||
f.commit();
|
||||
expect(f.key()).not.toBe(changedScript);
|
||||
});
|
||||
|
||||
it('fingerprints the workflow checkout separately from the release ref', () => {
|
||||
const f = fixture();
|
||||
const workflowRoot = temporary();
|
||||
write(
|
||||
workflowRoot,
|
||||
'.github/scripts/audio-prebuilds.mjs',
|
||||
'trusted helper',
|
||||
);
|
||||
const path = '.github/workflows/audio-capture-prebuilds.yml';
|
||||
write(workflowRoot, path, 'workflow at dispatch');
|
||||
const key = audioPrebuildKey(f.root, f.ref, workflowRoot);
|
||||
write(f.root, path, 'unrelated release checkout edit');
|
||||
expect(audioPrebuildKey(f.root, f.ref, workflowRoot)).toBe(key);
|
||||
write(workflowRoot, path, 'updated native toolchain');
|
||||
expect(audioPrebuildKey(f.root, f.ref, workflowRoot)).not.toBe(key);
|
||||
});
|
||||
|
||||
it('does not reuse refs without a native dependency lock', () => {
|
||||
const f = fixture();
|
||||
f.git('rm', 'pnpm-lock.yaml');
|
||||
f.commit();
|
||||
expect(f.key()).toBe('');
|
||||
});
|
||||
|
||||
it('fingerprints large native headers without a child-process buffer limit', () => {
|
||||
const f = fixture();
|
||||
const path = 'packages/audio-capture/native/miniaudio.h';
|
||||
write(f.root, path, 'x'.repeat(4 * 1024 * 1024));
|
||||
f.commit();
|
||||
const key = f.key();
|
||||
write(f.root, path, 'y'.repeat(4 * 1024 * 1024));
|
||||
f.commit();
|
||||
expect(f.key()).not.toBe(key);
|
||||
});
|
||||
});
|
||||
|
||||
describe('audio prebuild payload', () => {
|
||||
const targets = [
|
||||
'darwin-arm64',
|
||||
'darwin-x64',
|
||||
'linux-arm64',
|
||||
'linux-x64',
|
||||
'win32-x64',
|
||||
];
|
||||
const fixture = () => {
|
||||
const root = temporary();
|
||||
for (const target of targets)
|
||||
write(root, `${target}/audio.node`, 'native binary');
|
||||
return root;
|
||||
};
|
||||
it('accepts all five nonempty targets', () => {
|
||||
expect(AUDIO_TARGETS).toEqual(targets);
|
||||
expect(() => validateAudioPrebuilds(fixture())).not.toThrow();
|
||||
});
|
||||
it.each(AUDIO_TARGETS)('rejects missing %s', (target) => {
|
||||
const root = fixture();
|
||||
rmSync(join(root, target), { recursive: true });
|
||||
expect(() => validateAudioPrebuilds(root)).toThrow();
|
||||
});
|
||||
it.each(['empty', 'extra', 'symlink'])('rejects %s binaries', (kind) => {
|
||||
const root = fixture();
|
||||
if (kind === 'empty') write(root, 'linux-x64/audio.node', '');
|
||||
if (kind === 'extra') write(root, 'linux-x64/unexpected.js', 'code');
|
||||
if (kind === 'symlink') {
|
||||
rmSync(join(root, 'linux-x64/audio.node'));
|
||||
symlinkSync(
|
||||
join(root, 'linux-arm64/audio.node'),
|
||||
join(root, 'linux-x64/audio.node'),
|
||||
);
|
||||
}
|
||||
expect(() => validateAudioPrebuilds(root)).toThrow();
|
||||
});
|
||||
it('rejects symlinked platform directories', () => {
|
||||
const root = fixture();
|
||||
rmSync(join(root, 'linux-x64'), { recursive: true });
|
||||
symlinkSync(join(root, 'linux-arm64'), join(root, 'linux-x64'), 'junction');
|
||||
expect(() => validateAudioPrebuilds(root)).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('audio artifact provenance', () => {
|
||||
const repo = { owner: 'QwenLM', repo: 'qwen-code' };
|
||||
const fixture = () => {
|
||||
const artifact = {
|
||||
id: 12,
|
||||
name: 'audio-capture-prebuilds-key',
|
||||
expired: false,
|
||||
expires_at: '2100-01-01T00:00:00Z',
|
||||
workflow_run: {
|
||||
id: 34,
|
||||
head_branch: 'main',
|
||||
repository_id: 1,
|
||||
head_repository_id: 1,
|
||||
},
|
||||
};
|
||||
const run = {
|
||||
id: 34,
|
||||
status: 'completed',
|
||||
conclusion: 'success',
|
||||
head_branch: 'main',
|
||||
path: '.github/workflows/release.yml',
|
||||
event: 'workflow_dispatch',
|
||||
repository: { full_name: 'QwenLM/qwen-code' },
|
||||
head_repository: { full_name: 'QwenLM/qwen-code' },
|
||||
};
|
||||
const github = {
|
||||
rest: {
|
||||
actions: {
|
||||
getWorkflowRun: vi.fn(async () => ({ data: run })),
|
||||
listArtifactsForRepo: vi.fn(async () => ({
|
||||
data: { artifacts: [artifact] },
|
||||
})),
|
||||
},
|
||||
},
|
||||
};
|
||||
return { artifact, run, github };
|
||||
};
|
||||
it.each(['workflow_dispatch', 'schedule'])(
|
||||
'selects an immutable artifact ID from a successful main %s release',
|
||||
async (event) => {
|
||||
const f = fixture();
|
||||
f.run.event = event;
|
||||
await expect(
|
||||
findAudioArtifact(f.github, repo, 'key', vi.fn()),
|
||||
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
|
||||
},
|
||||
);
|
||||
it.each([
|
||||
['expired', true],
|
||||
['expires_at', 'invalid'],
|
||||
['expires_at', new Date(Date.now() - 1000).toISOString()],
|
||||
['name', 'other'],
|
||||
])('rejects artifact %s=%s', async (field, value) => {
|
||||
const f = fixture();
|
||||
f.artifact[field] = value;
|
||||
await expect(
|
||||
verifyAudioArtifact(f.github, repo, f.artifact, 'key'),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
it('requires selection headroom but consumes any unexpired artifact', async () => {
|
||||
const f = fixture();
|
||||
const now = Date.parse('2026-10-29T00:00:00Z');
|
||||
vi.spyOn(Date, 'now').mockReturnValue(now);
|
||||
expect(AUDIO_SELECTION_MARGIN_MS).toBe(48 * 60 * 60 * 1000);
|
||||
// Just over the selection margin: chosen by prepare, then consumed later.
|
||||
f.artifact.expires_at = new Date(
|
||||
now + AUDIO_SELECTION_MARGIN_MS + 60_000,
|
||||
).toISOString();
|
||||
await expect(
|
||||
verifyAudioArtifact(
|
||||
f.github,
|
||||
repo,
|
||||
f.artifact,
|
||||
'key',
|
||||
now,
|
||||
AUDIO_SELECTION_MARGIN_MS,
|
||||
),
|
||||
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
|
||||
await expect(
|
||||
findAudioArtifact(f.github, repo, 'key', vi.fn()),
|
||||
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
|
||||
// The downstream consumers re-verify after the release has already
|
||||
// committed to reuse and skipped the fresh build, so they must not
|
||||
// re-apply the margin: the same artifact still has ~48h left.
|
||||
await expect(
|
||||
verifyAudioArtifact(f.github, repo, f.artifact, 'key', now + 120_000),
|
||||
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
|
||||
// Too thin to survive the release: never selected in the first place.
|
||||
f.artifact.expires_at = new Date(now + 60_000).toISOString();
|
||||
await expect(
|
||||
verifyAudioArtifact(
|
||||
f.github,
|
||||
repo,
|
||||
f.artifact,
|
||||
'key',
|
||||
now,
|
||||
AUDIO_SELECTION_MARGIN_MS,
|
||||
),
|
||||
).rejects.toThrow();
|
||||
await expect(
|
||||
findAudioArtifact(f.github, repo, 'key', vi.fn()),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
it.each([
|
||||
['conclusion', 'failure'],
|
||||
['status', 'in_progress'],
|
||||
['head_branch', 'feature'],
|
||||
['path', '.github/workflows/ci.yml'],
|
||||
['event', 'pull_request'],
|
||||
['head_repository', { full_name: 'attacker/fork' }],
|
||||
])('rejects producer %s', async (field, value) => {
|
||||
const f = fixture();
|
||||
f.run[field] = value;
|
||||
await expect(
|
||||
verifyAudioArtifact(f.github, repo, f.artifact, 'key'),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
it('rejects a fork artifact before requesting its run', async () => {
|
||||
const f = fixture();
|
||||
f.artifact.workflow_run.head_repository_id = 2;
|
||||
await expect(
|
||||
verifyAudioArtifact(f.github, repo, f.artifact, 'key'),
|
||||
).rejects.toThrow();
|
||||
expect(f.github.rest.actions.getWorkflowRun).not.toHaveBeenCalled();
|
||||
});
|
||||
it('falls back on API failure and skips rejected candidates', async () => {
|
||||
const f = fixture();
|
||||
const warn = vi.fn();
|
||||
f.github.rest.actions.listArtifactsForRepo.mockRejectedValueOnce(
|
||||
new Error('offline'),
|
||||
);
|
||||
await expect(
|
||||
findAudioArtifact(f.github, repo, 'key', warn),
|
||||
).resolves.toBeUndefined();
|
||||
f.github.rest.actions.listArtifactsForRepo.mockResolvedValueOnce({
|
||||
data: {
|
||||
artifacts: [{ ...f.artifact, id: 13, expired: true }, f.artifact],
|
||||
},
|
||||
});
|
||||
await expect(
|
||||
findAudioArtifact(f.github, repo, 'key', warn),
|
||||
).resolves.toEqual({ artifact_id: 12, run_id: 34 });
|
||||
expect(warn).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
@@ -264,11 +264,11 @@ describe('audio release slots', () => {
|
||||
expect(route(audio.jobs.collect, {}, '')).toEqual(['ubuntu-latest']);
|
||||
});
|
||||
|
||||
it('builds the selected release commit without a cache dependency', () => {
|
||||
it('builds the selected release commit', () => {
|
||||
expect(release.jobs.audio_capture_prebuilds.uses).toBe(
|
||||
'./.github/workflows/audio-capture-prebuilds.yml',
|
||||
);
|
||||
expect(release.jobs.audio_capture_prebuilds.with).toEqual({
|
||||
expect(release.jobs.audio_capture_prebuilds.with).toMatchObject({
|
||||
ref: '${{ needs.prepare.outputs.release_sha }}',
|
||||
});
|
||||
expect(
|
||||
@@ -276,9 +276,6 @@ describe('audio release slots', () => {
|
||||
s.uses?.startsWith('actions/checkout@'),
|
||||
).with.ref,
|
||||
).toBe('${{ inputs.ref || github.sha }}');
|
||||
expect(JSON.stringify(release)).not.toMatch(
|
||||
/audio_artifact_id|audio_prebuild_key|audio-prebuilds\.mjs/,
|
||||
);
|
||||
for (const job of [audio.jobs.build, audio.jobs.collect]) {
|
||||
expect(job.steps[0].uses).toBe(
|
||||
'$/.github/actions/prepare-release-runner',
|
||||
@@ -288,15 +285,16 @@ describe('audio release slots', () => {
|
||||
|
||||
it('keeps newer host glibc out of Linux native binaries', () => {
|
||||
const steps = audio.jobs.build.steps;
|
||||
const container = steps.find(
|
||||
(s) => s.name === 'Build Linux prebuild on ECS',
|
||||
);
|
||||
expect(container.if).toBe("${{ runner.environment == 'self-hosted' }}");
|
||||
expect(container.run).toContain('node:22-bookworm');
|
||||
const container = steps.find((s) => s.name === 'Build Linux prebuild');
|
||||
expect(container.if).toBe("${{ startsWith(matrix.os, 'ubuntu') }}");
|
||||
expect(container.run).toContain('node:22.22.3-bookworm');
|
||||
expect(container['working-directory']).toBe('.');
|
||||
expect(container.run).toContain('--frozen-lockfile --ignore-scripts');
|
||||
expect(container.run).toContain('npm run prebuildify -- --target 22.22.3');
|
||||
expect(container.run).toContain('--user "$(id -u):$(id -g)"');
|
||||
for (const name of ['Install build deps', 'Build prebuild']) {
|
||||
expect(steps.find((s) => s.name === name).if).toBe(
|
||||
"${{ runner.environment != 'self-hosted' }}",
|
||||
"${{ !startsWith(matrix.os, 'ubuntu') }}",
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -28,6 +28,7 @@ it
|
||||
'normal',
|
||||
'workspace-leaf-link',
|
||||
'workspace-parent-link',
|
||||
'workspace-contained-parent-link',
|
||||
'runner-link',
|
||||
'workspace-outside',
|
||||
'workspace-equals-runner',
|
||||
@@ -58,6 +59,11 @@ it
|
||||
symlinkSync(outside, join(runner, 'parent'), 'dir');
|
||||
mkdirSync(join(outside, 'repo'));
|
||||
workspaceInput = join(runner, 'parent', 'repo');
|
||||
} else if (kind === 'workspace-contained-parent-link') {
|
||||
mkdirSync(workspace);
|
||||
writeFileSync(join(workspace, 'sentinel'), 'KEEP_INTERNAL');
|
||||
symlinkSync(runner, join(runner, 'parent'), 'dir');
|
||||
workspaceInput = join(runner, 'parent', 'repo');
|
||||
} else if (kind === 'runner-link') {
|
||||
mkdirSync(workspace);
|
||||
symlinkSync(runner, join(root, 'runner-link'), 'dir');
|
||||
@@ -99,6 +105,11 @@ it
|
||||
'KEEP_EXTERNAL',
|
||||
);
|
||||
assert.equal(statSync(join(outside, 'sentinel')).mode & 0o777, 0o444);
|
||||
if (kind === 'workspace-contained-parent-link')
|
||||
assert.equal(
|
||||
readFileSync(join(workspace, 'sentinel'), 'utf8'),
|
||||
'KEEP_INTERNAL',
|
||||
);
|
||||
if (success) {
|
||||
assert.ok(lstatSync(workspace).isDirectory());
|
||||
assert.deepEqual(readdirSync(workspace), []);
|
||||
|
||||
@@ -577,7 +577,7 @@ describe('release workflow', () => {
|
||||
// The tarball gate rides this checkout too: run from the selected
|
||||
// ref's own `scripts/`, it is missing on any ref that predates it.
|
||||
'sparse-checkout':
|
||||
'/.github/scripts\n/scripts/assert-release-version.mjs\n/scripts/check-release-tarball-exec-bits.mjs\n/scripts/release-packages.mjs\n/scripts/workspaces.js\n/package.json\n/packages/*/package.json\n/packages/channels/*/package.json\n/integrations/*/package.json',
|
||||
'/.github/scripts\n/.github/workflows/audio-capture-prebuilds.yml\n/scripts/assert-release-version.mjs\n/scripts/check-release-tarball-exec-bits.mjs\n/scripts/release-packages.mjs\n/scripts/workspaces.js\n/package.json\n/packages/*/package.json\n/packages/channels/*/package.json\n/integrations/*/package.json',
|
||||
'sparse-checkout-cone-mode': false,
|
||||
path: '.release-workflow',
|
||||
});
|
||||
@@ -3884,6 +3884,22 @@ describe('release artifact handoff', () => {
|
||||
expectedBuild: true,
|
||||
expectedPublish: true,
|
||||
},
|
||||
{
|
||||
name: 'enabled, reused audio',
|
||||
audio: 'skipped',
|
||||
artifact: '123',
|
||||
expectedBuild: true,
|
||||
expectedPublish: true,
|
||||
},
|
||||
{
|
||||
name: 'unset switch, reused audio',
|
||||
enabled: '',
|
||||
audio: 'skipped',
|
||||
artifact: '123',
|
||||
build: 'skipped',
|
||||
expectedBuild: false,
|
||||
expectedPublish: true,
|
||||
},
|
||||
{
|
||||
name: 'missing audio',
|
||||
audio: 'skipped',
|
||||
@@ -3944,6 +3960,7 @@ describe('release artifact handoff', () => {
|
||||
const needs = {
|
||||
prepare: {
|
||||
result: scenario.prepare ?? 'success',
|
||||
outputs: { audio_artifact_id: scenario.artifact ?? '' },
|
||||
},
|
||||
quality: { result: scenario.quality ?? 'success' },
|
||||
integration_none: { result: 'success' },
|
||||
|
||||
Reference in new issue
Block a user