mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
fix(release): cover every spawned vendored binary in the tarball exec-bit gate (#15008)
* fix(release): cover every spawned vendored binary in the tarball exec-bit gate The gate matched only vendor/ripgrep/*/rg, but the tarball spawns a second vendored executable: vendor/landlock-run/<arch>-linux/qwen-landlock-run. Its own runtime heal shares the ripgrep heal's hard limit — it can only chmod an install the running user owns, so a root-owned sudo install degrades the same silent way. Widen the expected set with a small spec list keyed on the leaf binary's presence (the landlock-run/src source directory falls out structurally), parse every regular-file tar entry instead of a single ripgrep regex, and extend the fixture tests to cover both trees with pass/reject cases. Also correct the stale comment above the landlock heal: it claimed npm archives strip non-bin exec bits, which is the inverse of what #12668 established — pnpm's packer wrote 0644 for every non-bin entry from 0.24.4 to 0.25.3; npm keeps working-tree modes. Refs #12668 * fix(release): keep missing-binary strictness in the widened exec-bit gate Deriving the expected set from the leaf binary's presence meant a platform directory whose binary was dropped during staging silently shrank the expected set: a dist without vendor/ripgrep/x64-linux/rg but with the directory would pass with five entries. Platform membership now comes from the directory name alone — ripgrep skips '-win32', landlock-run skips the 'src' source directory — so a missing binary fails as 'missing from the packed tarball' again, matching the pre-widening strictness. Also update the function doc block, which still claimed the gate covered ripgrep only. Regression test: an empty ripgrep platform directory fails with the missing-entry diagnostic even when every other binary passes.
This commit is contained in:
4 files changed
+133
-67
No files matched your search
@@ -171,8 +171,8 @@ case "${step}" in
|
||||
}
|
||||
npm run prepare:package
|
||||
# Last check before the publish step: the tarball the registry will serve
|
||||
# must carry the exec bit on every vendored ripgrep (#12668). The gate rides
|
||||
# the workflow-pinned checkout, not the selected release ref, which may
|
||||
# must carry the exec bit on every spawned vendored binary (#12668). The gate
|
||||
# rides the workflow-pinned checkout, not the selected release ref, which may
|
||||
# predate it; `dist` still resolves against the release ref's package.
|
||||
node .release-workflow/scripts/check-release-tarball-exec-bits.mjs dist
|
||||
;;
|
||||
|
||||
@@ -51,7 +51,9 @@ export function resolveLandlockRunner(requestedPath?: string): string {
|
||||
if (!existsSync(requested))
|
||||
throw new Error(`Landlock helper is missing: ${requested}`);
|
||||
const runner = realpathSync(requested);
|
||||
// npm archives strip executable bits from files outside the bin entries.
|
||||
// pnpm-packed releases (0.24.4–0.25.3) shipped every non-`bin` entry as
|
||||
// 0644 — only those installs need the bit restored, and a read-only or
|
||||
// root-owned install cannot be chmod'ed by the running user at all (#12668).
|
||||
if (requestedPath === undefined && (statSync(runner).mode & 0o111) === 0)
|
||||
chmodSync(runner, 0o755);
|
||||
return runner;
|
||||
|
||||
@@ -7,24 +7,24 @@
|
||||
*/
|
||||
|
||||
/**
|
||||
* Release gate for the vendored ripgrep's exec bits in the published CLI
|
||||
* tarball (#12668).
|
||||
* Release gate for the exec bits of spawned vendored binaries in the
|
||||
* published CLI tarball (#12668).
|
||||
*
|
||||
* From 0.24.4 on the release published the generated CLI with `pnpm publish`,
|
||||
* whose packer writes 0755 for the entries named in the manifest's `bin` map
|
||||
* and 0644 for every other entry — so `vendor/ripgrep/<platform>/rg` shipped
|
||||
* without an exec bit and a fresh install could not spawn the bundled ripgrep.
|
||||
* `npm pack` keeps the working-tree mode. This gate packs the prepared CLI
|
||||
* with the same packer the publish step uses and reads the modes recorded in
|
||||
* the archive (no extraction, so the umask cannot mask them), failing when any
|
||||
* unix vendored ripgrep lacks all three exec bits: a root-owned
|
||||
* and 0644 for every other entry — so every vendored binary shipped without
|
||||
* an exec bit and a fresh install could not spawn it. `npm pack` keeps the
|
||||
* working-tree mode. This gate packs the prepared CLI with the same packer
|
||||
* the publish step uses and reads the modes recorded in the archive (no
|
||||
* extraction, so the umask cannot mask them), failing when any spawned
|
||||
* vendored binary lacks all three exec bits: a root-owned
|
||||
* `sudo npm install -g` is only executable by others with 0111 set, and the
|
||||
* runtime heal cannot chmod such an install.
|
||||
* runtime heals cannot chmod such an install.
|
||||
*
|
||||
* Usage: node scripts/check-release-tarball-exec-bits.mjs [package-dir]
|
||||
* Exit 0 = every unix vendored ripgrep keeps 0111; exit 1 = one does not, or
|
||||
* the gate could not measure (a Windows host, an unreadable directory, no
|
||||
* tarball, or a missing `vendor/ripgrep`). It never passes vacuously.
|
||||
* Exit 0 = every spawned vendored binary keeps 0111; exit 1 = one does not,
|
||||
* or the gate could not measure (a Windows host, an unreadable directory, no
|
||||
* tarball, or no spawned vendored binaries). It never passes vacuously.
|
||||
*/
|
||||
|
||||
import { spawnSync } from 'node:child_process';
|
||||
@@ -33,9 +33,27 @@ import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
// `rg.exe` is a 0644 Windows binary by design, so only a bare unix `rg` name
|
||||
// matches; `-win32` directories carry nothing else this gate must inspect.
|
||||
const UNIX_RIPGREP_ENTRY = /^package\/vendor\/ripgrep\/[^/]+\/rg$/;
|
||||
// Vendored binaries the CLI spawns directly, one expected entry per platform
|
||||
// directory. Platform membership comes from the directory name alone: a
|
||||
// platform directory whose binary is missing is itself a broken package and
|
||||
// must fail the gate, never shrink the expected set. `rg.exe` is a 0644
|
||||
// Windows binary by design, so `-win32` platforms are excluded;
|
||||
// `landlock-run/src` is a source directory, not a platform, and is excluded
|
||||
// by name. Any future vendored binary spawned the same way belongs here —
|
||||
// its runtime heal shares the ripgrep heal's limit of being unable to chmod
|
||||
// a root-owned or read-only install (#12668).
|
||||
const SPAWNED_VENDORED_BINARIES = [
|
||||
{
|
||||
root: ['vendor', 'ripgrep'],
|
||||
leaf: 'rg',
|
||||
skip: (platform) => platform.endsWith('-win32'),
|
||||
},
|
||||
{
|
||||
root: ['vendor', 'landlock-run'],
|
||||
leaf: 'qwen-landlock-run',
|
||||
skip: (platform) => platform === 'src',
|
||||
},
|
||||
];
|
||||
|
||||
// read/write/exec bit values for owner, group and other, in tar order.
|
||||
const MODE_CLASSES = [
|
||||
@@ -65,8 +83,8 @@ function symbolicModeToOctal(mode) {
|
||||
return bits;
|
||||
}
|
||||
|
||||
/** Maps every unix `vendor/ripgrep/<platform>/rg` entry to its archive mode. */
|
||||
function listRipgrepEntries(tarball) {
|
||||
/** Maps every regular-file entry in the archive to its recorded mode. */
|
||||
function listTarEntryModes(tarball) {
|
||||
const listing = spawnSync('tar', ['-tvzf', tarball], { encoding: 'utf8' });
|
||||
if (listing.status !== 0) {
|
||||
throw new Error(
|
||||
@@ -75,16 +93,19 @@ function listRipgrepEntries(tarball) {
|
||||
}
|
||||
const entries = new Map();
|
||||
for (const line of listing.stdout.split('\n')) {
|
||||
if (line[0] !== '-') continue;
|
||||
const name = line.trim().split(/\s+/).pop();
|
||||
if (name === undefined || !UNIX_RIPGREP_ENTRY.test(name)) continue;
|
||||
entries.set(name, symbolicModeToOctal(line.slice(0, 10)));
|
||||
if (name !== undefined) {
|
||||
entries.set(name, symbolicModeToOctal(line.slice(0, 10)));
|
||||
}
|
||||
}
|
||||
return entries;
|
||||
}
|
||||
|
||||
/**
|
||||
* Packs `packageDir` the way the release publishes it and asserts every unix
|
||||
* vendored ripgrep entry keeps 0111. Throws with a per-entry report otherwise.
|
||||
* Packs `packageDir` the way the release publishes it and asserts every
|
||||
* spawned vendored binary entry keeps 0111. Throws with a per-entry report
|
||||
* otherwise.
|
||||
*/
|
||||
export function checkReleaseTarballExecBits(packageDir = 'dist') {
|
||||
if (process.platform === 'win32') {
|
||||
@@ -92,11 +113,11 @@ export function checkReleaseTarballExecBits(packageDir = 'dist') {
|
||||
'the release tarball exec-bit gate cannot read POSIX tar entry modes on Windows',
|
||||
);
|
||||
}
|
||||
const expected = expectedRipgrepEntries(packageDir);
|
||||
const expected = expectedSpawnedEntries(packageDir);
|
||||
const destination = mkdtempSync(join(tmpdir(), 'release-tarball-exec-bits-'));
|
||||
try {
|
||||
const tarball = pack(packageDir, destination);
|
||||
const actual = listRipgrepEntries(tarball);
|
||||
const actual = listTarEntryModes(tarball);
|
||||
const problems = [];
|
||||
for (const name of expected) {
|
||||
const mode = actual.get(name);
|
||||
@@ -109,7 +130,7 @@ export function checkReleaseTarballExecBits(packageDir = 'dist') {
|
||||
if (problems.length > 0) {
|
||||
throw new Error(
|
||||
[
|
||||
`the CLI tarball ships a vendored ripgrep without all exec bits (${packageDir})`,
|
||||
`the CLI tarball ships a spawned vendored binary without all exec bits (${packageDir})`,
|
||||
...problems.map((problem) => ` ${problem}`),
|
||||
].join('\n'),
|
||||
);
|
||||
@@ -123,25 +144,32 @@ export function checkReleaseTarballExecBits(packageDir = 'dist') {
|
||||
}
|
||||
}
|
||||
|
||||
function expectedRipgrepEntries(packageDir) {
|
||||
const vendored = join(packageDir, 'vendor', 'ripgrep');
|
||||
let platforms;
|
||||
try {
|
||||
platforms = readdirSync(vendored, { withFileTypes: true })
|
||||
.filter((entry) => entry.isDirectory() && !entry.name.endsWith('-win32'))
|
||||
.map((entry) => entry.name)
|
||||
.sort();
|
||||
} catch (error) {
|
||||
function expectedSpawnedEntries(packageDir) {
|
||||
const expected = [];
|
||||
for (const spec of SPAWNED_VENDORED_BINARIES) {
|
||||
const root = join(packageDir, ...spec.root);
|
||||
let platforms;
|
||||
try {
|
||||
platforms = readdirSync(root, { withFileTypes: true });
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
`cannot read ${root}: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
for (const platform of platforms) {
|
||||
if (!platform.isDirectory() || spec.skip(platform.name)) continue;
|
||||
expected.push(
|
||||
`package/${spec.root.join('/')}/${platform.name}/${spec.leaf}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (expected.length === 0) {
|
||||
throw new Error(
|
||||
`cannot read ${vendored}: ${error instanceof Error ? error.message : String(error)}`,
|
||||
`no spawned vendored executables under ${join(packageDir, 'vendor')}; the gate refuses to pass vacuously`,
|
||||
);
|
||||
}
|
||||
if (platforms.length === 0) {
|
||||
throw new Error(
|
||||
`no unix vendored ripgrep under ${vendored}; the gate refuses to pass vacuously`,
|
||||
);
|
||||
}
|
||||
return platforms.map((platform) => `package/vendor/ripgrep/${platform}/rg`);
|
||||
expected.sort();
|
||||
return expected;
|
||||
}
|
||||
|
||||
function pack(packageDir, destination) {
|
||||
@@ -174,7 +202,7 @@ function main(argv) {
|
||||
try {
|
||||
const { tarball, entries } = checkReleaseTarballExecBits(packageDir);
|
||||
process.stdout.write(
|
||||
`ok: ${entries.length} vendored ripgrep entries keep 0111 in ${tarball} (${packageDir})\n`,
|
||||
`ok: ${entries.length} spawned vendored binaries keep 0111 in ${tarball} (${packageDir})\n`,
|
||||
);
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
|
||||
@@ -51,27 +51,32 @@ const heapCap = (value) => value?.match(/--max-old-space-size=(\d+)/)?.[1];
|
||||
|
||||
/**
|
||||
* Writes a throwaway package shaped like the generated CLI for the
|
||||
* vendored-ripgrep packing cases: `vendor/ripgrep/<platform>/rg` (or `rg.exe`
|
||||
* for a `-win32` platform) at the given mode, and the manifest entry that makes
|
||||
* a packer pick the tree up.
|
||||
* vendored-binary packing cases: `vendor/ripgrep/<platform>/rg` (or `rg.exe`
|
||||
* for a `-win32` platform) at the given modes, the manifest entry that makes
|
||||
* a packer pick the tree up, and optionally
|
||||
* `vendor/landlock-run/<platform>/qwen-landlock-run` binaries.
|
||||
*/
|
||||
function writeVendoredRipgrepFixture(directory, platforms) {
|
||||
function writeVendoredBinaryFixture(directory, platforms, landlock = {}) {
|
||||
writeFileSync(
|
||||
path.join(directory, 'package.json'),
|
||||
JSON.stringify({
|
||||
name: 'vendored-ripgrep-fixture',
|
||||
name: 'vendored-binary-fixture',
|
||||
version: '1.0.0',
|
||||
files: ['vendor'],
|
||||
}),
|
||||
);
|
||||
const binaries = [];
|
||||
for (const [platform, mode] of Object.entries(platforms)) {
|
||||
const binary = path.join(
|
||||
directory,
|
||||
'vendor',
|
||||
'ripgrep',
|
||||
platform,
|
||||
platform.endsWith('-win32') ? 'rg.exe' : 'rg',
|
||||
);
|
||||
binaries.push([
|
||||
`vendor/ripgrep/${platform}/${platform.endsWith('-win32') ? 'rg.exe' : 'rg'}`,
|
||||
mode,
|
||||
]);
|
||||
}
|
||||
for (const [platform, mode] of Object.entries(landlock)) {
|
||||
binaries.push([`vendor/landlock-run/${platform}/qwen-landlock-run`, mode]);
|
||||
}
|
||||
for (const [relative, mode] of binaries) {
|
||||
const binary = path.join(directory, relative);
|
||||
mkdirSync(path.dirname(binary), { recursive: true });
|
||||
writeFileSync(binary, '#!/bin/sh\n');
|
||||
chmodSync(binary, mode);
|
||||
@@ -1852,20 +1857,25 @@ describe('package scripts', () => {
|
||||
});
|
||||
|
||||
it.skipIf(!arePermissionBitsEnforced())(
|
||||
'npm pack keeps the exec bit on every vendored ripgrep in the CLI tarball',
|
||||
'npm pack keeps the exec bit on every spawned vendored binary in the CLI tarball',
|
||||
() => {
|
||||
// The archive is where the bit is lost (#12668), so assert on its
|
||||
// recorded modes rather than on the staged tree. A `-win32` directory
|
||||
// ships a 0644 `rg.exe` by design and the gate excludes it structurally.
|
||||
const fixture = mkdtempSync(path.join(tmpdir(), 'cli-rg-exec-bit-'));
|
||||
try {
|
||||
writeVendoredRipgrepFixture(fixture, {
|
||||
'x64-linux': 0o755,
|
||||
'arm64-darwin': 0o755,
|
||||
'x64-win32': 0o644,
|
||||
});
|
||||
writeVendoredBinaryFixture(
|
||||
fixture,
|
||||
{
|
||||
'x64-linux': 0o755,
|
||||
'arm64-darwin': 0o755,
|
||||
'x64-win32': 0o644,
|
||||
},
|
||||
{ 'x64-linux': 0o755 },
|
||||
);
|
||||
const { entries } = checkReleaseTarballExecBits(fixture);
|
||||
expect(entries.map(([name]) => name)).toEqual([
|
||||
'package/vendor/landlock-run/x64-linux/qwen-landlock-run',
|
||||
'package/vendor/ripgrep/arm64-darwin/rg',
|
||||
'package/vendor/ripgrep/x64-linux/rg',
|
||||
]);
|
||||
@@ -1879,24 +1889,50 @@ describe('package scripts', () => {
|
||||
);
|
||||
|
||||
it.skipIf(!arePermissionBitsEnforced())(
|
||||
'release tarball gate rejects a vendored ripgrep without an exec bit',
|
||||
'release tarball gate rejects a spawned vendored binary without an exec bit',
|
||||
() => {
|
||||
const fixture = mkdtempSync(path.join(tmpdir(), 'cli-rg-gate-'));
|
||||
try {
|
||||
writeVendoredRipgrepFixture(fixture, { 'x64-linux': 0o644 });
|
||||
writeVendoredBinaryFixture(
|
||||
fixture,
|
||||
{ 'x64-linux': 0o755 },
|
||||
{ 'x64-linux': 0o644 },
|
||||
);
|
||||
expect(() => checkReleaseTarballExecBits(fixture)).toThrow(
|
||||
/package\/vendor\/ripgrep\/x64-linux\/rg: 0644 lacks an exec bit/,
|
||||
/package\/vendor\/landlock-run\/x64-linux\/qwen-landlock-run: 0644 lacks an exec bit/,
|
||||
);
|
||||
|
||||
// A package whose vendor/ripgrep holds no unix platform must not pass
|
||||
// A platform directory whose binary was dropped during staging must
|
||||
// fail, not silently shrink the expected set.
|
||||
const shrunk = mkdtempSync(path.join(tmpdir(), 'cli-rg-missing-'));
|
||||
try {
|
||||
writeVendoredBinaryFixture(
|
||||
shrunk,
|
||||
{ 'arm64-linux': 0o755 },
|
||||
{ 'x64-linux': 0o755 },
|
||||
);
|
||||
mkdirSync(path.join(shrunk, 'vendor', 'ripgrep', 'x64-linux'), {
|
||||
recursive: true,
|
||||
});
|
||||
expect(() => checkReleaseTarballExecBits(shrunk)).toThrow(
|
||||
/package\/vendor\/ripgrep\/x64-linux\/rg: missing from the packed tarball/,
|
||||
);
|
||||
} finally {
|
||||
rmSync(shrunk, { force: true, recursive: true });
|
||||
}
|
||||
|
||||
// A package whose vendor trees ship no spawned binary must not pass
|
||||
// vacuously: the gate refuses before it ever packs.
|
||||
const empty = mkdtempSync(path.join(tmpdir(), 'cli-rg-none-'));
|
||||
try {
|
||||
mkdirSync(path.join(empty, 'vendor', 'ripgrep'), {
|
||||
recursive: true,
|
||||
});
|
||||
mkdirSync(path.join(empty, 'vendor', 'landlock-run'), {
|
||||
recursive: true,
|
||||
});
|
||||
expect(() => checkReleaseTarballExecBits(empty)).toThrow(
|
||||
/no unix vendored ripgrep under/,
|
||||
/no spawned vendored executables under/,
|
||||
);
|
||||
} finally {
|
||||
rmSync(empty, { force: true, recursive: true });
|
||||
|
||||
Reference in new issue
Block a user