Files
2bda063cc9 test(integration): isolate the QWEN_HOME-unset case from the real $HOME (#10900) (#10920)
* test(integration): isolate the QWEN_HOME-unset case from the real $HOME (#10900)

Test 1d deleted QWEN_HOME to exercise the default ~/.qwen fallback, which resolves through os.homedir() to the runner's real home. On the shared ECS release pool that home can retain root-owned files from an earlier containerized job, so ~/.qwen was unwritable and the spawned CLI crashed at startup with EACCES, failing integration_none and blocking the v0.23.0 release.

globalSetup already redirects QWEN_HOME to a hermetic temp dir for exactly this pool-contamination reason (#10085, #10325); test 1d was the one case that undid that isolation. Point HOME at a writable temp dir under the test root so the unset-QWEN_HOME default still resolves through os.homedir() but lands hermetically. The fallback routing under test is unchanged.

* test(integration): let 1d pin the routing instead of the host's auth (#10900)

Correcting the record: the release run linked from #10900 did not fail this case. Its only red was an interactive /quit test, this file passed 7/7 including 1d, and the log carries no EACCES; the rerun went green with no code change. The redirect the previous commit added is prophylactic hardening against a latent exposure, not a fix for an observed failure. The exposure is real — 1d deletes QWEN_HOME and so undoes globalSetup's hermetic isolation, leaving it the only case here whose outcome depends on the real $HOME being writable — and so is the mechanism: with an unwritable $HOME the spawned CLI dies during startup, before any config-dir routing, with EACCES on mkdir '<home>/.qwen'.

Pointing HOME at an empty temp dir also severed the ambient file-based credentials globalSetup.carryOverHostConfig deliberately preserves. 1d then failed on any host whose model auth lives in ~/.qwen/settings.json rather than env vars: the child resolved its global dir to the empty redirect, found no credentials, and exited 1 with "No auth type is selected". Every CI lane injects OPENAI_* as process env, so no lane could see it.

Rather than copy credentials into a routing test to keep an assertion alive, drop the assertion. "The prompt run succeeds" measured the ambient host, not the QWEN_HOME fallback. Assert what 1a-1c already assert — that config.initialize() materialized installation_id in the resolved global dir — now under the redirected HOME. That pins the redirect itself: removing the HOME assignment turns it red on any host, and under an unwritable $HOME the startup crash leaves no installation_id to find.

* test(integration): make 1d's run succeed without borrowing the host's auth (#10900)

The previous commit kept 1d hermetic by dropping its success requirement, and in doing so dropped the suite's only end-to-end guard of the unset-QWEN_HOME path. globalSetup pins QWEN_HOME for every other spawned CLI, so nothing else exercises the os.homedir() fallback; the bare catch then swallowed every failure after config.initialize(), while the surviving existsSync still found the installation_id the crashing run had already written. Reproduced: with ambient auth scrubbed the child exits 1 on "No auth type is selected" and 1d reports green in 790ms instead of 7.2s. The case named "CLI functions normally" no longer required the CLI to function.

Hermeticity and the success requirement were never in conflict — only the way round 2 reached hermeticity was. cli/list_directory.test.ts already runs a prompt to completion with no host auth: a fake OpenAI server plus explicit CLI flags, which outrank ambient auth and any settings.json so the run cannot be routed to a real endpoint. Do that here, keeping the HOME redirect so the run still never depends on the real $HOME being writable.

Both guards are witnessed by mutation. Pointing --openai-base-url at an unreachable port fails the run after installation_id is written and turns 1d red, where the assertion alone would have stayed green. Removing the HOME stub turns 1d red on the location assertion. With all ambient auth scrubbed the case now passes on a real round trip, so it no longer measures the host.

* test(integration): make 1d's success requirement pin the model round-trip (#10900)

The previous commit restored 1d's success requirement as `expect(result).toBeTruthy()`, which pins nothing beyond exit 0 — and exit 0 is already enforced by `TestRig.run`, which rejects on any non-zero code. On exit 0 the rig appends a `StdErr:` section carrying the captured stderr to the resolved output, so any run that emits anything on either stream yields a truthy result. A regression that lets a prompt run exit 0 without ever reaching the model — an early-return bootstrap path that warns on stderr and exits cleanly, or a swallowed model-invocation failure — passes that guard with zero requests on the fake server the case already starts. The requirement the doc block calls load-bearing reported green on exactly the class it claimed to catch.

Assert the round-trip instead: `expect(result).toContain('Hello!')`, the fake server's canned content and the suite's own convention (`cli/run_shell_command.test.ts` asserts `expect(result).toContain(<model output>)` on a `rig.run` result, in lanes that include the container sandbox). Same line count, strictly stronger — a truthy result is implied by containing the model's output. Not adding the request-count assertion the review also proposed: `'Hello!'` cannot reach the output without the round-trip, so a second guard on the same fact would be redundant.

Witnessed by mutation, with a stub CLI standing in for the bundle — it writes installation_id under the redirected HOME, warns on stderr, and exits 0 without an HTTP call. Against the old guard 1d stays green in 206ms; against the new one it fails. Both arms reverted afterwards and the restore verified byte-exact.

Also corrects the doc block, which claimed this is the suite's only end-to-end guard of the unset path. It is not on the container lanes: `packages/cli/src/serve/sandbox.ts` pushes `--env QWEN_HOME=<container path>` unconditionally, so the containerized CLI resolves the global dir from that and runs the QWEN_HOME-set path, and the case goes green there while only the host-side launcher has exercised the unset one. The comment now says so, and says "must complete the round-trip" rather than "is required to exit 0".

* test(integration): isolate Windows home directory

---------

Co-authored-by: Qwen Autofix <[email protected]>
Co-authored-by: yiliang114 <[email protected]>
2026-09-11 15:05:08 +00:00

433 lines
15 KiB
TypeScript

/**
* @license
* Copyright 2025 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
/**
* E2E integration tests for the QWEN_HOME environment variable.
*
* These tests verify that when QWEN_HOME is set, all global config files
* (installation_id, settings.json, memory.md, etc.) are routed to the
* custom directory instead of ~/.qwen/.
*
* Based on the test plan at:
* .claude/docs/PLAN-qwen-config-dir-e2e-tests.md
*
* NOTE: Most tests require a full prompt run (config.initialize() must run to
* write installation_id). Only scenario 2b can use --help because settings
* migration runs before arg parsing.
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import {
CONTAINER_SANDBOX_NO_PROXY,
fakeServerHostOptions,
IS_CONTAINER_SANDBOX,
TestRig,
} from '../test-helper.js';
import { startFakeOpenAIServer } from '../fake-openai-server.js';
import {
existsSync,
mkdirSync,
writeFileSync,
readdirSync,
readFileSync,
} from 'node:fs';
import { join, resolve } from 'node:path';
// Keep in sync with SETTINGS_VERSION in packages/cli/src/config/settings.ts.
const CURRENT_SETTINGS_VERSION = 4;
// The fake server is reached over loopback, so a runner-level proxy must not
// be allowed to intercept it.
const NO_PROXY = IS_CONTAINER_SANDBOX
? CONTAINER_SANDBOX_NO_PROXY
: '127.0.0.1,localhost';
// Helper: list files under a directory recursively, returning relative paths
function listFilesRecursive(dir: string, base = dir): string[] {
if (!existsSync(dir)) return [];
const results: string[] = [];
for (const entry of readdirSync(dir, { withFileTypes: true })) {
const full = join(dir, entry.name);
if (entry.isDirectory()) {
results.push(...listFilesRecursive(full, base));
} else {
results.push(full.slice(base.length + 1));
}
}
return results;
}
describe('QWEN_HOME environment variable', () => {
let rig: TestRig;
let customConfigDir: string;
beforeEach(() => {
rig = new TestRig();
});
afterEach(async () => {
// Always clean up env vars regardless of test outcome
vi.unstubAllEnvs();
delete process.env['QWEN_HOME'];
delete process.env['QWEN_RUNTIME_DIR'];
delete process.env['QWEN_DEBUG_LOG_FILE'];
await rig.cleanup();
});
// -------------------------------------------------------------------------
// Group 1: Basic environment variable behaviour
// -------------------------------------------------------------------------
describe('Group 1: Basic env var behaviour', () => {
/**
* 1a. CLI uses custom config dir for settings and initialization.
*
* A full prompt run is required because installation_id is only written
* during config.initialize() → logStartSession() → getInstallationId().
* --help exits before that point.
*/
it('1a: installation_id is written inside QWEN_HOME, not ~/.qwen', async () => {
await rig.setup('qwen-home-1a-installation-id');
customConfigDir = join(rig.testDir!, 'custom-config');
mkdirSync(customConfigDir, { recursive: true });
process.env['QWEN_HOME'] = customConfigDir;
// A full prompt run is needed to trigger config.initialize()
try {
await rig.run('say hello');
} catch {
// May fail without a valid API key; that is acceptable — we only
// need config.initialize() to run far enough to create installation_id
}
const installationIdPath = join(customConfigDir, 'installation_id');
expect(
existsSync(installationIdPath),
`Expected installation_id at ${installationIdPath}`,
).toBe(true);
});
/**
* 1b. CLI creates the config dir structure when the path does not yet exist.
*/
it('1b: config dir is created when it does not exist', async () => {
await rig.setup('qwen-home-1b-dir-creation');
// Point to a path that does NOT exist yet
customConfigDir = join(rig.testDir!, 'nonexistent-config');
expect(existsSync(customConfigDir)).toBe(false);
process.env['QWEN_HOME'] = customConfigDir;
try {
await rig.run('say hello');
} catch {
// May fail without a valid API key — tolerate the error
}
// The directory must have been created
expect(
existsSync(customConfigDir),
`Expected ${customConfigDir} to be created`,
).toBe(true);
// installation_id signals that config.initialize() ran inside it
const installationIdPath = join(customConfigDir, 'installation_id');
expect(
existsSync(installationIdPath),
`Expected installation_id inside newly created dir`,
).toBe(true);
});
/**
* 1c. Relative path is resolved correctly.
*
* TestRig sets cwd to testDir when spawning the child process, so a
* relative path like "./custom-qwen" resolves to
* <testDir>/custom-qwen inside the subprocess.
*/
it('1c: relative QWEN_HOME path is resolved against subprocess cwd', async () => {
await rig.setup('qwen-home-1c-relative-path');
const relativePath = './custom-qwen';
process.env['QWEN_HOME'] = relativePath;
try {
await rig.run('say hello');
} catch {
// May fail without a valid API key — tolerate the error
}
// Resolve the expected absolute path the same way the subprocess does
const expectedAbsPath = resolve(rig.testDir!, 'custom-qwen');
const installationIdPath = join(expectedAbsPath, 'installation_id');
expect(
existsSync(installationIdPath),
`Expected installation_id at resolved path ${installationIdPath}`,
).toBe(true);
});
/**
* 1d. Default behaviour is preserved when QWEN_HOME is unset.
*
* HOME is redirected to a writable temp dir so the unset-QWEN_HOME default
* (~/.qwen, resolved via os.homedir()) lands hermetically. This is the one
* case in the file that otherwise undoes globalSetup's isolation and needs
* the real $HOME to be writable — the exposure #10085 and #10325 isolated
* QWEN_HOME against. It is fatal before any routing happens: with an
* unwritable $HOME the CLI dies during startup with `EACCES: permission
* denied, mkdir '<home>/.qwen'`.
*
* The redirected home carries no credentials, so the run drives the fake
* OpenAI server through explicit CLI flags, and it must complete the
* round-trip rather than tolerate failure like its siblings — tolerating
* one would report green on a regression that kills the run after
* config.initialize(). This is the only case that runs the unset path; in a
* container sandbox that coverage stops at the host-side launcher, because
* the launcher re-pins QWEN_HOME for the containerized CLI.
*/
it('1d: CLI functions normally when QWEN_HOME is not set', async () => {
await rig.setup('qwen-home-1d-default-behaviour');
// Explicitly ensure QWEN_HOME is absent for this test
delete process.env['QWEN_HOME'];
// Resolve the default ~/.qwen into a writable dir so the run never
// depends on the real $HOME being writable (see the note above).
const homeDir = join(rig.testDir!, 'home');
mkdirSync(homeDir, { recursive: true });
vi.stubEnv('HOME', homeDir);
vi.stubEnv('USERPROFILE', homeDir);
vi.stubEnv('NO_PROXY', NO_PROXY);
vi.stubEnv('no_proxy', NO_PROXY);
const fakeServer = await startFakeOpenAIServer(
() => ({ content: 'Hello!' }),
fakeServerHostOptions(),
);
try {
// Explicit CLI flags outrank ambient auth and any settings.json, so
// the run cannot be routed to a real model endpoint.
const result = await rig.run(
'say hello',
'--auth-type',
'openai',
'--model',
'fake-model',
'--openai-base-url',
fakeServer.baseUrl,
'--openai-api-key',
'fake-key',
);
expect(result).toContain('Hello!');
} finally {
await fakeServer.close();
}
// The default global dir must have materialized under the redirected
// HOME rather than the real one.
const installationIdPath = join(homeDir, '.qwen', 'installation_id');
expect(
existsSync(installationIdPath),
`Expected the default global dir at ${installationIdPath}`,
).toBe(true);
});
});
// -------------------------------------------------------------------------
// Group 2: Feature-specific config dir routing
// -------------------------------------------------------------------------
describe('Group 2: Feature-specific routing', () => {
/**
* 2b. Settings migration runs against the custom config dir.
*
* `extensions list` is sufficient here because it is a yargs subcommand
* that runs through `main()` and reaches `loadSettings()` (which triggers
* migration), without needing an API key or interactive session.
* (Note: `--help` cannot be used — yargs intercepts it and exits the
* process before `loadSettings()` runs.)
*/
it('2b: settings migration runs in QWEN_HOME dir', async () => {
await rig.setup('qwen-home-2b-settings-migration');
customConfigDir = join(rig.testDir!, 'migration-config');
mkdirSync(customConfigDir, { recursive: true });
process.env['QWEN_HOME'] = customConfigDir;
// Write a V1-format settings file into the custom config dir
const v1Settings = {
$version: 1,
theme: 'dark',
autoAccept: true,
};
writeFileSync(
join(customConfigDir, 'settings.json'),
JSON.stringify(v1Settings, null, 2),
);
// `extensions list` triggers loadSettings() (migration) without needing
// an API key.
try {
await rig.runCommand(['extensions', 'list']);
} catch {
// Tolerate non-zero exit; migration runs regardless.
}
// Read migrated settings
const migratedRaw = readFileSync(
join(customConfigDir, 'settings.json'),
'utf-8',
);
const migrated = JSON.parse(migratedRaw) as Record<string, unknown>;
expect(migrated['$version']).toBe(CURRENT_SETTINGS_VERSION);
});
});
// -------------------------------------------------------------------------
// Group 3: Isolation — project-level .qwen/ is NOT affected
// -------------------------------------------------------------------------
describe('Group 3: Project-level isolation', () => {
/**
* 3a. Project-level workspace settings work independently of QWEN_HOME.
*
* We put already-current settings in QWEN_HOME and V1 settings in the
* workspace .qwen/settings.json. Running `extensions list` triggers
* loadSettings() (migration). If the CLI is correctly reading workspace
* settings from <testDir>/.qwen/, the workspace settings.json will be
* migrated. If it mistakenly read from QWEN_HOME, the workspace file
* would be untouched.
*
* `extensions list` runs through `main()` and reaches `loadSettings()`
* (which triggers migration) without needing an API key.
*/
it('3a: workspace settings are read from project .qwen/, not from QWEN_HOME', async () => {
await rig.setup('qwen-home-3a-isolation');
customConfigDir = join(rig.testDir!, 'global-config');
mkdirSync(customConfigDir, { recursive: true });
process.env['QWEN_HOME'] = customConfigDir;
// Seed QWEN_HOME with the current schema version so it shouldn't migrate.
writeFileSync(
join(customConfigDir, 'settings.json'),
JSON.stringify(
{
$version: CURRENT_SETTINGS_VERSION,
customKey: 'in-global-dir',
},
null,
2,
),
);
// Overwrite the workspace settings.json with V1 format so migration is observable
const workspaceSettingsPath = join(
rig.testDir!,
'.qwen',
'settings.json',
);
writeFileSync(
workspaceSettingsPath,
JSON.stringify(
{
$version: 1,
theme: 'dark',
autoAccept: false,
customWorkspaceKey: 'workspace-value',
},
null,
2,
),
);
// `extensions list` triggers loadSettings() (including migration)
// without needing an API key.
try {
await rig.runCommand(['extensions', 'list']);
} catch {
// Tolerate non-zero exit; migration runs regardless.
}
// The workspace settings.json must have been migrated to the current
// settings version, proving the CLI read it from the workspace dir, not
// from QWEN_HOME.
const workspaceRaw = readFileSync(workspaceSettingsPath, 'utf-8');
const workspaceSettings = JSON.parse(workspaceRaw) as Record<
string,
unknown
>;
expect(workspaceSettings['$version']).toBe(CURRENT_SETTINGS_VERSION);
expect(workspaceSettings['customWorkspaceKey']).toBe('workspace-value');
// The QWEN_HOME settings.json must be unchanged (still at the version we wrote)
const globalRaw = readFileSync(
join(customConfigDir, 'settings.json'),
'utf-8',
);
const globalSettings = JSON.parse(globalRaw) as Record<string, unknown>;
expect(globalSettings['customKey']).toBe('in-global-dir');
});
});
// -------------------------------------------------------------------------
// Group 4: Interaction with QWEN_RUNTIME_DIR
// -------------------------------------------------------------------------
describe('Group 4: Interaction with QWEN_RUNTIME_DIR', () => {
/**
* 4a. QWEN_HOME and QWEN_RUNTIME_DIR can be set independently.
*
* Config files (installation_id) go to QWEN_HOME.
* Runtime files (debug logs) go to QWEN_RUNTIME_DIR.
*/
it('4a: config files land in QWEN_HOME and runtime files land in QWEN_RUNTIME_DIR', async () => {
await rig.setup('qwen-home-4a-independence');
customConfigDir = join(rig.testDir!, 'config-dir');
const runtimeDir = join(rig.testDir!, 'runtime-dir');
mkdirSync(customConfigDir, { recursive: true });
mkdirSync(runtimeDir, { recursive: true });
process.env['QWEN_HOME'] = customConfigDir;
process.env['QWEN_RUNTIME_DIR'] = runtimeDir;
process.env['QWEN_DEBUG_LOG_FILE'] = '1';
try {
await rig.run('say hello');
} catch {
// May fail without a valid API key — tolerate the error
}
// Config file must be inside QWEN_HOME
const installationIdPath = join(customConfigDir, 'installation_id');
expect(
existsSync(installationIdPath),
`Expected installation_id in QWEN_HOME at ${installationIdPath}`,
).toBe(true);
// Debug logs must be inside QWEN_RUNTIME_DIR (under debug/)
const debugDir = join(runtimeDir, 'debug');
const debugFiles = listFilesRecursive(debugDir);
expect(
debugFiles.length,
`Expected debug log files in ${debugDir}`,
).toBeGreaterThan(0);
// installation_id must NOT appear in the runtime dir
const runtimeInstallationId = join(runtimeDir, 'installation_id');
expect(
existsSync(runtimeInstallationId),
`Did NOT expect installation_id inside QWEN_RUNTIME_DIR`,
).toBe(false);
});
});
});