Files
6dfd4e4747 fix(runtime): deliver bounded provider images and PDFs (#13606)
* fix(runtime): deliver bounded provider images and PDFs

Use invocation-local media capabilities for actual file reads so
managed providers can deliver images and PDFs without a model or
inherited Omni uploads.

- Enforce inline media and complete result budgets before settlement
- Preserve accepted media across execute, status, and cancel responses
- Propagate PDF cancellation and wait for child cleanup
- Cover capability isolation, reply loss, and Broker restart recovery
- Document provider media limits and refresh dependency notices

* fix(runtime): repair media review validation gates

Keep provider media validation usable in standard CI while retaining
explicit activation of the full MySQL media scenario.

- Gate media integration on the existing bundle-ready opt-in
- Align PDF call assertions with the optional cancellation signal
- Keep the maximum-result fixture writable without changing its budget
- Explain the complete-result gate and restore baseline notices

* fix(runtime): preserve media diagnostics and clean up cancelled reads

Keep admitted media observations intact while retaining useful Hook and
PDF diagnostics. Bound fixture cleanup so cancelled reads and stalled
processes do not leave unused files or consume test resources.

- Fit Hook reasons and errors against the actual response budget
- Handle PDF ranges beyond document length and retain size errors
- Preserve read failures and propagate PDF fetch cancellation
- Add regression witnesses and instrument restarted media workers
- Bound MySQL commands and include media scratch paths in cleanup
- Record Computer Use comparisons and the typed browser facade plan

* fix(runtime): bound PDF cancellation and strengthen media probes

Prevent cancelled PDF reads from waiting forever for an unresponsive
child and make provider-media verification reflect the live runtime.

- Escalate cancelled PDF children and bound the remaining close wait
- Track the restarted provisioner and share fixture scratch prefixes
- Preserve MySQL initialization diagnostics during cleanup
- Skip redundant page probes and verify delivered ranges and statuses
- Update media design guidance for diagnostics and cancellation

* fix(runtime): tolerate the expected media probe reply loss

Keep lost-execute response loss from poisoning recovery evidence
while preserving fatal errors for all other requests.

- Consume and count one drop for the matching execute call
- Adapt media tests to the borrowed descriptor source
- Assert oversized descriptor media is refused before reading

* fix(runtime): verify media reply loss at the worker probe

The media driver checked a worker route that its broker client never
traverses, so the required loss counter always remained zero.

- Observe the actual Java probe drop instead of a proxy-only counter
- Allow broker recovery while preserving the lost-start assertions
- Verify the real drop, file replacement and single-use evidence

* fix(core): remove duplicate filesystem service import

The automatic main merge retained both copies of the same import and
failed the TypeScript build with TS2300.

- Keep one upstream import for the existing filesystem test fixtures

* fix(runtime): align media probe with the managed schema

Keep the provider media acceptance probe usable after the main schema
rollout replaces legacy tables and logical storage ownership keys.

- Read execution and runtime evidence from canonical managed tables
- Resolve physical ownership through the tenant storage alias
- Cover prepared, settled and released evidence on the fresh schema

* style(runtime): align media probes with Java checks

Keep the media integration gate compatible with the Java rules
introduced on main.

- Sort imports and remove the obsolete charset import
- Brace existing conditions and loops without changing behavior

* fix(runtime): adapt media validation to server migration

Keep provider media validation runnable after the Java server move
and the managed schema's epoch-millisecond timestamp conversion.

- Point the MySQL launcher at the migrated server POM
- Bind a numeric last-active time in the canonical schema fixture

* style(runtime): fix inherited MCP fault test checkstyle

Keep the module verification gate passing after integrating the new
upstream MCP fault test.

- Remove an unused CompletableFuture import
- Normalize blank lines without changing test logic or assertions

* test(runtime): align media probe with execution membership

Keep the media acceptance probe compatible with shared runtime
execution membership after integrating main.

- Check the original binding, generation and physical storage member
- Reject mismatched or exclusive members and leaked released membership

* docs(runtime): add the English provider media design and drop unrelated CUA docs

Translate the provider media design so the bilingual pair linked from the
Chinese document resolves.

Drop two CUA planning documents that were carried by the media commit but
describe unrelated browser work.

The Java/MySQL provider-media scenario passes locally against an isolated
MySQL instance: 12 cases, 24 model requests, one dropped start, every
reference dispatched once and released after broker restart.

* test(runtime): adapt the provider-media M4 driver to the broker client

main replaced the high-level broker provider with
ManagedRuntimeBrokerClient. Add a test-only adapter that exposes the
ManagedToolV2Client surface the M4 driver uses, backed by the new client,
and update the driver import and status narrowing.

Verified: integration typecheck, ESLint, and the Java/MySQL M4 scenario
(12 cases, 24 model requests, one dropped start, all references dispatched
once and released after Broker restart).

* style(tests): add the broker media adapter final newline

Restore Prettier compliance for the provider-media integration helper
so the formatting lane can pass.

---------

Co-authored-by: wangheyang <[email protected]>
Co-authored-by: wenshao <[email protected]>
2026-10-25 18:49:56 +08:00

326 lines
11 KiB
TypeScript

/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { spawn } from 'node:child_process';
import { existsSync } from 'node:fs';
import {
mkdir,
mkdtemp,
readFile,
readdir,
rm,
utimes,
writeFile,
} from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import {
ACP_HOME_PREFIX,
HOSTED_HOME_PREFIX,
HOSTED_STORE_PREFIX,
} from './scratch-dir.js';
import {
PROVIDER_MEDIA_MYSQL_PREFIX,
PROVIDER_MEDIA_HARNESS_PREFIX,
PROVIDER_MEDIA_FIXTURES_PREFIX,
PROVIDER_MEDIA_PROBE_PREFIX,
} from './helpers/provider-media-process-cleanup.mjs';
// The env keys globalSetup's setup() writes, saved so a case can restore the
// suite-wide values after re-importing the module and running its lifecycle.
const SETUP_ENV_KEYS = [
'INTEGRATION_TEST_FILE_DIR',
'QWEN_CODE_INTEGRATION_TEST',
'TELEMETRY_LOG_FILE',
'E2E_TEST_FILE_DIR',
'TEST_CLI_PATH',
'VERBOSE',
'KEEP_OUTPUT',
'QWEN_CODE_MODELS_DEV_REFRESH',
] as const;
describe('globalSetup memory-file save/restore', () => {
let qwenHome: string;
let savedEnv: Map<string, string | undefined>;
beforeEach(async () => {
qwenHome = await mkdtemp(join(tmpdir(), 'qwen-globalsetup-test-'));
savedEnv = new Map(
[...SETUP_ENV_KEYS, 'QWEN_HOME'].map((key) => [key, process.env[key]]),
);
process.env['QWEN_HOME'] = qwenHome;
// Let teardown remove the run directories this case creates.
process.env['KEEP_OUTPUT'] = 'false';
});
afterEach(async () => {
for (const [key, value] of savedEnv) {
if (value === undefined) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
vi.resetModules();
await rm(qwenHome, { recursive: true, force: true });
});
// memoryFilePath is captured at module import time, so point QWEN_HOME at
// the scratch dir BEFORE a fresh import of the module.
async function loadGlobalSetup() {
vi.resetModules();
return import('./globalSetup.js');
}
it('restores the saved memory file after the run', async () => {
await writeFile(join(qwenHome, 'QWEN.md'), 'original content', 'utf-8');
const { setup, teardown } = await loadGlobalSetup();
await setup();
await writeFile(join(qwenHome, 'QWEN.md'), 'mutated by tests', 'utf-8');
await expect(teardown()).resolves.toBeUndefined();
await expect(readFile(join(qwenHome, 'QWEN.md'), 'utf-8')).resolves.toBe(
'original content',
);
});
it('does not exit an all-green run red when the restore cannot write', async () => {
// The persistent pool runners can carry a readable-but-unwritable
// QWEN.md left behind by a privileged job; before #10325 the teardown
// restore threw on it and exited every all-green E2E run on that host
// red with no failing test. Swap the file for a directory after setup()
// read it — the write then fails regardless of privilege, since root
// bypasses permission bits.
await writeFile(join(qwenHome, 'QWEN.md'), 'original content', 'utf-8');
const { setup, teardown } = await loadGlobalSetup();
await setup();
await rm(join(qwenHome, 'QWEN.md'), { force: true });
await mkdir(join(qwenHome, 'QWEN.md'));
await expect(teardown()).resolves.toBeUndefined();
});
});
describe('globalSetup hermetic qwen home', () => {
let tmpRoot: string;
let savedEnv: Map<string, string | undefined>;
beforeEach(async () => {
tmpRoot = await mkdtemp(join(tmpdir(), 'qwen-globalsetup-tmp-'));
savedEnv = new Map(
[...SETUP_ENV_KEYS, 'QWEN_HOME', 'TMPDIR'].map((key) => [
key,
process.env[key],
]),
);
// The scratch home is only created when nothing has pinned QWEN_HOME, and
// it lands in the OS temp dir — redirect that so the case owns what the
// module picks. Both are read at import time, so they must be set first.
delete process.env['QWEN_HOME'];
process.env['QWEN_CODE_MODELS_DEV_REFRESH'] = 'on';
process.env['TMPDIR'] = tmpRoot;
process.env['KEEP_OUTPUT'] = 'false';
});
afterEach(async () => {
for (const [key, value] of savedEnv) {
if (value === undefined) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
vi.resetModules();
await rm(tmpRoot, { recursive: true, force: true });
});
async function loadGlobalSetup() {
vi.resetModules();
return import('./globalSetup.js');
}
it('points the run at its own qwen home', async () => {
const { setup, teardown } = await loadGlobalSetup();
await setup();
const home = process.env['QWEN_HOME'];
expect(home?.startsWith(tmpRoot)).toBe(true);
expect(process.env['QWEN_CODE_MODELS_DEV_REFRESH']).toBe('off');
await expect(teardown()).resolves.toBeUndefined();
expect(existsSync(home!)).toBe(false);
});
it.each([
['per-agent ACP home', ACP_HOME_PREFIX],
['Hosted process root', HOSTED_HOME_PREFIX],
['Hosted Session Store root', HOSTED_STORE_PREFIX],
['provider media MySQL root', PROVIDER_MEDIA_MYSQL_PREFIX],
['provider media harness root', PROVIDER_MEDIA_HARNESS_PREFIX],
['provider media fixture root', PROVIDER_MEDIA_FIXTURES_PREFIX],
['provider media probe root', PROVIDER_MEDIA_PROBE_PREFIX],
])(
'sweeps a stale %s a torn-down run left behind',
async (_label, prefix) => {
// cli/acp-integration.test.ts and the Hosted process helpers create their
// scratch roots directly under the OS temp dir, and a worker torn down
// mid-test never runs the cleanup that removes them. A root is reclaimed
// only while its prefix stays nested under the sweeper's — each prefix is
// the exact string its creation site builds the name from.
const leakedHome = join(tmpRoot, `${prefix}leaked`);
await mkdir(leakedHome, { recursive: true });
const leakedAt = new Date(Date.now() - 48 * 60 * 60 * 1000);
await utimes(leakedHome, leakedAt, leakedAt);
const { setup, teardown } = await loadGlobalSetup();
await setup();
expect(existsSync(leakedHome)).toBe(false);
await teardown();
},
);
it('does not exit an all-green run red when the scratch home cannot be removed', async () => {
// A CLI child that outlives its test keeps writing under `debug/`, so the
// removal walk reaches a directory that refills before the rmdir and
// throws ENOTEMPTY. That is how this cleanup first exited an all-green
// E2E run red, the same way the memory-file restore did in #10325. Stand
// in for that child with a writer that keeps the directory refilling —
// unlike a permission trick, it does not depend on not running as root.
const { setup, teardown } = await loadGlobalSetup();
await setup();
const debugDir = join(process.env['QWEN_HOME']!, 'debug');
await mkdir(debugDir, { recursive: true });
const writer = spawn(
process.execPath,
[
'-e',
// A tight loop, not a timer: the removal walk only fails when a file
// appears between its readdir and its rmdir, and a millisecond timer
// is far too slow to land inside that window. Self-limiting, so a
// child that outlives the kill below cannot spin indefinitely.
`const fs = require('node:fs');
const dir = ${JSON.stringify(debugDir)};
const stopAt = Date.now() + 30_000;
while (Date.now() < stopAt) {
try {
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(dir + '/' + process.hrtime.bigint() + '.log', 'x');
} catch {}
}`,
],
{ stdio: 'ignore' },
);
try {
// Node takes tens of milliseconds to boot, and the removal walk finishes
// in about one — without waiting for the writer to actually be producing,
// teardown would clear an idle directory and the case would pass against
// the very bug it exists to catch.
const deadline = Date.now() + 10_000;
while ((await readdir(debugDir)).length < 50) {
if (Date.now() > deadline) {
throw new Error('writer never started refilling the debug directory');
}
await new Promise((resolve) => setTimeout(resolve, 20));
}
await expect(teardown()).resolves.toBeUndefined();
} finally {
writer.kill('SIGKILL');
}
});
});
describe('globalSetup host config carry-over', () => {
let tmpRoot: string;
let homeDir: string;
let savedEnv: Map<string, string | undefined>;
beforeEach(async () => {
tmpRoot = await mkdtemp(join(tmpdir(), 'qwen-globalsetup-ci-'));
homeDir = await mkdtemp(join(tmpdir(), 'qwen-globalsetup-home-'));
savedEnv = new Map(
[...SETUP_ENV_KEYS, 'QWEN_HOME', 'HOME', 'CI', 'TMPDIR'].map((key) => [
key,
process.env[key],
]),
);
delete process.env['QWEN_HOME'];
process.env['HOME'] = homeDir;
process.env['TMPDIR'] = tmpRoot;
process.env['KEEP_OUTPUT'] = 'false';
process.env['QWEN_CODE_MODELS_DEV_REFRESH'] = 'on';
await mkdir(join(homeDir, '.qwen'), { recursive: true });
await writeFile(
join(homeDir, '.qwen', 'settings.json'),
JSON.stringify({ security: { auth: { selectedType: 'qwen-oauth' } } }),
'utf-8',
);
await writeFile(
join(homeDir, '.qwen', 'oauth_creds.json'),
'{"access_token":"host"}',
'utf-8',
);
});
afterEach(async () => {
for (const [key, value] of savedEnv) {
if (value === undefined) {
delete process.env[key];
} else {
process.env[key] = value;
}
}
vi.resetModules();
await rm(tmpRoot, { recursive: true, force: true });
await rm(homeDir, { recursive: true, force: true });
});
async function loadGlobalSetup() {
vi.resetModules();
return import('./globalSetup.js');
}
it('does not carry host auth state into the hermetic home on CI', async () => {
process.env['CI'] = 'true';
const { setup, teardown } = await loadGlobalSetup();
await setup();
const home = process.env['QWEN_HOME']!;
await expect(
readFile(join(home, 'settings.json'), 'utf-8'),
).rejects.toMatchObject({ code: 'ENOENT' });
await expect(
readFile(join(home, 'oauth_creds.json'), 'utf-8'),
).rejects.toMatchObject({ code: 'ENOENT' });
await teardown();
});
it('carries host config into the hermetic home outside CI', async () => {
delete process.env['CI'];
const { setup, teardown } = await loadGlobalSetup();
await setup();
const home = process.env['QWEN_HOME']!;
await expect(
readFile(join(home, 'settings.json'), 'utf-8'),
).resolves.toContain('qwen-oauth');
await expect(
readFile(join(home, 'oauth_creds.json'), 'utf-8'),
).resolves.toContain('host');
await teardown();
});
});