feat(managed-agent): support Workspace file uploads (#15586)

This commit is contained in:
jinye authored and GitHub committed 2026-10-26 01:10:18 +00:00
1 parent 338f13366c
commit 06d82d786a
44 files changed
+3613 -69

No files matched your search

@@ -0,0 +1,154 @@
# Managed Workspace uploads
[English](2026-10-26-managed-workspace-upload.md) | [简体中文](2026-10-26-managed-workspace-upload.zh-CN.md)
## Problem and scope
Managed WebShell can attach text and images to a message and snapshot small
Workspace text files. It cannot upload browser files into persistent Workspace
storage. The existing file service resolves registered LOCAL and SSH_LOCAL
storage through a bounded CLI bootstrap exchange; CSI file access is unsupported.
Implement uploads for these existing storage kinds, including empty files and
binary files, with an 8 MiB per-file limit. Reuse the composer picker, drop
choice, progress/error rows, cancellation and session-switch fencing. Uploads
create files in the selected working directory or a browsed existing directory.
An occupied destination returns a conflict and is never overwritten. Folder
uploads, automatic directory creation, CSI, and an object-storage service are
outside this change.
Uploads require no outstanding execution-storage members. Ordinary settled
Turns release these members; a persistent MCP Runtime may still retain one and
therefore return `workspace_busy`. This change does not weaken that ownership
boundary to treat an apparently idle Session as proof of a stopped tool process.
## Ownership and API
The new WebShell `POST /api/agent/web-shell/v1/workspaces/files/upload` route is
persisted-Workspace scoped. Query parameters are `workspaceId`, `cwdRelative`,
`path`, `expectedScopeRevision` and UUID `uploadId`; the request body is
`application/octet-stream`. It requires a trusted actor with OPERATOR or OWNER
access and enabled Workspace file access. The server validates the registered
storage identity and revision, and never resolves a daemon primary Workspace.
The binary body is read with a hard limit. A matching repeat of an upload ID
reconciles its original command; a changed request conflicts.
The response includes `scopeRevision`, `uploadId`, `path`, `sizeBytes` and
`sha256`. A Workspace file query advertises optional `maxUploadBytes` only when
the selected CLI supports the independent `workspaceFileUpload: 1` handshake.
Old deployments retain text references and keep upload disabled. OpenAPI, its
generated client types, surface registry and capability parity advance together.
## Durable admission and recovery
A short SQL transaction admits a `WORKSPACE_UPLOAD` operation in the existing
operation ledger and takes the physical storage owner hold. It freezes the
actor, request digest, ContextBinding, storage identity and original bootstrap
target; binary bytes are not stored in SQL. Existing tool acquisition, storage
fencing and retirement must respect this hold. No subprocess or SSH exchange
runs inside a SQL transaction. A completed or definitively failed command
releases only its own hold; a timeout, lost response or control-plane restart
keeps it until the original command is reconciled.
The runtime keeps an owner-private receipt under its configured state directory,
keyed by `uploadId`. It atomically claims the operation before writing, records
the original PID, and publishes a terminal receipt only after all file mutation
and temporary-file cleanup has finished. Reconciliation never executes an upload.
It returns a terminal receipt, conservatively waits for a live original process,
or, after definite original-process absence, checks the original destination's
length and digest. If no claim exists, reconciliation atomically publishes a
failure tombstone so a delayed original command cannot start. PID reuse may
delay reconciliation; it never authorizes replay. No replacement Runtime,
repeated mutation, or rollback deletion is used to resolve an unknown outcome.
## Runtime filesystem boundary
The existing bootstrap verifies allowed roots, storage identity and working
directory. Upload requests use protocol version 1 with operations `upload` and
`upload-status`, `uploadId`, relative `path`, `sizeBytes`, `sha256`, and base64
`data` for upload only. Responses use `state: completed | failed | pending`,
with terminal `code` on failure. Transport input is bounded separately from the
64 KiB response and ordinary bootstrap limits.
Reuse normalized relative-path validation and directory identity checks. Reject
symlinks, traversal, reserved managed storage paths and non-regular files. Write
an owner-only temporary file, sync it, check the directory anchors again, and
publish with an atomic no-clobber hard link. Never overwrite an occupied path.
Detected path or storage identity changes fail closed. These checks retain the
existing Node filesystem race limitations; they do not claim an `openat` sandbox.
## Composer behavior
A Managed upload target is supplied explicitly to the shared composer, without
enabling unrelated daemon features. The target identity includes provider,
Workspace, working directory, revision and session. An upload completion from a
previous target cannot append to the current draft. The existing customization
switch can still disable uploads.
A successful upload inserts a scoped path reference. On send, the Managed
adapter checks its scope and includes the relative path in model input; binary
bytes are not turned into a text attachment or read through the small-text
snapshot API. Existing text references keep their immutable snapshot behavior.
Cancelling a browser request does not promise deletion of an already committed
file; uncertain outcomes reconcile by the original upload ID.
## Validation and acceptance
- Baseline: the installed CLI lacks the upload handshake and the Managed menu
cannot upload to Workspace storage.
- Runtime: exact binary/empty bytes, 8 MiB boundaries, existing files, nested and
Unicode paths, symlinks/traversal, changed storage identity, concurrent/repeated
IDs, terminal receipts and crash/late-start reconciliation.
- Control plane: actor/tenant isolation, roles, stale scope, disabled/unsupported
storage, durable holds, concurrent tools/fencing, request mismatch, unknown
results, original-target recovery and response byte/digest validation.
- UI: enabled capability path, old capability disabled path, binary references,
failed uploads, target switches and unchanged text snapshots.
- Run focused package tests, root build/typecheck/bundle, OpenAPI contract and
surface gates, plus focused Java checks within the module's shared ten-minute
local verification budget. SQL/concurrency cases need real MySQL evidence;
unrun SSH, CSI, Linux or real-provider paths remain explicit acceptance gaps.
Implemented across the control plane, bundled CLI and WebShell. Local
verification passed root build/typecheck/bundle, focused TypeScript and Java
tests, contract/surface gates and Checkstyle. The selected Java run covered 99
distinct cases; the new upload suite also passed 10 cases on disposable MySQL
8.4.11. Initial Java setup, checks and cleanup used 569 seconds; the final
review regression, direct compilation and style checks stayed within the same
shared 600-second local budget. Independent review identified a blocked stdin
writer competing with stdout readers in the common pool. An independent JVM
reproduced the timeout; moving the writer to a virtual thread passed both that
probe and the permanent forked-process regression. Review then found no
additional confirmed defect.
The actual bundled LOCAL Runtime passed 15 smoke-test groups (27 subprocess
commands). Earlier Chromium verification passed five groups using the real UI
and bundled Runtime with controlled Java HTTP responses. It exposed and
verified the repair of an existing-Session submission path that discarded
prepared text-only file context; that evidence is limited to its controlled
HTTP setup.
After rebasing onto main `338f13366c`, root build/typecheck/bundle, 452 selected
WebShell tests, 267 selected CLI tests, 92 selected Java tests, executable server
packaging and Checkstyle passed. A fresh MySQL 8.4.11 case initialized the full
current V1 baseline and verified that the original process hold excludes a
second upload and physical fencing. The user explicitly added 30 minutes to
the shared Java verification budget for this deployment work.
The actual `qwen serve --managed` deployment used isolated data, signed
Gateway-to-Java requests, fresh H2, Hosted Harness, and the production bundled
LOCAL Runtime without HTTP interception. Ten API groups passed binary/empty
and 8 MiB uploads, oversize rejection, no-clobber/idempotent results, path safety
and independent actor/tenant isolation. Chromium uploaded through the actual
full application menu and submitted a new Session with scoped path-only input.
The configured Kimi Turn produced no assistant or tool result within 150
seconds and was cancelled; real-provider file reading is not accepted. A
controlled restart preserved all four completed upload receipts, exact disk
digests and scope revision. The original Session subsequently required operator
recovery, so its disabled upload menu followed that failure boundary and an
existing-Session binary submission was not verified by this deployment run.
Remote SSH, Linux, real-provider reading, recovery of that Session and full
Java/Hosted/fault CI gates remain pending. CSI remains unsupported. Server, CLI
and WebShell must be deployed together; the pre-release schema policy requires
recreating development catalogs after this V1 baseline change.
@@ -0,0 +1,53 @@
# Managed Workspace 文件上传
[English](2026-10-26-managed-workspace-upload.md) | [简体中文](2026-10-26-managed-workspace-upload.zh-CN.md)
## 问题与范围
Managed WebShell 已支持随消息提交文本和图片附件,以及引用 Workspace 内的小文本文件快照,但不能把浏览器文件上传到持久 Workspace 存储。现有文件服务通过有界 CLI bootstrap 访问已登记的 LOCAL 和 SSH_LOCAL 存储;CSI 文件访问尚不支持。
为这两种已有存储补齐上传,支持空文件和二进制文件,单文件上限 8 MiB。复用输入框的文件选择、拖放选择、进度与错误展示、取消和 Session 切换保护。文件写入当前工作目录或正在浏览的已有目录。目标已存在时返回冲突,绝不覆盖。文件夹上传、自动创建目录、CSI 和对象存储服务不在本次范围内。
上传要求没有尚未释放的 execution-storage 成员。普通 Turn 结束后会释放成员;持久 MCP Runtime 可能仍保留成员,因此返回 `workspace_busy`。本次不会放宽这条所有权边界,也不会用 Session 表面空闲来证明工具进程已停止。
## 归属与 API
新增 WebShell `POST /api/agent/web-shell/v1/workspaces/files/upload` 接口,归属于持久 Workspace。查询参数为 `workspaceId`、`cwdRelative`、`path`、`expectedScopeRevision` 和 UUID `uploadId`,请求正文为 `application/octet-stream`。接口要求可信 actor 拥有 OPERATOR 或 OWNER 权限,并已启用 Workspace 文件访问。服务端校验登记的存储身份与修订,绝不回退到 daemon 的主 Workspace。二进制正文受硬性大小限制。同一上传 ID 的相同请求只对账原命令;请求内容改变则返回冲突。
响应包含 `scopeRevision`、`uploadId`、`path`、`sizeBytes` 和 `sha256`。Workspace 文件查询仅在所选 CLI 支持独立的 `workspaceFileUpload: 1` 握手时公布可选 `maxUploadBytes`。旧部署继续支持文本引用,上传保持禁用。OpenAPI、生成的客户端类型、surface registry 和能力映射同步更新。
## 持久准入与恢复
短 SQL 事务在现有操作账本登记 `WORKSPACE_UPLOAD` 并取得物理存储 owner hold,冻结 actor、请求摘要、ContextBinding、存储身份和原始 bootstrap 目标;二进制内容不存入 SQL。现有工具执行、存储 fencing 和退役必须尊重这个 hold。子进程和 SSH 调用都在 SQL 事务外执行。确认完成或明确失败后,只释放本操作的 hold;超时、响应丢失或控制面重启时保留占用,直至原命令完成对账。
Runtime 在配置的私有状态目录下按 `uploadId` 保存仅 owner 可访问的回执。写入前原子认领操作并记录原始 PID,所有文件变更和临时文件清理结束后才发布终态回执。对账绝不执行上传:返回终态回执、保守等待仍存活的原进程,或在明确确认原进程不存在后检查原目标的大小与摘要。如果尚无 claim,对账会原子发布失败墓碑,阻止迟到的原命令开始执行。PID 复用可能延迟对账,但不能授权重放。未知结果不会通过替换 Runtime、重复写入或回滚删除来处理。
## Runtime 文件系统边界
沿用 bootstrap 对允许根目录、存储身份和工作目录的校验。上传请求使用协议版本 1,操作为 `upload` 和 `upload-status`,字段包含 `uploadId`、相对 `path`、`sizeBytes`、`sha256`,仅上传携带 base64 `data`。响应的 `state` 为 `completed | failed | pending`;失败终态包含 `code`。上传输入使用独立的有界传输限制,不扩大普通 bootstrap 和响应的 64 KiB 限制。
复用规范化相对路径校验与目录身份检查。拒绝符号链接、目录穿越、Managed 保留路径和特殊文件。先写入仅 owner 可读写的临时文件并同步,复查目录锚点,再通过原子、不覆盖的硬链接发布。已有路径绝不被覆盖。发现路径或存储身份变化时拒绝。这些检查沿用 Node 文件系统已有的竞态限制,不宣称提供 `openat` 沙箱。
## 输入框行为
向共享输入框显式提供 Managed 上传目标,不因此启用其他 daemon 功能。目标身份包含 provider、Workspace、工作目录、修订和 Session。旧目标的上传完成后不能向当前草稿插入引用。已有定制开关仍可禁用上传。
上传成功后插入带作用域的路径引用。发送时 Managed 适配层校验其作用域,并把相对路径放入模型输入;不会把二进制内容转换成文本附件,也不会调用小文本快照 API。已有文本引用继续保留不可变快照行为。取消浏览器请求不承诺删除已经提交的文件;不确定结果按原上传 ID 对账。
## 验证与验收
- 基线:全局 CLI 缺少上传握手,Managed 菜单无法上传到 Workspace 存储。
- Runtime:二进制和空文件精确字节、8 MiB 边界、已有文件、嵌套与 Unicode 路径、符号链接与穿越、存储身份变化、并发和重复 ID、终态回执以及崩溃和迟到命令对账。
- 控制面:actor 与租户隔离、角色、过期作用域、禁用与不支持的存储、持久 hold、工具执行与 fencing 并发、请求变化、未知结果、原目标恢复以及响应大小和摘要校验。
- UI:能力启用路径、旧能力禁用路径、二进制引用、上传失败、目标切换和原有文本快照。
- 执行定向包测试、根目录 build/typecheck/bundle、OpenAPI 契约和 surface 门禁,以及 Java 模块共享十分钟本地验证预算内的定向检查。SQL 和并发案例需要真实 MySQL 证据;未运行的 SSH、CSI、Linux 或真实模型路径必须明确保留为验收缺口。
控制面、CLI bundle 和 WebShell 均已实现。根目录 build/typecheck/bundle、定向 TypeScript 与 Java 测试、契约与 surface 门禁、Checkstyle 均通过。Java 定向运行覆盖 99 个不同案例;新增上传测试还在一次性 MySQL 8.4.11 上通过 10 个案例。初轮 Java 环境准备、检查与清理使用 569 秒;审查后追加的回归、直接编译和样式检查仍在同一共享 600 秒本地预算内。独立审查发现 stdin writer 与 stdout reader 争用 common pool 时会阻塞;独立 JVM 复现了超时,改用虚拟线程后,原 probe 和永久 fork 子进程回归均通过。复查未再确认其他缺陷。
实际 LOCAL Runtime bundle 通过 15 组 smoke 检查(27 次子进程调用)。早期 Chromium 使用真实 UI 和 Runtime bundle、受控 Java HTTP 响应,通过五组检查,发现并验证修复了已有 Session 提交时丢弃文件路径文本上下文的问题;这部分证据仅覆盖受控 HTTP 环境。
同步主干 `338f13366c` 后,根目录 build/typecheck/bundle、452 个定向 WebShell 测试、267 个定向 CLI 测试、92 个定向 Java 测试、可执行 Server 打包和 Checkstyle 均通过。一次新的 MySQL 8.4.11 案例初始化当前完整 V1 基线,并验证原进程 hold 拒绝第二次上传及物理 fencing。用户明确为本次部署工作追加了 30 分钟共享 Java 验证预算。
真实 `qwen serve --managed` 使用隔离数据目录、Gateway 到 Java 的签名请求、新 H2、Hosted Harness 和正式构建的 LOCAL Runtime,全程没有 HTTP 拦截。十组 API 检查通过二进制/空文件/8 MiB 上传、超限拒绝、不覆盖与幂等结果、路径安全和独立 actor/租户隔离。Chromium 通过完整应用的实际菜单完成上传,并以带作用域的纯路径输入提交新 Session。配置的 Kimi Turn 在 150 秒内没有 assistant 或工具结果,已取消,不能宣称真实模型读取通过。受控重启后,四个已完成上传的回执、磁盘摘要和作用域修订均保持一致。原 Session 随后需要 operator recovery,上传菜单按该失败边界禁用,因此本次部署没有验证该已有 Session 的二进制提交。
远端 SSH、Linux、真实模型读取、上述 Session 的恢复及完整 Java/Hosted/故障 CI 门禁仍待验收。CSI 仍不支持。部署时需要同步更新 Server、CLI 和 WebShell;按正式发布前的 schema 策略,本次 V1 基线变更后需重建开发数据库。
+24
View File
@@ -56,6 +56,11 @@ const mocks = vi.hoisted(() => ({
runWorkspaceRecoveryWorker: vi.fn(),
runSessionSendMcp: vi.fn(),
runManagedRuntimeAttestationWorker: vi.fn(),
runManagedRuntimeBootstrap: vi.fn(),
}));
vi.mock('./serve/managed-runtime-bootstrap.js', () => ({
runManagedRuntimeBootstrap: mocks.runManagedRuntimeBootstrap,
}));
vi.mock('./commands/agents/session-send-mcp-server.js', () => ({
@@ -810,6 +815,25 @@ describe('runCliEntry', () => {
expect(mocks.tryRunServeFastPath).not.toHaveBeenCalled();
});
it.each([
{ args: [], help: false, upload: false },
{ args: ['--help'], help: true, upload: false },
{ args: ['--workspace-file-upload'], help: false, upload: true },
])(
'dispatches the private bootstrap entry $args before normal startup',
async ({ args, help, upload }) => {
process.env['QWEN_CODE_MANAGED_NPM_UPDATE_VERSION'] = '2.0.0';
await runCliEntry(['managed-runtime-bootstrap', ...args]);
expect(mocks.runManagedRuntimeBootstrap).toHaveBeenCalledWith(
help,
upload,
);
expect(mocks.installManagedNpmUpdate).not.toHaveBeenCalled();
expect(mocks.tryRunServeFastPath).not.toHaveBeenCalled();
expect(mocks.main).not.toHaveBeenCalled();
},
);
it('serves session_send without normal startup or any stdout output', async () => {
await runCliEntry([
'agents',
+7 -2
View File
@@ -589,12 +589,17 @@ export async function runCliEntry(
): Promise<void> {
if (
rawArgv[0] === 'managed-runtime-bootstrap' &&
(rawArgv.length === 1 || (rawArgv.length === 2 && rawArgv[1] === '--help'))
(rawArgv.length === 1 ||
(rawArgv.length === 2 &&
['--help', '--workspace-file-upload'].includes(rawArgv[1]!)))
) {
const { runManagedRuntimeBootstrap } = await import(
'./serve/managed-runtime-bootstrap.js'
);
await runManagedRuntimeBootstrap(rawArgv[1] === '--help');
await runManagedRuntimeBootstrap(
rawArgv[1] === '--help',
rawArgv[1] === '--workspace-file-upload',
);
return;
}
if (
+5 -1
View File
@@ -307,7 +307,10 @@ export const CAPABILITY_PARITY: Record<ServeFeature, ParityRow> = {
standing: 'managed',
evidence: 'tool:write_file,tool:edit',
},
workspace_file_upload: NO_MANAGED_EQUIVALENT('no Managed file upload route'),
workspace_file_upload: {
standing: 'managed',
evidence: 'surface:WORKSPACE_FILES_UPLOAD',
},
workspace_file_delete: NO_MANAGED_EQUIVALENT('no Managed file delete route'),
workspace_directory_create: NO_MANAGED_EQUIVALENT(
'no Managed directory create route',
@@ -570,6 +573,7 @@ export const MANAGED_ONLY_NOT_A_GAP = new Set([
'WORKSPACE_DIRECTORY_CHECK',
'WORKSPACE_FILES_QUERY',
'WORKSPACE_FILES_READ',
'WORKSPACE_FILES_UPLOAD_QUERY',
// Tenant resources and admission the daemon has no counterpart for.
'AGENT_DEFINITION_GET',
'AGENT_DEFINITION_UPDATE',
@@ -5,7 +5,7 @@
*/
import { execFileSync, spawn } from 'node:child_process';
import { createHmac, randomUUID } from 'node:crypto';
import { createHash, createHmac, randomUUID } from 'node:crypto';
import { constants, promises as fs } from 'node:fs';
import path from 'node:path';
import { isDeepStrictEqual } from 'node:util';
@@ -13,7 +13,15 @@ import { getCliVersion } from '../utils/version.js';
import {
readWorkspaceFiles,
WorkspaceFileError,
workspaceFilePath,
} from './managed-workspace-files.js';
import {
inspectWorkspaceUpload,
workspaceUploadRequest,
workspaceUploadResult,
writeWorkspaceUpload,
type WorkspaceUploadResult,
} from './managed-workspace-upload.js';
import {
createManagedContextAttestationResponse,
isManagedContextReady,
@@ -24,6 +32,7 @@ import {
const MARKER = '.qwen-managed-storage.json';
const MAX_BYTES = 64 * 1024;
const MAX_UPLOAD_REQUEST_BYTES = 12 * 1024 * 1024;
const ID = /^[a-zA-Z0-9_.:-]{1,512}$/;
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
let hostId: string | undefined;
@@ -674,6 +683,123 @@ async function ensure(
return observe(saved);
}
async function workspaceUpload(
request: Record<string, unknown>,
expected: { device: string; inode: string },
verifyStorage: () => Promise<void>,
): Promise<WorkspaceUploadResult> {
const { metadata, bytes } = workspaceUploadRequest(request['files']);
const root = absolute(request['root']);
const cwd = workspaceFilePath(request['cwdRelative']);
const target = {
nodeId: request['nodeId'],
root,
cwd,
expected,
identity: request['identity'] ?? null,
marker: request['marker'] ?? null,
...metadata,
};
const canonical = (value: unknown): unknown =>
value && typeof value === 'object' && !Array.isArray(value)
? Object.fromEntries(
Object.entries(value)
.sort(([a], [b]) => a.localeCompare(b))
.map(([key, item]) => [key, canonical(item)]),
)
: value;
const requestDigest = createHash('sha256')
.update(JSON.stringify(canonical(target)))
.digest('hex');
const stateDirectory = absolute(request['stateDirectory']);
await privateDirectory(stateDirectory);
const directory = path.join(
stateDirectory,
'workspace-uploads',
metadata.uploadId,
);
await privateDirectory(directory);
const claimPath = path.join(directory, 'claim.json');
const receiptPath = path.join(directory, 'receipt.json');
let claim = await optionalJson(claimPath);
let started = false;
if (claim === undefined) {
if ((await optionalJson(receiptPath)) !== undefined)
refuse('Workspace upload claim is unavailable.');
const proposed = {
version: 1,
target,
requestDigest,
pid: bytes === undefined ? null : process.pid,
};
started = await createSshPrivateJson(claimPath, proposed);
claim = started ? proposed : await readSshPrivateJson(claimPath);
}
const saved = object(claim);
exact(saved, ['version', 'target', 'requestDigest', 'pid']);
if (
saved['version'] !== 1 ||
saved['requestDigest'] !== requestDigest ||
!isDeepStrictEqual(saved['target'], target)
)
throw new WorkspaceFileError('ssh_runtime_file_upload_conflict');
const pid = saved['pid'];
if (pid !== null && (!Number.isSafeInteger(pid) || Number(pid) <= 1))
refuse('Workspace upload process identity is invalid.');
const receipt = await optionalJson(receiptPath);
if (receipt !== undefined) {
const result = object(receipt);
if (
!['completed', 'failed'].includes(String(result['state'])) ||
(result['state'] === 'failed' && typeof result['code'] !== 'string') ||
!isDeepStrictEqual(
receipt,
workspaceUploadResult(
metadata,
result['state'] as 'completed' | 'failed',
result['code'] as string | undefined,
),
)
)
refuse('Workspace upload receipt is invalid.');
return receipt as WorkspaceUploadResult;
}
let result: WorkspaceUploadResult;
if (pid === null) {
result = workspaceUploadResult(
metadata,
'failed',
'ssh_runtime_file_upload_not_started',
);
} else if (started && bytes !== undefined) {
result = await writeWorkspaceUpload(
root,
cwd,
metadata,
bytes,
expected,
verifyStorage,
);
} else {
let live = true;
try {
live = alive(Number(pid));
} catch {
// An unknown process state cannot authorize cleanup or a terminal receipt.
}
if (live) return workspaceUploadResult(metadata, 'pending');
result = await inspectWorkspaceUpload(
root,
cwd,
metadata,
expected,
verifyStorage,
);
}
await writeSshPrivateJson(receiptPath, result);
return result;
}
export async function executeSshBootstrap(
value: unknown,
): Promise<Record<string, unknown>> {
@@ -703,6 +829,10 @@ export async function executeSshBootstrap(
if (within(root, stateDirectory) || within(stateDirectory, root))
refuse('Runtime state and Workspace roots must not overlap.');
if (request['files'] !== undefined) {
const fileOperation = object(request['files'])['operation'];
const uploading = ['upload', 'upload-status'].includes(
String(fileOperation),
);
const identity =
request['identity'] === undefined
? undefined
@@ -711,19 +841,24 @@ export async function executeSshBootstrap(
? markerDocument(request['marker'], identity)
: undefined;
if (identity && identity.root !== root) refuse('Storage root conflicts.');
if (identity) await verifySshStorage(identity, marker);
if (identity && !uploading) await verifySshStorage(identity, marker);
const expected = identity ?? object(request['localRootIdentity']);
try {
const files = await readWorkspaceFiles(
root,
text(request['cwdRelative']),
request['files'],
{
device: text(expected.device),
inode: text(expected.inode),
},
);
if (identity) await verifySshStorage(identity, marker);
const fileIdentity = {
device: text(expected.device),
inode: text(expected.inode),
};
const files = uploading
? await workspaceUpload(request, fileIdentity, async () => {
if (identity) await verifySshStorage(identity, marker);
})
: await readWorkspaceFiles(
root,
text(request['cwdRelative']),
request['files'],
fileIdentity,
);
if (identity && !uploading) await verifySshStorage(identity, marker);
return { qwenVersion: await getCliVersion(), files };
} catch (error) {
if (error instanceof WorkspaceFileError)
@@ -876,10 +1011,13 @@ export async function executeSshBootstrap(
);
}
export async function runManagedRuntimeBootstrap(help = false): Promise<void> {
export async function runManagedRuntimeBootstrap(
help = false,
workspaceFileUpload = false,
): Promise<void> {
if (help) {
process.stdout.write(
`${JSON.stringify({ protocolVersion: 1, capabilities: { workspaceFiles: 1 } })}\n`,
`${JSON.stringify({ protocolVersion: 1, capabilities: { workspaceFiles: 1, workspaceFileUpload: 1 } })}\n`,
);
return;
}
@@ -890,17 +1028,30 @@ export async function runManagedRuntimeBootstrap(help = false): Promise<void> {
for await (const chunk of process.stdin) {
const bytes = Buffer.from(chunk as Uint8Array);
size += bytes.length;
if (size > MAX_BYTES)
if (size > (workspaceFileUpload ? MAX_UPLOAD_REQUEST_BYTES : MAX_BYTES))
refuse('Runtime bootstrap request exceeds its limit.');
chunks.push(bytes);
}
const source = new TextDecoder('utf-8', { fatal: true }).decode(
Buffer.concat(chunks),
);
const request = object(JSON.parse(source) as unknown);
const files = request['files'];
const upload =
files && typeof files === 'object' && !Array.isArray(files)
? ['upload', 'upload-status'].includes(
String((files as Record<string, unknown>)['operation']),
)
: false;
if (
workspaceFileUpload !== upload ||
(upload && request['operation'] !== 'probe')
)
refuse('Workspace upload requires its dedicated bootstrap entry.');
response = {
protocolVersion: 1,
ok: true,
result: await executeSshBootstrap(JSON.parse(source) as unknown),
result: await executeSshBootstrap(request),
};
} catch (error) {
const known =
@@ -8,6 +8,8 @@ import { createHash } from 'node:crypto';
import { constants, promises as fs, type BigIntStats } from 'node:fs';
import path from 'node:path';
export const MAX_WORKSPACE_UPLOAD_BYTES = 8 * 1024 * 1024;
export class WorkspaceFileError extends Error {
constructor(readonly code: string) {
super('Workspace file request is unavailable.');
@@ -51,16 +53,11 @@ function signature(stat: BigIntStats): string {
].join(':');
}
export async function readWorkspaceFiles(
export async function workspaceFileDirectories(
root: string,
cwdRelative: string,
value: unknown,
expected: { device: string; inode: string },
): Promise<Record<string, unknown>> {
if (!value || typeof value !== 'object' || Array.isArray(value))
refuse('invalid_request');
const request = value as Record<string, unknown>;
if (request['protocolVersion'] !== 1) refuse('unsupported');
) {
const cwd = workspaceFilePath(cwdRelative);
const anchors = new Map<string, string>();
async function directory(relative: string): Promise<string> {
@@ -98,11 +95,33 @@ export async function readWorkspaceFiles(
const base = await directory(cwd);
const rooted = (relative: string) =>
cwd === '.' ? relative : relative === '.' ? cwd : `${cwd}/${relative}`;
return {
base,
directory: (relative: string) => directory(rooted(relative)),
verify,
};
}
export async function readWorkspaceFiles(
root: string,
cwdRelative: string,
value: unknown,
expected: { device: string; inode: string },
): Promise<Record<string, unknown>> {
if (!value || typeof value !== 'object' || Array.isArray(value))
refuse('invalid_request');
const request = value as Record<string, unknown>;
if (request['protocolVersion'] !== 1) refuse('unsupported');
const { base, directory, verify } = await workspaceFileDirectories(
root,
cwdRelative,
expected,
);
if (request['operation'] === 'read') {
const relative = workspaceFilePath(request['path']);
if (relative === '.') refuse('invalid_path');
const parent = path.posix.dirname(relative);
await directory(rooted(parent));
await directory(parent);
const filename = path.join(base, relative);
const handle = await fs.open(
filename,
@@ -175,7 +194,7 @@ export async function readWorkspaceFiles(
const queue = [{ relative, depth: 0 }];
while (queue.length) {
const next = queue.shift()!;
const dir = await directory(rooted(next.relative));
const dir = await directory(next.relative);
const stream = await fs.opendir(dir);
for await (const entry of stream) {
if (++scanned > 4096 || Date.now() > deadline || entries.length >= 50) {
@@ -218,5 +237,11 @@ export async function readWorkspaceFiles(
}
}
await verify();
return { protocolVersion: 1, directory: relative, entries, truncated };
return {
protocolVersion: 1,
directory: relative,
entries,
truncated,
maxUploadBytes: MAX_WORKSPACE_UPLOAD_BYTES,
};
}
@@ -0,0 +1,222 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process';
import { createHash, randomUUID } from 'node:crypto';
import { promises as fs } from 'node:fs';
import { createRequire } from 'node:module';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
const api = pathToFileURL(
createRequire(import.meta.url).resolve('tsx/esm/api'),
).href;
const source = new URL('./managed-runtime-bootstrap.ts', import.meta.url).href;
const tsconfig = fileURLToPath(new URL('../../tsconfig.json', import.meta.url));
describe('Managed Workspace upload bootstrap processes', () => {
let temporary: string;
let root: string;
let entry: string;
let envelope: Record<string, unknown>;
const children = new Set<ChildProcessWithoutNullStreams>();
beforeEach(async () => {
temporary = await fs.realpath(
await fs.mkdtemp(path.join(os.tmpdir(), 'qwen-upload-process-')),
);
root = path.join(temporary, 'workspace');
await fs.mkdir(root);
const stat = await fs.stat(root, { bigint: true });
envelope = {
protocolVersion: 1,
operation: 'probe',
nodeId: 'node-a',
root,
stateDirectory: path.join(temporary, 'private'),
allowedRoots: [temporary],
localRootIdentity: { device: String(stat.dev), inode: String(stat.ino) },
cwdRelative: '.',
};
entry = path.join(temporary, 'entry.mjs');
await fs.writeFile(
entry,
`import {register} from ${JSON.stringify(api)};\nregister({tsconfig:${JSON.stringify(tsconfig)}});\nawait (await import(${JSON.stringify(source)})).runManagedRuntimeBootstrap(process.argv.includes('--help'),process.argv.includes('--workspace-file-upload'));\n`,
);
});
afterEach(async () => {
for (const child of children) {
if (child.exitCode === null && child.signalCode === null) {
child.kill('SIGKILL');
await new Promise((resolve) => child.once('exit', resolve));
}
}
children.clear();
await fs.rm(temporary, { recursive: true, force: true });
});
function metadata(bytes: Buffer) {
return {
protocolVersion: 1,
uploadId: randomUUID(),
path: 'file.bin',
sizeBytes: bytes.length,
sha256: createHash('sha256').update(bytes).digest('hex'),
};
}
function launch(
input: string,
args = ['--workspace-file-upload'],
executable = entry,
) {
const child = spawn(process.execPath, [executable, ...args], {
stdio: ['pipe', 'pipe', 'pipe'],
});
children.add(child);
let stdout = '';
let stderr = '';
child.stdout.on('data', (chunk: Buffer) => {
stdout += chunk.toString();
});
child.stderr.on('data', (chunk: Buffer) => {
stderr += chunk.toString();
});
child.stdin.on('error', () => {
/* A bounded reader may close before a rejected oversized write finishes. */
});
child.stdin.end(input);
const finished = new Promise<Record<string, unknown>>((resolve, reject) => {
child.once('error', reject);
child.once('exit', (code) => {
if (code !== 0)
return reject(new Error(`Bootstrap exited ${code}: ${stderr}`));
try {
expect(Buffer.byteLength(stdout)).toBeLessThan(64 * 1024);
expect(stdout.trim().split('\n')).toHaveLength(1);
resolve(JSON.parse(stdout) as Record<string, unknown>);
} catch (error) {
reject(error);
}
});
});
return { child, finished };
}
it('advertises the upload handshake separately from read access', async () => {
const response = await launch('', ['--help']).finished;
expect(response).toEqual({
protocolVersion: 1,
capabilities: { workspaceFiles: 1, workspaceFileUpload: 1 },
});
});
it('accepts a large binary payload only through the bounded dedicated entry', async () => {
const bytes = Buffer.alloc(128 * 1024, 0xfa);
const info = metadata(bytes);
const request = JSON.stringify({
...envelope,
files: { ...info, operation: 'upload', data: bytes.toString('base64') },
});
expect(await launch(request, []).finished).toMatchObject({ ok: false });
expect(await fs.readdir(root)).toEqual([]);
expect(await launch(request).finished).toMatchObject({
ok: true,
result: { files: { ...info, state: 'completed' } },
});
expect(await fs.readFile(path.join(root, info.path))).toEqual(bytes);
});
it('rejects non-upload operations and transport input beyond 12 MiB', async () => {
expect(await launch(JSON.stringify(envelope)).finished).toMatchObject({
ok: false,
});
expect(
await launch(' '.repeat(12 * 1024 * 1024 + 1)).finished,
).toMatchObject({ ok: false });
expect(await fs.readdir(root)).toEqual([]);
});
it('tombstones a never-started upload before a later process can mutate', async () => {
const bytes = Buffer.from('late');
const info = metadata(bytes);
const status = {
...envelope,
files: { ...info, operation: 'upload-status' },
};
const response = await launch(JSON.stringify(status)).finished;
expect(response).toMatchObject({
ok: true,
result: {
files: { state: 'failed', code: 'ssh_runtime_file_upload_not_started' },
},
});
expect(
await launch(
JSON.stringify({
...envelope,
files: {
...info,
operation: 'upload',
data: bytes.toString('base64'),
},
}),
).finished,
).toEqual(response);
expect(await fs.readdir(root)).toEqual([]);
});
it('keeps a live writer pending and recovers the original path after its process dies', async () => {
const bytes = Buffer.from([0, 0xff, 0x80, 10]);
const info = metadata(bytes);
const entered = path.join(temporary, 'before-cleanup');
const paused = path.join(temporary, 'paused.mjs');
await fs.writeFile(
paused,
`import {register} from ${JSON.stringify(api)};\nimport {promises as fs} from 'node:fs';\nregister({tsconfig:${JSON.stringify(tsconfig)}});\nconst unlink=fs.unlink.bind(fs);\nfs.unlink=async function(file){if(String(file).endsWith(${JSON.stringify(`.qwen-managed-upload-${info.uploadId}.tmp`)})){await fs.writeFile(${JSON.stringify(entered)},'ready');await new Promise(()=>setInterval(()=>{},1000));}return unlink(file);};\nawait (await import(${JSON.stringify(source)})).runManagedRuntimeBootstrap(false,true);\n`,
);
const original = launch(
JSON.stringify({
...envelope,
files: { ...info, operation: 'upload', data: bytes.toString('base64') },
}),
[],
paused,
);
const stopped = original.finished.catch(() => undefined);
for (let attempt = 0; ; attempt++) {
if (
await fs.access(entered).then(
() => true,
() => false,
)
)
break;
if (attempt >= 100)
throw new Error('Original upload did not reach cleanup.');
await new Promise((resolve) => setTimeout(resolve, 25));
}
const status = JSON.stringify({
...envelope,
files: { ...info, operation: 'upload-status' },
});
expect(await launch(status).finished).toMatchObject({
ok: true,
result: { files: { state: 'pending' } },
});
original.child.kill('SIGKILL');
await stopped;
expect(await launch(status).finished).toMatchObject({
ok: true,
result: { files: { state: 'completed', ...info } },
});
expect(await fs.readFile(path.join(root, info.path))).toEqual(bytes);
expect(await fs.readdir(root)).toEqual([info.path]);
});
});
@@ -0,0 +1,422 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { createHash, randomUUID } from 'node:crypto';
import { promises as fs } from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { executeSshBootstrap } from './managed-runtime-bootstrap.js';
import { MAX_WORKSPACE_UPLOAD_BYTES } from './managed-workspace-files.js';
import {
workspaceUploadRequest,
type WorkspaceUploadMetadata,
type WorkspaceUploadResult,
} from './managed-workspace-upload.js';
describe('Managed Workspace upload receipts and filesystem', () => {
let temporary: string;
let root: string;
let stateDirectory: string;
let identity: { device: string; inode: string };
beforeEach(async () => {
temporary = await fs.realpath(
await fs.mkdtemp(path.join(os.tmpdir(), 'qwen-upload-test-')),
);
root = path.join(temporary, 'workspace');
stateDirectory = path.join(temporary, 'private');
await fs.mkdir(root);
await fs.mkdir(stateDirectory, { mode: 0o700 });
const stat = await fs.stat(root, { bigint: true });
identity = { device: String(stat.dev), inode: String(stat.ino) };
});
afterEach(async () => {
vi.restoreAllMocks();
await fs.rm(temporary, { recursive: true, force: true });
});
function metadata(
bytes: Buffer,
relative = 'file.bin',
): WorkspaceUploadMetadata {
return {
uploadId: randomUUID(),
path: relative,
sizeBytes: bytes.length,
sha256: createHash('sha256').update(bytes).digest('hex'),
};
}
async function request(
info: WorkspaceUploadMetadata,
bytes?: Buffer,
cwdRelative = '.',
) {
const result = await executeSshBootstrap({
protocolVersion: 1,
operation: 'probe',
nodeId: 'node-a',
root,
stateDirectory,
allowedRoots: [temporary],
localRootIdentity: identity,
cwdRelative,
files: {
protocolVersion: 1,
operation: bytes === undefined ? 'upload-status' : 'upload',
...info,
...(bytes === undefined ? {} : { data: bytes.toString('base64') }),
},
});
return result['files'] as WorkspaceUploadResult;
}
const receiptDirectory = (info: WorkspaceUploadMetadata) =>
path.join(stateDirectory, 'workspace-uploads', info.uploadId);
it('preserves binary bytes, Unicode paths and cwd and advertises its size limit', async () => {
await fs.mkdir(path.join(root, 'working', 'nested'), { recursive: true });
const bytes = Buffer.from([0, 255, 128, 10, 1]);
const info = metadata(bytes, 'nested/你好.bin');
expect(await request(info, bytes, 'working')).toEqual({
protocolVersion: 1,
...info,
state: 'completed',
});
const filename = path.join(root, 'working', info.path);
expect(await fs.readFile(filename)).toEqual(bytes);
expect((await fs.stat(filename)).mode & 0o777).toBe(0o600);
expect(await fs.readdir(path.dirname(filename))).toEqual(['你好.bin']);
const listing = await executeSshBootstrap({
protocolVersion: 1,
operation: 'probe',
nodeId: 'node-a',
root,
stateDirectory,
allowedRoots: [temporary],
localRootIdentity: identity,
cwdRelative: 'working',
files: { protocolVersion: 1, operation: 'query' },
});
expect(listing['files']).toMatchObject({
maxUploadBytes: MAX_WORKSPACE_UPLOAD_BYTES,
});
});
it.each([0, MAX_WORKSPACE_UPLOAD_BYTES])(
'supports %i exact bytes',
async (size) => {
const bytes = Buffer.alloc(size, 0xff);
const info = metadata(bytes);
expect(await request(info, bytes)).toMatchObject({ state: 'completed' });
expect(
(await fs.readFile(path.join(root, info.path))).equals(bytes),
).toBe(true);
},
);
it('refuses oversized, corrupt and noncanonical payloads before creating a claim', async () => {
const bytes = Buffer.from('payload');
const info = metadata(bytes);
await expect(
request({ ...info, sizeBytes: MAX_WORKSPACE_UPLOAD_BYTES + 1 }, bytes),
).rejects.toThrow();
await expect(
request({ ...info, sha256: '0'.repeat(64) }, bytes),
).rejects.toThrow();
await expect(
request({ ...info, uploadId: '../not-an-id' }, bytes),
).rejects.toThrow();
const oneByte = metadata(Buffer.from([0]));
expect(() =>
workspaceUploadRequest({
protocolVersion: 1,
operation: 'upload',
...oneByte,
data: 'AB==',
}),
).toThrow();
expect(() =>
workspaceUploadRequest({
protocolVersion: 1,
operation: 'upload-status',
...oneByte,
data: '',
}),
).toThrow();
expect(await fs.readdir(stateDirectory)).toEqual([]);
expect(await fs.readdir(root)).toEqual([]);
});
it('reconciles repeated IDs without executing again and rejects a changed request', async () => {
const bytes = Buffer.from('original');
const info = metadata(bytes);
const completed = await request(info, bytes);
expect(await request(info)).toEqual(completed);
await fs.unlink(path.join(root, info.path));
expect(await request(info, bytes)).toEqual(completed);
await expect(
request({ ...info, path: 'different' }, bytes),
).rejects.toMatchObject({ code: 'ssh_runtime_file_upload_conflict' });
expect(await fs.readdir(root)).toEqual([]);
});
it('claims concurrent repeats once and never exposes a partial destination', async () => {
const bytes = Buffer.alloc(256 * 1024, 0x97);
const info = metadata(bytes);
const results = await Promise.all(
Array.from({ length: 8 }, () => request(info, bytes)),
);
expect(
results.every((result) =>
['completed', 'pending'].includes(result.state),
),
).toBe(true);
expect(await request(info)).toMatchObject({ state: 'completed' });
expect(await fs.readFile(path.join(root, info.path))).toEqual(bytes);
expect(await fs.readdir(root)).toEqual([info.path]);
});
it('refuses occupied paths without overwrite or automatic numbering', async () => {
const bytes = Buffer.from('new');
const info = metadata(bytes);
await fs.writeFile(path.join(root, info.path), 'original');
const result = await request(info, bytes);
expect(result).toMatchObject({
state: 'failed',
code: 'ssh_runtime_file_exists',
});
expect(await request(info)).toEqual(result);
expect(await fs.readFile(path.join(root, info.path), 'utf8')).toBe(
'original',
);
expect(await fs.readdir(root)).toEqual([info.path]);
});
it('publishes only one destination under concurrent different uploads', async () => {
const first = Buffer.from('first');
const second = Buffer.from('second');
const results = await Promise.all([
request(metadata(first), first),
request(metadata(second), second),
]);
expect(results.map((result) => result.state).sort()).toEqual([
'completed',
'failed',
]);
const completed = results.find((result) => result.state === 'completed')!;
const actual = await fs.readFile(path.join(root, completed.path));
expect(createHash('sha256').update(actual).digest('hex')).toBe(
completed.sha256,
);
expect(await fs.readdir(root)).toEqual([completed.path]);
});
it.each([
'../escape',
'/absolute',
'.git/config',
'.qwen-managed-storage.json',
'.qwen-child-workspaces/file',
'a//b',
'a\\b',
'.',
])('rejects reserved or invalid path %s', async (relative) => {
const bytes = Buffer.from('file');
await expect(request(metadata(bytes, relative), bytes)).rejects.toThrow();
expect(await fs.readdir(root)).toEqual([]);
});
it('refuses symlink destinations, symlink ancestors and absent directories', async () => {
const outside = path.join(temporary, 'outside');
await fs.mkdir(outside);
await fs.writeFile(path.join(outside, 'original'), 'original');
await fs.symlink(outside, path.join(root, 'alias'));
await fs.symlink(path.join(outside, 'original'), path.join(root, 'link'));
const bytes = Buffer.from('new');
for (const relative of ['alias/new', 'link', 'absent/new']) {
expect(await request(metadata(bytes, relative), bytes)).toMatchObject({
state: 'failed',
});
}
expect(await fs.readdir(outside)).toEqual(['original']);
expect(await fs.readFile(path.join(outside, 'original'), 'utf8')).toBe(
'original',
);
expect((await fs.readdir(root)).sort()).toEqual(['alias', 'link']);
});
it('fails a changed root identity without creating a file', async () => {
identity = { ...identity, inode: '0' };
const bytes = Buffer.from('file');
expect(await request(metadata(bytes), bytes)).toMatchObject({
state: 'failed',
});
expect(await fs.readdir(root)).toEqual([]);
});
it('tombstones an unstarted operation before a delayed mutation arrives', async () => {
const bytes = Buffer.from('late');
const info = metadata(bytes);
const result = await request(info);
expect(result).toMatchObject({
state: 'failed',
code: 'ssh_runtime_file_upload_not_started',
});
expect(await request(info, bytes)).toEqual(result);
expect(await fs.readdir(root)).toEqual([]);
});
it('returns private terminal receipts after registered SSH storage changes while fencing new filesystem access', async () => {
const probe = await executeSshBootstrap({
protocolVersion: 1,
operation: 'probe',
nodeId: 'node-a',
root,
stateDirectory,
allowedRoots: [temporary],
});
const sshIdentity = probe['identity'] as Record<string, unknown>;
const marker = {
...sshIdentity,
version: 2,
tenantId: 'tenant-a',
storageId: 'storage-a',
registrationId: randomUUID(),
};
await executeSshBootstrap({
protocolVersion: 1,
operation: 'register-storage',
nodeId: 'node-a',
stateDirectory,
identity: sshIdentity,
marker,
});
const send = async (files: Record<string, unknown>) =>
(
await executeSshBootstrap({
protocolVersion: 1,
operation: 'probe',
nodeId: 'node-a',
root,
stateDirectory,
allowedRoots: [temporary],
cwdRelative: '.',
identity: sshIdentity,
marker,
files: { protocolVersion: 1, ...files },
})
)['files'];
const bytes = Buffer.from('original');
const info = metadata(bytes);
const result = await send({
...info,
operation: 'upload',
data: bytes.toString('base64'),
});
expect(result).toMatchObject({ state: 'completed' });
await fs.rename(root, `${root}-original`);
await fs.mkdir(root);
expect(await send({ ...info, operation: 'upload-status' })).toEqual(result);
await expect(send({ operation: 'query' })).rejects.toThrow();
expect(
await send({ ...metadata(bytes), operation: 'upload-status' }),
).toMatchObject({
state: 'failed',
code: 'ssh_runtime_file_upload_not_started',
});
expect(
await send({
...metadata(bytes),
operation: 'upload',
data: bytes.toString('base64'),
}),
).toMatchObject({ state: 'failed' });
expect(await fs.readdir(root)).toEqual([]);
});
it('keeps a live or unverified original process pending and leaves its temporary file', async () => {
const bytes = Buffer.from('original');
const info = metadata(bytes);
await request(info, bytes);
await fs.unlink(path.join(receiptDirectory(info), 'receipt.json'));
const temp = path.join(root, `.qwen-managed-upload-${info.uploadId}.tmp`);
await fs.writeFile(temp, 'in progress');
expect(await request(info)).toMatchObject({ state: 'pending' });
vi.spyOn(process, 'kill').mockImplementation(() => {
throw Object.assign(new Error('unknown'), { code: 'EPERM' });
});
expect(await request(info)).toMatchObject({ state: 'pending' });
expect(await fs.readFile(temp, 'utf8')).toBe('in progress');
});
it.each([true, false])(
'reconciles only after definite process absence with matching destination %s',
async (matching) => {
const bytes = Buffer.from('original');
const info = metadata(bytes);
await request(info, bytes);
await fs.unlink(path.join(receiptDirectory(info), 'receipt.json'));
if (!matching)
await fs.writeFile(path.join(root, info.path), 'different');
const temp = path.join(root, `.qwen-managed-upload-${info.uploadId}.tmp`);
await fs.writeFile(temp, 'leftover');
vi.spyOn(process, 'kill').mockImplementation(() => {
throw Object.assign(new Error('absent'), { code: 'ESRCH' });
});
const result = await request(info);
expect(result.state).toBe(matching ? 'completed' : 'failed');
expect(await fs.readdir(root)).toEqual([info.path]);
expect(await request(info, bytes)).toEqual(result);
expect(await fs.readFile(path.join(root, info.path), 'utf8')).toBe(
matching ? 'original' : 'different',
);
},
);
it('leaves an uncertain cleanup without a terminal receipt', async () => {
const bytes = Buffer.from('original');
const info = metadata(bytes);
const unlink = fs.unlink.bind(fs);
vi.spyOn(fs, 'unlink').mockImplementation(async (filename) => {
if (
String(filename).endsWith(`.qwen-managed-upload-${info.uploadId}.tmp`)
)
throw Object.assign(new Error('cannot clean'), { code: 'EACCES' });
return unlink(filename);
});
await expect(request(info, bytes)).rejects.toThrow('cannot clean');
await expect(
fs.stat(path.join(receiptDirectory(info), 'receipt.json')),
).rejects.toMatchObject({ code: 'ENOENT' });
expect(await request(info)).toMatchObject({ state: 'pending' });
});
it('cleans an interrupted prepublication temporary file without executing it', async () => {
const bytes = Buffer.from('original');
const info = metadata(bytes);
await request(info, bytes);
await fs.unlink(path.join(receiptDirectory(info), 'receipt.json'));
await fs.unlink(path.join(root, info.path));
await fs.writeFile(
path.join(root, `.qwen-managed-upload-${info.uploadId}.tmp`),
bytes,
);
vi.spyOn(process, 'kill').mockImplementation(() => {
throw Object.assign(new Error('absent'), { code: 'ESRCH' });
});
const result = await request(info);
expect(result).toMatchObject({
state: 'failed',
code: 'ssh_runtime_file_upload_incomplete',
});
expect(await fs.readdir(root)).toEqual([]);
expect(await request(info, bytes)).toEqual(result);
expect(await fs.readdir(root)).toEqual([]);
});
});
@@ -0,0 +1,240 @@
/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import { createHash } from 'node:crypto';
import { constants, promises as fs } from 'node:fs';
import path from 'node:path';
import {
MAX_WORKSPACE_UPLOAD_BYTES,
WorkspaceFileError,
workspaceFileDirectories,
workspaceFilePath,
} from './managed-workspace-files.js';
export interface WorkspaceUploadMetadata {
uploadId: string;
path: string;
sizeBytes: number;
sha256: string;
}
export interface WorkspaceUploadResult extends WorkspaceUploadMetadata {
protocolVersion: 1;
state: 'completed' | 'failed' | 'pending';
code?: string;
}
function refuse(code: string): never {
throw new WorkspaceFileError(`ssh_runtime_file_${code}`);
}
function failed(metadata: WorkspaceUploadMetadata, error: unknown) {
return workspaceUploadResult(
metadata,
'failed',
error instanceof WorkspaceFileError
? error.code
: (error as NodeJS.ErrnoException).code === 'EEXIST'
? 'ssh_runtime_file_exists'
: 'ssh_runtime_file_unavailable',
);
}
export function workspaceUploadRequest(value: unknown): {
metadata: WorkspaceUploadMetadata;
bytes?: Buffer;
} {
if (!value || typeof value !== 'object' || Array.isArray(value))
refuse('invalid_request');
const request = value as Record<string, unknown>;
if (
request['protocolVersion'] !== 1 ||
!['upload', 'upload-status'].includes(String(request['operation'])) ||
typeof request['uploadId'] !== 'string' ||
!/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(
request['uploadId'],
) ||
!Number.isSafeInteger(request['sizeBytes']) ||
Number(request['sizeBytes']) < 0 ||
Number(request['sizeBytes']) > MAX_WORKSPACE_UPLOAD_BYTES ||
typeof request['sha256'] !== 'string' ||
!/^[0-9a-f]{64}$/.test(request['sha256'])
)
refuse('invalid_request');
const relative = workspaceFilePath(request['path']);
if (relative === '.') refuse('invalid_path');
const metadata = {
uploadId: request['uploadId'],
path: relative,
sizeBytes: Number(request['sizeBytes']),
sha256: request['sha256'],
};
if (request['operation'] === 'upload-status') {
if (request['data'] !== undefined) refuse('invalid_request');
return { metadata };
}
const data = request['data'];
if (
typeof data !== 'string' ||
data.length !== Math.ceil(metadata.sizeBytes / 3) * 4
)
refuse('invalid_content');
const bytes = Buffer.from(data, 'base64');
if (
bytes.length !== metadata.sizeBytes ||
bytes.toString('base64') !== data ||
createHash('sha256').update(bytes).digest('hex') !== metadata.sha256
)
refuse('invalid_content');
return { metadata, bytes };
}
export function workspaceUploadResult(
metadata: WorkspaceUploadMetadata,
state: WorkspaceUploadResult['state'],
code?: string,
): WorkspaceUploadResult {
return {
protocolVersion: 1,
...metadata,
state,
...(code === undefined ? {} : { code }),
};
}
function temporaryPath(directory: string, metadata: WorkspaceUploadMetadata) {
return path.join(directory, `.qwen-managed-upload-${metadata.uploadId}.tmp`);
}
async function removeTemporary(temporary: string): Promise<void> {
try {
await fs.unlink(temporary);
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
}
}
async function syncDirectory(directory: string): Promise<void> {
const handle = await fs.open(directory, 'r');
try {
await handle.sync();
} finally {
await handle.close();
}
}
export async function writeWorkspaceUpload(
root: string,
cwd: string,
metadata: WorkspaceUploadMetadata,
bytes: Buffer,
expected: { device: string; inode: string },
verifyStorage: () => Promise<void>,
): Promise<WorkspaceUploadResult> {
let directories: Awaited<ReturnType<typeof workspaceFileDirectories>>;
let parent: string;
try {
await verifyStorage();
directories = await workspaceFileDirectories(root, cwd, expected);
parent = await directories.directory(path.posix.dirname(metadata.path));
} catch (error) {
return failed(metadata, error);
}
const destination = path.join(directories.base, metadata.path);
const temporary = temporaryPath(parent, metadata);
let ownsTemporary = false;
let result: WorkspaceUploadResult;
try {
const handle = await fs.open(temporary, 'wx', 0o600);
ownsTemporary = true;
try {
await handle.writeFile(bytes);
await handle.sync();
} finally {
await handle.close();
}
await directories.verify();
await verifyStorage();
await fs.link(temporary, destination);
await directories.verify();
await verifyStorage();
result = workspaceUploadResult(metadata, 'completed');
} catch (error) {
result = failed(metadata, error);
} finally {
if (ownsTemporary) {
await directories.verify();
await removeTemporary(temporary);
await syncDirectory(parent);
}
}
return result;
}
export async function inspectWorkspaceUpload(
root: string,
cwd: string,
metadata: WorkspaceUploadMetadata,
expected: { device: string; inode: string },
verifyStorage: () => Promise<void>,
): Promise<WorkspaceUploadResult> {
await verifyStorage();
const directories = await workspaceFileDirectories(root, cwd, expected);
const parent = await directories.directory(path.posix.dirname(metadata.path));
const destination = path.join(directories.base, metadata.path);
let completed = false;
try {
const handle = await fs.open(
destination,
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK,
);
try {
const before = await handle.stat({ bigint: true });
if (before.isFile() && before.size === BigInt(metadata.sizeBytes)) {
const bytes = Buffer.alloc(metadata.sizeBytes + 1);
let size = 0;
while (size < bytes.length) {
const read = await handle.read(
bytes,
size,
bytes.length - size,
size,
);
if (!read.bytesRead) break;
size += read.bytesRead;
}
const after = await handle.stat({ bigint: true });
const current = await fs.lstat(destination, { bigint: true });
completed =
size === metadata.sizeBytes &&
before.dev === current.dev &&
before.ino === current.ino &&
before.mtimeNs === after.mtimeNs &&
before.ctimeNs === after.ctimeNs &&
before.size === after.size &&
after.ctimeNs === current.ctimeNs &&
createHash('sha256').update(bytes.subarray(0, size)).digest('hex') ===
metadata.sha256;
}
} finally {
await handle.close();
}
} catch (error) {
if (
!['ENOENT', 'ELOOP'].includes((error as NodeJS.ErrnoException).code ?? '')
)
throw error;
}
await directories.verify();
await verifyStorage();
await removeTemporary(temporaryPath(parent, metadata));
await syncDirectory(parent);
return workspaceUploadResult(
metadata,
completed ? 'completed' : 'failed',
completed ? undefined : 'ssh_runtime_file_upload_incomplete',
);
}
+6
View File
@@ -844,6 +844,12 @@ packaged-stack E2E, which have no ingress,
then trusted as the actor for the request's tenant. It is disabled by default;
never enable it where untrusted clients can reach the server.
### Upload files into a Workspace
With Workspace file access enabled and compatible Server, CLI and WebShell artifacts, select a persisted LOCAL or SSH_LOCAL Workspace and choose **Add file → Upload to workspace**. Uploads accept binary and empty files up to 8 MiB each, use an existing destination directory, and reject occupied paths without overwriting them. The composer inserts a scoped path reference for tools to use; binary bytes are not converted into a small-text snapshot or message attachment. Old CLIs leave upload disabled. CSI, folder uploads and automatic directory creation are unsupported.
Uploads require an authenticated actor with OPERATOR or OWNER access and verified storage. An execution-storage member, including a retained MCP Runtime, keeps the Workspace busy. A durable physical-storage hold excludes new execution and retirement until the original upload has a definite result. If the response is uncertain, query the original upload ID; recovery does not replay the write on another Runtime. Cancelling the browser request does not delete a file already committed. See the [English design](../../docs/design/2026-10-26-managed-workspace-upload.md) and [Chinese design](../../docs/design/2026-10-26-managed-workspace-upload.zh-CN.md) for the protocol, recovery and acceptance boundaries.
### Create empty Workspaces from WebShell
Set `QWEN_MANAGED_AGENT_WORKSPACE_CREATION_ROOT` to an existing canonical, server-owned directory to enable **New workspace → Start from scratch** in the Managed composer. The default is disabled. The root must be separate from static Workspace mounts, the default Workspace cwd, and private Runtime state. Enable Workspace files, the Session Store, the durable Session-isolated local-process Broker and verified local Workspace recovery first. Enabling this setting authorizes authenticated tenant actors to allocate independent empty roots and receive OWNER access; only enable it for tenants allowed to consume this host's storage.
@@ -1,24 +1,32 @@
package com.alibaba.qwen.code.managedagent.api;
import com.alibaba.qwen.code.managedagent.exception.ManagedAgentException;
import com.alibaba.qwen.code.managedagent.model.TenantContext;
import com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileStore;
import com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileUploadStore;
import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.databind.JsonNode;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.validation.Valid;
import jakarta.validation.constraints.NotBlank;
import jakarta.validation.constraints.Size;
import org.springframework.http.CacheControl;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class WorkspaceFileController {
private final WorkspaceFileStore files;
private final WorkspaceFileUploadStore uploads;
public WorkspaceFileController(WorkspaceFileStore files) {
public WorkspaceFileController(WorkspaceFileStore files, WorkspaceFileUploadStore uploads) {
this.files = files;
this.uploads = uploads;
}
public record QueryRequest(@NotBlank String workspaceId, @NotBlank String cwdRelative,
@@ -38,6 +46,13 @@ public class WorkspaceFileController {
}
}
public record UploadQueryRequest(@NotBlank @Size(max = 36) String uploadId) {
@JsonAnySetter
public void rejectUnknown(String name, JsonNode value) {
throw new IllegalArgumentException("Unknown Workspace upload query field");
}
}
@PostMapping("/api/agent/web-shell/v1/workspaces/files/query")
public ResponseEntity<WorkspaceFileStore.QueryResult> query(TenantContext tenant,
@Valid @RequestBody QueryRequest request) {
@@ -52,4 +67,24 @@ public class WorkspaceFileController {
return ResponseEntity.ok().cacheControl(CacheControl.noStore()).body(files.read(tenant.tenantId(),
tenant.requireActorId(), request.workspaceId(), request.cwdRelative(), request.path(), request.expectedScopeRevision()));
}
@PostMapping(value = "/api/agent/web-shell/v1/workspaces/files/upload", consumes = MediaType.APPLICATION_OCTET_STREAM_VALUE)
public ResponseEntity<WorkspaceFileUploadStore.UploadResult> upload(TenantContext tenant,
@RequestParam String workspaceId, @RequestParam String cwdRelative, @RequestParam String path,
@RequestParam String expectedScopeRevision, @RequestParam String uploadId, HttpServletRequest request) throws java.io.IOException {
String actor = tenant.requireActorId();
String encoding = request.getHeader("Content-Encoding");
if (encoding != null && !"identity".equalsIgnoreCase(encoding)) {
throw new ManagedAgentException(HttpStatus.UNSUPPORTED_MEDIA_TYPE, "invalid_request", "Encoded Workspace uploads are unsupported.");
}
return ResponseEntity.ok().cacheControl(CacheControl.noStore()).body(uploads.upload(tenant.tenantId(), actor,
workspaceId, cwdRelative, path, expectedScopeRevision, uploadId, request.getInputStream(), request.getContentLengthLong()));
}
@PostMapping("/api/agent/web-shell/v1/workspaces/files/uploads/query")
public ResponseEntity<WorkspaceFileUploadStore.UploadStatus> queryUpload(TenantContext tenant,
@Valid @RequestBody UploadQueryRequest request) {
return ResponseEntity.ok().cacheControl(CacheControl.noStore()).body(uploads.query(tenant.tenantId(),
tenant.requireActorId(), request.uploadId()));
}
}
@@ -17,6 +17,7 @@ final class LocalStorageOwners {
static final String OPERATION_AUTHORITY_SQL = "SELECT 1 FROM ma_operation WHERE storage_key = ?"
+ " AND ((operation_kind IN ('WORKSPACE_RECOVERY', 'WORKSPACE_OPERATOR_RECOVERY', 'STORAGE_MIGRATION')"
+ " AND state IN ('CAPTURING', 'VERIFYING', 'PREPARED', 'ATTESTED', 'RETIRING', 'RETIRED', 'PREPARING', 'INVALIDATED'))"
+ " OR (operation_kind = 'WORKSPACE_UPLOAD' AND state = 'RUNNING')"
+ " OR (operation_kind = 'CHILD_WORKSPACE' AND state IN ('"
+ String.join("', '", ChildWorkspaceStore.PREPARING, ChildWorkspaceStore.READY, ChildWorkspaceStore.MERGING,
ChildWorkspaceStore.APPLYING, ChildWorkspaceStore.APPLIED, ChildWorkspaceStore.DISCARDING,
@@ -46,7 +46,9 @@ public class WorkspaceFileStore {
public record Entry(String path, String name, String kind) {
}
public record QueryResult(String scopeRevision, String directory, List<Entry> entries, boolean truncated) {
public record QueryResult(String scopeRevision, String directory, List<Entry> entries, boolean truncated,
@com.fasterxml.jackson.annotation.JsonInclude(com.fasterxml.jackson.annotation.JsonInclude.Include.NON_NULL)
Integer maxUploadBytes) {
}
public record FileResult(String scopeRevision, String path, String name, String mimeType,
@@ -64,8 +66,10 @@ public class WorkspaceFileStore {
|| entry.path() == null || entry.name() == null || entry.kind() == null || !List.of("file", "directory").contains(entry.kind()))) {
throw unavailable();
}
Integer maximum = mapper.convertValue(result.files().get("maxUploadBytes"), Integer.class);
return new QueryResult(result.revision(), directory, List.of(entries),
Boolean.TRUE.equals(result.files().get("truncated")));
Boolean.TRUE.equals(result.files().get("truncated")),
Integer.valueOf(WorkspaceFileProcess.MAX_UPLOAD_BYTES).equals(maximum) ? maximum : null);
} catch (IllegalArgumentException error) {
throw unavailable();
}
@@ -89,7 +93,7 @@ public class WorkspaceFileStore {
}
}
private record Scope(ContextBinding binding, String kind, String revision) {
record Scope(ContextBinding binding, String kind, String revision) {
}
private record Exchange(String revision, Map<String, Object> files) {
@@ -143,7 +147,7 @@ public class WorkspaceFileStore {
}
}
private Scope scope(String tenant, String actor, String workspace, String cwd) {
Scope scope(String tenant, String actor, String workspace, String cwd) {
return Objects.requireNonNull(transaction.execute(status -> {
var binding = registry.resolveForCreation(tenant, actor, new WorkspaceSelection(workspace, cwd)).binding();
if (!properties.getHarness().isWorkspaceFilesEnabled()) {
@@ -0,0 +1,455 @@
package com.alibaba.qwen.code.managedagent.persistence;
import com.alibaba.qwen.code.managedagent.config.ManagedAgentProperties;
import com.alibaba.qwen.code.managedagent.exception.ManagedAgentException;
import com.alibaba.qwen.code.managedagent.workspace.SshWorkspaceService;
import com.alibaba.qwen.code.managedagent.workspace.SshWorkspaceService.FileTarget;
import com.alibaba.qwen.code.managedagent.workspace.WorkspaceSelection;
import com.alibaba.qwen.code.runtimebroker.BrokerValues;
import com.alibaba.qwen.code.runtimebroker.JdbcRuntimeBindingRepository;
import com.alibaba.qwen.code.runtimebroker.RuntimeBrokerException;
import com.alibaba.qwen.code.runtimebroker.managedschema.ManagedSchemaKeys;
import com.alibaba.qwen.code.runtimebroker.managedschema.ManagedSchemaTime;
import com.alibaba.qwen.code.runtimebroker.provisioning.WorkspaceFileProcess;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.commons.codec.digest.DigestUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.http.HttpStatus;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.support.TransactionTemplate;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Executor;
import java.util.concurrent.Semaphore;
@Component
public class WorkspaceFileUploadStore {
private static final Logger LOG = LoggerFactory.getLogger(WorkspaceFileUploadStore.class);
private static final String KIND = "WORKSPACE_UPLOAD";
private static final int LIMIT = WorkspaceFileProcess.MAX_UPLOAD_BYTES;
private final WorkspaceFileStore files;
private final ManagedWorkspaceRegistry registry;
private final WorkspaceStorageGuard guard;
private final SshWorkspaceService ssh;
private final ManagedAgentProperties properties;
private final JdbcTemplate jdbc;
private final TransactionTemplate transaction;
private final ObjectMapper mapper;
private final Executor executor;
private final Semaphore slots = new Semaphore(4);
private final Set<String> running = ConcurrentHashMap.newKeySet();
public WorkspaceFileUploadStore(WorkspaceFileStore files, ManagedWorkspaceRegistry registry,
WorkspaceStorageGuard guard, SshWorkspaceService ssh, ManagedAgentProperties properties,
JdbcTemplate jdbc, PlatformTransactionManager manager, ObjectMapper mapper,
@Qualifier("managedAgentExecutor") Executor executor) {
this.files = files;
this.registry = registry;
this.guard = guard;
this.ssh = ssh;
this.properties = properties;
this.jdbc = jdbc;
this.transaction = new TransactionTemplate(manager);
this.mapper = mapper;
this.executor = executor;
}
public record UploadResult(String scopeRevision, String uploadId, String path, int sizeBytes, String sha256) {
}
@JsonInclude(JsonInclude.Include.NON_NULL)
public record UploadStatus(String scopeRevision, String uploadId, String state, String path,
int sizeBytes, String sha256, String code) {
}
private record Frozen(String tenant, String actor, String workspace, String cwd, String revision,
String uploadId, String path, int sizeBytes, String sha256, String physical,
String node, String entry, Map<String, Object> request, FileTarget remote) {
}
private record Operation(String key, String digest, Frozen target, UploadStatus result, boolean fresh) {
}
public UploadResult upload(String tenant, String actor, String workspace, String cwd, String path,
String revision, String uploadId, InputStream input, long contentLength) {
validate(path, revision, uploadId);
cwd = new WorkspaceSelection(workspace, cwd).cwdRelative();
if (!".".equals(cwd)) {
validate(cwd, revision, uploadId);
}
if (!slots.tryAcquire()) {
throw busy();
}
String key = key(tenant, uploadId);
try {
Operation saved = find(tenant, uploadId, false);
if (saved == null) {
WorkspaceFileStore.Scope scope = files.scope(tenant, actor, workspace, cwd);
if (!revision.equals(scope.revision())) {
throw changed();
}
} else {
authorize(saved, actor);
}
byte[] bytes = read(input, contentLength);
String digest = digest(List.of(actor, workspace, cwd, path, revision, uploadId,
bytes.length, DigestUtils.sha256Hex(bytes)));
Operation operation = admit(tenant, actor, workspace, cwd, path, revision, uploadId,
bytes.length, DigestUtils.sha256Hex(bytes), digest);
running.add(key);
UploadStatus result = operation.result();
if (result == null) {
try {
result = exchange(operation, operation.fresh() ? bytes : null);
} catch (RuntimeBrokerException error) {
if (operation.fresh() && "workspace_files_unsupported".equals(error.getCode())) {
result = finish(operation, status(operation.target(), "failed", "workspace_files_unsupported"));
} else {
defer(operation.key());
throw pending();
}
} catch (RuntimeException error) {
defer(operation.key());
throw pending();
}
}
authorize(operation, actor);
if ("pending".equals(result.state())) {
throw pending();
}
if ("failed".equals(result.state())) {
throw new ManagedAgentException(HttpStatus.CONFLICT, result.code(), "Workspace upload failed.");
}
return new UploadResult(result.scopeRevision(), result.uploadId(), result.path(), result.sizeBytes(), result.sha256());
} finally {
running.remove(key);
slots.release();
}
}
public UploadStatus query(String tenant, String actor, String uploadId) {
validateId(uploadId);
Operation operation = find(tenant, uploadId, false);
if (operation == null) {
throw notFound();
}
authorize(operation, actor);
if (operation.result() != null) {
return operation.result();
}
if (!slots.tryAcquire()) {
return status(operation.target(), "pending", null);
}
try {
UploadStatus result;
try {
result = exchange(operation, null);
} catch (RuntimeException unavailable) {
defer(operation.key());
result = status(operation.target(), "pending", null);
}
authorize(operation, actor);
return result;
} finally {
slots.release();
}
}
@Scheduled(initialDelay = 5000, fixedDelay = 5000)
public void reconcile() {
var due = jdbc.queryForList("SELECT tenant_id, operation_id FROM ma_operation WHERE operation_kind = ?"
+ " AND state = 'RUNNING' AND available_at <= " + ManagedSchemaTime.SQL_NOW
+ " ORDER BY available_at, operation_key LIMIT 4", KIND);
for (var row : due) {
String tenant = text(row.get("tenant_id"));
String uploadId = text(row.get("operation_id"));
String key = key(tenant, uploadId);
if (!running.add(key)) {
continue;
}
if (!slots.tryAcquire()) {
running.remove(key);
return;
}
try {
executor.execute(() -> {
try {
Operation operation = find(tenant, uploadId, false);
if (operation != null && operation.result() == null) {
exchange(operation, null);
}
} catch (RuntimeException error) {
LOG.debug("Workspace upload reconciliation deferred: {}", error.getClass().getSimpleName());
} finally {
try {
defer(key);
} finally {
running.remove(key);
slots.release();
}
}
});
} catch (RuntimeException error) {
running.remove(key);
slots.release();
throw error;
}
}
}
private Operation admit(String tenant, String actor, String workspace, String cwd, String path,
String revision, String uploadId, int size, String sha256, String digest) {
return transaction.execute(status -> {
WorkspaceStorageKindGuard.lockDomain(jdbc, tenant);
Operation previous = find(tenant, uploadId, true);
if (previous != null) {
authorize(previous, actor);
if (!previous.digest().equals(digest)) {
throw new ManagedAgentException(HttpStatus.CONFLICT, "workspace_upload_conflict", "Upload ID belongs to another request.");
}
return previous;
}
var scope = files.scope(tenant, actor, workspace, cwd);
if (!revision.equals(scope.revision())) {
throw changed();
}
if (!Set.of("LOCAL", "SSH_LOCAL").contains(scope.kind())) {
throw new ManagedAgentException(HttpStatus.CONFLICT, "workspace_files_unsupported", "This Workspace storage does not support uploads.");
}
WorkspaceMigrationAdmission.requireOpen(jdbc, tenant, scope.binding().getStorageId());
String physical = jdbc.queryForObject("SELECT physical_key FROM ma_storage WHERE alias_key = ?", String.class,
JdbcRuntimeBindingRepository.storageAliasKey(tenant, scope.binding().getStorageId()));
var owners = jdbc.queryForList("SELECT holder_key, maintenance_id, mount_state FROM ma_storage_owner WHERE storage_key = ? FOR UPDATE", physical);
if (owners.size() != 1 || !"READY".equals(owners.getFirst().get("mount_state"))) {
throw changed();
}
if (owners.getFirst().get("holder_key") != null || owners.getFirst().get("maintenance_id") != null
|| WorkspaceExecutionStore.hasExecutionMembers(jdbc, physical)) {
throw busy();
}
Map<String, Object> request = null;
FileTarget remote = null;
if ("LOCAL".equals(scope.kind())) {
var root = guard.fileRoot(scope.binding());
request = Map.of("protocolVersion", 1, "operation", "probe", "nodeId", "local-files",
"stateDirectory", properties.getRuntimeBroker().getStateDirectory(), "root", root.get("root"),
"allowedRoots", List.of(root.get("root")), "cwdRelative", cwd,
"localRootIdentity", Map.of("device", root.get("device"), "inode", root.get("inode")));
} else {
remote = ssh.freezeFiles(scope.binding());
}
Frozen target = new Frozen(tenant, actor, workspace, cwd, revision, uploadId, path, size, sha256, physical,
properties.getRuntimeBroker().getNodeExecutable(), properties.getRuntimeBroker().getWorkerEntry(), request, remote);
String key = key(tenant, uploadId);
jdbc.update("INSERT INTO ma_operation (operation_key, scope_key, operation_kind, operation_id, tenant_id, workspace_id,"
+ " target_kind, target_key, target_id, storage_key, physical_key, actor_key, request_digest, request_json, state, phase,"
+ " available_at, gmt_create, gmt_modified) VALUES (?, ?, ?, ?, ?, ?, 'STORAGE', ?, ?, ?, ?, ?, ?, ?, 'RUNNING', 'RUNNING', "
+ ManagedSchemaTime.SQL_NOW + " + 60000, " + ManagedSchemaTime.SQL_NOW + ", " + ManagedSchemaTime.SQL_NOW + ")",
key, ManagedSchemaKeys.key(tenant), KIND, uploadId.getBytes(StandardCharsets.UTF_8), tenant.getBytes(StandardCharsets.UTF_8),
workspace.getBytes(StandardCharsets.UTF_8), ManagedSchemaKeys.key(scope.binding().getStorageId()),
scope.binding().getStorageId().getBytes(StandardCharsets.UTF_8),
JdbcRuntimeBindingRepository.storageAliasKey(tenant, scope.binding().getStorageId()), physical,
ManagedWorkspaceRegistry.actorKey(tenant, actor), digest, encode(target));
if (jdbc.update("UPDATE ma_storage_owner SET holder_key = ? WHERE storage_key = ? AND holder_key IS NULL AND maintenance_id IS NULL", key, physical) != 1) {
throw busy();
}
return new Operation(key, digest, target, null, true);
});
}
private UploadStatus exchange(Operation operation, byte[] bytes) {
Frozen target = operation.target();
Map<String, Object> message = new LinkedHashMap<>();
message.put("protocolVersion", 1);
message.put("operation", bytes == null ? "upload-status" : "upload");
message.put("uploadId", target.uploadId());
message.put("path", target.path());
message.put("sizeBytes", target.sizeBytes());
message.put("sha256", target.sha256());
if (bytes != null) {
message.put("data", Base64.getEncoder().encodeToString(bytes));
}
Map<String, Object> response;
if (target.remote() != null) {
response = ssh.upload(target.remote(), message);
} else {
Map<String, Object> request = new LinkedHashMap<>(target.request());
request.put("files", message);
response = WorkspaceFileProcess.upload(target.node(), target.entry(), request);
}
if (!(response.get("files") instanceof Map<?, ?> result)
|| !Long.valueOf(1).equals(BrokerValues.exactLong(result.get("protocolVersion")))
|| !target.uploadId().equals(result.get("uploadId")) || !target.path().equals(result.get("path"))
|| !Long.valueOf(target.sizeBytes()).equals(BrokerValues.exactLong(result.get("sizeBytes")))
|| !target.sha256().equals(result.get("sha256"))
|| !(result.get("state") instanceof String state) || !Set.of("completed", "failed", "pending").contains(state)) {
throw pending();
}
String code = result.get("code") instanceof String value ? value : null;
if ("failed".equals(state) && (code == null || !code.matches("ssh_runtime_file_[a-z_]{1,80}"))) {
throw pending();
}
UploadStatus upload = status(target, state, code);
if ("pending".equals(state)) {
defer(operation.key());
return upload;
}
return finish(operation, upload);
}
private UploadStatus finish(Operation operation, UploadStatus result) {
return transaction.execute(status -> {
Frozen target = operation.target();
WorkspaceStorageKindGuard.lockDomain(jdbc, target.tenant());
Operation current = find(target.tenant(), target.uploadId(), true);
if (current == null || !current.digest().equals(operation.digest())) {
throw pending();
}
if (current.result() != null) {
return current.result();
}
if (jdbc.update("UPDATE ma_storage_owner SET holder_key = NULL WHERE storage_key = ? AND holder_key = ?"
+ " AND maintenance_id IS NULL", target.physical(), operation.key()) != 1) {
throw pending();
}
jdbc.update("UPDATE ma_operation SET state = ?, phase = ?, result_json = ?, completed_at = "
+ ManagedSchemaTime.SQL_NOW + ", gmt_modified = " + ManagedSchemaTime.SQL_NOW + " WHERE operation_key = ? AND state = 'RUNNING'",
result.state(), result.state(), encode(result), operation.key());
return result;
});
}
private Operation find(String tenant, String uploadId, boolean lock) {
var rows = jdbc.queryForList("SELECT request_digest, request_json, result_json FROM ma_operation"
+ " WHERE operation_key = ? AND operation_kind = ?" + (lock ? " FOR UPDATE" : ""), key(tenant, uploadId), KIND);
if (rows.isEmpty()) {
return null;
}
var row = rows.getFirst();
try {
Frozen target = mapper.readValue((String) row.get("request_json"), Frozen.class);
if (!tenant.equals(target.tenant()) || !uploadId.equals(target.uploadId())) {
throw pending();
}
UploadStatus result = row.get("result_json") == null ? null
: mapper.readValue((String) row.get("result_json"), UploadStatus.class);
return new Operation(key(tenant, uploadId), (String) row.get("request_digest"), target, result, false);
} catch (IOException error) {
throw pending();
}
}
private void authorize(Operation operation, String actor) {
if (!operation.target().actor().equals(actor)
|| !registry.accessOf(operation.target().tenant(), actor, operation.target().workspace()).canCreate()) {
throw notFound();
}
}
private void defer(String key) {
jdbc.update("UPDATE ma_operation SET available_at = " + ManagedSchemaTime.SQL_NOW
+ " + 5000 WHERE operation_key = ? AND operation_kind = ? AND state = 'RUNNING'", key, KIND);
}
private String encode(Object value) {
try {
return mapper.writeValueAsString(value);
} catch (com.fasterxml.jackson.core.JsonProcessingException error) {
throw new IllegalStateException(error);
}
}
private String digest(Object value) {
return DigestUtils.sha256Hex(encode(value));
}
private static UploadStatus status(Frozen target, String state, String code) {
return new UploadStatus(target.revision(), target.uploadId(), state, target.path(), target.sizeBytes(), target.sha256(), code);
}
private static byte[] read(InputStream input, long declared) {
if (declared > LIMIT) {
throw tooLarge();
}
try {
byte[] bytes = input.readNBytes(LIMIT + 1);
if (bytes.length > LIMIT) {
throw tooLarge();
}
if (declared >= 0 && declared != bytes.length) {
throw invalid();
}
return bytes;
} catch (IOException error) {
throw invalid();
}
}
private static void validate(String path, String revision, String uploadId) {
validateId(uploadId);
if (revision == null || !revision.matches("[a-f0-9]{64}") || path == null || path.length() > 1024
|| path.codePoints().anyMatch(value -> Character.isISOControl(value) || value >= 0xd800 && value <= 0xdfff)
|| path.contains("\\") || java.util.Arrays.stream(path.split("/", -1)).anyMatch(part -> part.isEmpty()
|| part.equals(".") || part.equals("..") || part.equals(".git") || part.equals(".qwen-child-workspaces")
|| part.startsWith(".qwen-managed-"))) {
throw invalid();
}
}
private static void validateId(String uploadId) {
try {
if (uploadId == null || !UUID.fromString(uploadId).toString().equals(uploadId)) {
throw invalid();
}
} catch (IllegalArgumentException error) {
throw invalid();
}
}
private static String key(String tenant, String uploadId) {
return ManagedSchemaKeys.key(KIND, tenant, uploadId);
}
private static String text(Object value) {
return value instanceof byte[] bytes ? new String(bytes, StandardCharsets.UTF_8) : (String) value;
}
private static ManagedAgentException invalid() {
return new ManagedAgentException(HttpStatus.BAD_REQUEST, "invalid_request", "Workspace upload request is invalid.");
}
private static ManagedAgentException changed() {
return new ManagedAgentException(HttpStatus.CONFLICT, "workspace_file_scope_changed", "Workspace changed; select the upload destination again.");
}
private static ManagedAgentException pending() {
return new ManagedAgentException(HttpStatus.SERVICE_UNAVAILABLE, "workspace_upload_pending", "Workspace upload outcome is pending; query the original upload ID.");
}
private static ManagedAgentException busy() {
return new ManagedAgentException(HttpStatus.CONFLICT, "workspace_busy", "Workspace file access is busy; try again.");
}
private static ManagedAgentException notFound() {
return new ManagedAgentException(HttpStatus.NOT_FOUND, "workspace_upload_not_found", "Workspace upload not found.");
}
private static ManagedAgentException tooLarge() {
return new ManagedAgentException(HttpStatus.PAYLOAD_TOO_LARGE, "workspace_upload_too_large", "Workspace upload exceeds 8 MiB.");
}
}
@@ -182,6 +182,42 @@ public class SshWorkspaceService implements AutoCloseable, AcceptanceEvidence {
return result;
}
public record FileTarget(SshRuntimeNode node, Map<String, Object> request, String qwenVersion) {
}
public FileTarget freezeFiles(ContextBinding binding) {
verifyBinding(binding);
var storage = runtimeStorage(binding);
var node = configuredNode(storage.nodeId()).runtimeNode();
Map<String, Object> request = new LinkedHashMap<>();
request.put("protocolVersion", 1);
request.put("operation", "probe");
request.put("nodeId", node.nodeId());
request.put("stateDirectory", node.stateDirectory());
request.put("root", storage.mountRoot());
request.put("allowedRoots", node.allowedRoots());
request.put("cwdRelative", binding.getCwdRelative());
request.put("identity", storage.registration().get("identity"));
request.put("marker", storage.registration().get("marker"));
return new FileTarget(node, request, (String) storage.registration().get("qwenVersion"));
}
public Map<String, Object> upload(FileTarget target, Map<String, Object> files) {
if (TransactionSynchronizationManager.isActualTransactionActive()) {
throw new IllegalStateException("SSH file access must run outside SQL transactions");
}
if (!configuredNode(target.node().nodeId()).runtimeNode().equals(target.node())) {
throw unavailable();
}
Map<String, Object> request = new LinkedHashMap<>(target.request());
request.put("files", files);
var result = connections().upload(target.node(), request);
if (!target.qwenVersion().equals(result.get("qwenVersion"))) {
throw unavailable();
}
return result;
}
private SshRuntimeSettings.Node admit(String tenant, String actor, String id) {
if (!properties.getRuntimeBroker().getSsh().isEnabled()) {
throw new ManagedAgentException(HttpStatus.NOT_FOUND, "ssh_runtime_disabled", "SSH Workspace registration is disabled.");
@@ -72,6 +72,7 @@ public final class ManagedSchemaContract {
STORAGE_MIGRATION,
CSI_RETIREMENT,
WORKSPACE_RETIREMENT,
WORKSPACE_UPLOAD,
CHANNEL_INGRESS,
SUBMIT_INPUT,
CANCEL
@@ -104,8 +104,16 @@ public final class SshConnectionManager implements AutoCloseable {
/** Sends credentials only through SSH stdin; command arguments contain trusted paths alone. */
public Map<String, Object> bootstrap(SshRuntimeNode node, Map<String, Object> fields) {
return bootstrap(node, fields, false);
}
public Map<String, Object> upload(SshRuntimeNode node, Map<String, Object> fields) {
return bootstrap(node, fields, true);
}
private Map<String, Object> bootstrap(SshRuntimeNode node, Map<String, Object> fields, boolean upload) {
byte[] input = JsonCodec.encode(fields);
if (input.length > RECORD_LIMIT) {
if (input.length > (upload ? WorkspaceFileProcess.MAX_UPLOAD_REQUEST_BYTES : RECORD_LIMIT)) {
throw new IllegalArgumentException("SSH bootstrap request exceeds its size limit");
}
CommandResult exchange;
@@ -121,7 +129,10 @@ public final class SshConnectionManager implements AutoCloseable {
if (fields.containsKey("files")) {
List<String> help = new ArrayList<>(command);
help.set(help.size() - 1, help.getLast() + " --help");
WorkspaceFileProcess.requireCapability(runner.run(List.copyOf(help), new byte[]{'{', '}'}, COMMAND_TIMEOUT));
WorkspaceFileProcess.requireCapability(runner.run(List.copyOf(help), new byte[]{'{', '}'}, COMMAND_TIMEOUT), upload);
}
if (upload) {
command.set(command.size() - 1, command.getLast() + " --workspace-file-upload");
}
exchange = runner.run(List.copyOf(command), input, COMMAND_TIMEOUT);
} finally {
@@ -645,12 +656,17 @@ public final class SshConnectionManager implements AutoCloseable {
throw unavailable(failure);
}
});
try (var stdin = process.getOutputStream()) {
stdin.write(input);
}
CompletableFuture<Void> writer = CompletableFuture.runAsync(() -> {
try (var stdin = started.getOutputStream()) {
stdin.write(input);
} catch (IOException failure) {
throw unavailable(failure);
}
}, task -> Thread.ofVirtual().start(task));
if (!process.waitFor(timeout.toMillis(), TimeUnit.MILLISECONDS)) {
throw unavailable(null);
}
writer.get(1, TimeUnit.SECONDS);
return new CommandResult(process.exitValue(), output.get(1, TimeUnit.SECONDS));
} catch (InterruptedException failure) {
Thread.currentThread().interrupt();
@@ -8,45 +8,70 @@ import java.time.Duration;
import java.util.List;
import java.util.Map;
/** Bounded, read-only bootstrap exchange using the configured local CLI. */
/** Bounded Workspace file exchanges using the configured local CLI. */
public final class WorkspaceFileProcess {
public static final int MAX_UPLOAD_BYTES = 8 * 1024 * 1024;
public static final int MAX_UPLOAD_REQUEST_BYTES = 12 * 1024 * 1024;
private WorkspaceFileProcess() {
}
public static Map<String, Object> read(String node, String entry, Map<String, Object> request) {
return exchange(node, entry, request, false);
}
public static Map<String, Object> upload(String node, String entry, Map<String, Object> request) {
return exchange(node, entry, request, true);
}
private static Map<String, Object> exchange(String node, String entry, Map<String, Object> request, boolean upload) {
if (node == null || node.isBlank() || entry == null || entry.isBlank()) {
throw unsupported();
}
byte[] input = JsonCodec.encode(request);
if (input.length > 65536) {
if (input.length > (upload ? MAX_UPLOAD_REQUEST_BYTES : 65536)) {
throw unsupported();
}
var help = SshConnectionManager.runCommand(List.of(node, entry, "managed-runtime-bootstrap", "--help"),
new byte[]{'{', '}'}, Duration.ofSeconds(15));
requireCapability(help);
var exchange = SshConnectionManager.runCommand(List.of(node, entry, "managed-runtime-bootstrap"),
requireCapability(help, upload);
List<String> command = upload
? List.of(node, entry, "managed-runtime-bootstrap", "--workspace-file-upload")
: List.of(node, entry, "managed-runtime-bootstrap");
var exchange = SshConnectionManager.runCommand(command,
input, Duration.ofSeconds(15));
if (exchange.exitCode() != 0 || exchange.stdout().length > 65536) {
throw unsupported();
throw upload ? uncertain() : unsupported();
}
Map<String, Object> envelope;
try {
envelope = JsonCodec.parseObject(exchange.stdout(), "Workspace file response");
} catch (IllegalArgumentException error) {
} catch (RuntimeException error) {
if (upload) {
throw uncertain();
}
throw unsupported();
}
if (Long.valueOf(1).equals(BrokerValues.exactLong(envelope.get("protocolVersion")))
&& Boolean.TRUE.equals(envelope.get("ok")) && envelope.get("result") instanceof Map<?, ?> result) {
return JsonCodec.parseObject(JsonCodec.encode(result), "Workspace file result");
}
throw new RuntimeBrokerException(409, "workspace_file_unavailable", "Workspace file cannot be read.", false);
throw upload ? uncertain()
: new RuntimeBrokerException(409, "workspace_file_unavailable", "Workspace file cannot be read.", false);
}
public static RuntimeBrokerException unsupported() {
return new RuntimeBrokerException(409, "workspace_files_unsupported", "Workspace file access requires an updated CLI.", false);
}
private static RuntimeBrokerException uncertain() {
return new RuntimeBrokerException(503, "workspace_upload_pending", "Original Workspace upload must be reconciled.", true);
}
static void requireCapability(SshConnectionManager.CommandResult help) {
requireCapability(help, false);
}
static void requireCapability(SshConnectionManager.CommandResult help, boolean upload) {
try {
if (help.exitCode() != 0 || help.stdout().length > 65536) {
throw unsupported();
@@ -54,7 +79,7 @@ public final class WorkspaceFileProcess {
var document = JsonCodec.parseObject(help.stdout(), "Workspace file capability");
if (!Long.valueOf(1).equals(BrokerValues.exactLong(document.get("protocolVersion")))
|| !(document.get("capabilities") instanceof Map<?, ?> capabilities)
|| !Long.valueOf(1).equals(BrokerValues.exactLong(capabilities.get("workspaceFiles")))) {
|| !Long.valueOf(1).equals(BrokerValues.exactLong(capabilities.get(upload ? "workspaceFileUpload" : "workspaceFiles")))) {
throw unsupported();
}
} catch (RuntimeException error) {
@@ -684,7 +684,7 @@ CREATE TABLE ma_operation (
CONSTRAINT uq_ma_operation_request UNIQUE (scope_key, operation_kind, target_key, actor_key, idempotency_digest),
CONSTRAINT ck_ma_operation_generation CHECK (claim_generation >= 0 AND attempt_count >= 0 AND payload_version > 0),
CONSTRAINT ck_ma_operation_mode CHECK ((CASE execution_mode WHEN 'SYNC' THEN 4 WHEN 'ASYNC' THEN 5 ELSE -1 END = CHAR_LENGTH(execution_mode))),
CONSTRAINT ck_ma_operation_kind CHECK ((CASE operation_kind WHEN 'CLOSE' THEN 5 WHEN 'ARCHIVE' THEN 7 WHEN 'DELETE' THEN 6 WHEN 'UNARCHIVE' THEN 9 WHEN 'RENAME' THEN 6 WHEN 'CWD_CHANGE' THEN 10 WHEN 'CONTEXT_COMPRESSION' THEN 19 WHEN 'ACTION_RESPONSE' THEN 15 WHEN 'TASK_CANCEL' THEN 11 WHEN 'CHILD_SESSION_START' THEN 19 WHEN 'CHILD_SESSION_CLOSE' THEN 19 WHEN 'CHILD_WORKSPACE' THEN 15 WHEN 'OUTPUT_UPLOAD' THEN 13 WHEN 'OUTPUT_VERIFY' THEN 13 WHEN 'OUTPUT_FINISH' THEN 13 WHEN 'OBJECT_PUT' THEN 10 WHEN 'TOOL_RESULT_PROJECTION' THEN 22 WHEN 'SESSION_RESOURCE_GC' THEN 19 WHEN 'CAPTURE_GC' THEN 10 WHEN 'WORKSPACE_OPERATOR_RECOVERY' THEN 27 WHEN 'WORKSPACE_RECOVERY' THEN 18 WHEN 'STORAGE_MIGRATION' THEN 17 WHEN 'CSI_RETIREMENT' THEN 14 WHEN 'CHANNEL_INGRESS' THEN 15 WHEN 'WORKSPACE_CREATE' THEN 16 WHEN 'SUBMIT_INPUT' THEN 12 WHEN 'CANCEL' THEN 6 WHEN 'WORKSPACE_RETIREMENT' THEN 20 ELSE -1 END = CHAR_LENGTH(operation_kind))),
CONSTRAINT ck_ma_operation_kind CHECK ((CASE operation_kind WHEN 'CLOSE' THEN 5 WHEN 'ARCHIVE' THEN 7 WHEN 'DELETE' THEN 6 WHEN 'UNARCHIVE' THEN 9 WHEN 'RENAME' THEN 6 WHEN 'CWD_CHANGE' THEN 10 WHEN 'CONTEXT_COMPRESSION' THEN 19 WHEN 'ACTION_RESPONSE' THEN 15 WHEN 'TASK_CANCEL' THEN 11 WHEN 'CHILD_SESSION_START' THEN 19 WHEN 'CHILD_SESSION_CLOSE' THEN 19 WHEN 'CHILD_WORKSPACE' THEN 15 WHEN 'OUTPUT_UPLOAD' THEN 13 WHEN 'OUTPUT_VERIFY' THEN 13 WHEN 'OUTPUT_FINISH' THEN 13 WHEN 'OBJECT_PUT' THEN 10 WHEN 'TOOL_RESULT_PROJECTION' THEN 22 WHEN 'SESSION_RESOURCE_GC' THEN 19 WHEN 'CAPTURE_GC' THEN 10 WHEN 'WORKSPACE_OPERATOR_RECOVERY' THEN 27 WHEN 'WORKSPACE_RECOVERY' THEN 18 WHEN 'STORAGE_MIGRATION' THEN 17 WHEN 'CSI_RETIREMENT' THEN 14 WHEN 'CHANNEL_INGRESS' THEN 15 WHEN 'WORKSPACE_CREATE' THEN 16 WHEN 'SUBMIT_INPUT' THEN 12 WHEN 'CANCEL' THEN 6 WHEN 'WORKSPACE_RETIREMENT' THEN 20 WHEN 'WORKSPACE_UPLOAD' THEN 16 ELSE -1 END = CHAR_LENGTH(operation_kind))),
progress_format VARCHAR(64),
progress_version BIGINT,
progress_stage VARCHAR(16),
@@ -2,7 +2,7 @@
"openapi": "3.1.0",
"info": {
"title": "Qwen Managed Agent Public and WebShell API",
"version": "1.93.0",
"version": "1.93.1",
"description": "Canonical first-release contract for the public Managed Agent resources and the WebShell adapter. Implemented routes and fields describe the current matching Server, Broker, Harness and WebShell build. Partial entries have remaining implementation or contract-test work; planned entries are not current server capabilities and must be excluded from production SDKs. Turn submission and cancellation always return the durable operation identifier and its persisted admission and delivery state. Session-list cursors are bound to the normalized archive state and title query. Earlier development API shapes, checkpoints and database catalogs are unsupported; initialize fresh state for this build. Runtime feature capabilities and deployment policy still govern optional execution modes."
},
"servers": [
@@ -6980,6 +6980,178 @@
}
}
}
},
"/api/agent/web-shell/v1/workspaces/files/upload": {
"post": {
"tags": ["WebShell"],
"operationId": "webShellWorkspaceFilesUpload",
"x-qwen-implementation-status": "implemented",
"summary": "Upload one binary file to a persisted Workspace",
"description": "Requires a trusted actor with OPERATOR or OWNER access, enabled Workspace files, verified LOCAL or SSH_LOCAL storage and an updated CLI. Uploads are limited to 8 MiB, require an existing parent directory and never overwrite an occupied path. A durable physical storage hold excludes tool execution and retirement until the original upload has a definite terminal result. Repeating the same upload ID reconciles its original command without replaying mutation; changed requests conflict. CSI is unsupported and never falls back to a primary Workspace. An uncertain result returns 503 workspace_upload_pending and must be queried by the original upload ID.",
"parameters": [
{
"name": "workspaceId",
"in": "query",
"required": true,
"schema": {
"type": "string",
"minLength": 1,
"maxLength": 128
}
},
{
"name": "cwdRelative",
"in": "query",
"required": true,
"schema": {
"type": "string",
"minLength": 1,
"maxLength": 1024
}
},
{
"name": "path",
"in": "query",
"required": true,
"schema": {
"type": "string",
"minLength": 1,
"maxLength": 1024
}
},
{
"name": "expectedScopeRevision",
"in": "query",
"required": true,
"schema": {
"type": "string",
"minLength": 1,
"maxLength": 64
}
},
{
"name": "uploadId",
"in": "query",
"required": true,
"schema": {
"type": "string",
"minLength": 1,
"maxLength": 36,
"format": "uuid"
}
}
],
"requestBody": {
"required": true,
"content": {
"application/octet-stream": {
"schema": {
"type": "string",
"format": "binary"
}
}
}
},
"responses": {
"200": {
"description": "Workspace upload result.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/WebShellWorkspaceFileUploadResult"
}
}
}
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"403": {
"$ref": "#/components/responses/Forbidden"
},
"404": {
"$ref": "#/components/responses/NotFound"
},
"409": {
"$ref": "#/components/responses/Conflict"
},
"503": {
"$ref": "#/components/responses/Unavailable"
},
"413": {
"description": "The upload exceeds the 8 MiB binary limit.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
}
},
"415": {
"description": "Uploads require unencoded application/octet-stream bytes.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorEnvelope"
}
}
}
}
}
}
},
"/api/agent/web-shell/v1/workspaces/files/uploads/query": {
"post": {
"tags": ["WebShell"],
"operationId": "webShellWorkspaceFilesUploadQuery",
"x-qwen-implementation-status": "implemented",
"summary": "Reconcile the original Workspace upload",
"description": "Only the admitting actor with a current Workspace operator grant can query the upload. Status checks use the frozen original storage target and never execute an upload or select a replacement Runtime. Reconciliation remains available through storage fencing; pending commands retain their durable hold. The server also reconciles pending uploads after client cancellation or restart.",
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/WebShellWorkspaceFileUploadQueryRequest"
}
}
}
},
"responses": {
"200": {
"description": "Workspace upload result.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/WebShellWorkspaceFileUploadStatus"
}
}
}
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"403": {
"$ref": "#/components/responses/Forbidden"
},
"404": {
"$ref": "#/components/responses/NotFound"
},
"409": {
"$ref": "#/components/responses/Conflict"
},
"503": {
"$ref": "#/components/responses/Unavailable"
}
}
}
}
},
"components": {
@@ -14320,6 +14492,12 @@
},
"truncated": {
"type": "boolean"
},
"maxUploadBytes": {
"type": "integer",
"minimum": 8388608,
"maximum": 8388608,
"description": "Present only when the selected CLI supports durable Workspace uploads."
}
}
},
@@ -15846,6 +16024,95 @@
"$ref": "#/components/schemas/ManagedExtensionApplySelection"
}
}
},
"WebShellWorkspaceFileUploadResult": {
"type": "object",
"additionalProperties": false,
"required": [
"scopeRevision",
"uploadId",
"path",
"sizeBytes",
"sha256"
],
"properties": {
"scopeRevision": {
"type": "string",
"pattern": "^[a-f0-9]{64}$"
},
"uploadId": {
"type": "string",
"format": "uuid"
},
"path": {
"type": "string",
"minLength": 1,
"maxLength": 1024
},
"sizeBytes": {
"type": "integer",
"minimum": 0,
"maximum": 8388608
},
"sha256": {
"type": "string",
"pattern": "^[a-f0-9]{64}$"
}
}
},
"WebShellWorkspaceFileUploadStatus": {
"type": "object",
"additionalProperties": false,
"required": [
"scopeRevision",
"uploadId",
"path",
"sizeBytes",
"sha256",
"state"
],
"properties": {
"scopeRevision": {
"type": "string",
"pattern": "^[a-f0-9]{64}$"
},
"uploadId": {
"type": "string",
"format": "uuid"
},
"path": {
"type": "string",
"minLength": 1,
"maxLength": 1024
},
"sizeBytes": {
"type": "integer",
"minimum": 0,
"maximum": 8388608
},
"sha256": {
"type": "string",
"pattern": "^[a-f0-9]{64}$"
},
"state": {
"type": "string",
"enum": ["completed", "failed", "pending"]
},
"code": {
"type": "string"
}
}
},
"WebShellWorkspaceFileUploadQueryRequest": {
"type": "object",
"additionalProperties": false,
"required": ["uploadId"],
"properties": {
"uploadId": {
"type": "string",
"format": "uuid"
}
}
}
}
}
@@ -175,6 +175,22 @@ class ManagedAgentApiContractTest {
entry(ApiModels.WebShellToolResultRequest.class, List.of("WebShellToolResultRequest")),
entry(SshWorkspaceController.DirectoryRequest.class, List.of("WebShellWorkspaceDirectoryRequest")),
entry(WorkspaceDiscoveryController.DirectoryRequest.class, List.of("WebShellWorkspaceDirectoryQueryRequest")),
entry(com.alibaba.qwen.code.managedagent.api.WorkspaceFileController.QueryRequest.class,
List.of("WebShellWorkspaceFileQueryRequest")),
entry(com.alibaba.qwen.code.managedagent.api.WorkspaceFileController.ReadRequest.class,
List.of("WebShellWorkspaceFileReadRequest")),
entry(com.alibaba.qwen.code.managedagent.api.WorkspaceFileController.UploadQueryRequest.class,
List.of("WebShellWorkspaceFileUploadQueryRequest")),
entry(com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileStore.QueryResult.class,
List.of("WebShellWorkspaceFilePage")),
entry(com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileStore.FileResult.class,
List.of("WebShellWorkspaceFileSnapshot")),
entry(com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileStore.Entry.class,
List.of("WebShellWorkspaceFileEntry")),
entry(com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileUploadStore.UploadResult.class,
List.of("WebShellWorkspaceFileUploadResult")),
entry(com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileUploadStore.UploadStatus.class,
List.of("WebShellWorkspaceFileUploadStatus")),
entry(SshWorkspaceController.CreateRequest.class, List.of("WebShellWorkspaceRegisterRequest")),
entry(SshWorkspaceController.OperationRequest.class, List.of("WebShellWorkspaceOperationRequest")),
entry(SshWorkspaceService.NodeSummary.class, List.of("WebShellRuntimeNode")),
@@ -1459,6 +1475,16 @@ class ManagedAgentApiContractTest {
post(WEB_SHELL + "/workspaces/files/read")
.header(TENANT, workspaceTenant).principal(actor),
"{\"workspaceId\":\"ws-default\",\"cwdRelative\":\".\",\"path\":\"a.txt\",\"expectedScopeRevision\":\"old\"}");
exchange(drift, "webShellWorkspaceFilesUpload", 409,
post(WEB_SHELL + "/workspaces/files/upload")
.header(TENANT, workspaceTenant).principal(actor)
.param("workspaceId", "ws-default").param("cwdRelative", ".").param("path", "file.bin")
.param("expectedScopeRevision", "a".repeat(64)).param("uploadId", UUID.randomUUID().toString())
.contentType(MediaType.APPLICATION_OCTET_STREAM).content(new byte[] {0}), null);
exchange(drift, "webShellWorkspaceFilesUploadQuery", 404,
post(WEB_SHELL + "/workspaces/files/uploads/query")
.header(TENANT, workspaceTenant).principal(actor),
"{\"uploadId\":\"" + UUID.randomUUID() + "\"}");
String publicBoundId = json(exchange(drift, "createSession", 202,
post("/v1/agents/sessions").header(TENANT, workspaceTenant)
.principal(actor).header(IDEMPOTENCY_KEY, "bound-public"),
@@ -400,6 +400,14 @@ public enum SurfaceRegistry {
"/api/agent/web-shell/v1/workspaces/files/read",
Surface.WEBSHELL, RuleClass.WORKSPACE_FILES,
EnumSet.of(Capability.WORKSPACE_FILES_READ)),
WEBSHELL_WORKSPACE_FILES_UPLOAD(Method.POST,
"/api/agent/web-shell/v1/workspaces/files/upload",
Surface.WEBSHELL, RuleClass.WORKSPACE_FILES,
EnumSet.of(Capability.WORKSPACE_FILES_UPLOAD)),
WEBSHELL_WORKSPACE_FILES_UPLOAD_QUERY(Method.POST,
"/api/agent/web-shell/v1/workspaces/files/uploads/query",
Surface.WEBSHELL, RuleClass.WORKSPACE_FILES,
EnumSet.of(Capability.WORKSPACE_FILES_UPLOAD_QUERY)),
WEBSHELL_SESSION_OPTIONS(Method.POST,
"/api/agent/web-shell/v1/session-options/query",
Surface.WEBSHELL, RuleClass.SESSION_OPTIONS,
@@ -701,6 +709,8 @@ public enum SurfaceRegistry {
WORKSPACE_DIRECTORIES,
WORKSPACE_FILES_QUERY,
WORKSPACE_FILES_READ,
WORKSPACE_FILES_UPLOAD,
WORKSPACE_FILES_UPLOAD_QUERY,
SESSION_OPTIONS,
SESSION_APPROVAL,
SESSION_APPROVAL_CHANGE,
@@ -78,6 +78,10 @@ class SurfaceRegistryGateTest {
.isEqualTo(SurfaceRegistry.RuleClass.WORKSPACE_FILES);
assertThat(SurfaceRegistry.WEBSHELL_WORKSPACE_FILES_READ.ruleClass())
.isEqualTo(SurfaceRegistry.RuleClass.WORKSPACE_FILES);
assertThat(SurfaceRegistry.WEBSHELL_WORKSPACE_FILES_UPLOAD.ruleClass())
.isEqualTo(SurfaceRegistry.RuleClass.WORKSPACE_FILES);
assertThat(SurfaceRegistry.WEBSHELL_WORKSPACE_FILES_UPLOAD_QUERY.ruleClass())
.isEqualTo(SurfaceRegistry.RuleClass.WORKSPACE_FILES);
}
@Test
@@ -0,0 +1,92 @@
package com.alibaba.qwen.code.managedagent.api;
import com.alibaba.qwen.code.managedagent.model.TenantContext;
import com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileStore;
import com.alibaba.qwen.code.managedagent.persistence.WorkspaceFileUploadStore;
import org.apache.commons.codec.digest.DigestUtils;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import java.io.InputStream;
import java.util.UUID;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verifyNoInteractions;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.header;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
class WorkspaceFileControllerTest {
WorkspaceFileUploadStore uploads;
MockMvc mvc;
@BeforeEach
void setup() {
uploads = mock(WorkspaceFileUploadStore.class);
mvc = MockMvcBuilders.standaloneSetup(new WorkspaceFileController(mock(WorkspaceFileStore.class), uploads))
.setControllerAdvice(new ApiExceptionHandler())
.setCustomArgumentResolvers(new TenantContextArgumentResolver()).build();
}
@Test
void rawUploadPreservesBinaryBytesAndReturnsTheScopedResult() throws Exception {
byte[] bytes = new byte[] {0, (byte) 0xff, 1};
String id = UUID.randomUUID().toString();
when(uploads.upload(anyString(), anyString(), anyString(), anyString(), anyString(), anyString(), anyString(), any(), anyLong()))
.thenAnswer(call -> {
assertThat(call.<String>getArgument(0)).isEqualTo("tenant");
assertThat(call.<String>getArgument(1)).isEqualTo("actor");
assertThat(call.<InputStream>getArgument(7).readAllBytes()).isEqualTo(bytes);
return new WorkspaceFileUploadStore.UploadResult("a".repeat(64), id, "binary.bin", bytes.length, DigestUtils.sha256Hex(bytes));
});
mvc.perform(post("/api/agent/web-shell/v1/workspaces/files/upload")
.requestAttr(TenantContextFilter.ATTRIBUTE, new TenantContext("tenant", "actor"))
.param("workspaceId", "workspace").param("cwdRelative", ".").param("path", "binary.bin")
.param("expectedScopeRevision", "a".repeat(64)).param("uploadId", id)
.contentType(MediaType.APPLICATION_OCTET_STREAM).content(bytes))
.andExpect(status().isOk()).andExpect(header().string("Cache-Control", "no-store"))
.andExpect(jsonPath("$.uploadId").value(id)).andExpect(jsonPath("$.sizeBytes").value(3));
}
@Test
void uploadsRequireATrustedActorAndUnencodedOctetStream() throws Exception {
for (boolean actor : new boolean[] {false, true}) {
mvc.perform(post("/api/agent/web-shell/v1/workspaces/files/upload")
.requestAttr(TenantContextFilter.ATTRIBUTE, new TenantContext("tenant", actor ? "actor" : null))
.param("workspaceId", "workspace").param("cwdRelative", ".").param("path", "binary.bin")
.param("expectedScopeRevision", "a".repeat(64)).param("uploadId", UUID.randomUUID().toString())
.header("Content-Encoding", "gzip").contentType(MediaType.APPLICATION_OCTET_STREAM).content(new byte[] {0}))
.andExpect(status().is(actor ? 415 : 401));
}
mvc.perform(post("/api/agent/web-shell/v1/workspaces/files/upload")
.requestAttr(TenantContextFilter.ATTRIBUTE, new TenantContext("tenant", "actor"))
.contentType(MediaType.APPLICATION_JSON).content("{}"))
.andExpect(status().isUnsupportedMediaType());
verifyNoInteractions(uploads);
}
@Test
void statusQueryReturnsPendingMetadataAndRejectsUnknownFields() throws Exception {
String id = UUID.randomUUID().toString();
when(uploads.query("tenant", "actor", id)).thenReturn(new WorkspaceFileUploadStore.UploadStatus(
"a".repeat(64), id, "pending", "binary.bin", 0, DigestUtils.sha256Hex(new byte[0]), null));
mvc.perform(post("/api/agent/web-shell/v1/workspaces/files/uploads/query")
.requestAttr(TenantContextFilter.ATTRIBUTE, new TenantContext("tenant", "actor"))
.contentType(MediaType.APPLICATION_JSON).content("{\"uploadId\":\"" + id + "\"}"))
.andExpect(status().isOk()).andExpect(jsonPath("$.state").value("pending"))
.andExpect(jsonPath("$.path").value("binary.bin"));
mvc.perform(post("/api/agent/web-shell/v1/workspaces/files/uploads/query")
.requestAttr(TenantContextFilter.ATTRIBUTE, new TenantContext("tenant", "actor"))
.contentType(MediaType.APPLICATION_JSON).content("{\"uploadId\":\"" + id + "\",\"root\":\"/tmp\"}"))
.andExpect(status().isBadRequest());
}
}
@@ -0,0 +1,35 @@
package com.alibaba.qwen.code.managedagent.persistence;
import org.junit.jupiter.api.AfterEach;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import javax.sql.DataSource;
import java.util.UUID;
class WorkspaceFileUploadMySqlIT extends WorkspaceFileUploadStoreTest {
private JdbcTemplate admin;
private String schema;
@Override
protected DataSource source() {
String url = System.getProperty("mysql.url");
if (url == null || !url.matches("jdbc:mysql://[^/]+/[^?]+(?:\\?.*)?")) {
throw new IllegalArgumentException("A MySQL test database URL is required");
}
String user = System.getProperty("mysql.user");
String password = System.getProperty("mysql.password", "");
admin = new ManagedSchemaJdbcTemplate(new DriverManagerDataSource(url, user, password));
schema = "workspace_upload_" + UUID.randomUUID().toString().replace("-", "");
admin.execute("CREATE DATABASE " + schema);
return new DriverManagerDataSource(url.replaceFirst("/[^/?]+(?=\\?|$)", "/" + schema), user, password);
}
@AfterEach
void dropSchema() {
if (admin != null && schema != null) {
admin.execute("DROP DATABASE IF EXISTS " + schema);
}
}
}
@@ -0,0 +1,351 @@
package com.alibaba.qwen.code.managedagent.persistence;
import com.alibaba.qwen.code.managedagent.config.ManagedAgentProperties;
import com.alibaba.qwen.code.managedagent.exception.ManagedAgentException;
import com.alibaba.qwen.code.managedagent.workspace.SshWorkspaceService;
import com.alibaba.qwen.code.runtimebroker.RuntimeBrokerException;
import com.alibaba.qwen.code.runtimebroker.managedschema.ManagedSchema;
import com.alibaba.qwen.code.runtimebroker.provisioning.WorkspaceFileProcess;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.commons.codec.digest.DigestUtils;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.jdbc.datasource.DriverManagerDataSource;
import org.springframework.transaction.support.TransactionSynchronizationManager;
import javax.sql.DataSource;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Clock;
import java.time.Duration;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.awaitility.Awaitility.await;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
class WorkspaceFileUploadStoreTest {
@TempDir
Path temp;
JdbcTemplate jdbc;
ManagedAgentProperties properties;
ManagedWorkspaceRegistry registry;
WorkspaceStorageGuard guard;
SshWorkspaceService ssh;
WorkspaceFileStore files;
WorkspaceFileUploadStore uploads;
DataSourceTransactionManager manager;
ObjectMapper mapper;
String workspace;
protected DataSource source() {
return new DriverManagerDataSource("jdbc:h2:mem:uploads-" + UUID.randomUUID()
+ ";MODE=MySQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE", "sa", "");
}
@BeforeEach
void setUp() throws Exception {
DataSource source = source();
ManagedSchema.initialize(source);
jdbc = new ManagedSchemaJdbcTemplate(source);
properties = new ManagedAgentProperties();
properties.getHarness().setWorkspaceFilesEnabled(true);
properties.getRuntimeBroker().setWorkspaceCreationRoot(Files.createDirectory(temp.resolve("created")).toRealPath().toString());
properties.getRuntimeBroker().setVerifiedWorkspaceRecoveryEnabled(true);
properties.getRuntimeBroker().setNodeExecutable("node");
properties.getRuntimeBroker().setWorkerEntry("/original/cli.js");
manager = new DataSourceTransactionManager(source);
registry = new ManagedWorkspaceRegistry(jdbc);
guard = new WorkspaceStorageGuard(jdbc, manager, properties,
path -> new WorkspaceStorageGuard.Identity(path.toString(), "test-host", "1", "2", "2026-10-15T00:00:00.123456789Z"));
mapper = new ObjectMapper();
workspace = new WorkspaceCreationStore(jdbc, manager, guard, registry, properties, mapper, Clock.systemUTC())
.create("tenant", "actor", "Uploads", "create").workspaceId();
ssh = mock(SshWorkspaceService.class);
files = new WorkspaceFileStore(registry, guard, ssh, properties, jdbc, manager, mapper);
uploads = store();
}
WorkspaceFileUploadStore store() {
return new WorkspaceFileUploadStore(files, registry, guard, ssh, properties, jdbc, manager, mapper, Runnable::run);
}
String revision() {
return files.scope("tenant", "actor", workspace, ".").revision();
}
WorkspaceFileUploadStore.UploadResult upload(String id, String path, String revision, byte[] bytes) {
return uploads.upload("tenant", "actor", workspace, ".", path, revision, id,
new ByteArrayInputStream(bytes), bytes.length);
}
@SuppressWarnings("unchecked")
static Map<String, Object> response(Map<String, Object> request, String state, String code) {
Map<String, Object> file = new LinkedHashMap<>((Map<String, Object>) request.get("files"));
file.remove("operation");
file.remove("data");
file.put("state", state);
if (code != null) {
file.put("code", code);
}
return Map.of("files", file);
}
@Test
void persistsOnlyMetadataHoldsStorageOutsideTransactionsAndReplaysCompletedResult() {
String id = UUID.randomUUID().toString();
String revision = revision();
byte[] bytes = new byte[] {0, (byte) 0xff, 1, 2};
try (var process = mockStatic(WorkspaceFileProcess.class)) {
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any())).thenAnswer(call -> {
assertThat(TransactionSynchronizationManager.isActualTransactionActive()).isFalse();
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isEqualTo(1);
assertThat(jdbc.queryForObject("SELECT request_json FROM ma_operation WHERE operation_kind = 'WORKSPACE_UPLOAD'", String.class))
.doesNotContain("data\":");
assertThatThrownBy(() -> guard.fence("tenant", storage(), 1, UUID.randomUUID().toString()))
.isInstanceOf(RuntimeException.class);
return response(call.getArgument(2), "completed", null);
});
var result = upload(id, "二进制.bin", revision, bytes);
assertThat(result.sha256()).isEqualTo(DigestUtils.sha256Hex(bytes));
assertThat(result.sizeBytes()).isEqualTo(4);
assertThat(result.scopeRevision()).isEqualTo(revision);
assertThat(upload(id, "二进制.bin", revision, bytes)).isEqualTo(result);
assertThat(uploads.query("tenant", "actor", id).state()).isEqualTo("completed");
process.verify(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any()));
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isZero();
}
}
@Test
void unknownOutcomeKeepsHoldAndRestartQueriesOnlyTheOriginalTargetThroughFencing() {
String id = UUID.randomUUID().toString();
String revision = revision();
AtomicInteger calls = new AtomicInteger();
try (var process = mockStatic(WorkspaceFileProcess.class)) {
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any())).thenAnswer(call -> {
Map<String, Object> request = call.getArgument(2);
Map<?, ?> message = (Map<?, ?>) request.get("files");
if (calls.getAndIncrement() == 0) {
assertThat(message.get("operation")).isEqualTo("upload");
throw new RuntimeBrokerException(503, "ssh_runtime_unavailable", "unknown", true);
}
assertThat(call.<String>getArgument(1)).isEqualTo("/original/cli.js");
assertThat(message.get("operation")).isEqualTo("upload-status");
assertThat(message.containsKey("data")).isFalse();
return response(request, "completed", null);
});
assertThatThrownBy(() -> upload(id, "file.bin", revision, new byte[] {1}))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> {
assertThat(error.getStatus().value()).isEqualTo(503);
assertThat(error.getCode()).isEqualTo("workspace_upload_pending");
});
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isEqualTo(1);
jdbc.update("UPDATE ma_storage SET fence_operation_id = 'retirement'");
jdbc.update("UPDATE ma_workspace SET workspace_generation = workspace_generation + 1");
properties.getRuntimeBroker().setWorkerEntry("/replacement/cli.js");
uploads = store();
var result = uploads.query("tenant", "actor", id);
assertThat(result.state()).isEqualTo("completed");
assertThat(result.scopeRevision()).isEqualTo(revision);
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isZero();
}
}
@Test
void rejectsChangedRequestsAndNeverReplaysMutationWhilePending() {
String id = UUID.randomUUID().toString();
String revision = revision();
AtomicInteger calls = new AtomicInteger();
try (var process = mockStatic(WorkspaceFileProcess.class)) {
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any())).thenAnswer(call -> {
Map<String, Object> request = call.getArgument(2);
assertThat(((Map<?, ?>) request.get("files")).get("operation"))
.isEqualTo(calls.getAndIncrement() == 0 ? "upload" : "upload-status");
return response(request, "pending", null);
});
assertThatThrownBy(() -> upload(id, "same.bin", revision, new byte[] {1})).isInstanceOf(ManagedAgentException.class);
assertThatThrownBy(() -> upload(id, "same.bin", revision, new byte[] {2}))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getCode()).isEqualTo("workspace_upload_conflict"));
assertThatThrownBy(() -> upload(id, "same.bin", revision, new byte[] {1})).isInstanceOf(ManagedAgentException.class);
assertThatThrownBy(() -> upload(UUID.randomUUID().toString(), "other.bin", revision, new byte[] {1}))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getCode()).isEqualTo("workspace_busy"));
assertThat(calls).hasValue(2);
}
}
@Test
void oldCliAndDefiniteFailureReleaseOnlyTheirOwnHold() {
String revision = revision();
try (var process = mockStatic(WorkspaceFileProcess.class)) {
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any())).thenThrow(
new RuntimeBrokerException(409, "workspace_files_unsupported", "updated CLI required", false));
assertThatThrownBy(() -> upload(UUID.randomUUID().toString(), "old.bin", revision, new byte[0]))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getCode()).isEqualTo("workspace_files_unsupported"));
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isZero();
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any()))
.thenAnswer(call -> response(call.getArgument(2), "failed", "ssh_runtime_file_exists"));
assertThatThrownBy(() -> upload(UUID.randomUUID().toString(), "exists.bin", revision, new byte[0]))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getCode()).isEqualTo("ssh_runtime_file_exists"));
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isZero();
}
}
@Test
void actorGrantAndScopeChecksPrecedeReadingTheBody() {
InputStream input = new InputStream() {
@Override
public int read() {
throw new AssertionError("unauthorized body must not be read");
}
};
String revision = revision();
for (String actor : List.of("stranger", "actor")) {
jdbc.update("UPDATE ma_workspace_access SET role = 'READER'");
assertThatThrownBy(() -> uploads.upload("tenant", actor, workspace, ".", "file.bin", revision,
UUID.randomUUID().toString(), input, 0)).isInstanceOf(ManagedAgentException.class);
}
jdbc.update("UPDATE ma_workspace_access SET role = 'OPERATOR'");
assertThatThrownBy(() -> uploads.upload("tenant", "actor", workspace, ".", "file.bin", "0".repeat(64),
UUID.randomUUID().toString(), input, 0)).isInstanceOf(ManagedAgentException.class);
properties.getHarness().setWorkspaceFilesEnabled(false);
assertThatThrownBy(() -> uploads.upload("tenant", "actor", workspace, ".", "file.bin", revision,
UUID.randomUUID().toString(), input, 0)).isInstanceOf(ManagedAgentException.class);
}
@Test
void rejectsOversizeTraversalAndUnsupportedStorageWithoutSubprocesses() {
String revision = revision();
try (var process = mockStatic(WorkspaceFileProcess.class)) {
assertThatThrownBy(() -> uploads.upload("tenant", "actor", workspace, ".", "large.bin", revision,
UUID.randomUUID().toString(), InputStream.nullInputStream(), WorkspaceFileProcess.MAX_UPLOAD_BYTES + 1L))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getStatus().value()).isEqualTo(413));
assertThatThrownBy(() -> upload(UUID.randomUUID().toString(), "../escape", revision, new byte[0]))
.isInstanceOf(ManagedAgentException.class);
for (String reserved : List.of(".git", ".qwen-managed-cache", ".qwen-child-workspaces")) {
assertThatThrownBy(() -> uploads.upload("tenant", "actor", workspace, reserved, "file.bin", revision,
UUID.randomUUID().toString(), InputStream.nullInputStream(), 0)).isInstanceOf(ManagedAgentException.class);
}
jdbc.update("UPDATE ma_storage SET storage_kind = 'CSI'");
String csiRevision = revision();
assertThatThrownBy(() -> upload(UUID.randomUUID().toString(), "csi.bin", csiRevision, new byte[0]))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getCode()).isEqualTo("workspace_files_unsupported"));
process.verifyNoInteractions();
}
}
@Test
void malformedResponseRetainsHoldAndBackgroundReconciliationSettlesWithoutMutation() {
String id = UUID.randomUUID().toString();
String revision = revision();
try (var process = mockStatic(WorkspaceFileProcess.class)) {
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any())).thenAnswer(call -> {
Map<String, Object> request = call.getArgument(2);
Map<?, ?> file = (Map<?, ?>) request.get("files");
if ("upload".equals(file.get("operation"))) {
return Map.of("files", Map.of("state", "completed", "sha256", "wrong"));
}
return response(request, "failed", "ssh_runtime_file_unavailable");
});
assertThatThrownBy(() -> upload(id, "file.bin", revision, new byte[0])).isInstanceOf(ManagedAgentException.class);
jdbc.update("UPDATE ma_operation SET available_at = 0 WHERE operation_kind = 'WORKSPACE_UPLOAD'");
uploads.reconcile();
assertThat(uploads.query("tenant", "actor", id).state()).isEqualTo("failed");
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isZero();
assertThatThrownBy(() -> uploads.query("other-tenant", "actor", id)).isInstanceOf(ManagedAgentException.class);
assertThatThrownBy(() -> uploads.query("tenant", "stranger", id)).isInstanceOf(ManagedAgentException.class);
jdbc.update("DELETE FROM ma_workspace_access");
assertThatThrownBy(() -> uploads.query("tenant", "actor", id)).isInstanceOf(ManagedAgentException.class);
}
}
@Test
void anotherUploadAndPhysicalFencingCannotEnterWhileTheOriginalProcessOwnsStorage() throws Exception {
Path entry = temp.resolve("gated-upload.mjs");
Files.writeString(entry, """
import fs from 'node:fs/promises';
if(process.argv.includes('--help')) console.log(JSON.stringify({protocolVersion:1,capabilities:{workspaceFileUpload:1}}));
else {
let input=''; for await (const bytes of process.stdin) input+=bytes;
const request=JSON.parse(input);
const stop=Date.now()+10000;
while(true) {
try { await fs.access(new URL('./release-upload', import.meta.url)); break; }
catch { if(Date.now()>stop) process.exit(9); await new Promise(resolve=>setTimeout(resolve,10)); }
}
const {operation,data,...file}=request.files;
console.log(JSON.stringify({protocolVersion:1,ok:true,result:{files:{...file,state:'completed'}}}));
}
""");
properties.getRuntimeBroker().setWorkerEntry(entry.toString());
String revision = revision();
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
var original = executor.submit(() -> upload(UUID.randomUUID().toString(), "first.bin", revision, new byte[0]));
try {
await().atMost(Duration.ofSeconds(5)).untilAsserted(() -> assertThat(jdbc.queryForObject(
"SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isEqualTo(1));
assertThatThrownBy(() -> upload(UUID.randomUUID().toString(), "second.bin", revision, new byte[0]))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getCode()).isEqualTo("workspace_busy"));
assertThatThrownBy(() -> guard.fence("tenant", storage(), 1, UUID.randomUUID().toString()))
.isInstanceOf(RuntimeException.class);
} finally {
Files.writeString(temp.resolve("release-upload"), "release");
}
assertThat(original.get(10, TimeUnit.SECONDS).sizeBytes()).isZero();
}
assertThat(jdbc.queryForObject("SELECT COUNT(*) FROM ma_storage_owner WHERE holder_key IS NOT NULL", Integer.class)).isZero();
}
@Test
void terminalEvidenceCannotReleaseADifferentStorageHolder() {
String id = UUID.randomUUID().toString();
String revision = revision();
try (var process = mockStatic(WorkspaceFileProcess.class)) {
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any())).thenAnswer(call -> {
jdbc.update("UPDATE ma_storage_owner SET holder_key = ?", "f".repeat(64));
return response(call.getArgument(2), "completed", null);
});
assertThatThrownBy(() -> upload(id, "file.bin", revision, new byte[0]))
.isInstanceOfSatisfying(ManagedAgentException.class, error -> assertThat(error.getCode()).isEqualTo("workspace_upload_pending"));
assertThat(jdbc.queryForObject("SELECT holder_key FROM ma_storage_owner", String.class)).isEqualTo("f".repeat(64));
assertThat(jdbc.queryForObject("SELECT state FROM ma_operation WHERE operation_kind = 'WORKSPACE_UPLOAD'", String.class)).isEqualTo("RUNNING");
}
}
@Test
void normalizesWorkingDirectoryAliasesBeforeFreezingAndComparingRequests() {
String id = UUID.randomUUID().toString();
String revision = revision();
try (var process = mockStatic(WorkspaceFileProcess.class)) {
process.when(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any())).thenAnswer(call -> {
Map<String, Object> request = call.getArgument(2);
assertThat(request.get("cwdRelative")).isEqualTo(".");
return response(request, "completed", null);
});
var first = uploads.upload("tenant", "actor", workspace, "./", "file.bin", revision, id,
InputStream.nullInputStream(), 0);
assertThat(upload(id, "file.bin", revision, new byte[0])).isEqualTo(first);
process.verify(() -> WorkspaceFileProcess.upload(anyString(), anyString(), any()));
}
}
private String storage() {
return jdbc.queryForObject("SELECT storage_id FROM ma_workspace WHERE workspace_id = ?", String.class, workspace);
}
}
@@ -9,15 +9,18 @@ import org.junit.jupiter.api.condition.OS;
import org.junit.jupiter.api.io.TempDir;
import java.net.URI;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.attribute.PosixFilePermissions;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Executors;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicBoolean;
@@ -219,6 +222,50 @@ class SshConnectionManagerTest {
assertEquals(0, result.stdout().length);
}
@Test
void writesStdinWhenTheCommonPoolHasOnlyOneFreeWorker() throws Exception {
Process probe = new ProcessBuilder(Path.of(System.getProperty("java.home"), "bin", "java").toString(),
"-Djava.util.concurrent.ForkJoinPool.common.parallelism=2", "-cp", System.getProperty("java.class.path"),
StdinProbe.class.getName()).redirectErrorStream(true).start();
try {
assertTrue(probe.waitFor(5, TimeUnit.SECONDS));
assertEquals(0, probe.exitValue(), new String(probe.getInputStream().readAllBytes(), StandardCharsets.UTF_8));
} finally {
if (probe.isAlive()) {
probe.destroyForcibly();
}
}
}
public static final class StdinProbe {
public static void main(String[] arguments) throws Exception {
var entered = new CountDownLatch(1);
var release = new CountDownLatch(1);
var blocker = ForkJoinPool.commonPool().submit(() -> {
entered.countDown();
try {
release.await();
} catch (InterruptedException error) {
Thread.currentThread().interrupt();
throw new IllegalStateException(error);
}
});
try {
if (!entered.await(1, TimeUnit.SECONDS)) {
throw new IllegalStateException("Common pool blocker did not start");
}
byte[] input = "workspace upload".getBytes(StandardCharsets.UTF_8);
var result = SshConnectionManager.runCommand(List.of("/bin/cat"), input, Duration.ofSeconds(2));
if (result.exitCode() != 0 || !Arrays.equals(input, result.stdout())) {
throw new IllegalStateException("Command did not receive its stdin");
}
} finally {
release.countDown();
blocker.get(1, TimeUnit.SECONDS);
}
}
}
@Test
void keepsCredentialsInStdinAndQuotesOnlyTrustedRemoteEntryPaths() {
var commands = new ArrayList<List<String>>();
@@ -317,10 +364,14 @@ class SshConnectionManagerTest {
var release = new CountDownLatch(1);
var lease = new RuntimeLease("runtime", URI.create("http://127.0.0.1:41234"), "token", "lease", 1);
try (var manager = manager(URI.create("http://127.0.0.1:8080"), (command, input, timeout) -> {
if (input.length == 0) return success();
if (input.length == 0) {
return success();
}
entered.countDown();
try {
if (!release.await(3, TimeUnit.SECONDS)) throw new AssertionError("Bootstrap was not released");
if (!release.await(3, TimeUnit.SECONDS)) {
throw new AssertionError("Bootstrap was not released");
}
} catch (InterruptedException failure) {
Thread.currentThread().interrupt();
throw new AssertionError(failure);
@@ -7,6 +7,8 @@ import org.junit.jupiter.api.io.TempDir;
import java.nio.file.Files;
import java.nio.file.Path;
import java.time.Duration;
import java.util.List;
import java.util.Map;
import static org.assertj.core.api.Assertions.assertThat;
@@ -29,6 +31,14 @@ class WorkspaceFileProcessTest {
.isInstanceOfSatisfying(RuntimeBrokerException.class,
error -> assertThat(error.getCode()).isEqualTo("workspace_files_unsupported"));
assertThat(temp.resolve("unexpected")).doesNotExist();
Files.writeString(entry, """
if (process.argv.includes('--help')) console.log(JSON.stringify({protocolVersion:1,capabilities:{workspaceFiles:1}}));
else { for await (const bytes of process.stdin) {} console.log('not json'); }
""");
assertThatThrownBy(() -> WorkspaceFileProcess.read(System.getProperty("node.executable", "node"),
entry.toString(), Map.of("files", Map.of("operation", "read"))))
.isInstanceOfSatisfying(RuntimeBrokerException.class,
error -> assertThat(error.getCode()).isEqualTo("workspace_files_unsupported"));
}
@Test
@@ -49,4 +59,58 @@ class WorkspaceFileProcessTest {
assertThat(entries).hasSize(1);
}
}
@Test
void uploadRequiresIndependentCapabilityAndUsesItsBoundedIngressFlag() throws Exception {
Path entry = temp.resolve("upload.mjs");
Files.writeString(entry, """
import fs from 'node:fs';
if (process.argv.includes('--help')) console.log(JSON.stringify({protocolVersion:1,capabilities:{workspaceFiles:1}}));
else { fs.writeFileSync(new URL('./unexpected', import.meta.url), 'executed'); }
""");
var request = Map.<String, Object>of("files", Map.of("operation", "upload", "data", "a".repeat(100000)));
assertThatThrownBy(() -> WorkspaceFileProcess.upload(System.getProperty("node.executable", "node"), entry.toString(), request))
.isInstanceOfSatisfying(RuntimeBrokerException.class, error -> assertThat(error.getCode()).isEqualTo("workspace_files_unsupported"));
assertThat(temp.resolve("unexpected")).doesNotExist();
Files.writeString(entry, """
if (process.argv.includes('--help')) console.log(JSON.stringify({protocolVersion:1,capabilities:{workspaceFiles:1,workspaceFileUpload:1}}));
else {
if (!process.argv.includes('--workspace-file-upload')) process.exit(8);
let input=''; for await (const bytes of process.stdin) input+=bytes;
console.log(JSON.stringify({protocolVersion:1,ok:true,result:{received:input.length}}));
}
""");
assertThat(((Number) WorkspaceFileProcess.upload(System.getProperty("node.executable", "node"), entry.toString(), request).get("received")).intValue())
.isEqualTo(JsonCodec.encode(request).length);
}
@Test
void outputFailuresAfterUploadInvocationAreUnknownAndCannotReleaseStorageAsUnsupported() throws Exception {
Path entry = temp.resolve("unknown.mjs");
for (String output : List.of("console.log('not json')", "process.exit(7)", "console.log('{}')")) {
Files.writeString(entry, """
import fs from 'node:fs';
if (process.argv.includes('--help')) console.log(JSON.stringify({protocolVersion:1,capabilities:{workspaceFileUpload:1}}));
else {
for await (const bytes of process.stdin) {}
fs.writeFileSync(new URL('./invoked', import.meta.url), 'effect');
""" + output + "\n}\n");
assertThatThrownBy(() -> WorkspaceFileProcess.upload(System.getProperty("node.executable", "node"),
entry.toString(), Map.of("files", Map.of("operation", "upload"))))
.isInstanceOfSatisfying(RuntimeBrokerException.class, error -> {
assertThat(error.getCode()).isEqualTo("workspace_upload_pending");
assertThat(error.getStatusCode()).isEqualTo(503);
});
assertThat(temp.resolve("invoked")).hasContent("effect");
}
}
@Test
void commandDeadlineAlsoBoundsAChildThatNeverReadsTheUploadPipe() {
long start = System.nanoTime();
assertThatThrownBy(() -> SshConnectionManager.runCommand(List.of(System.getProperty("node.executable", "node"),
"-e", "process.stdin.pause(); setTimeout(() => {}, 60000)"), new byte[1024 * 1024], Duration.ofMillis(150)))
.isInstanceOf(RuntimeBrokerException.class);
assertThat(Duration.ofNanos(System.nanoTime() - start)).isLessThan(Duration.ofSeconds(5));
}
}
@@ -1,5 +1,5 @@
# Generated by ManagedSurfaceCaptureTest (capture).
# sha256 of the route lines below: d9a9309deb7c45b81616f53167ed1968da3d105280ad54b8f778085aaa175092
# sha256 of the route lines below: f14c283b28ea706833d0fc5f15494f1f5f6cd8bf64f1a36e06a36c246d1c2f50
AUTOMATION_RETIRE DELETE /v1/agent-automations/{automationId} public
SESSION_DELETE DELETE /v1/agents/sessions/{sessionId} public
STORE_RESOURCE_GET GET /internal/managed-session-store/v1/sessions/{sessionId}/resources/{resourceId} internal
@@ -103,6 +103,8 @@ WORKSPACE_CREATE POST /api/agent/web-shell/v1/workspaces/create webshell
WORKSPACE_DIRECTORIES POST /api/agent/web-shell/v1/workspaces/directories/query webshell
WORKSPACE_FILES_QUERY POST /api/agent/web-shell/v1/workspaces/files/query webshell
WORKSPACE_FILES_READ POST /api/agent/web-shell/v1/workspaces/files/read webshell
WORKSPACE_FILES_UPLOAD POST /api/agent/web-shell/v1/workspaces/files/upload webshell
WORKSPACE_FILES_UPLOAD_QUERY POST /api/agent/web-shell/v1/workspaces/files/uploads/query webshell
WORKSPACE_GET POST /api/agent/web-shell/v1/workspaces/get webshell
WORKSPACE_REGISTRATION_GET POST /api/agent/web-shell/v1/workspaces/operations/get webshell
WORKSPACE_LIST POST /api/agent/web-shell/v1/workspaces/query webshell
@@ -465,6 +465,7 @@ interface ChatEditorRenderProps
atWorkspaceCwd?: string;
composerScopeKey?: string;
workspaceFeaturesEnabled?: boolean;
fileUploadTarget?: ComponentProps<typeof ChatEditor>['fileUploadTarget'];
attachmentsEnabled?: boolean;
sessionId?: string;
customization?: WebShellCustomization;
@@ -5244,6 +5245,57 @@ describe('ChatEditor file upload gating', () => {
).not.toBeNull();
});
it('uploads through an explicit Managed target without enabling daemon features', async () => {
const metadata = { managedUpload: { source: 'managed-workspace' } };
const client = {
uploadWorkspaceFile: vi.fn(async () => ({
kind: 'file_upload' as const,
path: 'a.pdf',
sizeBytes: 3,
hash: `sha256:${'a'.repeat(64)}` as const,
})),
};
const target = {
client,
targetKey: 'managed-workspace',
maxBytes: 8 * 1024 * 1024,
createReference: (path: string): WebShellComposerTag => ({
id: `file:${path}`,
kind: 'file',
value: path,
serialized: `@${path}`,
metadata,
}),
};
const container = renderChatEditor({
workspaceFeaturesEnabled: false,
attachmentsEnabled: false,
fileUploadTarget: target,
});
const input = container.querySelector<HTMLInputElement>(
'[data-web-shell-upload-input]',
)!;
expect(input).not.toBeNull();
act(() => composerCoreState.onFileUploadRequest?.('.'));
Object.defineProperty(input, 'files', {
value: [new File(['pdf'], 'a.pdf')],
});
await act(async () =>
input.dispatchEvent(new Event('change', { bubbles: true })),
);
expect(client.uploadWorkspaceFile).toHaveBeenCalledTimes(1);
expect(composerCoreState.addTags).toHaveBeenCalledWith(
[expect.objectContaining({ value: 'a.pdf', metadata })],
{ placement: 'inline', position: 'end' },
);
rerenderChatEditor(container, {
workspaceFeaturesEnabled: false,
fileUploadTarget: target,
customization: { fileUploadEnabled: false },
});
expect(container.querySelector('[data-web-shell-upload-input]')).toBeNull();
});
it('fileUploadEnabled={false} force-disables even with the capability', () => {
uploadWorkspaceState.current = makeWorkspace(['workspace_file_upload']);
const container = renderChatEditor({
@@ -57,7 +57,11 @@ import {
getComposerTagValue,
} from '../hooks/useComposerCore';
import { AtMentionPanel } from './AtMentionPanel';
import { useFileUpload, type FileUploadItem } from '../hooks/useFileUpload';
import {
useFileUpload,
type FileUploadItem,
type UseFileUploadOptions,
} from '../hooks/useFileUpload';
import { fileReferenceInsertText } from '../hooks/useAtMentionMenu';
import { AddMenu } from './composer/AddMenu';
import { computePrependSkillTransaction } from './composer/prependSkillInvocation';
@@ -326,6 +330,9 @@ interface ChatEditorProps {
shellModeEnabled?: boolean;
workspaceFeaturesEnabled?: boolean;
fileReferenceActions?: AtMentionWorkspaceActions;
fileUploadTarget?: UseFileUploadOptions & {
createReference(path: string): WebShellComposerTag;
};
persistFileReferences?: boolean;
attachmentsEnabled?: boolean;
onChatWidthModeChange?: (mode: '1000' | 'wide') => void;
@@ -1610,6 +1617,7 @@ export const ChatEditor = memo(
shellModeEnabled = true,
workspaceFeaturesEnabled = true,
fileReferenceActions,
fileUploadTarget,
persistFileReferences,
attachmentsEnabled = workspaceFeaturesEnabled,
onChatWidthModeChange,
@@ -1667,12 +1675,11 @@ export const ChatEditor = memo(
// item can be capability-gated (hidden on daemons without the feature).
const uploadWorkspace = useOptionalWorkspace();
const uploadTarget = useMemo(() => {
if (!uploadWorkspace || !workspaceFeaturesEnabled) return undefined;
// The host prop can force-disable upload even when the daemon advertises
// the capability; it does NOT bypass the capability check. Both must
// allow: `fileUploadEnabled === false` short-circuits, otherwise the
// `workspace_file_upload` capability is still required.
// The host switch also applies to an explicitly scoped Managed target;
// daemon targets require their own capability and trust checks below.
if (fileUploadEnabled === false) return undefined;
if (fileUploadTarget?.client) return fileUploadTarget;
if (!uploadWorkspace || !workspaceFeaturesEnabled) return undefined;
const features = uploadWorkspace.capabilities?.features ?? [];
if (!features.includes('workspace_file_upload')) return undefined;
if (atWorkspaceCwd) {
@@ -1687,6 +1694,7 @@ export const ChatEditor = memo(
return {
client: uploadWorkspace.client.workspaceByCwd(atWorkspaceCwd),
targetKey: atWorkspaceCwd,
createReference: undefined,
};
}
const primaryMatches = (
@@ -1694,15 +1702,21 @@ export const ChatEditor = memo(
).filter((workspace) => workspace.primary);
if (primaryMatches.length !== 1 || primaryMatches[0].trusted !== true)
return undefined;
return { client: uploadWorkspace.client, targetKey: '<primary>' };
return {
client: uploadWorkspace.client,
targetKey: '<primary>',
createReference: undefined,
};
}, [
uploadWorkspace,
atWorkspaceCwd,
fileUploadEnabled,
fileUploadTarget,
workspaceFeaturesEnabled,
]);
const uploadEnabled = uploadTarget !== undefined;
const maxUploadBytes =
fileUploadTarget?.maxBytes ??
uploadWorkspace?.capabilities?.limits?.maxWorkspaceFileUploadBytes ??
50 * 1024 * 1024;
const uploadTargetKey = `${sessionId ?? '<no-session>'}:${
@@ -1825,7 +1839,7 @@ export const ChatEditor = memo(
const serialized = fileReferenceInsertText(path).trim();
addComposerTags(
[
{
uploadTarget?.createReference?.(path) ?? {
id: `file:${serialized}`,
kind: 'file',
value: path,
@@ -1835,7 +1849,7 @@ export const ChatEditor = memo(
{ placement: 'inline', position: 'end' },
);
},
[addComposerTags],
[addComposerTags, uploadTarget],
);
const uploadStatusText = (upload: FileUploadItem): string => {
switch (upload.status) {
@@ -19,6 +19,7 @@ import type {
import type { EditorHandle } from '../../hooks/useComposerCore';
import { getComposerDraftStorageKey } from '../../hooks/useComposerCore';
import { getSlashCommandCompletionResult } from '../../completions/slashCompletion';
import { createInputAnnotationsFromComposerTags } from '../../utils/composerTag';
import type { ComponentProps } from 'react';
import {
ToolDetailsContext,
@@ -924,6 +925,67 @@ describe('ManagedSessionsPage', () => {
);
});
it('keeps prepared upload path context in the durable Turn input without a text attachment', async () => {
provider = {
...provider,
workspaceFiles: {
query: vi.fn(async () => ({
scopeRevision: 'registered',
directory: '.',
entries: [],
truncated: false,
maxUploadBytes: 8 * 1024 * 1024,
})),
read: vi.fn(),
upload: vi.fn(async (request, data) => ({
scopeRevision: 'registered',
uploadId: request.uploadId,
path: request.path,
sizeBytes: data.size,
sha256: 'a'.repeat(64),
})),
},
};
mocks.client.getSession.mockResolvedValue(
summary('s1', {
capabilities: {
canSend: true,
canCancel: false,
inputAttachments: true,
workspaceTurns: true,
},
workspace: { workspaceId: 'ws', cwdRelative: '.' },
}),
);
await render('s1');
const target = mocks.editorProps!.fileUploadTarget!;
const result = await target.client!.uploadWorkspaceFile({
path: '报告.bin',
data: new Blob(['binary']),
});
const tag = target.createReference(result.path);
const text = tag.serialized!;
await act(async () => {
mocks.editorProps!.onSubmit(text, undefined, undefined, vi.fn(), {
inputAnnotations: createInputAnnotationsFromComposerTags(text, [tag]),
});
await flush();
});
expect(mocks.client.submitPrompt).toHaveBeenCalledOnce();
expect(mocks.client.submitPrompt.mock.calls[0][1]).toMatchObject({
text,
input: [
{
type: 'input_text',
text: expect.stringContaining(
'Uploaded Workspace files (paths relative to the working directory):',
),
},
],
});
expect(provider.workspaceFiles!.read).not.toHaveBeenCalled();
});
it.each(['default', 'auto-edit', 'yolo'] as const)(
'shows persisted policy as fixed even on an unbound Session (%s)',
async (approvalMode) => {
@@ -1067,7 +1067,7 @@ function ManagedSessionsContent({
sessionId,
...(input.some(
(block) =>
block.type === 'input_file' || block.type === 'input_image',
block.type !== 'input_text' || block.text !== submittedText,
)
? { input }
: {}),
@@ -2236,6 +2236,7 @@ function ManagedSessionsContent({
skillsLoaded={skills.loaded}
workspaceFeaturesEnabled={false}
fileReferenceActions={referenceActions}
fileUploadTarget={fileReferences.uploadTarget}
persistFileReferences
attachmentsEnabled={
attachmentCapability.current.enabled && !active
@@ -1620,6 +1620,7 @@ export function WorkspaceBindingCreator({
skillsLoadError={skills.error}
skillsLoaded={skills.loaded}
fileReferenceActions={fileReferences.actions}
fileUploadTarget={fileReferences.uploadTarget}
persistFileReferences
existingFolderWorkspaceSupported={
registrationAvailable && Boolean(binding.registration)
@@ -1036,6 +1036,46 @@ export interface paths {
patch?: never;
trace?: never;
};
"/api/agent/web-shell/v1/workspaces/files/upload": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
/**
* Upload one binary file to a persisted Workspace
* @description Requires a trusted actor with OPERATOR or OWNER access, enabled Workspace files, verified LOCAL or SSH_LOCAL storage and an updated CLI. Uploads are limited to 8 MiB, require an existing parent directory and never overwrite an occupied path. A durable physical storage hold excludes tool execution and retirement until the original upload has a definite terminal result. Repeating the same upload ID reconciles its original command without replaying mutation; changed requests conflict. CSI is unsupported and never falls back to a primary Workspace. An uncertain result returns 503 workspace_upload_pending and must be queried by the original upload ID.
*/
post: operations["webShellWorkspaceFilesUpload"];
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/api/agent/web-shell/v1/workspaces/files/uploads/query": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
/**
* Reconcile the original Workspace upload
* @description Only the admitting actor with a current Workspace operator grant can query the upload. Status checks use the frozen original storage target and never execute an upload or select a replacement Runtime. Reconciliation remains available through storage fencing; pending commands retain their durable hold. The server also reconciles pending uploads after client cancellation or restart.
*/
post: operations["webShellWorkspaceFilesUploadQuery"];
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
}
export type webhooks = Record<string, never>;
export interface components {
@@ -2106,6 +2146,8 @@ export interface components {
directory: string;
entries: components["schemas"]["WebShellWorkspaceFileEntry"][];
truncated: boolean;
/** @description Present only when the selected CLI supports durable Workspace uploads. */
maxUploadBytes?: number;
};
WebShellWorkspaceFileSnapshot: {
scopeRevision: string;
@@ -2484,6 +2526,29 @@ export interface components {
idempotencyKey: string;
command: components["schemas"]["ManagedExtensionApplySelection"];
};
WebShellWorkspaceFileUploadResult: {
scopeRevision: string;
/** Format: uuid */
uploadId: string;
path: string;
sizeBytes: number;
sha256: string;
};
WebShellWorkspaceFileUploadStatus: {
scopeRevision: string;
/** Format: uuid */
uploadId: string;
path: string;
sizeBytes: number;
sha256: string;
/** @enum {string} */
state: "completed" | "failed" | "pending";
code?: string;
};
WebShellWorkspaceFileUploadQueryRequest: {
/** Format: uuid */
uploadId: string;
};
};
responses: {
/** @description The authenticated actor belongs to another tenant than X-Qwen-Tenant-Id or has an invalid ID (actor_scope_mismatch, which the tenant filter answers on every /v1/agents/ and WebShell route), or the resource is readable but the actor lacks this operation or the original Action responder permission. Artifact content reads may also return artifact_content_forbidden when the actor cannot read the original bytes. */
@@ -4632,4 +4697,88 @@ export interface operations {
};
};
};
webShellWorkspaceFilesUpload: {
parameters: {
query: {
workspaceId: string;
cwdRelative: string;
path: string;
expectedScopeRevision: string;
uploadId: string;
};
header?: never;
path?: never;
cookie?: never;
};
requestBody: {
content: {
"application/octet-stream": string;
};
};
responses: {
/** @description Workspace upload result. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["WebShellWorkspaceFileUploadResult"];
};
};
400: components["responses"]["BadRequest"];
401: components["responses"]["Unauthorized"];
403: components["responses"]["Forbidden"];
404: components["responses"]["NotFound"];
409: components["responses"]["Conflict"];
/** @description The upload exceeds the 8 MiB binary limit. */
413: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorEnvelope"];
};
};
/** @description Uploads require unencoded application/octet-stream bytes. */
415: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["ErrorEnvelope"];
};
};
503: components["responses"]["Unavailable"];
};
};
webShellWorkspaceFilesUploadQuery: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody: {
content: {
"application/json": components["schemas"]["WebShellWorkspaceFileUploadQueryRequest"];
};
};
responses: {
/** @description Workspace upload result. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["WebShellWorkspaceFileUploadStatus"];
};
};
400: components["responses"]["BadRequest"];
401: components["responses"]["Unauthorized"];
403: components["responses"]["Forbidden"];
404: components["responses"]["NotFound"];
409: components["responses"]["Conflict"];
503: components["responses"]["Unavailable"];
};
};
}
@@ -23,6 +23,61 @@ afterEach(() => {
});
describe('JavaManagedAgentClient', () => {
it('uploads raw bytes to the authenticated Workspace route and queries the same upload ID', async () => {
const result = {
scopeRevision: 'scope',
uploadId: 'upload-id',
path: 'folder/你好.pdf',
sizeBytes: 4,
sha256: 'a'.repeat(64),
};
const fetchImpl = vi
.fn<typeof fetch>()
.mockImplementation(async () => jsonResponse(result));
const client = new JavaManagedAgentClient({
baseUrl: 'https://product.example/gateway',
getHeaders: () => ({
authorization: 'Bearer scoped',
'content-type': 'application/json',
}),
fetch: fetchImpl,
});
const request = {
workspaceId: 'workspace',
cwdRelative: 'src',
path: result.path,
expectedScopeRevision: 'scope',
uploadId: result.uploadId,
};
const data = new Blob([new Uint8Array([0, 255, 128, 1])]);
const signal = new AbortController().signal;
expect(await client.uploadWorkspaceFile(request, data, signal)).toEqual(
result,
);
const [url, init] = fetchImpl.mock.calls[0];
const parsed = new URL(String(url));
expect(parsed.pathname).toBe(
'/gateway/api/agent/web-shell/v1/workspaces/files/upload',
);
expect(Object.fromEntries(parsed.searchParams)).toEqual(request);
expect(init?.body).toBe(data);
expect(init?.signal).toBe(signal);
expect(init?.credentials).toBe('include');
expect(new Headers(init?.headers).get('content-type')).toBe(
'application/octet-stream',
);
expect(new Headers(init?.headers).get('authorization')).toBe(
'Bearer scoped',
);
await client.queryWorkspaceFileUpload(result.uploadId, signal);
expect(fetchImpl.mock.calls[1][0]).toBe(
'https://product.example/gateway/api/agent/web-shell/v1/workspaces/files/uploads/query',
);
expect(JSON.parse(String(fetchImpl.mock.calls[1][1]?.body))).toEqual({
uploadId: result.uploadId,
});
});
it('renames through the authenticated WebShell POST using the original key and signal', async () => {
const result = { sessionId: 's1', title: 'Confirmed', replayed: false };
const fetchImpl = vi
@@ -246,6 +246,35 @@ export class JavaManagedAgentClient {
return this.post('/workspaces/files/read', request, signal);
}
async uploadWorkspaceFile(
request: {
workspaceId: string;
cwdRelative: string;
path: string;
expectedScopeRevision: string;
uploadId: string;
},
data: Blob,
signal?: AbortSignal,
): Promise<Schemas['WebShellWorkspaceFileUploadResult']> {
const query = new URLSearchParams(request);
const response = await this.fetchRequest(
`/workspaces/files/upload?${query}`,
data,
signal,
false,
'application/octet-stream',
);
return (await response.json()) as Schemas['WebShellWorkspaceFileUploadResult'];
}
queryWorkspaceFileUpload(
uploadId: string,
signal?: AbortSignal,
): Promise<Schemas['WebShellWorkspaceFileUploadStatus']> {
return this.post('/workspaces/files/uploads/query', { uploadId }, signal);
}
workspaceDirectories(
request: Schemas['WebShellWorkspaceDirectoryQueryRequest'],
signal?: AbortSignal,
@@ -959,11 +988,30 @@ export class JavaManagedAgentClient {
body: unknown,
signal: AbortSignal | undefined,
stream: boolean,
): Promise<Response> {
return this.fetchRequest(
path,
JSON.stringify(body),
signal,
stream,
'application/json',
);
}
private async fetchRequest(
path: string,
body: BodyInit,
signal: AbortSignal | undefined,
stream: boolean,
contentType: string,
): Promise<Response> {
const supplied = await this.options.getHeaders?.();
const headers = new Headers(supplied);
if (!headers.has('content-type')) {
headers.set('content-type', 'application/json');
if (
contentType === 'application/octet-stream' ||
!headers.has('content-type')
) {
headers.set('content-type', contentType);
}
headers.set('accept', stream ? 'text/event-stream' : 'application/json');
const response = await this.fetchImpl(
@@ -972,7 +1020,7 @@ export class JavaManagedAgentClient {
method: 'POST',
headers,
credentials: this.credentials,
body: JSON.stringify(body),
body,
signal,
},
);
@@ -215,6 +215,10 @@ export function createJavaManagedAgentProvider(
client.queryWorkspaceFiles(request, command.signal),
read: (request, command) =>
client.readWorkspaceFile(request, command.signal),
upload: (request, data, command) =>
client.uploadWorkspaceFile(request, data, command.signal),
queryUpload: (uploadId, command) =>
client.queryWorkspaceFileUpload(uploadId, command.signal),
},
canCancel: true,
titleSearch: true,
@@ -562,6 +562,21 @@ export interface ManagedAgentProvider {
request: components['schemas']['WebShellWorkspaceFileReadRequest'],
options: ManagedAgentRequestOptions,
): Promise<components['schemas']['WebShellWorkspaceFileSnapshot']>;
upload?(
request: {
workspaceId: string;
cwdRelative: string;
path: string;
expectedScopeRevision: string;
uploadId: string;
},
data: Blob,
options: ManagedAgentRequestOptions,
): Promise<components['schemas']['WebShellWorkspaceFileUploadResult']>;
queryUpload?(
uploadId: string,
options: ManagedAgentRequestOptions,
): Promise<components['schemas']['WebShellWorkspaceFileUploadStatus']>;
};
readonly workspaceBinding?: {
readonly agentId: string;
@@ -391,4 +391,144 @@ describe('Managed file reference snapshots', () => {
encodeManagedInput(prepared.text, [], prepared.files),
).resolves.toHaveLength(3);
});
function enableUploads() {
const api = provider.workspaceFiles!;
const original = vi.mocked(api.query).getMockImplementation()!;
vi.mocked(api.query).mockImplementation(async (request, options) => ({
...(await original(request, options)),
maxUploadBytes: 8 * 1024 * 1024,
}));
api.upload = vi.fn(async (request, data) => ({
scopeRevision: revision,
uploadId: request.uploadId,
path: request.path,
sizeBytes: data.size,
sha256: 'a'.repeat(64),
}));
api.queryUpload = vi.fn();
return api;
}
it('keeps upload disabled on a read-only CLI even when the provider has an upload method', async () => {
const api = provider.workspaceFiles!;
api.upload = vi.fn();
await render();
expect(current.actions).toBeDefined();
expect(current.uploadTarget).toBeUndefined();
expect(api.upload).not.toHaveBeenCalled();
});
it('uploads binary bytes and sends a scoped path without reading a text snapshot', async () => {
const api = enableUploads();
await render();
const target = current.uploadTarget!;
const data = new Blob([new Uint8Array([0, 255, 128, 1])]);
const result = await target.client.uploadWorkspaceFile({
path: '报告.pdf',
data,
});
expect(api.upload).toHaveBeenCalledWith(
{
workspaceId: 'workspace',
cwdRelative: '.',
path: '报告.pdf',
expectedScopeRevision: revision,
uploadId: expect.any(String),
},
data,
{ clientId: 'client', signal: undefined },
);
const tag = target.createReference(result.path);
const prompt = `${tag.serialized} inspect`;
const prepared = await current.prepareInput(prompt, [], {
inputAnnotations: createInputAnnotationsFromComposerTags(prompt, [tag]),
});
expect(prepared.text).toContain('paths relative to the working directory');
expect(prepared.text).toContain('"报告.pdf"');
expect(prepared.files).toEqual([]);
expect(api.read).not.toHaveBeenCalled();
});
it('does not admit an upload completion or restored reference into a different Session', async () => {
const api = enableUploads();
await render();
const target = current.uploadTarget!;
const result = await target.client.uploadWorkspaceFile({
path: 'binary.pdf',
data: new Blob(['pdf']),
});
const tag = target.createReference(result.path);
const original = vi.mocked(api.upload!).getMockImplementation()!;
let finish!: () => void;
vi.mocked(api.upload!).mockImplementationOnce(
(request, data, options) =>
new Promise((resolve) => {
finish = () => void original(request, data, options).then(resolve);
}),
);
const pending = target.client
.uploadWorkspaceFile({
path: 'pending.pdf',
data: new Blob(['pdf']),
})
.catch((error: unknown) => error);
await render('.', 'different-session');
finish();
expect(await pending).toBeInstanceOf(Error);
expect(((await pending) as Error).message).toContain('Workspace changed');
await expect(
current.prepareInput('@binary.pdf', [], {
inputAnnotations: createInputAnnotationsFromComposerTags(
'@binary.pdf',
[tag],
),
}),
).rejects.toThrow('Workspace changed');
});
it('reconciles a lost upload response by its original ID without sending the mutation again', async () => {
const api = enableUploads();
await render();
vi.mocked(api.upload!).mockRejectedValueOnce(
new TypeError('Connection lost'),
);
vi.mocked(api.queryUpload!).mockImplementation(async (uploadId) => ({
scopeRevision: revision,
uploadId,
state: 'completed',
path: 'a.bin',
sizeBytes: 1,
sha256: 'a'.repeat(64),
}));
await expect(
current.uploadTarget!.client.uploadWorkspaceFile({
path: 'a.bin',
data: new Blob(['a']),
}),
).resolves.toMatchObject({ path: 'a.bin' });
expect(api.upload).toHaveBeenCalledTimes(1);
expect(api.queryUpload).toHaveBeenCalledWith(
vi.mocked(api.upload!).mock.calls[0][0].uploadId,
{ clientId: 'client', signal: undefined },
);
});
it('never treats an occupied destination as an uploaded reference', async () => {
const api = enableUploads();
await render();
vi.mocked(api.upload!).mockRejectedValueOnce(
Object.assign(new Error('The file already exists.'), { status: 409 }),
);
await expect(
current.uploadTarget!.client.uploadWorkspaceFile({
path: 'a.bin',
data: new Blob(['a']),
}),
).rejects.toThrow('already exists');
expect(api.queryUpload).not.toHaveBeenCalled();
expect(() => current.uploadTarget!.createReference('a.bin')).toThrow(
'has not completed',
);
});
});
@@ -4,6 +4,10 @@ import type { ComposerSubmitMetadata } from '../../hooks/useComposerCore';
import type { WebShellComposerTag } from '../../customization';
import type { ManagedAgentProvider } from './managed-agent-provider';
import type { PromptFile } from '../../adapters/promptTypes';
import {
useManagedFileUpload,
type ManagedUploadReference,
} from './use-managed-file-upload';
interface Snapshot {
version: 1;
@@ -58,6 +62,11 @@ export function useManagedFileReferences(
const currentSource = useRef(source);
currentSource.current = source;
const [ready, setReady] = useState<string>();
const [uploadScope, setUploadScope] = useState<{
source: string;
scopeRevision: string;
maxUploadBytes: number;
}>();
const pendingReads = useRef(new Map<string, number>());
const [busySource, setBusySource] = useState<string>();
const [failure, setFailure] = useState<{ source: string; message: string }>();
@@ -97,6 +106,11 @@ export function useManagedFileReferences(
if (!active.current || currentSource.current !== source)
throw new Error('Workspace changed; select the file again.');
revision = result.scopeRevision;
setUploadScope({
source,
scopeRevision: result.scopeRevision,
maxUploadBytes: result.maxUploadBytes ?? 0,
});
return result;
} catch (error) {
if (signal?.aborted) throw error;
@@ -212,6 +226,17 @@ export function useManagedFileReferences(
};
}, [actions, source]);
const uploadTarget = useManagedFileUpload(
provider,
clientId,
ready === source &&
uploadScope?.source === source &&
workspaceId &&
cwdRelative
? { ...uploadScope, workspaceId, cwdRelative }
: undefined,
);
function assertCurrent() {
if (!active.current || currentSource.current !== source)
throw new Error('Workspace changed; select the file again.');
@@ -227,13 +252,32 @@ export function useManagedFileReferences(
'Wait for the selected file to finish loading before sending.',
);
const selected: Snapshot[] = [];
const uploaded: ManagedUploadReference[] = [];
for (const annotation of metadata?.inputAnnotations ?? []) {
if (
annotation.type !== 'reference' ||
annotation.reference.kind !== 'file'
)
continue;
const item = object(object(annotation.reference.metadata)['managedFile']);
const details = object(annotation.reference.metadata);
const upload = object(details['managedUpload']);
if (upload['version'] === 1) {
if (
upload['source'] !== source ||
typeof upload['scopeRevision'] !== 'string' ||
typeof upload['path'] !== 'string' ||
upload['path'] !== details['filePath'] ||
!upload['path'] ||
upload['path']
.split('/')
.some((part) => !part || part === '.' || part === '..')
)
throw new Error('Workspace changed; select the file again.');
if (!uploaded.some((item) => item.path === upload['path']))
uploaded.push(upload as unknown as ManagedUploadReference);
continue;
}
const item = object(details['managedFile']);
if (
item['version'] !== 1 ||
item['source'] !== source ||
@@ -249,7 +293,8 @@ export function useManagedFileReferences(
if (!selected.some((snapshot) => snapshot.path === item['path']))
selected.push(item as unknown as Snapshot);
}
if (!selected.length) return { text, files: [...files] };
if (!selected.length && !uploaded.length)
return { text, files: [...files] };
if (!api || !workspaceId || !cwdRelative)
throw new Error('Workspace file access is unavailable.');
const scope = await api.query(
@@ -259,7 +304,9 @@ export function useManagedFileReferences(
if (
!active.current ||
currentSource.current !== source ||
selected.some((item) => item.scopeRevision !== scope.scopeRevision)
[...selected, ...uploaded].some(
(item) => item.scopeRevision !== scope.scopeRevision,
)
)
throw new Error('Workspace changed; select the file again.');
const attached = [...files];
@@ -287,13 +334,26 @@ export function useManagedFileReferences(
);
}
assertCurrent();
const context = [
...(selected.length
? [
`Workspace files (relative path → attachment):\n${mapping.join('\n')}`,
]
: []),
...(uploaded.length
? [
`Uploaded Workspace files (paths relative to the working directory):\n${uploaded.map((item) => JSON.stringify(item.path)).join('\n')}`,
]
: []),
];
return {
text: `${text}\n\nWorkspace files (relative path → attachment):\n${mapping.join('\n')}`,
text: `${text}\n\n${context.join('\n\n')}`,
files: attached,
};
}
return {
actions: ready === source ? actions : undefined,
uploadTarget,
busy: busySource === source && (pendingReads.current.get(source) ?? 0) > 0,
prepareInput,
assertCurrent,
@@ -0,0 +1,161 @@
import { useEffect, useMemo, useRef } from 'react';
import { fileReferenceInsertText } from '../../hooks/useAtMentionSources';
import type { FileUploadClient } from '../../hooks/useFileUpload';
import type { WebShellComposerTag } from '../../customization';
import type { ManagedAgentProvider } from './managed-agent-provider';
export interface ManagedUploadReference {
version: 1;
source: string;
scopeRevision: string;
path: string;
}
interface UploadScope {
workspaceId: string;
cwdRelative: string;
source: string;
scopeRevision: string;
maxUploadBytes: number;
}
export function useManagedFileUpload(
provider: ManagedAgentProvider,
clientId: string,
scope: UploadScope | undefined,
) {
const api = provider.workspaceFiles;
const workspaceId = scope?.workspaceId;
const cwdRelative = scope?.cwdRelative;
const source = scope?.source;
const scopeRevision = scope?.scopeRevision;
const targetKey =
source && scopeRevision
? JSON.stringify([source, scopeRevision])
: undefined;
const currentTarget = useRef(targetKey);
currentTarget.current = targetKey;
useEffect(() => {
currentTarget.current = targetKey;
return () => {
if (currentTarget.current === targetKey)
currentTarget.current = undefined;
};
}, [targetKey]);
const maxBytes = scope?.maxUploadBytes;
return useMemo(() => {
if (
!api?.upload ||
!workspaceId ||
!cwdRelative ||
!source ||
!scopeRevision ||
!targetKey ||
!maxBytes ||
maxBytes <= 0 ||
!Number.isSafeInteger(maxBytes) ||
maxBytes > 8 * 1024 * 1024
)
return undefined;
const uploaded = new Map<string, ManagedUploadReference>();
function assertCurrent(signal?: AbortSignal) {
signal?.throwIfAborted();
if (currentTarget.current !== targetKey)
throw new Error('Workspace changed; select the file again.');
}
const client: FileUploadClient = {
async uploadWorkspaceFile(request) {
assertCurrent(request.signal);
const data =
request.data instanceof Blob
? request.data
: new Blob([
request.data instanceof Uint8Array
? new Uint8Array(request.data).buffer
: request.data,
]);
const uploadId = crypto.randomUUID();
let result: Awaited<ReturnType<NonNullable<typeof api.upload>>>;
try {
result = await api.upload!(
{
workspaceId,
cwdRelative,
path: request.path,
expectedScopeRevision: scopeRevision,
uploadId,
},
data,
{ clientId, signal: request.signal },
);
} catch (error) {
assertCurrent(request.signal);
const status =
error && typeof error === 'object' && 'status' in error
? Number(error.status)
: undefined;
if ((status && status < 500) || !api.queryUpload) throw error;
const recovered = await api.queryUpload(uploadId, {
clientId,
signal: request.signal,
});
if (recovered.state !== 'completed')
throw new Error(
recovered.state === 'pending'
? 'Upload outcome is not confirmed. Check the Workspace files before trying again.'
: 'The file could not be uploaded. Try again.',
);
result = recovered;
}
assertCurrent(request.signal);
if (
result.uploadId !== uploadId ||
result.path !== request.path ||
result.sizeBytes !== data.size ||
result.scopeRevision !== scopeRevision ||
!/^[a-f0-9]{64}$/.test(result.sha256)
)
throw new Error('The uploaded file could not be confirmed.');
uploaded.set(result.path, {
version: 1,
source,
scopeRevision,
path: result.path,
});
return {
kind: 'file_upload',
path: result.path,
sizeBytes: result.sizeBytes,
hash: `sha256:${result.sha256}`,
};
},
};
return {
client,
targetKey,
maxBytes,
createReference(path: string): WebShellComposerTag {
assertCurrent();
const managedUpload = uploaded.get(path);
if (!managedUpload) throw new Error('The upload has not completed.');
const serialized = fileReferenceInsertText(path).trim();
return {
id: `file:${serialized}`,
kind: 'file',
value: path,
serialized,
metadata: { fileKind: 'file', filePath: path, managedUpload },
};
},
};
}, [
api,
clientId,
workspaceId,
cwdRelative,
source,
scopeRevision,
maxBytes,
targetKey,
]);
}
+1
View File
@@ -539,6 +539,7 @@ describe('scripts/cli-entry.js production entry', () => {
it.each([
['managed-runtime-bootstrap'],
['managed-runtime-bootstrap', '--workspace-file-upload'],
['managed-runtime-bootstrap-worker', '/private/runtime/resource'],
])(
'keeps pinned %s commands in this process and skips active updates',