mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-11-07 06:19:43 -10:00
fix(runtime): deliver bounded provider images and PDFs (#13606)
* fix(runtime): deliver bounded provider images and PDFs Use invocation-local media capabilities for actual file reads so managed providers can deliver images and PDFs without a model or inherited Omni uploads. - Enforce inline media and complete result budgets before settlement - Preserve accepted media across execute, status, and cancel responses - Propagate PDF cancellation and wait for child cleanup - Cover capability isolation, reply loss, and Broker restart recovery - Document provider media limits and refresh dependency notices * fix(runtime): repair media review validation gates Keep provider media validation usable in standard CI while retaining explicit activation of the full MySQL media scenario. - Gate media integration on the existing bundle-ready opt-in - Align PDF call assertions with the optional cancellation signal - Keep the maximum-result fixture writable without changing its budget - Explain the complete-result gate and restore baseline notices * fix(runtime): preserve media diagnostics and clean up cancelled reads Keep admitted media observations intact while retaining useful Hook and PDF diagnostics. Bound fixture cleanup so cancelled reads and stalled processes do not leave unused files or consume test resources. - Fit Hook reasons and errors against the actual response budget - Handle PDF ranges beyond document length and retain size errors - Preserve read failures and propagate PDF fetch cancellation - Add regression witnesses and instrument restarted media workers - Bound MySQL commands and include media scratch paths in cleanup - Record Computer Use comparisons and the typed browser facade plan * fix(runtime): bound PDF cancellation and strengthen media probes Prevent cancelled PDF reads from waiting forever for an unresponsive child and make provider-media verification reflect the live runtime. - Escalate cancelled PDF children and bound the remaining close wait - Track the restarted provisioner and share fixture scratch prefixes - Preserve MySQL initialization diagnostics during cleanup - Skip redundant page probes and verify delivered ranges and statuses - Update media design guidance for diagnostics and cancellation * fix(runtime): tolerate the expected media probe reply loss Keep lost-execute response loss from poisoning recovery evidence while preserving fatal errors for all other requests. - Consume and count one drop for the matching execute call - Adapt media tests to the borrowed descriptor source - Assert oversized descriptor media is refused before reading * fix(runtime): verify media reply loss at the worker probe The media driver checked a worker route that its broker client never traverses, so the required loss counter always remained zero. - Observe the actual Java probe drop instead of a proxy-only counter - Allow broker recovery while preserving the lost-start assertions - Verify the real drop, file replacement and single-use evidence * fix(core): remove duplicate filesystem service import The automatic main merge retained both copies of the same import and failed the TypeScript build with TS2300. - Keep one upstream import for the existing filesystem test fixtures * fix(runtime): align media probe with the managed schema Keep the provider media acceptance probe usable after the main schema rollout replaces legacy tables and logical storage ownership keys. - Read execution and runtime evidence from canonical managed tables - Resolve physical ownership through the tenant storage alias - Cover prepared, settled and released evidence on the fresh schema * style(runtime): align media probes with Java checks Keep the media integration gate compatible with the Java rules introduced on main. - Sort imports and remove the obsolete charset import - Brace existing conditions and loops without changing behavior * fix(runtime): adapt media validation to server migration Keep provider media validation runnable after the Java server move and the managed schema's epoch-millisecond timestamp conversion. - Point the MySQL launcher at the migrated server POM - Bind a numeric last-active time in the canonical schema fixture * style(runtime): fix inherited MCP fault test checkstyle Keep the module verification gate passing after integrating the new upstream MCP fault test. - Remove an unused CompletableFuture import - Normalize blank lines without changing test logic or assertions * test(runtime): align media probe with execution membership Keep the media acceptance probe compatible with shared runtime execution membership after integrating main. - Check the original binding, generation and physical storage member - Reject mismatched or exclusive members and leaked released membership * docs(runtime): add the English provider media design and drop unrelated CUA docs Translate the provider media design so the bilingual pair linked from the Chinese document resolves. Drop two CUA planning documents that were carried by the media commit but describe unrelated browser work. The Java/MySQL provider-media scenario passes locally against an isolated MySQL instance: 12 cases, 24 model requests, one dropped start, every reference dispatched once and released after broker restart. * test(runtime): adapt the provider-media M4 driver to the broker client main replaced the high-level broker provider with ManagedRuntimeBrokerClient. Add a test-only adapter that exposes the ManagedToolV2Client surface the M4 driver uses, backed by the new client, and update the driver import and status narrowing. Verified: integration typecheck, ESLint, and the Java/MySQL M4 scenario (12 cases, 24 model requests, one dropped start, all references dispatched once and released after Broker restart). * style(tests): add the broker media adapter final newline Restore Prettier compliance for the provider-media integration helper so the formatting lane can pass. --------- Co-authored-by: wangheyang <[email protected]> Co-authored-by: wenshao <[email protected]>
This commit is contained in:
33 files changed
+4848
-106
No files matched your search
@@ -58,12 +58,13 @@ Session's workspace is refused the same way, since core's shell tool would ask
|
||||
and a preapproved Session never asks. The worker checks again just before the
|
||||
call runs, resolving the path afresh the way the kernel follows it, and settles
|
||||
the call as an error if a link has since moved it out. `mediaContext` is
|
||||
admitted for `read_file` only; it binds the
|
||||
tool's model-facing description to the Harness's modalities, while the read
|
||||
itself decides media delivery from this worker's own content-generator
|
||||
modalities, which it does not have, so a media file is still answered with the
|
||||
unsupported-type placeholder. Delivering media through the worker is follow-up
|
||||
work. Foreign Session references and unknown fields are refused
|
||||
admitted for `read_file` only. Its invocation-local effective modality view
|
||||
governs both description and actual reads, enabling images and PDFs only when
|
||||
requested; audio/video remain disabled. The worker does not initialize a model
|
||||
or upload through an inherited Omni environment opt-in. PDFs retain native-byte,
|
||||
text-first and image-rendering behavior within the
|
||||
[provider media budget](2026-10-07-managed-runtime-provider-media.md).
|
||||
Foreign Session references and unknown fields are refused
|
||||
before dispatch. The Broker also refuses, with 400
|
||||
`runtime_control_operation_invalid`, an operation with an unpaired surrogate in
|
||||
any key or string, which the JSON writer would otherwise send as `?`.
|
||||
@@ -81,7 +82,7 @@ Control requests and responses are bounded at 8 MiB for `bind-history`,
|
||||
`checkpoint` and `history`, and 1 MiB for other operations. Tool arguments
|
||||
also have the existing core limit of 256 KiB of canonical JSON; fitting the
|
||||
outer envelope does not bypass that limit. An `execute`, `status` or `cancel`
|
||||
result that would exceed its operation's response budget is fitted rather
|
||||
non-media result that would exceed its operation's response budget is fitted rather
|
||||
than refused: the worker first evicts oldest progress events (announced
|
||||
through `firstAvailableSeq`/`progressGap`), then cuts bulk text fields
|
||||
head-and-tail with an inline notice and sets `truncated` on shell displays.
|
||||
@@ -89,8 +90,14 @@ When even fully cut text could not fit beside what the cut cannot reach, a
|
||||
structured display (such as an edit's file diff, which only feeds the UI), then
|
||||
artifacts, which also only feed a client surface, and then hook results are
|
||||
dropped before any text is cut, and content the cut
|
||||
cannot reach at all (inline media) turns the model content into an explicit
|
||||
stub.
|
||||
cannot reach at all turns the non-media model content into an explicit stub.
|
||||
Media reads have a fixed 768 KiB aggregate base64 ceiling and a 896 KiB complete
|
||||
ToolResult ceiling. ReadFile settles budget errors as `FILE_TOO_LARGE` before
|
||||
Runtime terminal retention. Accepted media-bearing ToolResults stay identical
|
||||
through execute/status/cancel fitting; progress and auxiliary Hook details may
|
||||
be omitted, but a Hook stop decision remains. Invalid or unadmitted media is a
|
||||
protocol fault, never a rewritten successful stub. The 1 MiB envelope cap is
|
||||
unchanged. This does not enable the public Hosted raw file loop's media path.
|
||||
The cut is measured in JSON-encoded UTF-8 bytes, the unit of the wire limit,
|
||||
and removes whole code points, so a surrogate pair is never split; the notice
|
||||
reports how many characters (code points) were omitted. When several fields
|
||||
|
||||
@@ -45,9 +45,10 @@ core 只对 `notebook_edit` 应用内容修改，而本 profile 只暴露 `read_
|
||||
工作区之外的 `run_shell_command` 同样以此拒绝，因为 core 的 shell 工具只会询问，而预先
|
||||
批准的 Session 从不询问。调用执行前 worker 会再检查一次，按内核跟随链接的方式重新解析
|
||||
路径；若链接已把它移出工作区，该调用以错误结算。`mediaContext` 只对
|
||||
`read_file` 生效：它把该工具面向模型的描述绑定到 Harness 的模态上，而读取本身按本
|
||||
worker 自己的 content-generator 模态决定是否交付媒体；本 worker 没有这些模态，所以
|
||||
媒体文件仍以“不支持的类型”占位文本作答。经 worker 交付媒体留作后续工作。派发前拒绝外来 Session
|
||||
`read_file` 生效：invocation 独立的有效模态视图同时决定描述和真实读取，只按请求
|
||||
开启图片/PDF，音视频仍关闭。worker 不初始化模型，也不因继承 Omni 环境开关而上传。
|
||||
PDF 在[provider 媒体预算](2026-10-07-managed-runtime-provider-media.zh-CN.md)内保留
|
||||
原生字节、文本优先和页面渲染行为。派发前拒绝外来 Session
|
||||
reference 和未知字段。Broker 还会以 400 `runtime_control_operation_invalid` 拒绝任何键或
|
||||
字符串中含未配对代理项的操作，否则 JSON 写入器会把它发成 `?`。
|
||||
不支持的版本与操作明确失败，不回退到旧路由。
|
||||
@@ -59,13 +60,17 @@ Content-Encoding。TypeScript 客户端保留限长原因。这些诊断信息
|
||||
|
||||
`bind-history`、`checkpoint` 和 `history` 的控制请求与响应限制为 8 MiB，其他操作为
|
||||
1 MiB。工具参数还受 core 既有的 256 KiB 规范化 JSON 限制；满足外层信封限制并不绕过
|
||||
参数限制。`execute`、`status` 或 `cancel` 的结果超出所属操作响应预算时会被适配而非
|
||||
参数限制。`execute`、`status` 或 `cancel` 的非媒体结果超出所属操作响应预算时会被适配而非
|
||||
拒绝：worker 先淘汰最旧的 progress 事件（通过 `firstAvailableSeq`/`progressGap`
|
||||
告知），再把大文本字段按首尾截断并内联标记，shell 展示置 `truncated`；若大文本
|
||||
全部截到最短，仍放不下截断够不到的部分，则在截断任何文本之前，先丢弃结构化的展示
|
||||
（例如 edit 的文件 diff，它只供界面使用），再丢弃同样只供客户端界面使用的 artifacts，
|
||||
最后丢弃 hook 结果；截断完全够不到的内容
|
||||
（内联媒体）会让模型内容变为明确的占位存根。截断按 JSON
|
||||
最后丢弃 hook 结果；截断完全够不到的非媒体内容会变为明确的占位存根。
|
||||
媒体读取固定限制为聚合 base64 768 KiB、完整 ToolResult 896 KiB。ReadFile 在
|
||||
Runtime 保留终态前把预算错误结算为 `FILE_TOO_LARGE`。已接受的含媒体 ToolResult
|
||||
在 execute/status/cancel fitting 中保持相同；可以省略进度和 Hook 辅助内容，但
|
||||
保留 Hook 停止决定。畸形或未经准入的媒体属于协议故障，不能改写为成功占位文本。
|
||||
1 MiB 包装上限保持不变，这不开放公开 Hosted raw 文件循环的媒体路径。截断按 JSON
|
||||
编码后的 UTF-8 字节计量，与线上限制的单位一致；删除时以完整码点为单位，因此不会拆开
|
||||
代理对；标记注明省略了多少个字符（按码点计）。多个字段同时超长时，截到同一个大小，
|
||||
不会出现一个字段被清空、另一个字段仍保留大部分文本的情况。因此已结算的执行始终保有终态观察，
|
||||
|
||||
@@ -0,0 +1,337 @@
|
||||
# Managed Runtime Provider Media Delivery
|
||||
|
||||
[English](2026-10-07-managed-runtime-provider-media.md) | [简体中文](2026-10-07-managed-runtime-provider-media.zh-CN.md)
|
||||
|
||||
Status: provider/1 implemented; production deployment acceptance tracked separately.
|
||||
Date: 2026-10-07
|
||||
Issue: [#13039](https://github.com/QwenLM/qwen-code/issues/13039), including the [root-cause comment](https://github.com/QwenLM/qwen-code/issues/13039#issuecomment-5895577269).
|
||||
Code baseline: `a764fb9698e6e1152e0a8bc5808c30a765744152` for both upstream and local.
|
||||
|
||||
## 1. Decision and scope
|
||||
|
||||
Enable `read_file` to deliver real images and PDFs through the existing
|
||||
`managed-runtime-provider/1` worker, Java Runtime Broker, and
|
||||
`BrokerManagedRuntimeProvider` client. Preserve the seven-field prepared
|
||||
reference, approval, preflight, original execution observations, and release
|
||||
protocol. The Runtime receives only modality booleans, not model configuration,
|
||||
prompts, credentials, or model-interface access.
|
||||
|
||||
Tool description and actual file reads both use
|
||||
`Config.getEffectiveInputModalities()`. Add a worker-owned inline media ceiling
|
||||
to the existing ReadFile path. Response fitting preserves valid media;
|
||||
over-limit reads return an explicit, settled tool error.
|
||||
|
||||
This issue implements the provider path; it does not migrate every Hosted tool
|
||||
profile. The public Hosted file loop currently uses raw Tool v2 and a separate
|
||||
64 KiB inline Session resource. Replacing that loop, opening public media, a new
|
||||
artifact API, audio/video delivery, Vision Bridge inference inside the Runtime,
|
||||
and object-storage media transport are all separate changes. Successful provider
|
||||
delivery does not prove those capabilities exist.
|
||||
|
||||
## 2. Current contract and defects
|
||||
|
||||
| Scope | Verified baseline behavior |
|
||||
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `managed-runtime-provider-worker.ts:createRuntime` | Creates an execution Config with no content generator. The media-bound ReadFile only covers `getEffectiveInputModalities`. |
|
||||
| `fileUtils.ts:processSingleFileContent` | Reads `getContentGeneratorConfig()?.modalities`, so images remain unsupported, and native PDF and PDF-to-image fallback are unavailable; PDF text extraction already exists. |
|
||||
| `managed-tool-runtime.ts:prepare` | Copies `mediaContext` and includes it in the preparation digest; changing media context on a live invocation conflicts. |
|
||||
| `managed-runtime-provider-protocol.ts` / Java `ProviderRuntimeProtocol` | Closes `mediaContext: { inputModalities }` and allows boolean keys `image`, `pdf`, `audio`, `video`. Normal responses, including execute/status/cancel, are 1 MiB; historical operations are 8 MiB. |
|
||||
| `fitManagedRuntimeProviderResult` | Evicts progress and truncates text, and finally replaces non-truncatable inline media with a generic stub; it operates on the HTTP copy, not the execution result retained by the Runtime. |
|
||||
| Java `HttpRuntimeTransport` | The seven-field reference selects provider execute/status/cancel; the four-field raw Tool v2 uses a different route and result shape. |
|
||||
| `managed-tool-media.ts` | The larger validated-media exception belongs to ACP/raw Tool v2, not provider/1, and does not raise its 1 MiB ceiling. |
|
||||
| `hosted-workspace-tool-turn.ts` | Public Hosted file selection uses raw v2; the persistent outcome/message limits can independently reject media and are not determined by transport success. |
|
||||
|
||||
The issue comment suggested overriding content-generator configuration. This
|
||||
design prefers the existing effective-modality getter: a production Config
|
||||
returns modalities from the same resolved configuration by default, avoiding
|
||||
forged model and auth configuration for execution workers. The
|
||||
`getContentGeneratorConfig()?.modalities ?? {}` fallback is retained only for
|
||||
existing local Config test doubles that do not implement the effective getter.
|
||||
An empty object from the effective getter is authoritative; the fallback must
|
||||
not re-enable a modality that was turned off.
|
||||
|
||||
## 3. Architecture and sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant H as Harness/provider caller
|
||||
participant B as Java Runtime Broker
|
||||
participant R as Provider worker
|
||||
participant F as Workspace ReadFile
|
||||
H->>B: prepare(identity, read_file, input, mediaContext)
|
||||
B->>R: authenticated provider/1 prepare
|
||||
R->>R: copy and bind modalities for this invocation
|
||||
R-->>B: seven-field prepared reference
|
||||
B-->>H: seven-field prepared reference
|
||||
H->>B: confirmation / confirm / preflight
|
||||
H->>B: reserve / start(reference)
|
||||
B->>R: execute the original prepared reference
|
||||
R->>F: read under the fixed worker media budget
|
||||
F-->>R: valid inline parts or an explicit bounded tool error
|
||||
R->>R: retain terminal execution evidence
|
||||
R-->>B: bounded provider execution result
|
||||
B->>B: persist the original execution result
|
||||
B-->>H: original result containing inlineData
|
||||
H->>H: consume through the existing model-side media conversion
|
||||
Note over H,R: a lost reply only queries status for the original reference; it does not reread
|
||||
```
|
||||
|
||||
Routing continues to follow current ownership: worker control belongs to the
|
||||
live-session-owner; Broker start/status/cancel uses persistent execution
|
||||
ownership and the original Runtime generation. `mediaContext` expresses model
|
||||
capability and does not grant file permission. Existing workspace
|
||||
authorization, session binding, approval, and file-service checks remain in the
|
||||
read path. A model change only affects new preparations.
|
||||
|
||||
## 4. Modality binding and read semantics
|
||||
|
||||
1. Resolve and copy the existing media context during prepare. An omitted context
|
||||
preserves the worker's existing text-only media behavior, including PDF text
|
||||
extraction.
|
||||
2. Create an independent derived Config for the invocation. This provider profile
|
||||
enables image/PDF only when the copied corresponding boolean is true;
|
||||
audio/video remain disabled even if the model supports them. Description and
|
||||
execution use the same effective view. Do not modify the Session or
|
||||
process-global Config, do not initialize the generator, and do not infer
|
||||
capabilities from model names.
|
||||
3. Preserve the file service, target directory, permission rules, disabled read
|
||||
cache, and file-history service. Non-ReadFile tools continue to reject
|
||||
`mediaContext`. Do not register `zoom_image` or advertise zoom capability that
|
||||
this four-tool worker does not have. Fix `isOmniEnabled` to false on the
|
||||
worker's derived Config so inherited environment switches cannot load the
|
||||
media upload flow. A normal local CLI Config keeps the existing Omni behavior.
|
||||
4. Change `processSingleFileContent` to read modalities through the effective
|
||||
getter. Local CLI/Legacy continues to use the existing resolved modalities.
|
||||
The local Managed factory already overrides this getter and must be
|
||||
regression-tested, because it changes from affecting only the description to
|
||||
also affecting the real read.
|
||||
|
||||
| Input and capability | Required result |
|
||||
| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Supported image, `image: true` | Reuse the native overview and safe-format handling, and deliver a valid image inlineData within budget; preserve existing corrupt/unsafe-format messages. |
|
||||
| Image with image disabled or omitted | Existing unsupported-image message, no inline bytes. |
|
||||
| PDF without `pages`, `pdf: true` | Deliver native `application/pdf` inlineData when it fits; otherwise an explicit FILE_TOO_LARGE with guidance to use `pages`. |
|
||||
| With `pages`, or without native PDF | Existing text-first extraction and token/page checks; text-only models can still read text PDFs. |
|
||||
| Scanned/high-density PDF, `image: true` | Existing page rendering, constrained by the smaller provider aggregate budget. |
|
||||
| Audio/video | Existing unsupported message, no inline bytes, no remote upload. |
|
||||
| SVG/notebook/text | Preserve existing non-media behavior and permission rules. |
|
||||
|
||||
The generic manifest does not promise delivery of every media type. A
|
||||
preparation with media and the actual output are the acceptance evidence. The
|
||||
Harness caller passes the resolved capability snapshot through the fifth
|
||||
parameter of the existing `ManagedToolV2Client.prepare`; the published Hosted
|
||||
raw loop is not currently a production caller of that parameter. The default
|
||||
product capability must not be claimed open merely because the client can
|
||||
forward it.
|
||||
|
||||
## 5. Byte policy and terminal observability
|
||||
|
||||
provider/1 remains 1 MiB. The first-version policy is fixed in worker code; it
|
||||
adds no public configuration or caller budget:
|
||||
|
||||
| Budget | Unit and value |
|
||||
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES` | The sum of all inlineData base64 strings is 768 KiB, at most 576 KiB decoded; the inflation formula is `4 * ceil(decodedBytes / 3)`. |
|
||||
| `MAX_PROVIDER_MEDIA_RESULT_BYTES` | The complete ReadFile ToolResult is 896 KiB as UTF-8 JSON, including text, MIME/display metadata, and returnDisplay. |
|
||||
| Existing provider response ceiling | 1 MiB, including protocol, complete session identity, execution/status wrappers, and retained progress. |
|
||||
|
||||
These are transport ceilings and do not prove a specific model still has enough
|
||||
context. The Harness continues to enforce its own model and persistent-store
|
||||
checks. The 896 KiB check only constrains results that intend to deliver inline
|
||||
media. Text, SVG, notebook, and text-extracted PDF results continue to use
|
||||
existing text fitting, including legitimate text results larger than 896 KiB.
|
||||
|
||||
Add an optional trusted construction-time read policy to `ReadFileTool`, passed
|
||||
through `ProcessSingleFileContentOptions`. Only the provider worker sets it;
|
||||
local callers omit it and keep the current limits. It is not a tool parameter and
|
||||
cannot be enlarged through `mediaContext` or user settings. Apply it internally,
|
||||
before returning inline parts:
|
||||
|
||||
- Native PDF and raw-image reads constrain the actual bytes read, not just
|
||||
`stat.size`; a growing or replaced stream is rejected after at most the decoded
|
||||
limit plus one byte. Handles are closed on success, failure, and abort. The new
|
||||
helper must preserve existing file-service and permission semantics.
|
||||
- Image overviews measure the actual rendered Buffer before allocating base64.
|
||||
Preserve existing source-file/decoded limits; this feature introduces no
|
||||
arbitrary re-compression, resolution reduction, or new formats.
|
||||
- Add an optional aggregate base64 ceiling to the existing PDF renderer. The
|
||||
smaller policy also constrains the first page; the current "first page is
|
||||
always retained" exception no longer applies. Stop before a page that cannot
|
||||
fit in the read/encoding budget. With an explicit page range, reject incomplete
|
||||
renders; without a range, allow only a whole-page prefix with an explicit
|
||||
notice that later pages were omitted, and return FILE_TOO_LARGE if not even one
|
||||
page fits, preserving the specific single-page over-limit cause and
|
||||
recommending a smaller page or a smaller document. When the document page count
|
||||
is known, clamp the explicit range's end page to the actual count and report
|
||||
the document length; an unknown page count or a genuinely missing page still
|
||||
rejects an incomplete render.
|
||||
- Check both the encoded size of the complete media-bearing ReadFile ToolResult
|
||||
and the media bytes. Do not truncate base64, split PDF bytes, replace
|
||||
successful images with a generic success stub, or silently drop pages.
|
||||
|
||||
Pass the invocation AbortSignal into the `pdfinfo`, `pdftotext`, and `pdftoppm`
|
||||
operations owned by this call. `extractPDFText` already accepts a signal, but
|
||||
fileUtils must actually pass it; the renderer and page-count helper gain the
|
||||
same optional signal. Reuse `execCommand`/`execFile` cancellation, and normally
|
||||
wait for the owned child to exit and the renderer `finally` cleanup before
|
||||
returning cancellation. After cancellation, wait at most 1 second, then send
|
||||
SIGKILL; if close still does not arrive, wait at most another 1 second, unref the
|
||||
process and stdio event-loop handles, and reject the call, avoiding an unbounded
|
||||
wait. The renderer `finally` still performs cleanup; an OS process that cannot
|
||||
exit is not guaranteed to disappear immediately. Abort must be rethrown, not
|
||||
interpreted as an extraction failure that starts another fallback. The current
|
||||
helper can return abort as a failure value, so the signal must be checked
|
||||
explicitly after await, including before entering any fallback. Web Fetch
|
||||
deletes the unused persisted file when PDF extraction is cancelled and refunds
|
||||
its disk budget after a successful delete; cancellation continues to propagate.
|
||||
Do not bind a single invocation's signal to the shared availability probe
|
||||
Promise; those remain bounded shared probes. Tests cancel after each child
|
||||
process has started, not only before ReadFile execution.
|
||||
|
||||
Read budget errors use a normal `ToolResult.error` and
|
||||
`ToolErrorType.FILE_TOO_LARGE`, and the Runtime records `executionStatus: error`.
|
||||
Return a settled provider observation through HTTP 200 so the Broker saves the
|
||||
result and releases ownership, rather than turning it into a transport 413 or
|
||||
UNKNOWN. Malformed/over-limit protocol input continues to use existing rejection
|
||||
rules; a transport failure does not prove the read settled.
|
||||
|
||||
The response fitter may evict progress and trim auxiliary fields, but execute and
|
||||
settled status/cancel must preserve the complete accepted media-bearing
|
||||
ToolResult byte-for-byte. Error classification happens inside ReadFile, before
|
||||
`ManagedToolRuntime.run` retains the terminal state, and must not manufacture a
|
||||
different terminal delivery error in the HTTP copy. The 896 KiB ToolResult leaves
|
||||
128 KiB for wrappers and bounded auxiliary data; remove progress first, then
|
||||
reuse existing auxiliary fitting. Boundary tests must use a complete valid
|
||||
identity and the real execution/status wrappers to prove an admitted media
|
||||
result cannot enter the generic media-stub branch. If an anomalous result
|
||||
violates this invariant, reject it as a protocol fault rather than rewriting or
|
||||
ACKing the retained terminal evidence; this is an implementation error, not an
|
||||
ordinary size rejection. Do not add a second media projection cache or
|
||||
settlement state. The same reference must expose the same accepted model content;
|
||||
observations must not reread the file. After trimming Hook auxiliary fields,
|
||||
preserve a stop reason and Hook error that fit; if still over limit, first replace
|
||||
the largest diagnostic with an explicit omission notice. The fitter mutates the
|
||||
caller-held HTTP copy and may already have trimmed auxiliary fields before
|
||||
rejecting the response wrapper; the caller must discard the rejected copy. Reuse
|
||||
existing MIME/canonical base64 validation where possible, recognize only known
|
||||
inlineData leaves in llmContent, and do not exempt arbitrary metadata media.
|
||||
Java and TypeScript wrapper limits must stay consistent.
|
||||
|
||||
## 6. Storage, compatibility, and failure boundaries
|
||||
|
||||
Java stores bounded provider results in the existing tool execution ledger; no
|
||||
schema change or media table is needed. Media is private tool evidence: do not
|
||||
copy base64 into public token events, previews, logs, or error diagnostics. File
|
||||
content is untrusted data and cannot provide execution instructions.
|
||||
|
||||
Keep providerProtocol `/1` and the request/response shapes, because this only
|
||||
fulfills the existing mediaContext capability within the existing ceiling. An
|
||||
old worker's unsupported placeholder is identifiable during acceptance and must
|
||||
not be silently retried through Legacy or raw v2. The Harness's raw-v2 media
|
||||
ceiling is larger, so the provider must not send a larger response because of it.
|
||||
|
||||
When the caller's Session contract requires persistence, the exact accepted
|
||||
result must be committed before continuing to the model. The public Hosted raw
|
||||
file path's 64 KiB outcome/history limit cannot accept a 768 KiB provider
|
||||
payload. This design does not enlarge or bypass it. A future public Hosted media
|
||||
profile needs independent raw-tool modality binding and persistent media
|
||||
resource/message representation; a workspace path or public artifact preview
|
||||
alone is not model-recovery evidence.
|
||||
|
||||
status/cancel use the original prepared reference. After an execute reply is
|
||||
lost, status must recover the same media even if the file changed. After a
|
||||
Broker restart, read the saved terminal receipt; if the worker is lost before
|
||||
settlement, continue unknown and do not redispatch. release/new Turn eviction
|
||||
retain the existing lifecycle. Media is bounded per invocation, but retained
|
||||
invocation count can still consume substantial memory; acceptance covers that
|
||||
cost and does not claim a new global quota.
|
||||
|
||||
## 7. Implementation slices
|
||||
|
||||
| Slice | Files and concrete changes | Exit check |
|
||||
| ----------------------------- | ---------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| M1: single capability source | `packages/core/src/utils/fileUtils.ts`, provider worker media binding, local factory regression | Description and real read agree for image/PDF true/false/omitted, without depending on generator/credentials. |
|
||||
| M2: bounded read policy | `packages/core/src/tools/read-file.ts`, `utils/fileUtils.ts`, `utils/pdf.ts`, worker trusted construction policy | Real images, native PDFs, and rendered pages are deliverable; explicit size errors settle; local default behavior is unchanged. |
|
||||
| M3: retain media observations | `packages/cli/src/serve/managed-runtime-provider-protocol.ts`, worker routing, shared validator if needed | execute/status/cancel retain media; progress does not cause a stub; malformed payloads are not exempt. |
|
||||
| M4: Broker and consumption | `broker-managed-runtime-provider.test.ts`, Java `ProviderRuntimeTransportTest`, real-process driver | A real provider worker and Java Broker deliver bytes to a fake multimodal provider; the original reference recovers across a lost ACK. |
|
||||
|
||||
M1-M4 do not modify public OpenAPI, SQL migrations, or Hosted profile selection.
|
||||
After implementation, update the provider-control design in both English and
|
||||
Chinese; its current placeholder description is a historical baseline, not
|
||||
evidence of successful delivery.
|
||||
|
||||
M1 also adds `isOmniEnabled` to `DerivedConfigOverrides` in `config.ts`, set by
|
||||
the worker override and read by file processing; it adds no public setting and
|
||||
does not change the normal CLI Config default.
|
||||
|
||||
## 8. Verification and acceptance
|
||||
|
||||
Use small real PNG/JPEG/WebP/GIF and PDF fixtures to verify decoded bytes or
|
||||
rendered MIME/dimensions, not just success, descriptions, or "contains no
|
||||
placeholder". Fixtures live in the worker workspace; Harness decoy paths must not
|
||||
be read. PDF coverage includes native bytes, explicit text pages, image fallback,
|
||||
and missing Poppler. Renderer tests in the designated CI lane must actually run
|
||||
Poppler; a skip is not positive evidence.
|
||||
|
||||
Must cover:
|
||||
|
||||
1. Core: effective getter precedence, local mock fallback, local CLI/Managed
|
||||
regression, modalities on/off, text/SVG/notebook behavior.
|
||||
2. Worker: prepare different modalities in the same and different Sessions
|
||||
concurrently; retry conflict on changed context; omitted context must not
|
||||
inherit earlier capabilities; do not initialize model/auth/Vision
|
||||
Bridge/upload; permissions and workspace identity preserved.
|
||||
3. Budgets: exact boundary and one byte over, base64 inflation/padding,
|
||||
multibyte/escaped metadata, multi-page aggregate including the first page,
|
||||
growing raw file, corrupt image/PDF, explicit page rejection/implicit prefix
|
||||
notice, cancellation and renderer cleanup.
|
||||
4. Wire: real provider HTTP wrappers, maximum legal Session identity, chunked
|
||||
responses, execute with large progress and settled status/cancel; canonical
|
||||
base64/MIME rejection and over-limit non-media fields. Never split inlineData;
|
||||
never report a size rejection as successful delivery.
|
||||
5. Integration: real packaged worker, Java Broker, and MySQL ledger. Drive
|
||||
provider controls through `BrokerManagedRuntimeProvider.getToolV2Client`, not
|
||||
raw v2. Assert the seven-field reference, provider/1 worker routing, and
|
||||
nested `result.llmContent`; do not use raw `responseParts`. Complete
|
||||
prepare/approve/preflight/reserve/start. Set
|
||||
`execution = await client.execute(reference)`, confirm success and a result,
|
||||
then pass `execution.result.llmContent` to `convertToFunctionResponse` in
|
||||
`coreToolScheduler.ts`, then through the existing Harness model conversion to
|
||||
a deterministic multimodal fixture, and inspect the real outbound request.
|
||||
Native PDF acceptance uses the existing Chat Completions converter
|
||||
(`application/pdf` becomes a file part), and its image conversion must also
|
||||
pass. The Responses converter currently supports image results but turns PDFs
|
||||
into an unsupported placeholder; verify that existing limitation separately
|
||||
and do not claim native PDF support on that route. No real model credentials
|
||||
are needed. Lose start/execute replies, change the file, query the same
|
||||
reference, restart the Broker, and check the original bytes and that the tool
|
||||
invocation executes exactly once. The first sniff/render may require multiple
|
||||
file reads; recovery observations must not add reads or processing. release
|
||||
after error/cancellation must still respond.
|
||||
|
||||
During implementation, run the repository build/typecheck/bundle, per-package
|
||||
Core/CLI tests, Java Broker tests/Checkstyle, and the real MySQL process gate.
|
||||
Distinguish these results from a real-model smoke test. Media delivery assertions
|
||||
must fail when the effective getter or byte gate is removed, or when the fitter
|
||||
drops inlineData.
|
||||
|
||||
Acceptance requires image and native PDF bytes to reach the model fixture through
|
||||
the provider Broker path, PDF fallback to work, rejections to be bounded and
|
||||
honestly classified, capability snapshot isolation, and original-reference
|
||||
recovery consistency. Library tests or transport mocks passing alone do not
|
||||
constitute #13039 completion.
|
||||
|
||||
## 9. Risks and follow-up decisions
|
||||
|
||||
- The 768 KiB aggregate ceiling intentionally supports a bounded subset of local
|
||||
CLI media sizes. Enlarging it requires a versioned end-to-end transport,
|
||||
storage, and capacity design; changing one receiver is not enough.
|
||||
- PDF rendering and image decoding consume more memory/disk than the delivered
|
||||
payload. Before enabling in deployment, verify existing source-file/page/time
|
||||
limits and cumulative invocation retention; a bounded single response is not a
|
||||
global memory bound.
|
||||
- Poppler is a deployment dependency for text extraction and scanned-page
|
||||
rendering; when missing, return an honest tool error.
|
||||
- Public Hosted media, audio/video, object media, zoom, and custom ceilings each
|
||||
need their own consumer and contract; they are not implicit prerequisites for
|
||||
provider/1 image/PDF delivery.
|
||||
@@ -0,0 +1,266 @@
|
||||
# Managed Runtime Provider 媒体交付
|
||||
|
||||
[English](2026-10-07-managed-runtime-provider-media.md) | [简体中文](2026-10-07-managed-runtime-provider-media.zh-CN.md)
|
||||
|
||||
状态：provider/1 已实现；生产部署验收独立推进。
|
||||
日期：2026-10-07
|
||||
Issue：[#13039](https://github.com/QwenLM/qwen-code/issues/13039)，包含[根因评论](https://github.com/QwenLM/qwen-code/issues/13039#issuecomment-5895577269)。
|
||||
代码基线：上游与本地均为 `a764fb9698e6e1152e0a8bc5808c30a765744152`。
|
||||
|
||||
## 1. 决策与范围
|
||||
|
||||
让 `read_file` 通过现有 `managed-runtime-provider/1` worker、Java Runtime
|
||||
Broker 与 `BrokerManagedRuntimeProvider` 客户端交付真实图片和 PDF 内容。
|
||||
保留七字段 prepared reference、审批、preflight、原执行观察与释放协议。
|
||||
Runtime 只接收模态布尔值，不接收模型配置、Prompt、凭证或模型接口访问能力。
|
||||
|
||||
工具描述与实际文件读取统一使用 `Config.getEffectiveInputModalities()`。
|
||||
在现有 ReadFile 处理路径增加由 worker 持有的内联媒体上限。响应 fitting 保留
|
||||
有效媒体；超限读取返回明确且已结算的工具错误。
|
||||
|
||||
本 issue 实现 provider 路径，不迁移所有 Hosted 工具 profile。公开 Hosted 文件
|
||||
循环目前使用 raw Tool v2 和独立的 64 KiB 内联 Session 资源。替换该循环、开放
|
||||
公开媒体、新 artifact API、音视频交付、Runtime 内 Vision Bridge 推理与对象
|
||||
存储媒体传输均属于独立改动。provider 成功交付不能证明这些能力已经成立。
|
||||
|
||||
## 2. 当前契约与缺陷
|
||||
|
||||
| 范围 | 基线已核实行为 |
|
||||
| ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `managed-runtime-provider-worker.ts:createRuntime` | 创建无 content generator 的执行 Config。媒体绑定 ReadFile 仅覆盖 `getEffectiveInputModalities`。 |
|
||||
| `fileUtils.ts:processSingleFileContent` | 读取 `getContentGeneratorConfig()?.modalities`，因此图片仍不支持，原生 PDF 与 PDF 转图片回退也不可用；PDF 文本提取已存在。 |
|
||||
| `managed-tool-runtime.ts:prepare` | 复制 `mediaContext` 并纳入准备摘要；同一存活调用改变媒体上下文会冲突。 |
|
||||
| `managed-runtime-provider-protocol.ts` / Java `ProviderRuntimeProtocol` | 闭合 `mediaContext: { inputModalities }`，允许布尔键 `image`、`pdf`、`audio`、`video`。普通响应包括 execute/status/cancel 均为 1 MiB，历史操作为 8 MiB。 |
|
||||
| `fitManagedRuntimeProviderResult` | 淘汰进度并裁剪文本，最后把不可裁剪的内联媒体替换为通用 stub；处理的是 HTTP 副本，不是 Runtime 保留的执行结果。 |
|
||||
| Java `HttpRuntimeTransport` | 七字段引用选择 provider execute/status/cancel；四字段 raw Tool v2 使用不同路由和结果形态。 |
|
||||
| `managed-tool-media.ts` | 较大的验证媒体例外属于 ACP/raw Tool v2，不属于 provider/1，不会扩大其 1 MiB 上限。 |
|
||||
| `hosted-workspace-tool-turn.ts` | 公开 Hosted 文件选择 raw v2；持久 outcome/message 上限可独立拒绝媒体，不由传输成功决定。 |
|
||||
|
||||
Issue 评论建议覆盖 content-generator 配置。本方案优先使用既有有效模态 getter：
|
||||
生产 Config 默认从同一已解析配置返回模态，避免为执行 worker 伪造模型和鉴权
|
||||
配置。仅为未实现有效 getter 的既有局部 Config 测试替身，保留
|
||||
`getContentGeneratorConfig()?.modalities ?? {}` 回退。有效 getter 返回空对象
|
||||
就是权威结果，不能用回退重新启用已经关闭的模态。
|
||||
|
||||
## 3. 架构与时序
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant H as Harness/provider 调用方
|
||||
participant B as Java Runtime Broker
|
||||
participant R as Provider worker
|
||||
participant F as Workspace ReadFile
|
||||
H->>B: prepare(identity, read_file, input, mediaContext)
|
||||
B->>R: 已认证 provider/1 prepare
|
||||
R->>R: 为本次 invocation 复制并绑定模态
|
||||
R-->>B: 七字段 prepared reference
|
||||
B-->>H: 七字段 prepared reference
|
||||
H->>B: confirmation / confirm / preflight
|
||||
H->>B: reserve / start(reference)
|
||||
B->>R: execute 原 prepared reference
|
||||
R->>F: 按固定 worker 媒体预算读取
|
||||
F-->>R: 有效 inline parts 或明确有界工具错误
|
||||
R->>R: 保留终态执行证据
|
||||
R-->>B: 有界 provider 执行结果
|
||||
B->>B: 持久化原执行结果
|
||||
B-->>H: 原结果包含 inlineData
|
||||
H->>H: 通过既有模型侧媒体转换消费
|
||||
Note over H,R: 丢应答只查原 reference 的 status，不再次读取
|
||||
```
|
||||
|
||||
路由继续遵循当前归属：worker control 属于 live-session-owner；Broker
|
||||
start/status/cancel 使用持久执行所有权和原 Runtime generation。
|
||||
`mediaContext` 表达模型能力，不授予文件权限。既有 Workspace 授权、Session
|
||||
绑定、审批与 file-service 检查继续位于读取链路中。模型改变只影响新的准备。
|
||||
|
||||
## 4. 模态绑定与读取语义
|
||||
|
||||
1. 在 prepare 解析并复制现有媒体上下文。省略上下文保持 worker 现有的纯文本
|
||||
媒体行为，包括 PDF 文本提取。
|
||||
2. 为 invocation 创建独立 derived Config。本 provider profile 仅在复制的
|
||||
对应布尔值为 true 时启用 image/PDF；即使模型支持，audio/video 也保持关闭。
|
||||
描述与执行使用同一有效视图。不修改 Session 或进程全局 Config，不初始化
|
||||
generator，不根据模型名称推断能力。
|
||||
3. 保留 file service、target directory、权限规则、禁用的 read cache 和
|
||||
file-history service。非 ReadFile 工具继续拒绝 `mediaContext`。不注册
|
||||
`zoom_image`，也不广告此四工具 worker 并不存在的 zoom 能力。
|
||||
在 worker 的 derived Config 上把 `isOmniEnabled` 固定为 false，避免继承
|
||||
的环境开关加载媒体上传流程。本地普通 CLI Config 保留既有 Omni 行为。
|
||||
4. 把 `processSingleFileContent` 的模态读取改为有效 getter。本地 CLI/Legacy
|
||||
继续使用既有解析模态。本地 Managed factory 已覆盖此 getter，必须做回归，
|
||||
因为它会从只影响描述变成也影响真实读取。
|
||||
|
||||
| 输入与能力 | 要求结果 |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------- |
|
||||
| 支持的图片，`image: true` | 复用原生 overview 与安全格式处理，在预算内交付有效 image inlineData；保留既有损坏/不安全格式提示。 |
|
||||
| 图片且 image 关闭或缺省 | 原 unsupported-image 提示，无内联字节。 |
|
||||
| 无 `pages` 的 PDF，`pdf: true` | 可容纳时交付原生 `application/pdf` inlineData，否则明确 FILE_TOO_LARGE 并提示使用 `pages`。 |
|
||||
| 带 `pages`，或不支持原生 PDF | 既有文本优先提取与 token/页数检查，纯文本模型仍可读文本 PDF。 |
|
||||
| 扫描/高密度 PDF，`image: true` | 既有页面渲染，受更小的 provider 聚合预算约束。 |
|
||||
| 音视频 | 既有 unsupported 提示，不内联字节，不远程上传。 |
|
||||
| SVG/notebook/text | 保持既有非媒体行为与权限规则。 |
|
||||
|
||||
通用 manifest 不承诺交付每种媒体。带媒体的 preparation 和真实输出才是验收
|
||||
证据。Harness 调用方通过既有 `ManagedToolV2Client.prepare` 第五参数传入
|
||||
已解析能力快照；已发布的 Hosted raw 循环目前不是此参数的生产调用方。
|
||||
不能仅凭客户端会转发就宣称默认产品能力已开放。
|
||||
|
||||
## 5. 字节策略与终态可观察性
|
||||
|
||||
provider/1 保持 1 MiB。首版策略固定在 worker 代码内，不增加公开配置或调用方预算：
|
||||
|
||||
| 预算 | 单位与值 |
|
||||
| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
|
||||
| `MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES` | 全部 inlineData base64 字符串之和为 768 KiB，最多对应 576 KiB 解码字节；膨胀公式为 `4 * ceil(decodedBytes / 3)`。 |
|
||||
| `MAX_PROVIDER_MEDIA_RESULT_BYTES` | 完整 ReadFile ToolResult 的 JSON UTF-8 编码为 896 KiB，含文本、MIME/display 元数据和 returnDisplay。 |
|
||||
| 既有 provider 响应上限 | 1 MiB，含协议、完整 Session 身份、execution/status 包装与保留进度。 |
|
||||
|
||||
这些是传输上限，不证明特定模型还有足够上下文。Harness 继续执行自身模型和
|
||||
持久存储校验。
|
||||
896 KiB 检查只约束包含拟内联交付媒体的结果。text、SVG、notebook 和文本
|
||||
提取 PDF 继续使用既有文本 fitting，包括大于 896 KiB 的合法文本结果。
|
||||
|
||||
为 `ReadFileTool` 增加可选的可信构造期读取策略，经
|
||||
`ProcessSingleFileContentOptions` 转交。仅 provider worker 设置，本地调用
|
||||
省略并保持现有上限。这不是工具参数，不能通过 `mediaContext` 或用户设置扩大。
|
||||
在处理内部、返回 inline part 前执行：
|
||||
|
||||
- 原生 PDF 与 raw-image 读取约束实际读取字节，不能仅信任 `stat.size`；增长
|
||||
或替换的流最多读解码上限加一字节就拒绝。成功、失败和 abort 都关闭 handle。
|
||||
新 helper 必须保留现有 file-service 与权限语义。
|
||||
- 图片 overview 在分配 base64 之前测量真实渲染 Buffer。保留既有源文件/解码
|
||||
上限；本 feature 不引入任意重压缩、降低分辨率或新增格式。
|
||||
- 给现有 PDF renderer 增加可选聚合 base64 上限。更小策略也约束第一页，不能
|
||||
沿用当前“第一页总是保留”的例外。在读取/编码无法容纳的页面前停止。有明确
|
||||
page range 时拒绝不完整渲染；无 range 时只允许带明确省略后续页面提示的
|
||||
整页前缀；一页都容纳不下则 FILE_TOO_LARGE，并保留具体单页超限原因，建议选择更小的页面或缩小文档。
|
||||
已知文档页数时，显式范围的结束页按实际页数收敛并告知文档长度；
|
||||
未知页数或真正缺页仍拒绝不完整渲染。
|
||||
- 同时检查完整含媒体 ReadFile ToolResult 的编码大小和媒体字节。不裁剪 base64，
|
||||
不切分 PDF 字节，不把成功图片替换为通用 success stub，不静默丢页。
|
||||
|
||||
把 invocation AbortSignal 传入本次调用拥有的 `pdfinfo`、`pdftotext` 与
|
||||
`pdftoppm` 操作。`extractPDFText` 已接受 signal，但 fileUtils 必须真正传入；
|
||||
renderer 和页数 helper 增加同样的可选 signal。复用 `execCommand`/`execFile`
|
||||
取消，通常在返回取消前等待所属子进程退出和 renderer `finally` 清理。
|
||||
取消后最多等待 1 秒，再发送 SIGKILL；若仍未收到 close，最多再等待 1 秒，
|
||||
解除进程和 stdio 的事件循环引用并拒绝该调用，避免无限等待。renderer 的
|
||||
`finally` 仍执行清理；无法退出的 OS 进程不保证立即消失。abort 必须
|
||||
重新抛出，不能解释为提取失败再启动另一条回退。当前 helper 可以把 abort 返回成
|
||||
失败值，必须在 await 后显式检查 signal，包括进入任意回退之前。
|
||||
Web Fetch 在 PDF 提取取消时删除未使用的持久文件，并在成功删除后退还其磁盘预算；取消继续传播。
|
||||
不能把单 invocation 的 signal 绑到共享 availability probe Promise，
|
||||
它们仍是有界共享探测。测试在每种子进程
|
||||
已经启动之后取消，不能仅覆盖 ReadFile 执行前取消。
|
||||
|
||||
读取预算错误使用正常 `ToolResult.error` 与 `ToolErrorType.FILE_TOO_LARGE`，
|
||||
Runtime 记录 `executionStatus: error`。通过 HTTP 200 返回已结算的 provider
|
||||
观察，让 Broker 保存结果并释放所有权，不变成传输 413 或 UNKNOWN。
|
||||
畸形/超限协议输入继续使用既有拒绝规则；传输失败不证明读取已结算。
|
||||
|
||||
响应 fitter 可以淘汰 progress 并裁剪辅助字段，但 execute 与已结算
|
||||
status/cancel 必须逐字节保留完整已接受的含媒体 ToolResult。错误分类位于
|
||||
ReadFile 内部，在 `ManagedToolRuntime.run` 保留终态之前确定，不能在 HTTP
|
||||
副本中临时制造不同的终态交付错误。896 KiB ToolResult 为包装和有界辅助数据
|
||||
留出 128 KiB；先去掉 progress，再复用既有辅助 fitting。用完整合法身份和真实
|
||||
execution/status 包装做边界测试，证明已准入媒体结果不能进入通用 media-stub
|
||||
分支。若异常结果违反此不变量，按协议故障拒绝，不改写或 ACK 已保留的终态证据；
|
||||
这是实现错误，不是普通大小拒绝。不增加第二套媒体投影 cache 或结算状态。
|
||||
同一 reference 必须暴露同一已接受模型内容；观察不能重新读取文件。
|
||||
裁剪 Hook 辅助字段后，保留能容纳的停止原因和 Hook 错误；仍然超限时，先将最大的诊断替换为明确的省略提示。fitter 修改调用方持有的 HTTP 副本，
|
||||
最终拒绝响应包装前可能已经裁剪辅助字段，调用方必须丢弃被拒绝的副本。
|
||||
尽可能复用既有 MIME/规范 base64 校验，只识别 llmContent 中已知
|
||||
inlineData 叶子，不能给任意元数据媒体豁免。Java 与 TypeScript 包装上限保持一致。
|
||||
|
||||
## 6. 存储、兼容与失败边界
|
||||
|
||||
Java 把有界 provider 结果保存在既有工具执行 ledger，不需要 schema 变更或
|
||||
媒体表。媒体属于私有工具证据，不把 base64 复制到公开 token 事件、preview、
|
||||
日志或错误诊断。文件内容是非可信数据，不能提供执行指令。
|
||||
|
||||
保持 providerProtocol `/1` 和请求/响应形态，因为只是在既有上限内兑现已有
|
||||
mediaContext 能力。旧 worker 的 unsupported 占位可在验收中识别，不能静默
|
||||
经 Legacy 或 raw v2 重试。Harness 的 raw-v2 媒体上限更大，不允许 provider
|
||||
因此发送更大的响应。
|
||||
|
||||
当调用方 Session 契约要求持久化时，必须先提交精确接受的结果再继续模型。
|
||||
公开 Hosted raw 文件路径的 64 KiB outcome/history 上限不能接纳 768 KiB
|
||||
provider 负载。本设计不扩大或绕过它。未来公开 Hosted 媒体 profile 需要独立的
|
||||
raw-tool 模态绑定和持久媒体资源/message 表达，不能仅用 Workspace 路径或
|
||||
公开 artifact preview 作为模型恢复证据。
|
||||
|
||||
status/cancel 使用原 prepared reference。execute 应答丢失后，即使文件改变，
|
||||
status 也必须恢复同一媒体。Broker 重启读取已保存终态 receipt；结算前 worker
|
||||
丢失继续 unknown，不允许重新派发。release/新 Turn 淘汰保留既有生命周期。
|
||||
媒体每 invocation 有界，但现有调用数量保留仍可消耗大量内存；验收包含该成本，
|
||||
不宣称新增全局 quota。
|
||||
|
||||
## 7. 实施切片
|
||||
|
||||
| 切片 | 文件与具体改动 | 退出检查 |
|
||||
| --------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| M1：单一能力来源 | `packages/core/src/utils/fileUtils.ts`、provider worker 媒体绑定、本地 factory 回归 | image/PDF true/false/缺省时描述与真实读取一致，不依赖 generator/凭证。 |
|
||||
| M2：有界读取策略 | `packages/core/src/tools/read-file.ts`、`utils/fileUtils.ts`、`utils/pdf.ts`，worker 可信构造策略 | 真实图片、原生 PDF、渲染页可交付，明确大小错误可结算，本地默认行为保持。 |
|
||||
| M3：保留媒体观察 | `packages/cli/src/serve/managed-runtime-provider-protocol.ts`、worker 路由，必要时共享 validator | execute/status/cancel 保留媒体，progress 不造成 stub，畸形负载不能豁免。 |
|
||||
| M4：Broker 与消费证明 | `broker-managed-runtime-provider.test.ts`、Java `ProviderRuntimeTransportTest`、真实进程 driver | 真实 provider worker 与 Java Broker 把字节交给假多模态 provider，原引用恢复经受 ACK 丢失。 |
|
||||
|
||||
M1-M4 不修改公开 OpenAPI、SQL migration 或 Hosted profile 选择。实现后同步
|
||||
修改 provider-control 的中英文设计；其中当前占位描述属于历史基线，不是成功
|
||||
交付证据。
|
||||
|
||||
M1 还在 `config.ts` 的 `DerivedConfigOverrides` 中增加 `isOmniEnabled`，
|
||||
由 worker 设置覆盖、文件处理读取，不增加公开设置或改变普通 CLI Config 默认值。
|
||||
|
||||
## 8. 验证与验收
|
||||
|
||||
使用小型真实 PNG/JPEG/WebP/GIF 与 PDF fixture，验证解码字节或渲染 MIME/
|
||||
尺寸，不能只看 success、描述或“不含占位”。fixture 在 worker Workspace，
|
||||
Harness 诱饵路径不能被读取。PDF 覆盖原生字节、明确文本页、图片回退与 Poppler
|
||||
缺失。指定 CI lane 的渲染测试必须实际运行 Poppler；skip 不能作为正向证据。
|
||||
|
||||
必须覆盖：
|
||||
|
||||
1. Core：有效 getter 优先级、局部 mock 回退、本地 CLI/Managed 回归、模态开启/
|
||||
关闭、text/SVG/notebook 行为。
|
||||
2. Worker：同 Session 与跨 Session 同时准备不同模态；改上下文重试冲突；省略
|
||||
上下文不能继承先前能力；不初始化模型/auth/Vision Bridge/upload；权限与
|
||||
Workspace 身份保持。
|
||||
3. 预算：精确边界与多一字节、base64 膨胀/padding、多字节/转义元数据、含首
|
||||
页的多页聚合、增长 raw 文件、损坏图片/PDF、显式页拒绝/隐式前缀提示、取消
|
||||
和 renderer 清理。
|
||||
4. Wire：真实 provider HTTP 包装、最大合法 Session 身份、chunked 响应、带
|
||||
大 progress 的 execute 与 settled status/cancel；规范 base64/MIME 拒绝及
|
||||
超限非媒体字段。绝不切 inlineData，绝不把大小拒绝报告为成功交付。
|
||||
5. 集成：真实打包 worker、Java Broker 和 MySQL ledger。用
|
||||
`BrokerManagedRuntimeProvider.getToolV2Client` 驱动 provider controls，不
|
||||
直接走 raw v2。断言七字段引用、provider/1 worker 路由与嵌套
|
||||
`result.llmContent`，不能使用 raw `responseParts`。完成
|
||||
prepare/approve/preflight/reserve/start。令
|
||||
`execution = await client.execute(reference)`，确认成功且包含结果，再将
|
||||
`execution.result.llmContent` 传入 `coreToolScheduler.ts` 的
|
||||
`convertToFunctionResponse`，再经既有 Harness 模型转换把返回媒体
|
||||
送到确定性多模态 fixture，检查真实出站请求。原生 PDF 验收使用既有
|
||||
Chat Completions converter（`application/pdf` 转为 file part），其图片转换
|
||||
也必须通过。Responses converter 当前支持图片结果，但会把 PDF 变成
|
||||
unsupported 占位；单独验证此既有限制，不宣称该路由支持原生 PDF。
|
||||
不需要真实模型凭证。丢 start/execute 应答、修改文件、查同一引用、重启
|
||||
Broker，核对原字节和工具 invocation 只执行一次。首次 sniff/渲染可以需要
|
||||
多次文件读取；恢复观察不得新增读取或处理。错误/取消后 release 仍可回答。
|
||||
|
||||
实现阶段执行仓库 build/typecheck/bundle、按包 Core/CLI 测试、Java Broker
|
||||
测试/Checkstyle、真实 MySQL 进程门禁。区分这些结果与真实模型 smoke。
|
||||
移除有效 getter、字节 gate 或让 fitter 丢 inlineData 时，媒体交付断言必须失败。
|
||||
|
||||
验收要求图片和原生 PDF 字节经 provider Broker 路径到达模型 fixture，PDF
|
||||
回退成立、拒绝有界且分类诚实、能力快照隔离、原引用恢复一致。仅库测试或
|
||||
传输 mock 通过不构成 #13039 完成。
|
||||
|
||||
## 9. 风险与后续决策
|
||||
|
||||
- 768 KiB 聚合上限有意支持本地 CLI 媒体大小的有界子集。扩大需要版本化的
|
||||
端到端传输、存储和容量设计，不能只改一个接收方。
|
||||
- PDF 渲染和图片解码的内存/磁盘消耗大于交付负载。部署开启前验证既有源文件/
|
||||
页数/时间限制与累积 invocation 保留，不把单次响应有界等同全局内存有界。
|
||||
- Poppler 是文本提取和扫描页渲染的部署依赖，缺失时返回诚实工具错误。
|
||||
- 公开 Hosted 媒体、音视频、对象媒体、zoom、自定义上限需要各自的消费者和
|
||||
契约，不是 provider/1 图片/PDF 交付的隐含前置。
|
||||
@@ -25,6 +25,13 @@ import {
|
||||
HOSTED_STORE_PREFIX,
|
||||
} from './scratch-dir.js';
|
||||
|
||||
import {
|
||||
PROVIDER_MEDIA_MYSQL_PREFIX,
|
||||
PROVIDER_MEDIA_HARNESS_PREFIX,
|
||||
PROVIDER_MEDIA_FIXTURES_PREFIX,
|
||||
PROVIDER_MEDIA_PROBE_PREFIX,
|
||||
} from './helpers/provider-media-process-cleanup.mjs';
|
||||
|
||||
// The env keys globalSetup's setup() writes, saved so a case can restore the
|
||||
// suite-wide values after re-importing the module and running its lifecycle.
|
||||
const SETUP_ENV_KEYS = [
|
||||
@@ -155,6 +162,10 @@ describe('globalSetup hermetic qwen home', () => {
|
||||
['per-agent ACP home', ACP_HOME_PREFIX],
|
||||
['Hosted process root', HOSTED_HOME_PREFIX],
|
||||
['Hosted Session Store root', HOSTED_STORE_PREFIX],
|
||||
['provider media MySQL root', PROVIDER_MEDIA_MYSQL_PREFIX],
|
||||
['provider media harness root', PROVIDER_MEDIA_HARNESS_PREFIX],
|
||||
['provider media fixture root', PROVIDER_MEDIA_FIXTURES_PREFIX],
|
||||
['provider media probe root', PROVIDER_MEDIA_PROBE_PREFIX],
|
||||
])(
|
||||
'sweeps a stale %s a torn-down run left behind',
|
||||
async (_label, prefix) => {
|
||||
|
||||
@@ -0,0 +1,235 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
// Test-only adapter from the Managed Runtime Broker client to the
|
||||
// ManagedToolV2Client surface used by the provider-media M4 driver.
|
||||
|
||||
import { setTimeout as delay } from 'node:timers/promises';
|
||||
import type {
|
||||
ManagedToolExecutionResult,
|
||||
ManagedToolInvocationStatus,
|
||||
ManagedToolV2Client,
|
||||
} from '@qwen-code/qwen-code-core/tools/managed-tool-runtime.js';
|
||||
import type { ManagedToolInvocationReference } from '@qwen-code/qwen-code-core/tools/managed-tool-protocol.js';
|
||||
import {
|
||||
ManagedRuntimeBrokerClient,
|
||||
type ManagedRuntimeBrokerClientOptions,
|
||||
} from '../../packages/cli/src/serve/managed-runtime-broker-client.js';
|
||||
import type { ManagedRuntimePrepareRequest } from '../../packages/cli/src/serve/managed-runtime-protocol.js';
|
||||
import {
|
||||
parseManagedRuntimeProviderOperation,
|
||||
parseManagedRuntimeProviderResult,
|
||||
type ManagedRuntimeProviderOperation,
|
||||
} from '../../packages/cli/src/serve/managed-runtime-provider-protocol.js';
|
||||
|
||||
const REQUEST_TIMEOUT_MS = 60_000;
|
||||
const POLL_DELAY_MS = 100;
|
||||
|
||||
interface SessionEntry {
|
||||
readonly request: ManagedRuntimePrepareRequest;
|
||||
readonly harnessSessionId: string;
|
||||
readonly executions: Map<string, string>;
|
||||
}
|
||||
|
||||
interface ExecutionIdentity {
|
||||
harnessSessionId: string;
|
||||
runtimeSessionId: string;
|
||||
executionCallId: string;
|
||||
afterSeq?: number;
|
||||
}
|
||||
|
||||
type Inspection =
|
||||
| { outcome: 'known'; status: ManagedToolInvocationStatus }
|
||||
| { outcome: 'unknown' };
|
||||
|
||||
export class BrokerManagedRuntimeProvider {
|
||||
private readonly client: ManagedRuntimeBrokerClient;
|
||||
private readonly lifetime = new AbortController();
|
||||
private readonly sessions = new Map<string, SessionEntry>();
|
||||
private readonly released = new Set<string>();
|
||||
|
||||
constructor(options: ManagedRuntimeBrokerClientOptions) {
|
||||
this.client = new ManagedRuntimeBrokerClient(options);
|
||||
}
|
||||
|
||||
async getToolV2Client(
|
||||
request: ManagedRuntimePrepareRequest,
|
||||
context?: { harnessSessionId?: string },
|
||||
): Promise<ManagedToolV2Client> {
|
||||
const harnessSessionId = context?.harnessSessionId?.trim();
|
||||
if (!harnessSessionId) {
|
||||
throw new Error('Harness Session identity is required.');
|
||||
}
|
||||
await this.client.acquire(request, harnessSessionId, this.signal());
|
||||
const entry: SessionEntry = {
|
||||
request,
|
||||
harnessSessionId,
|
||||
executions: new Map(),
|
||||
};
|
||||
this.sessions.set(request.sessionId, entry);
|
||||
return this.createClient(entry);
|
||||
}
|
||||
|
||||
async release(
|
||||
sessionId: string,
|
||||
request?: ManagedRuntimePrepareRequest,
|
||||
_options?: { terminal?: boolean },
|
||||
): Promise<boolean> {
|
||||
if (this.released.has(sessionId)) return true;
|
||||
const entry = this.sessions.get(sessionId);
|
||||
const harnessSessionId = entry?.harnessSessionId ?? request?.sessionId;
|
||||
if (!harnessSessionId) return false;
|
||||
const released = await this.client.release(
|
||||
sessionId,
|
||||
harnessSessionId,
|
||||
this.signal(),
|
||||
);
|
||||
if (released) {
|
||||
this.released.add(sessionId);
|
||||
this.sessions.delete(sessionId);
|
||||
}
|
||||
return released;
|
||||
}
|
||||
|
||||
async inspectExecution(identity: ExecutionIdentity): Promise<Inspection> {
|
||||
const status = await this.client.getExecution(
|
||||
identity.runtimeSessionId,
|
||||
identity.harnessSessionId,
|
||||
identity.executionCallId,
|
||||
identity.afterSeq,
|
||||
this.signal(),
|
||||
);
|
||||
return { outcome: 'known', status };
|
||||
}
|
||||
|
||||
dispose(): void {
|
||||
this.lifetime.abort();
|
||||
}
|
||||
|
||||
private signal(): AbortSignal {
|
||||
return AbortSignal.any([
|
||||
this.lifetime.signal,
|
||||
AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
||||
]);
|
||||
}
|
||||
|
||||
private createClient(entry: SessionEntry): ManagedToolV2Client {
|
||||
const control = async (operation: ManagedRuntimeProviderOperation) => {
|
||||
const session = {
|
||||
harnessSessionId: entry.harnessSessionId,
|
||||
runtimeSessionId: entry.request.sessionId,
|
||||
turnKind: entry.request.turnKind,
|
||||
};
|
||||
const parsed = parseManagedRuntimeProviderOperation(operation, session);
|
||||
return parseManagedRuntimeProviderResult(
|
||||
parsed,
|
||||
await this.client.control(
|
||||
entry.request.sessionId,
|
||||
entry.harnessSessionId,
|
||||
parsed,
|
||||
this.signal(),
|
||||
),
|
||||
session,
|
||||
);
|
||||
};
|
||||
const executionCallId = (
|
||||
reference: ManagedToolInvocationReference,
|
||||
): string => {
|
||||
const id = entry.executions.get(reference.invocationId);
|
||||
if (!id) {
|
||||
throw new Error('Managed Runtime execution was not prepared.');
|
||||
}
|
||||
return id;
|
||||
};
|
||||
return {
|
||||
fileHistory: {
|
||||
bind: (binding) => control({ kind: 'bind-history', binding }),
|
||||
checkpoint: (promptId) => control({ kind: 'checkpoint', promptId }),
|
||||
snapshot: () => control({ kind: 'history' }),
|
||||
},
|
||||
manifest: () => control({ kind: 'manifest' }),
|
||||
beginTurn: async (identity) => {
|
||||
await control({ kind: 'begin-turn', identity });
|
||||
},
|
||||
prepare: (identity, toolName, input, modification, mediaContext) =>
|
||||
control({
|
||||
kind: 'prepare',
|
||||
identity,
|
||||
toolName,
|
||||
input,
|
||||
...(modification === undefined ? {} : { modification }),
|
||||
...(mediaContext === undefined ? {} : { mediaContext }),
|
||||
}),
|
||||
confirmation: (reference) => control({ kind: 'confirmation', reference }),
|
||||
confirm: async (reference, outcome, payload, phase) => {
|
||||
await control({
|
||||
kind: 'confirm',
|
||||
reference,
|
||||
outcome,
|
||||
...(payload === undefined ? {} : { payload }),
|
||||
...(phase === undefined ? {} : { phase }),
|
||||
});
|
||||
},
|
||||
preflight: (reference) => control({ kind: 'preflight', reference }),
|
||||
prepareExecution: async (reference) => {
|
||||
const reservation = await this.client.prepareExecution(
|
||||
entry.request.sessionId,
|
||||
entry.harnessSessionId,
|
||||
reference,
|
||||
this.signal(),
|
||||
);
|
||||
entry.executions.set(
|
||||
reference.invocationId,
|
||||
reservation.executionCallId,
|
||||
);
|
||||
return {
|
||||
executionCallId: reservation.executionCallId,
|
||||
invocationBindingId: entry.request.sessionId,
|
||||
};
|
||||
},
|
||||
execute: async (reference) => {
|
||||
const callId = executionCallId(reference);
|
||||
let status = await this.client.startExecution(
|
||||
entry.request.sessionId,
|
||||
entry.harnessSessionId,
|
||||
callId,
|
||||
this.signal(),
|
||||
);
|
||||
while (status.state !== 'settled') {
|
||||
await delay(POLL_DELAY_MS, undefined, {
|
||||
signal: this.lifetime.signal,
|
||||
});
|
||||
status = await this.client.getExecution(
|
||||
entry.request.sessionId,
|
||||
entry.harnessSessionId,
|
||||
callId,
|
||||
undefined,
|
||||
this.signal(),
|
||||
);
|
||||
}
|
||||
return status.result as ManagedToolExecutionResult;
|
||||
},
|
||||
status: (reference, afterSeq) =>
|
||||
this.client.getExecution(
|
||||
entry.request.sessionId,
|
||||
entry.harnessSessionId,
|
||||
executionCallId(reference),
|
||||
afterSeq,
|
||||
this.signal(),
|
||||
),
|
||||
cancel: (reference) =>
|
||||
// The broker client has no execution-cancel route, and the M4 driver
|
||||
// only cancels after settlement; return the same settled observation.
|
||||
this.client.getExecution(
|
||||
entry.request.sessionId,
|
||||
entry.harnessSessionId,
|
||||
executionCallId(reference),
|
||||
undefined,
|
||||
this.signal(),
|
||||
),
|
||||
} as ManagedToolV2Client;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,628 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import assert from 'node:assert/strict';
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||
import { createServer } from 'node:http';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import type { GenerateContentParameters, Part } from '@google/genai';
|
||||
import OpenAI from 'openai';
|
||||
import sharp from 'sharp';
|
||||
import { convertToFunctionResponse } from '@qwen-code/qwen-code-core/core/coreToolScheduler.js';
|
||||
import { convertLlmRequestToOpenAI } from '@qwen-code/qwen-code-core/core/openaiContentGenerator/converter.js';
|
||||
import { convertGeminiContentsToResponsesInput } from '@qwen-code/qwen-code-core/core/openaiResponsesContentGenerator/responses-converter.js';
|
||||
import { ToolConfirmationOutcome } from '@qwen-code/qwen-code-core/tools/tools.js';
|
||||
import {
|
||||
parseManagedToolInvocationReference,
|
||||
type ManagedToolMediaContext,
|
||||
} from '@qwen-code/qwen-code-core/tools/managed-tool-protocol.js';
|
||||
import type {
|
||||
ManagedToolExecutionResult,
|
||||
ManagedToolInvocationStatus,
|
||||
} from '@qwen-code/qwen-code-core/tools/managed-tool-runtime.js';
|
||||
import { BrokerManagedRuntimeProvider } from './broker-tool-v2-provider.js';
|
||||
import { waitUntil } from './hosted-harness-process.js';
|
||||
import { mediaFixture, type MediaCase } from './provider-media-fixtures.js';
|
||||
import { PROVIDER_MEDIA_HARNESS_PREFIX } from './provider-media-process-cleanup.mjs';
|
||||
|
||||
const configPath = process.argv[2];
|
||||
const config = JSON.parse(await readFile(configPath, 'utf8')) as {
|
||||
tenantId: string;
|
||||
brokerUrl: string;
|
||||
statusGateUrl: string;
|
||||
sessions: Array<{
|
||||
sessionId: string;
|
||||
workspaceId: string;
|
||||
directory: string;
|
||||
fault: MediaCase;
|
||||
}>;
|
||||
};
|
||||
assert(config.sessions.length > 0);
|
||||
const digest = (value: string | Buffer) =>
|
||||
createHash('sha256').update(value).digest('hex');
|
||||
let brokerUrl = config.brokerUrl;
|
||||
let dropStart = false;
|
||||
let startDrops = 0;
|
||||
let modelRequests = 0;
|
||||
let expectedCallId = '';
|
||||
let expectedSplitToolMedia = true;
|
||||
let expectedMedia: Array<{
|
||||
mime: string;
|
||||
hash: string;
|
||||
bytes: number;
|
||||
width?: number;
|
||||
height?: number;
|
||||
}> = [];
|
||||
let proxyFailure: unknown;
|
||||
const proxy = createServer(async (req, res) => {
|
||||
try {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of req) chunks.push(Buffer.from(chunk));
|
||||
const bytes = Buffer.concat(chunks);
|
||||
const upstream = await fetch(new URL(req.url!, brokerUrl), {
|
||||
method: req.method,
|
||||
headers: {
|
||||
authorization: 'Bearer hosted-tools-broker-token',
|
||||
'content-type': 'application/json',
|
||||
},
|
||||
...(bytes.length ? { body: bytes } : {}),
|
||||
signal: AbortSignal.timeout(45_000),
|
||||
});
|
||||
const response = Buffer.from(await upstream.arrayBuffer());
|
||||
if (dropStart && req.url!.endsWith(':start') && upstream.status === 200) {
|
||||
dropStart = false;
|
||||
startDrops++;
|
||||
res.destroy();
|
||||
return;
|
||||
}
|
||||
res.writeHead(upstream.status, { 'content-type': 'application/json' });
|
||||
res.end(response);
|
||||
} catch (cause) {
|
||||
proxyFailure = cause;
|
||||
res.destroy();
|
||||
}
|
||||
});
|
||||
const model = createServer(async (req, res) => {
|
||||
let stage = 'request';
|
||||
try {
|
||||
assert.equal(req.url, '/v1/chat/completions');
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of req) chunks.push(Buffer.from(chunk));
|
||||
const request = JSON.parse(Buffer.concat(chunks).toString()) as {
|
||||
model: string;
|
||||
messages: Array<{
|
||||
role: string;
|
||||
tool_call_id?: string;
|
||||
content?:
|
||||
| string
|
||||
| Array<{
|
||||
type: string;
|
||||
image_url?: { url: string };
|
||||
file?: { file_data: string };
|
||||
}>;
|
||||
}>;
|
||||
};
|
||||
assert.equal(request.model, 'provider-media-fixture');
|
||||
stage = 'tool-call-correlation';
|
||||
const tools = request.messages.filter((message) => message.role === 'tool');
|
||||
assert.equal(tools.length, 1);
|
||||
assert.equal(
|
||||
tools[0].tool_call_id,
|
||||
expectedCallId,
|
||||
'model HTTP original call correlation',
|
||||
);
|
||||
const mediaOf = (
|
||||
message: (typeof request.messages)[number] | undefined,
|
||||
) => {
|
||||
const parts =
|
||||
message && Array.isArray(message.content) ? message.content : [];
|
||||
return parts.flatMap((part) => {
|
||||
const url = part.image_url?.url ?? part.file?.file_data;
|
||||
return url ? [{ type: part.type, url }] : [];
|
||||
});
|
||||
};
|
||||
stage = 'media-role';
|
||||
const toolMedia = mediaOf(tools[0]);
|
||||
const following = request.messages[request.messages.indexOf(tools[0]) + 1];
|
||||
const userMedia = following?.role === 'user' ? mediaOf(following) : [];
|
||||
if (expectedSplitToolMedia) {
|
||||
assert.equal(
|
||||
toolMedia.length,
|
||||
0,
|
||||
'default tool reply must contain only text',
|
||||
);
|
||||
if (expectedMedia.length)
|
||||
assert.equal(
|
||||
following?.role,
|
||||
'user',
|
||||
'default media follows the original tool reply',
|
||||
);
|
||||
} else
|
||||
assert.equal(
|
||||
userMedia.length,
|
||||
0,
|
||||
'compatibility media stays inside the tool reply',
|
||||
);
|
||||
const media = expectedSplitToolMedia ? userMedia : toolMedia;
|
||||
stage = 'media-count';
|
||||
assert.equal(
|
||||
request.messages.flatMap(mediaOf).length,
|
||||
expectedMedia.length,
|
||||
'no duplicated or misplaced HTTP media',
|
||||
);
|
||||
assert.equal(media.length, expectedMedia.length, 'model HTTP media count');
|
||||
for (const [index, part] of media.entries()) {
|
||||
stage = 'data-uri';
|
||||
const match = /^data:([^;]+);base64,([A-Za-z0-9+/=]+)$/.exec(part.url);
|
||||
assert(match, 'model media must be an inline data URI');
|
||||
const expected = expectedMedia[index];
|
||||
stage = 'mime';
|
||||
assert.equal(match[1], expected.mime);
|
||||
assert.equal(
|
||||
part.type,
|
||||
expected.mime === 'application/pdf' ? 'file' : 'image_url',
|
||||
);
|
||||
const bytes = Buffer.from(match[2], 'base64');
|
||||
stage = 'decoded-length';
|
||||
assert.equal(bytes.length, expected.bytes, 'model HTTP decoded length');
|
||||
stage = 'decoded-sha256';
|
||||
assert.equal(digest(bytes), expected.hash, 'model HTTP decoded bytes');
|
||||
if (expected.width !== undefined) {
|
||||
stage = 'image-dimensions';
|
||||
const metadata = await sharp(bytes).metadata();
|
||||
assert.equal(metadata.width, expected.width);
|
||||
assert.equal(metadata.height, expected.height);
|
||||
assert.equal(`image/${metadata.format}`, expected.mime);
|
||||
}
|
||||
}
|
||||
modelRequests++;
|
||||
res.writeHead(200, { 'content-type': 'application/json' });
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
id: 'media-proof',
|
||||
object: 'chat.completion',
|
||||
created: 1,
|
||||
model: request.model,
|
||||
choices: [
|
||||
{
|
||||
index: 0,
|
||||
message: { role: 'assistant', content: 'MEDIA_ACCEPTED' },
|
||||
finish_reason: 'stop',
|
||||
},
|
||||
],
|
||||
}),
|
||||
);
|
||||
} catch {
|
||||
// Assertions must never expose inline bytes in Maven's captured output.
|
||||
proxyFailure = new Error(
|
||||
`Deterministic model rejected the outgoing media request at ${stage}.`,
|
||||
);
|
||||
res
|
||||
.writeHead(500)
|
||||
.end(JSON.stringify({ error: `media assertion failed at ${stage}` }));
|
||||
}
|
||||
});
|
||||
async function listen(server: ReturnType<typeof createServer>) {
|
||||
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
|
||||
const address = server.address();
|
||||
assert(address && typeof address !== 'string');
|
||||
return `http://127.0.0.1:${address.port}`;
|
||||
}
|
||||
const provider = new BrokerManagedRuntimeProvider({
|
||||
baseUrl: await listen(proxy),
|
||||
token: 'hosted-tools-broker-token',
|
||||
});
|
||||
const modelClient = new OpenAI({
|
||||
apiKey: 'local-fixture-only',
|
||||
baseURL: (await listen(model)) + '/v1',
|
||||
maxRetries: 0,
|
||||
});
|
||||
async function evidence(sessionId: string, phase: string, fields: object = {}) {
|
||||
if (proxyFailure) throw proxyFailure;
|
||||
const response = await fetch(
|
||||
new URL(`/media/${sessionId}/${phase}`, config.statusGateUrl),
|
||||
{
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify(fields),
|
||||
signal: AbortSignal.timeout(45_000),
|
||||
},
|
||||
);
|
||||
assert.equal(response.status, 200, `media probe ${phase}`);
|
||||
return (await response.json()) as { brokerUrl?: string; done?: boolean };
|
||||
}
|
||||
const reports: Array<Record<string, unknown>> = [];
|
||||
const harnessDirectory = await mkdtemp(
|
||||
path.join(tmpdir(), PROVIDER_MEDIA_HARNESS_PREFIX),
|
||||
);
|
||||
const originalCwd = process.cwd();
|
||||
process.chdir(harnessDirectory);
|
||||
try {
|
||||
for (const session of config.sessions) {
|
||||
const name = session.fault;
|
||||
const fixture = await mediaFixture(name);
|
||||
const replacement = await mediaFixture(name, true);
|
||||
const file = path.join(session.directory, fixture.filename);
|
||||
await writeFile(file, fixture.bytes);
|
||||
await writeFile(file + '.replacement', replacement.bytes);
|
||||
const decoy = path.join(harnessDirectory, fixture.filename);
|
||||
await writeFile(decoy, replacement.bytes);
|
||||
assert.notEqual(
|
||||
digest(fixture.bytes),
|
||||
digest(replacement.bytes),
|
||||
'Harness decoy differs from Workspace fixture',
|
||||
);
|
||||
const request = {
|
||||
protocolVersion: 1 as const,
|
||||
tenantId: config.tenantId,
|
||||
workspaceId: session.workspaceId,
|
||||
workspaceCwd: session.directory,
|
||||
sessionId: randomUUID(),
|
||||
turnKind: 'bootstrap' as const,
|
||||
};
|
||||
const client = await provider.getToolV2Client(request, {
|
||||
harnessSessionId: session.sessionId,
|
||||
});
|
||||
const manifest = await client.manifest();
|
||||
const identity = {
|
||||
sessionId: request.sessionId,
|
||||
promptId: randomUUID(),
|
||||
callId: randomUUID(),
|
||||
capabilityDigest: manifest.capabilityDigest,
|
||||
policyRevision: manifest.policyRevision,
|
||||
};
|
||||
await client.fileHistory!.bind({
|
||||
ownerSessionId: session.sessionId,
|
||||
ownerRuntimeSessionId: request.sessionId,
|
||||
executionCwd: session.directory,
|
||||
snapshots: [],
|
||||
});
|
||||
await client.beginTurn(identity);
|
||||
const input = {
|
||||
file_path: file,
|
||||
...(name === 'pdf-render' || name === 'pdf-text' ? { pages: '1' } : {}),
|
||||
};
|
||||
const context: ManagedToolMediaContext | undefined =
|
||||
name === 'image-missing'
|
||||
? undefined
|
||||
: {
|
||||
inputModalities: {
|
||||
image: name !== 'image-disabled',
|
||||
pdf: name !== 'pdf-render' && name !== 'pdf-text',
|
||||
},
|
||||
};
|
||||
// Concurrent preparations with opposite capabilities must keep separate snapshots.
|
||||
const [prepared] = await Promise.all([
|
||||
client.prepare(identity, 'read_file', input, undefined, context),
|
||||
client.prepare(
|
||||
{ ...identity, callId: randomUUID() },
|
||||
'read_file',
|
||||
input,
|
||||
undefined,
|
||||
{
|
||||
inputModalities: {
|
||||
image: name === 'image-disabled' || name === 'image-missing',
|
||||
pdf: false,
|
||||
},
|
||||
},
|
||||
),
|
||||
]);
|
||||
const reference = parseManagedToolInvocationReference({
|
||||
...identity,
|
||||
invocationId: prepared.invocationId,
|
||||
argsDigest: prepared.argsDigest,
|
||||
});
|
||||
assert.equal(Object.keys(reference).length, 7);
|
||||
expectedCallId = reference.callId;
|
||||
assert.equal('responseParts' in prepared, false);
|
||||
await assert.rejects(() =>
|
||||
client.prepare(identity, 'read_file', input, undefined, {
|
||||
inputModalities: {
|
||||
image: context?.inputModalities.image !== true,
|
||||
pdf: false,
|
||||
},
|
||||
}),
|
||||
);
|
||||
const confirmation = await client.confirmation(reference);
|
||||
if (confirmation || prepared.defaultPermission === 'ask')
|
||||
await client.confirm(reference, ToolConfirmationOutcome.ProceedOnce);
|
||||
await client.preflight(reference);
|
||||
const reservation = await client.prepareExecution!(reference);
|
||||
await evidence(session.sessionId, 'prepared', {
|
||||
reference,
|
||||
executionCallId: reservation.executionCallId,
|
||||
});
|
||||
dropStart = name === 'lost-start';
|
||||
let execution: ManagedToolExecutionResult | undefined;
|
||||
if (dropStart) {
|
||||
await assert.rejects(() => client.execute(reference));
|
||||
await waitUntil(
|
||||
async () =>
|
||||
(await evidence(session.sessionId, 'worker-done')).done === true,
|
||||
);
|
||||
await writeFile(file, replacement.bytes);
|
||||
} else {
|
||||
try {
|
||||
execution = await client.execute(reference);
|
||||
} catch (cause) {
|
||||
if (name !== 'lost-execute') throw cause;
|
||||
}
|
||||
if (name === 'lost-execute') {
|
||||
await waitUntil(
|
||||
async () =>
|
||||
(await evidence(session.sessionId, 'dropped')).done === true,
|
||||
);
|
||||
}
|
||||
}
|
||||
const beforeRecoveryCounts =
|
||||
name === 'lost-start' || name === 'lost-execute'
|
||||
? await readFile(
|
||||
path.join(session.directory, '.provider-media-reads.json'),
|
||||
'utf8',
|
||||
)
|
||||
: undefined;
|
||||
let status: ManagedToolInvocationStatus | undefined;
|
||||
await waitUntil(async () => {
|
||||
try {
|
||||
status = await client.status(reference);
|
||||
return status !== undefined && status.state === 'settled';
|
||||
} catch (cause) {
|
||||
if (name !== 'lost-execute') throw cause;
|
||||
return false;
|
||||
}
|
||||
});
|
||||
execution ??= status!.result!;
|
||||
const resultHash = digest(JSON.stringify(execution));
|
||||
const resultBytes = Buffer.byteLength(JSON.stringify(execution));
|
||||
const assertResult = (observed: unknown, label: string) => {
|
||||
const encoded = JSON.stringify(observed);
|
||||
assert.equal(Buffer.byteLength(encoded), resultBytes, `${label} length`);
|
||||
assert.equal(digest(encoded), resultHash, `${label} SHA256`);
|
||||
};
|
||||
assertResult(status!.result, 'execute and status retained result');
|
||||
await evidence(session.sessionId, 'settled', { resultHash, resultBytes });
|
||||
const readCounts = await readFile(
|
||||
path.join(session.directory, '.provider-media-reads.json'),
|
||||
'utf8',
|
||||
);
|
||||
if (beforeRecoveryCounts !== undefined)
|
||||
assert.equal(
|
||||
readCounts,
|
||||
beforeRecoveryCounts,
|
||||
'first lost-reply observation added no worker reads',
|
||||
);
|
||||
if (name === 'lost-start' || name === 'lost-execute') {
|
||||
await writeFile(file, replacement.bytes);
|
||||
assert.notEqual(
|
||||
digest(await readFile(file)),
|
||||
digest(fixture.bytes),
|
||||
'recovery fixture actually changed',
|
||||
);
|
||||
}
|
||||
assertResult(
|
||||
(await client.status(reference)).result,
|
||||
'original-reference recovery',
|
||||
);
|
||||
assertResult(
|
||||
(await client.cancel(reference)).result,
|
||||
'settled cancellation observation',
|
||||
);
|
||||
assert.equal(
|
||||
await readFile(
|
||||
path.join(session.directory, '.provider-media-reads.json'),
|
||||
'utf8',
|
||||
),
|
||||
readCounts,
|
||||
'observations added no worker file reads',
|
||||
);
|
||||
assert(
|
||||
execution.result && !('responseParts' in execution.result),
|
||||
'provider result must be nested ToolResult',
|
||||
);
|
||||
const content = execution.result.llmContent;
|
||||
const responseParts = convertToFunctionResponse(
|
||||
'read_file',
|
||||
reference.callId,
|
||||
content,
|
||||
);
|
||||
const media = responseParts
|
||||
.flatMap((part) => part.functionResponse?.parts ?? [])
|
||||
.flatMap((part) => (part.inlineData ? [part.inlineData] : []));
|
||||
const refused = name === 'pdf-too-large';
|
||||
const disabled = name === 'image-disabled' || name === 'image-missing';
|
||||
assert.equal(execution.executionStatus, refused ? 'error' : 'success');
|
||||
if (refused) assert.equal(execution.result.error?.type, 'file_too_large');
|
||||
assert.equal(
|
||||
media.length,
|
||||
refused || disabled || name === 'pdf-text' ? 0 : 1,
|
||||
`${name} inline media`,
|
||||
);
|
||||
if (name === 'pdf-text')
|
||||
assert(JSON.stringify(content).includes('provider media text proof'));
|
||||
expectedMedia = [];
|
||||
for (const part of media) {
|
||||
assert(part.data && part.mimeType, 'complete media part');
|
||||
const bytes = Buffer.from(part.data, 'base64');
|
||||
if (fixture.mime === 'application/pdf' && name !== 'pdf-render') {
|
||||
assert.equal(part.mimeType, 'application/pdf');
|
||||
assert.equal(
|
||||
bytes.length,
|
||||
fixture.bytes.length,
|
||||
'native PDF byte length',
|
||||
);
|
||||
assert.equal(digest(bytes), digest(fixture.bytes), 'native PDF bytes');
|
||||
expectedMedia.push({
|
||||
mime: part.mimeType,
|
||||
hash: digest(fixture.bytes),
|
||||
bytes: fixture.bytes.length,
|
||||
});
|
||||
} else {
|
||||
const metadata = await sharp(bytes).metadata();
|
||||
assert.equal(part.mimeType, `image/${metadata.format}`);
|
||||
if (name === 'pdf-render')
|
||||
assert(
|
||||
metadata.width! > 8 && metadata.height! > 6,
|
||||
'Poppler rendered a real page',
|
||||
);
|
||||
else {
|
||||
assert.equal(metadata.width, 8);
|
||||
assert.equal(metadata.height, 6);
|
||||
if (name === 'gif') {
|
||||
assert.equal(part.mimeType, 'image/gif', 'native GIF MIME');
|
||||
assert.equal(
|
||||
bytes.length,
|
||||
fixture.bytes.length,
|
||||
'native GIF length',
|
||||
);
|
||||
assert.equal(
|
||||
digest(bytes),
|
||||
digest(fixture.bytes),
|
||||
'native GIF SHA256',
|
||||
);
|
||||
} else
|
||||
assert.equal(
|
||||
part.mimeType,
|
||||
'image/jpeg',
|
||||
'safe image overview MIME',
|
||||
);
|
||||
const statistics = await sharp(bytes).stats();
|
||||
assert(
|
||||
statistics.channels[0].mean < 60 &&
|
||||
statistics.channels[2].mean > 210,
|
||||
'Workspace image pixels, not the red Harness decoy',
|
||||
);
|
||||
}
|
||||
expectedMedia.push({
|
||||
mime: part.mimeType,
|
||||
hash: digest(bytes),
|
||||
bytes: bytes.length,
|
||||
width: metadata.width,
|
||||
height: metadata.height,
|
||||
});
|
||||
}
|
||||
}
|
||||
const modelRequest: GenerateContentParameters = {
|
||||
model: 'provider-media-fixture',
|
||||
contents: [
|
||||
{
|
||||
role: 'model',
|
||||
parts: [
|
||||
{
|
||||
functionCall: {
|
||||
name: 'read_file',
|
||||
id: reference.callId,
|
||||
args: input,
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{ role: 'user', parts: responseParts as Part[] },
|
||||
],
|
||||
};
|
||||
for (const splitToolMedia of [true, false]) {
|
||||
expectedSplitToolMedia = splitToolMedia;
|
||||
const messages = convertLlmRequestToOpenAI(modelRequest, {
|
||||
model: modelRequest.model,
|
||||
modalities: { image: true, pdf: true },
|
||||
startTime: 0,
|
||||
splitToolMedia,
|
||||
});
|
||||
const completion = await modelClient.chat.completions.create({
|
||||
model: modelRequest.model,
|
||||
messages,
|
||||
});
|
||||
assert.equal(completion.choices[0].message.content, 'MEDIA_ACCEPTED');
|
||||
}
|
||||
if (name === 'native-pdf') {
|
||||
const responses = JSON.stringify(
|
||||
convertGeminiContentsToResponsesInput(modelRequest),
|
||||
);
|
||||
assert(
|
||||
responses.includes('application/pdf') &&
|
||||
responses.includes('Unsupported'),
|
||||
'Responses native PDF limitation',
|
||||
);
|
||||
assert(
|
||||
!responses.includes(fixture.bytes.toString('base64')),
|
||||
'Responses must not claim native PDF delivery',
|
||||
);
|
||||
}
|
||||
assert.equal(
|
||||
await provider.release(request.sessionId, request, { terminal: true }),
|
||||
true,
|
||||
);
|
||||
assert.equal(
|
||||
await provider.release(request.sessionId, request, { terminal: true }),
|
||||
true,
|
||||
);
|
||||
await evidence(session.sessionId, 'released', { resultHash, resultBytes });
|
||||
assert.equal(
|
||||
digest(await readFile(decoy)),
|
||||
digest(replacement.bytes),
|
||||
'Harness decoy untouched',
|
||||
);
|
||||
reports.push({
|
||||
...session,
|
||||
runtimeSessionId: request.sessionId,
|
||||
reference,
|
||||
executionCallId: reservation.executionCallId,
|
||||
resultHash,
|
||||
resultBytes,
|
||||
mediaCount: media.length,
|
||||
executionStatus: execution.executionStatus,
|
||||
});
|
||||
console.log(
|
||||
`M4_PROVIDER_MEDIA ${name}: media=${media.length} dispatch=1 released=true`,
|
||||
);
|
||||
}
|
||||
const restart = await evidence(config.sessions[0].sessionId, 'restart');
|
||||
assert(
|
||||
restart.brokerUrl && restart.brokerUrl !== brokerUrl,
|
||||
'fresh Embedded Broker endpoint',
|
||||
);
|
||||
brokerUrl = restart.brokerUrl;
|
||||
for (const report of reports) {
|
||||
const inspection = await provider.inspectExecution({
|
||||
harnessSessionId: report['sessionId'] as string,
|
||||
runtimeSessionId: report['runtimeSessionId'] as string,
|
||||
executionCallId: report['executionCallId'] as string,
|
||||
});
|
||||
assert.equal(inspection.outcome, 'known');
|
||||
assert(inspection.outcome === 'known');
|
||||
assert.equal(inspection.status.state, 'settled');
|
||||
assert.equal(
|
||||
Buffer.byteLength(JSON.stringify(inspection.status.result)),
|
||||
report['resultBytes'],
|
||||
'MySQL recovery length after Broker restart',
|
||||
);
|
||||
assert.equal(
|
||||
digest(JSON.stringify(inspection.status.result)),
|
||||
report['resultHash'],
|
||||
'MySQL recovery after Broker restart',
|
||||
);
|
||||
await evidence(report['sessionId'] as string, 'restarted', {
|
||||
resultHash: report['resultHash'],
|
||||
resultBytes: report['resultBytes'],
|
||||
});
|
||||
}
|
||||
assert.equal(startDrops, 1);
|
||||
assert.equal(modelRequests, reports.length * 2);
|
||||
if (proxyFailure) throw proxyFailure;
|
||||
await writeFile(configPath + '.results', JSON.stringify(reports));
|
||||
console.log(
|
||||
`HOSTED_PROVIDER_MEDIA_OK cases=${reports.length} modelRequests=${modelRequests} startDrops=${startDrops}`,
|
||||
);
|
||||
} finally {
|
||||
process.chdir(originalCwd);
|
||||
await rm(harnessDirectory, { recursive: true, force: true });
|
||||
provider.dispose();
|
||||
for (const server of [proxy, model]) {
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>((resolve) => server.close(() => resolve()));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFile, execFileSync, spawn } from 'node:child_process';
|
||||
import { once } from 'node:events';
|
||||
import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { test } from 'node:test';
|
||||
import { promisify } from 'node:util';
|
||||
import sharp from 'sharp';
|
||||
import { mediaFixture, pdfFixture } from './provider-media-fixtures.js';
|
||||
import './provider-media-read-probe.mjs';
|
||||
import {
|
||||
PROVIDER_MEDIA_FIXTURES_PREFIX,
|
||||
PROVIDER_MEDIA_PROBE_PREFIX,
|
||||
stopFixtureProcess,
|
||||
} from './provider-media-process-cleanup.mjs';
|
||||
|
||||
for (const alreadyExited of [false, true]) {
|
||||
test(
|
||||
`fixture cleanup handles ${alreadyExited ? 'an already signaled' : 'a running'} child`,
|
||||
{ timeout: 3000 },
|
||||
async () => {
|
||||
for (const graceful of [false, true]) {
|
||||
const child = spawn(
|
||||
process.execPath,
|
||||
['-e', 'setInterval(() => {}, 1000)'],
|
||||
{ stdio: 'ignore' },
|
||||
);
|
||||
try {
|
||||
await once(child, 'spawn');
|
||||
if (alreadyExited) {
|
||||
const exited = once(child, 'exit');
|
||||
child.kill('SIGKILL');
|
||||
await exited;
|
||||
assert.equal(child.exitCode, null);
|
||||
assert.equal(child.signalCode, 'SIGKILL');
|
||||
}
|
||||
let shutdowns = 0;
|
||||
await stopFixtureProcess(
|
||||
child,
|
||||
graceful
|
||||
? () => {
|
||||
shutdowns++;
|
||||
child.kill('SIGTERM');
|
||||
}
|
||||
: undefined,
|
||||
);
|
||||
assert.equal(shutdowns, graceful && !alreadyExited ? 1 : 0);
|
||||
assert.equal(child.exitCode, null);
|
||||
assert.equal(
|
||||
child.signalCode,
|
||||
graceful && !alreadyExited ? 'SIGTERM' : 'SIGKILL',
|
||||
);
|
||||
} finally {
|
||||
await stopFixtureProcess(child);
|
||||
}
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
test(
|
||||
'fixture cleanup kills a child that accepts shutdown but never exits',
|
||||
{ timeout: 3000 },
|
||||
async () => {
|
||||
const child = spawn(
|
||||
process.execPath,
|
||||
['-e', 'setInterval(() => {}, 1000)'],
|
||||
{ stdio: 'ignore' },
|
||||
);
|
||||
await once(child, 'spawn');
|
||||
try {
|
||||
await stopFixtureProcess(child, () => {}, 50);
|
||||
assert.equal(child.signalCode, 'SIGKILL');
|
||||
} finally {
|
||||
await stopFixtureProcess(child);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
test('worker read probe appends evidence across process generations', async () => {
|
||||
const temporary = await mkdtemp(
|
||||
path.join(tmpdir(), PROVIDER_MEDIA_PROBE_PREFIX),
|
||||
);
|
||||
try {
|
||||
const file = path.join(temporary, 'proof.pdf');
|
||||
await writeFile(file, 'proof');
|
||||
const probe = new URL('./provider-media-read-probe.mjs', import.meta.url)
|
||||
.href;
|
||||
const code = 'require("node:fs").readFileSync(process.argv[1])';
|
||||
execFileSync(process.execPath, ['--import', probe, '-e', code, file]);
|
||||
const first = await readFile(
|
||||
path.join(temporary, '.provider-media-reads.json'),
|
||||
'utf8',
|
||||
);
|
||||
execFileSync(process.execPath, ['--import', probe, '-e', code, file]);
|
||||
const second = await readFile(
|
||||
path.join(temporary, '.provider-media-reads.json'),
|
||||
'utf8',
|
||||
);
|
||||
assert(second.startsWith(first));
|
||||
assert(second.length > first.length);
|
||||
} finally {
|
||||
await rm(temporary, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('worker read probe preserves execFile promisify stdout and stderr', async () => {
|
||||
const result = await promisify(execFile)(process.execPath, [
|
||||
'-e',
|
||||
"process.stdout.write('out'); process.stderr.write('err');",
|
||||
]);
|
||||
assert.equal(result.stdout, 'out');
|
||||
assert.equal(result.stderr, 'err');
|
||||
});
|
||||
|
||||
test('real image fixtures decode with the expected MIME and dimensions', async () => {
|
||||
for (const name of ['png', 'jpeg', 'webp', 'gif'] as const) {
|
||||
const fixture = await mediaFixture(name);
|
||||
const metadata = await sharp(fixture.bytes).metadata();
|
||||
assert.equal(`image/${metadata.format}`, fixture.mime);
|
||||
assert.equal(metadata.width, 8);
|
||||
assert.equal(metadata.height, 6);
|
||||
assert(!(await mediaFixture(name, true)).bytes.equals(fixture.bytes));
|
||||
}
|
||||
});
|
||||
|
||||
test('real PDF fixtures distinguish text extraction, Poppler rendering and size refusal', async () => {
|
||||
const temporary = await mkdtemp(
|
||||
path.join(tmpdir(), PROVIDER_MEDIA_FIXTURES_PREFIX),
|
||||
);
|
||||
try {
|
||||
const text = path.join(temporary, 'text.pdf');
|
||||
const scan = path.join(temporary, 'scan.pdf');
|
||||
await writeFile(text, pdfFixture(true));
|
||||
await writeFile(scan, pdfFixture(false));
|
||||
assert(
|
||||
execFileSync('pdftotext', [text, '-'], { encoding: 'utf8' }).includes(
|
||||
'provider media text proof',
|
||||
),
|
||||
);
|
||||
assert.equal(
|
||||
execFileSync('pdftotext', [scan, '-'], { encoding: 'utf8' }).trim(),
|
||||
'',
|
||||
);
|
||||
execFileSync(
|
||||
'pdftoppm',
|
||||
['-f', '1', '-singlefile', '-png', scan, path.join(temporary, 'page')],
|
||||
{ stdio: 'pipe' },
|
||||
);
|
||||
const metadata = await sharp(
|
||||
await readFile(path.join(temporary, 'page.png')),
|
||||
).metadata();
|
||||
assert.equal(metadata.format, 'png');
|
||||
assert(metadata.width! > 8 && metadata.height! > 6);
|
||||
const oversized = await mediaFixture('pdf-too-large');
|
||||
assert(oversized.bytes.length > 576 * 1024);
|
||||
const large = path.join(temporary, 'large.pdf');
|
||||
await writeFile(large, oversized.bytes);
|
||||
assert(
|
||||
execFileSync('pdfinfo', [large], { encoding: 'utf8' }).includes('Pages:'),
|
||||
);
|
||||
const native = await mediaFixture('native-pdf');
|
||||
assert(
|
||||
native.bytes.length > 560 * 1024 && native.bytes.length < 576 * 1024,
|
||||
);
|
||||
await writeFile(large, native.bytes);
|
||||
assert(
|
||||
execFileSync('pdfinfo', [large], { encoding: 'utf8' }).includes('Pages:'),
|
||||
);
|
||||
} finally {
|
||||
await rm(temporary, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,111 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { deflateSync } from 'node:zlib';
|
||||
import sharp from 'sharp';
|
||||
|
||||
export const MEDIA_CASES = [
|
||||
'png',
|
||||
'jpeg',
|
||||
'webp',
|
||||
'gif',
|
||||
'native-pdf',
|
||||
'pdf-text',
|
||||
'pdf-render',
|
||||
'image-disabled',
|
||||
'image-missing',
|
||||
'pdf-too-large',
|
||||
'lost-start',
|
||||
'lost-execute',
|
||||
] as const;
|
||||
|
||||
export type MediaCase = (typeof MEDIA_CASES)[number];
|
||||
|
||||
export function pdfFixture(text: boolean, padding = 0): Buffer {
|
||||
const stream = text
|
||||
? Buffer.from('BT /F1 4 Tf 5 20 Td (provider media text proof) Tj ET')
|
||||
: Buffer.from('q 72 0 0 54 0 0 cm /Im1 Do Q');
|
||||
const image = deflateSync(Buffer.alloc(8 * 6 * 3, 93));
|
||||
const objects = [
|
||||
Buffer.from('<< /Type /Catalog /Pages 2 0 R >>'),
|
||||
Buffer.from('<< /Type /Pages /Kids [3 0 R] /Count 1 >>'),
|
||||
Buffer.from(
|
||||
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 72 54] ' +
|
||||
'/Resources << /Font << /F1 5 0 R >> /XObject << /Im1 6 0 R >> >> ' +
|
||||
'/Contents 4 0 R >>',
|
||||
),
|
||||
Buffer.concat([
|
||||
Buffer.from(`<< /Length ${stream.length} >>\nstream\n`),
|
||||
stream,
|
||||
Buffer.from('\nendstream'),
|
||||
]),
|
||||
Buffer.from('<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>'),
|
||||
Buffer.concat([
|
||||
Buffer.from(
|
||||
'<< /Type /XObject /Subtype /Image /Width 8 /Height 6 ' +
|
||||
'/ColorSpace /DeviceRGB /BitsPerComponent 8 /Filter /FlateDecode ' +
|
||||
`/Length ${image.length} >>\nstream\n`,
|
||||
),
|
||||
image,
|
||||
Buffer.from('\nendstream'),
|
||||
]),
|
||||
];
|
||||
const chunks = [Buffer.from('%PDF-1.4\n')];
|
||||
const offsets = [0];
|
||||
let size = chunks[0].length;
|
||||
for (const [index, object] of objects.entries()) {
|
||||
offsets.push(size);
|
||||
const chunk = Buffer.concat([
|
||||
Buffer.from(`${index + 1} 0 obj\n`),
|
||||
object,
|
||||
Buffer.from('\nendobj\n'),
|
||||
]);
|
||||
chunks.push(chunk);
|
||||
size += chunk.length;
|
||||
}
|
||||
chunks.push(
|
||||
Buffer.from(
|
||||
`xref\n0 ${objects.length + 1}\n0000000000 65535 f \n` +
|
||||
offsets
|
||||
.slice(1)
|
||||
.map((offset) => `${String(offset).padStart(10, '0')} 00000 n \n`)
|
||||
.join('') +
|
||||
`trailer\n<< /Size ${objects.length + 1} /Root 1 0 R >>\n` +
|
||||
`startxref\n${size}\n%%EOF\n` +
|
||||
(padding ? '%' + 'p'.repeat(padding) + '\n' : ''),
|
||||
),
|
||||
);
|
||||
return Buffer.concat(chunks);
|
||||
}
|
||||
|
||||
export async function mediaFixture(name: MediaCase, replacement = false) {
|
||||
if (name.includes('pdf') || name === 'lost-execute') {
|
||||
return {
|
||||
filename: 'proof.pdf',
|
||||
mime: 'application/pdf',
|
||||
bytes: pdfFixture(
|
||||
name === 'pdf-text',
|
||||
(name === 'pdf-too-large'
|
||||
? 600 * 1024
|
||||
: name === 'native-pdf' || name === 'lost-execute'
|
||||
? 560 * 1024
|
||||
: 0) + (replacement ? 17 : 0),
|
||||
),
|
||||
};
|
||||
}
|
||||
const format = ['jpeg', 'webp', 'gif'].includes(name) ? name : 'png';
|
||||
const bytes = await sharp({
|
||||
create: {
|
||||
width: 8,
|
||||
height: 6,
|
||||
channels: 3,
|
||||
background: replacement ? '#ed2161' : '#2161ed',
|
||||
},
|
||||
})
|
||||
.toFormat(format as 'png' | 'jpeg' | 'webp' | 'gif')
|
||||
.toBuffer();
|
||||
return { filename: `proof.${format}`, mime: `image/${format}`, bytes };
|
||||
}
|
||||
@@ -0,0 +1,194 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import assert from 'node:assert/strict';
|
||||
import console from 'node:console';
|
||||
import { spawn, execFileSync } from 'node:child_process';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { once } from 'node:events';
|
||||
import { chmod, mkdir, mkdtemp, readFile, rm } from 'node:fs/promises';
|
||||
import { createServer } from 'node:net';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import process from 'node:process';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { setTimeout as delay } from 'node:timers/promises';
|
||||
import {
|
||||
PROVIDER_MEDIA_MYSQL_PREFIX,
|
||||
stopFixtureProcess,
|
||||
} from './provider-media-process-cleanup.mjs';
|
||||
|
||||
const root = path.resolve(
|
||||
path.dirname(fileURLToPath(import.meta.url)),
|
||||
'../..',
|
||||
);
|
||||
const maven = process.env['MAVEN_EXECUTABLE'] ?? 'mvn';
|
||||
const settings = process.env['MAVEN_SETTINGS'];
|
||||
const prepareOnly = process.argv[2] === '--prepare-only';
|
||||
if (!prepareOnly) {
|
||||
assert.equal(
|
||||
process.argv[2],
|
||||
'--bundle-ready',
|
||||
'Requires the parent bundle-ready confirmation and SHA-256',
|
||||
);
|
||||
const hash = createHash('sha256')
|
||||
.update(await readFile(path.join(root, 'dist/cli.js')))
|
||||
.digest('hex');
|
||||
assert.equal(process.argv[3], hash, 'The confirmed packaged CLI has changed');
|
||||
console.log(`M4_BUNDLE_SHA256 ${hash}`);
|
||||
}
|
||||
const temporary = await mkdtemp(
|
||||
path.join(tmpdir(), PROVIDER_MEDIA_MYSQL_PREFIX),
|
||||
);
|
||||
const datadir = path.join(temporary, 'data');
|
||||
const socket = path.join(temporary, 'mysql.sock');
|
||||
let mysql;
|
||||
let mysqlFailure;
|
||||
let verification;
|
||||
try {
|
||||
await chmod(temporary, 0o700);
|
||||
await mkdir(datadir);
|
||||
const listener = createServer();
|
||||
let port;
|
||||
try {
|
||||
await new Promise((resolve, reject) =>
|
||||
listener.once('error', reject).listen(0, '127.0.0.1', resolve),
|
||||
);
|
||||
port = listener.address().port;
|
||||
} finally {
|
||||
await new Promise((resolve) => listener.close(resolve));
|
||||
}
|
||||
execFileSync(
|
||||
'mysqld',
|
||||
['--no-defaults', '--initialize-insecure', `--datadir=${datadir}`],
|
||||
{ stdio: 'pipe', timeout: 30_000, killSignal: 'SIGKILL' },
|
||||
);
|
||||
mysql = spawn(
|
||||
'mysqld',
|
||||
[
|
||||
'--no-defaults',
|
||||
`--datadir=${datadir}`,
|
||||
`--socket=${socket}`,
|
||||
'--bind-address=127.0.0.1',
|
||||
`--port=${port}`,
|
||||
'--mysqlx=OFF',
|
||||
`--pid-file=${temporary}/mysql.pid`,
|
||||
`--log-error=${temporary}/mysql.log`,
|
||||
],
|
||||
{ stdio: 'ignore' },
|
||||
);
|
||||
mysql.on('error', (cause) => {
|
||||
mysqlFailure = cause;
|
||||
});
|
||||
const startupDeadline = Date.now() + 30_000;
|
||||
for (;;) {
|
||||
try {
|
||||
execFileSync(
|
||||
'mysqladmin',
|
||||
['--no-defaults', `--socket=${socket}`, '-u', 'root', 'ping'],
|
||||
{ stdio: 'pipe', timeout: 1000, killSignal: 'SIGKILL' },
|
||||
);
|
||||
break;
|
||||
} catch {
|
||||
assert(
|
||||
Date.now() < startupDeadline &&
|
||||
mysql.exitCode === null &&
|
||||
mysql.signalCode === null &&
|
||||
mysqlFailure === undefined,
|
||||
'Isolated MySQL failed to start',
|
||||
);
|
||||
await delay(100);
|
||||
}
|
||||
}
|
||||
execFileSync(
|
||||
'mysql',
|
||||
[
|
||||
'--no-defaults',
|
||||
`--socket=${socket}`,
|
||||
'-u',
|
||||
'root',
|
||||
'-e',
|
||||
'CREATE DATABASE provider_media_m4',
|
||||
],
|
||||
{ stdio: 'pipe', timeout: 30_000, killSignal: 'SIGKILL' },
|
||||
);
|
||||
console.log(
|
||||
'M4_ISOLATED_MYSQL ' +
|
||||
execFileSync(
|
||||
'mysql',
|
||||
[
|
||||
'--no-defaults',
|
||||
`--socket=${socket}`,
|
||||
'-u',
|
||||
'root',
|
||||
'-N',
|
||||
'-e',
|
||||
'SELECT VERSION()',
|
||||
],
|
||||
{ encoding: 'utf8', timeout: 5000, killSignal: 'SIGKILL' },
|
||||
).trim(),
|
||||
);
|
||||
if (!prepareOnly) {
|
||||
verification = spawn(
|
||||
maven,
|
||||
[
|
||||
'-q',
|
||||
...(settings ? ['-s', settings] : []),
|
||||
'-f',
|
||||
'packages/managed-agent-server/pom.xml',
|
||||
'-Phosted-workspace-tools',
|
||||
'-Dit.test=HostedWorkspaceToolTurnIT#providerMediaReachesModelAndSurvivesReplyLossAndBrokerRestartOnMySql',
|
||||
'-Dqwen.provider.media.bundle-ready=true',
|
||||
`-Dnode.executable=${process.execPath}`,
|
||||
`-Dqwen.cli.entry=${root}/dist/cli.js`,
|
||||
`-Dmysql.url=jdbc:mysql://127.0.0.1:${port}/provider_media_m4?allowPublicKeyRetrieval=true&useSSL=false`,
|
||||
'-Dmysql.user=root',
|
||||
'test-compile',
|
||||
'failsafe:integration-test',
|
||||
'failsafe:verify',
|
||||
],
|
||||
{
|
||||
cwd: root,
|
||||
stdio: 'inherit',
|
||||
env: {
|
||||
...process.env,
|
||||
QWEN_HOME: temporary,
|
||||
QWEN_RUNTIME_DIR: `${temporary}/qwen-runtime`,
|
||||
},
|
||||
},
|
||||
);
|
||||
let code;
|
||||
try {
|
||||
[code] = await once(verification, 'exit');
|
||||
} catch {
|
||||
throw new Error('Maven failed to start. Check MAVEN_EXECUTABLE.');
|
||||
}
|
||||
assert.equal(code, 0, 'M4 real MySQL provider gate failed');
|
||||
const finalHash = createHash('sha256')
|
||||
.update(await readFile(path.join(root, 'dist/cli.js')))
|
||||
.digest('hex');
|
||||
assert.equal(
|
||||
finalHash,
|
||||
process.argv[3],
|
||||
'Packaged CLI changed during M4 acceptance',
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
await stopFixtureProcess(verification);
|
||||
await stopFixtureProcess(mysql, () => {
|
||||
try {
|
||||
execFileSync(
|
||||
'mysqladmin',
|
||||
['--no-defaults', `--socket=${socket}`, '-u', 'root', 'shutdown'],
|
||||
{ stdio: 'pipe', timeout: 1000, killSignal: 'SIGKILL' },
|
||||
);
|
||||
} catch {
|
||||
mysql.kill('SIGKILL');
|
||||
}
|
||||
});
|
||||
await rm(temporary, { recursive: true, force: true });
|
||||
console.log('M4_ISOLATED_MYSQL_CLEANED');
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { once } from 'node:events';
|
||||
import { setTimeout, clearTimeout } from 'node:timers';
|
||||
|
||||
export const PROVIDER_MEDIA_MYSQL_PREFIX =
|
||||
'qwen-e2e-home-provider-media-mysql-';
|
||||
export const PROVIDER_MEDIA_HARNESS_PREFIX =
|
||||
'qwen-e2e-home-provider-media-harness-';
|
||||
export const PROVIDER_MEDIA_FIXTURES_PREFIX =
|
||||
'qwen-e2e-home-provider-media-fixtures-';
|
||||
export const PROVIDER_MEDIA_PROBE_PREFIX =
|
||||
'qwen-e2e-home-provider-media-probe-';
|
||||
|
||||
export async function stopFixtureProcess(
|
||||
child,
|
||||
shutdown = () => child.kill('SIGKILL'),
|
||||
graceMs = 1000,
|
||||
) {
|
||||
if (!child?.pid || child.exitCode !== null || child.signalCode !== null)
|
||||
return;
|
||||
const exited = once(child, 'exit');
|
||||
const timer = setTimeout(() => child.kill('SIGKILL'), graceMs);
|
||||
try {
|
||||
try {
|
||||
shutdown();
|
||||
} catch {
|
||||
child.kill('SIGKILL');
|
||||
}
|
||||
await exited;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import fs from 'node:fs';
|
||||
import childProcess from 'node:child_process';
|
||||
import path from 'node:path';
|
||||
import process from 'node:process';
|
||||
import { syncBuiltinESMExports } from 'node:module';
|
||||
|
||||
const append = fs.appendFileSync;
|
||||
function record(file) {
|
||||
if (
|
||||
typeof file !== 'string' ||
|
||||
!/^proof\.(png|jpeg|webp|gif|pdf)$/.test(path.basename(file))
|
||||
)
|
||||
return;
|
||||
const key = path.resolve(file);
|
||||
append(
|
||||
path.join(path.dirname(key), '.provider-media-reads.json'),
|
||||
JSON.stringify({ file: key, pid: process.pid }) + '\n',
|
||||
);
|
||||
}
|
||||
for (const [owner, names] of [
|
||||
[fs.promises, ['open', 'readFile']],
|
||||
[fs, ['open', 'openSync', 'readFile', 'readFileSync', 'createReadStream']],
|
||||
]) {
|
||||
for (const name of names) {
|
||||
const original = owner[name];
|
||||
owner[name] = function (file, ...args) {
|
||||
record(file);
|
||||
return original.call(this, file, ...args);
|
||||
};
|
||||
}
|
||||
}
|
||||
const execFile = childProcess.execFile;
|
||||
childProcess.execFile = function (command, args, ...options) {
|
||||
if (
|
||||
['pdfinfo', 'pdftotext', 'pdftoppm'].includes(path.basename(command)) &&
|
||||
Array.isArray(args)
|
||||
) {
|
||||
for (const file of args) record(file);
|
||||
}
|
||||
return execFile.call(this, command, args, ...options);
|
||||
};
|
||||
Object.defineProperties(
|
||||
childProcess.execFile,
|
||||
Object.getOwnPropertyDescriptors(execFile),
|
||||
);
|
||||
syncBuiltinESMExports();
|
||||
@@ -5,11 +5,14 @@
|
||||
*/
|
||||
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { managedToolDigest } from '@qwen-code/qwen-code-core/tools/managed-tool-protocol.js';
|
||||
import { ToolConfirmationOutcome } from '@qwen-code/qwen-code-core/tools/tools.js';
|
||||
import {
|
||||
MANAGED_RUNTIME_PROVIDER_PROTOCOL,
|
||||
MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES,
|
||||
MAX_PROVIDER_MEDIA_RESULT_BYTES,
|
||||
ManagedRuntimeProviderProtocolError,
|
||||
MANAGED_RUNTIME_PROVIDER_ROUTE,
|
||||
MANAGED_WORKSPACE_CONTEXT_FILE_CHARS,
|
||||
@@ -56,6 +59,10 @@ const reference = {
|
||||
argsDigest: managedToolDigest({ file_path: 'note.txt' }),
|
||||
};
|
||||
|
||||
function resultDigest(value: unknown): string {
|
||||
return createHash('sha256').update(JSON.stringify(value)).digest('hex');
|
||||
}
|
||||
|
||||
describe('managed-runtime-provider/1', () => {
|
||||
it.each(fixtures.cases)('$name', ({ valid, request }) => {
|
||||
if (valid)
|
||||
@@ -896,7 +903,7 @@ describe('managed-runtime-provider/1', () => {
|
||||
expect(kept.length + omitted).toBe(20_000);
|
||||
});
|
||||
|
||||
it('stubs model content the cut cannot reach when nothing else is left', () => {
|
||||
it('refuses unadmitted media without replacing its retained outcome', () => {
|
||||
const limit = 1024 * 1024;
|
||||
const result = {
|
||||
executionStatus: 'success',
|
||||
@@ -907,11 +914,231 @@ describe('managed-runtime-provider/1', () => {
|
||||
] as unknown,
|
||||
},
|
||||
};
|
||||
const original = structuredClone(result);
|
||||
expect(() =>
|
||||
fitManagedRuntimeProviderResult(execute, result, limit),
|
||||
).toThrow('media result exceeds its admitted budget');
|
||||
expect(resultDigest(result)).toBe(resultDigest(original));
|
||||
});
|
||||
|
||||
it.each(['execute', 'status', 'cancel'] as const)(
|
||||
'preserves the entire maximum media result through %s fitting',
|
||||
(kind) => {
|
||||
const caption = { text: 'caption "中"\n' };
|
||||
const toolResult = {
|
||||
llmContent: [
|
||||
{
|
||||
inlineData: {
|
||||
mimeType: 'image/png',
|
||||
data: 'A'.repeat(MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES),
|
||||
},
|
||||
},
|
||||
caption,
|
||||
],
|
||||
returnDisplay: 'original display',
|
||||
};
|
||||
caption.text += 'x'.repeat(
|
||||
MAX_PROVIDER_MEDIA_RESULT_BYTES -
|
||||
Buffer.byteLength(JSON.stringify(toolResult)),
|
||||
);
|
||||
const original = structuredClone(toolResult);
|
||||
const terminal = {
|
||||
executionStatus: 'success',
|
||||
result: toolResult,
|
||||
postHook: { note: 'h'.repeat(140 * 1024) },
|
||||
};
|
||||
const value =
|
||||
kind === 'execute'
|
||||
? terminal
|
||||
: {
|
||||
state: 'settled',
|
||||
cancelRequested: false,
|
||||
lastSeq: Number.MAX_SAFE_INTEGER - 1,
|
||||
firstAvailableSeq: Number.MAX_SAFE_INTEGER - 1,
|
||||
progressGap: false,
|
||||
progress: [
|
||||
{
|
||||
seq: Number.MAX_SAFE_INTEGER - 1,
|
||||
output: 'p'.repeat(300 * 1024),
|
||||
},
|
||||
],
|
||||
result: terminal,
|
||||
};
|
||||
const maximumSession = {
|
||||
...session,
|
||||
harnessSessionId: 's'.repeat(512),
|
||||
};
|
||||
const envelope = (result: unknown) => ({
|
||||
protocolVersion: 1,
|
||||
providerProtocol: MANAGED_RUNTIME_PROVIDER_PROTOCOL,
|
||||
session: maximumSession,
|
||||
result,
|
||||
});
|
||||
const limit =
|
||||
1024 * 1024 - Buffer.byteLength(JSON.stringify(envelope(0))) + 1;
|
||||
fitManagedRuntimeProviderResult(
|
||||
kind === 'execute' ? execute : { kind, reference },
|
||||
value,
|
||||
limit,
|
||||
);
|
||||
expect(resultDigest(terminal.result)).toBe(resultDigest(original));
|
||||
expect(() =>
|
||||
parseManagedRuntimeProviderResult(
|
||||
kind === 'execute' ? execute : { kind, reference },
|
||||
value,
|
||||
maximumSession,
|
||||
),
|
||||
).not.toThrow();
|
||||
expect(terminal).not.toHaveProperty('postHook');
|
||||
expect(
|
||||
Buffer.byteLength(JSON.stringify(envelope(value))),
|
||||
).toBeLessThanOrEqual(1024 * 1024);
|
||||
if ('progress' in value) {
|
||||
expect(value.progressGap).toBe(true);
|
||||
expect(value.progress).toEqual([]);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it.each([
|
||||
{ mimeType: 'text/html', data: 'AA==' },
|
||||
{ mimeType: 'image/png', data: 'AR==' },
|
||||
{ mimeType: 'application/pdf', data: 'not-base64' },
|
||||
])(
|
||||
'rejects invalid inline media even when its envelope fits',
|
||||
(inlineData) => {
|
||||
const result = {
|
||||
executionStatus: 'success',
|
||||
result: { llmContent: { inlineData }, returnDisplay: 'image' },
|
||||
};
|
||||
const original = structuredClone(result);
|
||||
expect(() =>
|
||||
fitManagedRuntimeProviderResult(execute, result, 1024 * 1024),
|
||||
).toThrow('invalid inline media');
|
||||
expect(result).toEqual(original);
|
||||
},
|
||||
);
|
||||
|
||||
it('stubs non-media content that the text cut cannot reach', () => {
|
||||
const limit = 1024 * 1024;
|
||||
const result = {
|
||||
executionStatus: 'success',
|
||||
result: {
|
||||
llmContent: [
|
||||
{ functionResponse: { response: { text: 'x'.repeat(limit) } } },
|
||||
],
|
||||
},
|
||||
};
|
||||
fitManagedRuntimeProviderResult(execute, result, limit);
|
||||
expect(
|
||||
Buffer.byteLength(JSON.stringify(result), 'utf8'),
|
||||
).toBeLessThanOrEqual(limit);
|
||||
expect(typeof result.result.llmContent).toBe('string');
|
||||
expect(result.result.llmContent).toBe(
|
||||
'[Managed Runtime provider omitted this tool result to fit the wire limit.]',
|
||||
);
|
||||
expect(Buffer.byteLength(JSON.stringify(result))).toBeLessThanOrEqual(
|
||||
limit,
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses a media-bearing result above its complete JSON budget', () => {
|
||||
const result = {
|
||||
executionStatus: 'success',
|
||||
result: {
|
||||
llmContent: [
|
||||
{ inlineData: { mimeType: 'image/png', data: 'AA==' } },
|
||||
{ text: 'x'.repeat(MAX_PROVIDER_MEDIA_RESULT_BYTES) },
|
||||
],
|
||||
},
|
||||
};
|
||||
expect(() =>
|
||||
fitManagedRuntimeProviderResult(execute, result, 1024 * 1024),
|
||||
).toThrow('media result exceeds its admitted budget');
|
||||
});
|
||||
|
||||
it.each([28, 5000])(
|
||||
'preserves a %i-character Hook stop reason when fitting media',
|
||||
(length) => {
|
||||
const stopReason = 'x'.repeat(length);
|
||||
const hookError = 'Hook failed.';
|
||||
const result = {
|
||||
executionStatus: 'success',
|
||||
result: {
|
||||
llmContent: {
|
||||
inlineData: {
|
||||
mimeType: 'image/png',
|
||||
data: 'A'.repeat(MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES),
|
||||
},
|
||||
},
|
||||
},
|
||||
postHook: {
|
||||
shouldStop: true,
|
||||
stopReason,
|
||||
hookError,
|
||||
additionalContext: 'c'.repeat(512 * 1024),
|
||||
},
|
||||
};
|
||||
const original = resultDigest(result.result);
|
||||
fitManagedRuntimeProviderResult(execute, result, 1024 * 1024);
|
||||
expect(result.postHook).toEqual({
|
||||
shouldStop: true,
|
||||
stopReason,
|
||||
hookError,
|
||||
});
|
||||
expect(resultDigest(result.result)).toBe(original);
|
||||
},
|
||||
);
|
||||
|
||||
it.each(['stopReason', 'hookError'])(
|
||||
'trims only the oversized Hook %s diagnostic',
|
||||
(field) => {
|
||||
const result = {
|
||||
executionStatus: 'success',
|
||||
result: {
|
||||
llmContent: {
|
||||
inlineData: {
|
||||
mimeType: 'image/png',
|
||||
data: 'A'.repeat(MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES),
|
||||
},
|
||||
},
|
||||
},
|
||||
postHook: {
|
||||
shouldStop: true,
|
||||
stopReason: 'Stop requested.',
|
||||
hookError: 'Hook failed.',
|
||||
[field]: 'x'.repeat(1024 * 1024),
|
||||
},
|
||||
};
|
||||
const original = resultDigest(result.result);
|
||||
fitManagedRuntimeProviderResult(execute, result, 1024 * 1024);
|
||||
expect(result.postHook.shouldStop).toBe(true);
|
||||
expect(result.postHook[field as 'stopReason' | 'hookError']).toContain(
|
||||
'omitted to fit',
|
||||
);
|
||||
expect(
|
||||
result.postHook[field === 'stopReason' ? 'hookError' : 'stopReason'],
|
||||
).toBe(field === 'stopReason' ? 'Hook failed.' : 'Stop requested.');
|
||||
expect(resultDigest(result.result)).toBe(original);
|
||||
expect(Buffer.byteLength(JSON.stringify(result))).toBeLessThanOrEqual(
|
||||
1024 * 1024,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('keeps original media when an impossible observation budget is refused', () => {
|
||||
const result = {
|
||||
executionStatus: 'success',
|
||||
result: {
|
||||
llmContent: { inlineData: { mimeType: 'image/png', data: 'AA==' } },
|
||||
returnDisplay: 'original display',
|
||||
},
|
||||
postHook: { note: 'auxiliary' },
|
||||
failureHook: { note: 'auxiliary' },
|
||||
};
|
||||
const original = structuredClone(result);
|
||||
expect(() =>
|
||||
fitManagedRuntimeProviderResult(execute, result, 16),
|
||||
).toThrow('media observation exceeds its wire budget');
|
||||
expect(resultDigest(result.result)).toBe(resultDigest(original.result));
|
||||
expect(result).not.toHaveProperty('postHook');
|
||||
expect(result).not.toHaveProperty('failureHook');
|
||||
});
|
||||
|
||||
it('keeps hooks when cutting the text makes room', () => {
|
||||
|
||||
@@ -31,8 +31,11 @@ import { isShellResultDisplay } from '@qwen-code/qwen-code-core/utils/shell-resu
|
||||
import { ToolConfirmationOutcome } from '@qwen-code/qwen-code-core/tools/tools.js';
|
||||
import type { ToolConfirmationPayload } from '@qwen-code/qwen-code-core/tools/tools.js';
|
||||
import type { ManagedToolConfirmationPhase } from '@qwen-code/qwen-code-core/tools/managed-tool-runtime.js';
|
||||
import { managedToolResponseMediaBytes } from '../acp-integration/managed-tool-media.js';
|
||||
|
||||
export const MANAGED_RUNTIME_PROVIDER_PROTOCOL = 'managed-runtime-provider/1';
|
||||
export const MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES = 768 * 1024;
|
||||
export const MAX_PROVIDER_MEDIA_RESULT_BYTES = 896 * 1024;
|
||||
export const MANAGED_RUNTIME_PROVIDER_ROUTE = Object.freeze({
|
||||
key: 'provider-control',
|
||||
method: 'POST',
|
||||
@@ -763,6 +766,8 @@ function providerFitLevel(
|
||||
* through `firstAvailableSeq`/`progressGap`), then bulk text fields are cut
|
||||
* head-and-tail with an inline notice; `truncated` is set on shell displays.
|
||||
* Mutates and returns `value`; the caller owns a JSON-round-tripped copy.
|
||||
* Invalid or unadmitted media is refused before fitting. An impossible media
|
||||
* envelope is refused after auxiliary fields may have been trimmed.
|
||||
*/
|
||||
export function fitManagedRuntimeProviderResult(
|
||||
operation: ManagedRuntimeProviderOperation,
|
||||
@@ -779,11 +784,45 @@ export function fitManagedRuntimeProviderResult(
|
||||
const root = value as Record<string, unknown>;
|
||||
const fits = () =>
|
||||
Buffer.byteLength(JSON.stringify(root), 'utf8') <= budgetBytes;
|
||||
if (fits()) return value;
|
||||
const status = operation.kind === 'execute' ? undefined : root;
|
||||
const execution = (
|
||||
operation.kind === 'execute' ? root : status?.['result']
|
||||
) as Record<string, unknown> | undefined;
|
||||
const toolResult = execution?.['result'];
|
||||
const content =
|
||||
toolResult && typeof toolResult === 'object' && !Array.isArray(toolResult)
|
||||
? (toolResult as Record<string, unknown>)['llmContent']
|
||||
: undefined;
|
||||
const mediaResult = (Array.isArray(content) ? content : [content]).some(
|
||||
(part) =>
|
||||
part !== null &&
|
||||
typeof part === 'object' &&
|
||||
!Array.isArray(part) &&
|
||||
Object.hasOwn(part, 'inlineData'),
|
||||
);
|
||||
if (mediaResult) {
|
||||
let mediaBytes: number;
|
||||
try {
|
||||
mediaBytes = managedToolResponseMediaBytes(
|
||||
{ executionStatus: execution?.['executionStatus'], result: toolResult },
|
||||
'execute',
|
||||
);
|
||||
} catch {
|
||||
throw new ManagedRuntimeProviderProtocolError(
|
||||
'Managed Runtime provider returned invalid inline media.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
mediaBytes > MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES ||
|
||||
Buffer.byteLength(JSON.stringify(toolResult), 'utf8') >
|
||||
MAX_PROVIDER_MEDIA_RESULT_BYTES
|
||||
) {
|
||||
throw new ManagedRuntimeProviderProtocolError(
|
||||
'Managed Runtime provider media result exceeds its admitted budget.',
|
||||
);
|
||||
}
|
||||
}
|
||||
if (fits()) return value;
|
||||
|
||||
// 1. Evict oldest progress events; they re-derive from the settled result.
|
||||
const progress = status?.['progress'];
|
||||
@@ -820,6 +859,58 @@ export function fitManagedRuntimeProviderResult(
|
||||
}
|
||||
}
|
||||
|
||||
// The Runtime already retained this result. Only auxiliary fields may be
|
||||
// fitted; changing the media or its caption would create another receipt.
|
||||
if (mediaResult && execution) {
|
||||
if (!fits()) {
|
||||
const hook = execution['postHook'];
|
||||
if (
|
||||
hook &&
|
||||
typeof hook === 'object' &&
|
||||
!Array.isArray(hook) &&
|
||||
typeof (hook as Record<string, unknown>)['shouldStop'] === 'boolean'
|
||||
) {
|
||||
const fields = hook as Record<string, unknown>;
|
||||
const reason = fields['stopReason'];
|
||||
const hookError = fields['hookError'];
|
||||
execution['postHook'] = {
|
||||
shouldStop: fields['shouldStop'],
|
||||
...(typeof reason === 'string'
|
||||
? {
|
||||
stopReason: reason,
|
||||
}
|
||||
: {}),
|
||||
...(typeof hookError === 'string' ? { hookError } : {}),
|
||||
};
|
||||
} else {
|
||||
delete execution['postHook'];
|
||||
}
|
||||
delete execution['failureHook'];
|
||||
const retained = execution['postHook'] as
|
||||
| Record<string, unknown>
|
||||
| undefined;
|
||||
if (retained) {
|
||||
const diagnostics = ['stopReason', 'hookError']
|
||||
.filter((field) => typeof retained[field] === 'string')
|
||||
.sort(
|
||||
(a, b) =>
|
||||
Buffer.byteLength(retained[b] as string, 'utf8') -
|
||||
Buffer.byteLength(retained[a] as string, 'utf8'),
|
||||
);
|
||||
for (const field of diagnostics) {
|
||||
if (fits()) break;
|
||||
retained[field] =
|
||||
`Runtime Hook ${field} omitted to fit the media response.`;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!fits())
|
||||
throw new ManagedRuntimeProviderProtocolError(
|
||||
'Managed Runtime provider media observation exceeds its wire budget.',
|
||||
);
|
||||
return value;
|
||||
}
|
||||
|
||||
// 2. Cut the bulk text fields once, down to one common size. Each field is
|
||||
// cut at most once, so its notice counts everything omitted. When even
|
||||
// every field at its floor cannot meet the budget, step 3 takes over.
|
||||
@@ -870,8 +961,8 @@ export function fitManagedRuntimeProviderResult(
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Last resort: when even that cannot fit (content the cut cannot reach,
|
||||
// such as inline media), the model content becomes an explicit stub so
|
||||
// 3. Last resort: when other content the cut cannot reach cannot fit,
|
||||
// the model content becomes an explicit stub so
|
||||
// the terminal observation always fits.
|
||||
if (!fits() && execution) {
|
||||
const result = execution['result'];
|
||||
|
||||
@@ -55,6 +55,7 @@ import {
|
||||
MANAGED_RUNTIME_PROVIDER_ROUTE,
|
||||
MANAGED_WORKSPACE_CONTEXT_FILE_CHARS,
|
||||
managedRuntimeProviderLimit,
|
||||
MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES,
|
||||
type ManagedRuntimeProviderSession,
|
||||
} from './managed-runtime-provider-protocol.js';
|
||||
import { MANAGED_CONTEXT_PROTOCOL } from './managed-context-envelope.js';
|
||||
@@ -435,6 +436,30 @@ function reference(
|
||||
};
|
||||
}
|
||||
|
||||
function providerPdf(): Buffer {
|
||||
const stream = 'BT /F1 12 Tf 20 100 Td (Provider media fixture) Tj ET';
|
||||
const objects = [
|
||||
'<< /Type /Catalog /Pages 2 0 R >>',
|
||||
'<< /Type /Pages /Kids [3 0 R] /Count 1 >>',
|
||||
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << /Font << /F1 4 0 R >> >> /Contents 5 0 R >>',
|
||||
'<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>',
|
||||
`<< /Length ${stream.length} >>\nstream\n${stream}\nendstream`,
|
||||
];
|
||||
const offsets: number[] = [];
|
||||
let pdf = '%PDF-1.4\n';
|
||||
objects.forEach((object, index) => {
|
||||
offsets.push(Buffer.byteLength(pdf));
|
||||
pdf += `${index + 1} 0 obj\n${object}\nendobj\n`;
|
||||
});
|
||||
const xref = Buffer.byteLength(pdf);
|
||||
pdf += `xref\n0 6\n0000000000 65535 f \n`;
|
||||
pdf += offsets
|
||||
.map((offset) => `${String(offset).padStart(10, '0')} 00000 n \n`)
|
||||
.join('');
|
||||
pdf += `trailer\n<< /Size 6 /Root 1 0 R >>\nstartxref\n${xref}\n%%EOF\n`;
|
||||
return Buffer.from(pdf);
|
||||
}
|
||||
|
||||
describe('Managed Runtime provider worker', () => {
|
||||
let workspace: string;
|
||||
let storage: string;
|
||||
@@ -545,6 +570,215 @@ describe('Managed Runtime provider worker', () => {
|
||||
return control<T>({ kind: 'execute', reference: ref });
|
||||
}
|
||||
|
||||
it('delivers invocation-local images without inheriting Omni uploads or later file bytes', async () => {
|
||||
vi.stubEnv('QWEN_CODE_ENABLE_OMNI', '1');
|
||||
onTestFinished(() => {
|
||||
vi.unstubAllEnvs();
|
||||
});
|
||||
const upload = vi
|
||||
.spyOn(Config.prototype, 'loadOmniMediaReader')
|
||||
.mockRejectedValue(
|
||||
new Error('Execution worker must not load the upload pipeline.'),
|
||||
);
|
||||
const data = 'R0lGODlhAQABAIAAAP8AAP///yH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==';
|
||||
const file = path.join(workspace, 'pixel.gif');
|
||||
fs.writeFileSync(file, Buffer.from(data, 'base64'));
|
||||
await begin();
|
||||
const prepareImage = (callId: string, enabled?: boolean) =>
|
||||
control<ManagedToolPrepareResponse>({
|
||||
kind: 'prepare',
|
||||
identity: { ...identity, callId },
|
||||
toolName: 'read_file',
|
||||
input: { file_path: file },
|
||||
...(enabled === undefined
|
||||
? {}
|
||||
: { mediaContext: { inputModalities: { image: enabled } } }),
|
||||
});
|
||||
const [enabled, disabled, missing] = await Promise.all([
|
||||
prepareImage('image-enabled', true),
|
||||
prepareImage('image-disabled', false),
|
||||
prepareImage('image-missing'),
|
||||
]);
|
||||
type Execution = {
|
||||
executionStatus: string;
|
||||
result: { llmContent: unknown; returnDisplay: unknown };
|
||||
};
|
||||
const first = await execute<Execution>(reference(enabled));
|
||||
expect(first.executionStatus).toBe('success');
|
||||
expect(first.result.llmContent).toMatchObject({
|
||||
inlineData: { mimeType: 'image/gif', data },
|
||||
});
|
||||
for (const prepared of [disabled, missing]) {
|
||||
const answer = await execute<Execution>(reference(prepared));
|
||||
expect(answer.executionStatus).toBe('success');
|
||||
expect(answer.result.llmContent).toContain('Unsupported image file');
|
||||
}
|
||||
fs.writeFileSync(file, 'changed after settlement');
|
||||
const ref = reference(enabled);
|
||||
expect(await control({ kind: 'execute', reference: ref })).toEqual(first);
|
||||
for (const kind of ['status', 'cancel']) {
|
||||
expect(await control({ kind, reference: ref })).toMatchObject({
|
||||
state: 'settled',
|
||||
result: first,
|
||||
});
|
||||
}
|
||||
expect(upload).not.toHaveBeenCalled();
|
||||
expect(await control({ kind: 'release' })).toBe(true);
|
||||
});
|
||||
|
||||
it('delivers native PDF bytes and settles an oversized PDF as a tool error', async () => {
|
||||
const file = path.join(workspace, 'document.pdf');
|
||||
const pdf = providerPdf();
|
||||
fs.writeFileSync(file, pdf);
|
||||
await begin();
|
||||
const preparePdf = (callId: string) =>
|
||||
control<ManagedToolPrepareResponse>({
|
||||
kind: 'prepare',
|
||||
identity: { ...identity, callId },
|
||||
toolName: 'read_file',
|
||||
input: { file_path: file },
|
||||
mediaContext: { inputModalities: { pdf: true } },
|
||||
});
|
||||
const ref = reference(await preparePdf('native-pdf'));
|
||||
const answer = await execute(ref);
|
||||
expect(answer).toMatchObject({
|
||||
executionStatus: 'success',
|
||||
result: {
|
||||
llmContent: {
|
||||
inlineData: {
|
||||
mimeType: 'application/pdf',
|
||||
data: pdf.toString('base64'),
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
fs.appendFileSync(
|
||||
file,
|
||||
Buffer.alloc(MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES),
|
||||
);
|
||||
const large = reference(await preparePdf('large-pdf'));
|
||||
const refused = await execute(large);
|
||||
expect(refused).toMatchObject({
|
||||
executionStatus: 'error',
|
||||
result: { error: { type: 'file_too_large' } },
|
||||
});
|
||||
expect(await control({ kind: 'status', reference: large })).toMatchObject({
|
||||
state: 'settled',
|
||||
result: refused,
|
||||
});
|
||||
expect(await control({ kind: 'release' })).toBe(true);
|
||||
});
|
||||
|
||||
it('isolates media capabilities across concurrently prepared and executed Sessions', async () => {
|
||||
const data = 'R0lGODlhAQABAIAAAP8AAP///yH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==';
|
||||
const file = path.join(workspace, 'shared.gif');
|
||||
fs.writeFileSync(file, Buffer.from(data, 'base64'));
|
||||
await begin();
|
||||
const second: ManagedRuntimeProviderSession = {
|
||||
harnessSessionId: '550e8400-e29b-41d4-a716-446655440011',
|
||||
runtimeSessionId: '550e8400-e29b-41d4-a716-446655440012',
|
||||
turnKind: 'bootstrap',
|
||||
};
|
||||
await control({ kind: 'acquire' }, second);
|
||||
const manifest = await control<{
|
||||
capabilityDigest: string;
|
||||
policyRevision: string;
|
||||
}>({ kind: 'manifest' }, second);
|
||||
const otherIdentity = {
|
||||
...identity,
|
||||
sessionId: second.runtimeSessionId,
|
||||
capabilityDigest: manifest.capabilityDigest,
|
||||
policyRevision: manifest.policyRevision,
|
||||
};
|
||||
await control(
|
||||
{
|
||||
kind: 'bind-history',
|
||||
binding: {
|
||||
...binding(),
|
||||
ownerSessionId: second.harnessSessionId,
|
||||
ownerRuntimeSessionId: second.runtimeSessionId,
|
||||
},
|
||||
},
|
||||
second,
|
||||
);
|
||||
await control({ kind: 'begin-turn', identity: otherIdentity }, second);
|
||||
const prepareFor = (
|
||||
session: ManagedRuntimeProviderSession,
|
||||
call: ManagedToolCallIdentity,
|
||||
image: boolean,
|
||||
) =>
|
||||
control<ManagedToolPrepareResponse>(
|
||||
{
|
||||
kind: 'prepare',
|
||||
identity: call,
|
||||
toolName: 'read_file',
|
||||
input: { file_path: file },
|
||||
mediaContext: { inputModalities: { image } },
|
||||
},
|
||||
session,
|
||||
);
|
||||
const [enabled, disabled] = await Promise.all([
|
||||
prepareFor(SESSION, identity, true),
|
||||
prepareFor(second, otherIdentity, false),
|
||||
]);
|
||||
const executeFor = async (
|
||||
session: ManagedRuntimeProviderSession,
|
||||
prepared: ManagedToolPrepareResponse,
|
||||
) => {
|
||||
const ref = reference(prepared);
|
||||
await control({ kind: 'preflight', reference: ref }, session);
|
||||
return control<{
|
||||
executionStatus: string;
|
||||
result: { llmContent: unknown };
|
||||
}>({ kind: 'execute', reference: ref }, session);
|
||||
};
|
||||
const [first, other] = await Promise.all([
|
||||
executeFor(SESSION, enabled),
|
||||
executeFor(second, disabled),
|
||||
]);
|
||||
expect(first.executionStatus).toBe('success');
|
||||
expect(first.result.llmContent).toMatchObject({
|
||||
inlineData: { mimeType: 'image/gif', data },
|
||||
});
|
||||
expect(other.executionStatus).toBe('success');
|
||||
expect(other.result.llmContent).toContain('Unsupported image file');
|
||||
expect(await control({ kind: 'release' })).toBe(true);
|
||||
expect(await control({ kind: 'release' }, second)).toBe(true);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['audio', 'mp3'],
|
||||
['video', 'mp4'],
|
||||
])(
|
||||
'does not enable %s from a multimodal provider context',
|
||||
async (modality, extension) => {
|
||||
const file = path.join(workspace, `clip.${extension}`);
|
||||
fs.writeFileSync(
|
||||
file,
|
||||
Buffer.from('ID3\u0000\u0000\u0000\u0000\u0000\u0000\u0000'),
|
||||
);
|
||||
await begin();
|
||||
const prepared = await control<ManagedToolPrepareResponse>({
|
||||
kind: 'prepare',
|
||||
identity,
|
||||
toolName: 'read_file',
|
||||
input: { file_path: file },
|
||||
mediaContext: {
|
||||
inputModalities: { image: true, pdf: true, audio: true, video: true },
|
||||
},
|
||||
});
|
||||
const answer = await execute<{
|
||||
executionStatus: string;
|
||||
result: { llmContent: unknown };
|
||||
}>(reference(prepared));
|
||||
expect(answer.executionStatus).toBe('success');
|
||||
expect(answer.result.llmContent).toContain(
|
||||
`Unsupported ${modality} file`,
|
||||
);
|
||||
expect(await control({ kind: 'release' })).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
it('reports input errors separately from identity conflicts and permits corrected preparation', async () => {
|
||||
await begin();
|
||||
for (const [toolName, input, reason] of [
|
||||
|
||||
@@ -52,6 +52,8 @@ import {
|
||||
} from './managed-runtime-attestation-contract.js';
|
||||
import {
|
||||
MANAGED_RUNTIME_PROVIDER_ROUTE,
|
||||
MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES,
|
||||
MAX_PROVIDER_MEDIA_RESULT_BYTES,
|
||||
managedRuntimeProviderLimit,
|
||||
ManagedRuntimeProviderProtocolError,
|
||||
fitManagedRuntimeProviderResult,
|
||||
@@ -502,6 +504,8 @@ export class ManagedRuntimeProviderWorker {
|
||||
});
|
||||
const toolConfig = deriveConfig(config, {
|
||||
getFileHistoryService: () => history(value).service,
|
||||
// Execution workers never upload media through an inherited Omni opt-in.
|
||||
isOmniEnabled: () => false,
|
||||
});
|
||||
const shell = new WorkspaceShellTool(toolConfig, config);
|
||||
const tools = [
|
||||
@@ -529,10 +533,19 @@ export class ManagedRuntimeProviderWorker {
|
||||
conflict('Managed Runtime tool does not support media context.');
|
||||
return new ReadFileTool(
|
||||
deriveConfig(toolConfig, {
|
||||
getEffectiveInputModalities: () => ({ ...media.inputModalities }),
|
||||
getEffectiveInputModalities: () => ({
|
||||
image: media.inputModalities.image === true,
|
||||
pdf: media.inputModalities.pdf === true,
|
||||
audio: false,
|
||||
video: false,
|
||||
}),
|
||||
getFileReadCache: () => toolConfig.getFileReadCache(),
|
||||
getFileService: () => toolConfig.getFileService(),
|
||||
}),
|
||||
{
|
||||
maxInlineMediaBase64Bytes: MAX_PROVIDER_INLINE_MEDIA_BASE64_BYTES,
|
||||
maxMediaResultBytes: MAX_PROVIDER_MEDIA_RESULT_BYTES,
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
@@ -2492,6 +2492,7 @@ export type DerivedConfigOverrides = Partial<
|
||||
| 'getWorkspaceContext'
|
||||
| 'getFileService'
|
||||
| 'getEffectiveInputModalities'
|
||||
| 'isOmniEnabled'
|
||||
| 'getFileReadCache'
|
||||
| 'getFileHistoryService'
|
||||
| 'getToolRegistry'
|
||||
|
||||
@@ -5,6 +5,10 @@
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import fs from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import sharp from 'sharp';
|
||||
import type { Config } from '../config/config.js';
|
||||
import { ApprovalMode } from '../config/config.js';
|
||||
import { StandardFileSystemService } from '../services/fileSystemService.js';
|
||||
@@ -446,6 +450,174 @@ describe('ManagedToolRuntime', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('binds actual local factory reads to each preparation capability snapshot', async () => {
|
||||
const { ReadFileTool } = await import('./read-file.js');
|
||||
const { FileReadCache } = await import('../services/fileReadCache.js');
|
||||
const directory = await fs.mkdtemp(join(tmpdir(), 'managed-local-media-'));
|
||||
const gif = await sharp({
|
||||
create: { width: 1, height: 1, channels: 3, background: '#306090' },
|
||||
})
|
||||
.gif()
|
||||
.toBuffer();
|
||||
const image = join(directory, 'tiny.gif');
|
||||
const pdf = join(directory, 'native.pdf');
|
||||
await fs.writeFile(image, gif);
|
||||
await fs.writeFile(pdf, '%PDF-1.7');
|
||||
const generator = vi.fn(() => ({ modalities: { image: true, pdf: true } }));
|
||||
Object.assign(config, {
|
||||
getTargetDir: () => directory,
|
||||
getWorkspaceContext: () => ({ isPathWithinWorkspace: () => true }),
|
||||
getFileService: () => ({ shouldQwenIgnoreFile: () => false }),
|
||||
getFileSystemService: () => new StandardFileSystemService(),
|
||||
getFileReadCache: () => new FileReadCache(),
|
||||
getFileReadCacheDisabled: () => true,
|
||||
getEffectiveInputModalities: () => ({ image: true }),
|
||||
getContentGeneratorConfig: generator,
|
||||
getModel: () => 'local-test',
|
||||
getUsageStatisticsEnabled: () => false,
|
||||
getPlansDir: () => join(directory, 'plans'),
|
||||
storage: {
|
||||
getProjectTempDir: () => join(directory, 'tmp'),
|
||||
getProjectDir: () => directory,
|
||||
getUserSkillsDirs: () => [],
|
||||
getWorkflowRunsDir: () => join(directory, 'workflow-runs'),
|
||||
},
|
||||
isLsToolEnabled: () => false,
|
||||
});
|
||||
const readTool = new ReadFileTool(config);
|
||||
config.getToolRegistry = () =>
|
||||
({
|
||||
getTool: (name: string) =>
|
||||
name === ReadFileTool.Name ? readTool : undefined,
|
||||
ensureTool: vi.fn(async () => undefined),
|
||||
}) as unknown as ReturnType<Config['getToolRegistry']>;
|
||||
try {
|
||||
runtime = await createBuiltinManagedToolRuntime(config);
|
||||
const manifest = runtime.manifest();
|
||||
const call = {
|
||||
...identity,
|
||||
capabilityDigest: manifest.capabilityDigest,
|
||||
policyRevision: manifest.policyRevision,
|
||||
};
|
||||
await runtime.beginTurn(call);
|
||||
const enabled = await runtime.prepare(
|
||||
call,
|
||||
ReadFileTool.Name,
|
||||
{ file_path: image },
|
||||
undefined,
|
||||
{ inputModalities: { image: true } },
|
||||
);
|
||||
const disabled = await runtime.prepare(
|
||||
{ ...call, callId: 'disabled' },
|
||||
ReadFileTool.Name,
|
||||
{ file_path: image },
|
||||
undefined,
|
||||
{ inputModalities: {} },
|
||||
);
|
||||
const missing = await runtime.prepare(
|
||||
{ ...call, callId: 'missing' },
|
||||
ReadFileTool.Name,
|
||||
{ file_path: image },
|
||||
);
|
||||
const native = await runtime.prepare(
|
||||
{ ...call, callId: 'pdf' },
|
||||
ReadFileTool.Name,
|
||||
{ file_path: pdf },
|
||||
undefined,
|
||||
{ inputModalities: { pdf: true } },
|
||||
);
|
||||
const execute = async (prepared: ManagedToolPrepareResponse) => {
|
||||
const ref = reference(prepared);
|
||||
await runtime.preflight(ref);
|
||||
return runtime.execute(ref);
|
||||
};
|
||||
const delivered = await execute(enabled);
|
||||
expect(delivered.executionStatus).toBe('success');
|
||||
expect(delivered.result?.llmContent).toEqual({
|
||||
inlineData: {
|
||||
data: gif.toString('base64'),
|
||||
mimeType: 'image/gif',
|
||||
displayName: 'tiny.gif',
|
||||
},
|
||||
});
|
||||
expect((await execute(disabled)).result?.llmContent).toContain(
|
||||
'Unsupported image',
|
||||
);
|
||||
expect((await execute(missing)).result?.llmContent).toEqual(
|
||||
delivered.result?.llmContent,
|
||||
);
|
||||
expect((await execute(native)).result?.llmContent).toEqual({
|
||||
inlineData: {
|
||||
data: Buffer.from('%PDF-1.7').toString('base64'),
|
||||
mimeType: 'application/pdf',
|
||||
displayName: 'native.pdf',
|
||||
},
|
||||
});
|
||||
await fs.writeFile(image, 'changed');
|
||||
expect(await runtime.execute(reference(enabled))).toEqual(delivered);
|
||||
expect(runtime.status(reference(enabled)).result).toEqual(delivered);
|
||||
expect(generator).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
await runtime.dispose();
|
||||
await fs.rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('retains a ReadFile budget refusal as the original terminal tool error', async () => {
|
||||
const { ReadFileTool } = await import('./read-file.js');
|
||||
const { FileReadCache } = await import('../services/fileReadCache.js');
|
||||
const directory = await fs.mkdtemp(join(tmpdir(), 'managed-media-budget-'));
|
||||
const file = join(directory, 'native.pdf');
|
||||
await fs.writeFile(file, '%PDF-1.7');
|
||||
Object.assign(config, {
|
||||
getTargetDir: () => directory,
|
||||
getWorkspaceContext: () => ({ isPathWithinWorkspace: () => true }),
|
||||
getFileService: () => ({ shouldQwenIgnoreFile: () => false }),
|
||||
getFileSystemService: () => new StandardFileSystemService(),
|
||||
getFileReadCache: () => new FileReadCache(),
|
||||
getFileReadCacheDisabled: () => true,
|
||||
getEffectiveInputModalities: () => ({ pdf: true }),
|
||||
getUsageStatisticsEnabled: () => false,
|
||||
getPlansDir: () => join(directory, 'plans'),
|
||||
storage: {
|
||||
getProjectTempDir: () => join(directory, 'tmp'),
|
||||
getProjectDir: () => directory,
|
||||
getUserSkillsDirs: () => [],
|
||||
getWorkflowRunsDir: () => join(directory, 'workflow-runs'),
|
||||
},
|
||||
});
|
||||
const bounded = new ReadFileTool(config, {
|
||||
maxInlineMediaBase64Bytes: 768 * 1024,
|
||||
maxMediaResultBytes: 1,
|
||||
});
|
||||
runtime = new ManagedToolRuntime(
|
||||
config,
|
||||
() => [bounded],
|
||||
() => revision,
|
||||
);
|
||||
try {
|
||||
const call = {
|
||||
...identity,
|
||||
capabilityDigest: runtime.manifest().capabilityDigest,
|
||||
};
|
||||
await runtime.beginTurn(call);
|
||||
const ref = reference(
|
||||
await runtime.prepare(call, ReadFileTool.Name, { file_path: file }),
|
||||
);
|
||||
await runtime.preflight(ref);
|
||||
const result = await runtime.execute(ref);
|
||||
expect(result.executionStatus).toBe('error');
|
||||
expect(result.result?.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(JSON.stringify(result)).not.toContain('inlineData');
|
||||
await fs.writeFile(file, 'changed');
|
||||
expect(runtime.status(ref)).toMatchObject({ state: 'settled', result });
|
||||
expect(await runtime.execute(ref)).toEqual(result);
|
||||
} finally {
|
||||
await runtime.dispose();
|
||||
await fs.rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('binds media-capable builtins to the modality view and rejects media on other tools', async () => {
|
||||
const { ReadFileTool, buildReadFileDescription } = await import(
|
||||
'./read-file.js'
|
||||
|
||||
@@ -27,6 +27,8 @@ import { createMockWorkspaceContext } from '../test-utils/mockWorkspaceContext.j
|
||||
import { SchemaValidator } from '../utils/schemaValidator.js';
|
||||
import type { ToolResult } from './tools.js';
|
||||
import type { VisionBridgeNoticeDisplay } from '../services/visionBridge/vision-bridge-service.js';
|
||||
import { normalizeParts } from '../services/visionBridge/image-part-utils.js';
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
const visionBridgeMocks = vi.hoisted(() => ({
|
||||
runVisionBridge: vi.fn(),
|
||||
@@ -103,6 +105,7 @@ describe('ReadFileTool', () => {
|
||||
getFileService: () => new FileDiscoveryService(tempRootDir),
|
||||
getFileSystemService: () => new StandardFileSystemService(),
|
||||
getTargetDir: () => tempRootDir,
|
||||
getModel: () => 'test-model',
|
||||
getWorkspaceContext: () => createMockWorkspaceContext(tempRootDir),
|
||||
storage: {
|
||||
getProjectTempDir: () => path.join(tempRootDir, '.temp'),
|
||||
@@ -197,6 +200,343 @@ describe('ReadFileTool', () => {
|
||||
await fsp.rm(tempRootDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('trusted media limits', () => {
|
||||
const limits = {
|
||||
maxInlineMediaBase64Bytes: 768 * 1024,
|
||||
maxMediaResultBytes: 896 * 1024,
|
||||
};
|
||||
const boundedTool = (overrides = {}) =>
|
||||
new ReadFileTool(
|
||||
makeConfig({ getFileReadCacheDisabled: () => true, ...overrides }),
|
||||
limits,
|
||||
);
|
||||
|
||||
it.skipIf(!['linux', 'darwin'].includes(process.platform)).each([0, 1])(
|
||||
'applies the media ceiling to the borrowed PDF extent with %i extra bytes',
|
||||
async (extra) => {
|
||||
const bytes = Buffer.alloc(
|
||||
(limits.maxInlineMediaBase64Bytes / 4) * 3 + extra,
|
||||
);
|
||||
bytes.write('%PDF-1.7');
|
||||
const file = await put('borrowed.pdf', bytes);
|
||||
const service = new StandardFileSystemService();
|
||||
const producer = service.withReadFile.bind(service);
|
||||
let borrowed: fsp.FileHandle | undefined;
|
||||
service.withReadFile = async <T>(
|
||||
request: FileReadRequest,
|
||||
operation: (source: FileReadSource) => Promise<T>,
|
||||
) =>
|
||||
producer(request, async (source) => {
|
||||
if (source.kind !== 'descriptor')
|
||||
throw new Error('expected descriptor');
|
||||
borrowed = source.fileHandle;
|
||||
await fsp.rename(file, `${file}.original`);
|
||||
await fsp.writeFile(file, '%PDF-1.7\nreplacement');
|
||||
const reading = vi.spyOn(source.fileHandle, 'read');
|
||||
const result = await operation(source);
|
||||
if (extra) expect(reading).not.toHaveBeenCalled();
|
||||
return result;
|
||||
});
|
||||
const result = await read(
|
||||
file,
|
||||
boundedTool({
|
||||
getFileSystemService: () => service,
|
||||
getEffectiveInputModalities: () => ({ pdf: true }),
|
||||
}),
|
||||
);
|
||||
if (extra) {
|
||||
expect(result.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(result.llmContent).toContain('inline media byte limit');
|
||||
expect(JSON.stringify(result.llmContent)).not.toContain('inlineData');
|
||||
} else {
|
||||
expect(result.error).toBeUndefined();
|
||||
const inline = normalizeParts(result.llmContent)[0]?.inlineData;
|
||||
expect(inline?.data).toBe(bytes.toString('base64'));
|
||||
}
|
||||
await expect(borrowed!.stat()).rejects.toMatchObject({ code: 'EBADF' });
|
||||
},
|
||||
);
|
||||
|
||||
it('accepts native PDF bytes at the exact base64 ceiling and refuses one decoded byte over', async () => {
|
||||
const bytes = Buffer.alloc((limits.maxInlineMediaBase64Bytes / 4) * 3);
|
||||
bytes.write('%PDF-1.7');
|
||||
const file = await put('native.pdf', bytes);
|
||||
const result = await read(file, boundedTool());
|
||||
expect(result.error).toBeUndefined();
|
||||
const inline = normalizeParts(result.llmContent)[0]?.inlineData;
|
||||
expect(inline?.mimeType).toBe('application/pdf');
|
||||
expect(inline?.displayName).toBe('native.pdf');
|
||||
expect(typeof inline?.data).toBe('string');
|
||||
expect(inline!.data!.length).toBe(limits.maxInlineMediaBase64Bytes);
|
||||
expect(
|
||||
createHash('sha256')
|
||||
.update(Buffer.from(inline!.data!, 'base64'))
|
||||
.digest('hex'),
|
||||
).toBe(createHash('sha256').update(bytes).digest('hex'));
|
||||
await fsp.appendFile(file, 'x');
|
||||
const refused = await read(file, boundedTool());
|
||||
expect(refused.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(refused.llmContent).toContain('inline media byte limit');
|
||||
expect(refused.llmContent).not.toContain(
|
||||
'inline media result byte limit',
|
||||
);
|
||||
expect(JSON.stringify(refused)).not.toContain('inlineData');
|
||||
expect(
|
||||
(await read(file, new ReadFileTool(makeConfig()))).error,
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it('applies the tool-level aggregate limit to audio from a trusted config', async () => {
|
||||
const file = await put('clip.mp3', Buffer.alloc(589825));
|
||||
const result = await read(
|
||||
file,
|
||||
boundedTool({ getEffectiveInputModalities: () => ({ audio: true }) }),
|
||||
);
|
||||
expect(result.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(result.llmContent).toContain('inline media result byte limit');
|
||||
});
|
||||
|
||||
it('measures complete JSON UTF-8 bytes, including escaped and multibyte metadata', async () => {
|
||||
const file = await put('文档.pdf', '%PDF-1.7');
|
||||
pdfMocks.getPDFPageCount.mockResolvedValue(2);
|
||||
pdfMocks.renderPDFPagesToImages.mockResolvedValue({
|
||||
success: true,
|
||||
images: [{ data: 'YWJj', mimeType: 'image/jpeg' }],
|
||||
bytesTruncated: true,
|
||||
});
|
||||
const overrides = {
|
||||
getEffectiveInputModalities: () => ({ image: true }),
|
||||
getFileReadCacheDisabled: () => true,
|
||||
};
|
||||
const baseline = await read(file, boundedTool(overrides));
|
||||
expect(baseline.error).toBeUndefined();
|
||||
expect(JSON.stringify(baseline)).toContain('inlineData');
|
||||
expect(JSON.stringify(baseline)).toContain('\\"');
|
||||
const bytes = Buffer.byteLength(JSON.stringify(baseline), 'utf8');
|
||||
expect(bytes).toBeGreaterThan(JSON.stringify(baseline).length);
|
||||
const config = makeConfig(overrides);
|
||||
const exact = new ReadFileTool(config, {
|
||||
...limits,
|
||||
maxMediaResultBytes: bytes,
|
||||
});
|
||||
expect(await read(file, exact)).toEqual(baseline);
|
||||
const under = new ReadFileTool(config, {
|
||||
...limits,
|
||||
maxMediaResultBytes: bytes - 1,
|
||||
});
|
||||
const refused = await read(file, under);
|
||||
expect(refused.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(JSON.stringify(refused)).not.toContain('inlineData');
|
||||
});
|
||||
|
||||
it.each(['png', 'jpeg', 'webp', 'gif'] as const)(
|
||||
'delivers actual %s bytes and refuses an oversized rendered/raw result',
|
||||
async (format) => {
|
||||
const source = await sharp({
|
||||
create: { width: 2, height: 3, channels: 3, background: '#246080' },
|
||||
})
|
||||
.toFormat(format)
|
||||
.toBuffer();
|
||||
const file = await put(`tiny.${format}`, source);
|
||||
const result = await read(file, boundedTool());
|
||||
const inline = normalizeParts(result.llmContent).find(
|
||||
(part) => part.inlineData,
|
||||
)?.inlineData;
|
||||
expect(inline?.data).toBeTruthy();
|
||||
const decoded = Buffer.from(inline!.data!, 'base64');
|
||||
expect(await sharp(decoded).metadata()).toMatchObject({
|
||||
width: 2,
|
||||
height: 3,
|
||||
});
|
||||
if (format === 'gif') expect(decoded).toEqual(source);
|
||||
const refused = await read(
|
||||
file,
|
||||
new ReadFileTool(makeConfig(), {
|
||||
...limits,
|
||||
maxInlineMediaBase64Bytes: inline!.data!.length - 1,
|
||||
}),
|
||||
);
|
||||
expect(refused.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(JSON.stringify(refused)).not.toContain('inlineData');
|
||||
},
|
||||
);
|
||||
|
||||
it.each(['text.txt', 'shape.svg', 'cells.ipynb', 'text.pdf'])(
|
||||
'does not apply media-result limits to %s',
|
||||
async (name) => {
|
||||
const contents = name.endsWith('ipynb')
|
||||
? notebookJson([codeCell('x'.repeat(1024), 1, 'output')])
|
||||
: name.endsWith('pdf')
|
||||
? '%PDF-1.7'
|
||||
: 'x'.repeat(1024);
|
||||
const file = await put(name, contents);
|
||||
pdfMocks.getPDFPageCount.mockResolvedValue(1);
|
||||
pdfMocks.extractPDFText.mockResolvedValue({
|
||||
success: true,
|
||||
text: 'x'.repeat(1024),
|
||||
});
|
||||
const result = await read(
|
||||
file,
|
||||
new ReadFileTool(
|
||||
makeConfig({ getEffectiveInputModalities: () => ({}) }),
|
||||
{ maxInlineMediaBase64Bytes: 1, maxMediaResultBytes: 1 },
|
||||
),
|
||||
);
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(Buffer.byteLength(JSON.stringify(result))).toBeGreaterThan(1);
|
||||
expect(JSON.stringify(result)).not.toContain('inlineData');
|
||||
},
|
||||
);
|
||||
|
||||
it.each([2, null])(
|
||||
'handles a rendered range beyond the document with page count %s',
|
||||
async (pageCount) => {
|
||||
const file = await put('scan.pdf', '%PDF-1.7');
|
||||
pdfMocks.getPDFPageCount.mockResolvedValue(pageCount);
|
||||
pdfMocks.renderPDFPagesToImages.mockResolvedValue({
|
||||
success: true,
|
||||
images: [
|
||||
{ data: 'YWJj', mimeType: 'image/jpeg' },
|
||||
{ data: 'YWJj', mimeType: 'image/jpeg' },
|
||||
],
|
||||
bytesTruncated: false,
|
||||
});
|
||||
const result = await read(
|
||||
{ file_path: file, pages: '1-10' },
|
||||
boundedTool({ getEffectiveInputModalities: () => ({ image: true }) }),
|
||||
);
|
||||
if (pageCount === null) {
|
||||
expect(result.error?.type).toBe(ToolErrorType.READ_CONTENT_FAILURE);
|
||||
} else {
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(
|
||||
normalizeParts(result.llmContent).filter((part) => part.inlineData),
|
||||
).toHaveLength(2);
|
||||
expect(JSON.stringify(result.llmContent)).toContain(
|
||||
'The document has 2 pages; rendered pages 1-2.',
|
||||
);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('does not probe the page count or add a notice for a complete range', async () => {
|
||||
const file = await put('scan.pdf', '%PDF-1.7');
|
||||
pdfMocks.renderPDFPagesToImages.mockResolvedValue({
|
||||
success: true,
|
||||
images: [
|
||||
{ data: 'YWJj', mimeType: 'image/jpeg' },
|
||||
{ data: 'YWJj', mimeType: 'image/jpeg' },
|
||||
],
|
||||
bytesTruncated: false,
|
||||
});
|
||||
const result = await read(
|
||||
{ file_path: file, pages: '1-2' },
|
||||
boundedTool({ getEffectiveInputModalities: () => ({ image: true }) }),
|
||||
);
|
||||
expect(result.error).toBeUndefined();
|
||||
expect(normalizeParts(result.llmContent)).toHaveLength(2);
|
||||
expect(JSON.stringify(result.llmContent)).not.toContain(
|
||||
'The document has',
|
||||
);
|
||||
expect(pdfMocks.getPDFPageCount).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects an explicit incomplete rendered range but announces an implicit whole-page prefix', async () => {
|
||||
const file = await put('scan.pdf', '%PDF-1.7');
|
||||
pdfMocks.getPDFPageCount.mockResolvedValue(2);
|
||||
pdfMocks.renderPDFPagesToImages.mockResolvedValue({
|
||||
success: true,
|
||||
images: [{ data: 'YWJj', mimeType: 'image/jpeg' }],
|
||||
bytesTruncated: true,
|
||||
});
|
||||
const mediaTool = boundedTool({
|
||||
getEffectiveInputModalities: () => ({ image: true }),
|
||||
});
|
||||
const prefix = await read(file, mediaTool);
|
||||
expect(prefix.error).toBeUndefined();
|
||||
expect(JSON.stringify(prefix.llmContent)).toContain(
|
||||
'later pages were omitted',
|
||||
);
|
||||
const explicit = await read({ file_path: file, pages: '1-2' }, mediaTool);
|
||||
expect(explicit.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(pdfMocks.renderPDFPagesToImages).toHaveBeenLastCalledWith(
|
||||
['linux', 'darwin'].includes(process.platform)
|
||||
? expect.objectContaining({ read: expect.any(Function) })
|
||||
: file,
|
||||
{
|
||||
firstPage: 1,
|
||||
lastPage: 2,
|
||||
signal: abortSignal,
|
||||
maxTotalBase64Bytes: limits.maxInlineMediaBase64Bytes,
|
||||
},
|
||||
);
|
||||
pdfMocks.renderPDFPagesToImages.mockResolvedValue({
|
||||
success: true,
|
||||
images: [{ data: 'YWJj', mimeType: 'image/jpeg' }],
|
||||
bytesTruncated: false,
|
||||
});
|
||||
const shortRange = await read(
|
||||
{ file_path: file, pages: '1-2' },
|
||||
mediaTool,
|
||||
);
|
||||
expect(shortRange.error?.type).toBe(ToolErrorType.READ_CONTENT_FAILURE);
|
||||
expect(JSON.stringify(shortRange)).not.toContain('inlineData');
|
||||
pdfMocks.renderPDFPagesToImages.mockResolvedValue({
|
||||
success: false,
|
||||
error: 'first page too large',
|
||||
tooLarge: true,
|
||||
});
|
||||
const tooLarge = await read(file, mediaTool);
|
||||
expect(tooLarge.error?.type).toBe(ToolErrorType.FILE_TOO_LARGE);
|
||||
expect(tooLarge.llmContent).toBe('first page too large');
|
||||
});
|
||||
|
||||
it.each(['page-count', 'text', 'render'])(
|
||||
'propagates abort after %s without another fallback',
|
||||
async (stage) => {
|
||||
const file = await put('scan.pdf', '%PDF-1.7');
|
||||
const controller = new AbortController();
|
||||
pdfMocks.getPDFPageCount.mockResolvedValue(1);
|
||||
const target =
|
||||
stage === 'page-count'
|
||||
? pdfMocks.getPDFPageCount
|
||||
: stage === 'text'
|
||||
? pdfMocks.extractPDFText
|
||||
: pdfMocks.renderPDFPagesToImages;
|
||||
target.mockImplementation(async () => {
|
||||
controller.abort();
|
||||
return stage === 'page-count'
|
||||
? null
|
||||
: { success: false, error: 'cancelled' };
|
||||
});
|
||||
await expect(
|
||||
read(
|
||||
file,
|
||||
boundedTool({
|
||||
getEffectiveInputModalities: () => ({ image: true }),
|
||||
}),
|
||||
controller.signal,
|
||||
),
|
||||
).rejects.toThrow(/abort/i);
|
||||
expect(pdfMocks.getPDFPageCount).toHaveBeenCalledWith(
|
||||
['linux', 'darwin'].includes(process.platform)
|
||||
? expect.objectContaining({ read: expect.any(Function) })
|
||||
: file,
|
||||
controller.signal,
|
||||
);
|
||||
if (stage !== 'page-count')
|
||||
expect(pdfMocks.extractPDFText).toHaveBeenCalledWith(
|
||||
pdfMocks.getPDFPageCount.mock.calls[0]![0],
|
||||
{
|
||||
signal: controller.signal,
|
||||
},
|
||||
);
|
||||
if (stage !== 'render')
|
||||
expect(pdfMocks.renderPDFPagesToImages).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe('build', () => {
|
||||
it.skipIf(!['linux', 'darwin'].includes(process.platform)).each([
|
||||
['source.txt', 'ORIGINAL', 'REPLACEMENT', 'ORIGINAL'],
|
||||
|
||||
@@ -17,6 +17,8 @@ import type {
|
||||
import { BaseDeclarativeTool, BaseToolInvocation, Kind } from './tools.js';
|
||||
import { ToolNames, ToolDisplayNames } from './tool-names.js';
|
||||
import { getCurrentToolCallSource } from '../code-mode/tool-call-runtime.js';
|
||||
import type { ReadFileMediaLimits } from '../utils/inline-media.js';
|
||||
import { ToolErrorType } from './tool-error.js';
|
||||
|
||||
import type { PartListUnion, FunctionDeclaration } from '@google/genai';
|
||||
import type { PermissionDecision } from '../permissions/types.js';
|
||||
@@ -93,6 +95,7 @@ class ReadFileToolInvocation extends BaseToolInvocation<
|
||||
constructor(
|
||||
private config: Config,
|
||||
params: ReadFileToolParams,
|
||||
private readonly mediaLimits?: ReadFileMediaLimits,
|
||||
) {
|
||||
super(params);
|
||||
}
|
||||
@@ -272,6 +275,7 @@ class ReadFileToolInvocation extends BaseToolInvocation<
|
||||
preserveUnsupportedImage: prepareForVisionBridge,
|
||||
preparePdfForVisionBridge: prepareForVisionBridge,
|
||||
signal,
|
||||
mediaLimits: this.mediaLimits,
|
||||
readSource: source,
|
||||
},
|
||||
);
|
||||
@@ -406,7 +410,7 @@ class ReadFileToolInvocation extends BaseToolInvocation<
|
||||
),
|
||||
);
|
||||
|
||||
return {
|
||||
const toolResult: ToolResult = {
|
||||
llmContent,
|
||||
// Self-managed paging may have capped this body under a threshold the
|
||||
// scheduler never sees; re-reading just to count the dropped part is
|
||||
@@ -417,6 +421,30 @@ class ReadFileToolInvocation extends BaseToolInvocation<
|
||||
: {}),
|
||||
returnDisplay: this.toToolResultDisplay(result),
|
||||
};
|
||||
if (this.mediaLimits) {
|
||||
// Reading caps media bytes; the transport budget also includes metadata.
|
||||
const media = normalizeParts(llmContent).filter(
|
||||
(part) => part.inlineData !== undefined,
|
||||
);
|
||||
if (
|
||||
media.length > 0 &&
|
||||
(media.reduce(
|
||||
(sum, part) => sum + (part.inlineData?.data?.length ?? 0),
|
||||
0,
|
||||
) > this.mediaLimits.maxInlineMediaBase64Bytes ||
|
||||
Buffer.byteLength(JSON.stringify(toolResult), 'utf8') >
|
||||
this.mediaLimits.maxMediaResultBytes)
|
||||
) {
|
||||
const message =
|
||||
"File exceeds the inline media result byte limit. For PDFs, use 'pages' with a narrower range; otherwise use a smaller file.";
|
||||
return {
|
||||
llmContent: message,
|
||||
returnDisplay: message,
|
||||
error: { message, type: ToolErrorType.FILE_TOO_LARGE },
|
||||
};
|
||||
}
|
||||
}
|
||||
return toolResult;
|
||||
}
|
||||
|
||||
private async transcribePdfCandidate(
|
||||
@@ -662,7 +690,10 @@ export class ReadFileTool extends BaseDeclarativeTool<
|
||||
return Number.POSITIVE_INFINITY;
|
||||
}
|
||||
|
||||
constructor(private config: Config) {
|
||||
constructor(
|
||||
private config: Config,
|
||||
private readonly mediaLimits?: ReadFileMediaLimits,
|
||||
) {
|
||||
super(
|
||||
ReadFileTool.Name,
|
||||
ToolDisplayNames.READ_FILE,
|
||||
@@ -785,6 +816,6 @@ export class ReadFileTool extends BaseDeclarativeTool<
|
||||
protected createInvocation(
|
||||
params: ReadFileToolParams,
|
||||
): ToolInvocation<ReadFileToolParams, ToolResult> {
|
||||
return new ReadFileToolInvocation(this.config, params);
|
||||
return new ReadFileToolInvocation(this.config, params, this.mediaLimits);
|
||||
}
|
||||
}
|
||||
@@ -754,14 +754,12 @@ describe('WebFetchTool', () => {
|
||||
it('should not return a late side-query success after the user aborts', async () => {
|
||||
// The final stream chunk can already be queued when the user aborts:
|
||||
// the side query then resolves successfully despite the abort.
|
||||
const { result } = await runAnswering((_options, controller) => {
|
||||
controller.abort();
|
||||
return Promise.resolve({ text: 'Late success' });
|
||||
});
|
||||
|
||||
expect(result.error?.type).toBe(ToolErrorType.WEB_FETCH_FALLBACK_FAILED);
|
||||
expect(result.llmContent).not.toContain('Late success');
|
||||
expect(result.llmContent).not.toContain('Test content');
|
||||
await expect(
|
||||
runAnswering((_options, controller) => {
|
||||
controller.abort();
|
||||
return Promise.resolve({ text: 'Late success' });
|
||||
}),
|
||||
).rejects.toMatchObject({ name: 'AbortError' });
|
||||
});
|
||||
|
||||
it.each([
|
||||
@@ -789,28 +787,24 @@ describe('WebFetchTool', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('should return a proper error result when aborted with a non-Error reason', async () => {
|
||||
it('should propagate cancellation with a non-Error reason', async () => {
|
||||
// Session.ts aborts with a plain string reason; throwIfAborted rethrows
|
||||
// it as-is, so the error path must not assume an Error instance.
|
||||
const { result } = await runAnswering((_options, controller) => {
|
||||
controller.abort('qwen:user-cancel');
|
||||
return Promise.resolve({ text: 'Late success' });
|
||||
});
|
||||
|
||||
expect(result.error?.type).toBe(ToolErrorType.WEB_FETCH_FALLBACK_FAILED);
|
||||
expect(result.llmContent).toContain('qwen:user-cancel');
|
||||
expect(result.llmContent).not.toContain('undefined');
|
||||
expect(result.llmContent).not.toContain('Late success');
|
||||
await expect(
|
||||
runAnswering((_options, controller) => {
|
||||
controller.abort('qwen:user-cancel');
|
||||
return Promise.resolve({ text: 'Late success' });
|
||||
}),
|
||||
).rejects.toBe('qwen:user-cancel');
|
||||
});
|
||||
|
||||
it('should propagate a user abort instead of fabricating a raw-content result', async () => {
|
||||
const { result } = await runAnswering((_options, controller) => {
|
||||
controller.abort();
|
||||
return Promise.reject(new Error('Request was aborted.'));
|
||||
});
|
||||
|
||||
expect(result.error?.type).toBe(ToolErrorType.WEB_FETCH_FALLBACK_FAILED);
|
||||
expect(result.llmContent).not.toContain('Test content');
|
||||
await expect(
|
||||
runAnswering((_options, controller) => {
|
||||
controller.abort();
|
||||
return Promise.reject(new Error('Request was aborted.'));
|
||||
}),
|
||||
).rejects.toThrow('Request was aborted.');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -932,6 +926,48 @@ describe('WebFetchTool', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('removes and refunds a PDF cancelled during text extraction', async () => {
|
||||
stubFetch(PDF);
|
||||
const controller = new AbortController();
|
||||
const reason = new DOMException('cancelled', 'AbortError');
|
||||
mockExtractPDFText.mockImplementationOnce(async () => {
|
||||
controller.abort(reason);
|
||||
throw reason;
|
||||
});
|
||||
await expect(
|
||||
run('https://example.com/doc.pdf', 'read it', {
|
||||
signal: controller.signal,
|
||||
}),
|
||||
).rejects.toBe(reason);
|
||||
expect(vi.mocked(mockConfig.trackToolResultBytes).mock.calls).toEqual([
|
||||
[pdfBytes.length],
|
||||
[-pdfBytes.length],
|
||||
]);
|
||||
expect(fs.readdirSync(toolResultsDir)).toEqual([]);
|
||||
expect(mockGenerateContent).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('retains the disk charge if cancellation cleanup cannot delete the PDF', async () => {
|
||||
stubFetch(PDF);
|
||||
const controller = new AbortController();
|
||||
const reason = new DOMException('cancelled', 'AbortError');
|
||||
mockExtractPDFText.mockImplementationOnce(async (file: string) => {
|
||||
fs.unlinkSync(file);
|
||||
fs.mkdirSync(file);
|
||||
controller.abort(reason);
|
||||
throw reason;
|
||||
});
|
||||
await expect(
|
||||
run('https://example.com/doc.pdf', 'read it', {
|
||||
signal: controller.signal,
|
||||
}),
|
||||
).rejects.toBe(reason);
|
||||
expect(vi.mocked(mockConfig.trackToolResultBytes).mock.calls).toEqual([
|
||||
[pdfBytes.length],
|
||||
]);
|
||||
expect(fs.readdirSync(toolResultsDir)).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('should persist PDFs, extract their text, and summarize it', async () => {
|
||||
mockExtractPDFText.mockResolvedValue({
|
||||
success: true,
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
import { truncateToolOutput } from './truncation.js';
|
||||
import { measureToolOutput, type ToolOutputSize } from './tool-output-size.js';
|
||||
import { LRUCache } from 'mnemonist';
|
||||
import { rm } from 'node:fs/promises';
|
||||
import type { Config } from '../config/config.js';
|
||||
import {
|
||||
fetchWithPolicy,
|
||||
@@ -516,13 +517,27 @@ Status: ${entry.status} ${entry.statusText || 'OK'} | Content-Type: ${entry.cont
|
||||
|
||||
content = '';
|
||||
if (sniff.extension === 'pdf' && persistedPath) {
|
||||
const pdfText = await extractPDFText(persistedPath, { signal });
|
||||
if (pdfText.success && pdfText.text.trim()) {
|
||||
content = pdfText.text;
|
||||
} else if (!pdfText.success) {
|
||||
this.debugLogger.debug(
|
||||
`[WebFetchTool] PDF text extraction failed: ${pdfText.error}`,
|
||||
);
|
||||
try {
|
||||
const pdfText = await extractPDFText(persistedPath, { signal });
|
||||
signal.throwIfAborted();
|
||||
if (pdfText.success && pdfText.text.trim()) {
|
||||
content = pdfText.text;
|
||||
} else if (!pdfText.success) {
|
||||
this.debugLogger.debug(
|
||||
`[WebFetchTool] PDF text extraction failed: ${pdfText.error}`,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
try {
|
||||
await rm(persistedPath, { force: true });
|
||||
this.config.trackToolResultBytes(-response.body.length);
|
||||
} catch (cleanupError) {
|
||||
this.debugLogger.error(
|
||||
'[WebFetchTool] Failed to remove unused PDF',
|
||||
cleanupError,
|
||||
);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
} else if (contentType.includes('text/html')) {
|
||||
@@ -748,6 +763,7 @@ ${entry.content}
|
||||
: {}),
|
||||
};
|
||||
} catch (e) {
|
||||
if (signal.aborted) throw e;
|
||||
// e may be a non-Error abort reason (throwIfAborted rethrows whatever
|
||||
// the aborting caller passed — e.g. a plain string, or null).
|
||||
const errorMessage = `Error during fetch for ${this.params.url}: ${getErrorMessage(e)}`;
|
||||
|
||||
@@ -989,6 +989,80 @@ describe('fileUtils', () => {
|
||||
expect(result.error).toBeUndefined();
|
||||
};
|
||||
|
||||
it('uses the effective getter without consulting generator configuration', async () => {
|
||||
const bytes = await sharp({
|
||||
create: { width: 1, height: 1, channels: 3, background: '#306090' },
|
||||
})
|
||||
.gif()
|
||||
.toBuffer();
|
||||
const generator = vi.fn(() => {
|
||||
throw new Error('must not initialize a generator');
|
||||
});
|
||||
const config = withConfig({
|
||||
getEffectiveInputModalities: () => ({ image: true, pdf: true }),
|
||||
getContentGeneratorConfig: generator,
|
||||
});
|
||||
const image = await readMedia(
|
||||
'effective.gif',
|
||||
bytes,
|
||||
'image/gif',
|
||||
config,
|
||||
);
|
||||
expectInline(image, bytes, 'image/gif', 'effective.gif');
|
||||
const pdf = await readMedia(
|
||||
'effective.pdf',
|
||||
'%PDF-1.7',
|
||||
'application/pdf',
|
||||
config,
|
||||
);
|
||||
expectInline(pdf, '%PDF-1.7', 'application/pdf', 'effective.pdf');
|
||||
expect(generator).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([{}, { image: false, pdf: false }])(
|
||||
'keeps effective %j authoritative over enabled legacy modalities',
|
||||
async (modalities) => {
|
||||
const config = withConfig({
|
||||
getEffectiveInputModalities: () => modalities,
|
||||
getContentGeneratorConfig: () => ({
|
||||
modalities: { image: true, pdf: true },
|
||||
}),
|
||||
});
|
||||
const image = await readFakePng(config);
|
||||
expect(image.llmContent).toContain('Unsupported image');
|
||||
expect(JSON.stringify(image.llmContent)).not.toContain('inlineData');
|
||||
const pdf = await readMedia(
|
||||
'disabled.pdf',
|
||||
'%PDF-1.7',
|
||||
'application/pdf',
|
||||
config,
|
||||
);
|
||||
expect(pdf.errorType).toBe(ToolErrorType.READ_CONTENT_FAILURE);
|
||||
expect(JSON.stringify(pdf.llmContent)).not.toContain('inlineData');
|
||||
},
|
||||
);
|
||||
|
||||
it('retains legacy partial Config media reads when the effective getter is absent', async () => {
|
||||
const bytes = await sharp({
|
||||
create: { width: 1, height: 1, channels: 3, background: '#306090' },
|
||||
})
|
||||
.gif()
|
||||
.toBuffer();
|
||||
const config = withModalities({ image: true, pdf: true });
|
||||
expectInline(
|
||||
await readMedia('legacy.gif', bytes, 'image/gif', config),
|
||||
bytes,
|
||||
'image/gif',
|
||||
'legacy.gif',
|
||||
);
|
||||
expectInline(
|
||||
await readMedia('legacy.pdf', '%PDF-1.7', 'application/pdf', config),
|
||||
'%PDF-1.7',
|
||||
'application/pdf',
|
||||
'legacy.pdf',
|
||||
);
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
// Default: renderer unavailable, so PDF reads fall back to the text path
|
||||
// unless a test opts into rendering. Set after the global resetAllMocks.
|
||||
@@ -1725,6 +1799,7 @@ describe('fileUtils', () => {
|
||||
expect(mockRender).toHaveBeenCalledWith(testPdfFilePath, {
|
||||
firstPage,
|
||||
lastPage,
|
||||
signal: undefined,
|
||||
});
|
||||
const hasText = (parts: MediaPart[], pattern: RegExp) =>
|
||||
parts.some((p) => typeof p.text === 'string' && pattern.test(p.text));
|
||||
|
||||
@@ -25,6 +25,11 @@ import { createDebugLogger } from './debugLogger.js';
|
||||
import { getErrorMessage, isAbortError, isNodeError } from './errors.js';
|
||||
import type { InputModalities } from '../core/contentGenerator.js';
|
||||
import { detectEncodingFromBuffer } from './systemEncoding.js';
|
||||
import {
|
||||
base64ByteLength,
|
||||
readFileWithinBase64Limit,
|
||||
type ReadFileMediaLimits,
|
||||
} from './inline-media.js';
|
||||
import type { PDFRenderedImage } from './pdf.js';
|
||||
import {
|
||||
buildLargePDFGuidance,
|
||||
@@ -1129,6 +1134,7 @@ export interface ProcessSingleFileContentOptions {
|
||||
*/
|
||||
preparePdfForVisionBridge?: boolean;
|
||||
signal?: AbortSignal;
|
||||
mediaLimits?: ReadFileMediaLimits;
|
||||
/**
|
||||
* Large full-PDF text fallback returns a tool error by default. `@`-attached
|
||||
* PDFs use `reference` so the model gets guidance without a failed read.
|
||||
@@ -1216,6 +1222,7 @@ export async function processSingleFileContent(
|
||||
preserveUnsupportedImage = false,
|
||||
preparePdfForVisionBridge = false,
|
||||
signal,
|
||||
mediaLimits,
|
||||
largePdfBehavior = 'error',
|
||||
displayPath = filePath,
|
||||
} = options;
|
||||
@@ -1233,11 +1240,36 @@ export async function processSingleFileContent(
|
||||
signal,
|
||||
)
|
||||
: fs.promises.readFile(contentPath, signal ? { signal } : undefined));
|
||||
const readMediaBytes = async () => {
|
||||
if (!mediaLimits) return readBytes();
|
||||
if (!descriptor) {
|
||||
return readFileWithinBase64Limit(
|
||||
contentPath,
|
||||
mediaLimits.maxInlineMediaBase64Bytes,
|
||||
signal,
|
||||
);
|
||||
}
|
||||
if (
|
||||
base64ByteLength(descriptor.stats.size) >
|
||||
mediaLimits.maxInlineMediaBase64Bytes
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
return readBytes();
|
||||
};
|
||||
const relativePathForDisplay = (
|
||||
path.isAbsolute(displayPath)
|
||||
? path.relative(rootDirectory, displayPath)
|
||||
: displayPath
|
||||
).replace(/\\/g, '/');
|
||||
const mediaTooLarge = (
|
||||
message = "File exceeds the inline media byte limit. For PDFs, use 'pages' with a narrower range; otherwise use a smaller file.",
|
||||
): ProcessedFileReadResult => ({
|
||||
llmContent: message,
|
||||
returnDisplay: message,
|
||||
error: message,
|
||||
errorType: ToolErrorType.FILE_TOO_LARGE,
|
||||
});
|
||||
try {
|
||||
signal?.throwIfAborted();
|
||||
let stats: import('node:fs').Stats;
|
||||
@@ -1309,10 +1341,11 @@ export async function processSingleFileContent(
|
||||
const shouldRenderImageOverview =
|
||||
fileType === 'image' && CANONICAL_IMAGE_MIME_TYPES.has(mediaMimeType);
|
||||
const displayName = path.basename(displayPath);
|
||||
// Use optional call (`?.()`) so mock Configs that don't implement
|
||||
// getContentGeneratorConfig still work for non-media file types.
|
||||
// An empty effective view is authoritative; only legacy mocks fall back.
|
||||
const modalities: InputModalities =
|
||||
config.getContentGeneratorConfig?.()?.modalities ?? {};
|
||||
typeof config.getEffectiveInputModalities === 'function'
|
||||
? config.getEffectiveInputModalities()
|
||||
: (config.getContentGeneratorConfig?.()?.modalities ?? {});
|
||||
|
||||
// Vision-capable main model, explicit read (not `@`-reference): when text
|
||||
// extraction overflows or fails, render pages to images for the model
|
||||
@@ -1401,6 +1434,7 @@ export async function processSingleFileContent(
|
||||
}
|
||||
if (willExtractPdfText && !pageRange) {
|
||||
pdfPageCount = await getPDFPageCount(pdfSource, signal);
|
||||
signal?.throwIfAborted();
|
||||
const requirement = shouldRequirePDFPageRange(pdfPageCount, stats.size);
|
||||
// A vision render can hold up to PDF_MAX_PAGES_PER_READ pages, so only
|
||||
// require an explicit range past that ceiling; the text path keeps the
|
||||
@@ -1413,10 +1447,11 @@ export async function processSingleFileContent(
|
||||
`PDF full-text fallback gate: file=${relativePathForDisplay}, sizeMB=${fileSizeInMB.toFixed(2)}, pageCount=${pdfPageCount ?? 'unknown'}, required=${requirement.required}, rangeRequired=${rangeRequired}, effectivePageCount=${requirement.effectivePageCount}, hadPdfInfo=${requirement.hadPdfInfo}, behavior=${largePdfBehavior}`,
|
||||
);
|
||||
if (rangeRequired) {
|
||||
if (
|
||||
largePdfBehavior === 'error' &&
|
||||
!(await isPdftotextAvailable(descriptor?.fileHandle, signal))
|
||||
) {
|
||||
const textAvailable =
|
||||
largePdfBehavior !== 'error' ||
|
||||
(await isPdftotextAvailable(descriptor?.fileHandle, signal));
|
||||
signal?.throwIfAborted();
|
||||
if (largePdfBehavior === 'error' && !textAvailable) {
|
||||
return {
|
||||
llmContent: `[Cannot extract text from PDF: "${displayName}". ${PDF_TEXT_EXTRACTION_UNAVAILABLE_MESSAGE}]`,
|
||||
returnDisplay: `Failed to read pdf: ${relativePathForDisplay}`,
|
||||
@@ -1750,6 +1785,14 @@ export async function processSingleFileContent(
|
||||
viewSignal,
|
||||
)
|
||||
: await renderImageOverview(contentPath, viewSignal);
|
||||
signal?.throwIfAborted();
|
||||
if (
|
||||
mediaLimits &&
|
||||
base64ByteLength(view.bytes.length) >
|
||||
mediaLimits.maxInlineMediaBase64Bytes
|
||||
) {
|
||||
return mediaTooLarge();
|
||||
}
|
||||
const registry = config.getToolRegistry?.();
|
||||
const codeModeOnly = config.getCodeModeOnly?.();
|
||||
const declaredTools = new Set(
|
||||
@@ -1866,7 +1909,9 @@ export async function processSingleFileContent(
|
||||
returnDisplay: `Omitted unsupported image format: ${relativePathForDisplay} (${mediaMimeType})`,
|
||||
};
|
||||
}
|
||||
const contentBuffer = await readBytes();
|
||||
const contentBuffer = await readMediaBytes();
|
||||
if (contentBuffer === undefined) return mediaTooLarge();
|
||||
signal?.throwIfAborted();
|
||||
if (mediaMimeType === 'image/gif') {
|
||||
// GIFs skip the overview renderer, so nothing has validated the
|
||||
// bytes yet: a corrupt or mislabeled .gif would reach the provider
|
||||
@@ -1962,7 +2007,9 @@ export async function processSingleFileContent(
|
||||
// Otherwise, fall back to pdftotext for text extraction.
|
||||
if (!pageRange && modalities.pdf) {
|
||||
// Model supports PDF natively — send as base64
|
||||
const contentBuffer = await readBytes();
|
||||
const contentBuffer = await readMediaBytes();
|
||||
if (contentBuffer === undefined) return mediaTooLarge();
|
||||
signal?.throwIfAborted();
|
||||
const base64Data = contentBuffer.toString('base64');
|
||||
const base64SizeInMB = base64Data.length / (1024 * 1024);
|
||||
if (base64SizeInMB > 9.9) {
|
||||
@@ -2036,9 +2083,53 @@ export async function processSingleFileContent(
|
||||
lastPage: PDF_MAX_PAGES_PER_READ,
|
||||
}),
|
||||
signal,
|
||||
...(mediaLimits && {
|
||||
maxTotalBase64Bytes: mediaLimits.maxInlineMediaBase64Bytes,
|
||||
}),
|
||||
});
|
||||
signal?.throwIfAborted();
|
||||
if (
|
||||
(!render.success && render.tooLarge) ||
|
||||
(mediaLimits &&
|
||||
pageRange &&
|
||||
render.success &&
|
||||
render.bytesTruncated)
|
||||
) {
|
||||
return mediaTooLarge(!render.success ? render.error : undefined);
|
||||
}
|
||||
if (render.success && render.images.length > 0) {
|
||||
if (
|
||||
mediaLimits &&
|
||||
pageRange &&
|
||||
render.images.length !== pageRange.lastPage - startPage + 1
|
||||
) {
|
||||
pdfPageCount = await getPDFPageCount(pdfSource, signal);
|
||||
signal?.throwIfAborted();
|
||||
const lastPage = Math.min(
|
||||
pageRange.lastPage,
|
||||
pdfPageCount ?? pageRange.lastPage,
|
||||
);
|
||||
if (render.images.length !== lastPage - startPage + 1) {
|
||||
const message =
|
||||
'The requested PDF page range was not fully rendered. Use a range within the document.';
|
||||
return {
|
||||
llmContent: message,
|
||||
returnDisplay: message,
|
||||
error: message,
|
||||
errorType: ToolErrorType.READ_CONTENT_FAILURE,
|
||||
};
|
||||
}
|
||||
}
|
||||
const parts = toImageParts(render.images, startPage);
|
||||
if (
|
||||
pageRange &&
|
||||
pdfPageCount != null &&
|
||||
pageRange.lastPage > pdfPageCount
|
||||
) {
|
||||
parts.push({
|
||||
text: `[The document has ${pdfPageCount} pages; rendered pages ${startPage}-${startPage + render.images.length - 1}.]`,
|
||||
});
|
||||
}
|
||||
// Never drop pages silently. Two ways a no-page-range read can be
|
||||
// partial: the byte cap kicked in, or the render filled the page
|
||||
// ceiling (the page count was unknown/underestimated upstream, so
|
||||
@@ -2086,6 +2177,7 @@ export async function processSingleFileContent(
|
||||
if (renderForBridge && (!pdfResult.success || singlePageTextOverflow)) {
|
||||
if (pageRange && pdfPageCount === undefined) {
|
||||
pdfPageCount = await getPDFPageCount(pdfSource, signal);
|
||||
signal?.throwIfAborted();
|
||||
}
|
||||
const firstPage = pageRange?.firstPage ?? 1;
|
||||
const requestedLastPage =
|
||||
@@ -2106,11 +2198,24 @@ export async function processSingleFileContent(
|
||||
firstPage,
|
||||
lastPage,
|
||||
signal,
|
||||
...(mediaLimits && {
|
||||
maxTotalBase64Bytes: mediaLimits.maxInlineMediaBase64Bytes,
|
||||
}),
|
||||
})
|
||||
: {
|
||||
success: false as const,
|
||||
error: 'The requested page range is outside the PDF.',
|
||||
};
|
||||
signal?.throwIfAborted();
|
||||
if (
|
||||
(!render.success && 'tooLarge' in render && render.tooLarge) ||
|
||||
(mediaLimits &&
|
||||
pageRange &&
|
||||
render.success &&
|
||||
render.bytesTruncated)
|
||||
) {
|
||||
return mediaTooLarge(!render.success ? render.error : undefined);
|
||||
}
|
||||
if (render.success && render.images.length > 0) {
|
||||
const parts = toImageParts(render.images, firstPage);
|
||||
const renderedLastPage = firstPage + render.images.length - 1;
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
|
||||
import * as fs from 'node:fs/promises';
|
||||
import { constants } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { base64ByteLength, readFileWithinBase64Limit } from './inline-media.js';
|
||||
|
||||
vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>();
|
||||
return { ...actual, open: vi.fn(actual.open) };
|
||||
});
|
||||
|
||||
let directory: string;
|
||||
beforeEach(async () => {
|
||||
directory = await fs.mkdtemp(join(tmpdir(), 'inline-media-test-'));
|
||||
});
|
||||
afterEach(async () => {
|
||||
vi.restoreAllMocks();
|
||||
await fs.rm(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it.each([0, 1, 2, 3, 4, 5, 6])(
|
||||
'accounts for base64 padding at %i bytes',
|
||||
async (size) => {
|
||||
const bytes = Buffer.alloc(size, 42);
|
||||
const file = join(directory, 'media');
|
||||
await fs.writeFile(file, bytes);
|
||||
const limit = bytes.toString('base64').length;
|
||||
expect(base64ByteLength(size)).toBe(limit);
|
||||
expect(await readFileWithinBase64Limit(file, limit)).toEqual(bytes);
|
||||
await fs.appendFile(file, Buffer.alloc(3 - (size % 3) + 1));
|
||||
expect(await readFileWithinBase64Limit(file, limit)).toBeUndefined();
|
||||
},
|
||||
);
|
||||
|
||||
it('reads at most the decoded ceiling plus one byte even when the file grows', async () => {
|
||||
const file = join(directory, 'media');
|
||||
await fs.writeFile(file, 'abc');
|
||||
const handle = await fs.open(file, 'r');
|
||||
vi.spyOn(fs, 'open').mockResolvedValueOnce(handle);
|
||||
const originalRead = handle.read.bind(handle);
|
||||
let total = 0;
|
||||
let grown = false;
|
||||
vi.spyOn(handle, 'read').mockImplementation(async (options) => {
|
||||
if (!grown) {
|
||||
grown = true;
|
||||
await fs.appendFile(file, Buffer.alloc(256 * 1024));
|
||||
}
|
||||
const result = await originalRead(options);
|
||||
total += result.bytesRead;
|
||||
return result;
|
||||
});
|
||||
const close = vi.spyOn(handle, 'close');
|
||||
expect(await readFileWithinBase64Limit(file, 8)).toBeUndefined();
|
||||
expect(total).toBe(7);
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('closes the owned handle on abort during a read', async () => {
|
||||
const file = join(directory, 'media');
|
||||
await fs.writeFile(file, 'abc');
|
||||
const handle = await fs.open(file, 'r');
|
||||
vi.spyOn(fs, 'open').mockResolvedValueOnce(handle);
|
||||
const controller = new AbortController();
|
||||
const originalRead = handle.read.bind(handle);
|
||||
vi.spyOn(handle, 'read').mockImplementation(async (options) => {
|
||||
const result = await originalRead(options);
|
||||
controller.abort();
|
||||
return result;
|
||||
});
|
||||
const close = vi.spyOn(handle, 'close');
|
||||
await expect(
|
||||
readFileWithinBase64Limit(file, 4, controller.signal),
|
||||
).rejects.toThrow(/abort/i);
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('closes the owned handle on read failure and on success', async () => {
|
||||
const file = join(directory, 'media');
|
||||
await fs.writeFile(file, 'abc');
|
||||
for (const fail of [false, true]) {
|
||||
const handle = await fs.open(file, 'r');
|
||||
vi.spyOn(fs, 'open').mockResolvedValueOnce(handle);
|
||||
const close = vi.spyOn(handle, 'close');
|
||||
if (fail) {
|
||||
vi.spyOn(handle, 'read').mockRejectedValueOnce(new Error('read failure'));
|
||||
await expect(readFileWithinBase64Limit(file, 4)).rejects.toThrow(
|
||||
'read failure',
|
||||
);
|
||||
} else {
|
||||
expect(await readFileWithinBase64Limit(file, 4)).toEqual(
|
||||
Buffer.from('abc'),
|
||||
);
|
||||
}
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
}
|
||||
});
|
||||
|
||||
it('opens nonblocking and rejects a non-regular handle before reading', async () => {
|
||||
const file = join(directory, 'media');
|
||||
await fs.writeFile(file, 'abc');
|
||||
const handle = await fs.open(file, 'r');
|
||||
vi.spyOn(fs, 'open').mockClear().mockResolvedValueOnce(handle);
|
||||
vi.spyOn(handle, 'stat').mockResolvedValueOnce({
|
||||
isFile: () => false,
|
||||
} as Awaited<ReturnType<typeof handle.stat>>);
|
||||
const read = vi.spyOn(handle, 'read');
|
||||
const close = vi.spyOn(handle, 'close');
|
||||
await expect(readFileWithinBase64Limit(file, 4)).rejects.toThrow(
|
||||
'regular file',
|
||||
);
|
||||
expect(fs.open).toHaveBeenCalledWith(
|
||||
file,
|
||||
constants.O_RDONLY | constants.O_NONBLOCK,
|
||||
);
|
||||
expect(read).not.toHaveBeenCalled();
|
||||
expect(close).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it.each(['abort', 'read'])(
|
||||
'preserves the %s error when closing also fails',
|
||||
async (kind) => {
|
||||
const file = join(directory, 'media');
|
||||
await fs.writeFile(file, 'abc');
|
||||
const handle = await fs.open(file, 'r');
|
||||
vi.spyOn(fs, 'open').mockResolvedValueOnce(handle);
|
||||
const controller = new AbortController();
|
||||
const failure =
|
||||
kind === 'abort'
|
||||
? new DOMException('cancelled', 'AbortError')
|
||||
: new Error('read failure');
|
||||
vi.spyOn(handle, 'read').mockImplementationOnce(async () => {
|
||||
if (kind === 'abort') controller.abort(failure);
|
||||
throw failure;
|
||||
});
|
||||
const close = handle.close.bind(handle);
|
||||
vi.spyOn(handle, 'close').mockImplementationOnce(async () => {
|
||||
await close();
|
||||
throw new Error('close failure');
|
||||
});
|
||||
await expect(
|
||||
readFileWithinBase64Limit(file, 4, controller.signal),
|
||||
).rejects.toBe(failure);
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,54 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { constants } from 'node:fs';
|
||||
import { open } from 'node:fs/promises';
|
||||
|
||||
export interface ReadFileMediaLimits {
|
||||
readonly maxInlineMediaBase64Bytes: number;
|
||||
readonly maxMediaResultBytes: number;
|
||||
}
|
||||
|
||||
export function base64ByteLength(byteLength: number): number {
|
||||
return 4 * Math.ceil(byteLength / 3);
|
||||
}
|
||||
|
||||
export async function readFileWithinBase64Limit(
|
||||
filePath: string,
|
||||
maxBase64Bytes: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Buffer | undefined> {
|
||||
signal?.throwIfAborted();
|
||||
// Nonblocking open lets the handle check reject a replaced FIFO as well.
|
||||
const handle = await open(
|
||||
filePath,
|
||||
constants.O_RDONLY | constants.O_NONBLOCK,
|
||||
);
|
||||
try {
|
||||
signal?.throwIfAborted();
|
||||
const stats = await handle.stat();
|
||||
if (!stats.isFile()) {
|
||||
throw new Error('Inline media must be a regular file.');
|
||||
}
|
||||
const maxBytes = Math.floor(maxBase64Bytes / 4) * 3;
|
||||
if (stats.size > maxBytes) return undefined;
|
||||
const chunks: Buffer[] = [];
|
||||
let total = 0;
|
||||
while (total <= maxBytes) {
|
||||
signal?.throwIfAborted();
|
||||
const chunk = Buffer.alloc(Math.min(64 * 1024, maxBytes + 1 - total));
|
||||
const { bytesRead } = await handle.read({ buffer: chunk });
|
||||
signal?.throwIfAborted();
|
||||
if (bytesRead === 0) return Buffer.concat(chunks, total);
|
||||
total += bytesRead;
|
||||
if (total > maxBytes) return undefined;
|
||||
chunks.push(chunk.subarray(0, bytesRead));
|
||||
}
|
||||
return undefined;
|
||||
} finally {
|
||||
await handle.close().catch(() => {});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,236 @@
|
||||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { execFile } from 'node:child_process';
|
||||
import * as fs from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import sharp from 'sharp';
|
||||
import type { Config } from '../config/config.js';
|
||||
import { processSingleFileContent } from './fileUtils.js';
|
||||
import {
|
||||
extractPDFText,
|
||||
getPDFPageCount,
|
||||
isPdftoppmAvailable,
|
||||
isPdftotextAvailable,
|
||||
renderPDFPagesToImages,
|
||||
resetPdftoppmCache,
|
||||
resetPdftotextCache,
|
||||
} from './pdf.js';
|
||||
|
||||
vi.mock('node:child_process', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:child_process')>();
|
||||
return { ...actual, execFile: vi.fn(actual.execFile) };
|
||||
});
|
||||
vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('node:fs/promises')>();
|
||||
return { ...actual, mkdtemp: vi.fn(actual.mkdtemp) };
|
||||
});
|
||||
|
||||
function pdfFixture(text = ''): Buffer {
|
||||
const stream = text
|
||||
? `BT /F1 12 Tf 20 100 Td (${text}) Tj ET`
|
||||
: '0.2 0.4 0.6 rg 10 10 100 100 re f';
|
||||
const objects = [
|
||||
'<< /Type /Catalog /Pages 2 0 R >>',
|
||||
'<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>',
|
||||
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << /Font << /F1 6 0 R >> >> /Contents 5 0 R >>',
|
||||
'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Resources << /Font << /F1 6 0 R >> >> /Contents 5 0 R >>',
|
||||
`<< /Length ${Buffer.byteLength(stream)} >>\nstream\n${stream}\nendstream`,
|
||||
'<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>',
|
||||
];
|
||||
let pdf = '%PDF-1.4\n';
|
||||
const offsets = [0];
|
||||
for (const [index, object] of objects.entries()) {
|
||||
offsets.push(Buffer.byteLength(pdf));
|
||||
pdf += `${index + 1} 0 obj\n${object}\nendobj\n`;
|
||||
}
|
||||
const xref = Buffer.byteLength(pdf);
|
||||
pdf += `xref\n0 ${offsets.length}\n0000000000 65535 f \n`;
|
||||
pdf += offsets
|
||||
.slice(1)
|
||||
.map((offset) => `${String(offset).padStart(10, '0')} 00000 n \n`)
|
||||
.join('');
|
||||
pdf += `trailer\n<< /Size ${offsets.length} /Root 1 0 R >>\nstartxref\n${xref}\n%%EOF\n`;
|
||||
return Buffer.from(pdf);
|
||||
}
|
||||
|
||||
describe('real Poppler', () => {
|
||||
let directory: string;
|
||||
let available: boolean;
|
||||
beforeEach(async () => {
|
||||
const actual =
|
||||
await vi.importActual<typeof import('node:child_process')>(
|
||||
'node:child_process',
|
||||
);
|
||||
vi.mocked(execFile).mockReset().mockImplementation(actual.execFile);
|
||||
vi.mocked(fs.mkdtemp).mockClear();
|
||||
resetPdftoppmCache();
|
||||
resetPdftotextCache();
|
||||
directory = await fs.mkdtemp(join(tmpdir(), 'pdf-poppler-test-'));
|
||||
available = (await isPdftoppmAvailable()) && (await isPdftotextAvailable());
|
||||
});
|
||||
afterEach(async () => {
|
||||
vi.restoreAllMocks();
|
||||
await fs.rm(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('extracts actual text and renders decodable JPEG pages within the aggregate ceiling', async ({
|
||||
skip,
|
||||
}) => {
|
||||
if (!available) return skip();
|
||||
const textFile = join(directory, 'text.pdf');
|
||||
const pdf = pdfFixture('real PDF text');
|
||||
await fs.writeFile(textFile, pdf);
|
||||
const native = await processSingleFileContent(
|
||||
textFile,
|
||||
{
|
||||
getTargetDir: () => directory,
|
||||
getEffectiveInputModalities: () => ({ pdf: true }),
|
||||
} as Config,
|
||||
{
|
||||
mediaLimits: {
|
||||
maxInlineMediaBase64Bytes: 768 * 1024,
|
||||
maxMediaResultBytes: 896 * 1024,
|
||||
},
|
||||
},
|
||||
);
|
||||
expect(native.llmContent).toEqual({
|
||||
inlineData: {
|
||||
data: pdf.toString('base64'),
|
||||
mimeType: 'application/pdf',
|
||||
displayName: 'text.pdf',
|
||||
},
|
||||
});
|
||||
expect(await getPDFPageCount(textFile)).toBe(2);
|
||||
const text = await extractPDFText(textFile, { firstPage: 2, lastPage: 2 });
|
||||
expect(text.success && text.text).toContain('real PDF text');
|
||||
const scan = join(directory, 'scan.pdf');
|
||||
await fs.writeFile(scan, pdfFixture());
|
||||
const extracted = await extractPDFText(scan);
|
||||
expect(extracted.success).toBe(false);
|
||||
const render = await renderPDFPagesToImages(scan, {
|
||||
firstPage: 1,
|
||||
lastPage: 2,
|
||||
maxTotalBase64Bytes: 768 * 1024,
|
||||
});
|
||||
expect(render.success).toBe(true);
|
||||
if (!render.success) throw new Error(render.error);
|
||||
expect(render.images).toHaveLength(2);
|
||||
for (const image of render.images) {
|
||||
expect(image.mimeType).toBe('image/jpeg');
|
||||
expect(
|
||||
await sharp(Buffer.from(image.data, 'base64')).metadata(),
|
||||
).toMatchObject({ format: 'jpeg', width: 1600, height: 1600 });
|
||||
}
|
||||
const firstPageBytes = render.images[0]!.data.length;
|
||||
const prefix = await renderPDFPagesToImages(scan, {
|
||||
firstPage: 1,
|
||||
lastPage: 2,
|
||||
maxTotalBase64Bytes: firstPageBytes,
|
||||
});
|
||||
expect(prefix).toEqual({
|
||||
success: true,
|
||||
images: [render.images[0]],
|
||||
bytesTruncated: true,
|
||||
});
|
||||
expect(
|
||||
await renderPDFPagesToImages(scan, {
|
||||
firstPage: 1,
|
||||
lastPage: 2,
|
||||
maxTotalBase64Bytes: firstPageBytes - 1,
|
||||
}),
|
||||
).toMatchObject({ success: false, tooLarge: true });
|
||||
const renderDirectories = await Promise.all(
|
||||
vi
|
||||
.mocked(fs.mkdtemp)
|
||||
.mock.results.map(({ value }) => value as Promise<string>),
|
||||
);
|
||||
for (const path of renderDirectories.filter((path) =>
|
||||
path.includes('pdf-render-'),
|
||||
)) {
|
||||
await expect(fs.access(path)).rejects.toMatchObject({ code: 'ENOENT' });
|
||||
}
|
||||
});
|
||||
|
||||
for (const command of ['pdfinfo', 'pdftotext', 'pdftoppm'] as const) {
|
||||
it(`waits for real ${command} exit on cancellation and completes renderer cleanup`, async ({
|
||||
skip,
|
||||
}) => {
|
||||
if (!available || process.platform === 'win32') return skip();
|
||||
const file = join(directory, 'cancel.pdf');
|
||||
await fs.writeFile(file, pdfFixture('cancel this extraction'));
|
||||
const actual =
|
||||
await vi.importActual<typeof import('node:child_process')>(
|
||||
'node:child_process',
|
||||
);
|
||||
const controller = new AbortController();
|
||||
let child: ReturnType<typeof execFile> | undefined;
|
||||
let exited = false;
|
||||
let started!: () => void;
|
||||
const spawned = new Promise<void>((resolve) => {
|
||||
started = resolve;
|
||||
});
|
||||
vi.mocked(execFile).mockImplementation(
|
||||
(...args: Parameters<typeof execFile>) => {
|
||||
const process = actual.execFile(...args);
|
||||
if (args[0] === command) {
|
||||
child = process;
|
||||
process.once('spawn', () => {
|
||||
process.kill('SIGSTOP');
|
||||
started();
|
||||
});
|
||||
process.once('exit', () => {
|
||||
exited = true;
|
||||
});
|
||||
}
|
||||
return process;
|
||||
},
|
||||
);
|
||||
const operation =
|
||||
command === 'pdfinfo'
|
||||
? getPDFPageCount(file, controller.signal)
|
||||
: command === 'pdftotext'
|
||||
? extractPDFText(file, { signal: controller.signal })
|
||||
: renderPDFPagesToImages(file, {
|
||||
signal: controller.signal,
|
||||
maxTotalBase64Bytes: 768 * 1024,
|
||||
});
|
||||
const outcome = operation.then(
|
||||
() => undefined,
|
||||
(error: unknown) => error,
|
||||
);
|
||||
try {
|
||||
await spawned;
|
||||
expect(child?.exitCode).toBeNull();
|
||||
expect(child?.signalCode).toBeNull();
|
||||
controller.abort();
|
||||
} finally {
|
||||
child?.kill('SIGCONT');
|
||||
await outcome;
|
||||
}
|
||||
expect(await outcome).toMatchObject({ name: 'AbortError' });
|
||||
expect(exited).toBe(true);
|
||||
if (command === 'pdftoppm') {
|
||||
const directories = await Promise.all(
|
||||
vi
|
||||
.mocked(fs.mkdtemp)
|
||||
.mock.results.map(({ value }) => value as Promise<string>),
|
||||
);
|
||||
const renderDirectories = directories.filter((path) =>
|
||||
path.includes('pdf-render-'),
|
||||
);
|
||||
expect(renderDirectories).toHaveLength(1);
|
||||
for (const path of renderDirectories) {
|
||||
await expect(fs.access(path)).rejects.toMatchObject({
|
||||
code: 'ENOENT',
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
@@ -5,6 +5,7 @@
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { EventEmitter } from 'node:events';
|
||||
import {
|
||||
parsePDFPageRange,
|
||||
isPdftotextAvailable,
|
||||
@@ -51,6 +52,12 @@ vi.mock('./pdf-command.js', async (importOriginal) => {
|
||||
});
|
||||
|
||||
import { execFile } from 'node:child_process';
|
||||
import { readFileWithinBase64Limit } from './inline-media.js';
|
||||
|
||||
vi.mock('./inline-media.js', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('./inline-media.js')>();
|
||||
return { ...actual, readFileWithinBase64Limit: vi.fn() };
|
||||
});
|
||||
import type { FileHandle } from 'node:fs/promises';
|
||||
import { lstat, open, readdir, readFile, rm } from 'node:fs/promises';
|
||||
import { execPDFCommandFromHandle } from './pdf-command.js';
|
||||
@@ -119,6 +126,142 @@ describe('pdf utilities', () => {
|
||||
resetPdftoppmCache();
|
||||
});
|
||||
|
||||
it.each(['pdfinfo', 'pdftotext', 'pdftoppm'])(
|
||||
'does not settle cancelled %s at its abort callback before child close',
|
||||
async (command) => {
|
||||
if (command !== 'pdfinfo') mockExecResult();
|
||||
const child = Object.assign(new EventEmitter(), { pid: 123 });
|
||||
let callback!: ExecCallback;
|
||||
mockExecFile.mockImplementationOnce(
|
||||
(_command: unknown, _args: unknown, _options: unknown, cb: unknown) => {
|
||||
callback = cb as ExecCallback;
|
||||
return child as ReturnType<typeof execFile>;
|
||||
},
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const operation =
|
||||
command === 'pdfinfo'
|
||||
? getPDFPageCount('/test.pdf', controller.signal)
|
||||
: command === 'pdftotext'
|
||||
? extractPDFText('/test.pdf', { signal: controller.signal })
|
||||
: renderPDFPagesToImages('/test.pdf', {
|
||||
signal: controller.signal,
|
||||
});
|
||||
await vi.waitFor(() => expect(callback).toBeDefined());
|
||||
let settled = false;
|
||||
const outcome = operation.then(
|
||||
() => {
|
||||
settled = true;
|
||||
return undefined;
|
||||
},
|
||||
(error: unknown) => {
|
||||
settled = true;
|
||||
return error;
|
||||
},
|
||||
);
|
||||
controller.abort();
|
||||
callback(errorWith('cancelled', { name: 'AbortError' }), '', '');
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
expect(settled).toBe(false);
|
||||
if (command === 'pdftoppm') {
|
||||
const { rm } = await import('node:fs/promises');
|
||||
expect(rm).not.toHaveBeenCalled();
|
||||
}
|
||||
child.emit('close');
|
||||
expect(await outcome).toMatchObject({ name: 'AbortError' });
|
||||
if (command === 'pdftoppm') {
|
||||
const { rm } = await import('node:fs/promises');
|
||||
expect(rm).toHaveBeenCalledWith('/tmp/pdf-render-test', {
|
||||
recursive: true,
|
||||
force: true,
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it('does not bind a shared availability probe to one caller signal', async () => {
|
||||
let callback!: ExecCallback;
|
||||
mockExecFile.mockImplementationOnce(
|
||||
(_command: unknown, _args: unknown, _options: unknown, cb: unknown) => {
|
||||
callback = cb as ExecCallback;
|
||||
return {} as ReturnType<typeof execFile>;
|
||||
},
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const cancelled = extractPDFText('/first.pdf', {
|
||||
signal: controller.signal,
|
||||
}).catch((error: unknown) => error);
|
||||
const other = extractPDFText('/second.pdf');
|
||||
controller.abort();
|
||||
mockExecResult({ stdout: 'second caller text' });
|
||||
callback(null, '', '');
|
||||
expect(await cancelled).toMatchObject({ name: 'AbortError' });
|
||||
expect(await other).toEqual({ success: true, text: 'second caller text' });
|
||||
expect(mockExecFile.mock.calls[0]![2]).not.toHaveProperty('signal');
|
||||
expect(mockExecFile).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it.each(['pdfinfo', 'pdftotext', 'pdftoppm'])(
|
||||
'bounds cancelled %s even when SIGKILL produces no close event',
|
||||
async (command) => {
|
||||
if (command !== 'pdfinfo') mockExecResult();
|
||||
const child = Object.assign(new EventEmitter(), {
|
||||
pid: 123,
|
||||
kill: vi.fn(),
|
||||
unref: vi.fn(),
|
||||
stdout: { destroy: vi.fn() },
|
||||
stderr: { destroy: vi.fn() },
|
||||
});
|
||||
let callback!: ExecCallback;
|
||||
mockExecFile.mockImplementationOnce(
|
||||
(_command: unknown, _args: unknown, _options: unknown, cb: unknown) => {
|
||||
callback = cb as ExecCallback;
|
||||
return child as unknown as ReturnType<typeof execFile>;
|
||||
},
|
||||
);
|
||||
const controller = new AbortController();
|
||||
const operation =
|
||||
command === 'pdfinfo'
|
||||
? getPDFPageCount('/test.pdf', controller.signal)
|
||||
: command === 'pdftotext'
|
||||
? extractPDFText('/test.pdf', { signal: controller.signal })
|
||||
: renderPDFPagesToImages('/test.pdf', {
|
||||
signal: controller.signal,
|
||||
});
|
||||
await vi.waitFor(() => expect(callback).toBeDefined());
|
||||
let settled = false;
|
||||
const outcome = operation.catch((error: unknown) => {
|
||||
settled = true;
|
||||
return error;
|
||||
});
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
controller.abort();
|
||||
callback(errorWith('cancelled', { name: 'AbortError' }), '', '');
|
||||
await vi.advanceTimersByTimeAsync(999);
|
||||
expect(child.kill).not.toHaveBeenCalled();
|
||||
expect(settled).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(1);
|
||||
expect(child.kill).toHaveBeenCalledWith('SIGKILL');
|
||||
expect(settled).toBe(false);
|
||||
await vi.advanceTimersByTimeAsync(1000);
|
||||
expect(await outcome).toMatchObject({ name: 'AbortError' });
|
||||
expect(child.unref).toHaveBeenCalledOnce();
|
||||
expect(child.stdout.destroy).toHaveBeenCalledOnce();
|
||||
expect(child.stderr.destroy).toHaveBeenCalledOnce();
|
||||
if (command === 'pdftoppm') {
|
||||
const { rm } = await import('node:fs/promises');
|
||||
expect(rm).toHaveBeenCalledWith('/tmp/pdf-render-test', {
|
||||
recursive: true,
|
||||
force: true,
|
||||
});
|
||||
}
|
||||
expect(vi.getTimerCount()).toBe(0);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
},
|
||||
);
|
||||
it('propagates page-count cancellation instead of returning unknown', async () => {
|
||||
const controller = new AbortController();
|
||||
const cancelled = new Error('page-count-cancelled');
|
||||
@@ -668,6 +811,66 @@ describe('pdf utilities', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it.skipIf(!['linux', 'darwin'].includes(process.platform)).each([3, 4])(
|
||||
'checks a descriptor page of %i bytes before reading against a 4-byte base64 ceiling',
|
||||
async (size) => {
|
||||
vi.mocked(execPDFCommandFromHandle).mockResolvedValue({
|
||||
stdout: '',
|
||||
stderr: '',
|
||||
code: 0,
|
||||
maxBufferExceeded: false,
|
||||
timedOut: false,
|
||||
});
|
||||
const directory = {
|
||||
fd: 8,
|
||||
stat: async () => ({ dev: 1, ino: 2, isDirectory: () => true }),
|
||||
close: vi.fn(async () => undefined),
|
||||
};
|
||||
const output = {
|
||||
fd: 9,
|
||||
stat: async () => ({ size, isFile: () => true }),
|
||||
read: vi.fn(async (buffer: Buffer) => {
|
||||
Buffer.from('abc').copy(buffer);
|
||||
return { bytesRead: 3, buffer };
|
||||
}),
|
||||
close: vi.fn(async () => undefined),
|
||||
};
|
||||
vi.mocked(open)
|
||||
.mockResolvedValueOnce(directory as unknown as FileHandle)
|
||||
.mockResolvedValueOnce(output as unknown as FileHandle);
|
||||
vi.mocked(lstat).mockResolvedValue({
|
||||
dev: 1,
|
||||
ino: 2,
|
||||
isDirectory: () => true,
|
||||
} as unknown as Awaited<ReturnType<typeof lstat>>);
|
||||
mockReaddir.mockResolvedValueOnce(['page-1.jpg'] as never);
|
||||
const input = { fd: 3, close: vi.fn() } as unknown as FileHandle;
|
||||
const result = await renderPDFPagesToImages(input, {
|
||||
maxTotalBase64Bytes: 4,
|
||||
});
|
||||
if (size === 3) {
|
||||
expect(result).toEqual({
|
||||
success: true,
|
||||
images: [{ data: 'YWJj', mimeType: 'image/jpeg' }],
|
||||
bytesTruncated: false,
|
||||
});
|
||||
expect(output.read).toHaveBeenCalledOnce();
|
||||
} else {
|
||||
expect(result).toMatchObject({ success: false, tooLarge: true });
|
||||
expect(output.read).not.toHaveBeenCalled();
|
||||
}
|
||||
expect(output.close).toHaveBeenCalledOnce();
|
||||
expect(directory.close).toHaveBeenCalledOnce();
|
||||
expect(input.close).not.toHaveBeenCalled();
|
||||
expect(mockReadFile).not.toHaveBeenCalled();
|
||||
expect(readFileWithinBase64Limit).not.toHaveBeenCalled();
|
||||
expect(rm).toHaveBeenCalledWith('/tmp/pdf-render-test', {
|
||||
recursive: true,
|
||||
force: true,
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it('errors when pdftoppm produces no images', async () => {
|
||||
mockRendered([]);
|
||||
expectError(await renderPDFPagesToImages('/test.pdf'), 'no image output');
|
||||
@@ -689,5 +892,46 @@ describe('pdf utilities', () => {
|
||||
expect(result.bytesTruncated).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('applies the caller ceiling to the first page and cleans up', async () => {
|
||||
mockRendered(['page-1.jpg']);
|
||||
vi.mocked(readFileWithinBase64Limit).mockResolvedValueOnce(undefined);
|
||||
const result = await renderPDFPagesToImages('/test.pdf', {
|
||||
maxTotalBase64Bytes: 4,
|
||||
});
|
||||
expect(result).toMatchObject({ success: false, tooLarge: true });
|
||||
expect(readFileWithinBase64Limit).toHaveBeenLastCalledWith(
|
||||
'/tmp/pdf-render-test/page-1.jpg',
|
||||
4,
|
||||
undefined,
|
||||
);
|
||||
const { rm } = await import('node:fs/promises');
|
||||
expect(rm).toHaveBeenCalledWith('/tmp/pdf-render-test', {
|
||||
recursive: true,
|
||||
force: true,
|
||||
});
|
||||
expect(mockReadFile).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('accepts the exact aggregate ceiling and stops before encoding an oversized next page', async () => {
|
||||
mockRendered(['page-1.jpg', 'page-2.jpg']);
|
||||
vi.mocked(readFileWithinBase64Limit)
|
||||
.mockResolvedValueOnce(Buffer.from('abc'))
|
||||
.mockResolvedValueOnce(undefined);
|
||||
const result = await renderPDFPagesToImages('/test.pdf', {
|
||||
maxTotalBase64Bytes: 4,
|
||||
});
|
||||
expect(result).toEqual({
|
||||
success: true,
|
||||
images: [{ data: 'YWJj', mimeType: 'image/jpeg' }],
|
||||
bytesTruncated: true,
|
||||
});
|
||||
expect(readFileWithinBase64Limit).toHaveBeenLastCalledWith(
|
||||
'/tmp/pdf-render-test/page-2.jpg',
|
||||
0,
|
||||
undefined,
|
||||
);
|
||||
expect(mockReadFile).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -18,6 +18,7 @@ import {
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { estimateTextTokens } from './request-tokenizer/textTokenizer.js';
|
||||
import { base64ByteLength, readFileWithinBase64Limit } from './inline-media.js';
|
||||
import { execPDFCommandFromHandle, pdfDescriptorPath } from './pdf-command.js';
|
||||
import { readFileHandleBytes } from './file-read-source.js';
|
||||
|
||||
@@ -133,12 +134,29 @@ function execCommand(
|
||||
}> {
|
||||
if (fileHandle)
|
||||
return execPDFCommandFromHandle(command, args, fileHandle, options);
|
||||
return new Promise((resolve) => {
|
||||
execFile(
|
||||
options.signal?.throwIfAborted();
|
||||
return new Promise((resolve, reject) => {
|
||||
let completed: Parameters<typeof resolve>[0] | undefined;
|
||||
let closed = false;
|
||||
let executionError: Error | null = null;
|
||||
let cancellationTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
const finish = () => {
|
||||
if (!completed || !closed) return;
|
||||
clearTimeout(cancellationTimer);
|
||||
try {
|
||||
options.signal?.throwIfAborted();
|
||||
if (executionError?.name === 'AbortError') throw executionError;
|
||||
resolve(completed);
|
||||
} catch (error) {
|
||||
reject(error);
|
||||
}
|
||||
};
|
||||
const child = execFile(
|
||||
command,
|
||||
args,
|
||||
{ encoding: 'utf8', ...options },
|
||||
(error, stdout, stderr) => {
|
||||
executionError = error;
|
||||
if (error) {
|
||||
// Node sets error.code to the string 'ERR_CHILD_PROCESS_STDIO_MAXBUFFER'
|
||||
// when stdout or stderr exceeds the configured maxBuffer — the child
|
||||
@@ -163,24 +181,47 @@ function execCommand(
|
||||
(errAny.signal === 'SIGTERM' ||
|
||||
errAny.signal === undefined ||
|
||||
errAny.signal === null)));
|
||||
resolve({
|
||||
completed = {
|
||||
stdout: String(stdout ?? ''),
|
||||
stderr: String(stderr ?? ''),
|
||||
code: typeof error.code === 'number' ? error.code : 1,
|
||||
maxBufferExceeded,
|
||||
timedOut,
|
||||
});
|
||||
return;
|
||||
};
|
||||
} else {
|
||||
completed = {
|
||||
stdout: String(stdout ?? ''),
|
||||
stderr: String(stderr ?? ''),
|
||||
code: 0,
|
||||
maxBufferExceeded: false,
|
||||
timedOut: false,
|
||||
};
|
||||
}
|
||||
resolve({
|
||||
stdout: String(stdout ?? ''),
|
||||
stderr: String(stderr ?? ''),
|
||||
code: 0,
|
||||
maxBufferExceeded: false,
|
||||
timedOut: false,
|
||||
});
|
||||
// Give cancellation cleanup a bound even if the child cannot close.
|
||||
if (error?.name === 'AbortError' && !closed) {
|
||||
cancellationTimer = setTimeout(() => {
|
||||
child.kill('SIGKILL');
|
||||
if (closed) return;
|
||||
cancellationTimer = setTimeout(() => {
|
||||
child.unref();
|
||||
child.stdout?.destroy();
|
||||
child.stderr?.destroy();
|
||||
reject(options.signal?.reason ?? error);
|
||||
}, 1000);
|
||||
}, 1000);
|
||||
}
|
||||
// Abort callbacks can precede exit; normally wait before cleanup.
|
||||
queueMicrotask(finish);
|
||||
},
|
||||
);
|
||||
if (child.pid === undefined) {
|
||||
closed = true;
|
||||
} else {
|
||||
child.once('close', () => {
|
||||
closed = true;
|
||||
finish();
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -329,8 +370,9 @@ export async function getPDFPageCount(
|
||||
}
|
||||
const count = parseInt(match[1]!, 10);
|
||||
return isNaN(count) ? null : count;
|
||||
} catch {
|
||||
} catch (error) {
|
||||
signal?.throwIfAborted();
|
||||
if (error instanceof Error && error.name === 'AbortError') throw error;
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -350,6 +392,7 @@ export async function extractPDFText(
|
||||
source: string | FileHandle,
|
||||
options?: { firstPage?: number; lastPage?: number; signal?: AbortSignal },
|
||||
): Promise<PDFTextResult> {
|
||||
options?.signal?.throwIfAborted();
|
||||
const fileHandle = typeof source === 'string' ? undefined : source;
|
||||
const filePath = typeof source === 'string' ? source : pdfDescriptorPath();
|
||||
const available = await isPdftotextAvailable(fileHandle, options?.signal);
|
||||
@@ -394,9 +437,7 @@ export async function extractPDFText(
|
||||
// execCommand reports a signal-killed child as timedOut (killed +
|
||||
// SIGTERM); check the caller's abort first so a user cancel is not
|
||||
// misreported as a 30s timeout.
|
||||
if (options?.signal?.aborted) {
|
||||
return { success: false, error: 'PDF text extraction was cancelled.' };
|
||||
}
|
||||
options?.signal?.throwIfAborted();
|
||||
|
||||
if (timedOut) {
|
||||
return {
|
||||
@@ -471,6 +512,8 @@ export async function extractPDFText(
|
||||
|
||||
return { success: true, text: stdout };
|
||||
} catch (e: unknown) {
|
||||
options?.signal?.throwIfAborted();
|
||||
if (e instanceof Error && e.name === 'AbortError') throw e;
|
||||
return {
|
||||
success: false,
|
||||
error: `pdftotext execution failed: ${e instanceof Error ? e.message : String(e)}`,
|
||||
@@ -540,7 +583,7 @@ export interface PDFRenderedImage {
|
||||
|
||||
export type PDFRenderResult =
|
||||
| { success: true; images: PDFRenderedImage[]; bytesTruncated: boolean }
|
||||
| { success: false; error: string };
|
||||
| { success: false; error: string; tooLarge?: boolean };
|
||||
|
||||
/**
|
||||
* Compare two pdftoppm output filenames (e.g. "page-1.jpg", "page-10.jpg") by
|
||||
@@ -568,8 +611,14 @@ function comparePdfPageFilenames(a: string, b: string): number {
|
||||
*/
|
||||
export async function renderPDFPagesToImages(
|
||||
source: string | FileHandle,
|
||||
options?: { firstPage?: number; lastPage?: number; signal?: AbortSignal },
|
||||
options?: {
|
||||
firstPage?: number;
|
||||
lastPage?: number;
|
||||
signal?: AbortSignal;
|
||||
maxTotalBase64Bytes?: number;
|
||||
},
|
||||
): Promise<PDFRenderResult> {
|
||||
options?.signal?.throwIfAborted();
|
||||
const fileHandle = typeof source === 'string' ? undefined : source;
|
||||
const filePath = typeof source === 'string' ? source : pdfDescriptorPath();
|
||||
const available = await isPdftoppmAvailable(fileHandle, options?.signal);
|
||||
@@ -689,7 +738,10 @@ export async function renderPDFPagesToImages(
|
||||
let bytesTruncated = false;
|
||||
for (const name of entries) {
|
||||
options?.signal?.throwIfAborted();
|
||||
let buffer: Buffer;
|
||||
const remaining =
|
||||
(options?.maxTotalBase64Bytes ?? PDF_RENDER_MAX_TOTAL_BASE64_BYTES) -
|
||||
totalBytes;
|
||||
let buffer: Buffer | undefined;
|
||||
if (outputDirectory) {
|
||||
await verifyDirectory?.();
|
||||
const outputPath =
|
||||
@@ -704,28 +756,44 @@ export async function renderPDFPagesToImages(
|
||||
const stats = await output.stat();
|
||||
if (!stats.isFile())
|
||||
throw new Error('PDF page output is not a regular file.');
|
||||
buffer = await readFileHandleBytes(
|
||||
output,
|
||||
stats.size,
|
||||
options?.signal,
|
||||
);
|
||||
buffer =
|
||||
options?.maxTotalBase64Bytes !== undefined &&
|
||||
base64ByteLength(stats.size) > remaining
|
||||
? undefined
|
||||
: await readFileHandleBytes(output, stats.size, options?.signal);
|
||||
} finally {
|
||||
await output.close();
|
||||
}
|
||||
await verifyDirectory?.();
|
||||
} else {
|
||||
buffer = await readFile(join(tempDir, name));
|
||||
buffer =
|
||||
options?.maxTotalBase64Bytes === undefined
|
||||
? await readFile(join(tempDir, name), { signal: options?.signal })
|
||||
: await readFileWithinBase64Limit(
|
||||
join(tempDir, name),
|
||||
remaining,
|
||||
options.signal,
|
||||
);
|
||||
}
|
||||
const data = buffer.toString('base64');
|
||||
// Always keep the first page; afterwards stop before exceeding the cap so
|
||||
// one tool result can't balloon to tens of MB.
|
||||
options?.signal?.throwIfAborted();
|
||||
// Only the legacy ceiling permits an oversized first page.
|
||||
if (
|
||||
images.length > 0 &&
|
||||
totalBytes + data.length > PDF_RENDER_MAX_TOTAL_BASE64_BYTES
|
||||
buffer === undefined ||
|
||||
((options?.maxTotalBase64Bytes !== undefined || images.length > 0) &&
|
||||
base64ByteLength(buffer.length) > remaining)
|
||||
) {
|
||||
if (images.length === 0) {
|
||||
return {
|
||||
success: false,
|
||||
error:
|
||||
"The first rendered PDF page exceeds the inline media limit. Use the 'pages' parameter with a smaller page or a smaller document.",
|
||||
tooLarge: true,
|
||||
};
|
||||
}
|
||||
bytesTruncated = true;
|
||||
break;
|
||||
}
|
||||
const data = buffer.toString('base64');
|
||||
totalBytes += data.length;
|
||||
images.push({ data, mimeType: 'image/jpeg' });
|
||||
}
|
||||
@@ -733,6 +801,7 @@ export async function renderPDFPagesToImages(
|
||||
return { success: true, images, bytesTruncated };
|
||||
} catch (e: unknown) {
|
||||
options?.signal?.throwIfAborted();
|
||||
if (e instanceof Error && e.name === 'AbortError') throw e;
|
||||
return {
|
||||
success: false,
|
||||
error: `pdftoppm execution failed: ${
|
||||
|
||||
+311
@@ -0,0 +1,311 @@
|
||||
package com.alibaba.qwen.code.managedagent;
|
||||
|
||||
import com.alibaba.qwen.code.managedagent.workspace.runtime.EmbeddedRuntimeBroker;
|
||||
import com.alibaba.qwen.code.runtimebroker.transport.HttpRuntimeTransport;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.test.util.ReflectionTestUtils;
|
||||
|
||||
import java.net.ProxySelector;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.time.Duration;
|
||||
import java.util.Arrays;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import java.util.function.Supplier;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
final class HostedProviderMediaProbe implements AutoCloseable {
|
||||
private static final ObjectMapper JSON = new ObjectMapper();
|
||||
private final JdbcTemplate jdbc;
|
||||
private final String tenant;
|
||||
private final List<Map<String, Object>> sessions;
|
||||
private final EmbeddedRuntimeBroker original;
|
||||
private final Supplier<EmbeddedRuntimeBroker> restart;
|
||||
private volatile Object provisioner;
|
||||
private final HttpClient upstream = HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1).build();
|
||||
private final HttpClient forwarded;
|
||||
private final Map<String, Integer> dispatches = new ConcurrentHashMap<>();
|
||||
private final Map<String, JsonNode> executed = new ConcurrentHashMap<>();
|
||||
private final Map<String, JsonNode> references = new ConcurrentHashMap<>();
|
||||
private final Map<String, String> hashes = new ConcurrentHashMap<>();
|
||||
private final Map<String, Integer> lengths = new ConcurrentHashMap<>();
|
||||
private final Set<ProcessHandle> processes = ConcurrentHashMap.newKeySet();
|
||||
private final Set<String> finished = ConcurrentHashMap.newKeySet();
|
||||
private final ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
|
||||
private final AtomicReference<Throwable> failure = new AtomicReference<>();
|
||||
private volatile EmbeddedRuntimeBroker restarted;
|
||||
private volatile int dropped;
|
||||
|
||||
HostedProviderMediaProbe(JdbcTemplate jdbc, String tenant, List<Map<String, Object>> sessions,
|
||||
EmbeddedRuntimeBroker broker, HttpServer server, Supplier<EmbeddedRuntimeBroker> restart) {
|
||||
this.jdbc = jdbc;
|
||||
this.tenant = tenant;
|
||||
this.sessions = sessions;
|
||||
original = broker;
|
||||
this.restart = restart;
|
||||
server.setExecutor(executor);
|
||||
Object service = ReflectionTestUtils.getField(broker, "service");
|
||||
Object transport = ReflectionTestUtils.getField(service, "transport");
|
||||
provisioner = injectReadProbe(broker);
|
||||
forwarded = HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1)
|
||||
.proxy(ProxySelector.of(server.getAddress())).build();
|
||||
ReflectionTestUtils.setField(transport, "delegate", new HttpRuntimeTransport(forwarded));
|
||||
server.createContext("/", exchange -> {
|
||||
try {
|
||||
forward(exchange);
|
||||
} catch (Throwable cause) {
|
||||
// Payloads are private evidence: do not print JsonNode assertion diffs.
|
||||
recordFailure(cause, "worker proxy");
|
||||
} finally {
|
||||
exchange.close();
|
||||
}
|
||||
});
|
||||
server.createContext("/media/", exchange -> {
|
||||
try {
|
||||
assertThat(failure.get()).as("M4 worker proxy").isNull();
|
||||
String[] route = exchange.getRequestURI().getPath().split("/");
|
||||
assertThat(route).hasSize(4);
|
||||
var session = sessions.stream().filter(value -> value.get("sessionId").equals(route[2]))
|
||||
.findFirst().orElseThrow();
|
||||
JsonNode fields = JSON.readTree(exchange.getRequestBody());
|
||||
Map<String, Object> answer = check(session, route[3], fields);
|
||||
byte[] body = JSON.writeValueAsBytes(answer);
|
||||
exchange.sendResponseHeaders(200, body.length);
|
||||
exchange.getResponseBody().write(body);
|
||||
} catch (Throwable cause) {
|
||||
recordFailure(cause, "ledger probe");
|
||||
exchange.sendResponseHeaders(500, -1);
|
||||
} finally {
|
||||
exchange.close();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private Object injectReadProbe(EmbeddedRuntimeBroker broker) {
|
||||
Object service = ReflectionTestUtils.getField(broker, "service");
|
||||
Object provisioner = ReflectionTestUtils.getField(ReflectionTestUtils.getField(service, "provisioner"), "delegate");
|
||||
List<?> command = (List<?>) ReflectionTestUtils.getField(provisioner, "command");
|
||||
assertThat(command.size()).isEqualTo(3);
|
||||
assertThat(command.get(2)).isEqualTo("managed-runtime-worker");
|
||||
Path cli = Path.of(command.get(1).toString());
|
||||
Path readProbe = cli.getParent().getParent().resolve("integration-tests/helpers/provider-media-read-probe.mjs");
|
||||
ReflectionTestUtils.setField(provisioner, "command", List.of(command.get(0).toString(), "--import",
|
||||
readProbe.toUri().toString(), cli.toString(), "managed-runtime-worker"));
|
||||
return provisioner;
|
||||
}
|
||||
|
||||
private void recordFailure(Throwable cause, String stage) {
|
||||
String site = Arrays.stream(cause.getStackTrace())
|
||||
.filter(frame -> frame.getClassName().equals(HostedProviderMediaProbe.class.getName()))
|
||||
.findFirst().map(frame -> frame.getMethodName() + ":" + frame.getLineNumber()).orElse("unknown");
|
||||
String message = "M4_MEDIA_PROBE " + stage + " " + cause.getClass().getSimpleName() + " at " + site;
|
||||
if (failure.compareAndSet(null, new IllegalStateException(message))) {
|
||||
System.err.println(message);
|
||||
}
|
||||
}
|
||||
|
||||
private void forward(HttpExchange exchange) throws Exception {
|
||||
assertThat(exchange.getRequestMethod()).isEqualTo("POST");
|
||||
assertThat(exchange.getRequestURI().getHost()).isEqualTo("127.0.0.1");
|
||||
String path = exchange.getRequestURI().getPath();
|
||||
assertThat(path).doesNotContain("/v2/");
|
||||
byte[] bytes = exchange.getRequestBody().readAllBytes();
|
||||
JsonNode body = JSON.readTree(bytes);
|
||||
String kind = body.path("operation").path("kind").asText();
|
||||
boolean provider = path.equals("/internal/managed-runtime/provider/v1/control");
|
||||
if (provider) {
|
||||
assertThat(body.path("providerProtocol").asText()).isEqualTo("managed-runtime-provider/1");
|
||||
}
|
||||
if (kind.equals("execute")) {
|
||||
assertThat(provider).isTrue();
|
||||
JsonNode reference = body.path("operation").path("reference");
|
||||
assertThat(reference.size()).isEqualTo(7);
|
||||
String runtimeId = reference.path("sessionId").asText();
|
||||
dispatches.merge(runtimeId, 1, Integer::sum);
|
||||
executed.put(runtimeId, reference);
|
||||
captureProcesses();
|
||||
}
|
||||
HttpRequest.Builder request = HttpRequest.newBuilder(exchange.getRequestURI())
|
||||
.timeout(Duration.ofSeconds(40)).POST(HttpRequest.BodyPublishers.ofByteArray(bytes));
|
||||
for (String header : List.of("Authorization", "Cache-Control", "Content-Type",
|
||||
"X-Qwen-Managed-Lease-Id", "X-Qwen-Managed-Lease-Epoch")) {
|
||||
String value = exchange.getRequestHeaders().getFirst(header);
|
||||
if (value != null) {
|
||||
request.header(header, value);
|
||||
}
|
||||
}
|
||||
var response = upstream.send(request.build(), HttpResponse.BodyHandlers.ofByteArray());
|
||||
if (kind.equals("execute")) {
|
||||
assertThat(response.statusCode()).isEqualTo(200);
|
||||
JsonNode result = JSON.readTree(response.body()).path("result");
|
||||
assertThat(result.has("responseParts")).isFalse();
|
||||
assertThat(result.path("result").has("llmContent")).isTrue();
|
||||
String harness = body.path("session").path("harnessSessionId").asText();
|
||||
finished.add(harness);
|
||||
var session = sessions.stream().filter(value -> value.get("sessionId").equals(harness))
|
||||
.findFirst().orElseThrow();
|
||||
if (session.get("fault").equals("lost-execute") && dropped == 0) {
|
||||
Path file = Path.of(session.get("directory").toString()).resolve("proof.pdf");
|
||||
Files.write(file, Files.readAllBytes(file.resolveSibling("proof.pdf.replacement")));
|
||||
dropped++;
|
||||
return;
|
||||
}
|
||||
}
|
||||
for (String header : List.of("Content-Type", "Cache-Control")) {
|
||||
response.headers().firstValue(header).ifPresent(value -> exchange.getResponseHeaders().set(header, value));
|
||||
}
|
||||
exchange.sendResponseHeaders(response.statusCode(), response.body().length);
|
||||
exchange.getResponseBody().write(response.body());
|
||||
}
|
||||
|
||||
private Map<String, Object> execution(Map<String, Object> session) {
|
||||
var rows = jdbc.queryForList("SELECT * FROM ma_tool_execution WHERE harness_session_id = ?",
|
||||
session.get("sessionId"));
|
||||
assertThat(rows.size()).as("one SQL invocation per Session").isEqualTo(1);
|
||||
return rows.getFirst();
|
||||
}
|
||||
|
||||
private Map<String, Object> check(Map<String, Object> session, String phase, JsonNode fields) throws Exception {
|
||||
String harness = session.get("sessionId").toString();
|
||||
if (phase.equals("worker-done")) {
|
||||
return Map.of("done", finished.contains(harness));
|
||||
}
|
||||
if (phase.equals("dropped")) {
|
||||
assertThat(session.get("fault")).isEqualTo("lost-execute");
|
||||
return Map.of("done", dropped == 1);
|
||||
}
|
||||
if (phase.equals("restart")) {
|
||||
assertThat(restarted).isNull();
|
||||
for (var value : sessions) {
|
||||
check(value, "released", JSON.createObjectNode());
|
||||
}
|
||||
original.close();
|
||||
for (var process : processes) {
|
||||
if (process.isAlive()) {
|
||||
process.onExit().get(10, TimeUnit.SECONDS);
|
||||
}
|
||||
assertThat(process.isAlive()).as("original media worker stopped").isFalse();
|
||||
}
|
||||
restarted = restart.get();
|
||||
provisioner = injectReadProbe(restarted);
|
||||
return Map.of("brokerUrl", restarted.getBaseUri().toString());
|
||||
}
|
||||
var row = execution(session);
|
||||
String runtimeId = row.get("runtime_session_id").toString();
|
||||
JsonNode reference = JSON.readTree(row.get("reference_json").toString());
|
||||
if (phase.equals("prepared")) {
|
||||
assertThat(reference.size()).isEqualTo(7);
|
||||
assertThat(reference.equals(fields.path("reference"))).as("original seven-field reference").isTrue();
|
||||
references.put(harness, reference);
|
||||
assertThat(row.get("execution_call_id")).isEqualTo(fields.path("executionCallId").asText());
|
||||
}
|
||||
assertThat(reference.equals(references.get(harness))).as("immutable SQL reference").isTrue();
|
||||
boolean prepared = phase.equals("prepared");
|
||||
assertThat(row.get("execution_state")).isEqualTo(prepared ? "PREPARED" : "SETTLED");
|
||||
assertThat(((Number) row.get("dispatch_generation")).longValue()).isEqualTo(prepared ? 0 : 1);
|
||||
assertThat(dispatches.getOrDefault(runtimeId, 0)).as("physical provider execute count").isEqualTo(prepared ? 0 : 1);
|
||||
if (!prepared) {
|
||||
assertThat(reference.equals(executed.get(runtimeId))).as("executed original reference").isTrue();
|
||||
assertThat(row.get("execution_status")).isEqualTo(session.get("fault").equals("pdf-too-large") ? "error" : "success");
|
||||
byte[] encoded = JSON.writeValueAsBytes(JSON.readTree(row.get("result_json").toString()));
|
||||
String hash = HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(encoded));
|
||||
if (phase.equals("settled")) {
|
||||
hashes.put(harness, hash);
|
||||
lengths.put(harness, encoded.length);
|
||||
}
|
||||
assertThat(hash).isEqualTo(hashes.get(harness));
|
||||
assertThat(encoded.length).isEqualTo(lengths.get(harness));
|
||||
if (fields.has("resultHash")) {
|
||||
assertThat(hash).isEqualTo(fields.path("resultHash").asText());
|
||||
}
|
||||
if (fields.has("resultBytes")) {
|
||||
assertThat(encoded.length).isEqualTo(fields.path("resultBytes").asInt());
|
||||
}
|
||||
}
|
||||
String storageId = "storage-" + sessions.indexOf(session);
|
||||
String physical = jdbc.queryForObject("SELECT physical_key FROM ma_storage"
|
||||
+ " WHERE alias_key = ? AND tenant_id = ? AND storage_id = ?", String.class,
|
||||
com.alibaba.qwen.code.runtimebroker.JdbcRuntimeBindingRepository.storageAliasKey(tenant, storageId),
|
||||
tenant, storageId);
|
||||
assertThat(physical).isNotBlank();
|
||||
var member = jdbc.queryForMap("SELECT binding_id, runtime_generation, session_state,"
|
||||
+ " execution_storage_key, execution_exclusive FROM ma_runtime_session"
|
||||
+ " WHERE tenant_id = ? AND harness_session_id = ? AND runtime_session_id = ?",
|
||||
tenant, harness, runtimeId);
|
||||
boolean released = phase.equals("released") || phase.equals("restarted");
|
||||
assertThat(member.get("binding_id")).as("original runtime binding").isEqualTo(row.get("binding_id"));
|
||||
assertThat(member.get("runtime_generation")).as("original runtime generation").isEqualTo(row.get("runtime_generation"));
|
||||
assertThat(member.get("execution_storage_key")).as("physical execution membership")
|
||||
.isEqualTo(released ? null : physical);
|
||||
assertThat(member.get("execution_exclusive")).as("shared execution membership").isEqualTo(false);
|
||||
assertThat(member.get("session_state")).isEqualTo(released ? "RELEASED" : "READY");
|
||||
if (phase.equals("restarted")) {
|
||||
assertThat(restarted).isNotNull();
|
||||
assertThat(Files.readString(Path.of(session.get("directory").toString())
|
||||
.resolve(".provider-media-reads.json"))).isEqualTo(readCounts.get(harness));
|
||||
}
|
||||
if (phase.equals("released")) {
|
||||
readCounts.put(harness, Files.readString(Path.of(session.get("directory").toString())
|
||||
.resolve(".provider-media-reads.json")));
|
||||
}
|
||||
return Map.of("checked", phase);
|
||||
}
|
||||
|
||||
private final Map<String, String> readCounts = new ConcurrentHashMap<>();
|
||||
|
||||
void assertReport(Map<String, Object> session, JsonNode report) throws Exception {
|
||||
assertThat(failure.get()).as("M4 probe").isNull();
|
||||
check(session, "restarted", report);
|
||||
assertThat(dropped).isEqualTo(1);
|
||||
assertThat(execution(session).get("execution_call_id")).isEqualTo(report.path("executionCallId").asText());
|
||||
System.out.println("M4_MEDIA_LEDGER " + session.get("fault") + " dispatch=1 restarted=true owner=released");
|
||||
}
|
||||
|
||||
private void captureProcesses() {
|
||||
for (Object value : ((Map<?, ?>) ReflectionTestUtils.getField(provisioner, "owned")).values()) {
|
||||
Process process = (Process) ReflectionTestUtils.getField(value, "process");
|
||||
if (process.isAlive()) {
|
||||
String[] arguments = process.info().arguments().orElseThrow();
|
||||
assertThat(List.of(arguments)).contains("--import", "managed-runtime-worker");
|
||||
assertThat(List.of(arguments).stream().anyMatch(argument -> argument.endsWith("/dist/cli.js")))
|
||||
.as("packaged worker entry").isTrue();
|
||||
}
|
||||
processes.add(process.toHandle());
|
||||
processes.addAll(process.descendants().toList());
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() throws Exception {
|
||||
executor.shutdownNow();
|
||||
if (restarted != null) {
|
||||
restarted.close();
|
||||
}
|
||||
forwarded.shutdownNow();
|
||||
upstream.shutdownNow();
|
||||
captureProcesses();
|
||||
processes.forEach(ProcessHandle::destroyForcibly);
|
||||
for (var process : processes) {
|
||||
if (process.isAlive()) {
|
||||
process.onExit().get(10, TimeUnit.SECONDS);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
+195
@@ -0,0 +1,195 @@
|
||||
package com.alibaba.qwen.code.managedagent;
|
||||
|
||||
import com.alibaba.qwen.code.managedagent.persistence.ManagedSchemaJdbcTemplate;
|
||||
import com.alibaba.qwen.code.runtimebroker.JdbcRuntimeBindingRepository;
|
||||
import com.alibaba.qwen.code.runtimebroker.managedschema.ManagedSchema;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.sun.net.httpserver.Headers;
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import org.h2.jdbcx.JdbcDataSource;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.Timeout;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.springframework.test.util.ReflectionTestUtils;
|
||||
|
||||
import java.io.ByteArrayInputStream;
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.net.URI;
|
||||
import java.net.http.HttpClient;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.atomic.AtomicInteger;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
import static org.mockito.ArgumentMatchers.anyInt;
|
||||
import static org.mockito.ArgumentMatchers.anyLong;
|
||||
import static org.mockito.Mockito.CALLS_REAL_METHODS;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.never;
|
||||
import static org.mockito.Mockito.verify;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
class HostedProviderMediaProbeTest {
|
||||
@TempDir
|
||||
Path directory;
|
||||
|
||||
@Test
|
||||
@Timeout(10)
|
||||
void droppedEvidenceRequiresTheActualSingleLostExecuteInjection() throws Exception {
|
||||
ObjectMapper json = new ObjectMapper();
|
||||
Map<String, Object> session = Map.of("sessionId", "harness", "fault", "lost-execute",
|
||||
"directory", directory.toString());
|
||||
byte[] original = "original PDF".getBytes(StandardCharsets.UTF_8);
|
||||
byte[] replacement = "replacement PDF".getBytes(StandardCharsets.UTF_8);
|
||||
Files.write(directory.resolve("proof.pdf"), original);
|
||||
Files.write(directory.resolve("proof.pdf.replacement"), replacement);
|
||||
var probe = mock(HostedProviderMediaProbe.class, CALLS_REAL_METHODS);
|
||||
HttpClient client = HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1).build();
|
||||
ReflectionTestUtils.setField(probe, "upstream", client);
|
||||
ReflectionTestUtils.setField(probe, "sessions", List.of(session));
|
||||
ReflectionTestUtils.setField(probe, "dispatches", new ConcurrentHashMap<String, Integer>());
|
||||
ReflectionTestUtils.setField(probe, "executed", new ConcurrentHashMap<String, JsonNode>());
|
||||
ReflectionTestUtils.setField(probe, "finished", ConcurrentHashMap.newKeySet());
|
||||
ReflectionTestUtils.setField(probe, "processes", ConcurrentHashMap.newKeySet());
|
||||
ReflectionTestUtils.setField(probe, "provisioner", new EmptyOwned());
|
||||
assertThat(ReflectionTestUtils.<Map<String, Object>>invokeMethod(probe, "check",
|
||||
session, "dropped", json.createObjectNode())).containsEntry("done", false);
|
||||
|
||||
AtomicInteger workerRequests = new AtomicInteger();
|
||||
byte[] response = json.writeValueAsBytes(Map.of("result", Map.of("result", Map.of("llmContent", "original PDF"))));
|
||||
HttpServer worker = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
worker.createContext("/internal/managed-runtime/provider/v1/control", exchange -> {
|
||||
exchange.getRequestBody().readAllBytes();
|
||||
workerRequests.incrementAndGet();
|
||||
exchange.getResponseHeaders().set("Content-Type", "application/json");
|
||||
exchange.sendResponseHeaders(200, response.length);
|
||||
exchange.getResponseBody().write(response);
|
||||
exchange.close();
|
||||
});
|
||||
worker.start();
|
||||
try {
|
||||
URI uri = URI.create("http://127.0.0.1:" + worker.getAddress().getPort()
|
||||
+ "/internal/managed-runtime/provider/v1/control");
|
||||
byte[] body = json.writeValueAsBytes(Map.of("providerProtocol", "managed-runtime-provider/1",
|
||||
"session", Map.of("harnessSessionId", "harness"),
|
||||
"operation", Map.of("kind", "execute", "reference", Map.of(
|
||||
"sessionId", "runtime", "promptId", "prompt", "callId", "call",
|
||||
"invocationId", "invocation", "argsDigest", "args", "capabilityDigest", "capability",
|
||||
"policyRevision", "policy"))));
|
||||
HttpExchange exchange = mock(HttpExchange.class);
|
||||
ByteArrayOutputStream forwarded = new ByteArrayOutputStream();
|
||||
when(exchange.getRequestMethod()).thenReturn("POST");
|
||||
when(exchange.getRequestURI()).thenReturn(uri);
|
||||
when(exchange.getRequestBody()).thenAnswer(ignored -> new ByteArrayInputStream(body));
|
||||
when(exchange.getRequestHeaders()).thenReturn(new Headers());
|
||||
when(exchange.getResponseHeaders()).thenReturn(new Headers());
|
||||
when(exchange.getResponseBody()).thenReturn(forwarded);
|
||||
|
||||
ReflectionTestUtils.invokeMethod(probe, "forward", exchange);
|
||||
assertThat(workerRequests.get()).isEqualTo(1);
|
||||
assertThat(ReflectionTestUtils.<Map<String, Object>>invokeMethod(probe, "check",
|
||||
session, "dropped", json.createObjectNode())).containsEntry("done", true);
|
||||
assertThat(ReflectionTestUtils.getField(probe, "dropped")).isEqualTo(1);
|
||||
assertThat(Files.readAllBytes(directory.resolve("proof.pdf"))).isEqualTo(replacement);
|
||||
verify(exchange, never()).sendResponseHeaders(anyInt(), anyLong());
|
||||
assertThat(forwarded.size()).isZero();
|
||||
|
||||
ReflectionTestUtils.invokeMethod(probe, "forward", exchange);
|
||||
assertThat(workerRequests.get()).isEqualTo(2);
|
||||
assertThat(ReflectionTestUtils.getField(probe, "dropped")).isEqualTo(1);
|
||||
verify(exchange).sendResponseHeaders(200, response.length);
|
||||
assertThat(forwarded.toByteArray()).isEqualTo(response);
|
||||
} finally {
|
||||
worker.stop(0);
|
||||
client.close();
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void mediaEvidenceUsesTheCanonicalSchemaAndPhysicalExecutionMembership() throws Exception {
|
||||
JdbcDataSource source = new JdbcDataSource();
|
||||
source.setURL("jdbc:h2:mem:media-" + java.util.UUID.randomUUID()
|
||||
+ ";MODE=MySQL;DB_CLOSE_DELAY=-1;DATABASE_TO_LOWER=TRUE");
|
||||
ManagedSchema.initialize(source);
|
||||
var jdbc = new ManagedSchemaJdbcTemplate(source);
|
||||
String tenant = "media-租户";
|
||||
String physical = "a".repeat(64);
|
||||
Map<String, Object> session = Map.of("sessionId", "harness", "fault", "lost-execute",
|
||||
"directory", directory.toString());
|
||||
JsonNode reference = new ObjectMapper().valueToTree(Map.of(
|
||||
"sessionId", "runtime", "promptId", "prompt", "callId", "call",
|
||||
"invocationId", "invocation", "argsDigest", "args", "capabilityDigest", "capability",
|
||||
"policyRevision", "policy"));
|
||||
jdbc.update("INSERT INTO ma_tool_execution (execution_call_id_hash, execution_call_id,"
|
||||
+ " idempotency_key_hash, idempotency_key, binding_id, runtime_generation, harness_session_id,"
|
||||
+ " runtime_session_id, runtime_session_key, turn_id, tool_call_id, request_digest,"
|
||||
+ " reference_json, execution_state, last_sequence, cancel_requested, dispatch_generation,"
|
||||
+ " record_version) VALUES (?, 'execute', ?, 'key', 'binding', 1, 'harness', 'runtime', ?,"
|
||||
+ " 'turn', 'call', 'digest', ?, 'PREPARED', 0, FALSE, 0, 1)",
|
||||
"b".repeat(64), "c".repeat(64), "d".repeat(64), reference.toString());
|
||||
jdbc.update("INSERT INTO ma_runtime_session (scope_key, runtime_session_id, tenant_id, workspace_id,"
|
||||
+ " workspace_generation, canonical_cwd, capability_digest, isolation_class, harness_session_id,"
|
||||
+ " turn_kind, binding_id, runtime_generation, session_state, record_version, last_active_at,"
|
||||
+ " execution_storage_key)"
|
||||
+ " VALUES (?, 'runtime', ?, 'workspace', '1', '/', 'capability', 'process', 'harness',"
|
||||
+ " 'tool', 'binding', 1, 'READY', 1, ?, ?)", "e".repeat(64), tenant, System.currentTimeMillis(), physical);
|
||||
jdbc.update("INSERT INTO ma_storage (alias_key, tenant_key, storage_key, tenant_id, storage_id, physical_key)"
|
||||
+ " VALUES (?, ?, ?, ?, 'storage-0', ?)",
|
||||
JdbcRuntimeBindingRepository.storageAliasKey(tenant, "storage-0"),
|
||||
"f".repeat(64), "0".repeat(64), tenant, physical);
|
||||
jdbc.update("INSERT INTO ma_storage_owner (storage_key) VALUES (?)", physical);
|
||||
var probe = mock(HostedProviderMediaProbe.class, CALLS_REAL_METHODS);
|
||||
ReflectionTestUtils.setField(probe, "jdbc", jdbc);
|
||||
ReflectionTestUtils.setField(probe, "tenant", tenant);
|
||||
ReflectionTestUtils.setField(probe, "sessions", List.of(session));
|
||||
ReflectionTestUtils.setField(probe, "references", new ConcurrentHashMap<String, JsonNode>());
|
||||
ReflectionTestUtils.setField(probe, "dispatches", new ConcurrentHashMap<String, Integer>());
|
||||
ReflectionTestUtils.setField(probe, "executed", new ConcurrentHashMap<String, JsonNode>());
|
||||
ReflectionTestUtils.setField(probe, "hashes", new ConcurrentHashMap<String, String>());
|
||||
ReflectionTestUtils.setField(probe, "lengths", new ConcurrentHashMap<String, Integer>());
|
||||
ReflectionTestUtils.setField(probe, "readCounts", new ConcurrentHashMap<String, String>());
|
||||
var fields = new ObjectMapper().createObjectNode().put("executionCallId", "execute");
|
||||
fields.set("reference", reference);
|
||||
assertThat(ReflectionTestUtils.<Map<String, Object>>invokeMethod(probe, "check", session, "prepared", fields))
|
||||
.containsEntry("checked", "prepared");
|
||||
jdbc.update("UPDATE ma_runtime_session SET execution_storage_key=?", "9".repeat(64));
|
||||
assertThatThrownBy(() -> ReflectionTestUtils.invokeMethod(probe, "check", session, "prepared", fields))
|
||||
.isInstanceOf(AssertionError.class).hasMessageContaining("physical execution membership");
|
||||
jdbc.update("UPDATE ma_runtime_session SET execution_storage_key=?", physical);
|
||||
jdbc.update("UPDATE ma_runtime_session SET binding_id='other'");
|
||||
assertThatThrownBy(() -> ReflectionTestUtils.invokeMethod(probe, "check", session, "prepared", fields))
|
||||
.isInstanceOf(AssertionError.class).hasMessageContaining("original runtime binding");
|
||||
jdbc.update("UPDATE ma_runtime_session SET binding_id='binding', runtime_generation=2");
|
||||
assertThatThrownBy(() -> ReflectionTestUtils.invokeMethod(probe, "check", session, "prepared", fields))
|
||||
.isInstanceOf(AssertionError.class).hasMessageContaining("original runtime generation");
|
||||
jdbc.update("UPDATE ma_runtime_session SET runtime_generation=1, execution_exclusive=TRUE");
|
||||
assertThatThrownBy(() -> ReflectionTestUtils.invokeMethod(probe, "check", session, "prepared", fields))
|
||||
.isInstanceOf(AssertionError.class).hasMessageContaining("shared execution membership");
|
||||
jdbc.update("UPDATE ma_runtime_session SET execution_exclusive=FALSE");
|
||||
jdbc.update("UPDATE ma_tool_execution SET execution_state='SETTLED', execution_status='success',"
|
||||
+ " dispatch_generation=1, result_json='{}'");
|
||||
ReflectionTestUtils.setField(probe, "dispatches", Map.of("runtime", 1));
|
||||
ReflectionTestUtils.setField(probe, "executed", Map.of("runtime", reference));
|
||||
assertThat(ReflectionTestUtils.<Map<String, Object>>invokeMethod(probe, "check", session, "settled", fields))
|
||||
.containsEntry("checked", "settled");
|
||||
jdbc.update("UPDATE ma_runtime_session SET session_state='RELEASED'");
|
||||
assertThatThrownBy(() -> ReflectionTestUtils.invokeMethod(probe, "check", session, "released", fields))
|
||||
.isInstanceOf(AssertionError.class).hasMessageContaining("physical execution membership");
|
||||
jdbc.update("UPDATE ma_runtime_session SET execution_storage_key=NULL");
|
||||
Files.writeString(directory.resolve(".provider-media-reads.json"), "{}");
|
||||
assertThat(ReflectionTestUtils.<Map<String, Object>>invokeMethod(probe, "check", session, "released", fields))
|
||||
.containsEntry("checked", "released");
|
||||
}
|
||||
|
||||
private static class EmptyOwned {
|
||||
private final Map<String, Object> owned = Map.of();
|
||||
}
|
||||
}
|
||||
+34
-6
@@ -8,12 +8,15 @@ import com.alibaba.qwen.code.managedagent.persistence.ManagedSessionStore;
|
||||
import com.alibaba.qwen.code.managedagent.persistence.ToolPublicationAdmissionStore;
|
||||
import com.alibaba.qwen.code.managedagent.persistence.ToolPublicationDataStore;
|
||||
import com.alibaba.qwen.code.managedagent.persistence.ToolPublicationStore;
|
||||
import com.alibaba.qwen.code.managedagent.persistence.WorkspaceExecutionStore;
|
||||
import com.alibaba.qwen.code.managedagent.persistence.WorkspaceStorageGuard;
|
||||
import com.alibaba.qwen.code.managedagent.session.AgentStateStore;
|
||||
import com.alibaba.qwen.code.managedagent.session.HarnessEventProjector;
|
||||
import com.alibaba.qwen.code.managedagent.workspace.WorkspaceSelection;
|
||||
import com.alibaba.qwen.code.managedagent.workspace.runtime.EmbeddedRuntimeBroker;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeBindingRepository;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeSession;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeSessionRepository;
|
||||
import com.alibaba.qwen.code.runtimebroker.RuntimeTransport;
|
||||
import com.alibaba.qwen.code.runtimebroker.ToolExecutionRepository;
|
||||
import com.alibaba.qwen.code.runtimebroker.WorkspaceExecutionProfile;
|
||||
@@ -23,6 +26,7 @@ import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.Timeout;
|
||||
import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
|
||||
import org.junit.jupiter.api.extension.ExtendWith;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.junit.jupiter.params.ParameterizedTest;
|
||||
@@ -197,6 +201,16 @@ class HostedWorkspaceToolTurnIT {
|
||||
runDriver(cases, "provider-control");
|
||||
}
|
||||
|
||||
@Test
|
||||
@Timeout(300)
|
||||
@EnabledIfSystemProperty(named = "qwen.provider.media.bundle-ready", matches = "true")
|
||||
void providerMediaReachesModelAndSurvivesReplyLossAndBrokerRestartOnMySql() throws Exception {
|
||||
assertThat(System.getProperty("mysql.url")).as("M4 requires real MySQL").startsWith("jdbc:mysql:");
|
||||
assertThat(System.getProperty("mysql.user")).as("M4 requires -Dmysql.user").isNotBlank();
|
||||
runDriver(List.of("png", "jpeg", "webp", "gif", "native-pdf", "pdf-text", "pdf-render",
|
||||
"image-disabled", "image-missing", "pdf-too-large", "lost-start", "lost-execute"), "provider-media");
|
||||
}
|
||||
|
||||
private static void registerUnicodePublication(GenericApplicationContext context) {
|
||||
var objects = new ConcurrentHashMap<String, byte[]>();
|
||||
context.registerBean("unicodeObjects", ToolPublicationObjectStore.class, () -> new ToolPublicationObjectStore() {
|
||||
@@ -249,6 +263,7 @@ class HostedWorkspaceToolTurnIT {
|
||||
boolean sseGaps = driverName.equals("sse-gap");
|
||||
boolean shellOutput = driverName.equals("shell-output");
|
||||
boolean providerControl = driverName.equals("provider-control");
|
||||
boolean providerMedia = driverName.equals("provider-media");
|
||||
Path cli = Path.of(System.getProperty("qwen.cli.entry", "../../dist/cli.js")).toAbsolutePath().normalize();
|
||||
assertThat(cli).isRegularFile();
|
||||
String node = System.getProperty("node.executable");
|
||||
@@ -336,10 +351,14 @@ class HostedWorkspaceToolTurnIT {
|
||||
}
|
||||
if (faults) {
|
||||
var metadata = jdbc.queryForMap("SELECT VERSION() AS version, @@version_comment AS engine");
|
||||
System.out.println((shellOutput || providerControl ? "FG6F_DATABASE " : sseGaps ? "FG6E_DATABASE " : cancellations ? "FG6D_DATABASE " : storeFaults ? "FG6B_DATABASE " : "FG6A_DATABASE ") + metadata);
|
||||
System.out.println((providerMedia ? "M4_DATABASE " : shellOutput || providerControl ? "FG6F_DATABASE " : sseGaps ? "FG6E_DATABASE " : cancellations ? "FG6D_DATABASE " : storeFaults ? "FG6B_DATABASE " : "FG6A_DATABASE ") + metadata);
|
||||
try (var connection = jdbc.getDataSource().getConnection()) {
|
||||
assertThat(connection.getMetaData().getDatabaseProductName())
|
||||
.containsAnyOf("MySQL", "MariaDB");
|
||||
String product = connection.getMetaData().getDatabaseProductName();
|
||||
if (providerMedia) {
|
||||
assertThat(product).isEqualToIgnoringCase("MySQL");
|
||||
} else {
|
||||
assertThat(product).containsAnyOf("MySQL", "MariaDB");
|
||||
}
|
||||
}
|
||||
}
|
||||
ManagedAgentStore store = spring.getBean(ManagedAgentStore.class);
|
||||
@@ -410,13 +429,19 @@ class HostedWorkspaceToolTurnIT {
|
||||
? new HostedShellOutputProbe(jdbc, tenant, sessions, broker, gateServer) : null;
|
||||
HostedProviderControlProbe providerProbe = providerControl
|
||||
? new HostedProviderControlProbe(jdbc, tenant, sessions, broker, gateServer) : null;
|
||||
HostedProviderMediaProbe mediaProbe = providerMedia
|
||||
? new HostedProviderMediaProbe(jdbc, tenant, sessions, broker, gateServer,
|
||||
() -> new EmbeddedRuntimeBroker(spring.getBean(AgentStateStore.class),
|
||||
spring.getBean(ManagedAgentProperties.class), spring.getBean(RuntimeBindingRepository.class),
|
||||
spring.getBean(RuntimeSessionRepository.class), spring.getBean(ToolExecutionRepository.class),
|
||||
spring.getBean(WorkspaceExecutionStore.class))) : null;
|
||||
HostedWorkspaceColdLoadProbe coldLoadProbe = new HostedWorkspaceColdLoadProbe(jdbc, tenant, sessions,
|
||||
"http://127.0.0.1:" + spring.getWebServer().getPort(), gateServer);
|
||||
HostedOperatorRecoveryProbe operatorProbe = operatorRecovery
|
||||
? new HostedOperatorRecoveryProbe(spring, sessions, temporary.resolve("runtime"), gateServer) : null;
|
||||
gateServer.start();
|
||||
List<String> triggers = new ArrayList<>();
|
||||
try (shellProbe; providerProbe; coldLoadProbe; operatorProbe) {
|
||||
try (shellProbe; providerProbe; mediaProbe; coldLoadProbe; operatorProbe) {
|
||||
if (storeFaults || shellOutput) {
|
||||
for (Map<String, Object> session : sessions) {
|
||||
if (shellOutput ? session.get("fault").equals("receipt-failure")
|
||||
@@ -445,11 +470,12 @@ class HostedWorkspaceToolTurnIT {
|
||||
"integration-tests/helpers/hosted-" + driverName + "-driver.ts", config.toString())
|
||||
.directory(root.toFile()).redirectErrorStream(true).redirectOutput(log.toFile()).start();
|
||||
try {
|
||||
assertThat(driver.waitFor(faults || latency ? 130 : 270, TimeUnit.SECONDS)).as("Driver timeout: %s", Files.readString(log)).isTrue();
|
||||
assertThat(driver.waitFor(providerMedia ? 240 : faults || latency ? 130 : 270, TimeUnit.SECONDS)).as("Driver timeout: %s", Files.readString(log)).isTrue();
|
||||
assertThat(driver.exitValue()).as("Driver output: %s", Files.readString(log)).isZero();
|
||||
System.out.println(Files.readString(log));
|
||||
assertThat(Files.readString(log)).contains(unicode ? "HOSTED_SHELL_UNICODE_OK: " + String.join(",", cases)
|
||||
: operatorRecovery ? "HOSTED_OPERATOR_RECOVERY_OK" : latency ? "HOSTED_LATENCY_OK"
|
||||
: providerMedia ? "HOSTED_PROVIDER_MEDIA_OK"
|
||||
: providerControl ? "HOSTED_PROVIDER_FAULTS_OK"
|
||||
: shellOutput ? "HOSTED_SHELL_OUTPUT_FAULTS_OK"
|
||||
: sseGaps ? "HOSTED_SSE_GAP_OK"
|
||||
@@ -466,7 +492,9 @@ class HostedWorkspaceToolTurnIT {
|
||||
if (operatorRecovery) {
|
||||
continue;
|
||||
}
|
||||
if (unicode) {
|
||||
if (providerMedia) {
|
||||
mediaProbe.assertReport(sessions.get(index), reports.get(index));
|
||||
} else if (unicode) {
|
||||
assertThat(workspace.resolve("child/invalid.txt")).doesNotExist();
|
||||
assertThat(workspace.resolve("child/sibling.txt")).doesNotExist();
|
||||
assertThat(Files.readString(workspace.resolve("child/unicode-ok.txt")))
|
||||
|
||||
+51
-2
@@ -15,7 +15,10 @@ import java.math.BigDecimal;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.net.URI;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Base64;
|
||||
import java.util.HexFormat;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
@@ -53,6 +56,7 @@ class ProviderRuntimeTransportTest {
|
||||
private volatile Object acquireAnswer = true;
|
||||
private volatile UnaryOperator<Map<String, Object>> response = body -> body;
|
||||
private volatile byte[] rawBody;
|
||||
private volatile boolean chunked;
|
||||
/** A result spelled out as JSON text, for number forms a map cannot carry. */
|
||||
private volatile String rawResult;
|
||||
|
||||
@@ -84,8 +88,13 @@ class ProviderRuntimeTransportTest {
|
||||
if (contentEncoding != null) {
|
||||
exchange.getResponseHeaders().set("Content-Encoding", contentEncoding);
|
||||
}
|
||||
exchange.sendResponseHeaders(status, encoded.length);
|
||||
exchange.getResponseBody().write(encoded);
|
||||
exchange.sendResponseHeaders(status, chunked ? 0 : encoded.length);
|
||||
int split = chunked ? encoded.length / 2 : encoded.length;
|
||||
exchange.getResponseBody().write(encoded, 0, split);
|
||||
if (chunked) {
|
||||
exchange.getResponseBody().flush();
|
||||
exchange.getResponseBody().write(encoded, split, encoded.length - split);
|
||||
}
|
||||
exchange.close();
|
||||
});
|
||||
server.start();
|
||||
@@ -93,6 +102,46 @@ class ProviderRuntimeTransportTest {
|
||||
+ server.getAddress().getPort()), "secret", "lease", 3);
|
||||
}
|
||||
|
||||
@Test
|
||||
void preservesBoundedInlineMediaAcrossChunkedExecuteStatusAndCancel() throws Exception {
|
||||
chunked = true;
|
||||
String data = Base64.getEncoder().encodeToString(new byte[576 * 1024]);
|
||||
assertEquals(768 * 1024, data.length());
|
||||
for (String mime : List.of("image/png", "application/pdf")) {
|
||||
Map<String, Object> tool = Map.of("llmContent", List.of(Map.of("inlineData",
|
||||
Map.of("mimeType", mime, "data", data))), "returnDisplay", "media proof");
|
||||
Map<String, Object> execution = Map.of("executionStatus", "success", "result", tool);
|
||||
result = execution;
|
||||
Map<String, Object> received = transport.execute(lease, session, reference()).toCompletableFuture().join();
|
||||
assertMediaResult(execution, received, "execute");
|
||||
result = Map.of("state", "settled", "result", execution, "cancelRequested", false,
|
||||
"lastSeq", 1, "firstAvailableSeq", 0, "progressGap", false, "progress", List.of());
|
||||
Map<String, Object> observed = transport.status(lease, session, reference(), 0).toCompletableFuture().join();
|
||||
assertEquals("settled", observed.get("state"));
|
||||
assertMediaResult(execution, observed.get("result"), "status");
|
||||
Map<String, Object> cancelled = transport.cancel(lease, session, reference()).toCompletableFuture().join();
|
||||
assertEquals("settled", cancelled.get("state"));
|
||||
assertMediaResult(execution, cancelled.get("result"), "cancel");
|
||||
}
|
||||
assertEquals(List.of("execute", "status", "cancel", "execute", "status", "cancel"),
|
||||
requests.stream().map(request -> ProviderRuntimeProtocol.object(request.get("operation")).get("kind")).toList());
|
||||
for (Map<String, Object> request : requests) {
|
||||
Map<String, Object> operation = ProviderRuntimeProtocol.object(request.get("operation"));
|
||||
if (!operation.get("kind").equals("acquire")) {
|
||||
assertTrue(reference().equals(operation.get("reference")), "original seven-field provider reference");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static void assertMediaResult(Object expected, Object actual, String label) throws Exception {
|
||||
byte[] expectedBytes = JsonCodec.encode(expected);
|
||||
byte[] actualBytes = JsonCodec.encode(actual);
|
||||
assertEquals(expectedBytes.length, actualBytes.length, label + " result byte length");
|
||||
MessageDigest sha = MessageDigest.getInstance("SHA-256");
|
||||
assertEquals(HexFormat.of().formatHex(sha.digest(expectedBytes)),
|
||||
HexFormat.of().formatHex(sha.digest(actualBytes)), label + " result SHA256");
|
||||
}
|
||||
|
||||
@Test
|
||||
void rawFileHistoryDoesNotAcquireAProviderSession() {
|
||||
result = Map.of("ownerSessionId", HARNESS, "snapshots", List.of(), "files", Map.of());
|
||||
|
||||
Reference in new issue
Block a user