mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
fix(runner): page oversized assigned tool results (#16980)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work. > - Native agents receive assigned app-tool results through the durable Runner protocol. > - The assigned-tool bridge returned complete provider responses without a size bound. > - A large response could exceed the encrypted command limit after the upstream call had completed. > - The provider then waited for a result that the controller could not deliver. > - This pull request saves large governed responses and returns a small reference with a scoped page reader. > - Agents can retrieve the full result without repeating the original action. ## Linked Issues or Issue Description **What happened?** Large app-tool responses could leave native tool calls pending even though the upstream service completed. The durable command limit is about 510 KiB. The assigned-tool bridge did not account for that limit. A 720,048-byte regression response failed the delivery-size assertion before this change. **Expected behavior** Deliver a bounded response and a way to retrieve all saved content. Keep the original action at one execution. Preserve task, agent, company, responsible-user, and current-run access checks. **Steps to reproduce** 1. Run a native agent with an assigned tool that returns more than 510 KiB of JSON. 2. Let the upstream tool complete. 3. Observe that the original bridge returns the oversized object directly, so the durable controller cannot queue its result. Related: #14734 preserves uncertain tool outcomes across shutdown and restart. This change keeps that protection. #16517 handles native MCP content projection on a different transport; it does not bound native assigned-tool results. ## What Changed - Save governed assigned-tool responses above 500,000 encoded bytes using the existing response asset storage, quotas, and audit path. - Add `read_tool_result` for UTF-8 JSON pages from the same task and agent. Each encoded page is capped at 500,000 bytes, including JSON escaping and metadata, with remaining content available through the next byte offset. It works when generic API tools are disabled. - Reuse existing authenticated asset reads and responsible-user checks. Retrieval does not invoke the upstream service. - Preserve uncertain delivery if storage fails after an action completes. - Add regression, fault, authorization, and opt-in live Codex checks. Document the transport and access contract. ## Verification - Red: the original integration regression returned 720,048 bytes against unchanged production code and failed its bounded-response assertion. The follow-up regression also showed that a 480 KB result was unnecessarily saved under the original 24 KiB policy. - Green: results through 500,000 encoded bytes stay inline. Larger responses are saved in full. Real HTTP integration tests reconstruct the complete Unicode result after controller recreation in exactly two pages, in both standard and low-trust configurations, with one upstream execution. - The encoded-size boundary and escape-heavy Unicode tests verify that every page fits 500,000 bytes, cursors advance on valid UTF-8 boundaries, and concatenating pages exactly reconstructs the original JSON. Storage-failure tests preserve an uncertain outcome without repeating the action. - The focused unit/API run passes 1,957 tests; the changed real HTTP tests pass. `pnpm -r typecheck`, `pnpm build`, `node scripts/check-maps.mjs`, and `git diff --check` pass. - Real Codex acceptance passes with `gpt-6.1-sol` through Codex CLI 0.160.0 and native Runner, on macOS ARM64 with `low_trust_review` and generic API tools disabled. Codex requests every page in order, receives all 660,098 original bytes, and returns the final verification field. The two encoded pages are 500,000 and 160,543 bytes; the upstream operation executes once. Runner closes successfully in a 34.1-second probe. Only the disposable database's connection/worker counts were reduced to fit the host's remaining semaphore capacity. - The native-PRP regression also passes with escape-heavy Unicode pages over 1 MB in total and a separate inline result containing 200,000 backslashes. A scripted provider verifies both complete results by SHA-256. It proves the intermediate text wrapper can exceed 700 KB because the native transport decodes it before queuing the bounded PRP result; every result object stays within 500,000 encoded bytes. Native binaries are required for the local run, preventing a silent skip. - A second real-Codex probe with escaped Unicode, quotes, backslashes, newlines, and NULs also passes: every original byte of a 720,098-byte JSON response is retrieved in two encoded pages of 500,000 and 460,543 bytes. The model returns the final field, the upstream call executes once, and Runner closes in 32.6 seconds. This uses the same production source with only the synthetic fixture content varied. - The live probe exercises real PRP, persistence, and authenticated asset reads. It does not claim the original staging task resumed or full heartbeat checkpoint acceptance. - Earlier broader local testing passed 103 tests but could not start the second database for the existing `rejects a stale source-run question over public HTTP and accepts it from the next run` test. Extending its timeout exposed PostgreSQL initialization failure on this semaphore-exhausted host. The earlier local `pnpm test:run` attempt was interrupted and is not claimed as passing. Full CI passes on current head `cd32237ee5429765b8f349402ac3c79cceec3e6b`. GitHub reports 55 successful checks, two skipped Storybook checks, and one neutral Paperclip agent review. Greptile is 5/5 on this head with no review threads. - Additional Paperclip agent review is incomplete: it withdrew the earlier double-escaping finding after inspecting the native regression, found no new actionable defects, and reported stalled large-source reads prevented independent inspection of the transport decoder. Its neutral assessment is [recorded here](https://github.com/paperclipai/paperclip/pull/16980#issuecomment-6295579705). This limitation remains separate from the passing local native transport, real Codex acceptance, full CI, and Greptile results. ## Risks - Large responses now consume existing response-storage quota and require page reads. Larger pages use more provider context. Results are preserved in full; each encoded page leaves about 22 KiB below the PRP command limit for the command envelope. - A storage failure after execution retains an uncertain delivery outcome. The bridge must not report that the upstream action failed or invite a replay. - The dedicated reader grants access only to saved responses for the current task and agent. The asset route also checks the original responsible user. - Runner command limits and receipt rules remain unchanged. No migration or separate Runner protocol rollout is required. - This PR does not deploy the repair or resume an existing blocked task. ## Model Used OpenAI Codex (GPT-6), with reasoning, tool use, and code execution. The exact serving model ID and context-window size are not exposed in this session. The opt-in acceptance probe requests `gpt-6.1-sol` through Codex CLI 0.160.0. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass for the changed behavior (focused unit/API, real HTTP, and real Codex acceptance); the broader local database limitation is documented above - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <[email protected]>
This commit is contained in:
DottaandPaperclip authored and GitHub committed
1 parent
6b804bfeba
commit
9225796a4e
11 files changed
+608
-38
No files matched your search
@@ -125,6 +125,19 @@ Paperclip plays two roles in the MCP graph, and confusing them is the most commo
|
||||
|
||||
Operators usually mean *gateway* when they say "MCP access governance". For Paperclip-managed local adapter runs, Paperclip writes adapter MCP config that points at named gateway endpoints with short-lived scoped bearer tokens. Policies, approvals, and the audit log only exist for calls that enter gateway mode.
|
||||
|
||||
Native assigned-tool responses larger than 500,000 encoded bytes are saved after gateway
|
||||
validation and redaction. The provider receives a bounded asset reference and
|
||||
uses `read_tool_result` to retrieve UTF-8 JSON pages. The entire result is preserved:
|
||||
each page includes the next byte offset until EOF.
|
||||
The reader fits each encoded page within 500,000 bytes, including JSON escaping,
|
||||
with space left for PRP framing. Reading the saved response
|
||||
does not call the upstream tool again. The reader requires a currently authorized
|
||||
run, the same task and agent, and the existing asset/responsible-user access checks.
|
||||
It works when generic API tools are disabled. Storage uses the existing response
|
||||
capture quotas and audit path. A failure to save a completed action remains an
|
||||
uncertain delivery outcome; it must not invite an automatic repeat of the action.
|
||||
Saved provider content remains untrusted.
|
||||
|
||||
Connected tool names reserve the `mcp__paperclip-assigned__` provider prefix
|
||||
within the 128-character limit. Short existing names stay compatible. Longer
|
||||
names use a readable prefix and a stable hash of the connection, application,
|
||||
|
||||
@@ -1676,6 +1676,16 @@ runnerd recovery, and durable control-plane readers. These paths synchronously
|
||||
read and parse the full JSON file, so memory use and parse time grow with file
|
||||
size. Remote checkpoint archive and expanded-size limits remain 64 MiB.
|
||||
|
||||
Native assigned app tools save governed responses above 500,000 encoded bytes before returning
|
||||
a bounded reference through PRP. `read_tool_result` retrieves the saved JSON in
|
||||
UTF-8 pages under the same task, agent, and responsible-user access checks.
|
||||
Each encoded page fits within 500,000 bytes, including JSON escaping and metadata,
|
||||
leaving space for the PRP command envelope. Follow the next byte offset until EOF
|
||||
to reconstruct the full original JSON; paging never discards remaining content.
|
||||
Retrieval never repeats the upstream action. Storage failure after execution
|
||||
retains an uncertain result-delivery outcome, preserving the no-replay rule.
|
||||
Runner command and receipt size limits remain unchanged.
|
||||
|
||||
Sparse remote workspace checkpoints capture at most eight regular files at once.
|
||||
Each file retains the content hash and mutation checks. A failed capture settles
|
||||
its active readers before closing the manifest and publishes no success receipt.
|
||||
|
||||
@@ -89,6 +89,24 @@ remaining operations before qualifying the whole capability.
|
||||
Gaps: Scoped sessions remain agent/run-bound. MCP requires its existing protocol
|
||||
client; named gateways and gateway-token management retain board permissions.
|
||||
|
||||
Native assigned-tool results above 500,000 encoded bytes are stored as scoped response assets.
|
||||
`read_tool_result` reads bounded JSON pages without repeating the upstream call,
|
||||
including when generic `call_api` is disabled.
|
||||
Each encoded page is at most 500,000 bytes and continues by byte offset until the
|
||||
entire result has been read. Current run authority, same task
|
||||
and agent, and asset/responsible-user access remain required. Provider content
|
||||
stays untrusted. The
|
||||
[response integration tests](../../server/src/services/native-runtime/runner-api.integration.test.ts)
|
||||
cover full Unicode reconstruction after controller recreation, low-trust access,
|
||||
foreign assets, paused agents, and one upstream execution. The
|
||||
[native PRP regression](../../server/src/services/native-runtime/runner-tool-results-prp.integration.test.ts)
|
||||
also verifies complete delivery of escape-heavy pages through native Runner,
|
||||
including a text wrapper larger than the PRP command limit that is decoded before
|
||||
command serialization. The
|
||||
[opt-in live probe](../../server/src/__tests__/helpers/verify-assigned-tool-results-live.ts)
|
||||
uses real Codex and Runner with a synthetic large upstream response; it does not
|
||||
publish to an external service.
|
||||
|
||||
Discovery collapses identical descriptors with the same gateway name across
|
||||
providers, including native assigned-tool startup and fresh searches. Conflicting
|
||||
definitions remain an error. Distinct names and the normal invocation grants,
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
/** Opt-in paid smoke:
|
||||
* PAPERCLIP_LIVE_TOOL_RESULTS=true pnpm --filter @paperclipai/server exec vitest run src/services/native-runtime/runner-tool-results.live.test.ts
|
||||
* Requires a built native runner and an existing Codex login. Uses disposable
|
||||
* database/storage and a private credential copy. The app response is synthetic;
|
||||
* Codex, Runner, PRP, persistence and authenticated asset reads are real.
|
||||
*/
|
||||
import assert from "node:assert/strict";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { copyFile, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { homedir, tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { eq } from "drizzle-orm";
|
||||
import { agents, heartbeatRuns, issues } from "@paperclipai/db";
|
||||
import { startRunnerApiTestServer } from "./runner-api-server.js";
|
||||
import { createAssignedMcpTools } from "../../services/native-runtime/assigned-mcp-tools.js";
|
||||
import { PaperclipRunnerToolAuthority } from "../../services/native-runtime/paperclip-runner-tool-authority.js";
|
||||
import type { ToolGatewayService } from "../../services/tool-gateway.js";
|
||||
import { createRunnerdCodexTransport } from "../../vendor/paperclip-runner/index.js";
|
||||
import { RUNNER_TOOL_RESULT_MAX_BYTES } from "../../services/native-runtime/runner-tool-results.js";
|
||||
|
||||
const root = await mkdtemp(join(tmpdir(), "paperclip-tool-result-live-"));
|
||||
let server: Awaited<ReturnType<typeof startRunnerApiTestServer>> | undefined;
|
||||
let bundle: ReturnType<typeof createRunnerdCodexTransport> | undefined;
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
try {
|
||||
const codexHome = join(root, "codex");
|
||||
await mkdir(codexHome, { mode: 0o700 });
|
||||
await copyFile(join(process.env.PAPERCLIP_LIVE_CODEX_HOME ?? homedir() + "/.codex", "auth.json"), join(codexHome, "auth.json"));
|
||||
await writeFile(join(codexHome, "config.toml"), 'model_reasoning_effort = "low"\n');
|
||||
process.env.PAPERCLIP_HOME = join(root, "instance");
|
||||
process.env.PAPERCLIP_AGENT_JWT_SECRET = randomUUID();
|
||||
process.env.PAPERCLIP_TELEMETRY_DISABLED = "1";
|
||||
server = await startRunnerApiTestServer();
|
||||
const fixture = await server.fixture({ disableWakeOnDemand: true, apiToolsEnabled: false });
|
||||
await server.db.update(agents).set({ permissions: { trustPreset: "low_trust_review", authorizationPolicy: {
|
||||
trustBoundary: { mode: "low_trust_review", companyId: fixture.companyId, issueIds: [fixture.issueId], allowedAgentIds: [fixture.agentId] },
|
||||
} } }).where(eq(agents.id, fixture.agentId));
|
||||
const marker = `verified-${randomUUID()}`;
|
||||
const original = { untrusted: true, content: "x".repeat(660_000), verificationCode: marker };
|
||||
let executions = 0;
|
||||
const assignedMcpTools = await createAssignedMcpTools({
|
||||
gatewayPublicId: "fixture", bearerToken: "fixture",
|
||||
gateway: {
|
||||
listToolsForNamedGateway: async () => [{ name: "fixture:read_review_file", displayName: "Read review file",
|
||||
description: "Read the synthetic review file.", parametersSchema: { type: "object", properties: {}, additionalProperties: false },
|
||||
pluginId: "fixture", providerType: "mcp_remote_http", risk: "read" }],
|
||||
executeTool: async () => { executions++; return { status: "completed", result: original }; },
|
||||
} as unknown as ToolGatewayService,
|
||||
});
|
||||
const authority = new PaperclipRunnerToolAuthority(server.db, { ...fixture.authority.binding, assignedMcpTools });
|
||||
const toolName = assignedMcpTools.definitions()[0]!.name as string;
|
||||
const tools = authority.definitions().filter(tool => [toolName, "read_tool_result"].includes(String(tool.name)));
|
||||
const calls: Array<{ tool: string; resultBytes: number }> = [];
|
||||
const pages: Array<{ offsetBytes: number; nextOffsetBytes: number | null }> = [];
|
||||
let retrieved = "";
|
||||
const started = Date.now();
|
||||
bundle = createRunnerdCodexTransport({
|
||||
codexCommand: process.env.PAPERCLIP_LIVE_CODEX_COMMAND ?? fileURLToPath(new URL("../../../../node_modules/.pnpm/@[email protected]/node_modules/@openai/codex/bin/codex.js", import.meta.url)),
|
||||
sourceCodexHome: codexHome, environment: { ...process.env, CODEX_HOME: codexHome },
|
||||
disableCodexProjectInstructions: true,
|
||||
stateDirectory: join(root, "runner"),
|
||||
lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null },
|
||||
prpIdentity: { runnerInstanceId: "tool-result-live", environmentLeaseId: "tool-result-lease",
|
||||
runId: fixture.runId, normalizedSessionId: "tool-result-session", turnId: "tool-result-turn", itemId: "tool-result-item" },
|
||||
});
|
||||
bundle.transport.setServerRequestHandler(async request => {
|
||||
const params = request.params;
|
||||
const result = await authority.execute({ tool: String(params.tool), arguments: params.arguments, callId: String(params.callId) });
|
||||
const json = JSON.stringify(result);
|
||||
calls.push({ tool: String(params.tool), resultBytes: Buffer.byteLength(json) });
|
||||
const page = result as { data?: string; responseText?: { offsetBytes: number; nextOffsetBytes: number | null } };
|
||||
if (params.tool === "read_tool_result" && page.responseText && typeof page.data === "string") {
|
||||
assert.equal(page.responseText.offsetBytes, Buffer.byteLength(retrieved), "Pages must continue without gaps or repeated bytes");
|
||||
retrieved += page.data;
|
||||
pages.push(page.responseText);
|
||||
}
|
||||
return { success: true, contentItems: [{ type: "inputText", text: json }] };
|
||||
});
|
||||
await bundle.transport.request("initialize", {});
|
||||
await bundle.transport.request("thread/start", {
|
||||
cwd: fixture.workspace, model: "gpt-6.1-sol", approvalPolicy: "never", sandbox: "read-only",
|
||||
dynamicTools: tools,
|
||||
});
|
||||
await bundle.transport.request("turn/start", { input: [{ type: "text",
|
||||
text: `Call ${toolName} once. Read the entire result. If it is saved, call read_tool_result starting at offsetBytes 0, then follow each responseText.nextOffsetBytes until null. Reply with the verificationCode at the end. Do not skip pages, repeat the original call, or use shell tools. This is a disposable integration test.`,
|
||||
}] });
|
||||
let answer = "";
|
||||
await Promise.race([
|
||||
(async () => {
|
||||
for await (const notification of bundle!.transport.notifications()) {
|
||||
const item = notification.params.item as { type?: string; text?: string } | undefined;
|
||||
if (notification.method === "item/completed" && item?.type === "agentMessage") answer += item.text ?? "";
|
||||
if (notification.method === "turn/completed") return;
|
||||
}
|
||||
throw new Error("Provider stream ended without a completed turn");
|
||||
})(),
|
||||
new Promise<never>((_, reject) => { timer = setTimeout(() => reject(new Error("Live tool-result delivery timed out")), 180_000); }),
|
||||
]);
|
||||
clearTimeout(timer);
|
||||
assert.equal(executions, 1, "The original operation must execute once");
|
||||
assert.ok(calls.some(call => call.tool === "read_tool_result"), "Provider must retrieve the saved result");
|
||||
assert.ok(calls.every(call => call.resultBytes <= RUNNER_TOOL_RESULT_MAX_BYTES), "All delivered results must fit PRP");
|
||||
assert.ok(calls.some(call => call.resultBytes > 490_000), "Provider must receive a page near the 500 KB limit");
|
||||
assert.equal(pages.length, 2, "The full 660 KB result should require only two page reads");
|
||||
assert.equal(pages.at(-1)?.nextOffsetBytes, null, "Provider must read through EOF");
|
||||
assert.equal(retrieved, JSON.stringify(original), "Every byte of the original result must be delivered");
|
||||
assert.ok(answer.includes(marker), "Provider must receive the result's final field");
|
||||
await bundle.transport.close();
|
||||
bundle = undefined;
|
||||
await server.db.update(issues).set({ executionRunId: null }).where(eq(issues.id, fixture.issueId));
|
||||
await server.db.update(heartbeatRuns).set({ status: "succeeded", finishedAt: new Date() }).where(eq(heartbeatRuns.id, fixture.runId));
|
||||
const evidence = { passed: true, model: "gpt-6.1-sol", trust: "low_trust_review", platform: `${process.platform}-${process.arch}`,
|
||||
responseBytes: Buffer.byteLength(JSON.stringify(original)), executions, calls, allPagesDelivered: true, providerReceivedFinalField: true,
|
||||
runnerClosed: true, durationMs: Date.now() - started, upstream: "synthetic governed response" };
|
||||
if (process.env.PAPERCLIP_LIVE_EVIDENCE_PATH) await writeFile(process.env.PAPERCLIP_LIVE_EVIDENCE_PATH, JSON.stringify(evidence, null, 2) + "\n");
|
||||
console.log(JSON.stringify(evidence));
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
await bundle?.transport.close();
|
||||
await server?.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
@@ -19,6 +19,7 @@ import { publicChatTaskUrl } from "../chat-task-url.js";
|
||||
import { agentCommentaryToolInputSchema } from "@paperclipai/shared";
|
||||
import { submitAgentCommentary } from "../agent-commentary.js";
|
||||
import type { createAssignedMcpTools } from "./assigned-mcp-tools.js";
|
||||
import { captureRunnerToolResult, fitRunnerToolResultPage, READ_TOOL_RESULT, READ_TOOL_RESULT_DEFINITION, readToolResultSchema, RUNNER_TOOL_RESULT_MAX_BYTES } from "./runner-tool-results.js";
|
||||
import { assertAssignableAgent } from "../agent-assignability.js";
|
||||
import { approvalReadSqlCondition, canActorReadApproval, authorizationService, issueReadSqlCondition, type AuthorizationActor } from "../authorization.js";
|
||||
import { resolveCoreTrustPreset } from "../trust-preset-resolver.js";
|
||||
@@ -51,7 +52,7 @@ import { runnerApiSensitiveResponse } from "./runner-api-sensitive-response.js";
|
||||
import { workspaceFileResourceService } from "../workspace-file-resources.js";
|
||||
import { badRequest, forbidden, notFound, HttpError } from "../../errors.js";
|
||||
import { runnerApiCatalog, searchRunnerApi } from "./runner-api-catalog.js";
|
||||
import { executeRunnerApi, validateRunnerApiCall, RUNNER_API_MAX_BYTES, type RunnerApiFile } from "./runner-api-client.js";
|
||||
import { executeRunnerApi, validateRunnerApiCall, RUNNER_API_MAX_BYTES, type RunnerApiFile, type RunnerApiIo } from "./runner-api-client.js";
|
||||
import { acquireRunnerApiResponseSlot, runnerApiCompanyCaptureMaxBytes, RUNNER_API_RESPONSE_MAX_BYTES, RUNNER_API_RESPONSE_RUN_MAX_BYTES, RunnerApiResponseLimitError } from "./runner-api-response-limits.js";
|
||||
import { and, desc, eq, inArray, isNull, notInArray, sql } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
@@ -312,6 +313,7 @@ export class PaperclipRunnerToolAuthority {
|
||||
definitions.push(...runnerBridgeDefinitions({ workspace: !this.binding.firstPartyOnly && !!this.binding.workspaceRoot && this.binding.workspaceBridge === true, skills: !!this.binding.runtimeContext, api: runnerApiToolsEnabled(this.binding.companyId, this.binding.apiToolsEnabled), mode: workMode }));
|
||||
const connectionTools = this.binding.firstPartyOnly ? [] : [...RUNTIME_CONNECTION_TOOL_DEFINITIONS,
|
||||
...(this.binding.connectorAssignments ?? []).flatMap((assignment) => assignment.tools)];
|
||||
if (!this.binding.firstPartyOnly && this.binding.assignedMcpTools) definitions.push(READ_TOOL_RESULT_DEFINITION);
|
||||
const assignedTools = this.binding.firstPartyOnly ? [] : this.binding.assignedMcpTools?.definitions((tools) =>
|
||||
runnerCodexDynamicToolsFit([...connectionTools, ...tools, ...definitions])) ?? [];
|
||||
const all = [...connectionTools, ...assignedTools, ...definitions];
|
||||
@@ -400,11 +402,13 @@ export class PaperclipRunnerToolAuthority {
|
||||
const { operationId, ...response } = record(await this.#callApi(call.callId, { operationId: operation.operationId, pathParams: { id: input.taskId, ...(input.key ? { key: input.key } : {}) }, ...(body ? { body } : {}) }));
|
||||
return { ...response, apiOperationId: operationId };
|
||||
}
|
||||
if (this.binding.firstPartyOnly && (this.binding.assignedMcpTools?.has(call.tool) || isConnectorTool(call.tool)
|
||||
if (this.binding.firstPartyOnly && (call.tool === READ_TOOL_RESULT || this.binding.assignedMcpTools?.has(call.tool) || isConnectorTool(call.tool)
|
||||
|| RUNTIME_CONNECTION_TOOL_DEFINITIONS.some(tool => tool.name === call.tool))) throw forbidden("This Grok Bot connection exposes first-party Paperclip operations. Use your own provider apps under their existing permissions.");
|
||||
if (call.tool === READ_TOOL_RESULT) return this.#readToolResult(call.arguments);
|
||||
if (this.binding.assignedMcpTools?.has(call.tool)) {
|
||||
const current = await this.#boundContext();
|
||||
return this.binding.assignedMcpTools.execute(call, current.issue.workMode as "standard" | "planning" | "ask");
|
||||
const result = await this.binding.assignedMcpTools.execute(call, current.issue.workMode as "standard" | "planning" | "ask");
|
||||
return captureRunnerToolResult(result, this.#responseCaptureIo("runner.assigned_tool"));
|
||||
}
|
||||
if (isConnectorTool(call.tool)) {
|
||||
if (!(this.binding.connectorAssignments ?? []).some((assignment) => assignment.tools.some((tool) => tool.name === call.tool))) throw forbidden("Connector tool is not available to this run");
|
||||
@@ -809,8 +813,6 @@ export class PaperclipRunnerToolAuthority {
|
||||
const token = this.#apiToken(bound);
|
||||
if (!token) throw new Error("Paperclip run authentication is unavailable");
|
||||
const execute = async () => {
|
||||
let reservationId: string | undefined;
|
||||
let storageAttempted = false;
|
||||
const current = await this.#boundContext();
|
||||
if (!runnerApiToolsEnabled(this.binding.companyId, this.binding.apiToolsEnabled)) throw new Error("paperclip_runner_tool_not_advertised");
|
||||
const result = await executeRunnerApi(input, { ...context, workMode: current.issue.workMode }, {
|
||||
@@ -822,38 +824,7 @@ export class PaperclipRunnerToolAuthority {
|
||||
await this.#assertDotApiFileAccess(operation.path, input);
|
||||
},
|
||||
readFile: (file) => this.#readApiFile(file),
|
||||
reserveResponseCapture: async () => {
|
||||
const reservation = await this.#reserveApiResponseCapture();
|
||||
reservationId = reservation.id;
|
||||
return (completedBytes, cleaned) => reservation.settle(completedBytes, cleaned, storageAttempted);
|
||||
},
|
||||
saveResponse: async (bytes, contentType) => {
|
||||
if (!reservationId) throw new Error("API response storage reservation is missing");
|
||||
const storage = this.binding.storage ?? getStorageService();
|
||||
storageAttempted = true;
|
||||
const saved = await storage.putFile({ companyId: this.binding.companyId, namespace: "runner-api", originalFilename: contentType.includes("json") ? "response.json" : "response.bin", contentType, ...(Buffer.isBuffer(bytes) ? { body: bytes } : { body: createReadStream(bytes.path), byteSize: bytes.byteSize, sha256: bytes.sha256 }) });
|
||||
const asset = await this.db.transaction(async tx => {
|
||||
await this.#lockApiCaptureCompany(tx as unknown as Db);
|
||||
const asset = await assetService(tx as unknown as Db).create(this.binding.companyId, { ...saved, createdByAgentId: this.binding.agentId });
|
||||
await tx.update(runnerApiResponseReservations).set({ assetId: asset.id, reservedBytes: asset.byteSize }).where(and(eq(runnerApiResponseReservations.id, reservationId!), eq(runnerApiResponseReservations.companyId, this.binding.companyId)));
|
||||
return asset;
|
||||
}).catch(async error => {
|
||||
// A failed commit acknowledgement can still leave a committed
|
||||
// asset. A locking read waits out that transaction before proving
|
||||
// the reservation is unlinked and safe to compensate.
|
||||
const reservation = await this.db.select({ assetId: runnerApiResponseReservations.assetId })
|
||||
.from(runnerApiResponseReservations).where(and(eq(runnerApiResponseReservations.id, reservationId!), eq(runnerApiResponseReservations.companyId, this.binding.companyId))).for("update")
|
||||
.then(rows => rows[0]).catch(() => undefined);
|
||||
if (reservation?.assetId === null) {
|
||||
const removed = await storage.deleteObject(this.binding.companyId, saved.objectKey).then(() => true, () => false);
|
||||
if (removed) storageAttempted = false;
|
||||
}
|
||||
throw error;
|
||||
});
|
||||
const activity = await persistActivity(this.db, { companyId: this.binding.companyId, actorType: "agent", actorId: this.binding.agentId, agentId: this.binding.agentId, runId: this.binding.runId, issueId: this.binding.issueId, action: "asset.created", entityType: "asset", entityId: asset.id, details: { source: "runner.call_api", byteSize: saved.byteSize } });
|
||||
publishActivity(activity.publication);
|
||||
return { artifactId: asset.id, url: `/api/assets/${asset.id}/content`, contentType, byteSize: saved.byteSize, sha256: saved.sha256 };
|
||||
},
|
||||
...this.#responseCaptureIo("runner.call_api"),
|
||||
});
|
||||
await this.#assertDotApiFileAccess(operation.path, input);
|
||||
return result;
|
||||
@@ -909,6 +880,82 @@ export class PaperclipRunnerToolAuthority {
|
||||
return result;
|
||||
}
|
||||
|
||||
async #readToolResult(value: unknown): Promise<unknown> {
|
||||
if (!this.binding.assignedMcpTools) throw forbidden("Saved app-tool results are unavailable to this run");
|
||||
const input = readToolResultSchema.parse(value);
|
||||
const context = await this.#boundContext();
|
||||
const authorize = async () => {
|
||||
await this.#boundContext();
|
||||
const [capture] = await this.db.select({ id: runnerApiResponseReservations.id })
|
||||
.from(runnerApiResponseReservations)
|
||||
.innerJoin(heartbeatRuns, eq(heartbeatRuns.id, runnerApiResponseReservations.runId))
|
||||
.where(and(
|
||||
eq(runnerApiResponseReservations.assetId, input.artifactId),
|
||||
eq(runnerApiResponseReservations.companyId, this.binding.companyId),
|
||||
eq(heartbeatRuns.companyId, this.binding.companyId),
|
||||
eq(heartbeatRuns.agentId, this.binding.agentId),
|
||||
eq(heartbeatRuns.nativeIssueId, this.binding.issueId),
|
||||
)).limit(1);
|
||||
if (!capture) throw forbidden("Saved tool result is not available to this task");
|
||||
};
|
||||
await authorize();
|
||||
const apiUrl = this.binding.apiUrl ?? process.env.PAPERCLIP_API_URL;
|
||||
const token = this.#apiToken(context);
|
||||
if (!apiUrl || !token) throw new Error("Paperclip run authentication is unavailable");
|
||||
// This dedicated read grants only a saved result from this task. It remains
|
||||
// available without generic call_api and uses the same authenticated asset
|
||||
// route, range/UTF-8 decoding, and responsible-user checks as API reads.
|
||||
const { operationId, ...response } = await executeRunnerApi({
|
||||
operationId: "GET /api/assets/{assetId}/content",
|
||||
pathParams: { assetId: input.artifactId },
|
||||
responseText: { offsetBytes: input.offsetBytes },
|
||||
}, { ...this.binding, workMode: context.issue.workMode }, {
|
||||
apiUrl, token, beforeDispatch: authorize, responseTextPageBytes: RUNNER_TOOL_RESULT_MAX_BYTES,
|
||||
readFile: async () => { throw new Error("Saved result reads cannot upload files"); },
|
||||
saveResponse: async () => { throw new Error("Saved result reads cannot create assets"); },
|
||||
});
|
||||
return fitRunnerToolResultPage({ ...response, apiOperationId: operationId });
|
||||
}
|
||||
|
||||
#responseCaptureIo(source: "runner.call_api" | "runner.assigned_tool"): Pick<RunnerApiIo, "reserveResponseCapture" | "saveResponse"> {
|
||||
let reservationId: string | undefined;
|
||||
let storageAttempted = false;
|
||||
return {
|
||||
reserveResponseCapture: async () => {
|
||||
const reservation = await this.#reserveApiResponseCapture();
|
||||
reservationId = reservation.id;
|
||||
return (completedBytes, cleaned) => reservation.settle(completedBytes, cleaned, storageAttempted);
|
||||
},
|
||||
saveResponse: async (bytes, contentType) => {
|
||||
if (!reservationId) throw new Error("API response storage reservation is missing");
|
||||
const storage = this.binding.storage ?? getStorageService();
|
||||
storageAttempted = true;
|
||||
const saved = await storage.putFile({ companyId: this.binding.companyId, namespace: "runner-api", originalFilename: contentType.includes("json") ? "response.json" : "response.bin", contentType, ...(Buffer.isBuffer(bytes) ? { body: bytes } : { body: createReadStream(bytes.path), byteSize: bytes.byteSize, sha256: bytes.sha256 }) });
|
||||
const asset = await this.db.transaction(async tx => {
|
||||
await this.#lockApiCaptureCompany(tx as unknown as Db);
|
||||
const asset = await assetService(tx as unknown as Db).create(this.binding.companyId, { ...saved, createdByAgentId: this.binding.agentId });
|
||||
await tx.update(runnerApiResponseReservations).set({ assetId: asset.id, reservedBytes: asset.byteSize }).where(and(eq(runnerApiResponseReservations.id, reservationId!), eq(runnerApiResponseReservations.companyId, this.binding.companyId)));
|
||||
return asset;
|
||||
}).catch(async error => {
|
||||
// A failed commit acknowledgement can still leave a committed
|
||||
// asset. A locking read waits out that transaction before proving
|
||||
// the reservation is unlinked and safe to compensate.
|
||||
const reservation = await this.db.select({ assetId: runnerApiResponseReservations.assetId })
|
||||
.from(runnerApiResponseReservations).where(and(eq(runnerApiResponseReservations.id, reservationId!), eq(runnerApiResponseReservations.companyId, this.binding.companyId))).for("update")
|
||||
.then(rows => rows[0]).catch(() => undefined);
|
||||
if (reservation?.assetId === null) {
|
||||
const removed = await storage.deleteObject(this.binding.companyId, saved.objectKey).then(() => true, () => false);
|
||||
if (removed) storageAttempted = false;
|
||||
}
|
||||
throw error;
|
||||
});
|
||||
const activity = await persistActivity(this.db, { companyId: this.binding.companyId, actorType: "agent", actorId: this.binding.agentId, agentId: this.binding.agentId, runId: this.binding.runId, issueId: this.binding.issueId, action: "asset.created", entityType: "asset", entityId: asset.id, details: { source, byteSize: saved.byteSize } });
|
||||
publishActivity(activity.publication);
|
||||
return { artifactId: asset.id, url: `/api/assets/${asset.id}/content`, contentType, byteSize: saved.byteSize, sha256: saved.sha256 };
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async #lockApiCaptureCompany(tx: Db): Promise<void> {
|
||||
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${`runner-api-capture:${this.binding.companyId}`}, 0))`);
|
||||
}
|
||||
|
||||
@@ -39,6 +39,8 @@ export interface RunnerApiIo {
|
||||
apiUrl: string;
|
||||
token: string;
|
||||
fetch?: typeof fetch;
|
||||
/** Server-owned page size for the dedicated saved-result reader. */
|
||||
responseTextPageBytes?: number;
|
||||
/** Revalidate server-owned authority after asynchronous file preparation. */
|
||||
beforeDispatch?(): Promise<void>;
|
||||
/** Reserve before spilling. Settle completed bytes; failed captures retain their reservation. */
|
||||
@@ -127,7 +129,8 @@ export async function executeRunnerApi(input: RunnerApiCall, context: RunnerApiC
|
||||
if (!io.token) throw new Error("Paperclip run authentication is unavailable");
|
||||
const headers = new Headers({ Authorization: `Bearer ${io.token}`, "X-Paperclip-Run-Id": context.runId });
|
||||
const textOffset = input.responseText?.offsetBytes ?? 0;
|
||||
const textLimit = input.responseText?.limitBytes ?? RUNNER_API_INLINE_BYTES;
|
||||
const textLimit = input.responseText?.limitBytes
|
||||
?? (input.responseText ? io.responseTextPageBytes : undefined) ?? RUNNER_API_INLINE_BYTES;
|
||||
const assetTextRange = operation.operationId === "GET /api/assets/{assetId}/content"
|
||||
? { start: Math.max(0, textOffset - 1), end: Math.min(Number.MAX_SAFE_INTEGER, textOffset + textLimit) }
|
||||
: undefined;
|
||||
|
||||
@@ -15,6 +15,9 @@ import { createLocalAgentJwt } from "../../agent-auth-jwt.js";
|
||||
import { RUNNER_API_RESPONSE_MAX_BYTES, RUNNER_API_RESPONSE_RUN_MAX_BYTES } from "./runner-api-response-limits.js";
|
||||
import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js";
|
||||
import { prepareHeartbeatTaskContext } from "../heartbeat/task-context.js";
|
||||
import { createAssignedMcpTools } from "./assigned-mcp-tools.js";
|
||||
import type { ToolGatewayService } from "../tool-gateway.js";
|
||||
import { RUNNER_TOOL_RESULT_MAX_BYTES } from "./runner-tool-results.js";
|
||||
|
||||
describe("runner API against real HTTP routes", () => {
|
||||
let server: Awaited<ReturnType<typeof startRunnerApiTestServer>>;
|
||||
@@ -38,6 +41,62 @@ describe("runner API against real HTTP routes", () => {
|
||||
else process.env.PAPERCLIP_AGENT_JWT_SECRET = oldSecret;
|
||||
});
|
||||
|
||||
it.each(["standard", "low_trust_review"])("delivers oversized assigned results without repeating the action (%s)", async trust => {
|
||||
const fixture = await server.fixture({ apiToolsEnabled: false });
|
||||
if (trust === "low_trust_review") {
|
||||
await server.db.update(agents).set({ permissions: { trustPreset: trust, authorizationPolicy: {
|
||||
trustBoundary: { mode: trust, companyId: fixture.companyId, issueIds: [fixture.issueId], allowedAgentIds: [fixture.agentId] },
|
||||
} } }).where(eq(agents.id, fixture.agentId));
|
||||
}
|
||||
const original = { data: { content: "🧭é".repeat(120_000), last: "complete-result" } };
|
||||
const executeTool = vi.fn().mockResolvedValue({ status: "completed", result: original });
|
||||
const assignedMcpTools = await createAssignedMcpTools({
|
||||
gatewayPublicId: "fixture", bearerToken: "fixture",
|
||||
gateway: {
|
||||
listToolsForNamedGateway: async () => [{ name: "app:write", displayName: "Write", description: "Fixture action",
|
||||
parametersSchema: { type: "object" }, pluginId: "fixture", providerType: "mcp_remote_http", risk: "write" }],
|
||||
executeTool,
|
||||
} as unknown as ToolGatewayService,
|
||||
});
|
||||
const binding = { ...fixture.authority.binding, assignedMcpTools };
|
||||
const authority = new PaperclipRunnerToolAuthority(server.db, binding);
|
||||
const result: any = await authority.execute({ tool: assignedMcpTools.definitions()[0]!.name as string, callId: "large-action", arguments: {} });
|
||||
expect(Buffer.byteLength(JSON.stringify(result))).toBeLessThan(32 * 1024);
|
||||
expect(result).toMatchObject({ artifact: { artifactId: expect.any(String) }, readWith: { tool: "read_tool_result" } });
|
||||
// Recreate the authority to prove that pages do not depend on a live cache.
|
||||
const reopened = new PaperclipRunnerToolAuthority(server.db, binding);
|
||||
expect(reopened.definitions().some(tool => tool.name === "read_tool_result")).toBe(true);
|
||||
let offsetBytes: number | null = 0;
|
||||
let text = "";
|
||||
let pages = 0;
|
||||
do {
|
||||
const page: any = await reopened.execute({ tool: "read_tool_result", callId: randomUUID(),
|
||||
arguments: { artifactId: result.artifact.artifactId, offsetBytes } });
|
||||
expect(page).not.toHaveProperty("operationId");
|
||||
expect(page.ok).toBe(true);
|
||||
expect(Buffer.byteLength(JSON.stringify(page))).toBeLessThanOrEqual(RUNNER_TOOL_RESULT_MAX_BYTES);
|
||||
if (pages === 0) expect(Buffer.byteLength(page.data)).toBeGreaterThan(490_000);
|
||||
pages++;
|
||||
text += page.data;
|
||||
offsetBytes = page.responseText.nextOffsetBytes;
|
||||
} while (offsetBytes !== null);
|
||||
expect(JSON.parse(text)).toEqual(original);
|
||||
expect(pages).toBe(2);
|
||||
expect(executeTool).toHaveBeenCalledOnce();
|
||||
expect((await fixture.snapshot()).assets.filter(asset => asset.id === result.artifact.artifactId)).toHaveLength(1);
|
||||
const foreign = await server.fixture();
|
||||
const foreignAuthority = new PaperclipRunnerToolAuthority(server.db, { ...foreign.authority.binding, assignedMcpTools });
|
||||
await expect(foreignAuthority.execute({ tool: "read_tool_result", callId: "foreign",
|
||||
arguments: { artifactId: result.artifact.artifactId } })).rejects.toThrow();
|
||||
await expect(reopened.execute({ tool: "read_tool_result", callId: "unrelated",
|
||||
arguments: { artifactId: fixture.artifactId } })).rejects.toThrow("not available to this task");
|
||||
await expect(reopened.execute({ tool: "call_api", callId: "disabled",
|
||||
arguments: { operationId: "GET /api/companies/{companyId}/projects" } })).rejects.toThrow("not_advertised");
|
||||
await server.db.update(agents).set({ status: "paused" }).where(eq(agents.id, fixture.agentId));
|
||||
await expect(reopened.execute({ tool: "read_tool_result", callId: "paused",
|
||||
arguments: { artifactId: result.artifact.artifactId } })).rejects.toThrow("binding_not_authorized");
|
||||
});
|
||||
|
||||
it.skipIf(!process.env.PAPERCLIP_REQUIRE_RUNNER_API_INTEGRATION && !existsSync(defaultCapabilityRunnerdBinary())).each(["current", "legacy_http"])("runs runnerd → PRP → authority → actual authenticated HTTP (%s receipt)", async (receiptFormat) => {
|
||||
const fixture = await server.fixture();
|
||||
const provider = join(server.root, `scripted-api-provider-${receiptFormat}.mjs`);
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
import { existsSync } from "node:fs";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { chmod, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { afterAll, beforeAll, describe, expect, it, vi } from "vitest";
|
||||
import { startRunnerApiTestServer } from "../../__tests__/helpers/runner-api-server.js";
|
||||
import { createRunnerdCodexTransport, defaultCapabilityRunnerdBinary } from "../../vendor/paperclip-runner/index.js";
|
||||
import { createAssignedMcpTools } from "./assigned-mcp-tools.js";
|
||||
import { PaperclipRunnerToolAuthority } from "./paperclip-runner-tool-authority.js";
|
||||
import { RUNNER_TOOL_RESULT_MAX_BYTES } from "./runner-tool-results.js";
|
||||
import type { ToolGatewayService } from "../tool-gateway.js";
|
||||
|
||||
describe("saved assigned results through actual PRP", () => {
|
||||
let server: Awaited<ReturnType<typeof startRunnerApiTestServer>>;
|
||||
const oldSecret = process.env.PAPERCLIP_AGENT_JWT_SECRET;
|
||||
beforeAll(async () => {
|
||||
process.env.PAPERCLIP_AGENT_JWT_SECRET = randomUUID();
|
||||
server = await startRunnerApiTestServer();
|
||||
}, 60_000);
|
||||
afterAll(async () => {
|
||||
await server?.close();
|
||||
if (oldSecret === undefined) delete process.env.PAPERCLIP_AGENT_JWT_SECRET;
|
||||
else process.env.PAPERCLIP_AGENT_JWT_SECRET = oldSecret;
|
||||
});
|
||||
|
||||
it.skipIf(!process.env.PAPERCLIP_REQUIRE_RUNNER_API_INTEGRATION && !existsSync(defaultCapabilityRunnerdBinary()))("delivers escape-heavy 500 KB pages through actual PRP without truncation", async () => {
|
||||
const fixture = await server.fixture({ apiToolsEnabled: false });
|
||||
const original = { content: '🧭é"\\\n\u0000'.repeat(60_000), last: "complete-result" };
|
||||
const expected = JSON.stringify(original);
|
||||
const digest = createHash("sha256").update(expected).digest("hex");
|
||||
const inline = { content: "\\".repeat(200_000) };
|
||||
const inlineDigest = createHash("sha256").update(JSON.stringify(inline)).digest("hex");
|
||||
const executeTool = vi.fn(async (call: { parameters: Record<string, unknown> }) => ({
|
||||
status: "completed" as const, result: call.parameters.variant === "inline" ? inline : original,
|
||||
}));
|
||||
const assignedMcpTools = await createAssignedMcpTools({
|
||||
gatewayPublicId: "fixture", bearerToken: "fixture",
|
||||
gateway: {
|
||||
listToolsForNamedGateway: async () => [{ name: "app:read", displayName: "Read", description: "Fixture read",
|
||||
parametersSchema: { type: "object", properties: { variant: { type: "string", enum: ["large", "inline"] } } },
|
||||
pluginId: "fixture", providerType: "mcp_remote_http", risk: "read" }],
|
||||
executeTool,
|
||||
} as unknown as ToolGatewayService,
|
||||
});
|
||||
const authority = new PaperclipRunnerToolAuthority(server.db, { ...fixture.authority.binding, assignedMcpTools });
|
||||
const provider = join(server.root, "scripted-saved-result-provider.mjs");
|
||||
await writeFile(provider, `#!${process.execPath}
|
||||
import { createInterface } from 'node:readline';
|
||||
import { createHash } from 'node:crypto';
|
||||
const send = value => process.stdout.write(JSON.stringify(value)+'\\n');
|
||||
let step=0, artifactId, restored='', inlinePending=false;
|
||||
const call = (tool,args) => { step++; send({id:'call-'+step,method:'item/tool/call',params:{threadId:'saved-thread',turnId:'saved-turn',itemId:'item-'+step,callId:'saved-call-'+step,tool,arguments:args}}); };
|
||||
for await (const line of createInterface({input:process.stdin})) {
|
||||
const m=JSON.parse(line);
|
||||
if (!m.method) {
|
||||
if (String(m.id).startsWith('call-')) {
|
||||
const envelope=JSON.parse(m.result.contentItems[0].text);
|
||||
const result=envelope.result ?? envelope;
|
||||
if (inlinePending) {
|
||||
if(result.artifact || createHash('sha256').update(JSON.stringify(result)).digest('hex')!=='${inlineDigest}') throw new Error('Inline result was not delivered intact');
|
||||
send({method:'item/completed',params:{threadId:'saved-thread',turnId:'saved-turn',item:{id:'saved-answer',type:'agentMessage',text:'all-pages-verified'}}});
|
||||
send({method:'turn/completed',params:{turn:{id:'saved-turn',status:'completed'}}});
|
||||
}
|
||||
else if (step===1) { artifactId=result.artifact.artifactId; call('read_tool_result',{artifactId,offsetBytes:0}); }
|
||||
else {
|
||||
if(!result.ok || result.responseText.offsetBytes!==Buffer.byteLength(restored)) throw new Error('Invalid page receipt');
|
||||
restored+=result.data;
|
||||
if(result.responseText.nextOffsetBytes!==null) call('read_tool_result',{artifactId,offsetBytes:result.responseText.nextOffsetBytes});
|
||||
else {
|
||||
if(createHash('sha256').update(restored).digest('hex')!=='${digest}') throw new Error('Provider received incomplete JSON');
|
||||
inlinePending=true;
|
||||
call('${assignedMcpTools.definitions()[0]!.name}',{variant:'inline'});
|
||||
}
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if(m.method==='initialize') send({id:m.id,result:{userAgent:'scripted-saved-result-provider'}});
|
||||
else if(m.method==='thread/start') send({id:m.id,result:{thread:{id:'saved-thread',sessionId:'saved-session'}}});
|
||||
else if(m.method==='turn/start') {send({id:m.id,result:{turn:{id:'saved-turn',status:'inProgress'}}});send({method:'turn/started',params:{turn:{id:'saved-turn'}}});call('${assignedMcpTools.definitions()[0]!.name}',{variant:'large'});}
|
||||
else if(m.id!==undefined) send({id:m.id,result:{}});
|
||||
}
|
||||
`);
|
||||
await chmod(provider, 0o700);
|
||||
const bundle = createRunnerdCodexTransport({
|
||||
runnerBinary: defaultCapabilityRunnerdBinary(), codexCommand: provider, codexArgs: [],
|
||||
stateDirectory: join(server.root, "scripted-saved-result-runner"), lifecyclePolicy: { mode: "per_turn", idleTimeoutMs: null },
|
||||
prpIdentity: { runnerInstanceId: "saved-test", environmentLeaseId: "saved-test-lease", runId: fixture.runId,
|
||||
normalizedSessionId: "saved-test-session", turnId: "saved-test-turn", itemId: "saved-test-item" },
|
||||
});
|
||||
const sizes: Array<{ result: number; wrapper: number }> = [];
|
||||
bundle.transport.setServerRequestHandler(async request => {
|
||||
const params = request.params;
|
||||
const result = await authority.execute({ tool: String(params.tool), arguments: params.arguments, callId: String(params.callId) });
|
||||
const response = { success: true, contentItems: [{ type: "inputText", text: JSON.stringify(result) }] };
|
||||
sizes.push({ result: Buffer.byteLength(JSON.stringify(result)), wrapper: Buffer.byteLength(JSON.stringify(response)) });
|
||||
return response;
|
||||
});
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
let verified = false;
|
||||
const notifications: Array<{ method: string; type: unknown; status?: string }> = [];
|
||||
try {
|
||||
await bundle.transport.request("initialize", {});
|
||||
await bundle.transport.request("thread/start", { cwd: fixture.workspace, dynamicTools: authority.definitions() });
|
||||
await bundle.transport.request("turn/start", { input: [{ type: "text", text: "Read every saved result page" }] });
|
||||
await Promise.race([
|
||||
(async () => { for await (const notification of bundle.transport.notifications()) {
|
||||
notifications.push({ method: notification.method, type: (notification.params.item as { type?: string })?.type,
|
||||
status: (notification.params.turn as { status?: string })?.status });
|
||||
if (notification.method === "item/completed" && (notification.params.item as { text?: string })?.text === "all-pages-verified") verified = true;
|
||||
if (notification.method === "turn/completed") return;
|
||||
} })(),
|
||||
new Promise<never>((_, reject) => { timer = setTimeout(() => reject(new Error("Escaped pages did not reach the provider")), 15_000); }),
|
||||
]);
|
||||
expect(verified, JSON.stringify({ sizes, notifications, diagnostics: bundle.evidence().diagnostics })).toBe(true);
|
||||
expect(executeTool.mock.calls.map(([call]) => call.parameters)).toEqual([{ variant: "large" }, { variant: "inline" }]);
|
||||
expect(sizes.every(size => size.result <= RUNNER_TOOL_RESULT_MAX_BYTES)).toBe(true);
|
||||
// The provider-facing text wrapper is decoded before the PRP command.
|
||||
// Its larger JSON encoding must not be mistaken for the command's size.
|
||||
expect(sizes.some(size => size.wrapper > 700_000)).toBe(true);
|
||||
expect(bundle.evidence().diagnostics).toContain("runnerd authenticated to the durable PRP control plane");
|
||||
} finally { clearTimeout(timer); await bundle.transport.close(); }
|
||||
}, 30_000);
|
||||
});
|
||||
@@ -0,0 +1,6 @@
|
||||
import { it } from "vitest";
|
||||
|
||||
// Opt-in only: this uses the existing Codex login and makes a paid model call.
|
||||
it.skipIf(process.env.PAPERCLIP_LIVE_TOOL_RESULTS !== "true")("delivers a large assigned result to live Codex through Runner", async () => {
|
||||
await import("../../__tests__/helpers/verify-assigned-tool-results-live.js");
|
||||
}, 240_000);
|
||||
@@ -0,0 +1,79 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { SemanticToolOutcomeUnknownError } from "../../vendor/paperclip-runner/index.js";
|
||||
import { captureRunnerToolResult, fitRunnerToolResultPage, RUNNER_TOOL_RESULT_MAX_BYTES } from "./runner-tool-results.js";
|
||||
|
||||
describe("assigned tool result capture", () => {
|
||||
it("leaves small governed results inline without storage admission", async () => {
|
||||
const result = { data: { untrusted: true, content: "small result" } };
|
||||
const io = { reserveResponseCapture: vi.fn(), saveResponse: vi.fn() };
|
||||
expect(await captureRunnerToolResult(result, io)).toBe(result);
|
||||
expect(io.reserveResponseCapture).not.toHaveBeenCalled();
|
||||
expect(io.saveResponse).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("delivers a 480 KB result inline without extra page reads", async () => {
|
||||
const result = { content: "x".repeat(480_000) };
|
||||
const io = { reserveResponseCapture: vi.fn(async () => async () => {}),
|
||||
saveResponse: vi.fn(async () => ({ artifactId: "saved" })) };
|
||||
expect(await captureRunnerToolResult(result, io)).toBe(result);
|
||||
expect(io.saveResponse).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([500_000, 500_001])("captures only results above the encoded budget (%s bytes)", async byteSize => {
|
||||
const result = { content: "x".repeat(byteSize - Buffer.byteLength(JSON.stringify({ content: "" }))) };
|
||||
const saveResponse = vi.fn(async (bytes: Buffer) => ({ artifactId: "saved", byteSize: bytes.length }));
|
||||
const response = await captureRunnerToolResult(result, {
|
||||
reserveResponseCapture: async () => async () => {}, saveResponse,
|
||||
});
|
||||
if (byteSize === 500_000) {
|
||||
expect(response).toBe(result);
|
||||
expect(saveResponse).not.toHaveBeenCalled();
|
||||
} else {
|
||||
expect(response).toMatchObject({ artifact: { byteSize } });
|
||||
expect(saveResponse.mock.calls[0]![0].toString("utf8")).toBe(JSON.stringify(result));
|
||||
}
|
||||
});
|
||||
|
||||
it("fits escape-heavy Unicode pages without losing text or corrupting offsets", () => {
|
||||
const original = JSON.stringify({ content: '🧭é"\\\n\u0000'.repeat(60_000), last: "complete-result" });
|
||||
const bytes = Buffer.from(original);
|
||||
let offsetBytes = 0;
|
||||
let restored = "";
|
||||
let pages = 0;
|
||||
do {
|
||||
// Match the HTTP reader's UTF-8-safe raw byte window.
|
||||
let end = Math.min(offsetBytes + 500_000, bytes.length);
|
||||
while (end < bytes.length && (bytes[end]! & 0xc0) === 0x80) end--;
|
||||
const response = fitRunnerToolResultPage({ ok: true,
|
||||
responseText: { offsetBytes, nextOffsetBytes: end < bytes.length ? end : null, totalBytes: bytes.length },
|
||||
data: bytes.subarray(offsetBytes, end).toString("utf8") });
|
||||
const encoded = Buffer.byteLength(JSON.stringify(response));
|
||||
expect(encoded).toBeLessThanOrEqual(RUNNER_TOOL_RESULT_MAX_BYTES);
|
||||
if (response.responseText!.nextOffsetBytes !== null) expect(encoded).toBeGreaterThan(499_980);
|
||||
restored += response.data;
|
||||
pages++;
|
||||
const next = response.responseText!.nextOffsetBytes;
|
||||
if (next === null) break;
|
||||
expect(next).toBeGreaterThan(offsetBytes);
|
||||
expect(next).toBe(Buffer.byteLength(restored));
|
||||
offsetBytes = next;
|
||||
} while (pages < 20);
|
||||
expect(restored).toBe(original);
|
||||
expect(JSON.parse(restored)).toEqual(JSON.parse(original));
|
||||
expect(pages).toBeLessThan(10);
|
||||
});
|
||||
|
||||
it.each(["reserve", "save", "settle"])("preserves completed actions as uncertain when %s fails", async stage => {
|
||||
const fail = async () => { throw new Error("private provider response must not be exposed"); };
|
||||
const settle = vi.fn(stage === "settle" ? fail : async () => undefined);
|
||||
const io = {
|
||||
reserveResponseCapture: vi.fn(stage === "reserve" ? fail : async () => settle),
|
||||
saveResponse: vi.fn(stage === "save" ? fail : async () => ({ artifactId: "saved" })),
|
||||
};
|
||||
const error = await captureRunnerToolResult({ content: "x".repeat(600_000) }, io).catch(error => error);
|
||||
expect(error).toBeInstanceOf(SemanticToolOutcomeUnknownError);
|
||||
expect(error).toMatchObject({ message: "The completed app-tool result could not be saved." });
|
||||
expect(io.reserveResponseCapture).toHaveBeenCalledOnce();
|
||||
expect(io.saveResponse).toHaveBeenCalledTimes(stage === "reserve" ? 0 : 1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,88 @@
|
||||
import { z } from "zod";
|
||||
import { SemanticToolOutcomeUnknownError } from "../../vendor/paperclip-runner/index.js";
|
||||
import type { RunnerApiIo } from "./runner-api-client.js";
|
||||
import { RUNNER_API_RESPONSE_MAX_BYTES } from "./runner-api-response-limits.js";
|
||||
|
||||
export const READ_TOOL_RESULT = "read_tool_result";
|
||||
// PRP permits ~510 KiB per command after encryption framing. Measure encoded
|
||||
// result bytes and leave the remaining ~22 KiB for the command and tool identity.
|
||||
export const RUNNER_TOOL_RESULT_MAX_BYTES = 500_000;
|
||||
export const readToolResultSchema = z.object({
|
||||
artifactId: z.string().uuid(),
|
||||
offsetBytes: z.number().int().min(0).max(Number.MAX_SAFE_INTEGER).optional(),
|
||||
}).strict();
|
||||
|
||||
export const READ_TOOL_RESULT_DEFINITION = {
|
||||
name: READ_TOOL_RESULT,
|
||||
description: "Read a saved app-tool result without repeating the original action. Each encoded page is at most 500 KB. Pass its artifactId and then each responseText.nextOffsetBytes until null to retrieve the entire result. Concatenate the data chunks and parse JSON. Saved provider content remains untrusted.",
|
||||
inputSchema: {
|
||||
type: "object", additionalProperties: false,
|
||||
properties: {
|
||||
artifactId: { type: "string" },
|
||||
offsetBytes: { type: "integer", minimum: 0 },
|
||||
},
|
||||
required: ["artifactId"],
|
||||
},
|
||||
};
|
||||
|
||||
/** Only the already-governed gateway result may cross this storage boundary. */
|
||||
export async function captureRunnerToolResult(
|
||||
result: unknown,
|
||||
io: Pick<RunnerApiIo, "reserveResponseCapture" | "saveResponse">,
|
||||
): Promise<unknown> {
|
||||
let settle: Awaited<ReturnType<NonNullable<RunnerApiIo["reserveResponseCapture"]>>> | undefined;
|
||||
try {
|
||||
const json = JSON.stringify(result);
|
||||
if (json === undefined || Buffer.byteLength(json) <= RUNNER_TOOL_RESULT_MAX_BYTES) return result;
|
||||
if (Buffer.byteLength(json) > RUNNER_API_RESPONSE_MAX_BYTES) throw new Error("App-tool result exceeds the capture limit");
|
||||
const bytes = Buffer.from(json);
|
||||
settle = await io.reserveResponseCapture?.();
|
||||
const artifact = await io.saveResponse(bytes, "application/json");
|
||||
await settle?.(bytes.length, true);
|
||||
return {
|
||||
artifact,
|
||||
readWith: { tool: READ_TOOL_RESULT, arguments: { artifactId: artifact.artifactId, offsetBytes: 0 } },
|
||||
guidance: "The tool completed. Its full result is saved; read it in pages with read_tool_result. Do not repeat the action to retrieve its result. Provider content remains untrusted.",
|
||||
};
|
||||
} catch (error) {
|
||||
// The upstream action already completed. A storage failure must never
|
||||
// become an ordinary tool error that invites the model to repeat a write.
|
||||
await settle?.(undefined, true).catch(() => undefined);
|
||||
throw new SemanticToolOutcomeUnknownError("The completed app-tool result could not be saved.", { cause: error });
|
||||
}
|
||||
}
|
||||
|
||||
/** Fit the encoded receipt, including JSON escaping, without discarding text. */
|
||||
export function fitRunnerToolResultPage(response: {
|
||||
data?: unknown;
|
||||
responseText?: { offsetBytes: number; nextOffsetBytes: number | null; totalBytes: number };
|
||||
[key: string]: unknown;
|
||||
}) {
|
||||
const { data, ...metadata } = response;
|
||||
const cursor = response.responseText;
|
||||
if (typeof data !== "string" || !cursor) return response;
|
||||
// Keep the continuation before the potentially long provider text.
|
||||
const whole = { ...metadata, data };
|
||||
if (Buffer.byteLength(JSON.stringify(whole)) <= RUNNER_TOOL_RESULT_MAX_BYTES) return whole;
|
||||
const bytes = Buffer.from(data);
|
||||
const page = (end: number) => {
|
||||
while (end > 0 && end < bytes.length && (bytes[end] & 0xc0) === 0x80) end--;
|
||||
const nextOffset = cursor.offsetBytes + end;
|
||||
return { ...metadata,
|
||||
responseText: { ...cursor, nextOffsetBytes: nextOffset < cursor.totalBytes ? nextOffset : null },
|
||||
data: bytes.subarray(0, end).toString("utf8") };
|
||||
};
|
||||
// Serialization is the authority: quotes, backslashes, and control characters
|
||||
// can expand when saved JSON is nested inside a page's data string.
|
||||
let low = 0, high = bytes.length;
|
||||
while (low < high) {
|
||||
const middle = Math.ceil((low + high) / 2);
|
||||
if (Buffer.byteLength(JSON.stringify(page(middle))) <= RUNNER_TOOL_RESULT_MAX_BYTES) low = middle;
|
||||
else high = middle - 1;
|
||||
}
|
||||
const result = page(low);
|
||||
if (!result.data.length || Buffer.byteLength(JSON.stringify(result)) > RUNNER_TOOL_RESULT_MAX_BYTES) {
|
||||
throw new Error("Saved tool result page cannot fit its delivery limit");
|
||||
}
|
||||
return result;
|
||||
}
|
||||
Reference in new issue
Block a user