fix(opencode): export bounded tool argument diagnostics (#18699)

## Thinking Path

> - Paperclip manages work across agent runtimes.
> - Company observability sends allowed records to the operator's OTLP
receiver.
> - Native OpenCode exports tool results but omits submitted argument
diagnostics.
> - Exact argument values are excluded by the current capture policy.
> - Safe type information can distinguish malformed envelopes without
copying private values.
> - This change exports a fixed diagnostic shape from authenticated
dispatch and schema-rejection events.
> - Operators can compare rejected and accepted calls under the existing
capture controls.

## Linked Issues or Issue Description

**What happened?**

OpenCode tool spans omit argument diagnostics. The exporter reads native
tool events. The authenticated MCP boundary has the submitted arguments,
but its dispatch item events do not create tool spans. Calls rejected by
its schema check do not emit dispatch items.

**Expected behavior**

With tool details and tool content enabled, Paperclip tool spans include
the method, diagnostic schema version and allowed argument field types.
Exact values and arbitrary field names stay excluded.

**Steps to reproduce**

1. Configure a company OTLP receiver and enable native OpenCode export.
2. Enable tool details and tool content.
3. Submit a rejected Paperclip tool call and then a valid call.
4. Inspect the input diagnostics and outcome of each span.

Related work: #17989 proposes metadata-only native export. #18098 added
the normalized content exporter. This change repairs argument
diagnostics in that exporter.

## What Changed

- Export `paperclip.opencode.tool-diagnostics.v1` as `braintrust.input`
for authenticated Paperclip tool calls.
- Include `tools/call`, the argument type and types for a fixed list of
envelope fields.
- Use dispatch IDs for Paperclip tool spans. Keep separately named
`opencode tool ...` provider records for failures before dispatch. Mark
each span source.
- Preserve schema-rejected calls as failed items without invoking the
controller handler. Keep authentication and the closed tool catalog.
- Bound schema-rejection error text in stored events and transcripts
with the existing redaction and 8 KiB limit. Keep the provider response
unchanged.
- Preserve the controller opt-in, tool-detail and tool-content switches,
redaction, limits and run identity.
- Add MCP-to-receiver regression coverage and update the capture policy
and capability maps.

### Hypothesis and thesis for a fix

The exporter drops the dispatch diagnostic signal. Raw input omission is
intentional. The repair belongs in the OpenCode driver and exporter.
Shared protocol, tool permissions and task completion stay unchanged.

The regression sends schema-rejected, handler-rejected and valid
argument envelopes through a fixture OpenCode subprocess and the
authenticated MCP bridge. It checks the downstream rejection and
receipt, completion event and actual local OTLP request. The baseline
has no dispatch spans. The candidate has separate spans with safe type
diagnostics. Two warm runs and disabled capture use the same path.

Source: [OpenCode diagnostic
pattern](https://www.braintrust.dev/app/Paperclip/p/Paperclip%20Staging/patterns/f172172e-9c65-453f-bb3a-7c6ba4f5a400).

## Verification

- Baseline: `a92f616cd0a457c8ba85d994b24e1f92c2e4d1de`. The new
integration regression fails because tool spans are absent. The same
regression passes with this change.
- Exporter and MCP bridge suites: 37 tests pass. Three final driver
controls pass. Ten selected controls also passed before the final
provider-record repair. Runner TypeScript typecheck and build pass after
the review repair.
- The review regression failed with two spans before the repair. It
passes with three spans after the repair. Unauthenticated and unassigned
calls do not reach the rejection observer. A broken observer cannot
change the schema rejection.
- The full OpenCode/exporter/MCP run passes 94 tests and fails one
existing final-answer selection test. The same failure occurs in an
untouched baseline checkout. It is not claimed as a pass.
- A further filtered run reached the default five-second timeout in two
existing continuation tests during concurrent repository checks. All
five selected dispatch, warm-session and continuation controls pass with
a 15-second test deadline. The earlier timeouts remain recorded.
- `node scripts/check-maps.mjs` and `git diff --check` pass.
- Runner TypeScript checks and build pass. Full repository typecheck and
build stop at Rust because `cargo` is not installed. The full repository
test command was stopped before workspace handoff. No full-suite pass is
claimed. Local source commits:
`c48a8af1c186e415c802fa0ac95209b16b4a3122` `6744991f6`, and `a8ccf4693`.
The remote branch is based on newer master
`4814d1ab691c176170510a93a175dafdc9e021c7`. All initial eight changed
files match between those bases before the patch.
- The integration provider and controller are fixtures. The OTLP
receiver is a real local HTTP server. A separate real OpenCode 1.18.34
probe opened a session. The remote free model returned `Free usage limit
reached`. No tool call or receipt was produced. No live recovery is
claimed.

- Review follow-up: a real MCP request with 12,000 invalid array
elements produced a 264,908-character event error before the repair. The
same regression now stores 8,192 characters in both events and
transcript. The MCP error receipt stays unchanged and no handler runs.
All 37 exporter/MCP tests and 12 selected driver/exporter/MCP controls
pass. Runner TypeScript checks and build pass. Local follow-up commit:
`64d4fa583`.
- The previous remote head passed Runner, typecheck, build and E2E. Its
unchanged attention-retention shard failed twice with 15-second
timeouts, then cleanup deadlock and duplicate fixture IDs. Both affected
retention suites passed all 70 tests locally unchanged. The new head
needs fresh CI and review; no full CI pass is claimed.

- Current-master qualification (2026-11-06, merge `2959ecdb8d`):
normally merge master `53645a0104`, including the existing test-only
retention-fixture repair from #18774 reused by #18770. Keep the OpenCode
diagnostics feature and upstream durable MCP and completion-feedback
behavior. No test deadline or assertion changed. The three old CI
timeouts did not reproduce locally: the four completed-blocker-history
cases passed on the old head in 29.52 seconds, then passed on the merged
head in 13.23 seconds. The fixture rollback/provenance regression also
passed. All 116 exporter, MCP bridge and OpenCode driver fixture tests
passed. Runner TypeScript checks and both TypeScript compilation targets
passed; ACPX profiles match the release manifest and protocol-contract
checks pass. These checks use local PostgreSQL, HTTP and process
fixtures. No hosted provider call or deployed recovery is claimed. Fresh
CI and review are pending for this merge.

- Map-scope review follow-up (`61e662e179`): remove only the inherited
diagnostics paragraphs from the company capability maps. This change
adds logging detail within an existing capability. `node
scripts/check-maps.mjs` and `git diff --check` pass. Production and test
inputs are unchanged from the qualified merge; fresh CI and review are
pending for the final head.

- Test compiler follow-up (`6675b46377`): reproduce CI TS2722 with `tsc
-p packages/paperclip-runner/tsconfig.test.json --noEmit`. The harness
contract makes transcript retrieval optional. The OpenCode bounds
regression now fails explicitly if that capability is absent, then
inspects the narrowed method. The same complete Runner test compiler
project passes, and the actual local schema-rejection fixture passes
with all receipt and 8 KiB assertions retained. Production inputs are
unchanged. Fresh final-head CI and review are pending.

- Final-head qualification: `6675b4637743aa269e5214756b158e1e0a4561a5`
passes all 56 applicable CI checks (three skipped, one neutral review).
[CI
result](https://github.com/paperclipai/paperclip/actions/runs/42253119577).
Greptile gives 5/5 with no open findings. Hosted task recovery remains
pending verified rollout.

## Risks

- Authenticated spans use dispatch IDs. Provider records use separate
names and a source attribute. Consumers must choose the source they need
to avoid counting two observations of the same tool action.
- Safe types cannot reconstruct nested arguments or explain every
value-level validation failure.
- Export remains best effort and bounded. Lost native frames no longer
remove dispatch diagnostics, but abrupt process termination can still
lose a turn.
- Full local Rust checks and a deployed product journey are not yet
verified. The pull request is unmerged and not deployed.

## Model Used

OpenAI GPT-6 through Codex, with reasoning, tool use and code execution.
The exact runtime variant and context-window size are not exposed to
this session.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <[email protected]>
This commit is contained in:
DottaandPaperclip authored and GitHub committed 2026-11-06 11:01:25 -06:00
1 parent 952ecdbb39
commit 0d03610d2f
8 files changed
+291 -13

No files matched your search

+17 -3
View File
@@ -23,13 +23,27 @@ Metrics, raw API bodies and Codex Guardian assessments are disabled.
Native OpenCode 1.18.34 uses a runner-owned HTTP/JSON exporter. Each completed,
failed or cancelled turn produces a trace with Paperclip company, agent, run and
turn IDs. Tool outcomes and message-level token/cost receipts are child spans;
turn IDs. Authenticated Paperclip tool dispatch outcomes, native tool outcomes and
message-level token/cost receipts are child spans;
repeated usage updates replace the same message receipt. Missing prices remain
unknown. This projection excludes hidden reasoning, raw provider payloads and
system instructions. Prompt, response, tool-detail and tool-content switches
apply before export. Content is bounded to 8 KiB per field and 64 tools plus 64
usage records per turn; truncation is marked. Tool inputs and full model request
bodies are not captured. OpenCode's performance-only OTLP sink stays separate.
usage records per turn; truncation is marked. With both tool-detail and tool-content
capture enabled, Paperclip tool spans include `braintrust.input` diagnostics
(`paperclip.opencode.tool-diagnostics.v1`): the upstream `tools/call` method,
argument JSON type and types for a fixed list of envelope fields. These are
shape diagnostics, not a copy of the arguments. Arbitrary field names, argument
values, nested payloads and full model request bodies are not captured.
Authenticated schema rejections also produce failed tool spans without invoking
the controller handler. Conflicting request IDs produce separate failed items,
so a rejection cannot overwrite a previous dispatch outcome.
Paperclip tool spans use authenticated dispatch IDs and the name `tool <name>`.
Provider SSE records for those declared tools use `opencode tool <name>` and
remain available when failure occurs before dispatch, such as an oversized
request. These are separate observations, not correlated calls. The
`paperclip.tool.source` attribute distinguishes dispatch and provider records.
Native tools retain their provider spans without submitted-argument diagnostics. OpenCode's performance-only OTLP sink stays separate.
The exporter sends once at turn completion with a three-second deadline, at most
four requests in flight and no idle polling or persistent queue. Closing drains
@@ -117,3 +117,52 @@ it("does not block event delivery when a receiver stalls and cancels the request
expect(aborted).toBe(true);
expect(diagnostics).toEqual(["Company observability export unavailable (export_failed)."]);
}, 5000);
it("bounds dispatch diagnostics without traversing arbitrary keys or nested values", async () => {
const { observer, bodies } = setup();
observer.observe(event("turn.submitted"));
const args = { name: "provider-secret", arguments: { secret: "x".repeat(100000) }, ["private-key".repeat(10000)]: true };
observer.observe(event("item.started", { kind: "dynamicToolCall", item: { name: "paperclip_call_assigned_tool", arguments: args } }, "run-one", "call"));
observer.observe(event("item.completed", { kind: "dynamicToolCall", item: { result: { message: "provider-secret secret-canary" } } }, "run-one", "call"));
observer.observe(event("turn.completed"));
await observer.close();
const spans = bodies[0].resourceSpans[0].scopeSpans[0].spans;
const attributes = spanAttributes(spans[1]);
expect(JSON.parse(String(attributes["braintrust.input"]))).toEqual({ schema: "paperclip.opencode.tool-diagnostics.v1", method: "tools/call", argumentType: "object", fields: { name: "string", arguments: "object" } });
expect(attributes["braintrust.output"]).toBe('{"message":"[REDACTED] [REDACTED]"}');
expect(Buffer.byteLength(String(attributes["braintrust.input"]))).toBeLessThan(1024);
expect(JSON.stringify(bodies)).not.toContain("private-key");
});
it.each([null, [], "private-string"])("exports only the type for non-object arguments %j", async args => {
const { observer, bodies } = setup();
observer.observe(event("turn.submitted"));
observer.observe(event("item.started", { kind: "dynamicToolCall", item: { name: "tool", arguments: args } }, "run-one", "call"));
observer.observe(event("turn.completed"));
await observer.close();
const spans = bodies[0].resourceSpans[0].scopeSpans[0].spans;
expect(JSON.parse(String(spanAttributes(spans[1])["braintrust.input"])).fields).toEqual({});
expect(JSON.stringify(bodies)).not.toContain("private-string");
});
it("does not export dispatch spans when tool details are disabled", async () => {
const { observer, bodies } = setup({ ...environment, OTEL_LOG_TOOL_DETAILS: "0" });
observer.observe(event("turn.submitted"));
observer.observe(event("item.started", { kind: "dynamicToolCall", item: { name: "tool", arguments: { name: "private-name" } } }, "run-one", "call"));
observer.observe(event("turn.completed"));
await observer.close();
expect(bodies[0].resourceSpans[0].scopeSpans[0].spans).toHaveLength(1);
expect(JSON.stringify(bodies)).not.toContain("private-name");
});
it("retains provider failures for declared tools even before any dispatch", async () => {
const bodies: any[] = [];
const observer = createOpenCodeCompanyObservability({ environment, model: "test", sensitiveValues: [], managedToolNames: ["paperclip_call_assigned_tool"],
fetch: async (_url, init) => { bodies.push(JSON.parse(String(init?.body))); return new Response("{}"); } })!;
observer.observe(event("turn.submitted"));
observer.observe(event("tool.execution.completed", { name: "paperclip_call_assigned_tool", status: "failed", output: "request exceeded limit" }, "run-one", "native-rejection"));
observer.observe(event("turn.completed"));
await observer.close();
const spans = bodies[0].resourceSpans[0].scopeSpans[0].spans;
expect(spans[1].name).toBe("opencode tool paperclip_call_assigned_tool");
expect(spans[1].status.code).toBe(2);
expect(spanAttributes(spans[1])).toMatchObject({ "paperclip.tool.source": "provider", "braintrust.output": "request exceeded limit" });
});
@@ -23,9 +23,27 @@ function pairs(value: string | undefined): Record<string, string> {
}));
}
function argumentDiagnostics(value: unknown): string {
const type = (entry: unknown) => entry === null ? "null" : Array.isArray(entry) ? "array" : typeof entry;
const fields = record(value);
return JSON.stringify({
schema: "paperclip.opencode.tool-diagnostics.v1", method: "tools/call",
argumentType: type(value),
fields: Object.fromEntries([
"name", "arguments", "schema", "reportedWorkDisposition", "summary",
"completionClaim", "verification", "evidence", "artifacts", "continuation", "blocker",
].filter(key => Object.hasOwn(fields, key)).map(key => [key, type(fields[key])])),
});
}
function outputText(value: unknown): string {
if (typeof value === "string") return value;
try { return JSON.stringify(value) ?? ""; } catch { return ""; }
}
/** A controller opt-in, not OpenCode's raw OTLP. Projected PRP events never include reasoning or raw provider payloads. */
export function createOpenCodeCompanyObservability(input: {
environment: NodeJS.ProcessEnv | undefined; model: string; sensitiveValues: readonly string[];
managedToolNames?: readonly string[];
fetch?: typeof globalThis.fetch; onDiagnostic?: (message: string) => void;
}) {
const env = input.environment;
@@ -42,6 +60,8 @@ export function createOpenCodeCompanyObservability(input: {
["paperclip.company.id", "paperclip.agent.id", "paperclip.observability.revision"]
.filter(key => supplied[key]).map(key => [key, supplied[key]])) };
} catch { diagnostic("configuration_invalid"); return undefined; }
const managedTools = new Set(["paperclip_finish", "paperclip_block", ...(input.managedToolNames ?? [])]
.flatMap(name => [name, `paperclip_${name}`]));
const capture = (key: string) => env[key] === "1";
const clean = (value: unknown, limit = maxTextBytes): string => {
if (typeof value !== "string") return "";
@@ -123,19 +143,28 @@ export function createOpenCodeCompanyObservability(input: {
if (event.eventType === "item.completed" && payload.kind === "agentMessage" && capture("OTEL_LOG_ASSISTANT_RESPONSES")) {
turn.root.attributes["braintrust.output"] = clean(payload.text);
}
if (event.eventType.startsWith("tool.execution.") && event.itemId && capture("OTEL_LOG_TOOL_DETAILS")) {
let span = turn.tools.get(event.itemId);
const dynamic = payload.kind === "dynamicToolCall" && ["item.started", "item.completed"].includes(event.eventType);
const item = record(payload.item);
const canonicalTool = event.eventType.startsWith("tool.execution.");
if ((dynamic || canonicalTool) && event.itemId && capture("OTEL_LOG_TOOL_DETAILS")) {
const key = `${dynamic ? "dispatch" : "native"}:${event.itemId}`;
let span = turn.tools.get(key);
if (!span && turn.tools.size < maxRows) {
const name = clean(payload.name, 128);
span = makeSpan(`tool ${name || "execution"}`, time, {
"gen_ai.operation.name": "execute_tool", "gen_ai.tool.name": name });
turn.tools.set(event.itemId, span);
const name = clean(dynamic ? item.name : payload.name, 128);
span = makeSpan(`${!dynamic && managedTools.has(String(payload.name)) ? "opencode tool" : "tool"} ${name || "execution"}`, time, {
"gen_ai.operation.name": "execute_tool", "gen_ai.tool.name": name,
"paperclip.tool.source": dynamic ? "dispatch" : "provider" });
turn.tools.set(key, span);
}
if (span) {
span.end = time; span.failed = payload.status === "failed";
span.end = time; span.failed = dynamic ? item.is_error === true : payload.status === "failed";
if (capture("OTEL_LOG_TOOL_CONTENT")) {
span.attributes["braintrust.output"] = clean(payload.output);
if (payload.outputTruncated === true) turn.truncated = true;
if (dynamic && event.eventType === "item.started") {
span.attributes["braintrust.input"] = argumentDiagnostics(item.arguments);
} else {
span.attributes["braintrust.output"] = clean(dynamic ? outputText(item.result ?? item.error) : payload.output);
}
if (!dynamic && payload.outputTruncated === true) turn.truncated = true;
}
} else turn.truncated = true;
}
@@ -326,3 +326,37 @@ describe("OpenCode MCP bridge", () => {
});
});
});
it("reports admitted schema rejections without invoking handlers or widening authentication", async () => {
const handler = vi.fn(async () => ({ accepted: true }));
const rejected = vi.fn();
const bridge = await startOpenCodeMcpBridge({
tools: [{ name: "probe", inputSchema: { type: "object" } }], handler, onRejectedCall: rejected,
});
bridges.push(bridge);
const body = { id: "bad", method: "tools/call", params: { name: "probe", arguments: [] } };
expect((await rpc(bridge, body, "wrong")).status).toBe(401);
await rpc(bridge, { ...body, params: { name: "unassigned", arguments: [] } });
expect(rejected).not.toHaveBeenCalled();
expect(await (await rpc(bridge, body)).json()).toMatchObject({ result: { isError: true } });
expect(rejected).toHaveBeenCalledExactlyOnceWith({ tool: "probe", callId: "bad", arguments: [] }, expect.stringContaining("Invalid tool input"));
expect(handler).not.toHaveBeenCalled();
});
it("keeps schema rejection intact when its observer fails", async () => {
const handler = vi.fn(async () => ({}));
const bridge = await startOpenCodeMcpBridge({ handler, onRejectedCall: () => { throw new Error("observer unavailable"); } });
bridges.push(bridge);
expect(await (await rpc(bridge, { id: 1, method: "tools/call", params: { name: "paperclip_finish", arguments: {} } })).json()).toMatchObject({ result: { isError: true } });
expect(handler).not.toHaveBeenCalled();
});
it("reports conflicting request IDs without repeating the accepted action", async () => {
const handler = vi.fn(async () => ({ accepted: true }));
const rejected = vi.fn();
const bridge = await startOpenCodeMcpBridge({ tools: [{ name: "probe", inputSchema: { type: "object" } }], handler, onRejectedCall: rejected });
bridges.push(bridge);
const body = { id: "same", method: "tools/call", params: { name: "probe", arguments: { name: "first" } } };
await rpc(bridge, body);
expect(await (await rpc(bridge, { ...body, params: { name: "probe", arguments: { name: "second" } } })).json()).toMatchObject({ result: { isError: true } });
expect(handler).toHaveBeenCalledOnce();
expect(rejected).toHaveBeenCalledExactlyOnceWith({ tool: "probe", callId: "same", arguments: { name: "second" } }, "Duplicate call identity conflict.");
});
@@ -29,6 +29,7 @@ export interface OpenCodeMcpBridgeOptions {
/** Runner-owned operations callable only by a trusted harness extension and never advertised to the model. */
privateTools?: readonly Readonly<Record<string, unknown>>[];
handler: (call: OpenCodeMcpCall) => Promise<unknown>;
onRejectedCall?: (call: Pick<OpenCodeMcpCall, "tool" | "callId" | "arguments">, error: string) => void;
/** Null leaves durable requests with the owning Runner lease and bridge close. */
timeoutMs?: number | null;
/** Human-gated private operations may remain pending much longer than model tool calls. */
@@ -103,6 +104,7 @@ export async function startOpenCodeMcpBridge(
calls,
controllers,
handler: options.handler,
onRejectedCall: options.onRejectedCall,
diagnoseFinishInput,
timeoutMs: options.timeoutMs === null ? null : options.timeoutMs ?? DEFAULT_TIMEOUT_MS,
privateToolTimeoutMs: options.privateToolTimeoutMs ?? options.timeoutMs ?? DEFAULT_TIMEOUT_MS,
@@ -186,6 +188,7 @@ async function handleRequest(
calls: Map<string, { fingerprint: string; promise: Promise<unknown> }>;
controllers: Map<string, AbortController>;
handler: OpenCodeMcpBridgeOptions["handler"];
onRejectedCall: OpenCodeMcpBridgeOptions["onRejectedCall"];
timeoutMs: number | null;
privateToolTimeoutMs: number;
privateToolNames: ReadonlySet<string>;
@@ -283,15 +286,18 @@ async function handleRequest(
context.diagnoseFinishInput({ input: params.arguments, callId, validationCategory });
}
const detail = validator?.errors?.map((error) => `${error.instancePath || "/"} ${error.message ?? "is invalid"}`).join("; ");
const error = `Invalid tool input${detail ? `: ${detail}` : "."}`;
try { context.onRejectedCall?.({ tool, callId, arguments: args }, error); } catch {}
writeRpc(response, id, {
isError: true,
content: [{ type: "text", text: `Invalid tool input${detail ? `: ${detail}` : "."}` }],
content: [{ type: "text", text: error }],
});
return;
}
const fingerprint = canonicalJson({ tool, args });
const existing = context.calls.get(callId);
if (existing && existing.fingerprint !== fingerprint) {
try { context.onRejectedCall?.({ tool, callId, arguments: args }, "Duplicate call identity conflict."); } catch {}
writeRpc(response, id, { isError: true, content: [{ type: "text", text: "Duplicate call identity conflict." }] });
return;
}
@@ -154,6 +154,103 @@ afterAll(async () => {
});
describe("OpenCodeServerDriver", () => {
it("bounds schema rejection errors in events and transcript without changing the MCP receipt", async () => {
await chmod(fixture, 0o755);
const root = await mkdtemp(join(tmpdir(), "paperclip-large-rejection-")); roots.push(root);
const handler = vi.fn();
const driver = new OpenCodeServerDriver({ model: "openrouter/test-model", command: fixture, runtimeDirectory: root,
environment: { PATH: process.env.PATH },
dynamicTools: [{ name: "large_call", inputSchema: { type: "array", items: { type: "string" } } }],
dynamicToolHandler: handler,
});
const session = await driver.openSession({ runId: "large-rejection", normalizedSessionId: "large-rejection", workingDirectory: root });
try {
await session.startTurn({ message: { role: "user", text: "oversized-schema-rejection" } });
const events = await collectTurnEvents(session.events());
expect(events.at(-1)?.eventType).toBe("turn.completed");
const receipt = JSON.parse(await readFile(join(root, "fake-large-rejection-receipt.json"), "utf8"));
expect(receipt.result.isError).toBe(true);
expect(receipt.result.content[0].text.length).toBeGreaterThan(8_192);
expect(handler).not.toHaveBeenCalled();
if (!session.transcript) throw new Error("OpenCode session must expose its transcript");
for (const recorded of [events, (await session.transcript()).events]) {
const errors = recorded.filter(event => event.eventType === "item.completed")
.map(event => event.payload.item as { is_error?: boolean; error?: string }).filter(item => item?.is_error);
expect(errors).toHaveLength(1);
expect(errors[0]!.error).toContain("Invalid tool input");
expect(errors[0]!.error).toHaveLength(8_192);
}
} finally { await session.close({ reason: "test" }); }
});
it.each(["1", "0"])("exports safe dispatch argument diagnostics with tool content=%s", async capture => {
await chmod(fixture, 0o755);
const bodies: any[] = [];
const collector = createHttpServer((request, response) => {
const chunks: Buffer[] = [];
request.on("data", chunk => chunks.push(chunk));
request.on("end", () => { bodies.push(JSON.parse(Buffer.concat(chunks).toString("utf8"))); response.end("{}"); });
});
await new Promise<void>(resolve => collector.listen(0, "127.0.0.1", resolve));
const address = collector.address();
if (!address || typeof address === "string") throw new Error("collector did not bind");
const root = await mkdtemp(join(tmpdir(), "paperclip-argument-diagnostics-")); roots.push(root);
const calls: unknown[] = [];
const driver = new OpenCodeServerDriver({ model: "openrouter/test-model", command: fixture, runtimeDirectory: root,
environment: { PATH: process.env.PATH, PAPERCLIP_OBSERVABILITY_OPENCODE: "1",
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: `http://127.0.0.1:${address.port}/v1/traces`,
OTEL_LOG_TOOL_DETAILS: "1", OTEL_LOG_TOOL_CONTENT: capture },
taskEnvelope: createCodexTaskEnvelope({ objective: "argument-diagnostics" }),
dynamicTools: [{ name: "paperclip_call_assigned_tool", inputSchema: { type: "object" } }],
dynamicToolHandler: async call => {
calls.push(call.arguments);
if (typeof (call.arguments as { name?: unknown }).name !== "string") throw new Error("name must be a string");
return { receipt: "benign-receipt" };
},
});
let session: Awaited<ReturnType<typeof driver.openSession>> | undefined;
try {
session = await driver.openSession({ runId: "first", normalizedSessionId: "diagnostics", workingDirectory: root });
for (const runId of ["first", "second"]) {
if (runId === "second") session.attachRun!({ runId });
await session.startTurn({ message: { role: "user", text: JSON.stringify({ message: "argument-diagnostics", completionContract: createCodexTaskEnvelope({ objective: "argument-diagnostics" }).completionContract }) } });
const events = await collectTurnEvents(session.events());
expect(events.at(-1)?.eventType).toBe("turn.completed");
expect(events.filter(event => event.eventType === "run.result.proposed")).toHaveLength(1);
const receipts = JSON.parse(await readFile(join(root, "fake-diagnostic-receipts.json"), "utf8"));
expect(receipts[0].result.isError).toBe(true);
expect(receipts[1].result.isError).toBe(true);
expect(receipts[2].result.content[0].text).toContain("benign-receipt");
}
await session.close({ reason: "test" }); session = undefined;
expect(calls).toHaveLength(4);
expect(bodies).toHaveLength(2);
for (const [index, body] of bodies.entries()) {
const spans = body.resourceSpans[0].scopeSpans[0].spans;
const attrs = (span: any) => Object.fromEntries(span.attributes.map((a: any) => [a.key, a.value.stringValue ?? a.value.boolValue]));
const tools = spans.filter((span: any) => span.name === "tool paperclip_call_assigned_tool");
expect(tools).toHaveLength(3);
const native = spans.filter((span: any) => span.name === "opencode tool paperclip_call_assigned_tool");
expect(native).toHaveLength(1);
expect(attrs(native[0])["paperclip.tool.source"]).toBe("provider");
expect(attrs(native[0])).not.toHaveProperty("braintrust.input");
expect(attrs(tools[0])["paperclip.tool.source"]).toBe("dispatch");
expect(tools.map((span: any) => span.status.code)).toEqual([2, 2, 1]);
expect(attrs(tools[0])["paperclip.run.id"]).toBe(index === 0 ? "first" : "second");
if (capture === "1") {
expect(JSON.parse(attrs(tools[0])["braintrust.input"])).toMatchObject({ argumentType: "array", fields: {} });
expect(JSON.parse(attrs(tools[1])["braintrust.input"])).toMatchObject({ schema: "paperclip.opencode.tool-diagnostics.v1", method: "tools/call", argumentType: "object", fields: { name: "number", arguments: "object" } });
expect(JSON.parse(attrs(tools[2])["braintrust.input"]).fields.name).toBe("string");
} else for (const tool of tools) expect(attrs(tool)).not.toHaveProperty("braintrust.input");
}
for (const excluded of ["private-argument-canary", "private-key-canary", "private-value-canary", "benign-operation"]) expect(JSON.stringify(bodies)).not.toContain(excluded);
} finally {
await session?.close({ reason: "test" });
await new Promise<void>(resolve => { collector.close(() => resolve()); collector.closeAllConnections(); });
}
});
it("forwards configured bindings to the provider shell and redacts their values", async () => {
const root = await mkdtemp(join(tmpdir(), "paperclip-opencode-configured-env-"));
roots.push(root);
@@ -90,6 +90,7 @@ import {
canonicalOpenCodeMcpToolName,
startOpenCodeMcpBridge,
type OpenCodeMcpBridge,
type OpenCodeMcpBridgeOptions,
} from "./mcp-bridge.js";
import { nativeMcpLaunchBinding } from "../native-mcp.js";
import {
@@ -399,6 +400,7 @@ export class OpenCodeServerDriver implements HarnessDriver {
root,
cwd,
trace,
onRejectedCall: (call, error) => session?.observeRejectedToolCall(call, error),
dispatch: (call) => {
if (session === null)
throw new Error("OpenCode session is not ready for tool calls");
@@ -450,6 +452,7 @@ export class OpenCodeServerDriver implements HarnessDriver {
snapshot,
now: this.#options.now ?? (() => new Date()),
observabilityEnvironment: this.#options.environment,
managedToolNames: this.#options.dynamicTools?.map(tool => text(tool.name)).filter(Boolean),
onDiagnostic: this.#options.onDiagnostic,
});
session.startEventPump();
@@ -573,11 +576,13 @@ class OpenCodeHarnessSession implements HarnessSession {
snapshot: PersistedHarnessSession | null;
now: () => Date;
observabilityEnvironment?: NodeJS.ProcessEnv;
managedToolNames?: readonly string[];
onDiagnostic?: (message: string) => void;
}) {
this.#runtime = input.runtime;
this.#companyObservability = createOpenCodeCompanyObservability({
environment: input.observabilityEnvironment, model: input.model, sensitiveValues: input.runtime.sensitiveValues,
managedToolNames: input.managedToolNames,
onDiagnostic: input.onDiagnostic ?? ((message) => console.warn(message)),
});
this.#fetch = input.fetcher;
@@ -1128,6 +1133,18 @@ class OpenCodeHarnessSession implements HarnessSession {
);
}
observeRejectedToolCall(call: { tool: string; callId: string; arguments: unknown }, error: string): void {
const turnId = this.#activeTurnId;
if (!turnId || this.#closed) return;
const itemId = `${call.callId}:rejected:${randomBytes(12).toString("hex")}`;
this.#emit("item.started", {
kind: "dynamicToolCall", item: { type: "tool_call", id: itemId, name: call.tool, arguments: call.arguments },
}, { turnId, itemId });
this.#emit("item.completed", {
kind: "dynamicToolCall", item: { type: "tool_result", id: itemId, tool_use_id: itemId, is_error: true, error: redact(error) },
}, { turnId, itemId });
}
async dispatchTool(call: {
tool: string;
callId: string;
@@ -2425,6 +2442,7 @@ async function startRuntime(input: {
root: string;
cwd: string;
trace: ProviderTraceFileSink | null;
onRejectedCall?: OpenCodeMcpBridgeOptions["onRejectedCall"];
dispatch: (call: {
tool: string;
callId: string;
@@ -2463,6 +2481,7 @@ async function startRuntime(input: {
const bridge = await startOpenCodeMcpBridge({
tools: input.options.dynamicTools,
handler: input.dispatch,
onRejectedCall: input.onRejectedCall,
onDiagnostic: input.options.onDiagnostic,
...(input.options.durableToolRequests ? { timeoutMs: null } : {}),
}).catch(async (error) => {
@@ -535,6 +535,36 @@ const server = createServer(async (request, response) => {
emit({ type: "session.idle", properties: { sessionID: session.id } });
return;
}
if (String(parsedPrompt.message ?? "").includes("oversized-schema-rejection")) {
const receipt = await mcpRequest("tools/call", {
name: "paperclip_large_call", arguments: Array(12_000).fill(0),
});
await writeFile(join(process.cwd(), "fake-large-rejection-receipt.json"), JSON.stringify(receipt));
await callTerminalTool(promptPayload);
emit({ type: "session.idle", properties: { sessionID: session.id } });
return;
}
if (String(parsedPrompt.message ?? parsedPrompt.task?.prompt ?? "").includes("argument-diagnostics")) {
const receipts = [await mcpRequest("tools/call", {
name: "paperclip_paperclip_call_assigned_tool", arguments: [],
})];
for (const name of [42, "benign-operation"]) {
receipts.push(await mcpRequest("tools/call", {
name: "paperclip_paperclip_call_assigned_tool",
arguments: { name, arguments: { token: "private-argument-canary" }, "private-key-canary": "private-value-canary" },
}));
}
emit({ type: "message.updated", id: "diagnostic-message-event", properties: { sessionID: session.id,
info: { id: "diagnostic-message", sessionID: session.id, role: "assistant" } } });
emit({ type: "message.part.updated", id: "diagnostic-part-event", properties: { sessionID: session.id,
part: { id: "diagnostic-provider-row", messageID: "diagnostic-message", sessionID: session.id,
type: "tool", tool: "paperclip_paperclip_call_assigned_tool",
callID: "unrelated-provider-call", state: { status: "completed", output: "provider-receipt" } } } });
await writeFile(join(process.cwd(), "fake-diagnostic-receipts.json"), JSON.stringify(receipts));
await callTerminalTool(promptPayload);
emit({ type: "session.idle", properties: { sessionID: session.id } });
return;
}
if (!String(parsedPrompt.message ?? parsedPrompt.task?.prompt ?? "").includes("current-wake-completion-feedback")) await callFirstPaperclipTool();
if (String(parsedPrompt.message ?? "").includes("managed-github-operation")) {
const result = await new Promise(resolve => {