fix(cli): Restore-Packet M2b R2 — Claude parser nested tool_use/tool_result handling

Guard BLOCKED M2b at openrig bdf4cf6: Claude transcript parser silently
dropped `tool_use` parts (inside assistant content) and `tool_result`
parts (inside user content) without counting them against the omitted-
record-class enum, leading to all-zero `omittedCounts` on real Claude
transcripts where every assistant turn typically carries a `tool_use`
and most user turns carry a `tool_result`. M1 contract § 5 requires
`omitted_classes` to enumerate the classes ACTUALLY excluded from
transcript output; the previous parser would emit a contract-violating
all-zero summary at M2c.

R2 fix in src/restore-packet/claude-transcript-parser.ts:

- Replace `textFromClaudeMessage` with `walkClaudeContent` returning
  `{ text, toolUseCount, toolResultCount, toolPaths }`. The walker
  inspects each content part once, branching on `part.type`:
  - `"text"` → contribute to user-visible text.
  - `"tool_use"` → increment `function_call_output` counter; serialize
    `part.input` and route to path extraction.
  - `"tool_result"` → increment `raw_tool_outputs` counter; flatten
    `part.content` (string OR array of `{ type: "text" }` parts) and
    route to path extraction.
- The kept-message branch now records nested-part omissions BEFORE
  deciding whether to drop a record without visible text. A user turn
  that's pure tool_result (no visible text; common in real transcripts)
  no longer emits a kept message but DOES count its tool_result against
  the omitted-class counter and contributes the tool_result content to
  the path inventory.
- Path extraction from omitted nested parts mirrors the Codex parser's
  behavior of walking function_call args / custom_tool_call inputs for
  paths even though the records themselves are omitted from messages.

TDD discipline (per memory feedback_tdd_scope): 8 new tests authored to
fail FIRST. Test commit: 7 fail, 1 pass (a happy-path baseline). Then
the parser fix made all 8 pass. The committed state is the green state.

Tests added (8 new in `M2b R2 Claude parser nested-content-part handling`
describe block):

- guard reproducer fixture (the exact reproducer from the BLOCK
  artifact): assistant with tool_use + text, user with tool_result
  → omittedCounts.function_call_output=1, raw_tool_outputs=1, paths
  contains "/Users/wrandom/code/projects/openrig-hub".
- counts each tool_use part in a multi-tool assistant turn (tool_use*2 +
  text → function_call_output=2; path "/Users/wrandom/code/x.md"
  extracted from input).
- counts each tool_result part in a multi-result user turn
  (tool_result*2 → raw_tool_outputs=2; both paths extracted).
- mixed text + tool_use: visible text kept; tool_use counted.
- tool_use with non-path input: counted but contributes no paths
  (the `cmd: "echo hello"` doesn't match the /Users/wrandom/ or
  recognized-prefix patterns).
- tool_result with non-path content: same — counted but no paths.
- tool_result with array content shape: walks the array's text parts
  for paths.
- kept text-only assistant turn: counters stay at zero (no regression
  on the simple-text path).

Re-verification (per dispatch's required-fix list):

- npx vitest run test/restore-packet.test.ts → 63/63 pass (55 prior +
  8 new).
- npx tsc --noEmit → exit 0.
- npm run build → exit 0; dist/schemas/restore-summary.schema.json
  copied; dist/restore-packet/claude-transcript-parser.js emitted with
  the fix.
- Compiled-output runtime check using the guard's exact reproducer:
    node -e 'import(...claude-transcript-parser.js).then(...)'
  → output: `function_call_output: 1 raw_tool_outputs: 1 paths_count: 1`.
  Confirms the fix is reachable from compiled JS, not just source TS.
- Full CLI suite re-run: 703/707 pass. M2b baseline was 695/699; R2 adds
  8 tests, all green → 703/707. Same 4 pre-existing environmental flakes:
  - test/chatroom.test.ts > Chatroom wait detects newly arriving post-start
    message (10s timing flake)
  - test/config.test.ts > DaemonClient config integration > ConfigStore
    non-default host/port (env pollution)
  - test/preflight.test.ts > rig preflight --json (host port 7433 in use)
  - test/preflight.test.ts > rig preflight check marks (host port 7433 in use)
  All four touch zero files in this R2 commit. No regression.

Scope (R2 stays narrow per dispatch):

- Codex parser path-inventory shape NOT changed (the Claude walker
  mirrors Codex behavior; no Codex parser regressions).
- Velocity prior-art .mjs FROZEN reference UNCHANGED.
- M2c surfaces (packet-writer.ts, atomic emission, daemon full-read
  transcript route, daemon route tests) NOT touched.
- restore-packet write stays M2a-deferred error stub.
- restore-packet read + validate stay M3-pending stubs.
- M3/M4/M5/M6 surfaces NOT touched.
- No openrig source changes outside the two R2 files.
- No daemon code changes.
- No unrelated dirty work staged.

Quality lessons applied:

- v6 boundary isolation: R2 stays inside packages/cli/src/restore-packet/.
- v7 refined: R2 cites M2b R1 BLOCK artifact in this commit message; does
  NOT extend M2b R1 grant.
- v8 honest baseline: full CLI suite re-run; named flakes baseline-checked.
- v9 synthetic tokens only: all R2 fixtures use synthetic strings.
- v10 schema-packaging gap: schema-validator self-contained pattern
  unaffected; R2 modules are pure TypeScript with no runtime asset
  loading.
- v11 candidate (NEW from this BLOCK): "provider transcript parser
  nested-content-part coverage." Parser acceptance tests for provider
  transcript adapters MUST cover provider-native nested tool-call /
  tool-result / inline-tool-use shapes — not only top-level synthetic
  omitted records. R2 applies this lesson: 8 new focused tests cover
  Claude's nested-content-part shape AND the guard reproducer fixture
  is asserted byte-for-byte.

Posture: full QA/testing window (per dispatch).

Source qitem: qitem-20260502011841-fb53d7fd (M2b R2 dispatch).
Guard verdict that triggered this: rigs/openrig-velocity/state/orch/restore-packet-M2b-guard-checkpoint-block-2026-05-02.md.
This commit is contained in:
mvschwarz committed 2026-05-01 18:22:27 -07:00
1 parent bdf4cf6844
commit 5c87145e54
2 files changed
+313 -27

No files matched your search

@@ -50,35 +50,108 @@ function firstLine(text: string): string {
}
/**
* Extract user-visible text from a Claude `message.content` field.
* Handles three shapes:
* - string (legacy / user): use directly.
* - array of part objects (assistant or tool-bearing user): collect
* the `text` field of each `{ type: "text", text }` entry.
* - other / null / undefined: empty string.
* Result of walking a Claude `message.content` field. Returns:
* - `text`: concatenated user-visible text from `{ type: "text" }` parts.
* - `toolUseCount`: number of `{ type: "tool_use" }` parts seen.
* These are nested function-call records inside an assistant turn;
* per M1 contract § 5 they map to `function_call_output`.
* - `toolResultCount`: number of `{ type: "tool_result" }` parts seen.
* These are nested tool-output records inside a user turn; per
* contract § 5 they map to `raw_tool_outputs`.
* - `toolPaths`: paths extracted from omitted tool_use inputs and
* tool_result content (mirrors `codex-jsonl-parser.ts`'s tool-call
* path-inventory behavior; the tool record itself is NOT in the
* message text but its file references stay in the touched-files
* inventory).
*
* Tool-use parts (`{ type: "tool_use", ... }`) and tool-result parts
* (`{ type: "tool_result", ... }`) are intentionally NOT included in
* the visible message text; those are captured separately as
* raw_tool_outputs / function_call_output via the attachment-record path.
* R2 fix (per M2b R2 dispatch qitem-20260502011841-fb53d7fd): inspect
* nested content parts BEFORE discarding non-text. Previously dropped
* tool_use / tool_result silently, leading to misleading all-zero
* omittedCounts on real Claude transcripts where nearly every assistant
* turn has tool_use and most user turns have tool_result.
*/
function textFromClaudeMessage(message: unknown): string {
if (!message || typeof message !== "object") return "";
const obj = message as Record<string, unknown>;
const content = obj.content;
interface ContentWalkResult {
text: string;
toolUseCount: number;
toolResultCount: number;
toolPaths: string[];
}
/**
* Extract `text` candidates from a tool_result `content` field. The
* Claude API allows two shapes for `tool_result.content`:
* - string (the most common; e.g., a command's stdout): use directly.
* - array of part objects (when the tool returned structured content):
* collect each part's `text` field.
* Returns a flat string suitable for path extraction.
*/
function toolResultText(content: unknown): string {
if (typeof content === "string") return content;
if (!Array.isArray(content)) return "";
return content
.map((part) => {
if (typeof part === "string") return part;
if (!part || typeof part !== "object") return "";
const parts: string[] = [];
for (const part of content) {
if (typeof part === "string") {
parts.push(part);
} else if (part && typeof part === "object") {
const p = part as Record<string, unknown>;
if (p.type === "text" && typeof p.text === "string") return p.text;
// Skip tool_use / tool_result; they're not user-visible message text.
return "";
})
.filter((s) => s.length > 0)
.join("\n");
if (p.type === "text" && typeof p.text === "string") {
parts.push(p.text);
}
}
}
return parts.join("\n");
}
function walkClaudeContent(message: unknown): ContentWalkResult {
const out: ContentWalkResult = { text: "", toolUseCount: 0, toolResultCount: 0, toolPaths: [] };
if (!message || typeof message !== "object") return out;
const obj = message as Record<string, unknown>;
const content = obj.content;
if (typeof content === "string") {
out.text = content;
return out;
}
if (!Array.isArray(content)) return out;
const textParts: string[] = [];
for (const part of content) {
if (typeof part === "string") {
textParts.push(part);
continue;
}
if (!part || typeof part !== "object") continue;
const p = part as Record<string, unknown>;
const partType = typeof p.type === "string" ? p.type : "";
if (partType === "text" && typeof p.text === "string") {
textParts.push(p.text);
continue;
}
if (partType === "tool_use") {
out.toolUseCount += 1;
// Extract paths from the tool_use input. Mirrors the Codex parser's
// tool-call path-inventory behavior at codex-jsonl-parser.ts.
const input = p.input;
const inputStr = typeof input === "string" ? input : JSON.stringify(input ?? {});
out.toolPaths.push(inputStr);
continue;
}
if (partType === "tool_result") {
out.toolResultCount += 1;
// tool_result.content can be string OR array; both extract a flat
// text body for path scanning.
const resultText = toolResultText(p.content);
out.toolPaths.push(resultText);
continue;
}
// Other part types (image, etc.) are not user-visible text and not
// counted in the M1 contract enums; skip without counting.
}
out.text = textParts.filter((s) => s.length > 0).join("\n");
return out;
}
interface ClaudeRecord {
@@ -153,11 +226,30 @@ export function parseClaudeTranscript(content: string): StructuredTranscript {
// Kept message.
const role: "user" | "assistant" = recordType === "user" ? "user" : "assistant";
const rawText = textFromClaudeMessage(record.message);
if (hasSecretPattern(rawText)) {
const walked = walkClaudeContent(record.message);
// R2 fix: count nested tool_use / tool_result parts BEFORE deciding
// whether to emit a visible-text message. These counters fire
// regardless of whether the record contributes a message to the
// visible transcript — capturing them here is what M1 contract § 5
// requires for an honest `omitted_classes` field downstream.
for (let i = 0; i < walked.toolUseCount; i++) {
omittedCounter.recordOmission("function_call_output");
}
for (let i = 0; i < walked.toolResultCount; i++) {
omittedCounter.recordOmission("raw_tool_outputs");
}
// Path extraction from omitted tool parts (mirrors Codex parser
// behavior of walking tool-call args / inputs for path inventory
// even though the records themselves are omitted from messages).
for (const toolText of walked.toolPaths) {
extractPaths(toolText, pathCounts);
}
if (hasSecretPattern(walked.text)) {
omittedCounter.recordRedaction();
}
const text = redact(rawText).trim();
const text = redact(walked.text).trim();
if (!text) continue;
messageCount++;
+194
View File
@@ -605,3 +605,197 @@ describe("M2b interaction: parse → redact → omitted-records counter chain",
expect(result.messages[1]!.text).toBe("clean reply");
});
});
// ─────────────────────────────────────────────────────────────────────
// M2b R2: Claude parser nested tool_use / tool_result handling.
//
// Per guard's M2b BLOCK: the parser silently dropped tool_use parts
// (inside assistant content) and tool_result parts (inside user
// content) without counting them as omitted classes. Real Claude
// transcripts have these on nearly every turn; the previous M2b
// commit produced misleading omittedCounts of all-zero.
//
// R2 fix: walk message.content parts inside kept user/assistant
// records; count tool_use → function_call_output, tool_result →
// raw_tool_outputs; extract paths from omitted parts (mirroring
// Codex parser's behavior at codex-jsonl-parser.ts where function_call
// args + custom_tool_call input are walked for paths even though the
// records themselves are omitted from the message stream).
// ─────────────────────────────────────────────────────────────────────
describe("M2b R2 Claude parser nested-content-part handling", () => {
it("guard reproducer fixture: tool_use + tool_result both counted; paths extracted", () => {
const content = JSON.stringify({
type: "assistant",
message: {
role: "assistant",
content: [
{ type: "tool_use", id: "toolu_1", name: "Bash", input: { cmd: "pwd" } },
{ type: "text", text: "done" },
],
},
cwd: "/x",
sessionId: "s",
}) + "\n" + JSON.stringify({
type: "user",
message: {
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "toolu_1",
content: "/Users/wrandom/code/projects/openrig-hub",
},
],
},
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
// Both top-level records are KEPT (the parser still emits the visible
// text "done" from the assistant turn; the user turn has only
// tool_result content so no visible text — but the record was
// classified as kept; without visible text it's dropped from
// messages but the nested tool_result must STILL be counted.)
expect(r.omittedCounts.function_call_output).toBe(1);
expect(r.omittedCounts.raw_tool_outputs).toBe(1);
expect(r.paths.some((p) => p.path === "/Users/wrandom/code/projects/openrig-hub")).toBe(true);
});
it("counts each tool_use part in a multi-tool assistant turn", () => {
const content = JSON.stringify({
type: "assistant",
message: {
role: "assistant",
content: [
{ type: "tool_use", id: "t1", name: "Bash", input: { cmd: "ls" } },
{ type: "tool_use", id: "t2", name: "Read", input: { path: "/Users/wrandom/code/x.md" } },
{ type: "text", text: "running both" },
],
},
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
expect(r.omittedCounts.function_call_output).toBe(2);
expect(r.paths.some((p) => p.path === "/Users/wrandom/code/x.md")).toBe(true);
});
it("counts each tool_result part in a multi-result user turn", () => {
// Note: the Velocity-prior-art path pattern is greedy on the
// /Users/wrandom prefix and includes trailing whitespace and word
// chars until end-of-line; using \n separators here so each path
// is matched cleanly.
const content = JSON.stringify({
type: "user",
message: {
role: "user",
content: [
{ type: "tool_result", tool_use_id: "t1", content: "/Users/wrandom/code/a.md\nrunning" },
{ type: "tool_result", tool_use_id: "t2", content: "/Users/wrandom/code/b.md\nrunning" },
],
},
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
expect(r.omittedCounts.raw_tool_outputs).toBe(2);
expect(r.paths.some((p) => p.path === "/Users/wrandom/code/a.md")).toBe(true);
expect(r.paths.some((p) => p.path === "/Users/wrandom/code/b.md")).toBe(true);
});
it("mixed text + tool_use: visible text kept; tool_use counted", () => {
const content = JSON.stringify({
type: "assistant",
message: {
role: "assistant",
content: [
{ type: "text", text: "I'll run a command:" },
{ type: "tool_use", id: "t1", name: "Bash", input: { cmd: "pwd" } },
{ type: "text", text: "and report back" },
],
},
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
expect(r.messageCount).toBe(1);
expect(r.messages[0]!.text).toContain("I'll run a command:");
expect(r.messages[0]!.text).toContain("and report back");
expect(r.omittedCounts.function_call_output).toBe(1);
});
it("tool_use with non-path input: counted but contributes no paths", () => {
const content = JSON.stringify({
type: "assistant",
message: {
role: "assistant",
content: [
{ type: "tool_use", id: "t1", name: "Bash", input: { cmd: "echo hello" } },
{ type: "text", text: "done" },
],
},
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
expect(r.omittedCounts.function_call_output).toBe(1);
// No /Users/wrandom or recognized prefix path in the input.
expect(r.paths.length).toBe(0);
});
it("tool_result with non-path content: counted but contributes no paths", () => {
const content = JSON.stringify({
type: "user",
message: {
role: "user",
content: [
{ type: "tool_result", tool_use_id: "t1", content: "command exited with status 0" },
],
},
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
expect(r.omittedCounts.raw_tool_outputs).toBe(1);
expect(r.paths.length).toBe(0);
});
it("tool_result with array content shape: walks the array for paths", () => {
// Claude tool_result content can be either a string OR an array of
// { type: "text", text } parts (the same shape as message content).
const content = JSON.stringify({
type: "user",
message: {
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "t1",
content: [
{ type: "text", text: "found at /Users/wrandom/code/projects/openrig-hub/README.md" },
],
},
],
},
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
expect(r.omittedCounts.raw_tool_outputs).toBe(1);
expect(r.paths.some((p) => p.path === "/Users/wrandom/code/projects/openrig-hub/README.md")).toBe(true);
});
it("kept text-only assistant turn (no tools): counters stay at zero", () => {
const content = JSON.stringify({
type: "assistant",
message: { role: "assistant", content: [{ type: "text", text: "hello" }] },
cwd: "/x",
sessionId: "s",
});
const r = parseClaudeTranscript(content);
expect(r.messageCount).toBe(1);
expect(r.omittedCounts.function_call_output).toBe(0);
expect(r.omittedCounts.raw_tool_outputs).toBe(0);
});
});