feat(evidence): discover missing Evidence links without transferring records (#8633)

* feat(evidence): discover missing Evidence links without transferring records

An agent could only learn that a retained record's source was absent by reading
complete records or by having a downstream analysis refuse. The summary said
nothing about links, and complete-record retention was easy to mistake for
source closure.

- Add detail: links to get_evidence_bundle and CLI inspect-evidence-bundle:
  paged link rows with retained or missing status and no record payloads, for
  selected records against the whole session or file.
- Add retained-closure traversal, the walk composed workflows require, reporting
  complete or incomplete. Direct reads make no closure claim.
- Count direct and missing direct links in the summary.

Fixes #8454

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(evidence): count a repeated Evidence link once in every delivery

The Evidence schema does not deduplicate evidence_links, so the summary counted
a repeated link twice while link rows and counts reported it once. Both now read
one helper for a record's distinct direct links.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(evidence): validate link delivery against the advertised schemas

The test never called tools/list, so neither the shared validator nor the MCP
client checked its replies against what each profile advertises. A schema that
rejected the links result could have passed. Fetch the advertised definition and
validate every request and reply against it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Kaoru0822andClaude Opus 5.5 authored and GitHub committed 2026-11-02 05:14:34 -08:00
1 parent 93b8b82db7
commit 710fad8acf
17 files changed
+736 -44

No files matched your search

+18 -1
View File
@@ -390,9 +390,26 @@ that revision. If a list response overflows, retry on the same connection with
the same filters and offset and a smaller limit; export if one head still cannot
fit.
To find missing sources before a downstream analysis refuses them, read
`missing_direct_links` from the summary, or call `get_evidence_bundle` with
`detail: "links"`. The result (`kind: "evidence-link-inspection"`) transfers no
record payload. `filters` select the starting records, while membership is
checked against every record in the session or file, so a filtered-out source is
`retained`, not missing. Rows (`evidence_id`, `linked_evidence_id`, `status`) are
in canonical order, default to `link_status: "missing"` and page with `offset` and
`limit` (default 64); `links` counts every inspected link regardless of that
selection. `traversal: "direct"` (the default) reads the selected records' own
links and sets `retained_closure` to null, because it makes no closure claim.
`traversal: "retained-closure"` also follows retained links, as composed workflows
such as `build_call_path` do, and reports `complete` or `incomplete`; anything
beyond a missing link remains unknown. Missing means absent from this session or
file, not nonexistent, and neither status describes analysis coverage. Importing
a missing source changes this membership without changing any Evidence ID.
For a portable exported bundle, the CLI shares this projection:
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
`"detail":"records"` with an `evidence_id` filter returns that complete record.
`"detail":"records"` with an `evidence_id` filter returns that complete record,
and `"detail":"links"` reads link membership.
The CLI validates and materializes that local bundle under its existing memory
budget; summary delivery does not make arbitrary large files cheap to import.
+3 -1
View File
@@ -297,10 +297,12 @@ const projectWrite = (
export const inspectEvidenceBundleCommand = async (
input: {
readonly path: string;
readonly detail: "complete" | "summary" | "records";
readonly detail: "complete" | "summary" | "records" | "links";
readonly filters?: EvidenceRecordFilters | undefined;
readonly offset?: number | undefined;
readonly limit?: number | undefined;
readonly traversal?: "direct" | "retained-closure" | undefined;
readonly link_status?: "missing" | "retained" | "all" | undefined;
},
signal?: AbortSignal,
) => {
@@ -2,24 +2,41 @@ import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
inspectEvidenceLinks,
selectEvidenceRecords,
summarizeEvidenceBundle,
type EvidenceBundleSummary,
type EvidenceLinkInspection,
type EvidenceRecordFilters,
type EvidenceRecordSelection,
} from "../../domain/evidenceRecordSummary.js";
/** Select complete bundle delivery, discovery metadata, or a page of complete selected records. */
type InspectionDetail = "complete" | "summary" | "records" | "links";
/** Fields owned by particular deliveries, refused elsewhere rather than ignored. */
const DELIVERY_FIELDS = [
["offset", ["records", "links"]],
["limit", ["records", "links"]],
["traversal", ["links"]],
["link_status", ["links"]],
] as const;
/** Select complete delivery, discovery metadata, complete records, or link membership. */
export const inspectEvidenceBundle = (
bundle: EvidenceBundle,
options: {
readonly detail: "complete" | "summary" | "records";
readonly detail: InspectionDetail;
readonly filters?: EvidenceRecordFilters | undefined;
readonly offset?: number | undefined;
readonly limit?: number | undefined;
readonly traversal?: "direct" | "retained-closure" | undefined;
readonly link_status?: "missing" | "retained" | "all" | undefined;
},
): Result<
EvidenceBundle | EvidenceBundleSummary | EvidenceRecordSelection,
| EvidenceBundle
| EvidenceBundleSummary
| EvidenceRecordSelection
| EvidenceLinkInspection,
AnalysisInputError
> => {
if (options.detail === "complete" && options.filters !== undefined)
@@ -29,29 +46,34 @@ export const inspectEvidenceBundle = (
path: ["filters"],
reason: "invalid_value",
message:
"Record filters require detail: summary or records; complete returns the canonical bundle.",
expected: "detail: summary or records",
"Record filters require detail: summary, records or links; complete returns the canonical bundle.",
expected: "detail: summary, records or links",
},
]),
);
if (options.detail !== "records") {
const paging = (["offset", "limit"] as const).filter(
(field) => options[field] !== undefined,
);
if (paging.length > 0)
return err(
new AnalysisInputError(
"get_evidence_bundle",
undefined,
paging.map((field) => ({
const misplaced = DELIVERY_FIELDS.filter(
([field, details]) =>
options[field] !== undefined &&
!(details as readonly InspectionDetail[]).includes(options.detail),
);
if (misplaced.length > 0)
return err(
new AnalysisInputError(
"get_evidence_bundle",
undefined,
misplaced.map(([field, details]) => {
const expected = details
.map((detail) => `detail: ${detail}`)
.join(" or ");
return {
path: [field],
reason: "invalid_value" as const,
message: `${field} pages complete records and requires detail: records.`,
expected: "detail: records",
})),
),
);
}
message: `${field} requires ${expected}.`,
expected,
};
}),
),
);
if (options.detail === "records")
return ok(
selectEvidenceRecords(bundle, options.filters, {
@@ -59,6 +81,15 @@ export const inspectEvidenceBundle = (
limit: options.limit,
}),
);
if (options.detail === "links")
return ok(
inspectEvidenceLinks(bundle, options.filters, {
traversal: options.traversal,
linkStatus: options.link_status,
offset: options.offset,
limit: options.limit,
}),
);
return ok(
options.detail === "summary"
? summarizeEvidenceBundle(bundle, options.filters)
+2 -2
View File
@@ -26,12 +26,12 @@ export const registerAnalysisViewCommands = (
): void => {
cli.command(CLI_COMMANDS.inspectEvidenceBundle, {
description:
"Inspect a local Evidence bundle completely, as discovery metadata, or as a page of selected complete records",
"Inspect a local Evidence bundle completely, as discovery metadata, as a page of selected complete records, or as direct-link and retained-closure membership.",
args: z.object({
inputJson: z
.string()
.describe(
"JSON or JSON file with absolute path, detail, optional record filters, and records offset/limit",
"JSON or JSON file with absolute path, detail, optional record filters, records or links offset/limit, and links traversal/link_status",
),
}),
run: ({ args }) =>
+1 -1
View File
@@ -173,7 +173,7 @@ export const SESSION_TOOL_CONTRACTS = [
),
session(
"get_evidence_bundle",
"Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. detail=records returns a page (offset, limit) of the matching complete records with unchanged IDs and their missing direct evidence_links; it is not an importable bundle. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply to summary and records delivery.",
"Return every retained Evidence record and residual unknown as a complete canonical bundle by default. detail=summary discovers retained records without copying or transferring their payloads; optionally filter by exact Evidence ID, operation, target SHA-256, analysis-profile digest or canonical native procedure address. Metadata distinguishes complete record retention from native value-flow truncation or unavailable/unknown coverage. detail=records returns a page (offset, limit) of the matching complete records with unchanged IDs and their missing direct evidence_links; it is not an importable bundle. detail=links reads which evidence_links of the selected records this session or bundle retains, without their payloads: traversal retained-closure also follows retained links, the walk composed workflows require, and rows default to missing links. Use a discovered evidence_id with inspect_analysis_view without restarting a provider. Filters apply to summary, records and links delivery.",
getEvidenceBundleInputSchema,
),
{
+19 -4
View File
@@ -5,26 +5,41 @@ import { bundleComparisonInputSchema } from "../domain/bundleComparison.js";
import { prefixedDigestSchema } from "./../domain/digests.js";
import {
EVIDENCE_LINK_INSPECTION_LIMIT,
EVIDENCE_RECORD_SELECTION_LIMIT,
evidenceLinkStatusFilterSchema,
evidenceLinkTraversalSchema,
evidenceRecordFiltersSchema,
} from "../domain/evidenceRecordSummary.js";
import { residualUnknownSchema } from "../domain/residualUnknown.js";
/** Complete retained Evidence, selected discovery metadata, or a page of selected complete records. */
/** Complete retained Evidence, discovery metadata, a page of complete records, or link membership. */
export const getEvidenceBundleInputSchema = z.strictObject({
detail: z.enum(["complete", "summary", "records"]).default("complete"),
detail: z
.enum(["complete", "summary", "records", "links"])
.default("complete"),
filters: evidenceRecordFiltersSchema.optional(),
offset: z
.int()
.nonnegative()
.optional()
.describe("records only: first matching record to return; default 0."),
.describe("records or links: first matching row to return; default 0."),
limit: z
.int()
.positive()
.optional()
.describe(
`records only: maximum complete records to return; default ${String(EVIDENCE_RECORD_SELECTION_LIMIT)}.`,
`records: maximum complete records, default ${String(EVIDENCE_RECORD_SELECTION_LIMIT)}. links: maximum link rows, default ${String(EVIDENCE_LINK_INSPECTION_LIMIT)}.`,
),
traversal: evidenceLinkTraversalSchema
.optional()
.describe(
"links only: direct (default) reads the selected records' own links; retained-closure also follows retained links.",
),
link_status: evidenceLinkStatusFilterSchema
.optional()
.describe(
"links only: rows to return, default missing. Counts always cover every inspected link.",
),
});
+2
View File
@@ -46,6 +46,7 @@ import { jsonRecordSchema } from "../domain/jsonRecord.js";
import { byteSearchResultSchema } from "../domain/native/bytePattern.js";
import {
evidenceBundleSummarySchema,
evidenceLinkInspectionSchema,
evidenceRecordSelectionSchema,
} from "../domain/evidenceRecordSummary.js";
import { liveBinaryTargetSummaryVariants } from "../domain/liveBinaryTargetSummary.js";
@@ -537,6 +538,7 @@ export const sessionOutputSchemas = {
evidenceBundleSchema,
evidenceBundleSummarySchema,
evidenceRecordSelectionSchema,
evidenceLinkInspectionSchema,
]),
),
get_navigation_context: lifecycleResultOf(
+4
View File
@@ -56,6 +56,10 @@ export const MCP_TRANSPORT_EVIDENCE_DISCOVERY = `Discover retained records with
/** Next step when one MCP frame cannot carry a selected page of complete records. */
export const MCP_TRANSPORT_EVIDENCE_RECORDS = `Request fewer records with get_evidence_bundle detail: records, a smaller limit or an exact evidence_id filter; inspect compatible records in pages through inspect_analysis_view. A single record larger than one response needs export. ${MCP_TRANSPORT_EVIDENCE_EXPORT}`;
/** Next step when one MCP frame cannot carry a selected page of link rows. */
export const MCP_TRANSPORT_EVIDENCE_LINKS =
"Request fewer rows with get_evidence_bundle detail: links, a smaller limit or an exact evidence_id filter, then follow next_offset.";
/** Name the next step that is true for how long the caller keeps REA's runtime and session. */
export const analysisErrorRemediationAction = (
error: AnalysisError,
+160 -1
View File
@@ -53,6 +53,12 @@ export const evidenceBundleSummarySchema = z.strictObject({
total_retained_records: countSchema,
matching_records: countSchema,
unknown_revisions: countSchema,
direct_links: countSchema.describe(
"Distinct evidence_links of each matching record; detail links names them.",
),
missing_direct_links: countSchema.describe(
"Those direct links whose records this session or bundle does not retain. Zero establishes no transitive closure or analysis coverage.",
),
records: z.array(
z.strictObject({
evidence_id: evidenceSchema.shape.evidence_id,
@@ -149,13 +155,24 @@ function* matchingEvidenceRecords(
}
}
/** A record's distinct direct links; the Evidence schema does not deduplicate them. */
const directLinksOf = ({ evidence_links: links }: Evidence): string[] =>
uniqueSorted(links);
/** Discover retained Evidence without reading or copying its complete payloads. */
export const summarizeEvidenceBundle = (
bundle: EvidenceBundle,
filters: EvidenceRecordFilters = {},
): EvidenceBundleSummary => {
const records: EvidenceBundleSummary["records"] = [];
for (const { evidence, native } of matchingEvidenceRecords(bundle, filters))
const retained = new Set(bundle.records.map(({ evidence_id: id }) => id));
let directLinks = 0;
let missingDirectLinks = 0;
for (const { evidence, native } of matchingEvidenceRecords(bundle, filters)) {
for (const link of directLinksOf(evidence)) {
directLinks += 1;
if (!retained.has(link)) missingDirectLinks += 1;
}
records.push({
evidence_id: evidence.evidence_id,
subject: evidence.subject,
@@ -168,12 +185,15 @@ export const summarizeEvidenceBundle = (
limitation_count: evidence.limitations.length,
native_dossier: native,
});
}
return {
kind: "evidence-bundle-summary",
filters: selectedFilters(filters),
total_retained_records: bundle.records.length,
matching_records: records.length,
unknown_revisions: bundle.unknowns.length,
direct_links: directLinks,
missing_direct_links: missingDirectLinks,
records,
};
};
@@ -243,3 +263,142 @@ export const selectEvidenceRecords = (
limitations: [...EVIDENCE_RECORD_SELECTION_LIMITATIONS],
};
};
/** How far link inspection reads from the selected records. */
export const evidenceLinkTraversalSchema = z.enum([
"direct",
"retained-closure",
]);
/** Which inspected links a page returns; counts always cover all of them. */
export const evidenceLinkStatusFilterSchema = z.enum([
"missing",
"retained",
"all",
]);
/** Default page size for link rows, which carry two identifiers each. */
export const EVIDENCE_LINK_INSPECTION_LIMIT = 64;
/** Bounds what link membership establishes, in every inspection. */
export const EVIDENCE_LINK_INSPECTION_LIMITATIONS = [
"Link status is membership in this session or bundle: missing means absent here, not nonexistent, and neither status describes analysis coverage or a record's meaning.",
] as const;
/** Direct links, or the retained closure, of selected records without their payloads. */
export const evidenceLinkInspectionSchema = z.strictObject({
kind: z.literal("evidence-link-inspection"),
filters: z.record(z.string(), z.string()),
traversal: evidenceLinkTraversalSchema,
link_status: evidenceLinkStatusFilterSchema,
total_retained_records: countSchema,
matching_records: countSchema,
examined_records: countSchema.describe(
"Retained records whose evidence_links were read: the matching records, plus those reached through retained links when traversal is retained-closure.",
),
links: z
.strictObject({
total: countSchema,
retained: countSchema,
missing: countSchema,
})
.describe("Every link read from the examined records, before link_status."),
retained_closure: z
.enum(["complete", "incomplete"])
.nullable()
.describe(
"With retained-closure traversal, incomplete means a link could not be followed, so what lies beyond it remains unknown. Null for direct traversal or an empty selection, which make no closure claim.",
),
offset: countSchema,
limit: z.int().positive(),
matching_links: countSchema,
next_offset: countSchema.nullable(),
rows: z.array(
z.strictObject({
evidence_id: prefixedDigestSchema("ev"),
linked_evidence_id: prefixedDigestSchema("ev"),
status: z.enum(["retained", "missing"]),
}),
),
limitations: z.array(z.string()),
});
export type EvidenceLinkInspection = z.infer<
typeof evidenceLinkInspectionSchema
>;
/** Read link membership without copying or parsing any record payload. */
export const inspectEvidenceLinks = (
bundle: EvidenceBundle,
filters: EvidenceRecordFilters = {},
options: {
readonly traversal?:
| z.infer<typeof evidenceLinkTraversalSchema>
| undefined;
readonly linkStatus?:
| z.infer<typeof evidenceLinkStatusFilterSchema>
| undefined;
readonly offset?: number | undefined;
readonly limit?: number | undefined;
} = {},
): EvidenceLinkInspection => {
const traversal = options.traversal ?? "direct";
const linkStatus = options.linkStatus ?? "missing";
const offset = options.offset ?? 0;
const limit = options.limit ?? EVIDENCE_LINK_INSPECTION_LIMIT;
const retained = new Map(
bundle.records.map((record) => [record.evidence_id, record]),
);
const examined = new Set<string>();
const pending: string[] = [];
for (const { evidence } of matchingEvidenceRecords(bundle, filters))
pending.push(evidence.evidence_id);
const matching = pending.length;
while (pending.length > 0) {
const id = pending.pop();
if (id === undefined || examined.has(id)) continue;
examined.add(id);
if (traversal === "direct") continue;
for (const link of retained.get(id)?.evidence_links ?? [])
if (retained.has(link) && !examined.has(link)) pending.push(link);
}
const rows: EvidenceLinkInspection["rows"] = [];
const counts = { total: 0, retained: 0, missing: 0 };
let matchingLinks = 0;
// Bundle order is canonical, so a page stays stable across requests.
for (const evidence of bundle.records) {
const id = evidence.evidence_id;
if (!examined.has(id)) continue;
for (const link of directLinksOf(evidence)) {
const status = retained.has(link) ? "retained" : "missing";
counts.total += 1;
counts[status] += 1;
if (linkStatus !== "all" && linkStatus !== status) continue;
if (matchingLinks >= offset && rows.length < limit)
rows.push({ evidence_id: id, linked_evidence_id: link, status });
matchingLinks += 1;
}
}
const end = offset + rows.length;
return {
kind: "evidence-link-inspection",
filters: selectedFilters(filters),
traversal,
link_status: linkStatus,
total_retained_records: bundle.records.length,
matching_records: matching,
examined_records: examined.size,
links: counts,
// Nothing selected, or a direct read, supports no claim about a closure.
retained_closure:
traversal === "direct" || examined.size === 0
? null
: counts.missing === 0
? "complete"
: "incomplete",
offset,
limit,
matching_links: matchingLinks,
next_offset: end < matchingLinks ? end : null,
rows,
limitations: [...EVIDENCE_LINK_INSPECTION_LIMITATIONS],
};
};
+3
View File
@@ -16,6 +16,7 @@ import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js"
import {
MCP_TRANSPORT_EVIDENCE_DISCOVERY,
MCP_TRANSPORT_EVIDENCE_EXPORT,
MCP_TRANSPORT_EVIDENCE_LINKS,
MCP_TRANSPORT_EVIDENCE_RECORDS,
} from "../domain/analysisErrorPresentation.js";
import {
@@ -85,6 +86,8 @@ const evidenceBundleOverflowStep = (value: JsonValue): string => {
// A summary already used detail: summary; export writes outside one frame.
if (projection === "evidence-bundle-summary")
return MCP_TRANSPORT_EVIDENCE_EXPORT;
if (projection === "evidence-link-inspection")
return MCP_TRANSPORT_EVIDENCE_LINKS;
return projection === "evidence-record-selection"
? MCP_TRANSPORT_EVIDENCE_RECORDS
: MCP_TRANSPORT_EVIDENCE_DISCOVERY;
@@ -1,3 +1,4 @@
import { evidenceBundleSchema } from "../../../src/domain/evidenceBundle.js";
import { build } from "plist";
import { lstat, mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
@@ -216,11 +217,13 @@ cliTest.skipIf(process.platform !== "darwin")(
const retained = await call("get_evidence_bundle", {});
expect(retained.isError).not.toBe(true);
const evidenceBundle = toolContract(
"get_evidence_bundle",
).outputSchema.parse(retained.structuredContent);
const evidenceBundle = evidenceBundleSchema.parse(
toolContract("get_evidence_bundle").outputSchema.parse(
retained.structuredContent,
).result,
);
for (const operation of resourceOperations) {
const record = evidenceBundle.result.records.find(
const record = evidenceBundle.records.find(
(record) => record.operation === operation,
);
expectResourceEvidence(record, operation, firstBundle);
@@ -1,3 +1,4 @@
import { evidenceBundleSchema } from "../../../src/domain/evidenceBundle.js";
import { spawn } from "node:child_process";
import { once } from "node:events";
import { readFile, writeFile } from "node:fs/promises";
@@ -117,9 +118,11 @@ it("preserves file URL scheme spelling and script identity after real Inspector
const retained = await call("get_evidence_bundle", {});
expect(retained.isError, JSON.stringify(retained.content)).not.toBe(true);
const records = toolContract("get_evidence_bundle").outputSchema.parse(
retained.structuredContent,
).result.records;
const { records } = evidenceBundleSchema.parse(
toolContract("get_evidence_bundle").outputSchema.parse(
retained.structuredContent,
).result,
);
const original = records.find(
(record) => record.evidence_id === first.evidence_id,
);
@@ -1,3 +1,4 @@
import { evidenceBundleSchema } from "../../../src/domain/evidenceBundle.js";
import { expect, it } from "vitest";
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
@@ -141,9 +142,11 @@ it.each(["pseudocode", "overall"] as const)(
claims: { items: [{ unknown_ids: [] }] },
});
const exported = await call("get_evidence_bundle", { detail: "complete" });
const bundle = toolContract("get_evidence_bundle").outputSchema.parse(
exported.structuredContent,
).result;
const bundle = evidenceBundleSchema.parse(
toolContract("get_evidence_bundle").outputSchema.parse(
exported.structuredContent,
).result,
);
for (const source of [left, right, comparison])
expect(
bundle.records.find(
@@ -719,6 +719,54 @@ cliTest(
},
);
cliTest(
"reads link membership from a deliberately partial portable bundle",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-evidence-links-cli-");
const path = join(root, "partial.json");
const absent = analysisViewLayoutEvidence();
const linked = createEvidence(
undefined,
{ id: "fixture", name: "Fixture", version: "1" },
{
operation: "trace_feature",
parameters: {},
result: {},
evidenceLinks: [absent.evidence_id],
},
);
const bundle = createEvidenceBundle([linked]);
await writeFile(path, JSON.stringify(bundle));
const options = {
detail: "links",
traversal: "retained-closure",
} as const;
const result = await cli.run({
arguments: [
"inspect-evidence-bundle",
JSON.stringify({ path, ...options }),
"--json",
],
environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
});
expect(result.exitCode).toBe(0);
const expected = inspectEvidenceBundle(bundle, options);
if (!expected.ok) throw expected.error;
expect(result.json).toEqual(expected.value);
expect(result.json).toMatchObject({
kind: "evidence-link-inspection",
retained_closure: "incomplete",
rows: [
{
evidence_id: linked.evidence_id,
linked_evidence_id: absent.evidence_id,
status: "missing",
},
],
});
},
);
cliTest(
"formats the full caller-admitted native inventory page as Markdown and JSON",
async ({ cli }) => {
@@ -0,0 +1,398 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterAll, expect, it } from "vitest";
import { z } from "zod";
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
import { MCP_TRANSPORT_EVIDENCE_LINKS } from "../../../src/domain/analysisErrorPresentation.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
import { createEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
import {
EVIDENCE_LINK_INSPECTION_LIMIT,
EVIDENCE_LINK_INSPECTION_LIMITATIONS,
evidenceBundleSummarySchema,
evidenceLinkInspectionSchema,
} from "../../../src/domain/evidenceRecordSummary.js";
import { createToolResultDelivery } from "../../../src/server/toolResult.js";
import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const schemaValidation = createMcpSchemaValidation();
afterAll(() => schemaValidation.clear());
const GHIDRA = { id: "ghidra", name: "Ghidra", version: "12.1.4" };
const TARGET = {
path: "/fixtures/a.macho",
format: "mach-o" as const,
sha256: "a".repeat(64),
};
const record = (operation: string, evidenceLinks: readonly string[] = []) =>
createEvidence(TARGET, GHIDRA, {
operation,
parameters: {},
result: { operation },
evidenceLinks,
});
// top -> middle -> absent source; top -> leaf; unrelated stands alone.
const source = record("xrefs");
const leaf = record("procedure_address");
const middle = record("trace_feature", [source.evidence_id]);
const top = record("analyze_function_summary", [
middle.evidence_id,
leaf.evidence_id,
]);
const unrelated = record("list_names");
const PARTIAL = [top, middle, leaf, unrelated];
const connect = async (
options: {
readonly profile?: "full" | "compact";
readonly records?: readonly ReturnType<typeof record>[];
readonly maximumResponseBytes?: number;
} = {},
) => {
const session = createTestBinarySession(() => {
throw new Error("Link inspection must not open a provider");
});
const { client } = await createApplicationMcpHarness({
session,
clientIdentity: { name: "evidence-link-inspection", version: "1" },
schemaValidator: schemaValidation,
serverOptions: {
environment: {
...process.env,
REA_MCP_INPUT_SCHEMA_PROFILE: options.profile ?? "full",
REA_MCP_OUTPUT_SCHEMA_PROFILE: options.profile ?? "full",
},
...(options.maximumResponseBytes === undefined
? {}
: { delivery: createToolResultDelivery(options.maximumResponseBytes) }),
},
});
for (const retained of options.records ?? PARTIAL)
expect(session.recordEvidence(retained).ok).toBe(true);
const reply = (args: Record<string, unknown>) =>
client.callTool({ name: "get_evidence_bundle", arguments: args });
// Every delivered result must satisfy what this profile actually advertises.
const { tools } = await client.listTools();
const advertised = tools.find(({ name }) => name === "get_evidence_bundle");
if (advertised?.outputSchema === undefined)
throw new Error("get_evidence_bundle advertises no output schema");
const inputSchema: Record<string, unknown> = advertised.inputSchema;
const outputSchema: Record<string, unknown> = advertised.outputSchema;
const validation = schemaValidation.assertions;
expect(validation.validateSchema(inputSchema)).toBe(true);
expect(validation.validateSchema(outputSchema)).toBe(true);
const resultOf = async (args: Record<string, unknown>) => {
expect(
validation.validate(inputSchema, args),
JSON.stringify(validation.errors),
).toBe(true);
const result = await reply(args);
expect(result.isError, JSON.stringify(result)).not.toBe(true);
expect(
validation.validate(outputSchema, result.structuredContent),
JSON.stringify(validation.errors),
).toBe(true);
return z.object({ result: z.unknown() }).parse(result.structuredContent)
.result;
};
const links = async (args: Record<string, unknown> = {}) =>
evidenceLinkInspectionSchema.parse(
await resultOf({ detail: "links", ...args }),
);
const summary = async (args: Record<string, unknown> = {}) =>
evidenceBundleSummarySchema.parse(
await resultOf({ detail: "summary", ...args }),
);
return { client, session, reply, links, summary };
};
const exactly = (record_: { readonly evidence_id: string }) => ({
filters: { evidence_id: record_.evidence_id },
});
it.each(["full", "compact"] as const)(
"reads direct membership against the whole session, not the selection (%s)",
async (profile) => {
const { links, summary } = await connect({ profile });
// A retained-but-unselected source is retained, and direct makes no closure claim.
const selected = await links(exactly(top));
expect(selected).toEqual({
kind: "evidence-link-inspection",
filters: exactly(top).filters,
traversal: "direct",
link_status: "missing",
total_retained_records: 4,
matching_records: 1,
examined_records: 1,
links: { total: 2, retained: 2, missing: 0 },
retained_closure: null,
offset: 0,
limit: EVIDENCE_LINK_INSPECTION_LIMIT,
matching_links: 0,
next_offset: null,
rows: [],
limitations: [...EVIDENCE_LINK_INSPECTION_LIMITATIONS],
});
// The record that names the absent source reports it without any payload.
const missing = await links(exactly(middle));
expect(missing.links).toEqual({ total: 1, retained: 0, missing: 1 });
expect(missing.rows).toEqual([
{
evidence_id: middle.evidence_id,
linked_evidence_id: source.evidence_id,
status: "missing",
},
]);
expect(JSON.stringify(missing)).not.toContain("trace_feature");
// The summary counts direct links only for its matching records.
expect(await summary()).toMatchObject({
matching_records: 4,
direct_links: 3,
missing_direct_links: 1,
});
expect(await summary(exactly(top))).toMatchObject({
matching_records: 1,
direct_links: 2,
missing_direct_links: 0,
});
},
);
it("follows retained links only when asked and leaves the rest unknown", async () => {
const { links } = await connect();
const walk = { traversal: "retained-closure" };
// Top looks closed directly, but its closure reaches the absent source.
const closure = await links({ ...exactly(top), ...walk });
expect(closure).toMatchObject({
traversal: "retained-closure",
matching_records: 1,
examined_records: 3,
links: { total: 3, retained: 2, missing: 1 },
retained_closure: "incomplete",
rows: [
{
evidence_id: middle.evidence_id,
linked_evidence_id: source.evidence_id,
status: "missing",
},
],
});
// A branch without an absent link is complete, including a record with no links.
expect(await links({ ...exactly(leaf), ...walk })).toMatchObject({
examined_records: 1,
links: { total: 0, retained: 0, missing: 0 },
retained_closure: "complete",
});
// Selecting nothing supports no claim about a closure.
expect(
await links({ filters: { operation: "absent_operation" }, ...walk }),
).toMatchObject({
matching_records: 0,
examined_records: 0,
links: { total: 0 },
retained_closure: null,
rows: [],
});
});
it("counts a repeated link once in every delivery", async () => {
const repeated = record("trace_feature", [
source.evidence_id,
source.evidence_id,
]);
const { links, summary } = await connect({ records: [repeated] });
expect(await summary()).toMatchObject({
direct_links: 1,
missing_direct_links: 1,
});
const inspected = await links();
expect(inspected.links).toEqual({ total: 1, retained: 0, missing: 1 });
expect(inspected.rows).toHaveLength(1);
});
it("pages selected link rows in a stable order without skips or duplicates", async () => {
const { links } = await connect();
const everything = await links({ link_status: "all" });
expect(everything).toMatchObject({
link_status: "all",
links: { total: 3, retained: 2, missing: 1 },
matching_links: 3,
next_offset: null,
});
const key = ({
evidence_id: id,
linked_evidence_id: link,
}: (typeof everything.rows)[number]) => `${id}>${link}`;
const order = everything.rows.map(key);
expect(order).toEqual(
[...order].sort((left, right) => (left < right ? -1 : 1)),
);
const paged: string[] = [];
let offset: number | null = 0;
while (offset !== null) {
const page: z.output<typeof evidenceLinkInspectionSchema> = await links({
link_status: "all",
offset,
limit: 1,
});
expect(page.rows).toHaveLength(1);
// Counts describe every inspected link on every page.
expect(page.links.total).toBe(3);
paged.push(...page.rows.map(key));
offset = page.next_offset;
}
expect(paged).toEqual(order);
const retainedOnly = await links({ link_status: "retained" });
expect(retainedOnly.matching_links).toBe(2);
expect(retainedOnly.rows.every(({ status }) => status === "retained")).toBe(
true,
);
expect(await links({ link_status: "all", offset: 9 })).toMatchObject({
matching_links: 3,
next_offset: null,
rows: [],
});
});
it("reports the source a composed workflow refuses, and clears it on import", async () => {
const dossier = FUNCTION_COMPARISON_EXAMPLE.left;
if (dossier.subject === null) throw new Error("Expected a subject");
const health = record("get_project_health");
const linked = createEvidence(
{
path: dossier.subject.local_path,
sha256: dossier.subject.digest.sha256,
format: dossier.subject.format,
},
dossier.provider,
{
operation: dossier.operation,
parameters: dossier.parameters,
result: dossier.normalized_result,
confidence: dossier.confidence,
authority: dossier.authority,
evidenceLinks: [health.evidence_id],
},
);
const { client, links, summary } = await connect({ records: [linked] });
const walk = { traversal: "retained-closure" };
expect(await links(walk)).toMatchObject({
retained_closure: "incomplete",
rows: [
{
evidence_id: linked.evidence_id,
linked_evidence_id: health.evidence_id,
},
],
});
expect(await summary()).toMatchObject({ missing_direct_links: 1 });
const procedure = z
.looseObject({ procedure: z.looseObject({ address: z.string() }) })
.parse(dossier.normalized_result).procedure.address;
const callPath = () =>
client.callTool({
name: "build_call_path",
arguments: {
start: { address: procedure },
goal: { address: procedure },
functions: [
{ kind: "retained-evidence", evidence_id: linked.evidence_id },
],
},
});
// The inspection predicts the same absent ID the workflow refuses.
expect(JSON.stringify(parseMcpToolError(await callPath()))).toContain(
health.evidence_id,
);
const root = await createTestTempDirectory("rea-evidence-links-");
const bundlePath = join(root, "source.json");
await writeFile(bundlePath, JSON.stringify(createEvidenceBundle([health])));
const imported = await client.callTool({
name: "import_evidence_bundle",
arguments: { path: bundlePath },
});
expect(imported.isError, JSON.stringify(imported)).not.toBe(true);
// Membership changes, the original record does not, and the workflow proceeds.
expect(await links(walk)).toMatchObject({
examined_records: 2,
links: { total: 1, retained: 1, missing: 0 },
retained_closure: "complete",
rows: [],
});
expect(await summary()).toMatchObject({ missing_direct_links: 0 });
const proceeded = await callPath();
expect(proceeded.isError, JSON.stringify(proceeded)).not.toBe(true);
expect(parseEvidence(proceeded.structuredContent).evidence_links).toContain(
linked.evidence_id,
);
});
it("refuses link selections outside link delivery and names a smaller page", async () => {
const { reply } = await connect();
for (const [detail, field, value, expected] of [
["summary", "traversal", "direct", "detail: links"],
["records", "link_status", "all", "detail: links"],
["complete", "traversal", "retained-closure", "detail: links"],
["summary", "offset", 1, "detail: records or detail: links"],
["summary", "limit", 1, "detail: records or detail: links"],
] as const)
expect(
parseMcpToolError(await reply({ detail, [field]: value })).error,
).toMatchObject({
code: "invalid_request",
details: {
issues: [expect.objectContaining({ path: [field], expected })],
},
});
expect(
parseMcpToolError(
await reply({ detail: "complete", filters: exactly(top).filters }),
).error,
).toMatchObject({ code: "invalid_request" });
// Many links on one record overflow a small response budget with a named step.
const many = Array.from(
{ length: 64_000 },
(_, index) => `ev_${index.toString(16).padStart(64, "0")}`,
);
const crowded = await connect({
records: [record("trace_feature", many)],
maximumResponseBytes: 10 * 1024 * 1024,
});
const large = await crowded.reply({ detail: "links", limit: 64_000 });
expect(parseMcpToolError(large).error).toMatchObject({
code: "resource_constraint",
remediation: {
action: expect.stringContaining(MCP_TRANSPORT_EVIDENCE_LINKS),
},
});
expect(
evidenceLinkInspectionSchema.parse(
z
.object({ result: z.unknown() })
.parse(
(await crowded.reply({ detail: "links", limit: 2 }))
.structuredContent,
).result,
),
).toMatchObject({
links: { total: 64_000, missing: 64_000 },
next_offset: 2,
});
});
@@ -356,9 +356,11 @@ describe("MCP composed workflow snapshot replay", () => {
arguments: {},
});
expect(retained.isError, JSON.stringify(retained.content)).not.toBe(true);
const retainedBundle = toolContract(
"get_evidence_bundle",
).outputSchema.parse(retained.structuredContent).result;
const retainedBundle = evidenceBundleSchema.parse(
toolContract("get_evidence_bundle").outputSchema.parse(
retained.structuredContent,
).result,
);
expect(
retainedBundle.records.find(
({ evidence_id }) => evidence_id === primitiveEvidence.evidence_id,
@@ -199,6 +199,8 @@ it("retains summary discovery in the generic lifecycle reader but refuses it as
total_retained_records: 0,
matching_records: 0,
unknown_revisions: 0,
direct_links: 0,
missing_direct_links: 0,
records: [],
};
expect(