Compare commits

...
Author SHA1 Message Date
morluto bd234dedfa Merge commit 'ac30fbd' into HEAD 2026-11-07 23:15:35 +08:00
morluto 1f4edf466c fix: preserve call discovery across facet limits 2026-11-07 23:15:08 +08:00
morluto 95c8fc2aa5 Merge remote-tracking branch 'origin/main' into HEAD 2026-11-07 22:53:59 +08:00
Cursor Agent 367be83922 merge(main): revise call-path profiles without collapsing value traces
trace_call_path and trace_native_values both use workflow revision 2.
Value-trace absence stays specific to trace_native_values, and an
unrevised operation still matches the default profile.
2026-11-07 11:53:30 +00:00
morluto 670d025e33 merge: integrate current main into call discovery scope 2026-11-04 08:31:28 +08:00
morluto ef5a7e68b8 merge: integrate current main into call discovery scope 2026-11-02 07:47:23 +08:00
morluto b7160bd7c6 test: declare discovery in reconstruction controls 2026-11-01 14:43:11 +08:00
morluto 8dd4cf4a9d fix(evidence): retain discovery limits in call verification 2026-11-01 14:29:34 +08:00
morluto 30f5df2cd9 fix: preserve call discovery reasons in path evidence 2026-11-01 14:07:07 +08:00
morluto fd3b42b285 fix: preserve compact call coverage schema guidance 2026-11-01 14:03:20 +08:00
morluto 3310d82413 fix!: qualify call paths by provider discovery 2026-11-01 13:53:43 +08:00
40 changed files with 1136 additions and 158 deletions

No files matched your search

+21 -6
View File
@@ -129,7 +129,15 @@ side declares a required facet unavailable. An empty legacy facet without a
coverage declaration is also unknown. A producer that actually observed an
empty facet can declare it available so observed absence remains comparable.
Call comparison additionally retains the Evidence relationship coverage
checks. Ghidra and Hopper declare their observed dossier facets and complete
checks. Its call status compares the provider-observed sets; discovery
limitations stay on that dimension even when the observed sets match or differ.
The calls dimension's `discovery_status` records whether discovery is complete,
partial, or unknown across both sides and directions; older comparisons may omit
it. Incomplete or unreported discovery also remains a residual unknown and an
unresolved `find_changed_behavior` finding, including when observed sets match.
A structural reconstruction claim about calls remains unknown unless both
directions have established static direct-call discovery. An independent
observed change in another dimension can still fail an overall claim. Ghidra and Hopper declare their observed dossier facets and complete
provider-observed caller/callee sets, including empty sets. Missing decompiler
output remains unavailable; complete call coverage does not claim every runtime
target. When Ghidra's decompiler fails on one function, its dossier is still
@@ -206,11 +214,18 @@ relationship coverage and read-specific `limitations`, alongside observed
`calls`. When a provider does not classify its caveats, they remain on that
node and coverage is `unknown`. An empty call list with partial or unknown
coverage is not proof that the function has no calls.
`trace_call_path` reports each limited expansion in `residual_unknowns` and
sets `truncated: true`; any path already supported by observed edges remains
visible. When a requested goal is not reached but an expansion is unresolved,
`goal_status` is `unknown`; `not_reached` means traversal exhausted without
unresolved relationship reads. Cancellation remains a tool error.
`trace_call_path` reports limited relationship reads in `residual_unknowns` and
sets `truncated: true`; any path supported by observed edges remains visible.
`search_scope.observed_goal_status` describes the search of the provider-observed
graph, while `search_scope.discovery` describes whether discovery of applicable
direct relationships is established. A missing goal is `unknown` when either
discovery or a relationship read is unestablished. A found path is shortest
within the observed graph; undiscovered calls may produce a shorter path.
`truncated` describes relationship reads, not unestablished discovery.
`build_call_path` applies the same distinction to supplied function Evidence:
its `search_scope.exhaustive` describes the supplied graph and
`search_scope.observed_status` preserves that graph's result. Evidence without
discovery metadata has unknown discovery. Cancellation remains a tool error.
`trace_feature` validates each reference's containing-procedure resolution using
the same contract as navigation context. Explicit absence remains a successful
+5 -1
View File
@@ -178,7 +178,11 @@ does not establish another instance.
provider-observed direct relationship set. `complete` means that set was
returned without a known read limit; it does not claim every possible runtime
edge. Provider and execution caveats stay in `limitations` and do not imply
incomplete relationship coverage. Missing coverage remains `unknown`.
incomplete relationship coverage. Directional `discovery` separately describes
whether applicable static relationships could have been missed by discovery;
missing discovery is unknown. Preserve paths supported by observed edges,
and qualify negative and shortest-path claims by discovery scope. Missing
relationship coverage remains `unknown`.
- Declare process, filesystem, network, UI, and mutation effects truthfully;
enforce the requested target and lifecycle.
- Add limits only for real format, protocol, authority, or measured resource
+65 -33
View File
@@ -1,5 +1,6 @@
import type { CallRelationshipDirection } from "../contracts/enhancedInputs.js";
import {
CALL_RELATIONSHIP_TRAVERSALS,
cleanupStoppedTraversal,
readCallRelationships,
type CallRelationshipObserver,
@@ -18,7 +19,11 @@ import {
type EnhancedResult,
} from "./EnhancedToolTypes.js";
import { resolveProcedureAddress } from "./ProcedureAddressResolution.js";
import type { RelationshipCoverage } from "../domain/relationshipCoverage.js";
import {
combineRelationshipDiscovery,
type RelationshipCoverage,
type RelationshipDiscoveryStatus,
} from "../domain/relationshipCoverage.js";
type AnalysisCall = (
name: AnalysisOperation,
@@ -57,6 +62,7 @@ interface TraceState {
readonly residual: Set<string>;
readonly limitations: Set<string>;
readonly relationshipCoverage: Map<string, RelationshipCoverage>;
discovery: RelationshipDiscoveryStatus;
queueIndex: number;
traversalPath: string[];
}
@@ -150,6 +156,7 @@ const createTraceState = (input: CallPathTraceInput): TraceState => ({
residual: new Set(),
limitations: new Set(),
relationshipCoverage: new Map(),
discovery: "complete",
queueIndex: 0,
traversalPath: input.goal === input.start ? [input.start] : [],
});
@@ -159,12 +166,20 @@ const expandAddress = async (
expansion: AddressExpansion,
): Promise<Result<boolean, AnalysisError>> => {
const { input, state, address, current, signal } = expansion;
const { relation } = CALL_RELATIONSHIP_TRAVERSALS[input.direction];
const read = await readCallRelationships(
call,
input.direction,
address,
signal,
);
if (read.status !== "observed") {
state.discovery = combineRelationshipDiscovery(
state.discovery,
null,
relation,
);
}
// A provider failure delivered with cancellation remains the selected outcome.
if (
read.status === "stopped" ||
@@ -193,6 +208,11 @@ const expandAddress = async (
reason: "Provider did not report relationship coverage.",
};
state.relationshipCoverage.set(address, coverage);
state.discovery = combineRelationshipDiscovery(
state.discovery,
coverage,
relation,
);
for (const diagnostic of "diagnostics" in coverage &&
Array.isArray(coverage.diagnostics)
? coverage.diagnostics
@@ -255,47 +275,59 @@ const recordEdge = (
});
};
const projectTrace = (input: CallPathTraceInput, state: TraceState) => ({
start: input.start,
goal: input.goal ?? null,
direction: input.direction,
goal_status:
const projectTrace = (input: CallPathTraceInput, state: TraceState) => {
const observedGoalStatus =
input.goal === undefined
? "not_requested"
: state.traversalPath.length > 0
? "reached"
: state.residual.size > 0
? "unknown"
: "not_reached",
nodes: [...state.visited.entries()]
.map(([address, { depth }]) => ({ address, depth }))
.sort(
: "not_reached";
return {
start: input.start,
goal: input.goal ?? null,
direction: input.direction,
goal_status:
observedGoalStatus === "not_reached" && state.discovery !== "complete"
? "unknown"
: observedGoalStatus,
search_scope: {
graph: "provider_observed_graph",
discovery: state.discovery,
observed_goal_status: observedGoalStatus,
},
nodes: [...state.visited.entries()]
.map(([address, { depth }]) => ({ address, depth }))
.sort(
(left, right) =>
left.depth - right.depth ||
compareAddress(left.address, right.address),
),
edges: [...state.edges.values()].sort(
(left, right) =>
left.depth - right.depth || compareAddress(left.address, right.address),
left.discovery_depth - right.discovery_depth ||
compareAddress(left.source_address, right.source_address) ||
compareAddress(left.target_address, right.target_address),
),
edges: [...state.edges.values()].sort(
(left, right) =>
left.discovery_depth - right.discovery_depth ||
compareAddress(left.source_address, right.source_address) ||
compareAddress(left.target_address, right.target_address),
),
traversal_path: state.traversalPath,
failures: state.failures,
traversal: {
nodes_visited: state.visited.size,
},
relationship_coverage: Object.fromEntries(
[...state.relationshipCoverage.entries()].sort(([left], [right]) =>
compareAddress(left, right),
traversal_path: state.traversalPath,
failures: state.failures,
traversal: {
nodes_visited: state.visited.size,
},
relationship_coverage: Object.fromEntries(
[...state.relationshipCoverage.entries()].sort(([left], [right]) =>
compareAddress(left, right),
),
),
),
truncated: state.residual.size > 0,
residual_unknowns: [...state.residual],
limitations: [
"Direct provider call relationships may omit unresolved indirect calls.",
...state.limitations,
],
});
truncated: state.residual.size > 0,
residual_unknowns: [...state.residual],
limitations: [
"Direct provider call relationships may omit unresolved indirect calls.",
...state.limitations,
],
};
};
const compareAddress = (left: string, right: string): number =>
left < right ? -1 : left > right ? 1 : 0;
+2 -1
View File
@@ -24,7 +24,8 @@ export const workflowAnalysisProfile = (
? { workflow_revision: 4 }
: operation === "get_call_graph"
? { workflow_revision: 3 }
: operation === "trace_native_values"
: operation === "trace_call_path" ||
operation === "trace_native_values"
? { workflow_revision: 2 }
: { workflow_revision: 1 }),
});
@@ -1490,7 +1490,7 @@ describe("executions that report truncated references", () => {
});
});
it("revises the trace_feature workflow profile so earlier results are not replayed", () => {
it("revises trace workflow profiles so earlier results are not replayed", () => {
const profile = ANALYSIS_SNAPSHOT_PROFILE;
expect(
@@ -1502,9 +1502,12 @@ it("revises the trace_feature workflow profile so earlier results are not replay
workflowAnalysisProfile(profile),
);
expect(
workflowAnalysisProfile(profile, "trace_call_path"),
"other workflows keep their profile",
).toEqual(workflowAnalysisProfile(profile));
workflowAnalysisProfile(profile, "trace_call_path").parameters
.workflow_revision,
).toBe(2);
expect(workflowAnalysisProfile(profile, "trace_call_path")).not.toEqual(
workflowAnalysisProfile(profile),
);
});
it("binds native value-trace absence semantics without changing other workflows", () => {
@@ -1514,7 +1517,7 @@ it("binds native value-trace absence semantics without changing other workflows"
"recorded-versus-unavailable-v1",
);
expect(valueTrace.digest).not.toBe(workflowAnalysisProfile(profile).digest);
expect(workflowAnalysisProfile(profile, "trace_call_path")).toEqual(
expect(workflowAnalysisProfile(profile, "xrefs")).toEqual(
workflowAnalysisProfile(profile),
);
});
@@ -1,5 +1,8 @@
import { functionComparisonInputSchema } from "../../domain/functionComparisonSchemas.js";
import { FUNCTION_COMPARISON_PROVIDER } from "../../domain/comparisonEvidenceIdentity.js";
import {
FUNCTION_COMPARISON_PROFILE,
FUNCTION_COMPARISON_PROVIDER,
} from "../../domain/comparisonEvidenceIdentity.js";
import { compareFunctions } from "../../domain/functionComparison.js";
import { parseFunctionEvidence } from "../../domain/functionDossierEvidence.js";
import { createEvidence } from "../../domain/evidence.js";
@@ -50,6 +53,7 @@ export const compareFunctionEvidence = (
const evidence = createEvidence(undefined, FUNCTION_COMPARISON_PROVIDER, {
predicateType: "rea.function-comparison",
operation: "compare_functions",
analysisProfile: FUNCTION_COMPARISON_PROFILE,
parameters: {
left_evidence_id: leftEvidence.evidence_id,
right_evidence_id: rightEvidence.evidence_id,
@@ -81,29 +85,44 @@ const comparisonUnknownInput = (
leftIds: readonly string[],
rightIds: readonly string[],
comparisonEvidenceId: string,
): RecordUnknownInput | undefined =>
functionUnknownInput({
status: comparison.status,
dimensions: comparison.dimensions
.filter(({ status }) =>
comparison.status === "unknown"
? status === "unknown"
: status === "changed",
)
.map(({ dimension }) => dimension),
): RecordUnknownInput | undefined => {
const discoveryUnresolved = comparison.dimensions.some(
({ dimension, discovery_status }) =>
dimension === "calls" && discovery_status !== "complete",
);
const status =
comparison.status === "unchanged" && discoveryUnresolved
? "unknown"
: comparison.status;
const dimensions = comparison.dimensions
.filter(
({ dimension, status: dimensionStatus }) =>
(status === "unknown"
? dimensionStatus === "unknown"
: dimensionStatus === "changed") ||
(dimension === "calls" && discoveryUnresolved),
)
.map(({ dimension }) => dimension);
return functionUnknownInput({
status,
dimensions,
discoveryUnresolved,
leftIds,
rightIds,
comparisonEvidenceId,
});
};
const functionUnknownInput = ({
status,
dimensions,
discoveryUnresolved,
leftIds,
rightIds,
comparisonEvidenceId,
}: {
status: ReturnType<typeof compareFunctions>["status"];
discoveryUnresolved: boolean;
dimensions: readonly ReturnType<
typeof compareFunctions
>["dimensions"][number]["dimension"][];
@@ -118,7 +137,7 @@ const functionUnknownInput = ({
const contradictingIds = rightIds.filter((id) => !leftIds.includes(id));
const contradictory = status === "changed";
return {
question: `Function comparison is ${status} (comparison ${comparisonEvidenceId})`,
question: `Function comparison is ${status}${discoveryUnresolved ? "; call discovery is not fully established" : ""} (comparison ${comparisonEvidenceId})`,
severity: status === "changed" ? "medium" : "high",
domain: "function-comparison",
comparison_conclusion_scope: {
@@ -9,6 +9,7 @@ import {
import { buildCallPath } from "../../domain/callPath.js";
import { correlateStaticAndRuntime } from "../../domain/staticRuntimeCorrelation.js";
import { createEvidence, type Evidence } from "../../domain/evidence.js";
import { createAnalysisProfile } from "../../domain/analysisProfile.js";
import { readImmutableEvidence } from "../../domain/evidenceValidation.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
@@ -92,6 +93,9 @@ export const buildCallPathEvidence = (
const evidence = createEvidence(undefined, CALL_PATH_PROVIDER, {
predicateType: "rea.call-path",
operation: "build_call_path",
analysisProfile: createAnalysisProfile(CALL_PATH_PROVIDER, {
workflow_revision: 2,
}),
parameters: {
start: selected.value.input.start.address,
goal: selected.value.input.goal.address,
@@ -112,7 +116,7 @@ export const buildCallPathEvidence = (
{
operation: "analyze_function",
rationale:
"Collect complete callee dossiers for the reported frontier addresses.",
"Inspect reachable code and compare independent provider observations where call discovery is uncertain.",
},
],
});
+1 -1
View File
@@ -87,7 +87,7 @@ export const ENHANCED_TOOL_CONTRACTS = [
),
enhanced(
"trace_call_path",
"Resolve start and optional goal symbols or addresses to canonical procedure entries, then trace direct callers or callees until the graph is exhausted or the goal is reached. Results use canonical addresses; Evidence preserves supplied selectors. Returns visited nodes, direct-call edges, a shortest traversal path, provider failures, and residual unknowns. A requested goal is unknown when unresolved expansions prevent proving it absent; unresolved indirect calls remain unknown.",
"Resolve start and optional goal symbols or addresses, then trace direct callers or callees. Returns observed edges and a shortest path within the provider-observed graph. search_scope distinguishes the observed-graph result from whether discovery is established; a missing goal is unknown when relationship reads or discovery are incomplete. Unresolved indirect calls remain unknown.",
enhancedInputSchemas.trace_call_path,
),
enhanced(
+5 -1
View File
@@ -1,5 +1,8 @@
import { createEvidence } from "../domain/evidence.js";
import { FUNCTION_COMPARISON_PROVIDER } from "../domain/comparisonEvidenceIdentity.js";
import {
FUNCTION_COMPARISON_PROFILE,
FUNCTION_COMPARISON_PROVIDER,
} from "../domain/comparisonEvidenceIdentity.js";
import { compareProcessCaptureEvidence } from "../domain/process/processComparisonEvidence.js";
import { PROCESS_PROVIDER } from "../domain/process/processEvidenceProvider.js";
import { parseProcessCapture } from "../domain/process/processCaptureParsing.js";
@@ -26,6 +29,7 @@ export const FUNCTION_COMPARISON_EVIDENCE = createEvidence(
{
predicateType: "rea.function-comparison",
operation: "compare_functions",
analysisProfile: FUNCTION_COMPARISON_PROFILE,
parameters: {
left_evidence_id: left.value.evidence.evidence_id,
right_evidence_id: right.value.evidence.evidence_id,
+10
View File
@@ -279,6 +279,16 @@ const callPathTraceOutput = evidenceResultOf(
goal: z.string().nullable(),
direction: z.enum(["forward", "backward"]),
goal_status: z.enum(["not_requested", "reached", "not_reached", "unknown"]),
search_scope: z.object({
graph: z.literal("provider_observed_graph"),
discovery: z.enum(["complete", "partial", "unknown"]),
observed_goal_status: z.enum([
"not_requested",
"reached",
"not_reached",
"unknown",
]),
}),
nodes: z.array(
z.object({
address: z.string(),
+5 -1
View File
@@ -18,8 +18,10 @@ const referenceCoverage = (
source.diagnostics === undefined
? {}
: { diagnostic_ids: source.diagnostics.map(intern) };
const discovery =
source.discovery === undefined ? {} : { discovery: source.discovery };
if (source.status === "complete")
return { status: "complete", ...diagnostic };
return { status: "complete", ...diagnostic, ...discovery };
const directions =
source.limited_directions === undefined
? {}
@@ -30,12 +32,14 @@ const referenceCoverage = (
limitation_ids: source.limitations.map(intern),
...directions,
...diagnostic,
...discovery,
}
: {
status: "unknown",
reason_id: intern(source.reason),
...directions,
...diagnostic,
...discovery,
};
};
+44 -13
View File
@@ -196,7 +196,15 @@ describe("call path reconstruction", () => {
});
it("only reports not_found for exhaustive closure", () => {
const notFound = run([observe("0x1000", []), observe("0x4000", [])]);
const notFound = run([
observe("0x1000", [], "rea-workflow", {
relationshipCoverage: {
status: "complete",
discovery: { callees: { status: "complete" } },
},
}),
observe("0x4000", []),
]);
expect(notFound.status).toBe("not_found");
expect(
callPathResultSchema.safeParse({
@@ -227,12 +235,6 @@ describe("call path reconstruction", () => {
);
});
it("searches the complete supplied graph", () => {
expect(
run([observe("0x1000", ["0x2000"]), observe("0x2000", [])]).status,
).toBe("not_found");
});
it("returns all shortest paths inline without paging", () => {
const result = run([
observe("0x1000", ["0x2000", "0x3000"]),
@@ -246,6 +248,19 @@ describe("call path reconstruction", () => {
});
});
it("keeps an exhaustive supplied graph distinct from unknown discovery", () => {
expect(
run([observe("0x1000", ["0x2000"]), observe("0x2000", [])]),
).toMatchObject({
status: "unknown",
search_scope: {
exhaustive: true,
discovery: "unknown",
observed_status: "not_found",
},
});
});
describe("call path topology and capacity", () => {
it("retains every distinct generic limitation beyond the push argument limit", () => {
const limitations = Array.from(
@@ -430,6 +445,7 @@ describe("directional call relationship coverage", () => {
status: "partial",
limitations: [limitation],
limited_directions: directions,
discovery: { callees: { status: "complete" } },
};
const result = run([
observe("0x1000", ["0x2000"], "rea-workflow", { relationshipCoverage }),
@@ -514,14 +530,14 @@ describe("call path search extent", () => {
label: "expanded cycle without the goal",
graph: { "0x1000": ["0x2000"], "0x2000": ["0x1000", "0x2000"] },
goal: "0x4000",
status: "not_found",
status: "unknown",
explored: { nodes: 2, edges: 3, depth_reached: 1 },
},
{
label: "unreachable goal with its own dossier",
graph: { "0x1000": ["0x2000"], "0x2000": [], "0x4000": ["0x5000"] },
goal: "0x4000",
status: "not_found",
status: "unknown",
explored: { nodes: 2, edges: 1, depth_reached: 1 },
},
{
@@ -630,10 +646,18 @@ describe("explicit call relationship coverage", () => {
"Callers: Upstream xrefs do not establish direct caller relationships.";
const result = run([
observe("0x1000", ["0x2000"], "rea-workflow", {
relationshipCoverage: { status: "unknown", reason },
relationshipCoverage: {
status: "unknown",
reason,
discovery: { callees: { status: "complete" } },
},
}),
observe("0x2000", [], "rea-workflow", {
relationshipCoverage: { status: "unknown", reason },
relationshipCoverage: {
status: "unknown",
reason,
discovery: { callees: { status: "complete" } },
},
}),
]);
expect(result).toMatchObject({
@@ -683,7 +707,10 @@ describe("explicit call relationship coverage", () => {
const reason = "An unrelated function's callees were not observed.";
const result = run([
observe("0x1000", [], "rea-workflow", {
relationshipCoverage: { status: "complete" },
relationshipCoverage: {
status: "complete",
discovery: { callees: { status: "complete" } },
},
}),
observe("0x2000", ["0x5000"], "rea-workflow", {
relationshipCoverage: { status: "unknown", reason },
@@ -732,7 +759,11 @@ describe("call path evidence admission", () => {
run([observe("0x001000", [])], {
start: { address: "0X00001000" },
}),
).toMatchObject({ status: "not_found", start: "0x1000" });
).toMatchObject({
status: "unknown",
start: "0x1000",
search_scope: { observed_status: "not_found" },
});
expect(() =>
run([observe("0x1000", [])], { start: { address: "1000" } }),
).toThrow();
+31 -3
View File
@@ -1,5 +1,9 @@
import { z } from "zod";
import { relationshipCoverageLimitations } from "./relationshipCoverage.js";
import {
combineRelationshipDiscovery,
relationshipCoverageLimitations,
type RelationshipDiscoveryStatus,
} from "./relationshipCoverage.js";
import {
callPathInputSchema,
callPathResultSchema,
@@ -79,6 +83,7 @@ interface SearchState {
*/
readonly explored: CallPathResult["explored"];
readonly exhaustive: boolean;
readonly discovery: RelationshipDiscoveryStatus;
readonly limitations: readonly string[];
}
@@ -126,7 +131,16 @@ export const buildCallPath = (input: unknown): CallPathResult => {
evidence_links: uniqueEvidence(snapshots.values()),
limitations: [...new Set(limitations)].sort(compareUnicodeCodePoints),
};
const searchScope = { exhaustive };
const searchScope = {
exhaustive,
graph: "supplied_evidence" as const,
discovery: search.discovery,
observed_status: found
? ("found" as const)
: exhaustive
? ("not_found" as const)
: ("unknown" as const),
};
if (found)
return callPathResultSchema.parse({
...resultContext,
@@ -137,7 +151,8 @@ export const buildCallPath = (input: unknown): CallPathResult => {
});
return callPathResultSchema.parse({
...resultContext,
status: exhaustive ? "not_found" : "unknown",
status:
exhaustive && search.discovery === "complete" ? "not_found" : "unknown",
shortest_hops: null,
search_scope: searchScope,
paths: [],
@@ -259,6 +274,7 @@ const inspectSearch = ({
const queue = [start];
const limitations: string[] = [];
let incompleteExpandedCoverage = false;
let discovery: RelationshipDiscoveryStatus = "complete";
let expandedEdges = 0;
let depthReached = 0;
for (let index = 0; index < queue.length; index += 1) {
@@ -282,6 +298,16 @@ const inspectSearch = ({
// cannot add a path to the goal, so its dossier limits are contextual
// but do not make the searched start-to-goal closure incomplete.
if (node === goal) continue;
const discoveryFact = coverage?.discovery?.callees;
if (
discoveryFact !== undefined &&
discoveryFact.status !== "complete" &&
discoveryFact.reason !== undefined
)
limitations.push(
`Call discovery for ${node} is ${discoveryFact.status}: ${discoveryFact.reason}`,
);
discovery = combineRelationshipDiscovery(discovery, coverage, "callees");
if (snapshot.unresolvedCallCaveats.length > 0)
incompleteExpandedCoverage = true;
const callees = snapshot.dossier.facet_availability?.callees;
@@ -306,6 +332,7 @@ const inspectSearch = ({
continue;
}
if (snapshot === undefined) {
discovery = combineRelationshipDiscovery(discovery, null, "callees");
limitations.push(
`No analyze_function Evidence covers reachable function ${node}`,
);
@@ -331,6 +358,7 @@ const inspectSearch = ({
depth_reached: depthReached,
},
exhaustive: !incompleteExpandedCoverage,
discovery,
limitations,
};
};
+29 -3
View File
@@ -60,6 +60,11 @@ const resultContextShape = {
evidence_links: z.array(evidenceIdSchema).min(1),
limitations: z.array(z.string()),
};
const searchScope = {
graph: z.literal("supplied_evidence"),
discovery: z.enum(["complete", "partial", "unknown"]),
observed_status: z.enum(["found", "not_found", "unknown"]),
};
/** Evidence-cited directed call-path result containing every shortest path. */
export const callPathResultSchema = z.union([
@@ -67,21 +72,42 @@ export const callPathResultSchema = z.union([
...resultContextShape,
status: z.literal("found"),
shortest_hops: z.int().min(0),
search_scope: z.object({ exhaustive: z.boolean() }),
search_scope: z.object({
...searchScope,
exhaustive: z.boolean(),
observed_status: z.literal("found"),
}),
paths: z.array(pathSchema).min(1),
}),
z.object({
...resultContextShape,
status: z.literal("not_found"),
shortest_hops: z.null(),
search_scope: z.object({ exhaustive: z.literal(true) }),
search_scope: z.object({
...searchScope,
exhaustive: z.literal(true),
discovery: z.literal("complete"),
observed_status: z.literal("not_found"),
}),
paths: emptyArraySchema,
}),
z.object({
...resultContextShape,
status: z.literal("unknown"),
shortest_hops: z.null(),
search_scope: z.object({ exhaustive: z.literal(false) }),
search_scope: z.union([
z.object({
...searchScope,
exhaustive: z.literal(false),
observed_status: z.literal("unknown"),
}),
z.object({
...searchScope,
exhaustive: z.literal(true),
discovery: z.enum(["partial", "unknown"]),
observed_status: z.literal("not_found"),
}),
]),
paths: emptyArraySchema,
}),
]);
+32 -18
View File
@@ -297,24 +297,38 @@ const functionFindings = (
>,
): Finding[] => {
const { evidence, result } = comparison;
return result.dimensions.flatMap((dimension) =>
dimension.status === "unchanged"
? []
: [
makeFinding({
evidence,
dimension: dimension.dimension,
scope: "static_candidate",
status: dimension.status,
classification:
dimension.status === "unknown"
? "unresolved_branch"
: dimension.conclusion_kind,
limitations: dimension.limitations,
links: dimension.evidence_links,
}),
],
);
return result.dimensions.flatMap((dimension) => {
const finding = (status: "changed" | "unknown") =>
makeFinding({
evidence,
dimension: dimension.dimension,
scope: "static_candidate",
status,
classification:
status === "unknown"
? "unresolved_branch"
: dimension.conclusion_kind,
limitations:
status === "unknown" &&
dimension.dimension === "calls" &&
dimension.discovery_status === undefined
? [
...dimension.limitations,
"Static call discovery coverage was not recorded in the comparison.",
]
: dimension.limitations,
links: dimension.evidence_links,
});
const findings =
dimension.status === "unchanged" ? [] : [finding(dimension.status)];
if (
dimension.dimension === "calls" &&
dimension.status !== "unknown" &&
dimension.discovery_status !== "complete"
)
findings.push(finding("unknown"));
return findings;
});
};
const artifactFindings = (
+6
View File
@@ -1,4 +1,5 @@
import type { Evidence } from "./evidence.js";
import { createAnalysisProfile } from "./analysisProfile.js";
import {
PROCESS_PROVIDER,
isProcessEvidenceProvider,
@@ -18,6 +19,11 @@ export const FUNCTION_COMPARISON_PROVIDER = {
version: "1",
} as const;
export const FUNCTION_COMPARISON_PROFILE = createAnalysisProfile(
FUNCTION_COMPARISON_PROVIDER,
{ workflow_revision: 2 },
);
const COMPARISON_EVIDENCE_IDENTITIES = {
compare_artifacts: {
predicate: "rea.artifact-comparison",
+4
View File
@@ -110,6 +110,10 @@ describe("assessFunctionCallCoverage", () => {
],
unresolvedCallCaveats: ["Unresolved call at 0x1004: Computed target."],
diagnostics: ["Indirect calls are unresolved."],
discoveryStatus: "unknown",
discoveryLimitations: [
"Static direct-call discovery is not established for callers and callees.",
],
});
});
});
+38 -1
View File
@@ -1,6 +1,7 @@
import {
relationshipCoverageLimitations,
type RelationshipCoverage,
type RelationshipDiscoveryStatus,
} from "./relationshipCoverage.js";
import type { FunctionDossier } from "./native/functionDossier.js";
import { uniqueSorted } from "./canonicalOrdering.js";
@@ -20,7 +21,18 @@ export const combineFunctionRelationshipCoverage = (
const diagnostics = uniqueSorted(
reads.flatMap(({ coverage }) => coverage?.diagnostics ?? []),
);
const details = diagnostics.length > 0 ? { diagnostics } : {};
const discovery = {
...(callers?.discovery?.callers === undefined
? {}
: { callers: callers.discovery.callers }),
...(callees?.discovery?.callees === undefined
? {}
: { callees: callees.discovery.callees }),
};
const details = {
...(diagnostics.length > 0 ? { diagnostics } : {}),
...(Object.keys(discovery).length === 0 ? {} : { discovery }),
};
const callersLimited = callers?.status !== "complete";
const calleesLimited = callees?.status !== "complete";
const limited_directions: Extract<
@@ -85,6 +97,10 @@ export interface FunctionCallCoverageAssessment {
readonly unresolvedCallCaveats: readonly string[];
/** Provider caveats that remain relevant even with complete coverage. */
readonly diagnostics: readonly string[];
/** Discovery limits do not invalidate comparisons of the observed sets. */
readonly discoveryLimitations: readonly string[];
/** Whether all requested call discovery directions are established. */
readonly discoveryStatus: RelationshipDiscoveryStatus;
}
/** Keep observed-set blockers separate from descriptive provider caveats. */
@@ -97,6 +113,25 @@ export const assessFunctionCallCoverage = (
({ address, reason }) => `Unresolved call at ${address}: ${reason}`,
),
);
const missingDiscovery: string[] = [];
const discoveryLimitations: string[] = [];
let discoveryStatus: RelationshipDiscoveryStatus = "complete";
for (const direction of ["callers", "callees"] as const) {
const discovery = coverage?.discovery?.[direction];
if (discovery?.status === "partial") discoveryStatus = "partial";
else if (discovery?.status !== "complete" && discoveryStatus !== "partial")
discoveryStatus = "unknown";
if (discovery?.status === "complete") continue;
if (discovery?.reason === undefined) missingDiscovery.push(direction);
else
discoveryLimitations.push(
`Static direct-call discovery for ${direction} is ${discovery.status}: ${discovery.reason}`,
);
}
if (missingDiscovery.length > 0)
discoveryLimitations.push(
`Static direct-call discovery is not established for ${missingDiscovery.join(" and ")}.`,
);
return {
blockers: uniqueSorted([
...relationshipCoverageLimitations(coverage),
@@ -104,5 +139,7 @@ export const assessFunctionCallCoverage = (
]),
unresolvedCallCaveats,
diagnostics: uniqueSorted(coverage?.diagnostics ?? []),
discoveryLimitations,
discoveryStatus,
};
};
+16 -2
View File
@@ -117,7 +117,13 @@ const observe = (
digit: string,
value: ReturnType<typeof dossier>,
providerId = "rea-workflow",
relationshipCoverage: RelationshipCoverage | null = { status: "complete" },
relationshipCoverage: RelationshipCoverage | null = {
status: "complete",
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
},
): Evidence =>
createEvidence(
{
@@ -499,7 +505,7 @@ describe("function call relationship coverage", () => {
conclusion_kind: "unresolved_branch",
left_count: 0,
right_count: 0,
limitations: [`Right: ${limitation}`],
limitations: expect.arrayContaining([`Right: ${limitation}`]),
});
expect(result.status).toBe("unknown");
},
@@ -921,10 +927,18 @@ describe("function comparison", () => {
const left = observe("a", dossier("return 0;", "0x1000"), "rea-workflow", {
status: "complete",
diagnostics: [diagnostic],
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
});
const right = observe("b", dossier("return 0;", "0x2000"), "rea-workflow", {
status: "complete",
diagnostics: [diagnostic],
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
});
const comparison = compareFunctions(
functionSnapshotFixture(left),
+50 -26
View File
@@ -78,13 +78,18 @@ export const compareDimensions = (
];
return limitations.length === 0
? result
: unresolvedDimension({
dimension: result.dimension,
links: context.links,
leftCount: null,
rightCount: null,
limitations: [...result.limitations, ...limitations],
});
: {
...unresolvedDimension({
dimension: result.dimension,
links: context.links,
leftCount: null,
rightCount: null,
limitations: [...result.limitations, ...limitations],
}),
...(result.discovery_status === undefined
? {}
: { discovery_status: result.discovery_status }),
};
});
};
@@ -594,30 +599,49 @@ const compareCalls = (
const coverageDiagnostics = [
...left.callCoverageDiagnostics.map((reason) => `Left: ${reason}`),
...right.callCoverageDiagnostics.map((reason) => `Right: ${reason}`),
...left.callDiscoveryLimitations.map((reason) => `Left: ${reason}`),
...right.callDiscoveryLimitations.map((reason) => `Right: ${reason}`),
];
const discovery_status =
left.callDiscoveryStatus === "partial" ||
right.callDiscoveryStatus === "partial"
? "partial"
: left.callDiscoveryStatus === "complete" &&
right.callDiscoveryStatus === "complete"
? "complete"
: "unknown";
if (coverageLimitations.length > 0)
return unresolvedDimension({
dimension: "calls",
links: context.links,
leftCount: comparison.leftCount,
rightCount: comparison.rightCount,
limitations: [...coverageLimitations, ...coverageDiagnostics],
});
return {
...unresolvedDimension({
dimension: "calls",
links: context.links,
leftCount: comparison.leftCount,
rightCount: comparison.rightCount,
limitations: [...coverageLimitations, ...coverageDiagnostics],
}),
discovery_status,
};
if (comparison.status === "unknown")
return unresolvedDimension({
return {
...unresolvedDimension({
dimension: "calls",
links: context.links,
leftCount: comparison.leftCount,
rightCount: comparison.rightCount,
limitations: [...comparison.limitations, ...coverageDiagnostics],
}),
discovery_status,
};
return {
...dimensionResult({
dimension: "calls",
links: context.links,
leftCount: comparison.leftCount,
rightCount: comparison.rightCount,
...comparison,
...context,
textDelta: null,
limitations: [...comparison.limitations, ...coverageDiagnostics],
});
return dimensionResult({
dimension: "calls",
...comparison,
...context,
textDelta: null,
limitations: [...comparison.limitations, ...coverageDiagnostics],
});
}),
discovery_status,
};
};
const compareCfg = (
+6
View File
@@ -36,6 +36,12 @@ const functionDimensionContextShape = {
right_count: z.int().min(0).nullable(),
evidence_links: z.array(evidenceIdSchema).min(2),
limitations: z.array(z.string()),
discovery_status: z
.enum(["complete", "partial", "unknown"])
.describe(
"Static call discovery coverage for the compared observations; omission means unreported.",
)
.exactOptional(),
};
const functionDimensionSchema = z.union([
+4
View File
@@ -21,6 +21,8 @@ export interface FunctionSnapshot {
readonly callCoverageBlockers: readonly string[];
readonly unresolvedCallCaveats: readonly string[];
readonly callCoverageDiagnostics: readonly string[];
readonly callDiscoveryLimitations: readonly string[];
readonly callDiscoveryStatus: "complete" | "partial" | "unknown";
}
/** Artifact identity and provider-owned mapping jointly establish coordinate applicability. */
@@ -110,5 +112,7 @@ export const parseFunctionEvidence = (
callCoverageBlockers: callCoverage.blockers,
unresolvedCallCaveats: callCoverage.unresolvedCallCaveats,
callCoverageDiagnostics: callCoverage.diagnostics,
callDiscoveryLimitations: callCoverage.discoveryLimitations,
callDiscoveryStatus: callCoverage.discoveryStatus,
});
};
@@ -91,7 +91,15 @@ const source = (
confidence: "derived",
authority: "shipped-artifact",
...(hasCoverage
? { relationshipCoverage: { status: "complete" as const } }
? {
relationshipCoverage: {
status: "complete" as const,
discovery: {
callers: { status: "complete" as const },
callees: { status: "complete" as const },
},
},
}
: {}),
...observation,
},
@@ -354,10 +362,24 @@ describe("function reconstruction call coverage", () => {
it("preserves complete call diagnostics while passing equal call claims", () => {
const diagnostic = "Indirect calls remain unresolved.";
const leftWithDiagnostic = source("a", {
relationshipCoverage: { status: "complete", diagnostics: [diagnostic] },
relationshipCoverage: {
status: "complete",
diagnostics: [diagnostic],
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
},
});
const rightWithDiagnostic = source("b", {
relationshipCoverage: { status: "complete", diagnostics: [diagnostic] },
relationshipCoverage: {
status: "complete",
diagnostics: [diagnostic],
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
},
});
const observed = compareFunctions(
functionSnapshotFixture(leftWithDiagnostic),
@@ -454,7 +476,9 @@ describe("function reconstruction call coverage", () => {
},
});
});
});
describe("function reconstruction with independent changes", () => {
it("keeps a confirmed historical overall change as a failure despite unknown calls", () => {
const changedDossier = jsonValueSchema.parse({
...functionDossierSchema.parse(dossier),
+4 -1
View File
@@ -236,7 +236,10 @@ const readValidatedSourceFacts = (
);
functionSources.push({
evidence_id: source.evidence_id,
callCoverageBlockers: callCoverage.blockers,
callCoverageBlockers: [
...callCoverage.blockers,
...callCoverage.discoveryLimitations,
],
callCoverageDiagnostics: callCoverage.diagnostics,
});
} else {
+29
View File
@@ -3,9 +3,20 @@ import { z } from "zod";
import { fixedTupleSchema } from "./fixedTupleSchema.js";
const diagnostics = z.array(z.string().min(1)).min(1).exactOptional();
/** Discovery describes possible static calls beyond the returned provider graph. */
export const relationshipDiscoveryDirectionSchema = z.strictObject({
status: z.enum(["complete", "partial", "unknown"]),
reason: z.string().min(1).exactOptional(),
});
export const relationshipDiscoverySchema = z.strictObject({
callers: relationshipDiscoveryDirectionSchema.exactOptional(),
callees: relationshipDiscoveryDirectionSchema.exactOptional(),
});
const discovery = relationshipDiscoverySchema.exactOptional();
export const completeRelationshipCoverageSchema = z.strictObject({
status: z.literal("complete"),
diagnostics,
discovery,
});
const limitedDirections = z.union([
fixedTupleSchema([z.literal("callers")]),
@@ -17,12 +28,14 @@ export const partialRelationshipCoverageSchema = z.strictObject({
limitations: z.array(z.string().min(1)).min(1),
limited_directions: limitedDirections.exactOptional(),
diagnostics,
discovery,
});
export const unknownRelationshipCoverageSchema = z.strictObject({
status: z.literal("unknown"),
reason: z.string().min(1),
limited_directions: limitedDirections.exactOptional(),
diagnostics,
discovery,
});
/** Explicit coverage state for a provider's observed caller/callee set. */
@@ -34,6 +47,22 @@ export const relationshipCoverageSchema = z.discriminatedUnion("status", [
export type RelationshipCoverage = z.infer<typeof relationshipCoverageSchema>;
export type RelationshipDiscoveryStatus = z.infer<
typeof relationshipDiscoveryDirectionSchema
>["status"];
/** Combine discovery facts without treating missing historical metadata as proof. */
export const combineRelationshipDiscovery = (
current: RelationshipDiscoveryStatus,
coverage: RelationshipCoverage | null | undefined,
direction: "callers" | "callees",
): RelationshipDiscoveryStatus => {
const next = coverage?.discovery?.[direction]?.status ?? "unknown";
if (current === "partial" || next === "partial") return "partial";
if (current === "unknown" || next === "unknown") return "unknown";
return "complete";
};
/** Reasons the observed relationship inventory cannot establish completeness. */
export const relationshipCoverageLimitations = (
coverage: RelationshipCoverage | null | undefined,
+1
View File
@@ -272,6 +272,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
schema_version: 4,
bodies: false,
edge_kind: "call",
discovery: "body-classification-v1",
},
xrefs: { command: "xrefs", direction: "to", exact_address: true },
strings: {
+15 -1
View File
@@ -172,6 +172,18 @@ export class KunaReferenceQueries {
},
};
});
const calleeDiscovery =
node.kind === "data"
? {
discovery: {
callees: {
status: "partial" as const,
reason:
"Kuna classifies this graph node as data and does not analyze it as a function body.",
},
},
}
: {};
return ok({
callers: identities(graph.callers.get(address) ?? []),
callees: identities(graph.callees.get(address) ?? []),
@@ -191,8 +203,9 @@ export class KunaReferenceQueries {
limitations: [
"Kuna reports indirect calls in this function without every target; additional callees remain unknown.",
],
...calleeDiscovery,
}
: { status: "complete" },
: { status: "complete", ...calleeDiscovery },
source: this.#source(graph, node),
});
}
@@ -208,6 +221,7 @@ export class KunaReferenceQueries {
has_indirect_calls: graph.hasIndirectCalls,
function: {
address: node.address,
kind: node.kind,
has_indirect_calls: node.hasIndirectCalls,
forwards_to: node.forwardsTo,
forwarding_target_in_graph:
+4
View File
@@ -15,6 +15,7 @@ import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { Result } from "../domain/result.js";
import { toolContract } from "../contracts/toolContracts.js";
import { createEvidence } from "../domain/evidence.js";
import { createAnalysisProfile } from "../domain/analysisProfile.js";
import { recordWorkflowEvidence } from "./registerInvestigationTools/helpers.js";
import { recordDerivedEvidence } from "./recordDerivedEvidence.js";
import { runDerivedOperation } from "./runDerivedOperation.js";
@@ -159,6 +160,9 @@ const registerReconstruction = (
const evidence = createEvidence(undefined, RECONSTRUCTION_PROVIDER, {
predicateType: "rea.reconstruction-verification",
operation: contract.name,
analysisProfile: createAnalysisProfile(RECONSTRUCTION_PROVIDER, {
workflow_revision: 2,
}),
parameters: {
specification_sha256: result.specification_sha256,
claim_ids: input.specification.claims.map(
@@ -7,8 +7,35 @@ import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
import { reconstructionVerificationResultSchema } from "../../../src/domain/reconstructionVerificationSchemas.js";
import { connectLocalToolsMcp } from "../../fixtures/localToolsMcp.js";
const unchangedRight = () => {
const left = FUNCTION_COMPARISON_EXAMPLE.left;
const completeDiscovery = {
callers: { status: "complete" },
callees: { status: "complete" },
} as const;
const unchangedLeft = () => {
const fixture = FUNCTION_COMPARISON_EXAMPLE.left;
return createEvidence(
{
path: fixture.subject!.local_path,
sha256: fixture.subject!.digest.sha256,
format: fixture.subject!.format,
},
fixture.provider,
{
operation: fixture.operation,
parameters: fixture.parameters,
result: fixture.normalized_result,
confidence: fixture.confidence,
authority: fixture.authority,
relationshipCoverage: {
status: "complete",
discovery: completeDiscovery,
},
},
);
};
const unchangedRight = (left: ReturnType<typeof unchangedLeft>) => {
return createEvidence(
{ path: "/fixture/rebuilt", sha256: "2".repeat(64), format: "mach-o" },
left.provider,
@@ -18,7 +45,10 @@ const unchangedRight = () => {
result: left.normalized_result,
confidence: left.confidence,
authority: left.authority,
relationshipCoverage: { status: "complete" },
relationshipCoverage: {
status: "complete",
discovery: completeDiscovery,
},
},
);
};
@@ -50,8 +80,8 @@ it.each(["pseudocode", "overall"] as const)(
"keeps an overall claim unknown for an active %s question through stdio MCP",
async (dimension) => {
const { call } = await connectLocalToolsMcp();
const left = FUNCTION_COMPARISON_EXAMPLE.left;
const right = unchangedRight();
const left = unchangedLeft();
const right = unchangedRight(left);
const compared = await call("compare_functions", { left, right });
expect(compared.isError, JSON.stringify(compared)).not.toBe(true);
const comparison = parseEvidence(compared.structuredContent);
@@ -64,6 +64,7 @@ const childFailure = (operation: string, cleanup: boolean) =>
const relationshipPort = (
relationships: Relationships,
reads: string[],
completeDiscovery = false,
): AnalysisOperationPort => ({
execute: async (operation, parameters) => {
const procedure = String(parameters.procedure);
@@ -77,7 +78,17 @@ const relationshipPort = (
? err(answer)
: ok(
createAnalysisExecution(answer, provider, {
relationshipCoverage: { status: "complete" },
relationshipCoverage: {
status: "complete",
...(completeDiscovery
? {
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
}
: {}),
},
}),
);
},
@@ -96,6 +107,27 @@ const call = (
});
describe.each(directions)("%s relationship reads", (direction, operation) => {
it("keeps discovery unknown when cleanup stops an otherwise complete trace", async () => {
const reads: string[] = [];
const client = await connect(
relationshipPort(
{ "0x1": ["0x2"], "0x2": childFailure(operation, true) },
reads,
true,
),
);
const response = await client.callTool(call("trace_call_path", direction));
const { error } = parseMcpToolError(response);
expect(error.code).toBe("cleanup_incomplete");
expect(reads).toEqual([`${operation}:0x1`, `${operation}:0x2`]);
expect(
parseEvidence(error.details?.partial_observation).normalized_result,
).toMatchObject({
goal_status: "unknown",
search_scope: { discovery: "unknown", observed_goal_status: "unknown" },
});
});
it.each(tools)(
"%s stops at unconfirmed cleanup and reports it with the admitted result",
async (tool) => {
@@ -329,7 +329,7 @@ describe("trace_call_path MCP outcome semantics", () => {
});
});
it("reports an unreachable goal as absent after complete traversal", async () => {
it("keeps a complete observed traversal distinct from unknown discovery", async () => {
const client = await connect({
execute: async (name, arguments_) => {
if (name === "procedure_address")
@@ -347,7 +347,11 @@ describe("trace_call_path MCP outcome semantics", () => {
expect(trace.isError).not.toBe(true);
expect(trace.structuredContent).toMatchObject({
normalized_result: {
goal_status: "not_reached",
goal_status: "unknown",
search_scope: {
discovery: "unknown",
observed_goal_status: "not_reached",
},
truncated: false,
residual_unknowns: [],
},
@@ -84,6 +84,7 @@ it.each(profiles)("preserves directional gaps (%s)", async (profile) => {
status: "partial",
limitations: [limitation],
limited_directions: directions,
discovery: { callees: { status: "complete" } },
};
const observation = observe(coverage);
expect(session.recordEvidence(observation).ok).toBe(true);
@@ -1,3 +1,4 @@
import { Ajv2020 } from "ajv/dist/2020.js";
import { expect, it } from "vitest";
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
@@ -6,14 +7,14 @@ import { functionDossierSchema } from "../../../src/domain/native/functionDossie
import { jsonValueSchema } from "../../../src/domain/jsonValue.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js";
import type { RelationshipCoverage } from "../../../src/domain/relationshipCoverage.js";
import { functionComparisonResultSchema } from "../../../src/domain/functionComparisonSchemas.js";
const sourceWithCoverage = (
template: typeof FUNCTION_COMPARISON_EXAMPLE.left,
observation: string,
result: typeof template.normalized_result,
relationshipCoverage:
| { readonly status: "complete" }
| { readonly status: "unknown"; readonly reason: string },
relationshipCoverage: RelationshipCoverage,
) => {
if (template.subject === null)
throw new Error("Function comparison fixtures must be artifact-bound");
@@ -35,6 +36,316 @@ const sourceWithCoverage = (
);
};
const expectAdvertisedComparisonSchema = async (
client: Awaited<ReturnType<typeof createApplicationMcpHarness>>["client"],
payload: unknown,
) => {
const advertised = (await client.listTools()).tools.find(
({ name }) => name === "compare_functions",
);
if (advertised?.outputSchema === undefined)
throw new Error("compare_functions did not advertise its output schema");
const outputSchema: Record<string, unknown> = advertised.outputSchema;
const validator = new Ajv2020({ strict: false, validateFormats: false });
expect(validator.validateSchema(outputSchema)).toBe(true);
const validate = validator.compile(outputSchema);
expect(validate(payload), JSON.stringify(validate.errors)).toBe(true);
};
const expectDiscoveryConsumers = async (
client: Awaited<ReturnType<typeof createApplicationMcpHarness>>["client"],
comparison: ReturnType<typeof parseEvidence>,
label: "partial" | "unreported" | "complete",
) => {
const unknowns = await client.callTool({
name: "list_unknowns",
arguments: { domain: "function-comparison" },
});
expect(unknowns.isError).not.toBe(true);
const unknownItems = (
unknowns.structuredContent as { result: { items: unknown[] } }
).result.items;
if (label === "complete") expect(unknownItems).toHaveLength(0);
else
expect(unknownItems).toEqual(
expect.arrayContaining([
expect.objectContaining({
comparison_conclusion_scope: expect.objectContaining({
comparison_evidence_id: comparison.evidence_id,
dimensions: expect.arrayContaining(["calls"]),
}),
}),
]),
);
const changed = await client.callTool({
name: "find_changed_behavior",
arguments: { comparisons: [comparison] },
});
expect(changed.isError, JSON.stringify(changed.structuredContent)).not.toBe(
true,
);
const changedEvidence = parseEvidence(changed.structuredContent);
const findings = (
changedEvidence.normalized_result as { findings: { items: unknown[] } }
).findings.items;
if (label === "complete") expect(findings).toHaveLength(0);
else
expect(findings).toEqual(
expect.arrayContaining([
expect.objectContaining({
dimension: "calls",
classification: "unresolved_branch",
}),
]),
);
};
const expectHistoricalDiscovery = async (
session: ReturnType<typeof createTestBinarySession>,
client: Awaited<ReturnType<typeof createApplicationMcpHarness>>["client"],
comparison: ReturnType<typeof parseEvidence>,
) => {
const current = functionComparisonResultSchema.parse(
comparison.normalized_result,
);
const historical = createEvidence(undefined, comparison.provider, {
predicateType: comparison.predicate_type,
operation: comparison.operation,
parameters: comparison.parameters,
result: {
...current,
dimensions: current.dimensions.map((dimension) => {
if (dimension.dimension !== "calls") return dimension;
const { discovery_status: _discovery, ...historicalDimension } =
dimension;
return { ...historicalDimension, limitations: [] };
}),
},
confidence: comparison.confidence,
authority: comparison.authority,
limitations: comparison.limitations,
evidenceLinks: comparison.evidence_links,
});
expect(session.recordEvidence(historical).ok).toBe(true);
const prior = await verify(
(name, arguments_) => client.callTool({ name, arguments: arguments_ }),
historical,
["calls"],
);
expect(prior.normalized_result).toMatchObject({
claims: {
items: [{ status: "unknown", observed_status: "unchanged" }],
},
});
const historicalChanged = await client.callTool({
name: "find_changed_behavior",
arguments: { comparisons: [historical] },
});
expect(historicalChanged.isError).not.toBe(true);
expect(
parseEvidence(historicalChanged.structuredContent).normalized_result,
).toMatchObject({
findings: {
items: expect.arrayContaining([
expect.objectContaining({
dimension: "calls",
classification: "unresolved_branch",
}),
]),
},
});
expect(session.evidenceById(historical.evidence_id)).toEqual(historical);
};
it.each(["complete", "partial"] as const)(
"retains %s discovery when a call facet is unavailable",
async (discoveryStatus) => {
const session = createTestBinarySession(() => {
throw new Error("offline comparison must not launch a provider");
});
const { client } = await createApplicationMcpHarness({ session });
const original = functionDossierSchema.parse(
FUNCTION_COMPARISON_EXAMPLE.left.normalized_result,
);
const unavailable = {
...original,
facet_availability: {
...original.facet_availability,
callers: { available: false as const, reason: "Read unavailable." },
},
};
const coverage: RelationshipCoverage = {
status: "complete",
discovery: {
callers: { status: "complete" },
callees:
discoveryStatus === "complete"
? { status: "complete" }
: { status: "partial", reason: "Some bodies unclassified." },
},
};
const left = sourceWithCoverage(
FUNCTION_COMPARISON_EXAMPLE.left,
"facet-left",
unavailable,
coverage,
);
const right = sourceWithCoverage(
FUNCTION_COMPARISON_EXAMPLE.right,
"facet-right",
original,
coverage,
);
const compared = await client.callTool({
name: "compare_functions",
arguments: { left, right },
});
expect(compared.isError).not.toBe(true);
const comparison = parseEvidence(compared.structuredContent);
expect(comparison.normalized_result).toMatchObject({
dimensions: expect.arrayContaining([
expect.objectContaining({
dimension: "calls",
status: "unknown",
discovery_status: discoveryStatus,
limitations: expect.arrayContaining([
"Left: callers: Read unavailable.",
]),
}),
]),
});
const changed = await client.callTool({
name: "find_changed_behavior",
arguments: { comparisons: [comparison] },
});
expect(changed.isError).not.toBe(true);
const findings = (
parseEvidence(changed.structuredContent).normalized_result as {
findings: {
items: Array<{ dimension: string; limitations: string[] }>;
};
}
).findings.items;
const calls = findings.find((finding) => finding.dimension === "calls");
expect(calls).toBeDefined();
expect(calls?.limitations).not.toContain(
"Static call discovery coverage was not recorded in the comparison.",
);
},
);
it.each(["full", "compact"] as const)(
"qualifies observed calls and verifies discovery independently (%s)",
async (profile) => {
for (const [label, coverage, status] of [
[
"partial",
{
status: "complete",
discovery: {
callers: { status: "complete" },
callees: {
status: "partial",
reason: "The fixture body is classified as data.",
},
},
},
"unknown",
],
["unreported", { status: "complete" }, "unknown"],
[
"complete",
{
status: "complete",
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
},
"pass",
],
] as const) {
const session = createTestBinarySession(() => {
throw new Error("offline comparison must not launch a provider");
});
const { client } = await createApplicationMcpHarness({
session,
serverOptions: {
environment: { REA_MCP_OUTPUT_SCHEMA_PROFILE: profile },
},
});
const left = sourceWithCoverage(
FUNCTION_COMPARISON_EXAMPLE.left,
`${label}-left`,
FUNCTION_COMPARISON_EXAMPLE.left.normalized_result,
coverage,
);
const right = sourceWithCoverage(
FUNCTION_COMPARISON_EXAMPLE.right,
`${label}-right`,
FUNCTION_COMPARISON_EXAMPLE.left.normalized_result,
coverage,
);
const compared = await client.callTool({
name: "compare_functions",
arguments: { left, right },
});
if (compared.isError) throw new Error(JSON.stringify(compared));
const comparison = parseEvidence(compared.structuredContent);
if (label === "partial")
await expectAdvertisedComparisonSchema(
client,
compared.structuredContent,
);
expect(comparison.normalized_result).toMatchObject({
dimensions: expect.arrayContaining([
expect.objectContaining({
dimension: "calls",
status: "unchanged",
...(label === "partial"
? {
limitations: expect.arrayContaining([
expect.stringContaining(
"The fixture body is classified as data.",
),
]),
}
: {}),
}),
]),
});
await expectDiscoveryConsumers(client, comparison, label);
const result = await verify(
(name, arguments_) => client.callTool({ name, arguments: arguments_ }),
comparison,
["calls"],
);
expect(result.normalized_result).toMatchObject({
claims: {
items: [
{
claim_id: "calls",
status,
observed_status: "unchanged",
...(label === "partial"
? {
limitations: expect.arrayContaining([
expect.stringContaining(
"The fixture body is classified as data.",
),
]),
}
: {}),
},
],
},
});
if (label === "unreported")
await expectHistoricalDiscovery(session, client, comparison);
}
},
);
const verify = async (
call: (
name: string,
@@ -64,6 +375,89 @@ const verify = async (
return parseEvidence(response.structuredContent);
};
it("keeps an observed call-set difference without claiming a verified difference", async () => {
const session = createTestBinarySession(() => {
throw new Error("offline comparison must not launch a provider");
});
const { client } = await createApplicationMcpHarness({ session });
const original = functionDossierSchema.parse(
FUNCTION_COMPARISON_EXAMPLE.left.normalized_result,
);
const coverage: RelationshipCoverage = {
status: "complete",
discovery: {
callers: { status: "complete" },
callees: { status: "partial", reason: "Not all bodies were classified." },
},
};
const left = sourceWithCoverage(
FUNCTION_COMPARISON_EXAMPLE.left,
"difference-left",
jsonValueSchema.parse({ ...original, callees: [] }),
coverage,
);
const {
signature: _signature,
locals: _locals,
...helper
} = original.procedure;
const right = sourceWithCoverage(
FUNCTION_COMPARISON_EXAMPLE.right,
"difference-right",
jsonValueSchema.parse({
...original,
callees: [{ ...helper, address: "0x2000", name: "helper" }],
}),
coverage,
);
const response = await client.callTool({
name: "compare_functions",
arguments: { left, right },
});
if (response.isError) throw new Error(JSON.stringify(response));
const comparison = parseEvidence(response.structuredContent);
expect(comparison.normalized_result).toMatchObject({
dimensions: expect.arrayContaining([
expect.objectContaining({
dimension: "calls",
status: "changed",
limitations: expect.arrayContaining([
expect.stringContaining("Not all bodies were classified."),
]),
}),
]),
});
const changed = await client.callTool({
name: "find_changed_behavior",
arguments: { comparisons: [comparison] },
});
expect(changed.isError).not.toBe(true);
expect(
parseEvidence(changed.structuredContent).normalized_result,
).toMatchObject({
findings: {
items: expect.arrayContaining([
expect.objectContaining({
dimension: "calls",
classification: "derived_relationship",
}),
expect.objectContaining({
dimension: "calls",
classification: "unresolved_branch",
}),
]),
},
});
const result = await verify(
(name, arguments_) => client.callTool({ name, arguments: arguments_ }),
comparison,
["calls"],
);
expect(result.normalized_result).toMatchObject({
claims: { items: [{ status: "unknown", observed_status: "changed" }] },
});
});
const assertChangedDespiteUnknownCalls = async (
call: Parameters<typeof verify>[0],
left: ReturnType<typeof parseEvidence>,
@@ -168,7 +562,11 @@ it("keeps incomplete calls unknown through public comparison and verification wh
conclusion_kind: "unresolved_branch",
left_count: 0,
right_count: 0,
limitations: ["Right: Call graph unavailable."],
limitations: expect.arrayContaining([
"Right: Call graph unavailable.",
"Left: Static direct-call discovery is not established for callers and callees.",
"Right: Static direct-call discovery is not established for callers and callees.",
]),
evidence_links: [left.evidence_id, right.evidence_id],
}),
]),
@@ -106,7 +106,7 @@ it.each([
);
it.each([
{ comparisonStatus: "unchanged", verificationStatus: "pass" },
{ comparisonStatus: "unchanged", verificationStatus: "unknown" },
{ comparisonStatus: "changed", verificationStatus: "fail" },
] as const)(
"verifies public $comparisonStatus function comparison as $verificationStatus",
@@ -202,15 +202,15 @@ it.each([
status: verificationStatus,
summary: {
total: 4,
passed: verificationStatus === "pass" ? 4 : 2,
passed: comparisonStatus === "unchanged" ? 3 : 2,
failed: verificationStatus === "fail" ? 2 : 0,
unknown: 0,
unknown: comparisonStatus === "unchanged" ? 1 : 0,
},
claims: {
items: expect.arrayContaining([
expect.objectContaining({
dimension: "pseudocode",
status: verificationStatus,
status: comparisonStatus === "unchanged" ? "pass" : "fail",
observed_status: comparisonStatus,
comparison_evidence_id: comparison.evidence_id,
left_evidence_ids: [left.evidence_id],
@@ -447,7 +447,7 @@ describe("function comparison input-kind diagnostics", () => {
it.each([
{ status: "changed", expected: "fail" },
{ status: "unknown", expected: "unknown" },
{ status: "unchanged", expected: "pass" },
{ status: "unchanged", expected: "unknown" },
] as const)(
"checks the aggregate status of imported $status function comparisons",
async ({ status, expected }) => {
@@ -170,7 +170,13 @@ const evidence = (
},
confidence: "derived",
authority: "shipped-artifact",
relationshipCoverage: { status: "complete" },
relationshipCoverage: {
status: "complete",
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
},
},
);
@@ -273,9 +273,36 @@ it.each([
coverage: { status: "complete" } as RelationshipCoverage,
providerLimitations: ["The provider version identity is unavailable."],
dossierLimitations: ["The dossier has no native value-flow observation."],
status: "unknown",
exhaustive: true,
},
{
label: "complete discovery",
coverage: {
status: "complete",
discovery: { callees: { status: "complete" } },
} as RelationshipCoverage,
providerLimitations: [],
dossierLimitations: [],
status: "not_found",
exhaustive: true,
},
{
label: "partially discovered code",
coverage: {
status: "complete",
discovery: {
callees: {
status: "partial",
reason: "This node was classified as data.",
},
},
} as RelationshipCoverage,
providerLimitations: [],
dossierLimitations: [],
status: "unknown",
exhaustive: true,
},
{
label: "partial",
coverage: {
@@ -313,7 +340,7 @@ it.each([
coverage: undefined,
providerLimitations: [],
dossierLimitations: [],
status: "not_found",
status: "unknown",
exhaustive: true,
},
])(
@@ -408,6 +435,11 @@ it.each([
...scenario.providerLimitations,
...scenario.dossierLimitations,
...reasons,
...(coverage?.discovery?.callees?.reason === undefined
? []
: [
`Call discovery for 0x1000 is ${coverage.discovery.callees.status}: ${coverage.discovery.callees.reason}`,
]),
]),
});
expect(evidence.evidence_links).toEqual([dossier.evidence_id]);
@@ -428,7 +460,7 @@ it.each([
});
expect(rejected.isError, JSON.stringify(rejected)).toBe(true);
}
if (scenario.exhaustive)
if (scenario.status !== "unknown")
expect(session.listUnknowns({ domain: "call-path" })).toEqual([]);
else
expect(session.listUnknowns({ domain: "call-path" })).toMatchObject([
@@ -624,7 +656,27 @@ const close = async (
it("keeps an aggregate function reconstruction unknown while a constituent dimension is unresolved", async () => {
const { session, server, client } = await connected();
try {
const left = FUNCTION_COMPARISON_EXAMPLE.left;
const fixture = FUNCTION_COMPARISON_EXAMPLE.left;
const discovery = {
callers: { status: "complete" },
callees: { status: "complete" },
} as const;
const left = createEvidence(
{
path: fixture.subject!.local_path,
sha256: fixture.subject!.digest.sha256,
format: fixture.subject!.format,
},
fixture.provider,
{
operation: fixture.operation,
parameters: fixture.parameters,
result: fixture.normalized_result,
confidence: fixture.confidence,
authority: fixture.authority,
relationshipCoverage: { status: "complete", discovery },
},
);
const right = createEvidence(
{
path: "/tmp/rebuilt-function",
@@ -638,7 +690,7 @@ it("keeps an aggregate function reconstruction unknown while a constituent dimen
result: left.normalized_result,
confidence: left.confidence,
authority: left.authority,
relationshipCoverage: { status: "complete" },
relationshipCoverage: { status: "complete", discovery },
},
);
const compared = await client.callTool({
@@ -175,6 +175,12 @@ it("follows native overflow continuations through MCP without repeating provider
relationshipCoverage: {
status: "partial",
limitations: ["Indirect edges unknown"],
discovery: {
callees: {
status: "partial",
reason: "Body was classified as data.",
},
},
},
},
),
@@ -234,6 +240,24 @@ it("follows native overflow continuations through MCP without repeating provider
kind: "native-observation",
parent_operation: name,
});
if (name === "get_call_graph")
expect(evidence.normalized_result).toMatchObject({
items: {
type: "graph-nodes",
rows: [
{
relationship_coverage: {
discovery: {
callees: {
status: "partial",
reason: "Body was classified as data.",
},
},
},
},
],
},
});
expect(reads).toBe(before);
}
} finally {
+3 -2
View File
@@ -71,6 +71,7 @@ export interface FakeKunaBehavior {
| "with-bodies"
| "noisy";
readonly graphIndirectCalls?: boolean;
readonly graphDataFunction?: string;
/** Bytes of an extra root member in the graph JSON. */
readonly graphPadding?: number;
readonly xrefs?: "ok" | "hang" | "not-json";
@@ -145,11 +146,11 @@ if (args[0] === "--version") {
else {
if (behavior.graph === "noisy") process.stderr.write(Buffer.alloc(9 * 1024 * 1024, 0x78));
if (args[args.indexOf("--functions") + 1] !== "") throw new Error("graph must request no bodies");
const nodes = [...functions, { name: "puts", addr: "0x2000" }].map((fn) => ({ address: parseInt(fn.addr, 16), name: fn.name, kind: "normal", hasIndirectCalls: behavior.graphIndirectCalls !== false && fn === functions[0], forwardsTo: null, codeC: behavior.graph === "with-bodies" ? "unrequested code" : null, assembly: null, error: null }));
const nodes = [...functions, { name: "puts", addr: "0x2000" }].map((fn) => ({ address: parseInt(fn.addr, 16), name: fn.name, kind: fn.name === behavior.graphDataFunction ? "data" : "normal", hasIndirectCalls: behavior.graphIndirectCalls !== false && fn === functions[0], forwardsTo: null, codeC: behavior.graph === "with-bodies" ? "unrequested code" : null, assembly: null, error: null }));
const edges = [];
if (functions[1]) edges.push({ callerAddress: nodes[0].address, calleeAddress: nodes[1].address, kind: "call", calleeOrder: 0 });
if (functions[2]) edges.push({ callerAddress: nodes[0].address, calleeAddress: nodes[2].address, kind: functions[2].error ? "jump" : "call", calleeOrder: 1 });
if (functions[1]?.name === "parse") edges.push({ callerAddress: nodes[1].address, calleeAddress: 8192, kind: "call", calleeOrder: 0 });
if (functions[1]?.name === "parse" && nodes[1].kind !== "data") edges.push({ callerAddress: nodes[1].address, calleeAddress: 8192, kind: "call", calleeOrder: 0 });
setTimeout(() => process.stdout.write(JSON.stringify({ schemaVersion: 4, binary: { sourcePath: args[1], functionCount: nodes.length + (behavior.graph === "miscounted" ? 1 : 0), edgeCount: edges.length }, functions: nodes, edges, padding: "x".repeat(behavior.graphPadding ?? 0) })), behavior.referenceDelayMs ?? 0);
}
} else if (args[0] === "xrefs") {
@@ -7,6 +7,63 @@ import { FUNCTIONS, run, session } from "./kunaSessionFixture.js";
const it = test.skipIf(process.platform === "win32");
it("preserves observed edges while qualifying undiscovered calls in both directions", async () => {
const { open } = await session("rea-kuna-data-call-path-", {
graphIndirectCalls: false,
graphDataFunction: "parse",
});
const client = open();
try {
const tools = new EnhancedTools(client);
const forward = await tools.execute("trace_call_path", {
start: "main",
goal: "0x2000",
direction: "forward",
});
if (!forward.ok) throw forward.error;
expect(forward.value).toMatchObject({
goal_status: "unknown",
search_scope: {
discovery: "partial",
observed_goal_status: "not_reached",
},
relationship_coverage: {
"0x1040": {
status: "complete",
discovery: { callees: { status: "partial" } },
},
},
truncated: false,
});
const backward = await tools.execute("trace_call_path", {
start: "0x2000",
goal: "main",
direction: "backward",
});
if (!backward.ok) throw backward.error;
expect(backward.value).toMatchObject({
goal_status: "unknown",
search_scope: {
discovery: "unknown",
observed_goal_status: "not_reached",
},
});
const observed = await tools.execute("trace_call_path", {
start: "main",
goal: "parse",
direction: "forward",
});
if (!observed.ok) throw observed.error;
expect(observed.value).toMatchObject({
goal_status: "reached",
traversal_path: ["0x1000", "0x1040"],
search_scope: { discovery: "unknown", observed_goal_status: "reached" },
});
} finally {
expect(await client.close()).toEqual({ ok: true, value: null });
}
});
it("resolves graph-only imports as graph roots and path endpoints without exporting C", async () => {
const { open, kuna } = await session("rea-kuna-graph-only-import-", {
inventoryImports: false,
@@ -66,7 +66,15 @@ it("matches a simple-path oracle for every three-node directed graph in both dir
createAnalysisExecution(
[...related, ...related],
{ id: "oracle", name: "Oracle", version: null },
{ relationshipCoverage: { status: "complete" } },
{
relationshipCoverage: {
status: "complete",
discovery: {
callers: { status: "complete" },
callees: { status: "complete" },
},
},
},
),
);
},