Compare commits

...
Author SHA1 Message Date
morluto 2de78b704e chore: merge formatting fix from main 2026-11-07 23:15:33 +08:00
morluto ac30fbd658 style(test): format CLI help default annotations (#9280) 2026-11-07 23:02:44 +08:00
Nianze Wuandmorluto 5d21bddb18 fix(desktop): read JSON with deeply nested overwritten values (#9277)
* fix(desktop): read JSON with deeply nested overwritten values

* fix(desktop): bound exported JSON construction by view depth

* fix(desktop): preserve deep JSON numeric facts within bounded memory

---------

Co-authored-by: morluto <[email protected]>
2026-11-07 22:57:15 +08:00
morlutoandCursor Agent 2e2f981ccf test: assert absolute-path facts without frozen sentences (#9231)
* test: assert absolute-path facts without frozen sentences

Sibling path tests now require the field path, invalid_format, and regex
expectation, and they record that each negative case ran. NUL rejection
follows the argument that contains the byte instead of searching for the
word NUL.

* test: retain actionable NUL path diagnostics

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 22:46:17 +08:00
morluto f410fbb1c1 chore: preserve unrelated main imports 2026-11-07 22:42:16 +08:00
morluto 516d1a9f62 fix(mcp): reconcile native value flow status with main 2026-11-07 22:41:38 +08:00
Heo Hao f8bb792f15 fix(cli): stop restating schema defaults in help descriptions (#9278)
The help renderer appends "(default: X)" whenever the option field declares a
schema default, so descriptions that also spell the default out print it twice:

  --block-size <value>  Power-of-two block size from 256 to 65536 bytes
                        (default 4096) (default: 4096)

Remove the hand-written default from the eleven options that own a schema
default across inspect-byte-entropy, inspect-devicetree,
inspect-packet-capture, inspect-packet-fields, inspect-signal-decoders and
decode-signal-recording, matching the rule in cliDecimalField's contract that
the domain field owns bounds, defaults and help.

Options without a schema default keep their prose: --match_limit, --line_limit,
--limit and --packet_limit are nullable or optional, so the renderer emits no
annotation and the description is the only place the effective value appears.

Add a boundary test asserting each such option states its default exactly once,
so a restated default cannot come back unnoticed.
2026-11-07 22:38:03 +08:00
morlutoandCursor Agent 0ecf3a4cbf docs(mcp): document native function analysis scope (#8389)
Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 22:14:55 +08:00
ShuaixinHuang 8842092671 fix(pe): reject misaligned certificate table offsets (#9276)
* fix(pe): reject misaligned certificate table offsets

* test: align ghidra limitation wording with main

* style: format ghidra limitation assertion
2026-11-07 22:14:29 +08:00
morluto 7333602b85 test(mcp): share schema validation and assert status facts 2026-11-03 20:43:04 +08:00
morluto e870ea43ac fix(mcp): report native value flow support in session status 2026-11-03 20:32:03 +08:00
morluto 465e8b1e06 fix(native)!: preserve value-flow absence semantics 2026-11-03 20:02:14 +08:00
32 changed files with 1159 additions and 234 deletions

No files matched your search

+2 -2
View File
@@ -310,8 +310,8 @@ entry it does not read. The module reads the central directory itself, as REA
does, and shows `zipfile` only the records it lists, reading each entry in
bounded chunks no further than a byte past its declared size. A JSON file
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
or nested more than 500 levels deep, gets an `error` instead of a view, as in
REA, while text that is not one JSON
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
document, such as JSON Lines, stays text. Format detection follows REA's
rules. Files larger than 64 MiB get an `error` instead of a view
and are not read. XML is read with Python's `expat` twice: with namespace
+13
View File
@@ -972,6 +972,18 @@ finish without reading relationships, even when neither traversal mode is
available. Discovery does not block that direct invocation or replace its typed
partial/error outcomes.
`trace_native_values` includes `value_flow_support` for the provider routed to
`analyze_function`. `supported` declares that the provider may supply value flow;
it does not guarantee a model for the selected function. `unavailable` includes
the provider's reason, and `unknown` means no declaration. The provider ID is
preserved, or null if no analysis provider is routed. This declaration is separate
from operation readiness and the availability of call-target resolution. When
call-target resolution is missing, a direct trace may still return a local graph
with call-resolution unknowns; even `max_depth: 0` attempts to resolve call sites.
When support is unknown but the declared operation prerequisites are ready,
availability can still be true; it does not promise that value flow is present.
The trace's returned observations and unknowns remain authoritative for that call.
`trace_native_ui_action` reports authored-UI and native-seed modes separately.
Apple dispatch metadata requires a Mach-O target; another provider may supply
the symbol-only alternative on other formats. If neither mode is available,
@@ -2001,6 +2013,7 @@ bundle import/export; it does not include those view and settings tools.
| Web investigation | `browser,pcap,session,javascript-runtime` | page, bundle and session inspection, packet captures, JS runtime observation, evidence workflow |
| Android application analysis | `android,adb,apktool,session` | class and method inspection, device bridge, resource decoding; add `frida` for dynamic instrumentation, `jeb` for JEB projects, `flutter` for Flutter builds |
| Native macOS or iOS binary | `native,artifact,session` | Mach-O, signature and property-list inspection plus container extraction |
| Native analysis of a known function | `session,enhanced,application` | open a binary, analyze a known function, page retained pseudocode, and export Evidence; add `direct` for focused procedure search or assembly |
| Managed .NET assembly | `managed,session` | managed member comparison and reconstruction against the evidence workflow |
| Electron or browser-hosted JavaScript application | `browser,electron,application,javascript-runtime,session` | runtime targets, live page inspection, bundle and capture tools, feature tracing, JavaScript semantics and reconstruction obligations |
| Evidence bookkeeping only | `session` | open and close binaries, bundles, comparisons, unknown tracking |
+4
View File
@@ -317,6 +317,10 @@ export type CapabilityDescriptor = CapabilityAvailability & {
readonly operation: Exclude<AnalysisOperation, "health">;
readonly effects: CapabilityEffects;
readonly limitations: readonly string[];
/** Whether analyze_function can supply value flow for some procedures. */
readonly nativeValueFlow?:
| { readonly status: "supported" }
| { readonly status: "unavailable"; readonly reason: string };
};
/** Factory and capability declaration for an analysis implementation. */
+52 -9
View File
@@ -24,7 +24,6 @@ import {
toolAvailabilitySummaryInventory,
TOOL_ANNOTATION_NAMES,
TOOL_EFFECT_NAMES,
type ProviderCapability,
type ToolAvailabilityInventory,
type ToolAvailabilitySummaryInventory,
type ToolAvailability,
@@ -71,7 +70,7 @@ const statusSchema = z.object({
type ToolAvailabilityReason = "available" | ToolUnavailabilityReason;
type ProviderDescriptor = ProviderCapability;
type ProviderDescriptor = ReturnType<typeof expandProviderCapabilities>[number];
type UnavailableDescriptor = Extract<ProviderDescriptor, { available: false }>;
/** Optional integrations whose verified hosts exclude some platforms. */
@@ -310,6 +309,22 @@ export const buildCapabilityInventory = (
const facts = {
name: contract.name,
surface: contract.kind,
...(contract.name === "trace_native_values"
? {
value_flow_support: (() => {
const source = descriptors.get("analyze_function");
return source?.native_value_flow === undefined
? {
status: "unknown" as const,
provider_id: source?.provider.id ?? null,
}
: {
...source.native_value_flow,
provider_id: source.provider.id,
};
})(),
}
: {}),
...(availability.defaultModeAvailable === undefined
? {}
: { default_mode_available: availability.defaultModeAvailable }),
@@ -377,6 +392,9 @@ export function compactToolAvailability(
available: entry.available,
reason: entry.reason,
remediation: entry.remediation,
...(entry.value_flow_support === undefined
? {}
: { value_flow_support: entry.value_flow_support }),
...(entry.default_mode_available === undefined
? {}
: { default_mode_available: entry.default_mode_available }),
@@ -933,13 +951,38 @@ const composedAvailability = (
reason: "provider_missing",
remediation: "No provider composition is declared for this operation.",
};
return "modes" in requirements
? availabilityForModes(
requirements.modes,
requirements.defaultMode,
descriptors,
)
: composedAvailabilityFor(requirements, descriptors).availability;
const result =
"modes" in requirements
? availabilityForModes(
requirements.modes,
requirements.defaultMode,
descriptors,
)
: composedAvailabilityFor(requirements, descriptors).availability;
if (name !== "trace_native_values") return result;
const source = descriptors.get("analyze_function");
if (source?.available !== true) return result;
if (source.native_value_flow?.status === "unavailable")
return {
reason:
result.reason === "provider_unavailable" ||
result.reason === "unsupported_host"
? result.reason
: "provider_missing",
remediation: `${source.native_value_flow.reason} Select a provider that can report native value-flow observations; cross-function tracing also needs call-target resolution.${(result.reason === "provider_unavailable" || result.reason === "unsupported_host") && result.remediation !== null ? ` ${result.remediation}` : ""}`,
};
if (
result.reason === "provider_missing" &&
descriptors.get("resolve_native_call_targets") === undefined
)
return {
...result,
remediation:
source.native_value_flow?.status === "supported"
? "A direct trace may still return local value-flow observations with call-resolution unknowns. Configure a provider with resolve_native_call_targets for call resolution and cross-function tracing."
: "Value-flow support is undeclared. A direct trace may return partial results or an unknown. Configure a provider with resolve_native_call_targets for call resolution and cross-function tracing.",
};
return result;
};
/** Evaluate source groups once so discovery and missing-operation facts agree. */
@@ -10,6 +10,7 @@ import {
analysisProviderCandidate,
analysisProviderCandidateSummary,
expandProviderCapabilities,
expandToolAvailability,
sessionProviderFacts,
type ProviderCapabilitySet,
} from "../../contracts/toolOutputSchemaPrimitives.js";
@@ -30,6 +31,7 @@ import {
import { NATIVE_MACOS_PROVIDER_IDENTITY } from "../../native/NativeMacosProviderIdentity.js";
import {
buildCapabilityInventory,
compactToolAvailability,
MULTI_MODE_REMEDIATION,
} from "../CapabilityInventory.js";
@@ -74,7 +76,9 @@ type DescriptorOptions = {
| {
readonly available: false;
readonly reason?: string;
readonly availabilityCode?: "architecture_unsupported";
readonly availabilityCode?:
| "architecture_unsupported"
| "unsupported_host";
}
);
@@ -850,3 +854,91 @@ it("groups readiness repairs exactly and drops one replaced by a runtime observa
}),
]);
});
it.each([
{ detail: "summary", supported: true },
{ detail: "complete", supported: true },
{ detail: "summary", supported: false },
{ detail: "complete", supported: false },
] as const)(
"retains routed value-flow ownership and runtime failures in $detail discovery when supported=$supported",
async ({ detail, supported }) => {
const source = {
...descriptor(providerA, "analyze_function", []),
nativeValueFlow: supported
? { status: "supported" as const }
: { status: "unavailable" as const, reason: "No value flow." },
};
const resolver = descriptor(providerB, "resolve_native_call_targets", []);
const providerC = { id: "provider-c", name: "Provider C", version: "3" };
const unselected = {
...descriptor(providerC, "analyze_function", []),
nativeValueFlow: { status: "unavailable" as const, reason: "Unselected" },
};
const status = await projectStatus(
[source, resolver],
[{ ...candidate(providerC, [unselected]), selected: false }],
{
target: dispatchTarget("elf"),
detail,
runtimeUnavailability: new Map([
["resolve_native_call_targets", { reason: "Resolver failed." }],
]),
},
);
const projected = z
.object({ capabilities: sessionProviderFacts.shape.capabilities })
.parse(status);
const route = expandProviderCapabilities(projected.capabilities);
expect(
route.find((item) => item.operation === "analyze_function"),
).toMatchObject({
provider: providerA,
native_value_flow: { status: supported ? "supported" : "unavailable" },
});
const inventory = buildCapabilityInventory(status, {
processCaptureEnabled: false,
});
const item = expandToolAvailability(
compactToolAvailability(inventory, detail),
).find((candidate) => candidate.name === "trace_native_values");
expect(item).toMatchObject({
available: false,
reason: "provider_unavailable",
remediation: expect.stringContaining("Resolver failed."),
value_flow_support: {
status: supported ? "supported" : "unavailable",
provider_id: providerA.id,
},
});
if (!supported) expect(item?.remediation).toContain("No value flow.");
},
);
it("preserves an unsupported resolver host alongside unavailable value flow", async () => {
const source = {
...descriptor(providerA, "analyze_function", []),
nativeValueFlow: {
status: "unavailable" as const,
reason: "No value flow.",
},
};
const resolver = {
...descriptor(providerB, "resolve_native_call_targets", [], {
available: false,
reason: "Resolver host is unsupported.",
availabilityCode: "unsupported_host",
}),
remediation: "Use a supported resolver host.",
};
const status = await projectStatus([source, resolver], [], {
target: dispatchTarget("elf"),
detail: "summary",
});
const item = buildCapabilityInventory(status, {
processCaptureEnabled: false,
}).find((entry) => entry.name === "trace_native_values");
expect(item).toMatchObject({ available: false, reason: "unsupported_host" });
expect(item?.remediation).toContain("No value flow.");
expect(item?.remediation).toContain("Use a supported resolver host.");
});
@@ -417,6 +417,7 @@ export const providerCapabilitySets = (
const key = JSON.stringify([
effects,
limitations,
descriptor.nativeValueFlow ?? null,
status.available,
status.reason,
status.availability_code,
@@ -429,6 +430,9 @@ export const providerCapabilitySets = (
operations: [descriptor.operation],
effects,
limitations,
...(descriptor.nativeValueFlow === undefined
? {}
: { native_value_flow: descriptor.nativeValueFlow }),
...status,
});
else group.operations.push(descriptor.operation);
+6 -8
View File
@@ -266,12 +266,10 @@ export const registerBinaryDiagnosticsCommands = (
// "0x10" or "1e2".
blockSize: cliDecimalField(
inspectByteEntropyInputSchema.shape.block_size,
).describe(
"Power-of-two block size from 256 to 65536 bytes (default 4096)",
),
).describe("Power-of-two block size from 256 to 65536 bytes"),
offset: cliDecimalField(
inspectByteEntropyInputSchema.shape.offset,
).describe("Window start as a decimal file byte offset (default 0)"),
).describe("Window start as a decimal file byte offset"),
length: cliDecimalField(
inspectByteEntropyInputSchema.shape.length,
).describe("Window length in bytes; omit to extend through EOF"),
@@ -429,16 +427,16 @@ export const registerBinaryDiagnosticsCommands = (
options: z.object({
maxFileBytes: cliDecimalField(
inspectDevicetreeInputSchema.shape.max_file_bytes,
).describe("Maximum FDT file size in bytes (default 4194304)"),
).describe("Maximum FDT file size in bytes"),
maxNodes: cliDecimalField(
inspectDevicetreeInputSchema.shape.max_nodes,
).describe("Maximum number of device-tree nodes (default 8192)"),
).describe("Maximum number of device-tree nodes"),
maxProperties: cliDecimalField(
inspectDevicetreeInputSchema.shape.max_properties,
).describe("Maximum number of device-tree properties (default 32768)"),
).describe("Maximum number of device-tree properties"),
maxDepth: cliDecimalField(
inspectDevicetreeInputSchema.shape.max_depth,
).describe("Maximum device-tree nesting depth (default 64)"),
).describe("Maximum device-tree nesting depth"),
...analysisEvidenceDeliveryOptions,
}),
run: ({ args, options }) =>
+1 -1
View File
@@ -123,7 +123,7 @@ describe("CLI JSON path normalization", () => {
expect.objectContaining({
path: ["executable_path"],
code: "invalid_format",
message: expect.stringContaining("absolute local filesystem path"),
format: "regex",
}),
]);
});
@@ -53,6 +53,29 @@ export const requireOutputSchema = <
/** The readiness owner's repair instruction; never the failure description. */
const providerRemediation = z.string().min(1);
const nativeValueFlowDeclaration = z.discriminatedUnion("status", [
z.strictObject({ status: z.literal("supported") }),
z.strictObject({
status: z.literal("unavailable"),
reason: z.string().min(1),
}),
]);
const nativeValueFlowSupport = z.discriminatedUnion("status", [
z.strictObject({
status: z.literal("supported"),
provider_id: z.string().min(1),
}),
z.strictObject({
status: z.literal("unavailable"),
provider_id: z.string().min(1),
reason: z.string().min(1),
}),
z.strictObject({
status: z.literal("unknown"),
provider_id: z.string().min(1).nullable(),
}),
]);
const providerCapabilityFacts = {
operation: z.string(),
effects: z.object({
@@ -65,6 +88,7 @@ const providerCapabilityFacts = {
requires_root: z.boolean(),
}),
limitations: z.array(z.string()),
native_value_flow: nativeValueFlowDeclaration.exactOptional(),
};
export const providerCapability = z.discriminatedUnion("available", [
@@ -100,6 +124,7 @@ const providerCapabilityGroupFacts = {
operations: z.array(z.string().min(1)).min(1),
effects: z.array(providerCapabilityFacts.effects.keyof()),
limitations: z.array(z.string()),
native_value_flow: nativeValueFlowDeclaration.exactOptional(),
};
const providerCapabilityGroup = z.discriminatedUnion("available", [
@@ -138,6 +163,9 @@ export const expandProviderCapabilities = (
group.operations.map((operation) => ({
...providerCapability.parse({
operation,
...(group.native_value_flow === undefined
? {}
: { native_value_flow: group.native_value_flow }),
available: group.available,
reason: group.reason,
availability_code: group.availability_code,
@@ -375,6 +403,11 @@ const toolAvailabilityFacts = {
surface: z.string(),
default_mode_available: z.boolean().optional(),
modes: z.array(toolAvailabilityMode).optional(),
value_flow_support: nativeValueFlowSupport
.describe(
"The named provider's declared ability to supply value flow for some functions, separate from runtime readiness. Supported does not guarantee a model for a particular function; unknown means no declaration.",
)
.exactOptional(),
client_requirements: z.object({
required: z.array(clientFeatureNameSchema),
optional: z.array(clientFeatureNameSchema),
@@ -440,6 +473,7 @@ const toolAvailabilityStatusGroupFacts = {
names: z.array(z.string().min(1)).min(1),
default_mode_available: toolAvailabilityFacts.default_mode_available,
modes: toolAvailabilityFacts.modes,
value_flow_support: toolAvailabilityFacts.value_flow_support,
client_requirements: toolAvailabilityFacts.client_requirements.optional(),
};
@@ -449,6 +483,7 @@ const toolAvailabilityGroupFacts = {
default_mode_available:
toolAvailabilityStatusGroupFacts.default_mode_available,
modes: toolAvailabilityStatusGroupFacts.modes,
value_flow_support: toolAvailabilityStatusGroupFacts.value_flow_support,
client_requirements: toolAvailabilityStatusGroupFacts.client_requirements,
effects: z.array(toolAvailabilityFacts.effects.keyof()),
annotations: z.array(toolAvailabilityFacts.annotations.keyof()),
+1 -1
View File
@@ -51,7 +51,7 @@ const captureInputShape = z.strictObject({
.int()
.nonnegative()
.default(0)
.describe("Skip this many matching packets after filtering; default 0"),
.describe("Skip this many matching packets after filtering"),
match_limit: z
.number()
.int()
+2 -4
View File
@@ -5,9 +5,7 @@ export const inspectPacketFieldsInputSchema = z.strictObject({
kind: z
.enum(["protocols", "fields"])
.default("protocols")
.describe(
"Return protocol registrations or field definitions; defaults to protocols",
),
.describe("Return protocol registrations or field definitions"),
protocol: z
.string()
.min(1)
@@ -30,7 +28,7 @@ export const inspectPacketFieldsInputSchema = z.strictObject({
.nonnegative()
.max(Number.MAX_SAFE_INTEGER)
.default(0)
.describe("Number of matching records to skip; defaults to 0"),
.describe("Number of matching records to skip"),
limit: z
.number()
.int()
+3 -142
View File
@@ -1,3 +1,4 @@
import { PYTHON_JSON_READER } from "./appExportPythonJson.js";
import { PYTHON_XML_READER } from "./appExportPythonXml.js";
import { PYTHON_ZIP_HELPERS } from "./appExportPythonZip.js";
@@ -12,6 +13,7 @@ export const PYTHON_READY_CHECK = `
export const PYTHON_IMPORTS = `from __future__ import annotations
from array import array
import contextlib
import copy
import hashlib
@@ -57,11 +59,6 @@ MAX_PARSED_BYTES = 4 * 1024 * 1024
PARSED_TOO_LARGE = "larger than the 4 MiB limit for JSON and key file views"
MAX_JSON_VIEW_DEPTH = 500
JSON_TOO_DEEP = f"nested deeper than the {MAX_JSON_VIEW_DEPTH} levels a JSON view keeps"
#: A JSON token after any whitespace, by group: an opening, a closing, a comma, a colon, a string's quote, or a number
#: or literal. Strings are read by json's scanner, since a pattern repeating per escape holds memory for each.
JSON_TOKEN = re.compile(r'[ \t\n\r]*(?:([\[{])|([\]}])|(,)|(:)|(")|(-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?'
r"|true|false|null))")
JSON_SPACE = re.compile(r"[ \t\n\r]*")
def _respelled(text: str, home: Path) -> str:
@@ -254,143 +251,7 @@ def _keyfile(text: str) -> dict:
${PYTHON_XML_READER}
def _not_json(constant: str):
raise ValueError(f"{constant} is not JSON")
class _Inexact(Exception):
"""A JSON number that REA's state view refuses, since reading it would change it."""
def _digits(text: str) -> str:
return re.split(r"[eE]", text.lstrip("-"))[0].replace(".", "").strip("0") or "0"
def _js_string(value: float) -> str:
"""How JavaScript's String() writes a finite double: an integer below 1e21 in full, any other in shortest form."""
return str(int(value)) if value.is_integer() and abs(value) < 1e21 else repr(value)
def _safe_number(text: str) -> bool:
"""lossless-json's isSafeNumber: an integer spelling within 2^53, or a float spelling that keeps its digits."""
if re.fullmatch(r"-?[0-9]+", text):
return len(text.lstrip("-")) <= 16 and abs(int(text)) <= MAX_SAFE_INTEGER
value = float(text)
if not math.isfinite(value):
return False
written = _js_string(value)
return written == text or _digits(written) == _digits(text)
def _exact_number(text: str):
"""A JSON number read only when reading it changes nothing, as REA's state view rules: its spelling and its
written-back form both safe, and not -0."""
value = float(text)
if (not _safe_number(text) or not math.isfinite(value) or not _safe_number(_js_string(value))
or (value == 0 and math.copysign(1, value) < 0)):
raise _Inexact(text)
return int(text) if re.fullmatch(r"-?[0-9]+", text) else value
_INEXACT = object()
_LEVEL_END = object()
def _number_or_mark(spelling: str):
"""A JSON number read exactly, or a mark for one REA's state view refuses."""
try:
return _exact_number(spelling)
except _Inexact:
return _INEXACT
def _holds_inexact(value) -> bool:
"""Whether a parsed JSON value still holds a marked number once duplicate keys kept their last value; one
iterator per nesting level, so no child list is copied."""
levels = [iter((value,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif item is _INEXACT:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _deeper_than(root, limit: int) -> bool:
"""Whether a parsed JSON value nests deeper than a limit, as REA checks it: a value's depth is the number of
iterators open above it, one per nesting level, so no child list is copied."""
levels = [iter((root,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif len(levels) - 1 > limit:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _is_json(text: str) -> bool:
"""Whether text is one JSON value as REA's state view reads it, checked a token at a time without building it or
recursing: it holds a byte per open container and one string at a time, so text too deep for Python's parser or
too large to parse into a view is still told apart from other text."""
containers, expect, empty, at = bytearray(), "value", False, 0
while True:
token = JSON_TOKEN.match(text, at)
if token is None:
break
kind, at = token.lastindex, token.end()
if kind == 5:
try:
at = json.decoder.scanstring(text, at)[1]
except ValueError:
return False
if kind == 1 and expect == "value":
containers.append(ord(text[at - 1]))
expect, empty = ("value" if text[at - 1] == "[" else "key"), True
# A closing bracket's code is its opening's plus two.
elif kind == 2 and (expect == "after" or empty) and containers and containers[-1] == ord(text[at - 1]) - 2:
containers.pop()
expect, empty = "after", False
elif kind == 3 and expect == "after" and containers:
expect = "value" if containers[-1] == ord("[") else "key"
elif kind == 4 and expect == "colon":
expect = "value"
elif kind >= 5 and (expect == "value" or (kind == 5 and expect == "key")):
expect, empty = ("colon" if expect == "key" else "after"), False
else:
return False
return expect == "after" and not containers and JSON_SPACE.fullmatch(text, at) is not None
def _json_view(text: str, size: int):
"""A JSON text's format and view, or why it has none, as REA reads it; None when it is not JSON. Text over the
parsed-view limit, or nested past Python's recursion limit, is told apart without building its value."""
if size > MAX_PARSED_BYTES:
return {"format": "json", "view": None, "error": PARSED_TOO_LARGE} if _is_json(text) else None
# The whole text must parse, and duplicate keys settle, before an inexact number makes it JSON with an error.
try:
view = json.loads(text, parse_constant=_not_json, parse_float=_number_or_mark, parse_int=_number_or_mark)
except RecursionError:
# Nesting too deep for Python to parse is far deeper than REA's view keeps, once the rest of it is JSON too.
return {"format": "json", "view": None, "error": JSON_TOO_DEEP} if _is_json(text) else None
except ValueError:
return None
if _holds_inexact(view):
return {"format": "json", "view": None,
"error": "a number in it cannot be read exactly, such as an integer above 2^53"}
if _deeper_than(view, MAX_JSON_VIEW_DEPTH):
return {"format": "json", "view": None, "error": JSON_TOO_DEEP}
return {"format": "json", "view": view, "truncated": False}
${PYTHON_JSON_READER}
def _text_view(name: str, text: str, size: int) -> dict:
"""The format REA's state view detects for decoded text of size bytes, and its view; a JSON or key file over the
+231
View File
@@ -0,0 +1,231 @@
/** JSON parsing and bounded retained-view checks for exported desktop environments. */
export const PYTHON_JSON_READER = String.raw`
#: A JSON token after any whitespace, by group: an opening, a closing, a comma, a colon, a string's quote, or a number
#: or literal. Strings are read by json's scanner, since a pattern repeating per escape holds memory for each.
JSON_TOKEN = re.compile(r'[ \t\n\r]*(?:([\[{])|([\]}])|(,)|(:)|(")|(-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?'
r"|true|false|null))")
JSON_SPACE = re.compile(r"[ \t\n\r]*")
class _Inexact(Exception):
"""A JSON number that REA's state view refuses, since reading it would change it."""
def _digits(text: str) -> str:
return re.split(r"[eE]", text.lstrip("-"))[0].replace(".", "").strip("0") or "0"
def _js_string(value: float) -> str:
"""How JavaScript's String() writes a finite double: an integer below 1e21 in full, any other in shortest form."""
return str(int(value)) if value.is_integer() and abs(value) < 1e21 else repr(value)
def _safe_number(text: str) -> bool:
"""lossless-json's isSafeNumber: an integer spelling within 2^53, or a float spelling that keeps its digits."""
if re.fullmatch(r"-?[0-9]+", text):
return len(text.lstrip("-")) <= 16 and abs(int(text)) <= MAX_SAFE_INTEGER
value = float(text)
if not math.isfinite(value):
return False
written = _js_string(value)
return written == text or _digits(written) == _digits(text)
def _exact_number(text: str):
"""A JSON number read only when reading it changes nothing, as REA's state view rules: its spelling and its
written-back form both safe, and not -0."""
value = float(text)
if (not _safe_number(text) or not math.isfinite(value) or not _safe_number(_js_string(value))
or (value == 0 and math.copysign(1, value) < 0)):
raise _Inexact(text)
return int(text) if re.fullmatch(r"-?[0-9]+", text) else value
_INEXACT = object()
_DEEP = object()
_DEEP_INEXACT = object()
_LEVEL_END = object()
_INVALID_JSON = object()
JSON_ATOMS = {"true": True, "false": False, "null": None}
def _number_or_mark(spelling: str):
"""A JSON number read exactly, or a mark for one REA's state view refuses."""
try:
return _exact_number(spelling)
except _Inexact:
return _INEXACT
def _holds_inexact(value) -> bool:
"""Whether a parsed JSON value still holds a marked number once duplicate keys kept their last value; one
iterator per nesting level, so no child list is copied."""
levels = [iter((value,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif item is _INEXACT or item is _DEEP_INEXACT:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _deeper_than(root, limit: int) -> bool:
"""Whether a parsed JSON value nests deeper than a limit, as REA checks it: a value's depth is the number of
iterators open above it, one per nesting level, so no child list is copied."""
levels = [iter((root,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif item is _DEEP or item is _DEEP_INEXACT:
return True
elif len(levels) - 1 > limit:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _read_json(text: str, build: bool = True):
"""One JSON value, or _INVALID_JSON, read without recursion so duplicate keys settle before depth is checked.
Deep containers are summarized rather than built; exact-number marks still follow last-key-wins semantics.
With build=False, only syntax is checked: one byte per open container and one decoded string at a time."""
containers, expect, empty, at = bytearray(), "value", False, 0
values, keys, root = [], [], _INVALID_JSON
deep_bad, deep_key_at, bad_keys = bytearray(), array("I"), {}
def keep(value) -> None:
nonlocal root
if not values:
root = value
elif containers[-1] == ord("["):
values[-1].append(value)
else:
values[-1][keys[-1]] = value
def record_deep(bad: bool) -> None:
"""Summarize a finished child; keep only object keys with live inexact values."""
depth = len(deep_bad) - 1
if containers[-1] == ord("["):
deep_bad[-1] |= 2 | bad
return
marked = bad_keys.get(depth)
if bad or marked is not None:
key, _ = json.decoder.scanstring(text, deep_key_at[-1])
if bad:
if marked is None:
marked = key
elif isinstance(marked, str):
if marked != key:
marked = {marked, key}
else:
marked.add(key)
else:
if isinstance(marked, str):
if marked == key:
marked = None
else:
marked.discard(key)
if len(marked) == 1:
marked = next(iter(marked))
if marked is None:
bad_keys.pop(depth, None)
else:
bad_keys[depth] = marked
deep_bad[-1] = 2 | (marked is not None)
else:
deep_bad[-1] |= 2
while True:
token = JSON_TOKEN.match(text, at)
if token is None:
break
kind, at = token.lastindex, token.end()
if kind == 5:
string_at = at
try:
value, at = json.decoder.scanstring(text, at)
except ValueError:
return _INVALID_JSON
if kind == 1 and expect == "value":
if build:
if deep_bad or len(containers) >= MAX_JSON_VIEW_DEPTH:
deep_bad.append(0)
deep_key_at.append(0)
else:
value = [] if text[at - 1] == "[" else {}
keep(value)
values.append(value)
keys.append(None)
containers.append(ord(text[at - 1]))
expect, empty = ("value" if text[at - 1] == "[" else "key"), True
# A closing bracket's code is its opening's plus two.
elif kind == 2 and (expect == "after" or empty) and containers and containers[-1] == ord(text[at - 1]) - 2:
opening = containers.pop()
if build:
if deep_bad:
state = deep_bad.pop()
bad = bool(state & 1)
deep_key_at.pop()
bad_keys.pop(len(deep_bad), None)
if deep_bad:
record_deep(bad)
else:
keep((_DEEP_INEXACT if bad else _DEEP) if state & 2 else ([] if opening == ord("[") else {}))
else:
values.pop()
keys.pop()
expect, empty = "after", False
elif kind == 3 and expect == "after" and containers:
expect = "value" if containers[-1] == ord("[") else "key"
elif kind == 4 and expect == "colon":
expect = "value"
elif kind >= 5 and (expect == "value" or (kind == 5 and expect == "key")):
if build:
if expect == "key":
if deep_bad:
deep_key_at[-1] = string_at
else:
keys[-1] = value
else:
if kind == 6:
value = JSON_ATOMS.get(token[6], _INVALID_JSON)
if value is _INVALID_JSON:
value = _number_or_mark(token[6])
if deep_bad:
record_deep(value is _INEXACT)
else:
keep(value)
expect, empty = ("colon" if expect == "key" else "after"), False
else:
return _INVALID_JSON
if expect != "after" or containers or JSON_SPACE.fullmatch(text, at) is None:
return _INVALID_JSON
return root if build else True
def _json_view(text: str, size: int):
"""A JSON text's format and view, or why it has none, as REA reads it; None when it is not JSON. Text over the
parsed-view limit is told apart without building its value."""
if size > MAX_PARSED_BYTES:
valid = _read_json(text, build=False) is not _INVALID_JSON
return {"format": "json", "view": None, "error": PARSED_TOO_LARGE} if valid else None
# The whole text must parse, and duplicate keys settle, before an inexact number makes it JSON with an error.
view = _read_json(text)
if view is _INVALID_JSON:
return None
if _holds_inexact(view):
return {"format": "json", "view": None,
"error": "a number in it cannot be read exactly, such as an integer above 2^53"}
if _deeper_than(view, MAX_JSON_VIEW_DEPTH):
return {"format": "json", "view": None, "error": JSON_TOO_DEEP}
return {"format": "json", "view": view, "truncated": False}
`;
+20 -1
View File
@@ -21,11 +21,20 @@ const PATHS = [
];
it("matches the host path module for absolute local paths", () => {
const checked: string[] = [];
for (const path of PATHS) {
checked.push(path);
expect(absoluteLocalPathStringSchema.safeParse(path).success).toBe(
isAbsolute(path),
);
}
expect(checked).toEqual(
expect.arrayContaining([
"/absolute/path",
"relative/path",
"file:///absolute/path",
]),
);
expect(absoluteLocalPathStringSchema.safeParse("").success).toBe(false);
expect(
absoluteLocalPathStringSchema.safeParse("/absolute/with\0nul").success,
@@ -40,8 +49,18 @@ it("keeps the Windows absolute-path pattern valid in every client mode", () => {
() => new RegExp(windowsAbsolutePathPattern.source, flags),
flags,
).not.toThrow();
for (const path of PATHS)
const checked: string[] = [];
for (const path of PATHS) {
checked.push(path);
expect(windowsAbsolutePathPattern.test(path), path).toBe(
win32.isAbsolute(path),
);
}
expect(checked).toEqual(
expect.arrayContaining([
"C:\\absolute\\path",
"C:relative/path",
"/absolute/path",
]),
);
});
+1 -3
View File
@@ -24,9 +24,7 @@ export const inspectSignalDecodersInputSchema = z.strictObject({
.nonnegative()
.max(Number.MAX_SAFE_INTEGER)
.default(0)
.describe(
"Number of matching decoders or help lines to skip; defaults to 0",
),
.describe("Number of matching decoders or help lines to skip"),
limit: z
.number()
.int()
+1 -1
View File
@@ -31,7 +31,7 @@ const signalInputShape = z.strictObject({
.int()
.nonnegative()
.default(0)
.describe("Skip this many literal LF-separated output lines; default 0"),
.describe("Skip this many literal LF-separated output lines"),
line_limit: z
.number()
.int()
+3
View File
@@ -234,6 +234,9 @@ export const CAPABILITIES: readonly CapabilityDescriptor[] = Object.freeze(
return Object.freeze({
provider: GHIDRA_PROVIDER_IDENTITY,
operation,
...(operation === "analyze_function"
? { nativeValueFlow: { status: "supported" as const } }
: {}),
available: true,
reason: null,
// Project writes are private import machinery. Analysis reads are bound
+11
View File
@@ -29,6 +29,9 @@ export const KUNA_STRING_LIMITATION =
export const KUNA_XREF_LIMITATION =
"Kuna's static reference walk unifies forwarding aliases and omits references internal to that alias class. REA keeps only exact-address incoming edges; an empty result does not prove no reference exists.";
export const KUNA_VALUE_FLOW_UNAVAILABLE =
"REA's Kuna provider does not report native def-use observations.";
/** Whether an operation is one Kuna answers. */
export const isKunaOperation = (
operation: AnalysisOperation,
@@ -104,6 +107,14 @@ export const kunaCapabilities = (
KUNA_OPERATIONS.map((operation) => ({
provider: KUNA_PROVIDER_IDENTITY,
operation,
...(operation === "analyze_function"
? {
nativeValueFlow: {
status: "unavailable" as const,
reason: KUNA_VALUE_FLOW_UNAVAILABLE,
},
}
: {}),
...(availability.status === "available"
? ({ available: true, reason: null } as const)
: ({
+2 -5
View File
@@ -86,6 +86,7 @@ import {
isKunaOperation,
KUNA_RELATIONSHIP_LIMITATION,
KUNA_SINGLE_FUNCTION_LIMITATION,
KUNA_VALUE_FLOW_UNAVAILABLE,
kunaLimitations,
type KunaOperation,
} from "./KunaProviderCapabilities.js";
@@ -332,10 +333,6 @@ const kunaVerificationLimitations = (
/** Kuna names a procedure by its inventory entry or by an exact entry address. */
const procedureSelectorSchema = z.string().min(1);
/** REA's Kuna integration does not supply a def-use model for any procedure. */
export const KUNA_NATIVE_VALUE_FLOW_UNAVAILABLE =
"REA's Kuna provider does not report native def-use observations.";
/**
* One Kuna analysis session for one target.
*
@@ -1574,7 +1571,7 @@ export class KunaSessionClient implements AnalysisClient {
native_api: null,
native_value_flow: {
available: false,
reason: KUNA_NATIVE_VALUE_FLOW_UNAVAILABLE,
reason: KUNA_VALUE_FLOW_UNAVAILABLE,
limitations: [],
},
facet_availability: {
+15
View File
@@ -105,6 +105,21 @@ it("reports unsigned attribute OIDs and incomplete nested coverage", () => {
}
});
it.each([false, true])(
"rejects a misaligned certificate table (PE32+ = %s)",
(plus) => {
const aligned = peCertificateFixture(plus, undefined, undefined, true);
expect(parse(aligned).coverage.status).toBe("complete");
const shifted = Buffer.concat([
aligned.subarray(0, 0x200),
Buffer.alloc(1),
aligned.subarray(0x200),
]);
shifted.writeUInt32LE(0x201, 0x98 + (plus ? 112 : 96) + 4 * 8);
expect(() => parse(shifted)).toThrow(/certificate table offset.*aligned/u);
},
);
it("accepts only zero eight-byte alignment padding included in dwLength", () => {
for (const plus of [false, true]) {
const bytes = peCertificateFixture(plus, undefined, undefined, true);
+2
View File
@@ -79,6 +79,8 @@ export const parsePeCertificates = (
const directoryAt = optionalHeaderOffset + security.offset;
if (tableSize === 0 || tableOffset < headerSize)
fail("PE certificate table offset overlaps headers or has zero size.");
if (tableOffset % 8 !== 0)
fail("PE certificate table offset must be aligned to eight bytes.");
requireRange(bytes, tableOffset, tableSize);
const table = bytes.subarray(tableOffset, tableOffset + tableSize);
const entries: PeCertificates["entries"] = [];
@@ -255,11 +255,11 @@ cliTest(
code: "invalid_request",
details: {
issues: [
{
expect.objectContaining({
path: ["executable_path"],
reason: "invalid_format",
message: expect.stringContaining("absolute local filesystem path"),
},
expected: "regex",
}),
],
},
});
@@ -4,6 +4,7 @@ import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { z } from "zod";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
@@ -329,21 +330,84 @@ const pathRequests: readonly [string, Record<string, unknown>][] = [
["start_gdb_session", { inferior_tty: nul }],
];
const nulIssuePaths = (
value: unknown,
path: readonly (string | number)[] = [],
): (string | number)[][] => {
if (typeof value === "string") return value.includes("\0") ? [[...path]] : [];
if (Array.isArray(value))
return value.flatMap((item, index) =>
nulIssuePaths(item, [...path, index]),
);
if (typeof value === "object" && value !== null)
return Object.entries(value).flatMap(([key, item]) =>
nulIssuePaths(item, [...path, key]),
);
return [];
};
it("rejects OS-invalid local inputs through the real advertised MCP contracts", async () => {
const { call } = await connectLocalToolsMcp();
const checked: string[] = [];
for (const [name, arguments_] of pathRequests) {
const response = await call(name, arguments_);
expect(response.isError, name).toBe(true);
expect(JSON.stringify(response.content), name).toContain("NUL");
const paths = nulIssuePaths(arguments_);
expect(paths.length, name).toBeGreaterThan(0);
const error = parseMcpToolError(await call(name, arguments_)).error;
expect(error, name).toMatchObject({
code: "invalid_request",
details: { operation: name },
});
const issues = z
.object({
issues: z.array(
z.object({
path: z.array(z.union([z.string(), z.number()])),
reason: z.string(),
message: z.string(),
expected: z.unknown().optional(),
}),
),
})
.parse(error.details).issues;
expect(issues, name).toEqual(
expect.arrayContaining(
paths.map((path) =>
expect.objectContaining({
path,
reason: "invalid_format",
expected: "regex",
message: expect.stringMatching(/\bNUL\b|null byte|zero byte/i),
}),
),
),
);
checked.push(name);
}
expect(checked).toEqual(
expect.arrayContaining([
"extract_firmware",
"inspect_wasm_artifact",
"demangle_swift",
"capture_electron_scenario",
]),
);
const relative = await call("extract_firmware", {
path: "/tmp/firmware.bin",
output_directory: "relative-output",
});
expect(relative.isError).toBe(true);
expect(JSON.stringify(relative.content)).toContain(
"absolute local filesystem path",
);
expect(parseMcpToolError(relative).error).toMatchObject({
code: "invalid_request",
details: {
operation: "extract_firmware",
issues: [
expect.objectContaining({
path: ["output_directory"],
reason: "invalid_format",
expected: "regex",
}),
],
},
});
}, 60_000);
const ignoredOptions = [
@@ -0,0 +1,87 @@
import { describe, expect, it } from "vitest";
import { createCli } from "../../../src/cli.js";
type CliInstance = ReturnType<typeof createCli>;
const serve = async (
cli: CliInstance,
argv: readonly string[],
): Promise<{ readonly stdout: string; readonly exitCode: number }> => {
let stdout = "";
let exitCode = 0;
await cli.serve([...argv], {
env: {},
exit: (code) => {
exitCode = code;
},
stdout: (text) => {
stdout += text;
},
});
return { stdout, exitCode };
};
/**
* Options whose owning field already declares a schema default. The help
* renderer appends that default, so the description must not restate it.
*/
const singleDefaultOptions: readonly {
readonly command: string;
readonly options: readonly string[];
}[] = [
{ command: "inspect-byte-entropy", options: ["--block-size", "--offset"] },
{
command: "inspect-devicetree",
options: [
"--max-file-bytes",
"--max-nodes",
"--max-properties",
"--max-depth",
],
},
{
command: "decode-signal-recording",
options: ["--line_offset", "--line_limit"],
},
{
command: "inspect-packet-capture",
options: ["--match_offset", "--match_limit"],
},
{
command: "inspect-packet-fields",
options: ["--kind", "--offset"],
},
{ command: "inspect-signal-decoders", options: ["--offset"] },
];
const optionLine = (help: string, option: string): string | undefined => {
const line = help
.split(/\r?\n/u)
.find((candidate) => candidate.trimStart().startsWith(`${option} `));
return line?.trim();
};
describe("CLI help default annotations", () => {
it("states each schema-owned option default exactly once", async () => {
const cli = createCli({});
const repeated: string[] = [];
for (const { command, options } of singleDefaultOptions) {
const { stdout, exitCode } = await serve(cli, [command, "--help"]);
expect(exitCode, command).toBe(0);
for (const option of options) {
const line = optionLine(stdout, option);
if (line === undefined) {
repeated.push(`${command} ${option}: option line missing`);
continue;
}
const statements = line.match(/default/giu)?.length ?? 0;
if (statements !== 1)
repeated.push(
`${command} ${option}: ${statements} default statements`,
);
}
}
expect(repeated).toEqual([]);
});
});
@@ -13,7 +13,7 @@ interface PathInputCase {
readonly field: string;
readonly valid: Readonly<Record<string, unknown>>;
readonly invalid: Readonly<Record<string, unknown>>;
readonly issuePath?: readonly string[];
readonly issuePath: readonly string[];
}
const createMcp = async (
@@ -104,7 +104,13 @@ const inputCases = (
field: string,
valid: Readonly<Record<string, unknown>>,
invalid: Readonly<Record<string, unknown>>,
): PathInputCase => ({ name, field, valid, invalid });
): PathInputCase => ({
name,
field,
valid,
invalid,
issuePath: field.split("."),
});
return [
pathCase("open_binary", "path", { path: absolute }, { path: relative }),
@@ -246,8 +252,7 @@ it("advertises full host-absolute path constraints after SDK schema conversion",
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const absolute = resolve("rea-absolute-input");
const relative = "relative-input";
for (const { name, field, valid, invalid } of [
const schemaCases = [
...inputCases(absolute, relative),
{
name: "inspect_go_binary",
@@ -276,7 +281,9 @@ it("advertises full host-absolute path constraints after SDK schema conversion",
right_bundle_path: absolute,
},
},
]) {
];
const checked: string[] = [];
for (const { name, field, valid, invalid } of schemaCases) {
const schema = listed.tools.find(
(tool) => tool.name === name,
)?.inputSchema;
@@ -288,7 +295,16 @@ it("advertises full host-absolute path constraints after SDK schema conversion",
validate(valid),
`${name}.${field}: ${JSON.stringify(validate.errors)}`,
).toBe(true);
checked.push(`${name}.${field}`);
}
expect(checked).toEqual(
expect.arrayContaining([
"open_binary.path",
"extract_firmware.output_directory",
"inspect_go_binary.path NUL",
"compare_bundles.left_bundle_path empty",
]),
);
expect(firmwareCalls).toEqual([]);
} finally {
await close();
@@ -299,19 +315,39 @@ it("rejects relative paths through in-memory MCP handlers", async () => {
const firmwareCalls: string[] = [];
const { client, close } = await createMcp(firmwareCalls);
try {
for (const { name, field, invalid, issuePath } of inputCases(
resolve("rea-absolute-input"),
"relative-input",
)) {
const cases = inputCases(resolve("rea-absolute-input"), "relative-input");
const checked: string[] = [];
for (const { name, field, invalid, issuePath } of cases) {
const result = await client.callTool({ name, arguments: invalid });
expect(result.isError, `${name}.${field}`).toBe(true);
expect(JSON.stringify(result), `${name}.${field}`).toContain("absolute");
if (issuePath !== undefined)
expect(parseMcpToolError(result).error.details).toMatchObject({
operation: name,
issues: [{ path: issuePath, reason: "invalid_format" }],
});
expect(parseMcpToolError(result).error, `${name}.${field}`).toMatchObject(
{
code: "invalid_request",
details: {
operation: name,
issues: [
expect.objectContaining({
path: issuePath,
reason: "invalid_format",
expected: "regex",
}),
],
},
},
);
checked.push(`${name}.${field}`);
}
expect(checked).toEqual(
expect.arrayContaining([
"open_binary.path",
"open_binary.snapshot_path",
"extract_firmware.path",
"extract_firmware.output_directory",
"inspect_analysis_view.source.path",
"capture_browser_scenario.browser.executable_path",
"capture_electron_scenario.application_root",
]),
);
expect(firmwareCalls).toEqual([]);
} finally {
await close();
+1 -3
View File
@@ -703,9 +703,7 @@ describe("Ghidra MCP evidence parity", () => {
},
limitations: expect.arrayContaining([
expect.stringContaining("default auto-analysis complete"),
expect.stringContaining(
"original executable bytes are never written",
),
expect.stringContaining("original executable bytes are unchanged"),
]),
});
const directAnalyzed = await new EnhancedTools(session).execute(
@@ -0,0 +1,176 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterAll, expect, it } from "vitest";
import type { AnalysisProvider } from "../../../src/application/AnalysisProvider.js";
import {
valueTraceDiamond,
valueTraceProvider,
} from "../../../src/application/native/NativeValueTrace.fixture.js";
import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { functionDossierSchema } from "../../../src/domain/native/functionDossier.js";
import { ok } from "../../../src/domain/result.js";
import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { advertisedMcpToolSchemas } from "../../fixtures/advertisedMcpTool.js";
import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js";
const schemaValidation = createMcpSchemaValidation();
afterAll(() => schemaValidation.clear());
it.each([
{ profile: "compact", state: "supported" },
{ profile: "full", state: "supported" },
{ profile: "compact", state: "unknown" },
{ profile: "full", state: "unknown" },
{ profile: "compact", state: "unavailable" },
{ profile: "full", state: "unavailable" },
] as const)(
"reports $state value flow without a resolver in $profile MCP",
async ({ profile, state }) => {
const target = join(
await createTestTempDirectory("flow-capability-"),
"fixture.com",
);
await writeFile(target, Buffer.from([0xb8, 0x00, 0x4c, 0xcd, 0x21]));
const graph = valueTraceDiamond({ sessionSubject: true });
const provider: AnalysisProvider = {
identity: () => valueTraceProvider,
capabilities: () => [
{
provider: valueTraceProvider,
operation: "analyze_function",
available: true,
reason: null,
...(state === "unknown"
? {}
: state === "supported"
? { nativeValueFlow: { status: "supported" as const } }
: {
nativeValueFlow: {
status: "unavailable" as const,
reason: "Fixture has no value flow.",
},
}),
effects: {
mutatesArtifact: false,
launchesProcess: false,
mayShowUi: false,
mayAccessNetwork: false,
mayWriteFilesystem: false,
changesPermissions: false,
requiresRoot: false,
},
limitations: [],
},
],
resolveAnalysisProfile: async () =>
ok({
profile: createAnalysisProfile(valueTraceProvider, { fixture: true }),
}),
createClient: () => ({
execute: async (operation, parameters, options) => {
const execution = await graph.port.execute(
operation,
parameters,
options,
);
if (
!execution.ok ||
state !== "unavailable" ||
operation !== "analyze_function"
)
return execution;
const dossier = functionDossierSchema.parse(execution.value.result);
return ok({
...execution.value,
result: {
...dossier,
native_value_flow: {
available: false,
reason: "Fixture has no value flow.",
limitations: [],
},
},
});
},
close: async () => ok(null),
}),
};
const session = createTestBinarySession(provider);
const { client } = await createApplicationMcpHarness({
session,
schemaValidator: schemaValidation,
serverOptions: {
environment: {
...process.env,
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
REA_MCP_OUTPUT_SCHEMA_PROFILE: profile,
},
},
});
const call = async (name: string, args: Record<string, unknown>) => {
const schema = await advertisedMcpToolSchemas(
client,
name,
schemaValidation.assertions,
);
const reply = await client.callTool(
{ name, arguments: args },
{ toolDefinition: schema.toolDefinition },
);
expect(reply.isError, JSON.stringify(reply)).not.toBe(true);
expect(
schema.ajv.validate(schema.outputSchema, reply.structuredContent),
JSON.stringify(schema.ajv.errors),
).toBe(true);
return reply.structuredContent;
};
await call("open_binary", { path: target, format: "dos-com" });
const status = (await call("binary_session", { detail: "summary" })) as {
result: {
tool_availability: {
groups: Array<{
names: string[];
available: boolean;
value_flow_support?: { status: string; provider_id: string };
remediation: string | null;
}>;
};
};
};
const traceStatus = status.result.tool_availability.groups.find((g) =>
g.names.includes("trace_native_values"),
);
expect(traceStatus).toMatchObject({
available: false,
value_flow_support: { status: state, provider_id: "fixture" },
remediation: expect.any(String),
});
if (state === "unavailable")
expect(traceStatus?.remediation).toContain("Fixture has no value flow.");
else
expect(traceStatus?.remediation).toContain("resolve_native_call_targets");
const trace = parseEvidence(
await call("trace_native_values", { procedure: "0x1000", max_depth: 0 }),
);
const value = trace.normalized_result as {
total_nodes: number;
unknowns: Array<{ reason: string }>;
};
if (state === "unavailable") {
expect(value.total_nodes).toBe(0);
expect(value.unknowns).toContainEqual(
expect.objectContaining({ reason: "Fixture has no value flow." }),
);
} else {
expect(value.total_nodes).toBeGreaterThan(0);
expect(
value.unknowns.some((unknown) =>
unknown.reason.includes("resolve_native_call_targets"),
),
).toBe(true);
}
},
);
+26 -4
View File
@@ -709,8 +709,19 @@ describe("session filesystem path boundaries over MCP", () => {
name: "export_evidence_bundle",
arguments: { path: "relative-bundle.json" },
});
expect(exported.isError, JSON.stringify(exported.content)).toBe(true);
expect(JSON.stringify(exported.content)).toContain("absolute");
expect(parseMcpToolError(exported).error).toMatchObject({
code: "invalid_request",
details: {
operation: "export_evidence_bundle",
issues: [
expect.objectContaining({
path: ["path"],
reason: "invalid_format",
expected: "regex",
}),
],
},
});
const opened = await mcp.callTool({
name: "open_binary",
@@ -721,8 +732,19 @@ describe("session filesystem path boundaries over MCP", () => {
name: "close_binary",
arguments: { snapshot_path: "relative-analysis.json" },
});
expect(closed.isError, JSON.stringify(closed.content)).toBe(true);
expect(JSON.stringify(closed.content)).toContain("absolute");
expect(parseMcpToolError(closed).error).toMatchObject({
code: "invalid_request",
details: {
operation: "close_binary",
issues: [
expect.objectContaining({
path: ["snapshot_path"],
reason: "invalid_format",
expected: "regex",
}),
],
},
});
await mcp.callTool({ name: "close_binary", arguments: {} });
}, 10_000);
});
+22 -14
View File
@@ -383,7 +383,7 @@ it("attributes rejected WASM paths to their selected fields before provider work
},
});
onTestFinished(harness.close);
for (const [input, paths] of [
const relativeCases = [
[{ path: "relative.wasm" }, [["path"]]],
[
{ path: value.artifact.path, glue_paths: ["relative.js"] },
@@ -412,7 +412,9 @@ it("attributes rejected WASM paths to their selected fields before provider work
},
[["path"], ["glue_paths", 0], ["candidate_paths", 0]],
],
] as const) {
] as const;
const checked: string[] = [];
for (const [input, paths] of relativeCases) {
const response = await harness.client.callTool({
name: "inspect_wasm_artifact",
arguments: input,
@@ -423,14 +425,19 @@ it("attributes rejected WASM paths to their selected fields before provider work
.object({ issues: z.array(z.unknown()) })
.parse(failure.details);
expect(issues).toEqual(
paths.map((path) => ({
path,
reason: "invalid_format",
expected: "regex",
message: "must be an absolute local filesystem path",
})),
paths.map((path) =>
expect.objectContaining({
path,
reason: "invalid_format",
expected: "regex",
}),
),
);
checked.push(paths.map((path) => path.join(".")).join("+"));
}
expect(checked).toEqual(
expect.arrayContaining(["path", "glue_paths.0", "candidate_paths.0"]),
);
expect(inspect).not.toHaveBeenCalled();
const valid = await harness.client.callTool({
name: "inspect_wasm_artifact",
@@ -760,12 +767,13 @@ it("rejects malformed paths, refuses WASM summary delivery before provider work
expect(
z.object({ issues: z.array(z.unknown()) }).parse(rejected.details).issues,
).toEqual(
[["path"], ["glue_paths", 0]].map((path) => ({
path,
reason: "invalid_format",
expected: "regex",
message: "must be an absolute local filesystem path",
})),
[["path"], ["glue_paths", 0]].map((path) =>
expect.objectContaining({
path,
reason: "invalid_format",
expected: "regex",
}),
),
);
expect(inspect).not.toHaveBeenCalled();
for (const input of [
@@ -0,0 +1,205 @@
import { execFileSync } from "node:child_process";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import {
PYTHON_HELPERS,
PYTHON_IMPORTS,
} from "../../../src/domain/desktop/appExportPython.js";
import {
detectStateFormat,
textualView,
} from "../../../src/domain/desktop/stateFormats.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const arrays = "[".repeat(1500) + "1" + "]".repeat(1500);
const objects = '{"a":'.repeat(1500) + "1" + "}".repeat(1500);
const cases = [
["discarded arrays", `{"value":${arrays},"value":"done"}`, true],
["discarded objects", `{"value":${objects},"value":"done"}`, true],
["escaped duplicate key", `{"val\\u0075e":${arrays},"value":"done"}`, true],
["retained deep value", `{"value":${arrays}}`, false],
[
"retained deep inexact number",
`{"value":${"[".repeat(600)}9007199254740993${"]".repeat(600)}}`,
false,
],
[
"deep overwritten inexact number",
`{"value":${"[".repeat(600)}{"x":9007199254740993,"x":1}${"]".repeat(600)}}`,
false,
],
[
"deep empty-key overwrite",
`{"value":${"[".repeat(600)}{"":9007199254740993,"":1}${"]".repeat(600)}}`,
false,
],
[
"deep multiple inexact keys with one overwritten",
`{"value":${"[".repeat(600)}{"a":9007199254740993,"b":9007199254740993,"a":1}${"]".repeat(600)}}`,
false,
],
[
"deep multiple inexact keys all overwritten",
`{"value":${"[".repeat(600)}{"a":9007199254740993,"b":9007199254740993,"a":1,"b":2}${"]".repeat(600)}}`,
false,
],
[
"deep inexact key replaced by inexact",
`{"value":${"[".repeat(600)}{"a":9007199254740993,"a":9007199254740995}${"]".repeat(600)}}`,
false,
],
[
"deep escaped duplicate overwritten",
`{"value":${"[".repeat(600)}{"\\u0061":9007199254740993,"a":1}${"]".repeat(600)}}`,
false,
],
[
"discarded deep inexact subtree",
`{"value":${"[".repeat(600)}9007199254740993${"]".repeat(600)},"value":"done"}`,
true,
],
["last admitted depth", "[".repeat(500) + "1" + "]".repeat(500), true],
["empty container at limit", "[".repeat(501) + "]".repeat(501), true],
["empty container over limit", "[".repeat(502) + "]".repeat(502), false],
["trailing comma", `{"value":${arrays},"value":"done",}`, false],
["missing value", `{"value":${arrays},"value":}`, false],
["wrong closing bracket", `{"value":${arrays},"value":"done"]`, false],
] as const;
it.each(cases)("reads JSON with %s", async (_name, text, readable) => {
const root = await createTestTempDirectory("desktop-json-depth-");
const script = join(root, "read.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}${PYTHON_HELPERS}
text = json.load(sys.stdin)
print(json.dumps(_text_view("state.json", text, len(text.encode("utf-8")))))
`,
);
const output = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", "-X", "utf8", script],
{ input: JSON.stringify(text), encoding: "utf8" },
),
) as { format: string; view: unknown; error?: string };
const bytes = Buffer.from(text);
const format = detectStateFormat("state.json", bytes);
if (format !== "json" && format !== "text")
throw new Error(`Unexpected native format ${format}`);
const native = textualView(format, bytes);
expect(format === "json" && native.ok).toBe(readable);
expect(output.format).toBe(format);
if (native.ok) {
expect(output.error).toBeUndefined();
// Compare JSON values on their wire form, independent of object prototypes.
expect(JSON.stringify(output.view)).toBe(JSON.stringify(native.value));
if (
readable &&
(_name.startsWith("discarded") || _name === "escaped duplicate key")
)
expect(output.view).toEqual({ value: "done" });
} else {
expect(output.view).toBeNull();
expect(output.error).toBe(native.error);
}
});
it("bounds construction before refusing retained excessive JSON depth", async () => {
const root = await createTestTempDirectory("desktop-json-construction-");
const script = join(root, "bounded.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}${PYTHON_HELPERS}
import tracemalloc
text = "[" * 20_000 + "0" + "]" * 20_000
recursion_limit = sys.getrecursionlimit()
tracemalloc.start()
entry = _json_view(text, len(text))
peak = tracemalloc.get_traced_memory()[1]
tracemalloc.stop()
print(json.dumps({"entry": entry, "peak": peak, "recursion_unchanged": sys.getrecursionlimit() == recursion_limit}))
`,
);
const output = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", "-X", "utf8", script],
{ encoding: "utf8" },
),
) as { entry: unknown; peak: number; recursion_unchanged: boolean };
expect(output.entry).toEqual({
format: "json",
view: null,
error: "nested deeper than the 500 levels a JSON view keeps",
});
expect(output.recursion_unchanged).toBe(true);
// Python heap only, excluding source construction and interpreter/OS memory.
// A value retained only to refuse it must not allocate every input container.
expect(output.peak).toBeLessThan(1024 * 1024);
});
it("bounds allocations when a very deep value is overwritten", async () => {
const root = await createTestTempDirectory("desktop-json-memory-");
const script = join(root, "read.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}\nimport tracemalloc\n${PYTHON_HELPERS}
text = '{"value":' + '[' * 200000 + '1' + ']' * 200000 + ',"value":"done"}'
tracemalloc.start()
result = _json_view(text, len(text))
_, peak = tracemalloc.get_traced_memory()
print(json.dumps({"view": result["view"], "peak": peak}))
`,
);
const measured = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", script],
{ encoding: "utf8", timeout: 30000 },
),
) as { view: unknown; peak: number };
expect(measured.view).toEqual({ value: "done" });
expect(measured.peak).toBeLessThan(8 * 1024 * 1024);
}, 40000);
it("bounds a wide container already beyond the retained view depth", async () => {
const root = await createTestTempDirectory("desktop-json-wide-depth-");
const script = join(root, "read.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}\nimport tracemalloc\n${PYTHON_HELPERS}
text = '[' * 501 + ','.join(['"abc"'] * 600000) + ']' * 501
tracemalloc.start()
result = _json_view(text, len(text))
_, peak = tracemalloc.get_traced_memory()
print(json.dumps({"error": result.get("error"), "peak": peak}))
`,
);
const measured = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", script],
{ encoding: "utf8", timeout: 30000 },
),
) as { error: string; peak: number };
expect(measured.error).toBe(
"nested deeper than the 500 levels a JSON view keeps",
);
expect(measured.peak).toBeLessThan(8 * 1024 * 1024);
}, 40000);
@@ -34,7 +34,7 @@ const execPython = (
try {
writeFileSync(script, args[codeIndex + 1]!, "utf8");
return execFileSync(
"python3",
process.platform === "win32" ? "python" : "python3",
[
"-X",
"utf8",
@@ -375,15 +375,16 @@ it("checks text for JSON the way REA does, token by token, without parsing it",
"-I",
"-B",
"-c",
`${imports}${PYTHON_HELPERS}\nprint(json.dumps([_is_json(text) for text in json.load(sys.stdin)]))`,
`${imports}${PYTHON_HELPERS}\ntexts = json.load(sys.stdin)\nprint(json.dumps({"syntax": [_read_json(text, build=False) is not _INVALID_JSON for text in texts], "built": [_read_json(text) is not _INVALID_JSON for text in texts]}))`,
],
{ input: JSON.stringify(texts), encoding: "utf8" },
),
) as boolean[];
) as { syntax: boolean[]; built: boolean[] };
const parsed = texts.map((text) => safeParseJson(text).ok);
expect(texts.map((text) => isJsonText(text))).toEqual(parsed);
expect(python).toEqual(parsed);
expect(python.syntax).toEqual(parsed);
expect(python.built).toEqual(parsed);
expect(parsed.filter(Boolean)).toHaveLength(10);
});
@@ -445,9 +446,13 @@ print(json.dumps([_text_view("settings.ini", text, len(text.encode()))["view"]["
const root = await createTestTempDirectory("rea-keyfile-python-");
const script = join(root, "fixture.py");
await writeFile(script, program, "utf8");
const output = execFileSync("python3", ["-I", "-B", "-X", "utf8", script], {
input: JSON.stringify(inputs),
encoding: "utf8",
});
const output = execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", "-X", "utf8", script],
{
input: JSON.stringify(inputs),
encoding: "utf8",
},
);
expect(JSON.parse(output)).toEqual(inputs.map(keyfileView));
});
@@ -20,7 +20,7 @@ import { OwnedFileHandle } from "../../../../src/filesystem/OwnedFileHandle.js";
import { AnalysisProviderRegistry } from "../../../../src/application/binary/AnalysisProviderRegistry.js";
import { composeBinarySession } from "../../../../src/application/binary/BinarySessionComposition.js";
import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js";
import { KUNA_NATIVE_VALUE_FLOW_UNAVAILABLE } from "../../../../src/kuna/KunaSessionClient.js";
import { KUNA_VALUE_FLOW_UNAVAILABLE } from "../../../../src/kuna/KunaProviderCapabilities.js";
import { KunaProvider } from "../../../../src/kuna/KunaProvider.js";
import { KUNA_FUNCTION_TEXT_BYTES } from "../../../../src/kuna/KunaLimits.js";
import { KunaProjectCache } from "../../../../src/kuna/KunaProjectCache.js";
@@ -512,7 +512,7 @@ it("builds a dossier with text and call relationships, leaving unsupported facet
assembly: ["main: ; 0x1000", " 00001000: c3 RET"],
native_value_flow: {
available: false,
reason: KUNA_NATIVE_VALUE_FLOW_UNAVAILABLE,
reason: KUNA_VALUE_FLOW_UNAVAILABLE,
},
facet_availability: {
pseudocode: { available: true },