Compare commits

..
Author SHA1 Message Date
morlutoandCursor Agent e47bb152c2 feat(native): retry failed directory decompiles independently (#8811)
* feat(kuna): allow independent decompile retries

* fix(kuna): correct independent retry recovery

* feat(native): retain decompile export source facts

* test(native): verify historical export source facts

* test(kuna): bound retry lifecycle acknowledgement

* test(native): retain source after export conflict

* feat(kuna): allow independent decompile retries

* fix(kuna): correct independent retry recovery

* test(kuna): bound retry lifecycle acknowledgement

* feat(native): retry failed directory decompiles independently

* fix(native): guide directory retry after provider failure

* fix(native): point retry caveats to the journal

* style(test): format CLI help default annotations (#9280)

(cherry picked from commit ac30fbd658)

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:51:21 +08:00
morlutoandCursor Agent 81c09888bf feat(kuna): allow independent decompile retries (#8793)
* feat(kuna): allow independent decompile retries

* fix(kuna): correct independent retry recovery

* test(kuna): bound retry lifecycle acknowledgement

* style(test): format CLI help default annotations (#9280)

(cherry picked from commit ac30fbd658)

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:37:21 +08:00
morlutoandCursor Agent 106ba1730f fix!: preserve address context resource recovery (#8424)
* fix: retain address context cleanup failures

* fix!: preserve address context input errors

BREAKING CHANGE: Strict clients that cached the previous full inspect_address_context output schema must refresh tools/list and accept the typed error field on unavailable facets.

* test: use session contract for address context

* fix!: preserve address context resource recovery

BREAKING CHANGE: Clients validating unavailable context facets against a cached strict full output schema must refresh tools/list to accept resource_constraint errors.

* fix: expose authenticated records refused by the evidence ledger (#8426)

* fix: preserve completed evidence on retention refusal

* fix: preserve context when unretained evidence exceeds delivery limits

* fix(mcp): recover oversized context from session evidence

* fix(evidence): preserve completed atomic write results

* fix(cli): preserve caller path in unretained evidence

* style(test): format CLI help default annotations (#9280)

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:34:20 +08:00
morlutoandCursor Agent f7dc5489b3 feat(kuna): preserve stored pointer references (#8851)
* feat(kuna): verify release 1.990

* feat(kuna): preserve stored pointer references

* fix(kuna): group image-wide reference coverage gaps

* fix(evidence): retain producer-required xref observations

* fix(mcp): clarify xref address page scope

* fix(kuna): preserve reference target aliases

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:33:41 +08:00
3e8a5f76d0 feat(native): retain LLDB source coordinates and verify Pascal calls (#2800)
* feat(native): retain LLDB source coordinates and verify Pascal call entries

Preserve partial line-table attribution for native locations and historical
evidence compatibility. Verify owned Free Pascal call entries through CLI/MCP.

Refs #2382, #2372, #2397

Co-Authored-By: Codex <[email protected]>

* docs: namespace native specifications by domain and feature

Use stable descriptive paths instead of numeric prefixes to avoid parallel contribution collisions.

Co-Authored-By: Codex <[email protected]>

* fix(native): bound Pascal verification and compare caller coordinates

Address review findings at their producing boundaries and preserve directly affected consumers. Source inspection and git diff --check completed. Tests and automated checks were not run at the user’s explicit request.

* fix(native): bound compiler shutdown and source-coordinate fixtures

* docs: clarify historical native source verification

* fix(native): preserve source attribution and compiler ownership

* docs(native): qualify retained source attribution

* fix(ci): clean up native Pascal verifier processes

* test(native): type the Pascal verifier helper import after merging main

Main now typechecks this boundary test, which imports the untyped
canonical-temporary-directory helper and reads an unknown catch value.
Add the helper's declaration file and narrow the caught error.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(native): verify partial source attribution caveat

* fix(native): retain verifier compiler cleanup ownership

* fix(native): forward verifier cancellation and measure path bytes

CLI and MCP artifact runs observe the verifier abort signal, and a
failed Pascal verifier test stops the compiler group before deleting
its directory. Repeated source locations grow by measured UTF-8 JSON
bytes; workflow tokens, CPU and peak memory stay unmeasured.

---------

Co-authored-by: Matheus Breguêz <[email protected]>
Co-authored-by: Codex <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:25:20 +08:00
morluto ac30fbd658 style(test): format CLI help default annotations (#9280) 2026-11-07 23:02:44 +08:00
Nianze Wuandmorluto 5d21bddb18 fix(desktop): read JSON with deeply nested overwritten values (#9277)
* fix(desktop): read JSON with deeply nested overwritten values

* fix(desktop): bound exported JSON construction by view depth

* fix(desktop): preserve deep JSON numeric facts within bounded memory

---------

Co-authored-by: morluto <[email protected]>
2026-11-07 22:57:15 +08:00
morlutoandCursor Agent 2e2f981ccf test: assert absolute-path facts without frozen sentences (#9231)
* test: assert absolute-path facts without frozen sentences

Sibling path tests now require the field path, invalid_format, and regex
expectation, and they record that each negative case ran. NUL rejection
follows the argument that contains the byte instead of searching for the
word NUL.

* test: retain actionable NUL path diagnostics

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 22:46:17 +08:00
101 changed files with 5235 additions and 1008 deletions

No files matched your search

+15
View File
@@ -243,6 +243,20 @@ def _hex(value):
return "0x%x" % value
def _source_location(address):
"""Preserve available line-entry facets without acquiring source files."""
entry = address.GetLineEntry()
if not entry.IsValid():
return None
file_spec = entry.GetFileSpec()
file_path = str(file_spec) if file_spec.IsValid() else None
line = entry.GetLine() or None
column = entry.GetColumn() or None
if not file_path and line is None and column is None:
return None
return {"file_path": file_path or None, "line": line, "column": column}
def _location(address, target):
"""Load and file address, module and symbol of one code address."""
module = address.GetModule()
@@ -256,6 +270,7 @@ def _location(address, target):
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
"symbol": symbol.GetName() if symbol.IsValid() else None,
"source_location": _source_location(address),
}
return location
+88 -13
View File
@@ -25,6 +25,45 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
def address(valid=True, file_valid=True, line=23, column=7):
class SourceFileSpec:
def IsValid(self):
return file_valid
def __str__(self):
return "/missing/é space/fixture.pas"
file_spec = SourceFileSpec()
entry = types.SimpleNamespace(
IsValid=lambda: valid,
GetFileSpec=lambda: file_spec,
GetLine=lambda: line,
GetColumn=lambda: column,
)
return types.SimpleNamespace(GetLineEntry=lambda: entry)
self.assertEqual(
tracer._source_location(address()),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(column=0)),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
)
self.assertEqual(
tracer._source_location(address(file_valid=False)),
{"file_path": None, "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(line=0)),
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
)
self.assertIsNone(tracer._source_location(address(valid=False)))
self.assertIsNone(
tracer._source_location(address(file_valid=False, line=0, column=0))
)
def test_invalid_launch_process_reports_explicit_no_creation(self):
class InvalidProcess:
def IsValid(self):
@@ -390,6 +429,9 @@ class NativeTraceBudgetTests(unittest.TestCase):
return FileSpec()
class Address:
def GetLineEntry(self):
return types.SimpleNamespace(IsValid=lambda: False)
def GetModule(self):
return Module()
@@ -491,7 +533,24 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetFileSpec(self):
return FileSpec()
class SourceFileSpec:
def IsValid(self):
return True
def __str__(self):
return source_path
class Address:
def GetLineEntry(self):
if source_path is None:
return types.SimpleNamespace(IsValid=lambda: False)
return types.SimpleNamespace(
IsValid=lambda: True,
GetFileSpec=SourceFileSpec,
GetLine=lambda: 15,
GetColumn=lambda: 3,
)
def GetModule(self):
return Module()
@@ -518,19 +577,35 @@ class NativeTraceBudgetTests(unittest.TestCase):
breakpoint_count = 1000
created = [[Breakpoint()] for _ in range(breakpoint_count)]
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
with self.subTest(source_path_present=source_path is not None):
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(
sum(row["location_count"] for row in report), 64_000
)
self.assertLess(
sum(len(row["locations"]) for row in report), 64_000
)
self.assertLessEqual(
retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES
)
if source_path is not None:
self.assertTrue(
any(
location["source_location"]["file_path"] == source_path
for row in report
for location in row["locations"]
)
)
if __name__ == "__main__":
+2 -2
View File
@@ -310,8 +310,8 @@ entry it does not read. The module reads the central directory itself, as REA
does, and shows `zipfile` only the records it lists, reading each entry in
bounded chunks no further than a byte past its declared size. A JSON file
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
or nested more than 500 levels deep, gets an `error` instead of a view, as in
REA, while text that is not one JSON
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
document, such as JSON Lines, stays text. Format detection follows REA's
rules. Files larger than 64 MiB get an `error` instead of a view
and are not read. XML is read with Python's `expat` twice: with namespace
+2 -1
View File
@@ -147,6 +147,7 @@ that rollback is resolved; a clean rollback permits another setup attempt.
## Provenance and limitations
- If a completed project export reports a failed function, `procedure_pseudo_code` with `analysis: "independent_fresh"` or `rea decompile --analysis independent_fresh` requests a new single-function attempt. It bypasses the project result and prior single-function and snapshot results; it does not replace the original observation. On a cold session it obtains an inventory without starting a project export. The independent attempt can use different analysis context and may also fail. Identical results can share an Evidence ID because Evidence is content-addressed.
- Addresses and bytes are Kuna's observations of the image. Procedure entries and extents are its derivations: extents are upper bounds clipped at the next entry, and Kuna can split one function at an entry its own analysis infers. Pseudocode is its reconstruction, never original source.
- Pseudocode and assembly are exact Kuna text; text from Kuna and from another provider is not comparable as equal or different.
- Results are re-read rather than replayed from REA's analysis snapshots, because a cold answer can differ from the export's.
@@ -183,7 +184,7 @@ REA_KUNA_COMMAND=/absolute/path/to/kuna GHIDRA_INSTALL_DIR=/absolute/path/to/ghi
npm run verify:kuna -- --targets DIR --functions inflate,crc32 --compare-ghidra
```
The lane compiles a small source-owned fixture with the host C compiler and verifies the MCP workflow end to end. `--targets DIR --functions a,b` adds caller-built binaries. `--compare-ghidra` (with `GHIDRA_INSTALL_DIR`) runs those same functions through Ghidra; it requires `--targets` and `--functions` and refuses empty matrices or requested function names missing from every target. The report lists examined caller binaries and their matching requested symbols, including inputs omitted from analysis. See [testing](testing.md#kuna-lane) for coverage.
The lane compiles a small source-owned fixture with the host C compiler and verifies the MCP workflow end to end. `--targets DIR --functions a,b` adds caller-built binaries. `--compare-ghidra` (with `GHIDRA_INSTALL_DIR`) runs those same functions through Ghidra; it requires `--targets` and `--functions` and refuses empty matrices or requested function names missing from every target. See [testing](testing.md#kuna-lane) for coverage.
Failed runs also report process-lineage evidence and exit unsuccessfully. If provider or transport cleanup is unconfirmed, the report says `cleanup: "incomplete"` and records `retained_workspace`; that directory is preserved for investigation and cleanup retries.
+30 -3
View File
@@ -318,6 +318,8 @@ Retained `xrefs` Evidence also supports `native-observation` address pages and
default and does not accept `detail`; when its MCP response exceeds the budget,
follow the returned continuation to page the retained result. These addresses
remain untyped; paging does not add reference kinds or establish completeness.
The page tells callers that any additional provider detail recorded in the parent
requires retrieving or exporting the complete Evidence.
These views reference complete source context through `source_context.digest`
and caveat counts. Read it using the same parent with
@@ -1296,6 +1298,22 @@ Those raw observations are not recorded in the Evidence bundle. If the SDK
suppresses the error reply after client cancellation, Evidence tools cannot
recover them. An empty bundle does not prove that no partial work or side
effect occurred; output receipts and cleanup observations remain distinct.
If an authenticated Evidence record exceeds the investigation's retention
allowance, the refusal carries it inline as
`details.partial_observation: {kind: "unretained-evidence", evidence: ...}`
when it fits the response budget. Its ID identifies the supplied content; it
is not a retained-session reference. If authentication itself failed or lacked
headroom, no authenticated Evidence is claimed. When the completed record does
not fit the MCP response, the original retention error carries
`details.partial_observation_omitted` with the response constraint if the
remaining error fits; the observation remains unretained. Otherwise the usual
oversized-error recovery applies.
When an atomic derived-Evidence and Unknown write exceeds the allowance after
authentication, the completed Evidence is returned the same way if absent from
the investigation. If that exact Evidence was already present, the error
instead carries `{kind: "retained-evidence", evidence_id: ...}` for the current
session. The refused write adds neither a record nor an Unknown revision; a
previously retained record or Unknown remains available.
The broader retention decision is tracked in
[#4276](https://github.com/morluto/rea/issues/4276).
@@ -1577,12 +1595,21 @@ cursor outside a procedure is represented as
procedure resolution returns typed unreadable output; provider errors never
become absence from their message text. `inspect_address_context` requires an explicit address and
returns name, procedure, comment, inline-comment, and bookmark facets;
unsupported facets are local `unavailable` outcomes. A facet whose input is
rejected includes its canonical `error` projection, including any structured
issues, alongside the other facet outcomes. Use `current_document`,
unsupported facets are local `unavailable` outcomes. A facet rejected for
invalid input or a resource constraint includes its canonical `error`
projection, including structured issues or reported limits and recovery,
alongside the other facet outcomes. Use `current_document`,
`current_address`, and `current_procedure` for direct single-field lookups; use
the aggregate tools when you need related context together.
If unretained facet Evidence makes the context response exceed an MCP delivery
limit, REA omits those partial records in estimated size order until the reply
fits. Each affected facet identifies its unretained omission and the delivery
constraint; other facets and partials remain in the response when they fit. The
size estimate is used only for ordering and can omit more records than a perfect
wire-size ordering would. If the remaining context still cannot fit, call the
needed address operations individually and follow their delivery diagnostics.
## Integrity attribution and reconstruction claims
Artifact comparison results include `integrity_mismatch_evidence_ids`, naming
+30 -1
View File
@@ -252,8 +252,18 @@ The manifest reports the procedure-inventory limitations of the current call in
call's filters, selected and excluded counts, and inventory limitations; older
runs remain represented only by their retained function records, so a later
inventory warning is not presented as the history of every file.
Use `retry_failed: "independent_fresh"` (CLI: `--retry-failed independent_fresh`)
to request an independent attempt after a null pseudocode observation that the
provider explicitly reports as a function failure. The failed observation is
appended before the retry, so cancellation or a later failure keeps it. The
retry can use different analysis context. Other errors and entries merely
missing from an export are not retried. A null without a reported function
failure carries a limitation explaining why no independent retry was attempted.
A provider that does not report this distinction leaves the cause unknown.
Each function's `decompile-index.json` record, whatever its status, keeps the
complete limitations of the observations it used.
complete limitations of the observations it used. The index keeps the latest
record for each function; `decompile-journal.jsonl` keeps earlier attempts and
their separate limitations.
When available, `provider_source` records the source of that attempt's
pseudocode observation: a file path, SHA-256 and byte range, or a
single-function decompile. The latter describes the method, not a unique run. A file
@@ -1154,6 +1164,25 @@ arguments. The receiver class comes from LLDB's Objective-C runtime reader
without running target code; no expression is evaluated, and REA never
attaches to a process it did not launch.
Resolved breakpoint locations, entries and caller frames also retain
`source_location` from LLDB's line tables when available. Its `file_path`,
one-based `line` and one-based `column` are independently nullable. An invalid
line entry is null; historical captures may omit the field. This is debug metadata
attribution: REA does not read that source path or verify the source file bytes.
Absent coordinates do not imply that the executable has no source code.
Free Pascal programs with retained native symbols and usable DWARF can use this
same bounded call observation on macOS. Select an exact emitted symbol from the
binary rather than assuming a source procedure name. The owned Pascal fixture
uses an explicit C calling convention and a public symbol; its verification
covers call entries, raw integer registers and source coordinates, without
establishing typed Pascal locals or interactive stepping. After building REA,
run the real lane with an existing caller-selected compiler:
```sh
REA_VERIFY_FPC_COMMAND=/absolute/path/to/fpc npm run verify:native:pascal-calls
```
The run ends when the process exits, `max_events` entries are recorded,
`duration_ms` elapses, or the aggregate trace resource budget is exhausted.
Bounded termination reports partial coverage. REA checks process termination
@@ -0,0 +1,39 @@
# Implementation plan
## Boundary and contract
Extend nativeCodeLocationSchema in src/domain/native/nativeCallObservation.ts.
Reuse that shape for events, breakpoint locations, backtraces and retained journal
events. Keep the field optional for older evidence and nullable for current LLDB
observations with unavailable metadata. Each inner facet can be independently
unavailable; avoid dropping a valid line number because a file path is absent.
Read SBAddress.GetLineEntry once in the existing Python _location producer. A
valid SBFileSpec supplies its displayed path without filesystem access. Convert
zero line/column to null. An invalid entry returns null. Existing estimation and
serialization budget checks already traverse the location object recursively.
## Incremental work and validation
1. Define contracts and criteria before code; add failing producer tests.
2. Extend the canonical schema and _location producer. Update existing fake
address objects to implement the actual LLDB line-entry interface.
3. Extend provider/MCP boundary coverage and retained-journal coverage for the
additive field; preserve omission compatibility.
4. Extend the existing real macOS verifier with source-coordinate assertions
against its own Objective-C/C fixture built with debug information.
5. Update the native investigation guide; run Python tests, focused TypeScript
tests, repository checks and the real verifier if host authority permits.
## Effects and compatibility
The existing LLDB operation already owns its breakpoint callbacks and process
lifecycle. This adds a metadata read within that observation. It does not introduce
target execution or a new provider beyond existing behavior. No source file is
acquired from a path in debug information. Consumers of historical evidence retain
omission; consumers of new captures see either a partial/full object or null.
## References
- https://lldb.llvm.org/python_api/lldb.SBAddress.html
- https://lldb.llvm.org/python_api/lldb.SBLineEntry.html
@@ -0,0 +1,36 @@
# Native call source coordinates
Issue: https://github.com/morluto/rea/issues/2382
## Problem and outcome
The existing bounded macOS LLDB call observation retains module, symbol and
addresses, but loses compiler-provided source coordinates. Retain line-entry
metadata at the same producer boundary for breakpoint locations, call entries
and caller frames.
## Requirements
- Add optional nullable source_location to the canonical native code location.
A present object contains nullable file_path, line and column. Positive line
and column values are one-based. Provider zero values mean unavailable.
- Preserve partial facts from a valid line entry. Invalid entries or entries
without any usable facet produce null. Historical captures may omit the field.
- Read only SBAddress line metadata; do not resolve paths, read source contents,
evaluate expressions or change the existing launch, stop or cleanup policy.
- The field establishes LLDB line-table attribution, not verified source bytes.
- Existing trace and breakpoint byte budgets include the new representation.
## Acceptance
Producer tests cover valid, invalid and partial entries, Unicode paths and absent
coordinates. Existing evidence without the field remains accepted. CLI/MCP and
partial retained-journal paths preserve the canonical field. The real native-call
verifier checks source coordinates for its owned debug build, including caller
frames and breakpoint locations, and reports actual host/provider coverage.
## Out of scope
New debugger adapters, interactive stepping, source breakpoints, source-file
association verification, source reads, Pascal/Delphi compatibility claims and
Java/Android/CLR runtime support.
@@ -0,0 +1,14 @@
# Tasks
The checked items below record work completed on an earlier revision. The
current revision's final caller-frame assertions, marker-derived line
expectations, timeout handling, and Unicode budget case remain source-reviewed
only; the full native-call verifier is incomplete.
- [x] Define the single-feature scope, contract and implementation plan.
- [x] Reproduce lost source coordinates with producer regression tests.
- [x] Retain line-entry metadata in the producer and canonical schema.
- [x] Verify boundary/MCP propagation and historical omission.
- [x] Historical: verify real CLI/MCP call coordinates on an owned Free Pascal 3.2.2 program.
- [x] Historical: update documentation; pass Python, focused TypeScript, static and discovery checks.
- [ ] Complete the full native-call verifier after investigating #2397.
@@ -0,0 +1,45 @@
# Historical verification
The records below describe earlier revisions, before the final caller-frame
assertions, marker-derived line expectations, forced compiler timeout termination,
and large Unicode source-location budget coverage. They do not validate the
current revision. Those final changes were reviewed from source only; tests and
automated checks were not run at the user’s explicit request.
Host: macOS ARM64. Compiler: caller-installed Free Pascal 3.2.2, Darwin AArch64.
Debugger: LLDB 2103.0.34.103 from Command Line Tools.
- Python producer/budget/output tests: 10 passed.
- Focused native provider, retained-journal and MCP tests: 18 passed.
- Real Pascal CLI and stdio MCP: three expected calls, integer register values
0/1/2, source path and positive line attribution, successful target exit.
- Repository static checks: six tasks and architecture guards passed.
- Test discovery: all 982 test files discovered exactly once.
The complete native-call verifier did not pass. One run reported exited instead
of event-limit for capped overlapping breakpoints. Another reported unavailable
launch start identity in a later capped scenario. Investigation:
https://github.com/morluto/rea/issues/2397. These results have not yet been isolated
on unchanged upstream. The source-coordinate PR was draft at that historical checkpoint.
The first real native scenario and CLI/MCP source-coordinate comparisons completed
before those broader failures. The Pascal lane verifies bounded native call
observation; it does not establish typed Pascal local values, source breakpoints,
interactive stepping, Delphi compatibility or a GDB provider.
## Unmeasured costs and outstanding execution
Repeated source paths expand event, caller-frame and breakpoint-location payloads.
`nativeCallObservation.test.ts` measures that growth as UTF-8 JSON bytes: three
copies of one location grow by three times the difference between the
`source_location` object and `null`. That byte delta is not a token count.
Equivalent before/after completed workflows have not been measured for agent tokens,
elapsed time, CPU time or peak process-family memory, for either cold or warm runs.
The first compiler probe is only `fpc -iV` with a 30s deadline. Darwin ownership-helper
compilation uses `REA_SWIFTC_COMPILE_TIMEOUT_MS`, default 120s, inside the later
180s `observe-native-calls` run.
The current caller-frame equality checks, marker-derived source lines, forced
compiler timeout termination and Unicode source-path budget case remain unverified
by execution. Earlier Pascal CLI/MCP observations establish only the historical
coverage described above.
+1 -10
View File
@@ -448,16 +448,7 @@ caller-built functions through Ghidra (it refuses to run without `--targets` and
reports latency, sampled process-tree CPU and peak memory, UTF-8 response size,
and goto and warning-marker counts. Those counts are not a correctness judgement,
and model tokens are not measured. Reference timings and response sizes are
reported separately; reference caches expire when the server closes. The report's
`caller_target_selection` retains the selected directory, the original functions
argument, and each examined binary's requested names matched against `nm` text
symbols. `symbol_scan_complete` means the scan of non-`.stripped` entries in
the immediate flavor directories finished; it does not cover other directory
entries. An `inspection_failed` entry and `symbol_scan_complete: false` retain
partial discovery when symbol inspection fails. A missing symbol is not proof
that code is absent, and selection does not establish that a stripped counterpart
was verified; consult `results` for completed variants. `--report PATH` writes
the JSON report.
reported separately; reference caches expire when the server closes. `--report PATH` writes the JSON report.
The first matrix run used Kuna 1.833 and Ghidra 12.1.4 with JDK 21 on macOS
arm64. It covered zlib 1.3.2 `minigzip`, coreutils 9.9 `sort` and the sqlite
+1
View File
@@ -294,6 +294,7 @@
"prepare": "node scripts/prepare.mjs",
"verify:asset-catalog": "npm run build:cached && node scripts/verify/apple/asset-catalog.mjs",
"verify:native-calls": "npm run build:cached && node scripts/verify-native-calls.mjs",
"verify:native:pascal-calls": "npm run build:cached && node scripts/verify-native-pascal-calls.mjs",
"verify:native-ui:permissions": "npm run build:cached && node scripts/verify-native-ui.mjs --allow-permission-denial",
"test:focused": "node scripts/run-development-tests.mjs focused",
"test:sqlite": "npm run test:prepare && npm run test:sqlite:run",
+124 -46
View File
@@ -1,9 +1,8 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { fileURLToPath } from "node:url";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { createVerifierCommands } from "./verify-commands.mjs";
import { createOwnedMcpStdioTransport } from "./owned-mcp-stdio.mjs";
import { withVerifierCleanup } from "./verifier-cleanup.mjs";
import { INLINE_EVIDENCE_MCP_ENVIRONMENT } from "./mcp-verification-environment.mjs";
@@ -75,6 +74,23 @@ export async function artifactMcpCall(client, parameters, options = {}) {
return client.callTool(parameters, { ...options, toolDefinition });
}
/** Retain the exact command owner when a verifier cannot finish cleanup. */
class ArtifactVerifierCleanupError extends Error {
constructor(cause, cleanupOwner) {
super("Artifact verifier cleanup is incomplete", { cause });
this.name = "ArtifactVerifierCleanupError";
this.cleanupOwner = cleanupOwner;
}
}
const closeOwnedCommands = async (owner) => {
try {
await owner.close();
} catch (cause) {
throw new ArtifactVerifierCleanupError(cause, owner);
}
};
const environment = () => {
const compileCache =
process.env.NODE_COMPILE_CACHE ??
@@ -109,68 +125,130 @@ export async function artifactCliEvidence(
command,
target,
arguments_ = [],
{ timeoutMs = 60000 } = {},
{ timeoutMs = 60000, signal, commands } = {},
) {
const { stdout } = await promisify(execFile)(
process.execPath,
[
fileURLToPath(new URL("../rea.mjs", import.meta.url)),
command,
target,
...arguments_,
"--json",
],
{ env: environment(), timeout: timeoutMs, maxBuffer: 16 * 1024 * 1024 },
);
signal?.throwIfAborted();
const owner = commands ?? (await createVerifierCommands());
const execute = () =>
owner.exec(
process.execPath,
[
fileURLToPath(new URL("../rea.mjs", import.meta.url)),
command,
target,
...arguments_,
"--json",
],
{
env: environment(),
timeout: timeoutMs,
maxBuffer: 16 * 1024 * 1024,
signal,
},
);
const { stdout } =
commands === undefined
? await withVerifierCleanup(execute, [() => closeOwnedCommands(owner)])
: await execute();
signal?.throwIfAborted();
const evidence = JSON.parse(stdout);
assert.equal(evidence.error, undefined, stdout);
assert.ok(evidence.normalized_result, "CLI omitted normalized result");
return evidence;
}
const withOpenedArtifactMcp = async (target, openTool, verify) => {
const transport = new StdioClientTransport({
const withOpenedArtifactMcp = async (
target,
openTool,
verify,
options = {},
) => {
options.signal?.throwIfAborted();
const owner = options.commands ?? (await createVerifierCommands());
const transport = await createOwnedMcpStdioTransport({
command: process.execPath,
args: [fileURLToPath(new URL("../rea.mjs", import.meta.url)), "mcp"],
env: { ...environment(), ...INLINE_EVIDENCE_MCP_ENVIRONMENT },
stderr: "pipe",
arguments: [fileURLToPath(new URL("../rea.mjs", import.meta.url)), "mcp"],
hostEnvironment: { ...environment(), ...INLINE_EVIDENCE_MCP_ENVIRONMENT },
cwd: process.cwd(),
expectedCommand: null,
sweepTokenOwnedProcesses: true,
signal: options.signal,
});
owner.adopt(transport, "artifact MCP process");
const client = new Client({ name: "artifact-real-e2e", version: "1" });
return withVerifierCleanup(async () => {
await client.connect(transport);
const opened = await artifactMcpCall(client, {
name: openTool,
arguments: { path: target },
});
assert.notEqual(opened.isError, true, JSON.stringify(opened));
if (openTool === "open_bundle_resources")
assert.equal(opened.structuredContent?.result?.caller_path, target);
await verify(client);
}, [
() => {
toolBindings.delete(client);
},
() => client.close(),
() => transport.close(),
]);
let clientClose;
const closeClient = () =>
(clientClose ??= Promise.resolve().then(() => client.close()));
const abortTransport = () => {
// Keep the same promise for finalization; concurrent SDK closes can return
// before the first close has observed the child exit.
void closeClient().catch(() => undefined);
};
options.signal?.addEventListener("abort", abortTransport, { once: true });
try {
const result = await withVerifierCleanup(async () => {
options.signal?.throwIfAborted();
await client.connect(transport);
options.signal?.throwIfAborted();
const opened = await artifactMcpCall(
client,
{
name: openTool,
arguments: { path: target },
},
{ signal: options.signal },
);
assert.notEqual(opened.isError, true, JSON.stringify(opened));
if (openTool === "open_bundle_resources")
assert.equal(opened.structuredContent?.result?.caller_path, target);
await verify(client);
}, [
() => {
toolBindings.delete(client);
},
() => closeClient(),
() => transport.close(),
...(options.commands === undefined
? [() => closeOwnedCommands(owner)]
: []),
]);
options.signal?.throwIfAborted();
return result;
} finally {
options.signal?.removeEventListener("abort", abortTransport);
}
};
/** Run a real stdio MCP subprocess with an explicitly opened binary target. */
export async function withArtifactMcp(target, verify) {
return withOpenedArtifactMcp(target, "open_binary", verify);
export async function withArtifactMcp(target, verify, options = {}) {
return withOpenedArtifactMcp(target, "open_binary", verify, options);
}
/** Run a real stdio MCP subprocess with an explicitly opened bundle resource root. */
export async function withBundleResourcesMcp(target, verify) {
return withOpenedArtifactMcp(target, "open_bundle_resources", verify);
export async function withBundleResourcesMcp(target, verify, options = {}) {
return withOpenedArtifactMcp(
target,
"open_bundle_resources",
verify,
options,
);
}
/** Require a successful structured MCP tool result rather than its text rendering. */
export async function artifactMcpResult(client, name, arguments_ = {}) {
const result = await artifactMcpCall(client, {
name,
arguments: arguments_,
});
export async function artifactMcpResult(
client,
name,
arguments_ = {},
options = {},
) {
const result = await artifactMcpCall(
client,
{
name,
arguments: arguments_,
},
options,
);
assert.notEqual(result.isError, true, JSON.stringify(result));
assert.ok(
result.structuredContent?.normalized_result,
@@ -0,0 +1,5 @@
/** Allocate under a caller-selected parent and return the host's canonical spelling. */
export declare const createCanonicalTemporaryDirectory: (
prefix: string,
parent?: string,
) => Promise<{ readonly path: string; readonly allocatedPath: string }>;
@@ -0,0 +1,17 @@
import { mkdtemp, realpath, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
/** Allocate under a caller-selected parent and return the host's canonical spelling. */
export const createCanonicalTemporaryDirectory = async (
prefix,
parent = tmpdir(),
) => {
const allocatedPath = await mkdtemp(join(parent, prefix));
try {
return { path: await realpath(allocatedPath), allocatedPath };
} catch (cause) {
await rm(allocatedPath, { recursive: true, force: true });
throw cause;
}
};
+24
View File
@@ -0,0 +1,24 @@
import { importVerifierSource } from "./verifier-source-loader.mjs";
let implementation;
/** Create an SDK-compatible transport which retains the exact process owner. */
export async function createOwnedMcpStdioTransport(options) {
implementation ??= importVerifierSource(
new URL("../../src/process/OwnedMcpStdioTransport.ts", import.meta.url)
.href,
import.meta.url,
);
const { OwnedMcpStdioTransport } = await implementation;
return new OwnedMcpStdioTransport(options, {
provider: "artifact-verifier",
label: "Artifact verifier",
// Match the SDK's default maximum buffered stdio message size.
maxFrameBytes: 10 * 1024 * 1024,
// Unlike the JADX adapter, artifact verifiers do not impose a cumulative
// per-connection output limit; the SDK bounds each buffered message.
maxOperationBytes: Infinity,
gracefulInputCloseMs: 2000,
maxDiagnosticBytes: 64 * 1024,
});
}
+18
View File
@@ -61,6 +61,10 @@ const stable = (result) =>
file_address: event.file_address,
selector: event.selector,
receiver_class: event.receiver_class,
source_location: event.source_location,
caller_source_locations: event.backtrace.map(
(frame) => frame.source_location,
),
counter:
event.symbol === GREET
? event.registers[3].value
@@ -96,6 +100,18 @@ const checkScenario = async (binary) => {
});
const greet = result.events.filter(({ symbol }) => symbol === GREET);
const add = result.events.filter(({ symbol }) => symbol === "rea_call_add");
for (const location of [
...result.events,
...result.events.flatMap((event) => event.backtrace),
...result.breakpoints.flatMap((breakpoint) => breakpoint.locations),
]) {
assert.equal(location.source_location?.file_path, SOURCE);
assert.ok(location.source_location.line > 0);
assert.ok(
location.source_location.column === null ||
location.source_location.column > 0,
);
}
assert.equal(greet.length, 3, JSON.stringify(result));
assert.equal(add.length, 3, JSON.stringify(result));
for (const [index, event] of greet.entries()) {
@@ -269,6 +285,8 @@ try {
const binary = join(directory, "ReaCalls");
await exec("/usr/bin/xcrun", [
"clang",
"-g",
"-O0",
"-fobjc-arc",
"-framework",
"Foundation",
+186
View File
@@ -0,0 +1,186 @@
import assert from "node:assert/strict";
import { readFile, rm } from "node:fs/promises";
import { isAbsolute, join } from "node:path";
import { fileURLToPath } from "node:url";
import { createPackageVerifierCommands } from "./lib/verify-package-commands.mjs";
import { withVerifierCleanup } from "./lib/verifier-cleanup.mjs";
import {
artifactCli,
artifactMcpResult,
withArtifactMcp,
} from "./lib/artifact-e2e.mjs";
import { createCanonicalTemporaryDirectory } from "./lib/canonical-temporary-directory.mjs";
const cancellation = new AbortController();
let shutdownSignal;
const signalExitCodes = { SIGINT: 130, SIGTERM: 143 };
const forwardCancellation = (signal) => {
if (shutdownSignal !== undefined) return;
shutdownSignal = signal;
cancellation.abort(new Error(`Verification cancelled by ${signal}`));
};
const signalHandlers = Object.fromEntries(
Object.keys(signalExitCodes).map((signal) => [
signal,
() => forwardCancellation(signal),
]),
);
for (const [signal, handler] of Object.entries(signalHandlers))
process.on(signal, handler);
const compiler = process.env.REA_VERIFY_FPC_COMMAND;
if (process.platform !== "darwin" || !isAbsolute(compiler ?? ""))
throw new Error(
"Requires macOS and an absolute caller-selected REA_VERIFY_FPC_COMMAND; build the runtime first.",
);
const source = fileURLToPath(
new URL("../tests/conformance/native/calls.pas", import.meta.url),
);
const fixtureLines = (await readFile(source, "utf8")).split(/\r?\n/u);
const callerLines = new Set(
["REA_VERIFY_CALLER_LOOP", "REA_VERIFY_CALLER_CALL"].map((marker) => {
const matches = fixtureLines.flatMap((line, index) =>
line.includes(marker) ? [index + 1] : [],
);
assert.equal(matches.length, 1, `fixture has one ${marker} marker`);
return matches[0];
}),
);
const allocatedDirectory =
await createCanonicalTemporaryDirectory("rea-pascal-calls-");
let directory = allocatedDirectory.path;
const commands = await createPackageVerifierCommands({
workspace: {
path: allocatedDirectory.allocatedPath,
close: () =>
rm(allocatedDirectory.allocatedPath, { recursive: true, force: true }),
},
});
const execute = (command, arguments_, { timeout }) =>
commands.exec(command, arguments_, {
timeout,
diagnosticBytes: 1024 * 1024,
signal: cancellation.signal,
});
const input = {
breakpoints: [{ kind: "function", name: "rea_pascal_call" }],
argument_registers: 1,
backtrace_frames: 1,
duration_ms: 30_000,
};
const inspect = (result) => {
assert.equal(result.process.outcome, "exited");
assert.equal(result.process.exit_status, 0);
assert.equal(result.process.terminated, true);
assert.equal(result.process.stdout.text, "pascal 0\npascal 1\npascal 2\n");
assert.equal(result.events.length, 3);
assert.deepEqual(
result.events.map((event) => event.registers[0].value),
["0x0", "0x1", "0x2"],
);
assert.equal(result.breakpoints[0].location_count, 1);
for (const event of result.events) {
assert.equal(event.backtrace.length, 1);
assert.equal(event.backtrace[0].source_location?.file_path, source);
assert.equal(event.backtrace[0].module_path, join(directory, "calls"));
assert.ok(
callerLines.has(event.backtrace[0].source_location.line),
"caller is the Pascal loop/call site",
);
}
for (const location of [
...result.events,
...result.breakpoints[0].locations,
]) {
assert.equal(location.source_location?.file_path, source);
assert.ok(location.source_location.line > 0);
}
};
const stable = (result) =>
result.events.map((event) => ({
file_address: event.file_address,
source_location: event.source_location,
registers: event.registers,
callers: event.backtrace.map((frame) => ({
file_address: frame.file_address,
symbol: frame.symbol,
source_location: frame.source_location,
})),
}));
try {
const result = await withVerifierCleanup(async () => {
const version = (
await execute(compiler, ["-iV"], { timeout: 30_000 })
).stdout.trim();
const architecture = (
await execute(compiler, ["-iTP"], {
timeout: 30_000,
})
).stdout.trim();
await execute(
compiler,
[
"-dDEBUG",
"-gw3",
"-O-",
"-Xs-",
`-FE${directory}`,
`-FU${directory}`,
source,
],
{ timeout: 120_000 },
);
const binary = join(directory, "calls");
const cli = await artifactCli(
"observe-native-calls",
binary,
[JSON.stringify(input)],
{ timeoutMs: 180_000, signal: cancellation.signal, commands },
);
cancellation.signal.throwIfAborted();
inspect(cli);
await withArtifactMcp(
binary,
async (client) => {
const mcp = await artifactMcpResult(
client,
"observe_native_calls",
input,
{ signal: cancellation.signal },
);
cancellation.signal.throwIfAborted();
inspect(mcp);
assert.deepEqual(stable(mcp), stable(cli));
},
{ signal: cancellation.signal, commands },
);
cancellation.signal.throwIfAborted();
return {
ok: true,
mocked: false,
compiler,
version,
architecture,
debugger: cli.debugger,
cli: true,
stdio_mcp: true,
scope:
"bounded native call entries and compiler-provided source coordinates; no typed Pascal locals or interactive stepping",
};
}, [() => commands.close()]);
cancellation.signal.throwIfAborted();
console.log(JSON.stringify(result));
} catch (cause) {
const cancelled =
cause === cancellation.signal.reason ||
(cause?.reason === "cancelled" &&
cause.cancellation?.reason === cancellation.signal.reason &&
cause.cleanupFailure === null);
if (shutdownSignal === undefined || !cancelled) throw cause;
console.error(`Verification cancelled by ${shutdownSignal}.`);
process.exitCode = signalExitCodes[shutdownSignal];
} finally {
for (const signal of Object.keys(signalExitCodes))
process.off(signal, signalHandlers[signal]);
}
+13 -49
View File
@@ -595,37 +595,15 @@ const measureGhidra = async (path, functions) => {
return answers;
};
const describeFailure = (cause) => ({
message: cause instanceof Error ? cause.message : String(cause),
...(cause instanceof AggregateError
? { causes: cause.errors.map(describeFailure) }
: {}),
});
/** Inspect symbol-bearing caller binaries, retaining both selected and unmatched inputs. */
const targetPairs = async (root, names, selection) => {
/** Symbol-bearing caller binaries with the requested functions, beside their stripped copies. */
const targetPairs = async (root, names) => {
const pairs = [];
for (const flavor of (await readdir(root)).sort()) {
const directory = join(root, flavor);
if (!(await stat(directory)).isDirectory()) continue;
for (const file of (await readdir(directory)).sort()) {
if (file.endsWith(".stripped")) continue;
let symbols;
try {
symbols = await symbolAddresses(join(directory, file), names);
} catch (cause) {
selection.examined.push({
binary: `${flavor}/${file}`,
status: "inspection_failed",
error: describeFailure(cause),
});
throw cause;
}
selection.examined.push({
binary: `${flavor}/${file}`,
status: symbols.size > 0 ? "selected" : "no_matching_text_symbols",
matched_functions: [...symbols.keys()],
});
const symbols = await symbolAddresses(join(directory, file), names);
if (symbols.size > 0) pairs.push({ flavor, directory, file, symbols });
}
}
@@ -646,18 +624,6 @@ const verifyVariant = async ({ flavor, directory, symbols }, variant) => {
const results = [];
let timeoutRecovery;
const failures = [];
const callerTargetSelection =
values.targets === undefined
? undefined
: {
directory: values.targets,
functions_argument: values.functions ?? null,
requested_functions: (values.functions ?? "")
.split(",")
.filter(Boolean),
symbol_scan_complete: false,
examined: [],
};
let inspectionResourcesClean = true;
try {
const installation = await inspectKunaInstallation(
@@ -670,18 +636,13 @@ try {
);
kunaVersion = `kuna ${installation.value.version}`;
let callerPairs = [];
if (callerTargetSelection !== undefined) {
const names = callerTargetSelection.requested_functions;
if (values.targets !== undefined) {
const names = (values.functions ?? "").split(",").filter(Boolean);
assert.ok(names.length > 0, "--targets requires --functions");
callerPairs = await targetPairs(
callerTargetSelection.directory,
names,
callerTargetSelection,
);
callerTargetSelection.symbol_scan_complete = true;
callerPairs = await targetPairs(values.targets, names);
assert.ok(
callerPairs.length > 0,
`No caller-built targets contain the requested functions in ${callerTargetSelection.directory}`,
`No caller-built targets contain the requested functions in ${values.targets}`,
);
const found = new Set(
callerPairs.flatMap(({ symbols }) => [...symbols.keys()]),
@@ -769,15 +730,18 @@ if (cleanupComplete)
cleanupComplete = false;
failures.push(cause);
}
const describeFailure = (cause) => ({
message: cause instanceof Error ? cause.message : String(cause),
...(cause instanceof AggregateError
? { causes: cause.errors.map(describeFailure) }
: {}),
});
const report = {
lane: "kuna",
status: failures.length === 0 ? "passed" : "failed",
kuna_version: kunaVersion,
platform: `${process.platform}/${process.arch}`,
verifier_run: lifecycle,
...(callerTargetSelection === undefined
? {}
: { caller_target_selection: callerTargetSelection }),
results,
...(timeoutRecovery === undefined
? {}
+20 -266
View File
@@ -1,275 +1,29 @@
import {
deserializeMessage,
serializeMessage,
type JSONRPCMessage,
type Transport,
} from "@modelcontextprotocol/client";
import { randomUUID } from "node:crypto";
import { finished } from "node:stream/promises";
import type {
OwnedProviderProcessSpawnOptions,
SpawnedOwnedProviderProcess,
} from "../process/ProviderProcess.js";
import type { AnalysisCapturedOutput } from "../domain/analysisErrorBase.js";
import {
ProviderProcessSupervisor,
spawnOwnedProviderProcess,
} from "../process/ProviderProcess.js";
import { cleanupOwnedProcessGroup } from "../process/ProcessOwnership.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
OwnedMcpStdioTransport,
type OwnedMcpLauncher,
type OwnedMcpTransportFailure,
} from "../process/OwnedMcpStdioTransport.js";
import type { OwnedProviderProcessSpawnOptions } from "../process/ProviderProcess.js";
const MAX_FRAME_BYTES = 8 * 1024 * 1024;
const MAX_OPERATION_BYTES = 32 * 1024 * 1024;
/** Injectable acquisition seam; production owns a POSIX group or Windows job. */
export type JadxLauncher = (
options: OwnedProviderProcessSpawnOptions,
) => Promise<SpawnedOwnedProviderProcess>;
/** The first observed failure, recorded before notifying the SDK or beginning cleanup. */
export type JadxTransportFailure =
| { readonly kind: "process"; readonly cause: unknown }
| { readonly kind: "exit" }
| { readonly kind: "protocol"; readonly cause: Error }
| { readonly kind: "interrupted"; readonly reason: unknown };
/** SDK-framed stdio with REA-owned process cleanup rather than an unowned child. */
export class JadxMcpTransport implements Transport {
onclose: Transport["onclose"];
onerror: Transport["onerror"];
onmessage: Transport["onmessage"];
readonly runId = randomUUID();
#spawned: SpawnedOwnedProviderProcess | undefined;
#supervisor: ProviderProcessSupervisor | undefined;
#frameBuffer = Buffer.alloc(0);
#frameLength = 0;
#bytes = 0;
#started = false;
#closed = false;
#closePromise: Promise<void> | undefined;
#starting: Promise<void> | undefined;
#failure: JadxTransportFailure | undefined;
export type JadxLauncher = OwnedMcpLauncher;
export type JadxTransportFailure = OwnedMcpTransportFailure;
/** The JADX provider's bounded stdio protocol transport. */
export class JadxMcpTransport extends OwnedMcpStdioTransport {
constructor(
readonly options: Omit<OwnedProviderProcessSpawnOptions, "runId" | "stdin">,
readonly launcher: JadxLauncher = spawnOwnedProviderProcess,
) {}
/** Acquires exactly one protocol process after the SDK installs callbacks. */
start(): Promise<void> {
if (this.#started || this.#closed)
throw new Error("JADX transport cannot be restarted");
this.#started = true;
this.#starting = this.#acquire().catch((cause: unknown) => {
this.#failure ??= { kind: "process", cause };
throw cause;
});
return this.#starting;
}
async #acquire(): Promise<void> {
const spawned = await this.launcher({
...this.options,
runId: this.runId,
stdin: "pipe",
});
this.#spawned = spawned;
this.#supervisor = new ProviderProcessSupervisor(
options: Omit<OwnedProviderProcessSpawnOptions, "runId" | "stdin">,
launcher?: JadxLauncher,
) {
super(
options,
{
...spawned,
ownsProcessLifetime: true,
cleanup:
spawned.cleanup ??
(() => cleanupOwnedProcessGroup(spawned.ownership)),
provider: "jadx",
label: "JADX",
maxFrameBytes: 8 * 1024 * 1024,
maxOperationBytes: 32 * 1024 * 1024,
notifyOnExit: true,
},
{ captureStdout: false },
launcher,
);
if (
spawned.process.stdin === undefined ||
spawned.process.stdin === null ||
spawned.process.stdout === null
)
throw new Error("JADX launcher did not supply protocol stdin/stdout");
spawned.process.stdout.on("data", this.#read);
spawned.process.stdout.on("error", this.#error);
spawned.process.stderr?.on("data", this.#countStderr);
spawned.process.on("error", this.#error);
spawned.process.once("exit", this.#exit);
spawned.process.stdin.on("error", this.#error);
if (this.#closed) throw new Error("JADX transport closed during startup");
}
/** Writes SDK-serialized messages and observes stream backpressure. */
async send(message: JSONRPCMessage): Promise<void> {
const input = this.#spawned?.process.stdin;
if (this.#closed || input === undefined || input === null)
throw new Error("JADX protocol process is not writable");
await new Promise<void>((resolve, reject) => {
try {
input.write(serializeMessage(message), (cause) => {
if (cause === null || cause === undefined) resolve();
else {
this.#failure ??= { kind: "process", cause };
reject(cause);
}
});
} catch (cause) {
this.#failure ??= { kind: "process", cause };
reject(cause);
}
});
}
/** Verifies owned process exit before declaring the transport closed. */
close(): Promise<void> {
this.#closePromise ??= this.#stop().catch((cause: unknown) => {
this.#closePromise = undefined;
throw cause;
});
return this.#closePromise;
}
/** Keep the interruption reason distinct from errors produced by its teardown. */
interrupt(reason: unknown): Promise<void> {
this.#failure ??= { kind: "interrupted", reason };
return this.close();
}
/** Captured provider stderr without persisting inherited environment values. */
diagnostics(): AnalysisCapturedOutput | undefined {
const owner = this.#supervisor;
if (owner === undefined) return undefined;
const snapshot = owner.snapshot();
return {
stdout: "",
stderr: snapshot.stderr.text,
stderr_bytes: snapshot.stderr.observedBytes,
truncated:
snapshot.diagnosticTruncated === true || !owner.observedOutputClose(),
...(snapshot.exitCode === undefined
? {}
: { exit_code: snapshot.exitCode }),
...(snapshot.signal === undefined ? {} : { signal: snapshot.signal }),
};
}
/** Preserve the failed protocol/resource constraint after the SDK closes requests. */
observedFailure(): JadxTransportFailure | undefined {
return this.#failure;
}
/** Scope both the byte budget and diagnostic retention to the next serialized request. */
beginOperation(): void {
this.#bytes = 0;
this.#supervisor?.resetOutput();
}
async #stop(): Promise<void> {
this.#closed = true;
this.#frameBuffer = Buffer.alloc(0);
this.#frameLength = 0;
// A cancellation may race acquisition. Join it before inspecting ownership.
await this.#starting?.catch(() => undefined);
const process = this.#spawned?.process;
process?.stdout?.off("data", this.#read);
process?.stdout?.off("error", this.#error);
process?.stderr?.off("data", this.#countStderr);
process?.off("error", this.#error);
process?.off("exit", this.#exit);
const input = process?.stdin;
// A job termination can complete an in-flight pipe write asynchronously.
// Observe its errors until the owned Writable has finished closing.
const inputClosed =
input === undefined || input === null
? undefined
: finished(input, { readable: false, cleanup: true }).catch(
() => undefined,
);
const stopped = await this.#supervisor?.stop();
input?.destroy();
await inputClosed;
input?.off("error", this.#error);
this.onclose?.();
if (stopped?.status === "incomplete")
throw new ProviderCleanupError("jadx", [this.runId], {
reason: stopped.reason,
stderr: this.diagnostics()?.stderr ?? "",
});
}
readonly #error = (error: Error): void => {
this.#failure ??= { kind: "process", cause: error };
this.onerror?.(error);
// Reject pending SDK work after joining the same owned teardown on every path.
void this.close().catch((cause: unknown) =>
this.onerror?.(cause instanceof Error ? cause : new Error(String(cause))),
);
};
readonly #exit = (): void => {
this.#failure ??= { kind: "exit" };
this.onclose?.();
};
readonly #countStderr = (chunk: Buffer | string): void => {
this.#bytes += Buffer.byteLength(chunk);
if (this.#bytes > MAX_OPERATION_BYTES)
this.#fail(
new Error(
`JADX operation output exceeds ${MAX_OPERATION_BYTES} bytes; no complete result is available`,
),
);
};
readonly #read = (chunk: Buffer | string): void => {
if (this.#closed) return;
const bytes = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
this.#bytes += bytes.length;
if (this.#bytes > MAX_OPERATION_BYTES) {
this.#fail(
new Error(
`JADX operation output exceeds ${MAX_OPERATION_BYTES} bytes; no complete result is available`,
),
);
return;
}
try {
let offset = 0;
while (offset < bytes.length) {
const newline = bytes.indexOf(10, offset);
const end = newline === -1 ? bytes.length : newline;
const fragment = bytes.subarray(offset, end);
const frameBytes = this.#frameLength + fragment.length;
if (frameBytes > MAX_FRAME_BYTES)
throw new Error(`JADX MCP frame exceeds ${MAX_FRAME_BYTES} bytes`);
this.#appendFrame(fragment, frameBytes);
if (newline === -1) return;
const line = new TextDecoder("utf-8", {
fatal: true,
// Preserve a leading BOM so the JSON-RPC parser can reject it.
ignoreBOM: true,
}).decode(this.#frameBuffer.subarray(0, this.#frameLength));
this.#frameBuffer = Buffer.alloc(0);
this.#frameLength = 0;
this.onmessage?.(deserializeMessage(line));
offset = newline + 1;
}
} catch (cause) {
this.#fail(cause instanceof Error ? cause : new Error(String(cause)));
}
};
#appendFrame(fragment: Buffer, frameBytes: number): void {
if (frameBytes > this.#frameBuffer.length) {
const capacity = Math.min(
MAX_FRAME_BYTES,
Math.max(frameBytes, this.#frameBuffer.length * 2, 1),
);
const grown = Buffer.allocUnsafe(capacity);
this.#frameBuffer.copy(grown, 0, 0, this.#frameLength);
this.#frameBuffer = grown;
}
fragment.copy(this.#frameBuffer, this.#frameLength);
this.#frameLength = frameBytes;
}
#fail(error: Error): void {
this.#failure ??= { kind: "protocol", cause: error };
this.#error(error);
}
}
+65 -3
View File
@@ -9,8 +9,10 @@ import {
AnalysisCapabilityUnavailableError,
AnalysisInputError,
AnalysisOutputError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import {
omitUnretainedEvidenceFromError,
projectAnalysisError,
type AnalysisErrorProjection,
} from "../domain/analysisErrorProjection.js";
@@ -21,6 +23,7 @@ import { analysisBookmarkSchema } from "../domain/native/nativeInventory.js";
import { parseContainingProcedureResolution } from "../domain/native/nativeObservation.js";
import { nativeAddressCoordinateInputSchema } from "../domain/native/nativeAddress.js";
import { resolveAnalysisDocument } from "./AnalysisDocument.js";
import { estimateJsonStorage } from "./JsonAuthenticationHeadroom.js";
import { err, ok, type Result } from "../domain/result.js";
type Facet =
@@ -32,6 +35,16 @@ type Facet =
readonly error?: AnalysisErrorProjection;
};
type AddressContext = {
readonly address: string;
readonly document: string;
readonly name: Facet;
readonly procedure: Facet;
readonly comment: Facet;
readonly inline_comment: Facet;
readonly bookmarks: Facet;
};
/** Facets inspected after an explicit or provider-resolved document is known. */
export const ADDRESS_CONTEXT_FACET_OPERATIONS = [
"address_name",
@@ -99,12 +112,55 @@ export const getNavigationContext = async (
});
};
/** Preserve other facets and smaller partials if unretained Evidence cannot fit. */
export function* omitContextEvidenceForDelivery(
context: AddressContext,
limit: {
readonly constraint: "receive-buffer" | "string-length";
readonly bytesAtLeast: number;
},
budgetBytes: number,
): Generator<AddressContext> {
const names = [
"name",
"procedure",
"comment",
"inline_comment",
"bookmarks",
] as const;
const candidates = names.flatMap((name) => {
const facet = context[name];
if (facet.state !== "unavailable" || facet.error === undefined) return [];
const compact = omitUnretainedEvidenceFromError(facet.error, limit);
if (compact === undefined) return [];
const partial = facet.error.details?.partial_observation;
const size = estimateJsonStorage(partial, {
maximumValueBytes: budgetBytes,
includeSerialized: false,
}).valueBytes;
return [{ name, facet, compact, size }];
});
candidates.sort((left, right) => right.size - left.size);
let current = context;
for (const { name, facet, compact } of candidates) {
current = {
...current,
[name]: {
...facet,
remediation: compact.remediation.action,
error: compact,
},
};
yield current;
}
}
/** Inspect bounded provider-neutral facets at one explicit reproducible address. */
export const inspectAddressContext = async (
analysis: AnalysisOperationPort,
input: { readonly address: string; readonly document?: string | undefined },
signal?: AbortSignal,
): Promise<Result<JsonValue, AnalysisError>> => {
): Promise<Result<AddressContext, AnalysisError>> => {
const document = await resolveAnalysisDocument(
analysis,
input.document,
@@ -225,9 +281,15 @@ const unavailableFacet = (
: "Review provider availability in binary_session with detail complete and select another supported provider if appropriate. After switching, if a name is known and search_procedures is available, use it to find candidates and verify their identity and coordinates.",
};
}
if (error instanceof AnalysisInputError) {
if (
error instanceof AnalysisInputError ||
error instanceof AnalysisResourceConstraintError
) {
const projected = projectAnalysisError(error);
if (projected.code === "invalid_request")
if (
projected.code === "invalid_request" ||
projected.code === "resource_constraint"
)
return {
state: "unavailable",
reason: projected.message,
+43
View File
@@ -19,6 +19,10 @@ import type {
EvidenceSubjectTarget,
} from "../domain/evidence.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import {
pseudocodeAbsenceSchema,
type PseudocodeAbsence,
} from "../domain/native/pseudocodeAbsence.js";
import {
analysisSourceDescriptorSchema,
type AnalysisSourceDescriptor,
@@ -65,6 +69,8 @@ export type AnalysisOperation =
/** Atomic successful provider observation with exact producing provenance. */
export interface AnalysisExecution extends AnalysisObservation {
readonly provider: ProviderIdentity;
/** Provider-observed cause of a null pseudocode result, when known. */
readonly pseudocodeAbsence?: PseudocodeAbsence;
/**
* Compact provider-owned facts for this execution. This is not an Evidence
* record and is not restored by snapshot replay; consumers must treat its
@@ -82,6 +88,22 @@ export interface AnalysisExecution extends AnalysisObservation {
* output.
*/
readonly referencesTruncated?: boolean;
/** The producer needs this observation retained as source Evidence for composed workflows. */
readonly requiresSourceEvidence?: true;
/** Known reference-discovery gaps; absent means coverage was not reported. */
readonly referenceCoverageGaps?: readonly {
readonly code: string;
readonly scope: "image" | "observation";
readonly description: string;
}[];
/** Typed source facts for this live xref observation; absent means unreported. */
readonly referenceClassifications?: readonly {
readonly sourceAddress: string;
readonly targetAddress: string;
readonly kind: "stored_pointer";
readonly provenance: string;
readonly width: number;
}[];
/**
* Sibling operations known to be unavailable at the provider level when this
* operation answered from a standalone or fallback path.
@@ -91,12 +113,16 @@ export interface AnalysisExecution extends AnalysisObservation {
interface CreateAnalysisExecutionOptions {
readonly rawResult?: unknown;
readonly pseudocodeAbsence?: PseudocodeAbsence;
readonly sourceDescriptor?: AnalysisSourceDescriptor;
readonly analysisProfile?: AnalysisProfileCommitment;
readonly cachePolicy?: "live";
readonly limitations?: readonly string[];
readonly relationshipCoverage?: RelationshipCoverage;
readonly referencesTruncated?: boolean;
readonly requiresSourceEvidence?: true;
readonly referenceClassifications?: AnalysisExecution["referenceClassifications"];
readonly referenceCoverageGaps?: AnalysisExecution["referenceCoverageGaps"];
readonly siblingUnavailability?: AnalysisCapabilityUnavailableError;
readonly locations?: readonly EvidenceLocation[];
readonly sourceEvidence?: readonly Evidence[];
@@ -132,6 +158,13 @@ export function createAnalysisExecution(
? null
: jsonValueSchema.parse(options.rawResult),
provider,
...(options.pseudocodeAbsence === undefined
? {}
: {
pseudocodeAbsence: pseudocodeAbsenceSchema.parse(
options.pseudocodeAbsence,
),
}),
...(options.sourceDescriptor === undefined
? {}
: {
@@ -156,6 +189,15 @@ export function createAnalysisExecution(
...(options.referencesTruncated === undefined
? {}
: { referencesTruncated: options.referencesTruncated }),
...(options.requiresSourceEvidence === undefined
? {}
: { requiresSourceEvidence: options.requiresSourceEvidence }),
...(options.referenceCoverageGaps === undefined
? {}
: { referenceCoverageGaps: options.referenceCoverageGaps }),
...(options.referenceClassifications === undefined
? {}
: { referenceClassifications: options.referenceClassifications }),
...(options.siblingUnavailability === undefined
? {}
: { siblingUnavailability: options.siblingUnavailability }),
@@ -311,6 +353,7 @@ export type CapabilityDescriptor = CapabilityAvailability & {
* exclusions apply to every policy.
*/
readonly cachePolicy?: "snapshot" | "live";
readonly independentDecompile?: true;
/** Exact source profile for an operation implemented by a versioned local reader. */
readonly analysisProfile?: AnalysisProfileCommitment;
readonly provider: ProviderIdentity;
+9 -3
View File
@@ -25,8 +25,10 @@ import {
retainAnalysisResultSources,
type AnalysisResultWithSources,
} from "./investigation/recordEvidenceSources.js";
import { resolveInvestigationEvidenceClosure } from "./investigation/InvestigationEvidence.js";
import { createInvestigationEvidenceClosureChecker } from "./investigation/InvestigationEvidence.js";
import {
resolveInvestigationEvidenceClosure,
createInvestigationEvidenceClosureChecker,
} from "./investigation/InvestigationEvidence.js";
import type { Evidence } from "../domain/evidence.js";
import type { NativeToolName } from "../contracts/native/nativeToolContracts.js";
import type { EnhancedToolName } from "../contracts/enhancedInputs.js";
@@ -46,7 +48,10 @@ import {
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { access, stat } from "node:fs/promises";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { requiresNativeSourceAdmissionBeforeReplay } from "./binary/AnalysisSnapshotCache.js";
import {
requestsFreshIndependentDecompile,
requiresNativeSourceAdmissionBeforeReplay,
} from "./binary/AnalysisSnapshotCache.js";
import { classifyBinaryTargetFilesystemFailure } from "./BinaryTargetResolver.js";
import {
snapshotBindingMismatch,
@@ -488,6 +493,7 @@ const runAnalysis = async (
) &&
tool !== "trace_native_ui_action" &&
tool !== "decompile_all" &&
!requestsFreshIndependentDecompile(tool, arguments_) &&
routeAllowsSnapshotReplay(route, tool)
) {
const cached = snapshotEvidenceForQuery(snapshot, {
+18 -13
View File
@@ -33,19 +33,24 @@ export const projectDirectAnalysisFailure = (
callerInputPath: string,
): ReturnType<typeof projectAnalysisError> => {
const projection = projectInvocationFailure(error);
const observation = tryParseEvidence(projection.details?.partial_observation);
return observation === undefined
? projection
: {
...projection,
details: {
...projection.details,
partial_observation: withEvidenceCallerInputPath(
observation,
callerInputPath,
),
},
};
const partial = projection.details?.partial_observation;
const wrapped =
typeof partial === "object" &&
partial !== null &&
!Array.isArray(partial) &&
partial.kind === "unretained-evidence";
const observation = tryParseEvidence(wrapped ? partial.evidence : partial);
if (observation === undefined) return projection;
const rebound = withEvidenceCallerInputPath(observation, callerInputPath);
return {
...projection,
details: {
...projection.details,
partial_observation: wrapped
? { ...partial, evidence: rebound }
: rebound,
},
};
};
/** Await one-shot cleanup and retain either the outcome or both failures. */
+29
View File
@@ -137,6 +137,7 @@ const traceReferences = async (
const tracedTargets = new Set<string>();
// A source address is resolved once however many matches reference it.
const containing = new Map<string, TraceContainingProcedure>();
const imageCoverageGaps = new Set<string>();
for (const match of request.matches) {
if (tracedTargets.has(match.address)) continue;
tracedTargets.add(match.address);
@@ -154,6 +155,17 @@ const traceReferences = async (
`References to ${match.address} were truncated by the provider; additional references may exist.`,
);
}
for (const gap of observed.value.referenceCoverageGaps ?? []) {
if (gap.scope === "observation") {
residual.push(`References to ${match.address}: ${gap.description}`);
} else {
const key = JSON.stringify([observed.value.provider.id, gap.code]);
if (!imageCoverageGaps.has(key)) {
imageCoverageGaps.add(key);
residual.push(gap.description);
}
}
}
const sources = observed.value.result;
if (!Array.isArray(sources))
return err(
@@ -162,6 +174,20 @@ const traceReferences = async (
"provider returned a non-array result",
),
);
const classifications = new Map<
string,
{ kind: "stored_pointer"; provenance: string; width: number }[]
>();
for (const fact of observed.value.referenceClassifications ?? []) {
if (fact.targetAddress !== match.address) continue;
const values = classifications.get(fact.sourceAddress) ?? [];
values.push({
kind: fact.kind,
provenance: fact.provenance,
width: fact.width,
});
classifications.set(fact.sourceAddress, values);
}
for (const source of sources) {
if (typeof source !== "string")
return err(
@@ -211,6 +237,9 @@ const traceReferences = async (
target_address: match.address,
source_address: source,
containing_procedure: containingProcedure,
...(classifications.has(source)
? { source_classifications: classifications.get(source)! }
: {}),
});
}
}
+1 -1
View File
@@ -21,7 +21,7 @@ export const workflowAnalysisProfile = (
: operation === "batch_decompile"
? { workflow_revision: 4 }
: operation === "trace_feature"
? { workflow_revision: 4 }
? { workflow_revision: 8 }
: operation === "get_call_graph"
? { workflow_revision: 3 }
: operation === "trace_native_values"
+1 -1
View File
@@ -63,7 +63,7 @@ const DECOMPILE_ALL_LIMITATIONS: ComposedLimitationBudget = {
perItemOperations: new Set(["procedure_info", "procedure_pseudo_code"]),
maxDistinct: 8,
retainedIn:
"each function's record in decompile-index.json keeps the caveats of the observations it used",
"decompile-index.json keeps the latest record for each function; decompile-journal.jsonl keeps each attempt and its caveats",
};
/** The caveat budget for a composed workflow, when it has one. */
@@ -864,6 +864,73 @@ describe("document-scoped snapshot admission", () => {
});
});
it("retains producer-required xrefs without replaying a filesystem-writing query", () => {
const descriptor: CapabilityDescriptor = {
operation: "xrefs",
provider: ANALYSIS_SNAPSHOT_PROVIDER,
available: true,
reason: null,
effects: {
mutatesArtifact: false,
launchesProcess: true,
mayShowUi: false,
mayAccessNetwork: false,
mayWriteFilesystem: false,
changesPermissions: false,
requiresRoot: false,
},
limitations: [],
};
const xrefs = descriptor;
expect(
isSourceEvidenceRetainable("xrefs", xrefs, { address: "0x1000" }),
).toBe(false);
expect(isSnapshotCacheable("xrefs", xrefs, { address: "0x1000" })).toBe(
false,
);
expect(isSourceEvidenceRetainable("xrefs", xrefs, {})).toBe(false);
const writingXrefs = {
...xrefs,
effects: { ...xrefs.effects, mayWriteFilesystem: true },
};
expect(
isSourceEvidenceRetainable("xrefs", writingXrefs, { address: "0x1000" }),
).toBe(false);
expect(
isSourceEvidenceRetainable(
"xrefs",
writingXrefs,
{ address: "0x1000" },
{ requiresSourceEvidence: true },
),
).toBe(true);
expect(
isSourceEvidenceRetainable(
"xrefs",
writingXrefs,
{},
{ requiresSourceEvidence: true },
),
).toBe(false);
expect(
isSourceEvidenceRetainable(
"xrefs",
{
...writingXrefs,
effects: { ...writingXrefs.effects, mutatesArtifact: true },
},
{ address: "0x1000" },
{ requiresSourceEvidence: true },
),
).toBe(false);
expect(
isSnapshotCacheable("xrefs", writingXrefs, {
address: "0x1000",
document: "fixture",
}),
).toBe(false);
});
describe("analysis snapshot cache partitioning", () => {
it.each<{
operation: Exclude<AnalysisOperation, "health">;
@@ -1490,13 +1557,43 @@ describe("executions that report truncated references", () => {
});
});
it("does not replay references when the snapshot would lose their classifications", () => {
const cache = new AnalysisSnapshotCache();
const parameters = { address: "0x1000", document: "fixture" };
cache.record({
target: ANALYSIS_SNAPSHOT_TARGET,
profile: ANALYSIS_SNAPSHOT_PROFILE,
operation: "xrefs",
parameters,
execution: createAnalysisExecution(["0x2000"], ANALYSIS_SNAPSHOT_PROVIDER, {
referenceClassifications: [
{
sourceAddress: "0x2000",
targetAddress: "0x1000",
kind: "stored_pointer",
provenance: "relocation",
width: 8,
},
],
}),
});
expect(
cache.lookup(
ANALYSIS_SNAPSHOT_TARGET,
ANALYSIS_SNAPSHOT_PROFILE,
"xrefs",
parameters,
),
).toBeUndefined();
});
it("revises the trace_feature workflow profile so earlier results are not replayed", () => {
const profile = ANALYSIS_SNAPSHOT_PROFILE;
expect(
workflowAnalysisProfile(profile, "trace_feature").parameters
.workflow_revision,
).toBe(4);
).toBe(8);
expect(workflowAnalysisProfile(profile, "trace_feature")).not.toEqual(
workflowAnalysisProfile(profile),
@@ -117,28 +117,41 @@ const DOCUMENTLESS_SOURCE_EVIDENCE_OPERATIONS: ReadonlySet<AnalysisOperation> =
/**
* Whether a source observation is safe to retain as Evidence.
* A missing document selector blocks snapshot replay. Only caller and callee
* walks stay in the ledger without one.
* A missing document selector blocks snapshot replay. A producer can require
* its observation as Evidence independently of whether it can be replayed.
*/
export const isSourceEvidenceRetainable = (
operation: AnalysisOperation,
descriptor: CapabilityDescriptor | undefined,
parameters: Readonly<Record<string, JsonValue>>,
context?: ExecutionOptions["snapshotContext"],
options: {
readonly snapshotContext?: ExecutionOptions["snapshotContext"];
readonly requiresSourceEvidence?: boolean;
} = {},
): descriptor is CapabilityDescriptor =>
!STATEFUL_OPERATIONS.has(operation) &&
!RESOURCE_SCOPED_OPERATIONS.has(operation) &&
(!DOCUMENT_SCOPED_OPERATIONS.has(operation) ||
typeof parameters.document === "string" ||
DOCUMENTLESS_SOURCE_EVIDENCE_OPERATIONS.has(operation) ||
isOneShotPseudocode(operation, descriptor, context)) &&
options.requiresSourceEvidence === true ||
isOneShotPseudocode(operation, descriptor, options.snapshotContext)) &&
(!CURSOR_DEFAULT_OPERATIONS.has(operation) ||
typeof parameters.address === "string") &&
descriptor?.effects.mutatesArtifact === false &&
(descriptor.cachePolicy === "snapshot" ||
(options.requiresSourceEvidence === true ||
descriptor.cachePolicy === "snapshot" ||
descriptor.effects.mayWriteFilesystem === false) &&
descriptor.effects.changesPermissions === false;
/** Explicit independent decompiles must run even if a snapshot has a matching entry. */
export const requestsFreshIndependentDecompile = (
operation: string,
parameters: Readonly<Record<string, JsonValue>>,
): boolean =>
operation === "procedure_pseudo_code" &&
parameters.analysis === "independent_fresh";
/** Whether an observation can additionally be replayed as a snapshot. */
export const isSnapshotCacheable = (
operation: AnalysisOperation,
@@ -146,7 +159,10 @@ export const isSnapshotCacheable = (
parameters: Readonly<Record<string, JsonValue>>,
context?: ExecutionOptions["snapshotContext"],
): descriptor is CapabilityDescriptor =>
isSourceEvidenceRetainable(operation, descriptor, parameters, context) &&
!requestsFreshIndependentDecompile(operation, parameters) &&
isSourceEvidenceRetainable(operation, descriptor, parameters, {
snapshotContext: context,
}) &&
(replayDocumentSelected(operation, parameters) ||
isOneShotPseudocode(operation, descriptor, context)) &&
descriptor.cachePolicy !== "live";
@@ -589,9 +605,15 @@ export class AnalysisSnapshotCache {
): void {
const { target, profile, operation, parameters, execution } = input;
this.select(target, profile);
// The snapshot format does not carry reference truncation; a replay would
// present a capped read as complete, so it is not recorded.
if (execution.referencesTruncated === true) return;
// The snapshot format does not carry reference truncation or stored-pointer
// classifications. Replaying either would present a capped or untyped read
// as the live observation.
if (
execution.referencesTruncated === true ||
execution.referenceClassifications !== undefined ||
execution.referenceCoverageGaps !== undefined
)
return;
let sourceBinding: SnapshotReplayBinding | undefined;
if ((execution.sourceEvidence?.length ?? 0) > 0) {
if (execution.evidenceId === undefined || lookup === undefined) return;
+9 -7
View File
@@ -1539,12 +1539,10 @@ export class BinarySession
}
if (
execution.ok &&
isSourceEvidenceRetainable(
name,
capability,
arguments_,
options?.snapshotContext,
) &&
isSourceEvidenceRetainable(name, capability, arguments_, {
snapshotContext: options?.snapshotContext,
requiresSourceEvidence: execution.value.requiresSourceEvidence === true,
}) &&
profile !== undefined
) {
const evidence = createAnalysisExecutionEvidence(
@@ -1561,7 +1559,11 @@ export class BinarySession
"observed",
);
const recorded = this.recordEvidence(evidence);
if (!recorded.ok) return recorded;
if (!recorded.ok) {
// The provider finished. A full ledger must not discard that observation.
recorded.error.retainPartialObservation(evidence);
return recorded;
}
const retained = { ...execution.value, evidenceId: evidence.evidence_id };
if (cacheable && execution.value.cachePolicy !== "live")
this.recordSnapshot({
@@ -19,6 +19,7 @@ import type {
} from "../AnalysisProvider.js";
import {
isSnapshotCacheable,
requestsFreshIndependentDecompile,
requiresNativeSourceAdmissionBeforeReplay,
} from "./AnalysisSnapshotCache.js";
import type { SessionProviderRoute } from "./SessionProviderRouter.js";
@@ -89,6 +90,17 @@ export const prepareSessionExecution = (
),
);
}
if (
requestsFreshIndependentDecompile(operation, parameters) &&
capability.independentDecompile !== true
)
return err(
new AnalysisCapabilityUnavailableError(
capability.provider.id,
operation,
"This provider does not declare fresh independent decompilation.",
),
);
const profile =
active.profile !== null &&
capability.provider.id === active.profile.provider.id
@@ -417,6 +417,7 @@ export const providerCapabilitySets = (
const key = JSON.stringify([
effects,
limitations,
descriptor.independentDecompile ?? null,
status.available,
status.reason,
status.availability_code,
@@ -429,6 +430,9 @@ export const providerCapabilitySets = (
operations: [descriptor.operation],
effects,
limitations,
...(descriptor.independentDecompile === undefined
? {}
: { independent_decompile: descriptor.independentDecompile }),
...status,
});
else group.operations.push(descriptor.operation);
@@ -1416,9 +1416,9 @@ describe("decompile_all journal recovery", () => {
expect(limitations).toContainEqual(
expect.stringContaining("terminal call may cut the body"),
);
expect(limitations.at(-1)).toBe(
"33 per-item provider limitation observations were omitted from this answer; each function's record in decompile-index.json keeps the caveats of the observations it used.",
);
expect(limitations.at(-1)).toContain("33 per-item provider limitation");
expect(limitations.at(-1)).toContain("decompile-index.json");
expect(limitations.at(-1)).toContain("decompile-journal.jsonl");
const last = await readFile(
join(directory(), "functions/0x1130.c"),
"utf8",
@@ -153,6 +153,7 @@ export const decompileAll = async (
identity: loaded.value.identity,
signal,
limit: input.limit,
retryFailed: input.retry_failed,
state,
});
limitations = exportLimitations(
@@ -361,6 +362,7 @@ interface PublishInput {
readonly identity: ExportIdentity;
readonly signal: AbortSignal | undefined;
readonly limit: number | undefined;
readonly retryFailed: DecompileDirectoryInput["retry_failed"];
readonly state: PublishState;
}
@@ -485,6 +487,7 @@ const publishOne = async (input: {
readonly signal: AbortSignal | undefined;
readonly state: PublishState;
readonly bodySupport: "unknown" | "unavailable";
readonly retryFailed: DecompileDirectoryInput["retry_failed"];
}): Promise<{
readonly result: Result<void, AnalysisError>;
readonly bodySupport: "unknown" | "unavailable";
@@ -515,7 +518,8 @@ const publishOne = async (input: {
bodySupport: body.support,
};
}
const decompiled = await input.analysis.execute(
let attemptedIndependent = false;
let decompiled = await input.analysis.execute(
"procedure_pseudo_code",
{ procedure: input.procedure.address },
executionOptions(input.signal),
@@ -525,6 +529,47 @@ const publishOne = async (input: {
return { result: decompiled, bodySupport: body.support };
if (isAborted(input.signal))
return { result: err(cancelled(input.signal)), bodySupport: body.support };
if (
decompiled.ok &&
input.retryFailed === "independent_fresh" &&
decompiled.value.result === null &&
decompiled.value.pseudocodeAbsence?.kind === "provider-failure" &&
observationMismatch(input.identity, decompiled.value) === undefined
) {
const original = decompiled.value;
const originalSource: NonNullable<
DecompileDirectoryFunctionRecord["provider_source"]
> =
original.sourceDescriptor === undefined
? { status: "unknown", reason: "source_not_available" }
: { status: "reported", source: original.sourceDescriptor };
const recorded = await input.session.append(
failureRecord(input.procedure, body.fact, pseudocodeFailure(null), {
limitations: [...new Set([...infoCaveats, ...original.limitations])],
providerSource: originalSource,
pseudocodeAbsence: original.pseudocodeAbsence,
}),
);
if (!recorded.ok) return { result: recorded, bodySupport: body.support };
if (isAborted(input.signal))
return {
result: err(cancelled(input.signal)),
bodySupport: body.support,
};
attemptedIndependent = true;
decompiled = await input.analysis.execute(
"procedure_pseudo_code",
{ procedure: input.procedure.address, analysis: "independent_fresh" },
executionOptions(input.signal),
);
if (!decompiled.ok && decompiled.error.cleanupIncomplete)
return { result: decompiled, bodySupport: body.support };
if (isAborted(input.signal))
return {
result: err(cancelled(input.signal)),
bodySupport: body.support,
};
}
if (!decompiled.ok) {
if (decompiled.error instanceof AnalysisCancelledError)
return { result: decompiled, bodySupport: body.support };
@@ -553,7 +598,17 @@ const publishOne = async (input: {
};
}
const caveats = [
...new Set([...infoCaveats, ...decompiled.value.limitations]),
...new Set([
...infoCaveats,
...decompiled.value.limitations,
...(input.retryFailed === "independent_fresh" &&
!attemptedIndependent &&
decompiled.value.result === null
? [
"Independent retry was not attempted because the provider did not report a function failure for this observation.",
]
: []),
]),
];
const observationSource: NonNullable<
DecompileDirectoryFunctionRecord["provider_source"]
@@ -563,13 +618,25 @@ const publishOne = async (input: {
: { status: "reported", source: decompiled.value.sourceDescriptor };
const pseudocode = decompiled.value.result;
if (typeof pseudocode !== "string" || pseudocode.length === 0) {
const reason = pseudocodeFailure(pseudocode);
const reason =
input.retryFailed === undefined &&
pseudocode === null &&
decompiled.value.pseudocodeAbsence?.kind === "provider-failure"
? {
code: "no_pseudocode",
message:
"The provider reported a function decompilation failure. If it supports independent decompilation, resume this export with retry_failed: independent_fresh (CLI: --resume --retry-failed independent_fresh). The retry may use different analysis context.",
}
: pseudocodeFailure(pseudocode);
recordFailure(input.state, input.procedure, reason);
return {
result: await input.session.append(
failureRecord(input.procedure, body.fact, reason, {
limitations: caveats,
providerSource: observationSource,
...(decompiled.value.pseudocodeAbsence === undefined
? {}
: { pseudocodeAbsence: decompiled.value.pseudocodeAbsence }),
}),
),
bodySupport: body.support,
@@ -923,6 +990,7 @@ const failureRecord = (
/** Caveats of the observations this attempt used. */
readonly limitations?: readonly string[];
readonly providerSource?: DecompileDirectoryFunctionRecord["provider_source"];
readonly pseudocodeAbsence?: DecompileDirectoryFunctionRecord["pseudocode_absence"];
} = {},
): DecompileDirectoryFunctionRecord => ({
address: procedure.address,
@@ -933,6 +1001,9 @@ const failureRecord = (
...(options.providerSource === undefined
? {}
: { provider_source: options.providerSource }),
...(options.pseudocodeAbsence === undefined
? {}
: { pseudocode_absence: options.pseudocodeAbsence }),
...(options.relativePath === undefined
? {}
: { relative_path: options.relativePath }),
@@ -9,7 +9,10 @@ import {
validateResidualUnknownAddition,
type EvidenceBundle,
} from "../../domain/evidenceBundle.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import {
AnalysisInputError,
AnalysisResourceConstraintError,
} from "../../domain/analysisErrorCore.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import {
@@ -17,6 +20,7 @@ import {
evidenceDisplayMetadataAgree,
type Evidence,
} from "../../domain/evidence.js";
import { freezeJsonSnapshot } from "../../domain/immutableJson.js";
import { safeParseImmutableEvidence } from "../../domain/evidenceValidation.js";
import {
createResidualUnknown,
@@ -111,7 +115,8 @@ export class EvidenceLedger {
"record_evidence",
new Map([[evidence.evidence_id, evidence]]),
);
if (!admitted.ok) return admitted;
if (!admitted.ok)
return err(this.#exposeRefusedEvidence(admitted.error, evidence));
this.#records.set(evidence.evidence_id, evidence);
this.#commitAccounting(admitted.value);
return ok("added");
@@ -366,14 +371,38 @@ export class EvidenceLedger {
if (creationUnknown.revision_digest !== unknown.revision_digest)
return err(new UnknownRegistryError("already-exists"));
const checked = this.#appendIncremental([evidence]);
if (!checked.ok) return checked;
if (!checked.ok)
return err(this.#exposeRefusedEvidence(checked.error, evidence));
return ok(null);
}
const checked = this.#appendIncremental([evidence, mutation], unknown);
if (!checked.ok) return checked;
if (!checked.ok)
return err(this.#exposeRefusedEvidence(checked.error, evidence));
return ok(structuredClone(unknown));
}
/** Report authenticated completed Evidence without inferring membership from this attempt. */
#exposeRefusedEvidence(
error: AnalysisError,
evidence: Evidence,
): AnalysisError {
if (
!(error instanceof AnalysisResourceConstraintError) ||
error.resource !== "capacity"
)
return error;
return error.retainPartialObservation(
freezeJsonSnapshot(
this.#records.has(evidence.evidence_id)
? {
kind: "retained-evidence" as const,
evidence_id: evidence.evidence_id,
}
: { kind: "unretained-evidence" as const, evidence },
),
);
}
/** Apply one approved full-state update using optimistic revision matching. */
updateUnknown(
input: UpdateUnknownInput,
@@ -96,7 +96,9 @@ describe("investigation record ownership", () => {
...unknownInput("Is the missing Evidence present?"),
supporting_evidence_ids: [`ev_${"f".repeat(64)}`],
};
expect(records.recordEvidenceWithUnknown(evidence, invalid).ok).toBe(false);
const refused = records.recordEvidenceWithUnknown(evidence, invalid);
if (refused.ok) throw new Error("Expected invalid Unknown references");
expect(refused.error.partialObservation).toBeUndefined();
expect(records.hasEvidence(evidence.evidence_id)).toBe(false);
expect(records.exportEvidenceBundle()).toEqual(createEvidenceBundle([]));
});
@@ -12,6 +12,7 @@ import {
type ResidualUnknown,
} from "../../domain/residualUnknown.js";
import { InvestigationRecords } from "./InvestigationRecords.js";
import { isImmutableJsonSnapshot } from "../../domain/immutableJson.js";
const provider = { id: "fixture", name: "Retention fixture", version: "1" };
const observe = (index: number, path = "/fixture") =>
@@ -57,6 +58,39 @@ const recordCost = (evidence: Evidence): number => {
return records.statistics().retainedRepresentationBytes;
};
it("returns only authenticated Evidence when a completed record exceeds retention capacity", () => {
const evidence = observe(7);
const records = new InvestigationRecords({ maximumRepresentationBytes: 0 });
const refused = records.recordEvidence(evidence);
if (refused.ok) throw new Error("Expected retention refusal");
expect(refused.error._tag).toBe("AnalysisResourceConstraintError");
expect(refused.error.partialObservation).toEqual({
kind: "unretained-evidence",
evidence,
});
expect(isImmutableJsonSnapshot(refused.error.partialObservation)).toBe(true);
expect(records.hasEvidence(evidence.evidence_id)).toBe(false);
const invalid = records.recordEvidence({
...evidence,
normalized_result: "tampered",
});
if (invalid.ok) throw new Error("Expected integrity refusal");
expect(invalid.error._tag).toBe("EvidenceIntegrityError");
expect(invalid.error.partialObservation).toBeUndefined();
const mutable = createEvidence(undefined, provider, {
operation: "health",
parameters: {},
result: { value: 1 },
});
const noHeadroom = new InvestigationRecords({
availableAuthenticationBytes: () => 0,
}).recordEvidence(mutable);
if (noHeadroom.ok)
throw new Error("Expected authentication headroom refusal");
expect(noHeadroom.error._tag).toBe("AnalysisResourceConstraintError");
expect(noHeadroom.error.partialObservation).toBeUndefined();
});
it("admits exact capacity, rejects cumulative growth atomically, and releases accounting on clear", () => {
const first = observe(1);
const second = observe(2);
@@ -1,5 +1,10 @@
import type { ElectronBoundarySummary } from "../../domain/javascript/javascriptApplicationAnalysis.js";
import { electronSemanticsState } from "./ElectronBoundaryAnalysis.js";
import {
electronSemanticsState,
createElectronIpcContext,
summarizeElectronBoundaries,
type ElectronIpcContext,
} from "./ElectronBoundaryAnalysis.js";
import { ELECTRON_IDENTITY_LIMITATION } from "../../domain/javascript/javascriptElectronMemberWrites.js";
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
import {
@@ -47,11 +52,6 @@ import {
} from "./JavaScriptArtifactGraphStructure.js";
import { addJavaScriptBundlerNodes } from "./JavaScriptArtifactGraphBundlers.js";
import { addElectronBoundaries } from "./ElectronBoundaryGraph.js";
import {
createElectronIpcContext,
summarizeElectronBoundaries,
type ElectronIpcContext,
} from "./ElectronBoundaryAnalysis.js";
/**
* Build a validated application graph with cooperative immutable ownership.
+5
View File
@@ -36,6 +36,7 @@ import { localPathStringSchema } from "../domain/localPath.js";
import type { ExecutableFormatHint } from "../domain/dosCom.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { registerDecompileDirectoryCommand } from "./decompileDirectoryCommand.js";
import { decompileAnalysisSelection } from "../contracts/officialToolContracts.js";
const decompileSelectorGuidance = `${addressOrProcedureGuidance} Alternatively, use --file-offset alone.`;
const namedAddressGuidance = requiredSelectorGuidance(
@@ -221,6 +222,7 @@ const registerDecompileCommand = (
.describe(
`Ghidra source-file byte offset; mapped internally to an analysis address. ${decompileSelectorGuidance}`,
),
analysis: decompileAnalysisSelection,
snapshot: snapshotSelectionOption,
...nativeOpenOptionFields,
}),
@@ -245,6 +247,9 @@ const registerDecompileCommand = (
: "file-offset",
"procedure_pseudo_code",
),
...(options.analysis === undefined
? {}
: { analysis: options.analysis }),
},
directAnalysisOptionsFromCli(logger, options),
).then(projectCliAnalysisResult),
+5
View File
@@ -44,6 +44,7 @@ export const registerDecompileDirectoryCommand = (
.describe(
"Continue a matching export. Never overwrites a function file.",
),
retryFailed: decompileDirectoryInputSchema.shape.retry_failed,
name: z
.string()
.min(1)
@@ -96,6 +97,7 @@ const decompileAllArguments = (options: {
readonly output: string;
readonly layout: "address" | "dwarf";
readonly resume: boolean;
readonly retryFailed?: "independent_fresh" | undefined;
readonly name?: string | undefined;
readonly namespace?: string | undefined;
readonly addressStart?: string | undefined;
@@ -105,6 +107,9 @@ const decompileAllArguments = (options: {
output_directory: absoluteCallerPath(options.output),
layout: options.layout,
resume: options.resume,
...(options.retryFailed === undefined
? {}
: { retry_failed: options.retryFailed }),
...(options.name === undefined ? {} : { name: options.name }),
...(options.namespace === undefined ? {} : { namespace: options.namespace }),
...(options.addressStart === undefined
+1 -1
View File
@@ -123,7 +123,7 @@ describe("CLI JSON path normalization", () => {
expect.objectContaining({
path: ["executable_path"],
code: "invalid_format",
message: expect.stringContaining("absolute local filesystem path"),
format: "regex",
}),
]);
});
+8
View File
@@ -3,6 +3,7 @@ import { z } from "zod";
import { digestSchema } from "../domain/digests.js";
import { absoluteLocalPathStringSchema } from "../domain/localPath.js";
import { analysisSourceDescriptorSchema } from "../domain/analysisSource.js";
import { pseudocodeAbsenceSchema } from "../domain/native/pseudocodeAbsence.js";
/** On-disk journal. Append-only so a crash keeps committed function files. */
export const DECOMPILE_DIRECTORY_JOURNAL_NAME = "decompile-journal.jsonl";
@@ -102,6 +103,12 @@ export const decompileDirectoryInputSchema = z.strictObject({
.describe(
"Continue a journal for the same target SHA-256, provider, analysis profile, and layout. Both digests must be known. Verified files and functions recorded as external are skipped, conflicting files are left unchanged, and functions not yet tried run before retries of recorded failures.",
),
retry_failed: z
.enum(["independent_fresh"])
.optional()
.describe(
"For a null result explicitly reported as a function failure, retain the failure and try an independent decompile when supported. It can use different analysis context; errors and omissions without that typed fact are not retried.",
),
name: z
.string()
.min(1)
@@ -267,6 +274,7 @@ export const decompileDirectoryFunctionSchema = z
}),
])
.optional(),
pseudocode_absence: pseudocodeAbsenceSchema.optional(),
body: bodySchema,
failure: z
.strictObject({
+1 -1
View File
@@ -54,7 +54,7 @@ export const ENHANCED_TOOL_CONTRACTS = [
),
enhanced(
"decompile_all",
"Write provider pseudocode for every analyzed function, or a filtered subset, into a caller-selected directory and return only a manifest. Each file records the provider function name, canonical address, inclusive body ranges when procedure_info supplies them, and that function's procedure_info and pseudocode limitations. Each function's index record keeps its complete limitations. Evidence keeps at most eight first-observed distinct per-function limitations within an encoded-text budget; the omitted count measures observations, not distinct warnings. The manifest reports counts, the output directory, the index path, at most 32 failure details within 2,048 JSON-encoded UTF-8 bytes (names can be omitted; full names remain in the index), provider identity, and the analysis-profile digest. Pseudocode stays in the files. An existing directory is refused unless resume is set and the journal matches the target SHA-256, provider, and analysis profile; verified files are skipped and conflicting files are left unchanged. layout dwarf reports that compilation-unit paths are unavailable and names files by address. A failure of one function does not stop the others. Functions procedure_info classifies as external are counted as external, get no file, and are not decompiled; a resume skips them. Cancellation leaves files already written. Per-function limits do not bound the whole call; use limit and resume to select work that fits the client deadline, including import, listing and file work.",
"Write provider pseudocode for every analyzed function, or a filtered subset, into a caller-selected directory and return only a manifest. Each file records the provider function name, canonical address, inclusive body ranges when procedure_info supplies them, and that function's procedure_info and pseudocode limitations. Each function's index record keeps its latest attempt and its complete limitations; the journal keeps earlier attempts and their limitations. Evidence keeps at most eight first-observed distinct per-function limitations within an encoded-text budget; the omitted count measures observations, not distinct warnings. The manifest reports counts, the output directory, the index path, at most 32 failure details within 2,048 JSON-encoded UTF-8 bytes (names can be omitted; full names remain in the index), provider identity, and the analysis-profile digest. Pseudocode stays in the files. An existing directory is refused unless resume is set and the journal matches the target SHA-256, provider, and analysis profile; verified files are skipped and conflicting files are left unchanged. layout dwarf reports that compilation-unit paths are unavailable and names files by address. Set retry_failed: independent_fresh to retry a null result explicitly reported as a function failure when supported; the failed attempt stays in the journal and the new attempt may use different analysis context. Errors and omissions without that typed fact are not retried. A failure of one function does not stop the others. Functions procedure_info classifies as external are counted as external, get no file, and are not decompiled; a resume skips them. Cancellation leaves files already written. Per-function limits do not bound the whole call; use limit and resume to select work that fits the client deadline, including import, listing and file work.",
enhancedInputSchemas.decompile_all,
),
enhanced(
+5 -1
View File
@@ -36,6 +36,10 @@ const generic = <Code extends string>(code: Code) =>
/** The canonical invalid-input projection, also used for facet-local refusals. */
export const analysisInputErrorProjectionSchema = generic("invalid_request");
/** The canonical resource refusal, including its reported limits and recovery. */
export const analysisResourceErrorProjectionSchema = generic(
"resource_constraint",
);
/** Stable discriminated schema shared by every CLI and MCP error surface. */
export const analysisErrorProjectionSchema = z.discriminatedUnion("code", [
@@ -47,7 +51,7 @@ export const analysisErrorProjectionSchema = z.discriminatedUnion("code", [
generic("unsupported_target"),
generic("provider_unavailable"),
generic("provider_timeout"),
generic("resource_constraint"),
analysisResourceErrorProjectionSchema,
generic("cancelled"),
z
.object({
+14 -2
View File
@@ -23,6 +23,13 @@ import { nativeSearchInput } from "../domain/native/nativeSearch.js";
const hopperRegexDeadline =
"Regex dialect depends on the provider: Ghidra uses Java Pattern; Kuna, IDA and Hopper use ECMAScript Unicode. See binary_session for selected-provider limits. Hopper regex mode uses a cancellable worker. The matching deadline is REA_REGEX_MATCHING_DEADLINE_MS (a positive decimal integer of milliseconds from 1 to 2147483647, default 5000). Unset keeps the default. A deadline failure names the effective milliseconds, source (default or setting), the default, and the maximum, and leaves the API usable. The worker old-generation request is REA_REGEX_WORKER_HEAP_REQUEST_MIB (a decimal integer of mebibytes from 7 to 16384, default 128). Unset keeps the default. A positive Node --max-old-space-size, including NODE_OPTIONS, overrides that request and is not limited by the setting's floor or maximum. A heap failure names the effective request, its source (default or setting), the default, the floor, and the maximum.";
export const decompileAnalysisSelection = z
.enum(["independent_fresh"])
.optional()
.describe(
"Request a new independent decompile, bypassing project and snapshot results. Use only when the routed provider's procedure_pseudo_code capability declares independent_decompile; otherwise the request is refused. This may use different analysis context.",
);
const official = <
Name extends keyof typeof officialOutputSchemas,
Schema extends z.ZodObject,
@@ -207,8 +214,13 @@ export const OFFICIAL_TOOL_CONTRACTS = [
),
official(
"procedure_pseudo_code",
"Decompile one analyzed procedure by symbol name or provider-normalized address. Ghidra also accepts file-offset:0x... selectors and falls back to a unique source-file mapping when a 0x-prefixed number resolves to no procedure; existing analysis addresses take precedence and ambiguous mappings fail explicitly. Returns provider-specific pseudocode, never original source or cross-provider text equivalence, and may return null; request procedure_assembly when instruction precision matters. Ghidra no-return assumptions may cut the analyzed body; Evidence limitations report observed terminal calls and available callee identities. Select detail summary to retain complete Evidence and receive text availability and counts; inspect_analysis_view summary or native/pseudocode pages read the original observation without decompiling again. Complete remains the default.",
z.object({ procedure, document, detail: analysisDetailSchema }),
"Decompile one analyzed procedure by symbol name or provider-normalized address. To request a fresh independent attempt, use analysis: independent_fresh when binary_session reports independent_decompile for this provider operation; otherwise it is refused. The independent result may have different analysis context. Ghidra also accepts file-offset:0x... selectors and falls back to a unique source-file mapping when a 0x-prefixed number resolves to no procedure; existing analysis addresses take precedence and ambiguous mappings fail explicitly. Returns provider-specific pseudocode, never original source or cross-provider text equivalence, and may return null. When supplied, raw_result.pseudocode_absence distinguishes provider-failure from not-in-export; absence of that fact means unknown. Request procedure_assembly when instruction precision matters. Ghidra no-return assumptions may cut the analyzed body; Evidence limitations report observed terminal calls and available callee identities. Select detail summary to retain complete Evidence and receive text availability and counts; inspect_analysis_view summary or native/pseudocode pages read the original observation without decompiling again. Complete remains the default.",
z.object({
procedure,
document,
analysis: decompileAnalysisSelection,
detail: analysisDetailSchema,
}),
{ evidenceParameters: "omit-detail" },
),
official(
+1 -1
View File
@@ -187,7 +187,7 @@ export const SESSION_TOOL_CONTRACTS = [
{
...session(
"inspect_address_context",
"Inspect one explicit reproducible address for its analyzed name, containing procedure, regular and inline comments, and matching bookmarks. Each unsupported facet returns a typed unavailable outcome. A facet rejected for invalid input retains its canonical error and issues alongside any successful facets. Use xrefs, assembly, or pseudocode for those deeper views.",
"Inspect one explicit reproducible address for its analyzed name, containing procedure, regular and inline comments, and matching bookmarks. Each unsupported facet returns a typed unavailable outcome. A facet rejected for invalid input or a resource constraint retains its canonical error and recovery alongside any successful facets. Use xrefs, assembly, or pseudocode for those deeper views.",
addressContextInputSchema,
),
requiresActiveTarget: true,
+7 -2
View File
@@ -101,6 +101,7 @@ import { decompileDirectoryManifestSchema } from "./decompileDirectory.js";
import {
analysisErrorProjectionSchema,
analysisInputErrorProjectionSchema,
analysisResourceErrorProjectionSchema,
} from "./errorSchemas.js";
import { relationshipCoverageSchema } from "../domain/relationshipCoverage.js";
import { nativeDispatchMetadataResultSchema } from "../domain/native/objcSwiftMetadata.js";
@@ -122,13 +123,17 @@ import {
import { processCaptureSchema } from "../domain/process/processCapture.js";
import { processScenarioPlanSchema } from "../domain/process/processScenario.js";
import { electronFuseReportSchema } from "../domain/electron/electronFuses.js";
const contextFacetErrorSchema = z.discriminatedUnion("code", [
analysisInputErrorProjectionSchema,
analysisResourceErrorProjectionSchema,
]);
const contextFacetSchema = z.discriminatedUnion("state", [
z.object({ state: z.literal("available"), value: jsonValueSchema }),
z.object({
state: z.literal("unavailable"),
reason: z.string(),
remediation: z.string(),
error: analysisInputErrorProjectionSchema.optional(),
error: contextFacetErrorSchema.optional(),
}),
]);
const bookmarkFacetSchema = z.discriminatedUnion("state", [
@@ -140,7 +145,7 @@ const bookmarkFacetSchema = z.discriminatedUnion("state", [
state: z.literal("unavailable"),
reason: z.string(),
remediation: z.string(),
error: analysisInputErrorProjectionSchema.optional(),
error: contextFacetErrorSchema.optional(),
}),
]);
@@ -65,6 +65,7 @@ const providerCapabilityFacts = {
requires_root: z.boolean(),
}),
limitations: z.array(z.string()),
independent_decompile: z.literal(true).exactOptional(),
};
export const providerCapability = z.discriminatedUnion("available", [
@@ -100,6 +101,7 @@ const providerCapabilityGroupFacts = {
operations: z.array(z.string().min(1)).min(1),
effects: z.array(providerCapabilityFacts.effects.keyof()),
limitations: z.array(z.string()),
independent_decompile: providerCapabilityFacts.independent_decompile,
};
const providerCapabilityGroup = z.discriminatedUnion("available", [
@@ -138,6 +140,9 @@ export const expandProviderCapabilities = (
group.operations.map((operation) => ({
...providerCapability.parse({
operation,
...(group.independent_decompile === undefined
? {}
: { independent_decompile: group.independent_decompile }),
available: group.available,
reason: group.reason,
availability_code: group.availability_code,
+2
View File
@@ -402,6 +402,8 @@ import type { WebArtifactAcquisition } from "./webArtifactAcquisition.js";
/** Provider-neutral evidence collected before a typed analysis failure. */
export type AnalysisPartialObservation =
| { readonly kind: "unretained-evidence"; readonly evidence: Evidence }
| { readonly kind: "retained-evidence"; readonly evidence_id: string }
| GraphqlContract
| PlaywrightTrace
| SavedAnalysisSnapshotPartialObservation
+80 -6
View File
@@ -67,13 +67,76 @@ import { invocationSettingLimit } from "./analysisSettingDefinitions.js";
export const projectAnalysisError = (
error: AnalysisError,
callerLifetime: AnalysisCallerLifetime = "continuing",
): AnalysisErrorProjection => projectError(error, true, callerLifetime);
): AnalysisErrorProjection =>
projectError(
error,
{ capturedOutput: true, partialObservation: true },
callerLifetime,
);
/** Cleanup history references output retained by its enclosing error owner. */
export const projectAnalysisErrorWithoutCapturedOutput = (
error: AnalysisError,
callerLifetime: AnalysisCallerLifetime = "continuing",
): AnalysisErrorProjection => projectError(error, false, callerLifetime);
): AnalysisErrorProjection =>
projectError(
error,
{ capturedOutput: false, partialObservation: true },
callerLifetime,
);
/** Bounded fallback when the completed observation itself cannot be delivered. */
export const projectAnalysisErrorWithoutPartialObservation = (
error: AnalysisError,
callerLifetime: AnalysisCallerLifetime = "continuing",
): AnalysisErrorProjection =>
projectError(
error,
{ capturedOutput: true, partialObservation: false },
callerLifetime,
);
/** Omit only explicitly unretained Evidence when a complete reply cannot fit. */
export const omitUnretainedEvidenceFromError = (
projected: AnalysisErrorProjection,
limit: {
readonly constraint: "receive-buffer" | "string-length";
readonly bytesAtLeast: number;
},
): AnalysisErrorProjection | undefined => {
const partial = projected.details?.partial_observation;
if (
typeof partial !== "object" ||
partial === null ||
Array.isArray(partial) ||
partial.kind !== "unretained-evidence"
)
return undefined;
const { partial_observation: _partial, ...details } = projected.details!;
const evidence = partial.evidence;
const evidenceId =
typeof evidence === "object" &&
evidence !== null &&
!Array.isArray(evidence)
? evidence.evidence_id
: undefined;
return {
...projected,
remediation: {
action: `${projected.remediation.action} The completed observation was not retained and could not fit within MCP delivery limits.`,
},
details: {
...details,
partial_observation_omitted: {
reason: "mcp-response-limit",
retention: "unretained",
constraint: limit.constraint,
original_response_bytes_at_least: limit.bytesAtLeast,
...(typeof evidenceId === "string" ? { evidence_id: evidenceId } : {}),
},
},
};
};
/** A child failure embedded in a composed error's diagnostics. */
export interface AnalysisChildFailure {
@@ -95,7 +158,15 @@ export const childFailureDiagnostics = (
const project = (
{ error, capturedOutput }: AnalysisChildFailure,
callerLifetime: AnalysisCallerLifetime,
) => projectError(error, capturedOutput !== "omit", callerLifetime);
) =>
projectError(
error,
{
capturedOutput: capturedOutput !== "omit",
partialObservation: true,
},
callerLifetime,
);
const continuing = {
...diagnostics,
...Object.fromEntries(
@@ -137,7 +208,10 @@ const projectionDependsOnCallerLifetime = (error: AnalysisError): boolean => {
const projectError = (
error: AnalysisError,
includeCapturedOutput: boolean,
options: {
readonly capturedOutput: boolean;
readonly partialObservation: boolean;
},
callerLifetime: AnalysisCallerLifetime,
): AnalysisErrorProjection => {
const composed = error;
@@ -218,14 +292,14 @@ const projectError = (
primaryDetails?.execution_failure ?? underlyingCode,
}
: {}),
...(!includeCapturedOutput || capturedOutput === undefined
...(!options.capturedOutput || capturedOutput === undefined
? {}
: {
captured_output: {
...capturedOutput,
},
}),
...(partialObservation === undefined
...(!options.partialObservation || partialObservation === undefined
? {}
: {
partial_observation: isImmutableJsonSnapshot(partialObservation)
@@ -30,6 +30,10 @@ import {
} from "./nativeObservationSchemas.js";
import { projectNativeGraphRows } from "./nativeGraphView.js";
/** What an address page leaves in the complete parent, when recorded. */
export const XREFS_ADDRESS_PAGE_LIMITATION =
"This page contains source addresses only, not reference kinds or provenance. Retrieve or export the complete parent Evidence for any additional provider details it recorded.";
const readAddresses = createAnalysisViewResultReader(nativeAddressListSchema);
const readAddress = createAnalysisViewResultReader(
nativeObservationSourceSchemas.procedure_address,
@@ -347,6 +351,9 @@ export const projectNativeObservationView = (
const { total, ...payload } = page;
const observation = {
...common,
...(source.value.operation === "xrefs"
? { unknowns: [...common.unknowns, XREFS_ADDRESS_PAGE_LIMITATION] }
: {}),
...payload,
summary: { observations: total },
coverage: pageViewCoverage(selection.offset, page.items.rows.length, total),
+3 -142
View File
@@ -1,3 +1,4 @@
import { PYTHON_JSON_READER } from "./appExportPythonJson.js";
import { PYTHON_XML_READER } from "./appExportPythonXml.js";
import { PYTHON_ZIP_HELPERS } from "./appExportPythonZip.js";
@@ -12,6 +13,7 @@ export const PYTHON_READY_CHECK = `
export const PYTHON_IMPORTS = `from __future__ import annotations
from array import array
import contextlib
import copy
import hashlib
@@ -57,11 +59,6 @@ MAX_PARSED_BYTES = 4 * 1024 * 1024
PARSED_TOO_LARGE = "larger than the 4 MiB limit for JSON and key file views"
MAX_JSON_VIEW_DEPTH = 500
JSON_TOO_DEEP = f"nested deeper than the {MAX_JSON_VIEW_DEPTH} levels a JSON view keeps"
#: A JSON token after any whitespace, by group: an opening, a closing, a comma, a colon, a string's quote, or a number
#: or literal. Strings are read by json's scanner, since a pattern repeating per escape holds memory for each.
JSON_TOKEN = re.compile(r'[ \t\n\r]*(?:([\[{])|([\]}])|(,)|(:)|(")|(-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?'
r"|true|false|null))")
JSON_SPACE = re.compile(r"[ \t\n\r]*")
def _respelled(text: str, home: Path) -> str:
@@ -254,143 +251,7 @@ def _keyfile(text: str) -> dict:
${PYTHON_XML_READER}
def _not_json(constant: str):
raise ValueError(f"{constant} is not JSON")
class _Inexact(Exception):
"""A JSON number that REA's state view refuses, since reading it would change it."""
def _digits(text: str) -> str:
return re.split(r"[eE]", text.lstrip("-"))[0].replace(".", "").strip("0") or "0"
def _js_string(value: float) -> str:
"""How JavaScript's String() writes a finite double: an integer below 1e21 in full, any other in shortest form."""
return str(int(value)) if value.is_integer() and abs(value) < 1e21 else repr(value)
def _safe_number(text: str) -> bool:
"""lossless-json's isSafeNumber: an integer spelling within 2^53, or a float spelling that keeps its digits."""
if re.fullmatch(r"-?[0-9]+", text):
return len(text.lstrip("-")) <= 16 and abs(int(text)) <= MAX_SAFE_INTEGER
value = float(text)
if not math.isfinite(value):
return False
written = _js_string(value)
return written == text or _digits(written) == _digits(text)
def _exact_number(text: str):
"""A JSON number read only when reading it changes nothing, as REA's state view rules: its spelling and its
written-back form both safe, and not -0."""
value = float(text)
if (not _safe_number(text) or not math.isfinite(value) or not _safe_number(_js_string(value))
or (value == 0 and math.copysign(1, value) < 0)):
raise _Inexact(text)
return int(text) if re.fullmatch(r"-?[0-9]+", text) else value
_INEXACT = object()
_LEVEL_END = object()
def _number_or_mark(spelling: str):
"""A JSON number read exactly, or a mark for one REA's state view refuses."""
try:
return _exact_number(spelling)
except _Inexact:
return _INEXACT
def _holds_inexact(value) -> bool:
"""Whether a parsed JSON value still holds a marked number once duplicate keys kept their last value; one
iterator per nesting level, so no child list is copied."""
levels = [iter((value,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif item is _INEXACT:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _deeper_than(root, limit: int) -> bool:
"""Whether a parsed JSON value nests deeper than a limit, as REA checks it: a value's depth is the number of
iterators open above it, one per nesting level, so no child list is copied."""
levels = [iter((root,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif len(levels) - 1 > limit:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _is_json(text: str) -> bool:
"""Whether text is one JSON value as REA's state view reads it, checked a token at a time without building it or
recursing: it holds a byte per open container and one string at a time, so text too deep for Python's parser or
too large to parse into a view is still told apart from other text."""
containers, expect, empty, at = bytearray(), "value", False, 0
while True:
token = JSON_TOKEN.match(text, at)
if token is None:
break
kind, at = token.lastindex, token.end()
if kind == 5:
try:
at = json.decoder.scanstring(text, at)[1]
except ValueError:
return False
if kind == 1 and expect == "value":
containers.append(ord(text[at - 1]))
expect, empty = ("value" if text[at - 1] == "[" else "key"), True
# A closing bracket's code is its opening's plus two.
elif kind == 2 and (expect == "after" or empty) and containers and containers[-1] == ord(text[at - 1]) - 2:
containers.pop()
expect, empty = "after", False
elif kind == 3 and expect == "after" and containers:
expect = "value" if containers[-1] == ord("[") else "key"
elif kind == 4 and expect == "colon":
expect = "value"
elif kind >= 5 and (expect == "value" or (kind == 5 and expect == "key")):
expect, empty = ("colon" if expect == "key" else "after"), False
else:
return False
return expect == "after" and not containers and JSON_SPACE.fullmatch(text, at) is not None
def _json_view(text: str, size: int):
"""A JSON text's format and view, or why it has none, as REA reads it; None when it is not JSON. Text over the
parsed-view limit, or nested past Python's recursion limit, is told apart without building its value."""
if size > MAX_PARSED_BYTES:
return {"format": "json", "view": None, "error": PARSED_TOO_LARGE} if _is_json(text) else None
# The whole text must parse, and duplicate keys settle, before an inexact number makes it JSON with an error.
try:
view = json.loads(text, parse_constant=_not_json, parse_float=_number_or_mark, parse_int=_number_or_mark)
except RecursionError:
# Nesting too deep for Python to parse is far deeper than REA's view keeps, once the rest of it is JSON too.
return {"format": "json", "view": None, "error": JSON_TOO_DEEP} if _is_json(text) else None
except ValueError:
return None
if _holds_inexact(view):
return {"format": "json", "view": None,
"error": "a number in it cannot be read exactly, such as an integer above 2^53"}
if _deeper_than(view, MAX_JSON_VIEW_DEPTH):
return {"format": "json", "view": None, "error": JSON_TOO_DEEP}
return {"format": "json", "view": view, "truncated": False}
${PYTHON_JSON_READER}
def _text_view(name: str, text: str, size: int) -> dict:
"""The format REA's state view detects for decoded text of size bytes, and its view; a JSON or key file over the
+231
View File
@@ -0,0 +1,231 @@
/** JSON parsing and bounded retained-view checks for exported desktop environments. */
export const PYTHON_JSON_READER = String.raw`
#: A JSON token after any whitespace, by group: an opening, a closing, a comma, a colon, a string's quote, or a number
#: or literal. Strings are read by json's scanner, since a pattern repeating per escape holds memory for each.
JSON_TOKEN = re.compile(r'[ \t\n\r]*(?:([\[{])|([\]}])|(,)|(:)|(")|(-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?'
r"|true|false|null))")
JSON_SPACE = re.compile(r"[ \t\n\r]*")
class _Inexact(Exception):
"""A JSON number that REA's state view refuses, since reading it would change it."""
def _digits(text: str) -> str:
return re.split(r"[eE]", text.lstrip("-"))[0].replace(".", "").strip("0") or "0"
def _js_string(value: float) -> str:
"""How JavaScript's String() writes a finite double: an integer below 1e21 in full, any other in shortest form."""
return str(int(value)) if value.is_integer() and abs(value) < 1e21 else repr(value)
def _safe_number(text: str) -> bool:
"""lossless-json's isSafeNumber: an integer spelling within 2^53, or a float spelling that keeps its digits."""
if re.fullmatch(r"-?[0-9]+", text):
return len(text.lstrip("-")) <= 16 and abs(int(text)) <= MAX_SAFE_INTEGER
value = float(text)
if not math.isfinite(value):
return False
written = _js_string(value)
return written == text or _digits(written) == _digits(text)
def _exact_number(text: str):
"""A JSON number read only when reading it changes nothing, as REA's state view rules: its spelling and its
written-back form both safe, and not -0."""
value = float(text)
if (not _safe_number(text) or not math.isfinite(value) or not _safe_number(_js_string(value))
or (value == 0 and math.copysign(1, value) < 0)):
raise _Inexact(text)
return int(text) if re.fullmatch(r"-?[0-9]+", text) else value
_INEXACT = object()
_DEEP = object()
_DEEP_INEXACT = object()
_LEVEL_END = object()
_INVALID_JSON = object()
JSON_ATOMS = {"true": True, "false": False, "null": None}
def _number_or_mark(spelling: str):
"""A JSON number read exactly, or a mark for one REA's state view refuses."""
try:
return _exact_number(spelling)
except _Inexact:
return _INEXACT
def _holds_inexact(value) -> bool:
"""Whether a parsed JSON value still holds a marked number once duplicate keys kept their last value; one
iterator per nesting level, so no child list is copied."""
levels = [iter((value,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif item is _INEXACT or item is _DEEP_INEXACT:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _deeper_than(root, limit: int) -> bool:
"""Whether a parsed JSON value nests deeper than a limit, as REA checks it: a value's depth is the number of
iterators open above it, one per nesting level, so no child list is copied."""
levels = [iter((root,))]
while levels:
item = next(levels[-1], _LEVEL_END)
if item is _LEVEL_END:
levels.pop()
elif item is _DEEP or item is _DEEP_INEXACT:
return True
elif len(levels) - 1 > limit:
return True
elif isinstance(item, dict):
levels.append(iter(item.values()))
elif isinstance(item, list):
levels.append(iter(item))
return False
def _read_json(text: str, build: bool = True):
"""One JSON value, or _INVALID_JSON, read without recursion so duplicate keys settle before depth is checked.
Deep containers are summarized rather than built; exact-number marks still follow last-key-wins semantics.
With build=False, only syntax is checked: one byte per open container and one decoded string at a time."""
containers, expect, empty, at = bytearray(), "value", False, 0
values, keys, root = [], [], _INVALID_JSON
deep_bad, deep_key_at, bad_keys = bytearray(), array("I"), {}
def keep(value) -> None:
nonlocal root
if not values:
root = value
elif containers[-1] == ord("["):
values[-1].append(value)
else:
values[-1][keys[-1]] = value
def record_deep(bad: bool) -> None:
"""Summarize a finished child; keep only object keys with live inexact values."""
depth = len(deep_bad) - 1
if containers[-1] == ord("["):
deep_bad[-1] |= 2 | bad
return
marked = bad_keys.get(depth)
if bad or marked is not None:
key, _ = json.decoder.scanstring(text, deep_key_at[-1])
if bad:
if marked is None:
marked = key
elif isinstance(marked, str):
if marked != key:
marked = {marked, key}
else:
marked.add(key)
else:
if isinstance(marked, str):
if marked == key:
marked = None
else:
marked.discard(key)
if len(marked) == 1:
marked = next(iter(marked))
if marked is None:
bad_keys.pop(depth, None)
else:
bad_keys[depth] = marked
deep_bad[-1] = 2 | (marked is not None)
else:
deep_bad[-1] |= 2
while True:
token = JSON_TOKEN.match(text, at)
if token is None:
break
kind, at = token.lastindex, token.end()
if kind == 5:
string_at = at
try:
value, at = json.decoder.scanstring(text, at)
except ValueError:
return _INVALID_JSON
if kind == 1 and expect == "value":
if build:
if deep_bad or len(containers) >= MAX_JSON_VIEW_DEPTH:
deep_bad.append(0)
deep_key_at.append(0)
else:
value = [] if text[at - 1] == "[" else {}
keep(value)
values.append(value)
keys.append(None)
containers.append(ord(text[at - 1]))
expect, empty = ("value" if text[at - 1] == "[" else "key"), True
# A closing bracket's code is its opening's plus two.
elif kind == 2 and (expect == "after" or empty) and containers and containers[-1] == ord(text[at - 1]) - 2:
opening = containers.pop()
if build:
if deep_bad:
state = deep_bad.pop()
bad = bool(state & 1)
deep_key_at.pop()
bad_keys.pop(len(deep_bad), None)
if deep_bad:
record_deep(bad)
else:
keep((_DEEP_INEXACT if bad else _DEEP) if state & 2 else ([] if opening == ord("[") else {}))
else:
values.pop()
keys.pop()
expect, empty = "after", False
elif kind == 3 and expect == "after" and containers:
expect = "value" if containers[-1] == ord("[") else "key"
elif kind == 4 and expect == "colon":
expect = "value"
elif kind >= 5 and (expect == "value" or (kind == 5 and expect == "key")):
if build:
if expect == "key":
if deep_bad:
deep_key_at[-1] = string_at
else:
keys[-1] = value
else:
if kind == 6:
value = JSON_ATOMS.get(token[6], _INVALID_JSON)
if value is _INVALID_JSON:
value = _number_or_mark(token[6])
if deep_bad:
record_deep(value is _INEXACT)
else:
keep(value)
expect, empty = ("colon" if expect == "key" else "after"), False
else:
return _INVALID_JSON
if expect != "after" or containers or JSON_SPACE.fullmatch(text, at) is None:
return _INVALID_JSON
return root if build else True
def _json_view(text: str, size: int):
"""A JSON text's format and view, or why it has none, as REA reads it; None when it is not JSON. Text over the
parsed-view limit is told apart without building its value."""
if size > MAX_PARSED_BYTES:
valid = _read_json(text, build=False) is not _INVALID_JSON
return {"format": "json", "view": None, "error": PARSED_TOO_LARGE} if valid else None
# The whole text must parse, and duplicate keys settle, before an inexact number makes it JSON with an error.
view = _read_json(text)
if view is _INVALID_JSON:
return None
if _holds_inexact(view):
return {"format": "json", "view": None,
"error": "a number in it cannot be read exactly, such as an integer above 2^53"}
if _deeper_than(view, MAX_JSON_VIEW_DEPTH):
return {"format": "json", "view": None, "error": JSON_TOO_DEEP}
return {"format": "json", "view": view, "truncated": False}
`;
+20 -1
View File
@@ -21,11 +21,20 @@ const PATHS = [
];
it("matches the host path module for absolute local paths", () => {
const checked: string[] = [];
for (const path of PATHS) {
checked.push(path);
expect(absoluteLocalPathStringSchema.safeParse(path).success).toBe(
isAbsolute(path),
);
}
expect(checked).toEqual(
expect.arrayContaining([
"/absolute/path",
"relative/path",
"file:///absolute/path",
]),
);
expect(absoluteLocalPathStringSchema.safeParse("").success).toBe(false);
expect(
absoluteLocalPathStringSchema.safeParse("/absolute/with\0nul").success,
@@ -40,8 +49,18 @@ it("keeps the Windows absolute-path pattern valid in every client mode", () => {
() => new RegExp(windowsAbsolutePathPattern.source, flags),
flags,
).not.toThrow();
for (const path of PATHS)
const checked: string[] = [];
for (const path of PATHS) {
checked.push(path);
expect(windowsAbsolutePathPattern.test(path), path).toBe(
win32.isAbsolute(path),
);
}
expect(checked).toEqual(
expect.arrayContaining([
"C:\\absolute\\path",
"C:relative/path",
"/absolute/path",
]),
);
});
+12
View File
@@ -7,6 +7,18 @@ export const literalFeatureReferenceSchema = z.object({
target_address: z.string(),
source_address: z.string(),
containing_procedure: traceContainingProcedureSchema,
source_classifications: z
.array(
z.strictObject({
kind: z.literal("stored_pointer"),
provenance: z.string(),
width: z.int().positive(),
}),
)
.exactOptional()
.describe(
"Provider-reported source classifications; omission means none were supplied. The list need not exhaust all kinds.",
),
});
/** Complete result of a native literal feature trace. */
@@ -125,6 +125,18 @@ export const nativeCodeLocationSchema = z.strictObject({
module: z.string().nullable(),
module_path: z.string().nullable(),
symbol: z.string().nullable(),
/** LLDB line-table attribution; does not verify source file contents. */
source_location: z
.strictObject({
file_path: z.string().min(1).nullable(),
line: z.number().int().positive().nullable(),
column: z.number().int().positive().nullable(),
})
.nullable()
.optional()
.describe(
"Compiler or debugger source attribution only; REA does not read or verify the named source-file bytes.",
),
});
/** One observed entry into a requested function or method. */
+15
View File
@@ -0,0 +1,15 @@
import { z } from "zod";
/** Provider-observed reasons for a null pseudocode result. */
export const pseudocodeAbsenceSchema = z.discriminatedUnion("kind", [
z.strictObject({
kind: z.literal("provider-failure"),
reason: z.string().min(1),
}),
z.strictObject({
kind: z.literal("not-in-export"),
reason: z.string().min(1),
}),
]);
export type PseudocodeAbsence = z.infer<typeof pseudocodeAbsenceSchema>;
+1 -23
View File
@@ -341,28 +341,6 @@ export const copyAndHash = async (
};
};
/**
* The same run takes as long again, so only a longer deadline helps. Only a
* session that outlives this request can also read the later project export.
*/
export const kunaTimeoutRecovery = (
setting: string,
subcommand: string,
): {
readonly remediationAction: string;
readonly continuingCallerGuidance?: { readonly remediationAction: string };
} => {
const remediationAction = `Raise ${setting} in the environment that starts REA, then restart REA; it reads Kuna settings when it starts.`;
return subcommand === "decompile"
? {
remediationAction,
continuingCallerGuidance: {
remediationAction: `${remediationAction} Within a session, pseudocode is also read from the project export once it finishes.`,
},
}
: { remediationAction };
};
/** Map an owned-command failure to its typed error, keeping unproven cleanup owned. */
const commandFailure = (
cause: unknown,
@@ -413,7 +391,7 @@ const commandFailure = (
primary = new AnalysisTimeoutError(operation, ms, {
...details,
userMessage: `kuna ${subcommand} did not finish within ${setting} (${String(ms)} ms).`,
...kunaTimeoutRecovery(setting, subcommand),
remediationAction: `Raise ${setting} in the environment that starts REA, then restart REA; it reads Kuna settings when it starts.`,
});
}
// The same input overflows the same fixed buffer; waiting cannot help.
+19 -6
View File
@@ -35,6 +35,7 @@ import {
type KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
import type { PseudocodeAbsence } from "../domain/native/pseudocodeAbsence.js";
import { KUNA_FUNCTION_TEXT_BYTES } from "./KunaLimits.js";
import { kunaUnavailable } from "./KunaUnavailable.js";
@@ -48,6 +49,8 @@ export interface KunaProjectFileSpan {
export interface KunaProjectText {
readonly result: string | null;
readonly unavailableReason?: string;
readonly pseudocodeAbsence?: PseudocodeAbsence;
readonly files?: readonly KunaProjectFileSpan[];
readonly limitations: readonly string[];
}
@@ -314,13 +317,16 @@ export class KunaProjectReader {
this.project.functions.map((block) => [block.address, block]),
);
const block = this.#blocks.get(address);
if (block === undefined)
if (block === undefined) {
const reason =
"Kuna's project export contains no decompiled block for this inventory entry. Import slots may have no body; other omissions are not classified.";
return ok({
result: null,
limitations: [
"Kuna's export decompiles code entries only; this inventory entry, such as an import slot, has no body.",
],
unavailableReason: reason,
pseudocodeAbsence: { kind: "not-in-export", reason },
limitations: [reason],
});
}
const files = [
{
facet: "pseudocode" as const,
@@ -329,12 +335,19 @@ export class KunaProjectReader {
byteLength: block.cLength,
},
];
if (block.error !== null)
if (block.error !== null) {
const reason = `Kuna could not decompile this function: ${block.error}`;
return ok({
result: null,
unavailableReason: reason,
pseudocodeAbsence: { kind: "provider-failure", reason },
files,
limitations: [`Kuna could not decompile this function: ${block.error}`],
limitations: [
reason,
"To try an independent decompile, call procedure_pseudo_code with analysis: independent_fresh, or use rea decompile --analysis independent_fresh. It may use different analysis context and does not replace this observation.",
],
});
}
const text = await readSpan(
operation,
join(this.project.directory, this.project.manifest.files.c.path),
+6 -1
View File
@@ -273,7 +273,12 @@ export class KunaProvider implements AnalysisProviderCandidate {
bodies: false,
edge_kind: "call",
},
xrefs: { command: "xrefs", direction: "to", exact_address: true },
xrefs: {
command: "xrefs",
direction: "to",
exact_address: true,
stored_pointers: true,
},
strings: {
command: "strings",
encoding: "all",
+3
View File
@@ -104,6 +104,9 @@ export const kunaCapabilities = (
KUNA_OPERATIONS.map((operation) => ({
provider: KUNA_PROVIDER_IDENTITY,
operation,
...(operation === "procedure_pseudo_code"
? { independentDecompile: true as const }
: {}),
...(availability.status === "available"
? ({ available: true, reason: null } as const)
: ({
+77 -1
View File
@@ -224,6 +224,32 @@ const xrefRow = z.object({
to_function: xrefFunction.nullable(),
instruction: z.string(),
});
const xrefPointer = z.object({
kind: z.literal("pointer"),
from_address: address,
from_address_hex: address,
from_name: z.string().nullable(),
to_address: address,
to_address_hex: address,
to_name: z.string().nullable(),
width: z.int().positive(),
source: z.string().min(1),
});
const xrefWalkCoverage = z.object({
decoded: z.boolean(),
section: z.string().nullable(),
code_section: z.boolean().nullable(),
executable: z.boolean().nullable(),
unwalked: z.array(
z.object({
name: z.string().nullable(),
address,
address_hex: address,
section: z.string(),
}),
),
notes: z.array(z.string()),
});
const xrefsDocument = z.object({
binary: z.string(),
direction: z.literal("to"),
@@ -235,6 +261,10 @@ const xrefsDocument = z.object({
aliases: z.array(xrefFunction),
}),
xrefs: z.array(z.unknown()),
coverage: xrefWalkCoverage.optional(),
data_refs_coverage: z.string().optional(),
data_refs_count: count.optional(),
data_refs: z.array(xrefPointer).optional(),
});
export interface KunaIncomingReferences {
@@ -243,6 +273,15 @@ export interface KunaIncomingReferences {
readonly bytes: number;
readonly rows: number;
readonly excludedAliasRows: number;
readonly targetAliases: readonly {
readonly name: string;
readonly address: string;
}[];
readonly walkCoverage: z.infer<typeof xrefWalkCoverage> | null;
readonly pointerCoverage: string | null;
readonly pointers: Array<
Omit<z.infer<typeof xrefPointer>, "from_address_hex" | "to_address_hex">
> | null;
}
/** Code from incoming-reference admission, plus the address field when one failed. */
@@ -275,8 +314,34 @@ export const parseKunaIncomingReferences = (
throw new KunaDocumentError("xref_alias_hex");
aliases.add(alias.address);
}
if (
(document.data_refs === undefined) !==
(document.data_refs_count === undefined)
)
throw new KunaDocumentError("xref_count");
if (
document.data_refs !== undefined &&
document.data_refs.length !== document.data_refs_count
)
throw new KunaDocumentError("xref_count");
for (const item of document.coverage?.unwalked ?? [])
if (item.address !== item.address_hex)
throw new KunaDocumentError("xref_endpoints");
for (const pointer of document.data_refs ?? []) {
if (
pointer.from_address !== pointer.from_address_hex ||
pointer.to_address !== pointer.to_address_hex
)
throw new KunaDocumentError("xref_endpoints");
if (!aliases.has(pointer.to_address))
throw new KunaDocumentError("xref_target");
}
const addresses = new Set<string>();
let excludedAliasRows = 0;
for (const pointer of document.data_refs ?? []) {
if (pointer.to_address === target) addresses.add(pointer.from_address);
else excludedAliasRows += 1;
}
for (const raw of document.xrefs) {
const row = xrefRow.parse(raw);
if (
@@ -296,8 +361,19 @@ export const parseKunaIncomingReferences = (
}
return ok({
addresses: [...addresses].sort(compareKunaAddresses),
rows: document.xrefs.length,
rows: document.xrefs.length + (document.data_refs?.length ?? 0),
excludedAliasRows,
targetAliases: document.target.aliases.map(({ name, address }) => ({
name,
address,
})),
walkCoverage: document.coverage ?? null,
pointerCoverage: document.data_refs_coverage ?? null,
pointers:
document.data_refs?.map(
({ from_address_hex: _from, to_address_hex: _to, ...pointer }) =>
pointer,
) ?? null,
bytes: Buffer.byteLength(text),
sha256: createHash("sha256").update(text).digest("hex"),
});
+118 -5
View File
@@ -108,6 +108,7 @@ import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import type { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import { kunaQueryFailureForCaller } from "./KunaQueryErrors.js";
import type { PseudocodeAbsence } from "../domain/native/pseudocodeAbsence.js";
/** How often a request waiting on the project export reports that it still waits. */
const WAIT_PROGRESS_INTERVAL_MS = 10_000;
@@ -173,13 +174,17 @@ interface SingleFunctionText {
interface Answer {
readonly result: JsonValue;
readonly pseudocodeAbsence?: PseudocodeAbsence;
readonly source: AnswerSource | null;
readonly files?: readonly KunaProjectFileSpan[];
readonly locations: readonly EvidenceLocation[];
readonly limitations: readonly string[];
readonly referenceSource?: JsonValue;
readonly requiresSourceEvidence?: true;
readonly stringSource?: JsonValue;
readonly relationshipCoverage?: RelationshipCoverage;
readonly referenceClassifications?: AnalysisExecution["referenceClassifications"];
readonly referenceCoverageGaps?: AnalysisExecution["referenceCoverageGaps"];
readonly siblingUnavailability?: AnalysisCapabilityUnavailableError;
}
@@ -539,7 +544,8 @@ export class KunaSessionClient implements AnalysisClient {
}),
);
if (
operation === "procedure_pseudo_code" ||
(operation === "procedure_pseudo_code" &&
parameters.analysis !== "independent_fresh") ||
operation === "procedure_assembly" ||
operation === "analyze_function"
)
@@ -695,14 +701,21 @@ export class KunaSessionClient implements AnalysisClient {
},
{
...(sourceDescriptor === undefined ? {} : { sourceDescriptor }),
...(answer.pseudocodeAbsence === undefined
? {}
: { pseudocodeAbsence: answer.pseudocodeAbsence }),
// Kuna's identity is in the analysis profile; the raw result records
// only where file-backed text came from.
...(source === null &&
answer.pseudocodeAbsence === undefined &&
answer.referenceSource === undefined &&
answer.stringSource === undefined
? {}
: {
rawResult: {
...(answer.pseudocodeAbsence === undefined
? {}
: { pseudocode_absence: answer.pseudocodeAbsence }),
...(source === null
? {}
: {
@@ -725,6 +738,15 @@ export class KunaSessionClient implements AnalysisClient {
...(answer.relationshipCoverage === undefined
? {}
: { relationshipCoverage: answer.relationshipCoverage }),
...(answer.requiresSourceEvidence === undefined
? {}
: { requiresSourceEvidence: answer.requiresSourceEvidence }),
...(answer.referenceCoverageGaps === undefined
? {}
: { referenceCoverageGaps: answer.referenceCoverageGaps }),
...(answer.referenceClassifications === undefined
? {}
: { referenceClassifications: answer.referenceClassifications }),
...(answer.siblingUnavailability === undefined
? {}
: { siblingUnavailability: answer.siblingUnavailability }),
@@ -836,6 +858,33 @@ export class KunaSessionClient implements AnalysisClient {
limitations: classification,
});
case "procedure_pseudo_code": {
if (parameters.analysis === "independent_fresh") {
const signal =
options.signal === undefined
? this.#session.signal
: AbortSignal.any([options.signal, this.#session.signal]);
const standalone = await this.#singleFunction(
operation,
selected.address,
installation,
signal,
true,
);
if (standalone === EXPORT_READY)
throw new Error(
"Independent decompilation unexpectedly yielded to project",
);
if (!standalone.ok) return standalone;
return ok({
result: standalone.value,
source: { kind: "single-function" },
locations,
limitations: [
"This is a fresh independent single-function decompile; it does not use the project export's whole-program prototypes.",
...classification,
],
});
}
const pseudocode = await this.#pseudocode(
operation,
selected,
@@ -1005,11 +1054,59 @@ export class KunaSessionClient implements AnalysisClient {
options.signal,
);
if (!references.ok) return references;
const coverageGaps = [
...(references.value.walkCoverage !== null &&
references.value.walkCoverage.unwalked.length > 0
? [
{
code: "unwalked-code",
scope: "observation" as const,
description:
"Kuna reports unwalked code candidates whose outgoing references are not included. Their presence does not establish a reference to the selected target; the source observation identifies them.",
},
]
: []),
...(references.value.pointerCoverage === "none"
? [
{
code: "pointer-relocations-unavailable",
scope: "image" as const,
description:
"Kuna reports no pointer-relocation coverage for this image; stored pointers may still exist.",
},
]
: []),
];
return ok({
result: [...references.value.addresses],
source: null,
locations,
limitations,
limitations: [
...limitations,
...(references.value.pointerCoverage === null
? []
: [
"Kuna pointer references come from relocation metadata. The reported metadata source does not establish successful or complete enumeration.",
]),
...coverageGaps.map(({ description }) => description),
],
...(coverageGaps.length === 0
? {}
: { referenceCoverageGaps: coverageGaps }),
...(references.value.pointers === null
? {}
: {
referenceClassifications: references.value.pointers
.filter((pointer) => pointer.to_address === address)
.map((pointer) => ({
sourceAddress: pointer.from_address,
targetAddress: pointer.to_address,
kind: "stored_pointer" as const,
provenance: pointer.source,
width: pointer.width,
})),
}),
requiresSourceEvidence: true,
referenceSource: {
kind: "kuna-xrefs",
direction: "to",
@@ -1018,6 +1115,12 @@ export class KunaSessionClient implements AnalysisClient {
bytes: references.value.bytes,
examined_rows: references.value.rows,
excluded_alias_rows: references.value.excludedAliasRows,
target_aliases: references.value.targetAliases.map(
({ name, address }) => ({ name, address }),
),
walk_coverage: references.value.walkCoverage,
pointer_coverage: references.value.pointerCoverage,
pointers: references.value.pointers,
},
});
}
@@ -1251,7 +1354,14 @@ export class KunaSessionClient implements AnalysisClient {
);
if (!text.ok) return text;
return ok({
...text.value,
result: text.value.result,
limitations: text.value.limitations,
...(text.value.pseudocodeAbsence === undefined
? {}
: { pseudocodeAbsence: text.value.pseudocodeAbsence }),
...(text.value.files === undefined
? {}
: { files: text.value.files }),
source: { kind: "project-export", project: reader.project },
});
} catch (cause: unknown) {
@@ -1431,10 +1541,13 @@ export class KunaSessionClient implements AnalysisClient {
address: string,
installation: KunaInstallation,
signal: AbortSignal,
independent = false,
): Promise<SingleFunctionOutcome> {
const prepared = await this.#commands.prepare();
if (!prepared.ok) return prepared;
const admitted = await this.#admitSingleFunction(signal);
const admitted = independent
? await this.#runSlots.acquire(signal)
: await this.#admitSingleFunction(signal);
if (admitted === EXPORT_READY) return admitted;
if (admitted === undefined)
return err(
@@ -1581,7 +1694,7 @@ export class KunaSessionClient implements AnalysisClient {
pseudocode:
text === null
? unavailable(
pseudocode.value.limitations.at(-1) ??
pseudocode.value.unavailableReason ??
"Kuna did not supply pseudocode.",
)
: { available: true },
+1
View File
@@ -82,6 +82,7 @@ export const projectLldbPartialObservation = (input: {
limitations: [
"The LLDB run did not produce a complete native call observation result.",
"The target SHA-256 identifies the selected artifact; LLDB did not verify the bytes loaded into the process.",
"Source paths and lines are LLDB line-table attribution; REA did not read or verify the referenced source bytes.",
"Captured output byte counts are observed lower bounds because LLDB did not report final stream counters.",
...input.journal.limitations,
...(input.journalLimitations ?? []),
+1
View File
@@ -36,6 +36,7 @@ export const NATIVE_CALL_OBSERVATION_LIMITATIONS = [
"Registers are the raw integer argument registers at entry. For Objective-C methods the first two hold the receiver and the selector.",
"Receiver classes come from LLDB's Objective-C runtime reader without running target code; they are null for class methods and for receivers LLDB cannot classify.",
"Calls that never reach a symbol's entry, such as inlined code or objc_direct methods called directly, are not observed.",
"Source locations come from LLDB line-table metadata; REA does not read or verify the named source-file contents.",
"The target is launched directly under LLDB, not through LaunchServices; LLDB disables ASLR for it, so load addresses can equal file addresses.",
"The observation window is checked about once a second, and recording pauses the process at every entry, which slows it.",
] as const;
+303
View File
@@ -0,0 +1,303 @@
import {
deserializeMessage,
serializeMessage,
type JSONRPCMessage,
type Transport,
} from "@modelcontextprotocol/client";
import { randomUUID } from "node:crypto";
import { finished } from "node:stream/promises";
import type {
OwnedProviderProcessSpawnOptions,
SpawnedOwnedProviderProcess,
} from "./ProviderProcess.js";
import type { AnalysisCapturedOutput } from "../domain/analysisErrorBase.js";
import {
ProviderProcessSupervisor,
spawnOwnedProviderProcess,
} from "./ProviderProcess.js";
import { cleanupOwnedProcessGroup } from "./ProcessOwnership.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
/** Injectable acquisition seam; production owns a POSIX group or Windows job. */
export type OwnedMcpLauncher = (
options: OwnedProviderProcessSpawnOptions,
) => Promise<SpawnedOwnedProviderProcess>;
/** The first observed failure, recorded before notifying the SDK or beginning cleanup. */
export type OwnedMcpTransportFailure =
| { readonly kind: "process"; readonly cause: unknown }
| { readonly kind: "exit" }
| { readonly kind: "protocol"; readonly cause: Error }
| { readonly kind: "interrupted"; readonly reason: unknown };
/** SDK-framed stdio with REA-owned process cleanup rather than an unowned child. */
export class OwnedMcpStdioTransport implements Transport {
onclose: Transport["onclose"];
onerror: Transport["onerror"];
onmessage: Transport["onmessage"];
readonly runId = randomUUID();
#spawned: SpawnedOwnedProviderProcess | undefined;
#supervisor: ProviderProcessSupervisor | undefined;
#frameBuffer = Buffer.alloc(0);
#frameLength = 0;
#bytes = 0;
#started = false;
#closed = false;
#closePromise: Promise<void> | undefined;
#starting: Promise<void> | undefined;
#failure: OwnedMcpTransportFailure | undefined;
constructor(
readonly options: Omit<OwnedProviderProcessSpawnOptions, "runId" | "stdin">,
readonly settings: {
readonly provider: string;
readonly label: string;
readonly maxFrameBytes: number;
readonly maxOperationBytes: number;
readonly gracefulInputCloseMs?: number;
readonly maxDiagnosticBytes?: number;
readonly notifyOnExit?: boolean;
},
readonly launcher: OwnedMcpLauncher = spawnOwnedProviderProcess,
) {}
/** Acquires exactly one protocol process after the SDK installs callbacks. */
start(): Promise<void> {
if (this.#started || this.#closed)
throw new Error(`${this.settings.label} transport cannot be restarted`);
this.#started = true;
this.#starting = this.#acquire().catch((cause: unknown) => {
this.#failure ??= { kind: "process", cause };
throw cause;
});
return this.#starting;
}
async #acquire(): Promise<void> {
const spawned = await this.launcher({
...this.options,
runId: this.runId,
stdin: "pipe",
});
this.#spawned = spawned;
this.#supervisor = new ProviderProcessSupervisor(
{
...spawned,
ownsProcessLifetime: true,
cleanup:
spawned.cleanup ??
(() => cleanupOwnedProcessGroup(spawned.ownership)),
},
{
captureStdout: false,
...(this.settings.maxDiagnosticBytes === undefined
? {}
: { maxDiagnosticBytes: this.settings.maxDiagnosticBytes }),
},
);
if (
spawned.process.stdin === undefined ||
spawned.process.stdin === null ||
spawned.process.stdout === null
)
throw new Error(
`${this.settings.label} launcher did not supply protocol stdin/stdout`,
);
spawned.process.stdout.on("data", this.#read);
spawned.process.stdout.on("error", this.#error);
spawned.process.stderr?.on("data", this.#countStderr);
spawned.process.on("error", this.#error);
spawned.process.once("exit", this.#exit);
spawned.process.once("close", this.#outputClosed);
spawned.process.stdin.on("error", this.#error);
if (this.#closed)
throw new Error(`${this.settings.label} transport closed during startup`);
}
/** Writes SDK-serialized messages and observes stream backpressure. */
async send(message: JSONRPCMessage): Promise<void> {
const input = this.#spawned?.process.stdin;
if (this.#closed || input === undefined || input === null)
throw new Error(
`${this.settings.label} protocol process is not writable`,
);
await new Promise<void>((resolve, reject) => {
try {
input.write(serializeMessage(message), (cause) => {
if (cause === null || cause === undefined) resolve();
else {
this.#failure ??= { kind: "process", cause };
reject(cause);
}
});
} catch (cause) {
this.#failure ??= { kind: "process", cause };
reject(cause);
}
});
}
/** Verifies owned process exit before declaring the transport closed. */
close(): Promise<void> {
this.#closePromise ??= this.#stop().catch((cause: unknown) => {
this.#closePromise = undefined;
throw cause;
});
return this.#closePromise;
}
/** Keep the interruption reason distinct from errors produced by its teardown. */
interrupt(reason: unknown): Promise<void> {
this.#failure ??= { kind: "interrupted", reason };
return this.close();
}
/** Captured provider stderr without persisting inherited environment values. */
diagnostics(): AnalysisCapturedOutput | undefined {
const owner = this.#supervisor;
if (owner === undefined) return undefined;
const snapshot = owner.snapshot();
return {
stdout: "",
stderr: snapshot.stderr.text,
stderr_bytes: snapshot.stderr.observedBytes,
truncated:
snapshot.diagnosticTruncated === true || !owner.observedOutputClose(),
...(snapshot.exitCode === undefined
? {}
: { exit_code: snapshot.exitCode }),
...(snapshot.signal === undefined ? {} : { signal: snapshot.signal }),
};
}
/** Preserve the failed protocol/resource constraint after the SDK closes requests. */
observedFailure(): OwnedMcpTransportFailure | undefined {
return this.#failure;
}
/** Scope both the byte budget and diagnostic retention to the next serialized request. */
beginOperation(): void {
this.#bytes = 0;
this.#supervisor?.resetOutput();
}
async #stop(): Promise<void> {
this.#closed = true;
this.#frameBuffer = Buffer.alloc(0);
this.#frameLength = 0;
// A cancellation may race acquisition. Join it before inspecting ownership.
await this.#starting?.catch(() => undefined);
const process = this.#spawned?.process;
process?.stdout?.off("data", this.#read);
process?.stdout?.off("error", this.#error);
process?.stderr?.off("data", this.#countStderr);
process?.off("error", this.#error);
process?.off("exit", this.#exit);
process?.off("close", this.#outputClosed);
const input = process?.stdin;
// A job termination can complete an in-flight pipe write asynchronously.
// Observe its errors until the owned Writable has finished closing.
const inputClosed =
input === undefined || input === null
? undefined
: finished(input, { readable: false, cleanup: true }).catch(
() => undefined,
);
const gracefulInputCloseMs = this.settings.gracefulInputCloseMs ?? 0;
if (input !== undefined && input !== null && gracefulInputCloseMs > 0) {
input.end();
await this.#supervisor?.waitForExit(gracefulInputCloseMs);
}
const stopped = await this.#supervisor?.stop();
input?.destroy();
await inputClosed;
input?.off("error", this.#error);
this.onclose?.();
if (stopped?.status === "incomplete")
throw new ProviderCleanupError(this.settings.provider, [this.runId], {
reason: stopped.reason,
stderr: this.diagnostics()?.stderr ?? "",
});
}
readonly #error = (error: Error): void => {
this.#failure ??= { kind: "process", cause: error };
this.onerror?.(error);
// Reject pending SDK work after joining the same owned teardown on every path.
void this.close().catch((cause: unknown) =>
this.onerror?.(cause instanceof Error ? cause : new Error(String(cause))),
);
};
readonly #exit = (): void => {
this.#failure ??= { kind: "exit" };
if (this.settings.notifyOnExit === true) this.onclose?.();
};
readonly #outputClosed = (): void => {
if (this.settings.notifyOnExit !== true) this.onclose?.();
};
readonly #countStderr = (chunk: Buffer | string): void => {
this.#bytes += Buffer.byteLength(chunk);
if (this.#bytes > this.settings.maxOperationBytes)
this.#fail(
new Error(
`${this.settings.label} operation output exceeds ${this.settings.maxOperationBytes} bytes; no complete result is available`,
),
);
};
readonly #read = (chunk: Buffer | string): void => {
if (this.#closed) return;
const bytes = Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk);
this.#bytes += bytes.length;
if (this.#bytes > this.settings.maxOperationBytes) {
this.#fail(
new Error(
`${this.settings.label} operation output exceeds ${this.settings.maxOperationBytes} bytes; no complete result is available`,
),
);
return;
}
try {
let offset = 0;
while (offset < bytes.length) {
const newline = bytes.indexOf(10, offset);
const end = newline === -1 ? bytes.length : newline;
const fragment = bytes.subarray(offset, end);
const frameBytes = this.#frameLength + fragment.length;
if (frameBytes > this.settings.maxFrameBytes)
throw new Error(
`${this.settings.label} MCP frame exceeds ${this.settings.maxFrameBytes} bytes`,
);
this.#appendFrame(fragment, frameBytes);
if (newline === -1) return;
const line = new TextDecoder("utf-8", {
fatal: true,
// Preserve a leading BOM so the JSON-RPC parser can reject it.
ignoreBOM: true,
}).decode(this.#frameBuffer.subarray(0, this.#frameLength));
this.#frameBuffer = Buffer.alloc(0);
this.#frameLength = 0;
this.onmessage?.(deserializeMessage(line));
offset = newline + 1;
}
} catch (cause) {
this.#fail(cause instanceof Error ? cause : new Error(String(cause)));
}
};
#appendFrame(fragment: Buffer, frameBytes: number): void {
if (frameBytes > this.#frameBuffer.length) {
const capacity = Math.min(
this.settings.maxFrameBytes,
Math.max(frameBytes, this.#frameBuffer.length * 2, 1),
);
const grown = Buffer.allocUnsafe(capacity);
this.#frameBuffer.copy(grown, 0, 0, this.#frameLength);
this.#frameBuffer = grown;
}
fragment.copy(this.#frameBuffer, this.#frameLength);
this.#frameLength = frameBytes;
}
#fail(error: Error): void {
this.#failure ??= { kind: "protocol", cause: error };
this.#error(error);
}
}
+9 -2
View File
@@ -44,7 +44,10 @@ import {
AnalysisResourceConstraintError,
type AnalysisInputIssue,
} from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
projectAnalysisError,
projectAnalysisErrorWithoutPartialObservation,
} from "../domain/analysisErrorProjection.js";
import { projectInputIssueGroupsWithinBudget } from "../domain/inputIssueProjection.js";
import type { z } from "zod";
import { createEvidence } from "../domain/evidence.js";
@@ -936,7 +939,11 @@ class ErrorEvidenceTransport implements Transport {
}
: { retention: recorded === undefined ? "unavailable" : "failed" }),
...(recorded !== undefined && !recorded.ok
? { retention_error: projectAnalysisError(recorded.error) }
? {
retention_error: projectAnalysisErrorWithoutPartialObservation(
recorded.error,
),
}
: {}),
},
targetContent,
+47 -4
View File
@@ -7,6 +7,7 @@ import type { BinaryOpenResult } from "../application/binary/BinarySessionPort.j
import type { Evidence } from "../domain/evidence.js";
import { safeParseEvidence } from "../domain/evidenceValidation.js";
import {
omitUnretainedEvidenceFromError,
projectAnalysisError,
type AnalysisErrorProjection,
} from "../domain/analysisErrorProjection.js";
@@ -31,6 +32,7 @@ import {
MCP_RESULT_STRING_LIMIT,
NODE_MAX_STRING_LENGTH,
TARGET_CONTENT_RESULT_META,
type ToolResultEncoding,
} from "./toolResultEncoding.js";
/** Project a response-budget refusal without labeling REA's reserved string limit as Node's ceiling. */
@@ -128,12 +130,17 @@ export class ToolResultDelivery {
result: Result<JsonValue, AnalysisError>,
contract: ToolContract,
recovery: Readonly<Record<string, JsonValue>> = {},
alternatives?: (
failure: Extract<ToolResultEncoding, { readonly ok: false }>,
budgetBytes: number,
) => Iterable<JsonValue>,
): CallToolResult {
return result.ok
? this.successResult(
contract.kind === "session" ? { result: result.value } : result.value,
contract,
recovery,
alternatives,
)
: this.toErrorToolResult(result.error, contract.returnsTargetContent);
}
@@ -319,15 +326,31 @@ export class ToolResultDelivery {
projected: AnalysisErrorProjection,
targetContent: boolean,
): CallToolResult {
const structuredContent = { error: projected };
let structuredContent = { error: projected };
// Errors reach the wire as text only; the structured copy is a private
// carrier the transport removes after its own budget check.
const encoded = encodeToolResult(
let encoded = encodeToolResult(
structuredContent,
this.resultBudgetBytes,
"error-text",
targetContent,
);
if (!encoded.ok) {
const compact = omitUnretainedEvidenceFromError(projected, encoded);
if (compact !== undefined) {
const compactContent = { error: compact };
const compactEncoded = encodeToolResult(
compactContent,
this.resultBudgetBytes,
"error-text",
targetContent,
);
if (compactEncoded.ok) {
structuredContent = compactContent;
encoded = compactEncoded;
}
}
}
// The transport retains an oversized projected error before replacing it
// with a recoverable delivery constraint, using this same budget.
return {
@@ -342,13 +365,31 @@ export class ToolResultDelivery {
value: JsonValue,
contract: ToolContract,
recovery: Readonly<Record<string, JsonValue>> = {},
alternatives?: (
failure: Extract<ToolResultEncoding, { readonly ok: false }>,
budgetBytes: number,
) => Iterable<JsonValue>,
): CallToolResult {
const encoded = encodeToolResult(
let encoded = encodeToolResult(
value,
this.resultBudgetBytes,
"structured-and-text",
contract.returnsTargetContent,
);
if (!encoded.ok && alternatives !== undefined)
for (const candidate of alternatives(encoded, this.resultBudgetBytes)) {
const candidateEncoding = encodeToolResult(
candidate,
this.resultBudgetBytes,
"structured-and-text",
contract.returnsTargetContent,
);
if (candidateEncoding.ok) {
value = candidate;
encoded = candidateEncoding;
break;
}
}
if (encoded.ok)
return {
content: [{ type: "text", text: encoded.text }],
@@ -369,7 +410,9 @@ export class ToolResultDelivery {
? "Call list_unknowns on this same connection with the same filters and offset, using a smaller limit, then follow next_offset. If a single unknown cannot fit, export the session with export_evidence_bundle to recover it."
: contract.name === "get_evidence_bundle"
? evidenceBundleOverflowStep(value)
: "Use complete CLI JSON output to consume the analysis.";
: contract.name === "inspect_address_context"
? "Call get_evidence_bundle with detail summary to discover retained facet observations, then export_evidence_bundle to inspect complete records. Some facets may not have been retained; a new analysis can differ."
: "Use complete CLI JSON output to consume the analysis.";
const projected = mcpResponseOverflowProjection({
completed: true,
constraint: encoded.constraint,
+24 -4
View File
@@ -8,6 +8,7 @@ import { workflowUnknownInput } from "../application/investigation/WorkflowUnkno
import {
getNavigationContext,
inspectAddressContext,
omitContextEvidenceForDelivery,
} from "../application/AnalysisContextQueries.js";
import type { BinarySessionPort } from "../application/binary/BinarySessionPort.js";
import {
@@ -323,11 +324,30 @@ const registerContextTools = (
const admitted = await admission(
addressContract.name,
context.mcpReq.signal,
async (analysis) =>
server.delivery.toCallToolResult(
await inspectAddressContext(analysis, input, context.mcpReq.signal),
async (analysis) => {
const result = await inspectAddressContext(
analysis,
input,
context.mcpReq.signal,
);
if (!result.ok)
return server.delivery.toCallToolResult(result, addressContract);
const value = result.value;
return server.delivery.toCallToolResult(
result,
addressContract,
),
{},
(failure, budgetBytes) =>
(function* () {
for (const alternative of omitContextEvidenceForDelivery(
value,
failure,
budgetBytes,
))
yield { result: alternative };
})(),
);
},
);
return admitted.ok
? admitted.value
@@ -255,11 +255,11 @@ cliTest(
code: "invalid_request",
details: {
issues: [
{
expect.objectContaining({
path: ["executable_path"],
reason: "invalid_format",
message: expect.stringContaining("absolute local filesystem path"),
},
expected: "regex",
}),
],
},
});
@@ -4,6 +4,7 @@ import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { z } from "zod";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
@@ -329,21 +330,84 @@ const pathRequests: readonly [string, Record<string, unknown>][] = [
["start_gdb_session", { inferior_tty: nul }],
];
const nulIssuePaths = (
value: unknown,
path: readonly (string | number)[] = [],
): (string | number)[][] => {
if (typeof value === "string") return value.includes("\0") ? [[...path]] : [];
if (Array.isArray(value))
return value.flatMap((item, index) =>
nulIssuePaths(item, [...path, index]),
);
if (typeof value === "object" && value !== null)
return Object.entries(value).flatMap(([key, item]) =>
nulIssuePaths(item, [...path, key]),
);
return [];
};
it("rejects OS-invalid local inputs through the real advertised MCP contracts", async () => {
const { call } = await connectLocalToolsMcp();
const checked: string[] = [];
for (const [name, arguments_] of pathRequests) {
const response = await call(name, arguments_);
expect(response.isError, name).toBe(true);
expect(JSON.stringify(response.content), name).toContain("NUL");
const paths = nulIssuePaths(arguments_);
expect(paths.length, name).toBeGreaterThan(0);
const error = parseMcpToolError(await call(name, arguments_)).error;
expect(error, name).toMatchObject({
code: "invalid_request",
details: { operation: name },
});
const issues = z
.object({
issues: z.array(
z.object({
path: z.array(z.union([z.string(), z.number()])),
reason: z.string(),
message: z.string(),
expected: z.unknown().optional(),
}),
),
})
.parse(error.details).issues;
expect(issues, name).toEqual(
expect.arrayContaining(
paths.map((path) =>
expect.objectContaining({
path,
reason: "invalid_format",
expected: "regex",
message: expect.stringMatching(/\bNUL\b|null byte|zero byte/i),
}),
),
),
);
checked.push(name);
}
expect(checked).toEqual(
expect.arrayContaining([
"extract_firmware",
"inspect_wasm_artifact",
"demangle_swift",
"capture_electron_scenario",
]),
);
const relative = await call("extract_firmware", {
path: "/tmp/firmware.bin",
output_directory: "relative-output",
});
expect(relative.isError).toBe(true);
expect(JSON.stringify(relative.content)).toContain(
"absolute local filesystem path",
);
expect(parseMcpToolError(relative).error).toMatchObject({
code: "invalid_request",
details: {
operation: "extract_firmware",
issues: [
expect.objectContaining({
path: ["output_directory"],
reason: "invalid_format",
expected: "regex",
}),
],
},
});
}, 60_000);
const ignoredOptions = [
@@ -5,6 +5,7 @@ import {
type JSONRPCMessage,
} from "@modelcontextprotocol/client";
import { expect, it } from "vitest";
import { OwnedMcpStdioTransport } from "../../../src/process/OwnedMcpStdioTransport.js";
import type { ProviderProcessHandle } from "../../../src/process/ProviderProcess.js";
import {
JadxMcpTransport,
@@ -125,6 +126,47 @@ it("decodes split UTF-8 and newline bytes, then routes multiple complete frames"
await transport.close();
});
it("delivers buffered protocol output before reporting process close", async () => {
const process = new FixtureProcess();
const transport = new OwnedMcpStdioTransport(
{ command: "fixture", arguments: [] },
{
provider: "fixture",
label: "Fixture",
maxFrameBytes: 1024,
maxOperationBytes: 4096,
},
fixtureLauncher(process),
);
const events: string[] = [];
transport.onmessage = () => events.push("message");
transport.onclose = () => events.push("close");
await transport.start();
process.emit("exit", 0, null);
expect(transport.observedFailure()).toEqual({ kind: "exit" });
expect(events).toEqual([]);
process.stdout.write(serializeMessage({ jsonrpc: "2.0", id: 1, result: {} }));
process.emit("close", 0, null);
expect(events).toEqual(["message", "close"]);
await transport.close();
});
it("reports JADX exit even when a descendant holds the protocol pipes", async () => {
const process = new FixtureProcess();
const transport = new JadxMcpTransport(
{ command: "fixture", arguments: [] },
fixtureLauncher(process),
);
let closed = false;
transport.onclose = () => {
closed = true;
};
await transport.start();
process.emit("exit", 0, null);
expect(closed).toBe(true);
await transport.close();
});
it.each([
["invalid UTF-8 byte", Buffer.from([0xff])],
["truncated UTF-8 sequence", Buffer.from([0xe2, 0x82])],
@@ -221,3 +263,46 @@ it("retries an incomplete owned stop while joining concurrent close callers", as
await transport.close();
await transport.close();
});
it("joins late acquisition before releasing the protocol process", async () => {
const process = new FixtureProcess();
let deliver:
| ((launch: Awaited<ReturnType<JadxLauncher>>) => void)
| undefined;
let acquired = false;
let cleaned = false;
const transport = new JadxMcpTransport(
{ command: "fixture", arguments: [] },
async ({ runId }) =>
new Promise((resolve) => {
acquired = true;
deliver = resolve;
// The owned process is returned only when the test releases acquisition.
void runId;
}),
);
const startup = transport.start();
expect(acquired).toBe(true);
let finished = false;
const closing = transport.close().then(() => {
finished = true;
});
await new Promise<void>((resolve) => setImmediate(resolve));
expect(finished).toBe(false);
deliver?.({
process,
ownership: { runId: transport.runId, leaderPid: 1, processGroupId: 1 },
cleanup: async () => {
cleaned = true;
process.emit("exit", 0, null);
process.emit("close", 0, null);
process.stdout.end();
process.stderr.end();
process.stdin.end();
return { cleaned: true, signaled: false };
},
});
await expect(startup).rejects.toThrow("closed during startup");
await closing;
expect(cleaned).toBe(true);
});
@@ -55,10 +55,7 @@ const singleDefaultOptions: readonly {
{ command: "inspect-signal-decoders", options: ["--offset"] },
];
const optionLine = (
help: string,
option: string,
): string | undefined => {
const optionLine = (help: string, option: string): string | undefined => {
const line = help
.split(/\r?\n/u)
.find((candidate) => candidate.trimStart().startsWith(`${option} `));
@@ -80,7 +77,9 @@ describe("CLI help default annotations", () => {
}
const statements = line.match(/default/giu)?.length ?? 0;
if (statements !== 1)
repeated.push(`${command} ${option}: ${statements} default statements`);
repeated.push(
`${command} ${option}: ${statements} default statements`,
);
}
}
expect(repeated).toEqual([]);
@@ -1870,6 +1870,163 @@ it("retains a native dispatch workflow snapshot without a deep provider profile"
);
});
it.each([false, true])(
"does not replay a fresh decompile from a selected snapshot (document supplied: %s)",
async (withDocument) => {
const fixture = await createSnapshotReplayFixture(
"rea-fresh-decompile-snapshot-",
);
const original = fixture.createProvider({
supportedOperations: ["procedure_pseudo_code"],
});
const provider: AnalysisProvider = {
...original,
capabilities: () =>
original.capabilities().map((descriptor) => ({
...descriptor,
independentDecompile: true as const,
})),
};
const args = {
procedure: "fixture",
analysis: "independent_fresh",
...(withDocument ? { document: fixture.path } : {}),
};
const first = await runDirectAnalysis(
fixture.dependencies(provider),
fixture.path,
"procedure_pseudo_code",
args,
{ snapshotPath: fixture.snapshotPath },
);
const second = await runDirectAnalysis(
fixture.dependencies(provider),
fixture.path,
"procedure_pseudo_code",
args,
{ snapshotPath: fixture.snapshotPath },
);
expect(first.ok).toBe(true);
expect(second.ok).toBe(true);
expect(
fixture.calls.filter(
(operation) => operation === "procedure_pseudo_code",
),
).toHaveLength(2);
const saved = await readAnalysisSnapshot(fixture.snapshotPath);
if (!saved.ok) throw saved.error;
expect(
saved.value.entries.some(
(entry) => entry.operation === "procedure_pseudo_code",
),
).toBe(false);
},
);
it.each([true, false])(
"does not replay an imported fresh decompile (provider declares support: %s)",
async (declared) => {
const fixture = await createSnapshotReplayFixture(
"rea-fresh-imported-snapshot-",
);
const original = fixture.createProvider({
supportedOperations: ["procedure_pseudo_code"],
});
const provider: AnalysisProvider = {
...original,
capabilities: () =>
original.capabilities().map((descriptor) => ({
...descriptor,
...(declared ? { independentDecompile: true as const } : {}),
})),
};
const parameters = { procedure: "fixture", document: fixture.path };
const dependencies = fixture.dependencies(provider);
const first = await runDirectAnalysis(
dependencies,
fixture.path,
"procedure_pseudo_code",
parameters,
{ snapshotPath: fixture.snapshotPath },
);
if (!first.ok) throw first.error;
const saved = await readAnalysisSnapshot(fixture.snapshotPath);
if (!saved.ok) throw saved.error;
const entry = saved.value.entries.find(
(item) => item.operation === "procedure_pseudo_code",
);
const evidence = saved.value.evidence_bundle.records.find(
(item) => item.operation === "procedure_pseudo_code",
);
if (
entry === undefined ||
evidence === undefined ||
evidence.analysis_profile === null
)
throw new Error("Missing saved decompile");
const freshParameters = { ...parameters, analysis: "independent_fresh" };
const historical = createEvidence(
{
path: fixture.path,
sha256: saved.value.target.sha256,
format: saved.value.target.format,
},
evidence.provider,
{
operation: evidence.operation,
parameters: freshParameters,
result: evidence.normalized_result,
rawResult: evidence.raw_result,
analysisProfile: evidence.analysis_profile,
confidence: evidence.confidence,
limitations: evidence.limitations,
locations: evidence.locations,
},
);
const written = await withArtifactResourceScope((resources) =>
writeAnalysisSnapshot(
{
...saved.value,
entries: [
{
...entry,
parameters: freshParameters,
query_id: analysisQueryId(
saved.value.target,
saved.value.binding,
entry.operation,
freshParameters,
),
},
],
evidence_bundle: createEvidenceBundle([historical]),
},
fixture.snapshotPath,
true,
resources,
),
);
if (!written.ok) throw written.error;
const fresh = await runDirectAnalysis(
dependencies,
fixture.path,
"procedure_pseudo_code",
freshParameters,
{ snapshotPath: fixture.snapshotPath },
);
expect(fresh.ok).toBe(declared);
if (!fresh.ok)
expect(fresh.error, JSON.stringify(fresh.error)).toMatchObject({
code: "capability_unavailable",
});
expect(
fixture.calls.filter(
(operation) => operation === "procedure_pseudo_code",
),
).toHaveLength(declared ? 2 : 1);
},
);
it("replays source-bearing direct analysis only while snapshot source Evidence is present", async () => {
const fixture = await createSnapshotReplayFixture(
"rea-direct-source-replay-",
@@ -0,0 +1,123 @@
import { spawn } from "node:child_process";
import {
chmod,
readFile,
readdir,
realpath,
rm,
symlink,
writeFile,
} from "node:fs/promises";
import { basename, join } from "node:path";
import { expect, it } from "vitest";
import { createCanonicalTemporaryDirectory } from "../../../scripts/lib/canonical-temporary-directory.mjs";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
it("uses the canonical spelling for a verifier directory allocated through a symlink", async () => {
const root = await createTestTempDirectory("rea-pascal-verifier-path-");
const alias = join(root, "alias");
await symlink(root, alias, process.platform === "win32" ? "junction" : "dir");
const allocated = await createCanonicalTemporaryDirectory(
"rea-pascal-calls-",
alias,
);
try {
expect(allocated.path).toBe(await realpath(allocated.allocatedPath));
expect(allocated.path).not.toBe(
join(alias, basename(allocated.allocatedPath)),
);
} finally {
await rm(allocated.allocatedPath, { recursive: true, force: true });
}
});
it.skipIf(process.platform !== "darwin")(
"forwards verifier SIGTERM to the owned compiler group and removes its temporary directory",
async () => {
const root = await createTestTempDirectory("rea-pascal-verifier-cancel-");
const fakeCompiler = join(root, "fake-fpc");
const pidFile = join(root, "compiler-pids.txt");
await writeFile(
fakeCompiler,
'#!/usr/bin/env node\nimport { spawn } from "node:child_process";\nimport { writeFileSync } from "node:fs";\nconst child = spawn(process.execPath, ["-e", "setInterval(() => undefined, 1000)"], { stdio: "ignore" });\nwriteFileSync(process.env.REA_VERIFY_TEST_PID_FILE, `${process.pid} ${child.pid}\\n`);\nawait new Promise(() => undefined);\n',
);
await chmod(fakeCompiler, 0o700);
const verifier = spawn(
process.execPath,
["scripts/verify-native-pascal-calls.mjs"],
{
cwd: process.cwd(),
env: {
...process.env,
TMPDIR: root,
REA_VERIFY_FPC_COMMAND: fakeCompiler,
REA_VERIFY_TEST_PID_FILE: pidFile,
},
stdio: "ignore",
},
);
try {
const deadline = Date.now() + 10_000;
let pids: number[] | undefined;
while (Date.now() < deadline) {
try {
pids = (await readFile(pidFile, "utf8"))
.trim()
.split(/\s+/u)
.map(Number);
break;
} catch (cause: unknown) {
if ((cause as NodeJS.ErrnoException).code !== "ENOENT") throw cause;
await new Promise((resolve) => setTimeout(resolve, 20));
}
}
expect(pids, "the fake compiler started").toHaveLength(2);
verifier.kill("SIGTERM");
const result = await new Promise<{
code: number | null;
signal: NodeJS.Signals | null;
}>((resolve, reject) => {
verifier.once("error", reject);
verifier.once("close", (code, signal) => resolve({ code, signal }));
});
expect(result).toEqual({ code: 143, signal: null });
for (const pid of pids ?? [])
expect(() => process.kill(pid, 0)).toThrow();
expect(
(await readdir(root)).filter((name) =>
name.startsWith("rea-pascal-calls-"),
),
).toEqual([]);
} finally {
if (verifier.exitCode === null && verifier.signalCode === null) {
verifier.kill("SIGTERM");
await new Promise((resolve) => {
const timer = setTimeout(() => {
verifier.kill("SIGKILL");
resolve(undefined);
}, 1_000);
verifier.once("close", () => {
clearTimeout(timer);
resolve(undefined);
});
});
}
try {
const recorded = (await readFile(pidFile, "utf8"))
.trim()
.split(/\s+/u)
.map(Number);
for (const pid of recorded) {
try {
process.kill(pid, "SIGKILL");
} catch {
// The owned compiler group already exited.
}
}
} catch (cause: unknown) {
if ((cause as NodeJS.ErrnoException).code !== "ENOENT") throw cause;
}
await rm(root, { recursive: true, force: true });
}
},
);
@@ -13,7 +13,7 @@ interface PathInputCase {
readonly field: string;
readonly valid: Readonly<Record<string, unknown>>;
readonly invalid: Readonly<Record<string, unknown>>;
readonly issuePath?: readonly string[];
readonly issuePath: readonly string[];
}
const createMcp = async (
@@ -104,7 +104,13 @@ const inputCases = (
field: string,
valid: Readonly<Record<string, unknown>>,
invalid: Readonly<Record<string, unknown>>,
): PathInputCase => ({ name, field, valid, invalid });
): PathInputCase => ({
name,
field,
valid,
invalid,
issuePath: field.split("."),
});
return [
pathCase("open_binary", "path", { path: absolute }, { path: relative }),
@@ -246,8 +252,7 @@ it("advertises full host-absolute path constraints after SDK schema conversion",
const ajv = new Ajv2020({ strict: false, validateFormats: false });
const absolute = resolve("rea-absolute-input");
const relative = "relative-input";
for (const { name, field, valid, invalid } of [
const schemaCases = [
...inputCases(absolute, relative),
{
name: "inspect_go_binary",
@@ -276,7 +281,9 @@ it("advertises full host-absolute path constraints after SDK schema conversion",
right_bundle_path: absolute,
},
},
]) {
];
const checked: string[] = [];
for (const { name, field, valid, invalid } of schemaCases) {
const schema = listed.tools.find(
(tool) => tool.name === name,
)?.inputSchema;
@@ -288,7 +295,16 @@ it("advertises full host-absolute path constraints after SDK schema conversion",
validate(valid),
`${name}.${field}: ${JSON.stringify(validate.errors)}`,
).toBe(true);
checked.push(`${name}.${field}`);
}
expect(checked).toEqual(
expect.arrayContaining([
"open_binary.path",
"extract_firmware.output_directory",
"inspect_go_binary.path NUL",
"compare_bundles.left_bundle_path empty",
]),
);
expect(firmwareCalls).toEqual([]);
} finally {
await close();
@@ -299,19 +315,39 @@ it("rejects relative paths through in-memory MCP handlers", async () => {
const firmwareCalls: string[] = [];
const { client, close } = await createMcp(firmwareCalls);
try {
for (const { name, field, invalid, issuePath } of inputCases(
resolve("rea-absolute-input"),
"relative-input",
)) {
const cases = inputCases(resolve("rea-absolute-input"), "relative-input");
const checked: string[] = [];
for (const { name, field, invalid, issuePath } of cases) {
const result = await client.callTool({ name, arguments: invalid });
expect(result.isError, `${name}.${field}`).toBe(true);
expect(JSON.stringify(result), `${name}.${field}`).toContain("absolute");
if (issuePath !== undefined)
expect(parseMcpToolError(result).error.details).toMatchObject({
operation: name,
issues: [{ path: issuePath, reason: "invalid_format" }],
});
expect(parseMcpToolError(result).error, `${name}.${field}`).toMatchObject(
{
code: "invalid_request",
details: {
operation: name,
issues: [
expect.objectContaining({
path: issuePath,
reason: "invalid_format",
expected: "regex",
}),
],
},
},
);
checked.push(`${name}.${field}`);
}
expect(checked).toEqual(
expect.arrayContaining([
"open_binary.path",
"open_binary.snapshot_path",
"extract_firmware.path",
"extract_firmware.output_directory",
"inspect_analysis_view.source.path",
"capture_browser_scenario.browser.executable_path",
"capture_electron_scenario.application_root",
]),
);
expect(firmwareCalls).toEqual([]);
} finally {
await close();
@@ -0,0 +1,145 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { expect, it } from "vitest";
import { createTestProviderRouter } from "../../fixtures/binarySession.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js";
import { BinarySession } from "../../../src/application/binary/BinarySession.js";
import { InvestigationRecords } from "../../../src/application/investigation/InvestigationRecords.js";
import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js";
import { err, ok } from "../../../src/domain/result.js";
import { AnalysisCapabilityUnavailableError } from "../../../src/domain/analysisErrorCore.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { SESSION_TOOL_CONTRACTS } from "../../../src/contracts/sessionToolContracts.js";
import { createServer } from "../../../src/server/createServer.js";
it.each(["compact", "full"] as const)(
"preserves retained siblings and capacity recovery (%s schema)",
async (profile) => {
const identity = {
id: "fixture",
name: "Fixture analysis provider",
version: "1",
};
const directory = await createTestTempDirectory("rea-context-capacity-");
const target = join(directory, "fixture.com");
await writeFile(target, Buffer.from([0xb8, 0, 0x4c, 0xcd, 0x21]));
const router = createTestProviderRouter(
() => ({
execute: async (operation) => {
if (operation === "address_name")
return ok(createAnalysisExecution("entry", identity));
if (operation === "resolve_containing_procedure")
return ok(
createAnalysisExecution(
{
query_address: "0x100",
found: true,
procedure: {
address: "0x100",
name: "f".repeat(16000),
classification: null,
body: { available: false, reason: "not recorded" },
},
},
identity,
),
);
return err(
new AnalysisCapabilityUnavailableError(
identity.id,
operation,
"not provided",
),
);
},
close: async () => ok(null),
}),
{
resolveAnalysisProfile: async () =>
ok({ profile: createAnalysisProfile(identity, { fixture: true }) }),
},
);
const records = new InvestigationRecords({
maximumRepresentationBytes: 8000,
});
const session = new BinarySession(router, records);
const server = createServer(
{ kind: "session", session },
{
environment: { ...process.env, REA_MCP_OUTPUT_SCHEMA_PROFILE: profile },
},
);
const client = new Client({ name: "context-capacity", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const definitions = new Map(
(await client.listTools()).tools.map((tool) => [tool.name, tool]),
);
const call = (name: string, args: Record<string, string>) => {
const toolDefinition = definitions.get(name);
if (toolDefinition === undefined)
throw new Error(`Tool ${name} was not advertised`);
return client.callTool({ name, arguments: args }, { toolDefinition });
};
expect(
(
await call("open_binary", {
path: target,
format: "dos-com",
provider_id: "fixture",
})
).isError,
).not.toBe(true);
const arguments_ = { document: "fixture", address: "0x100" };
const response = await call("inspect_address_context", arguments_);
expect(response.isError).not.toBe(true);
const contract = SESSION_TOOL_CONTRACTS.find(
(
candidate,
): candidate is Extract<
(typeof SESSION_TOOL_CONTRACTS)[number],
{ readonly name: "inspect_address_context" }
> => candidate.name === "inspect_address_context",
);
if (contract === undefined)
throw new Error("inspect_address_context contract is missing");
const context = contract.outputSchema.parse(
response.structuredContent,
).result;
const direct = await call("resolve_containing_procedure", arguments_);
expect(direct.isError).toBe(true);
const expected = parseMcpToolError(direct).error;
expect(expected.code).toBe("resource_constraint");
expect(expected.details).toMatchObject({
partial_observation: {
kind: "unretained-evidence",
evidence: {
operation: "resolve_containing_procedure",
normalized_result: { found: true, query_address: "0x100" },
},
},
});
expect(context.name).toEqual({ state: "available", value: "entry" });
expect(context.procedure).toEqual({
state: "unavailable",
reason: expected.message,
remediation: expected.remediation.action,
error: expected,
});
expect(records.statistics().evidenceRecords).toBe(1);
expect(session.immutableEvidenceBundle().records[0]).toMatchObject({
operation: "address_name",
normalized_result: "entry",
});
} finally {
await client.close();
await server.close();
await session.close();
}
},
);
@@ -0,0 +1,331 @@
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { expect, it } from "vitest";
import { createTestProviderRouter } from "../../fixtures/binarySession.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js";
import { BinarySession } from "../../../src/application/binary/BinarySession.js";
import { InvestigationRecords } from "../../../src/application/investigation/InvestigationRecords.js";
import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js";
import { err, ok } from "../../../src/domain/result.js";
import { AnalysisCapabilityUnavailableError } from "../../../src/domain/analysisErrorCore.js";
import { createServer } from "../../../src/server/createServer.js";
const verifyContext = (
content: unknown,
twoLarge: boolean,
session: BinarySession,
): void => {
expect(content).toMatchObject({
result: {
name: { state: "available", value: "entry" },
procedure: {
state: "unavailable",
error: {
code: "resource_constraint",
details: {
partial_observation_omitted: {
reason: "mcp-response-limit",
retention: "unretained",
original_response_bytes_at_least: expect.any(Number),
evidence_id: expect.stringMatching(/^ev_[a-f0-9]{64}$/u),
},
},
},
},
},
});
const procedureDetails = (
content as {
result: {
procedure: {
error: {
details: {
partial_observation_omitted: { evidence_id: string };
};
};
};
};
}
).result.procedure.error.details;
expect(procedureDetails).not.toHaveProperty("partial_observation");
expect(procedureDetails).not.toHaveProperty("evidence_reference");
expect(
session.hasEvidence(
procedureDetails.partial_observation_omitted.evidence_id,
),
).toBe(false);
const smallFacet = twoLarge ? "inline_comment" : "comment";
expect(content).toMatchObject({
result: {
[smallFacet]: {
state: "unavailable",
error: {
code: "resource_constraint",
details: {
partial_observation: {
kind: "unretained-evidence",
evidence: {
operation: smallFacet,
normalized_result: "short note",
},
},
},
},
},
...(twoLarge
? {
comment: {
state: "unavailable",
error: {
details: {
partial_observation_omitted: {
reason: "mcp-response-limit",
retention: "unretained",
},
},
},
},
}
: {}),
},
});
};
it.each([
["compact", false],
["full", false],
["compact", true],
] as const)(
"keeps context and smaller partials when unretained observations exceed the MCP budget (%s, two large=%s)",
async (profile, twoLarge) => {
const identity = {
id: "fixture",
name: "Fixture analysis provider",
version: "1",
};
const directory = await createTestTempDirectory("rea-context-capacity-");
const target = join(directory, "fixture.com");
await writeFile(target, Buffer.from([0xb8, 0, 0x4c, 0xcd, 0x21]));
const router = createTestProviderRouter(
() => ({
execute: async (operation) => {
if (operation === "address_name")
return ok(createAnalysisExecution("entry", identity));
if (operation === "comment")
return ok(
createAnalysisExecution(
twoLarge ? "g".repeat(6 * 1024 * 1024) : "short note",
identity,
),
);
if (operation === "inline_comment" && twoLarge)
return ok(createAnalysisExecution("short note", identity));
if (operation === "resolve_containing_procedure")
return ok(
createAnalysisExecution(
{
query_address: "0x100",
found: true,
procedure: {
address: "0x100",
name: "f".repeat(6 * 1024 * 1024),
classification: null,
body: { available: false, reason: "not recorded" },
},
},
identity,
),
);
return err(
new AnalysisCapabilityUnavailableError(
identity.id,
operation,
"not provided",
),
);
},
close: async () => ok(null),
}),
{
resolveAnalysisProfile: async () =>
ok({ profile: createAnalysisProfile(identity, { fixture: true }) }),
},
);
const records = new InvestigationRecords({
maximumRepresentationBytes: 8000,
});
const session = new BinarySession(router, records);
const server = createServer(
{ kind: "session", session },
{
environment: {
...process.env,
REA_MCP_TOOL_FAMILIES: "session",
REA_MCP_OUTPUT_SCHEMA_PROFILE: profile,
},
},
);
const client = new Client({ name: "context-capacity", version: "1" });
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
try {
await server.connect(serverTransport);
await client.connect(clientTransport);
const definitions = new Map(
(await client.listTools()).tools.map((tool) => [tool.name, tool]),
);
const call = (name: string, args: Record<string, string>) => {
const toolDefinition = definitions.get(name);
if (toolDefinition === undefined)
throw new Error(`Tool ${name} was not advertised`);
return client.callTool({ name, arguments: args }, { toolDefinition });
};
expect(
(
await call("open_binary", {
path: target,
format: "dos-com",
provider_id: "fixture",
})
).isError,
).not.toBe(true);
const arguments_ = { document: "fixture", address: "0x100" };
const response = await call("inspect_address_context", arguments_);
expect(response.isError).not.toBe(true);
verifyContext(response.structuredContent, twoLarge, session);
const bundle = await client.callTool({
name: "get_evidence_bundle",
arguments: { detail: "complete" },
});
expect(bundle.structuredContent).toMatchObject({
result: {
records: [{ operation: "address_name", normalized_result: "entry" }],
},
});
} finally {
await client.close();
await server.close();
await session.close();
}
},
);
it.each([true, false])(
"recovers retained context facets when possible (profile=%s)",
async (profile) => {
const identity = {
id: "fixture",
name: "Fixture analysis provider",
version: "1",
};
const target = join(
await createTestTempDirectory("rea-context-oversize-success-"),
"fixture.com",
);
await writeFile(target, Buffer.from([0xb8, 0, 0x4c, 0xcd, 0x21]));
const router = createTestProviderRouter(
() => ({
execute: async (operation) =>
operation === "health"
? ok(createAnalysisExecution(null, identity))
: operation === "address_name"
? ok(
createAnalysisExecution(
"x".repeat(6 * 1024 * 1024),
identity,
),
)
: err(
new AnalysisCapabilityUnavailableError(
identity.id,
operation,
"not provided",
),
),
close: async () => ok(null),
}),
profile
? {
resolveAnalysisProfile: async () =>
ok({
profile: createAnalysisProfile(identity, { fixture: true }),
}),
}
: {},
);
const session = new BinarySession(router);
const server = createServer(
{ kind: "session", session },
{ environment: { ...process.env, REA_MCP_TOOL_FAMILIES: "session" } },
);
const client = new Client({
name: "context-oversize-success",
version: "1",
});
const [ct, st] = InMemoryTransport.createLinkedPair();
try {
await server.connect(st);
await client.connect(ct);
const opened = await client.callTool({
name: "open_binary",
arguments: { path: target, format: "dos-com" },
});
expect(opened.isError, JSON.stringify(opened.content)).not.toBe(true);
const result = await client.callTool({
name: "inspect_address_context",
arguments: { address: "0x100", document: "fixture" },
});
expect(result.isError).toBe(true);
const error = parseMcpToolError(result).error;
expect(error).toMatchObject({
code: "resource_constraint",
details: { reported_limits: { boundary: "mcp-response" } },
});
expect(error.remediation.action).toContain("get_evidence_bundle");
expect(error.remediation.action).toContain("export_evidence_bundle");
expect(error.remediation.action).toContain("may not have been retained");
const advertised = new Set(
(await client.listTools()).tools.map((tool) => tool.name),
);
expect(advertised.has("get_evidence_bundle")).toBe(true);
expect(advertised.has("export_evidence_bundle")).toBe(true);
expect(advertised.has("address_name")).toBe(false);
const summary = await client.callTool({
name: "get_evidence_bundle",
arguments: {
detail: "summary",
filters: { operation: "address_name" },
},
});
expect(summary.isError).not.toBe(true);
expect(summary.structuredContent).toMatchObject({
result: { matching_records: profile ? 1 : 0 },
});
const path = join(
await createTestTempDirectory("rea-context-recover-"),
"evidence.json",
);
const exported = await client.callTool({
name: "export_evidence_bundle",
arguments: { path },
});
expect(exported.isError).not.toBe(true);
const bundle = JSON.parse(await readFile(path, "utf8")) as {
records: { operation: string; normalized_result: unknown }[];
};
const record = bundle.records.find(
(entry) => entry.operation === "address_name",
);
if (profile)
expect(record?.normalized_result).toBe("x".repeat(6 * 1024 * 1024));
else expect(record).toBeUndefined();
} finally {
await client.close();
await server.close();
await session.close();
}
},
);
@@ -140,6 +140,106 @@ const trace = async (analysis: AnalysisOperationPort) => {
return result.structuredContent as { normalized_result: unknown };
};
it("reports known reference-discovery gaps without claiming truncation or a missing edge", async () => {
const baseline = provider();
const gap =
"An unwalked candidate may have outgoing references; its presence does not establish a reference to the selected target.";
const output = await trace({
execute: (operation, input, options) =>
operation === "xrefs" && input.address === "0x10"
? Promise.resolve(
ok(
createAnalysisExecution([], PROVIDER, {
referenceCoverageGaps: [
{ code: "unwalked", scope: "observation", description: gap },
],
}),
),
)
: baseline.execute(operation, input, options),
});
expect(output.normalized_result).toMatchObject({
truncated: false,
residual_unknowns: [`References to 0x10: ${gap}`],
references: [{ target_address: "0x11", source_address: "0x21" }],
});
});
it("deduplicates an image-wide gap without merging target-specific gaps", async () => {
const baseline = provider();
const output = await trace({
execute: async (operation, input, options) => {
const result = await baseline.execute(operation, input, options);
if (operation !== "xrefs" || !result.ok) return result;
return ok({
...result.value,
referenceCoverageGaps: [
{
code: "missing-relocations",
scope: "image",
description: "Image relocation coverage unavailable.",
},
{
code: "unwalked",
scope: "observation",
description: "This observation has unwalked sources.",
},
],
});
},
});
expect(output.normalized_result).toMatchObject({
truncated: false,
residual_unknowns: [
"Image relocation coverage unavailable.",
"References to 0x10: This observation has unwalked sources.",
"References to 0x11: This observation has unwalked sources.",
],
});
});
it("carries reported pointer classifications without inferring other reference kinds", async () => {
const baseline = provider();
const output = await trace({
execute: (operation, input, options) =>
operation === "xrefs" && input.address === "0x10"
? Promise.resolve(
ok(
createAnalysisExecution(["0x20"], PROVIDER, {
referenceClassifications: [
{
sourceAddress: "0x20",
targetAddress: "0x10",
kind: "stored_pointer",
provenance: "chained-fixups",
width: 8,
},
{
sourceAddress: "0x20",
targetAddress: "0x99",
kind: "stored_pointer",
provenance: "other",
width: 4,
},
],
}),
),
)
: baseline.execute(operation, input, options),
});
expect(output.normalized_result).toMatchObject({
references: [
{
source_address: "0x20",
source_classifications: [
{ kind: "stored_pointer", provenance: "chained-fixups", width: 8 },
],
},
{ source_address: "0x21" },
],
});
});
it("preserves references and explicitly unknown owners without claiming truncation", async () => {
const baseline = provider();
const output = await trace({
@@ -0,0 +1,108 @@
import { expect, it } from "vitest";
import { createApplicationMcpHarness } from "../../fixtures/applicationMcpHarness.js";
import { createTestProviderRouter } from "../../fixtures/binarySession.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { BinarySession } from "../../../src/application/binary/BinarySession.js";
import { InvestigationRecords } from "../../../src/application/investigation/InvestigationRecords.js";
import { compareFunctionEvidence } from "../../../src/application/investigation/FunctionComparisonEvidence.js";
import { FUNCTION_COMPARISON_EXAMPLE } from "../../../src/contracts/functionComparisonExample.js";
import { safeParseEvidence } from "../../../src/domain/evidenceValidation.js";
import { parseEvidenceBundle } from "../../../src/domain/evidenceBundle.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
import { analysisErrorProjectionSchema } from "../../../src/contracts/errorSchemas.js";
it.each([
{ profile: "compact", preexisting: false },
{ profile: "compact", preexisting: true },
{ profile: "full", preexisting: false },
{ profile: "full", preexisting: true },
] as const)(
"preserves the completed comparison on atomic retention failure ($profile schema, preexisting: $preexisting)",
async ({ profile, preexisting }) => {
const input = { ...FUNCTION_COMPARISON_EXAMPLE };
const computed = compareFunctionEvidence(input);
if (!computed.ok) throw computed.error;
expect(computed.value.unknown).toBeDefined();
const initial = [
input.left,
input.right,
...(preexisting ? [computed.value.evidence] : []),
];
const sizing = new InvestigationRecords();
for (const evidence of initial)
expect(sizing.recordEvidence(evidence).ok).toBe(true);
const records = new InvestigationRecords({
maximumRepresentationBytes:
sizing.statistics().retainedRepresentationBytes,
});
for (const evidence of initial)
expect(records.recordEvidence(evidence).ok).toBe(true);
const session = new BinarySession(
createTestProviderRouter(() => {
throw new Error("Comparison must not launch a provider");
}),
records,
);
const harness = await createApplicationMcpHarness({
session,
serverOptions: {
environment: { ...process.env, REA_MCP_OUTPUT_SCHEMA_PROFILE: profile },
},
});
try {
const reply = await harness.client.callTool({
name: "compare_functions",
arguments: input,
});
expect(reply.isError).toBe(true);
const error = parseMcpToolError(reply).error;
analysisErrorProjectionSchema.parse(error);
expect(error).toMatchObject({
code: "resource_constraint",
details: { operation: "record_unknown", resource: "capacity" },
});
const partial = error.details?.partial_observation;
if (preexisting) {
expect(partial).toEqual({
kind: "retained-evidence",
evidence_id: computed.value.evidence.evidence_id,
});
} else {
expect(partial).toMatchObject({
kind: "unretained-evidence",
evidence: {
evidence_id: computed.value.evidence.evidence_id,
operation: "compare_functions",
},
});
const authenticated = safeParseEvidence(
(partial as { evidence: unknown }).evidence,
);
if (!authenticated.ok) throw authenticated.error;
expect(authenticated.value).toEqual(computed.value.evidence);
}
expect(session.hasEvidence(computed.value.evidence.evidence_id)).toBe(
preexisting,
);
expect(session.hasEvidence(input.left.evidence_id)).toBe(true);
expect(session.hasEvidence(input.right.evidence_id)).toBe(true);
expect(records.statistics().unknownRevisions).toBe(0);
const bundle = await harness.client.callTool({
name: "get_evidence_bundle",
arguments: { detail: "complete" },
});
expect(bundle.isError).not.toBe(true);
const result = toolContract("get_evidence_bundle").outputSchema.parse(
bundle.structuredContent,
);
const ids = parseEvidenceBundle(result.result).records.map(
({ evidence_id }) => evidence_id,
);
expect(ids.includes(computed.value.evidence.evidence_id)).toBe(
preexisting,
);
} finally {
await harness.close();
}
},
);
@@ -0,0 +1,250 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server";
import { expect, it } from "vitest";
import { z } from "zod";
import { createTestProviderRouter } from "../../fixtures/binarySession.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { BinarySession } from "../../../src/application/binary/BinarySession.js";
import { InvestigationRecords } from "../../../src/application/investigation/InvestigationRecords.js";
import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js";
import { safeParseEvidence } from "../../../src/domain/evidenceValidation.js";
import { createEvidence } from "../../../src/domain/evidence.js";
import { ok } from "../../../src/domain/result.js";
import { createServer } from "../../../src/server/createServer.js";
import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js";
import { createToolResultDelivery } from "../../../src/server/toolResult.js";
it.each(["compact", "full"] as const)(
"delivers authenticated but unretained provider Evidence inline (%s schema)",
async (profile) => {
const identity = {
id: "fixture",
name: "Fixture analysis provider",
version: "1",
};
const target = join(
await createTestTempDirectory("rea-evidence-refusal-"),
"fixture.com",
);
await writeFile(target, Buffer.from([0xb8, 0, 0x4c, 0xcd, 0x21]));
const router = createTestProviderRouter(() => ({
execute: async () =>
ok(
createAnalysisExecution("entry", identity, {
limitations: ["fixture limitation"],
}),
),
close: async () => ok(null),
}));
const records = new InvestigationRecords({ maximumRepresentationBytes: 0 });
const session = new BinarySession(router, records);
const server = createServer(
{ kind: "session", session },
{
environment: { ...process.env, REA_MCP_OUTPUT_SCHEMA_PROFILE: profile },
},
);
const client = new Client({ name: "unretained-evidence", version: "1" });
const [ct, st] = InMemoryTransport.createLinkedPair();
try {
await server.connect(st);
await client.connect(ct);
const definitions = new Map(
(await client.listTools()).tools.map((tool) => [tool.name, tool]),
);
const definition = definitions.get("address_name");
if (definition === undefined)
throw new Error("address_name was not advertised");
expect(
(
await client.callTool({
name: "open_binary",
arguments: { path: target, format: "dos-com" },
})
).isError,
).not.toBe(true);
const result = await client.callTool(
{
name: "address_name",
arguments: { address: "0x100", document: "fixture" },
},
{ toolDefinition: definition },
);
expect(result.isError).toBe(true);
const error = parseMcpToolError(result).error;
expect(error.code).toBe("resource_constraint");
expect(error.details).toMatchObject({
operation: "record_evidence",
resource: "capacity",
partial_observation: {
kind: "unretained-evidence",
evidence: {
operation: "address_name",
normalized_result: "entry",
provider: identity,
limitations: ["fixture limitation"],
},
},
});
const partial = error.details?.partial_observation as {
kind: string;
evidence: unknown;
};
const authenticated = safeParseEvidence(partial.evidence);
expect(authenticated.ok).toBe(true);
if (!authenticated.ok) throw authenticated.error;
expect(session.hasEvidence(authenticated.value.evidence_id)).toBe(false);
expect(records.statistics().evidenceRecords).toBe(0);
expect(error.details).not.toHaveProperty("evidence_reference");
const bundle = await client.callTool({
name: "get_evidence_bundle",
arguments: { detail: "complete" },
});
expect(bundle.isError).not.toBe(true);
expect(bundle.structuredContent).toMatchObject({
result: { records: [] },
});
} finally {
await client.close();
await server.close();
await session.close();
}
},
);
it("keeps an oversized unretained observation failure bounded and the connection usable", async () => {
const identity = {
id: "fixture",
name: "Fixture analysis provider",
version: "1",
};
const target = join(
await createTestTempDirectory("rea-evidence-overflow-"),
"fixture.com",
);
await writeFile(target, Buffer.from([0xb8, 0, 0x4c, 0xcd, 0x21]));
const value = "x".repeat(STDIO_DEFAULT_MAX_BUFFER_SIZE + 65536);
const router = createTestProviderRouter(() => ({
execute: async () => ok(createAnalysisExecution(value, identity)),
close: async () => ok(null),
}));
const records = new InvestigationRecords({ maximumRepresentationBytes: 0 });
const session = new BinarySession(router, records);
const server = createServer({ kind: "session", session });
const client = new Client({
name: "unretained-evidence-overflow",
version: "1",
});
const [ct, st] = InMemoryTransport.createLinkedPair();
try {
await server.connect(st);
await client.connect(ct);
expect(
(
await client.callTool({
name: "open_binary",
arguments: { path: target, format: "dos-com" },
})
).isError,
).not.toBe(true);
const result = await client.callTool(
{
name: "address_name",
arguments: { address: "0x100", document: "fixture" },
},
{ timeout: 120000 },
);
expect(result.isError).toBe(true);
const error = parseMcpToolError(result).error;
expect(error).toMatchObject({
code: "resource_constraint",
details: {
operation: "record_evidence",
resource: "capacity",
reported_limits: { maximum_representation_bytes: 0 },
partial_observation_omitted: {
reason: "mcp-response-limit",
constraint: "receive-buffer",
},
},
});
expect(error.remediation.action).toContain("not retained");
expect(error.details).not.toHaveProperty("partial_observation");
expect(error.details).not.toHaveProperty("evidence_reference");
expect(Buffer.byteLength(JSON.stringify(result))).toBeLessThan(
STDIO_DEFAULT_MAX_BUFFER_SIZE,
);
expect(records.statistics().evidenceRecords).toBe(0);
expect(
(
await client.callTool({
name: "get_evidence_bundle",
arguments: { detail: "summary" },
})
).isError,
).not.toBe(true);
} finally {
await client.close();
await server.close();
await session.close();
}
}, 120000);
it("bounds fallback when the retention error itself has oversized diagnostics", async () => {
const records = new InvestigationRecords({ maximumRepresentationBytes: 0 });
const evidence = createEvidence(
undefined,
{ id: "fixture", name: "Fixture", version: "1" },
{ operation: "address_name", parameters: {}, result: "entry" },
{ immutable: true },
);
const refused = records.recordEvidence(evidence);
if (refused.ok) throw new Error("Expected retention refusal");
refused.error.retainDiagnostics({
oversized: "x".repeat(STDIO_DEFAULT_MAX_BUFFER_SIZE + 65536),
});
const delivery = createToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE);
const server = new EvidenceMcpServer(
{ name: "retention-overflow", version: "1" },
{ capabilities: {} },
(record) => records.recordEvidence(record),
delivery,
);
server.registerTool(
"large_retention_failure",
{ inputSchema: z.object({}) },
async () => delivery.toErrorToolResult(refused.error, true),
);
const client = new Client({ name: "retention-overflow", version: "1" });
const [ct, st] = InMemoryTransport.createLinkedPair();
try {
await server.connect(st);
await client.connect(ct);
const result = await client.callTool({
name: "large_retention_failure",
arguments: {},
});
const error = parseMcpToolError(result).error;
expect(error).toMatchObject({
code: "resource_constraint",
details: {
reported_limits: {
boundary: "mcp-response",
original_error_code: "resource_constraint",
retention: "failed",
retention_error: { code: "resource_constraint" },
},
},
});
expect(Buffer.byteLength(JSON.stringify(result))).toBeLessThan(
STDIO_DEFAULT_MAX_BUFFER_SIZE,
);
expect(records.statistics().evidenceRecords).toBe(0);
} finally {
await client.close();
await server.close();
}
}, 120000);
@@ -67,7 +67,16 @@ const tracer: NativeCallTracer & { calls: number } = {
breakpoints: request.input.breakpoints.map((_, index) => ({
index,
location_count: 1,
locations: [],
locations: [
{
load_address: "0x1000",
file_address: "0x1000",
module: "Tool",
module_path: request.executable,
symbol: "open",
source_location: { file_path: null, line: 23, column: null },
},
],
})),
events: [],
resource_limit_reached: false,
@@ -133,6 +142,12 @@ it("routes observe_native_calls through MCP with schema-checked input and output
.parse(called.structuredContent).normalized_result,
);
expect(result.process).toMatchObject({ pid: 99, outcome: "exited" });
expect(result.breakpoints[0]?.locations[0]?.source_location).toEqual({
file_path: null,
line: 23,
column: null,
});
expect(result.target.environment).toEqual(
JSON.parse(
'{"__proto__":"selected literal value","constructor":"ordinary value"}',
@@ -12,6 +12,7 @@ import {
analysisViewResultSchema,
inspectAnalysisViewInputSchema,
} from "../../../src/domain/analysisView/analysisView.js";
import { XREFS_ADDRESS_PAGE_LIMITATION } from "../../../src/domain/analysisView/nativeObservationView.js";
import { NATIVE_PRIMITIVE_OBSERVATION_OPERATIONS } from "../../../src/domain/native/nativeObservation.js";
import { connectGhidraMcp } from "./ghidraMcpHarness.js";
import { createMcpSchemaValidation } from "../../fixtures/mcpSchemaValidation.js";
@@ -138,6 +139,9 @@ it("delivers native primitive and graph summaries with exact retained parents an
},
coverage: { total: 1, exhausted: true },
});
expect(
analysisViewResultSchema.parse(page.normalized_result).unknowns,
).toContain(XREFS_ADDRESS_PAGE_LIMITATION);
const caveats = await call("inspect_analysis_view", {
source,
view: { kind: "native-context", facet: "limitations" },
+26 -4
View File
@@ -709,8 +709,19 @@ describe("session filesystem path boundaries over MCP", () => {
name: "export_evidence_bundle",
arguments: { path: "relative-bundle.json" },
});
expect(exported.isError, JSON.stringify(exported.content)).toBe(true);
expect(JSON.stringify(exported.content)).toContain("absolute");
expect(parseMcpToolError(exported).error).toMatchObject({
code: "invalid_request",
details: {
operation: "export_evidence_bundle",
issues: [
expect.objectContaining({
path: ["path"],
reason: "invalid_format",
expected: "regex",
}),
],
},
});
const opened = await mcp.callTool({
name: "open_binary",
@@ -721,8 +732,19 @@ describe("session filesystem path boundaries over MCP", () => {
name: "close_binary",
arguments: { snapshot_path: "relative-analysis.json" },
});
expect(closed.isError, JSON.stringify(closed.content)).toBe(true);
expect(JSON.stringify(closed.content)).toContain("absolute");
expect(parseMcpToolError(closed).error).toMatchObject({
code: "invalid_request",
details: {
operation: "close_binary",
issues: [
expect.objectContaining({
path: ["snapshot_path"],
reason: "invalid_format",
expected: "regex",
}),
],
},
});
await mcp.callTool({ name: "close_binary", arguments: {} });
}, 10_000);
});
+22 -14
View File
@@ -383,7 +383,7 @@ it("attributes rejected WASM paths to their selected fields before provider work
},
});
onTestFinished(harness.close);
for (const [input, paths] of [
const relativeCases = [
[{ path: "relative.wasm" }, [["path"]]],
[
{ path: value.artifact.path, glue_paths: ["relative.js"] },
@@ -412,7 +412,9 @@ it("attributes rejected WASM paths to their selected fields before provider work
},
[["path"], ["glue_paths", 0], ["candidate_paths", 0]],
],
] as const) {
] as const;
const checked: string[] = [];
for (const [input, paths] of relativeCases) {
const response = await harness.client.callTool({
name: "inspect_wasm_artifact",
arguments: input,
@@ -423,14 +425,19 @@ it("attributes rejected WASM paths to their selected fields before provider work
.object({ issues: z.array(z.unknown()) })
.parse(failure.details);
expect(issues).toEqual(
paths.map((path) => ({
path,
reason: "invalid_format",
expected: "regex",
message: "must be an absolute local filesystem path",
})),
paths.map((path) =>
expect.objectContaining({
path,
reason: "invalid_format",
expected: "regex",
}),
),
);
checked.push(paths.map((path) => path.join(".")).join("+"));
}
expect(checked).toEqual(
expect.arrayContaining(["path", "glue_paths.0", "candidate_paths.0"]),
);
expect(inspect).not.toHaveBeenCalled();
const valid = await harness.client.callTool({
name: "inspect_wasm_artifact",
@@ -760,12 +767,13 @@ it("rejects malformed paths, refuses WASM summary delivery before provider work
expect(
z.object({ issues: z.array(z.unknown()) }).parse(rejected.details).issues,
).toEqual(
[["path"], ["glue_paths", 0]].map((path) => ({
path,
reason: "invalid_format",
expected: "regex",
message: "must be an absolute local filesystem path",
})),
[["path"], ["glue_paths", 0]].map((path) =>
expect.objectContaining({
path,
reason: "invalid_format",
expected: "regex",
}),
),
);
expect(inspect).not.toHaveBeenCalled();
for (const input of [
@@ -124,3 +124,15 @@ it.each(["duplicate-first", "duplicate-last"])(
expect(result.error?.message).toMatch(/Duplicate MCP tool/u);
},
);
it("preserves cancellation received during final transport cleanup", async () => {
const result = await probe("cancel-finalization");
expect(result.accepted).toBe(false);
expect(result.error?.message).toBe("cancelled during final cleanup");
});
it("retains a work failure when cancellation arrives during cleanup", async () => {
const result = await probe("failure-cancel-finalization");
expect(result.accepted).toBe(false);
expect(result.error?.message).toBe("work failed before cancellation");
});
@@ -0,0 +1,66 @@
import { EventEmitter } from "node:events";
import { access, mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { PassThrough } from "node:stream";
import { expect, it } from "vitest";
import { createPackageVerifierCommands } from "../../../scripts/lib/verify-package-commands.mjs";
import { OwnedMcpStdioTransport } from "../../../src/process/OwnedMcpStdioTransport.js";
it("retains the workspace until an owned protocol cleanup can be retried", async () => {
const path = await mkdtemp(join(tmpdir(), "rea-verifier-transport-"));
const process = Object.assign(new EventEmitter(), {
pid: 1,
stdout: new PassThrough(),
stderr: new PassThrough(),
stdin: new PassThrough(),
exitCode: null,
signalCode: null,
kill: () => true,
});
let refuse = true;
const transport = new OwnedMcpStdioTransport(
{ command: "fixture", arguments: [] },
{
provider: "fixture",
label: "Fixture",
maxFrameBytes: 1024,
maxOperationBytes: 4096,
},
async ({ runId }) => ({
process,
ownership: { runId, leaderPid: 1, processGroupId: 1 },
cleanup: async () => {
if (refuse)
return { cleaned: false, reason: "injected cleanup failure" };
process.emit("close", 0, null);
process.stdout.end();
process.stderr.end();
process.stdin.end();
return { cleaned: true, signaled: false };
},
}),
);
const commands = await createPackageVerifierCommands({
workspace: {
path,
close: () => rm(path, { recursive: true, force: true }),
},
});
commands.adopt(transport, "protocol child");
try {
await transport.start();
// The group can still contain work after the leader's exit.
process.emit("exit", 0, null);
await expect(commands.close()).rejects.toMatchObject({
path,
cleanupOwner: commands,
});
await expect(access(path)).resolves.toBeUndefined();
refuse = false;
await commands.close();
await expect(access(path)).rejects.toMatchObject({ code: "ENOENT" });
} finally {
await rm(path, { recursive: true, force: true });
}
});
@@ -0,0 +1,205 @@
import { execFileSync } from "node:child_process";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it } from "vitest";
import {
PYTHON_HELPERS,
PYTHON_IMPORTS,
} from "../../../src/domain/desktop/appExportPython.js";
import {
detectStateFormat,
textualView,
} from "../../../src/domain/desktop/stateFormats.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
const arrays = "[".repeat(1500) + "1" + "]".repeat(1500);
const objects = '{"a":'.repeat(1500) + "1" + "}".repeat(1500);
const cases = [
["discarded arrays", `{"value":${arrays},"value":"done"}`, true],
["discarded objects", `{"value":${objects},"value":"done"}`, true],
["escaped duplicate key", `{"val\\u0075e":${arrays},"value":"done"}`, true],
["retained deep value", `{"value":${arrays}}`, false],
[
"retained deep inexact number",
`{"value":${"[".repeat(600)}9007199254740993${"]".repeat(600)}}`,
false,
],
[
"deep overwritten inexact number",
`{"value":${"[".repeat(600)}{"x":9007199254740993,"x":1}${"]".repeat(600)}}`,
false,
],
[
"deep empty-key overwrite",
`{"value":${"[".repeat(600)}{"":9007199254740993,"":1}${"]".repeat(600)}}`,
false,
],
[
"deep multiple inexact keys with one overwritten",
`{"value":${"[".repeat(600)}{"a":9007199254740993,"b":9007199254740993,"a":1}${"]".repeat(600)}}`,
false,
],
[
"deep multiple inexact keys all overwritten",
`{"value":${"[".repeat(600)}{"a":9007199254740993,"b":9007199254740993,"a":1,"b":2}${"]".repeat(600)}}`,
false,
],
[
"deep inexact key replaced by inexact",
`{"value":${"[".repeat(600)}{"a":9007199254740993,"a":9007199254740995}${"]".repeat(600)}}`,
false,
],
[
"deep escaped duplicate overwritten",
`{"value":${"[".repeat(600)}{"\\u0061":9007199254740993,"a":1}${"]".repeat(600)}}`,
false,
],
[
"discarded deep inexact subtree",
`{"value":${"[".repeat(600)}9007199254740993${"]".repeat(600)},"value":"done"}`,
true,
],
["last admitted depth", "[".repeat(500) + "1" + "]".repeat(500), true],
["empty container at limit", "[".repeat(501) + "]".repeat(501), true],
["empty container over limit", "[".repeat(502) + "]".repeat(502), false],
["trailing comma", `{"value":${arrays},"value":"done",}`, false],
["missing value", `{"value":${arrays},"value":}`, false],
["wrong closing bracket", `{"value":${arrays},"value":"done"]`, false],
] as const;
it.each(cases)("reads JSON with %s", async (_name, text, readable) => {
const root = await createTestTempDirectory("desktop-json-depth-");
const script = join(root, "read.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}${PYTHON_HELPERS}
text = json.load(sys.stdin)
print(json.dumps(_text_view("state.json", text, len(text.encode("utf-8")))))
`,
);
const output = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", "-X", "utf8", script],
{ input: JSON.stringify(text), encoding: "utf8" },
),
) as { format: string; view: unknown; error?: string };
const bytes = Buffer.from(text);
const format = detectStateFormat("state.json", bytes);
if (format !== "json" && format !== "text")
throw new Error(`Unexpected native format ${format}`);
const native = textualView(format, bytes);
expect(format === "json" && native.ok).toBe(readable);
expect(output.format).toBe(format);
if (native.ok) {
expect(output.error).toBeUndefined();
// Compare JSON values on their wire form, independent of object prototypes.
expect(JSON.stringify(output.view)).toBe(JSON.stringify(native.value));
if (
readable &&
(_name.startsWith("discarded") || _name === "escaped duplicate key")
)
expect(output.view).toEqual({ value: "done" });
} else {
expect(output.view).toBeNull();
expect(output.error).toBe(native.error);
}
});
it("bounds construction before refusing retained excessive JSON depth", async () => {
const root = await createTestTempDirectory("desktop-json-construction-");
const script = join(root, "bounded.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}${PYTHON_HELPERS}
import tracemalloc
text = "[" * 20_000 + "0" + "]" * 20_000
recursion_limit = sys.getrecursionlimit()
tracemalloc.start()
entry = _json_view(text, len(text))
peak = tracemalloc.get_traced_memory()[1]
tracemalloc.stop()
print(json.dumps({"entry": entry, "peak": peak, "recursion_unchanged": sys.getrecursionlimit() == recursion_limit}))
`,
);
const output = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", "-X", "utf8", script],
{ encoding: "utf8" },
),
) as { entry: unknown; peak: number; recursion_unchanged: boolean };
expect(output.entry).toEqual({
format: "json",
view: null,
error: "nested deeper than the 500 levels a JSON view keeps",
});
expect(output.recursion_unchanged).toBe(true);
// Python heap only, excluding source construction and interpreter/OS memory.
// A value retained only to refuse it must not allocate every input container.
expect(output.peak).toBeLessThan(1024 * 1024);
});
it("bounds allocations when a very deep value is overwritten", async () => {
const root = await createTestTempDirectory("desktop-json-memory-");
const script = join(root, "read.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}\nimport tracemalloc\n${PYTHON_HELPERS}
text = '{"value":' + '[' * 200000 + '1' + ']' * 200000 + ',"value":"done"}'
tracemalloc.start()
result = _json_view(text, len(text))
_, peak = tracemalloc.get_traced_memory()
print(json.dumps({"view": result["view"], "peak": peak}))
`,
);
const measured = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", script],
{ encoding: "utf8", timeout: 30000 },
),
) as { view: unknown; peak: number };
expect(measured.view).toEqual({ value: "done" });
expect(measured.peak).toBeLessThan(8 * 1024 * 1024);
}, 40000);
it("bounds a wide container already beyond the retained view depth", async () => {
const root = await createTestTempDirectory("desktop-json-wide-depth-");
const script = join(root, "read.py");
const imports = PYTHON_IMPORTS.split("\n")
.filter((line) => !line.includes("agentenv_protocol"))
.join("\n");
await writeFile(
script,
`${imports}\nimport tracemalloc\n${PYTHON_HELPERS}
text = '[' * 501 + ','.join(['"abc"'] * 600000) + ']' * 501
tracemalloc.start()
result = _json_view(text, len(text))
_, peak = tracemalloc.get_traced_memory()
print(json.dumps({"error": result.get("error"), "peak": peak}))
`,
);
const measured = JSON.parse(
execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", script],
{ encoding: "utf8", timeout: 30000 },
),
) as { error: string; peak: number };
expect(measured.error).toBe(
"nested deeper than the 500 levels a JSON view keeps",
);
expect(measured.peak).toBeLessThan(8 * 1024 * 1024);
}, 40000);
@@ -34,7 +34,7 @@ const execPython = (
try {
writeFileSync(script, args[codeIndex + 1]!, "utf8");
return execFileSync(
"python3",
process.platform === "win32" ? "python" : "python3",
[
"-X",
"utf8",
@@ -375,15 +375,16 @@ it("checks text for JSON the way REA does, token by token, without parsing it",
"-I",
"-B",
"-c",
`${imports}${PYTHON_HELPERS}\nprint(json.dumps([_is_json(text) for text in json.load(sys.stdin)]))`,
`${imports}${PYTHON_HELPERS}\ntexts = json.load(sys.stdin)\nprint(json.dumps({"syntax": [_read_json(text, build=False) is not _INVALID_JSON for text in texts], "built": [_read_json(text) is not _INVALID_JSON for text in texts]}))`,
],
{ input: JSON.stringify(texts), encoding: "utf8" },
),
) as boolean[];
) as { syntax: boolean[]; built: boolean[] };
const parsed = texts.map((text) => safeParseJson(text).ok);
expect(texts.map((text) => isJsonText(text))).toEqual(parsed);
expect(python).toEqual(parsed);
expect(python.syntax).toEqual(parsed);
expect(python.built).toEqual(parsed);
expect(parsed.filter(Boolean)).toHaveLength(10);
});
@@ -445,9 +446,13 @@ print(json.dumps([_text_view("settings.ini", text, len(text.encode()))["view"]["
const root = await createTestTempDirectory("rea-keyfile-python-");
const script = join(root, "fixture.py");
await writeFile(script, program, "utf8");
const output = execFileSync("python3", ["-I", "-B", "-X", "utf8", script], {
input: JSON.stringify(inputs),
encoding: "utf8",
});
const output = execFileSync(
process.platform === "win32" ? "python" : "python3",
["-I", "-B", "-X", "utf8", script],
{
input: JSON.stringify(inputs),
encoding: "utf8",
},
);
expect(JSON.parse(output)).toEqual(inputs.map(keyfileView));
});
@@ -77,10 +77,16 @@ test("keeps each Kuna no-pseudocode observation's limitations on its batch item"
const broken = await direct("broken");
const importSlot = await direct("puts");
expect(broken.normalized_result).toBeNull();
expect(broken.raw_result).toMatchObject({
pseudocode_absence: { kind: "provider-failure" },
});
expect(broken.limitations).toContain(
"Kuna could not decompile this function: Flow exceeded maximum instructions",
);
expect(importSlot.normalized_result).toBeNull();
expect(importSlot.raw_result).toMatchObject({
pseudocode_absence: { kind: "not-in-export" },
});
const batch = parseEvidence(
(
@@ -0,0 +1,459 @@
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterAll, expect, it as test, vi } from "vitest";
import { AnalysisProviderRegistry } from "../../../../src/application/binary/AnalysisProviderRegistry.js";
import { composeBinarySession } from "../../../../src/application/binary/BinarySessionComposition.js";
import { parseEvidence } from "../../../../src/domain/evidence.js";
import { KunaProvider } from "../../../../src/kuna/KunaProvider.js";
import { elfNoteFixture } from "../../../../src/native/elf/ElfNotes.fixture.js";
import { createApplicationMcpHarness } from "../../../fixtures/applicationMcpHarness.js";
import { advertisedMcpToolSchemas } from "../../../fixtures/advertisedMcpTool.js";
import { createMcpSchemaValidation } from "../../../fixtures/mcpSchemaValidation.js";
import { awaitAcknowledgement } from "../../../fixtures/lifecycleAcknowledgement.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { fakeKunaCalls, writeFakeKuna } from "./fakeKuna.js";
import { FUNCTIONS, unprovenCleanupLauncher } from "./kunaSessionFixture.js";
const it = test.skipIf(process.platform === "win32");
const schemas = createMcpSchemaValidation();
afterAll(() => schemas.clear());
const expectPriorFailureInJournal = async (output: string) => {
type Attempt = {
address?: string;
pseudocode_absence?: { kind: string; reason: string };
limitations?: string[];
};
const journal = (
await readFile(join(output, "decompile-journal.jsonl"), "utf8")
)
.trim()
.split("\n")
.map((line) => JSON.parse(line) as Attempt);
const index = JSON.parse(
await readFile(join(output, "decompile-index.json"), "utf8"),
) as { functions: Attempt[] };
const failed = journal.find(
(row) =>
row.address === "0x1080" &&
row.pseudocode_absence?.kind === "provider-failure",
);
const latest = index.functions.find((row) => row.address === "0x1080");
expect(failed?.pseudocode_absence?.reason).toEqual(expect.any(String));
expect(failed?.limitations).toContain(failed?.pseudocode_absence?.reason);
expect(latest?.limitations).not.toContain(failed?.pseudocode_absence?.reason);
};
it.each(["compact", "full"] as const)(
"guides resume and retries only reported project failures (%s)",
async (profile) => {
const root = await createTestTempDirectory("rea-directory-independent-");
const kuna = await writeFakeKuna(root, { functions: FUNCTIONS });
const target = join(root, "program");
const output = join(root, "output");
await writeFile(target, elfNoteFixture({ bits: 64, type: 2 }));
const provider = new KunaProvider(
{ REA_KUNA_COMMAND: kuna, REA_KUNA_CACHE_DIR: join(root, "cache") },
{ platform: process.platform },
);
const session = composeBinarySession(
new AnalysisProviderRegistry([provider], "kuna"),
);
const harness = await createApplicationMcpHarness({
session,
schemaValidator: schemas,
serverOptions: {
environment: {
...process.env,
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
REA_MCP_OUTPUT_SCHEMA_PROFILE: profile,
},
},
});
try {
const call = async (name: string, args: Record<string, unknown>) => {
const schema = await advertisedMcpToolSchemas(
harness.client,
name,
schemas.assertions,
);
const result = await harness.client.callTool(
{ name, arguments: args },
{ toolDefinition: schema.toolDefinition },
);
expect(result.isError, JSON.stringify(result)).not.toBe(true);
expect(
schema.ajv.validate(schema.outputSchema, result.structuredContent),
JSON.stringify(schema.ajv.errors),
).toBe(true);
return result.structuredContent;
};
await call("open_binary", { path: target, provider_id: "kuna" });
await call("procedure_assembly", { procedure: "main" });
const initial = parseEvidence(
await call("decompile_all", { output_directory: output }),
);
expect(initial.normalized_result).toMatchObject({
counts: { written: 2, failed: 2 },
failures: expect.arrayContaining([
expect.objectContaining({
address: "0x1080",
message: expect.stringContaining("retry_failed"),
}),
expect.objectContaining({
address: "0x2000",
code: "no_pseudocode",
}),
]),
});
const evidence = parseEvidence(
await call("decompile_all", {
output_directory: output,
resume: true,
retry_failed: "independent_fresh",
}),
);
expect(evidence.normalized_result).toMatchObject({
counts: { written: 1, failed: 1, skipped: 2 },
});
const index = JSON.parse(
await readFile(join(output, "decompile-index.json"), "utf8"),
) as {
functions: Array<{
address: string;
status: string;
provider_source?: unknown;
}>;
};
expect(
index.functions.find((item) => item.address === "0x1080"),
).toMatchObject({
status: "written",
provider_source: {
status: "reported",
source: { kind: "single-function-decompile" },
},
});
expect(
index.functions.find((item) => item.address === "0x2000"),
).toMatchObject({
status: "failed",
pseudocode_absence: { kind: "not-in-export" },
});
const journal = (
await readFile(join(output, "decompile-journal.jsonl"), "utf8")
)
.trim()
.split("\n")
.map(
(line) =>
JSON.parse(line) as {
address?: string;
status?: string;
provider_source?: unknown;
},
);
expect(journal.filter((row) => row.address === "0x1080")).toEqual([
expect.objectContaining({
status: "failed",
pseudocode_absence: expect.objectContaining({
kind: "provider-failure",
}),
failure: expect.objectContaining({
message: expect.stringContaining("retry_failed"),
}),
}),
expect.objectContaining({
status: "failed",
pseudocode_absence: expect.objectContaining({
kind: "provider-failure",
}),
provider_source: {
status: "reported",
source: expect.objectContaining({ kind: "file-range" }),
},
}),
expect.objectContaining({
status: "written",
provider_source: {
status: "reported",
source: { kind: "single-function-decompile" },
},
}),
]);
await expectPriorFailureInJournal(output);
expect(
(await fakeKunaCalls(kuna))
.filter(({ argv }) => argv[0] === "decompile")
.map(({ argv }) => argv[2]),
).toEqual(["0x1080"]);
} finally {
await harness.close();
const closed = await provider.closeOwnedResources();
if (!closed.ok) throw closed.error;
}
},
);
it("retains the project failure when the independent retry times out", async () => {
const root = await createTestTempDirectory(
"rea-directory-independent-timeout-",
);
const kuna = await writeFakeKuna(root, {
functions: FUNCTIONS,
decompile: "hang",
});
const target = join(root, "program");
const output = join(root, "output");
await writeFile(target, elfNoteFixture({ bits: 64, type: 2 }));
const provider = new KunaProvider(
{
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
REA_KUNA_TIMEOUT_MS: "300",
},
{ platform: process.platform },
);
const session = composeBinarySession(
new AnalysisProviderRegistry([provider], "kuna"),
);
const harness = await createApplicationMcpHarness({ session });
try {
const call = async (name: string, args: Record<string, unknown>) => {
const result = await harness.client.callTool({ name, arguments: args });
expect(result.isError, JSON.stringify(result)).not.toBe(true);
return result.structuredContent;
};
await call("open_binary", { path: target, provider_id: "kuna" });
await call("procedure_assembly", { procedure: "main" });
const evidence = parseEvidence(
await call("decompile_all", {
output_directory: output,
address_start: "0x1080",
address_end: "0x1080",
retry_failed: "independent_fresh",
}),
);
expect(evidence.normalized_result).toMatchObject({
counts: { written: 0, failed: 1 },
failures: [{ address: "0x1080", code: "deadline" }],
});
const journal = (
await readFile(join(output, "decompile-journal.jsonl"), "utf8")
)
.trim()
.split("\n")
.map(
(line) =>
JSON.parse(line) as {
address?: string;
status?: string;
failure?: { code: string };
},
);
const attempts = journal.filter((row) => row.address === "0x1080");
expect(attempts).toHaveLength(2);
expect(attempts[0]).toMatchObject({
status: "failed",
pseudocode_absence: { kind: "provider-failure" },
});
expect(attempts[1]).toMatchObject({
status: "failed",
failure: { code: "deadline" },
});
} finally {
await harness.close();
const closed = await provider.closeOwnedResources();
if (!closed.ok) throw closed.error;
}
});
it("retains the project failure when cancellation stops the independent retry", async () => {
const root = await createTestTempDirectory(
"rea-directory-independent-cancel-",
);
const kuna = await writeFakeKuna(root, {
functions: FUNCTIONS,
decompile: "hang",
});
const target = join(root, "program");
const output = join(root, "output");
await writeFile(target, elfNoteFixture({ bits: 64, type: 2 }));
const provider = new KunaProvider(
{
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
REA_KUNA_TIMEOUT_MS: "10000",
},
{ platform: process.platform },
);
const session = composeBinarySession(
new AnalysisProviderRegistry([provider], "kuna"),
);
const harness = await createApplicationMcpHarness({ session });
const controller = new AbortController();
let pending: Promise<unknown> | undefined;
try {
expect(
(
await harness.client.callTool({
name: "open_binary",
arguments: { path: target, provider_id: "kuna" },
})
).isError,
).not.toBe(true);
expect(
(
await harness.client.callTool({
name: "procedure_assembly",
arguments: { procedure: "main" },
})
).isError,
).not.toBe(true);
pending = harness.client
.callTool(
{
name: "decompile_all",
arguments: {
output_directory: output,
address_start: "0x1080",
address_end: "0x1080",
retry_failed: "independent_fresh",
},
},
{ signal: controller.signal, timeout: 15000 },
)
.catch(() => undefined);
const launched = vi.waitFor(
async () => {
expect(
(await fakeKunaCalls(kuna)).some(
({ argv }) => argv[0] === "decompile" && argv[2] === "0x1080",
),
).toBe(true);
},
{ timeout: 5000, interval: 20 },
);
await awaitAcknowledgement(
launched,
pending,
"independent retry launch",
6000,
);
controller.abort();
await pending;
expect(
(await harness.client.callTool({ name: "close_binary", arguments: {} }))
.isError,
).not.toBe(true);
const journal = (
await readFile(join(output, "decompile-journal.jsonl"), "utf8")
)
.trim()
.split("\n")
.map((line) => JSON.parse(line) as { address?: string; status?: string });
expect(journal.filter((row) => row.address === "0x1080")).toEqual([
expect.objectContaining({
status: "failed",
pseudocode_absence: expect.objectContaining({
kind: "provider-failure",
}),
}),
]);
} finally {
controller.abort();
await pending;
await harness.close();
const closed = await provider.closeOwnedResources();
if (!closed.ok) throw closed.error;
}
});
it("retains failed attempt and cleanup ownership when retry stop is unproven", async () => {
const root = await createTestTempDirectory(
"rea-directory-independent-cleanup-",
);
const kuna = await writeFakeKuna(root, {
functions: FUNCTIONS,
decompile: "hang",
});
const target = join(root, "program");
const output = join(root, "output");
await writeFile(target, elfNoteFixture({ bits: 64, type: 2 }));
const unproven = unprovenCleanupLauncher("decompile");
const provider = new KunaProvider(
{
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
REA_KUNA_TIMEOUT_MS: "300",
},
{
platform: process.platform,
launcher: unproven.launcher,
singleFunctionRuns: 1,
},
);
const session = composeBinarySession(
new AnalysisProviderRegistry([provider], "kuna"),
);
const harness = await createApplicationMcpHarness({ session });
try {
expect(
(
await harness.client.callTool({
name: "open_binary",
arguments: { path: target, provider_id: "kuna" },
})
).isError,
).not.toBe(true);
expect(
(
await harness.client.callTool({
name: "procedure_assembly",
arguments: { procedure: "main" },
})
).isError,
).not.toBe(true);
const failed = await harness.client.callTool({
name: "decompile_all",
arguments: {
output_directory: output,
address_start: "0x1080",
address_end: "0x1080",
retry_failed: "independent_fresh",
},
});
expect(failed.isError).toBe(true);
expect(
JSON.parse((failed.content[0] as { text: string }).text),
).toMatchObject({ error: { code: "cleanup_incomplete" } });
const journal = (
await readFile(join(output, "decompile-journal.jsonl"), "utf8")
)
.trim()
.split("\n")
.map((line) => JSON.parse(line) as { address?: string; status?: string });
expect(journal.filter((row) => row.address === "0x1080")).toEqual([
expect.objectContaining({
status: "failed",
pseudocode_absence: expect.objectContaining({
kind: "provider-failure",
}),
}),
]);
expect(
(await harness.client.callTool({ name: "close_binary", arguments: {} }))
.isError,
).toBe(true);
unproven.release();
expect(
(await harness.client.callTool({ name: "close_binary", arguments: {} }))
.isError,
).not.toBe(true);
} finally {
unproven.release();
await harness.close();
const closed = await provider.closeOwnedResources();
if (!closed.ok) throw closed.error;
}
});
@@ -0,0 +1,333 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterAll, expect, it as test, vi } from "vitest";
import { AnalysisProviderRegistry } from "../../../../src/application/binary/AnalysisProviderRegistry.js";
import { composeBinarySession } from "../../../../src/application/binary/BinarySessionComposition.js";
import { parseEvidence } from "../../../../src/domain/evidence.js";
import { KunaProvider } from "../../../../src/kuna/KunaProvider.js";
import { KunaRunSlots } from "../../../../src/kuna/KunaRunSlots.js";
import { elfNoteFixture } from "../../../../src/native/elf/ElfNotes.fixture.js";
import { createApplicationMcpHarness } from "../../../fixtures/applicationMcpHarness.js";
import { advertisedMcpToolSchemas } from "../../../fixtures/advertisedMcpTool.js";
import { createMcpSchemaValidation } from "../../../fixtures/mcpSchemaValidation.js";
import { awaitAcknowledgement } from "../../../fixtures/lifecycleAcknowledgement.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import {
fakeKunaCalls,
setFakeKunaBehavior,
writeFakeKuna,
} from "./fakeKuna.js";
import { FUNCTIONS, unprovenCleanupLauncher } from "./kunaSessionFixture.js";
const it = test.skipIf(process.platform === "win32");
const schemas = createMcpSchemaValidation();
afterAll(() => schemas.clear());
it("runs a cold independent request without starting a project export", async () => {
const root = await createTestTempDirectory("rea-kuna-independent-cold-");
const kuna = await writeFakeKuna(root, { functions: FUNCTIONS });
const target = join(root, "program");
await writeFile(target, elfNoteFixture({ bits: 64, type: 2 }));
const provider = new KunaProvider(
{
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
REA_KUNA_TIMEOUT_MS: "300",
},
{ platform: process.platform },
);
const session = composeBinarySession(
new AnalysisProviderRegistry([provider], "kuna"),
);
const harness = await createApplicationMcpHarness({ session });
try {
const call = async (name: string, args: Record<string, unknown>) => {
const result = await harness.client.callTool({ name, arguments: args });
expect(result.isError, JSON.stringify(result)).not.toBe(true);
return result.structuredContent;
};
await call("open_binary", { path: target, provider_id: "kuna" });
const evidence = parseEvidence(
await call("procedure_pseudo_code", {
procedure: "main",
analysis: "independent_fresh",
}),
);
expect(evidence.raw_result).toMatchObject({
source: { kind: "single-function" },
project_export: { state: "not-started" },
});
const calls = await fakeKunaCalls(kuna);
expect(calls.filter(({ argv }) => argv[0] === "functions")).toHaveLength(1);
expect(calls.filter(({ argv }) => argv[0] === "decompile")).toHaveLength(1);
expect(
calls.filter(({ argv }) => argv[0] === "decompile-project"),
).toHaveLength(0);
await setFakeKunaBehavior(kuna, {
functions: FUNCTIONS,
decompile: "hang",
});
const timeout = await harness.client.callTool({
name: "procedure_pseudo_code",
arguments: { procedure: "main", analysis: "independent_fresh" },
});
expect(timeout.isError).toBe(true);
const failure = JSON.parse(
(timeout.content[0] as { text: string }).text,
) as {
error: { code: string; remediation: { action: string } };
};
expect(failure.error).toMatchObject({
code: "provider_timeout",
remediation: {
action:
"Raise REA_KUNA_TIMEOUT_MS in the environment that starts REA, then restart REA; it reads Kuna settings when it starts.",
},
details: { timeout_ms: 300 },
});
expect(
(await fakeKunaCalls(kuna)).filter(
({ argv }) => argv[0] === "decompile-project",
),
).toHaveLength(0);
await setFakeKunaBehavior(kuna, { functions: FUNCTIONS });
await call("procedure_assembly", { procedure: "main" });
expect(
(await fakeKunaCalls(kuna)).filter(
({ argv }) => argv[0] === "decompile-project",
),
).toHaveLength(1);
} finally {
await harness.close();
const closed = await provider.closeOwnedResources();
if (!closed.ok) throw closed.error;
}
});
it.each(["compact", "full"] as const)(
"keeps the failed project observation when an explicit fresh retry succeeds (%s)",
async (profile) => {
const root = await createTestTempDirectory("rea-kuna-independent-");
const kuna = await writeFakeKuna(root, { functions: FUNCTIONS });
const target = join(root, "program");
await writeFile(target, elfNoteFixture({ bits: 64, type: 2 }));
const provider = new KunaProvider(
{ REA_KUNA_COMMAND: kuna, REA_KUNA_CACHE_DIR: join(root, "cache") },
{ platform: process.platform },
);
const session = composeBinarySession(
new AnalysisProviderRegistry([provider], "kuna"),
);
const harness = await createApplicationMcpHarness({
session,
schemaValidator: schemas,
serverOptions: {
environment: {
...process.env,
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
REA_MCP_OUTPUT_SCHEMA_PROFILE: profile,
},
},
});
try {
const call = async (name: string, args: Record<string, unknown>) => {
const schema = await advertisedMcpToolSchemas(
harness.client,
name,
schemas.assertions,
);
const result = await harness.client.callTool(
{ name, arguments: args },
{ toolDefinition: schema.toolDefinition },
);
expect(result.isError, JSON.stringify(result)).not.toBe(true);
expect(
schema.ajv.validate(schema.outputSchema, result.structuredContent),
JSON.stringify(schema.ajv.errors),
).toBe(true);
return result.structuredContent;
};
await call("open_binary", { path: target, provider_id: "kuna" });
await call("procedure_assembly", { procedure: "broken" });
const original = parseEvidence(
await call("procedure_pseudo_code", { procedure: "broken" }),
);
expect(original.normalized_result).toBeNull();
expect(original.limitations).toEqual(
expect.arrayContaining([expect.stringContaining("independent_fresh")]),
);
const status = (await call("binary_session", { detail: "summary" })) as {
result: {
capabilities: Array<{
groups: Array<{
operations: string[];
independent_decompile?: boolean;
}>;
}>;
};
};
expect(
status.result.capabilities
.flatMap((set) => set.groups)
.find((group) => group.operations.includes("procedure_pseudo_code")),
).toMatchObject({ independent_decompile: true });
const first = parseEvidence(
await call("procedure_pseudo_code", {
procedure: "broken",
analysis: "independent_fresh",
}),
);
const second = parseEvidence(
await call("procedure_pseudo_code", {
procedure: "broken",
analysis: "independent_fresh",
}),
);
expect(first.normalized_result).toEqual(
expect.stringContaining("single-function"),
);
expect(first.parameters.analysis).toBe("independent_fresh");
expect(first.evidence_id).not.toBe(original.evidence_id);
expect(second.evidence_id).toBe(first.evidence_id);
expect(
(await fakeKunaCalls(kuna)).filter(
({ argv }) => argv[0] === "decompile",
),
).toHaveLength(2);
const after = parseEvidence(
await call("procedure_pseudo_code", { procedure: "broken" }),
);
expect(after.normalized_result).toBeNull();
const bundle = (await call("get_evidence_bundle", {})) as {
result: { records: Array<{ evidence_id: string }> };
};
expect(bundle.result.records.map((record) => record.evidence_id)).toEqual(
expect.arrayContaining([original.evidence_id, first.evidence_id]),
);
} finally {
await harness.close();
const closed = await provider.closeOwnedResources();
if (!closed.ok) throw closed.error;
}
},
);
it("retains the independent run slot until an unproven stop is resolved", async () => {
const root = await createTestTempDirectory("rea-kuna-independent-cleanup-");
const kuna = await writeFakeKuna(root, {
functions: FUNCTIONS,
decompile: "hang",
});
const target = join(root, "program");
await writeFile(target, elfNoteFixture({ bits: 64, type: 2 }));
const unproven = unprovenCleanupLauncher("decompile");
const provider = new KunaProvider(
{
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
REA_KUNA_TIMEOUT_MS: "300",
},
{
platform: process.platform,
launcher: unproven.launcher,
singleFunctionRuns: 1,
},
);
const session = composeBinarySession(
new AnalysisProviderRegistry([provider], "kuna"),
);
const harness = await createApplicationMcpHarness({
session,
schemaValidator: schemas,
});
const caller = new AbortController();
let restoreAcquire: (() => void) | undefined;
let queued: Promise<unknown> | undefined;
try {
const call = (
name: string,
args: Record<string, unknown>,
signal?: AbortSignal,
) =>
harness.client.callTool(
{ name, arguments: args },
{ ...(signal === undefined ? {} : { signal }), timeout: 5000 },
);
expect(
(await call("open_binary", { path: target, provider_id: "kuna" }))
.isError,
).not.toBe(true);
expect(
(await call("procedure_assembly", { procedure: "broken" })).isError,
).not.toBe(true);
const first = await call("procedure_pseudo_code", {
procedure: "broken",
analysis: "independent_fresh",
});
expect(first.isError).toBe(true);
const failure = JSON.parse((first.content[0] as { text: string }).text) as {
error: { code: string };
};
expect(failure.error.code).toBe("cleanup_incomplete");
const acquire = KunaRunSlots.prototype.acquire;
let arrived!: () => void;
const admission = new Promise<void>((resolve) => {
arrived = resolve;
});
let slotSettled = false;
const spy = vi
.spyOn(KunaRunSlots.prototype, "acquire")
.mockImplementation(function (this: KunaRunSlots, signal) {
const result = acquire.call(this, signal);
void result.then(() => {
slotSettled = true;
});
arrived();
return result;
});
restoreAcquire = () => spy.mockRestore();
queued = call(
"procedure_pseudo_code",
{ procedure: "main", analysis: "independent_fresh" },
caller.signal,
).catch(() => undefined);
await awaitAcknowledgement(
admission,
queued,
"independent run slot admission",
);
await new Promise<void>((resolve) => setImmediate(resolve));
expect(slotSettled).toBe(false);
expect(
(await fakeKunaCalls(kuna)).filter(({ argv }) => argv[0] === "decompile"),
).toHaveLength(1);
caller.abort();
await queued;
restoreAcquire();
restoreAcquire = undefined;
expect((await call("close_binary", {})).isError).toBe(true);
unproven.release();
expect((await call("close_binary", {})).isError).not.toBe(true);
await setFakeKunaBehavior(kuna, { functions: FUNCTIONS });
expect(
(await call("open_binary", { path: target, provider_id: "kuna" }))
.isError,
).not.toBe(true);
const recovered = await call("procedure_pseudo_code", {
procedure: "main",
analysis: "independent_fresh",
});
expect(recovered.isError).not.toBe(true);
expect(
(await fakeKunaCalls(kuna)).filter(({ argv }) => argv[0] === "decompile"),
).toHaveLength(2);
} finally {
caller.abort();
unproven.release();
await queued;
restoreAcquire?.();
await harness.close();
const closed = await provider.closeOwnedResources();
if (!closed.ok) throw closed.error;
}
});
@@ -441,9 +441,21 @@ it("reports a failed function and an import slot as unavailable text, not errors
procedure: "broken",
});
expect(broken.result).toBeNull();
expect(broken.limitations.at(-1)).toBe(
expect(broken.limitations).toContain(
"Kuna could not decompile this function: Flow exceeded maximum instructions",
);
const dossier = await run(client, "analyze_function", {
procedure: "broken",
});
expect(dossier.result).toMatchObject({
facet_availability: {
pseudocode: {
available: false,
reason:
"Kuna could not decompile this function: Flow exceeded maximum instructions",
},
},
});
const imported = await run(client, "procedure_pseudo_code", {
procedure: "puts",
});
@@ -671,6 +683,20 @@ it("keeps answering pseudocode one function at a time when the export fails", as
"REA does not expose per-function records from a failed project export.",
]),
);
const fresh = await run(client, "procedure_pseudo_code", {
procedure: "main",
analysis: "independent_fresh",
});
expect(fresh.rawResult).toMatchObject({
source: { kind: "single-function" },
project_export: {
state: "failed",
reason: expect.stringContaining("not in recognized object file format"),
},
});
expect(fresh.limitations).toContainEqual(
expect.stringContaining("fresh independent single-function"),
);
expect(pseudocode.limitations.join(" ")).not.toContain("still building");
expect(pseudocode.limitations.join(" ")).toContain(
"larger project-export spans remain in the local project export",
@@ -286,6 +286,226 @@ it("keeps exact endpoints, deduplicates sites and counts excluded forwarding ali
});
});
it("preserves pointer and walk coverage separately and rejects inconsistent pointer identity", () => {
const pointer = {
kind: "pointer",
from_address: "@16384",
from_address_hex: "0x4000",
from_name: "slot",
to_address: "@8192",
to_address_hex: "0x2000",
to_name: "callee",
width: 8,
source: "chained-fixups",
};
const source = {
...xrefs(),
coverage: {
decoded: true,
section: "__TEXT,__text",
code_section: true,
executable: true,
unwalked: [
{
name: "middle",
address: "@20480",
address_hex: "0x5000",
section: "__TEXT,__const",
},
],
notes: ["middle was not walked"],
},
data_refs_coverage: "chained-fixups",
data_refs_count: 1,
data_refs: [pointer],
};
expect(
parseKunaIncomingReferences(wire(source), input, "0x2000"),
).toMatchObject({
ok: true,
value: {
addresses: ["0x1004", "0x4000"],
pointerCoverage: "chained-fixups",
pointers: [
{
from_address: "0x4000",
to_address: "0x2000",
source: "chained-fixups",
},
],
walkCoverage: { unwalked: [{ address: "0x5000" }] },
},
});
pointer.to_address_hex = "0x2001";
expect(
parseKunaIncomingReferences(wire(source), input, "0x2000"),
).toMatchObject({
ok: false,
error: { code: "xref_endpoints" },
});
});
it("counts pointer rows excluded because they target a forwarding alias", () => {
const pointer = (from: number, to: number) => ({
kind: "pointer",
from_address: `@${from}`,
from_address_hex: `0x${from.toString(16)}`,
from_name: null,
to_address: `@${to}`,
to_address_hex: `0x${to.toString(16)}`,
to_name: null,
width: 8,
source: "dynamic-relocations",
});
const source = {
...xrefs(),
data_refs_coverage: "dynamic-relocations",
data_refs_count: 2,
data_refs: [pointer(16384, 8192), pointer(16392, 12288)],
};
expect(
parseKunaIncomingReferences(wire(source), input, "0x2000"),
).toMatchObject({
ok: true,
value: {
addresses: ["0x1004", "0x4000"],
rows: 5,
excludedAliasRows: 2,
targetAliases: [{ name: "slot", address: "0x3000" }],
pointers: [
{ from_address: "0x4000", to_address: "0x2000" },
{ from_address: "0x4008", to_address: "0x3000" },
],
},
});
});
it("preserves full-width pointer and unwalked addresses without numeric rounding", () => {
const source = {
binary: input,
direction: "to",
count: 0,
target: {
name: null,
address: "@18446744073709551615",
address_hex: "0xffffffffffffffff",
aliases: [],
},
xrefs: [],
coverage: {
decoded: false,
section: null,
code_section: null,
executable: null,
unwalked: [
{
name: "candidate",
address: "@9007199254740993",
address_hex: "0x20000000000001",
section: ".text",
},
],
notes: [],
},
data_refs_coverage: "dynamic-relocations",
data_refs_count: 1,
data_refs: [
{
kind: "pointer",
from_address: "@9007199254740995",
from_address_hex: "0x20000000000003",
from_name: null,
to_address: "@18446744073709551615",
to_address_hex: "0xffffffffffffffff",
to_name: null,
width: 8,
source: "dynamic-relocations",
},
],
};
expect(
parseKunaIncomingReferences(wire(source), input, "0xffffffffffffffff"),
).toMatchObject({
ok: true,
value: {
addresses: ["0x20000000000003"],
rows: 1,
excludedAliasRows: 0,
pointers: [
{ from_address: "0x20000000000003", to_address: "0xffffffffffffffff" },
],
walkCoverage: { unwalked: [{ address: "0x20000000000001" }] },
},
});
});
it("distinguishes unreported pointer coverage from an explicitly empty pointer list", () => {
const source = xrefs();
expect(
parseKunaIncomingReferences(wire(source), input, "0x2000"),
).toMatchObject({
ok: true,
value: { pointerCoverage: null, pointers: null, walkCoverage: null },
});
expect(
parseKunaIncomingReferences(
wire({
...source,
data_refs_coverage: "none",
data_refs_count: 0,
data_refs: [],
}),
input,
"0x2000",
),
).toMatchObject({
ok: true,
value: { pointerCoverage: "none", pointers: [], walkCoverage: null },
});
});
it("rejects partial or inconsistent pointer inventories rather than reporting them empty", () => {
const source = xrefs();
for (const fields of [
{ data_refs: [] },
{ data_refs_count: 0 },
{ data_refs_count: 1, data_refs: [] },
]) {
expect(
parseKunaIncomingReferences(
wire({ ...source, ...fields }),
input,
"0x2000",
),
).toEqual({
ok: false,
error: { code: "xref_count" },
});
}
});
it("preserves unknown coverage for an address outside known sections", () => {
const source = {
...xrefs(),
coverage: {
decoded: false,
section: null,
code_section: null,
executable: null,
unwalked: [],
notes: [],
},
};
expect(
parseKunaIncomingReferences(wire(source), input, "0x2000"),
).toMatchObject({
ok: true,
value: {
walkCoverage: { section: null, code_section: null, executable: null },
},
});
});
it.each([
[
"count",
@@ -1,10 +1,7 @@
import { expect, it as test, vi } from "vitest";
import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js";
import {
kunaTimeoutRecovery,
type KunaProcessLauncher,
} from "../../../../src/kuna/KunaCommand.js";
import type { KunaProcessLauncher } from "../../../../src/kuna/KunaCommand.js";
import { KunaProjectCache } from "../../../../src/kuna/KunaProjectCache.js";
import { spawnOwnedProviderProcess } from "../../../../src/process/ProviderProcess.js";
@@ -373,20 +370,11 @@ it("names the setting that owns a single-function deadline", async () => {
message:
"kuna decompile did not finish within REA_KUNA_TIMEOUT_MS (300 ms).",
remediation: {
action: expect.stringMatching(
/^Raise REA_KUNA_TIMEOUT_MS .* restart REA.*project export/u,
),
action:
"Raise REA_KUNA_TIMEOUT_MS in the environment that starts REA, then restart REA; it reads Kuna settings when it starts.",
},
details: { timeout_ms: 300 },
});
// A finished invocation cannot read a later export from this session.
const recovery = kunaTimeoutRecovery("REA_KUNA_TIMEOUT_MS", "decompile");
expect(
projectAnalysisError(result.error, "invocation").remediation.action,
).toBe(recovery.remediationAction);
expect(projectAnalysisError(result.error).remediation.action).toBe(
recovery.continuingCallerGuidance?.remediationAction,
);
} finally {
expect(await client.close()).toEqual({ ok: true, value: null });
}
@@ -1,6 +1,5 @@
import { spawnSync } from "node:child_process";
import { createHash } from "node:crypto";
import { copyFile, mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { pathToFileURL } from "node:url";
@@ -11,164 +10,6 @@ import { fakeKunaCalls, writeFakeKuna } from "./fakeKuna.js";
const it = test.skipIf(process.platform === "win32");
const createCallerMatrix = async (root: string) => {
const targets = join(root, "targets");
const binaries: string[] = [];
for (const [flavor, name, multiplier] of [
["left", "audit_left", 3],
["right", "audit_right", 7],
["unmatched", "audit_other", 11],
] as const) {
const directory = join(targets, flavor);
await mkdir(directory, { recursive: true });
const source = join(root, `${flavor}.c`);
const binary = join(directory, "program");
await writeFile(
source,
`__attribute__((noinline,used)) int ${name}(int x) { return x * ${multiplier} + 1; }\nint main(void) { return ${name}(2); }\n`,
);
const compiled = spawnSync(
process.env.REA_CC ?? "cc",
["-O1", "-o", binary, source],
{ encoding: "utf8", timeout: 30_000 },
);
expect(compiled.error).toBeUndefined();
expect(compiled.status, compiled.stderr).toBe(0);
const stripped = `${binary}.stripped`;
await copyFile(binary, stripped);
const strip = spawnSync(process.env.REA_STRIP ?? "strip", [stripped], {
encoding: "utf8",
timeout: 30_000,
});
expect(strip.error).toBeUndefined();
expect(strip.status, strip.stderr).toBe(0);
binaries.push(binary, stripped);
}
return { targets, binaries };
};
const expectedCallerSelection = (targets: string) => ({
directory: targets,
functions_argument: "audit_left,audit_right",
requested_functions: ["audit_left", "audit_right"],
symbol_scan_complete: true,
examined: [
{
binary: "left/program",
status: "selected",
matched_functions: ["audit_left"],
},
{
binary: "right/program",
status: "selected",
matched_functions: ["audit_right"],
},
{
binary: "unmatched/program",
status: "no_matching_text_symbols",
matched_functions: [],
},
],
});
it("accepts requested functions distributed across caller targets", async () => {
const root = await createTestTempDirectory(
"rea-kuna-verifier-matrix-boundary-",
);
const { targets } = await createCallerMatrix(root);
const kuna = await writeFakeKuna(root);
const result = spawnSync(
process.execPath,
[
"scripts/verify-real-kuna.mjs",
"--targets",
targets,
"--functions",
"audit_left,audit_right",
],
{
encoding: "utf8",
timeout: 10_000,
env: {
PATH: process.env.PATH,
HOME: root,
REA_KUNA_COMMAND: kuna,
REA_CC: join(root, "compiler-must-not-run"),
},
},
);
expect(result.error).toBeUndefined();
expect(result.status).toBe(1);
expect(result.stderr).toContain("compiler-must-not-run");
expect(JSON.parse(result.stdout)).toMatchObject({
status: "failed",
cleanup: "complete",
caller_target_selection: expectedCallerSelection(targets),
});
});
test.skipIf(
process.platform === "win32" || process.env.REA_KUNA_COMMAND === undefined,
)(
"reports the complete selection of a passing real-provider matrix",
async () => {
const root = await createTestTempDirectory("rea-kuna-verifier-matrix-");
const { targets, binaries } = await createCallerMatrix(root);
const hashes = async () =>
Promise.all(
binaries.map(async (path) =>
createHash("sha256")
.update(await readFile(path))
.digest("hex"),
),
);
const before = await hashes();
const reportPath = join(root, "report.json");
const result = spawnSync(
process.execPath,
[
"scripts/verify-real-kuna.mjs",
"--targets",
targets,
"--functions",
"audit_left,audit_right",
"--report",
reportPath,
],
{
encoding: "utf8",
timeout: 100_000,
maxBuffer: 10 * 1024 * 1024,
env: process.env,
},
);
expect(result.error).toBeUndefined();
expect(result.status, result.stderr).toBe(0);
const report = JSON.parse(await readFile(reportPath, "utf8"));
expect(JSON.parse(result.stdout)).toEqual(report);
expect(report).toMatchObject({
status: "passed",
cleanup: "complete",
verifier_run: {
process_lineage: { status: "verified", descendants: [] },
},
caller_target_selection: expectedCallerSelection(targets),
});
expect(
report.results.map(({ binary }: { binary: string }) => binary),
).toEqual([
"fixture/rea-kuna-fixture",
"fixture/rea-kuna-fixture.stripped",
"left/program",
"left/program.stripped",
"right/program",
"right/program.stripped",
]);
expect(await hashes()).toEqual(before);
},
120_000,
);
it("accepts empty caller inventories but requires source-owned fixture literals", async () => {
const root = await createTestTempDirectory("rea-kuna-verifier-empty-");
const result = spawnSync(
@@ -207,39 +48,16 @@ await assert.rejects(() => verifyKunaStrings({ ...options, expectedStrings: ["re
});
it.each([
{
populated: false,
names: "main",
reason: "No caller-built targets",
examined: [],
},
{
populated: true,
names: "typo",
reason: "No caller-built targets",
examined: [
{
binary: "linux/program",
status: "no_matching_text_symbols",
matched_functions: [],
},
],
},
{ populated: false, names: "main", reason: "No caller-built targets" },
{ populated: true, names: "typo", reason: "No caller-built targets" },
{
populated: true,
names: "main,typo",
reason: "Requested functions were not found",
examined: [
{
binary: "linux/program",
status: "selected",
matched_functions: ["main"],
},
],
},
])(
"rejects an unverified caller matrix: $names, populated=$populated",
async ({ populated, names, reason, examined }) => {
async ({ populated, names, reason }) => {
const root = await createTestTempDirectory("rea-kuna-verifier-selection-");
const kuna = await writeFakeKuna(root);
const targets = join(root, "targets");
@@ -286,91 +104,12 @@ it.each([
expect(result.status).toBe(1);
expect(result.stderr).toContain(reason);
expect(result.stderr).not.toContain("compiler-must-not-run");
const report = JSON.parse(result.stdout);
expect(report.caller_target_selection).toEqual({
directory: targets,
functions_argument: names,
requested_functions: names.split(","),
symbol_scan_complete: true,
examined,
});
expect(report.cleanup).toBe("complete");
expect((await fakeKunaCalls(kuna)).map(({ argv }) => argv)).toEqual([
["--version"],
]);
},
);
it("retains examined caller targets when a later symbol inspection fails", async () => {
const root = await createTestTempDirectory("rea-kuna-verifier-inspection-");
const kuna = await writeFakeKuna(root);
const targets = join(root, "targets");
await mkdir(join(targets, "first"), { recursive: true });
await mkdir(join(targets, "second"), { recursive: true });
await writeFile(join(targets, "first", "program"), "not analyzed");
await writeFile(join(targets, "second", "broken"), "not analyzed");
const nm = join(root, "nm");
await writeFile(
nm,
`#!${process.execPath}\nif (process.argv.at(-1).endsWith("broken")) { process.stderr.write("inspection failed"); process.exit(9); }\nprocess.stdout.write("00001000 T main\\n");\n`,
{ mode: 0o755 },
);
const reportPath = join(root, "report.json");
const result = spawnSync(
process.execPath,
[
"scripts/verify-real-kuna.mjs",
"--targets",
targets,
"--functions",
"main",
"--report",
reportPath,
],
{
encoding: "utf8",
timeout: 10_000,
env: {
PATH: process.env.PATH,
HOME: root,
REA_KUNA_COMMAND: kuna,
REA_NM: nm,
REA_CC: join(root, "compiler-must-not-run"),
},
},
);
expect(result.error).toBeUndefined();
expect(result.status).toBe(1);
const report = JSON.parse(await readFile(reportPath, "utf8"));
expect(JSON.parse(result.stdout)).toEqual(report);
expect(report).toMatchObject({
status: "failed",
cleanup: "complete",
caller_target_selection: {
directory: targets,
functions_argument: "main",
requested_functions: ["main"],
symbol_scan_complete: false,
examined: [
{
binary: "first/program",
status: "selected",
matched_functions: ["main"],
},
{
binary: "second/broken",
status: "inspection_failed",
error: { message: expect.stringContaining("inspection failed") },
},
],
},
});
expect(result.stderr).not.toContain("compiler-must-not-run");
expect((await fakeKunaCalls(kuna)).map(({ argv }) => argv)).toEqual([
["--version"],
]);
});
it.each([
{
rejectClose: false,
@@ -55,6 +55,26 @@ const observation = (
};
describe("LLDB retained observations", () => {
it("preserves partial source coordinates while accepting historical omission", () => {
const attributed = {
...event,
source_location: {
file_path: "/missing/fixture.pas",
line: 23,
column: null,
},
};
const journal =
[event, attributed]
.map((entry) => JSON.stringify({ kind: "event", event: entry }))
.join("\n") + "\n";
const partial = observation("cancelled", journal);
expect(partial.events).toEqual([event, attributed]);
expect(partial.limitations).toContain(
"Source paths and lines are LLDB line-table attribution; REA did not read or verify the referenced source bytes.",
);
});
it.each([
["cancellation", "cancelled"],
["bridge failure", "tracer-failure"],
@@ -35,6 +35,11 @@ const LOCATION = {
module: "Fixture",
module_path: "/tmp/Fixture",
symbol: "-[Greeter greet:times:]",
source_location: {
file_path: "/not-acquired/é space/fixture.pas",
line: 23,
column: null,
},
};
const run = (
@@ -84,7 +89,7 @@ const run = (
{ name: "x0", value: "0x6000" },
{ name: "x1", value: "0x7000" },
],
backtrace: [],
backtrace: [{ ...LOCATION, source_location: null }],
},
],
other_stops: [],
@@ -207,7 +212,12 @@ describe("observe_native_calls projection: projects completed observations", ()
expect(result.events[0]).toMatchObject({
receiver_class: "Greeter",
selector: "greet:times:",
source_location: LOCATION.source_location,
backtrace: [{ ...LOCATION, source_location: null }],
});
expect(result.breakpoints[0]?.locations[0]?.source_location).toEqual(
LOCATION.source_location,
);
expect(result.coverage).toEqual({
status: "complete",
event_limit_reached: false,
@@ -581,3 +591,24 @@ it("retains completed call facts when cancellation arrives with the tracer resul
reason: "cancelled",
});
});
it("grows repeated source-location JSON by the location object, not by a token estimate", () => {
const path = "/not-acquired/é space/fixture.pas";
const located = {
...LOCATION,
source_location: { file_path: path, line: 23, column: null },
};
const absent = { ...located, source_location: null };
const copies = 3;
const withPath = Buffer.byteLength(
JSON.stringify(Array.from({ length: copies }, () => located)),
);
const withoutPath = Buffer.byteLength(
JSON.stringify(Array.from({ length: copies }, () => absent)),
);
const perCopy =
Buffer.byteLength(JSON.stringify(located.source_location)) -
Buffer.byteLength("null");
expect(withPath - withoutPath).toBe(copies * perCopy);
expect(perCopy).toBeGreaterThan(Buffer.byteLength(path));
});
@@ -51,8 +51,11 @@ import { observed } from "../fixtures/analysisExecution.js";
import {
createBinarySessionTargets,
createTestBinarySession,
createTestProviderRouter,
} from "../fixtures/binarySession.js";
import { createTestTempDirectory } from "../fixtures/temporaryDirectory.js";
import { BinarySession } from "../../src/application/binary/BinarySession.js";
import { InvestigationRecords } from "../../src/application/investigation/InvestigationRecords.js";
import { AnalysisCleanupLease } from "../../src/artifacts/AnalysisCleanupLease.js";
const cleanupError = new ProviderCleanupError(
@@ -278,6 +281,61 @@ it.each([false, true])(
},
);
it("preserves the caller path on an authenticated record refused by the ledger", async () => {
const [path] = await createBinarySessionTargets();
const callerPath = `${dirname(path)}/./${basename(path)}`;
const records = new InvestigationRecords({ maximumRepresentationBytes: 0 });
const session = new BinarySession(
createTestProviderRouter(
() => ({
execute: async (operation) =>
observed(operation === "health" ? null : { byte: 42 }),
close: async () => ok(null),
}),
{
resolveAnalysisProfile: async () =>
ok({
profile: createAnalysisProfile(
{
id: "fixture",
name: "Fixture analysis provider",
version: "1",
},
{ fixture: true },
),
}),
},
),
records,
);
const createSession = () => session;
const result = analysisCliErrorEnvelopeSchema.parse(
await runDirectAnalysis(
{
readConfiguration: () => parseConfig({}),
createBinarySession: createSession,
createArtifactBinarySession: createSession,
createNativeDispatchBinarySession: createSession,
createNativeBinarySession: createSession,
createManagedBinarySession: createSession,
},
callerPath,
"read_bytes",
{ address: "0x100", document: "fixture" },
).then(projectCliAnalysisResult),
);
expect(result.code).toBe("resource_constraint");
const partial = jsonObjectSchema.parse(
jsonObjectSchema.parse(result.details).partial_observation,
);
expect(partial.kind).toBe("unretained-evidence");
const evidence = parseEvidence(partial.evidence);
expect(evidence.caller_input_path).toBe(callerPath);
expect(evidence.subject?.local_path).toBe(path);
expect(evidence.normalized_result).toEqual({ byte: 42 });
expect(records.statistics().evidenceRecords).toBe(0);
});
describe("one-shot finalization cancellation", () => {
it.each(["read_bytes", "analyze_function"] as const)(
"retains completed %s Evidence when cancellation arrives during final close",
+16
View File
@@ -0,0 +1,16 @@
program ReaPascalCalls;
{$mode objfpc}
{$inline off}
procedure ReaPascalCall(Value: LongInt); cdecl; public name 'rea_pascal_call';
begin
WriteLn('pascal ', Value);
end;
var
Index: LongInt;
begin
for Index := 0 to 2 do { REA_VERIFY_CALLER_LOOP }
ReaPascalCall(Index); { REA_VERIFY_CALLER_CALL }
end.
Loaded 100 of 101 files, more files were not shown because too many files have changed in this diff. Show more