Compare commits

...
Author SHA1 Message Date
morluto a2b9a95d47 docs(mcp): qualify evidence view guidance 2026-11-01 23:46:05 +08:00
ShuaixinHuang aa48d494e9 fix(pe): enforce certificate name string alphabets (#8494)
* fix(pe): enforce certificate name string alphabets

* test(mcp): assert structured absolute-path diagnostics

* test(mcp): assert typed selector and stdio diagnostics

* fix(pe): bound IA5 name scans before validation
2026-11-01 21:24:01 +08:00
ShuaixinHuang 32c253ba31 fix(electron): preserve application argv during active capture (#8492) 2026-11-01 21:23:46 +08:00
morluto bafe3784cd fix(native): report dispatch target applicability (#8400)
* fix(native): report dispatch target applicability

* fix(native): clarify unavailable mode recovery

* test(kuna): avoid overlapping verifier imports

* docs(native): explain mode-specific discovery

* fix(native): preserve unbound provider recovery
2026-11-01 21:17:55 +08:00
morluto d684125eed docs(providers): verify prerequisite recovery and sibling failures 2026-11-01 21:14:27 +08:00
morluto 53363166f0 fix(evidence)!: validate native view parent predicates (#8451)
Reject native view sources whose predicate disagrees with the selected operation. Use the original predicate for existing Evidence; do not alter a record without recomputing its identity.
2026-11-01 20:27:34 +08:00
morluto 3842809c37 docs(setup): clarify Hopper failures and installer verification 2026-11-01 20:27:15 +08:00
gensokyo-dev e5e0dc5542 fix(ci): include hidden VitePress configuration in CodeQL (#8486) 2026-11-01 19:05:37 +08:00
2SinceJuly 1aa6beb5dc fix(decompile): name the differing resume identity (#8465)
Resume refused every identity mismatch with one generic message and
generic input-schema recovery. Report the differing target, profile,
layout, or provider facts with the journal and current values so callers
can choose the matching provider/profile or a new output directory.

Fixes #8349
2026-11-01 18:42:27 +08:00
gensokyo-dev 71746351d3 fix(ci): preserve the package verifier HTTP boundary (#8483) 2026-11-01 18:36:21 +08:00
morluto 293db94dd0 docs: prioritize product contracts over skill guidance 2026-11-01 18:29:34 +08:00
morluto 60c0cf977a docs: clarify recovery for valid lookup failures 2026-11-01 17:53:37 +08:00
morluto 7c755f2147 fix: retain address context cleanup failures (#8417) 2026-11-01 17:24:01 +08:00
morluto 1c1c8f6286 fix(snapshot): report binding mismatches (#8432)
* fix(snapshot): report binding mismatches

* test(snapshot): assert structured mismatch recovery

* docs(snapshot): explain profile comparison recovery

* fix(snapshot): include profile in recovery advice

* docs(snapshot): explain fresh-session profile discovery

* refactor(snapshot): derive admission from binding comparison
2026-11-01 17:23:11 +08:00
gensokyo-dev a6caa79bf4 ci(codeql): separate production scans from test fixtures (#8450) 2026-11-01 15:24:29 +08:00
gensokyo-dev e18594b247 refactor(composition): unify capability bindings and cleanup ownership (#8441)
Bind typed service creation, observed availability and retryable cleanup in one composition owner. Reuse selected CLI/MCP factories and canonical contract groups without copying provider operation inventories.

Closes #8429. Refs #740.

Validated paired real application workflows, actual local Codex CLI Terra/high use, complete Evidence and Unknown checkpoint parity, compiler-backed import boundaries, SDK schemas and final real mixed routing. Automated review findings were fixed at their producing owners. Existing exact-base CodeQL alerts are documented on the PR under the owner-authorized unrelated-check exception; no global alerts or security rules were changed.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
2026-11-01 13:30:21 +08:00
morluto 6e9ee97a69 fix!: preserve feature traces when owner lookup is unavailable (#8368)
* fix!: preserve feature traces when owner lookup is unavailable

* test: strengthen feature trace cancellation assertions

* fix: preserve feature trace provider readiness errors

* test: verify all feature trace reference sources

* test: verify IDA guidance at its owning boundary

* style: format feature trace regression
2026-11-01 05:38:40 +08:00
gensokyo-dev f72638f6b1 feat(investigation)!: retain records across engine cleanup (#8361)
Separate engine cleanup, investigation reset and terminal disposal in the shared application owner. Keep stable Evidence IDs and full Unknown revision history through engine close and target changes; serialize reset with all record writers and cleanup owners, optionally acknowledging a complete checkpoint before clearing. Bound retained representation without eviction.

Validation: 43 focused files / 533 distinct tests, source/type/lint/schema/documentation checks, nine genuine source cases across eight families, real Ghidra session identity and cleanup, and nine actual local Codex CLI terra/high runs with independent full-result and checkpoint audits. See docs/verification/investigation-lifecycle.md for measurements, partial attempts and verification limits.

Fixes #8347.

BREAKING CHANGE: close_binary retains investigation records. Use reset_investigation to clear them. Cross-process recovery requires an explicitly acknowledged Evidence bundle checkpoint.

Assisted-by: Suika (gensokyo-skills:suika-scatter-gather-planning)
2026-10-31 21:34:45 +00:00
Kaoru0822andClaude Opus 5.5 d54a4661da fix(ghidra): compare the DOS verifier's CLI window before its own rename (#8339)
The DOS verifier compared a fresh CLI session with an MCP session that
its address-boundary helper had renamed and restored, so the entry's
symbol source differed (imported vs user_defined). Compare the CLI with
the pre-edit MCP window, then assert the restoration changes only the
symbol provenance.

Fixes #8189

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-31 11:13:04 -07:00
Kaoru0822andClaude Opus 5.5 d2518a19f8 feat(native): label binary overview counts with their inventory scope (#8338)
* feat(native): label binary overview counts with their inventory scope

`binary_overview` described its counts as "exhaustive", and its result was
a pair of bare scalars. An agent could read them as a discovery of every
function and string in the file, or as comparable across providers. They
are exact totals of the rows each provider's inventories return.

The shared `projectBinaryOverview` now adds one provider-neutral
`inventory_coverage` record, so Ghidra's direct summary and the
listing fallback carry the same record:
`{ scope: "provider_inventory", discovery_coverage: "unknown" }`.
The record reuses the `discovery_coverage` vocabulary that
`list_procedures({detail:"summary"})` already reports, now owned by one
domain schema. The tool description states the scope and that counts are
not comparable across providers. Counts, segment omission semantics and
limitations are unchanged.

Fixes #8287

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(native): label overviews in the workflow so older snapshots stay valid

Analysis snapshots replay the projected inventory summary. A snapshot saved
before inventory_coverage existed would then fail the overview output schema.
The record describes what overview counts mean, whoever produced them, so
the binary_overview workflow now adds it on both its provider-summary and
listing paths. Provider values and projectBinaryOverview are unchanged, and
the composed overview's linked atomic record stays as the provider
reported it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(native): refresh composed overviews from before inventory_coverage

Bump the binary_overview workflow revision to 5, so snapshot and CLI replay
recompute a revision-4 composed overview from its atomic record. Replay
would otherwise return the cached overview without its count labels. The
revision-4 case joins the existing refresh scenario. The atomic Evidence
is not rewritten.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-31 11:12:52 -07:00
Kaoru0822andClaude Opus 5.5 a36c9dd9c8 fix(kuna): omit Mach-O data symbols from the procedure inventory (#8335)
* fix(kuna): omit Mach-O data symbols from the procedure inventory

Kuna's `functions --json` registers every Mach-O export as a function
entry, so globals in `__DATA,__data`, constants in `__TEXT,__const` and
`__mh_execute_header` were listed, counted and decompiled as procedures.

The Kuna session now classifies each inventory entry with the target's own
section table, read by the shared `inspect_macho` load-command reader from
the header bytes kept while its private copy is hashed. Entries in sections
that declare no instructions, or in the header and load commands, are data:
listings and searches omit them and report per-region counts, and
procedure operations refuse them by name or address while `address_name`
and `xrefs` still accept them. Instruction sections, symbol stubs and
symbol-pointer import slots are kept; addresses outside every section stay
with an explicit limitation, an unreadable section table keeps the whole
inventory unclassified, and ELF inventories are unchanged. Composed
workflows such as `decompile_all` and `binary_overview` follow the listing.

Fixes #8168

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(kuna): keep an entry's classification caveat on direct answers

Resolution now returns the selected entry's Mach-O placement and the
caveat it leaves open, built by the same constructor as the listings: the
one-entry unplaced wording, or the unreadable section table's reason.
Pseudocode, assembly, analyze_function (including the dossier's own
limitations), procedure_address, call queries by name and address_name
carry it; code and import-slot entries carry none.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-31 11:12:41 -07:00
Kaoru0822andClaude Opus 5.5 8a6ddec83a fix(mcp): name the close_binary retry for retained cleanup (#8329)
* fix(mcp): name the close_binary retry for retained cleanup

A close_binary that retired its target but could not confirm cleanup
returned cleanup_incomplete with retryable: false and only the generic
"Review the reported local resources" remediation, although the session
keeps every failed owner and the next close_binary on the same connection
retries them. Agents could abandon the connection that owns the only retry.

The MCP close tool now passes its retry operation to the session. The
session binds that typed AnalysisCleanupRetry to a failed retirement only
while it retains the cleanup for its next close; the close tool binds it to
process-capture failures its own scope retains and to compositions whose
session part was offered. Projection then reports
recovery.kind=retry_cleanup and appends a conditional instruction: keep the
connection, let the condition settle or resolve it, call close_binary
without snapshot_path and check the result. The hint disappears once that
cleanup resolves. retryable keeps describing the failed operation.

One-shot CLI closes, open_binary, analysis tools and failed snapshot saves
(which keep the target open) offer no retry. Cleanup composition copies the
owner's remediation without a retry offer, so the text appears once.

Fixes #8211

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test: derive close retry guidance from its owning formatters

The new close_binary retry assertions froze ProviderCleanupError's
explanatory sentence, so a harmless wording edit would break them. The MCP
test now compares delivered remediation with analysisErrorRemediationAction
applied to the delivered typed error plus cleanupRetryRemediation. The
domain tests use the same formatters, check that unbound and released errors
carry no retry text, and check that a bound composition adds the offer once.
Recovery fields, cleanup state and retry behavior keep separate assertions.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-31 11:12:30 -07:00
Kaoru0822andClaude Opus 5.5 c534e249d1 perf(process): reuse a verified Darwin ownership helper across processes (#8318)
* perf(process): reuse a verified Darwin ownership helper across processes

Every REA process on macOS compiled the Swift ownership helper into a fresh
temporary root and then paid macOS's first-launch assessment of that new
executable: 408 ms + 325 ms of a 1.4 s `rea inspect-macho /bin/ls`.

With the default xcrun compiler and module cache, the reader now keeps one
content-addressed copy under ~/Library/Caches/rea/process-helper/<key>/.
The key covers the helper sources, toolchain selection, architecture and
Darwin release. A copy is executed only after the cache root, entry and
files are verified as current-user owned, not group- or world-writable,
not links, and digest-matched to the entry manifest. Malformed entries
owned by the user are removed and rebuilt; linked or foreign entries are
left untouched and the per-process compile runs as before. Any cache
failure falls back to that compile, so the cache never fails an ownership
operation.

A miss compiles exactly as before, then publishes a copy through an atomic
rename, keeping the first concurrent publisher, and runs the shared copy.
That way its first-launch assessment is paid once. Per-run roots keep their
existing cleanup; published entries are never removed by close or exit.
An explicit REA_DARWIN_SWIFTC_PATH or REA_DARWIN_MODULE_CACHE_PATH keeps the
per-process compile and writes nothing to the cache. `rea uninstall
--purge-data` removes the namespace.

Fixes #2483

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(process): harden the shared Darwin helper cache

Address review findings on the shared ownership-helper cache:

- Verify every real parent of the cache root: each must belong to the user
  or root and not be writable by others (sticky root-owned directories
  excepted), so no other user can rename a verified entry. Execute through
  the real path.
- Open manifest and helper through the owned non-blocking regular-file
  reader, so a FIFO cannot block preparation, and retain handles whose close
  fails.
- Key entries on the swiftc and SDK that xcrun itself resolves, including
  their real paths, sizes and modification times. DEVELOPER_DIR, TOOLCHAINS,
  xcode-select and in-place updates therefore select new entries.
- Never remove or replace an invalid or published entry. Removal could race
  another process that has verified or is running that path, so such an
  entry falls back to the per-process compile.
- Give the cache a close() that retries retained staging removals and
  handle closes and reports ProviderCleanupError while any remain. The
  reader closes it with its per-run root.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(process): keep cache lookups owned and fall back from unlaunchable helpers

Address the second round of review on the shared ownership-helper cache:

- Pass the preparation signal to both xcrun toolchain lookups, and wait for
  every started lookup before reporting the first failure. Cancellation and
  close then stop a stalled lookup instead of waiting out its deadline.
  A failed or cancelled key computation is retried by the next lookup.
- When the verified shared copy cannot launch (EACCES, EPERM, ENOEXEC,
  ENOENT or an unrequested SIGKILL, for example from a noexec home volume),
  the reader stops using the cache. It reuses its own compiled helper or
  compiles one, and retries the read once.
- Anchor the cache's returned cleanup errors as cleanupRoot does, so every
  error returned while cleanup was pending resolves after a retry succeeds.
- Build the cache path with appendCallerPath, so a home with symlinks and
  `..` keeps the caller's meaning instead of being normalized lexically.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(process): keep concurrent fallbacks and dual cleanup failures

Each helper read now records before awaiting whether it launched the shared
copy, so a concurrent read's fallback cannot hide its own. When the per-run
root and the cache's retained resources both fail one close, the returned
error composes both. It stays incomplete until both owners recover.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(process): refuse shared helpers that an ACL lets others write

macOS ACLs can grant writes that the POSIX mode bits do not show. On
Darwin, lookup and publication list the real ancestors, the entry and its
files with `ls -lde`, and refuse sharing when any ACL entry allows a
write-class right. The default `group:everyone deny delete` entries on
macOS home folders are unaffected. The unlaunchable-helper test now removes
the execute bit from a verified copy, which refuses launch on both Linux
and macOS.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-31 11:12:02 -07:00
Kaoru0822andClaude Opus 5.5 41e7333cb5 feat(guidance): distinguish focused matches from broad parent sections (#8305)
* feat(guidance): distinguish focused matches from broad parent sections

Guidance search treated a parent section, including every descendant, as one
match. Terms scattered across unrelated subsections made a 49K-character root
the only result for "switch provider annotations", and every ancestor of a
focused hit was listed alongside it.

The section parser now records each section's own text (heading and body
before its first subsection) and its parent. Search keeps the same admission
rule but classifies each match by match_scope: "section" when every term is
in the section's own text, "subsections" when the terms are only spread
across subsections. Ancestors whose terms all sit inside one subsection are
no longer listed, section matches rank first, and subsections matches carry
own_text_terms plus bounded greedy pointers to the subsections holding the
remaining terms. Every match reports content_length so callers can judge a
read before making it. Read still returns the complete section.

Own text is tokenized once per request instead of each expanded subtree.

Fixes #8290

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(guidance): require the scattered-match checks to run

Count the subsections matches the real-bundle test inspects and fail when
none were checked, and document that a parent whose own text holds every
term stays listed beside its matching subsection.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-31 11:11:05 -07:00
371 changed files with 15213 additions and 2457 deletions

No files matched your search

+3 -1
View File
@@ -13,7 +13,9 @@ documentation, and live catalogs may describe different versions.
For provider-backed work, exercise the analyst's task through REA and compare
relevant native provider and independent artifact observations where available.
Check applicable partial results, recovery and session boundaries; report what
was not exercised. Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification)
was not exercised. For failure guidance, follow [design recovery around the
failure](../../../docs/tool-design.md#design-recovery-around-the-failure).
Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification)
to select evidence for the proposed contract instead of seeking provider API
parity.
@@ -514,4 +514,8 @@ as complete while required questions remain open.
Explain findings in plain language and tie them to returned evidence. When the
user asks to build something, use normal coding tools and separate observed
behavior from design choices. Close an opened native session with
`close_binary` when the investigation is complete.
`close_binary` when its engine is no longer needed. Evidence references and Unknown
history remain in memory on the same MCP connection. End the investigation with
`reset_investigation`; supply `save_path` to checkpoint the complete bundle before
clearing. Import the saved bundle to restore original references and Unknown
history in a new process. Retention is bounded and is not automatic persistence.
@@ -242,9 +242,13 @@ observation arrays and runtime reconciliation's `static_layers[].analysis` and
`runtime_observations[]` also accept exact references where advertised. Each
source can independently be inline or retained, preserving identity, coverage
and unknowns. CLI workflows advertising `--evidence-bundle` resolve these inputs
from the explicitly selected bundle. `close_binary` clears retained references;
export a bundle before closing, or supply portable inline Evidence in another
connection. A missing reference includes its ID and recovery guidance.
from the explicitly selected bundle. `close_binary` releases the engine while
retaining references and complete Unknown history on the same MCP connection.
`reset_investigation` explicitly clears them; optional `save_path` checkpoints the
complete bundle before clearing. Import that bundle in another connection to
restore its original references. Retention is bounded, in-memory and never silently
evicted; `binary_session.investigation` reports the budget. A missing reference
includes its ID and recovery guidance.
For exact file/native metadata or processing gaps, use
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
@@ -1,8 +1,11 @@
# Native, managed, and packaged artifacts
For release comparisons, export `list_procedures` Evidence from each target before
closing it. Close clears session records: import both bundles after collecting the
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
For release comparisons on one MCP connection, retain `list_procedures` Evidence
from each target and reuse its exact IDs after target switches or `close_binary`.
For a handoff or new process, export the complete bundle and import it there,
including its source closure. `reset_investigation` explicitly clears retained
records; optional `save_path` checkpoints them before clearing.
`match_native_functions` pairs unique exact names and, when supplied,
complete observed body bytes. Reuse optional same-context `list_names`,
`procedure_info`, `read_bytes` and existing `analyze_function` dossiers as
`left_observations`/`right_observations`; an MCP `analyze_function` summary
+21
View File
@@ -0,0 +1,21 @@
name: REA production code
# Match the existing default-setup scan's local-input threat model.
threat-models: [local]
# Retain root traversal, including future production roots. Hidden directories
# need their own root; use a directory so Python can share this configuration.
paths:
- .
- docs/.vitepress
# Keep the default query suite and all production roots, including bridge/,
# install.sh, runtime scripts, release controllers, and GitHub Actions.
# Exclude test inputs at extraction time so they cannot taint production callers.
paths-ignore:
- tests
- scripts/fixtures
- "**/*.test.*"
- "**/*.fixture.*"
- "**/test_*.py"
- "**/*_test.py"
+61
View File
@@ -0,0 +1,61 @@
name: CodeQL production
on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
- cron: "37 5 * * 1"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: codeql-production-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
analyze:
# A repository administrator must disable default setup before enabling
# this workflow; default setup rejects advanced CodeQL result uploads.
if: vars.REA_CODEQL_ADVANCED == 'true'
name: Analyze production (${{ matrix.language }})
runs-on: ubuntu-latest
timeout-minutes: 45
permissions:
contents: read
security-events: write
strategy:
fail-fast: false
max-parallel: 1
matrix:
language: [javascript-typescript, python, actions]
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
with:
persist-credentials: false
- name: Initialize production analysis
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with:
languages: ${{ matrix.language }}
build-mode: none
config-file: .github/codeql/production.yml
threads: 2
ram: 4096
- name: Analyze production code
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
with:
category: /language:${{ matrix.language }}
output: ${{ runner.temp }}/codeql-production-results
threads: 2
ram: 4096
- name: Retain complete analysis evidence
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: codeql-production-${{ matrix.language }}
path: ${{ runner.temp }}/codeql-production-results
if-no-files-found: error
retention-days: 7
+13
View File
@@ -28,6 +28,10 @@ Assess brittleness in touched code during implementation, debugging, review, and
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
After confirming a bug, search sibling adapters, consumers and similar workflows
for the same failure. Verify suspected matches, record working comparisons, and
group findings by root cause instead of filing duplicate reports.
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
@@ -75,6 +79,9 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
When changing installers or setup recovery, follow
[Checking installers and publisher downloads](docs/testing.md#checking-installers-and-publisher-downloads).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
@@ -83,6 +90,12 @@ Coordinate expensive checks across agents. If a test unexpectedly consumes heavy
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
contract, and actionable result or recovery at the point of use. Document the
result in product docs when explanation is needed. Add skill guidance only for
workflow judgment or context that the tool cannot reasonably convey; do not use
a skill as the primary repair for a discoverability or contract defect.
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
+3 -4
View File
@@ -16,10 +16,9 @@ import inventory. For current provider wiring, start with
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
the other `src/composition/` factories for tool services, including
[`gpu.ts`](src/composition/gpu.ts) and [`model.ts`](src/composition/model.ts).
Live Frida and Cutter adapters are constructed directly in
[`src/cli/fridaCommands.ts`](src/cli/fridaCommands.ts),
[`src/cli/cutterCommands.ts`](src/cli/cutterCommands.ts), and
[`src/server/createServer.ts`](src/server/createServer.ts).
Live Frida, Cutter, GDB and Rizin construction also uses shared composition
factories. For service ownership, availability and catalog declarations, follow
the [capability composition guide](docs/capability-composition.md).
Production factories construct fresh providers without starting engines or
acquiring targets; optional adapter failures must preserve successful peers. Run
`npm run verify:test-discovery` after adding or moving tests. Resolved import
+5 -5
View File
@@ -8,7 +8,7 @@ graph TD
end
subgraph "Production Composition"
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>static WASM/EVM descriptors: named contracts, factories, retryable lifetimes"]
COMPOSITION["src/composition/<br/>fresh typed provider and session factories<br/>one-shot binding and independent optional loading<br/>typed capability bindings: construction, availability and retryable lifetimes<br/>pure capability declarations feed provider catalogs"]
end
subgraph "Adapters"
@@ -23,6 +23,7 @@ graph TD
SESSION["SessionProviderRouter + BinarySession<br/>one immutable deep binding per target<br/>disjoint auxiliary routing"]
REGISTRY["AnalysisProviderRegistry<br/>sorted deep candidates<br/>deterministic selection, no fallback<br/>ProviderInspectionScope: prerequisite observation lifetime"]
RECORDS["src/application/investigation/<br/>composed InvestigationRecords + narrow ports<br/>EvidenceLedger and Unknown ownership"]
LIFETIMES["CapabilityScope<br/>host-owned service close capabilities<br/>released peers removed; failed cleanup retained for retry"]
SNAPSHOT["BinarySessionRecords + AnalysisSnapshotCache<br/>binary target/profile snapshots and session file ownership<br/>record coordination and post-commit notifications"]
JS_RECON["src/application/javascript/ + src/artifacts/javascript/<br/>JavaScript artifact reconstruction<br/>directory/ASAR reads<br/>AST-only bundle projection"]
end
@@ -99,6 +100,7 @@ graph TD
MAIN --> COMPOSITION
CLI --> COMPOSITION
SERVER --> COMPOSITION
COMPOSITION --> LIFETIMES
COMPOSITION --> SESSION
APP --> FIRMWARE
FIRMWARE --> PROCESS
@@ -149,8 +151,7 @@ graph TD
OBJDUMP --> DOMAIN
COMPOSITION --> RIZIN
RIZIN --> DOMAIN
CLI --> GDB
SERVER --> GDB
COMPOSITION --> GDB
GDB --> PROCESS
GDB --> DOMAIN
HOPPER --> HOPPER_BRIDGE
@@ -206,8 +207,7 @@ graph TD
NATIVE_READERS --> DOMAIN
COMPOSITION --> FORMAT_READERS
FORMAT_READERS --> DOMAIN
SERVER --> RUNTIME
CLI --> RUNTIME
COMPOSITION --> RUNTIME
RUNTIME --> DOMAIN
PROCESS --> FILESYSTEM
AUX --> FILESYSTEM
+107
View File
@@ -0,0 +1,107 @@
# Capability composition and ownership
REA keeps domain contracts and shared workflows separate from provider adapters
and CLI/MCP translation. A capability can have several provider identities; a
provider can contribute operations to several capabilities. Those associations
are source facts, while host availability is an observation.
## Source declarations and runtime bindings
`src/composition/capabilityDeclarations.ts` contains static provider identities,
named contract-group references and precise operation subsets. Complete groups
refer to `TOOL_CONTRACT_GROUPS` in the existing contract owner; family definitions
and provider catalogs read the same arrays, so a new group operation is included
without updating a copied provider list. Metadata imports only the group ID type
and pure metadata leaves; it must not load factories, schemas, engines or
generated catalogs. Catalog generation resolves groups and subsets against the
canonical contracts and rejects unknown groups/operations and conflicting
identities. New provider provenance must come from the implementation
that actually produces the Evidence, including format-specific capture adapters
and derived workflows.
`src/composition/capabilityServices.ts` binds typed service factories to those
capability keys. Each runtime entry owns construction, cleanup and any available
host/configuration facts or live probes. A service entry requires a close
capability; entries adapting managers retain their existing `closeAll` operation.
The host maps entries without erasing their individual service/port types. There
is no generic JSON operation dispatcher.
These two declarations have different loading needs. Static facts can be read
without loading a schema or provider graph. The full runtime composition is
materialized only when the MCP runtime needs it. One-shot CLI adapters import the
selected factory rather than constructing a complete capability host. CLI and MCP
reuse the same provider/service factories and application workflows, while
keeping their respective argument translation and lifetime.
## Resource lifetime
`CapabilityScope` registers a module's actual close capability when its instance
is composed. There is no independent server provider-close list to update.
Concurrent shutdown calls join the same cleanup attempt. A successful peer is
removed from pending cleanup; failed peers remain owned and are retried on the
next close. Synchronous throws and rejections without a reason remain failures,
and cannot prevent other peers from attempting cleanup. Shutdown seals further
ownership registration.
The host distinguishes these lifetimes:
- Host-owned service instances are released by that host.
- Caller-owned optional observations remain borrowed by SDK server instances;
their process-level owner closes them, including discarded discovery probes.
- CLI invocations use their existing finalizers to join cleanup and preserve the
original observation, cancellation and cleanup errors.
- Binary engines, bundle readers and investigation records retain their existing
owners. The `BinarySession` transition/drain barrier still coordinates target
changes, reset and disposal. A module refactor must not create independent
barriers that allow late work to repopulate reset records.
An application workflow may borrow a shared resource scope, such as byte entropy
using the host's artifact scope. Its wrapper does not acquire a second ownership
claim over the borrowed scope.
## Availability and registration
Module bindings provide host support, selected configuration and precisely typed
live availability probes through `CapabilityAvailabilityFacts`. Constructors do
not run these probes or launch engines. Session discovery invokes the existing
provider-owned probes only for advertised families; live failures remain fresh
observations. Caller policy, selected environments and injected providers retain
their existing precedence.
MCP registration remains in the MCP adapter. It consumes composed services,
record ports, admission and logging; it must not add fallback constructors that
can escape the host owner. Named contracts bind handlers and schemas. The complete
tool-family registrar map is checked by TypeScript, and catalog order never
selects a handler or schema.
## Extending a capability
1. Add or reuse the domain contract and application workflow, with explicit
Evidence provenance, coverage, unknowns and caller choices.
2. Put concrete construction in `src/composition/` and keep its default factory
lazy. Reference adapter-owned pure identity metadata from both the provider
and its static capability declaration.
3. Add the runtime binding if a host-owned service is needed. Bind cleanup and
availability/probes there, preserving borrowed versus owned resources.
4. Add the CLI/MCP argument adapters and their named registrations. A one-shot
command uses the shared selected factory and existing finalization helpers.
5. Run source-boundary, discovery, SDK-schema and relevant public workflow checks.
Exercise partial cleanup, retry, cancellation and caller-owned peers at the
applicable production seam; use actual applications/providers for routing and
complete Evidence verification. Recording ports do not prove real providers.
The constructor boundary guard covers the centralized Frida, Cutter, GDB and
Rizin constructors without rejecting legitimate type-only injected interfaces.
Other settled pure/provider boundaries continue checking type imports too. The
shared parser records module-edge erasure once. Whole-declaration type imports
are erased; inline type bindings retain their runtime module edge under
`verbatimModuleSyntax`. Static dynamic imports remain runtime edges too.
Tests belong with their semantic owner and must be
found exactly once; provider roots need the corresponding adapter discovery lane.
This is an internal composition contract. External plugin distribution, automatic
provider installation and a universal provider interface are separate concerns.
The [initial verification report](verification/capability-composition-2026-11-01.md)
records the executed public workflows, cleanup boundaries, resource measurements
and remaining provider qualification gaps for this composition increment.
+24
View File
@@ -615,6 +615,30 @@ cached result. `close_binary` accepts `snapshot_path` and optional
`overwrite: true` to save before releasing provider resources. A failed save
leaves the session open so the caller can resolve the output failure.
When a validated snapshot does not match the selected binding, the error's
`details.binding_mismatch` reports confirmed target identities, provider
identities, or profile digests that differ. `snapshot` is the saved value and
`selected` is the requested value. Multiple differences can be reported. A
`profile_comparison: "not_compared"` means the profile was not available at
that boundary; it does not establish a match. A profileless snapshot can report
`workflow_scope: "incompatible"` even when its target and profile match.
Choose a matching target, provider, and profile, or analyze without the snapshot
and save a new one for the intended binding.
For a profile mismatch, the saved snapshot contains
`binding.analysis_profile.parameters`. If `binary_session` reports an active
profile whose digest matches `binding_mismatch.profile_digest.selected`, its
`analysis_provider_binding.analysis_profile.parameters` can be compared with
those saved parameters. A rejected switch leaves the previous session active,
so its profile may be different from the attempted selection; check the digest
before using it for this comparison. A configuration difference does not by
itself establish why analysis results differ.
If no target is open and you intend to inspect the selected binding, open that
target without the snapshot, then check its profile digest in `binary_session`
before comparing parameters. This may start the selected provider; the saved
snapshot remains a separate file.
If a selected snapshot's file descriptor cannot close, `open_binary` reports
`cleanup_incomplete`. `binary_session` reports `cleanup_pending` even when no
target opened. Retry `close_binary` to release the retained descriptor before
+6 -2
View File
@@ -316,8 +316,12 @@ request to change it.
`runtime-configuration:` check. Re-run `rea setup` for that client when the
setting is on the registration allowlist, then reconnect so the new process
sees it.
- `invalid_request`: correct the reported fields from the current schema or
CLI help. Do not repeat the unchanged call.
- `invalid_request`: follow the specific recovery and reported issue. Inspect
the schema or CLI help when the input violates the contract or the response
asks for `inspect_input_contract`. A valid selector can instead require a
different choice, such as selecting a reported address; an inventory miss
does not establish that the target lacks the item. Do not repeat the
unchanged call when the response says it cannot succeed unchanged.
A restart is required to adopt a changed launch environment or registration.
The curated runtime settings above require no REA restart. It is not the first step for a transient
+8
View File
@@ -147,6 +147,14 @@ window-targeted renderer reload/crash actions, and synthetic `open-url` or
WebContents identities, process metrics, and IPC channel and value-shape metadata.
Payload values are not retained.
For the exact Playwright launch sequence, the active hook removes only REA's
bootstrap arguments from the application-visible `process.argv`. The vector keeps
the selected entry point and every caller argument, in order, including repeated
hook paths, `.asar` filenames, and debug-like options. The private launch handoff
is consumed before application code runs. If the observed sequence cannot be
matched exactly, argv stays unchanged and capture coverage reports `hook_conflict`;
REA does not guess an entry point or remove arguments by filename patterns.
`window_index` selects from Playwright's current Page list at the start of each action. REA resolves that Page to its corresponding BrowserWindow once; the action's `target` uses the BrowserWindow's capture-scoped `window_id`, matching the window inventory. Reordering either enumeration after selection does not retarget the action. Handle cleanup failures retain the completed action records and remain owned for provider cleanup retry.
Run the real fixture verifier with an operator-provided Electron runtime:
+180
View File
@@ -52,6 +52,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -115,6 +124,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -178,6 +196,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -241,6 +268,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -304,6 +340,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -367,6 +412,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -430,6 +484,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -493,6 +556,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -556,6 +628,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -619,6 +700,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -682,6 +772,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -819,6 +918,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -882,6 +990,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -945,6 +1062,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -1008,6 +1134,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -1071,6 +1206,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -1134,6 +1278,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -1197,6 +1350,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -1260,6 +1422,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
@@ -1323,6 +1494,15 @@
},
"required": ["kind", "operation", "timeout_ms"],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"kind": { "type": "string", "const": "retry_cleanup" },
"operation": { "type": "string", "minLength": 1 }
},
"required": ["kind", "operation"],
"additionalProperties": false
}
]
},
+33 -1
View File
@@ -675,6 +675,20 @@ an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra
and IDA have their own provider-specific host requirements; Windows Ghidra uses
the [experimental P0 boundary](windows-ghidra-p0.md).
Bazzite uses RPM packages, but REA does not install Hopper automatically on it.
Setup skips the Hopper offer because Bazzite is outside the supported host list,
regardless of whether Xvfb is installed. Bazzite documents
[containers for installing Linux packages](https://docs.bazzite.gg/Installing_and_Managing_Software/Containers/).
If using a container, check its distribution and architecture against the list
above. Install REA and Hopper in that container, give it access to the files you
want to analyze, and configure your agent to launch REA there. Verify installation
and analysis before relying on that workflow.
`rea doctor` checks health; it does not offer to install Hopper. Run `rea setup`
for installation. Setup cannot prompt when its output is redirected or during a
container image build. To approve Hopper installation without prompts, run
`rea setup --yes --install-hopper`.
If the Linux package download receives an HTTP failure, setup includes the
observed HTTP status in its `download_failed` diagnostic. The status does not
establish the underlying cause or guarantee that retrying will succeed.
@@ -728,6 +742,12 @@ Rotating signing subkeys remain bound to that primary identity.
The feed row itself is not PGP-signed; doctor labels its display version as
`reported_release_version`, separately from the authenticated launcher identity.
Even a correctly signed package can contain a launcher whose hash disagrees with
the feed. REA stops before installing that package. When reporting this error,
include the release version, package format and both hashes from the diagnostic.
Each format needs its own check: a matching Ubuntu DEB does not establish that
the Fedora RPM matches.
The approval plan includes the caller-selected cache location. Approved setup
retains the signed archive, public key and release metadata under
`${XDG_CACHE_HOME:-$HOME/.cache}/rea/hopper-releases/<launcher-sha256>`.
@@ -761,6 +781,16 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
`sudo`. `rea doctor --provider hopper --json` reports the selected
strategy and both host and effective mount facts.
You do not need to start a separate `Xvfb :99` server. REA starts and manages its
own display. After installation, run `rea doctor --provider hopper --json` and
check `hopper-demo-runtime` for display and Python problems. Setup also runs these
health checks: installing the package is only one step toward a working provider.
If an image build ends with `exit status 1`, read the setup output immediately
before that summary to find the failing step. On minimal Ubuntu images, run
`apt-get update` before setup if an earlier layer deleted the package indexes.
REA's package installation commands do not refresh those indexes.
Hopper also embeds Python; the Python executable and standard library alone
are insufficient on Ubuntu. Approved setup checks the system interpreter
at `/usr/bin/python3` and, when necessary, installs its matching
@@ -971,7 +1001,9 @@ including while a shutdown request is in flight. A helper exit alone is
insufficient, and an application that never authenticated has no verified
process identity. If no matching bridge becomes available, inspect and close
the reported document in Hopper before ending the owning REA connection.
Export the Evidence bundle before `close_binary` clears session records.
`close_binary` preserves Evidence and complete Unknown history in memory on the
same connection. Export the bundle before ending the REA process to resume later;
`reset_investigation(save_path)` checkpoints and explicitly clears those records.
`binary_session.cleanup_pending` remains true while provider cleanup is retained,
and provider health remains visible even with `open: false`; that flag alone
does not establish successful cleanup.
+14 -3
View File
@@ -38,7 +38,7 @@ Windows is not supported yet. Supported targets are ELF and thin Mach-O executab
| Tool | Source |
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `list_procedures`, `search_procedures`, `procedure_address` | Kuna's callable inventory (`kuna functions --json`), including import slots it does not decompile |
| `list_procedures`, `search_procedures`, `procedure_address` | Kuna's callable inventory (`kuna functions --json`), including import slots but not [Mach-O data](#mach-o-data-entries) |
| `address_name` | Function entries, then the data labels Kuna's export assigns |
| `procedure_pseudo_code`, `batch_decompile` | The project export, or a single-function run while the export builds |
| `procedure_assembly` | The project export's labeled listing |
@@ -62,7 +62,7 @@ The first text or assembly request in a session looks for this target's export i
- Assembly and data names wait for the export; progress notifications report the wait every ten seconds. A session does not rebuild a failed export: requests that need it fail with `retryable: false`, the export's failure, its captured Kuna output and what to change before reopening the target. A setting such as `REA_KUNA_PROJECT_TIMEOUT_MS` or `REA_KUNA_JOBS` is read when REA starts, so changing it also needs a restart. Pseudocode keeps using single-function runs.
- A single-function answer says so in its limitations. It lacks the export's whole-program prototypes, so a later request can return different text for the same function.
Pseudocode, assembly, `analyze_function` and data-name answers record where their text came from in `raw_result.source`: `single-function`, or `project-export` with the cache directory. `procedure_pseudo_code` and `procedure_assembly` answers from the export also name the file, byte span and the file's SHA-256. Combined `analyze_function` answers retain both facet records in `raw_result.source.files`, each with its file, digest and byte span. Both texts come from one export; losing that project retries the complete report. Listings and procedure lookups carry no `raw_result`. Agents can open or search the export's `.c` and `.asm` there directly when that is cheaper than calling a tool per function. Text read that way is target-derived, like any decompiler output, but it bypasses the untrusted-content marker that REA's tool results carry.
Pseudocode, assembly, `analyze_function` and data-name answers record where their text came from in `raw_result.source`: `single-function`, or `project-export` with the cache directory. `procedure_pseudo_code` and `procedure_assembly` answers from the export also name the file, byte span and the file's SHA-256. Combined `analyze_function` answers retain both facet records in `raw_result.source.files`, each with its file, digest and byte span. Both texts come from one export; losing that project retries the complete report. Listings and procedure lookups carry no `raw_result`; omitted [Mach-O data entries](#mach-o-data-entries) are reported in their limitations. Agents can open or search the export's `.c` and `.asm` there directly when that is cheaper than calling a tool per function. Text read that way is target-derived, like any decompiler output, but it bypasses the untrusted-content marker that REA's tool results carry.
The background export belongs to the session. `close_binary`, or the end of a CLI command, cancels it unless another session still waits for it. A one-shot CLI decompile therefore answers from a single-function run; any request that needs the export, such as `procedure_assembly` or `analyze_function`, keeps it.
@@ -72,6 +72,17 @@ Regex procedure and string search uses ECMAScript Unicode matching in a cancella
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` with `detail: "complete"` (the CLI default) returns the complete dossier inline, which can still be large; later retained views do not remove that first transfer. The MCP default retains the dossier and returns its summary, so page the facets you need through `inspect_analysis_view`, or prefer the individual text tools for a focused question.
## Mach-O data entries
Kuna's inventory also lists Mach-O exports that are data, such as globals in `__DATA,__data`, constants in `__TEXT,__const` and `__mh_execute_header`. REA classifies each entry with the target's own section table, the same load-command facts `inspect_macho` reports, read from the leading bytes kept while the session's private copy is hashed:
- A section with `S_ATTR_PURE_INSTRUCTIONS` or `S_ATTR_SOME_INSTRUCTIONS`, or of type `S_SYMBOL_STUBS`, holds code; the entry stays.
- A lazy or non-lazy symbol-pointer section (`__got`, `__la_symbol_ptr`) holds import slots; the entry stays.
- Any other section, or the header and load commands before the first section, holds data. Listings and searches omit the entry and say how many they omitted in each region. Pseudocode, assembly, `analyze_function` and `procedure_address` refuse it by name or address, and call queries refuse its name, naming its region; call queries by exact address stay validated by Kuna's graph. `address_name` and `xrefs` still accept it.
- An address outside every section and the header stays, with a limitation saying REA cannot classify it. If the section table cannot be read, every entry stays and listings say why. Answers about one such entry, including pseudocode, assembly, `analyze_function`, `procedure_address`, call queries by name and `address_name`, carry the same limitation; code and import slots carry none.
A section's attributes are the format's declaration, not proof of what runs there, and a kept entry is not proven to be a function. ELF inventories are kept as Kuna reports them. On a Mach-O target, the first listing or lookup establishes the private copy that Kuna commands also use, once per session, even when a warm export answers it.
## Strings
`list_strings` and `search_strings` share one lazy `kuna strings --json --encoding all --min-length 5 --termination any --no-xrefs` scan of the digest-checked private input. Skipping references avoids program lifting, a call graph and a C export. REA validates the input, scan policy, every row and the total before caching any facts. Encoding, character count, byte length and terminator facts must agree; supplementary UTF-8 characters count as Unicode scalars. Malformed output is an error, not an empty search. REA parses the JSON as Kuna writes it instead of buffering it, so the scan has no fixed size: Kuna writes about 290 bytes per row, 31 MB for a 20 MB Ghostty binary and 179 MB (611k rows) for an arm64 Firefox `XUL`. Rows are validated one at a time, and the first contradiction stops Kuna. Text REA does not keep is dropped unassembled; kept text and retained rows are admitted against the REA process's live heap headroom as they arrive; a refusal is a `memory` resource constraint with `retryable: false`, and closing other targets or starting REA with a larger Node.js heap makes room. String scans use the whole-program `REA_KUNA_PROJECT_TIMEOUT_MS` deadline; `REA_KUNA_TIMEOUT_MS` applies to single-function decompilation. The parsed inventory belongs to one selected target/executable profile and expires on session close; Kuna diagnostics beyond their 8 MiB buffer are retained as a failure until close, while other failures are not cached.
@@ -137,7 +148,7 @@ the manifest's file identity as well as the export files. These refusal and
migration cases are verified with controlled Kuna-format exports; they do not
establish behavior of an independently installed Kuna release.
Builds publish by one rename, so a reader sees a complete export or none. If later staging retirement fails, the typed cleanup error retains the completed export's key, directory and manifest in `published_export`. The session adopts that verified project; later queries reuse its immutable files while `project_export.cleanup_incomplete` reports the retained owner's live state. `close_binary` retries only the remaining cleanup, and a successful retry resolves prior cleanup errors. A failed export followed by failed staging cleanup retains both failures; it does not invent a publication receipt. A session keeps the export it uses from eviction only within its own REA process. If another REA process sharing the cache evicts it, the session rebuilds it; meanwhile pseudocode falls back to single-function runs. A damaged or edited export is removed and rebuilt. REA retains the file state verified with each digest, checks it before warm-cache reuse, and checks open descriptors before and after reading text. Assembly indexes are checked against the same state; a changed file makes the whole report retry rather than mixing facets from different exports. Staging moves to `trash/` for removal only after REA proves its Kuna process stopped. Staging of a process it cannot prove stopped is never swept; REA keeps it owned, retries the stop on `close_binary`, and reports it until the stop is proven. Failed file removal also stays bound to its export and is retried by `close_binary`; closing does not report success while that cleanup remains. `rea doctor --provider kuna` reports the cache directory without creating it.
Builds publish by one rename, so a reader sees a complete export or none. If later staging retirement fails, the typed cleanup error retains the completed export's key, directory and manifest in `published_export`. The session adopts that verified project; later queries reuse its immutable files while `project_export.cleanup_incomplete` reports the retained owner's live state. `close_binary` retries only the remaining cleanup, and a successful retry resolves prior cleanup errors. A failed export followed by failed staging cleanup retains both failures; it does not invent a publication receipt. A session keeps the export it uses from eviction only within its own REA process. If another REA process sharing the cache evicts it, the session rebuilds it; meanwhile pseudocode falls back to single-function runs. A damaged or edited export is removed and rebuilt. REA retains the file state verified with each digest, checks it before warm-cache reuse, and checks open descriptors before and after reading text. Assembly indexes are checked against the same state; a changed file makes the whole report retry rather than mixing facets from different exports. Staging moves to `trash/` for removal only after REA proves its Kuna process stopped. Staging of a process it cannot prove stopped is never swept; REA keeps it owned, retries the stop on `close_binary`, and reports it until the stop is proven; that MCP error names the same-connection retry as `recovery.kind=retry_cleanup` ([retained cleanup retry](mcp-contracts.md#retained-cleanup-retry)). Failed file removal also stays bound to its export and is retried by `close_binary`; closing does not report success while that cleanup remains. `rea doctor --provider kuna` reports the cache directory without creating it.
## Verify with a real Kuna
+58 -13
View File
@@ -173,6 +173,9 @@ recommendation to change it. When it names no setting, reduce supported work;
do not invent one. Raising only the MCP
client's deadline cannot extend this budget. These hints never include the
complete catalog or caller arguments.
`recovery.kind=retry_cleanup` appears only on a `close_binary`
`cleanup_incomplete` error whose retained resources the session will retry; see
[retained cleanup](#retained-cleanup-retry).
Failures omit `structuredContent`, since each advertised `outputSchema`
describes successful results. Successful replies retain their schema-validated
structured data and matching text projection. Oversized errors are retained
@@ -207,8 +210,13 @@ unresolved relationship reads. Cancellation remains a tool error.
`trace_feature` validates each reference's containing-procedure resolution using
the same contract as navigation context. Explicit absence remains a successful
negative observation; malformed provider replies return `unreadable_output`
for `resolve_containing_procedure` before successful-result projection.
negative observation. If the provider reports a non-retryable capability refusal
attributed to the owner lookup without incomplete cleanup, the reference remains visible
with `containing_procedure.status: "unavailable"` and a residual unknown. That
unknown does not imply references were truncated.
Malformed provider replies return `unreadable_output` for
`resolve_containing_procedure` before successful-result projection;
cancellation and incomplete cleanup remain errors.
REA prepares complete MCP results within the pinned stdio client's 10 MiB
receive-buffer budget, including both text and structured representations and
@@ -317,9 +325,10 @@ To discover existing records without their large payloads, call
`get_evidence_bundle` with `detail: "summary"`. Its `result.records` contain
Evidence IDs, artifact/provider identities, analysis-profile digests and native
procedure metadata. Optional `filters` match exact `evidence_id`, `operation`,
`target_sha256`, `analysis_profile_digest` or `procedure_address`. Use a discovered
ID with `inspect_analysis_view`; neither discovery nor retained selection starts
a provider. `retention: "complete-record"` describes stored Evidence, while
`target_sha256`, `analysis_profile_digest` or `procedure_address`. For a supported parent operation, use a discovered ID with
`inspect_analysis_view`; its tool description lists supported operations. For
other records, export the complete bundle to a caller-selected path. Neither
discovery nor retained selection starts a provider. `retention: "complete-record"` describes stored Evidence, while
`native_dossier.value_flow` independently reports truncation, unavailable or
unknown metadata. Missing omission counts remain null. Summary output carries
every matching record; actual transport admission still applies. Omitted
@@ -521,7 +530,11 @@ names, the native API record, and high-pcode facets without starting a provider.
and limit default to 0 and 64; pseudocode uses UTF-16 code units and never
splits surrogate pairs, while other pages count rows. Unavailable facts and
provider limitations remain explicit. Source is
an exact same-session retained reference or portable inline Evidence. Views are
an exact same-session retained reference, portable inline Evidence, or an
Evidence record or bundle file. Native function, inventory, and observation
views require the `rea.analysis` predicate; function-matching views require
`rea.native-function-matching`. A mismatched predicate returns an
`invalid_request` identifying `source.predicate_type`. Views are
a summary, a layout mitigations or linkage facet, one section/symbol/module, or
a stable page with a caller-selected positive `limit`. Module pages include
JavaScript assets, bundled modules, and source modules. Select an exact
@@ -675,13 +688,24 @@ advertises a file selector; a retained ID does not authorize a file read.
References belong to the current connection's Evidence ledger. Opening another
target or restarting the current target preserves retained records, including
when provider cleanup fails; records remain available for export and retry.
`close_binary` clears them, even when no binary is active. A fresh connection
has its own ledger. A missing reference reports its exact ID and
`close_binary` releases the engine and replay cache while preserving these records,
even when no binary is active or cleanup fails. `reset_investigation` explicitly
clears all records and Unknown revisions after draining earlier work; a healthy
active engine remains open. Its optional absolute `save_path` saves a complete
canonical bundle before clearing. Failed save, cancellation observed before clearing or incomplete cleanup
preserves records, and a published file is reported even if later cleanup fails.
A fresh connection has its own in-memory ledger; import the saved bundle to restore
original Evidence IDs and complete Unknown history. A missing reference reports its exact ID and
`details.reason: "missing"`; the server cannot infer
whether it was never recorded, cleared, or retained by another connection.
Supply complete inline Evidence, repeat its producer, or import an exported
Evidence bundle before referencing that imported record. Export a bundle before
closing if the investigation needs it later.
Evidence bundle before referencing that imported record. Export a bundle or use
`reset_investigation(save_path)` before ending the process if the investigation
needs it later. Retention has no automatic persistence or silent eviction.
`binary_session.investigation` reports record/revision/head counts, estimated
retained representation bytes and the fixed allowance derived from initial V8
heap headroom. A capacity refusal preserves the existing investigation; checkpoint
and reset it, or start with a larger Node heap. These byte estimates are not RSS.
CLI workflows that advertise `--evidence-bundle` can resolve the same reference
JSON from one explicitly selected local bundle, including an MCP export. They
@@ -742,6 +766,13 @@ finish without reading relationships, even when neither traversal mode is
available. Discovery does not block that direct invocation or replace its typed
partial/error outcomes.
`trace_native_ui_action` reports authored-UI and native-seed modes separately.
Apple dispatch metadata requires a Mach-O target; another provider may supply
the symbol-only alternative on other formats. If neither mode is available,
read each mode's missing operations and remediation: a remedy for an authored
app-bundle route does not by itself describe the native-seed route. Availability
does not guarantee that every selector or provider operation will succeed.
With complete detail, the `effects` and `annotations` arrays list the true
members of their fixed vocabularies. A known member omitted from either array is false. This applies
only to these closed flag sets; it does not make an omitted tool, provider, mode,
@@ -861,6 +892,20 @@ inspection fails or is cancelled before a provider client exists. If no client
remains, no provider run or health observation is available; follow the cleanup
error's reported resources.
### Retained cleanup retry
A `close_binary` that retires the target clears engine replay state while
retaining investigation Evidence and Unknown history, including when cleanup fails. When the session keeps every reported owner for the next
`close_binary` on the same connection, its `cleanup_incomplete` error carries
`recovery: {"kind": "retry_cleanup", "operation": "close_binary"}`.
Keep the connection open, let the reported condition settle or resolve it, call
`close_binary` without `snapshot_path`, and check the result. The retry does not
rerun the failed analysis and can report `cleanup_incomplete` again.
`retryable` still describes the failed operation and stays `false`. A failed
`snapshot_path` save returns before retirement, keeps the target open and has
no `retry_cleanup` hint. `open_binary`, analysis tools and one-shot CLI commands
do not report this hint either, even when their cleanup remains pending.
## Initial target at startup
MCP normally starts without a target; use `open_binary` to select one during the
@@ -1252,11 +1297,11 @@ If the client timed out or cancelled during startup, the pending query may have
been interrupted. Reissuing `open_binary` for the same active target can reuse
the current client, so it is not a fresh-start recovery. On the same connection:
1. Keep any useful inline Evidence, or export a bundle before closing if retained
records are needed. `binary_session` can show the selected binding and its
1. Keep useful inline Evidence or use the retained investigation references;
export a bundle before ending the process to resume later. `binary_session` can show the selected binding and its
recorded state; target selection alone does not prove engine readiness.
2. Call `close_binary` and check its result. It drains owned work, closes the
provider, and clears retained session records. A `cleanup_incomplete` result
provider and engine replay state while preserving investigation Evidence and Unknown history. A `cleanup_incomplete` result
must be addressed according to its reported owned resources before retrying.
3. Call `open_binary` with the same caller-selected path and
`provider_id: "ghidra"`, then retry the first query with an appropriate client
+14 -2
View File
@@ -24,13 +24,25 @@ CLI equivalent is `rea guidance --query "exact JavaScript module"`, followed
by `rea guidance --action read --locator LOCATOR`. Search is limited to the
packaged managed skill files, works without providers or a target, and returns
deterministically paged matches. Every non-generic query word must occur in the
section; generic-only queries return no matches. Results are documentation; locators
section or its subsections; generic-only queries return no matches. Results are documentation; locators
are bound to the installed bundle and report stale or unavailable guidance without
returning cached section content. A no-match result says nothing about analysis
capability availability.
Each match reports its `match_scope`. A `section` match has every term in its
own text, meaning its heading and body before its first subsection. A
`subsections` match has the terms only spread across subsections, with no
single subsection holding them all. Its `own_text_terms` lists the terms in its
own text, and `subsections` lists up to 8 subsection locators, chosen greedily,
whose own text holds the rest; `omitted_subsections` counts any more that were
needed. Section matches rank first. A parent is not listed when one subsection
holds every term, because that subsection is already a match, unless the
parent's own text also holds them all. `content_length`
is the size of the section that read returns, so callers can decide before
reading a broad parent.
Search returns at most 20 matches per page with 500-code-unit context excerpts.
Read retrieves the complete selected section. Admission permits 1 MiB per file,
Read retrieves the complete selected section, including its subsections. Admission permits 1 MiB per file,
4 MiB per bundle, 4,096 sections, and 25,165,824 expanded UTF-16 code units before
section text is materialized. Unreadable or oversized bundles return
`bundle_unavailable`, with no partial index or inferred absence. Cleanup failures
+27
View File
@@ -652,6 +652,33 @@ that directory: it outlives each run, so the first compilation into a new
directory pays a cold-cache build and later runs reuse it. An absent or empty
value keeps the shared default.
With the default `xcrun` compiler and module cache, REA also keeps one verified
copy of the compiled helper in `~/Library/Caches/rea/process-helper/`, so later
REA processes skip both the compile and macOS's first-launch assessment of a
new executable. Each entry is keyed by the helper source, the `swiftc` and SDK
that `xcrun --find swiftc` and `xcrun --show-sdk-path` resolve (their real
paths, sizes and modification times, so `DEVELOPER_DIR`, `TOOLCHAINS`,
`xcode-select` and in-place toolchain updates all select a new entry), the CPU
architecture and the Darwin release. Before running an entry, REA requires the
cache directory, the entry and its files to be owned by the current user and
not writable by group or others, every real parent directory to be owned by
the current user or root and not writable by others (sticky root-owned
directories excepted), no ACL on any of them to allow another principal a
write-class right (the default `deny delete` entries are fine), the helper and
manifest to be regular files rather than
links or FIFOs, and the helper's SHA-256 to match the manifest. Anything else is
never executed and never removed, because another REA process may still be
verifying or running that path; REA then compiles a per-process helper as
before. A published entry is never replaced. If the cache is unavailable, for
example under a read-only home directory, the ownership operation proceeds
with the per-process compile. These checks reject corruption, loosened
permissions and other users, but they do not defend against another process
running as the same user, which could already modify REA itself. Setting
`REA_DARWIN_SWIFTC_PATH` or `REA_DARWIN_MODULE_CACHE_PATH` keeps the per-process
compile and writes nothing to this cache. REA does not prune old entries; each
is about 240 KiB. Delete the directory while no REA process is running, or run
`rea uninstall --purge-data`, to remove them.
`xcrun` also maintains a separate tool-lookup cache. Its `--no-cache` option
refreshes entries and does not guarantee a write-free lookup. To avoid that
lookup while compiling REA's helper, set `REA_DARWIN_SWIFTC_PATH` to the absolute
+101
View File
@@ -365,6 +365,41 @@ Do not report the warm result as a cold-suite improvement, and do not enable
the cache in coverage or benchmark CI without first showing that its
instrumentation remains equivalent.
## CodeQL production scope
[The production configuration](../.github/codeql/production.yml) excludes
`tests/`, `scripts/fixtures/`, `*.test.*`, `*.fixture.*`, `test_*.py`, and `*_test.py` inputs
during extraction. It retains the default queries and the existing `local`
threat model. Production source, bridges, runtime scripts, installation and
release controllers, and Actions remain in scope for the configured languages.
Verification scripts remain included unless they match an explicit test path;
excluding the entire `scripts/` directory would hide shipped entrypoints.
The `paths` list retains repository-root traversal and explicitly includes
`docs/.vitepress/`, which JavaScript extraction otherwise skips as a hidden
directory. Keep directory entries here: the shared Python extractor does not
accept individual files as traversal roots. When changing this list, compare the
complete production file inventory as well as alert counts.
These exclusions do not change which behavioral tests CI executes. CodeQL's
JavaScript/TypeScript, Python, and Actions lanes do not add shell, Java, or Swift
analysis; their existing checks remain responsible for those languages.
[The production workflow](../.github/workflows/codeql.yml) is inactive until a
repository administrator disables CodeQL **default setup** and sets the Actions
repository variable `REA_CODEQL_ADVANCED` to `true`. Disable default setup first:
it rejects advanced CodeQL uploads. Then enable the variable and manually run
**CodeQL production** on `main`; verify all three analyses complete before
relying on the new scope. While the variable is unset, the existing default
setup continues scanning. Adding the configuration file alone does not change
default setup in this user-owned repository.
Each analysis uses two threads and a 4096 MiB CodeQL memory budget; language
jobs run one at a time. Complete SARIF reports are retained for seven days.
For a scope comparison, use the same source SHA, CodeQL bundle, default query
suite, and `local` threat model on both runs. Change only extraction paths, inspect
the extracted-file inventory and diagnostics, and retain both complete reports.
Alert counts do not establish a false-positive rate or a speedup. Report elapsed
time, CPU time, and process-family memory with their measurement limits.
## Choose verification for the change
`test:local` is fast source-level feedback. For the strongest practical
@@ -408,12 +443,57 @@ affected consumer.
- Where relevant, continue after a mixed success, failed request, cancellation
or close/reopen. Verify the next supported recovery and preservation of prior
results and resource ownership. Use the applicable lifecycle regressions below.
- For lookup failures, distinguish malformed input from a valid missing or
ambiguous selector. Check the returned candidate and input identity, follow
an actionable recovery where possible, and verify that an inventory miss is
not presented as artifact-wide absence. Test schema-repair advice separately.
- Record the provider version, host, fixture, actual entrypoint, controlled
faults and skipped or unavailable coverage. A simulated provider tests the
integration boundary; a scripted sequence tests that sequence, not an agent's
ability to discover it. Run the matching real-provider and agent lanes when
their respective claims require them.
For prerequisite recovery, include a case where only the wrong alternative is
configured: Unblob cannot enable Binwalk inspection, and Binwalk cannot enable
Unblob extraction. Check the final readiness or capability output retains the
producer's repair instructions, including which launch environment must change.
When feasible, follow the advice and verify that the same operation now works.
Report unexecuted recovery steps explicitly.
### Checking installers and publisher downloads
When changing setup or investigating an install failure, check the stages affected
by the change:
1. **Does REA support installation on this host?** Check the distribution and CPU
architecture, including an unsupported case. Recognizing RPM or DEB packages
does not mean REA can install on every related distribution. Check prompts in
a terminal and with redirected output; image builds cannot answer interactive
prompts. `doctor` checks health; `setup` installs.
2. **Does the downloaded package match the publisher's metadata?** Record the
release version, package format, URL and check date. Verify the signature and
compare the launcher hash inside that package with the feed and installed
launcher. Check each format you claim works: a passing DEB check says nothing
about the RPM. Old fixtures cannot catch a newly published package mismatch.
Test both an invalid signature and a valid signature with a mismatched launcher
hash; setup must reject either before running the package manager.
3. **Can download failures be diagnosed?** Use REA's request headers and timeout.
Check whether the setup result identifies the failed URL, HTTP status or
timeout, and next action; report missing details. Use controlled failures for
repeatable tests. A fast download on one network does not rule out a timeout
elsewhere, and curl with different headers may get a different response.
4. **Does the installed provider run?** Check package-manager prerequisites and
run doctor for the selected provider. For Linux Hopper, check Python and REA's
private Xvfb display separately from package installation. Where relevant,
test an unusable `/tmp/.X11-unix` and a blocked user/mount namespace fallback.
Starting an extra X server does not test the display REA creates and cleans up.
For a full Linux Hopper setup check, run `npm run verify:setup:linux` on a clean
root Linux runner. Set `REA_VERIFY_SERVER_COMMAND` to the installed REA command.
This checks setup, doctor, CLI/MCP analysis and repeated setup on that runner.
Report which host, package format and steps you tested. Inspecting a Linux package
on macOS verifies package contents, not a working Linux installation.
## Test design and pruning
Prefer real public CLI/MCP workflows, then production-boundary integration,
@@ -570,6 +650,27 @@ and toolchain conditions when comparing timings.
## Lifecycle regressions
Investigation lifetime changes exercise public MCP target A/B switches, target-free
close and close cleanup failure/retry, with exact retained IDs and complete Unknown
revision histories. Verify engine replay/cache removal separately from retained
facts. An explicit reset must preserve a healthy engine and existing metadata
invalidation, drain an admitted signal-ignoring late result, block later work,
and prevent stale replay/source bindings from restoring old facts. Cover checkpoint
overwrite refusal, publication followed by cancellation or staging cleanup failure,
cleanup retry, then fresh-process bundle import with identical original references
and Unknown history. Capacity admission tests preserve the complete pre-state on
failed record/import/Unknown transactions, count duplicates once, account for
metadata size deltas and release budget on reset. Controlled providers and cleanup
seams establish ordering and failure semantics; real application/provider runs are
reported separately.
Hold successful and error replies at the MCP transport boundary while reset
queues behind their original admission. Verify checkpoint contents and publication
order, including SDK output validation failure, handler failure and cancellation
before or during send. Handler cleanup must settle even when the SDK suppresses
the reply. An outgoing server request with the same numeric ID as an incoming
tool request must not acknowledge that tool's response.
For changes to resource acquisition, readiness, cancellation, or cleanup, select
the affected owners and applicable scenarios below. Include temporary snapshots
and one-shot commands. Use public CLI/MCP workflows where practical and a
+84
View File
@@ -99,6 +99,49 @@ include the smallest useful reproduction, impact and coverage limits. Do not
claim autonomous-agent selection from a scripted trace. Check related reports
before proposing another issue, and avoid treating lack of API parity as a bug.
## Design recovery around the failure
Distinguish a request that violates its input contract from a valid selection
that the current target, provider inventory, or project cannot resolve. The
same `invalid_request` code can represent either. Suggest schema inspection
only when the input contract is the problem; for a valid lookup, preserve the
caller's exact selector, the observed reason and any candidates, and give a
next step that addresses that outcome. An inventory miss does not prove that
the artifact lacks the requested item.
For prerequisite failures, identify what the requested operation actually needs.
A missing executable, a wrong path, an incompatible host or version, denied
permission, and a download timeout need different repairs. Preserve the observed
failure instead of calling all of them "not installed".
Derive the required engine from the operation's execution contract. For example,
`inspect_firmware_regions` needs Binwalk; `extract_firmware` needs Unblob.
Configuring the other tool cannot enable the requested operation. An installed
JDK outside the selected PATH may need `JAVA_HOME`, not another installation.
Say where a setting must change: a shell export does not change an already
running MCP server's environment.
Keep repair instructions with the contract that owns the failure, and preserve
them through provider readiness, capability discovery, doctor and CLI/MCP output.
An error description is not a repair instruction. Do not make each consumer
guess the fix from diagnostic text or invent another list of prerequisites.
Keep recovery within the selected target, provider, project and authority. If
another context could help, describe it as an explicit choice and state any
relevant loss of state; do not switch or select a candidate automatically.
Preserve uncertainty, partial observations, and session or cleanup state. A
suggested retry must be safe and explain any required change or wait.
Verify recovery through the public workflow: reproduce the failure, follow
the suggested next step when feasible, and check the result and session state.
Include a malformed-input control so schema guidance remains available when it
is actually needed. Compare CLI and MCP when both expose the operation.
For each confirmed bug, check sibling adapters and other consumers of the same
facts. Reproduce suspected matches and retain a working comparison where one
exists. Group reports and fixes around the shared cause; similar wording alone
does not establish another instance.
## Define the contract
- Use strict inputs with explicit target identity and required fields. Preserve
@@ -228,6 +271,47 @@ artifact can execute successfully.
## Own the lifecycle
The investigation owns authenticated Evidence and complete Unknown revision
history independently of an engine. `close_binary` drains earlier admitted work,
releases engine resources and clears replay/invalidation state; investigation
references survive, including target-free closes and cleanup failures. A target
switch does not make historical observations current for its new engine.
`reset_investigation` is the explicit destructive boundary. It drains earlier
analysis, imports, exports and Unknown operations, then clears investigation and
replay/source bindings while retaining a healthy active engine. Metadata mutation
invalidation remains in force until that engine is recreated. Later admitted work
waits for the transition; earlier late results cannot repopulate cleared history.
Runtime disposal seals new acquisitions and releases records after successful
owned engine/file cleanup; failed cleanup keeps the owner available for retry.
Records are retained in memory under a fixed per-owner representation allowance
derived from one quarter of initial available V8 heap. Admission accounts for
Evidence, Unknown history and indexes before atomic commit; metadata replacement
uses its size delta and duplicates do not accumulate. There is no silent eviction.
`binary_session.investigation` reports committed counts, estimated representation
bytes and the allowance; these estimates are not process RSS measurements. Live
authentication headroom is checked separately. On refusal, export a checkpoint,
reset, and import only the required investigation, or start a process with a
larger heap. Retention does not promise automatic persistence.
An MCP request retains its original session admission until its handler cleanup
and response send settle. A reset queued behind that request therefore cannot
invalidate Evidence before its reply is sent. Nested work borrows the same
admission. Cancellation before sending releases a suppressed reply after handler
cleanup; cancellation after sending starts still waits for transport settlement.
Lifecycle tools drain earlier admissions without acquiring one themselves.
`reset_investigation(save_path)` saves the complete cross-domain canonical bundle
once before clearing. It awaits file sync, atomic publication and staging cleanup;
failed save, incomplete cleanup or cancellation observed before reset commit
preserves all records. Publication
can precede cancellation or cleanup failure; the error retains its committed file
receipt. Import that bundle to restore original Evidence IDs and all Unknown
revisions in a new process. Engine replay snapshots remain a separate target and
profile-bound mechanism. This checkpoint contract does not promise recovery from
an uncheckpointed process exit or parent-directory loss after a host crash.
`SafeOutputTree` publishes only after file readback, nested-directory fsync
from children to parents, and root-parent fsync on hosts that support directory
syncing. Windows skips directory fsync. Failed writes retain a separate pending
+1
View File
@@ -50,6 +50,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
36,src/main/reportMcpTransportError.ts:reportMcpTransportError,MCP error has no known validation or disconnection identity,MCP reported an error with unknown connection state. Check whether the client is still connected before following connection-loss recovery.,Connection state remains unknown,Keep current copy,implemented,injected unknown-error recovery regression
37,src/main.ts:serveStdio throw,MCP connection cannot be created,Failed to start MCP stdio transport,Exposes transport implementation and gives no recovery,"Retry once. If it fails again, tell the user REA could not start its MCP connection and ask them to restart the MCP client, then run `rea doctor`.",implemented,verified by injected transport startup-throw runtime test
38,src/main.ts:requestShutdown catch,Connection or analysis session close rejects,Failed to shut down cleanly,Does not explain duplicate-process risk or recovery,REA could not close cleanly. End the REA process before starting it again.,implemented,verified by injected handle-close rejection runtime test
38.5,src/domain/analysisErrorRecovery.ts:cleanupRetryRemediation,close_binary returns cleanup_incomplete while the session retains the failed cleanup owners,"Provider cleanup could not be fully confirmed. Review the reported local resources before opening another target. (retryable: false, no recovery)",Agent could abandon the connection that owns the only cleanup retry,"Append: This connection retains that cleanup. Keep it open; after the reported condition settles or is resolved, call close_binary without snapshot_path and check its result. That call retries the retained cleanup without rerunning the failed analysis, preserves investigation Evidence and Unknown history, and can report cleanup_incomplete again. Add recovery.kind=retry_cleanup naming close_binary; keep retryable false",implemented,MCP close_binary with the real KunaProvider and a fake Kuna whose export stop stays unproven; process-capture MCP retry; session tests for an unoffered one-shot close and for failed and completed snapshot saves; one-shot IDA CLI cleanup reports no hint; real Kuna not run
39,src/main.ts:entrypoint catch,Unhandled rejection escapes run(),Unexpected server startup failure,Vague and lacks diagnostic recovery,"Do not repeat this startup. Tell the user REA could not start and ask them to restart the MCP client, then run `rea doctor`.",implemented,verified by injected entrypoint rejection test
40,src/application/Update.ts:unknown-install-method,Running package is not owned by detectable global npm installation,Update manually with: npm install --global rea-agents@latest,Old global-npm advice does not identify the owning installation,"Update this installation with the package manager or source checkout that owns it. For a one-off latest invocation, use npx rea-agents@latest.",implemented,36 Update tests passed; public CLI unowned-installation refusal verified with isolated package-manager fixtures
41,src/application/Update.ts:install,npm global install subprocess exits unsuccessfully,npm install --global rea-agents@latest failed.,Old registry/global-install advice omits the actual version prefix and installer failure,"REA could not install {version} in {prefix}: {failure}. Repair this npm installation, then rerun rea update.",implemented,36 Update tests passed; public CLI owned npm fixture failure preserved version prefix and installer reason
1 harm_rank location trigger current_copy user_risk proposed_replacement implementation_status verification_result
50 36 src/main/reportMcpTransportError.ts:reportMcpTransportError MCP error has no known validation or disconnection identity MCP reported an error with unknown connection state. Check whether the client is still connected before following connection-loss recovery. Connection state remains unknown Keep current copy implemented injected unknown-error recovery regression
51 37 src/main.ts:serveStdio throw MCP connection cannot be created Failed to start MCP stdio transport Exposes transport implementation and gives no recovery Retry once. If it fails again, tell the user REA could not start its MCP connection and ask them to restart the MCP client, then run `rea doctor`. implemented verified by injected transport startup-throw runtime test
52 38 src/main.ts:requestShutdown catch Connection or analysis session close rejects Failed to shut down cleanly Does not explain duplicate-process risk or recovery REA could not close cleanly. End the REA process before starting it again. implemented verified by injected handle-close rejection runtime test
53 38.5 src/domain/analysisErrorRecovery.ts:cleanupRetryRemediation close_binary returns cleanup_incomplete while the session retains the failed cleanup owners Provider cleanup could not be fully confirmed. Review the reported local resources before opening another target. (retryable: false, no recovery) Agent could abandon the connection that owns the only cleanup retry Append: This connection retains that cleanup. Keep it open; after the reported condition settles or is resolved, call close_binary without snapshot_path and check its result. That call retries the retained cleanup without rerunning the failed analysis, preserves investigation Evidence and Unknown history, and can report cleanup_incomplete again. Add recovery.kind=retry_cleanup naming close_binary; keep retryable false implemented MCP close_binary with the real KunaProvider and a fake Kuna whose export stop stays unproven; process-capture MCP retry; session tests for an unoffered one-shot close and for failed and completed snapshot saves; one-shot IDA CLI cleanup reports no hint; real Kuna not run
54 39 src/main.ts:entrypoint catch Unhandled rejection escapes run() Unexpected server startup failure Vague and lacks diagnostic recovery Do not repeat this startup. Tell the user REA could not start and ask them to restart the MCP client, then run `rea doctor`. implemented verified by injected entrypoint rejection test
55 40 src/application/Update.ts:unknown-install-method Running package is not owned by detectable global npm installation Update manually with: npm install --global rea-agents@latest Old global-npm advice does not identify the owning installation Update this installation with the package manager or source checkout that owns it. For a one-off latest invocation, use npx rea-agents@latest. implemented 36 Update tests passed; public CLI unowned-installation refusal verified with isolated package-manager fixtures
56 41 src/application/Update.ts:install npm global install subprocess exits unsuccessfully npm install --global rea-agents@latest failed. Old registry/global-install advice omits the actual version prefix and installer failure REA could not install {version} in {prefix}: {failure}. Repair this npm installation, then rerun rea update. implemented 36 Update tests passed; public CLI owned npm fixture failure preserved version prefix and installer reason
@@ -0,0 +1,246 @@
# Capability composition verification — 2026-11-01
Scope: issue #8429, the service composition/ownership increment of #740. Baseline
`6e9ee97a69cb16c7150d8b550e705cba055ee4f0`; implementation `bd9bf76d8`, followed
by type-only binding cleanup `161c3c146`. This changes existing composition,
metadata ownership and cleanup bookkeeping; no new provider/target authority.
`BinarySession` keeps its existing transition/admission barrier.
## Executed source and boundary checks
Full typecheck, module boundaries, architecture guards, test discovery,
production reachability, documentation consistency and repository formatting
passed. Changed-file typed lint passed with one worker and `GOMEMLIMIT=512MiB`:
39.622 s elapsed, 39.261 s process-family CPU, 2,945,368,064 B kernel peak.
Initial whole-repository typed and ordinary lint attempts hit the configured
3 GiB early stop; no OOM occurred. Whole-repository lint is not locally
qualified; changed-file checks and the final ownership-boundary typed checks
passed separately.
Relevant executed groups: 216 parser/boundary/metadata/provider-owner tests;
5 scope/factory tests; 27 MCP availability/composition tests; 28 deferred
startup/protocol/publication tests; 50 CLI factory/cleanup/target-input tests;
16 SDK schema tests; 4 compiled-verifier metadata link tests; and investigation
lifecycle, unknown-history and transport cleanup regressions. These groups
contain overlapping repeated checks and are not a unique total. The final
Android transport group passed all 4 cases, including construction failure and
failed cleanup retry. The moved Frida test body was preserved byte-for-byte;
its discovery owner changed to the adapter lane.
The old manual provider-close list was temporarily restored as a negative
control: the new successful-peer cleanup regression failed, then the current
source was restored. Synchronous close throws, undefined rejection reasons,
concurrent close joining, borrowed optional observations and failed-owner retry
are controlled tests, rather than claims of natural failures in real engines.
An initial new transport assertion omitted the existing outer
`OwnedMcpServers` AggregateError; its expected nesting was corrected. Other
preserved harness failures were missing SDK catalog preparation and two wrong
CLI test paths; corrected commands passed. One formatting issue was corrected.
Ordinary Knip reports one unused exported `DarwinHelperCache` type at
`src/process/DarwinHelperCache.ts:465`. An independent run on the clean pinned
baseline reports the identical sole issue; that file is unchanged here.
## Real applications and complete-result parity
Every case used actual public CLI/MCP routes. Inputs were hashed before/after,
with no source mutation. Both arms used pinned Node 24.22.1/npm 11.16.0 and
fresh isolated processes/configuration; OS page caches were uncontrolled.
| Domain | Genuine application/artifact | Independent fact oracle |
| ---------- | ------------------------------------------------------------------- | ----------------------------------------------- |
| Go | installed GitHub CLI `/usr/bin/gh`, Go 1.19.6 | ELF Go build-info bytes/module data |
| Managed | SDK 8.0.427 `NuGet.CommandLine.XPlat.dll`, selected Program members | independent dnfile metadata/CIL facts |
| JavaScript | installed agent-browser 0.27, selected entry source | original source declarations/imports |
| Android | NewPipe 0.29.1 APK inventory/application graph | complete ZIP member census and 9 member digests |
| WASM | actual CSS WASM artifact with WABT 1.0.42 | independent raw sections/types/exports |
All five full public CLI Evidence results matched across arms. Explicit Go/JS
per-arm bundle output paths were verified separately, and their complete saved
parent Evidence matched. MCP performed 117 calls per arm, including six fresh
producer calls, retained derivation, repeated close, checkpoint/reset, expected
missing references, same-process restore and actual fresh-process restore.
Complete producer bundles, full derived Evidence, exact requests/IDs and
complete checkpoints matched across arms. The analyst Unknown revision 1→2
history and every canonical mutation/history record remained authenticated;
fresh re-exported checkpoints were byte-identical. Fresh consumers ran no
producer calls. Ten independent domain fact assessments passed.
| Completed workflow | Baseline elapsed / CPU (s) | Feature elapsed / CPU (s) | Baseline / feature kernel peak (B) |
| ------------------------ | -------------------------- | ------------------------- | ---------------------------------- |
| Five CLI tasks | 17.675 / 16.268 | 22.192 / 20.667 | 393,936,896 / 390,504,448 |
| Five MCP lifecycle tasks | 68.270 / 65.500 | 61.201 / 58.041 | 731,680,768 / 776,331,264 |
Times are single observations including setup and all calls, not a causal
speedup/regression claim. Fresh-process tasks are reported separately from
same-process/restored consumers; neither is a controlled cold OS-cache run.
All tasks ran serially under a one-CPU, 4 GiB hard/3 GiB early-stop, no-swap,
192-task guard. CPU and peak memory are kernel process-family counters at
payload exit while the observer remained alive; all completed tasks verified
owned cleanup and no OOM. Sampling/RSS is not substituted for these counters.
Scripted checks do not incur model tokens; model usage is recorded separately.
## Native engine and Codex CLI
Real Ghidra 12.1.4/JDK 21 on `/usr/bin/jq` passed 37 public MCP calls per
arm. Two new light address queries before/after healthy investigation reset
returned identical Evidence. The same MCP client retained the same owned JVM
PID, start time, process group and public run identity. Three retained native
compositions used authenticated historical dossiers; these dossiers were not
regenerated. Full derived results, original bundles and Unknown histories
matched across arms and survived same/fresh-process restore.
A mixed-routing task passed 58 calls per arm, with explicit `jq → APK → managed`
metadata target switches and five new static producer families through the final
registry. These metadata switches do not imply launching every target engine.
Complete originals, derived results and checkpoint history matched across arms.
| Completed task | Baseline elapsed / CPU (s) | Feature elapsed / CPU (s) | Baseline / feature kernel peak (B) |
| --------------------------- | -------------------------- | ------------------------- | ---------------------------------- |
| Ghidra/native lifecycle | 38.196 / 36.804 | 41.741 / 40.369 | 1,111,789,568 / 1,098,362,880 |
| Mixed target/domain routing | 27.557 / 26.984 | 27.255 / 26.527 | 1,039,253,504 / 1,037,910,016 |
The native task includes a fresh engine start; its second query is a healthy
same-engine observation. Historical dossier reuse and fresh process consumers
are separate phases. These are single observations with uncontrolled OS caches.
Local `codex-cli 0.165.0` ran one actual `client-code` trial with the requested
`gpt-6.1-terra` alias, reasoning `high`, and 19 selected tools. The resolved
server-side model identity is not inferred. It completed 32 MCP calls across
JavaScript, managed, Android and WASM: five new producer calls, retained
compositions, four expected missing-reference controls, close/reset/save/import
and Unknown revision 1→2. There were no unexpected MCP errors or timeout.
The model's JS self-check remained explicitly **partial/unverified**: it used
`JSON.stringify`, whose property ordering changed after canonical import.
Independent full structural comparisons of all three actual results and the
baseline proved identical fields, values, array order and Evidence ID. The
first ordering difference was only the keys of
`graph.nodes[0].observations[0].properties`. This distinction is retained in the
assessment; the model's self-check is not relabeled a complete pass.
An actual new, target-free MCP process replayed the model's checkpoint in
9 calls. Complete Evidence and Unknown revision/mutation histories authenticated,
and its 18,968,472 B export was byte-identical (17 Evidence records,
2 Unknown revisions, 1 head), SHA-256
`165188d74298f0173c5730973ecb5354a1b966ca54ec46ee177efa049ebbc755`.
All four derived results and their complete producer records matched the
independent baseline. The copied authentication profile was removed; runtime,
production source and selected input hashes remained unchanged.
Actual client usage for the single attempt: **336,490 input tokens**, of which
**295,680 were cached**; **3,659 output tokens** and **250 reasoning output
tokens** were reported. Cached input is a subset, not an additional token total.
Baseline model usage was not measured; neither token savings nor a speedup is
claimed. Scripted source-fact and fresh-process verifiers incur no model API
tokens and are separate from this model usage.
| Completed verification | Elapsed / process-family CPU (s) | Kernel family peak (B) |
| ----------------------------------- | -------------------------------- | ---------------------- |
| Actual Codex CLI trial | 74.644 / 26.818 | 1,061,597,184 |
| Independent fresh checkpoint replay | 6.142 / 5.934 | 757,895,168 |
Manual final-answer review verified the reported IDs, counts, digests, coverage
and Unknown state against complete baseline values, all actual model calls and
the independent replay. The outcome is supported with the explicit model JS
self-check limitation resolved independently. Maximum observed in-flight MCP
calls was one; no target command, web or file-change tool items occurred.
Factual-assessment SHA-256:
`33451102e5faf57f11c627481899f16c4fc47fa5cba5f12cd2ed781a3de54ccf`.
Final immutable input/runtime audit SHA-256:
`73abce493182a7b433b4b78737720640ceb41d6df435cb0fec915468a0a304c8`.
## Catalog and build identity
Both compact/full SDK catalogs are byte-identical to baseline: 234 tools,
6 prompts, 180 CLI commands. The provider projection changes from 63 to 67 rows
only for source-proven web-source/module tracing, Harmony and QLOG provenance
corrections. All other product sections are unchanged. Its expected provider
SHA-256 is `4234ed15d7e53f34599b59dfba1f678369c0ae95b2135c7ba3414d944ab156b6`.
The 66 static declarations have 49 runtime keys and 17 explicit source-owned
workflow/engine groups; 70 bindings union to 67 identities. Static imports reach
44 pure modules without schema, engine or factory runtime imports.
The baseline runtime is `0e94962f64f3f532cc96b2432916f328f60058e97cd0e8e25f5c3e6ed3cdec10`
(2,670 files). Static pairs used clean `bd9bf76d8`, runtime
`62018a023b185dd9c6839799ed1fe4f085810161c723e508fc817d2fccdb59e3`
(2,713 files). Final type-only cleanup `161c3c146` rebuilt to
`d7be1d17dc6b5c23f33468398656ac4d3961380368be1f70dc767f0ba1fac6bf`.
The complete earlier manifest is cryptographically reconstructed by removing
one emitted trailing comma in `dist/composition/capabilityServices.js`'s
existing named import; every other runtime file is byte-identical. This is not
a claim that the two runtime hashes are identical. Native/mixed/model cases use
the final runtime.
Raw inputs, outputs, commands, hashes, recorded failed checks and resource receipts
remain outside Git under
`/home/yann/yann/.rea-worktree-artifacts/20261101/capability-module-ownership/`.
Static-pair audit receipt SHA-256:
`801707569577c7cec848334cf72bfab26a7ff4a95c2a87872d71d78eef7765a3`.
## Review corrections and final qualification
Two actionable review findings were fixed at their producing owners:
- `980991573`: import-edge facts now distinguish fully erased declarations from
inline type bindings, which still load their module under
`verbatimModuleSyntax`. Compiler-backed parser/boundary regressions passed;
the old parser fails the new expectations. A separate three-file probe through
the actually selected compiler emitted both inline runtime edges and erased
whole-declaration/type-query edges, without compiling the repository.
- `de6a61636`: the existing contract owner exposes 47 named canonical groups.
Thirty-seven full provider bindings reference those shared arrays, while
33 deliberate provider-subset/engine selections remain precise. The actual
ADB metadata now includes a new valid canonical ADB operation without an edit
to the provider declaration; reverting to the copied production lists fails
the regression. Unknown groups, unknown names and identity conflicts fail
catalog generation.
Final full typecheck, module boundaries, test discovery and changed-file typed
lint passed. Post-review source regressions passed 228 cases and SDK/MCP
regressions passed 34; the two strengthened boundary/metadata files passed
31 cases separately. These overlap earlier groups. Product catalog SHA-256
remains exactly `e942c72047db651939dd47a31042e6818caaca7470092d63645801954a72b523`;
both SDK catalogs remain byte-identical to the original baseline. Static metadata
retains its pure loading boundary; canonical group IDs are imported only as types.
The final implementation freeze is `de6a61636f3f84862cb4263c9ea219b7d2d4de36`,
with 2,713 runtime files, SHA-256
`be1d60c7dcca57eeb7ad09d682efd172c7f1679215e796a291b29eb52e29bda1`.
Earlier model/engine measurements remain attributed to their executed revisions;
they are not relabeled as runs on this later freeze.
A new actual mixed-routing task on this final freeze passed all 58 calls:
32.970 s elapsed, 32.180282 s process-family CPU, 1,042,538,496 B kernel peak.
Independent five-domain source facts, complete producer records and all five
full derived results/requests/IDs matched baseline and the prior qualified
implementation. Same-process and true fresh-process restore preserved the full
Unknown 1→2 mutation history and byte-identical 18,978,040 B checkpoint
(20 Evidence records, 2 Unknown revisions, 1 head), SHA-256
`0316bd6271c6d401a18c10d552b2005d39c98f3bbbc5945ae4823875af13fd5f`.
Owned cleanup, runtime/production-source/input hashes and no OOM were verified.
Append-only final-routing receipt SHA-256:
`9c410c2c31bf38d7edc569deb95b93b7f9a2cf68a7b9a7f84d3ec924f0ad75cb`.
This bounded requalification adds no model invocation or repeated native
analysis query; earlier token measurements remain unchanged.
CodeQL's six reported alert IDs (215, 490, 491, 493, 494, 495) all have open
instances on exact-base main `6e9ee97`. All 24 SARIF paths were reviewed and
independently checked: the new factory-default `process.env` sources retain the
existing command/directory selection and explicit local authority. The shared
filesystem/process sink files are byte-identical to base; Cutter's sink line
shift only removes its old identity constant. No source suppression or global
alert dismissal is included. The PR records the exact-base instance evidence
and the owner's authorized exception for this unrelated existing-alert check.
## Qualification limits
NewPipe coverage is static inventory/graph, not real JADX decompilation: JADX,
apktool and aapt2 were unavailable. No genuine local GGUF/safetensors target was
available. Real Hopper, IDA, JEB, Cutter GUI, GDB/Rizin debugger and Frida target
attachment are not qualified by these application cases; relevant factory,
protocol and cleanup tests use controlled seams. Historical native dossiers
are explicitly labeled reused evidence, not newly reproduced function analysis.
Retained records remain bounded by existing investigation policies; this does
not promise unlimited retention or a new durability backend. Further module
qualification and `BinarySession` simplification remain under parent #740.
@@ -0,0 +1,298 @@
# Investigation record lifetime verification
Verified on Linux with Node v24.22.1 and local Codex CLI 0.165.0 for
[#8347](https://github.com/morluto/rea/issues/8347). The final production revision is
`40ec9fe79b221b7550bf6a652f55389ea0b7e54f`, including main
`d54a4661da20382ad8456564a08569fdd24131c5`. Its runtime manifest SHA-256 is
`f4e55abdc5cb9fc2a86efcec5c410f2b5bc3eb5a7109a8d5ab88f07f8511c48b`.
Earlier measurements below identify their older revisions explicitly. Documentation
was being updated during final qualification; production sources and runtime
remained unchanged.
The [MCP contract](../mcp-contracts.md) separates engine cleanup, investigation
reset and explicit checkpointing. Automatic journaling remains tracked in
[#4809](https://github.com/morluto/rea/issues/4809).
## Response publication qualification
The request-publication correction is pinned to
`40ec9fe79b221b7550bf6a652f55389ea0b7e54f`. An MCP request now retains its
originating session admission through handler cleanup and physical send
settlement. Nested operations borrow that admission, avoiding reentry behind a
queued reset. Cancellation before a suppressed send and during an already-started
send have separate completion rules. Outgoing server requests cannot acknowledge
an incoming tool response merely because their IDs coincide.
All nine genuine saved-source workflows were repeated: 313 MCP calls and 19 CLI
checks, 114.210 s guarded elapsed, 109.411889 s CPU and 1,055,035,392 B maximum
family peak. Complete original public results and source identities matched
through close, checkpoint refusal, reset and same/fresh-process recovery; all
Unknown histories were preserved. No producer replay, source/runtime changes,
OOM or unresolved owned cleanup occurred. These runs have uncontrolled OS caches
and are not a paired timing comparison with the earlier cohorts below.
A new local Codex CLI task requested `gpt-6.1-terra`/`high` with the scoped
four-domain profile. It completed 25 MCP calls, including four expected
missing-reference controls, with no producer replay. Usage was 232,677 input
tokens including the 199,552 cached subset, 3,613 output tokens and 195 separately
reported reasoning tokens; elapsed was 61.466 s, local CPU 11.031356 s and peak
806,146,048 B. Requested aliases do not establish the server-resolved model
identity, and remote model compute is unmeasured.
An independent fresh stdio process authenticated that task's 21-record,
three-revision checkpoint: 16,138,657 B, SHA-256
`a4395246473397cdc9040cb643e3175fd6f8b24db40597f8b680f0e065643df5`.
Nine calls reproduced all four complete original results and source records and
reexported byte-identical bundle bytes. This took 6.353 s elapsed, 6.060161 s CPU
and 592,621,568 B peak, with verified cleanup and no OOM.
Independent full-result/source/history authentication and manual answer review
supported the model's stated conclusions. No task or model retry was needed in
this cohort; separate checker and report-aggregation CPU was not measured.
The actual Ghidra healthy-engine control also passed on this runtime: 49 calls,
51.115 s elapsed, 50.000976 s CPU and 1,124,110,336 B peak. The verifier-owned JVM
PID/start identity/process group, public analysis run and MCP client were the same
across reset. Two explicit `address_name` queries returned identical Evidence;
no dossier, inventory or decompile producer was replayed. Full checkpoint/history
recovery and owned cleanup passed, with no OOM. This observes retained-engine
behavior rather than a before/after speedup.
Across all three model cohorts, nine actual runs plus one prelaunch failure used
2,771,744 input tokens including the 2,468,608 cached subset, 23,334 output tokens
and 1,035 separately reported reasoning tokens. Guarded elapsed totals 480.694 s
and local CPU 87.103671 s; maximum family peak is 922,148,864 B. All earlier
failed/partial attempts remain included. These are execution costs, not a paired
model-token savings claim.
All nine controlled success/error publication cases passed. Reinstating only the
old admission factory made both successful-publication cases fail at the original
admission boundary; the corrected source was restored and all nine passed again.
The final boundary run used 12.872 s elapsed, 12.647482 s CPU and 941,723,648 B
peak. Existing lifecycle/deferred/shutdown tests passed 79/79; the full HTTP suite
passed 74/74. These controlled seams establish ordering, separately from real
application behavior.
CI also exposed the 1500-file JavaScript timeout fixture's retention headroom.
At a 512 MiB parent heap, the original timeout exceeded the 10 MiB wire budget;
fallback authentication projected 771,325,184 B against 411,666,312 B available
and correctly refused. The transport/timeout fixture now uses a finite 1536 MiB
parent heap while retaining all 1500 files and every recovery/query assertion.
Its worker remains at 128 MiB; the separate memory-pressure case retains its
512 MiB parent. All three tests passed in 23.556 s elapsed, 22.989758 s CPU and
1,064,308,736 B peak, including a public assertion that retained records fit the
fixed allowance. No production limit was raised for this fixture.
## Earlier review qualification
Review corrections were qualified at clean
`6d875b3ff73d40b365a713cf4ba69a10cca257b7`, with runtime manifest SHA-256
`328f05dc2d7b114c33dc0406e488c3046a43e1c12187e15ad9263be65df7cbaa`.
SDK error Evidence retention and reply publication now share session admission;
reset retries the MCP artifact scope alongside process cleanup before saving or
clearing. Controlled public MCP regressions cover both publication orderings,
queued cancellation, terminal shutdown, ordinary export cleanup failure, mixed
cleanup outcomes and scope reuse. Reinstating the former wiring made four of the
five new regression cases fail; all five pass with the correction. Controlled
waits also use bounded acknowledgements and release/join held work in `finally`.
The nine genuine saved-source cases below were repeated against this runtime:
**313 MCP calls and 19 CLI checks**, 118.510 s guarded elapsed, 114.859544 s CPU,
and 1,065,390,080 B maximum kernel-family peak. Every complete original result,
ID, source, coverage and Unknown revision chain matched through close, checkpoint
collision refusal, reset, same-process import and fresh-process recovery. No
producer or target open was replayed in this cohort. Input/checkpoint/runtime
immutability and owned cleanup were verified, with no OOM. These normal workflows
complement the controlled error/cleanup timing regressions; they do not reproduce
those timing faults themselves.
One further local Codex CLI task requested `gpt-6.1-terra`/`high` with the
scoped four-family code-mode profile. Its 25 MCP calls included four expected
missing-reference controls and no unexpected model errors or producer replay.
Independent full-result/source/history authentication and answer review supported
the answer within its stated scope; it did not claim to query every other
Unknown. The run used 58.269 s elapsed, 9.837083 s local CPU and 885,448,704 B peak,
with 239,931 input tokens including 215,168 cached, 3,721 output tokens and 277
separately reported reasoning tokens.
A real fresh stdio client restored that same new checkpoint: 21 records and
three Unknown revisions in 16,138,749 B, SHA-256
`760fc74951abc292a3ff2fe1771391ac0d4345416b2345dfa4e85cbb6ce96181`.
Nine calls reproduced all four complete original results and source records,
then reexported byte-identical bundle bytes. This took 5.443 s elapsed,
5.253212 s CPU and 595,218,432 B peak, with verified cleanup and no OOM.
Two external-checker failures were preserved and corrected against actual event
ordinals and query filter/paging semantics; neither required model or backend
replay. Separate checker CPU and remote model compute were not measured.
Across those two earlier cohorts, eight actual model runs plus one prelaunch failure used
2,539,067 input tokens including the 2,269,056 cached subset, 19,721 output tokens,
and 840 separately reported reasoning tokens. Guarded elapsed totals 419.228 s
and local CPU 76.072315 s; maximum family peak remains 922,148,864 B. Initial
failed/partial attempts below remain included. This is execution accounting,
not a paired model-token performance comparison.
That runtime also repeated the actual Ghidra healthy-engine control:
49 calls, 44.333 s elapsed, 43.348651 s CPU and 1,093,464,064 B peak. The same
verifier-owned JVM PID/start identity/process group, public analysis run and MCP
client survived reset; two `address_name` queries returned identical Evidence.
Startup-inclusive and warm requests took 24,953.573 ms and 6.762 ms respectively,
without dossier, inventory or decompile replay. These are startup/reuse
observations rather than a PR before/after speedup. Full checkpoint/history
recovery and owned cleanup passed, with no OOM.
## Initial main-integration workflows
Main was integrated in `4e20698b7b9883f659e445c059c011aa0e5de834`, with runtime
manifest SHA-256
`ee23cbaf0ecbf8d896fb5dc7a88317e2ab8eb77bd3a4467bf5793c3471ca9c7d`.
Nine genuine saved-source cases covered eight target families:
| Family | Actual source | Saved-source bundle SHA-256 |
| ---------- | ---------------------- | ------------------------------------------------------------------ |
| Native | jq observations | `0a3adad6ab39bfe29320e1f070a8e598382d2f17c4d6c19079bd6ff16dfe0937` |
| JavaScript | agent-browser 0.27.0 | `66232b6eb01b3dbf435949a1cd94cb2532b7f989b91cdf952a105b3cee231bda` |
| Managed | NuGet CLI, SDK 8.0.427 | `11ac63842299c0ac641a320ec9b2a5161b454a8a322d57724fd5f2ffb2fddfa4` |
| Android | NewPipe 0.29.1 | `fa499d0464a40ddc2f5159c893d68b8f50aa8e6d2b0080de22f2f3e5fc326825` |
| Harmony | Shark 1.0.8 APP | `3a526ab0113f80ada26ff5a0631fc95dac91485946f20584fc5a76f5ee8eb880` |
| Harmony | Shark inner HAP | `3431c0a347f331c3effacf7984ecb890f3fef46d8b00b03b50980023b065cd4a` |
| Apple | Rectangle 2.0.4 | `8daa6286b28aadf5ba0d41038046805e5c69874eaa0bc5daf0e265360fcae79d` |
| WASM | tree-sitter-css | `acb0cb6c3e0d520f83144e106e171c3ecebba70c5c0498bbb0113185a3e4bf95` |
| Process | Controlled jq captures | `55da0f130dc2265aae54261d49949471e5c929163c2b7314b42e95df3325b739` |
Each case imported authenticated observations, composed exact retained references,
closed without an active target, refused an existing checkpoint destination,
saved and reset, checked missing references, and restored into the same process
and a fresh public stdio MCP process. Complete original public results, source
links, coverage and Evidence IDs matched throughout. Every case included an
explicit analyst question with revisions 1→2; complete revision histories and
mutation Evidence authenticated after recovery. Bundle hashes above identify
the saved observations rather than release binaries. The actual jq target was
`/usr/bin/jq`, SHA-256
`90b1eef4498f31836342898fc6e97cfeb8a197c4366773a0fb120ae02a45ea52`.
That cohort completed **313 MCP calls and 19 public CLI checks**, without
source producer replay. Summed guarded elapsed was 119.445 s and CPU was
115.040362 s; maximum kernel cgroup-family peak was 1,047,826,432 B. All pinned
input/runtime hashes stayed unchanged, owned cleanup was verified, and no resource
stop or OOM occurred.
A separate actual Ghidra 12.1.4 control reset while its jq engine remained open.
The same verifier-owned JVM PID/start identity/process group, reported analysis
run and MCP client survived reset. Two identical `address_name` queries at
`0x104500` returned the same Evidence. The first request including startup took
21,023.963 ms; the post-reset request took 6.672 ms. This observes startup versus
reuse, rather than a PR before/after speedup. No function dossiers or procedure
inventories were regenerated. The complete close/checkpoint/import/fresh-process
control used 49 calls, 39.351 s elapsed, 38.222523 s CPU and 1,094,340,608 B peak;
reset acknowledged 15 records and three Unknown revisions with `retained_target`.
Earlier actual-provider controls also passed Ghidra→Rectangle provider-family
switching and Rectangle→NewPipe target switching, with reset retaining the healthy
second target. Those pre-sync runs preserved their complete original results and
histories; they are separate from the latest-runtime qualification above.
## Equivalent completed-task measurements
These pairs used exact baseline `60149282fcfc05ac32fbc94434c107710120da6f` and
implementation `3c9fb7f44a4496d18d893beb903c5f2ba68c06b4`, before the later main
integration. The baseline exported, closed and imported before reference
composition; the candidate closed and reused references directly. Both returned
the same complete original result for the same authentic inputs.
| Saved source | MCP calls, before→after | Elapsed s | CPU s | Kernel-family peak B |
| ------------ | ----------------------- | ----------- | ----------------- | ----------------------- |
| Native | 9→7 | 5.846→4.938 | 5.609106→4.760475 | 582,672,384→511,090,688 |
| JavaScript | 8→6 | 3.529→3.333 | 3.371675→3.163932 | 394,170,368→387,428,352 |
| WASM | 7→5 | 4.236→3.731 | 4.035871→3.520731 | 658,063,360→492,548,096 |
Each row is one sequential pair in fresh isolated processes. OS caches were
uncontrolled; builds and fingerprinting were measured separately. These samples
establish two fewer necessary calls, rather than statistical speedups or model
token savings.
## Initial Codex tasks
Local Codex requested `gpt-6.1-terra` with `high` reasoning. Native used the full
advertised surface without an enabled-tool filter or exact-name guide, both
before and after the latest main integration. JavaScript, packages and the mixed
managed/WASM/process/Harmony task used explicit scoped diagnostic profiles. The
mixed task used the existing experimental client code mode; its development
warning was retained.
All four task scopes passed independent canonical bundle/history authentication,
complete original-result comparison and final-answer review. These conclusions
were checked separately from process exit and token accounting. Native's final
12 calls preserved the original comparison, acknowledged a 14-record/three-revision
checkpoint, observed the expected missing-reference control, and restored both
Unknown heads. The mixed model used 25 calls; a separate latest-runtime fresh
stdio verifier used nine calls to restore its same 21-record/three-revision
checkpoint, reproduce all four complete originals, and reexport byte-identical
bundle bytes.
The initial cohort table includes **all attempts**, partial runs and the native task repeated
after main integration:
| Task/profile | Input tokens (cached subset) | Output tokens | Elapsed s / CPU s | Kernel-family peak B |
| ----------------------------------- | ---------------------------- | ------------- | ------------------- | -------------------- |
| Native, full discovery | 841,133 (769,536) | 4,326 | 101.845 / 18.383645 | 888,786,944 |
| JavaScript, scoped | 531,546 (452,224) | 3,421 | 88.236 / 14.456771 | 922,148,864 |
| Rectangle→NewPipe, scoped | 629,186 (567,424) | 4,384 | 108.062 / 22.589319 | 878,800,896 |
| Four-family mixed, scoped code mode | 297,271 (264,704) | 3,869 | 62.816 / 10.805497 | 914,604,032 |
Seven actual model runs plus one prelaunch failure consumed 2,299,136 input tokens
including the 2,053,888 cached subset, and 16,000 output tokens. Input already
includes cached input. Reported reasoning output was 563 tokens, recorded
separately. Canonical verification costs are separate from client/model costs.
The latest SDK catalog had 234 tools and 3,919,838 JSON bytes; those representation
bytes are not model tokens. No paired baseline model cohort ran, so these counters
do not establish causal whole-task token savings.
Three failed or partial attempts were preserved:
- Native's first attempt stopped before model/MCP launch because of a redundant
template-only harness argument. Its 0.718 s/0.473821 CPU s remains included.
- JavaScript saved its six-record/two-revision checkpoint before the isolated
client denied `binary_session`. The harness approval was corrected; a new
process restored the same checkpoint and identical trace without another
producer or reset. Joint accounting includes 16 client attempts, 15 served
calls and the client denial.
- The package model selected an unsupported generic view for a derived Apple
projection and stopped before checkpointing. Its continuation rehydrated
original authenticated saved inputs after matching current IDs and observed
summaries/projections, then used the correct projectors. Two inventory producers
ran once total; one additional NewPipe metadata reopen restored active state.
This qualifies explicit saved-source recovery. The first run's unsaved full
inventory body was not directly transported or checkpointed.
## Resource and regression boundaries
The genuine multi-domain/model workflows used `--max-old-space-size=512` for
MCP/CLI Node children; retention had no quota override. The controlled timeout
fixture's separate parent allowance is documented above. An earlier cumulative
default-policy control admitted all nine genuine
bundles: 34 records, 87,532,230 estimated representation bytes against its observed
150,244,580-byte allowance. Exact duplicates did not grow committed state. This
corpus did not reach default capacity; injected-policy regressions establish
atomic refusal, metadata replacement accounting and budget release on reset.
Local checks passed across 43 focused source/public-boundary files, 533 distinct
tests using each file's latest passing run, and 93 Python tests. The main
integration specifically reran eight lifecycle/recovery files with 61 passing
tests, including retained candidate cleanup failure/recovery and healthy-client
preservation. Typecheck, bounded lint, formatting, module/architecture guards,
documentation and production reachability passed. Controlled providers and
failure seams qualify cancellation, late work and cleanup orderings separately
from actual-provider observations.
Heavy tasks ran serially with one CPU, 4 GiB hard memory, 3 GiB early stop, zero
swap and 192 tasks maximum. Kernel CPU and memory.peak were observed at payload
exit while a cgroup observer remained alive. They include drivers/helpers and the
observer, exclude preflight outside the guard, and are distinct from process RSS
and retained-representation estimates. Failed model attempts, partial results,
assessments and raw receipts remain local; binaries and authentication data are
excluded from the repository.
Explicit acknowledged checkpoints qualify cross-process recovery. Uncheckpointed
process loss, automatic journaling, parent-directory power-loss recovery,
Windows, IDA/Hopper and a second actual deep engine were not qualified here.
+4 -1
View File
@@ -12,6 +12,7 @@ import { planTestSelection } from "./test-selection.mjs";
// Actionlint validates workflow syntax only. The semantic suite asserts these
// workflows' release, publication and self-hosted runner safeguards.
const GUARDED_WORKFLOWS = new Set([
".github/workflows/codeql.yml",
".github/workflows/pr-title.yml",
".github/workflows/pr-readiness.yml",
".github/workflows/ci-full-pr.yml",
@@ -275,10 +276,12 @@ if (tier === "reused") {
for (const owner of owners) selected[owner] = true;
if (
file.startsWith(".github/workflows/") ||
file.startsWith(".github/codeql/") ||
file === ".github/actionlint.yaml"
) {
selected.workflows = true;
if (GUARDED_WORKFLOWS.has(file)) selected.code = true;
if (GUARDED_WORKFLOWS.has(file) || file.startsWith(".github/codeql/"))
selected.code = true;
if (file !== ".github/workflows/ci.yml") continue;
}
// The Notes example is also the source-owned agent evaluation fixture.
+54
View File
@@ -28,6 +28,7 @@ const setAdd = bindBuiltinMethod(Set.prototype.add);
const hasOwnProperty = bindBuiltinMethod(Object.prototype.hasOwnProperty);
const { isProxy } = require("node:util").types;
const { createHash } = require("node:crypto");
const { EventEmitter: NodeEventEmitter } = require("node:events");
const nodeEventEmitterMethods = Object.create(null);
for (const method of [
@@ -1241,6 +1242,59 @@ const boundaries = createElectronActiveBoundaryPatches({
shape,
});
// Admit only this provider's exact launch tail and Playwright's owned prefix.
// Caller arguments remain opaque, including repeated hook paths and debug flags.
const restoreApplicationArgv = () => {
const handoff = process.env.REA_ELECTRON_CAPTURE_ARGV;
if (handoff === undefined) return;
delete process.env.REA_ELECTRON_CAPTURE_ARGV;
const plan = JSON.parse(handoff);
if (
plan === null ||
typeof plan !== "object" ||
!Number.isSafeInteger(plan.length) ||
plan.length < 3 ||
typeof plan.sha256 !== "string" ||
!/^[a-f0-9]{64}$/u.test(plan.sha256)
)
throw new Error("Capture argument handoff is not this hook's launch plan");
const tailIndex = process.argv.length - plan.length;
const prefixIndex = process.argv[1] === "--no-sandbox" ? 2 : 1;
if (
tailIndex !== prefixIndex + 2 ||
process.argv[prefixIndex] !== "--inspect=0" ||
process.argv[prefixIndex + 1] !== "--remote-debugging-port=0" ||
process.argv[tailIndex] !== "-r" ||
process.argv[tailIndex + 1] !== __filename ||
typeof process.argv[tailIndex + 2] !== "string" ||
process.argv[tailIndex + 2].length === 0
)
throw new Error("Observed argv does not match Playwright's capture prefix");
const tail = sliceArguments(process.argv, tailIndex);
if (
createHash("sha256").update(JSON.stringify(tail)).digest("hex") !==
plan.sha256
)
throw new Error(
"Observed argv does not match the capture argument handoff",
);
for (let index = 2; index < tail.length; index++)
process.argv[index - 1] = tail[index];
process.argv.length = tail.length - 1;
};
try {
restoreApplicationArgv();
} catch (cause) {
globalThis.__reaElectronActiveHookError = true;
recordRuntime(
"error",
cause instanceof Error ? cause.message : String(cause),
"failed",
{ target: "active-hook-arguments", error: true },
);
}
try {
const electron = require("electron");
patchEmitter(electron.app, (event, args) =>
+57
View File
@@ -0,0 +1,57 @@
interface ProviderBinding {
readonly identity: {
readonly id: string;
readonly name: string;
readonly version: string | null;
};
readonly operations: readonly string[] | { readonly group: string };
}
interface ProviderCatalogSources {
readonly toolContracts: {
readonly TOOL_CONTRACTS: readonly { readonly name: string }[];
};
readonly toolFamilies: {
readonly TOOL_CONTRACT_GROUPS: Readonly<
Record<string, readonly { readonly name: string }[]>
>;
};
readonly capabilityDeclarations: {
readonly CAPABILITY_DECLARATIONS: Readonly<
Record<string, { readonly providers: readonly ProviderBinding[] }>
>;
};
}
export function providerCatalog(sources: ProviderCatalogSources): {
readonly id: string;
readonly name: string;
readonly version: string | null;
readonly capabilities: readonly string[];
}[];
export function toolFamilyCatalog(sources: {
readonly toolFamilies: {
readonly TOOL_FAMILY_DEFINITIONS: readonly {
readonly id: string;
readonly surface: string;
readonly contracts: readonly { readonly name: string }[];
}[];
};
readonly toolContracts: {
readonly TOOL_CONTRACTS: readonly unknown[];
};
readonly catalogIdentity: {
readonly FULL_CATALOG_IDENTITY: {
readonly counts: { readonly mcp_tools: number };
};
};
}): {
readonly total: number;
readonly families: readonly {
readonly id: string;
readonly surface: string;
readonly count: number;
readonly tools: readonly string[];
}[];
};
+44 -403
View File
@@ -23,414 +23,55 @@ export const toolFamilyCatalog = (sources) => {
return { total: familyTotal, families };
};
/** Build provider identities with their sorted capability names. */
/** Derive provider capabilities from the same static declarations used by composition. */
export const providerCatalog = (sources) => {
const contractsByName = new Map(
sources.toolContracts.TOOL_CONTRACTS.map((contract) => [
contract.name,
contract,
]),
const contractNames = new Set(
sources.toolContracts.TOOL_CONTRACTS.map(({ name }) => name),
);
const declaredContracts = (providerId, operations) =>
operations.map((operation) => {
const contract = contractsByName.get(operation);
if (contract === undefined)
const providers = new Map();
for (const declaration of Object.values(
sources.capabilityDeclarations.CAPABILITY_DECLARATIONS,
)) {
for (const { identity, operations } of declaration.providers) {
let provider = providers.get(identity.id);
if (provider === undefined) {
provider = { ...identity, capabilities: new Set() };
providers.set(identity.id, provider);
} else if (
provider.name !== identity.name ||
provider.version !== identity.version
) {
throw new TypeError(
`Provider ${providerId} declares an unknown analyst operation: ${operation}`,
`Provider ${identity.id} has conflicting source identities`,
);
return contract;
});
const {
applicationContracts,
reconciliationContracts,
observationContracts,
activeContracts,
fuseContracts,
} = electronContractSlices(sources);
return [
{
identity: sources.cutterProvider.CUTTER_PROVIDER_IDENTITY,
contracts: sources.cutterContracts.CUTTER_TOOL_CONTRACTS,
},
{
identity: sources.objdumpProvider.OBJDUMP_PROVIDER_IDENTITY,
contracts:
sources.reverseEngineeringContracts.REVERSE_ENGINEERING_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_with_objdump",
),
},
{
identity: sources.rizinProvider.RIZIN_PROVIDER_IDENTITY,
contracts:
sources.reverseEngineeringContracts.REVERSE_ENGINEERING_TOOL_CONTRACTS.filter(
({ name }) => name === "execute_rizin_command",
),
},
{
identity: sources.gdbProvider.GDB_PROVIDER_IDENTITY,
contracts: sources.gdbContracts.GDB_TOOL_CONTRACTS,
},
{
identity: sources.rizinDebugProvider.RIZIN_DEBUGGER_PROVIDER_IDENTITY,
contracts: sources.rizinDebugContracts.RIZIN_DEBUG_TOOL_CONTRACTS,
},
{
identity: sources.fridaProvider.FRIDA_PROVIDER_IDENTITY,
contracts: sources.fridaContracts.FRIDA_TOOL_CONTRACTS,
},
{
identity: sources.hopperProvider.HOPPER_PROVIDER_IDENTITY,
contracts: declaredContracts(
"hopper",
sources.hopperProvider.HOPPER_OPERATIONS,
}
let selectedOperations = operations;
if (!Array.isArray(operations)) {
const contracts =
sources.toolFamilies.TOOL_CONTRACT_GROUPS[operations.group];
if (contracts === undefined)
throw new TypeError(
`Provider ${identity.id} declares an unknown contract group: ${operations.group}`,
);
selectedOperations = contracts.map(({ name }) => name);
}
for (const operation of selectedOperations) {
if (!contractNames.has(operation))
throw new TypeError(
`Provider ${identity.id} declares an unknown analyst operation: ${operation}`,
);
provider.capabilities.add(operation);
}
}
}
return [...providers.values()]
.map(({ id, name, version, capabilities }) => ({
id,
name,
version,
capabilities: [...capabilities].sort((left, right) =>
left.localeCompare(right),
),
},
{
identity: sources.ghidraProvider.GHIDRA_PROVIDER_IDENTITY,
contracts: declaredContracts(
"ghidra",
sources.ghidraProvider.GHIDRA_OPERATIONS,
),
},
{
identity: sources.idaProvider.IDA_PROVIDER_IDENTITY,
contracts: declaredContracts("ida", sources.idaProvider.IDA_OPERATIONS),
},
{
identity: sources.kunaProvider.KUNA_PROVIDER_IDENTITY,
contracts: declaredContracts(
"kuna",
sources.kunaProvider.KUNA_OPERATIONS,
),
},
{
identity: sources.nativeProvider.NATIVE_MACOS_PROVIDER_IDENTITY,
contracts: sources.nativeContracts.NATIVE_TOOL_CONTRACTS,
},
{
identity: sources.pwntoolsProvider.PWNTOOLS_PROVIDER_IDENTITY,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) =>
name === "inspect_binary_layout" ||
name === "inspect_recorded_crash",
),
},
{
identity: sources.artifactProviders.PE_RESOURCES_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_pe_resources",
),
},
{
identity: sources.artifactProviders.PE_CERTIFICATES_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_pe_certificates",
),
},
{
identity: sources.artifactProviders.PE_IMPORTS_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_pe_imports",
),
},
{
identity: sources.artifactProviders.DEVICETREE_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_devicetree",
),
},
{
identity: sources.artifactProviders.PE_LOAD_CONFIG_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_pe_load_config",
),
},
{
identity: sources.artifactProviders.ELF_NOTES_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_elf_notes",
),
},
{
identity: sources.artifactProviders.APPLE_IPS_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_apple_ips",
),
},
{
identity: sources.artifactProviders.KERNEL_MODULE_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_kernel_module",
),
},
{
identity: sources.artifactProviders.BYTE_ENTROPY_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_byte_entropy",
),
},
{
identity: sources.artifactProviders.NATIVE_EXPORTS_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_native_exports",
),
},
{
identity: sources.captureProvider.TSHARK_PROVIDER_IDENTITY,
contracts: sources.captureContracts.CAPTURE_TOOL_CONTRACTS,
},
{
identity: sources.signalProvider.SIGROK_PROVIDER_IDENTITY,
contracts: sources.signalContracts.SIGNAL_TOOL_CONTRACTS,
},
{
identity: sources.wasmProvider.WABT_PROVIDER_IDENTITY,
contracts: sources.wasmContracts.WASM_TOOL_CONTRACTS.filter(
({ kind }) => kind === "artifact-provider",
),
},
{
identity: sources.apkSignatureProvider.APK_SIGNATURE_PROVIDER_IDENTITY,
contracts: sources.apkSignatureContracts.APK_SIGNATURE_TOOL_CONTRACTS,
},
{
identity: sources.evmProvider.EVMOLE_PROVIDER_IDENTITY,
contracts: sources.evmContracts.EVM_TOOL_CONTRACTS,
},
{
identity: sources.goProvider.GO_BINARY_PROVIDER_IDENTITY,
contracts: sources.goContracts.GO_TOOL_CONTRACTS,
},
{
identity: sources.gpuProvider.GPU_CODE_OBJECTS_PROVIDER_IDENTITY,
contracts: sources.gpuContracts.GPU_TOOL_CONTRACTS,
},
{
identity: sources.il2cppProvider.IL2CPP_METADATA_PROVIDER_IDENTITY,
contracts: sources.il2cppContracts.IL2CPP_TOOL_CONTRACTS,
},
{
identity: sources.modelProvider.MODEL_ARTIFACT_PROVIDER_IDENTITY,
contracts: sources.modelContracts.MODEL_TOOL_CONTRACTS,
},
{
identity: sources.rustProvider.RUST_BINARY_PROVIDER_IDENTITY,
contracts: sources.rustContracts.RUST_TOOL_CONTRACTS,
},
{
identity: sources.pythonPycProvider.PYTHON_PYC_PROVIDER_IDENTITY,
contracts: sources.pythonPycContracts.PYTHON_PYC_TOOL_CONTRACTS,
},
{
identity: sources.erlangProvider.ERLANG_BEAM_PROVIDER_IDENTITY,
contracts: sources.erlangContracts.ERLANG_BEAM_TOOL_CONTRACTS,
},
{
identity: sources.electronFuseProvider.ELECTRON_FUSE_PROVIDER_IDENTITY,
contracts: fuseContracts,
},
{
identity: sources.sqliteDomain.SQLITE_PROVIDER_IDENTITY,
contracts: sources.sqliteContracts.SQLITE_TOOL_CONTRACTS,
},
{
identity: sources.artifactProviders.ARTIFACT_GRAPH_PROVIDER,
contracts: [
...sources.artifactContracts.ARTIFACT_TOOL_CONTRACTS,
...sources.desktopContracts.DESKTOP_APPLICATION_TOOL_CONTRACTS,
],
},
{
identity: sources.artifactProviders.MANAGED_STATIC_PROVIDER,
contracts: sources.managedContracts.MANAGED_TOOL_CONTRACTS,
},
{
identity: sources.artifactProviders.MANAGED_WORKFLOW_PROVIDER,
contracts:
sources.managedWorkflowContracts.MANAGED_WORKFLOW_TOOL_CONTRACTS,
},
{
identity: sources.firmwareProvider.BINWALK_PROVIDER_IDENTITY,
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_firmware_regions",
),
},
{
identity: sources.firmwareProvider.UNBLOB_PROVIDER_IDENTITY,
contracts: sources.firmwareContracts.FIRMWARE_TOOL_CONTRACTS.filter(
({ name }) => name === "extract_firmware",
),
},
{
identity: sources.androidProvider.JADX_PROVIDER_IDENTITY,
contracts: sources.androidContracts.ANDROID_TOOL_CONTRACTS,
},
{
identity: sources.pcapProvider.PCAP_PROVIDER_IDENTITY,
contracts: sources.pcapContracts.PCAP_TOOL_CONTRACTS,
},
{
identity: sources.flutterProvider.FLUTTER_PROVIDER_IDENTITY,
contracts: sources.flutterContracts.FLUTTER_TOOL_CONTRACTS,
},
{
identity: sources.apktoolProvider.APKTOOL_PROVIDER_IDENTITY,
contracts: sources.apktoolContracts.APKTOOL_TOOL_CONTRACTS,
},
{
identity: sources.jebProvider.JEB_PROVIDER_IDENTITY,
contracts: sources.jebContracts.JEB_TOOL_CONTRACTS,
},
{
identity: sources.adbProvider.ADB_PROVIDER_IDENTITY,
contracts: sources.adbContracts.ADB_TOOL_CONTRACTS,
},
{
identity: sources.webNetworkCaptureProvider.HAR_CAPTURE_PROVIDER_IDENTITY,
contracts:
sources.webNetworkCaptureContracts.WEB_NETWORK_CAPTURE_TOOL_CONTRACTS,
},
{
identity: sources.webNetworkCaptureProvider.SAZ_CAPTURE_PROVIDER_IDENTITY,
contracts:
sources.webNetworkCaptureContracts.WEB_NETWORK_CAPTURE_TOOL_CONTRACTS,
},
{
identity:
sources.webNetworkCaptureProvider.MITMPROXY_CAPTURE_PROVIDER_IDENTITY,
contracts:
sources.webNetworkCaptureContracts.WEB_NETWORK_CAPTURE_TOOL_CONTRACTS,
},
{
identity: sources.browserIdentities.CDP_BROWSER_PROVIDER_IDENTITY,
contracts: [
...sources.browserContracts.BROWSER_TOOL_CONTRACTS,
...sources.webRuntimeContracts.WEB_RUNTIME_TOOL_CONTRACTS,
],
},
{
identity:
sources.browserIdentities.PLAYWRIGHT_BROWSER_SCENARIO_PROVIDER_IDENTITY,
contracts:
sources.browserScenarioContracts.BROWSER_SCENARIO_TOOL_CONTRACTS,
},
{
identity: sources.browserIdentities.CDP_ELECTRON_PROVIDER_IDENTITY,
contracts: observationContracts,
},
{
identity:
sources.browserIdentities.PLAYWRIGHT_ELECTRON_ACTIVE_PROVIDER_IDENTITY,
contracts: activeContracts,
},
{
identity: sources.v8InspectorProvider.V8_INSPECTOR_PROVIDER_IDENTITY,
contracts:
sources.javascriptRuntimeObservationContracts
.JAVASCRIPT_RUNTIME_OBSERVATION_TOOL_CONTRACTS,
},
{
identity: sources.artifactProviders.JAVASCRIPT_APPLICATION_PROVIDER,
contracts: applicationContracts,
},
{
identity:
sources.artifactProviders.JAVASCRIPT_RUNTIME_RECONCILIATION_PROVIDER,
contracts: reconciliationContracts,
},
{
identity:
sources.artifactProviders.JAVASCRIPT_APPLICATION_WORKFLOW_PROVIDER,
contracts: sources.applicationContracts.APPLICATION_TOOL_CONTRACTS.filter(
({ name }) =>
name !== "project_android_application_graph" &&
name !== "project_apple_application_graph",
),
},
{
identity: sources.artifactProviders.ANALYSIS_VIEW_PROVIDER,
contracts: sources.analysisViewContracts.ANALYSIS_VIEW_TOOL_CONTRACTS,
},
{
identity: sources.artifactProviders.ANDROID_APPLICATION_PROVIDER,
contracts: sources.applicationContracts.APPLICATION_TOOL_CONTRACTS.filter(
({ name }) => name === "project_android_application_graph",
),
},
{
identity: sources.artifactProviders.APPLE_APPLICATION_PROVIDER,
contracts: sources.applicationContracts.APPLICATION_TOOL_CONTRACTS.filter(
({ name }) => name === "project_apple_application_graph",
),
},
{
identity: sources.artifactProviders.WEB_SCRIPT_EXPORT_PROVIDER,
contracts: sources.webScriptContracts.WEB_SCRIPT_TOOL_CONTRACTS,
},
{
identity: sources.javascriptRecoveryProvider.WAKARU_PROVIDER_IDENTITY,
contracts:
sources.javascriptRecoveryContracts.JAVASCRIPT_RECOVERY_TOOL_CONTRACTS,
},
]
.map(({ identity, contracts }) => ({
id: identity.id,
name: identity.name,
version: identity.version,
capabilities: names(contracts),
}))
.sort((left, right) => left.id.localeCompare(right.id));
};
const electronContractSlices = (sources) => {
const applicationContracts =
sources.electronContracts.ELECTRON_TOOL_CONTRACTS.filter(
({ name }) => name === "analyze_javascript_application",
);
const reconciliationContracts =
sources.electronContracts.ELECTRON_TOOL_CONTRACTS.filter(
({ name }) => name === "reconcile_javascript_runtime",
);
const observationContracts =
sources.electronContracts.ELECTRON_TOOL_CONTRACTS.filter(
({ name }) =>
name !== "analyze_javascript_application" &&
name !== "reconcile_javascript_runtime" &&
name !== "capture_electron_scenario" &&
name !== "inspect_electron_fuses",
);
const activeContracts =
sources.electronContracts.ELECTRON_TOOL_CONTRACTS.filter(
({ name }) => name === "capture_electron_scenario",
);
const fuseContracts =
sources.electronContracts.ELECTRON_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_electron_fuses",
);
if (
applicationContracts.length !== 1 ||
reconciliationContracts.length !== 1 ||
observationContracts.length !== 2 ||
activeContracts.length !== 1 ||
fuseContracts.length !== 1
)
throw new Error("Electron provider capability ownership drifted");
return {
applicationContracts,
reconciliationContracts,
observationContracts,
activeContracts,
fuseContracts,
};
};
+1 -124
View File
@@ -39,131 +39,8 @@ const SOURCE_PATHS = {
cli: "dist/cli.js",
toolContracts: "dist/contracts/toolContracts.js",
toolFamilies: "dist/contracts/toolFamilies.js",
binaryDiagnosticsContracts:
"dist/contracts/native/binaryDiagnosticsToolContracts.js",
cutterContracts: "dist/contracts/cutterToolContracts.js",
cutterProvider: "dist/cutter/CutterBridgeClient.js",
reverseEngineeringContracts:
"dist/contracts/reverseEngineeringToolContracts.js",
gdbContracts: "dist/contracts/gdbToolContracts.js",
gdbProvider: "dist/gdb/GdbSessionManager.js",
rizinDebugContracts: "dist/contracts/rizinDebugToolContracts.js",
objdumpProvider: "dist/objdump/ObjdumpCommand.js",
rizinProvider: "dist/rizin/RizinCommand.js",
rizinDebugProvider: "dist/rizin/RizinDebugSessionManager.js",
fridaContracts: "dist/contracts/fridaToolContracts.js",
fridaProvider: "dist/frida/FridaInstrumentationManager.js",
pwntoolsProvider: "dist/native/pwntools/PwntoolsRelease.js",
nativeContracts: "dist/contracts/native/nativeToolContracts.js",
captureContracts: "dist/contracts/captures/captureToolContracts.js",
captureProvider: "dist/captures/TsharkCaptureProvider.js",
signalContracts: "dist/contracts/signals/signalToolContracts.js",
signalProvider: "dist/signals/SigrokRecordingProvider.js",
wasmContracts: "dist/contracts/wasm/wasmToolContracts.js",
wasmProvider: "dist/wasm/WabtRelease.js",
apkSignatureContracts:
"dist/contracts/apkSignature/apkSignatureToolContracts.js",
apkSignatureProvider: "dist/apkSignature/ApkSignatureRelease.js",
evmContracts: "dist/contracts/evm/evmToolContracts.js",
sqliteContracts: "dist/contracts/sqlite/sqliteToolContracts.js",
sqliteDomain: "dist/domain/sqlite/sqliteDatabase.js",
evmProvider: "dist/evm/EvmoleRelease.js",
goContracts: "dist/contracts/go/goToolContracts.js",
goProvider: "dist/go/GoBinaryProvider.js",
gpuContracts: "dist/contracts/gpu/gpuToolContracts.js",
gpuProvider: "dist/gpu/GpuCodeObjectsProvider.js",
modelContracts: "dist/contracts/model/modelToolContracts.js",
modelProvider: "dist/model/ModelArtifactProvider.js",
il2cppContracts: "dist/contracts/il2cpp/il2cppToolContracts.js",
il2cppProvider: "dist/il2cpp/Il2cppMetadataProvider.js",
rustContracts: "dist/contracts/rust/rustToolContracts.js",
rustProvider: "dist/rust/RustBinaryProvider.js",
pythonPycContracts: "dist/contracts/python/pythonPycToolContracts.js",
pythonPycProvider: "dist/python/PythonPycProvider.js",
erlangContracts: "dist/contracts/erlang/beamToolContracts.js",
erlangProvider: "dist/erlang/BeamProvider.js",
artifactContracts: "dist/contracts/artifactToolContracts.js",
desktopContracts: "dist/contracts/desktopApplicationToolContracts.js",
managedContracts: "dist/contracts/managed/managedToolContracts.js",
firmwareContracts: "dist/contracts/firmware/firmwareToolContracts.js",
firmwareProvider: "dist/firmware/FirmwareRelease.js",
androidContracts: "dist/contracts/android/androidToolContracts.js",
flutterContracts: "dist/contracts/flutter/flutterToolContracts.js",
flutterProvider: "dist/flutter/FlutterBuildProvider.js",
androidProvider: "dist/android/JadxRelease.js",
apktoolContracts: "dist/contracts/apktool/apktoolToolContracts.js",
pcapContracts: "dist/contracts/pcap/pcapToolContracts.js",
pcapProvider: "dist/pcap/PcapProvider.js",
apktoolProvider: "dist/apktool/ApktoolProvider.js",
jebContracts: "dist/contracts/jeb/jebToolContracts.js",
jebProvider: "dist/jeb/JebProvider.js",
adbContracts: "dist/contracts/adb/adbToolContracts.js",
adbProvider: "dist/adb/AdbProvider.js",
managedWorkflowContracts:
"dist/contracts/managed/managedWorkflowToolContracts.js",
browserContracts: "dist/contracts/browserToolContracts.js",
webRuntimeContracts: "dist/contracts/webRuntimeToolContracts.js",
webNetworkCaptureContracts:
"dist/contracts/webNetworkCaptureToolContracts.js",
webNetworkCaptureProvider: "dist/browser/history/CaptureRelease.js",
browserScenarioContracts: "dist/contracts/browserScenarioToolContracts.js",
electronContracts: "dist/contracts/javascript/electronToolContracts.js",
electronFuseProvider: "dist/electron/ElectronFuseProvider.js",
javascriptRuntimeObservationContracts:
"dist/contracts/javascript/javascriptRuntimeObservationToolContracts.js",
applicationContracts: "dist/contracts/applicationToolContracts.js",
analysisViewContracts: "dist/contracts/analysisViewToolContracts.js",
webScriptContracts: "dist/contracts/webScriptToolContracts.js",
javascriptRecoveryContracts:
"dist/contracts/javascript/javascriptRecoveryToolContracts.js",
javascriptRecoveryProvider: "dist/javascript/recovery/WakaruRelease.js",
capabilityDeclarations: "dist/composition/capabilityDeclarations.js",
supportedClients: "dist/application/SupportedClients.js",
hopperProvider: "dist/hopper/HopperProviderCapabilities.js",
ghidraProvider: "dist/ghidra/GhidraProviderCapabilities.js",
idaProvider: "dist/ida/IdaProviderCapabilities.js",
kunaProvider: "dist/kuna/KunaProviderCapabilities.js",
nativeProvider: "dist/native/NativeMacOSProviderMetadata.js",
artifactProviders: "dist/domain/investigationProviders.js",
browserIdentities: "dist/browser/providerIdentities.js",
v8InspectorProvider: "dist/inspector/providerIdentity.js",
evidence: "dist/domain/evidence.js",
evidenceBundle: "dist/domain/evidenceBundle.js",
processCapture: "dist/domain/process/processCapture.js",
analysisSnapshot: "dist/domain/analysisSnapshot.js",
artifactGraph: "dist/domain/artifactGraph.js",
browserObservation: "dist/domain/browserObservation.js",
browserScenario: "dist/domain/browserScenario.js",
browserScenarioCapture: "dist/domain/browserScenarioCapture.js",
browserScenarioDiff: "dist/domain/browserScenarioDiff.js",
browserSession: "dist/domain/browserSession.js",
electronObservation: "dist/domain/javascript/electronObservation.js",
electronActiveObservation:
"dist/domain/javascript/electronActiveObservation.js",
javascriptRuntimeObservation:
"dist/domain/javascript/javascriptRuntimeObservation.js",
webBundleAnalysis: "dist/domain/webBundleAnalysis.js",
webCaptureDiff: "dist/domain/webCaptureDiff.js",
managedArtifact: "dist/domain/managed/managedArtifact.js",
managedComparison: "dist/domain/managed/managedMemberComparison.js",
managedNativeVerification: "dist/domain/managed/managedNativeVerification.js",
webMcpDiscovery: "dist/domain/webMcpDiscovery.js",
webScreenshot: "dist/domain/webScreenshot.js",
javascriptApplicationGraph:
"dist/domain/javascript/javascriptApplicationGraph.js",
javascriptApplicationAnalysis:
"dist/domain/javascript/javascriptApplicationAnalysis.js",
javascriptSemanticGraph: "dist/domain/javascript/javascriptSemanticGraph.js",
javascriptSemanticQuery:
"dist/domain/javascript/javascriptSemanticQuerySchemas.js",
javascriptSemanticTrace:
"dist/domain/javascript/javascriptSemanticTraceSchemas.js",
javascriptFeatureTrace:
"dist/domain/javascript/javascriptFeatureTraceSchemas.js",
javascriptVersionComparison:
"dist/domain/javascript/javascriptApplicationVersionComparisonSchemas.js",
reconstructionVerification:
"dist/domain/reconstructionVerificationSchemas.js",
residualUnknown: "dist/domain/residualUnknown.js",
};
/** Load every compiled source module referenced by the catalog. */
+2
View File
@@ -2,6 +2,8 @@
export interface ImportSpecifierFact {
readonly specifier: string;
readonly line: number;
/** True only when TypeScript erases this module edge under verbatimModuleSyntax. */
readonly typeOnly: boolean;
}
/** Static string specifier on one import or re-export node, when it has one. */
+21 -1
View File
@@ -26,6 +26,22 @@ export const importSpecifier = (node) => {
}
};
/** Whether TypeScript erases this module edge under verbatimModuleSyntax. */
const isTypeOnly = (node) => {
switch (node.type) {
case "TSImportType":
return true;
case "ImportDeclaration":
case "TSImportEqualsDeclaration":
return node.importKind === "type";
case "ExportNamedDeclaration":
case "ExportAllDeclaration":
return node.exportKind === "type";
default:
return false;
}
};
/**
* Static import and re-export specifiers in source order.
* Comments and string literals are not imports. Non-string dynamic imports
@@ -37,7 +53,11 @@ export const readImportSpecifiers = (source) => {
traverseFast(tree, (node) => {
const specifier = importSpecifier(node);
if (specifier === undefined) return;
specifiers.push({ specifier, line: node.loc?.start.line ?? 1 });
specifiers.push({
specifier,
line: node.loc?.start.line ?? 1,
typeOnly: isTypeOnly(node),
});
});
return specifiers;
};
+40 -1
View File
@@ -21,6 +21,8 @@ export const verifyKunaReferences = async ({
address,
expectedCaller,
nativeIncoming,
verifyNamedTrace = false,
minimumTraceReferences = 1,
}) => {
const before = await Promise.all([
entries(join(cacheRoot, "projects")),
@@ -80,7 +82,44 @@ export const verifyKunaReferences = async ({
direction: "forward",
});
assert.equal(path.value.goal_status, "reached");
composed = { graph_ms: graph.ms, path_ms: path.ms };
let traceMs;
if (verifyNamedTrace) {
const trace = await call("trace_feature", { query: name });
traceMs = trace.ms;
assert.ok(
trace.value.matches.some(
(match) => match.type === "procedure" && match.address === address,
),
`${name} was not retained in its feature trace`,
);
assert.ok(
incoming.value.length >= minimumTraceReferences,
`${name} fixture did not produce the expected incoming references`,
);
const references = trace.value.references.filter(
(row) => row.target_address === address,
);
assert.deepEqual(
references.map((row) => row.source_address).sort(),
[...incoming.value].sort(),
`${name} lost or duplicated an observed incoming reference`,
);
for (const reference of references) {
assert.equal(reference.containing_procedure.status, "unavailable");
assert.equal(reference.containing_procedure.provider_id, "kuna");
assert.ok(
trace.value.residual_unknowns.some((unknown) =>
unknown.includes(reference.source_address),
),
);
}
assert.equal(trace.value.truncated, false);
}
composed = {
graph_ms: graph.ms,
path_ms: path.ms,
...(traceMs === undefined ? {} : { feature_trace_ms: traceMs }),
};
}
assert.deepEqual(
await Promise.all([
+11 -3
View File
@@ -24,6 +24,8 @@ const MIGRATED_PROVIDER_ROOTS = new Set([
"inspector",
"javascript",
]);
const CENTRALIZED_CONSTRUCTOR_OWNER =
/^src\/(?:frida\/FridaInstrumentationManager|cutter\/CutterBridge(?:Client|Service)|gdb\/GdbSessionManager|rizin\/RizinDebugSessionManager)\.(?:js|ts)$/u;
const repositoryPath = (root, path) =>
relative(root, path).replaceAll("\\", "/");
@@ -35,7 +37,7 @@ const sourceTarget = (file, specifier) => {
: undefined;
};
const failedBoundary = (file, target) => {
const failedBoundary = (file, target, typeOnly) => {
if (!file.startsWith("src/") || !target.startsWith("src/")) return undefined;
const layer = file.split("/")[1];
const targetLayer = target.split("/")[1];
@@ -75,6 +77,12 @@ const failedBoundary = (file, target) => {
}
if (layer === "application" && targetLayer === "composition")
return "application-composition";
if (
(layer === "application" || layer === "server") &&
!typeOnly &&
CENTRALIZED_CONSTRUCTOR_OWNER.test(target)
)
return "provider-construction";
if (
(layer === "application" || layer === "server") &&
(MIGRATED_PROVIDER_ROOTS.has(targetLayer) ||
@@ -92,11 +100,11 @@ export const inspectModuleBoundaries = (file, source, root) => {
const absoluteFile = resolve(root, file);
const importer = repositoryPath(root, absoluteFile);
const violations = [];
for (const { specifier, line } of readImportSpecifiers(source)) {
for (const { specifier, line, typeOnly } of readImportSpecifiers(source)) {
const resolved = sourceTarget(absoluteFile, specifier);
if (resolved === undefined) continue;
const target = repositoryPath(root, resolved);
const boundary = failedBoundary(importer, target);
const boundary = failedBoundary(importer, target, typeOnly);
if (boundary !== undefined)
violations.push({ file: importer, line, specifier, target, boundary });
}
+1 -1
View File
@@ -1,5 +1,5 @@
import assert from "node:assert/strict";
import { HOPPER_PROVIDER_IDENTITY } from "../../dist/hopper/HopperProviderCapabilities.js";
import { HOPPER_PROVIDER_IDENTITY } from "../../dist/hopper/HopperProviderMetadata.js";
import {
requireMcpEvidenceResult,
requireEvidenceProvider,
+1 -1
View File
@@ -1,4 +1,4 @@
import { HOPPER_PROVIDER_IDENTITY } from "../../dist/hopper/HopperProviderCapabilities.js";
import { HOPPER_PROVIDER_IDENTITY } from "../../dist/hopper/HopperProviderMetadata.js";
/** Verify the selected Hopper candidate and concrete provider binding. */
export const requireHopperSelection = (status, expected) => {
+1 -1
View File
@@ -19,7 +19,7 @@ export const SOURCE_TEST_PROJECTS = [
group: "source",
include: [
"src/*.test.ts",
"src/{android,apkSignature,apktool,jeb,adb,artifacts,browser,config,cutter,dotnet,electron,erlang,evm,firmware,flutter,pcap,ghidra,go,gpu,hopper,ida,inspector,javascript,main,native,process,providers,python,reference,rust,server,sqlite,wasm,windows}/**/*.test.ts",
"src/{android,apkSignature,apktool,jeb,adb,artifacts,browser,config,cutter,dotnet,electron,erlang,evm,firmware,flutter,frida,pcap,ghidra,go,gpu,hopper,ida,inspector,javascript,main,native,process,providers,python,reference,rust,server,sqlite,wasm,windows}/**/*.test.ts",
],
},
{
+14
View File
@@ -0,0 +1,14 @@
import type { PackageVerifierCommands } from "./lib/verify-package-commands.mjs";
export function verifyPackageUpdate(options: {
readonly prefix: string;
readonly packageRoot: string;
readonly tarball: string;
readonly workspace: string;
readonly environment: NodeJS.ProcessEnv;
readonly commands: Pick<PackageVerifierCommands, "exec">;
}): Promise<{
readonly installed_version: string;
readonly exact_release: true;
readonly maintenance_read_only: true;
}>;
+13 -3
View File
@@ -35,13 +35,20 @@ export async function verifyPackageUpdate({
response.destroy();
return;
}
response.writeHead(500);
response.writeHead(500, {
"Content-Type": "text/plain; charset=utf-8",
"X-Content-Type-Options": "nosniff",
});
response.end(String(error));
});
});
let registry;
async function serveRegistry(request, response) {
const pathname = new URL(request.url, registry).pathname;
// A leading // belongs to an origin-form HTTP path, not a URL authority.
const requested = new URL(
request.url.startsWith("/") ? `${registry}${request.url}` : request.url,
);
const { pathname } = requested;
if (pathname === "/rea-agents") {
response.setHeader("Content-Type", "application/json");
response.end(
@@ -65,7 +72,10 @@ export async function verifyPackageUpdate({
response.setHeader("Content-Type", "application/json");
response.end("{}");
} else {
const upstream = await fetch(`https://registry.npmjs.org${request.url}`);
const upstreamUrl = new URL("https://registry.npmjs.org");
upstreamUrl.pathname = pathname;
upstreamUrl.search = requested.search;
const upstream = await fetch(upstreamUrl);
const body = Buffer.from(await upstream.arrayBuffer());
response.writeHead(upstream.status, {
"Content-Type":
+1 -1
View File
@@ -510,7 +510,7 @@ try {
);
const { GHIDRA_OPERATIONS } = await import(
pathToFileURL(
join(packageRoot, "dist/ghidra/GhidraProviderCapabilities.js"),
join(packageRoot, "dist/ghidra/GhidraProviderMetadata.js"),
)
);
const expected = GHIDRA_OPERATIONS.filter(
+18 -2
View File
@@ -378,17 +378,33 @@ try {
arguments: { procedure: "0xffffffffffffffff" },
});
assert.equal(invalid.isError, true, "Unmapped address did not fail");
// The CLI reads a fresh session, so compare it with MCP before the boundary
// helper's temporary rename and restoration.
const entryWindow = await call("read_function_instructions", {
procedure: identities.entry.address,
});
const cliWindow = await cliEvidence("instructions", identities.entry.address);
assert.deepEqual(cliWindow.normalized_result, entryWindow);
await verifyGhidraAddressBoundaries(
client,
identities.entry.address,
cliEvidence,
);
const cliWindow = await cliEvidence("instructions", identities.entry.address);
const entrySymbol = entryWindow.procedure.symbol;
assert.ok(entrySymbol != null, "The DOS entry needs observed symbol facts");
// Restoring the name is itself a user edit, so Ghidra records user provenance.
assert.deepEqual(
cliWindow.normalized_result,
await call("read_function_instructions", {
procedure: identities.entry.address,
}),
{
...entryWindow,
procedure: {
...entryWindow.procedure,
symbol: { ...entrySymbol, source: "user_defined" },
},
},
"Restoring the entry name changed more than its symbol provenance",
);
assert.equal(
digest(await readFile(targetPath)),
+3 -3
View File
@@ -8,20 +8,20 @@ import { fileURLToPath } from "node:url";
import { parseConfig } from "../dist/config/parseConfig.js";
import { resolveGhidraAnalysisProfile } from "../dist/ghidra/GhidraAnalysisProfile.js";
import { GhidraClient } from "../dist/ghidra/GhidraClient.js";
import { GHIDRA_FUNCTION_OPERATIONS } from "../dist/ghidra/GhidraFunctionOperations.js";
import {
GHIDRA_FUNCTION_OPERATIONS,
parseGhidraFunctionInput,
parseGhidraFunctionResult,
} from "../dist/ghidra/GhidraFunctionValues.js";
import { GHIDRA_INVENTORY_OPERATIONS } from "../dist/ghidra/GhidraInventoryOperations.js";
import {
GHIDRA_INVENTORY_OPERATIONS,
parseGhidraInventoryInput,
parseGhidraInventoryResult,
} from "../dist/ghidra/GhidraInventoryValues.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js";
import { GhidraHeadlessLauncher } from "../dist/ghidra/GhidraLauncher.js";
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProviderCapabilities.js";
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProviderMetadata.js";
import {
WINDOWS_P0_NATIVE_PE_ARCHITECTURES,
windowsP0Capabilities,
+1 -1
View File
@@ -13,7 +13,7 @@ import { GhidraClient } from "../dist/ghidra/GhidraClient.js";
import { inspectVerifierGhidraInstallation } from "./lib/ghidra-installation-inspection.mjs";
import { SUPPORTED_GHIDRA_VERSION } from "../dist/ghidra/GhidraReleasePolicy.js";
import { GhidraHeadlessLauncher } from "../dist/ghidra/GhidraLauncher.js";
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProviderCapabilities.js";
import { GHIDRA_PROVIDER_IDENTITY } from "../dist/ghidra/GhidraProviderMetadata.js";
import {
assertCleanup,
+1 -1
View File
@@ -11,7 +11,7 @@ import { AnalysisCleanupLease } from "../dist/artifacts/AnalysisCleanupLease.js"
import { resolveHopperAnalysisProfile } from "../dist/hopper/HopperAnalysisProfile.js";
import { HopperApplicationLauncher } from "../dist/hopper/BridgeLauncher.js";
import { HopperClient } from "../dist/hopper/HopperClient.js";
import { HOPPER_PROVIDER_IDENTITY } from "../dist/hopper/HopperProviderCapabilities.js";
import { HOPPER_PROVIDER_IDENTITY } from "../dist/hopper/HopperProviderMetadata.js";
import { loadRealHopperFixtureTargets } from "./lib/real-hopper-fixture.mjs";
// These options belong to the native client, below the CLI/MCP tool surface.
+1 -1
View File
@@ -11,7 +11,7 @@ import {
requireFunctionDossier,
requirePseudocode,
} from "./lib/real-hopper-assertions.mjs";
import { HOPPER_PROVIDER_IDENTITY } from "../dist/hopper/HopperProviderCapabilities.js";
import { HOPPER_PROVIDER_IDENTITY } from "../dist/hopper/HopperProviderMetadata.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/process/HopperLease.js";
import { REA_WORKFLOW_PROVIDER } from "../dist/domain/investigationProviders.js";
import { loadRealHopperFixtureTargets } from "./lib/real-hopper-fixture.mjs";
+48 -7
View File
@@ -35,6 +35,7 @@ import {
verifyKunaStrings,
} from "./lib/kuna-string-verifier.mjs";
import { nativeKunaIncomingSites } from "./lib/kuna-native-reference-verifier.mjs";
import { MULTI_MODE_REMEDIATION } from "../dist/application/CapabilityInventory.js";
import {
assertVerifierCleanupComplete,
completeVerifierRun,
@@ -115,7 +116,9 @@ __attribute__((noinline)) int rea_classify(int c) {
}
int main(int argc, char **argv) {
if (argc < 2) return 2;
printf("%u %d\n", rea_checksum((const unsigned char *)argv[1], strlen(argv[1])),
size_t length = strlen(argv[1]);
printf("%u %u %d\n", rea_checksum((const unsigned char *)argv[1], length),
rea_checksum((const unsigned char *)argv[1], length ? length - 1 : 0),
rea_classify(argv[1][0]));
return 0;
}
@@ -282,8 +285,12 @@ const quality = (text) => ({
const verifyKuna = async (
path,
functions,
expectedCaller,
expectedStrings = [],
{
expectedCaller,
expectedStrings = [],
verifyNamedTrace = false,
minimumTraceReferences = {},
} = {},
) => {
const sha256 = createHash("sha256")
.update(await readFile(path))
@@ -292,6 +299,36 @@ const verifyKuna = async (
const answers = [];
let coldFailure;
try {
const status = await cold.call("binary_session", { detail: "complete" });
if (status.value.format === "elf") {
const availability = status.value.tool_availability;
const dispatch = availability.find(
({ name }) => name === "inspect_native_dispatch_metadata",
);
const trace = availability.find(
({ name }) => name === "trace_native_ui_action",
);
assert.equal(dispatch?.available, false);
assert.equal(
trace?.modes?.find(({ name }) => name === "native_seed_call_trace")
?.available,
false,
);
if (process.platform === "darwin") {
const authored = trace?.modes?.find(
({ name }) => name === "authored_ui_dispatch",
);
const nativeSeed = trace?.modes?.find(
({ name }) => name === "native_seed_call_trace",
);
assert.equal(trace?.remediation, MULTI_MODE_REMEDIATION);
assert.notEqual(authored?.remediation, nativeSeed?.remediation);
assert.ok(
authored?.missing_operations.includes("decode_interface_builder"),
);
assert.ok(nativeSeed?.missing_operations.includes("list_names"));
}
}
answers.push({
strings: await verifyKunaStrings({
call: cold.call,
@@ -320,6 +357,8 @@ const verifyKuna = async (
name,
address,
expectedCaller,
verifyNamedTrace,
minimumTraceReferences: minimumTraceReferences[name],
nativeIncoming: (targetAddress) =>
nativeKunaIncomingSites({
executable: kuna,
@@ -549,10 +588,12 @@ try {
for (const binary of [built, stripped])
results.push({
binary: binary.slice(workspace.length + 1),
kuna: await verifyKuna(binary, [...fixtureFunctions], fixtureMain, [
"_φ( °-°)/ REA unicode marker",
"REA wide marker",
]),
kuna: await verifyKuna(binary, [...fixtureFunctions], {
expectedCaller: fixtureMain,
expectedStrings: ["_φ( °-°)/ REA unicode marker", "REA wide marker"],
verifyNamedTrace: binary === built,
minimumTraceReferences: { rea_checksum: 2 },
}),
});
// Caller-built matrix.
+1 -7
View File
@@ -1,3 +1,4 @@
import { ADB_PROVIDER_IDENTITY } from "./AdbProviderIdentity.js";
import {
absoluteCallerPath,
appendCallerPath,
@@ -152,13 +153,6 @@ import {
type AdbCommandContext,
} from "./AdbDeviceOperations.js";
/** Stable identity for the ADB device-acquisition provider. */
export const ADB_PROVIDER_IDENTITY = {
id: "adb",
name: "Android Debug Bridge",
version: null,
} as const;
const SHELL_TIMEOUT_MS = 30_000;
const PACKAGE_LIST_TIMEOUT_MS = 60_000;
const PACKAGE_LIST_MAX_OUTPUT_BYTES = 8 * 1024 * 1024;
+6
View File
@@ -0,0 +1,6 @@
/** Stable identity for the ADB device-acquisition provider. */
export const ADB_PROVIDER_IDENTITY = {
id: "adb",
name: "Android Debug Bridge",
version: null,
} as const;
+1 -7
View File
@@ -1,3 +1,4 @@
import { APKTOOL_PROVIDER_IDENTITY } from "./ApktoolProviderIdentity.js";
import { countDecodedFiles } from "./ApktoolDecodedInventory.js";
import { ApktoolWorkspace } from "./ApktoolWorkspace.js";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
@@ -70,13 +71,6 @@ import {
parseManifestPackage,
} from "./ApktoolDecodeOutput.js";
/** Stable identity for the Apktool resource-decoding provider. */
export const APKTOOL_PROVIDER_IDENTITY = {
id: "apktool",
name: "Apktool",
version: null,
} as const;
const DECODE_MAX_OUTPUT_BYTES = 4 * 1024 * 1024;
const MANIFEST_MAX_BYTES = 512 * 1024;
const MAX_DECODED_ENTRIES_VISITED = 100_000;
+6
View File
@@ -0,0 +1,6 @@
/** Stable identity for the Apktool resource-decoding provider. */
export const APKTOOL_PROVIDER_IDENTITY = {
id: "apktool",
name: "Apktool",
version: null,
} as const;
@@ -1,4 +1,5 @@
import { describe, expect, it } from "vitest";
import { createDeferred } from "../../tests/fixtures/binarySession.js";
import {
getNavigationContext,
@@ -9,10 +10,19 @@ import {
type AnalysisOperationPort,
} from "./AnalysisProvider.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisCancelledError,
AnalysisOutputError,
AnalysisProtocolError,
} from "../domain/analysisErrorCore.js";
import { AnalysisCleanupLease } from "../artifacts/AnalysisCleanupLease.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
primaryAnalysisError,
type AnalysisError,
} from "../domain/analysisErrorBase.js";
import type { AnalysisExecution } from "./AnalysisProvider.js";
import type { Result } from "../domain/result.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok } from "../domain/result.js";
@@ -437,3 +447,262 @@ it("does not start address facets after document selection completes under cance
expect(result.error.cause).toBe(reason);
expect(calls).toEqual(["current_document"]);
});
it("retains both facet cleanup owners until their independent retries succeed", async () => {
const first = new AnalysisCapabilityUnavailableError(
provider.id,
"comment",
"failed",
{
cleanup: { reason: "first release failed", resources: ["first-owner"] },
},
);
const second = new AnalysisCapabilityUnavailableError(
provider.id,
"inline_comment",
"failed",
{
cleanup: { reason: "second release failed", resources: ["second-owner"] },
},
);
let firstRetryAllowed = false;
const firstLease = new AnalysisCleanupLease(async () => {
if (!firstRetryAllowed) throw new Error("still owned");
}, ["first-owner"]);
const secondLease = new AnalysisCleanupLease(async () => {}, [
"second-owner",
]);
firstLease.bindCleanupError(first);
secondLease.bindCleanupError(second);
const result = await inspectAddressContext(
{
execute: async (operation) =>
operation === "comment"
? err(first)
: operation === "inline_comment"
? err(second)
: ok(createAnalysisExecution(null, provider)),
},
{ address: "0x401000", document: "fixture" },
);
if (result.ok) throw new Error("expected retained cleanup failures");
expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete");
expect(result.error.cleanupResources).toEqual(
expect.arrayContaining(["first-owner", "second-owner"]),
);
await expect(firstLease.release()).rejects.toThrow("still owned");
expect(result.error.cleanupResources).toEqual(
expect.arrayContaining(["first-owner", "second-owner"]),
);
await secondLease.release();
expect(result.error.cleanupIncomplete).toBe(true);
expect(result.error.cleanupResources).toEqual(["first-owner"]);
firstRetryAllowed = true;
await firstLease.release();
expect(result.error.cleanupIncomplete).toBe(false);
expect(result.error.cleanupResources).toEqual([]);
expect(projectAnalysisError(result.error).code).toBe(
"capability_unavailable",
);
});
it("waits for late facet cleanup after cancellation and retains its owner", async () => {
const controller = new AbortController();
const started = createDeferred<void>();
const late = createDeferred<Result<AnalysisExecution, AnalysisError>>();
const failure = new AnalysisCapabilityUnavailableError(
provider.id,
"comment",
"failed",
{
cleanup: { reason: "late cleanup failed", resources: ["late-owner"] },
},
);
const lease = new AnalysisCleanupLease(async () => {}, ["late-owner"]);
lease.bindCleanupError(failure);
const analysis: AnalysisOperationPort = {
execute: (operation) => {
if (operation === "comment") {
started.resolve();
return late.promise;
}
return Promise.resolve(ok(createAnalysisExecution(null, provider)));
},
};
let settled = false;
const pending = inspectAddressContext(
analysis,
{ address: "0x401000", document: "fixture" },
controller.signal,
).then((result) => {
settled = true;
return result;
});
await started.promise;
controller.abort();
await Promise.resolve();
expect(settled).toBe(false);
late.resolve(err(failure));
const result = await pending;
if (result.ok) throw new Error("expected cancellation with cleanup failure");
expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete");
expect(result.error.cleanupResources).toEqual(["late-owner"]);
await lease.release();
expect(projectAnalysisError(result.error).code).toBe("cancelled");
});
it("reports cancellation after a late successful facet that ignored its signal", async () => {
const controller = new AbortController();
const started = createDeferred<void>();
const late = createDeferred<Result<AnalysisExecution, AnalysisError>>();
const pending = inspectAddressContext(
{
execute: (operation) => {
if (operation === "comment") {
started.resolve();
return late.promise;
}
return Promise.resolve(ok(createAnalysisExecution(null, provider)));
},
},
{ address: "0x401000", document: "fixture" },
controller.signal,
);
await started.promise;
controller.abort();
late.resolve(ok(createAnalysisExecution("finished", provider)));
const result = await pending;
if (result.ok) throw new Error("expected cancellation");
expect(projectAnalysisError(result.error).code).toBe("cancelled");
});
it("does not start facets for an already-cancelled explicit document", async () => {
const controller = new AbortController();
controller.abort();
const calls: string[] = [];
const result = await inspectAddressContext(
{
execute: async (operation) => {
calls.push(operation);
return ok(createAnalysisExecution(null, provider));
},
},
{ address: "0x401000", document: "fixture" },
controller.signal,
);
if (result.ok) throw new Error("expected cancellation");
expect(projectAnalysisError(result.error).code).toBe("cancelled");
expect(calls).toEqual([]);
});
it("preserves a provider's cancellation for the same signal", async () => {
const controller = new AbortController();
const started = createDeferred<void>();
const reason = { request: "controlled cancellation" };
let providerCancellation: AnalysisCancelledError | undefined;
const pending = inspectAddressContext(
{
execute: (operation) => {
if (operation !== "address_name")
return Promise.resolve(ok(createAnalysisExecution(null, provider)));
started.resolve();
return new Promise((resolve) => {
controller.signal.addEventListener(
"abort",
() => {
providerCancellation = new AnalysisCancelledError(operation, {
cause: controller.signal.reason,
diagnostics: { owner: "controlled provider" },
});
resolve(err(providerCancellation));
},
{ once: true },
);
});
},
},
{ address: "0x401000", document: "fixture" },
controller.signal,
);
await started.promise;
controller.abort(reason);
const result = await pending;
if (result.ok) throw new Error("expected cancellation");
expect(result.error).toBe(providerCancellation);
expect(projectAnalysisError(result.error).details).toMatchObject({
diagnostics: { owner: "controlled provider" },
});
});
it.each([false, true])(
"keeps an earlier cleanup failure primary when cancellation arrives later (cancellation cleanup=%s)",
async (cancelCleanup) => {
const controller = new AbortController();
const firstDone = createDeferred<void>();
const failure = new AnalysisCapabilityUnavailableError(
provider.id,
"comment",
"failed",
{
cleanup: { reason: "cleanup failed", resources: ["first-owner"] },
},
);
const lease = new AnalysisCleanupLease(async () => {}, ["first-owner"]);
lease.bindCleanupError(failure);
const cancelled = new AnalysisCancelledError(
"address_name",
cancelCleanup
? {
cleanup: {
reason: "cancel cleanup failed",
resources: ["cancel-owner"],
},
}
: undefined,
);
const cancellationLease = new AnalysisCleanupLease(async () => {}, [
"cancel-owner",
]);
if (cancelCleanup) cancellationLease.bindCleanupError(cancelled);
const analysis: AnalysisOperationPort = {
execute: async (operation) => {
if (operation === "address_name")
return new Promise((resolve) => {
controller.signal.addEventListener(
"abort",
() => resolve(err(cancelled)),
{ once: true },
);
});
if (operation === "comment") {
firstDone.resolve();
return err(failure);
}
return ok(createAnalysisExecution(null, provider));
},
};
const pending = inspectAddressContext(
analysis,
{ address: "0x401000", document: "fixture" },
controller.signal,
);
await firstDone.promise;
await Promise.resolve();
await Promise.resolve();
controller.abort();
const result = await pending;
if (result.ok) throw new Error("expected retained cleanup failure");
expect(primaryAnalysisError(result.error)).toBe(failure);
expect(failure.secondaryFailures).toContain(cancelled);
expect(projectAnalysisError(result.error).code).toBe("cleanup_incomplete");
await lease.release();
if (cancelCleanup) {
expect(result.error.cleanupIncomplete).toBe(true);
expect(result.error.cleanupResources).toEqual(["cancel-owner"]);
await cancellationLease.release();
}
expect(projectAnalysisError(result.error).code).toBe(
"capability_unavailable",
);
},
);
+91 -12
View File
@@ -4,7 +4,12 @@ import type {
AnalysisOperationPort,
} from "./AnalysisProvider.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import {
AnalysisCancelledError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { analysisErrorWithCancellation } from "../domain/analysisErrorCancellation.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { analysisBookmarkSchema } from "../domain/native/nativeInventory.js";
import { parseContainingProcedureResolution } from "../domain/native/nativeObservation.js";
@@ -98,18 +103,39 @@ export const inspectAddressContext = async (
"inline_comment",
"list_bookmarks",
] as const satisfies readonly AnalysisOperation[];
const results = await Promise.all(
operations.map((operation) =>
analysis.execute(
operation,
operation === "list_bookmarks" ? parameters(document.value) : args,
executionOptions(signal),
),
),
);
for (const result of results) {
if (!result.ok && isRequestLevelFailure(result.error)) return result;
const completed: Result<AnalysisExecution, AnalysisError>[] = [];
if (signal?.aborted === true)
return err(
new AnalysisCancelledError("inspect_address_context", {
cause: signal.reason,
}),
);
let signalCancellation: AnalysisCancelledError | undefined;
const cancelled = () => {
signalCancellation = new AnalysisCancelledError("inspect_address_context", {
cause: signal?.reason,
});
completed.push(err(signalCancellation));
};
signal?.addEventListener("abort", cancelled, { once: true });
let results: Result<AnalysisExecution, AnalysisError>[];
try {
results = await Promise.all(
operations.map(async (operation) => {
const result = await analysis.execute(
operation,
operation === "list_bookmarks" ? parameters(document.value) : args,
executionOptions(signal),
);
completed.push(result);
return result;
}),
);
} finally {
signal?.removeEventListener("abort", cancelled);
}
const fatal = contextFailure(completed, signalCancellation);
if (fatal !== undefined) return err(fatal);
const name = facet(results[0], operations[0]);
const procedureResult = results[1];
const resolution = procedureResult?.ok
@@ -160,6 +186,59 @@ const isRequestLevelFailure = (error: AnalysisError): boolean =>
error._tag === "AnalysisTimeoutError" ||
error._tag === "NoBinaryOpenError";
/** Wait for all facets and retain every unresolved cleanup owner. */
const contextFailure = (
results: readonly Result<AnalysisExecution, AnalysisError>[],
signalCancellation?: AnalysisCancelledError,
): AnalysisError | undefined => {
const failures = [
...new Set(results.flatMap((result) => (result.ok ? [] : [result.error]))),
];
// A provider's cancellation for this exact signal carries its own diagnostic
// and cleanup facts. Attribute that returned error to the abort event rather
// than replacing it with our generic observation of the same signal.
const marker =
signalCancellation === undefined
? -1
: failures.indexOf(signalCancellation);
if (marker >= 0) {
const reported = failures.findIndex(
(failure, index) =>
index > marker &&
failure instanceof AnalysisCancelledError &&
failure.cause === signalCancellation?.cause,
);
if (reported >= 0) {
failures[marker] = failures[reported]!;
failures.splice(reported, 1);
}
}
const first = failures.find(
(failure) => isRequestLevelFailure(failure) || failure.cleanupIncomplete,
);
if (first === undefined) return undefined;
let combined = first;
for (const failure of failures) {
if (failure === first) continue;
if (failure instanceof AnalysisCancelledError)
combined = analysisErrorWithCancellation(combined, failure);
if (failure.cleanupIncomplete) {
const previous = combined;
combined = analysisErrorWithCleanupFailure(
previous,
failure,
"inspect_address_context",
).bindCleanupDependencies([previous, failure]);
} else if (
!(failure instanceof AnalysisCancelledError) &&
isRequestLevelFailure(failure)
) {
combined.retainSecondaryFailure(failure);
}
}
return combined;
};
const matchingBookmarks = (value: JsonValue, address: string): Facet => {
if (!Array.isArray(value))
return {
+3
View File
@@ -2,6 +2,7 @@ import type { AnalysisWorkScope } from "./AnalysisWorkScope.js";
import type { AnalysisObservation } from "../domain/analysisObservation.js";
import type { ProviderInspectionScope } from "./ProviderInspectionScope.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import type { TargetApplicability } from "../domain/targetApplicability.js";
import type { OfficialToolName } from "../contracts/officialToolContracts.js";
import type { EnhancedToolName } from "../contracts/enhancedInputs.js";
import type { NativeToolName } from "../contracts/native/nativeToolContracts.js";
@@ -260,6 +261,8 @@ type CapabilityAvailability =
};
export type CapabilityDescriptor = CapabilityAvailability & {
/** Provider-owned static target requirements, evaluated for an open target. */
readonly targetApplicability?: TargetApplicability;
/**
* Snapshot opts immutable, profile-bound reads into replay despite incidental
* private filesystem writes. Live state must always be re-observed. Omitting
+18 -1
View File
@@ -28,7 +28,16 @@ it("uses the provider-owned overview without requesting complete inventories", a
);
},
});
expect(await tools.execute("binary_overview", {})).toEqual(ok(result));
// A provider or a snapshot saved before the record existed omits it.
expect(await tools.execute("binary_overview", {})).toEqual(
ok({
...result,
inventory_coverage: {
scope: "provider_inventory",
discovery_coverage: "unknown",
},
}),
);
expect(calls).toEqual(["read_binary_inventory_summary"]);
});
@@ -140,6 +149,10 @@ it("counts canonical fallback inventories including typed and display-only strin
document: "selected",
procedure_count: 1,
string_count: 2,
inventory_coverage: {
scope: "provider_inventory",
discovery_coverage: "unknown",
},
}),
);
expect(await fallbackOverview([], []).execute("binary_overview", {})).toEqual(
@@ -147,6 +160,10 @@ it("counts canonical fallback inventories including typed and display-only strin
document: "selected",
procedure_count: 0,
string_count: 0,
inventory_coverage: {
scope: "provider_inventory",
discovery_coverage: "unknown",
},
}),
);
});
+50
View File
@@ -0,0 +1,50 @@
import type { ProviderAvailability } from "./AnalysisProvider.js";
import type { JebAnalysisAvailability } from "./jeb/JebAnalysisPort.js";
import type { AvailabilityPolicy } from "./CapabilityInventory.js";
import type { OptionalProviderLoadFailures } from "./OptionalObservationProviders.js";
export interface CapabilityAvailabilityFacts {
readonly probes?: Partial<CapabilityAvailabilityProbes>;
readonly observeProviderLoadFailures?: () => OptionalProviderLoadFailures;
readonly optionalProviderLoadFailures?:
| OptionalProviderLoadFailures
| undefined;
readonly hostUnsupported?: AvailabilityPolicy["hostUnsupported"];
readonly optionalFeatures?: Pick<
AvailabilityPolicy,
| "packetCaptureEnabled"
| "signalRecordingEnabled"
| "wasmArtifactEnabled"
| "evmInterfaceEnabled"
| "browserObservationEnabled"
| "browserScenarioEnabled"
| "electronObservationEnabled"
| "electronAutomationEnabled"
| "v8InspectorObservationEnabled"
| "javascriptRecoveryEnabled"
| "webModuleResolutionEnabled"
| "binaryLayoutEnabled"
| "recordedCrashEnabled"
| "firmwareInspectionEnabled"
| "firmwareExtractionEnabled"
>;
}
/** Provider-owned live probes used by session discovery. */
export interface CapabilityAvailabilityProbes {
readonly androidAnalysisAvailability: (
signal: AbortSignal,
) => Promise<ProviderAvailability>;
readonly adbAnalysisAvailability?: (
signal: AbortSignal,
) => Promise<ProviderAvailability>;
readonly apktoolAnalysisAvailability?: (
signal: AbortSignal,
) => Promise<ProviderAvailability>;
readonly jebAnalysisAvailability?: (
signal: AbortSignal,
) => Promise<JebAnalysisAvailability>;
readonly evmLimiterAvailability?: (
signal: AbortSignal,
) => Promise<ProviderAvailability | undefined>;
}
@@ -776,6 +776,23 @@ describe("trace_feature requirements", () => {
reason: null,
}));
it("remains available when owner lookup is unavailable", () => {
expect(
entry(
"trace_feature",
status({
open: true,
kind: "executable",
capabilities: capabilities([
"search_strings",
"search_procedures",
"xrefs",
]),
}),
),
).toMatchObject({ available: true, reason: "available" });
});
it("needs the search operations the workflow actually calls", () => {
expect(
entry(
+14 -9
View File
@@ -126,6 +126,10 @@ type AvailabilityModeResult =
| (Extract<ToolAvailabilityMode, { available: false }> & {
readonly reason: ToolUnavailabilityReason;
});
/** Recovery when unavailable modes require different actions. */
export const MULTI_MODE_REMEDIATION =
"No mode is currently advertised as available. Inspect each mode's missing operations and remediation.";
type AvailabilityContext = {
readonly name: string;
readonly kind: ToolKind;
@@ -215,12 +219,7 @@ const ENHANCED_REQUIREMENTS: ReadonlyMap<string, ProviderComposition> = new Map(
],
analyze_function: ["analyze_function"],
inspect_native_api: ["analyze_function"],
trace_feature: [
"search_strings",
"search_procedures",
"xrefs",
"resolve_containing_procedure",
],
trace_feature: ["search_strings", "search_procedures", "xrefs"],
} satisfies Readonly<Record<EnhancedContractName, ProviderComposition>>),
);
@@ -787,6 +786,7 @@ const targetAvailability = ({
const descriptor = descriptors.get(name);
if (
targetOpen &&
kind !== "enhanced" &&
descriptor?.available === false &&
(descriptor.availability_code === "target_kind_unsupported" ||
descriptor.availability_code === "target_format_unsupported")
@@ -984,6 +984,10 @@ const availabilityForModes = (
)[0];
const specificFailureRemediation =
failure?.reason === "provider_missing" ? undefined : failure?.remediation;
const distinctModeRemediations =
defaultMode === undefined &&
failures.length > 1 &&
new Set(failures.map((mode) => mode.remediation)).size > 1;
const facts = {
...(defaultMode === undefined
? {}
@@ -1001,9 +1005,10 @@ const availabilityForModes = (
return {
...facts,
reason: failure?.reason ?? "provider_missing",
remediation:
specificFailureRemediation ??
`Configure a provider for one of: ${missingOperations.join(", ")}.`,
remediation: distinctModeRemediations
? MULTI_MODE_REMEDIATION
: (specificFailureRemediation ??
`Configure a provider for one of: ${missingOperations.join(", ")}.`),
};
};
+116
View File
@@ -0,0 +1,116 @@
import { expect, it, vi } from "vitest";
import { awaitAcknowledgement } from "../../tests/fixtures/lifecycleAcknowledgement.js";
import { CapabilityScope } from "./CapabilityScope.js";
it("retains each failed cleanup while releasing successful peers across retries", async () => {
const scope = new CapabilityScope("Controlled mixed capability cleanup");
const synchronousFailure = new Error("Synchronous close refusal");
const asynchronousFailure = new Error("Asynchronous close refusal");
let allowSynchronous = false;
let allowAsynchronous = false;
const successful = vi.fn(async () => undefined);
const synchronous = vi.fn(() => {
if (!allowSynchronous) throw synchronousFailure;
return Promise.resolve();
});
const asynchronous = vi.fn(async () => {
if (!allowAsynchronous) throw asynchronousFailure;
});
const rejectedWithoutReason = vi.fn(() =>
allowAsynchronous ? Promise.resolve() : Promise.reject(undefined),
);
scope.own("released-peer", {}, successful);
scope.own("synchronous-owner", {}, synchronous);
scope.own("asynchronous-owner", {}, asynchronous);
scope.own("undefined-rejection-owner", {}, rejectedWithoutReason);
try {
const first: unknown = await scope.close().catch((cause: unknown) => cause);
expect(first).toBeInstanceOf(AggregateError);
if (!(first instanceof AggregateError))
throw new Error("Expected all cleanup failures");
expect(first.message).toBe("Controlled mixed capability cleanup");
expect(first.errors).toEqual([
synchronousFailure,
asynchronousFailure,
undefined,
]);
for (const close of [
successful,
synchronous,
asynchronous,
rejectedWithoutReason,
])
expect(close).toHaveBeenCalledTimes(1);
expect(() => scope.own("late-owner", {}, successful)).toThrow(
"after host shutdown",
);
allowAsynchronous = true;
const second: unknown = await scope
.close()
.catch((cause: unknown) => cause);
expect(second).toBeInstanceOf(AggregateError);
if (!(second instanceof AggregateError))
throw new Error("Expected the remaining owner failure");
expect(second.errors).toEqual([synchronousFailure]);
expect(successful).toHaveBeenCalledTimes(1);
expect(synchronous).toHaveBeenCalledTimes(2);
expect(asynchronous).toHaveBeenCalledTimes(2);
expect(rejectedWithoutReason).toHaveBeenCalledTimes(2);
allowSynchronous = true;
await scope.close();
await scope.close();
expect(synchronous).toHaveBeenCalledTimes(3);
expect(successful).toHaveBeenCalledTimes(1);
expect(asynchronous).toHaveBeenCalledTimes(2);
expect(rejectedWithoutReason).toHaveBeenCalledTimes(2);
} finally {
allowSynchronous = true;
allowAsynchronous = true;
await scope.close();
}
});
it("joins concurrent shutdowns and seals ownership before asynchronous cleanup settles", async () => {
const scope = new CapabilityScope();
let acknowledge = () => {};
const entered = new Promise<void>((resolve) => {
acknowledge = resolve;
});
let release = () => {};
const released = new Promise<void>((resolve) => {
release = resolve;
});
const close = vi.fn(async () => {
acknowledge();
await released;
});
scope.own("controlled-owner", {}, close);
const first = scope.close();
let second: Promise<void> | undefined;
try {
await awaitAcknowledgement(entered, first, "capability cleanup");
second = scope.close();
expect(second).toBe(first);
expect(close).toHaveBeenCalledTimes(1);
expect(() => scope.own("later-owner", {}, async () => undefined)).toThrow(
"after host shutdown",
);
release();
await Promise.all([first, second]);
expect(() =>
scope.own("post-shutdown-owner", {}, async () => undefined),
).toThrow("after host shutdown");
await scope.close();
expect(close).toHaveBeenCalledTimes(1);
} finally {
release();
await Promise.allSettled([
first,
...(second === undefined ? [] : [second]),
]);
await scope.close();
}
});
+40
View File
@@ -0,0 +1,40 @@
/** One host owns these exact close capabilities; a released peer stays released. */
export class CapabilityScope {
readonly #pending = new Map<string, () => Promise<void>>();
#accepting = true;
#closing: Promise<void> | undefined;
constructor(readonly failureMessage = "REA provider cleanup failed") {}
own<Value>(id: string, value: Value, close: () => Promise<void>): Value {
if (!this.#accepting)
throw new Error(`Cannot acquire capability ${id} after host shutdown`);
if (this.#pending.has(id))
throw new Error(`Capability ${id} already belongs to this host`);
this.#pending.set(id, close);
return value;
}
/** Join concurrent closes, retaining only failed owners for the next retry. */
close(): Promise<void> {
this.#accepting = false;
this.#closing ??= Promise.allSettled(
[...this.#pending].map(async ([id, close]) => {
// A synchronous adapter throw must not prevent other owners' cleanup.
await Promise.resolve().then(close);
this.#pending.delete(id);
}),
)
.then((results) => {
const failures = results.flatMap((result) =>
result.status === "rejected" ? [result.reason] : [],
);
if (failures.length > 0)
throw new AggregateError(failures, this.failureMessage);
})
.finally(() => {
this.#closing = undefined;
});
return this.#closing;
}
}
+8 -2
View File
@@ -42,8 +42,8 @@ import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { requiresNativeSourceAdmissionBeforeReplay } from "./binary/AnalysisSnapshotCache.js";
import { classifyBinaryTargetFilesystemFailure } from "./BinaryTargetResolver.js";
import {
snapshotBindingMismatch,
snapshotEvidenceForQuery,
snapshotMatchesTarget,
} from "../domain/analysisSnapshot.js";
import type { AnalysisProfileCommitment } from "../domain/analysisProfile.js";
import { profilelessNativeDispatchProfile } from "./binary/ProfilelessNativeWorkflowSnapshot.js";
@@ -742,10 +742,16 @@ const prepareSnapshot = async (options: {
...targetSelection(options),
});
if (!target.ok) return target;
if (!snapshotMatchesTarget(loaded.value.target, target.value))
const mismatch = snapshotBindingMismatch(
loaded.value,
target.value,
undefined,
);
if (mismatch?.target !== undefined)
return err(
new AnalysisSnapshotMismatchError(
"Analysis snapshot target does not match the requested binary",
mismatch,
),
);
return ok({ snapshot: loaded.value, target: target.value });
+45 -11
View File
@@ -4,12 +4,16 @@ import type {
AnalysisExecution,
AnalysisOperation,
} from "./AnalysisProvider.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { JsonValue } from "../domain/jsonValue.js";
import {
parseContainingProcedureResolution,
type ContainingProcedureResolution,
type TraceContainingProcedure,
} from "../domain/native/nativeObservation.js";
import { err, ok, type Result } from "../domain/result.js";
import type { EnhancedResult, TraceReference } from "./EnhancedToolTypes.js";
@@ -70,7 +74,7 @@ export const traceLiteralFeature = async (
search_mode: "literal",
matches: searched.value.matches,
references: traced.value.references,
truncated: traced.value.residual.length > 0,
truncated: traced.value.truncated,
residual_unknowns: traced.value.residual,
});
};
@@ -121,14 +125,18 @@ const traceReferences = async (
observe: ObserveCall,
request: ReferenceTraceRequest,
): Promise<
Result<{ references: TraceReference[]; residual: string[] }, AnalysisError>
Result<
{ references: TraceReference[]; residual: string[]; truncated: boolean },
AnalysisError
>
> => {
const references: TraceReference[] = [];
const residual: string[] = [];
let truncated = false;
// References identify targets by address, independently of search alternatives.
const tracedTargets = new Set<string>();
// A source address is resolved once however many matches reference it.
const containing = new Map<string, ContainingProcedureResolution>();
const containing = new Map<string, TraceContainingProcedure>();
for (const match of request.matches) {
if (tracedTargets.has(match.address)) continue;
tracedTargets.add(match.address);
@@ -140,10 +148,12 @@ const traceReferences = async (
if (!observed.ok) return observed;
// Only an explicit producer report of truncation is a coverage gap; a
// provider that reports nothing keeps its previous, unqualified result.
if (observed.value.referencesTruncated === true)
if (observed.value.referencesTruncated === true) {
truncated = true;
residual.push(
`References to ${match.address} were truncated by the provider; additional references may exist.`,
);
}
const sources = observed.value.result;
if (!Array.isArray(sources))
return err(
@@ -167,10 +177,34 @@ const traceReferences = async (
{ address: source },
request.signal,
);
if (!resolved.ok) return resolved;
const parsed = parseContainingProcedureResolution(resolved.value);
if (!parsed.ok) return parsed;
containingProcedure = parsed.value;
if (!resolved.ok) {
if (
!(resolved.error instanceof AnalysisCapabilityUnavailableError) ||
resolved.error.operation !== "resolve_containing_procedure" ||
resolved.error.cleanupIncomplete ||
resolved.error.retryable
)
return resolved;
if (request.signal?.aborted === true)
return err(
new AnalysisCancelledError("trace_feature", {
cause: request.signal.reason,
}),
);
containingProcedure = {
query_address: source,
status: "unavailable",
provider_id: resolved.error.providerId,
reason: resolved.error.reason,
};
residual.push(
`Containing procedure for ${source} is unavailable: ${resolved.error.reason}`,
);
} else {
const parsed = parseContainingProcedureResolution(resolved.value);
if (!parsed.ok) return parsed;
containingProcedure = parsed.value;
}
containing.set(source, containingProcedure);
}
references.push({
@@ -180,5 +214,5 @@ const traceReferences = async (
});
}
}
return ok({ references, residual });
return ok({ references, residual, truncated });
};
+2 -2
View File
@@ -3,7 +3,7 @@ import type { z } from "zod";
import { analysisInputErrorFromIssues } from "../domain/inputIssueProjection.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { ContainingProcedureResolution } from "../domain/native/nativeObservation.js";
import type { TraceContainingProcedure } from "../domain/native/nativeObservation.js";
import { err, type Result } from "../domain/result.js";
import {
enhancedInputSchemas,
@@ -14,7 +14,7 @@ export type EnhancedResult = Promise<Result<JsonValue, AnalysisError>>;
export type TraceReference = {
readonly target_address: string;
readonly source_address: string;
readonly containing_procedure: ContainingProcedureResolution;
readonly containing_procedure: TraceContainingProcedure;
};
/** One enhanced call whose input was parsed at its owning adapter boundary. */
+11 -7
View File
@@ -1,4 +1,5 @@
import { permitsCapabilityFallback } from "./CapabilityFallback.js";
import { withOverviewInventoryCoverage } from "../domain/native/inventoryCoverage.js";
import type { NativeCallGraph } from "../domain/native/nativeCallGraph.js";
import { nativeProcedureInputSchema } from "../domain/native/nativeProcedureInput.js";
import { CALL_RELATIONSHIP_TRAVERSALS } from "./CallRelationshipTraversal.js";
@@ -782,7 +783,8 @@ export class EnhancedTools {
{},
signal,
);
if (direct.ok || !permitsCapabilityFallback(direct.error)) return direct;
if (direct.ok) return ok(withOverviewInventoryCoverage(direct.value));
if (!permitsCapabilityFallback(direct.error)) return direct;
const document = await resolveAnalysisDocument(
this.analysis,
undefined,
@@ -817,12 +819,14 @@ export class EnhancedTools {
if (!stringCount.ok) return stringCount;
return ok(
projectBinaryOverview({
document: document.value,
...(segments?.ok ? { segments: segments.value } : {}),
procedure_count: procedureCount.value,
string_count: stringCount.value,
}),
withOverviewInventoryCoverage(
projectBinaryOverview({
document: document.value,
...(segments?.ok ? { segments: segments.value } : {}),
procedure_count: procedureCount.value,
string_count: stringCount.value,
}),
),
);
}
+38 -5
View File
@@ -1,4 +1,6 @@
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { EvidenceBundleExportReceipt } from "../domain/investigationLifecycle.js";
import { jsonValueValidationIssue } from "../domain/jsonValue.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { requireAuthenticationHeadroom } from "./JsonAuthenticationHeadroom.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
@@ -7,6 +9,7 @@ import { describeValidationFailure } from "../domain/validationFailure.js";
import {
safeParseImmutableEvidenceBundle,
safeParseEvidenceBundle,
isImmutableEvidenceBundle,
type EvidenceBundle,
} from "../domain/evidenceBundle.js";
import {
@@ -35,13 +38,14 @@ type EvidenceReadFailure =
type EvidenceWriteFailure =
| EvidenceFileError
| EvidenceIntegrityError
| AnalysisCancelledError;
| AnalysisCancelledError
| AnalysisResourceConstraintError;
/** Project a published bundle with record and residual-unknown revision counts. */
export const evidenceBundleExportReceipt = (
bundle: EvidenceBundle,
published: PublishedEvidenceFile,
) => ({
): EvidenceBundleExportReceipt => ({
path: published.path,
bytes: published.bytes,
records: bundle.records.length,
@@ -160,15 +164,28 @@ export const authenticateEvidenceBundleValue = (
}),
);
}
return validateBundlePayloads(parsed.value);
return validateBundlePayloads(parsed.value, path, operation);
};
/** Validate typed payloads without restricting historical workflow or provider identities. */
const validateBundlePayloads = (
bundle: EvidenceBundle,
): Result<EvidenceBundle, EvidenceIntegrityError> => {
path: string,
operation: string,
): Result<
EvidenceBundle,
EvidenceIntegrityError | AnalysisResourceConstraintError
> => {
for (const record of bundle.records) {
if (record.predicate_type !== "rea.process-capture") continue;
// A sealed bundle borrows records, but this typed parser still allocates a
// capture copy. Admit its actual working set against current headroom.
const headroom = requireAuthenticationHeadroom(record.normalized_result, {
operation,
path: absoluteCallerPath(path),
description: `Insufficient heap headroom to validate process capture ${record.evidence_id} in the Evidence bundle`,
});
if (!headroom.ok) return headroom;
const parsedCapture = safeParseProcessCapture(record.normalized_result);
if (!parsedCapture.success) {
return err(
@@ -202,6 +219,22 @@ export const writeEvidenceBundle = async (
cause: signal.reason,
}),
);
if (!isImmutableEvidenceBundle(bundle)) {
const issue = jsonValueValidationIssue(bundle);
if (issue !== undefined)
return err(
new EvidenceIntegrityError("Evidence checkpoint is not ordinary JSON", {
userMessage: issue,
}),
);
const headroom = requireAuthenticationHeadroom(bundle, {
operation,
path: absoluteCallerPath(path),
description:
"Insufficient heap headroom to authenticate the Evidence checkpoint",
});
if (!headroom.ok) return headroom;
}
const checked = safeParseEvidenceBundle(bundle);
if (!checked.ok) {
return err(
@@ -210,7 +243,7 @@ export const writeEvidenceBundle = async (
}),
);
}
const validated = validateBundlePayloads(checked.value);
const validated = validateBundlePayloads(checked.value, path, operation);
if (!validated.ok) return validated;
return writeTextParts(
bufferedJsonParts(canonicalJsonParts(validated.value)),
+178
View File
@@ -318,3 +318,181 @@ it("uses only the fixed managed guide inventory for injected reads", async () =>
);
expect(requested).toEqual([...MANAGED_SKILL_FILES]);
});
const nestedGuide = [
"# Native guide",
"",
"Overview of native work.",
"",
"## Apple app resources",
"",
"Switch provider bundles here.",
"",
"## Native targets",
"",
"Annotations in Ghidra last until close.",
"",
"### Function annotations in Ghidra",
"",
"Rename functions; annotations last until close.",
"",
"## Combined recovery",
"",
"Recovery steps.",
"",
"### Step alpha",
"",
"Use alphaword here.",
"",
"### Step beta",
"",
"Use betaword here.",
"",
].join("\n");
const nestedReader = async (path: string): Promise<string> =>
path === "references/native-and-artifacts.md"
? nestedGuide
: `# ${path}\n\nUnrelated guidance.\n`;
const matchesFor = async (query: string, limit = 20) => {
const result = await valueOf(
searchGuidanceWithReader(
{ query, limit },
nestedReader,
new ArtifactResourceScope(),
),
);
if (result.status !== "matches") throw new Error(`No matches: ${query}`);
return result.matches;
};
it("reports a focused subsection without repeating every enclosing parent", async () => {
const matches = await matchesFor("function annotations");
expect(matches).toEqual([
expect.objectContaining({
heading: "Function annotations in Ghidra",
match_scope: "section",
matched_terms: ["function", "annotations"],
}),
]);
expect(matches[0]).not.toHaveProperty("subsections");
expect(matches[0]).not.toHaveProperty("own_text_terms");
});
it("keeps a parent whose own text holds every term alongside its matching subsection", async () => {
const matches = await matchesFor("annotations close");
expect(
matches.map(({ heading, match_scope }) => [heading, match_scope]),
).toEqual([
["Function annotations in Ghidra", "section"],
["Native targets", "section"],
]);
expect(matches[1]?.context).not.toContain("Rename functions");
});
it("labels terms scattered across sibling subsections and points to where each occurs", async () => {
const matches = await matchesFor("switch provider annotations");
expect(matches).toHaveLength(1);
const [root] = matches;
expect(root).toMatchObject({
heading: "Native guide",
match_scope: "subsections",
matched_terms: ["switch", "provider", "annotations"],
content_length: nestedGuide.trim().length,
own_text_terms: [],
relevance: expect.stringContaining("no single subsection"),
});
expect(
root?.subsections?.map(({ heading, matched_terms }) => [
heading,
matched_terms,
]),
).toEqual([
["Apple app resources", ["switch", "provider"]],
["Native targets", ["annotations"]],
]);
const pointer = root?.subsections?.[0];
if (pointer === undefined) throw new Error("Expected a subsection pointer");
const read = await readGuidanceSectionWithReader(
{ locator: pointer.locator },
nestedReader,
new ArtifactResourceScope(),
);
expect(read).toMatchObject({
ok: true,
value: { status: "section", heading: "Apple app resources" },
});
const complete = await readGuidanceSectionWithReader(
{ locator: root?.locator },
nestedReader,
new ArtifactResourceScope(),
);
if (!complete.ok || complete.value.status !== "section")
throw new Error("Expected the complete parent section");
expect(complete.value.content).toHaveLength(root?.content_length ?? -1);
expect(complete.value.content).toContain("### Step beta");
});
it("separates a parent's own terms from those found in one subsection", async () => {
const [combined, ...rest] = await matchesFor("recovery alphaword");
expect(rest).toEqual([]);
expect(combined).toMatchObject({
heading: "Combined recovery",
match_scope: "subsections",
own_text_terms: ["recovery"],
});
expect(combined?.subsections?.map(({ heading }) => heading)).toEqual([
"Step alpha",
]);
});
it("ranks co-located terms before broader matches and treats heading words as own text", async () => {
const matches = await matchesFor("alphaword betaword");
expect(matches.map(({ heading }) => heading)).toEqual(["Combined recovery"]);
expect(await matchesFor("apple resources")).toEqual([
expect.objectContaining({
heading: "Apple app resources",
match_scope: "section",
}),
]);
// The scattered parent has a heading term and the higher score.
const focusedFirst = await valueOf(
searchGuidanceWithReader(
{ query: "recovery alphaword" },
async (path) =>
path === "SKILL.md"
? "# REA\n\n## Focused\n\nThe alphaword recovery recipe.\n"
: nestedReader(path),
new ArtifactResourceScope(),
),
);
if (focusedFirst.status !== "matches") throw new Error("Expected matches");
expect(
focusedFirst.matches.map(({ heading, match_scope }) => [
heading,
match_scope,
]),
).toEqual([
["Focused", "section"],
["Combined recovery", "subsections"],
]);
});
it("bounds subsection pointers and counts the remainder", async () => {
const terms = Array.from({ length: 10 }, (_, index) => `scatterterm${index}`);
const guide = [
"# Scattered",
"",
...terms.flatMap((term, index) => [`## Part ${index}`, "", term, ""]),
].join("\n");
const result = await valueOf(
searchGuidanceWithReader(
{ query: terms.join(" ") },
async (path) => (path === "SKILL.md" ? guide : `# ${path}\n`),
new ArtifactResourceScope(),
),
);
if (result.status !== "matches") throw new Error("Expected matches");
expect(result.matches).toHaveLength(1);
expect(result.matches[0]?.subsections).toHaveLength(8);
expect(result.matches[0]?.omitted_subsections).toBe(2);
});
+189 -42
View File
@@ -6,6 +6,7 @@ import { AnalysisCleanupLease } from "../artifacts/AnalysisCleanupLease.js";
import { MANAGED_SKILL_FILES, readSkillFile } from "./SetupSkill.js";
import { PRODUCT_IDENTITY } from "../identity.js";
import {
MAX_SUBSECTION_POINTERS,
readGuidanceSectionInputSchema,
readGuidanceSectionOutputSchema,
type ReadGuidanceSectionOutput,
@@ -327,41 +328,11 @@ const searchGuidanceInScope = async (
);
const queryWords = words(parsed.data.query);
const specificQueryWords = queryWords.filter(
(word) => !GENERIC_QUERY_WORDS.has(word),
const { coverage, ranked } = rankSections(
bundle.sections,
queryWords,
queryWords.filter((word) => !GENERIC_QUERY_WORDS.has(word)),
);
const queryWordsToMatch = specificQueryWords;
const ranked = bundle.sections
.flatMap((section) => {
const sectionWords = new Set(words(section.content));
const matchedSpecific = queryWordsToMatch.filter((word) =>
sectionWords.has(word),
);
if (
queryWordsToMatch.length === 0 ||
matchedSpecific.length !== queryWordsToMatch.length
)
return [];
const matched = queryWords.filter((word) => sectionWords.has(word));
const headingWords = new Set(words(section.heading));
const score =
matched.length +
matched.filter((word) => headingWords.has(word)).length * 2;
return {
section,
matched,
score,
wordCount: sectionWords.size,
};
})
.sort(
(left, right) =>
right.score - left.score ||
// Prefer a focused recipe over a long overview when term scores tie.
left.wordCount - right.wordCount ||
compareText(left.section.guide, right.section.guide) ||
left.section.ordinal - right.section.ordinal,
);
if (ranked.length === 0)
return ok(
@@ -396,14 +367,7 @@ const searchGuidanceInScope = async (
package: `${PRODUCT_IDENTITY.packageName}@${PRODUCT_IDENTITY.packageVersion}`,
bundle_sha256: bundle.digest,
query: parsed.data.query,
matches: page.map(({ section, matched }) => ({
guide: section.guide,
heading: section.heading,
locator: sectionLocator(bundle, section),
matched_terms: matched,
relevance: `Matched ${matched.join(", ")} in this section.`,
context: contextFor(section.content, matched),
})),
matches: page.map((match) => presentMatch(bundle, match, coverage)),
total_matches: ranked.length,
offset,
limit: parsed.data.limit,
@@ -422,6 +386,189 @@ const searchGuidanceInScope = async (
);
};
interface TermCoverage {
readonly section: GuideSection;
/** Query words in the section's own text, in query order. */
readonly own: readonly string[];
/** Query words anywhere in the section, including subsections. */
readonly subtree: Set<string>;
/** One subsection alone contains every specific query word. */
childCovers: boolean;
}
interface RankedSection {
readonly section: GuideSection;
readonly scope: "section" | "subsections";
readonly matched: readonly string[];
readonly score: number;
/** Query words in the section's own text. */
readonly own: readonly string[];
/** Specific words absent from the section's own text. */
readonly scattered: readonly string[];
/** Position of this section in document-ordered coverage. */
readonly index: number;
}
interface SectionRanking {
readonly coverage: readonly TermCoverage[];
readonly ranked: readonly RankedSection[];
}
/**
* Own text is tokenized once; subtree coverage is folded upward in reverse
* document order, so every subsection is complete before its parent reads it.
*/
const rankSections = (
sections: readonly GuideSection[],
queryWords: readonly string[],
specificWords: readonly string[],
): SectionRanking => {
if (specificWords.length === 0) return { coverage: [], ranked: [] };
const bySection = new Map<GuideSection, TermCoverage>();
const coverage = sections.map((section) => {
const own = new Set(words(section.ownContent));
const present = queryWords.filter((word) => own.has(word));
const fact = {
section,
own: present,
subtree: new Set(present),
childCovers: false,
};
bySection.set(section, fact);
return fact;
});
const covers = (terms: ReadonlySet<string>): boolean =>
specificWords.every((word) => terms.has(word));
for (const fact of coverage.toReversed()) {
const parent =
fact.section.parent === undefined
? undefined
: bySection.get(fact.section.parent);
if (parent === undefined) continue;
for (const word of fact.subtree) parent.subtree.add(word);
if (covers(fact.subtree)) parent.childCovers = true;
}
const ranked = coverage
.flatMap((fact, index): RankedSection[] => {
if (!covers(fact.subtree)) return [];
const own = new Set(fact.own);
const focused = covers(own);
// A subsection holding every term is already a match of its own.
if (!focused && fact.childCovers) return [];
const matched = focused
? fact.own
: queryWords.filter((word) => fact.subtree.has(word));
const headingWords = new Set(words(fact.section.heading));
return [
{
section: fact.section,
scope: focused ? "section" : "subsections",
matched,
score:
matched.length +
matched.filter((word) => headingWords.has(word)).length * 2,
own: fact.own,
scattered: specificWords.filter((word) => !own.has(word)),
index,
},
];
})
.sort(
(left, right) =>
// Co-located terms rank before terms gathered across subsections.
Number(left.scope === "subsections") -
Number(right.scope === "subsections") ||
right.score - left.score ||
// Prefer a focused recipe over a long overview when term scores tie.
left.section.content.length - right.section.content.length ||
compareText(left.section.guide, right.section.guide) ||
left.section.ordinal - right.section.ordinal,
);
return { coverage, ranked };
};
const isWithin = (section: GuideSection, ancestor: GuideSection): boolean => {
for (let parent = section.parent; parent !== undefined;) {
if (parent === ancestor) return true;
parent = parent.parent;
}
return false;
};
/**
* Greedy cover of the terms absent from a scattered match's own text: each
* pointer is the subsection whose own text locates the most remaining terms,
* earliest in the document on ties.
*/
const subsectionPointers = (
match: RankedSection,
coverage: readonly TermCoverage[],
): { readonly pointers: readonly TermCoverage[]; readonly omitted: number } => {
const descendants: TermCoverage[] = [];
// Document order keeps every subsection contiguous after its parent.
for (let index = match.index + 1; index < coverage.length; index += 1) {
const fact = coverage[index];
if (fact === undefined || !isWithin(fact.section, match.section)) break;
descendants.push(fact);
}
const remaining = new Set(match.scattered);
const pointers: TermCoverage[] = [];
while (remaining.size > 0) {
let best: TermCoverage | undefined;
let bestCount = 0;
for (const fact of descendants) {
const count = fact.own.filter((word) => remaining.has(word)).length;
if (count > bestCount) {
best = fact;
bestCount = count;
}
}
// Admission guarantees every scattered term has a subsection owner.
if (best === undefined) break;
for (const word of best.own) remaining.delete(word);
pointers.push(best);
}
return {
pointers: pointers.slice(0, MAX_SUBSECTION_POINTERS),
omitted: Math.max(0, pointers.length - MAX_SUBSECTION_POINTERS),
};
};
const presentMatch = (
bundle: GuidanceBundle,
match: RankedSection,
coverage: readonly TermCoverage[],
) => {
const { section, scope, matched } = match;
const common = {
guide: section.guide,
heading: section.heading,
locator: sectionLocator(bundle, section),
match_scope: scope,
matched_terms: matched,
content_length: section.content.length,
};
if (scope === "section")
return {
...common,
relevance: `Matched ${matched.join(", ")} in this section's own text.`,
context: contextFor(section.ownContent, matched),
};
const { pointers, omitted } = subsectionPointers(match, coverage);
return {
...common,
relevance: `Matched ${matched.join(", ")} only across different subsections; no single subsection contains every term.`,
context: contextFor(section.content, matched),
own_text_terms: match.own,
subsections: pointers.map((pointer) => ({
heading: pointer.section.heading,
locator: sectionLocator(bundle, pointer.section),
matched_terms: pointer.own,
})),
...(omitted === 0 ? {} : { omitted_subsections: omitted }),
};
};
export const readGuidanceSection = async (
input: unknown,
resources: ArtifactResourceScope,
+36 -6
View File
@@ -2,12 +2,18 @@
export interface GuideSection {
readonly guide: string;
readonly heading: string;
/** The complete section, including every subsection. */
readonly content: string;
/** Heading and body before the first subsection, where terms co-occur. */
readonly ownContent: string;
readonly ordinal: number;
/** Enclosing section of the same guide; top-level headings have none. */
readonly parent: GuideSection | undefined;
}
// The shipped bundle has tens of headings. These ceilings bound object count
// and UTF-16 text retained by a complete index before allocating section text.
// Own text needs no ceiling of its own: each code unit has exactly one owner.
const MAX_SECTIONS = 4096;
// ATX has at most six enclosing levels over the admitted 4 MiB bundle.
const MAX_EXPANDED_CHARACTERS = 6 * 4_194_304;
@@ -34,6 +40,8 @@ interface Heading {
readonly level: number;
readonly title: string;
readonly start: number;
readonly ordinal: number;
readonly parent: number | undefined;
end: number;
}
@@ -82,7 +90,13 @@ const readHeadings = (
const parent = enclosing.pop();
if (parent !== undefined) parent.end = start;
}
const heading = { ...parsed, start, end: content.length };
const heading = {
...parsed,
start,
end: content.length,
ordinal: headings.length,
parent: enclosing.at(-1)?.ordinal,
};
headings.push(heading);
enclosing.push(heading);
}
@@ -92,7 +106,10 @@ const readHeadings = (
return headings;
};
/** Bound the whole bundle before materializing an expanded section index. */
/**
* Bound the whole bundle before materializing an expanded section index.
* Sections are returned in document order, so subsections follow their parent.
*/
export const parseGuidanceSections = (
files: ReadonlyMap<string, string>,
): readonly GuideSection[] | undefined => {
@@ -111,13 +128,26 @@ export const parseGuidanceSections = (
documents.push({ guide, content, headings });
}
const sections: GuideSection[] = [];
for (const { guide, content, headings } of documents)
for (const [ordinal, heading] of headings.entries())
sections.push({
for (const { guide, content, headings } of documents) {
const guideSections: GuideSection[] = [];
for (const [ordinal, heading] of headings.entries()) {
// The next heading of any level ends this section's own text: it is
// either the first subsection or the end of the whole section.
const ownEnd = headings[ordinal + 1]?.start ?? heading.end;
const section = {
guide,
ordinal,
heading: heading.title,
content: content.slice(heading.start, heading.end).trim(),
});
ownContent: content.slice(heading.start, ownEnd).trim(),
parent:
heading.parent === undefined
? undefined
: guideSections[heading.parent],
};
guideSections.push(section);
sections.push(section);
}
}
return sections;
};
@@ -1,6 +1,23 @@
import { describe, expect, it } from "vitest";
import { requireAuthenticationHeadroom } from "./JsonAuthenticationHeadroom.js";
import {
estimateJsonStorage,
requireAuthenticationHeadroom,
} from "./JsonAuthenticationHeadroom.js";
import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js";
it("stops a bounded representation scan before visiting an over-budget tail", () => {
const value = {
first: "x".repeat(1_024),
get tail() {
throw new Error("The rejected representation must not read its tail");
},
};
const measured = estimateJsonStorage(value, {
maximumValueBytes: 512,
includeSerialized: false,
});
expect(measured.valueBytes).toBeGreaterThan(512);
});
describe("JSON authentication headroom", () => {
it("does not queue every row while deciding whether a wide report can fit", () => {
let pending = 0;
+17 -5
View File
@@ -6,14 +6,24 @@ import { err, ok, type Result } from "../domain/result.js";
/** Authentication clones parsed JSON and hashes its canonical commitment. */
const AUTHENTICATION_COPIES = 2;
/** Parsed working copies and conservative UTF-16 storage for escaped JSON. */
const estimateJsonStorage = (
/**
* Parsed storage and optional UTF-16 serialization work. A bounded value scan
* returns only a lower bound after exceeding maximumValueBytes.
*/
export const estimateJsonStorage = (
value: unknown,
options: {
readonly maximumValueBytes?: number;
readonly includeSerialized?: boolean;
} = {},
): { valueBytes: number; serializedUtf16Bytes: number } => {
const maximum = options.maximumValueBytes ?? Number.POSITIVE_INFINITY;
const serialized = options.includeSerialized !== false;
let valueBytes = 0;
let serializedCharacters = 0;
const stack: Iterator<unknown>[] = [[value][Symbol.iterator]()];
while (stack.length > 0) {
if (valueBytes > maximum) break;
const next = stack[stack.length - 1]!.next();
if (next.done) {
stack.pop();
@@ -22,7 +32,8 @@ const estimateJsonStorage = (
const item = next.value;
if (typeof item === "string") {
valueBytes += 64 + item.length * 2;
serializedCharacters += jsonStringCodeUnits(item);
if (serialized && valueBytes <= maximum)
serializedCharacters += jsonStringCodeUnits(item);
} else if (Array.isArray(item)) {
valueBytes += 128 + item.length * 16;
serializedCharacters += 2 + Math.max(0, item.length - 1);
@@ -36,8 +47,9 @@ const estimateJsonStorage = (
for (const key in item) {
if (!Object.hasOwn(item, key)) continue;
valueBytes += 64 + key.length * 2;
serializedCharacters +=
jsonStringCodeUnits(key) + 1 + (count++ === 0 ? 0 : 1);
if (serialized && valueBytes <= maximum)
serializedCharacters +=
jsonStringCodeUnits(key) + 1 + (count++ === 0 ? 0 : 1);
yield (item as Record<string, unknown>)[key];
}
})(),
+4
View File
@@ -25,6 +25,7 @@ import {
defaultKunaCacheDirectory,
sameLexicalDirectory,
} from "../config/kunaCacheDirectory.js";
import { defaultDarwinHelperCacheDirectory } from "../process/DarwinHelperCache.js";
import { linuxHopperReleaseCacheDirectory } from "./LinuxHopperCacheDirectory.js";
import { ClientConfigurationPublication } from "./ClientConfigurationPublication.js";
@@ -1163,6 +1164,9 @@ const purgeSelectedData = async (
),
);
}
// The ownership helper cache has no caller-selected location to retain.
if (platform === "darwin")
directories.push(defaultDarwinHelperCacheDirectory(home));
if (platform === "linux") {
const hopper = linuxHopperReleaseCacheDirectory(environment, home);
if (hopper.ok) directories.push(hopper.value);
+2 -5
View File
@@ -1,3 +1,4 @@
import { WEB_MODULE_TRACE_PROVIDER } from "../domain/webTraceProviders.js";
import { closeOwnedWebResources } from "./closeOwnedWebResources.js";
import { isDeepStrictEqual } from "node:util";
import { z } from "zod";
@@ -273,11 +274,7 @@ export class WebModuleTraceService {
sha256: data.sourceFile.sha256,
format: "file",
},
{
id: "rea-web-module-trace",
name: "REA captured module relationship trace",
version: "1",
},
WEB_MODULE_TRACE_PROVIDER,
{
operation: OPERATION,
parameters: input,
+2 -5
View File
@@ -1,3 +1,4 @@
import { WEB_SOURCE_LOCATION_PROVIDER } from "../domain/webTraceProviders.js";
import { closeOwnedWebResources } from "./closeOwnedWebResources.js";
import { createHash } from "node:crypto";
import { z } from "zod";
@@ -209,11 +210,7 @@ export class WebSourceLocationService {
sha256: data.sourceFile.sha256,
format: "file",
},
{
id: "rea-web-source-location",
name: "REA captured source-map point trace",
version: "1",
},
WEB_SOURCE_LOCATION_PROVIDER,
{
operation: OPERATION,
parameters: input,
+9 -5
View File
@@ -13,9 +13,13 @@ export const workflowAnalysisProfile = (
createAnalysisProfile(REA_WORKFLOW_PROVIDER, {
upstream_analysis_profile: upstream,
source_limitations: "provider-operation-caveats-v1",
...(operation === "binary_overview" || operation === "batch_decompile"
? { workflow_revision: 4 }
: operation === "get_call_graph" || operation === "trace_feature"
? { workflow_revision: 3 }
: { workflow_revision: 1 }),
...(operation === "binary_overview"
? { workflow_revision: 5 }
: operation === "batch_decompile"
? { workflow_revision: 4 }
: operation === "trace_feature"
? { workflow_revision: 4 }
: operation === "get_call_graph"
? { workflow_revision: 3 }
: { workflow_revision: 1 }),
});
@@ -225,6 +225,7 @@ const projectResolvedAnalysisView = (
const projected = projection.project({
evidenceId: parent.evidence_id,
operation: parent.operation,
predicateType: parent.predicate_type,
provider: parent.provider,
analysisProfile: parent.analysis_profile,
authority: parent.authority,
@@ -436,6 +436,7 @@ describe("browser scenario network observation", () => {
const project = (capture: unknown) => {
const result = projectBrowserScenarioView(
{
predicateType: "rea.browser-scenario-capture",
evidenceId: `ev:${"a".repeat(64)}`,
operation: "capture_browser_scenario",
normalizedResult: capture,
@@ -302,6 +302,7 @@ it("rejects collection-inapplicable fields and malformed sibling facts even for
).toBe(false);
const report = source();
const parent = {
predicateType: "rea.analysis",
evidenceId: "ev_" + "b".repeat(64),
operation: "inspect_native_load_image",
normalizedResult: report,
@@ -1248,6 +1248,11 @@ describe("executions that report truncated references", () => {
it("revises the trace_feature workflow profile so earlier results are not replayed", () => {
const profile = ANALYSIS_SNAPSHOT_PROFILE;
expect(
workflowAnalysisProfile(profile, "trace_feature").parameters
.workflow_revision,
).toBe(4);
expect(workflowAnalysisProfile(profile, "trace_feature")).not.toEqual(
workflowAnalysisProfile(profile),
);
+32 -21
View File
@@ -8,7 +8,6 @@ import {
} from "../../domain/evidenceErrors.js";
import { NoBinaryOpenError } from "../../domain/configurationErrors.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import {
rebindEvidenceSubjectTargetPath,
type Evidence,
@@ -21,8 +20,8 @@ import {
createAnalysisSnapshotEntry,
createAnalysisSnapshotWorkflowEntry,
snapshotBinding,
snapshotBindingMismatch,
snapshotMatchesProfile,
snapshotMatchesTarget,
snapshotTarget,
parseAnalysisSnapshot,
immutableAnalysisSnapshot,
@@ -31,6 +30,7 @@ import {
type AnalysisSnapshotWorkflowEntry,
type AnalysisSnapshotBinding,
type AnalysisSnapshotTarget,
type SnapshotBindingMismatch,
} from "../../domain/analysisSnapshot.js";
import type {
AnalysisExecution,
@@ -159,13 +159,22 @@ export class AnalysisSnapshotCache {
target: BinaryTarget,
profile: AnalysisProfileCommitment | undefined,
): boolean {
return (
(this.#target === undefined && this.#binding === undefined) ||
(this.#target !== undefined &&
this.#binding !== undefined &&
profile !== undefined &&
snapshotMatchesTarget(this.#target, target) &&
snapshotMatchesProfile(this.#binding, profile))
return this.mismatch(target, profile) === undefined;
}
/** Explain staged mismatch without treating missing binding facts as absent. */
mismatch(
target: BinaryTarget,
profile: AnalysisProfileCommitment | undefined,
): SnapshotBindingMismatch | undefined {
if (this.#target === undefined && this.#binding === undefined)
return undefined;
if (this.#target === undefined || this.#binding === undefined)
return { staged_binding: "incomplete" };
return snapshotBindingMismatch(
{ target: this.#target, binding: this.#binding },
target,
profile,
);
}
@@ -377,9 +386,7 @@ export class AnalysisSnapshotCache {
readonly profile: AnalysisProfileCommitment;
}
| undefined,
mergeEvidence: (
bundle: EvidenceBundle,
) => Result<number, EvidenceIntegrityError | AnalysisInputError>,
mergeEvidence: (bundle: EvidenceBundle) => Result<number, AnalysisError>,
): Result<number, AnalysisError> {
let validated: AnalysisSnapshot;
try {
@@ -391,16 +398,20 @@ export class AnalysisSnapshotCache {
}),
);
}
if (
active !== undefined &&
(!snapshotMatchesTarget(validated.target, active.target) ||
!snapshotMatchesProfile(validated.binding, active.profile))
)
return err(
new AnalysisSnapshotMismatchError(
"Analysis snapshot profile_mismatch: target, provider, or analysis profile does not match the active binary",
),
if (active !== undefined) {
const mismatch = snapshotBindingMismatch(
validated,
active.target,
active.profile,
);
if (mismatch !== undefined)
return err(
new AnalysisSnapshotMismatchError(
"Analysis snapshot target, provider, or analysis profile does not match the active binary",
mismatch,
),
);
}
const importedEvidence = mergeEvidence(validated.evidence_bundle);
return importedEvidence.ok ? ok(this.stage(validated)) : importedEvidence;
}
+155 -25
View File
@@ -1,4 +1,9 @@
import { randomUUID } from "node:crypto";
import {
writeEvidenceBundle,
evidenceBundleExportReceipt,
} from "../EvidenceBundleFiles.js";
import type { ResetInvestigationReceipt } from "../../domain/investigationLifecycle.js";
import { retainSavedAnalysisSnapshot } from "../../domain/savedAnalysisSnapshot.js";
@@ -60,6 +65,7 @@ import {
} from "./BinarySessionExecution.js";
import { closeAnalysisClient } from "../AnalysisClientCleanup.js";
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
import type { AnalysisCleanupRetry } from "../../domain/analysisErrorRecovery.js";
import { analysisErrorWithCancellation } from "../../domain/analysisErrorCancellation.js";
import { ProviderAdapterError } from "../../domain/providerAdapterError.js";
import { profilelessNativeDispatchProfile } from "./ProfilelessNativeWorkflowSnapshot.js";
@@ -310,11 +316,7 @@ export class BinarySession
options.signal,
"open_bundle_resources",
);
const closed = await this.#closeActive(
{},
false,
cleanupFailures.onFailure,
);
const closed = await this.#closeActive({}, cleanupFailures.onFailure);
if (!closed.ok) {
this.#pendingBundleResourceCleanup.push(candidate);
const candidateClosed = await this.#retryPendingBundleResourceCleanup(
@@ -457,8 +459,8 @@ export class BinarySession
current: this.#active,
target,
options,
stagedSnapshotMatches: (target, profile) =>
this.matchesSnapshot(target, profile),
stagedSnapshotMismatch: (target, profile) =>
this.snapshotMismatch(target, profile),
}).then((resolved) =>
resolved.ok && isAborted(options.signal)
? err(
@@ -486,8 +488,8 @@ export class BinarySession
current: this.#active,
path,
options: admitted,
stagedSnapshotMatches: (target, profile) =>
this.matchesSnapshot(target, profile),
stagedSnapshotMismatch: (target, profile) =>
this.snapshotMismatch(target, profile),
}),
options,
);
@@ -505,8 +507,8 @@ export class BinarySession
...resolved,
current: this.#active,
options,
stagedSnapshotMatches: (target, profile) =>
this.matchesSnapshot(target, profile),
stagedSnapshotMismatch: (target, profile) =>
this.snapshotMismatch(target, profile),
}),
),
options,
@@ -811,7 +813,7 @@ export class BinarySession
this.#externalMutationCleanupFailure =
!value.ok && value.error.cleanupIncomplete ? value.error : undefined;
if (retireTarget) {
const retired = await this.#closeActive({}, false);
const retired = await this.#closeActive({});
if (!retired.ok) {
const primary = completionFailure(value, retired.error);
const failure =
@@ -863,15 +865,19 @@ export class BinarySession
return err(new AnalysisCancelledError(operation));
}
/** Close the active target, if any. */
/** Release the engine and replay state while retaining investigation history. */
close(
options: Pick<ExecutionOptions, "progress"> & {
readonly retainProviderDocuments?: boolean;
readonly cleanupRetry?: AnalysisCleanupRetry;
} = {},
): Promise<Result<null, AnalysisError>> {
return this.#serialize(async (admittedThrough) => {
await this.#drainCalls(admittedThrough);
return this.#closeActive(options);
return this.#offerCleanupRetry(
await this.#closeActive(options),
options.cleanupRetry,
);
});
}
@@ -879,7 +885,9 @@ export class BinarySession
closeWithSnapshot(
path: string,
overwrite: boolean,
options: Pick<ExecutionOptions, "progress"> = {},
options: Pick<ExecutionOptions, "progress"> & {
readonly cleanupRetry?: AnalysisCleanupRetry;
} = {},
): Promise<Result<SavedAnalysisSnapshot, AnalysisError>> {
return this.#serialize(async (admittedThrough) => {
await this.#drainCalls(admittedThrough);
@@ -897,18 +905,137 @@ export class BinarySession
workflow_entries: snapshot.value.workflow_entries.length,
evidence_records: snapshot.value.evidence_bundle.records.length,
};
const closed = await this.#closeActive(options);
// A failed export or save returned above with the target still open, so
// only a failed retirement can offer the caller's cleanup retry.
const closed = this.#offerCleanupRetry(
await this.#closeActive(options),
options.cleanupRetry,
);
return closed.ok
? ok(receipt)
: err(retainSavedAnalysisSnapshot(closed.error, receipt));
});
}
/** Explicit whole-runtime disposal, separate from reusable engine cleanup. */
dispose(
options: Pick<ExecutionOptions, "progress"> & {
readonly retainProviderDocuments?: boolean;
} = {},
): Promise<Result<null, AnalysisError>> {
this.beginShutdown();
return this.#serialize(async (admittedThrough) => {
await this.#drainCalls(admittedThrough);
const closed = await this.#closeActive(options);
if (closed.ok) this.disposeInvestigationRecords();
return closed;
});
}
/** Save once at an explicit boundary, then end the investigation atomically. */
resetInvestigation(
options: {
readonly savePath?: string;
readonly overwrite?: boolean;
readonly signal?: AbortSignal;
readonly retryCleanup?: () => Promise<Result<null, AnalysisError>>;
} = {},
): Promise<Result<ResetInvestigationReceipt, AnalysisError>> {
if (this.#shutdownStarted)
return Promise.resolve(
err(new AnalysisCancelledError("reset_investigation")),
);
return this.#serialize(async (admittedThrough) => {
await this.#drainCalls(admittedThrough);
const cancelled = () =>
new AnalysisCancelledError("reset_investigation", {
cause: options.signal?.reason,
});
if (isAborted(options.signal)) return err(cancelled());
const adapterResources = await options.retryCleanup?.();
if (adapterResources !== undefined && !adapterResources.ok)
return adapterResources;
if (this.#externalMutationCleanupFailure?.cleanupIncomplete === true)
return err(this.#externalMutationCleanupFailure);
const pending = await this.#retryPendingCleanup();
if (!pending.ok) return pending;
const bundles = await this.#retryPendingBundleResourceCleanup();
if (!bundles.ok) return bundles;
const files = await this.closeFileResources();
if (!files.ok) return files;
if (this.#providerRouter.hasPendingOwnedResources()) {
const candidates = await this.#providerRouter.closeOwnedResources();
if (!candidates.ok) return candidates;
}
if (isAborted(options.signal)) return err(cancelled());
const statistics = this.investigationStatistics();
let saved: ResetInvestigationReceipt["saved_bundle"];
if (options.savePath !== undefined) {
const bundle = this.immutableEvidenceBundle();
const written = await writeEvidenceBundle(
bundle,
options.savePath,
options.overwrite ?? false,
this.fileResources,
options.signal,
"reset_investigation",
);
if (!written.ok) return written;
saved = evidenceBundleExportReceipt(bundle, written.value);
}
// Publication can win cancellation. Report the committed file but retain
// the investigation if cancellation arrived during publication/cleanup.
if (isAborted(options.signal))
return err(
new AnalysisCancelledError("reset_investigation", {
cause: options.signal?.reason,
...(saved === undefined
? {}
: { diagnostics: { saved_bundle: saved } }),
}),
);
this.clearInvestigationRecords();
return ok({
cleared_evidence_records: statistics.evidence_records,
cleared_unknown_revisions: statistics.unknown_revisions,
cleared_unknown_heads: statistics.unknown_heads,
retained_target:
this.#active !== undefined ||
this.#activeBundleResources !== undefined,
...(saved === undefined ? {} : { saved_bundle: saved }),
});
});
}
/** Offer the caller's retry only while this session retains the failed cleanup. */
#offerCleanupRetry(
closed: Result<null, AnalysisError>,
retry: AnalysisCleanupRetry | undefined,
): Result<null, AnalysisError> {
if (
!closed.ok &&
retry !== undefined &&
closed.error.cleanupIncomplete &&
this.#retainsCloseCleanup()
)
closed.error.bindCleanupRetry(retry);
return closed;
}
/** Owners the next close retries; external mutation owners retry elsewhere. */
#retainsCloseCleanup(): boolean {
return (
this.#pendingCleanup !== undefined ||
this.#pendingBundleResourceCleanup.length > 0 ||
this.hasPendingFileResources ||
this.#providerRouter.hasPendingOwnedResources()
);
}
async #closeActive(
options: Pick<ExecutionOptions, "progress"> & {
readonly retainProviderDocuments?: boolean;
},
clearRecords = true,
onCleanupFailure?: (failure: AnalysisError) => void,
): Promise<Result<null, AnalysisError>> {
const previous = this.#active;
@@ -917,14 +1044,14 @@ export class BinarySession
this.#activeBundleResources = undefined;
if (previousResources !== undefined)
this.#pendingBundleResourceCleanup.push(previousResources);
const closed =
const closing =
previous === undefined
? await this.#retryPendingCleanup(
? this.#retryPendingCleanup(
options.progress,
options.retainProviderDocuments,
onCleanupFailure,
)
: await this.#retireClient(previous, {
: this.#retireClient(previous, {
...(options.progress === undefined
? {}
: { progress: options.progress }),
@@ -933,8 +1060,11 @@ export class BinarySession
: {}),
...(onCleanupFailure === undefined ? {} : { onCleanupFailure }),
});
if (clearRecords) this.clearSessionRecords();
// Register the retiring resource owner before notifying observers that
// the engine/cache is gone. Cleanup is still pending at this boundary.
this.clearEngineRecords();
this.#clearRuntimeAvailability();
const closed = await closing;
const fileResources = await this.closeFileResources();
if (!fileResources.ok) onCleanupFailure?.(fileResources.error);
const candidateResources = await this.#providerRouter.closeOwnedResources();
@@ -1112,6 +1242,7 @@ export class BinarySession
const client = run?.client;
return binarySessionStatus({
...options,
investigation: this.investigationStatistics(),
target: this.#active?.target,
...(resourceRun === undefined
? {}
@@ -1128,10 +1259,7 @@ export class BinarySession
providerOperationHealth: client?.operationHealthSnapshot?.() ?? null,
cleanupPending:
this.#externalMutationCleanupFailure?.cleanupIncomplete === true ||
this.#pendingCleanup !== undefined ||
this.#pendingBundleResourceCleanup.length > 0 ||
this.hasPendingFileResources ||
this.#providerRouter.hasPendingOwnedResources(),
this.#retainsCloseCleanup(),
});
}
@@ -1217,6 +1345,8 @@ export class BinarySession
signal: AbortSignal | undefined,
operation: (analysis: AnalysisOperationPort) => Promise<Value>,
): Promise<Result<Value, AnalysisError>> {
if (this.#shutdownStarted)
return Promise.resolve(err(new AnalysisCancelledError(operationName)));
const generation = this.#transitionGeneration;
const transition = this.#transition;
const call = (async (): Promise<Result<Value, AnalysisError>> => {
@@ -0,0 +1,570 @@
import { access, mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { expect, it, vi } from "vitest";
import { createEvidence } from "../../domain/evidence.js";
import {
createEvidenceBundle,
parseEvidenceBundle,
} from "../../domain/evidenceBundle.js";
import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { err, ok } from "../../domain/result.js";
import type { RecordUnknownInput } from "../../domain/residualUnknown.js";
import { OwnedStagedOutput } from "../../filesystem/OwnedStagedOutput.js";
import type { AnalysisProviderCandidate } from "../AnalysisProvider.js";
import { AnalysisProviderRegistry } from "./AnalysisProviderRegistry.js";
import { BinarySession } from "./BinarySession.js";
import { SessionProviderRouter } from "./SessionProviderRouter.js";
import {
ControllableAnalysisClient,
createBinarySessionTargets,
createCacheProvider,
createDeferred,
createTestBinarySession,
} from "../../../tests/fixtures/binarySession.js";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { awaitAcknowledgement } from "../../../tests/fixtures/lifecycleAcknowledgement.js";
const observation = (label: string) =>
createEvidence(
undefined,
{ id: "fixture", name: "Lifecycle fixture", version: "1" },
{ operation: label, parameters: {}, result: { fixture: label } },
);
const unknownRequirements: Omit<RecordUnknownInput, "question" | "domain"> = {
severity: "medium",
supporting_evidence_ids: [],
contradicting_evidence_ids: [],
required_authority: "shipped-artifact",
required_confidence: "observed",
required_environment: null,
recommended_probes: [],
relationships: [],
};
const unknownInput: RecordUnknownInput = {
...unknownRequirements,
question: "Which source revision produced this lifecycle fixture?",
domain: "provenance",
};
const seed = (session: ReturnType<typeof createTestBinarySession>) => {
expect(session.recordEvidence(observation("retained_fixture")).ok).toBe(true);
const recorded = session.recordUnknown(unknownInput);
if (!recorded.ok) throw recorded.error;
expect(
session.updateUnknown({
...unknownRequirements,
unknown_id: recorded.value.unknown_id,
expected_revision: 1,
status: "investigating",
resolution: null,
}).ok,
).toBe(true);
return session.exportEvidenceBundle();
};
it("publishes engine retirement with its pending cleanup owner already visible", async () => {
const [target] = await createBinarySessionTargets();
const entered = createDeferred<void>();
const release = createDeferred<void>();
const provider = createCacheProvider([]);
const session = createTestBinarySession({
...provider,
createClient: (...arguments_) => ({
...provider.createClient(...arguments_),
close: async () => {
entered.resolve();
await release.promise;
return ok(null);
},
}),
});
const notifications: ReturnType<typeof session.status>[] = [];
const pending: Promise<unknown>[] = [];
let unsubscribe: () => void = () => undefined;
try {
expect((await session.open(target)).ok).toBe(true);
expect(
(
await session.execute("address_name", {
address: "0x1000",
document: "fixture",
})
).ok,
).toBe(true);
unsubscribe = session.onAnalysisSnapshotChanged(() => {
notifications.push(session.status());
});
const closing = session.close();
pending.push(closing);
await awaitAcknowledgement(entered.promise, closing, "engine close entry");
expect(notifications.length).toBeGreaterThan(0);
for (const status of await Promise.all(notifications))
expect(status).toMatchObject({ open: false, cleanup_pending: true });
release.resolve();
expect((await closing).ok).toBe(true);
expect(await session.status()).toMatchObject({
open: false,
cleanup_pending: false,
});
} finally {
unsubscribe();
release.resolve();
await Promise.allSettled(pending);
await session.close();
}
});
it("preserves metadata invalidation across reset until the mutated engine is retired", async () => {
const [target] = await createBinarySessionTargets();
const calls: string[] = [];
const base = createCacheProvider(calls);
const createClient = vi.fn(base.createClient);
const session = createTestBinarySession({ ...base, createClient });
const parameters = { address: "0x1000", document: "fixture" };
try {
expect((await session.open(target)).ok).toBe(true);
expect((await session.execute("address_name", parameters)).ok).toBe(true);
const replay = session.exportAnalysisSnapshot();
if (!replay.ok) throw replay.error;
expect(replay.value.entries).toHaveLength(1);
// The fixture declares this operation as a metadata mutation. No real
// provider or target bytes are changed by the simulated observation.
expect(
(
await session.execute("set_address_name", {
...parameters,
name: "renamed",
})
).ok,
).toBe(true);
expect(session.exportAnalysisSnapshot()).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
const before = session.exportEvidenceBundle();
expect(await session.resetInvestigation()).toEqual({
ok: true,
value: {
cleared_evidence_records: before.records.length,
cleared_unknown_revisions: 0,
cleared_unknown_heads: 0,
retained_target: true,
},
});
expect(session.activeTarget()?.path).toBe(target);
expect(createClient).toHaveBeenCalledTimes(1);
expect(session.exportAnalysisSnapshot()).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
expect(session.importAnalysisSnapshot(replay.value)).toMatchObject({
ok: false,
error: { _tag: "EvidenceIntegrityError" },
});
expect((await session.execute("address_name", parameters)).ok).toBe(true);
expect((await session.execute("address_name", parameters)).ok).toBe(true);
expect(
calls.filter((operation) => operation === "address_name"),
).toHaveLength(3);
expect((await session.close()).ok).toBe(true);
expect((await session.open(target)).ok).toBe(true);
expect(session.exportAnalysisSnapshot().ok).toBe(true);
} finally {
await session.dispose();
}
});
it("preserves records through failed disposal and clears them only after its owned cleanup succeeds", async () => {
const [target] = await createBinarySessionTargets();
const client = new ControllableAnalysisClient();
const realClose = client.close.bind(client);
const failure = new ProviderCleanupError(
"fixture",
["fixture-owned-engine"],
{ reason: "Controlled disposal cleanup refusal" },
);
let allowed = false;
const close = vi
.spyOn(client, "close")
.mockImplementation(() =>
allowed ? realClose() : Promise.resolve(err(failure)),
);
const createClient = vi.fn(() => client);
const session = createTestBinarySession(createClient);
try {
expect((await session.open(target)).ok).toBe(true);
expect(
(
await session.execute("address_name", {
address: "0x1000",
document: "fixture",
})
).ok,
).toBe(true);
const before = seed(session);
for (let attempt = 0; attempt < 2; attempt += 1) {
expect(await session.dispose()).toMatchObject({
ok: false,
error: { cleanupIncomplete: true },
});
expect(session.exportEvidenceBundle()).toEqual(before);
expect(session.listUnknowns()[0]?.revision).toBe(2);
expect(session.activeTarget()).toBeUndefined();
}
expect((await session.open(target)).ok).toBe(false);
expect(createClient).toHaveBeenCalledTimes(1);
allowed = true;
expect(await session.dispose()).toEqual(ok(null));
expect(session.exportEvidenceBundle()).toEqual(createEvidenceBundle([]));
expect(session.listUnknowns()).toEqual([]);
const closeCalls = close.mock.calls.length;
expect(await session.dispose()).toEqual(ok(null));
expect(close).toHaveBeenCalledTimes(closeCalls);
} finally {
allowed = true;
close.mockRestore();
await session.dispose();
}
});
it.each(["retired", "healthy"] as const)(
"refuses reset while separate candidate cleanup is pending with a %s engine",
async (engineState) => {
const [target] = await createBinarySessionTargets();
const root = await createTestTempDirectory("rea-reset-candidate-cleanup-");
const ownedRoot = join(root, "candidate-owned-inspection");
const checkpoint = join(root, "checkpoint.json");
const originalCheckpoint = Buffer.from("caller-owned checkpoint");
await writeFile(checkpoint, originalCheckpoint);
const providerCalls: string[] = [];
const provider = createCacheProvider(providerCalls);
let pending = false;
let cleanupAllowed = false;
const acquireCandidateOwner = async () => {
await mkdir(ownedRoot);
await writeFile(join(ownedRoot, "observation"), "candidate-owned facts");
pending = true;
};
const failure = new ProviderCleanupError("fixture", [ownedRoot], {
reason: "Controlled registry-owned inspection cleanup refusal",
});
const closeClient = vi.fn(async () => {
if (engineState === "retired") await acquireCandidateOwner();
return ok(null);
});
const createClient = vi.fn<typeof provider.createClient>(
(...arguments_) => ({
...provider.createClient(...arguments_),
close: closeClient,
}),
);
const closeOwnedResources = vi.fn(async () => {
if (!pending) return ok(null);
if (!cleanupAllowed) return err(failure);
await rm(ownedRoot, { recursive: true, force: true });
pending = false;
return ok(null);
});
const candidate: AnalysisProviderCandidate = {
identity: provider.identity,
capabilities: provider.capabilities,
createClient,
resolveAnalysisProfile: (selected, options) => {
if (provider.resolveAnalysisProfile === undefined)
throw new Error("The fixture requires its declared profile resolver");
return provider.resolveAnalysisProfile(selected, options);
},
inspectAvailability: () => ({
status: "available",
code: null,
reason: null,
diagnostics: {},
}),
inspectTargetSupport: () => ({
status: "supported",
code: null,
reason: null,
diagnostics: {},
}),
hasPendingOwnedResources: () => pending,
closeOwnedResources,
};
// The real registry owns this candidate predicate and cleanup callback;
// only the candidate's resource failure and native execution are simulated.
const registry = new AnalysisProviderRegistry([candidate]);
const session = new BinarySession(
SessionProviderRouter.selectable(registry, []),
);
const parameters = { address: "0x1000", document: "fixture" };
try {
expect((await session.open(target)).ok).toBe(true);
expect((await session.execute("address_name", parameters)).ok).toBe(true);
expect(createClient).toHaveBeenCalledTimes(1);
expect(registry.hasPendingOwnedResources()).toBe(false);
if (engineState === "healthy") {
const callsBefore = closeOwnedResources.mock.calls.length;
expect((await session.resetInvestigation()).ok).toBe(true);
expect(closeOwnedResources).toHaveBeenCalledTimes(callsBefore);
expect(closeClient).not.toHaveBeenCalled();
}
const before = seed(session);
if (engineState === "retired") {
expect(await session.close()).toEqual(err(failure));
expect(closeClient).toHaveBeenCalledTimes(1);
expect(session.activeTarget()).toBeUndefined();
} else {
await acquireCandidateOwner();
expect(session.activeTarget()?.path).toBe(target);
}
expect(registry.hasPendingOwnedResources()).toBe(true);
const cleanupCallsBefore = closeOwnedResources.mock.calls.length;
expect(
await session.resetInvestigation({
savePath: checkpoint,
overwrite: true,
}),
).toEqual(err(failure));
expect(closeOwnedResources).toHaveBeenCalledTimes(cleanupCallsBefore + 1);
expect((await readFile(checkpoint)).equals(originalCheckpoint)).toBe(
true,
);
expect(await readFile(join(ownedRoot, "observation"), "utf8")).toBe(
"candidate-owned facts",
);
expect(session.exportEvidenceBundle()).toEqual(before);
expect(session.listUnknowns()[0]?.revision).toBe(2);
expect(registry.hasPendingOwnedResources()).toBe(true);
expect(closeClient).toHaveBeenCalledTimes(
engineState === "retired" ? 1 : 0,
);
cleanupAllowed = true;
const reset = await session.resetInvestigation({
savePath: checkpoint,
overwrite: true,
});
expect(reset).toMatchObject({
ok: true,
value: {
cleared_evidence_records: before.records.length,
cleared_unknown_revisions: before.unknowns.length,
cleared_unknown_heads: 1,
retained_target: engineState === "healthy",
},
});
expect(closeOwnedResources).toHaveBeenCalledTimes(cleanupCallsBefore + 2);
expect(registry.hasPendingOwnedResources()).toBe(false);
await expect(access(ownedRoot)).rejects.toMatchObject({ code: "ENOENT" });
expect(
parseEvidenceBundle(JSON.parse(await readFile(checkpoint, "utf8"))),
).toEqual(before);
expect(session.exportEvidenceBundle()).toEqual(createEvidenceBundle([]));
expect(createClient).toHaveBeenCalledTimes(1);
expect(closeClient).toHaveBeenCalledTimes(
engineState === "retired" ? 1 : 0,
);
if (engineState === "healthy") {
expect(session.activeTarget()?.path).toBe(target);
expect((await session.execute("address_name", parameters)).ok).toBe(
true,
);
expect(createClient).toHaveBeenCalledTimes(1);
expect(
providerCalls.filter((operation) => operation === "address_name"),
).toHaveLength(2);
}
} finally {
cleanupAllowed = true;
await session.dispose();
}
},
);
it("drains prior metadata writes and refuses later admissions and direct writes after disposal", async () => {
const [target] = await createBinarySessionTargets();
const calls: string[] = [];
const session = createTestBinarySession(createCacheProvider(calls));
const entered = createDeferred<void>();
const release = createDeferred<void>();
let committedBeforeClear = false;
const pending: Promise<unknown>[] = [];
try {
expect((await session.open(target)).ok).toBe(true);
expect(
(
await session.execute("address_name", {
address: "0x1000",
document: "fixture",
})
).ok,
).toBe(true);
const snapshot = session.exportAnalysisSnapshot();
if (!snapshot.ok) throw snapshot.error;
const before = session.withAdmittedAnalysis(
"before_dispose",
undefined,
async () => {
entered.resolve();
await release.promise;
const record = observation("admitted_before_disposal");
const written = session.recordEvidence(record);
committedBeforeClear =
written.ok && session.hasEvidence(record.evidence_id);
return written;
},
);
pending.push(before);
await awaitAcknowledgement(entered.promise, before, "metadata write entry");
const disposing = session.dispose();
pending.push(disposing);
const late = vi.fn(async () =>
session.importEvidenceBundle(
createEvidenceBundle([observation("late_import")]),
),
);
const after = session.withAdmittedAnalysis(
"import_evidence_bundle",
undefined,
late,
);
pending.push(after);
release.resolve();
expect((await before).ok).toBe(true);
expect(committedBeforeClear).toBe(true);
expect(await disposing).toEqual(ok(null));
expect((await after).ok).toBe(false);
expect(late).not.toHaveBeenCalled();
const afterSettled = vi.fn(async () => session.recordUnknown(unknownInput));
expect(
(
await session.withAdmittedAnalysis(
"record_unknown",
undefined,
afterSettled,
)
).ok,
).toBe(false);
expect(afterSettled).not.toHaveBeenCalled();
expect(session.recordEvidence(observation("after_dispose")).ok).toBe(false);
expect(
session.importEvidenceBundle(
createEvidenceBundle([observation("after_dispose_import")]),
).ok,
).toBe(false);
expect(session.importAnalysisSnapshot(snapshot.value).ok).toBe(false);
expect(session.recordUnknown(unknownInput).ok).toBe(false);
expect(
session.recordEvidenceWithUnknown(
observation("after_dispose_unknown"),
unknownInput,
).ok,
).toBe(false);
expect((await session.open(target)).ok).toBe(false);
expect((await session.openBundleResources(target)).ok).toBe(false);
expect(session.exportEvidenceBundle()).toEqual(createEvidenceBundle([]));
} finally {
release.resolve();
await Promise.allSettled(pending);
await session.dispose();
}
});
it.each(["preaborted", "queued"] as const)(
"preserves exact reset cancellation reason when %s",
async (phase) => {
const root = await createTestTempDirectory("rea-reset-cancel-reason-");
const session = createTestBinarySession(
() => new ControllableAnalysisClient(),
);
const before = seed(session);
const entered = createDeferred<void>();
const release = createDeferred<void>();
const reason = { caller: "explicit reset cancellation", phase };
const controller = new AbortController();
const path = join(root, "must-not-publish.json");
let blocker: Promise<unknown> | undefined;
try {
if (phase === "queued") {
blocker = session.withAdmittedAnalysis(
"before_reset",
undefined,
async () => {
entered.resolve();
await release.promise;
},
);
await awaitAcknowledgement(
entered.promise,
blocker,
"reset blocker entry",
);
} else controller.abort(reason);
const resetting = session.resetInvestigation({
savePath: path,
signal: controller.signal,
});
if (phase === "queued") controller.abort(reason);
release.resolve();
await blocker;
const result = await resetting;
if (result.ok) throw new Error("Cancelled reset reported success");
expect(result.error._tag).toBe("AnalysisCancelledError");
expect(result.error.cause).toBe(reason);
expect(session.exportEvidenceBundle()).toEqual(before);
await expect(access(path)).rejects.toMatchObject({ code: "ENOENT" });
} finally {
release.resolve();
await blocker;
await session.dispose();
}
},
);
it("retains exact cancellation reason and a complete checkpoint during final cleanup", async () => {
const root = await createTestTempDirectory("rea-reset-final-cancel-");
const path = join(root, "published.json");
const session = createTestBinarySession(
() => new ControllableAnalysisClient(),
);
const before = seed(session);
const reason = { caller: "cancel reset after actual staged-output cleanup" };
const controller = new AbortController();
const original = OwnedStagedOutput.prototype.close;
let interrupted = false;
const seam = vi
.spyOn(OwnedStagedOutput.prototype, "close")
.mockImplementation(async function (this: OwnedStagedOutput) {
await original.call(this);
if (dirname(this.directory) === root && !interrupted) {
interrupted = true;
controller.abort(reason);
}
});
try {
const result = await session.resetInvestigation({
savePath: path,
signal: controller.signal,
});
if (result.ok) throw new Error("Cancelled reset reported success");
expect(interrupted).toBe(true);
expect(result.error._tag).toBe("AnalysisCancelledError");
expect(result.error.cause).toBe(reason);
const saved = await readFile(path);
expect(parseEvidenceBundle(JSON.parse(saved.toString("utf8")))).toEqual(
before,
);
expect(result.error.diagnostics?.saved_bundle).toEqual({
path,
bytes: saved.length,
records: before.records.length,
unknowns: before.unknowns.length,
});
expect(session.exportEvidenceBundle()).toEqual(before);
expect((await session.resetInvestigation()).ok).toBe(true);
expect(session.exportEvidenceBundle()).toEqual(createEvidenceBundle([]));
} finally {
seam.mockRestore();
await session.dispose();
}
});
+35 -31
View File
@@ -8,9 +8,10 @@ import { artifactTargetSelectedPath } from "../../domain/binaryTargetTypes.js";
import type { AnalysisProviderSelector } from "../../domain/providerSelection.js";
import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js";
import {
snapshotMatchesBinding,
snapshotBindingMismatch,
snapshotMatchesTarget,
snapshotTarget,
type SnapshotBindingMismatch,
} from "../../domain/analysisSnapshot.js";
import type { AnalysisSnapshot } from "../../domain/analysisSnapshot.js";
import { parseBinaryTarget } from "../BinaryTargetResolver.js";
@@ -23,7 +24,7 @@ import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import { err, type Result } from "../../domain/result.js";
import type { SessionProviderRoute } from "./SessionProviderRouter.js";
import {
isProfilelessNativeDispatchSnapshot,
profilelessSnapshotMismatch,
profilelessNativeDispatchProfile,
} from "./ProfilelessNativeWorkflowSnapshot.js";
import { SessionProviderRouter } from "./SessionProviderRouter.js";
@@ -68,10 +69,10 @@ interface ResolveSessionOpenInput {
readonly current: CurrentOpenBinding | undefined;
readonly path: string;
readonly options: BinarySessionOpenOptions;
readonly stagedSnapshotMatches: (
readonly stagedSnapshotMismatch: (
target: BinaryTarget,
profile: AnalysisProfileCommitment | null,
) => boolean;
) => SnapshotBindingMismatch | undefined;
}
interface ResolveSessionTargetInput {
@@ -79,10 +80,10 @@ interface ResolveSessionTargetInput {
readonly current: CurrentOpenBinding | undefined;
readonly target: BinaryTarget;
readonly options: BinarySessionOpenOptions;
readonly stagedSnapshotMatches: (
readonly stagedSnapshotMismatch: (
target: BinaryTarget,
profile: AnalysisProfileCommitment | null,
) => boolean;
) => SnapshotBindingMismatch | undefined;
}
/** Parse a target, resolve its provider route, and validate snapshot binding. */
@@ -134,7 +135,7 @@ export const validateResolvedSessionTarget = (
readonly route: SessionProviderRoute;
},
): Result<ResolvedSessionOpen, AnalysisError> => {
const { current, target, route, options, stagedSnapshotMatches } = input;
const { current, target, route, options, stagedSnapshotMismatch } = input;
const sameTarget = sameSessionTarget(current?.target, target);
const snapshotError = validateSnapshot({
snapshot: options.snapshot,
@@ -142,7 +143,7 @@ export const validateResolvedSessionTarget = (
profile: route.profile,
profilelessWorkflowProfile: profilelessNativeDispatchProfile(route),
current,
stagedSnapshotMatches,
stagedSnapshotMismatch,
});
if (snapshotError !== undefined) return err(snapshotError);
return { ok: true, value: { target, route, sameTarget } };
@@ -166,10 +167,10 @@ interface ValidateSnapshotInput {
readonly profile: AnalysisProfileCommitment | null;
readonly profilelessWorkflowProfile: AnalysisProfileCommitment | undefined;
readonly current: CurrentOpenBinding | undefined;
readonly stagedSnapshotMatches: (
readonly stagedSnapshotMismatch: (
target: BinaryTarget,
profile: AnalysisProfileCommitment | null,
) => boolean;
) => SnapshotBindingMismatch | undefined;
}
const validateSnapshot = ({
@@ -178,31 +179,34 @@ const validateSnapshot = ({
profile,
profilelessWorkflowProfile,
current,
stagedSnapshotMatches,
stagedSnapshotMismatch,
}: ValidateSnapshotInput): EvidenceIntegrityError | undefined => {
if (
snapshot !== undefined &&
(profile === null
? profilelessWorkflowProfile === undefined ||
!isProfilelessNativeDispatchSnapshot(
snapshot,
if (snapshot !== undefined) {
const mismatch =
profile === null
? profilelessSnapshotMismatch(
snapshot,
target,
profilelessWorkflowProfile,
)
: snapshotBindingMismatch(snapshot, target, profile);
if (mismatch !== undefined)
return new AnalysisSnapshotMismatchError(
"Analysis snapshot target, provider, or analysis profile does not match the requested binary",
mismatch,
);
}
const stagedMismatch =
current === undefined
? stagedSnapshotMismatch(
target,
profilelessWorkflowProfile,
profile ?? profilelessWorkflowProfile ?? null,
)
: !snapshotMatchesBinding(snapshot, target, profile))
)
: undefined;
if (stagedMismatch !== undefined)
return new AnalysisSnapshotMismatchError(
"Analysis snapshot profile_mismatch: target, provider, or analysis profile does not match the requested binary",
);
if (
current === undefined &&
!stagedSnapshotMatches(
target,
profile ?? profilelessWorkflowProfile ?? null,
)
)
return new AnalysisSnapshotMismatchError(
"Analysis snapshot profile_mismatch: staged target, provider, or analysis profile does not match",
"Analysis snapshot does not match the staged target, provider, or analysis profile",
stagedMismatch,
);
return undefined;
};
+26 -1
View File
@@ -1,5 +1,6 @@
import type { EspFlashSelection } from "../../domain/firmware/espFlashLocation.js";
import type { InvestigationRecordPort } from "../investigation/InvestigationRecordPort.js";
import type { ResetInvestigationReceipt } from "../../domain/investigationLifecycle.js";
import type { ExecutableFormatHint } from "../../domain/dosCom.js";
import type {
BinaryArchitecture,
@@ -11,6 +12,7 @@ import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js"
import type { AnalysisProviderSelector } from "../../domain/providerSelection.js";
import type { BinarySessionDetail } from "../../contracts/sessionStatusContract.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { AnalysisCleanupRetry } from "../../domain/analysisErrorRecovery.js";
import type { Result } from "../../domain/result.js";
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { AnalysisSnapshotLoader } from "./BinarySessionOpen.js";
@@ -76,16 +78,39 @@ export interface BinarySessionPort
path: string,
options?: Pick<ExecutionOptions, "signal">,
): Promise<Result<ResolvedAppBundleResourceRoot, AnalysisError>>;
/**
* `cleanupRetry` names the caller's same-session retry; the session binds it
* to a failed close only while it retains that cleanup for the next close.
*/
close(
options?: Pick<ExecutionOptions, "progress"> & {
readonly retainProviderDocuments?: boolean;
readonly cleanupRetry?: AnalysisCleanupRetry;
},
): Promise<Result<null, AnalysisError>>;
/** Seal runtime acquisition and release investigation state after successful cleanup. */
dispose(
options?: Pick<ExecutionOptions, "progress"> & {
readonly retainProviderDocuments?: boolean;
},
): Promise<Result<null, AnalysisError>>;
/** Seal future acquisition/admission while previously admitted work drains. */
beginShutdown(): void;
/** Drain work, optionally checkpoint, and clear records without closing the active engine. */
resetInvestigation(options?: {
readonly savePath?: string;
readonly overwrite?: boolean;
readonly signal?: AbortSignal;
/** Adapter-owned cleanup joins the drained transaction before clearing records. */
readonly retryCleanup?: () => Promise<Result<null, AnalysisError>>;
}): Promise<Result<ResetInvestigationReceipt, AnalysisError>>;
/** Save a snapshot and close while excluding later provider requests. */
closeWithSnapshot(
path: string,
overwrite: boolean,
options?: Pick<ExecutionOptions, "progress">,
options?: Pick<ExecutionOptions, "progress"> & {
readonly cleanupRetry?: AnalysisCleanupRetry;
},
): Promise<Result<SavedAnalysisSnapshot, AnalysisError>>;
/** Default complete detail; summary omits provider diagnostics and candidate capabilities. */
status(options?: {
+61 -22
View File
@@ -11,9 +11,10 @@ import {
AnalysisSnapshotMismatchError,
} from "../../domain/evidenceErrors.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { InvestigationStatistics } from "../../domain/investigationLifecycle.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
type AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
import type { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import type {
@@ -46,7 +47,7 @@ import { ProviderCleanupError } from "../../domain/providerCleanupError.js";
import { workflowAnalysisProfile } from "../WorkflowAnalysisProfile.js";
import {
hasOnlyProfilelessNativeDispatchEntries,
isProfilelessNativeDispatchSnapshot,
profilelessSnapshotMismatch,
} from "./ProfilelessNativeWorkflowSnapshot.js";
const SNAPSHOT_MUTATION_RECOVERY =
@@ -102,6 +103,7 @@ export abstract class BinarySessionRecords {
}
readonly #snapshot = new AnalysisSnapshotCache();
#snapshotInvalidated = false;
#investigationDisposed = false;
readonly #snapshotListeners = new Set<() => void | Promise<void>>();
constructor(records: InvestigationRecords = new InvestigationRecords()) {
@@ -116,7 +118,9 @@ export abstract class BinarySessionRecords {
recordEvidence(
evidence: Evidence,
): Result<"added" | "duplicate", EvidenceIntegrityError> {
): Result<"added" | "duplicate", AnalysisError> {
if (this.#investigationDisposed)
return err(new AnalysisCancelledError("record_evidence"));
const recorded = this.#records.recordEvidence(evidence);
if (recorded.ok && recorded.value === "added") this.#emitSnapshotChanged();
return recorded;
@@ -183,10 +187,9 @@ export abstract class BinarySessionRecords {
importEvidenceBundle(
bundle: unknown,
): Result<
EvidenceImportReceipt,
EvidenceIntegrityError | AnalysisInputError
> {
): Result<EvidenceImportReceipt, AnalysisError> {
if (this.#investigationDisposed)
return err(new AnalysisCancelledError("import_evidence_bundle"));
const imported = this.#records.mergeEvidenceBundle(bundle);
if (!imported.ok) return imported;
this.#snapshot.refreshEvidenceMetadata(imported.value.replacements);
@@ -194,7 +197,7 @@ export abstract class BinarySessionRecords {
return ok({
recordsAdded: imported.value.recordsAdded,
unknownsAdded: imported.value.unknownsAdded,
totalRecords: this.#records.immutableEvidenceBundle().records.length,
totalRecords: this.#records.statistics().evidenceRecords,
});
}
@@ -251,6 +254,8 @@ export abstract class BinarySessionRecords {
importAnalysisSnapshot(
snapshot: AnalysisSnapshot,
): Result<number, AnalysisError> {
if (this.#investigationDisposed)
return err(new AnalysisCancelledError("import_analysis_snapshot"));
const active = this.activeAnalysisBinding();
if (active !== undefined && this.#snapshotInvalidated)
return err(
@@ -262,16 +267,20 @@ export abstract class BinarySessionRecords {
),
);
const profile = active?.profile ?? active?.profilelessWorkflowProfile;
if (
active?.profile === null &&
(profile === undefined ||
!isProfilelessNativeDispatchSnapshot(snapshot, active.target, profile))
)
return err(
new AnalysisSnapshotMismatchError(
"Analysis snapshot profile_mismatch: the snapshot is not bound to the active native dispatch reader",
),
if (active?.profile === null) {
const mismatch = profilelessSnapshotMismatch(
snapshot,
active.target,
profile,
);
if (mismatch !== undefined)
return err(
new AnalysisSnapshotMismatchError(
"Analysis snapshot is not bound to the active native dispatch reader",
mismatch,
),
);
}
const imported = this.#snapshot.import(
snapshot,
active === undefined
@@ -290,11 +299,11 @@ export abstract class BinarySessionRecords {
return imported;
}
protected matchesSnapshot(
protected snapshotMismatch(
target: BinaryTarget,
profile: AnalysisProfileCommitment | null,
): boolean {
return this.#snapshot.matches(target, profile ?? undefined);
) {
return this.#snapshot.mismatch(target, profile ?? undefined);
}
protected selectSnapshot(
@@ -391,13 +400,37 @@ export abstract class BinarySessionRecords {
this.#emitSnapshotChanged();
}
protected clearSessionRecords(): void {
this.#records.clear();
/** Clear replay and engine invalidation state when the engine is retired. */
protected clearEngineRecords(): void {
this.#snapshot.clear();
this.#snapshotInvalidated = false;
this.#emitSnapshotChanged();
}
/** End investigation history without making mutated engine state replayable again. */
protected clearInvestigationRecords(): void {
this.#records.clear();
this.clearSnapshot();
}
/** Terminal runtime disposal also seals direct record-port writes. */
protected disposeInvestigationRecords(): void {
this.#investigationDisposed = true;
this.clearInvestigationRecords();
}
protected investigationStatistics(): InvestigationStatistics {
const statistics = this.#records.statistics();
return {
evidence_records: statistics.evidenceRecords,
unknown_revisions: statistics.unknownRevisions,
unknown_heads: statistics.unknownHeads,
retained_representation_bytes: statistics.retainedRepresentationBytes,
maximum_representation_bytes: statistics.maximumRepresentationBytes,
persistence: "memory",
};
}
#emitSnapshotChanged(): void {
for (const listener of this.#snapshotListeners) {
try {
@@ -418,6 +451,8 @@ export abstract class BinarySessionRecords {
recordUnknown(
input: RecordUnknownInput,
): Result<ResidualUnknown, AnalysisError> {
if (this.#investigationDisposed)
return err(new AnalysisCancelledError("record_unknown"));
const target = this.activeAnalysisBinding()?.target;
const recorded = this.#records.recordUnknown(input, target);
if (recorded.ok) this.#emitSnapshotChanged();
@@ -428,6 +463,8 @@ export abstract class BinarySessionRecords {
evidence: Evidence,
input: RecordUnknownInput,
): Result<ResidualUnknown | null, AnalysisError> {
if (this.#investigationDisposed)
return err(new AnalysisCancelledError("record_evidence_with_unknown"));
const recorded = this.#records.recordEvidenceWithUnknown(evidence, input);
if (recorded.ok) this.#emitSnapshotChanged();
return recorded;
@@ -436,6 +473,8 @@ export abstract class BinarySessionRecords {
updateUnknown(
input: UpdateUnknownInput,
): Result<ResidualUnknown, AnalysisError> {
if (this.#investigationDisposed)
return err(new AnalysisCancelledError("update_unknown"));
const target = this.activeAnalysisBinding()?.target;
const updated = this.#records.updateUnknown(input, target);
if (updated.ok) this.#emitSnapshotChanged();
@@ -25,9 +25,13 @@ import {
} from "./BinarySessionStatus.js";
import {
APPLE_DISPATCH_METADATA_ANALYSIS_PROFILE,
NATIVE_MACOS_PROVIDER_IDENTITY,
nativeMacOSCapabilities,
} from "../../native/NativeMacOSProviderMetadata.js";
import { NATIVE_MACOS_PROVIDER_IDENTITY } from "../../native/NativeMacosProviderIdentity.js";
import {
buildCapabilityInventory,
MULTI_MODE_REMEDIATION,
} from "../CapabilityInventory.js";
const projectedStatusSchema = z.object({
capabilities: sessionProviderFacts.shape.capabilities,
@@ -172,6 +176,14 @@ const projectStatus = async (
candidateStatuses: async () => candidates,
};
return binarySessionStatus({
investigation: {
evidence_records: 0,
unknown_revisions: 0,
unknown_heads: 0,
retained_representation_bytes: 0,
maximum_representation_bytes: 0,
persistence: "memory",
},
target: options.target,
route,
router,
@@ -509,6 +521,146 @@ it.each(["elf", "pe"] as const)(
},
);
const dispatchTarget = (format: "elf" | "mach-o"): BinaryTarget => ({
kind: "executable",
format,
path: "/fixture",
sha256: "a".repeat(64),
architecture: "x86_64",
availableArchitectures: ["x86_64"],
});
it.each([
{ format: "elf", symbols: false, nativeAvailable: false, route: false },
{ format: "elf", symbols: true, nativeAvailable: false, route: true },
{ format: "mach-o", symbols: false, nativeAvailable: true, route: true },
] as const)(
"projects $format dispatch applicability and symbol fallback ($symbols)",
async ({ format, symbols, nativeAvailable, route }) => {
const target = dispatchTarget(format);
const native = nativeMacOSCapabilities("darwin").find(
(item) => item.operation === "inspect_native_dispatch_metadata",
);
if (native === undefined) throw new Error("Missing native dispatch reader");
const capabilities = [
native,
descriptor(providerA, "decode_interface_builder", []),
descriptor(providerA, "procedure_address", []),
descriptor(providerA, "procedure_callees", []),
...(symbols ? [descriptor(providerA, "list_names", [])] : []),
];
const status = await projectStatus(capabilities, [], { target });
const parsed = projectedStatusSchema.parse(status);
const projected = expandCanonicalCapabilities(parsed.capabilities).find(
(item) => item.operation === "inspect_native_dispatch_metadata",
);
expect(projected).toMatchObject({
available: nativeAvailable,
availability_code: nativeAvailable ? null : "target_format_unsupported",
});
const inventory = buildCapabilityInventory(status, {
processCaptureEnabled: false,
});
const dispatch = inventory.find(
(item) => item.name === "inspect_native_dispatch_metadata",
);
expect(dispatch?.available).toBe(route);
const trace = inventory.find(
(item) => item.name === "trace_native_ui_action",
);
expect(
trace?.modes?.find((mode) => mode.name === "native_seed_call_trace"),
).toMatchObject({
available: route,
missing_operations: route
? []
: ["inspect_native_dispatch_metadata", "list_names"],
});
if (!route) {
expect(trace?.remediation).toBe(MULTI_MODE_REMEDIATION);
const authored = trace?.modes?.find(
(mode) => mode.name === "authored_ui_dispatch",
);
const nativeSeed = trace?.modes?.find(
(mode) => mode.name === "native_seed_call_trace",
);
expect(authored?.remediation).not.toBe(nativeSeed?.remediation);
expect(authored?.missing_operations).toContain(
"decode_interface_builder",
);
}
},
);
it("does not impose the Apple reader's target rule on another provider", async () => {
const target = dispatchTarget("elf");
const status = await projectStatus(
[descriptor(providerA, "inspect_native_dispatch_metadata", [])],
[],
{ target },
);
const projected = expandCanonicalCapabilities(
projectedStatusSchema.parse(status).capabilities,
);
expect(projected).toMatchObject([
{ operation: "inspect_native_dispatch_metadata", available: true },
]);
const inventory = buildCapabilityInventory(status, {
processCaptureEnabled: false,
});
expect(
inventory.find((item) => item.name === "inspect_native_dispatch_metadata"),
).toMatchObject({ available: true });
});
it("keeps distinct recovery for an unbound non-bundle session", async () => {
const native = nativeMacOSCapabilities("darwin").find(
(item) => item.operation === "inspect_native_dispatch_metadata",
);
if (native === undefined) throw new Error("Missing native dispatch reader");
const status = await projectStatus(
[native, descriptor(providerA, "decode_interface_builder", [])],
[],
{ target: dispatchTarget("mach-o") },
);
const inventory = buildCapabilityInventory(status, {
processCaptureEnabled: false,
});
const trace = inventory.find(
(item) => item.name === "trace_native_ui_action",
);
expect(trace).toMatchObject({
available: false,
reason: "provider_unavailable",
remediation: MULTI_MODE_REMEDIATION,
});
const authored = trace?.modes?.find(
(mode) => mode.name === "authored_ui_dispatch",
);
const nativeSeed = trace?.modes?.find(
(mode) => mode.name === "native_seed_call_trace",
);
expect(authored?.missing_operations).toContain("decode_interface_builder");
expect(nativeSeed?.missing_operations).toContain("procedure_address");
expect(authored?.remediation).not.toBe(nativeSeed?.remediation);
});
it("keeps host unavailability ahead of target applicability", async () => {
const native = nativeMacOSCapabilities("linux").find(
(item) => item.operation === "inspect_native_dispatch_metadata",
);
if (native === undefined) throw new Error("Missing native dispatch reader");
const target = dispatchTarget("elf");
const projected = await project([native], [], new Map(), target);
expect(expandCanonicalCapabilities(projected.capabilities)).toMatchObject([
{
operation: "inspect_native_dispatch_metadata",
available: false,
availability_code: "unsupported_host",
},
]);
});
describe("binary session summary projection", () => {
it("keeps each candidate's status, code and reason in summary without diagnostics or capability sets", async () => {
const unavailable: AnalysisProviderCandidateStatus = {
+11 -1
View File
@@ -1,5 +1,7 @@
import { z } from "zod";
import type { InvestigationStatistics } from "../../domain/investigationLifecycle.js";
import { artifactTargetRefusal } from "../../domain/apple/artifactTargetApplicability.js";
import { targetApplicabilityRefusal } from "../../domain/targetApplicability.js";
import type { BinaryTarget } from "../../domain/binaryTargetTypes.js";
import type { ResolvedAppBundleResourceRoot } from "../AppBundleExecutable.js";
import { binaryTargetSummary } from "./BinaryTargetSummary.js";
@@ -26,6 +28,7 @@ import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js"
import type { BinarySessionDetail } from "../../contracts/sessionStatusContract.js";
interface BinarySessionStatusInput {
readonly investigation: InvestigationStatistics;
readonly target: BinaryTarget | undefined;
readonly bundleResourceRoot?: ResolvedAppBundleResourceRoot;
readonly resourceCapabilities?: readonly CapabilityDescriptor[];
@@ -46,6 +49,7 @@ interface BinarySessionStatusInput {
/** Project internal provider routing state into the caller-visible session status. */
export const binarySessionStatus = async ({
investigation,
target,
bundleResourceRoot,
resourceCapabilities = [],
@@ -73,7 +77,12 @@ export const binarySessionStatus = async ({
const targetRefusal =
target === undefined
? undefined
: artifactTargetRefusal(descriptor.operation, target);
: descriptor.targetApplicability === undefined
? artifactTargetRefusal(descriptor.operation, target)
: targetApplicabilityRefusal(
descriptor.targetApplicability,
target,
);
if (targetRefusal !== undefined && descriptor.available)
return {
...descriptor,
@@ -130,6 +139,7 @@ export const binarySessionStatus = async ({
},
);
const common = {
investigation,
cleanup_pending: cleanupPending,
capabilities,
analysis_provider_binding: analysisProviderBinding,
@@ -1,8 +1,9 @@
import { analysisProfilesEqual } from "../../domain/analysisProfile.js";
import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js";
import {
snapshotMatchesBinding,
snapshotBindingMismatch,
type AnalysisSnapshot,
type SnapshotBindingMismatch,
} from "../../domain/analysisSnapshot.js";
import type { BinaryTarget } from "../../domain/binaryTargetTypes.js";
import { workflowAnalysisProfile } from "../WorkflowAnalysisProfile.js";
@@ -28,15 +29,22 @@ export const profilelessNativeDispatchProfile = (
return structuredClone(profile);
};
/** Require only exact native dispatch workflow observations for this target/profile. */
export const isProfilelessNativeDispatchSnapshot = (
/** Preserve the difference between a binding mismatch and incompatible contents. */
export const profilelessSnapshotMismatch = (
snapshot: AnalysisSnapshot,
target: BinaryTarget,
profile: AnalysisProfileCommitment,
): boolean =>
snapshotMatchesBinding(snapshot, target, profile) &&
snapshot.entries.length === 0 &&
hasOnlyProfilelessNativeDispatchEntries(snapshot.workflow_entries, profile);
profile: AnalysisProfileCommitment | undefined,
): SnapshotBindingMismatch | undefined => {
const mismatch = snapshotBindingMismatch(snapshot, target, profile);
if (mismatch !== undefined) return mismatch;
if (
profile === undefined ||
snapshot.entries.length !== 0 ||
!hasOnlyProfilelessNativeDispatchEntries(snapshot.workflow_entries, profile)
)
return { workflow_scope: "incompatible" };
return undefined;
};
/** Reject mixed operations and entries derived from a different source profile. */
export const hasOnlyProfilelessNativeDispatchEntries = (
@@ -138,6 +138,7 @@ const harness = (options?: {
Record<string, "error" | "timeout" | "empty" | "null">
>;
readonly info?: "ok" | "unavailable" | "error";
readonly listProfile?: typeof profile;
readonly subject?: boolean;
readonly mismatch?: string;
readonly unprofiled?: string;
@@ -170,7 +171,7 @@ const harness = (options?: {
})),
provider,
{
analysisProfile: profile,
analysisProfile: options?.listProfile ?? profile,
...(options?.subject === false ? {} : { subject }),
},
),
@@ -614,6 +615,37 @@ describe("decompile_all resume", () => {
await readFile(join(directory(), "functions/0x1000.c"), "utf8"),
).toContain(sentinel);
});
it("names the resume identity that differs instead of directing callers to the input schema", async () => {
expect((await run(harness().tools, directory())).ok).toBe(true);
const resumed = await run(
harness({ listProfile: otherProfile }).tools,
directory(),
{ resume: true },
);
expect(resumed.ok).toBe(false);
if (resumed.ok) return;
expect(resumed.error).toBeInstanceOf(AnalysisInputError);
if (!(resumed.error instanceof AnalysisInputError)) return;
expect(resumed.error.userMessage).toBe(
"The decompile journal does not match the current analysis profile digest.",
);
expect(resumed.error.conformsToInputContract).toBe(true);
expect(resumed.error.remediationAction).toContain("new output directory");
expect(resumed.error.issues).toEqual([
expect.objectContaining({
path: ["output_directory"],
expected: expect.objectContaining({
analysis_profile_digest: profile.digest,
}),
observed: expect.objectContaining({
analysis_profile_digest: otherProfile.digest,
}),
}),
]);
});
});
describe("decompile_all filters and layout", () => {
@@ -29,9 +29,9 @@ import {
type DecompileJournalHeader,
} from "../../contracts/decompileDirectory.js";
import {
AnalysisInputError,
AnalysisOutputError,
AnalysisResourceConstraintError,
type AnalysisInputError,
} from "../../domain/analysisErrorCore.js";
import { AnalysisError } from "../../domain/analysisErrorBase.js";
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
@@ -663,20 +663,18 @@ const readJournal = async (
if (!parsed.ok) return parsed;
const unknown = unknownIdentity(parsed.value.header, identity);
if (unknown !== undefined) return err(unknown);
return sameIdentity(parsed.value.header, identity)
? ok({
records: parsed.value.records,
written: parsed.value.written,
paths: parsed.value.paths,
committedBytes: parsed.value.committedBytes,
journalBytes: parsed.value.journalBytes,
})
: err(
decompileInputError(
"output_directory",
"The decompile journal does not match this target, provider, analysis profile, or layout.",
),
);
const mismatches = identityMismatches(parsed.value.header, identity);
if (mismatches.length > 0)
return err(
journalIdentityMismatch(parsed.value.header, identity, mismatches),
);
return ok({
records: parsed.value.records,
written: parsed.value.written,
paths: parsed.value.paths,
committedBytes: parsed.value.committedBytes,
journalBytes: parsed.value.journalBytes,
});
};
const parseJournal = (
@@ -747,16 +745,62 @@ const unknownIdentity = (
return undefined;
};
const sameIdentity = (
/** Identity components in the order presented to the caller. */
const identityMismatches = (
header: DecompileJournalHeader,
identity: ExportIdentity,
): boolean =>
header.target_sha256 === identity.targetSha256 &&
header.analysis_profile_digest === identity.analysisProfileDigest &&
header.layout === identity.layout &&
header.provider.id === identity.provider.id &&
header.provider.name === identity.provider.name &&
header.provider.version === identity.provider.version;
): readonly string[] => [
...(header.target_sha256 === identity.targetSha256 ? [] : ["target SHA-256"]),
...(header.analysis_profile_digest === identity.analysisProfileDigest
? []
: ["analysis profile digest"]),
...(header.layout === identity.layout ? [] : ["layout"]),
...(header.provider.id === identity.provider.id ? [] : ["provider id"]),
...(header.provider.name === identity.provider.name ? [] : ["provider name"]),
...(header.provider.version === identity.provider.version
? []
: ["provider version"]),
];
/**
* Report the concrete resume identity that differs. The request is valid, so
* generic input-schema recovery would send the caller to the wrong boundary.
*/
const journalIdentityMismatch = (
header: DecompileJournalHeader,
identity: ExportIdentity,
mismatches: readonly string[],
): AnalysisInputError => {
const message = `The decompile journal does not match the current ${mismatches.join(", ")}.`;
return new AnalysisInputError(
"decompile_all",
{
userMessage: message,
conformsToInputContract: true,
remediationAction:
"Resume with the target, provider, analysis profile, and layout that created this directory, or choose a new output directory. The journal and existing files were left unchanged.",
},
[
{
path: ["output_directory"],
reason: "invalid_value",
message,
expected: {
target_sha256: header.target_sha256,
analysis_profile_digest: header.analysis_profile_digest,
layout: header.layout,
provider: header.provider,
},
observed: {
target_sha256: identity.targetSha256,
analysis_profile_digest: identity.analysisProfileDigest,
layout: identity.layout,
provider: identity.provider,
},
},
],
);
};
const incompatibleJournal = (): AnalysisInputError =>
decompileInputError(
+1 -1
View File
@@ -1,6 +1,6 @@
import { expect, it } from "vitest";
import { BeamService } from "./BeamService.js";
import { ERLANG_BEAM_PROVIDER_IDENTITY } from "../../erlang/BeamProvider.js";
import { ERLANG_BEAM_PROVIDER_IDENTITY } from "../../erlang/ErlangBeamProviderIdentity.js";
import type { ErlangBeam } from "../../domain/erlang/beam.js";
import { ok } from "../../domain/result.js";
import {
+234 -68
View File
@@ -1,10 +1,10 @@
import { InvalidResidualUnknownGraphError } from "../../domain/residualUnknownGraphError.js";
import canonicalize from "canonicalize";
import { z } from "zod";
import {
createEvidenceBundle,
createImmutableEvidenceBundle,
isImmutableEvidenceBundle,
safeParseImmutableEvidenceBundle,
validateResidualUnknownAddition,
type EvidenceBundle,
@@ -12,10 +12,7 @@ import {
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import {
evidenceSemanticContent,
type Evidence,
} from "../../domain/evidence.js";
import { isImmutableEvidence, type Evidence } from "../../domain/evidence.js";
import { safeParseImmutableEvidence } from "../../domain/evidenceValidation.js";
import {
createResidualUnknown,
@@ -27,11 +24,22 @@ import {
} from "../../domain/residualUnknown.js";
import { err, ok, type Result } from "../../domain/result.js";
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
import type { UnknownPage, UnknownQuery } from "./InvestigationRecordPort.js";
import type {
InvestigationStatistics,
UnknownPage,
UnknownQuery,
} from "./InvestigationRecordPort.js";
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import {
admitInvestigationAuthentication,
admitInvestigationRetention,
investigationMaximumRepresentationBytes,
investigationValueBytes,
type InvestigationRetentionPolicy,
} from "./InvestigationRetention.js";
type EvidenceLedgerFailure = EvidenceIntegrityError;
type RecordResult = Result<"added" | "duplicate", EvidenceLedgerFailure>;
type RecordResult = Result<"added" | "duplicate", AnalysisError>;
/** Same authenticated semantic identity with new local display/request metadata. */
export interface EvidenceMetadataReplacement {
@@ -48,19 +56,41 @@ export interface EvidenceImportDelta {
readonly replacements: readonly EvidenceMetadataReplacement[];
}
type ImportResult = Result<
EvidenceImportDelta,
EvidenceLedgerFailure | AnalysisInputError
>;
type ImportResult = Result<EvidenceImportDelta, AnalysisError>;
interface RetentionChanges {
readonly deltaBytes: number;
readonly recordBytes: ReadonlyMap<string, number>;
}
/** Session-owned set of immutable evidence records. */
/** Bounded immutable facts owned by an investigation or a task-local workflow. */
export class EvidenceLedger {
#records = new Map<string, Evidence>();
#unknownRevisions = new Map<string, ResidualUnknown>();
#unknownHeads = new Map<string, ResidualUnknown>();
#retainedBytes = 0;
#recordBytes = new Map<string, number>();
readonly #maximumBytes: number;
constructor(
private readonly retentionPolicy: InvestigationRetentionPolicy = {},
) {
this.#maximumBytes =
investigationMaximumRepresentationBytes(retentionPolicy);
}
/** Read committed counts without constructing or hashing a bundle. */
statistics(): InvestigationStatistics {
return {
evidenceRecords: this.#records.size,
unknownRevisions: this.#unknownRevisions.size,
unknownHeads: this.#unknownHeads.size,
retainedRepresentationBytes: this.#retainedBytes,
maximumRepresentationBytes: this.#maximumBytes,
};
}
/** Record evidence idempotently; conflicting content is rejected. */
record(input: Evidence): RecordResult {
const authenticated = readLedgerEvidence(
const authenticated = this.#readEvidence(
input,
"Evidence record validation failed",
);
@@ -68,20 +98,31 @@ export class EvidenceLedger {
const evidence = authenticated.value;
const existing = this.#records.get(evidence.evidence_id);
if (existing !== undefined) {
// A borrowed record is already authenticated and deeply immutable. Its
// identity cannot change between reads, so comparing document-sized
// canonical strings again would add no integrity check.
if (existing === evidence) return ok("duplicate");
return recordsAgree(existing, evidence)
? ok("duplicate")
: err(new EvidenceIntegrityError("Conflicting evidence record"));
// Both records were authenticated and sealed at ingestion. Their exact
// semantic IDs establish identity even when local display paths differ;
// serializing both complete payloads again adds no integrity proof.
return ok("duplicate");
}
const admitted = this.#admitChanges(
"record_evidence",
new Map([[evidence.evidence_id, evidence]]),
);
if (!admitted.ok) return admitted;
this.#records.set(evidence.evidence_id, evidence);
this.#commitAccounting(admitted.value);
return ok("added");
}
/** Validate an entire bundle before atomically merging its records. */
import(input: unknown): ImportResult {
if (!isImmutableEvidenceBundle(input)) {
const headroom = admitInvestigationAuthentication(
input,
"import_evidence_bundle",
this.retentionPolicy,
);
if (!headroom.ok) return headroom;
}
const parsed = safeParseImmutableEvidenceBundle(input);
if (!parsed.ok) {
const { kind, message, cause } = parsed.error;
@@ -100,8 +141,8 @@ export class EvidenceLedger {
);
}
const bundle = parsed.value;
// Stage record conflicts and display metadata once, before either commit
// path. Unknown history additionally requires validating the merged graph.
// Incoming identities were authenticated above. Stage display replacements
// once; Unknown history additionally requires validating the merged graph.
const recordChanges = new Map<string, Evidence>();
let recordsAdded = 0;
const replacements: EvidenceMetadataReplacement[] = [];
@@ -112,8 +153,7 @@ export class EvidenceLedger {
recordsAdded += 1;
continue;
}
if (!recordsAgree(existing, evidence))
return err(new EvidenceIntegrityError("Conflicting evidence record"));
if (existing === evidence) continue;
if (!displayMetadataAgree(existing, evidence)) {
recordChanges.set(evidence.evidence_id, evidence);
replacements.push({
@@ -125,8 +165,14 @@ export class EvidenceLedger {
}
// A record-only bundle cannot change the already-valid unknown graph.
if (bundle.unknowns.length === 0) {
const admitted = this.#admitChanges(
"import_evidence_bundle",
recordChanges,
);
if (!admitted.ok) return admitted;
for (const evidence of recordChanges.values())
this.#records.set(evidence.evidence_id, evidence);
this.#commitAccounting(admitted.value);
return ok({
recordsAdded,
unknownsAdded: 0,
@@ -134,14 +180,11 @@ export class EvidenceLedger {
replacements,
});
}
const pending = new Map(this.#records);
for (const evidence of recordChanges.values())
pending.set(evidence.evidence_id, evidence);
const pendingUnknowns = new Map(this.#unknownRevisions);
const unknownChanges = new Map<string, ResidualUnknown>();
let unknownsAdded = 0;
for (const unknown of bundle.unknowns) {
const key = unknownRevisionKey(unknown);
const existing = pendingUnknowns.get(key);
const existing = this.#unknownRevisions.get(key);
if (
existing !== undefined &&
existing.revision_digest !== unknown.revision_digest
@@ -156,11 +199,33 @@ export class EvidenceLedger {
},
}),
);
pendingUnknowns.set(key, unknown);
if (existing === undefined) unknownsAdded += 1;
if (existing === undefined) {
unknownChanges.set(key, unknown);
unknownsAdded += 1;
}
}
if (recordChanges.size === 0 && unknownChanges.size === 0)
return ok({
recordsAdded: 0,
unknownsAdded: 0,
changed: false,
replacements: [],
});
const admitted = this.#admitChanges(
"import_evidence_bundle",
recordChanges,
[...unknownChanges.values()],
);
if (!admitted.ok) return admitted;
const pending = new Map(this.#records);
for (const evidence of recordChanges.values())
pending.set(evidence.evidence_id, evidence);
const pendingUnknowns = new Map(this.#unknownRevisions);
for (const [key, unknown] of unknownChanges)
pendingUnknowns.set(key, unknown);
const checked = this.#validateCandidate(pending, pendingUnknowns);
if (!checked.ok) {
if (!(checked.error instanceof UnknownRegistryError)) return checked;
return err(
new EvidenceIntegrityError("Evidence bundle unknown registry failed", {
cause: checked.error,
@@ -168,6 +233,7 @@ export class EvidenceLedger {
);
}
this.#commit(checked.value);
this.#commitAccounting(admitted.value);
return ok({
recordsAdded,
unknownsAdded,
@@ -219,13 +285,17 @@ export class EvidenceLedger {
recordUnknown(
input: RecordUnknownInput,
mutationEvidence: Evidence,
): Result<
ResidualUnknown,
EvidenceIntegrityError | UnknownRegistryError | AnalysisInputError
> {
const authenticated = readLedgerEvidence(
): Result<ResidualUnknown, AnalysisError> {
const headroom = admitInvestigationAuthentication(
input,
"record_unknown",
this.retentionPolicy,
);
if (!headroom.ok) return headroom;
const authenticated = this.#readEvidence(
mutationEvidence,
"Unknown mutation evidence is invalid",
"record_unknown",
);
if (!authenticated.ok) return authenticated;
const parsedMutation = authenticated.value;
@@ -254,28 +324,26 @@ export class EvidenceLedger {
evidenceInput: Evidence,
input: RecordUnknownInput,
mutationInput: Evidence,
): Result<
ResidualUnknown | null,
EvidenceIntegrityError | UnknownRegistryError
> {
const authenticatedEvidence = readLedgerEvidence(
): Result<ResidualUnknown | null, AnalysisError> {
const headroom = admitInvestigationAuthentication(
input,
"record_unknown",
this.retentionPolicy,
);
if (!headroom.ok) return headroom;
const authenticatedEvidence = this.#readEvidence(
evidenceInput,
"Atomic Evidence validation failed",
);
if (!authenticatedEvidence.ok) return authenticatedEvidence;
const authenticatedMutation = readLedgerEvidence(
const authenticatedMutation = this.#readEvidence(
mutationInput,
"Atomic Evidence validation failed",
"record_unknown",
);
if (!authenticatedMutation.ok) return authenticatedMutation;
const evidence = authenticatedEvidence.value;
const mutation = authenticatedMutation.value;
const existingEvidence = this.#records.get(evidence.evidence_id);
if (
existingEvidence !== undefined &&
!recordsAgree(existingEvidence, evidence)
)
return err(new EvidenceIntegrityError("Conflicting evidence record"));
let unknown: ResidualUnknown;
try {
unknown = createResidualUnknown(
@@ -306,15 +374,18 @@ export class EvidenceLedger {
updateUnknown(
input: UpdateUnknownInput,
mutationEvidence: Evidence,
): Result<
ResidualUnknown,
EvidenceIntegrityError | UnknownRegistryError | AnalysisInputError
> {
): Result<ResidualUnknown, AnalysisError> {
const current = this.#unknownHeads.get(input.unknown_id);
if (current === undefined)
return err(new UnknownRegistryError("not-found"));
if (current.revision !== input.expected_revision)
return err(new UnknownRegistryError("revision-conflict"));
const headroom = admitInvestigationAuthentication(
input,
"update_unknown",
this.retentionPolicy,
);
if (!headroom.ok) return headroom;
let unknown: ResidualUnknown;
try {
unknown = updateResidualUnknown(
@@ -330,9 +401,10 @@ export class EvidenceLedger {
}),
);
}
const authenticated = readLedgerEvidence(
const authenticated = this.#readEvidence(
mutationEvidence,
"Unknown mutation evidence is invalid",
"update_unknown",
);
if (!authenticated.ok) return authenticated;
return this.#appendUnknown(unknown, authenticated.value);
@@ -400,17 +472,19 @@ export class EvidenceLedger {
});
}
/** Clear records when the owning session closes. */
/** Explicit reset/disposal releases records and accounting; the ceiling stays stable. */
clear(): void {
this.#records.clear();
this.#unknownRevisions.clear();
this.#unknownHeads.clear();
this.#retainedBytes = 0;
this.#recordBytes.clear();
}
#appendUnknown(
unknown: ResidualUnknown,
mutationEvidence: Evidence,
): Result<ResidualUnknown, EvidenceIntegrityError | UnknownRegistryError> {
): Result<ResidualUnknown, AnalysisError> {
const checked = this.#appendIncremental([mutationEvidence], unknown);
if (!checked.ok) return checked;
return ok(structuredClone(unknown));
@@ -420,15 +494,13 @@ export class EvidenceLedger {
#appendIncremental(
records: readonly Evidence[],
unknown?: ResidualUnknown,
): Result<void, EvidenceIntegrityError | UnknownRegistryError> {
): Result<void, AnalysisError> {
const additions = new Map<string, Evidence>();
for (const evidence of records) {
const existing =
additions.get(evidence.evidence_id) ??
this.#records.get(evidence.evidence_id);
if (existing !== undefined) {
if (!recordsAgree(existing, evidence))
return err(new EvidenceIntegrityError("Conflicting evidence record"));
continue;
}
additions.set(evidence.evidence_id, evidence);
@@ -451,11 +523,18 @@ export class EvidenceLedger {
if (!(cause instanceof InvalidResidualUnknownGraphError)) throw cause;
return err(new UnknownRegistryError(cause.constraint, { cause }));
}
const admitted = this.#admitChanges(
unknown === undefined ? "record_evidence" : mutationOperation(unknown),
additions,
unknown === undefined ? [] : [unknown],
);
if (!admitted.ok) return admitted;
for (const [id, evidence] of additions) this.#records.set(id, evidence);
if (unknown !== undefined && unknownKey !== undefined) {
this.#unknownRevisions.set(unknownKey, unknown);
this.#unknownHeads.set(unknown.unknown_id, unknown);
}
this.#commitAccounting(admitted.value);
return ok(undefined);
}
@@ -463,13 +542,19 @@ export class EvidenceLedger {
#validateCandidate(
records: ReadonlyMap<string, Evidence>,
unknowns: ReadonlyMap<string, ResidualUnknown>,
): Result<EvidenceBundle, UnknownRegistryError> {
): Result<EvidenceBundle, AnalysisError> {
// Evidence records borrow their immutable authentication proof. The bundle
// owner still parses the merged Unknown history into a new sealed snapshot.
const revisions = [...unknowns.values()];
const headroom = admitInvestigationAuthentication(
revisions,
"import_evidence_bundle",
this.retentionPolicy,
);
if (!headroom.ok) return headroom;
try {
return ok(
createImmutableEvidenceBundle(
[...records.values()],
[...unknowns.values()],
),
createImmutableEvidenceBundle([...records.values()], revisions),
);
} catch (cause: unknown) {
if (
@@ -505,8 +590,93 @@ export class EvidenceLedger {
this.#unknownRevisions = unknownRevisions;
this.#unknownHeads = unknownHeads;
}
#readEvidence(
input: Evidence,
message: string,
operation = "record_evidence",
): Result<Evidence, AnalysisError> {
if (!isImmutableEvidence(input)) {
const headroom = admitInvestigationAuthentication(
input,
operation,
this.retentionPolicy,
);
if (!headroom.ok) return headroom;
}
return readLedgerEvidence(input, message);
}
/** Stage all byte-index changes; refused inputs never grow owner metadata. */
#admitChanges(
operation: string,
records: ReadonlyMap<string, Evidence>,
unknowns: readonly ResidualUnknown[] = [],
): Result<RetentionChanges, AnalysisError> {
let delta = 0;
const recordBytes = new Map<string, number>();
// Account for every replacement before additions so an atomic shrinking
// replacement can provide room regardless of incoming bundle order.
for (const id of records.keys()) {
const previousBytes = this.#recordBytes.get(id);
if (previousBytes !== undefined)
delta -= LEDGER_ENTRY_BYTES + previousBytes;
}
for (const record of records.values()) {
const remaining = this.#maximumBytes - this.#retainedBytes - delta;
const bytes = investigationValueBytes(
record,
remaining - LEDGER_ENTRY_BYTES,
);
delta += LEDGER_ENTRY_BYTES + bytes;
const admitted = admitInvestigationRetention(
operation,
this.#retainedBytes,
delta,
this.#maximumBytes,
);
if (!admitted.ok) return admitted;
recordBytes.set(record.evidence_id, bytes);
}
const newHeads = new Set<string>();
for (const unknown of unknowns) {
const key = unknownRevisionKey(unknown);
const indexBytes = LEDGER_ENTRY_BYTES + 64 + key.length * 2;
const headBytes =
this.#unknownHeads.has(unknown.unknown_id) ||
newHeads.has(unknown.unknown_id)
? 0
: LEDGER_ENTRY_BYTES;
newHeads.add(unknown.unknown_id);
const remaining = this.#maximumBytes - this.#retainedBytes - delta;
delta +=
indexBytes +
headBytes +
investigationValueBytes(unknown, remaining - indexBytes - headBytes);
const admitted = admitInvestigationRetention(
operation,
this.#retainedBytes,
delta,
this.#maximumBytes,
);
if (!admitted.ok) return admitted;
}
return ok({ deltaBytes: delta, recordBytes });
}
#commitAccounting(changes: RetentionChanges): void {
for (const [id, bytes] of changes.recordBytes)
this.#recordBytes.set(id, bytes);
this.#retainedBytes += changes.deltaBytes;
}
}
// Map entries, their key/value references and the owner's committed size index.
// Payloads are charged separately using the shared JSON storage estimator.
const LEDGER_ENTRY_BYTES = 256;
const mutationOperation = (unknown: ResidualUnknown): string =>
unknown.revision === 1 ? "record_unknown" : "update_unknown";
const readLedgerEvidence = (
input: unknown,
message: string,
@@ -524,10 +694,6 @@ const readLedgerEvidence = (
const unknownRevisionKey = (unknown: ResidualUnknown): string =>
`${unknown.unknown_id}:${String(unknown.revision)}`;
const recordsAgree = (left: Evidence, right: Evidence): boolean =>
canonicalize(evidenceSemanticContent(left)) ===
canonicalize(evidenceSemanticContent(right));
const displayMetadataAgree = (left: Evidence, right: Evidence): boolean =>
left.subject?.name === right.subject?.name &&
left.subject?.local_path === right.subject?.local_path &&
@@ -1,8 +1,6 @@
import type { Evidence } from "../../domain/evidence.js";
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import type { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import type {
RecordUnknownInput,
@@ -27,7 +25,7 @@ export interface EvidenceReader {
export interface EvidenceWriter {
recordEvidence(
evidence: Evidence,
): Result<"added" | "duplicate", EvidenceIntegrityError>;
): Result<"added" | "duplicate", AnalysisError>;
}
/** Atomic derived Evidence and linked Unknown writes. */
@@ -84,6 +82,15 @@ export interface EvidenceImportReceipt {
readonly totalRecords: number;
}
/** Committed map counts and conservative retained representation accounting. */
export interface InvestigationStatistics {
readonly evidenceRecords: number;
readonly unknownRevisions: number;
readonly unknownHeads: number;
readonly retainedRepresentationBytes: number;
readonly maximumRepresentationBytes: number;
}
/** Investigation record surface; binary execution and snapshots remain separate. */
export interface InvestigationRecordPort
extends
@@ -93,5 +100,5 @@ export interface InvestigationRecordPort
UnknownRegistryPort {
importEvidenceBundle(
bundle: unknown,
): Result<EvidenceImportReceipt, EvidenceIntegrityError | AnalysisInputError>;
): Result<EvidenceImportReceipt, AnalysisError>;
}
@@ -1,9 +1,7 @@
import type { BinaryTarget } from "../../domain/binaryTargetTypes.js";
import type { Evidence } from "../../domain/evidence.js";
import type { EvidenceBundle } from "../../domain/evidenceBundle.js";
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import type { UnknownRegistryError } from "../../domain/unknownRegistryError.js";
import type {
RecordUnknownInput,
@@ -21,7 +19,12 @@ import type {
UnknownRegistryPort,
UnknownPage,
UnknownQuery,
InvestigationStatistics,
} from "./InvestigationRecordPort.js";
import {
admitInvestigationAuthentication,
type InvestigationRetentionPolicy,
} from "./InvestigationRetention.js";
/** One investigation's validated record owner, independent of binary execution. */
export class InvestigationRecords
@@ -31,12 +34,24 @@ export class InvestigationRecords
EvidenceUnknownWriter,
UnknownRegistryPort
{
readonly #evidence = new EvidenceLedger();
readonly #evidence: EvidenceLedger;
/** Snapshot a heap-derived representation allowance for this investigation. */
constructor(
private readonly retentionPolicy: InvestigationRetentionPolicy = {},
) {
this.#evidence = new EvidenceLedger(retentionPolicy);
}
/** O(1) committed state; no bundle construction, cloning or hashing. */
statistics(): InvestigationStatistics {
return this.#evidence.statistics();
}
/** Record immutable Evidence idempotently; reject conflicting content. */
recordEvidence(
evidence: Evidence,
): Result<"added" | "duplicate", EvidenceIntegrityError> {
): Result<"added" | "duplicate", AnalysisError> {
return this.#evidence.record(evidence);
}
@@ -68,11 +83,11 @@ export class InvestigationRecords
/** Atomically merge a bundle and report changes for the owner's post-commit observers. */
mergeEvidenceBundle(
bundle: unknown,
): Result<EvidenceImportDelta, EvidenceIntegrityError | AnalysisInputError> {
): Result<EvidenceImportDelta, AnalysisError> {
return this.#evidence.import(bundle);
}
/** Clear records when the owning runtime applies its existing lifecycle policy. */
/** Explicit reset/disposal releases the investigation's durable records. */
clear(): void {
this.#evidence.clear();
}
@@ -82,6 +97,12 @@ export class InvestigationRecords
input: RecordUnknownInput,
target?: BinaryTarget,
): Result<ResidualUnknown, AnalysisError> {
const admitted = admitInvestigationAuthentication(
input,
"record_unknown",
this.retentionPolicy,
);
if (!admitted.ok) return admitted;
return this.#evidence.recordUnknown(
input,
unknownMutationEvidence(target, { operation: "record_unknown", input }),
@@ -93,6 +114,12 @@ export class InvestigationRecords
evidence: Evidence,
input: RecordUnknownInput,
): Result<ResidualUnknown | null, AnalysisError> {
const admitted = admitInvestigationAuthentication(
input,
"record_unknown",
this.retentionPolicy,
);
if (!admitted.ok) return admitted;
return this.#evidence.recordWithUnknown(
evidence,
input,
@@ -108,6 +135,12 @@ export class InvestigationRecords
input: UpdateUnknownInput,
target?: BinaryTarget,
): Result<ResidualUnknown, AnalysisError> {
const admitted = admitInvestigationAuthentication(
input,
"update_unknown",
this.retentionPolicy,
);
if (!admitted.ok) return admitted;
const evidence = unknownMutationEvidence(target, {
operation: "update_unknown",
input,
@@ -0,0 +1,298 @@
import { expect, it, vi } from "vitest";
import { createEvidence, type Evidence } from "../../domain/evidence.js";
import {
createEvidenceBundle,
createImmutableEvidenceBundle,
isImmutableEvidenceBundle,
} from "../../domain/evidenceBundle.js";
import {
recordUnknownInputSchema,
updateUnknownInputSchema,
type ResidualUnknown,
} from "../../domain/residualUnknown.js";
import { InvestigationRecords } from "./InvestigationRecords.js";
const provider = { id: "fixture", name: "Retention fixture", version: "1" };
const observe = (index: number, path = "/fixture") =>
createEvidence(
{ path, sha256: "a".repeat(64), format: "file" },
provider,
{ operation: "health", parameters: { index }, result: { value: index } },
{ immutable: true },
);
const unknownInput = recordUnknownInputSchema.parse({
question: "Which observation completes this investigation?",
severity: "medium",
domain: "retention-fixture",
required_authority: null,
required_confidence: "derived",
required_environment: null,
recommended_probes: [],
relationships: [],
});
const updateInput = (unknown: ResidualUnknown) =>
updateUnknownInputSchema.parse({
unknown_id: unknown.unknown_id,
expected_revision: unknown.revision,
status: "investigating",
severity: unknown.severity,
supporting_evidence_ids: unknown.supporting_evidence_ids,
contradicting_evidence_ids: unknown.contradicting_evidence_ids,
required_authority: unknown.required_authority,
required_confidence: unknown.required_confidence,
required_environment: unknown.required_environment,
recommended_probes: unknown.recommended_probes,
relationships: unknown.relationships,
resolution: unknown.resolution,
});
/** A completed unrestricted control supplies admission boundaries without repeating the estimator. */
const recordCost = (evidence: Evidence): number => {
const records = new InvestigationRecords({
maximumRepresentationBytes: Number.MAX_SAFE_INTEGER,
});
const result = records.recordEvidence(evidence);
if (!result.ok) throw result.error;
return records.statistics().retainedRepresentationBytes;
};
it("admits exact capacity, rejects cumulative growth atomically, and releases accounting on clear", () => {
const first = observe(1);
const second = observe(2);
const maximum = recordCost(first);
const records = new InvestigationRecords({
maximumRepresentationBytes: maximum,
});
expect(records.recordEvidence(first)).toEqual({ ok: true, value: "added" });
const before = records.statistics();
const borrowed = records.evidenceForAnalysis(first.evidence_id);
expect(records.recordEvidence(borrowed ?? first)).toEqual({
ok: true,
value: "duplicate",
});
expect(records.statistics()).toEqual(before);
expect(records.recordEvidence(second)).toMatchObject({
ok: false,
error: { _tag: "AnalysisResourceConstraintError", resource: "capacity" },
});
expect(records.statistics()).toEqual(before);
expect(records.evidenceForAnalysis(first.evidence_id)).toBe(borrowed);
expect(records.hasEvidence(second.evidence_id)).toBe(false);
records.clear();
expect(records.statistics()).toEqual({
evidenceRecords: 0,
unknownRevisions: 0,
unknownHeads: 0,
retainedRepresentationBytes: 0,
maximumRepresentationBytes: maximum,
});
expect(records.recordEvidence(first)).toEqual({ ok: true, value: "added" });
expect(records.statistics()).toEqual(before);
});
it("admits complete bundle transactions before committing any new record", () => {
const first = observe(1);
const second = observe(2);
const records = new InvestigationRecords({
maximumRepresentationBytes: recordCost(first),
});
const before = records.statistics();
expect(
records.mergeEvidenceBundle(createImmutableEvidenceBundle([first, second])),
).toMatchObject({
ok: false,
error: { _tag: "AnalysisResourceConstraintError", resource: "capacity" },
});
expect(records.statistics()).toEqual(before);
expect(records.exportEvidenceBundle()).toEqual(createEvidenceBundle([]));
expect(records.recordEvidence(first).ok).toBe(true);
const committed = records.statistics();
expect(
records.mergeEvidenceBundle(createImmutableEvidenceBundle([first, second]))
.ok,
).toBe(false);
expect(records.statistics()).toEqual(committed);
expect(records.hasEvidence(second.evidence_id)).toBe(false);
});
it("accounts metadata replacement deltas and refuses a growing replacement without losing the old owner", () => {
const small = observe(1, "/small");
const large = observe(1, `/large/${"x".repeat(8_192)}`);
const addition = observe(2);
expect(large.evidence_id).toBe(small.evidence_id);
const largeCost = recordCost(large);
expect(recordCost(small) + recordCost(addition)).toBeLessThan(largeCost);
const records = new InvestigationRecords({
maximumRepresentationBytes: largeCost,
});
expect(records.recordEvidence(large).ok).toBe(true);
expect(
records.mergeEvidenceBundle(
createImmutableEvidenceBundle([small, addition]),
),
).toMatchObject({ ok: true, value: { recordsAdded: 1, changed: true } });
expect(
records.evidenceForAnalysis(small.evidence_id)?.subject?.local_path,
).toBe("/small");
expect(records.statistics().retainedRepresentationBytes).toBe(
recordCost(small) + recordCost(addition),
);
const before = records.statistics();
expect(
records.mergeEvidenceBundle(createImmutableEvidenceBundle([large])).ok,
).toBe(false);
expect(records.statistics()).toEqual(before);
expect(
records.evidenceForAnalysis(small.evidence_id)?.subject?.local_path,
).toBe("/small");
expect(records.hasEvidence(addition.evidence_id)).toBe(true);
});
it("admits Evidence, mutation and Unknown together and makes exact creation replay zero-growth", () => {
const source = observe(1);
const control = new InvestigationRecords();
const created = control.recordEvidenceWithUnknown(source, unknownInput);
if (!created.ok) throw created.error;
const cost = control.statistics().retainedRepresentationBytes;
const refused = new InvestigationRecords({
maximumRepresentationBytes: cost - 1,
});
expect(refused.recordEvidenceWithUnknown(source, unknownInput)).toMatchObject(
{
ok: false,
error: { _tag: "AnalysisResourceConstraintError", resource: "capacity" },
},
);
expect(refused.statistics()).toMatchObject({
evidenceRecords: 0,
unknownRevisions: 0,
unknownHeads: 0,
retainedRepresentationBytes: 0,
});
const records = new InvestigationRecords({
maximumRepresentationBytes: cost,
});
expect(records.recordEvidenceWithUnknown(source, unknownInput).ok).toBe(true);
const before = records.statistics();
expect(records.recordEvidenceWithUnknown(source, unknownInput)).toEqual({
ok: true,
value: null,
});
expect(records.statistics()).toEqual(before);
expect(records.statistics()).toMatchObject({
evidenceRecords: 2,
unknownRevisions: 1,
unknownHeads: 1,
});
});
it("preserves the current Unknown revision and mutation history when an update exceeds capacity", () => {
const control = new InvestigationRecords();
const created = control.recordUnknown(unknownInput);
if (!created.ok) throw created.error;
const base = control.immutableEvidenceBundle();
const maximum = control.statistics().retainedRepresentationBytes;
const records = new InvestigationRecords({
maximumRepresentationBytes: maximum,
});
expect(records.mergeEvidenceBundle(base).ok).toBe(true);
const before = records.statistics();
expect(records.updateUnknown(updateInput(created.value))).toMatchObject({
ok: false,
error: { _tag: "AnalysisResourceConstraintError", resource: "capacity" },
});
expect(records.statistics()).toEqual(before);
expect(records.listUnknowns()).toEqual([created.value]);
expect(records.exportEvidenceBundle()).toEqual(base);
const updated = control.updateUnknown(updateInput(created.value));
if (!updated.ok) throw updated.error;
expect(control.statistics()).toMatchObject({
unknownRevisions: 2,
unknownHeads: 1,
});
expect(
records.mergeEvidenceBundle(control.immutableEvidenceBundle()).ok,
).toBe(false);
expect(records.statistics()).toEqual(before);
expect(records.exportEvidenceBundle()).toEqual(base);
});
it("rejects metadata growth atomically even when the import also carries existing Unknown history", () => {
const source = observe(1, "/small");
const control = new InvestigationRecords();
expect(control.recordEvidenceWithUnknown(source, unknownInput).ok).toBe(true);
const base = control.immutableEvidenceBundle();
const records = new InvestigationRecords({
maximumRepresentationBytes:
control.statistics().retainedRepresentationBytes,
});
expect(records.mergeEvidenceBundle(base).ok).toBe(true);
const enlarged = observe(1, `/large/${"x".repeat(4_096)}`);
const replacement = createImmutableEvidenceBundle(
base.records.map((record) =>
record.evidence_id === source.evidence_id ? enlarged : record,
),
base.unknowns,
);
const before = records.statistics();
expect(records.mergeEvidenceBundle(replacement).ok).toBe(false);
expect(records.statistics()).toEqual(before);
expect(records.exportEvidenceBundle()).toEqual(base);
});
it("borrows authenticated immutable inputs without budgeting nonexistent authentication clones", () => {
const first = observe(1);
const second = observe(2);
const records = new InvestigationRecords({
availableAuthenticationBytes: () => 0,
});
expect(records.recordEvidence(first).ok).toBe(true);
const bundle = createImmutableEvidenceBundle([first, second]);
expect(isImmutableEvidenceBundle(bundle)).toBe(true);
expect(
isImmutableEvidenceBundle(Object.freeze(createEvidenceBundle([first]))),
).toBe(false);
expect(records.mergeEvidenceBundle(bundle).ok).toBe(true);
const before = records.statistics();
expect(records.mergeEvidenceBundle(bundle)).toMatchObject({
ok: true,
value: { recordsAdded: 0, unknownsAdded: 0, changed: false },
});
expect(records.statistics()).toEqual(before);
expect(records.recordEvidence(structuredClone(first))).toMatchObject({
ok: false,
error: { _tag: "AnalysisResourceConstraintError", resource: "memory" },
});
expect(records.mergeEvidenceBundle(structuredClone(bundle))).toMatchObject({
ok: false,
error: { _tag: "AnalysisResourceConstraintError", resource: "memory" },
});
expect(records.statistics()).toEqual(before);
});
it("reads statistics without asking either bundle-export path to copy retained facts", () => {
const records = new InvestigationRecords();
expect(records.recordEvidence(observe(1)).ok).toBe(true);
const detached = vi
.spyOn(records, "exportEvidenceBundle")
.mockImplementation(() => {
throw new Error("Statistics must not export a detached bundle");
});
const borrowed = vi
.spyOn(records, "immutableEvidenceBundle")
.mockImplementation(() => {
throw new Error("Statistics must not construct a borrowed bundle");
});
expect(records.statistics()).toMatchObject({
evidenceRecords: 1,
unknownHeads: 0,
});
expect(detached).not.toHaveBeenCalled();
expect(borrowed).not.toHaveBeenCalled();
});
@@ -0,0 +1,94 @@
import { getHeapStatistics } from "node:v8";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import { AnalysisResourceConstraintError } from "../../domain/analysisErrorCore.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import { jsonValueValidationIssue } from "../../domain/jsonValue.js";
import { err, ok, type Result } from "../../domain/result.js";
import {
estimateJsonStorage,
requireAuthenticationHeadroom,
} from "../JsonAuthenticationHeadroom.js";
/** Instance policy; the live authentication allowance is never retained state. */
export interface InvestigationRetentionPolicy {
readonly maximumRepresentationBytes?: number;
readonly availableAuthenticationBytes?: () => number;
}
/**
* Reserve one retained representation, two authentication/canonical working
* copies and one detached export/control representation from current V8 room.
* This is conservative representation accounting, not a process RSS guarantee.
*/
export const investigationMaximumRepresentationBytes = (
policy: InvestigationRetentionPolicy,
): number => {
const maximum =
policy.maximumRepresentationBytes ??
Math.floor(getHeapStatistics().total_available_size / 4);
if (!Number.isSafeInteger(maximum) || maximum < 0)
throw new RangeError(
"Investigation representation allowance must be a nonnegative safe integer",
);
return maximum;
};
/** Validate borrowed JSON before the authentication estimator accesses it. */
export const admitInvestigationAuthentication = (
input: unknown,
operation: string,
policy: InvestigationRetentionPolicy,
): Result<void, AnalysisError> => {
const issue = jsonValueValidationIssue(input);
if (issue !== undefined)
return err(
new EvidenceIntegrityError("Investigation input is not ordinary JSON", {
userMessage: issue,
}),
);
return requireAuthenticationHeadroom(input, {
operation,
path: "investigation",
description:
"Insufficient live heap headroom to authenticate investigation records",
...(policy.availableAuthenticationBytes === undefined
? {}
: { availableBytes: policy.availableAuthenticationBytes() }),
});
};
/** Exact for admitted data; above the budget it establishes only a lower bound. */
export const investigationValueBytes = (
input: unknown,
remainingBytes: number,
): number =>
estimateJsonStorage(input, {
maximumValueBytes: Math.max(0, remainingBytes),
includeSerialized: false,
}).valueBytes;
export const admitInvestigationRetention = (
operation: string,
retainedBytes: number,
deltaBytes: number,
maximumBytes: number,
): Result<void, AnalysisError> =>
retainedBytes + deltaBytes <= maximumBytes
? ok(undefined)
: err(
new AnalysisResourceConstraintError(
operation,
"capacity",
"The investigation's retained representation allowance is exhausted.",
{
retained_representation_bytes: retainedBytes,
requested_delta_bytes: deltaBytes,
maximum_representation_bytes: maximumBytes,
},
{
remediationAction:
"Export the retained investigation and start a new investigation, or increase the parent Node process heap before starting it. Retention never evicts existing Evidence or Unknown history.",
},
),
);
@@ -1,5 +1,4 @@
import type { EvidenceWriter } from "./InvestigationRecordPort.js";
import { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { Evidence } from "../../domain/evidence.js";
import { err, ok, type Result } from "../../domain/result.js";
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
@@ -52,7 +51,7 @@ export const retainAnalysisResultSources = <Value>(
export const recordEvidenceSources = (
recordEvidence: EvidenceWriter["recordEvidence"] | undefined,
sources: readonly Evidence[],
): Result<null, EvidenceIntegrityError> => {
): Result<null, AnalysisError> => {
for (const source of sources) {
const recorded = recordEvidence?.(source);
if (recorded !== undefined && !recorded.ok) return recorded;
@@ -20,6 +20,7 @@ import { resolveNativeMatchingSources } from "./NativeMatchingSources.js";
import type { ExecutionOptions } from "../AnalysisProvider.js";
import type { NativeMatchingAllowance } from "../../config/nativeMatchingAllowance.js";
import { FUNCTION_COMPARISON_PROVIDER } from "../../domain/comparisonEvidenceIdentity.js";
import { NATIVE_FUNCTION_MATCHING_PREDICATE } from "../../domain/native/functionMatching.js";
const OPERATION = "match_native_functions";
@@ -144,7 +145,7 @@ const matchWithAdmission = async (
await checkpoint();
const evidence = createEvidence(undefined, FUNCTION_COMPARISON_PROVIDER, {
operation: OPERATION,
predicateType: "rea.native-function-matching",
predicateType: NATIVE_FUNCTION_MATCHING_PREDICATE,
confidence: "derived",
authority: "analyst-inference",
parameters: {
@@ -119,6 +119,7 @@ it("rebuilds native row selection when a caller's sealed flag covers only a shal
);
Object.freeze(report);
const parent = {
predicateType: matched.value.evidence.predicate_type,
evidenceId: matched.value.evidence.evidence_id,
operation: matched.value.evidence.operation,
normalizedResult: report,
@@ -1,7 +1,11 @@
import type { MatchNativeFunctionsInput } from "../../contracts/native/functionMatchingContracts.js";
import { digestCanonicalValue } from "../../domain/canonicalDigest.js";
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
import { isImmutableEvidence, type Evidence } from "../../domain/evidence.js";
import {
ANALYSIS_EVIDENCE_PREDICATE,
isImmutableEvidence,
type Evidence,
} from "../../domain/evidence.js";
import { readEvidence } from "../../domain/evidenceValidation.js";
import { retainedEvidenceReferenceSchema } from "../../domain/evidenceReference.js";
import {
@@ -82,7 +86,7 @@ export const resolveNativeMatchingSources = async (
): Promise<Evidence> => {
const evidence = await resolve(raw, path, borrowable);
if (
evidence.predicate_type !== "rea.analysis" ||
evidence.predicate_type !== ANALYSIS_EVIDENCE_PREDICATE ||
!operations.includes(evidence.operation) ||
evidence.subject === null
) {
@@ -1,6 +1,6 @@
import { satisfies } from "semver";
import { toolContract } from "../../contracts/toolContracts.js";
import { SQLITE_PROVIDER_IDENTITY } from "../../domain/sqlite/sqliteDatabase.js";
import { SQLITE_PROVIDER_IDENTITY } from "../../domain/sqlite/SqliteProviderIdentity.js";
const SQLITE_RUNTIME_REQUIREMENT =
"SQLite inspection requires Node.js 24.x >=24.15 or 26+ with the native DatabaseSync.limits API. Other REA features retain their published runtime requirements. The inspection worker verifies the API before querying the snapshot.";
@@ -3,6 +3,8 @@ import { cleanupElectronProcesses } from "./ElectronProcessCleanup.js";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { createHash } from "node:crypto";
import { fileURLToPath } from "node:url";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import {
@@ -30,6 +32,27 @@ it("passes the selected environment and ownership token to Electron launch", asy
vi.stubEnv("LANG", "ambient-language");
vi.stubEnv("XAUTHORITY", "/ambient/.Xauthority");
let launchedEnvironment: unknown;
const callerArguments = [
"--",
"/caller/plugin.asar",
"-r",
"caller-hook.cjs",
"\\server\\share\\".repeat(150),
];
const launchPlan = [
"-r",
fileURLToPath(
new URL("../../scripts/electron-active-hook.cjs", import.meta.url),
),
process.execPath,
...callerArguments,
];
const handoff = JSON.stringify({
length: launchPlan.length,
sha256: createHash("sha256")
.update(JSON.stringify(launchPlan))
.digest("hex"),
});
const provider = new PlaywrightElectronActiveProvider(
{
LANG: "selected-language",
@@ -38,6 +61,7 @@ it("passes the selected environment and ownership token to Electron launch", asy
XAUTHORITY: "/selected/display-authorization",
WAYLAND_DISPLAY: "selected-wayland",
REA_PRIVATE_VALUE: "not-a-launch-setting",
REA_ELECTRON_CAPTURE_ARGV: "forged ambient launch plan",
},
async (options) => {
launchedEnvironment = options?.env;
@@ -48,6 +72,7 @@ it("passes the selected environment and ownership token to Electron launch", asy
...electronActiveObservationInputSchema.parse({
executable_path: process.execPath,
application_path: process.execPath,
args: callerArguments,
}),
application_root: dirname(process.execPath),
});
@@ -60,7 +85,9 @@ it("passes the selected environment and ownership token to Electron launch", asy
XAUTHORITY: "/selected/display-authorization",
WAYLAND_DISPLAY: "selected-wayland",
REA_PROCESS_RUN_ID: expect.any(String),
REA_ELECTRON_CAPTURE_ARGV: handoff,
});
expect(handoff.length).toBeLessThan(128);
});
it("passes required Windows runtime variables without inheriting unrelated values", () => {
@@ -6,7 +6,7 @@ import { ElectronCaptureCleanup } from "./ElectronCaptureCleanup.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { electronFuseFailure } from "../electron/ElectronFuseFailures.js";
import { randomUUID } from "node:crypto";
import { createHash, randomUUID } from "node:crypto";
import { fileURLToPath } from "node:url";
import type { _electron as electron } from "playwright-core";
@@ -155,11 +155,25 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
await systemProcessOwnershipHost.captureBaseline?.(options.signal);
if (options.signal?.aborted)
throw new BrowserObservationError(OPERATION, "cancelled");
const applicationArgs = [
"-r",
hookPath,
paths.application,
...input.args,
];
const launchOptions: Parameters<typeof electron.launch>[0] = {
executablePath: paths.executable,
cwd: paths.root,
env: safeElectronEnvironment(this.environment, runId),
args: ["-r", hookPath, paths.application, ...input.args],
env: {
...safeElectronEnvironment(this.environment, runId),
REA_ELECTRON_CAPTURE_ARGV: JSON.stringify({
length: applicationArgs.length,
sha256: createHash("sha256")
.update(JSON.stringify(applicationArgs))
.digest("hex"),
}),
},
args: applicationArgs,
timeout: Math.max(1, startupDeadline - performance.now()),
};
application = await containLaunchRejections(() => {
Loaded 100 of 371 files, more files were not shown because too many files have changed in this diff. Show more