Compare commits

..
Author SHA1 Message Date
Cursor Agent 5e979c1c3b chore: merge main into codex/audit-settings-applicability 2026-11-11 02:26:38 +00:00
2885347cde test(ghidra): remove the retired processor-context suite (#9915)
* test(ghidra): keep the bridge context contract out of an empty suite

Source tests collect src/ghidra/GhidraFunctionValues.test.ts directly.
The empty describe failed that file with "No test found in suite" even
though every shard passed. Assert the bridge bindings at file level.

* test(ghidra): retire superseded source-text context assertions

Co-authored-by: gensokyo-dev <[email protected]>

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: N0zoM1z0 <[email protected]>
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-10 21:20:07 -05:00
N0zoM1z0andgensokyo-dev a6a2527ade fix(artifacts): revalidate cached MFT observations
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev f8fc572e4a fix(ntfs): report contradictory declared record identities
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev ab9e5fa204 fix(ntfs): require space for the first attribute marker
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 8d45cb45c4 fix(ntfs): validate first attribute bounds in FILE records
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 7712e3185e fix(ntfs): retain incomplete coverage alongside fixup mismatches
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 466149169e fix(ntfs): validate aligned FILE record extents
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 1c2626edbd fix(ntfs): reject reserved update sequence values
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev b16d18f198 test(artifacts): cover independently cancelled MFT traversals
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 14030e2f11 fix(ntfs): expose unrecognized FILE record flag bits
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 9505c73824 fix(artifacts): retain only the bounded MFT header sample
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 1a446cdade fix(ntfs): label the system flag independently of extensions
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 1ef7f02cae test(ntfs): group MFT header and fixup regressions
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 7b86474be7 test(artifacts): use the MFT traversal iterable contract
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 5d10337709 feat(artifacts): expose a bounded sample of MFT record headers
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 8eadb872f2 fix(ntfs): preserve full-width MFT log sequence numbers
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev f170175ec8 fix(artifacts): retry interrupted MFT preparation
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev fbe09f84df fix(artifacts): verify MFT source before publishing metadata
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev ebb480ba22 fix(ntfs): buffer the admitted MFT byte range once
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev da1f029c2f fix(artifacts): admit MFT captures through CLI and MCP
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 4db9c166ef fix(ntfs): report contradictory FILE record extents
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 29fd6a7d03 fix(ntfs): accept older FILE record headers
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev d7338876d6 fix(ntfs): reject malformed MFT update sequence arrays
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 9df3f7174f fix(ntfs): keep record identity across invalid MFT slots
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
N0zoM1z0andgensokyo-dev 79e476204f fix(ntfs): preserve MFT reference sequence numbers
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-11 10:03:35 +08:00
2SinceJuly 9c95218374 style(artifacts): apply the repository formatter to the MFT reader
The new NTFS $MFT files were committed unformatted, so the Static checks lane
failed on format:check. Formatting only; no behaviour change.

(cherry picked from commit dc6a6d40ff)
2026-11-11 10:03:35 +08:00
2SinceJuly d9d2641981 feat(artifacts): inventory NTFS $MFT record headers offline
An acquired $MFT stream had no reader, so an analyst holding one could only
treat it as an opaque file. Recognize the FILE signature of the first record
and inventory every record's header: sequence number, flags, used and total
bytes, base record reference and update-sequence array.

Each record's update-sequence fixups are verified against its sector
trailers. A mismatch keeps the header facts, is reported on the record and is
counted in the metadata, so a record that was not cleanly written is visible
rather than silently accepted. A missing or short array is reported as
unverified, a wrong signature stops the scan at its offset, and a trailing
partial record is reported as a byte count. Attributes, filenames and the
$UsnJrnl:$J journal are not decoded, so no path or change history is claimed.

Part of #9867

(cherry picked from commit ccb45f5712)
2026-11-11 10:03:35 +08:00
morluto e2585aa7ce test(ghidra): qualify instruction mode with real provider 2026-11-10 17:58:04 -05:00
morluto 1d1c7f98ff test(ci): repair verifier fixtures and cleanup assertions 2026-11-10 17:58:04 -05:00
morluto b1f1b3eff3 fix(javascript): recognize inherited environment for NaN 2026-11-10 17:58:04 -05:00
morluto c1d0f2bb45 fix(environment): retain response cleanup through shutdown 2026-11-10 17:58:04 -05:00
morluto 04de5f3200 fix(artifacts): recover interrupted reader preparation 2026-11-10 17:58:04 -05:00
morluto fe791ff159 fix(browser): mask unused PNG transparency key bits (#9890) 2026-11-10 17:39:53 -05:00
morluto dcd7d75706 fix(settings): report Ghidra limits for the operations that use them 2026-11-10 17:39:05 -05:00
morluto 4082375e73 fix(verifiers): preserve results and own cleanup through completion
Attempt independent resource cleanup without losing primary failures, retain unresolved owners, and publish completion after release. Bound existing producer commands, preserve helper return values, and settle startup and fixture children on failure.

Validation: 29 focused boundary tests passed; actual helper-value and import-bound filesystem cleanup controls passed. The real inert-child termination control and targeted JavaScript binding checks passed, alongside typecheck, lint and formatting. Live Ghidra, Cutter, Electron GUI and Windows addon qualification was not run.
2026-11-10 17:03:59 -05:00
morluto dcee59f273 test: strengthen independent oracles and consolidate boundary coverage
Require delivered facts, exact cancellation and malformed-input boundaries, and independent metadata expectations. Transfer unique assertions into stronger keepers before removing duplicate cases and overwritten fixture writes; preserve all consumed core fixture bytes.

Validation: 1,190 focused tests passed with 39 platform-dependent skips; 134 composition and conformance tests passed. Typecheck, lint, formatting, discovery and Knip passed. Independent preservation reviews and targeted regression controls passed.
2026-11-10 17:03:59 -05:00
morluto 31018cf037 test: retain failed prompt harness cleanup for retry 2026-11-10 16:50:24 -05:00
morluto 17f0c6944c test: require exact public views and export receipts 2026-11-10 16:50:24 -05:00
morluto 360cd6f438 test: isolate managed result algebra and preserve partial facts 2026-11-10 16:50:24 -05:00
morluto b129d39ac9 test: verify CLI failure status and cancellation stage ownership 2026-11-10 16:50:24 -05:00
morluto fbb785293b test: reject swallowed native binding assertions and cloned causes 2026-11-10 16:50:24 -05:00
morluto 1b8a7ee7f6 test: name the observed IL2CPP image retention witness 2026-11-10 16:50:15 -05:00
morluto 86b1b1431a test: preserve method-free IL2CPP overflow verification 2026-11-10 16:48:48 -05:00
morluto d5e2d59eb6 test: reject incomplete calibration receipts and section drift 2026-11-10 16:48:48 -05:00
morluto 03e43c0f1b test: bound compiler and startup fixture commands with existing owner 2026-11-10 16:47:10 -05:00
morluto 724211b468 test: finish independent session cleanup and completion gates 2026-11-10 16:47:10 -05:00
morluto fe1a5b1498 test: join the admitted command after caller signal assertions 2026-11-10 16:47:10 -05:00
morluto fc3f18b748 test: prove caller cancellation reaches owned commands 2026-11-10 16:44:32 -05:00
morluto 0a66fb2acf test: require independently observed verifier outcomes 2026-11-10 16:44:32 -05:00
morluto 56d2ba36b8 test: preserve provider harness cleanup and parser admission 2026-11-10 16:43:39 -05:00
morluto 0ea642843a test: retain independent ownership of fake GDB processes 2026-11-10 16:43:39 -05:00
morluto 0c37c22d51 test: remove copied authentication after command cleanup failure 2026-11-10 16:43:39 -05:00
morluto b32da4aefa test: remove the retired reactive process fixture 2026-11-10 16:43:39 -05:00
morluto c8a52fd478 test: preserve relational policy and cumulative read coverage 2026-11-10 16:43:39 -05:00
morluto 67f402fb81 test: consolidate duplicate artifact and session coverage 2026-11-10 16:41:40 -05:00
morluto 1ef28be5bc test: tighten filesystem configuration and read contracts 2026-11-10 16:41:40 -05:00
morluto de0c50ead7 test: assert artifact integrity and retained descriptor ownership 2026-11-10 16:41:40 -05:00
morluto fe4105187d test: remove fixtures for retired controlled replay 2026-11-10 16:40:47 -05:00
morluto b05f472592 test: exercise CI trust and remote cache authority policies 2026-11-10 16:40:47 -05:00
morluto 38286d4bc3 test: preserve repeated attribute URL node identity 2026-11-10 16:36:38 -05:00
morluto 15e3a73ba6 fix: pass verifier timeouts through current MCP SDK options 2026-11-10 16:35:41 -05:00
morluto 38dc9d246b test: own and bound Electron hook fixture processes 2026-11-10 16:32:53 -05:00
morluto 6736b78693 test: own and bound public CLI regression processes 2026-11-10 16:32:53 -05:00
morluto b179755e4f test: preserve independent MCP harness cleanup failures 2026-11-10 16:32:53 -05:00
morluto 42f61205c3 test: strengthen authority parser and result oracles 2026-11-10 16:32:53 -05:00
morluto b4c8799123 test: verify deadline settlement and retire cleanup fixtures 2026-11-10 16:32:53 -05:00
morluto fd584d4495 refactor: remove test-only planning and native UI runtime seams 2026-11-10 16:32:53 -05:00
morluto 3c4be703cd test: consolidate JavaScript and provider fixture coverage 2026-11-10 16:32:53 -05:00
morluto 439c197c3f test: harden artifact cancellation and native boundary coverage 2026-11-10 16:32:53 -05:00
morluto a8e815d664 test: preserve exact failure controls after consolidation 2026-11-10 16:18:18 -05:00
morluto c1e3d7dc6e test: strengthen Kuna and managed boundary result oracles 2026-11-10 16:02:45 -05:00
morluto 8661bb38d5 test: strengthen source oracles and prune duplicate proof 2026-11-10 16:02:45 -05:00
morluto 0ec73a8102 test: assert exact cancellation reasons across browser and desktop 2026-11-10 16:02:45 -05:00
morluto 93dc8d044d test: strengthen provider contracts and consolidate duplicate cases 2026-11-10 16:02:45 -05:00
morluto 6377a1831f Merge remote-tracking branch 'origin/main' into fix/full-test-audit-20261110 2026-11-10 16:02:00 -05:00
morluto a9882a58b2 test: use a portable producer for HTTP/3 cleanup 2026-11-10 16:02:00 -05:00
morluto 25f50eea02 test: use independent diagnostics and filesystem oracles 2026-11-10 16:02:00 -05:00
morluto b899c35419 test: retain completed binary observations on late cancellation 2026-11-10 16:02:00 -05:00
morluto a4b4957346 fix(artifacts): recover Deno inventory after cancelled preparation 2026-11-10 15:59:54 -05:00
morluto ddb8abf855 fix(artifacts): recover Deno inventory after cancelled preparation 2026-11-10 15:59:29 -05:00
RRFZ 99e79026c5 fix(elf): report the link-time kernel banner, not the kbuild placeholder (#9690)
Since Linux 6.1, kbuild compiles init/version.o with a temporary
UTS_VERSION. Its build number and timestamp stay empty unless
KBUILD_BUILD_VERSION or KBUILD_BUILD_TIMESTAMP is set. That weak
linux_banner stays in the image ahead of the banner from
init/version-timestamp.o, and the first-match scan reported it. The WSL
6.18 kernel showed '# SMP PREEMPT_DYNAMIC ' instead of
'#1 SMP PREEMPT_DYNAMIC Tue Oct  6 19:13:07 UTC 2026'.

Treat a banner with a space right after '#' or at its end as the
placeholder. Keep scanning and report a later banner that repeats its
text up to '#' and is not a placeholder. Report the placeholder when no
such banner follows.
2026-11-10 15:51:53 -05:00
RRFZ b4c3a028fe fix(windows): leave a shortcut icon comparison unknown across code pages (#9691)
* fix(windows): leave a shortcut icon comparison unknown across code pages

Windows stores the LinkInfo LocalBasePath only as ANSI when the system
code page can represent it, while StringData stays UTF-16. The parser
projects the ANSI bytes as Latin1, so on a code page 936 host a path such
as F:\百度网盘\BaiduNetdisk\BaiduNetdisk.exe never matched the same
Unicode IconLocation, and icon_location_differs_from_target reported true
for a shortcut whose icon is its own target.

Record whether each target path is an ANSI projection. Compare it with
the icon only when both use the same encoding or the ANSI text is ASCII.
A full target that cannot be compared makes the result null, like a
%VAR% target.

* fix(windows): keep each LinkInfo part's encoding when joining paths

A joined LinkInfo target was marked ANSI when any part was ANSI. An ASCII
ANSI share with a UTF-16 suffix such as \\server\share + 工具\app.exe
then could not be compared with a Unicode icon, although no byte needs a
code page. Record which encoding each part's non-ASCII text came from,
and compare two paths when their non-ASCII text shares one encoding.
2026-11-10 15:39:04 -05:00
ShuaixinHuang 5e4055b39e fix(process): mark Darwin compiler timeouts retryable (#9859) 2026-11-10 15:38:46 -05:00
morluto 5b34c4d3f5 test: strengthen boundary assertions and consolidate duplicate coverage 2026-11-10 15:32:41 -05:00
morluto db6400d775 style(tests): apply repository formatter to source closure checks 2026-11-10 15:32:41 -05:00
morluto dc7594e85f fix(cli): verify partial Evidence closure before publication 2026-11-10 15:25:36 -05:00
3345019fe1 fix(process): identify inaccessible Linux namespace peers (#7147)
* fix(process): recognize permission-denied user namespace peers

* fix(process): fall back to UID maps when namespace links are denied

---------

Co-authored-by: dhruvac29 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-11 03:36:38 +08:00
9dfec80e93 fix(desktop): preserve reader source inspection failures (#9773)
* fix(desktop): preserve reader source inspection failures

* docs(desktop): restore task reader and screen judge guide anchors

* test(ci): align callee discovery and invalid model extent expectations

* test(ci): preserve CLI source closure and initialize alias fixture state

* fix(cli): retain capture caller paths in environment builds

---------

Co-authored-by: hhang <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-11 03:36:18 +08:00
JG Trompandmorluto 005435ca78 fix(cli): publish partial Evidence to --evidence_output on failure (#7797)
* fix(cli): publish partial Evidence to --evidence_output on failure

publishCliAnalysisEvidence returned a failed result unchanged before any
--evidence_output handling. When a source reached its deadline, or another
failure carried partial Evidence, the requested file was not written and
--detail summary still printed the complete partial Evidence inline, which
reached more than 64 MB on a large build.

A failure whose details.partial_observation is authenticated Evidence now
publishes it through the same projection and new-file publication as a
successful result. The failure keeps its code, message and remediation.
With --detail summary, details.partial_observation holds the summary
view, and details.diagnostics.evidence_bundle reports the path and bytes.
A refused publication keeps the complete partial Evidence inline and adds
details.diagnostics.evidence_bundle_error with the refusal and a
limitation. Without --evidence_output, for cancellation, and for raw
provider observations the failure output is unchanged.

Fixes #7787

* fix(cli): keep committed partial Evidence receipts on refused publication

When publishing the partial Evidence of a failed analysis, a refusal can
follow a commit. Staging cleanup that stays incomplete after its retry
returns cleanup_incomplete with published_path and bytes, and a
cancellation after commit returns cancelled with
diagnostics.evidence_bundle. The failure path read only the second form,
so a committed file with incomplete cleanup was reported as not
published, without its receipt or the cleanup resources, and the full
partial Evidence went back inline.

Keep the original analysis failure and record the receipt from either
form in details.diagnostics.evidence_bundle. Incomplete publication
cleanup adds published, published_path, bytes, cleanup and resources to
details, the shape a successful run's cleanup_incomplete reports;
resources join any the analysis failure already reports, and its own
cleanup reason is kept. Once the file is committed, the delivered view
(the summary with --detail summary) replaces the inline partial Evidence.
Receipts omit a path that holds an excluded sensitive value, as on
success.

Tests drive the real staged publication through its commit point: cleanup
that stays incomplete, cleanup confirmed by retry, and SIGINT after
commit. A composition test checks receipt redaction on the failure path.

* fix(cli): report incomplete partial publication cleanup

---------

Co-authored-by: morluto <[email protected]>
2026-11-11 03:36:14 +08:00
dbfab4a65d fix(cli): refuse an unsupported Node.js release before linking (#9297)
* fix(cli): refuse an unsupported Node.js release before linking

The dispatcher imported the compiled runtime before checking the runtime
range, so a release outside `engines.node` failed while linking a module
that imports `createZstdDecompress`. Callers saw a module-linking stack
trace from every command, including `--version` and `doctor`, instead of
the refusal the installer and `engines` already promise.

Check `engines.node` from the package read the dispatcher already performs,
before the first compiled import. The helper has no imports of its own: the
dispatcher must admit the runtime before any dependency can be resolved.

Fixes #9296

* fix(cli): ship and align runtime admission

* fix(ci): cover packaged runtime admission helper

* test(cli): assert Node admission with structured range results

Export satisfiesNodeRange and cover caret, minimum, union, and
non-release versions. The dispatcher proves the compiled runtime never
loads by the absence of its marker file, and compares stderr to the
helper's own message.

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
2026-11-10 13:53:58 -05:00
Beqa b2f3dc81fa fix(cli): reject invalid JavaScript application paths (#9789)
Validate analyze-javascript-application paths with the shared localPathStringSchema before anchoring or provider execution. Empty paths now return CLI field errors; NUL bytes are rejected at the same boundary. Preserve selected-path Evidence for valid inputs and update the CLI regressions. Closes #9747.
2026-11-10 13:50:42 +00:00
morluto a4da4e7230 docs(mcp): clarify tool selection and recovery guidance (#9817)
* docs(mcp): clarify tool selection and recovery guidance

* test(mcp): prune wording assertions and preserve scope checks
2026-11-10 07:49:23 -05:00
morluto 433fd9486d fix(ci): align desktop assertions and calibration entry points 2026-11-10 07:27:32 -05:00
morluto 94ca4b656a docs(skills): add focused tool audit and agent trial guidance 2026-11-10 05:44:04 -05:00
d2f9b8b7de feat(desktop): build several tasks from one capture, and measure each task's pass rate within a budget (#9216)
* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(doctor): compare desktop settings with resolved defaults

* fix(desktop): refuse a build runtime whose export takes no task ends

Several tasks pass --task-end to the build computer's export, so a runtime
packed before it now fails at the start of the capture with the repack fix,
as a runtime without synthetic seeds already did, rather than at export.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(desktop): rewrite a download's launch keys written with spaces around =

The desktop entry parser accepts `Exec = notes %F` and an indented
`TryExec =notes`, so the install script now matches those keys the same way
instead of leaving the original program in place. Adds the feature's tasks.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep why a measured run failed in its solve record

A solve that returned an outcome dropped the runner's error, so a run that
failed to deploy, solve or verify was retried with no reason recorded. The
record, the measure result and difficulty.json now carry it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* test(desktop): run installer scripts at boundary

* style(desktop): remove trailing blank line

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(setup): skip Hopper launch after cancelled copy

* fix: preserve task measurement limits and cleanup

* fix(desktop): retain reported solve sandbox cleanup

* test(desktop): check retained log seed admission

* fix(doctor): query agentenv for model endpoint

* fix(doctor): preserve uncertainty for empty endpoint override

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): keep ordinary text checks exact

* fix(desktop): verify registered build host paths

* fix(desktop): retain record identity in draft checks

* test(desktop): preserve text reference numbers

* fix(desktop): bound download install input reads

* fix(desktop): match placed notes by whole text

* test(desktop): match multiple task export results

* fix(desktop): retain failed measurement file closes

* fix(setup): retain completed actions on cancellation

* test(setup): retain cancelled installer result

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* style(desktop): format review fixes

* fix(desktop): inspect archives with registered runner

* test(desktop): verify registered archive interpreter

* style(desktop): format capture review fixes

* style(desktop): format review changes

* feat(desktop): target agentenv-framework 0.9.1464

Desktop bundles, setup's --install-agentenv and the doctor's exact-version
check now target 0.9.1464, which keeps an exported app's data/get call,
holds the computer fixture above boot dialogs, leaves an unresolved setting's
build unverified rather than failed, and mounts lock packages instead of
copying them into the image. The web environment lane stays on 0.9.1431.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain per-solve sandbox cleanup roots

* test(desktop): expect the solve cleanup helper among the shipped agentenv helpers

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(desktop): clean up a stopped solve's sandboxes through agentenv 0.9.1482

agentenv-framework 0.9.1482 lists the local sandboxes under a root
(local_sandbox_ids) and rebuilds one killed while it wrote its port map for
teardown. The solve cleanup helper now terminates each sandbox agentenv
lists, keeping its Docker and host-process checks, and reports anything else
left under the root. Desktop bundles, setup's --install-agentenv and the
doctor's version check target 0.9.1482.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): verify a killed solve's cleanup once its sandbox root is empty

A solve's sandboxes are all local under its private root, and agentenv now
keeps a sandbox's folder until its teardown is confirmed. So when the helper
ends without a report and REA's cleanup empties that root, the cleanup is
verified: a cancelled solve ends as a cancellation and a solve past its
deadline as a failed run, rather than as cleanup the measure cannot vouch for.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(desktop): target agentenv-framework 0.9.1485

0.9.1485 keeps a local sandbox's folder until its teardown is confirmed, which
the solve cleanup's verified result relies on. (0.9.1484 failed to publish.)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): record a solve stopped before it reports as failed, not as a refused request

A helper killed from outside or stopped at its deadline left no report, so the
measure refused with invalid input and told the caller to fix its model
endpoint. A helper a signal ended now gives a failed solve of unknown cost,
which the measure records and stops on; one that exits on its own without a
report still ends the measure as a solve that could not run.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): check verifier behavior after selector changes

* test(windows): verify rejected writes during cleanup by effect

* fix(desktop): preserve selected alias target kinds in exports

* fix(cli): retain caller capture paths in environment builds

* fix(ghidra): classify oversized endpoint records consistently

* test: align model and Evidence expectations with current contracts

* fix(desktop): reconcile upstream sandbox cleanup recovery

* fix(docs): link cleanup qualification as repository evidence

---------

Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-10 05:26:28 -05:00
morluto cb4e5cfbff feat(analysis): make resource budgets recoverable across CLI and MCP (#9706)
* refactor(analysis): centralize resource budgets and recovery facts

* feat(analysis): expose ASAR and Ghidra runtime admission controls

* feat(artifacts): expose reader admission and preserve limited recognition

* feat(javascript): expose recoverable semantic precision budgets

* feat(web): add recoverable source-map budgets and compact retained views

* feat(web): make historical capture decoding budgets recoverable

* fix(desktop): enforce selected view policies across capture and export

* feat(runtime): make Frida and LLDB retention budgets selectable

* feat(native): expose diagnostic budgets and preserve exhaustion causes

* feat(managed): expose recoverable snapshot and representation budgets

* feat(metadata): make Go and Rust inspection budgets recoverable

* feat(wasm): expose representation and producer admissions

* feat(evm): expose per-call worker resource budgets

* feat(android): make resource and string-pool omissions recoverable

* feat(producers): expose command and recorded-capture budgets

* docs(analysis): record recovery contracts and integrated validation

* fix(docs): link calibration oracle from published verification pages

* fix(docs): restore desktop task anchors and receipt links

* test(analysis): align fixtures with selectable budget contracts

* fix(analysis): remove source-location projector dependency cycle

* test(analysis): refresh budget recovery and verifier regressions

* test(analysis): preserve format and publication recovery expectations

* test(windows): align conformance with owned runtime admission

* fix(desktop): preserve selected alias kinds in exported state

* test(analysis): align provider fixtures with recoverable budgets

* test(desktop): validate emitted alias metadata structurally

* fix(ghidra): classify bounded Windows endpoint refusals

* test(cli): assert selectable artifact admission recovery

* fix(cli): preserve capture provenance in environment builds
2026-11-10 05:11:20 -05:00
Ysopwnableandgensokyo-dev 55b6d2ca20 perf(ci): compact merged test console output (#9787)
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-10 07:43:44 +01:00
95ebd38d3f fix(ci): restore main workflows and captured-state replay (#9763)
* fix(cli): record the caller HAR path in environment-build evidence

environment-build inspected a HAR without the caller path that
inspect-web-network-capture records, so the published evidence.json
omitted caller_input_path and failed the byte-for-byte portable check.

Also update three stale portable expectations that already fail on main:
the no-reader verifier digest matches the #9185 selector parent, uncapped
IDA callee pages report complete discovery, and a CLI refusal of a
malformed artifact inspection includes the undelivered source closure.

* fix: restore portable docs, alias, offset, and Windows cleanup checks

The witness page linked task anchors that were never written, so any
docs-touching PR failed the route check. A selected state directory was
restored as a dangling symlink by a duplicated helper. An empty
safetensors file reports the extent as outside the file before offset
conversion. Windows cleanup refusal uses the runtime's in-progress message.

* fix(cli): keep the caller's HAR spelling in environment-build evidence

inspect-web-network-capture records the path the caller typed. The
one-shot build had already absolutized that path, so a relative capture
disagreed with the three-step bundle. The byte-for-byte check now runs
from the capture directory and requires caller_input_path grafana.har.

* fix(desktop): create alias targets only for directory selections

A missing settings file was created as a directory because the helper
treated any absent path as a directory. The capture's selected path kind
now travels with the alias, and only a directory alias creates its target.
An absent file alias stays a file the app can create.

* fix(desktop): keep legacy alias kind unknown

Captures written before selected_paths.kind leave the alias unknown.
Defaulting those to a directory makes an omitted settings file a
directory, so a later write through the alias fails with EISDIR.

* fix(desktop): keep alias kind a file, directory, or unknown

The export plan was widening a missing selected path kind to string.
Record file or directory only when the capture says so, and unknown otherwise.

* fix(desktop): narrow a missing alias kind to unknown

The selected path kind was widened to string, so the export plan could
not keep the file, directory, or unknown distinction.

* test(desktop): expect alias kind in the rendered app module

The export now records whether a selected alias is a directory. The
rendered STATE_ALIASES tuples include that kind.

* fix(ghidra): reject an oversized Windows TCP endpoint as invalid

The POSIX read maps a record past the compact endpoint bound to
"Ghidra TCP endpoint is invalid". The Windows lease reported that same
bound as a generic observation failure, so an expanded endpoint failed
the Windows lane with a different message.

* fix(desktop): create a legacy alias when a child proves it is a directory

A capture written before selected path kind can still nest one alias
inside another. That parent is a directory. A legacy leaf stays unknown
so a missing settings file is not created as a directory.

* fix(desktop): restore selected aliases when the app owns no files

An empty state directory does not become an owned path, so the class
was ownless and never called _ensure_state_aliases. The ownless class
now restores aliases whenever the export recorded some.

* fix(desktop): create the parent of a missing file alias

A file alias must stay a file. When its parent directory was not seeded,
the symlink was dangling and a write through it failed with ENOENT.

* ci: retrigger checks after an unrelated Windows timeout

The Windows lane timed out in webModuleCli, which this diff does not touch,
and the rerun API is unavailable.

* test(evidence): preserve complete CLI refusal diagnostics

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-10 06:37:51 +01:00
Beqa d4a7c20e04 docs: clarify coordination for overlapping PR fixes (#8931)
Say how to point at an overlapping change and then rebase, narrow, or close it, without implying a review date.
2026-11-10 02:13:39 +00:00
2SinceJuly 27edc5c83f test(ghidra): assert projected limitations instead of a stale literal (#9268)
Project analyze_function limitations from the provider instead of a sentence #9246 removed, so the boundary test follows the producer and main CI can go green.
2026-11-10 02:13:16 +00:00
ShuaixinHuang c52a9b9d79 fix(pe): reject misaligned certificate table offsets (#9666)
* fix(pe): reject misaligned certificate table offsets

* test: align ghidra limitation wording with main

* style: format ghidra limitation assertion

* test(ghidra): assert current provider limitations
2026-11-09 17:31:41 -05:00
morluto 8dfc47b107 fix(ci): restore compilation and static checks on main 2026-11-09 16:11:47 -05:00
ShuaixinHuangandFelix e9b6c87c1c fix(cli): align ADB package-name schemas (#9675)
Co-authored-by: Felix <[email protected]>
2026-11-09 10:28:15 -05:00
387bcb795d feat(desktop): isolate declared secrets during app builds (#9225)
* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): bound private key redaction spans

* fix(desktop): retain published bundle on copy failure

* fix(desktop): retain secret file cleanup owners

* fix(desktop): retain both published output cleanup owners

* test(desktop): verify UTF-16 credential refusal

* test(desktop): verify truncated secret redaction over MCP

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): scan private key runs in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* Format appTaskChecks.ts with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* fix(desktop): report unchecked client build paths

* docs(desktop): clarify gzip scrub limits

* test(desktop): build the fixture JWT from its parts and reformat

The literal token made every edit of this file trip secret scanners; the test
checks the same string. Also applies the pinned oxfmt to the key-run test.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(desktop): find no private key in NSS's key4.db password check

An Electron app's NSS key database holds a password-check value shaped like
an encrypted private key, so every such export was refused. An encrypted key
now counts only under a password-based encryption scheme, never PBKDF2 or
PBMAC1, and with at least the 48 bytes of encrypted data the smallest key
needs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): compare desktop settings with resolved defaults

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): close five withheld-state gaps a review found

- A sign-in window whose walk hit its entry limit, could not read a path or
  left a change unconfirmed is refused, rather than trusted as complete.
- The export refuses app dconf keys the sign-in window changed and the seed
  would ship with a different value, naming the keys, not their values.
- A bundle holding a gzip file that expands past what REA scans is refused.
- Withheld bytes are kept under a handle of their own, created exclusively and
  never replaced; a later build's different bytes take the next free handle,
  and a handle the computer's answer names is checked before any write.
- Setting names a Python keyword or a leading digit would break get file_,
  and names that collide get _2, _3 in a stable order.
- Adds the feature's tasks.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): redact a cut secret's start only at REA's own cuts

The boundary match for a secret cut short hid the start of any ellipsis-ended
text, so "Please…" became a redaction. It now applies only where text ends
one of REA's value cuts, whose lengths the producers and the redactor share,
and the host redacts the app's stderr and progress before it quotes their
end. Redacting before anything cuts stays a follow-up.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): publish secret files atomically, decompress a gzip seed once, and scan JSON strings linearly

- A secret file is written under a private temporary name and linked to its
  handle, so a concurrent export never reads a half-written file and reuses
  its handle for other bytes.
- The export decompresses each gzip seed file once and shares the result
  with the secret scan, the credential scan and the size refusal.
- The cut-secret stopgap finds JSON string literals in one forward pass
  instead of restarting at every quote.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): retain temporary secret file cleanup

* fix(desktop): preserve unmarked token-shaped values

* fix(desktop): withhold drafts after secret use

* fix(desktop): guard drafts against shared secret changes

* test(desktop): compare preserved seed bytes without enumeration

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(doctor): query agentenv for model endpoint

* fix(doctor): preserve uncertainty for empty endpoint override

* fix(desktop): protect host controls and quote withheld paths

* fix: redact declared form-encoded secrets

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): retain record identity in draft checks

* fix(desktop): refuse unchecked declared secrets

* fix(desktop): retain partial withheld export receipts

* fix(desktop): detect escaped secrets in JSON state

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* fix(desktop): scan escaped secrets in UTF-16 JSON

* style(desktop): format secret scan changes

* style(desktop): format review fixes

* style(desktop): format capture review fixes

* style(desktop): format review changes

---------

Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 23:23:13 +08:00
Ryan S e5ddbe8909 feat(native): decode Android .note.android.ident ELF notes (#9502)
* feat(native): decode Android .note.android.ident ELF notes

Decode Android NDK NT_ANDROID_TYPE_IDENT (.note.android.ident, owner Android, type 1) notes in inspect_elf_notes across legacy 4-byte and modern >=132-byte descriptors with bounded TextBudget accounting.

* fix(native): isolate Android note scan budget test, scope contract import, and add feature spec

* test(native): use domain tool contract in elfNotesMcp boundary test
2026-11-09 10:21:38 -05:00
018f19d8c8 feat(desktop): target agentenv-framework 0.9.1464 (#9302)
* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): exclude unseeded locks from task checks

* fix(desktop): preserve draft bounds and launch URLs

* fix(desktop): render draft values using their source format

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* Format appTaskChecks.ts with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain partial XML syntax facts

* feat(desktop): target agentenv-framework 0.9.1464

Desktop bundles, setup's --install-agentenv and the doctor's exact-version
check now target 0.9.1464, which keeps an exported app's data/get call,
holds the computer fixture above boot dialogs, leaves an unresolved setting's
build unverified rather than failed, and mounts lock packages instead of
copying them into the image. The web environment lane stays on 0.9.1431.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* fix(desktop): reuse export exclusions in task drafts

* fix(desktop): report unchecked client build paths

* docs(desktop): clarify gzip seed scrubbing

* fix(desktop): report unchecked client build paths

* docs(desktop): clarify gzip scrub limits

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(setup): own desktop build process cleanup

* fix(doctor): compare desktop settings with resolved defaults

* fix(doctor): compare desktop settings with resolved defaults

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(setup): propagate cancellation to desktop actions

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* test(desktop): use the framework pin in doctor fixtures

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(doctor): query agentenv for model endpoint

* fix(doctor): preserve uncertainty for empty endpoint override

* style(desktop): format framework pin fixture

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): retain record identity in draft checks

* fix(desktop): inspect supported runtime archives

* fix(setup): retain completed actions on cancellation

* docs(desktop): clarify document lock exclusion

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* style(desktop): format review fixes

* style(desktop): format capture review fixes

* fix(desktop): retain merged installer behavior

---------

Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 23:12:10 +08:00
fe2985a0b4 feat(desktop): check apps whose files REA can't read, through the app's own headless mode or a screen judge (#9217)
* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): exclude unseeded locks from task checks

* fix(desktop): preserve draft bounds and launch URLs

* fix(desktop): render draft values using their source format

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* Format appTaskChecks.ts with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* fix(desktop): report unchecked client build paths

* docs(desktop): clarify gzip scrub limits

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): keep the judge command on one line

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(doctor): compare desktop settings with resolved defaults

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* test(desktop): pin reader-free verifier to selector parent

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* test(desktop): check retained log seed admission

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(desktop): compare screen task views by value

* fix(desktop): reject inexact reader numbers

* fix(desktop): budget export for both reader passes

* fix(desktop): bound reader runner argument size

* test(desktop): verify chunked reader payloads and parent output

* fix(doctor): query agentenv for model endpoint

* style(desktop): format reader payload assertion

* fix(doctor): preserve uncertainty for empty endpoint override

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): retain record identity in draft checks

* fix(desktop): align timestamp selector digits

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* fix(desktop): retain remote reader cleanup ownership

* style(desktop): format capture review fixes

* fix(desktop): retain reader leases across verifier reloads

* test(desktop): overlap reader cleanup retries

* fix(desktop): run long exports within exec limits

---------

Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 23:09:52 +08:00
26d81afedd feat(desktop): install an app from an AppImage or an upstream tarball, pinned by its SHA-256 (#9215)
* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* Format appTaskChecks.ts with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* fix(desktop): report unchecked client build paths

* docs(desktop): clarify gzip scrub limits

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(doctor): compare desktop settings with resolved defaults

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(desktop): rewrite a download's launch keys written with spaces around =

The desktop entry parser accepts `Exec = notes %F` and an indented
`TryExec =notes`, so the install script now matches those keys the same way
instead of leaving the original program in place. Adds the feature's tasks.md.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* test(desktop): run installer scripts at boundary

* style(desktop): remove trailing blank line

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(setup): skip Hopper launch after cancelled copy

* fix(doctor): query agentenv for model endpoint

* fix(doctor): preserve uncertainty for empty endpoint override

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): keep ordinary text checks exact

* fix(desktop): verify registered build host paths

* fix(desktop): retain record identity in draft checks

* test(desktop): preserve text reference numbers

* fix(desktop): bound download install input reads

* fix(desktop): match placed notes by whole text

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* style(desktop): format review fixes

* fix(desktop): inspect archives with registered runner

* test(desktop): verify registered archive interpreter

* style(desktop): format capture review fixes

---------

Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 23:06:05 +08:00
d71e35a5ae fix(desktop): find ids in archive entries and draft nothing for a state directory's lock (#9210)
* fix(desktop): align nested state selections with aliases

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): exclude unseeded locks from task checks

* fix(desktop): preserve draft bounds and launch URLs

* fix(desktop): render draft values using their source format

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* Format appTaskChecks.ts with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* fix(desktop): report unchecked client build paths

* docs(desktop): clarify gzip scrub limits

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(doctor): compare desktop settings with resolved defaults

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(doctor): query agentenv for model endpoint

* fix(doctor): preserve uncertainty for empty endpoint override

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): keep ordinary text checks exact

* fix(desktop): verify registered build host paths

* fix(desktop): retain record identity in draft checks

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* style(desktop): format review fixes

* fix(desktop): inspect archives with registered runner

* test(desktop): verify registered archive interpreter

* style(desktop): format capture review fixes

---------

Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 23:03:54 +08:00
240f08d406 feat(desktop): set up and check a desktop build host in one command (#9199)
* fix(desktop): keep an empty env value empty

An empty assignment is not a path. Resolving it named the working directory
and rewrote the value when that directory's spelling moved.

* fix(desktop): keep a run's font on the run, not the frame's text

A formatting attribute such as Scribus FONT is not the document's words.
Joining it across runs made a frame-wide text check of ItalicBold that
failed when a run was split. The font stays a value check on its own run.

* fix(desktop): do not pair JSON records by a selector-shaped field

A field named @record_id matched the id suffix and became [@record_id=one],
which reads an XML attribute. Records paired while their pointers matched
nothing. Names that start with @ or > are not id fields.

* fix(desktop): count a save-numbered id once when pairing members

xml:id and draw:id values such as id123456 were both generated and
numbered, so the same selector was counted twice and refused. A text
edit of two elements keyed only by those ids now keeps its check.

* fix(desktop): count members of root JSON arrays

* fix(desktop): preserve selected state directory aliases

* docs(desktop): explain tool and skill refresh prerequisites

* fix(desktop): align nested state selections with aliases

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): exclude unseeded locks from task checks

* fix(desktop): preserve draft bounds and launch URLs

* fix(desktop): render draft values using their source format

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* Format appTaskChecks.ts with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* fix(desktop): report unchecked client build paths

* docs(desktop): clarify gzip scrub limits

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* fix(setup): own desktop build process cleanup

* fix(doctor): compare desktop settings with resolved defaults

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(doctor): inspect client-recorded desktop runners

* fix(doctor): reuse matching runner probes

* fix(setup): propagate cancellation to desktop actions

* fix(setup): stop Hopper on cancel, never report a cancelled setup ready, and keep a failed action's client setting

- The Hopper install gets setup's signal: Linux already honoured it, and the
  macOS installer now stops its downloads and stops before mounting or copying.
- runSetup checks the signal after the final doctor and close, so a
  cancellation there raises instead of returning ready.
- A failed pack or agentenv install no longer writes its archive or runner to
  client configs, so the setting the client already had stays.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(setup): keep the completed setup result in a cancellation after cleanup

A cancellation during the final doctor or host close carried only the applied
actions, dropping each client's result and backup path and the doctor report.
It now carries the whole result as setup_result, as doctor's cancellation
carries its report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* fix(setup): retain diagnostics when cancellation overlaps failure

* refactor(setup): keep command cancellation at its owner

* test(setup): verify cancellation at the application boundary

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* fix(doctor): resolve empty registered desktop runner

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): preserve archive paths and bound doctor reads

* fix(doctor): refuse non-files before descriptor admission

* fix(setup): track remaining cleanup resources across retries

* test(desktop): specify XML format in heap pairing fixture

* fix(doctor): query agentenv for model endpoint

* fix(doctor): preserve uncertainty for empty endpoint override

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): create selected state alias targets

* fix(desktop): inspect supported runtime archives

* fix(desktop): retain record identity in draft checks

* fix(desktop): scrub XML independently of reader compatibility

* fix(desktop): preserve file aliases and reader diagnostics

* style(desktop): format review fixes

* style(desktop): format capture review fixes

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 23:02:31 +08:00
61ab3f0c4f feat(desktop): name a record by a nested field or a child's attribute (#9185)
* fix(desktop): keep a selected folder that is named like a database log

A folder the caller selected stays a selected root when its name ends in
-journal or -wal. A generated log beside a database file still does not.

* fix(desktop): refuse a build command that exceeds the argument vector

Splitting one argument still leaves the whole command under Linux's E2BIG
limit. A request past that budget is refused before the process starts.

* fix(desktop): remap a captured path in an env assignment

The program word stays the program. A NAME=value assignment keeps its name
and follows a captured path value when that path's spelling moves.

* fix(desktop): keep oversized text out of draft guards

A value or line longer than the draft's text bound is not copied into a
guard, so one large frame cannot fill the task by itself.

* docs(desktop): correct dconf and check-preview guidance

A dconf version kept without its bytes is unreadable, not an empty key
list. A values preview is copied into a check only when it is an uncut scalar.

* fix(desktop): keep an empty env value empty

An empty assignment is not a path. Resolving it named the working directory
and rewrote the value when that directory's spelling moved.

* fix(desktop): keep a run's font on the run, not the frame's text

A formatting attribute such as Scribus FONT is not the document's words.
Joining it across runs made a frame-wide text check of ItalicBold that
failed when a run was split. The font stays a value check on its own run.

* fix(desktop): do not pair JSON records by a selector-shaped field

A field named @record_id matched the id suffix and became [@record_id=one],
which reads an XML attribute. Records paired while their pointers matched
nothing. Names that start with @ or > are not id fields.

* fix(desktop): count a save-numbered id once when pairing members

xml:id and draw:id values such as id123456 were both generated and
numbered, so the same selector was counted twice and refused. A text
edit of two elements keyed only by those ids now keeps its check.

* fix(desktop): count members of root JSON arrays

* fix(desktop): preserve selected state directory aliases

* docs(desktop): explain tool and skill refresh prerequisites

* fix(desktop): align nested state selections with aliases

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): exclude unseeded locks from task checks

* fix(desktop): preserve draft bounds and launch URLs

* fix(desktop): render draft values using their source format

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* Format appTaskChecks.ts with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): give the root-array draft test its own describe

The records describe grew past the 150-line lint limit with it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* fix(desktop): carry document format into selector evaluation

* test(desktop): separate markup boundary regression group

* fix(desktop): propagate formats through draft selectors

* fix(desktop): switch selector format within archive entries

* test(desktop): remove duplicate seed filename case

* test(desktop): pass JSON format in exported index parity

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* test(desktop): specify XML format in heap pairing fixture

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): retain record identity in draft checks

* fix(desktop): scrub XML independently of reader compatibility

* style(desktop): format review fixes

* style(desktop): format capture review fixes

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 23:01:22 +08:00
fd39977c32 docs(desktop): set up a desktop build host from a fresh Linux machine (#9184)
* fix(desktop): keep a program word and a directory's neighbor

The program after env stays the program even when a captured file sits in a
folder of that name. A selected directory is not a database, so a neighboring
file keeps the parent folder's spelling.

* fix(desktop): record whether a selected path is a directory

Capture stores the walked path's kind. A neighboring file of a selected
directory keeps the parent folder's spelling even when that directory never
changes and its children are not content roots.

* fix(desktop): mark containing paths without a pairwise scan

A recorded path marks its known ancestors as directories in one walk up
the path, instead of comparing every recorded path with every other.

* fix(desktop): keep a selected folder that is named like a database log

A folder the caller selected stays a selected root when its name ends in
-journal or -wal. A generated log beside a database file still does not.

* fix(desktop): refuse a build command that exceeds the argument vector

Splitting one argument still leaves the whole command under Linux's E2BIG
limit. A request past that budget is refused before the process starts.

* fix(desktop): remap a captured path in an env assignment

The program word stays the program. A NAME=value assignment keeps its name
and follows a captured path value when that path's spelling moves.

* fix(desktop): keep oversized text out of draft guards

A value or line longer than the draft's text bound is not copied into a
guard, so one large frame cannot fill the task by itself.

* docs(desktop): correct dconf and check-preview guidance

A dconf version kept without its bytes is unreadable, not an empty key
list. A values preview is copied into a check only when it is an uncut scalar.

* fix(desktop): keep an empty env value empty

An empty assignment is not a path. Resolving it named the working directory
and rewrote the value when that directory's spelling moved.

* fix(desktop): keep a run's font on the run, not the frame's text

A formatting attribute such as Scribus FONT is not the document's words.
Joining it across runs made a frame-wide text check of ItalicBold that
failed when a run was split. The font stays a value check on its own run.

* fix(desktop): do not pair JSON records by a selector-shaped field

A field named @record_id matched the id suffix and became [@record_id=one],
which reads an XML attribute. Records paired while their pointers matched
nothing. Names that start with @ or > are not id fields.

* fix(desktop): count a save-numbered id once when pairing members

xml:id and draw:id values such as id123456 were both generated and
numbered, so the same selector was counted twice and refused. A text
edit of two elements keyed only by those ids now keeps its check.

* fix(desktop): preserve selected state directory aliases

* docs(desktop): explain tool and skill refresh prerequisites

* fix(desktop): align nested state selections with aliases

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): exclude unseeded locks from task checks

* fix(desktop): preserve draft bounds and launch URLs

* fix(desktop): render draft values using their source format

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* test(desktop): separate markup boundary regression group

* test(desktop): remove duplicate seed filename case

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* test(desktop): remove duplicate history regression

* fix(desktop): align XML names with exported reader

* fix(desktop): scrub XML independently of reader compatibility

* style(desktop): format review fixes

* style(desktop): format capture review fixes

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 22:59:45 +08:00
012861ce1d feat(desktop): draft child-text selectors, member counts and attribute text, and fold in a third batch's findings (#9132)
* fix(desktop): follow a database mapping and own limitation text

A selected database keeps its log when a parent folder moves or the file
sits outside the home. Trash and seed-lock tests compare the owning notice
with the paths, and launch options that are not captured paths stay as written.

* test(desktop): pass the selected log into the draft

The export's selected home paths decide which log stays in the draft.
The line-guard test now names that log instead of relying on its reason.

* fix(desktop): retarget a settings folder that holds captured files

A nested profile directory is a captured path when a seeded file is inside
it. An option value that only resolves under the home stays as written.

* fix(desktop): keep a program word and a directory's neighbor

The program after env stays the program even when a captured file sits in a
folder of that name. A selected directory is not a database, so a neighboring
file keeps the parent folder's spelling.

* fix(desktop): record whether a selected path is a directory

Capture stores the walked path's kind. A neighboring file of a selected
directory keeps the parent folder's spelling even when that directory never
changes and its children are not content roots.

* fix(desktop): mark containing paths without a pairwise scan

A recorded path marks its known ancestors as directories in one walk up
the path, instead of comparing every recorded path with every other.

* fix(desktop): keep a selected folder that is named like a database log

A folder the caller selected stays a selected root when its name ends in
-journal or -wal. A generated log beside a database file still does not.

* fix(desktop): refuse a build command that exceeds the argument vector

Splitting one argument still leaves the whole command under Linux's E2BIG
limit. A request past that budget is refused before the process starts.

* fix(desktop): remap a captured path in an env assignment

The program word stays the program. A NAME=value assignment keeps its name
and follows a captured path value when that path's spelling moves.

* fix(desktop): keep oversized text out of draft guards

A value or line longer than the draft's text bound is not copied into a
guard, so one large frame cannot fill the task by itself.

* docs(desktop): correct dconf and check-preview guidance

A dconf version kept without its bytes is unreadable, not an empty key
list. A values preview is copied into a check only when it is an uncut scalar.

* fix(desktop): keep an empty env value empty

An empty assignment is not a path. Resolving it named the working directory
and rewrote the value when that directory's spelling moved.

* fix(desktop): keep a run's font on the run, not the frame's text

A formatting attribute such as Scribus FONT is not the document's words.
Joining it across runs made a frame-wide text check of ItalicBold that
failed when a run was split. The font stays a value check on its own run.

* fix(desktop): do not pair JSON records by a selector-shaped field

A field named @record_id matched the id suffix and became [@record_id=one],
which reads an XML attribute. Records paired while their pointers matched
nothing. Names that start with @ or > are not id fields.

* fix(desktop): count a save-numbered id once when pairing members

xml:id and draw:id values such as id123456 were both generated and
numbered, so the same selector was counted twice and refused. A text
edit of two elements keyed only by those ids now keeps its check.

* fix(desktop): preserve selected state directory aliases

* fix(desktop): align nested state selections with aliases

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): exclude unseeded locks from task checks

* fix(desktop): preserve draft bounds and launch URLs

* fix(desktop): render draft values using their source format

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve captured URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* fix(desktop): classify added line values in one pass

* fix(desktop): apply XML quote rules to text drafts

* fix(desktop): distinguish XML attributes from quoted text

* fix(desktop): preserve parsed XML line context

* fix(desktop): retain partial XML syntax facts

* fix(desktop): recognize markdown autolinks in drafts

* fix(desktop): reuse export exclusions in task drafts

* docs(desktop): clarify gzip scrub limits

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* fix(desktop): exclude covered text runs from draft counts

* fix: preserve text when drafting desktop checks

* fix: restore nested desktop state aliases

* refactor: share UUID recognition across draft checks

* fix: retain multiline XML attribute context

* fix: retain relationship identifiers in task drafts

* docs: clarify desktop draft identifier handling

* test(desktop): type formatting run cases as pairs

* test(desktop): separate markup boundary regression group

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* fix: align desktop numeric seed comparisons

* test(desktop): keep one draft exclusion fixture import

* test(desktop): retain log eligibility across draft test groups

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix: retain quoted identifier text in task drafts

* fix: recognize Unicode XML in desktop state readers

* fix(desktop): bound nested draft checks

* fix(desktop): explain missing arrays in truncated views

* fix(desktop): find XML type references in archives

* test(desktop): remove duplicated capture assertions

* fix(desktop): retain nested omitted text boundaries

* style(desktop): format draft value imports

* test(desktop): group model and draft value checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): retain placed-item sibling guards

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* style(desktop): format capture review fixes

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 22:56:44 +08:00
0dcd0dfc20 feat(desktop): check text kept in attributes, name a record by a child's text, and count members (#9114)
* fix(desktop): keep a lone lock a document and drop one beside its file

A selected name such as notes.lck stays a document, so launch and the draft
still treat it as the caller's file. A lock beside a recorded sibling is left
out of the seed unless that lock path itself was selected.

* fix(desktop): require the added record and leave save times out of list checks

A record a selector names is checked itself, so its words have to be in that
record. When an unnamed record only repeats words the seed already has, a
list that holds a save time is checked by the text around that time. A later
save of the same records still passes, and a missing or rewritten record fails.

* fix(desktop): judge the older index helper by pointer segments only

A leading zero in a file name or a URL does not change how the older index
helper reads a pointer. The witness still rejects that helper when a pointer
segment is a number such as 01.

* fix(desktop): keep a document whose name ends in .lock

A lock beside a recorded file still stays out of the seed unless that lock
was selected. A document such as Cargo.lock in the home is the user's file,
so the seed keeps it. An application lock in the home, such as poedit.lock,
stays out.

* fix(desktop): do not suggest list text the seed already passes

A repeated record is shown by text the seed does not hold. The seed's own
text is not a check. A pointer inside a check value is not a pointer segment
for the older index helper.

* fix(desktop): do not let a left-out lock hide settings checks

A sibling lock the seed omits is not the document the draft follows. Settings
changes stay suggested. The unused sibling-lock name helper is removed.

* fix(desktop): keep a linked settings file in the folder the link points at

A selected ~/.config alias no longer moves MuseScore settings to that alias.
The seed keeps settings/MuseScore or Documents, which is the path the export
opens the note from.

* fix(desktop): split home fetches by answer size and keep a size refusal

A batch of tiny files stops before each path's envelope exceeds the host
answer budget. A capture output past that ceiling stays a resource error
instead of becoming a generic exit.

* fix(desktop): seed a selected link under the path the app uses

The selected spelling wins, so ~/.config/MuseScore is not rewritten to the
folder it points at and the note stays the one launched document. A database
wal and journal follow that database. A launch argument is respelled the same
way. Teardown signals a detached PID only when its start identity still matches.

* fix(desktop): keep a witness's actions and an incomplete checkpoint

A failed witness save retries the action log it first recorded. A deletion
left unconfirmed between the seed and the end is not reported as no change.
Windows packs npm through a shell, which can start npm.cmd.

* fix(desktop): read verifier pointers from the parsed checks

A brace inside a check value is not an object boundary. A leading-zero
pointer after that value still rejects the older index helper.

* fix(desktop): leave out a sibling lock under a selected folder

A file under a selected folder inherits selected_path. That does not make
the lock beside its map a chosen document, so a settings change stays drafted.

* fix(desktop): prefer the most specific home spelling and retarget launch files

A selected parent directory no longer hides a settings folder walked through
a link. A launch filename under that folder is replaced with the seeded
file's absolute path.

* fix(desktop): pack the build runtime through npm's JavaScript entry

npm pack and npm ci no longer spawn npm.cmd through a shell, so a temporary
directory whose path contains spaces stays one argument.

* fix(desktop): spell launch settings and keep a database log exact

A database log matches only that file, so a neighboring folder is not moved.
A launch option and the working directory use the same home spelling as the seed.

* fix(desktop): draft a lock the export selected exactly

The draft uses the same selected paths as the export. A lock chosen by
itself stays in the draft. A lock under a selected folder stays out.

* fix(desktop): move a state database's logs with the database

The write-ahead log and journal follow the spelling the database received,
including a walked state link. A neighboring folder stays put. A separate
--config argument uses that spelling too.

* fix(desktop): keep a selected file ahead of a database-log guess

A file selected as itself keeps that spelling even when its name ends in
-journal. Relative launch arguments are resolved in the captured working
directory before that directory moves.

* fix(desktop): keep outside-home database logs and option values

A selected database outside the home keeps its write-ahead log. Launch
options are rewritten only when they name a captured file or settings path.

* fix(desktop): follow a database mapping and own limitation text

A selected database keeps its log when a parent folder moves or the file
sits outside the home. Trash and seed-lock tests compare the owning notice
with the paths, and launch options that are not captured paths stay as written.

* fix(desktop): retarget a settings folder that holds captured files

A nested profile directory is a captured path when a seeded file is inside
it. An option value that only resolves under the home stays as written.

* fix(desktop): keep a program word and a directory's neighbor

The program after env stays the program even when a captured file sits in a
folder of that name. A selected directory is not a database, so a neighboring
file keeps the parent folder's spelling.

* fix(desktop): record whether a selected path is a directory

Capture stores the walked path's kind. A neighboring file of a selected
directory keeps the parent folder's spelling even when that directory never
changes and its children are not content roots.

* fix(desktop): mark containing paths without a pairwise scan

A recorded path marks its known ancestors as directories in one walk up
the path, instead of comparing every recorded path with every other.

* fix(desktop): keep a selected folder that is named like a database log

A folder the caller selected stays a selected root when its name ends in
-journal or -wal. A generated log beside a database file still does not.

* fix(desktop): refuse a build command that exceeds the argument vector

Splitting one argument still leaves the whole command under Linux's E2BIG
limit. A request past that budget is refused before the process starts.

* fix(desktop): remap a captured path in an env assignment

The program word stays the program. A NAME=value assignment keeps its name
and follows a captured path value when that path's spelling moves.

* fix(desktop): keep oversized text out of draft guards

A value or line longer than the draft's text bound is not copied into a
guard, so one large frame cannot fill the task by itself.

* docs(desktop): correct dconf and check-preview guidance

A dconf version kept without its bytes is unreadable, not an empty key
list. A values preview is copied into a check only when it is an uncut scalar.

* fix(desktop): keep an empty env value empty

An empty assignment is not a path. Resolving it named the working directory
and rewrote the value when that directory's spelling moved.

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* fix(xml): avoid scanning selected ancestors per tag

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* style(desktop): format capture review fixes

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 22:53:18 +08:00
35f1c3517f feat(desktop): install extra packages beside an archive app, and keep a package file's dependencies at export (#9112)
* fix(desktop): keep added-record checks off ids, dates, and joined timestamps

A position-named record's fields keep their paths, so an identifier, a stamped date, and text past the limit are not suggested checks. A whole-text check does not climb into an ancestor whose text joins a timestamp.

* fix(desktop): bound home fetches and keep unsettled edits readable

Split home-folder reads by the guest answer budget, keep a capacity error
from the capture output, and drop the superseded task-argument refusal.
Grade kept text per line, leave unretained dconf unread, and reserve
by-position notes for XML children. Map a linked database's sidecars
through the selected spelling and stop a detached capture only when its
start identity still owns the pid.

* fix(desktop): describe unretained dconf as unread

The key comparison treats a missing copy as unknown. The comment still
said that copy counted as an empty key list.

* test(desktop): keep a capture capacity error through endedCapture

An output past the guest answer ceiling stays the resource constraint,
with its heap recovery and non-retryable status, instead of a generic
capture-exit message.

* fix(desktop): keep nested save times and formatting runs out of checks

A field check drops save-time leaves before it joins the field, so a nested
timestamp is not part of the suggested text. A text check beside a save time
uses the paragraph when the diff names a formatting run.

* test(desktop): read paragraph pointers from checks that have them

* test(desktop): expect a large task to export intact

The 96 KiB refusal is gone. A prompt past that size reaches the computer
and comes back unchanged, instead of failing before export.

* fix(desktop): keep real edits that save-side heuristics were dropping

Repeated unnamed fields stay content checks. A formula that still has
text is not renumbered with its ExprID. One gained style attribute is
the edit; a repeated fill-in stays upkeep. Phone numbers and plain
clock times stay text, and a decimal is the element's whole text. An
older verifier matches on the task it states, not its evaluator source.

* fix(desktop): cover repeated records and keep timestamps out of checks

An added record whose words are already in the seed still gets the list's
text when that text fits. Readiness is a start reason, so a final check
names seed locks without matching the message. Field text split by a
timestamp stays separate, and a timestamp line is not offered as a guard.

* fix(desktop): compare a cell's joined decimal within tolerance

The whole-element check keeps numeric tolerance, so 437.80 and a long
float still match 437.8, including when a run splits the digits. A
different number or other text does not. Both evaluators read an
element's text for that comparison.

* fix(desktop): keep selected logs and stop treating unknown walks as complete

Match an app name as a whole token, treat an omitted unchecked_paths count as incomplete, seed Chromium user_prefs.json, and keep a caller-selected log even when that path is also a state directory. A log inside the app's own folder still marks that folder so settings beside it stay seeded, while a lock in the home does not. Sibling locks under a selected folder stay out unless that lock was the selected path.

* fix(desktop): grade a number from a complete XML element only

A tolerance check joins an element's text only in an XML file or an XML
archive entry, and leaves that check undecided when the view is cut, so a
dropped descendant cannot turn the kept runs into a passing number. A JSON
object with the same shape stays the value itself. Equivalent decimals and a
split run on a whole cell still match.

* fix(desktop): accept an older grader only when its index helper agrees

The witness still matches a bundle whose evaluator is this REA's, or the older
index expression alone when no path segment is a leading-zero number. A
truncated verifier, a changed grader, or that older helper beside a segment
such as 01 no longer counts as the same task.

* fix(desktop): keep a lone lock a document and drop one beside its file

A selected name such as notes.lck stays a document, so launch and the draft
still treat it as the caller's file. A lock beside a recorded sibling is left
out of the seed unless that lock path itself was selected.

* fix(desktop): require the added record and leave save times out of list checks

A record a selector names is checked itself, so its words have to be in that
record. When an unnamed record only repeats words the seed already has, a
list that holds a save time is checked by the text around that time. A later
save of the same records still passes, and a missing or rewritten record fails.

* fix(desktop): judge the older index helper by pointer segments only

A leading zero in a file name or a URL does not change how the older index
helper reads a pointer. The witness still rejects that helper when a pointer
segment is a number such as 01.

* fix(desktop): keep a document whose name ends in .lock

A lock beside a recorded file still stays out of the seed unless that lock
was selected. A document such as Cargo.lock in the home is the user's file,
so the seed keeps it. An application lock in the home, such as poedit.lock,
stays out.

* fix(desktop): do not suggest list text the seed already passes

A repeated record is shown by text the seed does not hold. The seed's own
text is not a check. A pointer inside a check value is not a pointer segment
for the older index helper.

* fix(desktop): do not let a left-out lock hide settings checks

A sibling lock the seed omits is not the document the draft follows. Settings
changes stay suggested. The unused sibling-lock name helper is removed.

* fix(desktop): keep a linked settings file in the folder the link points at

A selected ~/.config alias no longer moves MuseScore settings to that alias.
The seed keeps settings/MuseScore or Documents, which is the path the export
opens the note from.

* fix(desktop): split home fetches by answer size and keep a size refusal

A batch of tiny files stops before each path's envelope exceeds the host
answer budget. A capture output past that ceiling stays a resource error
instead of becoming a generic exit.

* fix(desktop): seed a selected link under the path the app uses

The selected spelling wins, so ~/.config/MuseScore is not rewritten to the
folder it points at and the note stays the one launched document. A database
wal and journal follow that database. A launch argument is respelled the same
way. Teardown signals a detached PID only when its start identity still matches.

* fix(desktop): keep a witness's actions and an incomplete checkpoint

A failed witness save retries the action log it first recorded. A deletion
left unconfirmed between the seed and the end is not reported as no change.
Windows packs npm through a shell, which can start npm.cmd.

* fix(desktop): read verifier pointers from the parsed checks

A brace inside a check value is not an object boundary. A leading-zero
pointer after that value still rejects the older index helper.

* fix(desktop): leave out a sibling lock under a selected folder

A file under a selected folder inherits selected_path. That does not make
the lock beside its map a chosen document, so a settings change stays drafted.

* fix(desktop): prefer the most specific home spelling and retarget launch files

A selected parent directory no longer hides a settings folder walked through
a link. A launch filename under that folder is replaced with the seeded
file's absolute path.

* fix(desktop): pack the build runtime through npm's JavaScript entry

npm pack and npm ci no longer spawn npm.cmd through a shell, so a temporary
directory whose path contains spaces stays one argument.

* fix(desktop): spell launch settings and keep a database log exact

A database log matches only that file, so a neighboring folder is not moved.
A launch option and the working directory use the same home spelling as the seed.

* fix(desktop): draft a lock the export selected exactly

The draft uses the same selected paths as the export. A lock chosen by
itself stays in the draft. A lock under a selected folder stays out.

* fix(desktop): move a state database's logs with the database

The write-ahead log and journal follow the spelling the database received,
including a walked state link. A neighboring folder stays put. A separate
--config argument uses that spelling too.

* fix(desktop): keep a selected file ahead of a database-log guess

A file selected as itself keeps that spelling even when its name ends in
-journal. Relative launch arguments are resolved in the captured working
directory before that directory moves.

* fix(desktop): keep outside-home database logs and option values

A selected database outside the home keeps its write-ahead log. Launch
options are rewritten only when they name a captured file or settings path.

* fix(desktop): follow a database mapping and own limitation text

A selected database keeps its log when a parent folder moves or the file
sits outside the home. Trash and seed-lock tests compare the owning notice
with the paths, and launch options that are not captured paths stay as written.

* fix(desktop): retarget a settings folder that holds captured files

A nested profile directory is a captured path when a seeded file is inside
it. An option value that only resolves under the home stays as written.

* fix(desktop): keep a program word and a directory's neighbor

The program after env stays the program even when a captured file sits in a
folder of that name. A selected directory is not a database, so a neighboring
file keeps the parent folder's spelling.

* fix(desktop): record whether a selected path is a directory

Capture stores the walked path's kind. A neighboring file of a selected
directory keeps the parent folder's spelling even when that directory never
changes and its children are not content roots.

* fix(desktop): mark containing paths without a pairwise scan

A recorded path marks its known ancestors as directories in one walk up
the path, instead of comparing every recorded path with every other.

* fix(desktop): keep a selected folder that is named like a database log

A folder the caller selected stays a selected root when its name ends in
-journal or -wal. A generated log beside a database file still does not.

* fix(desktop): refuse a build command that exceeds the argument vector

Splitting one argument still leaves the whole command under Linux's E2BIG
limit. A request past that budget is refused before the process starts.

* fix(desktop): remap a captured path in an env assignment

The program word stays the program. A NAME=value assignment keeps its name
and follows a captured path value when that path's spelling moves.

* fix(desktop): keep oversized text out of draft guards

A value or line longer than the draft's text bound is not copied into a
guard, so one large frame cannot fill the task by itself.

* fix(desktop): keep an empty env value empty

An empty assignment is not a path. Resolving it named the working directory
and rewrote the value when that directory's spelling moved.

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* fix(desktop): preserve explicitly selected journal documents

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* fix(desktop): retain app state logs in exported seeds

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* docs(desktop): clarify recovered seed format

* test(desktop): group shipped gzip seed checks

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* docs(desktop): clarify unreadable dconf versions

* style(desktop): format capture review fixes

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 22:51:50 +08:00
Evtim Kostadinovandmorluto eb8ba20c35 docs(i18n): add Bulgarian README (#9656)
* docs(i18n): add Bulgarian README

Add a complete Bulgarian (bg) translation of the README and list it in
every README language bar and the website language inventory.

Register bg with a pending website catalog that uses the labeled,
nonindexable English fallback, with the fallback notice translated.
Regenerate the shared language picker and hreflang links and refresh
messages.pot.

* docs(i18n): clarify Bulgarian project code license scope

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 09:13:49 -05:00
Nianze Wuandmorluto bdf7cc3b1e fix(pe): reject malformed certificate name structures (#9625)
* fix(pe): reject malformed certificate name structures

* fix(ci): bound the complete Kuna verifier workflow

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 22:09:34 +08:00
吴佳浩Albenandmorluto 3b2522bd3b fix(javascript): reject a tls protocol version Node refuses at the call site (#9557)
* fix(javascript): reject a tls protocol version Node refuses at the call site

`tls.connect` resolves `minVersion`/`maxVersion` before it opens a socket, so a bad literal
throws from the call itself and no socket exists. The acquisition filter decided the port
only, so it reported a complete built-in resource for a call that throws.

Measured on Node 24.22.1:

| `minVersion` / `maxVersion` | outcome |
| --- | --- |
| `"TLSv1"`, `"TLSv1.1"`, `"TLSv1.2"`, `"TLSv1.3"` | socket created |
| `null`, `undefined`, absent | socket created |
| `"TLSv1.0"`, `"tlsv1"`, `"tlsv1.2"`, `"TLSv1.4"`, `"nope"`, `""` | THREW `ERR_TLS_INVALID_PROTOCOL_VERSION` |
| `5`, `true`, `{}`, `[]` | THREW `ERR_TLS_INVALID_PROTOCOL_VERSION` |
| `1n` | THREW `TypeError` |

The accepted set is exact and case-sensitive, so it is a closed list rather than a pattern.
`net` ignores these members, so only `tls` is consulted. The member is read in write order,
which is why `{minVersion: "nope", minVersion: "TLSv1.2"}` keeps its acquisition; a spread,
a computed key or an accessor leaves the answer undecided rather than decided against.

22 cases, 11 failing before this commit. `src/domain/javascript`: 5040 to 5062 passing.
`tsc` 0; `build` 0; `npm run format:check` 0; `npm run lint` 0 errors; `knip` 0.

* fix(javascript): read every tls options slot and check both version members

Four review findings, each measured on Node 24.22.1. All four were real gaps in the first
version of this check.

**An accepted `minVersion` returned before `maxVersion` was read.** `tls.connect({port: 80,
minVersion: "TLSv1.2", maxVersion: "nope"})` throws, because both members are resolved
before the socket opens. The loop now continues past an accepted version and returns only
for a rejected one.

**Only argument zero was read.** `tls` also accepts `(port, options)`, `(port, host,
options)` and `(path, options)`, and all three throw on a bad version:
`tls.connect(443, {minVersion: "nope"})`, `tls.connect(443, "127.0.0.1", {minVersion:
"nope"})` and `tls.connect("/tmp/x.sock", {minVersion: "nope"})`. Every object literal in
the argument list is now read; a spread argument shifts those positions, so it leaves the
answer undecided instead of picking a wrong slot.

**A signed number was left unknown.** `minVersion: -1`, `+1` and `-0` all throw, but they
arrive as a `UnaryExpression` and matched none of the literal arms. `readExactJavaScriptLiteral`
already normalises `-1` and `+1`; `-0` is the one case it excludes, so a sign over a numeric
literal is decided explicitly here rather than treated as unknown.

13 cases added; reverting the slot and loop changes fails 4 of them.
`src/domain/javascript`: 5062 to 5075 passing. `tsc` 0; `build` 0;
`npm run format:check` 0; `npm run lint` 0 errors; `knip` 0.

* fix(javascript): merge the tls arguments and honour a supplied secure context

Four review findings, each measured on Node 24.22.1. All four were real: the previous
reader checked each object argument independently and ignored the context.

**Several object arguments merge in order, so the later one wins.** The reader rejected the
first object and stopped:

| call | Node 24.22.1 |
| --- | --- |
| `tls.connect({port: 443, minVersion: "nope"}, {minVersion: "TLSv1.2"})` | SOCKET |
| `tls.connect({port: 443, minVersion: "TLSv1.2"}, {minVersion: "nope"})` | THREW |
| `tls.connect({port: 443}, {minVersion: "nope"})` | THREW |

The members are now resolved across the whole argument list in write order, so a later
argument replaces an earlier one the way Node's merge does.

**A supplied `secureContext` bypasses version parsing entirely.** Node uses
`options.secureContext || tls.createSecureContext(options)`, so
`tls.connect({port: 80, secureContext: ctx, minVersion: "nope"})` connects while a `null`
context throws. An unreadable context leaves the answer undecided.

**An ordinary method member cannot install a version.** The reader skipped any object
containing a method, so `{checkServerIdentity() {}, minVersion: "nope"}` kept a socket
Node never creates. Only an accessor runs a body and leaves the member open; a plain method
is a value Node never reads as a version name. A `minVersion() {}` member is itself such a
value, so it rejects.

**Signed BigInt and function-shaped versions were left unknown.** `-1n`, `() => {}`,
`function () {}` and `class {}` are all truthy non-strings that Node refuses; none matched
an existing arm. They are now decided.

12 cases added; reverting the merge and context changes fails 2 of them and reverting the
method arm fails 6. `src/domain/javascript`: 5075 to 5087 passing. `tsc` 0; `build` 0;
`npm run format:check` 0; `npm run lint` 0 errors; `knip` 0.

* fix(javascript): read only the options slot tls.connect copies, and its accessors

Six review findings, each measured on Node 24.22.1. All six were real.

**A falsy `secureContext` does not bypass version parsing.** Node tests
`options.secureContext` for truth, so `{secureContext: false, minVersion: "nope"}`, and `0`
and `""` with it, all throw. The guard only recognised nullish syntax, so it dropped real
throws. `staticTruthiness` now decides the context: a known-falsy value keeps checking, a
known-truthy one ends the check, and an unreadable one stays undecided.

**Node copies at most one options object.** Measured:

| call | result |
| --- | --- |
| `(443, {minVersion: "TLSv1.2"}, {minVersion: "nope"})` | SOCKET — the third argument is ignored |
| `(443, {minVersion: "nope"}, {minVersion: "TLSv1.2"})` | THREW |
| `(443, "127.0.0.1", {minVersion: "nope"})` | THREW — the host fills slot 1 |
| `({port: 443, minVersion: "nope"}, {minVersion: "TLSv1.2"})` | SOCKET — the second object wins |

The reader treated every object argument as merged in order, which let an ignored object
decide. `tlsOptionsObjects` now models the two overloads: an object in slot 0 is merged with
slot 1, and a port or path in slot 0 makes the first object among slots 1 and 2 the only one
read.

**An exact computed key names the member.** `{["minVersion"]: "nope"}` throws, but the
reader marked every computed key unknown before the shared name resolver saw it. Only an
unresolvable key is unknown now, and the case that asserted the old behaviour uses a
binding key so it still covers the unknown path.

**An accessor can write the member first.** `{get side() { this.minVersion = "TLSv1.2"; },
minVersion: "nope"}` connects, because Node's own copy runs the getter before it reads
`minVersion`; and a `set minVersion` in an earlier argument absorbs the later write. Both
are bodies this reader does not run, so an accessor anywhere in the merged objects now
leaves every version member undecided. A plain method still rejects, since Node never reads
it as a version name.

**An unshadowed `NaN` or `Infinity` is a non-string version.** Both throw, and neither
matched an existing arm. They are decided the same way the adjacent port filter decides
them, through `isUnmodifiedGlobalReference`, so a shadowed binding stays unknown.

16 cases added; reverting the truthiness, accessor and computed-key changes fails 2 of them.
`src/domain/javascript`: 5075 to 5188 passing. `tsc` 0; `build` 0;
`npm run format:check` 0; `npm run lint` 0 errors.

* fix(javascript): use the nullable key owner and settle unary versions

Six review findings, each measured on Node 24.22.1. All six were real, and three of them
were removing real sockets.

**`semanticStaticPropertyName` encodes an unresolved key as `""`, never `undefined`.** The
guard tested `undefined`, so it never fired and an unresolved computed key was read as a
member literally named empty. `const k = "minVersion";
tls.connect({port: 443, minVersion: "nope", [k]: "TLSv1.2"})` connects on Node 24.22.1
because the later computed member wins, while the analysis filtered on the stale `"nope"`
and dropped the socket. `semanticStaticPropertyKey` is the nullable owner and is now used,
with `null` treated as unknown.

**A getter can replace a plain version method before Node copies it.** The method branch
ignored `accessorWritten`, so `{get side() { this.minVersion = "TLSv1.2"; }, minVersion()
{}}` still returned `"rejects"` although it connects. The same unknown result now applies to
methods when an accessor is present.

**A non-literal second argument is a merged options object, not an absent one.**
`const o = {minVersion: "TLSv1.2"}; tls.connect({port: 443, minVersion: "nope"}, o)`
connects. The slot reader mapped every non-object to `undefined`, so only the stale first
object was examined. A non-object in a copied slot now stays unknown.

**`-0` is falsy, so a signed-zero context does not bypass parsing.** The local truthiness
helper did not fold a unary `-`, so `{secureContext: -0, minVersion: "nope"}` was treated as
a possible bypass although it throws. The shared `staticJavaScriptTruthiness` owner already
handles signed zero and replaces the local helper, which is deleted.

**A unary operator over a settled operand is a settled non-string version.** `!0`, `!1`,
`~~0`, `!!0` and `typeof 1` all throw `ERR_TLS_INVALID_PROTOCOL_VERSION`; the previous arm
recognised only `+` and `-`, and the exact-literal and shape arms did not reach them.
`void <expr>` stays accepted, because it is `undefined`.

The slot reader also had to distinguish a host string in slot 1 from an options object, since
`tls.connect(443, "127.0.0.1", {minVersion: "nope"})` throws while
`tls.connect(443, {minVersion: "nope"})` is the two-slot form.

`src/domain/javascript`: 5188 to 5197 passing. Reverting the key owner fails 1 case and
reverting the unary arms fails 2. `tsc` 0; `build` 0; `npm run format:check` 0;
`npm run lint` 0 errors.

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(javascript): share TLS option facts and preserve merge uncertainty

* fix(javascript): separate TLS assignment authority from copy facts

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 22:09:29 +08:00
8bc01b48b4 feat(native): list ELF and Mach-O initializer and finalizer functions (#9574)
* feat(native): list ELF and Mach-O initializer and finalizer functions

inspect_native_initializers and inspect-native-initializers list the
functions an ELF or Mach-O image runs before and after main: ELF
DT_INIT, DT_FINI, preinit, init and fini arrays and .ctors/.dtors, and
Mach-O __mod_init_func, __mod_term_func and __init_offsets, with slot
addresses, resolved targets, symbols where available and relocation
facts for relocatable objects. Nothing is executed. Slot, symbol,
relocation and note budgets are shared across tables, sentinels are
matched at the image width, and target-derived names are bounded and
marked untrusted.

* fix(native): keep ELF initializer slots unresolved when relocation coverage is partial

A skipped, unreadable or over-budget section relocation table now keeps
every slot it could reach unresolved instead of reading the stored word as
raw or dropping a .ctors slot as a sentinel. Dynamic writes that only
partly overlap a slot, or write it twice, leave it unresolved. Linked
images no longer reapply retained non-SHF_ALLOC link-time records.

The dynamic relocation walk now pauses at the shared reader's step
boundaries and observes cancellation there, resolved targets need only
their first byte mapped, and unreadable symbol names make the report
incomplete.

* fix(native): stop reporting unchecked Mach-O initializer words as resolved

Object files now apply their own section relocations: an UNSIGNED entry
against a defined symbol gives symbol_addend, one against an undefined
symbol gives unresolved_import, and any other entry leaves the slot
unresolved. Linked images without a fixup decoder, such as 32-bit ones,
keep pointer slots unresolved when an LC_DYLD_INFO bind stream or
LC_DYSYMTAB external relocations could replace the stored word.

LC_DYLD_INFO rebases now report dyld_info_rebase instead of
chained_fixup, and unreadable symbol names make the report incomplete.
Section headers keep their declared reloff and nreloc for this.

* docs(native): move initializer guide under docs/guides/native

The documentation guide assigns analyst workflows to docs/guides/<domain>/.
Move the guide there, place its navigation and directory entries in the
Native binaries group, and update links from the docs index, CLI guide,
spec and authored skill. The public /native-initializers route is
unchanged.

The guide and spec also describe per-slot relocation coverage, retained
link-time records, Mach-O object relocations, undecoded bind streams,
the dyld_info_rebase resolution and unreadable names.

* fix(native): keep IFUNC and partly relocated ELF initializer slots unresolved

An absolute relocation against an STT_GNU_IFUNC symbol no longer reports
the resolver address as the target. Section relocation records now use
their real write width, so a record that writes only part of a slot,
starts inside it or repeats it leaves the slot unresolved instead of
reading a raw zero or dropping a .ctors slot as a sentinel.

On 32-bit ARM the Thumb bit is cleared when locating a target's section
and file offset, while value_va keeps the stored callable value.

* fix(native): locate Thumb Mach-O initializer targets and reject arm64_32 with a 64-bit header

On CPU_TYPE_ARM the Thumb bit is cleared when locating and naming a
target, since nlist values do not carry it. An arm64_32 image that
declares a 64-bit header is rejected as malformed rather than read with
8-byte slots. The guide and spec describe IFUNC relocations, partly
relocated section slots, Thumb targets and this rejection.

* fix(native): index initializer address lookups and keep ambiguous mappings unknown

ELF and Mach-O initializer lookups now build the shared file mapping
index once per image, so each slot and target lookup is logarithmic in
the number of load segments rather than a scan of every mapping.

An address that load segments map ambiguously no longer falls back to a
section header offset. Its file offset is reported as unknown and the
report is incomplete; only genuinely unmapped spans use the section
fallback.

* fix(native): report Mach-O bind names over the name budget

A bound import name longer than the 1024-byte name budget was dropped
without a trace, although it is the only identity of an unresolved
initializer target. It now counts toward the over-long name limitation
and makes the report incomplete.

* test(native): count mapping lookups instead of timing the load index regression

The load-mapping regression asserted a wall-clock bound, which is
sensitive to machine load. It now spies on the shared mapping module and
requires the same number of mapping scans and index builds for 8 and 64
slots, so a per-slot scan of every load segment fails deterministically
on a small fixture.

* fix(native): yield while decoding legacy rebase runs

* fix(native): preserve initializer mapping and coverage

* fix(native): index legacy fixup spans

* fix(native): preserve relocation uncertainty and bound bind lookup work

* fix(native): validate ELF slot mappings and yield section relocation scans

---------

Co-authored-by: Matt Van Horn <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 08:58:12 -05:00
morluto deae5fbe6d docs: correct workflow claims and clarify historical specifications (#9655)
* docs: correct lifecycle and analysis workflow claims

* docs: distinguish historical specifications from current guides

* docs: align translated setup and MCP recovery guidance
2026-11-09 08:54:23 -05:00
fa0fcf35ce fix(desktop): resolve desktop entry ids that name a nested entry (#9563)
* fix(desktop): resolve desktop entry ids that name a nested entry

The desktop file id of an entry below an applications folder replaces
the directory separators with dashes, so applications/vendor/foo.desktop
advertises vendor-foo.desktop. locateDesktopEntry only probed the flat
path, so such an entry was either not found or, when a lower-priority
XDG root installed a flat file with the same id, a different
application was launched under the requested id.

List each applications directory instead and form every entry's id from
the path that holds it, so a dash inside a directory name does not
change the answer. The walk is bounded and stops reporting absence once
it can no longer decide, so a truncated search refuses instead of
falling through to a lower-priority root.

* fix(desktop): search nested entry directories without skipping answers

Three defects in findNestedDesktopEntry, all of which could select a lower-priority application under the same desktop file id:

`readdir` failure was treated as an empty directory, so an applications root or nested directory that could not be listed silently yielded to a lower-priority root. Only ENOENT and ENOTDIR now mean the root is not installed; any other failure reports desktop_entry_search_incomplete.

A directory whose own name ends in `.desktop` was skipped before it was searched, so `vendor.desktop/app.desktop` (id `vendor.desktop-app.desktop`) was unreachable. The suffix now selects a candidate only for files; directories still descend.

The entry budget was applied after `readdir` had materialized every name, so a large directory allocated far more than the 4096 entries it may examine. The walk now enumerates with `opendir` and charges the budget per entry.

* fix(desktop): finish the reachable nested search before reporting a gap

An unreadable or over-budget subtree is not proof the entry is absent, but
returning truncated as soon as one was met stopped the walk before the
siblings that could still hold it. The entry then looked incomplete even
though this root could answer, so a lower-priority root was never consulted
and the search reported a gap instead of the entry.

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(desktop): resolve only applicable desktop ID paths

* test(desktop): cover applicable probe failure before a matching spelling

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 21:47:20 +08:00
217b6fdcdc feat(cli): expose provider selection diagnostics in session status (#9603)
* feat(cli): expose provider selection diagnostics in session status

The registry now records how each deep provider became selected and why a
candidate was rejected, but the session status projection dropped both
fields. CLI providers/capabilities and MCP binary_session therefore showed
a candidate list that could not explain an automatic or explicit choice,
so users had to re-derive the reason from prose in selection errors.

Project selection_source and rejections for every candidate. Complete detail
keeps the rejection remediation and diagnostics; summary keeps the stable
origin, code and reason and lists the dropped fields in omitted_fields.
Existing candidate fields are unchanged, so both projections stay additive.

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(session): preserve binding source in refreshed candidates

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 21:47:15 +08:00
Harsh Asalkarandmorluto 78343052c2 fix(mcp): narrow union branch diagnostics to matching discriminators (#9481) (#9587)
* fix(mcp): narrow union branch diagnostics to matching discriminators

* fix(mcp): merge union expected options at identical diagnostic paths

* fix(mcp): guide cross-kind sub-selector mistakes on inspect_analysis_view

* fix(mcp): support heterogeneous strict object variants in union discriminator narrowing

* fix(mcp): resolve union discriminators before narrowing and preserve nested alternatives

* fix(mcp): resolve setting discriminator before narrowing and preserve invalid_type failures

* fix(mcp): narrow branches by matched discriminators before synthesizing diagnostics

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(input): preserve selector failures within projection bounds

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 21:47:09 +08:00
Cipherandmorluto b105dae7c6 test(mcp): pin Kimi/Moonshot-accepted input schemas for #9581 (#9592)
* test(mcp): pin Kimi/Moonshot-accepted input schemas for #9581

Closes #9581

#9581 reports Kimi/Moonshot rejecting 9 tools whose input schemas
carried a $ref to a oneOf/anyOf with a nested $ref. The inlined
advertisement that fixes this is already on main (reused: "inline"
in toolInputSchemaWithMetadata, 554816be, unreleased): all 246
advertised input schemas are reference-free in both profiles. This
adds the regression coverage that pins it: the issue's minimal
trigger through the compact presenter, and the rejected reference
pattern checked across every served input schema in both profiles,
naming the nine tools from the issue.

Not verified against live Kimi/Moonshot (no access); verified locally
that no advertised input schema in either profile contains the
rejected pattern.

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* test(mcp): qualify reference detector and semantic inlining

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 21:47:03 +08:00
吴佳浩Albenandmorluto 7a1d89a801 fix(javascript): reject child-process option literals Node refuses at the call site (#9594)
* fix(javascript): reject a spawn stdio string Node refuses at the call site

`node:child_process` validates a literal `stdio` string before it creates anything, so a
bad value throws from the call itself and no child process exists. The analysis reported the
raw string as `stdioMode` and kept the spawn.

Measured on Node 24.22.1:

| `stdio` | outcome |
| --- | --- |
| `"pipe"`, `"overlapped"`, `"inherit"`, `"ignore"` | child starts |
| `"ipc"`, `"PIPE"`, `"pipe "`, `"pipeline"`, `"stream"`, `"stdio"` | THREW `ERR_INVALID_ARG_VALUE` |

The accepted set is exact and case-sensitive, so it is a closed list rather than a pattern.
`"ipc"` is the interesting one: it is valid as an array element but refused as the scalar
shorthand, which is the shape a caller reaches for when they mean the array form.

`fork` differs on one value because it appends `"ipc"` to a string and splits it, so `""`
becomes the lone element `"ipc"` and is refused:

| call | outcome |
| --- | --- |
| `spawn(..., {stdio: ""})` | child starts |
| `fork(..., {stdio: ""})` | THREW `ERR_INVALID_ARG_VALUE` |

`exec` is left alone, because it spreads its options and validates none of them. A number,
boolean or object `stdio` is also refused by Node, but `stdioMode` already reports those as
`"dynamic"`; deciding them here would claim more than the reported mode explains, so only
the string case is decided.

18 cases, 9 failing before this commit. `src/domain/javascript`: 5106 to 5124 passing.
`tsc` 0; `build` 0; `npm run format:check` 0; `npm run lint` 0 errors.

* fix(javascript): reject a non-string cwd or argv0 Node refuses at the call site

The same call-site validation the `stdio` string gets applies to `cwd` and `argv0`:
Node checks them before the child exists, so a non-string literal throws
`ERR_INVALID_ARG_TYPE` and the analysis still reported a spawn.

Measured on Node 24.22.1, identical on `spawn`, `fork`, `execFile` and `exec`:

| `cwd` / `argv0` | outcome |
| --- | --- |
| `"/tmp"` / `"x"` | child starts |
| `null` | child starts |
| `5`, `0`, `true`, `{}`, `[]` | THREW `ERR_INVALID_ARG_TYPE` |

`env` is deliberately not in the set: Node accepts any `env` value, including `5` and
`"x"`, so a non-object `env` is not a call-site failure.

Only an exactly readable literal is decided; a binding, a call or a member read stays
undecided, because its value is not settled by this expression.

The test file is renamed from `spawn_stdio_literal` to `spawn_option_literals` because it
now covers all three members. 16 cases added; reverting this check fails 9 of them.
`src/domain/javascript`: 5124 to 5140 passing. `tsc` 0; `build` 0;
`npm run format:check` 0; `npm run lint` 0 errors.

* fix(javascript): read the stdio and path options by value, not by AST kind

Seven review findings, each measured on Node 24.22.1. All seven were real, and two of them
were removing real children.

**`execFile` does not forward `stdio`.** Async `execFile` never passes `options.stdio` to its
underlying `spawn`, so `execFile(process.execPath, ["-e", ""], {stdio: "bad"})` returns a
real child — `stdioMode` already forces it to `"default"`. The new filter rejected it anyway
and dropped the spawn plus its linked interactions. `execFile` is now exempt, like `exec`.

**A no-substitution template literal is the same string to Node.** `readExactJavaScriptLiteral`
already resolves it, but the filter tested `t.isStringLiteral`, so ``{stdio: `ipc`}`` kept a
spawn Node refuses while ``{cwd: `/tmp`}`` — a valid path — was rejected because the same
branch called `readExactJavaScriptLiteral` and treated any hit as a non-string. Both sides
now read the value and test its type:

| call | Node 24.22.1 |
| --- | --- |
| ``spawn(..., {stdio: `ipc`})`` | THREW `ERR_INVALID_ARG_VALUE` |
| ``spawn(..., {stdio: `pipe`})`` | child starts |
| ``spawn(..., {cwd: `/tmp`})`` | child starts |
| ``spawn(..., {argv0: `x`})`` | child starts |

**A shape the literal owner does not price still has a settled type.** `{cwd: 1n}`,
`{cwd: /x/}`, `{cwd: () => {}}`, `{cwd: class {}}` and `{cwd: new String("x")}` all throw
`ERR_INVALID_ARG_TYPE`, and none matched the previous object/array/boolean arms.

**The `fork` explanation was wrong.** Node's `stdioStringToArray` rejects `stdio: ""` with an
explicit check before anything is appended; it does not split the string and it never builds
an IPC-only array. The rejection was right, the reason was not, so the comment now states the
real rule.

44 cases; reverting the `execFile` exemption fails 1, reverting the value-based read fails 1,
and reverting the extended non-string arms fails 6.
`src/domain/javascript`: 5158 to 5168 passing. `tsc` 0; `build` 0;
`npm run format:check` 0; `npm run lint` 0 errors.

* fix(javascript): keep a path-shaped cwd that Node accepts

`cwd` is a path, so Node takes a byte array or a file `URL` there, not only a string. The
blanket `NewExpression` arm rejected every constructor for both members, which dropped real
children.

Measured on Node 24.22.1, identical on `spawn`, `fork`, `execFile` and `exec`:

| `cwd` | outcome |
| --- | --- |
| `new Uint8Array([47, 116, 109, 112])` | child starts |
| `new Buffer("/tmp")` | child starts |
| `new URL("file:///C:/tmp/")` | child starts |
| `new String("/tmp")`, `new Number(1)`, `new Foo("/tmp")` | THREW `ERR_INVALID_ARG_TYPE` |

`argv0` is different: it is an argument string only, so `new Uint8Array([120])` and
`new URL(...)` both throw there. `isPathConstructor` names the constructors that can produce
a path, and only `cwd` consults it.

`URL` stays undecided rather than accepted, because whether a file URL resolves is a
platform property: `new URL("file:///tmp/")` throws `ERR_INVALID_FILE_URL_PATH` on Windows
while a drive-qualified path does not. Syntax alone cannot settle that, so the answer is
left open instead of guessed.

8 cases added; reverting the arm fails 3 of them.
`src/domain/javascript`: 5168 to 5176 passing. `tsc` 0; `build` 0;
`npm run format:check` 0; `npm run lint` 0 errors.

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(javascript): keep unresolved cwd constructors unknown

* refactor(javascript): name process option admission by its contract

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 21:46:57 +08:00
Nianze Wuandmorluto 759817ccbe fix(javascript): distinguish Promise and FileHandle releases (#9584)
* fix(javascript): distinguish Promise and FileHandle releases

* fix(javascript): preserve reassigned handle candidates

* test(javascript): select unisolated route fixtures explicitly

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(javascript): retain uncertain releases after opaque receiver writes

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 21:46:51 +08:00
RRFZandmorluto da3bb5aa7b fix(managed): report InternalCall methods as runtime-provided boundaries (#9547)
* fix(managed): report InternalCall methods as runtime-provided boundaries

inspect_managed_native_boundaries listed P/Invoke, native, runtime and
unmanaged methods, but not methods with MethodImplOptions.InternalCall
(0x1000). The runtime implements those bodies; mscorlib and
System.Private.CoreLib declare hundreds of them as FCalls. They now appear
in native_implementations with boundary_kind runtime-provided.

* fix(managed): keep InternalCall bodies out of CIL decoding

The native-boundary view reports InternalCall methods as runtime-provided.
Body admission still decoded CIL from a nonzero RVA when the row kept IL
and managed flags. CoreCLR rejects such a row (BFA_INTERNAL_METHOD_WITH_RVA).

Body admission now treats InternalCall like the Runtime code type. The
method gets a partial body with no instructions. The InternalCall test
covers both a zero and a nonzero RVA.

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(managed): honor InternalCall during body admission

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 21:46:45 +08:00
RRFZ ec7d09176b fix(mcp): retain partial Playwright trace Evidence on failed imports (#9543) 2026-11-09 21:46:39 +08:00
RRFZ bf1811bdbc test(electron): expect the native path of an authorized ASAR file (#9549) 2026-11-09 21:46:34 +08:00
RRFZ 74201434e7 test(ida): expect Windows to fold a missing parent in the workspace root (#9548)
The IDA doctor test selects workspaceRoot `<root>/missing/..` and expects
the workspace to be unavailable. Windows folds `..` lexically before the
filesystem sees the path, as workspace allocation does, so the parent is
usable there and the test failed on every Windows run. Keep the POSIX
expectation and accept availability on Windows.
2026-11-09 21:46:30 +08:00
RRFZ 769d92025d fix(ida): report the regex syntax error for invalid search patterns (#9544)
IDA checks a regex search pattern before it starts the provider. That early
check returned a generic "Supply a valid regular expression." issue, so IDA
callers never saw the parser explanation or the literal-mode recovery that
the shared regex worker gives Kuna and Hopper callers.

Build both refusals with one helper in RegexMatcher.
2026-11-09 21:46:25 +08:00
RRFZ ca1674473c fix(rust): read rustc commit paths written with a Windows separator (#9542) 2026-11-09 21:46:20 +08:00
22f28644fe feat(artifacts): inventory and extract Deno compile executables (#8527)
* feat(artifacts): inventory and extract Deno compile executables

Adds Deno standalone artifact support for Mach-O, ELF, and PE carriers,
exposing virtual files, entry points, recorded options, and file digests
without running the executable. Extraction uses the existing safe output
tree, preserves symlink targets as metadata, and assigns indexed paths
to unsafe or oversized names. Environment-file values and certificate
bytes remain in local Evidence; npm snapshots are reported by length
without decoding. The documentation describes supported layouts, parsing
limits, and refusal behavior.

* fix(artifacts): resolve nested Intel Deno carriers to the enclosing app

The backward trailer search stopped at the first confirmed sentinel, so an
Intel app storing another Intel Deno executable as a virtual file reported
the inner app. Search to the slice start, drop sentinels inside a confirmed
payload, and fail closed on side-by-side trailers that differ.

Correct the docs on retained environment values and generated regions.

* fix(artifacts): keep Deno coverage, remote paths and PE resources truthful

Report unknown coverage when the virtual filesystem cannot be listed, keep
resolved specifiers when another remote ID is missing, and percent-encode
characters in remote paths that a Windows destination would refuse. Reject a
directory at the PE resource language level, and correct the guide's
description of colliding generated regions.

* fix(artifacts): normalize Deno remote escapes so hex case cannot collide

Uppercase every percent escape in a generated remote path, so specifiers that
differ only in escape case reach the path registry as one name and the second
gets an indexed path instead of colliding on a case-insensitive disk.

* fix(artifacts): fail closed on truncated enclosing Deno trailers and encode reserved remote path characters

The Intel trailer scan rejected an overlong sentinel before reading its
marker, so a truncated enclosing app vanished and the nested app was
reported. Read the marker first and fail closed when a confirmed overlong
trailer encloses another trailer.

encodeURIComponent leaves `*` alone and the sanitizer covered only part of
the Windows reserved set. remoteDestination now encodes the whole set once
as uppercase percent hex; source_path keeps the original.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KuWHGvzJkrhCqjtEHUwqyH

* fix(artifacts): defer truncated Intel Deno trailer failure until enclosing trailers resolve

A damaged Intel asset stored before a valid one inside a valid app made the
backward trailer search fail before it reached the outer trailer. Hold that
failure until the search ends and drop it when a confirmed enclosing trailer
holds the sentinel as data.

* fix(pe): fail closed when a resource type branch is unreadable beside a found name

namedResourceTargets skipped a type branch whose names directory, or a
name entry whose string, could not be read. A duplicate RT_RCDATA entry
pointing out of bounds therefore vanished once another branch matched,
and locatePeResourcePayloads returned only the readable ranges while
parsePeResources rejected the same tree.

The walk now records unresolved branches. hasPeResourceName still
reports false for a tree that never reaches the name, but once
locatePeResourcePayloads finds the name it raises a format failure if
any branch was left unresolved.

* fix(artifacts): skip Intel Deno sentinels that a native segment maps

A non-Deno x86_64 Mach-O that stores an Intel Deno executable in __DATA
had the asset's trailer promoted to the root application, because the
backward sentinel search never checked who owned the bytes it scanned.

locateMachOSlice now records the file ranges of every segment other than
__SUI and __LINKEDIT (libsui extends __LINKEDIT over the trailer), and
the Intel search skips any sentinel inside one of them. A slice whose
load commands do not frame stays with native inventory, since ownership
cannot be shown there.

* fix(artifacts): reject a universal Deno image with an unreadable sibling slice

The universal Mach-O locator skipped any slice whose load commands could
not be read or framed, and any slice with a FAT range issue. When another
slice confirmed Deno, that slice was reported as the whole image, so a
sibling declaring a different Deno application went unexamined.

locateMachOSlice now reports why part of a slice was not examined
(unreadable or unframed load commands, a truncated slice, or the Intel
sentinel allowance). locateMachO collects those reasons for every
declared and hidden slice and fails closed once any slice confirms a
Deno section. With none confirmed, the image stays with native inventory
as before.

* fix(scripts): supervise the Deno verifier version check

Run `deno --version` through the verifier command owner with a deadline
instead of raw execFile, and close the owner when the version is refused.

* fix(artifacts): keep unconfirmed Deno carriers native and fail truncated Intel trailers

An ELF SUI note whose descriptor is too short for the full d3n0l4nd name
now stays unconfirmed instead of failing closed; failures after the name
matches still fail closed.

Every marker-confirmed Intel trailer whose payload runs past the slice is
now kept until the backward search ends and fails unless an enclosing
payload contains it, so an Intel Deno app missing its last byte reports a
Deno failure instead of native inventory.

The Deno MCP boundary test now calls open_binary and inspect_artifact
through artifactMcpCall, as docs/testing.md requires.

* test(deno): drop source-text checks from verifier pin test

Keep the wrong-version subprocess check, which covers the pinned
release through behavior, and stop asserting on the verifier script's
source spelling.

* test(deno): close readers in finally in symlink and cancellation tests

A failed assertion or read error in either test skipped reader.close(),
leaving the file open while temp-directory cleanup ran. Both tests now
close the reader in finally, as CONTRIBUTING.md requires. The
closed-reader assertion in the symlink test stays after the finally.

* docs(specs): add deno-standalone spec, plan and tasks

Add docs/specs/artifacts/deno-standalone/ with spec.md, plan.md and
tasks.md for the Deno compile executable reader, as AGENTS.md requires
for a feature, and list them in the specifications index.

* docs: list the Deno compile guide in the documentation sidebar

* fix(deno): confirm the d3n0l4nd section before claiming Mach-O damage

A __SUI segment alone no longer makes later load-command damage Deno's.
A Mach-O whose __SUI segment holds only a differently named section now
stays absent and keeps native inventory, while damage seen before or
beside a confirmed d3n0l4nd section still fails closed.

* fix(deno): keep a slice with an unparsed segment command unread

A Mach-O segment command that frames but cannot be parsed may be the
__SUI segment, so the slice is no longer treated as fully examined. A
universal image whose other slice confirms Deno now fails closed instead
of letting that sibling supply the whole inventory, and with no confirmed
sibling the image still keeps native inventory.

* fix(deno): confirm an ELF SUI note before rejecting its length

A SUI note whose descsz runs past its note table was recorded as plain
damage before its descriptor name was read, so a malformed Deno carrier
fell back to native inventory. The owner and the d3n0l4nd name prefix are
now read within the table apart from the claimed extent, and a confirmed
note with an oversized extent fails closed.

* fix(deno): confirm a truncated __SUI section record by its name

A section record cut short after its 16-byte name was skipped, so a
damaged Deno carrier fell back to native inventory. The name is now read
apart from the rest of the record, and a truncated record that names
d3n0l4nd fails closed. The range checks for a confirmed record move into
their own helper.

* fix(deno): reject nonzero bytes after the last ELF note

When fewer than 12 bytes remained in a note table after its last note,
the walk stopped without looking at them, so a Deno note was selected
with complete coverage beside unaccounted data. The remainder is now
read: zero padding is accepted, and nonzero bytes are recorded as damage,
which fails closed once a SUI note is confirmed and otherwise leaves the
ELF to native inventory, matching the native note parser.

* fix(deno): validate FAT slice identity before choosing the search

The locator took a universal slice's CPU type from its FAT row, so a row
that called an x86_64 slice arm64 skipped that slice's Intel trailer and
let the arm64 payload stand for the whole image. Each slice's header is
now checked against its row with validateFatSliceIdentity. A mismatch
leaves the slice unread, so a universal image whose other slice confirms
Deno fails closed. The Intel fixture header now states
CPU_SUBTYPE_X86_64_ALL so its FAT rows can match it exactly.

* fix(deno): check the Intel trailer marker before judging its length

A trailer length word of 2^53 or more was dropped before its d3n0l4nd
marker was read, so an enclosing app with that length was skipped and a
nested app's range stood for the whole image. The marker is now read
first, and a marker-confirmed trailer whose length runs past the slice,
safe or not, joins the deferred truncated candidates and fails closed
unless an enclosing trailer holds it as data.

* fix(deno): count short ELF note tables against the note allowance

A note table shorter than one note header made a read of its trailing
bytes but counted no note, so many distinct tiny tables could force one
sequential read each past the 256-note allowance. Every nonempty table
now counts at least one entry, both in the up-front estimate and while
scanning, so an image past the allowance records the limit instead of
reading every table. The trailing-byte check moves into its own helper.

* fix(deno): fail closed on ELF extended section numbering beside a SUI note

A zero e_shnum with a nonzero e_shoff moves the section count into
section zero, so the section table was skipped and its SHT_NOTE tables
were never compared. Record it as damage, as PN_XNUM already is, so a
confirmed SUI note fails closed instead of reporting full coverage.

* fix(deno): keep undersized Intel trailers until enclosing ownership is known

A trailer length word below 8 was dropped before its d3n0l4nd marker was
read, so an enclosing app with a length of 0 to 7 was skipped and a nested
app's trailer was selected as the root. The marker is now read first, and a
marker-confirmed trailer too short to hold it is kept as malformed and fails
closed unless an enclosing trailer holds it as data.

* fix(pe): treat non-directory RT_RCDATA branches as unresolved

A duplicate typeId root entry whose target is a data entry was skipped
without marking the walk unresolved, so locatePeResourcePayloads returned
the readable named resource as complete. Route non-directory type
branches through the same unresolved path as unreadable names
directories so the Deno reader fails closed.

* fix(deno): fail closed on a zero-fill d3n0l4nd Mach-O section

denoSectionRange admitted the declared offset and size of a confirmed
__SUI,d3n0l4nd record without checking its section type, so a
S_ZEROFILL, S_GB_ZEROFILL or S_THREAD_LOCAL_ZEROFILL section could be
read as a file payload while MachoImage reports it as not file backed.
Reject zero-fill types with the shared isMachoZeroFillSection predicate
before assigning a physical range.

* fix(deno): bound the d3n0l4nd Mach-O section by its segment file extent

denoSectionRange checked a confirmed __SUI,d3n0l4nd record only against
the slice and file bounds, so a section reaching past its segment's
fileoff/filesize was still read as a complete payload while MachoImage
reports the same section as not file backed. Reject a section outside
its enclosing segment's file extent before admitting the physical range.

* style(deno): apply oxfmt to the DenoStandaloneLocation embedding union

* fix(artifacts): preserve Deno carrier ownership

* fix(artifacts): admit Deno parent segments

* fix(deno): preserve native segment ownership

* test(deno): cover mismatched Mach-O segment widths

---------

Co-authored-by: Matt Van Horn <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 21:43:01 +08:00
43f06eb1e8 feat(native): recover ELF function ranges from unwind tables (#9585)
* feat(native): recover ELF function ranges from unwind tables

inspect_elf_unwind_ranges and inspect-elf-unwind-ranges read .eh_frame
and .eh_frame_hdr from one local ELF32 or ELF64 image and list the
function ranges its FDEs name, with CIE facts, LSDA and personality,
the containing section and an exact symbol where one starts at the
address. Nothing is executed and no disassembler is needed. Pointer
encodings follow the DW_EH_PE rules with raw zero as null, discarded
FDEs are counted and skipped, the .eh_frame_hdr search table is
compared with the decoded FDEs, and record, row and symbol budgets are
bounded. verify:elf:unwind-ranges compares the reported starts with
llvm-dwarfdump or readelf on a compiled C++ program.

* fix(elf): keep unresolved unwind facts out of reported ranges

Skip FDEs whose initial location is indirect and do not locate
.eh_frame through an indirect eh_frame_ptr, since pointer slots are
not functions. Stop an unbounded header-located scan at a CIE with an
unrecognized augmentation, report ignored duplicate PT_GNU_EH_FRAME
headers as incomplete coverage, and accept a null personality stored
under an indirect encoding.

Keep only symbols at decoded FDE starts, cache name observations per
string-table offset, and decode names only for the returned page under
a fixed 16 MiB text allowance, so repeated or long names no longer
grow memory with the symbol count.

* fix(scripts): run the ELF unwind verifier under owned process trees

Launch probes, compiler drivers, the stripper, the frame dump and the
inspection through createVerifierCommands, as verify-binary-hardening
does, so a timeout or interrupt stops the whole tree. The temporary
directory is removed only after the owned commands have closed.

* docs(elf): move ELF unwind ranges guide under docs/guides/native

The documentation guide assigns analyst workflows to docs/guides/<domain>/.
Move the guide there, place its navigation and directory entries in the
Native binaries group, and update links from the docs index, CLI guide,
spec and authored skill. The public /elf-unwind-ranges route is unchanged.

* fix(elf): give unwind text limits their own recovery and skip indirect tables

Throw a dedicated failure when a page's symbol names exceed the 16 MiB
text allowance, and map it to a resource error that tells the caller
to request a smaller limit rather than raise max_ranges.

Read a CIE augmentation with one bounded NUL search and keep it as a
byte view, decoded only for returned CIEs, so a long augmentation is
never built up character by character.

Leave an .eh_frame_hdr search table with an indirect encoding unread:
the header is incomplete, the comparison is unavailable, and the table
no longer bounds a header-located .eh_frame scan.

* fix(elf): resolve header count and table forms and name IFUNC resolvers

Leave a non-zero .eh_frame_hdr FDE count under an indirect encoding
unresolved, since it is the address of a slot that is not followed, and
mark the header incomplete without reading the table.

Treat a DW_EH_PE_omit table encoding as an omitted search table before
testing the indirect bit, so a header that declares a count without a
table stays present with complete coverage, and accept that form in
output validation.

Match STT_GNU_IFUNC symbols at exact FDE starts after function symbols,
with a limitation that the name is the resolver, not the implementation
it selects at run time.

* fix(elf): require allocated unwind pointer bases

* fix(elf): map unwind header relative base

* fix(elf): validate unwind load mappings

* test(elf): cover mapped nonallocated frame agreement

---------

Co-authored-by: Matt Van Horn <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 21:42:53 +08:00
f21fda0fc0 feat(elf): report a Linux kernel image's version banner and embedded build config (#9573)
* feat(elf): report a Linux kernel image's version banner and embedded build config

inspect_kernel_build_config and inspect-kernel-build-config read one
local vmlinux, raw arm64 Image, gzip bzImage or zImage, or configs.ko
and report its Linux version banner and the IKCONFIG .config embedded
between IKCFG_ST and IKCFG_ED, without executing it. A config is present
only when a gzip member ends exactly at IKCFG_ED, passes its CRC-32 and
ISIZE checks and parses as .config text; framing that fails is reported
malformed, never absent. bzImage gzip payloads are inflated within a
budget and other codecs are named as unsupported, using codec detection
now shared with the kernel module parser. The config search, inflate
sizes and option counts are bounded, scans honor cancellation, and
target-derived text is marked untrusted.

* fix(elf): reject a gzip member whose header checksum fails

When FHCRC is set, compare the low 16 bits of the header CRC-32 before
inflating, so a damaged header no longer yields a present config or an
inflated bzImage payload.

* fix(elf): detect the bzImage codec inside the declared payload

Codec detection now reads the same payload_offset/payload_length slice that
is inflated, so bytes past a short payload cannot name its codec.

* fix(elf): report a framed config that does not decode as malformed

A damaged or cut-off deflate stream between IKCFG_ST and IKCFG_ED was
skipped and could end as an absent config with complete coverage. It is
now malformed with config_gzip_invalid, and a later valid config still wins.

* fix(elf): scan config lines in place and accept long string values

Config text is read one line at a time instead of split into an array, and
a quoted value is validated before it is decoded once. The 4096-character
string cap is removed: Kconfig has no such limit and max_config_bytes
already bounds every line, so a long CONFIG_EXTRA_FIRMWARE list is no
longer reported as malformed.

* fix(elf): let a live cancellation reach the kernel config parser

The parser ran synchronously, so an abort raised during parsing could not
change the signal before it returned. Inflation now runs on the zlib thread
pool and each scan yields to the event loop once per MiB, so the existing
cancellation checks observe a real abort.

* refactor(elf): carry the refused kernel config budget as a typed fact

Budget failures are now KernelBuildConfigBudgetFailure with a knob field,
and the provider builds its recovery advice from that field instead of
searching the diagnostic message for an option name.

* docs(elf): move the kernel build config guide under docs/guides/native

The documentation guide assigns analyst workflows to docs/guides/<domain>/.
Move the guide there, place its navigation and directory entries in the
Native binaries group, and update links from the docs index, CLI guide,
spec and authored skill. The public /kernel-build-config route is unchanged.

* fix(elf): yield while checking one long config line

A config made of one long quoted value never reached the per-line yield, so
the control-byte check, value validation and SHA-256 ran in one synchronous
block. Each now works one MiB window per event-loop turn, and the line
patterns match only a bounded name prefix instead of the whole value.

* fix(elf): return paged config entries that do not pin the decoded text

Entry names and values were V8 slices of the whole decoded config, so a
one-entry page kept the full text alive. Entries in the returned page are
now standalone copies, so retained memory follows the page size.

* fix(elf): yield while kernel marker searches find nothing

IKCFG_ST, IKCFG_ED and banner searches used one Buffer.indexOf over the
rest of the image, so a large image with no marker never yielded. They now
search overlapping one MiB windows and yield after each window without a
match.

* fix(elf): preserve long kernel config names

* fix(elf): frame kernel configs by gzip member boundaries

---------

Co-authored-by: Matt Van Horn <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 21:41:56 +08:00
morluto 7b9b6cc78b fix(electron): preserve unknown indirect listener ownership (#9653)
* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

* fix(electron): preserve uncertainty in indirect listener ownership
2026-11-09 08:39:31 -05:00
morluto b1f473fcf8 docs: clarify publication recovery and Evidence ownership (#9654) 2026-11-09 08:39:20 -05:00
morlutoandhhang 24053f7ee2 fix(desktop): preserve escaped xprop text (#9652)
* fix(desktop): preserve xprop escaped window text

* test(mcp): cover source fitting and omitted delivery budgets

* test(javascript): distinguish package scope in route fixtures

* test(apple): separate identity and refusal fixture groups

---------

Co-authored-by: hhang <[email protected]>
2026-11-09 08:35:55 -05:00
morluto a344650a7b fix(artifacts): preserve bundle and extraction failures on abort (#9649)
* fix(dotnet): preserve read failures during cancellation

* refactor(artifacts): reuse chunk cancellation owner

* fix(artifacts): retain extraction source lookup failure
2026-11-09 08:14:02 -05:00
morluto 9e0bc71e78 fix(comparison): retain integrity conflicts for duplicate paths (#9647)
fix: retain integrity conflicts for duplicate artifact paths
2026-11-09 08:12:55 -05:00
1f9bf8af0e fix(providers): mark callee coverage partial at computed call sites (#8340)
* fix(call-path): keep computed-call targets out of complete callee coverage

Ghidra reported complete callee coverage when a computed call had one
ReferenceManager target, so trace_call_path, build_call_path and the
function comparisons turned the static candidate into a definite negative.
The bridge now keeps those targets as observed callees and marks callee
coverage partial, naming the computed call sites. IDA callee reads cannot
classify their call sites, so their callee coverage is partial too. The
Ghidra call-coverage profile revision refuses older snapshots.

A source-owned x86-64 bridge probe in the ghidra-bridge-sources job checks
the slot, register and direct-call controls against real Ghidra.

Fixes #8277

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(ci): match the observed call coverage probe marker

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 21:03:14 +08:00
morluto 419956f56e fix(apple): preserve unknown identity in IPS joins (#9643)
* fix: preserve unknown identity in Apple IPS joins

* fix: require known CPU identity for Apple IPS joins

* refactor: centralize x86 Mach-O subtype identity

* docs(apple): explain unknown join identities

* fix(apple): bound architecture aliases to map entries
2026-11-09 21:02:08 +08:00
morlutoandSonai Biswas c5ee066022 fix(process): preserve semantic comparison across capture-local timing (#9644)
fix(process): stabilize capture comparisons

Co-authored-by: Sonai Biswas <[email protected]>
2026-11-09 21:02:03 +08:00
3230f50b16 feat(sqlite)!: page selected rows with a stated order and row offset (#9331)
* feat(sqlite): page selected rows with a stated order and row offset

readRows read `NOT INDEXED LIMIT ?` with no ORDER BY and reported
order: "unspecified", so an agent could not reach rows past row_limit
deterministically: raising row_limit hits the fixed 16 MiB reply budget,
and the sampled order depends on whichever index SQLite happened to scan.

Read the window in a unique, stated order and expose the position:

- rowid tables order by rowid unless all of rowid, _rowid_ and oid are
  declared columns, in which case the order stays unspecified.
- WITHOUT ROWID tables order by the primary-key columns, reusing each
  key's DESC flag and declared collation from pragma_index_xinfo so the
  plan keeps index order instead of materialising a sorter.
- rows.order, rows.row_offset and rows.next_row_offset report the window;
  next_row_offset is null once the window reaches the end.
- row_offset (CLI --row-offset) requires table and defaults to 0. A
  non-zero offset is refused when the table declares no usable order,
  because the skipped prefix would be arbitrary.

The schema now checks next_row_offset against truncated and the offset so
a worker reply cannot claim a page it did not read.

One attribution fix came with it: adding ORDER BY text can push an
over-wide projection past SQLite's SQL-text limit, which fails as
SQLITE_TOOBIG before the result-column limit is reported. Attribute that
case to column capacity too, so a 250k-column request still reads as
capacity rather than a byte budget.

Verified: npm run check:fast, npm run format:check on the touched files,
npm run verify:module-boundaries, npm run docs:check, and 196 tests
across the sqlite boundary, application and CLI/MCP suites. A 1002-row
inspection and in-memory EXPLAIN QUERY PLAN checks confirmed the
WITHOUT ROWID path adds no sorter. Full check:pr and the complete test
suite were not run.

* fix(sqlite): resolve the rowid alias and keep paging claims reachable

Three defects in the row paging added alongside row_offset:

- readRowOrder emitted ORDER BY rowid for every rowid table, but a table
  that declares a rowid column shadows that spelling, so the sort used the
  stored column and could repeat or drop rows across pages. Order now names
  the first alias the table leaves unbound, read from raw name bytes so an
  unrelated malformed identifier is not decoded just to choose an order.

- The inventory projection could hide the primary key a WITHOUT ROWID table
  orders by, silently degrading a paginated read to unspecified order. The
  order key is now read from the table itself rather than from the columns
  the caller asked to see.

- A truncated table with no stated order reported a next_row_offset that the
  row_offset rule then refused. It is null there, and the reply limitation
  no longer claims an unspecified order for windows that have one.

rowEvaluationFailure also probed the first rows of the table instead of the
requested window, so a failure confined to a later page was attributed to
columns that evaluate fine there.

* test(sqlite): assert the concatenated pages

The paging regression checked only counts, offsets and flags, so a second
page that repeated the first page's rows would still pass. Assert the values
read across all three pages instead.

* fix(sqlite): decode row keys in the database encoding

readRowKeyFacts decoded primary-key names as UTF-8 and probed a fixed
hidden set, so two supported schemas could not be paged at all:

- A UTF-16 database stores key names as UTF-16 bytes, so "k" decoded to
  NUL-padded text and ORDER BY named an identifier that does not exist.
  Pass the detected encoding, as readColumns and the index-metadata path
  already do.
- A virtual generated column named rowid (hidden=2) was absent from the
  shadow check, so ORDER BY rowid evaluated the generated expression
  instead of the rowid and could sort non-uniquely or fail outright.
  Include generated columns in the alias probe.

Also stop decoding primary-key names for rowid tables: their ORDER BY
names an alias, never a declared key, so an unselected malformed key
name could abort an otherwise readable selection.

The page-concatenation assertion now narrows SqliteValue before reading
.value, which the null/blob arms do not carry; this was failing the
Static checks typecheck job.

* fix(sqlite): order WITHOUT ROWID rows by the index's own key sequence

`readRowOrder` filed `pragma_index_xinfo` key terms into a map keyed by
column name, so a valid composite key that repeats a column —
`PRIMARY KEY(a COLLATE BINARY, a COLLATE NOCASE DESC)` — collapsed to one
term. The generated clause then overstated uniqueness: it omitted the
BINARY term, so `A`/`a` ties survived, and the `row_offset` paging
advertised a `primary_key` order that the actual key does not establish.

Build the terms from the index rows in `seqno` order instead, keeping
repetitions, and bail to `unspecified` if any declared key column never
appears in the index. The declared collation stays explicit so a caller
column's own collation cannot re-sort the window.

Verified against a real SQLite database that the repeated key is accepted
and that the old clause orders A, a, B while the index orders A, B, a.

* fix(sqlite): accept a null next offset on unordered truncated rows

readRows already withholds next_row_offset when a table shadows rowid, _rowid_ and oid and declares no primary key, because every non-zero offset is refused there. The reply schema still required the offset whenever truncated was set, so the provider rejected its own worker result and the first sample never reached CLI or MCP. Require the offset only when the order is stated.

* fix(sqlite): probe row keys only for ordinary tables and name row_offset

readRowKeyFacts ran for a selected virtual shadow table before readRows
could reject it, so an unavailable module surfaced as SQLite's
"no such module" instead of the selection refusal. Probe the key only
when the selected kind is an ordinary table and leave the order
unspecified otherwise, since no rows are read for that kind.

The unordered-offset refusal also reported the generic selection reason,
which callers see against table. Give it its own reason so the failure is
attributed to row_offset.

* feat(sqlite)!: page selected rows with a stated order and row offset

Selected-row inspection currently reports only rows.order: "unspecified" and
offers no way to reach rows past the first sample deterministically. Main also
refuses non-zero row_offset today, so nothing relies on that omission; this
change makes the stated order and the offset the contract, reports the next
offset the caller feeds back, and keeps the existing byte, heap, statement and
deadline limits.

BREAKING CHANGE: rows.order becomes rowid | primary_key | unspecified, and
rows gains required row_offset and next_row_offset members. A client that
validates selected-table output against the previous schema (where order was
the only permitted value) rejects a now-valid reply. Migration: accept the two
new members and treat order: "rowid" | "primary_key" as the stated order the
offset applies to; a null next_row_offset means the window reached the end.

* docs(sqlite): say a null next offset also means no stated order

The selected-rows limitation described next_row_offset as null only when the window reached the end. An unordered table whose row_limit truncates the sample also publishes null, because an offset there would name a request the row-offset rule refuses, so the Evidence read as if the sample were complete. State the second cause and pin it in the existing regression.

* test(sqlite): drop the limitation wording assertion and state the unordered exception

The boundary test pinned the sentence a caller reads in `limitations`,
so a harmless rewrite of that text failed a paging test whose real
subject is `order`, `truncated` and `next_row_offset`.
docs/development/testing.md asks for facts over frozen explanatory prose,
and the three structured expectations already fail if the paging fact
changes.

The tool contract promised that retained rows always have a unique order
and can be paged. A rowid table that shadows rowid, _rowid_ and oid and
declares no primary key has no such order: it returns
order=unspecified with a null next offset and refuses a non-zero
row_offset. Say so in the description so callers do not plan a
continuation the tool always refuses.

* docs(sqlite): say unordered tables cannot be paged

The description told callers that a truncated table without a stated order
needs "the table read without offsets", but omitting row_offset starts at
zero again: with an unchanged row_limit the next call repeats the first
sample instead of continuing past it. Say that these tables cannot be
paged and point at row_limit for a larger first sample.

* style(cli): wrap the native data type path description

The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change.

* fix(sqlite): bound rowid alias comparison by name byte length

readRowKeyFacts decoded every column name before testing it against three short aliases, so a table declaring a multi-megabyte name could exhaust the worker heap even when the caller projected the column away. Reject names that cannot be an alias by their byte length, accounting for the database encoding, before building any string.

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 21:01:58 +08:00
Chiragandmorluto 0a15246d32 docs(mcp): name the accepted seed kinds on trace_application_feature (#8220)
* docs(mcp): name the accepted seed kinds on trace_application_feature

The compact input profile reduces this schema to shallow properties, which
drops the seed union and its kind enums, so agents could not see which
kinds a seed accepts without reading the source. Describe the seed field
with its kinds, match defaults and one example; shallow reduction keeps
per-property descriptions, so the guidance survives the compact profile.

Closes #8205

* test(mcp): isolate seed guidance catalog configuration

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 21:01:52 +08:00
9108d9a627 fix(javascript): keep strict module checks unproven when alias walks exhaust their budget (#9387)
* fix(javascript): keep strict module checks unproven when alias walks exhaust their budget

Fixes #8201.

* fix(javascript): examine queued module sources after a strict walk exhausts its budget

A strict check that exhausts the alias budget stopped the whole walk, so a module branch queued beside the growing alias was never examined. Stop expanding instead, and drain the queued sources.

* fix(javascript): drain exhausted strict walks by selected keys and finite expressions

After exhaustion the limiter widens every key, so draining by normalized paths could match process.foo as mainModule. Keep each queued source's selected keys, expand finite expressions such as sequences and conditionals, and stop only at bindings and calls.

* fix(javascript): follow bindings and calls breadth first in exhausted strict walks

An exhausted strict walk skipped every binding and call, so (flag ? m : r).require with const m = module lost the module branch when r was explored first. Drain the queue breadth first after exhaustion and follow bindings and calls under a second alias selection budget, so a growing alias cannot starve a sibling.

* fix(javascript): bound every state a strict walk creates after exhaustion

After the alias budget is exhausted, paths are no longer normalized, so a
cycle through containers or getters can keep growing its selected path
without crossing a binding or call. Count each new state against the
drain budget instead of only binding and call follow-ups.

* fix(javascript): retain uncertainty at alias cutoff

---------

Co-authored-by: Matt Van Horn <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 07:59:15 -05:00
morluto 3c3d405c00 fix(artifacts): preserve physical failures during cancellation (#9641)
* fix(ines): preserve physical read failures on cancellation

* fix(artifacts): preserve root identity failures on cancellation

* refactor(artifacts): reuse cancellation projection
2026-11-09 20:56:15 +08:00
morluto 43d9c9d19b fix(uimage): reject impossible component extents before capacity (#9640)
* fix(artifacts): validate uImage extents before count limit

* docs(testing): distinguish malformed and capacity controls
2026-11-09 20:56:10 +08:00
morluto 86059108b1 fix(artifacts): retry cancelled reader preparation (#9639) 2026-11-09 20:56:05 +08:00
morluto 35fbec9df5 fix(artifacts): validate Godot row extent before budget (#9638) 2026-11-09 20:56:01 +08:00
morluto b394d50943 fix(reconstruction): honor resolved unknown revisions (#9637) 2026-11-09 20:55:57 +08:00
morluto 694edef951 fix(decompile): retain published output receipts (#9633)
* fix(decompile): retain published index on failure

* fix(decompile): preserve publication failure facts

* docs(decompile): explain retained publication receipts

* fix(decompile): retain receipts across provider cleanup chains

* fix(evidence): retain diagnostics through cleanup primary chains

* refactor(evidence): retain receipts once on the returned error
2026-11-09 20:55:53 +08:00
morluto d4b9a33348 fix(native): disclose failed accessibility reads (#9632)
* fix(native): disclose unavailable accessibility actions

* fix(native): retain accessibility attribute failures

* fix(native): preserve unknown accessibility selection
2026-11-09 20:55:48 +08:00
morluto b06df804d6 fix(model): validate GGUF extents before capacity (#9631)
* fix(model): validate GGUF extents before capacity

* fix(model): validate raw extents before capacity admission
2026-11-09 20:55:44 +08:00
df2d018b30 feat(desktop): read gzip documents as the document inside (#9111)
* fix(desktop): release a capture when its start is cancelled after launch

Cancelling start_build_capture during the opened app's screenshot left
the capture running, so the next start was refused. The capture is now
kept for the owner's cleanup, and the next start stops it first.

* fix(desktop): keep a published witness error and tag an app data refusal

A second failure to close a published witness returns the original error,
so its record and cleanup lease stay bound. A reset the app itself
refuses is an AppDataRefusal; only a reply that is not that refusal stays
unsettled. A missing child of a cut view is undecided with the same
description in both evaluators, and the shared cases cover it.

* fix(desktop): align a cut view's missing text with the Python check

A missing child of a truncated view is undecided for the same sentence in both evaluators, including the pointer the view does not contain.

* fix(desktop): keep one published witness cleanup across retries

A published record and its bound cleanup error stay together. A later close failure returns that error, and the unused path that built a fresh adapter error is gone.

* fix(desktop): keep a bundle task read inside the witness result

A cancelled, refused, or failed read of the bundle's task files returns the witness's Result, including a cleanup lease when a handle would not close.

* test(desktop): pass the settle timing when reading a witness bundle

The witness takes the settle timing before the bundle path, so the bundle-read tests keep the path in its own argument.

* fix(desktop): bound a guest file answer and split a long controller argument

A data/get whose base64 answer would exceed a quarter of the V8 heap is refused before it is read. Controller JSON that exceeds Linux's single-argument limit travels as several arguments the controller joins.

* fix(desktop): keep an underreported guest file from raising its read

A data/get answer is admitted only when its base64 body fits in the heap quarter after the copies held during parse. Decoded bytes larger than the computer reported are dropped.

* fix(desktop): grade shipped seeds and keep selected settings and sqlite logs

Seed checks follow the files the bundle ships, so an omitted lock is absent. An incomplete checkpoint is not reported as no change. A selected link keeps its spelling, and a database's wal and journal stay supporting state. Closing the fake computer stops a detached capture.

* fix(desktop): keep added-record checks off ids, dates, and joined timestamps

A position-named record's fields keep their paths, so an identifier, a stamped date, and text past the limit are not suggested checks. A whole-text check does not climb into an ancestor whose text joins a timestamp.

* fix(desktop): bound home fetches and keep unsettled edits readable

Split home-folder reads by the guest answer budget, keep a capacity error
from the capture output, and drop the superseded task-argument refusal.
Grade kept text per line, leave unretained dconf unread, and reserve
by-position notes for XML children. Map a linked database's sidecars
through the selected spelling and stop a detached capture only when its
start identity still owns the pid.

* fix(desktop): describe unretained dconf as unread

The key comparison treats a missing copy as unknown. The comment still
said that copy counted as an empty key list.

* test(desktop): keep a capture capacity error through endedCapture

An output past the guest answer ceiling stays the resource constraint,
with its heap recovery and non-retryable status, instead of a generic
capture-exit message.

* fix(desktop): keep nested save times and formatting runs out of checks

A field check drops save-time leaves before it joins the field, so a nested
timestamp is not part of the suggested text. A text check beside a save time
uses the paragraph when the diff names a formatting run.

* test(desktop): read paragraph pointers from checks that have them

* test(desktop): expect a large task to export intact

The 96 KiB refusal is gone. A prompt past that size reaches the computer
and comes back unchanged, instead of failing before export.

* fix(desktop): keep real edits that save-side heuristics were dropping

Repeated unnamed fields stay content checks. A formula that still has
text is not renumbered with its ExprID. One gained style attribute is
the edit; a repeated fill-in stays upkeep. Phone numbers and plain
clock times stay text, and a decimal is the element's whole text. An
older verifier matches on the task it states, not its evaluator source.

* fix(desktop): cover repeated records and keep timestamps out of checks

An added record whose words are already in the seed still gets the list's
text when that text fits. Readiness is a start reason, so a final check
names seed locks without matching the message. Field text split by a
timestamp stays separate, and a timestamp line is not offered as a guard.

* fix(desktop): compare a cell's joined decimal within tolerance

The whole-element check keeps numeric tolerance, so 437.80 and a long
float still match 437.8, including when a run splits the digits. A
different number or other text does not. Both evaluators read an
element's text for that comparison.

* fix(desktop): grade a number from a complete XML element only

A tolerance check joins an element's text only in an XML file or an XML
archive entry, and leaves that check undecided when the view is cut, so a
dropped descendant cannot turn the kept runs into a passing number. A JSON
object with the same shape stays the value itself. Equivalent decimals and a
split run on a whole cell still match.

* fix(desktop): accept an older grader only when its index helper agrees

The witness still matches a bundle whose evaluator is this REA's, or the older
index expression alone when no path segment is a leading-zero number. A
truncated verifier, a changed grader, or that older helper beside a segment
such as 01 no longer counts as the same task.

* fix(desktop): require the added record and leave save times out of list checks

A record a selector names is checked itself, so its words have to be in that
record. When an unnamed record only repeats words the seed already has, a
list that holds a save time is checked by the text around that time. A later
save of the same records still passes, and a missing or rewritten record fails.

* fix(desktop): judge the older index helper by pointer segments only

A leading zero in a file name or a URL does not change how the older index
helper reads a pointer. The witness still rejects that helper when a pointer
segment is a number such as 01.

* fix(desktop): do not suggest list text the seed already passes

A repeated record is shown by text the seed does not hold. The seed's own
text is not a check. A pointer inside a check value is not a pointer segment
for the older index helper.

* fix(desktop): keep a linked settings file in the folder the link points at

A selected ~/.config alias no longer moves MuseScore settings to that alias.
The seed keeps settings/MuseScore or Documents, which is the path the export
opens the note from.

* fix(desktop): split home fetches by answer size and keep a size refusal

A batch of tiny files stops before each path's envelope exceeds the host
answer budget. A capture output past that ceiling stays a resource error
instead of becoming a generic exit.

* fix(desktop): seed a selected link under the path the app uses

The selected spelling wins, so ~/.config/MuseScore is not rewritten to the
folder it points at and the note stays the one launched document. A database
wal and journal follow that database. A launch argument is respelled the same
way. Teardown signals a detached PID only when its start identity still matches.

* fix(desktop): keep a witness's actions and an incomplete checkpoint

A failed witness save retries the action log it first recorded. A deletion
left unconfirmed between the seed and the end is not reported as no change.
Windows packs npm through a shell, which can start npm.cmd.

* fix(desktop): read verifier pointers from the parsed checks

A brace inside a check value is not an object boundary. A leading-zero
pointer after that value still rejects the older index helper.

* fix(desktop): prefer the most specific home spelling and retarget launch files

A selected parent directory no longer hides a settings folder walked through
a link. A launch filename under that folder is replaced with the seeded
file's absolute path.

* fix(desktop): pack the build runtime through npm's JavaScript entry

npm pack and npm ci no longer spawn npm.cmd through a shell, so a temporary
directory whose path contains spaces stays one argument.

* fix(desktop): spell launch settings and keep a database log exact

A database log matches only that file, so a neighboring folder is not moved.
A launch option and the working directory use the same home spelling as the seed.

* fix(desktop): move a state database's logs with the database

The write-ahead log and journal follow the spelling the database received,
including a walked state link. A neighboring folder stays put. A separate
--config argument uses that spelling too.

* fix(desktop): keep a selected file ahead of a database-log guess

A file selected as itself keeps that spelling even when its name ends in
-journal. Relative launch arguments are resolved in the captured working
directory before that directory moves.

* fix(desktop): keep outside-home database logs and option values

A selected database outside the home keeps its write-ahead log. Launch
options are rewritten only when they name a captured file or settings path.

* fix(desktop): follow a database mapping and own limitation text

A selected database keeps its log when a parent folder moves or the file
sits outside the home. Trash and seed-lock tests compare the owning notice
with the paths, and launch options that are not captured paths stay as written.

* fix(desktop): retarget a settings folder that holds captured files

A nested profile directory is a captured path when a seeded file is inside
it. An option value that only resolves under the home stays as written.

* fix(desktop): keep a program word and a directory's neighbor

The program after env stays the program even when a captured file sits in a
folder of that name. A selected directory is not a database, so a neighboring
file keeps the parent folder's spelling.

* fix(desktop): record whether a selected path is a directory

Capture stores the walked path's kind. A neighboring file of a selected
directory keeps the parent folder's spelling even when that directory never
changes and its children are not content roots.

* fix(desktop): mark containing paths without a pairwise scan

A recorded path marks its known ancestors as directories in one walk up
the path, instead of comparing every recorded path with every other.

* fix(desktop): keep a selected folder that is named like a database log

A folder the caller selected stays a selected root when its name ends in
-journal or -wal. A generated log beside a database file still does not.

* fix(desktop): refuse a build command that exceeds the argument vector

Splitting one argument still leaves the whole command under Linux's E2BIG
limit. A request past that budget is refused before the process starts.

* fix(desktop): remap a captured path in an env assignment

The program word stays the program. A NAME=value assignment keeps its name
and follows a captured path value when that path's spelling moves.

* fix(desktop): keep oversized text out of draft guards

A value or line longer than the draft's text bound is not copied into a
guard, so one large frame cannot fill the task by itself.

* fix(desktop): keep an empty env value empty

An empty assignment is not a path. Resolving it named the working directory
and rewrote the value when that directory's spelling moved.

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* test(desktop): identify capture independently of schema preflight

* fix(desktop): preserve document URLs and value formats

* fix(desktop): bound added checks and omit noisy text guards

* fix(desktop): preserve launch URLs and JSON draft values

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs for runtime home

* fix(desktop): distinguish phone numbers from save metadata

* fix(desktop): preserve launch URLs and user text

* fix(desktop): preserve local file URL authority

* fix(desktop): preserve local file URL authority

* fix(desktop): render captured file URLs at launch

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): classify numeric timestamps using field context

* fix(desktop): retain context for generated draft values

* fix(desktop): retain context for generated draft values

* test(desktop): cover generated IDs in text guards

* test(desktop): cover generated IDs in text guards

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): isolate launcher URL encoding

* fix(desktop): retain clean text in noisy record fields

* style(desktop): format the noisy-record draft files with the pinned oxfmt

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): move launch execution to boundary suite

* fix(desktop): redact build schema failure output

* refactor(desktop): normalize schema failure once

* fix(desktop): omit unsafe schema parser snippets

* fix(desktop): preserve draft change and selected path evidence

* test(desktop): group retained dconf note regressions

* fix(desktop): respect archive entry format in draft checks

* test(desktop): separate unowned-file draft note cases

* docs(desktop): clarify task checks before scratch export

* fix(desktop): respell selected paths in file URLs

* test(desktop): group captured document spelling cases

* fix(desktop): bound draft preview serialization

* fix(desktop): restore selected state aliases

* fix(desktop): respect entry format in draft noise

* test(desktop): match exported alias evidence at its JSON boundary

* style: fix desktop launch test formatting

* style: format native command path schema

* fix(desktop): map selected external launch paths

* style(desktop): format seed document regressions

* test(desktop): group launch URL regressions

* fix(desktop): omit redundant nested state aliases

* fix(desktop): compare added free text as a whole

* fix(desktop): check recompressed seeds as shipped

* fix(desktop): map captured launch paths once

* refactor: remove unused whole-text guard state

* refactor: stop passing unused text children

* test: align free-text checks with whole-note policy

* fix: validate recompressed desktop archive seeds

* style(desktop): format archive seed regression

* fix(desktop): redact runtime schema errors

* fix(desktop): normalize captured file URLs

* fix(desktop): report seeded files by capture path

* fix(desktop): retain startup readiness failure stage

* fix(desktop): retain JSON date checks

* fix(desktop): preserve archive formatting checks

* fix(desktop): preserve explicitly selected journal files

* fix(desktop): report build command transport limits

* style(desktop): format capture review fixes

---------

Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-11-09 20:48:08 +08:00
6c1faaed75 fix(analysis): order graph depths numerically (#8268)
Co-authored-by: juncheng.ye <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 20:28:46 +08:00
1fcf50518e fix(native): wrap the legacy rebase cursor at 64 bits like dyld (#8111)
dyld keeps the LC_DYLD_INFO rebase cursor in a uint64_t, so an
ADD_ADDR_ULEB, DO_REBASE_ADD_ADDR_ULEB or single-slot skip operand above
2^63 steps backward. REA tracked the cursor as an unbounded bigint, so the
same stream overflowed past 2^64, failed as "Rebase expansion leaves its
file-backed segment" and left every later slot unknown. The bind stream
already wraps the same way since 576b58cef.

Every advance now wraps at 64 bits. addRun still bounds each emitted slot
to its segment, so a wrapped cursor outside the segment and multi-slot
runs with a backward skip remain refused.

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 20:28:42 +08:00
morluto 437e65d349 fix(unity): validate directory extent before capacity (#9635) 2026-11-09 20:28:38 +08:00
morluto df5103e26d fix(artifacts): validate SEA table extents before limits (#9634) 2026-11-09 20:28:34 +08:00
morluto f328b49d22 fix(cli): retain desktop directory selection (#9627) 2026-11-09 20:28:29 +08:00
morluto 70e4653feb fix(setup): retain verified skill installation outcome (#9630)
* fix(setup): retain verified skill installation outcome

* fix(setup): settle skill cleanup after host close
2026-11-09 20:28:25 +08:00
morluto 1883578b67 fix(evidence): preserve unknown source closure (#9629) 2026-11-09 20:28:21 +08:00
97b5503f39 feat(apple): join IPS images to admitted Mach-O slices (#9570)
* feat(apple): join Apple IPS frames to a selected Mach-O or dSYM slice

inspect_apple_ips reported every image UUID as unknown and stated that joined
frames were unobserved. Accept an optional join_path, read its LC_UUID and
__TEXT vmaddr with the existing bounded stable-artifact reader, and match by
canonical UUID plus architecture.

A frame joins only when exactly one slice carries the image UUID and an
agreeing architecture, so arm64 and arm64e never cross, all-zero UUIDs never
join, and a conflicted or unreadable artifact stays typed instead of guessed.
Joined frames report the slice __TEXT vmaddr plus the recorded imageOffset as
a static address, which is the value atos resolves for the same report.
Symbolication is still not performed.

* refactor(apple): remove superseded join integer type import

* fix(apple): share FAT admission and preserve completed join facts

* fix(apple): bind standalone joins to admitted source identity

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 20:20:33 +08:00
Matt Van HornandMatt Van Horn 645735fc0f feat(windows): inspect Shell Link (.lnk) shortcut files (#7859)
* feat(windows): inspect Shell Link (.lnk) shortcut files

Add inspect_windows_shortcut / inspect-windows-shortcut to read one
local Shell Link without resolving or executing the target. The report
keeps header flags and FILETIMEs, IDList items, LinkInfo volume and
network names, StringData, ExtraData including Tracker droid facts
labelled derived, and derived observations such as argument length
and a minimized ShowCommand.

* fix(windows): tighten Shell Link string reads and coverage

- Decode counted StringData over its full CountCharacters payload.
- Locate LinkInfo string terminators before charging max_string_bytes
  and before decoding; an unterminated string is a coverage issue.
- Mark an IDList without a TerminalID and an out-of-range
  LinkInfoHeaderSize as partial coverage.
- Report arguments_present and argument_length as null when
  HasArguments is set but the field could not be read.
- Derive target_extension from the joined base path and suffix.
- Keep ISO dates for FILETIME and UUIDv1 timestamps before 1970.
- Retain the completed file snapshot on parse failures.
- Reject an empty CLI path before normalization.
- Select the shortcut contract from its family in focused tests.
- Fixture StringData no longer writes a NUL into the counted payload.

* style: format docs index and Shell Link boundary type

Apply oxfmt to the two files format:check reports so Static checks passes.

* fix(windows): reject network share paths for Shell Link reads

inspect_windows_shortcut is advertised as offline, but the Windows path
schema admits UNC paths, and lstat on one can contact an SMB server and
offer the caller's credentials.

Reject UNC, extended UNC, and other host-prefixed device paths in the
provider before any filesystem call. Extended drive paths such as
\\?\C:\x stay allowed. A mapped network drive cannot be told from a
local drive by its path, and the docs say so.

* fix(windows): decode Unicode Shell Link file-entry names

File-entry items with bit 0x04 of the type byte, such as 0x36, store a
UTF-16LE name, but the parser always read it as Latin-1. The read stopped
at the first zero byte, so report.exe became r and an IDList-only
shortcut reported the wrong target_extension. Read those names as
UTF-16LE.

* fix(windows): read volume labels by exact Shell Link discriminator

VolumeLabelOffset 0x14 selects VolumeLabelOffsetUnicode and is not itself a
label offset, and that Unicode field exists for no other value. The parser
treated any offset of 0x14 or more as having both, so a Unicode-only label such
as "OS" also produced a false ANSI label "O", and an ANSI label stored past 0x14
had its own bytes read as a Unicode offset.

Branch on the exact value in parseVolume, the one place that interprets it, and
report a VolumeID too short to hold the selected field. The fixture wrote both
encodings for a Unicode volume, which hid the false label, so it now writes one.

* fix(windows): limit network share rejection to Windows hosts

POSIX hosts read repeated leading slashes as one local root, so an
accepted selection such as ///tmp/shortcut.lnk is a local file there.
Apply the host-prefixed path check only on Windows, and take the
platform through the provider constructor so the Windows path forms
stay covered on every test host.

* fix(windows): compare shortcut icons only against full paths

A stored IDList filename or relative path cannot show that an icon at a
full path is elsewhere, so the comparison reported a false difference.
Report null unless the icon and a recorded target are both full paths,
and assert the missing TerminalID issue by code and offset instead of
its message.

* fix(windows): keep unread ExtraData blocks unknown

A walk that stopped before or inside ExtraData still reported the blocks
that HasExpString and HasExpIcon expect as absent, though they were never
checked for. Report a block as absent only after the TerminalBlock shows the
section was read in full. Without one the block stays unknown and coverage
stays partial.

* fix(native): ignore LNK network provider type unless ValidNetType is set

CommonNetworkRelativeLink NetworkProviderType is only meaningful when the
ValidNetType flag (0x2) is set. Return null otherwise.

* fix(native): accept any ExtraData TerminalSize below 4 as the terminal

MS-SHLLINK requires TerminalSize to be less than 4, not exactly 0. Treat
1, 2 and 3 as the terminal block instead of a malformed block.

* fix(native): derive shortcut target facts from the stored environment target

Include the EnvironmentVariableDataBlock literal target among the target
candidates for target_extension and the icon comparison. Variables are not
expanded and the target is not opened.

* fix(windows-shortcut): keep unexpanded variable paths unresolved in icon comparison

A stored environment target such as C:\Users\%USERNAME%\app.exe could be
compared as different from C:\Users\Alice\app.exe. Variable-bearing paths
are never expanded, so the icon comparison now returns null unless a fixed
target matches the icon exactly.

* test: drop redundant ok assertion before typed error guard

The testing guide asks for the typed error to be thrown before asserting a
successful Result value, and says a preceding .ok assertion adds no coverage.

* fix(windows): point out-of-range LinkInfo offsets at their field

A LinkInfo, VolumeID or CommonNetworkRelativeLink offset that points
past its structure was reported at base + offset. With an offset such
as 0xffffffff that position exceeds uint32, so the report failed its
schema and the service discarded the whole partial result.

Report such issues at the offending offset field instead, which keeps
every issue offset and coverage.failing_offset inside the file. Cover
each LinkInfo offset field in the parser tests and the
LocalBasePathOffset case through the stable filesystem service.

* fix(windows): require RFC 4122 variant before deriving Tracker UUID facts

A version nibble of 1 alone does not identify a version-1 UUID. Also require the clock_seq_hi_and_reserved top bits to be 10, so other GUID layouts keep their raw value and derive no MAC or timestamp.

* fix(windows): ignore DeviceNameOffset fields unless ValidDevice is set

MS-SHLLINK 2.3.2 says DeviceNameOffset and DeviceNameOffsetUnicode are
ignored when the ValidDevice flag is clear. parseNetwork read both
regardless, so garbage in those fields became device-name facts or
malformed_structure issues. Read the flags once and only read the device
name fields when ValidDevice is set.

* fix(windows): keep unsupported Tracker headers as raw data

Check TrackerDataBlock Length (0x58) and Version (0) before decoding.
A block with any other header is retained as raw bytes with a
malformed_structure issue, so it no longer yields MAC and timestamp
facts under complete coverage.

* fix(windows): compare shortcut icon paths by normalized location

Collapse "." and ".." segments and repeated separators in the icon
comparison as text only, never opening a path. ".." stops at the drive
or UNC share root, and \\?\ and \\.\ paths stay verbatim.

Join LocalBasePath with CommonPathSuffix as the target and use the bare
base only when the suffix is empty, as the network share branch does.

Read the selected shortcut with lstat and open only, so a rejected
symlink's destination is never resolved.

* fix(windows): keep the UNC share in the root after repeated separators

normalizePath took the host and the next split part as the UNC root, so
\\server\\share\..\app.exe kept an empty share and let .. remove the
real share. Drop empty parts before picking the root so paths on
different shares stay different.

* fix(cli): parse windows shortcut limits as decimals

The four inspect-windows-shortcut limit options now go through
cliDecimalField like the other inspection commands, so values such as
0x10 are rejected and cliDecimalArguments.test.ts passes.

* fix(native): reject linked parents of windows shortcuts

A parent directory symlink or junction pointing at a UNC share was
followed by the first lstat, since the network check reads only the
path text and the symlink rejection covers only the final component.
On Windows, lstat each ancestor from the drive root down and reject
the path at the first link, before anything follows it.

* docs(native): move Windows shortcut guide and add its feature spec

Move the Shell Link guide to docs/guides/native/windows-shortcut.md and
list it under Native binaries in the site navigation, keeping the
/windows-shortcut public route. Update README, translated README, CLI
page and authored skill links to the new physical path.

Add docs/specs/native/windows-shortcut/ with spec.md, plan.md and
tasks.md, as required by docs/development/documentation.md.

* fix(native): admit the windows shortcut parent walk

On Windows the ancestor link walk ran before the provider's resource
admission, so close() could finish while the walk kept calling lstat.
Run the walk inside the same admission as the read, and check the
caller's signal before each ancestor lstat.

* fix(native): walk the effective parents of windows shortcuts

The ancestor link walk checked every prefix of the path as written, so
C:\alias\..\real\a.lnk was refused when C:\alias is a directory link,
although Windows removes alias\.. before opening the file. Walk the
ancestors left after removing . and .. components, keeping the
caller's spelling, never climbing above an absolute root, and leaving
\\?\ paths as written since Windows does not normalize them.

* test(native): assert the linked parent as a structured fact

The parent link regression matched the phrase "junction, which" in the
refusal message, so a wording edit would fail it. Report the linked
ancestor as observed.linked_ancestor on the input issue and assert that
fact instead of the explanatory text.

* test(native): bound the shortcut close gate and release it in finally

Race the parent walk lstat acknowledgement against inspection settling
with awaitAcknowledgement, then release the gate and join the inspect
and close work in finally so a failure reports its cause instead of
waiting for the Vitest timeout.

* fix(native): walk Windows shortcut parents lazily

windowsAncestors built and kept every parent prefix before the first
lstat, so a path with thousands of components allocated memory
quadratic in its length. It now keeps only the normalized components
and yields each parent from a running prefix as the walk reaches it.

---------

Co-authored-by: Matt Van Horn <[email protected]>
2026-11-09 20:19:44 +08:00
吴佳浩Albenandmorluto 25885b05c0 feat(v8): inspect V8 code cache containers and retained byte patterns (#8993)
* feat(v8): inspect V8 code cache containers and their retained byte patterns

Bytenode-compiled `.jsc` files — the V8 code cache container that `vm.Script`,
`vm.compileFunction` and bytenode 2.x write, and that Node single-executable applications
and Electron apps ship — had no format-native reader. `inspect_binary_layout` shows the
surrounding ELF/PE and `inspect_artifact` inventories the file, but neither decodes the V8
structures, so an agent could not tell which V8 line produced the file, could not learn the
container is metadata rather than source, and had nothing to point at a version-pinned
decompiler. Issue #8543.

`inspect_v8_code_cache` joins the format-native family with `inspect_python_pyc`,
`inspect_erlang_beam` and `inspect_kernel_module`: one explicit local file, bounded parsing,
no execution, no installation, no child process.

**The container is version-dependent, so the reader reports roles with confidence instead
of assuming an ABI.** All eight header words are reported with byte offset, observed value,
the role this reader believes the word has, and whether that role is `confirmed`,
`inferred`, or `unconfirmed`. Only three facts gate admission: the header size this reader
pins, the V8 code cache magic high half `0xc0de` (the low half is
`ExternalReferenceTable::kSize` and varies per build, so it is a build fingerprint rather
than a constant), and a declared payload length that reaches exactly the end of the file.

I derived the field semantics from the producer rather than assuming them, and two findings
changed the design:

- **The word V8 calls the source hash holds the source length on the measured producers.**
  A 54-code-unit source stored `54`, and sources with the same code-unit length but
  different content stored the same value, including a source whose UTF-8 length was 154.
  The reader reports it as `source_length` and the docs say that this name is what the
  measurement supports, not a claim about what another build stores there.
- **`vm.compileFunction` sets a high bit in that word.** The reader masks it out and reports
  the producer family as `script` or `compile_function`, so the source length stays correct
  for both producers.

Embedded strings are reported as **byte-pattern observations with absolute byte ranges**,
labelled as such in the field name and in the limitations. A reported sequence is not proof
V8 retained it as a string, and a missing literal is not proof it is absent.

**Fail-closed boundaries, all covered by tests:** an unknown magic is a typed
`unsupported_target` failure because the file is not a V8 code cache; a short header or a
payload-length disagreement is a typed `invalid_request` failure. Neither yields a partial
parse and neither records Evidence.

Non-goals are explicit in the tool description and the docs: the payload is serialized V8
heap data, not source; recovering logic needs a version-pinned View8-style decompiler that
this tool neither provides nor depends on.

Fixtures are generated from the running V8 instead of committing a version-pinned binary, so
the tests exercise the real producer and stay valid across Node upgrades.

- Semantic owner and earliest changed stage: format-native V8 code cache container decoding.
- CLI and MCP/tool-catalog contract: one new `artifact-provider` tool and one new
  `inspect-v8-code-cache` CLI command, registered in `TOOL_CONTRACTS`, the `artifact`
  language family, `TOOL_EFFECTS`, the capability declarations and services, and the
  capability inventory. `docs:check` passes, so the generated catalog agrees.
- Provider, bridge, target-format, or platform compatibility: no provider runs and no
  external tool is required, so the tool is available on every supported Node host.
- Evidence, artifact, provenance, or reconstruction contract: complete Evidence with the
  payload byte range as its location; no artifact format, reader, or reconstruction change.
- Generated metadata, package, or installation impact: catalog and document facts
  regenerated by the existing checks; no new dependency.

Expected effects:

- [x] Target content is returned as bounded structured facts.

- `src/v8/V8CodeCacheParser.test.ts` (13 cases): real-producer header facts, per-field role
  and confidence, source length for both producers, an observation's absolute offset
  verified against the file bytes, the limitation text, foreign magic, payload-length
  mismatch, short header, missing payload bytes, a different build constant still admitted,
  and the observation budget.
- `tests/boundary/mcp/v8CodeCacheMcp.test.ts` (4 cases): advertisement and annotations,
  header facts over MCP, relative-path rejection, unsupported magic, malformed container.
- `tests/acceptance/v8CodeCacheCli.test.ts` (4 cases): the public CLI, relative-path
  resolution, byte-range observations, and the typed failure.

- `npx vitest run src/contracts src/v8 src/application/v8CodeCache src/domain/v8` — 242 passed
- `npx vitest run tests/boundary/mcp/v8CodeCacheMcp.test.ts tests/acceptance/v8CodeCacheCli.test.ts tests/boundary/filesystem/productCatalog.test.ts tests/acceptance/tools/localToolContracts.test.ts` — 20 passed, 4 skipped
- `npx tsc --noEmit -p .` — exit 0
- `npm run build` — exit 0
- `npm run lint` — 0 errors
- `npm run knip` — exit 0
- `npm run docs:check` — exit 0
- `npx oxfmt --check src tests docs` — exit 0

Gap: the header table is measured on node v24.22.1 / V8 13.6.233.17-node.58 only. No other
V8 build is available here, so a different build's field order is untested; that is exactly
why only three facts gate admission and the remaining roles carry a confidence level rather
than being asserted.

Additive. No existing tool, schema, or behaviour changes. The reader never executes the
selected file, never launches a process, and never accesses the network; its audited effects
are `mutatesSession: true` only, matching the other static format readers.

Human owner: @XiaoBinGan

- [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
- [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
- [x] I declare this PR ready for initial maintainer review.

- [x] This PR addresses a concrete problem or an agreed enhancement.
- [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
- [x] The PR has one focused outcome and the title follows `type(scope): outcome`.
- [x] Related issue: #8543.
- [x] Tests cover changed observable behavior and meaningful failure paths.
- [x] Owning docs, contracts, and generated metadata are updated where needed.
- [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

* chore: retrigger title and readiness checks after the title was corrected

* fix(v8): bound the payload scan and use the repository path conventions

Review feedback from @greptile-apps on four points.

**Bound the synchronous payload scan.** The scan is one synchronous pass over the
payload, so the 512 MiB ceiling allowed a single request to hold the process for
several seconds. The ceiling is now 64 MiB, matching the pinned CPython pyc reader,
and the default is 32 MiB. The scan stays synchronous, like the BEAM and ELF note
readers in this family, but the budget that bounds it is no longer large enough to
matter. The new `max_file_bytes` ceiling and default are documented.

**Use the absolute-path schema.** `path` now uses `absoluteLocalPathStringSchema`
instead of `localPathStringSchema`, matching `inspect_erlang_beam`. A relative name
previously passed validation and was resolved against the server's working
directory, so `module.jsc` could select a different file than the caller meant. The
rejection is now at the schema boundary.

**State the extraction step.** The handoff in `docs/v8-code-cache.md` now says a
cache embedded in a single-executable application or an Electron app must be
extracted first, because this reader requires the magic at byte zero and returns
`unsupported_target` for the surrounding ELF/PE executable.

**Load only the owning contract group in tests.** Both tests now import
`V8_CODE_CACHE_TOOL_CONTRACTS` directly instead of resolving the tool through the
full catalog, so a focused test no longer constructs every other tool's schema.

21 cases pass across the parser, MCP boundary and CLI acceptance files; `tsc` 0;
`build` 0; `knip` 0; `docs:check` 0; `oxfmt --check` 0.

* fix(v8): select the tool contract by name in the focused tests

Reverts the catalog-position selection I added for a review suggestion. This
repository has a lint rule (`rea(no-positional-tool-contract)`) that forbids
selecting a tool contract by catalog order, and `toolContract("inspect_v8_code_cache")`
is how every neighbouring focused test selects one. The positional form I tried
introduced two lint errors, so the named form stays.

The review concern was that resolving through the full catalog constructs every
other tool's schema. That cost is real but it is the repository's chosen trade:
catalog ordering must never select an operation or schema, and `toolContract` is
the named accessor that enforces it.

* test(v8): cover the new tool in the absolute-path boundary test

Main added `tests/boundary/mcp/absoluteLocalPathTools.test.ts` after this branch
was opened. It enumerates every tool whose MCP `tools/list` schema must reject a
relative path, and `inspect_v8_code_cache` uses `absoluteLocalPathStringSchema`,
so the tool belongs in that list. Adding it keeps the enumerated field set complete
instead of leaving one absolute-path tool untested.

`tests/boundary/mcp` + v8 + contracts: 256 passing. `tsc` 0; `build` 0; `knip` 0;
`docs:check` 0; `npm run lint` 0 errors.

* perf(v8): read the payload by index instead of allocating per byte

`for (const [index, byte] of payload.entries())` allocates a two-element array for
every byte of the cache payload. On a 1 MiB payload that allocation dominated the
scan.

Measured with a real cache built from this parser, alternated so the two forms run
against the same input:

| payload | entries() | indexed reads |
| --- | --- | --- |
| 1 MiB | 15.5ms | 4.4ms |

That is 3.49x on the scan, and the gap widens on smaller payloads where the
allocation cost is a larger share (7.3x at 64 KiB in a synthetic run of the same
shape). The scan is the whole cost of `inspect_v8_code_cache` once the file is read,
so this is the tool's own hot loop.

Output is unchanged: the parser returns one retained string for the 64 KiB fixture,
and `src/v8` plus the CLI and MCP boundary files stay at 21 passing. `tsc` 0;
`build` 0; `npm run lint` 0 errors.

* fix(v8): admit only decimal tokens on the numeric CLI options

Main added `tests/boundary/cli/cliDecimalArguments.test.ts` after this branch was
opened. It enumerates every numeric CLI option and requires `0x10` and `1e3` to be
rejected, because plain `Number` coercion would read them as 16 and 1000. This
command's three options forwarded the domain fields unchanged, so the test failed on
them:

    inspect-v8-code-cache options.maxFileBytes: Parser.parse accepts 1e3
    inspect-v8-code-cache options.maxStrings: Parser.parse accepts 0x10
    inspect-v8-code-cache options.maxStrings: Parser.parse accepts 1e3
    inspect-v8-code-cache options.minStringBytes: Parser.parse accepts 0x10

`cliDecimalField` publishes the input view that parses the decimal token first and
then lets the domain field own bounds, defaults and help, which is what the other
numeric commands use.

`tests/boundary/cli/cliDecimalArguments.test.ts` now passes. `tsc` 0; `build` 0;
`npm run lint` 0 errors; `knip` 0; `docs:check` 0.

* fix(v8): recognize module caches, and advise from remaining headroom

Two review findings, each measured on Node 24.22.1 before changing anything.

**A module cache was reported as a script.** `vm.SourceTextModule.createCachedData()`
sets bit 30 of the source-length word, which the source-length mask `0x1fffffff` leaves
out, and the producer check only tested the function bit at bit 29:

| producer | source word |
| --- | --- |
| `vm.Script` | `0x00000013` for a 19-unit source |
| `vm.compileFunction` | `0x20000009` for a 9-unit source |
| `vm.SourceTextModule` | `0x40000013` for a 19-unit source |

So a real module cache was labelled `script`, and `vm.Script` rejects that container. The
producer enum gains `module` and the module bit is tested first; the source length is
unchanged because both flags sit outside the mask.

**The budget advice recommended a retry that could not work.** Both limits have
ceilings — `max_strings` at 65536 and `min_string_bytes` at 256 — so an admitted file
with more than 65536 runs of 256 bytes reached the branch while every suggested retry
was a rejected input. `observationBudgetRemediation` now reads the remaining headroom
and only recommends a change that is still admissible, falling back to selecting a
smaller artifact when neither can move. The parser message states the fact and leaves
recovery to the provider, which is the only place that can see both budgets.

`MAX_INSPECT_V8_CODE_CACHE_MIN_STRING_BYTES` names the `min_string_bytes` ceiling that
the schema previously spelled inline, so the advice and the schema cannot drift.

`docs/v8-code-cache.md` describes both flag bits. `src/v8` plus the CLI and MCP boundary
files: 28 passing; 7 new cases fail on the base commit. `tsc` 0; `build` 0;
`npm run lint` 0 errors; `knip` 0; `docs:check` 0.

* refactor(v8): decide the budget recovery as data, and test the decision

The remediation tests matched fragments of the advice sentence and banned old phrases.
`docs/testing.md` asks for error codes, recorded limits and recovery behaviour instead of
frozen prose, so a harmless reword could fail those tests while wrong advice passed them.

`observationBudgetRecovery` now returns the decision — which retries remain within the
schema's admitted range, and whether a smaller artifact is the only option — and
`observationBudgetRemediation` renders one sentence from it. The tests assert the
decision, so the wording is free to change and an incorrect choice cannot pass.

This also fixes a bug in the previous shape: it took the `min_string_bytes` branch to mean
"lower it", but that branch is reachable with the value already at the schema minimum,
where lowering is not a legal retry. The decision now carries `canLowerMinStringBytes`
and a `requiresSmallerArtifact` flag for that case.

5 cases, all failing on the base commit, covering both headroom states, either ceiling
alone, and the run-length floor. `src/v8` plus the CLI and MCP boundary files: 28
passing. `tsc` 0; `build` 0; `npm run lint` 0 errors; `knip` 0; `docs:check` 0.

* ci: rerun after the wakaru release download recovered

The only failing job on this branch was `javascript_recovery / real-javascript-recovery`,
which failed on `curl: (22) The requested URL returned error: 504` while downloading
`wakaru-linux-x64.tar.gz`. The release is reachable again (HTTP 200), so this empty
commit re-triggers the lane.

* docs(v8): place the guide in the tree and record the feature

The guide sat at `docs/v8-code-cache.md`, outside the `docs/guides/` tree every other
guide uses, and neither index linked it, so it could not be reached from
`docs/README.md` or the VitePress sidebar.

Moved to `docs/guides/javascript/`, linked from both indexes, and recorded the feature
under `docs/specs/v8-code-cache-inspection/` with `spec.md`, `plan.md` and `tasks.md`.
The spec keeps only the premises that were measured on Node 24.22.1 — the three producer
words and the retained-string layout — rather than restating the implementation.

`npm run docs:check` 0; `npm run format:check` 0; `tsc` 0; `build` 0; `lint` 0 errors;
`knip` 0. V8 reader, CLI and MCP files: 28 passing.

* docs(v8): use the documented spec path and test the advice against its struct

Two review findings.

**The records were at `docs/specs/v8-code-cache-inspection/`.** `AGENTS.md:73` asks for
`docs/specs/<domain>/<feature-slug>/`, so they move to
`docs/specs/v8/code-cache-inspection/`.

**The verification section claimed more than the tests do.** It said the producer cases
were built from real `vm.Script`, `vm.compileFunction` and `vm.SourceTextModule` output,
and that the budget advice was compared with its owning formatter. In fact only
`vm.Script` and `vm.compileFunction` are real producers — the module case sets the
measured bit on a `vm.Script` cache — and only `observationBudgetRecovery` was asserted.

Corrected the claim, and closed the second gap rather than only describing it:
`observationBudgetRemediation` now has four cases holding the sentence to the struct it
renders from. The sentence must not advise a knob the struct closed, and must mention a
smaller artifact exactly when the struct requires one. Verified by making the sentence
advise `max_strings` after the ceiling was reached — the test fails. Wording stays free;
wrong advice does not.

`npm run docs:check` 0; `npm run format:check` 0; `tsc` 0; `build` 0; `lint` 0 errors;
`knip` 0. V8 reader, CLI, MCP, absolute-path and decimal-argument files: 34 passing.

* fix(v8): restore the absolute-path contract entry after the rebase

The rebase onto `main` conflicted in `tests/boundary/mcp/absoluteLocalPathTools.test.ts`:
`main` gained `inspect_api_contract`, `inspect_devicetree`, `inspect_playwright_trace` and
`inspect_python_pyc` in both the contract list and the path-field map while this branch
added `inspect_v8_code_cache`. Both sides are additive, so both are kept.

Without the resolution the branch carried conflict markers, which made `mergeable_state`
`dirty` and every check on it meaningless. Re-verified after the rebase: `tsc` 0, `build` 0,
`format:check` 0, `lint` 0 errors, `knip` 0, `docs:check` 0, and the V8 reader plus its CLI,
MCP, absolute-path and decimal-argument boundary files at 34 passing.

* test(v8): pin the producer from the runtime's own header word

An independent reviewer ran these files on Node 22.23.4, 24.22.1 and 26.12.1. Only 22.23.4
failed, on `separates the compileFunction producer bit from the source length`, expecting
`compile_function` and reading `script`.

The cause is the fixture, not the parser: `vm.compileFunction(...).cachedData` carries
`0x20000009` in the source-length word on Node 24.22.1, with bit 29 set, while Node 22.23.4
does not set that bit at all. Whether a given V8 build marks the word is a build detail, so
asserting one build's spelling made the check version-dependent.

The case now reads the word the runtime produced and asserts the parser maps it correctly,
and keeps the part that must hold everywhere: the source length is the low bits, so it
equals both `word & 0x1fffffff` and the source's length. The flag meanings themselves stay
pinned by the synthetic `withHeaderWord` cases above, which do not depend on any build.

Reverting the parser's `compile_function` arm fails this case and
`keeps the function and module flags distinct`. `src/v8` plus the CLI, MCP and absolute-path
boundary files: 24 and 34 passing. `tsc` 0; `build` 0; `npm run format:check` 0;
`npm run lint` 0 errors; `knip` 0.

* fix(v8): report an unflagged producer word as unknown, not as a script

`vm.compileFunction` marks bit 29 of the source-length word on Node 24.22.1 (`0x20000004`
for a 4-unit source) and does not mark it on Node 22.23.4 (`0x00000004` for the same call).
Whether a V8 build marks a function cache is therefore a build detail, and an unflagged
word is either a `vm.Script` cache or an unmarked function cache. The container carries
nothing that separates the two.

The reader reported `script` for that case, which is a confident claim the bytes do not
support. My previous commit made the test accept whichever value the runtime produced,
which recorded the ambiguity instead of resolving it in the contract. `unknown` is now a
producer value, the unflagged word maps to it, and the observed word stays in
`header.fields` for a caller that trusts a specific build fingerprint. A limitation line
says the same in the report itself.

`header.producer` widens to `["script", "compile_function", "module", "unknown"]`; `script`
is no longer produced, because no unflagged word can be shown to be one.

The spec's producer rule and its verification notes are updated to match. Reverting the
mapping to `script` fails 2 of the 24 `src/v8` cases. `src/v8` with the CLI, MCP and
absolute-path boundary files: 33 passing. `tsc` 0; `build` 0; `docs:check` 0;
`npm run format:check` 0; `npm run lint` 0 errors. `knip` reports one unused export
(`builtinDataEffectMethod`) on `main` as well, so it is not from this change.

* fix(artifacts): retain completed identity across native parser failures

* docs(v8): preserve unknown unflagged cache provenance

---------

Co-authored-by: morluto <[email protected]>
2026-11-09 20:17:25 +08:00
2c27c2a43b feat(electron): report V8 cachedData compilation boundaries (#9501)
* feat(electron): report V8 cachedData execution boundaries

analyze-javascript-application treated new vm.Script(source, {cachedData}) and vm.compileFunction(..., {cachedData}) as ordinary source. V8 deserializes previously compiled bytecode for those calls, so an Electron preload wrapper that delegates its whole contextBridge and IPC surface to a .cache/.jsc snapshot appeared to have been fully analyzed with zero exposed APIs.

Record each vm compilation with its inert cache path when the cachedData value traces to a readFileSync/readFile of path syntax, and project a cached-data execution unknown plus a loads edge from the owning source. Compilations without a cache stay recorded as no-cached-data so absence remains distinguishable from an unparsed snapshot. Count cachedData executions in the Electron summary and state the coverage limit in the graph limitations.

* style(cli): wrap the native data type path description

The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change.

* style(domain): reformat the pending dynamic-code accumulator field

`npm run format:check` fails on this branch, which is what `Static checks` reports: oxfmt wants the `pendingDynamicCode` union on one line. Applying it is the whole change; no behaviour is touched.

* fix(electron): preserve VM callable and cache source uncertainty

* docs(javascript): specify cached-data compilation boundaries

* fix(electron): exclude disabled caches from recovered boundaries

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 20:16:18 +08:00
ab1b9e35d0 fix(entropy)!: own selected Windows path spelling at admission (#9393)
* fix(evidence)!: keep one Evidence identity across Windows path case

On Windows, an existing caller path now takes its filesystem spelling when
that spelling differs from the resolved path by letter case alone. One file
spelled `C:\Temp\x.bin`, `c:\temp\x.bin` or `C:\TEMP\X.BIN` produces one
`parameters.path`, `normalized_result` path and `evidence_id`, matching the
existing separator and `.` folding. `caller_input_path` keeps the input.

Links, junctions, `subst` drives, 8.3 short names and paths that do not
exist yet keep the caller spelling, so no link is followed and requested
output names keep their case.

BREAKING CHANGE: Windows Evidence produced from a path spelled in a case
other than its filesystem spelling now gets a different `evidence_id`.

Fixes #9373

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(entropy): keep Windows casing lookup optional

* ci(entropy): select Windows identity verification

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 20:16:13 +08:00
morluto bf271ba3fb fix(capture): retain safe caller path selections (#9626)
* fix(cli)!: retain repeated WASM input paths

BREAKING CHANGE: caller_input_paths values may now be ordered arrays of strings as well as strings. Clients must accept both forms.

* fix(capture): retain safe caller path selections

* fix(evidence): retain auxiliary selections in direct summaries

* test(wasm): separate SDK schema and acquired-input coverage
2026-11-09 20:12:45 +08:00
morluto dda2d40295 fix(desktop): verify process generation for window attribution (#9628) 2026-11-09 19:53:29 +08:00
0b1a945d69 fix(electron): model view constructors as window-like surfaces (#9571)
* fix(electron): model view constructors as window-like surfaces

Electron's static pass recognized a window only through the literal
`BrowserWindow` constructor, so `BaseWindow`, `BrowserView` and
`WebContentsView` produced no node and their literal `webPreferences` were
invisible. IPC registrations reached through such a view's own
`webContents.ipc` were dropped too, because the receiver resolver had the
same single-constructor gate -- VS Code alone registers seven channels this
way, and a shipped sample of four applications put 29 percent of window
construction sites on an unmodeled constructor.

Read webPreferences through one constructor set shared by the construction
inspector and the IPC receiver resolver. The finding only carries the new
`window_constructor` field when the site is not `BrowserWindow`, so existing
projections and their evidence stay byte-identical. A non-BrowserWindow
owner is not admitted for `loadFile`/`loadURL` on itself: only BrowserWindow
exposes those directly.

* fix(electron): keep the view constructor label lookup nullable

Observation labels are nullable in the application graph schema, so the
byLabel lookup failed tsc with TS18047 and took the Static checks job
down with it.

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 19:53:11 +08:00
Taksh Kothariandmorluto 425581e838 fix(cli): validate paths before capture file admission (#9318)
Co-authored-by: morluto <[email protected]>
2026-11-09 19:43:21 +08:00
morluto f047624018 fix(gpu): reject impossible offload bundle counts before capacity (#9621)
fix(gpu): validate bundle count against directory extent
2026-11-09 19:43:13 +08:00
morluto 62a2814756 fix(windows): bound endpoint reads and retain failed file cleanup (#9620)
fix(windows): bound Ghidra endpoint reads
2026-11-09 19:43:07 +08:00
morluto 400a91b5f9 fix(cli)!: retain repeated WASM input paths (#9624)
BREAKING CHANGE: caller_input_paths values may now be ordered arrays of strings as well as strings. Clients must accept both forms.
2026-11-09 19:43:01 +08:00
morluto 8dc2d7bec4 fix(firmware): retain command output after report read failure (#9622)
* fix(javascript): preserve recovery report cancellation

* fix(javascript): retain recovery output on report failures

* fix(firmware): retain output on report read failure
2026-11-09 19:42:54 +08:00
908d9c0726 fix(javascript): keep export bindings out of module views (#9486)
* fix(javascript): keep export bindings out of module views

Exported symbol bindings reuse the javascript-module node kind, so the modules collection counted one row per export alongside the owning module. On a CommonJS or ESM file with N exports, inspect_analysis_view reported N+1 module units, module pages listed indistinguishable rows sharing the module path, and item selection by selector.path failed with "Multiple modules have path".

Treat a node whose observations carry semantic_role export-binding as an export of its owning module, not a module unit, and exclude it from module entries. Export bindings remain distinct graph entities for export-shape comparison and version matching.

Validation: a domain regression in analysisView.test.ts and a CLI acceptance regression over a real CommonJS analysis both fail on main and pass with this change. Also ran the analysis view, MCP boundary, CLI boundary and JavaScript acceptance suites (125 tests), npm run check:fast, and the MCP catalog schema test after regenerating .cache/mcp-tool-catalog.json.

* test(javascript): run the export-binding view case on the shared CLI

The case starts a compiled CLI process for each of its three commands,
paying startup and tool registration three times per run. The fixture
already provides a file-scoped process for calls with no stdin or explicit
timeout, so use it.

* style(javascript): format the constructed-evidence type in the module-view case

The acceptance case's inline graph type did not match what oxfmt emits for the
80-column limit, so `format:check` refused the tree. Reflow only; the assertions
are unchanged.

* style(cli): wrap the native data type path description

The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change.

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 19:42:47 +08:00
morlutoandhhang 6a24e801ea fix(elf): qualify Linux CORE notes by the declared ABI (#9613)
* fix(elf): qualify Linux CORE notes by the declared ABI

* test(mcp): cover source fitting and omitted delivery budgets

---------

Co-authored-by: hhang <[email protected]>
2026-11-09 19:42:40 +08:00
morluto 5e61ae0b0e fix(process): preserve failed Darwin helper compilation for waiters (#9610)
fix(process): preserve retiring Darwin compiler failures
2026-11-09 19:42:32 +08:00
7baed55130 fix(doctor): probe providers with the selected client's registration environment (#9108)
* fix(doctor): probe providers with the selected client's registration environment

rea doctor read provider settings only from the terminal running it. A client
registration that carries GHIDRA_INSTALL_DIR or REA_JADX_MCP_CONFIG under its
own env block therefore looked unconfigured: doctor answered not_configured
while the server that client starts inherited the settings and worked. This is
the recovery command the docs point agents at, so the false negative sent them
back to re-apply fixes that were already in place.

When the caller names exactly one client, read that registration's effective
environment and hand it to the provider checks. Terminal variables that
providers resolve outside server config (PATH, SYSTEMROOT) stay inherited, and
running doctor without --client still reports this terminal. A registration the
loader cannot read keeps its own registration check as the answer instead of
being re-reported as four missing providers.

Fixes #9102

* fix(doctor): keep setup's terminal settings and a failed close on the host

Greptile found two regressions in the selected-client environment read.

Provider probes now start from the terminal and overlay the selected
registration, instead of replacing the terminal with the registration
alone. `rea setup --client` passes its own client selection through the
same doctor call, so on a first run with GHIDRA_INSTALL_DIR exported and no
registration yet, the replacements made setup report providerConfigured:
false and offer Hopper over the selected BYO provider. The registration
still wins for every value it records.

The registration read now runs inside the caller's resource scope and, when
the close of the configuration file fails, retains the returned owner there
so host.close() retries it. Dropping that owner left an unretired handle
with no way to report the unfinished cleanup.

* fix(doctor): leave a registration's expression settings unknown

Greptile found the previous fallback filling a name the selected client
records as a Cordis expression with the terminal's value. `dsh` keeps such
a value as `!!js ...`, which is not a literal; the parser reports the name
in `unresolvedSettings`, and merging the terminal into it then made probes
certify a setting the client owns and this process cannot read.

Unresolved names are now removed after the overlay, and an unreadable,
malformed, or unresolvable registration returns no environment at all, so
the caller keeps the terminal instead of a partly-known one. Provider checks
report those settings as unconfigured, which is the honest answer for a
chain that resolves outside REA.

dsh's missing registration still inherits the terminal, which is the case
`rea setup` depends on.

* fix(doctor): fold registration settings by platform identity

`selectedProviderSettings` overlaid a selected client's registration on the
terminal with an object spread. On Windows a registration can record
`ghidra_install_dir` while the terminal exports `GHIDRA_INSTALL_DIR`; the spread
kept both spellings, and since provider checks read the canonical uppercase
names, doctor either certified the terminal's install or reported the setting as
unconfigured. An expression the registration marks unresolved was skipped under
one spelling while its terminal value survived under the other.

Merge the two layers through `mergeEnvironmentOverrides` and fold the result
with `snapshotEnvironment`, so the registration's value wins regardless of how
it spells the name and the provider sees the names it reads. Fold the unresolved
names the same way before deleting them, and export the helper so the two
platform identities can be covered directly.

* style(doctor): wrap the long import in the runtime test

oxfmt wants the three imported names on their own lines; the previous
commit left the single-line form, which fails `npm run format:check`.

* test(codebuddy): construct the registration command like its sibling clients

`npxRegistrationCommand` was removed in c77735fa ("fix(setup)!: bind managed
registrations to selected installation"). The CodeBuddy boundary test landed
after that refactor and still imported it, so `npm run typecheck` failed on
every branch that descends from merge fb841a95 — main included, where the
main push CI is still queued.

Use `currentClientRegistrationCommand()` the way the kimi, lmStudio and other
client configuration tests already do.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

* fix(doctor): retain failed registration file cleanup

* fix(doctor): preserve unknown client provider readiness

* fix(doctor): scope Kuna readiness to its environment

* test(doctor): respect Kuna host support

* fix(doctor): read a registration's unresolved settings only where the probe needs them

Every setting an expression made unresolved blocked readiness, even one the provider never consults for the configuration the client supplied. With a literal JAVA_HOME and jar path, JADX resolves Java by absolute path and does not read PATH, yet a dsh registration with an expression-backed PATH skipped the probe and reported the provider unknown. Ghidra is the same when both GHIDRA_INSTALL_DIR and JAVA_HOME are explicit.

Decide per provider which unresolved setting the probe actually reads, and skip only those. Kuna's readiness list also omitted HOME and XDG_CACHE_HOME, which its cache root falls back to, so an unresolved cache input could let the probe check a fallback directory and report available without knowing the client's real cache.

* fix(doctor): stop probing a registration expression as a literal

Two findings from the review, both about handing a probe a value REA cannot interpret.

REA_KUNA_CACHE_DIR made settingAffectsProbe ignore every unresolved Kuna setting, so an expression-backed REA_KUNA_COMMAND or REA_KUNA_JOBS silently fell back to a default and the provider could report ready. Limit the exception to HOME and XDG_CACHE_HOME, the two cache-location defaults.

HOME and XDG_CACHE_HOME were also added to KUNA_DOCTOR_SETTINGS, which registrationEnvironment copies wholesale, so setup persisted ambient values a child already inherits. Split the readiness inputs from the settings setup writes.

Qwen expands $VAR and ${VAR} in a recorded env value from the environment it runs with. REA cannot predict that, but it was handing the literal to the probes, so a registration appending to PATH made JADX search for a literal ${PATH} entry. Treat such values as unresolved for that client so the affected provider reports unknown instead of probing the literal text.

* fix(registration): recognize a client's env expression at the shared reader

Doctor decided which Qwen settings were expressions after the shared registration reader had already parsed them as literals, so doctor and setup/status held two interpretations of the same file. AGENTS.md requires normalizing once at the owning boundary, so move the recognition into ClientRegistrationEnvironment and let every consumer read the facts it returns.

The reader now takes the client name, marks a variable-expanding client's $VAR and ${VAR} values as unresolved, and reports those names through unresolvedSettings. Doctor drops its own parser; setup and status pick up the same facts, which also keeps an expression-valued timeout out of the resolved budget instead of trusting the literal.

* fix(doctor): preserve unresolved typed settings

* style(cli): wrap the native data type path description

The `inspect-native-data-type` argument description line is 82 columns, over the repository's 80-column `.oxfmtrc.json` limit, so `npm run format:check` fails on main and on every branch merged from it. It came in with #9465 and the following commits to this file did not touch it. Reformat the one call the formatter objects to; no behaviour change.

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: Claude Sonnet 5 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 19:42:25 +08:00
512bcf5cdb fix(javascript): analyze first-party sources before dependencies (#9341)
* fix(javascript): analyze first-party sources before dependencies

Every source in one application analysis charges the same parent result
budget, and sources were offered in lexical inventory order. On an Electron
app.asar, node_modules/ sorts before an electron-vite out-*/ directory, so
dependency files spent the budget and every first-party main, preload, and
renderer file had its semantic analysis not started and its application
projection rejected. The summary then reported zero windows, preloads, and
bridges while electron_semantics still said "recovered".

Offer application JavaScript before JavaScript under node_modules; reported
files keep inventory order. Count Electron boundary facts only from files
whose projection was admitted, and report electron_semantics "partial" (with
a limitation naming the count) when an Electron-loading source was not fully
examined or reported.

Fixes #9321

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(javascript): treat limited Electron examination as partial coverage

A static pass can stop early at the Electron wrapper expansion budget or
another resource limit without failing the worker. Such a source carries no
analysis or projection failure, so its retained IPC or bridge facts still
reported electron_semantics "recovered". Read the owned examination status
and count a limited Electron-loading source as incomplete.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 19:42:19 +08:00
morluto 31b6db72c6 fix(environment): avoid attributing overlapping writes (#9614) 2026-11-09 19:42:13 +08:00
morluto 3e1f70096b fix(cli): retain selected IL2CPP binary path (#9623) 2026-11-09 19:42:06 +08:00
morluto e8d1bbc98b fix(reference): preserve import cancellation provenance (#9597)
* fix(reference): preserve import cancellation reason

* test(reference): cover null cancellation during failed cleanup

* test(reference): always release fixture directory
2026-11-09 19:42:00 +08:00
morlutoandhhang e66e92904e fix(desktop): preserve trailing spaces in desktop entry values (#9612)
* fix(desktop): preserve trailing spaces in desktop entry values

* test(mcp): cover source fitting and omitted delivery budgets

---------

Co-authored-by: hhang <[email protected]>
2026-11-09 19:41:54 +08:00
morluto 8c64eb511f fix(ghidra): stage NativeAOT report until recovery completes (#9611)
* fix(ghidra): stage nativeaot recovery report until complete

* test(ghidra): exercise nativeaot rollback reporting
2026-11-09 19:39:51 +08:00
morluto e304de99bc fix(desktop): retain published capture evidence on late cancellation (#9591)
fix(desktop): retain published capture on late cancellation
2026-11-09 19:36:11 +08:00
morluto d307ece767 fix(flutter): validate ELF build ID note padding (#9609) 2026-11-09 19:35:13 +08:00
morluto 4fde832260 fix(rizin): retain failed TTY cleanup ownership (#9608)
fix(rizin): retain TTY admission cleanup ownership
2026-11-09 19:35:07 +08:00
morluto 97fe58d3fe fix(erlang): validate table framing before row capacity (#9606)
fix(erlang): validate BEAM table extents before capacity
2026-11-09 19:35:01 +08:00
morluto 7c821cfea1 fix(hopper): retain bridge after lost shutdown reply (#9604) 2026-11-09 19:34:55 +08:00
morluto f46e9a8479 fix(environment): retain committed bundle receipts after finalization failure (#9602)
* fix(environment): retain published bundle receipts

* fix(environment): retain receipt after tarball lookup failures
2026-11-09 19:34:48 +08:00
morluto 29768c6f53 fix(dotnet): reject negative bundle declarations as malformed (#9601)
* fix(artifacts): reject negative .NET bundle declarations

* test(artifacts): derive .NET fixture field positions
2026-11-09 19:34:35 +08:00
morluto cc13c5d095 fix(native): preserve unknown ARM64 subtype in hardening (#9600) 2026-11-09 19:34:28 +08:00
morluto 897caa814d fix(inspector): drain discovery requests after sibling failure (#9598)
fix(inspector): drain failed discovery requests
2026-11-09 19:34:22 +08:00
morluto a874cef9af fix(adb): report observed output after failed file pull (#9596) 2026-11-09 19:34:16 +08:00
morluto a6c10a28f4 fix(cli): retain JavaScript evidence through final cleanup (#9595)
fix(cli): retain JavaScript evidence after final cleanup
2026-11-09 19:34:10 +08:00
morluto 9536ebb7ca fix(firmware): exclude UF2 payloads overlapping MD5 metadata (#9593)
* fix(artifacts): exclude UF2 payloads overlapping MD5 metadata

* fix(artifacts): clarify DfuSe overlap ordering
2026-11-09 19:34:04 +08:00
morluto 8358cfcab3 fix(javascript): preserve cancellation during recovery report reads (#9605)
* fix(javascript): preserve recovery report cancellation

* fix(javascript): retain recovery output on report failures
2026-11-09 19:27:39 +08:00
Ziqian zhangandmorluto 94728785ea fix(setup): refuse multi-document DSH patches (#7512)
Co-authored-by: morluto <[email protected]>
2026-11-09 18:30:31 +08:00
morluto 38be5fd9f2 fix(reconstruction): preserve unknown process case coverage (#9599)
fix(reconstruction): avoid inferring unobserved process cases
2026-11-09 18:28:13 +08:00
Henglie f20f05536e feat(providers): canonicalize selection diagnostics (#9171)
* feat(providers): canonicalize selection diagnostics

Extend candidate statuses with selection source and stable rejections derived from readiness and target support. Automatic selection now keeps every rejected candidate attributable, while explicit and ambiguous outcomes remain fail-closed and deterministic.

* test(providers): surface typed selection errors

Follow the repository test guide: throw the typed Result error before
reading the selected value so a failure reports the provider error.
2026-11-09 17:41:13 +08:00
gongfpp 4df630e16a fix(javascript): keep unisolated program-var route values unknown (#6778)
* fix(javascript): mark unisolated program-var route values unproven

`JavaScriptRouteScan.#resolve` now records the identifier whenever a value
proof depends on a program-scope `var` in a script without a proven isolated
runtime. That binding is a global-object property, so `globalThis.NAME` or an
alias can replace it after the source initializer.

The mark lives in the shared resolver, so every consumer degrades from one
fact instead of each one opting in:

- path segments and navigation destinations stop asserting the source value
  and report an unknown interpretation;
- an index flag stays unproven, as before;
- a route container keeps its source route set and reports an incomplete
  interpretation, so `var routes=[...]` no longer loses every route.

The source initializer is still followed, which is what lets a container
consumer keep its facts while disclosing the gap.

* fix(javascript): keep literal child routes behind an unisolated children array

Rejecting the whole children configuration marked its records as claimed
by the interpretation, so the automatic-root loop skipped them and the
literal child paths disappeared even though their absolute paths need no
parent proof. Keep the source records reachable and let the walker report
the unproven ancestry as an interpretation gap.

* test(javascript): bound the index-route CLI acceptance matrix

The flag matrix runs sixteen CLI invocations in one test and passed at 20.7 s
against a 30 s default timeout, so it timed out on a loaded runner. The same
matrix is already asserted per case in javascriptRouteScan.test.ts, where each
case is an in-process parse; this file keeps one resolved case and one
unresolved case to prove both branches of the public CLI view.

* fix(javascript): retain ownership of unproven route literals

Keep source ownership separate from runtime value proof so rejected program-var route segments cannot reappear through the generic constant pass. Cover path, route, typed and alias forms, retain navigation controls and independent literals, and verify inventory and public CLI projections.

Run the existing CLI index cases independently so repeated process startup does not consume one combined test deadline. Preserve both cases and all assertions.

* fix(javascript): keep unisolated navigation data out of route literals

An unisolated program-var slot holds data, not a route. Refusing its
runtime identity released the object back to the independent route scan,
which then reported a query payload as a nested route. Report the object
as consumed navigation data and stop following it as a path.

* fix(javascript): follow rejected navigation data without trusting it

Claiming the whole object for a rejected value also claimed its sibling
properties, so an independent route in the same object disappeared.
Keep the property selection: ownership follows the slot, and only the
route consumers refuse an unproven runtime identity.
2026-11-09 17:40:57 +08:00
Adolanium c546e6d12c fix(validation): stop reporting an exclusive bound as an inclusive minimum (#7502)
* fix(validation): stop reporting an exclusive bound as an inclusive minimum

Input issues report minimum and maximum as inclusive bounds, as JSON Schema does, but the projection ignored Zod's inclusive flag. A .positive() field such as inspect_analysis_view view.limit or observe_native_ui pid rejected 0 and reported minimum: 0, naming the rejected value as allowed; .lt() limits had the same problem for maximum. An exclusive bound now keeps the schema's own message (for example 'expected number to be >0') instead of a false inclusive bound. Inclusive bounds are unchanged.

* test(validation): cover exclusive and inclusive bounds through MCP
2026-11-09 17:40:42 +08:00
Heo Hao 875f729070 fix(cli): advertise the source-location path and URL constraints (#9281)
* fix(cli): advertise the source-location path and URL constraints

trace-web-source-location declared manifestPath, sourceMapPath and sourceMapUrl
as bare strings, so `--schema --json` advertised them with no minLength and no
absolute-path pattern. The service then refused a relative manifest or map with
a post-parse invalid_request naming manifest_path and source_map.path, while
trace-web-module-imports already advertises those constraints from its domain
fields.

Bind the three positionals to webSourceLocationInputSchema so the domain field
owns the bounds, format and help, matching the rule in cliDecimalField that the
domain field owns bounds, defaults, nullability and help. The CLI now refuses a
relative or NUL-bearing value at its own boundary with a field-level message
instead of surfacing a service-level failure.

Add a boundary test that compiles the advertised args schema and asserts the
shared path and non-empty URL constraints, next to the existing sibling case.

* test(cli): isolate the NUL rejection from the absolute-path check

The boundary case for trace-web-source-location used "manifest\0.json", which is
also a relative path. The absolute-path constraint alone rejects it, so the case
would still pass if schema conversion dropped the NUL pattern. Use an absolute
path with an embedded NUL instead, and add the same case for sourceMapPath.

* style(test): wrap input schema assertions for oxfmt

The two unknown-byte cases exceeded the line width and tripped
format:check in CI. No assertion changes.
2026-11-09 17:40:25 +08:00
Vinícius Andrade dcf481c11a fix(cli): admit native data type selectors before opening a target (#7360)
* fix(cli): admit native data type selectors before opening a target

* fix(cli): advertise the native data type selector choice

* test(cli): reject invalid native data type selectors before target access

* test(cli): cover native data type selector discovery

* fix(cli): reject an empty native data type target before resolution

inspect-native-data-type still declared its positional target as a plain
string after #7956 moved its sibling local-file commands to
localPathStringSchema, so an empty target passed validation and resolved
to the working directory. Use the shared schema and cover the command in
the empty-target admission table.

* refactor(cli): share native data type selector guidance owner

* test(cli): bind native data type selector checks to owning contracts
2026-11-09 17:40:09 +08:00
2SinceJulyand2SinceJuly ca456efcbf fix(wasm): admit compatible 1.0.x WABT releases (#9223)
* fix(wasm): admit compatible 1.0.x WABT releases

REA_WABT_BIN_DIRECTORY admission compared each --version banner to the
literal 1.0.42, so a Homebrew or rolling-distribution WABT that advanced
past that build was refused even though wasm-validate, wasm-objdump and
wasm2wat accept the same flags and emit the same output shape. Callers had
to keep a hand-extracted 1.0.42 tarball alongside the system package.

Admission now accepts the release range ^1.0.42 and records the observed
banner, matching how the Wakaru adapter already admits compatible builds.
The verified release stays the reference: a banner other than 1.0.42 is
retained as tool_profile.release and provider.version and the report states
that only 1.0.42 was verified. All three tools must report the same release,
since one directory is one installation. Managed downloads stay pinned.

Fixes #9197

* fix(wasm): bound WABT admission to releases the report schema accepts

WABT_ACCEPTED_RANGE was "^1.0.42", which also admits 1.1.0 and later 1.x
releases, while wasmArtifactSchema only accepts 1.0.x banners. A configured
tool reporting 1.1.0 therefore passed version admission, ran the inspection,
and then failed at report parsing. Use the explicit ">=1.0.42 <1.1.0"
range so admission and the schema agree, and test the boundary.

Also carry the compatible-release warning into both partial-Evidence
builders. Previously only successful reports stated that REA verified
1.0.42, so a validation rejection or later read failure under a
compatible release silently dropped the uncertainty that a success reports.
Share the text through wabtReleaseLimitation so both paths stay aligned.

* fix(wasm): stop claiming the 1.0.42 commit for a newer admitted release

A caller-supplied 1.0.43 or 1.0.44 was admitted with the observed banner in
tool_profile.release, but source_commit still carried the 1.0.42 revision,
which is a claim about a build that did not run. Emit null for any release
other than the verified 1.0.42; the observed release and each executable
digest still identify what actually ran.

Also spell the accepted range as it is defined (>=1.0.42 <1.1.0) in the guide
and the failure fixture instead of ^1.0.42, which in semver admits 1.1.0.

---------

Co-authored-by: 2SinceJuly <[email protected]>
2026-11-09 17:39:49 +08:00
Rishikesh Gowda K K 891371e738 fix(objdump): reflect observed launch failure in discovery availability (#9051)
* fix(objdump): reflect observed launch failure in discovery availability

Distinguish a statically registered standalone command from an executable
whose launch has failed in the active server session. Discovery leaves
unprobed commands optimistically available, reports an observed launch
failure as provider_unavailable with the command remediation, keeps the
tool callable for retries, and clears the failure after a subsequent
successful launch.

Fixes #8826.

* test(reverse): verify overlapping command launch dispatch ordering

* test(reverse): unblock gated test workers before awaiting service closure

* test(reverse): bound lifecycle acknowledgement and join launched workers

* fix(reverse): distinguish host capacity exhaustion from command launch failures

* fix(reverse): preserve host capacity across Linux process ownership preparation

* fix(reverse): exclude process ownership admission failures from command launch state

* fix(reverse): preserve Darwin process ownership preparation error across command launch

* fix(reverse): disambiguate cleanup declaration in provider observation

* fix(reverse): preserve command deadline timeout during Darwin ownership preparation

* fix(reverse): isolate Darwin baseline ps errors and provide platform-aware admission guidance

* fix(reverse): rely on typed baseline admission failures and use owning repair formatter in tests

* test(reverse): use owning formatters for process ownership remediation assertions

* fix(reverse): restrict Linux ownership prerequisite remediation to missing ps and preserve system error codes

* fix(reverse): retain typed process ownership prerequisite with preserved error codes

* fix(reverse): remove unvalidated error label fallbacks and keep admission tags out of system_code

* fix(reverse): keep Windows native runner failures out of command-launch state

* fix(process): preserve current_process_unreported refusal in Linux provider readiness

* fix(process): distinguish no-code ps failure from missing ps and cover rizin launch failure

* fix(windows): classify process_spawn host-control rejections as RuntimeControlUnavailableError
2026-11-09 17:39:34 +08:00
068c319e82 fix(desktop): keep a failed library owner query unknown (#9240)
* fix(desktop): keep a failed library owner query unknown

dpkg-query -S exits 1 when some requested path has no owner and 2 when
it could not answer at all. LibraryPackageInventory.packageOwners only
rejected the missing and failed outcomes, so an exit of 2 with empty
stdout parsed as a successful empty owner map and every located library
was recorded in unpackaged_libraries. That turns a failed lookup into an
incorrect ownership fact and drops the reason, while the executable
owner lookup in DebianPackageInventory already treats exit codes above 1
as unknown.

Treat an exit code above 1 as a failed query for the batch, report it in
library_packages_reason, and leave the libraries it covered out of
unpackaged_libraries. Already established missing_libraries are kept.

* fix(desktop): retain completed library ownership queries

* fix(desktop): report ownership gaps per library

---------

Co-authored-by: 2SinceJuly <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 17:39:20 +08:00
morluto 4a1c4a59ba test(discovery): separate recovery guidance and owned home state 2026-11-09 14:58:35 +08:00
morluto 95382c2ba0 test(process): bind cleanup fixtures to selected resources 2026-11-09 14:58:35 +08:00
morluto 5d5fe72218 fix(plist): reserve Evidence framing in nesting allowance 2026-11-09 14:58:35 +08:00
morluto bd0642e0a4 test(javascript): give loader fixtures explicit source context 2026-11-09 14:58:35 +08:00
morluto 2c6f981667 fix(javascript): hash Electron call paths from canonical values 2026-11-09 14:58:24 +08:00
morluto 84f3401dcf perf(tests): reuse CLI worker for Evidence bundle data cases 2026-11-09 14:58:24 +08:00
morluto 99d00d1041 perf(tests): reuse CLI workers for Rust and certificate cases 2026-11-09 13:36:08 +08:00
morluto 228ca6a9da test(reference): separate loader fixture data from assertions 2026-11-09 13:29:53 +08:00
morluto 4b27c51724 perf(tests): reuse CLI reader for browser input diagnostics 2026-11-09 13:29:53 +08:00
morlutoandShuaixinHuang b46f4fcebd fix(javascript): preserve loader authority for exact suffixes (#9477)
* fix(javascript): require exact loader suffix case

* fix(javascript): share exact loader mode across import resolution

* test(process): align symlink fixture with filesystem checkpoints

* test(javascript): normalize the module resolution oracle path separators

* test(frida): assert the CLI spawn Evidence contract

* test(mcp): use an unsupported analysis-view parent

* test(browser): separate source-location line bounds from parity

* fix(javascript): preserve uncertain loader provenance

* test(javascript): cover uncertain require resolution

* fix(javascript): retain conditional Electron evidence

* fix(javascript): preserve timed semantic provenance

* fix(javascript): retain conditional builtin effects

* test(javascript): distinguish bundled host authority

* fix(javascript): preserve nested loader invocation identity

* test(javascript): qualify Electron fixture authority

* fix(javascript): preserve loader invocation authority

* fix(javascript): respect source order for resolver escapes

* fix(javascript): check nested loader callable mutations

* fix(javascript): retain conditional constructor requests

* fix(reference): retain constructor loader relationships

* fix(reference): follow nested loader callable paths

* refactor(javascript): share callable mutation interpretation

* fix(javascript): require exact package loader suffixes

* refactor(javascript): unify conditional relation evidence

* refactor(javascript): unify conditional node evidence

* fix(javascript): retain unknown authority for mixed case sources

* fix(javascript): preserve loader uncertainty for dotfiles

* fix(reference): classify javascript suffix dotfiles

* test(reference): cover suffix dotfiles through cli

* fix(javascript): retain equivalent conditional built-in origins

* fix(javascript): preserve wrapped resolver operation

* fix(javascript): retain nested conditional built-in receivers

* test(javascript): declare CommonJS in native addon fixtures

* fix(javascript): retain wrapped resolver candidates

* style: format merged native command schema

* fix(javascript): preserve uncertain loader authority

* test(javascript): require candidate preload evidence

* test(javascript): cover reference loader member writes

---------

Co-authored-by: ShuaixinHuang <[email protected]>
2026-11-09 13:21:24 +08:00
Ysopwnableandgensokyo-dev 78a84d75c0 test(mcp): size source recovery budget from delivered evidence (#9566)
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-09 06:03:43 +01:00
morluto c78c8aed40 test(process): isolate startup failure from shared helper state 2026-11-09 08:40:39 +08:00
morluto 6b3372e259 perf(tests): reduce repeated schema compilation and CLI startup 2026-11-09 08:40:39 +08:00
morlutoandFelix 52fe342c73 fix(evidence): preserve subjects in derived workflows (#9492)
* fix(javascript): preserve verified trace evidence subjects

Co-authored-by: Felix <[email protected]>

* fix(evidence): preserve selected subject names in views

* fix(evidence): preserve subjects across related workflows

* fix(evidence): retain inventory projection subjects

* fix(managed): validate projected graph root identity

* style: format evidence and native command code

* fix(javascript): validate static graph root identity

* test(javascript): update feature trace graph expectations

---------

Co-authored-by: Felix <[email protected]>
2026-11-09 08:12:17 +08:00
4d7c6efe9a feat(desktop): open the seed's document at launch, and say when a seed was made for the task (#9054)
* feat(desktop): capture, draft, export and witness a task inside a build computer

- start_build_capture runs REA's capture inside the computer through a
  small controller over computer-exec, after a user-space apt-get update;
  mark_build_checkpoint returns once the capture confirms the checkpoint;
  stop_build_capture returns the draft between seed and end.
- export_build_environment exports with the requester's task inside the
  computer and fetches the bundle through data/get in bounded batches.
- check_build_task grades a computer's data/get with REA's evaluator and,
  through seed, after_edit, near_miss and after_reset stages, records a
  witness with the action log and the screen recording; act_on_build_computer
  gains reset.
- Capture progress lines carry window, checkpoint and refusal events; apps
  start in the home; a pack script builds the runtime archive; long tools
  send progress notifications.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): make build capture and witness steps retryable, and close the near-miss reset hole

- A capture whose start answer is lost is stopped at once and kept
  unreleased until the stop is confirmed; a lost checkpoint answer is
  recovered from the capture's own log, and draft and export use the
  checkpoints its Evidence records; stop can be repeated to resume
  reading its result.
- A witness can check its seed again after a failed read, and keeps its
  stages in a saving state until the record is written, so a failed save
  can be retried without reading the app again.
- after_edit and near_miss count only click, drag, scroll, type and key,
  so a reset no longer stands in for the near-miss edit.
- A .deb build keeps its package file for capture and passes its URL to
  export; bundle downloads write each checked file before fetching the
  next; an apt-get update failure reports its exit code and log tail;
  reset sends progress while it waits.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep the Trash out of seeds, bundle folders unwritten and first frames drawn

From the blind Inkscape rebuild:
- Paths in ~/.local/share/Trash get a desktop_trash reason and the
  shared_desktop role, so a file trashed before seed is neither owned nor
  seeded; the export names it in a limitation.
- The keep helper runs with PYTHONDONTWRITEBYTECODE=1, so starting a
  computer from a bundle folder no longer writes __pycache__ into it.
- start_build_computer waits, bounded, until the first frame is not one
  colour before returning it.
From review: a .deb that apt also offers exports from apt, since export
passes --package-url only when the capture says the package must come
from its file.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): bound the first-frame wait's retries to the time left, with progress throughout

Each retry of the drawn-first-frame wait runs under an abort that fires at
the 10 s deadline, and the latest frame is returned when time runs out.
connect takes the first frame itself while its progress heartbeat still
runs, reporting "waiting for the desktop to draw". The wait moves to
DrawnScreenshot.ts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): serialize each build computer's calls, keep one recording id, and separate unreadable checks from failures

- One queue per computer: actions, capture steps and witness checks run one
  at a time, so two overlapping after_edit checks can no longer both be filed
  and let the witness hold without a near miss. Close cancels the running and
  queued calls; different computers still run concurrently.
- The witness keeps one recording request id per computer, so a seed retried
  after a lost reply recovers the recording it started.
- A check that could not read its file is undecided, not a failure, so a
  temporary read error can't satisfy seed, near_miss or after_reset; such a
  stage is refused and can be checked again.
- The export's file listing is written to a file and read back in bounded
  batches, so a large bundle no longer overruns exec's 1 MiB output.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): treat failures on cut views as undecided, require a decided seed failure at export, and keep a close's cancellation

- A presence check that misses its line, row or pointed value in a view cut
  short is undecided rather than failed, since the data may sit in the part
  left out; a witness stage can no longer be proved by such a failure.
- Export counts only decided failures on the seed, and refuses a task whose
  every seed failure is undecided.
- A step whose turn was aborted by a close or the caller returns cancelled
  even when its work finished, so start_build_capture no longer reports
  success for a computer being removed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep a step's saved result when a cancellation arrives during its commit

Reporting cancelled for any step whose turn was aborted after its work
replaced a completed export or witness save with a bare cancellation, losing
what was saved and leaving the witness unreadable. Steps now return what they
completed; only start_build_capture, which commits nothing, checks its turn
after the post-launch screenshot and reports cancelled. Asking again for a
saved witness's final check returns its record.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): refuse a saved witness's repeated check for another folder

Asking again for the check that ended a witness returned its saved record
whatever directory came with it, so a caller naming another folder got
success though nothing was saved there. A different folder is now refused,
as an unfinished seed's retry already is.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): assert the facts of cut-view and undecided outcomes, not their sentences

The new cut-view, missing-value, undecided-refusal and witness tests froze
whole descriptions, so a harmless rewording broke them. They now check each
outcome's id, passed and undecided exactly and the facts its message must
carry (file, pointer, line, row, stage) separately.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(desktop): read XML with a plain DOCTYPE, and check an element's whole text

- Both readers accept a DOCTYPE with only a name and an optional PUBLIC
  or SYSTEM identifier, record it as the view's doctype, and never fetch
  the DTD or expand entities beyond XML's five; an internal subset, an
  external entity and a parameter entity stay refused alike.
- text_equals and text_contains compare the concatenated text of the
  element (or text node or attribute) a pointer reaches, as XPath
  string() does, with optional normalize_whitespace; both evaluators
  agree, and a text check on a truncated element is undecided.
- Whitespace-only text between elements is dropped only when the element
  holds no other text, so text split into runs keeps its spaces.
- The draft suggests text_equals for an element whose text changed.

So an AbiWord .abw reads as XML, and its paragraph's text is checked
whole however AbiWord splits it into <c> runs, as Writer's spans are.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): draft whole-text checks on the paragraph, read only text as text, and skip asctime stamps

- A changed value is read at both ends, so text that became a run still
  marks its paragraph, and the drafted text_equals moves up out of mixed
  content to the nearest ancestor named by an attribute, such as
  AbiWord's p by its xid.
- Text checks accept only a text node or an attribute value; element
  names and doctype fields fail, in both evaluators.
- isTimestamp also recognises asctime dates (Www Mmm dd hh:mm:ss yyyy).
- XML files are drafted with the keyed diff the archive entries use, so
  no suggestion reaches an element by position; the shared logic moves to
  appTaskDraftNamed.ts.
- Text-check tests assert facts, not whole sentences.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep checks on a changed attribute inside a paragraph whose text check replaces its runs'

A drafted whole-text check on a paragraph hid every change inside it, so a
link whose label and href both changed drafted only the paragraph's text and
the new destination went unchecked. The text check now covers text and runs
added or removed whole; a changed name or attribute of an element both
versions hold keeps its own check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): read an attribute called name or attributes as an attribute when the draft keeps its check

The owner of `.../attributes/name` was taken as `.../attributes`, so a
changed `name` attribute inside a paragraph with a text check was dropped
again. The owner is now read by segment, attributes first.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): draft text checks from text in runs added whole, and name a text element by its own id before its container

Text typed into a run the diff lists whole was never read, so the draft kept
only a presence check on the run, which wrong text also passed. Such text now
marks the nearest element both ends hold, or, when none can take the check,
the added element's own text. The climb also names an element by an
identifier of its own (id, xml:id, xid) before a larger container, so a
headline under a long layer keeps a whole-text check that survives re-split
runs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): expect the whole-text check where #8988's pairing test meets text checks

On this branch a changed element's text is drafted as `text_equals` on the
element, so the identity pairing test now expects that check, under the
same `[@role=body]` key.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(skills): build a computer-app environment from a request

The product skill gains references/desktop-build.md, a procedure that
takes Claude from a request such as "an Inkscape environment where the
agent changes a poster's headline" to a witnessed bundle and a report,
through start_build_computer, act_on_build_computer, the build capture
tools, export_build_environment and check_build_task: questions asked
once upfront, pauses only when blocked, a seed made in the app, content
checks by selector, a witness with a near miss and a reset in a fresh
computer, and solver runs only within a budget. setup installs the new
reference; skillVersion 61.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): fold the blind Inkscape rebuild's friction into the build skill

key takes an array of one chord's keys; check for a text cursor before
typing in apps with single-key shortcuts; in Save As, click Documents and
type only the file name; anything saved before seed, the Trash included,
is part of the seed; dismiss first-run dialogs before seed; a bundle's app
starts with the seed's files but need not open the document; Inkscape's
window class is Inkscape.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): fold the held-out AbiWord build's friction into the build skill

Name the act fields and link the guide; a region screenshot becomes the
latest frame; a drawn caret is not focus; confirm dialogs with Return;
check the dialog's folder when the app shows only a name; an XML file the
draft shows as lines was read as text, so use short line checks; and keep
text checks to the value the edit types, since some apps save a reloaded
document differently. Later witness stages take no directory.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): state the export's seed rule, the Trash exception and draft refusals exactly

The export refuses a task unless at least one check fails on the seed and
every check passes at the end; a file moved to the Trash is left out of
the seed, so trash a stray file rather than starting over; a null draft
names its cause in draft_refusal; and a line check on a format read as
text only approximates a value, so bound it and call it approximate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): teach the build skill to use DOCTYPE and whole-text checks

The skill now checks text an app splits into runs whole with text_equals or
text_contains, finds an AbiWord paragraph by its xid, and drops the Writer
revision-id workaround.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): point text checks at the paragraph, and say what the scrub replaces

From the AbiWord rebuild: point a text check at the paragraph or another
element the edit keeps, never at a run a reopened document splits; how to
name an unchanged element; the export replaces the host name, user@host
and home path while a bare user name stays; AbiWord's entry and class.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep every text check on a cut element undecided, in both evaluators

The XML reader drops text deeper than its depth limit but keeps later
siblings, and an element's whole text joins both sides, so a text_contains
on a cut element could match text the file never holds ("at 3" + deep
"WRONG" + ":30 PM" read as "at 3:30 PM"). Any text check on an element of a
cut view is now undecided again, as #9050 had it, in REA's evaluator and the
exported verifier; a text node or attribute value stays decided. The text
check tests assert facts rather than sentences.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(desktop): open the seed's document at launch, and say when a seed was made for the task

An exported app now opens the seed's one document at launch when its desktop
entry's Exec takes files, so a solving agent starts in the document rather
than on the app's start screen; a command that already names a file is kept,
and several documents or an Exec without a file code keep the launch and say
why. `--synthetic-seed`, which the build export passes, words the README and
limitations for a seed made in the build computer instead of asking to keep
the user's data local.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): count only a document the launch names, absolute or from its working directory, as already opened

A settings path under the home, such as geany's --config, kept the seed's
document from opening, and a relative document argument was missed and
appended again. An argument now counts only when it resolves, from the
captured working directory, to a file the capture gave the document role.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): check the seed and launch wording against their formatters, and the facts on their own

The tests froze the README's and limitations' sentences, so a harmless
rewrite broke them. The wording now comes from exported formatters
(`seedWording`, `seedLimitation`, `launchDocumentNote`,
`NO_LAUNCH_DOCUMENT`), the tests compare against those, and check the facts
separately: which document opens, the respelled path in the command, the
README naming it, `--synthetic-seed` and the Evidence's `synthetic_seed`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* refactor(desktop): plan the export's seeds and launch in their own module

The merged export module passed the file and function size limits; the seed
digests and the launch command now come from `DesktopExportLaunch.ts`.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep a reset applied when the screenshot asked after it fails

Once data/reset and the ready wait succeed, act_on_build_computer's reset
is applied in the action log and returns success; a screenshot asked of it
that is then refused or throws is named in the result's screenshot_error,
with no frame, so the next after_reset stage is not refused. Cancellation
still propagates.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep checks for elements without text beside a text edit, and normalize DOCTYPE system id line endings

A review found two gaps.

- A paragraph's whole-text check covered every element added or removed
  inside it, including ones holding no text: an image frame, a line break.
  A solver who typed the text but never inserted the image passed. Such an
  element now keeps its own presence or absence check.
- The Python reader kept a DOCTYPE system literal's CRLF where TypeScript
  normalizes it to LF. Both now report LF.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): wait out long build commands, fail a reset that restores nothing, and tie the witness to its bundle

A review found five gaps.

- Commands in the computer answer only when they end, but fetch stops
  waiting for headers after 300 s. So a slow export, draft or start always
  failed. A command with its own deadline now waits on node:http, bounded
  by that deadline.
- after_reset held whenever some check failed, so a reset that changed
  nothing passed on the near-miss state. Each check must now come out as it
  did at the seed.
- A failed or partial apt-get update left lists that looked complete, so a
  second start skipped the update. The update is now recorded only when it
  succeeds, and rerun otherwise.
- A witness on a bundle computer graded whatever task it was given. Its
  seed stage now refuses a task whose steps or checks differ from the
  bundle's.
- A task too large for one command argument (96 KiB) is refused before the
  export runs, instead of failing with E2BIG in the computer.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): advertise a checkpoint-name pattern that compiles in every JS regex mode

main's advertised-pattern test compiles every input schema pattern in v
mode, which refuses an unescaped hyphen in a character class. The
checkpoint name's hyphen is now escaped, and it means the same in u mode.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): import textSyntaxFormat once in the state format tests

main's mixed-case JSON suffix fix and this branch both added the import,
and the merge kept both.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep a completed reset, an unread seed, and a failed apt update

A cancel after data/reset no longer drops the restore from the action log,
so a near miss cannot treat the restored seed as an edit. An unreadable
seed check that later passes leaves after_reset undecided instead of
failing the witness. apt-get update is kept only when Error-Mode reports
success and a package list was fetched. The controller reply schema lives
with the controller, and capture progress lines are read by their domain
parser.

* fix(desktop): keep a whitespace-only run under the paragraph text check

A run whose only text is whitespace is still text the paragraph check
reads. It no longer gets its own presence check, which a later save of
the same words without that run would fail. Images and breaks, whose
text is empty, still keep their own checks.

* fix(desktop): keep an unread reset and a cut text check undecided

after_reset stays undecided when any check was not read, even if another
check still fails and the unread result's boolean matches the seed. A
text check for a child past a truncated view is undecided in both
evaluators. A reset whose reply is lost blocks a near miss until a later
input or a reset whose reply arrives.

* fix(desktop): retry a witness recording and a published record's cleanup

A recording that fails to end or download is not written into the witness
folder, and a manifest already received is fetched again without ending
the recording twice. A folder that is published while its handles fail to
close keeps that record and retries the same cleanup instead of writing
the folder again.

* fix(desktop): keep a published witness error and tag an app data refusal

A second failure to close a published witness returns the original error,
so its record and cleanup lease stay bound. A reset the app itself
refuses is an AppDataRefusal; only a reply that is not that refusal stays
unsettled. A missing child of a cut view is undecided with the same
description in both evaluators, and the shared cases cover it.

* fix(desktop): align a cut view's missing text with the Python check

A missing child of a truncated view is undecided for the same sentence in both evaluators, including the pointer the view does not contain.

* fix(desktop): keep one published witness cleanup across retries

A published record and its bound cleanup error stay together. A later close failure returns that error, and the unused path that built a fresh adapter error is gone.

* fix(desktop): keep a bundle task read inside the witness result

A cancelled, refused, or failed read of the bundle's task files returns the witness's Result, including a cleanup lease when a handle would not close.

* fix(desktop): check build runtime export capability

* fix(desktop): provide complete runtime repack command

* fix(desktop): retain runtime probe failures

* fix(desktop): recognize captured file URL documents

* fix(desktop): render captured file URLs at launch

* fix(desktop): isolate launcher URL encoding

* test(desktop): move launch execution to boundary suite

* style: fix desktop launch test formatting

* style: format native command path schema

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-09 08:07:40 +08:00
morluto 0b42dc56d1 perf(tests): reuse CLI sessions for analysis data cases 2026-11-09 07:59:49 +08:00
morluto 0d3a2b815e perf(tests): consolidate MCP schema validation 2026-11-09 07:59:49 +08:00
morluto a961db280f fix(filesystem): explain refused file symlinks (#9476) 2026-11-09 07:57:52 +08:00
Ryan S 9d0e87f331 feat(browser): reuse admitted browser-socket transport and allow discovery_timeout_ms (#9203)
* feat(browser): reuse admitted browser-socket transport and allow discovery_timeout_ms

* fix(browser): close unhealthy shared sockets and clamp max discovery timer

* fix(browser): retry failed shared-socket cleanup while peer leases remain active

* test(browser): race observation readiness with awaitAcknowledgement and join before mock restore

* fix(browser): isolate shared-socket discovery aborts and coalesce cold connects

* fix(browser): observe pending cold-connect cleanup failures and isolate fake CDP sessions

* refactor(browser): carry typed connect cleanup owners and gate concurrent session test

* test(browser): cover loopback socket reuse-key normalization

* test(browser): verify distinct browser socket path IDs stay separate
2026-11-08 23:06:56 +00:00
Ryan S 94182f331e feat(go): recover bounded Go string literals from (ptr, len) headers (#9206)
* feat(go): recover bounded Go string literals from (ptr, len) headers

* fix(go): deduplicate header scan spans, bound rejected cache, and reject Cf

* fix(go): allow U+200C ZWNJ and U+200D ZWJ in Go string literal recovery

* fix(go): index Mach-O load regions, validate MCP schema, and add feature specs

* test(go): cover Mach-O region-index cancellation and load-region boundaries
2026-11-08 22:50:32 +00:00
morluto 5553acfe50 fix(elf): report incomplete kernel module coverage (#9475) 2026-11-09 06:09:01 +08:00
morluto 44d232fccb fix(providers): retry live waiters after abandoned startup drains (#9467)
* fix(providers): retry live waiters after abandoned startup drains

* test(providers): bound startup lifecycle acknowledgements
2026-11-09 06:07:27 +08:00
morluto 5a25d42f23 fix(desktop)!: require contemporaneous SQLite member evidence (#9470)
* fix(desktop)!: require contemporaneous SQLite member evidence

* fix(desktop): preserve incomplete baseline membership uncertainty
2026-11-09 06:07:07 +08:00
morluto 3602b64c45 fix(providers): retain cleanup ownership across probes and queued work (#9468)
* fix(providers): retain cleanup ownership across probes and queued work

* fix(jeb): retain primary discovery failures during cleanup

* test(providers): bound cleanup and readiness barriers

* test(jeb): scope availability cleanup fault injection

* test(jeb): retain caller error through cleanup recovery
2026-11-09 06:06:10 +08:00
morluto d9c9e6ea67 fix(cli): retain completed Evidence through finalization failures (#9469) 2026-11-08 22:05:17 +00:00
morluto 9e78606e2b fix(inspector)!: retain context lifetimes across protocol ID reuse (#9471) 2026-11-08 22:04:35 +00:00
morluto 00a6524b9a fix(managed): preserve native export coverage during verification (#9472) 2026-11-09 06:03:48 +08:00
ShuaixinHuang 90c95a6b64 fix(apk): validate the Zip64 end record size (#9432)
* fix(apk): validate the Zip64 end record size

* test(ghidra): assert current provider limitations
2026-11-08 21:55:42 +00:00
ShuaixinHuangandmorluto 4063bfda97 fix(protobuf): stop trailers at the HTTP field boundary (#9436)
Co-authored-by: morluto <[email protected]>
2026-11-08 21:54:51 +00:00
morluto 21a5179105 fix(analysis): preserve unknown lookup values (#9474) 2026-11-09 05:54:29 +08:00
morluto 4b4abcc2ee refactor(core): reuse owned validation and Evidence reference rules (#9473) 2026-11-08 21:51:31 +00:00
ShuaixinHuang d0fd1b05fc fix(qlog): mark non-string frame types incomplete (#9433)
* fix(qlog): mark non-string frame types incomplete

* test(ghidra): assert current provider limitations
2026-11-08 21:50:33 +00:00
ShuaixinHuang c0c6488ae9 fix(desktop): preserve UTF-8 ZIP names in exported views (#9431)
* fix(desktop): preserve UTF-8 ZIP names in exported views

* test(desktop): avoid requiring Python 3.12 ZIP overrides

* test(ghidra): assert current provider limitations
2026-11-08 21:47:42 +00:00
morluto 25d3c7237f fix(paths): enforce selected path contracts at admission (#9465) 2026-11-09 03:14:48 +08:00
morluto b4b881116f fix(agentenv): report incomplete folder inventories (#9466) 2026-11-09 03:14:35 +08:00
morlutoandCursor Agent 88cf498036 fix!: preserve results when source publication fails (#8444)
* fix: retain address context cleanup failures

* fix!: preserve address context input errors

BREAKING CHANGE: Strict clients that cached the previous full inspect_address_context output schema must refresh tools/list and accept the typed error field on unavailable facets.

* test: use session contract for address context

* fix!: preserve address context resource recovery

BREAKING CHANGE: Clients validating unavailable context facets against a cached strict full output schema must refresh tools/list to accept resource_constraint errors.

* fix: expose authenticated records refused by the evidence ledger (#8426)

* fix: preserve completed evidence on retention refusal

* fix: preserve context when unretained evidence exceeds delivery limits

* fix(mcp): recover oversized context from session evidence

* fix(evidence): preserve completed atomic write results

* fix: retain completed results when source publication fails

* test: cover bounded recovery with unknown retention

* fix(evidence): preserve recovery identity and finalization facts

* fix(evidence): clarify retention of omitted observations

* test(evidence): compare omission guidance with its owner

* fix(evidence): preserve completed composed workflows

* fix(evidence): clarify unretained recovery export

* test(evidence): cover mixed recovery guidance

* fix(evidence): preserve caller scope in secondary failures

* refactor(evidence): remove superseded cleanup projection

* test(evidence): align recovery checks with owners

* fix: preserve cleanup on partial recovery limits

* test: use owners for diagnostic wording

* fix: admit completed analysis before evidence construction

* fix: preserve selected paths when reusing evidence

* fix(cli): preserve caller path in unretained evidence

* fix(evidence): preserve recovery metadata and admission bounds

* refactor: consolidate evidence closure imports

* style(test): format CLI help default annotations (#9280)

* test(mcp): keep recovery checks focused on contract facts

* fix(evidence): preserve feature trace replay revision

* test: align evidence recovery checks with ownership

* test: align partial recovery and cleanup diagnostics

* fix(evidence): preserve nested partials during cleanup

* fix(evidence): respect outer recovery ownership

* fix(capture): discard rejected nested observations

* test(frida): assert inline evidence recovery

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-09 02:41:54 +08:00
Ysopwnableandgensokyo-dev 559083be29 docs(security): define REA threat model and security properties (#9384)
* docs(security): define REA threat model and security properties

* docs(security): clarify the HTTP MCP trust boundary

---------

Co-authored-by: gensokyo-dev <[email protected]>
2026-11-08 18:11:38 +01:00
morluto 5044e20171 test: focus diagnostics and evidence checks on behavior (#9303)
* test: focus configuration diagnostics on their owners

Preserve shape errors, selected paths, recovery guidance, and unchanged-file behavior across parser, setup, and uninstall boundaries.

* test: focus artifact and evidence checks on behavior

Retain format, identity, malformed-input, and cleanup coverage while removing fixed prose and closing the admitted test source.

* refactor(debugger): use configured session admission limits

* test: focus evidence and diagnostics on behavior

* fix(test): restore verifier export and formatting

* test(config): verify debugger limit ranges

* test(rizin): keep fixture executable authoritative

* test(rizin): isolate debugger fixture environment
2026-11-08 16:39:20 +00:00
Ysopwnableandgensokyo-dev 08637b0c24 fix(deps): patch Vite and isolate ASAR tooling (#9374)
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-08 17:15:52 +01:00
morluto c5ed046d6e fix(ghidra)!: qualify memory effect coverage (#8651)
* fix(ghidra): qualify memory effect coverage

* fix(native): retain value trace source limitations

* fix(native): distinguish value trace policy commitments

* fix(ghidra): bind value-flow scope to snapshot profile

* test(ghidra): verify legacy value-flow snapshot recovery
2026-11-08 15:05:51 +00:00
03a3fcbcea feat(skills): build a computer-app environment from a request (#9049)
* feat(desktop): capture, draft, export and witness a task inside a build computer

- start_build_capture runs REA's capture inside the computer through a
  small controller over computer-exec, after a user-space apt-get update;
  mark_build_checkpoint returns once the capture confirms the checkpoint;
  stop_build_capture returns the draft between seed and end.
- export_build_environment exports with the requester's task inside the
  computer and fetches the bundle through data/get in bounded batches.
- check_build_task grades a computer's data/get with REA's evaluator and,
  through seed, after_edit, near_miss and after_reset stages, records a
  witness with the action log and the screen recording; act_on_build_computer
  gains reset.
- Capture progress lines carry window, checkpoint and refusal events; apps
  start in the home; a pack script builds the runtime archive; long tools
  send progress notifications.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): make build capture and witness steps retryable, and close the near-miss reset hole

- A capture whose start answer is lost is stopped at once and kept
  unreleased until the stop is confirmed; a lost checkpoint answer is
  recovered from the capture's own log, and draft and export use the
  checkpoints its Evidence records; stop can be repeated to resume
  reading its result.
- A witness can check its seed again after a failed read, and keeps its
  stages in a saving state until the record is written, so a failed save
  can be retried without reading the app again.
- after_edit and near_miss count only click, drag, scroll, type and key,
  so a reset no longer stands in for the near-miss edit.
- A .deb build keeps its package file for capture and passes its URL to
  export; bundle downloads write each checked file before fetching the
  next; an apt-get update failure reports its exit code and log tail;
  reset sends progress while it waits.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep the Trash out of seeds, bundle folders unwritten and first frames drawn

From the blind Inkscape rebuild:
- Paths in ~/.local/share/Trash get a desktop_trash reason and the
  shared_desktop role, so a file trashed before seed is neither owned nor
  seeded; the export names it in a limitation.
- The keep helper runs with PYTHONDONTWRITEBYTECODE=1, so starting a
  computer from a bundle folder no longer writes __pycache__ into it.
- start_build_computer waits, bounded, until the first frame is not one
  colour before returning it.
From review: a .deb that apt also offers exports from apt, since export
passes --package-url only when the capture says the package must come
from its file.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): bound the first-frame wait's retries to the time left, with progress throughout

Each retry of the drawn-first-frame wait runs under an abort that fires at
the 10 s deadline, and the latest frame is returned when time runs out.
connect takes the first frame itself while its progress heartbeat still
runs, reporting "waiting for the desktop to draw". The wait moves to
DrawnScreenshot.ts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): serialize each build computer's calls, keep one recording id, and separate unreadable checks from failures

- One queue per computer: actions, capture steps and witness checks run one
  at a time, so two overlapping after_edit checks can no longer both be filed
  and let the witness hold without a near miss. Close cancels the running and
  queued calls; different computers still run concurrently.
- The witness keeps one recording request id per computer, so a seed retried
  after a lost reply recovers the recording it started.
- A check that could not read its file is undecided, not a failure, so a
  temporary read error can't satisfy seed, near_miss or after_reset; such a
  stage is refused and can be checked again.
- The export's file listing is written to a file and read back in bounded
  batches, so a large bundle no longer overruns exec's 1 MiB output.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): treat failures on cut views as undecided, require a decided seed failure at export, and keep a close's cancellation

- A presence check that misses its line, row or pointed value in a view cut
  short is undecided rather than failed, since the data may sit in the part
  left out; a witness stage can no longer be proved by such a failure.
- Export counts only decided failures on the seed, and refuses a task whose
  every seed failure is undecided.
- A step whose turn was aborted by a close or the caller returns cancelled
  even when its work finished, so start_build_capture no longer reports
  success for a computer being removed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep a step's saved result when a cancellation arrives during its commit

Reporting cancelled for any step whose turn was aborted after its work
replaced a completed export or witness save with a bare cancellation, losing
what was saved and leaving the witness unreadable. Steps now return what they
completed; only start_build_capture, which commits nothing, checks its turn
after the post-launch screenshot and reports cancelled. Asking again for a
saved witness's final check returns its record.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): refuse a saved witness's repeated check for another folder

Asking again for the check that ended a witness returned its saved record
whatever directory came with it, so a caller naming another folder got
success though nothing was saved there. A different folder is now refused,
as an unfinished seed's retry already is.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): assert the facts of cut-view and undecided outcomes, not their sentences

The new cut-view, missing-value, undecided-refusal and witness tests froze
whole descriptions, so a harmless rewording broke them. They now check each
outcome's id, passed and undecided exactly and the facts its message must
carry (file, pointer, line, row, stage) separately.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(desktop): read XML with a plain DOCTYPE, and check an element's whole text

- Both readers accept a DOCTYPE with only a name and an optional PUBLIC
  or SYSTEM identifier, record it as the view's doctype, and never fetch
  the DTD or expand entities beyond XML's five; an internal subset, an
  external entity and a parameter entity stay refused alike.
- text_equals and text_contains compare the concatenated text of the
  element (or text node or attribute) a pointer reaches, as XPath
  string() does, with optional normalize_whitespace; both evaluators
  agree, and a text check on a truncated element is undecided.
- Whitespace-only text between elements is dropped only when the element
  holds no other text, so text split into runs keeps its spaces.
- The draft suggests text_equals for an element whose text changed.

So an AbiWord .abw reads as XML, and its paragraph's text is checked
whole however AbiWord splits it into <c> runs, as Writer's spans are.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): draft whole-text checks on the paragraph, read only text as text, and skip asctime stamps

- A changed value is read at both ends, so text that became a run still
  marks its paragraph, and the drafted text_equals moves up out of mixed
  content to the nearest ancestor named by an attribute, such as
  AbiWord's p by its xid.
- Text checks accept only a text node or an attribute value; element
  names and doctype fields fail, in both evaluators.
- isTimestamp also recognises asctime dates (Www Mmm dd hh:mm:ss yyyy).
- XML files are drafted with the keyed diff the archive entries use, so
  no suggestion reaches an element by position; the shared logic moves to
  appTaskDraftNamed.ts.
- Text-check tests assert facts, not whole sentences.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep checks on a changed attribute inside a paragraph whose text check replaces its runs'

A drafted whole-text check on a paragraph hid every change inside it, so a
link whose label and href both changed drafted only the paragraph's text and
the new destination went unchecked. The text check now covers text and runs
added or removed whole; a changed name or attribute of an element both
versions hold keeps its own check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): read an attribute called name or attributes as an attribute when the draft keeps its check

The owner of `.../attributes/name` was taken as `.../attributes`, so a
changed `name` attribute inside a paragraph with a text check was dropped
again. The owner is now read by segment, attributes first.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): draft text checks from text in runs added whole, and name a text element by its own id before its container

Text typed into a run the diff lists whole was never read, so the draft kept
only a presence check on the run, which wrong text also passed. Such text now
marks the nearest element both ends hold, or, when none can take the check,
the added element's own text. The climb also names an element by an
identifier of its own (id, xml:id, xid) before a larger container, so a
headline under a long layer keeps a whole-text check that survives re-split
runs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): expect the whole-text check where #8988's pairing test meets text checks

On this branch a changed element's text is drafted as `text_equals` on the
element, so the identity pairing test now expects that check, under the
same `[@role=body]` key.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* feat(skills): build a computer-app environment from a request

The product skill gains references/desktop-build.md, a procedure that
takes Claude from a request such as "an Inkscape environment where the
agent changes a poster's headline" to a witnessed bundle and a report,
through start_build_computer, act_on_build_computer, the build capture
tools, export_build_environment and check_build_task: questions asked
once upfront, pauses only when blocked, a seed made in the app, content
checks by selector, a witness with a near miss and a reset in a fresh
computer, and solver runs only within a budget. setup installs the new
reference; skillVersion 61.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): fold the blind Inkscape rebuild's friction into the build skill

key takes an array of one chord's keys; check for a text cursor before
typing in apps with single-key shortcuts; in Save As, click Documents and
type only the file name; anything saved before seed, the Trash included,
is part of the seed; dismiss first-run dialogs before seed; a bundle's app
starts with the seed's files but need not open the document; Inkscape's
window class is Inkscape.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): fold the held-out AbiWord build's friction into the build skill

Name the act fields and link the guide; a region screenshot becomes the
latest frame; a drawn caret is not focus; confirm dialogs with Return;
check the dialog's folder when the app shows only a name; an XML file the
draft shows as lines was read as text, so use short line checks; and keep
text checks to the value the edit types, since some apps save a reloaded
document differently. Later witness stages take no directory.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): state the export's seed rule, the Trash exception and draft refusals exactly

The export refuses a task unless at least one check fails on the seed and
every check passes at the end; a file moved to the Trash is left out of
the seed, so trash a stray file rather than starting over; a null draft
names its cause in draft_refusal; and a line check on a format read as
text only approximates a value, so bound it and call it approximate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): teach the build skill to use DOCTYPE and whole-text checks

The skill now checks text an app splits into runs whole with text_equals or
text_contains, finds an AbiWord paragraph by its xid, and drops the Writer
revision-id workaround.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(skills): point text checks at the paragraph, and say what the scrub replaces

From the AbiWord rebuild: point a text check at the paragraph or another
element the edit keeps, never at a run a reopened document splits; how to
name an unchanged element; the export replaces the host name, user@host
and home path while a bare user name stays; AbiWord's entry and class.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep every text check on a cut element undecided, in both evaluators

The XML reader drops text deeper than its depth limit but keeps later
siblings, and an element's whole text joins both sides, so a text_contains
on a cut element could match text the file never holds ("at 3" + deep
"WRONG" + ":30 PM" read as "at 3:30 PM"). Any text check on an element of a
cut view is now undecided again, as #9050 had it, in REA's evaluator and the
exported verifier; a text node or attribute value stays decided. The text
check tests assert facts rather than sentences.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep a reset applied when the screenshot asked after it fails

Once data/reset and the ready wait succeed, act_on_build_computer's reset
is applied in the action log and returns success; a screenshot asked of it
that is then refused or throws is named in the result's screenshot_error,
with no frame, so the next after_reset stage is not refused. Cancellation
still propagates.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep checks for elements without text beside a text edit, and normalize DOCTYPE system id line endings

A review found two gaps.

- A paragraph's whole-text check covered every element added or removed
  inside it, including ones holding no text: an image frame, a line break.
  A solver who typed the text but never inserted the image passed. Such an
  element now keeps its own presence or absence check.
- The Python reader kept a DOCTYPE system literal's CRLF where TypeScript
  normalizes it to LF. Both now report LF.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): wait out long build commands, fail a reset that restores nothing, and tie the witness to its bundle

A review found five gaps.

- Commands in the computer answer only when they end, but fetch stops
  waiting for headers after 300 s. So a slow export, draft or start always
  failed. A command with its own deadline now waits on node:http, bounded
  by that deadline.
- after_reset held whenever some check failed, so a reset that changed
  nothing passed on the near-miss state. Each check must now come out as it
  did at the seed.
- A failed or partial apt-get update left lists that looked complete, so a
  second start skipped the update. The update is now recorded only when it
  succeeds, and rerun otherwise.
- A witness on a bundle computer graded whatever task it was given. Its
  seed stage now refuses a task whose steps or checks differ from the
  bundle's.
- A task too large for one command argument (96 KiB) is refused before the
  export runs, instead of failing with E2BIG in the computer.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): advertise a checkpoint-name pattern that compiles in every JS regex mode

main's advertised-pattern test compiles every input schema pattern in v
mode, which refuses an unescaped hyphen in a character class. The
checkpoint name's hyphen is now escaped, and it means the same in u mode.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(desktop): import textSyntaxFormat once in the state format tests

main's mixed-case JSON suffix fix and this branch both added the import,
and the merge kept both.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(desktop): keep a completed reset, an unread seed, and a failed apt update

A cancel after data/reset no longer drops the restore from the action log,
so a near miss cannot treat the restored seed as an edit. An unreadable
seed check that later passes leaves after_reset undecided instead of
failing the witness. apt-get update is kept only when Error-Mode reports
success and a package list was fetched. The controller reply schema lives
with the controller, and capture progress lines are read by their domain
parser.

* fix(desktop): keep a whitespace-only run under the paragraph text check

A run whose only text is whitespace is still text the paragraph check
reads. It no longer gets its own presence check, which a later save of
the same words without that run would fail. Images and breaks, whose
text is empty, still keep their own checks.

* fix(desktop): keep an unread reset and a cut text check undecided

after_reset stays undecided when any check was not read, even if another
check still fails and the unread result's boolean matches the seed. A
text check for a child past a truncated view is undecided in both
evaluators. A reset whose reply is lost blocks a near miss until a later
input or a reset whose reply arrives.

* fix(desktop): retry a witness recording and a published record's cleanup

A recording that fails to end or download is not written into the witness
folder, and a manifest already received is fetched again without ending
the recording twice. A folder that is published while its handles fail to
close keeps that record and retries the same cleanup instead of writing
the folder again.

* fix(desktop): keep a published witness error and tag an app data refusal

A second failure to close a published witness returns the original error,
so its record and cleanup lease stay bound. A reset the app itself
refuses is an AppDataRefusal; only a reply that is not that refusal stays
unsettled. A missing child of a cut view is undecided with the same
description in both evaluators, and the shared cases cover it.

* fix(desktop): align a cut view's missing text with the Python check

A missing child of a truncated view is undecided for the same sentence in both evaluators, including the pointer the view does not contain.

* fix(desktop): keep one published witness cleanup across retries

A published record and its bound cleanup error stay together. A later close failure returns that error, and the unused path that built a fresh adapter error is gone.

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: morluto <[email protected]>
2026-11-08 22:19:28 +08:00
gensokyo-devandgensokyo-dev 72d5e72e6d perf(tests): reuse CLI sessions in Electron call-path regressions (#9335)
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-08 07:14:10 +00:00
gensokyo-devandgensokyo-dev c818ca9042 fix(ci): consume owned command failure output in verifiers (#9334)
* fix(ci): read owned CLI failure output in Apple verifier

* fix(ci): preserve packaged Ghidra command failure diagnostics

---------

Co-authored-by: gensokyo-dev <[email protected]>
2026-11-08 06:32:49 +00:00
gensokyo-devandgensokyo-dev 2ac8578827 fix(ci): register standalone function bundle verifier (#9332)
Co-authored-by: gensokyo-dev <[email protected]>
2026-11-08 06:06:39 +00:00
morlutoandCursor Agent b81da90354 feat(native): page retained procedure references (#8588)
* feat(native): page retained feature traces

* fix(snapshot): retain linked source evidence for replay

* test(snapshot): verify linked foreign evidence stays scoped

* feat(native): page retained procedure references

* docs(mcp): clarify imported residual questions

* docs(mcp): guide trace reference classification

* fix(cli): report result sources when Evidence publication fails

A trace keeps linked sources on the result. Use those records for the
undelivered source closure as well as the bundle, so a refused output
does not report them as not captured.

* ci: rerun pull request checks

The merge of origin/main did not start the CI workflow for this head.

* fix(native): document shared procedure endpoints and the required selector

Describe kind shared and items.endpoints instead of inline procedure copies.
State that procedure-references requires a positional selector or --procedure.

* fix(native): count procedure-reference endpoints before keeping a row

Include the shared endpoint table in the page budget, with JSON punctuation,
and store a new procedure only when its row is accepted.

* fix(native): measure a procedure endpoint once before its comparison key

Size the candidate before building one canonical identity, and reuse that
key for the selected-procedure and shared-endpoint checks.

* fix(native): reject an oversized procedure endpoint before canonicalizing it

When the JSON size already exceeds every stored identity, skip the canonical
string and keep the row out of the page.

* fix(native): describe procedure-reference direction and coverage

Avoid execution wording in the tool description, and give the CLI direction
option the help text the catalog requires.

* fix(native): compare endpoint size with UTF-8 identity length

Keep one running maximum of stored identity bytes. A multibyte selected
procedure is recognized when its JSON no longer fits the page.

* test(cli): count procedure-references as a dedicated command

The MCP tool has a CLI command, so it is no longer an MCP-only exception.

* test(native): catch the string-length endpoint comparison

Use a 200,000-character multibyte selected procedure and a shorter ASCII
endpoint so the old JavaScript length check drops the selected row.

* ci: rerun after the binary-layout toolchain timeout

The required check failed because the toolchain install exceeded 15 minutes.
The same job completed on the previous head, and this commit does not change
the procedure-reference behavior.

* test: stabilize capture decoder exhaustion coverage

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 02:16:24 +00:00
morlutoandCursor Agent 887b99888e fix(ghidra)!: distinguish skipped overlays in byte search (#8608)
* fix(ghidra)!: distinguish skipped overlays in byte search

BREAKING CHANGE: Byte search results require skipped_overlays; loaded overlays no longer appear in skipped_not_loaded. The Ghidra profile changes so prior snapshots are not replayed.

* fix(cli): explain overlay byte search scope

* test(ghidra): exercise overlay search cursor handoff

* test(ghidra): avoid freezing byte search help wording

* test(ghidra): classify search start as an address bound

* docs(ghidra): match the raw lane's overlay coverage

The raw Ghidra lane checks a manually created overlay block through MCP.
Say that, and do not claim an overlapping-load ELF, CLI search, or snapshot
cursor handoff.

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 01:49:58 +00:00
morluto 0b028cc75a feat(mcp): refresh provider candidates without switching (#8487)
* fix(mcp): report provider candidate observation scope

* feat(mcp): refresh provider candidates without switching

* docs: clarify candidate inspection ownership

* fix(mcp): clarify provider reselection on reopen

* test(mcp): bound candidate refresh lifecycle gates

* test(mcp): keep temporary fixture cleanup with its owner
2026-11-08 01:35:39 +00:00
morluto 1a0e2c77f2 fix(ghidra): qualify selected project call coverage (#8391)
* fix(ghidra): qualify selected project call coverage

* test(ghidra): retain verifier MCP cleanup ownership

* test(ghidra): document trusted verifier compiler selection

* test(ghidra): document owned verifier exit marker

* test(ghidra): remove ineffective scan directives

* fix(ghidra): supervise project coverage verification

* fix(ghidra): supervise verifier compiler process
2026-11-08 08:57:05 +08:00
morlutoandCursor Agent b40b0b8665 feat: page recorded Evidence arrays (#8863)
* feat(kuna): verify release 1.990

* feat(kuna): preserve stored pointer references

* fix(kuna): group image-wide reference coverage gaps

* fix(evidence): retain producer-required xref observations

* fix(mcp): clarify xref address page scope

* fix(kuna): preserve reference target aliases

* feat: page recorded evidence arrays

* fix(mcp): guide large evidence views to files

* fix: preserve source context in recorded views

* fix: keep recorded rows accessible with large context

* fix: disclose alias scope in recorded references

* test(kuna): identify pointer fixture version

* fix(view): keep deep recorded-array pages inside Evidence depth

Parent parameters are valid on the Evidence root and one level too deep
under a page result. Omit those parameters, and refuse a row that still
cannot be nested, instead of throwing from Evidence parsing.

* test(view): narrow recorded-array depth pages before reading parameters

* fix(view): validate recorded page rows once

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 08:56:24 +08:00
morluto 015a02d57b feat(ghidra)!: identify containing code and data (#8777)
* feat(ghidra)!: identify containing listed instructions

* fix(ghidra): validate instruction containment status

* fix(ghidra): validate containment coordinates

* feat(ghidra): identify containing defined data

* test(ghidra): correct snapshot and containment controls

* docs(native): distinguish object and dynamic type sizes

* docs(cli): retain incoming reference guidance

* fix(mcp): clarify data object address guidance

* test(ghidra): preserve defined data in reference fixtures
2026-11-07 21:26:52 +00:00
morluto 688dca3e90 fix(mcp): report native value flow support in session status (#8786)
* fix(native)!: preserve value-flow absence semantics

* fix(mcp): report native value flow support in session status

* test(mcp): share schema validation and assert status facts

* chore: preserve unrelated main imports
2026-11-07 21:26:13 +00:00
morlutoandCursor Agent 197a296664 fix(session): guide unsupported provider recovery (#8567)
* fix(session): guide unsupported provider recovery

* fix(session): clarify provider switch state loss

* fix(session): distinguish evidence from provider edits

* fix(session): preserve workflow-specific recovery

* test(session): compare recovery with its owner

* fix(session): guide available entry-comment handoff

* fix(session): condition provider handoff on readiness

* fix(cli): clarify target-free provider discovery

* test(mcp): remove duplicated recovery wording checks

* test(mcp): keep provider-selection binding case within the line limit

The merged unknown-provider checks stay in the same scenario and pass
the workspace path explicitly.

* test(mcp): use one temporary directory owner

* test: avoid freezing retry remediation wording

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 21:14:10 +00:00
morluto aa2c1c61ce feat(mcp): report provider release verification in session (#8485)
* feat(mcp): report provider release verification in session

* test(kuna): verify release guidance at its owner

* fix(kuna): verify release against reported version

* test(mcp): keep temporary fixture cleanup with its owner
2026-11-07 21:13:49 +00:00
morlutoandCursor Agent 2d25a09c0f fix(native)!: recognize identical function evidence (#8442)
* fix(native)!: recognize identical function evidence

* test(native): verify same-evidence dimension citations

* fix(compare): reject impossible same-evidence changes

A comparison of one Evidence record cannot report a changed dimension,
unequal subject hashes, or unequal digests on an unchanged dimension.

* test(compare): admit the partial same-evidence fixture explicitly

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 21:06:39 +00:00
morlutoandCursor Agent 9109e6f6a7 fix(ghidra): guide ambiguous selector recovery (#8768)
* fix(ghidra): guide ambiguous procedure selection

* test(ghidra): distinguish malformed selectors

* fix(ghidra): distinguish ambiguous file mappings

* fix(ghidra): name mapping inspection workflows

* docs(mcp): expose strict Ghidra address selection

* docs(cli): explain Ghidra qualified address selection

* test(ghidra): cover published ambiguous selector codes

MCP and projection tests use procedure_ambiguous and file_offset_ambiguous,
and match the recovery main already publishes for those codes.

* fix(ghidra): name the space-qualified address that avoids offset fallback

file_offset_ambiguous now tells the caller to use a form such as
ram:0x1000. A bare 0x selector repeats the file-offset lookup. The MCP
test requires that route and inspect_native_load_image.

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 20:57:13 +00:00
morluto cc58263ee0 fix(trace): disclose address-only reference scope (#8575)
* fix(trace): disclose address-only reference scope

* test(trace): assert qualified reference facts

* test(contracts): distinguish address and process starts

* fix(trace): clarify procedure interior reference scope
2026-11-07 20:29:36 +00:00
morluto 56fb5219ce fix(cli): retain linked sources in function bundles (#8873)
* fix: preserve source closure during snapshot replay

* fix: revalidate restored snapshot sources lazily

* fix: bound and share source-closure checks

* fix(cli): retain function evidence sources in bundles

* fix(cli): preserve partially captured source closure

* test(ghidra): verify saved project evidence bundles

* fix(cli): retain sources across incomplete finalization

* fix: supervise function bundle verifier processes

* style(test): format CLI help default annotations (#9280)

(cherry picked from commit ac30fbd658)
2026-11-07 20:12:25 +00:00
morlutoandCursor Agent f3c21665d8 docs(mcp): clarify Ghidra edit lifetime on switches (#8484)
* docs(mcp): clarify Ghidra edit lifetime on switches

* test(guidance): expect focused provider switch guidance

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 19:50:51 +00:00
morlutoandCursor Agent ce45f92219 docs(mcp): clarify retained evidence view support (#8480)
* docs(mcp): clarify retained evidence view support

* docs(mcp): describe pageable pseudocode alternative

* docs(mcp): page retained procedure pseudocode in place

procedure_pseudo_code Evidence already supports summary and native
pseudocode views of the original observation. The guide no longer says
that paging requires a new analyze_function dossier.

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 19:42:26 +00:00
morlutoandCursor Agent f912a68417 fix(ghidra): respect Java Unix socket path limits (#8382)
Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 02:15:05 +08:00
morlutoandCursor Agent 6ca4e1368c fix(cli): guide analysts to typed incoming references (#8514)
Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 02:14:55 +08:00
morlutoandCursor Agent 1071ad867c fix(test): report caller target selection coverage (#8406)
* fix(test): report caller target selection coverage

* test(kuna): cover passing caller target selection

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 02:04:48 +08:00
morluto c06b53644b fix(ghidra)!: preserve facts in inventory searches (#8376)
* fix(ghidra)!: preserve facts in inventory searches

BREAKING CHANGE: Ghidra snapshots made before the search projection profile revision no longer match. Run fresh analysis and save a new snapshot.

* test(ghidra): update raw profile digest goldens
2026-11-08 02:04:30 +08:00
morluto 352a846127 test: reduce brittle assertions and test-only runtime paths (#9289)
* test: focus assertions on behavioral evidence

Prune redundant and wording-coupled checks while retaining diagnostic, parser, recovery, and cleanup coverage.

* refactor: remove test-only wrappers and duplicate checks

Exercise production operations through test fixtures, remove unused adapters, and keep managed-member uncertainty guidance under its existing owner.

* test: focus process validation on diagnostic facts

* refactor: remove test-only runtime helpers

Move semantic evidence fixture setup out of runtime and remove unused Windows settle recovery helpers while checking configured defaults independently.

* test: focus artifact diagnostics on format facts

* test: verify retained cleanup guidance delivery
2026-11-08 02:02:31 +08:00
morlutoandCursor Agent 5679ed66f4 fix(mcp): surface pageable pseudocode workflow (#8481)
* fix(mcp): surface pageable pseudocode workflow

* fix(mcp): clarify pseudocode paging capabilities

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 01:47:14 +08:00
a67525cdf1 docs: organize canonical documentation layout and verify new pages (#2791)
* docs: organize canonical documentation layout (URGENTE)

Group all existing authored guides by domain and ownership, expose complete navigation, and preserve their public routes. Update physical path consumers and release configuration while retaining historical records. Document domain-based specs and ADR slugs without global numbering.

Co-Authored-By: Codex <[email protected]>

* fix(docs): honor home layout in all-page route verification

* style: format reviewed source changes

* fix(docs): preserve routes and navigation cache inputs

* fix: repair the static-check failures on the default branch (#2895)

* fix: repair the static-check failures on the default branch

The default branch fails its own required `Static checks` job three ways, so
every pull request reports that job as failed regardless of its own change.

- `npm run format:check` reports `src/domain/functionComparison.test.ts` and
  `src/ida/IdaOperationRunner.ts`.
- `npm run lint` reports a duplicate `import { join } from "node:path"` in
  `tests/boundary/mcp/ghidraInventoryMcp.test.ts` (lines 4 and 17).
- `npm run typecheck` reports `format: "devin"` missing from the client format
  union in `src/application/SupportedClients.ts`, and an unchecked index in
  `tests/boundary/cli/clientProfileLifecycle.test.ts`, whose pair array is
  inferred as `string[][]`.

Apply the pinned formatter, drop the duplicate import, list the `"devin"` format
that its own definition already uses, and mark the pair array `as const`.

Validation: `npm run check:fast` and `npm run format:check` pass.

* fix(setup): keep Devin on the JSON configuration dialect

---------

Co-authored-by: 吴佳浩 <[email protected]>
Co-authored-by: morluto <[email protected]>

* fix(docs): update links after guide relocation

* docs: correct verified documentation route counts

* fix(docs): update browser capture guide link after move (#2820)

Refresh gettext protected-link comments alongside the authored href.

Co-authored-by: Codex <[email protected]>

* fix(docs): include http3 lab in stable routes

* fix(docs): preserve historical release evidence and links

* fix(ghidra): restore Windows verifier identity import

* docs(wakaru): preserve pinned upstream policy link

* docs(index): retain recently added guide links

* docs(mcp): preserve parent retention clarification

* fix(cli): preserve standalone import guidance

* fix(test): preserve translated guide link drift coverage

---------

Co-authored-by: Matheus Breguêz <[email protected]>
Co-authored-by: Codex <[email protected]>
Co-authored-by: morluto <[email protected]>
Co-authored-by: 吴佳浩Alben <[email protected]>
Co-authored-by: 吴佳浩 <[email protected]>
2026-11-08 01:42:28 +08:00
morlutoandCursor Agent d5b5808f91 fix(mcp): qualify retained Evidence view guidance (#8377)
* fix(mcp): qualify retained Evidence view guidance

* style(test): format CLI help default annotations (#9280)

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 01:37:50 +08:00
morluto cbd4bd4a66 fix(ghidra): explain literal search candidate evidence (#8816)
* fix(ghidra): explain literal search candidate evidence

* style(test): format CLI help default annotations (#9280)

(cherry picked from commit ac30fbd658)
2026-11-08 01:29:58 +08:00
morlutoandCursor Agent 31278c7b5a fix!: qualify call paths by provider discovery (#8443)
* fix!: qualify call paths by provider discovery

* fix: preserve compact call coverage schema guidance

* fix: preserve call discovery reasons in path evidence

* fix(evidence): retain discovery limits in call verification

* test: declare discovery in reconstruction controls

* fix: preserve call discovery across facet limits

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 01:21:29 +08:00
morluto db0b9ac20c fix(kuna): guide inventory selector recovery (#8397)
* fix(kuna): clarify xref name recovery

* test(kuna): bind xref guidance checks to its owner

* fix(kuna): guide unresolved procedure selectors

* fix(kuna): guide missing procedure address recovery
2026-11-08 01:20:57 +08:00
morlutoandCursor Agent 18cc16141c docs(mcp): clarify residual unknown target scope (#8547)
* docs(mcp): clarify residual unknown target scope

* docs(mcp): explain unknown update subject

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 01:10:54 +08:00
morlutoandCursor Agent 60f9a4c69a fix(native): show function match limitations by default (#8558)
Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 01:10:38 +08:00
morlutoandCursor Agent 4bb53000a3 fix(ghidra): clarify function annotation selectors (#8584)
* fix(ghidra): clarify function annotation selectors

* test(ghidra): compare catalog guidance with its contract

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-08 01:10:17 +08:00
morluto c928bb8d80 test: clarify CLI and process diagnostics 2026-11-08 00:25:43 +08:00
morluto ee54b2e7cf test: clarify process ownership diagnostics 2026-11-08 00:23:38 +08:00
morluto 70fd7923ef test: clarify process capture redaction limitation 2026-11-08 00:22:48 +08:00
morluto 10de107fd5 test: clarify native UI recovery guidance 2026-11-08 00:21:58 +08:00
morluto 407d7f698d test: clarify managed IL boundary diagnostics 2026-11-08 00:18:50 +08:00
morluto 79da7ea8fa test: clarify managed evidence limitations 2026-11-08 00:17:59 +08:00
morluto ff542ada47 test: prune duplicate managed comparison limitation 2026-11-08 00:16:32 +08:00
morluto 75ff321e9e test: clarify Mach-O UUID fallback diagnostics 2026-11-08 00:15:04 +08:00
morluto 33b4762d5c test: prune redundant native parser assertions 2026-11-08 00:13:54 +08:00
morluto 6cf4e768e8 test: clarify Mach-O ciphertext diagnostics 2026-11-08 00:12:43 +08:00
morluto ad597388a4 test: simplify Ghidra coverage propagation fixture 2026-11-08 00:11:24 +08:00
morluto 110932cd5e test: clarify source map and worker diagnostics 2026-11-08 00:08:18 +08:00
morluto 5cafea5ebe test: clarify inspector coverage limitations 2026-11-08 00:06:55 +08:00
morluto 872890bef3 test: clarify environment and desktop contract checks 2026-11-08 00:05:59 +08:00
morluto ab9dc4569f test: decouple desktop diagnostics from exact wording 2026-11-08 00:03:32 +08:00
morluto 1596d6a275 test: reduce wording coupling and prune redundant coverage 2026-11-08 00:01:18 +08:00
morluto 80427af8cc test: repair snapshot and setup isolation checks 2026-11-08 00:01:18 +08:00
morlutoandCursor Agent e47bb152c2 feat(native): retry failed directory decompiles independently (#8811)
* feat(kuna): allow independent decompile retries

* fix(kuna): correct independent retry recovery

* feat(native): retain decompile export source facts

* test(native): verify historical export source facts

* test(kuna): bound retry lifecycle acknowledgement

* test(native): retain source after export conflict

* feat(kuna): allow independent decompile retries

* fix(kuna): correct independent retry recovery

* test(kuna): bound retry lifecycle acknowledgement

* feat(native): retry failed directory decompiles independently

* fix(native): guide directory retry after provider failure

* fix(native): point retry caveats to the journal

* style(test): format CLI help default annotations (#9280)

(cherry picked from commit ac30fbd658)

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:51:21 +08:00
morlutoandCursor Agent 81c09888bf feat(kuna): allow independent decompile retries (#8793)
* feat(kuna): allow independent decompile retries

* fix(kuna): correct independent retry recovery

* test(kuna): bound retry lifecycle acknowledgement

* style(test): format CLI help default annotations (#9280)

(cherry picked from commit ac30fbd658)

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:37:21 +08:00
morlutoandCursor Agent 106ba1730f fix!: preserve address context resource recovery (#8424)
* fix: retain address context cleanup failures

* fix!: preserve address context input errors

BREAKING CHANGE: Strict clients that cached the previous full inspect_address_context output schema must refresh tools/list and accept the typed error field on unavailable facets.

* test: use session contract for address context

* fix!: preserve address context resource recovery

BREAKING CHANGE: Clients validating unavailable context facets against a cached strict full output schema must refresh tools/list to accept resource_constraint errors.

* fix: expose authenticated records refused by the evidence ledger (#8426)

* fix: preserve completed evidence on retention refusal

* fix: preserve context when unretained evidence exceeds delivery limits

* fix(mcp): recover oversized context from session evidence

* fix(evidence): preserve completed atomic write results

* fix(cli): preserve caller path in unretained evidence

* style(test): format CLI help default annotations (#9280)

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:34:20 +08:00
morlutoandCursor Agent f7dc5489b3 feat(kuna): preserve stored pointer references (#8851)
* feat(kuna): verify release 1.990

* feat(kuna): preserve stored pointer references

* fix(kuna): group image-wide reference coverage gaps

* fix(evidence): retain producer-required xref observations

* fix(mcp): clarify xref address page scope

* fix(kuna): preserve reference target aliases

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:33:41 +08:00
3e8a5f76d0 feat(native): retain LLDB source coordinates and verify Pascal calls (#2800)
* feat(native): retain LLDB source coordinates and verify Pascal call entries

Preserve partial line-table attribution for native locations and historical
evidence compatibility. Verify owned Free Pascal call entries through CLI/MCP.

Refs #2382, #2372, #2397

Co-Authored-By: Codex <[email protected]>

* docs: namespace native specifications by domain and feature

Use stable descriptive paths instead of numeric prefixes to avoid parallel contribution collisions.

Co-Authored-By: Codex <[email protected]>

* fix(native): bound Pascal verification and compare caller coordinates

Address review findings at their producing boundaries and preserve directly affected consumers. Source inspection and git diff --check completed. Tests and automated checks were not run at the user’s explicit request.

* fix(native): bound compiler shutdown and source-coordinate fixtures

* docs: clarify historical native source verification

* fix(native): preserve source attribution and compiler ownership

* docs(native): qualify retained source attribution

* fix(ci): clean up native Pascal verifier processes

* test(native): type the Pascal verifier helper import after merging main

Main now typechecks this boundary test, which imports the untyped
canonical-temporary-directory helper and reads an unknown catch value.
Add the helper's declaration file and narrow the caught error.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* test(native): verify partial source attribution caveat

* fix(native): retain verifier compiler cleanup ownership

* fix(native): forward verifier cancellation and measure path bytes

CLI and MCP artifact runs observe the verifier abort signal, and a
failed Pascal verifier test stops the compiler group before deleting
its directory. Repeated source locations grow by measured UTF-8 JSON
bytes; workflow tokens, CPU and peak memory stay unmeasured.

---------

Co-authored-by: Matheus Breguêz <[email protected]>
Co-authored-by: Codex <[email protected]>
Co-authored-by: Kaoru0822 <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 23:25:20 +08:00
morluto ac30fbd658 style(test): format CLI help default annotations (#9280) 2026-11-07 23:02:44 +08:00
Nianze Wuandmorluto 5d21bddb18 fix(desktop): read JSON with deeply nested overwritten values (#9277)
* fix(desktop): read JSON with deeply nested overwritten values

* fix(desktop): bound exported JSON construction by view depth

* fix(desktop): preserve deep JSON numeric facts within bounded memory

---------

Co-authored-by: morluto <[email protected]>
2026-11-07 22:57:15 +08:00
morlutoandCursor Agent 2e2f981ccf test: assert absolute-path facts without frozen sentences (#9231)
* test: assert absolute-path facts without frozen sentences

Sibling path tests now require the field path, invalid_format, and regex
expectation, and they record that each negative case ran. NUL rejection
follows the argument that contains the byte instead of searching for the
word NUL.

* test: retain actionable NUL path diagnostics

---------

Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 22:46:17 +08:00
Heo Hao f8bb792f15 fix(cli): stop restating schema defaults in help descriptions (#9278)
The help renderer appends "(default: X)" whenever the option field declares a
schema default, so descriptions that also spell the default out print it twice:

  --block-size <value>  Power-of-two block size from 256 to 65536 bytes
                        (default 4096) (default: 4096)

Remove the hand-written default from the eleven options that own a schema
default across inspect-byte-entropy, inspect-devicetree,
inspect-packet-capture, inspect-packet-fields, inspect-signal-decoders and
decode-signal-recording, matching the rule in cliDecimalField's contract that
the domain field owns bounds, defaults and help.

Options without a schema default keep their prose: --match_limit, --line_limit,
--limit and --packet_limit are nullable or optional, so the renderer emits no
annotation and the description is the only place the effective value appears.

Add a boundary test asserting each such option states its default exactly once,
so a restated default cannot come back unnoticed.
2026-11-07 22:38:03 +08:00
morlutoandCursor Agent 0ecf3a4cbf docs(mcp): document native function analysis scope (#8389)
Co-authored-by: Cursor Agent <[email protected]>
2026-11-07 22:14:55 +08:00
ShuaixinHuang 8842092671 fix(pe): reject misaligned certificate table offsets (#9276)
* fix(pe): reject misaligned certificate table offsets

* test: align ghidra limitation wording with main

* style: format ghidra limitation assertion
2026-11-07 22:14:29 +08:00
2713 changed files with 201934 additions and 27201 deletions

No files matched your search

@@ -10,8 +10,8 @@ equivalent completed investigations. A useful outcome can also be a measured
decision to leave an implementation alone.
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
[tool design](../../../docs/tool-design.md), and
[testing guide](../../../docs/testing.md). This skill supplies an optimization
[tool design](../../../docs/development/tool-design.md), and
[testing guide](../../../docs/development/testing.md). This skill supplies an optimization
workflow, not additional permission to execute targets, publish, or merge.
Use the current contracts, provider prerequisites, and user authorization.
+1 -1
View File
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
# REA changelog update
Use this for release-note preparation and contributor attribution. Read
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
publication authority and the existing checkpoint checks. This skill prepares
notes; it does not authorize merging, tagging, or publishing.
+1 -1
View File
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
removed without losing that protection. The outcome may be a verified fix,
an evidence-backed recommendation, or an explicit decision to stop.
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
and pinned toolchain, rather than commands remembered from an earlier audit.
Preserve the requested scope: investigation alone does not authorize edits;
+12 -5
View File
@@ -5,7 +5,7 @@ description: Audit, investigate, design, or change REA investigation tools, CLI/
# REA tool design
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
contract semantics, provider ownership, and discoverability. Inspect the current
contract and nearest existing tool relevant to the analyst question; source,
documentation, and live catalogs may describe different versions.
@@ -14,23 +14,30 @@ For provider-backed work, exercise the analyst's task through REA and compare
relevant native provider and independent artifact observations where available.
Check applicable partial results, recovery and session boundaries; report what
was not exercised. For failure guidance, follow [design recovery around the
failure](../../../docs/tool-design.md#design-recovery-around-the-failure).
Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification)
failure](../../../docs/development/tool-design.md#design-recovery-around-the-failure).
Use [workflow verification](../../../docs/development/testing.md#provider-backed-workflow-verification)
to select evidence for the proposed contract instead of seeking provider API
parity.
For an audit, follow [audit existing tools](../../../docs/tool-design.md#audit-existing-tools):
For an audit, follow [audit existing tools](../../../docs/development/tool-design.md#audit-existing-tools):
scope representative tasks, exercise the live workflow, and distinguish
reproduced bugs, supported limitations, design opportunities and unverified
claims. Check relevant rendered MCP prompts, guided workflows and packaged skill
advice against the live contracts and exercise their suggested handoffs. Report
working alternatives and evidence limits as well as findings.
For wording audits and agent trials, use
[focused tool-description trials](../../../docs/development/client-verification.md#focused-tool-description-trials).
Start with representative workflow boundaries and repeat failures or ambiguous
outcomes; keep actual arguments and supported final claims separate. Reuse the
repository's evaluation tooling and report actual runtime/model versions and
synthetic versus production coverage rather than copying a harness into a skill.
For a design request, return the proposed boundary, input/result shape, Evidence
semantics, nearest alternative, and verification approach. Implement when
requested, preserving the user's scope and existing authorization.
For implementation or verification, consult [testing](../../../docs/testing.md)
For implementation or verification, consult [testing](../../../docs/development/testing.md)
for the affected consumer and provider lanes, and
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
provider/workflow guides only as the task requires.
@@ -1,8 +1,8 @@
---
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "60"
version: "62"
---
# REA
@@ -97,6 +97,12 @@ explicit path or endpoint and do not need it.
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
module graph is the application layer. Extract stored modules and analyze the
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
- Deno `deno compile` executable: `open_binary`, then `inspect_artifact`; the
embedded virtual filesystem is the application layer. Extract stored files and
analyze compatible JavaScript. Environment-file values and CA certificate bytes
are retained in local Evidence with their counts and byte lengths. The npm
snapshot length is reported and not decoded, and the executable is not run. See
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
`open_bundle_resources`, then the selected resource tool. This does not
require `CFBundleExecutable` or a deep provider. Resource sessions have no
@@ -108,7 +114,7 @@ explicit path or endpoint and do not need it.
- Firmware image: `inspect_firmware_regions` when advertised. Use
`extract_firmware` when extraction is requested, with a caller-selected new
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
without a provider; see the ESP section of `references/native-and-artifacts.md`.
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
@@ -140,6 +146,10 @@ explicit path or endpoint and do not need it.
- Task for a desktop application's exported environment, from a
`capture-desktop-application` folder: read
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
- Environment of a Linux desktop app with a task, from a request such as
"build me an Inkscape environment where the agent changes a poster's
headline": read [desktop builds](references/desktop-build.md). You drive the
app on a build computer from `start_build_computer`, then export and witness.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
@@ -150,7 +160,7 @@ explicit path or endpoint and do not need it.
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
raise it. See
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
or establish network loss. Prefer detail summary, then kind qlog trace/stream
@@ -165,7 +175,7 @@ explicit path or endpoint and do not need it.
display filters and decode-as rules. Requires Linux, util-linux prlimit and
TShark 4.x on an absolute `PATH` entry or via the optional absolute
`REA_TSHARK_COMMAND` override; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
Packet limits count input packets before filtering. Output defaults to 50 matching
frames; use `match_offset` / `match_limit` for later pages, or explicit null
limit for complete matching output. Preserve the source digest across pages.
@@ -173,7 +183,7 @@ explicit path or endpoint and do not need it.
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
and capinfos, with Linux prlimit or Windows Job Objects; see
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic.
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
@@ -181,7 +191,7 @@ explicit path or endpoint and do not need it.
relevant details before choosing settings; this does not scan hardware.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
Output defaults to 50 literal lines; select annotation classes and page with
`line_offset` / `line_limit`, or explicit null for all remaining output.
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
@@ -190,7 +200,7 @@ explicit path or endpoint and do not need it.
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
Select JavaScript glue and candidate paths explicitly and preserve URL/path
ambiguity. No implicit fetch or target execution; WAT is not original source.
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
Prefer its summary or selected signer/digest/lineage page; retain complete
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
@@ -209,13 +219,17 @@ explicit path or endpoint and do not need it.
Evidence stays retained under its original identity. Use the returned
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
Candidate coverage does not establish interface completeness. See the
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
with the explicit local file. This built-in target-free inspection requires no
Go installation or disassembler. Preserve raw module bytes and source ranges;
absent metadata does not prove a binary is not Go, and unparsed text records
remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
- Embedded Go compiler, module, build-setting metadata, or static `(ptr, len)`
string literals: `inspect_go_binary` with the explicit local file (pass
`include_string_literals: true` and optional `max_string_literals` for bounded
inferred static string-header recovery, and page retained rows with
`inspect_analysis_view` kind `go` collection `string_literals`). This built-in
target-free inspection requires no Go installation or disassembler. Preserve
raw module bytes and source ranges; absent metadata does not prove a binary is
not Go, static string recovery excludes compiler-inlined stack or immediate
strings, and unparsed text records remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
first, then decode selected payloads and query retained GPU Evidence.
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
@@ -268,6 +282,14 @@ explicit path or endpoint and do not need it.
proof the image never imported APIs through another table. Bound snapshots
and loaded IAT contents remain unobserved. See the
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
- Windows Shell Link shortcuts: `inspect_windows_shortcut` with the explicit
local `.lnk` file. This built-in target-free inspection requires no Windows
SDK or COM. Preserve header flags and FILETIMEs, IDList items, LinkInfo
volume and network names, StringData arguments and icon location, ExtraData
signatures, and Tracker droid facts labelled derived. Unknown IDList items
and ExtraData signatures stay raw with SHA-256. The target path is not
resolved or executed. See the
[Windows shortcut guide](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
explicit local PE32/PE32+ file. This built-in target-free inspection requires
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
@@ -291,6 +313,23 @@ explicit path or endpoint and do not need it.
build-id or ABI-tag fields. A missing table is observed absence of those
declarations, not proof the image never carried notes. See the
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
- Code that runs before `main` or at exit in ELF or Mach-O:
`inspect_native_initializers` with the explicit local file (and
`architecture` for a universal Mach-O). This built-in inspection requires no
disassembler. Preserve slot and target addresses, the `resolution` of each
entry, and its limitations. An `unresolved` slot is not a zero pointer, and a
missing table is not proof that no constructor exists. See the
[initializer guide](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
- ELF unwind function ranges: `inspect_elf_unwind_ranges` with the explicit
local ELF32/ELF64 file. This built-in target-free inspection requires no
disassembler. Preserve FDE start, end, size, containing section, exact
symbol name, CIE index, LSDA, and personality. Page with `offset` and
`limit`. A missing `.eh_frame` is observed absence of that declaration,
not proof the image has no functions. `.debug_frame` is reported present
or absent and is not decoded. CFA instructions are counted and not
evaluated. See the
[ELF unwind ranges guide](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
- Apple `.ips` crash reports: `inspect_apple_ips` with the explicit local
file. This built-in target-free inspection requires no Xcode, atos, or
pwntools. Preserve first-line metadata, `bug_type` as a string, process and
@@ -309,6 +348,16 @@ explicit path or endpoint and do not need it.
zstd containers are unsupported; decompress into a separate file with an
external codec utility, then inspect the decompressed `ET_REL` image. See the
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
- Linux kernel version and build config (`CONFIG_*` options such as
`CONFIG_SECURITY_SELINUX`): `inspect_kernel_build_config` with the explicit
local `vmlinux`, raw `Image`, x86 `bzImage`, or `configs.ko`. This built-in
target-free inspection requires no Linux host or `extract-ikconfig`. Preserve
the banner version, config SHA-256, option count, and value kinds; use `name`
for one option. An `IKCFG_ST` string is a config only with a valid gzip member
and `IKCFG_ED`. An absent config may mean the kernel was built without
`CONFIG_IKCONFIG`. Non-gzip payloads are unsupported; decompress externally,
then inspect the result. See the
[kernel build config guide](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
identity: `inspect_python_pyc` with the explicit local file. This built-in
target-free inspection requires no Python installation. Preserve magic,
@@ -332,17 +381,17 @@ explicit path or endpoint and do not need it.
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
report truncation and have unspecified order. Use a quiescent snapshot rather
than assuming a live database/WAL pair was copied atomically. See the
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
- Explicit Linux ELF file for offline layout, symbols, relocations or static
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
caller-supplied pwntools without opening a disassembler database. Preserve its
raw locations and inference/coverage limits; see the
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
thread, raw note and source-bound register without opening a live target.
Historical PIDs do not select live processes. Request `include_debugger_context`
only when core-only mapping candidates help; see the
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
- PE resource type/name/language trees and static icon group references:
`inspect_pe_resources` with an explicit path. This portable operation requires
no active target or engine; opaque payloads retain file ranges and SHA-256.
@@ -358,9 +407,9 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
leaves its database open. The modern headless profile creates and closes an
owned private database without saving. Installation and registration reuse
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
For migration to the separate native IDA MCP plugin, follow the guide's
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/ida-provider.md#migrate-a-legacy-attached-registration).
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
Replace the upstream registration explicitly; modern mrexodia tools are not the
native profile. Native attachment preserves the database but does not provide
direct callers; check session availability and the guide's verification limits.
@@ -539,6 +588,8 @@ as complete while required questions remain open.
[references/environments.md](references/environments.md)
- Tasks for a desktop application's exported environment:
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
- A desktop app environment from a request, built and witnessed on build
computers: [references/desktop-build.md](references/desktop-build.md)
## Finish the task
@@ -22,7 +22,7 @@ Preserve slot/relocation provenance and source coverage: matched means observed
candidate presence; partial scans do not establish uniqueness or absence.
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
target code. Runtime registration and loading remain unknown. See the
[JNI guide](https://github.com/morluto/rea/blob/main/docs/android-analysis.md#link-native-declarations-to-jni-candidates).
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
@@ -11,7 +11,7 @@ main and an older release expose the same arguments. Tool-name prefixes belong
to the client: find the canonical REA operation in its actual tool listing and
server association before concluding that it is absent. A shared prefix alone
does not establish which server an approval wildcard covers. See
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/client-configuration.md#client-tool-names-and-discovery).
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
## Choose the remedy from the observed failure
@@ -0,0 +1,860 @@
# Desktop app environments from a request
Use this when someone asks for an environment of a Linux desktop app with a
task, such as "Build me an Inkscape environment where the agent changes a
poster's headline and saves it". You are the building agent: you drive the
app on a build computer through REA's MCP tools, write the task, export the
bundle and witness it in a fresh computer. REA runs no model. The tools are
MCP-only and need a Linux host with Docker, agentenv and REA's packed
runtime. When a start reports one missing, tell the person to run
`rea doctor --provider desktop_build` and apply its fixes, or
`rea setup --client <client> --desktop-build` and restart the client, and
stop.
For the whole build:
- Drive the app only with `act_on_build_computer`. Never script it, run
commands in the computer or write its files another way. The one command
you write for the computer is a task's reader, which REA runs to check a
file; it never makes the seed or the edit.
- Ask the upfront questions once, then work on your own. Pause only when
blocked.
- The seed's content is synthetic. Never put real personal data in it.
- Close every computer you start, on success and on failure.
- Note the time you start; the report gives the build time.
## Ask once, upfront
In one message, ask only what the request leaves open, and keep the list for
the report. Skip each question the request answers; when it answers them all,
start without asking:
1. The app and the task, if either is unclear: what changes, in which
document.
2. Whether they have a starting file of their own; otherwise you make one.
3. Whether they want the tasks measured after the witness: with which
model, within which budget, and whether to harden a task until it lands
in the band.
4. If so, whether they want to review the tasks before those solves.
When you cannot ask, as in a headless `claude -p` run, do not stop: take
these defaults for whatever the request leaves open, say which you took in
your first message and in the report, and go on. Install from the Ubuntu
archive, make a synthetic seed, skip the review, and measure each task
within $4 in all, with the model the request names, without hardening.
## When you are blocked
Stop, say what you need and wait only for:
- a licence the app or its installer asks someone to accept: never accept it
on your own; name it and where to read it, and go on only once the person
says they accept it. Then record their words with `record_build_consent`,
kind `licence`, and accept it on their behalf;
- credentials: ask the person to write each secret, and nothing else, in a
file named after a handle you choose, such as `keepassxc-master`, in
`~/.config/rea/build-secrets/` on this host (under `$XDG_CONFIG_HOME` when
it is set), readable only by them (`chmod 600`), and wait until they say it
is there; then type it with `type_secret` (see Drive the app). Never ask
for a password or key in chat, and never read, list or print that folder;
- a signed-in state in the seed: by default the export withholds it (see
Sign-in and licence state). Seed it only when the person says the bundle
may hold it: record their words with `record_build_consent`, kind
`seed_sign_in`, and export with `seed_sign_in: true`;
- an install this flow cannot do, below.
Decide everything else yourself and note it in the report.
## Sign-in and licence state
What an app writes when someone signs in or activates a licence is withheld
from the seed: the bundle's receiver supplies their own. Mark
`pre-sign-in` with `mark_build_checkpoint` just before the sign-in or
activation and `post-sign-in` just after, both before `seed`: every file
under the owned paths that changed between them is withheld, and the draft
leaves it out. A file the app writes at first launch, such as a trial's
licence file, changes before you can mark anything: name it in
`export_build_environment`'s `withhold`. The export lists each withheld file
in `withheld_files` with the environment variable a deploy reads it from and
the handle of the secret REA keeps its bytes in; a check may not name one.
The export refuses a seed file holding a withheld file's exact bytes, such as
a `.bak` copy, but a backup that differs from it is not caught: trash backups
before `seed`.
Explicitly withhold any other files the person identifies as sensitive;
REA does not infer secrecy from a value's shape or a file name.
## Choose the install
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
the first source that has the app:
| Source | `install` |
| ---------------------------- | ------------------------------------------------------------------------------------------- |
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
| AppImage by URL | `{"kind": "appimage", "url", "sha256", "architecture", "extra_packages"}` |
| Tarball by URL | `{"kind": "tarball", "url", "sha256", "architecture", "program", "name", "extra_packages"}` |
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
signing key saved on this host; for a PPA, fetch it from
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
- When the app exits at launch for want of a runtime piece its package does
not install, start a new build computer with that package in
`extra_packages`, up to 16 archive names; the export installs them too.
MyPaint needs
`{"kind": "archive", "package": "mypaint", "extra_packages": ["librsvg2-common"]}`
for its SVG icons, and Freeplane `["openjdk-17-jre"]`, since its launcher
refuses noble's Java 21. Name one up front, too, when a package the app
only recommends holds its content, such as QElectroTech's element library
`qelectrotech-data`, without which no component can be placed.
- Take the `.deb` path only when the app is not in the archive at all, not
to add a package. Ask the requester for its URL and SHA-256; do not fetch
a vendor's download page yourself. List its `Depends` as `dependencies`,
with any runtime piece it lacks: it installs with the network off.
- Take an AppImage or a tarball only when no package has the app. Pin it by
the release file's URL for this host's architecture and the SHA-256 its
project publishes, such as a GitHub release asset's digest or the
release's `SHA256SUMS`; ask the requester when the project publishes none.
The computer downloads it when the bundle is put, refuses other bytes, and
unpacks it under `/opt/rea-apps/<desktop_entry>` with the network off; the
export downloads the same file.
- An AppImage installs its own desktop entry, so `desktop_entry` is the
`.desktop` file at the top of the image, such as FreeCAD's
`org.freecad.FreeCAD`. A wrong name fails the start with the entries the
image has: start again with one of them. The image brings its libraries,
so `extra_packages` is rarely needed; FreeCAD 1.1.4's needs none.
- A tarball has no entry of its own. `program` is the program's path inside
the archive, such as `firefox/firefox`; REA writes an entry named
`desktop_entry` that runs it with the files it opens, and `name` is what
the Applications menu shows. A tarball uses the computer's libraries: name
what its program lacks in `extra_packages`. gzip, bzip2 and xz archives
are read, zstd ones are not.
- Flatpak and snaps are not supported: say so and stop. Flatpak's sandbox
needs user namespaces the computer's container does not grant, so a
Flathub app installs but never opens; ask for the project's AppImage,
tarball or `.deb` when it ships one. An archive package that only installs
a snap, such as noble's `firefox`, counts as a snap. A package with no
build for this host's architecture (some PPAs are amd64 only) cannot be
installed either.
- `desktop_entry` is the package's `.desktop` file name under
`/usr/share/applications` without `.desktop`, such as
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
- To find the entry, read the package's file list: its page on
packages.ubuntu.com, or `apt-get download PKG && dpkg -c PKG_*.deb` where
apt runs. The entry may ship in a companion `-data` or `-common` package,
as Planner's in `planner-data`: read that package's list too. The entry's `StartupWMClass` may not be the window's real
class, so do not take `window_class` from it: MuseScore 3's says
`mscore3`, its window's is `MuseScore3`. The export uses the class the
capture saw.
## Drive the app
- Pass `computer_id` and the action's fields at the top level: click
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
`[x0, y0, x1, y1]`. The rest are in
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
- Take a `screenshot` before pointing: points are pixels of the latest
frame. If a response is too large, ask for a `region`, or `jpeg` with a
lower `quality`. A screenshot with `region` does not capture the screen
again; it crops the latest frame, so take a full screenshot, or a `wait`
with `screenshot: true`, first. The crop then becomes the latest frame:
take a full screenshot before pointing again.
- A drawn caret does not prove keyboard focus. After using a toolbar
control, click into the document, then type a few characters and look
before a long text.
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
- An action's own screenshot is taken at once. After opening a menu, a
dialog or a window, look with
`{"action": "wait", "ms": 500, "screenshot": true}`.
- On a computer started from a request the app is not open at first, and
`start_build_capture` opens it: never open it yourself before that. Do not
close it while capturing; the capture ends when the app exits. When the
app is closed at any other time, open it from Applications → Other → the
environment's name.
- Type a secret with `{"action": "type_secret", "secret": "<handle>"}`,
only into a field that masks what is typed: check in a screenshot that it
is a password field, which shows dots. A screenshot shows any field that
shows its text, such as many licence-key boxes, so never type a secret into
one: say the app needs a secret the build cannot type unseen, and stop.
`type_secret` takes no `screenshot`; look with a `wait` after it. A refusal
names the secret's file and what is wrong with it: tell the person, never
the value.
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
array holding one chord's keys, such as
`{"action": "key", "keys": ["Control", "s"]}`, and `type` turns `\n` into
Enter.
- Before `type`, check in a screenshot that a text cursor or a focused
field is there. In apps with single-key shortcuts, such as Inkscape, text
typed without one runs as commands.
- Save As differs by app. Check the dialog's current folder first: most
open in the home, so click Documents in the sidebar, but Scribus opens in
`~/Documents` and its sidebar has no Documents entry. Type only the file
name. A dialog taller than the 1280×800 screen hides its buttons:
maximize it from its title bar, or confirm it with `key` Return, as when
HomeBank's Save button is off-screen. When a GTK Save dialog ignores
clicks or runs off the screen, type `~/Documents/<name>` in its Name field
and press Return.
- Answer the app's save questions, such as keeping the format, then check
the saved path in the title or status bar, or the dialog's folder when
the app shows only the name, and that it shows no unsaved mark, before
you mark a checkpoint or check. Where the title never shows one, as in
HomeBank, a greyed Save or Revert, or the file reopened from recent
files, confirms the save.
- Qt spin boxes change their value on scroll: move the pointer off them
before you scroll.
- gLabels edits an object's text in Object properties → Text, not on the
canvas.
- In Grisbi's transaction form Return moves to the next field; only Return
in the last field, Notes, records the transaction.
- ghostwriter autosaves a new document as `~/Documents/untitled-1.md`: name
it with File → Rename, not Save As; Ctrl+Shift+S does nothing, and Enter
after a checklist item starts another `- [ ]` item.
- KMyMoney's first launch opens the New File Setup wizard; Finish asks for a
storage type (XML), then a Save As where typing `Documents/<name>.kmy`
works. Its date field is a sectioned Qt editor: double-click the month and
type each part, since Ctrl+A garbles it. A new transaction's payee comes
prefilled and selected. Amounts are saved as reduced fractions, such as
`-1749/20`, so compare them exactly. Untick Help → Tip of the Day → "Show
tips on startup" before `seed`, or the tip appears at every boot.
- QElectroTech: an element dropped from Collections follows the pointer until
a click on the folio places it. Its title shows only the app, so it needs
`state_paths`; its settings are in `~/.qet`.
- Rosegarden: after a dialog its Transport window can take keyboard focus,
so save with File → Save.
- Pencil2D: the Tools dock is too narrow for its buttons, so use shortcuts
(B for the brush); double-click a layer's name to rename it.
- Umbrello: the title shows no unsaved mark, so confirm a save with a probe
checkpoint. An Untitled document autosaves `~/autosave.xmi`: delete it
before `seed`. Add attributes from the tree: right-click the class → New →
Attribute.
- Hydrogen: the computer has no audio device, so Hydrogen falls back to a
null driver and its BPM field snaps back to 120 within about 100 ms.
Before `seed`, set Preferences → Audio System → ALSA with device `null`
and click "Apply and restart output", which `~/.hydrogen/hydrogen.conf`
keeps for the seed; also set autosaves per hour to 0, and dismiss the
development-version dialog with "Don't show again". `pulseaudio` as an
extra package does not help.
- Poedit cannot add entries by hand. Use File → New, pick the language and
save to `Documents` first (Sources Paths stays locked until then), then
Properties → Sources Paths → + → Add Files with a source such as
`/usr/lib/python3.12/argparse.py`. It writes `fr.mo` beside the `.po`, and
the language box's autocomplete popup takes the first OK click.
- CherryTree: its first-run autosave "Choose Storage Type" dialog swallows
keys, so save once early as `.ctd` and untick autosave.
- VSCodium (its tarball, with `extra_packages: ["libnss3",
"libasound2t64"]`) runs on outside REA's reach, so `stop_build_capture`
fails to clean it up: after `end`, quit it with Ctrl+Q, then stop. Enter
accepts its word suggestions, so press Escape after typing a line.
- Glade: a new widget's id is empty until you set it, and a second save
writes a `.glade~` backup, which shows in the draft as an unrelated
change. A backup present at `seed` is seeded with the document, so trash
it before `seed`. The draft names a button's `<child>` by the id
its widget carries, `[name=child][>object@id=submit_button]`, which moving
the button keeps; give the widget an id before `seed` so it can.
- VYM: the title never names the map, so pass `Documents/<map>.vym` in
`state_paths`. Its settings live in `.config/InSilmaril` and are written
only at a clean exit. It writes `<map>.vym.lock` while a map is open, so
close the map before `seed`. Press A only after clicking a branch on the
canvas: keys typed in the tree editor run other shortcuts.
- KmPlot: Create → Cartesian Plot, then edit the formula in the Function
tab and press Return. It stores minus as U+2212 (`−`). Its file:// title
still names the file.
- RedNotebook keeps each month in `~/.rednotebook/data/YYYY-MM.txt`, YAML
read as text, folded at about column 80: pick check phrases the fold keeps
on one line.
- Qt5 apps saving with QDom, such as KmPlot, write attributes in a random
order in each process, so check their XML by element, never by whole line.
- Gnumeric's desktop entry is `org.gnumeric.gnumeric`. Its Save As dialog
runs off the 1280×800 screen: type `~/Documents/<name>` in Name and press
Return. Rename a sheet with right-click on its tab, then Rename….
- Parley's title shows the collection's title, so name the file in
`state_paths`. File → New opens the new collection's Properties dialog:
confirm it with OK, since Cancel keeps the old collection and its autosave
lock open. A new collection starts with 15 blank entries: delete them.
Pick languages from the dropdown; a typed name keeps the previous locale.
While a document is open KDE keeps an autosave and its `.lock` in
`~/.local/share/stalefiles/parley/`, and there is no File → Close: before
`seed`, use File → New, then reopen the collection from Open Recent.
- Sigil: pass `extra_packages: ["libqt5svg5"]`, or its SVG toolbar icons are
blank. Its title names the book, so it needs no `state_paths`. It unpacks
the open book into `.local/share/sigil-ebook/sigil/workspace/Sigil-XXXXXX/`
and deletes it on exit, so never name the whole `sigil-ebook` folder: that
seeds the stale copy.
- What the app saved under the home before `seed`, such as a document in
`~/Documents` or its settings, is part of the seed. A file you move to the
Trash is left out, and the export lists it in `limitations`. Trash a stray
file rather than starting over. JabRef 3.8 writes `<file>.bak` and
`<file>.sav` beside a `.bib` it saves; in the owned folder they are seeded
and show in the draft as noise, so trash them before `seed` if they should
not ship.
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
choice the app saves only later, such as Inkscape's "show this every
time", may not reach the seed, so expect the dialog in the witness too.
- Some apps write their settings late: Freeplane writes
`~/.config/freeplane/1.7.x/auto.properties` only once Preferences is
confirmed, and a seed without it reopens its documentation map. Before
`seed`, open the app's preferences and confirm them, then read a probe
checkpoint's `changed_files` to see the settings file written.
- Before `seed`, click into the document to clear any selection the app
would save: Gnote saves its cursor and selection with the note.
- An app whose state folder is named after it, such as
`~/.local/share/gnote`, or after it and its version, such as TreeLine's
`~/.treeline-3.1`, is found without `state_paths`.
- TreeLine: File → New opens a template window. Book List's CATEGORY makes
AUTHOR children by default; change it under Data → Configure Data Types →
Default Child Type. A new child opens an inline rename showing "", so
press Escape and fill its fields in the Data Edit tab. Ctrl+A adds a child
node even while a field has focus. Clicking a node already selected starts
an inline rename, so Ctrl+X then cuts its text: cut a node from its
context menu. It saves nodes sorted by their
generated uid1, so their order changes from run to run. The draft names a
new node by its format and a field of its `data`,
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and checks its other
fields through that, so give each new node a title no other node has.
- After an app crashes, capture again on a fresh computer, so the seed
carries no crash-recovery state, such as MyPaint's "Recover interrupted
work?".
- Give what the task changes a stable name where the app has one, such as an
id in Inkscape's Object Properties, so a check can select it.
- Expect apps to split text into runs: Writer's revision ids split an
edited paragraph into spans and may give it an automatic style such as
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
with `text_equals` (see Write the task), and find the paragraph by
something the edit keeps, such as AbiWord's `xid`.
- Freeplane's first launch opens a documentation map: close it with Ctrl+W.
Its Ctrl+N asks for a template.
- MyPaint 2.0.1 crashes on brush strokes on the build computer: build its
seed with flood fill and layer operations.
- GnuCash's Ctrl+S on a new book opens a Save dialog with no format choice:
use File > Save As to pick XML. Close the book (File > New File) before
`seed`, so its `.LCK` lock is not seeded.
## Build the seed and the edit
1. `start_build_computer` with
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
The first start of an app builds its image, and starts at the same time
on this host wait for each other; either can take minutes. A failed
start returns the build's error: fix the request, or stop when the
install cannot work, and close any `computer_id` the error names.
2. `start_build_capture`. It returns once the app's window is open. A
`package_lists_error` may stop the capture finding the package's
sources; the export says when that matters. When the app exits before
its window opens, the error gives its exit code and the end of its
stderr, which may name what it lacks, such as an icon loader or a Java
runtime: close the computer and start a new one with that package in
`extra_packages` (see Choose the install), or stop as for an install
this flow cannot do.
- REA counts a file as the app's document only when the app's window
title shows its file name or `~/` path. When the title shows something
else, such as MuseScore's score title, pass the path you will save the
document at in `state_paths`, relative to the home:
`{"computer_id": ..., "state_paths": ["Documents/MuseScore3/Scores/Morning_Song.mscz"]}`.
It need not exist yet. Add the app's settings folder when its name
differs from the app's, such as `.config/MuseScore`, so the seed keeps
the dismissed first-run dialogs. Do not pass the document as a launch
argument to make REA see it: `state_paths` is the way. gLabels' title
leaves out the file's extension and Grisbi's shows the entity's name,
so both need it.
- An app whose document is a folder of pages, such as a Zim notebook
(`Notebooks/Notes`, a `.txt` file per page), never shows a file in its
title: pass the folder in `state_paths`. Every page in it is then a
document, so the bundle opens none at launch and the app starts where
it keeps its place; check the page the task changes.
3. Make the seed through the app and save it from the app under
`~/Documents`: build the document in the app, or write its source where
the app edits its format as text, then save it again from the app. Make
it look real but synthetic: the thing the task changes, plus a few things
the change must leave alone. Keep it to one document: the bundle's app
opens the seed's document at launch only when there is exactly one, and
otherwise starts on its own screen. For a starting file of the requester's,
read it here and rebuild it in the app with synthetic data in place of
personal data; REA cannot copy a file into the computer.
- An app that keeps its own data and saves after every change, such as
Osmo's contacts in `~/.local/share/osmo/contacts_records.xml`, has no
document: that state is the seed. Build it in the app and mark `seed`
once the app has written. To see whether it has, mark a throwaway
checkpoint, such as `probe-1`, and read its `changed_files`: up to ten
files that changed since the checkpoint before, those a window title
named first, then those under `state_paths` and those the app held
open, caches last, with `more_changed_files` counting the rest.
- The app's own compressed format works when it is gzip, such as
Gnumeric's `.gnumeric`, Dia's `.dia` or Inkscape's `.svgz`: REA reads
the document inside, and checks point into it as into the same
document uncompressed. Compression may be off by default: Dia's
"Compress diagram files" is unticked, which REA reads as plain XML. A format REA reads as binary, such as one
compressed with xz, has no view: save in a format the app offers that
REA reads, then reopen the file once and check that a plain Ctrl+S
keeps that format. When the task is about the app's own binary
format, such as GIMP's `.xcf` or Blender's `.blend`, keep it and check
it through a reader (see Read a binary file with a reader).
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
5. Make the change the task asks for, as a user would, and save it.
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
For more tasks in this environment, make each next task's change on top
of the last as a user would, save it, and mark `end-<task name>`, such as
`end-add-bread`. Each task is checked at its own end, so its checks must
not read what a later task changes, and a solving agent starts each from
the seed, so a task must not need an earlier one done first.
Then, for each task you can, make a near miss of its change on top, a
plausible wrong edit such as a typo or a wrong number, save it and mark
`near-<task name>`: the export refuses a task whose near miss passes every
check. Take the near misses after the last end, since each later draft
starts from the end before it.
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
its net change and its `values`), `draft.task.checks` (suggested
checks), `draft.guards`, `draft.notes` and `limitations`; each
`later_drafts` entry is the same for one later task, from the end
checkpoint before its own, so it shows only that task's change. When
`draft` is null, read `draft_refusal` first: a missing `seed` or `end`
checkpoint means capturing again on a new build computer, while a failed
draft command names its own cause to fix. If a declared secret was used,
automatic drafting is withheld: write the task manually and export it.
When `draft.files` is empty,
`draft.notes` says why:
- `no_change`: nothing was saved between `seed` and `end`. Capture again
on a new build computer, and save before marking `end`. When the app
keeps the task's state only on screen, such as a calculator's result,
judge the screen instead (see Judge the screen).
- `unowned_changes` with `no_document`: the app saved your document
where REA did not count it as the app's. Its `files` list what changed,
with their roles; the document is among them. Close the computer and
capture again on a new one with that path in `state_paths`; the export
refuses a check on a file outside the owned paths.
- `uncheckable_changes`: the app saved the change in dconf, as gedit does
its preferences, and no check reads dconf. Saving again does not help:
choose a change the app saves in a file, and capture again on a new
build computer, or judge the screen (see Judge the screen).
- `unreadable_changes`: a dconf database changed but REA could not read
it; `error` says why. This is not evidence the app saved there: report
the error rather than changing the task.
A note with a `file` says why that file's suggestions are thin:
- `by_position`: only position names what changed; its `elements` lack
a naming attribute. Name a record that keeps its fields as child
elements by one of them, as Osmo's contact by
`[name=contact][>first_name=Delphine]`, a JSON record by a field
inside it, `[>data/Title=…]` (`~1` for the `/` inside a pointer), or
an element by an id its child carries, `[>object@id=…]`; count records
with `member_count`, `matching` `[name=contact]`; otherwise check an
ancestor's text, or give the element an id in the app and capture again.
- `view_state_only`: every change is view state, a renumbered id or a
time the app stamps, and its `pointers` are the checks kept. The edit
was likely not saved: capture again and save it before `end`.
- `long_text`: the element's text is too long to check whole; check a
phrase in it with `text_contains`.
- `cut_view` or `unreadable_view`: REA's view holds only part of the
file, or none a check can read, such as a binary format (step 3). For
a binary file, read it with a reader (see Read a binary file with a
reader).
## Write the task
Write `{"name", "prompt", "checks"}` from the draft and what you did, one
per task, each with its own name.
- `name`: lowercase letters, digits and hyphens, not `smoke`.
- `prompt`: what the requester wants, in their words and the app's terms,
starting with the app's name, naming the document as a user would and
ending with saving it: "In Inkscape, change the poster's headline in
Documents/poster.svg to Summer Bake Sale and save it." Name the app: the
solving agent's own shell cannot see the computer's files, and the app's
name sends it to the screen. Leave saving out for an app that saves on its own, after every change
or a few seconds after the last one, such as Osmo or Gnote: grading waits
until the app's files have held for 3 s, so the solving agent need not
wait for the save. A save more than 3 s after the agent stops is still
missed; name such an app in the report. Never say how the file stores it,
and never give the checks.
- Checks are file-based: content checks that prove the edit, never only that
a file exists or changed. Start from the suggestions; keep those that say
what the user did and drop what follows from how the app saves.
- Compare exact values, so a typo or a wrong number fails.
- Read each file's `values` before writing checks: every value the change
added, changed and removed, with `before` and `after` (an uncut scalar, or
a preview cut at 600 characters) at the pointer a check names it by. Copy
the pointer. Copy `after` into `value` only when it is an uncut scalar: a
string without a trailing `…`, a number, or a boolean. Compact XML and a
JSON array or object are preview strings, so a `value_equals` check needs
the structured value from the file's view, and element text uses
`text_equals` or `text_contains`.
Only XML files, archives, JSON and key files have values: a text file's
or SQLite database's `change` already lists its lines and rows.
`refers_to` says what a number or id stands for, such as `category="44"`
naming `<cat key="44" name="Groceries"/>`, or a new KMyMoney
transaction's splits naming their account and payee; use it for the
prompt's words. `alternatives` gives a changed value's pointer by the
element's other names, such as a label beside an id, to pick another key
without another export.
Skip entries whose `note` is `by_position` (no check can use that
pointer) or noise (`view_state`, `renumbered`, `volatile`,
`timestamp`, `upkeep`, `layout`). `values_cut` means a limit left some out.
- Select elements by what they hold, never by position: SVG text by id,
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
or an ODF paragraph by style,
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
A selector must match exactly one element, so give the target a unique id
or style in the seed. The draft's pointers already do this: a name-like
attribute such as Scribus's `ANNAME` before a coordinate, and two
attributes together where one is not enough, as a spreadsheet cell's
`[@Col=3][@Row=2]`. Only an element's text is read by position, as
`children/0`.
- When an app may split an element's text into runs, check it whole:
`text_equals` (or `text_contains` for a value inside a sentence) on the
element compares all its text, however it is split. Add
`normalize_whitespace: true` when the app may reflow spaces. Example, an
AbiWord paragraph by its stable id:
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
Point a text check at the paragraph or another element the edit keeps,
never at a run such as AbiWord's `[name=c]`: a reopened document splits
the text into several runs. The draft lists only what changed; an
unchanged element is in `draft.guards` when it is a like sibling of a
changed one, and otherwise use the app's usual numbering, such as another
paragraph's `xid`, and let the export confirm it.
- In an ODS spreadsheet no attribute names a row: the draft names it by its
label cell, `[name=table:table-row][>table:table-cell=February]`. Where no
cell names it alone, name the sheet in the seed, and check its whole
text: `text_equals` on
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:spreadsheet]/children/[name=table:table][@table:name=Q1 Sales]`,
or a stretch of it with `text_contains`. Cell texts join with nothing
between, so a row reads `February23100`.
- For a line added to free text, such as an item on a note's list, the
draft compares the whole text with `text_equals`. This rejects longer or
negated items, but can reject a correct edit with different ordering or
spacing. Report that limitation when the task permits such variations.
- For a note placed by its own `x` and `y`, such as Xournal++'s `<text>`
on a layer, the draft checks its text on the layer with `text_contains`,
which passes wherever the user puts it, and guards each note it kept.
- Some apps keep text in an attribute, one element per run, such as
Scribus's `<ITEXT CH=...>`: add `"text_attribute": "CH"` to `text_equals`
or `text_contains` on the frame,
`/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]`,
to read its runs' values joined.
- When the draft shows no values or no checks, a scratch export can expose
`envs/<env>/seed/` only after the task has at least one valid check: it must
pass on the captured end state and fail on the seed. Before the first
successful export, use `act_on_build_computer` to open the saved file in an
available GUI text editor or viewer and inspect its actual representation
without changing it. Use the visible saved content and the task's edit to
write a check; an export refusal can also name the end state's value, such
as `is "44", not "1"`. If the file cannot be inspected or expressed by a
supported check, report the limitation instead of guessing a check.
- Where easy, add one check that the rest is untouched, such as another
text's value the edit must leave alone. `draft.guards` offers them at
their seed values, which the end holds too: a field on the nearest like
siblings, one the user sets, such as Planner's `work`, before a default
they all share; the text an addition kept; a text file's lines beside its
new one. Copy one or two into `checks`; they pass on the seed, which is
fine, since the export needs only one check to fail there.
- The draft suggests a decimal the app writes as text, such as `146.83`,
as a number with `tolerance: 0.001`; keep the tolerance for any number
you check that way.
- When the draft shows an XML document's changes as `lines`, REA read it
as text, for example because its DOCTYPE declares entities. Selectors are
impossible there, and `line_count` with `contains` only approximates a
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
name the check as approximate in the report's limitations.
- Some apps save a document reloaded from disk differently from one typed
in the same session: AbiWord wraps each new insertion in its own markup.
Your build's end state hides this, which is why whole-text checks matter;
expect the witness to show the real format.
The kinds and pointer rules are in
[the desktop tasks section](environments.md#pick-the-checks).
### Read a binary file with a reader
When the draft notes `unreadable_view` on a binary document, add a reader to
the task: the app's own headless command, which REA runs in the computer on a
private copy of the file, and whose output a check reads instead of the file.
Write it in the app's scripting mode, print each fact on a line starting
with `REA ` (with the space), and name the copy `{file}`:
- GIMP `.xcf`, layer names, as `lines`:
```json
{
"id": "layers",
"file": "Documents/poster.xcf",
"argv": ["gimp-console-2.10", "-i", "-d", "-f", "-b", "-"],
"stdin": "(let ((image (car (gimp-file-load RUN-NONINTERACTIVE \"{file}\" \"{file}\")))) (for-each (lambda (layer) (display \"\\nREA layer \") (display (car (gimp-item-get-name layer)))) (vector->list (cadr (gimp-image-get-layers image)))) (newline))\n(gimp-quit 0)\n",
"output": "lines"
}
```
checked with `{"kind": "lines_present", "file": "Documents/poster.xcf", "reader": "layers", "lines": ["layer Summer Sale"]}`.
Script-Fu's prompt `ts> ` starts a line, so print a newline before each
marked line, as `\nREA` does. A text layer's text is
`(car (gimp-text-layer-get-text layer))`.
- Blender `.blend`, objects with their type and location, as `json`:
```json
{
"id": "objects",
"file": "Documents/scene.blend",
"argv": [
"blender",
"-b",
"--factory-startup",
"{file}",
"--python-expr",
"import bpy, json; print('REA ' + json.dumps({o.name: {'type': o.type, 'location': [round(v, 4) for v in o.location]} for o in bpy.data.objects}, sort_keys=True))"
],
"output": "json"
}
```
checked with `{"kind": "value_equals", "file": "Documents/scene.blend", "reader": "objects", "pointer": "/Crate/location/0", "value": 2, "tolerance": 0.001}`.
Print every value a check compares as a number or string a check can
name, rounded where the app stores floats.
Blender 4 shows its Quick Setup splash at every launch until its
preferences are saved: with `.config/blender` in `state_paths`, confirm
Quick Setup with Continue, save the preferences (Edit → Preferences, then
Save Preferences in the menu at its bottom left), and read a probe
checkpoint's `changed_files` for `.config/blender/4.0/config/userpref.blend`
before `seed`, so solves start on the scene. Blender often takes the first click on a
menu or a dialog's button only to focus it: click again.
- A format LibreOffice opens but REA cannot read, such as a legacy `.xls`,
converted with `--convert-to`:
```json
{
"id": "cells",
"file": "Documents/budget.xls",
"argv": [
"sh",
"-c",
"soffice --headless --convert-to csv --outdir out \"$1\" >/dev/null && sed 's/^/REA /' out/*.csv",
"sh",
"{file}"
],
"output": "lines",
"timeout_s": 120
}
```
Pass `{file}` to `sh -c` as an argument, as here, rather than inside the
script.
The reader runs without the display, in a fresh home with the app's default
settings, so use the app's headless or batch mode; set `display: true` only
for an app that has none. It must exit 0 within `timeout_s` (60 s by
default, at most 300) and print at most 1 MiB. Pick facts the change sets and
the near miss gets wrong, such as a name or a number, not the app's whole
output.
Give the task the reader in `readers`, and each check through it `reader`
and the reader's `file`. `json` readers serve `value_equals`, `value_absent`
and `member_count`; `lines` readers `lines_present`, `lines_absent` and
`line_count`. The export runs the reader in the build computer on the seed
and the end: when a check through it fails on the end, the refusal shows
what the reader printed, and a reader that fails shows its error and the end
of its stderr. Fix the reader or the check from that and export again.
### Judge the screen
Only when no file shows the change, as when the draft notes `no_change`
because the app keeps its state on screen, write the task's checks as
screen judges: `{"id": "shown", "kind": "screen_judge", "question": "Does the
calculator's display show 8.568?"}`. A model, the judge, answers the question
on a screenshot three times, and the check passes when most answers are yes.
- REA keeps a screenshot at each checkpoint. Mark `seed` with the screen
showing the state before the task, where the answer is no, and `end` with
the change showing, where it is yes: the bundle ships both as the judge's
examples.
- Ask about what the change makes visible, with exact values, so the near
miss is a no: a result, a label, a selected item. Avoid what a window
moved or a cursor blinking can change.
- The prompt asks to leave the result showing, since the judge sees only
the screen when the agent stops.
- A task with screen judges holds only screen judges and no readers. The
export refuses one when a file shows the change, and checks none of its
questions: the witness does, running the bundle's own judge with the
runner's model endpoint (`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`[model]` in `.agentenv/config.toml`) on a fresh screenshot at each stage.
Without an endpoint the witness's check fails: say what to set and stop.
- The judge's model is `[model.roles] judge`, else `[model] default`, else
`anthropic/claude-sonnet-5-5`. `agent-env run --judge-model` does not
reach it. Each grade costs about $0.02.
## Export
`export_build_environment` with `tasks` and `directory`, the absolute path of
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. Each
entry of `tasks` is `{"task": ..., "end_checkpoint": ...}`: leave
`end_checkpoint` out for the task checked at `end`, and give
`end-<task name>` for each later one. The
export seeds what was saved at `seed`, replaces the computer's host name,
`user@host` and home path (a bare user name, such as AbiWord's author
`user`, stays), and checks each task against `seed`, its own end and its
`near-<task name>` when you took one. It refuses a task
unless at least one check fails on the seed, every check passes at its
end and one fails at its near miss, and says which task and why, such as a
pointer that names nothing at the end. A
check that the rest is untouched passes on the seed too, which is fine. On
a refusal, fix the task and export again, into another folder if that one
is no longer empty. The bundle's eval lists every task, so a run of the
bundle runs them all. A refusal naming a secret's handle means a file of the
bundle holds that secret, most often an app's sign-in or licence state in
the seed: do not work around it, but report it and stop. Keep
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
report.
The limitations name the folders the app wrote that the bundle neither owns
nor seeds, such as `~/.config/MuseScore`, so the app starts without them:
expect its first-run wizard, a tour or a "restore session?" question at every
boot. If the witness shows one, capture again on a new build computer with
those folders in `state_paths`, and dismiss the dialogs before `seed`. Lock
files the app holds while it runs, such as LibreOffice's `.lock` and
`.~lock.<name>#`, GnuCash's `<book>.LCK` or Grisbi's `.<file>.lock`, are
left out of the seed and listed after "Left out of the seed". An app can
still start fine yet refuse every save when a lock was seeded, so close the
document before `seed` where the app lets you (apps other than LibreOffice,
such as Rosegarden, also write `.~lock.<name>#` and autosaves beside a
document, and closing it covers them too), and treat "file is locked"
on a save in the witness as a seeded lock: delete it from the bundle's seed
and witness again in a new computer. In LibreOffice, own the profile
(`state_paths: [".config/libreoffice"]`) for a clean boot, and close the
document before `seed`, then reopen it from Recent Documents to make the
change: the export also leaves out the recovery list LibreOffice writes while
a document is open, which would start the bundle into document recovery. A
limitation naming `~/.config/dconf/user` as unread means the app's dconf
settings, such as a first-run tip you turned off, are not seeded: capture
again on a new build computer. The limitations also name seeded WebKit,
Chromium or QtWebEngine storage and caches; when the witness's boot is clean,
list them in the report.
Keep the build computer open until the witness holds: a task fix exports
again from it.
## Witness
Witness each task in its own fresh computer of the bundle, from the prompt,
as a solving agent would; a computer is witnessed once, so the steps below
repeat for each task, each with its own record folder. Do not replay the build's actions or coordinates, and do not
check between stages to steer the edit: a solving agent never sees the
checks. Give every `check_build_task` the task exactly as exported.
1. `start_build_computer` with
`{"source": {"kind": "bundle", "path": "<bundle>"}}`, adding
`"settings": {"<variable>": "<handle>"}` for each of the export's
`withheld_files`, or the deploy is refused for want of them, and agentenv
keeps that bundle's build failed, so start from a fresh copy of it. The app starts
with the seed's files in place and opens the seed's document when the
seed holds exactly one and the app takes a file. When the export's
`limitations` say it opens no document, open it from the prompt's path,
as a solving agent would. Take a `wait` screenshot first, since the
desktop may not have drawn yet. When the start fails because the
computer never got ready, and its error names lock files in the bundle's
seed, delete them from the bundle and start again.
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
path of a new or empty folder for the record. A check must fail.
Recording starts. Later stages take no `directory`.
3. Do the task from the prompt with `act_on_build_computer` and save.
`check_build_task` with `stage: "after_edit"`: every check must pass.
Each check waits until the app's files have held for 3 s, as the
bundle's verifier does, so an app that saves on its own needs no wait of
yours. A result with `settled: false` means the files were still
changing at the 15 s bound, and `read_cut: true` that a slow read was
still running there.
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
a check must fail.
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
with `stage: "after_reset"`: a check must fail again.
6. The last stage ends the witness: `witness.record` has the verdict, the
path of `witness.json` and the recording's segments in `directory`.
`close_build_computer` on the witness computer.
A stage that does not hold ends the witness as `failed`. Read each check's
result and decide:
- `after_edit` failed though the saved file is right: the check is brittle.
Fix it, export again and witness in a new computer.
- `after_edit` failed because your edit was wrong: witness again in a new
computer; a computer is witnessed once.
- `near_miss` passed: the checks are too loose. Tighten them, export again
and witness again.
- `after_reset` passed: the reset did not restore the seed. Report it as a
limitation.
Stop after three witnesses of a task and report what still fails.
## Measure
Only if the requester asked for it, or by the headless default above. If
they wanted to review the tasks first, show them each prompt and checks and
wait for their approval. Then `measure_build_task` each task with `bundle`,
`task`, `model` and `budget_usd`, its share of the budget. It runs the task
in fresh computers, 3 times, then 2 more unless they passed none or all, and
keeps a task that passes 1 to 3 of 5 (`band: "in_band"`). It stops before a
solve the largest cost so far would take past the budget, and writes the
record into the bundle's `difficulty.json` and README. The solver needs a
model endpoint where REA starts: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`; when the
measure says a solve did not run for want of one, tell the person what to
set and stop. Read the result's `failures_per_check` and each solve's
`failed_checks` before calling a task hard: a check that fails in most failed
solves while the app shows the right change is brittle, so fix the check,
export and witness again, rather than count it as difficulty.
## Harder tasks
When the requester wants tasks worth evaluating or training on, harden a
task that came out `too_easy` and measure it again, within the budget. The
solver sees and fixes its slips on screen, so one edit with the value in the
prompt is solved almost every time; so far, on TreeLine with Sonnet 5.5, so
were one more dialog, a conditional move of seven records, and ten
corrections read from a second document and applied across the list. Try,
one at a time, measuring each:
- A longer chain of exact edits than that, each a value the checks compare
whole, so one slip in the chain fails the task.
- Facts read in one view and used in another: values the solver must read
from a second document the app opens, or from another record, rather than
from the prompt, such as "the same year as The Quiet Glacier's".
- Distractors beside each target: records with near-identical names or
values, and a guard check on each that it stays as it was.
- Mistakes that do not show once saved: a value the app displays rounded or
formatted unlike what it stores, or an edit the app keeps only when its
field loses focus.
Stop when the budget left cannot cover 3 solves of the hardened task, and
report the bands measured.
Each hardened task stays sound: its checks still fail on the seed and pass on
its end, its near miss fails, and another valid way of doing it passes, so
check values, never the order or method the solver used. Put the second
document in the seed. Count records with `member_count` and check each
value by a selector that names its record, since no check joins two records.
## Close and report
`close_build_computer` on every computer still open, also after a failure.
If a close fails, call it again; if it still fails, tell the requester what
is left.
Report:
- the bundle path and the environment name;
- each task's name, prompt and end checkpoint;
- each check, and why it proves the edit and fails the near miss, and each
reader's command;
- each task's witness stages with their results, the verdict, and the paths of
`witness.json` and the recording;
- the seed's contents: `seed_files` and what each holds;
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
the host or user, so it needs review before sharing. `recompressed_files`
are gzip seeds whose headers were cleared, their content unchanged: give
their count;
- limitations: the capture's, the export's and anything you could not do;
- each measured task's band, passes of finished solves, turns, cost and
failures per check, from `difficulty.json`;
- the build time and the questions you asked.
Never publish or share the bundle yourself.
@@ -294,7 +294,9 @@ Use this when the user has a folder from `rea capture-desktop-application` and
wants a task for the environment `rea export-desktop-application` makes from
it: a prompt for a computer-use agent and checks that grade what the agent
left in the app's files. You write the prompt and the checks; REA validates
the task against the capture and exports it. These commands are CLI only.
the task against the capture and exports it. These commands are CLI only. To
build the environment and its task from a request instead, follow
[desktop builds](desktop-build.md).
### Describe the change
@@ -306,14 +308,27 @@ Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
and last version, with its `role`, the `baseline` and `final` versions and
the net `change`. Text changes list each added and removed line once, and
`line_counts` gives lines found at both ends whose count changed.
- `values` in each of those files: what the change added, changed and
removed, `before` and `after`, at the pointer a check names it by, with
`refers_to` naming the element a key such as `category="44"` stands for.
Skip entries noted `by_position` or as noise. Only XML files, archives,
JSON and key files have values: a text file's or database's change
already lists its lines and rows.
- `task`: a draft whose `checks` are suggestions from the net change of the
documents the user worked on, or of the app's own state when no document
changed.
- `guards`: optional checks that the rest kept its seed values, such as a
like sibling's field or a text file's neighbouring lines; add one or two
to show the rest is untouched.
- `notes`: why `files` is empty, or why a file's suggestions are thin, such
as `by_position` when no attribute names the changed element.
### Write the prompt
Say what the user wants done, in their words and the app's terms: "Add buy
milk to my to-do list and save it". Never say how the file stores it: no
Say what the user wants done, in their words and the app's terms, naming the
app: "In Mousepad, add buy milk to my to-do list and save it". A solving
agent's own shell cannot see the computer's files; the app's name sends it to
the screen. Never say how the file stores it: no
paths the user would not name, line formats, XML, keys or SQL, and never the
checks. Replace the draft's placeholder; the export refuses it.
@@ -344,13 +359,18 @@ document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
entries: point into an entry's view, and name array members by selector
rather than position, as
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
so the check survives an app version that writes them in another order. Give
so the check survives an app version that writes them in another order. A
gzip-compressed document, such as Gnumeric's `.gnumeric`, reads as the
document inside: point into it as into the same document uncompressed. Give
`value_equals` a `tolerance` to compare a number the app writes as text, such
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
document with `text_equals` on its paragraph element, not `value_equals` on a
text node: apps such as AbiWord and Writer split a paragraph into runs as they
save, so its first text node holds only part of it; use `text_contains` for a
value within a sentence, such as a time. The
value within a sentence, such as a time. A binary document, such as GIMP's
`.xcf`, has no view: give the task a `readers` entry, the app's own headless
command printing `REA ` lines, and a check naming it in `reader`, as
[desktop builds](desktop-build.md#read-a-binary-file-with-a-reader) shows. The
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
guide has the exact rules.
@@ -361,7 +381,8 @@ Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
the task only when its checks fail on the seed and all pass on the captured
end state. A refusal names the check: one that fails on the end state does not
match what the user left; "every check already passes on the seed" means no
check captures the change.
check captures the change. For several tasks from one capture, repeat
`--task` and name each later task's end with `--task-end NAME=CHECKPOINT`.
When the export refuses an application that no apt repository installs, ask
the user for the `.deb` file they installed it from and capture with
@@ -372,5 +393,7 @@ user's to choose.
Tell the user the task's name, prompt and checks, and how to measure a solve
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
`--model MODEL` three times and count the runs reported `passed`. Do not run
`--model MODEL` three times and count the runs reported `passed`. The runs
need a model endpoint: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`. Do not run
agents unless asked. The bundle's seed is the user's real data: keep it local.
@@ -47,7 +47,7 @@ member comparison CLI still takes two assembly paths; process comparison takes
two capture Evidence JSON paths. Check command help for the installed version.
An input bundle grants no implicit file lookup: `evidence-file` and historical
graph file selectors are accepted only by contracts advertising them. See the
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/cli.md#import-export-and-compare-evidence).
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
Completed `inspect_artifact` inventory also supports a summary and
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
`additional_paths` when advertised, then analyze the returned `analysis_input`
directory together. Each input retains its own snapshot, digest and byte-range
provenance; inputs and recovered code are never executed. Recovery requires a
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
extracted tree. When the question concerns known source paths and the connected
@@ -67,11 +67,18 @@ See [desktop installations](https://github.com/morluto/rea/blob/main/docs/deskto
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
files need `architecture`) when the
question concerns the embedded compiler version, module/dependency versions,
local replacements or build settings. No `open_binary`, Go installation or
local replacements, build settings, or static Go string literals packed without
NUL terminators. Pass `include_string_literals: true`
(`--include-string-literals`, optional `max_string_literals` /
`--max-string-literals`) to recover bounded inferred word-aligned `(ptr, len)`
UTF-8 string slices from read-only data and page them through `go`
`string_literals` views. No `open_binary`, Go installation or
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
Preserve artifact SHA-256, original file ranges, framed module bytes and every
unparsed module-text line. A missing record is an unknown, not proof of a
non-Go binary; observed metadata does not authenticate its claims. Function,
non-Go binary; observed metadata does not authenticate its claims, and static
`(ptr, len)` string recovery excludes compiler-inlined stack/immediate strings
without proving runtime reachability. Function,
source and type recovery remain a separate native-analysis question.
## Model files
@@ -142,6 +149,20 @@ table or overlay. Bound snapshots, loaded IAT contents, and export-forward
resolution remain unobserved. See
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
## Windows Shell Link shortcuts
Use `inspect_windows_shortcut` for an explicit local `.lnk` file when the
question concerns the recorded target path, arguments, working directory, icon
location, volume serial, network share names, ExtraData blocks, or Tracker
droid identifiers. No `open_binary`, Windows SDK, COM, or target execution is
required. The CLI equivalent is `inspect-windows-shortcut PATH`. Preserve
artifact SHA-256, FILETIME raw decimals, IDList type bytes, and ExtraData
signatures. Unknown IDList items and ExtraData signatures remain raw with
SHA-256; they are not guessed. Version-1 UUID MAC and timestamp values are
labelled derived. A missing optional section is observed absence of that
structure. See
[Windows Shell Link shortcuts](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
## PE TLS callbacks and load configuration
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
@@ -178,6 +199,36 @@ absence of those declarations, not proof the image never carried notes in
another container. See
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
## ELF and Mach-O initializer functions
Use `inspect_native_initializers` for an explicit local ELF or Mach-O image
when the question is which functions run before `main` or at exit: ELF
`DT_PREINIT_ARRAY`, `DT_INIT`, `DT_INIT_ARRAY`, `DT_FINI`, `DT_FINI_ARRAY` and
legacy `.ctors`/`.dtors`, or Mach-O `__mod_init_func`, `__init_offsets` and
`__mod_term_func`. Pass `architecture` for a universal Mach-O. The CLI
equivalent is `inspect-native-initializers PATH`. PIE slots are resolved
through RELATIVE, RELR and section relocations, and Mach-O slots through
chained fixups. Keep each entry's `resolution`: `unresolved` means the value was
not established, and `unresolved_import` names an external symbol. Page with
`offset` and `limit`. See
[ELF and Mach-O initializer functions](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
## ELF unwind function ranges
Use `inspect_elf_unwind_ranges` for an explicit local ELF32/ELF64 image when
the question concerns function starts and lengths that survive `strip`, which
functions have an LSDA, or which ranges have no exact symbol. No
`open_binary` or target execution is required. The CLI equivalent is
`inspect-elf-unwind-ranges PATH`. Preserve artifact SHA-256, FDE start and
end, containing section, exact symbol name, CIE index, LSDA, and personality.
Page later ranges with `offset` and `limit`. A missing `.eh_frame` is
observed absence of that declaration and of `PT_GNU_EH_FRAME`, not proof the
image has no functions. Header disagreement is a limitation and does not drop
decoded ranges. `.debug_frame` is reported present or absent and is not
decoded. CFA instructions are counted and not evaluated. Indirect EH pointers
are not dereferenced. See
[ELF unwind function ranges](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
## Apple IPS crash reports
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
@@ -209,6 +260,27 @@ container. gzip, xz, and zstd whole-file containers are unsupported and name
decompress into a separate file, then call `inspect_kernel_module` on that file. See
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
## Linux kernel build config
Use `inspect_kernel_build_config` for an explicit local kernel image (`vmlinux`
ELF, raw arm64 `Image`, x86 `bzImage`/`zImage`, or `configs.ko`) when the
question is which kernel version it is or how it was configured. No
`open_binary`, Linux host, or `extract-ikconfig` is required. The CLI
equivalent is `inspect-kernel-build-config PATH`; pass `name` for one exact
`CONFIG_*` option and `offset`/`limit` to page the rest. Preserve the banner
version, builder, and compiler, the config SHA-256 and option count, and each
entry's value kind (`y`, `m`, `string`, `number`, `not_set`). A config is
accepted only when `IKCFG_ST` is immediately followed by a valid gzip member
and `IKCFG_ED`, so an ordinary `IKCFG_ST` string is not a config. Offsets with
`space: "inflated_payload"` are positions in an inflated bzImage payload, not
file offsets. An absent config describes only the searched bytes; the kernel
may have been built without `CONFIG_IKCONFIG` or with it as a module. Truncated
payloads are incomplete, not absent. A framed blob that does not decode, has a
bad CRC, or holds non-config text is `malformed`, not absent. xz, lz4, lzma, zstd, bzip2, and lzo
payloads are unsupported and name `external_decompression`: decompress with an
external utility, then call `inspect_kernel_build_config` on the result. See
[Linux kernel build config](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
## CPython pyc files
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
@@ -246,7 +318,7 @@ Choose `row_limit` when a finite sample is useful and keep truncation and
unspecified row order explicit. Virtual tables, shadow tables and views are
schema facts, not row-read targets. REA inspects private copies and leaves
the source files untouched; obtain a coherent quiescent database/WAL snapshot
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
## Recorded Linux crashes
@@ -261,7 +333,7 @@ The optional `include_debugger_context` facet adds mapping candidates through
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
establish current file identity; zero reported flags leave permissions unknown.
Requested unavailable context fails with setup guidance. See
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
for exact upstream profiles, bounds and verification coverage.
## Offline ELF layout
@@ -325,7 +397,8 @@ on the first MCP call. It retains the complete observation and returns the
original selection, text availability and length without the body. Use
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
follow `coverage.next_offset` for later pages. Offsets and limits count UTF-16
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
page exceeds the client or transport budget. Offsets and limits count UTF-16
code units, not lines or tokens. Pages reuse the same observation without
repeating decompilation. Null and empty text remain distinct. Omitted or
`complete` detail preserves full inline delivery. Summary delivery requires a
@@ -393,7 +466,7 @@ In ephemeral mode, choose a private absolute directory and set
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
after changing that launch configuration; no current tool call or runtime
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/installation.md#automatically-checkpoint-an-owned-ghidra-program).
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
With the setting enabled, normal opens restore compatible saved Programs
automatically. Initial completed analysis is saved, and `close_binary` saves
@@ -414,7 +487,7 @@ import overrides and repair scripts are unavailable in this mode. Treat original
analysis completeness/settings as unknown and prior edits as caller-project state.
The program's recorded import digest must match the target; changed project content
requires reopening for a fresh profile. See the
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/installation.md#reuse-an-existing-ghidra-project).
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
## Ghidra startup deadline recovery
@@ -448,7 +521,7 @@ and memory controls in place. Disable specific expensive analyzers only when
their missing coverage is acceptable and retain the resulting limitations.
Exported C and embedded resources remain useful independent evidence, but a
timeout does not establish complete analysis or absence of code. See the
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
## Bridge request ceiling
@@ -599,7 +672,7 @@ broader metadata recovery on Linux; it requires an explicitly configured
NativeAOT JAR. These are separate routes. The built-in parser also runs on
Windows Ghidra P0; read frozen literals from its report, because
`search_strings` does not include them. See the
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
for setup, supported layouts, and host restrictions. Preserve binary identity
and distinguish derived metadata from Ghidra database observations. Recovered
types and native pseudocode are not original C# source. Windows Ghidra P0 has
@@ -628,7 +701,7 @@ every executable in the bundle. It also lists missing or weak loads and earlier
provides them.
Use `extract_artifact` when materialized files from a supported format are needed.
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
It materializes all regular files into a fresh temporary directory chosen by
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
The result reports the materialized directory; callers do not choose the
@@ -656,7 +729,7 @@ temporary, not a retained export of the image's contents.
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
`--target-format dos-com`). Inspect the load image before trusting function
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
@@ -670,7 +743,7 @@ otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
superset of the 68000 family; Amiga/Atari containers and vector-table entry
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
@@ -684,6 +757,11 @@ numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
Fresh-import sessions admit annotations; existing-project reuse is read-only.
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
A provider switch or different target can retire the active Ghidra database and discard
its edits, even if the new provider fails on a later call. Same-target,
same-profile reopen keeps the active database. Retained annotation Evidence
records what happened; it does not restore the edits.
For `compare_functions`, obtain `analyze_function` Evidence before and after the
edit while the MCP session is open; annotation Evidence is not a comparison
input. Export the needed Evidence before close if comparison will happen later.
@@ -704,7 +782,7 @@ code pages, payload hashes and static icon-group references. Same-language icon
resolution is explicit; missing references and size mismatch remain visible.
Other payloads are opaque, and this operation does not decode images or simulate
Windows resource loading. Consult the budgets and supported tree profile in
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
## Offline PE and ELF export tables
@@ -714,7 +792,7 @@ directory or ELF dynamic symbol table on any host without opening a native
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
partial or unsupported coverage means the inventory is incomplete, not that
exports are absent. Use `inspect_macho` for Mach-O. See
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
For exact core-module import/export differences, use `compare_wasm_interfaces`
with two complete inspection Evidence records (or current-session retained
@@ -830,3 +908,24 @@ Layout labels describe record shapes, not a detected compiler version; unknown
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
executable is never run. See
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
## Deno compile executables
A `deno compile` executable stores its application in a Mach-O `__SUI,d3n0l4nd`
section, an Intel Mach-O `<~sui-data~>` trailer, an ELF `SUI` note of type
`0x53554901`, or a PE `RT_RCDATA` resource named `D3N0L4ND`.
`open_binary` then `inspect_artifact` reports root format `deno-standalone`
with the carrier, entry point, recorded options, and one occurrence per virtual
file, including size and SHA-256. Transpiled output, CJS export analysis, and
source maps are separate occurrences with the file's `index` and their own
`role`. They normally use a `.deno-*` suffix on the file path and move to an
indexed path on a collision, so use the reported `logical_path`. Symlink targets
stay recorded path parts. A virtual filesystem that cannot be listed leaves
coverage unknown.
Environment-file values and CA certificate bytes are retained in local Evidence
with their counts and byte lengths. The npm snapshot length is reported and not
decoded. `extract-artifact` writes
stored files through the safe output tree; route extracted JavaScript to
`analyze_javascript_application`. Other layouts are unsupported, and the
executable is never run. See
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
@@ -41,10 +41,16 @@ behavior unknown. Record truncation, unmatched breakpoints, changed target
identity and cleanup uncertainty as limitations, rather than treating no retained
event as proof that the behavior never occurs.
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/frida-instrumentation.md),
For same-session MCP comparisons, pass each capture inline or reuse it as
`{"kind":"retained-evidence","evidence_id":"ev_..."}`. Default comparison
keeps capture-local timing as provenance without treating scheduler or observer
latency as a behavior change; use an exact `trace_spec` event when timing is part
of the analyst question.
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
[process capture](https://github.com/morluto/rea/blob/main/docs/process-capture.md),
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/native-investigation.md#native-desktop-observation).
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
## Browser and Electron
@@ -130,5 +136,5 @@ Inspect native `.saz` snapshots with `inspect_web_network_capture` and
traffic analysis, connect Everywhere's separate MCP server and use its
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
`AgentCalls`). Preserve existing sessions and filters when inspecting a
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
for the workflow and Telerik's public setup and traffic-debugging skills.
+1 -1
View File
@@ -27,7 +27,7 @@ body:
attributes:
label: Documentation location
description: Link the page, section, command, API, skill, or inline comment.
placeholder: README.md#quick-start or docs/mcp-contracts.md
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
validations:
required: true
+3 -3
View File
@@ -71,8 +71,8 @@ source, link, or rendered-document checks. -->
- User-visible CLI/MCP output (if applicable):
- Remaining proof gaps:
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#boundary-regressions)
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#lifecycle-regressions)
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#boundary-regressions)
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#lifecycle-regressions)
covered by the evidence above, and explain relevant gaps.
For evidence-bearing changes:
@@ -99,7 +99,7 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
its new behavior and migration, and why compatibility/deprecation cannot
preserve it. Every release increments minor, including breaking changes;
keep breaking markers and migration notes so callers can assess compatibility.
See docs/releasing.md. -->
See docs/development/releasing.md. -->
- Real Hopper/Ghidra, browser, or OS coverage:
- Package or release metadata impact:
- Security, privacy, process, or containment review:
+5 -4
View File
@@ -544,7 +544,7 @@ jobs:
--coverage.reporter=text
--coverage.reporter=json-summary
--coverage.reporter=lcov
--reporter=default
--reporter=dot
--reporter=junit
--reporter=json
--reporter=./scripts/ci/test-timing-reporter.mjs
@@ -559,7 +559,7 @@ jobs:
needs.test-source.result == 'success' || needs.test-source.result == 'failure')
run: >-
npm exec -- vitest --merge-reports
--reporter=default
--reporter=dot
--reporter=junit
--reporter=json
--reporter=./scripts/ci/test-timing-reporter.mjs
@@ -924,6 +924,7 @@ jobs:
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/filesystem/typescriptCompiler.test.ts
tests/boundary/filesystem/releaseProposal.test.ts
tests/acceptance/byteEntropyWindowsIdentity.test.ts
tests/boundary/filesystem/binaryTarget.test.ts
src/contracts/sessionPathInputs.test.ts
tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts
@@ -1016,8 +1017,8 @@ jobs:
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run docs:check
- name: Render documentation and validate page links
run: npm run docs:build
- name: Verify published routes, source links and local anchors
run: npm run verify:docs:routes
- name: Generation leaves authored files unchanged
run: git diff --exit-code
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+42 -3
View File
@@ -25,7 +25,7 @@ jobs:
node-version-file: .nvmrc
- name: Validate the SDK pin against the provider's verified release
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
# The bridge probe imports only source-owned ARM/Thumb bytes and never
# The bridge probe imports only source-owned ARM, Thumb and MIPS bytes and never
# executes target code. All SDK/project state stays in runner storage.
- name: Acquire checksum-verified Ghidra SDK in isolated runner storage
shell: bash
@@ -46,7 +46,7 @@ jobs:
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
test -n "${!java_home_variable}"
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
- name: Inspect real ARM and Thumb instruction context through the bridge
- name: Inspect real ARM, Thumb and MIPS instruction context through the bridge
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
@@ -58,10 +58,21 @@ jobs:
-import "${probe_root}/arm-thumb.bin" -loader BinaryLoader -processor ARM:LE:32:v7 \
-noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaArmContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_ARM_CONTEXT_OK arm=0 thumb=1 unavailable=null (injected provider value)' "${probe_root}/output.log"
mkdir -p "${probe_root}/mips-project"
# MIPS32 addiu v0,a0,3; MIPS16 li v0,90. These bytes are never executed.
printf '\003\000\202\044\132\152\000\000' > "${probe_root}/mips.bin"
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/mips-project" MipsContext \
-import "${probe_root}/mips.bin" -loader BinaryLoader -processor MIPS:LE:32:default \
-noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaInstructionContextProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" mips \
> "${probe_root}/mips-output.log" 2>&1 || { cat "${probe_root}/mips-output.log"; exit 1; }
cat "${probe_root}/mips-output.log"
grep -F 'REA_MIPS_CONTEXT_OK standard=0 mips16=1' "${probe_root}/mips-output.log"
# Each probe imports source-owned bytes at 0x1000, away from the Cortex-M
# vector symbols that ARMCortex.pspec defines at 0, and decodes them from
# the language's default context, so M-profile Thumb state must hold at
@@ -95,3 +106,31 @@ jobs:
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
# A call through a writable slot keeps Ghidra's recorded target as an
# observed callee while callee coverage stays partial; a direct call is
# the complete control. The source-owned x86-64 bytes are never executed.
- name: Keep computed-call targets non-exhaustive in callee coverage
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-computed-call"
mkdir -p "${probe_root}/project"
{
printf '\270\001\000\000\000\303\314\314' # 0x1000: mov eax, 1; ret
printf '\270\002\000\000\000\303\314\314' # 0x1008: mov eax, 2; ret
printf '\377\025\052\000\000\000\303\314' # 0x1010: call [rip + 0x2a]; ret
printf '\377\320\303\314\314\314\314\314' # 0x1018: call rax; ret
printf '\350\333\377\377\377\303\314\314' # 0x1020: call 0x1000; ret
printf '\314%.0s' $(seq 24) # 0x1028: int3 padding
printf '\000\020\000\000\000\000\000\000' # 0x1040: slot = 0x1000
} > "${probe_root}/computed-call.bin"
test "$(wc -c < "${probe_root}/computed-call.bin")" -eq 72
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ComputedCall \
-import "${probe_root}/computed-call.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor x86:LE:64:default -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaComputedCallCoverageProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_COMPUTED_CALL_COVERAGE_OK slot=observed-complete-discovery-unknown register=partial direct=complete callers=discovery-unknown' "${probe_root}/output.log"
+1 -1
View File
@@ -27,4 +27,4 @@ jobs:
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run docs:build
- run: npm run verify:docs:routes
+12 -6
View File
@@ -14,13 +14,13 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
## Architecture and Cleanup
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
For lifecycle changes, follow [ownership and completion invariants](docs/development/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/development/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
@@ -68,6 +68,12 @@ do not establish fewer tokens or faster analysis.
## Development
Documentation ownership and authoring conventions live in
[docs/development/documentation.md](docs/development/documentation.md). A feature
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
Consult the [documentation directory](docs/README.md) before adding a guide.
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
- `npm ci`: install locked dependencies.
@@ -80,15 +86,15 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
When changing installers or setup recovery, follow
[Checking installers and publisher downloads](docs/testing.md#checking-installers-and-publisher-downloads).
[Checking installers and publisher downloads](docs/development/testing.md#checking-installers-and-publisher-downloads).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/development/testing.md#developer-commands).
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
## Tool and Test Changes
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
contract, and actionable result or recovery at the point of use. Document the
@@ -96,7 +102,7 @@ result in product docs when explanation is needed. Add skill guidance only for
workflow judgment or context that the tool cannot reasonably convey; do not use
a skill as the primary repair for a discoverability or contract defect.
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Select applicable [boundary regressions](docs/development/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
+7 -7
View File
@@ -132,8 +132,8 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
- **Kuna analysis:** Use a caller-installed Kuna for procedure listings, pseudocode, assembly, call relationships, exact-address xrefs and cached string searches. Project exports retain source spans and digests; repeated requests reuse completed work. Recorded real-provider coverage uses Kuna 1.833 on Linux; Windows, PE and universal Mach-O are unsupported. See [the provider guide](docs/kuna-provider.md). ([#3832](https://github.com/morluto/rea/pull/3832), [#3854](https://github.com/morluto/rea/pull/3854), [#4071](https://github.com/morluto/rea/pull/4071), [#4095](https://github.com/morluto/rea/pull/4095)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Retained JavaScript facts:** Query exact application paths, selected properties and filtered module references through `inspect_analysis_view`, with paging and complete row Evidence. These views reuse completed analysis and preserve source selection and unknown coverage. ([#4265](https://github.com/morluto/rea/pull/4265), [morluto/rea#4190](https://github.com/morluto/rea/issues/4190), [#4097](https://github.com/morluto/rea/pull/4097), [morluto/rea#4092](https://github.com/morluto/rea/issues/4092)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **APK locale tables:** Request several resource locales in one decode with repeatable CLI `--locales` or MCP `locales`. Each table reports its status and coverage; existing single-locale fields remain available. ([#4804](https://github.com/morluto/rea/pull/4804), [morluto/rea#4777](https://github.com/morluto/rea/issues/4777)). Thanks @abhay-2108, @N0zoM1z0.
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/providers/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/providers/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
- **JADX deadlines:** Set `REA_JADX_OPERATION_TIMEOUT_MS` to extend APK loading and upstream operation deadlines together. Doctor reports the effective value and setup forwards an explicit selection. ([#4466](https://github.com/morluto/rea/pull/4466), [morluto/rea#3918](https://github.com/morluto/rea/issues/3918)). Thanks @jaychou813, @SandstromPL.
- **Electron window options:** Resolve stable local constant `BrowserWindow` options while retaining source provenance. Relevant writes and escapes keep the result unknown. ([#4400](https://github.com/morluto/rea/pull/4400), [morluto/rea#4366](https://github.com/morluto/rea/issues/4366)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Language editions:** Add the Greek website edition and Malayalam documentation, with Korean typography and translation corrections across the site. ([#3520](https://github.com/morluto/rea/pull/3520), [#4792](https://github.com/morluto/rea/pull/4792), [#3821](https://github.com/morluto/rea/pull/3821), [morluto/rea#3820](https://github.com/morluto/rea/issues/3820), [#3799](https://github.com/morluto/rea/pull/3799), [morluto/rea#3798](https://github.com/morluto/rea/issues/3798), [#4344](https://github.com/morluto/rea/pull/4344), [#3856](https://github.com/morluto/rea/pull/3856), [#4166](https://github.com/morluto/rea/pull/4166)). Thanks @gensokyo-dev, @hi2242, @kalayciburak, @kaoru0822-kitauji, @kraken-503, @kukinghan, @N0zoM1z0, @yeseong0412.
@@ -192,7 +192,7 @@ Full contribution record: [571 commits and 236 merged PRs](https://github.com/mo
## [6.5.0](https://github.com/morluto/rea/compare/rea-agents-6.4.0...rea-agents-6.5.0) (2026-10-13)
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/testing.md).
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/development/testing.md).
### Highlights
@@ -220,7 +220,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **HarmonyOS packages:** inventory HAP/HSP/App Pack structures without executing code. The recorded real workflow covers one pinned HAP through repository-built CLI/MCP; installed-package and real HSP/App Pack coverage remain unverified. ([#1447](https://github.com/morluto/rea/pull/1447)). Thanks @BaaRai.
- **Ghidra raw inputs and references:** compatible Linux/macOS Ghidra profiles accept caller-selected raw architecture/entry information and expose typed incoming references at exact addresses. Windows Ghidra P0 admits the raw NMOS 6502 profile described below; other raw targets and the incoming-reference facet remain outside Windows P0. ([#1868](https://github.com/morluto/rea/pull/1868), [issue morluto/rea#1836](https://github.com/morluto/rea/issues/1836), [#1715](https://github.com/morluto/rea/pull/1715), [issue morluto/rea#1245](https://github.com/morluto/rea/issues/1245)). Thanks @bugstan, @casesolved-co-uk, @mvanhorn, @N0zoM1z0.
- **Ghidra MIPS:** Add bounded MIPS32r2/o32 relocatable objects, soft-float integer-register storage, admitted MIPS16 shared ELF, and a narrowly admitted recordless legacy MIPS32 executable profile. The latter pins an endian-matched 32-bit language and records inferred o32 as an analysis assumption. Admission follows each documented ABI/flag/relocation boundary. ([#1949](https://github.com/morluto/rea/pull/1949), [issue morluto/rea#1937](https://github.com/morluto/rea/issues/1937), [#1950](https://github.com/morluto/rea/pull/1950), [issue morluto/rea#1938](https://github.com/morluto/rea/issues/1938), [#1991](https://github.com/morluto/rea/pull/1991), [issue morluto/rea#1851](https://github.com/morluto/rea/issues/1851), [#1898](https://github.com/morluto/rea/pull/1898), [issue morluto/rea#1620](https://github.com/morluto/rea/issues/1620)). Thanks @AurelioB, @BaaRai, @bqw371, @kaoru0822-kitauji, @N0zoM1z0.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/providers/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **PSP static ELF profile:** the Ghidra profile is Linux x64, bounded little-endian ELF32 `ET_EXEC` with the documented PSP/ABI declarations and a compatible caller-installed Allegrex extension. PRX, shared/relocatable images, PBP/ISO/CSO and runtime emulation are outside scope. Historical extension experiments are separate from public-provider verification; real public Ghidra PSP coverage remains unverified. ([#1599](https://github.com/morluto/rea/pull/1599)). Thanks @jaelliot.
- **Ghidra analysis settings:** apply validated analyzer overrides before analysis while retaining the selected analysis profile. ([#1913](https://github.com/morluto/rea/pull/1913)). Thanks @akushonkamen, @kaoru0822-kitauji, @N0zoM1z0.
- **Attached native IDA MCP:** add an optional native bridge alongside existing legacy/headless profiles. Recorded native runs use Linux IDA Free 9.4 with a navigation correction and Windows IDA Pro 9.3 with a loopback-listener change; stock native switching, native macOS and Windows IDA Free remain unverified. Attached databases remain read-only and are never saved or closed. ([#1544](https://github.com/morluto/rea/pull/1544)). Thanks @imissapixel, @N0zoM1z0.
@@ -292,7 +292,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **Source maps:** original-source and mapping rows preserve a null source name when unavailable. Handle null explicitly instead of expecting an invented unknown-source string; new index fields remain optional when reading older Evidence. ([ee50ed1](https://github.com/morluto/rea/commit/ee50ed121e38cc6e1a2ff0dc8b789f15f71c5402)).
- **Streamed filtered JSON:** `--filter-output` now emits JSON `null` instead of `{}` when the selected value is null; consumers must distinguish a present null from a missing selection. Use `--full-output` when an object envelope is required. ([#2333](https://github.com/morluto/rea/pull/2333)). Thanks @Jackmeson1.
- **Call graphs:** `get_call_graph.normalized_result` is now `{ graph, limitations }`; read traversal-depth groups under `graph` and retain relationship-specific caveats. Contract/fixture validation does not establish new Ghidra or IDA engine coverage. ([5dfb9d0](https://github.com/morluto/rea/commit/5dfb9d050c2cc44a61b183dae765bef4d22a53b7)).
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/reference/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Provider admission:** MIPS targets with unresolved GNU ABI attributes are refused. Supply supported ABI declarations. Saved Hopper databases on macOS are unavailable because the launcher cannot run the analysis bridge when opening a database; select an executable target instead. Database support requires a script-capable database launch route. ([d4bd65c](https://github.com/morluto/rea/commit/d4bd65c53c0254d0195cc7ab8da163dcadfd3b9b)).
- **Segment extents:** `binary_overview` segment length may be null when Hopper coordinates cannot establish a finite extent in one address space. Preserve unknown rather than converting it to zero or a saturated value. ([36db19c](https://github.com/morluto/rea/commit/36db19c59)).
@@ -372,7 +372,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
@@ -540,7 +540,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
+27 -10
View File
@@ -11,7 +11,7 @@ incorrect user instructions warrant standalone corrections. Collect other
cosmetic-only suggestions in an issue rather than separate PRs.
For selected architectural owners and dependency paths, use the
[architecture map](docs/architecture.mermaid). It is not a complete directory or
[architecture map](docs/architecture/architecture.mermaid). It is not a complete directory or
import inventory. For current provider wiring, start with
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
the other `src/composition/` factories for tool services, including
@@ -32,7 +32,7 @@ removing it. Preserve validation, lazy loading, caller policy, and resource
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
formats, and independently versioned bridges require an explicit compatibility
decision; an internal representation alone does not establish that obligation.
See [interpreting boundary facts once](docs/tool-design.md#interpret-boundary-facts-once)
See [interpreting boundary facts once](docs/development/tool-design.md#interpret-boundary-facts-once)
for admission ownership and the distinction between repeated validation and
independent checks.
@@ -44,7 +44,7 @@ Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/S
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
compare equivalent real tasks before claiming an improvement.
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an admission ceiling, budget, or deadline, follow
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
for ownership, operation snapshots, provider integration, and the steps for
@@ -130,6 +130,19 @@ status checks alongside its existing checks. Deploy the workflow to each
protected target branch before requiring it there. The workflow reads PR
metadata using trusted default-branch code and writes only a commit status.
When an open PR may overlap another open change or a recently landed fix, note
the related PR, issue, or commit in the description or a comment as soon as the
overlap is known. Prefer one clear pointer over repeated status updates.
If an overlapping change merges while a PR awaits review, link that commit on
the open PR and state what remains useful. Rebase or otherwise align with the
current base when the remaining work is still wanted, narrow the PR to the
uncovered portion, or close it when the change is fully superseded. Keep the
description and validation current after that update. Maintainers may also note
the expected disposition (rebase, narrow, or close) when they land or recognize
an overlapping fix. Readiness still does not grant merge permission or a review
date.
Maintainers may return a PR to the author or close it when the problem is
unsupported, validation is missing, or the owner cannot explain the change.
Contributors are responsible for completing this work before requesting review.
@@ -234,7 +247,7 @@ guarantee.
Use the development toolchain pinned in `.nvmrc` and
`package.json#packageManager`. Run `nvm use` before installing dependencies;
installed-package runtime support is documented in
[installation](docs/installation.md#start-setup).
[installation](docs/getting-started/installation.md#start-setup).
Use Node.js and npm for source builds and verifier scripts; the installed
package's Bun runtime support does not extend to those development scripts.
@@ -289,16 +302,20 @@ base revision, or run the suite on Linux or WSL, before attributing them to a
change. The embedded-artifact parent-swap tests exercise the selected-path
replacement when the rename succeeds; if Windows refuses it while the admitted
file is open, they instead verify the refusal and cleanup. See
[testing](docs/testing.md#developer-commands) for command scopes
[testing](docs/development/testing.md#developer-commands) for command scopes
and lane selection.
`git submodule update --init --recursive` fails at the pinned Unblob submodule
on Windows and can leave later submodules unchecked out. Apply the
[Unblob checkout workaround](docs/firmware-analysis.md#windows-submodule-checkout),
[Unblob checkout workaround](docs/guides/firmware/firmware-analysis.md#windows-submodule-checkout),
then rerun the command.
## Documentation website
Use the [documentation directory](docs/README.md) and
[authoring guide](docs/development/documentation.md) for document ownership,
domain-based feature specifications and architectural decision records.
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
for live editing, `npm run docs:build` to check the production build and
links, and `npm run docs:preview` to preview that build at `/rea/`.
@@ -352,7 +369,7 @@ npm run check:fast
`test:local` runs source tests without building. `check:changed` combines cached
static checks with source tests affected since the merge base with `origin/main`.
Import-graph selection can miss runtime registration, bridges, and generated data;
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
for commands, test placement, pruning, and real-provider prerequisites.
Before requesting review, run relevant tests and `npm run check`; add
@@ -427,8 +444,8 @@ Run `npm run verify:hopper` on a supported macOS host or
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
native fixtures and verify the installed engine through CLI/MCP; package or
simulated-provider checks do not establish engine behavior. See
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
for supported hosts and unattended startup.
Use the default `build/conformance/manifest.json` unless explicitly verifying
@@ -437,7 +454,7 @@ Generated fixtures and manifests remain ignored.
## Maintainer release checklist
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
Release Please PR with the next version and changelog. Review its migration
notes, wait for the current CI checks to pass, and merge it into main to publish.
+72 -64
View File
@@ -1,7 +1,7 @@
<div align="center">
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md)
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md) · [Български](docs/readme/README_bg.md)
# REA: Reverse Engineer Anything
@@ -10,7 +10,7 @@
**See a feature you like. Understand how it works, down to the binary level.**
[![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/reference/mcp-contracts.md#generated-catalog)
[![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#what-you-can-analyze)
[![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything)
[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE)
@@ -72,7 +72,7 @@ Setup registers REA with your agent and installs matching workflow instructions.
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
checks. Recovery steps are in `docs/installation.md` under
checks. Recovery steps are in `docs/getting-started/installation.md` under
"Linux owned-process prerequisite".
### Set up your agent
@@ -91,8 +91,8 @@ available. Run package acquisition from a trusted directory outside the target
project. Installation and recovery details are in the setup guide below.
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
[other agents](docs/installation.md#supported-agents). See
[installation and setup](docs/installation.md) for provider configuration and
[other agents](docs/getting-started/installation.md#supported-agents). See
[installation and setup](docs/getting-started/installation.md) for provider configuration and
manual MCP registration and Claude Code/Codex plugin installation.
MCP uses compact input schemas by default to reduce context overhead. Set
@@ -135,7 +135,7 @@ rea --help
```
For native analysis, configure a provider first. See the
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
snapshots and scripting.
### Update REA
@@ -191,45 +191,47 @@ CLI commands use the same workflows.
REA requires Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+, plus npm.
Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
| Target | What REA returns | Requirements and guide |
| ----------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/guides/native/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
| Windows shortcuts | Shell Link target, arguments, working dir, ExtraData and Tracker droid facts | Built-in static LNK inspection; [Windows shortcuts](docs/guides/native/windows-shortcut.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/guides/javascript/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/guides/web/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
| Deno compile binaries | Virtual files, entry point, recorded options, npm snapshot length | None (static only); [Deno compile executables](docs/deno-standalone.md) |
Static JavaScript and .NET inspection read the supplied files without running
the application. Runtime capture runs or interacts with the selected target
@@ -243,15 +245,15 @@ or update either provider.
<a id="choosing-a-deep-analysis-provider"></a>
Native formats and host support vary by provider. See
[Hopper and Ghidra setup](docs/installation.md#hopper), the
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), the
[Kuna decompiler guide](docs/kuna-provider.md), and
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
For large binaries, raise the provider's startup deadline with
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
Check [release availability](docs/installation.md#released-package-and-main)
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
for features added since the latest npm release.
## Showcases
@@ -335,14 +337,20 @@ REA exports a runnable environment with a basic test to check that it starts and
[Environment reference](docs/agent-environments.md) ·
[Watch the demo on X](https://x.com/reatools/status/2112574822584918075?s=20)
### Desktop app environments
REA can also build an environment of a Linux desktop app, which an agent uses through screenshots, the mouse and the keyboard. From a one-line request, such as "Build me a Gnote environment where the agent adds olive oil to the shopping list note", your agent installs the app on a local build computer, writes the task, exports the environment and checks the task in a fresh computer. The machine that builds it needs Linux on amd64 or arm64, Docker and the agentenv framework version the guide pins.
[Set up desktop app environments](docs/desktop-build-computer.md)
## FAQ
<details>
<summary><strong>Which agents can use REA?</strong></summary>
Any agent that supports local MCP servers. Setup configures the
[supported agents](docs/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/installation.md#mcp-registry).
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
</details>
@@ -352,7 +360,7 @@ Any agent that supports local MCP servers. Setup configures the
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
and operation. Static JavaScript and .NET inspection work without a native
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
use your existing installations. See [provider setup](docs/installation.md#hopper)
use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper)
for the options and their supported targets.
</details>
@@ -364,7 +372,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
ask you to choose demo mode or activate your license. Complete that prompt in the
same user session before your first CLI/MCP operation or unattended job. Open
Hopper manually if needed. See
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
</details>
@@ -373,7 +381,7 @@ Hopper manually if needed. See
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
to register REA's MCP server and install the matching instructions, then restart
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
</details>
@@ -422,12 +430,12 @@ with your REA version, target type, steps to reproduce and error output.
Start with the website's [worked guides](https://rea.tools/guides/).
For exact options, prerequisites and result contracts:
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
Report vulnerabilities through [SECURITY.md](SECURITY.md).
@@ -462,8 +470,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
to improve the code or docs.
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
[testing](docs/testing.md) for verification lanes, and the
[architecture map](docs/architecture.mermaid) for selected layers and dependency paths.
[testing](docs/development/testing.md) for verification lanes, and the
[architecture map](docs/architecture/architecture.mermaid) for selected layers and dependency paths.
## License
+111 -27
View File
@@ -4,6 +4,117 @@
Security fixes are provided for the latest published version of REA. Upgrade to the latest release before reporting a problem that may already have been resolved.
## Security boundary
REA is a local reverse-engineering and automation tool. CLI and MCP use the
same application workflows and run with the host process's permissions.
REA is not a general-purpose sandbox or a multi-tenant authorization service.
### Threat model
- **Trusted authority:** the operating-system account running REA, the caller
or agent host that selects operations, and configured analysis providers,
their executables and extensions, and service endpoints. An MCP client can
invoke the operations exposed to it; its host must enforce the user's tool
permissions and approvals.
- **Untrusted data:** target binaries, archives, source files, project contents,
captures, web pages and returned target content can be malicious. Selecting
an input for static inspection does not authorize commands embedded in that
input. Agents must treat target text and Evidence as data rather
than instructions granting additional authority.
- **Operation-specific effects:** static inspection parses target data and may
launch an analysis engine. Runtime, debugger, instrumentation, browser,
desktop and environment workflows can deliberately execute code, attach to
processes, make network requests, write files or interact with applications
and devices. Review the selected operation's effects and limitations; static
inspection does not authorize launching or interacting with the target
application. See
[runtime and network responsibilities](DISCLAIMER.md#runtime-network-and-device-operations).
Standard client registrations use a dedicated stdio process. Opt-in
[HTTP MCP](docs/reference/mcp-contracts.md#transports) uses plain HTTP without
TLS or client authentication and shares provider/active-target state across
protocol sessions. Its session ID is not a user-authentication or isolation
boundary. Keep this endpoint within a trusted local or isolated environment.
### Intended security properties
Static JavaScript and .NET inspection do not run the target application.
Unexpected execution of input code through a data-only analysis path is a
security concern. Analysis engines themselves are executable software trusted
with the host account's permissions; parsing an input in an external engine
does not isolate that engine from the host.
REA authenticates its managed Hopper and Ghidra bridges with per-session random
capability tokens transferred through private session files, rather than
process arguments or environment variables. POSIX uses current-user Unix
sockets. Windows Ghidra uses authenticated loopback TCP and a private NTFS
runtime whose DACL admits only the current user and SYSTEM. These controls
protect the managed session boundary, not every external provider or debugger
endpoint. See [Windows Ghidra transport and lifecycle](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Provider ownership and mutation rules remain specific to each workflow:
- Ghidra's working projects are temporary and owned by REA. Opt-in
[checkpoints](docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
retain REA-owned Program state. A
[caller-selected existing project](docs/getting-started/installation.md#reuse-an-existing-ghidra-project)
is fingerprinted and copied into an owned runtime for read-only processing;
Ghidra does not open or save the original database through that workflow.
- [IDA analysis](docs/providers/ida-provider.md) is read-only. An attached GUI
database is never saved or closed by REA; owned headless sessions have a
separate lifecycle.
- Cleanup must stay within resources REA owns. Failed or unverified cleanup
can retain a workspace or session for retry; closing a session
does not imply that every investigation artifact has been erased.
`rea setup` shows planned configuration and installation changes and requires
approval before applying them. Ghidra, Java and IDA are user-managed. Approved
Linux Hopper installation authenticates the package's detached PGP signature
against the pinned publisher identity and checks the signed launcher against
the selected release's digest. The release feed's display version is separate
from that authenticated identity. See [installation and publisher verification](docs/getting-started/installation.md#hopper).
### Limits and caller responsibilities
- **Host isolation:** private runtimes and capability tokens do not protect
against malicious software already running as the same operating-system
user, or a compromised trusted client or provider. Use an appropriately
isolated account, VM or restricted environment when handling hostile targets.
- **Resource bounds:** timeouts, cancellation, streaming and size limits are
operation-specific. They are not a universal CPU, memory, disk or wall-time
limit on every target and third-party tool. Enforce hard host resource limits
externally when needed. Failures of a documented boundary or cancellation
control still warrant investigation.
- **Data and network exposure:** Evidence, caches, checkpoints, exports and
captures can retain private target data, paths, URLs and memory contents.
Local execution does not make these outputs safe to publish. Requested
network operations, agent hosts and provider features such as
[IDA Free's cloud decompiler](docs/providers/ida-provider.md) can communicate
outside the host. Review their data handling and secure any exposed endpoint.
- **Interpretation:** analysis, reconstruction and generated environments can
be incomplete or incorrect. Identity checks, signatures and successful
analysis do not establish that a target, provider or generated program is
safe to execute. Review the Evidence and proposed actions before relying on
them.
### Vulnerability scope
Please report suspected violations of these boundaries privately. Examples
include unintended code execution during data-only inspection, escaping a
documented extraction/output boundary, bypassing bridge authentication or
target/session identity checks, unauthorized effects on an unrelated resource,
and disclosure of transport credentials or explicitly marked sensitive values.
An attacker-triggered bypass of a documented resource or cancellation control
can also be security-relevant without code execution.
Incorrect analysis, unsupported formats, crashes, hangs and high resource use
without a security-boundary impact are generally ordinary bug reports. Effects
explicitly requested through an execution or interaction workflow are not by
themselves a vulnerability. A dependency advisory needs its actual REA exposure
assessed; a complete exploit or diagnosis is not required to report a concern.
If you are unsure whether a boundary is affected, use the private channel below.
## Reporting a vulnerability
Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
@@ -21,30 +132,3 @@ unrelated local paths from logs or examples.
The maintainer will acknowledge the report, assess its scope, and coordinate
remediation and disclosure timing with the reporter. Please allow time for
provider-dependent behavior to be reproduced safely.
## Security boundary
REA authenticates its Hopper and Ghidra bridges with per-session random
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Tokens are transferred through private session descriptors rather than process
arguments or environment variables.
This is not a sandbox and does not protect against malicious processes already
running as the same operating-system user. Opening an untrusted binary
delegates parsing and analysis to the selected local provider with that user's
permissions.
Ghidra sessions use a packaged Java `HeadlessScript`, an isolated temporary
project, and private home, cache, and runtime directories. REA imports only the
requested target, bounds startup and protocol messages, requests bounded CPU
and heap settings, and deletes the temporary project when the session closes.
These controls limit accidental persistence and resource use; they do not make
Ghidra a sandbox. REA does not open or modify user-owned Ghidra projects.
`GHIDRA_INSTALL_DIR` and optional `JAVA_HOME` identify user-managed
installations. Setup can add those values to detected client configuration only
after showing its plan and receiving approval. REA does not download, install,
upgrade, or modify Ghidra or Java.
On Linux, REA accepts release metadata only from Hopper's HTTPS endpoint, restricts package URLs to Hopper's public download origin, bounds the package size, and compares the downloaded bytes with Hopper's published checksum before requesting installation. The published SHA-1 value is a corruption check, not a modern package signature; HTTPS origin validation remains part of the trust boundary. Dependency installation is delegated without shell evaluation to `apt-get`, `dnf`, or `pacman`, directly as root or through `pkexec`. REA never invokes `sudo`.
+21 -21
View File
@@ -54,27 +54,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
executables and their dependency graphs are supplied separately. Git submodules
pin inspection sources; they are not included in the npm package.
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/getting-started/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
Ghidra's original notice credits software developed at the National Security
Agency. It explains that US-government-created portions are not subject to US
+267 -13
View File
@@ -1,7 +1,9 @@
"""Run REA inside an agentenv build computer for REA on the host: capture the app, draft its task and export it.
REA sends this file's source through the computer's ``computer-exec/v1`` route as
``python3 -I -c <source> <command> <JSON arguments>``; each command prints one JSON line and exits. ``start``
``python3 -I -c <source> <command> <JSON arguments>``. JSON that would exceed the bytes Linux
passes as one string travels as several arguments, which this file joins before parsing. Each command prints
one JSON line and exits. ``start``
leaves the capture running on its own: its stdin is a FIFO it holds open for reading and writing, so it never
sees the end of its input, and ``mark`` and ``stop`` write their requests there and wait for the capture's own
acknowledgement on its progress log. A small shell around the capture writes its exit status to a file, the
@@ -91,6 +93,13 @@ def _exit_code(capture: str):
return None
def _ended(capture: str, code: int) -> dict:
"""A capture that has exited: its status, the end of its progress log and its output's size, which REA reads
whole for the Evidence or failure it ended with, whichever step was waiting on it."""
return {"exited": code, "progress": _tail(os.path.join(capture, "progress.log")),
"output_bytes": os.path.getsize(os.path.join(capture, "capture.out"))}
def _event_line(path: str, offset: int, kinds: set, name):
"""The first progress line after ``offset`` whose event is of one of ``kinds``, for ``name`` when given."""
try:
@@ -119,17 +128,49 @@ def _wait(capture: str, offset: int, kinds: set, name, timeout: float) -> dict:
return {"line": line}
code = _exit_code(capture)
if code is not None:
return {"exited": code, "progress": _tail(log), "output": _tail(os.path.join(capture, "capture.out"))}
return _ended(capture, code)
if time.monotonic() > deadline:
return {"timed_out": True, "progress": _tail(log)}
time.sleep(POLL_S)
def _within(path: str, folder: str) -> bool:
return path == folder or path.startswith(folder + os.sep)
def _state_paths(values: list, work: str):
"""Each state path as an absolute path strictly under the home and outside REA's work folder, both as named
and with its links resolved, or the refusal of the first that is not. The capture resolves the links again
and retains what they lead to, so a link out of the home is refused here. Each passes on as named, so the
capture keeps what its spelling means, such as a hidden settings folder."""
home = os.path.normpath(os.path.expanduser("~"))
real_home, real_work = os.path.realpath(home), os.path.realpath(work)
spellings = {home, real_home}
paths = []
for value in values:
relative = value[2:] if value.startswith("~/") else value
path = os.path.normpath(os.path.join(home, relative))
if ".." in relative.split("/") or not any(path.startswith(spelling + os.sep) for spelling in spellings):
return None, f"The state path {value} is not under the computer's home, {home}."
# Every link on the way is followed, a dangling one too; a path that does not exist yet resolves
# through its nearest existing folder.
resolved = os.path.realpath(path)
if not resolved.startswith(real_home + os.sep):
return None, f"The state path {value} leads through a link to {resolved}, which is not under the computer's home, {home}."
if _within(path, work) or _within(resolved, real_work):
return None, f"The state path {value} is in REA's work folder {work}, which every capture leaves out."
paths.append(path)
return paths, None
def start(arguments: dict) -> int:
work, capture = os.path.abspath(arguments["work"]), os.path.abspath(arguments["capture"])
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer; start the computer from a request, which installs it."})
state_paths, refusal = _state_paths(arguments.get("state_paths", []), work)
if refusal is not None:
return _say({"refused": refusal})
if os.path.exists(capture):
return _say({"error": f"the capture folder {capture} is already used"})
apt = _apt(work, arguments.get("apt_timeout_s", 600))
@@ -143,20 +184,38 @@ def start(arguments: dict) -> int:
"--output", os.path.join(capture, "capture"), "--exclude", work, "--json"]
for value in arguments.get("arguments", []):
argv += ["--argument", value]
for path in state_paths:
argv += ["--state-path", path]
if arguments.get("package_file"):
argv += ["--package-file", arguments["package_file"]]
# Reading and writing, so the capture never reads the end of its input between requests.
reader = os.open(control, os.O_RDWR)
process = None
try:
with open(os.path.join(capture, "capture.out"), "wb") as out, \
open(os.path.join(capture, "progress.log"), "wb") as progress:
process = subprocess.Popen(
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
"sh", os.path.join(capture, "exit"), *argv],
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
start_new_session=True)
try:
with open(os.path.join(capture, "capture.out"), "wb") as out, \
open(os.path.join(capture, "progress.log"), "wb") as progress:
process = subprocess.Popen(
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
"sh", os.path.join(capture, "exit"), *argv],
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
start_new_session=True)
except Exception as cause:
# Nothing will write exit. Record one and drop the FIFO, so stop does not wait on it.
try:
with open(os.path.join(capture, "exit"), "w", encoding="utf-8") as handle:
handle.write("1")
except OSError:
pass
try:
os.unlink(control)
except OSError:
pass
return _say({"error": f"the capture did not start: {cause}"})
finally:
os.close(reader)
if process is None:
return _say({"error": "the capture did not start"})
waited = _wait(capture, 0, {"window_open"}, None, arguments.get("timeout_s", 120))
return _say({"pid": process.pid, "apt": apt, **waited})
@@ -175,11 +234,13 @@ def _request(capture: str, line: str) -> bool:
def mark(arguments: dict) -> int:
"""Request a checkpoint and answer with the capture's acknowledgement line as it wrote it: the checkpoint's
counts and up to ten of the files that changed, home-relative, with a count of the rest."""
capture, name = os.path.abspath(arguments["capture"]), arguments["name"]
code = _exit_code(capture)
log = os.path.join(capture, "progress.log")
if code is not None:
return _say({"exited": code, "progress": _tail(log)})
return _say(_ended(capture, code))
# A checkpoint already taken, whose answer was lost, is answered from the capture's own log.
taken = _event_line(log, 0, {"checkpoint"}, name)
if taken is not None:
@@ -213,6 +274,7 @@ def run(arguments: dict) -> int:
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer"})
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
for path, text in arguments.get("files", {}).items():
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
@@ -228,6 +290,187 @@ def run(arguments: dict) -> int:
"stderr": done.stderr[-TAIL_BYTES:].decode("utf-8", "replace")})
# The controller is sent with python -c, so a detached supervisor needs its
# own source. It alone holds the child process handle and signals its group.
LONG_RUN = r'''
import json, os, signal, subprocess, sys, threading, time
job = sys.argv[1]
result = {}
child = None
drainer = None
stderr_tail, stderr_errors = bytearray(), []
def drain(stream):
try:
while chunk := stream.read1(65536):
stderr_tail.extend(chunk)
del stderr_tail[:-4096]
except OSError as error:
stderr_errors.append(str(error))
finally:
try:
stream.close()
except OSError as error:
stderr_errors.append(str(error))
def timed_out_marker():
try:
open(os.path.join(job, "timed_out"), "x").close()
except FileExistsError:
pass
try:
request = json.loads("".join(sys.argv[2:]))
if os.path.exists(os.path.join(job, "cancel")):
result = {"absent": True}
else:
with open(request["stdout"], "wb") as out:
child = subprocess.Popen(request["argv"], cwd=request["home"], stdin=subprocess.DEVNULL,
stdout=out, stderr=subprocess.PIPE, start_new_session=True)
drainer = threading.Thread(target=drain, args=(child.stderr,), daemon=True)
drainer.start()
deadline = time.monotonic() + request["timeout_s"]
interrupted = False
timed_out = False
while child.poll() is None:
timed_out = timed_out or time.monotonic() >= deadline
if not interrupted and (timed_out or os.path.exists(os.path.join(job, "cancel"))):
interrupted = True
if timed_out:
timed_out_marker()
try:
os.killpg(child.pid, signal.SIGINT)
except OSError:
pass
time.sleep(0.2)
# A descendant can still hold stderr after the leader exits. Do
# not publish completion until that stream has actually closed.
while drainer.is_alive():
drainer.join(0.2)
if time.monotonic() >= deadline:
timed_out = True
timed_out_marker()
result = {"exit_code": child.returncode, "stdout_bytes": os.path.getsize(request["stdout"]),
"timed_out": timed_out, "cancelled": interrupted and not timed_out,
"stderr": bytes(stderr_tail).decode("utf-8", "replace")}
if stderr_errors:
result["error"] = f"The REA command's stderr could not be read: {stderr_errors[0]}"
except Exception as error:
result["error"] = f"The REA command supervisor failed: {error}"
# Keep the child handle until it exits, including when cancellation arrives
# after an unexpected supervisor error.
while child is not None and child.poll() is None:
if os.path.exists(os.path.join(job, "cancel")):
try:
os.killpg(child.pid, signal.SIGINT)
except OSError:
pass
time.sleep(0.2)
if drainer is not None and drainer.ident is not None:
drainer.join()
if child is not None:
result.setdefault("exit_code", child.returncode)
result.setdefault("stderr", bytes(stderr_tail).decode("utf-8", "replace"))
try:
result.setdefault("stdout_bytes", os.path.getsize(request["stdout"]))
except OSError:
pass
if child is None or child.poll() is not None:
temporary = os.path.join(job, "result.tmp")
with open(temporary, "w", encoding="utf-8") as out:
json.dump(result, out)
os.replace(temporary, os.path.join(job, "result.json"))
'''
def run_begin(arguments: dict) -> int:
"""Start one long REA command in its own capture-owned job folder."""
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer"})
job = os.path.abspath(arguments["job"])
os.makedirs(os.path.dirname(job), exist_ok=True)
try:
os.mkdir(job, 0o700)
except FileExistsError:
return _say({"refused": "the command job folder is already used"})
process = None
try:
with open(os.path.join(job, "owner"), "x", encoding="ascii") as out:
out.write(arguments["token"])
for path, text in arguments.get("files", {}).items():
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
handle.write(text)
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
request = json.dumps({"argv": [rea, *arguments["argv"]], "stdout": arguments["stdout"],
"home": os.path.expanduser("~"), "timeout_s": arguments["timeout_s"]})
chunks = [request[i:i + 60000] for i in range(0, len(request), 60000)]
with open(os.path.join(job, "supervisor.log"), "wb") as log:
process = subprocess.Popen([sys.executable, "-I", "-c", LONG_RUN, job, *chunks], cwd=os.path.expanduser("~"),
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL,
stdout=log, stderr=log, start_new_session=True)
except (OSError, ValueError) as error:
stage = "did not start" if process is None else "started but its setup did not finish"
result = {"error": f"The REA command supervisor {stage}: {error}"}
if process is None:
temporary = os.path.join(job, "result.tmp")
with open(temporary, "w", encoding="utf-8") as out:
json.dump(result, out)
os.replace(temporary, os.path.join(job, "result.json"))
return _say(result)
return _say({"started": True})
def _run_result(job: str):
try:
with open(os.path.join(job, "result.json"), encoding="utf-8") as source:
return json.load(source)
except FileNotFoundError:
return None
def _owns_run(job: str, token: str) -> bool:
try:
with open(os.path.join(job, "owner"), encoding="ascii") as owner:
return owner.read() == token
except OSError:
return False
def run_poll(arguments: dict) -> int:
job = os.path.abspath(arguments["job"])
if not _owns_run(job, arguments["token"]):
return _say({"error": "the command job ownership is not confirmed"})
deadline = time.monotonic() + arguments.get("timeout_s", 30)
while (result := _run_result(job)) is None and time.monotonic() < deadline:
time.sleep(POLL_S)
if result is None:
pending = {"pending": True, "timed_out": os.path.exists(os.path.join(job, "timed_out"))}
progress = _tail(os.path.join(job, "supervisor.log"))
return _say({**pending, **({"progress": progress} if progress else {})})
return _say({"finished": True, **result})
def run_cancel(arguments: dict) -> int:
job = os.path.abspath(arguments["job"])
os.makedirs(os.path.dirname(job), exist_ok=True)
try:
os.mkdir(job, 0o700)
except FileExistsError:
pass
else:
# A late begin cannot launch into a folder cancellation claimed.
with open(os.path.join(job, "owner"), "x", encoding="ascii") as owner:
owner.write(arguments["token"])
return _say({"absent": True})
if not _owns_run(job, arguments["token"]):
return _say({"error": "the command job ownership is not confirmed"})
with open(os.path.join(job, "cancel"), "a", encoding="utf-8"):
pass
return run_poll(arguments)
def listing(arguments: dict) -> int:
"""Every regular file under a folder, by its path relative to the home, with its size and SHA-256.
@@ -236,8 +479,18 @@ def listing(arguments: dict) -> int:
"""
home = os.path.expanduser("~")
files, skipped = [], []
for directory, folders, names in os.walk(os.path.abspath(arguments["root"])):
def fail_walk(error: OSError) -> None:
raise error
for directory, folders, names in os.walk(os.path.abspath(arguments["root"]), onerror=fail_walk):
folders.sort()
# os.walk does not follow directory symlinks; disclose them to the consumer.
for name in folders[:]:
path = os.path.join(directory, name)
if os.path.islink(path):
skipped.append(os.path.relpath(path, home))
folders.remove(name)
for name in sorted(names):
path = os.path.join(directory, name)
relative = os.path.relpath(path, home)
@@ -256,7 +509,8 @@ def listing(arguments: dict) -> int:
return _say({"listing_bytes": os.path.getsize(output)})
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing}
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing,
"run-begin": run_begin, "run-poll": run_poll, "run-cancel": run_cancel}
if __name__ == "__main__":
sys.exit(COMMANDS[sys.argv[1]](json.loads(sys.argv[2])))
sys.exit(COMMANDS[sys.argv[1]](json.loads("".join(sys.argv[2:]))))
+121
View File
@@ -0,0 +1,121 @@
"""Run one solve of a bundle's task in a fresh agentenv computer for REA and print how it ended.
REA runs ``solve.py <bundle folder> <task> <model> <agent>`` with the agentenv runner's own interpreter, so
``agent_env`` imports. It runs the task once on this machine with ``run_bundle``, the solver deployed as the agent
and asked on the model, the endpoint taken from the environment or agentenv's config as a run takes it, and the
computer torn down as the run ends. It prints one JSON line: the run's outcome, instance id, seconds, each check's
result, the solver's turns and cost from its trajectory when that is a local file, and agentenv's version; or
``{"error": ...}`` when the task could not run. Everything else, agentenv's progress included, goes to stderr. It exits
0 after a run and 2 when none ran.
"""
from __future__ import annotations
import json
import os
import sys
from importlib import metadata
from pathlib import Path
#: The most characters of a check's description or a run's error REA keeps.
MAX_TEXT = 400
def _protocol():
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
os.dup2(2, 1)
sys.stdout = sys.stderr
return protocol
def _say(protocol, value: dict) -> None:
protocol.write(json.dumps(value) + "\n")
protocol.flush()
def _field(value, name: str):
"""A field of agentenv's dataclasses, or of the dicts a stored context holds."""
return value.get(name) if isinstance(value, dict) else getattr(value, name, None)
def _trajectory_end(context) -> tuple:
"""The solver's turns and cost from the last record of its local trajectory, or None for each."""
for response in reversed(_field(context, "prompt_responses") or []):
url = _field(response, "agent_trajectory_object_url") or ""
if not url.startswith("file://"):
continue
try:
records = json.loads(Path(url[len("file://"):]).read_text(encoding="utf-8"))
except (OSError, ValueError):
return None, None
last = records[-1] if isinstance(records, list) and records and isinstance(records[-1], dict) else {}
turns, usd = last.get("num_turns"), last.get("total_cost_usd")
return (turns if isinstance(turns, int) else None,
float(usd) if isinstance(usd, (int, float)) and not isinstance(usd, bool) else None)
return None, None
def _checks(context, task: str) -> list:
"""Each check's result as the task's verifier recorded it under the instance's verifications."""
verifications = (_field(context, "metadata") or {}).get("verifications") or {}
found = verifications.get(task) or next(iter(verifications.values()), {})
results = found.get("results") if isinstance(found, dict) else None
return [{"id": str(result.get("id")), "passed": result.get("result") is True,
"description": str(result.get("description", ""))[:MAX_TEXT]}
for result in results or [] if isinstance(result, dict)]
def _version() -> str | None:
try:
return metadata.version("agentenv-framework")
except metadata.PackageNotFoundError:
return None
def _cleanup(result) -> dict:
"""Only a returned teardown report can confirm what agentenv released."""
if result is None:
return {"unknown": True, "sandboxes": []}
unknown, sandboxes = False, []
for run in result.runs:
report = getattr(run, "torn_down", None)
if report is None:
unknown = True
else:
sandboxes.extend({"id": item.sandbox_id, "type": item.sandbox_type} for item in report.still_up)
return {"unknown": unknown, "sandboxes": sandboxes}
def main(argv: list[str]) -> int:
protocol = _protocol()
bundle, task, model, agent = argv[1:5]
try:
from agent_env.bundle import run_bundle
except ImportError as error:
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}",
"cleanup": {"unknown": False, "sandboxes": []}})
return 2
try:
result = run_bundle(bundle, tasks=[task], model=model, agent=agent, sandbox="local", emulate=False,
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its run down
_say(protocol, {"error": f"{type(error).__name__}: {error}"[:MAX_TEXT * 4],
"cleanup": _cleanup(getattr(error, "result", None))})
return 2
run = result.runs[0] if result.runs else None
if run is None:
_say(protocol, {"error": "the bundle ran no task", "cleanup": _cleanup(result)})
return 2
turns, usd = _trajectory_end(run.context)
error = getattr(run, "error", None)
_say(protocol, {"outcome": run.outcome.value, "instance_id": run.instance_id,
"seconds": round(float(run.duration), 1), "checks": _checks(run.context, task),
"turns": turns, "usd": usd, "agentenv": _version(),
"error": None if error is None else f"{type(error).__name__}: {error}"[:MAX_TEXT],
"cleanup": _cleanup(result)})
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+63
View File
@@ -0,0 +1,63 @@
"""Recover local sandboxes under one stopped solve's private agentenv root."""
from __future__ import annotations
import asyncio
import contextlib
import json
import os
import subprocess
import sys
from pathlib import Path
def _docker(*args: str) -> str:
command = os.environ.get("REA_DOCKER_COMMAND") or "docker"
result = subprocess.run([command, *args], capture_output=True, text=True, check=False)
if result.returncode:
raise RuntimeError(f"Docker {' '.join(args[:2])} failed: {result.stderr[-1000:]}")
return result.stdout.strip()
async def cleanup(root: Path) -> None:
# A helper has already stopped. agentenv lists the sandboxes under the root
# (AGENT_ENV_LOCAL_SANDBOX_DIR); anything else there remains owned for inspection.
if not any(root.iterdir()):
return
from agent_env.providers.sandbox_providers.local_docker.provider import (
LocalSandboxProvider, local_sandbox_ids, remove_local_work_dir,
)
for sandbox_id in local_sandbox_ids():
await LocalSandboxProvider().terminate_sandbox(sandbox_id)
# The pinned provider reports incomplete teardown and retains the folder.
# Keep REA's independent Docker/Linux checks before releasing that owner.
if _docker("ps", "-aq", "--filter", f"name=^agent-{sandbox_id}$"):
raise RuntimeError(f"Sandbox {sandbox_id} still has its agent container")
if _docker("ps", "-aq", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled containers")
if _docker("network", "ls", "-q", "--filter", f"label=agentenv.sandbox={sandbox_id}"):
raise RuntimeError(f"Sandbox {sandbox_id} still has labeled networks")
if sys.platform != "linux":
raise RuntimeError(f"Cannot verify host processes for sandbox {sandbox_id} on {sys.platform}")
from agent_env.providers.sandbox_providers.local_docker.provider import _marked_pids
if _marked_pids(sandbox_id):
raise RuntimeError(f"Sandbox {sandbox_id} still has marked host processes")
if remove_local_work_dir(sandbox_id) is None:
raise RuntimeError(f"Could not confirm removal of sandbox {sandbox_id}'s work folder")
leftover = sorted(entry.name for entry in root.iterdir())
if leftover:
raise RuntimeError(f"Unexpected sandbox root entries: {', '.join(leftover)}")
def main() -> int:
try:
with contextlib.redirect_stdout(sys.stderr):
asyncio.run(cleanup(Path(os.environ["AGENT_ENV_LOCAL_SANDBOX_DIR"])))
print(json.dumps({"ok": True}))
return 0
except Exception as error:
print(json.dumps({"ok": False, "error": f"{type(error).__name__}: {error}"}))
return 1
if __name__ == "__main__":
sys.exit(main())
+232
View File
@@ -0,0 +1,232 @@
"""Exercise the capture controller's filesystem inventory boundary."""
import json
import io
from contextlib import redirect_stdout
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
from bridge.agentenv import capture
CONTROLLER = Path(__file__).with_name("capture.py")
class CaptureListingTest(unittest.TestCase):
def run_listing(self, root, output):
return subprocess.run(
[
sys.executable,
str(CONTROLLER),
"list",
json.dumps({"root": str(root), "output": str(output)}),
],
text=True,
capture_output=True,
check=False,
)
def test_directory_symlink_is_disclosed_without_following_it(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
root = parent / "root"
target = parent / "target"
output = parent / "listing.json"
root.mkdir()
target.mkdir()
(root / "present.txt").write_text("present")
(target / "outside.txt").write_text("outside")
try:
(root / "linked").symlink_to(target, target_is_directory=True)
except OSError as error:
self.skipTest(f"directory symlinks unavailable: {error}")
result = self.run_listing(root, output)
self.assertEqual(result.returncode, 0, result.stderr)
listing = json.loads(output.read_text())
self.assertEqual(
[item["path"] for item in listing["files"]],
[os.path.relpath(root / "present.txt", Path.home())],
)
self.assertEqual(
listing["skipped"], [os.path.relpath(root / "linked", Path.home())]
)
def test_missing_root_does_not_produce_an_empty_successful_inventory(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
output = parent / "listing.json"
result = self.run_listing(parent / "missing", output)
self.assertNotEqual(result.returncode, 0)
self.assertFalse(output.exists())
@unittest.skipIf(sys.platform == "win32", "desktop build computers use Linux")
class LongRunTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.root = Path(self.tmp.name)
self.bin = self.root / "bin"
self.bin.mkdir()
rea = self.bin / "rea"
rea.write_text(
f"#!{sys.executable}\n"
"import os, signal, subprocess, sys, time\n"
"signal.signal(signal.SIGINT, lambda *_: sys.exit(130))\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'wait':\n"
" open(sys.argv[2], 'w').close()\n"
" time.sleep(30)\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'ignore':\n"
" signal.signal(signal.SIGINT, signal.SIG_IGN)\n"
" open(sys.argv[2], 'w').close()\n"
" while not os.path.exists(sys.argv[3]): time.sleep(0.02)\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'stderr':\n"
" sys.stderr.write('x' * 100000 + 'TAIL')\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'hold':\n"
" subprocess.Popen([sys.executable, '-c', 'import os,sys,time\\nwhile not os.path.exists(sys.argv[1]): time.sleep(0.02)', sys.argv[2]], stdout=subprocess.DEVNULL, stderr=sys.stderr, start_new_session=True)\n"
" sys.exit(0)\n"
"print(os.environ.get('AGENTENV_CONTROL_TOKEN', 'no token'))\n"
)
rea.chmod(0o755)
self.environment = {**os.environ, "HOME": str(self.root), "PATH": f"{self.bin}{os.pathsep}{os.environ.get('PATH', '')}",
"AGENTENV_CONTROL_TOKEN": "not-for-the-child"}
self.job = self.root / "job"
self.output = self.root / "out.json"
def command(self, name, **values):
result = subprocess.run([sys.executable, str(CONTROLLER), name, json.dumps(values)],
env=self.environment, text=True, capture_output=True, timeout=10, check=True)
return json.loads(result.stdout)
def begin(self, argv, timeout=60):
return self.command("run-begin", job=str(self.job), work=str(self.root), stdout=str(self.output),
token="owner-token", argv=argv, files={}, timeout_s=timeout)
def test_runs_and_reports_its_output(self):
self.assertEqual(self.begin([]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(result["exit_code"], 0)
self.assertEqual(self.output.read_text().strip(), "no token")
def test_passes_large_command_without_persisting_its_arguments(self):
self.assertEqual(self.begin(["done", *("x" * 4000 for _ in range(20))]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(result["exit_code"], 0)
self.assertFalse((self.job / "request.json").exists())
def test_keeps_only_a_bounded_stderr_tail(self):
self.assertEqual(self.begin(["stderr"]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(len(result["stderr"]), 4096)
self.assertTrue(result["stderr"].endswith("TAIL"))
def test_cancels_a_running_child_and_confirms_exit(self):
ready = self.root / "ready"
self.assertEqual(self.begin(["wait", str(ready)]), {"started": True})
import time
deadline = time.monotonic() + 5
while not ready.exists() and time.monotonic() < deadline:
time.sleep(0.02)
self.assertTrue(ready.exists())
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["cancelled"])
self.assertEqual(result["exit_code"], 130)
def test_cancel_before_late_start_claims_the_job(self):
self.assertEqual(self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1), {"absent": True})
self.assertIn("already used", self.begin([])["refused"])
self.assertFalse(self.output.exists())
def test_cancel_refuses_a_folder_without_the_matching_owner(self):
self.job.mkdir()
(self.job / "owner").write_text("someone-else")
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1)
self.assertIn("ownership is not confirmed", result["error"])
self.assertFalse((self.job / "cancel").exists())
def test_reports_supervisor_output_if_completion_is_unconfirmed(self):
self.job.mkdir()
(self.job / "owner").write_text("owner-token")
(self.job / "supervisor.log").write_text("supervisor failed")
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=0)
self.assertEqual(result["progress"], "supervisor failed")
self.assertTrue(result["pending"])
def test_reports_a_timed_out_child_that_has_not_stopped(self):
ready, release = self.root / "ready", self.root / "release"
self.assertEqual(self.begin(["ignore", str(ready), str(release)], timeout=3), {"started": True})
try:
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
self.assertTrue(result["pending"])
self.assertTrue(result["timed_out"])
self.assertTrue(ready.exists())
finally:
release.touch()
self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
def test_waits_for_an_inherited_stderr_pipe_after_the_child_exits(self):
release = self.root / "release"
self.assertEqual(self.begin(["hold", str(release)], timeout=3), {"started": True})
try:
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
self.assertTrue(result["pending"])
self.assertTrue(result["timed_out"])
finally:
release.touch()
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["finished"])
def test_failure_closing_supervisor_log_after_launch_does_not_claim_exit(self):
ready = self.root / "ready"
original_open = open
original_popen = subprocess.Popen
supervisors = []
class FailingClose:
def __init__(self, file):
self.file = file
def __enter__(self):
return self.file
def __exit__(self, *args):
self.file.close()
raise OSError("log close failed")
def opening(path, *args, **kwargs):
file = original_open(path, *args, **kwargs)
return FailingClose(file) if str(path).endswith("supervisor.log") else file
def spawning(*args, **kwargs):
process = original_popen(*args, **kwargs)
supervisors.append(process)
def cleanup():
if process.poll() is None:
(self.job / "cancel").touch()
process.wait(timeout=5)
self.addCleanup(cleanup)
return process
arguments = {"job": str(self.job), "work": str(self.root), "stdout": str(self.output),
"token": "owner-token", "argv": ["wait", str(ready)], "files": {}, "timeout_s": 60}
with patch.dict(os.environ, self.environment, clear=True), patch("builtins.open", side_effect=opening), patch.object(capture.subprocess, "Popen", side_effect=spawning), redirect_stdout(io.StringIO()) as output:
capture.run_begin(arguments)
self.assertIn("started but its setup did not finish", json.loads(output.getvalue())["error"])
import time
deadline = time.monotonic() + 5
while not ready.exists() and time.monotonic() < deadline:
time.sleep(0.02)
self.assertTrue(ready.exists())
self.assertFalse((self.job / "result.json").exists())
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["cancelled"])
supervisors[0].wait(timeout=5)
if __name__ == "__main__":
unittest.main()
+108
View File
@@ -0,0 +1,108 @@
"""Exercise cleanup decisions without a live agentenv provider."""
import asyncio
import tempfile
import types
import unittest
from pathlib import Path
from unittest.mock import patch
from bridge.agentenv import solve_cleanup
class SolveCleanupTest(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.folder = self.root / "agent-env-local-a1b2c3d4-abcdef12"
self.folder.mkdir()
self.terminated = []
self.module = types.ModuleType("agent_env.providers.sandbox_providers.local_docker.provider")
test = self
class Provider:
async def terminate_sandbox(self, sandbox_id):
test.terminated.append(sandbox_id)
self.module.LocalSandboxProvider = Provider
self.module.local_sandbox_ids = self._listed
self.module.remove_local_work_dir = lambda sandbox_id: self._remove(sandbox_id)
self.module._marked_pids = lambda _sandbox_id: []
self.patch_module = patch.dict("sys.modules", {self.module.__name__: self.module})
self.patch_module.start()
self.addCleanup(self.patch_module.stop)
def _listed(self):
# As agentenv lists them: `agent-env-<id>-<suffix>` folders, whatever the id's shape.
found = set()
for entry in self.root.iterdir():
sandbox_id, _, suffix = entry.name.removeprefix("agent-env-").rpartition("-")
if entry.name.startswith("agent-env-") and sandbox_id and suffix and entry.is_dir():
found.add(sandbox_id)
return sorted(found)
def _remove(self, sandbox_id):
folder = next(self.root.glob(f"agent-env-{sandbox_id}-*"))
folder.rmdir()
return folder
def run_cleanup(self):
asyncio.run(solve_cleanup.cleanup(self.root))
def test_terminates_owned_sandbox_before_removing_its_folder(self):
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertEqual(self.terminated, ["local-a1b2c3d4"])
self.assertFalse(self.folder.exists())
def test_retains_folder_when_docker_reports_container_then_retries(self):
with patch.object(solve_cleanup, "_docker", return_value="still-running"), patch.object(solve_cleanup.sys, "platform", "linux"):
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
self.run_cleanup()
self.assertTrue(self.folder.exists())
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertFalse(self.folder.exists())
self.assertEqual(len(self.terminated), 2)
def test_partial_cleanup_and_late_folder_are_retried(self):
later = self.root / "agent-env-local-f1223344-abcdef12"
later.mkdir()
def docker(*args):
return "still-running" if "local-f1223344" in " ".join(args) else ""
with patch.object(solve_cleanup, "_docker", side_effect=docker), patch.object(solve_cleanup.sys, "platform", "linux"):
with self.assertRaisesRegex(RuntimeError, "still has its agent container"):
self.run_cleanup()
self.assertTrue(later.exists())
self.assertFalse(self.folder.exists())
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertEqual(list(self.root.iterdir()), [])
def test_macos_keeps_folder_when_host_process_cleanup_cannot_be_verified(self):
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "darwin"):
with self.assertRaisesRegex(RuntimeError, "Cannot verify host processes"):
self.run_cleanup()
self.assertTrue(self.folder.exists())
def test_cleans_up_each_sandbox_agentenv_lists_whatever_its_id_looks_like(self):
self.folder.rmdir()
listed = self.root / "agent-env-local-a1b2c3d4e5-abcdef12"
listed.mkdir()
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
self.run_cleanup()
self.assertEqual(self.terminated, ["local-a1b2c3d4e5"])
self.assertFalse(listed.exists())
def test_unexpected_entry_remains_owned(self):
self.folder.rmdir()
(self.root / "unexpected").mkdir()
with patch.object(solve_cleanup, "_docker", return_value=""), patch.object(solve_cleanup.sys, "platform", "linux"):
with self.assertRaisesRegex(RuntimeError, "Unexpected sandbox root entries: unexpected"):
self.run_cleanup()
self.assertEqual(self.terminated, [])
self.assertTrue((self.root / "unexpected").exists())
if __name__ == "__main__":
unittest.main()
+220 -25
View File
@@ -119,9 +119,9 @@ import ghidra.program.util.VarnodeContext;
public final class ReaGhidraBridge extends HeadlessScript {
// At most 24 KiB after worst-case JSON escaping (six bytes per UTF-16 unit).
private static final int PDB_ATTRIBUTE_UTF16_BUDGET = 4096;
private static final int MAX_HIGH_PCODE_OPS = 3000;
private static final int MAX_HIGH_PCODE_INPUTS_PER_OP = 64;
private static final int MAX_HIGH_PCODE_DEF_USE_EDGES = 12000;
private static final int DEFAULT_HIGH_PCODE_OPS = 3000;
private static final int DEFAULT_HIGH_PCODE_INPUTS_PER_OP = 64;
private static final int DEFAULT_HIGH_PCODE_DEF_USE_EDGES = 12000;
private static final Gson GSON = new GsonBuilder().serializeNulls().create();
private final Map<FileBytes, Boolean> targetSourceIdentity = new IdentityHashMap<>();
private String admittedTargetSha256;
@@ -179,10 +179,15 @@ public final class ReaGhidraBridge extends HeadlessScript {
private List<FunctionEntry> procedureInventory;
private List<InventoryItem> stringInventory;
private DecompInterface decompiler;
private boolean existingProject;
// A refused Program may only report its admission failure to the client.
private String identityAdmissionFailure;
/** Per-function native decompiler limit from the session; 0 leaves it unbounded. */
private int decompileTimeoutSeconds;
private Integer decompileMaxPayloadMiB;
// DecompileProcess converts MiB to bytes using a signed-int left shift.
private static final int MAX_DECOMPILE_PAYLOAD_MIB = Integer.MAX_VALUE >> 20;
private ValueFlowLimits valueFlowLimits = defaultValueFlowLimits();
// DecompileProcess converts seconds to an int millisecond timer delay.
private static final int MAX_DECOMPILE_TIMEOUT_SECONDS = Integer.MAX_VALUE / 1000;
private final ReaGhidraExtensions analysisExtensions = new ReaGhidraExtensions();
@@ -199,6 +204,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
Path descriptorPath = Path.of(arguments[0]);
checkpointDirectory = descriptorPath.toAbsolutePath().normalize().getParent();
SessionDescriptor descriptor = readDescriptor(descriptorPath);
existingProject = descriptor.existingProject;
decompileTimeoutSeconds = descriptor.decompileTimeoutSeconds();
// Windows retains a native immutable bearer descriptor lease until
// runtime_close; POSIX removes it after this read. Normal cleanup removes
@@ -424,6 +430,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
decompileTimeoutSeconds = request.decompileTimeoutSeconds() == null
? descriptor.decompileTimeoutSeconds() : request.decompileTimeoutSeconds();
valueFlowLimits = request.valueFlowLimits();
applyDecompilePayloadLimit(request.decompileMaxPayloadMiB());
writeSuccess(writer, request.id, handleRequest(request, descriptor));
}
catch (RequestFailure failure) {
@@ -876,6 +884,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonArray blocks = new JsonArray();
JsonArray skipped = new JsonArray();
JsonArray notLoaded = new JsonArray();
JsonArray overlays = new JsonArray();
List<ByteSearchBlock> eligible = new ArrayList<>();
for (MemoryBlock block : ordered) {
monitor.checkCancelled();
@@ -883,10 +892,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
Address high = block.getEnd();
Address bound = start == null ? end : start;
if (bound != null && !bound.getAddressSpace().equals(low.getAddressSpace())) continue;
if (bound == null && (!block.isLoaded() || block.isOverlay())) {
// Without bounds, search loaded non-overlay memory. Unloaded file
// parts and loaded overlays stay reachable through explicit bounds.
if (bound == null && !block.isLoaded()) {
notLoaded.add(byteSearchCoverage(block, low, high));
continue;
}
if (bound == null && block.isOverlay()) {
overlays.add(byteSearchCoverage(block, low, high));
continue;
}
if (start != null && low.compareTo(start) < 0) low = start;
if (end != null && high.compareTo(end) > 0) high = end;
if (low.compareTo(high) > 0) continue;
@@ -960,6 +975,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
searched.addProperty("bytes", covered);
searched.add("skipped_uninitialized", skipped);
searched.add("skipped_not_loaded", notLoaded);
searched.add("skipped_overlays", overlays);
JsonObject result = new JsonObject();
result.add("matches", matches);
result.add("searched", searched);
@@ -1028,11 +1044,26 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private void initializeDecompiler() {
private DecompileOptions decompileOptions(Integer payloadMiB) {
DecompileOptions options = new DecompileOptions();
options.setMaxInstructions(Integer.MAX_VALUE);
if (payloadMiB != null) options.setMaxPayloadMBytes(payloadMiB);
return options;
}
private void applyDecompilePayloadLimit(Integer payloadMiB) {
if (java.util.Objects.equals(decompileMaxPayloadMiB, payloadMiB)) return;
// DecompInterface supports live setOptions: it updates the existing
// DecompileProcess result-size bound before sending native options.
if (!decompiler.setOptions(decompileOptions(payloadMiB)))
throw new RequestFailure("decompile_options_failed", "Ghidra could not apply the selected native decompiler payload limit");
decompiler.flushCache();
decompileMaxPayloadMiB = payloadMiB;
}
private void initializeDecompiler() {
decompiler = new DecompInterface();
decompiler.setOptions(options);
decompiler.setOptions(decompileOptions(decompileMaxPayloadMiB));
decompiler.toggleCCode(true);
decompiler.toggleSyntaxTree(true);
decompiler.toggleJumpLoads(true);
@@ -1414,8 +1445,29 @@ public final class ReaGhidraBridge extends HeadlessScript {
flow.addProperty("conditional", false);
flow.addProperty("computed", false);
if (instruction == null) {
result.addProperty("status", !currentProgram.getMemory().contains(address) ? "outside-memory" : currentProgram.getListing().getInstructionContaining(address) != null
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "not-in-listing");
Instruction containing = currentProgram.getListing().getInstructionContaining(address);
boolean inMemory = currentProgram.getMemory().contains(address);
Data containingData = inMemory && containing == null
? currentProgram.getListing().getDefinedDataContaining(address) : null;
result.addProperty("status", !inMemory ? "outside-memory" : containing != null
? "not-instruction-boundary" : containingData != null ? "data" : "not-in-listing");
if (inMemory && containing == null && containingData != null) {
JsonObject parentData = new JsonObject();
parentData.addProperty("address", canonicalAddress(containingData.getAddress()));
parentData.addProperty("length", containingData.getLength());
parentData.addProperty("type_id", containingData.getDataType().getPathName());
parentData.addProperty("provenance", "ghidra-listing");
result.add("containing_data", parentData);
}
if (containing != null && currentProgram.getMemory().contains(address)) {
JsonObject containingInstruction = new JsonObject();
containingInstruction.addProperty("address", canonicalAddress(containing.getAddress()));
containingInstruction.addProperty("length", containing.getLength());
containingInstruction.addProperty("mnemonic", containing.getMnemonicString());
containingInstruction.addProperty("raw_disassembly", containing.toString());
containingInstruction.addProperty("provenance", "ghidra-listing");
result.add("containing_instruction", containingInstruction);
}
for (String key : List.of("bytes", "length", "mnemonic", "raw_disassembly")) result.add(key, JsonNull.INSTANCE);
} else {
result.addProperty("status", "decoded");
@@ -1825,7 +1877,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
while (operations.hasNext()) {
monitor.checkCancelled();
PcodeOpAST operation = operations.next();
if (selected.size() >= MAX_HIGH_PCODE_OPS) {
if (selected.size() >= valueFlowLimits.maxOperations()) {
omittedOperationsLowerBound = 1;
break;
}
@@ -1853,6 +1905,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonArray effects = new JsonArray();
int omittedInputs = 0;
int omittedEdges = 0;
boolean edgeLimitReached = false;
for (PcodeOpAST operation : selected) {
monitor.checkCancelled();
String operationId = operationIds.get(operation);
@@ -1868,14 +1921,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
value.addProperty("block_membership", operation.getParent() == null ? "detached" : "member");
JsonArray inputs = new JsonArray();
int inputCount = operation.getNumInputs();
int capturedInputs = Math.min(inputCount, MAX_HIGH_PCODE_INPUTS_PER_OP);
int capturedInputs = Math.min(inputCount, valueFlowLimits.maxInputsPerOperation());
for (int inputIndex = 0; inputIndex < capturedInputs; inputIndex += 1) {
Varnode input = operation.getInput(inputIndex);
inputs.add(varnodeValue(input));
if (input instanceof VarnodeAST astInput) {
if (astInput.getHigh() instanceof HighParam parameter) {
if (parameterUses.size() >= MAX_HIGH_PCODE_DEF_USE_EDGES) omittedEdges++;
else {
if (parameterUses.size() >= valueFlowLimits.maxEdges()) {
edgeLimitReached = true;
omittedEdges++;
} else {
JsonObject use = new JsonObject();
use.addProperty("ordinal", parameter.getSlot());
use.addProperty("use", operationId);
@@ -1886,7 +1941,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
PcodeOp definition = astInput.getDef();
if (definition instanceof PcodeOpAST astDefinition) {
String definitionId = operationIds.get(astDefinition);
if (definitionId == null || defUse.size() >= MAX_HIGH_PCODE_DEF_USE_EDGES) {
if (definitionId == null || defUse.size() >= valueFlowLimits.maxEdges()) {
if (defUse.size() >= valueFlowLimits.maxEdges()) edgeLimitReached = true;
omittedEdges += 1;
} else {
JsonObject edge = new JsonObject();
@@ -1909,6 +1965,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonObject result = new JsonObject();
result.addProperty("available", true);
result.addProperty("provenance", "ghidra-high-pcode");
JsonObject limits = new JsonObject();
limits.addProperty("max_operations", valueFlowLimits.maxOperations());
limits.addProperty("max_inputs_per_operation", valueFlowLimits.maxInputsPerOperation());
limits.addProperty("max_edges", valueFlowLimits.maxEdges());
result.add("limits", limits);
JsonArray limitsReached = new JsonArray();
if (omittedOperationsLowerBound > 0) limitsReached.add("max_operations");
if (omittedInputs > 0) limitsReached.add("max_inputs_per_operation");
if (edgeLimitReached) limitsReached.add("max_edges");
result.add("limits_reached", limitsReached);
result.add("operations", operationValues);
result.add("def_use", defUse);
result.add("parameters", parameterValues);
@@ -1930,19 +1996,42 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add(
"Def-use links do not cross function boundaries and may omit memory aliasing, call side effects, or unresolved indirect flow."
);
limitations.add(
"Memory read/write effects cover high-p-code LOAD and STORE operations only; direct memory address varnodes in other operations are not included. An empty effect list does not establish that no memory is accessed. Recorded instruction references may provide complementary evidence."
);
limitations.add(
"Memory read/write effects include stack and temporary memory operations; they do not identify persistent state or business meaning."
);
limitations.add(
"Operation, input, and def-use limits can truncate large functions; inspect truncation, omitted lower bounds, and known omitted counts before drawing conclusions."
"Operation, input, and def-use limits can truncate large functions; inspect the effective limits, limits_reached and omitted counts before drawing conclusions."
);
limitations.add(
"Known omitted-edge counts cover links omitted from retained operations; omitted operations may contain additional uncounted links."
);
if (omittedOperationsLowerBound > 0)
limitations.add(valueFlowLimitRecovery("max_operations", valueFlowLimits.maxOperations()));
if (omittedInputs > 0)
limitations.add(valueFlowLimitRecovery("max_inputs_per_operation", valueFlowLimits.maxInputsPerOperation()));
if (edgeLimitReached)
limitations.add(valueFlowLimitRecovery("max_edges", valueFlowLimits.maxEdges()));
result.add("limitations", limitations);
return result;
}
private static String valueFlowLimitRecovery(String field, int effective) {
String setting = "ghidra.value_flow_" + field;
String environment = "REA_GHIDRA_VALUE_FLOW_" + field.toUpperCase(Locale.ROOT);
String adjustment = effective < Integer.MAX_VALUE
? "In MCP, use update_analysis_settings with setting " + setting +
", action set and a larger integer value from 1 to " + Integer.MAX_VALUE +
", then repeat the selected function analysis on this Program without restarting REA. " +
"For a one-shot CLI retry, set " + environment +
" to a larger count in the environment that starts the next rea invocation, then repeat the selected analysis; the previous invocation's Program is no longer active."
: "The setting is at its Java integer maximum; select a smaller function for this projection.";
return setting + " reached its effective value of " + effective + ". " + adjustment +
" These controls do not raise the decompile deadline, bridge transport or MCP client limits.";
}
private String pcodeOperationId(PcodeOpAST operation) {
return canonicalAddress(operation.getSeqnum().getTarget()) + "#" +
operation.getSeqnum().getTime();
@@ -2376,7 +2465,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (!match.matched()) {
continue;
}
JsonObject item = resultItem(entry.item, null);
JsonObject item = resultItem(entry.item, "procedure");
if (!match.aliases().isEmpty()) {
JsonArray aliases = new JsonArray();
for (String alias : match.aliases()) aliases.add(alias);
@@ -2391,7 +2480,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (!matcher.matches(item.value)) {
continue;
}
items.add(resultItem(item, null));
items.add(resultItem(item, procedureSearch ? "procedure" : "string"));
}
return items;
}
@@ -3665,12 +3754,26 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (!results.decompileCompleted()) {
throw new RequestFailure(
"decompile_failed",
"Ghidra decompilation failed: " + diagnosticMessage(results.getErrorMessage())
"Ghidra decompilation failed: " + diagnosticMessage(results.getErrorMessage()) +
". The effective native decompiler payload allowance is " +
decompiler.getOptions().getMaxPayloadMBytes() +
" MiB. " + decompilePayloadRecovery()
);
}
return results;
}
private String decompilePayloadRecovery() {
int effective = decompiler.getOptions().getMaxPayloadMBytes();
if (effective >= MAX_DECOMPILE_PAYLOAD_MIB)
return "If the diagnostic reports payload exhaustion, the native payload allowance is already at its supported maximum; no higher value is accepted. Use analyze_function for available listing facts or explicitly select a smaller function for decompilation.";
return "If the diagnostic reports payload exhaustion, in MCP raise ghidra.decompile_max_payload_mib with update_analysis_settings " +
"(integer MiB from 1 to " + MAX_DECOMPILE_PAYLOAD_MIB +
") and repeat the selected operation on this Program without restarting REA. " +
"For a one-shot CLI retry, set REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB to a larger MiB value in the environment that starts the next rea invocation, then repeat the selected operation; the previous invocation's Program is no longer active. " +
"Runtime null or an omitted launch seed restores the installed Ghidra default. This does not raise the decompile deadline, high-pcode projection budgets or MCP client limits.";
}
private static String decompiledText(DecompileResults results) {
if (results == null) {
return null;
@@ -3716,9 +3819,23 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private static JsonObject callRelationshipCoverage(
private JsonObject callRelationshipCoverage(
CallBoundaryObservation calls, Integer missingCallers) {
JsonObject coverage = completeResolvedCallCoverage();
JsonObject discovery = new JsonObject();
if (missingCallers != null) {
JsonObject callers = new JsonObject();
callers.addProperty("status", "unknown");
callers.addProperty("reason", "Recorded incoming references do not establish additional computed callers elsewhere in the program.");
discovery.add("callers", callers);
}
if (calls != null && !calls.computedCallSites().isEmpty()) {
JsonObject callees = new JsonObject();
callees.addProperty("status", "unknown");
callees.addProperty("reason", computedCallTargetLimitation(calls.computedCallSites()));
discovery.add("callees", callees);
}
if (!discovery.isEmpty()) coverage.add("discovery", discovery);
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
JsonArray directions = new JsonArray();
if (missingCallers != null && missingCallers > 0) directions.add("callers");
@@ -3733,20 +3850,49 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (calls == null && missingCallers == null) directions.add("callers");
directions.add("callees");
coverage.add("limited_directions", directions);
return coverage;
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
}
JsonArray limitations = new JsonArray();
if (callerLimitation != null) limitations.add(callerLimitation);
if (omitted != null && omitted > 0) directions.add("callees");
if (omitted != null && omitted > 0) {
limitations.add("Ghidra's callee inventory omits targets of " + omitted +
" call reference(s) or decoded call(s) without destinations.");
directions.add("callees");
}
if (!limitations.isEmpty()) {
coverage.addProperty("status", "partial");
coverage.add("limitations", limitations);
coverage.add("limited_directions", directions);
}
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
}
private JsonObject qualifyProjectCallCoverage(
JsonObject coverage, CallBoundaryObservation calls, Integer missingCallers) {
if (!existingProject) return coverage;
String reason = "The selected project's prior analysis completeness is unknown; " +
"the recorded call relationships may omit undiscovered callers or callees.";
String status = coverage.get("status").getAsString();
// A single-direction read with known omissions is still partial; an
// unknown project history adds a limitation rather than erasing them.
if (status.equals("partial") && (calls == null || missingCallers == null)) {
coverage.getAsJsonArray("limitations").add(reason);
return coverage;
}
if (status.equals("unknown")) reason += " " + coverage.get("reason").getAsString();
if (status.equals("partial")) {
for (JsonElement limitation : coverage.getAsJsonArray("limitations"))
reason += " " + limitation.getAsString();
}
JsonArray directions = new JsonArray();
if (missingCallers != null || (calls == null && missingCallers == null))
directions.add("callers");
if (calls != null || (calls == null && missingCallers == null))
directions.add("callees");
coverage.addProperty("status", "unknown");
coverage.addProperty("reason", reason);
coverage.remove("limitations");
coverage.add("limited_directions", directions);
return coverage;
}
@@ -3761,6 +3907,24 @@ public final class ReaGhidraBridge extends HeadlessScript {
return coverage;
}
/** Name a few sites inline; reference reads and call-target queries recover the rest. */
private static final int NAMED_COMPUTED_CALL_SITES = 4;
/**
* A computed call's ReferenceManager targets are static candidates: Ghidra
* records no proof that other values cannot reach the call, so the callee
* set stays incomplete while the recorded candidates remain observed.
*/
private static String computedCallTargetLimitation(List<String> sites) {
if (sites.isEmpty()) return null;
int named = Math.min(sites.size(), NAMED_COMPUTED_CALL_SITES);
String listed = String.join(", ", sites.subList(0, named)) +
(sites.size() > named ? " and " + (sites.size() - named) + " more" : "");
return "Ghidra records call reference targets at " + sites.size() +
" computed call site(s) (" + listed + "); these static references do not establish" +
" every possible target, so additional callees remain unknown.";
}
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
if (!reference.getReferenceType().isCall()) return false;
Function caller = functionAtOrContaining(reference.getFromAddress());
@@ -3795,9 +3959,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
monitor.checkCancelled();
calls.observe(reference);
}
List<String> computedCallSites = new ArrayList<>();
for (Instruction instruction : instructions) {
boundary.observe(instruction);
if (!instruction.getFlowType().isCall()) continue;
FlowType flow = instruction.getFlowType();
if (!flow.isCall()) continue;
boolean destination = false;
for (Reference reference : instruction.getReferencesFrom()) {
if (!reference.getReferenceType().isCall()) continue;
@@ -3809,9 +3975,13 @@ public final class ReaGhidraBridge extends HeadlessScript {
calls.unresolved(instruction.getAddress(),
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
}
// The same decoded flow classifies resolve_native_call_targets'
// mechanism; its recorded targets are candidates, not a closed set.
else if (flow.isComputed()) computedCallSites.add(canonicalAddress(instruction.getAddress()));
}
return new CallBoundaryObservation(boundary.values(),
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls(),
List.copyOf(computedCallSites));
}
private final class FunctionBoundaryLimitations {
@@ -5055,7 +5225,28 @@ public final class ReaGhidraBridge extends HeadlessScript {
Set<String> keys = new HashSet<>(REQUEST_KEYS);
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_timeout_seconds"))
keys.add("decompile_timeout_seconds");
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("value_flow_limits"))
keys.add("value_flow_limits");
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_max_payload_mib"))
keys.add("decompile_max_payload_mib");
JsonObject object = requireObject(parsed, keys);
Integer payloadMiB = null;
if (object.has("decompile_max_payload_mib") && !object.get("decompile_max_payload_mib").isJsonNull()) {
payloadMiB = requireInteger(object, "decompile_max_payload_mib");
if (payloadMiB < 1 || payloadMiB > MAX_DECOMPILE_PAYLOAD_MIB)
throw new IllegalArgumentException("Invalid per-request native decompiler payload limit");
}
ValueFlowLimits flowLimits = defaultValueFlowLimits();
if (object.has("value_flow_limits")) {
JsonObject limits = requireObject(object.get("value_flow_limits"),
Set.of("max_operations", "max_inputs_per_operation", "max_edges"));
int operations = requireInteger(limits, "max_operations");
int inputs = requireInteger(limits, "max_inputs_per_operation");
int edges = requireInteger(limits, "max_edges");
if (operations < 1 || inputs < 1 || edges < 1)
throw new IllegalArgumentException("Invalid per-request value-flow limits");
flowLimits = new ValueFlowLimits(operations, inputs, edges);
}
Integer timeout = null;
if (object.has("decompile_timeout_seconds")) {
JsonElement value = object.get("decompile_timeout_seconds");
@@ -5071,7 +5262,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (params == null || !params.isJsonObject()) {
throw new IllegalArgumentException("Bridge request parameters are invalid");
}
return new Request(id, requireString(object, "method"), params.getAsJsonObject(), timeout);
return new Request(id, requireString(object, "method"), params.getAsJsonObject(), timeout, payloadMiB, flowLimits);
}
private static JsonObject requireObject(JsonElement element, Set<String> keys) {
@@ -5257,7 +5448,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {}
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls, List<String> computedCallSites) {}
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
private record SwiftLiteralLayout(
String representation,
@@ -5279,7 +5470,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
int decompileTimeoutSeconds,
boolean existingProject
) {}
private record Request(int id, String method, JsonObject params, Integer decompileTimeoutSeconds) {}
private static ValueFlowLimits defaultValueFlowLimits() {
return new ValueFlowLimits(DEFAULT_HIGH_PCODE_OPS, DEFAULT_HIGH_PCODE_INPUTS_PER_OP, DEFAULT_HIGH_PCODE_DEF_USE_EDGES);
}
private record ValueFlowLimits(int maxOperations, int maxInputsPerOperation, int maxEdges) {}
private record Request(int id, String method, JsonObject params, Integer decompileTimeoutSeconds, Integer decompileMaxPayloadMiB, ValueFlowLimits valueFlowLimits) {}
private static final class RequestFailure extends RuntimeException {
private final String code;
@@ -37,7 +37,7 @@ import org.xml.sax.InputSource;
/**
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
* program's own default prototype, as its {@link Policy} row and docs/ghidra-riscv.md state. The
* program's own default prototype, as its {@link Policy} row and docs/providers/ghidra-riscv.md state. The
* script installs the prototype as a specification extension and assigns it, with dynamic
* storage, to every non-thunk function, external function and function definition.
*/
+2 -1
View File
@@ -1399,7 +1399,8 @@ def _serve_connection(connection):
break
except (ConnectionError, OSError):
# A lost client cannot confirm document closure or revoke this run.
pass
# Keep the listener available for the owner to confirm cleanup.
should_stop = False
finally:
try:
file.close()
+13 -8
View File
@@ -18,7 +18,6 @@ from mitmproxy import ctx, version
from mitmproxy.io import tnetstring
PROFILE = "12.2.3"
OUTPUT_BYTES = 96 * 1024 * 1024
CREDENTIALS = {b"authorization", b"proxy-authorization", b"cookie", b"set-cookie"}
@@ -204,7 +203,7 @@ def without_userinfo(value, base_scheme=None):
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
def project_record(state, secrets):
def project_record(state, secrets, max_depth):
binaries, numbers, redactions = [], [], []
byte_secrets = []
for secret in secrets:
@@ -251,8 +250,8 @@ def project_record(state, secrets):
del redactions[before[2]:]
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if depth > max_depth:
raise CaptureFailure("input-limit", f"Native capture exceeds max_nesting_depth ({max_depth} levels).", path)
if credential:
validate_omitted(value, path, depth)
redactions.append({"pointer": path, "reason": "transport-credential"})
@@ -337,12 +336,12 @@ def decode(request):
except CaptureFailure:
raise
except RecursionError:
raise CaptureFailure("input-limit", "Native parser exceeded its recursion budget at byte offset " + str(start) + ".") from None
raise CaptureFailure("unsupported", "Native parser reached Python's recursion limit at byte offset " + str(start) + ". max_nesting_depth does not raise that upstream limit.") from None
except (ValueError, IndexError, UnicodeError, EOFError):
raise CaptureFailure("format", "Malformed native record at byte offset " + str(start) + ".") from None
if not isinstance(state, dict):
raise CaptureFailure("format", "Native flow record must be a dictionary at byte offset " + str(start) + ".")
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"], request["budgets"]["depth"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
return {"decoder": {"id": "mitmproxy-native-tnetstring", "name": "REA offline native mitmproxy adapter", "version": PROFILE}, "container": {"reported": None, "numeric_literals": [], "redactions": [], "records_pointer": None}, "total_records": len(records), "records": records}
@@ -350,6 +349,8 @@ def failure_reply(error):
"""Keep producer constraints, resource exhaustion and adapter defects distinct."""
if isinstance(error, CaptureFailure):
return {"ok": False, "reason": error.reason, "message": str(error), "pointer": error.pointer}
if isinstance(error, RecursionError):
return {"ok": False, "reason": "unsupported", "message": "Native projection reached Python's recursion limit; max_nesting_depth does not raise that upstream limit.", "pointer": ""}
if isinstance(error, MemoryError):
return {"ok": False, "reason": "resource-limit", "message": "Native decoder exhausted its 768 MiB address-space budget.", "pointer": ""}
return {"ok": False, "reason": "decoder", "message": "Offline native decoder failed unexpectedly: " + type(error).__name__ + ".", "pointer": ""}
@@ -362,9 +363,13 @@ class OfflineCapture:
def running(self):
resource.setrlimit(resource.RLIMIT_AS, (768 * 1024 * 1024, 768 * 1024 * 1024))
resource.setrlimit(resource.RLIMIT_CPU, (30, 30))
resource.setrlimit(resource.RLIMIT_FSIZE, (OUTPUT_BYTES, OUTPUT_BYTES))
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
request = json.loads(Path(ctx.options.rea_request_path).read_text(encoding="utf-8"))
budgets = request["budgets"]
if any(type(budgets[key]) is not int or budgets[key] < 1 for key in ("inputBytes", "outputBytes", "depth")):
raise ValueError("Invalid admitted capture budgets")
output_bytes = budgets["outputBytes"]
resource.setrlimit(resource.RLIMIT_FSIZE, (output_bytes, output_bytes))
try:
response = {"ok": True, "value": decode(request)}
except Exception as error:
@@ -374,7 +379,7 @@ class OfflineCapture:
output = json.dumps(response, ensure_ascii=True, allow_nan=False, separators=(",", ":")).encode("ascii")
except Exception as error:
output = json.dumps(failure_reply(error), separators=(",", ":")).encode("ascii")
if len(output) > OUTPUT_BYTES:
if len(output) > output_bytes:
output = b'{"ok":false,"reason":"limit","message":"Native reply exceeds the complete-evidence output budget.","pointer":""}'
Path(request["reply_path"]).write_bytes(output)
finally:
+4 -4
View File
@@ -36,7 +36,7 @@ def location(value, scheme=None):
state = {"response": {"headers": [[b"Location", value]]}}
if scheme is not None:
state["request"] = {"scheme": scheme}
record = capture.project_record(state, [])
record = capture.project_record(state, [], 64)
sidecar = next(
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
)
@@ -75,11 +75,11 @@ class TransportUserinfoTests(unittest.TestCase):
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
with self.subTest(value=value):
record = capture.project_record({"request": {"path": value}}, [])
record = capture.project_record({"request": {"path": value}}, [], 64)
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
self.assertEqual(record["binary_fields"][0]["state"], "retained")
self.assertEqual(record["redactions"], [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [], 64)
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
@@ -110,7 +110,7 @@ class TransportUserinfoTests(unittest.TestCase):
"request": {"scheme": b"https"},
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
},
}, [])
}, [], 64)
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
self.assertEqual(record["redactions"], [])
+13
View File
@@ -24,3 +24,16 @@ func captureAccessibilityValue(_ source: Any?) -> AccessibilityValue {
}
return AccessibilityValue(value: NSNull(), gap: "AXValue type \(type(of: source)) is not a supported JSON scalar")
}
/** Keep an unavailable action list distinct from a successful empty list. */
func captureActionNames(status: Int32, success: Int32, names: [String]?) -> (names: [String], error: Int32?) {
guard status == success else { return ([], status) }
return (names ?? [], nil)
}
/** AX absence is distinct from a failed attribute read. */
func captureAttributeValue<Value>(status: Int32, success: Int32, absent: [Int32], value: Value?) -> (value: Value?, error: Int32?) {
if status == success { return (value, nil) }
if absent.contains(status) { return (nil, nil) }
return (nil, status)
}
+45 -20
View File
@@ -18,9 +18,10 @@ struct Request: Decodable {
struct Action: Decodable { var kind: String; var path: [Int]?; var direction: String?; var text: String? }
struct BoundaryFailure: Error { var code: String; var message: String }
func fail(_ code: String, _ message: String) throws -> Never { throw BoundaryFailure(code: code, message: message) }
func attribute(_ element: AXUIElement, _ key: String) -> CFTypeRef? {
func attribute(_ element: AXUIElement, _ key: String) -> (value: CFTypeRef?, error: Int32?) {
var value: CFTypeRef?
return AXUIElementCopyAttributeValue(element, key as CFString, &value) == .success ? value : nil
let status = AXUIElementCopyAttributeValue(element, key as CFString, &value)
return captureAttributeValue(status: status.rawValue, success: AXError.success.rawValue, absent: [AXError.noValue.rawValue, AXError.attributeUnsupported.rawValue], value: value)
}
func childCount(_ element: AXUIElement) -> ChildCount {
var count = 0
@@ -34,13 +35,15 @@ func children(_ element: AXUIElement, count: Int) -> ChildBatch<AXUIElement> {
return value as? [AXUIElement]
}
}
func text(_ element: AXUIElement, _ key: String) -> Any { (attribute(element, key) as? String) ?? NSNull() as Any }
func windowBounds(_ element: AXUIElement) -> CGRect? {
guard let p = attribute(element, kAXPositionAttribute), let s = attribute(element, kAXSizeAttribute),
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return nil }
var point = CGPoint.zero; var size = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &size) else { return nil }
return CGRect(origin: point, size: size)
func windowBounds(_ element: AXUIElement) -> (bounds: CGRect?, error: Int32?) {
let position = attribute(element, kAXPositionAttribute)
let size = attribute(element, kAXSizeAttribute)
if let error = position.error ?? size.error { return (nil, error) }
guard let p = position.value, let s = size.value,
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return (nil, nil) }
var point = CGPoint.zero; var dimensions = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &dimensions) else { return (nil, nil) }
return (CGRect(origin: point, size: dimensions), nil)
}
func observe(_ request: Request) async throws -> [String: Any] {
guard let app = NSRunningApplication(processIdentifier: request.pid) else {
@@ -84,11 +87,22 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard AXIsProcessTrusted() else { try fail("accessibility-denied", "Grant Accessibility to the REA host in System Settings > Privacy & Security, then retry") }
let application = AXUIElementCreateApplication(request.pid)
AXUIElementSetMessagingTimeout(application, 2)
guard let axWindows = attribute(application, kAXWindowsAttribute) as? [AXUIElement] else {
try fail("accessibility-unavailable", "Selected application does not expose accessibility windows")
let windowsAttribute = attribute(application, kAXWindowsAttribute)
if let error = windowsAttribute.error {
try fail("accessibility-unavailable", "Cannot read accessibility windows: AXError \(error)")
}
let matches = axWindows.filter { windowBounds($0) == bounds }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(axWindows.map { String(describing: windowBounds($0)) })") }
guard let axWindows = windowsAttribute.value as? [AXUIElement] else {
try fail("accessibility-unavailable", "Accessibility windows are unavailable for the selected application")
}
let candidates = axWindows.map { (element: $0, observation: windowBounds($0)) }
if let error = candidates.compactMap({ $0.observation.error }).first {
try fail("accessibility-unavailable", "Cannot read accessibility window geometry: AXError \(error)")
}
guard candidates.allSatisfy({ $0.observation.bounds != nil }) else {
try fail("accessibility-unavailable", "Accessibility window geometry is unavailable; the selected window cannot be identified")
}
let matches = candidates.filter { $0.observation.bounds == bounds }.map { $0.element }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(candidates.map { String(describing: $0.observation.bounds) })") }
selected = matches[0]
}
if let action = request.action, let root = selected {
@@ -99,13 +113,16 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard let available = count.value else {
try fail("element-count-unavailable", "Cannot validate selected accessibility path because child count failed with AXError \(count.error ?? -1)")
}
guard index >= 0 && index < available,
AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild) == .success,
let item = (selectedChild as? [AXUIElement])?.first else { try fail("element-missing", "Selected accessibility path no longer exists") }
guard index >= 0 && index < available else { try fail("element-missing", "Selected accessibility path no longer exists") }
let childStatus = AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild)
guard childStatus == .success else { try fail("accessibility-unavailable", "Cannot read selected accessibility child: AXError \(childStatus.rawValue)") }
guard let item = (selectedChild as? [AXUIElement])?.first else { try fail("accessibility-unavailable", "Selected accessibility child was not returned") }
element = item
}
var owner: pid_t = 0
guard AXUIElementGetPid(element, &owner) == .success && owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
let ownerStatus = AXUIElementGetPid(element, &owner)
guard ownerStatus == .success else { try fail("accessibility-unavailable", "Cannot identify accessibility element owner: AXError \(ownerStatus.rawValue)") }
guard owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
let outcome: AXError
switch action.kind {
case "click": outcome = AXUIElementPerformAction(element, kAXPressAction as CFString)
@@ -122,10 +139,18 @@ func observe(_ request: Request) async throws -> [String: Any] {
if nodes.count >= request.max_nodes { truncated = true; break }
let childCountResult = childCount(element)
var actions: CFArray?
AXUIElementCopyActionNames(element, &actions)
let observedValue = captureAccessibilityValue(attribute(element, kAXValueAttribute))
let actionStatus = AXUIElementCopyActionNames(element, &actions)
let observedActions = captureActionNames(status: actionStatus.rawValue, success: AXError.success.rawValue, names: actions as? [String])
if let error = observedActions.error { gaps.append("AX actions unavailable at path \(path): AXError \(error)"); truncated = true }
let role = attribute(element, kAXRoleAttribute)
let title = attribute(element, kAXTitleAttribute)
let value = attribute(element, kAXValueAttribute)
for (name, error) in [("role", role.error), ("title", title.error), ("value", value.error)] {
if let error { gaps.append("AX \(name) unavailable at path \(path): AXError \(error)"); truncated = true }
}
let observedValue = captureAccessibilityValue(value.value)
if let gap = observedValue.gap { gaps.append("\(gap) at path \(path)"); truncated = true }
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": observedValue.value, "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
nodes.append(["path": path, "role": (role.value as? String) ?? NSNull() as Any, "title": (title.value as? String) ?? NSNull() as Any, "value": observedValue.value, "actions": observedActions.names, "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
guard let totalChildren = childCountResult.value else {
truncated = true
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
+46 -29
View File
@@ -25,24 +25,17 @@ ARGUMENT_REGISTERS = {
"x86_64": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
}
MAX_SELECTOR_BYTES = 1024
MAX_OTHER_STOPS = 256
MAX_TRACE_BYTES = 8 * 1024 * 1024
MAX_TRACE_FRAMES = 65_536
MAX_OBSERVATION_JOURNAL_BYTES = MAX_TRACE_BYTES
MAX_BREAKPOINT_LOCATION_BYTES = 8 * 1024 * 1024
MAX_TARGET_OUTPUT_BYTES = 1024 * 1024
OUTPUT_DRAIN_BYTES_PER_PASS = 1024 * 1024
OUTPUT_EOF_WAIT_SECONDS = 1.0
OUTPUT_READER_JOIN_SECONDS = 0.5
LISTENER_NAME = "rea-native-call-observation"
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames):
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames, limits):
"""Whether a complete event fits both aggregate trace budgets."""
return (
current_bytes + event_bytes <= MAX_TRACE_BYTES
and current_frames + event_frames <= MAX_TRACE_FRAMES
current_bytes + event_bytes <= limits["trace_bytes"]
and current_frames + event_frames <= limits["trace_frames"]
)
@@ -71,7 +64,7 @@ def _fits_trace_budget(value, remaining_bytes):
class _ObservationJournal:
"""Flush a bounded JSONL prefix so observations survive tracer failure."""
def __init__(self, handle, max_bytes=MAX_OBSERVATION_JOURNAL_BYTES):
def __init__(self, handle, max_bytes):
self.handle = handle
self.max_bytes = max_bytes
self.bytes = 0
@@ -243,6 +236,20 @@ def _hex(value):
return "0x%x" % value
def _source_location(address):
"""Preserve available line-entry facets without acquiring source files."""
entry = address.GetLineEntry()
if not entry.IsValid():
return None
file_spec = entry.GetFileSpec()
file_path = str(file_spec) if file_spec.IsValid() else None
line = entry.GetLine() or None
column = entry.GetColumn() or None
if not file_path and line is None and column is None:
return None
return {"file_path": file_path or None, "line": line, "column": column}
def _location(address, target):
"""Load and file address, module and symbol of one code address."""
module = address.GetModule()
@@ -256,6 +263,7 @@ def _location(address, target):
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
"symbol": symbol.GetName() if symbol.IsValid() else None,
"source_location": _source_location(address),
}
return location
@@ -356,7 +364,7 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
error = lldb.SBError()
text = process.ReadCStringFromMemory(
values[names[1]],
MAX_SELECTOR_BYTES,
config["capture_limits"]["selector_bytes"],
error,
)
selector = text if error.Success() and text else None
@@ -389,10 +397,16 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
_TRACE_CONTEXT = None
def _resource_limit(context, limit):
context["resource_limit"] = limit
return "resource-limit"
def _handle_breakpoint(context, frame, index):
"""Read one hit while LLDB's breakpoint callback owns the stopped frame."""
if not frame.IsValid() or frame.GetPC() == lldb.LLDB_INVALID_ADDRESS:
raise RuntimeError("LLDB breakpoint callback did not supply a readable frame")
limits = context["config"]["capture_limits"]
thread = frame.GetThread()
events = context["events"]
location = _location(frame.GetPCAddress(), context["target"])
@@ -409,49 +423,50 @@ def _handle_breakpoint(context, frame, index):
context["retained_frames"],
0,
event_frame_count,
limits,
):
return "resource-limit"
return _resource_limit(context, "trace_frames")
try:
remaining_bytes = MAX_TRACE_BYTES - context["retained_trace_bytes"]
remaining_bytes = limits["trace_bytes"] - context["retained_trace_bytes"]
record = _record(
context, thread, frame, index, len(events), location, remaining_bytes
)
except ValueError:
return "resource-limit"
return _resource_limit(context, "trace_bytes")
if record is not None:
if not _fits_trace_budget(record, remaining_bytes):
return "resource-limit"
return _resource_limit(context, "trace_bytes")
record_bytes = len(json.dumps(record).encode("utf-8"))
if context["retained_trace_bytes"] + record_bytes > MAX_TRACE_BYTES:
return "resource-limit"
if context["retained_trace_bytes"] + record_bytes > limits["trace_bytes"]:
return _resource_limit(context, "trace_bytes")
for depth in range(1, frame_count + 1):
try:
location = _location(
thread.GetFrameAtIndex(depth).GetPCAddress(), context["target"]
)
except ValueError:
return "resource-limit"
return _resource_limit(context, "trace_bytes")
if (
_estimated_json_size(location) + record_bytes + 1
> MAX_TRACE_BYTES - context["retained_trace_bytes"]
> limits["trace_bytes"] - context["retained_trace_bytes"]
):
return "resource-limit"
return _resource_limit(context, "trace_bytes")
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if record["backtrace"] else 0)
if (
context["retained_trace_bytes"]
+ record_bytes
+ addition
> MAX_TRACE_BYTES
> limits["trace_bytes"]
):
return "resource-limit"
return _resource_limit(context, "trace_bytes")
record["backtrace"].append(location)
record_bytes += addition
journal = context.get("observation_journal")
if journal is not None and not journal.append(
{"kind": "event", "event": record}
):
return "resource-limit"
return _resource_limit(context, "trace_bytes")
events.append(record)
context["retained_frames"] += event_frame_count
context["retained_trace_bytes"] += record_bytes
@@ -483,9 +498,9 @@ def _handle_stop(context, other_stops):
if journal is not None and not journal.append(
{"kind": "other-stop", "reason": stop_description}
):
return "resource-limit"
return _resource_limit(context, "trace_bytes")
other_stops.append(stop_description)
if len(other_stops) >= MAX_OTHER_STOPS:
if len(other_stops) >= context["config"]["capture_limits"]["other_stops"]:
return "stop-limit"
return None
@@ -505,7 +520,7 @@ def _breakpoint_report(target, created, config):
continue
count += 1
remaining_bytes = (
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
config["capture_limits"]["breakpoint_location_bytes"] - retained_location_bytes
)
if not _fits_trace_budget(location, remaining_bytes):
locations_truncated = True
@@ -636,7 +651,7 @@ def trace(debugger, config):
for breakpoint in breakpoints:
breakpoint_index[breakpoint.GetID()] = index
listener = lldb.SBListener(LISTENER_NAME)
output_budget = config.get("max_output_bytes", MAX_TARGET_OUTPUT_BYTES)
output_budget = config["capture_limits"]["output_bytes"]
stdout = _OutputPipe(
config["stdout_path"], config["stdout_capture_path"], output_budget
)
@@ -656,7 +671,8 @@ def trace(debugger, config):
observation_path = config.get("observation_path")
if observation_path:
observation_journal = _ObservationJournal(
open(observation_path, "wb")
open(observation_path, "wb"),
config["capture_limits"]["trace_bytes"],
)
process = target.Launch(_launch_info(target, config, listener), error)
process_created = process.IsValid()
@@ -847,6 +863,7 @@ def trace(debugger, config):
"breakpoint_locations_truncated": breakpoint_locations_truncated,
"events": events,
"resource_limit_reached": outcome == "resource-limit",
"resource_limit": context.get("resource_limit"),
"target_output": {
"stdout_bytes": stdout.bytes,
"stderr_bytes": stderr.bytes,
+126 -41
View File
@@ -25,6 +25,45 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
def address(valid=True, file_valid=True, line=23, column=7):
class SourceFileSpec:
def IsValid(self):
return file_valid
def __str__(self):
return "/missing/é space/fixture.pas"
file_spec = SourceFileSpec()
entry = types.SimpleNamespace(
IsValid=lambda: valid,
GetFileSpec=lambda: file_spec,
GetLine=lambda: line,
GetColumn=lambda: column,
)
return types.SimpleNamespace(GetLineEntry=lambda: entry)
self.assertEqual(
tracer._source_location(address()),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(column=0)),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
)
self.assertEqual(
tracer._source_location(address(file_valid=False)),
{"file_path": None, "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(line=0)),
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
)
self.assertIsNone(tracer._source_location(address(valid=False)))
self.assertIsNone(
tracer._source_location(address(file_valid=False, line=0, column=0))
)
def test_invalid_launch_process_reports_explicit_no_creation(self):
class InvalidProcess:
def IsValid(self):
@@ -82,6 +121,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
result = tracer.trace(Debugger(), {
"executable": "fixture", "architecture": "arm64", "expected_sha256": "expected",
"breakpoints": [], "working_directory": "fixture/absent",
"capture_limits": {"output_bytes": 1024},
"stdout_path": "stdout", "stdout_capture_path": "stdout-capture",
"stderr_path": "stderr", "stderr_capture_path": "stderr-capture",
})
@@ -281,37 +321,44 @@ class NativeTraceBudgetTests(unittest.TestCase):
return super().flush()
stream = CountingStream()
journal = tracer._ObservationJournal(stream)
journal = tracer._ObservationJournal(stream, max_bytes=4096)
journal.append({"kind": "event", "event": {"sequence": 0}})
journal.append({"kind": "other-stop", "reason": "signal"})
self.assertEqual((stream.writes, stream.flushes), (2, 2))
self.assertEqual(journal.bytes, len(stream.getvalue()))
def test_aggregate_trace_budget_admits_exact_boundary_and_rejects_next_record(self):
self.assertTrue(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
10,
2,
)
)
self.assertFalse(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
11,
2,
)
)
self.assertFalse(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
10,
3,
)
def test_aggregate_trace_budget_uses_the_selected_admission(self):
limits = {"trace_bytes": 100, "trace_frames": 10}
self.assertTrue(tracer._can_retain_trace(90, 8, 10, 2, limits))
self.assertFalse(tracer._can_retain_trace(90, 8, 11, 2, limits))
self.assertFalse(tracer._can_retain_trace(90, 8, 10, 3, limits))
raised = {"trace_bytes": 101, "trace_frames": 11}
self.assertTrue(tracer._can_retain_trace(90, 8, 11, 3, raised))
def test_breakpoint_reports_exact_exhausted_budget_and_accepts_more_room(self):
frame = types.SimpleNamespace(
IsValid=lambda: True, GetPC=lambda: 1,
GetThread=lambda: types.SimpleNamespace(GetNumFrames=lambda: 1),
GetPCAddress=lambda: object(),
)
context = {
"target": object(), "events": [], "retained_frames": 1,
"retained_trace_bytes": 0,
"config": {"breakpoints": [{}], "backtrace_frames": 0, "max_events": 10,
"capture_limits": {"trace_bytes": 4096, "trace_frames": 1}},
}
with patch.object(tracer, "_location", return_value={"symbol": "main"}), patch.object(
tracer, "_wanted", return_value=True
), patch.object(tracer, "_record", return_value={"backtrace": []}):
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
self.assertEqual(context["resource_limit"], "trace_frames")
context["config"]["capture_limits"] = {"trace_bytes": 8, "trace_frames": 2}
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
self.assertEqual(context["resource_limit"], "trace_bytes")
self.assertEqual(context["events"], [])
context["config"]["capture_limits"]["trace_bytes"] = 4096
self.assertIsNone(tracer._handle_breakpoint(context, frame, 0))
self.assertEqual(context["events"], [{"backtrace": []}])
def test_oversized_producer_string_is_rejected_before_serialization(self):
self.assertTrue(tracer._fits_trace_budget("method", 1024))
@@ -390,6 +437,9 @@ class NativeTraceBudgetTests(unittest.TestCase):
return FileSpec()
class Address:
def GetLineEntry(self):
return types.SimpleNamespace(IsValid=lambda: False)
def GetModule(self):
return Module()
@@ -417,6 +467,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
object(),
[[Breakpoint()]],
{
"capture_limits": {"breakpoint_location_bytes": 4096},
"breakpoints": [
{
"kind": "objc-method",
@@ -451,7 +502,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
side_effect=lambda address, _target: {"address": address, "symbol": None},
), patch.object(tracer, "_wanted", return_value=True):
report, locations_truncated = tracer._breakpoint_report(
object(), [[Breakpoint()]], {"breakpoints": [{}]}
object(), [[Breakpoint()]], {"breakpoints": [{}], "capture_limits": {"breakpoint_location_bytes": 4096}}
)
self.assertEqual(report[0]["location_count"], 65)
@@ -491,7 +542,24 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetFileSpec(self):
return FileSpec()
class SourceFileSpec:
def IsValid(self):
return True
def __str__(self):
return source_path
class Address:
def GetLineEntry(self):
if source_path is None:
return types.SimpleNamespace(IsValid=lambda: False)
return types.SimpleNamespace(
IsValid=lambda: True,
GetFileSpec=SourceFileSpec,
GetLine=lambda: 15,
GetColumn=lambda: 3,
)
def GetModule(self):
return Module()
@@ -515,22 +583,39 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetLocationAtIndex(self, index):
return Location()
breakpoint_count = 1000
breakpoint_count = 30
created = [[Breakpoint()] for _ in range(breakpoint_count)]
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count,
"capture_limits": {"breakpoint_location_bytes": 512 * 1024}}
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
with self.subTest(source_path_present=source_path is not None):
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(
sum(row["location_count"] for row in report), 1920
)
self.assertLess(
sum(len(row["locations"]) for row in report), 1920
)
self.assertLessEqual(
retained_bytes, 512 * 1024
)
if source_path is not None:
self.assertTrue(
any(
location["source_location"]["file_path"] == source_path
for row in report
for location in row["locations"]
)
)
if __name__ == "__main__":
+7 -5
View File
@@ -78,14 +78,9 @@ public final class NativeAotExtension {
new FrozenObjectAnnotator(program, manager).analyze(located.directory(), scan.getPointerLocations(), monitor, log);
}
monitor.checkCancelled();
report.addProperty("method_tables", manager.getMethodTableCount());
JsonObject coverage = auditFrozenObjects(program, scan.getPointerLocations(), monitor);
report.add("coverage", coverage);
if (coverage.get("frozen_objects_annotated").getAsInt() != coverage.get("frozen_object_candidates").getAsInt())
diagnostics.add("Some frozen-object candidates lack committed instance annotations; inspect coverage counts and Ghidra diagnostics.");
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
JsonArray tables = new JsonArray();
for (var mt : manager.getMethodTables()) {
monitor.checkCancelled();
@@ -94,6 +89,13 @@ public final class NativeAotExtension {
identity.addProperty("type", mt.getMTType().getPathName());
tables.add(identity);
}
// Publish recovery facts only after every fallible recovery and audit step.
// A failed transaction is rolled back, so it cannot report committed metadata.
report.addProperty("method_tables", manager.getMethodTableCount());
report.add("coverage", coverage);
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
report.add("types", tables);
commit = true;
return finish(log);
+3 -3
View File
@@ -5,7 +5,6 @@ from pathlib import Path
import resource
import gdb
OUTPUT_BYTES = 64 * 1024 * 1024
class ContextFailure(Exception):
@@ -79,6 +78,7 @@ def failure_exit(request_path, status, marker):
def main():
request_path = os.environ["REA_CORE_CONTEXT_REQUEST"]
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
output_bytes = request["producer_limits"]["output_bytes"]
reserve = bytearray(1024 * 1024)
try:
reply = {"ok": True, "value": inspect(request)}
@@ -94,8 +94,8 @@ def main():
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
try:
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
if len(encoded) > OUTPUT_BYTES:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds the 64 MiB reply budget."}).encode("utf-8")
if len(encoded) > output_bytes:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds producer_limits.output_bytes=" + str(output_bytes) + "."}).encode("utf-8")
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
+7 -6
View File
@@ -14,7 +14,7 @@ BRIDGE = Path(__file__).with_name("core_context.py")
class CoreContextTests(unittest.TestCase):
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable"):
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable", output_bytes=16384):
commands, reads, queries = [], [], []
state = {"executable": None, "symbols": None, "connection": None}
gdb = types.ModuleType("gdb")
@@ -63,7 +63,7 @@ class CoreContextTests(unittest.TestCase):
plugin.write_text("")
request = stage / "request.json"
reply = stage / "reply.json"
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply)}))
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply), "producer_limits": {"output_bytes": output_bytes}}))
with patch.dict(sys.modules, modules), patch.dict(os.environ, REA_CORE_CONTEXT_REQUEST=str(request)):
runpy.run_path(str(BRIDGE))
result = json.loads(reply.read_text())
@@ -104,14 +104,15 @@ class CoreContextTests(unittest.TestCase):
self.assertEqual(reads, [])
self.assertEqual(queries, [])
def test_output_limit_message_separates_the_words_and_names_the_limit(self):
result, _, reads, queries = self.inspect(objfile="m" * (64 * 1024 * 1024))
def test_selected_output_limit_reports_recovery_and_accepts_more_headroom(self):
result, _, reads, queries = self.inspect(objfile="m" * 2048, output_bytes=1024)
self.assertFalse(result["ok"])
self.assertEqual(result["reason"], "output-limit")
self.assertNotIn("exceeds64", result["message"])
self.assertEqual(result["message"], "Complete debugger context exceeds the 64 MiB reply budget.")
self.assertIn("producer_limits.output_bytes=1024", result["message"])
self.assertEqual(reads, [])
self.assertEqual(queries, ["maps"])
recovered, _, _, _ = self.inspect(objfile="m" * 2048, output_bytes=4096)
self.assertTrue(recovered["ok"], recovered)
def test_wrong_connection_or_remaining_symbols_never_queries_mappings(self):
for options in [{"connection": "native"}, {"symbols": "unexpected-symbols"}]:
+12 -9
View File
@@ -5,14 +5,12 @@ from pathlib import Path
import resource
import sys
OUTPUT_BYTES = 64 * 1024 * 1024
def lower_resource_limits():
def lower_resource_limits(policy):
limits = {}
for name, kind, maximum in (
("address_space_bytes", resource.RLIMIT_AS, 3 * 1024**3),
("cpu_seconds", resource.RLIMIT_CPU, 30),
("file_size_bytes", resource.RLIMIT_FSIZE, OUTPUT_BYTES),
("address_space_bytes", resource.RLIMIT_AS, policy["address_space_bytes"]),
("cpu_seconds", resource.RLIMIT_CPU, policy["cpu_seconds"]),
("file_size_bytes", resource.RLIMIT_FSIZE, policy["file_bytes"]),
):
soft, hard = resource.getrlimit(kind)
finite = [maximum] + [value for value in (soft, hard) if value != resource.RLIM_INFINITY]
@@ -25,6 +23,11 @@ def lower_resource_limits():
def main(request_path, inspect, failure_type, profile):
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
policy = request["producer_limits"]
for key in ("input_bytes", "output_bytes", "diagnostic_bytes", "address_space_bytes", "file_bytes", "cpu_seconds"):
value = policy[key]
if type(value) is not int or not 1 <= value <= 9007199254740991:
raise ValueError("producer_limits." + key + " must be a positive safe integer")
os.environ["PWNLIB_NOTERM"] = "1"
os.environ["PWNLIB_CACHE_DIR"] = str(Path(request_path).parent / "cache")
limits = None
@@ -32,7 +35,7 @@ def main(request_path, inspect, failure_type, profile):
# The outer exit-status contract also covers allocation failure during serialization.
memory_reserve = bytearray(1024 * 1024)
try:
limits = lower_resource_limits()
limits = lower_resource_limits(policy)
descriptor = os.open(Path(request_path).parent / "limits.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump(limits, handle)
@@ -51,8 +54,8 @@ def main(request_path, inspect, failure_type, profile):
else:
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
if len(encoded) > OUTPUT_BYTES:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds the 64 MiB reply budget."}).encode("utf-8")
if len(encoded) > policy["output_bytes"]:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds producer_limits.output_bytes=" + str(policy["output_bytes"]) + "."}).encode("utf-8")
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
+88 -117
View File
@@ -1,4 +1,18 @@
import { defineConfig } from "vitepress";
import { dirname, relative, resolve, sep } from "node:path";
import { fileURLToPath } from "node:url";
import {
documentationRewrites,
documentationSections,
} from "./documentation.js";
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
const rewrites: Record<string, string> = {
...documentationRewrites,
"README.md": "documentation.md",
"specs/README.md": "specs/index.md",
"adr/README.md": "adr/index.md",
};
export default defineConfig({
title: "REA",
@@ -6,135 +20,92 @@ export default defineConfig({
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
lang: "en-US",
base: "/rea/",
rewrites,
markdown: {
config(md) {
// Repository links resolve from their physical source. Render them from
// the stable public route instead, including links across categories.
md.core.ruler.after("inline", "documentation-source-links", (state) => {
const physicalPath =
state.env.realPath ??
(state.env.relativePath
? resolve(documentationRoot, state.env.relativePath)
: null);
if (physicalPath === null) return;
const renderedPath =
state.env.path ??
resolve(
documentationRoot,
rewrites[state.env.relativePath] ?? state.env.relativePath,
);
for (const block of state.tokens) {
for (const token of block.children ?? []) {
const attribute =
token.type === "link_open"
? "href"
: token.type === "image"
? "src"
: null;
if (attribute === null) continue;
const destination = token.attrGet(attribute);
if (
!destination ||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
)
continue;
const separator = destination.search(/[?#]/u);
const path =
separator === -1 ? destination : destination.slice(0, separator);
const suffix = separator === -1 ? "" : destination.slice(separator);
const source = relative(
documentationRoot,
resolve(dirname(physicalPath), path),
)
.split(sep)
.join("/");
const rendered = relative(
dirname(renderedPath),
resolve(
documentationRoot,
source.startsWith("public/")
? source.slice("public/".length)
: (rewrites[source] ?? source),
),
)
.split(sep)
.join("/");
token.attrSet(attribute, rendered + suffix);
}
}
});
},
},
srcExclude: ["readme/**"],
themeConfig: {
nav: [
{ text: "Get started", link: "/installation" },
{ text: "Documentation", link: "/documentation" },
{ text: "Reference", link: "/mcp-contracts" },
{ text: "Roadmap", link: "/roadmap" },
],
sidebar: [
{
text: "Getting started",
text: "Documentation directory",
items: [
{ text: "Installation and setup", link: "/installation" },
{ text: "Client configuration", link: "/client-configuration" },
{ text: "CLI and Evidence", link: "/cli" },
{ text: "Agent prompts", link: "/mcp-prompts" },
{ text: "Roadmap", link: "/roadmap" },
],
},
{
text: "Investigation guides",
items: [
{ text: "Native binaries", link: "/native-investigation" },
{ text: "Go build metadata", link: "/go-binaries" },
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
{ text: "Model artifacts", link: "/model-artifacts" },
{ text: "Rust dependencies", link: "/rust-binaries" },
{ text: "PE certificates", link: "/pe-certificates" },
{ text: "PE imports", link: "/pe-imports" },
{ text: "PE TLS/load config", link: "/pe-load-config" },
{ text: "ELF notes", link: "/elf-notes" },
{ text: "Binary hardening", link: "/binary-hardening" },
{ text: "Apple IPS crash reports", link: "/apple-ips" },
{ text: "Linux kernel modules", link: "/kernel-modules" },
{ text: "CPython pyc files", link: "/python-pyc" },
{ text: "Erlang BEAM", link: "/erlang-beam" },
{
text: "JavaScript and Electron",
link: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript workflows",
link: "/javascript-application-workflows",
},
{ text: "Browser observation", link: "/browser-observation" },
{
text: "Web network captures",
link: "/web-network-captures",
},
{
text: "Build RL environments",
link: "/agent-environments-quickstart",
},
{ text: "Environment reference", link: "/agent-environments" },
{ text: "Electron observation", link: "/electron-observation" },
{ text: "Electron fuses", link: "/electron-fuses" },
{ text: "Process capture", link: "/process-capture" },
{
text: "Desktop application capture",
link: "/desktop-application-capture",
},
{
text: "Desktop application state",
link: "/desktop-application-state",
},
{
text: "Desktop application export",
link: "/desktop-application-export",
},
{
text: "Desktop application tasks",
link: "/desktop-application-tasks",
},
{
text: "Desktop build computer",
link: "/desktop-build-computer",
},
{
text: "Building and witnessing",
link: "/desktop-build-witness",
},
{ text: ".NET assemblies", link: "/managed-code-analysis" },
{
text: "Python packaged applications",
link: "/python-packaged-applications",
},
{ text: "Android applications", link: "/android-analysis" },
{ text: "Apple applications", link: "/apple-application-analysis" },
{ text: "Desktop installations", link: "/desktop-applications" },
{ text: "Firmware", link: "/firmware-analysis" },
{ text: "iNES containers", link: "/ines-containers" },
{ text: "SQLite databases", link: "/sqlite-databases" },
{ text: "Packet captures", link: "/packet-captures" },
{ text: "Signal recordings", link: "/signal-recordings" },
],
},
{
text: "Provider guides",
collapsed: true,
items: [
{ text: "IDA", link: "/ida-provider" },
{ text: "Kuna", link: "/kuna-provider" },
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
{ text: "Ghidra raw binary", link: "/raw-binary" },
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
{ text: "Ghidra GameCube DOL", link: "/ghidra-dol" },
{ text: "Ghidra semantics", link: "/ghidra-provider" },
],
},
{
text: "Reference and development",
collapsed: true,
items: [
{ text: "MCP contracts", link: "/mcp-contracts" },
{ text: "Tool design", link: "/tool-design" },
{
text: "Admission limits and recovery",
link: "/design/limits-and-recovery",
},
{ text: "Testing", link: "/testing" },
{ text: "Client verification", link: "/client-verification" },
{
text: "Reconstruction readiness",
link: "/reconstruction-readiness",
},
{ text: "Releasing", link: "/releasing" },
{ text: "All guides", link: "/documentation" },
{ text: "Authoring conventions", link: "/development/documentation" },
{ text: "Feature specifications", link: "/specs/" },
{ text: "Architecture decisions", link: "/adr/" },
],
},
...documentationSections.map((section) => ({
text: section.text,
collapsed: section.text !== "Getting started",
items: section.items.map((item) => ({
text: item.text,
link: item.route,
})),
})),
],
search: { provider: "local" },
socialLinks: [
+684
View File
@@ -0,0 +1,684 @@
// Authored guide ownership: physical sources and stable public routes.
export const documentationSections = [
{
text: "Getting started",
items: [
{
text: "Installation and setup",
source: "getting-started/installation.md",
route: "/installation",
},
{
text: "Client configuration and troubleshooting",
source: "getting-started/client-configuration.md",
route: "/client-configuration",
},
{
text: "CLI and Evidence",
source: "getting-started/cli.md",
route: "/cli",
},
{
text: "Guided MCP prompts and completion",
source: "getting-started/mcp-prompts.md",
route: "/mcp-prompts",
},
],
},
{
text: "Native binaries",
items: [
{
text: "Native binary investigation",
source: "guides/native/native-investigation.md",
route: "/native-investigation",
},
{
text: "Go binary build metadata",
source: "guides/native/go-binaries.md",
route: "/go-binaries",
},
{
text: "Offline binary diagnostics",
source: "guides/native/binary-diagnostics.md",
route: "/binary-diagnostics",
},
{
text: "ELF and Mach-O initializer functions",
source: "guides/native/native-initializers.md",
route: "/native-initializers",
},
{
text: "ELF unwind function ranges",
source: "guides/native/elf-unwind-ranges.md",
route: "/elf-unwind-ranges",
},
{
text: "Linux kernel build config",
source: "guides/native/kernel-build-config.md",
route: "/kernel-build-config",
},
{
text: "Recorded crash evidence",
source: "guides/native/recorded-crashes.md",
route: "/recorded-crashes",
},
{
text: "Windows Shell Link shortcuts",
source: "guides/native/windows-shortcut.md",
route: "/windows-shortcut",
},
],
},
{
text: "JavaScript and Electron",
items: [
{
text: "JavaScript Application Graph",
source: "guides/javascript/javascript-application-graph.md",
route: "/javascript-application-graph",
},
{
text: "Cross-layer JavaScript application workflows",
source: "guides/javascript/javascript-application-workflows.md",
route: "/javascript-application-workflows",
},
{
text: "JavaScript artifact reconstruction",
source: "guides/javascript/javascript-artifact-reconstruction.md",
route: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript source recovery",
source: "guides/javascript/javascript-recovery.md",
route: "/javascript-recovery",
},
{
text: "Passive Node and Electron runtime observation",
source: "guides/javascript/javascript-runtime-observation.md",
route: "/javascript-runtime-observation",
},
{
text: "JavaScript static/runtime reconciliation",
source: "guides/javascript/javascript-runtime-reconciliation.md",
route: "/javascript-runtime-reconciliation",
},
{
text: "Electron file-page observation",
source: "guides/javascript/electron-observation.md",
route: "/electron-observation",
},
{
text: "V8 code cache inspection",
source: "guides/javascript/v8-code-cache.md",
route: "/v8-code-cache",
},
],
},
{
text: "Browser and captured websites",
items: [
{
text: "Website observation with CDP",
source: "guides/web/browser-observation.md",
route: "/browser-observation",
},
{
text: "Browser scenario contract",
source: "guides/web/browser-scenario-contract.md",
route: "/browser-scenario-contract",
},
{
text: "Website runtime source attribution",
source: "guides/web/web-runtime.md",
route: "/web-runtime",
},
{
text: "Historical web network captures",
source: "guides/web/web-network-captures.md",
route: "/web-network-captures",
},
{
text: "HTTP/3 capture lab",
source: "http3-capture-lab.md",
route: "/http3-capture-lab",
},
{
text: "Captured website source locations",
source: "guides/web/web-source-location.md",
route: "/web-source-location",
},
{
text: "Captured website module relationships",
source: "guides/web/website-module-trace.md",
route: "/website-module-trace",
},
{
text: "Captured website scripts and static JavaScript analysis",
source: "guides/web/website-script-export.md",
route: "/website-script-export",
},
],
},
{
text: "Android applications",
items: [
{
text: "Static Android APK analysis",
source: "guides/android/android-analysis.md",
route: "/android-analysis",
},
{
text: "Apktool resource decoding",
source: "guides/android/apktool-resource-analysis.md",
route: "/apktool-resource-analysis",
},
{
text: "ADB device inspection and acquisition",
source: "guides/android/adb-device-analysis.md",
route: "/adb-device-analysis",
},
{
text: "Flutter build identification",
source: "guides/android/flutter-analysis.md",
route: "/flutter-analysis",
},
],
},
{
text: "Apple applications",
items: [
{
text: "Apple Application Inventory Projection",
source: "guides/apple/apple-application-analysis.md",
route: "/apple-application-analysis",
},
],
},
{
text: ".NET assemblies",
items: [
{
text: "Managed-code analysis",
source: "guides/managed/managed-code-analysis.md",
route: "/managed-code-analysis",
},
],
},
{
text: "Firmware",
items: [
{
text: "Firmware analysis",
source: "guides/firmware/firmware-analysis.md",
route: "/firmware-analysis",
},
],
},
{
text: "HarmonyOS packages",
items: [
{
text: "Static HarmonyOS package inventory",
source: "guides/harmony/harmony-analysis.md",
route: "/harmony-analysis",
},
],
},
{
text: "WebAssembly",
items: [
{
text: "Offline WebAssembly artifacts",
source: "guides/wasm/wasm-artifacts.md",
route: "/wasm-artifacts",
},
],
},
{
text: "EVM contracts",
items: [
{
text: "Offline EVM bytecode",
source: "guides/evm/evm-bytecode.md",
route: "/evm-bytecode",
},
],
},
{
text: "Databases and recorded packets",
items: [
{
text: "SQLite database snapshots",
source: "guides/data/sqlite-databases.md",
route: "/sqlite-databases",
},
{
text: "Recorded packet capture inspection",
source: "guides/data/packet-captures.md",
route: "/packet-captures",
},
{
text: "Packet capture summaries",
source: "guides/data/pcap-analysis.md",
route: "/pcap-analysis",
},
],
},
{
text: "Process capture and instrumentation",
items: [
{
text: "Process capture",
source: "guides/runtime/process-capture.md",
route: "/process-capture",
},
{
text: "Frida instrumentation",
source: "guides/runtime/frida-instrumentation.md",
route: "/frida-instrumentation",
},
],
},
{
text: "Provider setup and formats",
items: [
{
text: "Ghidra provider semantics",
source: "providers/ghidra-provider.md",
route: "/ghidra-provider",
},
{
text: "DOS MZ and COM analysis with Ghidra",
source: "providers/ghidra-dos.md",
route: "/ghidra-dos",
},
{
text: "Optional NativeAOT metadata recovery",
source: "providers/ghidra-nativeaot.md",
route: "/ghidra-nativeaot",
},
{
text: "MIPS ELF analysis through Ghidra",
source: "providers/ghidra-mips.md",
route: "/ghidra-mips",
},
{
text: "PSP static ELF analysis through Ghidra",
source: "providers/ghidra-psp.md",
route: "/ghidra-psp",
},
{
text: "RISC-V ELF analysis through Ghidra",
source: "providers/ghidra-riscv.md",
route: "/ghidra-riscv",
},
{
text: "Windows Ghidra P0",
source: "providers/windows-ghidra-p0.md",
route: "/windows-ghidra-p0",
},
{
text: "Headerless raw-binary analysis with Ghidra",
source: "providers/raw-binary.md",
route: "/raw-binary",
},
{
text: "IDA provider",
source: "providers/ida-provider.md",
route: "/ida-provider",
},
{
text: "JEB instrumentation and analysis",
source: "providers/jeb-analysis.md",
route: "/jeb-analysis",
},
],
},
{
text: "Contracts and evidence",
items: [
{
text: "MCP runtime contracts",
source: "reference/mcp-contracts.md",
route: "/mcp-contracts",
},
{
text: "Reconstruction obligation ledgers",
source: "reference/reconstruction-obligation-ledgers.md",
route: "/reconstruction-obligation-ledgers",
},
],
},
{
text: "Development and verification",
items: [
{
text: "Settings and recovery",
source: "design/limits-and-recovery.md",
route: "/design/limits-and-recovery",
},
{
text: "MCP tool design",
source: "development/tool-design.md",
route: "/tool-design",
},
{
text: "Testing REA",
source: "development/testing.md",
route: "/testing",
},
{
text: "Client and agent verification",
source: "development/client-verification.md",
route: "/client-verification",
},
{
text: "Reconstruction readiness conformance",
source: "development/reconstruction-readiness.md",
route: "/reconstruction-readiness",
},
{
text: "Releasing REA",
source: "development/releasing.md",
route: "/releasing",
},
{
text: "REA roadmap",
source: "development/roadmap.md",
route: "/roadmap",
},
],
},
{
text: "Release migrations",
items: [
{
text: "Upgrading from REA 6.1 to 6.2",
source: "migrations/migration-6.2.md",
route: "/migration-6.2",
},
],
},
{
text: "Signal recordings",
items: [
{
text: "Recorded signal inspection",
source: "guides/signals/signal-recordings.md",
route: "/signal-recordings",
},
],
},
{
text: "Additional guides",
items: [
{
text: "Offline API contract inventory",
source: "api-contracts.md",
route: "/api-contracts",
},
{
text: "Bun standalone executables",
source: "bun-standalone.md",
route: "/bun-standalone",
},
{
text: "Deno compile executables",
source: "deno-standalone.md",
route: "/deno-standalone",
},
{
text: "Capability composition and ownership",
source: "capability-composition.md",
route: "/capability-composition",
},
{
text: ".NET single-file bundles",
source: "dotnet-bundles.md",
route: "/dotnet-bundles",
},
{
text: "ESP flash images",
source: "esp-flash-image.md",
route: "/esp-flash-image",
},
{
text: "U-Boot FIT images",
source: "fit-images.md",
route: "/fit-images",
},
{
text: "Measure MCP catalog costs in Codex",
source: "mcp-catalog-evaluation.md",
route: "/mcp-catalog-evaluation",
},
{
text: "Offline Playwright traces",
source: "playwright-traces.md",
route: "/playwright-traces",
},
{
text: "Testing lanes and provider verification",
source: "testing-lanes.md",
route: "/testing-lanes",
},
{
text: "UF2 and DfuSe images",
source: "uf2-dfuse.md",
route: "/uf2-dfuse",
},
{
text: "WeChat mini-program wxapkg archives",
source: "wxapkg.md",
route: "/wxapkg",
},
{
text: "Binary hardening",
source: "binary-hardening.md",
route: "/binary-hardening",
},
{
text: "Desktop application capture",
source: "desktop-application-capture.md",
route: "/desktop-application-capture",
},
{
text: "Desktop application export",
source: "desktop-application-export.md",
route: "/desktop-application-export",
},
{
text: "Desktop application state",
source: "desktop-application-state.md",
route: "/desktop-application-state",
},
{
text: "Desktop application tasks",
source: "desktop-application-tasks.md",
route: "/desktop-application-tasks",
},
{
text: "Desktop build computer",
source: "desktop-build-computer.md",
route: "/desktop-build-computer",
},
{
text: "Desktop build witness",
source: "desktop-build-witness.md",
route: "/desktop-build-witness",
},
{
text: "iNES containers",
source: "ines-containers.md",
route: "/ines-containers",
},
{
text: "Build RL environments with REA",
source: "agent-environments-quickstart.md",
route: "/agent-environments-quickstart",
},
{
text: "Agent environments from recorded captures",
source: "agent-environments.md",
route: "/agent-environments",
},
{
text: "APK signature schemes",
source: "apk-signature.md",
route: "/apk-signature",
},
{
text: "Apple IPS crash reports",
source: "apple-ips.md",
route: "/apple-ips",
},
{
text: "Static analysis in GitHub Actions",
source: "ci-analysis.md",
route: "/ci-analysis",
},
{
text: "Desktop installation directories",
source: "desktop-applications.md",
route: "/desktop-applications",
},
{
text: "Flattened devicetree inspection",
source: "devicetree.md",
route: "/devicetree",
},
{
text: "Electron fuse state",
source: "electron-fuses.md",
route: "/electron-fuses",
},
{ text: "ELF note tables", source: "elf-notes.md", route: "/elf-notes" },
{
text: "Erlang BEAM files",
source: "erlang-beam.md",
route: "/erlang-beam",
},
{ text: "ESP images", source: "esp-image.md", route: "/esp-image" },
{
text: "GameCube DOL analysis through Ghidra",
source: "ghidra-dol.md",
route: "/ghidra-dol",
},
{
text: "PowerPC ELF analysis through Ghidra",
source: "ghidra-ppc.md",
route: "/ghidra-ppc",
},
{
text: "Godot PCK archives",
source: "godot-pck.md",
route: "/godot-pck",
},
{
text: "Static CUDA and HIP code objects",
source: "gpu-code-objects.md",
route: "/gpu-code-objects",
},
{
text: "Intel HEX and S-record images",
source: "hex-records.md",
route: "/hex-records",
},
{
text: "IL2CPP metadata",
source: "il2cpp-metadata.md",
route: "/il2cpp-metadata",
},
{
text: "Static JavaScript route inventory",
source: "javascript-route-inventory.md",
route: "/javascript-route-inventory",
},
{
text: "Linux kernel modules",
source: "kernel-modules.md",
route: "/kernel-modules",
},
{
text: "Kuna provider",
source: "kuna-provider.md",
route: "/kuna-provider",
},
{
text: "Model file declarations",
source: "model-artifacts.md",
route: "/model-artifacts",
},
{
text: "Match native functions across builds",
source: "native-function-matching.md",
route: "/native-function-matching",
},
{
text: "Node.js single executable applications",
source: "node-sea.md",
route: "/node-sea",
},
{
text: "PE Authenticode certificate tables",
source: "pe-certificates.md",
route: "/pe-certificates",
},
{
text: "PE import and delay-load tables",
source: "pe-imports.md",
route: "/pe-imports",
},
{
text: "PE TLS callbacks and load configuration",
source: "pe-load-config.md",
route: "/pe-load-config",
},
{
text: "Python packaged applications",
source: "python-packaged-applications.md",
route: "/python-packaged-applications",
},
{
text: "CPython pyc inspection",
source: "python-pyc.md",
route: "/python-pyc",
},
{
text: "Reconstruction coverage",
source: "reconstruction-coverage.md",
route: "/reconstruction-coverage",
},
{
text: "Reverse-engineering tool integrations",
source: "reverse-engineering-tools.md",
route: "/reverse-engineering-tools",
},
{
text: "Rust binary dependency metadata",
source: "rust-binaries.md",
route: "/rust-binaries",
},
{
text: "U-Boot legacy uImage containers",
source: "uimage.md",
route: "/uimage",
},
{
text: "UnityFS AssetBundle archives",
source: "unityfs.md",
route: "/unityfs",
},
],
},
];
export const documentationRewrites = Object.fromEntries(
documentationSections.flatMap((section) =>
section.items.flatMap((item) => {
const target = `${item.route.slice(1)}.md`;
return item.source === target ? [] : [[item.source, target]];
}),
),
);
+205
View File
@@ -0,0 +1,205 @@
# Documentation directory
REA documentation is organized by reader task and document owner. This directory
links every authored guide; the [site](https://rea.tools/) also serves separate
product pages. The VitePress documentation source is this `docs/` tree.
## Getting started
- [Installation and setup](getting-started/installation.md)
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
- [CLI and Evidence](getting-started/cli.md)
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
## Native binaries
- [Native binary investigation](guides/native/native-investigation.md)
- [Go binary build metadata](guides/native/go-binaries.md)
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
- [ELF and Mach-O initializer functions](guides/native/native-initializers.md)
- [ELF unwind function ranges](guides/native/elf-unwind-ranges.md)
- [Linux kernel build config](guides/native/kernel-build-config.md)
- [Recorded crash evidence](guides/native/recorded-crashes.md)
- [Windows Shell Link shortcuts](guides/native/windows-shortcut.md)
## JavaScript and Electron
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
- [Electron file-page observation](guides/javascript/electron-observation.md)
- [V8 code cache inspection](guides/javascript/v8-code-cache.md)
## Browser and captured websites
- [Website observation with CDP](guides/web/browser-observation.md)
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
- [Website runtime source attribution](guides/web/web-runtime.md)
- [Historical web network captures](guides/web/web-network-captures.md)
- [HTTP/3 capture lab](http3-capture-lab.md)
- [Captured website source locations](guides/web/web-source-location.md)
- [Captured website module relationships](guides/web/website-module-trace.md)
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
## Android applications
- [Static Android APK analysis](guides/android/android-analysis.md)
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
- [Flutter build identification](guides/android/flutter-analysis.md)
## Apple applications
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
## .NET assemblies
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
## Firmware
- [Firmware analysis](guides/firmware/firmware-analysis.md)
## HarmonyOS packages
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
## WebAssembly
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
## EVM contracts
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
## Databases and recorded packets
- [SQLite database snapshots](guides/data/sqlite-databases.md)
- [Recorded packet capture inspection](guides/data/packet-captures.md)
- [Packet capture summaries](guides/data/pcap-analysis.md)
## Process capture and instrumentation
- [Process capture](guides/runtime/process-capture.md)
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
## Signal recordings
- [Recorded signal inspection](guides/signals/signal-recordings.md)
## Provider setup and formats
- [Ghidra provider semantics](providers/ghidra-provider.md)
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
- [IDA provider](providers/ida-provider.md)
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
## Contracts and evidence
- [MCP runtime contracts](reference/mcp-contracts.md)
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
## Development and verification
- [Settings and recovery](design/limits-and-recovery.md)
- [Flutter budget calibration](verification/flutter-budget-calibration.md)
- [Source-map budget calibration](verification/source-map-budget-calibration.md)
- [Source-location delivery qualification](verification/source-location-delivery.md)
- [JavaScript precision calibration](verification/javascript-precision-calibration.md)
- [MCP tool design](development/tool-design.md)
- [Testing REA](development/testing.md)
- [Client and agent verification](development/client-verification.md)
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
- [Releasing REA](development/releasing.md)
- [REA roadmap](development/roadmap.md)
## Release migrations
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
## Architecture and authoring
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
- [Documentation ownership and contribution model](development/documentation.md)
- [Feature specifications](specs/README.md)
- [Architectural decision records](adr/README.md)
## Machine-owned documentation
| Path | Owner and update command |
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
| `assets/` | Shared authored images referenced by guides |
Do not edit generated catalogs or evidence to change product facts. Change their
canonical contracts or producer and regenerate. Historical release records retain
the exact commit messages and references from the release they describe.
## Additional guides
- [Offline API contract inventory](api-contracts.md)
- [Bun standalone executables](bun-standalone.md)
- [Deno compile executables](deno-standalone.md)
- [Capability composition and ownership](capability-composition.md)
- [.NET single-file bundles](dotnet-bundles.md)
- [ESP flash images](esp-flash-image.md)
- [U-Boot FIT images](fit-images.md)
- [Measure MCP catalog costs in Codex](mcp-catalog-evaluation.md)
- [Offline Playwright traces](playwright-traces.md)
- [Testing lanes and provider verification](testing-lanes.md)
- [UF2 and DfuSe images](uf2-dfuse.md)
- [WeChat mini-program wxapkg archives](wxapkg.md)
- [Binary hardening](binary-hardening.md)
- [Desktop application capture](desktop-application-capture.md)
- [Desktop application export](desktop-application-export.md)
- [Desktop application state](desktop-application-state.md)
- [Desktop application tasks](desktop-application-tasks.md)
- [Desktop build computer](desktop-build-computer.md)
- [Desktop build witness](desktop-build-witness.md)
- [iNES containers](ines-containers.md)
- [Build RL environments with REA](agent-environments-quickstart.md)
- [Agent environments from recorded captures](agent-environments.md)
- [APK signature schemes](apk-signature.md)
- [Apple IPS crash reports](apple-ips.md)
- [Static analysis in GitHub Actions](ci-analysis.md)
- [Desktop installation directories](desktop-applications.md)
- [Flattened devicetree inspection](devicetree.md)
- [Electron fuse state](electron-fuses.md)
- [ELF note tables](elf-notes.md)
- [Erlang BEAM files](erlang-beam.md)
- [ESP images](esp-image.md)
- [GameCube DOL analysis through Ghidra](ghidra-dol.md)
- [PowerPC ELF analysis through Ghidra](ghidra-ppc.md)
- [Godot PCK archives](godot-pck.md)
- [Static CUDA and HIP code objects](gpu-code-objects.md)
- [Intel HEX and S-record images](hex-records.md)
- [IL2CPP metadata](il2cpp-metadata.md)
- [Static JavaScript route inventory](javascript-route-inventory.md)
- [Linux kernel modules](kernel-modules.md)
- [Kuna provider](kuna-provider.md)
- [Model file declarations](model-artifacts.md)
- [Match native functions across builds](native-function-matching.md)
- [Node.js single executable applications](node-sea.md)
- [PE Authenticode certificate tables](pe-certificates.md)
- [PE import and delay-load tables](pe-imports.md)
- [PE TLS callbacks and load configuration](pe-load-config.md)
- [Python packaged applications](python-packaged-applications.md)
- [CPython pyc inspection](python-pyc.md)
- [Reconstruction coverage](reconstruction-coverage.md)
- [Reverse-engineering tool integrations](reverse-engineering-tools.md)
- [Rust binary dependency metadata](rust-binaries.md)
- [U-Boot legacy uImage containers](uimage.md)
- [UnityFS AssetBundle archives](unityfs.md)
+9
View File
@@ -0,0 +1,9 @@
# Architectural decision records
Store one actual decision per `docs/adr/<domain>/<decision-slug>.md`, without a
global sequence number. Use meaningful names and the Michael Nygard sections:
Status, Context, Decision and Consequences. Link each decision from its owning
feature specification.
See [the authoring guide](../development/documentation.md#feature-lifecycle) for
the distinction between specifications, roadmaps and isolated decisions.
+5 -3
View File
@@ -11,14 +11,14 @@ with your RL framework to set task goals, check rewards, and run training.
## Before you start
Install [REA and its agent integration](installation.md). To record an app,
Install [REA and its agent integration](getting-started/installation.md). To record an app,
you also need a running test instance and a Chrome-family browser. Choose an
instance where the agent is allowed to create, edit, and delete data. REA
records the app you provide; you start it and prepare its test data.
To run the exported environment, install the
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
and start Docker. REA's [compatibility check](testing.md#exported-environment-bundles)
and start Docker. REA's [compatibility check](development/testing.md#exported-environment-bundles)
uses AgentEnv version `0.9.1431` with Python 3.12.
There are two ways to build an environment: ask your agent to record the app,
@@ -52,7 +52,7 @@ work and which need more evidence. The app is used during recording; the
exported environment runs from the saved data.
After your agent exports the folder, skip to
[run the environment](#3-run-the-environment). The next section is an
[run the environment](#run-the-environment). The next section is an
alternative for an existing recording.
## 2. Or build from a saved recording
@@ -91,6 +91,8 @@ A successful build means the folder was created. Check the report to confirm
it supports the operations your tasks need. `coverage: null` means the build
could not prepare state for this environment and it will use replay.
<a id="run-the-environment"></a>
## 3. Run the environment
Whether your agent exported the folder or you built it from a HAR, run it
+23 -17
View File
@@ -30,7 +30,7 @@ rea environment-serve manifest.json --evidence capture.json --port 8765
```
The first command inspects the capture; the second projects the
[environment manifest](web-network-captures.md#environment-manifest) view from
[environment manifest](guides/web/web-network-captures.md#environment-manifest) view from
it, where `view.json` wraps the capture Evidence as the inline source:
```json
@@ -107,7 +107,10 @@ refusal names its stage: `environment-build` for the build's own checks,
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
projection, `environment-export` for the bundle. SIGINT cancels the build
(`cancelled`); before the bundle stage nothing is written, and a cancellation
during the write names the bundle it wrote. Stage progress is logged to stderr
after publication preserves the bundle receipt. Before publication, a failed
write attempts to remove partial output and reports cleanup failures. See
[bundle publication and recovery](getting-started/cli.md#export-an-environment-bundle).
Stage progress is logged to stderr
at info level; the report is the only output on stdout:
```json
@@ -163,7 +166,7 @@ at info level; the report is the only output on stdout:
`coverage` is the state's read views and write plans summarized, and `null`
when the state is unavailable and the bundle serves replay. The
[environment coverage](web-network-captures.md#environment-coverage) view
[environment coverage](guides/web/web-network-captures.md#environment-coverage) view
reports the same summary for any capture without building, so an agent can
check it after each recording. The view summarizes the state as derived; it
does not apply the serve-time admission checks, so for an environment whose
@@ -209,9 +212,9 @@ warning does not establish that these bodies were checked. A browser scenario's
saved values (`save` and `save_response`) named like credentials but not marked
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
to 20 of them by parameter pointer. Names do not establish sensitivity,
so the build never refuses on them. Capture
so the build never refuses on them.
Evidence also records the capture file's absolute path (`subject.local_path`,
Capture Evidence also records the capture file's absolute path (`subject.local_path`,
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
carries it: build from a neutral folder when that path should not ship.
@@ -235,7 +238,7 @@ name `export_environment_bundle`; the others name their stage as
## From a browser scenario
The capture can also come from REA itself: a
[browser scenario](browser-scenario-contract.md) records a scripted session
[browser scenario](guides/web/browser-scenario-contract.md) records a scripted session
in a browser REA launches, and its Evidence is a manifest parent like a HAR
inspection. Select the `network` event family and response bodies. A replayed
reply carries up to 256 KiB of a recorded body, but the scenario keeps at most
@@ -321,7 +324,7 @@ declared secret replaced by its redaction marker (a body that redaction had to
empty replays no body).
To record writes, sign in through the UI and then send the application's API
calls with [request steps](browser-scenario-contract.md#request-steps): each
calls with [request steps](guides/web/browser-scenario-contract.md#request-steps): each
sends one request to the start origin through the page's own `fetch`, with the
session's cookies, and a header can carry the CSRF cookie or stored token the
application's own requests send. A write is served only when the capture shows
@@ -332,9 +335,12 @@ scenario is written and cannot use a value an earlier response returned, so an
id must be known before the run: one the caller chooses, such as a `uid` in the
create body, or one an earlier scenario created. When the server assigns ids,
read the list after each create (the create's response or the list shows the
new entity), and record updates and deletes in a second scenario that reads
the first one's objects by id before editing them, or make them through the
UI. The environment coverage view names each
new entity). A second scenario can record updates and deletes using those
known ids, but produces a separate environment: recordings are not combined.
That scenario must include its own before/after reads and every action the
environment should support; it does not inherit the first scenario's create
evidence. To capture create and subsequent edits together, make them through
the UI in one scenario. The environment coverage view names each
write it withholds and what to record next. A scenario with write requests
changes the application's data: record against an instance whose data may
change.
@@ -481,7 +487,7 @@ captured text without changing the tool names or schema fields clients use.
Replay answers each call from one recording, so the answers reflect whichever
moment of the capture recorded them. The
[environment state](web-network-captures.md#environment-state) view extracts
[environment state](guides/web/web-network-captures.md#environment-state) view extracts
instead the data the capture's reads observed before the first related write:
entity types with an identity field, entities with their recorded values,
documents for reads that are not entities, and a render plan per read tool,
@@ -503,14 +509,14 @@ see [Serve from the state](#serve-from-the-state).
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the
environment, then to `127.0.0.1` and an ephemeral port, exactly as
[`rea mcp --http`](cli.md#serve-mcp-over-streamable-http) resolves them. The
[`rea mcp --http`](getting-started/cli.md#serve-mcp-over-streamable-http) resolves them. The
bound MCP URL is written to stderr as one JSON log line
(`"msg":"MCP transport listening"` with `url`); `REA_LOG_LEVEL` overrides that
log level. `ENVIRONMENT_NAME` renames the served environment; otherwise the
manifest's `name` is used, and a set but blank value is refused as
`configuration_invalid` before binding rather than silently falling back. The
listener serves one MCP client at a time with the same session, `Host` header
and shutdown contract as [transports](mcp-contracts.md#transports) describes.
and shutdown contract as [transports](reference/mcp-contracts.md#transports) describes.
It runs until SIGINT or SIGTERM, then closes the listener, prints one
`{"status":"stopped", …}` document whose `state` member says whether it served
from the state, and exits `130` or `143`.
@@ -585,7 +591,7 @@ names the method and path template, then what the tool returns, one level deep:
`returns a JSON object with fields …`, `returns a JSON array of objects with
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
where a list is an array of objects found by the
[environment state](web-network-captures.md#environment-state)'s list rule (the
[environment state](guides/web/web-network-captures.md#environment-state)'s list rule (the
root array, or the one array of objects an envelope carries). Field and member
names are written as JSON strings, so a captured name cannot read as more of
the description; a member name too long to show whole is a quoted prefix marked
@@ -641,7 +647,7 @@ rea environment-serve manifest.json --evidence capture.json
```
Before it binds, the server derives the
[environment state](web-network-captures.md#environment-state) the capture
[environment state](guides/web/web-network-captures.md#environment-state) the capture
observed: it runs the `environment-state` view on the capture in-process with
the manifest's `template_rules` in the same order, exactly as
`inspect-analysis-view` projects it from the same capture, so the state
@@ -733,7 +739,7 @@ query model may describe supported paging; an empty later page cannot start
returning earlier entities merely because it was empty in the capture.
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
[write plan](web-network-captures.md#environment-state) for them is served:
[write plan](guides/web/web-network-captures.md#environment-state) for them is served:
the capture's later reads confirmed the effect the plan models. Other write
tools are not served, so no call claims a change it does not make; each one's
plan states why in its `reason`. A served write changes the live state, which
@@ -1187,7 +1193,7 @@ read, not that the write had no other effect.
## Compatibility verification
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
described in [testing](testing.md#exported-environment-bundles). It projects
described in [testing](development/testing.md#exported-environment-bundles). It projects
the Grafana-shaped source-owned capture through the built CLI, packs the
checkout, exports a bundle with `--package-tarball`, runs it with the pinned
`agentenv-framework` release on the runner's Docker and requires the smoke task
+67 -4
View File
@@ -85,9 +85,49 @@ canonical hexadecimal form), `EXC_GUARD` subtypes,
pointer-authentication suffixes, and dyld missing-library strings. A report
that matches none of those rules is `unclassified`.
Opened Mach-O and dSYM UUID joins remain unobserved. Symbol names printed
inside the report are recorded claims and are not verified. Linux ELF cores
use the separate [recorded crash](recorded-crashes.md) workflow.
## Joining frames to the crashed Mach-O or dSYM
Pass `join_path` (`--join-path` in the CLI) to select one local Mach-O
executable or dSYM. UUID bytes in the report and the artifact's `LC_UUID` load
commands are compared in hexadecimal, so an uppercase `dwarfdump --uuid` value
and a lowercase report `uuid` match.
```bash
rea inspect-apple-ips /absolute/path/to/report.ips \
--join-path /absolute/path/to/App.app/Contents/MacOS/App --json
```
Shared-cache image sources `S` and `C` remain unjoined: UUID agreement does not
establish that a standalone image has the cached layout.
For a FAT container every slice is examined. An image joins only when exactly
one slice has that UUID **and** an architecture that agrees with the report's
`arch`; `arm64` and `arm64e` stay distinct, and an all-zero UUID never joins. A
UUID match with a different architecture, more than one compatible slice, or a
slice without a readable `__TEXT` segment is reported as a rejected image with
its reason and stays unjoined rather than being guessed.
An unknown UUID in another slice prevents a unique join unless its architecture
rules out a match. Missing or unrecognized architecture and unsupported CPU
subtypes remain unknown; display-name fallbacks do not establish agreement.
The parsed slice facts remain available even when no match can be derived.
`join.status` is `selected` with the artifact path, SHA-256, byte length, every
slice's offset, arch, CPU type and subtype, UUID, and `__TEXT` vmaddr, plus
`image_matches`, per-frame results, and join-specific limitations. A joined
frame reports `static_address`, the slice `__TEXT` vmaddr plus the frame's
recorded `imageOffset`, in decimal and hexadecimal. An unjoined frame reports
a reason instead; an unknown image UUID, an absent `imageIndex` row, and a
frame without `imageOffset` all stay unjoined rather than treated as aligned.
An artifact that is not a Mach-O image or FAT container reports
`join.status: "unreadable"` with a typed reason and leaves every report fact
intact. Symbol names printed inside the report stay recorded claims. Function
or symbol lookup at a joined static address, and matching dSYM symbol tables,
are not performed by this step.
Symbol names printed inside the report are recorded claims and are not
verified. Linux ELF cores use the separate
[recorded crash](guides/native/recorded-crashes.md) workflow.
## Limits and failures
@@ -120,4 +160,27 @@ success or truncated integer facts.
rea inspect-apple-ips /absolute/path/to/report.ips --max-integer-text-bytes 16777216 --json
```
This workflow does not execute a target, launch atos, or search for dSYMs.
`max_join_bytes` (`--max-join-bytes`) independently bounds the join artifact
read, defaulting to 64 MiB. This workflow does not execute a target, launch
atos, or search for dSYMs.
## Join admission and failures
The join uses the shared Mach-O container, slice, command-table and segment
readers. Slice extents and `sizeofcmds` bound every identity observation; duplicate
UUIDs or malformed commands cannot select the first plausible value. Unreadable
join results retain the successfully read artifact's path, SHA-256 and byte count.
The integer-text allowance spans the report and join, including repeated integer
facts and derived static addresses. Reservation precedes derived address
expansion. If a selected join read, interpretation or allowance fails, the error
retains the completed IPS report. Join read limits and recovery use
`max_join_bytes`, even when the report and join paths are identical. Cancellation
retains the caller's reason independently of those completed facts.
When a failure follows a completed join-file read, `partial_observation` retains
both the decoded report and a `join_artifact` receipt with the independently
observed path, SHA-256 and byte count. Frame rejection reasons reference
`join.limitations`; source-sized architecture diagnostics are retained once per
image rather than copied into every frame. Universal-image admission includes
the shared reader's supported hidden ARM64 row and rejects overlapping slices.
File renamed without changes.
+18 -3
View File
@@ -24,7 +24,7 @@ entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regi
are separate children. Source names such as `/$bunfs/root/main.js` remain in
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
indexed path and retain their source spelling or bytes. Virtual names deeper
than 256 logical path parts also use an indexed path with an explicit limitation,
than the selected `path_parts` allowance (default 256) also use an indexed path with an explicit limitation,
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
separate occurrences; ordinary extraction collision rules apply.
@@ -43,12 +43,27 @@ compatible source files; bytecode is retained as evidence, not decoded.
Module parsing reads exact metadata ranges rather than loading stored asset
bodies or whole pages around small pointers. A cache retains at most 256 KiB
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
are limited to 256 MiB, checked before allocation or I/O. The module table,
default to 256 MiB, checked before allocation or I/O. The module table,
decoded strings and combined logical path tree have separate admission checks;
at most 250,000 distinct path prefixes are admitted before inventory construction.
by default at most 250,000 distinct path prefixes are admitted before inventory construction.
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
only for the same admitted file state, with selected-path checks before publication.
Both artifact operations accept `artifact_limits.bun`. Its independent controls
are `payload_bytes`, `module_count`, `representation_bytes`, `metadata_read_bytes`,
`recognition_table_bytes`, `recognition_load_command_bytes`, `path_parts` and
`path_nodes`. Failed recognition remains explicitly limited instead of proving
absence. ELF section names are examined at their actual offsets, including names
beyond a large section-name prefix. Select `artifact_limits.expanded_bytes` to
change the aggregate referenced-plaintext work allowance.
For a reported metadata-read refusal, keep the same carrier and retry with
`--artifact-limits '{"bun":{"metadata_read_bytes":536870912}}'` or the equivalent
MCP `artifact_limits` object. Omitted fields keep defaults. The response names
the effective limit and exact repair; accepted overrides are reported separately
from caller selections. Changes apply on the next call without restarting REA.
Summary and selected extraction do not change the producer's admission.
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
Synthetic tests cover the carriers and malformed declarations. Run
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
+2 -2
View File
@@ -65,14 +65,14 @@ cancellation can still prevent upload.
that analysis was complete. A successful operation can retain partial analysis,
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
coverage, or a policy gate for those findings. Inspect the saved Evidence and
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
coverage before making such a decision. See [output and exit status](getting-started/cli.md#output-and-exit-status).
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
host/capability, or another operation failure. Keep the original JSON error (when
emitted), stderr, and exit code, and use their category/message to diagnose it.
An early startup failure may leave `analysis.json` empty; do not parse that as
successful analysis. Select a supported operation or explicitly provision its
[documented prerequisites](installation.md); do not silently switch providers,
[documented prerequisites](getting-started/installation.md); do not silently switch providers,
execute the artifact, or ignore the failure to obtain a green job.
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
+26
View File
@@ -0,0 +1,26 @@
# Deno compile executables
`inspect-artifact` inventories a `deno compile` executable. `extract-artifact` writes the embedded virtual files through the existing safe output tree. Neither operation runs the executable or the Deno runtime.
```sh
rea inspect-artifact ./app --json
rea extract-artifact ./app --json
```
In MCP, open the executable with `open_binary`, then call `inspect_artifact`. Use `detail: "summary"` for a large virtual filesystem and `inspect_analysis_view` to page retained Evidence.
The recognized carriers are a Mach-O `__SUI` section named `d3n0l4nd`, an Intel Mach-O `<~sui-data~>` trailer, an ELF `PT_NOTE` or `SHT_NOTE` owned by `SUI` with type `0x53554901` and descriptor name `d3n0l4nd`, and a PE `RT_RCDATA` resource named `D3N0L4ND`. These are structural layouts. The metadata JSON has no Deno version field, so the compiler version stays unknown unless another record establishes it. Other layouts, including older eszip trailers, are unsupported.
Root metadata reports the carrier, entry point, argv, V8 flags, permission options, unstable configuration, application name, self-extracting marker, virtual-filesystem case sensitivity, and the npm snapshot byte length. The npm snapshot is not decoded. Each virtual file is one occurrence with its logical path, size, and SHA-256. Transpiled output, CJS export analysis, and source maps are separate occurrences beside that file. Each has the same `index` as the file and its own `role`. A region normally has the logical path of the file plus a `.deno-*` suffix. When that path belongs to another stored file, the region moves to an indexed path such as `deno-vfs/0.vfs-transpiled`, so read the reported `logical_path` instead of building it from the file name. They are not linked by `parent_occurrence_id`, which follows directories and expanded containers. Symlink targets stay recorded path parts and are never followed.
`Metadata.env_vars_from_env_file` holds values from a compile-time `--env-file`, and `ca_data` holds certificate bytes. Inventory retains the variable names and values, certificate bytes, counts, and byte lengths in local Evidence, so sharing that Evidence shares them. Extraction does not write them as files.
A virtual filesystem that cannot be listed reports `vfs_listing` as `unsupported` and leaves inventory and extraction coverage unknown, because its files are not examined.
Remote modules, redirects, and specifiers contribute counts and logical paths when their stored form is unambiguous. When it is not, they stay present with a byte length and an `unsupported` member listing. A module whose specifier is missing uses an indexed `deno-remote/<index>` path, which still exposes its stored bytes, and modules with resolved specifiers keep their paths. Colons, backslashes, and control characters in a specifier are percent-encoded in the logical path, so `http://localhost:8000/mod.ts` becomes `deno-remote/localhost%3A8000/mod.ts`. Percent escapes in a logical path use uppercase hex, and a specifier that reaches a path already taken uses an indexed path. `source_path` keeps the original specifier. A remote specifier is indexed the same way when a path component exceeds 255 UTF-8 bytes or the logical path exceeds 512 UTF-8 bytes, counting the `.deno-*` suffix of its stored regions. Unsafe, deep, or over-long virtual names use `deno-vfs/<index>` under the same rule and retain the original spelling when it fits the name allowance. A descendant does not present its own name as a complete source path when an ancestor spelling was dropped.
A length that overruns the section, or a closing marker that does not match, is malformed. An opening marker that is absent is unsupported. A confirmed section that cannot be parsed fails closed and does not fall through to a plain native inventory. An Intel Mach-O sentinel inside a confirmed trailer payload is data of that payload, so an Intel Deno executable stored as a virtual file does not replace the app that stores it. Trailers side by side with different bytes fail closed. A file is confirmed only by its carrier name, so an unrelated native file with malformed tables keeps its native inventory. That inventory does not establish that the file holds no Deno section.
See [limits and recovery](design/limits-and-recovery.md) for the named allowances. Synthetic tests cover the carriers and malformed declarations. Run `npm run verify:deno:standalone` for a local Deno compile of the host and, when the compiler can produce them, Linux, Windows, and Intel macOS targets. Deno is a fixture producer only and is not installed by REA. The produced executable is not run.
Layout reference: Deno `cli/lib/standalone/binary.rs` and `cli/lib/standalone/virtual_fs.rs` at 2.9.8, with libsui 0.16.4 section placement.
+162 -24
View File
@@ -3,7 +3,7 @@
This guide explains the runtime settings design for maintainers and developers:
who owns each value, when an update takes effect, how providers enforce it, and
how to extend the system. The public tool contract is in
[MCP contracts](../mcp-contracts.md#editable-analysis-settings); recorded runtime
[MCP contracts](../reference/mcp-contracts.md#editable-analysis-settings); recorded runtime
checks and their limitations are in the
[verification report](../verification/runtime-analysis-settings.md).
@@ -172,17 +172,20 @@ checks all happen before publishing the replacement snapshot; a rejected batch
cannot partially apply. Clients coordinating writes can supply
`expected_revision`; without it, valid updates apply to the current state.
| Runtime setting | Application point |
| ------------------------------------------------------------- | ---------------------------------------------------------------------- |
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
| `jadx.operation_timeout_ms` | Next JADX operation |
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
| `wasm.command_timeout_ms` | Next WABT operation |
| `native.matching_representation_bytes` | Next `match_native_functions` request |
| Runtime setting | Application point |
| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
| `asar.max_listing_bytes` | Next expanded ASAR listing, independently of packed header bytes |
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
| `ghidra.decompile_max_payload_mib` | Next native decompile; `null` preserves the selected provider's payload default |
| `ghidra.value_flow_max_operations`, `ghidra.value_flow_max_inputs_per_operation`, `ghidra.value_flow_max_edges` | Next function value-flow projection on the existing Program |
| `jadx.operation_timeout_ms` | Next JADX operation |
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
| `wasm.command_timeout_ms` | Next WABT operation |
| `native.matching_representation_bytes` | Next `match_native_functions` request |
### Precedence and lifetime
@@ -202,6 +205,48 @@ for an unknown or omitted value elsewhere.
A write does not extend active work, relaunch a provider, clear retained
cleanup, or invalidate successful Evidence.
Expanded ASAR listing admission defaults to 64 MiB and has its own launch seed,
`REA_ASAR_MAX_LISTING_BYTES`. Raising `asar.max_header_bytes` alone cannot admit
expanded paths and entries that exceed this separate budget. The listing bound
uses exact safe-integer byte accounting; the host heap and transport constraints
remain independent. A lower runtime value is enforced on the next listing even
when the archive header was already examined.
Ghidra value-flow controls default to 3,000 operations, 64 inputs per operation,
and 12,000 edges. They accept positive Java integer counts and use the respective
`REA_GHIDRA_VALUE_FLOW_MAX_OPERATIONS`,
`REA_GHIDRA_VALUE_FLOW_MAX_INPUTS_PER_OPERATION`, and
`REA_GHIDRA_VALUE_FLOW_MAX_EDGES` launch seeds. Truncation names the exhausted
control. Completed Evidence remains accessible, but a new function dossier is
observed with the accepted controls instead of replaying an earlier projection.
The native decompiler payload control is separate from those projections and
from decompile time. Its launch seed is
`REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB`. Accepted explicit values are 1..2047 MiB:
the supported Ghidra implementation shifts that value into a signed integer byte
capacity. Omission or runtime `null` keeps the provider's own default; REA does
not guess that default from a different provider version. Changing payload
options takes effect before the next native decompile.
### Per-call resource policies and selection
An agent can also use controls declared directly by an operation. These do not
require a runtime setter or restart. Keep omitted fields omitted in Evidence;
report the resolved policy separately where needed. See the owning guides for
[Apktool string pages](../guides/android/apktool-resource-analysis.md),
[source maps](../guides/web/web-source-location.md),
[historical captures](../guides/web/web-network-captures.md),
[JavaScript precision](../guides/javascript/javascript-application-graph.md),
[managed inspection](../guides/managed/managed-code-analysis.md), and
[Frida capture](../guides/runtime/frida-instrumentation.md).
The same accepted policy must reach acquisition, parsing, worker requests,
serialization, cached-result admission and generated consumers where those
stages exist. Desktop state views and emitted AgentEnv Python share one policy;
delivery pagination never substitutes for producer selection. Capture limits
must be selected before an experiment: another experiment cannot recover bytes
that were dropped from a previous live observation.
Overrides belong to one REA runtime, survive target close/open, and disappear
when REA exits. Connections sharing that runtime share its overrides. No files,
ambient environment, registrations or upstream software are edited. Overrides
@@ -399,7 +444,7 @@ application points, and resulting behavior. Check that adapters deliver the
canonical recovery guidance, with independent assertions for its underlying
facts. Avoid freezing prose or deriving all expected values from the same
registry used by the implementation; either can hide the wrong failure.
See [test design](../testing.md#test-design-and-pruning) for the assertion policy.
See [test design](../development/testing.md#test-design-and-pruning) for the assertion policy.
Runtime checks cover atomic validation, stale writes, reset, runtime isolation,
accepted-operation snapshots, and explicit argument precedence. Public MCP
@@ -419,14 +464,107 @@ editable setting or active worker telemetry.
## Bun standalone module graphs
`MAX_BUN_MODULE_COUNT` admits at most 100,000 module records.
`MAX_BUN_RETAINED_BYTES` admits at most 256 MiB of serialized graph payload and
separately bounds retained entry metadata. Physical metadata read work is also
limited to 256 MiB per preparation, including repeated uncached ranges. A
256 KiB / 1,024-entry exact-range cache avoids full-page amplification. The
combined logical path tree admits at most 250,000 distinct prefixes; this is
checked before adding nodes, with shared prefixes counted once. The shared
artifact expansion budget also bounds the sum of referenced member lengths. Refusals name the boundary.
There is no runtime override; additional host memory or `NODE_OPTIONS` does not
raise these ceilings. Use a smaller producer fixture or a separately reviewed
reader for larger graphs. No partial extraction is claimed on admission failure.
`inspect_artifact` and `extract_artifact` accept per-call `artifact_limits`.
The `bun` group defaults to 100,000 modules, independent 256 MiB payload,
retained-representation and physical metadata-read allowances, 256 path parts
and 250,000 logical path nodes. Recognition separately admits a 1 MiB ELF
section table and 16 MiB Mach-O load commands. Exhausted recognition remains
unknown rather than proving the carrier has no Bun graph. Inventory preserves
native-root facts or another confirmed pack and reports the exact unexamined
alternative inline; it does not assert Bun presence. ELF section names
are examined with exact bounded reads, including names beyond the former
prefix window.
For example, keep the same complete carrier and retry a reported recognition
limitation with `{"artifact_limits":{"bun":{"recognition_table_bytes":2097152}}}`.
The CLI spelling is `--artifact-limits '{"bun":{"recognition_table_bytes":2097152}}'`.
No REA restart is needed. A smaller later selection is re-admitted; previously
parsed facts cannot bypass it. The 256 KiB / 1,024-entry exact-range cache
avoids full-page amplification and remains an internal optimization, not an agent
resource setting. Shared logical path prefixes count once before nodes are added.
## Artifact reader admission
The same request provides `dotnet_bundle`, `node_sea` and `godot` groups. Their
controls apply when recognition or inventory encounters that format; supplying
a policy does not assert the format is present. Unsupported keys are rejected.
Each group follows one domain-owned default policy. See [.NET bundles](../dotnet-bundles.md),
[Node SEA](../node-sea.md), [Godot PCK](../godot-pck.md) and
[Bun standalone](../bun-standalone.md) for format semantics.
`artifact_limits.expanded_bytes` selects aggregate plaintext work for ZIP, Bun,
SEA and PCK readers, including repeated readable logical occurrences. Omission
retains each archive's encoded-size-derived default. The .NET bundle group's
own `expanded_bytes` controls its independently enforced bundle expansion.
Failures preserve `details.reader_limit` with the exact parameter, effective
value, units, supported range, required value when known, next-call scope and
`can_increase`. When `can_increase` is true, raise that field and retry the same
complete artifact. At the supported maximum, or when the required representation
exceeds it, the failure is not retryable by increasing that budget; inspect the
declared dimensions and input validity, or explicitly choose a suitable reader.
Summary, pagination and
selected extraction do not bypass producing admission. Intrinsic format,
safe-integer, path validity and runtime representation constraints remain fixed.
Evidence parameters retain the exact selected object, including omission or
`{}`. When an object was selected, inventory and extraction report the resolved
`effective_limits`; summaries preserve those facts. These are accepted policies,
not process-memory telemetry or a claim that every format reader ran. Complete
source identities and graph manifests remain content-addressed independently
of delivery selection. No partial extraction is claimed on admission failure.
## Deno compile sections
`MAX_DENO_METADATA_JSON_BYTES` admits at most 8 MiB of metadata JSON.
`MAX_DENO_VFS_HEADER_JSON_BYTES` admits at most 32 MiB of virtual-filesystem
header JSON. Specifier, redirect, and remote tables each admit at most 100,000
records. Specifier text admits at most 8 MiB, and remote module bodies admit at
most 256 MiB. The virtual filesystem admits at most 100,000 directory, file,
and symlink records and at most 512 nested levels (`MAX_DENO_VFS_DEPTH`). A
name longer than 4,096 UTF-16 code units is indexed and is not retained.
`MAX_DENO_VFS_RETAINED_UNITS` admits at most 32 million UTF-16 code units across
the source path built for every entry, directories included, and retained
symlink parts. A logical path deeper than 256 parts, a name that is not a safe
relative path, a path component longer than 255 UTF-8 bytes
(`MAX_DENO_PATH_COMPONENT_BYTES`, POSIX `NAME_MAX`), or a logical path longer
than 512 UTF-8 bytes (`MAX_DENO_LOGICAL_PATH_BYTES`, half of the Darwin
`PATH_MAX`) uses an indexed path and keeps the original spelling when that
spelling is within the name allowance. Component and path lengths include the
`.deno-*` suffix of the stored regions a member has. A descendant has no source
path when an ancestor spelling was not retained. Cancellation is checked before
the virtual-filesystem walk. The walk does not yield, so it does not observe a
cancellation that arrives while it is running; the retained allowance bounds its
work. The combined logical path tree admits at most 250,000 distinct prefixes.
`MAX_DENO_RETAINED_BYTES` admits at most 256 MiB of retained header and entry
metadata. The shared artifact expansion budget also bounds the sum of referenced
member lengths, with a 256 MiB floor.
Carrier location claims a file only after it confirms the carrier name: a
Mach-O `__SUI` segment, a `SUI` ELF note descriptor named `d3n0l4nd`, a PE
`D3N0L4ND` resource, or an Intel trailer opening with `d3n0l4nd`. Before that,
malformed tables and exhausted search budgets leave the file with native
inventory, and that inventory makes no claim that the file holds no Deno
section. ELF location reads ELF and note headers, not the whole image, and does
not hash note descriptors. An ELF larger than 512 MiB is still located when
those headers fit. Program and section tables are read at their canonical record
size, so the 16-bit header count bounds each read at 65,535 records. A table
that cannot be read, or that uses extended program numbering, is skipped and
another table can still hold the note. At most 256 notes are examined
(`MAX_INSPECT_ELF_NOTE_ENTRIES`). A damaged table or the note ceiling is a
refusal only when a `SUI` note was also confirmed. Intel Mach-O trailer search
reads each scanned byte at most once and examines at most 64 `<~sui-data~>`
sentinels (`MAX_DENO_INTEL_TRAILER_SENTINELS`). The search continues past a
confirmed trailer, because an enclosing trailer can hold it as a stored file, so
reaching that allowance after a confirmation is a refusal. PE location walks the resource directories to the named payload and
does not read the image whole, so a PE of any size is located when the
`D3N0L4ND` payload is within the 512 MiB resource reader ceiling. A larger
payload, or a confirmed resource that cannot be resolved, is a named refusal.
Refusals name the boundary. There is no runtime override; additional host memory
or `NODE_OPTIONS` does not raise these ceilings. Duplicate-section verification
rejects unequal lengths before hashing, stops at the first digest mismatch,
and reads at most the carrier file size in aggregate
payload bytes. Overlapping copies that exceed that allowance are refused.
Environment-file values and CA certificate bytes are retained in local Evidence
with their counts and byte lengths. The npm snapshot length is retained and its
bytes are not decoded.
+82 -16
View File
@@ -47,6 +47,17 @@ computer would not have that entry. An `--executable` runs from
the current directory, as it would from a shell, and a relative path to it is
resolved there.
A nested desktop file uses a dashed ID: `vendor/app.desktop` has ID
`vendor-app`. REA checks path spellings that can encode that ID within each
XDG root before trying the next root. Unrelated directories do not affect
selection. If an applicable path cannot be examined or the bounded search
cannot establish absence, capture reports an incomplete desktop-entry search
rather than selecting a different application from a later root.
Desktop-entry string values retain literal trailing spaces. For example,
`Path=/tmp/workspace ` and `Path=/tmp/workspace\s` name the same directory
ending in a space. Use the escaped spelling to make such a suffix visible.
Before writing anything the command refuses to start if a window of the
application is already open, or if the window list cannot be read. A launch
beside a running instance can hand off to it, and that instance's unsaved
@@ -56,6 +67,9 @@ environment, and reads commands from stdin, one per line. The application gets
REA's environment without the `AGENTENV_*` variables an agent environment sets,
such as its control token: an app the exported computer launches never has
them, and an app captured inside a computer must not hold a credential to it.
It also gets `LIBC_FATAL_STDERR_=1`, so glibc writes a fatal error, such as
an abort's reason, to the app's stderr rather than its terminal, and an app
that exits at launch reports it.
| Line | Effect |
| ------------------------------------------- | ----------------------------------------------------------- |
@@ -90,12 +104,20 @@ Progress lines on stderr are JSON, `{"rea_progress": {"phase":
"desktop_capture", "message": ...}}`. Those a driver waits on also carry an
`event`: `{"kind": "window_open"}` once the window opens; `{"kind":
"checkpoint", "name", "trigger", "at_ms", "entries", "changes",
"retained_files", "retained_bytes"}` once a checkpoint has been taken, the
baseline, `stop` and `exit` included, with the paths walked, the paths
changed since the checkpoint before and the files retained; and `{"kind":
"checkpoint_refused", "name", "reason"}` for a requested name it refused. A
driver that sends `checkpoint <name>` waits for the event naming it rather
than for a fixed time.
"retained_files", "retained_bytes", "changed_files", "more_changed_files"}`
once a checkpoint has been taken, the baseline, `stop` and `exit` included,
with the paths walked, the paths changed since the checkpoint before, the
files retained, and up to ten of the files (not folders) that changed,
home-relative: those under a `--state-path` or absolute `--argument` first,
then those the application held open, then those in its state directories,
then the rest, and last caches (a folder whose name ends in `cache`, such as
`~/.cache`) and WebKit or Chromium storage, each by path, with a count of the
rest. An application can rewrite dozens of cache files on a save, and KDE's
holds its service cache open, so the saved document would otherwise fall past
the ten;
and `{"kind": "checkpoint_refused", "name", "reason"}` for a requested name it
refused. A driver that sends `checkpoint <name>` waits for the event naming it
rather than for a fixed time.
When the session ends the capture sends SIGTERM to the launched process, its
process group and every process that carries the run token, waits up to five
@@ -107,7 +129,11 @@ remain. Interrupting the command (Ctrl-C) ends the session the same way
without a further checkpoint; the capture is still written with end reason
`cancelled`, and the command exits with status 130. Interrupted before the
application is launched, the command writes no capture and removes the
content it had retained.
content it had retained. If cancellation arrives during publication or final
cleanup, the already-written capture can instead retain its earlier end
reason. Inspect `capture.json` and the error's `details.partial_observation`
for the retained capture Evidence. Cleanup failures also report resources
that could not be released.
If the capture fails for any other reason before `capture.json` is written,
the application is closed the same way and the content already retained is
@@ -262,8 +288,14 @@ found nowhere are listed in
`missing_libraries`, and libraries no package owns in `unpackaged_libraries`.
`library_packages_reason` says why the lists are incomplete: the executable is
not an ELF image (a launcher script, for example), `ldconfig`, `dpkg-query` or
`apt-cache` is missing, a library could not be read or names more than 512
libraries, or the walk stopped after 512 libraries.
`apt-cache` is missing, `dpkg-query -S` could not tell which package owns a
library (it exits 2 rather than 1, which only says no package owns the paths
it did not name), a library could not be read or exceeds the selected dependency
inventory budget, or the walk reached that budget. A failed owner query does
not list the libraries it covered in `unpackaged_libraries`, since a lookup
that could not finish is not proof that no package owns them. Completed batches
retain their package and unowned-library facts. Paths containing package-query
pattern characters stay unknown unless another candidate identifies an owner.
A sources file, key or library whose handle REA cannot close fails the capture
with the file as its cleanup resource. When the repository check and the
@@ -296,20 +328,29 @@ dot-folder named after the package, the executable, the desktop entry's full
id and its last part, and the entry's name, with whether each existed at the
start. A folder in those places whose name is one of these in another letter
case joins them, such as GIMP's `~/.config/GIMP` for the `gimp` package; only
the whole name is compared, so `~/.config/GIMPshop` does not. Dotfiles in the
the whole name is compared, so `~/.config/GIMPshop` does not. A file under a
folder the application creates during the capture beside one of them, with
the same name in another letter case or with a version added (`-3.1` or
`3`), as TreeLine's `~/.treeline-3.1`, is its state too. Dotfiles in the
home named after the application (`.<name>rc`, or
`.<name>` followed by `-`, `.` or `_`, for a name of three or more characters)
that exist at the start join them, since an application may only read such a
file during the capture. A `flatpak run` launch is named after its application
file during the capture. So does KDE's settings file in the config folder,
`<name>rc` such as Tellico's `~/.config/tellicorc`, for each such name of three
or more characters, whether or not it exists yet. A `flatpak run` launch is named after its application
id rather than the `flatpak` launcher, and adds `~/.var/app/<id>`.
**Session.** When it started, how long it ran and why it ended; the launched
process's exit status; every process seen in the application's tree; every
client window with its class pair, titles, owner pid and type, marked when it
belongs to the application; when the first application window appeared; the
belongs to the application. Titles and class names decode xprop's quoted
delimiters, newline and tab escapes, and three-digit octal byte escapes;
unsupported escape spellings leave that string unknown rather than inventing
text. The session also records when the first application window appeared; the
most common class pair of the application's windows, for matching its window
later; files the application's processes held open under the walked roots; the first
8 KiB of the launched process's stderr; and whether cleanup was verified. Each
later; files the application's processes held open under the walked roots; the last
8 KiB of the launched process's stderr, where a failing app's last words are,
with `stderr_truncated` when earlier output was dropped; and whether cleanup was verified. Each
process records whether the capture started it: the launched process, a
process carrying the capture's run token, or a descendant of one, even when
its environment cannot be read, as for a zombie or a sandboxed renderer.
@@ -335,7 +376,9 @@ directory, which are named in the limitations instead; a selected path that is
not a directory brings the `-wal` and `-journal` paths beside it, so a database
selected on its own is captured with its logs), the state directories
as walked (their physical spelling, which differs from the inventory's when one
is a symlink), the paths every walk left out (the output folder and
is a symlink), the selected paths (each `--state-path` and content-root
`--argument` as named, absolute, beside the physical path walked, so a
symlinked settings folder keeps what its name means), the paths every walk left out (the output folder and
`--exclude` values), and the checkpoints in order: `baseline` before
launch, each requested checkpoint, `stop` or `time-limit`, and `exit` after the
application has gone. Each checkpoint records how many paths its walk saw,
@@ -358,9 +401,17 @@ remaining content budget is not read, unless stored content of the same size
could make it a duplicate. Files that could not be
retained are listed with the reason: too large, over the content budget,
changed since the walk, changed during the read, or unreadable.
The baseline also records the paths and types of non-file entries it observed,
including symlinks, without following them. This is required to distinguish an
unreadable SQLite sidecar from an absent one. Captures made before this field
was added must be made again before state inspection, task drafting or export.
Directories the application held files open in, but whose starting content was
not retained, are listed so a second capture can pass them with
`--state-path`.
`--state-path`, or `state_paths` on a build computer's `start_build_capture`,
and a limitation names them, spelled from `~/` under the home. A lock file
the application held open, such as Poedit's `~/poedit.lock` or Lucene's
`write.lock`, makes no folder such a directory, since it holds nothing to
retain.
**Limitations.** What the capture could not establish, such as a truncated
walk or a missing window.
@@ -377,3 +428,18 @@ evidence for attribution, not proof.
Clicks and keystrokes are not recorded. Window titles are sampled every half
second, so short-lived titles can be missed.
Dependency inventory headroom is selected for each capture with
`--library-policy '{"libraries":1024,"string_bytes":8192}'`. Unset fields retain
their defaults: `libraries: 512` bounds both the closure walk and distinct
DT_NEEDED offsets in each ELF image; `search_path_entries: 16` bounds distinct
DT_RUNPATH and DT_RPATH offsets per kind; `string_bytes: 4096` includes the NUL
terminator of each decoded string; `table_bytes: 1048576` bounds each program
header or dynamic table. Values must be positive safe integers. A budget refusal
names the field that needs headroom; partial package facts and unresolved
coverage remain visible. The policy applies during this capture's admission,
without restarting REA, and explicit selections are retained in Evidence. Raising
content retention or a worker heap does not raise these inventory budgets.
Choosing a new capture still launches the selected application; changing these
read budgets does not add execution authority or recover an earlier capture's
omitted inventory.
+300 -36
View File
@@ -15,23 +15,45 @@ agent-env run ./mousepad-bundle --dry-run
The export reads the capture and its [state view](desktop-application-state.md);
it does not run the application. `--name` sets the environment name; otherwise
it comes from the desktop entry id or the program, with an `app-` prefix when
it comes from the desktop entry id or the program: a reverse-DNS id's last
part (`org.xfce.mousepad` gives `mousepad`), or else the whole id without a
trailing version (`glabels-3.0` gives `glabels`), with an `app-` prefix when
that name would not be a portable bundle name (for example all digits). The
output directory must be new or empty. `--task` adds a task an agent wrote for
the capture, once REA has validated it against the capture; see
[desktop application tasks](desktop-application-tasks.md). `--package-url`
the capture, once REA has validated it against the capture; repeat it for
several tasks, each checked at its own end checkpoint, which
`--task-end NAME=CHECKPOINT` names; see
[desktop application tasks](desktop-application-tasks.md#several-tasks). `--package-url`
says where the computer downloads the package of an application installed from
a `.deb` file; see [Applications from a .deb file](#applications-from-a-deb-file).
`--download-install FILE` says which AppImage or tarball a build computer
unpacked the application from; see
[Applications from an AppImage or tarball](#applications-from-an-appimage-or-tarball).
`--extra-package NAME`, repeatable, installs another Ubuntu 24.04 archive
package beside the application, such as a runtime piece its package does not
install; a [build computer](desktop-build-witness.md) passes its request's
`extra_packages`, or a `.deb` request's `dependencies`, this way. A name that
is not a Debian package name is refused (`extra_package`). Names the bundle
installs anyway, the application's own package and the dependencies it names
for a package installed from its `.deb` file, are not repeated; the rest are
listed in the limitations and the README. A package the captured `.deb` lists
as one alternative, such as `openjdk-17-jre | java-runtime`, is kept when apt
installs the application by name, since apt may satisfy it with another.
`--seed-checkpoint` names the checkpoint whose state seeds the bundle, the
baseline by default, and `--end-checkpoint` the one a task must pass on, the
capture's last by default; see [The seed checkpoint](#the-seed-checkpoint).
`--synthetic-seed` says the seed was made for the task, as a build agent makes
it in a [build computer](desktop-build-witness.md), rather than taken from the
user's own data; see [A seed made for the task](#a-seed-made-for-the-task).
The command returns Evidence with operation `export_desktop_application` and
predicate type `rea.desktop-application-export`, listing the environment name,
the seed and end `checkpoints`, owned paths, seed files, the seeds it
rewrote to remove the capture host's name and user (`scrubbed_files`), the
bundle files that still hold them (`unscrubbed_files`), and written files,
the last three by their path in the bundle. A failed export, cancellation
gzip seeds it compressed again only to clear their headers
(`recompressed_files`), the bundle files that still hold them
(`unscrubbed_files`), and written files, the last four by their path in the
bundle. A failed export, cancellation
included, removes the bundle it had not yet published, so no partial copy of the
user's data remains. If the bundle was published but its output handles could
not be closed, the command reports a cleanup error that carries the export
@@ -39,17 +61,19 @@ Evidence; the bundle itself is complete.
## The bundle
| Path | Contents |
| -------------------------------- | -------------------------------------------------------------------------------------------------- |
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
| `tasks/smoke.json` | A task that deploys the app |
| `tasks/<task>.json` | With `--task`: the agent's task, its verifier in `artifacts/` and the eval in `evals/` |
| `README.md` | What was exported, from which capture, and what to keep in mind |
| Path | Contents |
| --------------------------------- | -------------------------------------------------------------------------------------------------- |
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
| `envs/<name>/install-download.sh` | With `--download-install`: the run script that unpacks the downloaded AppImage or tarball |
| `envs/<name>/applications/` | With a tarball's `--download-install`: the desktop entry REA wrote for its program |
| `tasks/smoke.json` | A task that deploys the app |
| `tasks/<task>.json` | With `--task`: each of the agent's tasks, its verifier in `artifacts/` and the eval in `evals/` |
| `README.md` | What was exported, from which capture, and what to keep in mind |
The class file is `<name>_app.py` with hyphens as underscores, for example
`mousepad_app.py`: agentenv imports it under its stem, so a plain `<name>.py`
@@ -59,9 +83,10 @@ an `App` suffix.
The class declares:
- `Packages(apt=[...])` with the package the capture found. A vendor package
is named `<package>/<release>`, its repository's release (the Suite apt
reports as `a=`, or else its Codename), so apt takes it from that repository
- `Packages(apt=[...])` with the package the capture found, then the
`--extra-package` names. A vendor package is named `<package>/<release>`,
its repository's release (the Suite apt reports as `a=`, or else its
Codename), so apt takes it from that repository
even when the Ubuntu archive offers a higher version or epoch, as noble's
transitional `firefox` does over packages.mozilla.org's; it is named alone
when the release names neither, or when the capture was written before
@@ -101,7 +126,8 @@ The class declares:
reason (a capture
written before restrictions were recorded counts as unrestricted); when
another usable repository lists the same version it is left out instead,
with a limitation. The export also refuses snaps, flatpaks, AppImages, and a
with a limitation. The export also refuses snaps, flatpaks, an AppImage or
tarball not named with `--download-install`, and a
package no recorded repository lists unless it was installed from a `.deb`
file the capture recorded and `--package-url` is given, since the computer
could not install them by name, and a capture written before credentials
@@ -124,6 +150,17 @@ The class declares:
desktop entry without `Path`), which a relative argument or program path
such as `./app` also needs, and still starts if that folder is missing. The
export refuses a capture that saw no application window.
- The seed's document, opened at launch, so the app shows it rather than its
start screen: when the seed holds exactly one file the state view gives the
`document` role, and the desktop entry's `Exec` takes files (a `%f`, `%F`,
`%u` or `%U` field code), its path under `$HOME` is appended to the command.
A command that already names a document, absolute or from its working
directory, as one captured with `--argument <file>` does, is kept as it is;
a settings path under the home does not count. A seed with no document launches
as captured. With several, no desktop entry or an `Exec` that takes no file,
the app launches as captured and the README and the result's limitations say
why. A reset
relaunches the app, so it opens the document again.
- For an Electron application (the capture found its fuse sentinel or a
`resources/app.asar`), `--no-sandbox` right after the program unless the
command already has it: Chromium's sandbox needs user namespaces the
@@ -131,7 +168,8 @@ The class declares:
README and the result's limitations say the sandbox is off.
- `paths`: the state directories holding the application's state files and the
folders holding its documents, nested folders collapsed. A folder beside a
state directory whose name differs from it only in letter case is owned
state directory whose name differs from it only in letter case, or that
adds a version, as TreeLine's `~/.treeline-3.1` beside `~/.treeline`, is owned
whole in the files' spelling, so a `~/.config/GIMP` profile GIMP created
during the capture beside the candidate `~/.config/gimp` is owned as
`.config/GIMP`, not as the folder of its first file. The home itself and
@@ -140,9 +178,13 @@ The class declares:
Files the capture recorded under a symlinked state directory's physical
spelling are mapped back to the innermost such directory's home spelling,
unless that spelling is outside the captured home, as when the home itself is
a symbolic link. When two files map to one home path, the export uses the one
present before the capture, else the first by path, and a limitation names
the other.
a symbolic link. When a selected link names an application state directory,
the bundle seeds the application's state path and restores the selected
alias at boot. Targets recorded as directories are created even when the seed
has no files there; file targets and targets of unknown kind are not turned
into directories. Selections nested under that alias are seeded under the same
state path. An existing incompatible alias stops setup without replacing it. When two files map to one home path, the export uses the one present
before the capture, else the first by path, and a limitation names the other.
- Data hooks, because declaring any hook switches off agentenv's default data
plane: the class seeds the home once per boot and keeps it as the initial
state with `keep_initial`, `data/add` and `data/reset` call agentenv's `add`
@@ -155,10 +197,76 @@ retain is not seeded: one in the state view without retained content at that
checkpoint, or one the capture omitted (for example too large or past the
content budget). The result lists them in `unseeded_files` and the README
names them. When the application held files open in folders whose starting
content the capture did not retain, a limitation names those folders. The
desktop Trash, `~/.local/share/Trash`, is never the application's, so a file
the user moved there is neither owned nor seeded, even when a window title
names it; a limitation lists the files the Trash held at the seed.
content the capture did not retain, a limitation names those folders, spelled
from `~/` under the home: a file there is seeded only when the capture
retained it, as it does one written during the capture, and the limitation
names the seed files there or says there are none. The desktop Trash,
`~/.local/share/Trash`, is never the application's, so a file the user moved
there is neither owned nor seeded, even when a window title names it; a
limitation lists the files the Trash held at the seed.
Files that changed during the capture outside the owned paths are neither
owned nor seeded, so the app starts without what it saved there: MuseScore
keeps its settings in `~/.config/MuseScore`, which its name rules miss, and
showed its first-run wizard on every boot. A limitation and the README name
those folders, leaving out desktop services' files, each collapsed to its
folder in `~/.config`, `~/.local/share`, `~/.local/state` or `~/.local`, the
hidden folder of the home it lies in, or else its own folder, up to eight and
a count of the rest. Capture again naming them with `--state-path`, or
`state_paths` on a build computer, to own them.
Lock files the app holds while it runs are left out of the seed: a copied one
makes the app find its profile or document in use, and a LibreOffice Calc
bundle whose seed held its profile's `.lock` showed no window at boot. They
are `lockfile`, `lock`, `.lock`, `.parentlock`, `parent.lock`,
`SingletonLock` and Lucene's index lock `write.lock` anywhere, as Poedit's
translation memory holds, LibreOffice's document lock `.~lock.<name>#`, a
`<file>.lck`, `.<file>.lock`, `.<file>.lck` or `<file>.<ext>.lock` beside a
file the capture recorded in the same folder, such as GnuCash's
`household.gnucash.LCK`, Grisbi's empty `.household.gsb.lock`, which made it
refuse every save, and VYM's `plan.vym.lock` (a project's `Gemfile.lock`
beside its `Gemfile` has no extension to its file, so it is seeded), and
`*.lock` or `*.lck` directly in the home, as wxWidgets' single-instance
`~/poedit.lock`, or directly in one of the application's state directories;
a lone `notes.lck` or `.notes.lock` elsewhere is seeded. A lock
never makes a path the application's own, so `~/poedit.lock` is neither owned
nor named among the folders the bundle does not own.
Elsewhere such a file may be a project's own, such as `Cargo.lock` or
`yarn.lock`, which pin dependencies, and is seeded, even under a folder whose
name starts with a dot. A file the state view makes a document, or one named
itself by `--state-path` or an absolute `--argument`, is never left out,
whatever its name. A limitation and the README list them, up to
eight and a count, after "Left out of the seed". They stay under the owned
paths, so `data/get` returns the lock the running app holds, and `data/reset`
restores the seed without them.
A seed file keeps the permission bits it had at the seed when the capture saw
it change, such as a `0600` file an app checks: the class module lists those
other than `0644` in `SEED_MODES`, and the first boot sets them when it copies
the seed. A file the capture never saw change has no recorded mode and is
written `0644`.
Files in app state are retained regardless of a `.log` extension or `log.<n>`
name. They can hold conversation history or database writes as well as
diagnostics. The normal host-identity scrub applies to retained text; filenames
alone do not identify disposable diagnostics. Locks beside these files follow
the lock rules above.
Other seed files the app keeps for its own
running are seeded as captured and named, collapsed to the folder that holds
them: WebKit or Chromium storage (such as `hsts-storage.sqlite`, `storage/`,
`mediakeys/`, `Local Storage/`, `IndexedDB/`, `Cookies`, `Visited Links`,
`Platform Notifications/`, and a whole `QtWebEngine/`
profile), which may hold the app's own data, and caches (a folder whose name ends in `cache`); a stale one
can change how the app starts. A seeded LibreOffice profile's
`registrymodifications.xcu` ships without its `RecoveryList` items, which
LibreOffice writes for the documents it has open and which would start the
bundle into document recovery instead of its window; a limitation and the
README name the file. REA finds the items with its XML parser, as `item`
elements whose `oor:path` is `/org.openoffice.Office.Recovery/RecoveryList`
however the tags are spelled, and cuts exactly those elements, so text in a
comment or CDATA and every other byte stay as captured. A registry that is
not well-formed XML ships unchanged.
The application's dconf keys at the seed, if any, are written once when the
app first starts, also for an application that owns no files, and the class adds
@@ -168,7 +276,14 @@ existing user database through a `file-db:` profile, then runs `dconf compile`
on that dump and the captured keys, in that name order so the captured keys
win, and replaces `~/.config/dconf/user`. If that fails, the server prints a
one-line warning and the app starts without the captured keys. `data/reset` does
not restore them.
not restore them. No file of the bundle holds them, so `data/get` does not
return them and no task check reads them; a draft whose only change between
its checkpoints is to the application's dconf keys says so with
`uncheckable_changes`, and one whose dconf database changed but could not be
read says so with `unreadable_changes`. When a dconf database changed by the
seed checkpoint but its version there could not be read, as without
`dconf-cli`, the export seeds no keys from it and a limitation names it with
the read error.
## The seed checkpoint
@@ -185,6 +300,49 @@ The export refuses a name the capture does not hold exactly once, and an end
checkpoint taken before the seed checkpoint; `inspect-desktop-application-state`
lists the names in `capture.checkpoints`.
## A seed made for the task
By default the seed is the user's real data: the README and the result's
limitations say to keep the bundle local and not publish it. With
`--synthetic-seed` they say instead that the seed was made for the task, and
to read `envs/<name>/seed/` before publishing the bundle; the export cannot
tell the two apart, so the flag is the caller's word. `export_build_environment`
passes it, since the building agent makes the seed in the build computer. The
Evidence records it as `synthetic_seed`.
## Withheld sign-in state
`--withhold-from <checkpoint> --withhold-to <checkpoint>` withholds every file
under the owned paths that changed after the first checkpoint up to the
second, as the capture recorded its changes; `--withhold-path <file>` adds a
home-relative file, such as one written at first launch. The draft
(`describe-desktop-application-task`) takes the same options, leaves those
files out and lists them as `withheld_files`; the export refuses a check that
names one. Each withheld file present at the seed is left out of
`envs/<name>/seed/` and becomes a secret setting:
- the class declares `Settings` with a `SecretStr` field per file, and
`WITHHELD`, each field's home path;
- `env.toml` sets each field to `env:REA_<APP>_<FILE>`, so agentenv refuses
the deploy, naming the variable, until it is set;
- at first boot, before keeping the initial state, the class writes each file
from its setting, the file's bytes in base64, mode `0600`, so `data/reset`
restores it, and `data/get` serves it neither as a part nor in the views;
- the README names each variable, with
`export REA_<APP>_<FILE>="$(base64 -w0 < ~/<file>)"` from the receiver's
own signed-in copy of the app.
A withheld file larger than 48 KiB is refused, since its base64 must fit one
environment variable. `--withheld-copy <folder>` copies the withheld files,
as captured, into a new or empty folder outside the bundle. The result lists
them as `withheld_files`, each with its `setting`, `variable` and `handle`.
`--consent <file>` ships the person's consents, a `consent.json` that
`record_build_consent` records, as the bundle's own `consent.json`.
`--seed-withheld` seeds the withheld files after all, refused unless that
file holds a `seed_sign_in` consent; the result lists them as
`consented_files`, and the README says they are there with consent.
## Host name and user
A seed can hold the capture host's name and the user's: Geany saved its mail
@@ -228,6 +386,28 @@ export removes them from what the bundle ships:
boot, as it did the captured home before. The replacements read as the
computer's own: its user is `user`, with its home in `/home/user`, and
`localhost` is the name every host has for itself.
- A gzip-compressed seed, such as a Gnumeric workbook, is decompressed, up to
256 MiB, scrubbed as the format the state view read its document as, and
compressed again whether or not its document changed, so no file name,
comment or time its header held ships: with no file name, comment or extra
field, modification time 0 and the operating system unknown, so the same
document gives the same bytes on every host. It is deflated at the fastest
level, or stored when deflating would make it expand more than the 100 times
a view reads, so a seed that read still reads. It is listed in
`scrubbed_files` when its document changed, and otherwise in
`recompressed_files`, which the README counts rather than lists, so the
example and template files an app copies on first run do not bury the
document scrubbed; one already in that form ships as captured. The class copies it at
boot as it is, so the home it holds stays `/home/user`. One that does not
decompress within the scrub limit ships as captured, its bytes searched.
A document the state view did not read can still be decompressed and
recompressed under the separate 256 MiB scrub limit; its content is searched
even when its format cannot be scrubbed. One whose document needs no edit but a member of
which has an extra field, where BGZF (`.vcf.gz`, `.bam`) and dictzip
(`.dict.dz`) files index their blocks, also ships as captured, its bytes
searched, so its blocks stay where its index says; one whose document
changed is still compressed again as one member, losing them, as the
export's limitation says.
- Binary files, SQLite databases among them, and text files over 256 MiB are
copied as captured. Then every file the bundle ships, the class module with
its dconf keys and the README included, is searched by a looser rule than
@@ -294,6 +474,73 @@ declared vendor repository installs the package already; when the URL is not
written as `http://` or `https://` or holds a user name or password; and when
the file is built for an architecture other than `amd64`, `arm64` or `all`.
## Applications from an AppImage or tarball
A [build computer](desktop-build-computer.md) can install an application from
its project's AppImage or tarball, pinned by URL and SHA-256, under
`/opt/rea-apps/<desktop entry>`. Neither holds a Debian package, so the
capture records the package as `unknown`, and the export takes the install
from `--download-install`, a JSON file the build computer writes from its
request:
```json
{
"kind": "appimage",
"desktop_entry": "org.freecad.FreeCAD",
"url": "https://github.com/FreeCAD/FreeCAD/releases/download/1.1.4/FreeCAD_1.1.4-Linux-aarch64-py311.AppImage",
"sha256": "675d1a4295b2183e0e1d816a4b5557dcb63aeced4e60aaa28d6c3686f4c71dc9",
"architecture": "arm64"
}
```
A tarball's adds `program`, its path inside the archive, and optionally
`name`. The class then declares
`downloads=[Download(url=..., sha256=..., to="vendor/app.AppImage", arch=...)]`
(`vendor/app.tar` for a tarball) and `run=[RunScript(script="install-download.sh")]`,
the same script the build computer ran, with no apt package for the app. The
bundle does not contain the application and supports only the download's
architecture; the URL must keep serving those bytes, and the application's
licence is the operator's to check.
The export refuses the file (`download_install`) unless it names an
`appimage` or `tarball` with an `http://` or `https://` URL and a SHA-256,
the capture launched its `desktop_entry`, the captured program lies under its
folder in `/opt/rea-apps`, and the capture ran on its architecture; it
refuses `--package-url` beside it.
## Grading once the app has saved
A task's verifier, `artifacts/<task>-check/verify.py`, grades the app's files
only once they have settled, because an app can save on its own timer after the
last change. Gnote saves a note 4 s after the last key, so a read taken as
soon as the agent stops finds the text on screen but not yet in the file. The
verifier reads the app's `data/get` every 0.5 s. It compares each file's bytes
between reads by their SHA-256, as `data/get` returns them, and grades once
every file has held for 3 s. The 3 s count from when a read returned the
current bytes, not when it began, so a slow read that brings a change late
does not count its own run as quiet. 15 s after the first read began, the
bound, the verifier grades the last completed read: no read starts at or
after the bound, and a read still running then is cut. Each result records
whether the files settled in `settled`, and whether the bound cut a read in
`read_cut`.
The window has a cost and a limit:
- An app whose files do not change during the check, such as one that saved
at once, is graded 3 s after the first read returns.
- A save that lands more than 3 s after the first read returns is still
missed. An app that saves 4 s after the last key is caught only when the
agent stops at least a second after that key.
- Files the app rewrites more often than every 3 s, such as a log, never
settle, so every check waits until the bound.
- The bound counts the reads too, so a slow `data/get` ends the check at
15 s. Only the first read, which every grade needs, runs to its own
timeout.
The bound and window are `SETTLE_BOUND_S` and `SETTLE_WINDOW_S` in the
verifier. `check_build_task` waits the same way before it grades each
witness stage; see [witnessing](desktop-build-witness.md#witness).
## Typed views
The views follow the [state view](desktop-application-state.md)'s formats and
@@ -308,13 +555,30 @@ as their entries, read with Python's `zipfile` from the file in place, with
REA's entry, size, compression ratio and archive limits and its wording for an
entry it does not read. The module reads the central directory itself, as REA
does, and shows `zipfile` only the records it lists, reading each entry in
bounded chunks no further than a byte past its declared size. A JSON file
bounded chunks no further than a byte past its declared size. A gzip file is
the view of the document inside it, with `"compression": "gzip"` beside its
format, decompressed with Python's `zlib` within the same entry size and ratio
limits, its members joined as REA joins them. A JSON file
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
or nested more than 500 levels deep, gets an `error` instead of a view, as in
REA, while text that is not one JSON
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
document, such as JSON Lines, stays text. Format detection follows REA's
rules. Files larger than 64 MiB get an `error` instead of a view
and are not read. XML is read with Python's `expat` twice: with namespace
and are not read. The selected `file_bytes` also applies to scrubbed seed task
views before admission, independently of how much seed data the export can
copy. SQLite table views have a shared `sqlite_view_bytes` allowance, default
16 MiB of compact UTF-8 JSON per table, checked before retaining each row.
An oversized table keeps its error while other tables remain readable. Raising
this view allowance does not raise the independent 16 MiB native SQLite
value/record limit or REA worker reply limit; worker facts include additional
schema and typed-cell metadata and may be refused earlier. The TypeScript
and exported Python readers count their respective native compact JSON encodings;
float and exponent spelling can make their exact byte thresholds differ even
when they report equal numeric facts. The Python byte counter quotes large
strings in bounded chunks before retention.
Exported SQLite readers require Python 3.11 or later for the native connection limits; the
AgentEnv compatibility workflow uses Python 3.12.
XML is read with Python's `expat` twice: with namespace
processing, which refuses what REA's parser refuses, such as an undeclared
prefix, then without it, so element and attribute names keep the prefixes the
document spells and namespace declarations stay attributes, as in REA's view;
@@ -337,6 +601,6 @@ when a DOCTYPE names a DTD. Known differences remain:
and then gives the file an `error`, and so does a table or column name that
is not UTF-8.
The seed is the user's real data. Keep the bundle local and do not publish it.
The class was checked against agentenv-framework 0.9.1431 with
`python -m agentenv_protocol.computer._declare` and `agent-env run --dry-run`.
Unless it was [made for the task](#a-seed-made-for-the-task), the seed is the
user's real data. Keep the bundle local and do not publish it.
The class targets agentenv-framework 0.9.1485.
+128 -5
View File
@@ -11,6 +11,65 @@ again.
rea inspect-desktop-application-state ./mousepad-capture --format json
```
Views use bounded defaults. When a view reports truncation or a byte-budget
refusal, retry the same capture with `--view-policy` JSON overrides:
```bash
rea inspect-desktop-application-state ./mousepad-capture \
--state-path /home/ada/Documents/notes.txt \
--view-policy '{"text_lines":10000,"file_bytes":134217728}' --format json
```
`--state-path` selects an exact original captured path and can be repeated.
Omitting it examines every captured path. Selection preserves the captured
checkpoint and digest; it does not substitute the file's current contents.
A path absent from the capture is an input error.
`--view-policy` accepts positive safe-integer overrides for `file_bytes`,
`sqlite_tables`, `sqlite_rows`, `sqlite_view_bytes`, `text_lines`, `xml_nodes`, `xml_depth`,
`parsed_bytes`, `json_depth`, `zip_entries`, `zip_entry_bytes`, `zip_bytes`,
and `zip_ratio`. Unspecified controls keep the defaults described below.
`xml_depth` admits at most 494 and `json_depth` at most 991: these bounds
follow the fixed Evidence JSON nesting limit, including the captured-version
and ZIP-member wrappers and XML's element/children representation. A request
above either bound is rejected before reading the capture.
Evidence parameters retain only the supplied `view_policy` overrides; omission
and an explicitly empty selection remain distinct. State inspection and task
drafts report the resolved policy separately as
`normalized_result.effective_view_policy`. Changes apply on this call, without
a restart. File and ZIP byte counts refer to retained file bytes or uncompressed
ZIP member bytes, not transport size. Gzip document views share `zip_entry_bytes`
and `zip_ratio` with ZIP entry views; `zip_bytes` bounds a ZIP archive, not a
gzip stream. These selections also apply to the decompressed document's format
view. `sqlite_view_bytes` instead bounds each
projected table's compact UTF-8 JSON, including column names, primary keys,
rows and truncation metadata; it defaults to 16 MiB. Rows are charged before
retention. A table that exceeds this allowance keeps an actionable `error`,
while other tables remain readable. Increase this control and retry the same
capture for a projected-view refusal. It does not raise the SQLite worker's
separate fixed value/record or complete-reply limits, which may refuse earlier.
Each reader counts its native compact JSON encoding. Float spelling can differ:
Python emits `16.0` where JavaScript emits `16`, and exponent spelling also
varies. Identical numeric facts can therefore cross this byte threshold at
different settings. This is not a common transport-byte guarantee.
The same option is available on `describe-desktop-application-task` and
`export-desktop-application`. Export uses the accepted policy for task
validation and emits it into the Python readers used by the environment's
`data/get`; re-export with another policy to change those reader budgets.
Task checks apply `file_bytes` to the actual scrubbed seed bytes, including any
growth from replacing the capture home with `/home/user`. A seed over that
view allowance may still be exported under the independent acquisition limit,
but its unreadable view cannot establish a known task failure.
The TypeScript and exported Python readers share the policy's defaults.
These desktop capture workflows remain CLI-only.
Raising a view budget cannot recover content the capture never retained,
change numeric exactness, enable encrypted or unsupported archive formats,
or raise SQLite worker, Evidence nesting, process-memory or transport limits.
Keep increases specific to the exhausted budget. Truncation and incomplete
comparison coverage remain visible until the requested facts were examined.
The result is Evidence with operation `inspect_desktop_application_state` and
predicate type `rea.desktop-application-state`. Its confidence is `inferred`,
because roles come from heuristics; the views themselves are read from the
@@ -25,7 +84,7 @@ changed later. Each path records the evidence the capture holds about it:
| ------------------------ | ------------------------------------------------------------------------- |
| `state_directory` | It lies in one of the application's state directories, in either spelling |
| `opened_by_application` | An application process held it open during a sample |
| `named_in_window_title` | An application window title spelled its name or `~/` path on its own |
| `named_in_window_title` | An application window title spelled its name, `~/` or absolute path whole |
| `selected_path` | It lies under a `--state-path` or absolute `--argument` of the capture |
| `dconf_application_keys` | It is a dconf database holding groups whose path names the application |
| `desktop_service` | It lies where desktop services and toolkits keep shared files, or caches |
@@ -40,11 +99,26 @@ The role follows from the reasons, in this order: `shared_desktop` for a file
in the desktop Trash, which the user deleted, whatever else names it;
`application_state` for application keys in dconf, or a state directory outside shared desktop
locations; `document` for a file named in a window title or selected by the
caller, even if the application holds it open; `application_state` for another
caller, even if the application holds it open, except that a selected file
under a hidden folder of the home, such as `~/.config/MuseScore`, a dotfile
there, or a lock an application holds, such as LibreOffice's `.~lock.<name>#`
beside a document, is `application_state`, judged by the path as the caller
named it, so a file reached through `~/.config/MuseScore` stays the application's
when that folder is a link to `~/settings/MuseScore`; the innermost selected
path holding a file decides, so a score selected itself inside the link's
target keeps its own name and is a document (a capture written before it
recorded selected paths is judged by their physical spelling);
`application_state` for another
open file outside shared locations; `shared_desktop` for other files in shared
locations; `unrelated` otherwise. A window title names a file only when no
other file name character adjoins the name, so `day.txt` does not match
`today.txt - Notes`.
`today.txt - Notes`, and never by a folder of a path the title shows: in
CherryTree's `home-notebook.ctd - /home/user/Documents - CherryTree 1.1.2`,
`user` names no file. A name without an extension, such as `Makefile`, must
also stand on its own rather than end a longer path, as in
`Makefile - Editor`. A file under `~/.cache` or `~/.config/dconf` is never
named by a title. The capture's checkpoint acknowledgements rank files by the
same rule.
Application names, which select dconf groups, come from the package, the
executable, the desktop entry's id and `StartupWMClass`, and the window match.
@@ -70,11 +144,21 @@ whether the file was present, the SHA-256 of its retained bytes, and a view:
| `dconf` | The `GVariant` header | Groups whose path names the application, plus the names of the other groups |
| `zip` | A zip local file header, `PK\x03\x04`, whatever the name | Its file entries in archive order, each with its own format and view |
| `json` | A `.json` name or a leading `{` or `[` that parses | The parsed value, from files up to 4 MiB nested up to 500 levels |
| `xml` | An XML name or declaration that parses | An element tree (`name`, `attributes`, `children`); text kept exactly, DTDs never read |
| `xml` | An XML name, a declaration or a leading element that parses | An element tree (`name`, `attributes`, `children`); text kept exactly, DTDs never read |
| `keyfile` | Only `[group]`, `key=value` and comment lines, with at least one group | Groups of `key` → string value, from files up to 4 MiB |
| `text` | Other UTF-8 text | Lines, up to 5,000 |
| `binary` | NUL bytes or invalid UTF-8 | None |
A gzip file is read as the document inside it; see
[compressed documents](#compressed-documents).
Text that opens with an element, such as a Freeplane map's
`<map version="freeplane 1.7.0">` with no declaration, or with a DOCTYPE that
has no internal subset and then an element, such as a KmPlot plot's
`<!DOCTYPE kmpdoc>` before `<kmpdoc version="4">`, is XML when the whole text
is well formed, whatever the file is called, a `.txt` file included;
otherwise it stays text. The exported verifier makes the same choice.
A JSON value holding a number JavaScript would round, such as an integer above
2^53, is not read rather than shown rounded.
@@ -128,7 +212,15 @@ could misstate its committed rows. The capture records them under its content
roots, which include both beside a database selected on its own, when its
baseline walk listed every directory, read every path, entered no other
device, did not stop early and recorded what it left out, and wherever they change; an excluded log, and any elsewhere, is
unknown, since an unchanged file and a missing one look alike. A table
unknown, since an unchanged file and a missing one look alike. After an
incomplete checkpoint, a database or sidecar without an explicit
later change remains unconfirmed even if a later walk completes. REA does not
combine that older file with a newly recorded member of the SQLite set. The
historical versions remain listed; the affected SQLite view has a `view_error`.
A baseline non-file at a sidecar path, or a non-directory ancestor, cannot
establish an absent file. Captures without the required baseline non-file
inventory must be made again before inspection.
A table
that cannot be read keeps its `error` in place of its rows, and changes to that
database are marked incomplete. dconf
databases are read with `dconf dump /` against a private file profile, so the
@@ -185,6 +277,37 @@ so an archive of hundreds of thousands of entries costs no more to read
than one of a thousand. An archive whose central directory cannot be read has a
`view_error` of its own.
## Compressed documents
Gnumeric's `.gnumeric`, Dia's `.dia`, Xournal++'s `.xopp`, GnuCash's default
book and Inkscape's `.svgz` are XML compressed with gzip. A file that starts
with gzip's magic bytes, `1f 8b 08`, whatever its name, is read as the
document inside it: its members joined, then given a format and view by the
rules above under the file's own name, so a pointer or check into it is
written exactly as for the same document uncompressed, and the task draft is
the same. The version records the compression beside the format:
```json
{
"format": "xml",
"compression": "gzip",
"view": {
"root": { "name": "gnm:Workbook", "attributes": {}, "children": [] },
"truncated": false
},
"view_error": null
}
```
The document is read only when it is at most 16 MiB and at most 100 times the
file's size, an archive entry's limits, so decompressing stops there; past
them, or when the stream is damaged or cut short, the version has `format`
null and a `view_error`, and a check into it is undecided. A SQLite or dconf
database inside keeps its format and has a `view_error`, an archive inside is
read as an archive, and another gzip stream inside is binary. A gzip file
inside an archive reads as binary. Other compression, such as xz, Zstandard
or bzip2, is not read: such a file is binary.
## Kinds of change
Between consecutive versions a change is `created`, `deleted`, or a
+682 -57
View File
@@ -3,11 +3,12 @@
A [desktop application export](desktop-application-export.md) can carry one
task: a prompt for a computer-use agent and checks that grade what the agent
left in the application's files. An agent, not REA, writes the prompt and the
checks from the change the capture recorded. REA validates the task against
the capture and keeps it only when its checks fail on the seed and all pass on
the captured end state. REA runs no model and no code the agent wrote: the
checks are declarative, and the exported verifier evaluates them the same way
REA does.
checks from the change the capture recorded. For file checks, REA validates
the task against the capture and keeps it only when its checks fail on the
seed and all pass on the captured end state. The checks are declarative;
optional [readers](#readers) run commands to interpret files, and
[screen judges](#screen-judges) use a model when no file shows the change.
The exported verifier uses the same checks and reader runner as REA.
```sh
rea describe-desktop-application-task ./mousepad-capture --format json > draft.json
@@ -42,35 +43,457 @@ operation `describe_desktop_application_task` and predicate type
database is compared by its rows, since its hash leaves out the `-wal`
file. Text changes list each line
once and add `line_counts`, the lines found at both ends whose number of
occurrences changed. Views appear only for the files the suggestions cover,
with arrays past 200 items and strings past 2,000 characters cut
occurrences changed. An XML file's change names its members by the
[selectors](#pointers-and-selectors) checks use, as an archive's already
does, and by position only where nothing names a member or where only the
members' order changed. Views appear only for the files the suggestions
cover, with arrays past 200 items and strings past 2,000 characters cut
(`view_cut`).
- `values`, in each of those files: the values its change added, changed and
removed, each with the `pointer` a check names it by, its `kind` (`added`,
`changed` or `removed`), and `before` and `after`: the scalar, an element as
compact XML with the text the XML reader kept, spaces between runs included,
or a JSON value as compact JSON, cut at 600 characters, and null at the end
that lacks it. A file created or deleted lists what its one end holds,
`added` or `removed`: an archive's entries, an XML file's root element, a
JSON file's members or a key file's keys, each whole once. A value only
position reaches has `note: "by_position"`, and its pointer cannot go into a
check; noise an app writes as it saves has `view_state`, `renumbered`,
`volatile`, `timestamp`, `upkeep` or `layout` (see `task`). A record of
text fields added to a list of its kind, as GnuCash's
`<gnc:transaction>`, is listed field by field, and a field that lists
records, as its `<trn:splits>`, record by record, so the cut never hides
one. `refers_to` names up to
five other elements a value equals the `key` or `id` (`xmi:id` among them)
of, outside a view the app keeps of the document, as Umbrello's
`listview` mirroring its model by id, with the
`attribute` holding it, as HomeBank's `category="44"` names
`<cat key="44" name="Groceries"/>`, or the id a child element holds, as a
GnuCash split's `<split:account>` names the account whose `<act:id>` it
is, shown with its name, `<gnc:account …><act:name>Groceries</act:name>…`;
an element added whole lists those of the records of attributes it nests
too, as a KMyMoney transaction's splits name their `<ACCOUNT>` and
`<PAYEE>`, each once. Where several elements share that key, the one
whose tag shares the attribute's stem, as `payee` and `<pay>`, and of
those the one named for the attribute, as `account` an `<ACCOUNT>` before
an `<ACCOUNTID>`. A plain number names an element only through an
attribute so named or ending in `id`, `ref` or `key`, so a list's
`count="5"` names no `<FIXVERSION id="5"/>`. A value that changed lists,
in `alternatives`, its pointer naming the element by each other identifier
or name-like attribute it holds alone among its tag's members, up to
three, as an Inkscape text's `[@inkscape:label=Headline]` beside its
`[@id=text12]`, for a check to name it by instead; an element its tag
alone names inside the first one a condition names, as an EPUB chapter's
`h1`, by its `[@id=chapter-two]`. What a check can
name comes first, text an edit changed before the rest, so no cut drops
the edit, then what only position reaches, then noise. A file lists
at most 20 values and 8 KB, the draft 24 KB, and `values_cut` says when a
limit left some out. Values cover XML, archives, JSON and key files; a text
or SQLite file's change and suggestions already carry its lines and rows.
- `task`: a draft with the name `change`, a placeholder prompt the export
refuses, and suggested checks derived from the net change of the files with
role `document`, or of the app's own state files when no document changed.
role `document`, or of the app's own state files when no document changed,
excluding locks the export leaves out of the seed. App state files ending
in `.log` remain eligible for checks.
A text file gains `lines_present` for its new lines and `lines_absent` for
lines it lost (when either is at most ten), and `line_count` for each line
holding a letter whose count changed; JSON and key files gain
`value_equals` and `value_absent` by pointer; SQLite tables gain rows added,
holding a letter whose count changed; a new line holding an XML attribute
REA recognizes as generated, such as a named epoch time or `ID_329023692`, gets a
`line_count` of the lines `contains` its text up to that value, as
`<node TEXT="Water filter" ID="`, at the end's number, where both views
hold the whole file, instead of a `lines_present` another save would
fail, or a `volatile_lines` note when no text comes before it. UUID-shaped
text stays exact because its shape alone does not show that it was generated;
JSON and key files gain
`value_equals` and `value_absent` by pointer, a JSON array's records named
by an id field each holds alone at both ends (`id`, `uid`, `uuid`, `guid`,
`key` or a name ending in `_id` or `Id`), as `/lists/[id=chores]`,
whatever order the app saves them in; where the app generated that id, a
record both ends hold is named by its kind field and its first name-like
string field (a key ending in `name`, `title`, `label` or `caption`),
nearest first and up to three keys deep, that holds the same string at
both ends and names it alone at each, as TreeLine's category by
`/nodes/[format=CATEGORY][>data~1NAME=Science]`, and by its id where none
does, as `/nodes/[uid=7584d494d9f511f1a873edd4f1920549]` once its name
changed; and a record only one end holds is named by such a field that
names it alone at its end and nothing at the other, as TreeLine's new book by
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, its other fields
checked through that selector and the fields the selector reads not, nor
a field holding what every like record (one of its kind) at the end holds
there, as the empty `children` of every book; and
by position only where nothing names a record, as a new book whose title
another book holds, with a `by_position` note listing those checks;
never a value that is a uid the app generates (a
GUID, or 32 hex digits as TreeLine's uuid1), nor a check the seed already
passes, as a new record's `format` where another record of that kind held
its place; a removed record is checked gone by its own pointer, as
`/lists/[id=chores]`, and only where the end holds nothing there, so a
record only position reached, whose place another took, is left to the
count; `member_count` checks of the
arrays whose members were added or removed, of the members' kind where
records share a `type`, `kind`, `format` or `category` field, as `/nodes`
matching `[format=BOOK]`, and a count guard of the innermost array an edit
went through, as for XML; SQLite tables gain rows added,
changed and removed, by primary key, and their new row count. An XML file,
and the JSON, key file and XML views of a zip archive's entries, gain checks
by [selectors](#pointers-and-selectors) and never through an element by
position: in XML, first [`text_equals`](#whole-text) for each element whose
text changed, then an element added whole that it exists, one removed whole
that it is absent, then values changed, added and removed, an entry or a
JSON value added whole through its members, and an empty array or object as
it is. The text check names the paragraph rather than a run inside it: it
goes up from an element sitting in its parent's text, and from one named by
its tag or position alone to the nearest ancestor named by an attribute, as
AbiWord's `p` by its `xid`; the text nodes and runs inside it get no other
check, but an element holding no text that is added or removed there, such
as an image or a line break, keeps its own, since the text check never
reads it. Whitespace an app lays out between elements is not checked, and
values that look like a time the app stamps, in ISO 8601 or as C's
`asctime` writes it (`Thu Nov 5 09:20:31 2026`), are left out.
text changed, then elements added or removed whole, then values changed,
added and removed, an entry or a JSON value added whole through its
members, and an empty array or object as it is.
- A selector names a member by its tag, then by its identifiers (`name`,
`id`, `xid`, `xml:id`), then by one other attribute, a name-like key such
as Scribus's `ANNAME` or `inkscape:label`, a worded value or an integer of
up to five digits first, then by two such attributes together, as
Gnumeric's cells by `[@Col=3][@Row=2]`, then by a coordinate, a long
number, a length with its unit, as `svg:height="2.629cm"`, an `xml:id`
or `draw:id` of the form `id2` that LibreOffice numbers anew as it saves,
so a Draw shape is named by its `draw:name`, or a token,
such as `XPOS` or `ItemID`, or an id of random case, as Umbrello's
`ubcgF2AWfBmHh` (`u` and twelve letters and digits), an automatic style name an app numbers as it
saves and renumbers on reload, as LibreOffice's
`presentation:style-name="pr1"` (`P1`, `T1`, `MT1`, `L2` on a
`*:style-name`), so a slide's frames are named by `presentation:class`,
or an attribute naming
view state, such as which layer is `selected`, and last by a child
element's text, `[>CHILD=VALUE]`, a name-like tag first, as Osmo's
record by `[>first_name=Delphine]`, MuseScore's `<Text>` by
`[>style=Title]`, a spreadsheet row by its label cell, or GnuCash's
transaction by `[>trn:description=Greenleaf Market]`: text of at most 64
characters, never a time or a token or id a program generates; then by
an identifier a child carries, `[>CHILD@ATTR=VALUE]`, as a GtkBuilder
`<child>` by `[>object@id=submit_button]`, never one an app generates or
numbers anew as it saves; and then by text a child joins from elements
of its own, as Glade's `[>packing=TrueTrue1]` from a widget's packing
properties, which moving the widget changes. A pair
counts every member holding both its values, whichever attributes that
member ranks first. An element with identifiers is named by those alone;
an id the app generates, as Freeplane's `ID_329023692`, or a GUID in
any attribute, braced or not, as VYM's `uuid="{18eadb5d-…}"`, ranks
last, after a child's text, so a new node is named by its `TEXT` and a
new VYM branch by `[>heading=Sunset boat tour]`, while a member
both versions hold pairs by such an id first, so records that keep their
UUIDs and swap titles read as two title edits. A value that stands for
none, as Grisbi's `Id="(null)"` on every transaction, names nothing, so a
new transaction is named by its `Nb`. A child's text is
counted on every sibling, one an identifier names included, so a title
two records share names neither. The selector must name the
member alone at both ends, with the same values; a child's text names a
member that is in one version only when the list's length changed, so a
row whose one cell an edit changed stays that row. An element whose one
identifier changed, at its place among its tag's members, with every
other attribute and its children alike, is that element renamed, as a
MyPaint layer renamed from `Layer` to `Sun`: it is paired by its first
other attribute naming it alone at both ends, as
`[@src=data/sun.png]`, so the rename reads as its `name` changed, and
the old name gets a check that it names nothing, unless its change is
noise, as an id the app renumbered. Where that attribute's values number
the members in document order, as MyPaint 2 writes `src` as
`data/layer-00.png`, `data/layer-01.png` and on along the stack and
renumbers them when a layer moves, each version is named by its own
value of the renamed identifier instead, `[@name=Sun]` and
`[@name=Layer]`.
- A child of the document's root that its tag alone names, and that holds
a list (two or more elements of one tag and no text of its own) and has
an identifier, also gets a condition that keeps it named if the app adds
a sibling of its tag, as Dia adds an empty `Background` layer each time
it reopens and saves a diagram: an id other than `name` both ends hold,
as Inkscape's `[name=g][@id=layer1]`, which a new sibling would not
share; else the text of a child both ends hold, as Dia's
`[name=dia:layer][>dia:object=#Receive order#]`, which an empty sibling
lacks; else its `name`, which a new one may repeat. Deeper, a member of
a list named for its tag, as Gnumeric's `gnm:Sheet` in `gnm:Sheets`,
holding elements and its tag alone naming it, gets an id, else a
name-like child's text, `[name=gnm:Sheet][>gnm:Name=Budget]`, else its
`name`. Other steps, and elements with neither, stay as they are.
An identifier the app numbers by position names nothing: where the
members of a tag hold it as 1..N (or 0..N-1) in document order at both
ends, members were inserted or removed, and every member another
attribute names alone at both ends kept its number or moved it the way
the count did, one at least, as Planner's task `id` after a task is
inserted, members are paired by their other attributes, so an insertion
reads as one task added. A value two members swap, or an id appended at
the end, as Tellico's next `<entry id="4">`, leaves the ids identity. An
identifier numbering the members 0..N-1 in document order at both ends,
where members came or went, as Parley numbers its `<entry id="6">` by
where it saves it, ranks after a child's text where a member has one, so
a new entry is `[>translation=library]` and its kept neighbours are named
alike; Tellico's ids, which count from 1, Tiled's and Rosegarden's,
whose members hold no text, stay identity. An element a change added that
holds nothing but the id of a record it added elsewhere, as Parley's
lesson membership `<entry id="6"/>`, is `upkeep`, listed with `refers_to`
on that record, never a check or a count.
- The text check names the paragraph rather than a run inside it: it goes up
from an element sitting in its parent's text, and from one named by its
tag or position alone to the nearest ancestor named by an attribute, as
AbiWord's `p` by its `xid`, unless it is a field its tag names in a
record of two or more fields holding text, as QElectroTech's `<text>`
beside `<info_name>` in a `<dynamic_elmt_text>`, or KmPlot's
`<equation-0>` beside `<arg-min>`, which is checked itself rather than
the record whose text joins its fields; one line of text the app lays
out between line breaks, as KmPlot's formula before the next element, is
checked, and its siblings guarded, with `normalize_whitespace: true`,
and text whose whitespace alone changed, as a formula the app moved to
an indented line of its own, gets no check; where no attribute names one, to the nearest
ancestor within three levels its tag names with no step by position, as
MuseScore's `VBox` around untitled `<Text>` elements; and where only
position reaches the element, to the ancestor above the outermost step by
position, by an attribute of its own that names it alone where one does,
as an ODS sheet's `table:table[@table:name=Q1 Sales]` above its rows. The
text nodes and runs inside it get no other check, but an element holding
no text that is added or removed there, such as an image or a line break,
keeps its own, since the text check never reads it. A decimal that is an
element's only text, such as a spreadsheet cell's, is checked by its
number instead. Free text that gains a line, such as a shopping list,
uses `text_equals` on the whole text: the current XML checks cannot
compare an unstructured line as a whole item. This can reject a correct
edit that puts the item elsewhere or changes spacing. Text an app keeps in an attribute, one element per run, as Scribus's
`<StoryText><ITEXT CH="Spring "/><ITEXT CH="Fair"/>`, gets a `text_equals`
with `text_attribute` on the nearest ancestor an attribute names, as
`[name=PAGEOBJECT][@ANNAME=Headline]` with `"text_attribute": "CH"`, where
that ancestor holds no text nodes and the attribute's holders are runs,
none inside another and none carrying an identifier or a name-like
attribute beside it, as Umbrello's `<ownedAttribute xmi:id="…"
name="email" visibility="private"/>`, a record whose attributes are
checked one by one, two or more or one holding words with a space: an
element's own attribute, as a Freeplane node's `TEXT` beside the nodes in
it, or a lone value, as `<Bool value="false"/>`, is checked where it is.
The check covers that attribute and the runs holding it; another change
in the frame, as a run's `FONT`, keeps its own check. A record added
or removed whole, an element holding only elements, its fields, such as
Tellico's `<entry>` or a record of repeated `<field>` elements, never
makes the list holding records of its kind a check, whose text would hold
the records the edit left alone, except a record removed whole that only
position names, whose absence only the list's text shows. Text made of
runs is no record: an element whose tag names a paragraph or a piece of
text (`p`, Writer's `text:p` and `text:h`, `h1` to `h6`, `li`, `para`,
SVG's `text` and `tspan`, a `span` or an `a`), one inside such an
element, as Word's `<w:r>` run in its `<w:p>`, one whose elements are all
inline formatting (`b`, `i`, `u`, `em`, `strong`, `bold`, `italic`, a
`span`, `a`, `code`, `sub`, `sup` and the like, Gnote's `size:` and
`link:` runs), as `<list-item><bold>olive oil</bold></list-item>`, or
whose runs split a sentence at a space, as `Meet ` and a linked
`Monday`. Nor does an element holding a date the app stamps in parts
take a text check.
- An element added to a list, where another element of its tag is at either
end, gets a `value_equals` for each of its attributes, best ranked first,
at most six, leaving out its selector's attributes, identifiers, times,
volatile values, view state and values standing for none, as HomeBank's
new `<ope>` gets its `amount`, `category` and `payee`; a value two or
more like siblings all hold, as Grisbi's `Cu="1"`, comes after those
that vary, as its `Pa`, `Ca`, `Dt` and `Am`. Records of attributes
alone that it nests, as a KMyMoney `<TRANSACTION>` its `<SPLITS>` of
`<SPLIT>`, get a `value_equals` of each attribute whose values vary among
the like records, its own and those under its like siblings, at most
three a record and six in all, as each split's `value` and `payee`, but
not one every split holds alike, as `price="1/1"`, nor one each holds
equal to an earlier one, as `shares` repeats `value`. Each record is
named by an identifier that the app neither generates nor numbers anew
in every record holding such a list, as KMyMoney's `S0001` in every
transaction, else by its best ranked attribute short of a coordinate or
an amount, as `[name=SPLIT][@account=A000001]`. An element whose other attributes, two
or more, a like sibling holds every one of, as a new Planner task copies
its neighbour's defaults, and any other element added whole, get a check
that it exists; of those copied, an attribute naming an element the seed
holds by an id that is no plain number, as an Umbrello attribute's
`type` its data type, is the user's choice and keeps its check. A record added to a list that a selector names, as
Tellico's `<entry id="4">` or Osmo's `<record>` by its first name, gets
a `text_equals` for each field, at most five, each by its selector among
the record's fields, as `[name=title]`, leaving out a field repeating the
record's identifier, the field its selector names it by unless it is
the only one left, as a new VYM branch's `<heading>`, a value every
like record holds in that field, as Osmo's `<group>None</group>`, noise,
a field holding ids of its own, a date the app stamps in parts, and blank
text or times; a field that lists records, as GnuCash's `<trn:splits>`,
gets its records' fields instead, each record by its own selector. An
attribute every like record holds alike, as GnuCash's `version`, is the
format's and gets no check. A record only position names also gets a
`text_contains` for each new field of three or more characters, at most
five, at the nearest ancestor a check can name, each absent from that
ancestor's text at the seed.
- Where members of a list were added or removed, a `member_count` of the
list's members of their tag, and of their kind where they share one
(`type`, `kind` or `shape`, as `[name=dia:object][@type=Flowchart - Box]`),
unless every member of the tag holds a value of its own, as Tiled's
objects a free-form `type` (`well`, `door`, `gate`), which labels each
rather than grouping them,
equal to the end's number, which differs from the seed's: at most two a
file, the shallowest lists first, after the other checks. None for a
list of one member at both ends, as a style an app adds for a run, for
members of an element a text check reads, which are runs of its text,
for an element in mixed content, or for noise.
- A decimal written as text, such as `146.83`, is suggested as a number
with `tolerance: 0.001`, unless the seed's value, read as the evaluator
reads a number (`10` and `1e1` included), is that close too. A
`value_equals` that would pass on the seed's value is not suggested.
- An element placed by its own position, holding numeric `x` and `y` as a
like sibling does, as Xournal++ places each `<text>` note on a layer, is
an item of its own, never a run of the others' text: one added whole
gets a `member_count` matching its whole text with whitespace collapsed,
which passes wherever it is placed, and that list's `member_count`.
- Noise leaves the suggestions when the file's change holds anything
else, a change only position reaches included: view state, which a tag
or attribute on the way names (the cursor, a selection, as VYM's
`<select>`, the layer, frame,
view or colour the app has current, as Pencil2D's `currentLayer`,
scrolling, zoom, the viewport, windows, splitters, a list or tree view
mirroring the document, as Umbrello's `listview`, Inkscape's `namedview`, view and
configuration settings, the printer, a save's history, edit time,
editing duration and cycles, last-saved time, the generator and
document statistics, and a preview or thumbnail the app renders, as
Xournal++'s `<preview>`), or the value of a name-like attribute or `key` on
the way, when it is one token of several words, as LibreOffice's
`config:config-item[@config:name=CursorPositionX]` inside
`[@config:name=ooo:view-settings]`, or a settings path ending in how a
panel shows (`visible`, `hidden`, `expanded`, `geometry`, `size` and the
like), as VYM's `<setting key="/treeeditor/visible">`; an identifier the
app numbers by position. An `Id` suffix alone does not show that an
attribute is generated, so relationship fields such as `accountId` are
retained; other save-generated IDs may need manual review; volatile
values, UUIDs and epoch times, binary data encoded as base64 (a PNG's
`iVBORw0KGgo`, or 256 or more base64 characters mixing cases and
digits), an id counter (`uniqueid`, `nextobjectid`), a value equal to an
id the change gave a new element where a counter or that element holds
it, as Umbrello's `lastobject` holds the new attribute's `xmi:id`, but
never a reference on an element both ends hold, as a KMyMoney split
moved to a new account, and durations and counts under a document's metadata; times the app stamps, also as a date written in
parts whose tag, or one it sits in, names when the app made, changed or
saved something, as Tellico's `<cdate>` of `<year>`, `<month>` and
`<day>` (`cdate`, `mdate`, `created`, `modified`, `saved`, `updated`,
`entered`), or written whole under such a tag, as GnuCash's
`<trn:date-entered><ts:date>2026-11-06 08:53:08 +0000`,
which no check ever reads, while a date a user enters in parts, as a
birthday, is an edit like any other; an id the app assigns, an `id`
element with `type="guid"`, as GnuCash's `<trn:id>`, and text of control
characters, as its unset options hold; `upkeep`, a change inside an
app's `slots`, as GnuCash's `book:slots`, which it adds to and fills on
reopening a book, a count an app keeps of its members, an attribute
whose key ends in `count` or an element whose tag names a count, whose
integer changed and at each end equals how many elements of one tag it,
or the element holding the text, holds there, the counter left out, as
VYM's root `branchCount`, KMyMoney's `<TRANSACTIONS count="5">` or
GnuCash's `<gnc:count-data>`, or, for a counter the change added, how
many of one tag, more than one, it holds as children, as FreeCAD's
`<ObjectDeps Count="2">`, while a count a user enters, as an item's
`<count>`, is an edit, and the `loext:tab-stop-distance` attributes
LibreOffice fills in `styles.xml` while another entry holds the edit,
and, while a change outside them is the edit, any
change in `styles.xml`, in a document's style containers or its master,
handout and notes pages, or to an attribute naming an automatic style
(`P1`, `MT1`) at both ends, as Impress after a reload wraps its master pages' fields in a new `MT1` style and
rewrites list style `L2`, and a text showing another field that changed
alike, its element beside a field naming that one's key, as
QElectroTech's `<dynamic_elmt_text>` shows a component's label in its
`<text>` beside `<info_name>label</info_name>`; and `layout`, attributes holding a list of
numbers, as Dia's `obj_bb` `6.3225,6.15;12.5775,8.15`, or a number a
name on its way calls layout (a position, point, box, corner, width,
height, side or offset, as `elem_width`), that changed in an element up
to three levels below one whose text changed, below the root's own
children, while their own element's text did not, as Dia moves and
resizes a box around its new text, or below an element whose tag names a
diagram, widget, canvas or scene while the model change is elsewhere, as
Umbrello grows a class's `classwidget` `height` when an attribute is
added; an order's `<amount value="12"/>` beside its new description is
content. Geometry that changed with
no text beside it, as a CAD sketch's, is content. However much noise
comes first, it never takes a check's place. When the change holds
nothing else, the checks stay and a note flags them: up to three in a
document, and as many as fit in the app's own state, where a view
setting may be the task.
- Whitespace an app lays out between elements is not checked, and values
that look like a time the app stamps, in ISO 8601, as C's `asctime`
writes it (`Thu Nov 5 09:20:31 2026`) or as a time of day to the
second (`05:16:21.675302540`), are left out; noise never makes its
ancestor's whole text a check.
- `guards`: checks an author may add to show the edit left the rest alone,
kept out of `task.checks`, with ids `guard-1` and on: for each suggested
`value_equals` or `text_equals` on a member of an XML list, or a
`value_equals` through a JSON record added whole, as TreeLine's new book's
`data/Author` gives
`/nodes/[format=BOOK][>data~1Title=Small Engines of Life]/data/Author`
(its like records are those of its kind, named as both ends pair them,
and it reads only the check's own field), a field on up to
two like siblings, those sharing most of the attributes its selector
names it by and its type, as a cell's `Row` or `Col` and `ValueType`,
then the nearest first, at its seed value, which the end holds
too, trying at most the 16 nearest, so a guard costs the same among
thousands of siblings; for a member named as a field, by a name-like attribute alone, as
QElectroTech's `elementInformation[@name=label]`, the like siblings of
the nearest record out that hold that field too come first, so the other
components' labels guard a component's label rather than its other
fields: the check's own field where like siblings hold different values in
it, and otherwise one that varies among them, as Planner's `work` rather
than the `type="normal"` every task shares, but not one numbering the
members in order, as MyPaint's `src`; never through a sibling only
position pairs. A guard never selects its sibling by
what it reads: a sibling its selector names by the attribute the guard
checks, or by a value that is or holds the text it reads, as Glade's
`[>object=CancelTrueTrueTrue]` for a button's `Cancel` label, is named by
another identifier, name-like attribute or child's text it holds alone
at both ends instead, as Tiled's objects by `[@id=2]` rather than the
`name` the guard reads, or Glade's button by `[>packing=TrueTrue0]`, or
else the guard reads another field, or there is none. A `text_equals` with `text_attribute`
reads its siblings the same way. A text file gets a `lines_present` of the
nearest two unchanged lines of its first new line's kind, list items with the same
marker, as Zim's other `[ ]` action items, or lines the same command
opens, as the other headings after LyX's `\begin_layout Section`, or lines
of the same keyword or `key:` in other records, as a gettext catalogue's
other `msgstr` lines rather than the edited entry's own `msgid`, or else
its nearest lines with words in them; each text both ends hold once, never
a line with no text of its own, a command such as `\end_layout`, tags
alone such as `<fonts>` or `</node>`, or a key alone such as a RedNotebook
day's `24:`, nor a line that repeats; an item placed by its own position
that an edit added, a `member_count` matching each of up to four kept items'
text, nearest first. Where members were only edited, a
`member_count` of the innermost list holding the edited element that an
edit goes through by a member with like siblings, a record rather than a
field of one (a member named by a name-like attribute alone that the
record's like siblings hold too), of the edited member's tag and kind, at
the seed's number, as Dia's boxes in their layer, MyPaint's layers in
their stack or Glade's buttons in their button box: no member of the kind
added or removed; never of runs in an element a text check reads, and
only where the end holds the same number. A guard of a coordinate comes
after those of text. At most four a file besides the count guard, which
sits outside that cap, so sibling guards never squeeze it out. They pass
on the seed, which the export allows, since one check failing there is enough.
- `notes`: why `files` is empty, or why a file's suggestions are thin. Each
note has a `code` and a `message`. For a draft with no files: `no_change`
when no file under the home changed between the two checkpoints and nor
did the application's dconf keys; `unowned_changes` when files changed but
none under the owned paths, with up to five of them in `files`, each with
its home path and role (desktop services' files last), and how many more
in `more`; `uncheckable_changes` when the application's keys in a dconf
database changed, with the database in `files`: no check reads dconf, and
the export only writes the keys as they were at the seed checkpoint, so
make the task's change one the app saves in a file; `unreadable_changes`
when a dconf database changed but REA could not read it, with the read
failure in `error` (install `dconf-cli` or set `REA_DCONF_COMMAND` when it
is missing), since whose keys changed is then unknown; and `no_document`
when no file has the document role, as when no window title shows the
file's name. A capture again with `--state-path` naming the file, or
`state_paths` on a build computer's `start_build_capture`, makes it the
app's. For a drafted file, named in `file`: `by_position` when
changes reach their element only by position, the tags no attribute names
alone in `elements`, or when only the elements' order changed;
`view_state_only` when every change is noise, the checks kept in
`pointers`; `long_text` when an element's text changed past the 2,000
characters a text check compares, the elements in `pointers`; `cut_view`
when the views hold only part of the file; `unreadable_view` when no
check can read it, as a document REA reads as binary;
`volatile_lines` when a text file gained lines holding a value the app
generates with no text before it to count them by; `generated_reference`
when an array references a newly generated record ID that cannot be checked
exactly across runs, with up to five affected `pointers`; and `read_as_text`
when a text file opens like XML, with a declaration, a DOCTYPE or an
element, but reads as text because its DOCTYPE has an internal subset or
it is not well formed, so its checks compare lines that hold attributes in
the order the app wrote them.
The suggestions are a starting point. Keep the ones that say what the user
did; drop the ones that only follow from how the app saves its file. A LibreCAD
did; drop the ones that only follow from how the app saves its file. Read
`values` for what changed and its pointers, and `refers_to` for what a number
stands for, before writing the prompt and any check of your own. A LibreCAD
capture that added four holes to a drawing rewrote it from DXF R12 to DXF 2007,
so most of its lines changed; the draft suggests counts of `SECTION`, `ENDSEC`
and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
@@ -80,7 +503,7 @@ and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
```json
{
"name": "add-milk",
"prompt": "Add buy milk to my to-do list in Mousepad and save it.",
"prompt": "In Mousepad, add buy milk to my to-do list and save it.",
"checks": [
{
"id": "milk",
@@ -93,31 +516,35 @@ and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
```
`name` is a portable bundle name other than `smoke` (lowercase letters, digits
and hyphens); `prompt` is what the user asks for; `checks` holds 1 to 32
checks. Each check has a `kind`, a `file` given as a path under the home such as
and hyphens); `prompt` is what the user asks for, naming the app, since a
solving agent's own shell cannot see the computer's files and the app's name
sends it to the screen; `checks` holds 1 to 32
checks. Each file check has a `kind`, a `file` given as a path under the home such as
`Documents/todo.txt`, and an optional `id` (`check-1`, `check-2` and so on by
position otherwise). A check reads the typed view `data/get` returns for that
file, in the shapes of the state view.
| Kind | Fields | Passes when |
| --------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------ |
| `file_present` | | The file exists |
| `file_absent` | | The file does not exist |
| `lines_present` | `lines` | Every listed line is a whole line of the text view |
| `lines_absent` | `lines` | None of the listed lines is |
| `line_count` | `exact` or `contains`; `equals`, `at_least` or `at_most` | The number of lines equal to `exact`, or holding `contains`, compares so |
| `value_equals` | `pointer`, `value`; optional `tolerance` | The pointer names a value equal to `value`, or within `tolerance` of it |
| `value_absent` | `pointer` | The pointer names no value |
| `text_equals` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's [whole text](#whole-text) equals `value` |
| `text_contains` | `pointer`, `value`; optional `normalize_whitespace` | The XML element's whole text holds `value` |
| `row_present` | `table`, `where` | A row of the SQLite table has every column in `where` equal |
| `row_absent` | `table`, `where` | No row does |
| `row_count` | `table`; optional `where`; `equals`, `at_least` or `at_most` | The number of rows, or of rows matching `where`, compares so |
| Kind | Fields | Passes when |
| --------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `file_present` | | The file exists |
| `file_absent` | | The file does not exist |
| `lines_present` | `lines` | Every listed line is a whole line of the text view |
| `lines_absent` | `lines` | None of the listed lines is |
| `line_count` | `exact` or `contains`; `equals`, `at_least` or `at_most` | The number of lines equal to `exact`, or holding `contains`, compares so |
| `value_equals` | `pointer`, `value`; optional `tolerance` | The pointer names a value equal to `value`, or within `tolerance` of it |
| `value_absent` | `pointer` | The pointer names no value |
| `text_equals` | `pointer`, `value`; optional `normalize_whitespace`, `text_attribute` | The XML element's [whole text](#whole-text) equals `value` |
| `text_contains` | `pointer`, `value`; optional `normalize_whitespace`, `text_attribute` | The XML element's whole text holds `value` |
| `member_count` | `pointer`; optional `matching`; `equals`, `at_least` or `at_most` | The number of [members of the array](#counting-members), or of those `matching` names, compares so |
| `row_present` | `table`, `where` | A row of the SQLite table has every column in `where` equal |
| `row_absent` | `table`, `where` | No row does |
| `row_count` | `table`; optional `where`; `equals`, `at_least` or `at_most` | The number of rows, or of rows matching `where`, compares so |
The rules both evaluators follow:
- An absent file reads as empty: no lines, values or rows. So `lines_absent`,
`value_absent` and `row_absent` pass on it and a count is 0.
`value_absent` and `row_absent` pass on it and a count of lines or rows is
0; `member_count` fails, since its pointer names no array.
- A file that has no view (binary, over the read limit, or unreadable) fails
every check but `file_present` and `file_absent`. Line checks need a text
view, row checks a SQLite view and text checks an XML view, in a file or an
@@ -129,7 +556,9 @@ The rules both evaluators follow:
`{"groups": {"GROUP": {"KEY": "VALUE"}}}`, so a setting is
`/groups/GROUP/KEY`; XML as `{"root": {"name", "attributes", "children"}}`;
SQLite as `{"tables": {"TABLE": {"columns", "rows"}}}`; a zip archive as
`{"entries": [{"name", "format", "view", …}]}`.
`{"entries": [{"name", "format", "view", …}]}`; a gzip file, such as a
Gnumeric `.gnumeric`, as the document inside it, so its pointers are those
of the same document uncompressed.
- With `tolerance`, a non-negative number, `value` must be a number, and the
value found passes when it is a number, or a string that is a decimal
numeral such as `"5.0000000000000000"`, at most `tolerance` from `value`.
@@ -141,20 +570,106 @@ The rules both evaluators follow:
- A check decided by what a view leaves out (`lines_absent`, `value_absent`,
`row_absent`, the counts, `value_equals` of a whole object or array, and
the text checks of an element) fails when that view is truncated; a check
satisfied by what the view holds is decided on it.
satisfied by what the view holds is decided on it. An element's text is not:
the reader may have dropped text from between the runs it kept, so
`text_contains` fails on a truncated view even where the kept runs hold its
value. A text node or attribute value is whole, and decided.
The export refuses, naming the check, a file outside the owned paths, a file
spelled with a leading slash, `.` or `..`, a pointer into a file that is text
at the seed, as the bundle ships it, or at the end checkpoint, the empty
pointer (the whole file), a value that names the captured home, or the
pointer (the whole file) except in `member_count`, a `matching` that is no
selector, a value that names the captured home, or the
capture host's name or `user@host` anywhere (which the
[export removes from the seed](desktop-application-export.md#host-name-and-user)), a
`tolerance` with a value that is not a number, an empty `text_contains`
value, or one of only whitespace with `normalize_whitespace`, and a value that
looks like a date and time, in ISO 8601 or `asctime`'s form. Other volatile
value, or one of only whitespace with `normalize_whitespace`, an empty
`text_attribute`, a count without exactly one of `equals`, `at_least` and
`at_most`, and a value that looks like a date and time, in ISO 8601 or
`asctime`'s form. Other volatile
content, such as a
counter the app bumps on every save, is for the author to leave out.
## Readers
For a file REA cannot interpret itself, such as an app's binary document,
the task can define `readers`: 1 to 8 commands using the app's own reading
capability. Each reader has a unique `id` of lowercase letters, digits and
hyphens, a home-relative `file`, an `argv` array and `output`, either `json`
or `lines`. Its `argv`, or optional `stdin` text, must contain `{file}`;
the runner replaces it with the path of a private copy of that file.
The command must be installed where the export runs and in the exported
computer where the task will be graded.
The command prints its answer in lines starting with `REA `; other stdout
is ignored. With `output: "json"`, those marked lines join into one JSON
value for `value_equals`, `value_absent` and `member_count`. With
`output: "lines"`, they supply the lines for `lines_present`, `lines_absent`
and `line_count`. A check selects a reader with `"reader": "READER-ID"`
and must give that reader's file as its `file`. Every reader must be used by
at least one check.
Readers execute commands, including any code in `argv` or `stdin`; review
them before exporting or running a task. The runner uses a private copy, a
fresh home, no session bus or computer control token, and no display unless
the reader requests `display: true`. This is execution isolation from the
running app's normal profile, not a filesystem sandbox. `timeout_s` defaults
to 60 seconds and accepts 1 to 300. A failed command, timeout, malformed
answer or missing marked output reports a reader error; it does not establish
that the document lacks the expected value. Correct the command or its output and retry; increase
`timeout_s` within its accepted range when the reader needs longer.
A source inspection failure, such as an inaccessible parent or a symlink loop,
also reports a reader error and leaves affected checks undecided. Only a missing
source or a source that is not a regular file is reported as absent.
Export runs readers on both retained ends, using the seed bytes as the bundle
actually ships them. The witness and exported verifier run them in the
computer after the app's files settle. Cleanup failures retain the reader's
reported resources: retry the check or close the build computer to retry
cleanup; when the command's outcome is unknown, confirm it stopped before
removing its folder.
## Screen judges
A task whose result exists only on screen can use `screen_judge` checks:
```json
{
"name": "show-list",
"prompt": "In Notes, show the shopping list.",
"checks": [
{
"id": "shown",
"kind": "screen_judge",
"question": "Does the window show the shopping list?"
}
]
}
```
Each question must be a nonempty yes-or-no question of at most 2,000
characters, false on the seed and true when the task is done. A screen task
contains only screen checks and takes no readers. Export refuses it when an
owned file's readable change could be checked instead, or when the build
computer's seed or end screenshot is missing or exceeds 2 MiB. Capture those
checkpoints in a build computer before exporting.
Export ships the two screenshots as labelled failing and passing examples;
it does not call the judge or establish that the questions distinguish them.
The [build witness](desktop-build-witness.md) runs the exported judge on a
fresh screenshot at each stage. The verifier takes a screenshot after the
solving agent stops. For each question, the judge sees both examples and
votes three times; a majority of yes answers passes the check.
The judge uses `[model.roles] judge` in the runner's
`.agentenv/config.toml`, then `[model] default`, then
`anthropic/claude-sonnet-5-5`. Configure its endpoint with
`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or `[model] base_url` and `api_key`.
`agent-env run --judge-model` does not reach this verifier. Without an
endpoint, grading raises and the run fails rather than scoring the agent;
configure the endpoint and rerun the witness or task.
## Pointers and selectors
An app that rewrites a document moves things: FreeCAD 1.0 records more
@@ -168,9 +683,34 @@ rather than by its position:
`[name=Object]`.
- `[@KEY=VALUE]` names the member whose `attributes` hold `KEY="VALUE"`, as
the XML view keeps an element's attributes.
- `[>PATH=VALUE]` names the member that holds `VALUE` at a path of one to
three `/`-separated steps inside it. `>` reads XML and JSON differently:
For XML, `[>text()=VALUE]` compares the member's own whole text, with
whitespace collapsed, to `VALUE`. In JSON, `text()` remains a field name.
| Form | In an XML element | In a JSON record |
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
| `[>CHILD=VALUE]` | a child element `CHILD` whose text, every text node in it with each run of whitespace collapsed to one space and none at either end, is `VALUE` | the string field `CHILD`, compared whole |
| `[>A/B=VALUE]`, up to `[>A/B/C=VALUE]` | a child `A` with a child `B` of that text | the string at `member.A.B`, compared whole |
| `[>A@ATTR=VALUE]`, `[>A/B@ATTR=VALUE]` | a child `A` (or its child `B`) whose attribute `ATTR` is `VALUE` | the field named `A@ATTR`: `@` is part of a JSON key |
Records that keep their fields as elements are named by a child's text:
Osmo's `<contact><first_name>Delphine</first_name>…` is
`[name=contact][>first_name=Delphine]`. A GtkBuilder `<child>` is named by
the id its widget carries, `[name=child][>object@id=submit_button]`, and a
TreeLine node by a field of its `data`,
`[format=BOOK][>data/Title=The Silk Roads]`. Any element on the way will
do; a path with more than three steps, or an empty one, is no selector;
and a missing step, or a value at its end that is not a string, matches
nothing.
- Conditions can follow one another, `[name=Object][@name=Hole]`, and all must
hold. They compare strings only, the selector is escaped like any pointer
segment (`~1` for `/`), and a value cannot hold `][`.
segment, so a path's `/` is `~1` inside a pointer,
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and a `/` in a key
cannot be a step; a value cannot hold `][`. A JSON field whose key begins
with `@` or `>` is selected through a path, `[>@type=Book]`.
A selector must name exactly one member. When several match, or none does
before the last segment, the check fails and says which selector matched what.
@@ -184,7 +724,7 @@ FreeCAD 0.21 and 1.0 alike:
```json
{
"name": "cut-hole",
"prompt": "Cut a 10 mm hole through the middle of the plate and save it.",
"prompt": "In FreeCAD, cut a 10 mm hole through the middle of the plate and save it.",
"checks": [
{
"id": "cut",
@@ -291,11 +831,56 @@ read the element's whole text instead:
and trims both ends before comparing, as XPath's `normalize-space()` does,
for an app that reflows text or a value written across lines. A no-break
space is not whitespace there.
- `text_attribute` reads text an app keeps in an attribute: the element's
text is then that attribute's values on it and its descendants, joined in
document order, and its text nodes are left out. Scribus keeps a text
frame's text as `<StoryText><ITEXT CH="Spring "/><ITEXT CH="Fair"/></StoryText>`,
one `ITEXT` per run of formatting, so the check below reads `Spring Fair`
however the runs are split. The pointer must name an element; a text node
or attribute value fails.
```json
{
"id": "headline",
"kind": "text_equals",
"file": "Documents/flyer.sla",
"pointer": "/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]",
"value": "Spring Fair",
"text_attribute": "CH"
}
```
## Counting members
`member_count` counts the members of the array its pointer names: a JSON
array, or an XML element's `children`. `matching`, a selector of one or more
conditions as [above](#pointers-and-selectors), `[>PATH=VALUE]` included,
counts only the members it matches, however many; without it every member
counts. An element's children hold its text nodes too, so give `matching`
for XML, such as `[name=entry]`. A Tellico task that adds one book to a
collection of 12:
```json
{
"id": "one-more-book",
"kind": "member_count",
"file": "Documents/books.tc",
"pointer": "/entries/[name=tellico.xml]/view/root/children/[name=collection]/children",
"matching": "[name=entry]",
"equals": 13
}
```
It takes exactly one of `equals`, `at_least` and `at_most`. The empty
pointer `""` names the whole file, for a JSON file whose top level is an
array. A pointer that names no array, an absent file's included, fails and
says what it found there. Like the other counts, it is undecided on a view
that may hold only part of the array.
## Validation and export
`rea export-desktop-application <capture> <directory> --task <task.json>`
evaluates the checks on the capture's state view: on each file as it was at
evaluates file checks on the capture's state view: on each file as it was at
the seed checkpoint, which is the seed, as the bundle ships it with the
capture host's name and user removed, and at the end checkpoint, the
captured end state. Those are the capture's first and last checkpoints unless
@@ -304,9 +889,7 @@ It refuses a task whose every check passes on the seed, and a check that fails
on the end state, with that check's reason. Only a failure on what the seed's
view shows counts: a task whose checks fail on the seed only because a view
could not be read or may have left out what they look for, such as a line past
the 5,000 a text view keeps, is refused, naming each failure. A capture yields
one task, so the option is taken once. The result's `task` names the task, its
number of checks and those that fail on what the seed shows. A refused task
the 5,000 a text view keeps, is refused, naming each failure. A refused task
writes no bundle.
The bundle then holds, besides the app and `tasks/smoke.json`:
@@ -315,17 +898,56 @@ The bundle then holds, besides the app and `tasks/smoke.json`:
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `tasks/<name>.json` | Deploy the app, deploy the agent and its skills, record the screen, prompt the agent, run the verifier, stop recording |
| `artifacts/<name>-check/verify.py` | Reads the app's `data/get` through the computer and evaluates the checks, one result per check |
| `evals/<env>.toml` | The task, so a run of the bundle runs it; the smoke task runs with `--task smoke` |
| `evals/<env>.toml` | Every task, so a run of the bundle runs them all; the smoke task runs with `--task smoke` |
The verifier finds the app among the computer's child environments, calls its
`data/get` under the run's grant of the computer's control token, and reads
the `state` data part. It raises when the app cannot be reached, so the run
fails rather than scoring the agent. The README lists the task, its prompt, its
the `state` data part. It reads again until the app's files settle, as
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved)
describes, and each result's `settled` says whether they did, and `read_cut`
whether the bound cut a read still running. It raises when the app cannot be reached, so the run
fails rather than scoring the agent. The README lists each task, its prompt, its
checks and the recipe below.
### Several tasks
One capture can yield several tasks: repeat `--task`, one file per task, each
with its own name. Do each task's change on top of the last one in the same
capture and take a checkpoint once it is saved; the first task's end is the
end checkpoint, and `--task-end NAME=CHECKPOINT`, repeatable, names another
task's. Each task is checked at its own end, so a task whose check fails there
is refused, naming the task and the checkpoint, even when it passes at the
last one; its checks should not read what a later task changes. A checkpoint
named `near-NAME`, when the capture took one, is that task's near miss: a
plausible wrong edit on top of its change, such as a typo. The export refuses
the task unless one of its checks fails on what the near miss's views show.
```sh
rea export-desktop-application ./notes-capture ./notes-bundle \
--seed-checkpoint seed --end-checkpoint end \
--task add-eggs.json --task add-bread.json --task-end add-bread=end-add-bread
```
Two tasks of one name, a `--task-end` naming none of the tasks or not in
`NAME=CHECKPOINT` form, and a checkpoint the capture did not take once at or
after the seed are refused. The result's `tasks` lists each task: its name,
its number of checks, its end checkpoint, those that fail on what the seed
shows, and its near miss's checkpoint, or null, with those that fail there.
`describe-desktop-application-task` drafts a later task from the end before
it: `--seed-checkpoint end --end-checkpoint end-add-bread`.
A task's readers run at the seed as the bundle ships it, at its own end and
at its near miss. A screen-judged task ships the seed's screenshot and its own
end's as the judge's examples, and is refused when a file shows its change
since the end before it, or the seed for the first; its near miss is left to
the witness, so its `near_checkpoint` is null.
## Solve rate
REA does not run agents. To measure how often an agent solves the task, run it
REA's CLI runs no agent. On a build host, the MCP tool `measure_build_task`
measures a bundle's task for a model, records it in the bundle's
`difficulty.json`, and keeps the task in the band when it passes 1 to 3 of 5;
see [measuring a task](desktop-build-witness.md#measure). By hand, run the task
k times, three by default, and count the runs whose summary line says
`passed`:
@@ -336,7 +958,10 @@ done
```
The agent defaults to Claude Code and the model is the caller's choice (Claude
Sonnet 5.5 is a reasonable reference). A run reported as `scored below 1`
Sonnet 5.5 is a reasonable reference). The agent calls a model endpoint, and
agentenv refuses the run without one: set `LITELLM_BASE_URL` and
`LITELLM_API_KEY` where the runs start, or `base_url` and `api_key` under
`[model]` in `.agentenv/config.toml`. A run reported as `scored below 1`
failed the checks and counts as a miss; one reported as `failed` did not finish,
as when the computer or the app could not be reached, and is run again rather
than counted. Each run's instance keeps the verifier's results per check under
+189 -27
View File
@@ -14,15 +14,70 @@ agent's computer-use actions to it through three MCP tools:
The tools are MCP-only, like the debugger sessions: the computer lives as long
as the REA MCP server that started it, or until it is closed. Capturing the
app, exporting its bundle and witnessing the task on such a computer are in
[Building and witnessing an environment](desktop-build-witness.md).
[Building and witnessing an environment](desktop-build-witness.md), and REA's
skill for doing all of it from a request is
[desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Set up a build host
On a Linux host with Node.js and Docker, from a REA
[checkout](getting-started/installation.md#run-rea-from-a-checkout):
```bash
rea setup --client claude_code --desktop-build --install-agentenv
rea doctor --provider desktop_build --client claude_code
```
Setup shows its plan and asks before it changes anything; `--dry-run` only
prints it and `--yes` approves it. It packs REA's runtime archive into
`~/.cache/rea/build-computer/rea-<version>.tar.xz` (under `$XDG_CACHE_HOME`
when set, `~/Library/Caches` on macOS), and packs again once REA's version
changes; an archive `REA_BUILD_COMPUTER_REA_ARCHIVE` already names is
recorded instead. It records `REA_AGENT_ENV_COMMAND`, `REA_DOCKER_COMMAND` and
`REA_BUILD_COMPUTER_REA_ARCHIVE` as absolute paths in that client's REA
registration, keeping a value the registration already has other than an
archive setup packed for an earlier REA version, and refreshes the skill.
`--install-agentenv` runs `uv tool install agentenv-framework==0.9.1485`
when `uv` is on `PATH`, and otherwise prints the command. Setup never
installs Docker or Node.js and never runs `sudo`. Restart the client
afterwards.
Doctor is read-only and checks, in order: a Linux amd64 or arm64 host,
Node.js, procps `ps`, the Docker CLI, a Docker daemon reachable without
`sudo`, 20 GB free where Docker keeps images, agentenv-framework 0.9.1485,
the runtime archive of this REA version, the build settings in the
registration or the environment, and the skill. Each failure names its fix.
It also reports, without counting them against `healthy`, whether a model
endpoint for solver runs is configured and which agentenv containers
earlier builds left. Pass `--json` for the full report.
When a setting is unset where the server starts, REA uses `agent-env` and
`docker` on `PATH` and the archive setup packed for this REA version. The
rest of this page is the same setup by hand.
## Requirements
- Docker, and an agentenv-framework install whose `agent-env` program is named
by the absolute `REA_AGENT_ENV_COMMAND`. Bundles target agentenv
0.9.1431. REA runs its helper, `bridge/agentenv/computer.py`, with that
runner's Python: the interpreter on the program's `#!` line, or the
`python3` beside it in its virtual environment.
- A Linux host on amd64 or arm64 with about 20 GB of free disk for the
images agentenv builds and pulls. amd64 and arm64 hosts both build
natively; a host builds only its own architecture.
- Node.js at the version in `.nvmrc`, installed with
[nvm](https://github.com/nvm-sh/nvm) or from
[nodejs.org](https://nodejs.org/en/download), and REA
[run from a checkout](getting-started/installation.md#run-rea-from-a-checkout).
- [Docker Engine](https://docs.docker.com/engine/install/) and its
[post-install steps](https://docs.docker.com/engine/install/linux-postinstall/):
add yourself to the `docker` group, then log in again.
- agentenv-framework 0.9.1485, the version bundles target. Install
[uv](https://docs.astral.sh/uv/getting-started/installation/) first, then
run `uv tool install agentenv-framework==0.9.1485`. Set
`REA_AGENT_ENV_COMMAND` to the absolute path of the resulting `agent-env`
program, normally `$HOME/.local/bin/agent-env`; `command -v agent-env`
shows the path when its install directory is on your `PATH`. Installing it
with pip into a virtual environment instead needs `python3.12-venv` on
stock Ubuntu. REA runs its
helper, `bridge/agentenv/computer.py`, with that runner's Python: the
interpreter on the program's `#!` line, or the `python3` beside it in its
virtual environment.
- `REA_DOCKER_COMMAND`, the absolute path of the Docker CLI, lets REA remove a
computer whose agentenv teardown was not confirmed. Without it, REA names the
command to run instead.
@@ -32,20 +87,83 @@ app, exporting its bundle and witnessing the task on such a computer are in
checkout, `npm run pack:build-computer-runtime -- /path/to/rea-prod.tar.xz`
writes one: `npm pack`, its package folder as
`rea-prod` with the lockfile, `npm ci --omit=dev --omit=optional`, then
`rea-prod` as `.tar.xz`.
`rea-prod` as `.tar.xz`. The archive is a copy of the checkout: pack it
again after `git pull` and a rebuild.
Set these where the MCP server starts; a shell export does not reach a server
that is already running.
## Claude Code
Without `--desktop-build`, `rea setup --client claude_code` registers REA
without these variables. Either
export them in the shell that starts `claude`, which starts REA's server with
its environment:
```bash
export REA_AGENT_ENV_COMMAND="$HOME/.local/bin/agent-env"
export REA_DOCKER_COMMAND="$(command -v docker)"
export REA_BUILD_COMPUTER_REA_ARCHIVE="$HOME/rea-runtime/rea-prod.tar.xz"
claude
```
or set them, as absolute paths, in the `env` of the `mcpServers.rea` entry
setup wrote, such as in `~/.claude.json`:
```json
{
"mcpServers": {
"rea": {
"command": "/absolute/path/to/node",
"args": ["/absolute/path/to/rea/scripts/rea.mjs", "mcp"],
"env": {
"REA_AGENT_ENV_COMMAND": "/home/ada/.local/bin/agent-env",
"REA_DOCKER_COMMAND": "/usr/bin/docker",
"REA_BUILD_COMPUTER_REA_ARCHIVE": "/home/ada/rea-runtime/rea-prod.tar.xz"
}
}
}
}
```
`measure_build_task` runs solves through agentenv, which refuses them without
a model endpoint: export `LITELLM_BASE_URL` and `LITELLM_API_KEY` in the shell
that starts `claude`, whose REA server inherits them, or set `base_url` and
`api_key` under `[model]` in `.agentenv/config.toml`. REA passes them on to
agentenv and never reads the key.
Claude Code asks before each MCP tool call its permission settings do not
allow, and a build makes dozens of `act_on_build_computer` calls. You can
allow REA's tools in those settings' `permissions.allow` list:
`mcp__rea__act_on_build_computer` allows that tool, and `mcp__rea` allows all
of REA's.
A headless run, `claude -p`, ends when the skill asks its upfront questions,
and refuses the tool calls its permission settings do not allow. Put the
answers in the request: the install source when the app is not in the Ubuntu
archive, a synthetic seed or the file to start from, whether to review the
task before it is measured, and the model and budget to measure it with, or
none:
```bash
claude -p 'Build me a Gnote environment where the agent adds olive oil to the
shopping list note. Make a synthetic note. Then measure it with <model>,
at most $4 in all, without a review first.'
```
## Start a computer
From a request, REA renders a bundle in a private temporary folder: a computer
app on the `ubuntu-24.04` base with the `reference` driver that installs the
app, opens it on demand with `gtk-launch <desktop entry>` from the computer
user's home, so its open and save dialogs start there rather than in the
framework's folder for the app, and keeps no seed and no data hooks. The app is not open when the computer starts: agentenv lists it
in the desktop's Applications menu under Other by the environment's name, such
as `mousepad`, and that entry launches it through the app's server.
framework's folder for the app, and keeps no seed and no data hooks. The app is not open when the computer starts:
`start_build_capture` opens it under REA's capture. When no capture is under
way, open it from the desktop's Applications menu, where agentenv lists it
under Other by the environment's name, such as `mousepad`; that entry launches
it through the app's server. The name is the desktop entry's last part when it
is reverse-DNS (`com.github.xournalpp.xournalpp` gives `xournalpp`), or else
the whole entry without a trailing version (`glabels-3.0` gives `glabels`).
When the app is Electron, set `electron` to true. The install rewrites that
desktop entry's command with the same policy an export uses: `--no-sandbox`
@@ -69,16 +187,51 @@ Applications menu entry can open the app.
| `kind` | Installs |
| ---------------- | ---------------------------------------------------------------------------------------------------------- |
| `archive` | `package` from the Ubuntu 24.04 archive, on amd64 and arm64 |
| `archive` | `package` from the Ubuntu 24.04 archive, on amd64 and arm64, with its optional `extra_packages` beside it |
| `apt_repository` | `package` from the repository at `uri` with `suite` and `components`, trusting the key file `key_file` |
| `package_file` | The `.deb` at `url`, checked against `sha256`, for one `architecture`, after its apt `dependencies` are in |
| `appimage` | The AppImage at `url`, checked against `sha256`, for one `architecture`, extracted with its own entry |
| `tarball` | The archive at `url`, checked against `sha256`, for one `architecture`, with an entry running `program` |
Build steps run with the network off, so a `.deb`'s dependencies must be
listed. The bundle adds REA in a build-only layer: Node.js
listed.
An AppImage or a tarball is downloaded by agentenv when the bundle is put,
refused unless its SHA-256 is the one named, and unpacked by a root build
step, `install-download.sh`, under `/opt/rea-apps/<desktop_entry>`; the
export writes the same step. An AppImage is extracted with its own
`--appimage-extract`, so no FUSE is needed, and the step installs the image's
own `<desktop_entry>.desktop` under `/usr/local/share/applications` with
each `Exec` program replaced by the image's `AppRun`, after any `env` prefix
and its assignments, without `TryExec`, and its icon; an image without that entry fails the build, naming the entries it
has. A tarball is read with Python's `tarfile`, since the computer has no
`xz` program, so gzip, bzip2 and xz archives work. Its `data` filter refuses
links that point outside the folder, absolute ones included, and special
files, so such an archive fails the build. Its `program` is a path
inside the archive, and REA writes the entry: `name` in the menu, or the
entry's last part, and `program` with the files it opens (`%F`). An
Electron app's command is rewritten after this step, on the entry it
installed. An image or tarball holds no Debian package, so the capture
records its package as `unknown`; the export checks instead that the
captured program lies under the folder the download unpacks to. The
downloaded file stays in the image's layer under the app's folder beside
what it unpacks, so a 721 MB AppImage that unpacks to 3.0 GB adds both. `extra_packages` names up to 16 more archive packages, by the same
name rules as `package`, for a runtime piece the app needs that its package
does not install: apt here skips Recommends, so MyPaint 2.0.1 exits at launch
without `librsvg2-common`, GTK's SVG icon loader, and Freeplane 1.7.10 needs
`openjdk-17-jre`, since its launcher refuses noble's default Java 21. The
export installs them beside the app too, as it does a `.deb`'s
`dependencies` when apt offers the `.deb`'s package; see
[build and witness](desktop-build-witness.md).
The bundle adds REA in a build-only layer: Node.js
24.22.1, downloaded per architecture and checked against its release checksum,
and REA from `REA_BUILD_COMPUTER_REA_ARCHIVE`, both unpacked into `/opt/rea` by
a root build step, with `node` and `rea` on `PATH`. An exported bundle carries
none of it.
a root build step, with `node` and `rea` on `PATH`. It also installs
`libgtk-3-bin`, for `gtk-launch`, and `dconf-cli`, so the capture reads the
app's settings in dconf, such as GnuCash's "Show tips at startup", and the
seed keeps them. An exported bundle carries none of it but `dconf-cli`, which
it installs when it loads the app's dconf keys.
From a bundle folder, such as one `export-desktop-application` wrote, REA
starts that bundle's `tasks/smoke.json` as it is:
@@ -94,6 +247,10 @@ in the folder and its files stay as they were shared.
REA runs the deploy with `run_bundle(keep=True, emulate=False)` on the local
sandbox provider, so only the host's own architecture is built. The first start
of a bundle builds its image and can take minutes; later starts reuse it.
agentenv makes a start wait while another agentenv run on this host writes its
bundle's ids, so starts at the same time wait for each other, several minutes
when a few run together; the progress line then says it is waiting for another
agent-env run to finish writing this bundle's ids.
While it waits, REA sends an MCP progress notification every five seconds with
agentenv's latest line, so a client that resets its request timeout on
progress keeps waiting. REA then waits until the computer's readiness checks
@@ -103,9 +260,12 @@ while it is one colour throughout. The retries share 10 s, each given only
what is left of it, and when that runs out REA returns the latest frame;
progress notifications continue meanwhile. Given `screenshot` with a
`region`, `format` or `quality`, REA then takes one with them, a `region`
cropping that frame. The result holds
cropping that frame rather than capturing the screen again. The result holds
`computer_id`, agentenv's `sandbox_id`, the environment name or bundle folder,
and the screenshot's frame as structured content beside its image.
`extra_packages`, what the request named beside the app for the export to
install too (an archive install's `extra_packages` or a `.deb`'s
`dependencies`, empty otherwise), and the screenshot's frame as structured
content beside its image.
A failed start returns the last 16 KiB of the helper's stderr, where a failing
build prints its error, and its exit status; a cancelled one stops the helper,
@@ -119,16 +279,16 @@ progress and waits for them before closing every computer.
`act_on_build_computer` takes `computer_id`, an `action` and the driver's own
arguments for `computer_<action>`, which REA passes through:
| `action` | Arguments |
| ------------ | --------------------------------------------------------------- |
| `screenshot` | `region`, `format` (`png` or `jpeg`), `quality` |
| `click` | `x`, `y`, `button`, `count` (2 double-clicks), `modifiers` |
| `drag` | `path` of `[x, y]` points, `button`, `duration_ms`, `modifiers` |
| `scroll` | `direction`, `x`, `y`, `amount`, `modifiers` |
| `type` | `text`, where `\n` presses Enter and `\t` Tab |
| `key` | `keys`, one chord pressed in order, and `repeat` |
| `wait` | `ms` |
| `reset` | `screenshot` |
| `action` | Arguments |
| ------------ | ----------------------------------------------------------------------- |
| `screenshot` | `region`, `format` (`png` or `jpeg`), `quality` |
| `click` | `x`, `y`, `button`, `count` (2 double-clicks), `modifiers` |
| `drag` | `path` of 2 to 50 `[x, y]` points, `button`, `duration_ms`, `modifiers` |
| `scroll` | `direction`, `x`, `y`, `amount`, `modifiers` |
| `type` | `text`, where `\n` presses Enter and `\t` Tab |
| `key` | `keys`, one chord pressed in order, and `repeat` |
| `wait` | `ms` |
| `reset` | `screenshot` |
`reset` is not a driver tool: REA calls the app's `data/reset`, which restores
the bundle's seed and relaunches the app, then waits until the computer's
@@ -152,7 +312,9 @@ The result carries the driver's content unchanged, its text and images, after
REA's own JSON: `action_index` and the driver's structured result. A refusal,
such as a stale frame, is an error carrying the computer's message. A response
larger than the MCP budget is refused with how to ask for a smaller image: a
`region`, or `jpeg` with a lower `quality`. That error keeps REA's own result,
`region`, or `jpeg` with a lower `quality`. A screenshot with `region` does not
capture the screen again; it crops the latest frame, so take a full
screenshot, or a `wait` with `screenshot: true`, first. That error keeps REA's own result,
`computer_id` included, as `details.result`, since the action or start has
happened.
+231 -41
View File
@@ -7,21 +7,31 @@ it writes, and then witnesses the task in a fresh computer of that bundle.
REA runs its own capture, task draft and export inside the build computer and
grades the witness with the same checks the exported verifier runs.
| Tool | What it does |
| -------------------------- | ------------------------------------------------------------------------------------------- |
| `start_build_capture` | Downloads the package lists as the computer's user and launches the app under REA's capture |
| `mark_build_checkpoint` | Takes a named checkpoint and returns the capture's acknowledgement with its counts |
| `stop_build_capture` | Stops the capture and returns its checkpoints and the task draft between `seed` and `end` |
| `export_build_environment` | Exports the bundle with the agent's task inside the computer and copies it to a folder here |
| `check_build_task` | Grades a task against the app's live state; with a `stage`, files the check in the witness |
| Tool | What it does |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `start_build_capture` | Downloads the package lists as the computer's user and launches the app under REA's capture |
| `mark_build_checkpoint` | Takes a named checkpoint and returns the capture's counts and the files that changed |
| `stop_build_capture` | Stops the capture and returns its checkpoints and the task drafts, from `seed` to `end` and on to each `end-<task name>` |
| `export_build_environment` | Exports the bundle with the agent's tasks inside the computer and copies it to a folder here |
| `check_build_task` | Grades a task against the app's live state; with a `stage`, files the check in the witness |
| `measure_build_task` | Runs a bundle's task in fresh computers for a model and records how hard it is in the bundle |
Like the computer tools, these are MCP-only and act on a `computer_id` from
`start_build_computer`.
REA's skill walks an agent from a request such as "build me an Inkscape
environment where the agent changes a poster's headline" to a witnessed
bundle with these tools: the questions it asks first, when it stops for the
person, the install, the seed, the task, the witness and its final report.
See [desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Build
1. `start_build_computer` from a request. REA is installed in the computer.
2. `start_build_capture`. REA writes an apt configuration under
2. `start_build_capture`. REA first checks the installed runtime's export
schema for synthetic-seed support. If the archive is too old, repack a
current REA, update `REA_BUILD_COMPUTER_REA_ARCHIVE`, restart the REA MCP
server and start a new build computer. REA then writes an apt configuration under
`~/.rea-build/apt` and runs `apt-get update` as the computer's user, so the
capture finds the package's sources (`package_lists` counts the lists,
and `package_lists_error` gives apt's exit status and last lines when the
@@ -32,6 +42,34 @@ Like the computer tools, these are MCP-only and act on a `computer_id` from
from a `.deb` file, the build layer keeps that file at
`/opt/rea/package.deb` and the capture records it with `--package-file`. Do not open the app from
the menu first: the capture refuses an app whose window is already open.
When the capture ends before the window opens, REA reads its output. An app
that exits at launch, such as one missing a runtime, is reported with its
program, exit code, run time and the last lines of its stderr, at most 8
lines and 800 characters of the last 8 KiB the capture keeps, and
`details.diagnostics` keeps the capture's record of the session. A capture
that failed, such as one refusing to start, gives its own message and
remediation, with the whole failure in `details.diagnostics.command_failure`
and, when its cleanup is incomplete, the run tokens and paths it left in
`details.resources`. A checkpoint or stop that finds the capture ended
reads its output the same way.
`state_paths` names files or folders in the computer's home that hold the
app's state, relative to the home, after `~/` or absolute under it, at
most 16; each becomes a `--state-path` of the capture, so it is retained
from the start and its files count as the app's even before they exist.
REA counts a file as a document only when an app window's title shows its
name or `~/` path, so name the file the app will save when its title shows
something else, as MuseScore's shows a score's title: a file outside
hidden folders, other than a lock such as LibreOffice's `.~lock.<name>#`,
becomes a document, which the bundle owns, seeds and opens
at launch, and one under a hidden folder such as `~/.config/MuseScore` the
app's own state, also when that folder is a link elsewhere in the home. A
path with `..`, or naming the home itself, is refused before the computer
is asked; the computer refuses an absolute path outside its home, one in
`~/.rea-build`, and one a link leads out of the home or into
`~/.rea-build`, resolving a path that does not exist yet through its
nearest existing folder, and starts nothing.
3. Make the seed through the app with `act_on_build_computer`, saving it
where the user keeps such files, such as `~/Documents`, then
`mark_build_checkpoint` `seed`.
@@ -39,33 +77,103 @@ Like the computer tools, these are MCP-only and act on a `computer_id` from
`mark_build_checkpoint` `end`.
5. `stop_build_capture`. The app is closed and the capture finishes. The
result lists the checkpoints with their counts, the package and the
sources the capture found, its limitations and the draft: each file that
changed between `seed` and `end` with its net change, and suggested
checks with a placeholder prompt.
6. Write the task: its name, the prompt in the user's own words, and checks
that hold after the change and not on the seed.
7. `export_build_environment` with the task and a new or empty `directory`.
sources the capture found, its limitations and the draft: each file under
the paths the export owns that changed between `seed` and `end`, with its
net change and, for XML files, archives, JSON and key files, its `values`,
the values the change added, changed and removed with what each was and
became, at the pointer a check names it by; suggested checks with a
placeholder prompt; and `guards`, optional checks that the rest kept its
seed values: like siblings' fields, each line kept beside new text, and
the count of a list whose members were only edited. The draft's
[task section](desktop-application-tasks.md#describe-the-change) describes
each. `notes`, each with a `code` and a `message`, say why a draft has no
files: `no_change` when no file under the home changed between them and
nor did the app's dconf keys; `unowned_changes` when files changed but
none under the owned paths, listing up to five in `files` with their
roles, desktop files last, and counting the rest in `more`;
`uncheckable_changes` when the app's keys in dconf were read and changed,
listing the database in `files`, since no check reads dconf (a version
the capture kept no copy of, as it keeps none of the database at the
start, is `unreadable_changes`, not an empty key list);
`unreadable_changes` when a dconf database changed but REA could not read
it, such as without `dconf-cli` or when a present version's bytes were
not retained, with the read `error`, so whose keys changed is unknown;
and `no_document` when no file has the document role.
`unowned_changes` and `no_document` point at `state_paths`. They also say,
for a drafted `file`, why its suggestions are thin: `by_position`,
`view_state_only`, `long_text`, `cut_view`, `unreadable_view`,
`volatile_lines`, `generated_reference` or `read_as_text`.
6. Write the task: its name, the prompt in the user's own words naming the
app (a solving agent's own shell cannot see the computer's files, and the
export's limitations say when a prompt names neither the app's title,
environment nor package), and checks
that hold after the change and not on the seed, from the suggestions,
`values` and any `guards` worth keeping.
7. `export_build_environment` with the tasks and a new or empty `directory`.
Each task is `{"task", "end_checkpoint"}`, its end `end` by default; a
later task's change made on top of the last, and marked `end-<task name>`,
is checked there, and `stop_build_capture`'s `later_drafts` drafts it from
the end before it. A checkpoint `near-<task name>` taken after a near miss
of that task's change must fail one of its checks. An end the capture did
not take is refused before anything runs.
The export runs inside the computer with `--seed-checkpoint seed
--end-checkpoint end`, so it seeds what was saved at `seed`, scrubs the
--end-checkpoint end` and each task's `--task-end`, so it seeds what was
saved at `seed`, scrubs the
computer's host name and user from it, and refuses a task whose checks
all pass on the seed. A task larger than 96 KiB as REA passes it to the
computer is refused before anything runs. A file moved to the desktop Trash is not seeded,
and the export's limitations name what the Trash held at `seed`. For a
all pass on the seed. It also passes `--synthetic-seed`, since the agent
made the seed for the task, so the README does not call it the user's real
data. When the seed holds exactly one document and the app's desktop entry
takes files, the bundle's app opens that document at launch. A file moved
to the desktop Trash is not seeded, and the export's limitations name what
the Trash held at `seed`. The limitations and the README also name the
folders where files changed during the capture that the bundle neither
owns nor seeds, such as settings that keep a first-run wizard away, up to
eight collapsed to their top folder, such as `~/.config/MuseScore`, and a
count of the rest; capture again with them in `state_paths` to own them.
Lock files the app holds while it runs, such as LibreOffice's profile
`.lock` and its document lock `.~lock.<name>#`, GnuCash's
`<book>.LCK` or Grisbi's `.<file>.lock` beside its file, are left out of
the seed, since a copied one can keep the app from starting or saving. The
limitations and README list them after "Left out of the seed". A project's
`Cargo.lock` or `yarn.lock`, a document or a path in `state_paths`, and app
logs that are not locks are seeded. The limitations name seeded
WebKit or Chromium storage and caches too, which the app keeps for its
own running. When the application held files open in a folder whose
starting content the capture did not retain, a limitation names the
folder and the seed files in it. For a
`.deb` app that no apt source the capture found offers, the export takes
the request's URL with `--package-url`; one apt offers installs from apt. REA lists the bundle's files with their sizes
and SHA-256 in the computer, into a file beside the bundle that it reads
through `data/get`, since a large bundle's list outgrows a command's
output; it then brings the files back through the driver's
`data/get` in answers of at most 8 MiB, and checks and writes each answer
into the folder before asking for the next, so one answer at a time is
held; a failure leaves no partial folder.
the request's URL with `--package-url`; one apt offers installs from apt.
Either way the request's `dependencies` reach the export as
`--extra-package`, as an archive request's `extra_packages` do, so a
runtime piece the agent named, such as MyPaint's `librsvg2-common`, is
installed beside the app; the limitations and the README list them,
except the app itself and dependencies the bundle already names for a
`.deb` it installs from the file. REA lists the bundle's files
with their sizes and SHA-256 in the computer, into a file beside the
bundle that it reads through `data/get`, since a large bundle's list
outgrows a command's output; it then brings the files back through the
driver's `data/get` in answers of at most 8 MiB, and checks and writes
each answer into the folder before asking for the next, so one answer at
a time is held; a failure leaves no partial folder.
8. `close_build_computer`.
A checkpoint name is lowercase letters, digits, dot, dash and underscore; the
capture's own `baseline`, `stop`, `time-limit` and `exit` are refused, and so
is a name already taken. The acknowledgement comes from the capture after it
has walked the home and retained what changed: `entries` walked, `changes`
since the checkpoint before, `retained_files` and `retained_bytes`.
since the checkpoint before, `retained_files` and `retained_bytes`, and up to
ten `changed_files`, home-relative, with `more_changed_files` counting the
rest. Folders are left out. Files an app window's title has named so far
come first, as a document saved and no longer held open is in Gnumeric, LyX
or Tiled; then files named themselves in `state_paths`; then those the app
holds open, those under a selected folder and those in its state
directories; then the rest; and caches and WebKit, Chromium or QtWebEngine
storage last, even under a selected folder such as `.config/libreoffice`
with its icon cache, each by path. So a save's document leads the list.
When the computer's REA is older than these lists,
as when `REA_BUILD_COMPUTER_REA_ARCHIVE` still names an earlier build, the
checkpoint is still taken and both are `null`: which files changed is
unknown. Pack the current REA into the archive to get them back.
Each step can be asked again after its answer was lost:
@@ -89,7 +197,9 @@ computer's control token. REA sends each command through agentenv's
`computer-exec/v1` route with that token, which stays in REA, and reads larger
outputs, such as the draft, the export's Evidence and the bundle's file
list, from files under `~/.rea-build` through `data/get`, since a command's
output is capped at 1 MiB.
output is capped at 1 MiB. In a command's Evidence or failure read this way,
as in its own output, REA replaces the token in every string, after checking
the Evidence's digest; the bundle's files arrive as they are.
Calls on one computer run one at a time, in the order REA receives them:
an action, a capture step, a check or a close waits until the call before
@@ -102,7 +212,12 @@ cancels the call in progress on it and refuses the calls still waiting.
## Witness
Start a fresh computer from the exported folder with `start_build_computer`
and `{"kind": "bundle", "path": ...}`; agentenv boots the app with the seed.
and `{"kind": "bundle", "path": ...}`; agentenv boots the app with the seed,
on the seed's document when the export opens one. When the computer never
gets ready and the bundle's seed holds lock files, the start's error names
them: delete them from the bundle, or export again, which leaves them out.
A bundle records no state directories, so the error names only files by a
profile lock's name and LibreOffice's document locks.
Then, with the task exactly as exported:
| `stage` | Before it | Its checks must |
@@ -115,13 +230,29 @@ Then, with the task exactly as exported:
`check_build_task` reads the app's `data/get`, takes the views it returns
beside the files and evaluates each check with REA's evaluator, the one the
export validates tasks with and the exported verifier mirrors, one result per
check. Without `stage` it only grades. The `seed` stage begins a recording
check. Once the files settle it runs the task's
[readers](specs/desktop-app-environments/file-readers/spec.md) in the computer through
`computer-exec/v1`, as the verifier does, and a check through a reader reads
that reader's view. A task of [screen judges](specs/desktop-app-environments/screen-judge/spec.md)
reads no files: REA takes a screenshot and runs the bundle's own judge on it
with the agentenv runner's Python and model endpoint. Like the verifier, it grades only once the app's files settle, since an
app such as Gnote saves a few seconds after the last key. It reads `data/get`
every 0.5 s and grades once every file's bytes have held for 3 s, counted
from the read that returned them. 15 s after the first read began, it grades
the last completed read: no read starts then, and one still running is cut.
It sends progress notifications while it waits. The result's `settled` says
whether the files settled, and `read_cut` whether the bound cut a read; each
stage in the record keeps both. Every check therefore costs at least 3 s,
and a save that lands more than 3 s after the first read returns is still
missed; see
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved).
Without `stage` it only grades. The `seed` stage begins a recording
of the display under REA's own request id, one per computer; a seed check
that fails after it, such as on a failed read of the app's state or a lost
reply to the recording's start, can be asked again, and asks for the
recording under the same id, which the computer answers with the recording
it already began. The stages come in order. The edit and the near miss each
need an applied click, drag, scroll, type or key since the stage before, with
need an applied click, drag, scroll, type, typed secret or key since the stage before, with
no reset after the last of them, since a reset puts the seed back rather than
changing the app; `after_reset` needs the reset as the last change, and a
reset that leaves any check unlike on the seed fails the witness, its
@@ -137,14 +268,17 @@ SQLite file locked past the reader's wait, or because what it looks for may
lie in what its view cut, is `undecided` in its result: it shows nothing
about the task. That covers a line `lines_present` misses in a text view cut
at 5,000 lines, a row `row_present` misses in a table whose view keeps only
its first rows, a `value_equals` pointer that names nothing in a truncated
view or past a cut zip listing, and the absence checks and counts on a cut
view. A check that passes on what the view kept stays decided, and so does a
value found there that differs from the one expected. The result's `passed`
and description are the exported verifier's, which has no undecided results.
A stage that only undecided checks would settle, such as a near miss whose
one failing check could not read its file, files nothing and is refused;
check it again once the files read whole.
its first rows, a `value_equals` or text check pointer that names nothing
in a truncated view or past a cut zip listing, the absence checks and counts
on a cut view, and `text_equals` or `text_contains` of an element on one,
even where the element's kept text holds the value, since the cut may have
dropped text from between its runs. Any other check that passes on what the
view kept stays decided, and so does a value found there that differs from
the one expected.
The result's `passed` and description are the exported verifier's, which has
no undecided results. A stage that only undecided checks would settle, such
as a near miss whose one failing check could not read its file, files nothing
and is refused; check it again once the files read whole.
A stage whose checks do not show what it must ends the witness, and so does
`after_reset`. REA then ends the recording, fetches its segments through
@@ -154,13 +288,14 @@ A stage whose checks do not show what it must ends the witness, and so does
- `recording/NNNN.mp4`, the recording's segments;
- `witness.json`, REA Evidence (`rea.desktop-build-witness`) holding the
verdict, the stage that failed if one did, the task's name and SHA-256,
each stage's results with how many logged actions came before it, the
each stage's results with how many logged actions came before it and
whether the app's files had settled (`settled`), the
computer's [action log](desktop-build-computer.md#act) as it keeps it, each
action with its arguments, timing and outcome, with `actions_dropped`
counting the earliest actions it no longer keeps, and the recording's
segments with their sizes and SHA-256. The record's limitations say when
the log dropped actions or cut arguments, which each entry names in
`truncated_arguments`.
`truncated_arguments`, and which stages graded files that had not settled.
The stages' rules read the log's count of actions and the last applied
action of each kind, which it keeps for the whole session, so they hold
@@ -169,7 +304,62 @@ folder something else wrote into, the witness keeps its stages: check the
stage that ended it again, with the same task, and REA writes the record
without checking again. Once it is saved, the same check returns the saved
record again, so a record saved as its call was cancelled is not lost. A
computer is witnessed once; witness again from a fresh computer.
computer is witnessed once; witness again from a fresh computer. A bundle of
several tasks is witnessed one task per fresh computer: the seed stage
refuses a task whose steps and verifier differ from the bundle's for that
task's name, so each task's record shows that task alone from the seed.
## Measure
`measure_build_task` takes a bundle folder, a task's name, a `model`, an
`agent` (`claude-code` by default) and `budget_usd`. It runs the task through
REA's helper, `bridge/agentenv/solve.py`, with the agentenv runner's Python on
this host, one solve at a time, each in a fresh computer the run tears down:
the bundle's own steps, with the agent deployed as the solver on the model.
The model endpoint is the one agentenv reads where REA starts,
`LITELLM_BASE_URL` and `LITELLM_API_KEY` or `[model]` in
`.agentenv/config.toml`; REA holds no model client and never reads the key.
Each solve uses a private agentenv sandbox root. If the helper stops before
reporting cleanup, REA terminates each sandbox agentenv lists under that root
(`local_sandbox_ids`), including one killed while it recorded its ports, and
keeps the root when cleanup fails. agentenv keeps a sandbox's folder until its
teardown is confirmed, so once that root is empty the solve's cleanup is
verified, report or not. On macOS, REA cannot verify host processes left by a
recovered sandbox.
| Solves finished | Passes | Band | Next |
| --------------- | --------- | ---------- | ------------- |
| 3 | 0 | `too_hard` | stop |
| 3 | 3 | `too_easy` | stop |
| 3 | 1 or 2 | | 2 more solves |
| 5 | 1, 2 or 3 | `in_band` | stop |
| 5 | 4 | `too_easy` | stop |
A solve agentenv reports `failed`, `unscored` or `cancelled` did not finish:
it is run again, at most twice, and not counted, though its cost is. Before
each solve the measure stops, `stopped: "budget"`, when the largest cost so
far would take the total past `budget_usd`; a solve's cost is known only when
it ends, from Claude Code's last trajectory record, so the first solve always
runs and a solve that reports no cost stops the measure. A solve whose helper
is stopped before it reports, by the solve's deadline or killed from outside,
counts as `failed` with no known cost, so the measure records it and stops. A
solve that could not run at all, such as for want of a model endpoint, ends the measure with
its error and records nothing.
The result and `difficulty.json`, at the bundle's root, keep per task: the
model, agent and time; the band and why it stopped; `k`, the finished solves,
the passes and unfinished solves; the cost in all and the budget; each
finished solve's turns; `failures_per_check`, how many finished solves failed
each check; each solve's outcome, turns, cost, seconds, failed checks,
instance id and the error agentenv reported for it; `task_sha256`, the
SHA-256 of the task's steps file followed by its verifier, so a record whose
task changed since shows it is stale; and
REA's and agentenv's versions. Measuring a task again replaces its record and
keeps the others. A `difficulty.json` or README REA cannot read or parse is refused
before any solve and left as it is, and measures of one bundle write one at a
time, so neither loses the other's record. The README gets a Measured difficulty section with one
line per task, which agentenv ignores like the file.
See the [specification](specs/desktop-app-environments/build-witness/spec.md)
for scope and verification.
@@ -19,7 +19,7 @@ initialization, discovery and target-free request flow; it does not verify a
client's configured launcher or its inherited environment. Test the generated
registration through the actual client, including a reduced `PATH` when checking
desktop or service startup. Record the client version and launch environment
separately from terminal doctor results. See [launcher troubleshooting](client-configuration.md#launchers-and-path).
separately from terminal doctor results. See [launcher troubleshooting](../getting-started/client-configuration.md#launchers-and-path).
## Published Pi code-mode renderer limit
@@ -33,7 +33,7 @@ shared form would leave later fields unknown instead. The existing
its character limit by advertising reduced nested validation structure, while
REA continues to validate calls against the complete canonical input schema.
Compact is the default for the server's advertised inputs; an explicit `full`
choice is preserved. See the [compact input schema profile](mcp-contracts.md#compact-input-schema-profile)
choice is preserved. See the [compact input schema profile](../reference/mcp-contracts.md#compact-input-schema-profile)
for details of its reduced forms.
Custom renderer hosts can raise `inputMaxChars` through `renderToolSignature` or
@@ -47,7 +47,7 @@ and [renderer usage notes](https://github.com/earendil-works/pi/blob/b45e0ebcc5a
## llama.cpp
The [manual stdio connection](client-configuration.md#llamacpp) was checked on
The [manual stdio connection](../getting-started/client-configuration.md#llamacpp) was checked on
macOS arm64 with Node.js 24.18.0, llama-server b11715 (`9d1878a43`), and the
published `[email protected]` package. `GET /tools` exposed 207 REA tools without
enabling llama.cpp's built-in tools. With the source checkout at `9e372c6`, all
@@ -486,7 +486,7 @@ existing static JavaScript analysis through CLI and stdio MCP, including
manifest readback, competing query variants, and resolved relative imports.
It uses an installed browser and accepts an optional installed REA entrypoint.
The complete `verify:browser` lane also exercises passive script export through
both public adapters. See [website script export](website-script-export.md).
both public adapters. See [website script export](../guides/web/website-script-export.md).
`verify:browser:modules` compares CLI and stdio MCP traces against an independent
real Chromium module-loading fixture: import-map scopes, package prefixes,
@@ -494,7 +494,7 @@ null/backtracking rejection, query/fragment identities, repeated module instance
lazy and computed unknowns, exact source/map identities and no implicit refetch.
Set `REA_BROWSER_EXECUTABLE`; the optional script argument selects an installed
package entrypoint. The complete `verify:browser` lane and existing conditional
Chrome CI include this verifier. See [module relationships](website-module-trace.md).
Chrome CI include this verifier. See [module relationships](../guides/web/website-module-trace.md).
## Agent evaluation and conformance records
@@ -643,3 +643,65 @@ npm run evidence:check
The records preserve unsupported and unverified coverage as explicit unknowns.
Run the matching real-tool prerequisites described in this guide.
### Focused tool-description trials
For a wording or discoverability question, begin with one representative case
per affected workflow boundary and its nearest valid alternative. Repeat a
concrete failure or ambiguous result after repair. Add repetitions when the
requested confidence or comparison needs them; a large matrix is not the default.
Use the user's selected model and budget, and verify the actual client and Node
executables used by the trial rather than assuming the development shell's
versions apply.
Reuse the existing optional lane and source-owned scenarios where they fit:
```bash
REA_AGENT_EVAL_SCENARIOS=javascript-module-view npm run verify:agent
```
Select the account-supported model with `REA_AGENT_EVAL_MODEL`. Read the selected
prompt and rubric before interpreting the gate: closed factual scenarios request
complete producer Evidence, while `javascript-module-view` exercises retained
views with heuristic scoring. Neither gate alone proves correct selection or
fully supported prose for a different task. Extend the narrow scenario or grader
in the existing evaluation tooling when necessary; reusable runners and fixtures
belong there, not in skills or copied temporary harnesses.
Capture the actual `initialize` and `tools/list` surface. Compact presentation may
remove parameter descriptions or nested selector shapes, so a concise exact call
example can remain necessary. Verify the producer-to-consumer handoff before
removing its visible syntax. For a wording comparison, hold prompts, schemas,
tool order, fixtures, permissions and host settings fixed; change one surface at
a time. Use fresh agent context without the audit's answers. Record later source,
fixture or runtime changes separately rather than claiming a controlled delta.
Score these outcomes separately:
- Tool selection, accepting other valid routes.
- Canonical argument validity and the exact requested paths, IDs, selectors,
fields, units and delivery. Structurally valid arguments can select the wrong
input; fixtures should refuse that selection instead of returning the expected
answer anyway.
- Recovery and cleanup. A valid unresolved lookup is not a malformed request;
normal status reads and cleanup-only retries are not automatically mistakes.
Do not infer that work stopped from a host-only timeout.
- Final claims against returned facts and coverage. Correct text does not erase
malformed-call loops, and valid calls do not prove every claim. A failed name
lookup is not a negative reference search; an exhausted partial page does not
establish absence in unexamined source. Generic responses without state facts
cannot establish that a session is closed.
Retain prompts, condition/catalog hashes, actual calls/results, failures, final
answers, model/client/runtime versions and cleanup outcomes. Report whole-task
input, cached input, output and elapsed time when available, and label unmeasured
CPU or process-family memory. Fewer description bytes do not establish fewer
task tokens or faster execution. Remove copied authentication after owned
processes settle and review retained artifacts before sharing them.
Controlled errors can test agent recovery without target effects. Label them as
simulated and verify applicable workflows separately through the production
adapter and real provider boundaries. State tool-subset versus full/progressive
catalog scope and remaining provider/host coverage. Small one-shot trials are
smoke checks, not statistical success rates or evidence that newer models need
less guidance in general.
+82
View File
@@ -0,0 +1,82 @@
# Documentation ownership and authoring
Use the [documentation directory](../README.md) to find existing material before
adding another guide. Preserve one canonical owner for each contract or workflow;
link to that owner instead of copying its claims into several guides.
## Choose a destination
| Document | Destination |
| -------------------------------------------------------- | -------------------------------------- |
| Installation, client setup and first CLI/MCP steps | `docs/getting-started/` |
| Analyst workflows and evidence interpretation | `docs/guides/<domain>/` |
| Analysis-engine configuration and format prerequisites | `docs/providers/` |
| Shared runtime contracts and evidence semantics | `docs/reference/` |
| Development, testing, release and authoring instructions | `docs/development/` |
| Composition diagrams | `docs/architecture/` |
| Compatibility changes for a released version | `docs/migrations/` |
| One feature specification, plan and tasks | `docs/specs/<domain>/<feature-slug>/` |
| One architectural decision | `docs/adr/<domain>/<decision-slug>.md` |
Use meaningful, lowercase domain and feature slugs. Global numeric prefixes are
unnecessary: different domains can add features independently, and branches do
not compete for the next sequence number. Check the existing domain directory
and linked issues before creating a feature with the same slug.
Guides describe existing behavior and limitations. Specs define one proposed
feature or domain change; an existing guide does not become a specification by
moving it into `specs/`.
## Feature lifecycle
Create the specification before implementation. Each feature directory contains:
- `spec.md`: problem, desired behavior, requirements, acceptance and non-goals.
- `plan.md`: the single implementation plan, contracts and architectural choices.
- `tasks.md`: incremental, testable steps linked to requirements and verification.
Split independent domains or separately reviewable features before writing their
specs. A broad `ROADMAP.md`, when useful, only lists child specifications, order
and dependencies; requirements and implementation tasks belong to each child.
Cross-repository contracts have one producer-owned specification. Consumers link
to that contract instead of maintaining competing copies.
Record an isolated architectural decision separately in
`docs/adr/<domain>/<decision-slug>.md`, with **Status**, **Context**,
**Decision** and **Consequences**. Link the ADR from affected specifications.
Do not create decision records for routine file moves or invent past decisions.
Specs do not replace the contribution-scope decision required for a new provider
or expanded execution authority. Follow
[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
## Links and public routes
Use relative links to physical Markdown files so GitHub readers can follow them.
Link repository files outside `docs/` with their full GitHub URLs. Preserve
existing heading anchors when moving a guide.
[`documentation.ts`](https://github.com/morluto/rea/blob/main/docs/.vitepress/documentation.ts)
owns categorized site navigation and the existing guides' stable public routes.
Update it when adding or moving a guide. VitePress rewrites categorized sources
to the existing public paths; its Markdown hook resolves repository links from
the physical source and renders them against those public routes. Keep basenames
and route assignments unique. Edit links use the physical source file.
For a move, search for both the basename and its old full path. Update README
translations, authored skills, verification instructions, documentation-fact
readers and release configuration when they consume that path. Preserve
historical release records and references bound to earlier commits or tags.
## Generated ownership and checks
The [documentation directory](../README.md#machine-owned-documentation) identifies
the machine-owned outputs and their producers. Their existing paths are retained
because scripts, package consumers and release history depend on them. Change
canonical sources and regenerate; do not fix a generated projection by hand.
Run `npm run docs:check` for documentation facts and generated consistency, and
`npm run verify:docs:routes` to build the complete production site and check local
links, anchors and physical edit targets.
When a physical path moves, also run the tests that read it. Follow
[development checks](testing.md#developer-commands) before requesting review.
@@ -181,7 +181,8 @@ When choosing the manual path, leave the automatic main release PR unmerged.
Select the application commit from main, create the frozen branch, and prepare
its own release PR using the steps below. Merging the main release PR starts
automatic publication instead. Close the superseded main PR after the manual
release has published and its metadata has been synchronized back to main.
release has published, before merging the metadata synchronization PR back
to main.
`release/VERSION` also sets the expected version for the workflow. Use an exact
SemVer such as `release/6.1.0` or `release/6.1.0-rc.1`, without build metadata.
@@ -192,7 +193,7 @@ registry metadata or changelog disagrees with that version before creating a tag
Before publishing a candidate, review its changelog's breaking changes and the
migration notes for the release being prepared. The
[6.1 to 6.2 migration guide](migration-6.2.md) records that release's changes;
[6.1 to 6.2 migration guide](../migrations/migration-6.2.md) records that release's changes;
it is a historical reference, not the compatibility checklist for later releases.
REA uses Release Please's `always-bump-minor` strategy: every release increments
@@ -5,8 +5,8 @@ The [platform tracker](https://github.com/morluto/rea/issues/32),
[browser tracker](https://github.com/morluto/rea/issues/39) track capability and
verification work. Available tools and setup are documented in the
[investigation guides](https://rea.tools/guides/),
[installation guide](installation.md), and
[generated catalog](mcp-contracts.md#generated-catalog).
[installation guide](../getting-started/installation.md), and
[generated catalog](../reference/mcp-contracts.md#generated-catalog).
## Priorities
@@ -27,7 +27,7 @@ fixture servers; custom orchestration needs a demonstrated requirement.
Provider and platform claims require their corresponding
[real verification lanes](testing.md#real-toolchain-verification-lanes).
See [tool design](tool-design.md) for choosing a tool boundary and
[Ghidra semantics](ghidra-provider.md) for the existing deep-provider contract.
[Ghidra semantics](../providers/ghidra-provider.md) for the existing deep-provider contract.
Setup remains additive, reuses existing tools, and requires approval for disclosed
configuration and Hopper installation changes. Additional installer choices need
+46 -40
View File
@@ -8,7 +8,7 @@ installed-client and agent runs.
Historical verification records may refer to `experiments/analysis-efficiency`.
Those experiment scripts are available in the [archived source tree](https://github.com/morluto/rea/tree/a2e2c29b59e8775b0e6aedb09a9eb16adf32646b/experiments/analysis-efficiency);
they are not included in current checkouts. The retained [test support](https://github.com/morluto/rea/blob/main/tests/support/analysis-efficiency/README.md)
contains deterministic checkers and fixtures. The [allocation report](verification/analysis-efficiency/mcp-integer-allocation.md)
contains deterministic checkers and fixtures. The [allocation report](../verification/analysis-efficiency/mcp-integer-allocation.md)
and its original receipt JSON remain documentation, checked by the report producer.
## Developer commands
@@ -382,7 +382,7 @@ instrumentation remains equivalent.
## CodeQL production scope
[The production configuration](../.github/codeql/production.yml) excludes
[The production configuration](https://github.com/morluto/rea/blob/main/.github/codeql/production.yml) excludes
`tests/`, `scripts/fixtures/`, `*.test.*`, `*.fixture.*`, `test_*.py`, and `*_test.py` inputs
during extraction. It retains the default queries and the existing `local`
threat model. Production source, bridges, runtime scripts, installation and
@@ -398,7 +398,7 @@ These exclusions do not change which behavioral tests CI executes. CodeQL's
JavaScript/TypeScript, Python, and Actions lanes do not add shell, Java, or Swift
analysis; their existing checks remain responsible for those languages.
[The production workflow](../.github/workflows/codeql.yml) is inactive until a
[The production workflow](https://github.com/morluto/rea/blob/main/.github/workflows/codeql.yml) is inactive until a
repository administrator disables CodeQL **default setup** and sets the Actions
repository variable `REA_CODEQL_ADVANCED` to `true`. Disable default setup first:
it rejects advanced CodeQL uploads. Then enable the variable and manually run
@@ -602,6 +602,12 @@ For partial Evidence delivery, verify that same-session readback preserves
identity and provenance without rerunning the provider. Include a no-writer
control that keeps completed Evidence inline and preserves the original failure.
For a seekable file with a known extent, distinguish a provably impossible wire
extent from an independent representation limit. Pair the malformed declaration
with a structurally possible over-limit control. A count-only truncated fixture
does not establish capacity behavior. This does not require decoding an opaque
compressed payload merely to apply its declared expansion limit.
When producer order is unspecified, synchronize fixtures on the observed request
or completion event. A filename, generated key, or fixed delay does not prove
which item completed before cancellation. Preserve real producer ordering and
@@ -702,16 +708,16 @@ the affected owners and applicable scenarios below. Include temporary snapshots
and one-shot commands. Use public CLI/MCP workflows where practical and a
controlled production seam for interleavings that cannot be reproduced reliably.
| Scenario | Assertion and control |
| --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Return typed cancellation with available partial observations and actual cleanup status. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
| Cleanup owners recover after an error was returned | Retain the original child and aggregate errors and project them after each owner releases. A successful peer stays released; the aggregate remains incomplete until all owners release. Preserve the first cause even when rejection is `undefined`. |
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
| Scenario | Assertion and control |
| --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Before the documented commit cutoff, return typed cancellation with available partial observations and actual cleanup status. After commit, assert the owning publication outcome and receipt. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
| Cleanup owners recover after an error was returned | Retain the original child and aggregate errors and project them after each owner releases. A successful peer stays released; the aggregate remains incomplete until all owners release. Preserve the first cause even when rejection is `undefined`. |
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
Shared process-supervision controls separate leader exit from output closure.
Cover output during held or failed cleanup, bounded retention and drain timeout,
@@ -909,7 +915,7 @@ covers deterministic evaluator logic.
reconstruction, native-boundary relationships and application graphs without
executing fixture PE files. Operator-local manifests and actual ILSpy oracles
remain optional, separately reported checks; the real Ghidra NativeAOT lane has
its own toolchain prerequisites. See [the managed guide](managed-code-analysis.md)
its own toolchain prerequisites. See [the managed guide](../guides/managed/managed-code-analysis.md)
for those configurations. Generated completion-ledger checks use the same owning
entrypoint and include its verifier/fixture files in their cache inputs.
@@ -1165,104 +1171,104 @@ public registry, and checks both the primary failure and live descendant absence
## Real-toolchain verification lanes
See [Real-toolchain verification lanes](testing-lanes.md#real-toolchain-verification-lanes).
See [Real-toolchain verification lanes](../testing-lanes.md#real-toolchain-verification-lanes).
## Android APK analysis
See [Android APK analysis](testing-lanes.md#android-apk-analysis).
See [Android APK analysis](../testing-lanes.md#android-apk-analysis).
## Apktool resource decoding
See [Apktool resource decoding](testing-lanes.md#apktool-resource-decoding).
See [Apktool resource decoding](../testing-lanes.md#apktool-resource-decoding).
## Flutter build identification
See [Flutter build identification](testing-lanes.md#flutter-build-identification).
See [Flutter build identification](../testing-lanes.md#flutter-build-identification).
## ADB device acquisition
See [ADB device acquisition](testing-lanes.md#adb-device-acquisition).
See [ADB device acquisition](../testing-lanes.md#adb-device-acquisition).
## Optional NativeAOT Ghidra analysis
See [Optional NativeAOT Ghidra analysis](testing-lanes.md#optional-nativeaot-ghidra-analysis).
See [Optional NativeAOT Ghidra analysis](../testing-lanes.md#optional-nativeaot-ghidra-analysis).
## IDA MCP adapter
See [IDA MCP adapter](testing-lanes.md#ida-mcp-adapter).
See [IDA MCP adapter](../testing-lanes.md#ida-mcp-adapter).
## Kuna lane
See [Kuna lane](testing-lanes.md#kuna-lane).
See [Kuna lane](../testing-lanes.md#kuna-lane).
## DOS Ghidra analysis
See [DOS Ghidra analysis](testing-lanes.md#dos-ghidra-analysis).
See [DOS Ghidra analysis](../testing-lanes.md#dos-ghidra-analysis).
## Apple Interface Builder archives
See [Apple Interface Builder archives](testing-lanes.md#apple-interface-builder-archives).
See [Apple Interface Builder archives](../testing-lanes.md#apple-interface-builder-archives).
## Native platform baseline in CI
See [Native platform baseline in CI](testing-lanes.md#native-platform-baseline-in-ci).
See [Native platform baseline in CI](../testing-lanes.md#native-platform-baseline-in-ci).
### Termux Android browser smoke test
See [Termux Android browser smoke test](testing-lanes.md#termux-android-browser-smoke-test).
See [Termux Android browser smoke test](../testing-lanes.md#termux-android-browser-smoke-test).
## Packet capture analysis
See [Packet capture analysis](testing-lanes.md#packet-capture-analysis).
See [Packet capture analysis](../testing-lanes.md#packet-capture-analysis).
## Apple native metadata and UI
See [Apple native metadata and UI](testing-lanes.md#apple-native-metadata-and-ui).
See [Apple native metadata and UI](../testing-lanes.md#apple-native-metadata-and-ui).
### Firmware adapters
See [Firmware adapters](testing-lanes.md#firmware-adapters).
See [Firmware adapters](../testing-lanes.md#firmware-adapters).
### JavaScript source recovery
See [JavaScript source recovery](testing-lanes.md#javascript-source-recovery).
See [JavaScript source recovery](../testing-lanes.md#javascript-source-recovery).
### Captured website source-map lane
See [Captured website source-map lane](testing-lanes.md#captured-website-source-map-lane).
See [Captured website source-map lane](../testing-lanes.md#captured-website-source-map-lane).
### JavaScript large-output lane
See [JavaScript large-output lane](testing-lanes.md#javascript-large-output-lane).
See [JavaScript large-output lane](../testing-lanes.md#javascript-large-output-lane).
### Website runtime attribution lane
See [Website runtime attribution lane](testing-lanes.md#website-runtime-attribution-lane).
See [Website runtime attribution lane](../testing-lanes.md#website-runtime-attribution-lane).
### Offline binary layout
See [Offline binary layout](testing-lanes.md#offline-binary-layout).
See [Offline binary layout](../testing-lanes.md#offline-binary-layout).
### Offline EVM interface
See [Offline EVM interface](testing-lanes.md#offline-evm-interface).
See [Offline EVM interface](../testing-lanes.md#offline-evm-interface).
### Recorded crash evidence
See [Recorded crash evidence](testing-lanes.md#recorded-crash-evidence).
See [Recorded crash evidence](../testing-lanes.md#recorded-crash-evidence).
### Exported environment bundles
See [Exported environment bundles](testing-lanes.md#exported-environment-bundles).
See [Exported environment bundles](../testing-lanes.md#exported-environment-bundles).
## Agent evaluation and conformance records
See [Agent evaluation and conformance records](testing-lanes.md#agent-evaluation-and-conformance-records).
See [Agent evaluation and conformance records](../testing-lanes.md#agent-evaluation-and-conformance-records).
## Frida instrumentation
See [Frida instrumentation](testing-lanes.md#frida-instrumentation).
See [Frida instrumentation](../testing-lanes.md#frida-instrumentation).
### Offline WASM artifacts
See [Offline WASM artifacts](testing-lanes.md#offline-wasm-artifacts).
See [Offline WASM artifacts](../testing-lanes.md#offline-wasm-artifacts).
@@ -60,23 +60,11 @@ or ordinary script path when available. Identify the specific gap and what a
new or extended contract would add; upstream capability alone does not require
a new tool or equal support from every provider.
Exercise the task through REA as a whole and compare relevant provider behavior
with an independent artifact or format observation where possible. Use the
caller's selected target and authority. Record the provider version, target and
observed coverage, and distinguish an unsupported operation from an empty
result or an untested path.
Include the boundary cases that could change the answer: provider labels versus
source facts, addresses versus file locations, partial results, and a useful
recovery after failure or session close when those apply to the task. Check that
composed results keep each observation's identity, limitations and source
association. Existing real-provider lanes and focused tests establish their
selected scenarios; they do not by themselves establish a different workflow or
provider equivalence. If a provider cannot be exercised, report that limitation
and continue with the available evidence. Measure agent tool selection only with
an actual agent evaluation; a scripted tool sequence does not establish it. Use
[testing guidance](testing.md#provider-backed-workflow-verification) to select
the cases that establish the proposed contract.
Use the caller's selected target and authority. Follow
[provider-backed workflow verification](testing.md#provider-backed-workflow-verification)
for independent observations, boundary controls and reporting unavailable
provider coverage. A scripted tool sequence does not establish agent selection;
that claim requires an actual agent evaluation.
## Audit existing tools
@@ -94,14 +82,10 @@ identity, session lifetime and available alternatives. Matching text between
two renderers does not establish that their shared advice is correct or that an
agent selected the workflow.
Exercise the selected public workflows, including the steps before and after a
tool call. Check task selection, input choices, returned facts and their source,
partial or unsupported outcomes, and applicable session and recovery behavior.
Compare native provider output and an independent artifact observation where
available. Inspect existing tests and real-provider lanes for the scenario they
actually cover; a passing test does not establish a different task or host.
Use [workflow verification](testing.md#provider-backed-workflow-verification)
for counterexamples and lifecycle controls relevant to the audit.
Exercise each selected workflow before and after the tool call. Use
[workflow verification](testing.md#provider-backed-workflow-verification) to
check returned facts, partial outcomes, recovery and the scope of existing
provider evidence.
Report working routes as well as findings. Separate a reproduced contract bug,
an intentional limitation, a design opportunity and an unverified hypothesis;
@@ -109,6 +93,38 @@ include the smallest useful reproduction, impact and coverage limits. Do not
claim autonomous-agent selection from a scripted trace. Check related reports
before proposing another issue, and avoid treating lack of API parity as a bug.
### Write descriptions for selection and a valid first call
Start with the analyst task, selected source and useful result. Put material
effects and the deciding boundary between neighboring tools early. Progressive
discovery previews the first 240 characters, so those characters should identify
the operation without requiring the rest of its reference documentation.
Keep exact call syntax where the advertised schema cannot convey it. The compact
input profile strips most annotation prose and can reduce nested controls to
`{type: "object"}`. Moving selectors into parameter descriptions alone does not
make them available to those clients. For example, retained pseudocode needs
both `source.kind: "retained-evidence"` and `view: {kind: "native", facet:
"pseudocode"}`; a short exact example can be more useful than many paragraphs.
Use the producer's handoff for other supported views rather than maintaining
another supposedly exhaustive list in the shared consumer description.
Keep configuration tutorials and detailed matching algorithms with their owning
parameter contracts, product guides and actionable failures. Retain applicability,
source identity, uncertainty, authority and cleanup facts needed to choose or
safely continue the operation. An unresolved symbol lookup is not a completed
reference search; an exhausted page does not establish absence in unexamined
source. Shorter copy must preserve these distinctions.
For wording changes, use a small set of representative agent tasks at the affected
boundaries. Hold schemas, tool order, fixtures and host configuration fixed when
comparing copy; add repetitions only for failures or ambiguous outcomes. Score
actual arguments, corrective calls and supported final claims separately. A
correct eventual answer can conceal malformed-call loops, while a valid call
can still produce an overstated answer. Report whole-task usage, including
retries and cached input, rather than treating fewer description bytes as proof
of lower token cost.
## Design recovery around the failure
Distinguish a request that violates its input contract from a valid selection
@@ -178,14 +194,18 @@ does not establish another instance.
provider-observed direct relationship set. `complete` means that set was
returned without a known read limit; it does not claim every possible runtime
edge. Provider and execution caveats stay in `limitations` and do not imply
incomplete relationship coverage. Missing coverage remains `unknown`.
incomplete relationship coverage. Directional `discovery` separately describes
whether applicable static relationships could have been missed by discovery;
missing discovery is unknown. Preserve paths supported by observed edges,
and qualify negative and shortest-path claims by discovery scope. Missing
relationship coverage remains `unknown`. Computed candidates and unestablished incoming targets qualify directional discovery; they do not by themselves mean that recorded edges were omitted.
- Declare process, filesystem, network, UI, and mutation effects truthfully;
enforce the requested target and lifecycle.
- Add limits only for real format, protocol, authority, or measured resource
constraints. Report pagination, truncation, cancellation, and partial failure.
An assumed agent budget is insufficient justification for a limit. Name the
ceiling and its recovery as described in
[admission limits and recovery](design/limits-and-recovery.md).
[admission limits and recovery](../design/limits-and-recovery.md).
### Compose declared Evidence inputs
@@ -229,8 +249,8 @@ MCP supplies the current ledger lookup. CLI commands that advertise
`--evidence-bundle` use one explicitly selected, authenticated bundle for the
command's lifetime. Keep cleanup with that owner. A retained reference does not
grant filesystem access: file selectors remain separate, explicit contracts
where supported. See [retained Evidence inputs](mcp-contracts.md#retained-application-evidence-inputs)
and [CLI reuse](cli.md#import-export-and-compare-evidence).
where supported. See [retained Evidence inputs](../reference/mcp-contracts.md#retained-application-evidence-inputs)
and [CLI reuse](../getting-started/cli.md#import-export-and-compare-evidence).
### Interpret boundary facts once
@@ -344,13 +364,15 @@ stages that never ran.
and resource release separate. A valid process handle, successful launcher,
or selected target establishes only its own fact. Publish available identity
and exit observations even when launch fails; retain unknown ownership.
- Process exit does not establish output closure. The shared process supervisor
retains at most 64 KiB of additional diagnostics from owned stop entry,
including pending cleanup and retries, and drains for up to one second after
verified stop. It marks output incomplete when closure cannot be
observed and releases owned event-loop references without changing the proven
process-cleanup outcome. Failed process cleanup retains capture for retry;
borrowed processes and streams retain their own lifetime and reference state.
- Process exit does not establish output closure. The
[shared supervisor](https://github.com/morluto/rea/blob/main/src/process/ProviderProcess.ts)
owns diagnostic retention and bounded draining through cleanup and retries.
Report incomplete output separately from process cleanup. After verified stop
and bounded draining, release owned event-loop references even if output closure
stays unconfirmed. Keep the output capture owned for retry while process cleanup
is unresolved. Borrowed processes and streams retain their own lifetime and
reference state; select the matching
[supervision controls](testing.md#lifecycle-regressions) when changing this code.
- Authenticate and capture identity through the same boundary on initial and
recovery handshakes. Bound abandoned unauthenticated listeners where the
protocol permits it; that deadline must not expire authenticated idle sessions
@@ -369,8 +391,9 @@ stages that never ran.
analysis snapshot is part of `open_binary`; cancellation keeps that operation
and the caller's reason through the shared JSON reader.
- For cancellable observations, carry cancellation through result consumption
and awaited cleanup. Before returning success, check the signal after the
last awaited finalization step.
and awaited cleanup. Check the signal after the last awaited finalization step
unless the operation has passed its documented commit cutoff. Preserve the
owning operation's committed outcome and receipt after that cutoff.
Retain collected observations under typed cancellation. Preserve an existing
primary failure and let incomplete cleanup retain its own classification.
Preserve the caller's abort reason as the typed cancellation error's cause,
@@ -543,5 +566,5 @@ metadata to either adapter.
When discoverability changes, evaluate representative broad, direct, and negative
tasks on the intended host. Record host/model/catalog, selected tools, arguments,
errors, retries, and task outcomes. Schema validity and keyword heuristics do not
establish successful tool selection. See [mcp-contracts.md](mcp-contracts.md) for
establish successful tool selection. See [mcp-contracts.md](../reference/mcp-contracts.md) for
shipped runtime behavior.
+14 -1
View File
@@ -50,7 +50,8 @@ explicitly with the managed tools to establish PE/CLI facts.
## Admission and containment
Metadata is range-read with a 64 KiB cache. Policy budgets are 16 MiB of
Metadata is range-read with a cache of at most 64 KiB, bounded by the remaining
selected metadata allowance. Default policy budgets are 16 MiB of
manifest metadata, 100,000 files, 64 KiB per UTF-8 string, 512 MiB of stored or
expanded bytes per file, 256 path parts per member, 250,000 distinct in-memory
path-tree nodes, and 2 GiB of aggregate expanded bytes. The node budget counts
@@ -60,6 +61,18 @@ streams 64 KiB chunks and reuses completed probe facts during extraction;
ordinary native files without a marker incur one extra
read pass before host hashing.
Both artifact tools accept `artifact_limits.dotnet_bundle` to select
`metadata_bytes`, `entry_count`, `string_bytes`, `member_bytes`,
`expanded_bytes`, `path_parts`, and `path_nodes` for the next operation. The
string allowance applies to every manifest string, including the bundle ID.
For example, retry a complete bundle whose manifest exceeds the default with
`{"artifact_limits":{"dotnet_bundle":{"metadata_bytes":33554432}}}`.
The CLI accepts the same selection through `--artifact-limits` JSON. Omitted
fields preserve defaults; errors name the exact selected control, effective
value and required minimum when known. No REA restart is needed. Summary
delivery and paging do not change producer admission, and larger reader
budgets do not raise MCP response or Node representation limits.
Extraction hashes the whole host again through the same descriptor used to read
members and compares it with the inventory's root digest before creating an
output directory. This adds a full-host read pass to extraction, including when
+17 -1
View File
@@ -44,6 +44,22 @@ file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`),
auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed
notes keep `core` null.
Linux `CORE` decoding requires System V/unspecified (0) or GNU/Linux (3)
OSABI with ABI version zero. An unspecified OSABI alone does not establish
Linux origin. An explicitly different OSABI or ABI version leaves `core` null
and its Linux-specific `type_name` unknown, with a limitation explaining the
profile. Owner/type numbers, descriptor ranges and hashes, GNU-owned notes,
and complete table-examination coverage remain available.
For Android `NT_ANDROID_TYPE_IDENT` (`.note.android.ident`, owner `Android`,
type `1`) notes, `android.android_api` is decoded from both legacy 4-byte
descriptors (where `ndk_version` and `ndk_build_number` remain null) and modern
132-byte or extended NDK descriptors, which also populate `android.ndk_version`
and `ndk_build_number` when their 64-byte slots hold NUL-padded
printable ASCII. Truncated or non-ASCII Android descriptors degrade `android`
to null while preserving the raw descriptor digest and preview, and other
owners keep `android` null.
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
appear twice when both tables declare it.
@@ -54,7 +70,7 @@ the image never carried notes in another container or overlay.
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
32 note entries are examined by default across every table. Retained note
names, section names, GNU descriptor text, and core dump string representations share a scan budget based on
names, section names, GNU descriptor text, core dump string representations, and Android ident strings share a scan budget based on
the source bytes plus any decoded section-name table, capped at
`max_file_bytes`. The raw source and decoded table must also fit together
within that ceiling. If admission fails, the error reports the required
+1 -1
View File
@@ -23,7 +23,7 @@ and consider setting both `REA_MCP_INPUT_SCHEMA_PROFILE` and
advertised output schema to 4,096 bytes by relaxing nested client validation;
REA still validates complete results against the canonical contracts. Pair
this with retained summaries and selected inventory views to limit the context
needed for one question. See [MCP contracts](mcp-contracts.md) for the profiles
needed for one question. See [MCP contracts](reference/mcp-contracts.md) for the profiles
and their client compatibility tradeoffs.
The layout facts follow ESP-IDF v6.1 (`esp_app_format.h`, `esp_app_desc.h`,
+7 -5
View File
@@ -22,9 +22,10 @@ use the current projection identity; there is no pre-coverage FIT summary varian
A FIT is a flattened devicetree (FDT v16 or v17) with a top-level `/images`
node. Recognition walks the structure block within the first 4 KiB of the file
only; the file name does not matter. A devicetree whose `/images` node begins
after 4 KiB, a devicetree without `/images`, and any other FDT version stay on
the plain devicetree route: `open_binary` refuses them and
[`inspect_devicetree`](devicetree.md) still parses them. A U-Boot control DTB
after 4 KiB or a devicetree without `/images` stays on the plain devicetree
route: `open_binary` refuses it and [`inspect_devicetree`](devicetree.md)
can parse it when its FDT version is 16 or 17. Other FDT versions are not
recognized as FIT and are also unsupported by `inspect_devicetree`. A U-Boot control DTB
that contains `/images` is treated as a FIT. The container format is `fit`.
Recognition is a probe. The complete devicetree parse decides, so a file whose
@@ -124,8 +125,9 @@ Failures apply to the whole file and return no partial inventory.
The devicetree, including any embedded image data, must be at most 16 MiB; a
larger `totalsize` fails as a memory limit. Store large images externally
(`mkimage -E`): external data of any size is streamed from the file and never
loaded into memory. A FIT may declare at most 1,024 images and 1,024
(`mkimage -E`): external image bytes are streamed from the file within their declared
32-bit sizes and the resource bounds below, without loading the whole image
into memory. A FIT may declare at most 1,024 images and 1,024
configurations, and an image at most 8 hash nodes; the devicetree parser's
node, property and depth bounds also apply. The sum of declared image sizes
is bounded by the shared archive expansion budget. These are fixed resource
-31
View File
@@ -1,31 +0,0 @@
# Frida instrumentation
REA exposes Frida device and process discovery, attach or spawn sessions, and caller-authored JavaScript instrumentation through the CLI and MCP. The Node bindings are an optional package dependency and load only when a Frida operation runs. REA does not install Frida server on a target.
For remote devices, pass a host with an optional port (`host[:port]`; bracket IPv6 addresses) and any token, certificate, origin, or keepalive options on that operation. The keepalive interval is measured in seconds, matching Frida Core 17.23.2. Authentication values are held only for the active call or session and are excluded from Evidence. Do not embed credentials in the endpoint address.
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
`instrument-with-frida --json` returns the Evidence record directly, matching MCP. Consumers of the earlier `{evidence, cleanup_error}` CLI wrapper should read the record at the JSON root and the session cleanup observation at `normalized_result.cleanup_error`. If final provider cleanup fails or cancellation arrives during it, the typed error retains completed Evidence in `details.partial_observation`. Discovery commands retain their device or process report in the same field.
For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply.
A summary's selected fields look like this (excerpt):
```json
{
"summary": {
"state": "running",
"messages_retained": 50,
"messages_truncated": false
}
}
```
Use `get_evidence_bundle` with `detail="summary"` and `filters={"evidence_id":"PARENT_ID"}` to discover parent metadata without message payloads. Filters apply only to summary delivery. Use `export_evidence_bundle` for the complete canonical bundle, then select the record whose `evidence_id` matches `normalized_result.parent_evidence_id` from the status summary. To use the existing CLI `inspect-analysis-view`, extract that record into a standalone Evidence JSON file and select it as an Evidence-file source with `view={"kind":"summary"}`. Reading retained Evidence does not refresh a live session or verify a process after exit or PID reuse.
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released.
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Local device lookup waits 1000 ms. This release has no runtime override for that lookup; another REA timeout does not raise it. Remote endpoints do not use that lookup. Message capture is bounded to 16 MiB and reports truncation. Metadata, type, and text field caps, and the JSON node cap, are also fixed. Evidence names a cap when that observation actually dropped data. This release has no runtime override for those capture caps. Script source is not stored in Evidence; its selected path and SHA-256 digest are. For a file, `source_sha256` identifies the original selected bytes. Invalid UTF-8 continues to use replacement decoding; Evidence then records that limitation and `raw_result.source_representation` with the encoding, decoding mode, and `loaded_source_sha256` of the UTF-8 string passed to Frida. Valid UTF-8 files and inline scripts do not need this additional representation fact. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. If cancellation is observed when the native call returns, partial Evidence retains the source identity and captured messages even if that call rejects, including messages emitted during awaited cleanup. The caller's cancellation reason remains primary and a native rejection is retained as a sanitized diagnostic. A failure already observed before cleanup stays primary if cancellation arrives during cleanup. The cancellation cutoff is REA's observation of native completion, rather than a claim about unobservable native event order. `script_id` is REA's load-attempt identity. When creation rejects without returning a handle, `script_cleanup: "no_handle"` means REA acquired no handle to unload; it does not prove that the engine performed no work. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
The [Frida Core API](https://frida.re/docs/frida-core-api/) documents device management, sessions, and script loading; the [JavaScript API](https://frida.re/docs/javascript-api/) documents target-side instrumentation. [Luma](https://luma.frida.re/) is Frida's standalone GUI; REA does not automate or control it. `npm run verify:frida` requires a Linux host with the optional Frida binding and uses a short-lived local Node.js fixture created by the verifier. Add `-- --address HOST:PORT --pid PID` to run the separate remote attach lane against a caller-selected target. Mock tests do not establish real device/server compatibility.
+185 -124
View File
@@ -4,7 +4,7 @@ Use the CLI for a direct inspection or a script. It uses the same application
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
separate process; save results when you want to use them in a later command.
For a pinned, noninteractive workflow with retained reports, see
[static analysis in GitHub Actions](ci-analysis.md).
[static analysis in GitHub Actions](../ci-analysis.md).
## Run a command
@@ -19,7 +19,7 @@ Use an extracted application directory or an ASAR as the target. Static
JavaScript analysis returns the application graph and Evidence directly.
Generic `rea analyze PATH` selects this workflow for directories and `.asar`
files when neither `--provider` nor `--snapshot` is supplied. See
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
[JavaScript artifact reconstruction](../guides/javascript/javascript-artifact-reconstruction.md)
for results, integrity checks and coverage.
The complete application Evidence can be hundreds of megabytes for a real
@@ -59,7 +59,7 @@ For example, a 334 MB Obsidian application Evidence file yields a summary view
of about 10 KB. Replace the view with
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
module page. See
[JavaScript application workflows](javascript-application-workflows.md) for
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md) for
views, feature traces and comparisons that take the same saved Evidence.
Add `--timings` to a command to write one JSON timing record to stderr. For
@@ -226,7 +226,7 @@ saving the complete original Evidence in a new bundle. Summary requires that
explicit destination; existing files are preserved. Later
`inspect-analysis-view` calls can page the exact parent from the saved bundle
without reopening the assembly. See
[CLI summary and saved member rows](managed-code-analysis.md#cli-summary-and-saved-member-rows)
[CLI summary and saved member rows](../guides/managed/managed-code-analysis.md#cli-summary-and-saved-member-rows)
for the complete command sequence. Omitted or `--detail complete` retains the
full inline response.
@@ -234,8 +234,8 @@ Choose the workflow from the artifact's contents rather than its `.exe` or
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
unpack .NET single-file hosts or decode IL2CPP. For the image, type and method
names in a Unity `global-metadata.dat`, use
[IL2CPP metadata](il2cpp-metadata.md). NativeAOT method bodies require native
analysis. See the [managed-code guide](managed-code-analysis.md)
[IL2CPP metadata](../il2cpp-metadata.md). NativeAOT method bodies require native
analysis. See the [managed-code guide](../guides/managed/managed-code-analysis.md)
for supported deployment forms, selection semantics, and coverage limits.
## SQLite snapshots
@@ -243,7 +243,7 @@ for supported deployment forms, selection semantics, and coverage limits.
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
table. This command does not require a native analysis provider. See
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
[SQLite database snapshots](../guides/data/sqlite-databases.md) for database/WAL identity,
source-file handling and row-value semantics.
## Native analysis
@@ -256,7 +256,7 @@ rea inspect-go-binary /absolute/path/to/program --json
```
The shared CLI/MCP workflow reports compiler version, modules, replacements,
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
build settings, original bytes and SHA-256. See [Go binaries](../guides/native/go-binaries.md)
for supported containers and absent or malformed metadata.
Inspect a Godot PCK without running the game:
@@ -267,7 +267,7 @@ rea extract-artifact game.pck --json
```
Formats 2, 3 and 4 retain header, directory and integrity facts. Compact delivery
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
keeps complete Evidence in the requested bundle. See [Godot PCK](../godot-pck.md)
for embedded packs, batch queries and unknown encrypted coverage.
Inspect a UnityFS AssetBundle without running Unity:
@@ -278,7 +278,7 @@ rea extract-artifact content.bundle --json
```
Format versions 6 through 8 retain header, block and directory facts with
none/LZ4/LZ4HC block decompression. See [UnityFS](unityfs.md) for encryption
none/LZ4/LZ4HC block decompression. See [UnityFS](../unityfs.md) for encryption
and LZMA coverage boundaries.
Inspect a WeChat mini-program package without WeChat or an unpacker:
@@ -290,7 +290,7 @@ rea extract-artifact app.wxapkg --json
Plaintext packages retain header, table and member facts with verbatim member
bytes. `V1MMWX` packages are detected as encrypted coverage and refused on
extraction. See [wxapkg](wxapkg.md) for layout and coverage boundaries.
extraction. See [wxapkg](../wxapkg.md) for layout and coverage boundaries.
Inspect an ESP32-family app or bootloader image without esptool:
@@ -301,7 +301,7 @@ rea extract-artifact firmware.bin --json
The inventory lists each segment with its ESP-IDF load classification, the
checksum and SHA-256 results, the descriptor and any Secure Boot v2 signature
blocks. See [ESP images](esp-image.md) for recognition, coverage and refusals.
blocks. See [ESP images](../esp-image.md) for recognition, coverage and refusals.
Inspect an ESP flash dump, or one read from a flash address other than 0:
@@ -313,7 +313,7 @@ rea extract-artifact flash.bin --json
The inventory lists the bootloader, the partition table and each partition,
with ESP image facts for app partitions and the OTA records. See
[ESP flash images](esp-flash-image.md) for table location, inputs and limits.
[ESP flash images](../esp-flash-image.md) for table location, inputs and limits.
Inspect a Node.js SEA carrier without running it:
@@ -322,7 +322,7 @@ rea inspect-artifact ./application --json
rea extract-artifact ./application --json
```
See [Node.js SEA](node-sea.md) for supported layouts, script and asset paths,
See [Node.js SEA](../node-sea.md) for supported layouts, script and asset paths,
fuse observations, and the real-fixture verification lane.
Inventory and extract Intel HEX or S-record regions:
@@ -332,7 +332,7 @@ rea inspect-artifact ./firmware.hex --json
rea extract-artifact ./firmware.hex --json
```
See [Intel HEX and S-record images](hex-records.md) for recognition, region
See [Intel HEX and S-record images](../hex-records.md) for recognition, region
names, metadata and limits.
Inspect and extract stored U-Boot legacy uImage payloads and components:
@@ -342,7 +342,7 @@ rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.
rea extract-artifact ./firmware.bin --json
```
See [U-Boot uImage](uimage.md) for verified CRCs, physical component ranges,
See [U-Boot uImage](../uimage.md) for verified CRCs, physical component ranges,
unknown names, resource limits and retained views. Compressed bytes stay stored.
`inspect-artifact` and `extract-artifact` also read ZIP members compressed with
@@ -361,7 +361,7 @@ rea inspect-artifact ./firmware.uf2 --json
rea extract-artifact ./firmware.dfu --json
```
See [UF2 and DfuSe images](uf2-dfuse.md) for recognition, families, numbering,
See [UF2 and DfuSe images](../uf2-dfuse.md) for recognition, families, numbering,
the DfuSe suffix CRC and limits.
Inventory and extract U-Boot FIT images:
@@ -371,7 +371,7 @@ rea inspect-artifact ./boot.itb --json
rea extract-artifact ./boot.itb --json
```
See [U-Boot FIT images](fit-images.md) for recognition, data sources, hashes
See [U-Boot FIT images](../fit-images.md) for recognition, data sources, hashes
and limits.
Inspect local model declarations without loading weights:
@@ -385,7 +385,7 @@ Default hashing reads only headers and leaves full-file identity unknown.
`--hash full` streams the complete file for SHA-256. Both return typed metadata,
original tensor declarations and derived totals. MCP summaries retain complete
Evidence for filtered metadata, tensor and type-total pages; CLI compact delivery
uses `--detail summary --evidence_output FILE`. See [model files](model-artifacts.md).
uses `--detail summary --evidence_output FILE`. See [model files](../model-artifacts.md).
Inspect Unity IL2CPP metadata names without Unity or a disassembler:
@@ -402,7 +402,7 @@ method instances and `--ghidra-seeds PATH` then writes Ghidra function seeds. It
with retained Evidence; query indexed rows with filters, batch indices and
selected fields. The CLI defaults to complete Evidence;
`--detail summary --evidence_output FILE` exports the full result to a new file
and returns a compact overview. See [IL2CPP metadata](il2cpp-metadata.md).
and returns a compact overview. See [IL2CPP metadata](../il2cpp-metadata.md).
Inspect an embedded cargo-auditable Rust dependency list without installing
Rust or configuring a disassembler:
@@ -414,7 +414,7 @@ rea inspect-rust-binary /absolute/path/to/program --json
The same workflow reports package names, versions, sources, the root package,
dependency edges, and the section's file range. A missing `.dep-v0` section is
null metadata, not proof that the binary has no dependencies. See
[Rust binaries](rust-binaries.md).
[Rust binaries](../rust-binaries.md).
Inspect a PE Authenticode certificate table without executing the image or
verifying signatures:
@@ -426,7 +426,7 @@ rea inspect-pe-certificates /absolute/path/to/application.exe --json
The workflow reports `WIN_CERTIFICATE` entries and, for PKCS#7 signedData,
embedded X.509 serial, issuer, subject, and validity fields. A missing security
directory is observed absence of that table. See
[PE Authenticode certificate tables](pe-certificates.md).
[PE Authenticode certificate tables](../pe-certificates.md).
Inspect PE import and delay-load tables without executing the image:
@@ -437,7 +437,20 @@ rea inspect-pe-imports /absolute/path/to/application.exe --json
The workflow reports DLL names, named and ordinal functions, delay-load
descriptors, and a pefile-compatible import hash of the standard import table.
A missing import directory is observed absence of that table. See
[PE import and delay-load tables](pe-imports.md).
[PE import and delay-load tables](../pe-imports.md).
Inspect a Windows Shell Link (`.lnk`) without resolving or executing the
target:
```bash
rea inspect-windows-shortcut /absolute/path/to/shortcut.lnk --json
```
The workflow reports header flags and FILETIMEs, IDList items, LinkInfo volume
and network names, StringData arguments and icon location, ExtraData blocks
including Tracker droid facts labelled derived, and derived observations such
as argument length and a minimized ShowCommand. See
[Windows Shell Link shortcuts](../guides/native/windows-shortcut.md).
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
@@ -447,7 +460,33 @@ rea inspect-elf-notes /absolute/path/to/program --json
The workflow reports note owners, types, descriptor hashes, and GNU build-id
or ABI-tag fields when those descriptors match. A missing note table is
observed absence of those declarations. See [ELF note tables](elf-notes.md).
observed absence of those declarations. See [ELF note tables](../elf-notes.md).
List ELF and Mach-O initializer and finalizer functions without executing the
image:
```bash
rea inspect-native-initializers /absolute/path/to/program --json
```
The workflow reports `DT_INIT`, init/fini arrays, legacy `.ctors`/`.dtors`, and
Mach-O `__mod_init_func`, `__init_offsets` and `__mod_term_func` entries with
their resolved targets. Universal Mach-O files need `--architecture`. See
[ELF and Mach-O initializer functions](../guides/native/native-initializers.md).
Inspect ELF unwind function ranges without executing the image:
```bash
rea inspect-elf-unwind-ranges /absolute/path/to/program --json
```
The workflow reports FDE start, end, and size, the containing section, an exact
symbol name when one exists, CIE identity, LSDA, and personality. Coverage
counts ranges that match a symbol, ranges without a symbol, and whether the
`.eh_frame_hdr` search table agrees with the decoded FDEs. `.debug_frame` is
reported present, absent, or incomplete and is not decoded. A missing unwind table is
observed absence of those declarations. See
[ELF unwind function ranges](../guides/native/elf-unwind-ranges.md).
Inspect an Apple `.ips` crash report without executing a target:
@@ -458,7 +497,7 @@ rea inspect-apple-ips /absolute/path/to/report.ips --json
The workflow reports first-line metadata, process/exception/termination facts,
threads, usedImages, lossless integer registers, and documented crash classes
when `bug_type` is `"309"`. Other DiagnosticReports types return no crash
result. See [Apple IPS crash reports](apple-ips.md).
result. See [Apple IPS crash reports](../apple-ips.md).
Inspect Linux kernel module metadata without executing the module or requiring
`modinfo`:
@@ -474,7 +513,23 @@ not decoded and the signature is not validated. Other ELF types are not
applicable. gzip, xz, and zstd containers are unsupported and name
`external_decompression` as the next probe: decompress with an external
utility, then inspect the decompressed file. See
[Linux kernel modules](kernel-modules.md).
[Linux kernel modules](../kernel-modules.md).
Read a Linux kernel image's version banner and embedded `CONFIG_IKCONFIG`
build config without executing it:
```bash
rea inspect-kernel-build-config /absolute/path/to/vmlinux --json
rea inspect-kernel-build-config /absolute/path/to/bzImage --name CONFIG_MODULE_SIG --json
```
The workflow reports the banner version, builder, and compiler, the config
option count, and paged `CONFIG_*` entries (`y`, `m`, strings, numbers, and
`not_set`). An `IKCFG_ST` string counts as a config only when a valid gzip
member follows it and `IKCFG_ED` follows that member. A gzip bzImage payload
is inflated and searched; other payload codecs are unsupported and name
`external_decompression` as the next probe. See
[Linux kernel build config](../guides/native/kernel-build-config.md).
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects (3.14 and
3.15 headers are qualified but their marshal layout is not pinned)
@@ -486,7 +541,7 @@ rea inspect-python-pyc /absolute/path/to/module.pyc --json
The workflow reports the pinned CPython version for the magic, PEP 552 hash or
mtime/size fields, nested function names, and bytecode SHA-256. Bytecode is
not decompiled. See [CPython pyc inspection](python-pyc.md).
not decompiled. See [CPython pyc inspection](../python-pyc.md).
Inspect an Erlang BEAM file without executing bytecode:
@@ -496,7 +551,7 @@ rea inspect-erlang-beam /absolute/path/to/app.beam --json
The workflow reports IFF chunk names, atom tables, exports, imports, lambdas,
and the Code chunk header. Missing optional tables stay null. See
[Erlang BEAM files](erlang-beam.md).
[Erlang BEAM files](../erlang-beam.md).
Complete `rea decompile --json` Evidence can be saved and inspected through
`inspect-analysis-view` with an inline source and
@@ -505,10 +560,10 @@ The projection reads that saved observation without starting a provider;
offsets/limits count UTF-16 code units. A summary view preserves the original
selector and distinguishes null from empty text. The MCP tool additionally
supports `detail: "summary"` on its first call to retain the complete source
without returning the body inline; see [native investigation](native-investigation.md).
without returning the body inline; see [native investigation](../guides/native/native-investigation.md).
Configure [Hopper or Ghidra](installation.md#hopper), the
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
[IDA adapter](../providers/ida-provider.md), or [Kuna](../kuna-provider.md) before analyzing a
native target. Substitute
your target, search text and function name or address in these examples:
@@ -536,11 +591,11 @@ MCP `analyze_function` defaults to the same summary and retains the complete
dossier in the session for `inspect_analysis_view`.
Run `rea --help` or a command's `--help` for arguments and output options.
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
The [generated catalog](../reference/mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
tools. Provider-specific target support is described in the
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
[raw binary guide](raw-binary.md), [GameCube DOL guide](ghidra-dol.md), and
[Windows Ghidra guide](windows-ghidra-p0.md).
[native guide](../guides/native/native-investigation.md), [DOS guide](../providers/ghidra-dos.md),
[raw binary guide](../providers/raw-binary.md), [GameCube DOL guide](../ghidra-dol.md), and
[Windows Ghidra guide](../providers/windows-ghidra-p0.md).
## Choose a provider
@@ -587,19 +642,20 @@ For Hopper, Ghidra, IDA, JADX, and Kuna, run
such as `pwntools-elf`, report their configuration requirements directly and are
not accepted by `doctor --provider`. See
[task readiness](installation.md#check-readiness-for-your-task) and
[provider selection](mcp-contracts.md#identity-and-discovery).
[provider selection](../reference/mcp-contracts.md#identity-and-discovery).
The session reports work still in progress through `analysis_activity`.
A client timeout can end its wait while the provider continues analyzing.
`cleanup_incomplete` identifies resources whose shutdown or removal could not
be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
be verified. See [MCP contracts](../reference/mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for import, client timeouts and recovery.
A one-shot command releases its provider session and REA runtime before it
prints a failure, so it omits the same-session reuse and
`update_analysis_settings` guidance an MCP connection receives. Its
`details.limit` reports `scope: "rea-invocation"` and
A one-shot command attempts provider-session and runtime cleanup before reporting
a failure. If cleanup remains incomplete, inspect the reported resources and
follow the error's remediation. [Same-session cleanup retry](../reference/mcp-contracts.md#retained-cleanup-retry),
reuse and `update_analysis_settings` apply to a live MCP connection.
For one-shot commands, `details.limit` reports `scope: "rea-invocation"` and
`applies_at: "next_invocation"`; set the named `environment` (or the per-call
`parameter`) for a new invocation and rerun. For example, a Ghidra decompile
deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`.
@@ -628,12 +684,10 @@ an exact result when the target bytes, operation, parameters, provider and
settings match. Mutations and cursor-dependent calls are excluded from the
cache. Snapshot files are local and use owner-only permissions.
One-shot binary CLI Evidence reports the exact path argument in `caller_input_path`,
including relative spelling and symlink aliases. Its subject identifies the
inspected artifact; `caller_input_path` does not change the Evidence ID or
snapshot query. A replay reports the current invocation's path while leaving
the saved Evidence unchanged. Provider inspections continue to include the
established `parameters.input_path` field.
A binary CLI replay reports the current invocation's selected path while leaving
the saved Evidence and snapshot query unchanged. See
[caller-selected paths](../reference/mcp-contracts.md#caller-selected-paths)
for primary and auxiliary path fields and their relation to Evidence identity.
Importing an Evidence bundle refreshes retained request and subject display metadata.
It preserves cached analysis results, including observations for other targets.
@@ -742,6 +796,12 @@ rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-
Migration: replace `rea compare-bundles` with `rea compare`; the former command
has been removed. The MCP tool remains `compare_bundles`.
`rea evidence-import` performs process-local bundle authentication and
validation, reporting record and Unknown-revision counts. `unknowns_added`
counts revisions, not distinct Unknown IDs. It does not retain records in a
separate or background MCP server process. To retain records across calls in an
active MCP session, use the MCP tool `import_evidence_bundle` in that session.
`rea compare` compares records by Evidence identity and explicit pairs. It does
not read artifact contents: the inventories of two builds are unresolved
records, or one opaque change when you pair them. Use `compare-artifacts`,
@@ -763,8 +823,8 @@ protection still applies. A genuine pathname refusal uses `resource_constraint`
with `details.reason: path-limit` and `boundary: filesystem-path`.
For application traces and version comparisons, see
[JavaScript application workflows](javascript-application-workflows.md).
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md).
MCP [retained Evidence references](../reference/mcp-contracts.md#retained-application-evidence-inputs)
belong to one connection. Separate CLI calls can consume full saved records or
resolve exact retained IDs from an explicitly selected local bundle:
@@ -972,23 +1032,23 @@ for percent signs, query marks and fragments.
Historical-source import requires safe no-follow file opens on Linux or
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
or another supported host. See
[source-to-bundle comparison](javascript-application-workflows.md#historical-source-to-bundle-comparison)
[source-to-bundle comparison](../guides/javascript/javascript-application-workflows.md#historical-source-to-bundle-comparison)
for mapping identities, inferences and unknowns.
## Capture runtime behavior
Choose the guide for the target and intended interaction:
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Electron fuses](electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
- [Desktop application capture](desktop-application-capture.md): launch a Linux desktop application and record its windows and home-directory state at checkpoints while it is used.
- [Desktop application state](desktop-application-state.md): read a capture folder into the application's state files, typed views of each version and kinds of change.
- [Desktop application export](desktop-application-export.md): write a capture folder as an agentenv computer app bundle with its class, skill and seed.
- [Desktop application tasks](desktop-application-tasks.md): describe how a capture's files changed, then validate an agent-written task against the capture and export it with its verifier.
- [Browser observation](../guides/web/browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](../guides/web/browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](../guides/javascript/electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Electron fuses](../electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](../guides/javascript/javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](../guides/runtime/process-capture.md): run an executable and compare terminal, exit and filesystem observations.
- [Desktop application capture](../desktop-application-capture.md): launch a Linux desktop application and record its windows and home-directory state at checkpoints while it is used.
- [Desktop application state](../desktop-application-state.md): read a capture folder into the application's state files, typed views of each version and kinds of change.
- [Desktop application export](../desktop-application-export.md): write a capture folder as an agentenv computer app bundle with its class, skill and seed.
- [Desktop application tasks](../desktop-application-tasks.md): describe how a capture's files changed, then validate an agent-written task against the capture and export it with its verifier.
Runtime requests name the target and actions. Launched targets run with your
user permissions; consult the chosen guide for host requirements and effects.
@@ -1027,7 +1087,7 @@ one client at a time: a session ends with the client's `DELETE`, and a client
that connects while another session is open takes it over, so a client that
exited without ending its session needs no restart. The listener runs until
SIGINT or SIGTERM and does not stop when stdin closes. See
[transports](mcp-contracts.md#transports) for the session, `Host` header and
[transports](../reference/mcp-contracts.md#transports) for the session, `Host` header and
shutdown contract. Client registrations written by setup and the MCP registry
entry keep stdio.
@@ -1052,7 +1112,7 @@ answers every method with JSON-RPC `-32601`. The stopped document's `state`
member is `{"served": true, "evidence_id": …}`,
`{"served": false, "reason": "replay-only"}` or
`{"served": false, "reason": "unavailable", "code": …, "message": …}`; see
[serve from the state](agent-environments.md#serve-from-the-state).
[serve from the state](../agent-environments.md#serve-from-the-state).
```sh
rea environment-serve manifest.json --evidence capture.json --port 8765
@@ -1072,7 +1132,7 @@ MCP_ALLOWED_HOSTS=127.0.0.1,rea.lan rea environment-serve manifest.json --eviden
the served environment; set but blank, it is refused as `configuration_invalid`
like a blank `MCP_HOST`. The admitted client URL is written to stderr as one JSON log line
(`"msg":"MCP transport listening"`). See
[agent environments](agent-environments.md) for what the state and replay
[agent environments](../agent-environments.md) for what the state and replay
return and do not establish.
## Build an environment bundle
@@ -1097,7 +1157,7 @@ command. `--dry-run` writes nothing and may omit the directory.
rea environment-build grafana.har ./grafana --json
```
See [agent environments](agent-environments.md#build-a-bundle-in-one-step)
See [agent environments](../agent-environments.md#build-a-bundle-in-one-step)
for the report, the hints and what the bundle ships.
## Export an environment bundle
@@ -1130,9 +1190,26 @@ name Windows reserves (`con`, `nul`, `com1` and the like) is refused. The
command prints one JSON document listing the written files with their sizes,
the pinned package source, the served `tool_count`, and `state` in the shape
of the serve command's stopped document. See
[agent environments](agent-environments.md#export-a-bundle) for the folder
[agent environments](../agent-environments.md#export-a-bundle) for the folder
layout and what the smoke task checks.
`rea environment-export` prints the written files and exits `0` on success.
Input-pair refusals, an occupied destination, a missing package tarball, and
invalid labels fail before writing output. Host permission failures identify
whether reading or writing was denied and retain the affected path.
A failure during writing can leave a partial folder. Inspect it before removing
it and retrying; the next attempt refuses a non-empty destination. When one
file caused the failure, the diagnostic names that file. A folder-sync failure
can report only its cause.
If `details.partial_observation.kind` is `environment-bundle-publication`, the
bundle was already published. Its `directory` and `files` record the committed
output, even when a later identity check or cleanup failed. Inspect that bundle
and resolve the reported failure before using or replacing it; the receipt is
not an instruction to delete published output. Cleanup resources remain a
separate obligation.
## Compare an environment with its application
`rea environment-compare <manifest.json> --evidence <capture.json> --probe <probe.json> [--probe <probe.json> ...]`
@@ -1143,7 +1220,7 @@ answer is compared with the application's after ids are mapped and clocks and ge
makes no network call. It prints the report and exits `0`, or prints a structured `invalid_request` error and exits `1`
for the same pair refusals as `environment-serve`, a probe that is not capture Evidence, or a browser probe captured
without response bodies, or without the request bodies its writes need. See
[agent environments](agent-environments.md#measure-fidelity-against-the-application) for the report.
[agent environments](../agent-environments.md#measure-fidelity-against-the-application) for the report.
```sh
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
@@ -1324,62 +1401,39 @@ retains failed stages for retry; the CLI exposes no separate retry command.
or prints the refusing stage's structured error and exits `1`, writing nothing
when the refusal comes before the bundle is written. SIGINT cancels it with a
`cancelled` error; a cancellation before the bundle stage writes nothing.
`rea environment-export` is one-shot: it prints the written files and exits
`0`, or prints a structured `invalid_request` error and exits `1` for the same
pair refusals, a non-empty or file-occupied directory, a missing package
tarball or a name that is not a kebab-case label or is a Windows device name,
writing nothing in each case. A file occupying the destination is reported as
requiring a directory; identity-change diagnostics are reserved for admitted
paths that fail later identity checks. Host permission failures identify whether
input reading or bundle writing was denied and retain the reported path.
A write that fails after the first file (a permission, disk-space or
pathname-length failure on a later path) propagates as the host's failure,
naming its cause and, when it failed on one bundle file or the package
tarball, that file. A failure while syncing the finished folder names only its
cause. Either way the partial folder stays in place for inspection; remove it
before retrying, since the next attempt refuses the non-empty directory.
A `cancelled` result does not by itself determine the exit status. Commands
that handle SIGINT or SIGTERM assign `130` or `143` after their operation and
cleanup settle; their structured result can be printed before that status is
assigned. Read the process status and typed result separately.
A `cancelled` result does not by itself determine the exit status; it depends on
the command's signal handling. Commands routed through the shared
`withCommandCancellation` handler in the CLI set status `130` for SIGINT or
`143` for SIGTERM after their operation settles. Current source registrations
use that handler for process capture and comparison (`capture-process`,
`compare-process-captures`), desktop application capture and state
(`capture-desktop-application`, `inspect-desktop-application-state`,
`export-desktop-application`, `describe-desktop-application-task`), JavaScript application analysis (`analyze` when it
selects the JavaScript lane, and `analyze-javascript-application`), selected
browser, web and Electron commands (including `analyze-web-bundle`,
`observe-web-execution`, `capture-browser-scenario` and
`capture-electron-scenario`), and the platform and provider command families
for Android/ADB, Apktool/JEB/Flutter, firmware, PCAP and signal analysis,
WASM/EVM, Go/Rust and binary diagnostics, Frida instrumentation, the Cutter
command, objdump, Rizin and GDB, and the agent environment server
(`environment-serve`). The handler also wraps `doctor`,
`inspect-sqlite-database`, `recover-javascript-sources`,
`inspect-il2cpp-metadata`, and `inspect-desktop-application`. In these paths, a returned structured
`cancelled` result can be printed before the wrapper assigns the
signal-derived status. This describes the current source routing; it is not a
claim that every command or provider runtime has been independently exercised.
Commands with `--evidence_output` save the complete Evidence bundle before
delivering the selected view. For committed-file receipts, retained partial
Evidence and missing source records, see
[Evidence publication and source recovery](../reference/mcp-contracts.md#evidence-publication-and-source-recovery).
Other outputs have their own receipt shapes; for example,
[environment export](#export-an-environment-bundle) reports a published directory.
Commands with `--evidence_output` select their delivered view, then publish the
complete Evidence bundle atomically. Cancellation before publication leaves no
new file. A publication that has committed can win a later cancellation; the
`cancelled` result then keeps the written path and byte count under
`details.diagnostics.evidence_bundle`. If staging cleanup stays incomplete
after commit, the `cleanup_incomplete` error reports the same file as
`details.published_path` and `details.bytes`.
When a failure retains a completed observation and no bundle committed,
`details.source_closure` states what that observation cites. `publication` is
`not_published`. `records` holds authentic source Evidence this invocation
captured, within 64 KiB. `omitted` lists up to 64 other identities and
`omitted_count` the total: `not_captured` means the invocation holds no such
record, so nothing is shown for it or its ancestors; `not_inlined` means it was
captured but is not in this response. `complete` is true only when every cited
record is in `records` or embedded in the observation. If examining the closure
itself fails, `collection_failure` names the error type and the closure state is
unknown; the original failure is still returned. Nothing is retained after exit,
and running the command again is a new observation.
A failure that carries partial Evidence in `details.partial_observation`, such
as `provider_timeout` or `resource_constraint`, publishes that partial Evidence
to `--evidence_output` under the same new-file and atomic rules, then keeps its
error code, message and remediation when cleanup is confirmed. With `--detail summary`,
`details.partial_observation` holds the summary view of the partial Evidence
instead of the complete record, and `details.diagnostics.evidence_bundle`
reports the written path and byte count. Linked source records must all be
available before publication; missing sources leave the destination unchanged
and are reported in `details.source_closure`. A provider cleanup failure can
still publish its captured Evidence without clearing the cleanup error or its
resource ownership. If publication is refused before commit, for example because
the file already exists, the complete partial Evidence stays inline and `details.diagnostics.evidence_bundle_error` gives the
refusal's code, message and a limitation. A refusal after commit keeps the
receipt in `details.diagnostics.evidence_bundle` and the delivered view: when
staging cleanup stays incomplete, the code becomes `cleanup_incomplete`, the
original failure is retained in `details.execution_failure`, and `details` also carries `published`,
`published_path`, `bytes`, `cleanup: "incomplete"` and `resources`, as for a
successful run, and a later cancellation reports `cancelled` in
`evidence_bundle_error` with the signal's exit status. Without
`--evidence_output`, and for cancellation before publication or raw provider
observations, the failure output is unchanged.
When the reader of piped output closes early, as `rea ... | head` does, the
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
@@ -1423,7 +1477,7 @@ the total token count and have no next offset.
Inspect an explicit local FDT v16/v17 blob without execution or an external tool.
Returns header, reservations, node/property tree, derived standard values and
selected phandle references. See [devicetree inspection](devicetree.md) for
selected phandle references. See [devicetree inspection](../devicetree.md) for
limits and retained-view pagination.
```bash
@@ -1439,7 +1493,7 @@ rea inspect-pe-load-config /absolute/path/to/application.exe --json
Reports TLS callbacks and size-covered load-config fields without execution.
Use `--max-callbacks` and `--max-file-bytes` to set read budgets. See
[PE TLS and load configuration](pe-load-config.md).
[PE TLS and load configuration](../pe-load-config.md).
### Binary hardening
@@ -1449,9 +1503,16 @@ rea inspect-binary-hardening /absolute/path/to/program --json
Reports ELF, PE, and Mach-O hardening rows without execution. A missing import
is `unknown`, not `disabled`. Use `--max-file-bytes` to set the read budget. See
[binary hardening](binary-hardening.md).
[binary hardening](../binary-hardening.md).
Bun standalone executables use the existing `inspect-artifact` and
`extract-artifact` commands to inventory and materialize embedded source,
sourcemaps and bytecode without running the executable. See
[Bun standalone executables](bun-standalone.md) for layout coverage and limits.
[Bun standalone executables](../bun-standalone.md) for layout coverage and limits.
Deno compile executables use the same commands to inventory and materialize the
embedded virtual filesystem, remote modules, and recorded options. Inventory retains
embedded environment-file values and certificate bytes in local Evidence with their
counts and byte lengths, and extraction does not write them as files. The npm snapshot
length is reported and not decoded, and the executable is not run. See
[Deno compile executables](../deno-standalone.md).
@@ -43,7 +43,7 @@ Rerun setup to review the registration and restart the client.
executable and environment of the terminal running doctor. It does not launch
the client's configured command or reproduce the client's environment. If doctor
passes but the client disconnects, inspect the client's startup error and registered
launcher; see [verification scope](client-verification.md#launch-environment-verification).
launcher; see [verification scope](../development/client-verification.md#launch-environment-verification).
`rea doctor` has the same limit for provider checks, and reports a provider setting
the registration's file does not carry as a
@@ -68,7 +68,7 @@ not update an already running MCP server or desktop client.
To select this profile for an agent, set `REA_MCP_INPUT_SCHEMA_PROFILE` in that
client's existing REA registration `env` object (`environment` for OpenCode) and
restart the client, preserving its other settings. A successful terminal doctor run does not verify the client's
launch environment. See [launch-environment verification](client-verification.md#launch-environment-verification)
launch environment. See [launch-environment verification](../development/client-verification.md#launch-environment-verification)
and the client-specific instructions below. REA setup forwards supported selections,
including `REA_MCP_INPUT_SCHEMA_PROFILE`, from its environment into the registration;
it does not copy unrelated environment values. Restoring the shell's previous
@@ -83,7 +83,7 @@ raising a REA startup or operation deadline, separately ensure the client allows
enough time for the full request. Use `inspect_analysis_settings` for the running
server's defaults; `rea doctor` checks launch configuration and cannot observe
another process's runtime overrides. See the
[settings design](design/limits-and-recovery.md) for ownership and lifetime.
[settings design](../design/limits-and-recovery.md) for ownership and lifetime.
## Client tool names and discovery
@@ -102,13 +102,15 @@ does not repair a client's naming behavior or infer permission drift from an
unfamiliar prefix. The unavailable fork reported in
[#5512](https://github.com/morluto/rea/issues/5512) has not been reverified.
<a id="hotcold-catalogs"></a>
## Hot/cold catalogs
For a client that supports MCP catalog refresh, set
`REA_MCP_TOOL_CATALOG_PROFILE=hot-cold` in REA's server environment and restart
the connection. REA then advertises a small initial catalog and loads other
named tools through `discover_tools`. Setup preserves this explicit setting.
See the [discovery contract](mcp-contracts.md#hotcold-catalog-discovery) for
See the [discovery contract](../reference/mcp-contracts.md#hotcold-catalog-discovery) for
pagination, identity and family-scope semantics.
Analysis tools, including `inspect_analysis_view`, load on demand. For example,
@@ -155,7 +157,7 @@ REA_MCP_TOOL_CATALOG_PROFILE=full
This advertises all tools without requiring catalog refresh. Compact output
schemas widen client-side validation; REA still validates complete results
against the canonical contracts. See [output schemas](mcp-contracts.md#output-schemas).
against the canonical contracts. See [output schemas](../reference/mcp-contracts.md#output-schemas).
On main `3e0599a94`, a real FastMCP 4.1.6 / Python MCP 2.3.0 HTTP client with
the default 1 MiB SSE event limit rejected the 4,108,975-byte full catalog.
@@ -179,7 +181,7 @@ receive limits and full Evidence access still apply.
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
preserving your REA client entry. See the
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
[setup instructions and public skills](../guides/web/web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
## VS Code
@@ -217,7 +219,7 @@ tool inventory on Linux using a loopback model fixture. After setup, add
settings, then restart the client. Compact advertisements can reduce nested
schema constraints; REA still validates complete canonical inputs. Full-profile
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
Windows remain unverified; see [native client verification](testing.md).
Windows remain unverified; see [native client verification](../development/testing.md).
## Claude Desktop
@@ -329,7 +331,7 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
override.
- To remove entire tool families from the upfront catalog, run
`rea setup --client claude_code --tool-families <ids>` with the families the
workflow needs (see [the advertising scope contract](mcp-contracts.md#tool-family-advertising-scope)).
workflow needs (see [the advertising scope contract](../reference/mcp-contracts.md#tool-family-advertising-scope)).
Out-of-scope tools are not advertised and their calls fail with the SDK's
disabled-tool error, so widen the selection when a task needs another
family. Setup writes the scope into the entry's `env` as
@@ -346,7 +348,7 @@ Both keep canonical result validation and complete structured results; `compact`
keeps a widened root shape and `off` advertises nothing. On catalogs large
enough that full input schemas alone approach the frame limit, also keep the
input profile at its `compact` default. See [MCP output schema
profiles](mcp-contracts.md#output-schemas).
profiles](../reference/mcp-contracts.md#output-schemas).
## Kimi Code
@@ -391,7 +393,7 @@ doctor reports its readiness separately. Explicit selections override discovered
defaults, including when setup runs again with a changed selection. Unrelated
environment variables, process ownership tokens and one-shot target paths are not
copied. For an explicitly configured startup target, see the
[initial-target settings](mcp-contracts.md#initial-target-at-startup). Existing
[initial-target settings](../reference/mcp-contracts.md#initial-target-at-startup). Existing
client environment overrides remain intact unless setup supplies the same key.
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
caller-selected values aligned and reports a non-blocking advisory when a tool
@@ -502,7 +504,7 @@ standalone CLI check.
A model API's `400 INVALID_ARGUMENT` alone does not identify a rejected schema,
tool-count limit or skill defect. Preserve the response's parameter path and
diagnostic, the actual running REA version, Gemini CLI version and selected
model. For schema rejection, review the [input presentation profiles](mcp-contracts.md#compact-input-schema-profile)
model. For schema rejection, review the [input presentation profiles](../reference/mcp-contracts.md#compact-input-schema-profile)
and reconnect after changing REA's registration environment. Compact is the
current default; an explicitly stored `full` choice is preserved. The generic
error in [#4620](https://github.com/morluto/rea/issues/4620) does not establish
@@ -583,7 +585,7 @@ Responses-compatible Kimi gateway as “infinite recursion”; expanding those
references succeeded. A separate full-profile input failed the provider's
schema-size limit. These are distinct reported failures, not proof that the
canonical schema is recursive. Current REA inputs are already self-contained;
see [schema advertisement](mcp-contracts.md#schema-advertisement).
see [schema advertisement](../reference/mcp-contracts.md#schema-advertisement).
For this failure, inspect the active MCP launch command and refresh the REA
installation that command actually runs. A configured package selector or the
@@ -832,7 +834,7 @@ in `~/.agents/skills` (`$DSH_AGENTS_HOME/skills` when configured); Harness
discovers that directory, and its native `skill` tool can load
`reverse-engineer-anything`.
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
REA's optional [native client verification](../development/testing.md#end-to-end-integration-and-golden-evidence)
checks headless chat, skill loading, all tool input schemas and actual analysis
using the installed Harness and a local model fixture. This establishes the
Harness integration, with live DeepSeek API compatibility remaining unverified.
@@ -898,7 +900,7 @@ servers if you are adding REA to an existing configuration. `timeout_ms` is
llama.cpp's per-tool-call timeout, including cold engine startup. The example
allows REA's default 330-second Ghidra startup budget plus 120 seconds for the
operation. Increase it if you configure a longer provider allowance or need more
operation time; see [provider deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
operation time; see [provider deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
Start the server with your own tool-capable GGUF model:
@@ -933,7 +935,7 @@ keep the server local, and select targets and operations you are authorized to u
See llama.cpp's [MCP server configuration](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md#mcp-servers)
and [function-calling guidance](https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md).
The [verification record](client-verification.md#llamacpp) distinguishes connection
The [verification record](../development/client-verification.md#llamacpp) distinguishes connection
checks from model behavior and lists the tested versions.
## Other client settings
@@ -3,14 +3,15 @@
Run REA, connect your selected agents, and check the tools needed for your task.
Setup previews configuration changes and asks for approval before applying them.
| Task | Section |
| ---------------------------------- | --------------------------------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Install a Claude Code/Codex plugin | [Agent plugins and marketplaces](#agent-plugins-and-marketplaces) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
| Task | Section |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Install a Claude Code/Codex plugin | [Agent plugins and marketplaces](#agent-plugins-and-marketplaces) |
| Run unreleased code | [Run REA from a checkout](#run-rea-from-a-checkout) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](../providers/ida-provider.md), or [Kuna](../kuna-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
## Start setup
@@ -374,7 +375,7 @@ choosing an operation; it works before a target is opened, and
`result.tool_availability` reports per-operation availability, reasons and
client requirements for the current session state. It does not guarantee that
a particular caller-supplied input will be valid. See
[MCP discovery](mcp-contracts.md#identity-and-discovery).
[MCP discovery](../reference/mcp-contracts.md#identity-and-discovery).
Select the part you want to check:
@@ -384,14 +385,22 @@ Select the part you want to check:
| One analysis engine | `rea doctor --provider ghidra --json` (or `hopper`, `ida`, `jadx`, `kuna`) |
| One agent registration | `rea doctor --client codex --json` (see [client IDs](#supported-agents)) |
| Installed workflow instructions | `rea doctor --skill --json` |
| Desktop app environment builds | `rea doctor --provider desktop_build --client claude_code --json` |
A scoped report has `scope.mode: "explicit"`. Its `scope_checks` determine
`healthy` and the exit status. Other checks appear in `informational_checks`;
`environment_healthy` summarizes the full audit. `--target` adds a target check,
but the report remains audit-wide unless a scope option is also supplied.
Provider checks read the terminal running doctor. A client-started server may
receive a different environment, and doctor cannot observe it. When an available
With one selected client, Ghidra, IDA, JADX and Kuna provider checks use the
client's recorded environment over the terminal it inherits. If that registration
cannot be read or a setting a provider needs cannot be resolved, its available
value is `null` and its check is `unknown`; its configured value is also `null`
when the configuration setting itself is unknown. Doctor does not infer
readiness from the terminal. A missing registration still uses terminal settings
for setup. Other checks, including Hopper, use the terminal running doctor. A
client-started server may receive a different environment, and doctor cannot
observe it. When an available
provider relies on a setting that a registration's file does not record, or
records with other text, doctor adds a non-blocking
`registration_environment:<client>` check with `classification: "unknown"`. The
@@ -595,6 +604,36 @@ feature selection. The same package version string in a development checkout
does not establish that its bytes match the npm tarball. Update a registration
through a reviewed, scoped setup plan, then restart/reconnect the agent.
### Run REA from a checkout
To run code that is not released yet, build a checkout with the Node.js
version in `.nvmrc`:
```bash
git clone https://github.com/morluto/rea.git
cd rea
npm ci
npm run build:cached
```
Then run `npm link` once in the checkout to put its `rea` on your `PATH`, or,
wherever these docs say `rea …`, run
`node /absolute/path/to/rea/scripts/rea.mjs …`. Setup run this way registers
the checkout's `scripts/rea.mjs` with your agents.
After `git pull`, run `npm ci` and `npm run build:cached` again. If the
checkout's skill changed, refresh each installed copy before restarting its
agent. For example, preview and review the Claude Code skill changes:
```bash
rea setup --skill-client claude_code --dry-run --json
```
Then apply the reviewed changes with `rea setup --skill-client claude_code --yes --json`.
Substitute the relevant skill client; use the checkout's
`node /absolute/path/to/rea/scripts/rea.mjs` form if `rea` is not linked. For desktop builds, also pack the [build computer's runtime
archive](../desktop-build-computer.md#requirements) again.
## Skill-only installation
```bash
@@ -698,7 +737,7 @@ Linux architectures; configure [Ghidra](#ghidra) there instead. REA itself runs
Linux distribution with a recognized apt, dnf, or pacman package family, where
an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra
and IDA have their own provider-specific host requirements; Windows Ghidra uses
the [experimental P0 boundary](windows-ghidra-p0.md).
the [experimental P0 boundary](../providers/windows-ghidra-p0.md).
Bazzite uses RPM packages, but REA does not install Hopper automatically on it.
Setup skips the Hopper offer because Bazzite is outside the supported host list,
@@ -1025,7 +1064,7 @@ output-stream closure; `output_closed: false` keeps that uncertainty explicit.
Later requests retain the known exit instead of reducing it to socket loss.
An external GUI's helper exit does not establish that the GUI itself exited.
For cursor navigation, string decoding, symbol-name limits, and regex matching,
see [Hopper analysis behavior](native-investigation.md#hopper-analysis-behavior).
see [Hopper analysis behavior](../guides/native/native-investigation.md#hopper-analysis-behavior).
Closing or switching a target closes its bound Hopper document, shuts down REA's
bridge and removes its temporary socket directory. The macOS Hopper application
@@ -1150,13 +1189,13 @@ session metadata edits leave executable bytes unchanged and are discarded on clo
require Hopper; Windows P0 remains read-only.
Windows P0 admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs,
[DOS MZ executables](ghidra-dos.md), plus explicitly
selected [raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied
[DOS MZ executables](../providers/ghidra-dos.md), plus explicitly
selected [raw NMOS 6502 images](../providers/raw-binary.md#nmos-6502) with a caller-supplied
entry at file offset 0, on fixed local NTFS volumes. Other raw CPU families remain
unsupported on Windows.
The npm package bundles native Job Object ownership, protected private runtime
DACLs, and handle-based path admission; no separate addon installation is needed.
See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and
See the [Windows Ghidra P0 guide](../providers/windows-ghidra-p0.md) for verified scope and
limitations.
Extract Ghidra and install the JDK outside REA, then export absolute paths:
@@ -1199,7 +1238,7 @@ REA skill after approval. Managed registrations use the absolute selected Node o
Bun runtime and REA entry script, including when setup ran through a package runner. Hopper
installation remains unavailable on Windows. Setup never installs Ghidra,
Java, or Python. It preserves valid detected Ghidra/JDK settings in agent
registrations. See [Windows Ghidra P0](windows-ghidra-p0.md) for provider diagnostics.
registrations. See [Windows Ghidra P0](../providers/windows-ghidra-p0.md) for provider diagnostics.
The separate `rea mcp add` builtin delegates most clients to Incur's package-runner
child launcher. Run that acquisition from a trusted directory; prefer managed
@@ -1317,7 +1356,7 @@ passes the deadline to each provider client; a startup failure is returned by th
without exposing partial analysis. This deadline is separate from MCP transport
initialization and the client's deadline for that individual tool call. A client
can time out earlier even when Ghidra would complete within its startup deadline.
See [Ghidra first-query deadlines and recovery](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
See [Ghidra first-query deadlines and recovery](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for client options and the close/reopen recovery flow.
The 330-second default can be changed on a running MCP server by setting
@@ -1370,7 +1409,7 @@ invocation instead. In MCP, `ghidra.decompile_timeout_seconds` can override the
value for newly accepted operations on the existing Program; explicit `null`
removes the deadline. A transient timeout remains uncached so the same function
can be retried after an update. See the
[settings design](design/limits-and-recovery.md#runtime-settings-through-mcp).
[settings design](../design/limits-and-recovery.md#runtime-settings-through-mcp).
Codex/Kimi registrations derived by `rea setup` add the selected launch value
to the tool deadline as a single-decompile estimate; a runtime update does not
change that registration. A bulk call can need that limit for
@@ -1558,6 +1597,11 @@ For supported header-selected MIPS ELF targets, the profile also binds the
selected target's ELF metadata; it does not claim a fresh-import MIPS lane was
used for the saved project.
A matching recorded import digest does not claim that the database is unedited.
Caller and callee coverage therefore remains unknown. A single-direction query
with known omissions remains partial and also reports the unknown project history.
Observed call edges are still available: a path to an observed caller or callee
can be reached, but a missing path is unknown when the project's prior analysis
cannot establish completeness.
Saved REA seed/analyzer reports remain historical facts in retained source Evidence;
they are not interpreted as overrides applied by this session.
Project content participates in snapshot identity, so different project states
@@ -1735,7 +1779,7 @@ contains the launch arguments and `/proc/<jvm-pid>/status` reports
session and retain the resource flags needed for the measurement rather than
copying full command lines or environments into reports. Heap and affinity
settings affect cost and latency; they are not part of REA's semantic analysis
profile identity. See [real-provider testing](testing.md) for bounded
profile identity. See [real-provider testing](../development/testing.md) for bounded
verification workflows.
### Ghidra diagnostics and verification
@@ -1845,7 +1889,7 @@ Code, `codex plugin list` in Codex).
For analysis engines, ask the connected server: have the agent call
`binary_session` and read `result.tool_availability`, which reports what the
server the plugin started can use, with reasons and remediation. See
[MCP contracts](mcp-contracts.md).
[MCP contracts](../reference/mcp-contracts.md).
A plugin does not put `rea` on your `PATH`. To check an engine from a
terminal, use the package runner at the version the plugin pins:
File renamed without changes.
+2 -2
View File
@@ -1,7 +1,7 @@
# GameCube DOL analysis through Ghidra
This lane imports 32-bit big-endian PowerPC GameCube/Wii DOL images through
Ghidra. It does not execute the image. DOL is a headerless format with no magic
Ghidra. It does not execute the image. DOL has a section and entry-point header but no magic
number, so REA never detects it: the caller must select it explicitly.
Only the cached MEM1 window `0x80000000..0x81800000` is admitted. Wii MEM2
@@ -29,7 +29,7 @@ MCP, with the existing `open_binary` tool:
}
```
The architecture comes from the DOL header (`powerpc`). Do not pass
Selecting the DOL format supplies the architecture (`powerpc`). Do not pass
`architecture` with `gamecube-dol`; it is rejected. `entry_point` is optional,
and if given it must equal the header entry.
+26 -1
View File
@@ -144,9 +144,34 @@ keys, decode Godot resources, decompile GDScript bytecode, analyze signals,
apply patch/removal entries or execute targets.
Readable logical member bytes, including repeated physical ranges, share the ZIP
archive expansion policy: at most the larger of 256 MiB or 500 times the PCK's
archive expansion policy by default: at most the larger of 256 MiB or 500 times the PCK's
encoded byte span. Embedded carrier bytes do not enlarge that budget. The full
directory is checked before any member is yielded, hashed or extracted. Encrypted
members are not read and do not consume this plaintext-work allowance. Small
bounded aliases remain valid. This is a processing/output resource bound, separate
from the directory metadata retention budget and format validity.
Directory retention defaults to 64 MiB of encoded metadata. Select
`artifact_limits.godot.directory_bytes` to change it; path/source facts reserve
worst-case JSON expansion before declared path bytes are read, and the reader
then charges actual retained metadata. A preallocation `required_value` is the
allowance needed by that admission check, not a measurement of final output size.
Select `artifact_limits.expanded_bytes` independently to change aggregate
plaintext work, including repeated readable logical occurrences. Both controls
apply to inspection and extraction, before any member is yielded.
For example:
```sh
rea inspect-artifact ./game.pck --artifact-limits '{"godot":{"directory_bytes":134217728},"expanded_bytes":1073741824}' --json
```
MCP uses the same object under `artifact_limits` on `inspect_artifact` or
`extract_artifact`. A refusal reports the exact `reader_limit` parameter,
effective value, unit, supported range and required value when known. Raise the
named admission and retry the same complete PCK on the next call; no REA restart
is needed. Summary or page size does not change producer admission. Omitted
policy selections stay omitted in Evidence parameters; a selected object records
the accepted selection and `effective_limits`, conditional on which readers are
encountered. Raising these policies does not admit malformed offsets, unsupported
flags, unsafe extraction paths or unavailable encrypted contents.
@@ -113,8 +113,8 @@ existing analysis families directly, for example
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves its Success/Failure verdict, records `replace_requested`, and leaves actual replacement `null` because prior package state is not observed |
| `uninstall_adb_package` | **mutates device** | Requests package removal for user 0 via `adb shell pm uninstall --user 0`, without data retention |
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -W -n`) |
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -W -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
@@ -188,7 +188,7 @@ and during shutdown.
property value remains available in `properties`.
- **Device paths are not host paths.** `base_apk_device_path` values live on
the device filesystem. Only pulls and captures produce host-local paths,
each paired with byte size and SHA-256 digest. A pulled file's host name is
with byte size and SHA-256 digest on success. A pulled file's host name is
the device path's final POSIX segment. REA preserves the original device
path and filename; a missing final segment or `.`/`..` is refused before
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
@@ -249,6 +249,10 @@ and during shutdown.
- Pull and capture outputs are additive: the destination must not already
exist. Re-running a completed pull refuses the existing directory instead
of overwriting evidence.
- If a single-file pull fails and REA observes a regular file at the destination,
the error includes its `local_path` with `transfer_status: "unconfirmed"`
and `digest_status: "unknown"`. The file remains in place; the receipt does
not establish that the transfer completed.
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
@@ -288,7 +292,7 @@ Record: verified against adb 34.0.5 on Linux against an Android 14
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
287 services, 92 features, a real two-APK split set pulled with byte-exact
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
complete package lifecycle; see `docs/testing.md` for the lane summary.
complete package lifecycle; see `docs/development/testing.md` for the lane summary.
Intent integer extras use signed decimal text. The ADB argument encoder removes
redundant leading zeros for Android's radix-sensitive `--ei` parser while retained
@@ -6,12 +6,12 @@ kept as an unmodified, commit-pinned Git submodule; see
application workflow and return artifact-bound Evidence inline.
This family is present on repository main and in npm 4.1.0. Check the
[released package boundary](installation.md#released-package-and-main) and the
[released package boundary](../../getting-started/installation.md#released-package-and-main) and the
connected server's tool list before selecting it.
On Linux, JADX launches an REA-owned process and requires a procps-compatible
`ps` on REA's `PATH`; see the shared
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
## Optional package delivery
@@ -545,7 +545,8 @@ and each upstream request and reply collected before the failure. Failed replies
remain available even when their payload cannot be interpreted. Cancellation
also stops target hashing before provider admission.
These operations do not execute the APK or provide split APK/AAB handling,
The JADX package, class, method and reference operations do not execute the APK
or provide split APK/AAB handling,
signature validation, full resource-table semantics, native-library analysis or
Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
Android tools and the application graph do not treat it as a standalone APK.
@@ -687,14 +688,14 @@ bounds each upstream call. Timeout recovery reports the effective milliseconds
and range, with `details.limit` identifying the runtime setting. An invalid
provider-local launch value can also be repaired by a runtime override; reset
exposes that invalid launch value again. See the
[settings design](design/limits-and-recovery.md#invalid-launch-configuration).
[settings design](../../design/limits-and-recovery.md#invalid-launch-configuration).
A passing JADX configuration check in `rea doctor` reports its launch-derived
`operation_timeout_ms`; it does not inspect another running server's overrides. Explicit
cancellation still ends the operation immediately, and partial observations
and owned-process cleanup are unchanged. Raising the deadline does not extend
the bridge's 30-second startup handshake or the upstream **90-second**
decompilation limit. MCP clients enforce their own tool-call timeouts, and
setup's derived [client tool timeout](client-configuration.md) does not include
setup's derived [client tool timeout](../../getting-started/client-configuration.md) does not include
this setting, so also raise the client timeout when selecting a longer
deadline.
@@ -66,6 +66,8 @@ rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR
rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR --string-tables-only
rea decode-android-resources /targets/Example.apk --no-strings
rea decode-android-resources /targets/Example.apk --decode-timeout-ms 900000
rea decode-android-resources /targets/Example.apk --string-offset 2000 --string-limit 100 --string-tables-only
rea decode-android-resources /targets/Example.apk --max-string-table-bytes 16777216 --max-total-string-table-bytes 67108864
```
Equivalent MCP methods are `inspect_apktool_client` and
@@ -87,6 +89,31 @@ responses and charges the aggregate output budget for one copy per table.
Omitting the option preserves existing responses. `include_strings: false`
still suppresses all string projection, including in this mode.
Use `string_offset` and `string_limit` (CLI `--string-offset` and
`--string-limit`) to select a page of valid parsed entries from each requested
table, in source order. Defaults are offset 0 and limit 2000. Explicit page or
budget controls include `string_tables` even for a single primary table; its
`page` gives the requested offset and limit, returned count, and `next_offset`.
Retry that offset with the same locale and confirm `target.sha256` matches.
Each call decodes a fresh owned snapshot; it does not retain a workspace or
promise identity across changed APKs. A null continuation means the page reaches
the end, while table coverage remains partial if earlier entries were omitted.
An offset at or beyond the end returns an empty, present table with known counts.
If the continuation equals the requested offset, no output entry fit: increase
`max_total_strings` or use `string_tables_only` before retrying.
`max_total_strings` (CLI `--max-total-strings`, default 10000) bounds returned
string copies across all tables. Raise it, request fewer locales, or use
`string_tables_only` when aggregate output is exhausted. The legacy
multi-table shape charges both copies of primary strings. `max_string_table_bytes`
(CLI `--max-string-table-bytes`, default 8388608) bounds one XML source read;
`max_total_string_table_bytes` (CLI `--max-total-string-table-bytes`, default 33554432) bounds XML reads across distinct physical tables. Locale aliases
share the physical read and parsed page. Raise the relevant read budget to
retry an oversized table; a larger string limit or timeout does not raise it.
These controls take nonnegative safe-integer offsets and positive safe-integer
budgets, apply on the next decode, and require no REA restart. Process memory
and response transport capacity still apply.
## Interpret the result
- **An owned APK snapshot is digested and decoded.** `target.sha256` and
@@ -117,8 +144,8 @@ still suppresses all string projection, including in this mode.
Malformed XML and a wrong document root are whole-document projection
failures: the result is partial, the failure reason is disclosed, and entry
counts remain unknown. They are not counted as one unsupported string.
The default table is bounded to 2000 entries with partial coverage when more
exist. `locales` lists observed regular `strings.xml` tables whose exact
The default page returns up to 2000 entries with partial coverage when more
exist; its limitation names the next `string_offset`. `locales` lists observed regular `strings.xml` tables whose exact
`values-<locale>` directory is selectable by the request's locale grammar.
Locale suffixes allow up to 23 ASCII characters, derived from Apktool's
language, script, region, and variant fields; for example,
@@ -133,9 +160,10 @@ still suppresses all string projection, including in this mode.
When multiple tables are requested with `locales`, `string_tables` records
each projected table with its locale, status (`present`, `missing`,
`oversized`, `malformed`, or `skipped`), entry counts, and coverage,
bounded at an aggregate budget of 10,000 strings and 32 MiB XML. When `locales`
is omitted and `string_tables_only` is not true, legacy single-table payloads are preserved
without adding `string_tables`.
with default aggregate budgets of 10,000 strings and 32 MiB XML. When `locales`
is omitted, `string_tables_only` is not true and no string page or budget
control is selected, legacy single-table payloads are preserved without adding
`string_tables`.
- **Smali is skipped by design** (`apktool d --no-src`): Java-source
recovery belongs to the JADX family. Resource smali edits and rebuild are
a follow-up lane with mutation authority.
@@ -156,8 +184,8 @@ still suppresses all string projection, including in this mode.
- The version probe waits 20000 ms and retains 65536 bytes. This release has
no runtime override for that probe. `decode_timeout_ms` does not raise it.
- Manifest reads are bounded to 512 KiB before decoding UTF-8. Strings XML
reads are bounded to 8 MiB; larger documents are skipped with partial
coverage and an explicit limit, rather than reported as missing.
reads default to 8 MiB per table; larger documents are skipped with partial
coverage and an explicit read-budget recovery, rather than reported as missing.
- `decoded_file_count` is capped at 100,000 files; the cap is reported in
the limitations when hit.
- An ordinary nonzero decode exit surfaces as `unsupported_target` naming
@@ -174,7 +202,7 @@ OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
locales — launcher version, on-disk digest agreement, apktool.yml metadata
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
tables projected. Parser goldens in the test suite come from the same
decode. See `docs/testing.md` for the lane summary.
decode. See `docs/development/testing.md` for the lane summary.
Multi-locale and table-only projection were also verified through real CLI and
stdio MCP workflows on Linux with Apktool 3.0.3/JDK 21 against NewPipe 0.29.1,
@@ -217,3 +217,47 @@ The Gallery workflow and all 147 focused Flutter tests also pass on macOS Sonoma
including the conflicting-load and note-range-limit CLI/MCP regressions. Windows
verification runs under a non-administrator account. Verifiers use a shared
temporary parent so each CLI/MCP process can own and release its own snapshots.
## Selecting and recovering string-pool observations
`inspect_dart_aot` accepts `pool_policy`; the CLI uses `--pool-policy` JSON:
```sh
rea inspect-dart-aot app.apk --pool-policy '{"contains":"package:my_app/"}'
rea inspect-dart-aot app.apk --pool-policy '{"uri_limit":2000,"token_limit":4000,"run_bytes":4096,"token_bytes":256}'
```
A literal, case-sensitive `contains` filter runs before distinct collection,
so an omitted URI or identifier can be selected without returning every other
string. URI/token counts describe matching entries; `printable_run_count` counts
complete printable runs before filtering. Unset budgets preserve the defaults:
500 returned entries per URI category, 2000 identifier-like tokens, 1024 bytes
per printable run, and 128 bytes per identifier. Budget values must be positive
safe integers. Collection retains at most twice the selected category limit;
counts remain lower bounds when that capacity is reached. A run exceeding
`run_bytes` is omitted whole, rather than reported as a fabricated prefix.
The result names the budget reached, and `string_pool.policy` records the
selected filter and effective values. A larger budget or a narrower selection
applies to the next call without a REA restart; explicit choices stay in Evidence.
For a larger package-URI inventory, select the category and its relevant budget:
```sh
rea inspect-dart-aot app.apk --pool-policy '{"contains":"package:","uri_limit":2048}'
```
The equivalent MCP input is `pool_policy: {contains: "package:", uri_limit: 2048}`.
This recovers all bounded package-URI observations in the measured Gallery and
LocalSend APKs. Check the result's limits and coverage for other targets; long
runs and unsupported snapshot layouts remain separate limitations. The
[calibration report](../../verification/flutter-budget-calibration.md)
keeps the default at 500 because increasing every complete overview also delivers
more strings when they are unnecessary. Use `detail: "summary"` for the MCP
identity/count task described above.
These controls do not raise APK, ELF, library or response budgets, deserialize
Dart objects, or establish typed class/function semantics. Retained
`inspect_analysis_view` pages cover the returned selection; exhausting those
pages does not recover facts that the producing pool scan omitted. Re-run the
producer with the reported selection or budget change to obtain a new
observation, checking its APK and library digests before comparing it.
@@ -117,8 +117,8 @@ are never passed to TShark.
Installed libraries, global configuration and plugins remain trusted, are not
fully fingerprinted, and can influence dissection. This is not an OS sandbox.
Input is limited to 32 MiB before reading, each command to 30 seconds, and
combined stdout/stderr to 4 MiB during collection. Output overflow fails instead
Default limits are 32 MiB input before reading, 30 seconds per command, and
4 MiB combined stdout/stderr during collection. Output overflow fails instead
of returning truncated success. Before execution, `prlimit` enforces **1 GiB
address space per process**, **64 MiB per file**, **30 CPU seconds** and no core
dumps. Stricter inherited soft/hard limits are preserved; Evidence records the effective
@@ -128,6 +128,23 @@ These are not aggregate process-memory or filesystem quotas. The selected
limiter is trusted; its path, digest and arguments are retained in Evidence.
No arbitrary process flags are accepted.
Each capture/recording operation accepts `producer_limits` with positive safe
integer overrides: `input_bytes`, `output_bytes`, `command_timeout_ms`,
`address_space_bytes`, `file_bytes`, and `cpu_seconds`. The wall-clock timeout
is at most 2,147,483,647 milliseconds, matching Node timers. Omitted fields keep
the defaults above. Discovery accepts the same controls except `input_bytes`,
since it does not read a capture or recording. Packet field discovery defaults
to 32 MiB output rather than 4 MiB.
CLI supplies this object as JSON with `--producer-limits`, for example
`--producer-limits '{"input_bytes":67108864,"output_bytes":8388608,"cpu_seconds":60}'`.
Values apply to the next call without restarting REA. Effective native ceilings
still preserve any tighter inherited soft and hard limit. Errors identify the
exhausted producer control and retain partial output and cleanup status; follow
that repair after cleanup settles. Raising producer budgets does not raise
separate parser representations, transport buffers or external client deadlines.
Never truncate the selected artifact to fit an input budget.
## Verification
```sh
@@ -182,8 +199,9 @@ proof that the field appears in a selected recording.
The adapter uses the documented `tshark -G fields` grammar under its existing
private empty profile and Linux process limits. TShark 4.6.9's full glossary was
measured at 22,303,197 bytes; discovery admits at most **32 MiB** combined command
output (capture inspection remains at **4 MiB**). Parsing walks lines once and
measured at 22,303,197 bytes; discovery defaults to **32 MiB** combined command
output (capture inspection defaults to **4 MiB**); `producer_limits.output_bytes`
can change either request budget. Parsing walks lines once and
constructs objects only for the selected page. Returned strings are copied so
retained pages do not retain the complete glossary string. Before constructing a selected
record it charges six times the source UTF-8 bytes plus 512 bytes against an
File renamed without changes.
@@ -90,9 +90,20 @@ column limit. The selected database is readable and is not reported as
malformed; request fewer columns per call instead. `row_limit`
requires `table`, defaults to 20 and accepts positive safe integers. Samples
remain subject to the byte and worker resource limits below, regardless of row count. The result reports
the selected limit, returned count and whether more rows were observed. Row
order is unspecified; a truncated sample does not prove anything about omitted
records. View, virtual-table and shadow-table row reads are unsupported.
the selected limit, returned count and whether more rows were observed.
Rows are read in a stated unique order reported as `rows.order`: `rowid` for
rowid tables, and `primary_key` for `WITHOUT ROWID` tables, reusing the
primary-key index's own direction and declared collation. `rows.row_offset`
reports the rows skipped before the window and `rows.next_row_offset` the offset
to request for the following page; it is `null` once the window reaches the end.
Pass `row_offset` (CLI `--row-offset`, requires `table`, default 0) to skip into
a table rather than repeating earlier rows. A table that declares all of
`rowid`, `_rowid_` and `oid` as columns and has no primary key reports
`order: "unspecified"` and refuses a non-zero `row_offset`, because the skipped
prefix would be arbitrary. Pages compose only while the reported database and
WAL SHA-256 match; re-inspecting a changed snapshot restarts the sequence.
View, virtual-table and shadow-table row reads are unsupported.
SQLite integers are decimal strings so every signed 64-bit value survives JSON
serialization. Real values retain their numbers; `"Infinity"`, `"-Infinity"`
@@ -118,16 +118,25 @@ CLI keeps its existing complete result.
## Resource and lifecycle limits
Inputs are limited to 4 MiB, complete replies to 16 MiB and combined diagnostic
output to 1 MiB. An owned worker has a 256 MiB JavaScript heap; its profile
maxima are 3 GiB virtual address space and 30 CPU seconds, with tighter inherited
limits retained. The worker command deadline is
`timeout_ms` (default 30000 ms, integer 1 through 2147483647) on the next call,
or `--timeout-ms` on `rea inspect-evm-interface`. Omission keeps 30000 ms.
Those byte, address-space, and CPU ceilings have no runtime override, and
`timeout_ms` does not raise them. An oversized selected carrier returns a
non-retryable `resource_constraint` with resource `file-size` and its fixed
input allowance in `reported_limits.maximum_input_bytes`.
Per-call `producer_limits` selects `input_bytes`, `output_bytes`,
`diagnostic_bytes`, `address_space_bytes`, `file_bytes` and `cpu_seconds`.
Omission keeps 4 MiB input, 16 MiB reply/file, 1 MiB diagnostics, 3 GiB address
space and 30 CPU seconds. Omitted `file_bytes` follows the selected
`output_bytes`, so raising the reply allowance does not silently meet the old
file-write allowance. Explicit file limits remain independent. These fields
accept positive safe integers; changes apply to the next call without a REA
restart. The CLI accepts the same exact policy as JSON, for example:
```sh
rea inspect-evm-interface ./runtime.hex hex --producer-limits '{"input_bytes":8388608,"output_bytes":33554432}'
```
The owned worker's 256 MiB JavaScript heap remains separate. The worker command
deadline is `timeout_ms` (default 30000 ms, integer 1 through 2147483647), or
`--timeout-ms` on the CLI. Raising it does not raise producer, heap or client
limits. An oversized carrier reports non-retryable `resource_constraint` with
`reported_limits.maximum_input_bytes` and recovery naming
`producer_limits.input_bytes`.
Complete-reply allowance refusals use the same non-retryable resource family,
with `reported_limits.maximum_reply_bytes`; malformed JSON remains unreadable
output. This refusal leaves no complete report to retrieve through another
@@ -135,7 +144,7 @@ delivery channel. Virtual address space differs from resident memory. The worker
soft/hard limits, reports its configured soft limits inline and leaves hard limits
unchanged. WASM trap reservations are disabled
so the engine can run within that virtual limit; actual memory is not equated to
its 3 GiB allowance. Limits fail without partial success. Stable snapshots,
the selected address-space allowance. Limits fail without partial success. Stable snapshots,
private files and owned processes are cleaned independently of caller cancellation.
Invalid-input, unsupported, cancellation, timeout and malformed-reply failures
retain bounded stdout/stderr under `details.captured_output`, including whether
@@ -11,7 +11,7 @@ select a returned ELF separately for supported Ghidra analysis.
On Linux, Binwalk and Unblob run as REA-owned processes and require a
procps-compatible `ps` on REA's `PATH`; see the shared
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
## Compare stored firmware components
@@ -21,7 +21,7 @@ For supported FIT and legacy uImage containers, use the built-in
Unblob or extraction: exact logical paths, stored-byte hashes and declared
metadata establish component changes, with source Evidence and explicit unknowns.
The outer container at `.` is separate from its children. See the
[FIT version comparison recipe](fit-images.md#comparison) for CLI,
[FIT version comparison recipe](../../fit-images.md#comparison) for CLI,
MCP, summary delivery and selected supporting facts. Its real-data verification
covers a pair of OpenWrt FIT releases; it does not establish comparison of
decoded root filesystems or arbitrary firmware formats.
@@ -209,7 +209,7 @@ Select a verified native child for the existing tools:
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
Native architecture and executable validation still apply. Generic
[MIPS ELF](ghidra-mips.md) and [RISC-V ELF](ghidra-riscv.md) lanes are separate
[MIPS ELF](../../providers/ghidra-mips.md) and [RISC-V ELF](../../providers/ghidra-riscv.md) lanes are separate
documents with their own admission boundaries. This milestone does
not add PPC decompilation, raw flash load profiles, bootloader/device
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
@@ -317,7 +317,7 @@ bootloader image, such as an OTA download or app partition, for its chip,
classified segments, checksum and SHA-256 results, descriptor and Secure Boot
v2 signature blocks. `extract-artifact` / `extract_artifact` writes one file
per segment and signature block. Reading is portable and needs no external
tool. See [ESP images](esp-image.md).
tool. See [ESP images](../../esp-image.md).
## Inspect ESP flash dumps
@@ -328,11 +328,11 @@ facts for the bootloader and each app partition, and the OTA records with a
derived selection; the running partition stays unknown. Extraction writes
each partition, `nvs_keys` included, as an ordinary file. For a dump that
does not start at flash address 0, select `esp-flash-image` with
`flash_offset`. See [ESP flash images](esp-flash-image.md).
`flash_offset`. See [ESP flash images](../../esp-flash-image.md).
## Inspect extracted devicetrees
Use [`inspect_devicetree`](devicetree.md) / `rea inspect-devicetree` on an
Use [`inspect_devicetree`](../../devicetree.md) / `rea inspect-devicetree` on an
extracted DTB for its node/property tree, reservations, hardware summary and
selected phandle relationships. FDT v16/v17 inspection is portable and requires
no external tool or disassembler.
@@ -343,7 +343,7 @@ Use `inspect-artifact` / `inspect_artifact` on an Intel HEX or Motorola S-record
file for its address regions, declared start address and record facts, then
`extract-artifact` / `extract_artifact` for one raw file per region. Decoding is
portable and requires no external tool. See
[Intel HEX and S-record images](hex-records.md).
[Intel HEX and S-record images](../../hex-records.md).
## Inspect U-Boot legacy uImages
@@ -356,7 +356,7 @@ is extractable as `payload.bin`, and multi-file and script images additionally
expose each sub-image as `image-<n>.bin` from the stored size table. Declared
addresses are facts from the header, not proven entry points, and compressed
payloads stay as stored. Intel HEX/S-record and FIT images are separate
readers and are not covered here. See [U-Boot legacy uImage](uimage.md) for
readers and are not covered here. See [U-Boot legacy uImage](../../uimage.md) for
component padding, preserved names, resource policies and verification scope.
## Inspect UF2 and DfuSe images
@@ -366,7 +366,7 @@ address regions and its block numbering, or on a DfuSe file for its targets,
elements and DFU suffix, then `extract-artifact` / `extract_artifact` for one
raw file per region or element. A DfuSe suffix CRC mismatch follows
`integrity_policy`. Decoding is portable and requires no external tool. See
[UF2 and DfuSe images](uf2-dfuse.md).
[UF2 and DfuSe images](../../uf2-dfuse.md).
## Inspect U-Boot FIT images
@@ -374,4 +374,4 @@ Use `inspect-artifact` / `inspect_artifact` on a U-Boot FIT (`.itb`) for its
images, declared load and entry addresses, configurations and verified hashes,
then `extract-artifact` / `extract_artifact` for one raw file per image.
Decoding is portable and requires no external tool. See
[U-Boot FIT images](fit-images.md).
[U-Boot FIT images](../../fit-images.md).
File renamed without changes.
@@ -6,7 +6,7 @@ Disconnecting the inspection's page session can clear caller-owned
`navigator.platform` emulation inside Chromium. REA cannot restore another
client's complete user-agent/client-hint configuration. Use a disposable target
or let the owning client reapply its configuration; see the shared
[CDP state limitation](browser-observation.md#provider-authorities).
[CDP state limitation](../web/browser-observation.md#provider-authorities).
This passive runtime surface is distinct from the target-free static
[`analyze_javascript_application`](javascript-artifact-reconstruction.md)
@@ -14,7 +14,7 @@ workflow. Static analysis reads a local ASAR or extracted directory without
executing application code. Passive observation inspects an existing Electron
page. The results keep static inferences and runtime observations distinct.
Packaged fuse state is a separate static read:
[`inspect_electron_fuses`](electron-fuses.md) reports whether this binary will
[`inspect_electron_fuses`](../../electron-fuses.md) reports whether this binary will
honor Inspector flags, `ELECTRON_RUN_AS_NODE`, embedded ASAR integrity, and
`app.asar`-only loading.
Use the separate
@@ -265,6 +265,9 @@ retained arrays and counters without invoking application iterators or Array
species.
Listener-chain recognition distinguishes owned, unowned and unknown metadata.
Only a directly owned wrapper establishes hook dispatch. An application's data
link to an owned wrapper remains unknown: `.listener` can name that wrapper
without invoking it.
Data links may be inherited; absence is established only after completing the
bounded prototype lookup. Exhausting either walk, encountering a getter or seeing a Proxy leaves
ownership unknown. The hook preserves native registration without adding another
@@ -328,7 +331,7 @@ budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, seeded by
`update_analysis_settings` can override `asar.max_header_bytes` for new
operations without restarting REA. Cached headers are checked against each
operation's admission, including after a decrease. See the
[settings design](design/limits-and-recovery.md#failure-and-retry).
[settings design](../../design/limits-and-recovery.md#failure-and-retry).
An oversized header is
not parsed; selecting a member path does not bypass admission, and the page
stays unauthorized. The header cache separately retains at most that many bytes
@@ -50,6 +50,9 @@ A path-filtered reference reports the caller-selected path while keeping its
shared node identity. Item selection also accepts any retained path when it
identifies one module; if several module kinds share that path, select the
reported `node_id`. Node-ID selection uses the stable representative path.
Exported symbol bindings reuse the `javascript-module` kind but name one export
of their owning module, so module pages, items, and summary counts include only
module units and an exported module never becomes an extra path candidate.
## Retained module facts
@@ -180,7 +183,7 @@ large application can exceed one MCP response. Over MCP, pass
the session and return only selected rows with the trace's identity and
coverage. Page the retained trace with `inspect_analysis_view`; an oversized
complete reply offers its first `paths` page as a ready-to-call continuation.
See [MCP contracts](mcp-contracts.md) for filters. A handoff binds
See [MCP contracts](../../reference/mcp-contracts.md) for filters. A handoff binds
the exact native artifact digest and requested exports from the application
graph. Trace direction selects the reachable addons; requested
exports for each selected addon come from the complete supplied graph, so an
@@ -223,6 +226,12 @@ registrations/removals and dispatch, Node timers and cancellation handles, async
`node:child_process` creation and ownership, configuration sources and direct
defaults, request construction and response consumers, parse/coercion/
validation boundaries, and built-in resource acquisition/release.
For `fs/promises.open`, a binding receiving the Promise is distinct from one
receiving the awaited FileHandle. Awaiting the Promise later does not replace
the original binding, but `h = await h` can replace it with the FileHandle.
Receiver writes can therefore retain a candidate release and its ownership
ambiguity; a still-known Promise has no direct FileHandle release. Acquisitions
remain retained, and unknown loader authority keeps its existing candidate tag.
Reassignments connect retained right-hand reads to the binding or property
write. Compound assignments also retain the previous-value input; logical
assignments keep conditional inputs as candidates. These edges describe
@@ -632,7 +641,7 @@ All six CLI commands accept inline JSON or a path to a JSON file. The CLI
returns an Evidence record directly. Later workflow inputs can contain full
records, or exact retained references when `--evidence-bundle PATH` selects an
authenticated exported bundle. A separate CLI process has no retained MCP
connection state. See [CLI Evidence reuse](cli.md#import-export-and-compare-evidence).
connection state. See [CLI Evidence reuse](../../getting-started/cli.md#import-export-and-compare-evidence).
File inputs must be regular files; symlinks to regular files are accepted.
Directories, named pipes and device files produce an input error before JSON
@@ -665,7 +674,7 @@ References are scoped to one connection; `close_binary` clears them. Export a
bundle before closing and import it on another connection, or supply the full
inline Evidence there. Versions before 4.1.0 accept full inline Evidence only; installing
newer skill instructions does not change that schema. See
[MCP Evidence inputs](mcp-contracts.md#retained-application-evidence-inputs).
[MCP Evidence inputs](../../reference/mcp-contracts.md#retained-application-evidence-inputs).
For two operator-provided directories or ASARs, run:
@@ -22,7 +22,9 @@ rea analyze-javascript-application /absolute/path/to/apps/app.asar \
Generic `rea analyze` selects this static provider for directories and `.asar`
paths when neither `--provider` nor `--snapshot` is supplied. The dedicated
command and the generic route return their complete results inline. Native
command and the generic route return complete results inline by default.
Summary and route-view delivery export the full Evidence to `--evidence_output`
and return a projection; see [route delivery](../../javascript-route-inventory.md). Native
targets, single JavaScript files, `.app` bundles, and explicit deep-provider or
snapshot requests retain the native deep-analysis route.
@@ -86,6 +88,11 @@ properties and remains readable without migration.
## Select sources before analysis
Explicit `source_paths` retain the acquisition owner's requested order, including
selected dependency files. First-party priority applies only to unscoped analysis.
Recovered syntax or incomplete semantic examination keeps Electron coverage
partial even when useful windows or IPC facts were retained.
For a question about known entry sources, supply exact artifact-relative file
paths in `source_paths`, or repeat `--source-path` on either CLI route:
@@ -160,7 +167,7 @@ must also satisfy the new admission. For CLI or future MCP processes, set
`REA_ASAR_MAX_HEADER_BYTES` in the launch environment. Changing that environment
requires a new REA process. Invalid launch values are configuration errors,
never a silent fallback. See the
[settings design](design/limits-and-recovery.md#failure-and-retry) for recovery
[settings design](../../design/limits-and-recovery.md#failure-and-retry) for recovery
metadata, precedence, and reset behavior.
The archive index is read before any member can be selected, so
@@ -181,7 +188,9 @@ coverage, limitations, and `normalized_result.parent_evidence_id`. Pass that ID
to `inspect_analysis_view` for module pages or items, or to the application
workflows as a retained reference. The summary does not repeat or truncate the
analysis; a server without session retention refuses summary detail with
`capability_unavailable`. CLI output is unchanged and always complete.
`capability_unavailable`. On the CLI, `--detail summary --evidence_output PATH`
exports the full Evidence and returns its summary; the default remains complete
inline delivery.
With a connected MCP `client` and the caller-selected absolute `inputPath`, this
recipe analyzes once and pages module identities using the returned ID:
@@ -255,7 +264,7 @@ those missing bytes. New ASAR file occurrences record `unpacked: true` for
companion files and `unpacked: false` for packed entries, including when the
file bytes are unavailable. An omitted flag in an older inventory remains
unknown; it does not establish either storage mode. See
[MCP integrity handling](mcp-contracts.md#integrity-record-and-continue) for the
[MCP integrity handling](../../reference/mcp-contracts.md#integrity-record-and-continue) for the
tool-result contract.
JavaScript whose syntax cannot be parsed retains unavailable semantic coverage
@@ -280,7 +289,7 @@ does not bypass ASAR header/index admission.
If a complete response exceeds the transport budget, use its retained Evidence
ID for a summary, module page, or item, or export the bundle. See
[MCP tool results](mcp-contracts.md#tool-results) for delivery limits, export, and
[MCP tool results](../../reference/mcp-contracts.md#tool-results) for delivery limits, export, and
matching server/client buffer settings. Follow-up results also need to fit that
budget. [Analyzer limits](#analyzer-and-result-admission) control analysis memory,
source deadlines, and semantic capacity separately.
@@ -306,8 +315,11 @@ and caller selection remain available without implying applicable AST work.
MCP returns a `partial_observation` retained reference after recording succeeds;
open it with `inspect_analysis_view` on the same connection. If the SDK stopped
waiting before receiving the reply, follow [MCP cancellation recovery](mcp-contracts.md#progress-and-cancellation)
waiting before receiving the reply, follow [MCP cancellation recovery](../../reference/mcp-contracts.md#progress-and-cancellation)
to find the record. CLI failure output contains the complete partial Evidence.
With `--evidence_output`, the CLI also publishes it to that file and, with
`--detail summary`, prints only its summary view; see
[evidence publication](../../getting-started/cli.md#output-and-exit-status).
Owned analyzer cleanup is verified before private files are released; uncertain
cleanup retains ownership for retry and is reported separately.
Typed cancellation or another primary interruption remains attached while
@@ -323,6 +335,12 @@ failure does not authorize expanding relationships that parent admission decline
## What is reconstructed
Electron `BrowserView` and `WebContentsView` expose `webContents` and accept
`webPreferences`, including declared preloads. `BaseWindow` has neither surface
of its own. Preload associations retain the parsed constructor's mechanism rather
than labeling every view as a BrowserWindow. These are static associations, not
proof that a renderer or preload executed.
The projector reuses the content-addressed artifact inventory and safe artifact
readers. It retains canonical artifact-relative paths, exact byte counts,
SHA-256 digests, inventory IDs, ASAR container identity, and `.asar.unpacked`
@@ -515,6 +533,25 @@ If fingerprint construction reaches its structural
bound, the fingerprint is unavailable and its status is `truncated`; a bounded
prefix is never presented as a complete stable fingerprint.
## Cached-data compilation boundaries
The [feature specification](../../specs/javascript/v8-cached-data/spec.md) defines
the static admission and uncertainty contract.
A `node:vm` compilation reports whether `cachedData` is declared, absent or
unknown. Explicitly supplied undefined bytes are treated as absent; shadowed
identifiers remain declarations. Source syntax alone does not establish valid cache bytes, V8 acceptance
or execution. The summary counts declared compilations, not executions.
A proven synchronous filesystem read or awaited promise-based read supplies a
cache path. The producer retains its path base: source-directory expressions
can name inventoried artifacts, while relative literal paths depend on the
target working directory and remain unresolved. Other supplied values retain
their bounded source expression. Declared paths, exact-file lookup
status, candidate paths and resolution limitations remain separate; an unresolved
path never becomes a resolved file. Opaque inventoried cache bytes can remain
unavailable to the JavaScript resolver even when their occurrence is known.
## Authority and unknowns
API identity is a derivation: a familiar method or property name does not
@@ -729,7 +766,7 @@ These safeguards are not caller-selectable output budgets. Source-map parsing
reports truncation when its format or parser safety boundary is reached. The
application graph has no aggregate node, edge, root, or observation prefix cap.
Semantic evaluation retains at most 256 distinct primitive candidates per
By default, semantic evaluation retains at most 256 distinct primitive candidates per
expression and at most 1 MiB of worst-case JSON string bytes across a normalized
primitive value's string candidates. The byte budget estimates six JSON bytes
per UTF-16 code unit, the expansion bound for escaped strings, and bounds both
@@ -749,13 +786,48 @@ available for endpoint queries, tracing and comparison. Each analysis records en
later projections: a selected healthy sibling can remain exact while whole-file
coverage reports the encountered bound. Projection preserves limiting values
from the shared property-slot owner. Expression
evaluation and provenance walks stop after 256 nested levels. Union products
evaluation and provenance walks default to 256 nested levels. Union products
and string lengths are checked before allocating combinations or concatenated
strings. When a boundary is reached, the value stays unknown, semantic coverage
becomes partial, and graph evidence retains the expression location and typed
limiting reason. The number of unresolved alternatives is unknown; it is not
reported as an exact omission count.
`semantic_precision` selects per-source evaluator budgets independently of
retained graph capacity, worker memory and deadlines. Both
`analyze_javascript_application` and `analyze_web_bundle` accept a partial object:
| Field | Default | Unit |
| ---------------------- | ------: | ------------------------------------------- |
| `primitive_candidates` | 256 | alternatives per normalized primitive value |
| `primitive_bytes` | 1048576 | estimated JSON bytes per primitive value |
| `expression_depth` | 256 | nested expression/provenance levels |
| `getter_expansions` | 512 | scheduled states per precise getter query |
| `alias_selections` | 512 | additional precise selector states |
| `wrapper_visits` | 1000000 | wrapper visits/substitutions per source |
| `wrapper_depth` | 128 | active symbolic wrapper levels |
`primitive_bytes` charges six estimated JSON bytes per UTF-16 code unit plus
quotes before creating concatenated strings. Ordinary ASCII can exhaust that
conservative estimate before its actual JSON encoding reaches the selected
byte count. The [calibration record](../../verification/javascript-precision-calibration.md)
keeps existing defaults for its measured literal-value tasks; it does not
establish optimal candidate, getter or alias budgets for larger applications.
All fields accept positive safe integers; worker memory, execution and JSON
representation bounds remain independently enforced. Omission keeps the defaults;
invalid values fail validation. For example, retry a finite candidate-limit
observation with `semantic_precision: {"primitive_candidates": 512}`. The CLI
accepts `--semantic-precision '{"primitive_candidates":512}'` or a JSON file.
An override applies to that next analysis without restarting REA, remains in
Evidence parameters and effective coverage, and cannot change already retained
facts. Increasing these budgets does not raise graph retention, worker heap,
transfer admission, or parent result memory. Cyclic or unsupported evaluations
remain unknown. Wrapper limitations name visits and depth separately so a
caller can change the resource actually exhausted. Finite inline literal object
spreads use iterative syntax traversal and no longer inherit a duplicate
semantic-depth cut.
Byte, entry, path, and graph-shape bounds are hard limits. The parse deadline is
checked before and between bounded parsing and traversal phases; the synchronous
Babel and JSON parser calls cannot be preempted mid-call, so their input byte
@@ -810,7 +882,7 @@ analyzer; the caller's `NODE_OPTIONS` still controls the CLI/MCP process heap.
For defaults across new MCP operations, use `update_analysis_settings` with
`javascript.max_heap_mb` or `javascript.analysis_timeout_ms`. Explicit per-call
arguments take precedence, and accepted work keeps its original settings. See
the [settings design](design/limits-and-recovery.md#precedence-and-lifetime).
the [settings design](../../design/limits-and-recovery.md#precedence-and-lifetime).
V8 old-space limits do not impose a process RSS limit. When REA runs on Bun,
the analyzer ignores the V8 heap flags; resource failures then report
`worker_heap_limit_bytes` as null instead of a requested or V8-reported limit.
@@ -149,6 +149,6 @@ non-retryable `resource_constraint` with `resource: "file-size"` and
valid UTF-8 is an `invalid_format` request issue. These limits apply per
operation and do not claim aggregate host containment across concurrent callers.
See [testing.md](testing.md#javascript-source-recovery) for the focused real
See [testing.md](../../development/testing.md#javascript-source-recovery) for the focused real
Wakaru CLI/MCP lane and its source-owned compiler fixtures. Other host platforms
and Bun/Vue recovery are not claimed by this adapter.
@@ -37,7 +37,7 @@ application starts, and REA never resumes it or sends
caller-owned debugger before requesting application runtime observations.
For a packaged Electron app, read
[`inspect_electron_fuses`](electron-fuses.md) before starting an `electron-main`
[`inspect_electron_fuses`](../../electron-fuses.md) before starting an `electron-main`
target. When `EnableNodeCliInspectArguments` is disabled or removed, Electron
ignores `--inspect` and `--inspect-brk`. A missing main-process target after
launch does not, by itself, show that fuse.
@@ -114,7 +114,8 @@ caller cancellation still reports cancellation.
The direct observations are:
- `Debugger.scriptParsed`, establishing that a bounded script location was
parsed in the observed execution context;
parsed with the reported execution-context ID; association with a particular
context lifetime is known only when its creation was observed;
- `Debugger.scriptFailedToParse`, retained only as a bounded invalid-script
count in `scripts.parse_failed_total`. A well-formed parse-failure
event is distinct from malformed protocol data and does not make loaded
@@ -140,7 +141,7 @@ variants of one file remain separate observations. Older Evidence may omit
## Node and Electron roles
The caller declares one role: `node`, `electron-main`, `electron-preload`, or
The caller may declare one role: `node`, `electron-main`, `electron-preload`, or
`electron-renderer`. Node and Electron-main declarations require a protocol
target of type `node`; preload and renderer declarations require `page`.
Inspector does not authenticate the operating-system PID or distinguish an
@@ -163,8 +164,13 @@ and collection overhead, and projected URL fields. Each CDP message also has a
16 MiB transport limit. A message over that limit fails with `payload_limit`;
when the aggregate budget is reached, later metadata events are dropped and
the partial result reports `events_dropped`, `truncated: true`, and
`retained_metadata_budget_exceeded`. Repeated context transitions update the
current context record instead of accumulating duplicate objects. Scripts are
`retained_metadata_budget_exceeded`. Context records retain each observed lifetime, and scripts refer to the context
lifetime active when parsed. The reported numeric execution-context ID is
retained separately on both records; a script with no observed context creation
has an unknown context key. Repeated transitions within one lifetime update its
record; the state records the last observed lifecycle event and does not prove
that the context is still live. This changes context keys and Evidence identities from older captures;
recapture observations to obtain lifetime associations. Scripts are
validated after capture, deduplicated by stable metadata, and canonically
sorted. Wall-clock timestamps and protocol script IDs are excluded from the
durable result, so identical inputs and captured metadata produce the same
+121
View File
@@ -0,0 +1,121 @@
# V8 code cache inspection
`inspect_v8_code_cache` reads one explicitly selected V8 code cache container as static
bytes: a bytenode `.jsc`, the bytes `vm.Script.createCachedData()` writes, or a cache
carried inside a Node single-executable application or an Electron app. It never executes
the file, never starts a V8 runtime, never installs bytenode, and never invokes a
decompiler.
```bash
rea inspect-v8-code-cache /absolute/path/to/module.jsc --json
rea inspect-v8-code-cache /absolute/path/to/module.jsc --minStringBytes 8 --json
```
The MCP tool is `inspect_v8_code_cache` with the same inputs and the same retained
Evidence.
## What is read
A V8 code cache is a producer-written container, not a documented file format. This reader
admits one container shape and reports everything else as an observation, so a build it
has never seen still yields facts instead of a false parse.
The 32-byte header is eight little-endian 32-bit words. Every word is reported in
`header.fields` with its byte offset, its observed value, the role this reader believes it
has, and how confident that role is:
| Offset | Role | Status |
| -------------- | -------------- | ----------- |
| `0x00` | magic | confirmed |
| `0x04` | version hash | inferred |
| `0x08` | source length | confirmed |
| `0x0c` | flag hash | inferred |
| `0x10` | build constant | inferred |
| `0x14` | payload length | confirmed |
| `0x18`, `0x1c` | reserved | unconfirmed |
`confirmed` means the role is demonstrated by the reader's own tests against the producer.
`inferred` means the word's behaviour is consistent with the role but a different producer
could plausibly differ. `unconfirmed` means the word is reported and nothing more is
claimed.
Only three facts gate admission:
1. the header size this reader pins (32 bytes);
2. the V8 code cache magic high half, `0xc0de`. The low half is
`ExternalReferenceTable::kSize` and varies per V8 build, so it is reported as a build
fingerprint and never required to match a fixed value;
3. the declared payload length must equal the bytes that follow the header exactly.
`header.source_length` is the producer's recorded source length in UTF-16 code units.
V8's serializer calls this field the source hash; on the producers measured here its value
equals the source length exactly, including sources whose UTF-8 byte length differs, so the
reader reports it under the name the measurement supports. Treat that name as an
observation, not as a claim about what any other build stores there.
`header.producer` names the producer family from the flag bits in that word. Bit 29
marks `vm.compileFunction` (`compile_function`); bit 30 marks `vm.SourceTextModule`
(`module`). When neither flag is present, the producer remains `unknown`:
`vm.Script`, bytenode output and older unflagged function caches cannot be
separated by those bits alone. The source-length mask is `0x1fffffff`, so both
flags sit outside the reported length. This describes the admitted header
profile rather than a version-independent V8 ABI.
## Failed inspection and provenance
After a stable read completes, format and observation-budget refusals retain a
`file-snapshot-hash` partial observation with the selected path, exact SHA-256 and
byte count. This proves which bytes were read; it does not claim the format was
successfully decoded. A refusal before read completion does not invent an identity.
Descriptor cleanup remains owned for retry if finalization fails.
## Embedded byte patterns
`embedded_strings` lists printable runs found in the payload, each with its absolute byte
offset in the selected file and its observed byte length. `min_string_bytes` (default 4)
sets the shortest reported run; `max_strings` caps how many may be reported, and exceeding
it fails without partial success.
These are **byte-pattern observations inside serialized V8 heap data, not parsed heap
objects**. A reported run is not proof V8 retained it as a string. A literal that is absent
from the payload is not proof it is absent from the program: V8 may encode a string in a
form the run scan does not surface, and a string can be produced at runtime rather than
retained.
A run longer than 4096 bytes is reported with `truncated: true` and a `retained_bytes`
prefix.
## Limitations
Every result states these, and they are part of the contract:
- the payload is serialized V8 heap data, not source; no logic is recovered and no
bytecode is decompiled;
- recovering logic needs a version-pinned V8 decompiler (View8-style) that this tool
neither provides nor depends on;
- header roles beyond magic, source length, and payload length are version-dependent
observations;
- the tool never executes the selected file and never claims recovered source.
## Failure behaviour
An unknown magic is a typed `unsupported_target` failure, because the file is not a V8
code cache. A short header, a payload-length disagreement, or bytes past the declared
payload is a typed `invalid_request` failure. Neither produces completed inspection
Evidence; the stable-read partial observation described above is retained when available.
`max_file_bytes` (default 32 MiB, ceiling 64 MiB) is the caller-selected read budget. The
payload SHA-256 is computed over the declared payload, so two containers with the same
payload length and content but different version hashes have different whole-file digests
and identical payload digests.
## Relationship to other tools
- `inspect_binary_layout` reads the surrounding ELF/PE container of a single-executable
application; it does not decode the embedded code cache.
- A cache embedded in a Node single-executable application or an Electron app must be
**extracted first**: this reader admits a container whose magic is at byte zero, so
passing the surrounding ELF/PE executable here returns `unsupported_target`. Run
`extract_artifact` on the application, then pass the extracted `.jsc` path to this tool.
- `inspect_artifact` inventories the file without decoding V8 structures.
- This tool is in the same family as `inspect_python_pyc`, `inspect_erlang_beam`, and
`inspect_kernel_module`: one explicit local file, bounded parsing, no execution.
@@ -24,7 +24,7 @@ Native-body bridge mapping and managed runtime execution are outside the
current tool set.
The canonical tool inventory is the
[build-generated catalog](mcp-contracts.md#generated-catalog).
[build-generated catalog](../../reference/mcp-contracts.md#generated-catalog).
## Inspect an assembly
@@ -212,13 +212,9 @@ rea inspect-managed-members /absolute/path/Assembly.dll --type Namespace.Type \
bundle format. Its parent directory must exist, and it never replaces an
existing path. Relative paths resolve against the command's working directory.
The option also works with complete delivery when a saved bundle is useful.
Summary without a destination is rejected before analysis. A publication failure
returns a typed error with the completed Evidence under
`details.partial_observation`; it does not report successful summary delivery.
Cancellation before publication prevents the new output. Once atomic
publication has started, it can complete despite cancellation; a cancelled
delivery then retains the committed path and byte count under
`details.diagnostics.evidence_bundle`.
Summary without a destination is rejected before analysis. For partial Evidence
and committed-file receipts after failure or cancellation, see
[Evidence publication and source recovery](../../reference/mcp-contracts.md#evidence-publication-and-source-recovery).
Copy `normalized_result.parent_evidence_id` from the summary into the
`source.evidence_id` field of the `managed-members` request above, then save that
@@ -245,13 +241,13 @@ The canonical parser admits PE/CLI bytes and their metadata/CIL without loading
an assembly. It reports observed implementation markers and unavailable facts;
it does not itself unpack .NET single-file hosts or infer a NativeAOT identity
from an ordinary native PE. It does not decode IL2CPP; use
[IL2CPP metadata](il2cpp-metadata.md) for the image, type and method names in a
[IL2CPP metadata](../../il2cpp-metadata.md) for the image, type and method names in a
Unity `global-metadata.dat`. Inputs without admitted CLI metadata do not
become managed assemblies through naming or routing guesses.
Inspect separately obtained components explicitly and preserve their identities.
For ordinary .NET single-file hosts, use the existing artifact tools to
[inventory and extract bundle members](dotnet-bundles.md), then inspect the
[inventory and extract bundle members](../../dotnet-bundles.md), then inspect the
extracted assembly explicitly with the managed tools.
A valid implementation marker establishes a candidate native boundary, not
@@ -268,12 +264,30 @@ is retained in the classification evidence list.
### Resource admission
The static provider admits regular input files up to 128 MiB before retaining
their bytes. Reads remain bounded if a file grows after admission. Metadata,
CIL decoding, and projected inspection records have a separate conservative
64 MiB representation budget; shared heap strings are cached, but repeated
output occurrences still consume that budget. These are inspection resource
policies, not PE format limits or a hard bound on total V8 heap use.
The static provider defaults to a 128 MiB eagerly retained PE snapshot and a
separate conservative 64 MiB budget for metadata, CIL decoding and projected
inspection records. All three static inspection tools accept
`max_snapshot_bytes` and `max_metadata_representation_bytes` as positive safe
integers in bytes. CLI inspection and path comparison expose
`--max-snapshot-bytes` and `--max-metadata-representation-bytes`. An omitted
value keeps its default; explicit values apply to that call without a restart.
Snapshot refusal names its selected byte budget and tells the caller to raise
it for the same complete file. Representation exhaustion retains partial or
unavailable coverage and names the selected budget in its `resource-limit`
issue; raise that budget on the next inspection, or select fewer `type_names`
for members. For example:
```bash
rea inspect-managed-members /absolute/path/Assembly.dll --max-snapshot-bytes 268435456 --max-metadata-representation-bytes 134217728
```
Reads remain bounded if a file grows after admission. A cached snapshot is
rechecked against the current call's snapshot allowance and on-disk digest.
Shared heap strings are cached, but repeated output occurrences still consume
the representation budget. These are per-call resource policies, not PE format
limits or a bound on total V8 heap use. Raising one does not raise the other,
transport limits or available host memory.
Path-based member comparison uses the same per-input snapshot policy and
inspects its two selected paths sequentially, retaining each inspection rather
@@ -330,7 +344,7 @@ original `resource-limit` coordinate. It does not return partially admitted
member collections: empty fields, methods, references and edges are unexamined,
not proof of absence; exported types, user strings and signature-resolution
attestation remain null. Summary and retained pages keep this partial coverage.
The capacity limit is unchanged. If metadata or the type inventory itself cannot
The selected capacity applies to the metadata and member projection together. If metadata or the type inventory itself cannot
be admitted, the result remains unavailable.
Use the returned `types[].full_name` values to select declaring types before
@@ -608,13 +622,13 @@ describes exported shared-library entry points.
REA offers a built-in read-only snapshot parser for x64 PE/RTR 9.1 metadata and
an optional Linux headless Ghidra adapter for annotations and broader recovery.
The adapter requires a locally built `REA_GHIDRA_NATIVEAOT_JAR`; REA does not
install it or its toolchain. The [NativeAOT guide](ghidra-nativeaot.md) defines
install it or its toolchain. The [NativeAOT guide](../../providers/ghidra-nativeaot.md) defines
these routes, their layout/host restrictions, derived-memory provenance, and
verification coverage.
REA's experimental Windows Ghidra boundary admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE
applications and DLLs for static analysis. For x64 PE NativeAOT, the read-only
parser also runs there; the [NativeAOT guide](ghidra-nativeaot.md) lists what a
parser also runs there; the [NativeAOT guide](../../providers/ghidra-nativeaot.md) lists what a
Windows host covers.
For NativeAOT analysis, preserve the selected native artifact's path and digest
@@ -623,6 +637,7 @@ to native function addresses requires verified
provider evidence; names or vtable similarity alone do not prove that a native
function implements a specific managed method.
<div id="managednative-boundary-rules"></div>
<div id="managed-native-boundaries"></div>
## Managed/native boundary rules
@@ -653,7 +668,8 @@ unchanged signature. Comparison retains exact byte-based pairs for malformed
signatures, including empty, truncated, or invalid headers, but reports them as
unknown with a `signature-malformed` limitation when no change is observed.
This applies to historical Evidence without signature-resolution attestation as
well. Runtime-provided methods are counted separately in
well. Runtime-provided methods have the Runtime code type or the InternalCall
implementation flag, as core library FCalls do. They are counted separately in
`native_implementations.runtime_provided`; they have no file-backed native body
for this workflow to map and do not increase `native_body_unresolved`. Historical
verification Evidence without this separate count leaves it unknown (`null`).
@@ -677,7 +693,7 @@ partial when unknown MethodDef ownership can hide a selected body.
Metadata tokens are not native addresses. Native body mapping for C++/CLI,
ReadyToRun, NativeAOT, and IL2CPP requires separately verified format and provider
support. The managed static tools do not recover those mappings. The optional
[Ghidra NativeAOT extension](ghidra-nativeaot.md) has its own supported boundary.
[Ghidra NativeAOT extension](../../providers/ghidra-nativeaot.md) has its own supported boundary.
Declared import inventory supports a bounded positive claim. Its absence does
not exclude dynamic resolution, generated code, protected code, native helpers,
@@ -849,7 +865,7 @@ Target platform is not host support. The canonical parser must inspect
Windows-produced PE/CLI bytes on REA's supported Linux and macOS hosts with the
same result. Pinned Windows oracle jobs may provide additional conformance, but
do not establish that the REA managed analysis workflow has been verified on a
Windows host. The [experimental Windows Ghidra boundary](windows-ghidra-p0.md)
Windows host. The [experimental Windows Ghidra boundary](../../providers/windows-ghidra-p0.md)
has its own host and provider verification scope. Native
ReadyToRun, C++/CLI, NativeAOT, or IL2CPP coverage is additionally constrained
by the selected Hopper/Ghidra host and format matrix.
@@ -860,7 +876,7 @@ The current lane uses source-owned byte-built fixtures and malformed-input
regressions, with optional operator-supplied applications and ILSpy verification.
Run the deterministic lane with `npm run verify:managed`. See
[testing](testing.md#real-toolchain-verification-lanes) for prerequisites and
[testing](../../development/testing.md#real-toolchain-verification-lanes) for prerequisites and
coverage. Generated outputs belong outside tracked fixture directories and must
not remain after verification or packaging.
@@ -120,23 +120,30 @@ CLI output are unchanged.
Complete results have a 32 MiB input, 64 MiB reply, 1 MiB combined diagnostics
and 30-second owned command deadline. `REA_PWNTOOLS_MAX_INPUT_BYTES` raises the
input bound alone, up to 1 GiB; the reply and deadline bounds are unchanged. The
launch input default alone, up to 1 GiB. Per-call `producer_limits` selects
`input_bytes`, `output_bytes`, `diagnostic_bytes`, `address_space_bytes`,
`file_bytes`, and `cpu_seconds`; positive safe integers apply on the next call
without restart. `input_bytes` overrides the launch default. File admission
follows selected `output_bytes` unless `file_bytes` is explicit. The separate
`pwntools_timeout_ms` controls each command wall deadline. On CLI pass
`--producer-limits '{"cpu_seconds":60,"output_bytes":134217728}'`. The
reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
copies and encoding overhead. The
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results).
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](../../reference/mcp-contracts.md#tool-results).
These settings affect delivery, not decoder coverage or limits. The large
extended-index verification uses 256 MiB and an explicit five-minute request
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to the selected policy: defaults are 3 GiB
virtual address space, 30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
retained, and the effective values are returned in the result limitations. Virtual address space is not RSS;
Unicorn needs a large virtual map. Limits fail with no partial success, and
owned process/root cleanup completes independently of cancellation.
Core files, GDB session/state/control, optional pwndbg enrichment and instruction
inspection are separate increments tracked by #969. This increment makes no
real support claim for those features or other operating systems/architectures.
For core files and optional pwndbg mapping context, use the
[recorded-crash workflow](recorded-crashes.md). The binary-layout operations
described here do not provide debugger session state or control, or establish
support for other operating systems/architectures.
Typed decoder failures retain bounded `captured_output` (stdout, stderr and an
explicit truncation flag) alongside the original failure category and reason.
@@ -272,6 +279,12 @@ regular files only and rejects symbolic links.
## Byte entropy triage
Entropy admission preserves the caller's spelling in `caller_input_path`. On
Windows, the selected regular file's case-only filesystem spelling is used
consistently in committed paths through both CLI and MCP. Symlink, junction and
short-name aliases are not replaced with a different path. General path
normalization remains lexical and does not probe source-discovered paths.
`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
without an analysis engine, execution, extraction or network access.
+177
View File
@@ -0,0 +1,177 @@
# ELF unwind function ranges
`inspect_elf_unwind_ranges` / `inspect-elf-unwind-ranges` reads `.eh_frame`
and `.eh_frame_hdr` from one selected local ELF32/ELF64 image and lists the
function ranges those tables name. It reads the file without executing it and
does not require a disassembler.
```bash
rea inspect-elf-unwind-ranges /absolute/path/to/program --json
```
The CLI resolves relative paths against the caller's working directory. MCP
requires an absolute filesystem path and preserves that exact selection:
```json
{
"name": "inspect_elf_unwind_ranges",
"arguments": { "path": "/artifacts/program" }
}
```
Pass `--offset` and `--limit` (MCP `offset` and `limit`) to page the ranges.
The default page starts at 0 and returns at most 256 ranges.
## Observations
The inline Evidence result contains the artifact path, SHA-256, and byte
length, plus ELF class, byte order, machine, and object type.
`eh_frame.status` is `present`, `absent`, `incomplete`, or `unknown`. A
present table names its source: `section` when section headers name
`.eh_frame`, or `program` when section headers are missing and
`PT_GNU_EH_FRAME` supplies the bytes. `location` is the file range that was
examined. A `.eh_frame` section that is `SHT_NOBITS` or lies outside the file
is `incomplete` with a null `location`. One that runs past the end of the
file is `incomplete`, `location` is clamped to its in-file bytes, and the
records in those bytes are still decoded. Only the first `.eh_frame` and
`.eh_frame_hdr` section is decoded, and only the first `PT_GNU_EH_FRAME`
program header. A duplicate section, or a duplicate `PT_GNU_EH_FRAME` when no
`.eh_frame_hdr` section supplies the header, keeps `coverage.status` at
`incomplete`.
`eh_frame_hdr` reports version, the three pointer encodings, the `.eh_frame`
pointer, the declared FDE count, and how many binary-search table entries
were decoded. A header section that is `SHT_NOBITS` or extends past the end of
the file is `incomplete` with a null `location` and is not decoded.
`coverage.eh_frame_hdr_agreement` is `agree` when the table has exactly one
row per decoded FDE, the sorted starts match including repeats, and each row
points at a distinct FDE with that start. It is `disagree` otherwise, and
`unavailable` when the header is absent, unknown, or incomplete. Disagreement
is a limitation. The decoded FDEs stay in the result. The search table is
read for at most `max_ranges` rows, and for no more rows than the header bytes
can hold. When the declared FDE count is larger than either bound, the header
is `incomplete`, `table_entries` counts only the rows read, the comparison is
`unavailable`, and a limitation names the cap. A declared count that does not fit
in 32 bits is not decoded: `fde_count` is null, the header is `incomplete`,
and the decoded FDEs stay in the result. A header whose FDE count encoding is
`DW_EH_PE_omit` (`0xff`), as GNU ld writes when it cannot build the search
table, is `present` with a null `fde_count`, no table rows, and comparison
`unavailable`; it does not make coverage incomplete. A header that declares
an FDE count but writes `DW_EH_PE_omit` as its table encoding is `present`
with that count, no table rows, and comparison `unavailable`, and also keeps
coverage complete. A non-zero FDE count under an indirect encoding is the
address of a slot that is not followed, so `fde_count` is null, the header is
`incomplete`, and the table is not read.
When section headers name both `.eh_frame_hdr` and `PT_GNU_EH_FRAME`, the
section is decoded. A limitation names the segment as ignored only when it
declares different bytes than the section. When no `.eh_frame_hdr` section is
named, the header is read from `PT_GNU_EH_FRAME` even if a `.eh_frame`
section is present.
A record whose length word is `0xffffffff` uses the 64-bit extended length
that follows it. In `.eh_frame` the CIE ID and CIE pointer after that length
stay 4 bytes wide.
`DW_EH_PE_datarel` fields in `.eh_frame_hdr` are relative to the header
address. In `.eh_frame` they are relative to the GOT: `DT_PLTGOT` when the
dynamic table names it, otherwise the `.got.plt` or `.got` section address.
A limitation names the base that was used. When no GOT base is declared,
those values are unresolved: the affected record is not reported and
`coverage.status` is `incomplete`.
`debug_frame.status` is `present`, `absent`, `incomplete`, or `unknown`. It
is `unknown` when the image has no section headers or unreadable section
names. It is `incomplete`, with a null `location` and `coverage.status` at
`incomplete`, when the section is `SHT_NOBITS` or its bytes are not all in
the file. The `.eh_frame` results stay in the report. This command does not
decode `.debug_frame`.
`cies` lists the CIEs that the returned page's ranges name, in index order,
so its length never exceeds the page size. Each `cie_index` in `ranges`
matches the `index` of one entry in `cies`. `coverage.total_cies` counts every
decoded CIE, including CIEs no FDE on this page uses. Each CIE reports
version 1 or 3, augmentation, code and data alignment,
return-address register, pointer encoding, LSDA encoding, personality address
when the augmentation carries one, signal-frame and B-key flags, and the byte
length of the CFA program. CFA instructions are counted and not evaluated.
A CIE whose augmentation has a letter other than `z`, `L`, `P`, `R`, `S`, or
`B` is not decoded: the encodings of later letters cannot be trusted, so its
FDEs are not decoded either. A limitation counts those FDEs and coverage is
`incomplete`.
`ranges` is one page of FDEs. Each range has `start`, `end`, decimal `size`,
the containing `SHF_ALLOC` section, an exact symbol name when a symbol starts
at that address, the CIE index, LSDA, and personality. A function symbol is
preferred, then an `STT_GNU_IFUNC` symbol, then an untyped one. An IFUNC name
is the resolver that starts at that address, not the implementation it
selects at run time, and a limitation says so. Symbol names, section names,
and augmentation strings are target-derived and untrusted. Indirect EH
pointers are not dereferenced. An FDE whose initial location is indirect names
a pointer slot, not its function, so it is not reported as a range; a
limitation counts those FDEs and coverage is `incomplete`. An indirect
`eh_frame_ptr` does not locate `.eh_frame`, and the header is `incomplete`.
A search table with an indirect encoding is not read: the header is
`incomplete`, the comparison is `unavailable`, and the table does not bound a
header-located `.eh_frame`.
An encoded pointer whose stored value is zero is a null pointer, as libgcc
reads it: no PC-relative, text, data, or function base is added and an
indirect pointer is not followed. A resolved address, and an FDE `end`,
wraps at the ELF address width (32 or 64 bits), as libgcc's pointer
arithmetic does. A zero LSDA pointer is reported as a null
`lsda`, even when the CIE declares an LSDA encoding. An FDE whose initial
location is zero marks a function the linker discarded. It is not reported
as a range, `examined_records` still counts it, and a limitation counts how
many were skipped. In a relocatable object an unrelocated initial location
is also zero, so those FDEs are skipped too.
`page` reports `offset`, `limit`, `returned`, `total`, and `next_offset`.
`next_offset` is null when the page reaches the end of the decoded list.
`coverage` counts every decoded range, every decoded CIE, how many ranges
match a symbol, and how many do not. `examined_records` counts CIE and FDE records visited, including
records that could not be decoded.
A missing `.eh_frame` section and a missing `PT_GNU_EH_FRAME` segment are
observed absence of those declarations. That absence is not proof the image
has no functions. Program headers come from the shared ELF section
directory, which rejects an image whose program-header table cannot be read,
so a missing `PT_GNU_EH_FRAME` is always observed absence. When section
headers are missing too, section names are unknown, so a missing header is
`unknown` rather than `absent`.
A `.eh_frame` located only through the `.eh_frame_hdr` `eh_frame_ptr` has no
section size, so it is read up to the end of its `PT_LOAD` segment. Other
data, such as `.gcc_except_table`, can follow it there. When the search table
was decoded, the scan stops after the last FDE the table names, as libgcc's
`PT_GNU_EH_FRAME` lookup only binary-searches that table. Later bytes are not
decoded and a limitation says so. Without a usable table, the scan stops at
the first record that fails to decode, as libgcc's linear FDE search does,
and `coverage.status` is `incomplete`. A CIE with an unrecognized
augmentation letter, or an FDE that uses one, also stops that scan. Reaching the end of the segment with
no terminator also keeps coverage `incomplete`, because the table's extent is
then unknown.
Addresses map to file offsets through `SHF_ALLOC` sections, then `PT_LOAD`
segments. A section without `SHF_ALLOC`, such as `.comment` or `.debug_*`,
is not loaded, so its `sh_addr` is never used for translation and it is never
reported as the section containing a range. A record that ends past the table, or non-zero bytes after
the terminator, keeps `coverage.status` at `incomplete`. Relocatable objects
stay incomplete because relocations are not applied.
## Limits and failures
The selected file is limited to 64 MiB by default (512 MiB maximum).
`max_ranges` defaults to 8192 and caps at 65536. That ceiling applies
separately to CIE records and to FDEs. Exceeding either budget fails without
partial success. One page returns at most 4096 ranges. Symbol lookup stops
after 65536 entries across `.symtab` and `.dynsym` combined. Reaching that
cap keeps `coverage.status` at `incomplete`. Only symbols at decoded FDE
starts are kept, and a name is decoded only for a returned range. The names
on one page share a fixed 16 MiB text allowance; a page that exceeds it fails
with a resource error whose recovery is a smaller `limit`, because
`max_ranges` and `max_file_bytes` do not raise it.
Malformed ELF headers fail without partial success. This workflow does not
execute the image, evaluate CFA programs, or follow indirect pointers.
Loaded 100 of 2713 files, more files were not shown because too many files have changed in this diff. Show more