Compare commits

..
3082 changed files with 22038 additions and 306878 deletions

No files matched your search

@@ -10,8 +10,8 @@ equivalent completed investigations. A useful outcome can also be a measured
decision to leave an implementation alone.
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
[tool design](../../../docs/development/tool-design.md), and
[testing guide](../../../docs/development/testing.md). This skill supplies an optimization
[tool design](../../../docs/tool-design.md), and
[testing guide](../../../docs/testing.md). This skill supplies an optimization
workflow, not additional permission to execute targets, publish, or merge.
Use the current contracts, provider prerequisites, and user authorization.
+1 -1
View File
@@ -6,7 +6,7 @@ description: Prepare or rewrite REA release changelogs and GitHub release notes
# REA changelog update
Use this for release-note preparation and contributor attribution. Read
[docs/development/releasing.md](../../../docs/development/releasing.md) from the repository root for
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
publication authority and the existing checkpoint checks. This skill prepares
notes; it does not authorize merging, tagging, or publishing.
+1 -1
View File
@@ -9,7 +9,7 @@ Establish which checks still catch real regressions and which costs can be
removed without losing that protection. The outcome may be a verified fix,
an evidence-backed recommendation, or an explicit decision to stop.
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/development/testing.md),
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
and pinned toolchain, rather than commands remembered from an earlier audit.
Preserve the requested scope: investigation alone does not authorize edits;
+6 -8
View File
@@ -5,7 +5,7 @@ description: Audit, investigate, design, or change REA investigation tools, CLI/
# REA tool design
Use the [tool-design guide](../../../docs/development/tool-design.md) for tool boundaries,
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
contract semantics, provider ownership, and discoverability. Inspect the current
contract and nearest existing tool relevant to the analyst question; source,
documentation, and live catalogs may describe different versions.
@@ -14,23 +14,21 @@ For provider-backed work, exercise the analyst's task through REA and compare
relevant native provider and independent artifact observations where available.
Check applicable partial results, recovery and session boundaries; report what
was not exercised. For failure guidance, follow [design recovery around the
failure](../../../docs/development/tool-design.md#design-recovery-around-the-failure).
Use [workflow verification](../../../docs/development/testing.md#provider-backed-workflow-verification)
failure](../../../docs/tool-design.md#design-recovery-around-the-failure).
Use [workflow verification](../../../docs/testing.md#provider-backed-workflow-verification)
to select evidence for the proposed contract instead of seeking provider API
parity.
For an audit, follow [audit existing tools](../../../docs/development/tool-design.md#audit-existing-tools):
For an audit, follow [audit existing tools](../../../docs/tool-design.md#audit-existing-tools):
scope representative tasks, exercise the live workflow, and distinguish
reproduced bugs, supported limitations, design opportunities and unverified
claims. Check relevant rendered MCP prompts, guided workflows and packaged skill
advice against the live contracts and exercise their suggested handoffs. Report
working alternatives and evidence limits as well as findings.
claims. Report working alternatives and evidence limits as well as findings.
For a design request, return the proposed boundary, input/result shape, Evidence
semantics, nearest alternative, and verification approach. Implement when
requested, preserving the user's scope and existing authorization.
For implementation or verification, consult [testing](../../../docs/development/testing.md)
For implementation or verification, consult [testing](../../../docs/testing.md)
for the affected consumer and provider lanes, and
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
provider/workflow guides only as the task requires.
@@ -1,8 +1,8 @@
---
name: reverse-engineer-anything
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
metadata:
version: "62"
version: "59"
---
# REA
@@ -97,12 +97,6 @@ explicit path or endpoint and do not need it.
- Bun `--compile` executable: `open_binary`, then `inspect_artifact`; the Bun
module graph is the application layer. Extract stored modules and analyze the
JavaScript. See [Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
- Deno `deno compile` executable: `open_binary`, then `inspect_artifact`; the
embedded virtual filesystem is the application layer. Extract stored files and
analyze compatible JavaScript. Environment-file values and CA certificate bytes
are retained in local Evidence with their counts and byte lengths. The npm
snapshot length is reported and not decoded, and the executable is not run. See
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
`open_bundle_resources`, then the selected resource tool. This does not
require `CFBundleExecutable` or a deep provider. Resource sessions have no
@@ -114,7 +108,7 @@ explicit path or endpoint and do not need it.
- Firmware image: `inspect_firmware_regions` when advertised. Use
`extract_firmware` when extraction is requested, with a caller-selected new
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/guides/firmware/firmware-analysis.md).
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
- ESP32-family app or bootloader image (`0xE9` magic): `inspect_artifact`
without a provider; see the ESP section of `references/native-and-artifacts.md`.
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
@@ -127,15 +121,6 @@ explicit path or endpoint and do not need it.
cleanup effects, and do not require a preliminary static session unless their
own contract says so.
- User-owned browser page already open: `list_browser_targets`.
- Passive traffic capture of a SPA: the default scope is the page's own origin,
and data APIs usually live on another origin. Discover API origins from
absolute URLs in loaded bundles or script text via JavaScript analysis tools
(another passive capture without those origins still scopes to the page
origin, and `completeness.excluded` counts lack URLs). Pass both the page
origin and those API origins in `allowed_origins` — the list replaces the
default scope — and read `completeness.excluded` before reporting that an
API is absent. For long captures with body shapes, use
`export_evidence_bundle` instead of expecting an inline result.
- Browser action the agent must perform (click, type, navigate):
`capture_browser_scenario` with declared actions. See
[runtime observation](references/runtime-observation.md) for connect/launch
@@ -143,13 +128,6 @@ explicit path or endpoint and do not need it.
- Agent environment (AgentEnv bundle) of a web application: read
[environments](references/environments.md) before recording; it records with
`capture_browser_scenario` and writes the bundle with `export_environment_bundle`.
- Task for a desktop application's exported environment, from a
`capture-desktop-application` folder: read
[desktop application tasks](references/environments.md#tasks-for-a-desktop-application-environment).
- Environment of a Linux desktop app with a task, from a request such as
"build me an Inkscape environment where the agent changes a poster's
headline": read [desktop builds](references/desktop-build.md). You drive the
app on a build computer from `start_build_computer`, then export and witness.
- Fiddler Everywhere traffic or a native `.saz` capture: read the
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
@@ -160,7 +138,7 @@ explicit path or endpoint and do not need it.
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
raise it. See
[historical captures](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md) for the exact upstream
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
or establish network loss. Prefer detail summary, then kind qlog trace/stream
@@ -175,7 +153,7 @@ explicit path or endpoint and do not need it.
display filters and decode-as rules. Requires Linux, util-linux prlimit and
TShark 4.x on an absolute `PATH` entry or via the optional absolute
`REA_TSHARK_COMMAND` override; see
[capture guide](https://github.com/morluto/rea/blob/main/docs/guides/data/packet-captures.md).
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
Packet limits count input packets before filtering. Output defaults to 50 matching
frames; use `match_offset` / `match_limit` for later pages, or explicit null
limit for complete matching output. Preserve the source digest across pages.
@@ -183,7 +161,7 @@ explicit path or endpoint and do not need it.
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
and capinfos, with Linux prlimit or Windows Job Objects; see
[packet summaries](https://github.com/morluto/rea/blob/main/docs/guides/data/pcap-analysis.md).
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
Select an exact DNS question for address pages; page exhaustion does not prove
network completeness. This reads recordings and does not acquire traffic.
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
@@ -191,7 +169,7 @@ explicit path or endpoint and do not need it.
relevant details before choosing settings; this does not scan hardware.
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/guides/signals/signal-recordings.md).
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
Output defaults to 50 literal lines; select annotation classes and page with
`line_offset` / `line_limit`, or explicit null for all remaining output.
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
@@ -200,7 +178,7 @@ explicit path or endpoint and do not need it.
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
Select JavaScript glue and candidate paths explicitly and preserve URL/path
ambiguity. No implicit fetch or target execution; WAT is not original source.
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/guides/wasm/wasm-artifacts.md).
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
Prefer its summary or selected signer/digest/lineage page; retain complete
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
@@ -219,17 +197,13 @@ explicit path or endpoint and do not need it.
Evidence stays retained under its original identity. Use the returned
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
Candidate coverage does not establish interface completeness. See the
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/guides/evm/evm-bytecode.md).
- Embedded Go compiler, module, build-setting metadata, or static `(ptr, len)`
string literals: `inspect_go_binary` with the explicit local file (pass
`include_string_literals: true` and optional `max_string_literals` for bounded
inferred static string-header recovery, and page retained rows with
`inspect_analysis_view` kind `go` collection `string_literals`). This built-in
target-free inspection requires no Go installation or disassembler. Preserve
raw module bytes and source ranges; absent metadata does not prove a binary is
not Go, static string recovery excludes compiler-inlined stack or immediate
strings, and unparsed text records remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/go-binaries.md).
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
with the explicit local file. This built-in target-free inspection requires no
Go installation or disassembler. Preserve raw module bytes and source ranges;
absent metadata does not prove a binary is not Go, and unparsed text records
remain explicit. See the
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
first, then decode selected payloads and query retained GPU Evidence.
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
@@ -282,14 +256,6 @@ explicit path or endpoint and do not need it.
proof the image never imported APIs through another table. Bound snapshots
and loaded IAT contents remain unobserved. See the
[PE import guide](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
- Windows Shell Link shortcuts: `inspect_windows_shortcut` with the explicit
local `.lnk` file. This built-in target-free inspection requires no Windows
SDK or COM. Preserve header flags and FILETIMEs, IDList items, LinkInfo
volume and network names, StringData arguments and icon location, ExtraData
signatures, and Tracker droid facts labelled derived. Unknown IDList items
and ExtraData signatures stay raw with SHA-256. The target path is not
resolved or executed. See the
[Windows shortcut guide](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
- PE TLS callbacks and load configuration: `inspect_pe_load_config` with the
explicit local PE32/PE32+ file. This built-in target-free inspection requires
no Windows SDK. Preserve callback VAs as strings with their RVA, file offset
@@ -298,38 +264,12 @@ explicit path or endpoint and do not need it.
linker declarations, not proof of enforcement. A missing directory is observed
absence of that declaration. See the
[PE load configuration guide](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
- ELF, PE, and Mach-O hardening: `inspect_binary_hardening` with the explicit
local file. This built-in target-free inspection requires no pwntools or other
host tool. One image per Mach-O slice. Each property is `enabled`, `disabled`,
`partial`, `unknown`, or `not_applicable`, with the header field or import
behind it. A missing stack-protector or fortify import is `unknown`, not
proof the protection was omitted. Authenticode and Mach-O code signatures
report presence, not validity. Hardened runtime stays unknown and names
`inspect_signature`. The image is not executed. See the
[binary hardening guide](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
file. This built-in target-free inspection requires no disassembler. Preserve
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
build-id or ABI-tag fields. A missing table is observed absence of those
declarations, not proof the image never carried notes. See the
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
- Code that runs before `main` or at exit in ELF or Mach-O:
`inspect_native_initializers` with the explicit local file (and
`architecture` for a universal Mach-O). This built-in inspection requires no
disassembler. Preserve slot and target addresses, the `resolution` of each
entry, and its limitations. An `unresolved` slot is not a zero pointer, and a
missing table is not proof that no constructor exists. See the
[initializer guide](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
- ELF unwind function ranges: `inspect_elf_unwind_ranges` with the explicit
local ELF32/ELF64 file. This built-in target-free inspection requires no
disassembler. Preserve FDE start, end, size, containing section, exact
symbol name, CIE index, LSDA, and personality. Page with `offset` and
`limit`. A missing `.eh_frame` is observed absence of that declaration,
not proof the image has no functions. `.debug_frame` is reported present
or absent and is not decoded. CFA instructions are counted and not
evaluated. See the
[ELF unwind ranges guide](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
- Apple `.ips` crash reports: `inspect_apple_ips` with the explicit local
file. This built-in target-free inspection requires no Xcode, atos, or
pwntools. Preserve first-line metadata, `bug_type` as a string, process and
@@ -348,16 +288,6 @@ explicit path or endpoint and do not need it.
zstd containers are unsupported; decompress into a separate file with an
external codec utility, then inspect the decompressed `ET_REL` image. See the
[kernel module guide](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
- Linux kernel version and build config (`CONFIG_*` options such as
`CONFIG_SECURITY_SELINUX`): `inspect_kernel_build_config` with the explicit
local `vmlinux`, raw `Image`, x86 `bzImage`, or `configs.ko`. This built-in
target-free inspection requires no Linux host or `extract-ikconfig`. Preserve
the banner version, config SHA-256, option count, and value kinds; use `name`
for one option. An `IKCFG_ST` string is a config only with a valid gzip member
and `IKCFG_ED`. An absent config may mean the kernel was built without
`CONFIG_IKCONFIG`. Non-gzip payloads are unsupported; decompress externally,
then inspect the result. See the
[kernel build config guide](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
identity: `inspect_python_pyc` with the explicit local file. This built-in
target-free inspection requires no Python installation. Preserve magic,
@@ -381,17 +311,17 @@ explicit path or endpoint and do not need it.
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
report truncation and have unspecified order. Use a quiescent snapshot rather
than assuming a live database/WAL pair was copied atomically. See the
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
- Explicit Linux ELF file for offline layout, symbols, relocations or static
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
caller-supplied pwntools without opening a disassembler database. Preserve its
raw locations and inference/coverage limits; see the
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
thread, raw note and source-bound register without opening a live target.
Historical PIDs do not select live processes. Request `include_debugger_context`
only when core-only mapping candidates help; see the
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md).
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
- PE resource type/name/language trees and static icon group references:
`inspect_pe_resources` with an explicit path. This portable operation requires
no active target or engine; opaque payloads retain file ranges and SHA-256.
@@ -407,9 +337,9 @@ The legacy attached profile binds the already-open GUI input by SHA-256 and
leaves its database open. The modern headless profile creates and closes an
owned private database without saving. Installation and registration reuse
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md).
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
For migration to the separate native IDA MCP plugin, follow the guide's
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/providers/ida-provider.md#migrate-a-legacy-attached-registration).
[legacy attached migration](https://github.com/morluto/rea/blob/main/docs/ida-provider.md#migrate-a-legacy-attached-registration).
Replace the upstream registration explicitly; modern mrexodia tools are not the
native profile. Native attachment preserves the database but does not provide
direct callers; check session availability and the guide's verification limits.
@@ -455,10 +385,6 @@ describe possible providers, while top-level session capabilities describe the
current route and `analysis_provider_binding` names the selected provider. Do
not treat those scopes as interchangeable.
For editable Ghidra analysis that must survive a later agent process, select
the [owned Program checkpoint workflow](references/native-and-artifacts.md#continue-editable-ghidra-analysis-across-processes)
before making metadata edits. It is separate from Evidence/Unknown persistence.
For a Ghidra startup timeout on a large native target, follow
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
before abandoning deep analysis or repeating the same cold import. The default
@@ -586,10 +512,6 @@ as complete while required questions remain open.
- Agent environments (AgentEnv bundles) of a web application, recorded from
a live instance or a HAR:
[references/environments.md](references/environments.md)
- Tasks for a desktop application's exported environment:
[references/environments.md](references/environments.md#tasks-for-a-desktop-application-environment)
- A desktop app environment from a request, built and witnessed on build
computers: [references/desktop-build.md](references/desktop-build.md)
## Finish the task
@@ -22,7 +22,7 @@ Preserve slot/relocation provenance and source coverage: matched means observed
candidate presence; partial scans do not establish uniqueness or absence.
This reads DEX/ELF bytes without loading the APK into the decompiler or executing
target code. Runtime registration and loading remain unknown. See the
[JNI guide](https://github.com/morluto/rea/blob/main/docs/guides/android/android-analysis.md#link-native-declarations-to-jni-candidates).
[JNI guide](https://github.com/morluto/rea/blob/main/docs/android-analysis.md#link-native-declarations-to-jni-candidates).
For an R8/ProGuard release build with its `mapping.txt`, pass the absolute
`mapping_path` to `inspect_android_class` or `inspect_android_method`. Prefer
@@ -11,7 +11,7 @@ main and an older release expose the same arguments. Tool-name prefixes belong
to the client: find the canonical REA operation in its actual tool listing and
server association before concluding that it is absent. A shared prefix alone
does not establish which server an approval wildcard covers. See
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/getting-started/client-configuration.md#client-tool-names-and-discovery).
[client tool names and discovery](https://github.com/morluto/rea/blob/main/docs/client-configuration.md#client-tool-names-and-discovery).
## Choose the remedy from the observed failure
@@ -1,784 +0,0 @@
# Desktop app environments from a request
Use this when someone asks for an environment of a Linux desktop app with a
task, such as "Build me an Inkscape environment where the agent changes a
poster's headline and saves it". You are the building agent: you drive the
app on a build computer through REA's MCP tools, write the task, export the
bundle and witness it in a fresh computer. REA runs no model. The tools are
MCP-only and need a Linux host with Docker, agentenv and REA's packed
runtime. When a start reports one missing, tell the person to run
`rea doctor --provider desktop_build` and apply its fixes, or
`rea setup --client <client> --desktop-build` and restart the client, and
stop.
For the whole build:
- Drive the app only with `act_on_build_computer`. Never script it, run
commands in the computer or write its files another way. The one command
you write for the computer is a task's reader, which REA runs to check a
file; it never makes the seed or the edit.
- Ask the upfront questions once, then work on your own. Pause only when
blocked.
- The seed's content is synthetic. Never put real personal data in it.
- Close every computer you start, on success and on failure.
- Note the time you start; the report gives the build time.
## Ask once, upfront
In one message, ask only what the request leaves open, and keep the list for
the report. Skip each question the request answers; when it answers them all,
start without asking:
1. The app and the task, if either is unclear: what changes, in which
document.
2. Whether they have a starting file of their own; otherwise you make one.
3. Whether they want solver runs after the witness: how many (the budget)
and with which model.
4. If so, whether they want to review the task before those runs.
When you cannot ask, as in a headless `claude -p` run, do not stop: take
these defaults for whatever the request leaves open, say which you took in
your first message and in the report, and go on. Install from the Ubuntu
archive, make a synthetic seed, skip the review, and make 3 solver runs
within $4 in all, with the model the request names or else the runner's
default.
## When you are blocked
Stop, say what you need and wait only for:
- a licence the app or its installer asks someone to accept: never accept it
on your own; name it and where to read it, and go on only once the person
says they accept it. Then record their words with `record_build_consent`,
kind `licence`, and accept it on their behalf;
- credentials: ask the person to write each secret, and nothing else, in a
file named after a handle you choose, such as `keepassxc-master`, in
`~/.config/rea/build-secrets/` on this host (under `$XDG_CONFIG_HOME` when
it is set), readable only by them (`chmod 600`), and wait until they say it
is there; then type it with `type_secret` (see Drive the app). Never ask
for a password or key in chat, and never read, list or print that folder;
- a signed-in state in the seed: by default the export withholds it (see
Sign-in and licence state). Seed it only when the person says the bundle
may hold it: record their words with `record_build_consent`, kind
`seed_sign_in`, and export with `seed_sign_in: true`;
- an install this flow cannot do, below.
Decide everything else yourself and note it in the report.
## Sign-in and licence state
What an app writes when someone signs in or activates a licence is withheld
from the seed: the bundle's receiver supplies their own. Mark
`pre-sign-in` with `mark_build_checkpoint` just before the sign-in or
activation and `post-sign-in` just after, both before `seed`: every file
under the owned paths that changed between them is withheld, and the draft
leaves it out. A file the app writes at first launch, such as a trial's
licence file, changes before you can mark anything: name it in
`export_build_environment`'s `withhold`. The export lists each withheld file
in `withheld_files` with the environment variable a deploy reads it from and
the handle of the secret REA keeps its bytes in; a check may not name one.
The export refuses a seed file holding a withheld file's exact bytes, such as
a `.bak` copy, but a backup that differs from it is not caught: trash backups
before `seed`.
Explicitly withhold any other files the person identifies as sensitive;
REA does not infer secrecy from a value's shape or a file name.
## Choose the install
The computer is Ubuntu 24.04 (noble) on this host's architecture only. Use
the first source that has the app:
| Source | `install` |
| ---------------------------- | ------------------------------------------------------------------------------------------- |
| Ubuntu archive | `{"kind": "archive", "package": "inkscape"}` |
| PPA or vendor apt repository | `{"kind": "apt_repository", "package", "uri", "suite", "components", "key_file"}` |
| `.deb` by URL | `{"kind": "package_file", "url", "sha256", "architecture", "dependencies"}` |
| AppImage by URL | `{"kind": "appimage", "url", "sha256", "architecture", "extra_packages"}` |
| Tarball by URL | `{"kind": "tarball", "url", "sha256", "architecture", "program", "name", "extra_packages"}` |
- A PPA's `uri` is `https://ppa.launchpadcontent.net/OWNER/NAME/ubuntu`,
`suite` `noble`, `components` `["main"]`. `key_file` is the repository's
signing key saved on this host; for a PPA, fetch it from
keyserver.ubuntu.com by the fingerprint its Launchpad page shows.
- When the app exits at launch for want of a runtime piece its package does
not install, start a new build computer with that package in
`extra_packages`, up to 16 archive names; the export installs them too.
MyPaint needs
`{"kind": "archive", "package": "mypaint", "extra_packages": ["librsvg2-common"]}`
for its SVG icons, and Freeplane `["openjdk-17-jre"]`, since its launcher
refuses noble's Java 21. Name one up front, too, when a package the app
only recommends holds its content, such as QElectroTech's element library
`qelectrotech-data`, without which no component can be placed.
- Take the `.deb` path only when the app is not in the archive at all, not
to add a package. Ask the requester for its URL and SHA-256; do not fetch
a vendor's download page yourself. List its `Depends` as `dependencies`,
with any runtime piece it lacks: it installs with the network off.
- Flatpak, AppImage and snaps are not supported yet: say so and stop. An
archive package that only installs a snap, such as noble's `firefox`,
counts as a snap. A package with no build for this host's architecture
(some PPAs are amd64 only) cannot be installed either.
- `desktop_entry` is the package's `.desktop` file name under
`/usr/share/applications` without `.desktop`, such as
`org.inkscape.Inkscape`. Add `window_class` when you know it; Inkscape's
is `Inkscape`. AbiWord's entry is `abiword` and its class `Abiword`.
- To find the entry, read the package's file list: its page on
packages.ubuntu.com, or `apt-get download PKG && dpkg -c PKG_*.deb` where
apt runs. The entry may ship in a companion `-data` or `-common` package,
as Planner's in `planner-data`: read that package's list too. The entry's `StartupWMClass` may not be the window's real
class, so do not take `window_class` from it: MuseScore 3's says
`mscore3`, its window's is `MuseScore3`. The export uses the class the
capture saw.
## Drive the app
- Pass `computer_id` and the action's fields at the top level: click
`{x, y, count}`, scroll `{x, y, direction, amount}`, a screenshot region
`[x0, y0, x1, y1]`. The rest are in
[the build-computer guide](https://github.com/morluto/rea/blob/main/docs/desktop-build-computer.md).
- Take a `screenshot` before pointing: points are pixels of the latest
frame. If a response is too large, ask for a `region`, or `jpeg` with a
lower `quality`. A screenshot with `region` does not capture the screen
again; it crops the latest frame, so take a full screenshot, or a `wait`
with `screenshot: true`, first. The crop then becomes the latest frame:
take a full screenshot before pointing again.
- A drawn caret does not prove keyboard focus. After using a toolbar
control, click into the document, then type a few characters and look
before a long text.
- Confirm a dialog with `key` Return, not a trailing `\n` in `type`.
- An action's own screenshot is taken at once. After opening a menu, a
dialog or a window, look with
`{"action": "wait", "ms": 500, "screenshot": true}`.
- On a computer started from a request the app is not open at first, and
`start_build_capture` opens it: never open it yourself before that. Do not
close it while capturing; the capture ends when the app exits. When the
app is closed at any other time, open it from Applications → Other → the
environment's name.
- Type a secret with `{"action": "type_secret", "secret": "<handle>"}`,
only into a field that masks what is typed: check in a screenshot that it
is a password field, which shows dots. A screenshot shows any field that
shows its text, such as many licence-key boxes, so never type a secret into
one: say the app needs a secret the build cannot type unseen, and stop.
`type_secret` takes no `screenshot`; look with a `wait` after it. A refusal
names the secret's file and what is wrong with it: tell the person, never
the value.
- Prefer shortcuts and typing to long pointer paths: `key` takes `keys`, an
array holding one chord's keys, such as
`{"action": "key", "keys": ["Control", "s"]}`, and `type` turns `\n` into
Enter.
- Before `type`, check in a screenshot that a text cursor or a focused
field is there. In apps with single-key shortcuts, such as Inkscape, text
typed without one runs as commands.
- Save As differs by app. Check the dialog's current folder first: most
open in the home, so click Documents in the sidebar, but Scribus opens in
`~/Documents` and its sidebar has no Documents entry. Type only the file
name. A dialog taller than the 1280×800 screen hides its buttons:
maximize it from its title bar, or confirm it with `key` Return, as when
HomeBank's Save button is off-screen. When a GTK Save dialog ignores
clicks or runs off the screen, type `~/Documents/<name>` in its Name field
and press Return.
- Answer the app's save questions, such as keeping the format, then check
the saved path in the title or status bar, or the dialog's folder when
the app shows only the name, and that it shows no unsaved mark, before
you mark a checkpoint or check. Where the title never shows one, as in
HomeBank, a greyed Save or Revert, or the file reopened from recent
files, confirms the save.
- Qt spin boxes change their value on scroll: move the pointer off them
before you scroll.
- gLabels edits an object's text in Object properties → Text, not on the
canvas.
- In Grisbi's transaction form Return moves to the next field; only Return
in the last field, Notes, records the transaction.
- ghostwriter autosaves a new document as `~/Documents/untitled-1.md`: name
it with File → Rename, not Save As; Ctrl+Shift+S does nothing, and Enter
after a checklist item starts another `- [ ]` item.
- KMyMoney's first launch opens the New File Setup wizard; Finish asks for a
storage type (XML), then a Save As where typing `Documents/<name>.kmy`
works. Its date field is a sectioned Qt editor: double-click the month and
type each part, since Ctrl+A garbles it. A new transaction's payee comes
prefilled and selected. Amounts are saved as reduced fractions, such as
`-1749/20`, so compare them exactly. Untick Help → Tip of the Day → "Show
tips on startup" before `seed`, or the tip appears at every boot.
- QElectroTech: an element dropped from Collections follows the pointer until
a click on the folio places it. Its title shows only the app, so it needs
`state_paths`; its settings are in `~/.qet`.
- Rosegarden: after a dialog its Transport window can take keyboard focus,
so save with File → Save.
- Pencil2D: the Tools dock is too narrow for its buttons, so use shortcuts
(B for the brush); double-click a layer's name to rename it.
- Umbrello: the title shows no unsaved mark, so confirm a save with a probe
checkpoint. An Untitled document autosaves `~/autosave.xmi`: delete it
before `seed`. Add attributes from the tree: right-click the class → New →
Attribute.
- Hydrogen: the computer has no audio device, so Hydrogen falls back to a
null driver and its BPM field snaps back to 120 within about 100 ms.
Before `seed`, set Preferences → Audio System → ALSA with device `null`
and click "Apply and restart output", which `~/.hydrogen/hydrogen.conf`
keeps for the seed; also set autosaves per hour to 0, and dismiss the
development-version dialog with "Don't show again". `pulseaudio` as an
extra package does not help.
- Poedit cannot add entries by hand. Use File → New, pick the language and
save to `Documents` first (Sources Paths stays locked until then), then
Properties → Sources Paths → + → Add Files with a source such as
`/usr/lib/python3.12/argparse.py`. It writes `fr.mo` beside the `.po`, and
the language box's autocomplete popup takes the first OK click.
- CherryTree: its first-run autosave "Choose Storage Type" dialog swallows
keys, so save once early as `.ctd` and untick autosave.
- VSCodium (its tarball, with `extra_packages: ["libnss3",
"libasound2t64"]`) runs on outside REA's reach, so `stop_build_capture`
fails to clean it up: after `end`, quit it with Ctrl+Q, then stop. Enter
accepts its word suggestions, so press Escape after typing a line.
- Glade: a new widget's id is empty until you set it, and a second save
writes a `.glade~` backup, which shows in the draft as an unrelated
change. A backup present at `seed` is seeded with the document, so trash
it before `seed`. The draft names a button's `<child>` by the id
its widget carries, `[name=child][>object@id=submit_button]`, which moving
the button keeps; give the widget an id before `seed` so it can.
- VYM: the title never names the map, so pass `Documents/<map>.vym` in
`state_paths`. Its settings live in `.config/InSilmaril` and are written
only at a clean exit. It writes `<map>.vym.lock` while a map is open, so
close the map before `seed`. Press A only after clicking a branch on the
canvas: keys typed in the tree editor run other shortcuts.
- KmPlot: Create → Cartesian Plot, then edit the formula in the Function
tab and press Return. It stores minus as U+2212 (`−`). Its file:// title
still names the file.
- RedNotebook keeps each month in `~/.rednotebook/data/YYYY-MM.txt`, YAML
read as text, folded at about column 80: pick check phrases the fold keeps
on one line.
- Qt5 apps saving with QDom, such as KmPlot, write attributes in a random
order in each process, so check their XML by element, never by whole line.
- Gnumeric's desktop entry is `org.gnumeric.gnumeric`. Its Save As dialog
runs off the 1280×800 screen: type `~/Documents/<name>` in Name and press
Return. Rename a sheet with right-click on its tab, then Rename….
- Parley's title shows the collection's title, so name the file in
`state_paths`. File → New opens the new collection's Properties dialog:
confirm it with OK, since Cancel keeps the old collection and its autosave
lock open. A new collection starts with 15 blank entries: delete them.
Pick languages from the dropdown; a typed name keeps the previous locale.
While a document is open KDE keeps an autosave and its `.lock` in
`~/.local/share/stalefiles/parley/`, and there is no File → Close: before
`seed`, use File → New, then reopen the collection from Open Recent.
- Sigil: pass `extra_packages: ["libqt5svg5"]`, or its SVG toolbar icons are
blank. Its title names the book, so it needs no `state_paths`. It unpacks
the open book into `.local/share/sigil-ebook/sigil/workspace/Sigil-XXXXXX/`
and deletes it on exit, so never name the whole `sigil-ebook` folder: that
seeds the stale copy.
- What the app saved under the home before `seed`, such as a document in
`~/Documents` or its settings, is part of the seed. A file you move to the
Trash is left out, and the export lists it in `limitations`. Trash a stray
file rather than starting over. JabRef 3.8 writes `<file>.bak` and
`<file>.sav` beside a `.bib` it saves; in the owned folder they are seeded
and show in the draft as noise, so trash them before `seed` if they should
not ship.
- Dismiss first-run dialogs, such as a welcome screen, before `seed`. A
choice the app saves only later, such as Inkscape's "show this every
time", may not reach the seed, so expect the dialog in the witness too.
- Some apps write their settings late: Freeplane writes
`~/.config/freeplane/1.7.x/auto.properties` only once Preferences is
confirmed, and a seed without it reopens its documentation map. Before
`seed`, open the app's preferences and confirm them, then read a probe
checkpoint's `changed_files` to see the settings file written.
- Before `seed`, click into the document to clear any selection the app
would save: Gnote saves its cursor and selection with the note.
- An app whose state folder is named after it, such as
`~/.local/share/gnote`, or after it and its version, such as TreeLine's
`~/.treeline-3.1`, is found without `state_paths`.
- TreeLine: File → New opens a template window. Book List's CATEGORY makes
AUTHOR children by default; change it under Data → Configure Data Types →
Default Child Type. A new child opens an inline rename showing "", so
press Escape and fill its fields in the Data Edit tab. Ctrl+A adds a child
node even while a field has focus. Clicking a node already selected starts
an inline rename, so Ctrl+X then cuts its text: cut a node from its
context menu. It saves nodes sorted by their
generated uid1, so their order changes from run to run. The draft names a
new node by its format and a field of its `data`,
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and checks its other
fields through that, so give each new node a title no other node has.
- After an app crashes, capture again on a fresh computer, so the seed
carries no crash-recovery state, such as MyPaint's "Recover interrupted
work?".
- Give what the task changes a stable name where the app has one, such as an
id in Inkscape's Object Properties, so a check can select it.
- Expect apps to split text into runs: Writer's revision ids split an
edited paragraph into spans and may give it an automatic style such as
`P1`, and AbiWord wraps each insertion in `<c>`. Check such text whole
with `text_equals` (see Write the task), and find the paragraph by
something the edit keeps, such as AbiWord's `xid`.
- Freeplane's first launch opens a documentation map: close it with Ctrl+W.
Its Ctrl+N asks for a template.
- MyPaint 2.0.1 crashes on brush strokes on the build computer: build its
seed with flood fill and layer operations.
- GnuCash's Ctrl+S on a new book opens a Save dialog with no format choice:
use File > Save As to pick XML. Close the book (File > New File) before
`seed`, so its `.LCK` lock is not seeded.
## Build the seed and the edit
1. `start_build_computer` with
`{"source": {"kind": "request", "desktop_entry": ..., "install": ...}}`.
The first start of an app builds its image, and starts at the same time
on this host wait for each other; either can take minutes. A failed
start returns the build's error: fix the request, or stop when the
install cannot work, and close any `computer_id` the error names.
2. `start_build_capture`. It returns once the app's window is open. A
`package_lists_error` may stop the capture finding the package's
sources; the export says when that matters. When the app exits before
its window opens, the error gives its exit code and the end of its
stderr, which may name what it lacks, such as an icon loader or a Java
runtime: close the computer and start a new one with that package in
`extra_packages` (see Choose the install), or stop as for an install
this flow cannot do.
- REA counts a file as the app's document only when the app's window
title shows its file name or `~/` path. When the title shows something
else, such as MuseScore's score title, pass the path you will save the
document at in `state_paths`, relative to the home:
`{"computer_id": ..., "state_paths": ["Documents/MuseScore3/Scores/Morning_Song.mscz"]}`.
It need not exist yet. Add the app's settings folder when its name
differs from the app's, such as `.config/MuseScore`, so the seed keeps
the dismissed first-run dialogs. Do not pass the document as a launch
argument to make REA see it: `state_paths` is the way. gLabels' title
leaves out the file's extension and Grisbi's shows the entity's name,
so both need it.
- An app whose document is a folder of pages, such as a Zim notebook
(`Notebooks/Notes`, a `.txt` file per page), never shows a file in its
title: pass the folder in `state_paths`. Every page in it is then a
document, so the bundle opens none at launch and the app starts where
it keeps its place; check the page the task changes.
3. Make the seed through the app and save it from the app under
`~/Documents`: build the document in the app, or write its source where
the app edits its format as text, then save it again from the app. Make
it look real but synthetic: the thing the task changes, plus a few things
the change must leave alone. Keep it to one document: the bundle's app
opens the seed's document at launch only when there is exactly one, and
otherwise starts on its own screen. For a starting file of the requester's,
read it here and rebuild it in the app with synthetic data in place of
personal data; REA cannot copy a file into the computer.
- An app that keeps its own data and saves after every change, such as
Osmo's contacts in `~/.local/share/osmo/contacts_records.xml`, has no
document: that state is the seed. Build it in the app and mark `seed`
once the app has written. To see whether it has, mark a throwaway
checkpoint, such as `probe-1`, and read its `changed_files`: up to ten
files that changed since the checkpoint before, those a window title
named first, then those under `state_paths` and those the app held
open, caches last, with `more_changed_files` counting the rest.
- The app's own compressed format works when it is gzip, such as
Gnumeric's `.gnumeric`, Dia's `.dia` or Inkscape's `.svgz`: REA reads
the document inside, and checks point into it as into the same
document uncompressed. Compression may be off by default: Dia's
"Compress diagram files" is unticked, which REA reads as plain XML. A format REA reads as binary, such as one
compressed with xz, has no view: save in a format the app offers that
REA reads, then reopen the file once and check that a plain Ctrl+S
keeps that format. When the task is about the app's own binary
format, such as GIMP's `.xcf` or Blender's `.blend`, keep it and check
it through a reader (see Read a binary file with a reader).
4. Only once the seed is saved, `mark_build_checkpoint` with name `seed`.
5. Make the change the task asks for, as a user would, and save it.
6. Only once the change is saved, `mark_build_checkpoint` with name `end`.
7. `stop_build_capture`. Read `draft.files` (each file that changed, with
its net change and its `values`), `draft.task.checks` (suggested
checks), `draft.guards`, `draft.notes` and `limitations`. When `draft`
is null, read `draft_refusal` first: a missing `seed` or `end`
checkpoint means capturing again on a new build computer, while a failed
draft command names its own cause to fix. If a declared secret was used,
automatic drafting is withheld: write the task manually and export it.
When `draft.files` is empty,
`draft.notes` says why:
- `no_change`: nothing was saved between `seed` and `end`. Capture again
on a new build computer, and save before marking `end`. When the app
keeps the task's state only on screen, such as a calculator's result,
judge the screen instead (see Judge the screen).
- `unowned_changes` with `no_document`: the app saved your document
where REA did not count it as the app's. Its `files` list what changed,
with their roles; the document is among them. Close the computer and
capture again on a new one with that path in `state_paths`; the export
refuses a check on a file outside the owned paths.
- `uncheckable_changes`: the app saved the change in dconf, as gedit does
its preferences, and no check reads dconf. Saving again does not help:
choose a change the app saves in a file, and capture again on a new
build computer, or judge the screen (see Judge the screen).
- `unreadable_changes`: a dconf database changed but REA could not read
it; `error` says why. This is not evidence the app saved there: report
the error rather than changing the task.
A note with a `file` says why that file's suggestions are thin:
- `by_position`: only position names what changed; its `elements` lack
a naming attribute. Name a record that keeps its fields as child
elements by one of them, as Osmo's contact by
`[name=contact][>first_name=Delphine]`, a JSON record by a field
inside it, `[>data/Title=…]` (`~1` for the `/` inside a pointer), or
an element by an id its child carries, `[>object@id=…]`; count records
with `member_count`, `matching` `[name=contact]`; otherwise check an
ancestor's text, or give the element an id in the app and capture again.
- `view_state_only`: every change is view state, a renumbered id or a
time the app stamps, and its `pointers` are the checks kept. The edit
was likely not saved: capture again and save it before `end`.
- `long_text`: the element's text is too long to check whole; check a
phrase in it with `text_contains`.
- `cut_view` or `unreadable_view`: REA's view holds only part of the
file, or none a check can read, such as a binary format (step 3). For
a binary file, read it with a reader (see Read a binary file with a
reader).
## Write the task
Write `{"name", "prompt", "checks"}` from the draft and what you did.
- `name`: lowercase letters, digits and hyphens, not `smoke`.
- `prompt`: what the requester wants, in their words and the app's terms,
starting with the app's name, naming the document as a user would and
ending with saving it: "In Inkscape, change the poster's headline in
Documents/poster.svg to Summer Bake Sale and save it." Name the app: the
solving agent's own shell cannot see the computer's files, and the app's
name sends it to the screen. Leave saving out for an app that saves on its own, after every change
or a few seconds after the last one, such as Osmo or Gnote: grading waits
until the app's files have held for 3 s, so the solving agent need not
wait for the save. A save more than 3 s after the agent stops is still
missed; name such an app in the report. Never say how the file stores it,
and never give the checks.
- Checks are file-based: content checks that prove the edit, never only that
a file exists or changed. Start from the suggestions; keep those that say
what the user did and drop what follows from how the app saves.
- Compare exact values, so a typo or a wrong number fails.
- Read each file's `values` before writing checks: every value the change
added, changed and removed, with `before` and `after` (an uncut scalar, or
a preview cut at 600 characters) at the pointer a check names it by. Copy
the pointer. Copy `after` into `value` only when it is an uncut scalar: a
string without a trailing `…`, a number, or a boolean. Compact XML and a
JSON array or object are preview strings, so a `value_equals` check needs
the structured value from the file's view, and element text uses
`text_equals` or `text_contains`.
Only XML files, archives, JSON and key files have values: a text file's
or SQLite database's `change` already lists its lines and rows.
`refers_to` says what a number or id stands for, such as `category="44"`
naming `<cat key="44" name="Groceries"/>`, or a new KMyMoney
transaction's splits naming their account and payee; use it for the
prompt's words. `alternatives` gives a changed value's pointer by the
element's other names, such as a label beside an id, to pick another key
without another export.
Skip entries whose `note` is `by_position` (no check can use that
pointer) or noise (`view_state`, `renumbered`, `volatile`,
`timestamp`, `upkeep`, `layout`). `values_cut` means a limit left some out.
- Select elements by what they hold, never by position: SVG text by id,
`/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0`,
or an ODF paragraph by style,
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Title]/children/0`.
A selector must match exactly one element, so give the target a unique id
or style in the seed. The draft's pointers already do this: a name-like
attribute such as Scribus's `ANNAME` before a coordinate, and two
attributes together where one is not enough, as a spreadsheet cell's
`[@Col=3][@Row=2]`. Only an element's text is read by position, as
`children/0`.
- When an app may split an element's text into runs, check it whole:
`text_equals` (or `text_contains` for a value inside a sentence) on the
element compares all its text, however it is split. Add
`normalize_whitespace: true` when the app may reflow spaces. Example, an
AbiWord paragraph by its stable id:
`{"kind": "text_contains", "file": "Documents/memo.abw", "pointer": "/root/children/[name=section]/children/[name=p][@xid=6]", "value": "at 3:30 PM"}`.
Point a text check at the paragraph or another element the edit keeps,
never at a run such as AbiWord's `[name=c]`: a reopened document splits
the text into several runs. The draft lists only what changed; an
unchanged element is in `draft.guards` when it is a like sibling of a
changed one, and otherwise use the app's usual numbering, such as another
paragraph's `xid`, and let the export confirm it.
- In an ODS spreadsheet no attribute names a row: the draft names it by its
label cell, `[name=table:table-row][>table:table-cell=February]`. Where no
cell names it alone, name the sheet in the seed, and check its whole
text: `text_equals` on
`/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:spreadsheet]/children/[name=table:table][@table:name=Q1 Sales]`,
or a stretch of it with `text_contains`. Cell texts join with nothing
between, so a row reads `February23100`.
- For a line added to free text, such as an item on a note's list, the
draft compares the whole text with `text_equals`. This rejects longer or
negated items, but can reject a correct edit with different ordering or
spacing. Report that limitation when the task permits such variations.
- For a note placed by its own `x` and `y`, such as Xournal++'s `<text>`
on a layer, the draft checks its text on the layer with `text_contains`,
which passes wherever the user puts it, and guards each note it kept.
- Some apps keep text in an attribute, one element per run, such as
Scribus's `<ITEXT CH=...>`: add `"text_attribute": "CH"` to `text_equals`
or `text_contains` on the frame,
`/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]`,
to read its runs' values joined.
- When the draft shows no values or no checks, a scratch export can expose
`envs/<env>/seed/` only after the task has at least one valid check: it must
pass on the captured end state and fail on the seed. Before the first
successful export, use `act_on_build_computer` to open the saved file in an
available GUI text editor or viewer and inspect its actual representation
without changing it. Use the visible saved content and the task's edit to
write a check; an export refusal can also name the end state's value, such
as `is "44", not "1"`. If the file cannot be inspected or expressed by a
supported check, report the limitation instead of guessing a check.
- Where easy, add one check that the rest is untouched, such as another
text's value the edit must leave alone. `draft.guards` offers them at
their seed values, which the end holds too: a field on the nearest like
siblings, one the user sets, such as Planner's `work`, before a default
they all share; the text an addition kept; a text file's lines beside its
new one. Copy one or two into `checks`; they pass on the seed, which is
fine, since the export needs only one check to fail there.
- The draft suggests a decimal the app writes as text, such as `146.83`,
as a number with `tolerance: 0.001`; keep the tolerance for any number
you check that way.
- When the draft shows an XML document's changes as `lines`, REA read it
as text, for example because its DOCTYPE declares entities. Selectors are
impossible there, and `line_count` with `contains` only approximates a
value: `3:30` also matches `13:30`. Bound it with neighbouring text and
name the check as approximate in the report's limitations.
- Some apps save a document reloaded from disk differently from one typed
in the same session: AbiWord wraps each new insertion in its own markup.
Your build's end state hides this, which is why whole-text checks matter;
expect the witness to show the real format.
The kinds and pointer rules are in
[the desktop tasks section](environments.md#pick-the-checks).
### Read a binary file with a reader
When the draft notes `unreadable_view` on a binary document, add a reader to
the task: the app's own headless command, which REA runs in the computer on a
private copy of the file, and whose output a check reads instead of the file.
Write it in the app's scripting mode, print each fact on a line starting
with `REA ` (with the space), and name the copy `{file}`:
- GIMP `.xcf`, layer names, as `lines`:
```json
{
"id": "layers",
"file": "Documents/poster.xcf",
"argv": ["gimp-console-2.10", "-i", "-d", "-f", "-b", "-"],
"stdin": "(let ((image (car (gimp-file-load RUN-NONINTERACTIVE \"{file}\" \"{file}\")))) (for-each (lambda (layer) (display \"\\nREA layer \") (display (car (gimp-item-get-name layer)))) (vector->list (cadr (gimp-image-get-layers image)))) (newline))\n(gimp-quit 0)\n",
"output": "lines"
}
```
checked with `{"kind": "lines_present", "file": "Documents/poster.xcf", "reader": "layers", "lines": ["layer Summer Sale"]}`.
Script-Fu's prompt `ts> ` starts a line, so print a newline before each
marked line, as `\nREA` does. A text layer's text is
`(car (gimp-text-layer-get-text layer))`.
- Blender `.blend`, objects with their type and location, as `json`:
```json
{
"id": "objects",
"file": "Documents/scene.blend",
"argv": [
"blender",
"-b",
"--factory-startup",
"{file}",
"--python-expr",
"import bpy, json; print('REA ' + json.dumps({o.name: {'type': o.type, 'location': [round(v, 4) for v in o.location]} for o in bpy.data.objects}, sort_keys=True))"
],
"output": "json"
}
```
checked with `{"kind": "value_equals", "file": "Documents/scene.blend", "reader": "objects", "pointer": "/Crate/location/0", "value": 2, "tolerance": 0.001}`.
Print every value a check compares as a number or string a check can
name, rounded where the app stores floats.
Blender 4 shows its Quick Setup splash at every launch until its
preferences are saved: with `.config/blender` in `state_paths`, confirm
Quick Setup with Continue, save the preferences (Edit → Preferences, then
Save Preferences in the menu at its bottom left), and read a probe
checkpoint's `changed_files` for `.config/blender/4.0/config/userpref.blend`
before `seed`, so solves start on the scene. Blender often takes the first click on a
menu or a dialog's button only to focus it: click again.
- A format LibreOffice opens but REA cannot read, such as a legacy `.xls`,
converted with `--convert-to`:
```json
{
"id": "cells",
"file": "Documents/budget.xls",
"argv": [
"sh",
"-c",
"soffice --headless --convert-to csv --outdir out \"$1\" >/dev/null && sed 's/^/REA /' out/*.csv",
"sh",
"{file}"
],
"output": "lines",
"timeout_s": 120
}
```
Pass `{file}` to `sh -c` as an argument, as here, rather than inside the
script.
The reader runs without the display, in a fresh home with the app's default
settings, so use the app's headless or batch mode; set `display: true` only
for an app that has none. It must exit 0 within `timeout_s` (60 s by
default, at most 300) and print at most 1 MiB. Pick facts the change sets and
the near miss gets wrong, such as a name or a number, not the app's whole
output.
Give the task the reader in `readers`, and each check through it `reader`
and the reader's `file`. `json` readers serve `value_equals`, `value_absent`
and `member_count`; `lines` readers `lines_present`, `lines_absent` and
`line_count`. The export runs the reader in the build computer on the seed
and the end: when a check through it fails on the end, the refusal shows
what the reader printed, and a reader that fails shows its error and the end
of its stderr. Fix the reader or the check from that and export again.
### Judge the screen
Only when no file shows the change, as when the draft notes `no_change`
because the app keeps its state on screen, write the task's checks as
screen judges: `{"id": "shown", "kind": "screen_judge", "question": "Does the
calculator's display show 8.568?"}`. A model, the judge, answers the question
on a screenshot three times, and the check passes when most answers are yes.
- REA keeps a screenshot at each checkpoint. Mark `seed` with the screen
showing the state before the task, where the answer is no, and `end` with
the change showing, where it is yes: the bundle ships both as the judge's
examples.
- Ask about what the change makes visible, with exact values, so the near
miss is a no: a result, a label, a selected item. Avoid what a window
moved or a cursor blinking can change.
- The prompt asks to leave the result showing, since the judge sees only
the screen when the agent stops.
- A task with screen judges holds only screen judges and no readers. The
export refuses one when a file shows the change, and checks none of its
questions: the witness does, running the bundle's own judge with the
runner's model endpoint (`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`[model]` in `.agentenv/config.toml`) on a fresh screenshot at each stage.
Without an endpoint the witness's check fails: say what to set and stop.
- The judge's model is `[model.roles] judge`, else `[model] default`, else
`anthropic/claude-sonnet-5-5`. `agent-env run --judge-model` does not
reach it. Each grade costs about $0.02.
## Export
`export_build_environment` with `task` and `directory`, the absolute path of
a new or empty folder here, such as `<home>/rea-builds/poster/bundle`. The
export seeds what was saved at `seed`, replaces the computer's host name,
`user@host` and home path (a bare user name, such as AbiWord's author
`user`, stays), and checks the task against `seed` and `end`. It refuses a task
unless at least one check fails on the seed and every check passes at the
end, and says why, such as a pointer that names nothing at the end. A
check that the rest is untouched passes on the seed too, which is fine. On
a refusal, fix the task and export again, into another folder if that one
is no longer empty. A refusal naming a secret's handle means a file of the
bundle holds that secret, most often an app's sign-in or licence state in
the seed: do not work around it, but report it and stop. Keep
`seed_files`, `scrubbed_files`, `unscrubbed_files` and `limitations` for the
report.
The limitations name the folders the app wrote that the bundle neither owns
nor seeds, such as `~/.config/MuseScore`, so the app starts without them:
expect its first-run wizard, a tour or a "restore session?" question at every
boot. If the witness shows one, capture again on a new build computer with
those folders in `state_paths`, and dismiss the dialogs before `seed`. Lock
files the app holds while it runs, such as LibreOffice's `.lock` and
`.~lock.<name>#`, GnuCash's `<book>.LCK` or Grisbi's `.<file>.lock`, are
left out of the seed and listed after "Left out of the seed". An app can
still start fine yet refuse every save when a lock was seeded, so close the
document before `seed` where the app lets you (apps other than LibreOffice,
such as Rosegarden, also write `.~lock.<name>#` and autosaves beside a
document, and closing it covers them too), and treat "file is locked"
on a save in the witness as a seeded lock: delete it from the bundle's seed
and witness again in a new computer. In LibreOffice, own the profile
(`state_paths: [".config/libreoffice"]`) for a clean boot, and close the
document before `seed`, then reopen it from Recent Documents to make the
change: the export also leaves out the recovery list LibreOffice writes while
a document is open, which would start the bundle into document recovery. A
limitation naming `~/.config/dconf/user` as unread means the app's dconf
settings, such as a first-run tip you turned off, are not seeded: capture
again on a new build computer. The limitations also name seeded WebKit,
Chromium or QtWebEngine storage and caches; when the witness's boot is clean,
list them in the report.
Keep the build computer open until the witness holds: a task fix exports
again from it.
## Witness
Witness in a fresh computer of the bundle, from the prompt, as a solving
agent would. Do not replay the build's actions or coordinates, and do not
check between stages to steer the edit: a solving agent never sees the
checks. Give every `check_build_task` the task exactly as exported.
1. `start_build_computer` with
`{"source": {"kind": "bundle", "path": "<bundle>"}}`, adding
`"settings": {"<variable>": "<handle>"}` for each of the export's
`withheld_files`, or the deploy is refused for want of them, and agentenv
keeps that bundle's build failed, so start from a fresh copy of it. The app starts
with the seed's files in place and opens the seed's document when the
seed holds exactly one and the app takes a file. When the export's
`limitations` say it opens no document, open it from the prompt's path,
as a solving agent would. Take a `wait` screenshot first, since the
desktop may not have drawn yet. When the start fails because the
computer never got ready, and its error names lock files in the bundle's
seed, delete them from the bundle and start again.
2. `check_build_task` with `stage: "seed"` and `directory`, the absolute
path of a new or empty folder for the record. A check must fail.
Recording starts. Later stages take no `directory`.
3. Do the task from the prompt with `act_on_build_computer` and save.
`check_build_task` with `stage: "after_edit"`: every check must pass.
Each check waits until the app's files have held for 3 s, as the
bundle's verifier does, so an app that saves on its own needs no wait of
yours. A result with `settled: false` means the files were still
changing at the 15 s bound, and `read_cut: true` that a slow read was
still running there.
4. Make a near miss on top of it: a plausible wrong edit, such as a typo or
a wrong number, and save. `check_build_task` with `stage: "near_miss"`:
a check must fail.
5. `act_on_build_computer` with `action: "reset"`, then `check_build_task`
with `stage: "after_reset"`: a check must fail again.
6. The last stage ends the witness: `witness.record` has the verdict, the
path of `witness.json` and the recording's segments in `directory`.
`close_build_computer` on the witness computer.
A stage that does not hold ends the witness as `failed`. Read each check's
result and decide:
- `after_edit` failed though the saved file is right: the check is brittle.
Fix it, export again and witness in a new computer.
- `after_edit` failed because your edit was wrong: witness again in a new
computer; a computer is witnessed once.
- `near_miss` passed: the checks are too loose. Tighten them, export again
and witness again.
- `after_reset` passed: the reset did not restore the seed. Report it as a
limitation.
Stop after three witnesses and report what still fails.
## Solver runs
Only if the requester asked for them, or by the headless default above. If
they wanted to review the task first, show them its prompt and checks and
wait for their approval. Then run the bundle within the budget, one run at a
time, with `REA_AGENT_ENV_COMMAND`'s runner, or else `agent-env` on PATH:
`agent-env run <bundle> --task <name> --agent claude-code --model <model>`.
The runner needs a model endpoint in your shell: `LITELLM_BASE_URL` and
`LITELLM_API_KEY`, or `base_url` and `api_key` under `[model]` in
`.agentenv/config.toml`. When a run says no model endpoint is set, tell the
person what to set and stop. Count runs reported `passed`. A run
`scored below 1` is a miss; one reported `failed` did not finish, so run it
again if the budget allows instead of counting it. Never run past the budget.
## Close and report
`close_build_computer` on every computer still open, also after a failure.
If a close fails, call it again; if it still fails, tell the requester what
is left.
Report:
- the bundle path and the environment name;
- the task's name and prompt;
- each check, and why it proves the edit and fails the near miss, and each
reader's command;
- each witness stage with its result, the verdict, and the paths of
`witness.json` and the recording;
- the seed's contents: `seed_files` and what each holds;
- `scrubbed_files` and `unscrubbed_files`; an unscrubbed file still names
the host or user, so it needs review before sharing. `recompressed_files`
are gzip seeds whose headers were cleared, their content unchanged: give
their count;
- limitations: the capture's, the export's and anything you could not do;
- solver runs, if any: the model, the runs and the passes;
- the build time and the questions you asked.
Never publish or share the bundle yourself.
@@ -8,9 +8,7 @@ confirmed. The workflow records the application once with
again if a hint names a gap, then writes the bundle with
`export_environment_bundle`. For a HAR the user already has, skip to
[export](#export-and-hand-off) through `inspect_web_network_capture`, or use
`rea environment-build <capture.har> <dir>`. For a task in a desktop
application's exported environment, read
[its section](#tasks-for-a-desktop-application-environment).
`rea environment-build <capture.har> <dir>`.
## Before recording
@@ -287,112 +285,3 @@ See
[agent environments](https://github.com/morluto/rea/blob/main/docs/agent-environments.md)
for what the state, replay and the bundle establish, and
[evidence workflows](evidence-workflows.md) for the manifest and state views.
## Tasks for a desktop application environment
Use this when the user has a folder from `rea capture-desktop-application` and
wants a task for the environment `rea export-desktop-application` makes from
it: a prompt for a computer-use agent and checks that grade what the agent
left in the app's files. You write the prompt and the checks; REA validates
the task against the capture and exports it. These commands are CLI only. To
build the environment and its task from a request instead, follow
[desktop builds](desktop-build.md).
### Describe the change
Run `rea describe-desktop-application-task CAPTURE --format json`. Read:
- `owned_paths`: the only paths whose files the app serves, so the only files
a check can read.
- `files`: each file there whose state differs between the capture's first
and last version, with its `role`, the `baseline` and `final` versions and
the net `change`. Text changes list each added and removed line once, and
`line_counts` gives lines found at both ends whose count changed.
- `values` in each of those files: what the change added, changed and
removed, `before` and `after`, at the pointer a check names it by, with
`refers_to` naming the element a key such as `category="44"` stands for.
Skip entries noted `by_position` or as noise. Only XML files, archives,
JSON and key files have values: a text file's or database's change
already lists its lines and rows.
- `task`: a draft whose `checks` are suggestions from the net change of the
documents the user worked on, or of the app's own state when no document
changed.
- `guards`: optional checks that the rest kept its seed values, such as a
like sibling's field or a text file's neighbouring lines; add one or two
to show the rest is untouched.
- `notes`: why `files` is empty, or why a file's suggestions are thin, such
as `by_position` when no attribute names the changed element.
### Write the prompt
Say what the user wants done, in their words and the app's terms, naming the
app: "In Mousepad, add buy milk to my to-do list and save it". A solving
agent's own shell cannot see the computer's files; the app's name sends it to
the screen. Never say how the file stores it: no
paths the user would not name, line formats, XML, keys or SQL, and never the
checks. Replace the draft's placeholder; the export refuses it.
### Pick the checks
Checks must express only the observed change:
- Keep a suggestion only if it says what the user did. Drop those that follow
from how the app saves: a format rewrite, renumbered lines, counters. A
LibreCAD drawing gaining four holes was rewritten from DXF R12 to 2007, and
the draft suggests counts of `SECTION`, `ENDSEC` and `CIRCLE`; only
`CIRCLE` equals 9 is the user's change.
- Never check unchanged content, whole files, or values the app stamps:
times, ids it assigns, window sizes. REA refuses ISO 8601 and `asctime`
date-times (`Thu Nov 5 09:20:31 2026`); the
rest is your judgment.
- Prefer a few robust checks: whole lines over positions, rows by meaningful
columns over generated ids, `at_least` when the user may add more.
Kinds: `file_present` and `file_absent`; `lines_present` and `lines_absent`
(`lines`); `line_count` (`exact` or `contains`, and `equals`, `at_least` or
`at_most`); `value_equals` and `value_absent` (a JSON `pointer` into the view,
such as `/groups/GROUP/KEY` for a key file); `text_equals` and
`text_contains` (an XML element's whole text at a `pointer`, optional
`normalize_whitespace`); `row_present` and `row_absent` (`table`, `where` by
column) and `row_count`. Each names its `file` as a path under the home. A
document saved as a zip archive, such as FreeCAD's `.FCStd`, reads as its
entries: point into an entry's view, and name array members by selector
rather than position, as
`/entries/[name=Document.xml]/view/root/children/[name=Object][@name=Hole]`,
so the check survives an app version that writes them in another order. A
gzip-compressed document, such as Gnumeric's `.gnumeric`, reads as the
document inside: point into it as into the same document uncompressed. Give
`value_equals` a `tolerance` to compare a number the app writes as text, such
as `"5.0000000000000000"`, with a number. Check what the user wrote in a
document with `text_equals` on its paragraph element, not `value_equals` on a
text node: apps such as AbiWord and Writer split a paragraph into runs as they
save, so its first text node holds only part of it; use `text_contains` for a
value within a sentence, such as a time. A binary document, such as GIMP's
`.xcf`, has no view: give the task a `readers` entry, the app's own headless
command printing `REA ` lines, and a check naming it in `reader`, as
[desktop builds](desktop-build.md#read-a-binary-file-with-a-reader) shows. The
[desktop application tasks](https://github.com/morluto/rea/blob/main/docs/desktop-application-tasks.md)
guide has the exact rules.
### Export the task
Write the task as JSON with a `name`, the `prompt` and the `checks`, then run
`rea export-desktop-application CAPTURE DIRECTORY --task TASK.json`. REA keeps
the task only when its checks fail on the seed and all pass on the captured
end state. A refusal names the check: one that fails on the end state does not
match what the user left; "every check already passes on the seed" means no
check captures the change.
When the export refuses an application that no apt repository installs, ask
the user for the `.deb` file they installed it from and capture with
`--package-file`, then ask where the computer may download that file and pass
it as `--package-url`. Never fetch a vendor's download page yourself: such
sites often refuse automated downloads, and the copy and its licence are the
user's to choose.
Tell the user the task's name, prompt and checks, and how to measure a solve
rate: run `agent-env run DIRECTORY --task NAME --agent claude-code` with
`--model MODEL` three times and count the runs reported `passed`. The runs
need a model endpoint: `LITELLM_BASE_URL` and `LITELLM_API_KEY`, or
`base_url` and `api_key` under `[model]` in `.agentenv/config.toml`. Do not run
agents unless asked. The bundle's seed is the user's real data: keep it local.
@@ -47,7 +47,7 @@ member comparison CLI still takes two assembly paths; process comparison takes
two capture Evidence JSON paths. Check command help for the installed version.
An input bundle grants no implicit file lookup: `evidence-file` and historical
graph file selectors are accepted only by contracts advertising them. See the
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/getting-started/cli.md#import-export-and-compare-evidence).
[CLI reuse guide](https://github.com/morluto/rea/blob/main/docs/cli.md#import-export-and-compare-evidence).
Completed `inspect_artifact` inventory also supports a summary and
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
@@ -6,7 +6,7 @@ For captured or bundled JavaScript that needs readable source recovery, use
`additional_paths` when advertised, then analyze the returned `analysis_input`
directory together. Each input retains its own snapshot, digest and byte-range
provenance; inputs and recovered code are never executed. Recovery requires a
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/guides/javascript/javascript-recovery.md).
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
extracted tree. When the question concerns known source paths and the connected
@@ -67,18 +67,11 @@ See [desktop installations](https://github.com/morluto/rea/blob/main/docs/deskto
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
files need `architecture`) when the
question concerns the embedded compiler version, module/dependency versions,
local replacements, build settings, or static Go string literals packed without
NUL terminators. Pass `include_string_literals: true`
(`--include-string-literals`, optional `max_string_literals` /
`--max-string-literals`) to recover bounded inferred word-aligned `(ptr, len)`
UTF-8 string slices from read-only data and page them through `go`
`string_literals` views. No `open_binary`, Go installation or
local replacements or build settings. No `open_binary`, Go installation or
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
Preserve artifact SHA-256, original file ranges, framed module bytes and every
unparsed module-text line. A missing record is an unknown, not proof of a
non-Go binary; observed metadata does not authenticate its claims, and static
`(ptr, len)` string recovery excludes compiler-inlined stack/immediate strings
without proving runtime reachability. Function,
non-Go binary; observed metadata does not authenticate its claims. Function,
source and type recovery remain a separate native-analysis question.
## Model files
@@ -149,20 +142,6 @@ table or overlay. Bound snapshots, loaded IAT contents, and export-forward
resolution remain unobserved. See
[PE import and delay-load tables](https://github.com/morluto/rea/blob/main/docs/pe-imports.md).
## Windows Shell Link shortcuts
Use `inspect_windows_shortcut` for an explicit local `.lnk` file when the
question concerns the recorded target path, arguments, working directory, icon
location, volume serial, network share names, ExtraData blocks, or Tracker
droid identifiers. No `open_binary`, Windows SDK, COM, or target execution is
required. The CLI equivalent is `inspect-windows-shortcut PATH`. Preserve
artifact SHA-256, FILETIME raw decimals, IDList type bytes, and ExtraData
signatures. Unknown IDList items and ExtraData signatures remain raw with
SHA-256; they are not guessed. Version-1 UUID MAC and timestamp values are
labelled derived. A missing optional section is observed absence of that
structure. See
[Windows Shell Link shortcuts](https://github.com/morluto/rea/blob/main/docs/guides/native/windows-shortcut.md).
## PE TLS callbacks and load configuration
Use `inspect_pe_load_config` for an explicit local PE32/PE32+ image when the
@@ -176,18 +155,6 @@ mitigations carry `evidence_kind: "inferred"`; enforcement remains unobserved.
See
[PE TLS callbacks and load configuration](https://github.com/morluto/rea/blob/main/docs/pe-load-config.md).
## Binary hardening
Use `inspect_binary_hardening` for an explicit local ELF, PE, or Mach-O file
when the question concerns PIE or ASLR, NX, RELRO, stack-protector or FORTIFY
signs, control-flow flags, or stripped state. No `open_binary`, pwntools, or
target execution is required. The CLI equivalent is
`inspect-binary-hardening PATH`. Universal Mach-O files return one image per
slice. A missing import is `unknown`, not proof the protection was omitted.
Authenticode and Mach-O code-signature rows report presence, not validity.
`hardened_runtime` stays unknown and names `inspect_signature`. See
[binary hardening](https://github.com/morluto/rea/blob/main/docs/binary-hardening.md).
## ELF note tables
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
@@ -199,36 +166,6 @@ absence of those declarations, not proof the image never carried notes in
another container. See
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
## ELF and Mach-O initializer functions
Use `inspect_native_initializers` for an explicit local ELF or Mach-O image
when the question is which functions run before `main` or at exit: ELF
`DT_PREINIT_ARRAY`, `DT_INIT`, `DT_INIT_ARRAY`, `DT_FINI`, `DT_FINI_ARRAY` and
legacy `.ctors`/`.dtors`, or Mach-O `__mod_init_func`, `__init_offsets` and
`__mod_term_func`. Pass `architecture` for a universal Mach-O. The CLI
equivalent is `inspect-native-initializers PATH`. PIE slots are resolved
through RELATIVE, RELR and section relocations, and Mach-O slots through
chained fixups. Keep each entry's `resolution`: `unresolved` means the value was
not established, and `unresolved_import` names an external symbol. Page with
`offset` and `limit`. See
[ELF and Mach-O initializer functions](https://github.com/morluto/rea/blob/main/docs/guides/native/native-initializers.md).
## ELF unwind function ranges
Use `inspect_elf_unwind_ranges` for an explicit local ELF32/ELF64 image when
the question concerns function starts and lengths that survive `strip`, which
functions have an LSDA, or which ranges have no exact symbol. No
`open_binary` or target execution is required. The CLI equivalent is
`inspect-elf-unwind-ranges PATH`. Preserve artifact SHA-256, FDE start and
end, containing section, exact symbol name, CIE index, LSDA, and personality.
Page later ranges with `offset` and `limit`. A missing `.eh_frame` is
observed absence of that declaration and of `PT_GNU_EH_FRAME`, not proof the
image has no functions. Header disagreement is a limitation and does not drop
decoded ranges. `.debug_frame` is reported present or absent and is not
decoded. CFA instructions are counted and not evaluated. Indirect EH pointers
are not dereferenced. See
[ELF unwind function ranges](https://github.com/morluto/rea/blob/main/docs/guides/native/elf-unwind-ranges.md).
## Apple IPS crash reports
Use `inspect_apple_ips` for an explicit local Apple `.ips` file when the
@@ -260,27 +197,6 @@ container. gzip, xz, and zstd whole-file containers are unsupported and name
decompress into a separate file, then call `inspect_kernel_module` on that file. See
[Linux kernel modules](https://github.com/morluto/rea/blob/main/docs/kernel-modules.md).
## Linux kernel build config
Use `inspect_kernel_build_config` for an explicit local kernel image (`vmlinux`
ELF, raw arm64 `Image`, x86 `bzImage`/`zImage`, or `configs.ko`) when the
question is which kernel version it is or how it was configured. No
`open_binary`, Linux host, or `extract-ikconfig` is required. The CLI
equivalent is `inspect-kernel-build-config PATH`; pass `name` for one exact
`CONFIG_*` option and `offset`/`limit` to page the rest. Preserve the banner
version, builder, and compiler, the config SHA-256 and option count, and each
entry's value kind (`y`, `m`, `string`, `number`, `not_set`). A config is
accepted only when `IKCFG_ST` is immediately followed by a valid gzip member
and `IKCFG_ED`, so an ordinary `IKCFG_ST` string is not a config. Offsets with
`space: "inflated_payload"` are positions in an inflated bzImage payload, not
file offsets. An absent config describes only the searched bytes; the kernel
may have been built without `CONFIG_IKCONFIG` or with it as a module. Truncated
payloads are incomplete, not absent. A framed blob that does not decode, has a
bad CRC, or holds non-config text is `malformed`, not absent. xz, lz4, lzma, zstd, bzip2, and lzo
payloads are unsupported and name `external_decompression`: decompress with an
external utility, then call `inspect_kernel_build_config` on the result. See
[Linux kernel build config](https://github.com/morluto/rea/blob/main/docs/guides/native/kernel-build-config.md).
## CPython pyc files
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
@@ -318,7 +234,7 @@ Choose `row_limit` when a finite sample is useful and keep truncation and
unspecified row order explicit. Virtual tables, shadow tables and views are
schema facts, not row-read targets. REA inspects private copies and leaves
the source files untouched; obtain a coherent quiescent database/WAL snapshot
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/guides/data/sqlite-databases.md).
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
## Recorded Linux crashes
@@ -333,7 +249,7 @@ The optional `include_debugger_context` facet adds mapping candidates through
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
establish current file identity; zero reported flags leave permissions unknown.
Requested unavailable context fails with setup guidance. See
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/guides/native/recorded-crashes.md)
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
for exact upstream profiles, bounds and verification coverage.
## Offline ELF layout
@@ -392,18 +308,6 @@ cross-references. Addresses and recovered pseudocode are analysis observations,
not original source. Provider unavailability and unsupported metadata remain
unknown rather than false.
For large pseudocode, request `procedure_pseudo_code` with `detail: "summary"`
on the first MCP call. It retains the complete observation and returns the
original selection, text availability and length without the body. Use
`normalized_result.parent_evidence_id` with `inspect_analysis_view` and
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`;
follow `coverage.next_offset` for later pages. Choose a smaller limit if one
page exceeds the client or transport budget. Offsets and limits count UTF-16
code units, not lines or tokens. Pages reuse the same observation without
repeating decompilation. Null and empty text remain distinct. Omitted or
`complete` detail preserves full inline delivery. Summary delivery requires a
retaining session and does not reduce the work or memory of decompilation.
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
and the exact retained Evidence ID to read selected facts without reanalysis.
@@ -454,28 +358,6 @@ operation's full ordered list. Candidate capabilities, returned with complete
detail, describe possible providers. Top-level session capabilities describe the current route, and
`analysis_provider_binding` identifies the selected provider.
## Continue editable Ghidra analysis across processes
For a task spanning agent sessions, check `binary_session` after selecting
Ghidra: the current provider's `capabilities[].shared_limitations` states whether
Program checkpoints are enabled or the session is ephemeral. Successful native
queries distinguish initial analysis from an actually restored packed Program;
do not infer that restoration happened merely because checkpoint mode is enabled.
In ephemeral mode, choose a private absolute directory and set
`REA_GHIDRA_CHECKPOINT_DIR` in the environment starting REA. Restart the MCP server
after changing that launch configuration; no current tool call or runtime
analysis-setting update enables it. Stock ELF/PE/Mach-O imports on Linux/macOS
are supported; see the [checkpoint setup and limitations](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program).
With the setting enabled, normal opens restore compatible saved Programs
automatically. Initial completed analysis is saved, and `close_binary` saves
later metadata edits; no separate save/restore tool is needed. Repair a reported
save failure and retry close. Cancellation or process failure can lose edits
after the last successful save; unfinished initial analysis cannot resume.
Keep the same checkpoint configuration for later processes. Evidence/Unknown
history and immutable REA query snapshots have separate persistence workflows.
## Reuse caller-selected Ghidra analysis
When the caller supplies an expanded project, configure `REA_GHIDRA_PROJECT_PATH`
@@ -487,7 +369,7 @@ import overrides and repair scripts are unavailable in this mode. Treat original
analysis completeness/settings as unknown and prior edits as caller-project state.
The program's recorded import digest must match the target; changed project content
requires reopening for a fresh profile. See the
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/getting-started/installation.md#reuse-an-existing-ghidra-project).
[setup and preparation guide](https://github.com/morluto/rea/blob/main/docs/installation.md#reuse-an-existing-ghidra-project).
## Ghidra startup deadline recovery
@@ -499,7 +381,7 @@ When the error names `ghidra.startup_timeout_ms`, call
`update_analysis_settings` with `action: "set"` and a larger value within the
reported bounds. Resolve retained cleanup before the next startup attempt.
No REA restart is needed; an already accepted startup keeps its original
budget. Follow the error's session-specific restart guidance, and preserve useful Evidence before closing
budget. A new attempt repeats import. Preserve useful Evidence before closing
and reopening the target when remediation requires that lifecycle step.
For a per-function timeout, update `ghidra.decompile_timeout_seconds` and retry
@@ -521,7 +403,7 @@ and memory controls in place. Disable specific expensive analyzers only when
their missing coverage is acceptable and retain the resulting limitations.
Exported C and embedded resources remain useful independent evidence, but a
timeout does not establish complete analysis or absence of code. See the
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
## Bridge request ceiling
@@ -672,7 +554,7 @@ broader metadata recovery on Linux; it requires an explicitly configured
NativeAOT JAR. These are separate routes. The built-in parser also runs on
Windows Ghidra P0; read frozen literals from its report, because
`search_strings` does not include them. See the
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-nativeaot.md)
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
for setup, supported layouts, and host restrictions. Preserve binary identity
and distinguish derived metadata from Ghidra database observations. Recovered
types and native pseudocode are not original C# source. Windows Ghidra P0 has
@@ -701,7 +583,7 @@ every executable in the bundle. It also lists missing or weak loads and earlier
provides them.
Use `extract_artifact` when materialized files from a supported format are needed.
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/reference/mcp-contracts.md).
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
It materializes all regular files into a fresh temporary directory chosen by
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
The result reports the materialized directory; callers do not choose the
@@ -729,7 +611,7 @@ temporary, not a retained export of the image's contents.
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
`--target-format dos-com`). Inspect the load image before trusting function
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/providers/ghidra-dos.md).
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
@@ -743,7 +625,7 @@ otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
superset of the 68000 family; Amiga/Atari containers and vector-table entry
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/providers/raw-binary.md).
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
Intel HEX, S-record, UF2 and DfuSe files are containers, not raw images:
`inspect_artifact` lists address regions (UF2 per familyID, DfuSe per element
@@ -755,12 +637,7 @@ numbering gaps leave coverage unknown; a DfuSe suffix CRC mismatch follows
### Function annotations in Ghidra
Fresh-import sessions admit annotations; existing-project reuse is read-only.
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits. Normal close saves them when owned Program checkpoint mode is enabled; ephemeral mode discards them. CLI `annotate-native-function` follows the same setting when closing after its updated result. Select the [checkpoint workflow](#continue-editable-ghidra-analysis-across-processes) before edits that must survive a later process. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
A provider switch or different target can retire the active Ghidra database and discard
its edits, even if the new provider fails on a later call. Same-target,
same-profile reopen keeps the active database. Retained annotation Evidence
records what happened; it does not restore the edits.
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier. Ghidra rejects batch addresses that request the same name or a name another selected address holds; `raw_result.failures` gives each failure's reason.
For `compare_functions`, obtain `analyze_function` Evidence before and after the
edit while the MCP session is open; annotation Evidence is not a comparison
@@ -782,7 +659,7 @@ code pages, payload hashes and static icon-group references. Same-language icon
resolution is explicit; missing references and size mismatch remain visible.
Other payloads are opaque, and this operation does not decode images or simulate
Windows resource loading. Consult the budgets and supported tree profile in
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
## Offline PE and ELF export tables
@@ -792,7 +669,7 @@ directory or ELF dynamic symbol table on any host without opening a native
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
partial or unsupported coverage means the inventory is incomplete, not that
exports are absent. Use `inspect_macho` for Mach-O. See
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/guides/native/binary-diagnostics.md).
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
For exact core-module import/export differences, use `compare_wasm_interfaces`
with two complete inspection Evidence records (or current-session retained
@@ -831,13 +708,6 @@ cache one decoded block and support retained filtered/paged views. No
decryption or SerializedFile object decoding is provided. See
[UnityFS](https://github.com/morluto/rea/blob/main/docs/unityfs.md).
WeChat mini-program packages (`.wxapkg` files with `0xBE`/`0xED` marks) list
members with declared names and absolute carrier offsets; member bytes are
stored verbatim. `V1MMWX` PC-client packages report `encrypted` directory
status with unknown coverage and are refused on extraction; no decryption is
attempted. See
[wxapkg](https://github.com/morluto/rea/blob/main/docs/wxapkg.md).
U-Boot legacy uImages use the same inspection/extraction workflows. Prefer a
retained summary and selected inventory views for header facts or component
IDs. Header addresses/CPU/OS are declarations; payloads remain as stored.
@@ -908,24 +778,3 @@ Layout labels describe record shapes, not a detected compiler version; unknown
or ambiguous layouts are unsupported. Bytecode is retained, not decoded, and the
executable is never run. See
[Bun standalone executables](https://github.com/morluto/rea/blob/main/docs/bun-standalone.md).
## Deno compile executables
A `deno compile` executable stores its application in a Mach-O `__SUI,d3n0l4nd`
section, an Intel Mach-O `<~sui-data~>` trailer, an ELF `SUI` note of type
`0x53554901`, or a PE `RT_RCDATA` resource named `D3N0L4ND`.
`open_binary` then `inspect_artifact` reports root format `deno-standalone`
with the carrier, entry point, recorded options, and one occurrence per virtual
file, including size and SHA-256. Transpiled output, CJS export analysis, and
source maps are separate occurrences with the file's `index` and their own
`role`. They normally use a `.deno-*` suffix on the file path and move to an
indexed path on a collision, so use the reported `logical_path`. Symlink targets
stay recorded path parts. A virtual filesystem that cannot be listed leaves
coverage unknown.
Environment-file values and CA certificate bytes are retained in local Evidence
with their counts and byte lengths. The npm snapshot length is reported and not
decoded. `extract-artifact` writes
stored files through the safe output tree; route extracted JavaScript to
`analyze_javascript_application`. Other layouts are unsupported, and the
executable is never run. See
[Deno compile executables](https://github.com/morluto/rea/blob/main/docs/deno-standalone.md).
@@ -41,16 +41,10 @@ behavior unknown. Record truncation, unmatched breakpoints, changed target
identity and cleanup uncertainty as limitations, rather than treating no retained
event as proof that the behavior never occurs.
For same-session MCP comparisons, pass each capture inline or reuse it as
`{"kind":"retained-evidence","evidence_id":"ev_..."}`. Default comparison
keeps capture-local timing as provenance without treating scheduler or observer
latency as a behavior change; use an exact `trace_spec` event when timing is part
of the analyst question.
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/guides/runtime/frida-instrumentation.md),
Detailed contracts and prerequisites: [Frida](https://github.com/morluto/rea/blob/main/docs/frida-instrumentation.md),
[GDB/Rizin/Cutter](https://github.com/morluto/rea/blob/main/docs/reverse-engineering-tools.md),
[process capture](https://github.com/morluto/rea/blob/main/docs/guides/runtime/process-capture.md),
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/guides/native/native-investigation.md#native-desktop-observation).
[process capture](https://github.com/morluto/rea/blob/main/docs/process-capture.md),
and [native calls/UI](https://github.com/morluto/rea/blob/main/docs/native-investigation.md#native-desktop-observation).
## Browser and Electron
@@ -136,5 +130,5 @@ Inspect native `.saz` snapshots with `inspect_web_network_capture` and
traffic analysis, connect Everywhere's separate MCP server and use its
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
`AgentCalls`). Preserve existing sessions and filters when inspecting a
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/guides/web/web-network-captures.md)
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
for the workflow and Telerik's public setup and traffic-debugging skills.
+1 -1
View File
@@ -27,7 +27,7 @@ body:
attributes:
label: Documentation location
description: Link the page, section, command, API, skill, or inline comment.
placeholder: README.md#quick-start or docs/reference/mcp-contracts.md
placeholder: README.md#quick-start or docs/mcp-contracts.md
validations:
required: true
+3 -3
View File
@@ -71,8 +71,8 @@ source, link, or rendered-document checks. -->
- User-visible CLI/MCP output (if applicable):
- Remaining proof gaps:
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#boundary-regressions)
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/development/testing.md#lifecycle-regressions)
For boundary fixes, identify the applicable [boundary regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#boundary-regressions)
and [lifecycle regressions](https://github.com/morluto/rea/blob/main/docs/testing.md#lifecycle-regressions)
covered by the evidence above, and explain relevant gaps.
For evidence-bearing changes:
@@ -99,7 +99,7 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
its new behavior and migration, and why compatibility/deprecation cannot
preserve it. Every release increments minor, including breaking changes;
keep breaking markers and migration notes so callers can assess compatibility.
See docs/development/releasing.md. -->
See docs/releasing.md. -->
- Real Hopper/Ghidra, browser, or OS coverage:
- Package or release metadata impact:
- Security, privacy, process, or containment review:
+2 -3
View File
@@ -924,7 +924,6 @@ jobs:
tests/process-global/filesystemSnapshotFallbackCancellation.test.ts
tests/boundary/filesystem/typescriptCompiler.test.ts
tests/boundary/filesystem/releaseProposal.test.ts
tests/acceptance/byteEntropyWindowsIdentity.test.ts
tests/boundary/filesystem/binaryTarget.test.ts
src/contracts/sessionPathInputs.test.ts
tests/boundary/filesystem/directAnalysisSnapshotReplay.test.ts
@@ -1017,8 +1016,8 @@ jobs:
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run docs:check
- name: Verify published routes, source links and local anchors
run: npm run verify:docs:routes
- name: Render documentation and validate page links
run: npm run docs:build
- name: Generation leaves authored files unchanged
run: git diff --exit-code
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
@@ -95,31 +95,3 @@ jobs:
decode cortex ARM:LE:32:Cortex mrs/4/msp bx/2/lr
decode v8m ARM:LE:32:v8-m mrs/4/msplim bx/2/lr
decode xtensa Xtensa:LE:32:default entry/3/a1 retw.n/2
# A call through a writable slot keeps Ghidra's recorded target as an
# observed callee while callee coverage stays partial; a direct call is
# the complete control. The source-owned x86-64 bytes are never executed.
- name: Keep computed-call targets non-exhaustive in callee coverage
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
export JAVA_HOME="${!java_home_variable}"
probe_root="${RUNNER_TEMP}/rea-computed-call"
mkdir -p "${probe_root}/project"
{
printf '\270\001\000\000\000\303\314\314' # 0x1000: mov eax, 1; ret
printf '\270\002\000\000\000\303\314\314' # 0x1008: mov eax, 2; ret
printf '\377\025\052\000\000\000\303\314' # 0x1010: call [rip + 0x2a]; ret
printf '\377\320\303\314\314\314\314\314' # 0x1018: call rax; ret
printf '\350\333\377\377\377\303\314\314' # 0x1020: call 0x1000; ret
printf '\314%.0s' $(seq 24) # 0x1028: int3 padding
printf '\000\020\000\000\000\000\000\000' # 0x1040: slot = 0x1000
} > "${probe_root}/computed-call.bin"
test "$(wc -c < "${probe_root}/computed-call.bin")" -eq 72
timeout 180s "${GHIDRA_INSTALL_DIR}/support/analyzeHeadless" "${probe_root}/project" ComputedCall \
-import "${probe_root}/computed-call.bin" -loader BinaryLoader -loader-baseAddr 0x1000 \
-processor x86:LE:64:default -noanalysis -readOnly -deleteProject \
-scriptPath "${GITHUB_WORKSPACE}/tests/conformance/ghidra;${GITHUB_WORKSPACE}/bridge/ghidra" \
-postScript ReaComputedCallCoverageProbe.java "${GITHUB_WORKSPACE}/bridge/ghidra" \
> "${probe_root}/output.log" 2>&1 || { cat "${probe_root}/output.log"; exit 1; }
cat "${probe_root}/output.log"
grep -F 'REA_COMPUTED_CALL_COVERAGE_OK slot=observed-complete-discovery-unknown register=partial direct=complete callers=discovery-unknown' "${probe_root}/output.log"
+1 -1
View File
@@ -27,4 +27,4 @@ jobs:
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- run: npm run verify:docs:routes
- run: npm run docs:build
+7 -8
View File
@@ -54,11 +54,10 @@ jobs:
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
# The pinned framework builds its gateway and service-db images only when pulling them from
# ghcr.io fails, from public ECR base images; public ECR caps anonymous pulls by data per source
# IP and hosted runners share IPs, so such a build can fail here for hours. mirror.gcr.io serves
# the same Docker official images under the same digests, so each base image those Dockerfiles
# name is pulled there and named as its Dockerfile names it, as the framework's own CI does.
# Public ECR caps anonymous pulls by data per source IP and hosted runners share IPs, so the
# framework's image builds can fail here for hours. mirror.gcr.io serves the same Docker
# official images under the same digests, so each base image the pinned framework builds from
# is pulled there and named as its Dockerfile names it, as the framework's own CI does.
# Digest-pinned names are set in containerd, whose images Docker reads only on the containerd
# image store; switching stores restarts dockerd, so it happens before anything uses Docker.
- name: Use the containerd image store
@@ -73,16 +72,16 @@ jobs:
run: |
runner_root="${RUNNER_TEMP}/rea-agentenv-runner"
python -m venv "${runner_root}"
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1431'
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1334'
"${runner_root}/bin/agent-env" --version
docker version --format '{{.Server.Version}}'
echo "REA_AGENT_ENV_COMMAND=${runner_root}/bin/agent-env" >> "${GITHUB_ENV}"
echo "REA_DOCKER_COMMAND=$(command -v docker)" >> "${GITHUB_ENV}"
- name: Pull the gateway and service-db public ECR base images from mirror.gcr.io
- name: Pull the pinned framework's public ECR base images from mirror.gcr.io
shell: bash
run: |
package="$("${REA_AGENT_ENV_COMMAND%/agent-env}/python" -c 'import agent_env, pathlib; print(pathlib.Path(agent_env.__file__).parent)')"
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}/env/gateway" "${package}/env/envs/service_db" | awk '{print $2}' | sort -u |
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}" | awk '{print $2}' | sort -u |
while read -r image; do
name=${image#public.ecr.aws/docker/library/}
source=mirror.gcr.io/library/$name
+6 -12
View File
@@ -14,13 +14,13 @@ Preserve caller-selected inputs and local evidence, including output, URLs, path
## Architecture and Cleanup
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture/architecture.mermaid](docs/architecture/architecture.mermaid) when changing composition.
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
For lifecycle changes, follow [ownership and completion invariants](docs/development/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/development/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.
@@ -68,12 +68,6 @@ do not establish fewer tokens or faster analysis.
## Development
Documentation ownership and authoring conventions live in
[docs/development/documentation.md](docs/development/documentation.md). A feature
uses `docs/specs/<domain>/<feature-slug>/spec.md`, one `plan.md` and `tasks.md`;
individual architectural decisions use `docs/adr/<domain>/<decision-slug>.md`.
Consult the [documentation directory](docs/README.md) before adding a guide.
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
- `npm ci`: install locked dependencies.
@@ -86,15 +80,15 @@ Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain.
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
When changing installers or setup recovery, follow
[Checking installers and publisher downloads](docs/development/testing.md#checking-installers-and-publisher-downloads).
[Checking installers and publisher downloads](docs/testing.md#checking-installers-and-publisher-downloads).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/development/testing.md#developer-commands).
Coordinate expensive checks across agents. If a test unexpectedly consumes heavy CPU, stop its owned process family and profile a bounded representative case before retrying. Inspect production work, fixture generation, subprocesses, and repeated setup; preserve regression coverage when optimizing. Follow the [test resource guidance](docs/testing.md#developer-commands).
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
## Tool and Test Changes
For tool design, follow [docs/development/tool-design.md](docs/development/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
For user- or agent-facing tool problems, prioritize fixing the owning behavior,
contract, and actionable result or recovery at the point of use. Document the
@@ -102,7 +96,7 @@ result in product docs when explanation is needed. Add skill guidance only for
workflow judgment or context that the tool cannot reasonably convey; do not use
a skill as the primary repair for a discoverability or contract defect.
For test changes or provider claims, use [docs/development/testing.md](docs/development/testing.md). Select applicable [boundary regressions](docs/development/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Select applicable [boundary regressions](docs/testing.md#boundary-regressions) for parser, inventory, snapshot, and projection fixes. Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
+7 -7
View File
@@ -132,8 +132,8 @@ Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto
- **Kuna analysis:** Use a caller-installed Kuna for procedure listings, pseudocode, assembly, call relationships, exact-address xrefs and cached string searches. Project exports retain source spans and digests; repeated requests reuse completed work. Recorded real-provider coverage uses Kuna 1.833 on Linux; Windows, PE and universal Mach-O are unsupported. See [the provider guide](docs/kuna-provider.md). ([#3832](https://github.com/morluto/rea/pull/3832), [#3854](https://github.com/morluto/rea/pull/3854), [#4071](https://github.com/morluto/rea/pull/4071), [#4095](https://github.com/morluto/rea/pull/4095)). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
- **Retained JavaScript facts:** Query exact application paths, selected properties and filtered module references through `inspect_analysis_view`, with paging and complete row Evidence. These views reuse completed analysis and preserve source selection and unknown coverage. ([#4265](https://github.com/morluto/rea/pull/4265), [morluto/rea#4190](https://github.com/morluto/rea/issues/4190), [#4097](https://github.com/morluto/rea/pull/4097), [morluto/rea#4092](https://github.com/morluto/rea/issues/4092)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **APK locale tables:** Request several resource locales in one decode with repeatable CLI `--locales` or MCP `locales`. Each table reports its status and coverage; existing single-locale fields remain available. ([#4804](https://github.com/morluto/rea/pull/4804), [morluto/rea#4777](https://github.com/morluto/rea/issues/4777)). Thanks @abhay-2108, @N0zoM1z0.
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/providers/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/providers/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
- **Windows ARM binaries:** Ghidra admits ARMNT/Thumb-2 and ARM64 PE applications and DLLs on the Windows x64 host lane. Recorded Ghidra 12.1.4 verification uses `WindowsResourceReference` disabled and raised startup budgets; default-analyzer completion, ARM64EC/ARM64X and Windows-on-Arm hosts remain unverified or unsupported. See [the exact scope](docs/windows-ghidra-p0.md#native-authority-and-target-identity). ([#3965](https://github.com/morluto/rea/pull/3965), [morluto/rea#2186](https://github.com/morluto/rea/issues/2186)). Thanks @N0zoM1z0, @WestLabsAI.
- **Raw m68k images:** Analyze explicitly selected big-endian m68k raw binaries using Ghidra’s fixed 68000 profile and a caller-supplied aligned entry. Real CLI/MCP verification used Ghidra 12.1.4 on macOS arm64; Linux is unverified and Windows P0 does not admit m68k. See [mapping requirements](docs/raw-binary.md). ([#4018](https://github.com/morluto/rea/pull/4018)). Thanks @icedac.
- **JADX deadlines:** Set `REA_JADX_OPERATION_TIMEOUT_MS` to extend APK loading and upstream operation deadlines together. Doctor reports the effective value and setup forwards an explicit selection. ([#4466](https://github.com/morluto/rea/pull/4466), [morluto/rea#3918](https://github.com/morluto/rea/issues/3918)). Thanks @jaychou813, @SandstromPL.
- **Electron window options:** Resolve stable local constant `BrowserWindow` options while retaining source provenance. Relevant writes and escapes keep the result unknown. ([#4400](https://github.com/morluto/rea/pull/4400), [morluto/rea#4366](https://github.com/morluto/rea/issues/4366)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Language editions:** Add the Greek website edition and Malayalam documentation, with Korean typography and translation corrections across the site. ([#3520](https://github.com/morluto/rea/pull/3520), [#4792](https://github.com/morluto/rea/pull/4792), [#3821](https://github.com/morluto/rea/pull/3821), [morluto/rea#3820](https://github.com/morluto/rea/issues/3820), [#3799](https://github.com/morluto/rea/pull/3799), [morluto/rea#3798](https://github.com/morluto/rea/issues/3798), [#4344](https://github.com/morluto/rea/pull/4344), [#3856](https://github.com/morluto/rea/pull/3856), [#4166](https://github.com/morluto/rea/pull/4166)). Thanks @gensokyo-dev, @hi2242, @kalayciburak, @kaoru0822-kitauji, @kraken-503, @kukinghan, @N0zoM1z0, @yeseong0412.
@@ -192,7 +192,7 @@ Full contribution record: [571 commits and 236 merged PRs](https://github.com/mo
## [6.5.0](https://github.com/morluto/rea/compare/rea-agents-6.4.0...rea-agents-6.5.0) (2026-10-13)
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/development/testing.md).
Provider prerequisites and recorded verification scope are listed in [the testing guide](docs/testing.md).
### Highlights
@@ -220,7 +220,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **HarmonyOS packages:** inventory HAP/HSP/App Pack structures without executing code. The recorded real workflow covers one pinned HAP through repository-built CLI/MCP; installed-package and real HSP/App Pack coverage remain unverified. ([#1447](https://github.com/morluto/rea/pull/1447)). Thanks @BaaRai.
- **Ghidra raw inputs and references:** compatible Linux/macOS Ghidra profiles accept caller-selected raw architecture/entry information and expose typed incoming references at exact addresses. Windows Ghidra P0 admits the raw NMOS 6502 profile described below; other raw targets and the incoming-reference facet remain outside Windows P0. ([#1868](https://github.com/morluto/rea/pull/1868), [issue morluto/rea#1836](https://github.com/morluto/rea/issues/1836), [#1715](https://github.com/morluto/rea/pull/1715), [issue morluto/rea#1245](https://github.com/morluto/rea/issues/1245)). Thanks @bugstan, @casesolved-co-uk, @mvanhorn, @N0zoM1z0.
- **Ghidra MIPS:** Add bounded MIPS32r2/o32 relocatable objects, soft-float integer-register storage, admitted MIPS16 shared ELF, and a narrowly admitted recordless legacy MIPS32 executable profile. The latter pins an endian-matched 32-bit language and records inferred o32 as an analysis assumption. Admission follows each documented ABI/flag/relocation boundary. ([#1949](https://github.com/morluto/rea/pull/1949), [issue morluto/rea#1937](https://github.com/morluto/rea/issues/1937), [#1950](https://github.com/morluto/rea/pull/1950), [issue morluto/rea#1938](https://github.com/morluto/rea/issues/1938), [#1991](https://github.com/morluto/rea/pull/1991), [issue morluto/rea#1851](https://github.com/morluto/rea/issues/1851), [#1898](https://github.com/morluto/rea/pull/1898), [issue morluto/rea#1620](https://github.com/morluto/rea/issues/1620)). Thanks @AurelioB, @BaaRai, @bqw371, @kaoru0822-kitauji, @N0zoM1z0.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/providers/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **Ghidra RISC-V:** admit little-endian RV32/RV64 `ET_EXEC` ELF declaring the supported double-float ABI. Relocatable/shared, big-endian and other ABI profiles remain refused; see [the profile and recorded lane](docs/ghidra-riscv.md). ([#2151](https://github.com/morluto/rea/pull/2151)). Thanks @NIU-123370.
- **PSP static ELF profile:** the Ghidra profile is Linux x64, bounded little-endian ELF32 `ET_EXEC` with the documented PSP/ABI declarations and a compatible caller-installed Allegrex extension. PRX, shared/relocatable images, PBP/ISO/CSO and runtime emulation are outside scope. Historical extension experiments are separate from public-provider verification; real public Ghidra PSP coverage remains unverified. ([#1599](https://github.com/morluto/rea/pull/1599)). Thanks @jaelliot.
- **Ghidra analysis settings:** apply validated analyzer overrides before analysis while retaining the selected analysis profile. ([#1913](https://github.com/morluto/rea/pull/1913)). Thanks @akushonkamen, @kaoru0822-kitauji, @N0zoM1z0.
- **Attached native IDA MCP:** add an optional native bridge alongside existing legacy/headless profiles. Recorded native runs use Linux IDA Free 9.4 with a navigation correction and Windows IDA Pro 9.3 with a loopback-listener change; stock native switching, native macOS and Windows IDA Free remain unverified. Attached databases remain read-only and are never saved or closed. ([#1544](https://github.com/morluto/rea/pull/1544)). Thanks @imissapixel, @N0zoM1z0.
@@ -292,7 +292,7 @@ Provider prerequisites and recorded verification scope are listed in [the testin
- **Source maps:** original-source and mapping rows preserve a null source name when unavailable. Handle null explicitly instead of expecting an invented unknown-source string; new index fields remain optional when reading older Evidence. ([ee50ed1](https://github.com/morluto/rea/commit/ee50ed121e38cc6e1a2ff0dc8b789f15f71c5402)).
- **Streamed filtered JSON:** `--filter-output` now emits JSON `null` instead of `{}` when the selected value is null; consumers must distinguish a present null from a missing selection. Use `--full-output` when an object envelope is required. ([#2333](https://github.com/morluto/rea/pull/2333)). Thanks @Jackmeson1.
- **Call graphs:** `get_call_graph.normalized_result` is now `{ graph, limitations }`; read traversal-depth groups under `graph` and retain relationship-specific caveats. Contract/fixture validation does not establish new Ghidra or IDA engine coverage. ([5dfb9d0](https://github.com/morluto/rea/commit/5dfb9d050c2cc44a61b183dae765bef4d22a53b7)).
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/reference/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **MCP discovery:** `binary_session` no longer repeats `server_identity.catalog.tools`. Use the connected server tool list, grouped `tool_availability` and provider capabilities; see [MCP contracts](docs/mcp-contracts.md). Origin: [6395e8d](https://github.com/morluto/rea/commit/6395e8d3007d5ee453477460e7ee1c51cff76be0). ([#1988](https://github.com/morluto/rea/pull/1988)). Thanks @kaoru0822-kitauji, @N0zoM1z0.
- **Provider admission:** MIPS targets with unresolved GNU ABI attributes are refused. Supply supported ABI declarations. Saved Hopper databases on macOS are unavailable because the launcher cannot run the analysis bridge when opening a database; select an executable target instead. Database support requires a script-capable database launch route. ([d4bd65c](https://github.com/morluto/rea/commit/d4bd65c53c0254d0195cc7ab8da163dcadfd3b9b)).
- **Segment extents:** `binary_overview` segment length may be null when Hopper coordinates cannot establish a finite extent in one address space. Preserve unknown rather than converting it to zero or a saturated value. ([36db19c](https://github.com/morluto/rea/commit/36db19c59)).
@@ -372,7 +372,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/guides/runtime/process-capture.md) and [browser scenario contract](docs/guides/web/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **captures:** Historical captures without current producer accounting or transaction identity, and web diffs missing accessibility or storage dimensions, are rejected. Preserve original captures and recapture with the current producer; update comparison consumers to include every dimension rather than synthesizing missing observations. See the [process capture guide](docs/process-capture.md) and [browser scenario contract](docs/browser-scenario-contract.md). ([3283a45](https://github.com/morluto/rea/commit/3283a45d09457c79e3adadb50a48620926d4bc5b))
* **browser:** JSON shape paths now contain typed property and array-element segments. WebMCP input_schema_shape is replaced by input_schema and input_schema_sha256, retaining declared examples and defaults.
* **process:** Filesystem digest omission entries now require system_code; budget and changed-file omissions use null, and unavailable-file omissions preserve the operating-system error code.
* **contracts:** Provider analysis outputs must explicitly supply metadata previously repaired with defaults; unsupported nullable facets must be reported as null.
@@ -540,7 +540,7 @@ Full contribution record: [336 commits and 163 merged PRs](https://github.com/mo
### ⚠ BREAKING CHANGES
Read the [6.1 to 6.2 migration guide](docs/migrations/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
Read the [6.1 to 6.2 migration guide](docs/migration-6.2.md) before upgrading. REA's `always-bump-minor` policy includes breaking changes; 6.2 is not a backward-compatibility promise. Preserve original saved files and regenerate missing observations rather than fabricating identities or facts.
* **MCP / Evidence / snapshots / artifacts:** Success results are direct canonical Evidence; read `normalized_result` rather than the old `result`/`evidence` wrapper. Evidence requires nullable `analysis_profile`, session provider aliases are removed, snapshots require complete profile-bound query records, and artifact roles/permissions belong to occurrences. Recreate incompatible saved data; adding fields without recomputing their semantic identity is invalid. ([63bf914](https://github.com/morluto/rea/commit/63bf91405cf97b114a84f337fb1272d98a267999))
* **Process captures:** Legacy v3 conversion is removed. Explicit executable identity and `event_journal` are required; retain old captures and recapture when current comparisons are needed instead of inventing launch identity or event order. ([2a77469](https://github.com/morluto/rea/commit/2a7746998d47446c4db6d4f524e0f183d6803f3f))
+10 -14
View File
@@ -11,7 +11,7 @@ incorrect user instructions warrant standalone corrections. Collect other
cosmetic-only suggestions in an issue rather than separate PRs.
For selected architectural owners and dependency paths, use the
[architecture map](docs/architecture/architecture.mermaid). It is not a complete directory or
[architecture map](docs/architecture.mermaid). It is not a complete directory or
import inventory. For current provider wiring, start with
[`src/composition/binary.ts`](src/composition/binary.ts) for deep candidates and
the other `src/composition/` factories for tool services, including
@@ -32,7 +32,7 @@ removing it. Preserve validation, lazy loading, caller policy, and resource
acquisition and cleanup supplied by the old boundary. Published APIs, persisted
formats, and independently versioned bridges require an explicit compatibility
decision; an internal representation alone does not establish that obligation.
See [interpreting boundary facts once](docs/development/tool-design.md#interpret-boundary-facts-once)
See [interpreting boundary facts once](docs/tool-design.md#interpret-boundary-facts-once)
for admission ownership and the distinction between repeated validation and
independent checks.
@@ -44,7 +44,7 @@ Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/S
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
compare equivalent real tasks before claiming an improvement.
When adding or changing an MCP tool, follow the [tool design guide](docs/development/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
When adding or changing an admission ceiling, budget, or deadline, follow
[analysis settings, limits, and recovery](docs/design/limits-and-recovery.md)
for ownership, operation snapshots, provider integration, and the steps for
@@ -234,7 +234,7 @@ guarantee.
Use the development toolchain pinned in `.nvmrc` and
`package.json#packageManager`. Run `nvm use` before installing dependencies;
installed-package runtime support is documented in
[installation](docs/getting-started/installation.md#start-setup).
[installation](docs/installation.md#start-setup).
Use Node.js and npm for source builds and verifier scripts; the installed
package's Bun runtime support does not extend to those development scripts.
@@ -289,20 +289,16 @@ base revision, or run the suite on Linux or WSL, before attributing them to a
change. The embedded-artifact parent-swap tests exercise the selected-path
replacement when the rename succeeds; if Windows refuses it while the admitted
file is open, they instead verify the refusal and cleanup. See
[testing](docs/development/testing.md#developer-commands) for command scopes
[testing](docs/testing.md#developer-commands) for command scopes
and lane selection.
`git submodule update --init --recursive` fails at the pinned Unblob submodule
on Windows and can leave later submodules unchecked out. Apply the
[Unblob checkout workaround](docs/guides/firmware/firmware-analysis.md#windows-submodule-checkout),
[Unblob checkout workaround](docs/firmware-analysis.md#windows-submodule-checkout),
then rerun the command.
## Documentation website
Use the [documentation directory](docs/README.md) and
[authoring guide](docs/development/documentation.md) for document ownership,
domain-based feature specifications and architectural decision records.
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
for live editing, `npm run docs:build` to check the production build and
links, and `npm run docs:preview` to preview that build at `/rea/`.
@@ -356,7 +352,7 @@ npm run check:fast
`test:local` runs source tests without building. `check:changed` combines cached
static checks with source tests affected since the merge base with `origin/main`.
Import-graph selection can miss runtime registration, bridges, and generated data;
select relevant boundary tests explicitly. See [testing](docs/development/testing.md#developer-commands)
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
for commands, test placement, pruning, and real-provider prerequisites.
Before requesting review, run relevant tests and `npm run check`; add
@@ -431,8 +427,8 @@ Run `npm run verify:hopper` on a supported macOS host or
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
native fixtures and verify the installed engine through CLI/MCP; package or
simulated-provider checks do not establish engine behavior. See
[real-toolchain verification](docs/development/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/getting-started/installation.md#hopper)
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
for supported hosts and unattended startup.
Use the default `build/conformance/manifest.json` unless explicitly verifying
@@ -441,7 +437,7 @@ Generated fixtures and manifests remain ignored.
## Maintainer release checklist
Use the [release guide](docs/development/releasing.md). Main pushes automatically refresh a
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
Release Please PR with the next version and changelog. Review its migration
notes, wait for the current CI checks to pass, and merge it into main to publish.
+63 -72
View File
@@ -1,7 +1,7 @@
<div align="center">
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md) · [Български](docs/readme/README_bg.md)
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md) · [Roman Urdu](docs/readme/README_ur-Latn.md) · [Română](docs/readme/README_ro.md)
# REA: Reverse Engineer Anything
@@ -10,7 +10,7 @@
**See a feature you like. Understand how it works, down to the binary level.**
[![npm version](https://img.shields.io/npm/v/rea-agents?style=flat-square&color=cb3837)](https://www.npmjs.com/package/rea-agents)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/reference/mcp-contracts.md#generated-catalog)
[![MCP tool catalog](https://img.shields.io/badge/MCP-tool_catalog-5c4ee5?style=flat-square)](docs/mcp-contracts.md#generated-catalog)
[![Node.js requirements](https://img.shields.io/badge/Node.js-requirements-339933?style=flat-square&logo=nodedotjs&logoColor=white)](#what-you-can-analyze)
[![skills.sh](https://skills.sh/b/morluto/rea?style=flat-square)](https://skills.sh/morluto/rea/reverse-engineer-anything)
[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](LICENSE)
@@ -72,7 +72,7 @@ Setup registers REA with your agent and installs matching workflow instructions.
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
checks. Recovery steps are in `docs/getting-started/installation.md` under
checks. Recovery steps are in `docs/installation.md` under
"Linux owned-process prerequisite".
### Set up your agent
@@ -91,8 +91,8 @@ available. Run package acquisition from a trusted directory outside the target
project. Installation and recovery details are in the setup guide below.
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
[other agents](docs/getting-started/installation.md#supported-agents). See
[installation and setup](docs/getting-started/installation.md) for provider configuration and
[other agents](docs/installation.md#supported-agents). See
[installation and setup](docs/installation.md) for provider configuration and
manual MCP registration and Claude Code/Codex plugin installation.
MCP uses compact input schemas by default to reduce context overhead. Set
@@ -135,7 +135,7 @@ rea --help
```
For native analysis, configure a provider first. See the
[CLI and Evidence guide](docs/getting-started/cli.md) for native commands, provider selection,
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
snapshots and scripting.
### Update REA
@@ -191,47 +191,44 @@ CLI commands use the same workflows.
REA requires Node.js 22.x (>=22.23.4), 24.x (>=24.22.1), or 26.12.1+, plus npm.
Additional tools and host support depend on the target:
| Target | What REA returns | Requirements and guide |
| ----------------------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/guides/native/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/guides/native/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
| Windows shortcuts | Shell Link target, arguments, working dir, ExtraData and Tracker droid facts | Built-in static LNK inspection; [Windows shortcuts](docs/guides/native/windows-shortcut.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| Binary hardening | PIE, NX, RELRO, CFG and related header flags for ELF, PE and Mach-O | Built-in static header inspection; [binary hardening](docs/binary-hardening.md) |
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/guides/evm/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/guides/native/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/guides/javascript/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/guides/web/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/guides/web/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/guides/managed/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/guides/android/android-analysis.md) |
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/guides/android/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/providers/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/guides/android/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/guides/android/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/guides/firmware/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/guides/data/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/guides/data/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/guides/signals/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/guides/javascript/javascript-artifact-reconstruction.md), [Apple applications](docs/guides/apple/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/guides/runtime/process-capture.md) |
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
| Deno compile binaries | Virtual files, entry point, recorded options, npm snapshot length | None (static only); [Deno compile executables](docs/deno-standalone.md) |
| Target | What REA returns | Requirements and guide |
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra, IDA or Kuna; [native analysis](docs/native-investigation.md) |
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
| PE imports | Import and delay-load DLL names, ordinals, IAT RVAs and pefile import hash | Built-in static PE inspection; [PE imports](docs/pe-imports.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| Apple IPS crashes | .ips metadata, threads, images, registers and documented crash classes | Built-in static JSON inspection; [Apple IPS](docs/apple-ips.md) |
| Linux kernel modules | .modinfo keys, __versions CRCs, this_module presence, and signature trailer | Built-in static ELF inspection; [kernel modules](docs/kernel-modules.md) |
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
Static JavaScript and .NET inspection read the supplied files without running
the application. Runtime capture runs or interacts with the selected target
@@ -245,15 +242,15 @@ or update either provider.
<a id="choosing-a-deep-analysis-provider"></a>
Native formats and host support vary by provider. See
[Hopper and Ghidra setup](docs/getting-started/installation.md#hopper), the
[IDA guide](docs/providers/ida-provider.md) (including an optional native MCP bridge), the
[Hopper and Ghidra setup](docs/installation.md#hopper), the
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
[Kuna decompiler guide](docs/kuna-provider.md), and
[experimental Windows Ghidra support](docs/providers/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/providers/ghidra-dos.md).
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
For large binaries, raise the provider's startup deadline with
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
For provider selection, see the [CLI guide](docs/getting-started/cli.md#choose-a-provider).
Check [release availability](docs/getting-started/installation.md#released-package-and-main)
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
Check [release availability](docs/installation.md#released-package-and-main)
for features added since the latest npm release.
## Showcases
@@ -337,20 +334,14 @@ REA exports a runnable environment with a basic test to check that it starts and
[Environment reference](docs/agent-environments.md) ·
[Watch the demo on X](https://x.com/reatools/status/2112574822584918075?s=20)
### Desktop app environments
REA can also build an environment of a Linux desktop app, which an agent uses through screenshots, the mouse and the keyboard. From a one-line request, such as "Build me a Gnote environment where the agent adds olive oil to the shopping list note", your agent installs the app on a local build computer, writes the task, exports the environment and checks the task in a fresh computer. The machine that builds it needs Linux on amd64 or arm64, Docker and the agentenv framework version the guide pins.
[Set up desktop app environments](docs/desktop-build-computer.md)
## FAQ
<details>
<summary><strong>Which agents can use REA?</strong></summary>
Any agent that supports local MCP servers. Setup configures the
[supported agents](docs/getting-started/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/getting-started/installation.md#mcp-registry).
[supported agents](docs/installation.md#supported-agents); other clients can use
[manual MCP registration](docs/installation.md#mcp-registry).
</details>
@@ -360,7 +351,7 @@ Any agent that supports local MCP servers. Setup configures the
Deep native analysis uses Hopper, Ghidra, IDA or Kuna, depending on the target
and operation. Static JavaScript and .NET inspection work without a native
analysis engine. Setup can install Hopper after approval; Ghidra, IDA and Kuna
use your existing installations. See [provider setup](docs/getting-started/installation.md#hopper)
use your existing installations. See [provider setup](docs/installation.md#hopper)
for the options and their supported targets.
</details>
@@ -372,7 +363,7 @@ REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
ask you to choose demo mode or activate your license. Complete that prompt in the
same user session before your first CLI/MCP operation or unattended job. Open
Hopper manually if needed. See
[Hopper startup and troubleshooting](docs/getting-started/installation.md#launcher-paths-and-troubleshooting).
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
</details>
@@ -381,7 +372,7 @@ Hopper manually if needed. See
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
to register REA's MCP server and install the matching instructions, then restart
your agent. See [skill-only installation](docs/getting-started/installation.md#skill-only-installation).
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
</details>
@@ -430,12 +421,12 @@ with your REA version, target type, steps to reproduce and error output.
Start with the website's [worked guides](https://rea.tools/guides/).
For exact options, prerequisites and result contracts:
- [Installation and setup](docs/getting-started/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/getting-started/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/getting-started/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/reference/mcp-contracts.md) and [agent prompts](docs/getting-started/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/reference/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/development/roadmap.md): planned work and capability trackers.
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
Report vulnerabilities through [SECURITY.md](SECURITY.md).
@@ -470,8 +461,8 @@ report a bug or suggest a feature, or [send a pull request](https://github.com/m
to improve the code or docs.
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
[testing](docs/development/testing.md) for verification lanes, and the
[architecture map](docs/architecture/architecture.mermaid) for selected layers and dependency paths.
[testing](docs/testing.md) for verification lanes, and the
[architecture map](docs/architecture.mermaid) for selected layers and dependency paths.
## License
+27 -111
View File
@@ -4,117 +4,6 @@
Security fixes are provided for the latest published version of REA. Upgrade to the latest release before reporting a problem that may already have been resolved.
## Security boundary
REA is a local reverse-engineering and automation tool. CLI and MCP use the
same application workflows and run with the host process's permissions.
REA is not a general-purpose sandbox or a multi-tenant authorization service.
### Threat model
- **Trusted authority:** the operating-system account running REA, the caller
or agent host that selects operations, and configured analysis providers,
their executables and extensions, and service endpoints. An MCP client can
invoke the operations exposed to it; its host must enforce the user's tool
permissions and approvals.
- **Untrusted data:** target binaries, archives, source files, project contents,
captures, web pages and returned target content can be malicious. Selecting
an input for static inspection does not authorize commands embedded in that
input. Agents must treat target text and Evidence as data rather
than instructions granting additional authority.
- **Operation-specific effects:** static inspection parses target data and may
launch an analysis engine. Runtime, debugger, instrumentation, browser,
desktop and environment workflows can deliberately execute code, attach to
processes, make network requests, write files or interact with applications
and devices. Review the selected operation's effects and limitations; static
inspection does not authorize launching or interacting with the target
application. See
[runtime and network responsibilities](DISCLAIMER.md#runtime-network-and-device-operations).
Standard client registrations use a dedicated stdio process. Opt-in
[HTTP MCP](docs/reference/mcp-contracts.md#transports) uses plain HTTP without
TLS or client authentication and shares provider/active-target state across
protocol sessions. Its session ID is not a user-authentication or isolation
boundary. Keep this endpoint within a trusted local or isolated environment.
### Intended security properties
Static JavaScript and .NET inspection do not run the target application.
Unexpected execution of input code through a data-only analysis path is a
security concern. Analysis engines themselves are executable software trusted
with the host account's permissions; parsing an input in an external engine
does not isolate that engine from the host.
REA authenticates its managed Hopper and Ghidra bridges with per-session random
capability tokens transferred through private session files, rather than
process arguments or environment variables. POSIX uses current-user Unix
sockets. Windows Ghidra uses authenticated loopback TCP and a private NTFS
runtime whose DACL admits only the current user and SYSTEM. These controls
protect the managed session boundary, not every external provider or debugger
endpoint. See [Windows Ghidra transport and lifecycle](docs/providers/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Provider ownership and mutation rules remain specific to each workflow:
- Ghidra's working projects are temporary and owned by REA. Opt-in
[checkpoints](docs/getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
retain REA-owned Program state. A
[caller-selected existing project](docs/getting-started/installation.md#reuse-an-existing-ghidra-project)
is fingerprinted and copied into an owned runtime for read-only processing;
Ghidra does not open or save the original database through that workflow.
- [IDA analysis](docs/providers/ida-provider.md) is read-only. An attached GUI
database is never saved or closed by REA; owned headless sessions have a
separate lifecycle.
- Cleanup must stay within resources REA owns. Failed or unverified cleanup
can retain a workspace or session for retry; closing a session
does not imply that every investigation artifact has been erased.
`rea setup` shows planned configuration and installation changes and requires
approval before applying them. Ghidra, Java and IDA are user-managed. Approved
Linux Hopper installation authenticates the package's detached PGP signature
against the pinned publisher identity and checks the signed launcher against
the selected release's digest. The release feed's display version is separate
from that authenticated identity. See [installation and publisher verification](docs/getting-started/installation.md#hopper).
### Limits and caller responsibilities
- **Host isolation:** private runtimes and capability tokens do not protect
against malicious software already running as the same operating-system
user, or a compromised trusted client or provider. Use an appropriately
isolated account, VM or restricted environment when handling hostile targets.
- **Resource bounds:** timeouts, cancellation, streaming and size limits are
operation-specific. They are not a universal CPU, memory, disk or wall-time
limit on every target and third-party tool. Enforce hard host resource limits
externally when needed. Failures of a documented boundary or cancellation
control still warrant investigation.
- **Data and network exposure:** Evidence, caches, checkpoints, exports and
captures can retain private target data, paths, URLs and memory contents.
Local execution does not make these outputs safe to publish. Requested
network operations, agent hosts and provider features such as
[IDA Free's cloud decompiler](docs/providers/ida-provider.md) can communicate
outside the host. Review their data handling and secure any exposed endpoint.
- **Interpretation:** analysis, reconstruction and generated environments can
be incomplete or incorrect. Identity checks, signatures and successful
analysis do not establish that a target, provider or generated program is
safe to execute. Review the Evidence and proposed actions before relying on
them.
### Vulnerability scope
Please report suspected violations of these boundaries privately. Examples
include unintended code execution during data-only inspection, escaping a
documented extraction/output boundary, bypassing bridge authentication or
target/session identity checks, unauthorized effects on an unrelated resource,
and disclosure of transport credentials or explicitly marked sensitive values.
An attacker-triggered bypass of a documented resource or cancellation control
can also be security-relevant without code execution.
Incorrect analysis, unsupported formats, crashes, hangs and high resource use
without a security-boundary impact are generally ordinary bug reports. Effects
explicitly requested through an execution or interaction workflow are not by
themselves a vulnerability. A dependency advisory needs its actual REA exposure
assessed; a complete exploit or diagnosis is not required to report a concern.
If you are unsure whether a boundary is affected, use the private channel below.
## Reporting a vulnerability
Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
@@ -132,3 +21,30 @@ unrelated local paths from logs or examples.
The maintainer will acknowledge the report, assess its scope, and coordinate
remediation and disclosure timing with the reporter. Please allow time for
provider-dependent behavior to be reproduced safely.
## Security boundary
REA authenticates its Hopper and Ghidra bridges with per-session random
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
Tokens are transferred through private session descriptors rather than process
arguments or environment variables.
This is not a sandbox and does not protect against malicious processes already
running as the same operating-system user. Opening an untrusted binary
delegates parsing and analysis to the selected local provider with that user's
permissions.
Ghidra sessions use a packaged Java `HeadlessScript`, an isolated temporary
project, and private home, cache, and runtime directories. REA imports only the
requested target, bounds startup and protocol messages, requests bounded CPU
and heap settings, and deletes the temporary project when the session closes.
These controls limit accidental persistence and resource use; they do not make
Ghidra a sandbox. REA does not open or modify user-owned Ghidra projects.
`GHIDRA_INSTALL_DIR` and optional `JAVA_HOME` identify user-managed
installations. Setup can add those values to detected client configuration only
after showing its plan and receiving approval. REA does not download, install,
upgrade, or modify Ghidra or Java.
On Linux, REA accepts release metadata only from Hopper's HTTPS endpoint, restricts package URLs to Hopper's public download origin, bounds the package size, and compares the downloaded bytes with Hopper's published checksum before requesting installation. The published SHA-1 value is a corruption check, not a modern package signature; HTTPS origin validation remains part of the trust boundary. Dependency installation is delegated without shell evaluation to `apt-get`, `dnf`, or `pacman`, directly as root or through `pkexec`. REA never invokes `sudo`.
+21 -21
View File
@@ -54,27 +54,27 @@ REA's adapters do not grant, replace, or waive these licenses. The upstream
executables and their dependency graphs are supplied separately. Git submodules
pin inspection sources; they are not included in the npm package.
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/getting-started/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
| Project | License and source | REA relationship |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra; see [supported versions](docs/installation.md#ghidra). Copies of the 12.1.4 [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.990/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.990/NOTICE) | Bring your own `kuna`; REA verified 1.990 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
Ghidra's original notice credits software developed at the National Security
Agency. It explains that US-government-created portions are not subject to US
-516
View File
@@ -1,516 +0,0 @@
"""Run REA inside an agentenv build computer for REA on the host: capture the app, draft its task and export it.
REA sends this file's source through the computer's ``computer-exec/v1`` route as
``python3 -I -c <source> <command> <JSON arguments>``. JSON that would exceed the bytes Linux
passes as one string travels as several arguments, which this file joins before parsing. Each command prints
one JSON line and exits. ``start``
leaves the capture running on its own: its stdin is a FIFO it holds open for reading and writing, so it never
sees the end of its input, and ``mark`` and ``stop`` write their requests there and wait for the capture's own
acknowledgement on its progress log. A small shell around the capture writes its exit status to a file, the
mark a later command reads. REA runs as the session user in the home, with no ``AGENTENV_*`` variable, so
neither REA nor the app it launches holds the computer's control token, and with ``APT_CONFIG`` naming the
package lists ``start`` downloads as that user, so the capture finds the package's sources.
"""
from __future__ import annotations
import hashlib
import json
import os
import re
import shutil
import subprocess
import sys
import time
POLL_S = 0.2
TAIL_BYTES = 4096
def _say(value: dict) -> int:
print(json.dumps(value), flush=True)
return 0
def _tail(path: str) -> str:
try:
with open(path, "rb") as handle:
handle.seek(max(0, os.path.getsize(path) - TAIL_BYTES))
return handle.read().decode("utf-8", "replace")
except OSError:
return ""
def _environment(work: str) -> dict:
environment = {name: value for name, value in os.environ.items() if not name.startswith("AGENTENV_")}
environment["APT_CONFIG"] = os.path.join(work, "apt", "apt.conf")
return environment
def _apt(work: str, timeout: float) -> dict:
"""Download the package lists into the work folder as the session user, until one update succeeds."""
root = os.path.join(work, "apt")
lists = os.path.join(root, "lists")
updated = os.path.join(root, "updated")
os.makedirs(os.path.join(lists, "partial"), exist_ok=True)
os.makedirs(os.path.join(root, "cache", "archives", "partial"), exist_ok=True)
with open(os.path.join(root, "apt.conf"), "w", encoding="utf-8") as conf:
conf.write(
f'Dir::State::Lists "{lists}";\n'
f'Dir::Cache "{root}/cache";\n'
'Debug::NoLocking "true";\n'
'APT::Update::Error-Mode "any";\n'
)
# apt keeps lists compressed as it fetched them, such as *_Packages.lz4 on Ubuntu.
count = lambda: len([name for name in os.listdir(lists) if re.search(r"_Packages(\.\w+)?$", name)]) # noqa: E731
# A marker left by an update that fetched nothing is not a success; download again.
if os.path.exists(updated) and count() > 0:
return {"lists": count(), "updated": False}
log = os.path.join(root, "update.log")
try:
with open(log, "wb") as output:
code = subprocess.run(["apt-get", "update"], env=_environment(work), stdin=subprocess.DEVNULL,
stdout=output, stderr=subprocess.STDOUT, timeout=timeout).returncode
except (OSError, subprocess.TimeoutExpired) as error:
return {"lists": count(), "updated": True, "error": f"{type(error).__name__}: {error}"}
lists_fetched = count()
if code == 0 and lists_fetched > 0:
# Only a whole update that fetched lists is kept, so a failed or empty one runs again.
open(updated, "w").close()
return {"lists": lists_fetched, "updated": True, "exit_code": code}
if code == 0:
return {"lists": lists_fetched, "updated": True, "exit_code": code,
"error": "apt-get update exited 0 without a package list"}
return {"lists": lists_fetched, "updated": True, "exit_code": code,
"error": f"apt-get update exited {code}: {_tail(log)[-1500:].strip()}"}
def _exit_code(capture: str):
try:
with open(os.path.join(capture, "exit"), encoding="utf-8") as handle:
return int(handle.read().strip() or "-1")
except (OSError, ValueError):
return None
def _ended(capture: str, code: int) -> dict:
"""A capture that has exited: its status, the end of its progress log and its output's size, which REA reads
whole for the Evidence or failure it ended with, whichever step was waiting on it."""
return {"exited": code, "progress": _tail(os.path.join(capture, "progress.log")),
"output_bytes": os.path.getsize(os.path.join(capture, "capture.out"))}
def _event_line(path: str, offset: int, kinds: set, name):
"""The first progress line after ``offset`` whose event is of one of ``kinds``, for ``name`` when given."""
try:
with open(path, "rb") as handle:
handle.seek(offset)
lines = handle.read().decode("utf-8", "replace").splitlines()
except OSError:
return None
for line in lines:
try:
event = json.loads(line)["rea_progress"]["event"]
except (ValueError, KeyError, TypeError):
continue
if isinstance(event, dict) and event.get("kind") in kinds and (name is None or event.get("name") == name):
return line
return None
def _wait(capture: str, offset: int, kinds: set, name, timeout: float) -> dict:
"""Wait for the capture's event, or for it to exit, or for time to run out."""
log = os.path.join(capture, "progress.log")
deadline = time.monotonic() + timeout
while True:
line = _event_line(log, offset, kinds, name)
if line is not None:
return {"line": line}
code = _exit_code(capture)
if code is not None:
return _ended(capture, code)
if time.monotonic() > deadline:
return {"timed_out": True, "progress": _tail(log)}
time.sleep(POLL_S)
def _within(path: str, folder: str) -> bool:
return path == folder or path.startswith(folder + os.sep)
def _state_paths(values: list, work: str):
"""Each state path as an absolute path strictly under the home and outside REA's work folder, both as named
and with its links resolved, or the refusal of the first that is not. The capture resolves the links again
and retains what they lead to, so a link out of the home is refused here. Each passes on as named, so the
capture keeps what its spelling means, such as a hidden settings folder."""
home = os.path.normpath(os.path.expanduser("~"))
real_home, real_work = os.path.realpath(home), os.path.realpath(work)
spellings = {home, real_home}
paths = []
for value in values:
relative = value[2:] if value.startswith("~/") else value
path = os.path.normpath(os.path.join(home, relative))
if ".." in relative.split("/") or not any(path.startswith(spelling + os.sep) for spelling in spellings):
return None, f"The state path {value} is not under the computer's home, {home}."
# Every link on the way is followed, a dangling one too; a path that does not exist yet resolves
# through its nearest existing folder.
resolved = os.path.realpath(path)
if not resolved.startswith(real_home + os.sep):
return None, f"The state path {value} leads through a link to {resolved}, which is not under the computer's home, {home}."
if _within(path, work) or _within(resolved, real_work):
return None, f"The state path {value} is in REA's work folder {work}, which every capture leaves out."
paths.append(path)
return paths, None
def start(arguments: dict) -> int:
work, capture = os.path.abspath(arguments["work"]), os.path.abspath(arguments["capture"])
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer; start the computer from a request, which installs it."})
state_paths, refusal = _state_paths(arguments.get("state_paths", []), work)
if refusal is not None:
return _say({"refused": refusal})
if os.path.exists(capture):
return _say({"error": f"the capture folder {capture} is already used"})
apt = _apt(work, arguments.get("apt_timeout_s", 600))
try:
os.makedirs(capture)
except FileExistsError:
return _say({"error": f"a stop claimed {capture} before the capture launched", "apt": apt})
control = os.path.join(capture, "control")
os.mkfifo(control, 0o600)
argv = [rea, "capture-desktop-application", "--desktop-entry", arguments["desktop_entry"],
"--output", os.path.join(capture, "capture"), "--exclude", work, "--json"]
for value in arguments.get("arguments", []):
argv += ["--argument", value]
for path in state_paths:
argv += ["--state-path", path]
if arguments.get("package_file"):
argv += ["--package-file", arguments["package_file"]]
# Reading and writing, so the capture never reads the end of its input between requests.
reader = os.open(control, os.O_RDWR)
process = None
try:
try:
with open(os.path.join(capture, "capture.out"), "wb") as out, \
open(os.path.join(capture, "progress.log"), "wb") as progress:
process = subprocess.Popen(
["sh", "-c", 'done=$1; shift; "$@"; code=$?; printf "%s" "$code" > "$done.tmp" && mv "$done.tmp" "$done"',
"sh", os.path.join(capture, "exit"), *argv],
cwd=os.path.expanduser("~"), env=_environment(work), stdin=reader, stdout=out, stderr=progress,
start_new_session=True)
except Exception as cause:
# Nothing will write exit. Record one and drop the FIFO, so stop does not wait on it.
try:
with open(os.path.join(capture, "exit"), "w", encoding="utf-8") as handle:
handle.write("1")
except OSError:
pass
try:
os.unlink(control)
except OSError:
pass
return _say({"error": f"the capture did not start: {cause}"})
finally:
os.close(reader)
if process is None:
return _say({"error": "the capture did not start"})
waited = _wait(capture, 0, {"window_open"}, None, arguments.get("timeout_s", 120))
return _say({"pid": process.pid, "apt": apt, **waited})
def _request(capture: str, line: str) -> bool:
"""Write one request to the running capture; False when nothing reads its FIFO any more."""
try:
fd = os.open(os.path.join(capture, "control"), os.O_WRONLY | os.O_NONBLOCK)
except OSError:
return False
try:
os.write(fd, (line + "\n").encode("utf-8"))
finally:
os.close(fd)
return True
def mark(arguments: dict) -> int:
"""Request a checkpoint and answer with the capture's acknowledgement line as it wrote it: the checkpoint's
counts and up to ten of the files that changed, home-relative, with a count of the rest."""
capture, name = os.path.abspath(arguments["capture"]), arguments["name"]
code = _exit_code(capture)
log = os.path.join(capture, "progress.log")
if code is not None:
return _say(_ended(capture, code))
# A checkpoint already taken, whose answer was lost, is answered from the capture's own log.
taken = _event_line(log, 0, {"checkpoint"}, name)
if taken is not None:
return _say({"line": taken})
offset = os.path.getsize(log)
if not _request(capture, f"checkpoint {name}"):
return _say({"error": "the capture is not reading its requests"})
return _say(_wait(capture, offset, {"checkpoint", "checkpoint_refused"}, name, arguments.get("timeout_s", 600)))
def stop(arguments: dict) -> int:
capture = os.path.abspath(arguments["capture"])
if _exit_code(capture) is None and not os.path.exists(os.path.join(capture, "control")):
# Never launched: claim the folder, so a start still on its way finds it used and launches nothing.
os.makedirs(capture, exist_ok=True)
return _say({"absent": True})
if _exit_code(capture) is None:
_request(capture, "stop")
deadline = time.monotonic() + arguments.get("timeout_s", 600)
while (code := _exit_code(capture)) is None and time.monotonic() < deadline:
time.sleep(POLL_S)
log = os.path.join(capture, "progress.log")
if code is None:
return _say({"timed_out": True, "progress": _tail(log)})
output = os.path.join(capture, "capture.out")
return _say({"exit_code": code, "progress": _tail(log), "output_bytes": os.path.getsize(output)})
def run(arguments: dict) -> int:
"""Run one REA command with its output in a file, after writing the files it reads."""
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer"})
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
for path, text in arguments.get("files", {}).items():
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
handle.write(text)
with open(arguments["stdout"], "wb") as out:
try:
done = subprocess.run([rea, *arguments["argv"]], cwd=os.path.expanduser("~"),
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL, stdout=out,
stderr=subprocess.PIPE, timeout=arguments.get("timeout_s", 1800))
except subprocess.TimeoutExpired:
return _say({"timed_out": True})
return _say({"exit_code": done.returncode, "stdout_bytes": os.path.getsize(arguments["stdout"]),
"stderr": done.stderr[-TAIL_BYTES:].decode("utf-8", "replace")})
# The controller is sent with python -c, so a detached supervisor needs its
# own source. It alone holds the child process handle and signals its group.
LONG_RUN = r'''
import json, os, signal, subprocess, sys, threading, time
job = sys.argv[1]
result = {}
child = None
drainer = None
stderr_tail, stderr_errors = bytearray(), []
def drain(stream):
try:
while chunk := stream.read1(65536):
stderr_tail.extend(chunk)
del stderr_tail[:-4096]
except OSError as error:
stderr_errors.append(str(error))
finally:
try:
stream.close()
except OSError as error:
stderr_errors.append(str(error))
def timed_out_marker():
try:
open(os.path.join(job, "timed_out"), "x").close()
except FileExistsError:
pass
try:
request = json.loads("".join(sys.argv[2:]))
if os.path.exists(os.path.join(job, "cancel")):
result = {"absent": True}
else:
with open(request["stdout"], "wb") as out:
child = subprocess.Popen(request["argv"], cwd=request["home"], stdin=subprocess.DEVNULL,
stdout=out, stderr=subprocess.PIPE, start_new_session=True)
drainer = threading.Thread(target=drain, args=(child.stderr,), daemon=True)
drainer.start()
deadline = time.monotonic() + request["timeout_s"]
interrupted = False
timed_out = False
while child.poll() is None:
timed_out = timed_out or time.monotonic() >= deadline
if not interrupted and (timed_out or os.path.exists(os.path.join(job, "cancel"))):
interrupted = True
if timed_out:
timed_out_marker()
try:
os.killpg(child.pid, signal.SIGINT)
except OSError:
pass
time.sleep(0.2)
# A descendant can still hold stderr after the leader exits. Do
# not publish completion until that stream has actually closed.
while drainer.is_alive():
drainer.join(0.2)
if time.monotonic() >= deadline:
timed_out = True
timed_out_marker()
result = {"exit_code": child.returncode, "stdout_bytes": os.path.getsize(request["stdout"]),
"timed_out": timed_out, "cancelled": interrupted and not timed_out,
"stderr": bytes(stderr_tail).decode("utf-8", "replace")}
if stderr_errors:
result["error"] = f"The REA command's stderr could not be read: {stderr_errors[0]}"
except Exception as error:
result["error"] = f"The REA command supervisor failed: {error}"
# Keep the child handle until it exits, including when cancellation arrives
# after an unexpected supervisor error.
while child is not None and child.poll() is None:
if os.path.exists(os.path.join(job, "cancel")):
try:
os.killpg(child.pid, signal.SIGINT)
except OSError:
pass
time.sleep(0.2)
if drainer is not None and drainer.ident is not None:
drainer.join()
if child is not None:
result.setdefault("exit_code", child.returncode)
result.setdefault("stderr", bytes(stderr_tail).decode("utf-8", "replace"))
try:
result.setdefault("stdout_bytes", os.path.getsize(request["stdout"]))
except OSError:
pass
if child is None or child.poll() is not None:
temporary = os.path.join(job, "result.tmp")
with open(temporary, "w", encoding="utf-8") as out:
json.dump(result, out)
os.replace(temporary, os.path.join(job, "result.json"))
'''
def run_begin(arguments: dict) -> int:
"""Start one long REA command in its own capture-owned job folder."""
rea = shutil.which("rea")
if rea is None:
return _say({"error": "REA is not installed in this computer"})
job = os.path.abspath(arguments["job"])
os.makedirs(os.path.dirname(job), exist_ok=True)
try:
os.mkdir(job, 0o700)
except FileExistsError:
return _say({"refused": "the command job folder is already used"})
process = None
try:
with open(os.path.join(job, "owner"), "x", encoding="ascii") as out:
out.write(arguments["token"])
for path, text in arguments.get("files", {}).items():
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
handle.write(text)
os.makedirs(os.path.dirname(arguments["stdout"]), exist_ok=True)
request = json.dumps({"argv": [rea, *arguments["argv"]], "stdout": arguments["stdout"],
"home": os.path.expanduser("~"), "timeout_s": arguments["timeout_s"]})
chunks = [request[i:i + 60000] for i in range(0, len(request), 60000)]
with open(os.path.join(job, "supervisor.log"), "wb") as log:
process = subprocess.Popen([sys.executable, "-I", "-c", LONG_RUN, job, *chunks], cwd=os.path.expanduser("~"),
env=_environment(os.path.abspath(arguments["work"])), stdin=subprocess.DEVNULL,
stdout=log, stderr=log, start_new_session=True)
except (OSError, ValueError) as error:
stage = "did not start" if process is None else "started but its setup did not finish"
result = {"error": f"The REA command supervisor {stage}: {error}"}
if process is None:
temporary = os.path.join(job, "result.tmp")
with open(temporary, "w", encoding="utf-8") as out:
json.dump(result, out)
os.replace(temporary, os.path.join(job, "result.json"))
return _say(result)
return _say({"started": True})
def _run_result(job: str):
try:
with open(os.path.join(job, "result.json"), encoding="utf-8") as source:
return json.load(source)
except FileNotFoundError:
return None
def _owns_run(job: str, token: str) -> bool:
try:
with open(os.path.join(job, "owner"), encoding="ascii") as owner:
return owner.read() == token
except OSError:
return False
def run_poll(arguments: dict) -> int:
job = os.path.abspath(arguments["job"])
if not _owns_run(job, arguments["token"]):
return _say({"error": "the command job ownership is not confirmed"})
deadline = time.monotonic() + arguments.get("timeout_s", 30)
while (result := _run_result(job)) is None and time.monotonic() < deadline:
time.sleep(POLL_S)
if result is None:
pending = {"pending": True, "timed_out": os.path.exists(os.path.join(job, "timed_out"))}
progress = _tail(os.path.join(job, "supervisor.log"))
return _say({**pending, **({"progress": progress} if progress else {})})
return _say({"finished": True, **result})
def run_cancel(arguments: dict) -> int:
job = os.path.abspath(arguments["job"])
os.makedirs(os.path.dirname(job), exist_ok=True)
try:
os.mkdir(job, 0o700)
except FileExistsError:
pass
else:
# A late begin cannot launch into a folder cancellation claimed.
with open(os.path.join(job, "owner"), "x", encoding="ascii") as owner:
owner.write(arguments["token"])
return _say({"absent": True})
if not _owns_run(job, arguments["token"]):
return _say({"error": "the command job ownership is not confirmed"})
with open(os.path.join(job, "cancel"), "a", encoding="utf-8"):
pass
return run_poll(arguments)
def listing(arguments: dict) -> int:
"""Every regular file under a folder, by its path relative to the home, with its size and SHA-256.
The list goes to the file ``output`` names, outside the folder, and the answer gives its size: a large
bundle's list outgrows the 1 MiB of stdout a command answers with, so REA reads it through ``data/get``.
"""
home = os.path.expanduser("~")
files, skipped = [], []
def fail_walk(error: OSError) -> None:
raise error
for directory, folders, names in os.walk(os.path.abspath(arguments["root"]), onerror=fail_walk):
folders.sort()
# os.walk does not follow directory symlinks; disclose them to the consumer.
for name in folders[:]:
path = os.path.join(directory, name)
if os.path.islink(path):
skipped.append(os.path.relpath(path, home))
folders.remove(name)
for name in sorted(names):
path = os.path.join(directory, name)
relative = os.path.relpath(path, home)
if os.path.islink(path) or not os.path.isfile(path):
skipped.append(relative)
continue
digest = hashlib.sha256()
with open(path, "rb") as handle:
for chunk in iter(lambda: handle.read(1 << 20), b""):
digest.update(chunk)
files.append({"path": relative, "size": os.path.getsize(path), "sha256": digest.hexdigest()})
output = os.path.abspath(arguments["output"])
os.makedirs(os.path.dirname(output), exist_ok=True)
with open(output, "w", encoding="utf-8") as handle:
json.dump({"files": files, "skipped": skipped}, handle)
return _say({"listing_bytes": os.path.getsize(output)})
COMMANDS = {"start": start, "mark": mark, "stop": stop, "run": run, "list": listing,
"run-begin": run_begin, "run-poll": run_poll, "run-cancel": run_cancel}
if __name__ == "__main__":
sys.exit(COMMANDS[sys.argv[1]](json.loads("".join(sys.argv[2:]))))
-83
View File
@@ -1,83 +0,0 @@
"""Keep one agentenv computer up for REA until stdin closes or SIGTERM arrives.
REA runs ``computer.py <bundle folder> [task]`` with the agentenv runner's own interpreter, so ``agent_env``
imports. It deploys the bundle's task, ``smoke`` by default, on this machine with
``run_bundle(keep=True, emulate=False)``, prints one JSON line naming the computer's address, MCP URL, control
token and sandbox id, or ``{"error": ...}``, and tears the computer down once its stdin closes or it gets SIGTERM
or SIGINT. Everything else it or agentenv prints, a build's output included, goes to stderr. It exits 0 once
the teardown left nothing up, 1 when it did, and 2 when no computer came up.
"""
from __future__ import annotations
import json
import os
import signal
import sys
import threading
def _protocol():
"""A private copy of stdout for the JSON line; fd 1 then points at stderr, as do children that inherit it."""
protocol = os.fdopen(os.dup(1), "w", encoding="utf-8")
os.dup2(2, 1)
sys.stdout = sys.stderr
return protocol
def _say(protocol, value: dict) -> None:
protocol.write(json.dumps(value) + "\n")
protocol.flush()
def _computer(result):
"""The run's deployed computer, or why there is none."""
run = result.runs[0] if result.runs else None
if run is None:
return None, "the bundle ran no task"
if run.error is not None or run.failed_steps:
return None, f"the task failed: {run.error or run.failed_steps}"
for env in run.context.deployed_envs:
if env.environment_url and env.mcp_url and getattr(env, "sandbox_id", None):
return env, None
return None, "the task deployed no computer"
def main(argv: list[str]) -> int:
protocol = _protocol()
bundle, task = argv[1], argv[2] if len(argv) > 2 else "smoke"
try:
from agent_env.bundle import run_bundle
except ImportError as error:
_say(protocol, {"error": f"agent_env does not import with {sys.executable}: {error}"})
return 2
try:
result = run_bundle(bundle, tasks=[task], keep=True, emulate=False, sandbox="local",
on_progress=lambda line: print(line, file=sys.stderr, flush=True))
except BaseException as error: # noqa: BLE001 - a signal's RunInterrupted too; run_bundle tore its runs down
_say(protocol, {"error": f"{type(error).__name__}: {error}"})
return 2
stop = threading.Event()
for signum in (signal.SIGTERM, signal.SIGINT):
signal.signal(signum, lambda *_: stop.set())
env, problem = _computer(result)
try:
if env is None:
_say(protocol, {"error": problem})
else:
_say(protocol, {"base_url": env.environment_url, "mcp_url": env.mcp_url,
"control_token": env.control_token, "sandbox_id": env.sandbox_id})
threading.Thread(target=lambda: (sys.stdin.buffer.read(), stop.set()), daemon=True).start()
while not stop.wait(0.5):
pass
finally:
torn_down = result.teardown()
still_up = [sandbox for run in torn_down.runs if run.torn_down is not None for sandbox in run.torn_down.still_up]
if still_up:
print(f"still up after the teardown: {still_up}", file=sys.stderr, flush=True)
return 1
return 0 if env is not None else 2
if __name__ == "__main__":
sys.exit(main(sys.argv))
-232
View File
@@ -1,232 +0,0 @@
"""Exercise the capture controller's filesystem inventory boundary."""
import json
import io
from contextlib import redirect_stdout
import os
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import patch
from bridge.agentenv import capture
CONTROLLER = Path(__file__).with_name("capture.py")
class CaptureListingTest(unittest.TestCase):
def run_listing(self, root, output):
return subprocess.run(
[
sys.executable,
str(CONTROLLER),
"list",
json.dumps({"root": str(root), "output": str(output)}),
],
text=True,
capture_output=True,
check=False,
)
def test_directory_symlink_is_disclosed_without_following_it(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
root = parent / "root"
target = parent / "target"
output = parent / "listing.json"
root.mkdir()
target.mkdir()
(root / "present.txt").write_text("present")
(target / "outside.txt").write_text("outside")
try:
(root / "linked").symlink_to(target, target_is_directory=True)
except OSError as error:
self.skipTest(f"directory symlinks unavailable: {error}")
result = self.run_listing(root, output)
self.assertEqual(result.returncode, 0, result.stderr)
listing = json.loads(output.read_text())
self.assertEqual(
[item["path"] for item in listing["files"]],
[os.path.relpath(root / "present.txt", Path.home())],
)
self.assertEqual(
listing["skipped"], [os.path.relpath(root / "linked", Path.home())]
)
def test_missing_root_does_not_produce_an_empty_successful_inventory(self):
with tempfile.TemporaryDirectory() as parent:
parent = Path(parent)
output = parent / "listing.json"
result = self.run_listing(parent / "missing", output)
self.assertNotEqual(result.returncode, 0)
self.assertFalse(output.exists())
@unittest.skipIf(sys.platform == "win32", "desktop build computers use Linux")
class LongRunTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.root = Path(self.tmp.name)
self.bin = self.root / "bin"
self.bin.mkdir()
rea = self.bin / "rea"
rea.write_text(
f"#!{sys.executable}\n"
"import os, signal, subprocess, sys, time\n"
"signal.signal(signal.SIGINT, lambda *_: sys.exit(130))\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'wait':\n"
" open(sys.argv[2], 'w').close()\n"
" time.sleep(30)\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'ignore':\n"
" signal.signal(signal.SIGINT, signal.SIG_IGN)\n"
" open(sys.argv[2], 'w').close()\n"
" while not os.path.exists(sys.argv[3]): time.sleep(0.02)\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'stderr':\n"
" sys.stderr.write('x' * 100000 + 'TAIL')\n"
"if len(sys.argv) > 1 and sys.argv[1] == 'hold':\n"
" subprocess.Popen([sys.executable, '-c', 'import os,sys,time\\nwhile not os.path.exists(sys.argv[1]): time.sleep(0.02)', sys.argv[2]], stdout=subprocess.DEVNULL, stderr=sys.stderr, start_new_session=True)\n"
" sys.exit(0)\n"
"print(os.environ.get('AGENTENV_CONTROL_TOKEN', 'no token'))\n"
)
rea.chmod(0o755)
self.environment = {**os.environ, "HOME": str(self.root), "PATH": f"{self.bin}{os.pathsep}{os.environ.get('PATH', '')}",
"AGENTENV_CONTROL_TOKEN": "not-for-the-child"}
self.job = self.root / "job"
self.output = self.root / "out.json"
def command(self, name, **values):
result = subprocess.run([sys.executable, str(CONTROLLER), name, json.dumps(values)],
env=self.environment, text=True, capture_output=True, timeout=10, check=True)
return json.loads(result.stdout)
def begin(self, argv, timeout=60):
return self.command("run-begin", job=str(self.job), work=str(self.root), stdout=str(self.output),
token="owner-token", argv=argv, files={}, timeout_s=timeout)
def test_runs_and_reports_its_output(self):
self.assertEqual(self.begin([]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(result["exit_code"], 0)
self.assertEqual(self.output.read_text().strip(), "no token")
def test_passes_large_command_without_persisting_its_arguments(self):
self.assertEqual(self.begin(["done", *("x" * 4000 for _ in range(20))]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(result["exit_code"], 0)
self.assertFalse((self.job / "request.json").exists())
def test_keeps_only_a_bounded_stderr_tail(self):
self.assertEqual(self.begin(["stderr"]), {"started": True})
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertEqual(len(result["stderr"]), 4096)
self.assertTrue(result["stderr"].endswith("TAIL"))
def test_cancels_a_running_child_and_confirms_exit(self):
ready = self.root / "ready"
self.assertEqual(self.begin(["wait", str(ready)]), {"started": True})
import time
deadline = time.monotonic() + 5
while not ready.exists() and time.monotonic() < deadline:
time.sleep(0.02)
self.assertTrue(ready.exists())
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["cancelled"])
self.assertEqual(result["exit_code"], 130)
def test_cancel_before_late_start_claims_the_job(self):
self.assertEqual(self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1), {"absent": True})
self.assertIn("already used", self.begin([])["refused"])
self.assertFalse(self.output.exists())
def test_cancel_refuses_a_folder_without_the_matching_owner(self):
self.job.mkdir()
(self.job / "owner").write_text("someone-else")
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=1)
self.assertIn("ownership is not confirmed", result["error"])
self.assertFalse((self.job / "cancel").exists())
def test_reports_supervisor_output_if_completion_is_unconfirmed(self):
self.job.mkdir()
(self.job / "owner").write_text("owner-token")
(self.job / "supervisor.log").write_text("supervisor failed")
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=0)
self.assertEqual(result["progress"], "supervisor failed")
self.assertTrue(result["pending"])
def test_reports_a_timed_out_child_that_has_not_stopped(self):
ready, release = self.root / "ready", self.root / "release"
self.assertEqual(self.begin(["ignore", str(ready), str(release)], timeout=3), {"started": True})
try:
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
self.assertTrue(result["pending"])
self.assertTrue(result["timed_out"])
self.assertTrue(ready.exists())
finally:
release.touch()
self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
def test_waits_for_an_inherited_stderr_pipe_after_the_child_exits(self):
release = self.root / "release"
self.assertEqual(self.begin(["hold", str(release)], timeout=3), {"started": True})
try:
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=4)
self.assertTrue(result["pending"])
self.assertTrue(result["timed_out"])
finally:
release.touch()
result = self.command("run-poll", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["finished"])
def test_failure_closing_supervisor_log_after_launch_does_not_claim_exit(self):
ready = self.root / "ready"
original_open = open
original_popen = subprocess.Popen
supervisors = []
class FailingClose:
def __init__(self, file):
self.file = file
def __enter__(self):
return self.file
def __exit__(self, *args):
self.file.close()
raise OSError("log close failed")
def opening(path, *args, **kwargs):
file = original_open(path, *args, **kwargs)
return FailingClose(file) if str(path).endswith("supervisor.log") else file
def spawning(*args, **kwargs):
process = original_popen(*args, **kwargs)
supervisors.append(process)
def cleanup():
if process.poll() is None:
(self.job / "cancel").touch()
process.wait(timeout=5)
self.addCleanup(cleanup)
return process
arguments = {"job": str(self.job), "work": str(self.root), "stdout": str(self.output),
"token": "owner-token", "argv": ["wait", str(ready)], "files": {}, "timeout_s": 60}
with patch.dict(os.environ, self.environment, clear=True), patch("builtins.open", side_effect=opening), patch.object(capture.subprocess, "Popen", side_effect=spawning), redirect_stdout(io.StringIO()) as output:
capture.run_begin(arguments)
self.assertIn("started but its setup did not finish", json.loads(output.getvalue())["error"])
import time
deadline = time.monotonic() + 5
while not ready.exists() and time.monotonic() < deadline:
time.sleep(0.02)
self.assertTrue(ready.exists())
self.assertFalse((self.job / "result.json").exists())
result = self.command("run-cancel", job=str(self.job), token="owner-token", timeout_s=5)
self.assertTrue(result["cancelled"])
supervisors[0].wait(timeout=5)
if __name__ == "__main__":
unittest.main()
+76 -350
View File
@@ -119,9 +119,9 @@ import ghidra.program.util.VarnodeContext;
public final class ReaGhidraBridge extends HeadlessScript {
// At most 24 KiB after worst-case JSON escaping (six bytes per UTF-16 unit).
private static final int PDB_ATTRIBUTE_UTF16_BUDGET = 4096;
private static final int DEFAULT_HIGH_PCODE_OPS = 3000;
private static final int DEFAULT_HIGH_PCODE_INPUTS_PER_OP = 64;
private static final int DEFAULT_HIGH_PCODE_DEF_USE_EDGES = 12000;
private static final int MAX_HIGH_PCODE_OPS = 3000;
private static final int MAX_HIGH_PCODE_INPUTS_PER_OP = 64;
private static final int MAX_HIGH_PCODE_DEF_USE_EDGES = 12000;
private static final Gson GSON = new GsonBuilder().serializeNulls().create();
private final Map<FileBytes, Boolean> targetSourceIdentity = new IdentityHashMap<>();
private String admittedTargetSha256;
@@ -179,21 +179,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
private List<FunctionEntry> procedureInventory;
private List<InventoryItem> stringInventory;
private DecompInterface decompiler;
private boolean existingProject;
// A refused Program may only report its admission failure to the client.
private String identityAdmissionFailure;
/** Per-function native decompiler limit from the session; 0 leaves it unbounded. */
private int decompileTimeoutSeconds;
private Integer decompileMaxPayloadMiB;
// DecompileProcess converts MiB to bytes using a signed-int left shift.
private static final int MAX_DECOMPILE_PAYLOAD_MIB = Integer.MAX_VALUE >> 20;
private ValueFlowLimits valueFlowLimits = defaultValueFlowLimits();
// DecompileProcess converts seconds to an int millisecond timer delay.
private static final int MAX_DECOMPILE_TIMEOUT_SECONDS = Integer.MAX_VALUE / 1000;
private final ReaGhidraExtensions analysisExtensions = new ReaGhidraExtensions();
private static AddressSpace sessionDefaultAddressSpace;
private Path checkpointDirectory;
private long checkpointModification;
@Override
public void run() throws Exception {
@@ -202,9 +193,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
throw new IllegalArgumentException("REA bridge requires one session descriptor");
}
Path descriptorPath = Path.of(arguments[0]);
checkpointDirectory = descriptorPath.toAbsolutePath().normalize().getParent();
SessionDescriptor descriptor = readDescriptor(descriptorPath);
existingProject = descriptor.existingProject;
decompileTimeoutSeconds = descriptor.decompileTimeoutSeconds();
// Windows retains a native immutable bearer descriptor lease until
// runtime_close; POSIX removes it after this read. Normal cleanup removes
@@ -235,17 +224,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
throw new IllegalStateException("Ghidra provider version does not match the session");
}
String importedSha256 = currentProgram.getExecutableSHA256();
if (importedSha256 == null || !importedSha256.matches("(?i)[0-9a-f]{64}")) {
identityAdmissionFailure = "import_identity_unavailable";
}
else if (!constantTimeEquals(importedSha256.toLowerCase(Locale.ROOT),
descriptor.targetSha256)) {
identityAdmissionFailure = "import_identity_mismatch";
}
if (identityAdmissionFailure != null) {
end(false);
serve(descriptor);
return;
if (importedSha256 == null ||
!constantTimeEquals(
importedSha256.toLowerCase(Locale.ROOT),
descriptor.targetSha256
)) {
throw new IllegalStateException(
"Ghidra imported-byte digest does not match the admitted target"
);
}
admittedTargetSha256 = descriptor.targetSha256;
// GhidraScript wraps run() in a transaction. End it before serving so
@@ -256,7 +242,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
throw new IOException("Windows Ghidra P0 does not admit metadata recovery or database mutation.");
analysisExtensions.analyze(descriptor.analysisExtensions, currentProgram, monitor);
initializeDecompiler();
checkpointModification = currentProgram.getModificationNumber();
serve(descriptor);
}
finally {
@@ -420,18 +405,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
writeSuccess(writer, request.id, result);
return;
}
if (identityAdmissionFailure != null) {
// Keep the producer alive until the client has consumed the
// refusal and closed or sent shutdown. Every other request
// remains refused, including further startup attempts.
writeFailure(writer, request.id, identityAdmissionFailure,
"Ghidra imported identity admission failed");
continue;
}
decompileTimeoutSeconds = request.decompileTimeoutSeconds() == null
? descriptor.decompileTimeoutSeconds() : request.decompileTimeoutSeconds();
valueFlowLimits = request.valueFlowLimits();
applyDecompilePayloadLimit(request.decompileMaxPayloadMiB());
writeSuccess(writer, request.id, handleRequest(request, descriptor));
}
catch (RequestFailure failure) {
@@ -463,7 +438,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (descriptor.existingProject && isMutatingOperation(request.method))
throw new RequestFailure("method_unavailable", "Existing Ghidra project sessions are read-only");
return switch (request.method) {
case "checkpoint_program" -> checkpointProgram(request.params, descriptor);
case "read_binary_inventory_summary" -> binaryInventorySummary(request.params);
case "inspect_native_load_image" -> inspectNativeLoadImage(request.params);
case "read_bytes" -> readMemoryBytes(request.params);
@@ -519,31 +493,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
return method.equals("annotate_native_function") || method.equals("set_address_name") || method.equals("set_addresses_names");
}
/** Internal lifecycle request: the destination is inside this owned runtime. */
private JsonObject checkpointProgram(JsonObject params, SessionDescriptor descriptor) throws Exception {
requireKeys(params, Set.of("path", "force"));
if (descriptor.existingProject || !descriptor.transport.equals("unix-socket"))
throw new RequestFailure("method_unavailable", "This Program has no checkpoint authority");
Path output = Path.of(requireString(params, "path")).toAbsolutePath().normalize();
if (!output.getParent().equals(checkpointDirectory) ||
!output.getFileName().toString().matches("checkpoint-[a-f0-9-]{36}\\.gzf"))
throw new RequestFailure("invalid_request", "Checkpoint destination must belong to this runtime");
JsonElement forced = params.get("force");
if (!forced.isJsonPrimitive() || !forced.getAsJsonPrimitive().isBoolean())
throw new RequestFailure("invalid_request", "Checkpoint force must be a boolean");
boolean force = forced.getAsBoolean();
JsonObject result = new JsonObject();
boolean changed = currentProgram.getModificationNumber() != checkpointModification;
result.addProperty("saved", force || changed);
if (force || changed) {
if (Files.exists(output)) throw new RequestFailure("invalid_request", "Checkpoint destination already exists");
currentProgram.saveToPackedFile(output.toFile(), monitor);
checkpointModification = currentProgram.getModificationNumber();
result.addProperty("path", output.toString());
}
return result;
}
private JsonObject sessionInfo(SessionDescriptor descriptor) throws Exception {
boolean timedOut = analysisTimeoutOccurred();
JsonObject provider = new JsonObject();
@@ -822,8 +771,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private record ByteSearchBlock(MemoryBlock block, Address low, Address high) {}
private JsonObject searchMemoryBytes(JsonObject params, SessionDescriptor descriptor) throws Exception {
// mask, start, end, and cursor are omitted when absent; JSON encoding drops those keys.
Set<String> keys = params.keySet();
@@ -884,24 +831,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonArray blocks = new JsonArray();
JsonArray skipped = new JsonArray();
JsonArray notLoaded = new JsonArray();
JsonArray overlays = new JsonArray();
List<ByteSearchBlock> eligible = new ArrayList<>();
long covered = 0;
String next = null;
for (MemoryBlock block : ordered) {
monitor.checkCancelled();
Address low = block.getStart();
Address high = block.getEnd();
Address bound = start == null ? end : start;
if (bound != null && !bound.getAddressSpace().equals(low.getAddressSpace())) continue;
// Without bounds, search loaded non-overlay memory. Unloaded file
// parts and loaded overlays stay reachable through explicit bounds.
if (bound == null && !block.isLoaded()) {
// Without bounds, search loaded memory only. Non-loaded file parts
// (such as .symtab) and overlays stay reachable through explicit bounds.
if (bound == null && (!block.isLoaded() || block.isOverlay())) {
notLoaded.add(byteSearchCoverage(block, low, high));
continue;
}
if (bound == null && block.isOverlay()) {
overlays.add(byteSearchCoverage(block, low, high));
continue;
}
if (start != null && low.compareTo(start) < 0) low = start;
if (end != null && high.compareTo(end) > 0) high = end;
if (low.compareTo(high) > 0) continue;
@@ -909,73 +852,46 @@ public final class ReaGhidraBridge extends HeadlessScript {
skipped.add(byteSearchCoverage(block, low, high));
continue;
}
eligible.add(new ByteSearchBlock(block, low, high));
}
long covered = 0;
String next = null;
for (int first = 0; first < eligible.size() && next == null;) {
monitor.checkCancelled();
int last = first;
while (last + 1 < eligible.size()) {
monitor.checkCancelled();
ByteSearchBlock previous = eligible.get(last);
ByteSearchBlock following = eligible.get(last + 1);
if (!previous.high().getAddressSpace().equals(following.low().getAddressSpace()) ||
previous.high().next() == null || !previous.high().next().equals(following.low())) break;
last++;
}
Address low = eligible.get(first).low();
Address high = eligible.get(last).high();
if (next != null) continue;
if (resume != null && low.compareTo(resume) < 0) low = resume;
if (low.compareTo(high) <= 0 && low.getAddressSpace().equals(high.getAddressSpace())) {
Address at = low;
Address examinedEnd = high;
while (at != null && at.compareTo(high) <= 0 && high.subtract(at) >= bytes.length - 1L) {
monitor.checkCancelled();
// findBytes bounds candidate starts. The entire candidate remains in this eligible run.
Address found = currentProgram.getMemory().findBytes(at, high.subtract(bytes.length - 1L), bytes, masks, true, monitor);
monitor.checkCancelled();
if (found == null || high.subtract(found) < bytes.length - 1L) break;
if (matches.size() == limit) {
next = binding + ":" + canonicalAddress(found);
examinedEnd = found.addNoWrap(bytes.length - 1L);
break;
}
MemoryBlock matchBlock = currentProgram.getMemory().getBlock(found);
JsonObject match = new JsonObject();
match.addProperty("address", canonicalAddress(found));
match.addProperty("segment", matchBlock.getName());
Long offset = targetFileOffset(matchBlock, found).offset();
match.add("file_offset", offset == null ? JsonNull.INSTANCE : GSON.toJsonTree(offset));
Function function = currentProgram.getFunctionManager().getFunctionContaining(found);
JsonObject procedure = new JsonObject();
if (function != null) {
procedure.addProperty("address", canonicalAddress(function.getEntryPoint()));
procedure.addProperty("name", function.getName(true));
}
match.add("procedure", function == null ? JsonNull.INSTANCE : procedure);
matches.add(match);
at = found.next();
if (low.compareTo(high) > 0 || !low.getAddressSpace().equals(high.getAddressSpace())) continue;
Address at = low;
Address examinedEnd = high;
while (at != null && at.compareTo(high) <= 0 && high.subtract(at) >= bytes.length - 1L) {
monitor.checkCancelled();
// Ghidra's end address bounds candidate starts, not the final matched byte.
Address found = currentProgram.getMemory().findBytes(at, high.subtract(bytes.length - 1L), bytes, masks, true, monitor);
monitor.checkCancelled();
if (found == null || high.subtract(found) < bytes.length - 1L) break;
if (matches.size() == limit) {
next = binding + ":" + canonicalAddress(found);
examinedEnd = found.addNoWrap(bytes.length - 1L);
break;
}
for (int index = first; index <= last; index++) {
monitor.checkCancelled();
ByteSearchBlock item = eligible.get(index);
Address coveredLow = item.low().compareTo(low) < 0 ? low : item.low();
Address coveredHigh = item.high().compareTo(examinedEnd) > 0 ? examinedEnd : item.high();
if (coveredLow.compareTo(coveredHigh) > 0) continue;
JsonObject coverage = byteSearchCoverage(item.block(), coveredLow, coveredHigh);
covered = Math.addExact(covered, coverage.get("bytes").getAsLong());
blocks.add(coverage);
JsonObject match = new JsonObject();
match.addProperty("address", canonicalAddress(found));
match.addProperty("segment", block.getName());
Long offset = targetFileOffset(block, found).offset();
match.add("file_offset", offset == null ? JsonNull.INSTANCE : GSON.toJsonTree(offset));
Function function = currentProgram.getFunctionManager().getFunctionContaining(found);
JsonObject procedure = new JsonObject();
if (function != null) {
procedure.addProperty("address", canonicalAddress(function.getEntryPoint()));
procedure.addProperty("name", function.getName(true));
}
match.add("procedure", function == null ? JsonNull.INSTANCE : procedure);
matches.add(match);
at = found.next();
}
first = last + 1;
JsonObject coverage = byteSearchCoverage(block, low, examinedEnd);
covered = Math.addExact(covered, coverage.get("bytes").getAsLong());
blocks.add(coverage);
}
JsonObject searched = new JsonObject();
searched.add("blocks", blocks);
searched.addProperty("bytes", covered);
searched.add("skipped_uninitialized", skipped);
searched.add("skipped_not_loaded", notLoaded);
searched.add("skipped_overlays", overlays);
JsonObject result = new JsonObject();
result.add("matches", matches);
result.add("searched", searched);
@@ -1044,26 +960,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private DecompileOptions decompileOptions(Integer payloadMiB) {
private void initializeDecompiler() {
DecompileOptions options = new DecompileOptions();
options.setMaxInstructions(Integer.MAX_VALUE);
if (payloadMiB != null) options.setMaxPayloadMBytes(payloadMiB);
return options;
}
private void applyDecompilePayloadLimit(Integer payloadMiB) {
if (java.util.Objects.equals(decompileMaxPayloadMiB, payloadMiB)) return;
// DecompInterface supports live setOptions: it updates the existing
// DecompileProcess result-size bound before sending native options.
if (!decompiler.setOptions(decompileOptions(payloadMiB)))
throw new RequestFailure("decompile_options_failed", "Ghidra could not apply the selected native decompiler payload limit");
decompiler.flushCache();
decompileMaxPayloadMiB = payloadMiB;
}
private void initializeDecompiler() {
decompiler = new DecompInterface();
decompiler.setOptions(decompileOptions(decompileMaxPayloadMiB));
decompiler.setOptions(options);
decompiler.toggleCCode(true);
decompiler.toggleSyntaxTree(true);
decompiler.toggleJumpLoads(true);
@@ -1109,7 +1010,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
} catch (RequestFailure failure) {
// Existing names/addresses win. Only an unresolved numeric input
// can fall back to source coordinates; ambiguity never falls back.
if (!failure.code.equals("procedure_not_found") || !selector.matches("0x[0-9a-fA-F]+"))
if (!failure.code.equals("not_found") || !selector.matches("0x[0-9a-fA-F]+"))
throw failure;
}
}
@@ -1136,17 +1037,17 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
List<Address> candidates = new ArrayList<>(mappedAddresses);
if (candidates.size() > 1)
throw new RequestFailure("file_offset_ambiguous", "File offset " + coordinate +
throw new RequestFailure("ambiguous", "File offset " + coordinate +
" has multiple loaded mappings: " + candidates.stream().map(ReaGhidraBridge::canonicalAddress).toList() +
"; select an explicit analysis address");
if (candidates.isEmpty())
throw new RequestFailure("file_offset_unmapped", "File offset " + coordinate +
throw new RequestFailure("not_found", "File offset " + coordinate +
" has no loaded source mapping for the admitted target file; Ghidra image base: " + canonicalAddress(currentProgram.getImageBase()) +
". Inspect mappings with rea inspect-native-load-image <path> --provider ghidra --json or MCP inspect_native_load_image.");
Address mapped = candidates.get(0);
Function function = functionAtOrContaining(mapped);
if (function == null)
throw new RequestFailure("procedure_not_found", "File offset " + coordinate +
throw new RequestFailure("not_found", "File offset " + coordinate +
" maps to analysis address " + canonicalAddress(mapped) +
", but no procedure starts at or contains that address in the current Ghidra analysis. " +
"Choose an observed procedure with CLI rea search <path> . --kind procedures --mode regex --provider ghidra --json or MCP list_procedures.");
@@ -1349,12 +1250,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (component.getDataType() instanceof ghidra.program.model.data.BitFieldDataType bitfield) {
field.addProperty("bit_size", bitfield.getBitSize());
field.addProperty("bit_offset", bitfield.getBitOffset());
JsonObject bitField = new JsonObject();
bitField.addProperty("base_type_id", bitfield.getBaseDataType().getPathName());
bitField.addProperty("declared_bit_size", bitfield.getDeclaredBitSize());
bitField.addProperty("component_type_lookup", typeLookupResult(bitfield));
bitField.addProperty("base_type_lookup", typeLookupResult(bitfield.getBaseDataType()));
field.add("bit_field", bitField);
}
field.addProperty("status", "observed");
field.addProperty("flexible_tail", "unknown");
@@ -1385,7 +1280,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
result.add("members", members);
result.addProperty("total_fields", totalFields);
result.addProperty("truncated", totalFields > 20_000);
result.add("limitations", GSON.toJsonTree(List.of("Layouts are Ghidra data-type database observations; original debug/source authority and flexible-tail semantics are not inferred. Component type paths do not necessarily identify separately registered types; bit-field lookups report whether the manager resolves equivalent types.", "At most 20000 fields or enum members are serialized per type to bound provider output. total_fields and truncated report omissions. Enum values preserve Ghidra's signed long representation.")));
result.add("limitations", GSON.toJsonTree(List.of("Layouts are Ghidra data-type database observations; original debug/source authority and flexible-tail semantics are not inferred. Nested and recursive types retain category-path identities for separate inspection.", "At most 20000 fields or enum members are serialized per type to bound provider output. total_fields and truncated report omissions. Enum values preserve Ghidra's signed long representation.")));
if (type != null) {
JsonObject metadata = analysisExtensions.inspectDataType(currentProgram, type, address, monitor);
if (metadata != null) result.add("metadata_recovery", metadata);
@@ -1393,11 +1288,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private String typeLookupResult(DataType type) {
DataType resolved = currentProgram.getDataTypeManager().getDataType(type.getPathName());
return resolved == null ? "not-found" : resolved.isEquivalent(type) ? "equivalent" : "different";
}
private JsonObject instructionReference(Reference reference) {
JsonObject item = new JsonObject();
JsonObject kind = referenceKind(reference);
@@ -1445,29 +1335,8 @@ public final class ReaGhidraBridge extends HeadlessScript {
flow.addProperty("conditional", false);
flow.addProperty("computed", false);
if (instruction == null) {
Instruction containing = currentProgram.getListing().getInstructionContaining(address);
boolean inMemory = currentProgram.getMemory().contains(address);
Data containingData = inMemory && containing == null
? currentProgram.getListing().getDefinedDataContaining(address) : null;
result.addProperty("status", !inMemory ? "outside-memory" : containing != null
? "not-instruction-boundary" : containingData != null ? "data" : "not-in-listing");
if (inMemory && containing == null && containingData != null) {
JsonObject parentData = new JsonObject();
parentData.addProperty("address", canonicalAddress(containingData.getAddress()));
parentData.addProperty("length", containingData.getLength());
parentData.addProperty("type_id", containingData.getDataType().getPathName());
parentData.addProperty("provenance", "ghidra-listing");
result.add("containing_data", parentData);
}
if (containing != null && currentProgram.getMemory().contains(address)) {
JsonObject containingInstruction = new JsonObject();
containingInstruction.addProperty("address", canonicalAddress(containing.getAddress()));
containingInstruction.addProperty("length", containing.getLength());
containingInstruction.addProperty("mnemonic", containing.getMnemonicString());
containingInstruction.addProperty("raw_disassembly", containing.toString());
containingInstruction.addProperty("provenance", "ghidra-listing");
result.add("containing_instruction", containingInstruction);
}
result.addProperty("status", !currentProgram.getMemory().contains(address) ? "outside-memory" : currentProgram.getListing().getInstructionContaining(address) != null
? "not-instruction-boundary" : currentProgram.getListing().getDefinedDataContaining(address) != null ? "data" : "not-in-listing");
for (String key : List.of("bytes", "length", "mnemonic", "raw_disassembly")) result.add(key, JsonNull.INSTANCE);
} else {
result.addProperty("status", "decoded");
@@ -1877,7 +1746,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
while (operations.hasNext()) {
monitor.checkCancelled();
PcodeOpAST operation = operations.next();
if (selected.size() >= valueFlowLimits.maxOperations()) {
if (selected.size() >= MAX_HIGH_PCODE_OPS) {
omittedOperationsLowerBound = 1;
break;
}
@@ -1905,7 +1774,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonArray effects = new JsonArray();
int omittedInputs = 0;
int omittedEdges = 0;
boolean edgeLimitReached = false;
for (PcodeOpAST operation : selected) {
monitor.checkCancelled();
String operationId = operationIds.get(operation);
@@ -1921,16 +1789,14 @@ public final class ReaGhidraBridge extends HeadlessScript {
value.addProperty("block_membership", operation.getParent() == null ? "detached" : "member");
JsonArray inputs = new JsonArray();
int inputCount = operation.getNumInputs();
int capturedInputs = Math.min(inputCount, valueFlowLimits.maxInputsPerOperation());
int capturedInputs = Math.min(inputCount, MAX_HIGH_PCODE_INPUTS_PER_OP);
for (int inputIndex = 0; inputIndex < capturedInputs; inputIndex += 1) {
Varnode input = operation.getInput(inputIndex);
inputs.add(varnodeValue(input));
if (input instanceof VarnodeAST astInput) {
if (astInput.getHigh() instanceof HighParam parameter) {
if (parameterUses.size() >= valueFlowLimits.maxEdges()) {
edgeLimitReached = true;
omittedEdges++;
} else {
if (parameterUses.size() >= MAX_HIGH_PCODE_DEF_USE_EDGES) omittedEdges++;
else {
JsonObject use = new JsonObject();
use.addProperty("ordinal", parameter.getSlot());
use.addProperty("use", operationId);
@@ -1941,8 +1807,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
PcodeOp definition = astInput.getDef();
if (definition instanceof PcodeOpAST astDefinition) {
String definitionId = operationIds.get(astDefinition);
if (definitionId == null || defUse.size() >= valueFlowLimits.maxEdges()) {
if (defUse.size() >= valueFlowLimits.maxEdges()) edgeLimitReached = true;
if (definitionId == null || defUse.size() >= MAX_HIGH_PCODE_DEF_USE_EDGES) {
omittedEdges += 1;
} else {
JsonObject edge = new JsonObject();
@@ -1965,16 +1830,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
JsonObject result = new JsonObject();
result.addProperty("available", true);
result.addProperty("provenance", "ghidra-high-pcode");
JsonObject limits = new JsonObject();
limits.addProperty("max_operations", valueFlowLimits.maxOperations());
limits.addProperty("max_inputs_per_operation", valueFlowLimits.maxInputsPerOperation());
limits.addProperty("max_edges", valueFlowLimits.maxEdges());
result.add("limits", limits);
JsonArray limitsReached = new JsonArray();
if (omittedOperationsLowerBound > 0) limitsReached.add("max_operations");
if (omittedInputs > 0) limitsReached.add("max_inputs_per_operation");
if (edgeLimitReached) limitsReached.add("max_edges");
result.add("limits_reached", limitsReached);
result.add("operations", operationValues);
result.add("def_use", defUse);
result.add("parameters", parameterValues);
@@ -1996,42 +1851,19 @@ public final class ReaGhidraBridge extends HeadlessScript {
limitations.add(
"Def-use links do not cross function boundaries and may omit memory aliasing, call side effects, or unresolved indirect flow."
);
limitations.add(
"Memory read/write effects cover high-p-code LOAD and STORE operations only; direct memory address varnodes in other operations are not included. An empty effect list does not establish that no memory is accessed. Recorded instruction references may provide complementary evidence."
);
limitations.add(
"Memory read/write effects include stack and temporary memory operations; they do not identify persistent state or business meaning."
);
limitations.add(
"Operation, input, and def-use limits can truncate large functions; inspect the effective limits, limits_reached and omitted counts before drawing conclusions."
"Operation, input, and def-use limits can truncate large functions; inspect truncation, omitted lower bounds, and known omitted counts before drawing conclusions."
);
limitations.add(
"Known omitted-edge counts cover links omitted from retained operations; omitted operations may contain additional uncounted links."
);
if (omittedOperationsLowerBound > 0)
limitations.add(valueFlowLimitRecovery("max_operations", valueFlowLimits.maxOperations()));
if (omittedInputs > 0)
limitations.add(valueFlowLimitRecovery("max_inputs_per_operation", valueFlowLimits.maxInputsPerOperation()));
if (edgeLimitReached)
limitations.add(valueFlowLimitRecovery("max_edges", valueFlowLimits.maxEdges()));
result.add("limitations", limitations);
return result;
}
private static String valueFlowLimitRecovery(String field, int effective) {
String setting = "ghidra.value_flow_" + field;
String environment = "REA_GHIDRA_VALUE_FLOW_" + field.toUpperCase(Locale.ROOT);
String adjustment = effective < Integer.MAX_VALUE
? "In MCP, use update_analysis_settings with setting " + setting +
", action set and a larger integer value from 1 to " + Integer.MAX_VALUE +
", then repeat the selected function analysis on this Program without restarting REA. " +
"For a one-shot CLI retry, set " + environment +
" to a larger count in the environment that starts the next rea invocation, then repeat the selected analysis; the previous invocation's Program is no longer active."
: "The setting is at its Java integer maximum; select a smaller function for this projection.";
return setting + " reached its effective value of " + effective + ". " + adjustment +
" These controls do not raise the decompile deadline, bridge transport or MCP client limits.";
}
private String pcodeOperationId(PcodeOpAST operation) {
return canonicalAddress(operation.getSeqnum().getTarget()) + "#" +
operation.getSeqnum().getTime();
@@ -2465,7 +2297,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (!match.matched()) {
continue;
}
JsonObject item = resultItem(entry.item, "procedure");
JsonObject item = resultItem(entry.item, null);
if (!match.aliases().isEmpty()) {
JsonArray aliases = new JsonArray();
for (String alias : match.aliases()) aliases.add(alias);
@@ -2480,7 +2312,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (!matcher.matches(item.value)) {
continue;
}
items.add(resultItem(item, procedureSearch ? "procedure" : "string"));
items.add(resultItem(item, null));
}
return items;
}
@@ -2626,7 +2458,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (matchProcedureNames(entry, selector).matched()) matches.add(entry.function);
}
if (matches.size() > 1)
throw new RequestFailure("procedure_ambiguous", "Ghidra procedure name is ambiguous: " + value +
throw new RequestFailure("ambiguous", "Ghidra procedure name is ambiguous: " + value +
"; select an exact entry address from " + matches.stream()
.map(function -> canonicalAddress(function.getEntryPoint())).toList());
if (matches.size() == 1) return matches.get(0);
@@ -2634,11 +2466,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
// Retain legacy bare-hex address input only when no symbol matches.
Address address = explicitAddress ? parseReaAddress(value) : tryParseAddress(value);
if (address == null)
throw new RequestFailure("procedure_not_found", "Unknown Ghidra procedure name or address: " + value);
throw new RequestFailure("not_found", "Unknown Ghidra procedure name or address: " + value);
Function function = functionAtOrContaining(address);
if (function == null)
// Other procedure operations retain their analysis-address semantics.
throw new RequestFailure("procedure_not_found", "No procedure exists at the requested address: " + value +
throw new RequestFailure("not_found", "No procedure exists at the requested address: " + value +
"; Ghidra image base: " + canonicalAddress(currentProgram.getImageBase()) +
". Choose an observed analysis address. CLI: rea search <path> . --kind procedures --mode regex --provider ghidra --json; " +
"MCP: list_procedures. Inspect the loaded base and source mappings with CLI rea analyze <path> --provider ghidra --json " +
@@ -3754,26 +3586,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (!results.decompileCompleted()) {
throw new RequestFailure(
"decompile_failed",
"Ghidra decompilation failed: " + diagnosticMessage(results.getErrorMessage()) +
". The effective native decompiler payload allowance is " +
decompiler.getOptions().getMaxPayloadMBytes() +
" MiB. " + decompilePayloadRecovery()
"Ghidra decompilation failed: " + diagnosticMessage(results.getErrorMessage())
);
}
return results;
}
private String decompilePayloadRecovery() {
int effective = decompiler.getOptions().getMaxPayloadMBytes();
if (effective >= MAX_DECOMPILE_PAYLOAD_MIB)
return "If the diagnostic reports payload exhaustion, the native payload allowance is already at its supported maximum; no higher value is accepted. Use analyze_function for available listing facts or explicitly select a smaller function for decompilation.";
return "If the diagnostic reports payload exhaustion, in MCP raise ghidra.decompile_max_payload_mib with update_analysis_settings " +
"(integer MiB from 1 to " + MAX_DECOMPILE_PAYLOAD_MIB +
") and repeat the selected operation on this Program without restarting REA. " +
"For a one-shot CLI retry, set REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB to a larger MiB value in the environment that starts the next rea invocation, then repeat the selected operation; the previous invocation's Program is no longer active. " +
"Runtime null or an omitted launch seed restores the installed Ghidra default. This does not raise the decompile deadline, high-pcode projection budgets or MCP client limits.";
}
private static String decompiledText(DecompileResults results) {
if (results == null) {
return null;
@@ -3819,23 +3637,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
return result;
}
private JsonObject callRelationshipCoverage(
private static JsonObject callRelationshipCoverage(
CallBoundaryObservation calls, Integer missingCallers) {
JsonObject coverage = completeResolvedCallCoverage();
JsonObject discovery = new JsonObject();
if (missingCallers != null) {
JsonObject callers = new JsonObject();
callers.addProperty("status", "unknown");
callers.addProperty("reason", "Recorded incoming references do not establish additional computed callers elsewhere in the program.");
discovery.add("callers", callers);
}
if (calls != null && !calls.computedCallSites().isEmpty()) {
JsonObject callees = new JsonObject();
callees.addProperty("status", "unknown");
callees.addProperty("reason", computedCallTargetLimitation(calls.computedCallSites()));
discovery.add("callees", callees);
}
if (!discovery.isEmpty()) coverage.add("discovery", discovery);
Integer omitted = calls == null ? null : calls.unrepresentedCalleeCalls();
JsonArray directions = new JsonArray();
if (missingCallers != null && missingCallers > 0) directions.add("callers");
@@ -3850,49 +3654,20 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (calls == null && missingCallers == null) directions.add("callers");
directions.add("callees");
coverage.add("limited_directions", directions);
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
return coverage;
}
JsonArray limitations = new JsonArray();
if (callerLimitation != null) limitations.add(callerLimitation);
if (omitted != null && omitted > 0) directions.add("callees");
if (omitted != null && omitted > 0) {
limitations.add("Ghidra's callee inventory omits targets of " + omitted +
" call reference(s) or decoded call(s) without destinations.");
directions.add("callees");
}
if (!limitations.isEmpty()) {
coverage.addProperty("status", "partial");
coverage.add("limitations", limitations);
coverage.add("limited_directions", directions);
}
return qualifyProjectCallCoverage(coverage, calls, missingCallers);
}
private JsonObject qualifyProjectCallCoverage(
JsonObject coverage, CallBoundaryObservation calls, Integer missingCallers) {
if (!existingProject) return coverage;
String reason = "The selected project's prior analysis completeness is unknown; " +
"the recorded call relationships may omit undiscovered callers or callees.";
String status = coverage.get("status").getAsString();
// A single-direction read with known omissions is still partial; an
// unknown project history adds a limitation rather than erasing them.
if (status.equals("partial") && (calls == null || missingCallers == null)) {
coverage.getAsJsonArray("limitations").add(reason);
return coverage;
}
if (status.equals("unknown")) reason += " " + coverage.get("reason").getAsString();
if (status.equals("partial")) {
for (JsonElement limitation : coverage.getAsJsonArray("limitations"))
reason += " " + limitation.getAsString();
}
JsonArray directions = new JsonArray();
if (missingCallers != null || (calls == null && missingCallers == null))
directions.add("callers");
if (calls != null || (calls == null && missingCallers == null))
directions.add("callees");
coverage.addProperty("status", "unknown");
coverage.addProperty("reason", reason);
coverage.remove("limitations");
coverage.add("limited_directions", directions);
return coverage;
}
@@ -3907,24 +3682,6 @@ public final class ReaGhidraBridge extends HeadlessScript {
return coverage;
}
/** Name a few sites inline; reference reads and call-target queries recover the rest. */
private static final int NAMED_COMPUTED_CALL_SITES = 4;
/**
* A computed call's ReferenceManager targets are static candidates: Ghidra
* records no proof that other values cannot reach the call, so the callee
* set stays incomplete while the recorded candidates remain observed.
*/
private static String computedCallTargetLimitation(List<String> sites) {
if (sites.isEmpty()) return null;
int named = Math.min(sites.size(), NAMED_COMPUTED_CALL_SITES);
String listed = String.join(", ", sites.subList(0, named)) +
(sites.size() > named ? " and " + (sites.size() - named) + " more" : "");
return "Ghidra records call reference targets at " + sites.size() +
" computed call site(s) (" + listed + "); these static references do not establish" +
" every possible target, so additional callees remain unknown.";
}
private boolean unrepresentedCaller(Reference reference, Set<Function> callers) {
if (!reference.getReferenceType().isCall()) return false;
Function caller = functionAtOrContaining(reference.getFromAddress());
@@ -3959,11 +3716,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
monitor.checkCancelled();
calls.observe(reference);
}
List<String> computedCallSites = new ArrayList<>();
for (Instruction instruction : instructions) {
boundary.observe(instruction);
FlowType flow = instruction.getFlowType();
if (!flow.isCall()) continue;
if (!instruction.getFlowType().isCall()) continue;
boolean destination = false;
for (Reference reference : instruction.getReferencesFrom()) {
if (!reference.getReferenceType().isCall()) continue;
@@ -3975,13 +3730,9 @@ public final class ReaGhidraBridge extends HeadlessScript {
calls.unresolved(instruction.getAddress(),
"Instruction decoder identifies a call but ReferenceManager supplies no call destination");
}
// The same decoded flow classifies resolve_native_call_targets'
// mechanism; its recorded targets are candidates, not a closed set.
else if (flow.isComputed()) computedCallSites.add(canonicalAddress(instruction.getAddress()));
}
return new CallBoundaryObservation(boundary.values(),
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls(),
List.copyOf(computedCallSites));
returnedCallees == null ? null : calls.unrepresented, calls.unresolvedCalls());
}
private final class FunctionBoundaryLimitations {
@@ -5165,7 +4916,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
// without retaining its cause. This producer reason is not bad syntax.
if (exception.getDescription().equals("Stack overflow during pattern compilation"))
throw regexStackExhausted(expression, "compiling");
throw new RequestFailure("invalid_regex_pattern", "Invalid regex pattern " + expression + ": " + exception.getDescription() + " at index " + exception.getIndex());
throw new RequestFailure("invalid_request", "Invalid regex pattern " + expression + ": " + exception.getDescription() + " at index " + exception.getIndex());
}
catch (StackOverflowError exhausted) {
throw regexStackExhausted(expression, "compiling");
@@ -5225,28 +4976,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
Set<String> keys = new HashSet<>(REQUEST_KEYS);
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_timeout_seconds"))
keys.add("decompile_timeout_seconds");
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("value_flow_limits"))
keys.add("value_flow_limits");
if (parsed.isJsonObject() && parsed.getAsJsonObject().has("decompile_max_payload_mib"))
keys.add("decompile_max_payload_mib");
JsonObject object = requireObject(parsed, keys);
Integer payloadMiB = null;
if (object.has("decompile_max_payload_mib") && !object.get("decompile_max_payload_mib").isJsonNull()) {
payloadMiB = requireInteger(object, "decompile_max_payload_mib");
if (payloadMiB < 1 || payloadMiB > MAX_DECOMPILE_PAYLOAD_MIB)
throw new IllegalArgumentException("Invalid per-request native decompiler payload limit");
}
ValueFlowLimits flowLimits = defaultValueFlowLimits();
if (object.has("value_flow_limits")) {
JsonObject limits = requireObject(object.get("value_flow_limits"),
Set.of("max_operations", "max_inputs_per_operation", "max_edges"));
int operations = requireInteger(limits, "max_operations");
int inputs = requireInteger(limits, "max_inputs_per_operation");
int edges = requireInteger(limits, "max_edges");
if (operations < 1 || inputs < 1 || edges < 1)
throw new IllegalArgumentException("Invalid per-request value-flow limits");
flowLimits = new ValueFlowLimits(operations, inputs, edges);
}
Integer timeout = null;
if (object.has("decompile_timeout_seconds")) {
JsonElement value = object.get("decompile_timeout_seconds");
@@ -5262,7 +4992,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
if (params == null || !params.isJsonObject()) {
throw new IllegalArgumentException("Bridge request parameters are invalid");
}
return new Request(id, requireString(object, "method"), params.getAsJsonObject(), timeout, payloadMiB, flowLimits);
return new Request(id, requireString(object, "method"), params.getAsJsonObject(), timeout);
}
private static JsonObject requireObject(JsonElement element, Set<String> keys) {
@@ -5448,7 +5178,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
}
}
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {}
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls, List<String> computedCallSites) {}
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
private record SwiftLiteralLayout(
String representation,
@@ -5470,11 +5200,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
int decompileTimeoutSeconds,
boolean existingProject
) {}
private static ValueFlowLimits defaultValueFlowLimits() {
return new ValueFlowLimits(DEFAULT_HIGH_PCODE_OPS, DEFAULT_HIGH_PCODE_INPUTS_PER_OP, DEFAULT_HIGH_PCODE_DEF_USE_EDGES);
}
private record ValueFlowLimits(int maxOperations, int maxInputsPerOperation, int maxEdges) {}
private record Request(int id, String method, JsonObject params, Integer decompileTimeoutSeconds, Integer decompileMaxPayloadMiB, ValueFlowLimits valueFlowLimits) {}
private record Request(int id, String method, JsonObject params, Integer decompileTimeoutSeconds) {}
private static final class RequestFailure extends RuntimeException {
private final String code;
@@ -37,7 +37,7 @@ import org.xml.sax.InputSource;
/**
* Apply the committed RISC-V calling-convention policy after analysis. Ghidra's generic RV32
* compiler specification models ilp32d. Each policy derives a program-specific prototype from the
* program's own default prototype, as its {@link Policy} row and docs/providers/ghidra-riscv.md state. The
* program's own default prototype, as its {@link Policy} row and docs/ghidra-riscv.md state. The
* script installs the prototype as a specification extension and assigns it, with dynamic
* storage, to every non-thunk function, external function and function definition.
*/
+1 -2
View File
@@ -1399,8 +1399,7 @@ def _serve_connection(connection):
break
except (ConnectionError, OSError):
# A lost client cannot confirm document closure or revoke this run.
# Keep the listener available for the owner to confirm cleanup.
should_stop = False
pass
finally:
try:
file.close()
+8 -13
View File
@@ -18,6 +18,7 @@ from mitmproxy import ctx, version
from mitmproxy.io import tnetstring
PROFILE = "12.2.3"
OUTPUT_BYTES = 96 * 1024 * 1024
CREDENTIALS = {b"authorization", b"proxy-authorization", b"cookie", b"set-cookie"}
@@ -203,7 +204,7 @@ def without_userinfo(value, base_scheme=None):
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
def project_record(state, secrets, max_depth):
def project_record(state, secrets):
binaries, numbers, redactions = [], [], []
byte_secrets = []
for secret in secrets:
@@ -250,8 +251,8 @@ def project_record(state, secrets, max_depth):
del redactions[before[2]:]
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
if depth > max_depth:
raise CaptureFailure("input-limit", f"Native capture exceeds max_nesting_depth ({max_depth} levels).", path)
if depth > 64:
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
if credential:
validate_omitted(value, path, depth)
redactions.append({"pointer": path, "reason": "transport-credential"})
@@ -336,12 +337,12 @@ def decode(request):
except CaptureFailure:
raise
except RecursionError:
raise CaptureFailure("unsupported", "Native parser reached Python's recursion limit at byte offset " + str(start) + ". max_nesting_depth does not raise that upstream limit.") from None
raise CaptureFailure("input-limit", "Native parser exceeded its recursion budget at byte offset " + str(start) + ".") from None
except (ValueError, IndexError, UnicodeError, EOFError):
raise CaptureFailure("format", "Malformed native record at byte offset " + str(start) + ".") from None
if not isinstance(state, dict):
raise CaptureFailure("format", "Native flow record must be a dictionary at byte offset " + str(start) + ".")
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"], request["budgets"]["depth"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
return {"decoder": {"id": "mitmproxy-native-tnetstring", "name": "REA offline native mitmproxy adapter", "version": PROFILE}, "container": {"reported": None, "numeric_literals": [], "redactions": [], "records_pointer": None}, "total_records": len(records), "records": records}
@@ -349,8 +350,6 @@ def failure_reply(error):
"""Keep producer constraints, resource exhaustion and adapter defects distinct."""
if isinstance(error, CaptureFailure):
return {"ok": False, "reason": error.reason, "message": str(error), "pointer": error.pointer}
if isinstance(error, RecursionError):
return {"ok": False, "reason": "unsupported", "message": "Native projection reached Python's recursion limit; max_nesting_depth does not raise that upstream limit.", "pointer": ""}
if isinstance(error, MemoryError):
return {"ok": False, "reason": "resource-limit", "message": "Native decoder exhausted its 768 MiB address-space budget.", "pointer": ""}
return {"ok": False, "reason": "decoder", "message": "Offline native decoder failed unexpectedly: " + type(error).__name__ + ".", "pointer": ""}
@@ -363,13 +362,9 @@ class OfflineCapture:
def running(self):
resource.setrlimit(resource.RLIMIT_AS, (768 * 1024 * 1024, 768 * 1024 * 1024))
resource.setrlimit(resource.RLIMIT_CPU, (30, 30))
resource.setrlimit(resource.RLIMIT_FSIZE, (OUTPUT_BYTES, OUTPUT_BYTES))
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
request = json.loads(Path(ctx.options.rea_request_path).read_text(encoding="utf-8"))
budgets = request["budgets"]
if any(type(budgets[key]) is not int or budgets[key] < 1 for key in ("inputBytes", "outputBytes", "depth")):
raise ValueError("Invalid admitted capture budgets")
output_bytes = budgets["outputBytes"]
resource.setrlimit(resource.RLIMIT_FSIZE, (output_bytes, output_bytes))
try:
response = {"ok": True, "value": decode(request)}
except Exception as error:
@@ -379,7 +374,7 @@ class OfflineCapture:
output = json.dumps(response, ensure_ascii=True, allow_nan=False, separators=(",", ":")).encode("ascii")
except Exception as error:
output = json.dumps(failure_reply(error), separators=(",", ":")).encode("ascii")
if len(output) > output_bytes:
if len(output) > OUTPUT_BYTES:
output = b'{"ok":false,"reason":"limit","message":"Native reply exceeds the complete-evidence output budget.","pointer":""}'
Path(request["reply_path"]).write_bytes(output)
finally:
+4 -4
View File
@@ -36,7 +36,7 @@ def location(value, scheme=None):
state = {"response": {"headers": [[b"Location", value]]}}
if scheme is not None:
state["request"] = {"scheme": scheme}
record = capture.project_record(state, [], 64)
record = capture.project_record(state, [])
sidecar = next(
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
)
@@ -75,11 +75,11 @@ class TransportUserinfoTests(unittest.TestCase):
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
with self.subTest(value=value):
record = capture.project_record({"request": {"path": value}}, [], 64)
record = capture.project_record({"request": {"path": value}}, [])
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
self.assertEqual(record["binary_fields"][0]["state"], "retained")
self.assertEqual(record["redactions"], [])
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [], 64)
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
@@ -110,7 +110,7 @@ class TransportUserinfoTests(unittest.TestCase):
"request": {"scheme": b"https"},
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
},
}, [], 64)
}, [])
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
self.assertEqual(record["redactions"], [])
-13
View File
@@ -24,16 +24,3 @@ func captureAccessibilityValue(_ source: Any?) -> AccessibilityValue {
}
return AccessibilityValue(value: NSNull(), gap: "AXValue type \(type(of: source)) is not a supported JSON scalar")
}
/** Keep an unavailable action list distinct from a successful empty list. */
func captureActionNames(status: Int32, success: Int32, names: [String]?) -> (names: [String], error: Int32?) {
guard status == success else { return ([], status) }
return (names ?? [], nil)
}
/** AX absence is distinct from a failed attribute read. */
func captureAttributeValue<Value>(status: Int32, success: Int32, absent: [Int32], value: Value?) -> (value: Value?, error: Int32?) {
if status == success { return (value, nil) }
if absent.contains(status) { return (nil, nil) }
return (nil, status)
}
+20 -45
View File
@@ -18,10 +18,9 @@ struct Request: Decodable {
struct Action: Decodable { var kind: String; var path: [Int]?; var direction: String?; var text: String? }
struct BoundaryFailure: Error { var code: String; var message: String }
func fail(_ code: String, _ message: String) throws -> Never { throw BoundaryFailure(code: code, message: message) }
func attribute(_ element: AXUIElement, _ key: String) -> (value: CFTypeRef?, error: Int32?) {
func attribute(_ element: AXUIElement, _ key: String) -> CFTypeRef? {
var value: CFTypeRef?
let status = AXUIElementCopyAttributeValue(element, key as CFString, &value)
return captureAttributeValue(status: status.rawValue, success: AXError.success.rawValue, absent: [AXError.noValue.rawValue, AXError.attributeUnsupported.rawValue], value: value)
return AXUIElementCopyAttributeValue(element, key as CFString, &value) == .success ? value : nil
}
func childCount(_ element: AXUIElement) -> ChildCount {
var count = 0
@@ -35,15 +34,13 @@ func children(_ element: AXUIElement, count: Int) -> ChildBatch<AXUIElement> {
return value as? [AXUIElement]
}
}
func windowBounds(_ element: AXUIElement) -> (bounds: CGRect?, error: Int32?) {
let position = attribute(element, kAXPositionAttribute)
let size = attribute(element, kAXSizeAttribute)
if let error = position.error ?? size.error { return (nil, error) }
guard let p = position.value, let s = size.value,
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return (nil, nil) }
var point = CGPoint.zero; var dimensions = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &dimensions) else { return (nil, nil) }
return (CGRect(origin: point, size: dimensions), nil)
func text(_ element: AXUIElement, _ key: String) -> Any { (attribute(element, key) as? String) ?? NSNull() as Any }
func windowBounds(_ element: AXUIElement) -> CGRect? {
guard let p = attribute(element, kAXPositionAttribute), let s = attribute(element, kAXSizeAttribute),
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return nil }
var point = CGPoint.zero; var size = CGSize.zero
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &size) else { return nil }
return CGRect(origin: point, size: size)
}
func observe(_ request: Request) async throws -> [String: Any] {
guard let app = NSRunningApplication(processIdentifier: request.pid) else {
@@ -87,22 +84,11 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard AXIsProcessTrusted() else { try fail("accessibility-denied", "Grant Accessibility to the REA host in System Settings > Privacy & Security, then retry") }
let application = AXUIElementCreateApplication(request.pid)
AXUIElementSetMessagingTimeout(application, 2)
let windowsAttribute = attribute(application, kAXWindowsAttribute)
if let error = windowsAttribute.error {
try fail("accessibility-unavailable", "Cannot read accessibility windows: AXError \(error)")
guard let axWindows = attribute(application, kAXWindowsAttribute) as? [AXUIElement] else {
try fail("accessibility-unavailable", "Selected application does not expose accessibility windows")
}
guard let axWindows = windowsAttribute.value as? [AXUIElement] else {
try fail("accessibility-unavailable", "Accessibility windows are unavailable for the selected application")
}
let candidates = axWindows.map { (element: $0, observation: windowBounds($0)) }
if let error = candidates.compactMap({ $0.observation.error }).first {
try fail("accessibility-unavailable", "Cannot read accessibility window geometry: AXError \(error)")
}
guard candidates.allSatisfy({ $0.observation.bounds != nil }) else {
try fail("accessibility-unavailable", "Accessibility window geometry is unavailable; the selected window cannot be identified")
}
let matches = candidates.filter { $0.observation.bounds == bounds }.map { $0.element }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(candidates.map { String(describing: $0.observation.bounds) })") }
let matches = axWindows.filter { windowBounds($0) == bounds }
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(axWindows.map { String(describing: windowBounds($0)) })") }
selected = matches[0]
}
if let action = request.action, let root = selected {
@@ -113,16 +99,13 @@ func observe(_ request: Request) async throws -> [String: Any] {
guard let available = count.value else {
try fail("element-count-unavailable", "Cannot validate selected accessibility path because child count failed with AXError \(count.error ?? -1)")
}
guard index >= 0 && index < available else { try fail("element-missing", "Selected accessibility path no longer exists") }
let childStatus = AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild)
guard childStatus == .success else { try fail("accessibility-unavailable", "Cannot read selected accessibility child: AXError \(childStatus.rawValue)") }
guard let item = (selectedChild as? [AXUIElement])?.first else { try fail("accessibility-unavailable", "Selected accessibility child was not returned") }
guard index >= 0 && index < available,
AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild) == .success,
let item = (selectedChild as? [AXUIElement])?.first else { try fail("element-missing", "Selected accessibility path no longer exists") }
element = item
}
var owner: pid_t = 0
let ownerStatus = AXUIElementGetPid(element, &owner)
guard ownerStatus == .success else { try fail("accessibility-unavailable", "Cannot identify accessibility element owner: AXError \(ownerStatus.rawValue)") }
guard owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
guard AXUIElementGetPid(element, &owner) == .success && owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
let outcome: AXError
switch action.kind {
case "click": outcome = AXUIElementPerformAction(element, kAXPressAction as CFString)
@@ -139,18 +122,10 @@ func observe(_ request: Request) async throws -> [String: Any] {
if nodes.count >= request.max_nodes { truncated = true; break }
let childCountResult = childCount(element)
var actions: CFArray?
let actionStatus = AXUIElementCopyActionNames(element, &actions)
let observedActions = captureActionNames(status: actionStatus.rawValue, success: AXError.success.rawValue, names: actions as? [String])
if let error = observedActions.error { gaps.append("AX actions unavailable at path \(path): AXError \(error)"); truncated = true }
let role = attribute(element, kAXRoleAttribute)
let title = attribute(element, kAXTitleAttribute)
let value = attribute(element, kAXValueAttribute)
for (name, error) in [("role", role.error), ("title", title.error), ("value", value.error)] {
if let error { gaps.append("AX \(name) unavailable at path \(path): AXError \(error)"); truncated = true }
}
let observedValue = captureAccessibilityValue(value.value)
AXUIElementCopyActionNames(element, &actions)
let observedValue = captureAccessibilityValue(attribute(element, kAXValueAttribute))
if let gap = observedValue.gap { gaps.append("\(gap) at path \(path)"); truncated = true }
nodes.append(["path": path, "role": (role.value as? String) ?? NSNull() as Any, "title": (title.value as? String) ?? NSNull() as Any, "value": observedValue.value, "actions": observedActions.names, "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": observedValue.value, "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
guard let totalChildren = childCountResult.value else {
truncated = true
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
+43 -58
View File
@@ -25,17 +25,25 @@ ARGUMENT_REGISTERS = {
"x86_64": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
"x86_64h": ["rdi", "rsi", "rdx", "rcx", "r8", "r9"],
}
MAX_SELECTOR_BYTES = 1024
MAX_REPORTED_LOCATIONS = 64
MAX_OTHER_STOPS = 256
MAX_TRACE_BYTES = 8 * 1024 * 1024
MAX_TRACE_FRAMES = 65_536
MAX_OBSERVATION_JOURNAL_BYTES = MAX_TRACE_BYTES
MAX_BREAKPOINT_LOCATION_BYTES = 8 * 1024 * 1024
MAX_TARGET_OUTPUT_BYTES = 1024 * 1024
OUTPUT_DRAIN_BYTES_PER_PASS = 1024 * 1024
OUTPUT_EOF_WAIT_SECONDS = 1.0
OUTPUT_READER_JOIN_SECONDS = 0.5
LISTENER_NAME = "rea-native-call-observation"
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames, limits):
def _can_retain_trace(current_bytes, current_frames, event_bytes, event_frames):
"""Whether a complete event fits both aggregate trace budgets."""
return (
current_bytes + event_bytes <= limits["trace_bytes"]
and current_frames + event_frames <= limits["trace_frames"]
current_bytes + event_bytes <= MAX_TRACE_BYTES
and current_frames + event_frames <= MAX_TRACE_FRAMES
)
@@ -64,7 +72,7 @@ def _fits_trace_budget(value, remaining_bytes):
class _ObservationJournal:
"""Flush a bounded JSONL prefix so observations survive tracer failure."""
def __init__(self, handle, max_bytes):
def __init__(self, handle, max_bytes=MAX_OBSERVATION_JOURNAL_BYTES):
self.handle = handle
self.max_bytes = max_bytes
self.bytes = 0
@@ -236,20 +244,6 @@ def _hex(value):
return "0x%x" % value
def _source_location(address):
"""Preserve available line-entry facets without acquiring source files."""
entry = address.GetLineEntry()
if not entry.IsValid():
return None
file_spec = entry.GetFileSpec()
file_path = str(file_spec) if file_spec.IsValid() else None
line = entry.GetLine() or None
column = entry.GetColumn() or None
if not file_path and line is None and column is None:
return None
return {"file_path": file_path or None, "line": line, "column": column}
def _location(address, target):
"""Load and file address, module and symbol of one code address."""
module = address.GetModule()
@@ -263,7 +257,6 @@ def _location(address, target):
"module": module.GetFileSpec().GetFilename() if module.IsValid() else None,
"module_path": str(module.GetFileSpec()) if module.IsValid() else None,
"symbol": symbol.GetName() if symbol.IsValid() else None,
"source_location": _source_location(address),
}
return location
@@ -364,7 +357,7 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
error = lldb.SBError()
text = process.ReadCStringFromMemory(
values[names[1]],
config["capture_limits"]["selector_bytes"],
MAX_SELECTOR_BYTES,
error,
)
selector = text if error.Success() and text else None
@@ -397,16 +390,10 @@ def _record(context, thread, frame, index, sequence, location, remaining_bytes):
_TRACE_CONTEXT = None
def _resource_limit(context, limit):
context["resource_limit"] = limit
return "resource-limit"
def _handle_breakpoint(context, frame, index):
"""Read one hit while LLDB's breakpoint callback owns the stopped frame."""
if not frame.IsValid() or frame.GetPC() == lldb.LLDB_INVALID_ADDRESS:
raise RuntimeError("LLDB breakpoint callback did not supply a readable frame")
limits = context["config"]["capture_limits"]
thread = frame.GetThread()
events = context["events"]
location = _location(frame.GetPCAddress(), context["target"])
@@ -423,50 +410,49 @@ def _handle_breakpoint(context, frame, index):
context["retained_frames"],
0,
event_frame_count,
limits,
):
return _resource_limit(context, "trace_frames")
return "resource-limit"
try:
remaining_bytes = limits["trace_bytes"] - context["retained_trace_bytes"]
remaining_bytes = MAX_TRACE_BYTES - context["retained_trace_bytes"]
record = _record(
context, thread, frame, index, len(events), location, remaining_bytes
)
except ValueError:
return _resource_limit(context, "trace_bytes")
return "resource-limit"
if record is not None:
if not _fits_trace_budget(record, remaining_bytes):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
record_bytes = len(json.dumps(record).encode("utf-8"))
if context["retained_trace_bytes"] + record_bytes > limits["trace_bytes"]:
return _resource_limit(context, "trace_bytes")
if context["retained_trace_bytes"] + record_bytes > MAX_TRACE_BYTES:
return "resource-limit"
for depth in range(1, frame_count + 1):
try:
location = _location(
thread.GetFrameAtIndex(depth).GetPCAddress(), context["target"]
)
except ValueError:
return _resource_limit(context, "trace_bytes")
return "resource-limit"
if (
_estimated_json_size(location) + record_bytes + 1
> limits["trace_bytes"] - context["retained_trace_bytes"]
> MAX_TRACE_BYTES - context["retained_trace_bytes"]
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if record["backtrace"] else 0)
if (
context["retained_trace_bytes"]
+ record_bytes
+ addition
> limits["trace_bytes"]
> MAX_TRACE_BYTES
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
record["backtrace"].append(location)
record_bytes += addition
journal = context.get("observation_journal")
if journal is not None and not journal.append(
{"kind": "event", "event": record}
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
events.append(record)
context["retained_frames"] += event_frame_count
context["retained_trace_bytes"] += record_bytes
@@ -498,9 +484,9 @@ def _handle_stop(context, other_stops):
if journal is not None and not journal.append(
{"kind": "other-stop", "reason": stop_description}
):
return _resource_limit(context, "trace_bytes")
return "resource-limit"
other_stops.append(stop_description)
if len(other_stops) >= context["config"]["capture_limits"]["other_stops"]:
if len(other_stops) >= MAX_OTHER_STOPS:
return "stop-limit"
return None
@@ -519,19 +505,20 @@ def _breakpoint_report(target, created, config):
if not _wanted(config["breakpoints"][index], location["symbol"] or ""):
continue
count += 1
remaining_bytes = (
config["capture_limits"]["breakpoint_location_bytes"] - retained_location_bytes
)
if not _fits_trace_budget(location, remaining_bytes):
locations_truncated = True
continue
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if locations else 0)
if addition > remaining_bytes:
locations_truncated = True
continue
locations.append(location)
retained_location_bytes += addition
if len(locations) < MAX_REPORTED_LOCATIONS:
remaining_bytes = (
MAX_BREAKPOINT_LOCATION_BYTES - retained_location_bytes
)
if not _fits_trace_budget(location, remaining_bytes):
locations_truncated = True
continue
location_bytes = len(json.dumps(location).encode("utf-8"))
addition = location_bytes + (1 if locations else 0)
if addition > remaining_bytes:
locations_truncated = True
continue
locations.append(location)
retained_location_bytes += addition
report.append({"index": index, "location_count": count, "locations": locations})
return report, locations_truncated
@@ -651,7 +638,7 @@ def trace(debugger, config):
for breakpoint in breakpoints:
breakpoint_index[breakpoint.GetID()] = index
listener = lldb.SBListener(LISTENER_NAME)
output_budget = config["capture_limits"]["output_bytes"]
output_budget = config.get("max_output_bytes", MAX_TARGET_OUTPUT_BYTES)
stdout = _OutputPipe(
config["stdout_path"], config["stdout_capture_path"], output_budget
)
@@ -671,8 +658,7 @@ def trace(debugger, config):
observation_path = config.get("observation_path")
if observation_path:
observation_journal = _ObservationJournal(
open(observation_path, "wb"),
config["capture_limits"]["trace_bytes"],
open(observation_path, "wb")
)
process = target.Launch(_launch_info(target, config, listener), error)
process_created = process.IsValid()
@@ -863,7 +849,6 @@ def trace(debugger, config):
"breakpoint_locations_truncated": breakpoint_locations_truncated,
"events": events,
"resource_limit_reached": outcome == "resource-limit",
"resource_limit": context.get("resource_limit"),
"target_output": {
"stdout_bytes": stdout.bytes,
"stderr_bytes": stderr.bytes,
+40 -153
View File
@@ -25,45 +25,6 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_source_coordinates_preserve_partial_metadata_without_reading_source(self):
def address(valid=True, file_valid=True, line=23, column=7):
class SourceFileSpec:
def IsValid(self):
return file_valid
def __str__(self):
return "/missing/é space/fixture.pas"
file_spec = SourceFileSpec()
entry = types.SimpleNamespace(
IsValid=lambda: valid,
GetFileSpec=lambda: file_spec,
GetLine=lambda: line,
GetColumn=lambda: column,
)
return types.SimpleNamespace(GetLineEntry=lambda: entry)
self.assertEqual(
tracer._source_location(address()),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(column=0)),
{"file_path": "/missing/é space/fixture.pas", "line": 23, "column": None},
)
self.assertEqual(
tracer._source_location(address(file_valid=False)),
{"file_path": None, "line": 23, "column": 7},
)
self.assertEqual(
tracer._source_location(address(line=0)),
{"file_path": "/missing/é space/fixture.pas", "line": None, "column": 7},
)
self.assertIsNone(tracer._source_location(address(valid=False)))
self.assertIsNone(
tracer._source_location(address(file_valid=False, line=0, column=0))
)
def test_invalid_launch_process_reports_explicit_no_creation(self):
class InvalidProcess:
def IsValid(self):
@@ -121,7 +82,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
result = tracer.trace(Debugger(), {
"executable": "fixture", "architecture": "arm64", "expected_sha256": "expected",
"breakpoints": [], "working_directory": "fixture/absent",
"capture_limits": {"output_bytes": 1024},
"stdout_path": "stdout", "stdout_capture_path": "stdout-capture",
"stderr_path": "stderr", "stderr_capture_path": "stderr-capture",
})
@@ -321,44 +281,37 @@ class NativeTraceBudgetTests(unittest.TestCase):
return super().flush()
stream = CountingStream()
journal = tracer._ObservationJournal(stream, max_bytes=4096)
journal = tracer._ObservationJournal(stream)
journal.append({"kind": "event", "event": {"sequence": 0}})
journal.append({"kind": "other-stop", "reason": "signal"})
self.assertEqual((stream.writes, stream.flushes), (2, 2))
self.assertEqual(journal.bytes, len(stream.getvalue()))
def test_aggregate_trace_budget_uses_the_selected_admission(self):
limits = {"trace_bytes": 100, "trace_frames": 10}
self.assertTrue(tracer._can_retain_trace(90, 8, 10, 2, limits))
self.assertFalse(tracer._can_retain_trace(90, 8, 11, 2, limits))
self.assertFalse(tracer._can_retain_trace(90, 8, 10, 3, limits))
raised = {"trace_bytes": 101, "trace_frames": 11}
self.assertTrue(tracer._can_retain_trace(90, 8, 11, 3, raised))
def test_breakpoint_reports_exact_exhausted_budget_and_accepts_more_room(self):
frame = types.SimpleNamespace(
IsValid=lambda: True, GetPC=lambda: 1,
GetThread=lambda: types.SimpleNamespace(GetNumFrames=lambda: 1),
GetPCAddress=lambda: object(),
def test_aggregate_trace_budget_admits_exact_boundary_and_rejects_next_record(self):
self.assertTrue(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
10,
2,
)
)
self.assertFalse(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
11,
2,
)
)
self.assertFalse(
tracer._can_retain_trace(
tracer.MAX_TRACE_BYTES - 10,
tracer.MAX_TRACE_FRAMES - 2,
10,
3,
)
)
context = {
"target": object(), "events": [], "retained_frames": 1,
"retained_trace_bytes": 0,
"config": {"breakpoints": [{}], "backtrace_frames": 0, "max_events": 10,
"capture_limits": {"trace_bytes": 4096, "trace_frames": 1}},
}
with patch.object(tracer, "_location", return_value={"symbol": "main"}), patch.object(
tracer, "_wanted", return_value=True
), patch.object(tracer, "_record", return_value={"backtrace": []}):
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
self.assertEqual(context["resource_limit"], "trace_frames")
context["config"]["capture_limits"] = {"trace_bytes": 8, "trace_frames": 2}
self.assertEqual(tracer._handle_breakpoint(context, frame, 0), "resource-limit")
self.assertEqual(context["resource_limit"], "trace_bytes")
self.assertEqual(context["events"], [])
context["config"]["capture_limits"]["trace_bytes"] = 4096
self.assertIsNone(tracer._handle_breakpoint(context, frame, 0))
self.assertEqual(context["events"], [{"backtrace": []}])
def test_oversized_producer_string_is_rejected_before_serialization(self):
self.assertTrue(tracer._fits_trace_budget("method", 1024))
@@ -437,9 +390,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
return FileSpec()
class Address:
def GetLineEntry(self):
return types.SimpleNamespace(IsValid=lambda: False)
def GetModule(self):
return Module()
@@ -467,7 +417,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
object(),
[[Breakpoint()]],
{
"capture_limits": {"breakpoint_location_bytes": 4096},
"breakpoints": [
{
"kind": "objc-method",
@@ -481,34 +430,6 @@ class NativeTraceBudgetTests(unittest.TestCase):
self.assertEqual(report[0]["locations"], [])
self.assertFalse(locations_truncated)
def test_breakpoint_report_keeps_all_locations_under_aggregate_budget(self):
class Location:
def __init__(self, index):
self.index = index
def GetAddress(self):
return self.index
class Breakpoint:
def GetNumLocations(self):
return 65
def GetLocationAtIndex(self, index):
return Location(index)
with patch.object(
tracer,
"_location",
side_effect=lambda address, _target: {"address": address, "symbol": None},
), patch.object(tracer, "_wanted", return_value=True):
report, locations_truncated = tracer._breakpoint_report(
object(), [[Breakpoint()]], {"breakpoints": [{}], "capture_limits": {"breakpoint_location_bytes": 4096}}
)
self.assertEqual(report[0]["location_count"], 65)
self.assertEqual(len(report[0]["locations"]), 65)
self.assertFalse(locations_truncated)
def test_breakpoint_report_bounds_aggregate_location_json(self):
tracer.lldb.LLDB_INVALID_ADDRESS = -1
symbol_name = (
@@ -542,24 +463,7 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetFileSpec(self):
return FileSpec()
class SourceFileSpec:
def IsValid(self):
return True
def __str__(self):
return source_path
class Address:
def GetLineEntry(self):
if source_path is None:
return types.SimpleNamespace(IsValid=lambda: False)
return types.SimpleNamespace(
IsValid=lambda: True,
GetFileSpec=SourceFileSpec,
GetLine=lambda: 15,
GetColumn=lambda: 3,
)
def GetModule(self):
return Module()
@@ -583,39 +487,22 @@ class NativeTraceBudgetTests(unittest.TestCase):
def GetLocationAtIndex(self, index):
return Location()
breakpoint_count = 30
breakpoint_count = 1000
created = [[Breakpoint()] for _ in range(breakpoint_count)]
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count,
"capture_limits": {"breakpoint_location_bytes": 512 * 1024}}
for source_path in (None, "/tmp/" + "源🚀" * 3000 + "/calls.pas"):
with self.subTest(source_path_present=source_path is not None):
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(
sum(row["location_count"] for row in report), 1920
)
self.assertLess(
sum(len(row["locations"]) for row in report), 1920
)
self.assertLessEqual(
retained_bytes, 512 * 1024
)
if source_path is not None:
self.assertTrue(
any(
location["source_location"]["file_path"] == source_path
for row in report
for location in row["locations"]
)
)
config = {"breakpoints": [{"kind": "objc-method"}] * breakpoint_count}
report, locations_truncated = tracer._breakpoint_report(
object(), created, config
)
retained_bytes = sum(
len(json.dumps(location).encode("utf-8"))
+ (1 if index > 0 else 0)
for row in report
for index, location in enumerate(row["locations"])
)
self.assertTrue(locations_truncated)
self.assertEqual(sum(row["location_count"] for row in report), 64_000)
self.assertLess(sum(len(row["locations"]) for row in report), 64_000)
self.assertLessEqual(retained_bytes, tracer.MAX_BREAKPOINT_LOCATION_BYTES)
if __name__ == "__main__":
+5 -7
View File
@@ -78,9 +78,14 @@ public final class NativeAotExtension {
new FrozenObjectAnnotator(program, manager).analyze(located.directory(), scan.getPointerLocations(), monitor, log);
}
monitor.checkCancelled();
report.addProperty("method_tables", manager.getMethodTableCount());
JsonObject coverage = auditFrozenObjects(program, scan.getPointerLocations(), monitor);
report.add("coverage", coverage);
if (coverage.get("frozen_objects_annotated").getAsInt() != coverage.get("frozen_object_candidates").getAsInt())
diagnostics.add("Some frozen-object candidates lack committed instance annotations; inspect coverage counts and Ghidra diagnostics.");
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
JsonArray tables = new JsonArray();
for (var mt : manager.getMethodTables()) {
monitor.checkCancelled();
@@ -89,13 +94,6 @@ public final class NativeAotExtension {
identity.addProperty("type", mt.getMTType().getPathName());
tables.add(identity);
}
// Publish recovery facts only after every fallible recovery and audit step.
// A failed transaction is rolled back, so it cannot report committed metadata.
report.addProperty("method_tables", manager.getMethodTableCount());
report.add("coverage", coverage);
report.addProperty("status", diagnostics.isEmpty() ? "complete" : "partial");
report.add("reason", JsonNull.INSTANCE);
report.add("derived_memory", derivedRange.deepCopy());
report.add("types", tables);
commit = true;
return finish(log);
+3 -3
View File
@@ -5,6 +5,7 @@ from pathlib import Path
import resource
import gdb
OUTPUT_BYTES = 64 * 1024 * 1024
class ContextFailure(Exception):
@@ -78,7 +79,6 @@ def failure_exit(request_path, status, marker):
def main():
request_path = os.environ["REA_CORE_CONTEXT_REQUEST"]
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
output_bytes = request["producer_limits"]["output_bytes"]
reserve = bytearray(1024 * 1024)
try:
reply = {"ok": True, "value": inspect(request)}
@@ -94,8 +94,8 @@ def main():
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
try:
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
if len(encoded) > output_bytes:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds producer_limits.output_bytes=" + str(output_bytes) + "."}).encode("utf-8")
if len(encoded) > OUTPUT_BYTES:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds the 64 MiB reply budget."}).encode("utf-8")
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
+6 -7
View File
@@ -14,7 +14,7 @@ BRIDGE = Path(__file__).with_name("core_context.py")
class CoreContextTests(unittest.TestCase):
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable", output_bytes=16384):
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable"):
commands, reads, queries = [], [], []
state = {"executable": None, "symbols": None, "connection": None}
gdb = types.ModuleType("gdb")
@@ -63,7 +63,7 @@ class CoreContextTests(unittest.TestCase):
plugin.write_text("")
request = stage / "request.json"
reply = stage / "reply.json"
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply), "producer_limits": {"output_bytes": output_bytes}}))
request.write_text(json.dumps({"gdbinit": str(plugin), "architecture": architecture, "snapshot_path": str(root / "object.snapshot"), "reply_path": str(reply)}))
with patch.dict(sys.modules, modules), patch.dict(os.environ, REA_CORE_CONTEXT_REQUEST=str(request)):
runpy.run_path(str(BRIDGE))
result = json.loads(reply.read_text())
@@ -104,15 +104,14 @@ class CoreContextTests(unittest.TestCase):
self.assertEqual(reads, [])
self.assertEqual(queries, [])
def test_selected_output_limit_reports_recovery_and_accepts_more_headroom(self):
result, _, reads, queries = self.inspect(objfile="m" * 2048, output_bytes=1024)
def test_output_limit_message_separates_the_words_and_names_the_limit(self):
result, _, reads, queries = self.inspect(objfile="m" * (64 * 1024 * 1024))
self.assertFalse(result["ok"])
self.assertEqual(result["reason"], "output-limit")
self.assertIn("producer_limits.output_bytes=1024", result["message"])
self.assertNotIn("exceeds64", result["message"])
self.assertEqual(result["message"], "Complete debugger context exceeds the 64 MiB reply budget.")
self.assertEqual(reads, [])
self.assertEqual(queries, ["maps"])
recovered, _, _, _ = self.inspect(objfile="m" * 2048, output_bytes=4096)
self.assertTrue(recovered["ok"], recovered)
def test_wrong_connection_or_remaining_symbols_never_queries_mappings(self):
for options in [{"connection": "native"}, {"symbols": "unexpected-symbols"}]:
+9 -12
View File
@@ -5,12 +5,14 @@ from pathlib import Path
import resource
import sys
def lower_resource_limits(policy):
OUTPUT_BYTES = 64 * 1024 * 1024
def lower_resource_limits():
limits = {}
for name, kind, maximum in (
("address_space_bytes", resource.RLIMIT_AS, policy["address_space_bytes"]),
("cpu_seconds", resource.RLIMIT_CPU, policy["cpu_seconds"]),
("file_size_bytes", resource.RLIMIT_FSIZE, policy["file_bytes"]),
("address_space_bytes", resource.RLIMIT_AS, 3 * 1024**3),
("cpu_seconds", resource.RLIMIT_CPU, 30),
("file_size_bytes", resource.RLIMIT_FSIZE, OUTPUT_BYTES),
):
soft, hard = resource.getrlimit(kind)
finite = [maximum] + [value for value in (soft, hard) if value != resource.RLIM_INFINITY]
@@ -23,11 +25,6 @@ def lower_resource_limits(policy):
def main(request_path, inspect, failure_type, profile):
request = json.loads(Path(request_path).read_text(encoding="utf-8"))
policy = request["producer_limits"]
for key in ("input_bytes", "output_bytes", "diagnostic_bytes", "address_space_bytes", "file_bytes", "cpu_seconds"):
value = policy[key]
if type(value) is not int or not 1 <= value <= 9007199254740991:
raise ValueError("producer_limits." + key + " must be a positive safe integer")
os.environ["PWNLIB_NOTERM"] = "1"
os.environ["PWNLIB_CACHE_DIR"] = str(Path(request_path).parent / "cache")
limits = None
@@ -35,7 +32,7 @@ def main(request_path, inspect, failure_type, profile):
# The outer exit-status contract also covers allocation failure during serialization.
memory_reserve = bytearray(1024 * 1024)
try:
limits = lower_resource_limits(policy)
limits = lower_resource_limits()
descriptor = os.open(Path(request_path).parent / "limits.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump(limits, handle)
@@ -54,8 +51,8 @@ def main(request_path, inspect, failure_type, profile):
else:
reply = {"ok": False, "reason": "decoder", "message": type(error).__name__ + ": " + str(error)}
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
if len(encoded) > policy["output_bytes"]:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds producer_limits.output_bytes=" + str(policy["output_bytes"]) + "."}).encode("utf-8")
if len(encoded) > OUTPUT_BYTES:
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete artifact result exceeds the 64 MiB reply budget."}).encode("utf-8")
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
+92 -88
View File
@@ -1,18 +1,4 @@
import { defineConfig } from "vitepress";
import { dirname, relative, resolve, sep } from "node:path";
import { fileURLToPath } from "node:url";
import {
documentationRewrites,
documentationSections,
} from "./documentation.js";
const documentationRoot = fileURLToPath(new URL("..", import.meta.url));
const rewrites: Record<string, string> = {
...documentationRewrites,
"README.md": "documentation.md",
"specs/README.md": "specs/index.md",
"adr/README.md": "adr/index.md",
};
export default defineConfig({
title: "REA",
@@ -20,92 +6,110 @@ export default defineConfig({
"Reverse engineer binaries, applications, and runtime behavior with one CLI and MCP server.",
lang: "en-US",
base: "/rea/",
rewrites,
markdown: {
config(md) {
// Repository links resolve from their physical source. Render them from
// the stable public route instead, including links across categories.
md.core.ruler.after("inline", "documentation-source-links", (state) => {
const physicalPath =
state.env.realPath ??
(state.env.relativePath
? resolve(documentationRoot, state.env.relativePath)
: null);
if (physicalPath === null) return;
const renderedPath =
state.env.path ??
resolve(
documentationRoot,
rewrites[state.env.relativePath] ?? state.env.relativePath,
);
for (const block of state.tokens) {
for (const token of block.children ?? []) {
const attribute =
token.type === "link_open"
? "href"
: token.type === "image"
? "src"
: null;
if (attribute === null) continue;
const destination = token.attrGet(attribute);
if (
!destination ||
/^(?:[a-z][a-z\d+.-]*:|[/#])/iu.test(destination)
)
continue;
const separator = destination.search(/[?#]/u);
const path =
separator === -1 ? destination : destination.slice(0, separator);
const suffix = separator === -1 ? "" : destination.slice(separator);
const source = relative(
documentationRoot,
resolve(dirname(physicalPath), path),
)
.split(sep)
.join("/");
const rendered = relative(
dirname(renderedPath),
resolve(
documentationRoot,
source.startsWith("public/")
? source.slice("public/".length)
: (rewrites[source] ?? source),
),
)
.split(sep)
.join("/");
token.attrSet(attribute, rendered + suffix);
}
}
});
},
},
srcExclude: ["readme/**"],
themeConfig: {
nav: [
{ text: "Get started", link: "/installation" },
{ text: "Documentation", link: "/documentation" },
{ text: "Reference", link: "/mcp-contracts" },
{ text: "Roadmap", link: "/roadmap" },
],
sidebar: [
{
text: "Documentation directory",
text: "Getting started",
items: [
{ text: "All guides", link: "/documentation" },
{ text: "Authoring conventions", link: "/development/documentation" },
{ text: "Feature specifications", link: "/specs/" },
{ text: "Architecture decisions", link: "/adr/" },
{ text: "Installation and setup", link: "/installation" },
{ text: "Client configuration", link: "/client-configuration" },
{ text: "CLI and Evidence", link: "/cli" },
{ text: "Agent prompts", link: "/mcp-prompts" },
{ text: "Roadmap", link: "/roadmap" },
],
},
{
text: "Investigation guides",
items: [
{ text: "Native binaries", link: "/native-investigation" },
{ text: "Go build metadata", link: "/go-binaries" },
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
{ text: "Model artifacts", link: "/model-artifacts" },
{ text: "Rust dependencies", link: "/rust-binaries" },
{ text: "PE certificates", link: "/pe-certificates" },
{ text: "PE imports", link: "/pe-imports" },
{ text: "PE TLS/load config", link: "/pe-load-config" },
{ text: "ELF notes", link: "/elf-notes" },
{ text: "Apple IPS crash reports", link: "/apple-ips" },
{ text: "Linux kernel modules", link: "/kernel-modules" },
{ text: "CPython pyc files", link: "/python-pyc" },
{ text: "Erlang BEAM", link: "/erlang-beam" },
{
text: "JavaScript and Electron",
link: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript workflows",
link: "/javascript-application-workflows",
},
{ text: "Browser observation", link: "/browser-observation" },
{
text: "Web network captures",
link: "/web-network-captures",
},
{
text: "Build RL environments",
link: "/agent-environments-quickstart",
},
{ text: "Environment reference", link: "/agent-environments" },
{ text: "Electron observation", link: "/electron-observation" },
{ text: "Electron fuses", link: "/electron-fuses" },
{ text: "Process capture", link: "/process-capture" },
{ text: ".NET assemblies", link: "/managed-code-analysis" },
{
text: "Python packaged applications",
link: "/python-packaged-applications",
},
{ text: "Android applications", link: "/android-analysis" },
{ text: "Apple applications", link: "/apple-application-analysis" },
{ text: "Desktop installations", link: "/desktop-applications" },
{ text: "Firmware", link: "/firmware-analysis" },
{ text: "iNES containers", link: "/ines-containers" },
{ text: "SQLite databases", link: "/sqlite-databases" },
{ text: "Packet captures", link: "/packet-captures" },
{ text: "Signal recordings", link: "/signal-recordings" },
],
},
{
text: "Provider guides",
collapsed: true,
items: [
{ text: "IDA", link: "/ida-provider" },
{ text: "Kuna", link: "/kuna-provider" },
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
{ text: "Ghidra raw binary", link: "/raw-binary" },
{ text: "Ghidra NativeAOT", link: "/ghidra-nativeaot" },
{ text: "Ghidra PowerPC", link: "/ghidra-ppc" },
{ text: "Ghidra GameCube DOL", link: "/ghidra-dol" },
{ text: "Ghidra semantics", link: "/ghidra-provider" },
],
},
{
text: "Reference and development",
collapsed: true,
items: [
{ text: "MCP contracts", link: "/mcp-contracts" },
{ text: "Tool design", link: "/tool-design" },
{
text: "Admission limits and recovery",
link: "/design/limits-and-recovery",
},
{ text: "Testing", link: "/testing" },
{ text: "Client verification", link: "/client-verification" },
{
text: "Reconstruction readiness",
link: "/reconstruction-readiness",
},
{ text: "Releasing", link: "/releasing" },
],
},
...documentationSections.map((section) => ({
text: section.text,
collapsed: section.text !== "Getting started",
items: section.items.map((item) => ({
text: item.text,
link: item.route,
})),
})),
],
search: { provider: "local" },
socialLinks: [
-684
View File
@@ -1,684 +0,0 @@
// Authored guide ownership: physical sources and stable public routes.
export const documentationSections = [
{
text: "Getting started",
items: [
{
text: "Installation and setup",
source: "getting-started/installation.md",
route: "/installation",
},
{
text: "Client configuration and troubleshooting",
source: "getting-started/client-configuration.md",
route: "/client-configuration",
},
{
text: "CLI and Evidence",
source: "getting-started/cli.md",
route: "/cli",
},
{
text: "Guided MCP prompts and completion",
source: "getting-started/mcp-prompts.md",
route: "/mcp-prompts",
},
],
},
{
text: "Native binaries",
items: [
{
text: "Native binary investigation",
source: "guides/native/native-investigation.md",
route: "/native-investigation",
},
{
text: "Go binary build metadata",
source: "guides/native/go-binaries.md",
route: "/go-binaries",
},
{
text: "Offline binary diagnostics",
source: "guides/native/binary-diagnostics.md",
route: "/binary-diagnostics",
},
{
text: "ELF and Mach-O initializer functions",
source: "guides/native/native-initializers.md",
route: "/native-initializers",
},
{
text: "ELF unwind function ranges",
source: "guides/native/elf-unwind-ranges.md",
route: "/elf-unwind-ranges",
},
{
text: "Linux kernel build config",
source: "guides/native/kernel-build-config.md",
route: "/kernel-build-config",
},
{
text: "Recorded crash evidence",
source: "guides/native/recorded-crashes.md",
route: "/recorded-crashes",
},
{
text: "Windows Shell Link shortcuts",
source: "guides/native/windows-shortcut.md",
route: "/windows-shortcut",
},
],
},
{
text: "JavaScript and Electron",
items: [
{
text: "JavaScript Application Graph",
source: "guides/javascript/javascript-application-graph.md",
route: "/javascript-application-graph",
},
{
text: "Cross-layer JavaScript application workflows",
source: "guides/javascript/javascript-application-workflows.md",
route: "/javascript-application-workflows",
},
{
text: "JavaScript artifact reconstruction",
source: "guides/javascript/javascript-artifact-reconstruction.md",
route: "/javascript-artifact-reconstruction",
},
{
text: "JavaScript source recovery",
source: "guides/javascript/javascript-recovery.md",
route: "/javascript-recovery",
},
{
text: "Passive Node and Electron runtime observation",
source: "guides/javascript/javascript-runtime-observation.md",
route: "/javascript-runtime-observation",
},
{
text: "JavaScript static/runtime reconciliation",
source: "guides/javascript/javascript-runtime-reconciliation.md",
route: "/javascript-runtime-reconciliation",
},
{
text: "Electron file-page observation",
source: "guides/javascript/electron-observation.md",
route: "/electron-observation",
},
{
text: "V8 code cache inspection",
source: "guides/javascript/v8-code-cache.md",
route: "/v8-code-cache",
},
],
},
{
text: "Browser and captured websites",
items: [
{
text: "Website observation with CDP",
source: "guides/web/browser-observation.md",
route: "/browser-observation",
},
{
text: "Browser scenario contract",
source: "guides/web/browser-scenario-contract.md",
route: "/browser-scenario-contract",
},
{
text: "Website runtime source attribution",
source: "guides/web/web-runtime.md",
route: "/web-runtime",
},
{
text: "Historical web network captures",
source: "guides/web/web-network-captures.md",
route: "/web-network-captures",
},
{
text: "HTTP/3 capture lab",
source: "http3-capture-lab.md",
route: "/http3-capture-lab",
},
{
text: "Captured website source locations",
source: "guides/web/web-source-location.md",
route: "/web-source-location",
},
{
text: "Captured website module relationships",
source: "guides/web/website-module-trace.md",
route: "/website-module-trace",
},
{
text: "Captured website scripts and static JavaScript analysis",
source: "guides/web/website-script-export.md",
route: "/website-script-export",
},
],
},
{
text: "Android applications",
items: [
{
text: "Static Android APK analysis",
source: "guides/android/android-analysis.md",
route: "/android-analysis",
},
{
text: "Apktool resource decoding",
source: "guides/android/apktool-resource-analysis.md",
route: "/apktool-resource-analysis",
},
{
text: "ADB device inspection and acquisition",
source: "guides/android/adb-device-analysis.md",
route: "/adb-device-analysis",
},
{
text: "Flutter build identification",
source: "guides/android/flutter-analysis.md",
route: "/flutter-analysis",
},
],
},
{
text: "Apple applications",
items: [
{
text: "Apple Application Inventory Projection",
source: "guides/apple/apple-application-analysis.md",
route: "/apple-application-analysis",
},
],
},
{
text: ".NET assemblies",
items: [
{
text: "Managed-code analysis",
source: "guides/managed/managed-code-analysis.md",
route: "/managed-code-analysis",
},
],
},
{
text: "Firmware",
items: [
{
text: "Firmware analysis",
source: "guides/firmware/firmware-analysis.md",
route: "/firmware-analysis",
},
],
},
{
text: "HarmonyOS packages",
items: [
{
text: "Static HarmonyOS package inventory",
source: "guides/harmony/harmony-analysis.md",
route: "/harmony-analysis",
},
],
},
{
text: "WebAssembly",
items: [
{
text: "Offline WebAssembly artifacts",
source: "guides/wasm/wasm-artifacts.md",
route: "/wasm-artifacts",
},
],
},
{
text: "EVM contracts",
items: [
{
text: "Offline EVM bytecode",
source: "guides/evm/evm-bytecode.md",
route: "/evm-bytecode",
},
],
},
{
text: "Databases and recorded packets",
items: [
{
text: "SQLite database snapshots",
source: "guides/data/sqlite-databases.md",
route: "/sqlite-databases",
},
{
text: "Recorded packet capture inspection",
source: "guides/data/packet-captures.md",
route: "/packet-captures",
},
{
text: "Packet capture summaries",
source: "guides/data/pcap-analysis.md",
route: "/pcap-analysis",
},
],
},
{
text: "Process capture and instrumentation",
items: [
{
text: "Process capture",
source: "guides/runtime/process-capture.md",
route: "/process-capture",
},
{
text: "Frida instrumentation",
source: "guides/runtime/frida-instrumentation.md",
route: "/frida-instrumentation",
},
],
},
{
text: "Provider setup and formats",
items: [
{
text: "Ghidra provider semantics",
source: "providers/ghidra-provider.md",
route: "/ghidra-provider",
},
{
text: "DOS MZ and COM analysis with Ghidra",
source: "providers/ghidra-dos.md",
route: "/ghidra-dos",
},
{
text: "Optional NativeAOT metadata recovery",
source: "providers/ghidra-nativeaot.md",
route: "/ghidra-nativeaot",
},
{
text: "MIPS ELF analysis through Ghidra",
source: "providers/ghidra-mips.md",
route: "/ghidra-mips",
},
{
text: "PSP static ELF analysis through Ghidra",
source: "providers/ghidra-psp.md",
route: "/ghidra-psp",
},
{
text: "RISC-V ELF analysis through Ghidra",
source: "providers/ghidra-riscv.md",
route: "/ghidra-riscv",
},
{
text: "Windows Ghidra P0",
source: "providers/windows-ghidra-p0.md",
route: "/windows-ghidra-p0",
},
{
text: "Headerless raw-binary analysis with Ghidra",
source: "providers/raw-binary.md",
route: "/raw-binary",
},
{
text: "IDA provider",
source: "providers/ida-provider.md",
route: "/ida-provider",
},
{
text: "JEB instrumentation and analysis",
source: "providers/jeb-analysis.md",
route: "/jeb-analysis",
},
],
},
{
text: "Contracts and evidence",
items: [
{
text: "MCP runtime contracts",
source: "reference/mcp-contracts.md",
route: "/mcp-contracts",
},
{
text: "Reconstruction obligation ledgers",
source: "reference/reconstruction-obligation-ledgers.md",
route: "/reconstruction-obligation-ledgers",
},
],
},
{
text: "Development and verification",
items: [
{
text: "Settings and recovery",
source: "design/limits-and-recovery.md",
route: "/design/limits-and-recovery",
},
{
text: "MCP tool design",
source: "development/tool-design.md",
route: "/tool-design",
},
{
text: "Testing REA",
source: "development/testing.md",
route: "/testing",
},
{
text: "Client and agent verification",
source: "development/client-verification.md",
route: "/client-verification",
},
{
text: "Reconstruction readiness conformance",
source: "development/reconstruction-readiness.md",
route: "/reconstruction-readiness",
},
{
text: "Releasing REA",
source: "development/releasing.md",
route: "/releasing",
},
{
text: "REA roadmap",
source: "development/roadmap.md",
route: "/roadmap",
},
],
},
{
text: "Release migrations",
items: [
{
text: "Upgrading from REA 6.1 to 6.2",
source: "migrations/migration-6.2.md",
route: "/migration-6.2",
},
],
},
{
text: "Signal recordings",
items: [
{
text: "Recorded signal inspection",
source: "guides/signals/signal-recordings.md",
route: "/signal-recordings",
},
],
},
{
text: "Additional guides",
items: [
{
text: "Offline API contract inventory",
source: "api-contracts.md",
route: "/api-contracts",
},
{
text: "Bun standalone executables",
source: "bun-standalone.md",
route: "/bun-standalone",
},
{
text: "Deno compile executables",
source: "deno-standalone.md",
route: "/deno-standalone",
},
{
text: "Capability composition and ownership",
source: "capability-composition.md",
route: "/capability-composition",
},
{
text: ".NET single-file bundles",
source: "dotnet-bundles.md",
route: "/dotnet-bundles",
},
{
text: "ESP flash images",
source: "esp-flash-image.md",
route: "/esp-flash-image",
},
{
text: "U-Boot FIT images",
source: "fit-images.md",
route: "/fit-images",
},
{
text: "Measure MCP catalog costs in Codex",
source: "mcp-catalog-evaluation.md",
route: "/mcp-catalog-evaluation",
},
{
text: "Offline Playwright traces",
source: "playwright-traces.md",
route: "/playwright-traces",
},
{
text: "Testing lanes and provider verification",
source: "testing-lanes.md",
route: "/testing-lanes",
},
{
text: "UF2 and DfuSe images",
source: "uf2-dfuse.md",
route: "/uf2-dfuse",
},
{
text: "WeChat mini-program wxapkg archives",
source: "wxapkg.md",
route: "/wxapkg",
},
{
text: "Binary hardening",
source: "binary-hardening.md",
route: "/binary-hardening",
},
{
text: "Desktop application capture",
source: "desktop-application-capture.md",
route: "/desktop-application-capture",
},
{
text: "Desktop application export",
source: "desktop-application-export.md",
route: "/desktop-application-export",
},
{
text: "Desktop application state",
source: "desktop-application-state.md",
route: "/desktop-application-state",
},
{
text: "Desktop application tasks",
source: "desktop-application-tasks.md",
route: "/desktop-application-tasks",
},
{
text: "Desktop build computer",
source: "desktop-build-computer.md",
route: "/desktop-build-computer",
},
{
text: "Desktop build witness",
source: "desktop-build-witness.md",
route: "/desktop-build-witness",
},
{
text: "iNES containers",
source: "ines-containers.md",
route: "/ines-containers",
},
{
text: "Build RL environments with REA",
source: "agent-environments-quickstart.md",
route: "/agent-environments-quickstart",
},
{
text: "Agent environments from recorded captures",
source: "agent-environments.md",
route: "/agent-environments",
},
{
text: "APK signature schemes",
source: "apk-signature.md",
route: "/apk-signature",
},
{
text: "Apple IPS crash reports",
source: "apple-ips.md",
route: "/apple-ips",
},
{
text: "Static analysis in GitHub Actions",
source: "ci-analysis.md",
route: "/ci-analysis",
},
{
text: "Desktop installation directories",
source: "desktop-applications.md",
route: "/desktop-applications",
},
{
text: "Flattened devicetree inspection",
source: "devicetree.md",
route: "/devicetree",
},
{
text: "Electron fuse state",
source: "electron-fuses.md",
route: "/electron-fuses",
},
{ text: "ELF note tables", source: "elf-notes.md", route: "/elf-notes" },
{
text: "Erlang BEAM files",
source: "erlang-beam.md",
route: "/erlang-beam",
},
{ text: "ESP images", source: "esp-image.md", route: "/esp-image" },
{
text: "GameCube DOL analysis through Ghidra",
source: "ghidra-dol.md",
route: "/ghidra-dol",
},
{
text: "PowerPC ELF analysis through Ghidra",
source: "ghidra-ppc.md",
route: "/ghidra-ppc",
},
{
text: "Godot PCK archives",
source: "godot-pck.md",
route: "/godot-pck",
},
{
text: "Static CUDA and HIP code objects",
source: "gpu-code-objects.md",
route: "/gpu-code-objects",
},
{
text: "Intel HEX and S-record images",
source: "hex-records.md",
route: "/hex-records",
},
{
text: "IL2CPP metadata",
source: "il2cpp-metadata.md",
route: "/il2cpp-metadata",
},
{
text: "Static JavaScript route inventory",
source: "javascript-route-inventory.md",
route: "/javascript-route-inventory",
},
{
text: "Linux kernel modules",
source: "kernel-modules.md",
route: "/kernel-modules",
},
{
text: "Kuna provider",
source: "kuna-provider.md",
route: "/kuna-provider",
},
{
text: "Model file declarations",
source: "model-artifacts.md",
route: "/model-artifacts",
},
{
text: "Match native functions across builds",
source: "native-function-matching.md",
route: "/native-function-matching",
},
{
text: "Node.js single executable applications",
source: "node-sea.md",
route: "/node-sea",
},
{
text: "PE Authenticode certificate tables",
source: "pe-certificates.md",
route: "/pe-certificates",
},
{
text: "PE import and delay-load tables",
source: "pe-imports.md",
route: "/pe-imports",
},
{
text: "PE TLS callbacks and load configuration",
source: "pe-load-config.md",
route: "/pe-load-config",
},
{
text: "Python packaged applications",
source: "python-packaged-applications.md",
route: "/python-packaged-applications",
},
{
text: "CPython pyc inspection",
source: "python-pyc.md",
route: "/python-pyc",
},
{
text: "Reconstruction coverage",
source: "reconstruction-coverage.md",
route: "/reconstruction-coverage",
},
{
text: "Reverse-engineering tool integrations",
source: "reverse-engineering-tools.md",
route: "/reverse-engineering-tools",
},
{
text: "Rust binary dependency metadata",
source: "rust-binaries.md",
route: "/rust-binaries",
},
{
text: "U-Boot legacy uImage containers",
source: "uimage.md",
route: "/uimage",
},
{
text: "UnityFS AssetBundle archives",
source: "unityfs.md",
route: "/unityfs",
},
],
},
];
export const documentationRewrites = Object.fromEntries(
documentationSections.flatMap((section) =>
section.items.flatMap((item) => {
const target = `${item.route.slice(1)}.md`;
return item.source === target ? [] : [[item.source, target]];
}),
),
);
-205
View File
@@ -1,205 +0,0 @@
# Documentation directory
REA documentation is organized by reader task and document owner. This directory
links every authored guide; the [site](https://rea.tools/) also serves separate
product pages. The VitePress documentation source is this `docs/` tree.
## Getting started
- [Installation and setup](getting-started/installation.md)
- [Client configuration and troubleshooting](getting-started/client-configuration.md)
- [CLI and Evidence](getting-started/cli.md)
- [Guided MCP prompts and completion](getting-started/mcp-prompts.md)
## Native binaries
- [Native binary investigation](guides/native/native-investigation.md)
- [Go binary build metadata](guides/native/go-binaries.md)
- [Offline binary diagnostics](guides/native/binary-diagnostics.md)
- [ELF and Mach-O initializer functions](guides/native/native-initializers.md)
- [ELF unwind function ranges](guides/native/elf-unwind-ranges.md)
- [Linux kernel build config](guides/native/kernel-build-config.md)
- [Recorded crash evidence](guides/native/recorded-crashes.md)
- [Windows Shell Link shortcuts](guides/native/windows-shortcut.md)
## JavaScript and Electron
- [JavaScript Application Graph](guides/javascript/javascript-application-graph.md)
- [Cross-layer JavaScript application workflows](guides/javascript/javascript-application-workflows.md)
- [JavaScript artifact reconstruction](guides/javascript/javascript-artifact-reconstruction.md)
- [JavaScript source recovery](guides/javascript/javascript-recovery.md)
- [Passive Node and Electron runtime observation](guides/javascript/javascript-runtime-observation.md)
- [JavaScript static/runtime reconciliation](guides/javascript/javascript-runtime-reconciliation.md)
- [Electron file-page observation](guides/javascript/electron-observation.md)
- [V8 code cache inspection](guides/javascript/v8-code-cache.md)
## Browser and captured websites
- [Website observation with CDP](guides/web/browser-observation.md)
- [Browser scenario contract](guides/web/browser-scenario-contract.md)
- [Website runtime source attribution](guides/web/web-runtime.md)
- [Historical web network captures](guides/web/web-network-captures.md)
- [HTTP/3 capture lab](http3-capture-lab.md)
- [Captured website source locations](guides/web/web-source-location.md)
- [Captured website module relationships](guides/web/website-module-trace.md)
- [Captured website scripts and static JavaScript analysis](guides/web/website-script-export.md)
## Android applications
- [Static Android APK analysis](guides/android/android-analysis.md)
- [Apktool resource decoding](guides/android/apktool-resource-analysis.md)
- [ADB device inspection and acquisition](guides/android/adb-device-analysis.md)
- [Flutter build identification](guides/android/flutter-analysis.md)
## Apple applications
- [Apple Application Inventory Projection](guides/apple/apple-application-analysis.md)
## .NET assemblies
- [Managed-code analysis](guides/managed/managed-code-analysis.md)
## Firmware
- [Firmware analysis](guides/firmware/firmware-analysis.md)
## HarmonyOS packages
- [Static HarmonyOS package inventory](guides/harmony/harmony-analysis.md)
## WebAssembly
- [Offline WebAssembly artifacts](guides/wasm/wasm-artifacts.md)
## EVM contracts
- [Offline EVM bytecode](guides/evm/evm-bytecode.md)
## Databases and recorded packets
- [SQLite database snapshots](guides/data/sqlite-databases.md)
- [Recorded packet capture inspection](guides/data/packet-captures.md)
- [Packet capture summaries](guides/data/pcap-analysis.md)
## Process capture and instrumentation
- [Process capture](guides/runtime/process-capture.md)
- [Frida instrumentation](guides/runtime/frida-instrumentation.md)
## Signal recordings
- [Recorded signal inspection](guides/signals/signal-recordings.md)
## Provider setup and formats
- [Ghidra provider semantics](providers/ghidra-provider.md)
- [DOS MZ and COM analysis with Ghidra](providers/ghidra-dos.md)
- [Optional NativeAOT metadata recovery](providers/ghidra-nativeaot.md)
- [MIPS ELF analysis through Ghidra](providers/ghidra-mips.md)
- [PSP static ELF analysis through Ghidra](providers/ghidra-psp.md)
- [RISC-V ELF analysis through Ghidra](providers/ghidra-riscv.md)
- [Windows Ghidra P0](providers/windows-ghidra-p0.md)
- [Headerless raw-binary analysis with Ghidra](providers/raw-binary.md)
- [IDA provider](providers/ida-provider.md)
- [JEB instrumentation and analysis](providers/jeb-analysis.md)
## Contracts and evidence
- [MCP runtime contracts](reference/mcp-contracts.md)
- [Reconstruction obligation ledgers](reference/reconstruction-obligation-ledgers.md)
## Development and verification
- [Settings and recovery](design/limits-and-recovery.md)
- [Flutter budget calibration](verification/flutter-budget-calibration.md)
- [Source-map budget calibration](verification/source-map-budget-calibration.md)
- [Source-location delivery qualification](verification/source-location-delivery.md)
- [JavaScript precision calibration](verification/javascript-precision-calibration.md)
- [MCP tool design](development/tool-design.md)
- [Testing REA](development/testing.md)
- [Client and agent verification](development/client-verification.md)
- [Reconstruction readiness conformance](development/reconstruction-readiness.md)
- [Releasing REA](development/releasing.md)
- [REA roadmap](development/roadmap.md)
## Release migrations
- [Upgrading from REA 6.1 to 6.2](migrations/migration-6.2.md)
## Architecture and authoring
- [Architecture map](https://github.com/morluto/rea/blob/main/docs/architecture/architecture.mermaid)
- [Documentation ownership and contribution model](development/documentation.md)
- [Feature specifications](specs/README.md)
- [Architectural decision records](adr/README.md)
## Machine-owned documentation
| Path | Owner and update command |
| ----------------------------------------- | ----------------------------------------------------------------------------------------- |
| `public/product-catalog.json` | Ignored contract projection; `npm run docs:generate` |
| `verification/managed-conformance-*.json` | Ignored managed conformance projections; `npm run evidence:generate` |
| `error-contract.schema.json` | Tracked schema from contracts; `npm run docs:error-schema` |
| `user-visible-errors.csv` | Authored error classification inventory; preserve its schema |
| `releases/<version>.contributions.json` | Durable release contribution record; follow the [release guide](development/releasing.md) |
| `assets/` | Shared authored images referenced by guides |
Do not edit generated catalogs or evidence to change product facts. Change their
canonical contracts or producer and regenerate. Historical release records retain
the exact commit messages and references from the release they describe.
## Additional guides
- [Offline API contract inventory](api-contracts.md)
- [Bun standalone executables](bun-standalone.md)
- [Deno compile executables](deno-standalone.md)
- [Capability composition and ownership](capability-composition.md)
- [.NET single-file bundles](dotnet-bundles.md)
- [ESP flash images](esp-flash-image.md)
- [U-Boot FIT images](fit-images.md)
- [Measure MCP catalog costs in Codex](mcp-catalog-evaluation.md)
- [Offline Playwright traces](playwright-traces.md)
- [Testing lanes and provider verification](testing-lanes.md)
- [UF2 and DfuSe images](uf2-dfuse.md)
- [WeChat mini-program wxapkg archives](wxapkg.md)
- [Binary hardening](binary-hardening.md)
- [Desktop application capture](desktop-application-capture.md)
- [Desktop application export](desktop-application-export.md)
- [Desktop application state](desktop-application-state.md)
- [Desktop application tasks](desktop-application-tasks.md)
- [Desktop build computer](desktop-build-computer.md)
- [Desktop build witness](desktop-build-witness.md)
- [iNES containers](ines-containers.md)
- [Build RL environments with REA](agent-environments-quickstart.md)
- [Agent environments from recorded captures](agent-environments.md)
- [APK signature schemes](apk-signature.md)
- [Apple IPS crash reports](apple-ips.md)
- [Static analysis in GitHub Actions](ci-analysis.md)
- [Desktop installation directories](desktop-applications.md)
- [Flattened devicetree inspection](devicetree.md)
- [Electron fuse state](electron-fuses.md)
- [ELF note tables](elf-notes.md)
- [Erlang BEAM files](erlang-beam.md)
- [ESP images](esp-image.md)
- [GameCube DOL analysis through Ghidra](ghidra-dol.md)
- [PowerPC ELF analysis through Ghidra](ghidra-ppc.md)
- [Godot PCK archives](godot-pck.md)
- [Static CUDA and HIP code objects](gpu-code-objects.md)
- [Intel HEX and S-record images](hex-records.md)
- [IL2CPP metadata](il2cpp-metadata.md)
- [Static JavaScript route inventory](javascript-route-inventory.md)
- [Linux kernel modules](kernel-modules.md)
- [Kuna provider](kuna-provider.md)
- [Model file declarations](model-artifacts.md)
- [Match native functions across builds](native-function-matching.md)
- [Node.js single executable applications](node-sea.md)
- [PE Authenticode certificate tables](pe-certificates.md)
- [PE import and delay-load tables](pe-imports.md)
- [PE TLS callbacks and load configuration](pe-load-config.md)
- [Python packaged applications](python-packaged-applications.md)
- [CPython pyc inspection](python-pyc.md)
- [Reconstruction coverage](reconstruction-coverage.md)
- [Reverse-engineering tool integrations](reverse-engineering-tools.md)
- [Rust binary dependency metadata](rust-binaries.md)
- [U-Boot legacy uImage containers](uimage.md)
- [UnityFS AssetBundle archives](unityfs.md)
@@ -25,21 +25,7 @@ REA never installs the Android SDK, an emulator, or a system image.
Readiness is `rea inspect-adb-client`: it runs `adb version` and reports the
resolved binary, its version, and where it was selected from. A missing
binary reports `capability_unavailable` with the `REA_ADB_PATH` remediation
rather than attempting an installation. A path that cannot resolve to a file
(`ENOENT`, `ENOTDIR`, `ELOOP` or `ENAMETOOLONG`) is reported as missing. A
selected path the host refuses to execute, such as a directory or a file
without execute permission (`EACCES`) or one the host does not permit
(`EPERM`), and any other spawn error report `capability_unavailable` naming
that path. Exhausted host capacity (`EAGAIN`, `EMFILE`, `ENFILE` or `ENOMEM`)
is not a configuration failure. `inspect_adb_client`, and any device
operation whose adb cannot start at its version probe or at a later adb
command, report a retryable `capacity` resource constraint rather than a
capability refusal or an adb protocol failure; the `binary_session` inventory
reports the provider as `runtime_unavailable`. Both say the host could not
start adb and ask for a retry after freeing processes or descriptors, without
blaming `REA_ADB_PATH`. When adb never started, the refusal diagnostics record
the spawn error as `filesystem_error` with `path_source`, never as an
`exit_code`.
rather than attempting an installation.
The `binary_session` inventory uses the same bounded client-version probe; it
does not query devices, start the adb server, or establish that a requested
serial is reachable. Device state is checked by each device operation.
@@ -113,8 +99,8 @@ existing analysis families directly, for example
| `push_adb_file` | **mutates device** | One local file to a device path (e.g. pushing tools into `/data/local/tmp`); checks for existing targets and refuses them without explicit overwrite (the check is not atomic); compares digests when device `sha256sum` is available |
| `install_adb_package` | **mutates device** | Local APK via `adb install [-r]`; preserves its Success/Failure verdict, records `replace_requested`, and leaves actual replacement `null` because prior package state is not observed |
| `uninstall_adb_package` | **mutates device** | Requests package removal for user 0 via `adb shell pm uninstall --user 0`, without data retention |
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -W -n`) |
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -W -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `start_adb_app` | **mutates device** | Resolves the package's launcher activity (`cmd package resolve-activity --brief`) and starts it (`am start -n`) |
| `start_adb_activity` | **mutates device** | Starts an intent (`am start -a` with optional URI, component, and up to 16 typed extras); preserves the activity manager's verdict |
| `stop_adb_app` | **mutates device** | Force-stops every process of a package, ending a dynamic-analysis run |
Screen capture validates the PNG signature, IHDR framing, dimensions, and IHDR
@@ -188,7 +174,7 @@ and during shutdown.
property value remains available in `properties`.
- **Device paths are not host paths.** `base_apk_device_path` values live on
the device filesystem. Only pulls and captures produce host-local paths,
with byte size and SHA-256 digest on success. A pulled file's host name is
each paired with byte size and SHA-256 digest. A pulled file's host name is
the device path's final POSIX segment. REA preserves the original device
path and filename; a missing final segment or `.`/`..` is refused before
output creation. Windows hosts follow [Windows filename rules](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file),
@@ -217,18 +203,12 @@ and during shutdown.
`uninstall_adb_package` declare `mutatesTarget` and never run implicitly:
pushes check for existing device files before transfer and refuse them
unless `overwrite` is set. A failed or unexpected existence probe stops
transfer even with `overwrite`. A completed `test -e` exit of 1 establishes
absence only with empty stdout and either empty stderr or a recognized,
completed adb-server startup notice sequence. Startup notices are host
observations, not remote-path diagnostics; successful push receipts retain
them in `raw_result.device_path_probes`. Incomplete startup, extra diagnostics,
truncation and cancellation do not establish absence. The check is not atomic, so a different
transfer even with `overwrite`; only a clean `test -e` exit of 1 with no
output is interpreted as absence. The check is not atomic, so a different
process can create the path before `adb push` writes it. When available, the device's
`sha256sum` verifies the transfer. A missing command, output without a
recognizable digest, or host capacity that prevents the digest check from
starting reports `device_digest_source: "unavailable"` with an
unverified-transfer limitation rather than a retryable error; installs
preserve the
`sha256sum` verifies the transfer. A missing command or output without a
recognizable digest reports `device_digest_source: "unavailable"` with an
unverified-transfer limitation; installs preserve the
device's `Failure [...]` reason verbatim.
- **Producer identity stays attached.** Every result embeds the producing
adb client (`binary_path`, `path_source`, `version`, `installed_path`).
@@ -249,10 +229,6 @@ and during shutdown.
- Pull and capture outputs are additive: the destination must not already
exist. Re-running a completed pull refuses the existing directory instead
of overwriting evidence.
- If a single-file pull fails and REA observes a regular file at the destination,
the error includes its `local_path` with `transfer_status: "unconfirmed"`
and `digest_status: "unknown"`. The file remains in place; the receipt does
not establish that the transfer completed.
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
@@ -292,7 +268,7 @@ Record: verified against adb 34.0.5 on Linux against an Android 14
(API 34) x86_64 emulator — all operations above, a 224-package inventory,
287 services, 92 features, a real two-APK split set pulled with byte-exact
digests, a verified push/pull roundtrip, a 1.3 MB screen capture, and the
complete package lifecycle; see `docs/development/testing.md` for the lane summary.
complete package lifecycle; see `docs/testing.md` for the lane summary.
Intent integer extras use signed decimal text. The ADB argument encoder removes
redundant leading zeros for Android's radix-sensitive `--ei` parser while retained
-9
View File
@@ -1,9 +0,0 @@
# Architectural decision records
Store one actual decision per `docs/adr/<domain>/<decision-slug>.md`, without a
global sequence number. Use meaningful names and the Michael Nygard sections:
Status, Context, Decision and Consequences. Link each decision from its owning
feature specification.
See [the authoring guide](../development/documentation.md#feature-lifecycle) for
the distinction between specifications, roadmaps and isolated decisions.
+4 -6
View File
@@ -11,15 +11,15 @@ with your RL framework to set task goals, check rewards, and run training.
## Before you start
Install [REA and its agent integration](getting-started/installation.md). To record an app,
Install [REA and its agent integration](installation.md). To record an app,
you also need a running test instance and a Chrome-family browser. Choose an
instance where the agent is allowed to create, edit, and delete data. REA
records the app you provide; you start it and prepare its test data.
To run the exported environment, install the
[AgentEnv framework](https://github.com/scaleapi/agentenv-framework#install)
and start Docker. REA's [compatibility check](development/testing.md#exported-environment-bundles)
uses AgentEnv version `0.9.1431` with Python 3.12.
and start Docker. REA's [compatibility check](testing.md#exported-environment-bundles)
uses AgentEnv version `0.9.1334` with Python 3.12.
There are two ways to build an environment: ask your agent to record the app,
or start from a recording you already have.
@@ -52,7 +52,7 @@ work and which need more evidence. The app is used during recording; the
exported environment runs from the saved data.
After your agent exports the folder, skip to
[run the environment](#run-the-environment). The next section is an
[run the environment](#3-run-the-environment). The next section is an
alternative for an existing recording.
## 2. Or build from a saved recording
@@ -91,8 +91,6 @@ A successful build means the folder was created. Check the report to confirm
it supports the operations your tasks need. `coverage: null` means the build
could not prepare state for this environment and it will use replay.
<a id="run-the-environment"></a>
## 3. Run the environment
Whether your agent exported the folder or you built it from a HAR, run it
+19 -26
View File
@@ -30,7 +30,7 @@ rea environment-serve manifest.json --evidence capture.json --port 8765
```
The first command inspects the capture; the second projects the
[environment manifest](guides/web/web-network-captures.md#environment-manifest) view from
[environment manifest](web-network-captures.md#environment-manifest) view from
it, where `view.json` wraps the capture Evidence as the inline source:
```json
@@ -107,10 +107,7 @@ refusal names its stage: `environment-build` for the build's own checks,
`inspect_web_network_capture` and `inspect_analysis_view` for inspection and
projection, `environment-export` for the bundle. SIGINT cancels the build
(`cancelled`); before the bundle stage nothing is written, and a cancellation
after publication preserves the bundle receipt. Before publication, a failed
write attempts to remove partial output and reports cleanup failures. See
[bundle publication and recovery](getting-started/cli.md#export-an-environment-bundle).
Stage progress is logged to stderr
during the write names the bundle it wrote. Stage progress is logged to stderr
at info level; the report is the only output on stdout:
```json
@@ -166,7 +163,7 @@ at info level; the report is the only output on stdout:
`coverage` is the state's read views and write plans summarized, and `null`
when the state is unavailable and the bundle serves replay. The
[environment coverage](guides/web/web-network-captures.md#environment-coverage) view
[environment coverage](web-network-captures.md#environment-coverage) view
reports the same summary for any capture without building, so an agent can
check it after each recording. The view summarizes the state as derived; it
does not apply the serve-time admission checks, so for an environment whose
@@ -212,9 +209,9 @@ warning does not establish that these bodies were checked. A browser scenario's
saved values (`save` and `save_response`) named like credentials but not marked
`secret: true` are recorded unredacted; `credential-named-saved-values` lists up
to 20 of them by parameter pointer. Names do not establish sensitivity,
so the build never refuses on them.
so the build never refuses on them. Capture
Capture Evidence also records the capture file's absolute path (`subject.local_path`,
Evidence also records the capture file's absolute path (`subject.local_path`,
`parameters.capture_path`, `locations`, `artifact.path`), and the bundle
carries it: build from a neutral folder when that path should not ship.
@@ -238,7 +235,7 @@ name `export_environment_bundle`; the others name their stage as
## From a browser scenario
The capture can also come from REA itself: a
[browser scenario](guides/web/browser-scenario-contract.md) records a scripted session
[browser scenario](browser-scenario-contract.md) records a scripted session
in a browser REA launches, and its Evidence is a manifest parent like a HAR
inspection. Select the `network` event family and response bodies. A replayed
reply carries up to 256 KiB of a recorded body, but the scenario keeps at most
@@ -324,7 +321,7 @@ declared secret replaced by its redaction marker (a body that redaction had to
empty replays no body).
To record writes, sign in through the UI and then send the application's API
calls with [request steps](guides/web/browser-scenario-contract.md#request-steps): each
calls with [request steps](browser-scenario-contract.md#request-steps): each
sends one request to the start origin through the page's own `fetch`, with the
session's cookies, and a header can carry the CSRF cookie or stored token the
application's own requests send. A write is served only when the capture shows
@@ -335,12 +332,9 @@ scenario is written and cannot use a value an earlier response returned, so an
id must be known before the run: one the caller chooses, such as a `uid` in the
create body, or one an earlier scenario created. When the server assigns ids,
read the list after each create (the create's response or the list shows the
new entity). A second scenario can record updates and deletes using those
known ids, but produces a separate environment: recordings are not combined.
That scenario must include its own before/after reads and every action the
environment should support; it does not inherit the first scenario's create
evidence. To capture create and subsequent edits together, make them through
the UI in one scenario. The environment coverage view names each
new entity), and record updates and deletes in a second scenario that reads
the first one's objects by id before editing them, or make them through the
UI. The environment coverage view names each
write it withholds and what to record next. A scenario with write requests
changes the application's data: record against an instance whose data may
change.
@@ -487,7 +481,7 @@ captured text without changing the tool names or schema fields clients use.
Replay answers each call from one recording, so the answers reflect whichever
moment of the capture recorded them. The
[environment state](guides/web/web-network-captures.md#environment-state) view extracts
[environment state](web-network-captures.md#environment-state) view extracts
instead the data the capture's reads observed before the first related write:
entity types with an identity field, entities with their recorded values,
documents for reads that are not entities, and a render plan per read tool,
@@ -509,14 +503,14 @@ see [Serve from the state](#serve-from-the-state).
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the
environment, then to `127.0.0.1` and an ephemeral port, exactly as
[`rea mcp --http`](getting-started/cli.md#serve-mcp-over-streamable-http) resolves them. The
[`rea mcp --http`](cli.md#serve-mcp-over-streamable-http) resolves them. The
bound MCP URL is written to stderr as one JSON log line
(`"msg":"MCP transport listening"` with `url`); `REA_LOG_LEVEL` overrides that
log level. `ENVIRONMENT_NAME` renames the served environment; otherwise the
manifest's `name` is used, and a set but blank value is refused as
`configuration_invalid` before binding rather than silently falling back. The
listener serves one MCP client at a time with the same session, `Host` header
and shutdown contract as [transports](reference/mcp-contracts.md#transports) describes.
and shutdown contract as [transports](mcp-contracts.md#transports) describes.
It runs until SIGINT or SIGTERM, then closes the listener, prints one
`{"status":"stopped", …}` document whose `state` member says whether it served
from the state, and exits `130` or `143`.
@@ -591,7 +585,7 @@ names the method and path template, then what the tool returns, one level deep:
`returns a JSON object with fields …`, `returns a JSON array of objects with
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
where a list is an array of objects found by the
[environment state](guides/web/web-network-captures.md#environment-state)'s list rule (the
[environment state](web-network-captures.md#environment-state)'s list rule (the
root array, or the one array of objects an envelope carries). Field and member
names are written as JSON strings, so a captured name cannot read as more of
the description; a member name too long to show whole is a quoted prefix marked
@@ -647,7 +641,7 @@ rea environment-serve manifest.json --evidence capture.json
```
Before it binds, the server derives the
[environment state](guides/web/web-network-captures.md#environment-state) the capture
[environment state](web-network-captures.md#environment-state) the capture
observed: it runs the `environment-state` view on the capture in-process with
the manifest's `template_rules` in the same order, exactly as
`inspect-analysis-view` projects it from the same capture, so the state
@@ -739,7 +733,7 @@ query model may describe supported paging; an empty later page cannot start
returning earlier entities merely because it was empty in the capture.
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
[write plan](guides/web/web-network-captures.md#environment-state) for them is served:
[write plan](web-network-captures.md#environment-state) for them is served:
the capture's later reads confirmed the effect the plan models. Other write
tools are not served, so no call claims a change it does not make; each one's
plan states why in its `reason`. A served write changes the live state, which
@@ -1055,9 +1049,8 @@ unknown recorded status stays unknown; it is not borrowed from another
record). A bundle with no such tool yields a verifier that checks the count
only and says so. `tasks/smoke.json` deploys the env on the local
sandbox and runs that verifier against the env's MCP endpoint; the first run
also pulls the framework's published infrastructure images from `ghcr.io`,
building one only if its pull fails, and the run tears everything down when it
ends.
also builds the framework's own infrastructure, and the run tears everything
down when it ends.
The README written into the bundle records the capture artifact's SHA-256, the
capture and manifest Evidence ids, the manifest view digest, the served tool
@@ -1193,7 +1186,7 @@ read, not that the write had no other effect.
## Compatibility verification
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
described in [testing](development/testing.md#exported-environment-bundles). It projects
described in [testing](testing.md#exported-environment-bundles). It projects
the Grafana-shaped source-owned capture through the built CLI, packs the
checkout, exports a bundle with `--package-tarball`, runs it with the pinned
`agentenv-framework` release on the runner's Docker and requires the smoke task
@@ -6,12 +6,12 @@ kept as an unmodified, commit-pinned Git submodule; see
application workflow and return artifact-bound Evidence inline.
This family is present on repository main and in npm 4.1.0. Check the
[released package boundary](../../getting-started/installation.md#released-package-and-main) and the
[released package boundary](installation.md#released-package-and-main) and the
connected server's tool list before selecting it.
On Linux, JADX launches an REA-owned process and requires a procps-compatible
`ps` on REA's `PATH`; see the shared
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
## Optional package delivery
@@ -73,9 +73,6 @@ spellings; the original SDK declaration strings remain unchanged.
it remains null for unknown provider target SDKs, nonliteral declarations, or
components with filters targeting API 31 or later that omit the now-required
explicit declaration. A limitation records the latter validation error.
As in Android's package parser, an activity, alias or receiver filter without
an `<action>` is discarded and does not count toward either rule; a service
keeps such a filter. `intent_filter_count` still counts every declared filter.
Provider authorities and all inventory dimensions are bounded. `status:
"partial"` and `limitations` identify omitted declarations or values; a
capacity limitation appears only when an actual collection limit was reached.
@@ -545,8 +542,7 @@ and each upstream request and reply collected before the failure. Failed replies
remain available even when their payload cannot be interpreted. Cancellation
also stops target hashing before provider admission.
The JADX package, class, method and reference operations do not execute the APK
or provide split APK/AAB handling,
These operations do not execute the APK or provide split APK/AAB handling,
signature validation, full resource-table semantics, native-library analysis or
Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
Android tools and the application graph do not treat it as a standalone APK.
@@ -688,14 +684,14 @@ bounds each upstream call. Timeout recovery reports the effective milliseconds
and range, with `details.limit` identifying the runtime setting. An invalid
provider-local launch value can also be repaired by a runtime override; reset
exposes that invalid launch value again. See the
[settings design](../../design/limits-and-recovery.md#invalid-launch-configuration).
[settings design](design/limits-and-recovery.md#invalid-launch-configuration).
A passing JADX configuration check in `rea doctor` reports its launch-derived
`operation_timeout_ms`; it does not inspect another running server's overrides. Explicit
cancellation still ends the operation immediately, and partial observations
and owned-process cleanup are unchanged. Raising the deadline does not extend
the bridge's 30-second startup handshake or the upstream **90-second**
decompilation limit. MCP clients enforce their own tool-call timeouts, and
setup's derived [client tool timeout](../../getting-started/client-configuration.md) does not include
setup's derived [client tool timeout](client-configuration.md) does not include
this setting, so also raise the client timeout when selecting a longer
deadline.
-178
View File
@@ -1,178 +0,0 @@
# Offline API contract inventory
`inspect_api_contract` / `rea inspect-api-contract` supports local OpenAPI JSON
and GraphQL SDL (`format: "graphql-sdl"`). Neither profile contacts endpoints.
## OpenAPI JSON
The default profile reads one local OpenAPI
3.0, 3.1 or 3.2 JSON document. It inventories inline operation slots under
`paths`; the facts describe declarations, not implemented, reachable or
authorized runtime behavior. YAML, AsyncAPI, callbacks/webhooks, schema graphs,
contract diffs and traffic correlation are outside this version.
The OpenAPI importer never resolves references, opens sibling documents, evaluates
examples or contacts declared servers. OpenAPI 3.2 `query` and
`additionalOperations` follow the [Path Item contract](https://spec.openapis.org/oas/v3.2.1.html#path-item-object),
including case-sensitive additional HTTP method spelling.
## Select a useful answer
MCP defaults to a summary retaining the complete inventory. An optional view
returns selected facts immediately, taking precedence over `detail`:
```json
{
"path": "/artifacts/openapi.json",
"view": {
"kind": "api-contract",
"collection": "operations",
"methods": ["GET"],
"limit": 20
}
}
```
`operations` supports exact `operation_ids`, `operation_keys`, `methods`,
`paths`, `declared_ids` and `states` filters. Filters combine by intersection;
empty arrays select nothing. `fields` chooses columns; identity, source pointer,
state and unknown reason remain. The other collections are
`path-item-references` (optional `reference_ids`) and `diagnostics` (optional
`codes`), also with fields and offset/limit. The default page contains 64 rows.
Query a retained result with `inspect_analysis_view`, using the same view and
`source: {"kind":"retained-evidence","evidence_id":"..."}`. Copy the complete
parent ID from `normalized_result.parent_evidence_id`, not the summary's own ID.
These pages use immutable retained facts without rereading the specification.
Complete Evidence remains available through `get_evidence_bundle`.
CLI defaults to complete Evidence. For a selected answer, save that complete
Evidence in an absent bundle path:
```sh
rea inspect-api-contract ./openapi.json --detail summary \
--view '{"kind":"api-contract","collection":"operations","methods":["GET"],"limit":20}' \
--evidence_output ./contract.evidence.json --json
```
Later `rea inspect-analysis-view` can read that bundle through an exact
`evidence-file` source selection, even after the original specification is
unavailable. Publication never overwrites an existing file. CLI preserves its
original path spelling in `caller_input_path`; analysis locations and MCP inputs
use absolute filesystem paths.
## Identity, provenance and coverage
Each operation has an artifact-scoped `operation_id`, an exact method/path
`operation_key`, an optional `declared_id` from `operationId`, and its original
JSON Pointer. Keys can correlate exact declarations in a caller-selected pair
of revisions; they do not establish equivalence across unrelated APIs, renamed
path templates or runtime handlers. Declared IDs are not assumed unique.
The artifact includes the verified byte length, SHA-256 and selected path.
Malformed siblings retain useful declarations plus diagnostics. A Path Item
`$ref`, including a local fragment, remains unresolved; adjacent inline slots
are unknown because referenced fields may overlap. Missing or malformed paths
are explicit, and missing required OpenAPI 3.0 paths does not become a complete
empty inventory. `source_coverage` is limited to inline paths and records that
reference resolution and full schema validation were not performed. Page
exhaustion describes the selected rows and never upgrades source coverage.
Structured transport userinfo credentials are excluded from reference URIs;
`credential_redacted` reports that change, and an ambiguous credential span leaves
the displayed reference unknown. Ordinary reference query/fragment text and
names are retained. Original bytes remain in the caller's artifact; complete
inventory Evidence does not embed the entire source JSON. Duplicate JSON members
use the shared parser's last-value-wins behavior, recorded in source coverage.
If decoding or later validation fails after a verified read, the typed error
retains a compact `file-snapshot-hash` observation with the exact path, digest
and byte length. Syntax diagnostics do not quote source JSON. Cancellation
does not replace a parser failure that has already completed.
## Resource bounds and verification
The file-byte allowance and conservative expanded-facts allowance are each
32 MiB resource policies, not OpenAPI format limits. `max_file_bytes` can lower
the input allowance. Native JSON parsing additionally checks available heap
against the shared representation-expansion estimate before decoding. The
shared JSON nesting limit is 1,000, including uninterpreted schema regions.
Capacity refusals are typed; reply paging does not increase parser allowances.
The committed [Petstore fixture](https://github.com/morluto/rea/blob/b7c8caedd4166e6032c49024cdc6fb1154f1a2df/tests/fixtures/openapi/README.md) is an
original document from the published application, replayed offline through
public CLI/MCP tests. These check advertised schemas, complete/selected parity,
retained paging, partial references, zero requests to a loopback reference
server and unchanged source bytes. They do not establish runtime endpoint
behavior or full OpenAPI conformance.
## GraphQL SDL and operation documents
Select `format: "graphql-sdl"` for SDL. Optionally supply `operations_path` to
one local document containing operations and fragments. Both MCP paths are
absolute; CLI accepts relative spellings:
```sh
rea inspect-api-contract ./schema.graphql --contractFormat graphql-sdl \
--operationsPath ./queries.graphql --detail summary \
--view '{"kind":"graphql-contract","collection":"references","limit":20}' \
--evidence_output ./graphql.evidence.json --json
```
Collections are `types`, `fields`, `definitions`, `references`, `fragment-spreads`
and `diagnostics`. Each supports `fields`, `offset` and `limit` (default 64).
Exact batch filters include type/field/definition/reference IDs, type names,
`field_keys` such as `Query.viewer`, owning type names, binding/spread states
and diagnostic codes. Filters intersect; an empty array selects nothing.
Identity, source and applicable uncertainty fields always remain. Retained pages
and bundle-based queries use the same `graphql-contract` views without rereading
SDL or rebuilding its schema model.
Types and fields retain individual declaration occurrences, including extensions,
list/non-null wrappers, implemented interfaces and union member names. Operation
and fragment definitions own field references with aliases and parent-reference
IDs. Each reference links to a supplied SDL field occurrence, remains `model-only`
for a library-provided field such as `__typename`, or records an exact unknown
reason. Rejected schema construction keeps declarations but cannot supply trusted
bindings. Missing or duplicate fragment names remain explicit. Fragments are
visited once in their declared type scope and linked by spread edges; cycles
are never recursively expanded.
The versioned `graphql-sdl-field-bindings-v1` profile records the GraphQL library
version. It performs syntax parsing and schema construction for field resolution;
it does not establish whole-schema/operation validity, spread applicability,
reachability, authorization or endpoint behavior. Arguments, defaults,
descriptions, directive behavior, enum value inventory, introspection JSON,
federation composition and version comparison remain outside this profile.
Every fact has a document role, SHA-256, UTF-16 start/end offsets and parser token
line/column. Offsets address decoded UTF-8 text, including an optional BOM, rather
than byte positions. The two files are independently verified snapshots, not an
atomic pair. Fact IDs identify source occurrences; `field_key` separately supports
exact name-based joins and does not assert equivalence across revisions. CLI
retains its auxiliary path spelling in `caller_input_paths.operations_path`,
outside semantic identity, as well as its primary `caller_input_path`.
Malformed operation text retains the SDL inventory with partial coverage and a
source-located diagnostic. A primary parser refusal or later resource/cancellation
failure retains known source identities and available interpreted Evidence;
syntax diagnostics do not quote source text. Complete Evidence contains facts,
not original document bytes. An exhausted page never upgrades source coverage.
Each file has the same configurable maximum of 32 MiB. Conservatively charged
facts share a 32 MiB allowance; text/AST allocation is admitted against available
heap. The parser admits at most 262,144 lexical tokens and 65,536 comments per
document; subsequent iterative AST admission limits traversal depth to 128.
These are REA resource policies, not GraphQL language limits. Very deep input can
be refused by the parser before AST admission. Selected replies reduce transport
and retained queries reuse immutable facts; they do not avoid the initial parse.
The committed SWAPI SDL is an unchanged public application fixture with provenance
in `tests/fixtures/graphql/README.md`. It supplies 53 types and 243 fields;
verification does not execute its API.
Large complete inventories can exceed the default MCP response budget even when
selected pages fit. Use `export_evidence_bundle` with a caller-selected local
path for complete retained Evidence, or CLI complete JSON/bundle publication.
A transport refusal does not erase a completed retained analysis. Narrow pages
and source coverage remain available on the same continuing connection.
@@ -66,8 +66,6 @@ rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR
rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR --string-tables-only
rea decode-android-resources /targets/Example.apk --no-strings
rea decode-android-resources /targets/Example.apk --decode-timeout-ms 900000
rea decode-android-resources /targets/Example.apk --string-offset 2000 --string-limit 100 --string-tables-only
rea decode-android-resources /targets/Example.apk --max-string-table-bytes 16777216 --max-total-string-table-bytes 67108864
```
Equivalent MCP methods are `inspect_apktool_client` and
@@ -89,31 +87,6 @@ responses and charges the aggregate output budget for one copy per table.
Omitting the option preserves existing responses. `include_strings: false`
still suppresses all string projection, including in this mode.
Use `string_offset` and `string_limit` (CLI `--string-offset` and
`--string-limit`) to select a page of valid parsed entries from each requested
table, in source order. Defaults are offset 0 and limit 2000. Explicit page or
budget controls include `string_tables` even for a single primary table; its
`page` gives the requested offset and limit, returned count, and `next_offset`.
Retry that offset with the same locale and confirm `target.sha256` matches.
Each call decodes a fresh owned snapshot; it does not retain a workspace or
promise identity across changed APKs. A null continuation means the page reaches
the end, while table coverage remains partial if earlier entries were omitted.
An offset at or beyond the end returns an empty, present table with known counts.
If the continuation equals the requested offset, no output entry fit: increase
`max_total_strings` or use `string_tables_only` before retrying.
`max_total_strings` (CLI `--max-total-strings`, default 10000) bounds returned
string copies across all tables. Raise it, request fewer locales, or use
`string_tables_only` when aggregate output is exhausted. The legacy
multi-table shape charges both copies of primary strings. `max_string_table_bytes`
(CLI `--max-string-table-bytes`, default 8388608) bounds one XML source read;
`max_total_string_table_bytes` (CLI `--max-total-string-table-bytes`, default 33554432) bounds XML reads across distinct physical tables. Locale aliases
share the physical read and parsed page. Raise the relevant read budget to
retry an oversized table; a larger string limit or timeout does not raise it.
These controls take nonnegative safe-integer offsets and positive safe-integer
budgets, apply on the next decode, and require no REA restart. Process memory
and response transport capacity still apply.
## Interpret the result
- **An owned APK snapshot is digested and decoded.** `target.sha256` and
@@ -144,8 +117,8 @@ and response transport capacity still apply.
Malformed XML and a wrong document root are whole-document projection
failures: the result is partial, the failure reason is disclosed, and entry
counts remain unknown. They are not counted as one unsupported string.
The default page returns up to 2000 entries with partial coverage when more
exist; its limitation names the next `string_offset`. `locales` lists observed regular `strings.xml` tables whose exact
The default table is bounded to 2000 entries with partial coverage when more
exist. `locales` lists observed regular `strings.xml` tables whose exact
`values-<locale>` directory is selectable by the request's locale grammar.
Locale suffixes allow up to 23 ASCII characters, derived from Apktool's
language, script, region, and variant fields; for example,
@@ -160,10 +133,9 @@ and response transport capacity still apply.
When multiple tables are requested with `locales`, `string_tables` records
each projected table with its locale, status (`present`, `missing`,
`oversized`, `malformed`, or `skipped`), entry counts, and coverage,
with default aggregate budgets of 10,000 strings and 32 MiB XML. When `locales`
is omitted, `string_tables_only` is not true and no string page or budget
control is selected, legacy single-table payloads are preserved without adding
`string_tables`.
bounded at an aggregate budget of 10,000 strings and 32 MiB XML. When `locales`
is omitted and `string_tables_only` is not true, legacy single-table payloads are preserved
without adding `string_tables`.
- **Smali is skipped by design** (`apktool d --no-src`): Java-source
recovery belongs to the JADX family. Resource smali edits and rebuild are
a follow-up lane with mutation authority.
@@ -184,8 +156,8 @@ and response transport capacity still apply.
- The version probe waits 20000 ms and retains 65536 bytes. This release has
no runtime override for that probe. `decode_timeout_ms` does not raise it.
- Manifest reads are bounded to 512 KiB before decoding UTF-8. Strings XML
reads default to 8 MiB per table; larger documents are skipped with partial
coverage and an explicit read-budget recovery, rather than reported as missing.
reads are bounded to 8 MiB; larger documents are skipped with partial
coverage and an explicit limit, rather than reported as missing.
- `decoded_file_count` is capped at 100,000 files; the cap is reported in
the limitations when hit.
- An ordinary nonzero decode exit surfaces as `unsupported_target` naming
@@ -202,7 +174,7 @@ OpenJDK 25 against a signed aapt2-built probe APK carrying two resource
locales — launcher version, on-disk digest agreement, apktool.yml metadata
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
tables projected. Parser goldens in the test suite come from the same
decode. See `docs/development/testing.md` for the lane summary.
decode. See `docs/testing.md` for the lane summary.
Multi-locale and table-only projection were also verified through real CLI and
stdio MCP workflows on Linux with Apktool 3.0.3/JDK 21 against NewPipe 0.29.1,
+4 -67
View File
@@ -85,49 +85,9 @@ canonical hexadecimal form), `EXC_GUARD` subtypes,
pointer-authentication suffixes, and dyld missing-library strings. A report
that matches none of those rules is `unclassified`.
## Joining frames to the crashed Mach-O or dSYM
Pass `join_path` (`--join-path` in the CLI) to select one local Mach-O
executable or dSYM. UUID bytes in the report and the artifact's `LC_UUID` load
commands are compared in hexadecimal, so an uppercase `dwarfdump --uuid` value
and a lowercase report `uuid` match.
```bash
rea inspect-apple-ips /absolute/path/to/report.ips \
--join-path /absolute/path/to/App.app/Contents/MacOS/App --json
```
Shared-cache image sources `S` and `C` remain unjoined: UUID agreement does not
establish that a standalone image has the cached layout.
For a FAT container every slice is examined. An image joins only when exactly
one slice has that UUID **and** an architecture that agrees with the report's
`arch`; `arm64` and `arm64e` stay distinct, and an all-zero UUID never joins. A
UUID match with a different architecture, more than one compatible slice, or a
slice without a readable `__TEXT` segment is reported as a rejected image with
its reason and stays unjoined rather than being guessed.
An unknown UUID in another slice prevents a unique join unless its architecture
rules out a match. Missing or unrecognized architecture and unsupported CPU
subtypes remain unknown; display-name fallbacks do not establish agreement.
The parsed slice facts remain available even when no match can be derived.
`join.status` is `selected` with the artifact path, SHA-256, byte length, every
slice's offset, arch, CPU type and subtype, UUID, and `__TEXT` vmaddr, plus
`image_matches`, per-frame results, and join-specific limitations. A joined
frame reports `static_address`, the slice `__TEXT` vmaddr plus the frame's
recorded `imageOffset`, in decimal and hexadecimal. An unjoined frame reports
a reason instead; an unknown image UUID, an absent `imageIndex` row, and a
frame without `imageOffset` all stay unjoined rather than treated as aligned.
An artifact that is not a Mach-O image or FAT container reports
`join.status: "unreadable"` with a typed reason and leaves every report fact
intact. Symbol names printed inside the report stay recorded claims. Function
or symbol lookup at a joined static address, and matching dSYM symbol tables,
are not performed by this step.
Symbol names printed inside the report are recorded claims and are not
verified. Linux ELF cores use the separate
[recorded crash](guides/native/recorded-crashes.md) workflow.
Opened Mach-O and dSYM UUID joins remain unobserved. Symbol names printed
inside the report are recorded claims and are not verified. Linux ELF cores
use the separate [recorded crash](recorded-crashes.md) workflow.
## Limits and failures
@@ -160,27 +120,4 @@ success or truncated integer facts.
rea inspect-apple-ips /absolute/path/to/report.ips --max-integer-text-bytes 16777216 --json
```
`max_join_bytes` (`--max-join-bytes`) independently bounds the join artifact
read, defaulting to 64 MiB. This workflow does not execute a target, launch
atos, or search for dSYMs.
## Join admission and failures
The join uses the shared Mach-O container, slice, command-table and segment
readers. Slice extents and `sizeofcmds` bound every identity observation; duplicate
UUIDs or malformed commands cannot select the first plausible value. Unreadable
join results retain the successfully read artifact's path, SHA-256 and byte count.
The integer-text allowance spans the report and join, including repeated integer
facts and derived static addresses. Reservation precedes derived address
expansion. If a selected join read, interpretation or allowance fails, the error
retains the completed IPS report. Join read limits and recovery use
`max_join_bytes`, even when the report and join paths are identical. Cancellation
retains the caller's reason independently of those completed facts.
When a failure follows a completed join-file read, `partial_observation` retains
both the decoded report and a `join_artifact` receipt with the independently
observed path, SHA-256 and byte count. Frame rejection reasons reference
`join.limitations`; source-sized architecture diagnostics are retained once per
image rather than copied into every frame. Universal-image admission includes
the shared reader's supported hidden ARM64 row and rejects overlapping slices.
This workflow does not execute a target, launch atos, or search for dSYMs.
File renamed without changes.
@@ -120,30 +120,23 @@ CLI output are unchanged.
Complete results have a 32 MiB input, 64 MiB reply, 1 MiB combined diagnostics
and 30-second owned command deadline. `REA_PWNTOOLS_MAX_INPUT_BYTES` raises the
launch input default alone, up to 1 GiB. Per-call `producer_limits` selects
`input_bytes`, `output_bytes`, `diagnostic_bytes`, `address_space_bytes`,
`file_bytes`, and `cpu_seconds`; positive safe integers apply on the next call
without restart. `input_bytes` overrides the launch default. File admission
follows selected `output_bytes` unless `file_bytes` is explicit. The separate
`pwntools_timeout_ms` controls each command wall deadline. On CLI pass
`--producer-limits '{"cpu_seconds":60,"output_bytes":134217728}'`. The
input bound alone, up to 1 GiB; the reply and deadline bounds are unchanged. The
reply budget applies to the decoder record; the CLI/MCP Evidence envelope adds
copies and encoding overhead. The
pinned MCP SDK defaults to a 10 MiB receive buffer. Large complete results need
a caller-configured `StdioClientTransport({ maxBufferSize: ... })` and a matching
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](../../reference/mcp-contracts.md#tool-results).
server `REA_MCP_MAX_RESPONSE_BYTES`. See [MCP delivery budgets](mcp-contracts.md#tool-results).
These settings affect delivery, not decoder coverage or limits. The large
extended-index verification uses 256 MiB and an explicit five-minute request
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to the selected policy: defaults are 3 GiB
virtual address space, 30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
timeout in its separate opt-in MCP lane. REA does not silently truncate the result. Python lowers resource soft limits to at most 3 GiB virtual address space,
30 CPU seconds and 64 MiB file output. Inherited tighter soft/hard limits are
retained, and the effective values are returned in the result limitations. Virtual address space is not RSS;
Unicorn needs a large virtual map. Limits fail with no partial success, and
owned process/root cleanup completes independently of cancellation.
For core files and optional pwndbg mapping context, use the
[recorded-crash workflow](recorded-crashes.md). The binary-layout operations
described here do not provide debugger session state or control, or establish
support for other operating systems/architectures.
Core files, GDB session/state/control, optional pwndbg enrichment and instruction
inspection are separate increments tracked by #969. This increment makes no
real support claim for those features or other operating systems/architectures.
Typed decoder failures retain bounded `captured_output` (stdout, stderr and an
explicit truncation flag) alongside the original failure category and reason.
@@ -279,12 +272,6 @@ regular files only and rejects symbolic links.
## Byte entropy triage
Entropy admission preserves the caller's spelling in `caller_input_path`. On
Windows, the selected regular file's case-only filesystem spelling is used
consistently in committed paths through both CLI and MCP. Symlink, junction and
short-name aliases are not replaced with a different path. General path
normalization remains lexical and does not probe source-discovered paths.
`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
without an analysis engine, execution, extraction or network access.
-62
View File
@@ -1,62 +0,0 @@
# Binary hardening
Use `inspect_binary_hardening` to read compile-time and link-time hardening from
an explicit local ELF, PE, or Mach-O file. No disassembler, pwntools install, or
target execution is required. Universal Mach-O files return one image per slice.
```sh
rea inspect-binary-hardening /absolute/path/to/program --json
rea inspect-binary-hardening /absolute/path/to/program --max-file-bytes 1048576 --json
```
MCP accepts `{ "path": "/absolute/path/to/program" }`. CLI paths may be relative
to the current directory; MCP paths must be absolute.
Each image has one row per property: `property`, `state`, `confidence`, `basis`,
`location`, `note`, and `next_probe`. States are `enabled`, `disabled`,
`partial`, `unknown`, and `not_applicable`. `confidence` is `observed` for a
single header or import, and `derived` when the state combines several fields,
such as ELF PIE or full RELRO. An image's `architecture` is its CPU family, or
null when REA has no family for it. `machine` and `slice.architecture` keep the
exact identity, such as `arm64e`.
ELF rows cover PIE (`ET_DYN` with `PT_INTERP` or `DF_1_PIE`), NX (`PT_GNU_STACK`;
a missing header is `unknown` and the architecture default is only noted), RELRO
(`PT_GNU_RELRO` plus `BIND_NOW`), stack protector and FORTIFY undefined dynamic
imports, CET and AArch64 BTI/PAC GNU property notes, `DT_RPATH`, `DT_RUNPATH`, and
whether a `SHT_SYMTAB` section is present. Only an undefined dynamic import
counts. Imports come from the table `DT_SYMTAB` declares. A `SHT_DYNSYM` section
only supplies the symbol count when `DT_HASH` and `DT_GNU_HASH` leave it open,
and only if it names the same symbol and string tables. Without that count the
import rows are `unknown` and give the reason. A missing `__stack_chk_fail` or fortify import is `unknown`. That
absence does not establish that the protection was left out of a static or
stripped build. A present import is a sign that some code uses the protection,
not proof that all code does. NX, RELRO, `DT_RPATH`, and `DT_RUNPATH` are
`not_applicable` when `e_type` is neither `ET_EXEC` nor `ET_DYN`, and `unknown`
for a linked image with no readable program headers.
PE rows read `DllCharacteristics` for dynamic base, high-entropy VA, NX
compatibility, force integrity, no SEH, and guard CF. `cfg_instrumented` is the
load-config `GuardFlags` linker claim when `Size` covers that field. An
unreadable load configuration, including a TLS failure in that parser, leaves
only `cfg_instrumented` `unknown`. Authenticode and debug rows report certificate
and debug directory presence, not signature validity or debug content. A
directory whose address and size disagree is `unknown`.
Mach-O rows read `MH_PIE`, `MH_ALLOW_STACK_EXECUTION`, `MH_NO_HEAP_EXECUTION`,
an `__RESTRICT` segment, `LC_CODE_SIGNATURE` presence, encryption from the
existing encryption reader, the arm64e subtype, and a stack-protector import.
`hardened_runtime` is `unknown` with `next_probe` `inspect_signature`.
CodeDirectory flags stay with that tool.
The report states presence of build-time features. It does not state
exploitability. `inspect_binary_layout` is unchanged.
The default read budget is 64 MiB (`max_file_bytes`, ceiling 512 MiB). A changed
file is refused. A malformed header, a Mach-O segment or encryption command that fails admission, or a program, section,
or segment payload declared outside the file returns no partial report, except that one bad slice in a universal Mach-O leaves the other slices. A symbol table or
symbol name over its budget leaves the import rows `unknown` and keeps the
header rows. A GNU property note over its budget, or a compressed note section
that may hold one, leaves the CET and BTI/PAC rows `unknown` and keeps the header
rows. A dynamic-table budget still fails the ELF report. The image is never
executed.
@@ -31,7 +31,7 @@ authority are validated independently before selection and connection.
For a compact route query, select `detail: "summary"` and a
`view: {kind: "javascript-routes", collection: "routes", limit: 20}`. The
shared [route inventory](../../javascript-route-inventory.md) preserves source IDs,
shared [route inventory](javascript-route-inventory.md) preserves source IDs,
precise locations, partial coverage, and the complete parent Evidence for
subsequent filtered pages. Static candidates never establish complete application
coverage or reachability.
@@ -120,7 +120,7 @@ correspondence merely from their different retained registrations.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
- Every surface has equivalent CLI and MCP contracts and returns Evidence provenance.
- By default, MCP tools return complete results inline and include the Evidence ID. Selected summary views retain the complete parent Evidence and return a projection. Session evidence can be exported with `export_evidence_bundle`.
- MCP tools return complete results inline and include the Evidence ID. Session evidence can be exported with `export_evidence_bundle`.
Target discovery returns the complete in-scope target array in one result.
@@ -147,7 +147,7 @@ Caller-owned JSON/base64 strings, input parsing and validation, transport
buffers, garbage-collector behavior, and other runtime memory are outside this
estimate; the budget does not promise a bound on total process memory.
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](../javascript/electron-observation.md).
Electron `file://` pages use a separate provider and target boundary; see [electron-observation.md](electron-observation.md).
## Provider authorities
@@ -177,7 +177,7 @@ See [the reproduction and Chromium ownership limitation](https://github.com/morl
Existing static application Evidence and passive web/Electron captures can be
combined later through
[JavaScript static/runtime reconciliation](../javascript/javascript-runtime-reconciliation.md).
[JavaScript static/runtime reconciliation](javascript-runtime-reconciliation.md).
## Start a browser
@@ -228,7 +228,7 @@ in an existing profile.
For such a browser, pass the browser socket instead of the HTTP endpoint:
```bash
rea list-browser-targets "ws://127.0.0.1:9222/devtools/browser/<id>"
rea list-browser-targets ws://127.0.0.1:9222/devtools/browser/<id>
```
Chrome writes the port and the `/devtools/browser/<id>` path to the
@@ -239,21 +239,10 @@ file for you.
`observe_web_session`, `discover_webmcp_tools`, and `capture_web_screenshot`
accept a browser socket. Website runtime attribution, Electron inspection,
and scenario attach mode still require HTTP discovery.
- An admitted browser socket is retained by the active browser provider across
calls to the same normalized browser WebSocket URL, and concurrent cold-start
calls share one handshake so MCP sessions do not re-open the browser socket on
every step. Scheme case, uncompressed IPv6 loopback spellings, and leading
port zeros normalize to one reuse key, while `127.0.0.1`, `[::1]`, and
distinct `/devtools/browser/<id>` paths remain separate sockets. Each target
operation detaches its flattened target session when finished: unconfirmed
target attach or detach cleanup closes the shared browser socket for session
containment, whereas a caller's own discovery deadline or cancellation
releases only its lease when transport state remains clean. Provider shutdown
closes the retained browser socket. One-shot CLI invocations close their
provider on exit, so Chrome may still ask you to approve each separate CLI
connection. Discovery (`Browser.getVersion`, `Target.getTargets`) defaults to
a 10-second deadline and can be adjusted per call with `discovery_timeout_ms`
(`--discovery-timeout-ms` on the CLI).
- Each call opens one WebSocket and uses it for discovery
(`Browser.getVersion`, `Target.getTargets`) and for the selected page.
Discovery must finish within the same 10-second deadline as HTTP discovery.
Chrome may ask you to approve each connection.
- The browser identifier controls access to the socket. Evidence records the
endpoint as `ws://127.0.0.1:9222/devtools/browser/[REDACTED]`.
- `browser.revision` is the socket's `Browser.getVersion` value, such as
@@ -609,13 +598,13 @@ an unknown size, comparison reports unknown rather than treating the observation
as a resource addition or removal; complete inventory coverage still establishes
when an entire group is absent.
CDP discovery defaults to a 10-second response deadline, configurable per call
on passive browser tools with `discovery_timeout_ms` (default 10000). HTTP
discovery documents use an 8 MiB budget. Browser-socket discovery
(`Browser.getVersion`, `Target.getTargets`) uses the 64 MiB protocol message
budget. Neither byte budget has a runtime override. `list_browser_targets`
accepts `discovery_timeout_ms` and no `command_timeout_ms`, because it performs
only discovery without a post-discovery target command. Other CDP tools use
CDP discovery has a 10-second response deadline. HTTP discovery documents
use an 8 MiB budget. Browser-socket discovery (`Browser.getVersion`,
`Target.getTargets`) uses the 64 MiB protocol message budget. Neither budget
nor the discovery deadline has a runtime override. `list_browser_targets` accepts no
`command_timeout_ms`: HTTP discovery and browser-socket discovery
(`Browser.getVersion`, `Target.getTargets`) stay on that fixed deadline, and
the socket closes without a later command. Other CDP tools use
`command_timeout_ms` (default 20000) for each WebSocket handshake and command
response, separate from the caller-selected observation window, discovery, and
the 5000 ms cleanup grace. A browser-level socket's handshake and the following
@@ -635,7 +624,7 @@ rejects it as unknown; other command rejections retain their reported failure.
Network and console coverage starts only after REA attaches and enables the relevant CDP domains. `prior_activity_available` is always `false`; absence from these arrays is not evidence that an event never occurred. Scripts are different: scripts the page parsed before REA attached are still reported under `scripts`. A page that loaded before attach can therefore list its scripts while `network.requests` lacks the requests that fetched them. If CDP reuses a request ID without supplying redirect metadata, REA retains the predecessor observation and marks network coverage incomplete.
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation uses `source_map_fetch_timeout_ms` (default 30000) and a default 64 MiB aggregate streamed-response budget across redirects. Select `source_map_limits.fetch_bytes` to change byte admission for the next call. A fetch overrun is an item limitation, not a thrown analysis timeout. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding defaults to 64 MiB for raw resolved identities and 32 MiB for aggregate inline evidence; `source_map_limits.fetch_bytes` and `source_map_limits.output_bytes` own those independent admissions. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation uses `source_map_fetch_timeout_ms` (default 30000) and a 64 MiB aggregate streamed-response budget across redirects. The byte budget has no runtime override. A fetch overrun is an item limitation, not a thrown analysis timeout. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding also bounds raw resolved identities to 64 MiB and aggregate inline evidence to 32 MiB. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
A fully read response that fails UTF-8, JSON, mapping, or structural validation
retains `artifact: null`, empty original-source, module-edge, and mapping
@@ -9,10 +9,7 @@ liveness deadlines. `INPUT_JSON` may be inline JSON or a JSON file path.
The minimal request contains only browser launch/connect selection, `start_url`,
and at least one `actions` entry. Launch `headless` defaults to `true` and can
be set to `false` where the host supports a visible browser. Launched Chromium
keeps its sandbox; on hosts that cannot provide one, such as root in a
container, set `REA_BROWSER_NO_SANDBOX=true` for the REA process. Termux on
Android always launches without it. Navigations follow
be set to `false` where the host supports a visible browser. Navigations follow
the destinations named by the scenario and the page's ordinary browser requests,
and [request steps](#request-steps) send only to the start URL's origin;
there is no separate origin allowlist. Request steps use a temporary dispatch
@@ -359,7 +356,7 @@ A `request` step sends one HTTP request from the page with the page's own
recorded in the step's network events like any request the application makes.
Use it to record an application's API, including writes, when driving the UI
for each call would be slower or less exact than calling the endpoint the UI
calls. This action directly sends a caller-declared HTTP request without a user
calls. It is the only action that changes application state without a user
gesture, so run scenarios with write requests only against an instance whose
data you may change. This scenario creates one item:
+3 -18
View File
@@ -24,7 +24,7 @@ entry-point status, sizes and SHA-256. Sourcemaps, bytecode and module-info regi
are separate children. Source names such as `/$bunfs/root/main.js` remain in
metadata; extraction uses a relative path. Unsafe or non-UTF-8 names receive an
indexed path and retain their source spelling or bytes. Virtual names deeper
than the selected `path_parts` allowance (default 256) also use an indexed path with an explicit limitation,
than 256 logical path parts also use an indexed path with an explicit limitation,
preventing unbounded path-tree work while retaining the source fact. Duplicate names remain
separate occurrences; ordinary extraction collision rules apply.
@@ -43,27 +43,12 @@ compatible source files; bytecode is retained as evidence, not decoded.
Module parsing reads exact metadata ranges rather than loading stored asset
bodies or whole pages around small pointers. A cache retains at most 256 KiB
and 1,024 ranges for one reader preparation. Cumulative uncached metadata reads
default to 256 MiB, checked before allocation or I/O. The module table,
are limited to 256 MiB, checked before allocation or I/O. The module table,
decoded strings and combined logical path tree have separate admission checks;
by default at most 250,000 distinct path prefixes are admitted before inventory construction.
at most 250,000 distinct path prefixes are admitted before inventory construction.
Stored bodies are range-streamed for hashing and extraction. Header/suffix observations are reused
only for the same admitted file state, with selected-path checks before publication.
Both artifact operations accept `artifact_limits.bun`. Its independent controls
are `payload_bytes`, `module_count`, `representation_bytes`, `metadata_read_bytes`,
`recognition_table_bytes`, `recognition_load_command_bytes`, `path_parts` and
`path_nodes`. Failed recognition remains explicitly limited instead of proving
absence. ELF section names are examined at their actual offsets, including names
beyond a large section-name prefix. Select `artifact_limits.expanded_bytes` to
change the aggregate referenced-plaintext work allowance.
For a reported metadata-read refusal, keep the same carrier and retry with
`--artifact-limits '{"bun":{"metadata_read_bytes":536870912}}'` or the equivalent
MCP `artifact_limits` object. Omitted fields keep defaults. The response names
the effective limit and exact repair; accepted overrides are reported separately
from caller selections. Changes apply on the next call without restarting REA.
Summary and selected extraction do not change the producer's admission.
See [limits and recovery](design/limits-and-recovery.md) for admission ceilings.
Synthetic tests cover the carriers and malformed declarations. Run
`node scripts/verify-bun-standalone.mjs` after `npm run build:cached` for local Bun
+2 -2
View File
@@ -65,14 +65,14 @@ cancellation can still prevent upload.
that analysis was complete. A successful operation can retain partial analysis,
unknowns, and limitations. Exit zero is not a security verdict, proof of complete
coverage, or a policy gate for those findings. Inspect the saved Evidence and
coverage before making such a decision. See [output and exit status](getting-started/cli.md#output-and-exit-status).
coverage before making such a decision. See [output and exit status](cli.md#output-and-exit-status).
A nonzero exit can represent invalid input, unavailable tooling, an unsupported
host/capability, or another operation failure. Keep the original JSON error (when
emitted), stderr, and exit code, and use their category/message to diagnose it.
An early startup failure may leave `analysis.json` empty; do not parse that as
successful analysis. Select a supported operation or explicitly provision its
[documented prerequisites](getting-started/installation.md); do not silently switch providers,
[documented prerequisites](installation.md); do not silently switch providers,
execute the artifact, or ignore the failure to obtain a green job.
To exercise the failure path, temporarily set `ARTIFACT_PATH` to
+125 -239
View File
@@ -4,7 +4,7 @@ Use the CLI for a direct inspection or a script. It uses the same application
workflows and evidence contracts as REA's MCP server. Each CLI invocation is a
separate process; save results when you want to use them in a later command.
For a pinned, noninteractive workflow with retained reports, see
[static analysis in GitHub Actions](../ci-analysis.md).
[static analysis in GitHub Actions](ci-analysis.md).
## Run a command
@@ -19,7 +19,7 @@ Use an extracted application directory or an ASAR as the target. Static
JavaScript analysis returns the application graph and Evidence directly.
Generic `rea analyze PATH` selects this workflow for directories and `.asar`
files when neither `--provider` nor `--snapshot` is supplied. See
[JavaScript artifact reconstruction](../guides/javascript/javascript-artifact-reconstruction.md)
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
for results, integrity checks and coverage.
The complete application Evidence can be hundreds of megabytes for a real
@@ -39,17 +39,6 @@ pass an absolute path. REA reads and authenticates the file, then projects it
like retained Evidence, so a large file is bounded by the file reader rather
than charged as inline input.
An argument that accepts inline JSON or a JSON file path also accepts `-`, which
reads the request from standard input, so a generated request needs no
temporary file:
```bash
jq -nc '{source: {kind: "evidence-file", path: "app-evidence.json"}, view: {kind: "summary"}}' | rea inspect-analysis-view -
```
Standard input is read once as a single UTF-8 JSON document without a
byte-order mark, and it is limited to 8 MiB. Put a larger request in a file.
Local file inspection commands reject empty paths and NUL bytes before resolving
relative inputs, auxiliary files or output destinations. An empty string never
selects the working directory; use an explicit `.` when selecting that directory
@@ -59,7 +48,7 @@ For example, a 334 MB Obsidian application Evidence file yields a summary view
of about 10 KB. Replace the view with
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
module page. See
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md) for
[JavaScript application workflows](javascript-application-workflows.md) for
views, feature traces and comparisons that take the same saved Evidence.
Add `--timings` to a command to write one JSON timing record to stderr. For
@@ -226,7 +215,7 @@ saving the complete original Evidence in a new bundle. Summary requires that
explicit destination; existing files are preserved. Later
`inspect-analysis-view` calls can page the exact parent from the saved bundle
without reopening the assembly. See
[CLI summary and saved member rows](../guides/managed/managed-code-analysis.md#cli-summary-and-saved-member-rows)
[CLI summary and saved member rows](managed-code-analysis.md#cli-summary-and-saved-member-rows)
for the complete command sequence. Omitted or `--detail complete` retains the
full inline response.
@@ -234,8 +223,8 @@ Choose the workflow from the artifact's contents rather than its `.exe` or
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
unpack .NET single-file hosts or decode IL2CPP. For the image, type and method
names in a Unity `global-metadata.dat`, use
[IL2CPP metadata](../il2cpp-metadata.md). NativeAOT method bodies require native
analysis. See the [managed-code guide](../guides/managed/managed-code-analysis.md)
[IL2CPP metadata](il2cpp-metadata.md). NativeAOT method bodies require native
analysis. See the [managed-code guide](managed-code-analysis.md)
for supported deployment forms, selection semantics, and coverage limits.
## SQLite snapshots
@@ -243,7 +232,7 @@ for supported deployment forms, selection semantics, and coverage limits.
For a local database snapshot, use `rea inspect-sqlite-database PATH --json`.
Add `--table NAME --row-limit 20` to read a bounded sample from an ordinary
table. This command does not require a native analysis provider. See
[SQLite database snapshots](../guides/data/sqlite-databases.md) for database/WAL identity,
[SQLite database snapshots](sqlite-databases.md) for database/WAL identity,
source-file handling and row-value semantics.
## Native analysis
@@ -256,7 +245,7 @@ rea inspect-go-binary /absolute/path/to/program --json
```
The shared CLI/MCP workflow reports compiler version, modules, replacements,
build settings, original bytes and SHA-256. See [Go binaries](../guides/native/go-binaries.md)
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
for supported containers and absent or malformed metadata.
Inspect a Godot PCK without running the game:
@@ -267,7 +256,7 @@ rea extract-artifact game.pck --json
```
Formats 2, 3 and 4 retain header, directory and integrity facts. Compact delivery
keeps complete Evidence in the requested bundle. See [Godot PCK](../godot-pck.md)
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
for embedded packs, batch queries and unknown encrypted coverage.
Inspect a UnityFS AssetBundle without running Unity:
@@ -278,20 +267,9 @@ rea extract-artifact content.bundle --json
```
Format versions 6 through 8 retain header, block and directory facts with
none/LZ4/LZ4HC block decompression. See [UnityFS](../unityfs.md) for encryption
none/LZ4/LZ4HC block decompression. See [UnityFS](unityfs.md) for encryption
and LZMA coverage boundaries.
Inspect a WeChat mini-program package without WeChat or an unpacker:
```sh
rea inspect-artifact app.wxapkg --detail summary --evidence_output evidence.json --json
rea extract-artifact app.wxapkg --json
```
Plaintext packages retain header, table and member facts with verbatim member
bytes. `V1MMWX` packages are detected as encrypted coverage and refused on
extraction. See [wxapkg](../wxapkg.md) for layout and coverage boundaries.
Inspect an ESP32-family app or bootloader image without esptool:
```sh
@@ -301,7 +279,7 @@ rea extract-artifact firmware.bin --json
The inventory lists each segment with its ESP-IDF load classification, the
checksum and SHA-256 results, the descriptor and any Secure Boot v2 signature
blocks. See [ESP images](../esp-image.md) for recognition, coverage and refusals.
blocks. See [ESP images](esp-image.md) for recognition, coverage and refusals.
Inspect an ESP flash dump, or one read from a flash address other than 0:
@@ -313,7 +291,7 @@ rea extract-artifact flash.bin --json
The inventory lists the bootloader, the partition table and each partition,
with ESP image facts for app partitions and the OTA records. See
[ESP flash images](../esp-flash-image.md) for table location, inputs and limits.
[ESP flash images](esp-flash-image.md) for table location, inputs and limits.
Inspect a Node.js SEA carrier without running it:
@@ -322,7 +300,7 @@ rea inspect-artifact ./application --json
rea extract-artifact ./application --json
```
See [Node.js SEA](../node-sea.md) for supported layouts, script and asset paths,
See [Node.js SEA](node-sea.md) for supported layouts, script and asset paths,
fuse observations, and the real-fixture verification lane.
Inventory and extract Intel HEX or S-record regions:
@@ -332,7 +310,7 @@ rea inspect-artifact ./firmware.hex --json
rea extract-artifact ./firmware.hex --json
```
See [Intel HEX and S-record images](../hex-records.md) for recognition, region
See [Intel HEX and S-record images](hex-records.md) for recognition, region
names, metadata and limits.
Inspect and extract stored U-Boot legacy uImage payloads and components:
@@ -342,17 +320,14 @@ rea inspect-artifact ./firmware.bin --detail summary --evidence_output firmware.
rea extract-artifact ./firmware.bin --json
```
See [U-Boot uImage](../uimage.md) for verified CRCs, physical component ranges,
See [U-Boot uImage](uimage.md) for verified CRCs, physical component ranges,
unknown names, resource limits and retained views. Compressed bytes stay stored.
`inspect-artifact` and `extract-artifact` also read ZIP members compressed with
Zstandard (standard method 93). They retain the shared ZIP CRC, declared-size,
overlap and archive expansion checks. The native streaming decoder keeps the
existing 128 MiB Zstandard window cap; frames requiring a larger window are
reported as a resource limit (`reason: limit`, `boundary: memory`), rather than
as archive corruption. The cap cannot be raised at runtime: recompress with a
window of at most 128 MiB, or decompress externally before retrying.
Deprecated method 20 and other unregistered ZIP codecs remain unsupported.
refused. Deprecated method 20 and other unregistered ZIP codecs remain unsupported.
Inventory and extract UF2 or DfuSe regions:
@@ -361,7 +336,7 @@ rea inspect-artifact ./firmware.uf2 --json
rea extract-artifact ./firmware.dfu --json
```
See [UF2 and DfuSe images](../uf2-dfuse.md) for recognition, families, numbering,
See [UF2 and DfuSe images](uf2-dfuse.md) for recognition, families, numbering,
the DfuSe suffix CRC and limits.
Inventory and extract U-Boot FIT images:
@@ -371,7 +346,7 @@ rea inspect-artifact ./boot.itb --json
rea extract-artifact ./boot.itb --json
```
See [U-Boot FIT images](../fit-images.md) for recognition, data sources, hashes
See [U-Boot FIT images](fit-images.md) for recognition, data sources, hashes
and limits.
Inspect local model declarations without loading weights:
@@ -385,7 +360,7 @@ Default hashing reads only headers and leaves full-file identity unknown.
`--hash full` streams the complete file for SHA-256. Both return typed metadata,
original tensor declarations and derived totals. MCP summaries retain complete
Evidence for filtered metadata, tensor and type-total pages; CLI compact delivery
uses `--detail summary --evidence_output FILE`. See [model files](../model-artifacts.md).
uses `--detail summary --evidence_output FILE`. See [model files](model-artifacts.md).
Inspect Unity IL2CPP metadata names without Unity or a disassembler:
@@ -402,7 +377,7 @@ method instances and `--ghidra-seeds PATH` then writes Ghidra function seeds. It
with retained Evidence; query indexed rows with filters, batch indices and
selected fields. The CLI defaults to complete Evidence;
`--detail summary --evidence_output FILE` exports the full result to a new file
and returns a compact overview. See [IL2CPP metadata](../il2cpp-metadata.md).
and returns a compact overview. See [IL2CPP metadata](il2cpp-metadata.md).
Inspect an embedded cargo-auditable Rust dependency list without installing
Rust or configuring a disassembler:
@@ -414,7 +389,7 @@ rea inspect-rust-binary /absolute/path/to/program --json
The same workflow reports package names, versions, sources, the root package,
dependency edges, and the section's file range. A missing `.dep-v0` section is
null metadata, not proof that the binary has no dependencies. See
[Rust binaries](../rust-binaries.md).
[Rust binaries](rust-binaries.md).
Inspect a PE Authenticode certificate table without executing the image or
verifying signatures:
@@ -426,7 +401,7 @@ rea inspect-pe-certificates /absolute/path/to/application.exe --json
The workflow reports `WIN_CERTIFICATE` entries and, for PKCS#7 signedData,
embedded X.509 serial, issuer, subject, and validity fields. A missing security
directory is observed absence of that table. See
[PE Authenticode certificate tables](../pe-certificates.md).
[PE Authenticode certificate tables](pe-certificates.md).
Inspect PE import and delay-load tables without executing the image:
@@ -437,20 +412,7 @@ rea inspect-pe-imports /absolute/path/to/application.exe --json
The workflow reports DLL names, named and ordinal functions, delay-load
descriptors, and a pefile-compatible import hash of the standard import table.
A missing import directory is observed absence of that table. See
[PE import and delay-load tables](../pe-imports.md).
Inspect a Windows Shell Link (`.lnk`) without resolving or executing the
target:
```bash
rea inspect-windows-shortcut /absolute/path/to/shortcut.lnk --json
```
The workflow reports header flags and FILETIMEs, IDList items, LinkInfo volume
and network names, StringData arguments and icon location, ExtraData blocks
including Tracker droid facts labelled derived, and derived observations such
as argument length and a minimized ShowCommand. See
[Windows Shell Link shortcuts](../guides/native/windows-shortcut.md).
[PE import and delay-load tables](pe-imports.md).
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
@@ -460,33 +422,7 @@ rea inspect-elf-notes /absolute/path/to/program --json
The workflow reports note owners, types, descriptor hashes, and GNU build-id
or ABI-tag fields when those descriptors match. A missing note table is
observed absence of those declarations. See [ELF note tables](../elf-notes.md).
List ELF and Mach-O initializer and finalizer functions without executing the
image:
```bash
rea inspect-native-initializers /absolute/path/to/program --json
```
The workflow reports `DT_INIT`, init/fini arrays, legacy `.ctors`/`.dtors`, and
Mach-O `__mod_init_func`, `__init_offsets` and `__mod_term_func` entries with
their resolved targets. Universal Mach-O files need `--architecture`. See
[ELF and Mach-O initializer functions](../guides/native/native-initializers.md).
Inspect ELF unwind function ranges without executing the image:
```bash
rea inspect-elf-unwind-ranges /absolute/path/to/program --json
```
The workflow reports FDE start, end, and size, the containing section, an exact
symbol name when one exists, CIE identity, LSDA, and personality. Coverage
counts ranges that match a symbol, ranges without a symbol, and whether the
`.eh_frame_hdr` search table agrees with the decoded FDEs. `.debug_frame` is
reported present, absent, or incomplete and is not decoded. A missing unwind table is
observed absence of those declarations. See
[ELF unwind function ranges](../guides/native/elf-unwind-ranges.md).
observed absence of those declarations. See [ELF note tables](elf-notes.md).
Inspect an Apple `.ips` crash report without executing a target:
@@ -497,7 +433,7 @@ rea inspect-apple-ips /absolute/path/to/report.ips --json
The workflow reports first-line metadata, process/exception/termination facts,
threads, usedImages, lossless integer registers, and documented crash classes
when `bug_type` is `"309"`. Other DiagnosticReports types return no crash
result. See [Apple IPS crash reports](../apple-ips.md).
result. See [Apple IPS crash reports](apple-ips.md).
Inspect Linux kernel module metadata without executing the module or requiring
`modinfo`:
@@ -513,26 +449,9 @@ not decoded and the signature is not validated. Other ELF types are not
applicable. gzip, xz, and zstd containers are unsupported and name
`external_decompression` as the next probe: decompress with an external
utility, then inspect the decompressed file. See
[Linux kernel modules](../kernel-modules.md).
[Linux kernel modules](kernel-modules.md).
Read a Linux kernel image's version banner and embedded `CONFIG_IKCONFIG`
build config without executing it:
```bash
rea inspect-kernel-build-config /absolute/path/to/vmlinux --json
rea inspect-kernel-build-config /absolute/path/to/bzImage --name CONFIG_MODULE_SIG --json
```
The workflow reports the banner version, builder, and compiler, the config
option count, and paged `CONFIG_*` entries (`y`, `m`, strings, numbers, and
`not_set`). An `IKCFG_ST` string counts as a config only when a valid gzip
member follows it and `IKCFG_ED` follows that member. A gzip bzImage payload
is inflated and searched; other payload codecs are unsupported and name
`external_decompression` as the next probe. See
[Linux kernel build config](../guides/native/kernel-build-config.md).
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects (3.14 and
3.15 headers are qualified but their marshal layout is not pinned)
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects
without executing bytecode or installing Python:
```bash
@@ -541,7 +460,7 @@ rea inspect-python-pyc /absolute/path/to/module.pyc --json
The workflow reports the pinned CPython version for the magic, PEP 552 hash or
mtime/size fields, nested function names, and bytecode SHA-256. Bytecode is
not decompiled. See [CPython pyc inspection](../python-pyc.md).
not decompiled. See [CPython pyc inspection](python-pyc.md).
Inspect an Erlang BEAM file without executing bytecode:
@@ -551,19 +470,10 @@ rea inspect-erlang-beam /absolute/path/to/app.beam --json
The workflow reports IFF chunk names, atom tables, exports, imports, lambdas,
and the Code chunk header. Missing optional tables stay null. See
[Erlang BEAM files](../erlang-beam.md).
Complete `rea decompile --json` Evidence can be saved and inspected through
`inspect-analysis-view` with an inline source and
`view: {kind: "native", facet: "pseudocode", offset: 0, limit: 20000}`.
The projection reads that saved observation without starting a provider;
offsets/limits count UTF-16 code units. A summary view preserves the original
selector and distinguishes null from empty text. The MCP tool additionally
supports `detail: "summary"` on its first call to retain the complete source
without returning the body inline; see [native investigation](../guides/native/native-investigation.md).
[Erlang BEAM files](erlang-beam.md).
Configure [Hopper or Ghidra](installation.md#hopper), the
[IDA adapter](../providers/ida-provider.md), or [Kuna](../kuna-provider.md) before analyzing a
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
native target. Substitute
your target, search text and function name or address in these examples:
@@ -591,11 +501,11 @@ MCP `analyze_function` defaults to the same summary and retains the complete
dossier in the session for `inspect_analysis_view`.
Run `rea --help` or a command's `--help` for arguments and output options.
The [generated catalog](../reference/mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
The [generated catalog](mcp-contracts.md#generated-catalog) lists all CLI commands and MCP
tools. Provider-specific target support is described in the
[native guide](../guides/native/native-investigation.md), [DOS guide](../providers/ghidra-dos.md),
[raw binary guide](../providers/raw-binary.md), [GameCube DOL guide](../ghidra-dol.md), and
[Windows Ghidra guide](../providers/windows-ghidra-p0.md).
[native guide](native-investigation.md), [DOS guide](ghidra-dos.md),
[raw binary guide](raw-binary.md), [GameCube DOL guide](ghidra-dol.md), and
[Windows Ghidra guide](windows-ghidra-p0.md).
## Choose a provider
@@ -642,20 +552,19 @@ For Hopper, Ghidra, IDA, JADX, and Kuna, run
such as `pwntools-elf`, report their configuration requirements directly and are
not accepted by `doctor --provider`. See
[task readiness](installation.md#check-readiness-for-your-task) and
[provider selection](../reference/mcp-contracts.md#identity-and-discovery).
[provider selection](mcp-contracts.md#identity-and-discovery).
The session reports work still in progress through `analysis_activity`.
A client timeout can end its wait while the provider continues analyzing.
`cleanup_incomplete` identifies resources whose shutdown or removal could not
be verified. See [MCP contracts](../reference/mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for import, client timeouts and recovery.
A one-shot command attempts provider-session and runtime cleanup before reporting
a failure. If cleanup remains incomplete, inspect the reported resources and
follow the error's remediation. [Same-session cleanup retry](../reference/mcp-contracts.md#retained-cleanup-retry),
reuse and `update_analysis_settings` apply to a live MCP connection.
For one-shot commands, `details.limit` reports `scope: "rea-invocation"` and
A one-shot command releases its provider session and REA runtime before it
prints a failure, so it omits the same-session reuse and
`update_analysis_settings` guidance an MCP connection receives. Its
`details.limit` reports `scope: "rea-invocation"` and
`applies_at: "next_invocation"`; set the named `environment` (or the per-call
`parameter`) for a new invocation and rerun. For example, a Ghidra decompile
deadline names `REA_GHIDRA_DECOMPILE_TIMEOUT_SECONDS`.
@@ -684,10 +593,12 @@ an exact result when the target bytes, operation, parameters, provider and
settings match. Mutations and cursor-dependent calls are excluded from the
cache. Snapshot files are local and use owner-only permissions.
A binary CLI replay reports the current invocation's selected path while leaving
the saved Evidence and snapshot query unchanged. See
[caller-selected paths](../reference/mcp-contracts.md#caller-selected-paths)
for primary and auxiliary path fields and their relation to Evidence identity.
One-shot binary CLI Evidence reports the exact path argument in `caller_input_path`,
including relative spelling and symlink aliases. Its subject identifies the
inspected artifact; `caller_input_path` does not change the Evidence ID or
snapshot query. A replay reports the current invocation's path while leaving
the saved Evidence unchanged. Provider inspections continue to include the
established `parameters.input_path` field.
Importing an Evidence bundle refreshes retained request and subject display metadata.
It preserves cached analysis results, including observations for other targets.
@@ -718,11 +629,6 @@ rea analyze /absolute/path/to/program --provider ghidra --snapshot /absolute/pat
```
An exact CLI cache hit is normally read before starting a provider process.
`decompile --provider ghidra --snapshot ...` also retains its single selected
Program's pseudocode result, even without an explicit document selector. Old
snapshots that contain only the Evidence need one new query to add that replay
entry. This does not retain every body from `decompile-all` or make a persistent
session's implicit document replayable.
On Windows, Ghidra first admits the selected source through its native snapshot
owner, so replay can start that provider.
In MCP, `open_binary` accepts `snapshot_path` to import a snapshot atomically
@@ -775,10 +681,6 @@ results and Evidence. These are cached observations, not a saved provider
database: only eligible exact queries can reuse a result, and new or live
queries can still require provider startup. The CLI uses the same snapshot
format and preserves these records when loading and updating it.
When an MCP session replays a provider observation, its delivered Evidence identifies
the original observation through `evidence_links` and reports that the
provider was not rerun. The original and its available source records remain
in the investigation bundle.
## Import, export and compare Evidence
@@ -796,23 +698,15 @@ rea compare left.json right.json --record_pairs '[{"left_evidence_id":"ev_<left-
Migration: replace `rea compare-bundles` with `rea compare`; the former command
has been removed. The MCP tool remains `compare_bundles`.
`rea evidence-import` performs process-local bundle authentication and
validation, reporting record and Unknown-revision counts. `unknowns_added`
counts revisions, not distinct Unknown IDs. It does not retain records in a
separate or background MCP server process. To retain records across calls in an
active MCP session, use the MCP tool `import_evidence_bundle` in that session.
`rea compare` compares records by Evidence identity and explicit pairs. It does
not read artifact contents: the inventories of two builds are unresolved
records, or one opaque change when you pair them. Use `compare-artifacts`,
described at the end of this section, to compare their contents.
`--record_pairs` accepts an inline JSON array or the path to a local JSON
file containing that array, matching MCP `compare_bundles.record_pairs`. For a
file, use `rea compare left.json right.json --record_pairs pairs.json --json`.
Pair IDs must exist on their named sides and be one-to-one. Without pairs,
one-sided membership remains unknown; REA does not infer correspondence from
position or display names.
`--record_pairs` supplies an explicit JSON array of observation pairs, matching
MCP `compare_bundles.record_pairs`. Pair IDs must exist on their named sides and
be one-to-one. Without pairs, one-sided membership remains unknown; REA does
not infer correspondence from position or display names.
Exports preserve an existing destination unless `--overwrite` is explicit.
@@ -823,8 +717,8 @@ protection still applies. A genuine pathname refusal uses `resource_constraint`
with `details.reason: path-limit` and `boundary: filesystem-path`.
For application traces and version comparisons, see
[JavaScript application workflows](../guides/javascript/javascript-application-workflows.md).
MCP [retained Evidence references](../reference/mcp-contracts.md#retained-application-evidence-inputs)
[JavaScript application workflows](javascript-application-workflows.md).
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
belong to one connection. Separate CLI calls can consume full saved records or
resolve exact retained IDs from an explicitly selected local bundle:
@@ -859,8 +753,7 @@ from the bundle for every supported view kind. Bundles exported from MCP work
for these commands. Its inline source form cannot also select a bundle.
A missing ID is refused with `details.reason: "missing"`;
supplying a bundle does not make incompatible Evidence valid for a workflow.
`compare-managed-members` keeps its two assembly paths and an optional
repeatable `--type` selection applied to both sides, and
`compare-managed-members` keeps its two assembly paths, and
`compare-process-captures` keeps its two capture Evidence JSON paths. The bundle
option above selects an input; producer options that write a new bundle, such
as `inspect-managed-members --evidence-bundle`, retain their existing meaning.
@@ -898,7 +791,6 @@ Import an older source tree to compare with the current artifact:
```bash
rea import-reference-source /absolute/path/to/source
rea import-reference-source /absolute/path/to/source --detail summary --outline-depth 2 --json
rea import-reference-source /absolute/path/to/source --exclude-paths '["vendor/","*.generated","!keep.generated"]'
```
The default (`--detail full`) returns the complete `HistoricalSourceGraph`.
@@ -1032,23 +924,19 @@ for percent signs, query marks and fragments.
Historical-source import requires safe no-follow file opens on Linux or
macOS. Native Windows returns `unsupported_host`; use Linux REA inside WSL
or another supported host. See
[source-to-bundle comparison](../guides/javascript/javascript-application-workflows.md#historical-source-to-bundle-comparison)
[source-to-bundle comparison](javascript-application-workflows.md#historical-source-to-bundle-comparison)
for mapping identities, inferences and unknowns.
## Capture runtime behavior
Choose the guide for the target and intended interaction:
- [Browser observation](../guides/web/browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](../guides/web/browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](../guides/javascript/electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Electron fuses](../electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](../guides/javascript/javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](../guides/runtime/process-capture.md): run an executable and compare terminal, exit and filesystem observations.
- [Desktop application capture](../desktop-application-capture.md): launch a Linux desktop application and record its windows and home-directory state at checkpoints while it is used.
- [Desktop application state](../desktop-application-state.md): read a capture folder into the application's state files, typed views of each version and kinds of change.
- [Desktop application export](../desktop-application-export.md): write a capture folder as an agentenv computer app bundle with its class, skill and seed.
- [Desktop application tasks](../desktop-application-tasks.md): describe how a capture's files changed, then validate an agent-written task against the capture and export it with its verifier.
- [Browser observation](browser-observation.md): inspect a selected page through CDP.
- [Browser scenarios](browser-scenario-contract.md): run declared interactions and capture their results.
- [Electron observation](electron-observation.md): inspect an Electron renderer or capture an application scenario.
- [Electron fuses](electron-fuses.md): read packaged fuse bytes before expecting `--inspect` to reach an Electron main process.
- [Node/Electron Inspector](javascript-runtime-observation.md): record script locations and execution contexts.
- [Process capture](process-capture.md): run an executable and compare terminal, exit and filesystem observations.
Runtime requests name the target and actions. Launched targets run with your
user permissions; consult the chosen guide for host requirements and effects.
@@ -1087,7 +975,7 @@ one client at a time: a session ends with the client's `DELETE`, and a client
that connects while another session is open takes it over, so a client that
exited without ending its session needs no restart. The listener runs until
SIGINT or SIGTERM and does not stop when stdin closes. See
[transports](../reference/mcp-contracts.md#transports) for the session, `Host` header and
[transports](mcp-contracts.md#transports) for the session, `Host` header and
shutdown contract. Client registrations written by setup and the MCP registry
entry keep stdio.
@@ -1112,7 +1000,7 @@ answers every method with JSON-RPC `-32601`. The stopped document's `state`
member is `{"served": true, "evidence_id": …}`,
`{"served": false, "reason": "replay-only"}` or
`{"served": false, "reason": "unavailable", "code": …, "message": …}`; see
[serve from the state](../agent-environments.md#serve-from-the-state).
[serve from the state](agent-environments.md#serve-from-the-state).
```sh
rea environment-serve manifest.json --evidence capture.json --port 8765
@@ -1132,7 +1020,7 @@ MCP_ALLOWED_HOSTS=127.0.0.1,rea.lan rea environment-serve manifest.json --eviden
the served environment; set but blank, it is refused as `configuration_invalid`
like a blank `MCP_HOST`. The admitted client URL is written to stderr as one JSON log line
(`"msg":"MCP transport listening"`). See
[agent environments](../agent-environments.md) for what the state and replay
[agent environments](agent-environments.md) for what the state and replay
return and do not establish.
## Build an environment bundle
@@ -1157,7 +1045,7 @@ command. `--dry-run` writes nothing and may omit the directory.
rea environment-build grafana.har ./grafana --json
```
See [agent environments](../agent-environments.md#build-a-bundle-in-one-step)
See [agent environments](agent-environments.md#build-a-bundle-in-one-step)
for the report, the hints and what the bundle ships.
## Export an environment bundle
@@ -1190,26 +1078,9 @@ name Windows reserves (`con`, `nul`, `com1` and the like) is refused. The
command prints one JSON document listing the written files with their sizes,
the pinned package source, the served `tool_count`, and `state` in the shape
of the serve command's stopped document. See
[agent environments](../agent-environments.md#export-a-bundle) for the folder
[agent environments](agent-environments.md#export-a-bundle) for the folder
layout and what the smoke task checks.
`rea environment-export` prints the written files and exits `0` on success.
Input-pair refusals, an occupied destination, a missing package tarball, and
invalid labels fail before writing output. Host permission failures identify
whether reading or writing was denied and retain the affected path.
A failure during writing can leave a partial folder. Inspect it before removing
it and retrying; the next attempt refuses a non-empty destination. When one
file caused the failure, the diagnostic names that file. A folder-sync failure
can report only its cause.
If `details.partial_observation.kind` is `environment-bundle-publication`, the
bundle was already published. Its `directory` and `files` record the committed
output, even when a later identity check or cleanup failed. Inspect that bundle
and resolve the reported failure before using or replacing it; the receipt is
not an instruction to delete published output. Cleanup resources remain a
separate obligation.
## Compare an environment with its application
`rea environment-compare <manifest.json> --evidence <capture.json> --probe <probe.json> [--probe <probe.json> ...]`
@@ -1220,7 +1091,7 @@ answer is compared with the application's after ids are mapped and clocks and ge
makes no network call. It prints the report and exits `0`, or prints a structured `invalid_request` error and exits `1`
for the same pair refusals as `environment-serve`, a probe that is not capture Evidence, or a browser probe captured
without response bodies, or without the request bodies its writes need. See
[agent environments](../agent-environments.md#measure-fidelity-against-the-application) for the report.
[agent environments](agent-environments.md#measure-fidelity-against-the-application) for the report.
```sh
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
@@ -1277,10 +1148,6 @@ such as `UNSUPPORTED_OUTPUT_COMBINATION` for `--json` with `--token-limit`. Use
Commands reject surplus positional arguments before execution, including commands
that accept no positionals. Use `rea <command> --help` for the declared argument
shape; a final variadic argument continues to accept multiple values.
`rea mcp add` accepts options only and rejects unknown options before reading or
backing up client settings or starting registration. Its `--agent` and
`--command`/`-c` options accept spaced and equals forms. Values must be nonempty; a following flag does not satisfy a spaced option's
missing value.
`rea mcp doctor --json` checks production MCP startup and reports child logs in
`diagnostics`. It drains stderr while retaining at most its first 64 KiB of raw
@@ -1321,12 +1188,9 @@ treated as malformed.
exact array path with `--filter-output` and provide comma-separated direct
scalar columns with `--columns`; column order is the output column order:
For an artifact inventory request that selects the `occurrences` collection
and includes `logical_path` and `artifact_id` in its view fields:
```bash
rea inspect-analysis-view request.json --format csv \
--filter-output normalized_result.items --columns logical_path,artifact_id
--filter-output normalized_result.rows --columns name,address
```
The first line is a REA metadata record,
@@ -1401,17 +1265,60 @@ retains failed stages for retry; the CLI exposes no separate retry command.
or prints the refusing stage's structured error and exits `1`, writing nothing
when the refusal comes before the bundle is written. SIGINT cancels it with a
`cancelled` error; a cancellation before the bundle stage writes nothing.
A `cancelled` result does not by itself determine the exit status. Commands
that handle SIGINT or SIGTERM assign `130` or `143` after their operation and
cleanup settle; their structured result can be printed before that status is
assigned. Read the process status and typed result separately.
`rea environment-export` is one-shot: it prints the written files and exits
`0`, or prints a structured `invalid_request` error and exits `1` for the same
pair refusals, a non-empty or file-occupied directory, a missing package
tarball or a name that is not a kebab-case label or is a Windows device name,
writing nothing in each case. A file occupying the destination is reported as
requiring a directory; identity-change diagnostics are reserved for admitted
paths that fail later identity checks. Host permission failures identify whether
input reading or bundle writing was denied and retain the reported path.
A write that fails after the first file (a permission, disk-space or
pathname-length failure on a later path) propagates as the host's failure,
naming its cause and, when it failed on one bundle file or the package
tarball, that file. A failure while syncing the finished folder names only its
cause. Either way the partial folder stays in place for inspection; remove it
before retrying, since the next attempt refuses the non-empty directory.
Commands with `--evidence_output` save the complete Evidence bundle before
delivering the selected view. For committed-file receipts, retained partial
Evidence and missing source records, see
[Evidence publication and source recovery](../reference/mcp-contracts.md#evidence-publication-and-source-recovery).
Other outputs have their own receipt shapes; for example,
[environment export](#export-an-environment-bundle) reports a published directory.
A `cancelled` result does not by itself determine the exit status; it depends on
the command's signal handling. Commands routed through the shared
`withCommandCancellation` handler in the CLI set status `130` for SIGINT or
`143` for SIGTERM after their operation settles. Current source registrations
use that handler for process capture and comparison (`capture-process`,
`compare-process-captures`), JavaScript application analysis (`analyze` when it
selects the JavaScript lane, and `analyze-javascript-application`), selected
browser, web and Electron commands (including `analyze-web-bundle`,
`observe-web-execution`, `capture-browser-scenario` and
`capture-electron-scenario`), and the platform and provider command families
for Android/ADB, Apktool/JEB/Flutter, firmware, PCAP and signal analysis,
WASM/EVM, Go/Rust and binary diagnostics, Frida instrumentation, the Cutter
command, objdump, Rizin and GDB, and the agent environment server
(`environment-serve`). The handler also wraps `doctor`,
`inspect-sqlite-database`, `recover-javascript-sources`,
`inspect-il2cpp-metadata`, and `inspect-desktop-application`. In these paths, a returned structured
`cancelled` result can be printed before the wrapper assigns the
signal-derived status. This describes the current source routing; it is not a
claim that every command or provider runtime has been independently exercised.
Commands with `--evidence_output` select their delivered view, then publish the
complete Evidence bundle atomically. Cancellation before publication leaves no
new file. A publication that has committed can win a later cancellation; the
`cancelled` result then keeps the written path and byte count under
`details.diagnostics.evidence_bundle`. If staging cleanup stays incomplete
after commit, the `cleanup_incomplete` error reports the same file as
`details.published_path` and `details.bytes`.
When a failure retains a completed observation and no bundle committed,
`details.source_closure` states what that observation cites. `publication` is
`not_published`. `records` holds authentic source Evidence this invocation
captured, within 64 KiB. `omitted` lists up to 64 other identities and
`omitted_count` the total: `not_captured` means the invocation holds no such
record, so nothing is shown for it or its ancestors; `not_inlined` means it was
captured but is not in this response. `complete` is true only when every cited
record is in `records` or embedded in the observation. If examining the closure
itself fails, `collection_failure` names the error type and the closure state is
unknown; the original failure is still returned. Nothing is retained after exit,
and running the command again is a new observation.
When the reader of piped output closes early, as `rea ... | head` does, the
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
@@ -1424,11 +1331,7 @@ status with the analysis failure status, so it exits `1`. Setup has its own cont
structured result and process status as separate signals when scripting
cancellation.
`setup --dry-run` returns `planned` and exits `0`. For `setup` and
`environment-build`, explicit `--dry-run=true` and `--dry-run=false` are
accepted; empty or other values fail with `PARSE_ERROR` before any changes.
Every supplied value is checked even when a later occurrence overrides it.
A cancelled setup also exits
`setup --dry-run` returns `planned` and exits `0`; a cancelled setup also exits
`0`. Setup returns `1` for `needs_confirmation` or `needs_human`.
`doctor` returns `1` when required checks in its selected readiness scope fail;
unavailable optional providers remain informational for unrelated tasks.
@@ -1455,7 +1358,7 @@ the total token count and have no next offset.
Inspect an explicit local FDT v16/v17 blob without execution or an external tool.
Returns header, reservations, node/property tree, derived standard values and
selected phandle references. See [devicetree inspection](../devicetree.md) for
selected phandle references. See [devicetree inspection](devicetree.md) for
limits and retained-view pagination.
```bash
@@ -1471,26 +1374,9 @@ rea inspect-pe-load-config /absolute/path/to/application.exe --json
Reports TLS callbacks and size-covered load-config fields without execution.
Use `--max-callbacks` and `--max-file-bytes` to set read budgets. See
[PE TLS and load configuration](../pe-load-config.md).
### Binary hardening
```sh
rea inspect-binary-hardening /absolute/path/to/program --json
```
Reports ELF, PE, and Mach-O hardening rows without execution. A missing import
is `unknown`, not `disabled`. Use `--max-file-bytes` to set the read budget. See
[binary hardening](../binary-hardening.md).
[PE TLS and load configuration](pe-load-config.md).
Bun standalone executables use the existing `inspect-artifact` and
`extract-artifact` commands to inventory and materialize embedded source,
sourcemaps and bytecode without running the executable. See
[Bun standalone executables](../bun-standalone.md) for layout coverage and limits.
Deno compile executables use the same commands to inventory and materialize the
embedded virtual filesystem, remote modules, and recorded options. Inventory retains
embedded environment-file values and certificate bytes in local Evidence with their
counts and byte lengths, and extraction does not write them as files. The npm snapshot
length is reported and not decoded, and the executable is not run. See
[Deno compile executables](../deno-standalone.md).
[Bun standalone executables](bun-standalone.md) for layout coverage and limits.
@@ -24,8 +24,8 @@ the registration uses them. Clearing either makes startup fail without resolving
another package from the client's project. Restore the same installation before
repairing it; an unidentified registration remains untouched. See
[installation and recovery](installation.md#package-runners-and-global-installation).
Setup forwards supported, explicitly selected REA settings into client
configuration, but does not copy unrelated ambient environment values. To refresh an older registration, preview
Setup does not copy your ambient environment into client
settings. To refresh an older registration, preview
`rea setup --client <client-id> --skill=false --dry-run`, then run the same command
without `--dry-run` and approve the displayed changes. Setup preserves unrelated
settings, backs up changed configuration and leaves an aligned registration
@@ -43,37 +43,12 @@ Rerun setup to review the registration and restart the client.
executable and environment of the terminal running doctor. It does not launch
the client's configured command or reproduce the client's environment. If doctor
passes but the client disconnects, inspect the client's startup error and registered
launcher; see [verification scope](../development/client-verification.md#launch-environment-verification).
launcher; see [verification scope](client-verification.md#launch-environment-verification).
`rea doctor` has the same limit for provider checks, and reports a provider setting
the registration's file does not carry as a
[non-blocking unknown](installation.md#check-readiness-for-your-task).
## Setting launch variables in PowerShell
Shell examples that prefix a command with `REA_MCP_INPUT_SCHEMA_PROFILE=compact`
use POSIX shell syntax. In PowerShell, set the process environment before
launching REA:
```powershell
$env:REA_MCP_INPUT_SCHEMA_PROFILE = "compact"
rea mcp doctor --json
```
The assignment applies to the current PowerShell process and commands it launches.
Restore the previous value when finished, or use
`Remove-Item Env:REA_MCP_INPUT_SCHEMA_PROFILE` if it was previously unset. It does
not update an already running MCP server or desktop client.
To select this profile for an agent, set `REA_MCP_INPUT_SCHEMA_PROFILE` in that
client's existing REA registration `env` object (`environment` for OpenCode) and
restart the client, preserving its other settings. A successful terminal doctor run does not verify the client's
launch environment. See [launch-environment verification](../development/client-verification.md#launch-environment-verification)
and the client-specific instructions below. REA setup forwards supported selections,
including `REA_MCP_INPUT_SCHEMA_PROFILE`, from its environment into the registration;
it does not copy unrelated environment values. Restoring the shell's previous
value does not undo a selection already saved by setup.
## Runtime settings and client deadlines
`update_analysis_settings` changes curated defaults inside the running REA
@@ -83,7 +58,7 @@ raising a REA startup or operation deadline, separately ensure the client allows
enough time for the full request. Use `inspect_analysis_settings` for the running
server's defaults; `rea doctor` checks launch configuration and cannot observe
another process's runtime overrides. See the
[settings design](../design/limits-and-recovery.md) for ownership and lifetime.
[settings design](design/limits-and-recovery.md) for ownership and lifetime.
## Client tool names and discovery
@@ -102,86 +77,11 @@ does not repair a client's naming behavior or infer permission drift from an
unfamiliar prefix. The unavailable fork reported in
[#5512](https://github.com/morluto/rea/issues/5512) has not been reverified.
<a id="hotcold-catalogs"></a>
## Hot/cold catalogs
For a client that supports MCP catalog refresh, set
`REA_MCP_TOOL_CATALOG_PROFILE=hot-cold` in REA's server environment and restart
the connection. REA then advertises a small initial catalog and loads other
named tools through `discover_tools`. Setup preserves this explicit setting.
See the [discovery contract](../reference/mcp-contracts.md#hotcold-catalog-discovery) for
pagination, identity and family-scope semantics.
Analysis tools, including `inspect_analysis_view`, load on demand. For example,
load `inspect_api_contract` and supply its `view` to answer an initial OpenAPI
or GraphQL question directly. When later questions need retained facts, load
`inspect_analysis_view` and use the complete `parent_evidence_id`; source files
need not be reopened. If both tools are needed from the start, request both
names in one `discover_tools` batch to avoid another catalog refresh. The same
pattern applies to selected Playwright trace and endpoint-comparison results.
This defers unused schema delivery and client validator preparation; it does
not reduce parsing work, alter Evidence or shorten requested facts. Clients
without catalog refresh should keep the default `full` profile.
Codex CLI 0.165.0 requires its experimental `mcp_tool_list_changed` feature for
this workflow. Merge these entries into the existing `~/.codex/config.toml`
without replacing other feature or REA settings:
```toml
[features]
mcp_tool_list_changed = true
[mcp_servers.rea.env]
REA_MCP_TOOL_CATALOG_PROFILE = "hot-cold"
```
The server setting alone does not enable client refresh. If a successful
discovery call is followed by a missing tool, check the client's refresh feature
and tool allow/deny lists. Clients without refresh should use the default
`REA_MCP_TOOL_CATALOG_PROFILE=full`; reconnect after changing it. The
[FastMCP HTTP trial](#fastmcp-and-unsloth-sse-catalog-limits) below covers
explicit refreshes; other clients' automatic refresh remains unverified.
## FastMCP and Unsloth SSE catalog limits
If an HTTP client reports `Server-sent event exceeded the 1048576 byte limit`
during `tools/list`, merge these settings into the **REA server process's**
environment, then reconnect:
```text
REA_MCP_INPUT_SCHEMA_PROFILE=compact
REA_MCP_OUTPUT_SCHEMA_PROFILE=compact
REA_MCP_TOOL_CATALOG_PROFILE=full
```
This advertises all tools without requiring catalog refresh. Compact output
schemas widen client-side validation; REA still validates complete results
against the canonical contracts. See [output schemas](../reference/mcp-contracts.md#output-schemas).
On main `3e0599a94`, a real FastMCP 4.1.6 / Python MCP 2.3.0 HTTP client with
the default 1 MiB SSE event limit rejected the 4,108,975-byte full catalog.
The settings above delivered all 240 tools in 611,305 bytes and a
`binary_session` call succeeded. These are measured SSE response sizes for
that catalog, not a size guarantee for future catalogs or other profiles.
For a client that refreshes tools, combine `hot-cold` with the compact schema
settings to reduce initial context further: the same trial delivered six
initial tools in 33,903 bytes, then explicitly refreshed through 34 loaded
tools. Hot/cold alone later exceeded the SSE limit during refresh. Follow
the [refresh requirements](#hotcold-catalogs).
The Unsloth report [#9093](https://github.com/morluto/rea/issues/9093) fails at
catalog delivery; the Unsloth application itself was not exercised in this
trial. Schema compaction does not bound analysis results. Continue to use
summary/detail selection and tool-specific paging for larger results; client
receive limits and full Evidence access still apply.
## Fiddler Everywhere alongside REA
Connect Fiddler Everywhere's MCP server separately for HTTP traffic debugging,
preserving your REA client entry. See the
[setup instructions and public skills](../guides/web/web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
[setup instructions and public skills](web-network-captures.md#fiddler-everywhere-mcp-and-public-skills).
## VS Code
@@ -219,7 +119,7 @@ tool inventory on Linux using a loopback model fixture. After setup, add
settings, then restart the client. Compact advertisements can reduce nested
schema constraints; REA still validates complete canonical inputs. Full-profile
`gpt-5.4` metadata is also verified locally. Live model API acceptance and
Windows remain unverified; see [native client verification](../development/testing.md).
Windows remain unverified; see [native client verification](testing.md).
## Claude Desktop
@@ -230,7 +130,7 @@ in the file the installed app reads:
- Without Claude Desktop's MSIX package storage, setup uses
`%APPDATA%\Claude\claude_desktop_config.json`.
- The legacy MSIX storage layout (package family `Claude_pzs8sxrjxfjjc`, installed under
- The MSIX build (package family `Claude_pzs8sxrjxfjjc`, installed under
`C:\Program Files\WindowsApps\Claude_<version>_x64__pzs8sxrjxfjjc`)
virtualizes AppData writes. Once it has saved its configuration, it reads
the package-private copy
@@ -239,11 +139,6 @@ in the file the installed app reads:
private copy when it exists. Before the package has saved one, they use the
`%APPDATA%` file it still reads; with neither file, setup creates the
private copy.
- If `%LOCALAPPDATA%\Claude-Data\claude_pzs8sxrjxfjjc` exists, setup, doctor
and uninstall report an unresolved location. Migrated MSIX installations
have been observed reading `claude_desktop_config.json` in this directory.
Quit Claude Desktop and verify its active configuration before registering
manually. See [#8968](https://github.com/morluto/rea/issues/8968).
- If `%LOCALAPPDATA%\Packages` holds a `Claude_<publisher ID>` package from
another publisher, an `AnthropicPBC.Claude_<publisher ID>` package, or cannot
be read, REA cannot establish which file Claude
@@ -331,7 +226,7 @@ and Claude Code reports that autocompact is thrashing. To avoid this:
override.
- To remove entire tool families from the upfront catalog, run
`rea setup --client claude_code --tool-families <ids>` with the families the
workflow needs (see [the advertising scope contract](../reference/mcp-contracts.md#tool-family-advertising-scope)).
workflow needs (see [the advertising scope contract](mcp-contracts.md#tool-family-advertising-scope)).
Out-of-scope tools are not advertised and their calls fail with the SDK's
disabled-tool error, so widen the selection when a task needs another
family. Setup writes the scope into the entry's `env` as
@@ -348,7 +243,7 @@ Both keep canonical result validation and complete structured results; `compact`
keeps a widened root shape and `off` advertises nothing. On catalogs large
enough that full input schemas alone approach the frame limit, also keep the
input profile at its `compact` default. See [MCP output schema
profiles](../reference/mcp-contracts.md#output-schemas).
profiles](mcp-contracts.md#output-schemas).
## Kimi Code
@@ -382,7 +277,7 @@ This headroom is a practical default, not a guarantee for every analysis
duration. Setup preserves positive timeout values already in the `rea` entry.
Setup also forwards explicitly selected input/output schema profiles,
`REA_MCP_MAX_RESPONSE_BYTES`, `REA_MCP_MAX_INPUT_ELEMENTS`, the Windows ChangeTime settle settings `REA_WINDOWS_CHANGE_TIME_DRIFT_MS`, `REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS`, and `REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS`, provider paths and supported Ghidra, JADX and Kuna
`REA_MCP_MAX_RESPONSE_BYTES`, `REA_MCP_MAX_INPUT_ELEMENTS`, provider paths and supported Ghidra, JADX and Kuna
settings. It also forwards explicit executable and resource selections for ADB,
apktool, packet capture, GDB/Rizin/objdump, Cutter, WABT, Wakaru, firmware and
signal tools, browser inspection, and pwntools/pwndbg. This includes their
@@ -393,7 +288,7 @@ doctor reports its readiness separately. Explicit selections override discovered
defaults, including when setup runs again with a changed selection. Unrelated
environment variables, process ownership tokens and one-shot target paths are not
copied. For an explicitly configured startup target, see the
[initial-target settings](../reference/mcp-contracts.md#initial-target-at-startup). Existing
[initial-target settings](mcp-contracts.md#initial-target-at-startup). Existing
client environment overrides remain intact unless setup supplies the same key.
You can edit `startupTimeoutMs` and `toolTimeoutMs` there; doctor keeps positive
caller-selected values aligned and reports a non-blocking advisory when a tool
@@ -415,22 +310,6 @@ Kimi Code also discovers personal skills in `~/.agents/skills` and project-local
and verifies the matching skill files through the normal shared-skill flow.
Start a new Kimi Code session to pick up MCP and skill changes.
## LM Studio
For [LM Studio](https://lmstudio.ai/docs/app/mcp), setup writes `mcpServers.rea`
into `mcp.json` in the LM Studio home directory. Setup finds that directory as
LM Studio does: the path stored in `~/.lmstudio-home-pointer`, otherwise
`~/.cache/lm-studio` when it exists, otherwise `~/.lmstudio`. Setup only reads
the pointer file. LM Studio parses `mcp.json` as strict JSON, so setup rejects
a file with comments, trailing commas, or a byte order mark without a backup
or a write. The entry carries `command`, `args`, and `env`, and setup keeps
every other server. Local models often have small context windows, so setup
selects `REA_MCP_INPUT_SCHEMA_PROFILE=compact` unless the stored entry or the
setup environment selects a profile explicitly. Doctor reports an entry with no
explicit profile as stale, as for Kimi Code. LM Studio 0.4.26 on macOS loaded
this registration, and a local model called an REA tool through it. Linux and
Windows remain unverified.
## Pi
For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
@@ -504,7 +383,7 @@ standalone CLI check.
A model API's `400 INVALID_ARGUMENT` alone does not identify a rejected schema,
tool-count limit or skill defect. Preserve the response's parameter path and
diagnostic, the actual running REA version, Gemini CLI version and selected
model. For schema rejection, review the [input presentation profiles](../reference/mcp-contracts.md#compact-input-schema-profile)
model. For schema rejection, review the [input presentation profiles](mcp-contracts.md#compact-input-schema-profile)
and reconnect after changing REA's registration environment. Compact is the
current default; an explicitly stored `full` choice is preserved. The generic
error in [#4620](https://github.com/morluto/rea/issues/4620) does not establish
@@ -585,7 +464,7 @@ Responses-compatible Kimi gateway as “infinite recursion”; expanding those
references succeeded. A separate full-profile input failed the provider's
schema-size limit. These are distinct reported failures, not proof that the
canonical schema is recursive. Current REA inputs are already self-contained;
see [schema advertisement](../reference/mcp-contracts.md#schema-advertisement).
see [schema advertisement](mcp-contracts.md#schema-advertisement).
For this failure, inspect the active MCP launch command and refresh the REA
installation that command actually runs. A configured package selector or the
@@ -711,18 +590,6 @@ The user-level MCP paths follow the editor's VS Code layout. Trae's official
documentation describes the raw JSON editor without specifying those paths;
loading this registration in Trae remains unverified.
## CodeBuddy
For CodeBuddy, setup writes `mcpServers.rea` into the global
`~/.codebuddy/mcp.json` (a standard `mcpServers` JSON file at the user home
root on every platform). Entries carry `command`, `args`, and `env`; stdio is
the default transport. CodeBuddy discovers personal skills from
`~/.codebuddy/skills`, so setup installs its skill copy there.
The global MCP path is a plain JSON file; setup preserves every other server
and top-level key when it upserts `rea`. Loading this registration, the
bundled skill, and real REA tool calls through CodeBuddy are verified.
## WorkBuddy
For [WorkBuddy](https://www.workbuddy.cn/docs/workbuddy/From-Beginner-to-Expert-Guide/Function-Description/MCP-Guide), setup writes `mcpServers.rea` into the
@@ -765,20 +632,21 @@ writing a registration ZCode cannot load.
## FreeBuff
For FreeBuff, `rea setup --client freebuff` writes `mcpServers.rea` into the
user-scope `~/.agents/mcp.json` shared-agent file. Entries carry `command`,
`args`, and `env`; stdio is the default transport. The personal skill location
is `~/.agents/skills`, so setup installs its skill copy there. Because the same
For FreeBuff, the proposed integration writes `mcpServers.rea` into the user-scope
`~/.agents/mcp.json` shared-agent file. Entries carry `command`, `args`, and
`env`; stdio is the default transport. The proposed personal skill location is
`~/.agents/skills`, so setup installs its skill copy there. Because the same
file doubles as ZCode's user-scope fallback (see above), a `freebuff`
registration is also in the file ZCode uses until ZCode defines its own native
server. Setup and doctor detect FreeBuff from its own profile directory
(`~/.config/freebuff-desktop`), not from the shared `.agents` folder that REA's
skill installs create for every client.
registration is also in the file ZCode uses until ZCode defines its own native server.
Setup and doctor detect FreeBuff from its own profile directory
(`~/.config/freebuff-desktop`), not from the shared `.agents` folder that
REA's skill installs create for every client.
FreeBuff MCP loading and skill discovery have not been verified with an
installed native FreeBuff client/version; filesystem tests cover REA's editing,
discovery, and ownership behavior only. Automatic applicability requires the
FreeBuff profile directory even when the shared MCP file exists.
The configuration write and personal skill locations are proposed integration
boundaries. FreeBuff MCP loading and skill discovery have not been verified
with an installed native FreeBuff client/version; filesystem tests cover REA's
editing, discovery, and ownership behavior only. Automatic applicability
requires the FreeBuff profile directory even when the shared MCP file exists.
When uninstall cannot establish FreeBuff or another shared-storage reader from
product discovery, a malformed or unreadable shared MCP file is retained
unchanged and does not stop other REA removals, including `--purge-data`. When
@@ -786,9 +654,8 @@ FreeBuff or ZCode product applicability is established or unresolved, the same
malformed shared file stops uninstall before removals. See
[Diagnose, update, and remove](installation.md#diagnose-update-and-remove).
FreeBuff setup support requires `[email protected]` or later (including the
current npm `latest` release). Older versions reject
`rea setup --client freebuff` as an unsupported client. See the
FreeBuff setup support requires `[email protected]` or later. Older versions
reject `rea setup --client freebuff` as an unsupported client. See the
[installation guide](installation.md#supported-agents) for client availability.
## DeepSeek Harness
@@ -834,110 +701,13 @@ in `~/.agents/skills` (`$DSH_AGENTS_HOME/skills` when configured); Harness
discovers that directory, and its native `skill` tool can load
`reverse-engineer-anything`.
REA's optional [native client verification](../development/testing.md#end-to-end-integration-and-golden-evidence)
REA's optional [native client verification](testing.md#end-to-end-integration-and-golden-evidence)
checks headless chat, skill loading, all tool input schemas and actual analysis
using the installed Harness and a local model fixture. This establishes the
Harness integration, with live DeepSeek API compatibility remaining unverified.
See the [Harness MCP guide](https://github.com/deepseek-ai/deepseek-harness/blob/639ed015397290b3745d163aafe02ffee4aa3f84/packages/mcp/mcp-client/README.md)
for its native configuration contract.
<a id="llamacpp"></a>
## llama.cpp
Use REA from llama.cpp's local Web UI through `llama-server`'s native stdio
MCP support. REA supplies the analysis tools; llama.cpp runs your chosen model.
It is not a replacement for a native analysis provider such as Ghidra or Hopper.
Install REA in a persistent location so the configuration below can use a stable
entrypoint. For a project-local installation, run this in your chosen directory:
```bash
npm install --save-exact rea-agents
```
Or install it globally:
```bash
npm install --global rea-agents
```
See [package runners and global installation](installation.md#package-runners-and-global-installation)
for runtime requirements and installation options. Running `npx rea-agents setup`
alone does not create either installation; do not use its temporary package-cache path.
Use a llama.cpp build whose `llama-server --help` lists
`--mcp-servers-config`; older builds without that option cannot use this route.
The configuration below uses an already installed REA package, so tool discovery
does not depend on downloading an npm package during server startup.
Save a JSON file, for example `rea-mcp.json`:
```json
{
"mcpServers": {
"rea": {
"command": "/absolute/path/to/node",
"args": ["/absolute/path/to/rea-agents/scripts/rea.mjs", "mcp"],
"timeout_ms": 450000
}
}
}
```
Replace both paths with your installation's actual paths. For a project-local npm
installation, run `npm root` in that project; the script is under
`$(npm root)/rea-agents/scripts/rea.mjs`. For a global installation, use
`$(npm root -g)/rea-agents/scripts/rea.mjs` instead. Write the resulting absolute
paths as JSON strings; llama-server does not expand shell expressions there.
On Windows, use forward slashes, such as `"C:/Program Files/nodejs/node.exe"`,
or double each backslash, as in `"C:\\Program Files\\nodejs\\node.exe"`.
Apply this to both `command` and the script path in `args`. Single backslashes
can make the JSON invalid or change the path through escape sequences.
`node -p "JSON.stringify(process.execPath)"` prints your current Node.js executable
as a quoted, escaped JSON string that can be used as the `command` value.
Use the version supported by REA. Preserve any other
servers if you are adding REA to an existing configuration. `timeout_ms` is
llama.cpp's per-tool-call timeout, including cold engine startup. The example
allows REA's default 330-second Ghidra startup budget plus 120 seconds for the
operation. Increase it if you configure a longer provider allowance or need more
operation time; see [provider deadlines](../reference/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
Start the server with your own tool-capable GGUF model:
```bash
llama-server -m /absolute/path/to/model.gguf \
--host 127.0.0.1 --port 8080 --parallel 1 --ctx-size 16384 \
--mcp-servers-config /absolute/path/to/rea-mcp.json
```
Open `http://127.0.0.1:8080`. REA tools appear with a `rea_` prefix, such as
`rea_analyze_javascript_application`. Select only the tools needed for the
conversation in the Web UI's tool settings: discovery of REA's complete catalog
does not mean that its schemas and results fit the selected model's context.
Choose a context size your model and machine support. For a first check, enable
`rea_analyze_javascript_application` and ask it to inspect a small JavaScript
directory you own. This task needs no native analysis engine. Inspect the
returned Evidence and limitations rather than relying only on the model's summary.
If the model prints a proposed call as ordinary text, REA has not been called.
Check the selected model and its chat template against llama.cpp's function-calling
guidance. A working MCP connection does not establish reliable tool selection by
every model.
`--tools all` is not needed; it enables llama.cpp's separate built-in tools.
`--ui-mcp-proxy` is also not needed for this server-side stdio connection.
REA's setup wizard does not register llama.cpp automatically; pass the JSON
file to llama-server explicitly and restart it after changes.
The MCP child runs with llama-server's permissions. Loopback binding does not
sandbox it or make every REA operation read-only. Enable only trusted commands,
keep the server local, and select targets and operations you are authorized to use.
See llama.cpp's [MCP server configuration](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md#mcp-servers)
and [function-calling guidance](https://github.com/ggml-org/llama.cpp/blob/master/docs/function-calling.md).
The [verification record](../development/client-verification.md#llamacpp) distinguishes connection
checks from model behavior and lists the tested versions.
## Other client settings
For OpenCode, setup writes the V1 `mcp.rea` entry, which OpenCode V1 and V2
@@ -19,7 +19,7 @@ initialization, discovery and target-free request flow; it does not verify a
client's configured launcher or its inherited environment. Test the generated
registration through the actual client, including a reduced `PATH` when checking
desktop or service startup. Record the client version and launch environment
separately from terminal doctor results. See [launcher troubleshooting](../getting-started/client-configuration.md#launchers-and-path).
separately from terminal doctor results. See [launcher troubleshooting](client-configuration.md#launchers-and-path).
## Published Pi code-mode renderer limit
@@ -33,7 +33,7 @@ shared form would leave later fields unknown instead. The existing
its character limit by advertising reduced nested validation structure, while
REA continues to validate calls against the complete canonical input schema.
Compact is the default for the server's advertised inputs; an explicit `full`
choice is preserved. See the [compact input schema profile](../reference/mcp-contracts.md#compact-input-schema-profile)
choice is preserved. See the [compact input schema profile](mcp-contracts.md#compact-input-schema-profile)
for details of its reduced forms.
Custom renderer hosts can raise `inputMaxChars` through `renderToolSignature` or
@@ -43,30 +43,6 @@ does not establish broader Pi compatibility. See the pinned upstream
[`declarations.ts`](https://github.com/earendil-works/pi/blob/b45e0ebcc5ae2fc42a6f2f23104d10e656e6bedb/packages/codemode/src/declarations.ts)
and [renderer usage notes](https://github.com/earendil-works/pi/blob/b45e0ebcc5ae2fc42a6f2f23104d10e656e6bedb/packages/codemode/README.md#declarations-for-the-model).
<a id="llamacpp"></a>
## llama.cpp
The [manual stdio connection](../getting-started/client-configuration.md#llamacpp) was checked on
macOS arm64 with Node.js 24.18.0, llama-server b11715 (`9d1878a43`), and the
published `[email protected]` package. `GET /tools` exposed 207 REA tools without
enabling llama.cpp's built-in tools. With the source checkout at `9e372c6`, all
207 forwarded names, descriptions and input schemas matched a direct MCP client.
Calls through `POST /tools` returned session metadata and JavaScript analysis
Evidence for a fixture at a Unicode path. A missing target returned REA's input
error, and a subsequent session call still succeeded. The owned llama-server
process exited successfully after the checks. No native analysis engine or
Windows host was exercised.
A local Qwen3-1.7B Q8_0 run selected the JavaScript analysis tool and received its
Evidence, but its follow-up answer reached the output-token limit. Other small
Qwen3 attempts returned proposed calls as ordinary JSON text rather than tool
calls. These runs do not establish dependable model-driven investigations.
A request with the full catalog and task history exceeded a 16,384-token context;
select tools for the task instead of assuming that discovery implies model fit.
The checks exercised HTTP and stdio, not browser interaction with the Web UI.
## End-to-end, integration and golden evidence
The optional `verify:qwen-client`, `verify:pi-client`, and `verify:hermes-client`
@@ -486,7 +462,7 @@ existing static JavaScript analysis through CLI and stdio MCP, including
manifest readback, competing query variants, and resolved relative imports.
It uses an installed browser and accepts an optional installed REA entrypoint.
The complete `verify:browser` lane also exercises passive script export through
both public adapters. See [website script export](../guides/web/website-script-export.md).
both public adapters. See [website script export](website-script-export.md).
`verify:browser:modules` compares CLI and stdio MCP traces against an independent
real Chromium module-loading fixture: import-map scopes, package prefixes,
@@ -494,7 +470,7 @@ null/backtracking rejection, query/fragment identities, repeated module instance
lazy and computed unknowns, exact source/map identities and no implicit refetch.
Set `REA_BROWSER_EXECUTABLE`; the optional script argument selects an installed
package entrypoint. The complete `verify:browser` lane and existing conditional
Chrome CI include this verifier. See [module relationships](../guides/web/website-module-trace.md).
Chrome CI include this verifier. See [module relationships](website-module-trace.md).
## Agent evaluation and conformance records
-26
View File
@@ -1,26 +0,0 @@
# Deno compile executables
`inspect-artifact` inventories a `deno compile` executable. `extract-artifact` writes the embedded virtual files through the existing safe output tree. Neither operation runs the executable or the Deno runtime.
```sh
rea inspect-artifact ./app --json
rea extract-artifact ./app --json
```
In MCP, open the executable with `open_binary`, then call `inspect_artifact`. Use `detail: "summary"` for a large virtual filesystem and `inspect_analysis_view` to page retained Evidence.
The recognized carriers are a Mach-O `__SUI` section named `d3n0l4nd`, an Intel Mach-O `<~sui-data~>` trailer, an ELF `PT_NOTE` or `SHT_NOTE` owned by `SUI` with type `0x53554901` and descriptor name `d3n0l4nd`, and a PE `RT_RCDATA` resource named `D3N0L4ND`. These are structural layouts. The metadata JSON has no Deno version field, so the compiler version stays unknown unless another record establishes it. Other layouts, including older eszip trailers, are unsupported.
Root metadata reports the carrier, entry point, argv, V8 flags, permission options, unstable configuration, application name, self-extracting marker, virtual-filesystem case sensitivity, and the npm snapshot byte length. The npm snapshot is not decoded. Each virtual file is one occurrence with its logical path, size, and SHA-256. Transpiled output, CJS export analysis, and source maps are separate occurrences beside that file. Each has the same `index` as the file and its own `role`. A region normally has the logical path of the file plus a `.deno-*` suffix. When that path belongs to another stored file, the region moves to an indexed path such as `deno-vfs/0.vfs-transpiled`, so read the reported `logical_path` instead of building it from the file name. They are not linked by `parent_occurrence_id`, which follows directories and expanded containers. Symlink targets stay recorded path parts and are never followed.
`Metadata.env_vars_from_env_file` holds values from a compile-time `--env-file`, and `ca_data` holds certificate bytes. Inventory retains the variable names and values, certificate bytes, counts, and byte lengths in local Evidence, so sharing that Evidence shares them. Extraction does not write them as files.
A virtual filesystem that cannot be listed reports `vfs_listing` as `unsupported` and leaves inventory and extraction coverage unknown, because its files are not examined.
Remote modules, redirects, and specifiers contribute counts and logical paths when their stored form is unambiguous. When it is not, they stay present with a byte length and an `unsupported` member listing. A module whose specifier is missing uses an indexed `deno-remote/<index>` path, which still exposes its stored bytes, and modules with resolved specifiers keep their paths. Colons, backslashes, and control characters in a specifier are percent-encoded in the logical path, so `http://localhost:8000/mod.ts` becomes `deno-remote/localhost%3A8000/mod.ts`. Percent escapes in a logical path use uppercase hex, and a specifier that reaches a path already taken uses an indexed path. `source_path` keeps the original specifier. A remote specifier is indexed the same way when a path component exceeds 255 UTF-8 bytes or the logical path exceeds 512 UTF-8 bytes, counting the `.deno-*` suffix of its stored regions. Unsafe, deep, or over-long virtual names use `deno-vfs/<index>` under the same rule and retain the original spelling when it fits the name allowance. A descendant does not present its own name as a complete source path when an ancestor spelling was dropped.
A length that overruns the section, or a closing marker that does not match, is malformed. An opening marker that is absent is unsupported. A confirmed section that cannot be parsed fails closed and does not fall through to a plain native inventory. An Intel Mach-O sentinel inside a confirmed trailer payload is data of that payload, so an Intel Deno executable stored as a virtual file does not replace the app that stores it. Trailers side by side with different bytes fail closed. A file is confirmed only by its carrier name, so an unrelated native file with malformed tables keeps its native inventory. That inventory does not establish that the file holds no Deno section.
See [limits and recovery](design/limits-and-recovery.md) for the named allowances. Synthetic tests cover the carriers and malformed declarations. Run `npm run verify:deno:standalone` for a local Deno compile of the host and, when the compiler can produce them, Linux, Windows, and Intel macOS targets. Deno is a fixture producer only and is not installed by REA. The produced executable is not run.
Layout reference: Deno `cli/lib/standalone/binary.rs` and `cli/lib/standalone/virtual_fs.rs` at 2.9.8, with libsui 0.16.4 section placement.
+27 -178
View File
@@ -3,7 +3,7 @@
This guide explains the runtime settings design for maintainers and developers:
who owns each value, when an update takes effect, how providers enforce it, and
how to extend the system. The public tool contract is in
[MCP contracts](../reference/mcp-contracts.md#editable-analysis-settings); recorded runtime
[MCP contracts](../mcp-contracts.md#editable-analysis-settings); recorded runtime
checks and their limitations are in the
[verification report](../verification/runtime-analysis-settings.md).
@@ -121,22 +121,9 @@ launch environment affects the next REA process.
is the byte-budget shape: a decimal safe integer with a 10 MiB floor
(`MINIMUM_RESPONSE_BYTES`), the same shared schema, and no silent fallback. A
change needs a REA restart and a matching client receive buffer. It does not
raise REA's reserved string limit or Node's string ceiling. Like the regex
deadline, it is in `SELECTED_REGISTRATION_SETTINGS`, so setup forwards an
explicit selection into the client registration.
Windows ChangeTime admission waits out a fresh timestamp before recording a
baseline. `REA_WINDOWS_CHANGE_TIME_DRIFT_MS` (default 2000 ms, floor 1 ms,
maximum 60000 ms), `REA_WINDOWS_CHANGE_TIME_SETTLE_POLL_MS` (default 25 ms,
floor 1 ms, maximum 1000 ms), and `REA_WINDOWS_CHANGE_TIME_SETTLE_MAX_POLLS`
(default 40, floor 1, maximum 1000) are launch settings. Unset keeps those
defaults. An invalid value is a configuration error: it names the setting, the
rejected value, source `setting`, the default, the floor, and the maximum, and
says to set a positive decimal integer in the environment that starts REA and
then restart REA. There is no runtime override. Setup forwards an explicit
selection. When the poll budget runs out, admission keeps the last observation
instead of failing the readable file. Raising a setting only changes how long
that wait can last. It does not change Windows ChangeTime itself.
raise REA's reserved string limit or Node's string ceiling. The setup
allowlist does not copy this setting. A new headroom setting should, following
the regex deadline.
A per-call budget can have a runtime default without having an environment
setting. JavaScript `max_heap_mb`
@@ -172,20 +159,17 @@ checks all happen before publishing the replacement snapshot; a rejected batch
cannot partially apply. Clients coordinating writes can supply
`expected_revision`; without it, valid updates apply to the current state.
| Runtime setting | Application point |
| --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
| `asar.max_listing_bytes` | Next expanded ASAR listing, independently of packed header bytes |
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
| `ghidra.decompile_max_payload_mib` | Next native decompile; `null` preserves the selected provider's payload default |
| `ghidra.value_flow_max_operations`, `ghidra.value_flow_max_inputs_per_operation`, `ghidra.value_flow_max_edges` | Next function value-flow projection on the existing Program |
| `jadx.operation_timeout_ms` | Next JADX operation |
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
| `wasm.command_timeout_ms` | Next WABT operation |
| `native.matching_representation_bytes` | Next `match_native_functions` request |
| Runtime setting | Application point |
| ------------------------------------------------------------- | ---------------------------------------------------------------------- |
| `asar.max_header_bytes` | Next archive operation, including full JavaScript application analysis |
| `ghidra.startup_timeout_ms`, `hopper.startup_timeout_ms` | Next provider startup attempt |
| `ghidra.decompile_timeout_seconds` | Next operation on the existing Ghidra Program; `null` means unbounded |
| `jadx.operation_timeout_ms` | Next JADX operation |
| `regex.matching_deadline_ms`, `regex.worker_heap_request_mib` | Next search / new regex worker |
| `capture.decoder_old_generation_mib` | Next owned capture decoder |
| `javascript.analysis_timeout_ms`, `javascript.max_heap_mb` | Next source analysis / new JavaScript worker |
| `wasm.command_timeout_ms` | Next WABT operation |
| `native.matching_representation_bytes` | Next `match_native_functions` request |
### Precedence and lifetime
@@ -205,48 +189,6 @@ for an unknown or omitted value elsewhere.
A write does not extend active work, relaunch a provider, clear retained
cleanup, or invalidate successful Evidence.
Expanded ASAR listing admission defaults to 64 MiB and has its own launch seed,
`REA_ASAR_MAX_LISTING_BYTES`. Raising `asar.max_header_bytes` alone cannot admit
expanded paths and entries that exceed this separate budget. The listing bound
uses exact safe-integer byte accounting; the host heap and transport constraints
remain independent. A lower runtime value is enforced on the next listing even
when the archive header was already examined.
Ghidra value-flow controls default to 3,000 operations, 64 inputs per operation,
and 12,000 edges. They accept positive Java integer counts and use the respective
`REA_GHIDRA_VALUE_FLOW_MAX_OPERATIONS`,
`REA_GHIDRA_VALUE_FLOW_MAX_INPUTS_PER_OPERATION`, and
`REA_GHIDRA_VALUE_FLOW_MAX_EDGES` launch seeds. Truncation names the exhausted
control. Completed Evidence remains accessible, but a new function dossier is
observed with the accepted controls instead of replaying an earlier projection.
The native decompiler payload control is separate from those projections and
from decompile time. Its launch seed is
`REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB`. Accepted explicit values are 1..2047 MiB:
the supported Ghidra implementation shifts that value into a signed integer byte
capacity. Omission or runtime `null` keeps the provider's own default; REA does
not guess that default from a different provider version. Changing payload
options takes effect before the next native decompile.
### Per-call resource policies and selection
An agent can also use controls declared directly by an operation. These do not
require a runtime setter or restart. Keep omitted fields omitted in Evidence;
report the resolved policy separately where needed. See the owning guides for
[Apktool string pages](../guides/android/apktool-resource-analysis.md),
[source maps](../guides/web/web-source-location.md),
[historical captures](../guides/web/web-network-captures.md),
[JavaScript precision](../guides/javascript/javascript-application-graph.md),
[managed inspection](../guides/managed/managed-code-analysis.md), and
[Frida capture](../guides/runtime/frida-instrumentation.md).
The same accepted policy must reach acquisition, parsing, worker requests,
serialization, cached-result admission and generated consumers where those
stages exist. Desktop state views and emitted AgentEnv Python share one policy;
delivery pagination never substitutes for producer selection. Capture limits
must be selected before an experiment: another experiment cannot recover bytes
that were dropped from a previous live observation.
Overrides belong to one REA runtime, survive target close/open, and disappear
when REA exits. Connections sharing that runtime share its overrides. No files,
ambient environment, registrations or upstream software are edited. Overrides
@@ -444,7 +386,7 @@ application points, and resulting behavior. Check that adapters deliver the
canonical recovery guidance, with independent assertions for its underlying
facts. Avoid freezing prose or deriving all expected values from the same
registry used by the implementation; either can hide the wrong failure.
See [test design](../development/testing.md#test-design-and-pruning) for the assertion policy.
See [test design](../testing.md#test-design-and-pruning) for the assertion policy.
Runtime checks cover atomic validation, stale writes, reset, runtime isolation,
accepted-operation snapshots, and explicit argument precedence. Public MCP
@@ -464,107 +406,14 @@ editable setting or active worker telemetry.
## Bun standalone module graphs
`inspect_artifact` and `extract_artifact` accept per-call `artifact_limits`.
The `bun` group defaults to 100,000 modules, independent 256 MiB payload,
retained-representation and physical metadata-read allowances, 256 path parts
and 250,000 logical path nodes. Recognition separately admits a 1 MiB ELF
section table and 16 MiB Mach-O load commands. Exhausted recognition remains
unknown rather than proving the carrier has no Bun graph. Inventory preserves
native-root facts or another confirmed pack and reports the exact unexamined
alternative inline; it does not assert Bun presence. ELF section names
are examined with exact bounded reads, including names beyond the former
prefix window.
For example, keep the same complete carrier and retry a reported recognition
limitation with `{"artifact_limits":{"bun":{"recognition_table_bytes":2097152}}}`.
The CLI spelling is `--artifact-limits '{"bun":{"recognition_table_bytes":2097152}}'`.
No REA restart is needed. A smaller later selection is re-admitted; previously
parsed facts cannot bypass it. The 256 KiB / 1,024-entry exact-range cache
avoids full-page amplification and remains an internal optimization, not an agent
resource setting. Shared logical path prefixes count once before nodes are added.
## Artifact reader admission
The same request provides `dotnet_bundle`, `node_sea` and `godot` groups. Their
controls apply when recognition or inventory encounters that format; supplying
a policy does not assert the format is present. Unsupported keys are rejected.
Each group follows one domain-owned default policy. See [.NET bundles](../dotnet-bundles.md),
[Node SEA](../node-sea.md), [Godot PCK](../godot-pck.md) and
[Bun standalone](../bun-standalone.md) for format semantics.
`artifact_limits.expanded_bytes` selects aggregate plaintext work for ZIP, Bun,
SEA and PCK readers, including repeated readable logical occurrences. Omission
retains each archive's encoded-size-derived default. The .NET bundle group's
own `expanded_bytes` controls its independently enforced bundle expansion.
Failures preserve `details.reader_limit` with the exact parameter, effective
value, units, supported range, required value when known, next-call scope and
`can_increase`. When `can_increase` is true, raise that field and retry the same
complete artifact. At the supported maximum, or when the required representation
exceeds it, the failure is not retryable by increasing that budget; inspect the
declared dimensions and input validity, or explicitly choose a suitable reader.
Summary, pagination and
selected extraction do not bypass producing admission. Intrinsic format,
safe-integer, path validity and runtime representation constraints remain fixed.
Evidence parameters retain the exact selected object, including omission or
`{}`. When an object was selected, inventory and extraction report the resolved
`effective_limits`; summaries preserve those facts. These are accepted policies,
not process-memory telemetry or a claim that every format reader ran. Complete
source identities and graph manifests remain content-addressed independently
of delivery selection. No partial extraction is claimed on admission failure.
## Deno compile sections
`MAX_DENO_METADATA_JSON_BYTES` admits at most 8 MiB of metadata JSON.
`MAX_DENO_VFS_HEADER_JSON_BYTES` admits at most 32 MiB of virtual-filesystem
header JSON. Specifier, redirect, and remote tables each admit at most 100,000
records. Specifier text admits at most 8 MiB, and remote module bodies admit at
most 256 MiB. The virtual filesystem admits at most 100,000 directory, file,
and symlink records and at most 512 nested levels (`MAX_DENO_VFS_DEPTH`). A
name longer than 4,096 UTF-16 code units is indexed and is not retained.
`MAX_DENO_VFS_RETAINED_UNITS` admits at most 32 million UTF-16 code units across
the source path built for every entry, directories included, and retained
symlink parts. A logical path deeper than 256 parts, a name that is not a safe
relative path, a path component longer than 255 UTF-8 bytes
(`MAX_DENO_PATH_COMPONENT_BYTES`, POSIX `NAME_MAX`), or a logical path longer
than 512 UTF-8 bytes (`MAX_DENO_LOGICAL_PATH_BYTES`, half of the Darwin
`PATH_MAX`) uses an indexed path and keeps the original spelling when that
spelling is within the name allowance. Component and path lengths include the
`.deno-*` suffix of the stored regions a member has. A descendant has no source
path when an ancestor spelling was not retained. Cancellation is checked before
the virtual-filesystem walk. The walk does not yield, so it does not observe a
cancellation that arrives while it is running; the retained allowance bounds its
work. The combined logical path tree admits at most 250,000 distinct prefixes.
`MAX_DENO_RETAINED_BYTES` admits at most 256 MiB of retained header and entry
metadata. The shared artifact expansion budget also bounds the sum of referenced
member lengths, with a 256 MiB floor.
Carrier location claims a file only after it confirms the carrier name: a
Mach-O `__SUI` segment, a `SUI` ELF note descriptor named `d3n0l4nd`, a PE
`D3N0L4ND` resource, or an Intel trailer opening with `d3n0l4nd`. Before that,
malformed tables and exhausted search budgets leave the file with native
inventory, and that inventory makes no claim that the file holds no Deno
section. ELF location reads ELF and note headers, not the whole image, and does
not hash note descriptors. An ELF larger than 512 MiB is still located when
those headers fit. Program and section tables are read at their canonical record
size, so the 16-bit header count bounds each read at 65,535 records. A table
that cannot be read, or that uses extended program numbering, is skipped and
another table can still hold the note. At most 256 notes are examined
(`MAX_INSPECT_ELF_NOTE_ENTRIES`). A damaged table or the note ceiling is a
refusal only when a `SUI` note was also confirmed. Intel Mach-O trailer search
reads each scanned byte at most once and examines at most 64 `<~sui-data~>`
sentinels (`MAX_DENO_INTEL_TRAILER_SENTINELS`). The search continues past a
confirmed trailer, because an enclosing trailer can hold it as a stored file, so
reaching that allowance after a confirmation is a refusal. PE location walks the resource directories to the named payload and
does not read the image whole, so a PE of any size is located when the
`D3N0L4ND` payload is within the 512 MiB resource reader ceiling. A larger
payload, or a confirmed resource that cannot be resolved, is a named refusal.
Refusals name the boundary. There is no runtime override; additional host memory
or `NODE_OPTIONS` does not raise these ceilings. Duplicate-section verification
rejects unequal lengths before hashing, stops at the first digest mismatch,
and reads at most the carrier file size in aggregate
payload bytes. Overlapping copies that exceed that allowance are refused.
Environment-file values and CA certificate bytes are retained in local Evidence
with their counts and byte lengths. The npm snapshot length is retained and its
bytes are not decoded.
`MAX_BUN_MODULE_COUNT` admits at most 100,000 module records.
`MAX_BUN_RETAINED_BYTES` admits at most 256 MiB of serialized graph payload and
separately bounds retained entry metadata. Physical metadata read work is also
limited to 256 MiB per preparation, including repeated uncached ranges. A
256 KiB / 1,024-entry exact-range cache avoids full-page amplification. The
combined logical path tree admits at most 250,000 distinct prefixes; this is
checked before adding nodes, with shared prefixes counted once. The shared
artifact expansion budget also bounds the sum of referenced member lengths. Refusals name the boundary.
There is no runtime override; additional host memory or `NODE_OPTIONS` does not
raise these ceilings. Use a smaller producer fixture or a separately reviewed
reader for larger graphs. No partial extraction is claimed on admission failure.
-445
View File
@@ -1,445 +0,0 @@
# Desktop application capture
`rea capture-desktop-application` launches an installed Linux desktop
application, watches it while a person or a driver uses it, and records what
it is, which windows it opened and how the files under the home directory
changed. The result is the starting point for turning the application into an
agent environment: the inventory says how to install and launch it, the windows
say how to recognise it, and the checkpoints hold the state it read and wrote.
The application runs with the current user's permissions, in the current
desktop session, against the user's real files. This is not a sandbox. The
capture folder keeps copies of real files from the home directory, so treat it
as private data and keep it local.
## Host support
Capture runs on Linux with an X11 display: `DISPLAY` must be set, and `xprop`
(from `x11-utils`) must be installed, or `REA_XPROP_COMMAND` must name it.
The window manager must support the EWMH client list (`_NET_CLIENT_LIST`,
listed in `_NET_SUPPORTED`), as xfwm4, Mutter, KWin and Openbox do; without it
the capture is refused, because no window could be seen. A supported list that
is unset, as xfwm4 leaves it while no window is open, means no windows. Applications that run under XWayland are visible;
native Wayland windows are not. Other hosts are refused before anything is
launched.
## Capture an application
Name the application by its desktop entry id, the entry's absolute path, or an
executable:
```sh
rea capture-desktop-application --desktop-entry org.xfce.mousepad --output ./mousepad-capture
rea capture-desktop-application --executable gnucash --argument /home/ada/books.gnucash --output ./gnucash-capture
```
Desktop entries are looked up in `$XDG_DATA_HOME/applications` and then each
`$XDG_DATA_DIRS` directory; an entry installed as a symbolic link, as flatpak
and Nix do, is followed to its file. The entry's `Exec` line is expanded with no files;
`--argument` values are appended in its place, and the application runs in the
entry's `Path` or the home directory. When the line starts with
`env NAME=value`, the program is found on the `PATH` those assignments set,
with an empty or relative entry read from the folder the application runs
in, and their XDG folders name the state directories. Otherwise the program is
looked up on the host's `PATH` in the order the launch searches it, and one
found only through a relative or empty entry is refused, since the exported
computer would not have that entry. An `--executable` runs from
the current directory, as it would from a shell, and a relative path to it is
resolved there.
A nested desktop file uses a dashed ID: `vendor/app.desktop` has ID
`vendor-app`. REA checks path spellings that can encode that ID within each
XDG root before trying the next root. Unrelated directories do not affect
selection. If an applicable path cannot be examined or the bounded search
cannot establish absence, capture reports an incomplete desktop-entry search
rather than selecting a different application from a later root.
Desktop-entry string values retain literal trailing spaces. For example,
`Path=/tmp/workspace ` and `Path=/tmp/workspace\s` name the same directory
ending in a space. Use the escaped spelling to make such a suffix visible.
Before writing anything the command refuses to start if a window of the
application is already open, or if the window list cannot be read. A launch
beside a running instance can hand off to it, and that instance's unsaved
state would be missing from the baseline. It then records the starting state,
launches the application in its own process group with a run token in its
environment, and reads commands from stdin, one per line. The application gets
REA's environment without the `AGENTENV_*` variables an agent environment sets,
such as its control token: an app the exported computer launches never has
them, and an app captured inside a computer must not hold a credential to it.
It also gets `LIBC_FATAL_STDERR_=1`, so glibc writes a fatal error, such as
an abort's reason, to the app's stderr rather than its terminal, and an app
that exits at launch reports it.
| Line | Effect |
| ------------------------------------------- | ----------------------------------------------------------- |
| empty, `checkpoint`, or `checkpoint <name>` | Record the current state as a checkpoint |
| any other text except `stop` | Record a checkpoint named by that text |
| `stop`, or the end of stdin | Record a final checkpoint, close the application and finish |
An unnamed checkpoint is named `checkpoint-<ms>` after its time in the
session. A name that repeats an earlier checkpoint, or is one of the capture's
own names (`baseline`, `stop`, `time-limit`, `exit`), is refused with a
progress line on stderr; no checkpoint is taken and the session continues.
Name the checkpoints a later step reads by what they mark. The
[export](desktop-application-export.md) seeds the bundle from the baseline
unless `--seed-checkpoint` names another, and a
[task](desktop-application-tasks.md) is checked against the last checkpoint
unless `--end-checkpoint` names another. So one capture can build a seed and
then the change: launch the app, create and save the document, send
`checkpoint seed`, make the edit and save it, send `checkpoint end`, then
`stop`.
The session also finishes when the launched process has exited and the
application's processes and windows are all gone in two consecutive samples,
or when `--max-duration-seconds` passes. Before deciding the processes are
gone, the capture looks for any process still carrying the run token, so a
worker whose launcher exited before it was sampled keeps the session open. A
sample whose window list could not be read is not taken as an empty one; the
limitations say how many there were. One queued line is handled between
samples, and reading stdin pauses while 16 lines wait.
Progress lines on stderr are JSON, `{"rea_progress": {"phase":
"desktop_capture", "message": ...}}`. Those a driver waits on also carry an
`event`: `{"kind": "window_open"}` once the window opens; `{"kind":
"checkpoint", "name", "trigger", "at_ms", "entries", "changes",
"retained_files", "retained_bytes", "changed_files", "more_changed_files"}`
once a checkpoint has been taken, the baseline, `stop` and `exit` included,
with the paths walked, the paths changed since the checkpoint before, the
files retained, and up to ten of the files (not folders) that changed,
home-relative: those under a `--state-path` or absolute `--argument` first,
then those the application held open, then those in its state directories,
then the rest, and last caches (a folder whose name ends in `cache`, such as
`~/.cache`) and WebKit or Chromium storage, each by path, with a count of the
rest. An application can rewrite dozens of cache files on a save, and KDE's
holds its service cache open, so the saved document would otherwise fall past
the ten;
and `{"kind": "checkpoint_refused", "name", "reason"}` for a requested name it
refused. A driver that sends `checkpoint <name>` waits for the event naming it
rather than for a fixed time.
When the session ends the capture sends SIGTERM to the launched process, its
process group and every process that carries the run token, waits up to five
seconds for them to exit, and then removes every process that still carries
the token, whether or not it stayed in the process group. If that cleanup
cannot be verified after a second attempt, the capture is still written, and
the command then fails, naming the run token and process group that may
remain. Interrupting the command (Ctrl-C) ends the session the same way
without a further checkpoint; the capture is still written with end reason
`cancelled`, and the command exits with status 130. Interrupted before the
application is launched, the command writes no capture and removes the
content it had retained. If cancellation arrives during publication or final
cleanup, the already-written capture can instead retain its earlier end
reason. Inspect `capture.json` and the error's `details.partial_observation`
for the retained capture Evidence. Cleanup failures also report resources
that could not be released.
If the capture fails for any other reason before `capture.json` is written,
the application is closed the same way and the content already retained is
removed from the capture folder. The error names any process or file that
could not be removed.
A person types these lines in the terminal. An agent can drive the
application with a tool such as `xdotool` and write the lines to a FIFO that
it passes as stdin. Because the end of stdin stops the session, the driver
must keep the FIFO open for writing between checkpoints, for example with
`exec 3>capture.fifo` in a shell, rather than open and close it for each
line.
| Option | Default | Meaning |
| ------------------------ | ------- | ----------------------------------------------------------------------------------------- |
| `--state-path` | none | File or directory whose starting content is retained too (repeatable) |
| `--exclude` | none | Path left out of every checkpoint walk (repeatable); the output folder is always left out |
| `--max-duration-seconds` | 3600 | Stop the session after this long |
| `--max-entries` | 250,000 | Paths recorded per checkpoint walk, its roots included |
| `--max-file-bytes` | 64 MiB | Largest file whose content is retained |
| `--max-content-bytes` | 1 GiB | Total retained content |
| `--package-file` | none | The `.deb` file the application was installed from; its facts are recorded, not the file |
`--state-path`, `--exclude` and `--output` refuse an empty value, which
would otherwise name the current directory.
## The capture folder
The output folder must be new, or empty and owned by the current user; the
command makes it private (mode 0700). It holds:
- `capture.json`: the Evidence record, also printed on stdout.
- `content/<sha256>`: the bytes of each retained file, stored once per
distinct content.
The Evidence operation is `capture_desktop_application` and its predicate type
is `rea.desktop-application-capture`. Its `host` names the platform,
architecture, display and home directory, and the host's name (`hostname`)
and the user's (`user`), which captured state can hold, as in a mail
template's `ada@e81232be457d`; the export removes them from the bundle. A
capture written before they were recorded has neither, and one is left out
when the system cannot say, as for a user with no passwd entry. The rest of
the result has four parts.
**Application.** The desktop entry (id, path, name, `Exec`, icon,
`StartupWMClass`, categories, MIME types), the launch command and working
directory, the executable as found on `PATH` and after resolving links, and the
package that installed it. A Debian package records its name, version,
architecture and the apt sources that list the installed version, with the
origin (`o=`) apt reports for each source in `source_origins`, such as `Ubuntu`
for the Ubuntu archive and its mirrors or `LP-PPA-<name>` for a PPA; an empty
source list means the version came from a local `.deb` or the package lists
are missing, which `package_lists_present` tells apart. apt's lists folder,
and the trusted keys below, are where `apt-config` resolves
`Dir::State::Lists`, `Dir::Etc::TrustedParts` and `Dir::Etc::Trusted` under
`Dir`, so an `APT_CONFIG` file that moves them is followed: on a computer
without lists, an ordinary user can run `apt-get update` with an `APT_CONFIG`
naming writable `Dir::State::Lists` and `Dir::Cache` folders and capture with
the same `APT_CONFIG`. Without `apt-config`, or when it fails, they are
`/var/lib/apt/lists`, `/etc/apt/trusted.gpg.d` and `/etc/apt/trusted.gpg`.
The Debian package is the one `dpkg-query -S` names as the owner of the
desktop entry the capture launched, or else of the executable, since the
entry's package is what installs the application: LibreOffice Writer's entry
belongs to `libreoffice-writer`, while the `/usr/bin/libreoffice` it runs
belongs to `libreoffice-common`, which does not bring Writer with it. When the
two differ, `executable_package` names the executable's package, which the
export installs beside the entry's, since the entry's package need not depend
on it (`vim-common` owns Vim's entry but does not depend on `vim`).
`executable_package_reason` says why the computer could not install it by
name, by the rule for library packages below: neither the Ubuntu 24.04 archive
nor a vendor repository the export declares offers it, or `apt-cache policy`
did not say. When a package owns the entry but none owns its program, such as
a `~/.local/bin/<name>` found first on `PATH`, the entry's package is recorded
without `executable_package`, and `executable_package_reason` says no package
owns the program, so the export refuses rather than install a package that
does not bring it. A program link that dpkg does not own counts as owned when
dpkg owns the file it resolves to, as an alternatives link does. An
`--executable` launch, or an entry no package owns, such as
one in `~/.local/share/applications`, records the executable's package. When
`dpkg-query` does not finish or exits with an error for the executable or the
entry, the package is `unknown` and the limitations say which path, since a
failed lookup is not proof that no package owns it. The sources, vendor
repositories and `--package-file` match below are the recorded package's; the
library walk still reads the executable.
Each source whose origin is not `Ubuntu` is also recorded in
`apt_repositories`. `apt-get indextargets` names the sources file entry apt
read it from and its release, so flat repositories match too, whether their
suite is `./` (Typora's `deb https://downloads.typora.io/linux ./`) or `/`
(pkgs.k8s.io's). The record holds that entry (`sources_file`, `uri`, `suite`,
`components`, none for a flat repository, and `architectures`, the entry's
`arch=` or `Architectures:` restriction, empty when it has none), `release`,
the release's Suite (apt's `a=`) or else its Codename, and the key that signs
it. A user name or password in an entry's URI (`https://user:password@host/…`)
is never recorded: REA removes it from every URI it reads, in the sources file
and in `apt-get indextargets` and `apt-cache policy` output alike, so entries
match whether or not apt prints it, and sets `uri_credentials_removed` on a repository whose URI held
one (a capture written before this was recorded omits the field). The key is
found by verification, not by name: the candidates are the
entry's `signed-by` keyrings, or else every `.gpg` and `.asc` key in apt's trusted
parts folder and its trusted keyring, and each is checked with
`gpgv` against the release's `InRelease` (or `Release` with `Release.gpg`)
beside its index in apt's lists folder. A deb822 entry whose `Signed-By` field
holds the armored key itself, as `add-apt-repository ppa:<owner>/<name>`
writes it on Ubuntu 24.04, has that key as its only candidate, as apt trusts
no other for it: REA reads the field's continuation lines as deb822 writes
them, a `.` line standing for an empty one, from the key block's `BEGIN` line
to its last `END` line, and records the key's path as the entry's sources
file. A key counts when gpgv reports a valid signature by it and no bad,
expired, revoked or unverifiable signature by it, whatever gpgv's exit status:
a release also signed by a key the host lacks makes gpgv fail. Each candidate
is copied, an armored key decoded to a binary keyring, into a private
temporary folder that is removed afterwards; a folder REA cannot remove
fails the capture with the folder as its cleanup resource. The key that
verifies is recorded in `signing_key` with its path,
SHA-256, format (`binary` or `armored`) and primary fingerprint, and its bytes
are retained under `content/`. A key file holding one key is retained as it
is; from one holding several, such as a legacy `/etc/apt/trusted.gpg`, only
the transferable key with the verifying fingerprint is retained, in binary, so
the bundle trusts no other key. When no candidate verifies, `gpgv` is
missing, apt holds no signed release, no sources entry matches, the key the
entry's `Signed-By` holds does not verify the release, or the key is larger
than `--max-file-bytes` or the content budget, `signing_key` is null and
`signing_key_reason` says why. Ubuntu Pro's ESM repositories (origins such as
`UbuntuESMApps`, host `esm.ubuntu.com`) are not checked: apt reaches them only
with the host's Ubuntu Pro credentials, which a computer does not have.
Unless the installed version comes from the Ubuntu 24.04 archive, so for a
vendor repository's package and for a local install that no apt source lists,
`library_packages` lists the packages that own the shared libraries the
executable loads at start; the walk runs beside the repository check, and not
at all for an archive package, whose export does not use it. REA reads
DT_NEEDED from the resolved executable and from each library it finds, and
looks each soname up as ld.so does: in the RPATH of the needing image and of
each image up its chain of loaders to the executable when the needing image
has no RUNPATH, then in its RUNPATH (with `$ORIGIN` expanded and paths
normalized), and then in the `ldconfig -p` cache entries for the executable's
machine. It asks `dpkg-query -S` for each library's package. Libraries in the
executable's own folder belong to the application: they are walked but not
listed, and neither is the application's own package. One batched
`apt-cache policy` then keeps only the packages the Ubuntu 24.04 archive or a
recorded vendor repository with a verified key offers, in any component its
sources entry lists, since the export declares them all; the rest, such as
packages from a PPA or a local `.deb`, are listed in
`uninstallable_library_packages`. The same query checks `executable_package`.
Each recorded repository's `supplies` lists the kept packages, and the
executable's package, that the archive does not offer and its package files
list, so the export can leave out those only a repository it does not declare
supplies, such as one whose key the content budget later dropped. Sonames
found nowhere are listed in
`missing_libraries`, and libraries no package owns in `unpackaged_libraries`.
`library_packages_reason` says why the lists are incomplete: the executable is
not an ELF image (a launcher script, for example), `ldconfig`, `dpkg-query` or
`apt-cache` is missing, `dpkg-query -S` could not tell which package owns a
library (it exits 2 rather than 1, which only says no package owns the paths
it did not name), a library could not be read or exceeds the selected dependency
inventory budget, or the walk reached that budget. A failed owner query does
not list the libraries it covered in `unpackaged_libraries`, since a lookup
that could not finish is not proof that no package owns them. Completed batches
retain their package and unowned-library facts. Paths containing package-query
pattern characters stay unknown unless another candidate identifies an owner.
A sources file, key or library whose handle REA cannot close fails the capture
with the file as its cleanup resource. When the repository check and the
library walk both fail, the capture reports both failures and every folder or
file either could not release.
An application installed from a `.deb` file you downloaded, such as VariCAD
from varicad.com, has no apt source a computer could install it from. Name
that file with `--package-file` so the export can have the computer download
it instead. REA reads the file's control fields with
`dpkg-deb --field FILE Package Version Architecture Depends Pre-Depends`, and
when its Package, Version and Architecture are the installed package's, it
reads the file once more for its SHA-256 and size and records `package_file`:
the file's `name` without its folder, `sha256`, `size`, `package`, `version`,
`architecture`, and the `depends` and `pre_depends` entries as the file
declares them. Otherwise `package_file_reason` says why, for example that
`dpkg-deb` is not installed, that the file holds another version than the
installed one, or that the file changed between the two reads (its device,
inode, size and times are compared), since its fields and its hash could then
describe different files. The capture never copies the file: it is the vendor's, and only
these facts are kept.
Snaps, flatpaks and
AppImages are recognised from their paths, and flatpaks also from a
`flatpak run` launch, whose application id is the first operand after
flatpak's options and their values. Electron applications are marked by
the fuse sentinel in their binary or a `resources/app.asar` beside it. State
directories are the XDG config, data, state and cache folders and the
dot-folder named after the package, the executable, the desktop entry's full
id and its last part, and the entry's name, with whether each existed at the
start. A folder in those places whose name is one of these in another letter
case joins them, such as GIMP's `~/.config/GIMP` for the `gimp` package; only
the whole name is compared, so `~/.config/GIMPshop` does not. A file under a
folder the application creates during the capture beside one of them, with
the same name in another letter case or with a version added (`-3.1` or
`3`), as TreeLine's `~/.treeline-3.1`, is its state too. Dotfiles in the
home named after the application (`.<name>rc`, or
`.<name>` followed by `-`, `.` or `_`, for a name of three or more characters)
that exist at the start join them, since an application may only read such a
file during the capture. So does KDE's settings file in the config folder,
`<name>rc` such as Tellico's `~/.config/tellicorc`, for each such name of three
or more characters, whether or not it exists yet. A `flatpak run` launch is named after its application
id rather than the `flatpak` launcher, and adds `~/.var/app/<id>`.
**Session.** When it started, how long it ran and why it ended; the launched
process's exit status; every process seen in the application's tree; every
client window with its class pair, titles, owner pid and type, marked when it
belongs to the application. Titles and class names decode xprop's quoted
delimiters, newline and tab escapes, and three-digit octal byte escapes;
unsupported escape spellings leave that string unknown rather than inventing
text. The session also records when the first application window appeared; the
most common class pair of the application's windows, for matching its window
later; files the application's processes held open under the walked roots; the last
8 KiB of the launched process's stderr, where a failing app's last words are,
with `stderr_truncated` when earlier output was dropped; and whether cleanup was verified. Each
process records whether the capture started it: the launched process, a
process carrying the capture's run token, or a descendant of one, even when
its environment cannot be read, as for a zombie or a sandboxed renderer.
Processes are told apart by their start time, so a pid reused by another
process is not counted as the application's.
A window belongs to the application when its owner pid is one of the
application's live processes or its class matches the desktop entry's
`StartupWMClass`, the executable's name, the entry's full id or its last part,
or a flatpak's application id. A flatpak's windows report pids from its
sandbox, so they are matched by class only.
Ownership is checked again at every sample, because a window can appear before
the process that owns it is found. A matching window's pid joins the
application's processes, so a launch handed to a process the capture did not
start, such as a D-Bus activated service, is still followed. That process does
not carry the run token, so the capture never signals it and leaves it
running.
**State.** The walked roots (the home directory plus any content root
outside it), the content roots (state directories, `--state-path` values and
absolute `--argument` values, except `/` or a directory that contains the home
directory, which are named in the limitations instead; a selected path that is
not a directory brings the `-wal` and `-journal` paths beside it, so a database
selected on its own is captured with its logs), the state directories
as walked (their physical spelling, which differs from the inventory's when one
is a symlink), the selected paths (each `--state-path` and content-root
`--argument` as named, absolute, beside the physical path walked, so a
symlinked settings folder keeps what its name means), the paths every walk left out (the output folder and
`--exclude` values), and the checkpoints in order: `baseline` before
launch, each requested checkpoint, `stop` or `time-limit`, and `exit` after the
application has gone. Each checkpoint records how many paths its walk saw,
whether it was truncated, how many paths it could not read or directories on
another device it did not enter (`unchecked_paths`), which paths were created, modified, deleted or
changed type since the previous checkpoint, and which files it retained. A
path counts as created or deleted only where the walk listed its nearest
existing directory completely; differences under a directory that could not be
listed, or that a truncated walk did not reach, are counted as unconfirmed
instead. A walk does not enter a directory on another filesystem than its
root, such as an NFS, sshfs or other FUSE mount under the home directory; the
limitations list each such mount point. The
baseline retains every file under the content roots; later checkpoints retain
every changed file. A change records the file's type, size, mode,
modification time and, when known, the SHA-256 of its content before and
after, plus whether it lies in a state directory and whether the application
held it open. A file is retained only if it is still the version the walk
recorded and does not change while it is read. A file larger than the
remaining content budget is not read, unless stored content of the same size
could make it a duplicate. Files that could not be
retained are listed with the reason: too large, over the content budget,
changed since the walk, changed during the read, or unreadable.
The baseline also records the paths and types of non-file entries it observed,
including symlinks, without following them. This is required to distinguish an
unreadable SQLite sidecar from an absent one. Captures made before this field
was added must be made again before state inspection, task drafting or export.
Directories the application held files open in, but whose starting content was
not retained, are listed so a second capture can pass them with
`--state-path`, or `state_paths` on a build computer's `start_build_capture`,
and a limitation names them, spelled from `~/` under the home. A lock file
the application held open, such as Poedit's `~/poedit.lock` or Lucene's
`write.lock`, makes no folder such a directory, since it holds nothing to
retain.
**Limitations.** What the capture could not establish, such as a truncated
walk or a missing window.
## What the capture does not establish
Changes are found by comparing walks, using each path's size, modification
time, inode and mode, so a change undone between two checkpoints is not seen
and a rewrite that keeps all four is missed. A change is not attributed to the
process that made it: other desktop services write under the home directory
too, such as the dconf database, the window manager's settings and GTK's
recently used list. The markers for state directories and open files are
evidence for attribution, not proof.
Clicks and keystrokes are not recorded. Window titles are sampled every half
second, so short-lived titles can be missed.
Dependency inventory headroom is selected for each capture with
`--library-policy '{"libraries":1024,"string_bytes":8192}'`. Unset fields retain
their defaults: `libraries: 512` bounds both the closure walk and distinct
DT_NEEDED offsets in each ELF image; `search_path_entries: 16` bounds distinct
DT_RUNPATH and DT_RPATH offsets per kind; `string_bytes: 4096` includes the NUL
terminator of each decoded string; `table_bytes: 1048576` bounds each program
header or dynamic table. Values must be positive safe integers. A budget refusal
names the field that needs headroom; partial package facts and unresolved
coverage remain visible. The policy applies during this capture's admission,
without restarting REA, and explicit selections are retained in Evidence. Raising
content retention or a worker heap does not raise these inventory budgets.
Choosing a new capture still launches the selected application; changing these
read budgets does not add execution authority or recover an earlier capture's
omitted inventory.
-603
View File
@@ -1,603 +0,0 @@
# Desktop application export
`rea export-desktop-application <capture-folder> <directory>` turns a folder
written by [desktop application capture](desktop-application-capture.md) into an
agentenv bundle holding one computer app environment: the real application,
installed from the distribution archive, its vendor's apt repository or the
`.deb` file it was installed from on the `ubuntu-24.04` computer base, launched
at boot, seeded with the user's state and serving that state through agentenv's
data operations.
```sh
rea export-desktop-application ./mousepad-capture ./mousepad-bundle
agent-env run ./mousepad-bundle --dry-run
```
The export reads the capture and its [state view](desktop-application-state.md);
it does not run the application. `--name` sets the environment name; otherwise
it comes from the desktop entry id or the program: a reverse-DNS id's last
part (`org.xfce.mousepad` gives `mousepad`), or else the whole id without a
trailing version (`glabels-3.0` gives `glabels`), with an `app-` prefix when
that name would not be a portable bundle name (for example all digits). The
output directory must be new or empty. `--task` adds a task an agent wrote for
the capture, once REA has validated it against the capture; see
[desktop application tasks](desktop-application-tasks.md). `--package-url`
says where the computer downloads the package of an application installed from
a `.deb` file; see [Applications from a .deb file](#applications-from-a-deb-file).
`--download-install FILE` says which AppImage or tarball a build computer
unpacked the application from; see
[Applications from an AppImage or tarball](#applications-from-an-appimage-or-tarball).
`--extra-package NAME`, repeatable, installs another Ubuntu 24.04 archive
package beside the application, such as a runtime piece its package does not
install; a [build computer](desktop-build-witness.md) passes its request's
`extra_packages`, or a `.deb` request's `dependencies`, this way. A name that
is not a Debian package name is refused (`extra_package`). Names the bundle
installs anyway, the application's own package and the dependencies it names
for a package installed from its `.deb` file, are not repeated; the rest are
listed in the limitations and the README. A package the captured `.deb` lists
as one alternative, such as `openjdk-17-jre | java-runtime`, is kept when apt
installs the application by name, since apt may satisfy it with another.
`--seed-checkpoint` names the checkpoint whose state seeds the bundle, the
baseline by default, and `--end-checkpoint` the one a task must pass on, the
capture's last by default; see [The seed checkpoint](#the-seed-checkpoint).
`--synthetic-seed` says the seed was made for the task, as a build agent makes
it in a [build computer](desktop-build-witness.md), rather than taken from the
user's own data; see [A seed made for the task](#a-seed-made-for-the-task).
The command returns Evidence with operation `export_desktop_application` and
predicate type `rea.desktop-application-export`, listing the environment name,
the seed and end `checkpoints`, owned paths, seed files, the seeds it
rewrote to remove the capture host's name and user (`scrubbed_files`), the
gzip seeds it compressed again only to clear their headers
(`recompressed_files`), the bundle files that still hold them
(`unscrubbed_files`), and written files, the last four by their path in the
bundle. A failed export, cancellation
included, removes the bundle it had not yet published, so no partial copy of the
user's data remains. If the bundle was published but its output handles could
not be closed, the command reports a cleanup error that carries the export
Evidence; the bundle itself is complete.
## The bundle
| Path | Contents |
| --------------------------------- | -------------------------------------------------------------------------------------------------- |
| `envs/<name>/env.toml` | `type = "computer_app"`, the module and class, `computer = "ubuntu-24.04"`, `driver = "reference"` |
| `envs/<name>/<file>.py` | The `ComputerAppEnvironment` class |
| `envs/<name>/SKILL.md` | What the app is, its window class and where its files live |
| `envs/<name>/seed/` | The application's files at the seed checkpoint, by home-relative path |
| `envs/<name>/apt-keys/` | The signing key of each vendor apt repository the app installs from |
| `envs/<name>/install-package.sh` | With `--package-url`: the run script that installs the downloaded package with dpkg |
| `envs/<name>/install-download.sh` | With `--download-install`: the run script that unpacks the downloaded AppImage or tarball |
| `envs/<name>/applications/` | With a tarball's `--download-install`: the desktop entry REA wrote for its program |
| `tasks/smoke.json` | A task that deploys the app |
| `tasks/<task>.json` | With `--task`: the agent's task, its verifier in `artifacts/` and the eval in `evals/` |
| `README.md` | What was exported, from which capture, and what to keep in mind |
The class file is `<name>_app.py` with hyphens as underscores, for example
`mousepad_app.py`: agentenv imports it under its stem, so a plain `<name>.py`
could shadow a standard library module such as `calendar`. A class name that
would collide with a name the module imports or defines, such as `Path`, takes
an `App` suffix.
The class declares:
- `Packages(apt=[...])` with the package the capture found, then the
`--extra-package` names. A vendor package is named `<package>/<release>`,
its repository's release (the Suite apt reports as `a=`, or else its
Codename), so apt takes it from that repository
even when the Ubuntu archive offers a higher version or epoch, as noble's
transitional `firefox` does over packages.mozilla.org's; it is named alone
when the release names neither, or when the capture was written before
releases were recorded. Its library packages follow, so a package that
leaves out dependencies on the libraries it loads, as Typora's Electron
package does, still starts; library packages the capture found neither the
Ubuntu 24.04 archive nor a declared repository offers are left out, with a
limitation, and so are those only repositories the export leaves out supply
(a repository's `supplies`), such as one whose key the capture could not
retain. An archive package's declared dependencies are trusted as they
are. When the desktop entry's package is not the one that owns the program
it runs, as `vim-common`'s entry runs `vim`'s program, the program's package
(`executable_package`) follows the app's by name, since the entry's package
need not depend on it; a capture that found no archive or declared
repository offering it records why, and the export refuses it with that
reason, as it does when only repositories it leaves out supply it, or when
no package owns the program at all (`executable_package_reason` without
`executable_package`). The package must come from the Ubuntu 24.04 archive
or a mirror of it,
or from a vendor apt repository whose signing key the capture verified and
retained. A source counts as the archive when apt reports its origin as
`Ubuntu` for a noble suite; a capture made before origins were recorded
counts only Canonical's own archive hosts.
- For a vendor repository, `sources=[AptSource(apt=..., suite=...,
components=[...], key="apt-keys/<name>.gpg")]`, with the retained key copied
to that path (`.asc` when it is armored, which agentenv tells by its
content). A PPA added with `add-apt-repository`, whose sources entry holds its
key in `Signed-By`, exports that key as `apt-keys/<name>.asc`. apt takes one
key per URI and suite, so sources that share them, such as one repository
listed for two architectures, are declared once, with their components
merged; one with a different key is left out. A repository
whose key the capture did not verify, an Ubuntu Pro (ESM) repository, one
whose sources entry's URI held a user name or password (which the capture
did not record and the export never writes; the reason names neither them
nor the URI), one not reached over http or https, or one whose `arch=`
restriction leaves out the capture host's architecture is refused with its
reason (a capture
written before restrictions were recorded counts as unrestricted); when
another usable repository lists the same version it is left out instead,
with a limitation. The export also refuses snaps, flatpaks, an AppImage or
tarball not named with `--download-install`, and a
package no recorded repository lists unless it was installed from a `.deb`
file the capture recorded and `--package-url` is given, since the computer
could not install them by name, and a capture written before credentials
were removed whose repository URI still holds them, since it counted that
repository's libraries as installable; capture it again.
- `@computer_app(supports=...)` with both `amd64` and `arm64` for an archive
package, which the archive serves for both, and only the capture host's
architecture for a vendor package, since its repository may serve no other.
A vendor package captured on any other architecture is refused. A package
installed from its `.deb` file supports only that file's architecture.
- A launch at boot with the captured command, ready when a window of the
captured class appears and one of them is listed in the taskbar and is not
typed as a splash: a splash screen or a dialog of the class skips the
taskbar or says it is a splash, and an app whose splash shows first would
otherwise count as ready before its own window opens. Arguments that are the captured home or under it, and
`<option>=<path>` values under it, are respelled under the computer user's
`$HOME`. agentenv starts the app in its own folder, where an open or save
dialog would show the framework's files, so the command first changes to the
captured working directory (`"$HOME"` when that was the home, as for a
desktop entry without `Path`), which a relative argument or program path
such as `./app` also needs, and still starts if that folder is missing. The
export refuses a capture that saw no application window.
- The seed's document, opened at launch, so the app shows it rather than its
start screen: when the seed holds exactly one file the state view gives the
`document` role, and the desktop entry's `Exec` takes files (a `%f`, `%F`,
`%u` or `%U` field code), its path under `$HOME` is appended to the command.
A command that already names a document, absolute or from its working
directory, as one captured with `--argument <file>` does, is kept as it is;
a settings path under the home does not count. A seed with no document launches
as captured. With several, no desktop entry or an `Exec` that takes no file,
the app launches as captured and the README and the result's limitations say
why. A reset
relaunches the app, so it opens the document again.
- For an Electron application (the capture found its fuse sentinel or a
`resources/app.asar`), `--no-sandbox` right after the program unless the
command already has it: Chromium's sandbox needs user namespaces the
computer's container does not grant, and Electron exits without them. The
README and the result's limitations say the sandbox is off.
- `paths`: the state directories holding the application's state files and the
folders holding its documents, nested folders collapsed. A folder beside a
state directory whose name differs from it only in letter case, or that
adds a version, as TreeLine's `~/.treeline-3.1` beside `~/.treeline`, is owned
whole in the files' spelling, so a `~/.config/GIMP` profile GIMP created
during the capture beside the candidate `~/.config/gimp` is owned as
`.config/GIMP`, not as the folder of its first file. The home itself and
the shared `.config`, `.local`, `.local/share`, `.local/state` and `.cache`
folders are never owned whole: a file directly in one is owned as itself.
Files the capture recorded under a symlinked state directory's physical
spelling are mapped back to the innermost such directory's home spelling,
unless that spelling is outside the captured home, as when the home itself is
a symbolic link. When a selected link names an application state directory,
the bundle seeds the application's state path and restores the selected
alias at boot. Selections nested under that alias are seeded under the same
state path. An existing incompatible alias stops setup without replacing it. When two files map to one home path, the export uses the one present
before the capture, else the first by path, and a limitation names the other.
- Data hooks, because declaring any hook switches off agentenv's default data
plane: the class seeds the home once per boot and keeps it as the initial
state with `keep_initial`, `data/add` and `data/reset` call agentenv's `add`
and `reset`, and `data/get` returns the files under `paths` plus a data part
with a typed JSON view of each, read in a worker thread. An application with
no observed state declares no paths and no hooks.
A file under the owned paths whose bytes at the seed the capture did not
retain is not seeded: one in the state view without retained content at that
checkpoint, or one the capture omitted (for example too large or past the
content budget). The result lists them in `unseeded_files` and the README
names them. When the application held files open in folders whose starting
content the capture did not retain, a limitation names those folders, spelled
from `~/` under the home: a file there is seeded only when the capture
retained it, as it does one written during the capture, and the limitation
names the seed files there or says there are none. The desktop Trash,
`~/.local/share/Trash`, is never the application's, so a file the user moved
there is neither owned nor seeded, even when a window title names it; a
limitation lists the files the Trash held at the seed.
Files that changed during the capture outside the owned paths are neither
owned nor seeded, so the app starts without what it saved there: MuseScore
keeps its settings in `~/.config/MuseScore`, which its name rules miss, and
showed its first-run wizard on every boot. A limitation and the README name
those folders, leaving out desktop services' files, each collapsed to its
folder in `~/.config`, `~/.local/share`, `~/.local/state` or `~/.local`, the
hidden folder of the home it lies in, or else its own folder, up to eight and
a count of the rest. Capture again naming them with `--state-path`, or
`state_paths` on a build computer, to own them.
Lock files the app holds while it runs are left out of the seed: a copied one
makes the app find its profile or document in use, and a LibreOffice Calc
bundle whose seed held its profile's `.lock` showed no window at boot. They
are `lockfile`, `lock`, `.lock`, `.parentlock`, `parent.lock`,
`SingletonLock` and Lucene's index lock `write.lock` anywhere, as Poedit's
translation memory holds, LibreOffice's document lock `.~lock.<name>#`, a
`<file>.lck`, `.<file>.lock`, `.<file>.lck` or `<file>.<ext>.lock` beside a
file the capture recorded in the same folder, such as GnuCash's
`household.gnucash.LCK`, Grisbi's empty `.household.gsb.lock`, which made it
refuse every save, and VYM's `plan.vym.lock` (a project's `Gemfile.lock`
beside its `Gemfile` has no extension to its file, so it is seeded), and
`*.lock` or `*.lck` directly in the home, as wxWidgets' single-instance
`~/poedit.lock`, or directly in one of the application's state directories;
a lone `notes.lck` or `.notes.lock` elsewhere is seeded. A lock
never makes a path the application's own, so `~/poedit.lock` is neither owned
nor named among the folders the bundle does not own.
Elsewhere such a file may be a project's own, such as `Cargo.lock` or
`yarn.lock`, which pin dependencies, and is seeded, even under a folder whose
name starts with a dot. A file the state view makes a document, or one named
itself by `--state-path` or an absolute `--argument`, is never left out,
whatever its name. A limitation and the README list them, up to
eight and a count, after "Left out of the seed". They stay under the owned
paths, so `data/get` returns the lock the running app holds, and `data/reset`
restores the seed without them.
A seed file keeps the permission bits it had at the seed when the capture saw
it change, such as a `0600` file an app checks: the class module lists those
other than `0644` in `SEED_MODES`, and the first boot sets them when it copies
the seed. A file the capture never saw change has no recorded mode and is
written `0644`.
Files in app state are retained regardless of a `.log` extension or `log.<n>`
name. They can hold conversation history or database writes as well as
diagnostics. The normal host-identity scrub applies to retained text; filenames
alone do not identify disposable diagnostics. Locks beside these files follow
the lock rules above.
Other seed files the app keeps for its own
running are seeded as captured and named, collapsed to the folder that holds
them: WebKit or Chromium storage (such as `hsts-storage.sqlite`, `storage/`,
`mediakeys/`, `Local Storage/`, `IndexedDB/`, `Cookies`, `Visited Links`,
`Platform Notifications/`, and a whole `QtWebEngine/`
profile), which may hold the app's own data, and caches (a folder whose name ends in `cache`); a stale one
can change how the app starts. A seeded LibreOffice profile's
`registrymodifications.xcu` ships without its `RecoveryList` items, which
LibreOffice writes for the documents it has open and which would start the
bundle into document recovery instead of its window; a limitation and the
README name the file. REA finds the items with its XML parser, as `item`
elements whose `oor:path` is `/org.openoffice.Office.Recovery/RecoveryList`
however the tags are spelled, and cuts exactly those elements, so text in a
comment or CDATA and every other byte stay as captured. A registry that is
not well-formed XML ships unchanged.
The application's dconf keys at the seed, if any, are written once when the
app first starts, also for an application that owns no files, and the class adds
`dconf-cli` to its apt packages for that. The server starts before the
desktop's D-Bus session, so the class does not use `dconf load`: it dumps any
existing user database through a `file-db:` profile, then runs `dconf compile`
on that dump and the captured keys, in that name order so the captured keys
win, and replaces `~/.config/dconf/user`. If that fails, the server prints a
one-line warning and the app starts without the captured keys. `data/reset` does
not restore them. No file of the bundle holds them, so `data/get` does not
return them and no task check reads them; a draft whose only change between
its checkpoints is to the application's dconf keys says so with
`uncheckable_changes`, and one whose dconf database changed but could not be
read says so with `unreadable_changes`. When a dconf database changed by the
seed checkpoint but its version there could not be read, as without
`dconf-cli`, the export seeds no keys from it and a limitation names it with
the read error.
## The seed checkpoint
By default the seed is the state before the capture, its `baseline`, and a
task's end state is the capture's last checkpoint. `--seed-checkpoint` and
`--end-checkpoint` name others, so one capture can make a seed and then a
change: the user creates and saves a document and takes `checkpoint seed`,
then edits and saves it and takes `checkpoint end`. Exported with
`--seed-checkpoint seed --end-checkpoint end`, the bundle seeds the document
as it was saved at `seed`, owned paths, dconf keys and unseeded files
included, and a task must fail on that seed and pass on the state at `end`.
A file's state at a checkpoint is its last version recorded at or before it.
The export refuses a name the capture does not hold exactly once, and an end
checkpoint taken before the seed checkpoint; `inspect-desktop-application-state`
lists the names in `capture.checkpoints`.
## A seed made for the task
By default the seed is the user's real data: the README and the result's
limitations say to keep the bundle local and not publish it. With
`--synthetic-seed` they say instead that the seed was made for the task, and
to read `envs/<name>/seed/` before publishing the bundle; the export cannot
tell the two apart, so the flag is the caller's word. `export_build_environment`
passes it, since the building agent makes the seed in the build computer. The
Evidence records it as `synthetic_seed`.
## Withheld sign-in state
`--withhold-from <checkpoint> --withhold-to <checkpoint>` withholds every file
under the owned paths that changed after the first checkpoint up to the
second, as the capture recorded its changes; `--withhold-path <file>` adds a
home-relative file, such as one written at first launch. The draft
(`describe-desktop-application-task`) takes the same options, leaves those
files out and lists them as `withheld_files`; the export refuses a check that
names one. Each withheld file present at the seed is left out of
`envs/<name>/seed/` and becomes a secret setting:
- the class declares `Settings` with a `SecretStr` field per file, and
`WITHHELD`, each field's home path;
- `env.toml` sets each field to `env:REA_<APP>_<FILE>`, so agentenv refuses
the deploy, naming the variable, until it is set;
- at first boot, before keeping the initial state, the class writes each file
from its setting, the file's bytes in base64, mode `0600`, so `data/reset`
restores it, and `data/get` serves it neither as a part nor in the views;
- the README names each variable, with
`export REA_<APP>_<FILE>="$(base64 -w0 < ~/<file>)"` from the receiver's
own signed-in copy of the app.
A withheld file larger than 48 KiB is refused, since its base64 must fit one
environment variable. `--withheld-copy <folder>` copies the withheld files,
as captured, into a new or empty folder outside the bundle. The result lists
them as `withheld_files`, each with its `setting`, `variable` and `handle`.
`--consent <file>` ships the person's consents, a `consent.json` that
`record_build_consent` records, as the bundle's own `consent.json`.
`--seed-withheld` seeds the withheld files after all, refused unless that
file holds a `seed_sign_in` consent; the result lists them as
`consented_files`, and the README says they are there with consent.
## Host name and user
A seed can hold the capture host's name and the user's: Geany saved its mail
template as `pref_template_mail=ada@e81232be457d`, the user `ada` on a
container named `e81232be457d`, and many files hold the home path
`/home/ada`. The capture records `host.hostname` and `host.user`, and the
export removes them from what the bundle ships:
- In seeded text files (text, JSON, key files and XML in the state view), in
UTF-16 files that start with a byte-order mark (keeping the mark, byte order
and a following U+FEFF content character), and in the dconf keys,
`user@host` becomes `user@localhost`, the
host name standing alone becomes `localhost`, and the captured home becomes
`/home/user` wherever it is followed by `/`, the end of the text, or a
character that cannot continue a file name. The host name counts only as a
whole name: not inside a longer word, before a `-`, or as a label of a
domain such as `e81232be457d.local`, so a host named `ubuntu` leaves
`ports.ubuntu.com/ubuntu-ports` alone. A host named `localhost` is left
alone, but its `user@localhost` still takes the computer's user. The user
name alone is never replaced, since a short one such as `ada` or `user` is
also ordinary text. A rewritten seed is written with its own SHA-256, and
`scrubbed_files` and the README list it.
- JSON and XML keep their syntax. In JSON only string values change, each
read as its decoded value, so `\u0065` escapes cannot hide the host, and is
written back as JSON only when it changed; numbers and `true`, `false` and
`null` stay as they are even for a host named `true`. Member names are never
rewritten, since `{"ubuntu": 1, "localhost": 2}` on a host named `ubuntu`
would get two `localhost` keys and lose a setting; a file whose names hold
the host is listed in `unscrubbed_files` instead. In XML only character
data, attribute values, comments and CDATA change, character references
decoded the same way; element and attribute names and processing
instructions stay. A UTF-16 file is scrubbed by the format its text would
have in UTF-8, so JSON or XML in UTF-16 follows these rules too.
- In the dconf keys only GVariant strings change, each read as its decoded
value and written back in its own quotes only when it changed. Group names,
keys, dictionary keys, bytestrings and everything outside quotes stay, so a
host named `true` leaves `wrap=true` alone. A value whose string does not
parse, or that holds an object path or signature, ships as captured.
- `SEED_HOME` in the class is `/home/user`, the home the seed spells, which
the class respells to the computer's own home when it seeds the home at
boot, as it did the captured home before. The replacements read as the
computer's own: its user is `user`, with its home in `/home/user`, and
`localhost` is the name every host has for itself.
- A gzip-compressed seed, such as a Gnumeric workbook, is decompressed, up to
256 MiB, scrubbed as the format the state view read its document as, and
compressed again whether or not its document changed, so no file name,
comment or time its header held ships: with no file name, comment or extra
field, modification time 0 and the operating system unknown, so the same
document gives the same bytes on every host. It is deflated at the fastest
level, or stored when deflating would make it expand more than the 100 times
a view reads, so a seed that read still reads. It is listed in
`scrubbed_files` when its document changed, and otherwise in
`recompressed_files`, which the README counts rather than lists, so the
example and template files an app copies on first run do not bury the
document scrubbed; one already in that form ships as captured. The class copies it at
boot as it is, so the home it holds stays `/home/user`. One that does not
decompress within the scrub limit ships as captured, its bytes searched.
A document the state view did not read can still be decompressed and
recompressed under the separate 256 MiB scrub limit; its content is searched
even when its format cannot be scrubbed. One whose document needs no edit but a member of
which has an extra field, where BGZF (`.vcf.gz`, `.bam`) and dictzip
(`.dict.dz`) files index their blocks, also ships as captured, its bytes
searched, so its blocks stay where its index says; one whose document
changed is still compressed again as one member, losing them, as the
export's limitation says.
- Binary files, SQLite databases among them, and text files over 256 MiB are
copied as captured. Then every file the bundle ships, the class module with
its dconf keys and the README included, is searched by a looser rule than
the replacement's: the host name and `user@host` anywhere, as in
`e81232be457d.local` or a seed named `e81232be457d-notes.txt`, and the
captured home as a whole path, in its path and its content, JSON and dconf
strings and XML references decoded and the replacements themselves set
aside. Each one found is listed in `unscrubbed_files`, and in the README
unless it is the README, to check before sharing the bundle. Bytes that are not text are
searched as they are, so a binary file that holds the host in another
encoding is not listed.
- A task check whose values name `user@host` or the host name anywhere, by
that rule, is refused, as one naming the captured home is: the bundle does
not hold them. Its field names, kind and id are not values, so a host named
`file` or `kind` refuses no check. A task's checks must fail on the seed as
the bundle ships it, scrubbed, so a check the scrub itself satisfies, such
as a mail setting the user changed from `ada@e81232be457d` to
`user@localhost`, does not count as failing there.
A capture written before the host's name and user were recorded has only its
home removed, with a limitation saying so; capture again to remove the rest.
## Applications from a .deb file
A closed-source application you installed from a `.deb` file you downloaded,
such as VariCAD from varicad.com, is not in any apt repository the computer can
use. Capture it with `--package-file` naming that file (see
[desktop application capture](desktop-application-capture.md)), then pass
`--package-url` to the export with an `http://` or `https://` URL the computer
can download that same file from:
```sh
rea capture-desktop-application --executable varicad --package-file ~/Downloads/varicad2026-en_2.03_amd64.deb --output ./varicad-capture
rea export-desktop-application ./varicad-capture ./varicad-bundle --package-url https://files.example.com/varicad2026-en_2.03_amd64.deb
```
The class then declares, beside `Packages`:
- `downloads=[Download(url=..., sha256=..., to="vendor/<file>", arch=...)]`
with the captured file's SHA-256. agentenv downloads the URL once, when the
bundle is put, and refuses the file unless its SHA-256 matches, so the
computer installs exactly the captured version. The bundle does not contain
the package.
- `Packages(apt=[...])` with the first alternative of each of the file's
Depends and Pre-Depends entries, without version constraints or `:any`
qualifiers, then the library packages the capture found, since a vendor's
binary may load libraries its Depends leaves out.
- `run=[RunScript(script="install-package.sh")]`, a script the export writes
into the environment folder. agentenv runs it as root from the app's folder
once the apt packages are installed and the download is in place, with the
network off: it runs `dpkg -i` on the download and then removes it.
The bundle supports only the file's architecture. Vendor sites often refuse
automated downloads, as varicad.com does, so the URL may need to point at a
copy you keep, such as a private mirror; you must be allowed to use that copy,
and the package's licence, such as a 30-day trial's terms, is yours to check.
The README and the result's limitations say so.
The export refuses a package no recorded repository lists when `--package-url`
is missing, saying to capture with `--package-file` if the capture lacks it. It
also refuses `--package-url` when the capture recorded no package file, giving
the capture's `package_file_reason` when there is one; when the archive or a
declared vendor repository installs the package already; when the URL is not
written as `http://` or `https://` or holds a user name or password; and when
the file is built for an architecture other than `amd64`, `arm64` or `all`.
## Applications from an AppImage or tarball
A [build computer](desktop-build-computer.md) can install an application from
its project's AppImage or tarball, pinned by URL and SHA-256, under
`/opt/rea-apps/<desktop entry>`. Neither holds a Debian package, so the
capture records the package as `unknown`, and the export takes the install
from `--download-install`, a JSON file the build computer writes from its
request:
```json
{
"kind": "appimage",
"desktop_entry": "org.freecad.FreeCAD",
"url": "https://github.com/FreeCAD/FreeCAD/releases/download/1.1.4/FreeCAD_1.1.4-Linux-aarch64-py311.AppImage",
"sha256": "675d1a4295b2183e0e1d816a4b5557dcb63aeced4e60aaa28d6c3686f4c71dc9",
"architecture": "arm64"
}
```
A tarball's adds `program`, its path inside the archive, and optionally
`name`. The class then declares
`downloads=[Download(url=..., sha256=..., to="vendor/app.AppImage", arch=...)]`
(`vendor/app.tar` for a tarball) and `run=[RunScript(script="install-download.sh")]`,
the same script the build computer ran, with no apt package for the app. The
bundle does not contain the application and supports only the download's
architecture; the URL must keep serving those bytes, and the application's
licence is the operator's to check.
The export refuses the file (`download_install`) unless it names an
`appimage` or `tarball` with an `http://` or `https://` URL and a SHA-256,
the capture launched its `desktop_entry`, the captured program lies under its
folder in `/opt/rea-apps`, and the capture ran on its architecture; it
refuses `--package-url` beside it.
## Grading once the app has saved
A task's verifier, `artifacts/<task>-check/verify.py`, grades the app's files
only once they have settled, because an app can save on its own timer after the
last change. Gnote saves a note 4 s after the last key, so a read taken as
soon as the agent stops finds the text on screen but not yet in the file. The
verifier reads the app's `data/get` every 0.5 s. It compares each file's bytes
between reads by their SHA-256, as `data/get` returns them, and grades once
every file has held for 3 s. The 3 s count from when a read returned the
current bytes, not when it began, so a slow read that brings a change late
does not count its own run as quiet. 15 s after the first read began, the
bound, the verifier grades the last completed read: no read starts at or
after the bound, and a read still running then is cut. Each result records
whether the files settled in `settled`, and whether the bound cut a read in
`read_cut`.
The window has a cost and a limit:
- An app whose files do not change during the check, such as one that saved
at once, is graded 3 s after the first read returns.
- A save that lands more than 3 s after the first read returns is still
missed. An app that saves 4 s after the last key is caught only when the
agent stops at least a second after that key.
- Files the app rewrites more often than every 3 s, such as a log, never
settle, so every check waits until the bound.
- The bound counts the reads too, so a slow `data/get` ends the check at
15 s. Only the first read, which every grade needs, runs to its own
timeout.
The bound and window are `SETTLE_BOUND_S` and `SETTLE_WINDOW_S` in the
verifier. `check_build_task` waits the same way before it grades each
witness stage; see [witnessing](desktop-build-witness.md#witness).
## Typed views
The views follow the [state view](desktop-application-state.md)'s formats and
shapes: SQLite tables (at most 64 ordinary tables, `tables_truncated` past
that, and per table `columns`, `primary_key`, at most 500 `rows` and
`truncated`, with blobs, unsafe integers and text that is not UTF-8 tagged as
REA tags them), JSON, XML as `{root, truncated}` with REA's node, depth and
whitespace rules and its `doctype` when the document declares one, key file
groups, text as
`{lines, line_count, truncated}` split on `\n` and `\r\n`, and zip archives
as their entries, read with Python's `zipfile` from the file in place, with
REA's entry, size, compression ratio and archive limits and its wording for an
entry it does not read. The module reads the central directory itself, as REA
does, and shows `zipfile` only the records it lists, reading each entry in
bounded chunks no further than a byte past its declared size. A gzip file is
the view of the document inside it, with `"compression": "gzip"` beside its
format, decompressed with Python's `zlib` within the same entry size and ratio
limits, its members joined as REA joins them. A JSON file
holding a number JavaScript cannot read exactly, such as an integer above 2^53,
or whose retained value nests more than 500 levels deep, gets an `error` instead of a view, as in
REA. Duplicate keys keep their last value before the depth check. Text that is not one JSON
document, such as JSON Lines, stays text. Format detection follows REA's
rules. Files larger than 64 MiB get an `error` instead of a view
and are not read. The selected `file_bytes` also applies to scrubbed seed task
views before admission, independently of how much seed data the export can
copy. SQLite table views have a shared `sqlite_view_bytes` allowance, default
16 MiB of compact UTF-8 JSON per table, checked before retaining each row.
An oversized table keeps its error while other tables remain readable. Raising
this view allowance does not raise the independent 16 MiB native SQLite
value/record limit or REA worker reply limit; worker facts include additional
schema and typed-cell metadata and may be refused earlier. The TypeScript
and exported Python readers count their respective native compact JSON encodings;
float and exponent spelling can make their exact byte thresholds differ even
when they report equal numeric facts. The Python byte counter quotes large
strings in bounded chunks before retention.
Exported SQLite readers require Python 3.11 or later for the native connection limits; the
AgentEnv compatibility workflow uses Python 3.12.
XML is read with Python's `expat` twice: with namespace
processing, which refuses what REA's parser refuses, such as an undeclared
prefix, then without it, so element and attribute names keep the prefixes the
document spells and namespace declarations stay attributes, as in REA's view;
text splits into nodes where REA's splits it, at CDATA sections, comments and
processing instructions. Both passes read a DOCTYPE as REA does: with
parameter entity parsing off, so the DTD it names is never read, and handlers
that refuse an internal subset, an external entity and an entity expat would
skip; the first pass also checks each start tag for a reference to an
undeclared entity, which expat drops from an attribute value without a word
when a DOCTYPE names a DTD. Known differences remain:
- dconf databases are reported as binary, since REA reads them with `dconf`;
inside an archive, one is `dconf` with an error, as in REA.
- Zip entry names: a name in an Info-ZIP Unicode Path extra field is used only
where Python's `zipfile` reads it (3.12 and later), and an entry whose local
header names it differently from the central directory is damaged rather
than read.
- SQLite databases are read in place, read-only, rather than from a staged copy;
a write lock the running app holds is waited out for a quarter of a second
and then gives the file an `error`, and so does a table or column name that
is not UTF-8.
Unless it was [made for the task](#a-seed-made-for-the-task), the seed is the
user's real data. Keep the bundle local and do not publish it.
The class was checked against agentenv-framework 0.9.1464 with
`python -m agentenv_protocol.computer._declare` and `agent-env run --dry-run`.
-334
View File
@@ -1,334 +0,0 @@
# Desktop application state
`rea inspect-desktop-application-state <capture-folder>` reads a folder written
by [desktop application capture](desktop-application-capture.md) and describes
the application's state: which observed files are the application's, what each
retained version contains as typed JSON, and how each file changed between
checkpoints. It reads only the capture folder; the application is not run
again.
```sh
rea inspect-desktop-application-state ./mousepad-capture --format json
```
Views use bounded defaults. When a view reports truncation or a byte-budget
refusal, retry the same capture with `--view-policy` JSON overrides:
```bash
rea inspect-desktop-application-state ./mousepad-capture \
--state-path /home/ada/Documents/notes.txt \
--view-policy '{"text_lines":10000,"file_bytes":134217728}' --format json
```
`--state-path` selects an exact original captured path and can be repeated.
Omitting it examines every captured path. Selection preserves the captured
checkpoint and digest; it does not substitute the file's current contents.
A path absent from the capture is an input error.
`--view-policy` accepts positive safe-integer overrides for `file_bytes`,
`sqlite_tables`, `sqlite_rows`, `sqlite_view_bytes`, `text_lines`, `xml_nodes`, `xml_depth`,
`parsed_bytes`, `json_depth`, `zip_entries`, `zip_entry_bytes`, `zip_bytes`,
and `zip_ratio`. Unspecified controls keep the defaults described below.
`xml_depth` admits at most 494 and `json_depth` at most 991: these bounds
follow the fixed Evidence JSON nesting limit, including the captured-version
and ZIP-member wrappers and XML's element/children representation. A request
above either bound is rejected before reading the capture.
Evidence parameters retain only the supplied `view_policy` overrides; omission
and an explicitly empty selection remain distinct. State inspection and task
drafts report the resolved policy separately as
`normalized_result.effective_view_policy`. Changes apply on this call, without
a restart. File and ZIP byte counts refer to retained file bytes or uncompressed
ZIP member bytes, not transport size. Gzip document views share `zip_entry_bytes`
and `zip_ratio` with ZIP entry views; `zip_bytes` bounds a ZIP archive, not a
gzip stream. These selections also apply to the decompressed document's format
view. `sqlite_view_bytes` instead bounds each
projected table's compact UTF-8 JSON, including column names, primary keys,
rows and truncation metadata; it defaults to 16 MiB. Rows are charged before
retention. A table that exceeds this allowance keeps an actionable `error`,
while other tables remain readable. Increase this control and retry the same
capture for a projected-view refusal. It does not raise the SQLite worker's
separate fixed value/record or complete-reply limits, which may refuse earlier.
Each reader counts its native compact JSON encoding. Float spelling can differ:
Python emits `16.0` where JavaScript emits `16`, and exponent spelling also
varies. Identical numeric facts can therefore cross this byte threshold at
different settings. This is not a common transport-byte guarantee.
The same option is available on `describe-desktop-application-task` and
`export-desktop-application`. Export uses the accepted policy for task
validation and emits it into the Python readers used by the environment's
`data/get`; re-export with another policy to change those reader budgets.
Task checks apply `file_bytes` to the actual scrubbed seed bytes, including any
growth from replacing the capture home with `/home/user`. A seed over that
view allowance may still be exported under the independent acquisition limit,
but its unreadable view cannot establish a known task failure.
The TypeScript and exported Python readers share the policy's defaults.
These desktop capture workflows remain CLI-only.
Raising a view budget cannot recover content the capture never retained,
change numeric exactness, enable encrypted or unsupported archive formats,
or raise SQLite worker, Evidence nesting, process-memory or transport limits.
Keep increases specific to the exhausted budget. Truncation and incomplete
comparison coverage remain visible until the requested facts were examined.
The result is Evidence with operation `inspect_desktop_application_state` and
predicate type `rea.desktop-application-state`. Its confidence is `inferred`,
because roles come from heuristics; the views themselves are read from the
retained bytes, which are checked against their SHA-256 before use.
## Paths and roles
The view covers every file retained at the baseline and every file that
changed later. Each path records the evidence the capture holds about it:
| Reason | Meaning |
| ------------------------ | ------------------------------------------------------------------------- |
| `state_directory` | It lies in one of the application's state directories, in either spelling |
| `opened_by_application` | An application process held it open during a sample |
| `named_in_window_title` | An application window title spelled its name, `~/` or absolute path whole |
| `selected_path` | It lies under a `--state-path` or absolute `--argument` of the capture |
| `dconf_application_keys` | It is a dconf database holding groups whose path names the application |
| `desktop_service` | It lies where desktop services and toolkits keep shared files, or caches |
| `desktop_trash` | It lies in the desktop Trash, `~/.local/share/Trash` |
A folder beside a state directory whose name differs from it only in letter
case counts as that state directory, so a profile the application first
creates during the capture, such as GIMP's `~/.config/GIMP` beside the
candidate `~/.config/gimp`, is its state too.
The role follows from the reasons, in this order: `shared_desktop` for a file
in the desktop Trash, which the user deleted, whatever else names it;
`application_state` for application keys in dconf, or a state directory outside shared desktop
locations; `document` for a file named in a window title or selected by the
caller, even if the application holds it open, except that a selected file
under a hidden folder of the home, such as `~/.config/MuseScore`, a dotfile
there, or a lock an application holds, such as LibreOffice's `.~lock.<name>#`
beside a document, is `application_state`, judged by the path as the caller
named it, so a file reached through `~/.config/MuseScore` stays the application's
when that folder is a link to `~/settings/MuseScore`; the innermost selected
path holding a file decides, so a score selected itself inside the link's
target keeps its own name and is a document (a capture written before it
recorded selected paths is judged by their physical spelling);
`application_state` for another
open file outside shared locations; `shared_desktop` for other files in shared
locations; `unrelated` otherwise. A window title names a file only when no
other file name character adjoins the name, so `day.txt` does not match
`today.txt - Notes`, and never by a folder of a path the title shows: in
CherryTree's `home-notebook.ctd - /home/user/Documents - CherryTree 1.1.2`,
`user` names no file. A name without an extension, such as `Makefile`, must
also stand on its own rather than end a longer path, as in
`Makefile - Editor`. A file under `~/.cache` or `~/.config/dconf` is never
named by a title. The capture's checkpoint acknowledgements rank files by the
same rule.
Application names, which select dconf groups, come from the package, the
executable, the desktop entry's id and `StartupWMClass`, and the window match.
The entry's display name is left out, since a generic name such as `Settings`
would claim other applications' groups.
## Versions and views
Each path lists its versions: the baseline, then its state after each
checkpoint where it, or a database's `-wal` or `-journal` file, changed. A
checkpoint that leaves an absent file absent adds no version. A file's state
at any checkpoint is its last version recorded at or before it, which is how
the [export](desktop-application-export.md#the-seed-checkpoint) and the
[task draft](desktop-application-tasks.md) read it at the seed and end
checkpoints they are given; `capture.checkpoints` lists the names in order.
Views hold the captured bytes as they are, the capture host's name and user
included; the export removes those from the bundle. A version records
whether the file was present, the SHA-256 of its retained bytes, and a view:
| Format | Detected by | View |
| --------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `sqlite` | The `SQLite format 3` header | Every ordinary table's columns, primary key and up to 500 rows |
| `dconf` | The `GVariant` header | Groups whose path names the application, plus the names of the other groups |
| `zip` | A zip local file header, `PK\x03\x04`, whatever the name | Its file entries in archive order, each with its own format and view |
| `json` | A `.json` name or a leading `{` or `[` that parses | The parsed value, from files up to 4 MiB nested up to 500 levels |
| `xml` | An XML name, a declaration or a leading element that parses | An element tree (`name`, `attributes`, `children`); text kept exactly, DTDs never read |
| `keyfile` | Only `[group]`, `key=value` and comment lines, with at least one group | Groups of `key` → string value, from files up to 4 MiB |
| `text` | Other UTF-8 text | Lines, up to 5,000 |
| `binary` | NUL bytes or invalid UTF-8 | None |
A gzip file is read as the document inside it; see
[compressed documents](#compressed-documents).
Text that opens with an element, such as a Freeplane map's
`<map version="freeplane 1.7.0">` with no declaration, or with a DOCTYPE that
has no internal subset and then an element, such as a KmPlot plot's
`<!DOCTYPE kmpdoc>` before `<kmpdoc version="4">`, is XML when the whole text
is well formed, whatever the file is called, a `.txt` file included;
otherwise it stays text. The exported verifier makes the same choice.
A JSON value holding a number JavaScript would round, such as an integer above
2^53, is not read rather than shown rounded.
An XML view names elements and attributes as the document spells them, prefix
included: a saved Inkscape drawing's root is `svg`, in its default namespace,
with `sodipodi:docname` and the declarations `xmlns` and `xmlns:sodipodi`
among its attributes, and a Writer paragraph is `text:p`. An element's
`children` hold its elements and text in order, text split where a CDATA
section, comment or processing instruction falls. Whitespace-only text between
child elements is formatting, and dropped, in an element that holds no other
text, unless `xml:space="preserve"` applies; in mixed content, an element with
text of its own beside its children such as a paragraph split into runs, it is
kept, so `<p>Meet on <s>Sunday</s> <s>evening</s></p>` keeps the space between
the runs. Whitespace here is XML's: space, tab, carriage return and line feed;
a no-break space is text. The view keeps up to 5,000 nodes, comments and
processing instructions counted, nested up to 64 levels, and sets `truncated`
when it leaves some out.
A document that begins with a DOCTYPE naming its DTD, as AbiWord's `.abw`
does, reads as the same element tree, and the view records the declaration
beside `root`, its public identifier normalized as XML matches it:
```json
{
"doctype": {
"name": "abiword",
"public_id": "-//ABISOURCE//DTD AWML 1.0 Strict//EN",
"system_id": "http://www.abisource.com/awml.dtd"
},
"root": { "name": "abiword", "attributes": {}, "children": [] },
"truncated": false
}
```
An identifier the DOCTYPE leaves out is `null`, and a document without one
has no `doctype`. The DTD is never fetched or read, so it adds no default
attributes and declares no entities. A DOCTYPE with an internal subset
(`<!DOCTYPE a [ … ]>`), which could declare entities, is not read as XML, nor is
a document that refers to any entity but XML's five (`&amp;`, `&lt;`, `&gt;`,
`&apos;`, `&quot;`); such a file is text.
Exported XML views use REA's same formatting-whitespace classification, so
retained XML text and the task checks that address it stay aligned.
SQLite databases are read through REA's SQLite worker from a private copy,
with any retained `-wal` or `-journal` file staged beside it; the worker
accepts an inactive rollback journal and refuses a hot one. A version is not
read when its `-wal` or `-journal` file existed but was not retained, or when
the capture did not record whether either existed, since the database alone
could misstate its committed rows. The capture records them under its content
roots, which include both beside a database selected on its own, when its
baseline walk listed every directory, read every path, entered no other
device, did not stop early and recorded what it left out, and wherever they change; an excluded log, and any elsewhere, is
unknown, since an unchanged file and a missing one look alike. After an
incomplete checkpoint, a database or sidecar without an explicit
later change remains unconfirmed even if a later walk completes. REA does not
combine that older file with a newly recorded member of the SQLite set. The
historical versions remain listed; the affected SQLite view has a `view_error`.
A baseline non-file at a sidecar path, or a non-directory ancestor, cannot
establish an absent file. Captures without the required baseline non-file
inventory must be made again before inspection.
A table
that cannot be read keeps its `error` in place of its rows, and changes to that
database are marked incomplete. dconf
databases are read with `dconf dump /` against a private file profile, so the
live database and the dconf service are not touched; this needs `dconf-cli`, or
`REA_DCONF_COMMAND`.
A version has a `view_error` when its bytes were not retained, are over the 64
MiB read limit, are missing or no longer match their digest, exceed a format's
limit above, or its reader failed.
## Archives
Many applications save a document as a zip archive of XML and other files:
FreeCAD's `.FCStd` and `.FCBak`, OpenDocument (`.odt`, `.ods`, whose
`content.xml` is the document), Office Open XML, EPUB and JAR. Such a file is
`zip`, and its view lists its file entries in archive order, leaving out
directories: an entry whose name ends in `/`, or whose attributes mark a Unix
directory or, from an MS-DOS host, the directory attribute:
```json
{
"entries": [
{
"name": "Document.xml",
"size": 11923,
"sha256": "…",
"format": "xml",
"view": {
"root": { "name": "Document", "attributes": {}, "children": [] },
"truncated": false
},
"view_error": null
}
],
"entry_count": 15,
"truncated": false
}
```
Each entry's bytes are read from the archive's verified bytes in memory,
checked against its CRC-32 and declared size, and given a `format` and `view`
by the rules and limits above, with the entry's name in place of a file name.
Nothing is extracted to disk. An entry has a `view_error`, with `sha256`,
`format` and `view` null, when it is encrypted, compressed other than stored or
deflated, larger than 16 MiB, more than 100 times its compressed size, past the
64 MiB read from one archive, or its data is damaged. A SQLite or dconf
database, or an archive, inside an archive keeps its format and `sha256` and
has a `view_error` instead of a view: it is not read. The view lists at most
1,000 entries; `entry_count` counts them all, and `truncated` is true when some
are not listed. REA reads the central directory itself, a record at a time,
through the Zip64 end record whenever a Zip64 locator precedes the end record,
as Python's `zipfile` does, and hands the zip reader only the records it lists,
so an archive of hundreds of thousands of entries costs no more to read
than one of a thousand. An archive whose central directory cannot be read has a
`view_error` of its own.
## Compressed documents
Gnumeric's `.gnumeric`, Dia's `.dia`, Xournal++'s `.xopp`, GnuCash's default
book and Inkscape's `.svgz` are XML compressed with gzip. A file that starts
with gzip's magic bytes, `1f 8b 08`, whatever its name, is read as the
document inside it: its members joined, then given a format and view by the
rules above under the file's own name, so a pointer or check into it is
written exactly as for the same document uncompressed, and the task draft is
the same. The version records the compression beside the format:
```json
{
"format": "xml",
"compression": "gzip",
"view": {
"root": { "name": "gnm:Workbook", "attributes": {}, "children": [] },
"truncated": false
},
"view_error": null
}
```
The document is read only when it is at most 16 MiB and at most 100 times the
file's size, an archive entry's limits, so decompressing stops there; past
them, or when the stream is damaged or cut short, the version has `format`
null and a `view_error`, and a check into it is undecided. A SQLite or dconf
database inside keeps its format and has a `view_error`, an archive inside is
read as an archive, and another gzip stream inside is binary. A gzip file
inside an archive reads as binary. Other compression, such as xz, Zstandard
or bzip2, is not read: such a file is binary.
## Kinds of change
Between consecutive versions a change is `created`, `deleted`, or a
format-specific kind with what was added, removed and changed (up to 100 of
each, with counts):
- `lines` for text: lines added and removed, as a multiset.
- `values` for JSON and XML: JSON pointers to values, containers included, so
an array that becomes an object with the same members is changed. A pointer
longer than 4096 characters is counted but not listed.
- `values` for zip archives too, with array members named by
[selectors](desktop-application-tasks.md#pointers-and-selectors) rather than
positions wherever a selector names one member in both versions: entries by
name, as `/entries/[name=Document.xml]`, and XML elements by tag and an
attribute, as `/view/root/children/[name=Object][@name=Hole]`, so an object
inserted before another does not shift it. A comparison is incomplete when
either view leaves out entries, or an entry's view is missing or truncated.
- `keys` for key files and dconf: `group/key`, with `~` and `/` in the key
written `~0` and `~1` as in a JSON pointer, so `[a/b]` `c` and `[a]` `b/c`
stay apart.
- `rows` for SQLite: `table:` and the row's primary key, or the whole row when
the table has none.
- `content` for binary files, and `unreadable` when either version has no
view.
-927
View File
@@ -1,927 +0,0 @@
# Desktop application tasks
A [desktop application export](desktop-application-export.md) can carry one
task: a prompt for a computer-use agent and checks that grade what the agent
left in the application's files. An agent, not REA, writes the prompt and the
checks from the change the capture recorded. For file checks, REA validates
the task against the capture and keeps it only when its checks fail on the
seed and all pass on the captured end state. The checks are declarative;
optional [readers](#readers) run commands to interpret files, and
[screen judges](#screen-judges) use a model when no file shows the change.
The exported verifier uses the same checks and reader runner as REA.
```sh
rea describe-desktop-application-task ./mousepad-capture --format json > draft.json
# edit the draft's task into task.json: a name, the user's request, the checks
rea export-desktop-application ./mousepad-capture ./mousepad-bundle --task task.json
agent-env run ./mousepad-bundle --task add-milk --agent claude-code
```
## Describe the change
`rea describe-desktop-application-task <capture-folder>` reads the capture and
its [state view](desktop-application-state.md) and returns Evidence with
operation `describe_desktop_application_task` and predicate type
`rea.desktop-application-task-draft`:
- `application`: the environment name the export would use, the title and the
window class.
- `owned_paths`: the paths the export owns, the only ones whose files the
app's `data/get` serves and checks can read.
- `checkpoints`: the two checkpoints compared, `baseline` for the seed and
`final` for the end. They are the capture's first and last unless
`--seed-checkpoint` and `--end-checkpoint` name others, as for the
[export](desktop-application-export.md#the-seed-checkpoint), which takes
the same options; a name the capture does not hold, or an end before the
seed, is refused.
- `files`: each file under them whose state differs between those
checkpoints, with its home path, role, its versions at both (`baseline` and
`final`, the last recorded at or before each) and the net change between them, from the state view's own diff (`lines`, `values`,
`keys` or `rows` with what was added, removed and changed, or `created`,
`deleted`, `format`, `content`, `unreadable`; `content` also when the
views hold only part of a file and match while the file changed). A
database is compared by its rows, since its hash leaves out the `-wal`
file. Text changes list each line
once and add `line_counts`, the lines found at both ends whose number of
occurrences changed. An XML file's change names its members by the
[selectors](#pointers-and-selectors) checks use, as an archive's already
does, and by position only where nothing names a member or where only the
members' order changed. Views appear only for the files the suggestions
cover, with arrays past 200 items and strings past 2,000 characters cut
(`view_cut`).
- `values`, in each of those files: the values its change added, changed and
removed, each with the `pointer` a check names it by, its `kind` (`added`,
`changed` or `removed`), and `before` and `after`: the scalar, an element as
compact XML with the text the XML reader kept, spaces between runs included,
or a JSON value as compact JSON, cut at 600 characters, and null at the end
that lacks it. A file created or deleted lists what its one end holds,
`added` or `removed`: an archive's entries, an XML file's root element, a
JSON file's members or a key file's keys, each whole once. A value only
position reaches has `note: "by_position"`, and its pointer cannot go into a
check; noise an app writes as it saves has `view_state`, `renumbered`,
`volatile`, `timestamp`, `upkeep` or `layout` (see `task`). A record of
text fields added to a list of its kind, as GnuCash's
`<gnc:transaction>`, is listed field by field, and a field that lists
records, as its `<trn:splits>`, record by record, so the cut never hides
one. `refers_to` names up to
five other elements a value equals the `key` or `id` (`xmi:id` among them)
of, outside a view the app keeps of the document, as Umbrello's
`listview` mirroring its model by id, with the
`attribute` holding it, as HomeBank's `category="44"` names
`<cat key="44" name="Groceries"/>`, or the id a child element holds, as a
GnuCash split's `<split:account>` names the account whose `<act:id>` it
is, shown with its name, `<gnc:account …><act:name>Groceries</act:name>…`;
an element added whole lists those of the records of attributes it nests
too, as a KMyMoney transaction's splits name their `<ACCOUNT>` and
`<PAYEE>`, each once. Where several elements share that key, the one
whose tag shares the attribute's stem, as `payee` and `<pay>`, and of
those the one named for the attribute, as `account` an `<ACCOUNT>` before
an `<ACCOUNTID>`. A plain number names an element only through an
attribute so named or ending in `id`, `ref` or `key`, so a list's
`count="5"` names no `<FIXVERSION id="5"/>`. A value that changed lists,
in `alternatives`, its pointer naming the element by each other identifier
or name-like attribute it holds alone among its tag's members, up to
three, as an Inkscape text's `[@inkscape:label=Headline]` beside its
`[@id=text12]`, for a check to name it by instead; an element its tag
alone names inside the first one a condition names, as an EPUB chapter's
`h1`, by its `[@id=chapter-two]`. What a check can
name comes first, text an edit changed before the rest, so no cut drops
the edit, then what only position reaches, then noise. A file lists
at most 20 values and 8 KB, the draft 24 KB, and `values_cut` says when a
limit left some out. Values cover XML, archives, JSON and key files; a text
or SQLite file's change and suggestions already carry its lines and rows.
- `task`: a draft with the name `change`, a placeholder prompt the export
refuses, and suggested checks derived from the net change of the files with
role `document`, or of the app's own state files when no document changed,
excluding locks the export leaves out of the seed. App state files ending
in `.log` remain eligible for checks.
A text file gains `lines_present` for its new lines and `lines_absent` for
lines it lost (when either is at most ten), and `line_count` for each line
holding a letter whose count changed; a new line holding an XML attribute
REA recognizes as generated, such as a named epoch time or `ID_329023692`, gets a
`line_count` of the lines `contains` its text up to that value, as
`<node TEXT="Water filter" ID="`, at the end's number, where both views
hold the whole file, instead of a `lines_present` another save would
fail, or a `volatile_lines` note when no text comes before it. UUID-shaped
text stays exact because its shape alone does not show that it was generated;
JSON and key files gain
`value_equals` and `value_absent` by pointer, a JSON array's records named
by an id field each holds alone at both ends (`id`, `uid`, `uuid`, `guid`,
`key` or a name ending in `_id` or `Id`), as `/lists/[id=chores]`,
whatever order the app saves them in; where the app generated that id, a
record both ends hold is named by its kind field and its first name-like
string field (a key ending in `name`, `title`, `label` or `caption`),
nearest first and up to three keys deep, that holds the same string at
both ends and names it alone at each, as TreeLine's category by
`/nodes/[format=CATEGORY][>data~1NAME=Science]`, and by its id where none
does, as `/nodes/[uid=7584d494d9f511f1a873edd4f1920549]` once its name
changed; and a record only one end holds is named by such a field that
names it alone at its end and nothing at the other, as TreeLine's new book by
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, its other fields
checked through that selector and the fields the selector reads not, nor
a field holding what every like record (one of its kind) at the end holds
there, as the empty `children` of every book; and
by position only where nothing names a record, as a new book whose title
another book holds, with a `by_position` note listing those checks;
never a value that is a uid the app generates (a
GUID, or 32 hex digits as TreeLine's uuid1), nor a check the seed already
passes, as a new record's `format` where another record of that kind held
its place; a removed record is checked gone by its own pointer, as
`/lists/[id=chores]`, and only where the end holds nothing there, so a
record only position reached, whose place another took, is left to the
count; `member_count` checks of the
arrays whose members were added or removed, of the members' kind where
records share a `type`, `kind`, `format` or `category` field, as `/nodes`
matching `[format=BOOK]`, and a count guard of the innermost array an edit
went through, as for XML; SQLite tables gain rows added,
changed and removed, by primary key, and their new row count. An XML file,
and the JSON, key file and XML views of a zip archive's entries, gain checks
by [selectors](#pointers-and-selectors) and never through an element by
position: in XML, first [`text_equals`](#whole-text) for each element whose
text changed, then elements added or removed whole, then values changed,
added and removed, an entry or a JSON value added whole through its
members, and an empty array or object as it is.
- A selector names a member by its tag, then by its identifiers (`name`,
`id`, `xid`, `xml:id`), then by one other attribute, a name-like key such
as Scribus's `ANNAME` or `inkscape:label`, a worded value or an integer of
up to five digits first, then by two such attributes together, as
Gnumeric's cells by `[@Col=3][@Row=2]`, then by a coordinate, a long
number, a length with its unit, as `svg:height="2.629cm"`, an `xml:id`
or `draw:id` of the form `id2` that LibreOffice numbers anew as it saves,
so a Draw shape is named by its `draw:name`, or a token,
such as `XPOS` or `ItemID`, or an id of random case, as Umbrello's
`ubcgF2AWfBmHh` (`u` and twelve letters and digits), an automatic style name an app numbers as it
saves and renumbers on reload, as LibreOffice's
`presentation:style-name="pr1"` (`P1`, `T1`, `MT1`, `L2` on a
`*:style-name`), so a slide's frames are named by `presentation:class`,
or an attribute naming
view state, such as which layer is `selected`, and last by a child
element's text, `[>CHILD=VALUE]`, a name-like tag first, as Osmo's
record by `[>first_name=Delphine]`, MuseScore's `<Text>` by
`[>style=Title]`, a spreadsheet row by its label cell, or GnuCash's
transaction by `[>trn:description=Greenleaf Market]`: text of at most 64
characters, never a time or a token or id a program generates; then by
an identifier a child carries, `[>CHILD@ATTR=VALUE]`, as a GtkBuilder
`<child>` by `[>object@id=submit_button]`, never one an app generates or
numbers anew as it saves; and then by text a child joins from elements
of its own, as Glade's `[>packing=TrueTrue1]` from a widget's packing
properties, which moving the widget changes. A pair
counts every member holding both its values, whichever attributes that
member ranks first. An element with identifiers is named by those alone;
an id the app generates, as Freeplane's `ID_329023692`, or a GUID in
any attribute, braced or not, as VYM's `uuid="{18eadb5d-…}"`, ranks
last, after a child's text, so a new node is named by its `TEXT` and a
new VYM branch by `[>heading=Sunset boat tour]`, while a member
both versions hold pairs by such an id first, so records that keep their
UUIDs and swap titles read as two title edits. A value that stands for
none, as Grisbi's `Id="(null)"` on every transaction, names nothing, so a
new transaction is named by its `Nb`. A child's text is
counted on every sibling, one an identifier names included, so a title
two records share names neither. The selector must name the
member alone at both ends, with the same values; a child's text names a
member that is in one version only when the list's length changed, so a
row whose one cell an edit changed stays that row. An element whose one
identifier changed, at its place among its tag's members, with every
other attribute and its children alike, is that element renamed, as a
MyPaint layer renamed from `Layer` to `Sun`: it is paired by its first
other attribute naming it alone at both ends, as
`[@src=data/sun.png]`, so the rename reads as its `name` changed, and
the old name gets a check that it names nothing, unless its change is
noise, as an id the app renumbered. Where that attribute's values number
the members in document order, as MyPaint 2 writes `src` as
`data/layer-00.png`, `data/layer-01.png` and on along the stack and
renumbers them when a layer moves, each version is named by its own
value of the renamed identifier instead, `[@name=Sun]` and
`[@name=Layer]`.
- A child of the document's root that its tag alone names, and that holds
a list (two or more elements of one tag and no text of its own) and has
an identifier, also gets a condition that keeps it named if the app adds
a sibling of its tag, as Dia adds an empty `Background` layer each time
it reopens and saves a diagram: an id other than `name` both ends hold,
as Inkscape's `[name=g][@id=layer1]`, which a new sibling would not
share; else the text of a child both ends hold, as Dia's
`[name=dia:layer][>dia:object=#Receive order#]`, which an empty sibling
lacks; else its `name`, which a new one may repeat. Deeper, a member of
a list named for its tag, as Gnumeric's `gnm:Sheet` in `gnm:Sheets`,
holding elements and its tag alone naming it, gets an id, else a
name-like child's text, `[name=gnm:Sheet][>gnm:Name=Budget]`, else its
`name`. Other steps, and elements with neither, stay as they are.
An identifier the app numbers by position names nothing: where the
members of a tag hold it as 1..N (or 0..N-1) in document order at both
ends, members were inserted or removed, and every member another
attribute names alone at both ends kept its number or moved it the way
the count did, one at least, as Planner's task `id` after a task is
inserted, members are paired by their other attributes, so an insertion
reads as one task added. A value two members swap, or an id appended at
the end, as Tellico's next `<entry id="4">`, leaves the ids identity. An
identifier numbering the members 0..N-1 in document order at both ends,
where members came or went, as Parley numbers its `<entry id="6">` by
where it saves it, ranks after a child's text where a member has one, so
a new entry is `[>translation=library]` and its kept neighbours are named
alike; Tellico's ids, which count from 1, Tiled's and Rosegarden's,
whose members hold no text, stay identity. An element a change added that
holds nothing but the id of a record it added elsewhere, as Parley's
lesson membership `<entry id="6"/>`, is `upkeep`, listed with `refers_to`
on that record, never a check or a count.
- The text check names the paragraph rather than a run inside it: it goes up
from an element sitting in its parent's text, and from one named by its
tag or position alone to the nearest ancestor named by an attribute, as
AbiWord's `p` by its `xid`, unless it is a field its tag names in a
record of two or more fields holding text, as QElectroTech's `<text>`
beside `<info_name>` in a `<dynamic_elmt_text>`, or KmPlot's
`<equation-0>` beside `<arg-min>`, which is checked itself rather than
the record whose text joins its fields; one line of text the app lays
out between line breaks, as KmPlot's formula before the next element, is
checked, and its siblings guarded, with `normalize_whitespace: true`,
and text whose whitespace alone changed, as a formula the app moved to
an indented line of its own, gets no check; where no attribute names one, to the nearest
ancestor within three levels its tag names with no step by position, as
MuseScore's `VBox` around untitled `<Text>` elements; and where only
position reaches the element, to the ancestor above the outermost step by
position, by an attribute of its own that names it alone where one does,
as an ODS sheet's `table:table[@table:name=Q1 Sales]` above its rows. The
text nodes and runs inside it get no other check, but an element holding
no text that is added or removed there, such as an image or a line break,
keeps its own, since the text check never reads it. A decimal that is an
element's only text, such as a spreadsheet cell's, is checked by its
number instead. Free text that gains a line, such as a shopping list,
uses `text_equals` on the whole text: the current XML checks cannot
compare an unstructured line as a whole item. This can reject a correct
edit that puts the item elsewhere or changes spacing. Text an app keeps in an attribute, one element per run, as Scribus's
`<StoryText><ITEXT CH="Spring "/><ITEXT CH="Fair"/>`, gets a `text_equals`
with `text_attribute` on the nearest ancestor an attribute names, as
`[name=PAGEOBJECT][@ANNAME=Headline]` with `"text_attribute": "CH"`, where
that ancestor holds no text nodes and the attribute's holders are runs,
none inside another and none carrying an identifier or a name-like
attribute beside it, as Umbrello's `<ownedAttribute xmi:id="…"
name="email" visibility="private"/>`, a record whose attributes are
checked one by one, two or more or one holding words with a space: an
element's own attribute, as a Freeplane node's `TEXT` beside the nodes in
it, or a lone value, as `<Bool value="false"/>`, is checked where it is.
The check covers that attribute and the runs holding it; another change
in the frame, as a run's `FONT`, keeps its own check. A record added
or removed whole, an element holding only elements, its fields, such as
Tellico's `<entry>` or a record of repeated `<field>` elements, never
makes the list holding records of its kind a check, whose text would hold
the records the edit left alone, except a record removed whole that only
position names, whose absence only the list's text shows. Text made of
runs is no record: an element whose tag names a paragraph or a piece of
text (`p`, Writer's `text:p` and `text:h`, `h1` to `h6`, `li`, `para`,
SVG's `text` and `tspan`, a `span` or an `a`), one inside such an
element, as Word's `<w:r>` run in its `<w:p>`, one whose elements are all
inline formatting (`b`, `i`, `u`, `em`, `strong`, `bold`, `italic`, a
`span`, `a`, `code`, `sub`, `sup` and the like, Gnote's `size:` and
`link:` runs), as `<list-item><bold>olive oil</bold></list-item>`, or
whose runs split a sentence at a space, as `Meet ` and a linked
`Monday`. Nor does an element holding a date the app stamps in parts
take a text check.
- An element added to a list, where another element of its tag is at either
end, gets a `value_equals` for each of its attributes, best ranked first,
at most six, leaving out its selector's attributes, identifiers, times,
volatile values, view state and values standing for none, as HomeBank's
new `<ope>` gets its `amount`, `category` and `payee`; a value two or
more like siblings all hold, as Grisbi's `Cu="1"`, comes after those
that vary, as its `Pa`, `Ca`, `Dt` and `Am`. Records of attributes
alone that it nests, as a KMyMoney `<TRANSACTION>` its `<SPLITS>` of
`<SPLIT>`, get a `value_equals` of each attribute whose values vary among
the like records, its own and those under its like siblings, at most
three a record and six in all, as each split's `value` and `payee`, but
not one every split holds alike, as `price="1/1"`, nor one each holds
equal to an earlier one, as `shares` repeats `value`. Each record is
named by an identifier that the app neither generates nor numbers anew
in every record holding such a list, as KMyMoney's `S0001` in every
transaction, else by its best ranked attribute short of a coordinate or
an amount, as `[name=SPLIT][@account=A000001]`. An element whose other attributes, two
or more, a like sibling holds every one of, as a new Planner task copies
its neighbour's defaults, and any other element added whole, get a check
that it exists; of those copied, an attribute naming an element the seed
holds by an id that is no plain number, as an Umbrello attribute's
`type` its data type, is the user's choice and keeps its check. A record added to a list that a selector names, as
Tellico's `<entry id="4">` or Osmo's `<record>` by its first name, gets
a `text_equals` for each field, at most five, each by its selector among
the record's fields, as `[name=title]`, leaving out a field repeating the
record's identifier, the field its selector names it by unless it is
the only one left, as a new VYM branch's `<heading>`, a value every
like record holds in that field, as Osmo's `<group>None</group>`, noise,
a field holding ids of its own, a date the app stamps in parts, and blank
text or times; a field that lists records, as GnuCash's `<trn:splits>`,
gets its records' fields instead, each record by its own selector. An
attribute every like record holds alike, as GnuCash's `version`, is the
format's and gets no check. A record only position names also gets a
`text_contains` for each new field of three or more characters, at most
five, at the nearest ancestor a check can name, each absent from that
ancestor's text at the seed.
- Where members of a list were added or removed, a `member_count` of the
list's members of their tag, and of their kind where they share one
(`type`, `kind` or `shape`, as `[name=dia:object][@type=Flowchart - Box]`),
unless every member of the tag holds a value of its own, as Tiled's
objects a free-form `type` (`well`, `door`, `gate`), which labels each
rather than grouping them,
equal to the end's number, which differs from the seed's: at most two a
file, the shallowest lists first, after the other checks. None for a
list of one member at both ends, as a style an app adds for a run, for
members of an element a text check reads, which are runs of its text,
for an element in mixed content, or for noise.
- A decimal written as text, such as `146.83`, is suggested as a number
with `tolerance: 0.001`, unless the seed's value, read as the evaluator
reads a number (`10` and `1e1` included), is that close too. A
`value_equals` that would pass on the seed's value is not suggested.
- An element placed by its own position, holding numeric `x` and `y` as a
like sibling does, as Xournal++ places each `<text>` note on a layer, is
an item of its own, never a run of the others' text: one added whole
gets a `text_contains` of its text on the element holding the items,
which passes wherever it is placed, and that list's `member_count`.
- Noise leaves the suggestions when the file's change holds anything
else, a change only position reaches included: view state, which a tag
or attribute on the way names (the cursor, a selection, as VYM's
`<select>`, the layer, frame,
view or colour the app has current, as Pencil2D's `currentLayer`,
scrolling, zoom, the viewport, windows, splitters, a list or tree view
mirroring the document, as Umbrello's `listview`, Inkscape's `namedview`, view and
configuration settings, the printer, a save's history, edit time,
editing duration and cycles, last-saved time, the generator and
document statistics, and a preview or thumbnail the app renders, as
Xournal++'s `<preview>`), or the value of a name-like attribute or `key` on
the way, when it is one token of several words, as LibreOffice's
`config:config-item[@config:name=CursorPositionX]` inside
`[@config:name=ooo:view-settings]`, or a settings path ending in how a
panel shows (`visible`, `hidden`, `expanded`, `geometry`, `size` and the
like), as VYM's `<setting key="/treeeditor/visible">`; an identifier the
app numbers by position. An `Id` suffix alone does not show that an
attribute is generated, so relationship fields such as `accountId` are
retained; other save-generated IDs may need manual review; volatile
values, UUIDs and epoch times, binary data encoded as base64 (a PNG's
`iVBORw0KGgo`, or 256 or more base64 characters mixing cases and
digits), an id counter (`uniqueid`, `nextobjectid`), a value equal to an
id the change gave a new element where a counter or that element holds
it, as Umbrello's `lastobject` holds the new attribute's `xmi:id`, but
never a reference on an element both ends hold, as a KMyMoney split
moved to a new account, and durations and counts under a document's metadata; times the app stamps, also as a date written in
parts whose tag, or one it sits in, names when the app made, changed or
saved something, as Tellico's `<cdate>` of `<year>`, `<month>` and
`<day>` (`cdate`, `mdate`, `created`, `modified`, `saved`, `updated`,
`entered`), or written whole under such a tag, as GnuCash's
`<trn:date-entered><ts:date>2026-11-06 08:53:08 +0000`,
which no check ever reads, while a date a user enters in parts, as a
birthday, is an edit like any other; an id the app assigns, an `id`
element with `type="guid"`, as GnuCash's `<trn:id>`, and text of control
characters, as its unset options hold; `upkeep`, a change inside an
app's `slots`, as GnuCash's `book:slots`, which it adds to and fills on
reopening a book, a count an app keeps of its members, an attribute
whose key ends in `count` or an element whose tag names a count, whose
integer changed and at each end equals how many elements of one tag it,
or the element holding the text, holds there, the counter left out, as
VYM's root `branchCount`, KMyMoney's `<TRANSACTIONS count="5">` or
GnuCash's `<gnc:count-data>`, or, for a counter the change added, how
many of one tag, more than one, it holds as children, as FreeCAD's
`<ObjectDeps Count="2">`, while a count a user enters, as an item's
`<count>`, is an edit, and the `loext:tab-stop-distance` attributes
LibreOffice fills in `styles.xml` while another entry holds the edit,
and, while a change outside them is the edit, any
change in `styles.xml`, in a document's style containers or its master,
handout and notes pages, or to an attribute naming an automatic style
(`P1`, `MT1`) at both ends, as Impress after a reload wraps its master pages' fields in a new `MT1` style and
rewrites list style `L2`, and a text showing another field that changed
alike, its element beside a field naming that one's key, as
QElectroTech's `<dynamic_elmt_text>` shows a component's label in its
`<text>` beside `<info_name>label</info_name>`; and `layout`, attributes holding a list of
numbers, as Dia's `obj_bb` `6.3225,6.15;12.5775,8.15`, or a number a
name on its way calls layout (a position, point, box, corner, width,
height, side or offset, as `elem_width`), that changed in an element up
to three levels below one whose text changed, below the root's own
children, while their own element's text did not, as Dia moves and
resizes a box around its new text, or below an element whose tag names a
diagram, widget, canvas or scene while the model change is elsewhere, as
Umbrello grows a class's `classwidget` `height` when an attribute is
added; an order's `<amount value="12"/>` beside its new description is
content. Geometry that changed with
no text beside it, as a CAD sketch's, is content. However much noise
comes first, it never takes a check's place. When the change holds
nothing else, the checks stay and a note flags them: up to three in a
document, and as many as fit in the app's own state, where a view
setting may be the task.
- Whitespace an app lays out between elements is not checked, and values
that look like a time the app stamps, in ISO 8601, as C's `asctime`
writes it (`Thu Nov 5 09:20:31 2026`) or as a time of day to the
second (`05:16:21.675302540`), are left out; noise never makes its
ancestor's whole text a check.
- `guards`: checks an author may add to show the edit left the rest alone,
kept out of `task.checks`, with ids `guard-1` and on: for each suggested
`value_equals` or `text_equals` on a member of an XML list, or a
`value_equals` through a JSON record added whole, as TreeLine's new book's
`data/Author` gives
`/nodes/[format=BOOK][>data~1Title=Small Engines of Life]/data/Author`
(its like records are those of its kind, named as both ends pair them,
and it reads only the check's own field), a field on up to
two like siblings, those sharing most of the attributes its selector
names it by and its type, as a cell's `Row` or `Col` and `ValueType`,
then the nearest first, at its seed value, which the end holds
too, trying at most the 16 nearest, so a guard costs the same among
thousands of siblings; for a member named as a field, by a name-like attribute alone, as
QElectroTech's `elementInformation[@name=label]`, the like siblings of
the nearest record out that hold that field too come first, so the other
components' labels guard a component's label rather than its other
fields: the check's own field where like siblings hold different values in
it, and otherwise one that varies among them, as Planner's `work` rather
than the `type="normal"` every task shares, but not one numbering the
members in order, as MyPaint's `src`; never through a sibling only
position pairs. A guard never selects its sibling by
what it reads: a sibling its selector names by the attribute the guard
checks, or by a value that is or holds the text it reads, as Glade's
`[>object=CancelTrueTrueTrue]` for a button's `Cancel` label, is named by
another identifier, name-like attribute or child's text it holds alone
at both ends instead, as Tiled's objects by `[@id=2]` rather than the
`name` the guard reads, or Glade's button by `[>packing=TrueTrue0]`, or
else the guard reads another field, or there is none. A `text_equals` with `text_attribute`
reads its siblings the same way. A text file gets a `lines_present` of the
nearest two unchanged lines of its first new line's kind, list items with the same
marker, as Zim's other `[ ]` action items, or lines the same command
opens, as the other headings after LyX's `\begin_layout Section`, or lines
of the same keyword or `key:` in other records, as a gettext catalogue's
other `msgstr` lines rather than the edited entry's own `msgid`, or else
its nearest lines with words in them; each text both ends hold once, never
a line with no text of its own, a command such as `\end_layout`, tags
alone such as `<fonts>` or `</node>`, or a key alone such as a RedNotebook
day's `24:`, nor a line that repeats; an item placed by its own position
that an edit added, a `text_contains` of each of up to four kept items'
text, nearest first. Where members were only edited, a
`member_count` of the innermost list holding the edited element that an
edit goes through by a member with like siblings, a record rather than a
field of one (a member named by a name-like attribute alone that the
record's like siblings hold too), of the edited member's tag and kind, at
the seed's number, as Dia's boxes in their layer, MyPaint's layers in
their stack or Glade's buttons in their button box: no member of the kind
added or removed; never of runs in an element a text check reads, and
only where the end holds the same number. A guard of a coordinate comes
after those of text. At most four a file besides the count guard, which
sits outside that cap, so sibling guards never squeeze it out. They pass
on the seed, which the export allows, since one check failing there is enough.
- `notes`: why `files` is empty, or why a file's suggestions are thin. Each
note has a `code` and a `message`. For a draft with no files: `no_change`
when no file under the home changed between the two checkpoints and nor
did the application's dconf keys; `unowned_changes` when files changed but
none under the owned paths, with up to five of them in `files`, each with
its home path and role (desktop services' files last), and how many more
in `more`; `uncheckable_changes` when the application's keys in a dconf
database changed, with the database in `files`: no check reads dconf, and
the export only writes the keys as they were at the seed checkpoint, so
make the task's change one the app saves in a file; `unreadable_changes`
when a dconf database changed but REA could not read it, with the read
failure in `error` (install `dconf-cli` or set `REA_DCONF_COMMAND` when it
is missing), since whose keys changed is then unknown; and `no_document`
when no file has the document role, as when no window title shows the
file's name. A capture again with `--state-path` naming the file, or
`state_paths` on a build computer's `start_build_capture`, makes it the
app's. For a drafted file, named in `file`: `by_position` when
changes reach their element only by position, the tags no attribute names
alone in `elements`, or when only the elements' order changed;
`view_state_only` when every change is noise, the checks kept in
`pointers`; `long_text` when an element's text changed past the 2,000
characters a text check compares, the elements in `pointers`; `cut_view`
when the views hold only part of the file; `unreadable_view` when no
check can read it, as a document REA reads as binary;
`volatile_lines` when a text file gained lines holding a value the app
generates with no text before it to count them by; `generated_reference`
when an array references a newly generated record ID that cannot be checked
exactly across runs, with up to five affected `pointers`; and `read_as_text`
when a text file opens like XML, with a declaration, a DOCTYPE or an
element, but reads as text because its DOCTYPE has an internal subset or
it is not well formed, so its checks compare lines that hold attributes in
the order the app wrote them.
The suggestions are a starting point. Keep the ones that say what the user
did; drop the ones that only follow from how the app saves its file. Read
`values` for what changed and its pointers, and `refers_to` for what a number
stands for, before writing the prompt and any check of your own. A LibreCAD
capture that added four holes to a drawing rewrote it from DXF R12 to DXF 2007,
so most of its lines changed; the draft suggests counts of `SECTION`, `ENDSEC`
and `CIRCLE`, and only `CIRCLE` going from 5 to 9 is the user's change.
## The task file
```json
{
"name": "add-milk",
"prompt": "In Mousepad, add buy milk to my to-do list and save it.",
"checks": [
{
"id": "milk",
"kind": "lines_present",
"file": "Documents/todo.txt",
"lines": ["buy milk"]
}
]
}
```
`name` is a portable bundle name other than `smoke` (lowercase letters, digits
and hyphens); `prompt` is what the user asks for, naming the app, since a
solving agent's own shell cannot see the computer's files and the app's name
sends it to the screen; `checks` holds 1 to 32
checks. Each file check has a `kind`, a `file` given as a path under the home such as
`Documents/todo.txt`, and an optional `id` (`check-1`, `check-2` and so on by
position otherwise). A check reads the typed view `data/get` returns for that
file, in the shapes of the state view.
| Kind | Fields | Passes when |
| --------------- | --------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `file_present` | | The file exists |
| `file_absent` | | The file does not exist |
| `lines_present` | `lines` | Every listed line is a whole line of the text view |
| `lines_absent` | `lines` | None of the listed lines is |
| `line_count` | `exact` or `contains`; `equals`, `at_least` or `at_most` | The number of lines equal to `exact`, or holding `contains`, compares so |
| `value_equals` | `pointer`, `value`; optional `tolerance` | The pointer names a value equal to `value`, or within `tolerance` of it |
| `value_absent` | `pointer` | The pointer names no value |
| `text_equals` | `pointer`, `value`; optional `normalize_whitespace`, `text_attribute` | The XML element's [whole text](#whole-text) equals `value` |
| `text_contains` | `pointer`, `value`; optional `normalize_whitespace`, `text_attribute` | The XML element's whole text holds `value` |
| `member_count` | `pointer`; optional `matching`; `equals`, `at_least` or `at_most` | The number of [members of the array](#counting-members), or of those `matching` names, compares so |
| `row_present` | `table`, `where` | A row of the SQLite table has every column in `where` equal |
| `row_absent` | `table`, `where` | No row does |
| `row_count` | `table`; optional `where`; `equals`, `at_least` or `at_most` | The number of rows, or of rows matching `where`, compares so |
The rules both evaluators follow:
- An absent file reads as empty: no lines, values or rows. So `lines_absent`,
`value_absent` and `row_absent` pass on it and a count of lines or rows is
0; `member_count` fails, since its pointer names no array.
- A file that has no view (binary, over the read limit, or unreadable) fails
every check but `file_present` and `file_absent`. Line checks need a text
view, row checks a SQLite view and text checks an XML view, in a file or an
archive entry; another format fails.
- Equality is JSON equality: numbers by value, never a boolean for a number,
objects by their keys whatever their order, arrays in order.
- Pointers follow RFC 6901 (`~1` for `/`, `~0` for `~`) and must start with
`/`; on an array a segment may be a selector, below. Key files read as
`{"groups": {"GROUP": {"KEY": "VALUE"}}}`, so a setting is
`/groups/GROUP/KEY`; XML as `{"root": {"name", "attributes", "children"}}`;
SQLite as `{"tables": {"TABLE": {"columns", "rows"}}}`; a zip archive as
`{"entries": [{"name", "format", "view", …}]}`; a gzip file, such as a
Gnumeric `.gnumeric`, as the document inside it, so its pointers are those
of the same document uncompressed.
- With `tolerance`, a non-negative number, `value` must be a number, and the
value found passes when it is a number, or a string that is a decimal
numeral such as `"5.0000000000000000"`, at most `tolerance` from `value`.
`tolerance: 0` compares numerically, so `"1.2500"` equals `1.25`. Anything
else fails as not a number.
- `contains` matches a line that holds it anywhere, case and all. Checks take
no regular expressions: a pattern run on what the agent wrote could stall
the verifier, and JavaScript and Python read some patterns differently.
- A check decided by what a view leaves out (`lines_absent`, `value_absent`,
`row_absent`, the counts, `value_equals` of a whole object or array, and
the text checks of an element) fails when that view is truncated; a check
satisfied by what the view holds is decided on it. An element's text is not:
the reader may have dropped text from between the runs it kept, so
`text_contains` fails on a truncated view even where the kept runs hold its
value. A text node or attribute value is whole, and decided.
The export refuses, naming the check, a file outside the owned paths, a file
spelled with a leading slash, `.` or `..`, a pointer into a file that is text
at the seed, as the bundle ships it, or at the end checkpoint, the empty
pointer (the whole file) except in `member_count`, a `matching` that is no
selector, a value that names the captured home, or the
capture host's name or `user@host` anywhere (which the
[export removes from the seed](desktop-application-export.md#host-name-and-user)), a
`tolerance` with a value that is not a number, an empty `text_contains`
value, or one of only whitespace with `normalize_whitespace`, an empty
`text_attribute`, a count without exactly one of `equals`, `at_least` and
`at_most`, and a value that looks like a date and time, in ISO 8601 or
`asctime`'s form. Other volatile
content, such as a
counter the app bumps on every save, is for the author to leave out.
## Readers
For a file REA cannot interpret itself, such as an app's binary document,
the task can define `readers`: 1 to 8 commands using the app's own reading
capability. Each reader has a unique `id` of lowercase letters, digits and
hyphens, a home-relative `file`, an `argv` array and `output`, either `json`
or `lines`. Its `argv`, or optional `stdin` text, must contain `{file}`;
the runner replaces it with the path of a private copy of that file.
The command must be installed where the export runs and in the exported
computer where the task will be graded.
The command prints its answer in lines starting with `REA `; other stdout
is ignored. With `output: "json"`, those marked lines join into one JSON
value for `value_equals`, `value_absent` and `member_count`. With
`output: "lines"`, they supply the lines for `lines_present`, `lines_absent`
and `line_count`. A check selects a reader with `"reader": "READER-ID"`
and must give that reader's file as its `file`. Every reader must be used by
at least one check.
Readers execute commands, including any code in `argv` or `stdin`; review
them before exporting or running a task. The runner uses a private copy, a
fresh home, no session bus or computer control token, and no display unless
the reader requests `display: true`. This is execution isolation from the
running app's normal profile, not a filesystem sandbox. `timeout_s` defaults
to 60 seconds and accepts 1 to 300. A failed command, timeout, malformed
answer or missing marked output reports a reader error; it does not establish
that the document lacks the expected value. Correct the command or its output and retry; increase
`timeout_s` within its accepted range when the reader needs longer.
Export runs readers on both retained ends, using the seed bytes as the bundle
actually ships them. The witness and exported verifier run them in the
computer after the app's files settle. Cleanup failures retain the reader's
reported resources: retry the check or close the build computer to retry
cleanup; when the command's outcome is unknown, confirm it stopped before
removing its folder.
## Screen judges
A task whose result exists only on screen can use `screen_judge` checks:
```json
{
"name": "show-list",
"prompt": "In Notes, show the shopping list.",
"checks": [
{
"id": "shown",
"kind": "screen_judge",
"question": "Does the window show the shopping list?"
}
]
}
```
Each question must be a nonempty yes-or-no question of at most 2,000
characters, false on the seed and true when the task is done. A screen task
contains only screen checks and takes no readers. Export refuses it when an
owned file's readable change could be checked instead, or when the build
computer's seed or end screenshot is missing or exceeds 2 MiB. Capture those
checkpoints in a build computer before exporting.
Export ships the two screenshots as labelled failing and passing examples;
it does not call the judge or establish that the questions distinguish them.
The [build witness](desktop-build-witness.md) runs the exported judge on a
fresh screenshot at each stage. The verifier takes a screenshot after the
solving agent stops. For each question, the judge sees both examples and
votes three times; a majority of yes answers passes the check.
The judge uses `[model.roles] judge` in the runner's
`.agentenv/config.toml`, then `[model] default`, then
`anthropic/claude-sonnet-5-5`. Configure its endpoint with
`LITELLM_BASE_URL` and `LITELLM_API_KEY`, or `[model] base_url` and `api_key`.
`agent-env run --judge-model` does not reach this verifier. Without an
endpoint, grading raises and the run fails rather than scoring the agent;
configure the endpoint and rerun the witness or task.
## Pointers and selectors
An app that rewrites a document moves things: FreeCAD 1.0 records more
properties than 0.21, so a hole's radius sits at another position among its
siblings, and a plate's entry moves once a hole is added before it. A pointer
segment applied to an array may therefore name a member by what it holds
rather than by its position:
- `[KEY=VALUE]` names the member whose string field `KEY` is `VALUE`, such as
a zip entry by `[name=Document.xml]` or an XML element by its tag,
`[name=Object]`.
- `[@KEY=VALUE]` names the member whose `attributes` hold `KEY="VALUE"`, as
the XML view keeps an element's attributes.
- `[>PATH=VALUE]` names the member that holds `VALUE` at a path of one to
three `/`-separated steps inside it. `>` reads XML and JSON differently:
| Form | In an XML element | In a JSON record |
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
| `[>CHILD=VALUE]` | a child element `CHILD` whose text, every text node in it with each run of whitespace collapsed to one space and none at either end, is `VALUE` | the string field `CHILD`, compared whole |
| `[>A/B=VALUE]`, up to `[>A/B/C=VALUE]` | a child `A` with a child `B` of that text | the string at `member.A.B`, compared whole |
| `[>A@ATTR=VALUE]`, `[>A/B@ATTR=VALUE]` | a child `A` (or its child `B`) whose attribute `ATTR` is `VALUE` | the field named `A@ATTR`: `@` is part of a JSON key |
Records that keep their fields as elements are named by a child's text:
Osmo's `<contact><first_name>Delphine</first_name>…` is
`[name=contact][>first_name=Delphine]`. A GtkBuilder `<child>` is named by
the id its widget carries, `[name=child][>object@id=submit_button]`, and a
TreeLine node by a field of its `data`,
`[format=BOOK][>data/Title=The Silk Roads]`. Any element on the way will
do; a path with more than three steps, or an empty one, is no selector;
and a missing step, or a value at its end that is not a string, matches
nothing.
- Conditions can follow one another, `[name=Object][@name=Hole]`, and all must
hold. They compare strings only, the selector is escaped like any pointer
segment, so a path's `/` is `~1` inside a pointer,
`/nodes/[format=BOOK][>data~1Title=The Silk Roads]`, and a `/` in a key
cannot be a step; a value cannot hold `][`. A JSON field whose key begins
with `@` or `>` is selected through a path, `[>@type=Book]`.
A selector must name exactly one member. When several match, or none does
before the last segment, the check fails and says which selector matched what.
When the last segment's selector matches none, the pointer names no value:
`value_absent` passes, which is how a check says an element was removed, and
`value_equals` fails. On an object, a bracketed segment is an ordinary key.
A FreeCAD task that checks the user cut a 10 mm hole through the plate, in
FreeCAD 0.21 and 1.0 alike:
```json
{
"name": "cut-hole",
"prompt": "In FreeCAD, cut a 10 mm hole through the middle of the plate and save it.",
"checks": [
{
"id": "cut",
"kind": "value_equals",
"file": "Documents/plate.FCStd",
"pointer": "/entries/[name=Document.xml]/view/root/children/[name=Objects]/children/[name=Object][@type=Part::Cut]/attributes/type",
"value": "Part::Cut"
},
{
"id": "radius",
"kind": "value_equals",
"file": "Documents/plate.FCStd",
"pointer": "/entries/[name=Document.xml]/view/root/children/[name=ObjectData]/children/[name=Object][@name=Hole]/children/[name=Properties]/children/[name=Property][@name=Radius]/children/[name=Float]/attributes/value",
"value": 5,
"tolerance": 0.01
}
]
}
```
Before the edit the archive has no `Part::Cut` and no `Hole`, so both checks
fail on the seed; after it, `Radius` is `"5.0000000000000000"` and both pass.
A check decided by what a view leaves out depends, in an archive, on the entry
its pointer goes into: it fails when that entry's view is missing or
truncated, or when the entry is not listed and the listing is truncated. A
selector must see every member to know it names one, so a check whose pointer
searches an array the view may hold only part of fails as undecided: any array
of a truncated XML view, of a truncated entry's view, or the listing when it
is cut. A position is decided on what the view holds. Like a file, an entry
that has no view (binary, not read, or a database or archive inside the
archive) fails every check into its `view`; its `null` there is no value, while
an entry holding the JSON document `null` reads as `null`. Checks on its `name`,
`size`, `sha256` and `format` still read.
XML that uses namespaces is named as the document spells it, in both
evaluators: an element or attribute by its prefixed name, such as `text:p`
or `inkscape:label`, an element in a default namespace by its bare name, such
as `svg`, and namespace declarations as the attributes `xmlns` and
`xmlns:PREFIX`. Text is a string among an element's `children`. These checks
hold a poster's headline in an Inkscape SVG and a date in a LibreOffice Writer
document:
```json
[
{
"id": "headline",
"kind": "value_equals",
"file": "Pictures/poster.svg",
"pointer": "/root/children/[name=g][@inkscape:label=Layer 1]/children/[name=text][@id=headline]/children/[name=tspan]/children/0",
"value": "Summer Bake Sale"
},
{
"id": "date",
"kind": "value_equals",
"file": "Documents/flyer.odt",
"pointer": "/entries/[name=content.xml]/view/root/children/[name=office:body]/children/[name=office:text]/children/[name=text:p][@text:style-name=Subtitle]/children/0",
"value": "Monday 24 November"
}
]
```
Inkscape writes a line of text as a `tspan` inside its `text` element, and
Writer keeps a paragraph's style in `text:style-name`. A prefix is the one the
document uses, so a check names the prefixes the app writes.
## Whole text
A text node is only the stretch of text between two pieces of markup, and
apps split a paragraph into runs as they save it. AbiWord wraps text typed
into a reopened document in `c` elements, so changing a time to `3:30` saves
`<p>The team meeting starts at <c>3</c>:<c>30</c> in room 4.</p>`; Writer,
which records revision IDs by default, writes text typed in one session as
`text:span` runs. A `value_equals` on the paragraph's `children/0` then reads
`"The team meeting starts at "` and fails. `text_equals` and `text_contains`
read the element's whole text instead:
```json
{
"id": "time",
"kind": "text_equals",
"file": "Documents/meeting.abw",
"pointer": "/root/children/[name=section]/children/[name=p][@xid=3]",
"value": "The team meeting starts at 3:30 in room 4."
}
```
- The pointer names an element of an XML view, in a file or in an archive
entry's `view`, or a text node or attribute value in one, whose text is its
value. Another format, a pointer into an archive outside an XML entry's
view, or any other value fails, an element's `name` and the `doctype`'s
fields included: they are strings of the view, not text of the document.
- An element's text is XPath's `string()` of what the view keeps: every text
node in it and in its descendants, CDATA included, in document order, with
nothing between them. The view drops whitespace-only text between child
elements only in an element that holds no other text, outside
`xml:space="preserve"`, so the space between two runs of a paragraph is
part of its text. Elements that stand for characters, such as Writer's
`text:s`, `text:tab` and `text:line-break`, add none.
- `text_equals` passes when the text equals `value` exactly, case and all;
`text_contains` when it holds `value`, as a time within a sentence, so the
check holds whatever the rest of the sentence says.
- `normalize_whitespace: true` collapses each run of XML whitespace (space,
tab, carriage return, line feed) in both the text and `value` to one space
and trims both ends before comparing, as XPath's `normalize-space()` does,
for an app that reflows text or a value written across lines. A no-break
space is not whitespace there.
- `text_attribute` reads text an app keeps in an attribute: the element's
text is then that attribute's values on it and its descendants, joined in
document order, and its text nodes are left out. Scribus keeps a text
frame's text as `<StoryText><ITEXT CH="Spring "/><ITEXT CH="Fair"/></StoryText>`,
one `ITEXT` per run of formatting, so the check below reads `Spring Fair`
however the runs are split. The pointer must name an element; a text node
or attribute value fails.
```json
{
"id": "headline",
"kind": "text_equals",
"file": "Documents/flyer.sla",
"pointer": "/root/children/[name=DOCUMENT]/children/[name=PAGEOBJECT][@ANNAME=Headline]",
"value": "Spring Fair",
"text_attribute": "CH"
}
```
## Counting members
`member_count` counts the members of the array its pointer names: a JSON
array, or an XML element's `children`. `matching`, a selector of one or more
conditions as [above](#pointers-and-selectors), `[>PATH=VALUE]` included,
counts only the members it matches, however many; without it every member
counts. An element's children hold its text nodes too, so give `matching`
for XML, such as `[name=entry]`. A Tellico task that adds one book to a
collection of 12:
```json
{
"id": "one-more-book",
"kind": "member_count",
"file": "Documents/books.tc",
"pointer": "/entries/[name=tellico.xml]/view/root/children/[name=collection]/children",
"matching": "[name=entry]",
"equals": 13
}
```
It takes exactly one of `equals`, `at_least` and `at_most`. The empty
pointer `""` names the whole file, for a JSON file whose top level is an
array. A pointer that names no array, an absent file's included, fails and
says what it found there. Like the other counts, it is undecided on a view
that may hold only part of the array.
## Validation and export
`rea export-desktop-application <capture> <directory> --task <task.json>`
evaluates file checks on the capture's state view: on each file as it was at
the seed checkpoint, which is the seed, as the bundle ships it with the
capture host's name and user removed, and at the end checkpoint, the
captured end state. Those are the capture's first and last checkpoints unless
`--seed-checkpoint` and `--end-checkpoint` name others.
It refuses a task whose every check passes on the seed, and a check that fails
on the end state, with that check's reason. Only a failure on what the seed's
view shows counts: a task whose checks fail on the seed only because a view
could not be read or may have left out what they look for, such as a line past
the 5,000 a text view keeps, is refused, naming each failure. A capture yields
one task, so the option is taken once. The result's `task` names the task, its
number of checks and those that fail on what the seed shows. A refused task
writes no bundle.
The bundle then holds, besides the app and `tasks/smoke.json`:
| Path | Contents |
| ---------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `tasks/<name>.json` | Deploy the app, deploy the agent and its skills, record the screen, prompt the agent, run the verifier, stop recording |
| `artifacts/<name>-check/verify.py` | Reads the app's `data/get` through the computer and evaluates the checks, one result per check |
| `evals/<env>.toml` | The task, so a run of the bundle runs it; the smoke task runs with `--task smoke` |
The verifier finds the app among the computer's child environments, calls its
`data/get` under the run's grant of the computer's control token, and reads
the `state` data part. It reads again until the app's files settle, as
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved)
describes, and each result's `settled` says whether they did, and `read_cut`
whether the bound cut a read still running. It raises when the app cannot be reached, so the run
fails rather than scoring the agent. The README lists the task, its prompt, its
checks and the recipe below.
## Solve rate
REA does not run agents. To measure how often an agent solves the task, run it
k times, three by default, and count the runs whose summary line says
`passed`:
```sh
for run in 1 2 3; do
agent-env run ./mousepad-bundle --task add-milk --agent claude-code --model MODEL
done
```
The agent defaults to Claude Code and the model is the caller's choice (Claude
Sonnet 5.5 is a reasonable reference). The agent calls a model endpoint, and
agentenv refuses the run without one: set `LITELLM_BASE_URL` and
`LITELLM_API_KEY` where the runs start, or `base_url` and `api_key` under
`[model]` in `.agentenv/config.toml`. A run reported as `scored below 1`
failed the checks and counts as a miss; one reported as `failed` did not finish,
as when the computer or the app could not be reached, and is run again rather
than counted. Each run's instance keeps the verifier's results per check under
`verifications`.
-9
View File
@@ -110,15 +110,6 @@ not establish full metadata validity or correspondence to a native binary;
version 24 inspection still applies only to supported header layouts. See
[IL2CPP metadata](il2cpp-metadata.md).
In a managed directory with a Unity Mono witness, `Assembly-CSharp.dll` and
`Assembly-CSharp-firstpass.dll` get an `inspect_managed_members` probe listed
first; project-named asmdef assemblies keep only the ordinary managed probes.
The same probe is offered for those names in `Managed/` of a
`<Product>_BackUpThisFolder_ButDontShipItWithYourGame` folder (or when that
folder is the root); whether its IL matches the shipped IL2CPP code is unknown.
Point the scan at the game folder: a root that is the `<Product>_Data` or
`Managed` folder itself has no Mono witness path and gets no member probe.
Candidates are derived or inferred from these facts and do not prove loading or
execution. BepInEx, MelonLoader, Easy Anti-Cheat and BattlEye labels identify
path conventions only; runtime presence stays unknown. No bypass is provided.
-338
View File
@@ -1,338 +0,0 @@
# Desktop build computer
A build computer is a local agentenv computer that an agent drives while it
builds a desktop app environment. Claude Code cannot attach a new MCP server
in the middle of a session, so REA keeps the computer up and forwards the
agent's computer-use actions to it through three MCP tools:
| Tool | What it does |
| ----------------------- | ------------------------------------------------------------------------------------------ |
| `start_build_computer` | Starts a computer from a request or a bundle folder; returns its id and a first screenshot |
| `act_on_build_computer` | Sends one action to the computer's driver and returns the driver's answer, images included |
| `close_build_computer` | Tears the computer down, by agentenv's teardown or by removing its sandbox's containers |
The tools are MCP-only, like the debugger sessions: the computer lives as long
as the REA MCP server that started it, or until it is closed. Capturing the
app, exporting its bundle and witnessing the task on such a computer are in
[Building and witnessing an environment](desktop-build-witness.md), and REA's
skill for doing all of it from a request is
[desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Set up a build host
On a Linux host with Node.js and Docker, from a REA
[checkout](getting-started/installation.md#run-rea-from-a-checkout):
```bash
rea setup --client claude_code --desktop-build --install-agentenv
rea doctor --provider desktop_build --client claude_code
```
Setup shows its plan and asks before it changes anything; `--dry-run` only
prints it and `--yes` approves it. It packs REA's runtime archive into
`~/.cache/rea/build-computer/rea-<version>.tar.xz` (under `$XDG_CACHE_HOME`
when set, `~/Library/Caches` on macOS), and packs again once REA's version
changes; an archive `REA_BUILD_COMPUTER_REA_ARCHIVE` already names is
recorded instead. It records `REA_AGENT_ENV_COMMAND`, `REA_DOCKER_COMMAND` and
`REA_BUILD_COMPUTER_REA_ARCHIVE` as absolute paths in that client's REA
registration, keeping a value the registration already has other than an
archive setup packed for an earlier REA version, and refreshes the skill.
`--install-agentenv` runs `uv tool install agentenv-framework==0.9.1464`
when `uv` is on `PATH`, and otherwise prints the command. Setup never
installs Docker or Node.js and never runs `sudo`. Restart the client
afterwards.
Doctor is read-only and checks, in order: a Linux amd64 or arm64 host,
Node.js, procps `ps`, the Docker CLI, a Docker daemon reachable without
`sudo`, 20 GB free where Docker keeps images, agentenv-framework 0.9.1464,
the runtime archive of this REA version, the build settings in the
registration or the environment, and the skill. Each failure names its fix.
It also reports, without counting them against `healthy`, whether a model
endpoint for solver runs is configured and which agentenv containers
earlier builds left. Pass `--json` for the full report.
When a setting is unset where the server starts, REA uses `agent-env` and
`docker` on `PATH` and the archive setup packed for this REA version. The
rest of this page is the same setup by hand.
## Requirements
- A Linux host on amd64 or arm64 with about 20 GB of free disk for the
images agentenv builds and pulls. amd64 and arm64 hosts both build
natively; a host builds only its own architecture.
- Node.js at the version in `.nvmrc`, installed with
[nvm](https://github.com/nvm-sh/nvm) or from
[nodejs.org](https://nodejs.org/en/download), and REA
[run from a checkout](getting-started/installation.md#run-rea-from-a-checkout).
- [Docker Engine](https://docs.docker.com/engine/install/) and its
[post-install steps](https://docs.docker.com/engine/install/linux-postinstall/):
add yourself to the `docker` group, then log in again.
- agentenv-framework 0.9.1464, the version bundles target. Install
[uv](https://docs.astral.sh/uv/getting-started/installation/) first, then
run `uv tool install agentenv-framework==0.9.1464`. Set
`REA_AGENT_ENV_COMMAND` to the absolute path of the resulting `agent-env`
program, normally `$HOME/.local/bin/agent-env`; `command -v agent-env`
shows the path when its install directory is on your `PATH`. Installing it
with pip into a virtual environment instead needs `python3.12-venv` on
stock Ubuntu. REA runs its
helper, `bridge/agentenv/computer.py`, with that runner's Python: the
interpreter on the program's `#!` line, or the `python3` beside it in its
virtual environment.
- `REA_DOCKER_COMMAND`, the absolute path of the Docker CLI, lets REA remove a
computer whose agentenv teardown was not confirmed. Without it, REA names the
command to run instead.
- For a request, `REA_BUILD_COMPUTER_REA_ARCHIVE`: the absolute path of a tar
archive of a runnable REA, with its production dependencies, whose one top
folder holds `scripts/rea.mjs`. The computer runs REA from it. From a
checkout, `npm run pack:build-computer-runtime -- /path/to/rea-prod.tar.xz`
writes one: `npm pack`, its package folder as
`rea-prod` with the lockfile, `npm ci --omit=dev --omit=optional`, then
`rea-prod` as `.tar.xz`. The archive is a copy of the checkout: pack it
again after `git pull` and a rebuild.
Set these where the MCP server starts; a shell export does not reach a server
that is already running.
## Claude Code
Without `--desktop-build`, `rea setup --client claude_code` registers REA
without these variables. Either
export them in the shell that starts `claude`, which starts REA's server with
its environment:
```bash
export REA_AGENT_ENV_COMMAND="$HOME/.local/bin/agent-env"
export REA_DOCKER_COMMAND="$(command -v docker)"
export REA_BUILD_COMPUTER_REA_ARCHIVE="$HOME/rea-runtime/rea-prod.tar.xz"
claude
```
or set them, as absolute paths, in the `env` of the `mcpServers.rea` entry
setup wrote, such as in `~/.claude.json`:
```json
{
"mcpServers": {
"rea": {
"command": "/absolute/path/to/node",
"args": ["/absolute/path/to/rea/scripts/rea.mjs", "mcp"],
"env": {
"REA_AGENT_ENV_COMMAND": "/home/ada/.local/bin/agent-env",
"REA_DOCKER_COMMAND": "/usr/bin/docker",
"REA_BUILD_COMPUTER_REA_ARCHIVE": "/home/ada/rea-runtime/rea-prod.tar.xz"
}
}
}
}
```
Solver runs are `agent-env run` commands the agent runs in its shell, and
agentenv refuses them without a model endpoint: export `LITELLM_BASE_URL` and
`LITELLM_API_KEY` in the shell that starts `claude`, whose Bash tool inherits
them, or set `base_url` and `api_key` under `[model]` in
`.agentenv/config.toml`.
Claude Code asks before each MCP tool call its permission settings do not
allow, and a build makes dozens of `act_on_build_computer` calls. You can
allow REA's tools in those settings' `permissions.allow` list:
`mcp__rea__act_on_build_computer` allows that tool, and `mcp__rea` allows all
of REA's.
A headless run, `claude -p`, ends when the skill asks its upfront questions,
and refuses the tool calls its permission settings do not allow. Put the
answers in the request: the install source when the app is not in the Ubuntu
archive, a synthetic seed or the file to start from, whether to review the
task before solver runs, and the solver runs' count, model and budget, or
none:
```bash
claude -p 'Build me a Gnote environment where the agent adds olive oil to the
shopping list note. Make a synthetic note. Then make 2 solver runs with
<model>, at most $4 in all, without a review first.'
```
## Start a computer
From a request, REA renders a bundle in a private temporary folder: a computer
app on the `ubuntu-24.04` base with the `reference` driver that installs the
app, opens it on demand with `gtk-launch <desktop entry>` from the computer
user's home, so its open and save dialogs start there rather than in the
framework's folder for the app, and keeps no seed and no data hooks. The app is not open when the computer starts:
`start_build_capture` opens it under REA's capture. When no capture is under
way, open it from the desktop's Applications menu, where agentenv lists it
under Other by the environment's name, such as `mousepad`; that entry launches
it through the app's server. The name is the desktop entry's last part when it
is reverse-DNS (`com.github.xournalpp.xournalpp` gives `xournalpp`), or else
the whole entry without a trailing version (`glabels-3.0` gives `glabels`).
When the app is Electron, set `electron` to true. The install rewrites that
desktop entry's command with the same policy an export uses: `--no-sandbox`
immediately after the program, unless the command already has it. The
computer's container does not grant the user namespaces Chromium's sandbox
needs, and Electron exits without the flag. With the rewritten command, the
Applications menu entry can open the app.
```json
{
"source": {
"kind": "request",
"desktop_entry": "org.xfce.mousepad",
"window_class": "Mousepad",
"install": { "kind": "archive", "package": "mousepad" }
}
}
```
`install` takes one of:
| `kind` | Installs |
| ---------------- | ---------------------------------------------------------------------------------------------------------- |
| `archive` | `package` from the Ubuntu 24.04 archive, on amd64 and arm64, with its optional `extra_packages` beside it |
| `apt_repository` | `package` from the repository at `uri` with `suite` and `components`, trusting the key file `key_file` |
| `package_file` | The `.deb` at `url`, checked against `sha256`, for one `architecture`, after its apt `dependencies` are in |
Build steps run with the network off, so a `.deb`'s dependencies must be
listed. `extra_packages` names up to 16 more archive packages, by the same
name rules as `package`, for a runtime piece the app needs that its package
does not install: apt here skips Recommends, so MyPaint 2.0.1 exits at launch
without `librsvg2-common`, GTK's SVG icon loader, and Freeplane 1.7.10 needs
`openjdk-17-jre`, since its launcher refuses noble's default Java 21. The
export installs them beside the app too, as it does a `.deb`'s
`dependencies` when apt offers the `.deb`'s package; see
[build and witness](desktop-build-witness.md).
The bundle adds REA in a build-only layer: Node.js
24.22.1, downloaded per architecture and checked against its release checksum,
and REA from `REA_BUILD_COMPUTER_REA_ARCHIVE`, both unpacked into `/opt/rea` by
a root build step, with `node` and `rea` on `PATH`. It also installs
`libgtk-3-bin`, for `gtk-launch`, and `dconf-cli`, so the capture reads the
app's settings in dconf, such as GnuCash's "Show tips at startup", and the
seed keeps them. An exported bundle carries none of it but `dconf-cli`, which
it installs when it loads the app's dconf keys.
From a bundle folder, such as one `export-desktop-application` wrote, REA
starts that bundle's `tasks/smoke.json` as it is:
```json
{ "source": { "kind": "bundle", "path": "/home/ada/mousepad-bundle" } }
```
REA runs its helper with `PYTHONDONTWRITEBYTECODE=1`, which agentenv's own
import of the bundle's app class inherits, so a start leaves no `__pycache__`
in the folder and its files stay as they were shared.
REA runs the deploy with `run_bundle(keep=True, emulate=False)` on the local
sandbox provider, so only the host's own architecture is built. The first start
of a bundle builds its image and can take minutes; later starts reuse it.
agentenv makes a start wait while another agentenv run on this host writes its
bundle's ids, so starts at the same time wait for each other, several minutes
when a few run together; the progress line then says it is waiting for another
agent-env run to finish writing this bundle's ids.
While it waits, REA sends an MCP progress notification every five seconds with
agentenv's latest line, so a client that resets its request timeout on
progress keeps waiting. REA then waits until the computer's readiness checks
pass and opens one MCP session to its driver. Readiness can pass before the
desktop has drawn, so REA takes a whole-screen PNG screenshot every 500 ms
while it is one colour throughout. The retries share 10 s, each given only
what is left of it, and when that runs out REA returns the latest frame;
progress notifications continue meanwhile. Given `screenshot` with a
`region`, `format` or `quality`, REA then takes one with them, a `region`
cropping that frame rather than capturing the screen again. The result holds
`computer_id`, agentenv's `sandbox_id`, the environment name or bundle folder,
`extra_packages`, what the request named beside the app for the export to
install too (an archive install's `extra_packages` or a `.deb`'s
`dependencies`, empty otherwise), and the screenshot's frame as structured
content beside its image.
A failed start returns the last 16 KiB of the helper's stderr, where a failing
build prints its error, and its exit status; a cancelled one stops the helper,
which tears down whatever agentenv brought up. When that teardown is not
confirmed, the error's `computer_id` names what REA keeps for
`close_build_computer` to retry. Closing the server cancels starts still in
progress and waits for them before closing every computer.
## Act
`act_on_build_computer` takes `computer_id`, an `action` and the driver's own
arguments for `computer_<action>`, which REA passes through:
| `action` | Arguments |
| ------------ | ----------------------------------------------------------------------- |
| `screenshot` | `region`, `format` (`png` or `jpeg`), `quality` |
| `click` | `x`, `y`, `button`, `count` (2 double-clicks), `modifiers` |
| `drag` | `path` of 2 to 50 `[x, y]` points, `button`, `duration_ms`, `modifiers` |
| `scroll` | `direction`, `x`, `y`, `amount`, `modifiers` |
| `type` | `text`, where `\n` presses Enter and `\t` Tab |
| `key` | `keys`, one chord pressed in order, and `repeat` |
| `wait` | `ms` |
| `reset` | `screenshot` |
`reset` is not a driver tool: REA calls the app's `data/reset`, which restores
the bundle's seed and relaunches the app, then waits until the computer's
readiness checks pass again, sending an MCP progress notification every five
seconds while it waits. With `screenshot: true` it then takes a screenshot; if
that fails, the reset still counts as applied and its result has no `frame`
but a `screenshot_error` with the computer's message. A computer started from
a request has no data operations, so it refuses a reset.
Pointer actions also take `frame_id`, `space` and `norm_scale`, and every
action but `screenshot` takes `screenshot: true` for an image afterwards. That
image is taken at once, so after an action that opens a menu or a window, ask
for it with a `wait` of a few hundred milliseconds instead.
Points are pixels of the latest frame, so take a screenshot first; with
`space: "norm"` they are on a `0..norm_scale` grid over it. Keys are W3C names
or one character; REA translates model spellings such as `ctrl+s`, `Return`,
`esc` and `cmd` and leaves other names for the driver to judge. An argument the
action does not take is refused before anything is sent.
The result carries the driver's content unchanged, its text and images, after
REA's own JSON: `action_index` and the driver's structured result. A refusal,
such as a stale frame, is an error carrying the computer's message. A response
larger than the MCP budget is refused with how to ask for a smaller image: a
`region`, or `jpeg` with a lower `quality`. A screenshot with `region` does not
capture the screen again; it crops the latest frame, so take a full
screenshot, or a `wait` with `screenshot: true`, first. That error keeps REA's own result,
`computer_id` included, as `details.result`, since the action or start has
happened.
Actions, the capture steps and the task checks on one computer run one at a
time, in the order REA receives them; one that has to wait for the call
before it sends an MCP progress notification every five seconds meanwhile.
Different computers run side by side.
Every action is logged on the computer in order with the arguments sent, its
start and end and whether it was applied, refused or failed. The log keeps the
latest 256 actions within 256 KiB of JSON, dropping the oldest first. Each
string argument, such as `type`'s `text`, is kept to its first 1 KiB and each
list, such as `drag`'s `path`, to its first 64 items; an entry names the
arguments it cut in `truncated_arguments`, and the driver gets them whole. The
log counts every action, so it says how many it dropped, and keeps the index of
the last applied action of each kind for rules over the whole session. The log
and the computer's recording hooks (`computer-recording/v1` start and end under
REA's own request id, and the segments fetched through the driver's `data/get`
one at a time, each read within its manifest size and checked against it) are
what the witness records; they are not tools of their own. A recording larger
than a quarter of the V8 heap is refused before any segment is read.
## Close
`close_build_computer` ends REA's MCP session and closes the helper's stdin,
on which the helper runs agentenv's teardown. When agentenv confirms nothing is
left, the result says `teardown: "agentenv"`. Otherwise REA removes the
containers of that exact sandbox: agentenv runs a computer as
`agent-<sandbox id>` without a label and labels its other containers
`agentenv.sandbox=<sandbox id>`, so REA lists both with `docker ps -aq
--filter`, removes them with `docker rm -f`, and says
`teardown: "container_label"`. When neither confirms it, the error names what
is left and the computer stays listed, so closing it again retries. A close
cancels the call in progress on the computer, refuses the calls waiting for
it, and tears down once that call has ended. Closing the
MCP server closes every computer the same way, and a helper whose REA server
exits sees its stdin close and tears down.
The control token that guards the computer's framework-only routes stays in
REA: no result, error or log carries it. Code running in the computer as its
user can still read it, which is acceptable for a local build.
See the [specification](specs/desktop-app-environments/build-computer/spec.md)
for scope and verification.
-302
View File
@@ -1,302 +0,0 @@
# Building and witnessing an environment
An agent such as Claude Code builds a desktop app environment from a request
on a [build computer](desktop-build-computer.md): it makes the seed through
the app, makes the change the task asks for, exports the bundle with the task
it writes, and then witnesses the task in a fresh computer of that bundle.
REA runs its own capture, task draft and export inside the build computer and
grades the witness with the same checks the exported verifier runs.
| Tool | What it does |
| -------------------------- | ------------------------------------------------------------------------------------------- |
| `start_build_capture` | Downloads the package lists as the computer's user and launches the app under REA's capture |
| `mark_build_checkpoint` | Takes a named checkpoint and returns the capture's counts and the files that changed |
| `stop_build_capture` | Stops the capture and returns its checkpoints and the task draft between `seed` and `end` |
| `export_build_environment` | Exports the bundle with the agent's task inside the computer and copies it to a folder here |
| `check_build_task` | Grades a task against the app's live state; with a `stage`, files the check in the witness |
Like the computer tools, these are MCP-only and act on a `computer_id` from
`start_build_computer`.
REA's skill walks an agent from a request such as "build me an Inkscape
environment where the agent changes a poster's headline" to a witnessed
bundle with these tools: the questions it asks first, when it stops for the
person, the install, the seed, the task, the witness and its final report.
See [desktop builds](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/references/desktop-build.md).
## Build
1. `start_build_computer` from a request. REA is installed in the computer.
2. `start_build_capture`. REA first checks the installed runtime's export
schema for synthetic-seed support. If the archive is too old, repack a
current REA, update `REA_BUILD_COMPUTER_REA_ARCHIVE`, restart the REA MCP
server and start a new build computer. REA then writes an apt configuration under
`~/.rea-build/apt` and runs `apt-get update` as the computer's user, so the
capture finds the package's sources (`package_lists` counts the lists,
and `package_lists_error` gives apt's exit status and last lines when the
download failed; a later start runs the update again until one succeeds). It then starts `rea capture-desktop-application
--desktop-entry <entry>`
in the computer, leaving `~/.rea-build` out of every checkpoint, and returns
once the app's window is open, with a screenshot. For an app installed
from a `.deb` file, the build layer keeps that file at
`/opt/rea/package.deb` and the capture records it with `--package-file`. Do not open the app from
the menu first: the capture refuses an app whose window is already open.
When the capture ends before the window opens, REA reads its output. An app
that exits at launch, such as one missing a runtime, is reported with its
program, exit code, run time and the last lines of its stderr, at most 8
lines and 800 characters of the last 8 KiB the capture keeps, and
`details.diagnostics` keeps the capture's record of the session. A capture
that failed, such as one refusing to start, gives its own message and
remediation, with the whole failure in `details.diagnostics.command_failure`
and, when its cleanup is incomplete, the run tokens and paths it left in
`details.resources`. A checkpoint or stop that finds the capture ended
reads its output the same way.
`state_paths` names files or folders in the computer's home that hold the
app's state, relative to the home, after `~/` or absolute under it, at
most 16; each becomes a `--state-path` of the capture, so it is retained
from the start and its files count as the app's even before they exist.
REA counts a file as a document only when an app window's title shows its
name or `~/` path, so name the file the app will save when its title shows
something else, as MuseScore's shows a score's title: a file outside
hidden folders, other than a lock such as LibreOffice's `.~lock.<name>#`,
becomes a document, which the bundle owns, seeds and opens
at launch, and one under a hidden folder such as `~/.config/MuseScore` the
app's own state, also when that folder is a link elsewhere in the home. A
path with `..`, or naming the home itself, is refused before the computer
is asked; the computer refuses an absolute path outside its home, one in
`~/.rea-build`, and one a link leads out of the home or into
`~/.rea-build`, resolving a path that does not exist yet through its
nearest existing folder, and starts nothing.
3. Make the seed through the app with `act_on_build_computer`, saving it
where the user keeps such files, such as `~/Documents`, then
`mark_build_checkpoint` `seed`.
4. Make the change the task will ask for, save it, and
`mark_build_checkpoint` `end`.
5. `stop_build_capture`. The app is closed and the capture finishes. The
result lists the checkpoints with their counts, the package and the
sources the capture found, its limitations and the draft: each file under
the paths the export owns that changed between `seed` and `end`, with its
net change and, for XML files, archives, JSON and key files, its `values`,
the values the change added, changed and removed with what each was and
became, at the pointer a check names it by; suggested checks with a
placeholder prompt; and `guards`, optional checks that the rest kept its
seed values: like siblings' fields, each line kept beside new text, and
the count of a list whose members were only edited. The draft's
[task section](desktop-application-tasks.md#describe-the-change) describes
each. `notes`, each with a `code` and a `message`, say why a draft has no
files: `no_change` when no file under the home changed between them and
nor did the app's dconf keys; `unowned_changes` when files changed but
none under the owned paths, listing up to five in `files` with their
roles, desktop files last, and counting the rest in `more`;
`uncheckable_changes` when the app's keys in dconf were read and changed,
listing the database in `files`, since no check reads dconf (a version
the capture kept no copy of, as it keeps none of the database at the
start, is `unreadable_changes`, not an empty key list);
`unreadable_changes` when a dconf database changed but REA could not read
it, such as without `dconf-cli` or when a present version's bytes were
not retained, with the read `error`, so whose keys changed is unknown;
and `no_document` when no file has the document role.
`unowned_changes` and `no_document` point at `state_paths`. They also say,
for a drafted `file`, why its suggestions are thin: `by_position`,
`view_state_only`, `long_text`, `cut_view`, `unreadable_view`,
`volatile_lines`, `generated_reference` or `read_as_text`.
6. Write the task: its name, the prompt in the user's own words naming the
app (a solving agent's own shell cannot see the computer's files, and the
export's limitations say when a prompt names neither the app's title,
environment nor package), and checks
that hold after the change and not on the seed, from the suggestions,
`values` and any `guards` worth keeping.
7. `export_build_environment` with the task and a new or empty `directory`.
The export runs inside the computer with `--seed-checkpoint seed
--end-checkpoint end`, so it seeds what was saved at `seed`, scrubs the
computer's host name and user from it, and refuses a task whose checks
all pass on the seed. It also passes `--synthetic-seed`, since the agent
made the seed for the task, so the README does not call it the user's real
data. When the seed holds exactly one document and the app's desktop entry
takes files, the bundle's app opens that document at launch. A file moved
to the desktop Trash is not seeded, and the export's limitations name what
the Trash held at `seed`. The limitations and the README also name the
folders where files changed during the capture that the bundle neither
owns nor seeds, such as settings that keep a first-run wizard away, up to
eight collapsed to their top folder, such as `~/.config/MuseScore`, and a
count of the rest; capture again with them in `state_paths` to own them.
Lock files the app holds while it runs, such as LibreOffice's profile
`.lock` and its document lock `.~lock.<name>#`, GnuCash's
`<book>.LCK` or Grisbi's `.<file>.lock` beside its file, are left out of
the seed, since a copied one can keep the app from starting or saving. The
limitations and README list them after "Left out of the seed". A project's
`Cargo.lock` or `yarn.lock`, a document or a path in `state_paths`, and app
logs that are not locks are seeded. The limitations name seeded
WebKit or Chromium storage and caches too, which the app keeps for its
own running. When the application held files open in a folder whose
starting content the capture did not retain, a limitation names the
folder and the seed files in it. For a
`.deb` app that no apt source the capture found offers, the export takes
the request's URL with `--package-url`; one apt offers installs from apt.
Either way the request's `dependencies` reach the export as
`--extra-package`, as an archive request's `extra_packages` do, so a
runtime piece the agent named, such as MyPaint's `librsvg2-common`, is
installed beside the app; the limitations and the README list them,
except the app itself and dependencies the bundle already names for a
`.deb` it installs from the file. REA lists the bundle's files
with their sizes and SHA-256 in the computer, into a file beside the
bundle that it reads through `data/get`, since a large bundle's list
outgrows a command's output; it then brings the files back through the
driver's `data/get` in answers of at most 8 MiB, and checks and writes
each answer into the folder before asking for the next, so one answer at
a time is held; a failure leaves no partial folder.
8. `close_build_computer`.
A checkpoint name is lowercase letters, digits, dot, dash and underscore; the
capture's own `baseline`, `stop`, `time-limit` and `exit` are refused, and so
is a name already taken. The acknowledgement comes from the capture after it
has walked the home and retained what changed: `entries` walked, `changes`
since the checkpoint before, `retained_files` and `retained_bytes`, and up to
ten `changed_files`, home-relative, with `more_changed_files` counting the
rest. Folders are left out. Files an app window's title has named so far
come first, as a document saved and no longer held open is in Gnumeric, LyX
or Tiled; then files named themselves in `state_paths`; then those the app
holds open, those under a selected folder and those in its state
directories; then the rest; and caches and WebKit, Chromium or QtWebEngine
storage last, even under a selected folder such as `.config/libreoffice`
with its icon cache, each by path. So a save's document leads the list.
When the computer's REA is older than these lists,
as when `REA_BUILD_COMPUTER_REA_ARCHIVE` still names an earlier build, the
checkpoint is still taken and both are `null`: which files changed is
unknown. Pack the current REA into the archive to get them back.
Each step can be asked again after its answer was lost:
- A start whose answer is lost, or whose window never opened, stops the
capture it may have left running. Until the computer confirms that stop,
REA keeps the capture and the next `start_build_capture` retries the stop
before starting another.
- A checkpoint the capture took whose acknowledgement was lost comes back
from the capture's own log when the same name is marked again.
- A stop whose summary or draft was not read finishes when stop is asked
again, and a finished stop repeats its result. The draft and the export
trust the checkpoints the capture's own Evidence records.
- A step that saved something, such as an export's bundle or the witness's
record, returns it even when its call is cancelled or the computer closes
as the save finishes. `start_build_capture` saves nothing, so a close or
cancellation during its screenshot of the opened app cancels it.
The capture, the draft and the export run as the computer's user in its home,
without any `AGENTENV_*` variable, so neither REA nor the app holds the
computer's control token. REA sends each command through agentenv's
`computer-exec/v1` route with that token, which stays in REA, and reads larger
outputs, such as the draft, the export's Evidence and the bundle's file
list, from files under `~/.rea-build` through `data/get`, since a command's
output is capped at 1 MiB. In a command's Evidence or failure read this way,
as in its own output, REA replaces the token in every string, after checking
the Evidence's digest; the bundle's files arrive as they are.
Calls on one computer run one at a time, in the order REA receives them:
an action, a capture step, a check or a close waits until the call before
it on that computer has answered, and sends progress notifications while
it waits. So a check reads the app with no action half done, and a witness
stage is admitted, read and filed before anything else on that computer
runs. Calls on different computers run side by side. Closing a computer
cancels the call in progress on it and refuses the calls still waiting.
## Witness
Start a fresh computer from the exported folder with `start_build_computer`
and `{"kind": "bundle", "path": ...}`; agentenv boots the app with the seed,
on the seed's document when the export opens one. When the computer never
gets ready and the bundle's seed holds lock files, the start's error names
them: delete them from the bundle, or export again, which leaves them out.
A bundle records no state directories, so the error names only files by a
profile lock's name and LibreOffice's document locks.
Then, with the task exactly as exported:
| `stage` | Before it | Its checks must |
| ------------- | ----------------------------------------------------------------- | ---------------------------- |
| `seed` | nothing; give `directory`, a new or empty folder for the record | fail at least once |
| `after_edit` | the task done through the app from its prompt alone | all pass |
| `near_miss` | a change that is almost right, such as a misspelled line | fail at least once |
| `after_reset` | `act_on_build_computer` with `action: "reset"` as the last change | each come out as on the seed |
`check_build_task` reads the app's `data/get`, takes the views it returns
beside the files and evaluates each check with REA's evaluator, the one the
export validates tasks with and the exported verifier mirrors, one result per
check. Once the files settle it runs the task's
[readers](desktop-application-tasks.md#readers) in the computer through
`computer-exec/v1`, as the verifier does, and a check through a reader reads
that reader's view. A task of [screen judges](desktop-application-tasks.md#screen-judges)
reads no files: REA takes a screenshot and runs the bundle's own judge on it
with the agentenv runner's Python and model endpoint. Like the verifier, it grades only once the app's files settle, since an
app such as Gnote saves a few seconds after the last key. It reads `data/get`
every 0.5 s and grades once every file's bytes have held for 3 s, counted
from the read that returned them. 15 s after the first read began, it grades
the last completed read: no read starts then, and one still running is cut.
It sends progress notifications while it waits. The result's `settled` says
whether the files settled, and `read_cut` whether the bound cut a read; each
stage in the record keeps both. Every check therefore costs at least 3 s,
and a save that lands more than 3 s after the first read returns is still
missed; see
[grading once the app has saved](desktop-application-export.md#grading-once-the-app-has-saved).
Without `stage` it only grades. The `seed` stage begins a recording
of the display under REA's own request id, one per computer; a seed check
that fails after it, such as on a failed read of the app's state or a lost
reply to the recording's start, can be asked again, and asks for the
recording under the same id, which the computer answers with the recording
it already began. The stages come in order. The edit and the near miss each
need an applied click, drag, scroll, type, typed secret or key since the stage before, with
no reset after the last of them, since a reset puts the seed back rather than
changing the app; `after_reset` needs the reset as the last change, and a
reset that leaves any check unlike on the seed fails the witness, its
stage's `remediation` naming those checks. On a computer started from a
bundle folder, the seed refuses a task whose `tasks/<name>.json` or verifier
in the bundle differs from what the export writes for it. A reset
relaunches the app, and REA waits until the computer is ready again before
answering, sending progress notifications while it waits.
Only a check that fails on what the app's view shows counts as a failure. A
check that fails because its file or table could not be read, such as an
SQLite file locked past the reader's wait, or because what it looks for may
lie in what its view cut, is `undecided` in its result: it shows nothing
about the task. That covers a line `lines_present` misses in a text view cut
at 5,000 lines, a row `row_present` misses in a table whose view keeps only
its first rows, a `value_equals` or text check pointer that names nothing
in a truncated view or past a cut zip listing, the absence checks and counts
on a cut view, and `text_equals` or `text_contains` of an element on one,
even where the element's kept text holds the value, since the cut may have
dropped text from between its runs. Any other check that passes on what the
view kept stays decided, and so does a value found there that differs from
the one expected.
The result's `passed` and description are the exported verifier's, which has
no undecided results. A stage that only undecided checks would settle, such
as a near miss whose one failing check could not read its file, files nothing
and is refused; check it again once the files read whole.
A stage whose checks do not show what it must ends the witness, and so does
`after_reset`. REA then ends the recording, fetches its segments through
`data/get`, each checked against the recording's manifest, and writes into
`directory`:
- `recording/NNNN.mp4`, the recording's segments;
- `witness.json`, REA Evidence (`rea.desktop-build-witness`) holding the
verdict, the stage that failed if one did, the task's name and SHA-256,
each stage's results with how many logged actions came before it and
whether the app's files had settled (`settled`), the
computer's [action log](desktop-build-computer.md#act) as it keeps it, each
action with its arguments, timing and outcome, with `actions_dropped`
counting the earliest actions it no longer keeps, and the recording's
segments with their sizes and SHA-256. The record's limitations say when
the log dropped actions or cut arguments, which each entry names in
`truncated_arguments`, and which stages graded files that had not settled.
The stages' rules read the log's count of actions and the last applied
action of each kind, which it keeps for the whole session, so they hold
however many entries it dropped. The result returns the same record. When writing it fails, such as for a
folder something else wrote into, the witness keeps its stages: check the
stage that ended it again, with the same task, and REA writes the record
without checking again. Once it is saved, the same check returns the saved
record again, so a record saved as its call was cancelled is not lost. A
computer is witnessed once; witness again from a fresh computer.
See the [specification](specs/desktop-app-environments/build-witness/spec.md)
for scope and verification.
-82
View File
@@ -1,82 +0,0 @@
# Documentation ownership and authoring
Use the [documentation directory](../README.md) to find existing material before
adding another guide. Preserve one canonical owner for each contract or workflow;
link to that owner instead of copying its claims into several guides.
## Choose a destination
| Document | Destination |
| -------------------------------------------------------- | -------------------------------------- |
| Installation, client setup and first CLI/MCP steps | `docs/getting-started/` |
| Analyst workflows and evidence interpretation | `docs/guides/<domain>/` |
| Analysis-engine configuration and format prerequisites | `docs/providers/` |
| Shared runtime contracts and evidence semantics | `docs/reference/` |
| Development, testing, release and authoring instructions | `docs/development/` |
| Composition diagrams | `docs/architecture/` |
| Compatibility changes for a released version | `docs/migrations/` |
| One feature specification, plan and tasks | `docs/specs/<domain>/<feature-slug>/` |
| One architectural decision | `docs/adr/<domain>/<decision-slug>.md` |
Use meaningful, lowercase domain and feature slugs. Global numeric prefixes are
unnecessary: different domains can add features independently, and branches do
not compete for the next sequence number. Check the existing domain directory
and linked issues before creating a feature with the same slug.
Guides describe existing behavior and limitations. Specs define one proposed
feature or domain change; an existing guide does not become a specification by
moving it into `specs/`.
## Feature lifecycle
Create the specification before implementation. Each feature directory contains:
- `spec.md`: problem, desired behavior, requirements, acceptance and non-goals.
- `plan.md`: the single implementation plan, contracts and architectural choices.
- `tasks.md`: incremental, testable steps linked to requirements and verification.
Split independent domains or separately reviewable features before writing their
specs. A broad `ROADMAP.md`, when useful, only lists child specifications, order
and dependencies; requirements and implementation tasks belong to each child.
Cross-repository contracts have one producer-owned specification. Consumers link
to that contract instead of maintaining competing copies.
Record an isolated architectural decision separately in
`docs/adr/<domain>/<decision-slug>.md`, with **Status**, **Context**,
**Decision** and **Consequences**. Link the ADR from affected specifications.
Do not create decision records for routine file moves or invent past decisions.
Specs do not replace the contribution-scope decision required for a new provider
or expanded execution authority. Follow
[contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
## Links and public routes
Use relative links to physical Markdown files so GitHub readers can follow them.
Link repository files outside `docs/` with their full GitHub URLs. Preserve
existing heading anchors when moving a guide.
[`documentation.ts`](https://github.com/morluto/rea/blob/main/docs/.vitepress/documentation.ts)
owns categorized site navigation and the existing guides' stable public routes.
Update it when adding or moving a guide. VitePress rewrites categorized sources
to the existing public paths; its Markdown hook resolves repository links from
the physical source and renders them against those public routes. Keep basenames
and route assignments unique. Edit links use the physical source file.
For a move, search for both the basename and its old full path. Update README
translations, authored skills, verification instructions, documentation-fact
readers and release configuration when they consume that path. Preserve
historical release records and references bound to earlier commits or tags.
## Generated ownership and checks
The [documentation directory](../README.md#machine-owned-documentation) identifies
the machine-owned outputs and their producers. Their existing paths are retained
because scripts, package consumers and release history depend on them. Change
canonical sources and regenerate; do not fix a generated projection by hand.
Run `npm run docs:check` for documentation facts and generated consistency, and
`npm run verify:docs:routes` to build the complete production site and check local
links, anchors and physical edit targets.
When a physical path moves, also run the tests that read it. Follow
[development checks](testing.md#developer-commands) before requesting review.
+1 -14
View File
@@ -50,8 +50,7 @@ explicitly with the managed tools to establish PE/CLI facts.
## Admission and containment
Metadata is range-read with a cache of at most 64 KiB, bounded by the remaining
selected metadata allowance. Default policy budgets are 16 MiB of
Metadata is range-read with a 64 KiB cache. Policy budgets are 16 MiB of
manifest metadata, 100,000 files, 64 KiB per UTF-8 string, 512 MiB of stored or
expanded bytes per file, 256 path parts per member, 250,000 distinct in-memory
path-tree nodes, and 2 GiB of aggregate expanded bytes. The node budget counts
@@ -61,18 +60,6 @@ streams 64 KiB chunks and reuses completed probe facts during extraction;
ordinary native files without a marker incur one extra
read pass before host hashing.
Both artifact tools accept `artifact_limits.dotnet_bundle` to select
`metadata_bytes`, `entry_count`, `string_bytes`, `member_bytes`,
`expanded_bytes`, `path_parts`, and `path_nodes` for the next operation. The
string allowance applies to every manifest string, including the bundle ID.
For example, retry a complete bundle whose manifest exceeds the default with
`{"artifact_limits":{"dotnet_bundle":{"metadata_bytes":33554432}}}`.
The CLI accepts the same selection through `--artifact-limits` JSON. Omitted
fields preserve defaults; errors name the exact selected control, effective
value and required minimum when known. No REA restart is needed. Summary
delivery and paging do not change producer admission, and larger reader
budgets do not raise MCP response or Node representation limits.
Extraction hashes the whole host again through the same descriptor used to read
members and compares it with the inventory's root digest before creating an
output directory. This adds a full-host read pass to extraction, including when
@@ -6,7 +6,7 @@ Disconnecting the inspection's page session can clear caller-owned
`navigator.platform` emulation inside Chromium. REA cannot restore another
client's complete user-agent/client-hint configuration. Use a disposable target
or let the owning client reapply its configuration; see the shared
[CDP state limitation](../web/browser-observation.md#provider-authorities).
[CDP state limitation](browser-observation.md#provider-authorities).
This passive runtime surface is distinct from the target-free static
[`analyze_javascript_application`](javascript-artifact-reconstruction.md)
@@ -14,7 +14,7 @@ workflow. Static analysis reads a local ASAR or extracted directory without
executing application code. Passive observation inspects an existing Electron
page. The results keep static inferences and runtime observations distinct.
Packaged fuse state is a separate static read:
[`inspect_electron_fuses`](../../electron-fuses.md) reports whether this binary will
[`inspect_electron_fuses`](electron-fuses.md) reports whether this binary will
honor Inspector flags, `ELECTRON_RUN_AS_NODE`, embedded ASAR integrity, and
`app.asar`-only loading.
Use the separate
@@ -265,9 +265,6 @@ retained arrays and counters without invoking application iterators or Array
species.
Listener-chain recognition distinguishes owned, unowned and unknown metadata.
Only a directly owned wrapper establishes hook dispatch. An application's data
link to an owned wrapper remains unknown: `.listener` can name that wrapper
without invoking it.
Data links may be inherited; absence is established only after completing the
bounded prototype lookup. Exhausting either walk, encountering a getter or seeing a Proxy leaves
ownership unknown. The hook preserves native registration without adding another
@@ -331,7 +328,7 @@ budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, seeded by
`update_analysis_settings` can override `asar.max_header_bytes` for new
operations without restarting REA. Cached headers are checked against each
operation's admission, including after a decrease. See the
[settings design](../../design/limits-and-recovery.md#failure-and-retry).
[settings design](design/limits-and-recovery.md#failure-and-retry).
An oversized header is
not parsed; selecting a member path does not bypass admission, and the page
stays unauthorized. The header cache separately retains at most that many bytes
+1 -17
View File
@@ -44,22 +44,6 @@ file memory regions and paths (`NT_FILE`), fault signal and code (`NT_SIGINFO`),
auxiliary vectors (`NT_AUXV`) across 32-bit and 64-bit images. Other owners or unparsed
notes keep `core` null.
Linux `CORE` decoding requires System V/unspecified (0) or GNU/Linux (3)
OSABI with ABI version zero. An unspecified OSABI alone does not establish
Linux origin. An explicitly different OSABI or ABI version leaves `core` null
and its Linux-specific `type_name` unknown, with a limitation explaining the
profile. Owner/type numbers, descriptor ranges and hashes, GNU-owned notes,
and complete table-examination coverage remain available.
For Android `NT_ANDROID_TYPE_IDENT` (`.note.android.ident`, owner `Android`,
type `1`) notes, `android.android_api` is decoded from both legacy 4-byte
descriptors (where `ndk_version` and `ndk_build_number` remain null) and modern
132-byte or extended NDK descriptors, which also populate `android.ndk_version`
and `ndk_build_number` when their 64-byte slots hold NUL-padded
printable ASCII. Truncated or non-ASCII Android descriptors degrade `android`
to null while preserving the raw descriptor digest and preview, and other
owners keep `android` null.
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
appear twice when both tables declare it.
@@ -70,7 +54,7 @@ the image never carried notes in another container or overlay.
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
32 note entries are examined by default across every table. Retained note
names, section names, GNU descriptor text, core dump string representations, and Android ident strings share a scan budget based on
names, section names, GNU descriptor text, and core dump string representations share a scan budget based on
the source bytes plus any decoded section-name table, capped at
`max_file_bytes`. The raw source and decoded table must also fit together
within that ceiling. If admission fails, the error reports the required
+1 -1
View File
@@ -23,7 +23,7 @@ and consider setting both `REA_MCP_INPUT_SCHEMA_PROFILE` and
advertised output schema to 4,096 bytes by relaxing nested client validation;
REA still validates complete results against the canonical contracts. Pair
this with retained summaries and selected inventory views to limit the context
needed for one question. See [MCP contracts](reference/mcp-contracts.md) for the profiles
needed for one question. See [MCP contracts](mcp-contracts.md) for the profiles
and their client compatibility tradeoffs.
The layout facts follow ESP-IDF v6.1 (`esp_app_format.h`, `esp_app_desc.h`,
@@ -118,25 +118,16 @@ CLI keeps its existing complete result.
## Resource and lifecycle limits
Per-call `producer_limits` selects `input_bytes`, `output_bytes`,
`diagnostic_bytes`, `address_space_bytes`, `file_bytes` and `cpu_seconds`.
Omission keeps 4 MiB input, 16 MiB reply/file, 1 MiB diagnostics, 3 GiB address
space and 30 CPU seconds. Omitted `file_bytes` follows the selected
`output_bytes`, so raising the reply allowance does not silently meet the old
file-write allowance. Explicit file limits remain independent. These fields
accept positive safe integers; changes apply to the next call without a REA
restart. The CLI accepts the same exact policy as JSON, for example:
```sh
rea inspect-evm-interface ./runtime.hex hex --producer-limits '{"input_bytes":8388608,"output_bytes":33554432}'
```
The owned worker's 256 MiB JavaScript heap remains separate. The worker command
deadline is `timeout_ms` (default 30000 ms, integer 1 through 2147483647), or
`--timeout-ms` on the CLI. Raising it does not raise producer, heap or client
limits. An oversized carrier reports non-retryable `resource_constraint` with
`reported_limits.maximum_input_bytes` and recovery naming
`producer_limits.input_bytes`.
Inputs are limited to 4 MiB, complete replies to 16 MiB and combined diagnostic
output to 1 MiB. An owned worker has a 256 MiB JavaScript heap; its profile
maxima are 3 GiB virtual address space and 30 CPU seconds, with tighter inherited
limits retained. The worker command deadline is
`timeout_ms` (default 30000 ms, integer 1 through 2147483647) on the next call,
or `--timeout-ms` on `rea inspect-evm-interface`. Omission keeps 30000 ms.
Those byte, address-space, and CPU ceilings have no runtime override, and
`timeout_ms` does not raise them. An oversized selected carrier returns a
non-retryable `resource_constraint` with resource `file-size` and its fixed
input allowance in `reported_limits.maximum_input_bytes`.
Complete-reply allowance refusals use the same non-retryable resource family,
with `reported_limits.maximum_reply_bytes`; malformed JSON remains unreadable
output. This refusal leaves no complete report to retrieve through another
@@ -144,7 +135,7 @@ delivery channel. Virtual address space differs from resident memory. The worker
soft/hard limits, reports its configured soft limits inline and leaves hard limits
unchanged. WASM trap reservations are disabled
so the engine can run within that virtual limit; actual memory is not equated to
the selected address-space allowance. Limits fail without partial success. Stable snapshots,
its 3 GiB allowance. Limits fail without partial success. Stable snapshots,
private files and owned processes are cleaned independently of caller cancellation.
Invalid-input, unsupported, cancellation, timeout and malformed-reply failures
retain bounded stdout/stderr under `details.captured_output`, including whether
@@ -11,20 +11,7 @@ select a returned ELF separately for supported Ghidra analysis.
On Linux, Binwalk and Unblob run as REA-owned processes and require a
procps-compatible `ps` on REA's `PATH`; see the shared
[Linux process prerequisite](../../getting-started/installation.md#linux-owned-process-prerequisite).
## Compare stored firmware components
For supported FIT and legacy uImage containers, use the built-in
`inspect-artifact` / `inspect_artifact` inventory and shared
`compare-artifacts` / `compare_artifacts` workflow. This route needs no Binwalk,
Unblob or extraction: exact logical paths, stored-byte hashes and declared
metadata establish component changes, with source Evidence and explicit unknowns.
The outer container at `.` is separate from its children. See the
[FIT version comparison recipe](../../fit-images.md#comparison) for CLI,
MCP, summary delivery and selected supporting facts. Its real-data verification
covers a pair of OpenWrt FIT releases; it does not establish comparison of
decoded root filesystems or arbitrary firmware formats.
[Linux process prerequisite](installation.md#linux-owned-process-prerequisite).
## Caller-supplied tools
@@ -209,7 +196,7 @@ Select a verified native child for the existing tools:
Then use `analyze_function`, `inspect_native_api` or the other native primitives.
Native architecture and executable validation still apply. Generic
[MIPS ELF](../../providers/ghidra-mips.md) and [RISC-V ELF](../../providers/ghidra-riscv.md) lanes are separate
[MIPS ELF](ghidra-mips.md) and [RISC-V ELF](ghidra-riscv.md) lanes are separate
documents with their own admission boundaries. This milestone does
not add PPC decompilation, raw flash load profiles, bootloader/device
tree semantics, vulnerability scanning, emulation or hardware runtime capture.
@@ -258,12 +245,9 @@ tree semantics, vulnerability scanning, emulation or hardware runtime capture.
3, allowed range 1–10.
Staging budgets are polled and rechecked after exit; short-lived overshoot is
possible. The per-file process limit is one byte above the larger of the selected
payload budget and the fixed report/diagnostic file allowances, so a small payload
selection still permits the mandatory report, and a staged file the kernel stops
at the limit exceeds its budget instead of passing as complete. A tighter
inherited host limit still applies and can remove that byte, so a staged file
that reaches the installed limit is rejected as possibly truncated.
possible. The per-file process limit is the larger of the selected payload budget
and the fixed report/diagnostic file allowances, so a small payload selection
still permits the mandatory report. A tighter inherited host limit still applies.
The inherited file-size limit bounds each file, not total storage;
address-space limits are per process, not an aggregate memory quota. Worker
count is not CPU affinity for every external extractor. These limits do not
@@ -317,7 +301,7 @@ bootloader image, such as an OTA download or app partition, for its chip,
classified segments, checksum and SHA-256 results, descriptor and Secure Boot
v2 signature blocks. `extract-artifact` / `extract_artifact` writes one file
per segment and signature block. Reading is portable and needs no external
tool. See [ESP images](../../esp-image.md).
tool. See [ESP images](esp-image.md).
## Inspect ESP flash dumps
@@ -328,11 +312,11 @@ facts for the bootloader and each app partition, and the OTA records with a
derived selection; the running partition stays unknown. Extraction writes
each partition, `nvs_keys` included, as an ordinary file. For a dump that
does not start at flash address 0, select `esp-flash-image` with
`flash_offset`. See [ESP flash images](../../esp-flash-image.md).
`flash_offset`. See [ESP flash images](esp-flash-image.md).
## Inspect extracted devicetrees
Use [`inspect_devicetree`](../../devicetree.md) / `rea inspect-devicetree` on an
Use [`inspect_devicetree`](devicetree.md) / `rea inspect-devicetree` on an
extracted DTB for its node/property tree, reservations, hardware summary and
selected phandle relationships. FDT v16/v17 inspection is portable and requires
no external tool or disassembler.
@@ -343,7 +327,7 @@ Use `inspect-artifact` / `inspect_artifact` on an Intel HEX or Motorola S-record
file for its address regions, declared start address and record facts, then
`extract-artifact` / `extract_artifact` for one raw file per region. Decoding is
portable and requires no external tool. See
[Intel HEX and S-record images](../../hex-records.md).
[Intel HEX and S-record images](hex-records.md).
## Inspect U-Boot legacy uImages
@@ -356,7 +340,7 @@ is extractable as `payload.bin`, and multi-file and script images additionally
expose each sub-image as `image-<n>.bin` from the stored size table. Declared
addresses are facts from the header, not proven entry points, and compressed
payloads stay as stored. Intel HEX/S-record and FIT images are separate
readers and are not covered here. See [U-Boot legacy uImage](../../uimage.md) for
readers and are not covered here. See [U-Boot legacy uImage](uimage.md) for
component padding, preserved names, resource policies and verification scope.
## Inspect UF2 and DfuSe images
@@ -366,7 +350,7 @@ address regions and its block numbering, or on a DfuSe file for its targets,
elements and DFU suffix, then `extract-artifact` / `extract_artifact` for one
raw file per region or element. A DfuSe suffix CRC mismatch follows
`integrity_policy`. Decoding is portable and requires no external tool. See
[UF2 and DfuSe images](../../uf2-dfuse.md).
[UF2 and DfuSe images](uf2-dfuse.md).
## Inspect U-Boot FIT images
@@ -374,4 +358,4 @@ Use `inspect-artifact` / `inspect_artifact` on a U-Boot FIT (`.itb`) for its
images, declared load and entry addresses, configurations and verified hashes,
then `extract-artifact` / `extract_artifact` for one raw file per image.
Decoding is portable and requires no external tool. See
[U-Boot FIT images](../../fit-images.md).
[U-Boot FIT images](fit-images.md).
+9 -132
View File
@@ -22,10 +22,9 @@ use the current projection identity; there is no pre-coverage FIT summary varian
A FIT is a flattened devicetree (FDT v16 or v17) with a top-level `/images`
node. Recognition walks the structure block within the first 4 KiB of the file
only; the file name does not matter. A devicetree whose `/images` node begins
after 4 KiB or a devicetree without `/images` stays on the plain devicetree
route: `open_binary` refuses it and [`inspect_devicetree`](devicetree.md)
can parse it when its FDT version is 16 or 17. Other FDT versions are not
recognized as FIT and are also unsupported by `inspect_devicetree`. A U-Boot control DTB
after 4 KiB, a devicetree without `/images`, and any other FDT version stay on
the plain devicetree route: `open_binary` refuses them and
[`inspect_devicetree`](devicetree.md) still parses them. A U-Boot control DTB
that contains `/images` is treated as a FIT. The container format is `fit`.
Recognition is a probe. The complete devicetree parse decides, so a file whose
@@ -125,9 +124,8 @@ Failures apply to the whole file and return no partial inventory.
The devicetree, including any embedded image data, must be at most 16 MiB; a
larger `totalsize` fails as a memory limit. Store large images externally
(`mkimage -E`): external image bytes are streamed from the file within their declared
32-bit sizes and the resource bounds below, without loading the whole image
into memory. A FIT may declare at most 1,024 images and 1,024
(`mkimage -E`): external data of any size is streamed from the file and never
loaded into memory. A FIT may declare at most 1,024 images and 1,024
configurations, and an image at most 8 hash nodes; the devicetree parser's
node, property and depth bounds also apply. The sum of declared image sizes
is bounded by the shared archive expansion budget. These are fixed resource
@@ -151,116 +149,10 @@ After extraction:
## Comparison
Use the shared `compare-artifacts` / `compare_artifacts` workflow for exact
component paths and stored content identities. Neither extraction nor an external
firmware engine is required. This compares the bytes exposed by this FIT reader;
it does not inspect files inside a compressed root filesystem, match renamed
components, or establish runtime equivalence.
### CLI
Retain each complete inspection while returning a summary:
```sh
rea inspect-artifact /absolute/baseline.itb --detail summary \
--evidence_output /absolute/baseline-evidence.json --json
rea inspect-artifact /absolute/candidate.itb --detail summary \
--evidence_output /absolute/candidate-evidence.json --json
```
The CLI returns a `{summary, evidence_bundle}` receipt. Use each response's
`summary.normalized_result.parent_evidence_id` as the corresponding
`evidence_id` below. Save this request as `comparison.json`, replacing the paths
and IDs with the returned values, then run `rea compare-artifacts comparison.json
--json`. Evidence-file paths must be absolute. Naming the exact complete parent
avoids ambiguity when a bundle contains several inspection or inventory records.
```json
{
"left": {
"kind": "evidence-file",
"path": "/absolute/baseline-evidence.json",
"evidence_id": "COMPLETE_BASELINE_EVIDENCE_ID"
},
"right": {
"kind": "evidence-file",
"path": "/absolute/candidate-evidence.json",
"evidence_id": "COMPLETE_CANDIDATE_EVIDENCE_ID"
}
}
```
### MCP
For each file, call `open_binary` with its path, then `inspect_artifact` with
`{"detail":"summary"}`. Opening the second file retains the first inspection's
Evidence. Compare the two complete parent IDs without resending their graphs:
```json
{
"name": "compare_artifacts",
"arguments": {
"left": {
"kind": "retained-evidence",
"evidence_id": "COMPLETE_BASELINE_EVIDENCE_ID"
},
"right": {
"kind": "retained-evidence",
"evidence_id": "COMPLETE_CANDIDATE_EVIDENCE_ID"
}
}
}
```
The result counts unchanged, added, removed, changed, unknown and contradictory
paths, returning every delta inline with source Evidence links. The root `.` is
the outer FIT and is counted separately from `images/<unit-name>.bin`. A changed
outer hash therefore does not imply every component changed. Additions and
removals require complete container-entry coverage; ambiguous duplicate paths or
missing observations cannot prove absence or equivalence.
Artifact comparison includes an image's payload interpretation, including load
and entry declarations. Equivalent external offset forms and inventory positions
leave those declarations unchanged. Unavailable interpretations remain unknown,
while independently known changes survive; a changed row with an `availability`
dimension does not establish the unavailable field's value.
For supporting facts, batch selected paths rather than requesting the entire
inspection again:
```json
{
"name": "inspect_analysis_view",
"arguments": {
"source": {
"kind": "retained-evidence",
"evidence_id": "COMPLETE_BASELINE_EVIDENCE_ID"
},
"view": {
"kind": "inventory",
"collection": "occurrences",
"logical_paths": ["images/kernel-1.bin", "images/fdt-1.bin"],
"fields": [
"artifact_id",
"logical_path",
"hash_status",
"source_location",
"source_path"
],
"offset": 0,
"limit": 64
}
}
}
```
Read `collection: "nodes"` with those `artifact_ids` and `fields: ["sha256",
"size"]` to obtain content facts. Join by `artifact_id`, not array position.
Continue using the returned page metadata when necessary. Source locations are
ranges in the original FIT, not decoded filesystem offsets. Complete Evidence
remains available through the retained parent and `export_evidence_bundle`.
Summary and field selection reduce delivery only; they do not skip inventory
hashing or change coverage.
Artifact comparison reports changes to an image's payload interpretation,
including load and entry declarations. Equivalent external offset forms and
inventory positions leave those declarations unchanged. Unavailable
interpretations remain unknown, while independently known changes survive.
## Verification
@@ -278,18 +170,3 @@ No image was executed.
For small containers, complete delivery avoids an additional view request.
Choose summary and selected views when the omitted facts justify that extra
request; summary delivery does not reduce the underlying inventory work.
The component comparison workflow was additionally checked on the official
[OpenWrt 24.10.0](https://downloads.openwrt.org/releases/24.10.0/targets/mediatek/filogic/)
and [24.10.1](https://downloads.openwrt.org/releases/24.10.1/targets/mediatek/filogic/)
`abt_asr3000-initramfs-recovery.itb` images (8,519,680 and 8,585,216 bytes).
Their SHA-256 values are
`083b174e81ed373f2d2f2ea5366f9291767d95375383528fd6b9955ad815b85a` and
`41129cce68ad6269e5c8c48ef110f97682d6c4202ff2198bfd0d300c3dac132a`.
An independent FDT byte-range/hash oracle found changed kernel and initrd bytes
and an identical DTB; REA reported those two changed components, the changed
outer `.` and one unchanged component. A controlled copy changing only the root
timestamp reported only `.` changed and all three components unchanged. Public
CLI comparison of the exported source Evidence matched the MCP Evidence exactly;
summary and selected inventory views matched the independent ranges and hashes.
These checks did not decompress either initrd or execute the firmware.
@@ -217,47 +217,3 @@ The Gallery workflow and all 147 focused Flutter tests also pass on macOS Sonoma
including the conflicting-load and note-range-limit CLI/MCP regressions. Windows
verification runs under a non-administrator account. Verifiers use a shared
temporary parent so each CLI/MCP process can own and release its own snapshots.
## Selecting and recovering string-pool observations
`inspect_dart_aot` accepts `pool_policy`; the CLI uses `--pool-policy` JSON:
```sh
rea inspect-dart-aot app.apk --pool-policy '{"contains":"package:my_app/"}'
rea inspect-dart-aot app.apk --pool-policy '{"uri_limit":2000,"token_limit":4000,"run_bytes":4096,"token_bytes":256}'
```
A literal, case-sensitive `contains` filter runs before distinct collection,
so an omitted URI or identifier can be selected without returning every other
string. URI/token counts describe matching entries; `printable_run_count` counts
complete printable runs before filtering. Unset budgets preserve the defaults:
500 returned entries per URI category, 2000 identifier-like tokens, 1024 bytes
per printable run, and 128 bytes per identifier. Budget values must be positive
safe integers. Collection retains at most twice the selected category limit;
counts remain lower bounds when that capacity is reached. A run exceeding
`run_bytes` is omitted whole, rather than reported as a fabricated prefix.
The result names the budget reached, and `string_pool.policy` records the
selected filter and effective values. A larger budget or a narrower selection
applies to the next call without a REA restart; explicit choices stay in Evidence.
For a larger package-URI inventory, select the category and its relevant budget:
```sh
rea inspect-dart-aot app.apk --pool-policy '{"contains":"package:","uri_limit":2048}'
```
The equivalent MCP input is `pool_policy: {contains: "package:", uri_limit: 2048}`.
This recovers all bounded package-URI observations in the measured Gallery and
LocalSend APKs. Check the result's limits and coverage for other targets; long
runs and unsupported snapshot layouts remain separate limitations. The
[calibration report](../../verification/flutter-budget-calibration.md)
keeps the default at 500 because increasing every complete overview also delivers
more strings when they are unnecessary. Use `detail: "summary"` for the MCP
identity/count task described above.
These controls do not raise APK, ELF, library or response budgets, deserialize
Dart objects, or establish typed class/function semantics. Retained
`inspect_analysis_view` pages cover the returned selection; exhausting those
pages does not recover facts that the producing pool scan omitted. Re-run the
producer with the reported selection or budget change to obtain a new
observation, checking its APK and library digests before comparing it.
+31
View File
@@ -0,0 +1,31 @@
# Frida instrumentation
REA exposes Frida device and process discovery, attach or spawn sessions, and caller-authored JavaScript instrumentation through the CLI and MCP. The Node bindings are an optional package dependency and load only when a Frida operation runs. REA does not install Frida server on a target.
For remote devices, pass a host with an optional port (`host[:port]`; bracket IPv6 addresses) and any token, certificate, origin, or keepalive options on that operation. The keepalive interval is measured in seconds, matching Frida Core 17.23.2. Authentication values are held only for the active call or session and are excluded from Evidence. Do not embed credentials in the endpoint address.
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
`instrument-with-frida --json` returns the Evidence record directly, matching MCP. Consumers of the earlier `{evidence, cleanup_error}` CLI wrapper should read the record at the JSON root and the session cleanup observation at `normalized_result.cleanup_error`. If final provider cleanup fails or cancellation arrives during it, the typed error retains completed Evidence in `details.partial_observation`. Discovery commands retain their device or process report in the same field.
For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply.
A summary's selected fields look like this (excerpt):
```json
{
"summary": {
"state": "running",
"messages_retained": 50,
"messages_truncated": false
}
}
```
Use `get_evidence_bundle` with `detail="summary"` and `filters={"evidence_id":"PARENT_ID"}` to discover parent metadata without message payloads. Filters apply only to summary delivery. Use `export_evidence_bundle` for the complete canonical bundle, then select the record whose `evidence_id` matches `normalized_result.parent_evidence_id` from the status summary. To use the existing CLI `inspect-analysis-view`, extract that record into a standalone Evidence JSON file and select it as an Evidence-file source with `view={"kind":"summary"}`. Reading retained Evidence does not refresh a live session or verify a process after exit or PID reuse.
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released.
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Local device lookup waits 1000 ms. This release has no runtime override for that lookup; another REA timeout does not raise it. Remote endpoints do not use that lookup. Message capture is bounded to 16 MiB and reports truncation. Metadata, type, and text field caps, and the JSON node cap, are also fixed. Evidence names a cap when that observation actually dropped data. This release has no runtime override for those capture caps. Script source is not stored in Evidence; its selected path and SHA-256 digest are. For a file, `source_sha256` identifies the original selected bytes. Invalid UTF-8 continues to use replacement decoding; Evidence then records that limitation and `raw_result.source_representation` with the encoding, decoding mode, and `loaded_source_sha256` of the UTF-8 string passed to Frida. Valid UTF-8 files and inline scripts do not need this additional representation fact. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. If cancellation is observed when the native call returns, partial Evidence retains the source identity and captured messages even if that call rejects, including messages emitted during awaited cleanup. The caller's cancellation reason remains primary and a native rejection is retained as a sanitized diagnostic. A failure already observed before cleanup stays primary if cancellation arrives during cleanup. The cancellation cutoff is REA's observation of native completion, rather than a claim about unobservable native event order. `script_id` is REA's load-attempt identity. When creation rejects without returning a handle, `script_cleanup: "no_handle"` means REA acquired no handle to unload; it does not prove that the engine performed no work. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
The [Frida Core API](https://frida.re/docs/frida-core-api/) documents device management, sessions, and script loading; the [JavaScript API](https://frida.re/docs/javascript-api/) documents target-side instrumentation. [Luma](https://luma.frida.re/) is Frida's standalone GUI; REA does not automate or control it. `npm run verify:frida` requires a Linux host with the optional Frida binding and uses a short-lived local Node.js fixture created by the verifier. Add `-- --address HOST:PORT --pid PID` to run the separate remote attach lane against a caller-selected target. Mock tests do not establish real device/server compatibility.
+2 -2
View File
@@ -1,7 +1,7 @@
# GameCube DOL analysis through Ghidra
This lane imports 32-bit big-endian PowerPC GameCube/Wii DOL images through
Ghidra. It does not execute the image. DOL has a section and entry-point header but no magic
Ghidra. It does not execute the image. DOL is a headerless format with no magic
number, so REA never detects it: the caller must select it explicitly.
Only the cached MEM1 window `0x80000000..0x81800000` is admitted. Wii MEM2
@@ -29,7 +29,7 @@ MCP, with the existing `open_binary` tool:
}
```
Selecting the DOL format supplies the architecture (`powerpc`). Do not pass
The architecture comes from the DOL header (`powerpc`). Do not pass
`architecture` with `gamecube-dol`; it is rejected. `entry_point` is optional,
and if given it must equal the header entry.
@@ -175,9 +175,8 @@ are separate from the initialized MZ module. Headerless COM requires the
from the filename or unrecognized bytes. DOS extenders without an admitted
MZ real-mode entry remain outside this import boundary.
DOS MZ is available through Ghidra, with no Hopper support claim. Windows Ghidra
P0 admits DOS MZ read-only: function annotations and naming edits are refused
there, and explicit COM is not admitted. macOS x64 DOS verification remains unverified.
DOS MZ is available through Ghidra, with no Hopper support claim. It is outside
the Windows native PE P0 boundary. macOS x64 DOS verification remains unverified.
## Real verification
@@ -79,7 +79,7 @@ separate ABI flags record, FP attribute values are admitted only when absent,
recordless profile. Parsed values are retained in the MIPS analysis profile.
A carved ELF whose section header table extends beyond the captured file is
refused when the target is opened, before provider selection, as is every ELF
declaration beyond EOF ([installation](../getting-started/installation.md#ghidra)). The refusal
declaration beyond EOF ([installation](installation.md#ghidra)). The refusal
names the declared range and the file size. It applies even when a
program-header ABI record was captured: an uncaptured section header could
carry an unsupported or conflicting declaration, so the captured bytes are not
@@ -8,7 +8,7 @@ labels, functions, or memory bytes. The hydration semantics are adapted from
`effeb734fc570c32650f88b159608979dc7b423e` (MIT), cross-checked against
.NET's `MethodTable.h` at runtime commit
`a3b2d40328be0be3f8dd43f0e5cc925b8827b044` (MIT). The required notice ships in
[`licenses/nativeaot-parser-MIT.txt`](https://github.com/morluto/rea/blob/main/licenses/nativeaot-parser-MIT.txt).
[`licenses/nativeaot-parser-MIT.txt`](../licenses/nativeaot-parser-MIT.txt).
The optional headless Ghidra adapter remains available for database annotations
and broader recovery. It uses the same pinned upstream commit as an opt-in
@@ -158,7 +158,7 @@ does not imply that ordinary native disassembly/decompilation is impossible.
## Verification
See [the dedicated lane](../development/testing.md#optional-nativeaot-ghidra-analysis).
See [the dedicated lane](testing.md#optional-nativeaot-ghidra-analysis).
Fixtures are source-built benign .NET programs with inheritance, an interface,
virtual dispatch, representative arithmetic and frozen strings. Independent
compiler/linker symbols and raw directory bytes provide the oracle. Stripped
@@ -2,28 +2,19 @@
REA imports one target into an ephemeral Ghidra database. Linux/macOS support
read-only analysis and atomic function-name/entry-comment edits; Windows P0 is
read-only. GUI controls and writes to caller-selected source databases are unavailable.
read-only. GUI controls and persistent database mutation are unavailable.
Executable bytes remain unchanged.
An opt-in [owned Program checkpoint](../getting-started/installation.md#automatically-checkpoint-an-owned-ghidra-program)
retains analysis across Linux/macOS processes. It saves initial completed analysis
and normal-close metadata changes to `REA_GHIDRA_CHECKPOINT_DIR`; cancellation
and process failure recover only the last successfully saved state. The restored
packed Program's digest is reported with restored observations. Initial analysis
keeps one stable generation through its first save; later saved states bind their
content digest in profile identity. Restoration does not turn prior
database state into a fresh discovery. The default remains ephemeral.
Use [installation](../getting-started/installation.md#ghidra) for supported hosts, prerequisites,
Use [installation](installation.md#ghidra) for supported hosts, prerequisites,
configuration, startup deadlines, resource controls, and cleanup recovery.
[Linux Ghidra launches](../getting-started/installation.md#linux-owned-process-prerequisite) also
[Linux Ghidra launches](installation.md#linux-owned-process-prerequisite) also
require a procps-compatible `ps` on REA's `PATH` for process ownership checks.
[Windows P0](windows-ghidra-p0.md) has its own admission and lifecycle boundary;
[NativeAOT observations](ghidra-nativeaot.md) include a built-in read-only PE
snapshot parser and a separate optional annotation extension. Check the
[release boundary](../getting-started/installation.md#released-package-and-main) when using npm.
[release boundary](installation.md#released-package-and-main) when using npm.
See [import language and analysis seeds](../getting-started/installation.md#ghidra-import-language-and-analysis-seeds)
See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds)
to override the header-detected language or seed analysis. An override that conflicts with a fixed raw, DOS, PSP, MIPS, RISC-V or GameCube DOL import
is a configuration error naming the conflicting settings. It does not make an
otherwise admitted target unsupported; REA preserves the requested settings.
@@ -52,7 +43,7 @@ not clear it or alter session readiness. Log paths refer to the owned ephemeral
runtime and are removed by normal cleanup.
Provider selection and profile identity follow
[MCP contracts](../reference/mcp-contracts.md#identity-and-discovery). One deep
[MCP contracts](mcp-contracts.md#identity-and-discovery). One deep
provider stays bound to the target; runtime failure never selects another
provider automatically. The contracts below describe Ghidra observations,
not equivalence with another engine's pseudocode or analysis database.
@@ -89,47 +80,6 @@ does not prove text is absent. Both list and search Evidence carry this
limitation. When an address is known, use `read_bytes` and `xrefs` to inspect
bytes and references.
String searches and `trace_feature` retain each matched row's Ghidra-reported
encoding, termination and byte length. These describe the database's defined
data, not independent proof of the original source encoding. Procedure searches
and trace matches likewise retain Ghidra's external and thunk classification.
The Ghidra analysis profile identifies this search projection, so older
snapshots with unqualified search rows cannot replay under the new profile.
## Decompiler and value-flow budgets
Use `inspect_analysis_settings` or a result's named recovery to inspect
`ghidra.value_flow_max_operations` (default 3000),
`ghidra.value_flow_max_inputs_per_operation` (default 64), and
`ghidra.value_flow_max_edges` (default 12000). Each accepts a positive Java
integer count. `update_analysis_settings` applies changes to the next operation
on the same Program without restarting REA. Launch seeds use the corresponding
`REA_GHIDRA_VALUE_FLOW_MAX_OPERATIONS`,
`REA_GHIDRA_VALUE_FLOW_MAX_INPUTS_PER_OPERATION`, and
`REA_GHIDRA_VALUE_FLOW_MAX_EDGES` environment settings.
For a one-shot CLI retry, set the named seed in the environment that launches
the next `rea` invocation; the completed invocation's Program is no longer
active. Runtime MCP changes preserve the current Program. A changed launch seed
only affects a running MCP server after that server restarts.
The returned `native_value_flow.limits` and `limits_reached` distinguish accepted
budgets from actual exhaustion. Truncation keeps omitted counts and names the
setting to adjust. The edge budget applies independently to def-use and
parameter-use collections. New function dossiers are reobserved rather than
replayed from snapshots; completed Evidence remains available for historical
inspection. Other observation caches keep their existing policy.
`ghidra.decompile_max_payload_mib` controls the native decompiler's response
payload, separately from p-code projection and decompile time. Its launch seed
is `REA_GHIDRA_DECOMPILE_MAX_PAYLOAD_MIB`; explicit values are 1..2047 MiB to
avoid overflow in Ghidra's signed integer byte conversion. Omission or runtime
`null` preserves the installed provider's default. Changes apply before the
next native decompile, and native option changes clear its old cached function
state. A failed decompile preserves the original diagnostic and reports the
effective payload allowance; payload recovery applies only when that diagnostic
identifies payload exhaustion. Raising these controls does not raise the MCP
client's deadline or receive buffer.
## PE debug-symbol coverage
The post-analysis handshake reports Ghidra's PDB Program Information attributes:
@@ -174,11 +124,6 @@ also accept source-file byte offsets. Existing symbols and analysis addresses
take precedence; a `0x`-prefixed number that resolves to no procedure falls back
to its observed source mapping. Use `--file-offset=0x...` on the CLI or
`procedure: "file-offset:0x..."` through MCP to select file coordinates explicitly.
To select an analysis address without the bare-hex fallback, qualify it with its
Ghidra address space, for example `ram:0x...`. The `inspect-native-load-image`
CLI command or MCP `inspect_native_load_image` reports Ghidra's observed default
address space; use the space appropriate to the address, and retain any
load-image verification limitations.
Multiple mapping candidates fail as ambiguous rather than selecting a bank or
address space. A successful conversion retains the file offset, mapped address
and procedure entry in raw Evidence and `locations`.
@@ -301,7 +246,7 @@ profile stages before retrying their retained descriptors.
| Function identity | Every function result carries the entry address and Ghidra FunctionManager classification for external, thunk, and resolved thunk target. These are observations, not proof that unresolved targetless calls have been recovered. |
| Assembly and pseudocode | Assembly is complete Ghidra Listing text; pseudocode is Ghidra decompiler output. Neither is original source, and cross-provider comparison never treats Hopper and Ghidra text as equal or unequal semantic facts. |
| Instruction fast path | `read_function_instructions` returns every raw Listing instruction for the requested function without invoking the decompiler or whole-program name/string inventories. |
| Calls and references | Callers/callees contain only resolved functions. Reference edges preserve exact ReferenceManager type and call/jump/data/read/write/indirect/computed/conditional/terminal/external facts. Targetless computed flow remains unknown. A computed call's recorded targets stay observed callees, but they are static candidates rather than an exhaustive set, so callee coverage is `partial` and names those call sites. Synthetic entry-point references without actionable memory sources are omitted explicitly. |
| Calls and references | Callers/callees contain only resolved functions. Reference edges preserve exact ReferenceManager type and call/jump/data/read/write/indirect/computed/conditional/terminal/external facts. Targetless computed flow remains unknown. Synthetic entry-point references without actionable memory sources are omitted explicitly. |
| CFG | Dossiers use `BasicBlockModel` and retain only non-call successors inside the function body. CFG topology is address-normalized for comparison; provider-specific block construction remains a declared difference. |
| P-code value flow | Function dossiers include up to 3,000 high-p-code operations, at most 64 inputs per operation, and 12,000 def-use edges. Results identify memory reads/writes, branches, and calls by p-code opcode, and mark decompiler-dead operations. Large results report truncation, an omitted-operation lower bound, and known omitted-input/edge counts. The dossier is an intra-function graph; `trace_native_values` composes derived parameter/argument and return/output dependencies across resolved calls. It does not resolve memory aliasing, call side effects, runtime behavior, persistent state, or business meaning. |
| Result extent | Function instruction scans and native API boundary observations remain complete. P-code flow is the explicit bounded exception; its count fields distinguish exact retained counts from lower-bound or known-omitted counts. Caller cancellation and provider failures remain distinct. |
@@ -310,21 +255,9 @@ Runtime updates apply to newly accepted operations on the existing Program.
They preserve accepted budgets and do not clear cleanup blockers or change the
client deadline. Transient decompile timeout failures remain uncached so the
same function can be retried after raising the limit. See the
[settings design](../design/limits-and-recovery.md) for snapshots, precedence,
[settings design](design/limits-and-recovery.md) for snapshots, precedence,
and provider integration.
When `inspect_native_instruction` selects a byte inside a listed instruction,
`containing_instruction` reports its separate start, length and Ghidra Listing
text, retaining any non-default address space. The requested address remains
unchanged, and a containing function may be absent. For defined data, the same
tool reports its containing object's start, length and type path; use the start
with `inspect_native_data_type` to inspect its layout. This does not select a
field or union member. These are listing observations, not proof of execution or
an alternate decode. Earlier Evidence can
omit the field. The analysis-profile revision prevents older snapshots from
replaying results without it; open the target without the old snapshot and save
a fresh snapshot when needed.
Function extents and instruction completeness refer to Ghidra's analyzed body.
An inferred no-return call can terminate that body before subsequent bytes.
Pseudocode, assembly, procedure info, instruction windows, incoming and outgoing
@@ -393,22 +326,13 @@ must be constrained.
Optional `start` and `end` addresses are inclusive and select one address space.
A match must end at or before `end`, so `end` set inside a match excludes it.
Without bounds, loaded non-overlay memory blocks are eligible, in Ghidra address
order, including across adjacent initialized blocks. Blocks that are not loaded
(file parts such as `.symtab`) are listed in `searched.skipped_not_loaded`;
loaded overlay blocks are listed separately in `searched.skipped_overlays`.
Neither is searched by default; pass `start` or `end` in its address space to
search it. A block that is both unloaded and an overlay is listed once under
`skipped_not_loaded`.
The separate overlay field changes the byte-search result contract and the
Ghidra analysis profile. Refresh cached MCP schemas; snapshots from the older
profile are not replayed. Previously exported Evidence remains importable as
historical data with its original result shape.
Overlapping matches are included, including across adjacent eligible initialized
blocks in the same address space. Gaps and excluded or uninitialized blocks break
a search run. Each match reports its address, starting segment, unique source file
offset for that starting address (or null), and containing procedure (or null).
A starting offset does not imply the full match is contiguous in the file.
Without bounds, loaded memory blocks are eligible, in Ghidra address order.
Blocks that are not loaded (file parts such as `.symtab`) and overlay blocks are
listed in `searched.skipped_not_loaded` and are not searched; pass `start` or
`end` in their address space to search them.
Overlapping matches are included; matches never cross block boundaries.
Each match reports its address, segment, unique source file offset (or null),
and containing procedure (or null).
Pages default to 100 matches and accept 1 to 1000. Pass `next_cursor` as `cursor`
with the same target, profile, pattern, mask and bounds to continue. `complete`
@@ -93,15 +93,8 @@ The maintained PSP fixture builds from `tests/conformance/psp/`, never executes,
and is independent of commercial game data. PSP GNU tools supply ELF/ABI,
raw ABI-record, symbol and encoding observations; REA's parsed target and saved
snapshot must agree with the independent ABI declaration. Public CLI/MCP must
recover known functions, a direct call, the immediate `0x1234`, BITREV, a
global word and a marker. The additional source-owned VFPU function contains
`vpfxs`, `vpfxt`, `vpfxd` and three `vadd.q` instructions.
The PSP lane checks their fixed encoded bytes and Ghidra's decoded mnemonics
through `inspect_native_instruction`; an analyzed dossier reports available
high-p-code opaque-operation and truncation observations. Neither decoded
instructions nor `CALLOTHER` counts establish prefix consumption, lane-level
arithmetic or emulator/hardware equivalence.
The verifier
recover known functions, a direct
call, the immediate `0x1234`, BITREV, a global word and a marker. The verifier
checks observed loader/language, original bytes, Evidence, snapshot reopening,
and an A-B-A target switch with different global contents at the same address.
A snapshot for A must not open B. Cleanup must finish before success is reported.
@@ -265,7 +265,7 @@ definitions use it.
Ghidra 12.1.4 PUBLIC with a 64-bit JDK 21+ is required, and on macOS the release
tree ships no host decompiler binary, so `Decompiler/os/<platform>/decompile` and
`sleigh` must come from your own build; see
[installation](../getting-started/installation.md#ghidra).
[installation](installation.md#ghidra).
Run `npm run verify:ghidra:riscv` with an existing Ghidra 12.1.4 installation,
JDK 21+, and a RISC-V-capable Clang, LLD, LLVM readelf and LLVM objdump. The
@@ -49,16 +49,14 @@ selected dependency versions without reopening the file:
}
```
Go views support `dependencies` (exact `paths`), `settings` (exact `keys`) and,
when collected with `include_string_literals: true`, `string_literals` (exact
`values` and substring `contains`), plus optional original `indices`, field
selection, and pagination. Filters intersect; empty filter arrays select no rows
and `fields: []` returns only row identity. Each row keeps its original `index`,
scoped to the parent Evidence and collection; duplicate module paths and setting
keys keep distinct indices. Page offsets count matching records. Exhausting
retained rows does not establish complete runtime dependencies or string
coverage. Original bytes, unknown lines and all complete metadata remain in the
parent and can be exported as an Evidence bundle.
Go views support `dependencies` (exact `paths`) and `settings` (exact `keys`),
optional original `indices`, field selection, and pagination. Filters intersect;
empty filter arrays select no rows and `fields: []` returns only row identity.
Each row keeps its original `index`, scoped to the parent Evidence and collection;
duplicate module paths and setting keys keep distinct indices. Page offsets
count matching records. Exhausting retained rows does not establish complete
runtime dependencies. Original bytes, unknown lines and all complete metadata
remain in the parent and can be exported as an Evidence bundle.
One-shot CLI summaries save complete Evidence before returning the projection:
@@ -115,26 +113,6 @@ their empty dependency array does not establish the absence of dependencies.
The result's limitations state this explicitly. A module-mode build with a main
module record and no dependency records has an observed empty dependency list.
When `include_string_literals: true` (CLI `--include-string-literals`, optional
`max_string_literals` / `--max-string-literals`, default `2000`)
is selected, `string_literals` reports `evidence_kind: "inferred"` alongside
deduplicated printable UTF-8 literals referenced by word-aligned two-word
`(ptr, len)` string headers in non-executable initialized data. Because Go packs
read-only string data contiguously without NUL terminators, slicing by `(ptr, len)`
recovers exact literal boundaries instead of concatenated C-string runs. Each
target slice (default `1`–`4096` bytes) must stay within a single read-only mapped
section or segment, decode as printable UTF-8 (`\t`, `\n`, `\r`, joiners
`U+200C` / `U+200D` and printable Unicode scalar values, excluding control and
other format characters), and fit within
a default 64 MiB header-scan budget and 512 KiB aggregate string budget
(`truncated: true` when capped). Select `max_string_literal_bytes`,
`max_string_bytes`, `max_string_header_scan_bytes`, or `max_string_candidates`
to change these independent budgets. Their CLI flags use hyphens, for example
`--max-string-literal-bytes 16384`. `limits_reached` names exhausted controls;
increase those controls for a new inspection of the same complete artifact. Compiler-inlined or
stack-constructed strings without static headers, unapplied relocation addends
and Mach-O `LC_DYLD_CHAINED_FIXUPS` pointer encodings are noted in `limitations`.
## Supported boundary
The built-in TypeScript reader supports ELF, PE and thin Mach-O images, and one
@@ -205,39 +183,23 @@ host permission denial, changed artifacts and cancellation retain distinct
errors. The selected file is never modified; this inspection creates no
temporary files, starts no subprocesses and accesses no network endpoint.
The existing default resource budgets are now explicit per-call controls:
An input above 256 MiB returns `resource_constraint` with resource `file-size`
and `reported_limits.maximum_input_bytes`. Select a suitable input within that
fixed allowance; changing an output-write or transport budget does not admit it.
| Parameter | Default | What it admits |
| ------------------------------ | ----------- | ------------------------------------------------------------- |
| `max_input_bytes` | 256 MiB | Complete stable input file |
| `max_report_bytes` | 8 MiB | Complete normalized metadata representation |
| `max_build_info_bytes` | 1 MiB | Aggregate embedded string contents, excluding length prefixes |
| `max_header_search_bytes` | 256 MiB | Cumulative build-info search work |
| `max_string_literals` | 2,000 | Returned inferred string rows |
| `max_string_literal_bytes` | 4,096 bytes | Each inferred string candidate |
| `max_string_bytes` | 512 KiB | Aggregate returned inferred text |
| `max_string_header_scan_bytes` | 64 MiB | String-header and candidate-byte examination |
| `max_string_candidates` | 100,000 | Distinct inferred candidates retained before projection |
Every value is a positive safe integer. Input and string byte maxima follow the
runtime Buffer/string representation, with base64 expansion accounted for in
build-info admission; schemas advertise their exact maxima. CLI options use the same names with
hyphens, such as `--max-build-info-bytes 2097152`. Inferred-string controls
require `include_string_literals: true`. Omitted controls keep the defaults
and stay omitted in request Evidence. Error recovery names the exhausted
parameter and effective bytes; increase that parameter on the next call,
without restarting REA. The original complete artifact and its identity are
preserved. An unchanged retry cannot increase a deterministic allowance.
The shared native container reader separately bounds structural table decoding
to 16 MiB. None of these Go controls changes that boundary. Inline strings must
also fit their actual mapped source range. Identical search spans with the same
virtual alignment are searched once; partially overlapping spans consume the
cumulative search budget. Oversized complete reports fail without partial
success and without allocating the full JSON serialization. MCP's separate
[response transport budget](../../reference/mcp-contracts.md) can still require
export or paging of successfully retained Evidence. These controls do not
establish operating-system memory or CPU isolation.
Resource guards bound the selected file and cumulative header searches to 256 MiB
each, aggregate embedded string contents to 1 MiB (varint length prefixes are
framing and do not count toward this allowance), structural table decoding to
16 MiB and the complete normalized metadata report to 8 MiB. Inline strings must
also fit within their actual mapped source range. Identical search spans with
the same virtual alignment are searched once; partially overlapping spans
consume the cumulative search budget
and fail with a CPU resource constraint when it is exhausted. Oversized results
fail without partial success. MCP also applies
its [response transport budget](mcp-contracts.md), which counts both text and
structured content; a report within the metadata limit can still require export
from retained Evidence. These limits are not operating-system memory or CPU
isolation.
Embedded metadata is evidence of file content. It does not authenticate a
compiler, dependency, commit or supply chain, and it does not establish runtime
+1 -53
View File
@@ -25,33 +25,6 @@ by default; `--integrity-policy record-and-continue` retains the observed bytes
and an explicit untrusted contradiction. An all-zero MD5 is an unknown claim,
not a verified digest.
Artifact comparison pairs members by `logical_path`, the stored path without
`res://`, and compares their bytes and occurrence facts. It does not compare
`godot-pck-entry` metadata. Godot 4.3
[stores `res://` paths](https://github.com/godotengine/godot/blob/77dcf97d82cbfe4e4615475fa52ca03da645dbd8/editor/export/editor_export_platform.cpp#L228)
and [keys members by them](https://github.com/godotengine/godot/blob/77dcf97d82cbfe4e4615475fa52ca03da645dbd8/core/io/file_access_pack.cpp#L50-L52).
Godot 4.4.1
[stores paths without the prefix](https://github.com/godotengine/godot/blob/49a5bc7b616bd04689a2c89e89bda41f50241464/editor/export/editor_export_platform.cpp#L257-L260)
and [removes it before keying](https://github.com/godotengine/godot/blob/49a5bc7b616bd04689a2c89e89bda41f50241464/core/io/file_access_pack.cpp#L48-L50).
Each version finds `res://scenes/main.gd` in its own export, but comparing the
stored `source_path` would mark every member changed between such exports.
`logical_path` names the member Godot's lookup finds, except in two cases.
REA normalizes Unicode to NFC, so a member renamed only between Unicode forms
compares unchanged, although Godot treats it as another path. Godot reads `\`
as `/` and REA keeps it literal, so a member spelled with backslashes compares
as removed and added. Directory order and offsets are layout. The encryption
bit, the only member flag REA admits, is the occurrence's `encrypted` field. A
member whose encryption changes compares `unknown` in `metadata` and
`availability`: the change is known, but REA does not read encrypted bytes,
so whether the script changed is unknown. Godot's exporter
[stores the MD5 of a member's original bytes](https://github.com/godotengine/godot/blob/a13da4feb8d8aefc283c3763d33a2f170a18d541/editor/export/editor_export_platform.cpp#L465-L468),
as the 4.3 and 4.4.1 exporters do, so the declared MD5 does not depend on the
encryption IV. Encrypted members whose declared MD5s differ therefore also
compare `unknown` in `metadata` and `availability`: the MD5s declare
different scripts, but REA cannot verify them, and equal MD5s add nothing for
the same reason. An encrypted member with an MD5 on only one side stays
`unknown`.
## CLI
Start with a compact summary and keep the complete Evidence bundle:
@@ -144,34 +117,9 @@ keys, decode Godot resources, decompile GDScript bytecode, analyze signals,
apply patch/removal entries or execute targets.
Readable logical member bytes, including repeated physical ranges, share the ZIP
archive expansion policy by default: at most the larger of 256 MiB or 500 times the PCK's
archive expansion policy: at most the larger of 256 MiB or 500 times the PCK's
encoded byte span. Embedded carrier bytes do not enlarge that budget. The full
directory is checked before any member is yielded, hashed or extracted. Encrypted
members are not read and do not consume this plaintext-work allowance. Small
bounded aliases remain valid. This is a processing/output resource bound, separate
from the directory metadata retention budget and format validity.
Directory retention defaults to 64 MiB of encoded metadata. Select
`artifact_limits.godot.directory_bytes` to change it; path/source facts reserve
worst-case JSON expansion before declared path bytes are read, and the reader
then charges actual retained metadata. A preallocation `required_value` is the
allowance needed by that admission check, not a measurement of final output size.
Select `artifact_limits.expanded_bytes` independently to change aggregate
plaintext work, including repeated readable logical occurrences. Both controls
apply to inspection and extraction, before any member is yielded.
For example:
```sh
rea inspect-artifact ./game.pck --artifact-limits '{"godot":{"directory_bytes":134217728},"expanded_bytes":1073741824}' --json
```
MCP uses the same object under `artifact_limits` on `inspect_artifact` or
`extract_artifact`. A refusal reports the exact `reader_limit` parameter,
effective value, unit, supported range and required value when known. Raise the
named admission and retry the same complete PCK on the next call; no REA restart
is needed. Summary or page size does not change producer admission. Omitted
policy selections stay omitted in Evidence parameters; a selected object records
the accepted selection and `effective_limits`, conditional on which readers are
encountered. Raising these policies does not admit malformed offsets, unsupported
flags, unsafe extraction paths or unavailable encrypted contents.
-121
View File
@@ -1,121 +0,0 @@
# V8 code cache inspection
`inspect_v8_code_cache` reads one explicitly selected V8 code cache container as static
bytes: a bytenode `.jsc`, the bytes `vm.Script.createCachedData()` writes, or a cache
carried inside a Node single-executable application or an Electron app. It never executes
the file, never starts a V8 runtime, never installs bytenode, and never invokes a
decompiler.
```bash
rea inspect-v8-code-cache /absolute/path/to/module.jsc --json
rea inspect-v8-code-cache /absolute/path/to/module.jsc --minStringBytes 8 --json
```
The MCP tool is `inspect_v8_code_cache` with the same inputs and the same retained
Evidence.
## What is read
A V8 code cache is a producer-written container, not a documented file format. This reader
admits one container shape and reports everything else as an observation, so a build it
has never seen still yields facts instead of a false parse.
The 32-byte header is eight little-endian 32-bit words. Every word is reported in
`header.fields` with its byte offset, its observed value, the role this reader believes it
has, and how confident that role is:
| Offset | Role | Status |
| -------------- | -------------- | ----------- |
| `0x00` | magic | confirmed |
| `0x04` | version hash | inferred |
| `0x08` | source length | confirmed |
| `0x0c` | flag hash | inferred |
| `0x10` | build constant | inferred |
| `0x14` | payload length | confirmed |
| `0x18`, `0x1c` | reserved | unconfirmed |
`confirmed` means the role is demonstrated by the reader's own tests against the producer.
`inferred` means the word's behaviour is consistent with the role but a different producer
could plausibly differ. `unconfirmed` means the word is reported and nothing more is
claimed.
Only three facts gate admission:
1. the header size this reader pins (32 bytes);
2. the V8 code cache magic high half, `0xc0de`. The low half is
`ExternalReferenceTable::kSize` and varies per V8 build, so it is reported as a build
fingerprint and never required to match a fixed value;
3. the declared payload length must equal the bytes that follow the header exactly.
`header.source_length` is the producer's recorded source length in UTF-16 code units.
V8's serializer calls this field the source hash; on the producers measured here its value
equals the source length exactly, including sources whose UTF-8 byte length differs, so the
reader reports it under the name the measurement supports. Treat that name as an
observation, not as a claim about what any other build stores there.
`header.producer` names the producer family from the flag bits in that word. Bit 29
marks `vm.compileFunction` (`compile_function`); bit 30 marks `vm.SourceTextModule`
(`module`). When neither flag is present, the producer remains `unknown`:
`vm.Script`, bytenode output and older unflagged function caches cannot be
separated by those bits alone. The source-length mask is `0x1fffffff`, so both
flags sit outside the reported length. This describes the admitted header
profile rather than a version-independent V8 ABI.
## Failed inspection and provenance
After a stable read completes, format and observation-budget refusals retain a
`file-snapshot-hash` partial observation with the selected path, exact SHA-256 and
byte count. This proves which bytes were read; it does not claim the format was
successfully decoded. A refusal before read completion does not invent an identity.
Descriptor cleanup remains owned for retry if finalization fails.
## Embedded byte patterns
`embedded_strings` lists printable runs found in the payload, each with its absolute byte
offset in the selected file and its observed byte length. `min_string_bytes` (default 4)
sets the shortest reported run; `max_strings` caps how many may be reported, and exceeding
it fails without partial success.
These are **byte-pattern observations inside serialized V8 heap data, not parsed heap
objects**. A reported run is not proof V8 retained it as a string. A literal that is absent
from the payload is not proof it is absent from the program: V8 may encode a string in a
form the run scan does not surface, and a string can be produced at runtime rather than
retained.
A run longer than 4096 bytes is reported with `truncated: true` and a `retained_bytes`
prefix.
## Limitations
Every result states these, and they are part of the contract:
- the payload is serialized V8 heap data, not source; no logic is recovered and no
bytecode is decompiled;
- recovering logic needs a version-pinned V8 decompiler (View8-style) that this tool
neither provides nor depends on;
- header roles beyond magic, source length, and payload length are version-dependent
observations;
- the tool never executes the selected file and never claims recovered source.
## Failure behaviour
An unknown magic is a typed `unsupported_target` failure, because the file is not a V8
code cache. A short header, a payload-length disagreement, or bytes past the declared
payload is a typed `invalid_request` failure. Neither produces completed inspection
Evidence; the stable-read partial observation described above is retained when available.
`max_file_bytes` (default 32 MiB, ceiling 64 MiB) is the caller-selected read budget. The
payload SHA-256 is computed over the declared payload, so two containers with the same
payload length and content but different version hashes have different whole-file digests
and identical payload digests.
## Relationship to other tools
- `inspect_binary_layout` reads the surrounding ELF/PE container of a single-executable
application; it does not decode the embedded code cache.
- A cache embedded in a Node single-executable application or an Electron app must be
**extracted first**: this reader admits a container whose magic is at byte zero, so
passing the surrounding ELF/PE executable here returns `unsupported_target`. Run
`extract_artifact` on the application, then pass the extracted `.jsc` path to this tool.
- `inspect_artifact` inventories the file without decoding V8 structures.
- This tool is in the same family as `inspect_python_pyc`, `inspect_erlang_beam`, and
`inspect_kernel_module`: one explicit local file, bounded parsing, no execution.
-177
View File
@@ -1,177 +0,0 @@
# ELF unwind function ranges
`inspect_elf_unwind_ranges` / `inspect-elf-unwind-ranges` reads `.eh_frame`
and `.eh_frame_hdr` from one selected local ELF32/ELF64 image and lists the
function ranges those tables name. It reads the file without executing it and
does not require a disassembler.
```bash
rea inspect-elf-unwind-ranges /absolute/path/to/program --json
```
The CLI resolves relative paths against the caller's working directory. MCP
requires an absolute filesystem path and preserves that exact selection:
```json
{
"name": "inspect_elf_unwind_ranges",
"arguments": { "path": "/artifacts/program" }
}
```
Pass `--offset` and `--limit` (MCP `offset` and `limit`) to page the ranges.
The default page starts at 0 and returns at most 256 ranges.
## Observations
The inline Evidence result contains the artifact path, SHA-256, and byte
length, plus ELF class, byte order, machine, and object type.
`eh_frame.status` is `present`, `absent`, `incomplete`, or `unknown`. A
present table names its source: `section` when section headers name
`.eh_frame`, or `program` when section headers are missing and
`PT_GNU_EH_FRAME` supplies the bytes. `location` is the file range that was
examined. A `.eh_frame` section that is `SHT_NOBITS` or lies outside the file
is `incomplete` with a null `location`. One that runs past the end of the
file is `incomplete`, `location` is clamped to its in-file bytes, and the
records in those bytes are still decoded. Only the first `.eh_frame` and
`.eh_frame_hdr` section is decoded, and only the first `PT_GNU_EH_FRAME`
program header. A duplicate section, or a duplicate `PT_GNU_EH_FRAME` when no
`.eh_frame_hdr` section supplies the header, keeps `coverage.status` at
`incomplete`.
`eh_frame_hdr` reports version, the three pointer encodings, the `.eh_frame`
pointer, the declared FDE count, and how many binary-search table entries
were decoded. A header section that is `SHT_NOBITS` or extends past the end of
the file is `incomplete` with a null `location` and is not decoded.
`coverage.eh_frame_hdr_agreement` is `agree` when the table has exactly one
row per decoded FDE, the sorted starts match including repeats, and each row
points at a distinct FDE with that start. It is `disagree` otherwise, and
`unavailable` when the header is absent, unknown, or incomplete. Disagreement
is a limitation. The decoded FDEs stay in the result. The search table is
read for at most `max_ranges` rows, and for no more rows than the header bytes
can hold. When the declared FDE count is larger than either bound, the header
is `incomplete`, `table_entries` counts only the rows read, the comparison is
`unavailable`, and a limitation names the cap. A declared count that does not fit
in 32 bits is not decoded: `fde_count` is null, the header is `incomplete`,
and the decoded FDEs stay in the result. A header whose FDE count encoding is
`DW_EH_PE_omit` (`0xff`), as GNU ld writes when it cannot build the search
table, is `present` with a null `fde_count`, no table rows, and comparison
`unavailable`; it does not make coverage incomplete. A header that declares
an FDE count but writes `DW_EH_PE_omit` as its table encoding is `present`
with that count, no table rows, and comparison `unavailable`, and also keeps
coverage complete. A non-zero FDE count under an indirect encoding is the
address of a slot that is not followed, so `fde_count` is null, the header is
`incomplete`, and the table is not read.
When section headers name both `.eh_frame_hdr` and `PT_GNU_EH_FRAME`, the
section is decoded. A limitation names the segment as ignored only when it
declares different bytes than the section. When no `.eh_frame_hdr` section is
named, the header is read from `PT_GNU_EH_FRAME` even if a `.eh_frame`
section is present.
A record whose length word is `0xffffffff` uses the 64-bit extended length
that follows it. In `.eh_frame` the CIE ID and CIE pointer after that length
stay 4 bytes wide.
`DW_EH_PE_datarel` fields in `.eh_frame_hdr` are relative to the header
address. In `.eh_frame` they are relative to the GOT: `DT_PLTGOT` when the
dynamic table names it, otherwise the `.got.plt` or `.got` section address.
A limitation names the base that was used. When no GOT base is declared,
those values are unresolved: the affected record is not reported and
`coverage.status` is `incomplete`.
`debug_frame.status` is `present`, `absent`, `incomplete`, or `unknown`. It
is `unknown` when the image has no section headers or unreadable section
names. It is `incomplete`, with a null `location` and `coverage.status` at
`incomplete`, when the section is `SHT_NOBITS` or its bytes are not all in
the file. The `.eh_frame` results stay in the report. This command does not
decode `.debug_frame`.
`cies` lists the CIEs that the returned page's ranges name, in index order,
so its length never exceeds the page size. Each `cie_index` in `ranges`
matches the `index` of one entry in `cies`. `coverage.total_cies` counts every
decoded CIE, including CIEs no FDE on this page uses. Each CIE reports
version 1 or 3, augmentation, code and data alignment,
return-address register, pointer encoding, LSDA encoding, personality address
when the augmentation carries one, signal-frame and B-key flags, and the byte
length of the CFA program. CFA instructions are counted and not evaluated.
A CIE whose augmentation has a letter other than `z`, `L`, `P`, `R`, `S`, or
`B` is not decoded: the encodings of later letters cannot be trusted, so its
FDEs are not decoded either. A limitation counts those FDEs and coverage is
`incomplete`.
`ranges` is one page of FDEs. Each range has `start`, `end`, decimal `size`,
the containing `SHF_ALLOC` section, an exact symbol name when a symbol starts
at that address, the CIE index, LSDA, and personality. A function symbol is
preferred, then an `STT_GNU_IFUNC` symbol, then an untyped one. An IFUNC name
is the resolver that starts at that address, not the implementation it
selects at run time, and a limitation says so. Symbol names, section names,
and augmentation strings are target-derived and untrusted. Indirect EH
pointers are not dereferenced. An FDE whose initial location is indirect names
a pointer slot, not its function, so it is not reported as a range; a
limitation counts those FDEs and coverage is `incomplete`. An indirect
`eh_frame_ptr` does not locate `.eh_frame`, and the header is `incomplete`.
A search table with an indirect encoding is not read: the header is
`incomplete`, the comparison is `unavailable`, and the table does not bound a
header-located `.eh_frame`.
An encoded pointer whose stored value is zero is a null pointer, as libgcc
reads it: no PC-relative, text, data, or function base is added and an
indirect pointer is not followed. A resolved address, and an FDE `end`,
wraps at the ELF address width (32 or 64 bits), as libgcc's pointer
arithmetic does. A zero LSDA pointer is reported as a null
`lsda`, even when the CIE declares an LSDA encoding. An FDE whose initial
location is zero marks a function the linker discarded. It is not reported
as a range, `examined_records` still counts it, and a limitation counts how
many were skipped. In a relocatable object an unrelocated initial location
is also zero, so those FDEs are skipped too.
`page` reports `offset`, `limit`, `returned`, `total`, and `next_offset`.
`next_offset` is null when the page reaches the end of the decoded list.
`coverage` counts every decoded range, every decoded CIE, how many ranges
match a symbol, and how many do not. `examined_records` counts CIE and FDE records visited, including
records that could not be decoded.
A missing `.eh_frame` section and a missing `PT_GNU_EH_FRAME` segment are
observed absence of those declarations. That absence is not proof the image
has no functions. Program headers come from the shared ELF section
directory, which rejects an image whose program-header table cannot be read,
so a missing `PT_GNU_EH_FRAME` is always observed absence. When section
headers are missing too, section names are unknown, so a missing header is
`unknown` rather than `absent`.
A `.eh_frame` located only through the `.eh_frame_hdr` `eh_frame_ptr` has no
section size, so it is read up to the end of its `PT_LOAD` segment. Other
data, such as `.gcc_except_table`, can follow it there. When the search table
was decoded, the scan stops after the last FDE the table names, as libgcc's
`PT_GNU_EH_FRAME` lookup only binary-searches that table. Later bytes are not
decoded and a limitation says so. Without a usable table, the scan stops at
the first record that fails to decode, as libgcc's linear FDE search does,
and `coverage.status` is `incomplete`. A CIE with an unrecognized
augmentation letter, or an FDE that uses one, also stops that scan. Reaching the end of the segment with
no terminator also keeps coverage `incomplete`, because the table's extent is
then unknown.
Addresses map to file offsets through `SHF_ALLOC` sections, then `PT_LOAD`
segments. A section without `SHF_ALLOC`, such as `.comment` or `.debug_*`,
is not loaded, so its `sh_addr` is never used for translation and it is never
reported as the section containing a range. A record that ends past the table, or non-zero bytes after
the terminator, keeps `coverage.status` at `incomplete`. Relocatable objects
stay incomplete because relocations are not applied.
## Limits and failures
The selected file is limited to 64 MiB by default (512 MiB maximum).
`max_ranges` defaults to 8192 and caps at 65536. That ceiling applies
separately to CIE records and to FDEs. Exceeding either budget fails without
partial success. One page returns at most 4096 ranges. Symbol lookup stops
after 65536 entries across `.symtab` and `.dynsym` combined. Reaching that
cap keeps `coverage.status` at `incomplete`. Only symbols at decoded FDE
starts are kept, and a name is decoded only for a returned range. The names
on one page share a fixed 16 MiB text allowance; a page that exceeds it fails
with a resource error whose recovery is a smaller `limit`, because
`max_ranges` and `max_file_bytes` do not raise it.
Malformed ELF headers fail without partial success. This workflow does not
execute the image, evaluate CFA programs, or follow indirect pointers.
-151
View File
@@ -1,151 +0,0 @@
# Linux kernel build config
`inspect_kernel_build_config` / `inspect-kernel-build-config` reads the
`Linux version` banner and the embedded `.config` from one selected local
kernel image without executing or booting it. Accepted inputs are a `vmlinux`
ELF, a raw arm64 `Image`, an x86 `bzImage` or `zImage` whose payload is gzip,
and a `configs.ko` module. It does not require a Linux host,
`extract-ikconfig`, or `/proc/config.gz`.
```bash
rea inspect-kernel-build-config /absolute/path/to/vmlinux --json
rea inspect-kernel-build-config /absolute/path/to/bzImage --name CONFIG_SECURITY_SELINUX --json
```
The CLI resolves relative paths against the caller's working directory. MCP
requires an absolute filesystem path and preserves that exact selection:
```json
{
"name": "inspect_kernel_build_config",
"arguments": { "path": "/artifacts/vmlinux", "name": "CONFIG_MODULE_SIG" }
}
```
## Observations
The inline Evidence result contains the artifact path, SHA-256, and byte
length. `format` is `elf` when the file starts with the ELF magic and `file`
otherwise.
`banner` is the first printable string that starts with `Linux version `
followed by a version token beginning with a digit and ends at a NUL or
newline within 512 bytes. `version` is that token. `builder` is the first
parenthesized group (`user@host`). `compiler` is the second group up to the
last `) #`, so nested parentheses in compiler strings are kept. The banner is
`absent` when no such string was found in the searched bytes and `unknown`
when the bytes were not searched.
`config` is `present` only when all of the following hold:
- `IKCFG_ST` is immediately followed by a gzip member (`1f 8b 08`).
- The member inflates within `max_config_bytes` and its CRC-32 and ISIZE
trailer match.
- The member trailer ends exactly at the first `IKCFG_ED` after the gzip
header. Inflation reads only the bytes up to that marker.
- Every non-blank line of the inflated text is a `#` comment,
`# CONFIG_X is not set`, or `CONFIG_X=value`, and the text is valid UTF-8
with no control bytes other than tab, CR, and LF.
An `IKCFG_ST` string that is not followed by gzip magic, or whose complete
member is not followed by `IKCFG_ED`, is skipped, so an ordinary string is never
reported as a config. When `IKCFG_ST`, gzip magic, and a later `IKCFG_ED` frame
a member that does not decode, a deflate stream cut off by that marker, a
member that fails its CRC-32 or ISIZE check, or text that fails the grammar,
the config is `malformed` with coverage
`incomplete`, and `marker`, `gzip`, and `end_marker` locate the blob. A
malformed blob is never reported as an absent config. A valid config later in
the searched bytes still wins. `marker`, `gzip`, and `end_marker` give
the three ranges. `sha256` hashes the inflated text. `option_count` counts
every `CONFIG_*` line. `matched_count` counts the options that match the
optional exact `name` filter, and `entries` is the `offset`/`limit` page of
those matches in file order.
Each entry has a `name`, a `kind`, and a `value`:
| `kind` | Source line | `value` |
| --------- | -------------------------- | ------------------------- |
| `y`, `m` | `CONFIG_X=y`, `CONFIG_X=m` | `y` or `m` |
| `number` | decimal or `0x` hex | the text as written |
| `string` | `CONFIG_X="..."` | decoded text (`\"`, `\\`) |
| `not_set` | `# CONFIG_X is not set` | `null` |
Banner text and configuration values are target-derived and untrusted.
## bzImage payloads
A file with `HdrS` at offset `0x202` is an x86 setup header. From boot
protocol 2.08, `payload_offset` and `payload_length` locate the compressed
kernel after the `(setup_sects + 1) * 512` setup code. `payload` reports the
container, codec, file range, and inflated length. A gzip payload is inflated
within `max_payload_bytes` and the inflated bytes are searched. Locations in
those bytes have `space: "inflated_payload"` and are offsets in the inflated
payload, not file offsets. Only the leading gzip member is inflated, and its
own ISIZE trailer gives the inflated length. Kbuild appends a separate 4-byte
size only for the non-gzip codecs. Any bytes after the gzip member inside the
payload are not read.
xz, lz4, lzma, zstd, bzip2, and lzo payloads are `unsupported`. The result
names the codec and `external_decompression` as the next probe. This is an
external step, not a REA operation: decompress the payload with the matching
utility into a separate file, then call `inspect_kernel_build_config` on that
file. A file that is itself a compressed stream, including a whole-file gzip
such as `Image.gz`, is reported as `unsupported` with its codec and is not
inflated. Only a gzip payload inside a bzImage is inflated.
## Limits and failures
The selected file is limited to 64 MiB by default (512 MiB maximum). An IKCFG
member may inflate to 4 MiB by default (32 MiB maximum) and a bzImage payload
to 128 MiB by default (512 MiB maximum). At most 16384 `CONFIG_*` options are
examined by default (65536 maximum). A page holds 256 entries by default
(4096 maximum). Exceeding any budget fails with no partial success and names
the option to raise. An IKCFG member past `max_config_bytes` fails that way only
when no later candidate is accepted, so an oversized decoy cannot hide a valid
config.
One search inflates at most 16 `IKCFG_ST` gzip candidates and at most four
times `max_config_bytes` of output across them. A candidate whose stream fails
partway through is charged its full per-candidate allowance, since its partial
output is not measured. When that search budget runs
out before a config is accepted, the config is `incomplete` with reason
`config_search_exhausted` instead of failing or reporting an absent config.
Inflation runs off the event loop. Every scan, including marker
searches that find nothing, config text checks, validation of one long value,
and hashing, yields to it at least once
per MiB, so a cancellation that arrives while the image is being parsed is honored
at the next check. One inflate call runs to its output bound once started.
Results that could not search or read the kernel bytes stay `incomplete`,
`malformed`, or `unsupported` and are never
reported as an absent config:
| `config.reason` | Meaning |
| ---------------------------------- | ---------------------------------------------------------------------------- |
| `config_gzip_invalid` | a framed `IKCFG_ST` gzip member does not decode or fails its CRC-32 or ISIZE |
| `config_text_invalid` | a framed `IKCFG_ST` gzip member is not `.config` text |
| `config_gzip_truncated` | an `IKCFG_ST` gzip member runs past the searched bytes |
| `config_search_exhausted` | decoy candidates used up the config search budget |
| `payload_truncated` | the declared bzImage payload or its gzip member ends early |
| `payload_gzip_invalid` | the bzImage gzip payload fails to inflate or its trailer |
| `payload_codec_unsupported` | a compressed payload or whole-file stream was not inflated |
| `unrecognized_payload_codec` | the bzImage payload starts with no known codec magic |
| `setup_header_payload_unavailable` | the setup header predates 2.08 or has no payload length |
A kernel built without `CONFIG_IKCONFIG` carries no embedded config, so an
`absent` config describes only the bytes that were searched. A kernel built
with `CONFIG_IKCONFIG=m` keeps the config in `configs.ko`; inspect that module
instead of the image. This workflow does not execute the image, compare the
config with a source tree, or evaluate Kconfig dependencies.
## Format references
The markers follow Linux v6.12's
[`kernel/configs.c`](https://github.com/torvalds/linux/blob/v6.12/kernel/configs.c)
and
[`scripts/extract-ikconfig`](https://github.com/torvalds/linux/blob/v6.12/scripts/extract-ikconfig).
The setup header fields follow the
[x86 boot protocol](https://docs.kernel.org/arch/x86/boot.html). The banner
format follows
[`init/version-timestamp.c`](https://github.com/torvalds/linux/blob/v6.12/init/version-timestamp.c).
These references establish the decoded layout, not runtime behavior.
-178
View File
@@ -1,178 +0,0 @@
# ELF and Mach-O initializer functions
`inspect_native_initializers` / `inspect-native-initializers` lists the
functions that one local ELF or Mach-O image declares to run before `main` and
at exit. It reads the file without executing it and does not need a
disassembler provider.
```bash
rea inspect-native-initializers /absolute/path/to/program --json
rea inspect-native-initializers ./Universal.app/Contents/MacOS/App --architecture arm64 --json
```
The CLI resolves relative paths against the caller's working directory. MCP
requires an absolute filesystem path:
```json
{
"name": "inspect_native_initializers",
"arguments": { "path": "/artifacts/program", "offset": 0, "limit": 64 }
}
```
A universal Mach-O requires `architecture` to select one slice. ELF files and
thin Mach-O files do not take it.
## Sources
ELF entries are listed in this order:
| Kind | Source |
| --------------- | ------------------------------------------------------------------------------------- |
| `preinit_array` | `DT_PREINIT_ARRAY` / `DT_PREINIT_ARRAYSZ`, else `.preinit_array` |
| `init` | `DT_INIT` |
| `init_array` | `DT_INIT_ARRAY` / `DT_INIT_ARRAYSZ`, else `SHT_INIT_ARRAY` or `.init_array*` sections |
| `ctors` | legacy `.ctors` and `.ctors.*` sections |
| `fini` | `DT_FINI` |
| `fini_array` | `DT_FINI_ARRAY` / `DT_FINI_ARRAYSZ`, else `SHT_FINI_ARRAY` or `.fini_array*` sections |
| `dtors` | legacy `.dtors` and `.dtors.*` sections |
Section fallback applies when the dynamic tag is absent, as in static
executables and relocatable objects. Mach-O entries come from sections typed
`S_MOD_INIT_FUNC_POINTERS` (`mod_init`), `S_INIT_FUNC_OFFSETS`
(`init_offsets`) and `S_MOD_TERM_FUNC_POINTERS` (`mod_term`).
## Entries
Each `entries[]` item reports:
- `slot_va`, `slot_file_offset`, `slot_bytes` and `slot_section`: where the
pointer or offset is stored. For a universal Mach-O, file offsets are
relative to the whole file, not the slice. For `init` and `fini` the slot is
the `DT_INIT` / `DT_FINI` value field in the dynamic section. When that field
is outside every load segment, `slot_va` and `slot_section` are null and a
limitation says so.
- `value_va`, `value_file_offset` and `section`: the resolved target address,
its file offset and its containing section. These are null when the value
was not established. A file offset is also null when load segments map the
address ambiguously (overlapping segments with different file offsets, or
file and zero-fill memory), and the report is then `incomplete`.
- `symbol`: a defined symbol whose value equals the target exactly, or the
import name of an unresolved symbolic relocation or chained bind.
- `resolution`: how the value was obtained.
| Resolution | Meaning |
| ------------------- | -------------------------------------------------------------------- |
| `raw` | The stored word, used when no relocation or fixup covers it |
| `relative_addend` | An `R_*_RELATIVE` relocation addend (RELA, or the REL in-place word) |
| `relr` | A `DT_RELR` packed relative relocation |
| `symbol_addend` | A section relocation against a defined symbol plus its addend |
| `chained_fixup` | A Mach-O chained-fixup rebase target |
| `dyld_info_rebase` | A Mach-O `LC_DYLD_INFO` rebase opcode target |
| `offset_table` | A 32-bit `__init_offsets` value added to the image base |
| `unresolved_import` | A relocation or bind against an undefined symbol |
| `unresolved` | A slot whose value this inspection cannot establish |
A bound import name over the 1024-byte name budget is reported as no name and
makes the report `incomplete` with a counted limitation.
Symbol and section names are untrusted file text and carry
`trust: "observed-untrusted"`.
For PIE executables and shared objects the array slots are usually zero on
disk. A slot is reported `unresolved`, never as its stored zero, when the
dynamic relocation tables use an encoding this inspection does not decode
(for example Android packed relocations or an unsupported machine). Relative
relocations are decoded for x86-64, i386, AArch64 and 32-bit ARM. A slot is
also `unresolved` when a dynamic relocation writes only part of it, starts
inside it, or writes it more than once with different values.
Section `REL` / `RELA` tables are applied to slots that no dynamic relocation
covers. When such a table is skipped for an unexpected entry size, cannot be
read, or lies past the record budget, every slot it could reach that has no
relocation found is `unresolved`, and a `.ctors` slot is not dropped as a
sentinel. A section record that writes only part of a slot, starts inside it,
or repeats it also leaves the slot `unresolved`. In a relocatable object a
table reaches only the section named by its `sh_info`. In a linked image, a relocation section without `SHF_ALLOC`
(such as one kept by `--emit-relocs`) was already applied by the static
linker, so the slot's stored word is used rather than applying the record
again.
An absolute relocation against an `STT_GNU_IFUNC` symbol is `unresolved`: the
symbol value is the resolver, and the loader stores whatever the resolver
returns.
On 32-bit ARM (ELF `EM_ARM` and Mach-O `CPU_TYPE_ARM`), bit 0 of a function
pointer selects Thumb state. `value_va` keeps the pointer as stored, while
`value_file_offset` and `section` describe the code address with that bit
cleared. A Mach-O symbol is matched at either address.
In a relocatable object (`ET_REL`) every section starts at address 0. Slot
and value addresses are offsets inside the named section, and a limitation
says so.
Mach-O pointer slots are decoded through chained fixups or `LC_DYLD_INFO`
opcodes for linked little-endian arm64 and x86-64 images. Other linked images,
including 32-bit ones, have no fixup decoder here: their stored word is used
only when the image declares no `LC_DYLD_INFO` bind stream, and a pointer slot
is `unresolved` otherwise. Legacy external relocations (`LC_DYSYMTAB`
`nextrel`) are not decoded, so they make an otherwise raw pointer slot
`unresolved` in any image. In an object file (`MH_OBJECT`), each section's own
relocation entries set the slot: a pointer-width `UNSIGNED` entry against a
defined symbol gives `symbol_addend`, one against an undefined symbol gives
`unresolved_import`, a section-ordinal entry keeps the stored address, and any
other entry leaves the slot `unresolved`. An `arm64_32` image that declares a
64-bit header is rejected as malformed, since its pointers are four bytes.
`.ctors` and `.dtors` values `0` and `-1` are list sentinels and are omitted.
`-1` means all ones at the image's pointer width: `0xffffffff` in ELF32 and
`0xffffffffffffffff` in ELF64. A 64-bit slot holding `0xffffffff` is listed.
A stored zero in an initializer array is listed, so the array length stays
visible.
## Paging and limits
`offset` and `limit` page the ordered list (default limit 64, maximum 4096).
`coverage.examined_entries` counts every slot read, including omitted
sentinels, and `total_entries` (the listed entries) is set when the inventory
is complete. At most 65536 slots are examined; beyond that, or when relocation
coverage is unknown, `status` and `coverage.status` are `incomplete`.
ELF symbol names come from at most 65536 symbols across all `SHT_SYMTAB` and
`SHT_DYNSYM` tables together, and section `REL` / `RELA` relocations from at
most 1048576 records across all relocation sections together. Mach-O symbol
names come from at most 65536 symbols across all `LC_SYMTAB` commands
together; a repeated `LC_SYMTAB` over the same ranges is ignored with a
limitation. Reaching any of these budgets makes the report `incomplete` with a
limitation.
A symbol name that cannot be read, because its string table is missing or its
offset lies outside the table, is reported as no name and makes the report
`incomplete` with a counted limitation. Name offset 0 is the empty name.
The dynamic relocation tables are read in steps that yield to other work and
observe cancellation, so a large `DT_RELR` table does not block the server.
`limitations` lists the standing limitations, then at most 24 distinct
observations from the image. Reasons a crafted table can repeat per row, such
as symbol names over the 1024-byte budget or relocation sections with an
unexpected entry size, are reported once with a count, and observations past
the 24-note budget are counted in one closing line. The list never exceeds 40
lines.
The selected file is limited to 64 MiB by default (`max_file_bytes`, 512 MiB
maximum). Malformed headers, multiple `PT_DYNAMIC` segments and truncated
dynamic sections fail without partial success.
This inspection does not prove that a listed address runs, and file order is
not the runtime order: `DT_FINI` is listed before `fini_array`, while the
dynamic linker runs `fini_array` first, and `.ctors` runs in reverse order. A
missing tag or section is observed absence of that declaration, not proof that
no code runs before `main`.
Legacy Mach-O bind lookup examines at most 4096 candidate runs per query and
at most twice the file byte count across the decoded image's queries. These
resource allowances bound sparse, overlapping run descriptions without expanding
them into per-address objects. Exhaustion leaves pointer applicability unresolved
and the report incomplete. Bind-stream parsing and initializer batches yield so
other requests and cancellation can run.
-94
View File
@@ -1,94 +0,0 @@
# Windows Shell Link shortcuts
`inspect_windows_shortcut` / `inspect-windows-shortcut` answers whether one
selected local file is a Windows Shell Link (`.lnk`), which target path,
arguments, working directory, and icon location it records, which volume and
network names `LinkInfo` stores, and which ExtraData blocks follow the
strings. It reads the file without resolving or executing the target and does
not require a Windows SDK, COM, or a network.
```bash
rea inspect-windows-shortcut /absolute/path/to/shortcut.lnk --json
```
The CLI resolves relative paths against the caller's working directory. MCP
and the advertised input schema require an absolute filesystem path and
preserve that exact selection:
```json
{
"name": "inspect_windows_shortcut",
"arguments": { "path": "/artifacts/shortcut.lnk" }
}
```
## Observations
The inline Evidence result contains the artifact path, SHA-256, and byte
length. `header` is the 76-byte `ShellLinkHeader`: `LinkFlags` and
`FileAttributes` as raw values plus documented names, creation/access/write
FILETIMEs as a raw 64-bit decimal and ISO-8601 UTC when the timestamp is in
range, file size, icon index, `ShowCommand` (`normal`, `maximized`, or
`minimized`), and hotkey modifiers. Undocumented `ShowCommand` values are
decoded as `normal`, as specified.
`id_list` is present when `HasLinkTargetIDList` is set. Each item keeps its
file offset, size, and type byte. Root-folder, volume, and file-entry items
are decoded when those layouts match documented shell-item headers, including
the 8.3 short name on a file entry. Every other item is `raw` with a SHA-256
of its payload. Unknown items are not guessed.
`link_info` reports volume drive type, serial, label, local base path, and
common network relative link names, with Unicode variants when the header
declares them. ANSI strings use a byte-preserving Latin1 projection; the
original system ANSI code page is unobserved.
`string_data` reports name, relative path, working directory, command-line
arguments, and icon location according to `LinkFlags`, each with encoding,
`CountCharacters`, byte length, and the decoded value. `IsUnicode` selects
UTF-16LE; otherwise the Latin1 projection is used.
`extra_data` enumerates every block: signature, size, and offset.
`EnvironmentVariableDataBlock` and `IconEnvironmentDataBlock` expose ANSI and
Unicode targets. `TrackerDataBlock` exposes the NetBIOS machine id and droid /
birth-droid GUIDs. Version-1 UUID MAC and timestamp values are labelled
`evidence_kind: "derived"`. `KnownFolderDataBlock` and `SpecialFolderDataBlock`
keep their folder identifiers. `ConsoleFEDataBlock` keeps the code page.
`PropertyStoreDataBlock` and unrecognized signatures stay raw with SHA-256.
Unknown signatures are listed and are not dropped.
`observations` are derived only from those documented fields: whether
arguments are present, argument character length, target extension, whether
`ShowCommand` is minimized, and whether the icon location differs from the
recorded target path. That comparison is `null` unless the icon and a recorded
target are both full paths: a stored filename, relative path, or `%VAR%` form
cannot show that two locations differ. This workflow does not emit verdicts
about intent.
A missing optional section is observed absence of that structure. It is not
proof the shortcut never carried that data in another container.
## Limits and failures
The selected file is limited to 1 MiB by default (16 MiB maximum). At most 256
IDList items (4096 maximum) and 64 ExtraData blocks (1024 maximum) are
examined by default. Retained string payload bytes across StringData,
LinkInfo, and ExtraData are limited to 64 KiB by default (1 MiB maximum).
Malformed `HeaderSize` or `LinkCLSID`, and exhausted budgets, fail without
partial success. Truncation after a valid header, ExtraData size overruns, and
odd Unicode lengths return partial coverage with `coverage.failing_offset`.
A LinkInfo offset that points outside its structure is reported at the
offset field itself, so every issue offset lies inside the file.
Coverage is never `complete` when issues are present.
The selected path must be a regular file. Symbolic links are rejected.
UNC paths and other host-prefixed device paths, such as `\\server\share\a.lnk`
and `\\?\UNC\server\share\a.lnk`, are rejected before any filesystem call. On
Windows, a path whose parent directory is a symbolic link or junction is
rejected before anything follows that link. The check covers the directories
Windows opens after removing `.` and `..`, so `C:\alias\..\real\a.lnk` is
checked at `C:\real` only; a `\\?\` path is checked as written. A mapped
network drive cannot be told apart from a local drive by its path and is not
rejected.
This workflow does not resolve, open, or execute the link target or call COM,
and it does not contact a network for a path on a local volume.
@@ -1,31 +0,0 @@
# Frida instrumentation
REA exposes Frida device and process discovery, attach or spawn sessions, and caller-authored JavaScript instrumentation through the CLI and MCP. The Node bindings are an optional package dependency and load only when a Frida operation runs. REA does not install Frida server on a target.
For remote devices, pass a host with an optional port (`host[:port]`; bracket IPv6 addresses) and any token, certificate, origin, or keepalive options on that operation. The keepalive interval is measured in seconds, matching Frida Core 17.23.2. Authentication values are held only for the active call or session and are excluded from Evidence. Do not embed credentials in the endpoint address.
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If cleanup fails, the session remains available for cleanup retry but refuses new script loads and resume requests. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
`instrument-with-frida --json` returns the Evidence record directly, matching MCP. Consumers of the earlier `{evidence, cleanup_error}` CLI wrapper should read the record at the JSON root and the session cleanup observation at `normalized_result.cleanup_error`. If final provider cleanup fails or cancellation arrives during it, the typed error retains completed Evidence in `details.partial_observation`. Discovery commands retain their device or process report in the same field.
For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply.
A summary's selected fields look like this (excerpt):
```json
{
"summary": {
"state": "running",
"messages_retained": 50,
"messages_truncated": false
}
}
```
Use `get_evidence_bundle` with `detail="summary"` and `filters={"evidence_id":"PARENT_ID"}` to discover parent metadata without message payloads. Filters apply only to summary delivery. Use `export_evidence_bundle` for the complete canonical bundle, then select the record whose `evidence_id` matches `normalized_result.parent_evidence_id` from the status summary. To use the existing CLI `inspect-analysis-view`, extract that record into a standalone Evidence JSON file and select it as an Evidence-file source with `view={"kind":"summary"}`. Reading retained Evidence does not refresh a live session or verify a process after exit or PID reuse.
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released.
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Local device lookup waits 1000 ms. This release has no runtime override for that lookup; another REA timeout does not raise it. Remote endpoints do not use that lookup. Message capture defaults to 16 MiB per session, 4 KiB per type, 32 KiB per error text field, 250,000 JSON nodes and depth 64. Select `capture_limits` on `start_frida_session` or `instrument_with_frida` before observing: `session_bytes`, `message_type_bytes`, `message_text_bytes`, `json_nodes`, and `json_depth`. Omitted fields keep these defaults. The session allowance includes JSON/base64 expansion, the envelope and metadata; a budget too small for one complete event retains no event. Depth is bounded to 992 levels, reserving eight enclosing message/status/Evidence containers within REA's shared JSON representation limit. These are retention budgets, independent of target authority, observation duration and MCP/client delivery limits. The selected policy is fixed for that session and returned as `capture_limits` in observations; no REA restart is required. The CLI exposes `--capture-session-bytes`, `--capture-message-type-bytes`, `--capture-message-text-bytes`, `--capture-json-nodes`, and `--capture-json-depth`. Evidence names the effective control when that observation dropped data. Changing a budget requires choosing a new session or experiment and cannot recover already discarded live observations. Script source is not stored in Evidence; its selected path and SHA-256 digest are. For a file, `source_sha256` identifies the original selected bytes. Invalid UTF-8 continues to use replacement decoding; Evidence then records that limitation and `raw_result.source_representation` with the encoding, decoding mode, and `loaded_source_sha256` of the UTF-8 string passed to Frida. Valid UTF-8 files and inline scripts do not need this additional representation fact. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. If cancellation is observed when the native call returns, partial Evidence retains the source identity and captured messages even if that call rejects, including messages emitted during awaited cleanup. The caller's cancellation reason remains primary and a native rejection is retained as a sanitized diagnostic. A failure already observed before cleanup stays primary if cancellation arrives during cleanup. The cancellation cutoff is REA's observation of native completion, rather than a claim about unobservable native event order. `script_id` is REA's load-attempt identity. When creation rejects without returning a handle, `script_cleanup: "no_handle"` means REA acquired no handle to unload; it does not prove that the engine performed no work. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
The [Frida Core API](https://frida.re/docs/frida-core-api/) documents device management, sessions, and script loading; the [JavaScript API](https://frida.re/docs/javascript-api/) documents target-side instrumentation. [Luma](https://luma.frida.re/) is Frida's standalone GUI; REA does not automate or control it. `npm run verify:frida` requires a Linux host with the optional Frida binding and uses a short-lived local Node.js fixture created by the verifier. Add `-- --address HOST:PORT --pid PID` to run the separate remote attach lane against a caller-selected target. Mock tests do not establish real device/server compatibility.
@@ -36,8 +36,6 @@ The MCP tool `project_harmony_application_graph` takes the same
- exact component paths, artifact IDs, and hashes (manifests, Ark bytecode,
resources, native libraries, JavaScript, signing)
- `unclassified_components` with the same paths and content identities for
authenticated files outside those categories and the App Pack child list
- a packaging-model hint: Stage is claimed only from an exactly cased `module.json` entry
outside a classified resource path, FA only from `config.json` outside a
classified resource path, otherwise `unknown`
@@ -57,24 +55,6 @@ being classified as code files. Ark and JavaScript runtime hints summarize the
classified bytecode and JavaScript lists; supported source extensions share the
same classification rule as their component rows.
`unclassified_components` preserves files such as `pkgContextInfo.json`,
`.pages.info`, a HarmonyOS `.har` shared library, and a non-certificate
`META-INF` entry without interpreting their contents. The `.har` suffix also
names HTTP Archive JSON, so identifying one needs verified ZIP bytes rather than
a path rule. Different paths sharing the same bytes remain separate entries. App
Pack children of an App Pack root are already reported in `app_pack_children`,
so they are not repeated as unclassified files. A nested `.hap` or `.hsp` member
of a HAP or HSP root is disclosed in `unclassified_components` instead, because
child-package inventory is not recursive. Unreadable or encrypted content remains
in `unavailable_components` with unknown identity instead.
`coverage.status: "complete-within-inventory"` describes the authenticated
source inventory and bounded bridge projection, not semantic understanding of
every file. Unclassified files do not make a complete inventory partial; missing
inventory pages, source coverage limits, unavailable content and bridge limits
still qualify coverage independently. An empty unclassified list on a partial
inventory does not establish that the whole package has no unclassified files.
Every claim is path-presence only. Manifest semantics (module identity,
abilities, entry points, permissions, dependencies), Ark bytecode contents and
versions, resource resolution, signing verification, and observed native
@@ -82,12 +62,6 @@ boundaries are out of scope for this increment and stay in the projection's
`limitations` array until a dedicated HarmonyOS provider exists. REA fails
closed on packages whose inventory cannot be authenticated.
`pkgContextInfo.json` is inventoried and listed as an unclassified component but
never read, so its `isSO` declarations do not narrow `bridge_candidates`. When
that entry is present the projection adds a limitation naming the unread
declaration, so a path-convention hypothesis is not mistaken for a package
declaration.
## Verification lane
`npm run fixtures:harmony` downloads the SHA-256-pinned 1.0.1 HAP release
+1 -4
View File
@@ -36,10 +36,7 @@ regions and are never filled.
Region `format_metadata` (`hex-record-region`) records `start_address`,
`end_address`, `length`, `record_count`, and `first_line`/`last_line`: the
1-based line span of the contributing data records. `compare_artifacts`
compares none of these fields: a region's name encodes its start and end
addresses, its length follows from them, and record and line counts are
layout.
1-based line span of the contributing data records.
Root `format_metadata` (`hex-records`) records:
@@ -51,19 +51,13 @@ was established; failed registrations remain unavailable. Correct
launch environment. Restart/reconnect that server after changing its environment;
an export in another shell does not update a running server.
An HTTP registration's `url` must use `http` or `https` without URL credentials,
and its host must be `localhost`, `127.0.0.1`, `[::1]`, or the Docker/Podman
host-gateway name `host.docker.internal` or `host.containers.internal`. Use a
gateway name when REA runs in a container and IDA MCP listens on the container
host. REA refuses every other host.
Open the **original input binary** in REA, not an `.idb` or `.i64` database. Its SHA-256 must match IDA's recorded input hash. REA checks the attached identity and image base around each analysis request. A mismatch or target switch fails with recovery guidance. The original input must remain readable by REA; WSL can use its mounted alias while the GUI runs on Windows. Attached profiles, including native mode, refuse universal Mach-O inputs at admission: every slice shares the recorded whole-file hash, so REA cannot verify which slice the GUI loaded. Open a thin file extracted with `lipo -thin` in both IDA and REA, or use headless mode, which analyzes the selected slice.
REA's close operation releases its MCP connection or proxy. It never saves or closes the existing GUI database. External edits remain visible because results are live and are not replayed from snapshots. These checks do not lock the GUI or prove that the database is unpatched.
## Optional native IDA MCP bridge
IDA Pro and IDA Free can use the separate [ida-free-mcp native plugin](https://github.com/0xshlomil/ida-free-mcp) through an explicit attached registration. Install and start that plugin using its upstream instructions, open the original binary in the IDA GUI, and wait for analysis. The native MCP 1.1.2 release binds `0.0.0.0` without a listen-address setting. Restrict inbound access with the host firewall or use an upstream build that binds only to loopback before starting it. REA's local URL restriction does not change the plugin's listener. REA does not install IDA, build the plugin, activate a license, or launch the GUI.
IDA Pro and IDA Free can use the separate [ida-free-mcp native plugin](https://github.com/0xshlomil/ida-free-mcp) through an explicit attached registration. Install and start that plugin using its upstream instructions, open the original binary in the IDA GUI, and wait for analysis. The native MCP 1.1.2 release binds `0.0.0.0` without a listen-address setting. Restrict inbound access with the host firewall or use an upstream build that binds only to loopback before starting it. REA's loopback URL restriction does not change the plugin's listener. REA does not install IDA, build the plugin, activate a license, or launch the GUI.
```json
{
@@ -233,7 +227,7 @@ admits it explicitly. Raw-binary and GameCube DOL targets are refused; open
them with Ghidra. Admission is not verification: the real workflows under
[Verification](#verification) cover x86_64 and arm64 only.
All profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Neither `callees` nor `callgraph` reports the call sites behind its edges, so a computed call's targets are never established as exhaustive; callee coverage is therefore `partial` even for an uncapped reply, and a call path that does not reach its goal remains `unknown`. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
All profiles support procedure/string inventory and literal or regex search, procedure resolution, pseudocode, assembly, raw function instructions, resolved callees, incoming address references, and a function dossier. Inventories and modern pseudocode/assembly follow upstream pagination. Upstream limits on modern xrefs and callees are retained as explicit truncation limitations. In the modern headless profile, targets that are not proven x86-only use processor-neutral `callgraph` edges instead of the x86 instruction checks in upstream `callees` (mrexodia/ida-pro-mcp#549). REA requests depth zero: the pinned upstream emits the root's outgoing edges without scanning child functions, then REA batches function-name lookup for those addresses. The graph covers recognized function destinations only; imports and other non-function targets remain unknown. A server without `callgraph` returns an empty relationship list with an explicit coverage limitation. Graph and path workflows retain those limitations alongside observed edges. A failed or inconsistent graph reply remains an error. A transport-level truncated preview is rejected rather than reported as complete evidence; the error preserves upstream retrieval guidance.
All profiles expose `list_documents` as the single target admitted for the session. Attached mode verifies its digest against IDA metadata before returning it; headless mode binds the digest-checked private input. `inspect_address_context`, `trace_feature`, and `binary_overview` can therefore resolve that identity without upstream document-listing tools. Segment enumeration is optional: when the selected profile cannot provide it, `binary_overview` still returns document and procedure/string counts and omits `segments` and `segment_count`; omission means unobserved, not empty. The modern headless profile also supports `resolve_containing_procedure`. An address resolves through upstream `lookup_funcs` to the canonical entry of the function that contains it; the body stays unavailable like other IDA identities. When no function contains it, REA reads the database-scoped minimal `survey_binary` and reports `not_in_procedure` for an address inside one of its segments (ends are exclusive) and `outside_segments` otherwise. A missing, truncated or inconsistent survey fails the read instead of guessing a reason. `trace_feature` marks a trace truncated, with a residual unknown for that address, when upstream caps the references it returned. Attached profiles do not declare address containment resolution.
@@ -245,7 +239,7 @@ Complete body ranges, typed reference edges, CFG, referenced data, source locals
## Verification
Run `npm run verify:ida -- --target /absolute/path/to/program --procedure main` with a configured upstream server. It exercises production CLI and MCP tools, validates their output contracts and Evidence, checks original-input preservation and session cleanup, and keeps local target data out of its console summary. Supply `--package-root` to verify an extracted or installed package. See [testing](../development/testing.md) for the distinction between adapter tests and real-provider evidence.
Run `npm run verify:ida -- --target /absolute/path/to/program --procedure main` with a configured upstream server. It exercises production CLI and MCP tools, validates their output contracts and Evidence, checks original-input preservation and session cleanup, and keeps local target data out of its console summary. Supply `--package-root` to verify an extracted or installed package. See [testing](testing.md) for the distinction between adapter tests and real-provider evidence.
Native verification requires `--second-procedure other_function --second-marker unique_second_body_text`; these options remain optional for the other profiles. The marker must occur in the second function's pseudocode and be absent from the first; the verifier also checks that returning to the first restores its text.
Loaded 100 of 3082 files, more files were not shown because too many files have changed in this diff. Show more